In this example, we are taking employee data as input and we are going to capture the duplicate and unique employees on the basis of EmployeeID.
Input Data :
Design the job as shown below:
Open Sort stage properties, select the key ’employeeID’ and select the Option ‘Create Key Change Column = True’, this will add one new output column ‘keychange’ code representing ‘0’ as duplicate record and ‘1’ as unique record.
In Sort stage properties, select Mapping tab to provide input-output mapping as shown below.
Open Filter stage properties, provide the where condition and the output link associated with it as shown below.
In Filter stage properties, select the Link Ordering tab to provide the output links.
In Filter stage properties, select the Mapping tab to provide the output mapping.
Save the job, compile and run it to see the results.
Output Data :
1. Duplicate Records
2. Unique Records