Filter Stage Example

It is one of the processing stages and filters out records of the input data as per the specified Conditions. It supports one input link and n number of output links. All the data which don’t satisfy the condition can be passed to an output link.

Example:

In below example we have Employee Department details as an input and we are going to filter out employees with Department as ‘A100’ to one file and ‘DV100’ to another.

Input Data:

filter1 1 - Filter Stage Example

Design the job as shown below:

FilterMain - Filter Stage Example

Open Filter stage properties, Under Stageà Properties tab select the Where clause to define the condition Where Clause = department = ‘A100’ and add Output link = 0 for the same. Similarly add another Where clause for department ‘DV100’

filter2 - Filter Stage Example

We can specify the Link order to let the stage know, where conditional data needs to flow to the corresponding link as shown below.

filter3 - Filter Stage Example

Under OutputàMapping tab of filter stage, drag and drop columns to be mapped from input to output i.e Dept_A100.

filter4 - Filter Stage Example

Similarly, Under OutputàMapping tab of filter stage, drag and drop columns to be mapped from input to output i.e. Dept_DV100.

filter6 1 - Filter Stage Example
Provide the target file path and name in the InputàProperties tab in Dataset stage as below for both the target files.
filter5 - Filter Stage Example
Save, Compile and run the job. Post successful job run the outputs will be as shown below.

Output Data:

filter7 - Filter Stage Example
filter8 - Filter Stage Example