Tuesday, February 04, 2014
DataStage Scenario - Design2 - job2
DataStage Scenario Problem --> DataStage Scenario - Problem2
Solution Design :
a) Job Design :
In job design, we are using Copy, Aggregator, Filter and Join stage to get the output.
b) Copy Stage Properties :
Simply map the input to both link output. first link goes to Aggregator and second link goes to Join stage.
c) Aggregator Stage Properties :
Input data contains only one column "No" , In Aggregator stage, we have group the data on the "No" column and calculate the rows for each Key ( No ).
d) Filter Stage Properties :
In Filter stage, we put 2 where condition count=1 and count>1. and assigned different links to both conditions.
From filter Stage, first link (count=1) map to output file ( which contains the unique records )
and second link we map with Join stage.
e) Join Stage Properties :
In join stage, we join the both input on key column (No).
Output from Join map with second output files which contains all the dups as occur in input.