Wednesday, October 01, 2014

Difference Between Normal Lookup and Sparse Lookup


Normal Lookup :-
  • Normal Lookup data needs to be in memory
  • Normal might provide poor performance if the reference data is huge as it has to put all the data in memory.
  • Normal Lookup can have more than one reference link.
  • Normal lookup can be used with any database.
Sparse Lookup :-
  • Sparse Lookup directly hits the database.
  • If the input stream data is less and reference data is more like 1:100 or more in such cases sparse lookup is better.
  • Sparse Lookup,we can only have one reference link.
  • Sparse lookup,we can only use for Oracle and DB2.
  • Sparse lookup sends individual sql statements for every incoming row.(Imagine if the reference data is  huge).
This Lookup type option can be found in Oracle or DB2 stages.Default is Normal.



Like the Facebook Page & join Group
https://www.facebook.com/DataStage4you
https://www.facebook.com/groups/DataStage4you

https://twitter.com/datastage4you
For WHATSAPP group , drop a msg to 91-88-00-906098


Tuesday, September 30, 2014

DataStage Warning : When checking operator: Operator of type "APT_TSortOperator": will partition despite the preserve-partitioning flag on the data set on input port 0


DataStage Warning : When checking operator: Operator of type "APT_TSortOperator": will partition despite the preserve-partitioning flag on the data set on input port 0.


These warnings are obtained when we are using some specific partitioning method in a stage in the job e.g hash/same etc but in the preceding stage , preserve portioning(also default) is set in the stage>advanced tab.


Sunday, September 28, 2014

DataStage Warning - APT_CombinedOperatorController


While we are running DataStage job, we got this warning like..

APT_CombinedOperatorController(1),0:  XXXXXXXXXXXXXXXXXXXXXXX
APT_CombinedOperatorController(2),1:  XXXXXXXXXXXXXXXXXXXXXXX

We don't get the stage name which is causing this issue ? How we will get the Culprit Stage :-) ?


Solution :

Friday, September 26, 2014

DataStage Warning - Agg,1: Hash table has grown to 16384 entries


Sometime when we are running Datastage job in which we are using Aggregator stage, we get below warning -

Agg,1: Hash table has grown to 16384 entries.
Agg,0: Hash table has grown to 16384 entries. 


Solution :