I am using SSIS to replicate data from an AS400 mainframe to a SQL destination. I am using a lookup column to see if the primary key is duplicated and if not, it will INSERT the row. This is all working fine. What I need to know is can I also use the Lookup transformation to look for differential data and then UPDATE the row? The primary key of the table will never change, however the data might and I need the package to recognize that this is a modified row on the mainframe and that the same row on the SQL destination server needs to be updated.
Thanks for any useful information.
I believe that I found a solution to my own problem. It looks like I need to use the Slowly Changing Dimension transformation. So far in running the test data, this is exactly what I need.|||Thank you so much for the answer to your own question! I've been using lookup and conditional split transformation to check if non-key columns have been modified. Writing an expression to compare all columns in a 40-column table has been making me go crazy. SCD does it all for you! In the literature I had in hand SCD is always used in the data warehouse context. Since I'm using IS for the data migration (only), I've never considered it as a transforamtion that could be of any use to me.
No comments:
Post a Comment