https://delta.io logo
r

Randy Sims

08/21/2023, 3:48 PM
Is it really a good idea to use time travel instead of the usual SCD Type 2 method with metadata fields? Does this defeat the VACUUM command? Does it cause more complications with maintenance of the delta?
k

Kashyap Bhatt

08/21/2023, 5:56 PM
IMHO, these are not comparable. 1. Storing a dimension as SCD Type 2 table vs 2. Storing a dimension as a snapshot table. With SCD Type 2 table you can run a query that joins your facts to different versions of a single dimension value. With option 2 you simply can't do that, you can use only one version of the dimension value in one query.
So decide, if you want versioned dimensions or not, if yes then go with 1, if not then option 2.
d

Dominique Brezinski

08/21/2023, 9:45 PM
Another way too look at it is time travel is nice with SCD Type 1...you can snap back to point in time and see the state of things. But if you really need Type 2, I wouldn't consider time travel a real substitute.
👍 1
r

Randy Sims

08/21/2023, 9:45 PM
Thanks for the confirmation