https://delta.io logo
v

Vladimir Prus

08/21/2023, 6:16 PM
The current recommendation for implementing SCD2 tables, at https://docs.delta.io/latest/delta-update.html#slowly-changing-data-scd-type-2-operation-into-delta-tables, suggest to first do a join with existing data, to get a list of rows that must be added. That seems somewhat wasteful - merge operation already does a join, do we're down to two joins. Are there any plans to make this more straightforwards, say by allowing a whenMatched to both update existing record and add a new one?