https://delta.io logo
c

Christina

07/16/2023, 11:16 PM
Does either OSS/Databricks managed delta lake support Merge on Read today?
d

Dominique Brezinski

07/17/2023, 4:09 PM
As of OSS 2.4 reading and writing to tables with Deletion Vectors is supported. Deletion Vectors are merged on read, and then physically merged in certain operations. https://docs.delta.io/latest/delta-deletion-vectors.html
c

Christina

07/18/2023, 1:39 AM
Thank you! Can I assume enabling deletion vector won’t affect merge on the same table as long as delete and merge are not concurrent? Will that result in any consistency issue? Is it recommended to run reorg / purge once in a while? Or do periodic optimize/Z order and compaction effectively purge the soft deleted records? Does deletion vectors performance optimization apply to merge and update operations?