https://delta.io logo
a

abhijeet_naib

02/16/2023, 4:20 PM
Hi All, if vacuum and merge are ran in parallel will vacuum delete files that are being committed ?
j

JosephK (exDatabricks)

02/16/2023, 4:21 PM
No
a

abhijeet_naib

02/16/2023, 4:21 PM
ok
g

Gerhard Brueckl

02/16/2023, 4:26 PM
How is it implemented? Is there also some threshold on the age of orphaned files?
j

JosephK (exDatabricks)

02/16/2023, 4:49 PM
To modify my no answer, it does appear if you vacuum 0 hours while another process has written but not committed files, you could lose data.
c

Christopher Grant

02/16/2023, 7:01 PM
theoretically, if your merge takes longer than your data retention period - which is 7 days by default - they could conflict. but jobs taking days is extremely rare.
👍 3
2 Views