Hi All,
if vacuum and merge are ran in parallel will vacuum delete files that are being committed ?
j
JosephK (exDatabricks)
02/16/2023, 4:21 PM
No
a
abhijeet_naib
02/16/2023, 4:21 PM
ok
g
Gerhard Brueckl
02/16/2023, 4:26 PM
How is it implemented? Is there also some threshold on the age of orphaned files?
j
JosephK (exDatabricks)
02/16/2023, 4:49 PM
To modify my no answer, it does appear if you vacuum 0 hours while another process has written but not committed files, you could lose data.
c
Christopher Grant
02/16/2023, 7:01 PM
theoretically, if your merge takes longer than your data retention period - which is 7 days by default - they could conflict. but jobs taking days is extremely rare.