https://delta.io logo
r

Rahul Sharma

02/05/2023, 1:39 PM
Hii Team , i am getting file not found exception ,The default retention threshold for the Delta files is set to 72 hours.
v

Vivek B

02/05/2023, 6:59 PM
Just wondering how you kept the default retention. Is it through a scheduled spark job or databricks managed?
r

Rahul Sharma

02/06/2023, 5:20 AM
this is through a scheduled spark job on EMR
Copy code
self.set_table_property('raw',self.raw_delta_table_path,self.deltla_log_retention_duration_raw,self.delete_file_retention_duration_raw)
                        
self.set_table_property('refine',self.refine_delta_table_path,self.deltla_log_retention_duration_refine,self.delete_file_retention_duration_refine)
                        
this is how i set the property
every day my vaccum job also run
please give me some suggestion bcz this is in our prod ENV issue
3 Views