https://delta.io logo
r

Rahul Sharma

02/16/2023, 4:16 AM
Hii Team , can we set max file size during performing z-order optimization bcz i am having issue to getting 30mb file size so it is creating issue during update the data, i have set below property but not working this property . any idea ?
Copy code
targetfilesize_refine='5000000' #byte

  
          alter table delta.`{table_loc}` 
            SET TBLPROPERTIES (
                'delta.targetFileSize'='{self.targetfilesize_refine}',
                'delta.tuneFileSizesForRewrites'='{self.tunefilesizesforrewrites_refine}'
                )
             """)
j

Jon Stockham

02/16/2023, 8:32 AM
try setting
spark.databricks.delta.optimize.maxFileSize
r

Rahul Sharma

02/16/2023, 8:42 AM
won’t effect also in z-order ?
7 Views