Are versioned buckets advised when we use the delta format C

Question

Are versioned buckets advised when we use the delta format Concerned it will blow up s3 costs

TD · Answer

no its not advised in fact there is no need

TD · Answer

delta s data is stored in immutable parquet files since objects are not modified there is no need to version the objects

TD · Answer

the table versions are maintained in the delta log which tracks which parquet files are relevant for each version of the table

rtyler · Answer

We deploy with versions but have aggressive lifecycle policies in place that purge noncurrent versions after a week in most cases We look at versions in S3 as a way to give a little extra protection in case of catastrophic stupidity when dropping or vacuuming stuck out tongue