Hi friends, As a best practice when some table is updated period to period, is it a better option to upsert it, or overwrites is also an option depending the situation ? Currently we are working like this :

result.write \

.format("delta") \

.mode("overwrite") \

.option("overwriteSchema", "true") \

.saveAsTable('schema.table')

Overwrite is better because upsert/merge will keep rows that aren’t in your

result

df. You probably don’t want that unless you want to track old rows.