There is also

whenNotMatchedBySourceDelete

thanks for sharing

( customers_table.alias("target") .merge(new_df.alias("source"), "target.id = source.id") .whenMatchedUpdate( set={"target.last_seen": "source.current_date", "target.status": "'active'"} ) .whenNotMatchedInsert( values={ "target.id": "source.id", "target.name": "source.name", "target.age": "source.age", "target.last_seen": "source.current_date", "target.status": "'active'", } ) .whenNotMatchedBySourceUpdate( condition="target.last_seen <= (current_date() - INTERVAL '30' DAY)", set={"target.status": "'inactive'"}, ) .execute() )

this is your example - this one works

and this one

def handle_merge(path, updates, current_dt): delta_state = DeltaTable.forPath(spark, path) delta_state.alias("s").merge(updates.alias("c"), "s.key = c.key and s.is_current = true") \ .whenMatchedUpdate( condition = "s.is_current = true AND s.value <> c.value", set = { "is_current": "false", "valid_to": "c.date" } ).whenNotMatchedInsert( values = { "key": "c.key", "value": "c.value", "value2": "c.value2", "is_current": "true", "date": "c.date", "valid_to": "null" } ).whenNotMatchedBySource(condition = "c.valid_to IS NULL").update(set = { "is_current": "false", "valid_to": current_dt }).whenNotMatchedBySource().delete().execute() delta_state = spark.read.format("delta").load("dummy_delta") delta_state.show(50) # fails with: AttributeError: 'DeltaMergeBuilder' object has no attribute 'whenNotMatchedBySource' handle_merge('dummy_delta', d2, current_dt=2)

my example fails consistently