Hi delta io guys Please check out our new blog on Delta Merge Optimization:- https://www.linkedin.com/posts/joydeep-banik-roy_delta-mergeoptimisation-strategies-a[…]867588489216-WDby?utm_source=share&utm_medium=member_desktop

not sure but wouldn't you want to also add

source.country = 'USA'

then to your merge condition?

<http://deltaTable.as|deltaTable.as>("target")
  .merge(<http://df.as|df.as>("source"), 
  //Earlier would have looked like "target.id = source.id and target.country = source.country"
  "target.id = source.id and target.country = 'USA'")
  .whenMatched
  .updateAll()
  .whenNotMatched()
  .insertAll()
  .execute()

True, sorry I missed that => also, in our case source was already partitioned by country, and we picked them separately but that might not be immediately apparent to the reader, let me add that part

also, how would you parallelize the load then as there is only 1 source_df?

maybe I am understanding the question wrong so we would launch multiple spark delta jobs with the same dataframe, is this what you are asking