https://delta.io logo
s

Sanjeeb Dey

02/06/2023, 5:43 AM
facing an issue (not sure if it one), but whenever I do a merge logic, the number of partitions in spark processing reduces to equivalent to the table partition
y

Yatharth Maheshwari

02/06/2023, 7:01 AM
Can you try setting this property.
Copy code
--conf spark.databricks.delta.merge.repartitionBeforeWrite.enabled=false
s

Sanjeeb Dey

02/06/2023, 5:55 PM
Yes, it worked. Thanks a lot
1
🙌 1
my mistake was, i was just removing the config instead of marking it as false 🤣
4 Views