https://delta.io logo
b

Benny Elgazar

09/07/2023, 9:37 PM
Hi, I am suffering with performance issue when using MERGE INTO command. The table have 8 columns, and types such int, string, float. The table count (*) = 1.2~ Trillion rows. The data Im trying to merge is 10-50 or 100 records tops in single batch. The table is partitioned by year_month field. The table uses 4 columns Z order optimization. The merge into using join keys that are Z-optimized. Anyone can help?
d

Dominique Brezinski

09/07/2023, 11:01 PM
4 columns might be too much for the z-order, depending on the cardinality and relationship between the columns. Are you joining on all 4 columns?
What does the query plan look like? What are the merge metrics after it completes?
s

Sasidharan Sangeeth

09/11/2023, 8:58 AM
@Dominique Brezinski how to see the merge metrics?