https://delta.io logo
e

ERevanthSai

02/15/2023, 6:01 AM
Write to delta table is failing due to parallel writes. I cannot use the partitions here. Is there any workaround to avoid such issues.
p

Praateek Mahajan

02/15/2023, 6:02 AM
Make it an append table? Logically MERGE can’t work since you might end up affecting the same files.
e

ERevanthSai

02/15/2023, 6:03 AM
It can't only be an append table. As we are performing updates on the table
a

Athi

02/15/2023, 1:42 PM
is this new table? if new table, create the table DDL upfront and try parallel writes.
d

Dominique Brezinski

02/16/2023, 7:14 PM
parallel writes that are not explicitly partition disjoint will always conflict, and one will fail. Delta does not work at the granularity of resolving if the parquet files being removed are disjoint between the two operations.
9 Views