Lori
09/18/2023, 7:01 PMConcurrentAppendException
when I write to the same table (use Delta merge
) from 2 EMR clusters (EMR version 6.10.0, Spark 3.3.1, Delta 2.2.0), and there was no entries in the DynamoDB after writing. It looks like I did not enable Delta Lake Multi-writer successfully. My questions are:
1. How can I debug this? I checked online, but I did not find the useful solutions/debug steps
2. I use the SparkSession.builder().appName().config(…)
to config the spark session. Is this correct? Besides the spark configuration listed in the Setup steps, is there any required Spark configurations I need to setup?
Thank you!