Shira Bodenstein02/14/2023, 12:40 PM
The deletion will be in a similar way. Now to the question: Since there are no partitions on the fields on the query, does it mean that Spark will have to open and read all files? Thanks in advance!
DeltaTable.forPath(sparkSession, hdfsPath) .alias("oldData") .merge(data.alias("newData"), deltaMergeQuery) .whenMatched() .updateAll() .whenNotMatched() .insertAll() .execute();
JosephK (exDatabricks)02/14/2023, 1:10 PM
Shira Bodenstein02/15/2023, 6:05 AM