Marius Grama01/10/2023, 6:27 AM
, add data into the column, perform modifications , drop the column and subsequently I add a new column
and add data into the newly added column and perform modifications. While calling
function I get the message :
Error in SQL statement: DeltaColumnMappingUnsupportedSchemaIncompatibleException: Change Data Feed (CDF) read is not supported on tables with column mapping schema changes (e.g. rename or drop).
I did add the property
You may force enable streaming read at your own risk by turning on spark.databricks.delta.changeDataFeed.unsafeBatchReadOnIncompatibleSchemaChanges.enabled.
to my test table (in
while creating the table) - however I still receive the above mentioned exception. Can somebody provide me a hint?
Tony S.01/10/2023, 10:15 AM
Scott Sandre (Delta Lake)01/11/2023, 6:31 PM
is a SQL configuration ... not a delta table property. Can you try adding it as a SQL conf to your spark session instead?