Alber Tadrous

06/06/2023, 8:08 PM
Hi Team, I store my delta tables schema definition in a central repo in json format, to enforce and validate schema of the incoming data, but when I tried to update a column data type from Timestamp to string using this commands: struct = StructType().fromJson(new_schema_string) df2 = spark.createDataFrame(data=[],schema=struct) df2.write.mode("append").format("delta").option("overwriteSchema", "true").saveAsTable("table_name") it throws this error: Failed to merge fields 'createdt' and 'createdt'. Failed to merge incompatible data types TimestampType and StringType what should I do?