Hi guys what's the difference between

mergeSchema

and

overwriteSchema

? When to use one instead other?

not all changes in schema can be merged - e.g. you cannot change the type of a column from

StringType()

to

IntegerType()

if you still need to do this, you need to overwrite schema and rewrite your date

I got an error using

overwriteSchema

, I was adding some columns to original table, change to

mergeSchema

it works

thats sounds weird though - what was the change you did? are you also using

.mode("overwrite")

?

It was complaining in

.mode("append")

, i changed also in

.mode("overwrite")

inside a if statement

What do you mean with inside an

if

? You either execute as append or as overwrite