Guys, when I want union two dfs, and sometimes their schema get changed what is the best practice to do it ?

Curious about this one myself. When strictly dealing with dataframes I guess the answer would be to use the most effective method to compare the dataframe schemas and add the missing columns as null values before the union.

spark (non-delta) questions belong in #random unionByName with allowMissingColumns should work!

@Christopher Grant sorry, by that Thanks for sharing the solution

all good