Hi guys , is it possible to upsert when a table is not delta, for example, saved in parquet format ?

No,

no, you'd need to read from the target table to dataframe, read from source table to dataframe, combine the two dataframes so that the result reflects the desired outcome and then overwrite the full target table.

@Martin In this way , can i capture inserts and updates? full=(t1.alias("a") .join(t2.alias("b"),col("ca")==col("cb"),"inner") .select(*cols)) (full.write .format("parquet") .mode("overwrite") .option("overwriteSchema","true") .save("path"))

you definitely can do it, but it's pretty complex and not efficient. it's why formats like delta lake are so convenient for users - as a user, these APIs are given to you.

I agree with that, the point is because only in gold layer they transform as a delta table, 😐

i dont know any specifics, but it's probably easier to just get the pipeline and tables over to delta rather than re-inventing the wheel with parquet.