I really hope that someone can clarify the issue we re curre

Question

I really hope that someone can clarify the issue we re currently facing with the `delta merge` operation What does it happen if during a merge operation the underlying data is changing Can this lead to data inconsistencies This would explain some issues that we re facing

TD · Answer

all writes are transactionally guaranteed to serializable

Alessandro Biagi · Answer

So these statements of chatgpt are false right I don t use it generally but for this particular doubt I gave it try and it seemed to confirm my suspicious < TD>

Alessandro Biagi · Answer

To clarify this I m referring to thecase where the input dataframe of delta merge operation is changing If you can also give more details about your answer that would be appreciated < TD>

Nick Karpov · Answer

this was an issue but should be solved by <https github com delta io delta commit b07257df6bf7de2cadf44c21b767200f63094cb9>

Alessandro Biagi · Answer

< Nick Karpov> ok but I m quite sure I m facing this same issue is there a way for me to confirm this

Nick Karpov · Answer

what version are you using

Nick Karpov · Answer

i believe this fix went out in <https github com delta io delta releases tag v2 2 0>

Alessandro Biagi · Answer

ok let me check this

Alessandro Biagi · Answer

< Nick Karpov> what s the fastest way to check deltalake version it s not straightforward from the docs

Alessandro Biagi · Answer

< Nick Karpov> apparently our version is the very old 1 1 0 so this could explain everything

Alessandro Biagi · Answer

< Nick Karpov> regarding this fix it would come out naturally with the version 2 2 0 or should I modify some spark configuration specifically to have the materialized view I m still facing a similar issue with 2 2 0 where my source looks like it wasn t materialized see my problem below in the chat