Hi team, anyone have any idea as to why does spark always set the nullable field as true even when i have mentioned it to be false in the schema? I am trying to create a delta table with empty dataframe and a schema

Can you please post your code snippet?

Hi Ian, Spark will convert the schema automatically to be

nullable=true

when writing a Delta Table. It definitely is a little surprising but appears to be the intended behavior. You can read more in this thread: https://github.com/delta-io/delta/issues/873#issuecomment-1006790092

Thanks Jim. I was surprised myself as well but continuing to read the thread it made sense. I will try the user workaround. thanks again.

No problem! I think a lot of users will find your question useful 😄

Sure.