Harun
03/08/2023, 6:25 PMwrite_deltalake
It raises `deltalake.PyDeltaTableError: Schema error: Invalid data type for Delta Lake: Null`error if there is a column with all none values.
Is there a way to solve this? E.g. define the schema for table or define column type
import pandas as pd
from deltalake.writer import write_deltalake
data = [['tom', None], ['nick', None], ['juli', None]]
data = pd.DataFrame(data, columns=['Name', 'City'])
write_deltalake("s3a://.....", data, mode='overwrite')
JosephK (exDatabricks)
03/08/2023, 6:27 PMHarun
03/09/2023, 9:05 AMJosephK (exDatabricks)
03/09/2023, 12:03 PM