Hello. I am running

0.10.1

and I currently get this error writing

GEOM

values:

File "/Users/simonthelin/Library/Caches/pypoetry/virtualenvs/7b-atlas-to-bronze-wsdENo_o-py3.9/lib/python3.9/site-packages/deltalake/writer.py", line 303, in write_deltalake
    ds.write_dataset(
  File "/Users/simonthelin/Library/Caches/pypoetry/virtualenvs/7b-atlas-to-bronze-wsdENo_o-py3.9/lib/python3.9/site-packages/pyarrow/dataset.py", line 999, in write_dataset
    _filesystemdataset_write(
  File "pyarrow/_dataset.pyx", line 3655, in pyarrow._dataset._filesystemdataset_write
  File "pyarrow/error.pxi", line 100, in pyarrow.lib.check_status
pyarrow.lib.ArrowInvalid: Column 1 named geom expected length 89939 but got length 57171

Is there any way I can align this on my pyarrow table before I call

write_deltalake

? I found this: https://github.com/huggingface/datasets/issues/1817 I assume this is expected behaviour of

write_deltalake

?