https://delta.io logo
m

Maks Lyzhkov

01/24/2023, 1:22 PM
Hi all! Is it writing to GCS currently supported from python bindings? I’m faced with this error:
Copy code
deltalake.PyDeltaTableError: Failed to read delta log object: Generic GCS error: Error performing copy request test_delta/_delta_log/_commit_0025723a-ff2f-4bb6-9f7f-7c80f95fd5b9.json.tmp: response error
......
<p>POST requests require a <code>Content-length</code> header.  <ins>That's all we know.</ins>
to reproduce it:
Copy code
import os

import pyarrow as pa
import deltalake

storage_options = {"SERVICE_ACCOUNT": os.environ.get("GOOGLE_APPLICATION_CREDENTIALS")}
pylist = [{'n_legs': 2, 'animals': 'Flamingo'}, {'year': 2021, 'animals': 'Centipede'}]
table = pa.Table.from_pylist(pylist)
deltalake.write_deltalake("<gs://some_bucket/test_delta>", table, storage_options=storage_options)
w

Will Jones

01/24/2023, 3:29 PM
Huh that error again. It was reported recently but I haven’t been able to reproduce yet. https://github.com/delta-io/delta-rs/issues/878 I’ll take a look soon.
m

Maks Lyzhkov

01/24/2023, 7:14 PM
Oh, it will be great) Thx!
4 Views