https://delta.io logo
j

Jim Hibbard

01/11/2023, 8:24 PM
Hey! I've been playing with delta-rs and really enjoying it. I had a couple questions though: 1. In the deltalake.table.DeltaTable init method, what is the purpose of the
without_files
parameter? 2. What's the best way to get a reference to the filesystem object being used by a DeltaTable if I want to pass it to another method but with the same configuration / credentials?
w

Will Jones

01/11/2023, 8:29 PM
without_files
skips resolving the list of current files. I don’t think
without_files
is useful yet in the Python bindings. It was implemented in Rust for append-only write use cases where they didn’t want to have to keep the list of files in memory.
And on (2) I don’t think we expose that yet, unfortunately
I think the best thing to do is keep
storage_options
around
Does that seem viable for your use case?
r

rtyler

01/11/2023, 8:43 PM
yes, but you can
get_storage_uri
off the
DeltaTable
too
j

Jim Hibbard

01/11/2023, 9:00 PM
Thanks for all the info. My use case is writing helper libraries and making pull requests to ML libraries that support reading parquet but not delta. I can generate the list of parquet files with delta-rs for a given version of a table, but it'd be nice to pass the filesystem object to these library's parquet reader methods too. The use case would be for users who are bought in on delta already but also want to use one of these ML libraries without duplicating effort on configuring credentials, generating the parquet file lists, etc. and want to do it in a consistent way without thinking about it.
r

rtyler

01/11/2023, 9:01 PM
I assume you're working largely in Python then?
j

Jim Hibbard

01/11/2023, 9:04 PM
Yes, largely in Python
5 Views