with delta 3.0 it's announced that the writes will be written in all three table formats (deltalake, iceberg, hudi). Will delta-rs also include that functionality for both read/write?
Will liquid partitioning too? I don't see it's in the PROTOCOL
r
rtyler
06/29/2023, 6:35 PM
Still being discussed here: https://github.com/delta-io/delta/issues/1874
I'm not sure whether that will be a runtime concern or a protocol concern based on my understanding of the proposal at this point
Yes, we have Z-order today in Python and Rust, but right now it requires a lot of memory for large unpartitioned tables. In the next release, we'll have much more efficient z-order for large unpartitioned tables (that's what the PR linked does).