06/29/2023, 6:19 PM
with delta 3.0 it's announced that the writes will be written in all three table formats (deltalake, iceberg, hudi). Will delta-rs also include that functionality for both read/write?
06/29/2023, 6:21 PM
It is in the protocol, so we'll pick it up soon
06/29/2023, 6:33 PM
Will liquid partitioning too? I don't see it's in the PROTOCOL
06/29/2023, 6:35 PM
Still being discussed here:
I'm not sure whether that will be a runtime concern or a protocol concern based on my understanding of the proposal at this point
06/29/2023, 6:41 PM
Does delta-rs do z-ordering today?
06/29/2023, 6:42 PM
indeed! there's some work
is actively doing here too:
06/29/2023, 6:52 PM
Yes, we have Z-order today in Python and Rust, but right now it requires a lot of memory for large unpartitioned tables. In the next release, we'll have much more efficient z-order for large unpartitioned tables (that's what the PR linked does).