@Robert I am working with the

OptimizeBuilder

tonight, and I'm curious why

with_filters

takes

PartitionFilter<&'a, &'a str>

where T =

&'a str

rather than an allocated `String? Lifetimes are not my strength but I'm not sure how it would be possible to create

PartitionFilter

with anything other than static strings for the partition values. I have tried a couple different options with no success, and I think the

T

used for

PartitionFIlter

is just too restrictive 🤔

That’s a good point! The partition filters have been around since before the operations, and beyond avoiding allocations, I am not aware of any reason wha this choice has been made.

Ah, you're highlighted on

git-annotate(1)

because you moved the code, not because you wrote it 🙂

The open PR for optimistic commits actually starts to use datafusions expressions more deeply. With the upcoming core activities though, I hope we get a lower cost abstraction (in terms of dependencies) ,,,

the definition of

PartitionFilter

isn't too awkward, the lifetime constraints are a little odd. I'll experiment with

OptimizeBuilder

and just a different

T

I do believe that table maintenance / optimisation is something where delta-rs might actually be very well suited.

(I’ve been trying to nerd snipe Rust devs into doing that upstream object store work for months 😆 )

@Robert FWIW I started looking at updating

PartitionFilter

with some changes and it got messy really quick, so I just hacked around the lifetime issue a bit: https://github.com/buoyant-data/lambda-delta-optimize/blob/main/src/main.rs#L37-L53 😆