Does delta-rs suport automatic schema evolution? (addition of columns/renaming columns/missing columns in newer versions of data) on append?

I don't think so. I think those features would require column mapping, which we don't yet support.

ah, thanks.

Yes, all volunteers. It is no ones day job to maintain delta-rs right now.

Ah thanks for the info!

I think there are certain companies using delta-rs in production, so they can about maintaining the particular pieces they use. But the overall maintenance and direction is volunteer driven.

there are a lot of people contributing to delta-rs in a professional context (myself included) but @Will Jones’s first point is spot on: it's nobody's job explicitly to maintain delta-rs 😆 My employer (Scribd) created it originally but our focus has always been on the write path, there's a few folks who specialize more on the read path, some with Rust, some with Python 🙂 We're a motley crew that's for sure

We use delta-rs in production, for automatic schema evolution you have to currently read in the whole table, align with the change and overwrite, it works for small tables but for really large on it is a pain atm. Right now we use spark for large tables so it works out okay. We also combine trino and delta-rs together in tandem and handle merges via trino and load in memory tmp delta tables via delta-rs to handle those scenarios atm.