With
https://github.com/delta-io/delta-rs/pull/1249 we're going to have to finally address the deprecation of arrow:
jsonreader:Decoder upstream. I have been working with that code a bit more lately so I might try to tackle it, but I pushed this change to a topic branch so others can collaborate on it if desired. I think the migration to
RawDecoder
will help some of the checkpoint memory pressure that
@Christian Williams and I have identified in production
#kafka-delta-ingest usage