I wanted to share some updates on the datafusion upgrade:
https://github.com/delta-io/delta-rs/pull/1249
Basically datafusion 21 incorporates a newer (not newest) version of arrow which has some bugs preventing our use of it. As such we need datafusion 22 to come out which upgrades to arrow 36 😦