#delta-rs
Title
c
Cole MacKenzie
05/31/2023, 11:55 PMDoes anyone know if it is possible to track stats for datafusion delta-rs queries? Basically I want to see how many files where considered, pruned and then how many bytes fetched from the object store.
r
Robert
06/01/2023, 5:47 AMNot via "first class" support so far, but this is definitely on my todo list. Once an execution plan is executed, you can inspect the metrics generated by it. Many metrics that the datafusion operators supply could already be gathered, but I wanted to look into generating custom metrics more related to delta as well.
well ... come to think of it. the metrics you wanted should at least partially be available from the Parquet exec. we inspect this in the scanned files tests https://github.com/delta-io/delta-rs/blob/b17f2863dc561834e51c1aa951260cce1a46d4f7/rust/tests/datafusion_test.rs#L58C45-L68
c
Cole MacKenzie
06/01/2023, 5:11 PMOh very cool