Matthew Powers
03/10/2023, 3:22 PMlevi.skipped_stats(delta_table, filters=[('a_float', '=', 4.5)])
function that returns this {'num_files': 2, 'num_files_skipped': 1, 'num_bytes_skipped': 996}
. This allows users to see how many files / how much data gets skipped for different predicates. It’ll help them figure out when they should Z ORDER, etc. The new get_add_actions
API is opening up all sorts of new possibilities to get insights on Delta tables. Here’s the code if you’re interested.