I seem to remember that delta lake only tracks metadata for

Question

I seem to remember that delta lake only tracks metadata for the first 30 columns and thus column ordering matters for improving query performance but I can t seem to find this in the docs can someone point me in the right direction

chris fish · Answer

chris fish · Answer

```By default Delta Lake on Databricks collects statistics on the first 32 columns defined in your table schema You can change this value using the table property delta dataSkippingNumIndexedCols Adding more columns to collect statistics would add more overhead as you write files Collecting statistics on long strings is an expensive operation To avoid collecting statistics on long strings you can either configure the table property delta dataSkippingNumIndexedCols to avoid columns containing long strings or move columns containing long strings to a column greater than delta dataSkippingNumIndexedCols using ALTER TABLE ALTER COLUMN See ALTER TABLE```

Lennart Skogmo · Answer

found something called delta dataSkippingNumIndexedCols here <https learn microsoft com en us azure databricks delta table properties>

chris fish · Answer

just fyi our docs search is a little messed up right now so using google and `site <http docs databricks com|docs databricks com>` will usually help you find the thing you re looking for

Jeremy Jordan · Answer

ahh perfect thank you

Jeremy Jordan · Answer

~i m assuming this applies to the OSS delta lake as well ~ found the relevant section here <https docs delta io latest optimizations oss html data skipping>