https://delta.io logo
r

Roshan Punnoose

05/26/2023, 10:56 AM
Hi there, I have an int64 column and would like to know if it is possible to force delta encoding when Delta Lake/Spark writes the parquet file. And second, if it is possible to introspect the parquet files to check which encoding is used.
n

Nick Karpov

05/28/2023, 8:30 PM
i don't think* it's possible to specify the encoding, but take a look at this code snippet https://arrow.apache.org/docs/python/parquet.html#inspecting-the-parquet-file-metadata to see introspecting the parquet files