Hey there, does delta (via databricks in this case) write out multiple parquet row groups in parquet files?
j
JosephK (exDatabricks)
04/10/2023, 7:26 PM
Depending on data size, yes. Why?
a
Alex Wilcoxson
04/10/2023, 7:47 PM
We are creating an arrow flight service over our delta lake tables using delta-rs and datafusion. On this datafusion discussion it was recommended to try to reduce rowgroup size: https://github.com/apache/arrow-datafusion/discussions/5901