Usman Zubair08/17/2023, 10:11 PM
for data arriving throughout the day. They currently use
but are switching to
. They are encountering some challenges: 1. Polars Categorical data type is not supported in a delta format (use strings as categories to minimize data size) 2. Write a pyarrow to Delta (they are able to cast a polars table as a pyarrow table) (1) is not supported yet so they would have to explicitly cast but is there a solution for (2)?