Channels
delta-social
h2g2-streaming
roapi
delta-oss
dais-2023
random
delta-rs
general
jobs
kafka-delta-ingest
deltalake-questions
events
lakefs
delta-cycling
deltalake-databricks-dbsql
delta-community
azuredatabricks
deltalake-on-aws
delta-sharing
flink-delta-connector
graph-delta-lake
dat
delta-sharing-rs
Powered by
#delta-rs
Title
i
Ian Joiner
05/23/2023, 5:17 PM
I wonder how hard it is for us to add new data types. In particular I need u128, u256, u512 as well as decimal512 & decimal1024 types.
r
rtyler
05/23/2023, 5:20 PM
the challenge is less at the delta-rs level and more at the arrow level, does arrow-rs already support them for parquet?
w
Will Jones
05/23/2023, 5:21 PM
Probably can be defined as Fixed-size-binary extension types. Though not sure what the extension types support looks like for arrow-rs and parquet-rs
👍 1
Ah Raphael beat to me that suggestion
👍 1
i
Ian Joiner
05/23/2023, 5:22 PM
@rtyler
Not yet. I just posted on the Arrow mailing list as well.
w
Will Jones
05/23/2023, 5:25 PM
relevant:
https://github.com/apache/arrow-rs/issues/2444
👍 1
Post