https://delta.io logo
p

Parthiban Jaganathan

07/18/2023, 6:50 PM
Does open source version of delta 3.0 supports glue catalog for delta uniform?
s

Scott Sandre (Delta Lake)

07/19/2023, 8:05 PM
UniForm in the delta 3.0 preview is file-system-based, not catalog-based, so no
1
p

Parthiban Jaganathan

07/19/2023, 8:18 PM
but 3.0 preview document talks about creating a table before publishing data. how can we create delta lake uniform without table?
s

Scott Sandre (Delta Lake)

07/21/2023, 6:17 PM
but 3.0 preview document talks about creating a table before publishing data
Can you please link me to this?
p

Parthiban Jaganathan

07/21/2023, 6:22 PM
@Scott Sandre (Delta Lake) Is there a way i can set the iceberg property against delta lake which resides in s3 without catalog reference?
s

Scott Sandre (Delta Lake)

07/21/2023, 7:07 PM
@Parthiban Jaganathan - I don't see the word "publishing" on that website. which part of the docs talk about "Creating a table before publishing data"
To set an iceberg table property, you must set the delta table property like so
Copy code
delta.universalformat.config.iceberg.<my_iceberg_table_property>
i.e. just use the prefix
delta.universalformat.config.iceberg.
1
p

Parthiban Jaganathan

07/21/2023, 7:15 PM
Copy code
spark.sql("ALTER TABLE delta.`path\to\delta\table` SET TBLPROPERTIES ('delta.universalFormat.enabledFormats' = 'iceberg')")
This would kickoff manifest generation for iceberg?
s

Scott Sandre (Delta Lake)

07/21/2023, 7:16 PM
when altering an existing table (to enable uniform), you also need to 1. enable column mapping name mode.
'delta.columnMapping.mode' = 'name'
2. enable the icebergcompatv1 table feature
'delta.enableIcebergCompatV1' = 'true'
1
yup, that will kick off the iceberg metadata conversion on each delta commit!
p

Parthiban Jaganathan

07/21/2023, 7:18 PM
Thank you @Scott Sandre (Delta Lake) I will test using the above setting. Hope we get glue catalog support sooner 🙏