https://delta.io logo
s

Starkadur Hrobjartsson

04/05/2023, 11:47 AM
Hi, I'm trying to create a delta table locally containing a generated column as follows:
CREATE TABLE IF NOT EXISTS test_schema.test_table
(
timestamp TIMESTAMP,
year INT GENERATED ALWAYS AS (YEAR(timestamp))
)
USING DELTA LOCATION '/tmp/delta-lake/delta-table'
TBLPROPERTIES(delta.minWriterVersion = '4')
I get the following error:
Syntax error at or near 'GENERATED'
The same table I can create in Databricks with the appropriate location, catalog and schema. Is this feature exclusive to Databricks or am I missing something?
j

JosephK (exDatabricks)

04/05/2023, 12:42 PM
Generate is databricks only
👍 1
j

Jim Hibbard

04/05/2023, 2:41 PM
Yes, it's only available in Databricks currently, can see some good discussion in the repo here. The feature depends on Spark 3.4 or greater and there's discussions within the Spark community how best to support the underlying capability. Hopefully we'll see it soon🤞
👍 1
4 Views