I'm trying to unit test code running locally, but the delta sql extensions/V2 datasource don't seem to be loaded:
got spark configuration with:
delta_core is version 2.2.0
spark is 3.3.2
CREATE TABLE IF NOT EXISTS test
year INT GENERATED ALWAYS AS (year(timestamp)))
PARTITIONED BY (year);"""
Syntax error at or near 'GENERATED'
year INT GENERATED ALWAYS AS year(timestamp))
(a) works fine on databricks (b) If I remove GENERATED clause works fine and create delta table in local test
Seems the SQL extension is not loaded? anything else I need to do?
Looks like not supported and need to use delta API
OPTIMIZE works, so extension registered