Martin
01/26/2023, 1:55 PMALTER TABLE x SET LOCATION
statement (https://spark.apache.org/docs/3.0.0-preview/sql-ref-syntax-ddl-alter-table.html#set-location-and-set-file-format). But if I run this after I changed the physical location, it fails since at this point the table in Hive metastore is already corrupt since there is no Delta Table to be found any more are the stored location.
I can work around this issue by creating a dummy delta table (without registering it in Hive metastore) at the original location after I moved the Delta Lake table folder to its destination, run the the ALTER TABLE x SET LOCATION
statement and then simply delete the dummy delta table at the original location.
Another way for moving unmanaged Tables would be to move Delta Lake table folder using OS, run DROP TABLE x
and then CREATE TABLE x USING DELTA LOCATION
(https://docs.delta.io/latest/delta-batch.html#control-data-location). Downside of this approach is that we lose table properties and metadata.
Both approaches feel very clumsy to my. In my point of view, this should be a build-in feature. Am I missing something?Gerhard Brueckl
01/26/2023, 2:00 PM