Bas Beelen

02/15/2023, 9:40 AM
I have an interesting/werid issue that I can not explain what is happening: I have several delta tables where
is enabled. And it has been working fine. Today though, for one of the tables we get a weird error message when we try to use
table_changes("the_table", <commit>)
, namely:
No file found in the directory: <gs://bucket/raw/.../_delta_log>
1. The
is at the location 2. I can query the table itself
select * from the_table
3. I can load previous versions
select * from the_table@v100
4. But this fails:
select * from table_changes("the_table", 100)
, with the above mentioned error Nothing in our logic has changed for the specific table where this is happening for. Other tables that run on the same logic of ingestion etc. don't have this specific issue. Anyone an idea what could be going on? We use Databricks on GCP

Yatharth Maheshwari

02/15/2023, 10:32 AM
Looks like this is because the _*delta_log* file have been deleted and now you cannot fetch the CDFs. The CDF files might or might not have been deleted based on your vacuum job.

Bas Beelen

02/15/2023, 11:01 AM
Thanks @Yatharth Maheshwari, looks like indeed I can actually run the table_changes from the latest version that is available and apparently I was tryhing to fetch from a version that was no longer present. TIL
🙌 1