https://delta.io logo
m

Marius Grama

02/23/2023, 11:29 AM
Is there a function in delta-lake OSS corresponding to
table_changes
(for reading from the change data feed) ? https://docs.databricks.com/sql/language-manual/functions/table_changes.html
While going through the delta-lake oss code I came over https://github.com/delta-io/delta/blob/e0e9b0095dcc5b1c4372474a54c87428340fe899/co[…]n/scala/org/apache/spark/sql/delta/commands/cdc/CDCReader.scala But there is apparently no SQL function exposing this functionality
c

Christopher Grant

02/23/2023, 5:52 PM
IMO, it's difficult to push delta-specific functionality - in this case, custom syntax - to Spark. feel free to open an issue on the Delta github if there isn't one already.
s

Scott Sandre (Delta Lake)

02/24/2023, 7:56 PM
Expect this in the next 2.3 Delta Lake release!
cc @Allison Portis
m

Marius Grama

02/24/2023, 7:57 PM
Thank you Scott. Great news. That’s a nice feedback.
You can update now https://github.com/delta-io/delta/issues/1307
Support querying Change Data Feed (CDF) using SQL queries
3 Views