Hi All. Delta Sharing has a python client library. Is it compatible with SQLAlchemy? My goal is to use Superset to access Delta tables using Delta Sharing.

AFAIK Delta-Sharing only exposes information and give access to data files that construct a table. There's some support for e.g. predicate pushdowns but it doesn't provide any sort of query/SQL API that SQLAlchemy could use. I assume it's possible to read a delta-sharing table into a Pandas dataframe and then use something like DuckDB to turn it into something that is SQL query-able, and wrapping that with a custom SQLAlchemy dialect 🤯 Not quite a trivial solution I guess 🙂

Thanks @Oz Katz. It’s unfortunate that Delta Sharing doesn’t support SQL API because it can provide more access to other tools. Yeah, I read a similar suggestion on using DuckDB. But as you have said, it’s a “messy”, and it doesn’t exploits many benefits of Delta features such as data skipping and predicate push down. So I doubt it will have a good performance.