I think we can't start on liquid clustering until a design doc is released with details. I don't think OSS Spark delta lake has this yet (but correct me if I'm wrong)
09/18/2023, 7:38 PM
Ah, I see, I wasn't aware that it wasn't open yet.
09/18/2023, 9:50 PM
Yeah, it's available in DBR 13.x from Databricks but not yet part of the upcoming Delta 3.0.0 release as far as I understand it.
I am still quite curious as to the implementation since there's some coordination between the read and write side of the equation to make it work 🤔