Can anybody tell me if spark.databricks.delta.merge.enableLowShuffle is currently an option for open source delta lake? Broader question, is there a place to see all applicable config options? I've looked through the docs but haven't found it.

I'd also appreciate an overview of the OSS supported configs. I think that would be very helpful. Concerning `spark.databricks.delta.merge.enableLowShuffle`: I think it is still Databricks proprietary and not OSS

There is no list and that's intentional. @Serge Smertin made a list once

Joseph, you keep behaving unprofessionally even outside of Databricks 😉

Jokes aside, it would be very helpful to have such a list. The one in the docs is not exhaustive afaik.

the problems is that such a list is impossible to keep up-to-date across different DBRs 🤷

But for OSS it would just be a property of the version, right? It's easy for me to say when I'm not doing the work, but I think once the initial list is made for the current release, it shouldn't require much effort to maintain for future releases when available options are added/removed

let's say, not all of the configuration properties are made to be for public use. not all of them are guaranteed to work. some of them are like feature flags for highly experimental things

I get that but I still think having a list of stable properties for OSS would be highly beneficial, especially for people that are new to Delta. It is sometimes really confusing to figure out which property is only available on DBX. If that is not possible it would be helpful if you could explicitly mention in the documentation that this list is not exhaustive for this reason.

Come on @JosephK (exDatabricks) - please refrain from personal attacks and follow our code of conduct https://github.com/delta-io/delta/blob/master/CODE_OF_CONDUCT.md. Personal attacks are grounds for removal from this community environment.

I edited my comment

Thanks! I always appreciate your candor @JosephK (exDatabricks) - just a little less spicy please 😉

@Denny Lee same list? 😛 for same reasons? 😛 that list could be automatically created, but the real difficulty is to know either or not certain config do work.

Fair enough - we will need to generate it, review it, and then publish it, eh?! 🙂

you have the wiki page i've created, right? 😛

You know the answer @Serge - why do you call me out 🙂

Did this ever get published or an issue created? Could someone link to the wiki page in question?