Chanukya Pekala

01/10/2023, 9:59 AM
Hi, I would like to understand Databricks JAR based workflow tasks. Can I interpret JAR based runs to be something like a spark-submit on a cluster? In the logs, I was expecting to see the
spark-submit --class <|> --num-executors 4
etc., And, then there is another task type introduced recently I think — spark submit task type. If we have a JAR to execute, does spark submit task type and JAR type are implicitly working similar or different?

JosephK (exDatabricks)

01/10/2023, 12:43 PM
This isn’t a delta lake question. may be a better place to ask.