09/08/2023, 2:15 PM
Hello i run my AWS Glue Jobs locally in the Docker Container. I want to convert a parquet file to delta file. I tried using pandas api with “to.Delta()” function and inside the container the command “spark-submit --deltalake delta” as argument. and i tried pip install delta-spark inside the container It fails always with “Failed to find data source: delta.”