Hi All, Is there any way to do ‘dos2unix’ cmd in Databricks notebook. My usecase is I have a csv file in ADLS location but with some junk chars so I need to remove the junk chars while reading the file using pyspark in my databricks notebook. I can handle this using dos2unix cmd running manually but this suppose to be automate. Can anyone plz help on this? Thanks!

You can use %sh to call shell commands, though dos2unix is not installed on the dbr images. You could use sed or awk to do the same thing.

Or just use Python do achieve the same results: https://pypi.org/project/dos2unix/