https://delta.io logo
a

Alber Tadrous

03/30/2023, 2:55 PM
Hi Team, I want to ingest a csv files to delta tables and I am facing a weird behavior , So I can read the file using spark context
spark.read.format("csv").load("s3://<file-path>")
but when I use Pandas it throws an error
j

Jim Hibbard

03/30/2023, 3:46 PM
Hi Alber, You could always load the data into spark first and then use the
toPandas()
function to retrieve your pandas DataFrame:
Copy code
df = spark.read.format('csv').load("s3://<file-path>").toPandas()
2 Views