https://delta.io logo
s

Suraj Malthumkar

05/25/2023, 7:22 AM
Hello Everyone, How to Read parquet data files using delta connector when the data files committed to delta table via standalone library ? Am using Apache Avro and Apache Parquet libraries directly to convert data to the Parquet format.
Copy code
System.out.println("Delta Read");
spark.read().format("delta")
        .load("<s3a://delta-laketest/spark_table>")
Also when am running this(above code) it Does not read the parquet files. As the logs and checkpoint files points to the parquet files registered. How do i read it using delta connector ? 2nd way: Reading Parquet data (single-JVM)
Copy code
DeltaLog log = DeltaLog.forTable(conf, "<s3a://delta-laketest/my_table>");
CloseableIterator<RowRecord> rowItr = log.snapshot().open(); //update().open();
Code above was able to Read parquet data files after committed to delta table via standalone library. Is this correct way to Read parquet data files using delta connector when the data files committed to delta table via standalone library ? I would like to understand whats the difference in both ways? I am pretty new to the delta, please guide me through this. Thank you for your help! :)