Tejaswini Gorthi
02/28/2023, 2:38 PMtimestamp-micros
to sync timestamp columns, but looks like the library is unable to read the data. I am getting the below exception
scala.MatchError: LongValue(1677595007976000) (of class shadedelta.com.github.mjakubowski84.parquet4s.LongValue)
But i am able to read the data fine using Azure spark pool and also read the parquet files fine. Is there a way to read timestamp-micros
using the library?0 = {StackTraceElement@18976} "shadedelta.com.github.mjakubowski84.parquet4s.TimeValueCodecs$.com$github$mjakubowski84$parquet4s$TimeValueCodecs$$decodeLocalDateTime(ValueCodec.scala:203)"
1 = {StackTraceElement@18977} "shadedelta.com.github.mjakubowski84.parquet4s.TimeValueCodecs$$anon$12.decodeNonNull(ValueCodec.scala:284)"
2 = {StackTraceElement@18978} "shadedelta.com.github.mjakubowski84.parquet4s.TimeValueCodecs$$anon$12.decodeNonNull(ValueCodec.scala:281)"
3 = {StackTraceElement@18979} "shadedelta.com.github.mjakubowski84.parquet4s.OptionalValueCodec$class.decode(ValueCodec.scala:89)"
4 = {StackTraceElement@18980} "shadedelta.com.github.mjakubowski84.parquet4s.TimeValueCodecs$$anon$12.decode(ValueCodec.scala:281)"
5 = {StackTraceElement@18981} "<http://io.delta.standalone.internal.data.RowParquetRecordImpl.io|io.delta.standalone.internal.data.RowParquetRecordImpl.io>$delta$standalone$internal$data$RowParquetRecordImpl$$decode(RowParquetRecordImpl.scala:181)"
6 = {StackTraceElement@18982} "io.delta.standalone.internal.data.RowParquetRecordImpl.getAs(RowParquetRecordImpl.scala:168)"
7 = {StackTraceElement@18983} "io.delta.standalone.internal.data.RowParquetRecordImpl.getString(RowParquetRecordImpl.scala:111)"
8 = {StackTraceElement@18984} "com.fivetran.warehouses.adls.ADLSTestWarehouse.getValue(ADLSTestWarehouse.java:117)"
9 = {StackTraceElement@18985} "com.fivetran.warehouses.adls.ADLSTestWarehouse.queryTable(ADLSTestWarehouse.java:79)"
Scott Sandre (Delta Lake)
03/01/2023, 1:23 AM