https://delta.io logo
#random
Title
s

sabari dass

04/03/2023, 3:26 PM
Hi All, I am getting below error when trying to get the value of timestamp field. Can anyone plz help me on this to fix it? Elastic Search data in my index- index1: { “id”: “123”, “__ts”: 1649289672103 } Pyspark code: es_df = spark.read.format(“es”).options(**es_conn_details) .options(“es.read.metadata”,”true”) .options(“es.read.metadata.version”,”true”) .options(“es.mapping.date.rich”,”false”) .load(“index1”) .select(“id”, “__ts”) es_xfm_df = es_df.selectExpr(“id”, “from_unixtime(__ts/1000, ‘yyyy-MM-dd HHmmss’) as __ts”) Error when try to execute display(es_xfm_df): RuntimeException: Error while encoding: Java.lang.RuntimeException: Java.lang.Long is not a valid external type for schema of string Caused by: RuntimeException: Java.lang.Long is not a valid external type for schema of string Thanks in Advance
2 Views