https://delta.io logo
h

Heet Bhimani

06/22/2023, 9:52 AM
Hellow!!, I'm currently experiencing a datatype mismatch and schema inconsistency issue when querying the same table in both Databricks and AWS Athena. Although the datatype is specified as a timestamp in both cases, when I use the typeof query, I see different output. I'm seeking assistance to understand if there are any missing configurations or settings that need to be adjusted. My goal is to have a timestamp without a time zone. 1st Databricks: 2nd AWS Athena:
m

Michael Nacey

06/27/2023, 8:20 PM
See this article about the same issue with Hudi. It looks like a Spark INT96 format problem more than a Delta or Hudi problem: https://github.com/apache/hudi/issues/2123