https://delta.io logo
a

Ainesh Pandey

05/17/2023, 3:47 PM
Hi, I have a PySpark DataFrame in Databricks. I included the following code multiple times during processing:
Copy code
df_final.where(col('dh_state_name')=='Wisconsin').where(col('dh_place_name')=='Madison').where(col('property_type')=='Single-family').display()
These should be showing the exact same results, but sometimes it randomly says "Query returned no results". There is no consistency either; the exact same code will show different results across runs. All of the code is deterministic, so I'm confused about why this is happening. Any thoughts?
j

JosephK (exDatabricks)

05/17/2023, 7:46 PM
Not a delta io question
51 Views