https://delta.io logo
s

Soukaina

05/24/2023, 11:10 AM
Hi everyone, is there a library or something that i can use so that i can partition my spatial dataset following the hilbert curve on the databricks runtime? Thank you in advance
j

JosephK (exDatabricks)

05/24/2023, 11:22 AM
zorder uses hilbert curves to do file layout which is similar to partitions
s

Soukaina

05/24/2023, 11:24 AM
when i read the github code of z-ordering it uses interleave bits which is purely in the z order curve algorithm not the hilbert curve one
j

JosephK (exDatabricks)

05/24/2023, 12:31 PM
that's open source, databricks is different
s

Soukaina

05/24/2023, 12:33 PM
so the z order in the open delta works with the z order curve while on databricks it works with hilbert curve?
j

JosephK (exDatabricks)

05/24/2023, 12:35 PM
yup
s

Soukaina

05/24/2023, 12:35 PM
i wanna compare the two layouts if it's by default hilbert curve on the databricks runtime how to switch to z curve
5 Views