I know We could use the deltalake python package for this pr

Question

I know We could use the deltalake python package for this problem chaos ensues gt ArrowNotImplementedError Unsupported cast from map gt to map gt no available cast function for target type zombie

rtyler · Answer

I think this is because of a column with `map lt string array lt string gt gt ` I am kind of hoping this is fixed in `main` but I cannot test at the moment

Will Jones · Answer

Do you have the full source and target type

rtyler · Answer

not sure what you mean by target type the type that I listed comes from the `describe table` output of Spark SQL

Will Jones · Answer

Is it basically that we can t read a map of string to array of string

rtyler · Answer

that is my understanding of the error since this is a primitive `<http dt to|dt to> pandas ` call

rtyler · Answer

MUHARHARHAR using `columns` to skip it does seem to solve the problem

rtyler · Answer

< Will Jones> I cannot share this data for hopefully obvious reasons is there a good approach to provide a useful bit of data for you others in a bug report

Will Jones · Answer

Are you able to get the arrow schema off the Python API I think that s the main hint we need

Will Jones · Answer

Hopefully it s not too much more complex than that

rtyler · Answer

I understand the words you re using but not how to translate that into code stuck out tongue I assume you re looking for something more than `dt schema `

rtyler · Answer

The result from point up is `Field request headers MapType PrimitiveType string ArrayType PrimitiveType string contains null=True value contains null=True nullable=True metadata= comment REDACTED `

Will Jones · Answer

I think there s just one to PyArrow you could chain on there but what you ve provided seems sufficient