Hi. Has anyone seen this error while creating a view over a complex struct type datasets:

Cannot have map type columns in DataFrame which calls set operations

To reproduce:

Select * from
(
SELECT struct('Spark', map(1,2)) as a
Union
SELECT struct('Spark', map(1,1)) as a
)

Please do share best practices on handling these cases

try

UNION ALL

.

UNION

tries to take only the distinct rows, and you can't do a

distinct

on MapType columns, i guess

Thank you so much. @Christopher Grant