https://delta.io logo
k

Kranthi Rao

05/15/2023, 2:53 PM
Do you need to use AWS Glue Crawler (with Native Delta Lake option) to be able to access delta table from AWS Redshift & Athena or are there other marketplace connector options ?
s

Samat Kurmanov

05/16/2023, 12:04 PM
1. Yes using crawlers 2. I think it is now working with Redshift tables 3. Just use Delta lake with Redshift Externals tables
k

Kranthi Rao

05/16/2023, 12:12 PM
Great. Thanks so much for your prompt response. For the 3rd point, we need to use delta lake table as source, do some transformations to the data and make it available in redshift for another workload. Since we need to perform Transformations to a rather large table (accounting for daily updates) , could you recommend a high level approach to handle incremental data ?