Hello everyone, can delta sharing be used inside Azure ML notebooks either via pandas or pyspark?

Well I suppose so. It is just http requests based on the deltasharing config. So if the networking allows for reaching the endpoint you’re good to go

Yes, Kris is right! Assuming there's no network issues, you should be good to go 🙂

Thanks for that. I've been trying to make delta sharing work via azure ML but seem to be running in a 403 "this request is not authorized" error. I've given the Azure ML compute instance's public IP the network access on the adls storage as well as pointed it to the "config.share" file that has the auth token. The same token works when I run delta sharing in a python poetry project and can load the tables as both pandas and spark dataframes. Do you folks have any suggestions on what could be happening on Azure ML?

I'm assuming this is at work, so correct me if that's wrong. I'd probably start by sending an email/ticket to your networking/firewall team to see if there's something set up to block this traffic before further troubleshooting. It's kinda tricky to give better advice remotely unfortunately since this is more of a runtime question than a delta-sharing question 😕

Hi Jim, thanks for such a detailed response. I understand it's hard to troubleshoot remotely. In our use case, Azure ML can get to the storage, the delta sharing post request returns a 200 (please see attached screen). But when it tries to retrieve the storage contents is when the 403 error pops up (please refer to attached screen). So I dont think this is a networking issue?

Awesome! I'm glad you were able to track it down so quickly, hopefully this saved you some time 🙌