Hi Team, I am looking to overwrite the databricks managed delta table using python Script,Not sure can i use delta-rs library for this ? We should’nt run the python script as datbricks job. Job can be executed in jenkins environment or AWS LAMBDA, execution of this script happens outside databricks env.

I am looking to overwrite the databricks managed delta table using python Script,Not sure can i use delta-rs library for this ?

All the metadata itself is contained in the table and not data bricks itself, so should be fine to do. The only danger is if you are using S3 and have multiple writers at the same time: outside of databricks they might overwrite each other. (This isn't an issue on other clouds)

@Will Jones , Thanks for response

I've done a ton of lambda work, I'm confused with what's being asked here @Kishore Shanmuganathan

@rtyler, i am looking for information regarding to pass AWS credentials or databricks tokens to delta-rs module for authentication . i didn’t find the relevant documentation.

does this help?https://www.buoyantdata.com/blog/2023-07-08-instance-authentication-delta-rust.html

Thank you, It really helped !!