David Schenk

05/08/2023, 12:55 PM
Hey guys, have any of you experience with setting up the dynamodb lock with delta-rs in rust? I mean I have read the documentation and I am sure that as far as I could follow everything is set correctly. But the
panicked at
with an
ObjectStore { source: Generic { store: "DeltaS3ObjectStore", source: LockClientRequired } }
. Do any of you have any ideas or experience with this? In thread I will post some code snippets to better understand how my setup looks like.
Copy code
let mut storage_options = HashMap::new();
    storage_options.insert("AWS_REGION".to_string(), "eu-central-1".to_string());
    storage_options.insert("AWS_PROFILE".to_string(), "the_profile".to_string());
    storage_options.insert("AWS_S3_LOCKING_PROVIDER".to_string(), "dynamodb".to_string());
    storage_options.insert("DYNAMO_LOCK_TABLE_NAME".to_string(), "delta_writer_rust".to_string());
    storage_options.insert("DYNAMO_LOCK_PARTITION_KEY_VALUE".to_string(), "lockID".to_string());

let object_store = DeltaObjectStore::try_new(table_uri.parse().unwrap(), StorageOptions::from(storage_options.clone())).unwrap();
DynamoDB table is created before and PK is set to lockID as String
Ok, seems it has something to do with the storage_options that are provided in the DeltaObjectStore. If I set the configuration parameter es environment variable I get the following panic:
Copy code
ObjectStore { source: Generic { store: "DeltaS3ObjectStore", source: Dynamo { source: GetItemError(Validation("The provided key element does not match the schema")) } } }
But it looks to me that the schema definition is invalid that is used by delta-rs.
Ok, I’ve crawled through the integration tests and found this
function: Recreated my dynamodb table with
as partition key and code runs without issues. What am I missing here?


05/08/2023, 3:59 PM
@David are you just trying to use locking for your own writes? That is all done for you when the appropriate environment variables are set. You should not need to implement any locking yourself.
set and
set to
will cover it. Here's some terraform that sets up the dynamo table properly:
🙌 1
(there are some environment variables can be overwritten to change the name of the table and the hash key, but this is the default)

David Schenk

05/09/2023, 6:50 AM
Hey @rtyler, thank you this is exactly what I was searching for. Just tried to set the PK to something I know since it is not mentioned in the docs (all other parameters have a documented default value). I figured it out by reading the integration tests but adding it to docs might safe some hours of researching 😄