Hey everyone! Does anyone know how to get the DynamoDB Log Store going within Databricks for writes to Delta? I tried doing something simple like

spark.conf.set("spark.delta.logStore.s3.impl", "io.delta.storage.S3DynamoDBLogStore")
spark.conf.set("spark.delta.DynamoDBLogStore.ddb.region", "us-west-2")
spark.conf.set("spark.io.delta.storage.S3DynamoDBLogStore.ddb.tableName", "ddb_test")

df = spark.createDataFrame(
    [
        (1, "foo"),
        (2, "bar"),
    ],
    ["id", "label"]
)

df.write.mode("append").format("delta").save(<S3 path>)

and didn't see a DynamoDB table created. Would appreciate any pointers, thanks!

what version of DBR?

13.2

hm, i don't think it works yet at session level, can you set these at the cluster level

spark.delta.logStore.class io.delta.storage.S3DynamoDBLogStore
spark.databricks.tahoe.logStore.allowOSSLogStores true
spark.io.delta.storage.S3DynamoDBLogStore.ddb.tableName <dynamodb_table_name>
spark.io.delta.storage.S3DynamoDBLogStore.ddb.region <dynamodb_region>