https://delta.io logo
p

Payra

05/01/2023, 10:20 PM
Hi Is it possible to create two delta table on same data ? Why is this required is, I want to keep the data in a shared storage, I want to create one delta table in my side and keep appending my data everyday at the storage through the delta table. At the same time allow other users to run queries on those data through maybe another delta table on their compute engine. Will it work if I create multiple delta table on the same data ?
j

JosephK (exDatabricks)

05/01/2023, 11:36 PM
You shouldn't do what youre trying to do. Just have 1 table and let people read it while others are appending
s

Steve Quan

05/02/2023, 1:21 AM
Yup, if you would like a consistent version when people reading from a Delta Table. You can either designate the version name or use "version as of {timestamp}"
p

Payra

05/02/2023, 2:35 AM
@JosephK (exDatabricks) @Steve Quan Thanks for the response. So here is the problem, If I allow user to do query from my delta table then the compute cost will be mine. I want to people to pay for what they use. I am not sure how to do it here. Any suggestion would be appreciated. Thanks.
s

Steve Quan

05/02/2023, 3:35 AM
Databricks is decoupling storage and compute. If you share your Delta Table and ask your customer to query from their own workspace. Nothing will come from your pocket 😁
p

Payra

05/02/2023, 4:28 AM
Thanks @Steve Quan I think thats what I need. I need to share my delta table. How can I do it ? Any document or reference do we have? Please share.
I found this one. Did you mean this feature ? https://docs.databricks.com/data-sharing/index.html
j

JosephK (exDatabricks)

05/02/2023, 11:10 AM
No, you just use separate clusters for each operation. The users with the query will use 1 cluster and the writers will use a different one. Separate compute/billing
s

Sumanth Bo3

05/10/2023, 6:51 AM
Hi i know the thread is closed but have a follow up question @Payra when writing new data to delta table does the read operations get affected like i mean will the records be locked when writing the data
j

JosephK (exDatabricks)

05/10/2023, 11:28 AM
Reads are not affected by writes. When you start a read, you read version x and if a write starts during that read, it will write a new version that won't affect previous versions.
🙌 1
gratitude thank you 1
3 Views