https://delta.io logo
p

Per Tengkvist

03/24/2023, 11:05 AM
Hi All, maybe a newbie question... I've setup a minio storage on localhost:9000; no problem accessing it with minio client and s3fs (everything connects via http rest api, works like a charm). Then I have setup a delta-sharing server (deltaio/delta-sharing-server docker image) with a yaml config file as in delta-io/delta-sharing/server/src/universal/conf/delta-sharing-server.yaml.template on github. But how to set the "tables: location:" attribute? There are only examples on how to connect to open online data sources which require no credentials (access_key, secret_key ), and how to configure credentials when connecting to aws and gcp through environment variables and azure through xml-config-file. How do I configure my delta-sharing-server to connect to my minio server at localhost:9000 with access_key=abc123 and secret_key=abc123 - an example would be much appreciated? Note: I can access the delta-sharing-server using the python lib delta-sharing.SharingClient("mydeltacfgfile") and see the share, schema and table I have configured, but to access the actual file I need a way to connect to the path on the minio server.
k

Kris Geusebroek

03/25/2023, 6:24 PM
Have a look at https://github.com/godatadriven/datamesh where I have an example for this specific setup
p

Per Tengkvist

03/28/2023, 6:22 PM
Thanks for the feedback! So as far as I can see, the datamesh/config/delta/core-site.xml is where the "magic" happens, i.e. the configuration settings for how the delta sharing server connects to the storage servers... correct?
k

Kris Geusebroek

03/28/2023, 8:19 PM
Yes. The Hadoop libraries are used for storage connection and this is the config file they use. I believe there was a issue in delta sharing to be able to set these config in the startup arguments but not sure if that is implemented
p

Per Tengkvist

03/31/2023, 10:56 AM
thx!