Hi All, maybe a newbie question... I've setup a minio storage on localhost:9000; no problem accessing it with minio client and s3fs (everything connects via http rest api, works like a charm). Then I have setup a delta-sharing server (deltaio/delta-sharing-server docker image) with a yaml config file as in delta-io/delta-sharing/server/src/universal/conf/delta-sharing-server.yaml.template on github.
But how to set the "tables: location:" attribute? There are only examples on how to connect to open online data sources which require no credentials (access_key, secret_key ), and how to configure credentials when connecting to aws and gcp through environment variables and azure through xml-config-file.
How do I configure my delta-sharing-server to connect to my minio server at localhost:9000 with access_key=abc123 and secret_key=abc123 - an example would be much appreciated?
Note: I can access the delta-sharing-server using the python lib delta-sharing.SharingClient("mydeltacfgfile") and see the share, schema and table I have configured, but to access the actual file I need a way to connect to the path on the minio server.
Thanks for the feedback! So as far as I can see, the datamesh/config/delta/core-site.xml is where the "magic" happens, i.e. the configuration settings for how the delta sharing server connects to the storage servers... correct?
03/28/2023, 8:19 PM
Yes. The Hadoop libraries are used for storage connection and this is the config file they use.
I believe there was a issue in delta sharing to be able to set these config in the startup arguments but not sure if that is implemented