🌟 Andrew Bauman contributed an official QuickStart Delta Lake Dockerfile! Perfect for testing tooling or for a dev environment! ➡️ This environment is packed with Delta Lake ecosystem tooling: ✅ delta-rs for Python & Rust ✅ PySpark ✅ Scala ✅ ROAPI... and much more! Check the docs for details! 🤓 [Detailed Information Here]

looks like the certificate has expired on docs.delta.io?

Hi everyone. I'm new to the community and excited to be here. I wanted to share a personal project I've been working on that I think might be of interest to some of you. It's a GitHub project called Kotosiro Sharing, and it is a Rust-based Delta Sharing Server that includes administration functionality. The implementation covers all the specification except CDF related APIs since I could not find the Rust crate equivalent of DeltaLog. If you're interested, you can check it out at GitHub. I would greatly appreciate any feedback or suggestions you may have. Thank you in advance for taking the time to review my work. I'm looking forward to getting to know all of you and contributing to the community.

Has anyone access External Location or Storage Credentials in UC enabled workspace in private endpoint.

Hello, I am currently learning about Delta Lake. I would like to utilize Delta Lake and Spark SQL for on-premises purposes. Specifically, I have CSV files stored in my Azure Blob Storage, and I would like to load these files into my Delta tables on-premises. Could you please provide me with guidance on where to begin, as I would like to avoid wasting time and effort by aimlessly searching for information? Thank you.

Hi, I'm using delta lake with AWS Glue. when i try to run the command --> deltaTable.optimize().executeCompaction() i get the following error --> AttributeError: 'DeltaTable' object has no attribute 'optimize' I'm using the jar --> delta-core_2.12-1.0.0.jar Pls help.

Atom / RSS feed for Delta Lake Blog Hi, I really enjoy the Delta Lake Blog: https://delta.io/blog Maybe I'm a little bit old-school but I still use an feed reader for my favorite sites. 👴 I was wondering if it would be possible to have a RSS feed for the Delta Blog so I'll get notified when there is a new article.

Hello I am running EMR cluster and moving delta jar file from S3 to

/usr/lib/spark/jars/

through bootstrap actions. I am able to read/write delta files. But, when I do

import delta

my job is failing with

ModuleNotFoundError: No module named 'delta'

Any help is much appreciated 🙂

Who's going to PyData Seattle next week? Would be great to meet up if you're attending! delta io

Do you have any event for free tshirt or paid tshirt for delta lake ?

Good morning! The NYC Department of Citywide Administrative Services is hosting an NYC Government Hiring Hall on Friday, April 28th in Queens! We're hiring for a wide variety of titles and positions, but especially Engineers, Architects and Inspectors! Engineers have a hand in all of the systems and structures that make our lives possible, like buildings, highways, parks, waterways, the electrical grid and more. They also play a major role in tackling some of New York City's greatest challenges. Many City agencies will be present, most of which will conduct on-the-spot interviews for select jobs in City government including: • Apprentice (Construction Laborer) • Assistant Landscape Architect • Construction Laborer Apprentice • Chemical Engineer • City Research Scientist • Civil Engineer • Electrical Engineer • Inspectors (Construction, Housing and Plumbing) • Mechanical Engineer • Stationary Engineer • And more! Event name: NYC Government Hiring Hall Date: Friday, April 28, 2023 Time: 10:00 a.m. – 3:00 p.m. Location: York College – 160-20 Liberty Avenue, Jamaica, Queens 11451. (Between 160th and Guy R Brewer Blvd.) Registration: bit.ly/april28nychiringhall Please share the above information and attached flyer with any friends and colleagues looking for an employment opportunity! We will be hosting these events on a regular basis and will continue to share them with you as the flyers are available. Candidates can also add a "save the date" for the next event, which will be held on Saturday, May 6th in Bronx. More information to follow. Please reach out to Citywidejobs@dcas.nyc.gov, if you have any questions. Sincerely, DCAS NYC Department of Citywide Administrative Services (EC) P: 212-386-1701 | Citywidejobs@dcas.nyc.gov Follow DCAS: Instagram | Twitter | Facebook | LinkedIn | Inside Citywide Podcast

Does Trino support delta sharing?

Otherwise, How can I query delta lake table in Unity Catalog?

I hope you enjoy the read 😃 😃 Any feedback would be much appreciated https://www.linkedin.com/posts/databeans_mergeschemaevolution-activity-7060311422156566529-8MyV?utm_source=share&utm_medium=member_ios

@Sukumar Nataraj happy to share with you the release of our lighthouse project which we discussed previously. I hope you are still interested https://databeans-blogs.medium.com/z-ordre-take-the-guesswork-out-bad0133d7895

Hi All, Can I use Mlflow to access delta tables within hive metastore? Looks like I need to provide delta table as one of the inputs to query it from within mlflow.

If someone is interested in event processing architecture here is an interesting article about how to build an integration hub :) https://medium.com/gastromatic/synchronizing-data-using-memphis-dev-a-case-study-2e6e9a7b5512

Hello team, I have 1 question. I am using Apache Spark Streaming to run Merge Query to Delta Lake table. My table turn on optimize performance. Meaning after run Merge Query complete, it is automatic run Optimize query. I want to turn of that to improve Streaming batch. How can I turn off this feature ?

Hi 👋

Microsoft just announced a new SAAS product at Microsoft Build: Microsoft Fabric. It is a data analytics and BI platform, and they chose Delta Lake as storage format. delta io https://learn.microsoft.com/de-de/fabric/

The state of data engineering 2023 is out. As data practitioners, im sure you'll find it interesting, and we look forward to your feedback. https://lakefs.io/blog/the-state-of-data-engineering-2023/

Happy to share with you a simple explanation of one of delta’s latest added features: deletion vectors. Enjoy the read. https://www.linkedin.com/posts/databeans_deltalake-deltalake-deltalake-activity-7069022797703847939-8tpX?utm_source=share&utm_medium=member_ios

I have started writing a blog post series that explains the delta-rs Python bindings: https://medium.com/@ognis1205/delta-rs-as-a-python-rust-ffi-example-part-1-b11f53b40f9c I hope you enjoy the read 🙂

This is my second part of the blog post series that explains the delta-rs Python bindings: https://medium.com/@ognis1205/delta-rs-as-a-python-rust-ffi-example-part-2-f0cb386a04bd I hope you enjoy the read 🙂

Hi all, this is my recent work on Delta, the library of Native Delta Lake Transaction Log Implementation in Go https://github.com/csimplestring/delta-go It now supports local file, azure blob, gcs and s3 (single driver) Any feedback/pull request/question are welcome.

my next plan is to make it more production-ready and integrated with some Go data processing tools, any suggestions are welcome!

Hi, Is there any slack group for Databricks? (not just Delta Lake)