Hey folks, This week is PrestoCon Day June 7th 🥳 We hope you can join us for a day of all things Presto; speakers from Bolt, Adobe, TikTok/Bytedance, Alibaba Cloud, IBM, Uber, and many more will be sharing how they use Presto at their companies. PrestoCon Day is virtual and free to register! Drop in when you can, just make sure to register at: https://events.linuxfoundation.org/prestocon-day/register/ Looking forward to seeing many of you there 👍

Upcoming Online Meetup ⭐ Tuesday, June 13 at 9:00 AM PDT Want to learn more about Databricks Connect and Spark Connect so you can use Spark from anywhere? 🌍 With Spark Connect, you have even more options to connect to Spark from your language of choice whether its PySpark, Scala, Go, Rust and/or your IDE of choice! Databricks Connect v2 leverages Spark Connect so you can connect to your Spark clusters within Databricks. Come join the awesome @Simon Whiteley, CTO of Advancing Analytics to discuss with a panel including @Martin Grund, Stefania Leone, and his partner in crime @Denny Lee. Don't forget we have a fun Spark Connect hackathon with 20K of prizes: So you think you can hack. See you soon! 🥳 RSVP: https://www.meetup.com/data-ai-online/events/293994300/

ONE WEEK from today! ⭐ Join @Robert, Sr. Digital Expert Cloud Native Machine Learning Platform and Technology Principal at BASF as he discusses the relationship between process engineering and data engineering alongside D3L2 host, @Denny Lee. 🦀 D3L2: How BASF achieves global sustainability with #DeltaLake w/ Robert Pack 🗓️ Thursday, June 15 🕝 9:00 - 10:00 AM PDT RSVP: https://community.linuxfoundation.org/events/details/lfhq-delta-lake-presents-d3l2-how-basf[…]s-global-sustainability-with-delta-lake-w-robert-pack/

Online Meetup ⭐ TOMORROW, June 13 at 9:00 AM PDT Learn more about Databricks Connect and Spark Connect so you can use Spark from anywhere! 🌍 Come join the awesome @Simon Whiteley, CTO of Advancing Analytics to discuss with a panel including @Martin Grund, Stefania Leone, and his partner in crime @Denny Lee. Don't forget we have a fun Spark Connect hackathon with 20K of prizes: So you think you can hack. See you soon! 🥳 RSVP ➡️ https://www.meetup.com/data-ai-online/events/293994300/

Tomorrow at 9:00AM PDT 🦀 D3L2: How BASF achieves global sustainability with #DeltaLake w/ Robert Pack! @Robert will discuss the relationship between process engineering and data engineering alongside D3L2 host, @Denny Lee. ⭐ Get ready for another live D3L2 session! RSVP: https://community.linuxfoundation.org/events/details/lfhq-delta-lake-presents-d3l2-how-basf[…]s-global-sustainability-with-delta-lake-w-robert-pack/

Not fully Delta Lake related, but I do plan on speaking a bit about it while I discuss Rust and INL's efforts in that direction at API Days Interface. I'm one of the keynotes on Day 2 🙂 https://www.apidays.global/interface/

LAST CHANCE! Space is filling up fast. Join us in San Francisco at the Moscone Center on June 26 - 29, 2023 for Data + AI Summit! 😁 🥳 Get the latest on Delta Lake — unlock the power of data engineering and streaming on the lakehouse as you discover innovations, best practices and how-tos, immersing yourself in a world of simplicity, unification and limitless possibilities. 🙌 Use code ETLINUX400 to save $400 off the regular price of the full conference pass. See you there! 👉 https://dbricks.co/3lvO1hz

Myself (@Scott Haines) and @Tristen will be presenting The Hitchhiker's Guide to Delta Lake Streaming in about 11 hours. Session Overview: https://www.databricks.com/dataaisummit/session/hitchhikers-guide-delta-lake-streaming/ (Room 12 if you are on site) Session Companion Material: bit.ly/hitchhikers-guide-to-dls Delta Lake Definitive Guide (Early Release): bit.ly/dldgv2 Please come and support us or any other Delta lake talk tomorrow.

About to present!

Hi everyone! If you missed Data+AI Summit in San Francisco, you can catch most session recordings on the Databricks YouTube channel. Here are just a few of my favorites: ⭐ Processing Delta Lake Tables on AWS Using AWS Glue, Amazon Athena, and Amazon Redshift (@Noritaka Sekiyama & @Akira Ajisaka): ⭐ Introducing Universal Format: Iceberg and Hudi Support in Delta Lake (@Himanshu Raja and @Ryan Johnson): ⭐ Massive Data Processing in Adobe Using Delta Lake: One Year In (Yeshwanth Vijayakuma): ⭐ Delta Kernel: Simplifying Building Connectors for Delta (Denny Lee & @TD): ⭐ Data + AI Summit Keynote, Thursday Part 3 - What's New with Delta Lake 3.0 (Michael Armbrust):

Hi everyone! We are excited to announce the establishment of a new study group in Tokyo called the Open Table Format Study Group Tokyo (OTF Tokyo). Our study group aims to share insights and knowledge about data formats for data lakes, including Hudi, Iceberg, Delta, and more. It's important to note that the topics presented in our study group are not limited to these data formats alone. We also cover surrounding technologies such as lower-level data formats like Parquet and Arrow, as well as other data lake-related technologies like data integration and data governance. The first session is scheduled to be held around September to October at the Amazon Japan Meguro Office. Since it also serves as a trial run, the first session will be a semi-closed event. However, if you are interested, please contact us. Presentations will be conducted in English and will be limited to in-person attendance only.

North Bay Python is a boutique one-track conference, held in Petaluma, California! This year, they're holding their conference on July 29th & 30th, in a barn, on a ranch overlooking the Petaluma River. Delta Lake is giving away 3️⃣ tickets to North Bay Python! Enter on LinkedIn here: https://www.linkedin.com/feed/update/urn:li:activity:7086836112760913921 The North Bay Python ticket giveaway ends this Friday, July 21st! We will select three winners at random and contact each winner directly. ⭐️

Don't miss Everything You've Wanted to Know About Delta Sharing LIVE this Thursday, August 3rd with Databricks Solution Architect & open source evangelist @will & Kubit CEO Alex Li! 🙌 Will will go over all the basics of Delta Sharing: how it works, and what types of data, workflows, and companies it’s a particularly good fit for. Alex will talk about the experience building the Kubit integration, and about why it make so much sense to leverage Delta Sharing for Product Analytics. Tune in LIVE on Kubit’s LinkedIn page on Thursday, August 3 at 11am PT / 2pm ET Event link*:* https://www.linkedin.com/events/7090698796313038848/

Hi Team this is Amit from Bangalore India, recently i build pipeline from MySQL binary log to Delta lake. Object storage - S3

Faced some issue related to merge and all but pipeline is kind of stable

This is batch job

Nice to meet you all thank you ☺️☺️☺️☺️

Hi everybody, Does anybody know if in the Netherlands a event/session is going to take place about Delta Live Tables? Thanks for your reaction in advanced

🎉 Language Creators Charity Fundraiser 🗓️ September 19, 2023 🕙 12:00 - 4:00PM PST 📍700 Bellevue Way NE · Bellevue, WA 🍿 The charity event brings together this unique group of computer science pioneers, unlike any event held before. These great minds come together for what will surely be a fantastic night of discussion as the panel delves into the past and future of programming language creation. The event will attract innovators and engineers from Seattle, the nation’s fastest-growing technology hub. The event is a benefit for Last Mile Education Fund and NumFOCUS. :rocket::ticket: Event ticket on sale now: https://pydata.org/language-creator-fundraiser/ 🚀 Limited edition T-shirt co designed and signed by Guido van Rossum are available as a donation add on item.

Open Source Summit Europe will be held in ⭐ Bilbao, Spain on Sept 19-21 ⭐! This is the premier event for open source developers, technologists, and community leaders to collaborate, share information, solve problems, and gain knowledge, furthering open source innovation and ensuring a sustainable open source ecosystem! The conference umbrella is composed of a collection of events covering the most important technologies, topics, and issues affecting open source today. 🎟️ Register + SAVE US$250: https://hubs.la/Q01ZFQ300

Hey, all you Rustaceans! Rust Global is the first Rust Foundation-hosted gathering dedicated to the use of Rust in global leadership settings. This half-day event (co-located with WasmCon) will feature networking opportunities and informative content focused on the future of Rust and its value to businesses, educators, governments, and professional software developers worldwide The event will be held on Wednesday, September 6 at 9:00am – 12:00pm in Bellevue, WA - Register today: https://events.linuxfoundation.org/rust-global/

Future roadmap visibility is a biggest challenge in delta lake 🤞

Hi there! If you are <!here>, you are going to definitely want to check out this giveaway. If you are attending #PyDataSeattle or will be in the Seattle area on September 19, you do NOT want to miss this event! The Language Creators Charity Fundraiser will feature the creators of C#, Java, Perl, Python, TypeScript, and Smalltalk in a conversation about programming language design. ⭐ To enter to win one of 10 VIP tickets to this celebration, simply share this image to LinkedIn and/or Twitter and use the hashtag #DeltaLake in your caption! Good luck!

REMINDER: If you are attending #PyDataSeattle or will be in the Seattle area on September 19, enter to win one of 10 VIP tickets to the Language Creators Charity Fundraiser The Language Creators Charity Fundraiser will feature the creators of C#, Java, Perl, Python, TypeScript, and Smalltalk in a conversation about programming language design. ⭐ To enter to win one of 10 VIP tickets to this celebration, simply share this image to LinkedIn and/or Twitter and use the hashtag #DeltaLake in your caption! Good luck!

We will be hosting a meetup event called Open Table Format Study Group Tokyo at Amazon Japan, which will center around topics related to open table formats! https://otfsg-tokyo.connpass.com/event/296440/

You can also find an English description on the webpage. Please check it out!

If you are attending PyData Seattle or will be in the Seattle area on September 19, enter to win one of the last few VIP tickets to the Language Creators Charity Fundraiser! The Language Creators Charity Fundraiser will feature the creators of C#, Java, Perl, Python, TypeScript, and Smalltalk in a conversation about programming language design. ⭐ Enter to win by hitting the 🥳 below! Winners will be chosen at random and contacted via Slack. Good luck! <!here>

https://fosdem.org/2024/news/2023-09-18-stands-cfp/ just a thought to share

Wednesday, Sept. 20 from 14:40 - 15:20 CEST @Florian Valeye presents Building Python Binding for the Delta Lake Library in Rust at OSSEU! 🦀 Florian will cover briefly what delta-rs is and its implementation, then focus on how combining Rust and Python code bases to have the best of two worlds by leveraging pyo3 library. You can tune in virtually on YouTube! 🔗 Check it out here:

LIVE this Friday, Sept 22nd at 10AM PT, don't miss Building Lakehouse on Delta Lake with @Matthew Powers and Xinran Waibel, Senior Data Engineer at Netflix and Founder of Data Engineer Things! In this webinar, Matthew will cover: ⭐ Data Lakehouse architecture compared to Data Warehouses / Data Lakes ⭐ Benefits of paring Delta Lake with Apache Spark ⭐ How Delta Lake can make Pandas / Polars / PyTorch analyses faster Tune in on YouTube ➡️