Hey Folks! If you are in SF on April 3rd, we have an exciting meetup for you to attend! lakeFS and DuckDB are hosting a meetup for all folks in the data community who want to learn and grow their Data skills. We'll have two sessions during this meetup: • Quacking DuckDB - local and in the cloud • ML experimentation made easy with lakeFS & DuckDB Details: • Date/Time: April 3rd, 5pm - 7pm • Location: San Francisco • Little Star Pizza, vegan food and drinks will be available https://www.meetup.com/lets-talk-data-sf/events/292111441/ Register in advance so we can make sure there is enough of everything for everyone 🙂

The PyData Seattle Conference is where users, contributors, and newcomers can share experiences to learn from one another and grow together! 🪴 This physical conference consists of 3-days of talks, tutorials, and discussions to bring attendees the latest project features along with cutting-edge use cases hosted at the Microsoft Conference center in Seattle, WA. 🗣️ Check out these awesome talks happening at the conference! • MLOps Deployment Patterns with Delta Lake and MLflow* * • Building Reliable, Open Lakehouses with Delta Lake • Scaling data workloads using the best of both worlds: pandas and Spark Delta Lake is excited to be a Diamond Sponsor! Get your tickets here.

Hey Seattle folks, we have a fun two-part Seattle Spark + AI Meetup coming up! 👏 Part 1️⃣ @Irena Kushner will present Migrating Bespoke ML to Horovod Part 2️⃣ @Simon Whiteley is hopping across the pond to join @Denny Lee LIVE to answer your data engineering questions from building a data platform to ingestion to ETL to analytics Come join us at the Databricks Bellevue Office on April 19th at 6 pm PT. In addition to the technical content and discussion, we will have great food including (but not limited to) Frelard Tamales, Dick's Burgers, Rachel's Ginger Beer, Hello Robin cookies, and other great local food joints! 🍔🍪😋 RSVP 👉 https://lnkd.in/gTRuf9cW

a twofer

Join @Gerhard Brueckl & @Denny Lee on April 27th for D3L2: Let's connect: Power BI to Delta Lake / VSCode Databricks Power Tools! 🦀 Let’s discuss the journey from traditional BI and data warehousing to highly scalable big data solutions using Apache Spark™ and Delta Lake through the lens of a Microsoft data guy and also see what role Power BI and Databricks played in this endeavor. RSVP ➡️ https://community.linuxfoundation.org/events/details/lfhq-delta-lake-presents-d3l2-lets-con[…]-power-bi-to-delta-lake-vscode-databricks-power-tools/

With so much focus on the developer, how do you do things like social, marketing, and events? Partnering with The Linux Foundation allows Delta Lake to do the many other important non-technical aspects that are just important to growing a community. 📣 Check out this OSSummit session: Focusing on the Developer and Partnering with The Linux Foundation Helps Delta Lake to Build a Flourishing Community by @Denny Lee and Carly Akerly! They will discuss the metrics and OKRs used to better understand the community, how Delta Lake has improved its global outreach and individual collaborations, and much more! 🙇 Learn about the session: https://lnkd.in/gyYTP-Rb ✏️ Register for OSS NA: (May 10-12) https://lnkd.in/eYv-iDi

Explore the latest advances in leading open source projects and industry technologies includes Delta Lake, MLflow, PyTorch, dbt, Presto/Trino, DuckDB & much more at #DataAISummit in San Francisco - June 26-29! 🎉 Check out the full range of sessions available to you. ➡️ https://go.delta.io/3KHV8vU BONUS: We have a special discount code for the Delta Lake open source community! Use code is ETLINUX400 (expires June 2) to save $400 off the regular price of the full conference pass. Register today 👉 https://go.delta.io/dais-2023-register

Join us TODAY, April 19th at Databricks Bellevue for a fun-filled and informative two-part Meetup. Get ready to enjoy a variety of local Seattle food/drink favorites including Dick's burgers (deluxe Dick's burgers, anyone?), Rachel's Ginger Beer, Ezelle's Fried Chicken, Paseo Caribbean Sandwiches, Hello Robin Cookies, & Frelard Tamales! We will have plenty of food in addition to the technical content so come HUNGRY! Event Program: 6:00 - 6:30 - Eat, beverages, and network! 6:30 - 7:15 - Talk: Irena Kushner - Migrating Bespoke ML to Horovod & Databricks 7:15 - 8:00 - Simon Whiteley + Denny Lee: Ask Us Anything 8:00 - 8:30 - Eat, drink, and network some more! Haven't RSVP'd? There's still time! Click here.

The PyData 2023 Pre-Conference meetup PARTY is next week! Join us on Tuesday, April 25th from 6:00 PM to 8:30 PM PDT at Databricks Bellevue. Event program: 6:00 - 6:30 - Eat, beverages and network. 6:30 - 6:40 - Announcements 6:40 - 7:10 - Talk: Chengyin Eng + Hyukjin Kwon- Introducing Spark Connect - The Power of Apache Spark, Everywhere 7:10 - 7:40 - Talk: Lu Qiu & Jasmine Wang Simplify and speed up cloud storage data access of AI training 7:40 - 8:20 - Panel Discussion - The past, present and future of open source. Denny Lee and Jules Damji - Learning Spark book authors 🌎📚💕 Host: Eloisa Elias T 🤠 WTM Ambassador 8:20 - 8:30 - Raffle 8:30 - 8:40 - Pie bar! 💯🎉🥧 RSVP: https://www.meetup.com/seattle-spark-meetup/events/291903857/

This Thursday, April 27 @ 9:00AM PDT! ⭐ D3L2: Let's connect: Power BI to Delta Lake / VSCode Databricks Power Tools! 🦀 Join @Gerhard Brueckl, Lead Data Engineer / Cloud Solution Architect at paiqo GmbH and key maintainer of the Power BI connector for Delta Lake, & host @Denny Lee to discuss Gerhard's journey from traditional BI and data warehousing technologies (e.g., Power BI, Azure, SQL Server) to building highly scalable big data solutions based on OSS technologies such as Apache Spark™ and Delta Lake. RSVP ➡️ https://community.linuxfoundation.org/events/details/lfhq-delta-lake-presents-d3l2-lets-con[…]-power-bi-to-delta-lake-vscode-databricks-power-tools/

🦀 D3L2: Ingesting Data from Delta Lake into SageMaker Studio and EMR 📆 Wed, May 17th 🕝 9:00 AM (PDT) Combining SageMaker Studio and Delta Lake brings state-of-the-art machine learning to your data lake. In this session with @Vedant Jain and @Denny Lee, we show how you can train ML models and how you can take advantage of the capabilities offered by Delta Lake using Amazon SageMaker Studio. RSVP: https://community.linuxfoundation.org/events/details/lfhq-delta-lake-presents-ingesting-data-from-delta-lake-into-sagemaker-studio-and-emr/

I wanted to give a shout out to @rtyler as Delta Lake's featured contributor! https://delta.io/profiles/tyler-croy. Give a shout out to all of his many contributions!

⭐ Session Spotlight ⭐ This year at Data + AI Summit, learn how Adobe Experience Platform (AEP) leverages Delta Lake to serve a large number of Adobe digital experience customers in this #DataAISummit session, Delta Lake Migration At Scale! 🦀 The session will cover the solutions to the above challenges and lessons learned from migration and they will discuss how scalable migration service becomes a core capability of the data platform. Don't miss the #DataAISummit on June 26-29! Use code ETLINUX400 (expires June 2) to save $400 off the regular price of the full conference pass. 📝View the schedule: https://go.delta.io/dais2023-schedule ✔️Register for Data+AI Summit: https://dbricks.co/3lvO1hz

This Wednesday, May 17th - Tune in live at 9:00AM PDT for another D3L2 session! ⭐ RSVP here. ⭐ Combining SageMaker Studio and Delta Lake brings state-of-the-art machine learning to your data lake. In this session, @Vedant Jain & @Denny Lee show how you can train ML models and how you can take advantage of the capabilities offered by Delta Lake using Amazon SageMaker Studio.

🦀 D3L2: Discussing Rust, Ballista, Ray SQL, Data Fusion with Andy Grove 📆 Thurs, May 25th 🕝 9:00 AM (PDT) Join this special D3L2 vidcast/podcast with @Andy Grove who has been specializing in query engines and distributed systems. Among many of his accolades, he started the DataFusion and Ballista query engine projects and donated both to the Apache Software Foundation as part of the Apache Arrow project. He also donated the initial Rust implementation of Apache Arrow and recently created Ray-SQL, a distributed SQL query engine in Python using Ray. RSVP: https://community.linuxfoundation.org/events/details/lfhq-delta-lake-presents-d3l2-discussing-rust-ballista-ray-sql-data-fusion-with-andy-grove/

📣 Data+AI Summit is one month away! Mark your calendars for June 26-29 and use code ETLINUX400 to save $400 off the regular price of the full conference pass! Code expires June 2. There will be plenty of awesome Delta Lake sessions including: • Why Delta Lake Is The Best Storage Format For Pandas Analyses by @Matthew Powers • What’s New With Data Sharing And Collaboration On The Lakehouse: From Delta Sharing To Clean Rooms with Steve Mahoney, @Celia Kung, and Erika Ehrli • Writing Data-Sharing Apps Using Node.Js And Delta Sharing by @will • Processing Delta Lake Tables On AWS Using AWS Glue, Amazon Athena, And Amazon Redshift with Noritaka Sekiyama • The Hitchhiker's Guide To Delta Lake Streaming with Scott Haines • & so many more! 💬 Explore DAIS sessions here: https://register.dataaisummit.com/flow/db/dais2023/sessioncatalog23/page/sessioncatalog?i[…]ga=2.65081006.1861864944.1684853375-235216433.1684450278 📝 Register for DAIS here: https://dbricks.co/3lvO1hz

🦀 Watch D3L2: Discussing Rust, Ballista, Ray SQL, DataFusion with Andy Grove here:

lol starting with #subaruchat

Join Robert Pack, Sr. Digital Expert Cloud Native Machine Learning Platform and Technology Principal at BASF as he discusses the relationship between process engineering and data engineering. ✅ In a connected world, there are very interesting graph-like relationships when we’re talking about processing chemicals in a sustainable way to efficiently fast querying of data to training and running machine learning models at large scale, high variety, and low cost. 📆 Thursday, June 15th 🕝 9:00AM PDT RSVP: https://community.linuxfoundation.org/events/details/lfhq-delta-lake-presents-d3l2-how-basf[…]s-global-sustainability-with-delta-lake-w-robert-pack/ cc @Robert & @Denny Lee

Uplevel your career with the Databricks training & certification program at Data+AI Summit in San Francisco, June 26-29. 👏 Learn LLMs, machine learning, and data engineering in hands-on trainings and take advantage of our complimentary onsite certifications to elevate your career. Don’t miss the chance to save 20% on all paid courses. Use discount code TR1mjrxeii before June 25. Register today! 😄

SAVE $400 off the regular price of your full conference Data+AI Summit pass! Use code is ETLINUX400. Code expires Friday, June 2. 🗓️ June 26-29 📍 San Francisco ✔️ Explore the latest advances in leading open-source projects and industry technologies including Delta Lake, MLflow, PyTorch, dbt, Presto/Trino, DuckDB & much more! Join the premier event for the global data community to learn about LLMs, lakehouse and all the latest innovations in data and AI! 👉 https://dbricks.co/3lvO1hz

Hey folks, This week is PrestoCon Day June 7th 🥳 We hope you can join us for a day of all things Presto; speakers from Bolt, Adobe, TikTok/Bytedance, Alibaba Cloud, IBM, Uber, and many more will be sharing how they use Presto at their companies. PrestoCon Day is virtual and free to register! Drop in when you can, just make sure to register at: https://events.linuxfoundation.org/prestocon-day/register/ Looking forward to seeing many of you there 👍

Upcoming Online Meetup ⭐ Tuesday, June 13 at 9:00 AM PDT Want to learn more about Databricks Connect and Spark Connect so you can use Spark from anywhere? 🌍 With Spark Connect, you have even more options to connect to Spark from your language of choice whether its PySpark, Scala, Go, Rust and/or your IDE of choice! Databricks Connect v2 leverages Spark Connect so you can connect to your Spark clusters within Databricks. Come join the awesome @Simon Whiteley, CTO of Advancing Analytics to discuss with a panel including @Martin Grund, Stefania Leone, and his partner in crime @Denny Lee. Don't forget we have a fun Spark Connect hackathon with 20K of prizes: So you think you can hack. See you soon! 🥳 RSVP: https://www.meetup.com/data-ai-online/events/293994300/

ONE WEEK from today! ⭐ Join @Robert, Sr. Digital Expert Cloud Native Machine Learning Platform and Technology Principal at BASF as he discusses the relationship between process engineering and data engineering alongside D3L2 host, @Denny Lee. 🦀 D3L2: How BASF achieves global sustainability with #DeltaLake w/ Robert Pack 🗓️ Thursday, June 15 🕝 9:00 - 10:00 AM PDT RSVP: https://community.linuxfoundation.org/events/details/lfhq-delta-lake-presents-d3l2-how-basf[…]s-global-sustainability-with-delta-lake-w-robert-pack/

Online Meetup ⭐ TOMORROW, June 13 at 9:00 AM PDT Learn more about Databricks Connect and Spark Connect so you can use Spark from anywhere! 🌍 Come join the awesome @Simon Whiteley, CTO of Advancing Analytics to discuss with a panel including @Martin Grund, Stefania Leone, and his partner in crime @Denny Lee. Don't forget we have a fun Spark Connect hackathon with 20K of prizes: So you think you can hack. See you soon! 🥳 RSVP ➡️ https://www.meetup.com/data-ai-online/events/293994300/

Tomorrow at 9:00AM PDT 🦀 D3L2: How BASF achieves global sustainability with #DeltaLake w/ Robert Pack! @Robert will discuss the relationship between process engineering and data engineering alongside D3L2 host, @Denny Lee. ⭐ Get ready for another live D3L2 session! RSVP: https://community.linuxfoundation.org/events/details/lfhq-delta-lake-presents-d3l2-how-basf[…]s-global-sustainability-with-delta-lake-w-robert-pack/

Not fully Delta Lake related, but I do plan on speaking a bit about it while I discuss Rust and INL's efforts in that direction at API Days Interface. I'm one of the keynotes on Day 2 🙂 https://www.apidays.global/interface/

LAST CHANCE! Space is filling up fast. Join us in San Francisco at the Moscone Center on June 26 - 29, 2023 for Data + AI Summit! 😁 🥳 Get the latest on Delta Lake — unlock the power of data engineering and streaming on the lakehouse as you discover innovations, best practices and how-tos, immersing yourself in a world of simplicity, unification and limitless possibilities. 🙌 Use code ETLINUX400 to save $400 off the regular price of the full conference pass. See you there! 👉 https://dbricks.co/3lvO1hz

Myself (@Scott Haines) and @Tristen will be presenting The Hitchhiker's Guide to Delta Lake Streaming in about 11 hours. Session Overview: https://www.databricks.com/dataaisummit/session/hitchhikers-guide-delta-lake-streaming/ (Room 12 if you are on site) Session Companion Material: bit.ly/hitchhikers-guide-to-dls Delta Lake Definitive Guide (Early Release): bit.ly/dldgv2 Please come and support us or any other Delta lake talk tomorrow.