#delta-oss
- m
Matthew Powers
05/03/2023, 2:11 PMHey everyone. The purpose of this channel is to discuss project like mack, levi, eren, jodie, hio, diane, and some cool other project that will be released soon.👀 1✅ 1 - m
Matthew Powers
05/03/2023, 2:12 PMThere is so much overlap between these projects and opportunities for collaboration. The transparent communication will benefit all of us!! - m
Matthew Powers
05/03/2023, 2:36 PMWelcome @Souvik Pratiher!!👋 1 - s
Souvik Pratiher
05/03/2023, 2:37 PMHi everyone!!delta lake - m
Matthew Powers
05/03/2023, 2:39 PM@Souvik Pratiher - I am working on a JumpSpark LinkedIn post right now. Other folks: this is a cool project to create a PySpark project with a single command: https://github.com/Spratiher9/JumpSpark🙏 1s- 2
- 1
- m
Matthew Powers
05/03/2023, 2:39 PM@Dan Crystal Beach - Welcome!!d- 2
- 1
- m
Matthew Powers
05/03/2023, 3:26 PMgratitude thank you 1apache spark 3 - j
Joydeep Banik Roy
05/03/2023, 5:04 PMHi everyone delta io - m
Matthew Powers
05/03/2023, 6:52 PMHey Joydeep, will be great for the mack contributors to learn about your awesome Change Data Feed contributions! - j
Joydeep Banik Roy
05/04/2023, 4:49 PMSure @Matthew Powers, do we port some of these functionalities to mackm- 2
- 1
- m
Matthew Powers
05/04/2023, 9:58 PMDatabeans just open source a tool called lighthouse to provide users with insights on when data maintenance tasks should be performed on their Delta tables. Check it out @Joydeep Banik Roy @Brayan Jules and @Nick Karpov: https://github.com/Databeans/lighthouse👀 1🥳 1 - m
Matthew Powers
05/04/2023, 10:04 PM@Abidi Gassen - welcome 👋✋ 1 - m
Matthew Powers
05/04/2023, 10:04 PMReally excited about your new project!🥳 1 - a
Abidi Gassen
05/04/2023, 10:23 PMThank you @Matthew Powers, indeed we’re super excited about the first release of #lighthouse project and looking forward to the community’s review and feedback. Looking forward to answer any questions about the project and discuss it’s future given the huge uncovered axis of cleaning up your lakehouse from a data layout perspective.👍 2 - a
Abidi Gassen
05/04/2023, 10:39 PMFor those interested in a more profound dive into the whys and hows of the project, I strongly recommand reading thses two blog posts: https://databeans-blogs.medium.com/z-ordre-take-the-guesswork-out-bad0133d7895 - a
Abidi Gassen
05/04/2023, 10:39 PM - j
Joydeep Banik Roy
05/05/2023, 6:36 AM@Abidi Gassen great project, I just checked it out. Have a few ideas. let's connect❤️ 3 - j
Joydeep Banik Roy
05/05/2023, 9:56 AMHi delta io guys Please check out my new blog on Delta Merge Optimization:- https://www.linkedin.com/posts/joydeep-banik-roy_delta-mergeoptimisation-strategies-a[…]867588489216-WDby?utm_source=share&utm_medium=member_desktop👍 1 - m
Matthew Powers
05/08/2023, 9:49 PMJust made a fairly large quinn refactoring to make the Ruff lint checks happy: https://github.com/MrPowers/quinn/pull/104👍 1 - m
Matthew Powers
05/08/2023, 9:52 PMHey @Sem Sinchenko, glad to have you in this chat!🖖 1 - m
Matthew Powers
05/09/2023, 1:02 PM@Joydeep Banik Roy - this PR to add users more insights into the sizes of files in their Delta table is cool. I wanted to make you aware of another project called levi that I am working on that is providing similar functionality. Levi uses delta-rs, which exposes easier APIs for figuring out the list of files / file sizes in a Delta Table. Let me know what you think. - m
Matthew Powers
05/12/2023, 6:39 PM@shingo - Hey there. This channel is a smaller group of folks that are collaborating on different projects in the Delta open source ecosystem. Happy to have you in this channel 👋 - n
Nick Karpov
05/12/2023, 6:42 PM👋 - s
shingo
05/12/2023, 11:46 PM👋 - s
shingo
05/12/2023, 11:47 PMThanks for the invitation! - m
Matthew Powers
05/15/2023, 5:38 PM@Michael Shtelma - welcome! Everyone else - Micheal just donated deltatorch to the delta-incubator organization. This is a great way to read Delta tables to perform PyTorch analyses. We will add some issues with the “good first issue” label soon if you’d like to contribute.👍 1s- 2
- 3
- m
Michael Shtelma
05/16/2023, 10:40 AMThanks a lot, @Matthew Powers - s
shingo
05/17/2023, 12:16 AMI just published a blog post on how to deploy a self-hosted Delta Sharing server on AWS. Please check it out, and any feedback would be welcome! https://medium.com/@ognis1205/how-to-deploy-self-hosted-delta-sharing-server-on-aws-7aa2a7f6f46👍 6md- 3
- 4
- s
shingo
06/01/2023, 9:32 AMI have started writing a blog post series that explains the delta-rs Python bindings: https://medium.com/@ognis1205/delta-rs-as-a-python-rust-ffi-example-part-1-b11f53b40f9c🙌 2rs- 3
- 6
- a
Abidi Gassen
06/12/2023, 11:02 PM