A Vast Majority Of Linux's Input Improvements Are Developed By One Individual
Google Ejects Open-Source WireGuard From Android Play Store Over Donation Link In App
LLVM Clang RISC-V Now Supports LTO
Big security flaw in Linux sudo command - MyBroadband
Predicting NFL play outcomes with Python and data science
If you made through part 1, congrats! You have the patience it takes to format data. In that article, I cleaned up my National Football League data set using a few Python libraries and some basic football knowledge. Picking up where I left off, it's time to take a closer look at my data set.
read more
Drupal shows leadership on diversity and inclusion
I didn't expect DrupalCon Seattle's opening keynote to address the barriers that hold people back from making open source contributions. So imagine my surprise when Dries Buytaert, Drupal's project lead and co-founder and CTO of Acquia, which created Drupal, used his time onstage to share an apology.
read more
The Delta Lake Project Turns to Linux Foundation to Become the Open Standard for Data Lakes
Increasing community engagement shows commitment to open source and encourages industry-wide innovation for data-driven initiatives
Amsterdam and San Francisco, October 16, 2019 – The Linux Foundation, the nonprofit organization enabling mass innovation through open source, today announced that it will host Delta Lake, a project focusing on improving the reliability, quality and performance of data lakes. Delta Lake, announced by Databricks earlier this year, has been adopted by thousands of organizations and has a thriving ecosystem of supporters, including Intel, Alibaba and Booz Allen Hamilton. To further drive adoption and contributions, Delta Lake will become a Linux Foundation project and use an open governance model.
Every organization aspires to get more value from data through data science, machine learning and analytics, but they are massively hindered by the lack of data reliability within data lakes. Delta Lake addresses data reliability challenges by making transactions ACID compliant enabling concurrent reads and writes. Its schema enforcement capability helps to ensure that the data lake is free of corrupt and not-conformant data. Since its launch in October 2017, Delta Lake has been adopted by over 4,000 organizations and processes over two exabytes of data each month.
“Bringing Delta Lake under the neutral home of the Linux Foundation will help the open source community dependent on the project develop the technology addressing how big data is stored and processed, both on-prem and in the cloud,” said Michael Dolan, VP of Strategic Programs at the Linux Foundation. “The Linux Foundation helps open source communities leverage an open governance model to enable broad industry contribution and consensus building, which will improve the state of the art for data storage and reliability.”
Databricks’ cofounders are the original creators of the open source Apache Spark project, the unified analytics engine that has become the defacto standard for large-scale data processing. Databricks’ CEO and cofounder Ali Ghodsi expressed excitement in going through this journey again with the Delta Lake project. “Our team has continued to create and contribute to open source projects because we know it is the fastest, most comprehensive way to innovate. To address organizations’ data challenges we want to ensure this project is open source in the truest form. Through the strength of the Linux Foundation community and contributions, we’re confident that Delta Lake will quickly become the standard for data storage in data lakes.”
Delta Lake will have an open governance model that encourages participation and technical contribution and will provide a framework for long-term stewardship by an ecosystem invested in Delta Lake’s success.
Although initially designed to work with Apache Spark, Delta Lake has developed a thriving community which is adding support for other open source data systems.
“As a major cloud provider, Alibaba has been a leader, contributor, consumer, and supporter for various open source initiatives, especially in the big data and AI area. We have been working with Databricks on a native Hive connector for Delta Lake on the open source front, and we are thrilled to see the project joining the Linux Foundation. We will continue to foster and contribute to the open source community.”
– Yangqing Jia, VP of Big Data / AI at Alibaba
“Intel and Databricks have a long history of working together to advance Apache Spark technology with innovative data analytics and AI solutions and to enable enterprise readiness. Databricks Delta Lake contribution to the Linux Foundation is an important open source storage technology that can help the ecosystem improve reliability for data lakes. We look forward to joining in the Delta Lake project and continuing our collaboration with Databricks and the Apache community.”
– Wei Li, Vice President, Intel Architecture, Graphics and Software and General Manager, Machine Learning Performance
“The Starburst team is excited about the development of Delta Lake and have already developed a native connector for Presto that is currently in beta testing. We believe this will enable companies creating or migrating their data lakes to the cloud the ability to finally realize the value that they were prom ised years ago and perform interactive SQL analytics on data lakes directly.”
– Justin Borgman, CEO, Starburst
“Booz Allen Hamilton is very excited about the potential of Delta Lake technology, especially its promise to provide an open, scalable data platform to enable a broad range of analytics – SQL analytics that powers reporting and dashboarding to data science and machine learning with R & Python. We are looking forward to making significant contributions to the Delta Lake project. We are starting with native integrations of Apache Nifi with Delta Lake.”
– Dan Tucker, VP at Booz Allen Hamilton
For more information please visit Delta.io or follow @DeltaLakeOSS on Twitter.
About The Linux Foundation
Founded in 2000, the Linux Foundation is supported by more than 1,000 members and is the world’s leading home for collaboration on open source software, open standards, open data, and open hardware. Linux Foundation’s projects are critical to the world’s infrastructure including Linux, Kubernetes, Node.js, and more. The Linux Foundation’s methodology focuses on leveraging best practices and addressing the needs of contributors, users and solution providers to create sustainable models for open collaboration. For more information, please visit us at linuxfoundation.org.
The Linux Foundation has registered trademarks and uses trademarks. For a list of trademarks of The Linux Foundation, please see its trademark usage page: www.linuxfoundation.org/trademark-usage. Linux is a registered trademark of Linus Torvalds.
###
Media Contacts
Beth Handoll
ReTHINKitMedia
beth@rethinkitmedia.com
+1 415 535 8658
The post The Delta Lake Project Turns to Linux Foundation to Become the Open Standard for Data Lakes appeared first on The Linux Foundation.
How to Fix ‘semanage command’ Not Found Error in CentOS/RHEL
Khronos Opens Door For Allowing More Open-Source Drivers To Reach Conformance Status
TURNIP Vulkan Driver Gets MSAA Working
Red Hat and SAS: Enabling enterprise intelligence across the hybrid cloud
Every day 2.5 quintillion bytes of big data is created - this data comes from externally sourced websites, blog posts, tweets, sensors of various types and public data initiatives such as the human genome project as well as audio and video recordings from smart devices/apps and the Internet of Things (IoT).
Working together to raise mental health awareness: How Red Hat observed World Mental Health Day
Cultivating a diverse and inclusive workspace is an important part of Red Hat’s open culture. That’s why we work to create an environment where associates feel comfortable bringing their whole selves to work every single day. One way we achieve this mission is by making sure that Red Hatters who wish to share their mental health experiences, are met with compassion and understanding, but most importantly, without stigma. It is estimated that one in four adults suffers from mental illness every year.