Open-source News

If you made through part 1, congrats! You have the patience it takes to format data. In that article, I cleaned up my National Football League data set using a few Python libraries and some basic football knowledge. Picking up where I left off, it's time to take a closer look at my data set.

read more

I didn't expect DrupalCon Seattle's opening keynote to address the barriers that hold people back from making open source contributions. So imagine my surprise when Dries Buytaert, Drupal's project lead and co-founder and CTO of Acquia, which created Drupal, used his time onstage to share an apology.

read more

Increasing community engagement shows commitment to open source and encourages industry-wide innovation for data-driven initiatives

Amsterdam and San Francisco, October 16, 2019 – The Linux Foundation, the nonprofit organization enabling mass innovation through open source, today announced that it will host Delta Lake, a project focusing on improving the reliability, quality and performance of data lakes. Delta Lake, announced by Databricks earlier this year, has been adopted by thousands of organizations and has a thriving ecosystem of supporters, including Intel, Alibaba and Booz Allen Hamilton. To further drive adoption and contributions, Delta Lake will become a Linux Foundation project and use an open governance model.

Every organization aspires to get more value from data through data science, machine learning and analytics, but they are massively hindered by the lack of data reliability within data lakes. Delta Lake addresses data reliability challenges by making transactions ACID compliant enabling concurrent reads and writes. Its schema enforcement capability helps to ensure that the data lake is free of corrupt and not-conformant data. Since its launch in October 2017, Delta Lake has been adopted by over 4,000 organizations and processes over two exabytes of data each month.

“Bringing Delta Lake under the neutral home of the Linux Foundation will help the open source community dependent on the project develop the technology addressing how big data is stored and processed, both on-prem and in the cloud,” said Michael Dolan, VP of Strategic Programs at the Linux Foundation. “The Linux Foundation helps open source communities leverage an open governance model to enable broad industry contribution and consensus building, which will improve the state of the art for data storage and reliability.”

Databricks’ cofounders are the original creators of the open source Apache Spark project, the unified analytics engine that has become the defacto standard for large-scale data processing. Databricks’ CEO and cofounder Ali Ghodsi expressed excitement in going through this journey again with the Delta Lake project. “Our team has continued to create and contribute to open source projects because we know it is the fastest, most comprehensive way to innovate. To address organizations’ data challenges we want to ensure this project is open source in the truest form. Through the strength of the Linux Foundation community and contributions, we’re confident that Delta Lake will quickly become the standard for data storage in data lakes.”

Delta Lake will have an open governance model that encourages participation and technical contribution and will provide a framework for long-term stewardship by an ecosystem invested in Delta Lake’s success.

Although initially designed to work with Apache Spark, Delta Lake has developed a thriving community which is adding support for other open source data systems.

“As a major cloud provider, Alibaba has been a leader, contributor, consumer, and supporter for various open source initiatives, especially in the big data and AI area. We have been working with Databricks on a native Hive connector for Delta Lake on the open source front, and we are thrilled to see the project joining the Linux Foundation. We will continue to foster and contribute to the open source community.”

– Yangqing Jia, VP of Big Data / AI at Alibaba 

“Intel and Databricks have a long history of working together to advance Apache Spark technology with innovative data analytics and AI solutions and to enable enterprise readiness. Databricks Delta Lake contribution to the Linux Foundation is an important open source storage technology that can help the ecosystem improve reliability for data lakes. We look forward to joining in the Delta Lake project and continuing our collaboration with Databricks and the Apache community.”

– Wei Li, Vice President, Intel Architecture, Graphics and Software and General Manager, Machine Learning Performance

“The Starburst team is excited about the development of Delta Lake and have already developed a native connector for Presto that is currently in beta testing. We believe this will enable companies creating or migrating their data lakes to the cloud the ability to finally realize the value that they were prom ised years ago and perform interactive SQL analytics on data lakes directly.”

– Justin Borgman, CEO, Starburst

“Booz Allen Hamilton is very excited about the potential of Delta Lake technology, especially its promise to provide an open, scalable data platform to enable a broad range of analytics – SQL analytics that powers reporting and dashboarding to data science and machine learning with R & Python. We are looking forward to making significant contributions to the Delta Lake project. We are starting with native integrations of Apache Nifi with Delta Lake.”

– Dan Tucker, VP at Booz Allen Hamilton

For more information please visit Delta.io or follow @DeltaLakeOSS on Twitter.

About The Linux Foundation

Founded in 2000, the Linux Foundation is supported by more than 1,000 members and is the world’s leading home for collaboration on open source software, open standards, open data, and open hardware. Linux Foundation’s projects are critical to the world’s infrastructure including Linux, Kubernetes, Node.js, and more.  The Linux Foundation’s methodology focuses on leveraging best practices and addressing the needs of contributors, users and solution providers to create sustainable models for open collaboration. For more information, please visit us at linuxfoundation.org.

The Linux Foundation has registered trademarks and uses trademarks. For a list of trademarks of The Linux Foundation, please see its trademark usage page: www.linuxfoundation.org/trademark-usage. Linux is a registered trademark of Linus Torvalds.

###

Media Contacts

Beth Handoll
ReTHINKitMedia
beth@rethinkitmedia.com
+1 415 535 8658

The post The Delta Lake Project Turns to Linux Foundation to Become the Open Standard for Data Lakes appeared first on The Linux Foundation.

I am trying to configure a Samba share with correct boolean and security context values using semanage command to make changes in SELinux policy to allow access to Samba share from remote clients on...

Shadow of the Tomb Raider Definitive Edition to arrive on MacOS/Linux  TweakTown

Khronos president Neil Trevett was at this month's XDC2019 conference in Montreal and he clarified their position on accepting conformance submissions by the open-source drivers...

Mesa's TURNIP Vulkan driver that provides open-source Vulkan API support for Qualcomm Adreno hardware in recent weeks has been back to seeing new activity and this week more useful contributions are being made...

Databricks brings its Delta Lake project to the Linux Foundation  TechCrunch

Databricks brings its Delta Lake project to the Linux Foundation  TechCrunch

Every day 2.5 quintillion bytes of big data is created - this data comes from externally sourced websites, blog posts, tweets, sensors of various types and public data initiatives such as the human genome project as well as audio and video recordings from smart devices/apps and the Internet of Things (IoT).

Cultivating a diverse and inclusive workspace is an important part of Red Hat’s open culture. That’s why we work to create an environment where associates feel comfortable bringing their whole selves to work every single day. One way we achieve this mission is by making sure that Red Hatters who wish to share their mental health experiences, are met with compassion and understanding, but most importantly, without stigma. It is estimated that one in four adults suffers from mental illness every year.

Beyond Active Directory: Mac, Linux, SaaS, & IaaS  Security Boulevard

Longtime FUSE developer Miklos Szeredi of Red Hat has been working on a new "FUSE2" FUSE kernel driver for implementing file-systems in user-space...

Shadow of the Tomb Raider Definitive Edition coming to macOS and Linux on 5 November  GamingLyfe Network

Highly Threaded Linux Software Running Under CFS Quotas See Big Performance Fix  Phoronix

Thanks to a Linux kernel fix that is likely to be back-ported to the various stable series, highly threaded software running under CFS quotas for enforcing CPU limits are about to be much faster. At least in a synthetic test case, the kernel fix yields a 30x improvement in performance...

Microsoft To Linux Community: ‘We Are An Open Source Company’  Forbes

A few days after the xf86-video-amdgpu 19.1 release, xf86-video-ati 19.1 is out as the newest X.Org driver release for older ATI/AMD graphics processors...

Linux Gets ‘Shadow Of The Tomb Raider: Definitive Edition’ The Same Day As Windows 10 And Consoles  Forbes

Codasip and Munich University of Applied Science become members

SAN FRANCISCO, Oct. 15, 2019 — CHIPS Alliance, the leading consortium advancing common, open hardware for interfaces, processors and systems, today announced Codasip GmbH and Munich University of Applied Science have joined the CHIPS Alliance. In addition, on November 14–15, CHIPS Alliance will be joining the university for a workshop on open source design verification.

CHIPS Alliance is a project hosted by the Linux Foundation to foster a collaborative environment to accelerate the creation and deployment of open SoCs, peripherals and software tools for use in mobile, computing, consumer electronics, and Internet of Things (IoT) applications. The CHIPS Alliance project develops high-quality open source Register Transfer Level (RTL) code relevant to the design of open source CPUs, RISC-V-based SoCs, and complex peripherals for Field Programmable Gate Arrays (FPGAs) and custom silicon.

Codasip is a leading supplier of configurable RISC-V® embedded processor IP. Codasip provides a portfolio of various RISC-V implementations along with a suite of processor developers tools to allow for rapid core customization, and will contribute to working groups on verification platforms and open cores.

“Codasip has years of processor development experience and has shown its dedication to open platforms by its contributions to open source compiler and compliance projects. We welcome their participation in the CHIPS Alliance to facilitate the adoption of open architectures,” said Zvonimir Bandić, senior director of next-generation platforms architecture at Western Digital and Chairman, CHIPS Alliance.

“Codasip is excited to join the CHIPS Alliance and support the community in its efforts to ease the path of adoption of RISC-V processors in leading-edge SOC applications,” said Karel Masařík, CEO of Codasip. “The CHIPS Alliance is the logical next step in providing chip designers more choices when it comes to processor architectures.”

Munich University of Applied Sciences aims to secure an outstanding position as a university of applied sciences. It recognizes the future demands of society and industry, and is changing with a critical yet open vision for current issues, such as the ongoing digitalization of all areas of life. The university focuses on continuous improvement of quality and on constant development in research, teaching, and continuing education.

“We strongly believe in open source silicon and design flows,” said Stefan Wallentowitz, professor for computer architecture at MUAS. “We look forward to improving open source verification tools together with innovative companies in that field.”

In cooperation with Munich University of Applied Science, the CHIPS Alliance is conducting an open source design verification workshop in Munich. The workshop invites contributions from industry, academia and hobbyists as talks or tutorials. Registration is open now for the November 14–15 event.

About Codasip

Codasip delivers leading-edge processor IP and high-level design tools, providing ASIC designers with all the advantages of the RISC-V open ISA, along with the unique ability to automatically optimize the processor IP. As a founding member of the RISC-V Foundation and a long-term supplier of LLVM and GNU-based processor solutions, Codasip is committed to open standards for embedded processors. Formed in 2006 and headquartered in Munich, Germany, Codasip currently has offices in the US and Europe, with representatives in Asia and Israel. For more information about our products and services, visit www.codasip.com.

About the CHIPS Alliance

The CHIPS Alliance is an organization which develops and hosts high-quality, open source hardware code (IP cores), interconnect IP (physical and logical protocols), and open source software development tools for design, verification, and more. The main aim is to provide a barrier-free collaborative environment, to lower the cost of developing IP and tools for hardware development. The CHIPS Alliance is hosted by the Linux Foundation. For more information, visit chipsalliance.org.

About the Linux Foundation

The Linux Foundation was founded in 2000 and has since become the world’s leading home for collaboration on open source software, open standards, open data, and open hardware. Today, the Foundation is supported by more than 1,000 members and its projects are critical to the world’s infrastructure, including Linux, Kubernetes, Node.js and more. The Linux Foundation focuses on employing best practices and addressing the needs of contributors, users, and solution providers to create sustainable models for open collaboration. For more information, visit linuxfoundation.org.

The post CHIPS Alliance growth continues with new members and design workshop this November appeared first on The Linux Foundation.