Open-source News

Juju and Charmed Operators Accelerating FINOS Open Source Projects Adoption

The Linux Foundation - Wed, 06/08/2022 - 00:03

The article by Srikrishna ‘Kris’ Sharma with Canonical originally appeared in the FINOS Project’s Community Blog. It is another example of enterprises open sourcing their code so that they can “collectively solve common problems so they can separately innovate and differentiate on top of the common baseline.” Read more about Why Do Enterprises Use and Contribute to Open Source Software.

Orchestrating Legend with Juju

Goldman Sachs open sourced the code and contributed its internally developed Legend data management platform into FINOS in October 2020.  Legend provides an end-to-end data platform experience covering the full data lifecycle. It encompasses a suite of data management and governance components known as the Legend Platform. Legend enables breaking down silos and building a critical bridge over the historical divide between business and engineering, allowing companies to build data-driven applications and insightful business intelligence dashboards.

Accelerate FINOS Open Source Project Adoption

Ease and speed of deployment enables innovation and lowers the barrier of entry to open source consumption and contribution. Engineering experience is about leveraging software ops automation to demonstrate impact of an open source project to the community. An awesome engineering experience is more often required to enable wider adoption and contribution to an open source project.

Over the last few months, Canonical has been working closely with FINOS and its community members to offer a consistent way to deploy and manage enterprise applications using Juju and Charmed Operators with a focus on Day 2 operations. The idea is to provide a software ops automation framework and toolkit that enables the DevOps teams at financial institutions to realise the benefits of rapid deployment/ testing and application management using a platform that is 100% open source, vendor-agnostic and hybrid-multi-cloud ready.

What is Juju and Charmed Operator? Charmed Operator:

A charmed operator (also known, more simply, as a “charm”) encapsulates a single application and all the code and know-how it takes to operate it, such as how to combine and work with other related applications or how to upgrade it. Charms are programmed to understand a single application, its operations, and its potential to integrate with other applications. A charm defines and enables the channels by which applications connect. Hundreds of charms are available at charmhub.io.

Juju Operator Lifecycle Manager (OLM) is a hybrid-cloud application management and orchestration system for installation and day 2 operations. It helps deploy, configure, scale, integrate, maintain, and manage Kubernetes native, container-native and VM-native applications—and the relations between them.

Juju allows anyone to deploy and operate charmed operators (charms) in any cloud–including Kubernetes, VMs and Metal. Charms encapsulate the application plus deployment and operations knowledge into one single reusable artefact. Juju manages the lifecycle of applications and infrastructure stacks from cloud to the edge. Juju is cloud-vendor agnostic and hybrid-multi-cloud by nature: it can manage the lifecycle of applications in public clouds, private clouds, or on bare metal. Once bootstrapped, Juju will offer the same deployment and operations experience regardless of the cloud vendor.

The Legend Charm Bundle

In the spirit of providing an enterprise-grade automated deployment and maintenance experience to FINOS members, Canonical created a charmed bundle for Legend and contributed it to FINOS.

The Legend Charm Bundle provides a simple, efficient and enterprise-ready way to deploy and orchestrate a Legend instance in various environments across the CI/CD pipeline, from developer’s workstation to production environment. The bundle includes several Charmed Operators, one for each Legend component.

Why a Legend Charm Bundle?
  1. A simple way to evaluate Legend
    One can spin up a Legend environment from scratch using one single command juju deploy finos-legend-bundle
  2. An intuitive approach (for banks and other financial institutions) to spin up production environments
  3. Provides orchestration capabilities, not only deployment scripting
  4. Easily plugs into Legend release lifecycle and simplifies Legend FINOS instance maintenance

The Legend charm documentation resides on finos/legend-integration-juju github repository and here is the link to related repositories.multiple components.

Detailed instructions are available for local and cloud installations if you would like to spin up your own Legend instance within a few mins and start using Legend either locally or on AWS EKS.

The post Juju and Charmed Operators Accelerating FINOS Open Source Projects Adoption appeared first on Linux Foundation.

How Garbage Collection works inside a Java Virtual Machine

opensource.com - Tue, 06/07/2022 - 21:54
How Garbage Collection works inside a Java Virtual Machine Jayashree Hutt… Tue, 06/07/2022 - 09:54 Register or Login to like Register or Login to like

Automatic Garbage Collection (GC) is one of the most important features that makes Java so popular. This article explains why GC is essential. It includes automatic and generational GC, how the Java Virtual Machine (JVM) divides heap memory, and finally, how GC works inside the JVM.

Java memory allocation

Java memory is divided into four sections:

  1. Heap: The memory for object instances is allocated in the heap. When the object declaration is made, there won't be any memory allocated in the heap. Instead, a reference is created for that object in the stack.
  2. Stack: This section allocates the memory for methods, local variables, and class instance variables.
  3. Code: Bytecode resides in this section.
  4. Static: Static data and methods are placed in this section.
What is automatic Garbage Collection (GC)?

Automatic GC is a process in which the referenced and unreferenced objects in heap memory are identified, and then unreferenced objects are considered for deletion. The term referenced objects means some part of your program is using those objects. Unreferenced objects are not currently being used by the program.

Programming languages like C and C++ require manual allocation and deallocation of memory. This is automatically handled by GC in Java, although you can trigger GC manually with the system.gc(); call in your code.

The fundamental steps of GC are:

1. Mark used and unused objects

In this step, the used and unused objects are marked separately. This is a time-consuming process, as all objects in memory must be scanned to determine whether they're in use or not.

Image by:

(Jayashree Huttanagoudar, CC BY-SA 4.0)

2. Sweep/Delete objects

There are two variations of sweep and delete.

Simple deletion: Only unreferenced objects are removed. However, the memory allocation for new objects becomes difficult as the free space is scattered across available memory.

Image by:

(Jayashree Huttanagoudar, CC BY-SA 4.0)

Deletion with compaction: Apart from deleting unreferenced objects, referenced objects are compacted. Memory allocation for new objects is relatively easy, and memory allocation performance is improved.

Image by:

(Jayashree Huttanagoudar, CC BY-SA 4.0)

More on Java What is enterprise Java programming? Red Hat build of OpenJDK Java cheat sheet Free online course: Developing cloud-native applications with microservices Fresh Java articles What is generational Garbage Collection (GC), and why is it needed?

As seen in the sweep and delete model, scanning all objects for memory reclamation from unused objects becomes difficult once the objects keep growing. An experimental study shows that most objects created during the program execution are short-lived.

The existence of short-lived objects can be used to improve the performance of GC. For that, the JVM divides the memory into different generations. Next, it categorizes the objects based on these memory generations and performs the GC accordingly. This approach is known as generational GC.

Heap memory generations and the generational Garbage Collection (GC) process

To improve the performance of the GC mark and sweep steps, the JVM divides the heap memory into three generations:

  • Young Generation
  • Old Generation
  • Permanent Generation
Image by:

(Jayashree Huttanagoudar, CC BY-SA 4.0)

Here is a description of each generation and its key features.

Young Generation

All created objects are present here. The young generation is further divided into:

  1. Eden: All newly created objects are allocated with the memory here.
  2. Survivor space (S0 and S1): After surviving one GC, the live objects are moved to one of these survivor spaces.
Image by:

(Jayashree Huttanagoudar, CC BY-SA 4.0)

The generational GC that happens in the Young Generation is known as Minor GC. All Minor GC cycles are "Stop the World" events that cause the other applications to pause until it completes the GC cycle. This is why Minor GC cycles are faster.

To summarize: Eden space has all newly created objects. Once Eden is full, the first Minor GC cycle is triggered.

Image by:

(Jayashree Huttanagoudar, CC BY-SA 4.0)

Minor GC: The live and dead objects are marked during this cycle. The live objects are moved to survivor space S0. Once all live objects are moved to S0, the unreferenced objects are deleted.

Image by:

(Jayashree Huttanagoudar, CC BY-SA 4.0)

The age of objects in S0 is 1 because they have survived one Minor GC. Now Eden and S1 are empty.

Once cleared, the Eden space is again filled with new live objects. As time elapses, some objects in Eden and S0 become dead (unreferenced), and Eden's space is full again, triggering the Minor GC.

Image by:

(Jayashree Huttanagoudar, CC BY-SA 4.0)

This time the dead and live objects in Eden and S0 are marked. The live objects from Eden are moved to S1 with an age increment of 1. The live objects from S0 are also moved to S1 with an age increment of 2 (because they've now survived two Minor GCs). At this point, S0 and Eden are empty. After every Minor GC, Eden and one of the survivor spaces are empty.

The same cycle of creating new objects in Eden continues. When the next Minor GC occurs, Eden and S1 are cleared by moving the aged objects to S0. The survivor spaces switch after every Minor GC.

Image by:

(Jayashree Huttanagoudar, CC BY-SA 4.0)

This process continues until the age of one of the surviving objects reaches a certain threshold, at which point it is moved to the so-called the Old Generation with a process called promotion.

Further, the -Xmn flag sets the Young Generation size.

Old Generation (Tenured Generation)

This generation contains the objects that have survived several Minor GCs and aged to reach an expected threshold.

Image by:

(Jayashree Huttanagoudar, CC BY-SA 4.0)

In the example diagram above, the threshold is 8. The GC in the Old Generation is known as a Major GC. Use the flags -Xms and -Xmx to set the initial and maximum size of the heap memory.

Permanent Generation

The Permanent Generation space stores metadata related to library classes and methods of an application, J2SE, and what's in use by the JVM itself. The JVM populates this data at runtime based on which classes and methods are in use. Once the JVM finds the unused classes, they are unloaded or collected, making space for used classes.

Use the flags -XX:PermGen and -XX:MaxPermGen to set the initial and maximum size of the Permanent Generation.

Metaspace

Metaspace was introduced in Java 8u and replaced PermGen. The advantage of this is automatic resizing, which avoids OutOfMemory errors.

Wrap up

This article discusses the various memory generations of JVM and how they are helpful for automatic generational Garbage Collection (GC). Understanding how Java handles memory isn't always necessary, but it can help you envision how the JVM deals with your variables and class instances. This understanding allows you to plan and troubleshoot your code and comprehend potential limitations inherent in a specific platform.

Understanding how Java handles memory isn't always necessary, but it can help you envision how the JVM deals with your variables and class instances.

Image by:

Pixabay. CC0.

Java What to read next Hard lessons learned about Kubernetes garbage collection This work is licensed under a Creative Commons Attribution-Share Alike 4.0 International License. Register or Login to post a comment.

Red Hat Enterprise Linux 9.0 Performing Well, Great Benefit To Newer Intel Xeon & AMD EPYC Servers

Phoronix - Tue, 06/07/2022 - 21:00
Last month RHEL 9.0 reached GA as the newest major update to Red Hat Enterprise Linux. Since then I've been trying out RHEL 9.0 on a few servers. To little surprise, especially for latest-generation Intel Xeon Scalable and AMD EPYC servers, RHEL 9.0 is offering significant uplift compared to the existing RHEL8 series. Here are osme Red Hat Enterprise Linux 9.0 benchmarks comparing the performance to RHEL 8.6.

SUSE Linux Enterprise 15 SP4 Released - Switches To NVIDIA's Open Kernel Driver, Adds AMD SEV-ES

Phoronix - Tue, 06/07/2022 - 20:00
In kicking off SUSECON, SUSE has announced the release of SUSE Linux Enterprise 15 Service Pack 4. Notable with SLE 15 SP4, SUSE is already switching to using NVIDIA's open-source GPU kernel-mode driver that NVIDIA open-sourced last month and is under active development and has a long road ahead before it's even feasible for upstreaming into the mainline kernel...

Intel Begins Adding Meteor Lake Support To Coreboot

Phoronix - Tue, 06/07/2022 - 18:05
Intel Meteor Lake as the successor to Raptor Lake has begun seeing Coreboot enablement work by Intel engineers...

Arm Looking To Make It Easier To Run Docker On AArch64 Linux

Phoronix - Tue, 06/07/2022 - 17:54
While Docker can already run on AArch64 Linux, the mainline Linux kernel's default configuration "defconfig" lacks a few features for allowing it to run out-of-the-box. An Arm engineer is proposing adjusting those defaults to make it more easily/straight-forward to run Docker on 64-bit Arm...

SDL Adds A DirectX 12 Renderer Backend

Phoronix - Tue, 06/07/2022 - 17:02
Somewhat surprisingly, the open-source SDL library that is widely used by games for cross-platform handling and abstracting of various software interfaces has added a Microsoft DirectX 12 renderer...

Ikey Doherty Returning To Work Full-Time On Serpent OS Linux Distribution

Phoronix - Tue, 06/07/2022 - 15:00
Ikey Doherty started the Solus Linux distribution then disappeared and wrote an open letter on Phoronix about his absence, also was employed by Intel for a year to work on their Clear Linux distribution, had a brief stint starting a game/software development company, and then started the Serpent OS Linux distribution. He's still been working on Serpent OS but then was working full-time elsewhere but now has decided to return full-time to working on his latest Linux distribution...

Pages