Open-source News

The post How to Integrate ONLYOFFICE in WordPress for Document Editing first appeared on Tecmint: Linux Howtos, Tutorials & Guides .

It’s no secret that WordPress is one of the most popular content management systems for websites and blogs all over the Internet. In fact, 43% of the web is built on the WordPress platform.

The post How to Integrate ONLYOFFICE in WordPress for Document Editing first appeared on Tecmint: Linux Howtos, Tutorials & Guides.

Resurrecting a "dead" "live" USB - Linux & Unix  BleepingComputer

This post originally appeared on the Delta Lake blog. 

We are happy to announce the release of the Delta Lake 2.0 (pypi, maven, release notes) on Apache Spark 3.2, with the following features including but not limited to:

• Support for change data feed,
• Z-Order clustering of data to reduce the amount of data to read,
• Support for idempotent writes to Delta tables,
• Support for dropping columns,
• Support for dynamic partition overwrite, and
• Experimental support for multi-part checkpoints.

The significance of Delta Lake 2.0 is not just a number – though it is timed quite nicely with Delta Lake’s 3rd birthday. It reiterates our collective commitment to the open-sourcing of Delta Lake, as announced by Michael Armbrust’s Day 1 keynote at Data + AI Summit 2022.

Michael Armbrust’s Day 1 keynote during Data + AI Summit 2022

What’s new in Delta Lake 2.0?

There have been a lot of new features released in the last year between Delta Lake 1.0, 1.2, and now 2.0. This blog will review a few of these specific features that are going to have a large impact on your workload.

Improving data skipping

When exploring or slicing data using dashboards, data practitioners will often run queries with a specific filter in place. As a result, the matching data is often buried in a large table, requiring Delta Lake to read a significant amount of data. With data skipping via column statistics and Z-Order, the data can be clustered by the most common filters used in queries — sorting the table to skip irrelevant data, which can dramatically increase query performance.

Support for data skipping via column statistics

When querying any table from HDFS or cloud object storage, by default, your query engine will scan all of the files that make up your table. This can be inefficient, especially if you only need a smaller subset of data. To improve this process, as part of the Delta Lake 1.2 release, we included support for data skipping by utilizing the Delta table’s column statistics.

For example, when running the following query, you do not want to unnecessarily read files outside of the year or uid ranges.

When Delta Lake writes a table, it will automatically collect the minimum and maximum values and store this directly into the Delta log (i.e. column statistics). Therefore, when a query engine reads the transaction log, those read queries can skip files outside the range of the min/max values as visualized below.

This approach is more efficient than row-group filtering within the Parquet file itself, as you do not need to read the Parquet footer. For more information on the latter process, please refer to How Apache Spark performs a fast count using the parquet metadata. For more information on data skipping, please refer to data skipping.

Support Z-Order clustering of data to reduce the amount of data read

But data skipping using column statistics is only one part of the solution. To maximize data skipping, what is also needed is the ability to skip with data clustering. As implied previously, data skipping is most effective when files have a very small minimum/maximum range. While sorting the data can help, this is most effective when applied to a single column.

Regular sorting of data by primary and secondary columns (left) and 2-dimensional Z-order data clustering for two columns (right).

But with ​​Z-order, its space-filling curve provides better multi-column data clustering. This data clustering allows column stats to be more effective in skipping data based on filters in a query. See the documentation and this blog for more details.

Support Change Data Feed on Delta tables

One of the biggest value propositions of Delta Lake is its ability to maintain data reliability in the face of changing records brought on by data streams. However, this requires scanning and reading the entire table, creating significant overhead that can slow performance.

With Change Data Feed (CDF), you can now read a Delta table’s change feed at the row level rather than the entire table to capture and manage changes for up-to-date silver and gold tables. This improves your data pipeline performance and simplifies its operations.

To enable CDF, you must explicitly use one of the following methods:

• New table: Set the table property delta.enableChangeDataFeed = true in the CREATE TABLE command.

  CREATE TABLE student (id INT, name STRING, age INT) TBLPROPERTIES (delta.enableChangeDataFeed = true)

• Existing table: Set the table property delta.enableChangeDataFeed = true in the ALTER TABLE command.

  ALTER TABLE myDeltaTable SET TBLPROPERTIES (delta.enableChangeDataFeed = true)

• All new tables:

  set spark.databricks.delta.properties.defaults.enableChangeDataFeed = true;

An important thing to remember is once you enable the change data feed option for a table, you can no longer write to the table using Delta Lake 1.2.1 or below. However, you can always read the table. In addition, only changes made after you enable the change data feed are recorded; past changes to a table are not captured.

So when should you enable Change Data Feed? The following use cases should drive when you enable the change data feed.

• Silver and Gold tables: When you want to improve Delta Lake performance by streaming row-level changes for up-to-date silver and gold tables. This is especially apparent when following MERGE, UPDATE, or DELETE operations accelerating and simplifying ETL operations.
• Transmit changes: Send a change data feed to downstream systems such as Kafka or RDBMS that can use the feed to process later stages of data pipelines incrementally.
• Audit trail table: Capture the change data feed as a Delta table provides perpetual storage and efficient query capability to see all changes over time, including when deletes occur and what updates were made.

See the documentation for more details.

Support for dropping columns

For versions of Delta Lake prior to 1.2, there was a requirement for Parquet files to store data with the same column name as the table schema. Delta Lake 1.2 introduced a mapping between the logical column name and the physical column name in those Parquet files. While the physical names remain unique, the logical column renames become a simple change in the mapping and logical column names can have arbitrary characters while the physical name remains Parquet-compliant.

As part of the Delta Lake 2.0 release, we leveraged column mapping so that dropping a column is a metadata operation. Therefore, instead of physically modifying all of the files of the underlying table to drop a column, this can be a simple modification to the Delta transaction log (i.e. a metadata operation) to reflect the column removal. Run the following SQL command to drop a column:

ALTER TABLE myDeltaTable DROP COLUMN myColumn

See documentation for more details.

Support for Dynamic Partition Overwrites

In addition, Delta Lake 2.0 now supports Delta dynamic partition overwrite mode for partitioned tables; that is, overwrite only the partitions with data written into them at runtime.

When in dynamic partition overwrite mode, we overwrite all existing data in each logical partition for which the write will commit new data. Any existing logical partitions for which the write does not contain data will remain unchanged. This mode is only applicable when data is being written in overwrite mode: either INSERT OVERWRITE in SQL, or a DataFrame write with df.write.mode("overwrite"). In SQL, you can run the following commands:

SET spark.sql.sources.partitionOverwriteMode=dynamic; INSERT OVERWRITE TABLE default.people10m SELECT * FROM morePeople;

Note, dynamic partition overwrite conflicts with the option replaceWhere for partitioned tables. For more information, see the documentation for details.

Additional Features in Delta Lake 2.0

In the spirit of performance optimizations, Delta Lake 2.0.0 also includes these additional features:

• Support for idempotent writes to Delta tables to enable fault-tolerant retry of Delta table writing jobs without writing the data multiple times to the table. See the documentation for more details.
• Experimental support for multi-part checkpoints to split the Delta Lake checkpoint into multiple parts to speed up writing the checkpoints and reading. See documentation for more details.
• Other notable changes
  • Improve the generated column data skipping by adding the support for skipping by nested column generated column
  • Improve the table schema validation by blocking the unsupported data types in Delta Lake.
  • Support creating a Delta Lake table with an empty schema.
  • Change the behavior of DROP CONSTRAINT to throw an error when the constraint does not exist. Before this version, the command used to return silently.
  • Fix the symlink manifest generation when partition values contain space in them.
  • Fix an issue where incorrect commit stats are collected.
  • More ways to access the Delta table OPTIMIZE file compaction command.
    • Python OPTIMIZE APIs (commit), API documentation.
    • Scala OPTIMIZE APIs (commit), API documentation.

Building a Robust Data Ecosystem

As noted in Michael Armbrust’s Day 1 keynote and our Dive into Delta Lake 2.0 session, a fundamental aspect of Delta Lake is the robustness of its data ecosystem.

As data volume and variety continue to rise, the need to integrate with the most common ingestion engines is critical. For example, we’ve recently announced integrations with Apache Flink, Presto, and Trino — allowing you to read and write to Delta Lake directly from these popular engines. Check out Delta Lake > Integrations for the latest integrations.

Delta Lake will be relied on even more to bring reliability and improved performance to data lakes by providing ACID transactions and unifying streaming and batch transactions on top of existing cloud data stores. By building connectors with the most popular compute engines and technologies, the appeal of Delta Lake will continue to increase — driving more growth in the community and rapid adoption of the technology across the most innovative and largest enterprises in the world.

Updates on Community Expansion and Growth

We are proud of the community and the tremendous work over the years to deliver the most reliable, scalable, and performant table storage format for the lakehouse to ensure consistent high-quality data. None of this would be possible without the contributions from the open-source community. In the span of a year, we have seen the number of downloads skyrocket from 685K monthly downloads to over 7M downloads/month. As noted in the following figure, this growth is in no small part due to the quickly expanding Delta ecosystem.

All of this activity and the growth in unique contributions — including commits, PRs, changesets, and bug fixes — has culminated in an increase in contributor strength by 633% during the last three years (Source: The Linux Foundation Insights).

But it is important to remember that we could not have done this without the contributions of the community.

Credits

Saying this, we wanted to provide a quick shout-out to all of those involved with the release of Delta Lake 2.0: Adam Binford, Alkis Evlogimenos, Allison Portis, Ankur Dave, Bingkun Pan, Burak Yilmaz, Chang Yong Lik, Chen Qingzhi, Denny Lee, Eric Chang, Felipe Pessoto, Fred Liu, Fu Chen, Gaurav Rupnar, Grzegorz Kołakowski, Hussein Nagree, Jacek Laskowski, Jackie Zhang, Jiaan Geng, Jintao Shen, Jintian Liang, John O’Dwyer, Junyong Lee, Kam Cheung Ting, Karen Feng, Koert Kuipers, Lars Kroll, Liwen Sun, Lukas Rupprecht, Max Gekk, Michael Mengarelli, Min Yang, Naga Raju Bhanoori, Nick Grigoriev, Nick Karpov, Ole Sasse, Patrick Grandjean, Peng Zhong, Prakhar Jain, Rahul Shivu Mahadev, Rajesh Parangi, Ruslan Dautkhanov, Sabir Akhadov, Scott Sandre, Serge Rielau, Shixiong Zhu, Shoumik Palkar, Tathagata Das, Terry Kim, Tyson Condie, Venki Korukanti, Vini Jaiswal, Wenchen Fan, Xinyi, Yijia Cui, Yousry Mohamed.

We’d also like to thank Nick Karpov and Scott Sandre for their help with this post.

How can you help?

We’re always excited to work with current and new community members. If you’re interested in helping the Delta Lake project, please join our community today through many forums, including GitHub, Slack, Twitter, LinkedIn, YouTube, and Google Groups.

The post Delta 2.0 – The Foundation of your Data Lakehouse is Open appeared first on Linux Foundation.

Kali Linux 2022.3 adds 5 new tools, updates Linux kernel, and more  BleepingComputer

Kali Linux 2022.3 adds 5 new tools, updates Linux kernel, more  BleepingComputer

Two Point Campus from the dev of Two Point Hospital out now  GamingOnLinux

In addition to Intel's busy Patch Tuesday, AMD today made public CVE-2021-46778 that university researchers have dubbed the "SQUIP" attack as a side channel vulnerability affecting the execution unit scheduler across Zen 1/2/3 processors...

How to upgrade to Linux Mint 21 using the Upgrade Tool  Ghacks

Today's busy patch Tuesday for Intel continues with the Linux kernel getting mitigated for EIBRS Post-barrier Return Stack Buffer (PBRSB). This PBRSB is the latest handling on the "CPU vulnerability nightmares front", the pull request calls it...

Kali Linux 2022.3 released: Packages for test labs, new tools, and a community Discord server  Help Net Security

6 Important Factors to Consider When Choosing a Linux Distro  MUO - MakeUseOf

Linux Kernel Patched For "PBRSB" After Intel eIBRS CPUs Found To Be Insufficient  Phoronix

As part of today's "Patch Tuesday", Intel has made a new round of security vulnerabilities public -- including a new processor advisory that affects their latest Xeon Scalable and Core wares resulting in new CPU microcode being required...

Solve: Legacy trusted.gpg keyring - APT Key 'apt-key' Deprecation on Ubuntu  Linux Shout

System76 Galago Pro 14 inch Linux laptop now available with 12th-gen Intel chips for $999 and up  Liliputing

System76 Galago Pro 14 inch Linux laptop now available with 12th-gen Intel chips for $999 and up  Liliputing

How to Type Less and Work Faster in the Linux Terminal  How-To Geek

How to Install And Configure DaVinci Resolve In Linux  Make Tech Easier

System76 Refreshes Its Affordable Galago Pro Linux Laptop with Alder Lake CPUs  9to5Linux

A guide to JVM interpretation and compilation Jayashree Hutt… Tue, 08/09/2022 - 10:36 Register or Login to like Register or Login to like

Java is a platform-independent language. Programs are converted to bytecode after compilation. This bytecode gets converted to machine code at runtime. An interpreter emulates the execution of bytecode instructions for the abstract machine on a specific physical machine. Just-in-time (JIT) compilation happens at some point during execution, and ahead-of-time (AOT) compilation happens during build time. 

This article explains when an interpreter comes into play and when JIT and AOT will occur. I also discuss the trade-offs between JIT and AOT.

Source code, bytecode, machine code

Applications are generally written using a programming language like C, C++, or Java. The set of instructions written using high-level programming languages is called source code. Source code is human readable. To execute it on the target machine, source code needs to be converted to machine code, which is machine readable. Source code is typically converted into machine code by a compiler. 

In Java, however, the source code is first converted into an intermediate form called bytecode. This bytecode is platform independent, which is why Java is well known as a platform-independent programming language. The primary Java compiler javac converts the Java source code into bytecode. Then, the bytecode is interpreted by the interpreter.

Here is a small Hello.java program:

//Hello.java
public class Hello {

    public static void main(String[] args) {
         System.out.println("Inside Hello World!");
         }
}

Compile it using javac to generate a Hello.class file containing the bytecode. 

$ javac Hello.java
$ ls
Hello.class  Hello.java

Now, use javap to disassemble the content of the Hello.class file. The output of javap depends on the options used. If you don't choose any options, it prints basic information, including which source file this class file is compiled from, the package name, public and protected fields, and methods of the class.

$ javap Hello.class
Compiled from "Hello.java"
public class Hello {
  public Hello();
  public static void main(java.lang.String[]);
}

To see the bytecode content in the .class file, use the -c option:

$ javap -c Hello.class
Compiled from "Hello.java"
public class Hello {
  public Hello();
        Code:
           0: aload_0
           1: invokespecial #1                      // Method java/lang/Object."":()V
           4: return

  public static void main(java.lang.String[]);
        Code:
           0: getstatic         #2                      // Field java/lang/System.out:Ljava/io/PrintStream;
           3: ldc               #3                      // String Inside Hello World!
           5: invokevirtual #4                      // Method    
java/io/PrintStream.println:(Ljava/lang/String;)V
           8: return
}

To get more detailed information, use the -v option:

$ javap -v Hello.classInterpreter, JIT, AOT

The interpreter is responsible for emulating the execution of bytecode instructions for the abstract machine on a specific physical machine. When compiling source code using javac and executing using the java command, the interpreter operates during runtime and serves its purpose.

$ javac Hello.java
$ java Hello
Inside Hello World!

The JIT compiler also operates at runtime. When the interpreter interprets a Java program, another component, called a runtime profiler, is silently monitoring the program's execution to observe which portion of the code is getting interpreted and how many times. These statistics help detect the hotspots of the program, that is, those portions of code frequently being interpreted. Once they're interpreted above a set threshold, they are eligible to be converted into machine code directly by the JIT compiler. The JIT compiler is also known as a profile-guided compiler. Conversion of bytecode to native code happens on the fly, hence the name just-in-time. JIT reduces overhead of the interpreter emulating the same set of instructions to machine code.

The AOT compiler compiles code during build time. Generating frequently interpreted and JIT-compiled code at build time improves the warm-up time of the Java Virtual Machine (JVM). This compiler was introduced in Java 9 as an experimental feature. The jaotc tool uses the Graal compiler, which is itself written in Java, for AOT compilation. 

Here's a sample use case for a Hello program:

//Hello.java
public class Hello {


    public static void main(String[] args) {
            System.out.println("Inside Hello World!");
            }
}


$ javac Hello.java
$ jaotc --output libHello.so Hello.class
$ java -XX:+UnlockExperimentalVMOptions -XX:AOTLibrary=./libHello.so Hello
Inside Hello World!When do interpreting and compiling come into play: an example

This example illustrates when Java uses an interpreter and when JIT and AOT pitch in. Consider a simple Java program, Demo.java:

//Demo.java
public class Demo {
  public int square(int i) throws Exception {
        return(i*i);
  }


  public static void main(String[] args) throws Exception {
        for (int i = 1; i <= 10; i++) {
          System.out.println("call " + Integer.valueOf(i));
          long a = System.nanoTime();
          Int r = new Demo().square(i);
        System.out.println("Square(i) = " + r);
          long b = System.nanoTime();
          System.out.println("elapsed= " + (b-a));
          System.out.println("--------------------------------");
        }
  }
}

This simple program has a main method that creates a Demo object instance, and calls the method square, which displays the square root of the for loop iteration value. Now, compile and run the code:

$ javac Demo.java
$ java Demo
1 iteration
Square(i) = 1
Time taken= 8432439
--------------------------------
2 iteration
Square(i) = 4
Time taken= 54631
--------------------------------
.
.
.
--------------------------------
10 iteration
Square(i) = 100
Time taken= 66498
--------------------------------

More on Java What is enterprise Java programming? Red Hat build of OpenJDK Java cheat sheet Free online course: Developing cloud-native applications with microservices Fresh Java articles

The question now is whether the output is a result of the interpreter, JIT, or AOT. In this case, it's wholly interpreted. How did I conclude that? Well, to get JIT to contribute to the compilation, the hotspots of the code must be interpreted above a defined threshold. Then and only then are those pieces of code queued for JIT compilation. To find the threshold for JDK 11:

$ java -XX:+PrintFlagsFinal -version | grep CompileThreshold
 intx CompileThreshold     = 10000                                      {pd product} {default}
[...]
openjdk version "11.0.13" 2021-10-19
OpenJDK Runtime Environment 18.9 (build 11.0.13+8)
OpenJDK 64-Bit Server VM 18.9 (build 11.0.13+8, mixed mode, sharing)

The above output demonstrates that a particular piece of code should be interpreted 10,000 times to be eligible for JIT compilation. Can this threshold be manually tuned, and is there some JVM flag that indicates whether a method is JIT compiled? Yes, there are multiple options to serve this purpose. 

One option for learning whether a method is JIT compiled is -XX:+PrintCompilation. Along with this option, the flag -Xbatch provides the output in a more readable way. If both interpretation and JIT are happening in parallel, the -Xbatch flag helps distinguish the output of both. Use these flags as follows:

$ java -Xbatch  -XX:+PrintCompilation  Demo
         34        1        b  3           java.util.concurrent.ConcurrentHashMap::tabAt (22 bytes)
         35        2         n 0           jdk.internal.misc.Unsafe::getObjectVolatile (native)   
         35        3        b  3           java.lang.Object::<init> (1 bytes)
[...]
210  269         n 0           java.lang.reflect.Array::newArray (native)   (static)
        211  270        b  3           java.lang.String::substring (58 bytes)
[...]
--------------------------------
10 iteration
Square(i) = 100
Time taken= 50150
-------------------------------- 

The output of the above command is too lengthy, so I've truncated the middle portion. Note that along with the Demo program code, the JDKs internal class functions are also getting compiled. This is why the output is so lengthy. Because my focus is Demo.java code, I'll use an option that can minimize the output by excluding the internal package functions. The command -XX:CompileCommandFile disables JIT for internal classes:

$ java -Xbatch -XX:+PrintCompilation -XX:CompileCommandFile=hotspot_compiler Demo

The file hotspot_compiler referenced by -XX:CompileCommandFile contains this code to exclude specific packages:

$ cat hotspot_compiler
quiet
exclude java/* *
exclude jdk/* *
exclude sun/* *

In the first line, quiet instructs the JVM not to write anything about excluded classes. To tune the JIT threshold, use -XX:CompileThreshold with the value set to 5, meaning that after interpreting five times, it's time for JIT:

$ java -Xbatch -XX:+PrintCompilation -XX:CompileCommandFile=hotspot_compiler \
-XX:CompileThreshold=5 Demo
        47      1       n 0     java.lang.invoke.MethodHandle::linkToStatic(LLLLLL)L (native)  
           (static)
        47      2       n 0     java.lang.invoke.MethodHandle::invokeBasic(LLLLL)L (native)  
        47      3       n 0     java.lang.invoke.MethodHandle::linkToSpecial(LLLLLLL)L (native)  
           (static)
        48      4       n 0     java.lang.invoke.MethodHandle::linkToStatic(L)I (native)   (static)
        48      5       n 0     java.lang.invoke.MethodHandle::invokeBasic()I (native)  
        48      6       n 0     java.lang.invoke.MethodHandle::linkToSpecial(LL)I (native)  
           (static)
[...]
        1 iteration
        69   40         n 0     java.lang.invoke.MethodHandle::linkToStatic(ILIIL)I (native)  
           (static)
[...]
Square(i) = 1
        78   48         n 0     java.lang.invoke.MethodHandle::linkToStatic(ILIJL)I (native)  
(static)
        79   49         n 0     java.lang.invoke.MethodHandle::invokeBasic(ILIJ)I (native)  
[...]
        86   54         n 0     java.lang.invoke.MethodHandle::invokeBasic(J)L (native)  
        87   55         n 0     java.lang.invoke.MethodHandle::linkToSpecial(LJL)L (native)  
(static)
Time taken= 8962738
--------------------------------
2 iteration
Square(i) = 4
Time taken= 26759
--------------------------------

10 iteration
Square(i) = 100
Time taken= 26492
--------------------------------

The output is still not different from interpreted output! This is because, as per Oracle's documentation, the -XX:CompileThreshold flag is effective only when TieredCompilation is disabled:

$ java -Xbatch -XX:+PrintCompilation -XX:CompileCommandFile=hotspot_compiler \
-XX:-TieredCompilation -XX:CompileThreshold=5 Demo
124     1       n       java.lang.invoke.MethodHandle::linkToStatic(LLLLLL)L (native)   (static)
127     2       n       java.lang.invoke.MethodHandle::invokeBasic(LLLLL)L (native)  
[...]
1 iteration
        187   40        n       java.lang.invoke.MethodHandle::linkToStatic(ILIIL)I (native)   (static)
[...]
(native)   (static)
        212   54        n       java.lang.invoke.MethodHandle::invokeBasic(J)L (native)  
        212   55        n       java.lang.invoke.MethodHandle::linkToSpecial(LJL)L (native)   (static)
Time taken= 12337415
[...]
--------------------------------
4 iteration
Square(i) = 16
Time taken= 37183
--------------------------------
5 iteration
        214   56        b       Demo::<init> (5 bytes)
        215   57        b       Demo::square (16 bytes)
Square(i) = 25
Time taken= 983002
--------------------------------
6 iteration
Square(i) = 36
Time taken= 81589
[...]
10 iteration
Square(i) = 100
Time taken= 52393

This section of code is now JIT compiled after the fifth interpretation:

--------------------------------
5 iteration
        214   56        b       Demo::<init> (5 bytes)
        215   57        b       Demo::square (16 bytes)
Square(i) = 25
Time taken= 983002
--------------------------------

Along with the square() method, the constructor is also getting JIT compiled because there is a Demo instance inside the for loop before calling square(). Hence, it will also reach the threshold and be JIT compiled. This example illustrates when JIT comes into play after interpretation. 

To see the compiled version of the code, use the -XX:+PrintAssembly flag, which works only if there is a disassembler in the library path. For OpenJDK, use the hsdis disassembler. Download a suitable disassembler library— in this case, hsdis-amd64.so— and place it under Java_HOME/lib/server. Make sure to use -XX:+UnlockDiagnosticVMOptions before -XX:+PrintAssembly. Otherwise, JVM will give you a warning. 

The entire command is as follows:

$ java -Xbatch -XX:+PrintCompilation -XX:CompileCommandFile=hotspot_compiler \ -XX:-TieredCompilation -XX:CompileThreshold=5 -XX:+UnlockDiagnosticVMOptions \ -XX:+PrintAssembly Demo
[...]
5 iteration
        178   56        b       Demo::<init> (5 bytes)
Compiled method (c2)    178   56                Demo::<init> (5 bytes)
 total in heap  [0x00007fd4d08dad10,0x00007fd4d08dafe0] = 720
 relocation     [0x00007fd4d08dae88,0x00007fd4d08daea0] = 24
[...]
 handler table  [0x00007fd4d08dafc8,0x00007fd4d08dafe0] = 24
[...]
 dependencies   [0x00007fd4d08db3c0,0x00007fd4d08db3c8] = 8
 handler table  [0x00007fd4d08db3c8,0x00007fd4d08db3f8] = 48
----------------------------------------------------------------------
Demo.square(I)I  [0x00007fd4d08db1c0, 0x00007fd4d08db2b8]  248 bytes
[Entry Point]
[Constants]
  # {method} {0x00007fd4b841f4b0} 'square' '(I)I' in 'Demo'
  # this:       rsi:rsi   = 'Demo'
  # parm0:      rdx     = int
  #             [sp+0x20]  (sp of caller)
[...]
[Stub Code]
  0x00007fd4d08db280: movabs $0x0,%rbx          ;   {no_reloc}
  0x00007fd4d08db28a: jmpq   0x00007fd4d08db28a  ;   {runtime_call}
  0x00007fd4d08db28f: movabs $0x0,%rbx          ;   {static_stub}
  0x00007fd4d08db299: jmpq   0x00007fd4d08db299  ;   {runtime_call}
[Exception Handler]
  0x00007fd4d08db29e: jmpq   0x00007fd4d08bb880  ;   {runtime_call ExceptionBlob}
[Deopt Handler Code]
  0x00007fd4d08db2a3: callq  0x00007fd4d08db2a8
  0x00007fd4d08db2a8: subq   $0x5,(%rsp)
  0x00007fd4d08db2ad: jmpq   0x00007fd4d08a01a0  ;   {runtime_call DeoptimizationBlob}
  0x00007fd4d08db2b2: hlt    
  0x00007fd4d08db2b3: hlt    
  0x00007fd4d08db2b4: hlt    
  0x00007fd4d08db2b5: hlt    
  0x00007fd4d08db2b6: hlt    
  0x00007fd4d08db2b7: hlt    
ImmutableOopMap{rbp=NarrowOop }pc offsets: 96
ImmutableOopMap{}pc offsets: 112
ImmutableOopMap{rbp=Oop }pc offsets: 148 Square(i) = 25
Time taken= 2567698
--------------------------------
6 iteration
Square(i) = 36
Time taken= 76752
[...]
--------------------------------
10 iteration
Square(i) = 100
Time taken= 52888

The output is lengthy, so I've included only the output related to Demo.java.

Now it's time for AOT compilation. This option was introduced in JDK9. AOT is a static compiler to generate the .so library. With AOT, the interested classes can be compiled to create an .so library that can be directly executed instead of interpreting or JIT compiling. If JVM doesn't find any AOT-compiled code, the usual interpretation and JIT compilation takes place. 

The command used for AOT compilation is as follows:

$ jaotc --output=libDemo.so Demo.class

To see the symbols in the shared library, use the following:

$ nm libDemo.so

To use the generated .so library, use -XX:AOTLibrary along with -XX:+UnlockExperimentalVMOptions as follows:

$ java -XX:+UnlockExperimentalVMOptions -XX:AOTLibrary=./libDemo.so Demo
1 iteration
Square(i) = 1
Time taken= 7831139
--------------------------------
2 iteration
Square(i) = 4
Time taken= 36619
[...]
10 iteration
Square(i) = 100
Time taken= 42085

This output looks as if it is an interpreted version itself. To make sure that the AOT compiled code is utilized, use -XX:+PrintAOT:

$ java -XX:+UnlockExperimentalVMOptions -XX:AOTLibrary=./libDemo.so -XX:+PrintAOT Demo
         28        1         loaded        ./libDemo.so  aot library
         80        1         aot[ 1]   Demo.main([Ljava/lang/String;)V
         80        2         aot[ 1]   Demo.square(I)I
         80        3         aot[ 1]   Demo.<init>()V
1 iteration
Square(i) = 1
Time taken= 7252921
--------------------------------
2 iteration
Square(i) = 4
Time taken= 57443
[...]
10 iteration
Square(i) = 100
Time taken= 53586

Just to make sure that JIT compilation hasn't happened, use the following:

$ java -XX:+UnlockExperimentalVMOptions -Xbatch -XX:+PrintCompilation \ -XX:CompileCommandFile=hotspot_compiler -XX:-TieredCompilation \ -XX:CompileThreshold=3 -XX:AOTLibrary=./libDemo.so -XX:+PrintAOT Demo
         19        1         loaded        ./libDemo.so  aot library
         77        1         aot[ 1]   Demo.square(I)I
         77        2         aot[ 1]   Demo.main([Ljava/lang/String;)V
         77        3         aot[ 1]   Demo.<init>()V
         77        2         aot[ 1]   Demo.main([Ljava/lang/String;)V   made not entrant
[...]
4 iteration
Square(i) = 16
Time taken= 43366
[...]
10 iteration
Square(i) = 100
Time taken= 59554

If any small change is made to the source code subjected to AOT, it's important to ensure that the corresponding .so is created again. Otherwise, the stale AOT-compiled .so won't have any effect. For example, make a small change to the square function such that now it's calculating cube:

//Demo.java
public class Demo {

  public int square(int i) throws Exception {
        return(i*i*i);
  }

  public static void main(String[] args) throws Exception {
        for (int i = 1; i <= 10; i++) {
          System.out.println("" + Integer.valueOf(i)+" iteration");
          long start = System.nanoTime();
          int r= new Demo().square(i);
          System.out.println("Square(i) = " + r);
          long end = System.nanoTime();
          System.out.println("Time taken= " + (end-start));
          System.out.println("--------------------------------");
        }
  }
}

Now, compile Demo.java again:

$ java Demo.java

But, don't create libDemo.so using jaotc. Instead, use this command:

$ java -XX:+UnlockExperimentalVMOptions -Xbatch -XX:+PrintCompilation -XX:CompileCommandFile=hotspot_compiler -XX:-TieredCompilation -XX:CompileThreshold=3 -XX:AOTLibrary=./libDemo.so -XX:+PrintAOT Demo
         20        1         loaded        ./libDemo.so  aot library
         74        1         n           java.lang.invoke.MethodHandle::linkToStatic(LLLLLL)L (native)   (static)
2 iteration
sqrt(i) = 8
Time taken= 43838
--------------------------------
3 iteration
        137   56        b            Demo::<init> (5 bytes)
        138   57        b            Demo::square (6 bytes)
sqrt(i) = 27
Time taken= 534649
--------------------------------
4 iteration
sqrt(i) = 64
Time taken= 51916
[...]
10 iteration
sqrt(i) = 1000
Time taken= 47132

Though the old version of libDemo.so is loaded, JVM detected it as a stale one. Every time a .class file is created, a fingerprint goes into the class file, and a class fingerprint is kept in the AOT library. Because the class fingerprint is different from the one in the AOT library, AOT-compiled native code is not used. Instead, the method is now JIT compiled, because the -XX:CompileThreshold is set to 3.

AOT or JIT?

If you are aiming to reduce the warm-up time of the JVM, use AOT, which reduces the burden during runtime. The catch is that AOT will not have enough data to decide which piece of code needs to be precompiled to native code.  By contrast, JIT pitches in during runtime and impacts the warm-up time. However, it will have enough profiling data to compile and decompile the code more efficiently.

Use interpretation, just-in-time compilation, and ahead-of-time compilation efficiently by understanding the differences among them.

Image by:

Image by WOCinTech Chat, CC BY 2.0

Java What to read next This work is licensed under a Creative Commons Attribution-Share Alike 4.0 International License. Register or Login to post a comment.