Open-source News

5 new improvements in Apache ShardingSphere Yacine Si Tayeb, PhD Tue, 10/04/2022 - 03:00

Apache ShardingSphere, a powerful distributed database, recently released a major update to optimize and enhance its features, performance, testing, documentation, and examples. In short, the project continues to work hard at development to make it easier for you to manage your organization's data.

1. SQL audit for data sharding

The problem: When a user executes an SQL query without the sharding feature in large-scale data sharding scenarios, the SQL query is routed to the underlying database for execution. As a result, many database connections are occupied, and businesses are severely affected by a timeout or other issues. Worse still, should the user perform an UPDATE/DELETE operation, a large amount of data may be incorrectly updated or deleted.

ShardingSphere's solution: As of version 5.2.0, ShardingSphere provides the SQL audit for data sharding feature and allows users to configure audit strategies. The strategy specifies multiple audit algorithms, and users can decide whether audit rules should be disabled. SQL execution is strictly prohibited if any audit algorithm fails to pass.

Here's the configuration of an SQL audit for data sharding:

rules:
- !SHARDING
  tables:
    t_order:
      actualDataNodes: ds_${0..1}.t_order_${0..1}
      tableStrategy:
        standard:
          shardingColumn: order_id
          shardingAlgorithmName: t_order_inline
      auditStrategy:
        auditorNames:
          - sharding_key_required_auditor
        allowHintDisable: true
  defaultAuditStrategy:
    auditorNames:
      - sharding_key_required_auditor
    allowHintDisable: true  auditors:
    sharding_key_required_auditor:
      type: DML_SHARDING_CONDITIONS

Given complex business scenarios, this new feature allows you to dynamically disable the audit algorithm by using SQL hints so that partial business SQL operations can be executed.

ShardingSphere has a built-in DML disables full-route audit algorithm. You can also implement a ShardingAuditAlgorithm interface to gain advanced SQL audit functions:

/* ShardingSphere hint: disableAuditNames=sharding_key_required_auditor */ SELECT * FROM t_order;2. SQL execution process management

The ShardingSphere MySQL database provides a SHOW PROCESSLIST statement, allowing you to view the currently running thread. You can kill the thread with the KILL statement for SQL that takes too long to be temporarily terminated.

Image by:

(Duan Zhengqiang, CC BY-SA 4.0)

The SHOW PROCESSLIST and KILL statements are widely used in daily operation and maintenance management. To enhance your ability to manage ShardingSphere, version 5.2.0 supports the MySQL SHOW PROCESSLIST and KILL statements. When you execute a DDL/DML statement through ShardingSphere, ShardingSphere automatically generates a unique UUID identifier and stores the SQL execution information in each instance.

When you execute the SHOW PROCESSLIST statement, ShardingSphere processes the SQL execution information based on the current operating mode.

If the current mode is cluster mode, ShardingSphere collects and synchronizes the SQL execution information of each compute node through the governance center and then returns the summary to the user. If the current mode is the standalone mode, ShardingSphere only returns SQL execution information in the current compute node.

You get to determine whether to execute the KILL statement based on the result returned by SHOW PROCESSLIST, and ShardingSphere cancels the SQL in execution based on the ID in the KILL statement.

More on data science What is data science? What is Python? How to become a data scientist Data scientist: A day in the life What is big data? Whitepaper: Data-intensive intelligent applications in a hybrid cloud blueprint MariaDB and MySQL cheat sheet Latest data science articles 3. Shardingsphere-on-cloud

Shardingsphere-on-cloud is a project of Apache ShardingSphere providing cloud-oriented solutions. Version 0.1.0 has been released, and it has been officially voted as a sub-project of Apache ShardingSphere.

Shardinsphere-on-cloud will continue releasing configuration templates, deployment scripts, and other automation tools for ShardingSphere on the cloud.

It will also polish the engineering practices in terms of high availability, data migration, observability, shadow DB, security, and audit, optimize the delivery mode of Helm Charts, and continue to enhance its cloud-native management capabilities through Kubernetes Operator. There are already introductory issues in the project repository to help those interested in getting Go, Database, and Cloud up and running quickly.

4. Access port

In version 5.2.0, ShardingSphere-Proxy can monitor specified IP addresses and integrate openGauss database drivers by default. ShardingSphere-JDBC supports c3p0 data sources, and a Connection.prepareStatement can specify the columns.

5. Distributed transaction

The original logical database-level transaction manager has been adjusted to a global manager, supporting distributed transactions across multiple logical databases. XA transactions are now automatically managed by ShardingSphere, which removes the XA statement's ability to control distributed transactions.

Use ShardingSphere for distributed data

ShardingSphere supports multiple databases, and each improvement takes more work off your hands. The establishment of the shardingsphere-on-cloud sub-project shows ShardingSphere's commitment to being cloud-native. The greater ShardingSphere community welcomes anyone interested in Go, databases, and the cloud to join the shardingsphere-on-cloud sub-project!

The article was first published on Medium.com and has been republished with permission.

ShardingSphere supports multiple databases, and each improvement in the 5.2.0 release takes more work off your hands.

Image by:

Opensource.com

Databases What to read next This work is licensed under a Creative Commons Attribution-Share Alike 4.0 International License. Register or Login to post a comment.

As a follow-up to the Rust infrastructure pull request for Linux 6.1, Linus Torvalds pulled the initial Rust code into the mainline Linux kernel this evening...

Java News Roundup: BellSoft Introduces Alpaquita Linux, GlassFish, WildFly, Hibernate, Tomcat  InfoQ.com

Lilbits: More Pixel 7 leaks, DIY magnetic charger for the Framework Laptop, and Linux 6.0  Liliputing

Obliteration is a brand new PS4 Emulator for PC (Windows/Linux)  DSOGaming

Debian Makes Major Change, Will Include Non-Free Firmware in Future Installers  MUO - MakeUseOf

AMD P-State Fixes, Fixing s2idle For Some Rembrandt Laptops Lead ACPI/PM For Linux 6.1  Phoronix

The ACPI, power management, and thermal subsystem pull requests have been sent in for the in-development Linux 6.1 kernel...

Git 2.38 was released on Sunday and most notable with this feature release is the inclusion of scalar as a Microsoft-developed repository management tool to make it easier dealing with very large Git reposotories...

How to Check Shutdown and Restart History on Linux  MUO - MakeUseOf

How to get started deploying containers with Podman  TechRepublic

How to enable systemd in Windows Subsystem for Linux (WSL)  XDA Developers

A kernel hardening security improvement on the way for Linux 6.1 is the ability to provide warning of possible memcpy() based overflows. Right now this is only a warning but it's work towards being able to address "trivially detectable" buffer overflow conditions within the kernel and in the future may be able to block such overflows from happening...

The top Linux courses on LinkedIn Learning  TechRepublic

Big changes coming in Debian 12: Some parts won't be FOSS  The Register

In this article we will try to teach you how to Clone Linux System Drive to a Larger SSD. Linux is an open-source operating system like other operating  BollyInside

Ryzen 7000 Runs Faster With Security Mitigations Enabled on Linux  Tom's Hardware

AMD Rembrandt CPUFreq vs. AMD P-State Linux Testing Review  Phoronix

Before getting busy with the AMD Ryzen 7000 "Zen 4" desktop testing, I recently wrapped up some benchmarks looking at the ACPI CPUFreq vs. AMD P-State frequency scaling drivers and various governor options for the AMD Ryzen 6000 "Rembrandt" mobile SoCs. If you are curious about the impact of CPUFreq/P-State and the various governors for the latest AMD laptops running Linux, this round of testing is for you.

IceWM 3.0 was released today as the latest major update to this X11 window manager that has been development since the late 90's with a particular focus on simplicity and speed...