Open-source News

Best Python IDEs for Ubuntu and Other Linux Distros  It's FOSS

Following AMDVLK 2022.Q4.1 from late October, AMDVLK 2022.Q4.2 is now available as AMD's latest official open-source Vulkan Linux driver update for both gamers and enterprise customers...

Sent out today was this week's batch of "drm-misc-next" code containing Direct Rendering Manager updates to the core infrastructure and smaller drivers of material that is ready for queuing ahead of the Linux 6.2 cycle...

Linux 6.2 To Introduce Allwinner A100 & D1 Display Support  Phoronix

Another point release to the ROCm 5.3 series for AMD's open-source compute stack is now available...

Radeon ROCm 5.3.2 Released With A Few Fixes  Phoronix

While the release next month of Blender 3.4 is planning to ship with Wayland enabled, Fedora Linux 37 users are expected to soon find their packaged Blender versions already running with the Wayland support enabled...

Fedora Workstation Bringing Early Support For Blender On Wayland  Phoronix

Linux Operating System Market Is Thriving Worldwide By 2026- Report Spread across 101 Pages  Digital Journal

AMD Ryzen Lenovo Laptop Linux Performance For Zen 2 / Zen 3 ...  Phoronix

8 Best Screen Recorders For Linux  Beebom

4 key differences between Twitter and Mastodon Don Watkins Thu, 11/10/2022 - 03:00

Social media is not always sociable, and sometimes we need a sufficient impetus to change what we do and what we read. I began using Twitter as a replacement for my RSS reader in 2008, which revolutionized how I read and learned up to that point. Tweets from educators and free and open source (FOSS) advocates worldwide kept me informed and engaged in a learning network that was without equal. That's changed over the past half dozen years, and recently a change in ownership and the shaping of what I read was driven more by an algorithm than by my personal interests and choices. During a yearly meetup of correspondents and editors of Opensource.com a few years ago, Seth Kenlon suggested giving Mastodon a try. I joined Fosstodon in 2019. Fosstodon is a Mastodon instance for a community of like-minded people who enjoy free and open source software.

Mastodon vs Twitter

Change is not easy. Being a creature of habit, I stayed with my old standby even though it was becoming increasingly tiresome. The threat of its sale in the spring of 2022 invited me to reconsider Fosstodon.

1. Favorite instead of like

The Mastodon interface is similar to Twitter. Rather than "liking" a post, you "favorite" a post on Mastodon by clicking the star icon under the post content.

Image by:

(Don Watkins, CC BY-SA 4.0)

2. Share a post

Re-sharing on my old network is a "retweet," but on Mastodon, it's a "boost." You click the double-arrow icon under the post content to boost a post.

Image by:

(Don Watkins, CC BY-SA 4.0)

3. Mastodon instances

Because anyone can run a Mastodon instance, different instances not only have unique communities (like the ones that form around specific hashtags on Twitter, but Mastodon also has hashtags). Some have a unique set of rules. For instance, unlike my former social network, there were content moderation rules on Fosstodon that seemed strict initially. I made a post unrelated to FOSS software, and my post was removed. I was told it had been removed because I'd not issued a "content warning." That irked me, so I looked for another instance and found a couple more to my liking. One was Mastodon.social, and the other Scholar.social. The former is a general server with no expectation about what you will post. The latter was an instance dedicated to academics. In all cases, there are well-enforced codes of conduct.

Each instance has rules, and while they differ slightly in the description, they clearly spell out what is and is not acceptable behavior. Fosstodon published its code of conduct, which established the rules and expectations of behavior on the site.

More great content Free online course: RHEL technical overview Learn advanced Linux commands Download cheat sheets Find an open source alternative Explore open source resources 4. Open source social networking

If you want to run your own Mastodon instance or help develop one, you'll be happy to know that Mastodon is open source. It uses an AGPLv3 license, and its source code is available as a Git repository. The software provides a social network server that uses the ActivityPub protocol to communicate with other servers worldwide.

Mastodon is not a single site on the internet but a series of sites spanning the globe and communicating with each other. This federated network is referred to as the "fediverse." Unlike other social networks, where there's a single owner of the network, Mastodon and other ActivityPub sites are owned by anyone who runs a server.

From a user's perspective, this doesn't matter at first. You can sign up on any Mastodon instance and then connect to all other instances.

There is power to this distributed design, though. If you encounter an instance with a community producing content you'd rather not see, you can block either a single user from that instance or the whole instance.

In the past month, I've returned to Fosstodon primarily because open source is my passion. I enjoy sharing open source content on Fosstodon because the other users of Fosstodon are generally receptive to posts about free and open source software. When I have something to share that's not considered appropriate on Fosstodon, I share it on Scholar.social or Mastodon.social.

Not all instances have topics they focus on, and even those that do often use their topical interests as a guideline rather than grounds for strict removal of posts. If you have a particular interest, you might be able to find a community built around that topic, and you're likely to see that you have an instant audience. Of course, you'll still always be able to communicate with users of other instances, too.

Try Mastodon

Mastodon is not a corporation. All of its instances are staffed and supported by each server's contributors. Some instances make it easy to support them with Patreon or PayPal.

I have found the fediverse a welcoming place that brings joy back into social networking. Have you joined Mastodon? What are your takeaways? Let us know in the comments.

Mastodon is not a corporation. All of its instances are staffed and supported by each server's contributors. Here are a few other advantages.

Image by:

Opensource.com

Alternatives What to read next This work is licensed under a Creative Commons Attribution-Share Alike 4.0 International License. Register or Login to post a comment.

Audit your sharding database algorithm Yacine Si Tayeb, PhD Thu, 11/10/2022 - 03:00

Thanks to the ShardingSphere community's continuous review and feedback to develop features such as data sharding and read/write splitting, our team found that some users create many shards when using the data sharding feature.

In such cases, there can be 1,000 physical tables corresponding to a sharding logical table, which disturbs users.

For instance, a SELECT * FROM t_order statement will lead to a full-route, which is obviously not the case for OLTP. This SQL can be placed in another Proxy to avoid blocking other requests.

However, if users are not familiar with Proxy or how to write a where condition and don't know that sharding is not supported in this condition, a full-route is still required.

A full-route can lower the performance of Proxy and even result in the failure of a reasonable request. Imagine that there are 1,000 shards in a physical database. If they are executed in parallel, 1,000 connections are needed, and if in serial, the request can lead to a timeout. For this reason, community users asked whether the unreasonable request could be intercepted directly.

Our team considered the issue for a while. One option is to simply block the full-route operation. Doing so requires a check in the code and adding a switch to the configuration file. On the other hand, if the user later needs to set a table to read-only or requires the update operation to carry a limit, does that mean the code and configuration change again? This approach obviously goes against the pluggable logic of Proxy.

In response to the above problems, the recently released Apache ShardingSphere 5.2.0 provides users with auditing for the SQL sharding function. The audit can either be an interception operation or a statistical operation. Similar to the sharding and unique key generation algorithms, the audit algorithm is plugin-oriented, user-defined, and configurable.

[ Related read 5 new improvements in Apache ShardingSphere ]

Next, I will elaborate on the implementation logic for auditing data sharding with specific SQL examples.

Audit for sharding interface

The entrance to Apache ShardingSphere's audit is in the org.apache.shardingsphere.infra.executor.check.SQLCheckEngine class, which will invoke the check method of the SQLChecker interface. Currently, the ShardingSphere audit contains an audit for permission (verify username and password) and an audit for sharding.

This example focuses on the parent interface implemented in the ShardingAuditChecker of audit for sharding.

Image by:

(Yacine Si Tayeb, CC BY-SA 4.0)

You can learn its working principles quickly by viewing the check code of org.apache.shardingsphere.sharding.checker.audit.ShardingAuditChecker.

public interface ShardingAuditAlgorithm extends ShardingSphereAlgorithm {
   
    /**
     * Sharding audit algorithm SQL check.
     *
     * @param sqlStatementContext SQL statement context
     * @param parameters SQL parameters
     * @param grantee grantee
     * @param database database
     * @return SQL check result
     */
    SQLCheckResult CHECK(SQLStatementContext<?> sqlStatementContext, List<Object> parameters, Grantee grantee, ShardingSphereDatabase DATABASE);
}

This method obtains the audit strategies of all the sharding tables involved and invokes the audit algorithms configured in each sharding table audit strategy. An exception is displayed to the user if an audit algorithm fails to pass.

Some users may wonder what disableAuditNames does here. The sharding audit also allows users to skip this process. In some cases, users may need to execute SQL that should have been blocked by the audit, and they are aware of the impact of this SQL.

Users can utilize the Hint: disableAuditNames to skip audit interception, which will be described with practical examples later. The Proxy Administrators can configure allowHintDisable to control whether to allow users to skip this process. The default value is true, indicating that a Hint-based skip is permitted.

More on data science What is data science? What is Python? How to become a data scientist Data scientist: A day in the life Use JupyterLab in the Red Hat OpenShift Data Science sandbox Whitepaper: Data-intensive intelligent applications in a hybrid cloud blueprint MariaDB and MySQL cheat sheet Latest data science articles Audit for sharding algorithm

The audit for sharding algorithm interface org.apache.shardingsphere.sharding.spi.ShardingAuditAlgorithm is inherited from SPI class ShardingSphereAlgorithm. It inherits type and props properties and defines its own check method. If you want to customize your audit algorithm, just implement the interface and add it to INF.services.

Image by:

(Yacine Si Tayeb, CC BY-SA 4.0)

public interface ShardingAuditAlgorithm extends ShardingSphereAlgorithm {
   
    /**
     * Sharding audit algorithm SQL check.
     *
     * @param sqlStatementContext SQL statement context
     * @param parameters SQL parameters
     * @param grantee grantee
     * @param database database
     * @return SQL check result
     */
    SQLCheckResult CHECK(SQLStatementContext<?> sqlStatementContext, List<Object> parameters, Grantee grantee, ShardingSphereDatabase DATABASE);
}

Apache ShardingSphere implements a general audit for sharding algorithm org.apache.shardingsphere.sharding.algorithm.audit.DMLShardingConditionsShardingAuditAlgorithm, namely the above-mentioned SQL statement that intercepts the full-route.

The algorithm makes decisions by determining whether the sharding condition is null. Of course, it won't intercept broadcast tables and non-sharding tables.

public final class DMLShardingConditionsShardingAuditAlgorithm implements ShardingAuditAlgorithm {
   
    @Getter
    private Properties props;
   
    @Override
    public void init(final Properties props) {
        this.props = props;
    }
   
    @SuppressWarnings({"rawtypes", "unchecked"})
    @Override
    public SQLCheckResult CHECK(final SQLStatementContext<?> sqlStatementContext, final List<Object> parameters, final Grantee grantee, final ShardingSphereDatabase DATABASE) {
        IF (sqlStatementContext.getSqlStatement() instanceof DMLStatement) {
            ShardingRule rule = DATABASE.getRuleMetaData().getSingleRule(ShardingRule.class);
            IF (rule.isAllBroadcastTables(sqlStatementContext.getTablesContext().getTableNames())
                    || sqlStatementContext.getTablesContext().getTableNames().stream().noneMatch(rule::isShardingTable)) {
                RETURN NEW SQLCheckResult(TRUE, "");
            }
            ShardingConditionEngine shardingConditionEngine = ShardingConditionEngineFactory.createShardingConditionEngine(sqlStatementContext, DATABASE, rule);
            IF (shardingConditionEngine.createShardingConditions(sqlStatementContext, parameters).isEmpty()) {
                RETURN NEW SQLCheckResult(FALSE, "Not allow DML operation without sharding conditions");
            }
        }
        RETURN NEW SQLCheckResult(TRUE, "");
    }
   
    @Override
    public String getType() {
        RETURN "DML_SHARDING_CONDITIONS";
    }
}

I'd like to introduce another audit for the sharding algorithm: LimitRequiredShardingAuditAlgorithm. This algorithm can intercept SQL without carrying a limit in the update and delete operations.

As this algorithm is less universal, it is not currently integrated into Apache ShardingSphere. As you can see, it is very easy to implement a custom algorithm, which is why the audit for sharding framework is needed. Thanks to its plugin-oriented architecture, ShardingSphere boasts great scalability.

public final class LimitRequiredShardingAuditAlgorithm implements ShardingAuditAlgorithm {
   
    @Getter
    private Properties props;
   
    @Override
    public void init(final Properties props) {
        this.props = props;
    }
   
    @SuppressWarnings({"rawtypes", "unchecked"})
    @Override
    public SQLCheckResult CHECK(final SQLStatementContext<?> sqlStatementContext, final List<Object> parameters, final Grantee grantee, final ShardingSphereDatabase DATABASE) {
        IF (sqlStatementContext instanceof UpdateStatementContext && !((MySQLUpdateStatement) sqlStatementContext.getSqlStatement()).getLimit().isPresent()) {
            RETURN NEW SQLCheckResult(FALSE, "Not allow update without limit");
        }
        IF (sqlStatementContext instanceof DeleteStatementContext && !((MySQLDeleteStatement) sqlStatementContext.getSqlStatement()).getLimit().isPresent()) {
            RETURN NEW SQLCheckResult(FALSE, "Not allow delete without limit");
        }
        RETURN NEW SQLCheckResult(TRUE, "");
    }
   
    @Override
    public String getType() {
        RETURN "LIMIT_REQUIRED";
    }
}Use audit for sharding

Audit for sharding requires you to configure an audit strategy for logical tables. To help you get started quickly, its configuration is the same as that of the sharding algorithm and the sharding key value generator.

There is an algorithm definition and strategy definition, and a default audit strategy is also supported. If the audit strategy is configured in the logical table, it affects only that logical table.

If defaultAuditStrategy is configured in the logical table, it takes effect for all the logical tables under the sharding rule. Auditors are similar to ShardingAlgorithms, auditStrategy to databaseStrategy, and defaultAuditStrategy to defaultDatabaseStrategy or defaultTableStrategy.

Please refer to the following example. Only the configuration of the audit for sharding is displayed. You must configure the sharding algorithm and data source yourself.

rules:
  - !SHARDING
    TABLES:
      t_order:
        actualDataNodes: ds_${0..1}.t_order_${0..1}
        auditStrategy:
          auditorNames:
            - sharding_key_required_auditor
          allowHintDisable: TRUE
    defaultAuditStrategy:
      auditorNames:
        - sharding_key_required_auditor
      allowHintDisable: TRUE
    auditors:
      sharding_key_required_auditor:
        TYPE: DML_SHARDING_CONDITIONS

Step 1: Execute a query operation. An error is displayed as the audit strategy for intercepting the full-database route is configured.

mysql> SELECT * FROM t_order;
ERROR 13000 (44000): SQL CHECK failed, error message: NOT allow DML operation WITHOUT sharding conditions

Step 2: Add HINT. The name of the HINT is /* ShardingSphere hint: disableAuditNames */，and disableAuditNames is followed by the auditorsNames configured in the preceding command.

If multiple names exist, separate them with spaces such as/* ShardingSphere hint: disableAuditNames=auditName1 auditName2*/. After using HINT, you can see that the SQL operation is successfully executed.

mysql> /* ShardingSphere hint: disableAuditNames=sharding_key_required_auditor */ SELECT * FROM t_order;
+----------+---------+------------+--------+
| order_id | user_id | address_id | STATUS |
+----------+---------+------------+--------+
|       30 |      20 |         10 | 20     |
|       32 |      22 |         10 | 20     |
+----------+---------+------------+--------+
2 ROWS IN SET (0.01 sec)

Note: HINT requires you to modify the server.yaml configuration of Proxy. In addition, if you are using MySQL terminal to connect to Proxy directly, you need to add the -c property—otherwise, HINT comments will be filtered out of the MySQL terminal and will not be parsed by Proxy on the backend.

rules:
  - !SQL_PARSER
    sqlCommentParseEnabled: TRUE
    sqlStatementCache:
      initialCapacity: 2000
      maximumSize: 65535
    parseTreeCache:
      initialCapacity: 128
      maximumSize: 1024
props:
  proxy-hint-enabled: TRUE
mysql -uroot -proot -h127.0.0.1 -P3307  -cDistSQL with audit for sharding

As you can see from the release notes, Apache ShardingSphere 5.2.0 supports the following DistSQL with audit for sharding function:

CREATE SHARDING AUDITOR
ALTER SHARDING AUDITOR
SHOW SHARDING AUDIT ALGORITHMS
The following DistSQL will be supported IN future releases:

DROP SHARDING AUDITOR
SHOW UNUSED SHARDING AUDIT ALGORITHMS
CREATE SHARDING TABLE RULE # including AUDIT_STRATEGY

This post introduced how audit for sharding works with specific examples. I believe you already have a basic understanding of this function and can use it whenever you need or use a custom algorithm.

You are also welcome to submit general algorithms to the community. If you have any ideas to contribute or encounter issues with your ShardingSphere, feel free to post them on GitHub.

This article originally appeared on ShardingSphere 5.2.0: Audit for sharding intercepts unreasonable requests in multi-shards scenarios and is republished with permission.

This demonstration of ShardingSphere 5.2.0 elaborates on the implementation logic for auditing data sharding with specific SQL examples.

Image by:

Image by Mapbox Uncharted ERG, CC-BY 3.0 US

Databases What to read next Improve your database knowledge with this MariaDB and MySQL cheat sheet This work is licensed under a Creative Commons Attribution-Share Alike 4.0 International License. Register or Login to post a comment.

8 Best Screen Recorders For Linux  Beebom

The post GhostBSD – A Simple Unix-like Desktop OS Based on FreeBSD first appeared on Tecmint: Linux Howtos, Tutorials & Guides .

Brief: This article describes the basic instructions on installing GhostBSD using the graphical installer using DVD / USB method. GhostBSD is an open-source Unix-like desktop operating system created upon the most recent release version

The post GhostBSD – A Simple Unix-like Desktop OS Based on FreeBSD first appeared on Tecmint: Linux Howtos, Tutorials & Guides.

Following last week's release of Mesa 22.3-rc1 that also marked the feature freeze for this quarter's release cycle, Mesa 22.3-rc2 is out today with an initial batch of bug fixes...

How to Add AppImages to Your System Menu on Linux  MUO - MakeUseOf

Atempo Partners with CIQ to Complete Certification Process for Rocky Linux on All Atempo Offerings  EIN News

Soulstone Survivors might dethrone the likes of Vampire Survivors for me  GamingOnLinux

Soulstone Survivors might dethrone the likes of Vampire Survivors for me  GamingOnLinux