Open-source News

EXT4 Sees Some Performance Optimizations, Folio Conversion With Linux 6.4

Phoronix - Tue, 04/25/2023 - 20:20
While often times the EXT4 file-system driver updates for new Linux kernel merge windows can be rather mundane given the maturity of this widely-used Linux file-system, this time around for Linux 6.4 it's a bit more exciting...

Linux 6.4 Can Run As A Confidential AMD SEV-SNP vTOM Guest On Microsoft Hyper-V

Phoronix - Tue, 04/25/2023 - 18:46
For those making use of Microsoft Hyper-V virtualization there are some notable additions to find with the in-development Linux 6.4 kernel...

Initial Apple M2 Support & Other 64-bit ARM Changes For Linux 6.4

Phoronix - Tue, 04/25/2023 - 18:26
On Monday the ARM64 (AArch64) architecture code changes were submitted for the in-development Linux 6.4 kernel along with the various SoC updates and various platform/machine additions for ARM hardware with this new kernel version...

What's new in GNOME 44?

opensource.com - Tue, 04/25/2023 - 15:00
What's new in GNOME 44? Jim Hall Tue, 04/25/2023 - 03:00

I use GNOME as my primary desktop environment on my Linux PC at home. GNOME gives me an easy-to-use graphical desktop that provides the flexibility I need yet doesn't get in my way when I focus on my work.

GNOME recently released GNOME 44 with a bunch of new features. I reached out to the GNOME team to ask about the latest version and what was in it. Here's what team members Caroline Henriksen (brand manager), Matthias Clasen (GNOME developer and release team member), and Allan Day (design team) had to share.

New GNOME features

Jim Hall: What are some of the new and updated features in GNOME 44 that you're most excited about?

GNOME Team: I am very excited to see how fresh and modern our user interfaces look. Not just in the core apps like Files (the file manager, Nautilus) but also in our Settings, which have seen a lot of work in the last cycle—many Settings panels have been improved. If you have a chance, you should try the new Mouse & Touchpad panel and enjoy the animated illustrations.

There's a lot to like in GNOME 44. For example, I think that a lot of people are going to be really happy about the new grid view in the file chooser, as well as being able to easily connect devices from the new Bluetooth menu in the quick settings.

Jim: The release notes mention GNOME Circle and that a few new apps have been added. What is GNOME Circle?

Team: GNOME Circle is a collection of fantastic apps that use the GNOME platform. It's GNOME's way of promoting the best apps that use our technologies and supporting app developers.

To be included in GNOME Circle, an app has to meet a set of requirements. Once it does, the developers get things like extra publicity and GNOME Foundation membership. That, in turn, gives them access to additional infrastructure and travel sponsorship. More information and how to apply can be found on the GNOME Circle page.

We're thrilled with how successful GNOME Circle has been. It contains more than 50 apps now! I particularly like that not all of these apps revolve around computing. You can find apps like a health tracker, a metronome, or a chess clock.

Jim: GNOME is the standard desktop in several Linux distributions. Where can we expect to see GNOME 44?

Team: The upcoming Fedora 38 release will include GNOME 44 and should be out sometime in April, as will Ubuntu 23.04. And GNOME 44 builds have already landed in openSUSE's Tumbleweed and MicroOS, to name just a few of the major distros.

More Linux resources Linux commands cheat sheet Advanced Linux commands cheat sheet Free online course: RHEL technical overview Linux networking cheat sheet SELinux cheat sheet Linux common commands cheat sheet What are Linux containers? Register for your free Red Hat account Our latest Linux articles The GNOME community

Jim: The release name for GNOME 44 is Kuala Lumpur. Where does this name come from?

Team: GNOME has two major yearly conferences, GUADEC in the middle of the year (the next conference will take place in Latvia in July 2023) and GNOME Asia towards the end of the year. We are very thankful to the local team in Malaysia who welcomed us for GNOME Asia 2022 in Kuala Lumpur.

Organizing these events takes a lot of effort and commitment from the GNOME staff and the local teams. As a small sign of our appreciation, GNOME releases are named after the location of the most recent conference. This naming scheme was introduced a number of years ago. GNOME 3.18, Gothenburg, was the first.

Jim: GNOME has a strong user community with active members. How does GNOME keep the community so engaged?

Team: GNOME has always been a community-driven project with a strong sense of collaboration and inclusivity. That's part of what makes being a GNOME contributor and user so rewarding. Being a member of the GNOME community means that you get to interact with people from all over the world to work on common goals and exchange ideas. It is an enriching and inspiring experience, and I think that is what helps keep our community excited and engaged.

One important aspect of fostering that engagement is meeting our community where they're at and making our events more accessible to people from all over the world. For example, our flagship conference, GUADEC, was hosted in Guadalajara, Mexico, last year. This was the first time GUADEC happened outside of Europe, and this helped make it easier for GNOME users and contributors in Latin America to attend.

We also make an effort to meet our community members not just online and at our own conferences but at other events such as Linux Application Summit, FOSDEM, or SCaLE. If you see a GNOME booth at any of these events, please stop by and say hi. You'll often find developers, designers, foundation staff, and board members all happy to chat and answer questions.

Get involved with GNOME

Jim: How can folks get started with writing their own apps for GNOME? If I wanted to learn how to write my first "hello world" app for GNOME, is there a tutorial I can follow?

Team: The Get started developing for GNOME site includes a collection of tutorials, including a guide on quickly creating your first app. With new technologies like Flatpak and GNOME Builder, it's amazing just how easy it is to create your own app nowadays. Fire up Builder, click "new project," fill in some details, and you'll have your own running GNOME app. It really is that easy.

Jim: What are some ways that people can contribute?

Team: If someone is interested in GNOME and is motivated to get involved, there are definitely things they can do to help. Participating in discussions on our Discourse instance or reporting issues is a great place to start if you're a beginner. There are also lots of non-technical jobs that need doing, like helping with our documentation, translating GNOME into different languages, or even helping organize our annual conferences. A lot of these activities have friendly teams working on them who will help you to get started.

Alternatively, if you have coding experience, you can browse our "newcomer" tickets for tasks that might interest you.

Another way to contribute is through donating to GNOME. As an open source project and a non-profit foundation, regular donations help us continue to build up GNOME, provide necessary infrastructure, and power new initiatives.

[ Get the guide to installing applications on Linux ]

The GNOME Linux desktop's latest release is now available. Find out about the new and improved Bluetooth, user interface, apps, and other features in GNOME 44.

Image by:

Gunnar Wortmann via Pixabay. Modified by Opensource.com. CC BY-SA 4.0.

Linux What to read next My first impression of GNOME Console on Linux Linux desktops: KDE vs GNOME 24 Linux desktops you need to try This work is licensed under a Creative Commons Attribution-Share Alike 4.0 International License. Register or Login to post a comment.
Categories: UR Solutions News

Retry your Python code until it fails

opensource.com - Tue, 04/25/2023 - 15:00
Retry your Python code until it fails Moshe Zadka Tue, 04/25/2023 - 03:00

Sometimes, a function is called with bad inputs or in a bad program state, so it fails. In languages like Python, this usually results in an exception.

But sometimes exceptions are caused by different issues or are transitory. Imagine code that must keep working in the face of caching data being cleaned up. In theory, the code and the cleaner could carefully agree on the clean-up methodology to prevent the code from trying to access a non-existing file or directory. Unfortunately, that approach is complicated and error-prone. However, most of these problems are transitory, as the cleaner will eventually create the correct structures.

Even more frequently, the uncertain nature of network programming means that some functions that abstract a network call fail because packets were lost or corrupted.

A common solution is to retry the failing code. This practice allows skipping past transitional problems while still (eventually) failing if the issue persists. Python has several libraries to make retrying easier. This is a common "finger exercise."

Tenacity

One library that goes beyond a finger exercise and into useful abstraction is tenacity. Install it with pip install tenacity or depend on it using a dependencies = tenacity line in your pyproject.toml file.

Set up logging

A handy built-in feature of tenacity is support for logging. With error handling, seeing log details about retry attempts is invaluable.

To allow the remaining examples display log messages, set up the logging library. In a real program, the central entry point or a logging configuration plugin does this. Here's a sample:

import logging logging.basicConfig( level=logging.INFO, format="%(asctime)s:%(name)s:%(levelname)s:%(message)s", ) TENACITY_LOGGER = logging.getLogger("Retrying")Selective failure

To demonstrate the features of tenacity, it's helpful to have a way to fail a few times before finally succeeding. Using unittest.mock is useful for this scenario.

from unittest import mock thing = mock.MagicMock(side_effect=[ValueError(), ValueError(), 3])

If you're new to unit testing, read my article on mock.

Before showing the power of tenacity, look at what happens when you implement retrying directly inside a function. Demonstrating this makes it easy to see the manual effort using tenacity saves.

def useit(a_thing): for i in range(3): try: value = a_thing() except ValueError: TENACITY_LOGGER.info("Recovering") continue else: break else: raise ValueError() print("the value is", value)

The function can be called with something that never fails:

>>> useit(lambda: 5) the value is 5

With the eventually-successful thing:

>>> useit(thing) 2023-03-29 17:00:42,774:Retrying:INFO:Recovering 2023-03-29 17:00:42,779:Retrying:INFO:Recovering the value is 3

Calling the function with something that fails too many times ends poorly:

try: useit(mock.MagicMock(side_effect=[ValueError()] * 5 + [4])) except Exception as exc: print("could not use it", repr(exc))

The result:

2023-03-29 17:00:46,763:Retrying:INFO:Recovering 2023-03-29 17:00:46,767:Retrying:INFO:Recovering 2023-03-29 17:00:46,770:Retrying:INFO:Recovering could not use it ValueError()Simple tenacity usage

For the most part, the function above was retrying code. The next step is to have a decorator handle the retrying logic:

import tenacity my_retry=tenacity.retry( stop=tenacity.stop_after_attempt(3), after=tenacity.after_log(TENACITY_LOGGER, logging.WARNING), )

Tenacity supports a specified number of attempts and logging after getting an exception.

The useit function no longer has to care about retrying. Sometimes it makes sense for the function to still consider retryability. Tenacity allows code to determine retryability by itself by raising the special exception TryAgain:

@my_retry def useit(a_thing): try: value = a_thing() except ValueError: raise tenacity.TryAgain() print("the value is", value)

Now when calling useit, it retries ValueError without needing custom retrying code:

useit(mock.MagicMock(side_effect=[ValueError(), ValueError(), 2]))

The output:

2023-03-29 17:12:19,074:Retrying:WARNING:Finished call to '__main__.useit' after 0.000(s), this was the 1st time calling it. 2023-03-29 17:12:19,080:Retrying:WARNING:Finished call to '__main__.useit' after 0.006(s), this was the 2nd time calling it. the value is 2Configure the decorator

The decorator above is just a small sample of what tenacity supports. Here's a more complicated decorator:

my_retry = tenacity.retry( stop=tenacity.stop_after_attempt(3), after=tenacity.after_log(TENACITY_LOGGER, logging.WARNING), before=tenacity.before_log(TENACITY_LOGGER, logging.WARNING), retry=tenacity.retry_if_exception_type(ValueError), wait=tenacity.wait_incrementing(1, 10, 2), reraise=True )

More Python resources What is an IDE? Cheat sheet: Python 3.7 for beginners Top Python GUI frameworks Download: 7 essential PyPI libraries Red Hat Developers Register for your free Red Hat account Latest Python articles

This is a more realistic decorator example with additional parameters:

  • before: Log before calling the function
  • retry: Instead of only retrying TryAgain, retry exceptions with the given criteria
  • wait: Wait between calls (this is especially important if calling out to a service)
  • reraise: If retrying failed, reraise the last attempt's exception

Now that the decorator also specifies retryability, remove the code from useit:

@my_retry def useit(a_thing): value = a_thing() print("the value is", value)

Here's how it works:

useit(mock.MagicMock(side_effect=[ValueError(), 5]))

The output:

2023-03-29 17:19:39,820:Retrying:WARNING:Starting call to '__main__.useit', this is the 1st time calling it. 2023-03-29 17:19:39,823:Retrying:WARNING:Finished call to '__main__.useit' after 0.003(s), this was the 1st time calling it. 2023-03-29 17:19:40,829:Retrying:WARNING:Starting call to '__main__.useit', this is the 2nd time calling it. the value is 5

Notice the time delay between the second and third log lines. It's almost exactly one second:

>>> useit(mock.MagicMock(side_effect=[5])) 2023-03-29 17:20:25,172:Retrying:WARNING:Starting call to '__main__.useit', this is the 1st time calling it. the value is 5

With more detail:

try: useit(mock.MagicMock(side_effect=[ValueError("detailed reason")]*3)) except Exception as exc: print("retrying failed", repr(exc))

The output:

2023-03-29 17:21:22,884:Retrying:WARNING:Starting call to '__main__.useit', this is the 1st time calling it. 2023-03-29 17:21:22,888:Retrying:WARNING:Finished call to '__main__.useit' after 0.004(s), this was the 1st time calling it. 2023-03-29 17:21:23,892:Retrying:WARNING:Starting call to '__main__.useit', this is the 2nd time calling it. 2023-03-29 17:21:23,894:Retrying:WARNING:Finished call to '__main__.useit' after 1.010(s), this was the 2nd time calling it. 2023-03-29 17:21:25,896:Retrying:WARNING:Starting call to '__main__.useit', this is the 3rd time calling it. 2023-03-29 17:21:25,899:Retrying:WARNING:Finished call to '__main__.useit' after 3.015(s), this was the 3rd time calling it. retrying failed ValueError('detailed reason')

Again, with KeyError instead of ValueError:

try: useit(mock.MagicMock(side_effect=[KeyError("detailed reason")]*3)) except Exception as exc: print("retrying failed", repr(exc))

The output:

2023-03-29 17:21:37,345:Retrying:WARNING:Starting call to '__main__.useit', this is the 1st time calling it. retrying failed KeyError('detailed reason')Separate the decorator from the controller

Often, similar retrying parameters are needed repeatedly. In these cases, it's best to create a retrying controller with the parameters:

my_retryer = tenacity.Retrying( stop=tenacity.stop_after_attempt(3), after=tenacity.after_log(TENACITY_LOGGER, logging.WARNING), before=tenacity.before_log(TENACITY_LOGGER, logging.WARNING), retry=tenacity.retry_if_exception_type(ValueError), wait=tenacity.wait_incrementing(1, 10, 2), reraise=True )

Decorate the function with the retrying controller:

@my_retryer.wraps def useit(a_thing): value = a_thing() print("the value is", value)

Run it:

>>> useit(mock.MagicMock(side_effect=[ValueError(), 5])) 2023-03-29 17:29:25,656:Retrying:WARNING:Starting call to '__main__.useit', this is the 1st time calling it. 2023-03-29 17:29:25,663:Retrying:WARNING:Finished call to '__main__.useit' after 0.008(s), this was the 1st time calling it. 2023-03-29 17:29:26,667:Retrying:WARNING:Starting call to '__main__.useit', this is the 2nd time calling it. the value is 5

This lets you gather the statistics of the last call:

>>> my_retryer.statistics {'start_time': 26782.847558759, 'attempt_number': 2, 'idle_for': 1.0, 'delay_since_first_attempt': 0.0075125470029888675}

Use these statistics to update an internal statistics registry and integrate with your monitoring framework.

Extend tenacity

Many of the arguments to the decorator are objects. These objects can be objects of subclasses, allowing deep extensionability.

For example, suppose the Fibonacci sequence should determine the wait times. The twist is that the API for asking for wait time only gives the attempt number, so the usual iterative way of calculating Fibonacci is not useful.

One way to accomplish the goal is to use the closed formula:

A little-known trick is skipping the subtraction in favor of rounding to the closest integer:

Which translates to Python as:

int(((1 + sqrt(5))/2)**n / sqrt(5) + 0.5)

This can be used directly in a Python function:

from math import sqrt def fib(n): return int(((1 + sqrt(5))/2)**n / sqrt(5) + 0.5)

The Fibonacci sequence counts from 0 while the attempt numbers start at 1, so a wait function needs to compensate for that:

def wait_fib(rcs): return fib(rcs.attempt_number - 1)

The function can be passed directly as the wait parameter:

@tenacity.retry( stop=tenacity.stop_after_attempt(7), after=tenacity.after_log(TENACITY_LOGGER, logging.WARNING), wait=wait_fib, ) def useit(thing): print("value is", thing()) try: useit(mock.MagicMock(side_effect=[tenacity.TryAgain()] * 7)) except Exception as exc: pass

Try it out:

2023-03-29 18:03:52,783:Retrying:WARNING:Finished call to '__main__.useit' after 0.000(s), this was the 1st time calling it. 2023-03-29 18:03:52,787:Retrying:WARNING:Finished call to '__main__.useit' after 0.004(s), this was the 2nd time calling it. 2023-03-29 18:03:53,789:Retrying:WARNING:Finished call to '__main__.useit' after 1.006(s), this was the 3rd time calling it. 2023-03-29 18:03:54,793:Retrying:WARNING:Finished call to '__main__.useit' after 2.009(s), this was the 4th time calling it. 2023-03-29 18:03:56,797:Retrying:WARNING:Finished call to '__main__.useit' after 4.014(s), this was the 5th time calling it. 2023-03-29 18:03:59,800:Retrying:WARNING:Finished call to '__main__.useit' after 7.017(s), this was the 6th time calling it. 2023-03-29 18:04:04,806:Retrying:WARNING:Finished call to '__main__.useit' after 12.023(s), this was the 7th time calling it.

Subtract subsequent numbers from the "after" time and round to see the Fibonacci sequence:

intervals = [ 0.000, 0.004, 1.006, 2.009, 4.014, 7.017, 12.023, ] for x, y in zip(intervals[:-1], intervals[1:]): print(int(y-x), end=" ")

Does it work? Yes, exactly as expected:

0 1 1 2 3 5 Wrap up

Writing ad-hoc retry code can be a fun distraction. For production-grade code, a better choice is a proven library like tenacity. The tenacity library is configurable and extendable, and it will likely meet your needs.

Use the Tenacity and Mock libraries to find the bugs hiding deep within your code.

Image by:

Photo by Jen Wike Huger, CC BY-SA; Original photo by Torkild Retvedt

Python What to read next This work is licensed under a Creative Commons Attribution-Share Alike 4.0 International License. Register or Login to post a comment.

18 Best NodeJS Frameworks for App Development in 2023

Tecmint - Tue, 04/25/2023 - 14:25
The post 18 Best NodeJS Frameworks for App Development in 2023 first appeared on Tecmint: Linux Howtos, Tutorials & Guides .

Node.js is used to build fast, highly scalable network applications based on an event-driven non-blocking input/output model, and single-threaded asynchronous programming. A web application framework is a combination of libraries, helpers, and tools that

The post 18 Best NodeJS Frameworks for App Development in 2023 first appeared on Tecmint: Linux Howtos, Tutorials & Guides.

NVM – Install Multiple Node.js Versions in Linux

Tecmint - Tue, 04/25/2023 - 13:31
The post NVM – Install Multiple Node.js Versions in Linux first appeared on Tecmint: Linux Howtos, Tutorials & Guides .

Node Version Manager (NVM in short) is a simple bash script to manage multiple active node.js versions on your Linux system. It allows you to install multiple node.js versions, and view all versions available

The post NVM – Install Multiple Node.js Versions in Linux first appeared on Tecmint: Linux Howtos, Tutorials & Guides.

Pages