Open-source News

The Rusticl Rust-based OpenCL driver implementation that was merged last year into Mesa 22.3 has been an extremely promising effort. One of the issues though is that with the current mainline state inside Mesa has lacked support for working with AMD Radeon GPUs via the RadeonSI Gallium3D driver, but that is now coming to fruition...

Linux Foundation Welcomes New Open Source Projects from Peraton Labs to Scale 5G Security  Yahoo Finance

Linux Foundation Welcomes New Open Source Projects from ...  PR Newswire

The x86 CPU updates have been merged for the Linux 6.3 kernel that include a few new features worth mentioning for AMD and Intel customers...

Intel LKGS, AMD Automatic IBRS & Caching AMD Debug Registers Merged For Linux 6.3  Phoronix

iXsystems Introduces TrueNAS Mini R Appliance with Linux-based ...  insideHPC

A Phoronix reader pointed out that there are initial code that landed for adding RISC-V processor support to Microsoft's .NET runtime...

HID-BPF Ready For Linux 6.3 Along With Steam Deck Controller ...  Phoronix

The HID subsystem updates for Linux 6.3 have been submitted and they contain a number of exciting input updates for this spring 2023 kernel version from the introduction of HID-BPF to native Steam Deck controller interface handling...

Linux Operating System Market Growth Research 2023-2027: by ...  Digital Journal

While Ubuntu Linux hasn't provided Flatpak support out-of-the-box due to their preference of using their own Snap app packaging/distribution format, Ubuntu flavors/spins have to this point been able to pre-install Flatpak support if they desired. However, for the 23.04 "Lunar Lobster" cycle and moving forward, Ubuntu flavors will no longer be permitted to install Flatpak packages by default...

Ubuntu Flavors/Spins Will No Longer Be Able To Install Flatpak By Default  Phoronix

Tailoring your Linux command prompt  Network World

A week after seeing X.Org display driver updates for old Trident and S3 Graphics hardware, a new release of xf86-video-ark is now available that provides the open-source UMS display driver support for old ARK Logic hardware. ARK Logic only lasted through the 1990's as a purveyor of PCI video cards...

Computop debuts three new Linux-based payment terminals  Finextra

As part of the Linux networking updates for the Linux 6.3 cycle in addition to the BIG TCP support for IPv4 that can yield higher throughput and lower latency, another networking change worth pointing out is the introduction of the "ath12k" driver for Qualcomm WiFi 7 hardware support...

Linux 6.3 Adds ath12k Driver For Qualcomm WiFi 7 Hardware Support  Phoronix

Linus Torvalds Reacts to Poorly-Executed Pull Requests for Linux 6.3  It's FOSS News

Microsoft .NET 8 bolsters Linux support  InfoWorld

How I do automated accessibility testing for my website dmundra Wed, 02/22/2023 - 03:00

This article covers adding accessibility tests to your site using Pa11y (pa11y-ci with axe) and Cypress (with cypress-axe) in GitLab CI/CD. I use a Jekyll website as an example, but any website technology that runs in CI/CD can leverage this setup.

Prep your website

In addition to getting your website to run in CI/CD, I recommend enabling an XML sitemap feature. A sitemap allows the accessibility tests to parse all URLs to find accessibility issues across the site. I recommend the jekyll-sitemap plugin for Jekyll sites.

Collecting a list of all major URLs is a good alternate step if a sitemap is not possible. The URLs should cover all potential layouts of the website, such as pages with the highest traffic or the most landings. This approach won't catch all accessibility issues, especially content level concerns, but it will test the layout and main pages.

This scenario requires the npm or yarn package managers. I used npm for this article. If your project doesn't have npm initialized, run the npm init command to create the package.json file.

Begin with Pa11y

Pa11y is a free and open source software that tests websites for accessibility issues. Pa11y-ci is the command line utility geared towards continuous integration (CI). Install pa11y-ci as a development dependency with npm:

$ npm i --save-dev pa11y-ci

After you complete the installation, edit the package.json and add the following commands to the scripts section:

"start-detached": "bundle exec jekyll serve --detach", "pa11y-ci:home": "pa11y-ci http://127.0.0.1:4000", "pa11y-ci:sitemap": "pa11y-ci --sitemap http://127.0.0.1:4000/sitemap.xml --sitemap-find https://accessibility.civicactions.com --sitemap-replace http://127.0.0.1:4000 --sitemap-exclude \"/*.pdf\""

• start-detached: Starts the web server that will run Jekyll for testing.
• pa11y-ci:home: Runs pa11y-ci tests on the home page. Useful for troubleshooting.
• pa11y-ci:sitemap: Runs pa11y-ci tests using the sitemap and excludes PDFs. The sitemap will refer to the live site URLs, so replace those with local URLs for testing in the CI pipeline.

Add a JSON file named .pa11yci that configures pa11y-ci with various options. Here is a sample file:

{ "defaults": { "concurrency": 1, "standard": "WCAG2AA", "runners": ["axe", "htmlcs"], "ignore": [ "color-contrast", "frame-tested" ], "chromeLaunchConfig": { "args": ["--disable-dev-shm-usage", "--no-sandbox", "--disable-gpu"] }, "reporters": [ "cli", ["./pa11y-reporter-junit.js", { "fileName": "./pa11y-report-junit.xml" }] ] } }

• concurrency: I reduced this set to 1 because increasing it caused errors (https://github.com/pa11y/pa11y-ci/issues/168 covers the bug, which might be fixed).
• standard: I have stuck with the default WCAG2AA as the goal for this site.
• runners: I ran axe (run tests using axe-core) and htmlcs (default, run tests using HTML CodeSniffer) to cover all potential accessibility issues.
• ignore: With newer versions of axe and some of the changes to the site, I ran into color contrast false positives. I also have an embedded iframe that requires separate testing that axe will report about. I have follow-up issues to examine the axe results, so I am ignoring those criteria for now.
• chromeLaunchConfig: pa11y-ci uses Chrome, and I found that the GitLab CI pipeline requires that the Chrome browser runs properly in the pipeline.
• reports: I use the default command line reporter, but I also added a custom reporter that reports on the pa11y-ci results in a junit format. This came in handy for reporting the results in the GitLab CI pipeline.

That's it. Run this setup locally using npm, and you will see the following output (truncated for brevity):

dmundra in ~/workspace/accessibility/accessibility on branch main > npm run start-detached > start-detached > bundle exec jekyll serve --detach Configuration file: /Users/dmundra/workspace/accessibility/accessibility/_config.yml Source: /Users/dmundra/workspace/accessibility/accessibility Destination: /Users/dmundra/workspace/accessibility/accessibility/_site Incremental build: disabled. Enable with --incremental Generating... done in 8.217 seconds. Auto-regeneration: disabled when running server detached. Server address: http://127.0.0.1:4000 Server detached with pid '14850'. Run `pkill -f jekyll' or `kill -9 14850' to stop the server. dmundra in ~/workspace/accessibility/accessibility on branch main > npm run pa11y-ci:sitemap > pa11y-ci:sitemap > pa11y-ci --sitemap http://localhost:4000/sitemap.xml --sitemap-exclude "/*.pdf" Running Pa11y on 110 URLs: > http://localhost:4000/guide/glossary - 0 errors > http://localhost:4000/guide/introduction - 0 errors > http://localhost:4000/guide/history - 0 errors > http://localhost:4000/guide/design - 0 errors ... ✔ 110/110 URLs passed

The site passes the tests. Here is an example job running in GitLab. The pa11y configuration continues to test all site pages for accessibility issues and report on them.

What does an error look like? Here is an example:

> http://localhost:4000/guide/introduction - 1 errors Errors in http://localhost:4000/guide/introduction: •

and
must only directly contain
1. , or elements (https://dequeuniversity.com/rules/axe/3.5/list?application=axeAPI) (#main-content > div:nth-child(2) > div > div > div > div:nth-child(1) > nav > ul)

   You get a count of the number of errors at a given URL and then details on the accessibility issue. It also displays a link to the criteria being violated and the location in the HTML of the issue.

   Try Cypress

   Cypress is a JavaScript testing framework and is very helpful in writing tests that interact with the site and assert that features work as expected. The setup for Cypress is very similar to pa11y-ci in terms of installation with npm.

   $ npm i --save-dev cypress cypress-axe cypress-real-events

   After the installation is complete, edit the package.json and add the following commands to the scripts section:

   "cypress-tests": "cypress run --browser chrome --headless"
   • cypress-tests: Run the Cypress tests with a headless Chrome browser.

   When launching Cypress for the first time, you get a wizard to create the configuration file. Here is a sample file:

   const { defineConfig } = require('cypress') module.exports = defineConfig({ video: true, videosFolder: 'cypress/results', reporter: 'junit', reporterOptions: { mochaFile: 'cypress/results/junit.[hash].xml', toConsole: false, }, screenshotsFolder: 'cypress/results/screenshots', e2e: { // We've imported your old cypress plugins here. // You may want to clean this up later by importing these. setupNodeEvents(on, config) { return require('./cypress/plugins/index.js')(on, config) }, baseUrl: 'http://localhost:4000', }, })
   • video: Take videos of the tests, which are helpful for troubleshooting.
   • videosFolder: Defines the video storage folder.
   • reporter: Set to junit to make it easier to report the results in the GitLab CI pipeline.
   • reporterOptions: Includes a path for the junit files and the keyword [hash] to preserve unique reports for each test file (otherwise, the file is overwritten). Skip the console output for the reporter and use the default output.
   • screenshotsFolder: Defines the screenshot storage folder (useful for troubleshooting).
   • e2e: References the local URL of the site and the plugins.

   After setting up Cypress and writing some tests (see below for examples), run the tests locally using npm. You will see the following output (truncated for brevity):

   dmundra in ~/workspace/accessibility/accessibility on branch main > npm run cypress-tests > cypress-tests > cypress run --browser chrome --headless ==================================================================================================== (Run Starting) ┌────────────────────────────────────────────────────────────────────────────────────────────────┐ │ Cypress: 11.2.0 │ │ Browser: Chrome 109 (headless) │ │ Node Version: v18.10.0 (/usr/local/Cellar/node/18.10.0/bin/node) │ │ Specs: 5 found (accordion.cy.js, home.cy.js, images.cy.js, menu.cy.js, search.cy.js) │ │ Searched: cypress/e2e/**/*.cy.{js,jsx,ts,tsx} │ └────────────────────────────────────────────────────────────────────────────────────────────────┘ ──────────────────────────────────────────────────────────────────────────────────────────────────── Running: search.cy.js (5 of 5) (Results) ┌────────────────────────────────────────────────────────────────────────────────────────────────┐ │ Tests: 1 │ │ Passing: 1 │ │ Failing: 0 │ │ Pending: 0 │ │ Skipped: 0 │ │ Screenshots: 0 │ │ Video: true │ │ Duration: 2 seconds │ │ Spec Ran: search.cy.js │ └────────────────────────────────────────────────────────────────────────────────────────────────┘ (Video) - Started processing: Compressing to 32 CRF - Finished processing: /Users/dmundra/workspace/accessibility/accessibility/cypres (0 seconds) s/results/search.cy.js.mp4 ... (Run Finished) Spec Tests Passing Failing Pending Skipped ┌────────────────────────────────────────────────────────────────────────────────────────────────┐ │ ✔ search.cy.js 00:02 1 1 - - - │ ...

   While Pa11y-ci can test interactivity, Cypress and its plugins can do much more. For a Jekyll site, I found that pa11y-ci did not catch any accessibility issues in mobile drop-down menu, dynamic search, or accordion features. I ran Cypress tests to interact with the elements (like performing searches, clicking menus, or clicking the accordion) and then checked if the results still passed accessibility tests. Here is the search example:

   describe('Search', () => { it('should be accessible', () => { cy.visit('/search') cy.get('#search-input').type('accessibility') cy.checkA11yWithMultipleViewPorts() }) })

   Here is a quick video of the running test.

   The above test visits the search page, types the word "accessibility" in the search field, and then checks the results for accessibility issues. I use the cypress-axe plugin to check accessibility issues with axe core, just like pa11y-ci. I have wrapped the cypress-axe functions in a function to test multiple window sizes and report on the issues in a table format.

   I also use the plugin cypress-real-events to interact with the site with a keyboard to check that the features are keyboard-accessible. Keyboard accessibility is a critical consideration (Operable principle of WCAG), and having an automated test that can confirm the features are keyboard accessible means that, maybe, there is one less test to run manually. You can see an example of the test here.

   Here is an example of what an error looks like:

   Running: a11y/anonymous_a11y.cy.js (1 of 36) cy.log(): Accessibility scanning: Home (/) cy.log(): 4 accessibility violations were detected ┌─────────┬────────────────────────┬────────────┬────────────────────────────────────────────────────────────────────────────────────┬───────┐ │ (index) │ id │ impact │ description │ nodes │ ├─────────┼────────────────────────┼────────────┼────────────────────────────────────────────────────────────────────────────────────┼───────┤ │ 0 │ 'image-alt' │ 'critical' │ 'Ensures elements have alternate text or a role of none or presentation' │ 4 │ │ 1 │ 'link-name' │ 'serious' │ 'Ensures links have discernible text' │ 3 │ │ 2 │ 'page-has-heading-one' │ 'moderate' │ 'Ensure that the page, or at least one of its frames contains a level-one heading' │ 1 │ │ 3 │ 'region' │ 'moderate' │ 'Ensures all page content is contained by landmarks' │ 2 │ └─────────┴────────────────────────┴────────────┴────────────────────────────────────────────────────────────────────────────────────┴───────┘

   Cypress logs provide a count of the number of errors at a given URL and then details on what the accessibility issue is, the impact, and its location.

   You can find additional details and examples in the Cypress folder.

   Use the GitLab CI/CD

   Now that you have pa11y-ci and Cypress running locally, see how to run automated accessibility tests in GitLab using CI/CD features. The GitLab repository is available here. Here is the .gitlab-ci.yml file setup:

   stages: - test cache: key: ${CI_COMMIT_REF_SLUG} paths: - node_modules/ - .npm/ - vendor/ruby default: image: ruby:2 before_script: - apt-get update - apt-get install -yq gconf-service libasound2 libatk1.0-0 libc6 libcairo2 libcups2 libdbus-1-3 libexpat1 libfontconfig1 libgcc1 libgconf-2-4 libgdk-pixbuf2.0-0 libglib2.0-0 libgtk-3-0 libnspr4 libpango-1.0-0 libpangocairo-1.0-0 libstdc++6 libx11-6 libx11-xcb1 libxcb1 libxcomposite1 libxcursor1 libxdamage1 libxext6 libxfixes3 libxi6 libxrandr2 libxrender1 libxss1 libxtst6 ca-certificates fonts-liberation libnss3 lsb-release xdg-utils wget libgbm1 xvfb - apt-get install -y nodejs npm - bundle install -j $(nproc) --path vendor/ruby - npm ci --cache .npm --prefer-offline - npm run start-detached pa11y-tests: stage: test script: - npm run pa11y-ci:sitemap artifacts: when: always reports: junit: - pa11y-report-junit.xml expire_in: 1 day cypress-tests: stage: test script: # Install chrome browser manually, taken from https://github.com/cypress-io/cypress-docker-images/blob/master/browsers/node16.14.2-slim-chrome100-ff99-edge/Dockerfile#L48 - wget --no-verbose -O /usr/src/google-chrome-stable_current_amd64.deb "http://dl.google.com/linux/chrome/deb/pool/main/g/google-chrome-stable/google-chrome-stable_105.0.5195.125-1_amd64.deb" - dpkg -i /usr/src/google-chrome-stable_current_amd64.deb - rm -f /usr/src/google-chrome-stable_current_amd64.deb - npm run cypress-tests artifacts: when: always paths: - cypress/results/ reports: junit: - cypress/results/*.xml expire_in: 1 day

   The file currently defines only one stage test and caches folders that store dependencies when installed. Then:

   1. Steps used by all stages:
      1. Use the Ruby version 2 image because it is compatible with the current Jekyll installation.
      2. I install many dependencies based on the documentation at running puppeteer on GitLab. Install node and npm to install site dependencies.
      3. Install the Jekyll Ruby dependencies.
      4. Install the Cypress and pa11y-ci dependencies via npm.
      5. Start the web server.
   2. Run the pa11y-ci to test the site and capture the output to a file.*
   3. Install the Chrome browser dependencies in cypress-tests using steps provided by Cypress in their Docker image configurations. Run the Cypress tests and capture the output to files.*

   * Capture the output of Cypress and pa11y-ci tests as junit XML files.

   Here is an example screenshot of the GitLab pipeline (taken from https://gitlab.com/civicactions/accessibility/-/pipelines/744894072):

   Image by:

   (Daniel Mundra, CC BY-SA 4.0)

   Here is an example of the test results in the same pipeline:

   Image by:

   (Daniel Mundra, CC BY-SA 4.0)

   Our favorite resources about open source Git cheat sheet Advanced Linux commands cheat sheet Open source alternatives Free online course: RHEL technical overview Check out more cheat sheets

   GitLab CI/CD automatically take junit XML files and outputs them in a clear format. Cypress tests provide the junit XML output as part of their features (see above). I created a custom reporter for pa11y-ci to output the format in junit (credit to macieklewkowicz/pa11y-reporter-junit).

   Note: GitLab version 12.8+ supports Pa11y accessibility tests (see https://docs.gitlab.com/ee/ci/testing/accessibility_testing.html for details). The above setup allows for customization of the pa11y-ci and also targeting of local URLs. I recommend using their options for live sites.

   Wrap up

   Using the above steps, you can provide accessibility testing for your site locally and in CI. This process helps you track and fix accessibility issues on your site and in the content. An important caveat about automated testing is that it only catches 57% of issues, so you definitely want to include manual testing with your accessibility testing.

   Further reading and examples
   • Automated accessibility testing: Leveraging GitHub Actions and pa11y-ci with axe.
   • Automated accessibility testing with Travis CI.
   • How to Automate Web Accessibility Testing.
   • Setting up Cypress with axe for accessibility - Tim Deschryver.
   • GitLab CI | Cypress Documentation.

   Thank you to Marissa Fox and Mike Gifford for your support, thoughts, and feedback.

   Follow along with this example of performing accessibility tests in GitLab with Pa11y and Cypress on a Jekyll website.

   Accessibility GitLab What to read next This work is licensed under a Creative Commons Attribution-Share Alike 4.0 International License. Register or Login to post a comment.