Open-source News

Get change alerts from any website with this open source tool Leigh Morresi Tue, 09/27/2022 - 03:00

The year was 2020, and news about COVID-19 came flooding in so quickly that everyone felt completely overwhelmed with similar news articles providing updates with varying degrees of accuracy.

But all I needed to know was when my official government guidelines changed. In the end, that's all that mattered to me.

Whether the concern is a pandemic or just the latest tech news, keeping ahead of changes in website content can be critical.

The changedetection.io project provides a simple yet highly capable, open source solution for website change detection and notification. It's easy to set up, and it can notify over 70 (and counting) different notification systems, such as Matrix, Mattermost, Nextcloud, Signal, Zulip, Home Assistant, email, and more. It also notifies proprietary applications like Discord, Office365, Reddit, Telegram, and many others.

But changedetection.io isn't just limited to watching web page content. You can also monitor XML and JSON feeds, and it will build an RSS feed of the websites that changed.

More great content Free online course: RHEL technical overview Learn advanced Linux commands Download cheat sheets Find an open source alternative Explore open source resources

Thanks to its built-in JSON simple storage system, there's no need to set up complicated databases to receive and store information. You can run it as a Docker image or install it with pip. The project has an extensive wiki help section, and most common questions are covered there.

For sites using complex JavaScript, you can connect your changedetection.io installation to a Chromium or Chrome browser with the built-in Playwright content fetcher.

Once running, access the application in your browser (http://localhost:5000, by default). You can set a password in the Settings section if your computer can be reached from an outside network.

Image by:

(Leigh Morresi, CC BY-SA 4.0)

Submit the URL of a page you want to monitor. There are several settings related to how the page is filtered. For example, you more than likely do not want to know when a company's stock price listed in their site footer has changed, but you may want to know when they post a news article to their blog.

Monitor a site

Imagine you want to add your favorite website, Opensource.com, to be monitored. You only want to know when the main call-out article contains the word "python" and you want to be notified over Matrix.

To do this, begin with the visual-selector tool. (This requires the playwright browser interface to be connected.)

Image by:

(Leigh Morresi, CC BY-SA 4.0)

The visual-selector tool automatically calculates the best Xpath or CSS filter to target the content. Otherwise, you would get a lot of noise from the daily page updates.

Next, visit the Filters & Triggers tab.

Image by:

(Leigh Morresi, CC BY-SA 4.0)

In CSS/JSON/XPATH Filter field (the blue circle), you can see the automatically generated CSS filter from the previous step.

There are several useful filters available, such as Remove elements (good for removing noisy elements), Ignore text, Trigger/wait for text, and Block change-detection if text matches (used for waiting for some text to disappear, like "sold out").

In Trigger/wait for text (the red circle), type in the keyword you want to monitor for. (That's "python" in this example.)

The final step is in the Notifications tab, where you configure where you want to receive your notification. Below I added a Matrix room as the notification target, using the Matrix API.

Image by:

(Leigh Morresi, CC BY-SA 4.0)

The notification URL is in the format of matrixs://username:password@matrix.org/#/room/#room-name:matrix.org

However, t2Bot format is also supported. Here are more Matrix notification options.

And that's it! Now you'll receive a message over Matrix whenever the content changes.

There's more

There's so much more to changedetection.io. If you prefer calling a custom JSON API, you don't have to use an API for notifications (use jsons://). You can also create a custom HTTP request (POST and GET), execute JavaScript before checking (perhaps to pre-fill a username and password login field), and many more interesting features, with more to come.

Stop browsing the web and start watching the web instead!

Use changedetection.io to get alerts when a website makes changes or updates.

Tools Web development What to read next This work is licensed under a Creative Commons Attribution-Share Alike 4.0 International License. Register or Login to post a comment.

8 GUI Package Managers for Arch Linux  It's FOSS

The post Top 5 Open Source Plugins for ONLYOFFICE Docs first appeared on Tecmint: Linux Howtos, Tutorials & Guides .

If you think that office software is exclusively designed for writing texts, making calculations in spreadsheets, and creating informative presentations, you are wrong. Some office suites are capable of doing much more things than

The post Top 5 Open Source Plugins for ONLYOFFICE Docs first appeared on Tecmint: Linux Howtos, Tutorials & Guides.

What is a Loop device in Linux?  It's FOSS

The Real Reason Why The Linux Logo Is A Penguin  SlashGear

This morning I called attention to some pending work around a 20 year old chipset workaround in the Linux kernel had been hurting modern AMD systems by erroneously still applying the change to modern hardware. Fortunately, that patch has now been picked up by Linus Torvalds in time for the Linux 6.0 kernel expected for its stable debut next weekend...

Lilbits: Pixel Watch leaked (again), improving Linux performance on modern AMD chips, and Linux for the Samsung Galaxy Tab 2  Liliputing

1. 20-year-old Linux workaround is still slowing down AMD systems  Ars Technica
2. A 20 Year Old Chipset Workaround Has Been Hurting Modern AMD Linux Systems  Phoronix
3. Old Linux patch slows down AMD Ryzen and Epyc  Aviation Analysis Wing
4. Ancient Linux patch slows down AMD Ryzen and Epyc  California18
5. View Full Coverage on Google News

20-year-old Linux workaround is still slowing down AMD systems  Ars Technica

Linux 6.0 should be with us very soon  TechRadar

How to Install and Use Thunderbird on Linux  MUO - MakeUseOf

How to install KVM on Ubuntu 20.04 LTS server  Linux Shout

How to install KVM on Ubuntu 20.04 LTS server  Linux Shout

Linux Fu: Atomic Power  Hackaday

Earlier this year it was shared that SUSE/openSUSE is developing the Adaptable Linux Platform "ALP" for next-generation SUSE Linux Enterprise. Coming out later this week is the first public preview of openSUSE ALP...

openSUSE ALP Prototype "Les Droites" Releasing This Week  Phoronix

How to set a static IP in Rocky Linux  TechRepublic

RemotePC Adds Linux Support - socaltech.com  socalTech.com

Why Debian Might Include Non-Free Firmware in Future Releases  MUO - MakeUseOf

Why Debian Might Include Non-Free Firmware in Future Releases  MUO - MakeUseOf