Open-source News

Passing the PyTorch  Protocol

How I recovered my Linux system using a Live USB device David Both Mon, 09/12/2022 - 03:00

I have a dozen or so physical computers in my home lab and even more VMs. I use most of these systems for testing and experimentation. I frequently write about using automation to make sysadmin tasks easier. I have also written in multiple places that I learn more from my own mistakes than I do in almost any other way.

I have learned a lot during the last couple of weeks.

I created a major problem for myself. Having been a sysadmin for years and written hundreds of articles and five books about Linux, I really should have known better. Then again, we all make mistakes, which is an important lesson: You're never too experienced to make a mistake.

I'm not going to discuss the details of my error. It's enough to tell you that it was a mistake and that I should have put a lot more thought into what I was doing before I did it. Besides, the details aren't really the point. Experience can't save you from every mistake you're going to make, but it can help you in recovery. And that's literally what this article is about: Using a Live USB distribution to boot and enter a recovery mode.

The problem

First, I created the problem, which was essentially a bad configuration for the /etc/default/grub file. Next, I used Ansible to distribute the misconfigured file to all my physical computers and run grub2-mkconfig. All 12 of them. Really, really fast.

All but two failed to boot. They crashed during the very early stages of Linux startup with various errors indicating that the /root filesystem could not be located.

I could use the root password to get into "maintenance" mode, but without /root mounted, it was impossible to access even the simplest tools. Booting directly to the recovery kernel did not work either. The systems were truly broken.

More Linux resources Linux commands cheat sheet Advanced Linux commands cheat sheet Free online course: RHEL technical overview Linux networking cheat sheet SELinux cheat sheet Linux common commands cheat sheet What are Linux containers? Our latest Linux articles Recovery mode with Fedora

The only way to resolve this problem was to find a way to get into recovery mode. When all else fails, Fedora provides a really cool tool: The same Live USB thumb drive used to install new instances of Fedora.

After setting the BIOS to boot from the Live USB device, I booted into the Fedora 36 Xfce live user desktop. I opened two terminal sessions next to each other on the desktop and switched to root privilege in both.

I ran lsblk in one for reference. I used the results to identify the / root partition and the boot and efi partitions. I used one of my VMs, as seen below. There is no efi partition in this case because this VM does not use UEFI.

# lsblk
NAME          MAJ:MIN RM  SIZE RO TYPE MOUNTPOINTS
loop0           7:0    0  1.5G  1 loop
loop1           7:1    0    6G  1 loop
├─live-rw     253:0    0    6G  0 dm   /
└─live-base   253:1    0    6G  1 dm  
loop2           7:2    0   32G  0 loop
└─live-rw     253:0    0    6G  0 dm   /
sda             8:0    0  120G  0 disk
├─sda1          8:1    0    1G  0 part
└─sda2          8:2    0  119G  0 part
  ├─vg01-swap 253:2    0    4G  0 lvm  
  ├─vg01-tmp  253:3    0   10G  0 lvm  
  ├─vg01-var  253:4    0   20G  0 lvm  
  ├─vg01-home 253:5    0    5G  0 lvm  
  ├─vg01-usr  253:6    0   20G  0 lvm  
  └─vg01-root 253:7    0    5G  0 lvm  
sr0            11:0    1  1.6G  0 rom  /run/initramfs/live
zram0         252:0    0    8G  0 disk [SWAP]

The /dev/sda1 partition is easily identifiable as /boot, and the root partition is pretty obvious as well.

In the other terminal session, I performed a series of steps to recover my systems. The specific volume group names and device partitions such as /dev/sda1 will differ for your systems. The commands shown here are specific to my situation.

The objective is to boot and get through startup using the Live USB, then mount only the necessary filesystems in an image directory and run the chroot command to run Linux in the chrooted image directory. This approach bypasses the damaged GRUB (or other) configuration files. However, it provides a complete running system with all the original filesystems mounted for recovery, both as the source of the tools required and the target of the changes to be made.

Here are the steps and related commands:

1. Create the directory /mnt/sysimage to provide a location for the chroot directory.

2. Mount the root partition on /mnt/sysimage:

# mount /dev/mapper/vg01-root /mnt/sysimage

3. Make /mnt/sysimage your working directory:

# cd /mnt/sysimage

4. Mount the /boot and /boot/efi filesystems.

5. Mount the other main filesystems. Filesystems like /home and /tmp are not needed for this procedure:

# mount /dev/mapper/vg01-usr usr

# mount /dev/mapper/vg01-var var

6. Mount important but already mounted filesystems that must be shared between the chrooted system and the original Live system, which is still out there and running:

# mount --bind /sys sys

# mount --bind /proc proc

7. Be sure to do the /dev directory last, or the other filesystems won't mount:

# mount --bind /dev dev

8. Chroot the system image:

# chroot /mnt/sysimage

The system is now ready for whatever you need to do to recover it to a working state. However, one time I was able to run my server for several days in this state until I could research and test real fixes. I don't really recommend that, but it can be an option in a dire emergency when things just need to get up and running–now!

The solution

The fix was easy once I got each system into recovery mode. Because my systems now worked just as if they had booted successfully, I simply made the necessary changes to /etc/default/grub and /etc/fstab and ran the grub2-mkconfig > boot/grub2/grub.cfg command. I used the exit command to exit from chroot and then rebooted the host.

Of course, I could not automate the recovery from my mishap. I had to perform this entire process manually on each host—a fitting bit of karmic retribution for using automation to quickly and easily propagate my own errors.

Lessons learned

Despite their usefulness, I used to hate the "Lessons Learned" sessions we would have at some of my sysadmin jobs, but it does appear that I need to remind myself of a few things. So here are my "Lessons Learned" from this self-inflicted fiasco.

First, the ten systems that failed to boot used a different volume group naming scheme, and my new GRUB configuration failed to consider that. I just ignored the fact that they might possibly be different.

• Think it through completely.
• Not all systems are alike.
• Test everything.
• Verify everything.
• Never make assumptions.

Everything now works fine. Hopefully, I am a little bit smarter, too.

The Fedora Live USB distribution provides an effective solution to boot and enter a recovery mode.

Image by:

Photo by Markus Winkler on Unsplash

Linux Sysadmin What to read next This work is licensed under a Creative Commons Attribution-Share Alike 4.0 International License. Register or Login to post a comment.

Control your home automation remotely with Raspberry Pi and Traefik Hub Nicolas Mengin Mon, 09/12/2022 - 03:00

Over the years, several friends have asked me for tips on managing their home networks. In most cases, they are setting up home automation and want to access their services from the outside.

Every time I helped them, each made the same comment: "Are you kidding? It cannot be so complicated to publish one simple application!"

Publishing applications that don't put your network or cluster at risk can indeed be quite complicated. When we started working on Traefik Hub—the latest product by Traefik Labs—I knew it would be a game-changer for publishing applications.

This article demonstrates the complexity of publishing services and how Traefik Hub makes your life a lot easier. I use the example of setting up a server to control your home automation remotely with Traefik Hub running on a Raspberry Pi.

The challenge

Setting up a server to manage your home automation is nice, but being able to control it remotely from anywhere in the world using only your mobile phone—that's even nicer!

However, with great power comes great responsibility. If you want access to your local network from the outside, you'd better ensure it's resilient and that you are the only one with access.

First, I'll look at the steps you would normally take to achieve that.

Reach your Home Assistant remotely any time

Home Assistant is a well-known solution to manage home automation devices. It's an open source project written in Python. It allows you to have Home Automation with a local installation: No data on the cloud and everything is kept private. I recommend this excellent article to help you install Home Assistant on your Raspberry Pi using Docker.

To reach your Home Assistant from the outside, you must expose your Raspberry Pi to the internet. To do so, you have to:

• Know the public IP of your router.
• Redirect the traffic from your router to your Raspberry Pi.

Note: Most internet providers assign dynamic public IPs—each time your router restarts, your IP will probably change. To build a resilient system, you would also need a dynamic domain.

More on Raspberry Pi What is Raspberry Pi? eBook: Guide to Raspberry Pi Getting started with Raspberry Pi cheat sheet eBook: Running Kubernetes on your Raspberry Pi Whitepaper: Data-intensive intelligent applications in a hybrid cloud blueprint Understanding edge computing Our latest on Raspberry Pi Encryption matters

When you communicate with your server, you send sensitive data, such as your username and password. You must verify and encrypt communication using a TLS certificate to avoid this data being stolen.

This requires:

• A reverse proxy to do the TLS termination.
• A piece of infrastructure to generate and renew your Let's Encrypt certificates.

TL;DR

To sum it up, after installing Home Assistant on your Raspberry Pi, you need to:

• Get your router public IP.
• Create a port forward to your Raspberry Pi.
• Buy a domain name.
• Create a dynamic domain.
• Install a reverse proxy and configure it for encrypted access using a TLS certificate.

Now, imagine if you could skip all of the steps above and publish your services in a few clicks!

Traefik Hub to the rescue

Traefik Hub is a cloud-native networking SaaS platform that allows users to publish their services at the edge quickly. Using Traefik Hub, you can publish your Home Assistant application in a few clicks.

Remember the challenges I mentioned earlier? Scratch that. Once you have Home Assistant installed on your Raspberry Pi, all you have to do is connect your Raspberry Pi to Traefik Hub. Traefik Hub handles everything for you, including:

• Making your service reachable from the internet.
• Providing a dynamic domain (for free).
• Encrypting communication with a TLS certificate and an Access Control Policy.

And now that I have introduced Traefik Hub, I'll get down to the business of configuring it.

Step 1: Connect your Raspberry Pi to Traefik Hub

First, head over to Traefik Hub and sign up for a free account. You can sign up via Google or GitHub.

You need to add a new agent to connect your Raspberry Pi to Traefik Hub.

Image by:

(Nicolas Mengin, CC BY-SA 4.0)

Traefik Hub provides several snippets that allow you to start from scratch.

Since the Home Assistant setup is a bit complex, you can get your token from the Hub UI and use the script below for this example. The token allows you to connect your agent to Traefik Hub. Traefik Hub then attaches this agent to your account, and you can start publishing your services.

Here's the script:

version: '3'

networks:
  traefik: {}

services:
  homeassistant:
    container_name: homeassistant
    image: "ghcr.io/home-assistant/home-assistant:stable"
    volumes:
                  # /!\\ Mount the custom configuration file described below /!\\
      - ./configuration.yaml:/config/configuration.yaml
      - /etc/localtime:/etc/localtime:ro
    restart: unless-stopped
    privileged: true
                networks:
      - traefik
                ports:
                        - 8123
 
        # Start the agent with the latest version
  hub-agent:
    image: ghcr.io/traefik/hub-agent-traefik:v0.7.2
    restart: "on-failure"
    container_name: hub-agent
    networks:
      - traefik
                command:
      - run
      - --hub.token= # Set your token here
      - --auth-server.advertise-url=http://hub-agent
      - --traefik.host=traefik
      - --traefik.tls.insecure=true
      - --hub.url=https://platform.hub.traefik.io/agent
      - --hub.ui.url=https://hub.traefik.io
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    depends_on:
      - traefik

  # Start Traefik with the latest version
  traefik:
    image: traefik:v2.8
    container_name: traefik
    networks:
      - traefik
    command:
      # Enable Hub communication (open the port 9900 and 9901 by default)
      - --experimental.hub=true
      - --hub.tls.insecure=true
      - --metrics.prometheus.addrouterslabels=true # ./configuration.yaml to mount on your home assistant container
# in /config/configuration.yaml

# These modifications are required by home assistant to be exposed using
# a third party software such as the Traefik Hub agent

# Loads default set of integrations. Do not remove.
default_config:

http:
  ip_ban_enabled: true
  login_attempts_threshold: 5
  use_x_forwarded_for: true
  trusted_proxies:
    - 192.168.1.0/24
    - 172.18.0.0/24
    - 127.0.0.1
    - ::1
    - fe80::/64
    - fe00::/64
    - fd00::/64

# Text to speech
tts:
  - platform: google_translateStep 2: Publish your service

Once you have installed the agent on your Raspberry Pi, Traefik Hub discovers every service running on your cluster so you can publish them without digging into your configuration files.

Image by:

(Nicolas Mengin, CC BY-SA 4.0)

Select your Home Assistant service, and click the Save and Publish button to publish it.

Image by:

(Nicolas Mengin, CC BY-SA 4.0)

And now let the magic happen!

Image by:

(Nicolas Mengin, CC BY-SA 4.0)

Once Hub notifies you that your service has been published, you can reach it from the internet using the domain Traefik Hub has generated. The connection is verifiable and encrypted, and your Home Assistant remains reachable even if your public IP changes.

Image by:

(Nicolas Mengin, CC BY-SA 4.0)

Behind the scenes

Your application is published. Next, I'll discuss a few things Traefik Hub takes care of behind the scenes to offer a seamless experience and some handy configuration options.

Traefik instance

When you installed the Traefik Hub Agent, you certainly noticed that it comes with a Traefik Proxy instance.

Traefik Hub creates a tunnel between its platform and the agent you installed on your Raspberry Pi to publish your service on the internet. The agent passes through the requests to open source Traefik Proxy, which is used as an Ingress Controller. Traefik Hub manages both the domain and the TLS certificate, and it shares the certificate with your Traefik instance to allow it to do the TLS termination.

Image by:

(Nicolas Mengin, CC BY-SA 4.0)

Access Control Policy

Another point to remember is that a deployed Home Assistant application comes with its own login system. However, when you publish a service using Traefik Hub, you can restrict access further by using an Access Control Policy such as JWT and Basic Auth.

Image by:

(Nicolas Mengin, CC BY-SA 4.0)

Kubernetes

If you are a Kubernetes user, you can also publish your Kubernetes Services. Traefik Hub can manage Kubernetes Services through the UI or a dedicated CRD.

Manage and monitor

Traefik Hub also provides a web UI that allows you to manage and monitor the exposition of services.

Image by:

(Nicolas Mengin, CC BY-SA 4.0)

Wrap up

This article started by going through a long and complicated list of tasks that come with publishing an application over an encrypted and verifiable connection. Setting up home automation is an excellent example of that level of complexity. But when things seem impossibly hard, there's always an easier alternative! Traefik Hub makes your life simpler by taking over most of the mundane operations tasks, saving time, and allowing developers to focus on building applications.

Now you can turn the lights on in your house, even if you're on the other side of the world!

If you're interested in learning more about Traefik Hub, check out this getting started article. Traefik Hub is currently in Beta, so please don't hesitate to give it a try and provide feedback—you can do so directly in the UI.

I hope you found this article helpful, and thank you for reading!

Employ open source networking to facilitate cloud-native apps.

Image by:

27707 via Pixabay, CC0. Modified by Jen Wike Huger.

Home automation Cloud Edge computing Raspberry Pi What to read next Why choose open source for your home automation project This work is licensed under a Creative Commons Attribution-Share Alike 4.0 International License. Register or Login to post a comment.

How To Play Epic Games On Linux  IWMBuzz

The post How to Install Universal Media Server to Stream Media to Any Devices first appeared on Tecmint: Linux Howtos, Tutorials & Guides .

Universal Media Server (UMS) is a cross-platform and free DLNA-compliant, HTTP(s) PnP Media server, which provides a number of capabilities such as sharing multimedia files such as images, videos, and audio between modern devices

The post How to Install Universal Media Server to Stream Media to Any Devices first appeared on Tecmint: Linux Howtos, Tutorials & Guides.

The past few years have transformed how we think about work. Our "new normal" presents unique challenges for companies thinking seriously about the future of work. At Red Hat, we’ve come to believe that the future of work is driven by one concept: flexibility.

Retbleed fix slugs Linux VM performance by up to 70 percent  The Register

Linus Torvalds has just announced Linux 6.0-rc5 as the latest test release of Linux 6.0 that is working its way toward a stable release in early October...

Linux 6.0-rc5 Released After A Calm Week Of Kernel Development  Phoronix

How to Self-Host Bitwarden Password Manager on Raspberry Pi Zero  MUO - MakeUseOf

Disclosed back in March was the Spectre-BHB / Branch History Injection (BHI) speculative execution vulnerability that on the Arm side affected CPUs from the likes of the Spectre-A15 through A78 series as well as the likes of the X1, X2, and A710, plus the Neoverse E1 / N1 / N2 / V1 CPUs. Now for Linux 6.1, a command-line option is being added for ARM64 to be able to disable the Spectre-BHB mitigation due to the "great impact" to performance...

Fly-Pie: A Mouse-Centric GNOME Shell Menu Launcher for Linux  Linux News, Apps and Tutorials

Linux 6.1 Adding Option To Disable Spectre-BHB On Arm Due To "Great Impact" On Performance  Phoronix

Papirus Icon Set’s September Update Adds 38 New Icons  OMG! Ubuntu!

For those recently picking up an ASUS laptop powered by AMD Ryzen Mobile 6000 series "Rembrandt" SoCs or considering such a device, AMD has prepared a set of fixes for the suspend-to-idle support...

AMD Prepares s2idle Fixes For AMD Ryzen 6000 Series Powered ASUS Laptops  Phoronix

The Device Tree (DT) files needed by the Linux kernel for Apple Macs powered by the M1 Pro, Max, and Ultra SoCs have been submitted on the kernel mailing list for review and working their way towards upstream...

Apple M1 Pro/Max/Ultra Device Trees Under Review For Linux  Phoronix

As part of the multi-year effort to overhaul the Linux kernel's printk() code there has been much work in recent months around threaded console printing so each registered console would have a kernel thread and console printing would be decoupled from the printk() callers. That work was aimed for Linux 5.19 but then reverted due to troubles. There is now a new implementation in the works...

Linux Sees A New Attempt At Threaded Console Printing  Phoronix