Open-source News

ARM + uboot + linux kernel debugging using debuggers (JTAG)  All About Circuits

Sent in as a "fix" this week for the Linux 5.18 kernel and to be found in tomorrow's 5.18-rc5 release is supporting sensor readings with the Gigabyte-WMI driver for the Gigabyte B660 GAMING X DDR4 motherboard...

Best Open Source Lightweight Web Browsers for Linux  BollyInside

Unity 7.6 Released for Public Testing as the First Major Update to Unity7 in 6 Years  9to5Linux

3 Ways to Test SSD Speed & Performance  MUO - MakeUseOf

8 Photoshop Alternatives for Creating 3D Elements  MUO - MakeUseOf

Build your Linux skills with this certification training bundle deal  BleepingComputer

How To Check Disk Usage in Linux  Tom's Hardware

Xfce’s Apps Update for April 2022: New Releases of Mousepad, Xfce Terminal, and Thunar  9to5Linux

AMD continues working on their open-source Linux driver support for next-gen GPUs... The latest patches posted on Friday are for "GFX11", pointing to the major new graphics IP version with RDNA3 graphics processors due out later this year...

As part of Mesa preparing to drop its old GLSL to TGSI code path and routing more Mesa drivers through using the NIR intermediate representation, the Nouveau Gallium3D driver code has switched to using this modern IR by default...

Fedora Linux is looking at tightening up its cryptographic policies with next year's Fedora 38/39 releases but for Fedora 37 later this year they will likely begin warning users around the planned changes...

Loongson engineers continue working on aiming to upstream their LoongArch CPU architecture support in the Linux kernel...

KDE this week saw more components converted to using Qt Quick, among other features and improvements to the KDE Plasma desktop...

Hands On With GNOME's New Terminal for Linux Users  It's FOSS

How to Download Youtube videos on Ubuntu 22.04 using GUI APP  Linux Shout

Parsing data with strtok in C Jim Hall Sat, 04/30/2022 - 03:00 Register or Login to like Register or Login to like

Some programs can just process an entire file at once, and other programs need to examine the file line-by-line. In the latter case, you likely need to parse data in each line. Fortunately, the C programming language has a standard C library function to do just that.

The strtok function breaks up a line of data according to "delimiters" that divide each field. It provides a streamlined way to parse data from an input string.

Reading the first token

Suppose your program needs to read a data file, where each line is separated into different fields with a semicolon. For example, one line from the data file might look like this:

102*103;K1.2;K0.5

In this example, store that in a string variable. You might have read this string into memory using any number of methods. Here's the line of code:

char string[] = "102*103;K1.2;K0.5";

Once you have the line in a string, you can use strtok to pull out "tokens." Each token is part of the string, up to the next delimiter. The basic call to strtok looks like this:

#include
char *strtok(char *string, const char *delim);

The first call to strtok reads the string, adds a null (\0) character at the first delimiter, then returns a pointer to the first token. If the string is already empty, strtok returns NULL.

#include
#include

int
main()
{
  char string[] = "102*103;K1.2;K0.5";
  char *token;

  token = strtok(string, ";");

  if (token == NULL) {
    puts("empty string!");
    return 1;
  }

  puts(token);

  return 0;
}

This sample program pulls off the first token in the string, prints it, and exits. If you compile this program and run it, you should see this output:

102*103

102*103 is the first part of the input string, up to the first semicolon. That's the first token in the string.

Note that calling strtok modifies the string you are examining. If you want the original string preserved, make a copy before using strtok.

Programming and development Red Hat Developers Blog Programming cheat sheets Try for free: Red Hat Learning Subscription eBook: An introduction to programming with Bash Bash shell scripting cheat sheet eBook: Modernizing Enterprise Java Reading the rest of the string as tokens

Separating the rest of the string into tokens requires calling strtok multiple times until all tokens are read. After parsing the first token with strtok, any further calls to strtok must use NULL in place of the string variable. The NULL allows strtok to use an internal pointer to the next position in the string.

Modify the sample program to read the rest of the string as tokens. Use a while loop to call strtok multiple times until you get NULL.

#include
#include

int
main()
{
  char string[] = "102*103;K1.2;K0.5";
  char *token;

  token = strtok(string, ";");

  if (token == NULL) {
    puts("empty string!");
    return 1;
  }

  while (token) {
    /* print the token */
    puts(token);

    /* parse the same string again */
    token = strtok(NULL, ";");
  }

  return 0;
}

By adding the while loop, you can parse the rest of the string, one token at a time. If you compile and run this sample program, you should see each token printed on a separate line, like this:

102*103
K1.2
K0.5Multiple delimiters in the input string

Using strtok provides a quick and easy way to break up a string into just the parts you're looking for. You can use strtok to parse all kinds of data, from plain text files to complex data. However, be careful that multiple delimiters next to each other are the same as one delimiter.

For example, if you were reading CSV data (comma-separated values, such as data from a spreadsheet), you might expect a list of four numbers to look like this:

1,2,3,4

But if the third "column" in the data was empty, the CSV might instead look like this:

1,2,,4

This is where you need to be careful with strtok. With strtok, multiple delimiters next to each other are the same as a single delimiter. You can see this by modifying the sample program to call strtok with a comma delimiter:

#include
#include

int
main()
{
  char string[] = "1,2,,4";
  char *token;

  token = strtok(string, ",");

  if (token == NULL) {
    puts("empty string!");
    return 1;
  }

  while (token) {
    puts(token);
    token = strtok(NULL, ",");
  }

  return 0;
}

If you compile and run this new program, you'll see strtok interprets the ,, as a single comma and parses the data as three numbers:

1
2
4

Knowing this limitation in strtok can save you hours of debugging.

Using multiple delimiters in strtok

You might wonder why the strtok function uses a string for the delimiter instead of a single character. That's because strtok can look for different delimiters in the string. For example, a string of text might have spaces and tabs between each word. In this case, you would use each of those "whitespace" characters as delimiters:

#include
#include

int
main()
{
  char string[] = "  hello \t world";
  char *token;

  token = strtok(string, " \t");

  if (token == NULL) {
    puts("empty string");
    return 1;
  }

  while (token) {
    puts(token);
    token = strtok(NULL, " \t");
  }

  return 0;
}

Each call to strtok uses both a space and tab character as the delimiter string, allowing strtok to parse the line correctly into two tokens.

Wrap up

The strtok function is a handy way to read and interpret data from strings. Use it in your next project to simplify how you read data into your program.

The strtok function is a handy way to read and interpret data from strings. Use it in your next project to simplify how you read data into your program.

Image by:

kris krüg

Programming What to read next This work is licensed under a Creative Commons Attribution-Share Alike 4.0 International License. Register or Login to post a comment.

Install Avidemux on Ubuntu 22.04 LTS Jammy Linux  Linux Shout

Rust-based Redox OS 0.7.0 Arrives with Enhanced Hardware Support  It's FOSS News

5 basic network commands for Linux and Windows  TechTarget