# Using SystemTap

short intro, contents of chapter

This chapter documents how to install SystemTap in the system and explains how to use the `stap` utility to run SystemTap scripts.

## Running SystemTap Scripts

- basic commands (e.g. stap), useful options per command (e.g. stap -vv), tool references (man pages, related kernel-doc), references within book (i.e. errors chapter) - running systemtap scripts

SystemTap is distributed with a number of command line tools that allow you to monitor the activities of the system. The `stap` command reads probing instructions from a SystemTap script, translates these instructions into C code, builds a kernel module, and loads it into the running Linux kernel. The `staprun` command runs SystemTap instrumentation, that is, a kernel module built from SystemTap scripts during a cross-instrumentation.

Running `stap` and `staprun` requires elevated privileges to the system. Because not all users can be granted root access just to run SystemTap, you can allow a non-privileged user to run SystemTap instrumentation on their machine by adding them to one of the following user groups:

stapdev

  Members of this group can use the `stap` command to run SystemTap scripts, or `staprun` to run SystemTap instrumentation modules.

  Running the `stap` command involves compiling SystemTap scripts into kernel modules and loading them into the kernel. This operation requires elevated privileges to the system, which are granted to stapdev members. Unfortunately, such privileges also grant effective root access to stapdev members. As a consequence, only grant stapdev group membership to users whom you can trust with root access.

Usage stapdev
stapdev Usage
stapusr

  Members of this group can only use the `staprun` command to run SystemTap instrumentation modules. In addition, they can only run modules from the `/lib/modules/kernel_version/systemtap/` directory. Note that this directory must be owned only by the root user, and must only be writable by the root user.

Usage stapusr
stapusr Usage

The `stap` command reads a SystemTap script either from a file, or from standard input. To tell `stap` to read a SystemTap script from a file, specify the file name on the command line:

    stap file_name

To instruct `stap` to read a SystemTap script from standard input, use the `-` switch instead of the file name. Note that any command-line options you wish to use must be inserted before the `-` switch. For example, to make the output of the `stap` command more verbose, type:

    echo "probe timer.s(1) {exit()}" | stap -v -

Below is a list of commonly used `stap` options:

-v

  Makes the output of the SystemTap session more verbose. You can repeat this option multiple times to provide more details on the script's execution, for example:

      stap -vvv script.stp

  This option is particularly useful if you encounter any errors in running the script. For more information about common SystemTap script errors, refer to .

-o file_name

  Sends the standard output to a file named file_name.

-S size,count

  Limits the maximum size of output files to size megabytes and the maximum number of stored files to count. This option implements logrotate operations for SystemTap and the resulting file names have a sequence number suffix.

-x process_id

  Sets the SystemTap handler function `target()` to the specified process ID. For more information about `target()`, refer to .

-c 'command'

  Sets the SystemTap handler function `target()` to the specified command and runs the SystemTap instrumentation for the duration of this command. For more information about `target()`, refer to .

-e 'script'

  Uses script rather than a file as input for the SystemTap translator.

-F

  Uses SystemTap's flight recorder mode and makes the script a background process. For more information about flight recorder mode, refer to .

stap options
any other useful options worth noting here for beginners?

For more information about the `stap` command, refer to the stap 1 man page. For more information about the `staprun` command and cross-instrumentation, refer to or the staprun 8 man page.

### SystemTap Flight Recorder Mode

SystemTap's flight recorder mode allows you to run a SystemTap script for long periods of time and just focus on recent output. The flight recorder mode limits the amount of output generated.

There are two variations of the flight recorder mode: in-memory and file mode. In both cases, the SystemTap script runs as a background process.

#### In-memory Flight Recorder

When flight recorder mode is used without a file name, SystemTap uses a buffer in kernel memory to store the output of the script. Once the SystemTap instrumentation module is loaded and the probes start running, the instrumentation detaches and is put in the background. When the interesting event occurs, you can reattach to the instrumentation to see the recent output in the memory buffer and any continuing output.

To run a SystemTap script by using the flight recorder in-memory mode, run the `stap` command with the `-F` command line option:

    stap -F iotime.stp

Once the script starts, `stap` prints a message similar to the following to provide you with the command to reconnect to the running script:

    Disconnecting from systemtap module.
    To reconnect, type "staprun -A stap_5dd0073edcb1f13f7565d8c343063e68_19556"

When the interesting event occurs, run the following command to connect to the currently running script, output the recent data in the memory buffer, and get continuing output:

    staprun -A stap_5dd0073edcb1f13f7565d8c343063e68_19556

By default, the kernel buffer is 1MB in size. You can increase this value by using the `-s` option with the size in megabytes (rounded up to the next power over 2) for the buffer. For example, `-s2` on the SystemTap command line would specify 2MB for the buffer.

flight recorder mode in-memory mode

#### File Flight Recorder

The flight recorder mode can also store data to files. You can control the number and size of the files kept by using the `-S` option followed by two numerical arguments separated by a comma: the first argument is the maximum size in megabytes for the each output file, the second argument is the number of recent files to keep. To specify the file name, use the `-o` option followed by the name. SystemTap automatically adds a number suffix to the file name to indicate the order of the files.

The following command starts SystemTap in file flight recorder mode with the output going to files named `/tmp/pfaults.log.[0-9]+`, each file 1MB or smaller, and keeping latest two files:

    stap -F -o /tmp/pfaults.log -S 1,2  pfaults.stp

The command prints the process ID to standard output. Sending a SIGTERM to the process terminates the SystemTap script and stops the data collection. For example, if the previous command listed 7590 as the process ID, the following command would stop the SystemTap script:

    kill -s SIGTERM 7590

In this example, only the most recent two files generated by the script are kept: SystemTap automatically removes older files. As a result, the `ls -sh /tmp/pfaults.log.*` command lists two files:

    1020K /tmp/pfaults.log.5    44K /tmp/pfaults.log.6

To examine the latest data, read the file with the highest number, in this case `/tmp/pfaults.log.6`.

flight recorder mode file mode
flight recorder mode
Usage running SystemTap scripts
running SystemTap scripts Usage
Usage stap
stap Usage
Usage staprun
staprun Usage
SystemTap scripts, how to run
Usage standard input, running scripts from
standard input, running scripts from Usage
running scripts from standard input
Usage options, stap
options, stap Usage
Using SystemTap