Red Hat Training

A Red Hat training course is available for Red Hat Enterprise Linux

3.5. Array Operations in SystemTap

This section enumerates some of the most commonly used array operations in SystemTap.

3.5.1. Assigning an Associated Value

Use = to set an associated value to indexed unique pairs, as in: 
array_name[index_expression] = value
Example 3.11, “Basic Array Statements” shows a very basic example of how to set an explicit associated value to a unique key. You can also use a handler function as both your index_expression and value. For example, you can use arrays to set a timestamp as the associated value to a process name (which you wish to use as your unique key), as in:

Example 3.12. Associating Timestamps to Process Names

arr[tid()] = gettimeofday_s()
Whenever an event invokes the statement in Example 3.12, “Associating Timestamps to Process Names”, SystemTap returns the appropriate tid() value (that is, the ID of a thread, which is then used as the unique key). At the same time, SystemTap also uses the function gettimeofday_s() to set the corresponding timestamp as the associated value to the unique key defined by the function tid(). This creates an array composed of key pairs containing thread IDs and timestamps.
In this same example, if tid() returns a value that is already defined in the array arr, the operator will discard the original associated value to it, and replace it with the current timestamp from gettimeofday_s().

3.5.2. Reading Values From Arrays

You can also read values from an array the same way you would read the value of a variable. To do so, include the array_name[index_expression] statement as an element in a mathematical expression. For example:

Example 3.13. Using Array Values in Simple Computations

delta = gettimeofday_s() - arr[tid()]
This example assumes that the array arr was built using the construct in Example 3.12, “Associating Timestamps to Process Names” (from Section 3.5.1, “Assigning an Associated Value”). This sets a timestamp that will serve as a reference point, to be used in computing for delta.
The construct in Example 3.13, “Using Array Values in Simple Computations” computes a value for the variable delta by subtracting the associated value of the key tid() from the current gettimeofday_s(). The construct does this by reading the value of tid() from the array. This particular construct is useful for determining the time between two events, such as the start and completion of a read operation.

Note

If the index_expression cannot find the unique key, it returns a value of 0 (for numerical operations, such as Example 3.13, “Using Array Values in Simple Computations”) or a null (empty) string value (for string operations) by default.

3.5.3. Incrementing Associated Values

Use ++ to increment the associated value of a unique key in an array, as in: 
array_name[index_expression] ++
Again, you can also use a handler function for your index_expression. For example, if you wanted to tally how many times a specific process performed a read to the virtual file system (using the vfs.read event), you can use the following probe:

Example 3.14. vfsreads.stp

probe vfs.read
{
  reads[execname()] ++
}
In Example 3.14, “vfsreads.stp”, the first time that the probe returns the process name gnome-terminal (that is, the first time gnome-terminal performs a VFS read), that process name is set as the unique key gnome-terminal with an associated value of 1. The next time that the probe returns the process name gnome-terminal, SystemTap increments the associated value of gnome-terminal by 1. SystemTap performs this operation for all process names as the probe returns them.

3.5.4. Processing Multiple Elements in an Array

Once you have collected enough information in an array, you will need to retrieve and process all elements in that array to make it useful. Consider Example 3.14, “vfsreads.stp”: the script collects information about how many VFS reads each process performs, but does not specify what to do with it. The simplest method of making Example 3.14, “vfsreads.stp” useful is to print the key pairs in the reads array.
The best way to process all key pairs in an array (as an iteration) is to use the foreach statement. Consider the following example:

Example 3.15. cumulative-vfsreads.stp

global reads
probe vfs.read
{ 
  reads[execname()] ++
}
probe timer.s(3)
{
  foreach (count in reads)
    printf("%s : %d \n", count, reads[count])
}
In the second probe of Example 3.15, “cumulative-vfsreads.stp”, the foreach statement uses the count variable to reference each iteration of a unique key in the reads array. The reads[count] array statement in the same probe retrieves the associated value of each unique key.
Given what we know about the first probe in Example 3.15, “cumulative-vfsreads.stp”, the script prints VFS-read statistics every 3 seconds, displaying names of processes that performed a VFS-read along with a corresponding VFS-read count.
Now, remember that the foreach statement in Example 3.15, “cumulative-vfsreads.stp” prints all iterations of process names in the array, and in no particular order. You can instruct the script to process the iterations in a particular order by using + (ascending) or - (descending). In addition, you can also limit the number of iterations the script needs to process with the limit value option.
For example, consider the following replacement probe: 
probe timer.s(3)
{
  foreach (count in reads- limit 10)
    printf("%s : %d \n", count, reads[count])
}
This foreach statement instructs the script to process the elements in the array reads in descending order (of associated value). The limit 10 option instructs the foreach to only process the first ten iterations (that is, print the first 10, starting with the highest value).

3.5.5. Clearing/Deleting Arrays and Array Elements

Sometimes, you may need to clear the associated values in array elements, or reset an entire array for re-use in another probe. Example 3.15, “cumulative-vfsreads.stp” in Section 3.5.4, “Processing Multiple Elements in an Array” allows you to track how the number of VFS reads per process grows over time, but it does not show you the number of VFS reads each process makes per 3-second period.
To do that, you will need to clear the values accumulated by the array. You can accomplish this using the delete operator to delete elements in an array, or an entire array. Consider the following example:

Example 3.16. noncumulative-vfsreads.stp

global reads
probe vfs.read
{ 
  reads[execname()] ++
}
probe timer.s(3)
{
  foreach (count in reads)
    printf("%s : %d \n", count, reads[count])
  delete reads	
}
In Example 3.16, “noncumulative-vfsreads.stp”, the second probe prints the number of VFS reads each process made within the probed 3-second period only. The delete reads statement clears the reads array within the probe.

Note

You can have multiple array operations within the same probe. Using the examples from Section 3.5.4, “Processing Multiple Elements in an Array” and Section 3.5.5, “Clearing/Deleting Arrays and Array Elements” , you can track the number of VFS reads each process makes per 3-second period and tally the cumulative VFS reads of those same processes. Consider the following example: 
global reads, totalreads
probe vfs.read
{
  reads[execname()] ++
  totalreads[execname()] ++
}
probe timer.s(3)
{
  printf("=======\n")
  foreach (count in reads-) 
    printf("%s : %d \n", count, reads[count])
  delete reads
}
probe end
{
  printf("TOTALS\n")
  foreach (total in totalreads-)
    printf("%s : %d \n", total, totalreads[total])
}
In this example, the arrays reads and totalreads track the same information, and are printed out in a similar fashion. The only difference here is that reads is cleared every 3-second period, whereas totalreads keeps growing.

3.5.6. Using Arrays in Conditional Statements

You can also use associative arrays in if statements. This is useful if you want to execute a subroutine once a value in the array matches a certain condition. Consider the following example:

Example 3.17. vfsreads-print-if-1kb.stp

global reads
probe vfs.read
{
  reads[execname()] ++
}
probe timer.s(3)
{
  printf("=======\n")
  foreach (count in reads-)
    if (reads[count] >= 1024)
      printf("%s : %dkB \n", count, reads[count]/1024)
    else
      printf("%s : %dB \n", count, reads[count])
}
Every three seconds, Example 3.17, “vfsreads-print-if-1kb.stp” prints out a list of all processes, along with how many times each process performed a VFS read. If the associated value of a process name is equal or greater than 1024, the if statement in the script converts and prints it out in kB.
Testing for Membership

You can also test whether a specific unique key is a member of an array. Further, membership in an array can be used in if statements, as in: 

if([index_expression] in array_name) statement
To illustrate this, consider the following example:

Example 3.18. vfsreads-stop-on-stapio2.stp

global reads
probe vfs.read
{
  reads[execname()] ++
}
probe timer.s(3)
{
  printf("=======\n")
  foreach (count in reads+) 
    printf("%s : %d \n", count, reads[count])
  if(["stapio"] in reads) {
    printf("stapio read detected, exiting\n")
    exit()
  }
}
The if(["stapio"] in reads) statement instructs the script to print stapio read detected, exiting once the unique key stapio is added to the array reads.

3.5.7. Computing for Statistical Aggregates

Statistical aggregates are used to collect statistics on numerical values where it is important to accumulate new data quickly and in large volume (storing only aggregated stream statistics). Statistical aggregates can be used in global variables or as elements in an array.
To add value to a statistical aggregate, use the operator <<< value.

Example 3.19. stat-aggregates.stp

global reads	
probe vfs.read
{
  reads[execname()] <<< count
}
In Example 3.19, “stat-aggregates.stp”, the operator <<< count stores the amount returned by count to the associated value of the corresponding execname() in the reads array. Remember, these values are stored; they are not added to the associated values of each unique key, nor are they used to replace the current associated values. In a manner of speaking, think of it as having each unique key (execname()) having multiple associated values, accumulating with each probe handler run.

Note

In the context of Example 3.19, “stat-aggregates.stp”, count returns the amount of data written by the returned execname() to the virtual file system.
To extract data collected by statistical aggregates, use the syntax format @extractor(variable/array index expression). extractor can be any of the following integer extractors:
count
Returns the number of all values stored into the variable/array index expression. Given the sample probe in Example 3.19, “stat-aggregates.stp”, the expression @count(writes[execname()]) will return how many values are stored in each unique key in array writes.
sum
Returns the sum of all values stored into the variable/array index expression. Again, given sample probe in Example 3.19, “stat-aggregates.stp”, the expression @sum(writes[execname()]) will return the total of all values stored in each unique key in array writes.
min
Returns the smallest among all the values stored in the variable/array index expression.
max
Returns the largest among all the values stored in the variable/array index expression.
avg
Returns the average of all values stored in the variable/array index expression.
When using statistical aggregates, you can also build array constructs that use multiple index expressions (to a maximum of 5). This is helpful in capturing additional contextual information during a probe. For example:

Example 3.20. Multiple Array Indexes

global reads
probe vfs.read
{
  reads[execname(),pid()] <<< 1
}
probe timer.s(3)
{
  foreach([var1,var2] in reads)
    printf("%s (%d) : %d \n", var1, var2, @count(reads[var1,var2]))
}
In Example 3.20, “Multiple Array Indexes”, the first probe tracks how many times each process performs a VFS read. What makes this different from earlier examples is that this array associates a performed read to both a process name and its corresponding process ID.
The second probe in Example 3.20, “Multiple Array Indexes” demonstrates how to process and print the information collected by the array reads. Note how the foreach statement uses the same number of variables (var1 and var2) contained in the first instance of the array reads from the first probe.