Red Hat Training

A Red Hat training course is available for Red Hat Enterprise Linux

9.2. Monitoring and Diagnosing Performance Problems

Red Hat Enterprise Linux 7 provides a number of tools that are useful for monitoring system performance and diagnosing performance problems related to the networking subsystem. This section outlines the available tools and gives examples of how to use them to monitor and diagnose network related performance issues.

9.2.1. ss

ss is a command-line utility that prints statistical information about sockets, allowing administrators to assess device performance over time. By default, ss lists open non-listening TCP sockets that have established connections, but a number of useful options are provided to help administrators filter out statistics about specific sockets.
Red Hat recommends ss over netstat in Red Hat Enterprise Linux 7.
ss is provided by the iproute package. For more information, see the man page:
$ man ss

9.2.2. ip

The ip utility lets administrators manage and monitor routes, devices, routing policies, and tunnels. The ip monitor command can continuously monitor the state of devices, addresses, and routes.
ip is provided by the iproute package. For details about using ip, see the man page:
$ man ip

9.2.3. dropwatch

Dropwatch is an interactive tool that monitors and records packets that are dropped by the kernel.
For further information, see the dropwatch man page:
$ man dropwatch

9.2.4. ethtool

The ethtool utility allows administrators to view and edit network interface card settings. It is useful for observing the statistics of certain devices, such as the number of packets dropped by that device.
You can view the status of a specified device's counters with ethtool -S and the name of the device you want to monitor.
$ ethtool -S devname
For further information, see the man page:
$ man ethtool

9.2.5. /proc/net/snmp

The /proc/net/snmp file displays data that is used by snmp agents for IP, ICMP, TCP and UDP monitoring and management. Examining this file on a regular basis can help administrators identify unusual values and thereby identify potential performance problems. For example, an increase in UDP input errors (InErrors) in /proc/net/snmp can indicate a bottleneck in a socket receive queue.

9.2.6. Network Monitoring with SystemTap

The Red Hat Enterprise Linux 7 SystemTap Beginner's Guide includes several sample scripts that are useful for profiling and monitoring network performance.
The following SystemTap example scripts relate to networking and may be useful in diagnosing network performance problems. By default they are installed to the /usr/share/doc/systemtap-client/examples/network directory.
nettop.stp
Every 5 seconds, prints a list of processes (process identifier and command) with the number of packets sent and received and the amount of data sent and received by the process during that interval.
socket-trace.stp
Instruments each of the functions in the Linux kernel's net/socket.c file, and prints trace data.
dropwatch.stp
Every 5 seconds, prints the number of socket buffers freed at locations in the kernel. Use the --all-modules option to see symbolic names.
The latencytap.stp script records the effect that different types of latency have on one or more processes. It prints a list of latency types every 30 seconds, sorted in descending order by the total time the process or processes spent waiting. This can be useful for identifying the cause of both storage and network latency. Red Hat recommends using the --all-modules option with this script to better enable the mapping of latency events. By default, this script is installed to the /usr/share/doc/systemtap-client-version/examples/profiling directory.