General Benchmarking

Latest response

I want a straighforward way of comparing two different systems in terms of their performance.

It does not seem that RedHat ships with any benchmarking utility of sufficient merit. I am looking for something comparable to Bench32.exe from Atto for Windows but I can't seem to find anything. Moreover, the benchmark utility in Disk Utility requires an unformatted disk to test write performance, but it is also able to take advantage of RAM. In this case I want to directly test DISK I/O performance and produce a nice plot of where the sweetspot is for the block size.

I'm curious as to what others are using to do this. Also any recommended CPU and memory benchmarking utilitites. They key is to keep it simple. I've looked at Phoronix Test Suite which is nice but a bit on the cumbersome side. It may come down to that though.

Any suggestions are welcomed.

Responses

Kevin,

Having never used Bench32.exe it's difficult to provide a comparative solution. Can you describe specifically what you want to benchmark? is it just diskio/throughput or are you benchmarking CPU/Memory/Network or a specific application? Single host or multiple hosts?

This thread may be of assistance:
https://access.redhat.com/solutions/173863

So the Atto tool basically does a very quick test of Disk performance in terms of MB/sec while doing a sweep of all of the different block sizes for read and write operations. This allows you to quickly see the optimal block size to set when creating file systems.

I'm going to attempt to attach a picture to the original post. Here is a link to a screenshot of the tool.
http://www.google.com/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&ved=0CAcQjRw&url=http%3A%2F%2Fwww.softpedia.com%2Fget%2FSystem%2FBenchmarks%2FATTO-Disk-Benchmark.shtml&ei=9aO1VP-4HsG-ggTQzIPoBg&bvm=bv.83640239,d.eXY&psig=AFQjCNGUE234Fr8J7TaPS0v2uj4uI0bJqw&ust=1421276523367676

Without going into any specialised tools, I think dd would be able to give you a similar result to the output you have attached, eg.

dd bs=8192K count=32 if=/dev/zero of=disktest oflag=direct

Then just modify the bs 'block size' value and count (could be written into a small loop).

If you want more than that I would recommend looking at Bonnie from the above thread I posted.

-edit-

Stole Jamie's oflag=direct from below as it seems more accurate.

Oh I just used direct I/O because it's what's ticked in the screenshot, I assume an application which does direct I/O is being used. Like all artificial benchmarking, results are only as accurate as the actual usage they are mimicing.

If you want to test for async I/O then replace the direct flags with conv=fsync which ensures all data and metadata is flushed from cache before completing. This allows async I/O to be merged for good performance, but doesn't skew your results by only writing to filesystem cache, as dd does when it's with no options.

btw if you have any suggestions for additional benchmarking tools for the kbase you linked, please let me know or comment on it, I'd love to add them. I compiled that list from a few sources but it seems there's always a good new benchmarking tool someone else knows!

Cheers for the info Jamie, I was wondering who went to the effort to put that list together!

Systemtap may be a worthy addition, but it's not exactly a point and shoot type of benchmarking tool.
https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/SystemTap_Beginners_Guide/index.html

This was my script to provide the same as screenshot but in terminal (Use at your own risk!). I will fix the division that provides total size when I get a minute.. as that will (slightly) impact stats and should be uniform.

#!/bin/bash
bs=1
ct=0

while [ ${bs} -le 8192 ]
do
ct=$(expr 250000 / ${bs})

echo "Block size $(printf "%4sK" ${bs}) Write: $(dd bs=${bs}K count=${ct} if=/dev/zero of=disktest oflag=direct 2>&1 | grep copied)"
echo "Block size $(printf "%4sK" ${bs}) Read : $(dd bs=${bs}K if=disktest of=/dev/zero iflag=direct 2>&1 | grep copied)"

bs=$(expr ${bs} \* 2)
done

if [ -f disktest ]; then
  rm disktest
fi

Output example:

Block size    1K Write: 256000000 bytes (256 MB) copied, 80.337 s, 3.2 MB/s
Block size    1K Read : 256000000 bytes (256 MB) copied, 57.2704 s, 4.5 MB/s
Block size    2K Write: 256000000 bytes (256 MB) copied, 39.8333 s, 6.4 MB/s
Block size    2K Read : 256000000 bytes (256 MB) copied, 28.9451 s, 8.8 MB/s
Block size    4K Write: 256000000 bytes (256 MB) copied, 20.4561 s, 12.5 MB/s
Block size    4K Read : 256000000 bytes (256 MB) copied, 14.4958 s, 17.7 MB/s
Block size    8K Write: 256000000 bytes (256 MB) copied, 11.3489 s, 22.6 MB/s
Block size    8K Read : 256000000 bytes (256 MB) copied, 8.65886 s, 29.6 MB/s
Block size   16K Write: 256000000 bytes (256 MB) copied, 6.42032 s, 39.9 MB/s
Block size   16K Read : 256000000 bytes (256 MB) copied, 4.99115 s, 51.3 MB/s
Block size   32K Write: 255983616 bytes (256 MB) copied, 4.28486 s, 59.7 MB/s
Block size   32K Read : 255983616 bytes (256 MB) copied, 3.0628 s, 83.6 MB/s
Block size   64K Write: 255983616 bytes (256 MB) copied, 2.73701 s, 93.5 MB/s
Block size   64K Read : 255983616 bytes (256 MB) copied, 2.17796 s, 118 MB/s
Block size  128K Write: 255983616 bytes (256 MB) copied, 2.43073 s, 105 MB/s
Block size  128K Read : 255983616 bytes (256 MB) copied, 2.00827 s, 127 MB/s
Block size  256K Write: 255852544 bytes (256 MB) copied, 2.37852 s, 108 MB/s
Block size  256K Read : 255852544 bytes (256 MB) copied, 1.83763 s, 139 MB/s
Block size  512K Write: 255852544 bytes (256 MB) copied, 2.14551 s, 119 MB/s
Block size  512K Read : 255852544 bytes (256 MB) copied, 1.85699 s, 138 MB/s
Block size 1024K Write: 255852544 bytes (256 MB) copied, 2.14587 s, 119 MB/s
Block size 1024K Read : 255852544 bytes (256 MB) copied, 1.76811 s, 145 MB/s
Block size 2048K Write: 255852544 bytes (256 MB) copied, 2.00662 s, 128 MB/s
Block size 2048K Read : 255852544 bytes (256 MB) copied, 1.78564 s, 143 MB/s
Block size 4096K Write: 255852544 bytes (256 MB) copied, 1.99437 s, 128 MB/s
Block size 4096K Read : 255852544 bytes (256 MB) copied, 1.49582 s, 171 MB/s
Block size 8192K Write: 251658240 bytes (252 MB) copied, 1.71451 s, 147 MB/s
Block size 8192K Read : 251658240 bytes (252 MB) copied, 1.35223 s, 186 MB/s

Nice! Great idea incorporating the "copied" output from dd in the display. Do IEC units and it'll be perfect ;)

Phoronix Test Suite is pretty good. It incorporates a lot of different benchmarking utilities, so I think their is value in adding it to your list. It is just unfortunately a little cumbersome.

Almost sounds like the types of things I used to use Bonnie++ for. Though, last time I was stress-testing storage subsystems, I used FIO.

Both are excellent tools for bringing storage systems to their knees.

It's not as pretty as the graph, but this script should do what you're after.

#!/bin/bash
if [ -z $1 ]; then
  echo "Usage: $0 [DEVICE|FILE]"
  exit 1
fi
SIZE="512"
COUNT="0"
MB="256"
while [ $SIZE != 8388608 ]; do
  COUNT="$((1048576*MB/SIZE))"
  echo "Running write test against $1: $COUNT direct I/Os of $SIZE bytes ($MB MiB)"
  dd if=/dev/zero of="$1" bs=$SIZE count=$COUNT oflag=direct
  echo "Running read test against $1: direct I/Os of $SIZE bytes"
  dd if="$1" of=/dev/null bs=$SIZE iflag=direct
  SIZE="$((SIZE*2))"
done
if [ -f $1 ]; then
  rm $1
fi

Extra points available for making this pretty in gnuplot, and for spawning something else to regularly send dd a USR1 so it prints current progress ;)

We have used iozone (http://www.iozone.org/) to test I/O performance and compare systems.

I was aware of iozone. I wasn't particularly happy with its output. Also, my system has large amounts of RAM so I don't want to have to create a 256GB test file just to exhuast the RAM to measure the true performance of the disk.

you can boot with mem=16G. Thus you will measure less of the memory

Thanks for the suggestions. I'll give some of these a shot. Ultimately I have to produce some graph to my manager so he can assess the cost/benefit ratio for system performance. I need charts that make that clear.

This thread raises a good point regarding general benchmarking. I think a tool similar to Ryan Sawhill's xsos (but for benchmarking) would be extremely well received!
https://access.redhat.com/discussions/469323

To all reading, if you agree with this, please do raise a support case and make a feature request. The more you can provide about your requirements and aims of performance testing, the better we can meet those needs. The more people ask, the more likely it is to happen. We absolutely love the opportunity to provide and build software which makes RHEL more useful for you!

Do you mean a tool like System Configuration Collector (http://sysconfcollect.sourceforge.net/)? It creates snapshots of systems and provides functionality to compare (part of) these snapshots (http://sysconfcollect.sourceforge.net/examples/compare_system.html).