Do you know about the Beowulf Performance Suite?

The Beowulf Performance Suite (BPS) was designed to provide a comprehensive and comparative way of measuring cluster performance. Although BPS contains many benchmarking programs, BPS is not designed to directly benchmark clusters. BPS is designed as an analysis tool to measure differences due to hardware or software changes on the same cluster. In addition, successfully running all the tests provides some assurances that the cluster is configured properly.

The suite can run any or all programs and produce HTML output files. The use of HTML makes it trivial to share your results with others on the web. The following tests are available:

• bonnie - hard drive performance
• stream - memory performance
• netperf - general network performance
• netpipe - detailed network performance
• nas - NAS parallel tests
• unixbench - general Unix benchmarks
• lmbench - micro Linux benchmarks

As part of the design, all the tests are are "open source" and the tar files for all programs are included in the binary RPM. Some of the tests are are already

Some users have questioned why the High Performance LINPACK (HPL) benchmark is not included in the suite. While HPL is a good measure of performance, the main concern was that the test suite would be used as a "my cluster is better than your cluster" set of tests due to the fact that HPL is used to rate the top supercomputers. Our goal was to provide a set of tools to measure performance increases (or decreases) when things are changed.

Obtaining/Installing BPS

Running The Suite

Before running BPS, it is best to examine the options. This step can be accomplished by entering "bps" at the command prompt. (Note: If you installed the tar file you will need to set your path to ~/bps/bin) The options are as follows.

Usage: /opt/bps/bin/bps 

Options:
  -b                            bonnie++
  -s                            stream
  -f <send node>,<receive node&GT netperf to remote node
  -p <send node>,<receive node&GT netpipe to remote node
  -n <compiler&GT,<#processors&GT,  NAS parallel benchmarks
     <test size&GT,<MPI&GT,         compiler={gnu,pgi,intel}
     <machine1,machine2,...&GT    test size={A,B,C,dummy}
                                MPI={mpich,lam,mpipro}
  -k                            keep NAS directory when finished
  -u                            unixbench
  -m                            lmbench
  -l <log_dir>                  benchmark log directory
  -w                            preserve existing log directory
  -i <mboard manufacturer>,     machine information
     <mboard model>,<memory>,
     <interconnect>,<linux ver>
  -v                            show version
  -h                            show this help

All output files from bps are placed in the log directory. If you do not specify a log directory, then the bps script will use default name "bps-logs". If a previous bps-logs directory exists, you may preserve these results by using the "-w" option where the old directory will be renamed to "bps-logs.some-unique-number" and a new bps-logs directory will be created.

The tests can be broken into three categories, single system (i.e. host machine), network (any two machines), and parallel (any number of machines). The host machine tests are stream, bonnie++, lmbench, and unixbench. The network tests are netperf and netpipe. The parallel test are the eight NAS parallel benchmarks (see below).

Some Examples

bps -s 

bps -f kronos01,kronos03

bps -w -b -m -u -p kronos04,kronos02

 bps -k -n gnu,8,A,lam,kronos,kronos01,kronos02,kronos03,kronos04,kronos05,kronos06,kronos00

As mentioned, if you are running the NAS tests, you may tell bps to keep ("-k") the NAS directory (npb) in your log directory. This can be useful in tracking down compile errors (see the npb/make.log). HTML output files can be produced from the results files in the log directory by executing:

bps-html bps-logs

where bps-logs is the bps log directory. The results can be viewed by pointing a browser at the index.html file in the log directory. An example of the results can be found here.

Important Notes

• All tests source files are archived in the /opt/bps/src/ directory.
• The BPS suite is best run as a normal user, not root. Some of the tests (i.e. NAS parallel) will not run as root.
• When using Netpipe and Netperf benchmarks, rsh with no password must be permitted between the nodes.
• Also, the tests have been designed so that the BPS rpm only needs to be installed on the master node. For this to work, the BPS log directory must be mounted on all nodes (e.g. under /home).
• NAS Parallel Benchmarks have been tested with specific versions of LAM and MPICH. Please see the NAS documentation for more information. Rather than limit potential BPS users, these are not made a part of the required packages list. The benchmark scripts have been written to rely on the three environment variables (LAM_HOME for LAM/MPI, MPICH_HOME for MPICH, and MPIPRO_HOME for MPI-PRO, and MPICH). If you are having problems with the NAS benchmarks, use the "-k" option to run the NAS tests and then go to bps-logs/npb directory and consult the README.bps file. Also, if you wish to use the Portland Group or the Intel Compilers make sure they are properly configured.