Editing Cluster Benchmarking Packages

Here are some links to benchmarks:

'''Benchmark Suites'''
* [http://icl.cs.utk.edu/hpcc/ HPCC Challange Benchmark] - uses seven tests to measure several different performance parameters.
* [http://www.clustermonkey.net//content/view/38/27/ Beowulf Perforamnce Suite (BPS)] - Example results are [http://clustermonkey.net/download/kronos/bps-logs/ here]. 
* [http://www.nas.nasa.gov/Resources/Software/npb.html NASA Parallel Benchmarks] - One of the most, if not the most, widely used set of benchmarks for clusters.
* [https://software.intel.com/en-us/articles/intel-mpi-benchmarks/ Intel MPI Benchmarks] - freely available.
* [http://perfbase.tigris.org/ perfbase] - a set of front end tools using a PostgreSQL database as backend, which together form a system for the management and analysis of the output of tests and experiments.
* [http://liinwww.ira.uka.de/~skampi/ SKaMPI] - a suite of tests designed to measure the performance of MPI. 
* [http://sourceforge.net/apps/trac/cbench/ Cbench] - a Perl-based scripting framework for building, running, and analyzing the output of various opensource codes 
* [http://accc.riken.jp/HPC_e/himenobmt_e.html Himeno]  Dr. Ryutaro Himeno, Director of the Advanced Center for Computing and Communication, has developedbenchmark to evaluate performance of incompressible fluid analysis code. This benchmark program takes measurements to proceed major loops in solving the Poisson's equation solution using the Jacobi iteration method. Being the code very simple and easy to compile and to execute, users can measure actual speed (in MFLOPS) immediately.
* [https://github.com/EPCCed ADEPT Benchmark] - a suite that addresses the power usage of parallel technologies. [http://www.adept-project.eu/ Adept] builds on the expertise of software developers from high-performance computing (HPC) to exploit parallelism for performance, and on the expertise of Embedded systems engineers in managing energy usage.


'''File Systems I/O'''

* [http://www.nus.edu.sg/comcen/svu/publications/hpc_nus/sep_2005/Performance.pdf GPFS vs. NFS evaluation]
* EXT3, EXT3, Reiser, JFS, XFS benchmarks [http://linuxgazette.net/102/piszcz.html Part 1] and [http://linuxgazette.net/122/piszcz.html Part 2]
* [http://www.clustermonkey.net/ Cluster Monkey] Series on Parallel File System Benchmarking
** [http://www.clustermonkey.net//content/view/62/28/ Benchmarking Parallel File Systems]
** [http://www.clustermonkey.net//content/view/87/32/  A Benchmark for Parallel File Systems]
** [http://www.clustermonkey.net//content/view/117/32/ Using the PIO Benchmark ]  
* [http://stromberg.dnsalias.org/~strombrg/nfs-test.html nfs-test] is a program that tries a bunch of different rsizes, wsizes, protocols (tcp vs udp) and NFS versions to optimize performance
* [http://www.iozone.org IOZONE] Filesystem Benchmark
* [http://public.lanl.gov/jnunez/benchmarks/mpiiotest.htm MPI-IO Test] LANL's MPI-IO Test (The best test for MPI-IO codes)
* [http://www.llnl.gov/icc/lc/siop/downloads/download.html Livermore File System Tests] A collection of tests from Lawrence Livermore for testing parallel IO file systems. This include IOR (a great test for parallel file systems) and a bunch of metadata tests

'''Individual Codes for Benchmarks'''

These are codes that you can download (freely) and use for benchmarking. Some of them have benchmark results available and some don't.

* [http://www.gromacs.org/ Gromacs] - A great benchmark code. GROMACS is a versatile package to perform molecular dynamics, i.e. simulate the Newtonian equations of motion for systems with hundreds to millions of particles. It is primarily designed for biochemical molecules like proteins and lipids that have a lot of complicated bonded interactions.\
* [http://hmmer.janelia.org/ HMMER] HMMER is used for profiling hidden Markov models (profile HMMs) to do sensitive database searching using statistical descriptions of a sequence family's consensus. HMMER is a freely distributable implementation of profile HMM software for protein sequence analysis.
* [http://gfs.sourceforge.net/ Gerris] - Gerris is an Open Source Free Software library for the solution of the partial differential equations describing fluid flow.
* [http://www.hlrs.de/people/resch/PROJECTS/PARACFD.html ParaCFD] While this is an old CFD benchmark, I think it is still useful. It contains both an OpenMP version and a MPI version.
* [http://www.mgnet.org/mgnet-codes.html MGNet] - A collection of Multi-Grid codes that you can use for benchmarking. There are no published results, but there are some MPI codes that you can use for benchmarks.

'''Network Benchmarks'''

These benchmarks can be used to test the networking aspects of cluster codes. Some of them can test the entire fabric and some test just a pair of nodes.
* [http://www.scl.ameslab.gov/netpipe/ NetPIPE] - One of the best network benchmarks. It tests a range of packet sizes and measures the time (bandwidth). It can use MPI as the message sending mechanism so it allows you to also test MPI implementations.
* [http://www.netperf.org/netperf/ Netperf] - Another good networking benchmark that is used frequently.
@@ Line 41: / Line 41: @@
 * [http://www.scl.ameslab.gov/netpipe/ NetPIPE] - One of the best network benchmarks. It tests a range of packet sizes and measures the time (bandwidth). It can use MPI as the message sending mechanism so it allows you to also test MPI implementations.
 * [http://www.netperf.org/netperf/ Netperf] - Another good networking benchmark that is used frequently.
-'''GPU Benchmarks/Tests'''
-* [https://sourceforge.net/projects/cudagpumemtest CUDA GPU memtest] is a GPU memory test utility for NVIDIA and AMD GPUs using well established patterns from memtest86/memtest86+ as well as additional stress tests. The tests are designed to find hardware and soft errors. The code is written in CUDA and OpenCL.
-* [https://github.com/ihaque/memtestCL MemtestCL] is a program to test the memory and logic of OpenCL-enabled GPUs, CPUs, and accelerators for errors. It is an OpenCL port of our CUDA- based tester for NVIDIA GPUs, MemtestG80.
-* [https://launchpad.net/clamity Calamity] - is a testing framework that allows you test how your GPU is working, by running various tests. With its plugin system it can be expanded to test for more than just the core set.
-* [https://github.com/Microway/gpu-burn GPU-Burn] thoroughly exercises the math units on NVIDIA GPUs. It is capable of stressing both the single-precision (32-bit) and double-precision (64-bit) math units of any GPU with CUDA support. Generally-speaking, this means any NVIDIA Tesla, Quadro or GeForce GPU released after 2010. During startup, all GPUs in the system are discovered and tests are run on each. As the tool runs, the status of each GPU is output (including number of passes, number of errors encountered, and current GPU temperatures).