PelicanHPC: A GNU/Linux distribution to create a HPC cluster for MPI based parallel computing

PelicanHPC
GNU Linux

News

29 Sept 2008 If you were looking for ParallelKnoppix and you arrived here, you're in the right place. PelicanHPC is the continuation of the ParallelKnoppix project. The PK web page is no longer available, and re-directs here. ISO images for the last version of PK are achived here on the download page, but you're encouraged to use PelicanHPC instead.
05 Sept 2008 v1.7 released

Open MPI 1.2.7rc2, LAM/MPI 7.1.2, Octave 3.0.1 with MPITB
kernel 2.6.26, syslinux 3.71, trying out atftpd
make_pelican adapted to latest live-helper
general sync to Debian Lenny
torrents available at Linuxtracker.org Thanks! ISO images are also available (see the download section, below)

11 July 2008 v1.6 released

boots into graphical environment (xfce4) to avoid confusing people born in the last few decades
the tutorial is included on the desktop
general updating and a little polishing here and there

Features

PelicanHPC is a live CD image that let's you set up a high performance computing cluster in a few minutes. A Pelican cluster allows you to do parallel computing using MPI. You can run Pelican on a single multiple core machine to use all cores to solve a problem, or you can network multiple computers together to make a cluster. The frontend node (either a real computer or a virtual machine) boots from the CD image. The compute nodes boot by PXE, using the frontend node as the server. All of the nodes of the cluster get their filesystems from the same CD image, so it is guaranteed that all nodes run the the same software. The CD image is created by running a single script, which takes advantage of the Debian Live infrastructure. If you need to add packages, it is very easy to create a custom version by adding the package names to the script and then running it.

Pelican is created using Debian Live. To make your own version you only need live_helper and the make_pelican script, which is provided below.
The LAM-MPI and OpenMPI implementations of MPI are installed. Both 32 and 64 bit versions are available. Debian testing (Lenny) is the base for both.
Contains extensive example programs using GNU Octave and MPITB. Also has the Linpack HPL benchmark.
You can use any Class C network you like. By default, the cluster is on 10.11.12.*
xfce4 window manager, konqueror for browsing and file management, ksysguard for monitoring the cluster, kate and nano for editing. As noted, it is very easy to add packages. Pelican is a bare-bones framework for setting up a HPC cluster.
Pelican releases and all testing is done using Debian testing (Lenny) as the base. Etch does not work, Sid may or may not.

Here are some shots of v1.3 (click to enlarge):

Download

I'd appreciate it if you would try to get a new release using bittorrent, if possible. Check for torrents here. When the torrent gets stale, use the normal download page or a mirror (see below). The make_pelican script and pre-built ISO images are always available on the normal download page. To use the script to make your own version, you need to install the live_helper package. Get the most recent version.

Mirrors (Thanks!):

Germany:
http://download.semmel.org/PK/ username "parallel" password "knoppix"

Documentation

The basic steps are to boot the CD, and then type "pelican_setup". To use X, type "startx". Here is a tutorial
that shows screenshots of the process. For security, you should change the user password (the default is "live").
You can help by contributing bits of knowledge to the forum.

User Forum

Highlights from the forum:

Help Wanted

Pelican is close to perfection but it's not quite there. Here are some problems I would like some help with:

infiniband support: I believe that a user managed to use IB on ParallelKnoppix. If so, it should not be hard to do on Pelican. I don't have hardware to try this.
documentation and examples of usage