Slackware Beowulf Cluster

[Aaron Collier]

Hello,

I have actually been working on a cluster-oriented Linux distribution 
based on Slackware (actually version 8.1) called WUBCOS (Widener 
University Beowulf Cluster OS).  I have been running the basic 
distribution on an eight node cluster I built as part of my senior project 
in computer science and mathematics.  Unfortunately, the distribution 
isn't very refined (but still stable) as I no longer have access to 
adequate resources to continue development because I just graduated.

WUBCOS is currently optimized for my particular hardware but I can tell 
you the software I have installed at the moment.  I compiled a custom 
kernel and actually enabled hyper-threading (simultaneous multithreading) 
which works fairly well despite what others may say.  I upgraded the 
kernel from 2.4.18 to 2.4.20 to add proper DMA support and HT 
support, but Slackware 9 includes kernel 2.4.20 (no source though).  Since 
the member nodes don't have CD-ROM drives I wrote a small network-based 
remote node installation script that mounts an NFS partition hosted on the 
main server simply containing a copy of the Slackware installation CD and 
custom packages I created.  I actually had to create a custom root disk 
set which includes the install script and the necessary nic drivers.  I 
also have been using channel bonding (the default round-robin packet 
distribution scheme) and haven't had any problems with stability.  The 
cluster uses both MPICH-MPI and LAM-MPI although MPICH has been patched to 
work more efficiently with Open PBS which is used as the only batch queue 
system.  Also included is a precompiled ATLAS package which has been 
optimized for the Pentium 4 Xeon with threaded libraries.  I also 
installed PVFS which works well enough.  The distro uses NIS to distribute 
user information, DNS for standard name resolution, and NFS to distribute 
user home directories and actually /usr/local as well (people will argue 
this isn't good but it works for me).  I also configured NTP because time 
travel was becoming all too common on the nodes.  I also went through all 
of the system initialization scripts (located in /etc/rc.d) and disabled 
all unnecessary system servies (like httpd) although these files differ 
for the head node and the regular compute nodes.

Speaking in terms of a general installation, since I had a large hard 
drive I installed everything (about 2GB) on the head node (actually I 
removed the sudo package).  For the installation of the member nodes I 
made custom tagfiles so a node installation only requires about 750MB.  
What you need to install will probably depend upon your anticipated use 
for the cluster.

Configuring Slackware to operate efficiently in a cluster environment can 
be fairly time consuming depending upon your needs but it is certainly 
well worth the effort.  I am glad someone else recognizes the merits of 
the Slackware distribution.

Please feel free to contact me if you have any additional questions.

Bye,
Aaron Collier

P.S. If anyone would like to hire me then please send me an e-mail :o)

_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf