Running perl scripts and non-mpi programs on scyld

[hanzl at noel.feld.cvut.cz]

> We have a Scyld Beowulf cluster currently running on 28cz-4 (we are
> getting -5 soon).  We have been running into a lot of problems with
> users that are trying to run scripts on the child nodes.   To start
> with, what is the best way to run serial (non-MPI) programs?
> ...
> Say I have a perl script.  (I  NFS mount /usr /lib etc. on the child
> nodes)
> 
> I want to run this perl script on N nodes with N DIFFERENT arguments.

For these types of jobs, we are using SGE on scyld-like cluster (we
are using HDDCS which is a variant of Clustermatic which is similar to
Scyld but this should not matter here).

SGE is quite nice and opensource batch spooling. Using it with
scyld-like cluster for this type of jobs is a bit tricky but quite
easy. We just create one 'queue' for every slave node and use node
number as a queue name. Then we use 'starter method' script like this:

file /usr/local/bin/sge-bproc-starter-method:

  #/bin/sh
  bpsh $QUEUE $*

All these queues are defined as running on master node but starter
method in fact moves perl scripts on individual slave nodes.

To run scripts on N nodes with N DIFFERENT arguments, you may use
'array jobs' or submit many individual jobs.

(And there is much more you can do with SGE, I highly recommend it.)

Regards

Vaclav Hanzl
_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf