[Beowulf] cluster scheduler for dynamic tree-structured jobs?

Guy Coates gmpc at sanger.ac.uk
Tue Jun 1 08:03:15 EDT 2010


On 15/05/10 11:24, Andrew Piskorski wrote:
> Folks, I could use some advice on which cluster job scheduler (batch
> queuing system) would be most appropriate for my particular needs.
> I've looked through docs for SGE, Slurm, etc., but without first-hand
> experience with each one it's not at all clear to me which I should
> choose...
> 

This may be late in the day but...
If you job dependencies are too complicated for you queuing system to
deal with, you may want to look at the Ensembl Hive system;

http://www.ensembl.org/info/docs/eHive/index.html

It is the system we use in-house for our genome-analysis pipelines,
which have lots of complicated dependencies. It sits on top of a
traditional queuing system which handles job-dispatch etc.

It has been de-coupled from the genome analysis workflow, so you should
(in theory) be able to use it for any analysis.

Cheers,

Guy

-- 
Dr. Guy Coates, Informatics System Group
The Wellcome Trust Sanger Institute, Hinxton, Cambridge, CB10 1HH, UK
Tel: +44 (0)1223 834244 x 6925
Fax: +44 (0)1223 496802


-- 
 The Wellcome Trust Sanger Institute is operated by Genome Research 
 Limited, a charity registered in England with number 1021457 and a 
 company registered in England with number 2742969, whose registered 
 office is 215 Euston Road, London, NW1 2BE. 
_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.



More information about the Beowulf mailing list