<HTML>
<HEAD>
<TITLE>Re: [Beowulf] assigning cores to queues with torque</TITLE>
</HEAD>
<BODY>
<FONT FACE="Calibri, Verdana, Helvetica, Arial"><SPAN STYLE='font-size:11pt'><BR>
<BR>
<BR>
On 3/8/10 10:14 AM, "Micha Feigin" <<a href="michf@post.tau.ac.il">michf@post.tau.ac.il</a>> wrote:<BR>
<BR>
</SPAN></FONT><BLOCKQUOTE><FONT FACE="Calibri, Verdana, Helvetica, Arial"><SPAN STYLE='font-size:11pt'>I have a small local cluster in our lab that I'm trying to setup with minimum<BR>
hustle to support both cpu and gpu processing where only some of the nodes have<BR>
a gpu and those have only two gpu for four cores.<BR>
<BR>
It is currently setup using torque from ubuntu (2.3.6) with the torque supplied<BR>
scheduler (set it up with maui initially but it was a bit of a pain for such a<BR>
small cluster so I switched)<BR>
<BR>
This cluster is used by very few people in a very controlled environment so I<BR>
don't really need any protection from each other, the queues are just for<BR>
convenience to allow remote execution<BR>
<BR>
The problem:<BR>
<BR>
I want to allow gpu related jobs to run only on the gpu equiped nodes (i.e more jobs then GPUs will be queued), I want to run other jobs on all nodes with either<BR>
1. a priority to use the gpu equiped nodes last<BR>
2. or better, use only two out of four cores on the gpu equiped nodes<BR>
<BR>
It doesn't seem though that I can map nodes or cores to queues with torque as far as I can tell<BR>
(i.e cpu queue uses 2 cores on gpu1, 2 cores on gpu2, all cores on everything else<BR>
gpu queue uses 2 cores on gpu1, 2 cores on gpu2)<BR>
<BR>
I can't seem to set user defined resources so that I can define gpu machines as having gpu resource and schedule according to that.<BR>
<BR>
Is it possible to achieve any of these two with torque, or is there any other<BR>
simple enough queue manager that can do this (preferably with a debian package<BR>
in some way to simplify maintanance). I only manage this cluster since no one<BR>
else knows how to and it's supposed to take as little of my time as possible<BR>
I'm looking for the simplest solution to implement and not the most versatile<BR>
one.<BR>
</SPAN></FONT></BLOCKQUOTE><FONT FACE="Calibri, Verdana, Helvetica, Arial"><SPAN STYLE='font-size:11pt'><BR>
<BR>
you can define a resource “gpu” in your TORQUE nodes file:<BR>
<BR>
hostname np=4 gpu<BR>
<BR>
and then users can request –l nodes=1:ppn=4:gpu to get assigned a node with a gpu, but to do anything more advanced you’ll need Maui or Moab. You should try the maui users mailing list, or the torque users mailing list to see if anyone else has some ideas</SPAN></FONT>
</BODY>
</HTML>