[Beowulf] Hypothetical Situation

[Brian Dobbins]

Hi Brent,

  A while back, I had done a portion of what you were looking for, just to 
make my job easier while testing new kernels on our nodes.  We had a small 
cluster connected to an APC masterswitch which could reboot the nodes (by 
powering them off, then on again).  The nodes themselves were diskless, 
and the process was:

 a) If a newer image was located on the file server, reboot with the new 
    image
 b) If we don't get a status OK from the node after 5 minutes (done via a 
    simple file write in rc.local in the ramdisk), revert back to the old 
    image.

  (This was all done on a fairly small scale, mind you.)

  For us, with diskless nodes, this was very easy - we didn't have to 
modify GRUB's boot parameters at all.  I'm not even remotely knowledgeable 
about GRUB's options, but perhaps there's an easy way to do configure it 
to boot a specified kernel just once, then revert back to a 'safe' one.  
(You can do this in startup, if you're guaranteed startup will succeed, 
but that's not likely in your situation!)

  If there's a chance you can do this disklessly in terms of booting, let 
me know, I can see if I have any of the old configuration files still 
around.  This doesn't mean you can't use disks, but with RAM so cheap 
these days, booting from a ramdisk doesn't cost you much, and makes some 
things a lot easier.  You can always mount the disk during init.

  (Ie, you'd have the script called from the job manager execute a 
'mkelf-linux' or 'mknbi' binary with a standard ramdisk, and the only 
thing that changes is the bzImage file supplied to the command.  From 
memory, something like:)

  mkelf-linux --append="root=/dev/ram" ${bzImage_custom} RamDisk.gz mem=xG 
> /tftpboot/${user}_${date}_${bzImage_custom}.elf

  Unfortunately, I can't help you on the job manager stuff, but I'm sure 
there are plenty of people here who know heaps more and will certainly 
chime in.

  Good luck!
  - Brian


Brian Dobbins
Yale University

_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf