[Beowulf] recommendation on crash cart for a cluster room: fullcluster KVM is not an option I suppose?

Gerry Creager gerry.creager at tamu.edu
Tue Oct 13 12:11:22 EDT 2009

Don Holmgren wrote:
> On Tue, 13 Oct 2009, Prentice Bisbal wrote:
>> Mark Hahn wrote:
>>>> Even with IPMI, you still need a crash cart of some type to initially
>>>> set up IPMI in the system's BIOS. At the minimum, you need to set 
>>>> the IP
>>>> address that the IMPI interface will listen on (if it's a shared NIC
>>> afaik, not really.  here's what I prefer: cluster nodes normally come
>>> out of the box with BIOS configured to try booting over the net before
>>> local HD.
>>> sometimes this is conditional on the local HD having no active 
>>> partition.
>>> great: so they boot from a special PXE image I set up as a catchall.
>>> (dhcpd lets you define a catchall for any not nodes which lack a 
>>> their own
>>> MAC-specific stanza.)  when nodes are in that state, I like to
>>> auto-configure
>>> the cluster's knowlege of them: collect MAC, add to dhcpd.conf, etc.  at
>>> this stage, you can also use local (open) ipmi on the node itself to
>>> configure the IPMI LAN interface:
>>>     ipmitool lan 2 set password pa55word
>>>     ipmitool lan 2 set defgw ipaddr
>>>     ipmitool lan 2 set ipsrc dhcp
>>> none of this precludes tricks like frobing the switch to find the 
>>> port-MAC
>>> mappings of course - the point is simply that if you let unconfigured 
>>> nodes
>>> autoboot into a useful image, that image can help you automate more 
>>> of the
>>> config.
>> My cluster nodes' IPMI share their physical port with the primary NIC.
>> Before using IPMI, I had to enable it in the BIOS and then assign it an
>> IP address in the BIOS, too. I didn't think of using ipmitool. I wonder
>> if I could do all that using ipmitool, without enabling IPMI in the BIOS
>> first.  Anyone know for sure?
>> -- 
>> Prentice
> With all (and various) flavors of IPMI and BMC hardware on Intel, 
> Supermicro, and Asus systems since about 2004, we've been able to use 
> ipmitool or equivalent software in Linux to setup IPMI LAN and other 
> parameters.
> Consistently, though, for systems with serial over LAN, we've always had 
> to configure serial port redirect settings in the BIOS.  The latter is 
> why we've always tried to get vendors to provide a mechanism for 
> replicating BIOS settings from machine to machine without using a crash 
> cart (not always successfully, unfortunately).

We've been working with one vendor to get IPMI working better and more 
predictably with DHCP. Manual config with a crash cart has been a bit 
problemmatical for us, too.  SuperMicro has been real good about custom 
BIOS mods to help us out in that regard.

Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

More information about the Beowulf mailing list