Cluster Poll Results (tangent into OS choices)

Vann H. Walke walkev at
Fri Oct 31 14:44:59 EST 2003

On Fri, 2003-10-31 at 12:38, John Hearns wrote:
> On Fri, 31 Oct 2003, Robert G. Brown wrote:
> > 
> > It is also very interesting to me that RH (for example) thinks that
> > there is something that they are going to provide that is worth e.g.
> > several hundred thousand dollars in the case of a 1000+ node cluster
> > running their "workstation" product.  Fifty dollars certainly.  Five
> > hundred dollars maybe.  A thousand dollars possibly, but only if they
> > come up with a cluster-specific installation with some actual added
> > value.
> > 
> I'll second that.
> There has been a debate running on this topic on the Fedora list
> over the last few days.
> Sorry to be so boring, but its something we should debate too.

Hmm... Let's take the case of a 1000 node system.  If we assume a
$3000/node cost (probably low once rack, UPS, hardware support, and
interconnect are added in), we arrive at an approximate hardware cost of
$3,000,000.  If we were to use the RHEL WS list price of $179/node, we
get $179,000 or about 6% of the hardware cost.  That is assuming RedHat
will not provide any discount on large volume purchases (unlikely).  Is
6% unreasonable?    

What are the alternatives?

- Keep using an existing RH distro:  Only if you're willing to move into
do it yourself mode when RH stop support (December?).  I expect very few
would be happy with this option.  However, if you have a working RH7.3
cluster, it works, and you don't have to worry too much about security,
why change?  For new clusters though....
- Fedora - Planned releases 2-3 times a year.  So, if I build a system
on the Fedora release scheduled this Monday, who will be providing
security patches for it 2 years from now (after 4-6 new releases have
been dropped).  My guess is no-one. Again, we're in the do it yourself
maintenance or frequent OS upgrade mode.
- SUSE - Not sure about this one.  Their commercial pricing model is
pretty close to RedHat's.  Are they going to keep developing consumer
releases?  What will the support be for those releases?  Can we really
expect more than we get from a purely community developed system? 
Perhaps someone with more SUSE knowledge could comment?
- Debian - Could be a good option, but to some extent you end up in the
same position as Fedora.  How often do the releases come out.  Who
supports the old releases?  What hardware / software will work on the
- Gentoo - Not reliable, stable enough to meet my needs for clustering
- Mandrake - Mandrake has their clustering distribution, which could be
a good possibility, but the cost is as high or higher than RedHat.  
- Scyld - Superior design, supported, but again very high cost and may
have to fight some compatibility issues since the it's market share in
the Linux world is less than tiny.
- OSCAR / Rocks / etc...  - generally installed on top of another
distribution.  We still have to pick a base distribution.

My conclusions - If you're in a research facility / university type
setting where limited amounts of down time are acceptable, a free or
nearly free system is perfect.  A new Fedora/Debian/SuSE release comes
out, shut the system down over Christmas break and rebuild it.  (As long
as you're happy spending a fair amount of time doing rebuilds and fixing
upgrade problems). 

If however you really need the thing to work - Corporate research sites,
satellite data processing, etc... the cost of the operating system may
be minuscule relative to the cost of having the system down.  If you
_really_ want a particular application to work having it certified and
supported on the OS may be important.  

The project on which I'm working - building sonar training simulators
for the US Navy Submarine force requires stable systems which should
operate without major maintenance / operational changes for many years. 
Knowing the RedHat will support the enterprise line for 5 years is a big
selling point.

The cluster management portion of the software stack would be great to
have integrated in to the product, but if third party vendors (Linux
Networx, OSCAR, Rocks, etc...) can provide the cluster management
portion on top of the distribution, a solution can be found.  In some
ways this is even better since your cluster management decision is
independent of the OS vendor.

I basically just want to make the point that the cluster space is filled
with people of many different needs.  Will everyone want RHEL?  My guess
is a resounding NO.  (In the days of RH7.3 you could almost say Yes.) 
But, there are situations in which a stable, supported product is
needed.  This is the market RedHat is trying to target and states so
pretty clearly ("Enterprise").  Small users and research systems get
somewhat left out in the cold, but we probably shouldn't complain after
having a free ride for the last 5+ years.  

So, is 6% unreasonable?


> _______________________________________________
> Beowulf mailing list, Beowulf at
> To change your subscription (digest mode or unsubscribe) visit

Beowulf mailing list, Beowulf at
To change your subscription (digest mode or unsubscribe) visit

More information about the Beowulf mailing list