renting time on a cluster

Robert G. Brown rgb at phy.duke.edu
Thu Apr 10 15:57:46 EDT 2003


On Thu, 10 Apr 2003, Jakob Oestergaard wrote:

> 
> So, is the original question a once-in-a-decade thing, or do people
> generally feel that there is interest in such a service?
> 
> (haven't seen many of those requests on this list, AFAIR)

The problem is that there is a fairly narrow profile of problems for
which such a service is optimal; in "most cases" the cost benefit of
doing it yourself or in your existing IT organization are superior, as
you have to pay any such service provider the real costs plus
depreciation plus a profit; in your own organization some parts of these
costs are low marginal cost rescalings of existing infrastructure or
opportunity cost time paid out of a pool of low priority competing
tasks or FTE surplus hours (i.e. free).

The same problem exists, actually, for "centralized" shared compute
resources at universities or supercomputer centers -- for these to be a
cost win they generally need a pool of clients that is:

  a) Big enough to keep their cluster operating close to capacity all
the time, since the only way to be the fixed costs of dead time is to
amortize it over active time, raising rates and starting a deadly spiral
of still fewer clients.

  b) With demand that can be spread out to keep the duty cycle high a la
a).  It does no good to have one cluster-year's worth of tasks for your
cluster if all your clients insist on having their work done in the same
three month time of the year -- you'll have to have a cluster 3x bigger
(and idle 3/4 of the year) or lose 2/3 of your clients and STILL be idle
3/4 of the year.  Oooo, hate to even do the math on that one.

  c) Poor enough in local computing resources that a locally purchased
and administered cluster doesn't make more sense.

  d) Almost by definition, with a problem that needs only a short,
intense burst of computation.  People with longrunning problems tend NOT
to use this sort of resource because they almost always are better off
with their own cluster.  It's people who need a 128 node cluster for a
month who can't make do with a 16 node cluster for a year that will be
your primary clients (along with a FEW of those local-resource poor
groups -- this is an important client base of a shared resource in a
University, for example).

A good sized campus is likely to have enough of a mix where a
centralized cluster can make sense, especially one that is "owned" by
the primary groups that operate it who effectively subscribe most of its
time with a clear understanding of how it is to be split up among long
runners and on demanders.

A commercial cluster is pretty tough.  I think you'd need a bunch of
long term "subscribers" there as well, contractually bound for periods
on the order of a year, to keep risks sane and costs reasonably
competitive with DIY.  If you had some sort of auction/market model
whereby you could resell idle time at or even below cost to keep from
losing money actively while charging a lot more than cost to the on
demand short term users (who would pay it as it is still cheaper than
building their own) you might work out a stable and profitable business.

You'd also do better reselling to businesses than to university or
government researchers.  We're notoriously cheap and like to DIY anyway.
Small businesses especially often have significant infrastructure
barriers that would make purchasing rented time desirable in at least
the short run, IF you could identify the small businesses that need
it...

   rgb

Robert G. Brown	                       http://www.phy.duke.edu/~rgb/
Duke University Dept. of Physics, Box 90305
Durham, N.C. 27708-0305
Phone: 1-919-660-2567  Fax: 919-660-2525     email:rgb at phy.duke.edu



_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf



More information about the Beowulf mailing list