Global Shared Memory and SCI/Dolphin

Mikhail Kuzminsky kus at
Mon Jul 23 12:44:16 EDT 2012

According to Joachim Worringen
> Franz Marini:
> >   being in the process of deciding which net infrastructure to use for =
> our
> > next cluster (Myrinet, SCI/Dolphin or Quadrics), I was looking at the
> > specs for the different types of hw.
> >   Provided that SCI/Dolphin implements RDMA, I was wondering why so lit=
> tle
> > effort seems to be put into implementing a GSM solution for x86 cluster=
> s.
> Because MPI is what most people want to achieve code- and=20
> peformance-portability.
  Partially I may agree, partially - not: MPI is not the best
in the sense of portability (for example, optimiziation requires
knowledge of interconnect topology, which may vary from cluster to cluster,
and of course from MPP to MPP computer). I think that if there is
relative cheap and effective way to build ccNUMA system from cluster - it may
have success. 
> > The only (maybe big, maybe not) problem I see in the Dolphin hw is the
> > lack of support for cache coherency.
> >
> >   I think that having GSM support in (almost) commodity clusters would =
> be
> > a really-nice-thing(tm).
> Martin Schulz (formerly TU M=FCnchen, now Cornell Theory Center) has deve=
> loped=20
> exactly the thing you are looking for. See=20
> . You will also =
> find=20
> his PhD thesis there which describes the complete software.
> I do not know about the exact status of the SW right now (his approach=20
> required some patches to the SCI driver, and it will probably be necessar=
> y to=20
> apply them to the current drivers). Very interesting approach, though.
> Other, non SCI approaches like MOSIX and the various DSM/SVM libraries al=
> so=20
> offer you some sort of global shared memory - but most do only use TCP/IP=
>  for=20
> communication.
>  Joachim
> Joachim Worringen - NEC C&C research lab St.Augustin
> fon +49-2241-9252.20 - fax .99 -
  Even hardware implementation of CPUs cache-coherence for large number
of processors may give bottleneck. Broadcasting MOESI gives high coherence
traffic, ccNUMA-systems use directory-based cache-coherence approach.
Software solutions are in general not efficient, but hardware solutions
(if they will exist) will be expensive :-(

Mikhail Kuzminsky
Zelinsky Institute of Organic Chemistry
Beowulf mailing list, Beowulf at
To change your subscription (digest mode or unsubscribe) visit

More information about the Beowulf mailing list