Global Shared Memory and SCI/Dolphin

James Cownie jcownie at
Thu Jul 17 05:01:37 EDT 2003

> > >   Partially I may agree, partially - not: MPI is not the best in the
> > > sense of portability (for example, optimiziation requires knowledge
> > > of interconnect topology, which may vary from cluster to cluster,
> > > and of course from MPP to MPP computer).
> > MPI has specific support for this in Rolf Hempel's topology code,
> > which is intended to allow you to have the system help you to choose a
> > good mapping of your processes onto the processors in the system.
>   Unfortunately I do not know about that codes :-( but for the best
> optimization I'll re-build the algorithm itself to "fit" for target
> topology.

Since it's a standard part of MPI it seems a bit unfair of you to be
saying that MPI doesn't support optimisation based on topology, when
all you mean is "I didn't RTFM so I don't know about that part of
the MPI standard".

See (for instance) chapter 6 in "MPI The Complete Reference" which
discusses the MPI topology routines at some length.
This is all MPI-1 stuff too, so it's not as if it's new ;-)

Of course it may well be that none of the vendors has bothered
actually to implement the topology routines in any way which gives you
a benefit. However it still seems unfair to blame the MPI _standard_
for failings in MPI _implementations_. After all the MPI forum spent
time arguing about this, so we were aware of the issue, and trying to
give you a solution to the problem.

> > This seems to me to be _more_ than you have in a portable way on the
> > ccNUMA machines, where you have to worry about
> > 
> > 1) where every page of data lives, not just how close each process is
> >    to another one (and you have more pages than processes/threads to
> >    worry about !)
> > 
> > 2) the scheduler choosing to move your processes/threads around the
> >    machine. 
>   Yes, but "by default" I believe that they are the tasks of
> operating system, or, as maximum, the information I'm supplying to
> OS, *after* translation and linking of the program.

Having seen the effect which layout has, and the contortions people go
to to try to get their SMP codes to work efficiently in non-portable
ways (re-coding to make "first touch" happen on the "right" processor,
use of machine specific system calls for page affinity control and so
on), I remain unconvinced.

-- Jim 

James Cownie	<jcownie at>
Etnus, LLC.     +44 117 9071438

Beowulf mailing list, Beowulf at
To change your subscription (digest mode or unsubscribe) visit

More information about the Beowulf mailing list