Robert G. Brown rgb at
Tue Jun 17 09:34:15 EDT 2003

On 16 Jun 2003, Carlos Lopez Nataren wrote:

> Hello, I was wondering if anybody has made a test on whether ssh can
> make a difference over rsh when using mpi (LAM/MPICH) and it is used for
> the slave nodes???

Make a difference of what sort?

Both rsh and ssh are in the range of negligible overhead for most
sensible job distribution schema.  They are invoked once per remote
process for the sole purpose of distributing the remote processes.
AFAIK, they aren't used as any sort of meaningful (long term)
communications pipeline by any of major parallel packages, which are
designed to provide their own, highly efficient communications and job

You can look back at the list archives -- I've benchmarked ssh and rsh
several times over the last few years.  I no longer can do rsh -- it
isn't even installed on dulug systems by default, although I am yumming
to see if an rsh rpm is still out there.  However, over a rather noisy
>>wireless<< connection from my laptop the time required to establish an
ssh connection is 0.4 seconds, on average.  The ssh-derived overhead in
the time required to spawn 100 remote jobs would thus be less than a
minute absolute; even allowing rsh to be "infinitely fast" by comparison
I'd say that this is negligible.

It is a bit more (marginally) expensive when it comes to scp.  When
copying files, its bidirectional encryption algorithm significantly adds
time relative to e.g. rsh or a raw socket pipe, and is also
significantly more CPU intensive.  I honestly don't know if any of the
MPI's use rcp to copy a binary image to the remote host to execute it --
pvm does not (relying on the binary already being available on the nodes
on a predetermined path).  For a large binary, this might not be

Overall, I'd say that if you are running something where the rsh/ssh
difference matters, you should be looking at Scyld or bproc anyway as
they reduce this sort of overhead to the theoretical minimum, at the
cost of providing virtually no node security (so you really have to run
your cluster inside a head node/firewall).  In all other cases, I'd
recommend using ssh.  It is time for rsh to just go away.  ssh has lots
of advantages besides just security, and as network and CPU speeds
continue to increase the marginal cost of using it (which was negligible
years ago) will continue to decrease.

> and one more, the same as before but now using NFS over IPSEC, can
> anyone tell???

No clue here, sorry.


Robert G. Brown	             
Duke University Dept. of Physics, Box 90305
Durham, N.C. 27708-0305
Phone: 1-919-660-2567  Fax: 919-660-2525     email:rgb at

Beowulf mailing list, Beowulf at
To change your subscription (digest mode or unsubscribe) visit

More information about the Beowulf mailing list