data storage location

Felix Rauch rauch at
Thu Sep 11 09:25:29 EDT 2003

On Wed, 10 Sep 2003, Guy Coates wrote:
> Recently we've had some success using multicast methods to distribute data
> to large numbers of nodes. udpcast is one of
> the better multicast codes, but you'll need to put some wrappers around it
> to make it useful. The multicast method is substantially faster than
> rsyncing data on large clusters.

I had two problems when testing UDPcast:

- A cheaper ATI CentreCom Fast-Ethernet switch multicasted data only
  with the speed of the slowest connected machine, even if that
  machine was *not* part of the multicast group. I had to unplug all
  10 Mbps links to speed up UDPcast above 1 MByte/s.

- With Gigabit Ethernet and a smaller cluster of 1 GHz PentiumIII
  machines I had to set a rate limitation on the sender. Otherwise the
  sender and the receivers lost synchronization and the transmission
  didn't work.

However, if you forgive me a short bit of advertising, we use the
"Dolly" programm to replicate large amounts of data in our clusters:

It replicates data with a multi-drop chain over TCP and scales quite
nicely. We only had performance problems on a switch with a rather
limited backplane, but otherwise we use it regularly in our 16- and
128-node clusters.

- Felix

Felix Rauch                      | Email: rauch at
Institute for Computer Systems   | Homepage:
ETH Zentrum / RZ H18             | Phone: +41 1 632 7489
CH - 8092 Zuerich / Switzerland  | Fax:   +41 1 632 1307

Beowulf mailing list, Beowulf at
To change your subscription (digest mode or unsubscribe) visit

More information about the Beowulf mailing list