[Beowulf] copying data between clusters
scott at cse.ucdavis.edu
Fri Mar 5 13:21:01 EST 2010
> I'd like to paralyze that across multiple nodes to drive the aggregate up
> I was hoping someone would pop up say, hey use this magical piece of
> software. (of which im unable to locate)..
Sounds like what we are doing with hadoop and gridftp-hdfs for our LHC
cluster. Basically you would run N gridftp servers where N is the number
of nodes (and preferably uncontended uplinks as well) on the destination
cluster. Then run something like bestman (it'll act as a director).
It is a non-trivial stack of software but it should get the job done.
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
More information about the Beowulf