[Beowulf] copying data between clusters

kyron kyron at neuralbs.com
Fri Mar 5 11:30:44 EST 2010

On Fri, 05 Mar 2010 11:22:14 -0500, Mike Davis <jmdavis1 at vcu.edu> wrote:
> Michael Di Domenico wrote:
>> How does one copy large (20TB) amounts of data from one cluster to
>> another?
>> Assuming that each node in the cluster can only do about 30MB/sec
>> between clusters and i want to preserve the uid/gid/timestamps, etc
> If the clusters are co-lo I wouldn't copy I would use shared storage. If

> they are not co-located I would use patience.
> Seriously though, for a one time copy, I would consider copying to an 
> external system and then physically moving that system. To do this and 
> preserve ownerships you will need to duplicate accounts and groups.

...and we are all assuming non-compressibility; otherwise, use pbzip2 ;) 
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

More information about the Beowulf mailing list