large filesystem & fileserver architecture issues.

Michael T. Prinkey mprinkey at
Tue Aug 5 11:45:20 EDT 2003

On 4 Aug 2003, Nicholas Henke wrote:

We have a lot of experience with IDE RAID arrays at client sites.  The DOE
lab in Morgantown, WV has about 4 TBs of IDE RAID that we built for them.  
The performance is quite good (840 GBs, 140 MB/sec read, 80 MB/sec write)
and the price is hard to beat.  The raid array that serves home
directories to their clusters and workstations is backed up nightly to a
second raid server, similarly to your system.  To speed things along we
installed an extra gigabit card in the primary and backup servers and
connected the two directly.  The nightly backup (cp -auf via NFS) of 410
GBs take just over an hour using the dedicated gbit link.  Rsync would
probably be faster.  Without the shortcircuit gigabit link, it used to run
four or five times longer and seriously impact NFS performance for the
rest of the systems on the LAN.

Hope this helps.


Mike Prinkey
Aeolus Research, Inc.

> Hey all -- here is our situation.
> We currently have several clusters that are configured with either IBM
> x342 or Dell 2650 serves with their respective vendors SCSI RAID arrays
> hanging off of them. Each server + array is good for around 600GB after
> RAID 5 and formatting. The IBM's have the added ability to do a RAID 50
> of multiple arrays ( which seems to work & perform quite nicely ). Each
> of the servers then exports the filesystem via NFS, and is mounted on
> the nodes. The clusters range from 24 to 128 nodes. For backups we
> maintain an offline server + array that we use to rsync the data
> nightly, then use our amanda server  and tape robot to backup. We use an
> offline sync, as we need a level 0 dump every 2 weeks, and doing a level
> 0 dump of 600GB just trashes the performance on a live server. As we are
> a .edu and all of the clusters were purchased by the individual groups,
> the options we can explore have to be very cost efficient for hardware,
> and free for software.
> Now for the problem...
> A couple of our clusters are using the available filespace quite
> rapidly, and we are looking to add space. The most cost efficient
> approach we have found is to buy a IDE RAID box, like those available
> from RaidZone or PogoLinux. This allows us to use the cheap IDE systems
> as the offline sync, and use the scsi systems as online servers.
> And the questions:
> 1) Is there a better way to backup the systems without the need for an
> offline sync? 
> 2) Does anyone have experience doing RAID 50 with Dell hardware? How bad
> does it bite ?
> 3) Are there any recommended IDE RAID systems? We are not looking for
> super stellar performance, just a solid system that does it's job as an
> offline sync for backups.

Beowulf mailing list, Beowulf at
To change your subscription (digest mode or unsubscribe) visit

More information about the Beowulf mailing list