[Beowulf] RE: SATA or SCSI drives - Multiple Read/write speeds.

[Lombard, David N]

From: Bill Broadley [mailto:bill at cse.ucdavis.edu]
> 
> On Tue, Dec 09, 2003 at 07:03:12AM -0800, Lombard, David N wrote:
> > Very big pro:  You can get much higher *sustained* bandwidth levels,
> > regardless of CPU load.  ATA/PATA requires CPU involvement, and
> > bandwidth tanks under moderate CPU load.
> 
> I've heard this before, I've yet to see it.  To what do you attribute
> this advantage?  DMA scatter gather?  Higher bitrate at the read head?

Non involvement of the CPU with direct disk activities (i.e., the bits
handled by the SCSI controller) plus *way* faster CPU to handle the
high-level RAID processing v. the pokey processors found on most RAID
cards.  With multiple controllers on separate busses, I don't funnel all
my I/O through one bus.  Note again, I only discuss maximal disk
bandwidth, which means RAID-0.

> Do you have a way to quantify this *sustained* bandwidth?  Care to
share?

Direct measurement with both standard testers and applications.
Sustained means a dataset substantially larger than memory to avoid
cache effects.

> > The highest SCSI bandwidth rates I've seen first hand are 290 MB/S
for
> > IA32 and 380 MB/S for IPF. Both had two controllers on independent
PCI-X
> > busses, 6 disks for IA32 and 12 for IPF in a s/w RAID-0 config.
                                                 ==========
> Was this RAID-5?  In Hardware?  In Software?  Which controllers?
See underlining immediately above.

> Do you have any reason to believe you wouldn't see similar with the
> same number of SATA drives on 2 independent PCI-X busses?

I have no info on SATA, thus the question later on.
 
> I've seen 250 MB/sec from a relatively vanilla single controller
setup.

What file size v. memory and what CPU load *not* associated with
actually driving the I/O?

> Check out: (no I don't really trust tom's that much):
> http://www6.tomshardware.com/storage/20031114/raidcore-
> 24.html#data_transfer_diagrams_raid_5
> 
> The RaidCore manages 250 MB/sec decaying to 180MB/sec on the slower
inner
> tracks of a drive.  Certainly seems like 2 of these on seperate busses
> would have a good change of hitting the above numbers.
> 
> Note the very similar SCSI 8 drive setups are slower.

I'll look at this.

> > Does SATA reduce the CPU requirement from ATA/PATA, or is it the
same?
> > Unless it's substantially lower, you still have a system best suited
for
> > low to moderate I/O needs.
> 
> Do you have any way to quantify this?  Care to share?  I've seen many
> similar
> comments but when I actually go measure I get very similar numbers,
often
> single disks managing 40-60 MB/sec and 10% cpu, and maximum disk
transfer
> rates around 300-400 MB/sec at fairly high rates of cpu usage.

Direct measurement with both standard testers and applications.
Sustained means a dataset substantially larger than memory to avoid
cache effects.

You repeated my comment, "fairly high rates of cpu usage" -- high cpu
usage _just_to_drive_the_I/O_ meaning it's unavailable for the
application.  Also, are you quoting a burst number, that can benefit
from caching, or a sustained number, where the cache was exhausted long
ago?

The high cpu load hurts scientific/engineering apps that want to access
lots of data on disk, and burst rates are meaningless. In addition, I've
repeatedly heard that same thing from sysadmins setting up NFS servers
-- the ATA/PATA disks have too great a *negative* impact on NFS server
performance -- here the burst rates should have been more significant,
but the CPU load got in the way.

> > BTW, http://www.iozone.org/ is a nice standard I/O benchmark.  But,
as
> > mentioned earlier in this thread, app-specific benchmarking is
*always*
> > best.
> 
> Agreed.  Iozone or bonnie++ seem to do fine on large sequential file
> benchmarking I prefer postmark for replicating database like access
> patterns.

Good to know. Thanks.

-- 
David N. Lombard

My comments represent my opinions, not those of Intel.

_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf