Recent AMD760MPX chipset PCI measurements??
patrick at myri.com
Wed Feb 13 00:26:19 EST 2002
Paul Lu wrote:
> Agreed. Greg Lindahl also pointed this out in a separate reply.
> I was hoping for greater bandwidth to handle the case of a full duplex
> transfer across the Myrinet (which could, in theory, saturate the PCI bus
> given the full 4 Gb/s bandwidth of duplex operation).
> In an email to me, Greg made the point that 4 Gb/s duplex
> is difficult to achieve in practice with the current drivers.
> I haven't tried it, so I defer to Greg (and you) on this issue,
> but I was hoping to have some extra headroom (for duplex transfers
> *and* in scenarios where there is also lots of, say, disk I/O on
> the PCI bus too).
Aargh, Greg beat me here, I was still lost in my endless mail queue.
Ok, briefly: the MPX chipset is, in my opinion a very good one. It's
certainely not as good as the serverworks one, but it's still very
attractive for several reasons:
* the PCI Read and PCI Write are well balanced. It's a sign of a good
internal design. The i860 is very unbalanced for example (PCI Read is
very very low compare to Write), even with the DT tweak.
* the MPX chipset is on a different segment than the Serverworks: PIII
are OK, but you want eventually to move to the upper segment that
includes Athlon and P4. In this segment, where you start to get several
flops per cycle, the MPX chipset is just the best one so far.
* it provides more than the Myrinet one-way link speed.
* it provides just the bidirectional bandwidth achievable today with
Let me develop the last point: The link speed is 2 Gb/s. A full duplex
link provides 2+2 Gb/s, so if the PCI is big enough you should be able
to reach the peak of the PCI by doing bidirectional traffic. Everything
on the board is designed to support 450-500 MB/s bidirectional, at least
on the C cards (L9). Unfortunaltely, the max bidirectional you get right
now is 300 MB/s, even if the PCI can sustain much more. My personal
guess is that the firmware is serializing some events in the state
machine and the pipeline is damaged. It's on my plate right now and I
have finished a precise embedded log mechanism today. Hopefully, it will
tell me which transition in the state machine is late so we can fix it.
So with the current firmware, you won't see a big difference between
ServerWorks and MPX because they are both more than the one-way link
speed and because they are better or just enough the bidirectional
bandwidth. However, I expect to know much more about the bidirectional
behaviour this week to change that.
And as the Athlon pump more flops than the PIII, MPX based machines are
more interesting than ServerWorks ones. Now, the plumas chipset from
Intel and the Grand Champion from ServerWorks are supposed to arrive
real soon, it may totally change the picture.
And something I like a lot about the MPX design is that the AGP bus is
bridged from the 64/66 PCI, and the bridged is implemented on both ways:
it's possible to send/recv to/from the memory on a AGP video card. Some
cool parallel rendering and video transfert can be done...
Hope it fill the hole.
| Patrick Geoffray, Ph.D. patrick at myri.com
| Myricom, Inc. http://www.myri.com
| Cell: 865-389-8852 685 Emory Valley Rd (B)
| Phone: 865-425-0978 Oak Ridge, TN 37830
Beowulf mailing list, Beowulf at beowulf.org
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
More information about the Beowulf