[Beowulf] Woodcrest Memory bandwidth
Keith D. Underwood
kdunder at sandia.gov
Mon Aug 14 22:48:19 EDT 2006
Um, I haven't looked closely at Woodcrest lately, but everyone does
remember that on a write, you have to fetch the cache line that you are
writing to, right? So, if you have a 10 GB/s memory system, the most a
copy should be able to do is:
read the source at 3.3 GB/s
read the destination at 3.3 GB/s
write the destination at 3.3 GB/s
What streams will report is 6.6 GB/s, which, um, matches the results
earlier in this thread.
On Mon, 2006-08-14 at 19:00 -0600, Stuart Midgley wrote:
> actually, latency determines bandwidth more than memory speed... see
> my rant(?) from a little over a year ago
> prefetch etc. help, but the limiting factor is still latency. Hence
> the opterons have significantly higher real memory bandwidth (their
> latency is about 1/3 that of xeons/p4's etc). If you look at the
> ia64's then they have even high latency again, but they can have a
> huge number of outstanding loads (from memory its >80), so their
> effective bandwidth is high.
> On 15/08/2006, at 8:10, Joe Landman wrote:
> > Hi Stu:
> > Stu Midgley wrote:
> >> sorry, forgot to reply all... don't you hate gmail's interface
> >> sometimes?
> >> What is the memory latency of the woodcrest machines? Since memory
> >> latency really determines your memory bandwidth.
> > Hmmm... not for large block sequential accesses. You can prefetch
> > these assuming enough intelligence in the code generator (heh), or
> > hardware if the memory access pattern is fairly consistent.
> > Latency really defines the random access local node GUPS, well, its
> > really more complex than that, but roughly that.
> > That said, I would like to measure this. I have an old code which
> > does
> > this, any pointers on code other people would like me to run? If
> > not too hard (e.g. less than 15 minutes) I might do a few.
> >> If Intel hasn't made any improvements in latency then the limited
> >> number of out-standing loads in the x86-64 architecture will limit
> >> the
> >> bandwidth regarless of the MB/s you throw at it.
> > Hmmm... Ok, you are implying that if your processor can consume the
> > load/store slots faster than it can launch them, and there are a
> > limited
> > number of memory operations in flight (2? as I remember, not
> > looking at
> > my notes now), it is going to be load-store pipeline limited, not
> > necessarily "bandwidth". That is, the memory system would be
> > significantly faster than the CPU can consume.
> > I haven't looked closely at the Woodcrest arch yet. Don't know
> > precisely what they are doing here and how it differs from AMD.
> > be interesting. So far I haven't been impressed with code that I
> > thought I should be really impressed with on this machine. Oddly
> > performance was about what we got out of the Core Duo on this
> > platform.
> Dr Stuart Midgley
> Industry Uptake Program Leader
> iVEC, 'The hub of advanced computing in Western Australia'
> 26 Dick Perry Avenue, Technology Park
> Kensington WA 6151
> Phone: +61 8 6436 8545
> Fax: +61 8 6436 8555
> Email: industry at ivec.org
> WWW: http://www.ivec.org
> Beowulf mailing list, Beowulf at beowulf.org
> To change your subscription (digest mode or unsubscribe) visit
Keith D. Underwood Scalable Computing Systems
Senior Member of Technical Staff Sandia National Laboratories
kdunder at sandia.gov
Beowulf mailing list, Beowulf at beowulf.org
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
More information about the Beowulf