[Beowulf] newb question - help with NUMA + mpich2 + GPUs
samuel at unimelb.edu.au
Mon Feb 20 22:31:12 EST 2012
-----BEGIN PGP SIGNED MESSAGE-----
On 21/02/12 09:52, Micha wrote:
> Things are actually more complex as this is a mixed GPU/GPU (CUDA)
> system so I'm also looking for how to effectively transfer data
> between GPUs siting on different PCIe slots and find the affinity
> between GPUs and CPUs. Also at what stage is the support for using
> MPI to copy between GPUs?
The hwloc library from the Open-MPI folks will probably help with some
It can show you which cores are near which PCI devices for instance
and lstopo is a fantastic tool for getting a quick overview of a node.
I *believe* that code is in the 1.5 series but it'd be well worth
asking the question on the open-mpi lists to get a definitive answer
from someone who knows what they're talking about. :-)
There was also a discussion on the Open-MPI devel list recently about
why MVAPICH2 appears to do better than it with GPUs (for the moment),
the summary is here:
Hope this helps!
Christopher Samuel - Senior Systems Administrator
VLSCI - Victorian Life Sciences Computation Initiative
Email: samuel at unimelb.edu.au Phone: +61 (0)3 903 55545
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
-----END PGP SIGNATURE-----
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.
More information about the Beowulf