From raq at cttc.upc.edu Wed Jun 8 13:32:45 2011 From: raq at cttc.upc.edu (Ramiro Alba) Date: Wed, 08 Jun 2011 19:32:45 +0200 Subject: [Beowulf] HPC and Free Air Cooling Message-ID: <1307554365.11247.98.camel@mundo> Hi everybody, We are about to deploy a Infiniband Cluster, using Free Air Cooling combined with HVAC equipment to deal with the Data Center cooling, and we try to contact people having similar experiences. We particularly would like to have your opinion on: - Relative Humidity control ranges (10 - 80% rather than 40 - 60%) - Impact on hardware degradation - Effects of air contaminants combined with high relative humidities I'll be glad to hear from you and your experiences on the concern. Best regards -- Ramiro Alba Centre Tecnol??gic de Tranfer??ncia de Calor http://www.cttc.upc.edu Escola T??cnica Superior d'Enginyeries Industrial i Aeron??utica de Terrassa Colom 11, E-08222, Terrassa, Barcelona, Spain Tel: (+34) 93 739 86 46 -- Aquest missatge ha estat analitzat per MailScanner a la cerca de virus i d'altres continguts perillosos, i es considera que est? net. -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. -------------- next part -------------- _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From stuartb at 4gh.net Thu Jun 9 09:32:35 2011 From: stuartb at 4gh.net (Stuart Barkley) Date: Thu, 9 Jun 2011 09:32:35 -0400 (EDT) Subject: [Beowulf] IPv6 and clusters? Message-ID: Yesterday was IPv6 day. Does IPv6 have any major place in clusters? Has anyone ever tried an IPv6 only cluster? My thought is that IPv4 is fine for internal cluster networks using rfc1918 networks. I expect the multihomed login/management nodes will eventually need IPv6 on the outside interfaces. Other thoughts? Stuart Barkley -- I've never been lost; I was once bewildered for three days, but never lost! -- Daniel Boone _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From prentice at ias.edu Thu Jun 9 10:17:47 2011 From: prentice at ias.edu (Prentice Bisbal) Date: Thu, 09 Jun 2011 10:17:47 -0400 Subject: [Beowulf] IPv6 and clusters? In-Reply-To: References: Message-ID: <4DF0D60B.1020306@ias.edu> On 06/09/2011 09:32 AM, Stuart Barkley wrote: > Yesterday was IPv6 day. > > Does IPv6 have any major place in clusters? Has anyone ever tried an > IPv6 only cluster? > > My thought is that IPv4 is fine for internal cluster networks using > rfc1918 networks. I expect the multihomed login/management nodes will > eventually need IPv6 on the outside interfaces. > The flip-side is that cluster networks are the perfect place for testing and getting familiar with IPv6 since they are almost always private networks that are completely isolated from the rest of your network, except for the master node. I wonder if there are any performance penalties for IPv6 compared to IPv4. Prentice _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From beckerjes at mail.nih.gov Thu Jun 9 10:44:22 2011 From: beckerjes at mail.nih.gov (Jesse Becker) Date: Thu, 9 Jun 2011 10:44:22 -0400 Subject: [Beowulf] IPv6 and clusters? In-Reply-To: <4DF0D60B.1020306@ias.edu> References: <4DF0D60B.1020306@ias.edu> Message-ID: <20110609144422.GY12195@mail.nih.gov> On Thu, Jun 09, 2011 at 10:17:47AM -0400, Prentice Bisbal wrote: >On 06/09/2011 09:32 AM, Stuart Barkley wrote: >> Yesterday was IPv6 day. >> >> Does IPv6 have any major place in clusters? Has anyone ever tried an >> IPv6 only cluster? >> >> My thought is that IPv4 is fine for internal cluster networks using >> rfc1918 networks. I expect the multihomed login/management nodes will >> eventually need IPv6 on the outside interfaces. >> > >The flip-side is that cluster networks are the perfect place for testing >and getting familiar with IPv6 since they are almost always private >networks that are completely isolated from the rest of your network, >except for the master node. I wonder if there are any performance >penalties for IPv6 compared to IPv4. Cisco seems to imply as much. I just recently saw one of their spec sheets for the 4900M cards[1] that shows "250 mpps for IPv4, 125 mpps for IPv6". http://www.cisco.com/en/US/prod/collateral/switches/ps5718/ps6021/ps9310/Data_Sheet_Cat_4900M.html -- Jesse Becker NHGRI Linux support (Digicon Contractor) _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From cap at nsc.liu.se Thu Jun 9 12:53:33 2011 From: cap at nsc.liu.se (Peter =?iso-8859-1?q?Kjellstr=F6m?=) Date: Thu, 9 Jun 2011 18:53:33 +0200 Subject: [Beowulf] IPv6 and clusters? In-Reply-To: References: Message-ID: <201106091853.36630.cap@nsc.liu.se> On Thursday, June 09, 2011 03:32:35 PM Stuart Barkley wrote: > Yesterday was IPv6 day. > > Does IPv6 have any major place in clusters? Has anyone ever tried an > IPv6 only cluster? Lots and lots of cluster-related software currently requires IPv4 so I suspect that this would be a challenge. One thing IPv4 makes hard (for most people) today that IPv6 will fix is to "afford" assigning global addresses to all nodes in a cluster (even though most people can live without that...). /Peter > My thought is that IPv4 is fine for internal cluster networks using > rfc1918 networks. I expect the multihomed login/management nodes will > eventually need IPv6 on the outside interfaces. > > Other thoughts? > > Stuart Barkley -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: This is a digitally signed message part. URL: -------------- next part -------------- _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From beat at 0x1b.ch Fri Jun 10 03:02:51 2011 From: beat at 0x1b.ch (Beat Rubischon) Date: Fri, 10 Jun 2011 09:02:51 +0200 Subject: [Beowulf] IPv6 and clusters? In-Reply-To: References: Message-ID: <4DF1C19B.2050209@0x1b.ch> Hello! On 09.06.11 15:32, Stuart Barkley wrote: > Does IPv6 have any major place in clusters? A good question which I asked myself many times. I'm operating my home network dual stack since 2005 and preparing regularly clusters for customers of my employer. Basically most of the tools used in clusters are still IPv4. Staring with PXE (DHCP, TFTP) and Management (IPMI). Going up to the job schedulers, parallel filesystems, MPIs. OpenMPI is an exception as it supports IPv6. Older interconnects are not IPv6 aware, Infiniband has some troubles when using large MTUs on IPoIB to improve TCP throughput - multicast drops are possible and multicast is essential for IPv6. So the first answer is no. You have to run your cluster with an IPv4 network. On the other hand there are many "cluster consolidations". People build Grids, sharing resources over multiple machines. This invents usually ugly networking hacks as the cluster networks are typically not routed. In this specific area IPv6 could be a solution. Running a cluster dual stack (private IPv4, public IPv6) would allow to interconnect clusters and distribute work and data easily. The needed services (ssh, Webservices) are IPv6 capable since the first days. So the second answer is yes. Still, I have no customer yet which was aware of the "new internet". Even large universities are not using IPv6, their policies, hardware and know how not ready. One case I know is a campus running IPv4 with a complex firewall setup: They think about deploying IPv6, but the stateless autoconfiguration and even more the privacy extensions are creating so much addresses that their firewalls are not able to handle the tables. And memory, at least with a C, F, or J brand, is pretty expensive. My conclusion about IPv6 in general: We have broken the internet in a way that the deployment of IPv6 needs a major effort. 10 years back at least academic networks were open and an additional protocol with the design of IPv6 wouldn't be a problem. But today our setups are so complex and firewalls such an essential part of the daily operation that an "open protocol" like IPv6 creates a huge amount of headache. Same is true for clusters. There are many lazy admins out there which are operation their clusters on a "never touch a running system" policy. In best case the headnode gets patches, compute nodes are often driven by a complete outdated Linux installation. Opening a large amount of identical systems to the world invents many questions in the security area. Of course, it would be possible to firewall a cluster so no access would be possible from outside. But in such a case the benefit of IPv6 and its open point to point architecture is lost. Nobody from the management will be able to pay you for a migration. So I'm still open and ready for an IPv6 deployment in a cluster. But I'm also pretty disaffected and don't believe it will happen soon. Beat -- \|/ Beat Rubischon ( 0-0 ) http://www.0x1b.ch/~beat/ oOO--(_)--OOo--------------------------------------------------- Meine Erlebnisse, Gedanken und Traeume: http://www.0x1b.ch/blog/ _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From hahn at mcmaster.ca Mon Jun 13 12:20:22 2011 From: hahn at mcmaster.ca (Mark Hahn) Date: Mon, 13 Jun 2011 12:20:22 -0400 (EDT) Subject: [Beowulf] transparent hugepage? Message-ID: Hi all, I'm at Ottawa Linux Symp, and see mentions of recent entry to the kernel of transparent hugepage support. if any of you are using this support in the HPC context, I'm interested in hearing about it. consider: a modern processor might have a TLB of as few as 1k entries (for 4k pages) - thus mapping only 4M without TLB misses...) OLS is a lot quieter this year, though last year wasn't buzzy either. not sure why - just the global economic state? _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From cbergstrom at pathscale.com Mon Jun 13 13:04:12 2011 From: cbergstrom at pathscale.com (=?ISO-8859-1?Q?=22C=2E_Bergstr=F6m=22?=) Date: Tue, 14 Jun 2011 00:04:12 +0700 Subject: [Beowulf] PathScale EKOPath goes open source Message-ID: <4DF6430C.9070604@pathscale.com> Hi all The story was leaked a little early on a popular news site, but... http://www.pathscale.com/ekopath4-open-source-announcement Special for beowulf users here's a direct download link http://c591116.r16.cf2.rackcdn.com/ekopath/nightly/Linux/ekopath-2011-06-12-installer.run Disclaimer - I'm not sure how long we'll leave this direct link up and I've not personally tested or reviewed the QA results. I have every reason to believe it's a solid build, but it's still a nightly. ./C @CTOPathScale _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From samuel at unimelb.edu.au Mon Jun 13 20:10:57 2011 From: samuel at unimelb.edu.au (Christopher Samuel) Date: Tue, 14 Jun 2011 10:10:57 +1000 Subject: [Beowulf] transparent hugepage? In-Reply-To: References: Message-ID: <4DF6A711.600@unimelb.edu.au> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 14/06/11 02:20, Mark Hahn wrote: > I'm at Ottawa Linux Symp, and see mentions of recent > entry to the kernel of transparent hugepage support. This is something I've been tracking and been very interested in testing out though I've not had a chance yet to do so due to time pressure and an unexpected stay in hospital. If I do get a chance once I've recuperated I'll certainly post any results on. cheers! Chris - -- Christopher Samuel - Senior Systems Administrator VLSCI - Victorian Life Sciences Computation Initiative Email: samuel at unimelb.edu.au Phone: +61 (0)3 903 55545 http://www.vlsci.unimelb.edu.au/ -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk32pxEACgkQO2KABBYQAh8JGgCfYUlZZzSlfUJXT/q06hX++VfX rO0AoIk1/g9imMZCX8gLi/GiWpEgJh5L =kZF8 -----END PGP SIGNATURE----- _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From samuel at unimelb.edu.au Mon Jun 13 20:22:20 2011 From: samuel at unimelb.edu.au (Christopher Samuel) Date: Tue, 14 Jun 2011 10:22:20 +1000 Subject: [Beowulf] PathScale EKOPath goes open source In-Reply-To: <4DF6430C.9070604@pathscale.com> References: <4DF6430C.9070604@pathscale.com> Message-ID: <4DF6A9BC.6010209@unimelb.edu.au> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 14/06/11 03:04, "C. Bergstr?m" wrote: > The story was leaked a little early on a popular news site, but... > http://www.pathscale.com/ekopath4-open-source-announcement Excellent news! What's the license BTW ? GPL, BSD ? > Special for beowulf users here's a direct download link > http://c591116.r16.cf2.rackcdn.com/ekopath/nightly/Linux/ekopath-2011-06-12-installer.run Thanks - I presume there'll be a source tarball at some point as well ? A query not a request, I realise you're busy.. :-) All the best, Chris - -- Christopher Samuel - Senior Systems Administrator VLSCI - Victorian Life Sciences Computation Initiative Email: samuel at unimelb.edu.au Phone: +61 (0)3 903 55545 http://www.vlsci.unimelb.edu.au/ -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk32qbsACgkQO2KABBYQAh9pugCfSUWoiaNQDigut6Oyi7329Mpv aDAAnAoxCxgUihJcgQCS9aKFgAXaTWA2 =resl -----END PGP SIGNATURE----- _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From cbergstrom at pathscale.com Mon Jun 13 20:32:10 2011 From: cbergstrom at pathscale.com (=?ISO-8859-1?Q?Christopher_Bergstr=F6m?=) Date: Tue, 14 Jun 2011 07:32:10 +0700 Subject: [Beowulf] PathScale EKOPath goes open source In-Reply-To: <4DF6A9BC.6010209@unimelb.edu.au> References: <4DF6430C.9070604@pathscale.com> <4DF6A9BC.6010209@unimelb.edu.au> Message-ID: On Tue, Jun 14, 2011 at 7:22 AM, Christopher Samuel wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On 14/06/11 03:04, "C. Bergstr?m" wrote: > >> The story was leaked a little early on a popular news site, but... >> http://www.pathscale.com/ekopath4-open-source-announcement > > Excellent news! ?What's the license BTW ? ?GPL, BSD ? Main compiler is GPLv3, v2+ and LGPL - Other parts are a mix of permissive licenses > >> Special for beowulf users here's a direct download link >> http://c591116.r16.cf2.rackcdn.com/ekopath/nightly/Linux/ekopath-2011-06-12-installer.run > > Thanks - I presume there'll be a source tarball at some > point as well ? A query not a request, I realise you're > busy.. :-) We'll be making more sources available github.com/pathscale > > All the best, You too! ./C _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From samuel at unimelb.edu.au Mon Jun 13 20:49:39 2011 From: samuel at unimelb.edu.au (Christopher Samuel) Date: Tue, 14 Jun 2011 10:49:39 +1000 Subject: [Beowulf] PathScale EKOPath goes open source In-Reply-To: References: <4DF6430C.9070604@pathscale.com> <4DF6A9BC.6010209@unimelb.edu.au> Message-ID: <4DF6B023.4030308@unimelb.edu.au> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 14/06/11 10:32, Christopher Bergstr?m wrote: > Main compiler is GPLv3, v2+ and LGPL - Other parts are > a mix of permissive licenses Great, thanks! > We'll be making more sources available github.com/pathscale Even better. :-) Thanks Chris! All the best, (another) Chris - -- Christopher Samuel - Senior Systems Administrator VLSCI - Victorian Life Sciences Computation Initiative Email: samuel at unimelb.edu.au Phone: +61 (0)3 903 55545 http://www.vlsci.unimelb.edu.au/ -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk32sCMACgkQO2KABBYQAh+b1QCeL4oBUxudQZAfz5mz+VqtmuGH axAAn3bm+AGPJ8FNcXRWKybdGYlNNGrm =X+i6 -----END PGP SIGNATURE----- _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From deadline at eadline.org Tue Jun 14 10:00:07 2011 From: deadline at eadline.org (Douglas Eadline) Date: Tue, 14 Jun 2011 10:00:07 -0400 (EDT) Subject: [Beowulf] PathScale EKOPath goes open source In-Reply-To: <4DF6430C.9070604@pathscale.com> References: <4DF6430C.9070604@pathscale.com> Message-ID: <39874.192.168.93.213.1308060007.squirrel@mail.eadline.org> Thanks! One question, how does EKOPath differ from the AMD Open64 version? -- Doug > Hi all > > The story was leaked a little early on a popular news site, but... > http://www.pathscale.com/ekopath4-open-source-announcement > > Special for beowulf users here's a direct download link > http://c591116.r16.cf2.rackcdn.com/ekopath/nightly/Linux/ekopath-2011-06-12-installer.run > > Disclaimer - I'm not sure how long we'll leave this direct link up and > I've not personally tested or reviewed the QA results. I have every > reason to believe it's a solid build, but it's still a nightly. > > ./C > > @CTOPathScale > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing > To change your subscription (digest mode or unsubscribe) visit > http://www.beowulf.org/mailman/listinfo/beowulf > > -- > This message has been scanned for viruses and > dangerous content by MailScanner, and is > believed to be clean. > -- Doug -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From hahn at mcmaster.ca Tue Jun 14 10:37:35 2011 From: hahn at mcmaster.ca (Mark Hahn) Date: Tue, 14 Jun 2011 10:37:35 -0400 (EDT) Subject: [Beowulf] PathScale EKOPath goes open source In-Reply-To: <39874.192.168.93.213.1308060007.squirrel@mail.eadline.org> References: <4DF6430C.9070604@pathscale.com> <39874.192.168.93.213.1308060007.squirrel@mail.eadline.org> Message-ID: > Thanks! One question, how does EKOPath differ from > the AMD Open64 version? I'm sure the answer is "history", since they are at least in abstract, both forks that have been separately developed for years. maybe not entirely separately, but certainly by different groups with different foci. (not to mention the fact that there seems to be at least 2 open64s.) but this begs the imvho better question: will ecopath be unified with open64? regards, mark. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From cbergstrom at pathscale.com Tue Jun 14 11:07:44 2011 From: cbergstrom at pathscale.com (=?ISO-8859-1?Q?=22C=2E_Bergstr=F6m=22?=) Date: Tue, 14 Jun 2011 22:07:44 +0700 Subject: [Beowulf] PathScale EKOPath goes open source In-Reply-To: References: <4DF6430C.9070604@pathscale.com> <39874.192.168.93.213.1308060007.squirrel@mail.eadline.org> Message-ID: <4DF77940.5010100@pathscale.com> On 06/14/11 09:37 PM, Mark Hahn wrote: >> Thanks! One question, how does EKOPath differ from >> the AMD Open64 version? > > I'm sure the answer is "history", since they are at least in abstract, > both forks that have been separately developed for years. maybe not > entirely separately, but certainly by different groups with different > foci. > (not to mention the fact that there seems to be at least 2 open64s.) > > but this begs the imvho better question: will ecopath be unified with > open64? There's more "open64's" and forks of EKOPath than I'd like, but I don't see any chance of unification in the foreseeable future. We have however been kicking around the idea to offer official support options on top of the AMD version. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From mathog at caltech.edu Tue Jun 14 12:22:02 2011 From: mathog at caltech.edu (David Mathog) Date: Tue, 14 Jun 2011 09:22:02 -0700 Subject: [Beowulf] PathScale EKOPath goes open source Message-ID: Mark Hahn wrote: >> Thanks! One question, how does EKOPath differ from >> the AMD Open64 version? >I'm sure the answer is "history", since they are at least in abstract, >both forks that have been separately developed for years. The history (possibly even the real history) is here: http://en.wikipedia.org/wiki/Open64 More useful than the history would be a performance comparison of the current versions of the Open64 variants vs. gcc vs. Intel vs. Portland compilers. Anybody seen a comparison that is not several years old? Thanks, David Mathog mathog at caltech.edu Manager, Sequence Analysis Facility, Biology Division, Caltech _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From deadline at eadline.org Tue Jun 14 15:30:37 2011 From: deadline at eadline.org (Douglas Eadline) Date: Tue, 14 Jun 2011 15:30:37 -0400 (EDT) Subject: [Beowulf] [Fwd: Re[2]: PathScale EKOPath goes open source] Message-ID: <53420.192.168.93.213.1308079837.squirrel@mail.eadline.org> Asking this for "Mikhail Kuzminsky" who can't seem to post to the list. ----------------------------------------------------- Does this (free) version support the work w/NVIDIA GPU ? Mikhail > > > > The story was leaked a little early on a popular news site, but... > > http://www.pathscale.com/ekopath4-open-source-announcement > > > > Special for beowulf users here's a direct download link > > > http://c591116.r16.cf2.rackcdn.com/ekopath/nightly/Linux/ekopath-2011-06-12-installer.run > > > > Disclaimer - I'm not sure how long we'll leave this direct link up and > > I've not personally tested or reviewed the QA results. I have every > > reason to believe it's a solid build, but it's still a nightly. > > > > ./C > > > > @CTOPathScale > > _______________________________________________ -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. -- Doug -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From cbergstrom at pathscale.com Tue Jun 14 15:39:03 2011 From: cbergstrom at pathscale.com (=?ISO-8859-1?Q?=22C=2E_Bergstr=F6m=22?=) Date: Wed, 15 Jun 2011 02:39:03 +0700 Subject: [Beowulf] [Fwd: Re[2]: PathScale EKOPath goes open source] In-Reply-To: <53420.192.168.93.213.1308079837.squirrel@mail.eadline.org> References: <53420.192.168.93.213.1308079837.squirrel@mail.eadline.org> Message-ID: <4DF7B8D7.2050404@pathscale.com> On 06/15/11 02:30 AM, Douglas Eadline wrote: > Asking this for "Mikhail Kuzminsky" > who can't seem to post to the list. > ----------------------------------------------------- > > Does this (free) version support the work w/NVIDIA GPU ? No ENZO has all the GPGPU work we've done _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From cbergstrom at pathscale.com Tue Jun 14 15:44:23 2011 From: cbergstrom at pathscale.com (=?ISO-8859-1?Q?Christopher_Bergstr=F6m?=) Date: Wed, 15 Jun 2011 02:44:23 +0700 Subject: [Beowulf] PathScale EKOPath goes open source In-Reply-To: References: Message-ID: On Tue, Jun 14, 2011 at 11:22 PM, David Mathog wrote: > Mark Hahn wrote: > >>> Thanks! One question, how does EKOPath differ from >>> the AMD Open64 version? > >>I'm sure the answer is "history", since they are at least in abstract, >>both forks that have been separately developed for years. > > > More useful than the history would be a performance comparison of the > current versions of the Open64 variants vs. gcc vs. Intel vs. Portland > compilers. ?Anybody seen a comparison that is not several years old? Please don't be mistaken EKOPath != Open64 as already mentioned. (In some cases the difference could be huge or small.. all depends on how much our code has diverged) The EKOPath nightly is available to download if anyone wants to pull and run benchmarks. More important to us is real customer code though. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From mdidomenico4 at gmail.com Tue Jun 14 15:48:11 2011 From: mdidomenico4 at gmail.com (Michael Di Domenico) Date: Tue, 14 Jun 2011 15:48:11 -0400 Subject: [Beowulf] regex expert needed Message-ID: I need to search for a keyword at the beginning of a line and then display each line up to the next blank line. is this grep'able? ala match: text text text not needed text not needed text _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From mdidomenico4 at gmail.com Tue Jun 14 15:53:40 2011 From: mdidomenico4 at gmail.com (Michael Di Domenico) Date: Tue, 14 Jun 2011 15:53:40 -0400 Subject: [Beowulf] regex expert needed In-Reply-To: References: Message-ID: apparently awk '/pattern1/,/pattern2/' does what i need thanks On Tue, Jun 14, 2011 at 3:48 PM, Michael Di Domenico wrote: > I need to search for a keyword at the beginning of a line and then > display each line up to the next blank line. ?is this grep'able? > > ala > > match: > ? text > ? text > ? text > > ?not needed text > ?not needed text > _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From reuti at staff.uni-marburg.de Tue Jun 14 16:00:32 2011 From: reuti at staff.uni-marburg.de (Reuti) Date: Tue, 14 Jun 2011 22:00:32 +0200 Subject: [Beowulf] regex expert needed In-Reply-To: References: Message-ID: <47851F68-8B51-4D63-BAE7-36B6333EF0C5@staff.uni-marburg.de> Am 14.06.2011 um 21:53 schrieb Michael Di Domenico: > apparently > > awk '/pattern1/,/pattern2/' does what i need Yep, or similar with sed and the same address type. -- Reuti > thanks > > On Tue, Jun 14, 2011 at 3:48 PM, Michael Di Domenico > wrote: >> I need to search for a keyword at the beginning of a line and then >> display each line up to the next blank line. is this grep'able? >> >> ala >> >> match: >> text >> text >> text >> >> not needed text >> not needed text >> > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing > To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf > _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From landman at scalableinformatics.com Wed Jun 15 10:27:01 2011 From: landman at scalableinformatics.com (Joe Landman) Date: Wed, 15 Jun 2011 10:27:01 -0400 Subject: [Beowulf] Anyone have experience with Cisco 6509 switches for a cluster? Message-ID: <4DF8C135.9060104@scalableinformatics.com> Hi folks A partner is rebuilding a beowulf rendering/post-production cluster, and they swapped out some smaller switches for a larger Cisco 6509 switch frame. They are encountering some issues, asked us for help. Unfortunately, I know very little about these switches and IOS (not the Apple bit), so I am hoping to get a pointer to what we should be looking for. If anyone is an IOS/Cisco expert that can help today, please contact me offlist. Here's the problem: iperf between 2 nodes, wire speed. 117 MB/s. iperf between 4 nodes, 1/2 wire speed, or 55 MB/s. As they increase the number of pairs doing iperf, performance keeps dropping. This suggests that all traffic is being serialized somehow, possibly transiting a single interface. Anyone out there ever see something like this before? Any clues as to how to handle it? Or how to diagnose/fix this? Thanks in advance. Joe -- Joseph Landman, Ph.D Founder and CEO Scalable Informatics Inc. email: landman at scalableinformatics.com web : http://scalableinformatics.com http://scalableinformatics.com/sicluster phone: +1 734 786 8423 x121 fax : +1 866 888 3112 cell : +1 734 612 4615 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From bcostescu at gmail.com Wed Jun 15 11:07:55 2011 From: bcostescu at gmail.com (Bogdan Costescu) Date: Wed, 15 Jun 2011 17:07:55 +0200 Subject: [Beowulf] Anyone have experience with Cisco 6509 switches for a cluster? In-Reply-To: <4DF8C135.9060104@scalableinformatics.com> References: <4DF8C135.9060104@scalableinformatics.com> Message-ID: On Wed, Jun 15, 2011 at 16:27, Joe Landman wrote: > ? This suggests that all traffic is being serialized somehow, possibly > transiting a single interface. I seem to remember on at least some Cisco cards ports being "bundled", organized like a 2-level tree. Each bundle of 8 ports communicate at wire speed between them, but share a common 1GB link to the upper tree level. The card itself might have had some further limitation when connecting to the backplane. Depending on the pattern of the nodes/ports that were involved in the test, this might explain the different results. Cheers, Bogdan _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From mdidomenico4 at gmail.com Wed Jun 15 19:09:22 2011 From: mdidomenico4 at gmail.com (Michael Di Domenico) Date: Wed, 15 Jun 2011 19:09:22 -0400 Subject: [Beowulf] Anyone have experience with Cisco 6509 switches for a cluster? In-Reply-To: <4DF8C135.9060104@scalableinformatics.com> References: <4DF8C135.9060104@scalableinformatics.com> Message-ID: Can we get a list of the Modules in the Switch? I'd be willing to bet they don't have an MSFC card, and are just using the SUP module On Wed, Jun 15, 2011 at 10:27 AM, Joe Landman wrote: > Hi folks > > ? A partner is rebuilding a beowulf rendering/post-production cluster, > and they swapped out some smaller switches for a larger Cisco 6509 > switch frame. ?They are encountering some issues, asked us for help. > Unfortunately, I know very little about these switches and IOS (not the > Apple bit), so I am hoping to get a pointer to what we should be looking > for. ?If anyone is an IOS/Cisco expert that can help today, please > contact me offlist. > > ? Here's the problem: > > ? iperf between 2 nodes, wire speed. ?117 MB/s. > > ? iperf between 4 nodes, 1/2 wire speed, or 55 MB/s. > > ? As they increase the number of pairs doing iperf, performance keeps > dropping. > > ? This suggests that all traffic is being serialized somehow, possibly > transiting a single interface. > > ? Anyone out there ever see something like this before? ?Any clues as > to how to handle it? ?Or how to diagnose/fix this? > > ? Thanks in advance. > > Joe > -- > Joseph Landman, Ph.D > Founder and CEO > Scalable Informatics Inc. > email: landman at scalableinformatics.com > web ?: http://scalableinformatics.com > ? ? ? ?http://scalableinformatics.com/sicluster > phone: +1 734 786 8423 x121 > fax ?: +1 866 888 3112 > cell : +1 734 612 4615 > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing > To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf > _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From brockp at umich.edu Thu Jun 16 10:20:00 2011 From: brockp at umich.edu (Brock Palen) Date: Thu, 16 Jun 2011 10:20:00 -0400 Subject: [Beowulf] Anyone have experience with Cisco 6509 switches for a cluster? In-Reply-To: <4DF8C135.9060104@scalableinformatics.com> References: <4DF8C135.9060104@scalableinformatics.com> Message-ID: We had a 6509 in one of our older clusters and got rid of it. Each group of 12 ports has some amount of shared bandwidth so you will see the results you see it is just how it is built. Though I think you should not see 55MB/s until 6 hosts on a group of 12 ports. At least in our case if you ran a nasty version of Gromacs (which performs very poorly) but flood the network our 6509 would lock up and we would have to restart every host connected, we had to have CIsco tech out and after a day of monkeying around a double super secret undocumented command fixed the lockup issue when pushing that many bytes. Looks like they may have fixed that in newer versions. Personally this experience has kept me away from Cisco gear for clusters that don't have another network like IB to take most the load. Though this was now a few years ago the situation with that hardware maybe much better. Brock Palen www.umich.edu/~brockp Center for Advanced Computing brockp at umich.edu (734)936-1985 On Jun 15, 2011, at 10:27 AM, Joe Landman wrote: > Hi folks > > A partner is rebuilding a beowulf rendering/post-production cluster, > and they swapped out some smaller switches for a larger Cisco 6509 > switch frame. They are encountering some issues, asked us for help. > Unfortunately, I know very little about these switches and IOS (not the > Apple bit), so I am hoping to get a pointer to what we should be looking > for. If anyone is an IOS/Cisco expert that can help today, please > contact me offlist. > > Here's the problem: > > iperf between 2 nodes, wire speed. 117 MB/s. > > iperf between 4 nodes, 1/2 wire speed, or 55 MB/s. > > As they increase the number of pairs doing iperf, performance keeps > dropping. > > This suggests that all traffic is being serialized somehow, possibly > transiting a single interface. > > Anyone out there ever see something like this before? Any clues as > to how to handle it? Or how to diagnose/fix this? > > Thanks in advance. > > Joe > -- > Joseph Landman, Ph.D > Founder and CEO > Scalable Informatics Inc. > email: landman at scalableinformatics.com > web : http://scalableinformatics.com > http://scalableinformatics.com/sicluster > phone: +1 734 786 8423 x121 > fax : +1 866 888 3112 > cell : +1 734 612 4615 > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing > To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf > > _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From lindahl at pbm.com Fri Jun 17 15:17:36 2011 From: lindahl at pbm.com (Greg Lindahl) Date: Fri, 17 Jun 2011 12:17:36 -0700 Subject: [Beowulf] New Tri-Labs cluster running QLogic HCAs/switches Message-ID: <20110617191735.GH11425@bx9.net> 20,000 nodes: http://www.hpcwire.com/hpcwire/2011-06-17/qlogic_wins_major_deployment_in_nnsa%27s_tri-labs_cluster.html Looks like Mellanox is finally getting some significant competition. Now if only blekko could pull the same trick off against google! -- greg _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From hahn at mcmaster.ca Sat Jun 18 11:53:19 2011 From: hahn at mcmaster.ca (Mark Hahn) Date: Sat, 18 Jun 2011 11:53:19 -0400 (EDT) Subject: [Beowulf] New Tri-Labs cluster running QLogic HCAs/switches In-Reply-To: <20110617191735.GH11425@bx9.net> References: <20110617191735.GH11425@bx9.net> Message-ID: > http://www.hpcwire.com/hpcwire/2011-06-17/qlogic_wins_major_deployment_in_nnsa%27s_tri-labs_cluster.html "...was the result of a highly competitive evaluation and benchmarking process that involved all major InfiniBand interconnect vendors." would be awesome to read the benchmark results - does anyone know where to look? > Now if only blekko could pull the same trick off against google! I did try blekko for relevant benchmarks ;) _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From mdidomenico4 at gmail.com Sat Jun 18 14:01:10 2011 From: mdidomenico4 at gmail.com (Michael Di Domenico) Date: Sat, 18 Jun 2011 14:01:10 -0400 Subject: [Beowulf] New Tri-Labs cluster running QLogic HCAs/switches In-Reply-To: References: <20110617191735.GH11425@bx9.net> Message-ID: On Sat, Jun 18, 2011 at 11:53 AM, Mark Hahn wrote: >> http://www.hpcwire.com/hpcwire/2011-06-17/qlogic_wins_major_deployment_in_nnsa%27s_tri-labs_cluster.html > > "...was the result of a highly competitive evaluation and benchmarking process > that involved all major InfiniBand interconnect vendors." > > would be awesome to read the benchmark results - > does anyone know where to look? i have limited knowledge of the deal, but i suspect or recall (can't decide which), is that most of the "benchmarks" were actual tri-labs code, not widely available benchmarks of standard applications >> Now if only blekko could pull the same trick off against google! > > I did try blekko for relevant benchmarks ;) > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing > To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf > _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From samuel at unimelb.edu.au Mon Jun 20 03:14:54 2011 From: samuel at unimelb.edu.au (Christopher Samuel) Date: Mon, 20 Jun 2011 17:14:54 +1000 Subject: [Beowulf] Japan knocks China off the #1 spot of the Top500 by 3X - a GRAPE machine ? Message-ID: <4DFEF36E.4070805@unimelb.edu.au> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 According to the NYT the new Top500 list (due out in the next few hours) will list the Japanese 'K' machine at the #1 spot at 8.2 PF. http://www.nytimes.com/2011/06/20/technology/20computer.html?_r=1 # The computer, known as "K Computer", is three times # faster than a Chinese rival that previously held the # top position, said Jack Dongarra, a professor of # electrical engineering and computer science at the # University of Tennessee at Knoxville who keeps the # official rankings of computer performance. [...] # K is made up of 672 cabinets filled with system # boards. Although considered energy-efficient, it # still uses enough electricity to power nearly # 10,000 homes at a cost of around $10 million # annually, Mr. Dongarra said. # # The research lab that houses K plans to increase # the computer?s size to 800 cabinets. That will # raise its speed, which already exceeds that of its # five closest competitors combined, Mr. Dongarra said. The excellent @HPC_Guru on Twitter said: # K Supercomputer Technical details: 80k+ SPARC64 VIIIfx # CPUs, 640K+ cores, 1PB+ RAM, 6-dimensional Mesh/Torus # interconnect But I have a reliable source who claims that this is using GRAPE cards as APUs to reach its performance without causing (another) meltdown in Japan.. cheers, Chris - -- Christopher Samuel - Senior Systems Administrator VLSCI - Victorian Life Sciences Computation Initiative Email: samuel at unimelb.edu.au Phone: +61 (0)3 903 55545 http://www.vlsci.unimelb.edu.au/ -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk3+824ACgkQO2KABBYQAh/HsQCcD1eNOe9hPG6BohYWWJ7IU74f SJAAoICP4/FOR7BSISMMI8QODOM8OHVR =V8Lo -----END PGP SIGNATURE----- _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From samuel at unimelb.edu.au Mon Jun 20 03:39:34 2011 From: samuel at unimelb.edu.au (Christopher Samuel) Date: Mon, 20 Jun 2011 17:39:34 +1000 Subject: [Beowulf] Japan knocks China off the #1 spot of the Top500 by 3X - a GRAPE machine ? In-Reply-To: <4DFEF36E.4070805@unimelb.edu.au> References: <4DFEF36E.4070805@unimelb.edu.au> Message-ID: <4DFEF936.9040404@unimelb.edu.au> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 20/06/11 17:14, Christopher Samuel wrote: > But I have a reliable source who claims that this is > using GRAPE cards as APUs to reach its performance > without causing (another) meltdown in Japan.. My reliable source appears to have been mistaken! The new Top500 press release says: # Unlike the Chinese system it displaced from the No. 1 # slot and other recent very large system, the K Computer # does not use graphics processors or other accelerators. Chips are (according to @HPC_Guru): # The K uses a SPARC6 VIIIfx CPU designed by #Fujitsu: # 8 cores, 45nm, 128 Gflops, 2.2 Gflops/Watt cheers, Chris - -- Christopher Samuel - Senior Systems Administrator VLSCI - Victorian Life Sciences Computation Initiative Email: samuel at unimelb.edu.au Phone: +61 (0)3 903 55545 http://www.vlsci.unimelb.edu.au/ -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk3++TYACgkQO2KABBYQAh8ahgCeMXhd/OTPfzhD7ECxP/h1CdOe KZ0AoIur7DhflHvVjarySs1YN4Wo+Ls4 =0riS -----END PGP SIGNATURE----- _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From hahn at mcmaster.ca Mon Jun 20 14:21:48 2011 From: hahn at mcmaster.ca (Mark Hahn) Date: Mon, 20 Jun 2011 14:21:48 -0400 (EDT) Subject: [Beowulf] Japan knocks China off the #1 spot of the Top500 by 3X - a GRAPE machine ? In-Reply-To: <4DFEF936.9040404@unimelb.edu.au> References: <4DFEF36E.4070805@unimelb.edu.au> <4DFEF936.9040404@unimelb.edu.au> Message-ID: > # The K uses a SPARC6 VIIIfx CPU designed by #Fujitsu: > # 8 cores, 45nm, 128 Gflops, 2.2 Gflops/Watt check the date: http://www.tomshardware.com/news/fujitsu-venus-sparc64-viiifx-cpu,7806.html http://www.tomshardware.com/news/sparc-venus-supercomputer-cray-riken,11386.html useful tech details: http://www.hotchips.org/archives/hc21/3_tues/HC21.25.500.ComputingAccelerators-Epub/HC21.25.51A.Maruyama-Fujitsu-Octo-Core-VIIIfx.pdf _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From tom.elken at qlogic.com Mon Jun 20 15:11:21 2011 From: tom.elken at qlogic.com (Tom Elken) Date: Mon, 20 Jun 2011 12:11:21 -0700 Subject: [Beowulf] New Tri-Labs cluster running QLogic HCAs/switches In-Reply-To: References: <20110617191735.GH11425@bx9.net> Message-ID: <35AAF1E4A771E142979F27B51793A4888838E723CE@AVEXMB1.qlogic.org> > -----Original Message----- > From: beowulf-bounces at beowulf.org [mailto:beowulf-bounces at beowulf.org] > On Behalf Of Mark Hahn > > http://www.hpcwire.com/hpcwire/2011-06- > 17/qlogic_wins_major_deployment_in_nnsa%27s_tri-labs_cluster.html > > "...was the result of a highly competitive evaluation and benchmarking > process > that involved all major InfiniBand interconnect vendors." > > would be awesome to read the benchmark results - > does anyone know where to look? The results are not published anywhere, nor will they be since they were run on pre-production hardware covered by NDAs. But I can provide some background on the benchmarks and the decision process. The TLCC2 RFP asked responders to provide ... " the compute node delivered MPI bandwidth, latency and messaging throughput benchmarks over IBA 4x QDR (or faster) HCA attached to PCIe2 (or faster) buss between two nodes utilizing 1 MPI task/node, 1 MPI task/socket and 1 MPI task/core on each node." QLogic provided these benchmark results to the OEMs/resellers that were bidding using the following benchmarks: osu_mbw_mr (the OSU Micro-Benchmark's Multi-bandwidth, Message Rate test) osu_multi_lat (Multiple latency: single or parallel latency tests between two nodes), LLNL's SQMR (Sequoia Message Rate) and osu_bw (OSU bandwidth). But some of the most important benchmarks were run later, in the final stages of making an interconnect decision, on a test cluster using the Intel Sandy Bridge CPUs with both QLogic and the competition's IB HCAs and switches. So the only difference was the IB hardware and the SW stacks to support them. To make their decision, the Tri-labs looked at technical risks, schedule risks, and ran their synthetic workload benchmarks (which also contained the microbenchmarks mentioned) on the test cluster to evaluate functionality, performance, and stability to help make their decision. -Tom Elken This message and any attached documents contain information from QLogic Corporation or its wholly-owned subsidiaries that may be confidential. If you are not the intended recipient, you may not read, copy, distribute, or use this information. If you have received this transmission in error, please notify the sender immediately by reply e-mail and then delete this message. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From deadline at eadline.org Wed Jun 22 09:20:32 2011 From: deadline at eadline.org (Douglas Eadline) Date: Wed, 22 Jun 2011 09:20:32 -0400 (EDT) Subject: [Beowulf] GP-GPU programming Message-ID: <53439.192.168.93.213.1308748832.squirrel@mail.eadline.org> For those interested, I just posted an article called "Harnessing GP-GPU Power the Easy Way" http://www.clustermonkey.net//content/view/305/1/ on ClusterMonkey.net. It provides an overview of the PGI Accelerator model for programming CPU/GPUs. i.e. you can modify existing Fortran and C programs to work on GPUs. Interesting idea. And, if you visit the site, take the GPU poll if you have not already. thanks -- Doug -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From cbergstrom at pathscale.com Wed Jun 22 09:30:30 2011 From: cbergstrom at pathscale.com (=?ISO-8859-1?Q?Christopher_Bergstr=F6m?=) Date: Wed, 22 Jun 2011 20:30:30 +0700 Subject: [Beowulf] GP-GPU programming In-Reply-To: <53439.192.168.93.213.1308748832.squirrel@mail.eadline.org> References: <53439.192.168.93.213.1308748832.squirrel@mail.eadline.org> Message-ID: On Wed, Jun 22, 2011 at 8:20 PM, Douglas Eadline wrote: > For those interested, > > I just posted an article called > > ?"Harnessing GP-GPU Power the Easy Way" > > ? http://www.clustermonkey.net//content/view/305/1/ > > on ClusterMonkey.net. It provides an > overview of the PGI Accelerator model for > programming CPU/GPUs. i.e. you can modify > existing Fortran and C programs to work > on GPUs. Interesting idea. > > And, if you visit the site, take the GPU poll > if you have not already. Have you looked at HMPP directives? it was announced as an open standard last year and now during ISC an official non-profit org to steward it forward is being formed. I'm happy to get you a beta version of our compiler if you're interested in doing a side-by-side comparison. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From landman at scalableinformatics.com Thu Jun 23 11:53:19 2011 From: landman at scalableinformatics.com (Joe Landman) Date: Thu, 23 Jun 2011 11:53:19 -0400 Subject: [Beowulf] Question: has anyone seen strange network timing bits on Nehalem/Westmere based motherboards? Message-ID: <4E03616F.70608@scalableinformatics.com> Hi folks I seem to recall, though I can't remember where, seeing about 6-8 months ago, reports of some strange network jitter for these platforms. This is Nehalem/Westmere specific, didn't impact the earlier CPUs. For some reason, my memory is of a strange driver-NUMA connection, but I am not sure at this time. Trying to trace something down, and gathering the background material first. Gigabit ethernet, Intel NICs on the motherboard. Single IOH. Thanks! Joe -- Joseph Landman, Ph.D Founder and CEO Scalable Informatics Inc. email: landman at scalableinformatics.com web : http://scalableinformatics.com http://scalableinformatics.com/sicluster phone: +1 734 786 8423 x121 fax : +1 866 888 3112 cell : +1 734 612 4615 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From eugen at leitl.org Thu Jun 23 16:00:59 2011 From: eugen at leitl.org (Eugen Leitl) Date: Thu, 23 Jun 2011 22:00:59 +0200 Subject: [Beowulf] PCI Express takes on Thunderbolt Message-ID: <20110623200059.GP26837@leitl.org> http://www.eetimes.com/General/DisplayPrintViewContent?contentItemId=4217190 PCI Express takes on Thunderbolt Rick Merritt 6/22/2011 6:06 PM EDT The PCI Special Interest Group will launch an effort in July to create a cabled version of PCI Express that will take on the Thunderbolt interconnect developed by Intel. SANTA CLARA, Calif. ? The PCI Special Interest Group will launch an effort in July to created a cabled version of PCI Express that will take on the Thunderbolt interconnect developed by Intel and Apple. Backers suggest the PCIe approach will be more open and more optimal than Thunderbolt for delivering high throughput I/O to tablets and thin notebooks. The new cable will be based on PCIe 3.0 which supports up to 8 GTransfers/second. It likely will support a maximum of four parallel lanes for throughput up to 32 Gbits/s and distances no longer than three meters. While initially focused on copper, the technology is expected to migrate to higher speed copper and optical links. The road map likely leads to a 16 GT/s version based on PCIe Gen4 in about four years as well as an optical version for longer reach and/or higher data rates at some point. The cable and connector itself are expected to be flatter than those of Thunderbolt. The PCIe cable also will support power to peripherals at levels likely less than 20W. Details of the new standard will be defined by a working group now being formed. The group is expected to deliver a standard system makers can implement in products before June 2013. The effort to write the spec could take nine to 18 months. The biggest part of the work is expected to be defining technical requirements and a new connector. The new spec is aimed at consumer uses for desktop and mobile PCs and tablets as well as their peripherals such as external storage devices. The PCI SIG has a separate cable group, chartered in 2005, that has already delivered a spec for the 2.5 and 5 GT/s versions PCIe 1.1 and 2.0, supporting distances up to eight meters and aimed for use in servers and other data center equipment. Representatives of the PCI SIG declined to comment in any way on Thunderbolt. However, the initiative is clearly aimed at similar applications including external disk and solid-state drives. "This will help proliferate PCI Express into new business opportunities," said Al Yanes, president of the PCI SIG, declining to give examples of how it will be used. "Right now we see a need from our members," Yanes said, declining to comment on Thunderbolt directly. "There are solutions [like this] in the industry--Thunderbolt is one of them, and some companies are doing own thing," he added. Comparing PCIe cable, Thunderbolt "The big issue here is proprietary versus industry standard," said Nathan Brookwood, principal of market watcher Insight64 (Saratoga, Calif.). "It's not clear third parties will have access to Thunderbolt on the same basis they get access to PCI Express," he said. Indeed, one chip maker on the show floor of the annual PCI SIG developers conference here said his company is working on a Thunderbolt design. However, the gating item to getting it completed is getting access to the technology from Intel, he said. The motivation for the PCIe cable "wasn?t spawned due to Thunderbolt, it was more about the shift to thin notebooks and tablets that means you just can't mechanically package things the same way we used to," said one source close to the effort who asked not to be named. "Thunderbolt was interesting, but it did not solve the problems we have the way we want to have them solved," the source said. Thunderbolt uses a router chip on either end of the connection to support multiple protocols and daisy chaining of devices. Apple "is fine with the extra cost of the router chips, but we don?t need [the multiprotocol support] and a couple extra chips don't make business sense for us," the source said. The use of four parallel channels and a thinner cable and connector are also expected to give the PCIe approach a leg up over Thunderbolt in throughput and ease of supporting thin systems. Intel introduced Thunderbolt in February when Apple debuted MacBook computers using it. It uses five wires to support two 10 Gbits/s bi-directional channels on a common transport layer that can carry 4x PCIe Gen 2 or DisplayPort traffic. A handful of system makers said they support Thunderbolt including executives in Canon's camera and video group. LaCie, Promise Technology and Western Digital said they will support the interconnect in external drives. A handful of other companies said they will provide support in mainly software products. Other than Apple, only Sony has so far been reported to have plans to support Thunderbolt. The PCI SIG's decision to create a competing technology suggests mainstream PC makers on the PCI SIG board such as Dell and Hewlett-Packard do not want to adopt Thunderbolt. When Thunderbolt was announced, at least one top PC maker said privately the company is moving ahead with USB 3.0 as a fast external interconnect. It is less interested in Thunderbolt than in seeing Intel more aggressively support USB 3.0, he said. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From Daniel.Kidger at bull.co.uk Fri Jun 24 11:52:57 2011 From: Daniel.Kidger at bull.co.uk (Daniel.Kidger at bull.co.uk) Date: Fri, 24 Jun 2011 16:52:57 +0100 Subject: [Beowulf] Execution time measurements In-Reply-To: References: Message-ID: Mikhail, I still think that there could be a NUMA issue here With no NUMA binding: - the one process case can migrate between cores on the core sockets - if its memory is on the first socket, then it will run a little slower when scheduled on the second socket. - with two process on a node, the first maybe be inhibited from moving to the other socket because there is already a process there consuming cpu. and vice versa. hence both will always run with local memory. Daniel From: "David Mathog" To: beowulf at beowulf.org Date: 24/05/2011 19:27 Subject: Re: [Beowulf] Execution time measurements Sent by: beowulf-bounces at beowulf.org Another message from Mikhail Kuzminsky, who for some reason or other cannot currently post directly to the list: BEGIN FORWARD 1st of all, I should mention that the effect is observed only for Opteron 2350/OpenSuSE 10.3. Execution of the same job w/the same binaries on Nehalem E5520/OpenSuSe 11.1 gives the same time for 1 and 2 simultaneously runnung jobs. Mon, 23 May 2011 12:32:33 -0700 ???????????? ???? "David Mathog" : > Mon, 23 May 2011 09:40:13 -0700 ???????????????????????? ???????? "David Mathog" > : > > > On Fri, May 20, 2011 at 02:26:31PM -0400, Mark Hahn forwarded a message: > > > > When I run 2 identical examples of the same batch job > > simultaneously, execution time of *each* job is > > > > LOWER than for single job run ! > I thought also about cpus frequency variations, but I think that null output > of > lsmod|grep freq > is enough for fixed CPU frequency. > > END FORWARD > Regarding the frequencies, better to use > cat /proc/cpuinfo | grep MHz I looked to cpuinfo, but only manually - some times (i.e. I didn't run any script w/periodical looking for CPU frequencies). All the frequencies of cores were fixed. > Did you verify that the results for each of the two simultaneous runs > are both correct? Yes, the results are the same. I looked also to number of iterations etc. But I'll check outputs again. >Ideally, tweak some parameter so they are slightly > different from each other. But I don't understand - if I change slightly some of input parameters, what may it give ? > David Mathog > mathog at caltech.edu > Manager, Sequence Analysis Facility, Biology Division, Caltech Fri, 20 May 2011 20:11:15 -0400 message from Serguei Patchkovskii : > Suse 10.3 is quite old; it uses a kernel which is less than perfect at scheduling jobs and allocating resources for >NUMA systems. Try running your test job using: > > numactl --cpunodebind=0 --membind=0 g98 numactl w/all things bound to node 1 gives "big" execution time ( 1 day 4 hours; 2 simultaneous jobs run faster), for forcing different nodes for cpu and memory - execution time is even higher (+1 h). Therefore effect observed don't looks as result of numa allocations :-( Mikhail END FORWARD My point about the two different parameter sets on the jobs was to determine if the two were truly independent, or if they might not be interacting with each other through checkpoint files or shared memory, or the like. Regards, David Mathog mathog at caltech.edu Manager, Sequence Analysis Facility, Biology Division, Caltech _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From tom.elken at qlogic.com Fri Jun 24 12:43:30 2011 From: tom.elken at qlogic.com (Tom Elken) Date: Fri, 24 Jun 2011 09:43:30 -0700 Subject: [Beowulf] Question: has anyone seen strange network timing bits on Nehalem/Westmere based motherboards? In-Reply-To: <4E03616F.70608@scalableinformatics.com> References: <4E03616F.70608@scalableinformatics.com> Message-ID: <35AAF1E4A771E142979F27B51793A4888838F22AEE@AVEXMB1.qlogic.org> > I seem to recall, though I can't remember where, seeing about 6-8 > months ago, reports of some strange network jitter for these platforms. > This is Nehalem/Westmere specific, didn't impact the earlier CPUs. > For some reason, my memory is of a strange driver-NUMA connection, but > I am not sure at this time. Hi Joe, Not sure if this is the issue, but for best latency benchmark performance, we recommend turning all C-states off in the BIOS. If it cannot be done there, you can add the kernel boot option: processor.max_cstate=0 and reboot. With C-states turned on, there is high and very variable MPI/IB latency measured if the benchmark is running on CPU0, the default, since extra OS interrupts are being serviced on CPU0. Once you've turned them off, You can look at these files: /proc/acpi/processor/*/power to make sure that only state C1 is being used in the list of states at the end. I think we've also seen this affecting throughput benchmarks. Again, I'm not sure if this affects GigE. -Tom > > Trying to trace something down, and gathering the background material > first. Gigabit ethernet, Intel NICs on the motherboard. Single IOH. > > Thanks! > > Joe > -- > Joseph Landman, Ph.D > Founder and CEO > Scalable Informatics Inc. > email: landman at scalableinformatics.com > web : http://scalableinformatics.com > http://scalableinformatics.com/sicluster > phone: +1 734 786 8423 x121 > fax : +1 866 888 3112 > cell : +1 734 612 4615 > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin > Computing > To change your subscription (digest mode or unsubscribe) visit > http://www.beowulf.org/mailman/listinfo/beowulf This message and any attached documents contain information from QLogic Corporation or its wholly-owned subsidiaries that may be confidential. If you are not the intended recipient, you may not read, copy, distribute, or use this information. If you have received this transmission in error, please notify the sender immediately by reply e-mail and then delete this message. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From orion at cora.nwra.com Fri Jun 24 16:54:35 2011 From: orion at cora.nwra.com (Orion Poplawski) Date: Fri, 24 Jun 2011 14:54:35 -0600 Subject: [Beowulf] Facebook's servers Message-ID: <4E04F98B.4010808@cora.nwra.com> http://opencompute.org/ I guess not commodity (yet), but interesting. -- Orion Poplawski Technical Manager 303-415-9701 x222 NWRA/CoRA Division FAX: 303-415-9702 3380 Mitchell Lane orion at cora.nwra.com Boulder, CO 80301 http://www.cora.nwra.com _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From raysonlogin at gmail.com Mon Jun 27 11:27:00 2011 From: raysonlogin at gmail.com (Rayson Ho) Date: Mon, 27 Jun 2011 11:27:00 -0400 Subject: [Beowulf] K computer powered by Open MPI Message-ID: >From the presentation, "Programming on the K computer", P. 11: http://www.fujitsu.com/downloads/TC/sc10/programming-on-k-computer.pdf Open MPI based: * Open Standard, Open Source, Multi-Platform including PC Cluster. * Adding extension to Open MPI for "Tofu" interconnect And it was confirmed that Open MPI (actually, Open MPI + Fujitsu's code for Tofu) was used in LINPACK to achieve 8 PFLOPS and get the #1 in TOP500: http://blogs.cisco.com/performance/open-mpi-powers-8-petaflops/ Other opensource stuff used by K: * Lustre - see the presentation, "An Overview of Fujitsu's Lustre Based File System": http://www.olcf.ornl.gov/wp-content/events/lug2011/4-12-2011/230-300_Shinji_Sumimoto_LUG2011-FJ-20110407-pub.pdf * Linux on SPARC - should make Ellison happy as he needs to ship SPARC T4 with Oracle Linux: http://www.pcworld.com/article/212564/ellison_oracle_enterprise_linux_coming_to_sparc.html Rayson _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From mathog at caltech.edu Mon Jun 27 11:29:20 2011 From: mathog at caltech.edu (David Mathog) Date: Mon, 27 Jun 2011 08:29:20 -0700 Subject: [Beowulf] Facebook's servers Message-ID: Orion Poplawski wrote > > I guess not commodity (yet), but interesting. > Thanks for posting that, it was interesting. The AMD board spec says it uses "address parity" memory, which I guess implies ECC, since elsewhere in the spec it discusses handling ECC. They implement reboot on lan (ROL, repurposing the WOL packet). The Intel board spec memory section doesn't say anything about parity/ECC, only that the memory must be registered, but again, elsewhere it discusses logging ECC errors, so I guess ECC is assumed. This board also implements ROL. The ROL sections seem to imply that both NICs will respond to such a packet, which could have some "interesting" security implications, at least for those of us where one interface is public. I can imagine a nightmare scenario where one machine is corrupted, it turns itself into a DHCP server and starts spraying WOL packets out onto the network, quickly converting more machines, which can carry on the same trick via their private interfaces. Normal WOL isn't nearly so hazardous on a public interface, it only becomes a security risk if the attacker has both access to another host on the subnet and some method that can remotely force the attacked system to do an orderly shutdown. Regards, David Mathog mathog at caltech.edu Manager, Sequence Analysis Facility, Biology Division, Caltech _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From orion at cora.nwra.com Thu Jun 30 16:33:21 2011 From: orion at cora.nwra.com (Orion Poplawski) Date: Thu, 30 Jun 2011 14:33:21 -0600 Subject: [Beowulf] Suggestions for used workhorse servers Message-ID: <4E0CDD91.2090405@cora.nwra.com> One can find some pretty inexpensive older servers on eBay that probably could yield a decent $/flop ratio. I was wondering if people here had suggestions for classic workhorse servers - basic 1U boxes that did/do pretty well be are a couple years old at this point. Thanks! -- Orion Poplawski Technical Manager 303-415-9701 x222 NWRA/CoRA Division FAX: 303-415-9702 3380 Mitchell Lane orion at cora.nwra.com Boulder, CO 80301 http://www.cora.nwra.com _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From prentice at ias.edu Thu Jun 30 16:48:39 2011 From: prentice at ias.edu (Prentice Bisbal) Date: Thu, 30 Jun 2011 16:48:39 -0400 Subject: [Beowulf] Suggestions for used workhorse servers In-Reply-To: <4E0CDD91.2090405@cora.nwra.com> References: <4E0CDD91.2090405@cora.nwra.com> Message-ID: <4E0CE127.9060605@ias.edu> I've always been pleased with the HP ProLiant systems, like the DL385 models. The seemed pretty reliable to me. I'd trust one of those over a Dell PowerEdge, however I'm sure you'll get as many opinions as there are subscribers on this list. -- Prentice On 06/30/2011 04:33 PM, Orion Poplawski wrote: > One can find some pretty inexpensive older servers on eBay that probably could > yield a decent $/flop ratio. I was wondering if people here had suggestions > for classic workhorse servers - basic 1U boxes that did/do pretty well be are > a couple years old at this point. > > Thanks! > _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From james.p.lux at jpl.nasa.gov Thu Jun 30 17:15:34 2011 From: james.p.lux at jpl.nasa.gov (Lux, Jim (337C)) Date: Thu, 30 Jun 2011 14:15:34 -0700 Subject: [Beowulf] Suggestions for used workhorse servers In-Reply-To: <4E0CDD91.2090405@cora.nwra.com> Message-ID: One might make a nice small cluster for learning purposes. With 8 nodes, you could do a lot of experimenting. Even 4 nodes works, but with 8, if your parallelization works, you get a pretty dramatic speedup. And, when you screw up, and need to reinstall all the software everywhere, 4-8 nodes is manageable by hand. You could also, if you have extra network cards, experiment with things like different interconnect architectures. There is significant value in a stack of boxes which you "own" and don't have to account for the use of (or lack), for that sort of "fooling around" For production purposes, you're probably better off buying newer computers: Power consumption, hassles, etc. On 6/30/11 1:33 PM, "Orion Poplawski" wrote: >One can find some pretty inexpensive older servers on eBay that probably >could >yield a decent $/flop ratio. I was wondering if people here had >suggestions >for classic workhorse servers - basic 1U boxes that did/do pretty well be >are >a couple years old at this point. > >Thanks! > >-- >Orion Poplawski >Technical Manager 303-415-9701 x222 >NWRA/CoRA Division FAX: 303-415-9702 >3380 Mitchell Lane orion at cora.nwra.com >Boulder, CO 80301 http://www.cora.nwra.com >_______________________________________________ >Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing >To change your subscription (digest mode or unsubscribe) visit >http://www.beowulf.org/mailman/listinfo/beowulf _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From raq at cttc.upc.edu Wed Jun 8 13:32:45 2011 From: raq at cttc.upc.edu (Ramiro Alba) Date: Wed, 08 Jun 2011 19:32:45 +0200 Subject: [Beowulf] HPC and Free Air Cooling Message-ID: <1307554365.11247.98.camel@mundo> Hi everybody, We are about to deploy a Infiniband Cluster, using Free Air Cooling combined with HVAC equipment to deal with the Data Center cooling, and we try to contact people having similar experiences. We particularly would like to have your opinion on: - Relative Humidity control ranges (10 - 80% rather than 40 - 60%) - Impact on hardware degradation - Effects of air contaminants combined with high relative humidities I'll be glad to hear from you and your experiences on the concern. Best regards -- Ramiro Alba Centre Tecnol??gic de Tranfer??ncia de Calor http://www.cttc.upc.edu Escola T??cnica Superior d'Enginyeries Industrial i Aeron??utica de Terrassa Colom 11, E-08222, Terrassa, Barcelona, Spain Tel: (+34) 93 739 86 46 -- Aquest missatge ha estat analitzat per MailScanner a la cerca de virus i d'altres continguts perillosos, i es considera que est? net. -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. -------------- next part -------------- _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From stuartb at 4gh.net Thu Jun 9 09:32:35 2011 From: stuartb at 4gh.net (Stuart Barkley) Date: Thu, 9 Jun 2011 09:32:35 -0400 (EDT) Subject: [Beowulf] IPv6 and clusters? Message-ID: Yesterday was IPv6 day. Does IPv6 have any major place in clusters? Has anyone ever tried an IPv6 only cluster? My thought is that IPv4 is fine for internal cluster networks using rfc1918 networks. I expect the multihomed login/management nodes will eventually need IPv6 on the outside interfaces. Other thoughts? Stuart Barkley -- I've never been lost; I was once bewildered for three days, but never lost! -- Daniel Boone _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From prentice at ias.edu Thu Jun 9 10:17:47 2011 From: prentice at ias.edu (Prentice Bisbal) Date: Thu, 09 Jun 2011 10:17:47 -0400 Subject: [Beowulf] IPv6 and clusters? In-Reply-To: References: Message-ID: <4DF0D60B.1020306@ias.edu> On 06/09/2011 09:32 AM, Stuart Barkley wrote: > Yesterday was IPv6 day. > > Does IPv6 have any major place in clusters? Has anyone ever tried an > IPv6 only cluster? > > My thought is that IPv4 is fine for internal cluster networks using > rfc1918 networks. I expect the multihomed login/management nodes will > eventually need IPv6 on the outside interfaces. > The flip-side is that cluster networks are the perfect place for testing and getting familiar with IPv6 since they are almost always private networks that are completely isolated from the rest of your network, except for the master node. I wonder if there are any performance penalties for IPv6 compared to IPv4. Prentice _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From beckerjes at mail.nih.gov Thu Jun 9 10:44:22 2011 From: beckerjes at mail.nih.gov (Jesse Becker) Date: Thu, 9 Jun 2011 10:44:22 -0400 Subject: [Beowulf] IPv6 and clusters? In-Reply-To: <4DF0D60B.1020306@ias.edu> References: <4DF0D60B.1020306@ias.edu> Message-ID: <20110609144422.GY12195@mail.nih.gov> On Thu, Jun 09, 2011 at 10:17:47AM -0400, Prentice Bisbal wrote: >On 06/09/2011 09:32 AM, Stuart Barkley wrote: >> Yesterday was IPv6 day. >> >> Does IPv6 have any major place in clusters? Has anyone ever tried an >> IPv6 only cluster? >> >> My thought is that IPv4 is fine for internal cluster networks using >> rfc1918 networks. I expect the multihomed login/management nodes will >> eventually need IPv6 on the outside interfaces. >> > >The flip-side is that cluster networks are the perfect place for testing >and getting familiar with IPv6 since they are almost always private >networks that are completely isolated from the rest of your network, >except for the master node. I wonder if there are any performance >penalties for IPv6 compared to IPv4. Cisco seems to imply as much. I just recently saw one of their spec sheets for the 4900M cards[1] that shows "250 mpps for IPv4, 125 mpps for IPv6". http://www.cisco.com/en/US/prod/collateral/switches/ps5718/ps6021/ps9310/Data_Sheet_Cat_4900M.html -- Jesse Becker NHGRI Linux support (Digicon Contractor) _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From cap at nsc.liu.se Thu Jun 9 12:53:33 2011 From: cap at nsc.liu.se (Peter =?iso-8859-1?q?Kjellstr=F6m?=) Date: Thu, 9 Jun 2011 18:53:33 +0200 Subject: [Beowulf] IPv6 and clusters? In-Reply-To: References: Message-ID: <201106091853.36630.cap@nsc.liu.se> On Thursday, June 09, 2011 03:32:35 PM Stuart Barkley wrote: > Yesterday was IPv6 day. > > Does IPv6 have any major place in clusters? Has anyone ever tried an > IPv6 only cluster? Lots and lots of cluster-related software currently requires IPv4 so I suspect that this would be a challenge. One thing IPv4 makes hard (for most people) today that IPv6 will fix is to "afford" assigning global addresses to all nodes in a cluster (even though most people can live without that...). /Peter > My thought is that IPv4 is fine for internal cluster networks using > rfc1918 networks. I expect the multihomed login/management nodes will > eventually need IPv6 on the outside interfaces. > > Other thoughts? > > Stuart Barkley -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: This is a digitally signed message part. URL: -------------- next part -------------- _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From beat at 0x1b.ch Fri Jun 10 03:02:51 2011 From: beat at 0x1b.ch (Beat Rubischon) Date: Fri, 10 Jun 2011 09:02:51 +0200 Subject: [Beowulf] IPv6 and clusters? In-Reply-To: References: Message-ID: <4DF1C19B.2050209@0x1b.ch> Hello! On 09.06.11 15:32, Stuart Barkley wrote: > Does IPv6 have any major place in clusters? A good question which I asked myself many times. I'm operating my home network dual stack since 2005 and preparing regularly clusters for customers of my employer. Basically most of the tools used in clusters are still IPv4. Staring with PXE (DHCP, TFTP) and Management (IPMI). Going up to the job schedulers, parallel filesystems, MPIs. OpenMPI is an exception as it supports IPv6. Older interconnects are not IPv6 aware, Infiniband has some troubles when using large MTUs on IPoIB to improve TCP throughput - multicast drops are possible and multicast is essential for IPv6. So the first answer is no. You have to run your cluster with an IPv4 network. On the other hand there are many "cluster consolidations". People build Grids, sharing resources over multiple machines. This invents usually ugly networking hacks as the cluster networks are typically not routed. In this specific area IPv6 could be a solution. Running a cluster dual stack (private IPv4, public IPv6) would allow to interconnect clusters and distribute work and data easily. The needed services (ssh, Webservices) are IPv6 capable since the first days. So the second answer is yes. Still, I have no customer yet which was aware of the "new internet". Even large universities are not using IPv6, their policies, hardware and know how not ready. One case I know is a campus running IPv4 with a complex firewall setup: They think about deploying IPv6, but the stateless autoconfiguration and even more the privacy extensions are creating so much addresses that their firewalls are not able to handle the tables. And memory, at least with a C, F, or J brand, is pretty expensive. My conclusion about IPv6 in general: We have broken the internet in a way that the deployment of IPv6 needs a major effort. 10 years back at least academic networks were open and an additional protocol with the design of IPv6 wouldn't be a problem. But today our setups are so complex and firewalls such an essential part of the daily operation that an "open protocol" like IPv6 creates a huge amount of headache. Same is true for clusters. There are many lazy admins out there which are operation their clusters on a "never touch a running system" policy. In best case the headnode gets patches, compute nodes are often driven by a complete outdated Linux installation. Opening a large amount of identical systems to the world invents many questions in the security area. Of course, it would be possible to firewall a cluster so no access would be possible from outside. But in such a case the benefit of IPv6 and its open point to point architecture is lost. Nobody from the management will be able to pay you for a migration. So I'm still open and ready for an IPv6 deployment in a cluster. But I'm also pretty disaffected and don't believe it will happen soon. Beat -- \|/ Beat Rubischon ( 0-0 ) http://www.0x1b.ch/~beat/ oOO--(_)--OOo--------------------------------------------------- Meine Erlebnisse, Gedanken und Traeume: http://www.0x1b.ch/blog/ _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From hahn at mcmaster.ca Mon Jun 13 12:20:22 2011 From: hahn at mcmaster.ca (Mark Hahn) Date: Mon, 13 Jun 2011 12:20:22 -0400 (EDT) Subject: [Beowulf] transparent hugepage? Message-ID: Hi all, I'm at Ottawa Linux Symp, and see mentions of recent entry to the kernel of transparent hugepage support. if any of you are using this support in the HPC context, I'm interested in hearing about it. consider: a modern processor might have a TLB of as few as 1k entries (for 4k pages) - thus mapping only 4M without TLB misses...) OLS is a lot quieter this year, though last year wasn't buzzy either. not sure why - just the global economic state? _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From cbergstrom at pathscale.com Mon Jun 13 13:04:12 2011 From: cbergstrom at pathscale.com (=?ISO-8859-1?Q?=22C=2E_Bergstr=F6m=22?=) Date: Tue, 14 Jun 2011 00:04:12 +0700 Subject: [Beowulf] PathScale EKOPath goes open source Message-ID: <4DF6430C.9070604@pathscale.com> Hi all The story was leaked a little early on a popular news site, but... http://www.pathscale.com/ekopath4-open-source-announcement Special for beowulf users here's a direct download link http://c591116.r16.cf2.rackcdn.com/ekopath/nightly/Linux/ekopath-2011-06-12-installer.run Disclaimer - I'm not sure how long we'll leave this direct link up and I've not personally tested or reviewed the QA results. I have every reason to believe it's a solid build, but it's still a nightly. ./C @CTOPathScale _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From samuel at unimelb.edu.au Mon Jun 13 20:10:57 2011 From: samuel at unimelb.edu.au (Christopher Samuel) Date: Tue, 14 Jun 2011 10:10:57 +1000 Subject: [Beowulf] transparent hugepage? In-Reply-To: References: Message-ID: <4DF6A711.600@unimelb.edu.au> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 14/06/11 02:20, Mark Hahn wrote: > I'm at Ottawa Linux Symp, and see mentions of recent > entry to the kernel of transparent hugepage support. This is something I've been tracking and been very interested in testing out though I've not had a chance yet to do so due to time pressure and an unexpected stay in hospital. If I do get a chance once I've recuperated I'll certainly post any results on. cheers! Chris - -- Christopher Samuel - Senior Systems Administrator VLSCI - Victorian Life Sciences Computation Initiative Email: samuel at unimelb.edu.au Phone: +61 (0)3 903 55545 http://www.vlsci.unimelb.edu.au/ -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk32pxEACgkQO2KABBYQAh8JGgCfYUlZZzSlfUJXT/q06hX++VfX rO0AoIk1/g9imMZCX8gLi/GiWpEgJh5L =kZF8 -----END PGP SIGNATURE----- _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From samuel at unimelb.edu.au Mon Jun 13 20:22:20 2011 From: samuel at unimelb.edu.au (Christopher Samuel) Date: Tue, 14 Jun 2011 10:22:20 +1000 Subject: [Beowulf] PathScale EKOPath goes open source In-Reply-To: <4DF6430C.9070604@pathscale.com> References: <4DF6430C.9070604@pathscale.com> Message-ID: <4DF6A9BC.6010209@unimelb.edu.au> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 14/06/11 03:04, "C. Bergstr?m" wrote: > The story was leaked a little early on a popular news site, but... > http://www.pathscale.com/ekopath4-open-source-announcement Excellent news! What's the license BTW ? GPL, BSD ? > Special for beowulf users here's a direct download link > http://c591116.r16.cf2.rackcdn.com/ekopath/nightly/Linux/ekopath-2011-06-12-installer.run Thanks - I presume there'll be a source tarball at some point as well ? A query not a request, I realise you're busy.. :-) All the best, Chris - -- Christopher Samuel - Senior Systems Administrator VLSCI - Victorian Life Sciences Computation Initiative Email: samuel at unimelb.edu.au Phone: +61 (0)3 903 55545 http://www.vlsci.unimelb.edu.au/ -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk32qbsACgkQO2KABBYQAh9pugCfSUWoiaNQDigut6Oyi7329Mpv aDAAnAoxCxgUihJcgQCS9aKFgAXaTWA2 =resl -----END PGP SIGNATURE----- _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From cbergstrom at pathscale.com Mon Jun 13 20:32:10 2011 From: cbergstrom at pathscale.com (=?ISO-8859-1?Q?Christopher_Bergstr=F6m?=) Date: Tue, 14 Jun 2011 07:32:10 +0700 Subject: [Beowulf] PathScale EKOPath goes open source In-Reply-To: <4DF6A9BC.6010209@unimelb.edu.au> References: <4DF6430C.9070604@pathscale.com> <4DF6A9BC.6010209@unimelb.edu.au> Message-ID: On Tue, Jun 14, 2011 at 7:22 AM, Christopher Samuel wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On 14/06/11 03:04, "C. Bergstr?m" wrote: > >> The story was leaked a little early on a popular news site, but... >> http://www.pathscale.com/ekopath4-open-source-announcement > > Excellent news! ?What's the license BTW ? ?GPL, BSD ? Main compiler is GPLv3, v2+ and LGPL - Other parts are a mix of permissive licenses > >> Special for beowulf users here's a direct download link >> http://c591116.r16.cf2.rackcdn.com/ekopath/nightly/Linux/ekopath-2011-06-12-installer.run > > Thanks - I presume there'll be a source tarball at some > point as well ? A query not a request, I realise you're > busy.. :-) We'll be making more sources available github.com/pathscale > > All the best, You too! ./C _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From samuel at unimelb.edu.au Mon Jun 13 20:49:39 2011 From: samuel at unimelb.edu.au (Christopher Samuel) Date: Tue, 14 Jun 2011 10:49:39 +1000 Subject: [Beowulf] PathScale EKOPath goes open source In-Reply-To: References: <4DF6430C.9070604@pathscale.com> <4DF6A9BC.6010209@unimelb.edu.au> Message-ID: <4DF6B023.4030308@unimelb.edu.au> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 14/06/11 10:32, Christopher Bergstr?m wrote: > Main compiler is GPLv3, v2+ and LGPL - Other parts are > a mix of permissive licenses Great, thanks! > We'll be making more sources available github.com/pathscale Even better. :-) Thanks Chris! All the best, (another) Chris - -- Christopher Samuel - Senior Systems Administrator VLSCI - Victorian Life Sciences Computation Initiative Email: samuel at unimelb.edu.au Phone: +61 (0)3 903 55545 http://www.vlsci.unimelb.edu.au/ -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk32sCMACgkQO2KABBYQAh+b1QCeL4oBUxudQZAfz5mz+VqtmuGH axAAn3bm+AGPJ8FNcXRWKybdGYlNNGrm =X+i6 -----END PGP SIGNATURE----- _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From deadline at eadline.org Tue Jun 14 10:00:07 2011 From: deadline at eadline.org (Douglas Eadline) Date: Tue, 14 Jun 2011 10:00:07 -0400 (EDT) Subject: [Beowulf] PathScale EKOPath goes open source In-Reply-To: <4DF6430C.9070604@pathscale.com> References: <4DF6430C.9070604@pathscale.com> Message-ID: <39874.192.168.93.213.1308060007.squirrel@mail.eadline.org> Thanks! One question, how does EKOPath differ from the AMD Open64 version? -- Doug > Hi all > > The story was leaked a little early on a popular news site, but... > http://www.pathscale.com/ekopath4-open-source-announcement > > Special for beowulf users here's a direct download link > http://c591116.r16.cf2.rackcdn.com/ekopath/nightly/Linux/ekopath-2011-06-12-installer.run > > Disclaimer - I'm not sure how long we'll leave this direct link up and > I've not personally tested or reviewed the QA results. I have every > reason to believe it's a solid build, but it's still a nightly. > > ./C > > @CTOPathScale > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing > To change your subscription (digest mode or unsubscribe) visit > http://www.beowulf.org/mailman/listinfo/beowulf > > -- > This message has been scanned for viruses and > dangerous content by MailScanner, and is > believed to be clean. > -- Doug -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From hahn at mcmaster.ca Tue Jun 14 10:37:35 2011 From: hahn at mcmaster.ca (Mark Hahn) Date: Tue, 14 Jun 2011 10:37:35 -0400 (EDT) Subject: [Beowulf] PathScale EKOPath goes open source In-Reply-To: <39874.192.168.93.213.1308060007.squirrel@mail.eadline.org> References: <4DF6430C.9070604@pathscale.com> <39874.192.168.93.213.1308060007.squirrel@mail.eadline.org> Message-ID: > Thanks! One question, how does EKOPath differ from > the AMD Open64 version? I'm sure the answer is "history", since they are at least in abstract, both forks that have been separately developed for years. maybe not entirely separately, but certainly by different groups with different foci. (not to mention the fact that there seems to be at least 2 open64s.) but this begs the imvho better question: will ecopath be unified with open64? regards, mark. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From cbergstrom at pathscale.com Tue Jun 14 11:07:44 2011 From: cbergstrom at pathscale.com (=?ISO-8859-1?Q?=22C=2E_Bergstr=F6m=22?=) Date: Tue, 14 Jun 2011 22:07:44 +0700 Subject: [Beowulf] PathScale EKOPath goes open source In-Reply-To: References: <4DF6430C.9070604@pathscale.com> <39874.192.168.93.213.1308060007.squirrel@mail.eadline.org> Message-ID: <4DF77940.5010100@pathscale.com> On 06/14/11 09:37 PM, Mark Hahn wrote: >> Thanks! One question, how does EKOPath differ from >> the AMD Open64 version? > > I'm sure the answer is "history", since they are at least in abstract, > both forks that have been separately developed for years. maybe not > entirely separately, but certainly by different groups with different > foci. > (not to mention the fact that there seems to be at least 2 open64s.) > > but this begs the imvho better question: will ecopath be unified with > open64? There's more "open64's" and forks of EKOPath than I'd like, but I don't see any chance of unification in the foreseeable future. We have however been kicking around the idea to offer official support options on top of the AMD version. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From mathog at caltech.edu Tue Jun 14 12:22:02 2011 From: mathog at caltech.edu (David Mathog) Date: Tue, 14 Jun 2011 09:22:02 -0700 Subject: [Beowulf] PathScale EKOPath goes open source Message-ID: Mark Hahn wrote: >> Thanks! One question, how does EKOPath differ from >> the AMD Open64 version? >I'm sure the answer is "history", since they are at least in abstract, >both forks that have been separately developed for years. The history (possibly even the real history) is here: http://en.wikipedia.org/wiki/Open64 More useful than the history would be a performance comparison of the current versions of the Open64 variants vs. gcc vs. Intel vs. Portland compilers. Anybody seen a comparison that is not several years old? Thanks, David Mathog mathog at caltech.edu Manager, Sequence Analysis Facility, Biology Division, Caltech _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From deadline at eadline.org Tue Jun 14 15:30:37 2011 From: deadline at eadline.org (Douglas Eadline) Date: Tue, 14 Jun 2011 15:30:37 -0400 (EDT) Subject: [Beowulf] [Fwd: Re[2]: PathScale EKOPath goes open source] Message-ID: <53420.192.168.93.213.1308079837.squirrel@mail.eadline.org> Asking this for "Mikhail Kuzminsky" who can't seem to post to the list. ----------------------------------------------------- Does this (free) version support the work w/NVIDIA GPU ? Mikhail > > > > The story was leaked a little early on a popular news site, but... > > http://www.pathscale.com/ekopath4-open-source-announcement > > > > Special for beowulf users here's a direct download link > > > http://c591116.r16.cf2.rackcdn.com/ekopath/nightly/Linux/ekopath-2011-06-12-installer.run > > > > Disclaimer - I'm not sure how long we'll leave this direct link up and > > I've not personally tested or reviewed the QA results. I have every > > reason to believe it's a solid build, but it's still a nightly. > > > > ./C > > > > @CTOPathScale > > _______________________________________________ -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. -- Doug -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From cbergstrom at pathscale.com Tue Jun 14 15:39:03 2011 From: cbergstrom at pathscale.com (=?ISO-8859-1?Q?=22C=2E_Bergstr=F6m=22?=) Date: Wed, 15 Jun 2011 02:39:03 +0700 Subject: [Beowulf] [Fwd: Re[2]: PathScale EKOPath goes open source] In-Reply-To: <53420.192.168.93.213.1308079837.squirrel@mail.eadline.org> References: <53420.192.168.93.213.1308079837.squirrel@mail.eadline.org> Message-ID: <4DF7B8D7.2050404@pathscale.com> On 06/15/11 02:30 AM, Douglas Eadline wrote: > Asking this for "Mikhail Kuzminsky" > who can't seem to post to the list. > ----------------------------------------------------- > > Does this (free) version support the work w/NVIDIA GPU ? No ENZO has all the GPGPU work we've done _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From cbergstrom at pathscale.com Tue Jun 14 15:44:23 2011 From: cbergstrom at pathscale.com (=?ISO-8859-1?Q?Christopher_Bergstr=F6m?=) Date: Wed, 15 Jun 2011 02:44:23 +0700 Subject: [Beowulf] PathScale EKOPath goes open source In-Reply-To: References: Message-ID: On Tue, Jun 14, 2011 at 11:22 PM, David Mathog wrote: > Mark Hahn wrote: > >>> Thanks! One question, how does EKOPath differ from >>> the AMD Open64 version? > >>I'm sure the answer is "history", since they are at least in abstract, >>both forks that have been separately developed for years. > > > More useful than the history would be a performance comparison of the > current versions of the Open64 variants vs. gcc vs. Intel vs. Portland > compilers. ?Anybody seen a comparison that is not several years old? Please don't be mistaken EKOPath != Open64 as already mentioned. (In some cases the difference could be huge or small.. all depends on how much our code has diverged) The EKOPath nightly is available to download if anyone wants to pull and run benchmarks. More important to us is real customer code though. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From mdidomenico4 at gmail.com Tue Jun 14 15:48:11 2011 From: mdidomenico4 at gmail.com (Michael Di Domenico) Date: Tue, 14 Jun 2011 15:48:11 -0400 Subject: [Beowulf] regex expert needed Message-ID: I need to search for a keyword at the beginning of a line and then display each line up to the next blank line. is this grep'able? ala match: text text text not needed text not needed text _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From mdidomenico4 at gmail.com Tue Jun 14 15:53:40 2011 From: mdidomenico4 at gmail.com (Michael Di Domenico) Date: Tue, 14 Jun 2011 15:53:40 -0400 Subject: [Beowulf] regex expert needed In-Reply-To: References: Message-ID: apparently awk '/pattern1/,/pattern2/' does what i need thanks On Tue, Jun 14, 2011 at 3:48 PM, Michael Di Domenico wrote: > I need to search for a keyword at the beginning of a line and then > display each line up to the next blank line. ?is this grep'able? > > ala > > match: > ? text > ? text > ? text > > ?not needed text > ?not needed text > _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From reuti at staff.uni-marburg.de Tue Jun 14 16:00:32 2011 From: reuti at staff.uni-marburg.de (Reuti) Date: Tue, 14 Jun 2011 22:00:32 +0200 Subject: [Beowulf] regex expert needed In-Reply-To: References: Message-ID: <47851F68-8B51-4D63-BAE7-36B6333EF0C5@staff.uni-marburg.de> Am 14.06.2011 um 21:53 schrieb Michael Di Domenico: > apparently > > awk '/pattern1/,/pattern2/' does what i need Yep, or similar with sed and the same address type. -- Reuti > thanks > > On Tue, Jun 14, 2011 at 3:48 PM, Michael Di Domenico > wrote: >> I need to search for a keyword at the beginning of a line and then >> display each line up to the next blank line. is this grep'able? >> >> ala >> >> match: >> text >> text >> text >> >> not needed text >> not needed text >> > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing > To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf > _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From landman at scalableinformatics.com Wed Jun 15 10:27:01 2011 From: landman at scalableinformatics.com (Joe Landman) Date: Wed, 15 Jun 2011 10:27:01 -0400 Subject: [Beowulf] Anyone have experience with Cisco 6509 switches for a cluster? Message-ID: <4DF8C135.9060104@scalableinformatics.com> Hi folks A partner is rebuilding a beowulf rendering/post-production cluster, and they swapped out some smaller switches for a larger Cisco 6509 switch frame. They are encountering some issues, asked us for help. Unfortunately, I know very little about these switches and IOS (not the Apple bit), so I am hoping to get a pointer to what we should be looking for. If anyone is an IOS/Cisco expert that can help today, please contact me offlist. Here's the problem: iperf between 2 nodes, wire speed. 117 MB/s. iperf between 4 nodes, 1/2 wire speed, or 55 MB/s. As they increase the number of pairs doing iperf, performance keeps dropping. This suggests that all traffic is being serialized somehow, possibly transiting a single interface. Anyone out there ever see something like this before? Any clues as to how to handle it? Or how to diagnose/fix this? Thanks in advance. Joe -- Joseph Landman, Ph.D Founder and CEO Scalable Informatics Inc. email: landman at scalableinformatics.com web : http://scalableinformatics.com http://scalableinformatics.com/sicluster phone: +1 734 786 8423 x121 fax : +1 866 888 3112 cell : +1 734 612 4615 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From bcostescu at gmail.com Wed Jun 15 11:07:55 2011 From: bcostescu at gmail.com (Bogdan Costescu) Date: Wed, 15 Jun 2011 17:07:55 +0200 Subject: [Beowulf] Anyone have experience with Cisco 6509 switches for a cluster? In-Reply-To: <4DF8C135.9060104@scalableinformatics.com> References: <4DF8C135.9060104@scalableinformatics.com> Message-ID: On Wed, Jun 15, 2011 at 16:27, Joe Landman wrote: > ? This suggests that all traffic is being serialized somehow, possibly > transiting a single interface. I seem to remember on at least some Cisco cards ports being "bundled", organized like a 2-level tree. Each bundle of 8 ports communicate at wire speed between them, but share a common 1GB link to the upper tree level. The card itself might have had some further limitation when connecting to the backplane. Depending on the pattern of the nodes/ports that were involved in the test, this might explain the different results. Cheers, Bogdan _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From mdidomenico4 at gmail.com Wed Jun 15 19:09:22 2011 From: mdidomenico4 at gmail.com (Michael Di Domenico) Date: Wed, 15 Jun 2011 19:09:22 -0400 Subject: [Beowulf] Anyone have experience with Cisco 6509 switches for a cluster? In-Reply-To: <4DF8C135.9060104@scalableinformatics.com> References: <4DF8C135.9060104@scalableinformatics.com> Message-ID: Can we get a list of the Modules in the Switch? I'd be willing to bet they don't have an MSFC card, and are just using the SUP module On Wed, Jun 15, 2011 at 10:27 AM, Joe Landman wrote: > Hi folks > > ? A partner is rebuilding a beowulf rendering/post-production cluster, > and they swapped out some smaller switches for a larger Cisco 6509 > switch frame. ?They are encountering some issues, asked us for help. > Unfortunately, I know very little about these switches and IOS (not the > Apple bit), so I am hoping to get a pointer to what we should be looking > for. ?If anyone is an IOS/Cisco expert that can help today, please > contact me offlist. > > ? Here's the problem: > > ? iperf between 2 nodes, wire speed. ?117 MB/s. > > ? iperf between 4 nodes, 1/2 wire speed, or 55 MB/s. > > ? As they increase the number of pairs doing iperf, performance keeps > dropping. > > ? This suggests that all traffic is being serialized somehow, possibly > transiting a single interface. > > ? Anyone out there ever see something like this before? ?Any clues as > to how to handle it? ?Or how to diagnose/fix this? > > ? Thanks in advance. > > Joe > -- > Joseph Landman, Ph.D > Founder and CEO > Scalable Informatics Inc. > email: landman at scalableinformatics.com > web ?: http://scalableinformatics.com > ? ? ? ?http://scalableinformatics.com/sicluster > phone: +1 734 786 8423 x121 > fax ?: +1 866 888 3112 > cell : +1 734 612 4615 > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing > To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf > _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From brockp at umich.edu Thu Jun 16 10:20:00 2011 From: brockp at umich.edu (Brock Palen) Date: Thu, 16 Jun 2011 10:20:00 -0400 Subject: [Beowulf] Anyone have experience with Cisco 6509 switches for a cluster? In-Reply-To: <4DF8C135.9060104@scalableinformatics.com> References: <4DF8C135.9060104@scalableinformatics.com> Message-ID: We had a 6509 in one of our older clusters and got rid of it. Each group of 12 ports has some amount of shared bandwidth so you will see the results you see it is just how it is built. Though I think you should not see 55MB/s until 6 hosts on a group of 12 ports. At least in our case if you ran a nasty version of Gromacs (which performs very poorly) but flood the network our 6509 would lock up and we would have to restart every host connected, we had to have CIsco tech out and after a day of monkeying around a double super secret undocumented command fixed the lockup issue when pushing that many bytes. Looks like they may have fixed that in newer versions. Personally this experience has kept me away from Cisco gear for clusters that don't have another network like IB to take most the load. Though this was now a few years ago the situation with that hardware maybe much better. Brock Palen www.umich.edu/~brockp Center for Advanced Computing brockp at umich.edu (734)936-1985 On Jun 15, 2011, at 10:27 AM, Joe Landman wrote: > Hi folks > > A partner is rebuilding a beowulf rendering/post-production cluster, > and they swapped out some smaller switches for a larger Cisco 6509 > switch frame. They are encountering some issues, asked us for help. > Unfortunately, I know very little about these switches and IOS (not the > Apple bit), so I am hoping to get a pointer to what we should be looking > for. If anyone is an IOS/Cisco expert that can help today, please > contact me offlist. > > Here's the problem: > > iperf between 2 nodes, wire speed. 117 MB/s. > > iperf between 4 nodes, 1/2 wire speed, or 55 MB/s. > > As they increase the number of pairs doing iperf, performance keeps > dropping. > > This suggests that all traffic is being serialized somehow, possibly > transiting a single interface. > > Anyone out there ever see something like this before? Any clues as > to how to handle it? Or how to diagnose/fix this? > > Thanks in advance. > > Joe > -- > Joseph Landman, Ph.D > Founder and CEO > Scalable Informatics Inc. > email: landman at scalableinformatics.com > web : http://scalableinformatics.com > http://scalableinformatics.com/sicluster > phone: +1 734 786 8423 x121 > fax : +1 866 888 3112 > cell : +1 734 612 4615 > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing > To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf > > _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From lindahl at pbm.com Fri Jun 17 15:17:36 2011 From: lindahl at pbm.com (Greg Lindahl) Date: Fri, 17 Jun 2011 12:17:36 -0700 Subject: [Beowulf] New Tri-Labs cluster running QLogic HCAs/switches Message-ID: <20110617191735.GH11425@bx9.net> 20,000 nodes: http://www.hpcwire.com/hpcwire/2011-06-17/qlogic_wins_major_deployment_in_nnsa%27s_tri-labs_cluster.html Looks like Mellanox is finally getting some significant competition. Now if only blekko could pull the same trick off against google! -- greg _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From hahn at mcmaster.ca Sat Jun 18 11:53:19 2011 From: hahn at mcmaster.ca (Mark Hahn) Date: Sat, 18 Jun 2011 11:53:19 -0400 (EDT) Subject: [Beowulf] New Tri-Labs cluster running QLogic HCAs/switches In-Reply-To: <20110617191735.GH11425@bx9.net> References: <20110617191735.GH11425@bx9.net> Message-ID: > http://www.hpcwire.com/hpcwire/2011-06-17/qlogic_wins_major_deployment_in_nnsa%27s_tri-labs_cluster.html "...was the result of a highly competitive evaluation and benchmarking process that involved all major InfiniBand interconnect vendors." would be awesome to read the benchmark results - does anyone know where to look? > Now if only blekko could pull the same trick off against google! I did try blekko for relevant benchmarks ;) _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From mdidomenico4 at gmail.com Sat Jun 18 14:01:10 2011 From: mdidomenico4 at gmail.com (Michael Di Domenico) Date: Sat, 18 Jun 2011 14:01:10 -0400 Subject: [Beowulf] New Tri-Labs cluster running QLogic HCAs/switches In-Reply-To: References: <20110617191735.GH11425@bx9.net> Message-ID: On Sat, Jun 18, 2011 at 11:53 AM, Mark Hahn wrote: >> http://www.hpcwire.com/hpcwire/2011-06-17/qlogic_wins_major_deployment_in_nnsa%27s_tri-labs_cluster.html > > "...was the result of a highly competitive evaluation and benchmarking process > that involved all major InfiniBand interconnect vendors." > > would be awesome to read the benchmark results - > does anyone know where to look? i have limited knowledge of the deal, but i suspect or recall (can't decide which), is that most of the "benchmarks" were actual tri-labs code, not widely available benchmarks of standard applications >> Now if only blekko could pull the same trick off against google! > > I did try blekko for relevant benchmarks ;) > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing > To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf > _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From samuel at unimelb.edu.au Mon Jun 20 03:14:54 2011 From: samuel at unimelb.edu.au (Christopher Samuel) Date: Mon, 20 Jun 2011 17:14:54 +1000 Subject: [Beowulf] Japan knocks China off the #1 spot of the Top500 by 3X - a GRAPE machine ? Message-ID: <4DFEF36E.4070805@unimelb.edu.au> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 According to the NYT the new Top500 list (due out in the next few hours) will list the Japanese 'K' machine at the #1 spot at 8.2 PF. http://www.nytimes.com/2011/06/20/technology/20computer.html?_r=1 # The computer, known as "K Computer", is three times # faster than a Chinese rival that previously held the # top position, said Jack Dongarra, a professor of # electrical engineering and computer science at the # University of Tennessee at Knoxville who keeps the # official rankings of computer performance. [...] # K is made up of 672 cabinets filled with system # boards. Although considered energy-efficient, it # still uses enough electricity to power nearly # 10,000 homes at a cost of around $10 million # annually, Mr. Dongarra said. # # The research lab that houses K plans to increase # the computer?s size to 800 cabinets. That will # raise its speed, which already exceeds that of its # five closest competitors combined, Mr. Dongarra said. The excellent @HPC_Guru on Twitter said: # K Supercomputer Technical details: 80k+ SPARC64 VIIIfx # CPUs, 640K+ cores, 1PB+ RAM, 6-dimensional Mesh/Torus # interconnect But I have a reliable source who claims that this is using GRAPE cards as APUs to reach its performance without causing (another) meltdown in Japan.. cheers, Chris - -- Christopher Samuel - Senior Systems Administrator VLSCI - Victorian Life Sciences Computation Initiative Email: samuel at unimelb.edu.au Phone: +61 (0)3 903 55545 http://www.vlsci.unimelb.edu.au/ -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk3+824ACgkQO2KABBYQAh/HsQCcD1eNOe9hPG6BohYWWJ7IU74f SJAAoICP4/FOR7BSISMMI8QODOM8OHVR =V8Lo -----END PGP SIGNATURE----- _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From samuel at unimelb.edu.au Mon Jun 20 03:39:34 2011 From: samuel at unimelb.edu.au (Christopher Samuel) Date: Mon, 20 Jun 2011 17:39:34 +1000 Subject: [Beowulf] Japan knocks China off the #1 spot of the Top500 by 3X - a GRAPE machine ? In-Reply-To: <4DFEF36E.4070805@unimelb.edu.au> References: <4DFEF36E.4070805@unimelb.edu.au> Message-ID: <4DFEF936.9040404@unimelb.edu.au> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 20/06/11 17:14, Christopher Samuel wrote: > But I have a reliable source who claims that this is > using GRAPE cards as APUs to reach its performance > without causing (another) meltdown in Japan.. My reliable source appears to have been mistaken! The new Top500 press release says: # Unlike the Chinese system it displaced from the No. 1 # slot and other recent very large system, the K Computer # does not use graphics processors or other accelerators. Chips are (according to @HPC_Guru): # The K uses a SPARC6 VIIIfx CPU designed by #Fujitsu: # 8 cores, 45nm, 128 Gflops, 2.2 Gflops/Watt cheers, Chris - -- Christopher Samuel - Senior Systems Administrator VLSCI - Victorian Life Sciences Computation Initiative Email: samuel at unimelb.edu.au Phone: +61 (0)3 903 55545 http://www.vlsci.unimelb.edu.au/ -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk3++TYACgkQO2KABBYQAh8ahgCeMXhd/OTPfzhD7ECxP/h1CdOe KZ0AoIur7DhflHvVjarySs1YN4Wo+Ls4 =0riS -----END PGP SIGNATURE----- _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From hahn at mcmaster.ca Mon Jun 20 14:21:48 2011 From: hahn at mcmaster.ca (Mark Hahn) Date: Mon, 20 Jun 2011 14:21:48 -0400 (EDT) Subject: [Beowulf] Japan knocks China off the #1 spot of the Top500 by 3X - a GRAPE machine ? In-Reply-To: <4DFEF936.9040404@unimelb.edu.au> References: <4DFEF36E.4070805@unimelb.edu.au> <4DFEF936.9040404@unimelb.edu.au> Message-ID: > # The K uses a SPARC6 VIIIfx CPU designed by #Fujitsu: > # 8 cores, 45nm, 128 Gflops, 2.2 Gflops/Watt check the date: http://www.tomshardware.com/news/fujitsu-venus-sparc64-viiifx-cpu,7806.html http://www.tomshardware.com/news/sparc-venus-supercomputer-cray-riken,11386.html useful tech details: http://www.hotchips.org/archives/hc21/3_tues/HC21.25.500.ComputingAccelerators-Epub/HC21.25.51A.Maruyama-Fujitsu-Octo-Core-VIIIfx.pdf _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From tom.elken at qlogic.com Mon Jun 20 15:11:21 2011 From: tom.elken at qlogic.com (Tom Elken) Date: Mon, 20 Jun 2011 12:11:21 -0700 Subject: [Beowulf] New Tri-Labs cluster running QLogic HCAs/switches In-Reply-To: References: <20110617191735.GH11425@bx9.net> Message-ID: <35AAF1E4A771E142979F27B51793A4888838E723CE@AVEXMB1.qlogic.org> > -----Original Message----- > From: beowulf-bounces at beowulf.org [mailto:beowulf-bounces at beowulf.org] > On Behalf Of Mark Hahn > > http://www.hpcwire.com/hpcwire/2011-06- > 17/qlogic_wins_major_deployment_in_nnsa%27s_tri-labs_cluster.html > > "...was the result of a highly competitive evaluation and benchmarking > process > that involved all major InfiniBand interconnect vendors." > > would be awesome to read the benchmark results - > does anyone know where to look? The results are not published anywhere, nor will they be since they were run on pre-production hardware covered by NDAs. But I can provide some background on the benchmarks and the decision process. The TLCC2 RFP asked responders to provide ... " the compute node delivered MPI bandwidth, latency and messaging throughput benchmarks over IBA 4x QDR (or faster) HCA attached to PCIe2 (or faster) buss between two nodes utilizing 1 MPI task/node, 1 MPI task/socket and 1 MPI task/core on each node." QLogic provided these benchmark results to the OEMs/resellers that were bidding using the following benchmarks: osu_mbw_mr (the OSU Micro-Benchmark's Multi-bandwidth, Message Rate test) osu_multi_lat (Multiple latency: single or parallel latency tests between two nodes), LLNL's SQMR (Sequoia Message Rate) and osu_bw (OSU bandwidth). But some of the most important benchmarks were run later, in the final stages of making an interconnect decision, on a test cluster using the Intel Sandy Bridge CPUs with both QLogic and the competition's IB HCAs and switches. So the only difference was the IB hardware and the SW stacks to support them. To make their decision, the Tri-labs looked at technical risks, schedule risks, and ran their synthetic workload benchmarks (which also contained the microbenchmarks mentioned) on the test cluster to evaluate functionality, performance, and stability to help make their decision. -Tom Elken This message and any attached documents contain information from QLogic Corporation or its wholly-owned subsidiaries that may be confidential. If you are not the intended recipient, you may not read, copy, distribute, or use this information. If you have received this transmission in error, please notify the sender immediately by reply e-mail and then delete this message. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From deadline at eadline.org Wed Jun 22 09:20:32 2011 From: deadline at eadline.org (Douglas Eadline) Date: Wed, 22 Jun 2011 09:20:32 -0400 (EDT) Subject: [Beowulf] GP-GPU programming Message-ID: <53439.192.168.93.213.1308748832.squirrel@mail.eadline.org> For those interested, I just posted an article called "Harnessing GP-GPU Power the Easy Way" http://www.clustermonkey.net//content/view/305/1/ on ClusterMonkey.net. It provides an overview of the PGI Accelerator model for programming CPU/GPUs. i.e. you can modify existing Fortran and C programs to work on GPUs. Interesting idea. And, if you visit the site, take the GPU poll if you have not already. thanks -- Doug -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From cbergstrom at pathscale.com Wed Jun 22 09:30:30 2011 From: cbergstrom at pathscale.com (=?ISO-8859-1?Q?Christopher_Bergstr=F6m?=) Date: Wed, 22 Jun 2011 20:30:30 +0700 Subject: [Beowulf] GP-GPU programming In-Reply-To: <53439.192.168.93.213.1308748832.squirrel@mail.eadline.org> References: <53439.192.168.93.213.1308748832.squirrel@mail.eadline.org> Message-ID: On Wed, Jun 22, 2011 at 8:20 PM, Douglas Eadline wrote: > For those interested, > > I just posted an article called > > ?"Harnessing GP-GPU Power the Easy Way" > > ? http://www.clustermonkey.net//content/view/305/1/ > > on ClusterMonkey.net. It provides an > overview of the PGI Accelerator model for > programming CPU/GPUs. i.e. you can modify > existing Fortran and C programs to work > on GPUs. Interesting idea. > > And, if you visit the site, take the GPU poll > if you have not already. Have you looked at HMPP directives? it was announced as an open standard last year and now during ISC an official non-profit org to steward it forward is being formed. I'm happy to get you a beta version of our compiler if you're interested in doing a side-by-side comparison. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From landman at scalableinformatics.com Thu Jun 23 11:53:19 2011 From: landman at scalableinformatics.com (Joe Landman) Date: Thu, 23 Jun 2011 11:53:19 -0400 Subject: [Beowulf] Question: has anyone seen strange network timing bits on Nehalem/Westmere based motherboards? Message-ID: <4E03616F.70608@scalableinformatics.com> Hi folks I seem to recall, though I can't remember where, seeing about 6-8 months ago, reports of some strange network jitter for these platforms. This is Nehalem/Westmere specific, didn't impact the earlier CPUs. For some reason, my memory is of a strange driver-NUMA connection, but I am not sure at this time. Trying to trace something down, and gathering the background material first. Gigabit ethernet, Intel NICs on the motherboard. Single IOH. Thanks! Joe -- Joseph Landman, Ph.D Founder and CEO Scalable Informatics Inc. email: landman at scalableinformatics.com web : http://scalableinformatics.com http://scalableinformatics.com/sicluster phone: +1 734 786 8423 x121 fax : +1 866 888 3112 cell : +1 734 612 4615 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From eugen at leitl.org Thu Jun 23 16:00:59 2011 From: eugen at leitl.org (Eugen Leitl) Date: Thu, 23 Jun 2011 22:00:59 +0200 Subject: [Beowulf] PCI Express takes on Thunderbolt Message-ID: <20110623200059.GP26837@leitl.org> http://www.eetimes.com/General/DisplayPrintViewContent?contentItemId=4217190 PCI Express takes on Thunderbolt Rick Merritt 6/22/2011 6:06 PM EDT The PCI Special Interest Group will launch an effort in July to create a cabled version of PCI Express that will take on the Thunderbolt interconnect developed by Intel. SANTA CLARA, Calif. ? The PCI Special Interest Group will launch an effort in July to created a cabled version of PCI Express that will take on the Thunderbolt interconnect developed by Intel and Apple. Backers suggest the PCIe approach will be more open and more optimal than Thunderbolt for delivering high throughput I/O to tablets and thin notebooks. The new cable will be based on PCIe 3.0 which supports up to 8 GTransfers/second. It likely will support a maximum of four parallel lanes for throughput up to 32 Gbits/s and distances no longer than three meters. While initially focused on copper, the technology is expected to migrate to higher speed copper and optical links. The road map likely leads to a 16 GT/s version based on PCIe Gen4 in about four years as well as an optical version for longer reach and/or higher data rates at some point. The cable and connector itself are expected to be flatter than those of Thunderbolt. The PCIe cable also will support power to peripherals at levels likely less than 20W. Details of the new standard will be defined by a working group now being formed. The group is expected to deliver a standard system makers can implement in products before June 2013. The effort to write the spec could take nine to 18 months. The biggest part of the work is expected to be defining technical requirements and a new connector. The new spec is aimed at consumer uses for desktop and mobile PCs and tablets as well as their peripherals such as external storage devices. The PCI SIG has a separate cable group, chartered in 2005, that has already delivered a spec for the 2.5 and 5 GT/s versions PCIe 1.1 and 2.0, supporting distances up to eight meters and aimed for use in servers and other data center equipment. Representatives of the PCI SIG declined to comment in any way on Thunderbolt. However, the initiative is clearly aimed at similar applications including external disk and solid-state drives. "This will help proliferate PCI Express into new business opportunities," said Al Yanes, president of the PCI SIG, declining to give examples of how it will be used. "Right now we see a need from our members," Yanes said, declining to comment on Thunderbolt directly. "There are solutions [like this] in the industry--Thunderbolt is one of them, and some companies are doing own thing," he added. Comparing PCIe cable, Thunderbolt "The big issue here is proprietary versus industry standard," said Nathan Brookwood, principal of market watcher Insight64 (Saratoga, Calif.). "It's not clear third parties will have access to Thunderbolt on the same basis they get access to PCI Express," he said. Indeed, one chip maker on the show floor of the annual PCI SIG developers conference here said his company is working on a Thunderbolt design. However, the gating item to getting it completed is getting access to the technology from Intel, he said. The motivation for the PCIe cable "wasn?t spawned due to Thunderbolt, it was more about the shift to thin notebooks and tablets that means you just can't mechanically package things the same way we used to," said one source close to the effort who asked not to be named. "Thunderbolt was interesting, but it did not solve the problems we have the way we want to have them solved," the source said. Thunderbolt uses a router chip on either end of the connection to support multiple protocols and daisy chaining of devices. Apple "is fine with the extra cost of the router chips, but we don?t need [the multiprotocol support] and a couple extra chips don't make business sense for us," the source said. The use of four parallel channels and a thinner cable and connector are also expected to give the PCIe approach a leg up over Thunderbolt in throughput and ease of supporting thin systems. Intel introduced Thunderbolt in February when Apple debuted MacBook computers using it. It uses five wires to support two 10 Gbits/s bi-directional channels on a common transport layer that can carry 4x PCIe Gen 2 or DisplayPort traffic. A handful of system makers said they support Thunderbolt including executives in Canon's camera and video group. LaCie, Promise Technology and Western Digital said they will support the interconnect in external drives. A handful of other companies said they will provide support in mainly software products. Other than Apple, only Sony has so far been reported to have plans to support Thunderbolt. The PCI SIG's decision to create a competing technology suggests mainstream PC makers on the PCI SIG board such as Dell and Hewlett-Packard do not want to adopt Thunderbolt. When Thunderbolt was announced, at least one top PC maker said privately the company is moving ahead with USB 3.0 as a fast external interconnect. It is less interested in Thunderbolt than in seeing Intel more aggressively support USB 3.0, he said. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From Daniel.Kidger at bull.co.uk Fri Jun 24 11:52:57 2011 From: Daniel.Kidger at bull.co.uk (Daniel.Kidger at bull.co.uk) Date: Fri, 24 Jun 2011 16:52:57 +0100 Subject: [Beowulf] Execution time measurements In-Reply-To: References: Message-ID: Mikhail, I still think that there could be a NUMA issue here With no NUMA binding: - the one process case can migrate between cores on the core sockets - if its memory is on the first socket, then it will run a little slower when scheduled on the second socket. - with two process on a node, the first maybe be inhibited from moving to the other socket because there is already a process there consuming cpu. and vice versa. hence both will always run with local memory. Daniel From: "David Mathog" To: beowulf at beowulf.org Date: 24/05/2011 19:27 Subject: Re: [Beowulf] Execution time measurements Sent by: beowulf-bounces at beowulf.org Another message from Mikhail Kuzminsky, who for some reason or other cannot currently post directly to the list: BEGIN FORWARD 1st of all, I should mention that the effect is observed only for Opteron 2350/OpenSuSE 10.3. Execution of the same job w/the same binaries on Nehalem E5520/OpenSuSe 11.1 gives the same time for 1 and 2 simultaneously runnung jobs. Mon, 23 May 2011 12:32:33 -0700 ???????????? ???? "David Mathog" : > Mon, 23 May 2011 09:40:13 -0700 ???????????????????????? ???????? "David Mathog" > : > > > On Fri, May 20, 2011 at 02:26:31PM -0400, Mark Hahn forwarded a message: > > > > When I run 2 identical examples of the same batch job > > simultaneously, execution time of *each* job is > > > > LOWER than for single job run ! > I thought also about cpus frequency variations, but I think that null output > of > lsmod|grep freq > is enough for fixed CPU frequency. > > END FORWARD > Regarding the frequencies, better to use > cat /proc/cpuinfo | grep MHz I looked to cpuinfo, but only manually - some times (i.e. I didn't run any script w/periodical looking for CPU frequencies). All the frequencies of cores were fixed. > Did you verify that the results for each of the two simultaneous runs > are both correct? Yes, the results are the same. I looked also to number of iterations etc. But I'll check outputs again. >Ideally, tweak some parameter so they are slightly > different from each other. But I don't understand - if I change slightly some of input parameters, what may it give ? > David Mathog > mathog at caltech.edu > Manager, Sequence Analysis Facility, Biology Division, Caltech Fri, 20 May 2011 20:11:15 -0400 message from Serguei Patchkovskii : > Suse 10.3 is quite old; it uses a kernel which is less than perfect at scheduling jobs and allocating resources for >NUMA systems. Try running your test job using: > > numactl --cpunodebind=0 --membind=0 g98 numactl w/all things bound to node 1 gives "big" execution time ( 1 day 4 hours; 2 simultaneous jobs run faster), for forcing different nodes for cpu and memory - execution time is even higher (+1 h). Therefore effect observed don't looks as result of numa allocations :-( Mikhail END FORWARD My point about the two different parameter sets on the jobs was to determine if the two were truly independent, or if they might not be interacting with each other through checkpoint files or shared memory, or the like. Regards, David Mathog mathog at caltech.edu Manager, Sequence Analysis Facility, Biology Division, Caltech _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From tom.elken at qlogic.com Fri Jun 24 12:43:30 2011 From: tom.elken at qlogic.com (Tom Elken) Date: Fri, 24 Jun 2011 09:43:30 -0700 Subject: [Beowulf] Question: has anyone seen strange network timing bits on Nehalem/Westmere based motherboards? In-Reply-To: <4E03616F.70608@scalableinformatics.com> References: <4E03616F.70608@scalableinformatics.com> Message-ID: <35AAF1E4A771E142979F27B51793A4888838F22AEE@AVEXMB1.qlogic.org> > I seem to recall, though I can't remember where, seeing about 6-8 > months ago, reports of some strange network jitter for these platforms. > This is Nehalem/Westmere specific, didn't impact the earlier CPUs. > For some reason, my memory is of a strange driver-NUMA connection, but > I am not sure at this time. Hi Joe, Not sure if this is the issue, but for best latency benchmark performance, we recommend turning all C-states off in the BIOS. If it cannot be done there, you can add the kernel boot option: processor.max_cstate=0 and reboot. With C-states turned on, there is high and very variable MPI/IB latency measured if the benchmark is running on CPU0, the default, since extra OS interrupts are being serviced on CPU0. Once you've turned them off, You can look at these files: /proc/acpi/processor/*/power to make sure that only state C1 is being used in the list of states at the end. I think we've also seen this affecting throughput benchmarks. Again, I'm not sure if this affects GigE. -Tom > > Trying to trace something down, and gathering the background material > first. Gigabit ethernet, Intel NICs on the motherboard. Single IOH. > > Thanks! > > Joe > -- > Joseph Landman, Ph.D > Founder and CEO > Scalable Informatics Inc. > email: landman at scalableinformatics.com > web : http://scalableinformatics.com > http://scalableinformatics.com/sicluster > phone: +1 734 786 8423 x121 > fax : +1 866 888 3112 > cell : +1 734 612 4615 > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin > Computing > To change your subscription (digest mode or unsubscribe) visit > http://www.beowulf.org/mailman/listinfo/beowulf This message and any attached documents contain information from QLogic Corporation or its wholly-owned subsidiaries that may be confidential. If you are not the intended recipient, you may not read, copy, distribute, or use this information. If you have received this transmission in error, please notify the sender immediately by reply e-mail and then delete this message. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From orion at cora.nwra.com Fri Jun 24 16:54:35 2011 From: orion at cora.nwra.com (Orion Poplawski) Date: Fri, 24 Jun 2011 14:54:35 -0600 Subject: [Beowulf] Facebook's servers Message-ID: <4E04F98B.4010808@cora.nwra.com> http://opencompute.org/ I guess not commodity (yet), but interesting. -- Orion Poplawski Technical Manager 303-415-9701 x222 NWRA/CoRA Division FAX: 303-415-9702 3380 Mitchell Lane orion at cora.nwra.com Boulder, CO 80301 http://www.cora.nwra.com _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From raysonlogin at gmail.com Mon Jun 27 11:27:00 2011 From: raysonlogin at gmail.com (Rayson Ho) Date: Mon, 27 Jun 2011 11:27:00 -0400 Subject: [Beowulf] K computer powered by Open MPI Message-ID: >From the presentation, "Programming on the K computer", P. 11: http://www.fujitsu.com/downloads/TC/sc10/programming-on-k-computer.pdf Open MPI based: * Open Standard, Open Source, Multi-Platform including PC Cluster. * Adding extension to Open MPI for "Tofu" interconnect And it was confirmed that Open MPI (actually, Open MPI + Fujitsu's code for Tofu) was used in LINPACK to achieve 8 PFLOPS and get the #1 in TOP500: http://blogs.cisco.com/performance/open-mpi-powers-8-petaflops/ Other opensource stuff used by K: * Lustre - see the presentation, "An Overview of Fujitsu's Lustre Based File System": http://www.olcf.ornl.gov/wp-content/events/lug2011/4-12-2011/230-300_Shinji_Sumimoto_LUG2011-FJ-20110407-pub.pdf * Linux on SPARC - should make Ellison happy as he needs to ship SPARC T4 with Oracle Linux: http://www.pcworld.com/article/212564/ellison_oracle_enterprise_linux_coming_to_sparc.html Rayson _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From mathog at caltech.edu Mon Jun 27 11:29:20 2011 From: mathog at caltech.edu (David Mathog) Date: Mon, 27 Jun 2011 08:29:20 -0700 Subject: [Beowulf] Facebook's servers Message-ID: Orion Poplawski wrote > > I guess not commodity (yet), but interesting. > Thanks for posting that, it was interesting. The AMD board spec says it uses "address parity" memory, which I guess implies ECC, since elsewhere in the spec it discusses handling ECC. They implement reboot on lan (ROL, repurposing the WOL packet). The Intel board spec memory section doesn't say anything about parity/ECC, only that the memory must be registered, but again, elsewhere it discusses logging ECC errors, so I guess ECC is assumed. This board also implements ROL. The ROL sections seem to imply that both NICs will respond to such a packet, which could have some "interesting" security implications, at least for those of us where one interface is public. I can imagine a nightmare scenario where one machine is corrupted, it turns itself into a DHCP server and starts spraying WOL packets out onto the network, quickly converting more machines, which can carry on the same trick via their private interfaces. Normal WOL isn't nearly so hazardous on a public interface, it only becomes a security risk if the attacker has both access to another host on the subnet and some method that can remotely force the attacked system to do an orderly shutdown. Regards, David Mathog mathog at caltech.edu Manager, Sequence Analysis Facility, Biology Division, Caltech _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From orion at cora.nwra.com Thu Jun 30 16:33:21 2011 From: orion at cora.nwra.com (Orion Poplawski) Date: Thu, 30 Jun 2011 14:33:21 -0600 Subject: [Beowulf] Suggestions for used workhorse servers Message-ID: <4E0CDD91.2090405@cora.nwra.com> One can find some pretty inexpensive older servers on eBay that probably could yield a decent $/flop ratio. I was wondering if people here had suggestions for classic workhorse servers - basic 1U boxes that did/do pretty well be are a couple years old at this point. Thanks! -- Orion Poplawski Technical Manager 303-415-9701 x222 NWRA/CoRA Division FAX: 303-415-9702 3380 Mitchell Lane orion at cora.nwra.com Boulder, CO 80301 http://www.cora.nwra.com _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From prentice at ias.edu Thu Jun 30 16:48:39 2011 From: prentice at ias.edu (Prentice Bisbal) Date: Thu, 30 Jun 2011 16:48:39 -0400 Subject: [Beowulf] Suggestions for used workhorse servers In-Reply-To: <4E0CDD91.2090405@cora.nwra.com> References: <4E0CDD91.2090405@cora.nwra.com> Message-ID: <4E0CE127.9060605@ias.edu> I've always been pleased with the HP ProLiant systems, like the DL385 models. The seemed pretty reliable to me. I'd trust one of those over a Dell PowerEdge, however I'm sure you'll get as many opinions as there are subscribers on this list. -- Prentice On 06/30/2011 04:33 PM, Orion Poplawski wrote: > One can find some pretty inexpensive older servers on eBay that probably could > yield a decent $/flop ratio. I was wondering if people here had suggestions > for classic workhorse servers - basic 1U boxes that did/do pretty well be are > a couple years old at this point. > > Thanks! > _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From james.p.lux at jpl.nasa.gov Thu Jun 30 17:15:34 2011 From: james.p.lux at jpl.nasa.gov (Lux, Jim (337C)) Date: Thu, 30 Jun 2011 14:15:34 -0700 Subject: [Beowulf] Suggestions for used workhorse servers In-Reply-To: <4E0CDD91.2090405@cora.nwra.com> Message-ID: One might make a nice small cluster for learning purposes. With 8 nodes, you could do a lot of experimenting. Even 4 nodes works, but with 8, if your parallelization works, you get a pretty dramatic speedup. And, when you screw up, and need to reinstall all the software everywhere, 4-8 nodes is manageable by hand. You could also, if you have extra network cards, experiment with things like different interconnect architectures. There is significant value in a stack of boxes which you "own" and don't have to account for the use of (or lack), for that sort of "fooling around" For production purposes, you're probably better off buying newer computers: Power consumption, hassles, etc. On 6/30/11 1:33 PM, "Orion Poplawski" wrote: >One can find some pretty inexpensive older servers on eBay that probably >could >yield a decent $/flop ratio. I was wondering if people here had >suggestions >for classic workhorse servers - basic 1U boxes that did/do pretty well be >are >a couple years old at this point. > >Thanks! > >-- >Orion Poplawski >Technical Manager 303-415-9701 x222 >NWRA/CoRA Division FAX: 303-415-9702 >3380 Mitchell Lane orion at cora.nwra.com >Boulder, CO 80301 http://www.cora.nwra.com >_______________________________________________ >Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing >To change your subscription (digest mode or unsubscribe) visit >http://www.beowulf.org/mailman/listinfo/beowulf _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean.