From alvin at Maggie.Linux-Consulting.com Sun Jun 1 05:21:29 2003 From: alvin at Maggie.Linux-Consulting.com (Alvin Oga) Date: Sun, 1 Jun 2003 02:21:29 -0700 (PDT) Subject: Cheap PCs from Wal-Mart In-Reply-To: <5.2.1.1.0.20030529163946.01d00a58@mail.toaster.net> Message-ID: hi ya robert On Thu, 29 May 2003, Robert wrote: > Hello > > these guys make small foot print 1U rackmountable low powered units and > they can also be wall mounted, real cool units for clusters for starter's. > > http://www.ironsystems.com/products/iservers/aclass/a110_low_power.htm for > $499.. and an even lower priced rackmountable 1U would be using a 1U shelf with holes for mb, power supply and disks ... ( no chassis cover, no chassis sides ... hotmail style .. also made by kingstar ) for cpu cooling ... i'd use a couple of regular 24" household fans to blow lots of air across the cpu and memory and disks cpu temperature vs lifespan graph http://www.Linux-1U.net/CPU - i've always wondered why nobody has household fans mounted on the sides of the cabinets to keep the cpu cooler ... ( minor detail is that there's no mounting holes in the fan ) for "small footprint 1U rackmounts" http://www.Linux-1U.net .... 9", 11", 14" .. deep chassis for microATX/flexITX 10" deep for mini-itx mb or microatx w/ p4-3.0G fsb-800 cpu and 2 ide disks ( 500GB total ) mini-itx is similar/same to the via eden series mb on the other extreme 25" deep 1U chassis for 8 drives per one 1U chassis ( 2TB per 1U ) have fun alvin _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From mikee at mikee.ath.cx Sun Jun 1 10:47:12 2003 From: mikee at mikee.ath.cx (Mike Eggleston) Date: Sun, 1 Jun 2003 09:47:12 -0500 Subject: Cheap PCs from Wal-Mart In-Reply-To: <5.2.1.1.0.20030529163946.01d00a58@mail.toaster.net>; from Robert@jaspreet.org on Thu, May 29, 2003 at 04:51:38PM -0700 References: <5.2.0.9.2.20030529152525.02e8cb68@mailhost4.jpl.nasa.gov> <5.2.1.1.0.20030529163946.01d00a58@mail.toaster.net> Message-ID: <20030601094712.C11031@mikee.ath.cx> On Thu, 29 May 2003, Robert wrote: > Hello > > these guys make small foot print 1U rackmountable low powered units and > they can also be wall mounted, real cool units for clusters for starter's. > > http://www.ironsystems.com/products/iservers/aclass/a110_low_power.htm for > $499.. I had not read of the Via chips and had been planning to use the AMD Athlon chips. Are the via chips ok? Will they run linux/mosix ok? Any specific thoughts about them? Mike _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From mikee at mikee.ath.cx Sun Jun 1 14:25:52 2003 From: mikee at mikee.ath.cx (Mike Eggleston) Date: Sun, 1 Jun 2003 13:25:52 -0500 Subject: Cheap PCs from Wal-Mart In-Reply-To: <3EDA426A.10102@tamu.edu>; from gerry.creager@tamu.edu on Sun, Jun 01, 2003 at 01:14:02PM -0500 References: <5.2.0.9.2.20030529152525.02e8cb68@mailhost4.jpl.nasa.gov> <20030601094712.C11031@mikee.ath.cx> <3EDA426A.10102@tamu.edu> Message-ID: <20030601132552.D11031@mikee.ath.cx> On Sun, 01 Jun 2003, Gerry Creager N5JXS wrote: > I've got a Via Epia Mini-ITX board I've gotten for another project. We > loaded RH 9 via PXE-Boot (with particular thanks to RGB!), as in, a > normal install rather than diskless boot. Works great. I've got the > slowest, equivalent to a 600 MHz Celeron, but that's all I needed for > this project... in fact, overkill. > > Runs pretty cool, even with just the 2 puny little case fans, and can be > driven off a 12v Wall Wart (?tm?; should I trade mark it? no...) at > less than 5a. The wart I have is rated for 4.5a, but I've not measured > it yet. > > Built-on sound, video, USB, 1394, ethernet, and video out. Would you please give the specifics for your borard, processor, memory, etc? I'm interested in your setup. Mike _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From hahn at physics.mcmaster.ca Sun Jun 1 16:56:02 2003 From: hahn at physics.mcmaster.ca (Mark Hahn) Date: Sun, 1 Jun 2003 16:56:02 -0400 (EDT) Subject: Cheap PCs from Wal-Mart In-Reply-To: <20030601094712.C11031@mikee.ath.cx> Message-ID: > I had not read of the Via chips and had been planning to use the AMD Athlon > chips. Are the via chips ok? Will they run linux/mosix ok? Any specific > thoughts about them? they're fairly slow, from what I've read, and still use the P5 pinout (which means you'll peak at around 500 MB/s). the chip is optimized for area/power/price first, not performance - VIA's been pushing it for use in embedded/appliance kinds of uses. there's nothing inherently wrong with using lots of low-power (both ways) chips. if you carefully evaluate your code, and find that higher-powered chips don't help much (for instance, you aren't actually bottlenecked on FP or memory bandwidth, or are drastically parallel.) as I see it, even if Amdahl's saw lets you take this path, you wind up spending more and more on non-computational hardware. for instance, compare: "light": 1U via/PIII-class server, $170 1U chassis - probably cheaper per unit. - probably cooler per unit. "heavy": 1U dual Xeon/Opteron server, $350 1U chassis - definitely faster (CPU, cache, dram, network). - definitely more reliable. - probably denser. - definitely hotter per unit. - definitely more expensive per unit. I consider heat the only serious issue with the 'heavy' approach, and there's just not much we can do about that except hold our breath for .09 ;) like heat, reliability is a huge concern mainly for large clusters - you definitely don't want 512 noname powersupplies, or the kind of chassis where the vendor doesn't realize that extra fans are a liability. it's easy to imagine situations where the light approach would win: a small, non-critical learning cluster, for instance, with not much space available, or cooling capacity. or even a google-ish DB-ish thing where you're really trying to make smart disk drives. some place with plentiful cheap labor. an interesting recent development is non-wimpy blade systems. the original blades had all the disadvantages of 'light' above, but also were extremely expensive and utterly proprietary. 'heavy blades' seem to be based on commodity motherboards (good), but apparently with a separate powersupply for each board (bad). I'd like to see someone market a redundant PS which could power several motherboards (not unlike the way minis and mainframes often use DC distribution). the motherboards would still be standard/commodity/non-proprietary, and could even have passive CPU heatsinks, and just a minimal tray to screw each MB onto. using wallmart PCs for "light" cluster nodes seems sensible to me, assuming they do actually have reasonable reliability. again, assuming density isn't an issue for you, and your code is happy with slow CPUs... _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From bob at drzyzgula.org Sun Jun 1 18:13:53 2003 From: bob at drzyzgula.org (Bob Drzyzgula) Date: Sun, 1 Jun 2003 18:13:53 -0400 Subject: Cheap PCs from Wal-Mart In-Reply-To: References: <20030601094712.C11031@mikee.ath.cx> Message-ID: <20030601181353.B25006@www2> On Sun, Jun 01, 2003 at 04:56:02PM -0400, Mark Hahn wrote: > > > I had not read of the Via chips and had been planning to use the AMD Athlon > > chips. Are the via chips ok? Will they run linux/mosix ok? Any specific > > thoughts about them? > > they're fairly slow, from what I've read, and still use the P5 pinout > (which means you'll peak at around 500 MB/s). the chip is optimized > for area/power/price first, not performance - VIA's been pushing it for > use in embedded/appliance kinds of uses. Actually, the lastest Via Cyrix/Centaur C3 chips use a Socket 370 interface, and support SSE. One thing that made them particularly slow until recently was that they ran the FPU at half the core speed; the 1GHz unit now runs it at full speed. Still, the C3 is probably a bad choice for a computationally intensive application. --Bob _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From Robert at jaspreet.org Sun Jun 1 15:37:42 2003 From: Robert at jaspreet.org (Robert) Date: Sun, 01 Jun 2003 12:37:42 -0700 Subject: Cheap PCs from Wal-Mart In-Reply-To: <3EDA426A.10102@tamu.edu> References: <20030601094712.C11031@mikee.ath.cx> <5.2.0.9.2.20030529152525.02e8cb68@mailhost4.jpl.nasa.gov> <5.2.1.1.0.20030529163946.01d00a58@mail.toaster.net> <20030601094712.C11031@mikee.ath.cx> Message-ID: <5.1.1.6.2.20030601123658.04d65d00@mail.toaster.net> Hello Gerry, Would u know if someone has worked on remote serial access for this EPIA Mini-ITX board. Rob At 01:14 PM 6/1/2003 -0500, Gerry Creager N5JXS wrote: >I've got a Via Epia Mini-ITX board I've gotten for another project. We >loaded RH 9 via PXE-Boot (with particular thanks to RGB!), as in, a normal >install rather than diskless boot. Works great. I've got the slowest, >equivalent to a 600 MHz Celeron, but that's all I needed for this >project... in fact, overkill. > >Runs pretty cool, even with just the 2 puny little case fans, and can be >driven off a 12v Wall Wart (?tm?; should I trade mark it? no...) at less >than 5a. The wart I have is rated for 4.5a, but I've not measured it yet. > >Built-on sound, video, USB, 1394, ethernet, and video out. > >gerry > > >Mike Eggleston wrote: >>On Thu, 29 May 2003, Robert wrote: >> >>>Hello >>> >>>these guys make small foot print 1U rackmountable low powered units and >>>they can also be wall mounted, real cool units for clusters for starter's. >>> >>>http://www.ironsystems.com/products/iservers/aclass/a110_low_power.htm >>>for $499.. >> >>I had not read of the Via chips and had been planning to use the AMD Athlon >>chips. Are the via chips ok? Will they run linux/mosix ok? Any specific >>thoughts about them? >>Mike >>_______________________________________________ >>Beowulf mailing list, Beowulf at beowulf.org >>To change your subscription (digest mode or unsubscribe) visit >>http://www.beowulf.org/mailman/listinfo/beowulf > >-- >Gerry Creager -- gerry.creager at tamu.edu >Network Engineering -- AATLT, Texas A&M University >Cell: 979.229.5301 Office: 979.458.4020 FAX: 979.847.8578 >Page: 979.228.0173 >Office: 903A Eller Bldg, TAMU, College Station, TX 77843 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From gerry.creager at tamu.edu Sun Jun 1 14:14:02 2003 From: gerry.creager at tamu.edu (Gerry Creager N5JXS) Date: Sun, 01 Jun 2003 13:14:02 -0500 Subject: Cheap PCs from Wal-Mart In-Reply-To: <20030601094712.C11031@mikee.ath.cx> References: <5.2.0.9.2.20030529152525.02e8cb68@mailhost4.jpl.nasa.gov> <5.2.1.1.0.20030529163946.01d00a58@mail.toaster.net> <20030601094712.C11031@mikee.ath.cx> Message-ID: <3EDA426A.10102@tamu.edu> I've got a Via Epia Mini-ITX board I've gotten for another project. We loaded RH 9 via PXE-Boot (with particular thanks to RGB!), as in, a normal install rather than diskless boot. Works great. I've got the slowest, equivalent to a 600 MHz Celeron, but that's all I needed for this project... in fact, overkill. Runs pretty cool, even with just the 2 puny little case fans, and can be driven off a 12v Wall Wart (?tm?; should I trade mark it? no...) at less than 5a. The wart I have is rated for 4.5a, but I've not measured it yet. Built-on sound, video, USB, 1394, ethernet, and video out. gerry Mike Eggleston wrote: > On Thu, 29 May 2003, Robert wrote: > > >>Hello >> >>these guys make small foot print 1U rackmountable low powered units and >>they can also be wall mounted, real cool units for clusters for starter's. >> >>http://www.ironsystems.com/products/iservers/aclass/a110_low_power.htm for >>$499.. > > > I had not read of the Via chips and had been planning to use the AMD Athlon > chips. Are the via chips ok? Will they run linux/mosix ok? Any specific > thoughts about them? > > Mike > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org > To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- Gerry Creager -- gerry.creager at tamu.edu Network Engineering -- AATLT, Texas A&M University Cell: 979.229.5301 Office: 979.458.4020 FAX: 979.847.8578 Page: 979.228.0173 Office: 903A Eller Bldg, TAMU, College Station, TX 77843 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From mikee at mikee.ath.cx Sun Jun 1 23:18:02 2003 From: mikee at mikee.ath.cx (Mike Eggleston) Date: Sun, 1 Jun 2003 22:18:02 -0500 Subject: Cheap PCs from Wal-Mart In-Reply-To: <20030601181353.B25006@www2>; from bob@drzyzgula.org on Sun, Jun 01, 2003 at 06:13:53PM -0400 References: <20030601094712.C11031@mikee.ath.cx> <20030601181353.B25006@www2> Message-ID: <20030601221802.A16442@mikee.ath.cx> On Sun, 01 Jun 2003, Bob Drzyzgula wrote: > On Sun, Jun 01, 2003 at 04:56:02PM -0400, Mark Hahn wrote: > > > > > I had not read of the Via chips and had been planning to use the AMD Athlon > > > chips. Are the via chips ok? Will they run linux/mosix ok? Any specific > > > thoughts about them? > > > > they're fairly slow, from what I've read, and still use the P5 pinout > > (which means you'll peak at around 500 MB/s). the chip is optimized > > for area/power/price first, not performance - VIA's been pushing it for > > use in embedded/appliance kinds of uses. > > Actually, the lastest Via Cyrix/Centaur C3 chips use a Socket 370 interface, > and support SSE. One thing that made them particularly slow until > recently was that they ran the FPU at half the core speed; the 1GHz > unit now runs it at full speed. > > Still, the C3 is probably a bad choice for a computationally intensive > application. Ok, but what about something like a cluster with mosix in front and some database servers in back? Or web servers in front, or email servers.... this sort of thing? Mike _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From jhearns at freesolutions.net Mon Jun 2 02:18:24 2003 From: jhearns at freesolutions.net (John Hearns) Date: 02 Jun 2003 07:18:24 +0100 Subject: ClusterKnoppix Message-ID: <1054534705.15377.8.camel@harwood.home> I saw this on Slashdot a few days ago. It is a customised Knoppix distribution, with an OpenMosix patched kernel, and PXE/DHCP enabled etc. I intend to try this out in the next few days. For those who haven't used Knoppix, it is a CD which contains a bootable Debian-based distribution which installs itself in memory, not touching the hard drive. Knoppix has excellent hardware detection, and sets up a good desktop. It 'just works'. Uses might include demonstrating a Linux desktop to friends/colleagues, testing if a PC can run Linux before buying/accepting it, or as a rescue disk for troubleshooting an already installed Linux system. John Hearns _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From alvin at Maggie.Linux-Consulting.com Mon Jun 2 03:02:24 2003 From: alvin at Maggie.Linux-Consulting.com (Alvin Oga) Date: Mon, 2 Jun 2003 00:02:24 -0700 (PDT) Subject: Cheap PCs from Wal-Mart - mini-itx In-Reply-To: <20030601132552.D11031@mikee.ath.cx> Message-ID: hi ya On Sun, 1 Jun 2003, Mike Eggleston wrote: > On Sun, 01 Jun 2003, Gerry Creager N5JXS wrote: > > > I've got a Via Epia Mini-ITX board I've gotten for another project. We > > loaded RH 9 via PXE-Boot (with particular thanks to RGB!), as in, a > > normal install rather than diskless boot. Works great. I've got the > > slowest, equivalent to a 600 MHz Celeron, but that's all I needed for > > this project... in fact, overkill. > > > > Runs pretty cool, even with just the 2 puny little case fans, and can be > > driven off a 12v Wall Wart (?tm?; should I trade mark it? no...) at > > less than 5a. The wart I have is rated for 4.5a, but I've not measured > > it yet. we did lots of testing on that 12v dc-dc input to the mini-itx mb ( well, www.mini-box.com folks did all the testing and ordering of the ( custom 12v dc-to-dc atx power supply we're using the freetech mini-itx mb that supposed to handle p4-3G ( our test will be to keep the cpu cool in our 1u box ) have fun alvin _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From alvin at Maggie.Linux-Consulting.com Mon Jun 2 03:32:59 2003 From: alvin at Maggie.Linux-Consulting.com (Alvin Oga) Date: Mon, 2 Jun 2003 00:32:59 -0700 (PDT) Subject: Cheap PCs from Wal-Mart - heat In-Reply-To: Message-ID: hi ya mark On Sun, 1 Jun 2003, Mark Hahn wrote: > for instance, compare: > "light": 1U via/PIII-class server, $170 1U chassis > - probably cheaper per unit. > - probably cooler per unit. > > "heavy": 1U dual Xeon/Opteron server, $350 1U chassis > - definitely faster (CPU, cache, dram, network). > - definitely more reliable. > - probably denser. > - definitely hotter per unit. > - definitely more expensive per unit. actually, both cases can be identical ... only major differences can be types of fans for airflow and wattage of the power supply thickness of the steel also makes a big difference to reliability of the 1U server > I consider heat the only serious issue with the 'heavy' approach, and there's > just not much we can do about that except hold our breath for .09 ;) > like heat, reliability is a huge concern mainly for large clusters - you > definitely don't want 512 noname powersupplies, or the kind of chassis > where the vendor doesn't realize that extra fans are a liability. fans vs cpu temp vs cooling mechanism implies a specific cooling mechanism instead of generic solutions with lots of "useless" fans?? - if you look at the airflow, most of the fans don't do anything since 8 good 40x40x20 fans is about $100 retail... it'd make sense to use some other more cost-effective cooling of the cpu heatsink - yes you can buy $3.oo fans... but, you get what you pay for ? > the motherboards would still be standard/commodity/non-proprietary, > and could even have passive CPU heatsinks, and just a minimal tray to > screw each MB onto. passive heatsink w/ proper airflow works much better than those itty-bitty fans clipped on top of the cpu heatsink - fan blades needs ( say 0.5" of air ) to bite into to generate airflow and cooling of the cpu - air flow blowing down onto the cpu and bending 90degrees sideways is not helping the coooling air stream to cool the cpu - we put our cpu fans on the side :-) have fun alvin _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From thomas.sinczak at wp.pl Mon Jun 2 13:05:10 2003 From: thomas.sinczak at wp.pl (Thomas Sinczak) Date: Mon, 02 Jun 2003 19:05:10 +0200 Subject: use of gimp with a claster?! Message-ID: <3EDB83C6.4090800@wp.pl> Dear all, I am new to claster world... Up to now I have tested only openMosix - with results that do not help me... my problem is as follows: I have two computers: 1,8GHz Celeron with 256MB RAM and 100T Ethernet 1,26GHz PIII with 512MB RAM and 100T Ethernet I am doing some graphics - let say 6 Mpixel photos (17MB in size raw) that are manipulated with GIMP and so my question is: is that possible that the gimp will utilise the power of both computers with the use of beowulf claster?? if yes then please show me any howto Best Regards ThomasS -- ____ thomas.sinczak at wp.pl Exchange student of Osaka University, Japan - OUSSEP Student of Technical University of Lodz, Poland - Mechatronics ---- Daddy, why is that man using Windows...? Do not stare at him son... - it`s impolite! ---- - join EU ! - are you jokeing?? check the facts!! then let us speak again! ---- end of message _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From jakob at unthought.net Mon Jun 2 06:15:58 2003 From: jakob at unthought.net (Jakob Oestergaard) Date: Mon, 2 Jun 2003 12:15:58 +0200 Subject: Gigabit performance issues and NFS In-Reply-To: <5.0.2.1.2.20030529092240.02aece48@pop.larc.nasa.gov> References: <5.0.2.1.2.20030529092240.02aece48@pop.larc.nasa.gov> Message-ID: <20030602101557.GB9276@unthought.net> On Thu, May 29, 2003 at 09:35:54AM -0400, Doug Farley wrote: > Fellow Wulfers, > > I know this isnt 100% wulf related, although it is part of my wulfs setup, > but this is the best forum where everyone has alot of good experience. NFS is 'wulf related, whether we like it or not :) > > Well Heres the deal, I have a nice 2TB Linux file server with an Intel > e1000 based nic in it. And I have an SGI O3 (master node) that is dumping > to it with a tigon series gigabit card. I've tuned both, and my ttcp and > netpipe performance average ~ 80-95MB/s which is more than reasonable for > me. Both the fibre channel on my SGI and the raid (3ware) on my Linux box > can write at 40MB/s sustained, read is a little faster for both maybe ~ > 50MB/s sustained. I can get ftp/http transfers between the two to go at > 39-40MB/s, which again i'm reasonably happy with. BUT, the part that is > killing me is nfs and scp. Both crawl in at around 8-11MB/s with no other > devices on the network. 11MB/sec with scp is quite good - considering everything is encrypted and what not... With NFS that's pretty poor though, I'd agree. > Any exports from the SGI i've exported with the > 32bitclients flag, and i've pumped my r&wsize windows up to 32K, and forced > nfs v3 on both Linux and Irix. After spending a week scouring the web I've > found nothing that has worked, and SGI support thinks its a Linux nfs > problem, which could be, but i'd like to get the opinion of this crowd in > hopes of some light! What does top and vmstat on your NFS server tell you? How many nfsd threads are busy (in R or D state), during the writes ? The default number of nfsd threads is 8, which may be a little low. I run 16 threads here, on a somewhat smaller NFS server (also with a Gbit NIC). If you only see one or two nfsd threads in R or D state, anywhere near the top of your "top", then this should not be the problem. Try specifying the "async" parameter for the given mount in your exports file on the NFS server. Just to see if this helps. There are some considerations you need to make here - if the client does a sync() and you use the async option on the server, you are not guaranteed that the data has reached the disk platters by the time the client sync() call returns. This may or may not matter for you. What does vmstat say during such a big write? Is the CPU idle or busy, is it spending all it's time in the kernel? How's the ping between the machines, when doing the write and when the network is more idle? You may have a switch in between that does store-and-forward instead of cut-through, when the network gets loaded. Latency hurts NFS. -- ................................................................ : jakob at unthought.net : And I see the elder races, : :.........................: putrid forms of man : : Jakob ?stergaard : See him rise and claim the earth, : : OZ9ABN : his downfall is at hand. : :.........................:............{Konkhra}...............: _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From d.l.farley at larc.nasa.gov Mon Jun 2 10:17:53 2003 From: d.l.farley at larc.nasa.gov (Doug Farley) Date: Mon, 02 Jun 2003 10:17:53 -0400 Subject: Gigabit performance issues and NFS In-Reply-To: <20030602101557.GB9276@unthought.net> References: <5.0.2.1.2.20030529092240.02aece48@pop.larc.nasa.gov> <5.0.2.1.2.20030529092240.02aece48@pop.larc.nasa.gov> Message-ID: <5.0.2.1.2.20030602080851.02ced0f0@pop.larc.nasa.gov> Jakob, For the switch I have a hp procurve 2708, from the netpipe results i'd say its a cut-through switch, however I may have not applied enough load to it to find out, and I have seen no data either way to determine otherwise. On writing from the SGI to my Linux NFS server I have 4 nfsd's running on the Linux box, with all 4 in either a R or D state. I've included copies of the top results for reference. In regards to the use of async instead of sync in the exports, the speed was 10-14MB/s, there were again 2 nfsd's in R state and 2 in D state as before. with a load average of ~ 4, each of the nfsd consuming ~ 3% cpu, and each of the biod's consuming about 6%. I hope that maybe some of these numbers/figures (while possibly excessive) might help the group shed some light onto my problem. Thank you all again, Doug Farley For SGI Writing to Linux: ping before: ----Linux-g.localdomain PING Statistics---- 5 packets transmitted, 5 packets received, 0.0% packet loss round-trip min/avg/max = 0.219/0.286/0.427 ms ----Linux-g.localdomain PING Statistics---- 22889 packets transmitted, 22888 packets received, 0.0% packet loss round-trip min/avg/max = 0.202/0.476/2.307 ms 2072.5 packets/sec sent, 2072.5 packets/sec received --- SGI-g.localdomain ping statistics --- 13 packets transmitted, 13 received, 0% loss, time 11993ms rtt min/avg/max/mdev = 0.108/0.178/0.230/0.036 ms /ping before ping during: ----Linux-g.localdomain PING Statistics---- 6 packets transmitted, 6 packets received, 0.0% packet loss round-trip min/avg/max = 0.240/0.333/0.380 ms ----Linux-g.localdomain PING Statistics---- 33140 packets transmitted, 33140 packets received, 0.0% packet loss round-trip min/avg/max = 0.149/0.264/2.670 ms 3687.1 packets/sec sent, 3687.1 packets/sec received --- SGI-g.localdomain ping statistics --- 9 packets transmitted, 9 received, 0% loss, time 7999ms rtt min/avg/max/mdev = 0.130/0.230/0.316/0.050 ms --- SGI-g.localdomain ping statistics --- 25450 packets transmitted, 25449 received, 0% loss, time 11371ms rtt min/avg/max/mdev = 0.076/0.243/2.221/0.059 ms, ipg/ewma 0.446/0.249 ms /ping during SGI: top header before: ---- IRIX64 SGI 6.5 IP35 load averages: 0.02 0.00 0.00 57 processes: 56 sleeping, 1 running 8 CPUs: 99.4% idle, 0.0% usr, 0.4% ker, 0.0% wait, 0.0% xbrk, 0.2% intr Memory: 8192M max, 7313M avail, 7264M free, 4096M swap, 4096M free swap /top header before top header during: IRIX64 SGI 6.5 IP35 load averages: 1.28 0.48 0.18 61 processes: 59 sleeping, 2 running 8 CPUs: 83.6% idle, 0.1% usr, 7.3% ker, 6.2% wait, 0.0% xbrk, 2.9% intr Memory: 8192M max, 7455M avail, 5703M free, 4096M swap, 4096M free swap PID PGRP USERNAME PRI SIZE RES STATE TIME WCPU% CPU% COMMAND 511570519 511570519 dfarley 20 416K 304K sleep 0:29 22.2 30.16 ln 511571530 511571530 root 21 0K 0K sleep 0:08 6.5 6.63 bio3d 8975393 8975393 root 21 0K 0K sleep 0:07 6.1 6.53 bio3d 511549378 511549378 root 21 0K 0K run/1 12:24 6.0 6.15 bio3d 511563810 511563810 root 21 0K 0K sleep 0:08 6.5 5.74 bio3d 511543776 511541950 root 20 0K 0K sleep 49:18 2.2 2.88 nfsd 511568500 511568500 dfarley 20 2208K 1536K run/3 0:00 0.2 0.24 top 511567378 511549203 dfarley 20 4208K 3104K sleep 0:00 0.0 0.02 sshd 8928 8928 root 20 1808K 1088K sleep 1:02 0.0 0.01 prngd 410 410 root 20 2512K 2512K sleep 1:34 0.0 0.01 ntpd 381 0 root 20 2816K 2064K sleep 0:32 0.0 0.00 ipmon /top header during /SGI Linux Box: vmstat before: procs memory swap io system cpu r b w swpd free buff cache si so bi bo in cs us sy id 0 0 0 18980 18012 516412 418384 0 0 22 40 44 41 0 1 47 /vmstat before vmstat during: procs memory swap io system cpu r b w swpd free buff cache si so bi bo in cs us sy id 0 0 5 18980 8552 518828 445648 0 0 22 43 45 44 0 1 47 /vmstat during top header before: up 9 days, 17:03, 2 users, load average: 0.33, 0.06, 0.00 72 processes: 70 sleeping, 2 running, 0 zombie, 0 stopped CPU states: 0.0% user, 1.0% system, 0.0% nice, 99.0% idle Mem: 1031204K av, 1015316K used, 15888K free, 0K shrd, 532128K buff Swap: 2048276K av, 18980K used, 2029296K free 422944K cached /top header before top header during: up 9 days, 17:27, 2 users, load average: 2.60, 0.76, 0.26 70 processes: 66 sleeping, 4 running, 0 zombie, 0 stopped CPU states: 0.0% user, 25.6% system, 0.0% nice, 74.4% idle Mem: 1031204K av, 1022200K used, 9004K free, 0K shrd, 517376K buff Swap: 2048276K av, 18980K used, 2029296K free 446512K cached PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME COMMAND 14732 root 15 0 0 0 0 RW 4.2 0.0 1:21 nfsd 14725 root 15 0 0 0 0 DW 3.8 0.0 0:55 nfsd 14726 root 14 0 0 0 0 RW 3.8 0.0 2:08 nfsd 14729 root 14 0 0 0 0 DW 3.6 0.0 2:52 nfsd 5 root 9 0 0 0 0 SW 0.6 0.0 1:12 kswapd 6 root 9 0 0 0 0 SW 0.6 0.0 32:40 kscand 14730 root 9 0 0 0 0 SW 0.2 0.0 1:23 nfsd 1 root 9 0 156 128 96 S 0.0 0.0 0:04 init 2 root 9 0 0 0 0 SW 0.0 0.0 0:00 keventd 3 root 9 0 0 0 0 SW 0.0 0.0 0:00 kapmd /top header during /Linux Box ============================== Doug Farley Data Analysis and Imaging Branch Systems Engineering Competency NASA Langley Research Center < D.L.FARLEY at LaRC.NASA.GOV > < Phone +1 757 864-8141 > At 12:15 PM 6/2/2003 +0200, Jakob Oestergaard wrote: >On Thu, May 29, 2003 at 09:35:54AM -0400, Doug Farley wrote: > > Fellow Wulfers, > > > > I know this isnt 100% wulf related, although it is part of my wulfs setup, > > but this is the best forum where everyone has alot of good experience. > >NFS is 'wulf related, whether we like it or not :) > > > > > Well Heres the deal, I have a nice 2TB Linux file server with an Intel > > e1000 based nic in it. And I have an SGI O3 (master node) that is dumping > > to it with a tigon series gigabit card. I've tuned both, and my ttcp and > > netpipe performance average ~ 80-95MB/s which is more than reasonable for > > me. Both the fibre channel on my SGI and the raid (3ware) on my Linux box > > can write at 40MB/s sustained, read is a little faster for both maybe ~ > > 50MB/s sustained. I can get ftp/http transfers between the two to go at > > 39-40MB/s, which again i'm reasonably happy with. BUT, the part that is > > killing me is nfs and scp. Both crawl in at around 8-11MB/s with no other > > devices on the network. > >11MB/sec with scp is quite good - considering everything is encrypted >and what not... > >With NFS that's pretty poor though, I'd agree. > > > Any exports from the SGI i've exported with the > > 32bitclients flag, and i've pumped my r&wsize windows up to 32K, and > forced > > nfs v3 on both Linux and Irix. After spending a week scouring the web > I've > > found nothing that has worked, and SGI support thinks its a Linux nfs > > problem, which could be, but i'd like to get the opinion of this crowd in > > hopes of some light! > >What does top and vmstat on your NFS server tell you? > >How many nfsd threads are busy (in R or D state), during the writes ? > >The default number of nfsd threads is 8, which may be a little low. I >run 16 threads here, on a somewhat smaller NFS server (also with a Gbit >NIC). If you only see one or two nfsd threads in R or D state, >anywhere near the top of your "top", then this should not be the >problem. > >Try specifying the "async" parameter for the given mount in your exports >file on the NFS server. Just to see if this helps. There are some >considerations you need to make here - if the client does a sync() and >you use the async option on the server, you are not guaranteed that the >data has reached the disk platters by the time the client sync() call >returns. This may or may not matter for you. > >What does vmstat say during such a big write? Is the CPU idle or busy, >is it spending all it's time in the kernel? > >How's the ping between the machines, when doing the write and when the >network is more idle? You may have a switch in between that does >store-and-forward instead of cut-through, when the network gets loaded. >Latency hurts NFS. > >-- >................................................................ >: jakob at unthought.net : And I see the elder races, : >:.........................: putrid forms of man : >: Jakob ?stergaard : See him rise and claim the earth, : >: OZ9ABN : his downfall is at hand. : >:.........................:............{Konkhra}...............: _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From gerry.creager at tamu.edu Sun Jun 1 23:47:25 2003 From: gerry.creager at tamu.edu (Gerry Creager N5JXS) Date: Sun, 01 Jun 2003 22:47:25 -0500 Subject: Cheap PCs from Wal-Mart In-Reply-To: <20030601132552.D11031@mikee.ath.cx> References: <5.2.0.9.2.20030529152525.02e8cb68@mailhost4.jpl.nasa.gov> <20030601094712.C11031@mikee.ath.cx> <3EDA426A.10102@tamu.edu> <20030601132552.D11031@mikee.ath.cx> Message-ID: <3EDAC8CD.1070109@tamu.edu> Via EPiA M-6000 motherboard/chipset combo (at least that's what I recall, but the EPiA V looks closer to what I installed...) with an SiS onboard audio chipset, 10/100 ethernet onboard VGA video chipset, and "tv-out" in both S-Video and composite (RCA jack). I'm running it with a 40 GB Maxtor (boought solely on price as there was nothing smaller available for less money at the time). I added a Hayes 56k V92 modem because what I'm planning to do is poll a bunch of automated weather sites for data and they are all dial-up... Check out: http://www.viavpsd.com/product/epia_v_spec.jsp?motherboardId=141 http://www.viavpsd.com/product/epia_m_spec.jsp?motherboardId=81 http://www.viavpsd.com/product/epia_mini_itx_spec.jsp?motherboardId=21 The case I found (at Fry's) was $79; short of going into the office tonight I'll have to look on outpost.com -- darn! Not there. I'll try to find the info on the case, but it comes with a wall wart for AC power, and there's a 12v coaxial DC jack on the case... I'm not at all sure I'd use it for a compute cluster, but it might do some low-end work and it is certainly is a low-heat/low power solution. Gerry Mike Eggleston wrote: > On Sun, 01 Jun 2003, Gerry Creager N5JXS wrote: > > >>I've got a Via Epia Mini-ITX board I've gotten for another project. We >>loaded RH 9 via PXE-Boot (with particular thanks to RGB!), as in, a >>normal install rather than diskless boot. Works great. I've got the >>slowest, equivalent to a 600 MHz Celeron, but that's all I needed for >>this project... in fact, overkill. >> >>Runs pretty cool, even with just the 2 puny little case fans, and can be >>driven off a 12v Wall Wart (?tm?; should I trade mark it? no...) at >>less than 5a. The wart I have is rated for 4.5a, but I've not measured >>it yet. >> >>Built-on sound, video, USB, 1394, ethernet, and video out. > > > Would you please give the specifics for your borard, processor, memory, > etc? I'm interested in your setup. > > Mike -- Gerry Creager -- gerry.creager at tamu.edu Network Engineering -- AATLT, Texas A&M University Cell: 979.229.5301 Office: 979.458.4020 FAX: 979.847.8578 Page: 979.228.0173 Office: 903A Eller Bldg, TAMU, College Station, TX 77843 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From jhearns at freesolutions.net Mon Jun 2 11:26:51 2003 From: jhearns at freesolutions.net (John Hearns) Date: Mon, 2 Jun 2003 16:26:51 +0100 Subject: Cheap PCs from Wal-Mart References: <5.2.0.9.2.20030529152525.02e8cb68@mailhost4.jpl.nasa.gov> <5.2.1.1.0.20030529163946.01d00a58@mail.toaster.net> <20030601094712.C11031@mikee.ath.cx> <3EDA426A.10102@tamu.edu> <3EDAFA58.5070907@nada.kth.se> Message-ID: <002401c3291b$67a52d80$8461cdc2@DREAD> I think the list knows that I'm a fan of the mini-ITX - I've got four of them at home. (I've had a paper accepted for the UKUUG Linux Developers Conference on mini-ITX also) re. the points on floating point performance, these are valid. But I think we should note the new Nehemiah boards which are just out. Well worth looking at I think - I intend to get one soon, and I'll report back if/when I do. http://www.mini-itx.com/reviews/nehemiah/?page=10#s21 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From tegner at nada.kth.se Mon Jun 2 03:18:48 2003 From: tegner at nada.kth.se (Jon Tegner) Date: Mon, 02 Jun 2003 09:18:48 +0200 Subject: Cheap PCs from Wal-Mart In-Reply-To: <3EDA426A.10102@tamu.edu> References: <5.2.0.9.2.20030529152525.02e8cb68@mailhost4.jpl.nasa.gov> <5.2.1.1.0.20030529163946.01d00a58@mail.toaster.net> <20030601094712.C11031@mikee.ath.cx> <3EDA426A.10102@tamu.edu> Message-ID: <3EDAFA58.5070907@nada.kth.se> I have used the Via Epia Mini-ITX board (EPIA M9000), and I would not recommend it on any application heavy on floating point operations. In my experience (CFD) it is closer to a 300 MHz Celeron than a 600 (but I guess it depends on the application...). /jon Gerry Creager N5JXS wrote: > I've got a Via Epia Mini-ITX board I've gotten for another project. > We loaded RH 9 via PXE-Boot (with particular thanks to RGB!), as in, a > normal install rather than diskless boot. Works great. I've got the > slowest, equivalent to a 600 MHz Celeron, but that's all I needed for > this project... in fact, overkill. > > Runs pretty cool, even with just the 2 puny little case fans, and can > be driven off a 12v Wall Wart (?tm?; should I trade mark it? no...) > at less than 5a. The wart I have is rated for 4.5a, but I've not > measured it yet. > > Built-on sound, video, USB, 1394, ethernet, and video out. > > gerry > > > Mike Eggleston wrote: > >> On Thu, 29 May 2003, Robert wrote: >> >> >>> Hello >>> >>> these guys make small foot print 1U rackmountable low powered units >>> and they can also be wall mounted, real cool units for clusters for >>> starter's. >>> >>> http://www.ironsystems.com/products/iservers/aclass/a110_low_power.htm >>> for $499.. >> >> >> >> I had not read of the Via chips and had been planning to use the AMD >> Athlon >> chips. Are the via chips ok? Will they run linux/mosix ok? Any specific >> thoughts about them? >> >> Mike >> _______________________________________________ >> Beowulf mailing list, Beowulf at beowulf.org >> To change your subscription (digest mode or unsubscribe) visit >> http://www.beowulf.org/mailman/listinfo/beowulf > > _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From gerry.creager at tamu.edu Sun Jun 1 23:59:31 2003 From: gerry.creager at tamu.edu (Gerry Creager N5JXS) Date: Sun, 01 Jun 2003 22:59:31 -0500 Subject: Cheap PCs from Wal-Mart In-Reply-To: <20030601221802.A16442@mikee.ath.cx> References: <20030601094712.C11031@mikee.ath.cx> <20030601181353.B25006@www2> <20030601221802.A16442@mikee.ath.cx> Message-ID: <3EDACBA3.6000501@tamu.edu> Mosix/web service, maybe. Not for MY database needs, though, although, now that you mention it, I may have to try! gerry Mike Eggleston wrote: > On Sun, 01 Jun 2003, Bob Drzyzgula wrote: > > >>On Sun, Jun 01, 2003 at 04:56:02PM -0400, Mark Hahn wrote: >> >>>>I had not read of the Via chips and had been planning to use the AMD Athlon >>>>chips. Are the via chips ok? Will they run linux/mosix ok? Any specific >>>>thoughts about them? >>> >>>they're fairly slow, from what I've read, and still use the P5 pinout >>>(which means you'll peak at around 500 MB/s). the chip is optimized >>>for area/power/price first, not performance - VIA's been pushing it for >>>use in embedded/appliance kinds of uses. >> >>Actually, the lastest Via Cyrix/Centaur C3 chips use a Socket 370 interface, >>and support SSE. One thing that made them particularly slow until >>recently was that they ran the FPU at half the core speed; the 1GHz >>unit now runs it at full speed. >> >>Still, the C3 is probably a bad choice for a computationally intensive >>application. > > > Ok, but what about something like a cluster with mosix in front > and some database servers in back? Or web servers in front, or email > servers.... this sort of thing? -- Gerry Creager -- gerry.creager at tamu.edu Network Engineering -- AATLT, Texas A&M University Cell: 979.229.5301 Office: 979.458.4020 FAX: 979.847.8578 Page: 979.228.0173 Office: 903A Eller Bldg, TAMU, College Station, TX 77843 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From samhdaniel at earthlink.net Mon Jun 2 11:10:29 2003 From: samhdaniel at earthlink.net (Sam Daniel) Date: 02 Jun 2003 11:10:29 -0400 Subject: Cheap PCs from Wal-Mart In-Reply-To: <5.2.1.1.0.20030529163946.01d00a58@mail.toaster.net> References: <5.2.0.9.2.20030529152525.02e8cb68@mailhost4.jpl.nasa.gov> <5.2.1.1.0.20030529163946.01d00a58@mail.toaster.net> Message-ID: <1054566629.4082.1.camel@wulf> The only problem with these is that the C3-933 runs all floating point at half the processor speed. If you're trying to really solve a problem, that could be a big drawback. For a home system, it wouldn't be so bad. Unless you really need the 1U form factor, you can beat the price by nearly 50% by putting together the same hardware in a mini-atx or mid-tower case of your own choosing. VIA recently released the C3-1GHz with a new core, called "Nehemiah", which is supposed to run the FPU at full speed, but I haven't seen confirmation of that in any tests yet. The cost will only be $10-$20 more than the C3-933. Using the new processor will make these systems a lot more attractive, with even lower power consumption than the C3-933. Sam On Thu, 2003-05-29 at 19:51, Robert wrote: > Hello > > these guys make small foot print 1U rackmountable low powered units and > they can also be wall mounted, real cool units for clusters for starter's. > http://www.ironsystems.com/products/iservers/aclass/a110_low_power.htm for > $499.. > > Rob > > > At 04:19 PM 5/29/2003 -0700, Alvin Oga wrote: > > >hi ya > > > >cheap PCs can be gotten almost anywhere ??? doesnt have to be > >walmart/circuit city/emachines/etc > > > >$ 30 cheap pc case ( that makes the PC their widget ) > >$ 70 generic motherboard w/ onboard nic, onboard svga > >$ 70 Celeron-1.7G 478pin fsb400 cpu > >$ 25 128MB pc-133 > >$ 25 50x cdrom > >$ 60 20GB ide disk > >---- ------------- > >$ 280 grand total > > > >$ 25 oem ms license > > > >mb, cpu, disks can be lot lower in $$ if you use p3 and pc-133 meory > > > >via series mb w/ p3-800 is about $85 total ( subtract ~ $60 from above ) > > > >same cost estimates for amd duron/athlon based systems > > > >you can save the shipping by bying locally... > >and might be zero sales tax in some states too > > > >stuff all that into a 1U chassis and add $100 - $250 extra ... > >and take out the cost of the "generic midtower case" > > > >and if there's a problem w/ the pc, i'd hate to worry about how to return > >it and get a better box back or is it, as typically the case, > >that they'd simply send out a different returned PC .. since its a > >warranty replacement, they dont have to send you a brand new pc > >like they would have to with a new order > > > >On Thu, 29 May 2003, Jim Lux wrote: > > > > > For those of you looking to build a cluster on the (real) cheap, Walmart > > > has mailorder PCs, with Lindows (a Linux variant) installed for $200 (plus > > > sales tax and shipping, of course). > > > > > > I just bought one of these for my daughter (with WinXP, for $300.. I guess > > > the MS license is $100) and while it's no ball of fire, and the keyboard > > > and mouse are what you'd expect for a $200 computer, it DOES work ok..at > > > >magic ! > > > >have fun > >alvin > > > >_______________________________________________ > >Beowulf mailing list, Beowulf at beowulf.org > >To change your subscription (digest mode or unsubscribe) visit > >http://www.beowulf.org/mailman/listinfo/beowulf > > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org > To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From hahn at physics.mcmaster.ca Mon Jun 2 13:08:40 2003 From: hahn at physics.mcmaster.ca (Mark Hahn) Date: Mon, 2 Jun 2003 13:08:40 -0400 (EDT) Subject: Gigabit performance issues and NFS In-Reply-To: <5.0.2.1.2.20030602080851.02ced0f0@pop.larc.nasa.gov> Message-ID: > ping before: > ----Linux-g.localdomain PING Statistics---- > 5 packets transmitted, 5 packets received, 0.0% packet loss > round-trip min/avg/max = 0.219/0.286/0.427 ms > > ----Linux-g.localdomain PING Statistics---- > 22889 packets transmitted, 22888 packets received, 0.0% packet loss > round-trip min/avg/max = 0.202/0.476/2.307 ms > 2072.5 packets/sec sent, 2072.5 packets/sec received > > --- SGI-g.localdomain ping statistics --- > 13 packets transmitted, 13 received, 0% loss, time 11993ms > rtt min/avg/max/mdev = 0.108/0.178/0.230/0.036 ms > /ping before uh, that sucks! here's what I see on a pair of gbe-connected nodes (dual-xeon/2400, i7500, e1000) on an SMC SMC8624T: | [hahn at cat48 hahn]$ ping -c100 -q cat47 | PING cat47 (10.0.0.47) from 10.0.0.48 : 56(84) bytes of data. | | --- cat47 ping statistics --- | 100 packets transmitted, 100 received, 0% loss, time 98998ms | rtt min/avg/max/mdev = 0.034/0.035/0.046/0.004 ms and here's two nodes on a (3com 48+2) 100bT switch: | [root at cat8 root]# ping -c100 -q cat2 | PING cat2 (10.0.0.2) from 10.0.0.8 : 56(84) bytes of data. | | --- cat2 ping statistics --- | 100 packets transmitted, 100 received, 0% loss, time 98996ms | rtt min/avg/max/mdev = 0.099/0.101/0.241/0.016 ms in other words, your network appears to be rather sluggish, unless I've misunderstood your ping numbers. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From josip at lanl.gov Mon Jun 2 13:21:02 2003 From: josip at lanl.gov (Josip Loncaric) Date: Mon, 02 Jun 2003 11:21:02 -0600 Subject: Gigabit performance issues and NFS In-Reply-To: <5.0.2.1.2.20030602080851.02ced0f0@pop.larc.nasa.gov> References: <5.0.2.1.2.20030529092240.02aece48@pop.larc.nasa.gov> <5.0.2.1.2.20030529092240.02aece48@pop.larc.nasa.gov> <5.0.2.1.2.20030602080851.02ced0f0@pop.larc.nasa.gov> Message-ID: <3EDB877E.7060308@lanl.gov> Doug, In addition to NFS tuning suggested by Jakob, you may want to check your gigabit ethernet driver parameters. This was the key problem we identified on Coral, our cluster at ICASE (my old LaRC home :-), and the fix was to increase the interrupt mitigation parameters for the acenic device driver. The reason is this: NFS in Linux uses UDP, and it takes 6 normal UDP frames (unless you are using jumbo frames) to transfer one 8KB NFS block (a typical NFS tuning choice). If the NFS server gets interrupted more than every 6th frame, chances are that some frames will be lost. If any of the 6 frames is lost, all 6 have to be re-sent, leading to further losses, etc. This can collapse NFS performance by 100:1. We found that the following line in /etc/modules.conf restored reasonable (but not great) NFS performance: options acenic tx_coal_tick=75 rx_coal_tick=75 max_tx_desc=32 max_rx_desc=6 The key bit is that last parameter "max_rx_desc=6" but the others help as well. Optimal choices may be hardware dependent, but the above is a good starting point for the acenic device driver. On the other hand, your NFS performance results are fairly typical. We found that acenic-based Gigabit Ethernet can deliver ~72% of its rated performance only under rare circumstances and with special effort. Normally, we got FTP performance of about 28 MB/s and NFS performance of about 13 MB/s. Sincerely, Josip _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From luis.licon at yakko.cimav.edu.mx Mon Jun 2 13:56:11 2003 From: luis.licon at yakko.cimav.edu.mx (Luis Fernando Licon Padilla) Date: Mon, 02 Jun 2003 11:56:11 -0600 Subject: ClusterKnoppix References: <1054534705.15377.8.camel@harwood.home> Message-ID: <3EDB8FBB.5060907@yakko.cimav.edu.mx> John Hearns wrote: >I saw this on Slashdot a few days ago. > >It is a customised Knoppix distribution, with an OpenMosix patched >kernel, and PXE/DHCP enabled etc. >I intend to try this out in the next few days. > >For those who haven't used Knoppix, it is a CD which contains a bootable >Debian-based distribution which installs itself in memory, not touching >the hard drive. Knoppix has excellent hardware detection, and sets up >a good desktop. It 'just works'. >Uses might include demonstrating a Linux desktop to friends/colleagues, >testing if a PC can run Linux before buying/accepting it, or as a rescue >disk for troubleshooting an already installed Linux system. > >John Hearns > >_______________________________________________ >Beowulf mailing list, Beowulf at beowulf.org >To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf > > > > Hi you all guys, I have a 12 dual nodes beowulf running on Red Hat Linux 8.0, but you know, I have the update problem since I do not want to pay the update service to Red Hat, so my question is, Does someone have experience about clusters running on both Debian and RedHat distributions, who can help me to decide on this issue? Thanks, Luis -- ISC Luis Fernando Licon Padilla Advanced Materials Research Center Miguel de Cervantes 120 Complejo Industrial Chihuahua C.P.31109 Chihuahua, Chih. Mexico Phone: 52 (614)4391154 Fax: 52 (614)4391112 alternative e-mail: lordsirion2002 at yahoo.com _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From hahn at physics.mcmaster.ca Mon Jun 2 14:42:32 2003 From: hahn at physics.mcmaster.ca (Mark Hahn) Date: Mon, 2 Jun 2003 14:42:32 -0400 (EDT) Subject: Cheap PCs from Wal-Mart In-Reply-To: <002401c3291b$67a52d80$8461cdc2@DREAD> Message-ID: > re. the points on floating point performance, these are valid. > But I think we should note the new Nehemiah boards which are just out. > Well worth looking at I think - I intend to get one soon, and I'll report > back if/when I do. > http://www.mini-itx.com/reviews/nehemiah/?page=10#s21 hmm. for rhetorical purposes, let's compare the numbers from this article to two high-end unis: dhry whet mmi mmf memi memf e800 1048 285 963 1588 194 208 e10k 1300 351 1193 1968 233 245 e10k-n 1591 366 2255 2285 664 389 p4 6809 9327 22170 13896 5050 5041 ath 3319 8855 13011 12217 2912 3080 I think it's pretty clear that you need to expect much lower performance from even the 'high-end' VIA chips. if your code more resembles dhrystone (mostly integer, cache friendly), then you can mostly expect to scale with clock speed, and the VIA chips might be attractive on a speed/(heat*cost) basis. for general clusters, where memory bandwidth and FP performance and integrated gigabit are big advantages, VIA doesn't compete. data is from the article above and http://www6.tomshardware.com/cpu/20030521/index.html _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rocky at atipa.com Mon Jun 2 14:09:34 2003 From: rocky at atipa.com (Rocky McGaugh) Date: Mon, 2 Jun 2003 13:09:34 -0500 (CDT) Subject: ClusterKnoppix In-Reply-To: <3EDB8FBB.5060907@yakko.cimav.edu.mx> Message-ID: On Mon, 2 Jun 2003, Luis Fernando Licon Padilla wrote: > Hi you all guys, > > I have a 12 dual nodes beowulf running on Red Hat Linux 8.0, but you > know, I have the update problem since I do not want to pay the update > service to Red Hat, so my question is, Does someone have experience > about clusters running on both Debian and RedHat distributions, who can > help me to decide on this issue? > > Thanks, > > Luis for redhat, yum is the best thing since sliced bread. http://linux.duke.edu/projects/yum/ -- Rocky McGaugh Atipa Technologies rocky at atipatechnologies.com rmcgaugh at atipa.com 1-785-841-9513 x3110 http://67.8450073/ perl -e 'print unpack(u, ".=W=W+F%T:7\!A+F-O;0H`");' _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From mikee at mikee.ath.cx Mon Jun 2 15:42:23 2003 From: mikee at mikee.ath.cx (Mike Eggleston) Date: Mon, 2 Jun 2003 14:42:23 -0500 Subject: ClusterKnoppix In-Reply-To: <3EDB8FBB.5060907@yakko.cimav.edu.mx>; from luis.licon@yakko.cimav.edu.mx on Mon, Jun 02, 2003 at 11:56:11AM -0600 References: <1054534705.15377.8.camel@harwood.home> <3EDB8FBB.5060907@yakko.cimav.edu.mx> Message-ID: <20030602144223.L17733@mikee.ath.cx> On Mon, 02 Jun 2003, Luis Fernando Licon Padilla wrote: > John Hearns wrote: > > >I saw this on Slashdot a few days ago. > > > >It is a customised Knoppix distribution, with an OpenMosix patched > >kernel, and PXE/DHCP enabled etc. > >I intend to try this out in the next few days. > > > >For those who haven't used Knoppix, it is a CD which contains a bootable > >Debian-based distribution which installs itself in memory, not touching > >the hard drive. Knoppix has excellent hardware detection, and sets up > >a good desktop. It 'just works'. > >Uses might include demonstrating a Linux desktop to friends/colleagues, > >testing if a PC can run Linux before buying/accepting it, or as a rescue > >disk for troubleshooting an already installed Linux system. > > > >John Hearns > > > >_______________________________________________ > >Beowulf mailing list, Beowulf at beowulf.org > >To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf > > > > > > > > > Hi you all guys, > > I have a 12 dual nodes beowulf running on Red Hat Linux 8.0, but you > know, I have the update problem since I do not want to pay the update > service to Red Hat, so my question is, Does someone have experience > about clusters running on both Debian and RedHat distributions, who can > help me to decide on this issue? Have you tried using apt? It seems to run a day or two behind the RedHat updates, but it gets all the updates. Try setting a nightly cron entry to execute 'apt-get update; apt-get update' for all your nodes. You can find apt at www.freshrpms.net. Mike _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From Robin.Laing at drdc-rddc.gc.ca Mon Jun 2 17:25:33 2003 From: Robin.Laing at drdc-rddc.gc.ca (Robin Laing) Date: Mon, 02 Jun 2003 15:25:33 -0600 Subject: Help: different kernel images for slaves. Message-ID: <3EDBC0CD.6090109@drdc-rddc.gc.ca> Hello, I am involved with adding some SMP machines to our small cluster and I am trying to find out how I tell the new machines to run the SMP kernel instead of the default in the /var/beowulf/config file. We are running RH 9 and BPpoc. Our present system is working with single processor boot image and the SMP machines will boot with only one processor working. I have compiled a SMP kernel to test but I cannot just change the boot kernel for all the other machines. Presently in /var/beowulf are: boot.img nodeboot.img nodeinfo I want to add boot_smp.img I have not seen any documentation on configuring /etc/beowulf/config to work with more than one boot image. Is it possible to define which nodes boot with which kernel? -- Robin Laing Instrumentation Technologist Voice: 1.403.544.4762 Military Engineering Section FAX: 1.403.544.4704 Defence R&D Canada - Suffield Email: Robin.Laing at DRDC-RDDC.gc.ca PO Box 4000, Station Main WWW: http://www.suffield.drdc-rddc.gc.ca Medicine Hat, AB, T1A 8K6 Canada _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From math at velocet.ca Mon Jun 2 18:22:34 2003 From: math at velocet.ca (Ken Chase) Date: Mon, 2 Jun 2003 18:22:34 -0400 Subject: Cheap PCs from Wal-Mart In-Reply-To: ; from alvin@Maggie.Linux-Consulting.com on Mon, Jun 02, 2003 at 02:18:06PM -0700 References: Message-ID: <20030602182234.E81856@velocet.ca> On Mon, Jun 02, 2003 at 02:18:06PM -0700, Alvin Oga's all... >> hmm. for rhetorical purposes, let's compare the numbers from this article >> to two high-end unis: >> >> dhry whet mmi mmf memi memf >> e800 1048 285 963 1588 194 208 >> e10k 1300 351 1193 1968 233 245 >> e10k-n 1591 366 2255 2285 664 389 >> p4 6809 9327 22170 13896 5050 5041 >> ath 3319 8855 13011 12217 2912 3080 > >I wonder what the specific details was for the aove p4/ath cpus. > >For the $$$ difference betwen a p4/amd and c3 ... it seems >like a non-issue for performance of spending the extra $50 - $100 >for better performance in the above benchmark numbers > >( but, its a bigger price/performance gap for the lastest FSB-800 cpus > and runs $300 - $500 or higher > >good stuff in either case ... Since power & cooling is almost always free for .edu's, yes, .edu cluster design and others' will be extremely different. Not to mention that .edu's cant stand losing nodes in a throwaway design that accomodates tolerable rates of failure. (Anyone operating a cluster in this mode? Im very curious as to how you surmounted the psychological objections to it!) When you say 'TCO' you REALLY have to define what you mean -- "total cost of OWNERSHIP" or "total cost of OPERATION"? You can own a really nice cluster for $x if you dont have to cut $x into slices to buy it AND pay for running power and cooling for it for n years. Anyone admit to being in the 'must pay for power and other operationals' camp and want to explain how their designs differ? (especially those that pay a premium for floor space - these edens start winning on two fronts. Low heat = low space requirements.) I suspect EDENs will be viable for people in the expensive power & floorspace areas (Bay area, etc?) /kc -- Ken Chase, math at velocet.ca * Velocet Communications Inc. * Toronto, Canada Wiznet Velocet DSL.ca Datavaults 24/7: 416-967-4414 tollfree: 1-866-353-0363 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From James.P.Lux at jpl.nasa.gov Mon Jun 2 18:25:47 2003 From: James.P.Lux at jpl.nasa.gov (Jim Lux) Date: Mon, 02 Jun 2003 15:25:47 -0700 Subject: Cheap PCs from Wal-Mart References: <002401c3291b$67a52d80$8461cdc2@DREAD> Message-ID: <5.2.0.9.2.20030602150917.02f2ba88@mailhost4.jpl.nasa.gov> Which P4 and Athlon were you comparing.. I went to the linked page at Toms Hardware and found reasonably close results Athlon XP 3200+, and P4 3GHz... Now, what's the power consumption of those two? I think the P4 dissipates a max of 85W (poking around on the same Tom's page) The C3 at 800 MHz dissipates 6.65W average and peaks at 12W The Nehemiah at 1 GHz dissipates 11.25/15 W ave/pk So, looking at the scaling and comparing P4 against Via C3 Nehemiah Speed 6809/1591 4.3:1 Power 85/15 5.7:1 So the Via looks like it does more computes/Joule... This is really a first order approximation. You'd have to look at peripheral power, memory power, and power supply conversion and power distribution efficiency. Peripherals and memory probably scale with memory speed fairly linearly. The higher powered device will have more I^2R losses in the power supply and distribution. (Supplying 50 or so Amps at 2-3V is no easy feat) Here's the upshot... if you're in an environment where ultimate performance is important, than you're probably best going with as few big fast processors as you can get (the original Cray paradigm). On the other hand, if you're in an environment where maximizing computation for a fixed amount of power/heat dissipation resources (i.e. spaceflight or portable operation), then things like the Via C3 start to look attractive, assuming your task scales well. Since I'm generally interested in resource(other than dollars) constrained computing, I'd like to see more attention paid to computing per joule or computing rate/watt. There are different tradeoffs whether you are concerned about total energy (i.e. you're running off a battery) or whether you have an instantaneous power dissipation limit (i.e. thermal radiators in space). (that is, you can decide, do I want to run the calculation in one hour at 200W, or over two hours at 100W) At 02:42 PM 6/2/2003 -0400, Mark Hahn wrote: > > re. the points on floating point performance, these are valid. > > But I think we should note the new Nehemiah boards which are just out. > > Well worth looking at I think - I intend to get one soon, and I'll report > > back if/when I do. > > http://www.mini-itx.com/reviews/nehemiah/?page=10#s21 > >hmm. for rhetorical purposes, let's compare the numbers from this article >to two high-end unis: > > dhry whet mmi mmf memi memf >e800 1048 285 963 1588 194 208 >e10k 1300 351 1193 1968 233 245 >e10k-n 1591 366 2255 2285 664 389 >p4 6809 9327 22170 13896 5050 5041 >ath 3319 8855 13011 12217 2912 3080 > >I think it's pretty clear that you need to expect much lower >performance from even the 'high-end' VIA chips. if your code >more resembles dhrystone (mostly integer, cache friendly), >then you can mostly expect to scale with clock speed, and the >VIA chips might be attractive on a speed/(heat*cost) basis. > >for general clusters, where memory bandwidth and FP performance >and integrated gigabit are big advantages, VIA doesn't compete. > >data is from the article above and > http://www6.tomshardware.com/cpu/20030521/index.html > >__ James Lux, P.E. Spacecraft Telecommunications Section Jet Propulsion Laboratory, Mail Stop 161-213 4800 Oak Grove Drive Pasadena CA 91109 tel: (818)354-2075 fax: (818)393-6875 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From josip at lanl.gov Mon Jun 2 18:39:54 2003 From: josip at lanl.gov (Josip Loncaric) Date: Mon, 02 Jun 2003 16:39:54 -0600 Subject: Cheap PCs from Wal-Mart In-Reply-To: References: Message-ID: <3EDBD23A.7020508@lanl.gov> Mark Hahn wrote: > > data is from the article above and > http://www6.tomshardware.com/cpu/20030521/index.html ... which leads to the following very interesting observation: "Our lab engineers made a curious discovery when they were examining the new 865PE board from MSI. By incorporating special logic circuitry, the manufacturer has succeeded in boosting the speed of the CPU through dynamic overclocking in such a way that it is not detectable with conventional benchmarking utilities like WCPUID, Intel Frequency Display, CPUZ, or SiSoft Sandra 2003. However, FSB and CPU speeds are only increased when applications are started or when benchmark programs have finished - subject to CPU usage reaching close to 100 percent." (see http://www6.tomshardware.com/cpu/20030521/800fsb-25.html) Neat... but beware of unintended consequences, e.g. variable OS event timing... Sincerely, Josip _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From alvin at Maggie.Linux-Consulting.com Mon Jun 2 18:54:37 2003 From: alvin at Maggie.Linux-Consulting.com (Alvin Oga) Date: Mon, 2 Jun 2003 15:54:37 -0700 (PDT) Subject: Cheap PCs from Wal-Mart - mini-itx - 12v In-Reply-To: <20030602182436.F81856@velocet.ca> Message-ID: hi ya ken On Mon, 2 Jun 2003, Ken Chase wrote: > >we did lots of testing on that 12v dc-dc input to the mini-itx mb > >( well, www.mini-box.com folks did all the testing and ordering of the > >( custom 12v dc-to-dc atx power supply these tests were for 1-z 2-z systems .. one dc-to-dc ps for each mini-itx motherboard in the stackable case-size of a standard cdrom > Any numbers on the PF of these ac-to-dv (then to dc) floor-brick-with- > power-cables xformers? I suspect this stuff isnt worth anything til you > run it off a site-wide 12V infrastructure to get a good PF rating. or one can operate like a colo ?? - take 110VAC in, convert to 12V dc batteries, generate 110VAC to powerup the colo'd servers or 48V dc ( like the phone systems ) have fun alvin _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From joelja at darkwing.uoregon.edu Mon Jun 2 18:48:22 2003 From: joelja at darkwing.uoregon.edu (Joel Jaeggli) Date: Mon, 2 Jun 2003 15:48:22 -0700 (PDT) Subject: Cheap PCs from Wal-Mart In-Reply-To: <20030602182234.E81856@velocet.ca> Message-ID: On Mon, 2 Jun 2003, Ken Chase wrote: > On Mon, Jun 02, 2003 at 02:18:06PM -0700, Alvin Oga's all... > >> hmm. for rhetorical purposes, let's compare the numbers from this article > >> to two high-end unis: > >> > >> dhry whet mmi mmf memi memf > >> e800 1048 285 963 1588 194 208 > >> e10k 1300 351 1193 1968 233 245 > >> e10k-n 1591 366 2255 2285 664 389 > >> p4 6809 9327 22170 13896 5050 5041 > >> ath 3319 8855 13011 12217 2912 3080 > > > >I wonder what the specific details was for the aove p4/ath cpus. > > > >For the $$$ difference betwen a p4/amd and c3 ... it seems > >like a non-issue for performance of spending the extra $50 - $100 > >for better performance in the above benchmark numbers > > > >( but, its a bigger price/performance gap for the lastest FSB-800 cpus > > and runs $300 - $500 or higher > > > >good stuff in either case ... > > Since power & cooling is almost always free for .edu's, yes, .edu cluster > design and others' will be extremely different. I'm not entirely clear on the compatibility of the phrase "Since power & cooling is almost always free for .edu's," and the phrase "massive chiller upgrade required". clusters especially large ones have real costs associated with power, space, and cooling that can't been hidden in your general budget... > Not to mention that .edu's cant stand losing nodes in a throwaway > design that accomodates tolerable rates of failure. (Anyone operating > a cluster in this mode? Im very curious as to how you surmounted the > psychological objections to it!) > > When you say 'TCO' you REALLY have to define what you mean -- "total cost > of OWNERSHIP" or "total cost of OPERATION"? You can own a really nice > cluster for $x if you dont have to cut $x into slices to buy it > AND pay for running power and cooling for it for n years. > > Anyone admit to being in the 'must pay for power and other operationals' > camp and want to explain how their designs differ? (especially those > that pay a premium for floor space - these edens start winning on two > fronts. Low heat = low space requirements.) > > I suspect EDENs will be viable for people in the expensive power & floorspace > areas (Bay area, etc?) > > /kc > -- -------------------------------------------------------------------------- Joel Jaeggli Academic User Services joelja at darkwing.uoregon.edu -- PGP Key Fingerprint: 1DE9 8FCA 51FB 4195 B42A 9C32 A30D 121E -- In Dr. Johnson's famous dictionary patriotism is defined as the last resort of the scoundrel. With all due respect to an enlightened but inferior lexicographer I beg to submit that it is the first. -- Ambrose Bierce, "The Devil's Dictionary" _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From math at velocet.ca Mon Jun 2 18:24:36 2003 From: math at velocet.ca (Ken Chase) Date: Mon, 2 Jun 2003 18:24:36 -0400 Subject: Cheap PCs from Wal-Mart - mini-itx In-Reply-To: ; from alvin@Maggie.Linux-Consulting.com on Mon, Jun 02, 2003 at 12:02:24AM -0700 References: <20030601132552.D11031@mikee.ath.cx> Message-ID: <20030602182436.F81856@velocet.ca> On Mon, Jun 02, 2003 at 12:02:24AM -0700, Alvin Oga's all... > >hi ya > >On Sun, 1 Jun 2003, Mike Eggleston wrote: > >> On Sun, 01 Jun 2003, Gerry Creager N5JXS wrote: >> >> > I've got a Via Epia Mini-ITX board I've gotten for another project. We >> > loaded RH 9 via PXE-Boot (with particular thanks to RGB!), as in, a >> > normal install rather than diskless boot. Works great. I've got the >> > slowest, equivalent to a 600 MHz Celeron, but that's all I needed for >> > this project... in fact, overkill. >> > >> > Runs pretty cool, even with just the 2 puny little case fans, and can be >> > driven off a 12v Wall Wart (?tm?; should I trade mark it? no...) at >> > less than 5a. The wart I have is rated for 4.5a, but I've not measured >> > it yet. > >we did lots of testing on that 12v dc-dc input to the mini-itx mb >( well, www.mini-box.com folks did all the testing and ordering of the >( custom 12v dc-to-dc atx power supply Any numbers on the PF of these ac-to-dv (then to dc) floor-brick-with- power-cables xformers? I suspect this stuff isnt worth anything til you run it off a site-wide 12V infrastructure to get a good PF rating. /kc -- Ken Chase, math at velocet.ca * Velocet Communications Inc. * Toronto, Canada Wiznet Velocet DSL.ca Datavaults 24/7: 416-967-4414 tollfree: 1-866-353-0363 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From alvin at Maggie.Linux-Consulting.com Mon Jun 2 17:18:06 2003 From: alvin at Maggie.Linux-Consulting.com (Alvin Oga) Date: Mon, 2 Jun 2003 14:18:06 -0700 (PDT) Subject: Cheap PCs from Wal-Mart In-Reply-To: Message-ID: hi ya On Mon, 2 Jun 2003, Mark Hahn wrote: > > re. the points on floating point performance, these are valid. > > But I think we should note the new Nehemiah boards which are just out. > > Well worth looking at I think - I intend to get one soon, and I'll report > > back if/when I do. > > http://www.mini-itx.com/reviews/nehemiah/?page=10#s21 > > hmm. for rhetorical purposes, let's compare the numbers from this article > to two high-end unis: > > dhry whet mmi mmf memi memf > e800 1048 285 963 1588 194 208 > e10k 1300 351 1193 1968 233 245 > e10k-n 1591 366 2255 2285 664 389 > p4 6809 9327 22170 13896 5050 5041 > ath 3319 8855 13011 12217 2912 3080 I wonder what the specific details was for the aove p4/ath cpus. For the $$$ difference betwen a p4/amd and c3 ... it seems like a non-issue for performance of spending the extra $50 - $100 for better performance in the above benchmark numbers ( but, its a bigger price/performance gap for the lastest FSB-800 cpus and runs $300 - $500 or higher good stuff in either case ... have fun alvin > I think it's pretty clear that you need to expect much lower > performance from even the 'high-end' VIA chips. if your code > more resembles dhrystone (mostly integer, cache friendly), > then you can mostly expect to scale with clock speed, and the > VIA chips might be attractive on a speed/(heat*cost) basis. > > for general clusters, where memory bandwidth and FP performance > and integrated gigabit are big advantages, VIA doesn't compete. > > data is from the article above and > http://www6.tomshardware.com/cpu/20030521/index.html > > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org > To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf > _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From hahn at physics.mcmaster.ca Mon Jun 2 19:15:12 2003 From: hahn at physics.mcmaster.ca (Mark Hahn) Date: Mon, 2 Jun 2003 19:15:12 -0400 (EDT) Subject: Cheap PCs from Wal-Mart In-Reply-To: Message-ID: > > hmm. for rhetorical purposes, let's compare the numbers from this article > > to two high-end unis: > > > > dhry whet mmi mmf memi memf > > e800 1048 285 963 1588 194 208 > > e10k 1300 351 1193 1968 233 245 > > e10k-n 1591 366 2255 2285 664 389 > > p4 6809 9327 22170 13896 5050 5041 > > ath 3319 8855 13011 12217 2912 3080 > > I wonder what the specific details was for the aove p4/ath cpus. it's not interesting or relevant. for the record, by "high-end", I meant "highest end currently available". that is, P4, FSB800, i875, 2-3 GHz. the highest-end Athlon is significantly lower-end (single ddr400 I think, PR3200). > For the $$$ difference betwen a p4/amd and c3 ... it seems sure, high-end desktops are more expensive than integrated appliance boards, no surprise there. the point is that they're also dramatically faster. being much faster means you can't just ignore how slow the VIA chips are, at least not in this context (compute clusters). remember that speed advantages are often *multiplicative*, so you might get 4x speedup from CPU, 7x from dram, 2x from disk, 8x from gigabit for a total of 448x! guaranteed not to exceed, of course, but the principle is sound... > like a non-issue for performance of spending the extra $50 - $100 > for better performance in the above benchmark numbers AMD is lagging in uniprocessors, no question, no surprise. it's astonishing that a non-fly-by-night company could simply forget to deal with that little issue of motherboards. afaikt, there are hundreds, possibly thousands of shops worldwide who could produce a basic 6-layer motherboard without breaking a sweat... _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From math at velocet.ca Mon Jun 2 19:55:24 2003 From: math at velocet.ca (Ken Chase) Date: Mon, 2 Jun 2003 19:55:24 -0400 Subject: Cheap PCs from Wal-Mart In-Reply-To: ; from joelja@darkwing.uoregon.edu on Mon, Jun 02, 2003 at 03:48:22PM -0700 References: <20030602182234.E81856@velocet.ca> Message-ID: <20030602195524.K81856@velocet.ca> On Mon, Jun 02, 2003 at 03:48:22PM -0700, Joel Jaeggli's all... >On Mon, 2 Jun 2003, Ken Chase wrote: >> Since power & cooling is almost always free for .edu's, yes, .edu cluster >> design and others' will be extremely different. > >I'm not entirely clear on the compatibility of the phrase "Since power & >cooling is almost always free for .edu's," and the phrase "massive chiller >upgrade required". clusters especially large ones have real costs >associated with power, space, and cooling that can't been hidden in your >general budget... I was generalizing a bit too harshly, but two of the designs we've shipped out to universities were free from any considerations of power, cooling or total cost of operation. :) Kinda nice to be free of the more difficult aspects of design! However, other designs of clusters and similar systems (server farms) are pathologically encumbered with such considerations to the point that they're they major concern. Both situations give rise to radically different designs, and whoever was questioning the use of the Eden must not have encountered these requirements. /kc > >> Not to mention that .edu's cant stand losing nodes in a throwaway >> design that accomodates tolerable rates of failure. (Anyone operating >> a cluster in this mode? Im very curious as to how you surmounted the >> psychological objections to it!) >> >> When you say 'TCO' you REALLY have to define what you mean -- "total cost >> of OWNERSHIP" or "total cost of OPERATION"? You can own a really nice >> cluster for $x if you dont have to cut $x into slices to buy it >> AND pay for running power and cooling for it for n years. >> >> Anyone admit to being in the 'must pay for power and other operationals' >> camp and want to explain how their designs differ? (especially those >> that pay a premium for floor space - these edens start winning on two >> fronts. Low heat = low space requirements.) >> >> I suspect EDENs will be viable for people in the expensive power & floorspace >> areas (Bay area, etc?) >> >> /kc >> > >-- >-------------------------------------------------------------------------- >Joel Jaeggli Academic User Services joelja at darkwing.uoregon.edu >-- PGP Key Fingerprint: 1DE9 8FCA 51FB 4195 B42A 9C32 A30D 121E -- > In Dr. Johnson's famous dictionary patriotism is defined as the last > resort of the scoundrel. With all due respect to an enlightened but > inferior lexicographer I beg to submit that it is the first. > -- Ambrose Bierce, "The Devil's Dictionary" > -- Ken Chase, math at velocet.ca * Velocet Communications Inc. * Toronto, Canada Wiznet Velocet DSL.ca Datavaults 24/7: 416-967-4414 tollfree: 1-866-353-0363 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From hahn at physics.mcmaster.ca Mon Jun 2 19:36:29 2003 From: hahn at physics.mcmaster.ca (Mark Hahn) Date: Mon, 2 Jun 2003 19:36:29 -0400 (EDT) Subject: Cheap PCs from Wal-Mart In-Reply-To: <5.2.0.9.2.20030602150917.02f2ba88@mailhost4.jpl.nasa.gov> Message-ID: > I went to the linked page at Toms Hardware and found reasonably close results > Athlon XP 3200+, and P4 3GHz... right, as I said "high-end". > Now, what's the power consumption of those two? > I think the P4 dissipates a max of 85W (poking around on the same Tom's page) that would be max TDP. > The C3 at 800 MHz dissipates 6.65W average and peaks at 12W > The Nehemiah at 1 GHz dissipates 11.25/15 W ave/pk I suspect VIA's "peak" is not equivalent (lower) than Intel's TDP. > So, looking at the scaling and comparing P4 against Via C3 Nehemiah > Speed 6809/1591 4.3:1 > Power 85/15 5.7:1 > > So the Via looks like it does more computes/Joule... well, for dhrystone at least. which isn't really surprising if you think about the fact that dhry is entirely on-chip, and much of the power dissipated by a high-end CPU is actually the external interface. if you consider a *real* workload which has some external load, you'd see much higher throughput on the thick system. > This is really a first order approximation. You'd have to look at > peripheral power, memory power, and power supply conversion and power > distribution efficiency. Peripherals and memory probably scale with memory > speed fairly linearly. huh? do you mean "the P4 drives ram much faster and so the ram will also dissipate more power"? > The higher powered device will have more I^2R > losses in the power supply and distribution. (Supplying 50 or so Amps at > 2-3V is no easy feat) well, the PS supplies 3.3V; the Vcc for the CPU only has to travel a couple of inches, and that's probably in a plane. I don't really thing that's an issue. > Here's the upshot... if you're in an environment where ultimate performance > is important, than you're probably best going with as few big fast > processors as you can get (the original Cray paradigm). On the other hand, > if you're in an environment where maximizing computation for a fixed amount > of power/heat dissipation resources (i.e. spaceflight or portable > operation), then things like the Via C3 start to look attractive, assuming > your task scales well. almost tautological, it's so unobjectionable ;) my whole point was that people must not even start to think that these low-power CPUs are close in performance. and even if they were, I don't think people really have an intuitive sense for what "scales well" means - I doubt *anyone* has a task that is as undemanding of the system as dhrystone is, for instance. > computing, I'd like to see more attention paid to computing per joule or > computing rate/watt. There are different tradeoffs whether you are sure, but please, let's use specFPrate per watt. even that's a bit dated, since it's so cache-friendly. but the real question is: can you afford to use a cluster node which has, say, 10-20% of the performance? you can stretch Amdahl's law a bit and see that the further you push wimpy nodes, the smaller a problem domain you can address (requires ever looser coupled programs, longer latency of individual work-units, etc). _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From lindahl at keyresearch.com Mon Jun 2 20:52:23 2003 From: lindahl at keyresearch.com (Greg Lindahl) Date: Mon, 2 Jun 2003 17:52:23 -0700 Subject: Cheap PCs from Wal-Mart In-Reply-To: <20030602182234.E81856@velocet.ca> References: <20030602182234.E81856@velocet.ca> Message-ID: <20030603005223.GA2037@greglaptop.internal.keyresearch.com> On Mon, Jun 02, 2003 at 06:22:34PM -0400, Ken Chase wrote: > I suspect EDENs will be viable for people in the expensive power & floorspace > areas (Bay area, etc?) Us? Floorspace is less than 80 cents per month per square foot, in a high-quality machineroom. There's a huge glut. Office space is cheap, too. Now 2 years ago, neither was true... -- greg _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rgb at phy.duke.edu Mon Jun 2 22:23:47 2003 From: rgb at phy.duke.edu (Robert G. Brown) Date: Mon, 2 Jun 2003 22:23:47 -0400 (EDT) Subject: Cheap PCs from Wal-Mart In-Reply-To: Message-ID: On Mon, 2 Jun 2003, Mark Hahn wrote: > but the real question is: can you afford to use a cluster node which > has, say, 10-20% of the performance? you can stretch Amdahl's law a bit > and see that the further you push wimpy nodes, the smaller a problem > domain you can address (requires ever looser coupled programs, longer > latency of individual work-units, etc). Ya. I've refrained from this discussion because I'm busy and y'all've been doing so very well anyway, but I think Mark has it dead right. Really, truly cheap-o nodes are a lovely thing for (as was proposed in the original post long ago) a tiny $1K hobby/play cluster, or even for a toy cluster for a high school cluster laboratory where $2-3K is "a lot of money" for them to raise. The purpose of such a cluster is NOT to get some piece of numerical work done as cost-effectively as possible. It is to teach a generation of kids about managing linux systems, writing programs, and the rudiments of cluster computing (Amdahl's law and so forth) in an inexpensive environment that they can "play" with. It is for cluster humans like me to have cheap miniclusters at home to play with to try things out in an environment they can "break the hell out of" without interfering with their production environment. For real numerical work, the following argue against using the cheapest possible nodes: a) MFLOPS/$$ are generally not peak for the cheapest hardware, even in raw aggregate. I "usually" find that I can get the most work done for the least money one or two clock generations back from peak/bleeding edge. b) Mark's repeated observations on "good benchmarks" in terms of which to find the cost-benefit winner are also dead on. The best benchmark is (of course) the application(s) you plan to run. Lacking numbers for that application, you'll have to do your best to guestimate it from e.g. specfp, stream, and other benchmarks. Most people on this list will find raw integer benchmarks to be somewhat irrelevant, as HPC performance tends to be dominated by floating point operations. Then there is the usual list of rate-limiting bottlenecks: cpu, memory, network. c) Cheap nodes are cheap, and in a tanstaafl world will likely break early, break often. For a few nodes in a toy cluster with some sort of warranty, your aggregate risk may be low and you may have the energy to deal with failures that occur. For 128 nodes in a production cluster, a failure a week or more will soon drive you mad. d) Cheap slow nodes probably draw MORE power per unit of work done than do faster newer more expensive nodes. I say "probably" because I haven't done all of the measurements to be able to say this for a fact, but note that a number of items in a "typical computer" chassis produce a more or less invariant draw. You have to power the power supply itself, a floppy, a NIC, a hdd, a motherboard, a video card. Some of those things actually run quieter and draw less in more expensive versions. Others draw at a rate that scales with clock (although not necessarily linearly). Cooling goes with heat. Both cost money. For example, consider an imaginary node where overhead is 40 watts and CPU/memory at speed "1" are 40 more watts. To get to speed 2 with perfect scaling you can buy two nodes and draw 160 watts total. Or you can buy one node at speed 2, forget scaling, and even if the 2x cpu draws 80 watts, end up drawing only 120 watts. If the faster CPU has a marginal cost of less than the second entire system, you end up winning on raw dollars as well (one of the things that favors fast nodes as in a). e) Finally, cheap nodes are almost obsolete when you buy them, so it should come as no surprise that in a production environment they'll be obsolete in a lot less than a typical three year lifetime. I've read cost benefit studies that suggest that from a TCO perspective even fairly MODERN nodes should be replaced every 12 to 18 months (although this particular timeframe depends a bit on a variety of overhead and management costs). Cheap nodes shouldn't even be brought home. Moore's Law is inexorable and unforgiving, and you'll find that your entire $1000 "cheap cluster" may be replaceable by a single $1000 desktop system that is twice as fast INSIDE a year from when you bought it if you're not careful. So for hobby/home, sure, wal-mart specials, cheapest homemade, web-special boxes are fine. Hell, I still run nodes at home as slow as 400 MHz Celerons -- but they aren't "production", they are for fun. At Duke I buy much more expensive, much more reliable, much faster systems, and curse the gods when even THEY break down from time to time. rgb -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb at phy.duke.edu _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From math at velocet.ca Mon Jun 2 22:40:09 2003 From: math at velocet.ca (Ken Chase) Date: Mon, 2 Jun 2003 22:40:09 -0400 Subject: Cheap PCs from Wal-Mart In-Reply-To: <5.2.0.9.2.20030602150917.02f2ba88@mailhost4.jpl.nasa.gov>; from James.P.Lux@jpl.nasa.gov on Mon, Jun 02, 2003 at 03:25:47PM -0700 References: <002401c3291b$67a52d80$8461cdc2@DREAD> <5.2.0.9.2.20030602150917.02f2ba88@mailhost4.jpl.nasa.gov> Message-ID: <20030602224009.L81856@velocet.ca> On Mon, Jun 02, 2003 at 03:25:47PM -0700, Jim Lux's all... >Which P4 and Athlon were you comparing.. >I went to the linked page at Toms Hardware and found reasonably close results >Athlon XP 3200+, and P4 3GHz... > >Now, what's the power consumption of those two? >I think the P4 dissipates a max of 85W (poking around on the same Tom's page) > >The C3 at 800 MHz dissipates 6.65W average and peaks at 12W >The Nehemiah at 1 GHz dissipates 11.25/15 W ave/pk > >So, looking at the scaling and comparing P4 against Via C3 Nehemiah >Speed 6809/1591 4.3:1 >Power 85/15 5.7:1 I found slightly different numbers for the C3-Ezra-T 800: (mind the spaces) http://www.via.com.tw/en/viac3/VIA C3 Ezra-T datasheet v1.0 .pdf taking this to its pathological conclusion: C3 Ezra T-800 C3 Nehemiah 1Ghz P4 ------------------------------------------------------------------------------ heat dissip. W 5.0/8.5pk 11.25av/15pk 85 (peak or avg?) dhrystones d 1048 1591 6809 d/W 209/123 141/106 80 ratio 2.6 / 1.5 1.8 / 1.3 1 Now I dont know which power number to choose for the C3's - the power ratings are listed as "average power dissipated while running Winstone 99 on Win98" in the PDF. And there's no numbers there for the E10k-n (the pdf link was off a June 2002 article from Tom's HW) for avg/peak power. These numbers dont really mean that much, except in a very general sense. Specific benchmarks and direct power usage readings would be much more useful, considering: - If you add a HD to the setup per CPU, you blow your total W/node figure (not to mention your price curve). - I dont know what dhrystones are an accurate measure of what everyone is doing with clusters these days - I dont know if what people do with clusters excercises average or peak power consumption for any CPU (or for some and not others). (I do know that burnMMX and burnK7 running together would blow our breakers for dual Tbird 1.3Ghz on Tyan 2460s with no HDs on 300W enermax power supplies and 512Mb ram at 8 boards/15 Amps, where as running G98 or gromacs code did not, so what's running is relevant.) Furthermore, it is to be noted that the E10K-n does require active cooling, changing the amount of space required ('volume' for airflow must be included) - though Im guessing that in a 'blade' type setup or with inline airflow over big heatsinks, you could avoid fans onboard. For computation power vs Watts, we should really be looking at the new low-power celerons, I bet they have some figures competitive to the C3. /kc >> >> dhry whet mmi mmf memi memf >>e800 1048 285 963 1588 194 208 >>e10k 1300 351 1193 1968 233 245 >>e10k-n 1591 366 2255 2285 664 389 >>p4 6809 9327 22170 13896 5050 5041 >>ath 3319 8855 13011 12217 2912 3080 >> >>I think it's pretty clear that you need to expect much lower >>performance from even the 'high-end' VIA chips. if your code >>more resembles dhrystone (mostly integer, cache friendly), >>then you can mostly expect to scale with clock speed, and the >>VIA chips might be attractive on a speed/(heat*cost) basis. >> >>for general clusters, where memory bandwidth and FP performance >>and integrated gigabit are big advantages, VIA doesn't compete. >> >>data is from the article above and >> http://www6.tomshardware.com/cpu/20030521/index.html >> >>__ > >James Lux, P.E. >Spacecraft Telecommunications Section >Jet Propulsion Laboratory, Mail Stop 161-213 >4800 Oak Grove Drive >Pasadena CA 91109 >tel: (818)354-2075 >fax: (818)393-6875 > >_______________________________________________ >Beowulf mailing list, Beowulf at beowulf.org >To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- Ken Chase, math at velocet.ca * Velocet Communications Inc. * Toronto, Canada Wiznet Velocet DSL.ca Datavaults 24/7: 416-967-4414 tollfree: 1-866-353-0363 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From math at velocet.ca Mon Jun 2 22:54:12 2003 From: math at velocet.ca (Ken Chase) Date: Mon, 2 Jun 2003 22:54:12 -0400 Subject: Cheap PCs from Wal-Mart - mini-itx - 12v In-Reply-To: ; from alvin@Maggie.Linux-Consulting.com on Mon, Jun 02, 2003 at 03:54:37PM -0700 References: <20030602182436.F81856@velocet.ca> Message-ID: <20030602225412.M81856@velocet.ca> On Mon, Jun 02, 2003 at 03:54:37PM -0700, Alvin Oga's all... > >hi ya ken > >On Mon, 2 Jun 2003, Ken Chase wrote: > >> >we did lots of testing on that 12v dc-dc input to the mini-itx mb >> >( well, www.mini-box.com folks did all the testing and ordering of the >> >( custom 12v dc-to-dc atx power supply > >these tests were for 1-z 2-z systems .. one dc-to-dc ps for each mini-itx >motherboard in the stackable case-size of a standard cdrom > >> Any numbers on the PF of these ac-to-dv (then to dc) floor-brick-with- >> power-cables xformers? I suspect this stuff isnt worth anything til you >> run it off a site-wide 12V infrastructure to get a good PF rating. > >or one can operate like a colo ?? > - take 110VAC in, convert to 12V dc batteries, generate 110VAC > to powerup the colo'd servers > >or 48V dc ( like the phone systems ) totally agreed. my point was that if you dont do this, you'll lose a fair bit in any power saved. I measured a couple laptops around the office with that horridly-named 'kill a watt' or whatever its called thing we ordered after someone on list suggested it - one shoddy old laptop xformer was running at 0.45pf! Most were in .55 to .6 range. Most PC atx ac to dc standard supplies were in .75 to .85 range. A few of our nicer servers had 0.98 to 1.0 pf. Furthermore, our UPS situation would be a fair bit more efficient I suspect were we backing up a fair bit more 12 and 48V than trying to rectify it back to AC, regardless of what losses the end systems incur (just think - ac to dc to charge the batteries, and then in an outtage, dc to ac back to dc. the horror!) Furthermore, it seems to be the only way to power these low-power systems, with DC to DC PS's - these things would probably require a 75 to 100W AC ATX PS for a diskless setup... Who makes those anymore - hard enough these days to find a 250. /kc > >have fun >alvin -- Ken Chase, math at velocet.ca * Velocet Communications Inc. * Toronto, Canada Wiznet Velocet DSL.ca Datavaults 24/7: 416-967-4414 tollfree: 1-866-353-0363 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From jhearns at freesolutions.net Tue Jun 3 00:00:35 2003 From: jhearns at freesolutions.net (John Hearns) Date: 03 Jun 2003 05:00:35 +0100 Subject: ClusterKnoppix In-Reply-To: <20030602144223.L17733@mikee.ath.cx> References: <1054534705.15377.8.camel@harwood.home> <3EDB8FBB.5060907@yakko.cimav.edu.mx> <20030602144223.L17733@mikee.ath.cx> Message-ID: <1054612836.4441.12.camel@harwood.home> On Mon, 2003-06-02 at 20:42, Mike Eggleston wrote: > Have you tried using apt? It seems to run a day or two behind the RedHat > updates, but it gets all the updates. Try setting a nightly cron entry to > execute 'apt-get update; apt-get update' for all your nodes. You can find > apt at www.freshrpms.net. > Mike, I use apt-for-rpm from Freshrpms also, and find it excellent. Last year I also did a lot of work with Current, which is an open source implementation of an up2date server: http://current.tigris.org/ This could also work well for Luis. We should also mention Yum, as Bob Brown is on the list! http://linux.duke.edu/projects/yum/ _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From math at velocet.ca Tue Jun 3 00:05:01 2003 From: math at velocet.ca (Ken Chase) Date: Tue, 3 Jun 2003 00:05:01 -0400 Subject: Cheap PCs from Wal-Mart In-Reply-To: ; from hahn@physics.mcmaster.ca on Mon, Jun 02, 2003 at 11:59:14PM -0400 References: <20030602224009.L81856@velocet.ca> Message-ID: <20030603000501.Q81856@velocet.ca> On Mon, Jun 02, 2003 at 11:59:14PM -0400, Mark Hahn's all... >> - If you add a HD to the setup per CPU, you blow your total W/node figure >> (not to mention your price curve). > >well, a desktop HD would be 5-10W, but I expect a laptop HD would be >a good match. Now you want to run this thing for years to recover the price diff between the two on power ;) What kind of data rates you get to laptop drives? /kc -- Ken Chase, math at velocet.ca * Velocet Communications Inc. * Toronto, Canada Wiznet Velocet DSL.ca Datavaults 24/7: 416-967-4414 tollfree: 1-866-353-0363 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From jhearns at freesolutions.net Mon Jun 2 23:56:23 2003 From: jhearns at freesolutions.net (John Hearns) Date: 03 Jun 2003 04:56:23 +0100 Subject: Linux Magazine article Message-ID: <1054612584.4445.6.camel@harwood.home> I just picked up the June edition of Linux Magazine here in the UK, and read it on the train back from Glasgow. This edition concerns Beowulf computing - and has an excellent article by our very own Bob Brown. Congratulations! For anyone in the UK, Linux Magazine is carried by the Borders bookshops chain. I got mine in the Buchanan Street branch. Oh, and that's the US magazine - not the excellent UK Linux Magazine, edited by John Southern. ps. Two small children sat opposite me on the train. I convinced the little boy that I lived on the train, and slept under the table at night. Wonder if he believedme, and is telling stories at school? _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From jhearns at freesolutions.net Tue Jun 3 00:15:50 2003 From: jhearns at freesolutions.net (John Hearns) Date: 03 Jun 2003 05:15:50 +0100 Subject: Cheap PCs from Wal-Mart In-Reply-To: References: Message-ID: <1054613751.4445.21.camel@harwood.home> On Tue, 2003-06-03 at 03:23, Robert G. Brown wrote: > > It is to teach a generation of kids about managing linux systems, > writing programs, and the rudiments of cluster computing (Amdahl's law > and so forth) in an inexpensive environment that they can "play" with. > It is for cluster humans like me to have cheap miniclusters at home to > play with to try things out in an environment they can "break the hell > out of" without interfering with their production environment. Good points. That's why I have my nodes - I live in a small apartment in London's Docklands. Not much spare space, and with the high temperatures in London at the moment, I'm glad of them! (Air conditioning is not that common in UK houses). Don't forget also Jim Lux's thoughts on portable/resource constrained computing. If anyone is interested, here is the abstract for my UKUUG Developers Talk. Maybe a good excuse to visit the Edinburgh Festival! http://www.ukuug.org/events/linux2003/prog/abstract-JHearns-1.shtml BTW, I'm to talk about C optimisations for the VIA processors during the talk. If anyone on the list is good with this sort of thing - specifically the latest gcc versions, or commercial compilers please drop me an email off list if I can pick your brains. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From Andrew.Cannon at nnc.co.uk Tue Jun 3 03:11:14 2003 From: Andrew.Cannon at nnc.co.uk (Cannon, Andrew) Date: Tue, 3 Jun 2003 08:11:14 +0100 Subject: Help: different kernel images for slaves. Message-ID: This might not help, but from what I've been told (I'm due to try it in our 16 node cluster soon) you can use the Kickstart program to distribute different versions of most of the software in RH across the network. I'm sure you should be able to do the same with the kernel too (I hope so...) Hope this helps a bit. Andrew -----Original Message----- From: Robin Laing [mailto:Robin.Laing at drdc-rddc.gc.ca] Sent: Monday, June 02, 2003 10:26 PM To: beowulf at beowulf.org Subject: Help: different kernel images for slaves. Hello, I am involved with adding some SMP machines to our small cluster and I am trying to find out how I tell the new machines to run the SMP kernel instead of the default in the /var/beowulf/config file. We are running RH 9 and BPpoc. Our present system is working with single processor boot image and the SMP machines will boot with only one processor working. I have compiled a SMP kernel to test but I cannot just change the boot kernel for all the other machines. Presently in /var/beowulf are: boot.img nodeboot.img nodeinfo I want to add boot_smp.img I have not seen any documentation on configuring /etc/beowulf/config to work with more than one boot image. Is it possible to define which nodes boot with which kernel? -- Robin Laing Instrumentation Technologist Voice: 1.403.544.4762 Military Engineering Section FAX: 1.403.544.4704 Defence R&D Canada - Suffield Email: Robin.Laing at DRDC-RDDC.gc.ca PO Box 4000, Station Main WWW: http://www.suffield.drdc-rddc.gc.ca Medicine Hat, AB, T1A 8K6 Canada _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf *********************************************************************************** NNC Limited Booths Hall Chelford Road Knutsford Cheshire WA16 8QZ Country of Registration: United Kingdom Registered Number: 1120437 This e-mail and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this e-mail in error please notify the NNC system manager by e-mail at eadm at nnc.co.uk. *********************************************************************************** _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From lange at informatik.Uni-Koeln.DE Tue Jun 3 05:14:59 2003 From: lange at informatik.Uni-Koeln.DE (Thomas Lange) Date: Tue, 3 Jun 2003 11:14:59 +0200 Subject: ClusterKnoppix In-Reply-To: <3EDB8FBB.5060907@yakko.cimav.edu.mx> References: <1054534705.15377.8.camel@harwood.home> <3EDB8FBB.5060907@yakko.cimav.edu.mx> Message-ID: <16092.26387.257370.530388@informatik.uni-koeln.de> >>>>> On Mon, 02 Jun 2003 11:56:11 -0600, Luis Fernando Licon Padilla said: > I have a 12 dual nodes beowulf running on Red Hat Linux 8.0, but > you know, I have the update problem since I do not want to pay > the update service to Red Hat, so my question is, Does someone > have experience about clusters running on both Debian and RedHat > distributions, who can help me to decide on this issue? Have a look at FAI, the fully automatic installation (and upgrade) for Debian. On the FAI web pages http://www.informatik.uni-koeln.de/fai/ you can find some success story of people how installed a beowulf cluster using my tool. Updates are very simple (and for free, of course) using apt-get. -- regards Thomas _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rgb at phy.duke.edu Tue Jun 3 07:05:33 2003 From: rgb at phy.duke.edu (Robert G. Brown) Date: Tue, 3 Jun 2003 07:05:33 -0400 (EDT) Subject: ClusterKnoppix In-Reply-To: <1054612836.4441.12.camel@harwood.home> Message-ID: On 3 Jun 2003, John Hearns wrote: > We should also mention Yum, as Bob Brown is on the list! > http://linux.duke.edu/projects/yum/ (Remembering that although I do love yum, it is the brainchild of Seth Vidal and Mike Stenner at Duke. I just use it, make a lot of yammering sounds on the list about making its config file xmlish, and sometimes have lunch with them and exchange a bit of banter about python, a language I've sworn not to learn so I CAN'T help with yum that I'll probably know by the end of summer...:-) rgb -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb at phy.duke.edu _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From James.P.Lux at jpl.nasa.gov Tue Jun 3 09:54:50 2003 From: James.P.Lux at jpl.nasa.gov (Jim Lux) Date: Tue, 3 Jun 2003 06:54:50 -0700 Subject: Cheap PCs from Wal-Mart - mini-itx References: <20030601132552.D11031@mikee.ath.cx> <20030602182436.F81856@velocet.ca> Message-ID: <003001c329d7$b3d9aac0$02a8a8c0@office1> > >> > Runs pretty cool, even with just the 2 puny little case fans, and can be > >> > driven off a 12v Wall Wart (?tm?; should I trade mark it? no...) at > >> > less than 5a. The wart I have is rated for 4.5a, but I've not measured > >> > it yet. > > > >we did lots of testing on that 12v dc-dc input to the mini-itx mb > >( well, www.mini-box.com folks did all the testing and ordering of the > >( custom 12v dc-to-dc atx power supply > > Any numbers on the PF of these ac-to-dv (then to dc) floor-brick-with- > power-cables xformers? I suspect this stuff isnt worth anything til you > run it off a site-wide 12V infrastructure to get a good PF rating. > > /kc Power factor, or efficiency? CE marking requires fairly good harmonic control and power factor, even for small wall warts or floor bricks. It's become fairly straightforward to design "good" power supplies, since there is huge consumer demand (via regulatory process) and the semiconductor mfrs have produced inexpensive chips to make PFC easier. Efficiency, though, isn't all that hot, there. Especially considered in the system context, because you have a linevoltage:12V conversion, and then another 12V:5 and 12V:3 conversion, each of which is probably 80-85% efficient. In a standard PC, you have only one conversion. However, you wouldn't want 12V distribution either. IR losses in the cabling would eat you alive, and maintaining regulation would be very difficult. One better solution would be to distribute unregulated 300VDC to all the mobos, and have them convert that to the required 12,5,and 3 in one step (essentially splitting the standard PC power supply down the middle). For safety reasons, you don't see many 300VDC distribution designs, though. Telco stuff uses 48VDC for distribution... high enough that IR losses aren't killing you, you can get it with batteries, and it's below the informal "50V is low voltage" shock hazard cutoff. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From James.P.Lux at jpl.nasa.gov Tue Jun 3 10:04:29 2003 From: James.P.Lux at jpl.nasa.gov (Jim Lux) Date: Tue, 3 Jun 2003 07:04:29 -0700 Subject: Cheap PCs from Wal-Mart References: Message-ID: <003d01c329d9$0cbf1fc0$02a8a8c0@office1> > > The C3 at 800 MHz dissipates 6.65W average and peaks at 12W > > The Nehemiah at 1 GHz dissipates 11.25/15 W ave/pk > > I suspect VIA's "peak" is not equivalent (lower) than Intel's TDP. Hard to know until someone sticks a current probe on the devices... > > > This is really a first order approximation. You'd have to look at > > peripheral power, memory power, and power supply conversion and power > > distribution efficiency. Peripherals and memory probably scale with memory > > speed fairly linearly. > > huh? do you mean "the P4 drives ram much faster and so the ram > will also dissipate more power"? Precisely.. CMOS, to a first order, has power dissipation proportional to clock frequency. Cycle the bus at 200 MHz and it draws twice as much power as cycling the bus at 100 MHz. I don't know if the P4 or C3 have the same bus width, too? Wider buses draw more power (for the line drivers/receivers). > > > The higher powered device will have more I^2R > > losses in the power supply and distribution. (Supplying 50 or so Amps at > > 2-3V is no easy feat) > > well, the PS supplies 3.3V; the Vcc for the CPU only has to travel > a couple of inches, and that's probably in a plane. I don't really > thing that's an issue. It's a huge issue... The efficiency of the PS is lower at 3.3V than at 5V or 12V, for instance. As far as the CPU core voltage regulator, the same applies... If you push power through anything, you're going to have more IR losses at 1.8V than at 2.5V. There's only so much copper available on the board to carry the current, and the pin or ball is only so big. On chip, there's the issue of the power carrying conductors in the metalization. The saving grace is that, for CMOS, the power dissipation also scales with voltage, so as the feature size goes down (increasing IR losses), the power consumed goes down too (reducing current). However, consider.. a contact/trace resistance of 5 milliohms, carrying a current of 20 amps, disspates 2 Watts... 5 mOhm is pretty darn low.. especially for a pin only 25 mils on a side. > > my whole point was that people must not even start to think that > these low-power CPUs are close in performance. and even if they were, > I don't think people really have an intuitive sense for what "scales well" > means - I doubt *anyone* has a task that is as undemanding of the system > as dhrystone is, for instance. > > > computing, I'd like to see more attention paid to computing per joule or > > computing rate/watt. There are different tradeoffs whether you are > > sure, but please, let's use specFPrate per watt. even that's a bit > dated, since it's so cache-friendly. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From Robin.Laing at drdc-rddc.gc.ca Tue Jun 3 10:51:44 2003 From: Robin.Laing at drdc-rddc.gc.ca (Robin Laing) Date: Tue, 03 Jun 2003 08:51:44 -0600 Subject: Help: different kernel images for slaves. Message-ID: <3EDCB600.4080500@drdc-rddc.gc.ca> Andrew, This would work but I forgot to mention that these are diskless workstations. Under bproc there is a setting in the /etc/beowulf/config for kernel image but I cannot find any documentation for assigning or allocating a different kernel for different machines. I will look at kickstart though to see if it would work and install on each system. > Message: 10 > From: "Cannon, Andrew" > To: beowulf at beowulf.org > Subject: RE: Help: different kernel images for slaves. > Date: Tue, 3 Jun 2003 08:11:14 +0100 > > This might not help, but from what I've been told (I'm due to try it in our > 16 node cluster soon) you can use the Kickstart program to distribute > different versions of most of the software in RH across the network. I'm > sure you should be able to do the same with the kernel too (I hope so...) > > Hope this helps a bit. > > Andrew > > -----Original Message----- > From: Robin Laing [mailto:Robin.Laing at drdc-rddc.gc.ca] > Sent: Monday, June 02, 2003 10:26 PM > To: beowulf at beowulf.org > Subject: Help: different kernel images for slaves. > > > Hello, > > I am involved with adding some SMP machines to our small cluster and I > am trying to find out how I tell the new machines to run the SMP > kernel instead of the default in the /var/beowulf/config file. > > We are running RH 9 and BPpoc. > > Our present system is working with single processor boot image and the > SMP machines will boot with only one processor working. I have > compiled a SMP kernel to test but I cannot just change the boot kernel > for all the other machines. > > Presently in /var/beowulf are: > boot.img nodeboot.img nodeinfo > > I want to add boot_smp.img > > I have not seen any documentation on configuring /etc/beowulf/config > to work with more than one boot image. > > Is it possible to define which nodes boot with which kernel? > > -- Robin Laing Instrumentation Technologist Voice: 1.403.544.4762 Military Engineering Section FAX: 1.403.544.4704 Defence R&D Canada - Suffield Email: Robin.Laing at DRDC-RDDC.gc.ca PO Box 4000, Station Main WWW: http://www.suffield.drdc-rddc.gc.ca Medicine Hat, AB, T1A 8K6 Canada _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- Robin Laing Instrumentation Technologist Voice: 1.403.544.4762 Military Engineering Section FAX: 1.403.544.4704 Defence R&D Canada - Suffield Email: Robin.Laing at DRDC-RDDC.gc.ca PO Box 4000, Station Main WWW: http://www.suffield.drdc-rddc.gc.ca Medicine Hat, AB, T1A 8K6 Canada _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From bryan at ayesha.phys.Virginia.EDU Tue Jun 3 10:36:24 2003 From: bryan at ayesha.phys.Virginia.EDU (Bryan K. Wright) Date: Tue, 03 Jun 2003 10:36:24 -0400 Subject: Data corruption w/ tg3 and bcm5700 drivers Message-ID: <200306031436.h53EaPJ29643@ayesha.phys.Virginia.EDU> Hi folks, We're having some problems with the tg3 and bcm5700 drivers, and I was hoping somebody out there might have a clue to share. We have several boxes based on the GA-7DPXDW+ motherboard, with 3c996 network cards. (According to the bcm5700 driver, these identify themselves as BCM95700A6.) We've tried both the bcm5700 driver and a couple of versions of the tg3 driver (1.2 and 1.5) with similar results. We've tried the tg3 drivers with both the "stock" Red Hat 2.4.18-5 kernel (from RH 7.2) and with the openMosix 2.4.20-openmosix2 kernel. We find that, transferring large (multi-gigabyte) files from one machine to another results in either an error that aborts the transfer (at a random time) or, if the transfer completes, we find that the transferred file has the same size as the original, but their checksums (from /usr/bin/sum) differ. If we try the transfer using scp, the transfer always aborts, often with a message about a corrupt Message Authentication Code. Using ftp, the transfer will sometimes complete. Interestingly, we find that if we do the experiment with a file full of zeros (e.g., dd if=/dev/zero of=junk.dat ...) the checksums match (on those occasions when the transfer completes, that is). Real data is always corrupted, though. We've tried this with nothing but a cross-connect cable between the boxes, with the same results. Any suggestions would be appreciated, either solutions or diagnostics we can try. Thanks in advance, Bryan Wright -- =============================================================================== Bryan Wright |"If you take cranberries and stew them like Physics Department | applesauce, they taste much more like prunes University of Virginia | than rhubarb does." -- Groucho Charlottesville, VA 22901 | (434) 924-7218 | bryan at virginia.edu =============================================================================== _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From jhearns at freesolutions.net Tue Jun 3 11:29:59 2003 From: jhearns at freesolutions.net (John Hearns) Date: Tue, 3 Jun 2003 16:29:59 +0100 Subject: Help: different kernel images for slaves. References: <3EDCB600.4080500@drdc-rddc.gc.ca> Message-ID: <01dd01c329e5$01ab29b0$8461cdc2@DREAD> ----- Original Message ----- From: "Robin Laing" To: Sent: Tuesday, June 03, 2003 3:51 PM Subject: RE: Help: different kernel images for slaves. > Andrew, > > This would work but I forgot to mention that these are diskless > workstations. Under bproc there is a setting in the > /etc/beowulf/config for kernel image but I cannot find any > documentation for assigning or allocating a different kernel for > different machines. > > I will look at kickstart though to see if it would work and install on > each system. You can do diskless installs using Linux Terminal Server Project http://www.ltsp.org Loading different kernels would be a matter of editing the /etc/dhcpd.conf and creating an entry for each workstation. The LTSP documentation gives an example, where 'filename' is the path to the kernel for that particular workstation: group { use-host-decl-names on; option log-servers 192.168.0.254; host ws001 { hardware ethernet 00:E0:18:E0:04:82; fixed-address 192.168.0.1; filename "/lts/vmlinuz.ltsp"; } } I stress that I haven't actually DONE this - though I have set up and demoed LTSP. The OSCAR people are also working on OSCAR-lite, though I don't know if you can distribute different kernels using it. LTSP might be a good way for you to go - I can help you getting it running, though the documentation is good. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rajesh at utdallas.edu Tue Jun 3 11:31:06 2003 From: rajesh at utdallas.edu (Rajesh Bhairampally) Date: Tue, 3 Jun 2003 10:31:06 -0500 Subject: Cheap PCs from Wal-Mart References: <1054613751.4445.21.camel@harwood.home> Message-ID: <020501c329e5$26ba77b0$c900a8c0@Rajesh> > > It is to teach a generation of kids about managing linux systems, > > writing programs, and the rudiments of cluster computing (Amdahl's law > > and so forth) in an inexpensive environment that they can "play" with. i am new to cluster computing environment and would like to learn with hand-on experience. i would like to start with two linux boxes (i recently placed an order for PCs from walmart) in my home environment. I am wondering if there are any tutorials/manuals for newbees to teach beowulf clustering concepts and hands-on tutorials. Any help is greately appreciated. Thanks, Rajesh _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From Robin.Laing at drdc-rddc.gc.ca Tue Jun 3 11:48:46 2003 From: Robin.Laing at drdc-rddc.gc.ca (Robin Laing) Date: Tue, 03 Jun 2003 09:48:46 -0600 Subject: Help: different kernel images for slaves. In-Reply-To: <01dd01c329e5$01ab29b0$8461cdc2@DREAD> References: <3EDCB600.4080500@drdc-rddc.gc.ca> <01dd01c329e5$01ab29b0$8461cdc2@DREAD> Message-ID: <3EDCC35E.8060301@drdc-rddc.gc.ca> Hello John, This is one of the things I am looking at. I just found out that the MB has DHCP or bootp built in. Our present cluster is using beoboot and the administrator (part time and I am going to be the other) wants to use beoboot. I am now reading documentation on this matter. John Hearns wrote: > ----- Original Message ----- > From: "Robin Laing" > To: > Sent: Tuesday, June 03, 2003 3:51 PM > Subject: RE: Help: different kernel images for slaves. > > > >>Andrew, >> >>This would work but I forgot to mention that these are diskless >>workstations. Under bproc there is a setting in the >>/etc/beowulf/config for kernel image but I cannot find any >>documentation for assigning or allocating a different kernel for >>different machines. >> >>I will look at kickstart though to see if it would work and install on >>each system. > > > You can do diskless installs using Linux Terminal Server Project > http://www.ltsp.org > > Loading different kernels would be a matter of editing the > /etc/dhcpd.conf and creating an entry for each workstation. > The LTSP documentation gives an example, > where 'filename' is the path to the kernel for that particular workstation: > > > group { > use-host-decl-names on; > option log-servers 192.168.0.254; > > host ws001 { > hardware ethernet 00:E0:18:E0:04:82; > fixed-address 192.168.0.1; > filename "/lts/vmlinuz.ltsp"; > } > } > > > > I stress that I haven't actually DONE this - though I have set up and demoed > LTSP. > > > The OSCAR people are also working on OSCAR-lite, though I don't know if you > can > distribute different kernels using it. > LTSP might be a good way for you to go - I can help you getting it running, > though the documentation > is good. > -- Robin Laing Instrumentation Technologist Voice: 1.403.544.4762 Military Engineering Section FAX: 1.403.544.4704 Defence R&D Canada - Suffield Email: Robin.Laing at DRDC-RDDC.gc.ca PO Box 4000, Station Main WWW: http://www.suffield.drdc-rddc.gc.ca Medicine Hat, AB, T1A 8K6 Canada _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rgb at phy.duke.edu Tue Jun 3 12:34:30 2003 From: rgb at phy.duke.edu (Robert G. Brown) Date: Tue, 3 Jun 2003 12:34:30 -0400 (EDT) Subject: Cheap PCs from Wal-Mart In-Reply-To: <020501c329e5$26ba77b0$c900a8c0@Rajesh> Message-ID: On Tue, 3 Jun 2003, Rajesh Bhairampally wrote: > > > It is to teach a generation of kids about managing linux systems, > > > writing programs, and the rudiments of cluster computing (Amdahl's law > > > and so forth) in an inexpensive environment that they can "play" with. > > i am new to cluster computing environment and would like to learn with > hand-on experience. > i would like to start with two linux boxes (i recently placed an order for > PCs from walmart) in > my home environment. I am wondering if there are any tutorials/manuals for > newbees to teach beowulf clustering concepts and hands-on tutorials. Any > help is greately appreciated. In Linux Magazine (already mentioned today) Forrest Hoffman (a long-time beowulfer) runs an "Extreme Linux" column that you might find very helpful. Forrest published material every month that ranages from newbie/tutorial to fairly advanced. On www.linux-mag.html his current online article appears to be "Message Passing for Master/Slave Programs". This article explores the use of MPI; previous columns have covered PVM and lots of other stuff. Both PVM and MPI have demo/examples in their installation kit(s). There is additional information on cluster setup and operation in many places -- so many I have a hard time keeping them straight myself. To help my own memory, I've collected a LOT of resources you might find useful on http://www.phy.duke.edu/brahma/index.php In particular, http://www.phy.duke.edu/brahma/Resources/links.php has close to everything clusteroid that I've been able to find -- FAQ, HOWTO, Underground, Scyld, bproc, FAI, online magazines devoted to clustering in whole or in part, newsletters, and more. This site is "almost" up to date -- I rebuilt it a short time ago, and noticed that it still has indentation problems while reviewing it a moment ago:-) BTW (to all the list cluster experts) I'd cherish suggestions for additions, deletions, or modifications to this page and its companion vendors.php page. Vendors should carefully read about the tee-shirt clause.;-) Tee shirts aren't really required, but (as I note) they can't hurt... rgb > > Thanks, > Rajesh > > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org > To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf > -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb at phy.duke.edu _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From luis.licon at yakko.cimav.edu.mx Tue Jun 3 14:28:14 2003 From: luis.licon at yakko.cimav.edu.mx (Luis Fernando Licon Padilla) Date: Tue, 03 Jun 2003 12:28:14 -0600 Subject: ClusterKnoppix References: <1054534705.15377.8.camel@harwood.home> <3EDB8FBB.5060907@yakko.cimav.edu.mx> Message-ID: <3EDCE8BE.1010502@yakko.cimav.edu.mx> Luis Fernando Licon Padilla wrote: > John Hearns wrote: > >> I saw this on Slashdot a few days ago. >> >> It is a customised Knoppix distribution, with an OpenMosix patched >> kernel, and PXE/DHCP enabled etc. >> I intend to try this out in the next few days. >> >> For those who haven't used Knoppix, it is a CD which contains a bootable >> Debian-based distribution which installs itself in memory, not touching >> the hard drive. Knoppix has excellent hardware detection, and sets up >> a good desktop. It 'just works'. >> Uses might include demonstrating a Linux desktop to friends/colleagues, >> testing if a PC can run Linux before buying/accepting it, or as a rescue >> disk for troubleshooting an already installed Linux system. >> >> John Hearns >> >> _______________________________________________ >> Beowulf mailing list, Beowulf at beowulf.org >> To change your subscription (digest mode or unsubscribe) visit >> http://www.beowulf.org/mailman/listinfo/beowulf >> >> >> >> > Hi you all guys, > > I have a 12 dual nodes beowulf running on Red Hat Linux 8.0, but you > know, I have the update problem since I do not want to pay the update > service to Red Hat, so my question is, Does someone have experience > about clusters running on both Debian and RedHat distributions, who > can help me to decide on this issue? > > Thanks, > > Luis > Thank you all guys for sharing your experiences with me, Sincerily, Luis -- ISC Luis Fernando Licon Padilla Advanced Materials Research Center Miguel de Cervantes 120 Complejo Industrial Chihuahua C.P.31109 Chihuahua, Chih. Mexico Phone: 52 (614)4391154 Fax: 52 (614)4391112 alternative e-mail: lordsirion2002 at yahoo.com _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From bogdan.costescu at iwr.uni-heidelberg.de Tue Jun 3 14:32:34 2003 From: bogdan.costescu at iwr.uni-heidelberg.de (Bogdan Costescu) Date: Tue, 3 Jun 2003 20:32:34 +0200 (CEST) Subject: Data corruption w/ tg3 and bcm5700 drivers In-Reply-To: <200306031436.h53EaPJ29643@ayesha.phys.Virginia.EDU> Message-ID: On Tue, 3 Jun 2003, Bryan K. Wright wrote: > We've tried both the bcm5700 driver and a couple of versions > of the tg3 driver (1.2 and 1.5) with similar results. We've tried the > tg3 drivers with both the "stock" Red Hat 2.4.18-5 kernel (from RH > 7.2) and with the openMosix 2.4.20-openmosix2 kernel. RedHat has released several kernels after 2.4.18-5 and IIRC some were related to bugs in tg3 driver. The current one just released is 2.4.20-18.7 (for 7.2). > We find that, transferring large (multi-gigabyte) files from > one machine to another results in either an error that aborts the > transfer (at a random time) or, if the transfer completes, we find > that the transferred file has the same size as the original, but > their checksums (from /usr/bin/sum) differ. This might also be a memory problem on one (or both) computer(s). Let memtest86 go through several cycles of its tests to make sure that this is not the case. -- Bogdan Costescu IWR - Interdisziplinaeres Zentrum fuer Wissenschaftliches Rechnen Universitaet Heidelberg, INF 368, D-69120 Heidelberg, GERMANY Telephone: +49 6221 54 8869, Telefax: +49 6221 54 8868 E-mail: Bogdan.Costescu at IWR.Uni-Heidelberg.De _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From wrp at alpha0.bioch.virginia.edu Tue Jun 3 15:55:11 2003 From: wrp at alpha0.bioch.virginia.edu (William R.Pearson) Date: Tue, 3 Jun 2003 15:55:11 -0400 Subject: Data corruption w/ tg3 and bcm5700 drivers Message-ID: <494D29CF-95FD-11D7-9C0C-003065F9A63A@alpha0.bioch.virginia.edu> We have had terrible problems with the tg3/bcm5700 drivers as well, but only on our head machines with dual 3Com996 boards. We have not tried a new kernel for the past 3 months, but before that we tried everything from 2.4.18-3 to 2.4.20, the ONLY two versions of the kernel that were stable for several hours were 2.4.18-3 (original with RH7.3) and 2.4.18-4. We have not had any problems with machines with a single 3Com996 NIC (our nodes). We are very frustrated with dual 3Com996 support, as there have been at least 4 kernel upgrades that specifically mentioned this problem, and none of them have fixed it. Our machines are dual 1 GHz PIII with supermicro MBs. Bill Pearson _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From xyzzy at speakeasy.org Tue Jun 3 16:11:39 2003 From: xyzzy at speakeasy.org (Trent Piepho) Date: Tue, 3 Jun 2003 13:11:39 -0700 (PDT) Subject: measuring power usage Message-ID: I seem to recall reading in a mailing list, I think this one, about a relatively inexpensive device that would measure the real wattage and power factor of electronics. I think that someone on the list bought one and posted a review. Does this ring a bell with anyone? I'd like to get some real numbers about what our cluster is drawing and how much we capacity we have for new hardware. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From selva at mailaps.org Tue Jun 3 16:26:12 2003 From: selva at mailaps.org (Selva Nair) Date: Tue, 3 Jun 2003 16:26:12 -0400 (EDT) Subject: Data corruption w/ tg3 and bcm5700 drivers In-Reply-To: <200306031436.h53EaPJ29643@ayesha.phys.Virginia.EDU> Message-ID: On Tue, 3 Jun 2003, Bryan K. Wright wrote: > Hi folks, > > We're having some problems with the tg3 and bcm5700 drivers, > and I was hoping somebody out there might have a clue to share. > > We have several boxes based on the GA-7DPXDW+ motherboard, > with 3c996 network cards. (According to the bcm5700 driver, these > identify themselves as BCM95700A6.) > > We've tried both the bcm5700 driver and a couple of versions > of the tg3 driver (1.2 and 1.5) with similar results. We've tried the > tg3 drivers with both the "stock" Red Hat 2.4.18-5 kernel (from RH > 7.2) and with the openMosix 2.4.20-openmosix2 kernel. I have had very similar issues with the 3com 996 cards using Redhat 7.0-&.2 kernels. I switched to the Broadcom BCM5700 driver distributed by Broadcom Corporation and the network is pretty stable under constant heavy load since then. I use their driver source version 2.0.32 (release date: 01/02/2002) rebuilt locally. I cant recall where I downloaded the sources from. Selva _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From v_454 at yahoo.com Tue Jun 3 16:19:57 2003 From: v_454 at yahoo.com (Steve Elliot) Date: Tue, 3 Jun 2003 13:19:57 -0700 (PDT) Subject: Large File Support for RedHat Linux 8.0 Kernal 2.4.18-14 Message-ID: <20030603201957.52444.qmail@web40506.mail.yahoo.com> Hello, I am trying to find out if there is a newer version of the LFS rpms. Currently I can only find the rpms for Kernal-2.2.12-20 and glibc-2.1 on sclyd's site. Any help would be appreciated and thanks in advance. Steve Elliot __________________________________ Do you Yahoo!? Yahoo! Calendar - Free online calendar with sync to Outlook(TM). http://calendar.yahoo.com _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From Robin.Laing at drdc-rddc.gc.ca Tue Jun 3 16:50:14 2003 From: Robin.Laing at drdc-rddc.gc.ca (Robin Laing) Date: Tue, 03 Jun 2003 14:50:14 -0600 Subject: Help: different kernel images for slaves - Solved Message-ID: <3EDD0A06.9060706@drdc-rddc.gc.ca> The administrator found the solution in the source tree Release Notes for beoboot-cm.1.4. In the /etc/beowulf/config [...] Support for multiple boot images has been added via the configuration file. For example: bootfile /var/beowulf/boot.img # this is the default boot image bootfile 5 /var/beowulf/boot2.img # Node 5 takes a different boot file. You can also supply comma separated node ranges like 5-10,100-120 [...] Now if only this would get put into the normal documentation. Isn't it strange how documentation in any field seems to be the last thing that gets completed. Of course this is a start in the process of getting the kernel to work. The process does find the correct kernel on boot and thus we are still working with only one procedure for the whole cluster. Now to get all the other little things to work. :) I hope that this info will be of use for others. -- Robin Laing Instrumentation Technologist Voice: 1.403.544.4762 Military Engineering Section FAX: 1.403.544.4704 Defence R&D Canada - Suffield Email: Robin.Laing at DRDC-RDDC.gc.ca PO Box 4000, Station Main WWW: http://www.suffield.drdc-rddc.gc.ca Medicine Hat, AB, T1A 8K6 Canada _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From becker at scyld.com Tue Jun 3 17:36:25 2003 From: becker at scyld.com (Donald Becker) Date: Tue, 3 Jun 2003 17:36:25 -0400 (EDT) Subject: Data corruption w/ tg3 and bcm5700 drivers In-Reply-To: <494D29CF-95FD-11D7-9C0C-003065F9A63A@alpha0.bioch.virginia.edu> Message-ID: On Tue, 3 Jun 2003, William R.Pearson wrote: > We have had terrible problems with the tg3/bcm5700 drivers as well, > but only on our head machines with dual 3Com996 boards. We use the bcm5700 driver with our distribution. Although we have to frequently update it to support new IDs, it has been much more reliable than the tg3 driver. The tg3 driver has had many new versions that claim to have fixed all of the serious problems with the previous version, and be even faster. But some of the new versions are closely tied to subtle interface changes in 2.5, which makes it risky to just back-port to production kernels without careful study of what the changes mean. [[ Of course, I have strong personal opinions about network driver API changes, backward compatible stable driver releases and the difficulty of keeping driver code working. I'm not above saying "not as easy as it looks from the outside, eh?". ]] -- Donald Becker becker at scyld.com Scyld Computing Corporation http://www.scyld.com 914 Bay Ridge Road, Suite 220 Scyld Beowulf cluster system Annapolis MD 21403 410-990-9993 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From lindahl at keyresearch.com Tue Jun 3 17:56:10 2003 From: lindahl at keyresearch.com (Greg Lindahl) Date: Tue, 3 Jun 2003 14:56:10 -0700 Subject: Data corruption w/ tg3 and bcm5700 drivers In-Reply-To: <494D29CF-95FD-11D7-9C0C-003065F9A63A@alpha0.bioch.virginia.edu> References: <494D29CF-95FD-11D7-9C0C-003065F9A63A@alpha0.bioch.virginia.edu> Message-ID: <20030603215610.GA1884@greglaptop.internal.keyresearch.com> On Tue, Jun 03, 2003 at 03:55:11PM -0400, William R.Pearson wrote: > We have had terrible problems with the tg3/bcm5700 drivers as well, > but only on our head machines with dual 3Com996 boards. Bill, Do you see the instability when you're only using 1 of the interfaces? Do you see the instability when you boot a uniprocessor kernel? These are the kinds of experiments you can do to narrow down the problem. greg _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From becker at scyld.com Tue Jun 3 17:43:10 2003 From: becker at scyld.com (Donald Becker) Date: Tue, 3 Jun 2003 17:43:10 -0400 (EDT) Subject: Large File Support for RedHat Linux 8.0 Kernal 2.4.18-14 In-Reply-To: <20030603201957.52444.qmail@web40506.mail.yahoo.com> Message-ID: On Tue, 3 Jun 2003, Steve Elliot wrote: > I am trying to find out if there is a newer version of > the LFS rpms. Currently I can only find the rpms for > Kernal-2.2.12-20 and glibc-2.1 on sclyd's site. Easy answer: we were the first commercial distribution with large file LFS support, way back in the days of 2.2 kernel (circa 2000). The 2.4 kernel and current C library have LFS support, thus new patch sets are not needed. -- Donald Becker becker at scyld.com Scyld Computing Corporation http://www.scyld.com 914 Bay Ridge Road, Suite 220 Scyld Beowulf cluster system Annapolis MD 21403 410-990-9993 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From selva at mailaps.org Tue Jun 3 18:11:06 2003 From: selva at mailaps.org (Selva Nair) Date: Tue, 3 Jun 2003 18:11:06 -0400 (EDT) Subject: measuring power usage In-Reply-To: Message-ID: On Tue, 3 Jun 2003, Trent Piepho wrote: > I seem to recall reading in a mailing list, I think this one, about a > relatively inexpensive device that would measure the real wattage and power > factor of electronics. I think that someone on the list bought one and posted > a review. Does this ring a bell with anyone? I'd like to get some real > numbers about what our cluster is drawing and how much we capacity we have for > new hardware. google "kill a watt" Selva _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rgb at phy.duke.edu Tue Jun 3 18:40:46 2003 From: rgb at phy.duke.edu (Robert G. Brown) Date: Tue, 3 Jun 2003 18:40:46 -0400 (EDT) Subject: measuring power usage In-Reply-To: Message-ID: On Tue, 3 Jun 2003, Selva Nair wrote: > On Tue, 3 Jun 2003, Trent Piepho wrote: > > > I seem to recall reading in a mailing list, I think this one, about a > > relatively inexpensive device that would measure the real wattage and power > > factor of electronics. I think that someone on the list bought one and posted > > a review. Does this ring a bell with anyone? I'd like to get some real > > numbers about what our cluster is drawing and how much we capacity we have for > > new hardware. > > google "kill a watt" Yes, very lovely thingie. There are links to at least one source on the brahma vendors page, and I >>have<< pictures of a kill-a-watt in action, if I ever overcome my innate inertia and post them on the brahma photo tour... rgb > > Selva > > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org > To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf > -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb at phy.duke.edu _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From dlane at ap.stmarys.ca Tue Jun 3 18:50:38 2003 From: dlane at ap.stmarys.ca (Dave Lane) Date: Tue, 03 Jun 2003 19:50:38 -0300 Subject: measuring power usage In-Reply-To: References: Message-ID: <5.2.0.9.0.20030603194551.023ddb08@crux.stmarys.ca> At 06:40 PM 6/3/2003 -0400, Robert G. Brown wrote: >On Tue, 3 Jun 2003, Selva Nair wrote: > > > On Tue, 3 Jun 2003, Trent Piepho wrote: > > > > > I seem to recall reading in a mailing list, I think this one, about a > > > relatively inexpensive device that would measure the real wattage and > power > > > factor of electronics. I think that someone on the list bought one > and posted > > > a review. Does this ring a bell with anyone? I'd like to get some real > > > numbers about what our cluster is drawing and how much we capacity we > have for > > > new hardware. > > > > google "kill a watt" > >Yes, very lovely thingie. There are links to at least one source on the >brahma vendors page, and I >>have<< pictures of a kill-a-watt in action, >if I ever overcome my innate inertia and post them on the brahma photo >tour... There is nothing in the flier that indicates that it measures rms amps and hence rms watts. Computers are not resistive loads and they draw current in a switching fashion. As a result they usually produce a lower current reading on a non-rms meter than they are actually using. From memory the measurements I did produced about a 20% difference in current measurements between Fluke rms and ordinary meters. ... Dave _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From alvin at Maggie.Linux-Consulting.com Tue Jun 3 19:19:38 2003 From: alvin at Maggie.Linux-Consulting.com (Alvin Oga) Date: Tue, 3 Jun 2003 16:19:38 -0700 (PDT) Subject: measuring power usage - meter In-Reply-To: <5.2.0.9.0.20030603194551.023ddb08@crux.stmarys.ca> Message-ID: On Tue, 3 Jun 2003, Dave Lane wrote: > At 06:40 PM 6/3/2003 -0400, Robert G. Brown wrote: .. > > > google "kill a watt" > > > >Yes, very lovely thingie. There are links to at least one source on the > >brahma vendors page, and I >>have<< pictures of a kill-a-watt in action, > >if I ever overcome my innate inertia and post them on the brahma photo > >tour... i hear a regular power meter is about $100 ... and i would assume it would be accurate down to the pennies since the local utilities supposed to be charging $0.xx per kwhr c ya alvin _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From xyzzy at speakeasy.org Tue Jun 3 19:35:56 2003 From: xyzzy at speakeasy.org (Trent Piepho) Date: Tue, 3 Jun 2003 16:35:56 -0700 (PDT) Subject: measuring power usage In-Reply-To: <5.2.0.9.0.20030603194551.023ddb08@crux.stmarys.ca> Message-ID: On Tue, 3 Jun 2003, Dave Lane wrote: > > > google "kill a watt" > > > >Yes, very lovely thingie. There are links to at least one source on the > >brahma vendors page, and I >>have<< pictures of a kill-a-watt in action, > >if I ever overcome my innate inertia and post them on the brahma photo > >tour... Thanks to all that responded. Cute pun for a name, but it completely thwarted my google search for terms like "power factor" and "meter". BTW, is there a real searchable list archive anywhere? google groups has lists.beowulf but it looks like they stopped archiving that in 2000. The archives on beowulf.org aren't searchable, unless you try to use a google web search to find pages that were indexed. > There is nothing in the flier that indicates that it measures rms amps and > hence rms watts. Computers are not resistive loads and they draw current in It measures power factor and frequency, so it should be doing what is necessary to find true rms. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From dlane at ap.stmarys.ca Tue Jun 3 20:00:21 2003 From: dlane at ap.stmarys.ca (Dave Lane) Date: Tue, 03 Jun 2003 21:00:21 -0300 Subject: measuring power usage In-Reply-To: References: <5.2.0.9.0.20030603194551.023ddb08@crux.stmarys.ca> Message-ID: <5.2.0.9.0.20030603205241.023df5a8@crux.stmarys.ca> At 04:35 PM 6/3/2003 -0700, Trent Piepho wrote: > > There is nothing in the flier that indicates that it measures rms amps and > > hence rms watts. Computers are not resistive loads and they draw > current in > >It measures power factor and frequency, so it should be doing what is >necessary to find true rms. Measuring true rms current depends on measuring in some way the area under the curve from the shape of the waveform. Non-rms meters usually rectify the voltage to all positive voltage and low pass filter it to convert to a DC value, then a fudge factor is used which assumes the waveform is a sine wave (which it isn't). ... Dave ps. what you really need for an accurate measurement of RMS power and power factor is something like: http://ca.fluke.com/caen/products/features.htm?cs_id=5179(FlukeProducts)&category=PHASE1(FlukeProducts) which unfortunely is about $1900US - ouch! There are likely other alternatives that are cheaper. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rgb at phy.duke.edu Tue Jun 3 21:09:23 2003 From: rgb at phy.duke.edu (Robert G. Brown) Date: Tue, 3 Jun 2003 21:09:23 -0400 (EDT) Subject: measuring power usage In-Reply-To: Message-ID: On Tue, 3 Jun 2003, Trent Piepho wrote: > > There is nothing in the flier that indicates that it measures rms amps and > > hence rms watts. Computers are not resistive loads and they draw current in > > It measures power factor and frequency, so it should be doing what is > necessary to find true rms. Yeah, it's hard to say whether it does or doesn't even from the brochure that comes with it. My recollection is that it "implies" that it does, since they go out of their way to indicate that appropriate usage is to measure draw of e.g. refrigerators and other loads that typically have PF<1. However, the (non-PFC) switching power supplies used in PC's only draw current for roughly the middle third of each half-cycle and have a significant harmonic load (and harmonic distortion) at 180 Hz; it isn't just a matter of evaluating _avg for w = 2\pi 60 and some small/measured delta. So it is anybody's guess if it does it right. Still, even with a 10% or so fudge on the power factor, I find the device to be very useful. Without it, it is fairly difficult to have ANY idea of the power draw of a given PC. Even if it is basically returning just (1/2)V_0 I_0 that's useful information. And all for less than $50 delivered... rgb -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb at phy.duke.edu _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From James.P.Lux at jpl.nasa.gov Wed Jun 4 00:25:19 2003 From: James.P.Lux at jpl.nasa.gov (Jim Lux) Date: Tue, 3 Jun 2003 21:25:19 -0700 Subject: Kill-A-Watt Re: measuring power usage References: <5.2.0.9.0.20030603194551.023ddb08@crux.stmarys.ca> Message-ID: <00b701c32a51$4e9dfc40$02a8a8c0@office1> I know someone who has reverse engineered the Kill-A-Watt, as well as done extensive testing at various power factors and waveforms. The thing really does measure actual instantaneous power (averaged over a few cycles), so weird current waveforms, etc. don't fool it. It definitely measures active power vs apparent power (RMS V * RMS I) as well as other parameters.. Accurate to a few percent, which is substantially better than most inexpensive DMMs. > >Yes, very lovely thingie. There are links to at least one source on the > >brahma vendors page, and I >>have<< pictures of a kill-a-watt in action, > >if I ever overcome my innate inertia and post them on the brahma photo > >tour... > > There is nothing in the flier that indicates that it measures rms amps and > hence rms watts. Computers are not resistive loads and they draw current in > a switching fashion. As a result they usually produce a lower current > reading on a non-rms meter than they are actually using. From memory the > measurements I did produced about a 20% difference in current measurements > between Fluke rms and ordinary meters. > _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From James.P.Lux at jpl.nasa.gov Wed Jun 4 00:41:19 2003 From: James.P.Lux at jpl.nasa.gov (Jim Lux) Date: Tue, 3 Jun 2003 21:41:19 -0700 Subject: Cheap PCs from Wal-Mart References: Message-ID: <00c301c32a53$8b3d2c00$02a8a8c0@office1> From: "Mark Hahn" To: "Jim Lux" Sent: Tuesday, June 03, 2003 4:32 PM Subject: Re: Cheap PCs from Wal-Mart > > > > This is really a first order approximation. You'd have to look at > > > > peripheral power, memory power, and power supply conversion and power > > > > distribution efficiency. Peripherals and memory probably scale with > > memory > > > > speed fairly linearly. > > > > > > huh? do you mean "the P4 drives ram much faster and so the ram > > > will also dissipate more power"? > > > > Precisely.. CMOS, to a first order, has power dissipation proportional to > > clock frequency. Cycle the bus at 200 MHz and it draws twice as much power > > as cycling the bus at 100 MHz. I don't know if the P4 or C3 have the same > > bus width, too? Wider buses draw more power (for the line > > drivers/receivers). > > your basic premise in all of this is that you have some kind of > computational task which simply doesn't need memory bandwidth > or FP power. yes, desktop/server CPUs are not at all optimized > for that kind of load, and never will be, so why aren't you using > a mips or arm chip? Because you want to take advantage of the millions (billions?) of dollars being spent on development for the consumer market. Sure, you could use a more highly optimized chipset, or you could even spend a few tens of millions of dollars and design your own. However, you'd not be able to reap the benefits of: 1) Extensive defacto testing of the consumer chip's internal logic... 2) Amortization of development costs across literally millions of units 3) Development tools at low cost 4) Large numbers of people familiar with the instruction set, and the development tools. One could, of course, acquire all these things with other approaches, however, at some point you have to decide, for a low volume application, whether you want to spend your money on one-off development, or just buying some number more processors. The latter approach has a graceful degradation path in the event of failures. Granted, this grossly oversimplifies what goes into those sorts of applications, but, the real point is that the fastest chip isn't always the best system solution. > > It's a huge issue... The efficiency of the PS is lower at 3.3V than at 5V or > > 12V, for instance. As far as the CPU core voltage regulator, the same > > applies... If you push power through anything, you're going to have more IR > > losses at 1.8V than at 2.5V. There's only so much copper available on the > > board to carry the current, and the pin or ball is only so big. On chip, > > except that we're talking about a whole *plane* devoted to VCC > (which can be modeled as a rather large wire), and literally hundreds > of pins. I'm guessing that loss is on the order of 2-5%. People are more than willing to spend hundreds of thousands of dollars to increase efficiency 5% on spaceflight applications. If all you got is 100W from solar panels, constrained by fundamental limits (how far you are from the sun, how much mass you're willing to devote to panels, etc.), then there are situations where 5% is a very big deal. Especially if it makes a big difference in thermal management. > > > However, consider.. a contact/trace resistance of 5 milliohms, carrying a > > current of 20 amps, disspates 2 Watts... 5 mOhm is pretty darn low.. > > especially for a pin only 25 mils on a side. > > except when there are 100 of them. P4, 478 pin package, 85 power pins, 180 ground pins.. who knows what sort of bond wires connect them internally, etc. Sure, you've got lots of pins, and lots of copper on the board, but, still, running 100W in small area is no trivial matter. > > as I said, I'm not denigrating the value of low-power chips in certain > domains. I just don't think there's a good argument for compute clusters > of low-power chips, at least not general-purpose ones. and for special > purpose, I'd suggest offloading all the hard stuff onto an FPGA or the like, > and using a CPU that doesn't even pretend to be competitive... Aha.. but isn't the whole point of cluster computing a'la Beowulf to accept some inefficiencies over a custom design in exchange for very attractive pricing of a commodity component? Otherwise, wouldn't everyone just go out and buy N-way Crays? > > regards, mark hahn. > _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From James.P.Lux at jpl.nasa.gov Wed Jun 4 00:43:17 2003 From: James.P.Lux at jpl.nasa.gov (Jim Lux) Date: Tue, 3 Jun 2003 21:43:17 -0700 Subject: measuring power usage References: <5.2.0.9.0.20030603194551.023ddb08@crux.stmarys.ca> <5.2.0.9.0.20030603205241.023df5a8@crux.stmarys.ca> Message-ID: <00ca01c32a53$d1433280$02a8a8c0@office1> Kill-A-Watt does just this.. measures actual waveforms. Quite a novel feat, really, to package it all up and sell it for $40.. What the Fluke gives you that the Kill-A-Watt doesn't is things like an external interface, calibration manuals, multiple ranges, etc. The Fluke's probably also potentially calibrateable to a NIST traceable standard, and is probably more reliable. ----- Original Message ----- From: "Dave Lane" To: "Beowulf Mailing list" Sent: Tuesday, June 03, 2003 5:00 PM Subject: Re: measuring power usage > At 04:35 PM 6/3/2003 -0700, Trent Piepho wrote: > > > > There is nothing in the flier that indicates that it measures rms amps and > > > hence rms watts. Computers are not resistive loads and they draw > > current in > > > >It measures power factor and frequency, so it should be doing what is > >necessary to find true rms. > > Measuring true rms current depends on measuring in some way the area under > the curve from the shape of the waveform. Non-rms meters usually rectify > the voltage to all positive voltage and low pass filter it to convert to a > DC value, then a fudge factor is used which assumes the waveform is a sine > wave (which it isn't). > > ... Dave > > ps. what you really need for an accurate measurement of RMS power and power > factor is something like: > http://ca.fluke.com/caen/products/features.htm?cs_id=5179(FlukeProducts)&cat egory=PHASE1(FlukeProducts) > which unfortunely is about $1900US - ouch! There are likely other > alternatives that are cheaper. > > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org > To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From dajelen at att.net Wed Jun 4 17:06:46 2003 From: dajelen at att.net (dajelen at att.net) Date: Wed, 04 Jun 2003 21:06:46 +0000 Subject: How to get started . . . Message-ID: <200306042100.h54L0K515113@NewBlue.Scyld.com> Hi all, I've been lurking on this list for a while and have finally gotten up the nerve to ask a question. How can I get started in the Beowulf arena? Ideally, I'd like to become active on a project somewhere and learn as I go. Does anyone have ideas on how this could be accomplished? Dave _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From shewa at inel.gov Wed Jun 4 18:06:51 2003 From: shewa at inel.gov (Andrew Shewmaker) Date: Wed, 04 Jun 2003 16:06:51 -0600 Subject: measuring power usage In-Reply-To: References: Message-ID: <3EDE6D7B.9030107@inel.gov> Trent Piepho wrote: >Thanks to all that responded. Cute pun for a name, but it completely thwarted >my google search for terms like "power factor" and "meter". > >BTW, is there a real searchable list archive anywhere? google groups has >lists.beowulf but it looks like they stopped archiving that in 2000. The >archives on beowulf.org aren't searchable, unless you try to use a google web >search to find pages that were indexed. > > Never mind my last email. I just remembered this one. http://marc.theaimsgroup.com/?l=beowulf&r=1&w=2 -Andrew _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From shewa at inel.gov Wed Jun 4 18:03:47 2003 From: shewa at inel.gov (Andrew Shewmaker) Date: Wed, 04 Jun 2003 16:03:47 -0600 Subject: measuring power usage In-Reply-To: References: Message-ID: <3EDE6CC3.3020608@inel.gov> Trent Piepho wrote: >Thanks to all that responded. Cute pun for a name, but it completely thwarted >my google search for terms like "power factor" and "meter". > >BTW, is there a real searchable list archive anywhere? google groups has >lists.beowulf but it looks like they stopped archiving that in 2000. The >archives on beowulf.org aren't searchable, unless you try to use a google web >search to find pages that were indexed. > I point google to geocrawler sometimes. Type "site:geocrawler.com beowulf" plus your query. I'm not sure how often they update their index of geocrawler.com though Andrew _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From Andrew.Cannon at nnc.co.uk Thu Jun 5 03:25:45 2003 From: Andrew.Cannon at nnc.co.uk (Cannon, Andrew) Date: Thu, 5 Jun 2003 08:25:45 +0100 Subject: How to get started . . . Message-ID: What sort of resources have you got? If you have a number of computers, I would recommend that you try networking them together and finding a task that they can all work on. Try using MPICH or PVM for a simple introduction to clustering. You will probably get lots of advice, some of it conflicting, some good and some bad. Search on Google etc too. The resources are out there. Have fun. Andy -----Original Message----- From: dajelen at att.net [mailto:dajelen at att.net] Sent: Wednesday, June 04, 2003 10:07 PM To: beowulf at beowulf.org Subject: How to get started . . . Hi all, I've been lurking on this list for a while and have finally gotten up the nerve to ask a question. How can I get started in the Beowulf arena? Ideally, I'd like to become active on a project somewhere and learn as I go. Does anyone have ideas on how this could be accomplished? Dave _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf *********************************************************************************** NNC Limited Booths Hall Chelford Road Knutsford Cheshire WA16 8QZ Country of Registration: United Kingdom Registered Number: 1120437 This e-mail and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this e-mail in error please notify the NNC system manager by e-mail at eadm at nnc.co.uk. *********************************************************************************** _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From mat.harris at genestate.com Thu Jun 5 05:38:20 2003 From: mat.harris at genestate.com (Mat Harris) Date: Thu, 5 Jun 2003 10:38:20 +0100 Subject: How to get started . . . In-Reply-To: References: Message-ID: <20030605093819.GA8024@genestate.com> hi, I am in the same boat. what is a good idea for a simple trial task? I am a 'beginner' c programmer so nothing to complex. I don't really know what i can do to test the small collection of machines I have. Will I have to program a certain way to get the 'cluster' effect? cheers On Thu, Jun 05, 2003 at 08:25:45 +0100, Cannon, Andrew wrote: > What sort of resources have you got? If you have a number of computers, I > would recommend that you try networking them together and finding a task > that they can all work on. Try using MPICH or PVM for a simple introduction > to clustering. You will probably get lots of advice, some of it conflicting, > some good and some bad. Search on Google etc too. The resources are out > there. > > Have fun. > > Andy > > -----Original Message----- > From: dajelen at att.net [mailto:dajelen at att.net] > Sent: Wednesday, June 04, 2003 10:07 PM > To: beowulf at beowulf.org > Subject: How to get started . . . > > > Hi all, > > I've been lurking on this list for a while and have finally gotten up > the nerve to ask a question. > > How can I get started in the Beowulf arena? > > Ideally, I'd like to become active on a project somewhere and learn > as I go. > > Does anyone have ideas on how this could be accomplished? > > Dave > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org > To change your subscription (digest mode or unsubscribe) visit > http://www.beowulf.org/mailman/listinfo/beowulf > > > *********************************************************************************** > NNC Limited > Booths Hall > Chelford Road > Knutsford > Cheshire > WA16 8QZ > > Country of Registration: United Kingdom > Registered Number: 1120437 > > This e-mail and any files transmitted with it are confidential and > intended solely for the use of the individual or entity to whom they > are addressed. If you have received this e-mail in error please notify > the NNC system manager by e-mail at eadm at nnc.co.uk. > *********************************************************************************** > > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org > To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- -- Mat Harris ( ) ASCII Ribbon Campaign mat.harris at genestate.com X against HTML on email / \ and Usenet posts * .* **. .**** .' *****. ** '. '*****. .*** **.. '****** .**** .***. '******' ****** .****. ****** ******* .****. '******' .*******. .***** ******' .*******. ******. .*****' .*******. ******. .*****' .******. .****** *****. .****** ..' *****. '**** .******. .****. .. .***** ***' .****. ..******' '***... '****. '** **** ..*******'' '*******.. **** ** ***. ..*******'' '*********. *** '* .** ..*******' '********.. ** * *' .*****'' ''*******. *** ***' ''******.. .***' ****. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 232 bytes Desc: not available URL: From dajelen at att.net Thu Jun 5 10:20:48 2003 From: dajelen at att.net (dajelen at att.net) Date: Thu, 05 Jun 2003 14:20:48 +0000 Subject: How to get started . . . Message-ID: <200306051414.h55EED514584@NewBlue.Scyld.com> Unfortunately (or fortunately), my experience is in programming, rather than physics or epidemiology or another area that had domain-specific knowledge. I've heard of some "hard" problems that would require the use of a Beowulf- type cluster, but don't know enough of the specifics to take it to programming the cluster at this point (optimization isn't even close to the table at this point.) I mentioned becoming active on a project with the idea that I could join something in process and learn how the tasks are identified and how they are solved - and what methods work best to solve problem X. How do I find a project and get to the point where I can begin to consider the programming questions? > > How can I get started in the Beowulf arena? > > find a project. something you want to compute. > > > Ideally, I'd like to become active on a project somewhere and learn > > as I go. > > cooperative projects really only work if participants > have a concrete goal in mind ("fix feature x"). > _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From dajelen at att.net Thu Jun 5 10:30:49 2003 From: dajelen at att.net (dajelen at att.net) Date: Thu, 05 Jun 2003 14:30:49 +0000 Subject: How to get started . . . Message-ID: <200306051424.h55EOG516251@NewBlue.Scyld.com> Hi Heidi, Thanks for the input, and I will check out your web page. I might have mis-stated my objective. Yes, I do have two old PCs that I can use to build a Beowulf cluster. (My resources are limited, so the pre-built cluster is not an option at the moment.) I understand the principles behind a Beowulf cluster and message-passing, problem partitioning, etc. My question was an attempt to determine/identify a project to join that could utilize the cluster (2-nodes) so that I could understand the programming better and learn more about the behavior of the cluster itself and how to optimize the cluster. I thought that joining a project that's already in process would give me a head-start - someone else would have already planned out the problem to be solved, determined the best way (or, at least, A way) to solve it, and was ready (or has been) implementing the solution. My background is programming, rather than a specific domain, so I feel that I can assist with any project that is out there. Which problem domains might be the most interesting? Are there projects that can use another programming hand? If I need to, how can I find a project and begin the process on my own? I do appreciate any and all direction you can provide, Dave > Hi Dave: > Do you have 2 old machines you could try and hook together? That's all > you need to get started.. I very detailed description of how to setup > is on my > Webpage.. > http://chaos.fullerton.edu/~heidi/heidi.html > > you can buy a ready made cluster online! An 8 node cluster will set you > back about $10k (depends on the components though.. that is a very good > one!) > heidi > > -----Original Message----- > From: dajelen at att.net [mailto:dajelen at att.net] > Sent: Wednesday, June 04, 2003 2:07 PM > To: beowulf at beowulf.org > Subject: How to get started . . . > > Hi all, > > I've been lurking on this list for a while and have finally gotten up > the nerve to ask a question. > > How can I get started in the Beowulf arena? > > Ideally, I'd like to become active on a project somewhere and learn > as I go. > > Does anyone have ideas on how this could be accomplished? > > Dave > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org > To change your subscription (digest mode or unsubscribe) visit > http://www.beowulf.org/mailman/listinfo/beowulf _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From beerli at csit.fsu.edu Wed Jun 4 23:53:29 2003 From: beerli at csit.fsu.edu (Peter Beerli) Date: Wed, 4 Jun 2003 23:53:29 -0400 (EDT) Subject: queuing systems and cpu usage? Message-ID: Hi all, I am a novice to all clustering and queueing systems. So far, my cluster needs were satisfied on ad hoc cluster to run embarrassingly parallel code (MCMC runs). After my move, I can use time on a IBM Regatta cluster (and I am impatiently waiting for my very own Linux Athlon cluster). The Regatta cluster is running loadleveler which seems to have an abysmal job-scheduling performance (it seems that currently out of the max=480 cpus only 288 are used [4 idle nodes out of 15] and many jobs (including mine) are in the queues, waiting). I would be glad to hear information what schedulers you prefer and (preferably) also get some numbers of how many nodes and cpus are idle under standard load** or other appropriate statistics (what is the appropriate statistics?). (this is not really a question for this list but some might be able to answer: is out regatta loadleveler misconfigured?) **standard load: too many users submit jobs for the queue with the longest time limit, very few medium, small length jobs, most of the jobs are obviously only using <32 cpus (a node on the regatta has 32 cpus) thanks, Peter ---- Peter Beerli, School of Computational Science and Information Technology (CSIT) Dirac Science Library, Florida State University Tallahassee, Florida 32306-4120 USA old webpage: http://evolution.genetics.washington.edu/PBhtmls/beerli.html _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From johnb at quadrics.com Thu Jun 5 12:18:03 2003 From: johnb at quadrics.com (John Brookes) Date: Thu, 5 Jun 2003 17:18:03 +0100 Subject: queuing systems and cpu usage? (Partly OT) Message-ID: <010C86D15E4D1247B9A5DD312B7F5AA7E5E1AF@stegosaurus.bristol.quadrics.com> Peter, I came across LoadLeveler on an SP3 in a former job and also found the scheduling to be pretty poor. To be fair to IBM, they readily admitted the fact. I quickly found the Maui scheduler from the Maui Supercomputing Centre (now defunct? its old url no longer works). At the time it was licensed (though free), but they were working on the legal issues to make it freely distributable. It's now a SourceForge project (so I assume they succeeded with the legal :) I'm not sure if/where a fully-supported version that plugs into LL can be found these days (or even whether the project retains the LL interface), but it did much better than either of the built-ins at that time and works well now (on other systems, at least). It's also highly configurable, so you can make it as nice (or nasty!) as you like. YM will almost certainly V, as my only experiences under IBM are from ~3yo versions, but if you can get it to work it'd probably be a Good Thing. Maui's often used as the scheduler for PBS and Sun GridEngine nowadays, so getting to know its foibles wouldn't be wasted once your Linux/Athlon cluster arrives. The project is at: http://mauischeduler.sourceforge.net/ Some information on Maui and the Maui/LL tie-in can be found at eg: http://supercluster.org/documentation/ Cheers, John Brookes Quadrics Ltd. > -----Original Message----- > From: Peter Beerli [mailto:beerli at csit.fsu.edu] > Sent: 05 June 2003 04:53 > To: Beowulf Mailing list > Subject: queuing systems and cpu usage? > > > Hi all, > I am a novice to all clustering and queueing systems. > So far, my cluster needs were satisfied on ad hoc cluster to run > embarrassingly parallel code (MCMC runs). After my move, I can use > time on a IBM Regatta cluster (and I am impatiently waiting > for my very > own Linux Athlon cluster). The Regatta cluster is running > loadleveler which seems to > have an abysmal job-scheduling performance (it seems that > currently out of the max=480 cpus > only 288 are used [4 idle nodes out of 15] and many jobs > (including mine) are in the queues, waiting). > > I would be glad to hear information what schedulers you > prefer and (preferably) > also get some numbers of how many nodes and cpus are idle > under standard load** > or other appropriate statistics (what is the appropriate statistics?). > > (this is not really a question for this list but some might > be able to answer: > is out regatta loadleveler misconfigured?) > > **standard load: too many users submit jobs for the queue > with the longest time limit, > very few medium, small length jobs, most of the jobs are > obviously only using <32 cpus > (a node on the regatta has 32 cpus) > > thanks, > Peter > ---- > Peter Beerli, > School of Computational Science and Information Technology (CSIT) > Dirac Science Library, Florida State University > Tallahassee, Florida 32306-4120 USA > old webpage: http://evolution.genetics.washington.edu/PBhtmls/beerli.html _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From sfrolov at accufo.vwh.net Thu Jun 5 10:20:47 2003 From: sfrolov at accufo.vwh.net (sfrolov at accufo.vwh.net) Date: Thu, 5 Jun 2003 08:20:47 -0600 (MDT) Subject: dual Xeon issue Message-ID: <1054822842.97498@accufo.vwh.net> Sorry for a noob-like question. We are in the process of buying a Xeon-based cluster for scientific computing (running MPI parallelized numerical models). We've benchmarked clasters offered by Microway and Aspen Systems and discovered that we are getting much better results using only one CPU per box (running the model on 6 boxes using one CPU in each box was about 50% faster than running it on 3 boxes using both CPUs). This was surprising to us since we've used to be limited by interprocessor communications, which should be a lot faster between CPUs in one box. Can anybody explain the reason for this and, more importantly, is there any way to improve this situation. Thanks, Sergei. P.S. Can anybody recommend the best performance/price ratio cluster? _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From shewa at inel.gov Thu Jun 5 12:45:54 2003 From: shewa at inel.gov (Andrew Shewmaker) Date: Thu, 05 Jun 2003 10:45:54 -0600 Subject: How to get started . . . In-Reply-To: <20030605093819.GA8024@genestate.com> References: <20030605093819.GA8024@genestate.com> Message-ID: <3EDF73C2.4080507@inel.gov> Mat Harris wrote: > hi, I am in the same boat. what is a good idea for a simple trial task? > > I am a 'beginner' c programmer so nothing to complex. I don't really know what i > can do to test the small collection of machines I have. Why, "hello world", of course =-) The Ohio Super Computer Center [1] has some good online courses and you should read Robert G. Brown's book on his site [2]. [1] http://oscinfo.osc.edu/training/ [2] http://www.phy.duke.edu/brahma/index.php > Will I have to program a certain way to get the 'cluster' effect? First, make sure you read as much as you can of the available material on the web. It sounds to me like you are still trying to understand what a beowulf cluster is. You are probably not distinguishing between high performance clusters running parallelized codes and High Availability (HA) clusters, which are concerned with failover and load balancing. The beowulf FAQ explains things like this (see www.beowulf.org) and www.lcic.org has links to information about all types of clusters. Beowulf clusters must be programmed in a certain way. They don't automatically make an application run faster. However, there are projects that are attempting to make a cluster of computers look more like one Single System Image. OpenMosix (openmosix.sf.net) and SSI for Linux (ssic-linux.sf.net) are two projects that are working on providing a single process space, distributed shared memory, and more. Both of these are more on the HA end of things, but you can still run parallel codes on them. Some of their features might increase the overhead and lower the performance of parallel applications. These systems may also have difficulty scaling up to several hundreds or thousands of nodes. Bproc (bproc.sf.net) was designed to create a second generation of beowulf clusters although first generation clusters, using rsh or ssh to start processes remotely, are still more common. Bproc provides a single process space for the cluster and scales up to at least 1024 nodes...Pink (www.lanl.gov/projects/pink/) is the largest one I know of. Only the master node has a full Linux installation while the slave nodes are totally dependent on the master. Scyld (www.scyld.com) and Clustermatic (www.clustermatic.org) can help you set up a bproc based cluster. -Andrew -- Andrew Shewmaker, Associate Engineer Phone: 1-208-526-1276 Idaho National Eng. and Environmental Lab. P.0. Box 1625, M.S. 3605 Idaho Falls, Idaho 83415-3605 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From timm at fnal.gov Thu Jun 5 14:45:11 2003 From: timm at fnal.gov (Steven Timm) Date: Thu, 5 Jun 2003 13:45:11 -0500 (CDT) Subject: dual Xeon issue In-Reply-To: <1054822842.97498@accufo.vwh.net> Message-ID: I'm not sure if I can explain the problem or not, but a crucial piece of information that anyone would need to know to explain it is--do you have the hyper-threading enabled or not? Up until now we have found that Linux tends to allocate two processes to the extra logical cpu's supplied with hyperthreading without much regard to whether they are different physical cpu's or not. This can slow performance down.. doubly so if you are dependent on memory bandwidth. Steve ------------------------------------------------------------------ Steven C. Timm (630) 840-8525 timm at fnal.gov http://home.fnal.gov/~timm/ Fermilab Computing Division/Core Support Services Dept. Assistant Group Leader, Scientific Computing Support Group Lead of Computing Farms Team On Thu, 5 Jun 2003 sfrolov at accufo.vwh.net wrote: > Sorry for a noob-like question. We are in the process of buying a Xeon-based cluster for scientific computing (running MPI parallelized numerical models). We've benchmarked clasters offered by Microway and Aspen Systems and discovered that we are getting much better results using only one CPU per box (running the model on 6 boxes using one CPU in each box was about 50% faster than running it on 3 boxes using both CPUs). This was surprising to us since we've used to be limited by interprocessor communications, which should be a lot faster between CPUs in one box. Can anybody explain the reason for this and, more importantly, is there any way to improve this situation. > > Thanks, > > Sergei. > > P.S. Can anybody recommend the best performance/price ratio cluster? > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org > To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf > _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From siegert at sfu.ca Thu Jun 5 15:17:03 2003 From: siegert at sfu.ca (Martin Siegert) Date: Thu, 5 Jun 2003 12:17:03 -0700 Subject: dual Xeon issue In-Reply-To: <1054822842.97498@accufo.vwh.net> References: <1054822842.97498@accufo.vwh.net> Message-ID: <20030605191703.GB6918@stikine.ucs.sfu.ca> On Thu, Jun 05, 2003 at 08:20:47AM -0600, sfrolov at accufo.vwh.net wrote: > Sorry for a noob-like question. We are in the process of buying a Xeon-based > cluster for scientific computing (running MPI parallelized numerical models). > We've benchmarked clasters offered by Microway and Aspen Systems and discovered > that we are getting much better results using only one CPU per box (running the > model on 6 boxes using one CPU in each box was about 50% faster than running it > on 3 boxes using both CPUs). This was surprising to us since we've used to be > limited by interprocessor communications, which should be a lot faster between > CPUs in one box. Can anybody explain the reason for this and, more importantly, > is there any way to improve this situation. Since you mentioned that you are limited by interprocessor communications the reason for the slowdown in the dual-processor configuration is probably that the communication between processors on different nodes becomes the bottleneck. Since the link between nodes is now shared between two processors on each node you effectively halve the per processor bandwidth. Remedy: either buy single processor boxes or double the bandwidth (cf channel bonding). -- Martin Siegert Manager, Research Services Academic Computing Services phone: (604) 291-4691 Simon Fraser University fax: (604) 291-4242 Burnaby, British Columbia email: siegert at sfu.ca Canada V5A 1S6 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From orion at cora.nwra.com Thu Jun 5 16:03:58 2003 From: orion at cora.nwra.com (Orion Poplawski) Date: Thu, 05 Jun 2003 14:03:58 -0600 Subject: HyperThreading in Cluster Message-ID: <3EDFA22E.8090902@cora.nwra.com> This is partly prompted by Steven Timm's recent comment about HyperThreading, as well as something I read in a magazine the other day stating the HPC clusters need to run with HyperThreading off. Can anyone shed more light on the current status of HyperThreading in parallel computing? I could see where allocating two compute threads to a single processor might bog things down (swapping data sets in/out of cache, etc.), but it would seem ideal to allocate one compute thread per processor, and let the other virtual processor help with OS/interrupt type operations. Does the linux scheduler have such control yet? -- Orion Poplawski System Administrator 303-415-9701 x222 Colorado Research Associates/NWRA FAX: 303-415-9702 3380 Mitchell Lane, Boulder CO 80301 www.co-ra.com _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From math at velocet.ca Thu Jun 5 15:06:32 2003 From: math at velocet.ca (Ken Chase) Date: Thu, 5 Jun 2003 15:06:32 -0400 Subject: dual Xeon issue In-Reply-To: <1054822842.97498@accufo.vwh.net>; from sfrolov@accufo.vwh.net on Thu, Jun 05, 2003 at 08:20:47AM -0600 References: <1054822842.97498@accufo.vwh.net> Message-ID: <20030605150632.J61960@velocet.ca> On Thu, Jun 05, 2003 at 08:20:47AM -0600, sfrolov at accufo.vwh.net's >P.S. Can anybody recommend the best performance/price ratio cluster? "DEPENDS" like always. Depends on what you're doing. Depends on if your code scales well, or superlinearly, or not at all. Depends on the cost of power, floor space, cooling, gear, gear being shipped to you, humans to assemble, install, test, and maintain it. Depends on whats free and what's not. (see my post on .edu vs .com clusters earlier this week). Even more generally, depends on what you mean by 'performance'. Does it have to sing and dance well for morons who like blinkenlights? Does it have to run one single job so freakin' fast that the organization can publish results as bragging rights, but makes it ass-slow running 30 jobs at the same time by researchers fighting over resources? Or did you mean 'best throughput in jobs per month'? What did you mean? 'depends'. /kc -- Ken Chase, math at velocet.ca * Velocet Communications Inc. * Toronto, Canada Wiznet Velocet DSL.ca Datavaults 24/7: 416-967-4414 tollfree: 1-866-353-0363 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From ctierney at hpti.com Thu Jun 5 15:35:16 2003 From: ctierney at hpti.com (Craig Tierney) Date: 05 Jun 2003 13:35:16 -0600 Subject: dual Xeon issue In-Reply-To: <1054822842.97498@accufo.vwh.net> References: <1054822842.97498@accufo.vwh.net> Message-ID: <1054841715.11421.136.camel@woody> On Thu, 2003-06-05 at 08:20, sfrolov at accufo.vwh.net wrote: > Sorry for a noob-like question. We are in the process of buying a Xeon-based cluster for scientific computing (running MPI parallelized numerical models). We've benchmarked clasters offered by Microway and Aspen Systems and discovered that we are getting much better results using only one CPU per box (running the model on 6 boxes using one CPU in each box was about 50% faster than running it on 3 boxes using both CPUs). This was surprising to us since we've used to be limited by interprocessor communications, which should be a lot faster between CPUs in one box. Can anybody explain the reason for this and, more importantly, is there any way to improve this situation. > You are probably pushing the memory bandwidth on the motherboard. On most Xeon motherboards, the memory bandwidth can be exhausted by 1 cpu. Rarely do you get a 2x speedup when using the 2nd cpu. Your results are typical. You could go with single CPU motherboards. The new 800Mhz FSB looks quite attractive. However, the odds are that when you include the extra interconnect and the other infrastructure that the dual Xeon system will provide better price/performance. It does all depend on your code. I doubt that a Microway or an Aspen Systems cluster is going to perform that differently if configured similarly. Make sure that Hyperthreading is off and you are using a good compiler (Intel). Look at what support you get for your money. Things are going to break, and knowing how the vendor is going fix your problems is important. Craig > Thanks, > > Sergei. > > P.S. Can anybody recommend the best performance/price ratio cluster? > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org > To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- Craig Tierney _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From glen at cert.ucr.edu Thu Jun 5 15:39:18 2003 From: glen at cert.ucr.edu (Glen Kaukola) Date: Thu, 05 Jun 2003 12:39:18 -0700 Subject: ups units Message-ID: <3EDF9C66.5090106@cert.ucr.edu> Hi everyone, I've got a question about UPS units. If they're rated at 1500VA, then how many computers with 400 watt power supplies should they be able to handle? And how many computers with 550 watt power supplies? If someone could let me know how you calculate the answer I'd really appricate it. The reason I ask is because we have a number of Cyberpower and APC UPS units rated at 1500VA. They each have 2 computers plugged into them, and seemed to be working fine, as me and my boss tested them by unplugging each while the computers plugged into them were running. But recently we had a power spike or whatever you'd call it, that lasted less than a second, and afterwards I noticed that many of the computers plugged into the Cyberpower units had rebooted. I contacted the company we purchased the Cyberpower units from, and he in turn contacted Cyberpower. Cyberpower informed him that there shouldn't be more than one computer plugged into each UPS unit, and said something which I didn't understand about 12 amps, 125 volts, and 400 X 2 watts being more than 1500AV. It all sounded rather fishy to me, so I was hoping that somebody on this list could clear things up for me. Thanks in advance, Glen _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From lindahl at keyresearch.com Thu Jun 5 16:13:37 2003 From: lindahl at keyresearch.com (Greg Lindahl) Date: Thu, 5 Jun 2003 13:13:37 -0700 Subject: dual Xeon issue In-Reply-To: <1054822842.97498@accufo.vwh.net> References: <1054822842.97498@accufo.vwh.net> Message-ID: <20030605201337.GB1513@greglaptop.internal.keyresearch.com> On Thu, Jun 05, 2003 at 08:20:47AM -0600, sfrolov at accufo.vwh.net wrote: > This was surprising to us since we've used to be limited by > interprocessor communications, which should be a lot faster between > CPUs in one box. How much memory bandwidth does your code want? Xeon systems usually have a memory bus which can be maxed out by a single cpu. As a by-the-way, Opteron systems, if you get the memory affinity going correctly, have memory bandwidth that scales as you add more processors. Also, your request for the "best performance/price ratio cluster" doesn't have an answer -- different people are going to see different performance depending on what their application is. That's doubly true for unusual apps like yours. -- greg _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From josip at lanl.gov Thu Jun 5 16:12:32 2003 From: josip at lanl.gov (Josip Loncaric) Date: Thu, 05 Jun 2003 14:12:32 -0600 Subject: dual Xeon issue In-Reply-To: <1054822842.97498@accufo.vwh.net> References: <1054822842.97498@accufo.vwh.net> Message-ID: <3EDFA430.9060102@lanl.gov> sfrolov at accufo.vwh.net wrote: > [...] We've benchmarked clasters offered by Microway and > Aspen Systems and discovered that we are getting much better results > using only one CPU per box (running the model on 6 boxes using one > CPU in each box was about 50% faster than running it on 3 boxes using > both CPUs). Your results are only slightly worse than expected. This is a generic situation with memory bandwidth limited codes, where a dual CPU box typically counts as only 1.5 single CPU boxes (approximately) due to shared resources (memory bandwidth, network card, certain OS functions). However, for a code which spins mostly in cache and whose processes to not communicate much, a dual CPU box can match two singles. Shared memory communication is much faster than network, and it can be a big win if used between tightly coupled processes within a dual CPU box, but there is a caveat. Polling two different MPI devices (shmem and net) can impose its own penalty. Moreover, since most problems do not partition neatly into loosely coupled pairs of tightly coupled processes, exploiting the higher bandwidth of shared memory communication is tricky. For all these reasons, some people prefer singles. Others like denser packaging so they buy duals. When a high end network is involved (Myriniet, Quadrics), duals are clearly more cost effective since they require fewer network ports for the same number of CPUs. Sincerely, Josip _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From vijayv at CSE.MsState.EDU Thu Jun 5 19:15:15 2003 From: vijayv at CSE.MsState.EDU (Vijay Velusamy) Date: Thu, 5 Jun 2003 18:15:15 -0500 (CDT) Subject: dual Xeon issue In-Reply-To: <1054822842.97498@accufo.vwh.net> Message-ID: Which MPI implementation are you using for the benchmarking. This would actually depend on the implementation specifics. The implementation needs to need to be aware of and take advantage of SMP's. Also are you using a SMP kernel for this? -Vijay On Thu, 5 Jun 2003 sfrolov at accufo.vwh.net wrote: > Sorry for a noob-like question. We are in the process of buying a Xeon-based cluster for scientific computing (running MPI parallelized numerical models). We've benchmarked clasters offered by Microway and Aspen Systems and discovered that we are getting much better results using only one CPU per box (running the model on 6 boxes using one CPU in each box was about 50% faster than running it on 3 boxes using both CPUs). This was surprising to us since we've used to be limited by interprocessor communications, which should be a lot faster between CPUs in one box. Can anybody explain the reason for this and, more importantly, is there any way to improve this situation. > > Thanks, > > Sergei. > > P.S. Can anybody recommend the best performance/price ratio cluster? > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org > To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf > _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From nuclearnine at prodigy.net Thu Jun 5 22:32:47 2003 From: nuclearnine at prodigy.net (Nuclear Nine) Date: Thu, 5 Jun 2003 22:32:47 -0400 Subject: How to get started . . . References: <200306051424.h55EOG516251@NewBlue.Scyld.com> Message-ID: <003e01c32bd3$ebbfdf60$88574b43@Homer> I am a newbie as well. What does everyone think about the Rocks cluster distribution? According to their website it seems like the easiest solution. Any comments. http://www.rocksclusters.org/Rocks/ ----- Original Message ----- From: To: "Fearn, Heidi" Cc: Sent: Thursday, June 05, 2003 10:30 AM Subject: RE: How to get started . . . Hi Heidi, Thanks for the input, and I will check out your web page. I might have mis-stated my objective. Yes, I do have two old PCs that I can use to build a Beowulf cluster. (My resources are limited, so the pre-built cluster is not an option at the moment.) I understand the principles behind a Beowulf cluster and message-passing, problem partitioning, etc. My question was an attempt to determine/identify a project to join that could utilize the cluster (2-nodes) so that I could understand the programming better and learn more about the behavior of the cluster itself and how to optimize the cluster. I thought that joining a project that's already in process would give me a head-start - someone else would have already planned out the problem to be solved, determined the best way (or, at least, A way) to solve it, and was ready (or has been) implementing the solution. My background is programming, rather than a specific domain, so I feel that I can assist with any project that is out there. Which problem domains might be the most interesting? Are there projects that can use another programming hand? If I need to, how can I find a project and begin the process on my own? I do appreciate any and all direction you can provide, Dave > Hi Dave: > Do you have 2 old machines you could try and hook together? That's all > you need to get started.. I very detailed description of how to setup > is on my > Webpage.. > http://chaos.fullerton.edu/~heidi/heidi.html > > you can buy a ready made cluster online! An 8 node cluster will set you > back about $10k (depends on the components though.. that is a very good > one!) > heidi > > -----Original Message----- > From: dajelen at att.net [mailto:dajelen at att.net] > Sent: Wednesday, June 04, 2003 2:07 PM > To: beowulf at beowulf.org > Subject: How to get started . . . > > Hi all, > > I've been lurking on this list for a while and have finally gotten up > the nerve to ask a question. > > How can I get started in the Beowulf arena? > > Ideally, I'd like to become active on a project somewhere and learn > as I go. > > Does anyone have ideas on how this could be accomplished? > > Dave > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org > To change your subscription (digest mode or unsubscribe) visit > http://www.beowulf.org/mailman/listinfo/beowulf _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From andrewxwang at yahoo.com.tw Thu Jun 5 23:18:16 2003 From: andrewxwang at yahoo.com.tw (=?big5?q?Andrew=20Wang?=) Date: Fri, 6 Jun 2003 11:18:16 +0800 (CST) Subject: queuing systems and cpu usage? In-Reply-To: Message-ID: <20030606031816.7347.qmail@web16804.mail.tpe.yahoo.com> If there are a lot of people submit jobs to Loadleveler, while Loadlevler is scheduling, then the performance is not good. To reduce the impact of job submition, make sure the spool directory is on a local file system. (a gridengine cluster was having problem because the spool directory was on NFS) -- I am assume Loadlevler is similar to gridengine -- spool directory access performance is important. Otherwise, it must be the scheduler performance, then use Maui. Lastly, is Loadlevler free? Thanks, Andrew. --- Peter Beerli ???? > Hi all, > I am a novice to all clustering and queueing > systems. > So far, my cluster needs were satisfied on ad hoc > cluster to run > embarrassingly parallel code (MCMC runs). After my > move, I can use > time on a IBM Regatta cluster (and I am impatiently > waiting for my very > own Linux Athlon cluster). The Regatta cluster is > running loadleveler which seems to > have an abysmal job-scheduling performance (it seems > that currently out of the max=480 cpus > only 288 are used [4 idle nodes out of 15] and many > jobs (including mine) are in the queues, waiting). > > I would be glad to hear information what schedulers > you prefer and (preferably) > also get some numbers of how many nodes and cpus are > idle under standard load** > or other appropriate statistics (what is the > appropriate statistics?). > > (this is not really a question for this list but > some might be able to answer: > is out regatta loadleveler misconfigured?) > > **standard load: too many users submit jobs for the > queue with the longest time limit, > very few medium, small length jobs, most of the > jobs are obviously only using <32 cpus > (a node on the regatta has 32 cpus) > > thanks, > Peter > ---- > Peter Beerli, > School of Computational Science and Information > Technology (CSIT) > Dirac Science Library, Florida State University > Tallahassee, Florida 32306-4120 USA > old webpage: > http://evolution.genetics.washington.edu/PBhtmls/beerli.html > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org > To change your subscription (digest mode or > unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf ----------------------------------------------------------------- ??? Yahoo!?? ??????? - ???????????? http://fate.yahoo.com.tw/ _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From smuelas at mecanica.upm.es Fri Jun 6 03:15:32 2003 From: smuelas at mecanica.upm.es (Santiago Muelas) Date: Fri, 6 Jun 2003 09:15:32 +0200 Subject: dual Xeon issue In-Reply-To: <1054822842.97498@accufo.vwh.net> References: <1054822842.97498@accufo.vwh.net> Message-ID: <20030606091532.6e0201fb.smuelas@mecanica.upm.es> I am also quite a "newby" to this beowulf-world. A couple of months ago I asked a question in this list because my program did not attained my expectations. I was convinced of an ethernet problem and asked for giga-ethernet... I receive many very nice answers, but one specially was the very right one. It came from "master" Brown. He told me to fine-tune my program before doing any other thing... and he was right. Now, I am learning with four Ahtlon XP2200 computers and sometime I tested them against a dual 2-Xeon-2200 Dell , expensive one, plus, sometime another dual with 2 Ahtlon 1800. So, as finally the results are --for me-- impressive and the size of my little "toy" could be similar to yours, and despite the fact that I am the perfect oppposite to an expert in clusters, here are my recomendations: - Do not expend money in "better" things. In this aspect, quantity is better than "theoretical-quality" - Normal ethernet is great and more than enough for 4,5 or 6 machines.(and an standard switch) - Choose the best price-quality possibility. - Use "xmpi" and work on your program until the "red" has almost disappeared. And here are my results, tested with a finite element program written by my friend Lee Margetts abd fine-tuned by him: - Time to run an iterative, 100.000 equations program on one Ahtlon: 58 sec. On 2 machines 37 sec.(I have limited the iterations to the minimum) - The same on one CPU with the Xeon: 62 sec. On 2 CPU 40 sec. The Xeon has 1Gb. memory for each processor. The Ahtlons 512Kb each. Now a bigger problem: 330.000 equations. In this case and due to memory requirements I was not able to run the program on just 2 Ahtlons, so I run it on the four. With the same total memory as the Dell-Xeon, I tested on this one with the two processors. Clearly, the results are not comparable but give an interesting result: - Time to run on the 2-Xeon: 2m56sc. - Time to run on the 4 Ahtlons: 1m24sc. So less than one half. Now, the cost: Each Athlon has 1 processor, 512Mb, one motherboard (standard with ethernet included) and one HD (chipest, but new). Total price in euros: 350-400. Cost of 4 Un. --> 1400 euros. Dell dual-Xeon with 2Gb. total ram, scsi disk and.... 4500-5000 euros. So the decission is simple. I will increase imediately the ram on the Ahtlons to attain 4Gb. (cost of 2 Gb= 200 euros. I use the standard cheapest OEM memory cards. Excellent, BTW) All that means that I can have a nice beowulf with 12 computers based on Ahtlons (now Ahtlon XP 2400 has the best quality/price), for the same cost as a prestigious dual computer. I think this is the reason of beowulf philosophy !! Thanks to all for your patience. On Thu, 5 Jun 2003 08:20:47 -0600 (MDT) sfrolov at accufo.vwh.net wrote: > Sorry for a noob-like question. We are in the process of buying a Xeon-based cluster for scientific computing (running MPI parallelized numerical models). We've benchmarked clasters offered by Microway and Aspen Systems and discovered that we are getting much better results using only one CPU per box (running the model on 6 boxes using one CPU in each box was about 50% faster than running it on 3 boxes using both CPUs). This was surprising to us since we've used to be limited by interprocessor communications, which should be a lot faster between CPUs in one box. Can anybody explain the reason for this and, more importantly, is there any way to improve this situation. > > Thanks, > > Sergei. > > P.S. Can anybody recommend the best performance/price ratio cluster? > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org > To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- Santiago Muelas Profesor de Resistencia de Materiales y C?lculo de Estructuras ETSI de Caminos, Canales y Puertos (U.P.M) smuelas at mecanica.upm.es http://w3.mecanica.upm.es/~smuelas _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From jhearns at freesolutions.net Fri Jun 6 05:23:38 2003 From: jhearns at freesolutions.net (John Hearns) Date: Fri, 6 Jun 2003 10:23:38 +0100 Subject: PS2 cluster Message-ID: <004d01c32c0d$6b442190$8461cdc2@DREAD> I remember when the PS2 first came out, ideas were kicked around on the Beowulf list about clustering them and using the vector units in them (and using Firewire networking) Looks like NCS are actually doing this, using the PS2 Linux kit and Ethernet: http://access.ncsa.uiuc.edu/Releases/05.27.03_Playing_th.html Wow! _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From Thomas.Alrutz at dlr.de Fri Jun 6 07:28:24 2003 From: Thomas.Alrutz at dlr.de (Thomas Alrutz) Date: Fri, 06 Jun 2003 13:28:24 +0200 Subject: dual Xeon issue References: <1054822842.97498@accufo.vwh.net> Message-ID: <3EE07AD8.70206@dlr.de> > Sorry for a noob-like question. We are in the process of buying a Xeon-based cluster > for scientific computing (running MPI parallelized numerical models). > We've benchmarked clasters offered by Microway and Aspen Systems and discovered that we > are getting much better results using only one CPU per box (running the model on 6 boxes > using one CPU in each box was about 50% faster than running it on 3 boxes using both CPUs). > This was surprising to us since we've used to be limited by interprocessor communications, > which should be a lot faster between CPUs in one box. Can anybody explain the reason for > this and, more importantly, is there any way to improve this situation. > > Thanks, > > Sergei. > The so called "poor performance" in parallel mode when using 2 CPUs per node (box), comes due to the fact that the memory bandwidth is limited. If you are employing 2 CPUs then the 2 jobs share there memory interface. Otherwise if you are employing only 1 CPU and no other job is running on the node, you have the full memory bandwidth exclusive. This yields to the performance gain of only 50% when both CPUs are working. Our benchmarks using the unstructured Navier-Stokes TAU-code solver for a numerical calculation had shown the same behavior. We used a wing-body-engine aircraft configuration with 2 million grid points for the benchmarks and employed a full multigrid cycle to test the communication (here MPI-calls for a domain decomposition model). The performance gain (faster main loop times for a iteration) that we got for the 2. CPU are : Athlon MP FSB 133 1.6 GHz 72% Xeon Rambus FSB 100 2.0 GHz 55% Xeon DDR FSB 100 2.4 GHz 47% Xeon DDR FSB 100 2.8 GHz 43% Xeon DDR FSB 133 2.4 GHz 50% These are the values for 1 node in use. We have watched a decrease of the performance gain for the 2. CPU, when more then 1 node were used for the calculation (e.g. Xeon 2.4 GHz FSB 100 on 8 nodes only 37% for the 2. CPU). So if you are using a code that needs a lot of memory transfers, then you have decide, if the gain of the performance is worth the cost of the 2. CPU! But if you are looking for good benchmark results you might try an Opteron system (NUMA instead of Bus). Thomas -- __/|__ | Dipl.-Math. Thomas Alrutz /_/_/_/ | DLR Institute of Aerodynamics and Flow Technology |/ | Numerical Methods DLR | Bunsenstr. 10 | D-37073 Goettingen/Germany _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rgb at phy.duke.edu Fri Jun 6 08:41:38 2003 From: rgb at phy.duke.edu (Robert G. Brown) Date: Fri, 6 Jun 2003 08:41:38 -0400 (EDT) Subject: dual Xeon issue In-Reply-To: <20030606091532.6e0201fb.smuelas@mecanica.upm.es> Message-ID: On Fri, 6 Jun 2003, Santiago Muelas wrote: > - Do not expend money in "better" things. In this aspect, quantity is > better than "theoretical-quality" > - Normal ethernet is great and more than enough for 4,5 or 6 > machines.(and an standard switch) > - Choose the best price-quality possibility. > - Use "xmpi" and work on your program until the "red" has almost > disappeared. Ah, chela, you have done well! Honorable Master beg to offer only one small correction. Suggest do not assume problem of others same as problem of self. Sometimes problem need better quality, expensive network, dual processors, sometimes not. YMMV is the True Tao. However, observation that True Tao is optimize price/performance make Master very happy, very happy. Chela is Enlightened. Master also pleased that chela discover superlinear speedup and more. Suggest chela investigate memory bound feature of program -- perhaps dual cpu, one memory bus a problem? rgb Da "master" feelin' a wee bit lunatic this morning:-) Curiously, I had just written the following poem and added it to Hot Tea...time to go look in the mirror and see if those bloodshot eyes are lookin' a little crazy once again. -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb at phy.duke.edu Brilliance and Madness True brilliance, my friend, on the edge of madness lies So that many a great master has lunatic eyes. 'Tis an irregular stair that mounts to the skies And thus we conclude: it's crazy to be wise! From: http://www.phy.duke.edu/~rgb/Poetry/hot_tea.php _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From canon at nersc.gov Fri Jun 6 11:53:48 2003 From: canon at nersc.gov (canon at nersc.gov) Date: Fri, 06 Jun 2003 08:53:48 -0700 Subject: queuing systems and cpu usage? In-Reply-To: Message from Peter Beerli of "Wed, 04 Jun 2003 23:53:29 EDT." Message-ID: <200306061553.h56FrmS4018724@pookie.nersc.gov> Peter, We use LL on our ~6000 cpu SP system (416 Nighthawk II with two plane colony interconnect). We are able to get > 90% scheduled utilization even with very big jobs (>2048). While LL is hardly perfect, you should be able to get more out of it than you stated. I don't handle the LL config for the system, so I can tell you any magic bullets. I'm sure our SP admins would be happy to give you some suggestions though. On our 200 node Linux cluster we currently use LSF which is quite good. However, we are looking hard at SGE since it is free, open source, and maturing rapidly. Let me know if you would like more info. --Shane ------------------------------------------------------------------------ Shane Canon voice: 510-486-6981 PSDF Project Lead fax: 510-486-7520 National Energy Research Scientific Computing Center 1 Cyclotron Road Mailstop 943-256 Berkeley, CA 94720 canon at nersc.gov ------------------------------------------------------------------------ _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From shewa at inel.gov Fri Jun 6 12:46:22 2003 From: shewa at inel.gov (Andrew Shewmaker) Date: Fri, 06 Jun 2003 10:46:22 -0600 Subject: How to get started . . . In-Reply-To: <003e01c32bd3$ebbfdf60$88574b43@Homer> References: <200306051424.h55EOG516251@NewBlue.Scyld.com> <003e01c32bd3$ebbfdf60$88574b43@Homer> Message-ID: <3EE0C55E.2040602@inel.gov> Nuclear Nine wrote: > I am a newbie as well. What does everyone think about the Rocks cluster > distribution? According to their website it seems like the easiest > solution. Any comments. Rocks is very easy to use. I found it a bit easier to set up than OSCAR, but I think OSCAR has some other advantages (e.g., it can be in can be installed on several distributions). Both Rocks and OSCAR are great ways to build first generation beowulf clusters, and I hear Mandrake's is nice too. However, I believe all of these use NFS for at least /home. This may not affect you much on a small clusters, but it depends on your application. I avoid using NFS whenever I can. When/if you encounter bottlenecks with NFS, you should consider looking at lustre (www.lustre.org), PVFS (www.parl.clemson.edu/pvfs/), and v9fs (www.clustermatic.org). In addition to using NFS, these first generation cluster environments use ssh or rsh to start processes on the compute nodes. These work okay, but bproc (bproc.sf.net) provides a better way IMO. -Andrew -- Andrew Shewmaker, Associate Engineer Phone: 1-208-526-1276 Idaho National Eng. and Environmental Lab. P.0. Box 1625, M.S. 3605 Idaho Falls, Idaho 83415-3605 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From wdinkel at atipa.com Fri Jun 6 12:39:20 2003 From: wdinkel at atipa.com (Will Dinkel) Date: 06 Jun 2003 11:39:20 -0500 Subject: HyperThreading in Cluster In-Reply-To: <3EDFA22E.8090902@cora.nwra.com> References: <3EDFA22E.8090902@cora.nwra.com> Message-ID: <1054917560.9645.265.camel@zappa> As far as I know, the majority of the kernels that you will find in today's Linux distributions are HyperThreading "aware", but not HyperThreading "intelligent". As you mention below, the scheduler doesn't know that allocating two compute threads to a single processor is a BAD thing, especially when it leaves the other physical processor idle. Ingo Molnar has a patch to the latest 2.5.X kernel releases that addresses this issue. He uses per-physical-cpu runqueues and load balancing to ensure that the system load is correctly balanced between physical and virtual cpus. It's experimental, updated fairly regularly, delivers promising results, and can be found here: http://people.redhat.com/mingo/O(1)-scheduler/ On Thu, 2003-06-05 at 15:03, Orion Poplawski wrote: > This is partly prompted by Steven Timm's recent comment about > HyperThreading, as well as something I read in a magazine the other day > stating the HPC clusters need to run with HyperThreading off. Can > anyone shed more light on the current status of HyperThreading in > parallel computing? > > I could see where allocating two compute threads to a single processor > might bog things down (swapping data sets in/out of cache, etc.), but > it would seem ideal to allocate one compute thread per processor, and > let the other virtual processor help with OS/interrupt type operations. > Does the linux scheduler have such control yet? -- Will Dinkel Atipa Technologies -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part URL: From vngalchin at yahoo.com Fri Jun 6 20:17:04 2003 From: vngalchin at yahoo.com (Galchin Vasili) Date: Fri, 6 Jun 2003 17:17:04 -0700 (PDT) Subject: How is fault detection handled in Beowulf? Message-ID: <20030607001704.38843.qmail@web12203.mail.yahoo.com> Hello, How is fault detection handled in Beowulf? Is any attempt made to take a node out of the pool if a fault is detected? Regards, Vasili Galchin __________________________________ Do you Yahoo!? Yahoo! Calendar - Free online calendar with sync to Outlook(TM). http://calendar.yahoo.com _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From andrewxwang at yahoo.com.tw Sat Jun 7 09:45:06 2003 From: andrewxwang at yahoo.com.tw (=?big5?q?Andrew=20Wang?=) Date: Sat, 7 Jun 2003 21:45:06 +0800 (CST) Subject: GridEngine WorkShop 2003 (SGE 6.0, Globus 2/3 integration, etc...) Message-ID: <20030607134506.60981.qmail@web16804.mail.tpe.yahoo.com> Is the workshop free? Is there a website to list the programs? Thanks, Andrew. --- Ron Chen wrote: > Last years presentation available at: > http://gridengine.sunsource.net/project/gridengine/workshop22-24.04.02/proceedings.html > > -Ron > > > --- Monika Grobecker wrote: > > Dear Grid Engine Partner > > > > You are cordially invited to our second Grid Engine > > Development Workshop, > > September 22 to 24, 2003, in Regensburg, Germany. > > > > The objectives of the workshop are: > > > > Get together with the SGE developers and our > > development partners - > > Give and get an indepth overview on the ongoing and > > next Grid Engine > > developments - > > Present and discuss the development planned by Sun > > for the next major Grid > > Engine release (6.0) - > > Exchange research and development plans and > > coordinate activities. > > > > See below for a rough tentative agenda and travel > > details. > > > > We need your workshop registration by June 20, 2003 > > to enable us making the > > necessary hotel and conference room arrangements. > > Please register by simply > > sending e-mail to monika.grobecker at sun.com. > > > > Tentative Agenda: > > > > Sept 22: Welcome and Grid Engine Status Update > > Welcome > > Grid Engine Project Update > > Project activity and site updates > > New subprojects (GEP, JAM, Jgrid) > > Grid Engine Roadmap > > Architecture changes and feature enhancements > > Timetable > > Sun development plan > > Project gaps and contribution opportunities > > > > Sept 23: Partner and Contributor's Day > > Partners and contributors present their work with or > > around Grid Engine, e.g. > > on: > > Development contributions > > Globus Toolkit 2 and 3 integration; OGSA/Web > > services relation > > Grid Engine Portal > > End user scenarios: problems solved and lessons > > learned > > > > Sept 24: SIG and Workshop Day > > Identify Special Interest Groups > > Parallel SIG workshops > > Presentation and discussion of workshop results > > Adjourn > > > > Travel is easiest through the airport of Munich and > > rental car. There is a > > choice of hotel rooms from EUR 62 to 109, including > > breakfast. Try to arrive > > Saturday. Oktoberfest in Munich starts September 20 > > this year. Regensburg is > > very much worthwile a visit too, as many of you > > know. > > Details will follow soon. > > > > Regards > > > > Wolfgang > > Fritz > > Monika > > > > ************************************************ > > Sun Microsystems Gridware GmbH > > Monika Grobecker > > Dr.-Leo-Ritter-Str. 7 > > D-93049 Regensburg Phone: +49-941-3075-100 > > Germany Fax: +49-941-3075-222 > > > > mailto:monika.grobecker at sun.com > > ************************************************ > > > > > > > > > ----------------------------------------------------------------- ??? Yahoo!?? ??????? - ???????????? http://fate.yahoo.com.tw/ _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From andrewxwang at yahoo.com.tw Sat Jun 7 09:45:08 2003 From: andrewxwang at yahoo.com.tw (=?big5?q?Andrew=20Wang?=) Date: Sat, 7 Jun 2003 21:45:08 +0800 (CST) Subject: GridEngine WorkShop 2003 (SGE 6.0, Globus 2/3 integration, etc...) Message-ID: <20030607134508.49866.qmail@web16802.mail.tpe.yahoo.com> Is the workshop free? Is there a website to list the programs? Thanks, Andrew. --- Ron Chen wrote: > Last years presentation available at: > http://gridengine.sunsource.net/project/gridengine/workshop22-24.04.02/proceedings.html > > -Ron > > > --- Monika Grobecker wrote: > > Dear Grid Engine Partner > > > > You are cordially invited to our second Grid Engine > > Development Workshop, > > September 22 to 24, 2003, in Regensburg, Germany. > > > > The objectives of the workshop are: > > > > Get together with the SGE developers and our > > development partners - > > Give and get an indepth overview on the ongoing and > > next Grid Engine > > developments - > > Present and discuss the development planned by Sun > > for the next major Grid > > Engine release (6.0) - > > Exchange research and development plans and > > coordinate activities. > > > > See below for a rough tentative agenda and travel > > details. > > > > We need your workshop registration by June 20, 2003 > > to enable us making the > > necessary hotel and conference room arrangements. > > Please register by simply > > sending e-mail to monika.grobecker at sun.com. > > > > Tentative Agenda: > > > > Sept 22: Welcome and Grid Engine Status Update > > Welcome > > Grid Engine Project Update > > Project activity and site updates > > New subprojects (GEP, JAM, Jgrid) > > Grid Engine Roadmap > > Architecture changes and feature enhancements > > Timetable > > Sun development plan > > Project gaps and contribution opportunities > > > > Sept 23: Partner and Contributor's Day > > Partners and contributors present their work with or > > around Grid Engine, e.g. > > on: > > Development contributions > > Globus Toolkit 2 and 3 integration; OGSA/Web > > services relation > > Grid Engine Portal > > End user scenarios: problems solved and lessons > > learned > > > > Sept 24: SIG and Workshop Day > > Identify Special Interest Groups > > Parallel SIG workshops > > Presentation and discussion of workshop results > > Adjourn > > > > Travel is easiest through the airport of Munich and > > rental car. There is a > > choice of hotel rooms from EUR 62 to 109, including > > breakfast. Try to arrive > > Saturday. Oktoberfest in Munich starts September 20 > > this year. Regensburg is > > very much worthwile a visit too, as many of you > > know. > > Details will follow soon. > > > > Regards > > > > Wolfgang > > Fritz > > Monika > > > > ************************************************ > > Sun Microsystems Gridware GmbH > > Monika Grobecker > > Dr.-Leo-Ritter-Str. 7 > > D-93049 Regensburg Phone: +49-941-3075-100 > > Germany Fax: +49-941-3075-222 > > > > mailto:monika.grobecker at sun.com > > ************************************************ > > > > > > > > > ----------------------------------------------------------------- ??? Yahoo!?? ??????? - ???????????? http://fate.yahoo.com.tw/ _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From andrewxwang at yahoo.com.tw Sat Jun 7 10:05:27 2003 From: andrewxwang at yahoo.com.tw (=?big5?q?Andrew=20Wang?=) Date: Sat, 7 Jun 2003 22:05:27 +0800 (CST) Subject: queuing systems and cpu usage? (Partly OT) In-Reply-To: <010C86D15E4D1247B9A5DD312B7F5AA7E5E1AF@stegosaurus.bristol.quadrics.com> Message-ID: <20030607140528.10165.qmail@web16812.mail.tpe.yahoo.com> Just one point to share. The latest version of Maui is available at: http://www.supercluster.org/maui/ The version on http://mauischeduler.sourceforge.net/ is the `Molokini Edition', from what I have learnt from other people, never use this version! This is what people on the xcat list told me, I did not try it, but seems like those are valid points: http://bohnsack.com/lists/archives/xcat-user/2385.html Thanks, Andrew. --- John Brookes >> Maui's often used as the scheduler for PBS and Sun > GridEngine nowadays, so > getting to know its foibles wouldn't be wasted once > your Linux/Athlon > cluster arrives. > > The project is at: > http://mauischeduler.sourceforge.net/ > > Some information on Maui and the Maui/LL tie-in can > be found at eg: > http://supercluster.org/documentation/ > > Cheers, > > John Brookes > Quadrics Ltd. > > > > -----Original Message----- > > From: Peter Beerli [mailto:beerli at csit.fsu.edu] > > Sent: 05 June 2003 04:53 > > To: Beowulf Mailing list > > Subject: queuing systems and cpu usage? > > > > > > Hi all, > > I am a novice to all clustering and queueing > systems. > > So far, my cluster needs were satisfied on ad hoc > cluster to run > > embarrassingly parallel code (MCMC runs). After my > move, I can use > > time on a IBM Regatta cluster (and I am > impatiently waiting > > for my very > > own Linux Athlon cluster). The Regatta cluster is > running > > loadleveler which seems to > > have an abysmal job-scheduling performance (it > seems that > > currently out of the max=480 cpus > > only 288 are used [4 idle nodes out of 15] and > many jobs > > (including mine) are in the queues, waiting). > > > > I would be glad to hear information what > schedulers you > > prefer and (preferably) > > also get some numbers of how many nodes and cpus > are idle > > under standard load** > > or other appropriate statistics (what is the > appropriate statistics?). > > > > (this is not really a question for this list but > some might > > be able to answer: > > is out regatta loadleveler misconfigured?) > > > > **standard load: too many users submit jobs for > the queue > > with the longest time limit, > > very few medium, small length jobs, most of the > jobs are > > obviously only using <32 cpus > > (a node on the regatta has 32 cpus) > > > > thanks, > > Peter > > ---- > > Peter Beerli, > > School of Computational Science and Information > Technology (CSIT) > > Dirac Science Library, Florida State University > > Tallahassee, Florida 32306-4120 USA > > old webpage: > http://evolution.genetics.washington.edu/PBhtmls/beerli.html > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org > To change your subscription (digest mode or > unsubscribe) visit > http://www.beowulf.org/mailman/listinfo/beowulf > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org > To change your subscription (digest mode or > unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf ----------------------------------------------------------------- ??? Yahoo!?? ??????? - ???????????? http://fate.yahoo.com.tw/ _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From James.P.Lux at jpl.nasa.gov Sat Jun 7 12:26:51 2003 From: James.P.Lux at jpl.nasa.gov (Jim Lux) Date: Sat, 7 Jun 2003 09:26:51 -0700 Subject: ups units References: <3EDF9C66.5090106@cert.ucr.edu> Message-ID: <005201c32d11$9aaf9cc0$02a8a8c0@office1> First off... VA is not watts... Watts is active power ( integral of instantaneous voltage * instantaneous current). VA is RMS volts * RMS amps. If you hook up an RMS ammeter and measure the current draw, multiply that by the RMS voltage, you'll actually be measuring VA (Volt-Amps) If, and only if, the load is resistive so the current and voltage waveforms are "in phase" and identical, will VA=Watts. For all other cases (where PC power supplies happen to be), the current isn't necessarily in phase with the voltage, nor is the current waveform a nice sinusoid. The ratio between the "watts" and the "VA" is called the power factor. So, if you have a device that runs at 120V, and draws 10 amps, it has a VA of 1200. But, maybe the current is out of phase a bit, or non-sinusoidal, so if you actually measure the power (i.e. how fast the wheel spins on the meter), (maybe you put the load in a sealed, insulated box and measure the temperature rise?)... you find that it dissipates, say 1000 Watts. The power factor, in this case, would be 1000/1200 or 0.83 (which, by the way, is a typical kind of pf for a switching power supply). The other thing to bear in mind is the "Watt" number advertised for a power supply (i.e. a 400W power supply) may not bear much connection to how much power is actually consumed or supplied. The nameplate must (in a regulatory sense) give you the RMS current and RMS voltage, so you can get VA. It will also usually give Watts. Depending on the marking requirements, it might give power factor, as well. (The accuracy and reliability of nameplate markings is whole 'nother story... Unless something catches fire, or gets potentially blamed for a disaster, it's unlikely that the stuff actually gets checked.. certainly not on a "per each unit manufactured" basis) --------- Now we come to the other big problem.. inrush current. The nameplate ratings are for the "steady state" situation. Many devices, particularly those with large capacitors inside (i.e. power supplies) draw s everal times their steady state current when power is first applied. (AC Motors also draw 3-4 times their running current when starting, too). The circuitbreaker in your panel doesn't trip because it has a "inverse time/current" characteristic.. A short overload won't trip it, but a sustained one will (remember that the purpose of the circuit breaker ("overcurrent protection" or OCP) is to keep the place from burning down; NOT to protect your equipment. The short inrush might lead to a "light blink" as the load pulls down the supply voltage a bit, but won't usually trip the breaker. In fuses, this is the difference between "fast blow" and "slow blow".. although there are literally dozens of curves available. For circuit breakers, they actually have ones with knobs to set the curve, used in industrial applications (where you're turning on 10 kW of lights at a shot, or starting a 50 HP compressor, etc.) Your UPS, on the other hand, probably does overcurrent protection with a very fast acting current sensor. The OCP on the load side of the UPS is designed to protect the UPS (i.e. don't fry the output transistors). I suspect that this is a difference between the inexpensive UPS's and the "big iron room full of batteries" UPSs.. The former is really designed to run one PC (and it's associated fairly small inrush), the latter is designed with slow trip and some overhead in the design. ----- Original Message ----- From: "Glen Kaukola" To: "Beowulf" Sent: Thursday, June 05, 2003 12:39 PM Subject: ups units > I contacted the company we purchased the Cyberpower units from, and he > in turn contacted Cyberpower. Cyberpower informed him that there > shouldn't be more than one computer plugged into each UPS unit, and said > something which I didn't understand about 12 amps, 125 volts, and 400 X > 2 watts being more than 1500AV. It all sounded rather fishy to me, so I > was hoping that somebody on this list could clear things up for me. 12A * 125V (which sounds like a worst case, to me.. 125 is really high for a line voltage) = 1500 VA... That's probably their max output spec.. Consider your 400W power supplies.. If they had 0.8 power factor , the current each will draw is (400/110) / 0.8 = 4.5 Amps A 2:1 inrush would be perfectly reasonable... _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rgb at phy.duke.edu Sat Jun 7 14:29:33 2003 From: rgb at phy.duke.edu (Robert G. Brown) Date: Sat, 7 Jun 2003 14:29:33 -0400 (EDT) Subject: HPC conference near Raleigh NC... Message-ID: Dear Raleigh area beowulfers, Just to let you know, there is a conference/workshop on HPC in the Raleigh-Durham-Chapel Hill area next week at Wake Tech Community College. Wake Tech is one of several area colleges that are participants in a major NSF project designed to foster training programs for cluster administrators and programmers to meet what are (I think correctly) perceived to be exponentially growing demand. I've been informally advising them through the grantseeking process, although I'm not an actual participant. They've got a nice list of speakers for Friday with tracks focussing on bioinformatics (one of the major interests in this area with all the genetics and drug projects and companies), infrastructure, and tools and programming. In addition some talks will focus on educational issues that cluster users don't ordinarily focus on. The conference is free, and will even provide complimentary meals to people who register (so they can get a head count ahead of time). Area students are especially encouraged to attend. The conference website is at: http://www.highperformancecomputing.org/wtcc/wtcc061203.html and as of a few minutes ago still doesn't have the actual agenda posted. I have a copy however (as I'm one of the speakers) and am putting a copy at http://www.phy.duke.edu/~rgb/agenda.pdf (as I don't think the list manager really likes 200K attachments:-). If you're in the area and are interested in grid/cluster/beowulf computing (or any of the specific topics covered by talks on the agenda), give it a look and consider attending -- it should be fun. Again, be sure to contact Michele Blackmon and register if you plan to attend so that they can be sure to have enough food and room and so forth; this is their first such conference and may be surprised at the number of people in the area that are interested. Duke people interested in going are welcome to contact me about hitching a ride -- I've got a big car;-) rgb -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb at phy.duke.edu _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rgb at phy.duke.edu Sat Jun 7 14:47:29 2003 From: rgb at phy.duke.edu (Robert G. Brown) Date: Sat, 7 Jun 2003 14:47:29 -0400 (EDT) Subject: ups units In-Reply-To: <005201c32d11$9aaf9cc0$02a8a8c0@office1> Message-ID: On Sat, 7 Jun 2003, Jim Lux wrote: > First off... > VA is not watts... Watts is active power ( integral of instantaneous voltage ^^^^^^^^ time average (to pick nits -- VA and Watts >>are<< identical units. But Jim knew that; he meant time integral divided by the time, or average:-) > * instantaneous current). VA is RMS volts * RMS amps. Everything else Jim said was just peachy. I'd amplify a couple of points about PF -- if all your systems have a PF of 0.8 to 0.9 (typical for a non-PFC switching power supply) then as he noted a system that draws 120 watts (average power) on a 120 volt line needs to draw perhaps 120 Amps, not the 100 Amps you might expect on the basis of the average. However, power companies have to be able to deliver PEAK currents, not average currents, and usually charge you on the basis of those peaks as it limits their ability to distribute power elsewhere on their grid. These higher, off phase currents also have negative implications for your primary supply transformers, which can heat and actually become damaged or have a shorted lifetime if continuously operated near their nominal peak capacity. Finally, voltage brownouts during the peak draw period can limit the systems' ability to draw current during precisely the period they need to be drawing current to provide sustained power. This, in turn, can cause systems to lose some of their innate ability to buffer and ride out line surges caused by (for example) lots of systems turning on and off at once. Which in turn means that there can be more downtime, more expense and lost work associated with downtime, more hassle. The moral of all which is: a) Consider using harmonic mitigating transformers and other line conditioning on new "large" facilities intended to hold lots of cluster nodes with switching power supplies and get the wiring done by competent people who understand the issues involved and not just joe electrician off of the street; b) Otherwise leave a healthy margin of surplus capacity on each line when considering what you can run on old circuits or even new circuits with no special power factor/harmonic correction measures taken. rgb -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb at phy.duke.edu _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From James.P.Lux at jpl.nasa.gov Sat Jun 7 19:23:51 2003 From: James.P.Lux at jpl.nasa.gov (Jim Lux) Date: Sat, 7 Jun 2003 16:23:51 -0700 Subject: ups units References: Message-ID: <000801c32d4b$dbb25340$02a8a8c0@office1> ----- Original Message ----- From: "Robert G. Brown" To: "Jim Lux" Cc: "Glen Kaukola" ; "Beowulf" Sent: Saturday, June 07, 2003 11:47 AM Subject: Re: ups units > On Sat, 7 Jun 2003, Jim Lux wrote: > > > First off... > > VA is not watts... Watts is active power ( integral of instantaneous voltage > ^^^^^^^^ > time average > > (to pick nits -- VA and Watts >>are<< identical units. But Jim knew > that; he meant time integral divided by the time, or average:-) Indeed.. from a dimensional analysis standpoint, both VA and Watts (and VARs for that matter) are all rate of energy.. Joules/sec And, yes, it's really the time average of the product of the instantaneous power (i.e. volts*amps) > > > * instantaneous current). VA is RMS volts * RMS amps. > > Everything else Jim said was just peachy. I'd amplify a couple of > points about PF -- if all your systems have a PF of 0.8 to 0.9 (typical > for a non-PFC switching power supply) then as he noted a system that > draws 120 watts (average power) on a 120 volt line needs to draw perhaps > 120 Amps, not the 100 Amps you might expect on the basis of the average. And, watch out for low line voltage... Switching supplies are kind of interesting.. they're "constant power" devices, so as the line voltage drops, the current increases. This is unlike most conventional resistive/inductive loads, where as voltage drops, so does the current. Gang up a raft o' PCs so that your line voltage gets sucked down to 110 or 105 volts, and the current goes up correspondingly.. increasing the IR losses in the distribution circuitry. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From crhea at mayo.edu Sun Jun 8 12:00:07 2003 From: crhea at mayo.edu (Cris Rhea) Date: Sun, 8 Jun 2003 11:00:07 -0500 (CDT) Subject: Oddball networking question... Message-ID: <200306081600.h58G07wT005701@sijer.mayo.edu> Currently, my small RedHat-based cluster is connected via 100Mb Ethernet (small Cisco switch). I'm adding another node to act as a dedicated NFS server and would like to get as much bandwidth as possible between that system and the switch. At this time, I'm not is a position to upgrade everything to GigE, but would like to use a pair of GigE ports between the NFS server and the switch. The Cisco switch is a 24 port 10/100 switch, but copper GigE cards are available for the "option slots". >From the reading I've done, Linux suports channel bonding; but all nodes would require multiple Ethernet cards and also be running the bonding drivers. I'm looking only to bond two GigE links between my NFS server and the switch. Is this possible? If so, what's it called under Linux so I can read up on it? TIA- --- Cris ---- Cristopher J. Rhea Mayo Foundation Research Computing Facility Pavilion 2-25 crhea at Mayo.EDU Rochester, MN 55905 Fax: (507) 266-4486 (507) 284-0587 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From jhearns at freesolutions.net Sun Jun 8 13:47:22 2003 From: jhearns at freesolutions.net (John Hearns) Date: 08 Jun 2003 18:47:22 +0100 Subject: Oddball networking question... In-Reply-To: <200306081600.h58G07wT005701@sijer.mayo.edu> References: <200306081600.h58G07wT005701@sijer.mayo.edu> Message-ID: <1055094442.1656.1.camel@harwood.home> On Sun, 2003-06-08 at 17:00, Cris Rhea wrote: > > The Cisco switch is a 24 port 10/100 switch, but copper GigE cards are available > for the "option slots". > > From the reading I've done, Linux suports channel bonding; but all nodes would > require multiple Ethernet cards and also be running the bonding drivers. Why not use a single GigE card and one GBIC in the switch. I don't understand why you think two might be needed. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From bryan at westernfire.com Sun Jun 8 14:25:16 2003 From: bryan at westernfire.com (Bryan Klein) Date: Sun, 8 Jun 2003 11:25:16 -0700 Subject: Adaptive MPI on Beowulf... Message-ID: <000101c32deb$5044da80$6401a8c0@Thunder> I was curious if anyone here has ever run jobs using Adaptive MPI. http://finesse.cs.uiuc.edu/research/ampi/ I run some fortran/c++ code using MPI and this Charm++ AMPI stuff caught my eye. Bryan Klein Western Fire Center, Inc. bryan at westernfire.com _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From nfalano at hotmail.com Sun Jun 8 13:32:41 2003 From: nfalano at hotmail.com (Norman Alano) Date: Mon, 09 Jun 2003 01:32:41 +0800 Subject: need help re: script Message-ID: greetings! i need help re: my scenario. how i can create a script that i will only create account on the server and it automatically creates account on the nodes. and how i can create a script that i will shutdown the server node and it will automatically shutdown the nodes. thank you very much for your help cheers norman alano _________________________________________________________________ MSN 8 with e-mail virus protection service: 2 months FREE* http://join.msn.com/?page=features/virus _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From jmdavis at mail2.vcu.edu Sun Jun 8 13:50:25 2003 From: jmdavis at mail2.vcu.edu (Mike Davis) Date: Sun, 08 Jun 2003 13:50:25 -0400 Subject: Oddball networking question... In-Reply-To: <200306081600.h58G07wT005701@sijer.mayo.edu> References: <200306081600.h58G07wT005701@sijer.mayo.edu> Message-ID: <3EE37761.5040605@mail2.vcu.edu> The cisco switches support trunking (which is what you're looking for), but I'm not sure if this ability is limited to only connections between cisco products (the only way that I've done it). I would think that for a small cluster, you wouldn't need to trunk the GigE, but your app may be so disk intensive that it could be necessary. My channel bonded clusters all use two switches rather than a virtual switch. But, I'm in the process of replacing the networking of these older clusters with GigE. My new clusters were purchased with GigE. Channel bonding even FastE offers good performance increases (up to 190Mb/sec) in netperf. The problems that I've had are occasional race conditions leading to inetd dying and nodes locking up on occasion. Good luck, Mike Cris Rhea wrote: >Currently, my small RedHat-based cluster is connected via 100Mb Ethernet >(small Cisco switch). > >I'm adding another node to act as a dedicated NFS server and would like to get >as much bandwidth as possible between that system and the switch. > >At this time, I'm not is a position to upgrade everything to GigE, but would >like to use a pair of GigE ports between the NFS server and the switch. > >The Cisco switch is a 24 port 10/100 switch, but copper GigE cards are available >for the "option slots". > >From the reading I've done, Linux suports channel bonding; but all nodes would >require multiple Ethernet cards and also be running the bonding drivers. > >I'm looking only to bond two GigE links between my NFS server and the switch. >Is this possible? If so, what's it called under Linux so I can read up on it? > >TIA- > >--- Cris > > > > >---- > Cristopher J. Rhea Mayo Foundation > Research Computing Facility Pavilion 2-25 > crhea at Mayo.EDU Rochester, MN 55905 > Fax: (507) 266-4486 (507) 284-0587 >_______________________________________________ >Beowulf mailing list, Beowulf at beowulf.org >To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf > > > _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From tekka99 at libero.it Mon Jun 9 05:10:00 2003 From: tekka99 at libero.it (=?iso-8859-1?Q?Gianluca_Cecchi?=) Date: Mon, 9 Jun 2003 11:10:00 +0200 Subject: =?iso-8859-1?Q?lm=5Fsensors_configuration_tips_for_dual_athlon_MB?= Message-ID: Hello, I have several dual athlon servers as comp. nodes that are not in CED rooms. To avoid too much noise I'm using fans with low rpm values (around 4200) for the cpus of those servers. The mainboards used are Chaintech 7KDD, based on AMD 762 + 768 chipset. The cpus used are MP2200+ (1800MHz). >From the bios I'm able to set as the highest shutdown temp 70 centigrad degrees and sometimes I have spontaneus shutdown due to these settings, I imagine (if I reboot immediately I see about 63 degrees for the Cpus and 55 degrees for the MB). I would like to have any suggestion based on the experience of this list's users. The poer supply I'm using is a 450wat extreme. The o.s. is Slackware 9.0 with 2.4.20 kernel (also tried some 2.4.21-rc kernels). On the systems there are four scsi drives, as they are used as head node also. I have installed lm_sensors, to disable bios check and replace with sw checks and alert messages. But I have some problems in configuring the sensors.conf file. Any hints for dual athlon based systems? Based on sensors-detect command, at boot I load these modules and run these commands: /sbin/modprobe i2c-proc # I2C adapter drivers /sbin/modprobe i2c-amd756 /sbin/modprobe i2c-isa # I2C chip drivers /sbin/modprobe it87 /sbin/modprobe eeprom /usr/local/bin/sensors -s but the output is quite strange. I would like also to have some feedback on maximum safe temp I can set for Athlon MP 2200+ cpus and for MB cpu, and if the bios settings available is actually too restrictive as I think, or if I'm tttally wrong and unwise... How can I gain, if I add a case fan? Any hint? Thanks in advance. Bye, Gianluca _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rmyers1400 at attbi.com Mon Jun 9 05:24:57 2003 From: rmyers1400 at attbi.com (Robert Myers) Date: Mon, 09 Jun 2003 05:24:57 -0400 Subject: lm_sensors configuration tips for dual athlon MB In-Reply-To: References: Message-ID: <3EE45269.4090209@attbi.com> Gianluca Cecchi wrote: >I have several dual athlon servers as comp. nodes that are not in CED rooms. >To avoid too much noise I'm using fans with low rpm values (around 4200) for >the cpus of those servers. > If the cabinet will permit it (and it is worth cutting some custom holes if you're up for that sort of thing), a ducted chassis fan is a good way to move alot of air quietly. Might not have to let your CPU's run so hot. RM _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From shewa at inel.gov Mon Jun 9 09:46:19 2003 From: shewa at inel.gov (Andrew Shewmaker) Date: Mon, 09 Jun 2003 07:46:19 -0600 Subject: need help re: script In-Reply-To: References: Message-ID: <3EE48FAB.6090308@inel.gov> Norman Alano wrote: > > greetings! > > i need help re: my scenario. how i can create a script that i will only > create account on the server and it automatically creates account on the > nodes. and how i can create a script that i will shutdown the server > node and it will automatically shutdown the nodes. You might want to take a look at the C3 tools. http://www.csm.ornl.gov/torc/C3/ Andrew -- Andrew Shewmaker, Associate Engineer Phone: 1-208-526-1276 Idaho National Eng. and Environmental Lab. P.0. Box 1625, M.S. 3605 Idaho Falls, Idaho 83415-3605 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From josip at lanl.gov Mon Jun 9 11:17:12 2003 From: josip at lanl.gov (Josip Loncaric) Date: Mon, 09 Jun 2003 09:17:12 -0600 Subject: ups units In-Reply-To: <005201c32d11$9aaf9cc0$02a8a8c0@office1> References: <3EDF9C66.5090106@cert.ucr.edu> <005201c32d11$9aaf9cc0$02a8a8c0@office1> Message-ID: <3EE4A4F8.2090405@lanl.gov> Jim Lux wrote: > First off... > VA is not watts... Watts is active power ( integral of instantaneous voltage > * instantaneous current). VA is RMS volts * RMS amps Exactly! VA rating is only an upper bound on Watts... > The ratio between the "watts" and the "VA" is called the power factor. So, > if you have a device that runs at 120V, and draws 10 amps, it has a VA of > 1200. But, maybe the current is out of phase a bit, or non-sinusoidal, so > if you actually measure the power (i.e. how fast the wheel spins on the > meter), (maybe you put the load in a sealed, insulated box and measure the > temperature rise?)... you find that it dissipates, say 1000 Watts. The > power factor, in this case, would be 1000/1200 or 0.83 (which, by the way, > is a typical kind of pf for a switching power supply). Kill-a-Watt (available at Radio Shack for $30) can measure VA, Watts, and power factor. At the input to my UPS unit (supporting a dual CPU server, a fast ethernet switch, etc.), it reports about 185 VA or 130 W or power factor 0.7 (somewhat lower than Jim's example). > Now we come to the other big problem.. inrush current. This server draws less than 1 A RMS, but when it is powered up it draws almost 2 A RMS briefly, which matches Glen's 2:1 inrush estimate. Since the power-on phase is brief, a slow acting 20 A circuit breaker can support 12 such machines. However, as Jim pointed out, one needs to be more careful in observing VA limits of commodity UPS units whose electronics are sensitive to instantaneous load. Finally, the power supply rating printed on your PC is only an upper bound. While your PC may require a 300-400 W power supply (mainly to supply sufficient inrush current to the PC's components), under normal circumstances the PC will dissipate less than half this maximum power rating. So, if you have a room with X PCs, here is an estimate of the sufficient air conditioner capacity: you will have to remove roughly X*150W of heat (i.e. roughly X*500 BTU/h) plus a safety margin. Sincerely, Josip P.S. This "half the power supply rating" power dissipation estimate is very crude, but can be useful when more accurate figures are hard to obtain. BTW, in sizing power supplies, the aggregate Watt rating does not tell the whole story, because a PS supplies multiple voltages, each with its own current limit. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From jhearns at freesolutions.net Mon Jun 9 12:08:26 2003 From: jhearns at freesolutions.net (John Hearns) Date: Mon, 9 Jun 2003 17:08:26 +0100 Subject: ups units References: <3EDF9C66.5090106@cert.ucr.edu> <005201c32d11$9aaf9cc0$02a8a8c0@office1> <3EE4A4F8.2090405@lanl.gov> Message-ID: <035101c32ea1$63364820$8461cdc2@DREAD> > > Kill-a-Watt (available at Radio Shack for $30) can measure VA, Watts, > and power factor. At the input to my UPS unit (supporting a dual CPU > server, a fast ethernet switch, etc.), it reports about 185 VA or 130 W > or power factor 0.7 (somewhat lower than Jim's example). > For us European types, a quick search of the Radio Shack UK site did not reveal this product. So I did some Googling, and found a european version at http://www.prodigit.com/e2000m.htm (referenced in a thread on Tesla coils, with Jim Lux as a contributor. Do you really have such a thing, Jim?) It looks as if it has a Euro-standard power plug/socket. A UK distributor is listed as: http://www.etps.co.uk/ They just told me there is no 'UK tooled' version - ie. no UK three pin socket. They are getting me some idea of pricings for the Euro model. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From gotero at linuxprophet.com Mon Jun 9 21:23:46 2003 From: gotero at linuxprophet.com (Glen Otero) Date: Mon, 9 Jun 2003 18:23:46 -0700 Subject: Linux clusters for bioinformatics Message-ID: <2EBBCCCA-9AE2-11D7-8F9E-000393911A90@linuxprophet.com> Beowulfers- Just wanted to announce my cluster distro project, BioBrew, to the list. BioBrew is an open source Linux cluster distribution that is enhanced for bioinformaticists and life scientists. BioBrew automates cluster installation, includes all the HPC software a cluster enthusiast needs, and contains the NCBI toolkit, BLAST, mpiBLAST, HMMER, ClustalW, GROMACS, WISE, and EMBOSS, and will be available free of charge at Bioinformatics.org. BioBrew is in late stage beta testing and will be released this month at ClusterWorld, June 24th-26th in San Jose, where I'll be giving a talk on BioBrew. Please drop by if you're in the neighborhood and contact me if you'd like a free pass to get into the exhibit hall. I'm conducting a free online presentation on BioBrew later this week. If you'd like information on the presentation, please contact me off list. Glen Otero, Ph.D. Linux Prophet gotero at linuxprophet.com _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From sgaudet at wildopensource.com Tue Jun 10 08:27:28 2003 From: sgaudet at wildopensource.com (Stephen Gaudet) Date: Tue, 10 Jun 2003 08:27:28 -0400 Subject: Two Linux server specialists combine Message-ID: <3EE5CEB0.40301@wildopensource.com> Penguin Computing, a server maker specializing in machines running Linux, has signed an agreement to acquire Scyld Computing, the company founded by a pioneer of "Beowulf" Linux supercomputers, the companies plan to announce Tuesday. more: http://msnbc-cnet.com.com/2100-1010_3-1014970.html?type=pt&part=msnbc&tag=alert&form=feed&subj=cnetnews -- Steve Gaudet Wild Open Source (home office) ---------------------- Bedford, NH 03110 pH:603-488-1599 cell:603-498-1600 http://www.wildopensource.com _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From mbasa at incas.ro Mon Jun 9 08:00:09 2003 From: mbasa at incas.ro (Mihai Basa) Date: Mon, 9 Jun 2003 21:00:09 +0900 Subject: need help re: script Message-ID: <20030609210009.M72045@incas.ro> Hi Norman, I'm curious what the others will respond to your question, but here's one answer: http://seti.home.ro/add_cluster_user.sh It's a script that we've used for a while (we're using ROCKS now). It uses C3 (Cluster Command and Control) which you can get from http://www.csm.ornl.gov/torc/C3/ - which is a great tool to have around a cluster anyhow, and pretty easy to install. The script works ok, but isn't 100% abuse-proof. It creates users on all nodes and sets them up for password-less ssh logins. You might actually want to write your own script, but I hope this could help you to a quick start. To shutdown the nodes (once you've got C3) all you need is to type: cexec poweroff ...and voila, instant silence! :) Mihai NA> greetings! NA> i need help re: my scenario. how i can create a script that i will only NA> create account on the server and it automatically creates account on the NA> nodes. and how i can create a script that i will shutdown the server node NA> and it will automatically shutdown the nodes. NA> thank you very much for your help NA> cheers NA> norman alano NA> _________________________________________________________________ NA> MSN 8 with e-mail virus protection service: 2 months FREE* NA> http://join.msn.com/?page=features/virus NA> _______________________________________________ NA> Beowulf mailing list, Beowulf at beowulf.org NA> To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rene.storm at emplics.com Tue Jun 10 10:37:47 2003 From: rene.storm at emplics.com (Rene Storm) Date: Tue, 10 Jun 2003 16:37:47 +0200 Subject: AW: need help re: script Message-ID: <29B376A04977B944A3D87D22C495FB23012746@vertrieb.emplics.com> Hi, use nis (ypserv) to manage your accounts. http://www.linux.org/docs/ldp/howto/NIS-HOWTO/index.html For your execution scripts you could do something like: #!/bin/bash # . /etc/profile for the environment, needs passwordless rsh NODES="node02 node01" for NODE in $NODES; do echo "---------- $NODE" rsh $NODE ". /etc/profile; $*" echo done If you halt or reboot your machines, this machine should be last, to execute the script on ;o) There are many tools for that out there. Take a look at sourceforge. Cu Ren? -----Urspr?ngliche Nachricht----- Von: Norman Alano [mailto:nfalano at hotmail.com] Gesendet: Sonntag, 8. Juni 2003 19:33 An: beowulf at beowulf.org Betreff: need help re: script greetings! i need help re: my scenario. how i can create a script that i will only create account on the server and it automatically creates account on the nodes. and how i can create a script that i will shutdown the server node and it will automatically shutdown the nodes. thank you very much for your help cheers norman alano _________________________________________________________________ MSN 8 with e-mail virus protection service: 2 months FREE* http://join.msn.com/?page=features/virus _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From andy at ideafix.litec.csic.es Tue Jun 10 12:40:23 2003 From: andy at ideafix.litec.csic.es (A.P.Manners) Date: Tue, 10 Jun 2003 17:40:23 +0100 Subject: NAS Parallel Benchmarks for Current Hardware Message-ID: <3EE609F7.BE430A1E@ideafix.litec.csic.es> I am looking to put together a small cluster for numerical simulation and have been surprised at how few NPB benchmark results using current hardware I can find via google. Pointers to either a repository or individual results gratefully received. Is there some other (reasonably widely used) suite of benchmarks that is relevant for large scale numerical simulation? Thanks for any info. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From 8nrf at qlink.queensu.ca Tue Jun 10 11:42:26 2003 From: 8nrf at qlink.queensu.ca (Nathan Fredrickson) Date: Tue, 10 Jun 2003 11:42:26 -0400 Subject: need help re: script In-Reply-To: <29B376A04977B944A3D87D22C495FB23012746@vertrieb.emplics.com> Message-ID: > For your execution scripts you could do something like: > > #!/bin/bash > # . /etc/profile for the environment, needs passwordless rsh > NODES="node02 node01" > > for NODE in $NODES; do > echo "---------- $NODE" > rsh $NODE ". /etc/profile; $*" > echo > done > >There are many tools for that out there. Take a look at sourceforge. The distributed shell is a perl script that is similar to above, but has more features. It has become my favorite cluster tool. It can be found at: http://dsh.sourceforge.net/ Nathan _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From M.Arndt at science-computing.de Tue Jun 10 12:30:36 2003 From: M.Arndt at science-computing.de (Michael Arndt) Date: Tue, 10 Jun 2003 18:30:36 +0200 Subject: Comparison of Cluster price ratios Message-ID: <20030610183036.A26812@blnsrv1.science-computing.de> Hello * some weeks ago someone mailed, that he collects on a website information concerning price/performance rations for different linux cluster types (CPU / Disk / Mem) i am looking for info about somewhat actual numbers what to calculate for management info as numbers for Dollars / Megabyte Diskspace for Megabucks / Gigabyte SCSI / IDE, dito Mem, and CPU --- i know about the limitations of those numbers, but if someone has done something in this direction for linux clusters, please remail me the pointer TIA Micha _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rgupta at cse.iitkgp.ernet.in Tue Jun 10 16:22:40 2003 From: rgupta at cse.iitkgp.ernet.in (Rakesh Gupta) Date: Wed, 11 Jun 2003 01:52:40 +0530 (IST) Subject: AW: need help re: script In-Reply-To: <29B376A04977B944A3D87D22C495FB23012746@vertrieb.emplics.com> Message-ID: Hi, A simple solution would be to create an account on the server node and then do a copy of /etc/passwd /etc/groups /etc/shadow ( use rcp or scp or ssh ) to the cleint node for creating and managing account. For shutdown write another script ( basically run shutdown using ssh on the cleint ) wait for sufficient time and the shutdown the server. I have the scripts for this and could mail you if you are interested. Regards rakesh On Tue, 10 Jun 2003, Rene Storm wrote: > Hi, > > > use nis (ypserv) to manage your accounts. > http://www.linux.org/docs/ldp/howto/NIS-HOWTO/index.html > > For your execution scripts you could do something like: > > #!/bin/bash > # . /etc/profile for the environment, needs passwordless rsh > NODES="node02 node01" > > for NODE in $NODES; do > echo "---------- $NODE" > rsh $NODE ". /etc/profile; $*" > echo > done > > If you halt or reboot your machines, this machine should be last, to execute the script on ;o) > > There are many tools for that out there. Take a look at sourceforge. > > Cu > Ren? > > > -----Urspr?ngliche Nachricht----- > Von: Norman Alano [mailto:nfalano at hotmail.com] > Gesendet: Sonntag, 8. Juni 2003 19:33 > An: beowulf at beowulf.org > Betreff: need help re: script > > > > greetings! > > i need help re: my scenario. how i can create a script that i will only > create account on the server and it automatically creates account on the > nodes. and how i can create a script that i will shutdown the server node > and it will automatically shutdown the nodes. > > thank you very much for your help > > cheers > > norman alano > > _________________________________________________________________ > MSN 8 with e-mail virus protection service: 2 months FREE* > http://join.msn.com/?page=features/virus > > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org > To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org > To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf > -- ---------------------------------------------------------------------- Rakesh Gupta Research Consultant Computer Science and Engineering Department IIT Kharagpur West Bengal India - 721302 URL: http://www.crx.iitkgp.ernet.in/~rakesh/ Phone: 09832117500 -------------------------------------------------------------------- _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rbw at ahpcrc.org Tue Jun 10 18:36:29 2003 From: rbw at ahpcrc.org (Richard Walsh) Date: Tue, 10 Jun 2003 17:36:29 -0500 Subject: Nice survey of higher end power supplies ... Message-ID: <200306102236.h5AMaTw31240@mycroft.ahpcrc.org> All, Intersecting the recent interesting discussion of power and such, Tom's Hardware has just put up a nice review of higher-end supplies rating them on effeciency, noise, etc. There is a nice summary table at the end. You can ask your supplier what they are using and then look at the table and say ... "I don't want that one." ;-) http://www17.tomshardware.com/howto/20030609/index.html Enjoy, rbw #--------------------------------------------------- # Richard Walsh # Project Manager, Cluster Computing, Computational # Chemistry and Finance # netASPx, Inc. # 1200 Washington Ave. So. # Minneapolis, MN 55415 # VOX: 612-337-3467 # FAX: 612-337-3400 # EMAIL: rbw at networkcs.com, richard.walsh at netaspx.com # rbw at ahpcrc.org # #--------------------------------------------------- # "Why waste time learning when ignornace is # instantaneous?" -Thomas Hobbes #--------------------------------------------------- _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From hahn at physics.mcmaster.ca Tue Jun 10 20:31:31 2003 From: hahn at physics.mcmaster.ca (Mark Hahn) Date: Tue, 10 Jun 2003 20:31:31 -0400 (EDT) Subject: Nice survey of higher end power supplies ... In-Reply-To: <200306102236.h5AMaTw31240@mycroft.ahpcrc.org> Message-ID: > etc. There is a nice summary table at the end. You can > ask your supplier what they are using and then look at > the table and say ... "I don't want that one." ;-) > > http://www17.tomshardware.com/howto/20030609/index.html it's interesting, but I don't think it's all that useful. for instance, how many of you have cluster nodes that need >400W PS's? as usual THG seems mainly focused on satisfying the gamer/overclocker that he's got Da B35t. one relevant and interesting factoid was that they claim the PS's all operate at about 70% efficiency. it's a little hard to tell how they measured that, but I think it's at peak load. it would be nice to know whether efficiency is better at sane/normal loads. I just received a pair of kill-a-watt's today, and am eager to plug things into them. a PIII/833/i815 256M ram, 40G + 120G SATA demo machine I have here was around 50W. but someone already made off with the single kill-a-watt I brought in ;) PF .74 on that machine, by the way. I was kind of surprised, since it was a compaq deskpro. pretty old, though, that's probably why. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From snkapadi at usc.edu Tue Jun 10 16:50:39 2003 From: snkapadi at usc.edu (shyam kapadia) Date: Tue, 10 Jun 2003 13:50:39 -0700 Subject: Call for Papers for S.C.A.L.E Message-ID: <000501c32f91$f36e2070$6502a8c0@mouse> Southern CAlifornia Linux Exposition (SCALE) 2003. The USC, Simi/Conejo, and UCLA Linux User Groups are proud to present the second annual Southern California Linux Expo scheduled for November 22nd, 2003 at the Los Angeles Convention Center. The Southern California Linux Expo will bring together Linux and Open Source Software companies, developers, and users. The conference is being held by Linux user groups from the local area: the USCLUG, SCLUG, and UCLALUG. Building on the tremendous success of last year's SCALE, the three LUGs will expand on previous LUG Fests to promote Linux and the Open Source Software community. We INVITE you to share your work on Linux and Open Source projects with the rest of the community. Presentations can be in the form of speeches, tutorials, or demonstrations. A speech on Linux's uses in multimedia, a tutorial on iptables (demonstrating routing and firewall applications in Linux), or a demonstration of projects such as games, are just some examples of appropriate material. Presentations are allotted time in increments of 30 minutes up to 2 hours. For speeches, please submit your paper along with a description of the speech format and intended audience. For tutorials and demonstrations, please submit your proposal and let us know of any special needs such as audio/video equipment. All proposals are to be sent to kapadia at socallinuxexpo.com. The deadline for submissions has been extended to Sunday, October 6th at midnight. Further details for last years expo can be found at http://www.socallinuxexpo.com. Thanks and Regards SoCAlLinuxExpo CFP Committee. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From jeff at aslab.com Tue Jun 10 21:49:25 2003 From: jeff at aslab.com (Jeff Nguyen) Date: Tue, 10 Jun 2003 18:49:25 -0700 Subject: Nice survey of higher end power supplies ... References: <200306102236.h5AMaTw31240@mycroft.ahpcrc.org> Message-ID: <151001c32fbb$b0156d90$6502a8c0@jeff> > Intersecting the recent interesting discussion of power > and such, Tom's Hardware has just put up a nice review > of higher-end supplies rating them on effeciency, noise, > etc. There is a nice summary table at the end. You can > ask your supplier what they are using and then look at > the table and say ... "I don't want that one." ;-) > > http://www17.tomshardware.com/howto/20030609/index.html > There is one important factor which is difficult for the author to provide in the review. That is the reliability factor of the power supply under normal operating environment. It the power supply performs well but does not last, it is not an ideal choice either. Jeff ASL Inc. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From exa at kablonet.com.tr Tue Jun 10 17:32:37 2003 From: exa at kablonet.com.tr (Eray Ozkural) Date: Wed, 11 Jun 2003 00:32:37 +0300 Subject: PS2 cluster In-Reply-To: <004d01c32c0d$6b442190$8461cdc2@DREAD> References: <004d01c32c0d$6b442190$8461cdc2@DREAD> Message-ID: <200306110032.38038.exa@kablonet.com.tr> On Friday 06 June 2003 12:23, John Hearns wrote: > I remember when the PS2 first came out, > ideas were kicked around on the Beowulf list about > clustering them and using the vector units in them > (and using Firewire networking) I could definitely build a really cool 2^6 hypercube with firewire hardware but last time I checked there weren't any free TCP/IP drivers for firewire that would support the kind of routing I'd like (could you have something like CT routing in software?). Anybody's got the latest scoop on firewire for linux networking? Cheers, -- Eray Ozkural (exa) Comp. Sci. Dept., Bilkent University, Ankara KDE Project: http://www.kde.org www: http://www.cs.bilkent.edu.tr/~erayo Malfunction: http://mp3.com/ariza GPG public key fingerprint: 360C 852F 88B0 A745 F31B EA0F 7C07 AE16 874D 539C _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From fryman at cc.gatech.edu Tue Jun 10 21:44:04 2003 From: fryman at cc.gatech.edu (Josh Fryman) Date: Tue, 10 Jun 2003 21:44:04 -0400 Subject: Nice survey of higher end power supplies ... In-Reply-To: References: <200306102236.h5AMaTw31240@mycroft.ahpcrc.org> Message-ID: <20030610214404.02a655e9.fryman@cc.gatech.edu> > I just received a pair of kill-a-watt's today, and am eager to > plug things into them. speaking of these... i bought one at radioshack for $20 or so, and am curious if anyone out there has reverse-engineered the insides? i'm getting ready to attempt such a task to get the Watts output tied directly to a little micro that can report the power consumption back into the network. after all, why look at the LCD in some random location when i could just display it on my desktop? :) -j _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From James.P.Lux at jpl.nasa.gov Wed Jun 11 00:35:38 2003 From: James.P.Lux at jpl.nasa.gov (Jim Lux) Date: Tue, 10 Jun 2003 21:35:38 -0700 Subject: Nice survey of higher end power supplies ... References: Message-ID: <000601c32fd2$e87f4b80$02a8a8c0@office1> > > > > http://www17.tomshardware.com/howto/20030609/index.html > > it's interesting, but I don't think it's all that useful. > for instance, how many of you have cluster nodes that need >400W PS's? > as usual THG seems mainly focused on satisfying the gamer/overclocker > that he's got Da B35t. > > one relevant and interesting factoid was that they claim the PS's > all operate at about 70% efficiency. it's a little hard to tell > how they measured that, but I think it's at peak load. it would > be nice to know whether efficiency is better at sane/normal loads. Probably worse efficiency at light loads. There's a fairly significant "overhead" power that is consumed at all times, as well as a "proportional to load" loss. Also, the power factor is probably much worse at light loads, unless the PS uses some form of harmonic/PF correction (e.g. it is CE marked, and actually complies with the mark). The usual input stage is a rectifier to a capacitor input filter, which has an input current waveform that is very "pulse like" at low loads. High pulse current >> high I^2*R losses > > PF .74 on that machine, by the way. I was kind of surprised, > since it was a compaq deskpro. pretty old, though, that's probably why. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rgb at phy.duke.edu Wed Jun 11 09:03:40 2003 From: rgb at phy.duke.edu (Robert G. Brown) Date: Wed, 11 Jun 2003 09:03:40 -0400 (EDT) Subject: Nice survey of higher end power supplies ... In-Reply-To: Message-ID: On Tue, 10 Jun 2003, Mark Hahn wrote: > one relevant and interesting factoid was that they claim the PS's > all operate at about 70% efficiency. it's a little hard to tell > how they measured that, but I think it's at peak load. it would > be nice to know whether efficiency is better at sane/normal loads. They could be confusing PF and efficiency (obviously NOT the same thing at all). To otherwise measure efficiency they'd have to put controlled loads (e.g. big resistors, light bulbs) on all the various DC out lines and measure power delivery to that load (relatively simple, VA = W for DC) while measuring average power in. The number is believable, though. > I just received a pair of kill-a-watt's today, and am eager to > plug things into them. a PIII/833/i815 256M ram, 40G + 120G SATA > demo machine I have here was around 50W. but someone already > made off with the single kill-a-watt I brought in ;) Tell them to go buy their own. For $40 or $50 delivered, why not? rgb -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb at phy.duke.edu _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From k_kolokoutsas at yahoo.co.uk Wed Jun 11 08:01:01 2003 From: k_kolokoutsas at yahoo.co.uk (=?iso-8859-1?q?kolokoutsas=20konstantinos?=) Date: Wed, 11 Jun 2003 13:01:01 +0100 (BST) Subject: ideal motherboard for diskless beowulf Message-ID: <20030611120101.60267.qmail@web9901.mail.yahoo.com> Hello, I am in the process of designing a 12-node diskless cluster. I am going with AMD CPUs, but not quite confident which mobo to use, for serving the following with a limited budget: - (must) onboard LAN - (must) proven Linux (RH7.2) compatibility - (ideal) micro-atx - (ideal) wide range of CPU support, i.e. from Duron sub-GHz up to XP 2600+ - (bonus) some overclocking capabilities In terms of chip thanks in advance, Dr. Kostas Kolokoutsas __________________________________________________ Yahoo! Plus - For a better Internet experience http://uk.promotions.yahoo.com/yplus/yoffer.html _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rgb at phy.duke.edu Wed Jun 11 09:10:49 2003 From: rgb at phy.duke.edu (Robert G. Brown) Date: Wed, 11 Jun 2003 09:10:49 -0400 (EDT) Subject: Nice survey of higher end power supplies ... In-Reply-To: <20030610214404.02a655e9.fryman@cc.gatech.edu> Message-ID: On Tue, 10 Jun 2003, Josh Fryman wrote: > > I just received a pair of kill-a-watt's today, and am eager to > > plug things into them. > > speaking of these... i bought one at radioshack for $20 or so, and > am curious if anyone out there has reverse-engineered the insides? > > i'm getting ready to attempt such a task to get the Watts output > tied directly to a little micro that can report the power consumption > back into the network. after all, why look at the LCD in some > random location when i could just display it on my desktop? :) Do it! The RS232 chips are pretty cheap, IIRC. Then sell the little mini-circuit back to KAW for their "KAWII" with a serial readout. At $20 retail they can't be making a lot of money any more, and I'd cheerily put one on every circuit line in our cluster for $40 or even $50 each, if I could read the total line input. In fact, being able to control the output via serial line would be lovely too. There are only a half-dozen buttons or so, so a three or four bit set of control codes would likely do fine... rgb -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb at phy.duke.edu _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rgb at phy.duke.edu Wed Jun 11 09:57:39 2003 From: rgb at phy.duke.edu (Robert G. Brown) Date: Wed, 11 Jun 2003 09:57:39 -0400 (EDT) Subject: ideal motherboard for diskless beowulf In-Reply-To: <20030611120101.60267.qmail@web9901.mail.yahoo.com> Message-ID: On Wed, 11 Jun 2003, kolokoutsas konstantinos wrote: > Hello, > > I am in the process of designing a 12-node diskless > cluster. I am going with AMD CPUs, but not quite > confident which mobo to use, for serving the following > with a limited budget: > > - (must) onboard LAN > - (must) proven Linux (RH7.2) compatibility > - (ideal) micro-atx > - (ideal) wide range of CPU support, i.e. from Duron > sub-GHz up to XP 2600+ > - (bonus) some overclocking capabilities UP or MP? MP can be a hair cheaper per CPU, if your tasks aren't resource bound. They won't fit in a micro-atx, they won't do Duron CPUs, but on AVERAGE they will take up even less than two micro-atx's, especially in a 1U form factor. I've used a variety of MSI boards and (except for the taiwan capacitor fiasco) been generally satisfied with them. I use Tyan's (e.g. Tyan 2466 or better) for duals -- they have onboard 100BT or better (current motherboards have dual GigE NICs onboard, as well as video). Just about any of these boards will work fine with any version of RHL from 7-9. Beware motherboards with onboard RTL 8139 NICs -- they are cheap, common, and suck. I'd plan on adding a decent NIC a la carte if I "had" to pick a motherboard with an onboard RTL. Finally, I'd recommend using RH 9, not 7.2, and 7.3 over 7.2 if it came to that. Linux really does get better, and an essential part of installing ANYTHING is arranging for regular updates to fix security problems and bugs. The older versions tend to get frozen out and no longer be aggressively updated, meaning that any vulnerabilities or bugs in the setup won't get fixed. rgb -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb at phy.duke.edu _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From josip at lanl.gov Wed Jun 11 10:16:35 2003 From: josip at lanl.gov (Josip Loncaric) Date: Wed, 11 Jun 2003 08:16:35 -0600 Subject: Nice survey of higher end power supplies ... In-Reply-To: References: Message-ID: <3EE739C3.5040308@lanl.gov> Mark Hahn wrote: > > PF .74 on that machine, by the way. I was kind of surprised, > since it was a compaq deskpro. pretty old, though, that's probably why. Power factor is about 0.65 on my own machine. I was kind of surprised too. According to Kill-a-watt, my 2.8 GHz P4 w/Asus motherboard, lots of other goodies, and a 2002-vintage Antec PS dissipates 130-180 Watts in operation depending on activity. Another disappointment was its "sleep" mode which dissipates 100 Watts (not much better than the 130 W idle state). When "off" it still uses 5 Watts... Sincerely, Josip P.S. PF 0.65 means that at 180 W the machine draws about 270 VA. This VA rating is important when sizing UPS units. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From jhearns at freesolutions.net Wed Jun 11 10:47:47 2003 From: jhearns at freesolutions.net (John Hearns) Date: Wed, 11 Jun 2003 15:47:47 +0100 Subject: ideal motherboard for diskless beowulf References: Message-ID: <03d501c33028$6fc89fb0$8461cdc2@DREAD> > > Finally, I'd recommend using RH 9, not 7.2, and 7.3 over 7.2 if it came > to that. Linux really does get better, and an essential part of > installing ANYTHING is arranging for regular updates to fix security > problems and bugs. The older versions tend to get frozen out and no > longer be aggressively updated, meaning that any vulnerabilities or bugs > in the setup won't get fixed. > Given that RH is at version 9, I think the current RH policy is that version 7.xx will be unsupported after one year. 7.2 and 7.3 have End of Life dates for errata updates of 31st December 2003 http://www.redhat.com/apps/support/errata/ I would guess you are specifying 7.2 or 7.3 due to your applications. You should consider testing/recompiling/certifying them on a more recent version. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rbw at ahpcrc.org Wed Jun 11 11:22:15 2003 From: rbw at ahpcrc.org (Richard Walsh) Date: Wed, 11 Jun 2003 10:22:15 -0500 Subject: Nice survey of higher end power supplies ... Message-ID: <200306111522.h5BFMFh14234@mycroft.ahpcrc.org> Wanting to make sure I have these power related terms straight, I wrote up definitions for myself based on recent discussions and would be grateful if folks would validate and/or correct them. 1. Idealized or VA Rated Power: An idealized measure of power delivered (or drawn) that assumes the voltage and amperage are in phase, sinusoidal, and of the magnitudes (RMS) used in the calculation. Most often used to rate/describe line power. 2. Active or Watt Rated Power: The power truly delivered (or drawn) and averaged over some period of time (integral divided by time). Typically less than 1 above (but measured in the same units of course) because most loads are not perfectly resistive and supplied power has non- sinusoidal waveforms, out of phase voltage and amperage, less than ideal magnitudes (RMS) for V and A. (The complicating effects of Bob's line harmonics, etc. would register here as well I suppose.) Often used to describe power drawn/used by a device (a PC power supply) and distinct from line power. The ratio of 2 to 1 is the Power Factor. 3. Power Efficiency: Not to be confused with the Power Factor, the ratio of the power delivered from a device to a given load to the power consumed on input by the device. In the case of a typical PC power supply, this would be the ratio of the power drawn (by lines [3.3, 5, 12V] in use for a particular load over a specific time) over the power consumed by the PC power supply which includes its own draw and heat related losses (my guess is that active power [2 above] should be used in the denominator here). Too wordy, but are these definitions accurate, sufficiently complete? rbw _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From mikee at mikee.ath.cx Wed Jun 11 11:24:29 2003 From: mikee at mikee.ath.cx (Mike Eggleston) Date: Wed, 11 Jun 2003 10:24:29 -0500 Subject: ideal motherboard for diskless beowulf In-Reply-To: <20030611120101.60267.qmail@web9901.mail.yahoo.com>; from k_kolokoutsas@yahoo.co.uk on Wed, Jun 11, 2003 at 01:01:01PM +0100 References: <20030611120101.60267.qmail@web9901.mail.yahoo.com> Message-ID: <20030611102429.A16195@mikee.ath.cx> On Wed, 11 Jun 2003, kolokoutsas konstantinos wrote: > Hello, > > I am in the process of designing a 12-node diskless > cluster. I am going with AMD CPUs, but not quite > confident which mobo to use, for serving the following > with a limited budget: > > - (must) onboard LAN > - (must) proven Linux (RH7.2) compatibility > - (ideal) micro-atx > - (ideal) wide range of CPU support, i.e. from Duron > sub-GHz up to XP 2600+ > - (bonus) some overclocking capabilities Don't know about ideal, but I am using the Biostar M7KVQ with an AMD XP 2100+ and 512MB. Mike _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rgb at phy.duke.edu Wed Jun 11 14:15:41 2003 From: rgb at phy.duke.edu (Robert G. Brown) Date: Wed, 11 Jun 2003 14:15:41 -0400 (EDT) Subject: Nice survey of higher end power supplies ... In-Reply-To: <200306111522.h5BFMFh14234@mycroft.ahpcrc.org> Message-ID: On Wed, 11 Jun 2003, Richard Walsh wrote: > Too wordy, but are these definitions accurate, sufficiently complete? Sure, as far as I can see. rgb -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb at phy.duke.edu _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From abignone at unige.it Tue Jun 10 17:02:35 2003 From: abignone at unige.it (Franco A. Bignone) Date: Tue, 10 Jun 2003 23:02:35 +0200 Subject: PS2 cluster References: <200306061127.h56BRl505710@NewBlue.Scyld.com> Message-ID: <3EE6476B.E12277B2@unige.it> > Reply-To: "John Hearns" > From: "John Hearns" > To: > Subject: PS2 cluster > Date: Fri, 6 Jun 2003 10:23:38 +0100 > > I remember when the PS2 first came out, > ideas were kicked around on the Beowulf list about > clustering them and using the vector units in them > (and using Firewire networking) > > Looks like NCS are actually doing this, using the PS2 Linux kit > and Ethernet: > > http://access.ncsa.uiuc.edu/Releases/05.27.03_Playing_th.html > > Wow! Right, if I have understood correctly they have set a 70 node cluster, plus some connecting hardware for a price tag of 20,000 $. Not bad. Is it effective in price/performance ? It seems nice route to follow if your software make use of the VU, even if I don't have clear how much is difficult to set it up correctly. FB. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From abignone at unige.it Tue Jun 10 17:03:41 2003 From: abignone at unige.it (Franco A. Bignone) Date: Tue, 10 Jun 2003 23:03:41 +0200 Subject: PS2 cluster References: <200306061127.h56BRl505710@NewBlue.Scyld.com> Message-ID: <3EE647AD.765F2189@unige.it> > Reply-To: "John Hearns" > From: "John Hearns" > To: > Subject: PS2 cluster > Date: Fri, 6 Jun 2003 10:23:38 +0100 > > I remember when the PS2 first came out, > ideas were kicked around on the Beowulf list about > clustering them and using the vector units in them > (and using Firewire networking) > > Looks like NCS are actually doing this, using the PS2 Linux kit > and Ethernet: > > http://access.ncsa.uiuc.edu/Releases/05.27.03_Playing_th.html > > Wow! Right, if I have understood correctly they have set a 70 node cluster, plus some connecting hardware for a price tag of 20,000 $. Not bad. Is it effective in price/performance ? It seems nice route to follow if your software make use of the VU, even if I don't have clear how much is difficult to set it up correctly. FB . _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From abignone at unige.it Tue Jun 10 17:02:59 2003 From: abignone at unige.it (Franco A. Bignone) Date: Tue, 10 Jun 2003 23:02:59 +0200 Subject: PS2 cluster References: <200306061127.h56BRl505710@NewBlue.Scyld.com> Message-ID: <3EE64783.38C39F3B@unige.it> > Reply-To: "John Hearns" > From: "John Hearns" > To: > Subject: PS2 cluster > Date: Fri, 6 Jun 2003 10:23:38 +0100 > > I remember when the PS2 first came out, > ideas were kicked around on the Beowulf list about > clustering them and using the vector units in them > (and using Firewire networking) > > Looks like NCS are actually doing this, using the PS2 Linux kit > and Ethernet: > > http://access.ncsa.uiuc.edu/Releases/05.27.03_Playing_th.html > > Wow! Right, if I have understood correctly they have set a 70 node cluster, plus some connecting hardware for a price tag of 20,000 $. Not bad. Is it effective in price/performance ? It seems nice route to follow if your software make use of the VU, even if I don't have clear how much is difficult to set it up correctly. FB . _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From abignone at unige.it Tue Jun 10 17:04:07 2003 From: abignone at unige.it (Franco A. Bignone) Date: Tue, 10 Jun 2003 23:04:07 +0200 Subject: PS2 cluster References: <200306061127.h56BRl505710@NewBlue.Scyld.com> Message-ID: <3EE647C7.6686E8F3@unige.it> > Reply-To: "John Hearns" > From: "John Hearns" > To: > Subject: PS2 cluster > Date: Fri, 6 Jun 2003 10:23:38 +0100 > > I remember when the PS2 first came out, > ideas were kicked around on the Beowulf list about > clustering them and using the vector units in them > (and using Firewire networking) > > Looks like NCS are actually doing this, using the PS2 Linux kit > and Ethernet: > > http://access.ncsa.uiuc.edu/Releases/05.27.03_Playing_th.html > > Wow! Right, if I have understood correctly they have set a 70 node cluster, plus some connecting hardware for a price tag of 20,000 $. Not bad. Is it effective in price/performance ? It seems nice route to follow if your software make use of the VU, even if I don't have clear how much is difficult to set it up correctly. FB. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From abignone at unige.it Tue Jun 10 17:04:25 2003 From: abignone at unige.it (Franco A. Bignone) Date: Tue, 10 Jun 2003 23:04:25 +0200 Subject: PS2 cluster. References: <200306061127.h56BRl505710@NewBlue.Scyld.com> Message-ID: <3EE647D9.4A18D7C7@unige.it> > Reply-To: "John Hearns" > From: "John Hearns" > To: > Subject: PS2 cluster > Date: Fri, 6 Jun 2003 10:23:38 +0100 > > I remember when the PS2 first came out, > ideas were kicked around on the Beowulf list about > clustering them and using the vector units in them > (and using Firewire networking) > > Looks like NCS are actually doing this, using the PS2 Linux kit > and Ethernet: > > http://access.ncsa.uiuc.edu/Releases/05.27.03_Playing_th.html > > Wow! Right, if I have understood correctly they have set a 70 node cluster, plus some connecting hardware for a price tag of 20,000 $. Not bad. Is it effective in price/performance ? It seems nice route to follow if your software make use of the VU, even if I don't have clear how much is difficult to set it up correctly. FB . _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From abignone at unige.it Tue Jun 10 17:02:22 2003 From: abignone at unige.it (Franco A. Bignone) Date: Tue, 10 Jun 2003 23:02:22 +0200 Subject: PS2 cluster References: <200306061127.h56BRl505710@NewBlue.Scyld.com> Message-ID: <3EE6475D.7E41876F@unige.it> > Reply-To: "John Hearns" > From: "John Hearns" > To: > Subject: PS2 cluster > Date: Fri, 6 Jun 2003 10:23:38 +0100 > > I remember when the PS2 first came out, > ideas were kicked around on the Beowulf list about > clustering them and using the vector units in them > (and using Firewire networking) > > Looks like NCS are actually doing this, using the PS2 Linux kit > and Ethernet: > > http://access.ncsa.uiuc.edu/Releases/05.27.03_Playing_th.html > > Wow! Right, if I have understood correctly they have set a 70 node cluster, plus some connecting hardware for a price tag of 20,000 $. Not bad. Is it effective in price/performance ? It seems nice route to follow if your software make use of the VU, even if I don't have clear how much is difficult to set it up correctly. FB _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From abignone at unige.it Tue Jun 10 17:04:15 2003 From: abignone at unige.it (Franco A. Bignone) Date: Tue, 10 Jun 2003 23:04:15 +0200 Subject: PS2 cluster References: <200306061127.h56BRl505710@NewBlue.Scyld.com> Message-ID: <3EE647CF.44C294D4@unige.it> > Reply-To: "John Hearns" > From: "John Hearns" > To: > Subject: PS2 cluster > Date: Fri, 6 Jun 2003 10:23:38 +0100 > > I remember when the PS2 first came out, > ideas were kicked around on the Beowulf list about > clustering them and using the vector units in them > (and using Firewire networking) > > Looks like NCS are actually doing this, using the PS2 Linux kit > and Ethernet: > > http://access.ncsa.uiuc.edu/Releases/05.27.03_Playing_th.html > > Wow! Right, if I have understood correctly they have set a 70 node cluster, plus some connecting hardware for a price tag of 20,000 $. Not bad. Is it effective in price/performance ? It seems nice route to follow if your software make use of the VU, even if I don't have clear how much is difficult to set it up correctly. FB . _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From abignone at unige.it Tue Jun 10 17:03:47 2003 From: abignone at unige.it (Franco A. Bignone) Date: Tue, 10 Jun 2003 23:03:47 +0200 Subject: PS2 cluster References: <200306061127.h56BRl505710@NewBlue.Scyld.com> Message-ID: <3EE647B3.E76EE35F@unige.it> > Reply-To: "John Hearns" > From: "John Hearns" > To: > Subject: PS2 cluster > Date: Fri, 6 Jun 2003 10:23:38 +0100 > > I remember when the PS2 first came out, > ideas were kicked around on the Beowulf list about > clustering them and using the vector units in them > (and using Firewire networking) > > Looks like NCS are actually doing this, using the PS2 Linux kit > and Ethernet: > > http://access.ncsa.uiuc.edu/Releases/05.27.03_Playing_th.html > > Wow! Right, if I have understood correctly they have set a 70 node cluster, plus some connecting hardware for a price tag of 20,000 $. Not bad. Is it effective in price/performance ? It seems nice route to follow if your software make use of the VU, even if I don't have clear how much is difficult to set it up correctly. FB _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From abignone at unige.it Tue Jun 10 17:03:59 2003 From: abignone at unige.it (Franco A. Bignone) Date: Tue, 10 Jun 2003 23:03:59 +0200 Subject: PS2 cluster References: <200306061127.h56BRl505710@NewBlue.Scyld.com> Message-ID: <3EE647BF.1E32F9E1@unige.it> > Reply-To: "John Hearns" > From: "John Hearns" > To: > Subject: PS2 cluster > Date: Fri, 6 Jun 2003 10:23:38 +0100 > > I remember when the PS2 first came out, > ideas were kicked around on the Beowulf list about > clustering them and using the vector units in them > (and using Firewire networking) > > Looks like NCS are actually doing this, using the PS2 Linux kit > and Ethernet: > > http://access.ncsa.uiuc.edu/Releases/05.27.03_Playing_th.html > > Wow! Right, if I have understood correctly they have set a 70 node cluster, plus some connecting hardware for a price tag of 20,000 $. Not bad. Is it effective in price/performance ? It seems nice route to follow if your software make use of the VU, even if I don't have clear how much is difficult to set it up correctly. FB _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From abignone at unige.it Tue Jun 10 17:03:08 2003 From: abignone at unige.it (Franco A. Bignone) Date: Tue, 10 Jun 2003 23:03:08 +0200 Subject: PS2 cluster References: <200306061127.h56BRl505710@NewBlue.Scyld.com> Message-ID: <3EE6478C.BCE8A42F@unige.it> > Reply-To: "John Hearns" > From: "John Hearns" > To: > Subject: PS2 cluster > Date: Fri, 6 Jun 2003 10:23:38 +0100 > > I remember when the PS2 first came out, > ideas were kicked around on the Beowulf list about > clustering them and using the vector units in them > (and using Firewire networking) > > Looks like NCS are actually doing this, using the PS2 Linux kit > and Ethernet: > > http://access.ncsa.uiuc.edu/Releases/05.27.03_Playing_th.html > > Wow! Right, if I have understood correctly they have set a 70 node cluster, plus some connecting hardware for a price tag of 20,000 $. Not bad. Is it effective in price/performance ? It seems nice route to follow if your software make use of the VU, even if I don't have clear how much is difficult to set it up correctly. FB . _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From JAI_RANGI at SDSTATE.EDU Wed Jun 11 16:19:17 2003 From: JAI_RANGI at SDSTATE.EDU (RANGI, JAI) Date: Wed, 11 Jun 2003 15:19:17 -0500 Subject: Octave on Beowulf Message-ID: Octave is Mat lab like program but free. Has any tried to run octave on Beowulf Cluster. Any hint or link will be appreciated. Thanks Jai Rangi _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From JAI_RANGI at SDSTATE.EDU Wed Jun 11 16:19:17 2003 From: JAI_RANGI at SDSTATE.EDU (RANGI, JAI) Date: Wed, 11 Jun 2003 15:19:17 -0500 Subject: Octave on Beowulf Message-ID: Octave is Mat lab like program but free. Has any tried to run octave on Beowulf Cluster. Any hint or link will be appreciated. Thanks Jai Rangi _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From math at velocet.ca Wed Jun 11 16:19:00 2003 From: math at velocet.ca (Ken Chase) Date: Wed, 11 Jun 2003 16:19:00 -0400 Subject: PS2 cluster In-Reply-To: <3EE647CF.44C294D4@unige.it>; from abignone@unige.it on Tue, Jun 10, 2003 at 11:04:15PM +0200 References: <200306061127.h56BRl505710@NewBlue.Scyld.com> <3EE647CF.44C294D4@unige.it> Message-ID: <20030611161859.T29733@velocet.ca> On Tue, Jun 10, 2003 at 11:04:15PM +0200, Franco A. Bignone's all... >> Reply-To: "John Hearns" >> From: "John Hearns" >> To: >> Subject: PS2 cluster >> Date: Fri, 6 Jun 2003 10:23:38 +0100 >> >> I remember when the PS2 first came out, >> ideas were kicked around on the Beowulf list about >> clustering them and using the vector units in them >> (and using Firewire networking) >> >> Looks like NCS are actually doing this, using the PS2 Linux kit >> and Ethernet: >> >> http://access.ncsa.uiuc.edu/Releases/05.27.03_Playing_th.html >> >> Wow! > >Right, if I have understood correctly they have set a 70 node cluster, >plus some connecting hardware for a price tag of 20,000 $. Not bad. >Is it effective in price/performance ? It seems nice route to follow >if your software make use of the VU, even if I don't have clear how >much is difficult to set it up correctly. > >FB >. Every msg is slightly different that you sent to the list, especially the . at the end. are you trying to do something here? Perhpas your have notMeToo set in your beowulf-l mailman prefs. Go visit the mailman URL for this list please. /kc > >_______________________________________________ >Beowulf mailing list, Beowulf at beowulf.org >To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- Ken Chase, math at velocet.ca * Velocet Communications Inc. * Toronto, Canada Wiznet Velocet DSL.ca Datavaults 24/7: 416-967-4414 tollfree: 1-866-353-0363 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From becker at scyld.com Wed Jun 11 16:40:31 2003 From: becker at scyld.com (Donald Becker) Date: Wed, 11 Jun 2003 16:40:31 -0400 (EDT) Subject: Nice survey of higher end power supplies ... In-Reply-To: <3EE739C3.5040308@lanl.gov> Message-ID: On Wed, 11 Jun 2003, Josip Loncaric wrote: > Mark Hahn wrote: > > > > PF .74 on that machine, by the way. I was kind of surprised, > > since it was a compaq deskpro. pretty old, though, that's probably why. > > Power factor is about 0.65 on my own machine. I was kind of surprised > too. I'm shocked at that low PF. I had believed that most modern power supplies had some PF correction. > According to Kill-a-watt, my 2.8 GHz P4 w/Asus motherboard, lots > of other goodies, and a 2002-vintage Antec PS dissipates 130-180 Watts > in operation depending on activity. Another disappointment was its > "sleep" mode which dissipates 100 Watts (not much better than the 130 W > idle state). When "off" it still uses 5 Watts... This doesn't surprise me at all. Sleep mode on a desktop is no better than executing a halt instruction. Unlike a laptop, where all devices are known and controlled, a desktop machine still has to leave most of the system active to support unknown PCI or ISA devices. This has the potential to change. When most devices implement PCI power management, the OS should be able to verify that there are unsupported devices and tell the ACPI firmware to really turn things off. But you won't find this fully implemented for currrent motherboards. -- Donald Becker becker at scyld.com Scyld Computing Corporation http://www.scyld.com 914 Bay Ridge Road, Suite 220 Scyld Beowulf cluster system Annapolis MD 21403 410-990-9993 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From michael.worsham at mci.com Wed Jun 11 16:03:57 2003 From: michael.worsham at mci.com (Michael Worsham) Date: Wed, 11 Jun 2003 16:03:57 -0400 Subject: Slackware Beowulf Cluster Message-ID: <003c01c33054$98d248d0$2a408c0a@Wcomnet.com> Has anyone attempted to create a beowulf cluster with Slackware? If so, what files are required for a base install? Thanks. -- M _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From laurenceliew at yahoo.com.sg Thu Jun 12 02:05:47 2003 From: laurenceliew at yahoo.com.sg (=?iso-8859-1?q?Laurence?=) Date: Thu, 12 Jun 2003 14:05:47 +0800 (CST) Subject: Octave on Beowulf In-Reply-To: Message-ID: <20030612060547.45199.qmail@web11904.mail.yahoo.com> Hi, Octave will run on any linux platform.. so that includes all Beowulf.... I guess the question really is will it run FASTER? I know SciLab has PVM bindings...., for Octave.. I am not sure if there are PVM or MPI bindings... but you may wish to try to use some of the freely available MPI bindings/libraries for MatLab: google Matlab MPI Else.. if you code is not MPI/PVM parallel, that is it is serial code, you can just use PBS/SGE/LSF to submit your Octave jobs to the cluster.... Hope this helps. Cheers! Laurence --- "RANGI, JAI" wrote: > Octave is Mat lab like program but free. > Has any tried to run octave on Beowulf Cluster. > Any hint or link will be appreciated. > Thanks > > Jai Rangi > > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org > To change your subscription (digest mode or > unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf ===== --- Cheers! Laurence __________________________________________________ Do You Yahoo!? Send free SMS from your PC! http://sg.sms.yahoo.com _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From smuelas at mecanica.upm.es Thu Jun 12 01:57:48 2003 From: smuelas at mecanica.upm.es (Santiago Muelas) Date: Thu, 12 Jun 2003 07:57:48 +0200 Subject: ideal motherboard for diskless beowulf In-Reply-To: <20030611120101.60267.qmail@web9901.mail.yahoo.com> References: <20030611120101.60267.qmail@web9901.mail.yahoo.com> Message-ID: <20030612075748.2e08bb37.smuelas@mecanica.upm.es> I have just done what you are going to do. Only with 8 PC's. I give you MY recommendations just in case they are of any interest for you as I am not at all an expert. So, remind please that these are the advice of a novice, but I will write freely to make things easier... - As far as I know, the best AMD in quality/price today in Europe is Ahtlon XP2400. Just 100 eur. - If you are going to work with the beast near you, buy coolers from "Artic Cooler" model Cooper Silent for the CPU: 18 eur. - If your budget allows it, buy aluminium cases. Mr. XP2400 will be happy. - I strongly recommend as I have tested it, the last micro-atx from Shuttle with the chipset from Nvidia: nforce2. It goes around 10% quicker in my installation than any other tested. Price: 100 eur. - Personally, I don't overclock. I have payed a good amount to risk anything (and I don't think it is worth). Those are my actual feelings. Please, once you have your "children" working, tell me what are your feelings. I am very interested in new, relatively small installations. Santiago Muelas On Wed, 11 Jun 2003 13:01:01 +0100 (BST) kolokoutsas konstantinos wrote: > Hello, > > I am in the process of designing a 12-node diskless > cluster. I am going with AMD CPUs, but not quite > confident which mobo to use, for serving the following > with a limited budget: > > - (must) onboard LAN > - (must) proven Linux (RH7.2) compatibility > - (ideal) micro-atx > - (ideal) wide range of CPU support, i.e. from Duron > sub-GHz up to XP 2600+ > - (bonus) some overclocking capabilities > > In terms of chip > > thanks in advance, > Dr. Kostas Kolokoutsas > > > __________________________________________________ > Yahoo! Plus - For a better Internet experience > http://uk.promotions.yahoo.com/yplus/yoffer.html > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org > To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- Santiago Muelas E.T.S. Ingenieros de Caminos, (U.P.M) Tf.: (34) 91 336 66 59 e-mail: smuelas at mecanica.upm.es Fax: (34) 91 336 67 61 www: http://w3.mecanica.upm.es/~smuelas _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From laurenceliew at yahoo.com.sg Thu Jun 12 02:05:47 2003 From: laurenceliew at yahoo.com.sg (=?iso-8859-1?q?Laurence?=) Date: Thu, 12 Jun 2003 14:05:47 +0800 (CST) Subject: Octave on Beowulf In-Reply-To: Message-ID: <20030612060547.45199.qmail@web11904.mail.yahoo.com> Hi, Octave will run on any linux platform.. so that includes all Beowulf.... I guess the question really is will it run FASTER? I know SciLab has PVM bindings...., for Octave.. I am not sure if there are PVM or MPI bindings... but you may wish to try to use some of the freely available MPI bindings/libraries for MatLab: google Matlab MPI Else.. if you code is not MPI/PVM parallel, that is it is serial code, you can just use PBS/SGE/LSF to submit your Octave jobs to the cluster.... Hope this helps. Cheers! Laurence --- "RANGI, JAI" wrote: > Octave is Mat lab like program but free. > Has any tried to run octave on Beowulf Cluster. > Any hint or link will be appreciated. > Thanks > > Jai Rangi > > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org > To change your subscription (digest mode or > unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf ===== --- Cheers! Laurence __________________________________________________ Do You Yahoo!? Send free SMS from your PC! http://sg.sms.yahoo.com _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From abignone at unige.it Thu Jun 12 05:47:11 2003 From: abignone at unige.it (Franco A. Bignone) Date: Thu, 12 Jun 2003 11:47:11 +0200 Subject: PS2 cluster References: <200306061127.h56BRl505710@NewBlue.Scyld.com> <3EE647CF.44C294D4@unige.it> <20030611161859.T29733@velocet.ca> Message-ID: <3EE84C1E.A7CCD4E4@unige.it> Sorry for the inconvenience, but my mail kept telling me that sending had failed, while sending the message out. So I sent by mistake multiple copies trying to figure out the problem, when I understood what was happening was too late. FB -- ************************************************************************* * Dr. Franco A. Bignone, I.S.T., National Cancer Institute, Lab. Exp. * * Oncology, Lr.go Rosanna Benzi, 10, 16132, Genova, Italy. * * e-mail: franco.bignone at istge.it, abignone at unige.it * * http://gendyn.ist.unige.it http://gendyn.istge.it * * ph. home: +39-010-247-3070 (answ.) * * job: +39-010-5600-213, +39-010-355839, +39-010-5600641, * * fax: +39-010-5600-217 * ************************************************************************* _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From andy at ideafix.litec.csic.es Thu Jun 12 06:21:33 2003 From: andy at ideafix.litec.csic.es (andy) Date: Thu, 12 Jun 2003 11:21:33 +0100 Subject: NAS Parallel Benchmarks for Current Hardware References: <200306111810.WAA01122@nocserv.free.net> Message-ID: <3EE8542D.93CE3B57@ideafix.litec.csic.es> Mikhail, > > I am looking to put together a small cluster for numerical simulation > > and have been surprised at how few NPB benchmark results using current > > hardware I can find via google. > > > It's common situation w/NPB (in opposition to Linpack, SPECcpu e.a.) :-( Unfortunately, I am not sure linpack and spec reliably measure what is important for numerical work on a beowulf. There is some good news. I have got in contact with the group at NASA and they have got some funding to employ someone next week for a 3 month period to get things rolling again with the NPB benchmarks. Hopefully, some useful numbers for current machines should soon start appearing at: www.nas.nasa.gov/Software/NPB _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From collier at cs.widener.edu Wed Jun 11 19:15:30 2003 From: collier at cs.widener.edu (Aaron Collier) Date: Wed, 11 Jun 2003 19:15:30 -0400 (EDT) Subject: Slackware Beowulf Cluster Message-ID: Hello, I have actually been working on a cluster-oriented Linux distribution based on Slackware (actually version 8.1) called WUBCOS (Widener University Beowulf Cluster OS). I have been running the basic distribution on an eight node cluster I built as part of my senior project in computer science and mathematics. Unfortunately, the distribution isn't very refined (but still stable) as I no longer have access to adequate resources to continue development because I just graduated. WUBCOS is currently optimized for my particular hardware but I can tell you the software I have installed at the moment. I compiled a custom kernel and actually enabled hyper-threading (simultaneous multithreading) which works fairly well despite what others may say. I upgraded the kernel from 2.4.18 to 2.4.20 to add proper DMA support and HT support, but Slackware 9 includes kernel 2.4.20 (no source though). Since the member nodes don't have CD-ROM drives I wrote a small network-based remote node installation script that mounts an NFS partition hosted on the main server simply containing a copy of the Slackware installation CD and custom packages I created. I actually had to create a custom root disk set which includes the install script and the necessary nic drivers. I also have been using channel bonding (the default round-robin packet distribution scheme) and haven't had any problems with stability. The cluster uses both MPICH-MPI and LAM-MPI although MPICH has been patched to work more efficiently with Open PBS which is used as the only batch queue system. Also included is a precompiled ATLAS package which has been optimized for the Pentium 4 Xeon with threaded libraries. I also installed PVFS which works well enough. The distro uses NIS to distribute user information, DNS for standard name resolution, and NFS to distribute user home directories and actually /usr/local as well (people will argue this isn't good but it works for me). I also configured NTP because time travel was becoming all too common on the nodes. I also went through all of the system initialization scripts (located in /etc/rc.d) and disabled all unnecessary system servies (like httpd) although these files differ for the head node and the regular compute nodes. Speaking in terms of a general installation, since I had a large hard drive I installed everything (about 2GB) on the head node (actually I removed the sudo package). For the installation of the member nodes I made custom tagfiles so a node installation only requires about 750MB. What you need to install will probably depend upon your anticipated use for the cluster. Configuring Slackware to operate efficiently in a cluster environment can be fairly time consuming depending upon your needs but it is certainly well worth the effort. I am glad someone else recognizes the merits of the Slackware distribution. Please feel free to contact me if you have any additional questions. Bye, Aaron Collier P.S. If anyone would like to hire me then please send me an e-mail :o) _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From josip at lanl.gov Thu Jun 12 12:48:37 2003 From: josip at lanl.gov (Josip Loncaric) Date: Thu, 12 Jun 2003 10:48:37 -0600 Subject: Nice survey of higher end power supplies ... In-Reply-To: References: Message-ID: <3EE8AEE5.2070607@lanl.gov> Donald Becker wrote: > On Wed, 11 Jun 2003, Josip Loncaric wrote: >> >>Power factor is about 0.65 on my own machine. I was kind of surprised >>too. > > > I'm shocked at that low PF. I had believed that most modern power > supplies had some PF correction. Apparently not this one (Antec True430). Power factor this low may be related to the fact that this power supply is a bit oversized. My computer uses only 30-40% of the power supply peak rating. The oversized 430W PS came with the Antec case, but a 350W unit would have been more than adequate. Sincerely, Josip _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From johnh at sjgeophysics.com Thu Jun 12 13:31:38 2003 From: johnh at sjgeophysics.com (John Harrop) Date: 12 Jun 2003 10:31:38 -0700 Subject: Configuring OpenSSH 3.5p1 Message-ID: <1055439098.27276.31.camel@orion-2> I'm currently building a new cluster which unlike the older, more isolated one will be using OpenSSH. How are others configuring their ssh? I've been trying to keep the system in the 'recommended' settings of the O'Reilly SSH book but I'm having trouble getting automatic password authentication. (The passphrase authentication is fine.) For example, ssh-agent handles the passphrase fine, but I can't seem to get /etc/shosts.equiv or .shosts to work with the password. Has the security tightened up on OpenSSH since the book? Assistance would be appreciated - I have a feeling I'm missing something simple in all the ssh options, files and permissions! Cheers, John Harrop Geologist, GIS Manager SJ Geophysics Ltd _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From chaimj at majorlinux.com Thu Jun 12 13:59:19 2003 From: chaimj at majorlinux.com (Major Chai Mee Joon) Date: Fri, 13 Jun 2003 01:59:19 +0800 Subject: ideal motherboard for diskless beowulf In-Reply-To: <200306111521.h5BFLQY28078@NewBlue.Scyld.com> References: <200306111521.h5BFLQY28078@NewBlue.Scyld.com> Message-ID: <20030612175919.GC2630@mail.majorlinux.com> Hi, > On Wed, 11 Jun 2003, kolokoutsas konstantinos wrote: > > > Hello, > > > > I am in the process of designing a 12-node diskless > > cluster. I am going with AMD CPUs, but not quite > > confident which mobo to use, for serving the following > > with a limited budget: > > > > - (must) onboard LAN > > - (must) proven Linux (RH7.2) compatibility > > - (ideal) micro-atx > > - (ideal) wide range of CPU support, i.e. from Duron > > sub-GHz up to XP 2600+ > > - (bonus) some overclocking capabilities You just have to be careful if you're using the the latest boards unless you like to get your hands dirty. eg. Asus A7N8X Deluxe Board based on nVidia's Nforce2-Chipset, as the onboard Nvidia Ethernet requires the driver supplied by nVidia. Regards, Major Chai MajorLinux.com _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rgb at phy.duke.edu Thu Jun 12 14:11:41 2003 From: rgb at phy.duke.edu (Robert G. Brown) Date: Thu, 12 Jun 2003 14:11:41 -0400 (EDT) Subject: Configuring OpenSSH 3.5p1 In-Reply-To: <1055439098.27276.31.camel@orion-2> Message-ID: On 12 Jun 2003, John Harrop wrote: > I'm currently building a new cluster which unlike the older, more > isolated one will be using OpenSSH. How are others configuring their > ssh? > > I've been trying to keep the system in the 'recommended' settings of the > O'Reilly SSH book but I'm having trouble getting automatic password > authentication. (The passphrase authentication is fine.) For example, > ssh-agent handles the passphrase fine, but I can't seem to get > /etc/shosts.equiv or .shosts to work with the password. > > Has the security tightened up on OpenSSH since the book? Assistance > would be appreciated - I have a feeling I'm missing something simple in > all the ssh options, files and permissions! Unfortunately there are several places where authentication can occur (or not) in an ssh connection. Running ssh -v when trying can often help figure out what fails. I'm not quite clear on what you mean by "getting automatic password authentication". Ssh/sshd usually come preconfigured in a mode that requires that you enter your password when ssh-connecting to any host running the sshd. To configure it so that you DON'T need a password to run a remote shell command (for example so you can do things like): rgb at lilith|T:104>ssh ganesh ls -ald .ssh drwxr-xr-x 3 rgb prof 4096 Nov 7 2002 .ssh you need to generate a key pair. The tool for this is ssh-keygen, and you can create either RSA (ssh 1) or DSA (ssh 2) keys. One place you COULD be failing is that sshd can be configured to permit logins using keypairs or not. Read man sshd_config. In most versions of ssh these days, I'm pretty sure .shosts and /etc/shosts.equiv is pretty much ignored in favor of the much stronger rsa/dsa host identification. I enclose below a trace of ssh -v in a successful remote login with no password. Note that it gets down to where it needs to authenticate, announces that it can do so via either publickey (a keypair) or a password (where it has already validated the host identity) and then uses the publickey successfully. HTH, rgb -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb at phy.duke.edu rgb at lilith|T:109>ssh -v ganesh OpenSSH_3.5p1, SSH protocols 1.5/2.0, OpenSSL 0x0090701f debug1: Reading configuration data /etc/ssh/ssh_config debug1: Applying options for * debug1: Rhosts Authentication disabled, originating port will not be trusted. debug1: ssh_connect: needpriv 0 debug1: Connecting to ganesh [152.3.182.51] port 22. debug1: Connection established. debug1: identity file /home/rgb/.ssh/identity type 0 debug1: identity file /home/rgb/.ssh/id_rsa type -1 debug1: identity file /home/rgb/.ssh/id_dsa type 2 debug1: Remote protocol version 1.99, remote software version OpenSSH_3.1p1 debug1: match: OpenSSH_3.1p1 pat OpenSSH_2.*,OpenSSH_3.0*,OpenSSH_3.1* debug1: Enabling compatibility mode for protocol 2.0 debug1: Local version string SSH-2.0-OpenSSH_3.5p1 debug1: SSH2_MSG_KEXINIT sent debug1: SSH2_MSG_KEXINIT received debug1: kex: server->client aes128-cbc hmac-md5 none debug1: kex: client->server aes128-cbc hmac-md5 none debug1: SSH2_MSG_KEX_DH_GEX_REQUEST sent debug1: expecting SSH2_MSG_KEX_DH_GEX_GROUP debug1: dh_gen_key: priv key bits set: 127/256 debug1: bits set: 1577/3191 debug1: SSH2_MSG_KEX_DH_GEX_INIT sent debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY debug1: Host 'ganesh' is known and matches the RSA host key. debug1: Found key in /home/rgb/.ssh/known_hosts2:1 debug1: bits set: 1617/3191 debug1: ssh_rsa_verify: signature correct debug1: kex_derive_keys debug1: newkeys: mode 1 debug1: SSH2_MSG_NEWKEYS sent debug1: waiting for SSH2_MSG_NEWKEYS debug1: newkeys: mode 0 debug1: SSH2_MSG_NEWKEYS received debug1: done: ssh_kex2. debug1: send SSH2_MSG_SERVICE_REQUEST debug1: service_accept: ssh-userauth debug1: got SSH2_MSG_SERVICE_ACCEPT debug1: authentications that can continue: publickey,password debug1: next auth method to try is publickey debug1: try privkey: /home/rgb/.ssh/id_rsa debug1: try pubkey: /home/rgb/.ssh/id_dsa debug1: input_userauth_pk_ok: pkalg ssh-dss blen 435 lastkey 0x808bb10 hint 2 debug1: read PEM private key done: type DSA debug1: ssh-userauth2 successful: method publickey debug1: channel 0: new [client-session] debug1: send channel open 0 debug1: Entering interactive session. debug1: ssh_session2_setup: id 0 debug1: channel request 0: pty-req debug1: Requesting X11 forwarding with authentication spoofing. debug1: channel request 0: x11-req debug1: channel request 0: shell debug1: fd 3 setting TCP_NODELAY debug1: channel 0: open confirm rwindow 0 rmax 32768 Last login: Thu Jun 12 13:56:58 2003 from rgb.adsl.duke.edu rgb at ganesh|T:101> _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From widyono at cis.upenn.edu Thu Jun 12 14:28:44 2003 From: widyono at cis.upenn.edu (Daniel Widyono) Date: Thu, 12 Jun 2003 14:28:44 -0400 Subject: Configuring OpenSSH 3.5p1 In-Reply-To: References: <1055439098.27276.31.camel@orion-2> Message-ID: <20030612182844.GA14762@central.cis.upenn.edu> .shosts and shosts.equiv are disabled for root (check the source, last I checked it was hard-coded with no option). Are you having trouble with root, and are able to use .shosts or shosts.equiv for any other account? Regards, Dan W. On Thu, Jun 12, 2003 at 02:11:41PM -0400, Robert G. Brown wrote: > On 12 Jun 2003, John Harrop wrote: > > ssh-agent handles the passphrase fine, but I can't seem to get > > /etc/shosts.equiv or .shosts to work with the password. > In most versions of ssh these days, I'm pretty sure .shosts and > /etc/shosts.equiv is pretty much ignored in favor of the much stronger > rsa/dsa host identification. -- -- Daniel Widyono http://www.cis.upenn.edu/~widyono -- Liniac Project, CIS Dept., SEAS, University of Pennsylvania -- Mail: CIS Dept, 302 Levine 3330 Walnut St Philadelphia, PA 19104 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From bryan at ayesha.phys.Virginia.EDU Thu Jun 12 16:20:43 2003 From: bryan at ayesha.phys.Virginia.EDU (Bryan K. Wright) Date: Thu, 12 Jun 2003 16:20:43 -0400 Subject: Data corruption w/ tg3 and bcm5700 drivers In-Reply-To: Message from "Bryan K. Wright" of "Tue, 03 Jun 2003 10:36:24 EDT." <200306031436.h53EaPJ29643@ayesha.phys.Virginia.EDU> Message-ID: <200306122020.h5CKKhx27905@ayesha.phys.Virginia.EDU> Hi folks, (A recap for those who weren't tuned in earlier: I've been having data-corruption problems with Broadcom-based 3com Gigabit cards.) Here's an interesting new datum: The cards were previously in a 64-bit PCI slot, but they'll also work in a 32-bit slot. Moving them to the 32-bit slot apparently gets rid of the data corruption problem (we're currently using the bcm5700 driver.) Bryan -- =============================================================================== Bryan Wright |"If you take cranberries and stew them like Physics Department | applesauce, they taste much more like prunes University of Virginia | than rhubarb does." -- Groucho Charlottesville, VA 22901 | (434) 924-7218 | bryan at virginia.edu =============================================================================== _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From hahn at physics.mcmaster.ca Thu Jun 12 18:28:00 2003 From: hahn at physics.mcmaster.ca (Mark Hahn) Date: Thu, 12 Jun 2003 18:28:00 -0400 (EDT) Subject: ideal motherboard for diskless beowulf In-Reply-To: Message-ID: > > I am in the process of designing a 12-node diskless > > cluster. I am going with AMD CPUs, but not quite why are you fixed on AMD? in my opinion, there are several very important features which are scarce in the Athlon world: - fast memory. the basic dual-Xeon now has 2xpc2100 ram, which is critically important to a lot of codes. Opterons are a whole different story, of course, but then again, uniprocessor 6.4 GB/s P4c's are, too. don't even get me started about quad Itanium2's ;) - onboard, PXE-enabled gigabit, preferably two. for quite a while, Intel didn't have the best smell in the Linux world when it came to NICs, but the e1000 seems quite fine now. - PCIX - fast implementations of PCI are critical for getting serious cluster interconnect to perform. course, "serious" interconnect means big bucks, which makes this relevant only for certain categories of cluster. > > confident which mobo to use, for serving the following > > with a limited budget: > > > > - (must) onboard LAN are there *any* athlon boards with gigabit onboard? some, I guess. > > - (must) proven Linux (RH7.2) compatibility that's not really saying much - I think you'd be fine with anything recent from VIA, and I think SiS support has gotten better recently. of course, AMD still makes the only dual-athlon chipset, right? choosing from among the top 5 motherboard vendors is a good way to get decent compatibility... > > - (ideal) micro-atx hmm. certainly possible, but that puts you into the small/cheap category of clusters (as opposed to the large and probably more expensive, also known as "server"). u-atx just so it fits in small (cheap) cases? > > - (ideal) wide range of CPU support, i.e. from Duron > > sub-GHz up to XP 2600+ that shouldn't be a problem. though I'm not sure it would make sense to build a cluster that mixed CPUs like that, or one that went with Durons (simply because Athlon's at their sweetspot offer quite a bit better price/performance.) > > - (bonus) some overclocking capabilities just say no. seriously. it's hard to imagine why you'd want a compute cluster if you couldn't trust the results. > UP or MP? MP can be a hair cheaper per CPU, if your tasks aren't far be it from be to disagree with rgb, but I'm not so sure about this. at the very least, you'd have to add that it depends on the number of duplicated components for UP. it's clear that MP boards tend to cost 2-4x as much as uni, and MP chips are noticably more expensive as well. so it boils down to an extra disk, PS, case, switch port, maybe video card. (assuming you have the same ram-per-cpu in either case.) > fiasco) been generally satisfied with them. I use Tyan's (e.g. Tyan > 2466 or better) for duals -- they have onboard 100BT or better (current > motherboards have dual GigE NICs onboard, as well as video). oh, cool, I hadn't looked at Tyan dual-athlon boards recently - they've got both 64x66 and gigabit. > from 7-9. Beware motherboards with onboard RTL 8139 NICs -- they are > cheap, common, and suck. I'd plan on adding a decent NIC a la carte if > I "had" to pick a motherboard with an onboard RTL. 8139's are fairly wimpy, but then again, 100bT is pretty miserable - do you really worry about whether your 8139 is consuming a few more cycles than a smarter NIC would, given that it's only doing ~9 MB/s? _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From Mark at markandrewsmith.co.uk Thu Jun 12 19:01:54 2003 From: Mark at markandrewsmith.co.uk (Mark Andrew Smith) Date: Fri, 13 Jun 2003 00:01:54 +0100 Subject: Data corruption w/ tg3 and bcm5700 drivers In-Reply-To: <200306122020.h5CKKhx27905@ayesha.phys.Virginia.EDU> Message-ID: May I ask what motherboards (and possibly BIOS) are you using? Cheers, Mark. -----Original Message----- From: beowulf-admin at scyld.com [mailto:beowulf-admin at scyld.com]On Behalf Of Bryan K. Wright Sent: 12 June 2003 21:21 To: beowulf at beowulf.org Cc: bryan at ayesha.phys.Virginia.EDU Subject: Re: Data corruption w/ tg3 and bcm5700 drivers Hi folks, (A recap for those who weren't tuned in earlier: I've been having data-corruption problems with Broadcom-based 3com Gigabit cards.) Here's an interesting new datum: The cards were previously in a 64-bit PCI slot, but they'll also work in a 32-bit slot. Moving them to the 32-bit slot apparently gets rid of the data corruption problem (we're currently using the bcm5700 driver.) Bryan -- ============================================================================ === Bryan Wright |"If you take cranberries and stew them like Physics Department | applesauce, they taste much more like prunes University of Virginia | than rhubarb does." -- Groucho Charlottesville, VA 22901 | (434) 924-7218 | bryan at virginia.edu ============================================================================ === _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf This email has been scanned for viruses by NetBenefit using Sophos anti-virus technology This email has been scanned for viruses by NetBenefit using Sophos anti-virus technology _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From smuelas at mecanica.upm.es Fri Jun 13 02:31:11 2003 From: smuelas at mecanica.upm.es (Santiago Muelas) Date: Fri, 13 Jun 2003 08:31:11 +0200 Subject: Configuring OpenSSH 3.5p1 In-Reply-To: References: <1055439098.27276.31.camel@orion-2> Message-ID: <20030613083111.1f0f6394.smuelas@mecanica.upm.es> I'm running RedHat-9. Everything works perfectly with the simple generation of the key (Using ssh-keygen and putting the result on directory .ssh). Very clear explained on "man ssh". Nothing more to be done. On Thu, 12 Jun 2003 14:11:41 -0400 (EDT) "Robert G. Brown" wrote: > On 12 Jun 2003, John Harrop wrote: > > > I'm currently building a new cluster which unlike the older, more > > isolated one will be using OpenSSH. How are others configuring their > > ssh? > > > > I've been trying to keep the system in the 'recommended' settings of the > > O'Reilly SSH book but I'm having trouble getting automatic password > > authentication. (The passphrase authentication is fine.) For example, > > ssh-agent handles the passphrase fine, but I can't seem to get > > /etc/shosts.equiv or .shosts to work with the password. > > > > Has the security tightened up on OpenSSH since the book? Assistance > > would be appreciated - I have a feeling I'm missing something simple in > > all the ssh options, files and permissions! > > Unfortunately there are several places where authentication can occur > (or not) in an ssh connection. Running ssh -v when trying can often > help figure out what fails. > > I'm not quite clear on what you mean by "getting automatic password > authentication". Ssh/sshd usually come preconfigured in a mode that > requires that you enter your password when ssh-connecting to any host > running the sshd. To configure it so that you DON'T need a password to > run a remote shell command (for example so you can do things like): > > rgb at lilith|T:104>ssh ganesh ls -ald .ssh > drwxr-xr-x 3 rgb prof 4096 Nov 7 2002 .ssh > > you need to generate a key pair. The tool for this is ssh-keygen, and > you can create either RSA (ssh 1) or DSA (ssh 2) keys. One place you > COULD be failing is that sshd can be configured to permit logins using > keypairs or not. Read man sshd_config. > > In most versions of ssh these days, I'm pretty sure .shosts and > /etc/shosts.equiv is pretty much ignored in favor of the much stronger > rsa/dsa host identification. > > I enclose below a trace of ssh -v in a successful remote login with no > password. Note that it gets down to where it needs to authenticate, > announces that it can do so via either publickey (a keypair) or a > password (where it has already validated the host identity) and then > uses the publickey successfully. > > HTH, > > rgb > > -- > Robert G. Brown http://www.phy.duke.edu/~rgb/ > Duke University Dept. of Physics, Box 90305 > Durham, N.C. 27708-0305 > Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb at phy.duke.edu > > > rgb at lilith|T:109>ssh -v ganesh > OpenSSH_3.5p1, SSH protocols 1.5/2.0, OpenSSL 0x0090701f > debug1: Reading configuration data /etc/ssh/ssh_config > debug1: Applying options for * > debug1: Rhosts Authentication disabled, originating port will not be > trusted. > debug1: ssh_connect: needpriv 0 > debug1: Connecting to ganesh [152.3.182.51] port 22. > debug1: Connection established. > debug1: identity file /home/rgb/.ssh/identity type 0 > debug1: identity file /home/rgb/.ssh/id_rsa type -1 > debug1: identity file /home/rgb/.ssh/id_dsa type 2 > debug1: Remote protocol version 1.99, remote software version > OpenSSH_3.1p1 > debug1: match: OpenSSH_3.1p1 pat OpenSSH_2.*,OpenSSH_3.0*,OpenSSH_3.1* > debug1: Enabling compatibility mode for protocol 2.0 > debug1: Local version string SSH-2.0-OpenSSH_3.5p1 > debug1: SSH2_MSG_KEXINIT sent > debug1: SSH2_MSG_KEXINIT received > debug1: kex: server->client aes128-cbc hmac-md5 none > debug1: kex: client->server aes128-cbc hmac-md5 none > debug1: SSH2_MSG_KEX_DH_GEX_REQUEST sent > debug1: expecting SSH2_MSG_KEX_DH_GEX_GROUP > debug1: dh_gen_key: priv key bits set: 127/256 > debug1: bits set: 1577/3191 > debug1: SSH2_MSG_KEX_DH_GEX_INIT sent > debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY > debug1: Host 'ganesh' is known and matches the RSA host key. > debug1: Found key in /home/rgb/.ssh/known_hosts2:1 > debug1: bits set: 1617/3191 > debug1: ssh_rsa_verify: signature correct > debug1: kex_derive_keys > debug1: newkeys: mode 1 > debug1: SSH2_MSG_NEWKEYS sent > debug1: waiting for SSH2_MSG_NEWKEYS > debug1: newkeys: mode 0 > debug1: SSH2_MSG_NEWKEYS received > debug1: done: ssh_kex2. > debug1: send SSH2_MSG_SERVICE_REQUEST > debug1: service_accept: ssh-userauth > debug1: got SSH2_MSG_SERVICE_ACCEPT > debug1: authentications that can continue: publickey,password > debug1: next auth method to try is publickey > debug1: try privkey: /home/rgb/.ssh/id_rsa > debug1: try pubkey: /home/rgb/.ssh/id_dsa > debug1: input_userauth_pk_ok: pkalg ssh-dss blen 435 lastkey 0x808bb10 > hint 2 > debug1: read PEM private key done: type DSA > debug1: ssh-userauth2 successful: method publickey > debug1: channel 0: new [client-session] > debug1: send channel open 0 > debug1: Entering interactive session. > debug1: ssh_session2_setup: id 0 > debug1: channel request 0: pty-req > debug1: Requesting X11 forwarding with authentication spoofing. > debug1: channel request 0: x11-req > debug1: channel request 0: shell > debug1: fd 3 setting TCP_NODELAY > debug1: channel 0: open confirm rwindow 0 rmax 32768 > Last login: Thu Jun 12 13:56:58 2003 from rgb.adsl.duke.edu > rgb at ganesh|T:101> > > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org > To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- Santiago Muelas Profesor de Resistencia de Materiales y C?lculo de Estructuras ETSI de Caminos, Canales y Puertos (U.P.M) smuelas at mecanica.upm.es http://w3.mecanica.upm.es/~smuelas _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From smuelas at mecanica.upm.es Fri Jun 13 02:32:27 2003 From: smuelas at mecanica.upm.es (Santiago Muelas) Date: Fri, 13 Jun 2003 08:32:27 +0200 Subject: ideal motherboard for diskless beowulf In-Reply-To: <20030612175919.GC2630@mail.majorlinux.com> References: <200306111521.h5BFLQY28078@NewBlue.Scyld.com> <20030612175919.GC2630@mail.majorlinux.com> Message-ID: <20030613083227.6088fc60.smuelas@mecanica.upm.es> Also the last Shuttle, but it is something very simple to install.... On Fri, 13 Jun 2003 01:59:19 +0800 Major Chai Mee Joon wrote: > Hi, > > > On Wed, 11 Jun 2003, kolokoutsas konstantinos wrote: > > > > > Hello, > > > > > > I am in the process of designing a 12-node diskless > > > cluster. I am going with AMD CPUs, but not quite > > > confident which mobo to use, for serving the following > > > with a limited budget: > > > > > > - (must) onboard LAN > > > - (must) proven Linux (RH7.2) compatibility > > > - (ideal) micro-atx > > > - (ideal) wide range of CPU support, i.e. from Duron > > > sub-GHz up to XP 2600+ > > > - (bonus) some overclocking capabilities > > You just have to be careful if you're using the the latest boards > unless you like to get your hands dirty. > > eg. Asus A7N8X Deluxe Board based on nVidia's Nforce2-Chipset, > as the onboard Nvidia Ethernet requires the driver supplied by nVidia. > > > Regards, > > Major Chai > > MajorLinux.com > > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org > To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- Santiago Muelas Profesor de Resistencia de Materiales y C?lculo de Estructuras ETSI de Caminos, Canales y Puertos (U.P.M) smuelas at mecanica.upm.es http://w3.mecanica.upm.es/~smuelas _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From julien.leduc at imag.fr Fri Jun 13 09:14:01 2003 From: julien.leduc at imag.fr (Julien Leduc) Date: Fri, 13 Jun 2003 13:14:01 +0000 Subject: Configuring OpenSSH 3.5p1 In-Reply-To: <1055439098.27276.31.camel@orion-2> References: <1055439098.27276.31.camel@orion-2> Message-ID: <20030613131401.4967e2d7.julien.leduc@imag.fr> On 12 Jun 2003 10:31:38 -0700 John Harrop wrote: > I'm currently building a new cluster which unlike the older, more > isolated one will be using OpenSSH. How are others configuring their > ssh? He we have a 121 nodes cluster. The main server, that handle open PBS server and NIS server, is due to perform some administration tasks through SSH (with rshp/mput tools that allow good perfs over ssh admin tasks such as running remote scripts or copying files efficienltly over the cluster), so here we need to ssh on every node without entering a password. To do this, we have 2 pairs of dsa keys (generated with ssh-keygen -t dsa) for root: one for the server and another that is common to all the computing nodes, and we allow the root ssh login from the server on all the nodes by copying his public dsa key in the authorized_keys file (so all you have to do is: scp /root/.ssh/id_dsa.pub root at node_without_pass:/root/.ssh/authorized_keys the first time, you will have to type the pass but then it works :) ). The main problem you will face is how to spread the authorized keys file on all the nodes (If you have a lot of nodes...). Julien Leduc Laboratoire ID-IMAG _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From chris_oubre at hotmail.com Fri Jun 13 15:17:01 2003 From: chris_oubre at hotmail.com (Chris Oubre) Date: Fri, 13 Jun 2003 14:17:01 -0500 Subject: Opteron Large Memory Compilers? Message-ID: <001e01c331e0$5e30fd10$25462a80@rice.edu> Hello, We are looking into the feasibility of using Opterons in on cluster. One major issue we had when we benchmarked on them was the lack of large memory support in the compilers we used. We tried to run large memory jobs on an Opteron using the Portland Compilers and found that it would not compile jobs larger than 2 Gigs. We contacted the Portland Group and they responded that it was a know bug. We haven't tried again in a while and we were wondering if anyone knows if this has been fixed? Has anyone run jobs > 2 Gigs on a Opteron? And if so which compiler did you use. And finally what is the current consensus on which Opteron compiler is best for large memory jobs. Thank you. **************************************************** Christopher D. Oubre * email: chris_oubre at hotmail.com * research: http://cmt.rice.edu/~coubre * Web: http://www.angelfire.com/la2/oubre * Hangout: http://pub44.ezboard.com/bsouthterrebonne * Phone:(713)348-3541 Fax: (713)348-4150 * Rice University * Department of Physics, M.S. 61 * 6100 Main St. ^-^ * Houston, Tx 77251-1892, USA (O O) * -= Phlax=- ( v ) * ************************************m*m************* _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From ctibirna at giref.ulaval.ca Fri Jun 13 16:06:15 2003 From: ctibirna at giref.ulaval.ca (Cristian Tibirna) Date: Fri, 13 Jun 2003 16:06:15 -0400 Subject: strange behavior. Hints? Message-ID: <200306131606.15570.ctibirna@giref.ulaval.ca> Hello I have a small (16x(2P-IV / 4G RAM / myri), redhat-7.3 with xCAT) cluster that we use with good results since a few months. I am about to prepare some reports about speedups of our in-house FEM code (parallelised with MPICH-GM and using PETSc for solvers) and I'm doing some tests which consist mostly of launching a same (rather big) FEM simulation on decreasing number of nodes: for n in `seq -f"%02d" 16 1`; do mpirun -np $n ./simulator; done (OK, the script is a bit more complicated than this, but you get the idea). A strange phenomenon started to appear a few weeks ago. The simulation works very well for all n, apart n=10 and n=11. For these, the program segfaults on 2 to 5 of the nodes and of course this locks the execution (MPI waits) and I have to kill it. This is reliably reproductible. I'm absolutely sure my code has no special code dealing with the number of nodes (highly generalised OOP C++ code). Now, I start to believe there's some strange bug in the Myrinet hardware/software. But I feel this is a really wild guess. I plan to start investigation by tearing all the components apart (mpi, petsc, myrinet drivers) and test them again. But this is a really big battle with little chances of success, given that I can positively see all working correctly most of the time (i.e., running on 16, 15, 4, 2 etc. nodes works OK). I wonder if anybody saw such a behavior before and has some (more valuable) hints (than my wild guesses) for where to look and how to do it, eventually. Thanks a lot for your attention. -- Cristian Tibirna (1-418-) 656-2131 / 4340 Laval University - Quebec, CAN ... http://www.giref.ulaval.ca/~ctibirna Research profesional at GIREF ... ctibirna at giref.ulaval.ca PhD Student - Chemical Engng ... tibirna at gch.ulaval.ca _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From patrick at myri.com Fri Jun 13 16:58:53 2003 From: patrick at myri.com (Patrick Geoffray) Date: 13 Jun 2003 16:58:53 -0400 Subject: strange behavior. Hints? In-Reply-To: <200306131606.15570.ctibirna@giref.ulaval.ca> References: <200306131606.15570.ctibirna@giref.ulaval.ca> Message-ID: <1055537934.573.184.camel@asterix> Hi Christian, On Fri, 2003-06-13 at 16:06, Cristian Tibirna wrote: > Now, I start to believe there's some strange bug in the Myrinet > hardware/software. But I feel this is a really wild guess. Open an help ticket with help at myri.com and we will investigate as soon as possible. If your application segfaults, you can recompile MPICH-GM and your application with "-O0 -g" and run it under a debugger (not a problem under gdb for 10 processes). Once we know where it segfaults (from the gdb trace), we can find why. Regards Patrick -- Patrick Geoffray Myricom, Inc. http://www.myri.com _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From v_454 at yahoo.com Fri Jun 13 15:51:44 2003 From: v_454 at yahoo.com (Steve Elliot) Date: Fri, 13 Jun 2003 12:51:44 -0700 (PDT) Subject: Large File Support for RedHat Linux 8.0 Kernal 2.4.18-14 Message-ID: <20030613195144.41888.qmail@web40505.mail.yahoo.com> Our RedHat 8.0 doesn't seem to support Large file system out of the box. We have a testing program running on an ext3 file system, with both _LARGEFILE_SOURCE and _FILE_OFFSET_BITS=64 defined, the lseek() function failed when the file pointer goes beyound 2G. What's the steps to turn the large file support on? and how do I know it is on? __________________________________ Do you Yahoo!? Yahoo! Calendar - Free online calendar with sync to Outlook(TM). http://calendar.yahoo.com _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From tlovie at pokey.mine.nu Fri Jun 13 17:19:41 2003 From: tlovie at pokey.mine.nu (Thomas Lovie) Date: Fri, 13 Jun 2003 17:19:41 -0400 Subject: Large File Support for RedHat Linux 8.0 Kernal 2.4.18-14 In-Reply-To: <20030613195144.41888.qmail@web40505.mail.yahoo.com> Message-ID: <000301c331f1$83f189f0$1806a8c0@fishnet.exigentsi.com> I might be mistaken here, but I didn't think that ext3 could support large files. I thought you have to go to XFS or reiserfs. Tom. -----Original Message----- From: beowulf-admin at scyld.com [mailto:beowulf-admin at scyld.com] On Behalf Of Steve Elliot Sent: Friday, June 13, 2003 3:52 PM To: beowulf at beowulf.org Subject: Large File Support for RedHat Linux 8.0 Kernal 2.4.18-14 Our RedHat 8.0 doesn't seem to support Large file system out of the box. We have a testing program running on an ext3 file system, with both _LARGEFILE_SOURCE and _FILE_OFFSET_BITS=64 defined, the lseek() function failed when the file pointer goes beyound 2G. What's the steps to turn the large file support on? and how do I know it is on? __________________________________ Do you Yahoo!? Yahoo! Calendar - Free online calendar with sync to Outlook(TM). http://calendar.yahoo.com _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From joe.griffin at mscsoftware.com Fri Jun 13 17:25:43 2003 From: joe.griffin at mscsoftware.com (Joe Griffin) Date: Fri, 13 Jun 2003 14:25:43 -0700 Subject: Large File Support for RedHat Linux 8.0 Kernal 2.4.18-14 In-Reply-To: <20030613195144.41888.qmail@web40505.mail.yahoo.com> References: <20030613195144.41888.qmail@web40505.mail.yahoo.com> Message-ID: <3EEA4157.8030101@mscsoftware.com> My RedHat 8.0 system writes files larger than 8 Gb. everest <106> cat bigwrite.f program bigwrite integer buf(1024) do i=1,1024 buf(i)=i enddo open( unit=21, access='direct', form='unformatted', 1 status='unknown', recl=4096, err=700 ) irec=1 c do i=1,524288 c do i=1,2*524288 do i=1,4*524288 irec=i c write(6,*) 'writing, irec =', irec write(21, rec=irec, err=800) buf enddo write(6,*) 'write done, irec =', irec close( unit=21 ) stop 700 continue write(6,*) 'open err' stop 800 continue write(6,*) 'write err, irec=', irec close( unit=21 ) end everest <107> ifc bigwrite.f program BIGWRITE 26 Lines Compiled everest <108> a.out write done, irec = 2097152 everest <109> ls -ltr fort.21 -rw-r--r-- 1 jjg develop 8589934592 Jun 13 14:27 fort.21 My file system is ext2. Regards, Joe Griffin Steve Elliot wrote: >Our RedHat 8.0 doesn't seem to support Large file >system out of the box. We have a testing program >running on an ext3 file system, with both >_LARGEFILE_SOURCE and _FILE_OFFSET_BITS=64 defined, >the lseek() function failed when the file pointer goes >beyound 2G. What's the steps to turn the large file >support on? and how do I know it is on? > >__________________________________ >Do you Yahoo!? >Yahoo! Calendar - Free online calendar with sync to Outlook(TM). >http://calendar.yahoo.com >_______________________________________________ >Beowulf mailing list, Beowulf at beowulf.org >To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf > > > > _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From jim at ks.uiuc.edu Fri Jun 13 17:49:21 2003 From: jim at ks.uiuc.edu (Jim Phillips) Date: Fri, 13 Jun 2003 16:49:21 -0500 (CDT) Subject: Large File Support for RedHat Linux 8.0 Kernal 2.4.18-14 In-Reply-To: <3EEA4157.8030101@mscsoftware.com> Message-ID: Hi, You also need to add the O_LARGEFILE flag to your open() call. -Jim > Steve Elliot wrote: > > >Our RedHat 8.0 doesn't seem to support Large file > >system out of the box. We have a testing program > >running on an ext3 file system, with both > >_LARGEFILE_SOURCE and _FILE_OFFSET_BITS=64 defined, > >the lseek() function failed when the file pointer goes > >beyound 2G. What's the steps to turn the large file > >support on? and how do I know it is on? _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From becker at scyld.com Fri Jun 13 16:56:28 2003 From: becker at scyld.com (Donald Becker) Date: Fri, 13 Jun 2003 16:56:28 -0400 (EDT) Subject: Opteron Large Memory Compilers? In-Reply-To: <001e01c331e0$5e30fd10$25462a80@rice.edu> Message-ID: On Fri, 13 Jun 2003, Chris Oubre wrote: > We are looking into the feasibility of using Opterons in on > cluster. One major issue we had when we benchmarked on them was the > lack of large memory support in the compilers we used. We tried to run > large memory jobs on an Opteron using the Portland Compilers and found > that it would not compile jobs larger than 2 Gigs. Could you please clarify? Does the compiler not compile and link an executable larger than 2GB, or Does the program fail when dynamic libraries put the total size >2GB, or Does the executable not work with a data set larger than 2GB? It must be the last, since even the most bloated programs don't go much over 100MB of compiled code. > Has anyone run jobs > 2 Gigs on a Opteron? And if so which > compiler did you use. And finally what is the current consensus on which > Opteron compiler is best for large memory jobs. -- Donald Becker becker at scyld.com Scyld Computing Corporation http://www.scyld.com 914 Bay Ridge Road, Suite 220 Scyld Beowulf cluster system Annapolis MD 21403 410-990-9993 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From glen at cert.ucr.edu Fri Jun 13 18:38:50 2003 From: glen at cert.ucr.edu (Glen Kaukola) Date: Fri, 13 Jun 2003 15:38:50 -0700 Subject: Large File Support for RedHat Linux 8.0 Kernal 2.4.18-14 In-Reply-To: <20030613195144.41888.qmail@web40505.mail.yahoo.com> References: <20030613195144.41888.qmail@web40505.mail.yahoo.com> Message-ID: <3EEA527A.1030303@cert.ucr.edu> Steve Elliot wrote: >Our RedHat 8.0 doesn't seem to support Large file >system out of the box. We have a testing program >running on an ext3 file system, with both >_LARGEFILE_SOURCE and _FILE_OFFSET_BITS=64 defined, >the lseek() function failed when the file pointer goes >beyound 2G. What's the steps to turn the large file >support on? and how do I know it is on? > We had a similar problem, and it turned to be tcsh. tcsh doesn't support files over 2GB it seems, so we ended up having to convert all of our scripts over to bash. Glen _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From landman at scalableinformatics.com Fri Jun 13 20:50:11 2003 From: landman at scalableinformatics.com (Joseph Landman) Date: 13 Jun 2003 20:50:11 -0400 Subject: Large File Support for RedHat Linux 8.0 Kernal 2.4.18-14 In-Reply-To: <3EEA527A.1030303@cert.ucr.edu> References: <20030613195144.41888.qmail@web40505.mail.yahoo.com> <3EEA527A.1030303@cert.ucr.edu> Message-ID: <1055551811.4661.67.camel@protein.scalableinformatics.com> On Fri, 2003-06-13 at 18:38, Glen Kaukola wrote: > We had a similar problem, and it turned to be tcsh. tcsh doesn't > support files over 2GB it seems, so we ended up having to convert all of > our scripts over to bash. I simply pulled down the latest TCSH sources, and recompiled for large file support. I replaced the installed tcsh with the compiled tcsh. See http://bioinformatics.org/pipermail/biodevelopers/2002-May/000026.html This should be a far simpler solution than converting all your shell scripts. It would introduce far fewer bugs I would think, than a port. Hopefully the distro vendors (RH and others) in question will catch the hint and make sure that half of their distro is not missing the right compilation options. Shells, editors, file managers, etc all need this. -- Joseph Landman, Ph.D Scalable Informatics LLC email: landman at scalableinformatics.com web: http://scalableinformatics.com phone: +1 734 612 4615 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From johnh at sjgeophysics.com Sat Jun 14 04:18:25 2003 From: johnh at sjgeophysics.com (John Harrop) Date: Sat, 14 Jun 2003 01:18:25 -0700 Subject: Configuring OpenSSH 3.5p1 In-Reply-To: <20030613083111.1f0f6394.smuelas@mecanica.upm.es> References: <1055439098.27276.31.camel@orion-2> <20030613083111.1f0f6394.smuelas@mecanica.upm.es> Message-ID: <1055578706.1991.14.camel@cr517045-a> Thanks for the suggestions and comments. You got me thinking on some new lines that solved the problem. Actually there were two problems: One of my keys had been damaged or changed so that caused some problems. I also found out that the $HOME/.ssh directory must have no greater than 744 permissions (and I think it installs as 766). I recall being warned about this before - but its not as obvious in the docs as permission requirements of some files. The new system I'm working on is also running on RedHat 9. I'm glad to hear I'm not the only one using it. Cheers, John Harrop Geologist GIS and Software Development Manager SJ Geophysics Ltd On Thu, 2003-06-12 at 23:31, Santiago Muelas wrote: > I'm running RedHat-9. Everything works perfectly with the simple generation of the key (Using ssh-keygen and putting the result on directory .ssh). Very clear explained on "man ssh". Nothing more to be done. > > > > > On Thu, 12 Jun 2003 14:11:41 -0400 (EDT) > "Robert G. Brown" wrote: > > > On 12 Jun 2003, John Harrop wrote: > > > > > I'm currently building a new cluster which unlike the older, more > > > isolated one will be using OpenSSH. How are others configuring their > > > ssh? > > > > > > I've been trying to keep the system in the 'recommended' settings of the > > > O'Reilly SSH book but I'm having trouble getting automatic password > > > authentication. (The passphrase authentication is fine.) For example, > > > ssh-agent handles the passphrase fine, but I can't seem to get > > > /etc/shosts.equiv or .shosts to work with the password. > > > > > > Has the security tightened up on OpenSSH since the book? Assistance > > > would be appreciated - I have a feeling I'm missing something simple in > > > all the ssh options, files and permissions! > > > > Unfortunately there are several places where authentication can occur > > (or not) in an ssh connection. Running ssh -v when trying can often > > help figure out what fails. > > > > I'm not quite clear on what you mean by "getting automatic password > > authentication". Ssh/sshd usually come preconfigured in a mode that > > requires that you enter your password when ssh-connecting to any host > > running the sshd. To configure it so that you DON'T need a password to > > run a remote shell command (for example so you can do things like): > > > > rgb at lilith|T:104>ssh ganesh ls -ald .ssh > > drwxr-xr-x 3 rgb prof 4096 Nov 7 2002 .ssh > > > > you need to generate a key pair. The tool for this is ssh-keygen, and > > you can create either RSA (ssh 1) or DSA (ssh 2) keys. One place you > > COULD be failing is that sshd can be configured to permit logins using > > keypairs or not. Read man sshd_config. > > > > In most versions of ssh these days, I'm pretty sure .shosts and > > /etc/shosts.equiv is pretty much ignored in favor of the much stronger > > rsa/dsa host identification. > > > > I enclose below a trace of ssh -v in a successful remote login with no > > password. Note that it gets down to where it needs to authenticate, > > announces that it can do so via either publickey (a keypair) or a > > password (where it has already validated the host identity) and then > > uses the publickey successfully. > > > > HTH, > > > > rgb > > > > -- > > Robert G. Brown http://www.phy.duke.edu/~rgb/ > > Duke University Dept. of Physics, Box 90305 > > Durham, N.C. 27708-0305 > > Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb at phy.duke.edu > > > > > > rgb at lilith|T:109>ssh -v ganesh > > OpenSSH_3.5p1, SSH protocols 1.5/2.0, OpenSSL 0x0090701f > > debug1: Reading configuration data /etc/ssh/ssh_config > > debug1: Applying options for * > > debug1: Rhosts Authentication disabled, originating port will not be > > trusted. > > debug1: ssh_connect: needpriv 0 > > debug1: Connecting to ganesh [152.3.182.51] port 22. > > debug1: Connection established. > > debug1: identity file /home/rgb/.ssh/identity type 0 > > debug1: identity file /home/rgb/.ssh/id_rsa type -1 > > debug1: identity file /home/rgb/.ssh/id_dsa type 2 > > debug1: Remote protocol version 1.99, remote software version > > OpenSSH_3.1p1 > > debug1: match: OpenSSH_3.1p1 pat OpenSSH_2.*,OpenSSH_3.0*,OpenSSH_3.1* > > debug1: Enabling compatibility mode for protocol 2.0 > > debug1: Local version string SSH-2.0-OpenSSH_3.5p1 > > debug1: SSH2_MSG_KEXINIT sent > > debug1: SSH2_MSG_KEXINIT received > > debug1: kex: server->client aes128-cbc hmac-md5 none > > debug1: kex: client->server aes128-cbc hmac-md5 none > > debug1: SSH2_MSG_KEX_DH_GEX_REQUEST sent > > debug1: expecting SSH2_MSG_KEX_DH_GEX_GROUP > > debug1: dh_gen_key: priv key bits set: 127/256 > > debug1: bits set: 1577/3191 > > debug1: SSH2_MSG_KEX_DH_GEX_INIT sent > > debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY > > debug1: Host 'ganesh' is known and matches the RSA host key. > > debug1: Found key in /home/rgb/.ssh/known_hosts2:1 > > debug1: bits set: 1617/3191 > > debug1: ssh_rsa_verify: signature correct > > debug1: kex_derive_keys > > debug1: newkeys: mode 1 > > debug1: SSH2_MSG_NEWKEYS sent > > debug1: waiting for SSH2_MSG_NEWKEYS > > debug1: newkeys: mode 0 > > debug1: SSH2_MSG_NEWKEYS received > > debug1: done: ssh_kex2. > > debug1: send SSH2_MSG_SERVICE_REQUEST > > debug1: service_accept: ssh-userauth > > debug1: got SSH2_MSG_SERVICE_ACCEPT > > debug1: authentications that can continue: publickey,password > > debug1: next auth method to try is publickey > > debug1: try privkey: /home/rgb/.ssh/id_rsa > > debug1: try pubkey: /home/rgb/.ssh/id_dsa > > debug1: input_userauth_pk_ok: pkalg ssh-dss blen 435 lastkey 0x808bb10 > > hint 2 > > debug1: read PEM private key done: type DSA > > debug1: ssh-userauth2 successful: method publickey > > debug1: channel 0: new [client-session] > > debug1: send channel open 0 > > debug1: Entering interactive session. > > debug1: ssh_session2_setup: id 0 > > debug1: channel request 0: pty-req > > debug1: Requesting X11 forwarding with authentication spoofing. > > debug1: channel request 0: x11-req > > debug1: channel request 0: shell > > debug1: fd 3 setting TCP_NODELAY > > debug1: channel 0: open confirm rwindow 0 rmax 32768 > > Last login: Thu Jun 12 13:56:58 2003 from rgb.adsl.duke.edu > > rgb at ganesh|T:101> > > > > _______________________________________________ > > Beowulf mailing list, Beowulf at beowulf.org > > To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf > > > -- > Santiago Muelas > Profesor de Resistencia de Materiales y C?lculo de Estructuras > ETSI de Caminos, Canales y Puertos (U.P.M) > smuelas at mecanica.upm.es http://w3.mecanica.upm.es/~smuelas > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org > To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From k_kolokoutsas at yahoo.co.uk Fri Jun 13 19:48:56 2003 From: k_kolokoutsas at yahoo.co.uk (=?iso-8859-1?q?kolokoutsas=20konstantinos?=) Date: Sat, 14 Jun 2003 00:48:56 +0100 (BST) Subject: ideal motherboard for diskless beowulf In-Reply-To: Message-ID: <20030613234856.5266.qmail@web9908.mail.yahoo.com> Thank you all for the input! This beowulf will be dedicated to running one particular Monte Carlo particle acceleration code already running within RH7.2 and quite dependant on it in many ways, thus the RH7.2 criterium. The 12-node config will serve as a test for a larger cluster, thus the very limited budget, and the choice of (the cheaper) AMD CPUs. The micro-atx form factor is of interest because "I was given the challenge..." of putting as many motherboards in one customized full tower box as possible. Dual/Quad CPU motherboards are not an option, while due to portability issues, racks are out of the question. Thanks once again, Kostas Kolokoutsas __________________________________________________ Yahoo! Plus - For a better Internet experience http://uk.promotions.yahoo.com/yplus/yoffer.html _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From brian at swbell.net Fri Jun 13 23:39:07 2003 From: brian at swbell.net (Brian Macy) Date: Fri, 13 Jun 2003 22:39:07 -0500 (CDT) Subject: Large File Support for RedHat Linux 8.0 Kernal 2.4.18-14 In-Reply-To: <3EEA527A.1030303@cert.ucr.edu> Message-ID: On Fri, 13 Jun 2003, Glen Kaukola wrote: > Steve Elliot wrote: > > >Our RedHat 8.0 doesn't seem to support Large file > >system out of the box. We have a testing program > >running on an ext3 file system, with both > >_LARGEFILE_SOURCE and _FILE_OFFSET_BITS=64 defined, > >the lseek() function failed when the file pointer goes > >beyound 2G. What's the steps to turn the large file > >support on? and how do I know it is on? > > > > We had a similar problem, and it turned to be tcsh. tcsh doesn't > support files over 2GB it seems, so we ended up having to convert all of > our scripts over to bash. > > > Glen > The other thing you need to check is that you're using an off_t type variable for the offset argument to lseek. If you're using an int or a long, you'll still have the 32-bit restriction. Brian -- Brian Macy bcmacy at swbell.net _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From becker at scyld.com Sat Jun 14 06:25:55 2003 From: becker at scyld.com (Donald Becker) Date: Sat, 14 Jun 2003 06:25:55 -0400 (EDT) Subject: Large File Support for RedHat Linux 8.0 Kernal 2.4.18-14 In-Reply-To: <3EEA527A.1030303@cert.ucr.edu> Message-ID: On Fri, 13 Jun 2003, Glen Kaukola wrote: > Steve Elliot wrote: > >Our RedHat 8.0 doesn't seem to support Large file > >system out of the box. We have a testing program > >running on an ext3 file system, with both > >_LARGEFILE_SOURCE and _FILE_OFFSET_BITS=64 defined, > >the lseek() function failed when the file pointer goes > >beyound 2G. What's the steps to turn the large file > >support on? and how do I know it is on? > > > > We had a similar problem, and it turned to be tcsh. tcsh doesn't > support files over 2GB it seems, so we ended up having to convert all of > our scripts over to bash. Use bash version 2.0 or later. Version 1 had LFS problems, some surprising. In some cases you couldn't pipe ("|") or redirect more than 2GB of data. You'll also want to verify your specific 'perl' binary, which is used like a shell. Other places we had to make modifications were the FTP servers and clients, most of which do not have LFS. -- Donald Becker becker at scyld.com Scyld Computing Corporation http://www.scyld.com 914 Bay Ridge Road, Suite 220 Scyld Beowulf cluster system Annapolis MD 21403 410-990-9993 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From douglas at shore.net Sat Jun 14 11:59:01 2003 From: douglas at shore.net (Douglas O'Flaherty) Date: Sat, 14 Jun 2003 11:59:01 -0400 Subject: Opteron Large Memory Compilers? Message-ID: <3EEB4645.4040405@shore.net> Chris: The GA release of the PGI Compiler which supports Opteron is set for 6/30. I don't know of a 'new' beta release since the Opteron launch. There is a 2GB limit on a single data object in the beta, but larger aggregate data sets are supported. The limit is described in the release notes. I certainly would expect it to be changed in the 6/30 release. Was that the issue? There was a new beta of the Absoft Opteron compiler this month, about which I know little. I assume you've already tried gcc. cheers, doug Douglas O'Flaherty _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From maurice at harddata.com Sat Jun 14 13:05:34 2003 From: maurice at harddata.com (Maurice Hilarius) Date: Sat, 14 Jun 2003 11:05:34 -0600 Subject: Opteron Large Memory Compilers? In-Reply-To: <200306141032.h5EAWTU28993@NewBlue.Scyld.com> Message-ID: <5.1.1.6.2.20030614105819.03f917c0@mail.harddata.com> >From: "Chris Oubre" >Subject: Opteron Large Memory Compilers? >Date: Fri, 13 Jun 2003 14:17:01 -0500 > >Hello, > We are looking into the feasibility of using Opterons in on >cluster. One major issue we had when we benchmarked on them was the >lack of large memory support in the compilers we used. We tried to run >large memory jobs on an Opteron using the Portland Compilers and found >that it would not compile jobs larger than 2 Gigs. We contacted the >Portland Group and they responded that it was a know bug. We haven't >tried again in a while and we were wondering if anyone knows if this has >been fixed? I suggest you try GCC. It is not saddled with this problem, and in our tests generally quite a bit better than PGI in performance. You also want to grab the Core Math Library from AMD: http://www.amd.com/us-en/Processors/DevelopWithAMD/0,,30_2252_2282,00.html With our best regards, Maurice W. Hilarius Telephone: 01-780-456-9771 Hard Data Ltd. FAX: 01-780-456-9772 11060 - 166 Avenue mailto:maurice at harddata.com Edmonton, AB, Canada http://www.harddata.com/ T5X 1Y3 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From maurice at harddata.com Mon Jun 16 01:23:11 2003 From: maurice at harddata.com (Maurice Hilarius) Date: Sun, 15 Jun 2003 23:23:11 -0600 Subject: Two CS undergraduates trying to build a prototype beowulf need help with bproc In-Reply-To: <200306151902.h5FJ2uU11623@NewBlue.Scyld.com> Message-ID: <5.1.1.6.2.20030615231836.03c746c0@mail.harddata.com> If you want a working and tested free bproc based distribution you may download it from our ftp server. It is based on RedHat 7.3 (8.0 is rather messy) ftp://ftp.harddata.com/pub/hddcs/3.2/2003-spring/ Docs are included. >Hello friends. >We tried searching the mailing list archive, but the link on the beowulf >site was off, so we decided to try our luck and consult the oracles. :) > <<>> >So, cutting down to our problem: > >We installed redhat 8.0 in two machines, named master and slave1. We >setup nfs and rsh/rexec/rlogin and everything is working fine at kernel >level. We can rsh from/to slave1/master without any password prompts, >and pam looks ok too. > >We installed the 3.2.4 bproc/beowulf kernel patch in kernel 2.4.20, >recompiled it and as it seems, everything looks fine. > >We sucessfully modprobed vmadump and bproc, configured our master's >/etc/beowulf/config and all looked fine to go. > >However, when we tried running bpmaster (as root) it started, then >segfaulted. <<>> >Any sugestions, ideas, flames (those in pvt, pls) and the like would be >very welcome. :) > >Thanks in Advance, > >-- >Marcos Hiroshi Umino With our best regards, Maurice W. Hilarius Telephone: 01-780-456-9771 Hard Data Ltd. FAX: 01-780-456-9772 11060 - 166 Avenue mailto:maurice at harddata.com Edmonton, AB, Canada http://www.harddata.com/ T5X 1Y3 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From egan at sense.net Sun Jun 15 17:59:13 2003 From: egan at sense.net (Egan Ford) Date: Sun, 15 Jun 2003 15:59:13 -0600 Subject: NAS Parallel Benchmarks for Current Hardware In-Reply-To: <3EE8542D.93CE3B57@ideafix.litec.csic.es> Message-ID: <033001c33389$5dde1f00$27b358c7@titan> > Unfortunately, I am not sure linpack and spec reliably measure what is > important for numerical work on a beowulf. I disagree. SPECCPU v1.2 FP benchmarks (http://www.spec.org/cpu2000/CFP2000/): 168.wupwise Fortran 77 Physics / Quantum Chromodynamics 171.swim Fortran 77 Shallow Water Modeling 172.mgrid Fortran 77 Multi-grid Solver: 3D Potential Field 173.applu Fortran 77 Parabolic / Elliptic Partial Differential Equations 177.mesa C 3-D Graphics Library 178.galgel Fortran 90 Computational Fluid Dynamics 179.art C Image Recognition / Neural Networks 183.equake C Seismic Wave Propagation Simulation 187.facerec Fortran 90 Image Processing: Face Recognition 188.ammp C Computational Chemistry 189.lucas Fortran 90 Number Theory / Primality Testing 191.fma3d Fortran 90 Finite-element Crash Simulation 200.sixtrack Fortran 77 High Energy Nuclear Physics Accelerator Design 301.apsi Fortran 77 Meteorology: Pollutant Distribution _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From japa at mhu.eti.br Sat Jun 14 16:12:28 2003 From: japa at mhu.eti.br (Marcos Hiroshi Umino) Date: Sat, 14 Jun 2003 17:12:28 -0300 Subject: Two CS undergraduates trying to build a prototype beowulf need help with bproc Message-ID: <20030614165721.DA2B.JAPA@mhu.eti.br> Hello friends. We tried searching the mailing list archive, but the link on the beowulf site was off, so we decided to try our luck and consult the oracles. :) We are brazilian computer sciences undergraduates from the contryside of Sao Paulo, in a city called Presidente Prudente. Our computer sciences university is new and we are students of the first class, so actually, we are making our lab. It is a great chance to play around, so we two decided to team up and try to build a small prototype beowulf. We are not linux veterans, unfortunately, but we're trying hard and doing some progress. So, cutting down to our problem: We installed redhat 8.0 in two machines, named master and slave1. We setup nfs and rsh/rexec/rlogin and everything is working fine at kernel level. We can rsh from/to slave1/master without any password prompts, and pam looks ok too. We installed the 3.2.4 bproc/beowulf kernel patch in kernel 2.4.20, recompiled it and as it seems, everything looks fine. We sucessfully modprobed vmadump and bproc, configured our master's /etc/beowulf/config and all looked fine to go. However, when we tried running bpmaster (as root) it started, then segfaulted. Unfortunately, our guidebook is a brazilian translation, and I must admit the translator wasn't very good at it. We are trying to search for complementary textbooks (we are students, so books sugestions are great, but unfortunately are out of question). We are a bit lost because the book we're following seems to use an outdated beowulf version, because bproc docs seems to point toward no need for rsh/rexec. Any sugestions, ideas, flames (those in pvt, pls) and the like would be very welcome. :) Thanks in Advance, -- Marcos Hiroshi Umino _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From BrianTJones at RealEdu.org Sat Jun 14 15:31:09 2003 From: BrianTJones at RealEdu.org (BrianTJones at RealEdu.org) Date: Sat, 14 Jun 2003 15:31:09 -0400 Subject: ups units In-Reply-To: <3EDF9C66.5090106@cert.ucr.edu> References: <3EDF9C66.5090106@cert.ucr.edu> Message-ID: Glen, 1 of the things you have to keep in mind is the watt rating & watt load are 2 separate animals; Two ways of calculating your load; 1. put the PC as the ONLY thing running on a DIGITAL power meter for your home or business 2. look up all your parts' active & inactive loads 2b or not. Watts=Volts x Amperage P=IV Brian Sidenote; If you have dead or surplus equipment Donate them! Don't let lead & other heavy metals go into landfills & the water tables!! We're organizing donations to schools & nonprofits throughout the US & helping with the repair work. DonateHW at RealEdu.org for more details _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From andy at ideafix.litec.csic.es Mon Jun 16 04:44:32 2003 From: andy at ideafix.litec.csic.es (andy) Date: Mon, 16 Jun 2003 09:44:32 +0100 Subject: NAS Parallel Benchmarks for Current Hardware References: <033001c33389$5dde1f00$27b358c7@titan> Message-ID: <3EED8370.ADD04801@ideafix.litec.csic.es> Egan Ford wrote: > > > Unfortunately, I am not sure linpack and spec reliably measure what is > > important for numerical work on a beowulf. > > I disagree. I admit I should have been more careful and inserted the word "our" befor "numerical work on beowulf". My concern is partly that they does not test what I want to know and partly that linpack and spec blob ratings shift a lot of hardware. What I need (and I supect many others but I will not speak for them this time) is: * a range of grid sizes so that I can see cache/memory effects * a range of common numercal solvers (covering most of those I use) with differing communication characteristics so that I can gauge the likely behaviour of a range of simulations. No single number can provide such information but something the like NAS parallel benchmarks do. 15-20 years ago linpack stopped being a measure of anything useful when manufacturers adapted their compilers/hardware to generate unrepresentatively large blob ratings. This was brought home when commissioning a small shared memory machine and we could not get near the quoted linpack blob rating using various combinations of optimisation flags (just over 10 blobs instead of nearly 30 blobs if I recall correctly). After a phone call we used the undocumented -linpack compiler flag and bingo. Evidence is growing that the spec blob ratings have suffered in a similar way (though to a lesser degree). See, for example, "Sun breaks another SPEC benchmark" in comp.benchmarks: http://groups.google.com/groups?hl=en&lr=&ie=UTF-8&safe=off&group=comp.benchmarks _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From dwarkanath.mb at vinciti.com Mon Jun 16 07:18:53 2003 From: dwarkanath.mb at vinciti.com (Dwarkanath) Date: Mon, 16 Jun 2003 16:48:53 +0530 Subject: Beowulf for MAC OSX Message-ID: Hello Group, I wish to know, whether Beowulf supports MAC OS X. If yes, can i know the details/pre-requisites/steps to accomplish the same. Thanking you in advance, Dwarkanath MB _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From way2hrishi at rediffmail.com Mon Jun 16 09:19:20 2003 From: way2hrishi at rediffmail.com (Hrishikesh Dewan) Date: 16 Jun 2003 13:19:20 -0000 Subject: Calculating Performance Bencmark Message-ID: <20030616131920.9262.qmail@webmail24.rediffmail.com> Dear Friends & Experts, We are interested in setting up a Beowulf in our Mathematics Department. But the problem that we are facing is calculating the Peak Performance of the cluster. Can anyone help us , point to any links that have such information. Any help will always be highly regarded. Regards, Hrishikesh Dewan thinking in colors -------------- next part -------------- An HTML attachment was scrubbed... URL: From egan at sense.net Mon Jun 16 12:02:49 2003 From: egan at sense.net (Egan Ford) Date: Mon, 16 Jun 2003 10:02:49 -0600 Subject: NAS Parallel Benchmarks for Current Hardware In-Reply-To: <3EED8370.ADD04801@ideafix.litec.csic.es> Message-ID: <03cc01c33420$bd0cb3f0$27b358c7@titan> Unfortunately benchmarking is a game. I tell my customers to benchmark their application. It is the only way to really know what will work best for any particular application. We offer a benchmark lab to assist with that. > -----Original Message----- > From: andy at ideafix.litec.csic.es [mailto:andy at ideafix.litec.csic.es] > Sent: Monday, June 16, 2003 2:45 AM > To: egan at sense.net; beowulf at beowulf.org > Subject: Re: NAS Parallel Benchmarks for Current Hardware > > > Egan Ford wrote: > > > > > Unfortunately, I am not sure linpack and spec reliably > measure what is > > > important for numerical work on a beowulf. > > > > I disagree. > > I admit I should have been more careful and inserted the word "our" > befor "numerical work on beowulf". My concern is partly that they does > not test what I want to know and partly that linpack and spec blob > ratings shift a lot of hardware. > > What I need (and I supect many others but I will not speak > for them this > time) is: > > * a range of grid sizes so that I can see cache/memory effects > > * a range of common numercal solvers (covering most of those > I use) with > differing communication characteristics so that I can gauge the likely > behaviour of a range of simulations. > > No single number can provide such information but something > the like NAS > parallel benchmarks do. > > 15-20 years ago linpack stopped being a measure of anything > useful when > manufacturers adapted their compilers/hardware to generate > unrepresentatively large blob ratings. This was brought home when > commissioning a small shared memory machine and we could not get near > the quoted linpack blob rating using various combinations of > optimisation flags (just over 10 blobs instead of nearly 30 blobs if I > recall correctly). After a phone call we used the > undocumented -linpack > compiler flag and bingo. > > Evidence is growing that the spec blob ratings have suffered in a > similar way (though to a lesser degree). See, for example, "Sun breaks > another SPEC benchmark" in comp.benchmarks: > http://groups.google.com/groups?hl=en&lr=&ie=UTF-8&safe=off&group=comp.bench marks _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rgb at phy.duke.edu Mon Jun 16 13:36:54 2003 From: rgb at phy.duke.edu (Robert G. Brown) Date: Mon, 16 Jun 2003 13:36:54 -0400 (EDT) Subject: ideal motherboard for diskless beowulf In-Reply-To: <20030613234856.5266.qmail@web9908.mail.yahoo.com> Message-ID: On Sat, 14 Jun 2003, kolokoutsas konstantinos wrote: > > Thank you all for the input! > > This beowulf will be dedicated to running one > particular Monte Carlo particle acceleration code > already running within RH7.2 and quite dependant on it > in many ways, thus the RH7.2 criterium. I'm sorry, how is that? a) Can't you just recompile under 7.3 or for that matter 9? b) I would expect most applications compiled for 7.2 to just plain "run" on 7.3, at least. Almost by definition, the major libraries don't change much between minor revision increments. It would certainly be worth it to try running it your application on 7.3. It would also be worth it to start porting your code into a source RPM format so that it can just be rebuilt in the future in five minutes whenever you want to run it on an RPM-supporting architecture.a > The 12-node config will serve as a test for a larger > cluster, thus the very limited budget, and the choice > of (the cheaper) AMD CPUs. > > The micro-atx form factor is of interest because "I > was given the challenge..." of putting as many > motherboards in one customized full tower box as > possible. Dual/Quad CPU motherboards are not an > option, while due to portability issues, racks are out > of the question. Hmm, I don't know what you mean by "portability issues either. I've built both tower/shelf clusters and rack clusters and set them up in e.g. Expo booths for a three day demo.. Tower/shelf clusters are a total PITA to transport. Rack-based clusters are often much easier and achieve a higher CPU density >>if<< you use a rolling rack. At Linux Expo three or four years ago we built a dual rack Netfinity cluster with dual P3 nodes, kindly loaned to us by IBM. The whole thing came off the truck and was set up and running (48 CPUs) in a matter of four hours or so. Taking it down was even faster -- a couple of hours start to finish. It took me almost as long to set up my much smaller tower cluster on a rolling shelf unit I brought from home, with all the cabling and carrying. Nowadays, there are some REALLY cool rolling four post racks. Check out the "department server rack" on http://www.phy.duke.edu/brahma/brahma_tour.php (right above "Seth, Looking Grumpy":-). I don't remember how many U it is, but at a guess perhaps 20U. One could EASILY fit 16 1U cases in it with room to spare, or 32 (or more) CPUs. Monitor and KVM (switch) on top, middlin' hefty UPS on the bottom, and you can literally roll it from room to room without even powering it down! And this is only one choice -- I'll bet there are a variety of options even here. When you are ready to scale up, just buy two post racks and move the nodes into a permanent home... If your money is REALLY tight perhaps you can't afford this, but if you are trying to "sell" the result a rolling rack is going to beat the pants off of a jury riggged tower setup in crowd appeal... rgb > > Thanks once again, > Kostas Kolokoutsas > > > > > __________________________________________________ > Yahoo! Plus - For a better Internet experience > http://uk.promotions.yahoo.com/yplus/yoffer.html > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org > To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf > -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb at phy.duke.edu _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From kpodesta at redbrick.dcu.ie Mon Jun 16 13:04:10 2003 From: kpodesta at redbrick.dcu.ie (Karl Podesta) Date: Mon, 16 Jun 2003 18:04:10 +0100 Subject: Beowulf for MAC OSX In-Reply-To: References: Message-ID: <20030616170410.GB17594@prodigy.Redbrick.DCU.IE> On Mon, Jun 16, 2003 at 04:48:53PM +0530, Dwarkanath wrote: > Hello Group, > > I wish to know, whether Beowulf supports MAC OS X. If yes, can i know the > details/pre-requisites/steps to accomplish the same. > > Thanking you in advance, > > Dwarkanath MB If you have machines which run OS X, then you could make a beowulf out of them. You just have to connect the machines together into a network, and then install the software you want to run (MPI/PVM, etc). Connecting the machines together should not be a problem - installing the software is where the bulk of your work is. You will need to compile MPI, PVM, or whatever software you want to run. There seems to be a short guide here: http://www.stat.ucla.edu/computing/clusters/deployment.php And you could also check out www.bioteam.net for a look at the Apple clusters they've put together, and the bioinformatics software they have available for Mac OS X. This is simplifying things, and assumes you just have a few spare Macs you want to use! If you're looking for a cluster to accomplish a specific task, obviously you need to think about your application, money you have to spare, etc. Kp -- Karl Podesta Dublin City University, Ireland _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From natorro at fisica.unam.mx Mon Jun 16 21:06:42 2003 From: natorro at fisica.unam.mx (Carlos Lopez Nataren) Date: 16 Jun 2003 20:06:42 -0500 Subject: SSH or RSH Message-ID: <1055812002.1628.6.camel@natorro.fisica.unam.mx> Hello, I was wondering if anybody has made a test on whether ssh can make a difference over rsh when using mpi (LAM/MPICH) and it is used for the slave nodes??? and one more, the same as before but now using NFS over IPSEC, can anyone tell??? Thanks a lot in advance. natorro -- Carlos Lopez Nataren Instituto de Fisica, UNAM _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From jhearns at freesolutions.net Tue Jun 17 04:08:22 2003 From: jhearns at freesolutions.net (John Hearns) Date: Tue, 17 Jun 2003 09:08:22 +0100 Subject: ideal motherboard for diskless beowulf References: Message-ID: <005b01c334a7$a1ef4060$8461cdc2@DREAD> ----- Original Message ----- From: "Robert G. Brown" To: "kolokoutsas konstantinos" Cc: Sent: Monday, June 16, 2003 6:36 PM Subject: Re: ideal motherboard for diskless beowulf > > > > This beowulf will be dedicated to running one > > particular Monte Carlo particle acceleration code > > already running within RH7.2 and quite dependant on it > > in many ways, thus the RH7.2 criterium. > > I'm sorry, how is that? > > a) Can't you just recompile under 7.3 or for that matter 9? You could, yes. However, HEP applications are linked with big libraries - such as the GEANT codes and the CERNLIBS. What happens is that there is a certification process, which checks out that the codes run on a certain version of (say) Redhat. However, the current certified version at CERN is 7.3, no? > > > > The micro-atx form factor is of interest because "I > > was given the challenge..." of putting as many > > motherboards in one customized full tower box as > > possible. Dual/Quad CPU motherboards are not an > > option, while due to portability issues, racks are out > > of the question. Kostas, if you are within reach of London you are welcome to have a look at the mini-ITX boards I have. I would be happy to help put the CERN environment on them and see how they run your codes. You certainly could make an easily portable cluster with these boards. (I am a paricle physicist, so do understand some of this stuff!) If you have a budget, there are boards using the same CPU which fit into the slots for a hard disk drive. That would fit your tower rack configuration - just get a full tower case, ordinary motherboard for the 'head' CPU, and populate all the drive bays with CPUs. You then use a small Ethernet switch to connect them. Again, I would be happy to help and advise on the boards, and helping get the Monte Carlo codes up and running. John Hearns _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From jhearns at freesolutions.net Tue Jun 17 04:41:47 2003 From: jhearns at freesolutions.net (John Hearns) Date: Tue, 17 Jun 2003 09:41:47 +0100 Subject: ideal motherboard for diskless beowulf References: Message-ID: <00f001c334ac$4d20f290$8461cdc2@DREAD> > > > > The micro-atx form factor is of interest because "I > > was given the challenge..." of putting as many > > motherboards in one customized full tower box as > > possible. Dual/Quad CPU motherboards are not an > > option, while due to portability issues, racks are out > > of the question. Kostas, you don't tell us your background. Are you a student or research fellow in one of the UK universities maybe? Which experiment are you working on? As I say, I did my PhD on ALEPH and have worked at CERN. I played around with ideas of putting lots of CPUs in a tower case last year. The discussion is on the Bewoulf list I think - the problem with that the little CPU boards were more expensive. I think that this configuration would be instructive to build, and would have a lot of applications. The boards I'm thinking of fit in a drive bay, and use power from a standard disk drive connector. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From phil.neumiller at convergys.com Mon Jun 16 17:49:35 2003 From: phil.neumiller at convergys.com (phil.neumiller at convergys.com) Date: Mon, 16 Jun 2003 17:49:35 -0400 Subject: Beowulf in Enterprise Environments Message-ID: Hi, Does anybody on the list know of folks using Beowulf clusters in what might be called traditional IT or Enterprise environments? Why/Why not? What about in cryptographic acceleration environments? Why/Why not? Perhaps for stuff even as benign as SSL/TLS/IPsec terminations??? Thanks, Phil -- "NOTICE: The information contained in this electronic mail transmission is intended by Convergys Corporation for the use of the named individual or entity to which it is directed and may contain information that is privileged or otherwise confidential. If you have received this electronic mail transmission in error, please delete it from your system without copying or forwarding it, and notify the sender of the error by reply email or by telephone (collect), so that the sender's address records can be corrected." _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From lathama at yahoo.com Tue Jun 17 00:52:35 2003 From: lathama at yahoo.com (Andrew Latham) Date: Mon, 16 Jun 2003 21:52:35 -0700 (PDT) Subject: new to clusters Message-ID: <20030617045235.93517.qmail@web80512.mail.yahoo.com> I have recently put together(just hardware) two small clusters. What would be a good simple project for a novice like myself to break myself and my clusters in. they are small and cheap but great for learning. 9 - Allen Bradley DataMytes - P133 - 64mb - Compaq NC3121 3 - IBM PC 365 6589-13U - Dual P-Pro200 - 128mb - Compaq NC3121 is there a project that would be good to run to warm these boys up and teach me about managing a cluster. ===== Andrew Latham LathamA.com lathama at lathama.com lathama at yahoo.com LathamA(lay-th-ham-eh) _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From jhearns at freesolutions.net Tue Jun 17 06:51:24 2003 From: jhearns at freesolutions.net (John Hearns) Date: Tue, 17 Jun 2003 11:51:24 +0100 Subject: Beowulf in Enterprise Environments References: Message-ID: <002301c334be$6837e4f0$8461cdc2@DREAD> ----- Original Message ----- From: To: Sent: Monday, June 16, 2003 10:49 PM Subject: Beowulf in Enterprise Environments > Hi, > > Does anybody on the list know of folks using Beowulf clusters in what might > be called > traditional IT or Enterprise environments? Why/Why not? > > What about in cryptographic acceleration environments? Why/Why not? > Perhaps for > stuff even as benign as SSL/TLS/IPsec terminations??? Depends what you mean by Beowulf of course :-) If we expand the definition to include Clusters of Workstations, or batch farms, then yes. People like car manufacturers, aerospace companies running simulations. Render farms for movie production. I guess financial institutions running Monte Carlos also. The Beowulf list however will (quite rightly) set the definition tighter, to mean a parallell cluster using MPI or PVM, and commodity hardware. Then I admit not to have much knowledge of uses in the Enterprise. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From cjereme at ucla.edu Tue Jun 17 08:39:56 2003 From: cjereme at ucla.edu (cjereme at ucla.edu) Date: Tue, 17 Jun 2003 4:39:56 PST Subject: rocks nacpi Message-ID: <200306171239.h5HCduH16662@webmail.my.ucla.edu> An embedded and charset-unspecified text was scrubbed... Name: not available URL: From rgb at phy.duke.edu Tue Jun 17 09:34:15 2003 From: rgb at phy.duke.edu (Robert G. Brown) Date: Tue, 17 Jun 2003 09:34:15 -0400 (EDT) Subject: SSH or RSH In-Reply-To: <1055812002.1628.6.camel@natorro.fisica.unam.mx> Message-ID: On 16 Jun 2003, Carlos Lopez Nataren wrote: > Hello, I was wondering if anybody has made a test on whether ssh can > make a difference over rsh when using mpi (LAM/MPICH) and it is used for > the slave nodes??? Make a difference of what sort? Both rsh and ssh are in the range of negligible overhead for most sensible job distribution schema. They are invoked once per remote process for the sole purpose of distributing the remote processes. AFAIK, they aren't used as any sort of meaningful (long term) communications pipeline by any of major parallel packages, which are designed to provide their own, highly efficient communications and job control. You can look back at the list archives -- I've benchmarked ssh and rsh several times over the last few years. I no longer can do rsh -- it isn't even installed on dulug systems by default, although I am yumming to see if an rsh rpm is still out there. However, over a rather noisy >>wireless<< connection from my laptop the time required to establish an ssh connection is 0.4 seconds, on average. The ssh-derived overhead in the time required to spawn 100 remote jobs would thus be less than a minute absolute; even allowing rsh to be "infinitely fast" by comparison I'd say that this is negligible. It is a bit more (marginally) expensive when it comes to scp. When copying files, its bidirectional encryption algorithm significantly adds time relative to e.g. rsh or a raw socket pipe, and is also significantly more CPU intensive. I honestly don't know if any of the MPI's use rcp to copy a binary image to the remote host to execute it -- pvm does not (relying on the binary already being available on the nodes on a predetermined path). For a large binary, this might not be negligible. Overall, I'd say that if you are running something where the rsh/ssh difference matters, you should be looking at Scyld or bproc anyway as they reduce this sort of overhead to the theoretical minimum, at the cost of providing virtually no node security (so you really have to run your cluster inside a head node/firewall). In all other cases, I'd recommend using ssh. It is time for rsh to just go away. ssh has lots of advantages besides just security, and as network and CPU speeds continue to increase the marginal cost of using it (which was negligible years ago) will continue to decrease. > and one more, the same as before but now using NFS over IPSEC, can > anyone tell??? No clue here, sorry. rgb -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb at phy.duke.edu _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From shewa at inel.gov Tue Jun 17 11:03:10 2003 From: shewa at inel.gov (Andrew Shewmaker) Date: Tue, 17 Jun 2003 09:03:10 -0600 Subject: Two CS undergraduates trying to build a prototype beowulf need help with bproc In-Reply-To: <20030614165721.DA2B.JAPA@mhu.eti.br> References: <20030614165721.DA2B.JAPA@mhu.eti.br> Message-ID: <3EEF2DAE.2080808@inel.gov> Marcos Hiroshi Umino wrote: > Hello friends. > > We tried searching the mailing list archive, but the link on the beowulf > site was off, so we decided to try our luck and consult the oracles. :) Try http://marc.theaimsgroup.com/?l=beowulf&r=1&w=2 > We installed redhat 8.0 in two machines, named master and slave1. We > setup nfs and rsh/rexec/rlogin and everything is working fine at kernel > level. We can rsh from/to slave1/master without any password prompts, > and pam looks ok too. > > We installed the 3.2.4 bproc/beowulf kernel patch in kernel 2.4.20, > recompiled it and as it seems, everything looks fine. > > We sucessfully modprobed vmadump and bproc, configured our master's > /etc/beowulf/config and all looked fine to go. > > However, when we tried running bpmaster (as root) it started, then > segfaulted. You know, bproc has a mailing list too. http://bproc.sf.net I remember running into a problem with bpmaster and my network interfaces. Do you have two interfaces? One for the public network and one for the beowulf? If your master node has only one NIC and you already configured the interface, then I think bpmaster will also try to initialize a network interface and you will have trouble. > Unfortunately, our guidebook is a brazilian translation, and I must > admit the translator wasn't very good at it. > > We are trying to search for complementary textbooks (we are students, so > books sugestions are great, but unfortunately are out of question). We > are a bit lost because the book we're following seems to use an outdated > beowulf version, because bproc docs seems to point toward no need for > rsh/rexec. Yes. With bproc you have no need of rsh/rexec. Although you can install bproc on a cluster with full Linux installs on every node, bproc based clusters usually only have a full install on the master. The slave nodes boot up a minimal kernel from the master. Andrew -- Andrew Shewmaker, Associate Engineer Phone: 1-208-526-1276 Idaho National Eng. and Environmental Lab. P.0. Box 1625, M.S. 3605 Idaho Falls, Idaho 83415-3605 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rgb at phy.duke.edu Tue Jun 17 11:06:42 2003 From: rgb at phy.duke.edu (Robert G. Brown) Date: Tue, 17 Jun 2003 11:06:42 -0400 (EDT) Subject: new to clusters In-Reply-To: <20030617045235.93517.qmail@web80512.mail.yahoo.com> Message-ID: On Mon, 16 Jun 2003, Andrew Latham wrote: > I have recently put together(just hardware) two small clusters. What would be a > good simple project for a novice like myself to break myself and my clusters > in. they are small and cheap but great for learning. > > 9 - Allen Bradley DataMytes - P133 - 64mb - Compaq NC3121 > 3 - IBM PC 365 6589-13U - Dual P-Pro200 - 128mb - Compaq NC3121 > > is there a project that would be good to run to warm these boys up and teach me > about managing a cluster. Check out e.g. Forrest Hoffman's "Extreme Linux" Column in Linux Magazine, www.linux-mag.com. Forrest has been running a series of tutorial articles on MPI (and in earlier issues, PVM). The June LM issue had a number of articles on cluster infrastructure and management. http://www.phy.duke.edu/brahma has links to lots of beowulf/cluster websites and other resources to help facilitate learning as well. Both PVM and MPI come with packages of "examples" of parallel programs, some of which are quite fun as demos (e.g. mandelbrot set toys, pvmpov) and others of which demonstrate at least some of the various parallel programming paradigms. Finally, there are a number of excellent books out there with tutorial sections on cluster programming. MIT press has a series of books on both MPI and PVM and a companion volume on beowulfs that is probably "the" beowulf book, again with numerous examples. There are a number of books on parallel algorithms: Amalsi and Gottlieb, Foster,... (I can't remember if I have a list linked to brahma or not -- I should probably put one there). That ought to be enough to get you going through the toy cluster stage to where you can be teaching and learning with a good appreciation for the basic toolset. rgb -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb at phy.duke.edu _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rgb at phy.duke.edu Tue Jun 17 11:17:24 2003 From: rgb at phy.duke.edu (Robert G. Brown) Date: Tue, 17 Jun 2003 11:17:24 -0400 (EDT) Subject: Beowulf in Enterprise Environments In-Reply-To: <002301c334be$6837e4f0$8461cdc2@DREAD> Message-ID: On Tue, 17 Jun 2003, John Hearns wrote: > > ----- Original Message ----- > From: > To: > Sent: Monday, June 16, 2003 10:49 PM > Subject: Beowulf in Enterprise Environments > > > > Hi, > > > > Does anybody on the list know of folks using Beowulf clusters in what > might > > be called > > traditional IT or Enterprise environments? Why/Why not? > > > > What about in cryptographic acceleration environments? Why/Why not? As Johns notes, "Beowulf" clusters per se, possibly not. Generalized clusters or LANs, they've been doing this longer than the term "beowulf" has been around -- really as long as COMPUTERS have been around. RC5 is cryptography spread out across the net in what is now a virtual architype of a "grid" computation. crack (the venerable Unix password cracking program) can be run embarrassingly parallel over a network and in fact has been programmed to do so on request for a rather long time now. The NSA has probably been doing this from back when the "cluster" they were using was a cluster of mainframes or a cluster of supercomputers. Indeed, the very first decryption "computers" used in WWII pretty much WERE clusters. ...etc. However, in the enterprise one has to carefully differentiate between "High Availability" (HA) cluster applications -- a distributed webserver, google, distributed databases -- and "High Performance Computing" (HPC) applications. HPC is where beowulfs and compute clusters live and is the primary topic of interest on this list. HA is where enterprise server farms of all flavors live, and is NOT generally discussed a lot on this list (although there are other lists where it is). "Extreme Linux" (once upon a time) referred to the union of HA and HPC and perhaps a bit of other linux automagic to makes it so that (as Sun used to so delicately and cleverly put it) "the network IS the computer". Corporations that need HPC, of course, do clusters all the time -- simulations, finite element analysis in engineering, etc. However, most of what one calls "enterprise" computing is likely to be a component of HA, or at best in a middle ground. The enterprise is a lot more about reliable delivery and load control and balancing than it is about screaming aggregate FLOPS. rgb -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb at phy.duke.edu _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From way2saurav at rediffmail.com Tue Jun 17 06:32:17 2003 From: way2saurav at rediffmail.com (Saurav Gohain) Date: 17 Jun 2003 10:32:17 -0000 Subject: what is a flop Message-ID: <20030617103217.2576.qmail@webmail28.rediffmail.com> Dear Sir/Friends I am new to cluster and hence asking this question ( may be a silly one for u all). In super computers and cluster, I find Peak Performance rated as TFLOPs, GFLOP's etc. Now, the question is what is a flop. From a beowulf pdf tutorial, i came to know that it is calculated as Total Peak Performance= No of Nodes * No of CPU's * Floating point operations per second * No of Cycles per second. Although, the first two and the last is easily derivable, what about the third one. I checked the specs of Opetron and P4 but there's isn't any mention about the Flops. How will i derive then ? In a comparison graph in Intel's website, I found a term SPEC*fp2000 ratings. Now this ratings are like 2.4 , 3.4 etc. Is it the number that I should input in the above formula to calculate the total peak performance in flops about a cluster. Kindly let me know the facts... Regards, Saurav -------------- next part -------------- An HTML attachment was scrubbed... URL: From jhearns at freesolutions.net Tue Jun 17 12:47:58 2003 From: jhearns at freesolutions.net (John Hearns) Date: Tue, 17 Jun 2003 17:47:58 +0100 Subject: ideal motherboard for diskless beowulf References: <005b01c334a7$a1ef4060$8461cdc2@DREAD> <3EEF6E4C.3698AB62@starpower.net> Message-ID: <011c01c334f0$384e5940$8461cdc2@DREAD> > [I'm a different responder..] > That idea has intrigued me for a while... I've been able to find a > number of 5.25 and even 3.5" form factor boards but no enclosures... > (well, only one but that appeared to be quite a bit too long to fit in a > standard bay...) > > The only integrated drive form factor device that I've been able to find > that is suitable to be a slave node is the one from TotalImpact. > Unfortunately that device has become a bit outdated and is VERY > expensive... (they also have an equally interesting/outdated/overpriced > quad G4 PCI board...) Repeated sessions with google has returned no > alternatives... > > Do you have any specific products in mind? The boards I looked at were from Nexcom. Their EBC series, for instance: http://www.nexcom.com/0330/nexweb/weben/ObjView.aspx?ObjID=Prod*10000083 I talked with the UK sales people, who put me onto a UK distributor. Sadly the price was about three times a mini-ITX board, so I couldn't afford them. Looking at their website now, there are 3.5" divices. Wow... http://www.nexcom.com/0330/nexweb/weben/ObjView.aspx?ObjID=Prod*10000088 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From roberto.ammendola at roma2.infn.it Tue Jun 17 14:34:48 2003 From: roberto.ammendola at roma2.infn.it (Roberto Ammendola) Date: Tue, 17 Jun 2003 20:34:48 +0200 Subject: what is a flop References: <20030617103217.2576.qmail@webmail28.rediffmail.com> Message-ID: <3EEF5F48.5020505@roma2.infn.it> Saurav Gohain wrote: > > From a beowulf pdf tutorial, i came to know that it is calculated as > Total Peak Performance= No of Nodes * No of CPU's * Floating point > operations per second * No of Cycles per second. > I would say: Total Peak Performance= No of Nodes * No of CPU's per Node * Floating point operations per clock cycle * No of Cycles per second. The "Floating point operations per clock cycle" depends on the processor, obviously, and on which instructions you use in your code. For example in a processor with the SSE instruction set you can perform 4 operations (on 32 bit register each) per clock cycle. One processor (Xeon or P4) running at 2.0 GHz can reach 8 GigaFlops. But this is only a peak performance. The real value is up to the cleverness of the programmer ;) cheers roberto _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From maurice at harddata.com Tue Jun 17 16:01:32 2003 From: maurice at harddata.com (Maurice Hilarius) Date: Tue, 17 Jun 2003 14:01:32 -0600 Subject: what is a flop (Roberto Ammendola) In-Reply-To: <200306171901.h5HJ1FU07223@NewBlue.Scyld.com> Message-ID: <5.1.1.6.2.20030617135556.03e7b9a0@mail.harddata.com> With regards to your message at 01:01 PM 6/17/03, beowulf-request at scyld.com. Where you stated: > Date: Tue, 17 Jun 2003 20:34:48 +0200 >From: Roberto Ammendola >Subject: Re: what is a flop > >Saurav Gohain wrote: > > > > > From a beowulf pdf tutorial, i came to know that it is calculated as > > Total Peak Performance= No of Nodes * No of CPU's * Floating point > > operations per second * No of Cycles per second. > > >I would say: > >Total Peak Performance= No of Nodes * No of CPU's per Node * Floating >point operations per clock cycle * No of Cycles per second. > >The "Floating point operations per clock cycle" depends on the >processor, obviously, and on which instructions you use in your code. >For example in a processor with the SSE instruction set you can perform >4 operations (on 32 bit register each) per clock cycle. One processor >(Xeon or P4) running at 2.0 GHz can reach 8 GigaFlops. But this is only >a peak performance. The real value is up to the cleverness of the >programmer ;) > >cheers >roberto And I would say dual CPU boards do not sale at a factor of 2:1 over singles. This is mainly ruled by 3 things: 1) efficiency of chipset for SMP operations, access to memory, buses, etc.. 2) efficiency of compiler at SMP/multithreaded code generation 3) efficiency of code design to complement the above factors. As a general ( really general as it changes a lot with code and compilers) the rule I know : Dual P3 ( VIA chipset): 1.5 : 1 Dual XEON P4 ( Intel 7501 chipset): 1.3 : 1 Dual AthlonMP ( AMD 760MPX chipset) 1.4 : 1 Dual Opteron (AMD 8xxx Chipset) 1.8 : 1 Dual Alpha (DEC Tsunami chipset) 1.8 : 1 With our best regards, Maurice W. Hilarius Telephone: 01-780-456-9771 Hard Data Ltd. FAX: 01-780-456-9772 11060 - 166 Avenue mailto:maurice at harddata.com Edmonton, AB, Canada http://www.harddata.com/ T5X 1Y3 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From M.Arndt at science-computing.de Tue Jun 17 12:21:27 2003 From: M.Arndt at science-computing.de (Michael Arndt) Date: Tue, 17 Jun 2003 18:21:27 +0200 Subject: Beowulf in Enterprise Environments In-Reply-To: ; from phil.neumiller@convergys.com on Mon, Jun 16, 2003 at 05:49:35PM -0400 References: Message-ID: <20030617182127.H21711@blnsrv1.science-computing.de> -you mix 3 questions ;-) we do a linux cluster concept ( like John before i would more define it as a "clustered peer to NOW" for complete german car industries and Chip development companies No Beowulf (in strict sense) there, reason being that main requirements favorise and priorize other strategies simple one: you have a application with around 100 000 $ license costs per user and the vendor guarantees function only for a certain unmodified ! distro ... there are other and more important nontechnical issues ... On the other hand: I assume there competent american companies able to do exactly the same on a beowulf base / concept when other needs and conditions are not the main issue ... If you take losse Beowulf == Linux Cluster then: Clusters are acepted and broad spread for productive and very important company issues ... e.g every german car manufacturer now does Crash Simulation on Linux > > Does anybody on the list know of folks using Beowulf clusters in what might > be called > traditional IT or Enterprise environments? Why/Why not? > > What about in cryptographic acceleration environments? Why/Why not? > Perhaps for > stuff even as benign as SSL/TLS/IPsec terminations??? there is a hardware company in germany called Clusterlabs ( active also in US) that does exactly this: Blades with Hardware SSL Cards for different uses i am sure there are others also .. Micha _________________________creating IT solutions Michael Arndt Bereichsleiter IT-Services Berlin phone +49(0)30 72 62 38-50 fax +49(0)30 72 62 38-59 m.arndt at science-computing.de science + computing ag Ehrenbergstrasse 19 D-10245 Berlin, Germany www.science-computing.de _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From phil.neumiller at convergys.com Tue Jun 17 16:23:56 2003 From: phil.neumiller at convergys.com (phil.neumiller at convergys.com) Date: Tue, 17 Jun 2003 16:23:56 -0400 Subject: Enterprise Beowulf and Beowulf on a Chip Message-ID: In my own search for finding out whether Beowulf makes sense for enterprise computing (specifically on-line transaction processing (OLTP)), I found a paper at Compaq research: http://research.compaq.com/wrl/projects/Database/ specifically the paper http://research.compaq.com/wrl/projects/Database/isca00.pdf This work emphasizes "large memory stall times" as being a primary culprit limiting OLTP performance. The solution promoted by Compaq researchers in the paper is chip multiprocessing (CMP). This makes me wonder if OLTP can benefit from simply lots of processors (with fast interconnect) to utilize more L1, L2 cache simultaneously (a might bit larger than a chip I might add!!)... Their Piranha system (a research prototype) integrates eight Alpha processor cores on a chip with a two level cache with cache coherence architecture (sort of a micro-sub-cluster). If this approach makes sense for OLTP doesn't a Beowulf make sense for OLTP work now? Also... If Beowulf makes sense on the macro level does it make sense in the micro-level or perhaps in the fractal sense of a self similar architecture (exploiting even more hierarchy)? -- "NOTICE: The information contained in this electronic mail transmission is intended by Convergys Corporation for the use of the named individual or entity to which it is directed and may contain information that is privileged or otherwise confidential. If you have received this electronic mail transmission in error, please delete it from your system without copying or forwarding it, and notify the sender of the error by reply email or by telephone (collect), so that the sender's address records can be corrected." _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From Daniel.Kidger at quadrics.com Tue Jun 17 12:28:54 2003 From: Daniel.Kidger at quadrics.com (Daniel Kidger) Date: Tue, 17 Jun 2003 17:28:54 +0100 Subject: what is a flop Message-ID: <010C86D15E4D1247B9A5DD312B7F5AA78DDF87@stegosaurus.bristol.quadrics.com> Dear Saurav, Actually the third term should be 'Floating point operations per cycle' Usualy further refined to mean specifically 64-bit floats ('double-precision') In the past most processor took many cycles to do each floating point operation. Now with pipelining most processors can issue at least one floating point operation every machine cycle (note though that very few can issue divides every cycle) The P4 can normally do one flop per cycle - but it can do two (using its SSE2 unit) but only if both are multiplies (or both adds) *and* both pairs of operands are contiguous in memory. Some architectures have two seperate floating point units: one just adds, the other multiplies (can add too) and hence also have a '2' here Some architectures most notably the vector machines have 16 or more floating point units. Many architectures have fused mutiply-add units (Alpha, Itanium, Mips, Power3, etc.) They can add the result of a multiply directly to another register. Thus these also have '2' flops per cycle. Some archtectures have 2 (or more) 'muladd' units. Hence iirc Power3/4 and Itanium2 can yield 4 flops per cycle. Yours, Daniel. -------------------------------------------------------------- Dr. Dan Kidger, Quadrics Ltd. daniel.kidger at quadrics.com One Bridewell St., Bristol, BS1 2AA, UK 0117 915 5505 ----------------------- www.quadrics.com -------------------- -----Original Message----- From: Saurav Gohain [mailto:way2saurav at rediffmail.com] Sent: 17 June 2003 11:32 To: beowulf at beowulf.org Subject: what is a flop Dear Sir/Friends I am new to cluster and hence asking this question ( may be a silly one for u all). In super computers and cluster, I find Peak Performance rated as TFLOPs, GFLOP's etc. Now, the question is what is a flop. >From a beowulf pdf tutorial, i came to know that it is calculated as Total Peak Performance= No of Nodes * No of CPU's * Floating point operations per second * No of Cycles per second. Although, the first two and the last is easily derivable, what about the third one. I checked the specs of Opetron and P4 but there's isn't any mention about the Flops. How will i derive then ? In a comparison graph in Intel's website, I found a term SPEC*fp2000 ratings. Now this ratings are like 2.4 , 3.4 etc. Is it the number that I should input in the above formula to calculate the total peak performance in flops about a cluster. Kindly let me know the facts... Regards, Saurav _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From xyzzy at speakeasy.org Tue Jun 17 16:25:05 2003 From: xyzzy at speakeasy.org (Trent Piepho) Date: Tue, 17 Jun 2003 13:25:05 -0700 (PDT) Subject: Rack or Cabinet? Message-ID: We're planning on buying a few rackmount systems, probably not so many that they take more than one rack, and I'm trying to decide between an open rack or a cabinet to house the systems+ups+switch+etc in. I don't have much experience with rackmount systems, so I'm wondering if anyone here has any experiences to share. The noise of the computer room is annoying the people near it, so I'd like to get a cabinet if they're quieter, but I'm concerned about the cooling issues. Does the enclosed space cause cabinets to overheat? I see some come with a few fans on the top, but is that really enough? We have a cooled computer room, but no fancy underfloor AC ducts. >From what I've gathered about the pro/cons of the two. Rack pros: cheaper, usually < $500 open design probably results in better cooling cons: noisy doesn't look as nice, with exposed rat's nests of wires, etc. Cabinet pros: Looks nice, with plexyglass doors and hidden wires, making non-techie types more impressed with how their computer dollars were spent dampens sound cons: expensive, typical > $1500 lack of airflow may result in poor cooling _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From James.P.Lux at jpl.nasa.gov Tue Jun 17 16:41:47 2003 From: James.P.Lux at jpl.nasa.gov (Jim Lux) Date: Tue, 17 Jun 2003 13:41:47 -0700 Subject: what is a flop References: <20030617103217.2576.qmail@webmail28.rediffmail.com> Message-ID: <001701c33510$df4e37e0$02a8a8c0@office1> FLOPS and the related MIPS are attempts to quantify the amount of computational work being done. FLOPS = Floating Point Operations per second, MIPS = Million Instructions per Second. The real problem with such measures is that they work fine for a classic, non-pipelined, one instruction per clock kind of processor. However, if a multiply takes 100 times as long as an add, how do you count up the ops? How do you count speed on a processor where the number of instructions per second is highly dependent on the sequence of instructions (virtually any modern pipelined processor) or on cache hits/misses. Even the venerable Z80 and 8080 microprocessors have variable numbers of clocks required for instructions (conditional jumps taking one of two different lengths of time). Only on processors designed for "lockstep" processing (like a lot of DSP cores) is the instruction rate reasonably constant (and there, things like interrupts can screw up the timing... a real problem if your code has to service an i/o device every N clocks, regardless) The best you can do with such simple measures is look at the time required to execute some fairly well defined mathematical operation, with a fairly well defined number and kind of operations required.. A floating point FFT with no fancy tricks might be a good example. The number of multiplies and adds is well known, and independent of the data being transformed. FFT performance compared with some other algorithm can tell you something about the caching and pipeline performance. As many will tell you, the true measure of performance is how fast it goes with YOUR application... ----- Original Message ----- From: Saurav Gohain To: beowulf at beowulf.org Sent: Tuesday, June 17, 2003 3:32 AM Subject: what is a flop Dear Sir/Friends I am new to cluster and hence asking this question ( may be a silly one for u all). In super computers and cluster, I find Peak Performance rated as TFLOPs, GFLOP's etc. Now, the question is what is a flop. >From a beowulf pdf tutorial, i came to know that it is calculated as Total Peak Performance= No of Nodes * No of CPU's * Floating point operations per second * No of Cycles per second. Although, the first two and the last is easily derivable, what about the third one. I checked the specs of Opetron and P4 but there's isn't any mention about the Flops. How will i derive then ? In a comparison graph in Intel's website, I found a term SPEC*fp2000 ratings. Now this ratings are like 2.4 , 3.4 etc. Is it the number that I should input in the above formula to calculate the total peak performance in flops about a cluster. Kindly let me know the facts... Regards, Saurav _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rbw at ahpcrc.org Tue Jun 17 17:40:25 2003 From: rbw at ahpcrc.org (Richard Walsh) Date: Tue, 17 Jun 2003 16:40:25 -0500 Subject: what is a flop Message-ID: <200306172140.h5HLePP05878@mycroft.ahpcrc.org> Dan Kidger wrote: >The P4 can normally do one flop per cycle - but it can do two (using its >SSE2 unit) but only if both are multiplies (or both adds) *and* both pairs >of operands are contiguous in memory. > >Some architectures have two seperate floating point units: one just adds, >the other multiplies (can add too) and hence also have a '2' here > >Some architectures most notably the vector machines have 16 or more floating >point units. > >Many architectures have fused mutiply-add units (Alpha, Itanium, Mips, >Power3, etc.) They can add the result of a multiply directly to another >register. Thus these also have '2' flops per cycle. > >Some archtectures have 2 (or more) 'muladd' units. Hence iirc Power3/4 and >Itanium2 can yield 4 flops per cycle. A glass ;-) menagerie ... if you will. Another point to be made of the this, is that peak performance per processor is slippery to define ... where does the boundary of the processor actually rest? At the edge of the chip, the module, ... a safe definition is perhaps a processing core consisting of a pairing of independent FPUs and dedicated registers. Cray quotes 12.8 Gflops for their X1 MSP ... but that is for a multi-chip module ... which includes 2 FPUs (mul and add) x 2 vector pipes x 4 vector cores ... this is gives you Daniel's 16 FPUs. As an exercise compute the vector clock of the X1, the peak performance of a Power 4 core running at 1.3 GHz? How about the whole Power 4 chip? ;-) Finally, the relevance of the peak performance ends right about ... here. Cheers, rbw _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From waitt at saic.com Tue Jun 17 19:30:37 2003 From: waitt at saic.com (Tim Wait) Date: Tue, 17 Jun 2003 19:30:37 -0400 Subject: what is a flop References: <200306172140.h5HLePP05878@mycroft.ahpcrc.org> Message-ID: <3EEFA49D.3070709@saic.com> > Finally, the relevance of the peak performance ends right about ... here. Hah! Very well put! May I add another? /home/me/bin/FLOPS_benchmark > /dev/null _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From joelja at darkwing.uoregon.edu Tue Jun 17 20:56:07 2003 From: joelja at darkwing.uoregon.edu (Joel Jaeggli) Date: Tue, 17 Jun 2003 17:56:07 -0700 (PDT) Subject: Rack or Cabinet? In-Reply-To: Message-ID: cabinets are really ideal when you have raised floors and forced-air cooling, in the absence of that, an open-rack will probably run cooler... joelja On Tue, 17 Jun 2003, Trent Piepho wrote: > We're planning on buying a few rackmount systems, probably not so many > that they take more than one rack, and I'm trying to decide between an open > rack or a cabinet to house the systems+ups+switch+etc in. > > I don't have much experience with rackmount systems, so I'm wondering if > anyone here has any experiences to share. The noise of the computer room is > annoying the people near it, so I'd like to get a cabinet if they're quieter, > but I'm concerned about the cooling issues. Does the enclosed space cause > cabinets to overheat? I see some come with a few fans on the top, but is that > really enough? We have a cooled computer room, but no fancy underfloor AC > ducts. > > >From what I've gathered about the pro/cons of the two. > > Rack > pros: > cheaper, usually < $500 > open design probably results in better cooling > > cons: > noisy > doesn't look as nice, with exposed rat's nests of wires, etc. > > Cabinet > pros: > Looks nice, with plexyglass doors and hidden wires, making non-techie types > more impressed with how their computer dollars were spent > dampens sound > cons: > expensive, typical > $1500 > lack of airflow may result in poor cooling > > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org > To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf > -- -------------------------------------------------------------------------- Joel Jaeggli Academic User Services joelja at darkwing.uoregon.edu -- PGP Key Fingerprint: 1DE9 8FCA 51FB 4195 B42A 9C32 A30D 121E -- In Dr. Johnson's famous dictionary patriotism is defined as the last resort of the scoundrel. With all due respect to an enlightened but inferior lexicographer I beg to submit that it is the first. -- Ambrose Bierce, "The Devil's Dictionary" _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From laurenceliew at yahoo.com.sg Tue Jun 17 21:44:17 2003 From: laurenceliew at yahoo.com.sg (=?iso-8859-1?q?Laurence?=) Date: Wed, 18 Jun 2003 09:44:17 +0800 (CST) Subject: Enterprise Beowulf and Beowulf on a Chip In-Reply-To: Message-ID: <20030618014417.99050.qmail@web11904.mail.yahoo.com> Hi, One of the accepted definition of a beowulf is the use of low-cost COMMODITY processor... mostly INTEL and AMD cpus. If and when INTEL/AMD have low-cost multicpu on a single chip.. and is available in your DIY shop next door.... yes.. I guess it would make for a faster beowulf... As for enterprise beowulf and OLTP if this is what you are interested in.... OLTP usually means a database... so unless the database is parallel eg like Oracle 9i RAC, that would still be your bottleneck... you could queue many requests in a farm of request/compute nodes... but if you have a large DB.. it may not make sense... Oracle 9i RAC runs on a cluster... but this is not the usual definition of a beowulf however... you may want to explore PVFS + MySQL or Oracle... using PVFS which gives you parallel IO in a beowulf cluster... you may be able to speed up the IO.. it would help maybe not so much in OLTP but I believe in data-mining type, an analogy would be the genomics/blast people... who blast there sequences against a flat file database... and they do experiment with such parallel IO techniques... when you think enterprise.. those that would use beowulf would be those with intensive computation needs... Volvo, Audi, Nissan for car simulation, Philips, Toshiba for chip design, multi-media design houses etc etc.... and others... for "normal" enterprise... a beowulf "MAY" not be very useful... until the day when that complex spreadsheets can be computed not on your computer.. but sent to a Spreadsheet farm in the data centre behind you..... Hope this helps. Cheers! laurence --- phil.neumiller at convergys.com wrote: > In my own search for finding out whether Beowulf > makes sense for > enterprise computing (specifically on-line > transaction processing > (OLTP)), I found a paper at Compaq research: > > http://research.compaq.com/wrl/projects/Database/ > > specifically the paper > > http://research.compaq.com/wrl/projects/Database/isca00.pdf > > This work emphasizes "large memory stall times" as > being > a primary culprit limiting OLTP performance. The > solution > promoted by Compaq researchers in the paper is chip > multiprocessing (CMP). This makes me wonder if OLTP > can benefit from simply lots of processors (with > fast interconnect) > to utilize more L1, L2 cache simultaneously (a might > bit larger > than a chip I might add!!)... > > Their Piranha system (a research prototype) > integrates eight > Alpha processor cores on a chip with a two level > cache with > cache coherence architecture (sort of a > micro-sub-cluster). > > If this approach makes sense for OLTP doesn't a > Beowulf > make sense for OLTP work now? > > Also... > > If Beowulf makes sense on the macro level does it > make > sense in the micro-level or perhaps in the fractal > sense of > a self similar architecture (exploiting even more > hierarchy)? > > > -- > "NOTICE: The information contained in this > electronic mail transmission is > intended by Convergys Corporation for the use of the > named individual or > entity to which it is directed and may contain > information that is > privileged or otherwise confidential. If you have > received this electronic > mail transmission in error, please delete it from > your system without > copying or forwarding it, and notify the sender of > the error by reply email > or by telephone (collect), so that the sender's > address records can be > corrected." > > > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org > To change your subscription (digest mode or > unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf ===== --- Cheers! Laurence __________________________________________________ Do You Yahoo!? Send free SMS from your PC! http://sg.sms.yahoo.com _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From franz.marini at mi.infn.it Wed Jun 18 04:53:17 2003 From: franz.marini at mi.infn.it (Franz Marini) Date: Wed, 18 Jun 2003 10:53:17 +0200 (CEST) Subject: SMP CPUs scaling factors (was "what is a flop") In-Reply-To: <5.1.1.6.2.20030617135556.03e7b9a0@mail.harddata.com> Message-ID: Hi, On Tue, 17 Jun 2003, Maurice Hilarius wrote: > And I would say dual CPU boards do not sale at a factor of 2:1 over singles. > ... > As a general ( really general as it changes a lot with code and compilers) > the rule I know : > > Dual P3 ( VIA chipset): 1.5 : 1 > Dual XEON P4 ( Intel 7501 chipset): 1.3 : 1 Hrm. Now, this is a figure I find hard to believe... This would mean that, for a "general" (and thus I think you mean "not-overly-optimized") parallel (threaded, I guess) code, you'd get only a 30% gain in a dual Xeon system over a single one... Don't know, it seems a somewhat conservative figure to me. > Dual AthlonMP ( AMD 760MPX chipset) 1.4 : 1 (The same considerations applied to the Xeon holds true here ;)) Does anyone have some real world application figures regarding the performance ratio between single and two-way (and maybe four-way) SMP systems based on the P4 Xeon processor ? I'm particularly interested in molecular dynamics code, like, e.g., Gromacs and CPMD. Have a nice day, Franz --------------------------------------------------------- Franz Marini Sys Admin and Software Analyst, Dept. of Physics, University of Milan, Italy. email : franz.marini at mi.infn.it --------------------------------------------------------- _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From Steve_L_Smith at dell.com Wed Jun 18 06:27:58 2003 From: Steve_L_Smith at dell.com (Steve_L_Smith at dell.com) Date: Wed, 18 Jun 2003 05:27:58 -0500 Subject: Rack or Cabinet Message-ID: <5D0C92E65C29D311A24D0090273C2168079973A6@BRKXMTCH02_b> Trent You should take the advice of the system vendor. At Dell (this is not an advert:-) we design our systems (as do all tier 1s) for mounting in enclosed racks (including side panels and doors). This is to ensure correct cooling. Running something like a Dell PowerEdge 1750 1U DP Xeon in an open cabinet is definitely not recommended. The systems are designed to have front to back cooling, allowing air to circulate randomly around the sides and tops upsets this airflow and will cause the heat within the server to rise, possibly leading to over-temperature situations. And if you do not fill a complete rack, you should also use blanking panels to maintain the airflow - also note that this also applies to e.g. a Myrinet switch - you should use blanking panels if you do not fully populate the switch chassis. Unfortunately noise is not something that we can do much about at the moment - putting hot processors in small spaces means we need big/fast (hence somewhat noisy) fans to push the air out of our box into your room! Hope this helps Steve ------------------------------------------------- Steve Smith HPC Business Manager Dell EMEA Dell Campus, Cain Road, Bracknell, RG12 1FA, UK Direct: +44 1344 372037 Switchboard: +44 1344 812000 Fax: +44 1344 372359 Mobile: +44 7802 594874 email: steve_l_smith at dell.com ------------------------------------------------------ --__--__-- Message: 2 Date: Tue, 17 Jun 2003 13:25:05 -0700 (PDT) From: Trent Piepho To: Beowulf Mailing list Subject: Rack or Cabinet? We're planning on buying a few rackmount systems, probably not so many that they take more than one rack, and I'm trying to decide between an open rack or a cabinet to house the systems+ups+switch+etc in. I don't have much experience with rackmount systems, so I'm wondering if anyone here has any experiences to share. The noise of the computer room is annoying the people near it, so I'd like to get a cabinet if they're quieter, but I'm concerned about the cooling issues. Does the enclosed space cause cabinets to overheat? I see some come with a few fans on the top, but is that really enough? We have a cooled computer room, but no fancy underfloor AC ducts. >From what I've gathered about the pro/cons of the two. Rack pros: cheaper, usually < $500 open design probably results in better cooling cons: noisy doesn't look as nice, with exposed rat's nests of wires, etc. Cabinet pros: Looks nice, with plexyglass doors and hidden wires, making non-techie types more impressed with how their computer dollars were spent dampens sound cons: expensive, typical > $1500 lack of airflow may result in poor cooling --__--__-- _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From alvin at Maggie.Linux-Consulting.com Wed Jun 18 06:53:47 2003 From: alvin at Maggie.Linux-Consulting.com (Alvin Oga) Date: Wed, 18 Jun 2003 03:53:47 -0700 (PDT) Subject: Rack or Cabinet In-Reply-To: <5D0C92E65C29D311A24D0090273C2168079973A6@BRKXMTCH02_b> Message-ID: hi ya steve On Wed, 18 Jun 2003 Steve_L_Smith at dell.com wrote: > Trent > > You should take the advice of the system vendor. At Dell (this is not an > advert:-) we design our systems (as do all tier 1s) for mounting in enclosed > racks (including side panels and doors). This is to ensure correct cooling. > Running something like a Dell PowerEdge 1750 1U DP Xeon in an open cabinet > is definitely not recommended. The systems are designed to have front to for the dual xeons or dual-p4s, what do your normal temperatures run at say using normal 25C or normal 50F computer room temps ??? > back cooling, allowing air to circulate randomly around the sides and tops > upsets this airflow and will cause the heat within the server to rise, > possibly leading to over-temperature situations. And if you do not fill a > complete rack, you should also use blanking panels to maintain the airflow - > also note that this also applies to e.g. a Myrinet switch - you should use > blanking panels if you do not fully populate the switch chassis. and if the air in the chassis is meant to go side to side ... the cabinet airflow should also be doing the same ... > Unfortunately noise is not something that we can do much about at the moment > - putting hot processors in small spaces means we need big/fast (hence > somewhat noisy) fans to push the air out of our box into your room! you can cut some noice by making bigger holes for fans instead of small holes ... holes as big as the diameter of the fan blades .. less interference from the air flow squeezing thru/pass the obstacle but than again, that's not possible/realistic for making 12" diameter holes if one is using 12" vent fans i have a p4-2.0G running at 28C ... am nicely cool and happy... in normal 25C room temp ... w/ normal fans (noisy) in a 1U mini-itx box.. -- am thinking the bios sensor must be whacky, but the air is in fact cool as is the heatsinks ... and its happily calculating away have fun alvin _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From Steve_L_Smith at dell.com Wed Jun 18 06:27:58 2003 From: Steve_L_Smith at dell.com (Steve_L_Smith at dell.com) Date: Wed, 18 Jun 2003 05:27:58 -0500 Subject: Rack or Cabinet Message-ID: <5D0C92E65C29D311A24D0090273C2168079973A6@BRKXMTCH02_b> Trent You should take the advice of the system vendor. At Dell (this is not an advert:-) we design our systems (as do all tier 1s) for mounting in enclosed racks (including side panels and doors). This is to ensure correct cooling. Running something like a Dell PowerEdge 1750 1U DP Xeon in an open cabinet is definitely not recommended. The systems are designed to have front to back cooling, allowing air to circulate randomly around the sides and tops upsets this airflow and will cause the heat within the server to rise, possibly leading to over-temperature situations. And if you do not fill a complete rack, you should also use blanking panels to maintain the airflow - also note that this also applies to e.g. a Myrinet switch - you should use blanking panels if you do not fully populate the switch chassis. Unfortunately noise is not something that we can do much about at the moment - putting hot processors in small spaces means we need big/fast (hence somewhat noisy) fans to push the air out of our box into your room! Hope this helps Steve ------------------------------------------------- Steve Smith HPC Business Manager Dell EMEA Dell Campus, Cain Road, Bracknell, RG12 1FA, UK Direct: +44 1344 372037 Switchboard: +44 1344 812000 Fax: +44 1344 372359 Mobile: +44 7802 594874 email: steve_l_smith at dell.com ------------------------------------------------------ --__--__-- Message: 2 Date: Tue, 17 Jun 2003 13:25:05 -0700 (PDT) From: Trent Piepho To: Beowulf Mailing list Subject: Rack or Cabinet? We're planning on buying a few rackmount systems, probably not so many that they take more than one rack, and I'm trying to decide between an open rack or a cabinet to house the systems+ups+switch+etc in. I don't have much experience with rackmount systems, so I'm wondering if anyone here has any experiences to share. The noise of the computer room is annoying the people near it, so I'd like to get a cabinet if they're quieter, but I'm concerned about the cooling issues. Does the enclosed space cause cabinets to overheat? I see some come with a few fans on the top, but is that really enough? We have a cooled computer room, but no fancy underfloor AC ducts. >From what I've gathered about the pro/cons of the two. Rack pros: cheaper, usually < $500 open design probably results in better cooling cons: noisy doesn't look as nice, with exposed rat's nests of wires, etc. Cabinet pros: Looks nice, with plexyglass doors and hidden wires, making non-techie types more impressed with how their computer dollars were spent dampens sound cons: expensive, typical > $1500 lack of airflow may result in poor cooling --__--__-- _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From alvin at Maggie.Linux-Consulting.com Wed Jun 18 06:53:47 2003 From: alvin at Maggie.Linux-Consulting.com (Alvin Oga) Date: Wed, 18 Jun 2003 03:53:47 -0700 (PDT) Subject: Rack or Cabinet In-Reply-To: <5D0C92E65C29D311A24D0090273C2168079973A6@BRKXMTCH02_b> Message-ID: hi ya steve On Wed, 18 Jun 2003 Steve_L_Smith at dell.com wrote: > Trent > > You should take the advice of the system vendor. At Dell (this is not an > advert:-) we design our systems (as do all tier 1s) for mounting in enclosed > racks (including side panels and doors). This is to ensure correct cooling. > Running something like a Dell PowerEdge 1750 1U DP Xeon in an open cabinet > is definitely not recommended. The systems are designed to have front to for the dual xeons or dual-p4s, what do your normal temperatures run at say using normal 25C or normal 50F computer room temps ??? > back cooling, allowing air to circulate randomly around the sides and tops > upsets this airflow and will cause the heat within the server to rise, > possibly leading to over-temperature situations. And if you do not fill a > complete rack, you should also use blanking panels to maintain the airflow - > also note that this also applies to e.g. a Myrinet switch - you should use > blanking panels if you do not fully populate the switch chassis. and if the air in the chassis is meant to go side to side ... the cabinet airflow should also be doing the same ... > Unfortunately noise is not something that we can do much about at the moment > - putting hot processors in small spaces means we need big/fast (hence > somewhat noisy) fans to push the air out of our box into your room! you can cut some noice by making bigger holes for fans instead of small holes ... holes as big as the diameter of the fan blades .. less interference from the air flow squeezing thru/pass the obstacle but than again, that's not possible/realistic for making 12" diameter holes if one is using 12" vent fans i have a p4-2.0G running at 28C ... am nicely cool and happy... in normal 25C room temp ... w/ normal fans (noisy) in a 1U mini-itx box.. -- am thinking the bios sensor must be whacky, but the air is in fact cool as is the heatsinks ... and its happily calculating away have fun alvin _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From k_kolokoutsas at yahoo.co.uk Wed Jun 18 06:09:37 2003 From: k_kolokoutsas at yahoo.co.uk (=?iso-8859-1?q?kolokoutsas=20konstantinos?=) Date: Wed, 18 Jun 2003 11:09:37 +0100 (BST) Subject: ideal motherboard for diskless beowulf In-Reply-To: <200306180855.h5I8tJU30389@NewBlue.Scyld.com> Message-ID: <20030618100937.9589.qmail@web9907.mail.yahoo.com> John, thanks for your comments. My background is in Stellar Astrophysics (Imperial), however the beowulf is for a friend postdoc doing relativistic shock particle acceleration at MPIfA. Not quite sure about the characteristics of her Monte Carlo code, so can't comment on library usage/dependencies etc. My interest in this particular exercise has more to do with hardware, and to identify what would be the best micro-atx Athlon/Athlon XP motherboard for a beowulf cluster within budget. Yes, the mini-ITX form factor is a quite interesting possibility. However, I have also come across the EBX form factor (5.25"). Such motherboards, primarily used in embedded systems are manufactured by www.ITOX.com, can take up to 1GHZ P3/Celeron CPUs, with various power supply configs and are prohibitely expensive(~$300 without CPU/memory), for the budget of this beowulf at least. Kostas _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From Steve_L_Smith at dell.com Wed Jun 18 08:34:29 2003 From: Steve_L_Smith at dell.com (Steve_L_Smith at dell.com) Date: Wed, 18 Jun 2003 07:34:29 -0500 Subject: Rack or Cabinet Message-ID: <5D0C92E65C29D311A24D0090273C2168079973A9@BRKXMTCH02_b> Hi Alvin Our systems are designed for a maximum inlet temperature of 35C, and a front to back rise of up to 5C - this is pretty extreme. Normally (good airflow through the system, not too many cables and a system that is not running at max - full of disks and memory and really working hard!) we'd expect a 2C rise, so your 28C in a 25C inlet room is about right. The other thing I didn't mention is that in a full rack (e.g. 42 x 2P Xeon systems) you would expect to see up to 11C rise in temperature from the bottom of the rack to the top - again extreme but worth bearing in mind when deploying full racks of hot boxes. Not sure I'd recommend people cutting big holes for the fans - unless it isn't a Dell of course:-) Cheers Steve -----Original Message----- From: Alvin Oga [mailto:alvin at Maggie.Linux-Consulting.com] Sent: 18 June 2003 11:54 To: Steve_L_Smith at exchange.dell.com Cc: beowulf at scyld.com; beowulf at beowulf.org Subject: Re:Rack or Cabinet hi ya steve On Wed, 18 Jun 2003 Steve_L_Smith at dell.com wrote: > Trent > > You should take the advice of the system vendor. At Dell (this is not an > advert:-) we design our systems (as do all tier 1s) for mounting in enclosed > racks (including side panels and doors). This is to ensure correct cooling. > Running something like a Dell PowerEdge 1750 1U DP Xeon in an open cabinet > is definitely not recommended. The systems are designed to have front to for the dual xeons or dual-p4s, what do your normal temperatures run at say using normal 25C or normal 50F computer room temps ??? > back cooling, allowing air to circulate randomly around the sides and tops > upsets this airflow and will cause the heat within the server to rise, > possibly leading to over-temperature situations. And if you do not fill a > complete rack, you should also use blanking panels to maintain the airflow - > also note that this also applies to e.g. a Myrinet switch - you should use > blanking panels if you do not fully populate the switch chassis. and if the air in the chassis is meant to go side to side ... the cabinet airflow should also be doing the same ... > Unfortunately noise is not something that we can do much about at the moment > - putting hot processors in small spaces means we need big/fast (hence > somewhat noisy) fans to push the air out of our box into your room! you can cut some noice by making bigger holes for fans instead of small holes ... holes as big as the diameter of the fan blades .. less interference from the air flow squeezing thru/pass the obstacle but than again, that's not possible/realistic for making 12" diameter holes if one is using 12" vent fans i have a p4-2.0G running at 28C ... am nicely cool and happy... in normal 25C room temp ... w/ normal fans (noisy) in a 1U mini-itx box.. -- am thinking the bios sensor must be whacky, but the air is in fact cool as is the heatsinks ... and its happily calculating away have fun alvin _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From Steve_L_Smith at dell.com Wed Jun 18 08:34:29 2003 From: Steve_L_Smith at dell.com (Steve_L_Smith at dell.com) Date: Wed, 18 Jun 2003 07:34:29 -0500 Subject: Rack or Cabinet Message-ID: <5D0C92E65C29D311A24D0090273C2168079973A9@BRKXMTCH02_b> Hi Alvin Our systems are designed for a maximum inlet temperature of 35C, and a front to back rise of up to 5C - this is pretty extreme. Normally (good airflow through the system, not too many cables and a system that is not running at max - full of disks and memory and really working hard!) we'd expect a 2C rise, so your 28C in a 25C inlet room is about right. The other thing I didn't mention is that in a full rack (e.g. 42 x 2P Xeon systems) you would expect to see up to 11C rise in temperature from the bottom of the rack to the top - again extreme but worth bearing in mind when deploying full racks of hot boxes. Not sure I'd recommend people cutting big holes for the fans - unless it isn't a Dell of course:-) Cheers Steve -----Original Message----- From: Alvin Oga [mailto:alvin at Maggie.Linux-Consulting.com] Sent: 18 June 2003 11:54 To: Steve_L_Smith at exchange.dell.com Cc: beowulf at scyld.com; beowulf at beowulf.org Subject: Re:Rack or Cabinet hi ya steve On Wed, 18 Jun 2003 Steve_L_Smith at dell.com wrote: > Trent > > You should take the advice of the system vendor. At Dell (this is not an > advert:-) we design our systems (as do all tier 1s) for mounting in enclosed > racks (including side panels and doors). This is to ensure correct cooling. > Running something like a Dell PowerEdge 1750 1U DP Xeon in an open cabinet > is definitely not recommended. The systems are designed to have front to for the dual xeons or dual-p4s, what do your normal temperatures run at say using normal 25C or normal 50F computer room temps ??? > back cooling, allowing air to circulate randomly around the sides and tops > upsets this airflow and will cause the heat within the server to rise, > possibly leading to over-temperature situations. And if you do not fill a > complete rack, you should also use blanking panels to maintain the airflow - > also note that this also applies to e.g. a Myrinet switch - you should use > blanking panels if you do not fully populate the switch chassis. and if the air in the chassis is meant to go side to side ... the cabinet airflow should also be doing the same ... > Unfortunately noise is not something that we can do much about at the moment > - putting hot processors in small spaces means we need big/fast (hence > somewhat noisy) fans to push the air out of our box into your room! you can cut some noice by making bigger holes for fans instead of small holes ... holes as big as the diameter of the fan blades .. less interference from the air flow squeezing thru/pass the obstacle but than again, that's not possible/realistic for making 12" diameter holes if one is using 12" vent fans i have a p4-2.0G running at 28C ... am nicely cool and happy... in normal 25C room temp ... w/ normal fans (noisy) in a 1U mini-itx box.. -- am thinking the bios sensor must be whacky, but the air is in fact cool as is the heatsinks ... and its happily calculating away have fun alvin _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From bogdan.costescu at iwr.uni-heidelberg.de Wed Jun 18 09:38:46 2003 From: bogdan.costescu at iwr.uni-heidelberg.de (Bogdan Costescu) Date: Wed, 18 Jun 2003 15:38:46 +0200 (CEST) Subject: SMP CPUs scaling factors (was "what is a flop") In-Reply-To: Message-ID: On Wed, 18 Jun 2003, Franz Marini wrote: > Does anyone have some real world application figures regarding the > performance ratio between single and two-way (and maybe four-way) SMP > systems based on the P4 Xeon processor ? I'm particularly interested in > molecular dynamics code, like, e.g., Gromacs and CPMD. If CHARMM figures are also valid: - when running 2 single-CPU jobs on a 2-way system, real (wall) time of each process is the same as if the process was running alone. - when running a parallel job with shmem/usysv the speed-up is close to 2 (1.98 or so). This is with "normal" MD. Using PME might make things different... -- Bogdan Costescu IWR - Interdisziplinaeres Zentrum fuer Wissenschaftliches Rechnen Universitaet Heidelberg, INF 368, D-69120 Heidelberg, GERMANY Telephone: +49 6221 54 8869, Telefax: +49 6221 54 8868 E-mail: Bogdan.Costescu at IWR.Uni-Heidelberg.De _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rgb at phy.duke.edu Wed Jun 18 11:20:03 2003 From: rgb at phy.duke.edu (Robert G. Brown) Date: Wed, 18 Jun 2003 11:20:03 -0400 (EDT) Subject: Rack or Cabinet? In-Reply-To: Message-ID: On Tue, 17 Jun 2003, Trent Piepho wrote: > Rack > pros: > cheaper, usually < $500 > open design probably results in better cooling > > cons: > noisy > doesn't look as nice, with exposed rat's nests of wires, etc. > > Cabinet > pros: > Looks nice, with plexyglass doors and hidden wires, making non-techie types > more impressed with how their computer dollars were spent > dampens sound > cons: > expensive, typical > $1500 > lack of airflow may result in poor cooling Yup, I think you've about got it. Except that I think that closed cabinets are often power-ventilated with directed AC so that cooling may not be that much of a problem, to be fair. To me, the more than $1000 difference in cost is pretty much everything. We keep all the racks in a room where sound doesn't matter and there are enough nodes that the impression factor doesn't matter much either. It's a workingperson's cluster vs a show cluster kind of thing...;-) rgb -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb at phy.duke.edu _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From lathama at yahoo.com Wed Jun 18 11:08:50 2003 From: lathama at yahoo.com (Andrew Latham) Date: Wed, 18 Jun 2003 08:08:50 -0700 (PDT) Subject: power supplies Message-ID: <20030618150850.70991.qmail@web80513.mail.yahoo.com> Has any one used or liked any AT/ATX external power supplies. My clusters are using minimal power. The power supplys are wasting energy converting(heat). I see the advantage to building a rackmount power converter and playing with independant ATX swithing. What are some ideas or comments about such a project. ===== Andrew Latham LathamA.com lathama at lathama.com lathama at yahoo.com LathamA(lay-th-ham-eh) _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From Daniel.Kidger at quadrics.com Wed Jun 18 13:54:10 2003 From: Daniel.Kidger at quadrics.com (Daniel Kidger) Date: Wed, 18 Jun 2003 18:54:10 +0100 Subject: what is a flop Message-ID: <010C86D15E4D1247B9A5DD312B7F5AA78DDF8E@stegosaurus.bristol.quadrics.com> -----Original Message----- >From: Mikhail Kuzminsky [mailto:kus at free.net] >Sent: None >To: roberto.ammendola at roma2.infn.it >Cc: beowulf at beowulf.org >Subject: Re: what is a flop > > >According to Roberto Ammendola >> The "Floating point operations per clock cycle" depends on the >> processor, obviously, and on which instructions you use in your code. >> For example in a processor with the SSE instruction set you can perform >> 4 operations (on 32 bit register each) per clock cycle. One processor >> (Xeon or P4) running at > Taking into account that throughput of FMUL and FADD units in >P4/Xeon is 2 cycles, i.e. FP result may be received on any 2nd sycle >only, the peak Performance of P4/2 Ghz must be 4 GFLOPS. IMHO You are both correct and also wrong at the same time. The P4/Xeon *can* do 8 Gflop/s but only in 'single-precision'. It can do this by issueing just one SSE2 instruction but that instruction does 4 muls (or adds) on a 128-bit load. (as 4*4byte consecutive floats). compare with doing 2 muls (or adds) on 2*16 consecutive 8byte 'doubles'). A Flop is usually defines as being on a number of at least 64bits. iirc the P4/Xeon can only issue one floating point instruction per cycle, and so outside of the SSE2 unit it can only achive clockspeed Gflops/s. Hence a 2.0 GHz P4/Xeon should be quoted as 4 GigaFlops peak As a side note SSE2 only works to the standard 64 bits, but the FPUs work to 80bits, hence you often get slightly different numerical results when comparing IEEE maths between the P4 FPU and the SSE2 or indeed between the P4 and say an Alpha. Yours, Daniel. -------------------------------------------------------------- Dr. Dan Kidger, Quadrics Ltd. daniel.kidger at quadrics.com One Bridewell St., Bristol, BS1 2AA, UK 0117 915 5505 ----------------------- www.quadrics.com -------------------- _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From joelja at darkwing.uoregon.edu Wed Jun 18 14:24:34 2003 From: joelja at darkwing.uoregon.edu (Joel Jaeggli) Date: Wed, 18 Jun 2003 11:24:34 -0700 (PDT) Subject: power supplies In-Reply-To: <20030618150850.70991.qmail@web80513.mail.yahoo.com> Message-ID: If you carry 3.3volts for any significant distance you'll have a substantial voltage drop the ammount of courrent you can will quickly result in larger conductors... you can get nebs compliant -48volt dc atx power supplies, then just use use a hanging huge rectifier (or two) to power the whole stack. but then you have the joy of working with large amounts of dc power... joelja On Wed, 18 Jun 2003, Andrew Latham wrote: > Has any one used or liked any AT/ATX external power supplies. My clusters are > using minimal power. The power supplys are wasting energy converting(heat). > I see the advantage to building a rackmount power converter and playing with > independant ATX swithing. What are some ideas or comments about such a project. > > > ===== > Andrew Latham > LathamA.com > lathama at lathama.com > lathama at yahoo.com > LathamA(lay-th-ham-eh) > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org > To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf > -- -------------------------------------------------------------------------- Joel Jaeggli Academic User Services joelja at darkwing.uoregon.edu -- PGP Key Fingerprint: 1DE9 8FCA 51FB 4195 B42A 9C32 A30D 121E -- In Dr. Johnson's famous dictionary patriotism is defined as the last resort of the scoundrel. With all due respect to an enlightened but inferior lexicographer I beg to submit that it is the first. -- Ambrose Bierce, "The Devil's Dictionary" _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From thakur at mcs.anl.gov Wed Jun 18 15:19:57 2003 From: thakur at mcs.anl.gov (Rajeev Thakur) Date: Wed, 18 Jun 2003 14:19:57 -0500 Subject: MPI-REDUCE-TO-ALL In-Reply-To: <200306181904.h5IJ4eU10906@NewBlue.Scyld.com> Message-ID: > Date: Wed, 18 Jun 2003 16:02:22 -0300 > From: Renato Silva > Cc: beowulf at beowulf.org > > Hi Folks > > Does anybody knows what are the communication algorithm used in the > mpich-2.5 > to do the Reduce-to-all ? > > Thanks > Renato Silva All the collective algorithms are described in the source code. See the file src/coll/intra_fns_new.c and look for MPI_Allreduce. The next release of MPICH will have a different algorithm for long-message allreduce. Rajeev _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rssr at lncc.br Wed Jun 18 15:02:22 2003 From: rssr at lncc.br (Renato Silva) Date: Wed, 18 Jun 2003 16:02:22 -0300 Subject: MPI-REDUCE-TO-ALL References: <033001c33389$5dde1f00$27b358c7@titan> <3EED8370.ADD04801@ideafix.litec.csic.es> Message-ID: <3EF0B73E.629EF049@lncc.br> Hi Folks Does anybody knows what are the communication algorithm used in the mpich-2.5 to do the Reduce-to-all ? Thanks Renato Silva _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From alvin at Maggie.Linux-Consulting.com Wed Jun 18 16:32:23 2003 From: alvin at Maggie.Linux-Consulting.com (Alvin Oga) Date: Wed, 18 Jun 2003 13:32:23 -0700 (PDT) Subject: power supplies In-Reply-To: <20030618150850.70991.qmail@web80513.mail.yahoo.com> Message-ID: hi ya andrew if you have access to the outside ... ( sending exhaust air outside ) than, i'd use 48v or 12 car batteries or sealed lead/acid batteries use a dc-dc converter to drive your +12vdc atx power supply or -48vdc atx power supply - but these +12vdc atx power supp is expensive compared to $20 110vac atx supply - we're using an $5 ac adaptor to give +12vdc out and plugs into a custom dc-dc convertor ( $50 ) to give us about 100W atx power supply capacity have fun alvin On Wed, 18 Jun 2003, Andrew Latham wrote: > Has any one used or liked any AT/ATX external power supplies. My clusters are > using minimal power. The power supplys are wasting energy converting(heat). > I see the advantage to building a rackmount power converter and playing with > independant ATX swithing. What are some ideas or comments about such a project. > _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From dinamica at webcable.com.br Wed Jun 18 15:41:02 2003 From: dinamica at webcable.com.br (Dinamica) Date: Wed, 18 Jun 2003 16:41:02 -0300 Subject: MPI and commons in FORTRAN Message-ID: <20030618194102.GA3233@webcable.com.br> Hi list, I am afraid this is a bit off-topic but i have searched the web and could not find any helpful pointer about this. Pls, let me know if you can help me or if there is another forum to post this: I would like to parallelize just one subroutine call (the func/error evaluation) of a large FORTRAN code (the genetic algorhythm by Carrol). I have tried to code this in a couple of ways: 1) MPI_INIT'ing in the main routine or 2) MPI_INIT'ing only in the routine that contains the loop (to be parallelized) and actually makes the call to the func/error evaluation. None of them work and i am afraid i am missing something regarding the way common blocks are handled in MPI/parallel codes. Since this is the first time i try something more elaborate (IMHO) with MPI and FORTRAN, I am a bit lost and would like to know if there is any documentation or examples i can read more about this. Sorry if this looks vague, i can give more details if you want. Thanks a lot for your help, Guilherme Menegon IQUSP, Brasil --------------------------- _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From mphil39 at hotmail.com Wed Jun 18 18:05:22 2003 From: mphil39 at hotmail.com (Matt Phillips) Date: Wed, 18 Jun 2003 18:05:22 -0400 Subject: Beowulf in Enterprise Environments Message-ID: On Tue, 17 Jun 2003, Robert G. Brown wrote: >crack (the venerable Unix password cracking program) can be run >embarrassingly parallel over a network and in fact has been programmed >to do so on request for a rather long time now. Sounds reasonable that it can be made embarrassingly parallel. However, its creator Alec Muffett continue s to deny that it can/should be parallelized. http://www.crypticide.org/users/alecm/security/c50-faq.html (Micsellany section Q 5&6 ) Matt _________________________________________________________________ Protect your PC - get McAfee.com VirusScan Online http://clinic.mcafee.com/clinic/ibuy/campaign.asp?cid=3963 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rgb at phy.duke.edu Wed Jun 18 21:05:59 2003 From: rgb at phy.duke.edu (Robert G. Brown) Date: Wed, 18 Jun 2003 21:05:59 -0400 (EDT) Subject: Beowulf in Enterprise Environments In-Reply-To: Message-ID: On Wed, 18 Jun 2003, Matt Phillips wrote: > > On Tue, 17 Jun 2003, Robert G. Brown wrote: > >crack (the venerable Unix password cracking program) can be run > >embarrassingly parallel over a network and in fact has been programmed > >to do so on request for a rather long time now. > > Sounds reasonable that it can be made embarrassingly parallel. However, its > creator Alec Muffett continue s to deny that it can/should be parallelized. > > http://www.crypticide.org/users/alecm/security/c50-faq.html (Micsellany > section Q 5&6 ) You mean the part of the README where network mode is described? As in: %< Snip ============== * conf/network.conf This is the file used to configure Crack for network running; this file contains lines, each of which has several fields: host:relpow:nfsbool:rshuser:crackdir Where: o host This is the name of the host to which Crack should "rsh", in order to despatch a job. There can be several instances of the same hostname in the file, if desired, in order to dispatch more than one job to a given host (if it has more than one CPU, for instance). %< Snip ============== Sounds like integrated, embarrassingly parallel operation to me... Works that way too, back when I used to run crack on passwd files;-) rgb -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb at phy.duke.edu _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From hahn at physics.mcmaster.ca Wed Jun 18 22:19:37 2003 From: hahn at physics.mcmaster.ca (Mark Hahn) Date: Wed, 18 Jun 2003 22:19:37 -0400 (EDT) Subject: power supplies In-Reply-To: Message-ID: > If you carry 3.3volts for any significant distance you'll have a > substantial voltage drop the ammount of courrent you can will quickly > result in larger conductors... I have this remarkable image in my mind of 20 or so lovely EATX dual-opteron motherboards arranged radially like fins around a couple of heavy-gauge copper cables running vertically through a cylindrical rack, with a monster fan at the bottom. it would be lovely, and perhaps even worthy of a cray-ish couch around it ;) heck, such an arrangement could even cut down on that murderous time-of-flight component in gigabit ethernet performance :) _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From hahn at physics.mcmaster.ca Wed Jun 18 23:03:09 2003 From: hahn at physics.mcmaster.ca (Mark Hahn) Date: Wed, 18 Jun 2003 23:03:09 -0400 (EDT) Subject: Enterprise Beowulf and Beowulf on a Chip In-Reply-To: Message-ID: > In my own search for finding out whether Beowulf makes sense for > enterprise computing (specifically on-line transaction processing > (OLTP)), OLTP has significant throughput-type parallelism, so can be done by clusters quite nicely. it could be that you'd want a low-latency interconnect for lockmanager-type synchronization, but wouldn't necessarily need much bandwidth. > This work emphasizes "large memory stall times" as being > a primary culprit limiting OLTP performance. The solution right: OLTP is computationally trivial. so you can either use CMP-type parallelism (where stalls still waste the CPU for the duration, but CPUs are cheap) or SMT (which they reject for a practical reason, namely that CMP is a lot easier to throw together.) > multiprocessing (CMP). This makes me wonder if OLTP > can benefit from simply lots of processors (with fast interconnect) > to utilize more L1, L2 cache simultaneously (a might bit larger > than a chip I might add!!)... it's not clear how much good cache does you for OLTP - if you look at figure 6, you see that miss rate decreases between P4 and P8 - that is, when you have 8 1M caches, you start to approach a working set. in figure 8, you see that misses are still something like 33% of execution time, though, so even 12M cache is not terribly effective. > If this approach makes sense for OLTP doesn't a Beowulf > make sense for OLTP work now? clusters are great for OLTP, but Beowulf is usually considered a fairly specific kind of cluster, which is tuned in ways that are not much of advantage for OLTP. > If Beowulf makes sense on the macro level does it make > sense in the micro-level or perhaps in the fractal sense of > a self similar architecture (exploiting even more hierarchy)? I do not believe there is any problem obtaining whatever OLTP performance you need. do you need 700K tpmC? I really don't think so. making OLTP cheap (even at modest performance) is an entirely different topic. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From auro at nietzsche.mit.edu Wed Jun 18 23:22:25 2003 From: auro at nietzsche.mit.edu (Stefano) Date: Wed, 18 Jun 2003 23:22:25 -0400 (EDT) Subject: cluster of AOD Opteron In-Reply-To: Message-ID: Hi guys, As I am going to receive some funding this fall, I was wondering of buying an opteron cluster for my research. Mainlym the cluster will run VASP (an ab-initio quantum program, written by a group in Wien), with myrinet. Is somebody who is using AMD opterons yet ? Any myrinet trouble ? Is somebody from AMD reading ? I think some fortran vendor has announced the port of their F90 to the opteron. Well, it would be nice to recompile VASP for 64bits and see how fast it goes. With the itanium2 (compiled in 2 version 32 and 64 bits), it not so fast to justify the HUGE cost of an itanium cluster. Maybe the opteron will shake high-performace scientific computing ! Regards, Stefano Curtarolo _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From kpodesta at redbrick.dcu.ie Thu Jun 19 07:20:49 2003 From: kpodesta at redbrick.dcu.ie (Karl Podesta) Date: Thu, 19 Jun 2003 12:20:49 +0100 Subject: Beowulf in Enterprise Environments In-Reply-To: References: Message-ID: <20030619112048.GE12882@prodigy.Redbrick.DCU.IE> On Wed, Jun 18, 2003 at 09:05:59PM -0400, Robert G. Brown wrote: > You mean the part of the README where network mode is described? As in: > - SNIP - > Sounds like integrated, embarrassingly parallel operation to me... > > Works that way too, back when I used to run crack on passwd files;-) > > rgb The author seems to be talking about parallelising the actual crypt() algorithm, to do a single crack accross many CPUs, and maintains it's like 9 women trying to produce 1 baby in 1 month :-) So perhaps a bit of mis-information on his side about parallelising operations? When he could simply run multiple instances of crack accross a network like you describe, a good oul bit of data/domain decomposition, maybe put an MPI wrapper round it to collate results or nicey it up for running it from a head node accross a cluster - that's maybe what people were requesting from him and he mis-understood. Kp -- Karl Podesta Dublin City University, Ireland _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rgb at phy.duke.edu Thu Jun 19 10:08:04 2003 From: rgb at phy.duke.edu (Robert G. Brown) Date: Thu, 19 Jun 2003 10:08:04 -0400 (EDT) Subject: Beowulf in Enterprise Environments In-Reply-To: <20030619112048.GE12882@prodigy.Redbrick.DCU.IE> Message-ID: On Thu, 19 Jun 2003, Karl Podesta wrote: > The author seems to be talking about parallelising the actual crypt() > algorithm, to do a single crack accross many CPUs, and maintains it's > like 9 women trying to produce 1 baby in 1 month :-) > > So perhaps a bit of mis-information on his side about parallelising > operations? When he could simply run multiple instances of crack accross > a network like you describe, a good oul bit of data/domain decomposition, > maybe put an MPI wrapper round it to collate results or nicey it up > for running it from a head node accross a cluster - that's maybe what > people were requesting from him and he mis-understood. You got me. As the crack README clearly states, it has been programmed to run (nearly EP) over a network from very early versions of crack. Crack works by applying rules and transformations to dictionaries to generate a trial password, crypting the trial password, and comparing the crypted result to the crypts in the passwd file, which is why the crypts were split off in public-unreadable /etc/shadow and made md5 to significantly increase the work required to generate the crypt. IIRC, the network version works by taking the ruleset to be tested and farming (chunks of) the ruleset out round-robin in a master-slave paradigm, in which the slaves return all the crypt hits they encounter to the master for cumulation. In some ways, it is a lovely model parallel application for relatively trivial parallelism -- a short message plus startup overhead to get a slave going, a short (well, one HOPES short:-) return from the slave, a whole lot of slave computation in between -- quite similar to the way a mandelbrot set generator is parallelized but without the baggage of PVM or MPI, scaling efficiency out to the high 90's for quite a few nodes in spite of the inefficiency of lots of rsh spawns. Parallelizing MD5 or libcrypt itself, well, that's just plain silly... like parallelizing a single threaded random number generator. Quite a lot like it, actually. Why, almost identical:-) Let's insert a really, really high latency network IPC hit in between ordinary arithmetic instructions in a single thread with innate sequential dependencies... hmmm. I don't know the timing because I can't remember when crack was first parallelized, but (to come back to the original thread about enterprise parallel applications) I'd bet that it preceded (and possibly inspired) even the venerable RC5 challenge etc. as one of the very first if not the first widely distributed and commonly used truly network parallel application programs, used in many an enterprise long before the "beowulf" per se was invented. I have no idea if the parallelization was done by black hats or white hats (or even both:-) -- the program was in pretty common use by both, back in the good old days when the ypx program could often as not provide you with a complete copy of any NIS domain's passwd file and crack could often as not give you a half dozen passwd hits from that file in the first four or five minutes of NON parallel operation on a 4 MFLOP Sun workstation. Or crackers would snoop a passwd from a telnet or rsh connection from some grad student logging back in to check mail over the summer and would then proceed to copy the local passwd file (pre-shadow) to work on at their liesure. Either you (as a white hat) ran crack on your passwd files and whomped users who thought their userid twice or a word like "unbreakable" was a "good password" upside the head or the black hats would cheerfully do it for you...:-) Needless to say, been there, coped with that. One of many reasons I'm a shameless advocate of eliminating rsh altogether from the world of programming. telnet can stay as a nice little daemon debugging tool, as long as telnetd is staked to an anthill along with rshd... rgb -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb at phy.duke.edu _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From edwardsa at plk.af.mil Thu Jun 19 12:02:31 2003 From: edwardsa at plk.af.mil (Arthur H. Edwards) Date: Thu, 19 Jun 2003 10:02:31 -0600 Subject: F95/F90 compilers Message-ID: <20030619160231.GA4799@plk.af.mil> I'm about to purchase a F95 compiler for a beowulf running Debian with mpich. I'm obviously looking for performance. I'm also looking for a compiler that doesn't choke or create garbage. Art Edwards -- Art Edwards Senior Research Physicist Air Force Research Laboratory Electronics Foundations Branch KAFB, New Mexico (505) 853-6042 (v) (505) 846-2290 (f) _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From gust_77ar at yahoo.com Thu Jun 19 12:16:58 2003 From: gust_77ar at yahoo.com (Gustavo Velardez) Date: Thu, 19 Jun 2003 09:16:58 -0700 (PDT) Subject: Cluster installation Message-ID: <20030619161658.44319.qmail@web14103.mail.yahoo.com> Hi everybody, We need to know the heat generated (that is, how much cooling is needed) by a beowulf cluster of 11 units. Each unit has 2 Pentium 4 processors. The master node has a power consumption of 338 W and each slave node 295 W. Where can I get this information? Thanx Gustavo __________________________________ Do you Yahoo!? SBC Yahoo! DSL - Now only $29.95 per month! http://sbc.yahoo.com _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From jhearns at freesolutions.net Thu Jun 19 12:36:22 2003 From: jhearns at freesolutions.net (John Hearns) Date: Thu, 19 Jun 2003 17:36:22 +0100 Subject: Network Booting References: <20030619090737.672f725f.torsten@howard.cc> Message-ID: <01a501c33680$ef5d5190$8461cdc2@DREAD> > 7. Does the Linux Terminal Server Project (LTSP) have anything useful > that I could/should be using? Yes indeed - I think there is an article linked from the LTSP page on installing a cluster using LTSP. And if you install LTSP, you will learn about PXE booting and DHCP _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From moor007 at bellsouth.net Thu Jun 19 12:56:02 2003 From: moor007 at bellsouth.net (moor007 at bellsouth.net) Date: Thu, 19 Jun 2003 11:56:02 -0500 Subject: cluster of AOD Opteron (Stefano) In-Reply-To: <200306191635.h5JGZGU30202@NewBlue.Scyld.com> Message-ID: <000401c33683$aaf403c0$0b01a8c0@redstorm> I just received my hardware yesterday for my opteron cluster. My tech will start putting it together today or tomorrow. I am building a 16 CPU cluster w/ the 240 processor onboard the Tyan 2880. I will be using the 2D wulfkit running SuSE enterprise server and Portland Group Server for the Opteron. I am hoping it will be fast. Of course, that is relative. Anyway, I said all that to say that I will begin posting performance benchmarks as they become available. Tim _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From egan at sense.net Thu Jun 19 12:54:49 2003 From: egan at sense.net (Egan Ford) Date: Thu, 19 Jun 2003 10:54:49 -0600 Subject: F95/F90 compilers In-Reply-To: <20030619160231.GA4799@plk.af.mil> Message-ID: <07c301c33683$805a19b0$27b358c7@titan> Both PGI and Intel have eval versions freely available. I recommend you try both and decide for yourself. I am assuming that you are using Intel/AMD-based clusters. My experience has been with both products is that price/performance is greater with the optimized compilers. > -----Original Message----- > From: beowulf-admin at scyld.com > [mailto:beowulf-admin at scyld.com] On Behalf Of Arthur H. Edwards > Sent: Thursday, June 19, 2003 10:03 AM > To: beowulf at beowulf.org > Subject: F95/F90 compilers > > > I'm about to purchase a F95 compiler for a beowulf running Debian with > mpich. I'm obviously looking for performance. I'm also looking for a > compiler that doesn't choke or create garbage. > > Art Edwards > > -- > Art Edwards > Senior Research Physicist > Air Force Research Laboratory > Electronics Foundations Branch > KAFB, New Mexico > > (505) 853-6042 (v) > (505) 846-2290 (f) > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org > To change your subscription (digest mode or unsubscribe) > visit http://www.beowulf.org/mailman/listinfo/beowulf > _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From torsten at howard.cc Thu Jun 19 09:07:37 2003 From: torsten at howard.cc (torsten) Date: Thu, 19 Jun 2003 09:07:37 -0400 Subject: Network Booting Message-ID: <20030619090737.672f725f.torsten@howard.cc> Good Morning: I've been struggling for days to make a single computer be able to network boot. I've read many documents, and I'm a little confused. Here are some questions, and I am very grateful for assistance, in advance. 1. I've seen references to bootp/dhcp, bootp or dhcp, bootp and dhcp, dhcp over bootp - Which one do I need. Do I need both bootp and dhcp? 2. My Lab is on the University subnet. Will my bootp/dhcp server conflict with the university system? Do I need to isolate my lab with IP masq first? 3. On which broadcast/subnet does PXE send bootp requests. Does it make a difference, with PXE, whether I use bootp or DHCP? 4. What are people using to setup/install partitions? I've seen bpbatch and ka-tools. I'm interested in GPL software, primarily. 5. Are there any known problems with Realtek PXE ROMS? My network cards are system integrated Realtek's. 6. Should I just download and setup the MOSIX stuff? 7. Does the Linux Terminal Server Project (LTSP) have anything useful that I could/should be using? Thanks, Torsten _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From lathama at yahoo.com Thu Jun 19 00:46:50 2003 From: lathama at yahoo.com (Andrew Latham) Date: Wed, 18 Jun 2003 21:46:50 -0700 (PDT) Subject: power supplies In-Reply-To: Message-ID: <20030619044650.61470.qmail@web80513.mail.yahoo.com> "... cray-ish couch ..." Now this was what I was thinking about :-) I understand the powerloss. However the heat produced by 9 AT powersupplies by using ~20 watts each is more than they are worth. I think that there is a more efficient method. Considering a cable run of 3 ft. a center 4U chasis with a sizable powersupply could carry nearly a full rack. These are ruff estimites mind you. Does this seem to be pos. or is the 48V method going to win. I would prefer to wire it myself and I found a great place for rackmount goodies (mouser.com) --- Mark Hahn wrote: > > If you carry 3.3volts for any significant distance you'll have a > > substantial voltage drop the ammount of courrent you can will quickly > > result in larger conductors... > > I have this remarkable image in my mind of 20 or so lovely EATX > dual-opteron motherboards arranged radially like fins around > a couple of heavy-gauge copper cables running vertically through > a cylindrical rack, with a monster fan at the bottom. it would > be lovely, and perhaps even worthy of a cray-ish couch around it ;) > > heck, such an arrangement could even cut down on that murderous > time-of-flight component in gigabit ethernet performance :) > > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org > To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf ===== Andrew Latham LathamA.com lathama at lathama.com lathama at yahoo.com LathamA(lay-th-ham-eh) _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From beerli at csit.fsu.edu Thu Jun 19 10:49:24 2003 From: beerli at csit.fsu.edu (Peter Beerli) Date: Thu, 19 Jun 2003 10:49:24 -0400 (EDT) Subject: seeking citation of EP Message-ID: Does someone know who came up with the term "Embarrassingly Parallel" and give the citation? If there are several contexts then the preferred context would be Markov chain Monte Carlo related. thanks, Peter ---- Peter Beerli, School of Computational Science and Information Technology (CSIT) Dirac Science Library, Florida State University Tallahassee, Florida 32306-4120 USA old webpage: http://evolution.genetics.washington.edu/PBhtmls/beerli.html _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From egan at sense.net Thu Jun 19 13:30:23 2003 From: egan at sense.net (Egan Ford) Date: Thu, 19 Jun 2003 11:30:23 -0600 Subject: Network Booting In-Reply-To: <20030619090737.672f725f.torsten@howard.cc> Message-ID: <07c501c33688$76ee0b70$27b358c7@titan> > 1. I've seen references to bootp/dhcp, bootp or dhcp, bootp > and dhcp, > dhcp over bootp - Which one do I need. Do I need both bootp and dhcp? Solutions like pxelinux.0, nbgrub, and etherboot all work with dhcp. I find dhcp easy to work with. > 2. My Lab is on the University subnet. Will my > bootp/dhcp server > conflict with the university system? Do I need to isolate > my lab with > IP masq first? Block dhcp/bootp traffic at the switch port that uplinks to the university system. If you have a multi homed management node running dhcp edit your dhcp startup script or config file to only bind on the eth device that is for your cluster. > > 3. On which broadcast/subnet does PXE send bootp requests. > Does it make > a difference, with PXE, whether I use bootp or DHCP? PXE uses DHCP to get an address. > 4. What are people using to setup/install partitions? I've > seen bpbatch > and ka-tools. I'm interested in GPL software, primarily. Use pxelinux.0 or nbgrub to network boot, then use the native network installer for your OS. e.g. kickstart for RH, autoyast for SuSE. Or use systemimager for any distro. > 6. Should I just download and setup the MOSIX stuff? I would recommend a cluster without MOSIX first, then later determine if MOSIX is the right solution for you. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From glen at cert.ucr.edu Thu Jun 19 12:44:34 2003 From: glen at cert.ucr.edu (Glen Kaukola) Date: Thu, 19 Jun 2003 09:44:34 -0700 Subject: F95/F90 compilers In-Reply-To: <20030619160231.GA4799@plk.af.mil> References: <20030619160231.GA4799@plk.af.mil> Message-ID: <3EF1E872.5070404@cert.ucr.edu> Arthur H. Edwards wrote: >I'm about to purchase a F95 compiler for a beowulf running Debian with >mpich. I'm obviously looking for performance. I'm also looking for a >compiler that doesn't choke or create garbage. > The Fortran compiler from Intel is free, as long as you're not going to be producing any commercial applications with it I think. And our experience with it, is that even on an AMD processor the code that it produces runs faster than the code produced by our portland group compiler. Glen _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From egan at sense.net Thu Jun 19 12:54:49 2003 From: egan at sense.net (Egan Ford) Date: Thu, 19 Jun 2003 10:54:49 -0600 Subject: seeking citation of EP In-Reply-To: Message-ID: <07c401c33683$80912f40$27b358c7@titan> Unrelated, but a HPC shop I visited preferred the term, "naturally parallel". Guess "embarrassingly" was negative. > -----Original Message----- > From: beowulf-admin at scyld.com > [mailto:beowulf-admin at scyld.com] On Behalf Of Peter Beerli > Sent: Thursday, June 19, 2003 8:49 AM > To: beowulf at beowulf.org > Subject: seeking citation of EP > > > Does someone know who came up with > the term > "Embarrassingly Parallel" > and give the citation? If there are several contexts then > the preferred context would be Markov chain Monte Carlo related. > > thanks, > Peter > ---- > Peter Beerli, > School of Computational Science and Information Technology (CSIT) > Dirac Science Library, Florida State University > Tallahassee, Florida 32306-4120 USA > old webpage: > http://evolution.genetics.washington.edu/PBhtm> ls/beerli.html > > > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org > To change your subscription (digest mode or unsubscribe) > visit http://www.beowulf.org/mailman/listinfo/beowulf > _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From hahn at physics.mcmaster.ca Thu Jun 19 14:01:58 2003 From: hahn at physics.mcmaster.ca (Mark Hahn) Date: Thu, 19 Jun 2003 14:01:58 -0400 (EDT) Subject: seeking citation of EP In-Reply-To: <07c401c33683$80912f40$27b358c7@titan> Message-ID: > Unrelated, but a HPC shop I visited preferred the term, "naturally > parallel". Guess "embarrassingly" was negative. euphemisms are usually dumb, but that's exceptionally so. EP is no more natural than tightly-coupled parallelism (ie, the opposite of EP.) if you care about honest, meaningful terms, use tight/loose coupling. personally, I like EP - an allusion to "embarassment of riches", I believe, and probably from Pfister's book. has that ring to it ;) _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rgb at phy.duke.edu Thu Jun 19 13:50:17 2003 From: rgb at phy.duke.edu (Robert G. Brown) Date: Thu, 19 Jun 2003 13:50:17 -0400 (EDT) Subject: Cluster installation In-Reply-To: <20030619161658.44319.qmail@web14103.mail.yahoo.com> Message-ID: On Thu, 19 Jun 2003, Gustavo Velardez wrote: > Hi everybody, > We need to know the heat generated (that is, how much > cooling is needed) by a beowulf cluster of 11 units. > Each unit has 2 Pentium 4 processors. The master node > has a power consumption of 338 W and each slave node > 295 W. > Where can I get this information? > Thanx > > Gustavo You mean, other than by adding 338+2950 = 3288 Watts (Joules per second)? This is a bit less than one ton of air conditioning (a "ton" of AC is the capacity to remove 3504 watts continuously). However, I'd be lavish and install a healthy overcapacity to allow for growth of your cluster and to keep the space >>cold<< and not just not hot. Also, human bodies add 100W or so each, lights ditto, and the walls or exterior ventilation can allow heat to diffuse in or out depending on whether the ambient space outside is warmer or cooler. Two or three tons wouldn't be out of the question, with adequate flow and delivery and warm air return. Note that a "ton" of AC doesn't necessarily mean a particularly large compressor/chiller. We're talking a bit more than a window unit (although there are some pretty big window units) but all the AC you need should sit nicely in less than a cubic meter of outdoor space, plus of course plumbing and the blower to actually deliver "cold" to the space to be cooled (or more properly, to remove the heat therefrom). rgb -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb at phy.duke.edu _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rgb at phy.duke.edu Thu Jun 19 13:40:47 2003 From: rgb at phy.duke.edu (Robert G. Brown) Date: Thu, 19 Jun 2003 13:40:47 -0400 (EDT) Subject: Network Booting In-Reply-To: <20030619090737.672f725f.torsten@howard.cc> Message-ID: On Thu, 19 Jun 2003, torsten wrote: > Good Morning: > > I've been struggling for days to make a single computer be able to > network boot. I've read many documents, and I'm a little confused. > Here are some questions, and I am very grateful for assistance, in > advance. Have you read in particular the HOWTOs on tldp.org? The DHCP and the Diskless Linux howtos: http://www.tldp.org/HOWTO/mini/DHCP/index.html http://www.tldp.org/HOWTO/Diskless-HOWTO.html http://www.tldp.org/HOWTO/Diskless-root-NFS-HOWTO.html http://www.tldp.org/HOWTO/Diskless-root-NFS-other-HOWTO.html are good, and the following aren't quite HOWTOs for PXE but they cross reference a lot of documents that can be used to help you out. http://www.kegel.com/linux/pxe.html http://www.stanford.edu/~alfw/PXE-Kickstart/PXE-Kickstart.html > 1. I've seen references to bootp/dhcp, bootp or dhcp, bootp and dhcp, > dhcp over bootp - Which one do I need. Do I need both bootp and dhcp? bootp is a very old network protocol supporting diskless booting. You send a bootp broadcast/request out on a network, and it returns your network identity (which it works out based on e.g. your MAC address) and a tftp (trivial file transfer protocol) address where you can find what you devoutly hope is the kernel you are supposed to boot. TFTP is a sucking wound from the point of view of security and hence must be used with caution -- in the old days it was pretty easy to set up tftp by accident so that anybody in the netverse could do a tftp connection to the server and ask nicely for /etc/passwd (see earlier comment on parallel crack:-) and it would just say uh, duh, sure... dhcp is a modern improvement (specified by various RFC's as is, for that matter, bootp itself) on bootp. In addition to giving the connecting host an IP number either from a pool or based on its explicit MAC address, it can send all kinds of other information back to the client. For example, mask information, nameserver(s), lease time, NIS information, NTP information, routing information -- enough to totally configure the interface. In addition, it can (like bootp) specify both server addresses and file paths on the servers. A lot of your conflict likely arises because for a while dhcp couldn't directly support pxe booting, so a pxe client was developed that could. We've used it here for a fairly long time and it works fine, and you can even find details of setting up pxelinux including sample configuration files on the brahma resources page (resources for beowulf builders). However, in recent linuces dhpc can directly handle PXE (bootp) requests. So all you SHOULD need is tftp and dhcp. Alas, the TLDP DHPC HOWTO seems not to have caught on to this yet, although the PXE HOWTO has gone away -- "interesting"... a TLDP gap. Still, as Kegel's lovely article makes clear, with just DHPC and TFTP you can give a system all kinds of things to boot: grub, pxelinux, etherboot, a network install image, a diskless boot image, or "whatever evil program you desire". The latter means that you need to look out for people running dhcp servers in your LAN, as in at least the old days the fastest server (when there is more than one) wins and gets to tell the booting host what to boot. [Ah, I remember well the days when SGI's came with bootparamd on by default and were always the fastest thing on any LAN and were never configured to support diskless boots of e.g. Suns and in fact were somewhat broken for that purpose. You could always tell when a new one was installed because all the diskless systems stopped booting until you went over and drilled a stake through the SGI's little daemonic heart.] Nowadays it's merely a security problem. Use whatever means necessary to ensure that no bootp/dhcp servers exist within the broadcast boundary of your LAN (up to the router/switch that filters broadcasts, if you've got one). Try not to kill anyone. Permanently that is. Shooting them in the kneecaps is generally held to be ok. > 2. My Lab is on the University subnet. Will my bootp/dhcp server > conflict with the university system? Do I need to isolate my lab with > IP masq first? See previous comments. I personally am fond of my kneecaps and imagine that you kinda like your own. Therefore, I would be very certain that your LAN is isolated from the University sytem if they indeed run a WAN dhpc server before setting up one of your own. Even if they aren't running a dhcp server on the WAN, you'll still want this isolation. Do YOU want to be giving out IP numbers and so forth to all the booting systems on campus? Of course not. You might do a really good job and find yourself saddled with it permamently. I think I'd rather get shot in the kneecaps than that. > 3. On which broadcast/subnet does PXE send bootp requests. Does it make > a difference, with PXE, whether I use bootp or DHCP? The broadcasts are ETHERNET broadcasts. That is, they aren't IP broadcasts at all as IP hasn't yet been configured on the host containing the device issuing the request (if that makes sense). The NIC shouts at the top of its metaphorical lungs IS THERE ANYBODY OUT THERE THAT HAS THE FAINTEST IDEA WHO I AM and hopefully the One True Server says "why certainly, you'd be 192.168.1.38, your netmask is 255.255.255.0, you can route through 192.168.1.250, your server is 192.168.1.1, and if you ask it nicely it will send you an image at /tftpboot/i386/kernel_RH9 so you can boot..." The security problem is obvious, as if my slightly faster server answers first, I can tell it to boot /tftpboot/i386/kernel_RH9_evil from my very own system, the one that behaves just like RH9's kernel except for the backdoor on port 47319 that I can get into if I enter the right 1337 password. Other reasons for limiting the range of the broadcast are similarly obvious. An ethernet broadcast is moderately expensive as it goes to every port on a switch and requires active processing by every host when it gets there. It actually eats a bit of CPU and net bandwidth from every host and branch of the LAN. This is why network segmentation was invented, in a way -- some old networks were notorious for generating truly incredible amounts of broadcast traffic, kind of like a perpetually running DoS attack that was perfectly "normal". It is pretty easy these days to create a boundary to a broadcast domain. Lots of (managed) switches permit you to turn off broadcast forwarding. Of course, a router or dual headed linux box used as a router accomplish the same thing even better. > 4. What are people using to setup/install partitions? I've seen bpbatch > and ka-tools. I'm interested in GPL software, primarily. I'm not sure what you mean here. Are you talking about e.g. fdisk/mkfs? Higher order front ends of same? Or something for setting up diskless systems specifically? We actually mostly use PXE/DHCP to fuel kickstart, which creates partitions and installs them flawlessly and even moderately intelligently. There are lots of ways to accomplish the same thing, though, including scripts that use fdisk and mkfs. Most of them are probably GPL -- I didn't realize that anybody was selling proprietary linux tools in this space (there's an oxymoron for you -- "proprietary linux tools";-). > 5. Are there any known problems with Realtek PXE ROMS? My network cards > are system integrated Realtek's. I will try to stifle any response, since I honestly don't know how their PXE cards behave. However, overall I loathe RTL NICs. I wouldn't be surprised if they behave badly. Then again, in the fullness of time even RealTek may get the humble NIC right... > 6. Should I just download and setup the MOSIX stuff? Depends on what you want to do. Generally speaking I'd answer "no" with a puzzled expression on my face, although for certain task mixes it isn't horribly unreasonable. One major problem is that it lags the major distributions pretty significantly, which can produce both security and user satisfaction gaps. However, it does work nicely as a user-transparent, moderately "expensive" job migration facility, when it works. > 7. Does the Linux Terminal Server Project (LTSP) have anything useful > that I could/should be using? See answer to 6. Conceivably, but it depends on what you're trying to do. So what are you trying to do? (Hope some of the above helps:-) rgb -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb at phy.duke.edu _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From josip at lanl.gov Thu Jun 19 16:26:04 2003 From: josip at lanl.gov (Josip Loncaric) Date: Thu, 19 Jun 2003 14:26:04 -0600 Subject: F95/F90 compilers In-Reply-To: <3EF1E872.5070404@cert.ucr.edu> References: <20030619160231.GA4799@plk.af.mil> <3EF1E872.5070404@cert.ucr.edu> Message-ID: <3EF21C5C.10207@lanl.gov> Glen Kaukola wrote: > Arthur H. Edwards wrote: > >> I'm about to purchase a F95 compiler for a beowulf running Debian with >> mpich. I'm obviously looking for performance. I'm also looking for a >> compiler that doesn't choke or create garbage. >> > > The Fortran compiler from Intel is free, as long as you're not going to > be producing any commercial applications with it I think. And our > experience with it, is that even on an AMD processor the code that it > produces runs faster than the code produced by our portland group compiler. Agreed -- but at my old institute, we had pretty good luck with the Portland Group Fortran. Intel's Fortran can be faster, but unfortunately, it sometimes generates incorrect code. Some ifc 7.0 versions incorrectly handled equivalence statements -- which are liberally used in many old Fortran packages. I'm not sure if this has been completely fixed even in the latest ifc 7.1 versions. Sincerely, Josip _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rgb at phy.duke.edu Thu Jun 19 15:53:05 2003 From: rgb at phy.duke.edu (Robert G. Brown) Date: Thu, 19 Jun 2003 15:53:05 -0400 (EDT) Subject: Network Booting In-Reply-To: <20030619144306.571533d6.torsten@howard.cc> Message-ID: On Thu, 19 Jun 2003, torsten wrote: > > >> 7. Does the Linux Terminal Server Project (LTSP) have anything useful > >> that I could/should be using? > > > >See answer to 6. Conceivably, but it depends on what you're trying to > >do. So what are you trying to do? > > > Thank you for the very complete answers. This solves a number of concerns > I have run into. > > I am trying setup a cluster to run MPICH. Some of the PC's are > headed workstations, the rest will be headless (for MPICH only). > > While we have manually (via CD-ROM) setup the workstations, I want to > automate the addition and maintenance of headless PC's. An excellent idea. There are a number of resources linked to the brahma page http://www.phy.duke.edu/brahma that should help you out. There are lots of ways to do it. We use DHCP+PXE+KICKSTART+YUM here (and there is moderate documentation of how to go about it on one of the brahma references) but there is also FAL (or is it FAI) for debian, there is Scyld (if you want a commercial solution that makes your cluster pretty much plug-n-play for MPI jobs), there is clustermatic and bproc, and Mandrake and SuSE have solutions of their own I'm less familiar with. > > I am trying to use DHCP with PXE booting to boot the PC's. Any advice > toward this general direction, I am grateful for. Hopefully some of the many links I've thrown at you will be of use. I don't think it is useful for me to write a HOWTO directly to the whole list, though... especially when there is so much excellent documentation now online. rgb P.S. -- I can now say that Mosix is not a good idea. I'd suggest picking a package-based distribution that supports automated installation AND an automated package maintenance tool and go with it OR pick Scyld, dicker out a reasonable license fee for your small cluster, and go that way. Your long term interests are not just automating installation but automating REinstallation, running maintenance package updates and upgrades, and LAN operation (accountspace, shared filespace and so forth). An e.g. RH based cluster is one approach that makes your cluster very much like a LAN of workstations, just some of those workstations have no heads and are used pretty much only for MPI or PVM or EP/script managed computations. The Scyld cluster makes your cluster nodes into "a supercomputer" -- a virtual MP machine -- typically with a single head, a custom task/file loader and clusterwide PID space. If I understand it correctly (I'm sure Don will correct me if I'm misrepresenting it:-). Your current model sounds more like a NOW cluster with multiple servers and points of access and nodes you can login to (you don't log into a "node" in a Scyld cluster any more than you log into a "processor" in an MP machine:-) but I don't know if that is by deliberate design or just what you knew how to do. -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb at phy.duke.edu _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From torsten at howard.cc Thu Jun 19 14:43:06 2003 From: torsten at howard.cc (torsten) Date: Thu, 19 Jun 2003 14:43:06 -0400 Subject: Network Booting In-Reply-To: References: <20030619090737.672f725f.torsten@howard.cc> Message-ID: <20030619144306.571533d6.torsten@howard.cc> >> 7. Does the Linux Terminal Server Project (LTSP) have anything useful >> that I could/should be using? > >See answer to 6. Conceivably, but it depends on what you're trying to >do. So what are you trying to do? Thank you for the very complete answers. This solves a number of concerns I have run into. I am trying setup a cluster to run MPICH. Some of the PC's are headed workstations, the rest will be headless (for MPICH only). While we have manually (via CD-ROM) setup the workstations, I want to automate the addition and maintenance of headless PC's. I am trying to use DHCP with PXE booting to boot the PC's. Any advice toward this general direction, I am grateful for. Torsten _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From James.P.Lux at jpl.nasa.gov Thu Jun 19 16:14:59 2003 From: James.P.Lux at jpl.nasa.gov (Jim Lux) Date: Thu, 19 Jun 2003 13:14:59 -0700 Subject: Cluster installation In-Reply-To: <20030619161658.44319.qmail@web14103.mail.yahoo.com> Message-ID: <5.2.0.9.2.20030619130907.02f5a960@mailhost4.jpl.nasa.gov> Simple.. all the electrical power is converted to heat. 1 unit @338W and 10 units at 295W gives you, to a first order, 2950+338 = 3288 Watts... So, in energy terms, that's 3.3 kJ/second. You can convert kJ to calories, BTU, tons of ice, etc. A comment: 295 and 338 appear to be overly precise numbers. I would venture to guess that the instantaneous power consumption of a PC varies somewhat, depending on instruction stream (fine scale) and environmental effects (line voltage, air temperature, etc.) You might just assume 350W per node and be done with it... Nit picky note: indeed, the information coming out of the cluster is somewhat more organized (i.e. lower entropy) than the information going in, and there is some small, but non-zero, amount of energy required to reduce the information entropy. Then again, one can also use the energy to randomize the data. At 09:16 AM 6/19/2003 -0700, you wrote: >Hi everybody, >We need to know the heat generated (that is, how much >cooling is needed) by a beowulf cluster of 11 units. >Each unit has 2 Pentium 4 processors. The master node >has a power consumption of 338 W and each slave node >295 W. >Where can I get this information? >Thanx > > Gustavo James Lux, P.E. Spacecraft Telecommunications Section Jet Propulsion Laboratory, Mail Stop 161-213 4800 Oak Grove Drive Pasadena CA 91109 tel: (818)354-2075 fax: (818)393-6875 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rbw at ahpcrc.org Thu Jun 19 17:13:09 2003 From: rbw at ahpcrc.org (Richard Walsh) Date: Thu, 19 Jun 2003 16:13:09 -0500 Subject: Cluster installation Message-ID: <200306192113.h5JLD9N24407@mycroft.ahpcrc.org> Jim Lux wrote: >Then again, one can also use the energy to randomize the data. Can one get paid to do this ... :-) ... ? Sounds like a case of turning lead into dross rather than gold. Nature is "nicht boshaft", but programs do this more often than we would like, especially when run as black boxen. May all your programming experiences be ... entropy reducing. Maxwell's Daemon (rbw) #--------------------------------------------------- # Richard Walsh # Project Manager, Cluster Computing, Computational # Chemistry and Finance # netASPx, Inc. # 1200 Washington Ave. So. # Minneapolis, MN 55415 # VOX: 612-337-3467 # FAX: 612-337-3400 # EMAIL: rbw at networkcs.com, richard.walsh at netaspx.com # rbw at ahpcrc.org # #--------------------------------------------------- # "Put a glass of wine in a barrel of sewage and you # have sewage. Put a glass of sewage in a barrel of # wine and you also have sewage. # -Schopenhauer #--------------------------------------------------- _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From mikee at mikee.ath.cx Thu Jun 19 17:25:06 2003 From: mikee at mikee.ath.cx (Mike Eggleston) Date: Thu, 19 Jun 2003 16:25:06 -0500 Subject: Network Booting In-Reply-To: <20030619144306.571533d6.torsten@howard.cc>; from torsten@howard.cc on Thu, Jun 19, 2003 at 02:43:06PM -0400 References: <20030619090737.672f725f.torsten@howard.cc> <20030619144306.571533d6.torsten@howard.cc> Message-ID: <20030619162506.A19914@mikee.ath.cx> On Thu, 19 Jun 2003, torsten wrote: > > >> 7. Does the Linux Terminal Server Project (LTSP) have anything useful > >> that I could/should be using? > > > >See answer to 6. Conceivably, but it depends on what you're trying to > >do. So what are you trying to do? > > > Thank you for the very complete answers. This solves a number of concerns > I have run into. > > I am trying setup a cluster to run MPICH. Some of the PC's are > headed workstations, the rest will be headless (for MPICH only). > > While we have manually (via CD-ROM) setup the workstations, I want to > automate the addition and maintenance of headless PC's. > > I am trying to use DHCP with PXE booting to boot the PC's. Any advice > toward this general direction, I am grateful for. I have used LTSP to setup an ad-hoc cluster for PVM. It worked well. Mike _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From ctierney at hpti.com Thu Jun 19 16:53:22 2003 From: ctierney at hpti.com (Craig Tierney) Date: 19 Jun 2003 14:53:22 -0600 Subject: F95/F90 compilers In-Reply-To: <3EF21C5C.10207@lanl.gov> References: <20030619160231.GA4799@plk.af.mil> <3EF1E872.5070404@cert.ucr.edu> <3EF21C5C.10207@lanl.gov> Message-ID: <1056056002.11801.134.camel@woody> On Thu, 2003-06-19 at 14:26, Josip Loncaric wrote: > Glen Kaukola wrote: > > Arthur H. Edwards wrote: > > > >> I'm about to purchase a F95 compiler for a beowulf running Debian with > >> mpich. I'm obviously looking for performance. I'm also looking for a > >> compiler that doesn't choke or create garbage. > >> > > > > The Fortran compiler from Intel is free, as long as you're not going to > > be producing any commercial applications with it I think. And our > > experience with it, is that even on an AMD processor the code that it > > produces runs faster than the code produced by our portland group compiler. > > Agreed -- but at my old institute, we had pretty good luck with the > Portland Group Fortran. Intel's Fortran can be faster, but > unfortunately, it sometimes generates incorrect code. Some ifc 7.0 > versions incorrectly handled equivalence statements -- which are > liberally used in many old Fortran packages. I'm not sure if this has > been completely fixed even in the latest ifc 7.1 versions. > And at times the Portland Group Fortran generates incorrect code as well. PG has a much longer history in high performance computing so they better support some features that users are used to having. Intel is trying to play catch up. They are doing a good job. I try and keep both compilers around just in case I have a problem with one or the other. Craig > Sincerely, > Josip > > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org > To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- Craig Tierney _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From jducom at nd.edu Thu Jun 19 17:43:12 2003 From: jducom at nd.edu (Jean-Christophe Ducom) Date: Thu, 19 Jun 2003 16:43:12 -0500 Subject: Channel bonding - CPU affinity Message-ID: <3EF22E70.6030800@nd.edu> Just wondering: is it possible to bind an ethernet interface to a specific cpu via cpu affinity mask? Or is it just simpler (and much better in terms of performance) to use channel bonding? Thanks for any comments. JC _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From yudong at lshp.gsfc.nasa.gov Thu Jun 19 16:44:00 2003 From: yudong at lshp.gsfc.nasa.gov (Yudong Tian) Date: Thu, 19 Jun 2003 16:44:00 -0400 Subject: Network Booting In-Reply-To: <20030619090737.672f725f.torsten@howard.cc> Message-ID: My linux nodes use RealTek chips and I have been doing network installation and booting fine. The procedure is a little involved, and I do have some step-by-step notes here: http://lis.gsfc.nasa.gov/yudong/notes/net-install.txt I hope this can help with your Q1,2,3. ------------------------------------------------------------ Falun Dafa: The Tao of Meditation (http://www.falundafa.org) ------------------------------------------------------------ Yudong Tian, Ph.D. NASA/GSFC (301) 286-2275 > -----Original Message----- > From: beowulf-admin at scyld.com [mailto:beowulf-admin at scyld.com]On Behalf > Of torsten > Sent: Thursday, June 19, 2003 9:08 AM > To: beowulf at beowulf.org > Subject: Network Booting > > > Good Morning: > > I've been struggling for days to make a single computer be able to > network boot. I've read many documents, and I'm a little confused. > Here are some questions, and I am very grateful for assistance, in > advance. > > 1. I've seen references to bootp/dhcp, bootp or dhcp, bootp and dhcp, > dhcp over bootp - Which one do I need. Do I need both bootp and dhcp? > > 2. My Lab is on the University subnet. Will my bootp/dhcp server > conflict with the university system? Do I need to isolate my lab with > IP masq first? > > 3. On which broadcast/subnet does PXE send bootp requests. Does it make > a difference, with PXE, whether I use bootp or DHCP? > > 4. What are people using to setup/install partitions? I've seen bpbatch > and ka-tools. I'm interested in GPL software, primarily. > > 5. Are there any known problems with Realtek PXE ROMS? My network cards > are system integrated Realtek's. > > 6. Should I just download and setup the MOSIX stuff? > > 7. Does the Linux Terminal Server Project (LTSP) have anything useful > that I could/should be using? > > Thanks, Torsten > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org > To change your subscription (digest mode or unsubscribe) visit > http://www.beowulf.org/mailman/listinfo/beowulf _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From nordquist at geosci.uchicago.edu Thu Jun 19 16:35:49 2003 From: nordquist at geosci.uchicago.edu (Russell Nordquist) Date: Thu, 19 Jun 2003 15:35:49 -0500 (CDT) Subject: Network Booting In-Reply-To: <20030619144306.571533d6.torsten@howard.cc> Message-ID: rgb mentioned kickstart and FAI. Also worth looking into is systemimager. it works quite well for relatively homogeneous machines and is distro agnostic. It is quite easy to use this with dhcp/pxe to clone/update machines. http://www.systemimager.org russell On Thu, 19 Jun 2003 at 14:43, torsten wrote: > > >> 7. Does the Linux Terminal Server Project (LTSP) have anything useful > >> that I could/should be using? > > > >See answer to 6. Conceivably, but it depends on what you're trying to > >do. So what are you trying to do? > > > Thank you for the very complete answers. This solves a number of concerns > I have run into. > > I am trying setup a cluster to run MPICH. Some of the PC's are > headed workstations, the rest will be headless (for MPICH only). > > While we have manually (via CD-ROM) setup the workstations, I want to > automate the addition and maintenance of headless PC's. > > I am trying to use DHCP with PXE booting to boot the PC's. Any advice > toward this general direction, I am grateful for. > > Torsten > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org > To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf > - - - - - - - - - - - - Russell Nordquist UNIX Systems Administrator Geophysical Sciences Computing http://geosci.uchicago.edu/computing NSIT, University of Chicago - - - - - - - - - - - _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From hahn at physics.mcmaster.ca Thu Jun 19 18:40:46 2003 From: hahn at physics.mcmaster.ca (Mark Hahn) Date: Thu, 19 Jun 2003 18:40:46 -0400 (EDT) Subject: Network Booting In-Reply-To: <20030619162506.A19914@mikee.ath.cx> Message-ID: > > I am trying to use DHCP with PXE booting to boot the PC's. Any advice > > toward this general direction, I am grateful for. > > I have used LTSP to setup an ad-hoc cluster for PVM. It worked well. I looked at LTSP before building my current cluster, since I wanted nodes to be completely ephemeral (nothing installed on local disk.) but there's not really anything unique in LTSP that you need - in fact, my system is basically pure RH8, with minor tweaks to mount /dev/shm in various places, populate a /var in one, start sshd, etc. I'm about to add 10-12 dual-opterons to the existing 48 dual-xeons, and it looks like I could actually get away with changing only the kernel. (one line in tftpd or pxelinux config files.) I don't know yet how much trouble it is to run opterons in extended mode (64b, extra regs). seems like the main benefit is fast dram, which you get either way... _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From douglas at shore.net Thu Jun 19 23:52:23 2003 From: douglas at shore.net (Douglas O'Flaherty) Date: Thu, 19 Jun 2003 23:52:23 -0400 Subject: 95/F90 compilers Message-ID: <3EF284F7.9090304@shore.net> Art: I cite Polyhedron as a good general source for comparative data and articles. The web site is http://www.polyhedron.co.uk/ Pay attention to the version numbers when looking at comparisons. The PGI compiler in the test matrix is 3.2-4 and PGI is releasing version 5.0 at the end of this month. I wish I knew of other broad sources. You can also search for others who run similar applications as you do and see if they have comparative compiler data. Don't assume that a compiler that does one application well will also optimize a different application. Also, don't forget to look at Math Library performance. They make a difference. doug _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From baaden at smplinux.de Fri Jun 20 05:35:08 2003 From: baaden at smplinux.de (Marc Baaden) Date: Fri, 20 Jun 2003 11:35:08 +0200 Subject: [OT] Maximum performance on single processor ? Message-ID: <200306200935.LAA17759@apex.ibpc.fr> Hi, I am sorry if this query is slightly off-topic for the list. But I guess that many people here have also experiences with respect to my enquiry. We are looking for the highest performance machine on single (or very few, let's say 4) processors, because we need to run unparallelized and/or poorly parallelized code. Our budget is of roughly 20000 $, and a linux/beowulf platform would be ideal. The current top Intel x86 or AMD processors are too slow, and we do not have access to other architectures. Thank you very much in advance, Marc Baaden -- Dr. Marc Baaden - Institut de Biologie Physico-Chimique, Paris mailto:baaden at smplinux.de - http://www.marc-baaden.de FAX: +49 697912 39550 - Tel: +33 15841 5176 ou +33 609 843217 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From mikee at mikee.ath.cx Fri Jun 20 08:07:36 2003 From: mikee at mikee.ath.cx (Mike Eggleston) Date: Fri, 20 Jun 2003 07:07:36 -0500 Subject: [OT] Maximum performance on single processor ? In-Reply-To: <200306200935.LAA17759@apex.ibpc.fr>; from baaden@smplinux.de on Fri, Jun 20, 2003 at 11:35:08AM +0200 References: <200306200935.LAA17759@apex.ibpc.fr> Message-ID: <20030620070736.A23866@mikee.ath.cx> On Fri, 20 Jun 2003, Marc Baaden wrote: > > Hi, > > I am sorry if this query is slightly off-topic for the list. > But I guess that many people here have also experiences with > respect to my enquiry. > > We are looking for the highest performance machine on single > (or very few, let's say 4) processors, because we need to run > unparallelized and/or poorly parallelized code. > > Our budget is of roughly 20000 $, and a linux/beowulf platform > would be ideal. > > The current top Intel x86 or AMD processors are too slow, and we > do not have access to other architectures. Do you have the source to what you're running? I once read of a study that showed how an Apple II could out-perform a Cray... the Apple II ran something like a quicksort where the Cray ran a bubblesort. Mike _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From joelja at darkwing.uoregon.edu Fri Jun 20 06:15:35 2003 From: joelja at darkwing.uoregon.edu (Joel Jaeggli) Date: Fri, 20 Jun 2003 03:15:35 -0700 (PDT) Subject: [OT] Maximum performance on single processor ? In-Reply-To: <200306200935.LAA17759@apex.ibpc.fr> Message-ID: On Fri, 20 Jun 2003, Marc Baaden wrote: > > Hi, > > I am sorry if this query is slightly off-topic for the list. > But I guess that many people here have also experiences with > respect to my enquiry. > > We are looking for the highest performance machine on single > (or very few, let's say 4) processors, because we need to run > unparallelized and/or poorly parallelized code. > > Our budget is of roughly 20000 $, and a linux/beowulf platform > would be ideal. actually by defintion if your code is can't be parallelized then a cluster of off-the-shelf hardware probably isn't appropriate. > The current top Intel x86 or AMD processors are too slow, and we > do not have access to other architectures. Then you're screwed. > Thank you very much in advance, > Marc Baaden > > -- -------------------------------------------------------------------------- Joel Jaeggli Academic User Services joelja at darkwing.uoregon.edu -- PGP Key Fingerprint: 1DE9 8FCA 51FB 4195 B42A 9C32 A30D 121E -- In Dr. Johnson's famous dictionary patriotism is defined as the last resort of the scoundrel. With all due respect to an enlightened but inferior lexicographer I beg to submit that it is the first. -- Ambrose Bierce, "The Devil's Dictionary" _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From baaden at smplinux.de Fri Jun 20 08:44:05 2003 From: baaden at smplinux.de (Marc Baaden) Date: Fri, 20 Jun 2003 14:44:05 +0200 Subject: [OT] Maximum performance on single processor ? In-Reply-To: Your message of "Fri, 20 Jun 2003 07:06:25 EDT." Message-ID: <200306201244.OAA22202@apex.ibpc.fr> Thanks for all your comments and suggestions. I realize (and some of you mentioned it) that you need more information to understand better what I am looking for. I have an existing application which is part of a project. I have the source code. It is Fortran. It *can* be parallelized, but we would rather spend our time on the other parts of the project which need to be written from scratch *first*. The application is to run in real time, that is the user does something and as a function of user input and the calculation with the fortran program that I described, there is a correponding feedback to the user on the screen (and in some Virtual Reality equipment). Right now, even on simple test cases, the "response time" (eg calculation time for a single step) of our program is on the order of the second. (this is for an athlon MP 2600+) We need to get that down to a fraction of seconds, best milli-seconds, in order to be usable in real time. (makes it a factor of roughly 1000) As I said the code can indeed be parallelized - maybe even simply cleaned up in some parts - but unfortunately there remains very much other important stuff to do. So we'd rather spend some money on a really fast CPU and not touch the code at the moment. So my question was more, what is the fastest CPU I can get for $20000 at the moment (without explicitly parallelizing, hyperthreading or vectorizing my code). Cheers, Marc Baaden -- Dr. Marc Baaden - Institut de Biologie Physico-Chimique, Paris mailto:baaden at smplinux.de - http://www.marc-baaden.de FAX: +49 697912 39550 - Tel: +33 15841 5176 ou +33 609 843217 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From jhearns at freesolutions.net Fri Jun 20 07:03:16 2003 From: jhearns at freesolutions.net (John Hearns) Date: Fri, 20 Jun 2003 12:03:16 +0100 Subject: [OT] Maximum performance on single processor ? References: <200306200935.LAA17759@apex.ibpc.fr> Message-ID: <019601c3371b$901cd840$8461cdc2@DREAD> > > I am sorry if this query is slightly off-topic for the list. > But I guess that many people here have also experiences with > respect to my enquiry. > > We are looking for the highest performance machine on single > (or very few, let's say 4) processors, because we need to run > unparallelized and/or poorly parallelized code. > > Our budget is of roughly 20000 $, and a linux/beowulf platform > would be ideal. > > The current top Intel x86 or AMD processors are too slow, and we > do not have access to other architectures. > Being mischevous, $20000 would probably buy you the cabinet for a http://www.sgi.com/servers/altix/ Seriously though, people on the list will ask for more information on your application. You say 'too slow' - exaclty what do you mean by this? If it is (say) an application like gene sequence matching, which takes a long time to match against databases, then you may be unable to speed up individual runs. But what you CAN do, and what will probably make your researchers happy, is to have a high throughput - so you get good utilisation of your machines, and results coming through the pipeline. The list really needs some specifics though - and some timings on the processors you have tried. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rgb at phy.duke.edu Fri Jun 20 07:06:25 2003 From: rgb at phy.duke.edu (Robert G. Brown) Date: Fri, 20 Jun 2003 07:06:25 -0400 (EDT) Subject: [OT] Maximum performance on single processor ? In-Reply-To: Message-ID: On Fri, 20 Jun 2003, Joel Jaeggli wrote: > > Our budget is of roughly 20000 $, and a linux/beowulf platform > > would be ideal. > > actually by defintion if your code is can't be parallelized then a cluster > of off-the-shelf hardware probably isn't appropriate. > > > The current top Intel x86 or AMD processors are too slow, and we > > do not have access to other architectures. > > Then you're screwed. Let me be a TEENY bit more optimistic than Joel. The one glimmer of hope I can hold out is that if your code is single threaded by highly vectorized, you might look at some exotic solutions (and some less exotic solutions) involving a vector co-processor, either integrated with the main CPU or as a programmable add-on. Some of the vector processors have extremely high floating point rates for the limited instruction set and data manipulations they are capable of. Three slightly off-the-beaten path solutions you might look into are: a) Power PC. Apple is actually pushing them for clusters these days (I just attended a talk where they were touted), and the upper end ones have both an L3 cache layer and an embedded Altivec vector unit. Although the CPU itself is around 1 GHz and hence "slow" in terms of cycles, the vector unit has some extremely high overall FLOPS rate for the right kind of code. I know nothing of the level of difficulty of programming for it, but Apple supports it with compilers and so forth. b) http://arrakis.ncsa.uiuc.edu/ps2/cluster.php. OK, yes, it's sort of a toy, sort of a joke, but there is at least one Sony Playstation 2 cluster out there. Why, you ask? It runs linux (the "emotion engine" core is a MIPS CPU) and it has two attached vector DSP's. Programming it so that it uses the VU's is PITA, and it may not be what you need, but you want exotica, and this is it. c) NEC has a strange one it does: http://www.silicon.com/news/500009/1/3016.html I know next to nothing about it, but maybe you can cut a deal with them to try it out. May or may not have linux support yet (but they might give you a couple if you were to offer to do the port:-). Again, 8 GFLOP vector processors. d) Vector units are available in a lot of contexts for Linux systems, few of them (alas) straightforward. DSP's, high end video cards. In most cases these units have staggering peak FLOPS (more then 10 GFLOPS in some cases IIRC) but in all cases programming them and optimizing to achieve a reasonable fraction of peak in real world throughput will be, um, a "challenge" unless your problem is shaped just right. But it might be, so give these a look. Otherwise, as Joel says, you're pretty much screwed, especially with a budget of only $20K. This assumes that you've already looked at Intel Itanium and AMD Opteron solutions, and already know about the SSE2 instructions available to those CPUs. In many cases, throughput is related to memory bandwidth in the limit, and these units are designed with fast wide memory. Make sure that you understand your application very well indeed and that it really isn't parallelizable. If it is vectorizable you may have hope. If it isn't vectorizable OR parallelizable then the best you will do anywhere is the fastest CPU du jour... however inadequate it may be. rgb -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb at phy.duke.edu _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From jhearns at freesolutions.net Fri Jun 20 09:01:45 2003 From: jhearns at freesolutions.net (John Hearns) Date: Fri, 20 Jun 2003 14:01:45 +0100 Subject: [OT] Maximum performance on single processor ? References: <200306201244.OAA22202@apex.ibpc.fr> Message-ID: <026701c3372c$1eb39480$8461cdc2@DREAD> > > Right now, even on simple test cases, the "response time" (eg calculation > time for a single step) of our program is on the order of the second. > (this is for an athlon MP 2600+) > We need to get that down to a fraction of seconds, best milli-seconds, > in order to be usable in real time. (makes it a factor of roughly 1000) > > As I said the code can indeed be parallelized - maybe even simply cleaned > up in some parts - but unfortunately there remains very much other important > stuff to do. So we'd rather spend some money on a really fast CPU and not > touch the code at the moment. I disagree. You really must do some code profiling. You may find some real big bottleneck, which is either unintentional or not necessary. Also, you should consider how the code is written - if you are manipulating large arrays then the order of access, and the stride length can make a big difference. You may be ranging all over a huge array, pulling things in from memory(*) when a different algorithm/different loop arrangement could keep things in cache. And Fortran, being 'closer to the machine' can expose these things (I may be wrong). Have you looked at books like: "High Performance Computing" http://www.oreilly.com/catalog/hpc2/ (*) Sorry to teach my granny to suck eggs, but your machine isn't going into swap I trust. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From landman at scalableinformatics.com Fri Jun 20 09:11:27 2003 From: landman at scalableinformatics.com (Joe Landman) Date: Fri, 20 Jun 2003 09:11:27 -0400 Subject: [OT] Maximum performance on single processor ? In-Reply-To: <200306201244.OAA22202@apex.ibpc.fr> References: <200306201244.OAA22202@apex.ibpc.fr> Message-ID: <3EF307FF.6010409@scalableinformatics.com> Marc Baaden wrote: > Right now, even on simple test cases, the "response time" (eg calculation > time for a single step) of our program is on the order of the second. > (this is for an athlon MP 2600+) > We need to get that down to a fraction of seconds, best milli-seconds, > in order to be usable in real time. (makes it a factor of roughly 1000) You are not going to get a 3 order of magnitude increase in performance by switching CPUs. You won't get an order of magnitude over the Athlon. > So my question was more, what is the fastest CPU I can get for $20000 > at the moment (without explicitly parallelizing, hyperthreading or > vectorizing my code). Before you pursue this any more, lets try to reset expectations to be more in line with what would need to occur. 1) the CPUs are generally within single digit factors of performance on a given code, and this is as much dominated by memory speed and/or latency for various codes, as it could be by CPU "speed" 2) As it does not appear that you have profiled the code (just a guess), I would strongly urge you do that as your next step. This will tell you (if you chose not to optimize your code) where the code is spending most of its time. If this is in fetching data from memory then a "faster" CPU will not help, as the CPU is not the bottleneck. 3) to get multiple orders of magnitude change in performance, you will need at minimum an algorithm shift. The algorithms used in your code now are taking time, and you need to know how much time. It would be terrible if your code takes 1/2 second to set up the calculation in the first place, and another 1/2 second to perform it. In this case, the first half second would impede any faster algorithm. If you comment out the computation, how long does the setup/teardown take for the code? Though you don't want to work on this code, I am not sure you will be able to get more than some percentage points with a new CPU without at minimum a profile of the code, and a hard look at what it is doing, and how it is doing it. > > > Cheers, > Marc Baaden > -- Joseph Landman, Ph.D Scalable Informatics LLC, email: landman at scalableinformatics.com web : http://scalableinformatics.com phone: +1 734 612 4615 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From seth at hogg.org Fri Jun 20 09:15:47 2003 From: seth at hogg.org (Simon Hogg) Date: Fri, 20 Jun 2003 14:15:47 +0100 Subject: [OT] Maximum performance on single processor ? In-Reply-To: <200306201244.OAA22202@apex.ibpc.fr> References: Message-ID: <4.3.2.7.2.20030620140207.00ae23a0@pop.freeuk.net> At 14:44 20/06/03 +0200, Marc Baaden wrote: >I have an existing application which is part of a project. I have >the source code. It is Fortran. It *can* be parallelized, but we >would rather spend our time on the other parts of the project >which need to be written from scratch *first*. > >The application is to run in real time, that is the user does something >and as a function of user input and the calculation with the fortran >program that I described, there is a correponding feedback to the >user on the screen (and in some Virtual Reality equipment). > >Right now, even on simple test cases, the "response time" (eg calculation >time for a single step) of our program is on the order of the second. >(this is for an athlon MP 2600+) >We need to get that down to a fraction of seconds, best milli-seconds, >in order to be usable in real time. (makes it a factor of roughly 1000) > >As I said the code can indeed be parallelized - maybe even simply cleaned >up in some parts - but unfortunately there remains very much other important >stuff to do. So we'd rather spend some money on a really fast CPU and not >touch the code at the moment. > >So my question was more, what is the fastest CPU I can get for $20000 >at the moment (without explicitly parallelizing, hyperthreading or >vectorizing my code). I'm sure some other people will give 'better' answers, but from having a look at your web pages, I would be tempted to go down the route of second-hand SGI equipment. For example (and no, I don't know how the performance stacks up, I'm looking partly at a general bio-informatics / SGI link if that makes sense) I can see for sale an Origin 2000 Quad 500MHz / 4GB RAM for UKP 15,725. Just another suggestion to add to the pile. -- Simon _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From hahn at physics.mcmaster.ca Fri Jun 20 10:29:57 2003 From: hahn at physics.mcmaster.ca (Mark Hahn) Date: Fri, 20 Jun 2003 10:29:57 -0400 (EDT) Subject: [OT] Maximum performance on single processor ? In-Reply-To: Message-ID: > Although the CPU itself is around 1 GHz and hence "slow" in terms of > cycles, the vector unit has some extremely high overall FLOPS rate for > the right kind of code. where "right kind of code" means "exquisitely cache-friendly". I suspect that some speed-hacked fortran is going to be very much not like that. as previous posts pointed out, unless the original code is remarkably stupid, hardware is simply not going to give you 1000x. finding a smart kid and setting him up with a $5k P4/i875 seems a lot more plausible to me. the real lateral-thinking approach is something like fpga's, but if you can't be bothered to sanity-check the original code, fpga's are out of the question... _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From jhearns at freesolutions.net Fri Jun 20 11:04:22 2003 From: jhearns at freesolutions.net (John Hearns) Date: Fri, 20 Jun 2003 16:04:22 +0100 Subject: [OT] Maximum performance on single processor ? References: Message-ID: <031901c3373d$3ea46600$8461cdc2@DREAD> My advice: Go find your local systems guru. This person is usually to be found in a dimly lit basement office. You can recognise it by the 'Nicht Fur Lookenpeepers' notices and the 1972 lineprinter ASCII art Snoopy calendar on the wall. The person will have disguised a spiffy top of the range monitor with carefully chosen junk. On that monitor, he/she will have many windows open. One will be tailing your mail file (joke). Bring gifts of beer. As him/her for advice: "Please help me to get my program running faster". _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From mikee at mikee.ath.cx Fri Jun 20 10:57:32 2003 From: mikee at mikee.ath.cx (Mike Eggleston) Date: Fri, 20 Jun 2003 09:57:32 -0500 Subject: [OT] Maximum performance on single processor ? In-Reply-To: <200306201244.OAA22202@apex.ibpc.fr>; from baaden@smplinux.de on Fri, Jun 20, 2003 at 02:44:05PM +0200 References: <200306201244.OAA22202@apex.ibpc.fr> Message-ID: <20030620095732.D23866@mikee.ath.cx> On Fri, 20 Jun 2003, Marc Baaden wrote: > > Thanks for all your comments and suggestions. I realize (and some > of you mentioned it) that you need more information to understand > better what I am looking for. > > I have an existing application which is part of a project. I have > the source code. It is Fortran. It *can* be parallelized, but we > would rather spend our time on the other parts of the project > which need to be written from scratch *first*. > > The application is to run in real time, that is the user does something > and as a function of user input and the calculation with the fortran > program that I described, there is a correponding feedback to the > user on the screen (and in some Virtual Reality equipment). > > Right now, even on simple test cases, the "response time" (eg calculation > time for a single step) of our program is on the order of the second. > (this is for an athlon MP 2600+) > We need to get that down to a fraction of seconds, best milli-seconds, > in order to be usable in real time. (makes it a factor of roughly 1000) > > As I said the code can indeed be parallelized - maybe even simply cleaned > up in some parts - but unfortunately there remains very much other important > stuff to do. So we'd rather spend some money on a really fast CPU and not > touch the code at the moment. > > So my question was more, what is the fastest CPU I can get for $20000 > at the moment (without explicitly parallelizing, hyperthreading or > vectorizing my code). Can the lisp technique of memoization help with your processing? Mike _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rmyers1400 at attbi.com Fri Jun 20 11:16:37 2003 From: rmyers1400 at attbi.com (Robert Myers) Date: Fri, 20 Jun 2003 11:16:37 -0400 Subject: [OT] Maximum performance on single processor ? In-Reply-To: <200306201244.OAA22202@apex.ibpc.fr> References: <200306201244.OAA22202@apex.ibpc.fr> Message-ID: <3EF32555.7040006@attbi.com> Marc Baaden wrote: >As I said the code can indeed be parallelized - maybe even simply cleaned >up in some parts - but unfortunately there remains very much other important >stuff to do. So we'd rather spend some money on a really fast CPU and not >touch the code at the moment. > >So my question was more, what is the fastest CPU I can get for $20000 >at the moment (without explicitly parallelizing, hyperthreading or >vectorizing my code). > > You may have done better service to the readers of this list than you have to yourself, since the readers of this list are naturally inclined to think about the sort of question you have asked, and the answers have been interesting to read. The fact that you are asking the question, though, begs an answer that others have partially given: no plausible hardware purchase you can make will be a substitute for you or someone you are working with to get smart about being clever with hardware and software. You don't want to invest in people, but people is what you must invest in. There is probably someone out there really hungry for work or just plain interested who could get you an awful lot of mileage for the money you want to spend. Without that sort of expertise, the chances of your getting much further than you already are through a hardware purchase are, to a very good approximation, zero. The application you describe sounds like something that might wind up in an ASIC or FPGA, but that involves serious people time. The only no-brains speedup strategy that could work is the most powerful speed-up strategy of all: wait a couple of years for off-the-shelf hardware to get faster. RM _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From Daniel.Kidger at quadrics.com Fri Jun 20 10:02:44 2003 From: Daniel.Kidger at quadrics.com (Daniel Kidger) Date: Fri, 20 Jun 2003 15:02:44 +0100 Subject: cluster of AMD Opteron Message-ID: <010C86D15E4D1247B9A5DD312B7F5AA78DDF91@stegosaurus.bristol.quadrics.com> From: Mikhail Kuzminsky [mailto:kus at free.net] >> Is somebody who is using AMD opterons yet ? > We tested 2-way SMP server based on RioWorks mobo. But I should >not recommend this motherboard for using: by default it has no >monitoring (temperature etc) chips on the board, it's necessary >to buy special additional card ! Unfortunately as a result I don't >have data about lm_sensors work. Moreover, the choice of SMP >boards is very restricted now: Tyan S2880 and MSI K8D. We have a small cluster of Newisys 2100 dual-Opteron 1U-nodes. our NDA prevents me from going into much praise, but I can say that they *do* have execellent environemental monitoring of just about everythinbg on the Mobo. You can even get a live output of the current power consumption :-) (which of course relates to another long running thread on the beowulf list). Yours, Daniel. -------------------------------------------------------------- Dr. Dan Kidger, Quadrics Ltd. daniel.kidger at quadrics.com One Bridewell St., Bristol, BS1 2AA, UK 0117 915 5505 ----------------------- www.quadrics.com -------------------- _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From maurice at harddata.com Fri Jun 20 12:09:54 2003 From: maurice at harddata.com (Maurice Hilarius) Date: Fri, 20 Jun 2003 10:09:54 -0600 Subject: [OT] Maximum performance on single processor ? In-Reply-To: <200306201502.h5KF2EU21424@NewBlue.Scyld.com> Message-ID: <5.1.1.6.2.20030620100637.03ca3450@mail.harddata.com> With regards to your message: >From: Marc Baaden <<>> >So my question was more, what is the fastest CPU I can get for $20000 >at the moment (without explicitly parallelizing, hyperthreading or >vectorizing my code). I believe the answer is an Opteron. Of course that much money will buy quite a lot of CPUs! With the memory bandwidth, floating point engine, and Hypertransport links on these, they are VERY fast for this type of thing. If you want access to a machine to run some code samples on, please contact me off-list and we can arrange some time for you on a machine. We have test/demo dual Opteron servers in place, available over SSH connections, for exactly this purpose. They are set up with compilers, libraries and tools suitable for this use. Thanks! With our best regards, Maurice W. Hilarius Telephone: 01-780-456-9771 Hard Data Ltd. FAX: 01-780-456-9772 11060 - 166 Avenue mailto:maurice at harddata.com Edmonton, AB, Canada http://www.harddata.com/ T5X 1Y3 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From jcmoore at atipa.com Fri Jun 20 12:24:20 2003 From: jcmoore at atipa.com (Curt Moore) Date: 20 Jun 2003 11:24:20 -0500 Subject: cluster of AOD Opteron In-Reply-To: <200306201342.RAA28782@nocserv.free.net> References: <200306201342.RAA28782@nocserv.free.net> Message-ID: <1056126260.9688.15.camel@picard.lab.atipa.com> The RioWorks HDAMA (Arima) motherboard does have on-board sensors, adm1026 based. We've recently deployed a 100 node, 200 CPU, cluster of these systems and all works very well. Arima does have planned both a mini BMC which does just management type functions and also a full BMC with will do other neat things, I believe, such as KVM over LAN. Below is a lm_sensors dump from an Arima HDAMA. adm1026-i2c-0-2c Adapter: SMBus AMD8111 adapter at 80e0 Algorithm: Non-I2C SMBus adapter in0: +1.15 V (min = +0.00 V, max = +2.99 V) in1: +1.59 V (min = +0.00 V, max = +2.99 V) in2: +1.57 V (min = +0.00 V, max = +2.99 V) in3: +1.19 V (min = +0.00 V, max = +2.99 V) in4: +1.18 V (min = +0.00 V, max = +2.99 V) in5: +1.14 V (min = +0.00 V, max = +2.99 V) in6: +1.24 V (min = +0.00 V, max = +2.49 V) in7: +1.59 V (min = +0.00 V, max = +2.49 V) in8: +0.00 V (min = +0.00 V, max = +2.49 V) in9: +0.45 V (min = +1.25 V, max = +0.98 V) in10: +2.70 V (min = +0.00 V, max = +3.98 V) in11: +3.33 V (min = +0.00 V, max = +4.42 V) in12: +3.38 V (min = +0.00 V, max = +4.42 V) in13: +5.12 V (min = +0.00 V, max = +6.63 V) in14: +1.57 V (min = +0.00 V, max = +2.99 V) in15: +11.88 V (min = +0.00 V, max = +15.94 V) in16: -12.03 V (min = +2.43 V, max = -16.00 V) fan0: 0 RPM (min = 0 RPM, div = 2) fan1: 0 RPM (min = 0 RPM, div = 2) fan2: 0 RPM (min = 0 RPM, div = 2) fan3: 0 RPM (min = 0 RPM, div = 2) fan4: 0 RPM (min = 0 RPM, div = 1) fan5: 0 RPM (min = 0 RPM, div = 1) fan6: -1 RPM (min = 0 RPM, div = 1) fan7: -1 RPM (min = 0 RPM, div = 1) temp1: +37?C (min = -128?C, max = +80?C) temp2: +46?C (min = -128?C, max = +100?C) temp3: +46?C (min = -128?C, max = +100?C) vid: +1.850 V (VRM Version 9.1) -Curt ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Curt Moore Systems Integration Engineer At?pa Technologies jcmoore at atipa.com (O) 785-813-0312 (Fax) 785-841-1809 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From alangrimes at starpower.net Fri Jun 20 15:26:01 2003 From: alangrimes at starpower.net (Alan Grimes) Date: Fri, 20 Jun 2003 12:26:01 -0700 Subject: [OT] Maximum performance on single processor ? References: <031901c3373d$3ea46600$8461cdc2@DREAD> Message-ID: <3EF35FC9.3084A9FF@starpower.net> om >From what little I've heard my advice for this rig is: 1. The athlon MP is limited by its 266mhz FSB... For truly single-threaded code, get an Athlon 3200 and a dual channel DDR board... This should earn you roughly a 1.6x speedup over your existing rig... I, myself, am using only 1.2ghz chips in my dual athlon because of the limited system bandwidth.. 2. The current benchmark king is the P4 3.0 ghz/ 4x200mhz FSB. You can profit from this even more if you have two threads going... (this would be roughly 2x over the MP you mentioned..)... 3. It still isn't clear what your simulation is (I might have missed it) but it _MIGHT_ be profitable to seperate out your front end.. If your front end requires quite a bit more than what can be farmed out to an OpenGL 1.3+ gfx accelerator, an the bandwidth to your engine is reasonable, consider putting your front end on the athlon mentioned above and your back-end on the P4 and link them over GBE... Then make sure there is as little overhead as possible on the P4... Take off all unnecessary software... Depending on how much CPU your front end takes and how well you can make your network work for you, this could earn you as much as a 2x speedup over the 1.6x above... -- Testimonial: I wasted 2,000 hours using Linux. http://users.rcn.com/alangrimes/ _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rgb at phy.duke.edu Fri Jun 20 12:32:10 2003 From: rgb at phy.duke.edu (Robert G. Brown) Date: Fri, 20 Jun 2003 12:32:10 -0400 (EDT) Subject: [OT] Maximum performance on single processor ? In-Reply-To: <200306201244.OAA22202@apex.ibpc.fr> Message-ID: On Fri, 20 Jun 2003, Marc Baaden wrote: > > Thanks for all your comments and suggestions. I realize (and some > of you mentioned it) that you need more information to understand > better what I am looking for. > > I have an existing application which is part of a project. I have > the source code. It is Fortran. It *can* be parallelized, but we > would rather spend our time on the other parts of the project > which need to be written from scratch *first*. > > The application is to run in real time, that is the user does something > and as a function of user input and the calculation with the fortran > program that I described, there is a correponding feedback to the > user on the screen (and in some Virtual Reality equipment). > > Right now, even on simple test cases, the "response time" (eg calculation > time for a single step) of our program is on the order of the second. > (this is for an athlon MP 2600+) > We need to get that down to a fraction of seconds, best milli-seconds, > in order to be usable in real time. (makes it a factor of roughly 1000) > > As I said the code can indeed be parallelized - maybe even simply cleaned > up in some parts - but unfortunately there remains very much other important > stuff to do. So we'd rather spend some money on a really fast CPU and not > touch the code at the moment. > > So my question was more, what is the fastest CPU I can get for $20000 > at the moment (without explicitly parallelizing, hyperthreading or > vectorizing my code). Ain't happenin'. The factor of 1000, I mean. Not no way, not no how, not without completely rewriting your code. >From this description, the ONLY thing you should be working on is rewriting the code from scratch, doing a serious job of algorithm analysis, optimization, vectorization, parallelization, and (IMHO) not in fortran. If it is graphics/visualization, consider studying things like computer games, render farms, openGL, and various libraries and hardware devices that can move the work required into custom processors. It could be that you are using Fortran to do various data transformations in poorly written software that are now done extremely rapidly in highly specialized hardware (not even on a "CPU" -- on an offboard coprocessor with its own memory and instruction set). Or not. However, you're not getting a factor of 1000 relative to an Athlon 2600 on any hardware in existence for $20K. I only wish...:-) rgb > > > Cheers, > Marc Baaden > > -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb at phy.duke.edu _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From Daniel.Kidger at quadrics.com Fri Jun 20 13:04:39 2003 From: Daniel.Kidger at quadrics.com (Daniel Kidger) Date: Fri, 20 Jun 2003 18:04:39 +0100 Subject: [OT] Maximum performance on single processor ? Message-ID: <010C86D15E4D1247B9A5DD312B7F5AA78DDF95@stegosaurus.bristol.quadrics.com> -----Original Message----- > I have an existing application which is part of a project. I have > the source code. It is Fortran. It *can* be parallelized, but we > would rather spend our time on the other parts of the project > which need to be written from scratch *first*. Fortran is not a problem - for numerical work it should be able to beat C 95% of the time IMHO > The application is to run in real time, that is the user does something > and as a function of user input and the calculation with the fortran > program that I described, there is a correponding feedback to the > user on the screen (and in some Virtual Reality equipment). A few years ago, I was involved in a project that involved doing Finite Elements within a VR environment. It can be done, but you must look closely at your application code to see *why* it is taking over 1000 milliseconds per step: Do you know which bit of code takes the most time? Can you save small arrays and such like to avoid recalculation? How are you getting new data to/from the Fortran - reading and writing files is easy to implement but far too slow for VR. Are you using the appropriate matrix maths libraries for your platform (eg mkl, atlas, libgoto, etc.)? I hope that you don't have any matrix multiplies as 3 nested Do-loops! and even.. Can you use any spare CPUs in a cluster to speculatively calculate parts of the next timestep(s) even if sometimes you have to throw these results away? Yours, Daniel. -------------------------------------------------------------- Dr. Dan Kidger, Quadrics Ltd. daniel.kidger at quadrics.com One Bridewell St., Bristol, BS1 2AA, UK 0117 915 5505 ----------------------- www.quadrics.com -------------------- _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rgb at phy.duke.edu Fri Jun 20 12:51:04 2003 From: rgb at phy.duke.edu (Robert G. Brown) Date: Fri, 20 Jun 2003 12:51:04 -0400 (EDT) Subject: [OT] Maximum performance on single processor ? In-Reply-To: <3EF32555.7040006@attbi.com> Message-ID: On Fri, 20 Jun 2003, Robert Myers wrote: > The only no-brains speedup strategy that could work is the most powerful > speed-up strategy of all: wait a couple of years for off-the-shelf > hardware to get faster. Hmm, Moore's law assuming a very optimistic annual doubling requires log_2(1024) = 10 years to get a kilospeedup. Your conclusions remains unchallenged -- this problem will only resolve with the investment of serious human energy redoing the application -- maybe. Or a (literal) "quantum leap" -- if quantum computers mature sooner than ten years. Assuming that QC's don't require novel, intrinsically parallel programming in their own right and won't just run off-the-shelf serial fortran all that fast. rgb > > RM > > > > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org > To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf > -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb at phy.duke.edu _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From abijur at craft-tech.com Fri Jun 20 13:07:54 2003 From: abijur at craft-tech.com (Ashwin Bijur) Date: Fri, 20 Jun 2003 13:07:54 -0400 Subject: diskless booting & channel bonding Message-ID: <3EF33F6A.60309@craft-tech.com> Can anyone point me to a resource that would give general (or step-by-step) instructions on booting a diskless cluster with channel bonding? Thanks, Ashwin Bijur. abijur at craft-tech.com _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From mikee at mikee.ath.cx Fri Jun 20 13:16:59 2003 From: mikee at mikee.ath.cx (Mike Eggleston) Date: Fri, 20 Jun 2003 12:16:59 -0500 Subject: [OT] Maximum performance on single processor ? In-Reply-To: <031901c3373d$3ea46600$8461cdc2@DREAD>; from jhearns@freesolutions.net on Fri, Jun 20, 2003 at 04:04:22PM +0100 References: <031901c3373d$3ea46600$8461cdc2@DREAD> Message-ID: <20030620121659.H23866@mikee.ath.cx> On Fri, 20 Jun 2003, John Hearns wrote: > My advice: > > Go find your local systems guru. > This person is usually to be found in a dimly lit basement office. You can > recognise it by the > 'Nicht Fur Lookenpeepers' notices and the 1972 lineprinter ASCII art Snoopy > calendar on the > wall. > The person will have disguised a spiffy top of the range monitor with > carefully chosen junk. > On that monitor, he/she will have many windows open. One will be tailing > your mail file (joke). > > Bring gifts of beer. > As him/her for advice: "Please help me to get my program running faster". I resemble that remark! Mike _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From baaden at smplinux.de Fri Jun 20 13:43:27 2003 From: baaden at smplinux.de (Marc Baaden) Date: Fri, 20 Jun 2003 19:43:27 +0200 Subject: [OT] Maximum performance on single processor ? In-Reply-To: Your message of "Fri, 20 Jun 2003 12:51:04 EDT." Message-ID: <200306201743.TAA27894@apex.ibpc.fr> Hi again, first thanks for all the helpful (though a bit discouraging) comments. Firstly I should probably say that the factor of 1000 was what we'd ideally need to be absolutely sure about the feasability. Maybe a factor 10-20 is sufficient, maybe less. Then, just out of pure curiosity, I'd still like to know what kind of speedup by investment I could get for 20k$, let's say eg with respect to the Athlon MB 2600 and which are currently the fastest processors in this pricerange. I think someone "estimated" a factor of 4-5 for a Power4+, is that right ? landman at scalableinformatics.com said: >> As it does not appear that you have profiled the code (just a guess), Half-right half-wrong guess. I have very rapidly profiled the code, just confirming what I already guessed, that the bottleneck (> 80% of the time) is the energy calculation subroutine, which calculates pairwise interactions between atoms in the system. There are indeed potential speedup factors in this code, like - introduction of a neighbour list - parallelization .. or maybe the use of a specialized board (MD-GRAPE) for just these calculations. Unfortunately I currently do lack the competence for parallelization or other optimizations (probably trying to use SSE2 code would be even better), and part of the more specialized methods seems like a complicated and particularly short-lived business. hahn at physics.mcmaster.ca said: >> do you mean you want uniprocessors? you also need to characterize the >> code MUCH better in order to choose a machine. for instance, power* >> and itanium chips are both very fast for some kinds of FP code, but >> miserable for integer-dominated code. We do use floating-point code. hahn at physics.mcmaster.ca said: >> have you actually done paper calculations to see whether the >> performance you observe is sensible/expected? ie, that you're not >> triggering some pathological behavior, even bugs? Yes. The code is tested (> 10 years). But we know it has not been developed with speed in mind (my contest for the understatement of the year) >> have you done the >> high-resolution measuring to know whether your user inputs are >> actually arriving without being blocked (for instance, on 20ms >> scheduler ticks)? I have measured the calculation separately for now. Not even doing the user part. But I do have an alternative program which already does run. So I know that the method in itself is working. It is just the calculation module which is in Fortran that is not yet adapted. hahn at physics.mcmaster.ca said: >> an athlon MP 2600+) >> a quite poor architecture for today; it was competitive a year ago. oops, we must have been very ill advised then, because the machine is just a month old, and was the "top" model from our vendor. Or is there such a gap between the US and the European market ? mikee at mikee.ath.cx said: >> Can the lisp technique of memoization help with your processing? Sorry, I don't know about this. What is it about ? beowulf-request at scyld.com said: >> Also, you should consider how the code is written - if you are >> manipulating large arrays then the order of access, and the stride >> length can make a big difference. You may be ranging all over a huge >> array, pulling things in from memory(*) when a different algorithm/ >> different loop arrangement could keep things in cache. So - if I interpret your and other people's statements right - depending on the code, memory might be as important/critical as CPU power ? Is there a way to check on what the code depends most ? >> And Fortran, >> being 'closer to the machine' can expose these things (I may be >> wrong). I don't see how (but I am not very experienced in this), any more direct ideas ? >> Have you looked at books like: "High Performance Computing" >> http://www.oreilly.com/catalog/hpc2/ No, not yet. I will definitely need some guidance into how to best optimize the code. beowulf-request at scyld.com said: >> (*) Sorry to teach my granny to suck eggs, but your machine isn't >> going into swap I trust. I was about to say no it is not .. but I shall give it a serious check. johnb at quadrics.com said: >> If it looks like it would be a major project, you could get a summer >> student to get on with doing it whilst the rest of the folks are doing >> the other stuff. It is (going to be) a major project. I would love having a summer student doing the job, but I certainly haven't been able to get hold of someone with the corresponding competences. (Also, in general the people I met who might be competent, did not program in Fortran :)) ) rgb at phy.duke.edu said: >> From this description, the ONLY thing you should be working on is >> rewriting the code from scratch, doing a serious job of algorithm >> analysis, optimization, vectorization, parallelization, and (IMHO) not >> in fortran. >> If it is graphics/visualization, consider studying things like >> computer games, render farms, openGL, and various libraries and >> hardware devices that can move the work required into custom >> processors. It could be that you are using Fortran to do various data >> transformations in poorly written software that are now done extremely >> rapidly in highly specialized hardware (not even on a "CPU" -- on an >> offboard coprocessor with its own memory and instruction set). Or >> not. These are all excellent remarks. But I do totally lack competence or knowledge in these fields, and it seems extremely difficult/dangerous to me to make the right choices in this context or to get an overview of what the best ways are to optimize the code/algorithm. Also I do not (yet) know the fortran code very well, as I am not developing/ maintaining it, and this does not really help in making those choices. Well, if you are still with me, then thanks for reading through all this. Cheers, Marc Baaden -- Dr. Marc Baaden - Institut de Biologie Physico-Chimique, Paris mailto:baaden at smplinux.de - http://www.marc-baaden.de FAX: +49 697912 39550 - Tel: +33 15841 5176 ou +33 609 843217 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From ctierney at hpti.com Fri Jun 20 13:43:39 2003 From: ctierney at hpti.com (Craig Tierney) Date: 20 Jun 2003 11:43:39 -0600 Subject: [OT] Maximum performance on single processor ? In-Reply-To: References: Message-ID: <1056131019.11801.1493.camel@woody> stuff deleted.... > Ain't happenin'. The factor of 1000, I mean. Not no way, not no how, > not without completely rewriting your code. > > >From this description, the ONLY thing you should be working on is > rewriting the code from scratch, doing a serious job of algorithm > analysis, optimization, vectorization, parallelization, and (IMHO) not > in fortran. If someone else had said this I would probably have had some snide comment and caused some sort of flame war. However after noticing the comment came from a helpful and knowledgeable list member I will have to hold back. However, I really want to know why 'not in fortran'? Not everyone is a programming god and can visualise how a particular piece of code is going to end up in assembler. You can get good performance out of C and C++ but you have to be careful, and I still don't know if you can beat Fortran in many cases. In a nutshell, without starting a flamewar (too late), what's wrong with Fortran for high performance numerical codes? PS. I still hate Fortran, never liked it. However I have found that it does the job. Craig > > If it is graphics/visualization, consider studying things like computer > games, render farms, openGL, and various libraries and hardware devices > that can move the work required into custom processors. It could be > that you are using Fortran to do various data transformations in poorly > written software that are now done extremely rapidly in highly > specialized hardware (not even on a "CPU" -- on an offboard coprocessor > with its own memory and instruction set). Or not. > > However, you're not getting a factor of 1000 relative to an Athlon 2600 > on any hardware in existence for $20K. I only wish...:-) > > rgb > > > > > > > Cheers, > > Marc Baaden > > > > -- Craig Tierney _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From waitt at saic.com Fri Jun 20 14:02:10 2003 From: waitt at saic.com (Tim Wait) Date: Fri, 20 Jun 2003 14:02:10 -0400 Subject: [OT] Maximum performance on single processor ? References: Message-ID: <3EF34C22.6090301@saic.com> As others have suggested, profile the code. You may be suffering from excessive cache misses. The Athlon 2600 only has 256k L2. A relatively cheap fix for that could be simply getting a proc with lots of cache. ie, alpha, MIPS, etc. You should be able to find a vendor to let you run some benchmarks. Tim _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From xyzzy at speakeasy.org Fri Jun 20 14:07:59 2003 From: xyzzy at speakeasy.org (Trent Piepho) Date: Fri, 20 Jun 2003 11:07:59 -0700 (PDT) Subject: [OT] Maximum performance on single processor ? In-Reply-To: <3EF32555.7040006@attbi.com> Message-ID: On Fri, 20 Jun 2003, Robert Myers wrote: > You may have done better service to the readers of this list than you > have to yourself, since the readers of this list are naturally inclined > to think about the sort of question you have asked, and the answers have > been interesting to read. > > The fact that you are asking the question, though, begs an answer that > others have partially given: no plausible hardware purchase you can make > will be a substitute for you or someone you are working with to get I'm in agreement with what everyone else has said, if you're using an AthlonMP 2600 now, no < $20k single cpu system will give you a factor of 1000 speedup. A factor of two is probably the best you can hope for. But, the original question is still unanswered. If you had a non-parallel scientific fortran code, what current system <$20k would run it the fastest? Opteron, Itanium, Xeon, Athlon, Power4? Of course, I realize that the correct answer is, "it depends." Things like memory access patterns, suitability for SIMD instructions, floating point mix, etc. all make a big difference. But without that knowledge, what would the best guess be? _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From mikee at mikee.ath.cx Fri Jun 20 14:28:51 2003 From: mikee at mikee.ath.cx (Mike Eggleston) Date: Fri, 20 Jun 2003 13:28:51 -0500 Subject: [OT] Maximum performance on single processor ? In-Reply-To: <5.1.1.6.2.20030620100637.03ca3450@mail.harddata.com>; from maurice@harddata.com on Fri, Jun 20, 2003 at 10:09:54AM -0600 References: <200306201502.h5KF2EU21424@NewBlue.Scyld.com> <5.1.1.6.2.20030620100637.03ca3450@mail.harddata.com> Message-ID: <20030620132851.J23866@mikee.ath.cx> On Fri, 20 Jun 2003, Maurice Hilarius wrote: > With regards to your message: > >From: Marc Baaden > > <<>> > > >So my question was more, what is the fastest CPU I can get for $20000 > >at the moment (without explicitly parallelizing, hyperthreading or > >vectorizing my code). > > > I believe the answer is an Opteron. > Of course that much money will buy quite a lot of CPUs! > > With the memory bandwidth, floating point engine, and Hypertransport links > on these, they are VERY fast for this type of thing. > If you want access to a machine to run some code samples on, please contact > me off-list and we can arrange some time for you on a machine. > We have test/demo dual Opteron servers in place, available over SSH > connections, for exactly this purpose. > They are set up with compilers, libraries and tools suitable for this use. If you have something like RH7+ on one of these boxes, would you please send me the output of 'cat /proc/cpuinfo'? Thanks. Mike _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From mikee at mikee.ath.cx Fri Jun 20 15:08:25 2003 From: mikee at mikee.ath.cx (Mike Eggleston) Date: Fri, 20 Jun 2003 14:08:25 -0500 Subject: [OT] Maximum performance on single processor ? In-Reply-To: ; from rgb@phy.duke.edu on Fri, Jun 20, 2003 at 12:51:04PM -0400 References: <3EF32555.7040006@attbi.com> Message-ID: <20030620140825.K23866@mikee.ath.cx> On Fri, 20 Jun 2003, Robert Myers wrote: > The only no-brains speedup strategy that could work is the most powerful > speed-up strategy of all: wait a couple of years for off-the-shelf > hardware to get faster. The code that executes the fastest is the code that does not exist. Mike _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From xyzzy at speakeasy.org Fri Jun 20 15:15:24 2003 From: xyzzy at speakeasy.org (Trent Piepho) Date: Fri, 20 Jun 2003 12:15:24 -0700 (PDT) Subject: [OT] Maximum performance on single processor ? In-Reply-To: <20030620132851.J23866@mikee.ath.cx> Message-ID: On Fri, 20 Jun 2003, Mike Eggleston wrote: > > With the memory bandwidth, floating point engine, and Hypertransport links > > on these, they are VERY fast for this type of thing. > > If you want access to a machine to run some code samples on, please contact > > me off-list and we can arrange some time for you on a machine. > > We have test/demo dual Opteron servers in place, available over SSH > > connections, for exactly this purpose. > > They are set up with compilers, libraries and tools suitable for this use. > > If you have something like RH7+ on one of these boxes, would you please send > me the output of 'cat /proc/cpuinfo'? Thanks. How about the kernel startup messages too? I want to a five digit bogomips value! It's like CPU pr0n..... _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rene at renestorm.de Fri Jun 20 18:01:35 2003 From: rene at renestorm.de (ReneStorm) Date: Sat, 21 Jun 2003 00:01:35 +0200 Subject: Pallas and LanaiXP, sporadic gaps Message-ID: Hi Folks, does anyone have experience with the Pallas Benchmark and the new Myrinet LanaiXP Cards ? Problem: I ve got some strang, sporadic gaps on our 32 Node (64 CPU) Cluster. Example: processes 32: (32 waiting in barrier) SendRecv byte Mb/sec 512 74.54 1024 129.78 2048 193.25 4096 252.89 8192 46.84 16384 70.79 32768 230.63 32768 256.37 Cluster Equipment: 32x Myrinet Lanai M3F-PCIXD-2 Myrinet-Fiber/PCI-X Interface, 64x 2.4 Ghz, 32 GB Mem All myrinet cards are running with 133 MHz, (PCI read/write above 800 MB/sec). Compiler: gcc, pgroup, intel The configuration looks fine. Cards are very fast. 8 byte PingPong below 7usec, Peak Bandwidth near 500 MB/sec Thanks in advance and have a nice weekend. Rene _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From patrick at myri.com Fri Jun 20 19:44:07 2003 From: patrick at myri.com (Patrick Geoffray) Date: 20 Jun 2003 19:44:07 -0400 Subject: Pallas and LanaiXP, sporadic gaps In-Reply-To: References: Message-ID: <1056152649.964.30.camel@asterix> Hi Rene, On Fri, 2003-06-20 at 18:01, ReneStorm wrote: > does anyone have experience with the Pallas Benchmark and the new Myrinet > LanaiXP Cards ? > > Problem: I ve got some strang, sporadic gaps on our 32 Node (64 CPU) > Cluster. I think this is a known bug that has been fixed recently by the GM-2 folks. They released an update (GM 2.0.1) yesterday. You should see the number of resends going up big time in the GM counters at the same time than the hiccups. Please let me know if you still see it with the latest GM. Patrick -- Patrick Geoffray Myricom, Inc. http://www.myri.com _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From bushnell at chem.ucsb.edu Fri Jun 20 20:19:44 2003 From: bushnell at chem.ucsb.edu (John Bushnell) Date: Fri, 20 Jun 2003 17:19:44 -0700 (PDT) Subject: [OT] Maximum performance on single processor ? In-Reply-To: <200306201743.TAA27894@apex.ibpc.fr> Message-ID: On Fri, 20 Jun 2003, Marc Baaden wrote: > landman at scalableinformatics.com said: > >> As it does not appear that you have profiled the code (just a guess), > > Half-right half-wrong guess. I have very rapidly profiled the code, just > confirming what I already guessed, that the bottleneck (> 80% of the time) > is the energy calculation subroutine, which calculates pairwise interactions > between atoms in the system. > There are indeed potential speedup factors in this code, like > - introduction of a neighbour list > - parallelization > .. or maybe the use of a specialized board (MD-GRAPE) for just these > calculations. Looking at some of the info on the Riken site who evidently makes these GRAPE cards, this sound like a good bet if pairwise energy calculations are really where most of the speedup is needed. They give a peak performance for the 4-cpu GRAPE-2 as 64GFLOPS (ouch). Very interesting. I've never heard of these cards. I wonder what the approx. cost is? They look very easy to implement, and even have a really simple piece of Fortran code showing how to use one on their web site. One of their benchmarks shows an almost 100x speedup using three cards on a Sun... - John _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From Serguei.Patchkovskii at sympatico.ca Fri Jun 20 21:16:44 2003 From: Serguei.Patchkovskii at sympatico.ca (Serguei Patchkovskii) Date: Fri, 20 Jun 2003 21:16:44 -0400 Subject: cluster of AOD Opteron References: <200306201342.RAA28782@nocserv.free.net> Message-ID: <005701c33792$c7c1ddf0$6501a8c0@sims.nrc.ca> > The price for Opteron- > based servers is high, and price/performance ratio in comparison > w/Xeon is not clear. Once you start populating your systems with "interesting" amounts of memory (i.e. anything above 2Gbytes), the price difference between dual Opterons and dual Xeons is really in the noise - at least at the places we buy. If your suppliers charge you a lot more for Opterons, may be you should look for another source? Serguei _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From hahn at physics.mcmaster.ca Sat Jun 21 03:01:32 2003 From: hahn at physics.mcmaster.ca (Mark Hahn) Date: Sat, 21 Jun 2003 03:01:32 -0400 (EDT) Subject: [OT] Maximum performance on single processor ? In-Reply-To: Message-ID: > > If you have something like RH7+ on one of these boxes, would you please send > > me the output of 'cat /proc/cpuinfo'? Thanks. > > How about the kernel startup messages too? I want to a five digit bogomips > value! It's like CPU pr0n..... they're not *that* interesting: Total of 2 processors activated (5570.56 BogoMIPS). if you're looking for bogo-max, you'd look for a larger machine (IBM ccNUMA? GS320 or Marvel?) _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From seth at hogg.org Sat Jun 21 06:23:46 2003 From: seth at hogg.org (Simon Hogg) Date: Sat, 21 Jun 2003 11:23:46 +0100 Subject: [OT] Maximum performance on single processor ? In-Reply-To: References: <200306201743.TAA27894@apex.ibpc.fr> Message-ID: <4.3.2.7.2.20030621111838.00aec9f0@pop.freeuk.net> At 17:19 20/06/03 -0700, John Bushnell wrote: > Looking at some of the info on the Riken site who evidently makes >these GRAPE cards, this sound like a good bet if pairwise energy >calculations are really where most of the speedup is needed. They >give a peak performance for the 4-cpu GRAPE-2 as 64GFLOPS (ouch). http://www.research.ibm.com/grape/ I hadn't heard of these either, but a quote from the IBM site; "These chips will be combined with other special chips at RIKEN later this year to make a Molecular Dynamics Machine, which will run at a whopping 100 Teraflops speed, the fastest for any computer in the world." There seems to be a fair bit of marketing speak at that site as well :-) -- Simon _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From christopher.yip at utoronto.ca Sat Jun 21 17:15:35 2003 From: christopher.yip at utoronto.ca (Christopher Yip) Date: Sat, 21 Jun 2003 17:15:35 -0400 Subject: MDGRAPE In-Reply-To: <200306211901.h5LJ17U15703@NewBlue.Scyld.com> Message-ID: <7F9D857E-A42D-11D7-A1EE-000A9587184C@utoronto.ca> Here's a link to that MD GRAPE info: http://www.peta.co.jp/ and a couple of their users: http://aci5.astronomy.pomona.edu/docs/grape/MD- GRAPE2_implementation.html Christopher M. Yip, Ph.D., P.Eng. Associate Professor - Canada Research Chair in Molecular Imaging Departments of Chemical Engineering and Applied Chemistry Department of Biochemistry Institute of Biomaterials and Biomedical Engineering 407 Rosebrugh Building Toronto, Ontario, CANADA M5S 3G9 (416) 978-7853 (416) 978-4317 (fax) christopher.yip at utoronto.ca http://bigten.ibme.utoronto.ca _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From iwao at rickey-net.com Sat Jun 21 23:55:47 2003 From: iwao at rickey-net.com (Iwao Makino) Date: Sun, 22 Jun 2003 12:55:47 +0900 Subject: [OT] Re:GRAPE In-Reply-To: References: Message-ID: For those of whom interested for Grape(s) ... There's another GRAPE(not MDGRAPE) This is for Astronomical simulation ONLY but... Is faster than earth simulator of NEC. Below site talk about both At 17:19 -0700 on 03.6.20, John Bushnell wrote regarding Re: [OT] Maximum performance on single processor ?: > Looking at some of the info on the Riken site who evidently makes >these GRAPE cards, this sound like a good bet if pairwise energy >calculations are really where most of the speedup is needed. They >give a peak performance for the 4-cpu GRAPE-2 as 64GFLOPS (ouch). >Very interesting. I've never heard of these cards. I wonder what >the approx. cost is? I see MDGRAPE-2 costs shown about $17,000 #Not sure they include booster card or not. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From landman at scalableinformatics.com Sat Jun 21 09:37:35 2003 From: landman at scalableinformatics.com (Joseph Landman) Date: 21 Jun 2003 09:37:35 -0400 Subject: [OT] Maximum performance on single processor ? In-Reply-To: <200306201743.TAA27894@apex.ibpc.fr> References: <200306201743.TAA27894@apex.ibpc.fr> Message-ID: <1056202654.4432.24.camel@protein.scalableinformatics.com> On Fri, 2003-06-20 at 13:43, Marc Baaden wrote: > landman at scalableinformatics.com said: > >> As it does not appear that you have profiled the code (just a guess), > > Half-right half-wrong guess. I have very rapidly profiled the code, just > confirming what I already guessed, that the bottleneck (> 80% of the time) > is the energy calculation subroutine, which calculates pairwise interactions > between atoms in the system. Excellent. I am glad I was half wrong :). > There are indeed potential speedup factors in this code, like > - introduction of a neighbour list > - parallelization > .. or maybe the use of a specialized board (MD-GRAPE) for just these > calculations. One of the fastest speedup methods I have used for someone to accelerate single processor performance of their code was to look at the pair interactions, which required a square-root operation, and recast that from force (which required the square-root operation) to potential energy, which did not. That gave me a factor of 5 in that loop which had been dominating the calculation. If you can live with 2 potential energy evaluations at a fixed interval, you might be able to get away from the square root. You also want to rewrite the innards to get away from division, even to the point of pre-calculating it and playing games there. Most x86 do not have a hardware accelerated 1/x, or 1/sqrt(x). Power4 ABI, MIPS n64 ABI, and the PA-RISC ABI do. The compilers are even "smart enough" in the case of the SGI compilers, or reasonable enough to help recast your calculations this way. Most of the pair-wise interactions I have encountered in research codes has been a high precision pair potential, or a high precision sum over all contributions from all interacting orbitals. If you are dealing wit MD (sounds like you are), you might look at whether or not you need as high a precision. Other codes I have worked on use cut-off radii which limit the number of neighbors (and the search through the system for neighbors with which to compute interactions). Square-roots tend to pop-up here as well, in the distance calculation. It is (usually) trivial (and still meaningful) to avoid distance, and do a distance squared, so you don't have to pay for the expensive square root. > Unfortunately I currently do lack the competence for parallelization or other > optimizations (probably trying to use SSE2 code would be even better), and > part of the more specialized methods seems like a complicated and particularly > short-lived business. Understood. You might explore with various groups whether or not they could help you. You might be able to work with a local computer science, or physics/chemistry department to locate an exceptional student who would gain from the experience of working with a company. You certainly could find a company to at least assess the situation for you and let you know whether or not it is feasable. If it were, I would suspect the changes would cost less than 20k$. [...] > hahn at physics.mcmaster.ca said: > >> an athlon MP 2600+) > >> a quite poor architecture for today; it was competitive a year ago. I disagree with the characterization of "quite poor". On certain classes of code, it is significantly better than its direct competition, though on other classes it is significantly worse. [...] > So - if I interpret your and other people's statements right - depending on > the code, memory might be as important/critical as CPU power ? > Is there a way to check on what the code depends most ? Yes. Using a combination of profiling techniques, and exploitation of CPU profiling counters, you should be able to track the most expensive portion of the calculation, and why it is most expensive. See the Oprofile bit on http://freshmeat.net (http://freshmeat.net/projects/oprofile/?topic_id=45) and Troy Baer's excellent lperfex (http://freshmeat.net/projects/lperfex/?topic_id=861). -- Joseph Landman, Ph.D Scalable Informatics LLC email: landman at scalableinformatics.com web: http://scalableinformatics.com phone: +1 734 612 4615 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From andrewxwang at yahoo.com.tw Mon Jun 23 23:22:59 2003 From: andrewxwang at yahoo.com.tw (=?big5?q?Andrew=20Wang?=) Date: Mon, 23 Jun 2003 20:22:59 -0700 (PDT) Subject: Opteron (x86-64) compute farms/clusters? Message-ID: <20030624032259.48447.qmail@web16809.mail.tpe.yahoo.com> How well the existing tools run on Opteron machines? Does LAM-MPI or MPICH run in 64-bit mode? Also, has anyone tried Gridengine or PBS on it? Lastly, is there an opensource Opteron compile farm that I can access? I would like to see if my code really runs correctly on them before buying! Andrew. __________________________________ Do you Yahoo!? SBC Yahoo! DSL - Now only $29.95 per month! http://sbc.yahoo.com _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From Thomas.Alrutz at dlr.de Tue Jun 24 04:19:48 2003 From: Thomas.Alrutz at dlr.de (Thomas Alrutz) Date: Tue, 24 Jun 2003 10:19:48 +0200 Subject: Opteron (x86-64) compute farms/clusters? References: <20030624032259.48447.qmail@web16809.mail.tpe.yahoo.com> Message-ID: <3EF809A4.1050802@dlr.de> I just made some benchmarks on a Opteron 240 (1.4 GHz) node running with Suse/United Linux Enterprise edition. I have sucessfully compiled mpich-1.2.4 in 64 bit without any problems (./configure -device=ch_p4 -commtype=shared). The default compiler is the gcc-3.2.2 (maybe a Suse patch) and is set to 64Bit, the Portland (5.0beta) compiler didn't worked at all ! I tried our CFD-code (TAU) to run 3 aerodynamik configurations on this machine with both CPUs and the results are better then estimated. We achieved in full multigrid (5 cycles, 1 equation turbulence model) a efficiency of about 97%, 92% and 101 % for the second CPU. Those results are much better as the results we get on the Intel Xeons (around 50%). Thomas -- __/|__ | Dipl.-Math. Thomas Alrutz /_/_/_/ | DLR Institut fuer Aerodynamik und Stroemungstechnik |/ | Numerische Verfahren DLR | Bunsenstr. 10 | D-37073 Goettingen/Germany _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rodmur at maybe.org Tue Jun 24 09:18:57 2003 From: rodmur at maybe.org (Dale Harris) Date: Tue, 24 Jun 2003 06:18:57 -0700 Subject: [OT] Omega Nomad OM-44 Message-ID: <20030624131857.GM7917@maybe.org> Hi, I realize this is kind of off topic. But don't exactly know where else to ask off hand. Just curious if anyone has played with one of the Omega Nomad OM-40 family of temperature sensors: http://www.omega.com/pptst/OM-40.html And got it running under Linux? I wasn't very successful in getting information out of Omega about it other than that they don't support Linux. It doesn't look like it should be too hard to do since it has a serial access. I thought it might be a fairly nice device for the machine room, it's really small, put it about it anywhere. Relatively cheap. -- Dale Harris rodmur at maybe.org /.-) _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From shewa at inel.gov Tue Jun 24 10:09:09 2003 From: shewa at inel.gov (Andrew Shewmaker) Date: Tue, 24 Jun 2003 08:09:09 -0600 Subject: Opteron (x86-64) compute farms/clusters? In-Reply-To: <20030624032259.48447.qmail@web16809.mail.tpe.yahoo.com> References: <20030624032259.48447.qmail@web16809.mail.tpe.yahoo.com> Message-ID: <3EF85B85.1090200@inel.gov> Andrew Wang wrote: > How well the existing tools run on Opteron machines? > > Does LAM-MPI or MPICH run in 64-bit mode? Also, has > anyone tried Gridengine or PBS on it? > > Lastly, is there an opensource Opteron compile farm > that I can access? I would like to see if my code > really runs correctly on them before buying! > > Andrew. Most vendors will give you a remote account or send you an evaluation unit. I imagine you'll probably be contacted off-list by several of them. I've compiled a 64-bit MPICH, GROMACS, and a few other codes with a GCC 3.3 prerelease. I have also used the beta PGI compiler with good results. Some build scripts require slight modification to recognize x86-64 as an architecture, but most porting is trivial. GROMACS has some optimized assembly that didn't come out quite right, but I bet they have it fixed by now. All my testing was a couple of weeks before the release, but I haven't gotten any in yet unfortunately. Andrew -- Andrew Shewmaker, Associate Engineer Phone: 1-208-526-1276 Idaho National Eng. and Environmental Lab. P.0. Box 1625, M.S. 3605 Idaho Falls, Idaho 83415-3605 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From Luc.Vereecken at chem.kuleuven.ac.be Tue Jun 24 13:22:12 2003 From: Luc.Vereecken at chem.kuleuven.ac.be (Luc Vereecken) Date: Tue, 24 Jun 2003 19:22:12 +0200 Subject: [OT] Omega Nomad OM-44 In-Reply-To: <20030624131857.GM7917@maybe.org> Message-ID: <3.0.6.32.20030624192212.008f1100@arrhenius.chem.kuleuven.ac.be> At 06:18 AM 6/24/03 -0700, Dale Harris wrote: >Just curious if anyone has played with one of the Omega Nomad >OM-40 family of temperature sensors: > >http://www.omega.com/pptst/OM-40.html > >And got it running under Linux? I wasn't very successful in getting >information out of Omega about it other than that they don't support >Linux. A nice alternative are the devices by picotech http://www.picotech.com which are supported under linux and windows with drivers and logging software. They have a linux driver that one can access easily through shared memory (C-example program can be downloaded). I use it to monitor the temperature in my computer room, log it, do temperature-related shutdowns, etc. all based on very simple perl scripts (I use the TH-03 with 3 thermistor-based temperature measurement channels for air-inlet, ambient air, and air-outlet). Nice equipment, works very well. Luc Vereecken _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From nixon at nsc.liu.se Tue Jun 24 15:39:30 2003 From: nixon at nsc.liu.se (nixon at nsc.liu.se) Date: Tue, 24 Jun 2003 21:39:30 +0200 Subject: [OT] Omega Nomad OM-44 In-Reply-To: <3.0.6.32.20030624192212.008f1100@arrhenius.chem.kuleuven.ac.be> (Luc Vereecken's message of "Tue, 24 Jun 2003 19:22:12 +0200") References: <3.0.6.32.20030624192212.008f1100@arrhenius.chem.kuleuven.ac.be> Message-ID: Luc Vereecken writes: > A nice alternative are the devices by picotech > http://www.picotech.com > which are supported under linux and windows with drivers and logging > software. They have a linux driver that one can access easily through > shared memory (C-example program can be downloaded). I use it to monitor > the temperature in my computer room, log it, do temperature-related > shutdowns, etc. all based on very simple perl scripts (I use the TH-03 with > 3 thermistor-based temperature measurement channels for air-inlet, ambient > air, and air-outlet). Nice equipment, works very well. I second that. I have a TH-03 and a RH-02 in our computer room, which have saved us a couple of times when the campus cooling system went down. And since the serial protocol is documented you can easily replace the supplied driver with your own. I've written a little Python daemon that polls the units and makes the measurements available via XMLRPC, which fits our monitoring setup nicely. -- Leif Nixon Systems expert ------------------------------------------------------------ National Supercomputer Centre Linkoping University ------------------------------------------------------------ _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From hahn at physics.mcmaster.ca Tue Jun 24 18:14:06 2003 From: hahn at physics.mcmaster.ca (Mark Hahn) Date: Tue, 24 Jun 2003 18:14:06 -0400 (EDT) Subject: [OT] Omega Nomad OM-44 In-Reply-To: <3.0.6.32.20030624192212.008f1100@arrhenius.chem.kuleuven.ac.be> Message-ID: > A nice alternative are the devices by picotech > http://www.picotech.com I prefer to use iButtons. these are button-sized devices that you hang off a two-wire bus, and plug into some computer's serial port. everything is utterly open, and it's cheap and flexible. I have 6 sensors in my machineroom: http://sharcnet.mcmaster.ca/~hahn/cgi-bin/temp.cgi which are very trivially collected and logged to the obvious kind of sql table. www.ibutton.com. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rodmur at maybe.org Tue Jun 24 19:21:28 2003 From: rodmur at maybe.org (Dale Harris) Date: Tue, 24 Jun 2003 16:21:28 -0700 Subject: [OT] Omega Nomad OM-44 In-Reply-To: References: <3.0.6.32.20030624192212.008f1100@arrhenius.chem.kuleuven.ac.be> Message-ID: <20030624232128.GR7917@maybe.org> On Tue, Jun 24, 2003 at 06:14:06PM -0400, Mark Hahn elucidated: > > http://www.picotech.com > > www.ibutton.com. > grr.. wish I had known about those before buying the Nomad. Dale (/me slaps forhead) _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rgb at phy.duke.edu Wed Jun 25 11:56:13 2003 From: rgb at phy.duke.edu (Robert G. Brown) Date: Wed, 25 Jun 2003 11:56:13 -0400 (EDT) Subject: [OT] Omega Nomad OM-44 In-Reply-To: <20030624232128.GR7917@maybe.org> Message-ID: On Tue, 24 Jun 2003, Dale Harris wrote: > On Tue, Jun 24, 2003 at 06:14:06PM -0400, Mark Hahn elucidated: > > > http://www.picotech.com > > > > www.ibutton.com. > > > > grr.. wish I had known about those before buying the Nomad. > Both of these are now on the brahma resources page, along with a third vendor's thermal sensor link. > (/me slaps forhead) You shouldn't feel too bad. It actually has taken years for there to be an acceptable crop of low-cost offerings in this arena. For a while there, one had to either DIY or use something as expensive as a Netbotz in order to get online environmental monitoring -- there was only one cheap device that would work in Linux and the tiny company that made/resold it bellied up. At least now there is choice, and the choices are all <$200! For an extra $100 or so, you can add a PC-TV card and an X10 and build yourself a netbotz out of any system with a bit of attention to spare for polling, or buy one of the legendary wal-mart specials and build a complete net monitoring appliance with video, thermal, humidity, door monitoring for <$600. rgb > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org > To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf > -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb at phy.duke.edu _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From philippe.blaise at cea.fr Wed Jun 25 05:28:26 2003 From: philippe.blaise at cea.fr (phblai) Date: Wed, 25 Jun 2003 11:28:26 +0200 Subject: Opteron (x86-64) compute farms/clusters? In-Reply-To: <200306241612.UAA09513@nocserv.free.net> References: <200306241612.UAA09513@nocserv.free.net> Message-ID: <3EF96B3A.70601@cea.fr> Mikhail Kuzminsky wrote: >According to Thomas Alrutz > > >>I just made some benchmarks on a Opteron 240 (1.4 GHz) node running with >>Suse/United Linux Enterprise edition. >>I have sucessfully compiled mpich-1.2.4 in 64 bit without any problems >>(./configure -device=ch_p4 -commtype=shared). The default compiler is >>the gcc-3.2.2 (maybe a Suse patch) and is set to 64Bit, the Portland >>(5.0beta) compiler didn't worked at all ! >> >>I tried our CFD-code (TAU) to run 3 aerodynamik configurations on this >>machine with both CPUs and the results are better then estimated. >>We achieved in full multigrid (5 cycles, 1 equation turbulence model) a >>efficiency of about 97%, 92% and 101 % for the second CPU. >>Those results are much better as the results we get on the Intel Xeons >>(around 50%). >> >> > It looks that this results are predictable: Xeon CPUs require high >memory bandwidth, but both CPUs share common system bus. Opteron CPUs >have own memory buses and scale in this sense excellent. Better SPECrate >results for Opteron (i.e. work on a mix of tasks) confirm (in particular) >this features. CFD codes, I beleive, require high memory throughput ... > >Mikhail Kuzminsky >Zelinsky Institute of Organic Chemistry >Moscow > > >_______________________________________________ >Beowulf mailing list, Beowulf at beowulf.org >To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf > > > > the 4 cpus (ccNUMA) AMD Opteron scales very well (each cpu has its own memory with a 5.3 GB/s bandwith access), compare to the (SMP) Intel / Itanium2 equivalent (I870 chipset), where the 4 cpus share a common 6.4GB/s memory access. The 4 cpus Opteron is cheap, but the Itanium2 can perform quite better on some benchs (the 3MB L3 cache has a 32 GB/s bandwith !). In fact, the AMD Opteron architecture is very close to the HP/Compaq/Digital EV7 Marvel one, that scales very well to 32 cpus (maybe 64 ?). But I don' know if AMD plans to make machines with more than 8 cpus ? Philippe Blaise _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From kuku at physik.rwth-aachen.de Wed Jun 25 09:46:38 2003 From: kuku at physik.rwth-aachen.de (Christoph Kukulies) Date: Wed, 25 Jun 2003 15:46:38 +0200 Subject: 3Com 905C and ASUS P4GX8 Message-ID: <200306251346.h5PDkcF19321@accms33.physik.rwth-aachen.de> Hi, I installed Linux RH 7.3 on a Dual 3GHz P4 CPU Asus P4GX8 MB and since the on board 100 MBit controller didn't seem to be supported by this Linux version (doubt whether any version does in the moment) I put in a 3Com 905c and run it with the 509 vortex driver. I'm seeing this in /var/log/messages and the machine is very slow on network copies and activity. Jun 25 10:24:34 accms07 kernel: 14: @c33a3580 length 800000be status 000100beJun 25 10:24:34 accms07 kernel: 15: @c33a35c0 length 800000be status 000100beJun 25 10:24:34 accms07 kernel: eth0: Transmit error, Tx status register 82. Jun 25 10:24:34 accms07 kernel: Probably a duplex mismatch. See Documentation/networking/vortex.txt Jun 25 10:24:34 accms07 kernel: Flags; bus-master 1, dirty 156838(6) current 156838(6) Jun 25 10:24:34 accms07 kernel: Transmit list 00000000 vs. c33a3380. Jun 25 10:24:34 accms07 kernel: 0: @c33a3200 length 800000be status 000100be Jun 25 10:24:34 accms07 kernel: 1: @c33a3240 length 800000be status 000100be Jun 25 10:24:34 accms07 kernel: 2: @c33a3280 length 800000be status 000100be Jun 25 10:24:34 accms07 kernel: 3: @c33a32c0 length 800000be status 000100be Jun 25 10:24:34 accms07 kernel: 4: @c33a3300 length 800000be status 000100be Jun 25 10:24:34 accms07 kernel: 5: @c33a3340 length 800000be status 800100be Jun 25 10:24:34 accms07 kernel: 6: @c33a3380 length 800000be status 000100be Jun 25 10:24:34 accms07 kernel: 7: @c33a33c0 length 800000be status 000100be Jun 25 10:24:34 accms07 kernel: 8: @c33a3400 length 800000be status 000100be Jun 25 10:24:34 accms07 kernel: 9: @c33a3440 length 800000be status 000100be Jun 25 10:24:34 accms07 kernel: 10: @c33a3480 length 800000be status 000100beJun 25 10:24:34 accms07 kernel: 11: @c33a34c0 length 800000be status 000100beJun 25 10:24:34 accms07 kernel: 12: @c33a3500 length 800000be status 000100beJun 25 10:24:34 accms07 kernel: 13: @c33a3540 length 800000be status 000100be Any clues? -- Chris Christoph P. U. Kukulies kukulies (at) rwth-aachen.de _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From luis.licon at yakko.cimav.edu.mx Wed Jun 25 17:53:21 2003 From: luis.licon at yakko.cimav.edu.mx (Luis Fernando Licon Padilla) Date: Wed, 25 Jun 2003 15:53:21 -0600 Subject: Off topic - G5 question Message-ID: <3EFA19D1.7010009@yakko.cimav.edu.mx> Hi you all guys, I've just checked the G5 processor out, what do you think about it, it seems like it's not good enough considering it's architecture and it's performance. http://www.apple.com/lae/g5 Cheerz, Luis -- ISC Luis Fernando Licon Padilla Telecommunications Department Advanced Materials Research Center Miguel de Cervantes 120 Complejo Industrial Chihuahua C.P.31109 Chihuahua, Chih. Mexico Phone: 52 (614)4391154 Fax: 52 (614)4391112 alternative e-mail: lordsirion2002 at yahoo.com _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From seth at hogg.org Wed Jun 25 18:42:59 2003 From: seth at hogg.org (Simon Hogg) Date: Wed, 25 Jun 2003 23:42:59 +0100 Subject: Off topic - G5 question In-Reply-To: <3EFA19D1.7010009@yakko.cimav.edu.mx> Message-ID: <4.3.2.7.2.20030625234133.00ac75a0@pop.freeuk.net> At 15:53 25/06/03 -0600, Luis Fernando Licon Padilla wrote: >Hi you all guys, I've just checked the G5 processor out, what do you think >about it, it seems like it's not good enough considering it's architecture >and it's performance. > >http://www.apple.com/lae/g5 There are some benchmarks linked from there which look impressive, but (obviously?) there are some question marks over the benchmarks; http://www.extremetech.com/article2/0,3973,1136018,00.asp has a discussion. Simon _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From hahn at physics.mcmaster.ca Wed Jun 25 20:20:00 2003 From: hahn at physics.mcmaster.ca (Mark Hahn) Date: Wed, 25 Jun 2003 20:20:00 -0400 (EDT) Subject: Off topic - G5 question In-Reply-To: <3EFA19D1.7010009@yakko.cimav.edu.mx> Message-ID: > Hi you all guys, I've just checked the G5 processor out, what do you > think about it, it seems like it's not good enough considering it's > architecture and it's performance. > > http://www.apple.com/lae/g5 it's a nice step forward for Apple - the first time they've actually taken dram seriously. but Apple drinks a little too much of their own koolaid, or at least gives their marketing weasels a little too much leash. if you haven't looked already, Apple paid one of those analyst companies (Veritest, nice name) to prove that the G5 spanks those unwashed, beige-boxed, Intel-using heathens. to accomplish this, they tuned the G5 quite nicely, and relentlessly de-tuned the PC - stopping just short of installing WinME on it ;) if you want to make your competition look bad, run their benchmarks for them. at a glance, it might seem sensible to use gcc 3.3 for both machines; as Veritest says: To be able to directly compare the performance of the hardware systems, the same compiler - GCC, with similar settings were used on both platforms. OK, well, first of all, even if you build gcc 3.3 from scratch on each system, it's not going to factor out compiler effects. gcc does slightly different things on each platform. next, they didn't actually use the same compiler - they used Apple's gcc 3.3 binary, and for the PC, one build from virgin sources. all of this begs the question of why they wouldn't use Intel's compilers, which for some old reason, do really well on SPEC sources... it gets worse. things like tuning the dual-G5 to avoid snoop traffic when running (uniprocessor) CPU2000 tests, but turning *on* HT on the (uniprocessor) P4, and running an SMP kernel. again, there's a nugget of plausibility there, since they presumably couldn't get a uniprocessor version of Mac OS X. sure enough: they turn HT off and install a uni kernel to test SPEC rate on the uniprocessor and disable HT on the SMP system. quite different compiler switches for the two platforms, as well. don't forget to install a special single-threaded malloc, but only for the uniprocessor G5 tests... they appear to be trying to make a same-to-same comparison, which is both difficult and strange. users want a best-to-best comparison. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rdn at uchicago.edu Thu Jun 26 02:45:18 2003 From: rdn at uchicago.edu (Russell Nordquist) Date: Thu, 26 Jun 2003 01:45:18 -0500 (CDT) Subject: is this a good candidate for bproc? Message-ID: I've been lurking on this list for a while and I blame this question on that. I keep seeing new stuff I want to play with. Yesterday it was the iButton and today it's bproc. Here is the situation....I have a four node "cluster" that will be running independent computational jobs. No mpi, etc just lots 'o non-parallel jobs. My initial inclination was to setup a traditional (ie ssh to nodes and start jobs) cluster with SGE (and hide the ssh to nodes part). That way I can limit one job per CPU and be done with it. But, then I got to thinking about single system images. One issue I have had with queue systems is jobs lingering after their queue slot is gone (admittedly this has only happened with mpich programs, but our clusters haven't done much else until now). Users don't know to check process running on the nodes (and shouldn't need to) so I get e-mails about slow jobs and find the non-scheduled run aways. bproc's ability to allow users to manage processes from the master node would take care of this (hey, it's like a twin tower origin 200 now :). and I won't have to deal with libraries, etc on the nodes. However, I've never admined bproc or even been on a bproc system. Does this sound like a good place for bproc? I would use bproc+SGE to keep the 1 job 1 cpu efficiency. Is it realistic to expect that any job that can be run using "ssh node1 -e ./job.exe" would run with "bpsh 1 ./job.exe", or are there code related gotchas (threads?)? I guess I could set up both and see how it each works, but I wanted to get a feel for how useful people found bproc and it's limitations. Although I really want to experiment with bproc, I need to keep the day to day admin time on this system low. thanks russell - - - - - - - - - - - - Russell Nordquist UNIX Systems Administrator Geophysical Sciences Computing http://geosci.uchicago.edu/computing NSIT, University of Chicago - - - - - - - - - - - _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From hanzl at noel.feld.cvut.cz Thu Jun 26 06:33:27 2003 From: hanzl at noel.feld.cvut.cz (hanzl at noel.feld.cvut.cz) Date: Thu, 26 Jun 2003 12:33:27 +0200 Subject: is this a good candidate for bproc? In-Reply-To: References: Message-ID: <20030626123327N.hanzl@unknown-domain> > independent computational jobs. No mpi, etc just lots 'o non-parallel jobs. > ... > I would use bproc+SGE to keep the 1 job 1 cpu efficiency. We have something like this, and I described it here: http://noel.feld.cvut.cz/magi/sge+bproc.html (It is a quick first draft yet uncorrected by any feedback, any opinions/additions/corrections from any list readers welcome). > Is it realistic to expect that any job that can be run using "ssh > node1 -e ./job.exe" would run with "bpsh 1 ./job.exe", or are there > code related gotchas (threads?)? Most compiled executables computing something are just fine. Running complex scripts or administrative programs and daemons on nodes often can be solved by using more NFS mounts than is usual in bproc world to get access to all the executables, libraries and config files. The only nasty problem I am aware of may be the absence of support for control terminal stuff on slave nodes. This stuff is one of the few little known dark corners of UNIX - it is related to kernel calls like setsid(2) and to delivery of signals to groups of processes. In userland this omission translates to non-working Ctrl-C and spurious error messages from some interactive programs. Best Regards Vaclav Hanzl _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rgb at phy.duke.edu Thu Jun 26 10:04:00 2003 From: rgb at phy.duke.edu (Robert G. Brown) Date: Thu, 26 Jun 2003 10:04:00 -0400 (EDT) Subject: Off topic - G5 question In-Reply-To: <3EFA19D1.7010009@yakko.cimav.edu.mx> Message-ID: On Wed, 25 Jun 2003, Luis Fernando Licon Padilla wrote: > Hi you all guys, I've just checked the G5 processor out, what do you > think about it, it seems like it's not good enough considering it's > architecture and it's performance. > > http://www.apple.com/lae/g5 Looks great (if you believe their white papers) if you're planning to run BLAST. Doesn't look horribly shabby at floating point performance overall, has a nice peppy FSB. I certainly wouldn't kick it out of my system room for chewing crackers. So to speak. Although I believe that it requires some fairly careful programming to get maximum speed out of the CPU and would be very curious about real-world benchmarks for code that is just slapped on it and compiled, no tuning or hand vectorization. However, the traditional litany requires that one examine price as well as raw performance, and comparing it to P4 (really P6) class CPUs is apples to oranges. A more interesting and fair comparison would be to opterons and itaniums, and including price. rgb > > > Cheerz, > > > Luis > > -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb at phy.duke.edu _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From seth at hogg.org Thu Jun 26 11:42:39 2003 From: seth at hogg.org (Simon Hogg) Date: Thu, 26 Jun 2003 16:42:39 +0100 Subject: Off topic - G5 question In-Reply-To: References: <3EFA19D1.7010009@yakko.cimav.edu.mx> Message-ID: <4.3.2.7.2.20030626164046.00aedb70@pop.freeuk.net> At 10:04 26/06/03 -0400, Robert G. Brown wrote: >On Wed, 25 Jun 2003, Luis Fernando Licon Padilla wrote: >> http://www.apple.com/lae/g5 >Looks great (if you believe their white papers) if you're planning to >run BLAST. [...] Although I believe that >it requires some fairly careful programming to get maximum speed out of >the CPU and would be very curious about real-world benchmarks for code >that is just slapped on it and compiled, no tuning or hand >vectorization. Aye, there's the rub ... If you believe their white papers ... Comparing A/G BLAST on the G5 with 'vanilla' BLAST on the Xeons. "Apple compared the performance of the dual 2GHz Power Mac G5 running A/G BLAST with a 3GHz Pentium 4?based system and a dual 3.06GHz Xeon-based system, both running Red Hat Linux 9.0 and NCBI BLAST." And what is A/G BLAST? "A/G BLAST is an optimized version of NCBI BLAST developed by Apple in collaboration with Genentech. Optimized for dual PowerPC G5 processors, the Velocity Engine, and the symmetric multiprocessing capabilities of Mac OS X[...]" So once again, it would be nice to see some 'real-world' comparisons of like-with-like. >However, the traditional litany requires that one examine price as well >as raw performance, and comparing it to P4 (really P6) class CPUs is >apples to oranges. A more interesting and fair comparison would be to >opterons and itaniums, and including price. Habsolutely agree on that! -- Simon _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From amitoj at cs.uh.edu Thu Jun 26 11:56:36 2003 From: amitoj at cs.uh.edu (Amitoj G. Singh) Date: Thu, 26 Jun 2003 10:56:36 -0500 (CDT) Subject: is this a good candidate for bproc? In-Reply-To: Message-ID: Worth a try. Look under /etc/security. There's some stuff that will allow you to put restrictions such as the 1 job 1 cpu limit you want. - Amitoj. On Thu, 26 Jun 2003, Russell Nordquist wrote: > I've been lurking on this list for a while and I blame this question on > that. I keep seeing new stuff I want to play with. Yesterday it was the > iButton and today it's bproc. > > Here is the situation....I have a four node "cluster" that will be running > independent computational jobs. No mpi, etc just lots 'o non-parallel jobs. > My initial inclination was to setup a traditional (ie ssh to nodes and > start jobs) cluster with SGE (and hide the ssh to nodes part). That way I > can limit one job per CPU and be done with it. > > But, then I got to thinking about single system images. One issue I have > had with queue systems is jobs lingering after their queue slot is gone > (admittedly this has only happened with mpich programs, but our clusters > haven't done much else until now). Users don't know to check process > running on the nodes (and shouldn't need to) so I get e-mails about slow > jobs and find the non-scheduled run aways. bproc's ability to allow users > to manage processes from the master node would take care of this (hey, > it's like a twin tower origin 200 now :). and I won't have to deal with > libraries, etc on the nodes. > > However, I've never admined bproc or even been on a bproc system. > Does this sound like a good place for bproc? I would use bproc+SGE to keep > the 1 job 1 cpu efficiency. Is it realistic to expect that any job that > can be run using "ssh node1 -e ./job.exe" would run with "bpsh 1 > ./job.exe", or are there code related gotchas (threads?)? > > I guess I could set up both and see how it each works, but I wanted to get > a feel for how useful people found bproc and it's limitations. Although I > really want to experiment with bproc, I need to keep the day to day admin > time on this system low. > > thanks > russell > > - - - - - - - - - - - - > Russell Nordquist > UNIX Systems Administrator > Geophysical Sciences Computing > http://geosci.uchicago.edu/computing > NSIT, University of Chicago > - - - - - - - - - - - > > > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org > To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf > _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rdn at uchicago.edu Thu Jun 26 13:19:46 2003 From: rdn at uchicago.edu (Russell Nordquist) Date: Thu, 26 Jun 2003 12:19:46 -0500 (CDT) Subject: is this a good candidate for bproc? In-Reply-To: <20030626123327N.hanzl@unknown-domain> Message-ID: On Thu, 26 Jun 2003 at 12:33, hanzl at noel.feld.cvut.cz wrote: > > independent computational jobs. No mpi, etc just lots 'o non-parallel jobs. > > ... > > I would use bproc+SGE to keep the 1 job 1 cpu efficiency. > > We have something like this, and I described it here: > > http://noel.feld.cvut.cz/magi/sge+bproc.html > > (It is a quick first draft yet uncorrected by any feedback, any > opinions/additions/corrections from any list readers welcome). looks good to me, thanks. I actually ran across this link when I was researching. Anyone got any opinions on the bproc vs the ssh/nfs approach? I've never used bproc, so I don't have a fell for it? russell > Best Regards > > Vaclav Hanzl > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org > To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf > - - - - - - - - - - - - Russell Nordquist UNIX Systems Administrator Geophysical Sciences Computing http://geosci.uchicago.edu/computing NSIT, University of Chicago - - - - - - - - - - - _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rgb at phy.duke.edu Thu Jun 26 13:55:25 2003 From: rgb at phy.duke.edu (Robert G. Brown) Date: Thu, 26 Jun 2003 13:55:25 -0400 (EDT) Subject: Off topic - G5 question In-Reply-To: <4.3.2.7.2.20030626164046.00aedb70@pop.freeuk.net> Message-ID: On Thu, 26 Jun 2003, Simon Hogg wrote: > And what is A/G BLAST? "A/G BLAST is an optimized version of NCBI BLAST > developed by Apple in collaboration with Genentech. Optimized for dual > PowerPC G5 processors, the Velocity Engine, and the symmetric > multiprocessing capabilities of Mac OS X[...]" Awwww, you-all is jes' so cynical...;-) Actually, I attended at least part of an HPC talk a week+ ago given by an Apple guy -- about half devoted to BLAST and genetic stuff and half devoted to marketing (as far as I could tell:-). From what I could dig out of him (and this is all from memory so don't shoot me if I'm wrong) the G5 has an embedded vector unit in it (the "Velocity Engine" in the marketspeak above?) that has a modestly spectacular peak throughput for instruction streams that match -- I want to say 12 GFLOPs or something like that. The CPU itself has a fairly hefty L3 cache outside of L2 and L1 and some cleverness designed to try to keep the gaping maw the vector unit represents filled. It actually looked like a thoughtful architecture at least at the block device/bus level. However, as is usually the case with vector sub- or co-processors, the real problem is getting code to use it at all, let alone effectively, as the compiler usually groks "CPU" and doesn't know offhand when or how to send code to the vector processor for a speedup or when to leave it alone and execute on the CPU itself. I would guess that code has to minimally be vectorized and instrumented to use the VP with pragma's or the like, or worse, require actual cross-compiled objects to inline in the code, but I really don't know or care enough to fine out. So I think that is what they are referring to as an "optimized version" -- one that somebody has taken the time to instrument for the VP and tune for their cache sizes and so forth. Apple was clearly very interested in targeting the genomics clustering people -- they may (correctly) see them as deep pockets backing an insatiable appetite for cluster power over the next decade or so as projects everywhere build up the lexicon of genetic expression of every living creature on the planet (let alone the "human genome") and be gunning for an inside edge with an architecture and application set deliberately tuned for good performance in this one market. Of course, this makes a LOT of things very difficult to properly benchmark. At a guess, if one hand-tuned almost any sort of vector-intensive floating point code for the VP architecture, one might see a fairly spectacular speedup, but note that the unit's integer performance (indicative of how well the CPU does everyday work) was distinctly unimpressive so a NON-vectorizable application might really suck on it. Also one needs to use SSE2 instrumented compilers and instructions to get fair comparisons from alternative architectures, for the same reasons -- if one compiles a patch of code that SSE2 would (possibly spectacularly) speed up with a non-SSE2 compiler, well, performance might well suck. Mark's observation about best-to-best comparisons are likely one approach, especially if you plan to examine vendor benchmarks. Let Apple tune the hell out of BLAST, but only compare results to an Intel-tuned BLAST on >>its<< favorite CPU/system du jour, etc. Alternatively, just compiling the same source with gcc with the same flags on both systems works for me, at least, as this is likely to be the extent of the effort I want to make tuning per architecture, ever. Maintaining architecture-specific optimizations in a changing world of hardware technology is very painful and expensive. > So once again, it would be nice to see some 'real-world' comparisons of > like-with-like. Amen, although the problem that has stumped philosophers over the ages is: just what IS the real world, anyway? Pardon me, I have to put a cat into a superposition state with a diabolical device...:-) rgb -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb at phy.duke.edu _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From henken at seas.upenn.edu Thu Jun 26 14:22:44 2003 From: henken at seas.upenn.edu (Nicholas Henke) Date: 26 Jun 2003 14:22:44 -0400 Subject: is this a good candidate for bproc? In-Reply-To: References: Message-ID: <1056651763.24158.17.camel@roughneck.liniac.upenn.edu> On Thu, 2003-06-26 at 02:45, Russell Nordquist wrote: > > However, I've never admined bproc or even been on a bproc system. > Does this sound like a good place for bproc? I would use bproc+SGE to keep > the 1 job 1 cpu efficiency. Is it realistic to expect that any job that > can be run using "ssh node1 -e ./job.exe" would run with "bpsh 1 > ./job.exe", or are there code related gotchas (threads?)? Java is really unhappy with bproc. As far as I can tell bproc does not support use of the clone system call yet, and java uses that to spawn its threads. Nic -- Nicholas Henke Penguin Herder & Linux Cluster System Programmer Liniac Project - Univ. of Pennsylvania _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From walkev at presearch.com Thu Jun 26 15:34:52 2003 From: walkev at presearch.com (Vann H. Walke) Date: 26 Jun 2003 15:34:52 -0400 Subject: Off topic - G5 question In-Reply-To: References: Message-ID: <1056656072.24962.47.camel@localhost.localdomain> > Alternatively, just compiling the same source with gcc with the same > flags on both systems works for me, at least, as this is likely to be > the extent of the effort I want to make tuning per architecture, ever. > Maintaining architecture-specific optimizations in a changing world of > hardware technology is very painful and expensive. I pretty much agree with all your points, but don't completely ignore optimizations. For some of our signal processing code, we get a ~2.5x speedup by using SSE primitives. It is isn't hard to justify a couple days of weeding through code to reduce your hardware requirements by half. Plus, at least for the SIMD type instructions porting to other SIMD architectures isn't excruciatingly painful. We were able to tune our code for both the Altivec and SSE instructions, thus giving us "real" performance comparisons. Vann -- Vann H. Walke Presearch, Inc. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From hahn at physics.mcmaster.ca Thu Jun 26 17:39:09 2003 From: hahn at physics.mcmaster.ca (Mark Hahn) Date: Thu, 26 Jun 2003 17:39:09 -0400 (EDT) Subject: Off topic - G5 question In-Reply-To: Message-ID: > unit represents filled. It actually looked like a thoughtful > architecture at least at the block device/bus level. it's dandy. the problem is Apple's marketing nonsense. SIMD is great, too. cache is great. but please don't claim that your in-cache SIMD FLOPS is a general/true FLOPS rate for the machine, since it's NOT, for anything other than Mandelbrot surfing. incidentally, I don't see *any* off-chip cache in the G5's. > Of course, this makes a LOT of things very difficult to properly > benchmark. I don't agree at all. I want to see precisely two sets of numbers when evaluating new hardware performance: - fairly clean microbenchmarks like Stream, streaming sequential file IO, zero-byte packet latency, etc. - best-case performance on real codes - for the compute cluster world, that means SpecFP and SpecFPrate. other fields may want tuned-BLAS or 45-filter photoshop. well, and price, but that's normally boringly predictable (I say "way too expensive!" a lot ;) > suck on it. Also one needs to use SSE2 instrumented compilers and > instructions to get fair comparisons from alternative architectures, for I find that many of my users spend most of their serious time inside tuned library functions, so the compiler is less relevant. though Intel's compilers *do* auto-vectorize onto SSE2. gcc can use SSE2, but doesn't currently autovectorize. incidentally, gcc 3.3 *does* also support intrinsics for Altivec vector operations; I'm not sure whether similar intrinsics are available for SSE2. > flags on both systems works for me, at least, as this is likely to be > the extent of the effort I want to make tuning per architecture, ever. I find this irresponsible (sorry rgb!). if you're going to consume more than a few CPU hours, and those cycles are coming from a shared resource, failing to optimize is just plain hostile. I still see users fling their code at a machine with g77 prog.f && ./a.out and wonder why it's so slow. usually, I stifle the obvious answer, and (hey, this is Canada) apologize for not providing the information they need to make supra-moronic use of the machine... regards, mark 'surly' hahn. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rgb at phy.duke.edu Thu Jun 26 18:12:22 2003 From: rgb at phy.duke.edu (Robert G. Brown) Date: Thu, 26 Jun 2003 18:12:22 -0400 (EDT) Subject: Off topic - G5 question In-Reply-To: Message-ID: On Thu, 26 Jun 2003, Mark Hahn wrote: > I find that many of my users spend most of their serious time inside > tuned library functions, so the compiler is less relevant. though Intel's > compilers *do* auto-vectorize onto SSE2. gcc can use SSE2, but doesn't > currently autovectorize. incidentally, gcc 3.3 *does* also support > intrinsics for Altivec vector operations; I'm not sure whether similar > intrinsics are available for SSE2. > > > flags on both systems works for me, at least, as this is likely to be > > the extent of the effort I want to make tuning per architecture, ever. > > I find this irresponsible (sorry rgb!). if you're going to consume > more than a few CPU hours, and those cycles are coming from a shared > resource, failing to optimize is just plain hostile. Oh, that's ok. I freely admit to utter irresponsibility. Laziness too! I was (of course) at least partly tongue in cheek, although the nugget of truth in what I was saying is what I'd call the "CM5 phenomenon". CM5's were a nifty architecture too -- basically (IIRC) a network of sparcstations each with an attached vector unit. Programming it was nightmarish -- it went beyond instrumenting code, one basically had to cross-compile code modules for the vector units and use the Sparcs as computers to run these modules and handle all the nitty gritty for them (like access to I/O and other resources). Then the CM5 bellied up (Duke sold its old one for a few thousand $$ to somebody interested in salvaging the gold in its gazillion contacts). All that effort totally wasted. We're talking years of a human life, not just a few weeks porting a simple application, and he was just one of Duke's many CM5 users...:-) Or if you prefer, the MPI phenomenon. Back when the CM5 had to be hand coded, so did nearly everything else. Code maintenance was nightmarish, as ever parallel supercomputer manufacturer had their own message passing API and so forth. So the government (major supercomputer purchaser that it is) finally said that enough was enough and that they were only going to use one API in the future and if machines didn't support it they weren't going to be bought, and lo, MPI was born so people didn't have to rewrite their code. Or related examples abound in all realms of computedom. The more you customize or optimize per architecture, the less portable your code becomes and the more expensive it becomes to change architectures until you find yourself in a golden trap, still buying IBM mainframes because you have all this custom stuff that your business relies on that (in its day) was heavily customized to take advantage of all of the nifty features of IBM mainframes. Curiously, a cluster of G5's with their embedded Altivecs, or a cluster of playstations with their embedded vector processor, bear a strong resemblance to the CM5 architecturally and (in the case of the PS2's) in how one has to program them with what amounts to a cross-compiler with instrumented calls to send code and data on to the vector unit for handling. This kind of architecture makes me very nervous. Optimizing is no longer just a matter of picking good compiler flags or writing core loops sensibly (things that can easily be changed or that are good ideas on nearly any architecture); it starts to become an issue of writing code "just for the Altivec" or "just for the PS2" -- hacking your application (a process that can take a LONG time, not just hours or days) irreversibly so that you have to spend hours or days again to change it back to run on a more normal architecture. It is this lack of portability that can, and I say can, be a killer to productivity and ultimately to one's ability to select freely from COTS architectures. Once the code is modified and optimized for (say) a G5, one has to take into account the ADDITIONAL cost of un/remodifying it and reoptimizing it for (say) an Opteron or Itanium or whateverisnextium. This makes Apple very happy, in precisely the way that Microsoft is made happy every time a businessman says "we have to run Windows because everybody is used to it and it would cost us too much time/money/stress to change" even when it is visibly less cost-effective head to head with alternatives. > I still see users fling their code at a machine with > g77 prog.f && ./a.out > and wonder why it's so slow. usually, I stifle the obvious answer, > and (hey, this is Canada) apologize for not providing the information > they need to make supra-moronic use of the machine... Right. They need to be running gcc -O3 -o prog prog.c prog instead. Obviously;-) (yoke, yoke, I was mayking a yoke to help you forget how screwed you are...:-) Still, you're right of course, especially where things like SSE2 or Altivec aware compilers can do a lot of the optimizing for you if your code isn't too complex, or the issue is using ATLAS-tuned blas instead of a really shitty off the shelf blas. Still, there is something to be said for writing just plain sound and efficient code (in C:-) and using mundane -O-level optimization as a basis for how good a machine is, especially if you don't plan to study the microarchitecture layout and figure out if you can get four flops in the pipeline per clock, but only if you rearrange your code in a completely nonintuitive way so it does a multiply/add in a certain way... > regards, mark 'surly' hahn. (and to cheer you up..:-) rg-let-me-show-you-my-hand-tuned-8087-library-some-day-b -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb at phy.duke.edu _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rrstump at sandia.gov Thu Jun 26 21:04:23 2003 From: rrstump at sandia.gov (Roland Stumpf) Date: Thu, 26 Jun 2003 18:04:23 -0700 Subject: Intel PRO/1000CT Gigabit ethernet with CSA Message-ID: <3EFB9817.6090001@sandia.gov> Hi everybody ! We are configuring a ~40 node cluster for parallel materials modeling applications (e.g. VASP code from Vienna). My guess is that the sweet spot in terms of communication speed versus cost is with gigabit ethernet. Does anybody have an opinion on the new Intel PRO/1000CT Gigabit NIC that travels through the CSA (Communications Streaming Architecture) bus ? The Intel fact sheet is, as often, short on quantitative detail but it looks promising: http://www.intel.com/design/network/products/lan/controllers/82547ei.htm How does this NIC and a switch compare to Myrinet and similar networks ? We would be especially interested in an improvement of the latency for MPI communication over other Gigabit or 100Mbit ethernet. Another concern is if the faster GigE cards could saturate a fully connected modern 24 port switch like the Dell PowerConnect 5224, leading to dropped packages and crashing jobs. Thanks, Roland _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From daniel.pfenniger at obs.unige.ch Fri Jun 27 02:54:33 2003 From: daniel.pfenniger at obs.unige.ch (Daniel Pfenniger) Date: Fri, 27 Jun 2003 08:54:33 +0200 Subject: Intel PRO/1000CT Gigabit ethernet with CSA In-Reply-To: <3EFB9817.6090001@sandia.gov> References: <3EFB9817.6090001@sandia.gov> Message-ID: <3EFBEA29.60602@obs.unige.ch> Hi, For a small experimental cluster (24 dual Xeon nodes) we decided to use InfiniBand technology, which from specs is 4 times faster (8Gb/s), 1.5 lower latency (~5musec) than Myrinet for approximately the same cost/port. Our estimate is that with current 3GHz class processors one needs larger bandwith and lower latency than 2xGigaEthernet in order to make efficient use of the rest of the hardware, this for typical MPI type runs. Of course there exist always sufficiently coarse grained parallel applications for which GigaEthernet would be the good choice. At the moment there are very few >32 port fat-tree switches for InfiniBand, but 96-128 port swicthes should be available in the next months. Dan Roland Stumpf wrote: > Hi everybody ! > > We are configuring a ~40 node cluster for parallel materials modeling > applications (e.g. VASP code from Vienna). My guess is that the sweet > spot in terms of communication speed versus cost is with gigabit > ethernet. Does anybody have an opinion on the new Intel PRO/1000CT > Gigabit NIC that travels through the CSA (Communications > Streaming Architecture) bus ? The Intel fact sheet is, as often, short > on quantitative detail but it looks promising: > > http://www.intel.com/design/network/products/lan/controllers/82547ei.htm > > How does this NIC and a switch compare to Myrinet and similar networks ? > We would be especially interested in an improvement of the latency for > MPI communication over other Gigabit or 100Mbit ethernet. Another > concern is if the faster GigE cards could saturate a fully connected > modern 24 port switch like the Dell PowerConnect 5224, leading to > dropped packages and crashing jobs. > > Thanks, > Roland _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From graham.mullier at syngenta.com Fri Jun 27 05:28:55 2003 From: graham.mullier at syngenta.com (graham.mullier at syngenta.com) Date: Fri, 27 Jun 2003 10:28:55 +0100 Subject: Off topic - G5 question Message-ID: <0B27450D68F1D511993E0001FA7ED2B3025E50F9@ukjhmbx12.ukjh.zeneca.com> I think Rob made an excellent point. I'd like to paraphrase: optimising for a very unusual architecture, with poor support from the compilation (etc) tools, should not be undertaken lightly. I've seen examples in my time of very promising (=hyped) add-on hardware units for Vaxes, IBM mainframes, plus complex specialised graphics systems (E&S PS390) that either didn't live up to the promised performance, or where the effort involved to make use of the specialised hardware made it difficult to justify the work. Vax - the extra units seemed to offer performance similar to the strange new (to me at the time) Unix machines. So we bought an SGI server and once we'd got over the shock found it very useful because it was so flexible, with good compilation tools. IBM - we never saw the supposed performance improvement, and the only way to get calculations (large ab initio molecular orbital calculations) onto the system was to hand-deliver a tape (1/4-inch reel...). We suggested that the vector unit wasn't worth it for our work. E&S PS390 - goodwhen someone else wrote the graphics code, nightmare when it was someone in our group. He produced an excellent general visualisation system for 3D scatter plots but when we stopped using the hardware we had to throw away the code. Not a good use of resources. I prefer good general-purpose systems where other people help me by working on the tools necessary to make best use of the hardware. Graham Graham Mullier Chemoinformatics Team Leader, Chemistry Design Group, Syngenta, Bracknell, RG42 6EY, UK. direct line: +44 (0) 1344 414163 mailto:Graham.Mullier at syngenta.com -----Original Message----- From: Robert G. Brown [mailto:rgb at phy.duke.edu] Sent: 26 June 2003 23:12 To: Mark Hahn Cc: Beowulf Mailing List Subject: Re: Off topic - G5 question [...] Oh, that's ok. I freely admit to utter irresponsibility. Laziness too! I was (of course) at least partly tongue in cheek, although the nugget of truth in what I was saying is what I'd call the "CM5 phenomenon". [...] This kind of architecture makes me very nervous. Optimizing is no longer just a matter of picking good compiler flags or writing core loops sensibly (things that can easily be changed or that are good ideas on nearly any architecture); [...] It is this lack of portability that can, and I say can, be a killer to productivity and ultimately to one's ability to select freely from COTS architectures. Once the code is modified and optimized for (say) a G5, one has to take into account the ADDITIONAL cost of un/remodifying it and reoptimizing it for (say) an Opteron or Itanium or whateverisnextium. [...] _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From wrankin at ee.duke.edu Fri Jun 27 07:57:15 2003 From: wrankin at ee.duke.edu (Bill Rankin) Date: 27 Jun 2003 07:57:15 -0400 Subject: Job Posting - cluster admin. Message-ID: <1056715034.2172.21.camel@rohgun.cse.duke.edu> FYI - we are seeking a Beowulf admin for our university cluster. If you know of anyone that is interested, please forward them this information. The web page for Duke HR: http://www.hr.duke.edu/jobs/main.html Go to "Job Postings" and enter the folllowing requisition number. CAM45496-061203 Or just search for "Beowulf" in the job descriptions. Thanks, -bill -- dr. bill rankin .................................... research scientist wrankin at ee.duke.edu .......................... center for computational duke university ...................... science engineering and medicine http://www.ee.duke.edu/~wrankin ................... http://cse.duke.edu _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From mikee at mikee.ath.cx Fri Jun 27 10:13:00 2003 From: mikee at mikee.ath.cx (Mike Eggleston) Date: Fri, 27 Jun 2003 09:13:00 -0500 Subject: Job Posting - cluster admin. In-Reply-To: <1056715034.2172.21.camel@rohgun.cse.duke.edu>; from wrankin@ee.duke.edu on Fri, Jun 27, 2003 at 07:57:15AM -0400 References: <1056715034.2172.21.camel@rohgun.cse.duke.edu> Message-ID: <20030627091300.A18502@mikee.ath.cx> On Fri, 27 Jun 2003, Bill Rankin wrote: > FYI - we are seeking a Beowulf admin for our university cluster. If you > know of anyone that is interested, please forward them this information. > > The web page for Duke HR: > > http://www.hr.duke.edu/jobs/main.html > > Go to "Job Postings" and enter the folllowing requisition number. > > CAM45496-061203 > > Or just search for "Beowulf" in the job descriptions. > > Thanks, > > -bill > > -- > dr. bill rankin .................................... research scientist > wrankin at ee.duke.edu .......................... center for computational > duke university ...................... science engineering and medicine > http://www.ee.duke.edu/~wrankin ................... http://cse.duke.edu I'd love to. Can I do it from Dallas? Mike _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From becker at scyld.com Thu Jun 26 03:08:36 2003 From: becker at scyld.com (Donald Becker) Date: Thu, 26 Jun 2003 00:08:36 -0700 (PDT) Subject: 3Com 905C and ASUS P4GX8 In-Reply-To: <200306251346.h5PDkcF19321@accms33.physik.rwth-aachen.de> Message-ID: On Wed, 25 Jun 2003, Christoph Kukulies wrote: > I installed Linux RH 7.3 on a Dual 3GHz P4 CPU Asus P4GX8 MB > and since the on board 100 MBit controller didn't seem to be > supported by this Linux version (doubt whether any version does > in the moment) The Scyld distribution ships with our driver package, and almost certainly supports the card. But your problem is likely a configuration issue, not a driver failure. > I'm seeing this in /var/log/messages and the machine > is very slow on network copies and activity. What is the detection message? > eth0: Transmit error, Tx status register 82. Read http://www.scyld.com/network/vortex.html Specifically the section that says 0x82 Out of window collision. This typically occurs when some other Ethernet host is incorrectly set to full duplex on a half duplex network. -- Donald Becker becker at scyld.com Scyld Computing Corporation http://www.scyld.com 914 Bay Ridge Road, Suite 220 Scyld Beowulf cluster system Annapolis MD 21403 410-990-9993 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From daniel.pfenniger at obs.unige.ch Fri Jun 27 13:46:28 2003 From: daniel.pfenniger at obs.unige.ch (Daniel Pfenniger) Date: Fri, 27 Jun 2003 19:46:28 +0200 Subject: Intel PRO/1000CT Gigabit ethernet with CSA In-Reply-To: <200306271701.VAA12659@nocserv.free.net> References: <200306271701.VAA12659@nocserv.free.net> Message-ID: <3EFC82F4.6040502@obs.unige.ch> Mikhail Kuzminsky wrote: > According to Daniel Pfenniger > >>For a small experimental cluster (24 dual Xeon nodes) >>we decided to use InfiniBand technology, which from specs is >>4 times faster (8Gb/s), 1.5 lower latency (~5musec) than >>Myrinet for approximately the same cost/port. > > Could you pls compare them a bit more detailed ? > Infiniband card costs (as I heard) about $1000-, (HCA-Net from > FabricNetworks, former InfiniSwitch ?), what is close to Myrinet. > But what is about switches (I heard about high prices) ? > > In particular, I'm interesting in very small switches; > FabricNetworks produce 8-port 800-series switch, but I don't > know about prices. May be there is 6 or 4 port switches ? I don't know about such small switches. FabricNetworks list prices are ~10K$ for a 12 port switch, ~15K$ for a 24 port switch. But these 2 switches are not fully fat. A fully fat tree 32 port switch list price is estimated by me around 30-40K$, but it is not official because very recent (such a switch has been shown at the Heidelberg HPC conference these days). The exact costs are presently not well fixed because several companies enter the market. The nice thing about IB is that it is an open standard, the components from different companies are compatible, which is good for pressing costs down. > BTW, is it possible to connect pair of nodes by means of > "cross-over" cable (as in Ethernet), i.e. w/o switch ? Yes. The same copper cable as for the switch should do (but I have not tested yet). Each HCA has *two* ports, so you should be able to build a fully connected no hop 3-node cluster without a switch. With two HCA per nodes you might raise this number to 5 and still have a reasonable cost/node with respect to the switched solution. The 3m cables cost something around 300$ though, not completely negligible. Daniel Pfenniger _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From patrick at myri.com Fri Jun 27 15:00:36 2003 From: patrick at myri.com (Patrick Geoffray) Date: 27 Jun 2003 15:00:36 -0400 Subject: Intel PRO/1000CT Gigabit ethernet with CSA In-Reply-To: <3EFC82F4.6040502@obs.unige.ch> References: <200306271701.VAA12659@nocserv.free.net> <3EFC82F4.6040502@obs.unige.ch> Message-ID: <1056740437.548.101.camel@asterix> On Fri, 2003-06-27 at 13:46, Daniel Pfenniger wrote: > The exact costs are presently not well fixed because several companies > enter the market. The nice thing about IB is that it is an open > standard, the components from different companies are compatible, > which is good for pressing costs down. With the slicon coming from one company (actually 2 but the second one does only switch chip), the price adjustment would mainly affect the reseller, where the margin are not that high. I don't expect much a price war in the Infiniband market, mainly because many IB shops are already just burning (limited) VC cash. The main point for price advantage of IB is if the volume goes up. It's a very different problem that the multiple-vendors-marketing-stuff. One can argue that HPC does not yield such high volumes, only a business market like the Databases one does. Remember Gigabit Ethernet. It was very expensive when the early adopters were the HPC crowd and the price didn't drop until it made its way to the desktop. It's the case for 10GE today. Patrick -- Patrick Geoffray Myricom, Inc. http://www.myri.com _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From daniel.pfenniger at obs.unige.ch Fri Jun 27 15:52:51 2003 From: daniel.pfenniger at obs.unige.ch (Daniel Pfenniger) Date: Fri, 27 Jun 2003 21:52:51 +0200 Subject: Intel PRO/1000CT Gigabit ethernet with CSA In-Reply-To: <1056740437.548.101.camel@asterix> References: <200306271701.VAA12659@nocserv.free.net> <3EFC82F4.6040502@obs.unige.ch> <1056740437.548.101.camel@asterix> Message-ID: <3EFCA093.4090006@obs.unige.ch> Patrick Geoffray wrote: > On Fri, 2003-06-27 at 13:46, Daniel Pfenniger wrote: >>The exact costs are presently not well fixed because several companies >>enter the market. The nice thing about IB is that it is an open >>standard, the components from different companies are compatible, >>which is good for pressing costs down. > > > With the slicon coming from one company (actually 2 but the second one > does only switch chip), the price adjustment would mainly affect the > reseller, where the margin are not that high. I don't expect much a > price war in the Infiniband market, mainly because many IB shops are > already just burning (limited) VC cash. > The main point for price advantage of IB is if the volume goes up. It's > a very different problem that the multiple-vendors-marketing-stuff. One > can argue that HPC does not yield such high volumes, only a business > market like the Databases one does. > > Remember Gigabit Ethernet. It was very expensive when the early adopters > were the HPC crowd and the price didn't drop until it made its way to > the desktop. It's the case for 10GE today. > > Patrick > > Patrick Geoffray > Myricom, Inc. Yes I mostly agree with your analysis, database is the only significant potential market for IB. However the problem with 1GBE or 10GBE is that the latency remains poor for HPC applications, while IB goes in the right direction. The real comparison to be made is not between GE and IB, but between IB and Myricom products, which belong to an especially protected niche. As a result for years the Myrinet products did hardly drop in price for a sub-Moore's-law increase in performance, because of a lack of competition (the price we paid for our Myricom cards and switch 18 months ago is today *exactly* the same). Now IB at least introduces a healthy stimulation for this low latency HPC sector. Daniel _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From keith.murphy at attglobal.net Fri Jun 27 17:36:37 2003 From: keith.murphy at attglobal.net (Keith Murphy) Date: Fri, 27 Jun 2003 14:36:37 -0700 Subject: Intel PRO/1000CT Gigabit ethernet with CSA References: <200306271701.VAA12659@nocserv.free.net> <3EFC82F4.6040502@obs.unige.ch> <1056740437.548.101.camel@asterix> <3EFCA093.4090006@obs.unige.ch> Message-ID: <01a401c33cf4$306ef250$02fea8c0@oemcomputer> There have been competitive priced low latency interconnect cards to Myricom for some years including products from Quadrix and Dolphin. The reason for the lack of movement on pricing, is the high performance cluster interconnect is still a very small market I am sure that even in the very best of years the total yearly quantity of low latency interconnect cards and switches sold, never exceeds the daily quantity of Ethernet cards and switches sold. Price reductions are not reflected in Moore's law, unless accompanied by a subsequent expotentitial leap in the size of the market. Keith Murphy Dolphin Interconnect T: 818-597-2114 F: 818-597-2119 C: 818-292-5100 www.dolphinics.com www.scali.com ----- Original Message ----- From: "Daniel Pfenniger" To: "Patrick Geoffray" Cc: "Beowulf cluster mailing list" Sent: Friday, June 27, 2003 12:52 PM Subject: Re: Intel PRO/1000CT Gigabit ethernet with CSA > Patrick Geoffray wrote: > > On Fri, 2003-06-27 at 13:46, Daniel Pfenniger wrote: > > >>The exact costs are presently not well fixed because several companies > >>enter the market. The nice thing about IB is that it is an open > >>standard, the components from different companies are compatible, > >>which is good for pressing costs down. > > > > > > With the slicon coming from one company (actually 2 but the second one > > does only switch chip), the price adjustment would mainly affect the > > reseller, where the margin are not that high. I don't expect much a > > price war in the Infiniband market, mainly because many IB shops are > > already just burning (limited) VC cash. > > The main point for price advantage of IB is if the volume goes up. It's > > a very different problem that the multiple-vendors-marketing-stuff. One > > can argue that HPC does not yield such high volumes, only a business > > market like the Databases one does. > > > > Remember Gigabit Ethernet. It was very expensive when the early adopters > > were the HPC crowd and the price didn't drop until it made its way to > > the desktop. It's the case for 10GE today. > > > > Patrick > > > > Patrick Geoffray > > Myricom, Inc. > > Yes I mostly agree with your analysis, database is the only significant > potential market for IB. > > However the problem with 1GBE or 10GBE is that the latency remains poor > for HPC applications, while IB goes in the right direction. > The real comparison to be made is not between GE and IB, but between > IB and Myricom products, which belong to an especially protected niche. > As a result for years the Myrinet products did hardly drop in price > for a sub-Moore's-law increase in performance, because of a lack of > competition (the price we paid for our Myricom cards and switch > 18 months ago is today *exactly* the same). > Now IB at least introduces a healthy stimulation for this low latency > HPC sector. > > Daniel > > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org > To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf > _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rrstump at sandia.gov Thu Jun 26 12:36:07 2003 From: rrstump at sandia.gov (Roland Stumpf) Date: Thu, 26 Jun 2003 09:36:07 -0700 Subject: Intel PRO/1000CT Gigabit ethernet with CSA Message-ID: <3EFB20F7.20007@sandia.gov> Hi everybody ! We are configuring a ~40 node cluster for parallel materials modeling applications (e.g. VASP code from Vienna). My guess is that the sweet spot in terms of communication speed versus cost is with gigabit ethernet. Does anybody have an opinion on the new Intel PRO/1000CT Gigabit LAN controller that travels through the CSA (Communications Streaming Architecture) bus ? The Intel fact sheet is, as often, short on quantitative detail but it looks promising: http://www.intel.com/design/network/products/lan/controllers/82547ei.htm We would be especially interested in an improvement of the latency for MPI communication over other Gigabit or 100Mbit ethernet. Another concern is if the faster GigE cards could saturate a fully connected modern 24 port switch like the Dell PowerConnect 5224, leading to dropped packages and crashing jobs. Thanks, Roland _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From toon at moene.indiv.nluug.nl Fri Jun 27 14:50:14 2003 From: toon at moene.indiv.nluug.nl (Toon Moene) Date: Fri, 27 Jun 2003 20:50:14 +0200 Subject: [Fwd: Re: Off topic - G5 question] Message-ID: <3EFC91E6.2080701@moene.indiv.nluug.nl> Oops, forgot to "reply all". -- Toon Moene - mailto:toon at moene.indiv.nluug.nl - phoneto: +31 346 214290 Saturnushof 14, 3738 XG Maartensdijk, The Netherlands Maintainer, GNU Fortran 77: http://gcc.gnu.org/onlinedocs/g77_news.html GNU Fortran 95: http://gcc-g95.sourceforge.net/ (under construction) -------------- next part -------------- An embedded message was scrubbed... From: unknown sender Subject: no subject Date: no date Size: 1227 URL: From mof at labf.org Thu Jun 26 12:13:34 2003 From: mof at labf.org (Mof) Date: Fri, 27 Jun 2003 01:43:34 +0930 Subject: Off topic - G5 question In-Reply-To: References: Message-ID: <200306270143.35048.mof@labf.org> On Thu, 26 Jun 2003 11:34 pm, Robert G. Brown wrote: > On Wed, 25 Jun 2003, Luis Fernando Licon Padilla wrote: > > Hi you all guys, I've just checked the G5 processor out, what do you > > think about it, it seems like it's not good enough considering it's > > architecture and it's performance. > > > > http://www.apple.com/lae/g5 > > Looks great (if you believe their white papers) if you're planning to > run BLAST. Doesn't look horribly shabby at floating point performance > overall, has a nice peppy FSB. I certainly wouldn't kick it out of my > system room for chewing crackers. So to speak. Although I believe that > it requires some fairly careful programming to get maximum speed out of > the CPU and would be very curious about real-world benchmarks for code > that is just slapped on it and compiled, no tuning or hand > vectorization. > > However, the traditional litany requires that one examine price as well > as raw performance, and comparing it to P4 (really P6) class CPUs is > apples to oranges. A more interesting and fair comparison would be to > opterons and itaniums, and including price. An article talking about some of the problems with the benchmarks : http://www.extremetech.com/article2/0,3973,1136018,00.asp Mof. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From widyono at cis.upenn.edu Wed Jun 25 21:11:58 2003 From: widyono at cis.upenn.edu (Daniel Widyono) Date: Wed, 25 Jun 2003 21:11:58 -0400 Subject: 3Com 905C and ASUS P4GX8 In-Reply-To: <200306251346.h5PDkcF19321@accms33.physik.rwth-aachen.de> References: <200306251346.h5PDkcF19321@accms33.physik.rwth-aachen.de> Message-ID: <20030626011158.GB27312@central.cis.upenn.edu> > Jun 25 10:24:34 accms07 kernel: Probably a duplex mismatch. See Documentation/networking/vortex.txt > Any clues? Yes. The clue is spelled out in the error message. :) Regards, Dan W. -- -- Daniel Widyono http://www.cis.upenn.edu/~widyono -- Liniac Project, CIS Dept., SEAS, University of Pennsylvania -- Mail: CIS Dept, 302 Levine 3330 Walnut St Philadelphia, PA 19104 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rmyers1400 at attbi.com Fri Jun 27 21:26:22 2003 From: rmyers1400 at attbi.com (Robert Myers) Date: Fri, 27 Jun 2003 21:26:22 -0400 Subject: Intel PRO/1000CT Gigabit ethernet with CSA (fwd) In-Reply-To: References: Message-ID: <3EFCEEBE.2060107@attbi.com> Donald Becker wrote: >This appears to connect directly to the internal motherboard chipset >crossbar, giving high-bandwidth access without (directly) conflicting >with the PCI bus. > >This architecture does reduce the bus access latency compared PCI, but >there will be no measurable reduction in network communication latency. > >If there were a Infiniband adapter connected to the same crossbar port, >you *would* be able to notice a significant reduction in single-word >transfer latency. > > There doesn't seem to be enough documentation anywhere even to decide whether such an enterprise is possible. CSA appears to be proprietary, so, Intel being Intel, you probably need a license to hook up to that port with an infiniband adapter of your own creation. Anyone have a clue otherwise? I feel almost silly asking, because every bit of evidence I've seen indicates that Intel is trying to create a competitive advantage for its own network silicon by giving it unique access to the MCH. RM RM _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From robl at mcs.anl.gov Fri Jun 27 21:50:56 2003 From: robl at mcs.anl.gov (Robert Latham) Date: Fri, 27 Jun 2003 20:50:56 -0500 Subject: Off topic - G5 question In-Reply-To: <3EFA19D1.7010009@yakko.cimav.edu.mx> References: <3EFA19D1.7010009@yakko.cimav.edu.mx> Message-ID: <20030628015056.GM9353@mcs.anl.gov> On Wed, Jun 25, 2003 at 03:53:21PM -0600, Luis Fernando Licon Padilla wrote: > Hi you all guys, I've just checked the G5 processor out, what do you > think about it, it seems like it's not good enough considering it's > architecture and it's performance. > > http://www.apple.com/lae/g5 a fast cpu is ok, but really (and i'm biased here :> ) IO is a much more limiting factor for high performance computing. so forget about the chip speeds and which one is faster... the G4 had pretty low heat dissipation. The itaniums are notorius for their heat dissipation (i've heard reports in the ballpark of 120 watts) while the G5 is reported to throw off 47 watts. But who can say what the real number will be when don't forget to factor operational costs into the price/performance calculation. ==rob -- Rob Latham Mathematics and Computer Science Division A215 0178 EA2D B059 8CDF Argonne National Labs, IL USA B29D F333 664A 4280 315B _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From becker at scyld.com Fri Jun 27 20:57:34 2003 From: becker at scyld.com (Donald Becker) Date: Fri, 27 Jun 2003 17:57:34 -0700 (PDT) Subject: Intel PRO/1000CT Gigabit ethernet with CSA (fwd) In-Reply-To: Message-ID: On Fri, 27 Jun 2003, Mark Hahn wrote: > Hi Don. do you happen to have any experience > with this new CSA feature - specifically whether it > actually lowers gigabit latency? This appears to connect directly to the internal motherboard chipset crossbar, giving high-bandwidth access without (directly) conflicting with the PCI bus. This architecture does reduce the bus access latency compared PCI, but there will be no measurable reduction in network communication latency. If there were a Infiniband adapter connected to the same crossbar port, you *would* be able to notice a significant reduction in single-word transfer latency. > Streaming Architecture) bus ? The Intel fact sheet is, as often, short > on quantitative detail but it looks promising: > > http://www.intel.com/design/network/products/lan/controllers/82547ei.htm Yup, this is very similar to the way earlier Intel Fast Ethernet chips connected to the Intel chipsets. If you look at the motherboards you'll see that those chips had significantly fewer pins than the PCI NICS, yet they appear to software to be a regular PCI device. > How does this NIC and a switch compare to Myrinet and similar > networks? Almost exactly the same as a 66Mhz PCI adapter. > modern 24 port switch like the Dell PowerConnect 5224, leading to > dropped packages and crashing jobs. You'll have the same constraints: Don't saturate with small packets Minimize multicast and broadcast packets Limit to low-rate service establishment (i.e. not back-to-back) Don't use for data transport. -- Donald Becker becker at scyld.com Scyld Computing Corporation http://www.scyld.com 914 Bay Ridge Road, Suite 220 Scyld Beowulf cluster system Annapolis MD 21403 410-990-9993 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From toon at moene.indiv.nluug.nl Sat Jun 28 06:13:03 2003 From: toon at moene.indiv.nluug.nl (Toon Moene) Date: Sat, 28 Jun 2003 12:13:03 +0200 Subject: [Fwd: Re: Off topic - G5 question] References: <3EFC91E6.2080701@moene.indiv.nluug.nl> Message-ID: <3EFD6A2F.50709@moene.indiv.nluug.nl> I wrote: > Well, I could make -O2 -ffast-math -funroll-loops the default (but then, > I don't think I'll get this past the IEEE754 police @ gcc.gnu.org :-) and Mark Hahn asked me (offline) whether -ffast-math is important. Well, as always: it depends. However, without -ffast-math, gcc and g77 zealously will follow IEEE 754 restrictions on floating point operations. Perhaps it's illustrative to see what that means by giving an example (culled from a bug report by one of Apple's compiler engineers). Consider this loop: DO IZ = 1, NZ DO IY = 1, NY DO IX = 1, NX CMAP(IX,IY,IZ) = DCONJG(CMAP(IX,IY,IZ)) / DFLOAT(NX*NY*NZ) END DO END DO END DO with I[XYZ], N[XYZ] integers and CMAP a double precision complex rank-3 array. DCONJG is expanded inline no matter what, so that's a rather basic operation. However, without -ffast-math, g77 is required to convert the double precision real DFLOAT(NX*NY*NZ) - a loop invariant - to double complex and than _perform a double complex division_ inside the loop. With -ffast-math, all that remains is two multiplications with the inverse of DFLOAT(NX*NY*NZ) for the real and imaginary parts of CMAP(IX,IY,IZ). -- Toon Moene - mailto:toon at moene.indiv.nluug.nl - phoneto: +31 346 214290 Saturnushof 14, 3738 XG Maartensdijk, The Netherlands Maintainer, GNU Fortran 77: http://gcc.gnu.org/onlinedocs/g77_news.html GNU Fortran 95: http://gcc-g95.sourceforge.net/ (under construction) _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From kuku at physik.rwth-aachen.de Sat Jun 28 03:24:17 2003 From: kuku at physik.rwth-aachen.de (Christoph P. Kukulies) Date: Sat, 28 Jun 2003 09:24:17 +0200 Subject: 3Com 905C and ASUS P4GX8 In-Reply-To: References: <200306251346.h5PDkcF19321@accms33.physik.rwth-aachen.de> Message-ID: <20030628072417.GA8289@gilberto.physik.rwth-aachen.de> On Thu, Jun 26, 2003 at 12:08:36AM -0700, Donald Becker wrote: > On Wed, 25 Jun 2003, Christoph Kukulies wrote: > > > I installed Linux RH 7.3 on a Dual 3GHz P4 CPU Asus P4GX8 MB > > and since the on board 100 MBit controller didn't seem to be > > supported by this Linux version (doubt whether any version does > > in the moment) > > The Scyld distribution ships with our driver package, and almost > certainly supports the card. But your problem is likely a configuration > issue, not a driver failure. > > > I'm seeing this in /var/log/messages and the machine > > is very slow on network copies and activity. > > What is the detection message? Jun 23 08:46:11 accms07 kernel: PCI: Found IRQ 9 for device 02:00.0 Jun 23 08:46:11 accms07 kernel: 3c59x: Donald Becker and others. www.scyld.com/network/vortex.html Jun 23 08:46:11 accms07 kernel: See Documentation/networking/vortex.txt Jun 23 08:46:11 accms07 kernel: 02:00.0: 3Com PCI 3c905C Tornado at 0xa800. Vers LK1.1.18-ac Jun 23 08:46:11 accms07 kernel: 00:01:02:1e:db:74, IRQ 9 Jun 23 08:46:11 accms07 kernel: product code 4552 rev 00.13 date 05-06-00 Jun 23 08:46:11 accms07 kernel: 8K byte-wide RAM 5:3 Rx:Tx split, autoselect/Autonegotiate interface. Jun 23 08:46:11 accms07 kernel: MII transceiver found at address 24, status 782d. Jun 23 08:46:11 accms07 kernel: Enabling bus-master transmits and whole-frame receives. Jun 23 08:46:11 accms07 kernel: 02:00.0: scatter/gather enabled. h/w checksums enabled Jun 23 08:46:11 accms07 kernel: Linux agpgart interface v0.99 (c) Jeff Hartmann > > > eth0: Transmit error, Tx status register 82. > > Read http://www.scyld.com/network/vortex.html > Specifically the section that says > 0x82 Out of window collision. This typically occurs when > some other Ethernet host is incorrectly set to full duplex on a half duplex > network. > > Thanks. I will check if the switch might be set to HD fixed instead of FD Auto. -- Chris Christoph P. U. Kukulies kukulies (at) rwth-aachen.de _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From hahn at physics.mcmaster.ca Sun Jun 29 12:37:45 2003 From: hahn at physics.mcmaster.ca (Mark Hahn) Date: Sun, 29 Jun 2003 12:37:45 -0400 (EDT) Subject: interconnect latency, disected. In-Reply-To: <200306291414.SAA12281@nocserv.free.net> Message-ID: does anyone have references handy for recent work on interconnect latency? specifically, I'm noticing two things: - many of the papers I see that contrast IB with GBE seem to claim that GBE latency is much larger than I measure (20-30 us). - I haven't found any recent studies on where even 25 us of GBE latency comes from. I recall a really great study from years ago that broke a single tx/rx down into an explicit timeline, but that was for ~100 MIP CPUs and 100bT, I think. - are there current/active efforts to use something other than TCP for implementing the usual MPI primitives? I'd love to see something that used ethernet's broad/multicast support, for instance. thanks, mark. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From lindahl at keyresearch.com Sun Jun 29 15:25:33 2003 From: lindahl at keyresearch.com (Greg Lindahl) Date: Sun, 29 Jun 2003 12:25:33 -0700 Subject: interconnect latency, disected. In-Reply-To: References: <200306291414.SAA12281@nocserv.free.net> Message-ID: <20030629192533.GA11418@greglaptop.greghome.keyresearch.com> On Sun, Jun 29, 2003 at 12:37:45PM -0400, Mark Hahn wrote: > - I haven't found any recent studies on where even 25 us of > GBE latency comes from. store-and-forward swithes? It's entertaining to measure back-to-back latency, but not very realistic. > - are there current/active efforts to use something other than > TCP for implementing the usual MPI primitives? I'd love to see > something that used ethernet's broad/multicast support, for instance. GAMMA and MVIA don't use TCP, but I don't think either has that. What kind of savings do you expect, given the mismatch between unreliable broadcast & multicast and MPI's semantics? I see a lot more enthusiasm for such features from people who haven't looked at the details... greg _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From fmahr at gmx.de Sun Jun 29 17:24:19 2003 From: fmahr at gmx.de (Ferdinand Mahr) Date: Sun, 29 Jun 2003 23:24:19 +0200 Subject: interconnect latency, disected. References: Message-ID: <3EFF5903.4E4E9D54@gmx.de> Hi everyone, Mark Hahn wrote: > > does anyone have references handy for recent work on interconnect latency? ...that's almost what I'm currently looking for. Does anyone have those references for a wider variety of interconnection technologies? I'm thinking of FE, GE, Myrinet, Dolphin, IB, Quadrics... I don't need the complete results from, say, a PALLAS MPI Benchmark run, but maybe the latency of a ping or something similar. It's not for a buying decision, but just to have a clue. Thanks, Ferdinand _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From sp at scali.com Sun Jun 29 16:22:33 2003 From: sp at scali.com (Steffen Persvold) Date: Sun, 29 Jun 2003 22:22:33 +0200 (CEST) Subject: Intel PRO/1000CT Gigabit ethernet with CSA In-Reply-To: <200306291414.SAA12281@nocserv.free.net> Message-ID: On Sun, 29 Jun 103, Mikhail Kuzminsky wrote: > According to Daniel Pfenniger > > Patrick Geoffray wrote: > > > On Fri, 2003-06-27 at 13:46, Daniel Pfenniger wrote: > > >>The exact costs are presently not well fixed because several companies > > >>enter the market. The nice thing about IB is that it is an open > > >>standard, the components from different companies are compatible, > > >>which is good for pressing costs down. > > > > > > With the slicon coming from one company (actually 2 but the second one > > > does only switch chip), the price adjustment would mainly affect the > > > reseller, where the margin are not that high. I don't expect much a > > > price war in the Infiniband market, mainly because many IB shops are > > > already just burning (limited) VC cash. > > > The main point for price advantage of IB is if the volume goes up. It's > > > a very different problem that the multiple-vendors-marketing-stuff. One > > > can argue that HPC does not yield such high volumes, only a business > > > market like the Databases one does. > > > > > > Remember Gigabit Ethernet. It was very expensive when the early adopters > > > were the HPC crowd and the price didn't drop until it made its way to > > > the desktop. It's the case for 10GE today. > > > ... > > > Patrick Geoffray > > > Myricom, Inc. > > > > Yes I mostly agree with your analysis, database is the only significant > > potential market for IB. > > > > However the problem with 1GBE or 10GBE is that the latency remains poor > > for HPC applications, while IB goes in the right direction. > > The real comparison to be made is not between GE and IB, but between > > IB and Myricom products, which belong to an especially protected niche. > > As a result for years the Myrinet products did hardly drop in price > > for a sub-Moore's-law increase in performance, because of a lack of > > competition (the price we paid for our Myricom cards and switch > > 18 months ago is today *exactly* the same). > > I agree with you both. From the viewpoint of HPC clusters the IB > competitor is Myrinet (and SCI etc). But there are many applications > w/coarse-grained parallelism, where bandwidth is the main thing, not the > latency (I think, quantum chemistry applications are bandwidth- > limited). In this case (i.e. if latnecy is less important) 10Gb Ethernet > is also IB competitor. Moreover, IB, I beleive, will be used for > TCP/IP connections also - in opposition to Myrinet etc. (I beleive > there is no TCP/IP drivers for Myrinet - am I correct ?) No, actually there is. The GM driver emulates an normal ethernet device the same way as IB does (and SCI). However this is not the strength of these high speed interconnects, and not the optimal way of using them (seen from our HPC/MPI perspective). > > Again, from the veiwpoint of some real appilications, there are some > applications which > use TCP/IP stack for parallelization (I agree that is bad, but ...) > - for example Linda tools (used in Gaussian) work over TCP/IP, Gamess-US > DDI "subsystem" works over TCP/IP. In the case of IB or 10Gb Ethernet > TCP/IP is possible. > I believe the current TCP/IP implementation in Linux is not capable of utilizing the possible bandwidth a 10Gb Ethernet connection can provide. Do do that you would need new protocols utilizing RDMA (doing true zerocopy as we do in MPI). And then of course you are into problems like userspace memory pinning etc... Regards, -- Steffen Persvold ,,, mailto: sp at scali.com Senior Software Engineer (o-o) http://www.scali.com -----------------------------oOO-(_)-OOo----------------------------- Scali AS, PObox 150, Oppsal, N-0619 Oslo, Norway, Tel: +4792484511 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From mprinkey at aeolusresearch.com Sun Jun 29 18:59:06 2003 From: mprinkey at aeolusresearch.com (Michael T. Prinkey) Date: Sun, 29 Jun 2003 18:59:06 -0400 (EDT) Subject: interconnect latency, disected. In-Reply-To: References: Message-ID: <50772.192.168.1.3.1056927546.squirrel@ra.aeolustec.com> There is at least two MPI Ethernet implementations that try to use transfer layers other than TCP: http://www.nersc.gov/research/FTG/mvich/ http://www.disi.unige.it/project/gamma/mpigamma/ Both claim ~10-13 usec latencies for older Gbit NICs. MVICH seems to be dead. Gamma is still not SMP safe. Both are pretty short on hardware support. AFAIK, neither support either of the Gbit NICs I have been using (tg3 and e1000). I recently tested Netgear GA622Ts end to end with netpipe. Latencies were roughly 22 usecs for packets <100 bytes, bandwidth was respectable (540 Mbps for 8k packets, 850 Mbps for ~100k packets). Latencies for Intel e1000 though a cheap 16-port switch were ~63 usecs. Bandwidth was lower across the board through the switch. Although I haven't done thorough testing, my best guess is that most of the latency is coming from the switch, so TCP vs UDP vs raw ethernet frames might be moot. > > does anyone have references handy for recent work on interconnect > latency? > > specifically, I'm noticing two things: > > - many of the papers I see that contrast IB with GBE seem to > claim that GBE latency is much larger than I measure (20-30 us). > > - I haven't found any recent studies on where even 25 us of > GBE latency comes from. I recall a really great study from years ago > that broke a single tx/rx down into an explicit timeline, > but that was for ~100 MIP CPUs and 100bT, I think. > > - are there current/active efforts to use something other than > TCP for implementing the usual MPI primitives? I'd love to see > something that used ethernet's broad/multicast support, for instance. > > thanks, mark. > > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org > To change your subscription (digest mode or unsubscribe) visit > http://www.beowulf.org/mailman/listinfo/beowulf _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From jcookeman at yahoo.com Sun Jun 29 23:20:16 2003 From: jcookeman at yahoo.com (Justin Cook) Date: Sun, 29 Jun 2003 20:20:16 -0700 (PDT) Subject: Hard link /etc/passwd Message-ID: <20030630032016.88507.qmail@web10607.mail.yahoo.com> Good day, I have an 11 node diskless cluster. All slave node roots are under /tftpboot/node1 ... /tftpboot/node2 ... so on. Is it safe to hard link the /etc/passwd and /etc/group file to the server nodes for consistency across the network? __________________________________ Do you Yahoo!? SBC Yahoo! DSL - Now only $29.95 per month! http://sbc.yahoo.com _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From jcownie at etnus.com Mon Jun 30 05:30:37 2003 From: jcownie at etnus.com (James Cownie) Date: Mon, 30 Jun 2003 10:30:37 +0100 Subject: [Fwd: Re: Off topic - G5 question] In-Reply-To: Message from Toon Moene of "Fri, 27 Jun 2003 20:50:14 +0200." <3EFC91E6.2080701@moene.indiv.nluug.nl> Message-ID: <19Wuzd-1Xe-00@etnus.com> Toon Moene wrote :- > Mark Hahn wrote: > > > I still see users fling their code at a machine with > > g77 prog.f && ./a.out > > and wonder why it's so slow. > > Well, I could make -O2 -ffast-math -funroll-loops the default (but then, > I don't think I'll get this past the IEEE754 police @ gcc.gnu.org :-) However there's nothing to stop Mark (assuming he has control of the machine) from setting the system up so that the "g77" you get by default is really a shell script which turns on appropriate optimisation flags, and also providing a command either to compile unoptimised, or to change the users $PATH so that they can get to a "raw" g77. I know that this is the approach used by Nick McLaren on the big Sun machines at Cambridge. http://www.hpcf.cam.ac.uk/prog_franklin_compilers.html Depending on the interests of your users (are they interested in their application domain, or in becoming compiler switch experts), they may prefer a system where someone has already defaulted the compiler to use an apporpiate set of magic optimisation flags. -- Jim James Cownie Etnus, LLC. +44 117 9071438 http://www.etnus.com _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From jhearns at freesolutions.net Mon Jun 30 05:39:33 2003 From: jhearns at freesolutions.net (John Hearns) Date: Mon, 30 Jun 2003 10:39:33 +0100 Subject: mpiBLAST article on The Register Message-ID: <3F000555.6000706@freesolutions.net> I think this has already been on the Beowulf list, so forgive me if that's so. An interesting article on MPI enabled BLAST on The Register: http://www.theregister.co.uk/content/61/31471.html "The 240 processor Linux cluster helped show that some scientific computing tasks will run with adequate performance and incredible reliability on a system that can fit in an average closest.(sic)" _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From tekka99 at libero.it Mon Jun 30 06:07:21 2003 From: tekka99 at libero.it (=?iso-8859-1?Q?Gianluca_Cecchi?=) Date: Mon, 30 Jun 2003 12:07:21 +0200 Subject: =?iso-8859-1?Q?Re:Hard_link_/etc/passwd?= Message-ID: > Good day, > I have an 11 node diskless cluster. All slave node > roots are under /tftpboot/node1 ... /tftpboot/node2 > ... so on. Is it safe to hard link the /etc/passwd > and /etc/group file to the server nodes for > consistency across the network? Apart from the beowulf related srategy, take in mind that hard links cannot traverse filesystems.. Bye, Gianluca _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From jcownie at etnus.com Mon Jun 30 05:00:16 2003 From: jcownie at etnus.com (James Cownie) Date: Mon, 30 Jun 2003 10:00:16 +0100 Subject: interconnect latency, dissected. In-Reply-To: Message from Ferdinand Mahr of "Sun, 29 Jun 2003 23:24:19 +0200." <3EFF5903.4E4E9D54@gmx.de> Message-ID: <19WuWG-1Uo-00@etnus.com> Mark Hahn wrote: > > does anyone have references handy for recent work on interconnect latency? Try http://www.cs.berkeley.edu/~bonachea/upc/netperf.pdf It doesn't have Inifinband, but does have Quadrics, Myrinet 2000, GigE and IBM. -- Jim James Cownie Etnus, LLC. +44 117 9071438 http://www.etnus.com _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From wrankin at ee.duke.edu Mon Jun 30 11:48:19 2003 From: wrankin at ee.duke.edu (Bill Rankin) Date: 30 Jun 2003 11:48:19 -0400 Subject: Job Posting - cluster admin. In-Reply-To: <20030627091300.A18502@mikee.ath.cx> References: <1056715034.2172.21.camel@rohgun.cse.duke.edu> <20030627091300.A18502@mikee.ath.cx> Message-ID: <1056988099.3706.20.camel@rohgun.cse.duke.edu> > I'd love to. Can I do it from Dallas? It would be a heck of a commute each morning. :-) -b _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From dieter at engr.uky.edu Mon Jun 30 17:15:21 2003 From: dieter at engr.uky.edu (William Dieter) Date: Mon, 30 Jun 2003 17:15:21 -0400 Subject: Hard link /etc/passwd In-Reply-To: <20030630032016.88507.qmail@web10607.mail.yahoo.com> Message-ID: You have to be careful when doing maintenance. For example, if you do: mv /etc/passwd /etc/passwd.bak cp /etc/passwd.bak /etc/passwd all of the copies will be linked to the backup copy. Normally you might not do this, but some text editors sometimes do similar things silently... A symbolic link might be safer. Bill. On Sunday, June 29, 2003, at 11:20 PM, Justin Cook wrote: > Good day, > I have an 11 node diskless cluster. All slave node > roots are under /tftpboot/node1 ... /tftpboot/node2 > ... so on. Is it safe to hard link the /etc/passwd > and /etc/group file to the server nodes for > consistency across the network? _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From clwang at csis.hku.hk Mon Jun 23 03:32:03 2003 From: clwang at csis.hku.hk (Cho-Li Wang) Date: Mon, 23 Jun 2003 15:32:03 +0800 Subject: Cluster2003: Final Call for Paper Message-ID: <3EF6ACF3.10D63009@csis.hku.hk> Dear Colleagues: Due to numerous requests from participants and especially the good news that Hong Kong is now removed from the SARS affected areas by WHO at 3:00pm today (June 23, Hong Kong time), we are pleased to further extend the submission deadline to June 30, 2003. If you have not submitted your paper to Cluster2003 yet, please consider to submit it before the new deadline. Best Regards, Cho-Li Wang and Daniel S. Katz Cluster2003 program Co-Chairs ----------------------------------------------------------------- ***************************************************** * Deadline further extended to June 30, 2003 (Firm) * ***************************************************** Cluster 2003 IEEE International Conference on Cluster Computing December 1-4, 2003, Hong Kong URL: http://www.csis.hku.hk/cluster2003/ Organized by Department of Computer Science and Information Systems, The University of Hong Kong Clusters today are an important element of mainstream computing. In less than a decade, "lowly" computer clusters have come to prominence as the most convenient and cost-effective tool for solving many complex computational problems. Being able to bring enough computing cycles together in a pool is one thing, but to be able to use these cycles efficiently so that the aggregate power of a cluster can be fully harnessed is another. Research and development efforts must continue to explore better strategies to make clusters an even greater tool when facing the grandest challenges of this age. Cluster 2003, to be held in the most dynamic city in the Orient, will provide an open forum for cluster researchers and system builders, as well as cluster users, to present and discuss new directions and results that could shape the future of cluster computing. Cluster 2003 welcomes paper submissions from engineers and scientists in academia, industry, and government describing their latest research findings in any aspects of cluster computing. Topics of interest include, but are not limited to: Cluster Middleware -- Single-System Image Services -- Software Environments and Tools -- Standard Software for Clusters -- I/O Libraries, File Systems, and Distributed RAID Cluster Networking -- High-Speed System Interconnects -- Lightweight Communication Protocols -- Fast Message Passing Libraries Managing Clusters -- Cluster Security and Reliability -- Tools for Managing Clusters -- Cluster Job and Resource Management -- High-Availability Cluster Solutions Applications -- Scientific and E-Commerce Applications -- Data Distribution and Load Balancing -- Innovative Cluster Applications Performance Analysis and Evaluation -- Benchmarking and Profiling Tools -- Performance Prediction and Modeling -- Analysis and Visualization Grid Computing and Clusters -- Grid Applications Integration -- Network-Based Distributed Computing -- Mobile Agents and Java for Clusters/Grids -- Middleware, Tools and Environments Paper Submission: Paper Format: Not to exceed 25 double-spaced pages (including figures, tables and references) using 10-12 point font on 8.5 x 11-inch pages. Show page number at each page. Include an abstract, five to ten keywords, and the corresponding author's e-mail address. Electronic Submission: Web-based submissions are strongly encouraged. The URL will be announced two weeks before the submission deadline at the conference web site. Authors should submit a PostScript (level 2) or PDF file. Postal submissions are not accepted. Journal Special Issue: Authors of the best papers from the conference will be invited to submit an expand version of their papers for possible publication in a special issue of the International Journal of High Performance Computing and Networking (IJHPCN), which will appear in 2004. Tutorials: For more information about proposals for organizing tutorials, please contact Tutorials Chair Ira Pramanick (ira.pramanick at sun.com) by June 30, 2003. Exhibitions: Companies interested in presenting exhibits at the meeting should contact one of the Exhibits Co-Chairs - Jim Ang (jaang at sandia.gov) or Nam Ng (Nng at cc.hku.hk) by August 31, 2003. Important Dates: -- Papers Submission Deadline: June 30, 2003 (Firm) -- Tutorial Proposals Due: June 30, 2003 -- Author Notification: (TBA) -- Exhibition Proposal Due: August 31, 2003 -- Final Manuscript Due: (TBA) -- Pre-Conference Tutorials: December 1, 2003 -- Conference: December 2-4, 2003 Proceedings of the conference and workshops will be available at the conference. ------------------ Chairs/Committees ------------------ General Co-Chairs Jack Dongarra (University of Tennessee) Lionel Ni (Hong Kong University of Science and Technology) General Vice Chair Francis C.M. Lau (The University of Hong Kong) Program Co-Chairs Daniel S. Katz (Jet Propulsion Laboratory) Cho-Li Wang (The University of Hong Kong) Program Vice Chairs Bill Gropp (Argonne National Laboratory) -- Middleware Wolfgang Rehm (Technische Universita"t Chemnitz) -- Hardware Zhiwei Xu (Chinese Academy of Sciences, China) -- Applications Tutorials Chair Ira Pramanick (Sun Microsystems) Workshops Chair Jiannong Cao (Hong Kong Polytechnic University) Exhibits/Sponsors Co-Chairs Jim Ang (Sandia National Lab) Nam Ng (The University of Hong Kong) Publications Chair Rajkumar Buyya (The University of Melbourne) Publicity Chair Arthur B. Maccabe (The University of New Mexico) Poster Chair Putchong Uthayopas (Kasetsart University) Finance/Registration Chair Alvin Chan (Hong Kong Polytechnic University) Local Arrangements Chair Anthony T.C. Tam (The University of Hong Kong) For further information please send e-mail to: cluster2003 at csis.hku.hk ************************************************************* To unsubscribe from srg-announce: send e-mail to: majordomo at csis.hku.hk, subject: anything, with content: unsubscribe srg-announce ************************************************************* _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From clwang at csis.hku.hk Mon Jun 23 03:32:03 2003 From: clwang at csis.hku.hk (Cho-Li Wang) Date: Mon, 23 Jun 2003 15:32:03 +0800 Subject: Cluster2003: Final Call for Paper Message-ID: <3EF6ACF3.10D63009@csis.hku.hk> Dear Colleagues: Due to numerous requests from participants and especially the good news that Hong Kong is now removed from the SARS affected areas by WHO at 3:00pm today (June 23, Hong Kong time), we are pleased to further extend the submission deadline to June 30, 2003. If you have not submitted your paper to Cluster2003 yet, please consider to submit it before the new deadline. Best Regards, Cho-Li Wang and Daniel S. Katz Cluster2003 program Co-Chairs ----------------------------------------------------------------- ***************************************************** * Deadline further extended to June 30, 2003 (Firm) * ***************************************************** Cluster 2003 IEEE International Conference on Cluster Computing December 1-4, 2003, Hong Kong URL: http://www.csis.hku.hk/cluster2003/ Organized by Department of Computer Science and Information Systems, The University of Hong Kong Clusters today are an important element of mainstream computing. In less than a decade, "lowly" computer clusters have come to prominence as the most convenient and cost-effective tool for solving many complex computational problems. Being able to bring enough computing cycles together in a pool is one thing, but to be able to use these cycles efficiently so that the aggregate power of a cluster can be fully harnessed is another. Research and development efforts must continue to explore better strategies to make clusters an even greater tool when facing the grandest challenges of this age. Cluster 2003, to be held in the most dynamic city in the Orient, will provide an open forum for cluster researchers and system builders, as well as cluster users, to present and discuss new directions and results that could shape the future of cluster computing. Cluster 2003 welcomes paper submissions from engineers and scientists in academia, industry, and government describing their latest research findings in any aspects of cluster computing. Topics of interest include, but are not limited to: Cluster Middleware -- Single-System Image Services -- Software Environments and Tools -- Standard Software for Clusters -- I/O Libraries, File Systems, and Distributed RAID Cluster Networking -- High-Speed System Interconnects -- Lightweight Communication Protocols -- Fast Message Passing Libraries Managing Clusters -- Cluster Security and Reliability -- Tools for Managing Clusters -- Cluster Job and Resource Management -- High-Availability Cluster Solutions Applications -- Scientific and E-Commerce Applications -- Data Distribution and Load Balancing -- Innovative Cluster Applications Performance Analysis and Evaluation -- Benchmarking and Profiling Tools -- Performance Prediction and Modeling -- Analysis and Visualization Grid Computing and Clusters -- Grid Applications Integration -- Network-Based Distributed Computing -- Mobile Agents and Java for Clusters/Grids -- Middleware, Tools and Environments Paper Submission: Paper Format: Not to exceed 25 double-spaced pages (including figures, tables and references) using 10-12 point font on 8.5 x 11-inch pages. Show page number at each page. Include an abstract, five to ten keywords, and the corresponding author's e-mail address. Electronic Submission: Web-based submissions are strongly encouraged. The URL will be announced two weeks before the submission deadline at the conference web site. Authors should submit a PostScript (level 2) or PDF file. Postal submissions are not accepted. Journal Special Issue: Authors of the best papers from the conference will be invited to submit an expand version of their papers for possible publication in a special issue of the International Journal of High Performance Computing and Networking (IJHPCN), which will appear in 2004. Tutorials: For more information about proposals for organizing tutorials, please contact Tutorials Chair Ira Pramanick (ira.pramanick at sun.com) by June 30, 2003. Exhibitions: Companies interested in presenting exhibits at the meeting should contact one of the Exhibits Co-Chairs - Jim Ang (jaang at sandia.gov) or Nam Ng (Nng at cc.hku.hk) by August 31, 2003. Important Dates: -- Papers Submission Deadline: June 30, 2003 (Firm) -- Tutorial Proposals Due: June 30, 2003 -- Author Notification: (TBA) -- Exhibition Proposal Due: August 31, 2003 -- Final Manuscript Due: (TBA) -- Pre-Conference Tutorials: December 1, 2003 -- Conference: December 2-4, 2003 Proceedings of the conference and workshops will be available at the conference. ------------------ Chairs/Committees ------------------ General Co-Chairs Jack Dongarra (University of Tennessee) Lionel Ni (Hong Kong University of Science and Technology) General Vice Chair Francis C.M. Lau (The University of Hong Kong) Program Co-Chairs Daniel S. Katz (Jet Propulsion Laboratory) Cho-Li Wang (The University of Hong Kong) Program Vice Chairs Bill Gropp (Argonne National Laboratory) -- Middleware Wolfgang Rehm (Technische Universita"t Chemnitz) -- Hardware Zhiwei Xu (Chinese Academy of Sciences, China) -- Applications Tutorials Chair Ira Pramanick (Sun Microsystems) Workshops Chair Jiannong Cao (Hong Kong Polytechnic University) Exhibits/Sponsors Co-Chairs Jim Ang (Sandia National Lab) Nam Ng (The University of Hong Kong) Publications Chair Rajkumar Buyya (The University of Melbourne) Publicity Chair Arthur B. Maccabe (The University of New Mexico) Poster Chair Putchong Uthayopas (Kasetsart University) Finance/Registration Chair Alvin Chan (Hong Kong Polytechnic University) Local Arrangements Chair Anthony T.C. Tam (The University of Hong Kong) For further information please send e-mail to: cluster2003 at csis.hku.hk ************************************************************* To unsubscribe from srg-announce: send e-mail to: majordomo at csis.hku.hk, subject: anything, with content: unsubscribe srg-announce ************************************************************* _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From alvin at Maggie.Linux-Consulting.com Sun Jun 1 05:21:29 2003 From: alvin at Maggie.Linux-Consulting.com (Alvin Oga) Date: Sun, 1 Jun 2003 02:21:29 -0700 (PDT) Subject: Cheap PCs from Wal-Mart In-Reply-To: <5.2.1.1.0.20030529163946.01d00a58@mail.toaster.net> Message-ID: hi ya robert On Thu, 29 May 2003, Robert wrote: > Hello > > these guys make small foot print 1U rackmountable low powered units and > they can also be wall mounted, real cool units for clusters for starter's. > > http://www.ironsystems.com/products/iservers/aclass/a110_low_power.htm for > $499.. and an even lower priced rackmountable 1U would be using a 1U shelf with holes for mb, power supply and disks ... ( no chassis cover, no chassis sides ... hotmail style .. also made by kingstar ) for cpu cooling ... i'd use a couple of regular 24" household fans to blow lots of air across the cpu and memory and disks cpu temperature vs lifespan graph http://www.Linux-1U.net/CPU - i've always wondered why nobody has household fans mounted on the sides of the cabinets to keep the cpu cooler ... ( minor detail is that there's no mounting holes in the fan ) for "small footprint 1U rackmounts" http://www.Linux-1U.net .... 9", 11", 14" .. deep chassis for microATX/flexITX 10" deep for mini-itx mb or microatx w/ p4-3.0G fsb-800 cpu and 2 ide disks ( 500GB total ) mini-itx is similar/same to the via eden series mb on the other extreme 25" deep 1U chassis for 8 drives per one 1U chassis ( 2TB per 1U ) have fun alvin _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From mikee at mikee.ath.cx Sun Jun 1 10:47:12 2003 From: mikee at mikee.ath.cx (Mike Eggleston) Date: Sun, 1 Jun 2003 09:47:12 -0500 Subject: Cheap PCs from Wal-Mart In-Reply-To: <5.2.1.1.0.20030529163946.01d00a58@mail.toaster.net>; from Robert@jaspreet.org on Thu, May 29, 2003 at 04:51:38PM -0700 References: <5.2.0.9.2.20030529152525.02e8cb68@mailhost4.jpl.nasa.gov> <5.2.1.1.0.20030529163946.01d00a58@mail.toaster.net> Message-ID: <20030601094712.C11031@mikee.ath.cx> On Thu, 29 May 2003, Robert wrote: > Hello > > these guys make small foot print 1U rackmountable low powered units and > they can also be wall mounted, real cool units for clusters for starter's. > > http://www.ironsystems.com/products/iservers/aclass/a110_low_power.htm for > $499.. I had not read of the Via chips and had been planning to use the AMD Athlon chips. Are the via chips ok? Will they run linux/mosix ok? Any specific thoughts about them? Mike _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From mikee at mikee.ath.cx Sun Jun 1 14:25:52 2003 From: mikee at mikee.ath.cx (Mike Eggleston) Date: Sun, 1 Jun 2003 13:25:52 -0500 Subject: Cheap PCs from Wal-Mart In-Reply-To: <3EDA426A.10102@tamu.edu>; from gerry.creager@tamu.edu on Sun, Jun 01, 2003 at 01:14:02PM -0500 References: <5.2.0.9.2.20030529152525.02e8cb68@mailhost4.jpl.nasa.gov> <20030601094712.C11031@mikee.ath.cx> <3EDA426A.10102@tamu.edu> Message-ID: <20030601132552.D11031@mikee.ath.cx> On Sun, 01 Jun 2003, Gerry Creager N5JXS wrote: > I've got a Via Epia Mini-ITX board I've gotten for another project. We > loaded RH 9 via PXE-Boot (with particular thanks to RGB!), as in, a > normal install rather than diskless boot. Works great. I've got the > slowest, equivalent to a 600 MHz Celeron, but that's all I needed for > this project... in fact, overkill. > > Runs pretty cool, even with just the 2 puny little case fans, and can be > driven off a 12v Wall Wart (?tm?; should I trade mark it? no...) at > less than 5a. The wart I have is rated for 4.5a, but I've not measured > it yet. > > Built-on sound, video, USB, 1394, ethernet, and video out. Would you please give the specifics for your borard, processor, memory, etc? I'm interested in your setup. Mike _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From hahn at physics.mcmaster.ca Sun Jun 1 16:56:02 2003 From: hahn at physics.mcmaster.ca (Mark Hahn) Date: Sun, 1 Jun 2003 16:56:02 -0400 (EDT) Subject: Cheap PCs from Wal-Mart In-Reply-To: <20030601094712.C11031@mikee.ath.cx> Message-ID: > I had not read of the Via chips and had been planning to use the AMD Athlon > chips. Are the via chips ok? Will they run linux/mosix ok? Any specific > thoughts about them? they're fairly slow, from what I've read, and still use the P5 pinout (which means you'll peak at around 500 MB/s). the chip is optimized for area/power/price first, not performance - VIA's been pushing it for use in embedded/appliance kinds of uses. there's nothing inherently wrong with using lots of low-power (both ways) chips. if you carefully evaluate your code, and find that higher-powered chips don't help much (for instance, you aren't actually bottlenecked on FP or memory bandwidth, or are drastically parallel.) as I see it, even if Amdahl's saw lets you take this path, you wind up spending more and more on non-computational hardware. for instance, compare: "light": 1U via/PIII-class server, $170 1U chassis - probably cheaper per unit. - probably cooler per unit. "heavy": 1U dual Xeon/Opteron server, $350 1U chassis - definitely faster (CPU, cache, dram, network). - definitely more reliable. - probably denser. - definitely hotter per unit. - definitely more expensive per unit. I consider heat the only serious issue with the 'heavy' approach, and there's just not much we can do about that except hold our breath for .09 ;) like heat, reliability is a huge concern mainly for large clusters - you definitely don't want 512 noname powersupplies, or the kind of chassis where the vendor doesn't realize that extra fans are a liability. it's easy to imagine situations where the light approach would win: a small, non-critical learning cluster, for instance, with not much space available, or cooling capacity. or even a google-ish DB-ish thing where you're really trying to make smart disk drives. some place with plentiful cheap labor. an interesting recent development is non-wimpy blade systems. the original blades had all the disadvantages of 'light' above, but also were extremely expensive and utterly proprietary. 'heavy blades' seem to be based on commodity motherboards (good), but apparently with a separate powersupply for each board (bad). I'd like to see someone market a redundant PS which could power several motherboards (not unlike the way minis and mainframes often use DC distribution). the motherboards would still be standard/commodity/non-proprietary, and could even have passive CPU heatsinks, and just a minimal tray to screw each MB onto. using wallmart PCs for "light" cluster nodes seems sensible to me, assuming they do actually have reasonable reliability. again, assuming density isn't an issue for you, and your code is happy with slow CPUs... _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From bob at drzyzgula.org Sun Jun 1 18:13:53 2003 From: bob at drzyzgula.org (Bob Drzyzgula) Date: Sun, 1 Jun 2003 18:13:53 -0400 Subject: Cheap PCs from Wal-Mart In-Reply-To: References: <20030601094712.C11031@mikee.ath.cx> Message-ID: <20030601181353.B25006@www2> On Sun, Jun 01, 2003 at 04:56:02PM -0400, Mark Hahn wrote: > > > I had not read of the Via chips and had been planning to use the AMD Athlon > > chips. Are the via chips ok? Will they run linux/mosix ok? Any specific > > thoughts about them? > > they're fairly slow, from what I've read, and still use the P5 pinout > (which means you'll peak at around 500 MB/s). the chip is optimized > for area/power/price first, not performance - VIA's been pushing it for > use in embedded/appliance kinds of uses. Actually, the lastest Via Cyrix/Centaur C3 chips use a Socket 370 interface, and support SSE. One thing that made them particularly slow until recently was that they ran the FPU at half the core speed; the 1GHz unit now runs it at full speed. Still, the C3 is probably a bad choice for a computationally intensive application. --Bob _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From Robert at jaspreet.org Sun Jun 1 15:37:42 2003 From: Robert at jaspreet.org (Robert) Date: Sun, 01 Jun 2003 12:37:42 -0700 Subject: Cheap PCs from Wal-Mart In-Reply-To: <3EDA426A.10102@tamu.edu> References: <20030601094712.C11031@mikee.ath.cx> <5.2.0.9.2.20030529152525.02e8cb68@mailhost4.jpl.nasa.gov> <5.2.1.1.0.20030529163946.01d00a58@mail.toaster.net> <20030601094712.C11031@mikee.ath.cx> Message-ID: <5.1.1.6.2.20030601123658.04d65d00@mail.toaster.net> Hello Gerry, Would u know if someone has worked on remote serial access for this EPIA Mini-ITX board. Rob At 01:14 PM 6/1/2003 -0500, Gerry Creager N5JXS wrote: >I've got a Via Epia Mini-ITX board I've gotten for another project. We >loaded RH 9 via PXE-Boot (with particular thanks to RGB!), as in, a normal >install rather than diskless boot. Works great. I've got the slowest, >equivalent to a 600 MHz Celeron, but that's all I needed for this >project... in fact, overkill. > >Runs pretty cool, even with just the 2 puny little case fans, and can be >driven off a 12v Wall Wart (?tm?; should I trade mark it? no...) at less >than 5a. The wart I have is rated for 4.5a, but I've not measured it yet. > >Built-on sound, video, USB, 1394, ethernet, and video out. > >gerry > > >Mike Eggleston wrote: >>On Thu, 29 May 2003, Robert wrote: >> >>>Hello >>> >>>these guys make small foot print 1U rackmountable low powered units and >>>they can also be wall mounted, real cool units for clusters for starter's. >>> >>>http://www.ironsystems.com/products/iservers/aclass/a110_low_power.htm >>>for $499.. >> >>I had not read of the Via chips and had been planning to use the AMD Athlon >>chips. Are the via chips ok? Will they run linux/mosix ok? Any specific >>thoughts about them? >>Mike >>_______________________________________________ >>Beowulf mailing list, Beowulf at beowulf.org >>To change your subscription (digest mode or unsubscribe) visit >>http://www.beowulf.org/mailman/listinfo/beowulf > >-- >Gerry Creager -- gerry.creager at tamu.edu >Network Engineering -- AATLT, Texas A&M University >Cell: 979.229.5301 Office: 979.458.4020 FAX: 979.847.8578 >Page: 979.228.0173 >Office: 903A Eller Bldg, TAMU, College Station, TX 77843 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From gerry.creager at tamu.edu Sun Jun 1 14:14:02 2003 From: gerry.creager at tamu.edu (Gerry Creager N5JXS) Date: Sun, 01 Jun 2003 13:14:02 -0500 Subject: Cheap PCs from Wal-Mart In-Reply-To: <20030601094712.C11031@mikee.ath.cx> References: <5.2.0.9.2.20030529152525.02e8cb68@mailhost4.jpl.nasa.gov> <5.2.1.1.0.20030529163946.01d00a58@mail.toaster.net> <20030601094712.C11031@mikee.ath.cx> Message-ID: <3EDA426A.10102@tamu.edu> I've got a Via Epia Mini-ITX board I've gotten for another project. We loaded RH 9 via PXE-Boot (with particular thanks to RGB!), as in, a normal install rather than diskless boot. Works great. I've got the slowest, equivalent to a 600 MHz Celeron, but that's all I needed for this project... in fact, overkill. Runs pretty cool, even with just the 2 puny little case fans, and can be driven off a 12v Wall Wart (?tm?; should I trade mark it? no...) at less than 5a. The wart I have is rated for 4.5a, but I've not measured it yet. Built-on sound, video, USB, 1394, ethernet, and video out. gerry Mike Eggleston wrote: > On Thu, 29 May 2003, Robert wrote: > > >>Hello >> >>these guys make small foot print 1U rackmountable low powered units and >>they can also be wall mounted, real cool units for clusters for starter's. >> >>http://www.ironsystems.com/products/iservers/aclass/a110_low_power.htm for >>$499.. > > > I had not read of the Via chips and had been planning to use the AMD Athlon > chips. Are the via chips ok? Will they run linux/mosix ok? Any specific > thoughts about them? > > Mike > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org > To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- Gerry Creager -- gerry.creager at tamu.edu Network Engineering -- AATLT, Texas A&M University Cell: 979.229.5301 Office: 979.458.4020 FAX: 979.847.8578 Page: 979.228.0173 Office: 903A Eller Bldg, TAMU, College Station, TX 77843 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From mikee at mikee.ath.cx Sun Jun 1 23:18:02 2003 From: mikee at mikee.ath.cx (Mike Eggleston) Date: Sun, 1 Jun 2003 22:18:02 -0500 Subject: Cheap PCs from Wal-Mart In-Reply-To: <20030601181353.B25006@www2>; from bob@drzyzgula.org on Sun, Jun 01, 2003 at 06:13:53PM -0400 References: <20030601094712.C11031@mikee.ath.cx> <20030601181353.B25006@www2> Message-ID: <20030601221802.A16442@mikee.ath.cx> On Sun, 01 Jun 2003, Bob Drzyzgula wrote: > On Sun, Jun 01, 2003 at 04:56:02PM -0400, Mark Hahn wrote: > > > > > I had not read of the Via chips and had been planning to use the AMD Athlon > > > chips. Are the via chips ok? Will they run linux/mosix ok? Any specific > > > thoughts about them? > > > > they're fairly slow, from what I've read, and still use the P5 pinout > > (which means you'll peak at around 500 MB/s). the chip is optimized > > for area/power/price first, not performance - VIA's been pushing it for > > use in embedded/appliance kinds of uses. > > Actually, the lastest Via Cyrix/Centaur C3 chips use a Socket 370 interface, > and support SSE. One thing that made them particularly slow until > recently was that they ran the FPU at half the core speed; the 1GHz > unit now runs it at full speed. > > Still, the C3 is probably a bad choice for a computationally intensive > application. Ok, but what about something like a cluster with mosix in front and some database servers in back? Or web servers in front, or email servers.... this sort of thing? Mike _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From jhearns at freesolutions.net Mon Jun 2 02:18:24 2003 From: jhearns at freesolutions.net (John Hearns) Date: 02 Jun 2003 07:18:24 +0100 Subject: ClusterKnoppix Message-ID: <1054534705.15377.8.camel@harwood.home> I saw this on Slashdot a few days ago. It is a customised Knoppix distribution, with an OpenMosix patched kernel, and PXE/DHCP enabled etc. I intend to try this out in the next few days. For those who haven't used Knoppix, it is a CD which contains a bootable Debian-based distribution which installs itself in memory, not touching the hard drive. Knoppix has excellent hardware detection, and sets up a good desktop. It 'just works'. Uses might include demonstrating a Linux desktop to friends/colleagues, testing if a PC can run Linux before buying/accepting it, or as a rescue disk for troubleshooting an already installed Linux system. John Hearns _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From alvin at Maggie.Linux-Consulting.com Mon Jun 2 03:02:24 2003 From: alvin at Maggie.Linux-Consulting.com (Alvin Oga) Date: Mon, 2 Jun 2003 00:02:24 -0700 (PDT) Subject: Cheap PCs from Wal-Mart - mini-itx In-Reply-To: <20030601132552.D11031@mikee.ath.cx> Message-ID: hi ya On Sun, 1 Jun 2003, Mike Eggleston wrote: > On Sun, 01 Jun 2003, Gerry Creager N5JXS wrote: > > > I've got a Via Epia Mini-ITX board I've gotten for another project. We > > loaded RH 9 via PXE-Boot (with particular thanks to RGB!), as in, a > > normal install rather than diskless boot. Works great. I've got the > > slowest, equivalent to a 600 MHz Celeron, but that's all I needed for > > this project... in fact, overkill. > > > > Runs pretty cool, even with just the 2 puny little case fans, and can be > > driven off a 12v Wall Wart (?tm?; should I trade mark it? no...) at > > less than 5a. The wart I have is rated for 4.5a, but I've not measured > > it yet. we did lots of testing on that 12v dc-dc input to the mini-itx mb ( well, www.mini-box.com folks did all the testing and ordering of the ( custom 12v dc-to-dc atx power supply we're using the freetech mini-itx mb that supposed to handle p4-3G ( our test will be to keep the cpu cool in our 1u box ) have fun alvin _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From alvin at Maggie.Linux-Consulting.com Mon Jun 2 03:32:59 2003 From: alvin at Maggie.Linux-Consulting.com (Alvin Oga) Date: Mon, 2 Jun 2003 00:32:59 -0700 (PDT) Subject: Cheap PCs from Wal-Mart - heat In-Reply-To: Message-ID: hi ya mark On Sun, 1 Jun 2003, Mark Hahn wrote: > for instance, compare: > "light": 1U via/PIII-class server, $170 1U chassis > - probably cheaper per unit. > - probably cooler per unit. > > "heavy": 1U dual Xeon/Opteron server, $350 1U chassis > - definitely faster (CPU, cache, dram, network). > - definitely more reliable. > - probably denser. > - definitely hotter per unit. > - definitely more expensive per unit. actually, both cases can be identical ... only major differences can be types of fans for airflow and wattage of the power supply thickness of the steel also makes a big difference to reliability of the 1U server > I consider heat the only serious issue with the 'heavy' approach, and there's > just not much we can do about that except hold our breath for .09 ;) > like heat, reliability is a huge concern mainly for large clusters - you > definitely don't want 512 noname powersupplies, or the kind of chassis > where the vendor doesn't realize that extra fans are a liability. fans vs cpu temp vs cooling mechanism implies a specific cooling mechanism instead of generic solutions with lots of "useless" fans?? - if you look at the airflow, most of the fans don't do anything since 8 good 40x40x20 fans is about $100 retail... it'd make sense to use some other more cost-effective cooling of the cpu heatsink - yes you can buy $3.oo fans... but, you get what you pay for ? > the motherboards would still be standard/commodity/non-proprietary, > and could even have passive CPU heatsinks, and just a minimal tray to > screw each MB onto. passive heatsink w/ proper airflow works much better than those itty-bitty fans clipped on top of the cpu heatsink - fan blades needs ( say 0.5" of air ) to bite into to generate airflow and cooling of the cpu - air flow blowing down onto the cpu and bending 90degrees sideways is not helping the coooling air stream to cool the cpu - we put our cpu fans on the side :-) have fun alvin _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From thomas.sinczak at wp.pl Mon Jun 2 13:05:10 2003 From: thomas.sinczak at wp.pl (Thomas Sinczak) Date: Mon, 02 Jun 2003 19:05:10 +0200 Subject: use of gimp with a claster?! Message-ID: <3EDB83C6.4090800@wp.pl> Dear all, I am new to claster world... Up to now I have tested only openMosix - with results that do not help me... my problem is as follows: I have two computers: 1,8GHz Celeron with 256MB RAM and 100T Ethernet 1,26GHz PIII with 512MB RAM and 100T Ethernet I am doing some graphics - let say 6 Mpixel photos (17MB in size raw) that are manipulated with GIMP and so my question is: is that possible that the gimp will utilise the power of both computers with the use of beowulf claster?? if yes then please show me any howto Best Regards ThomasS -- ____ thomas.sinczak at wp.pl Exchange student of Osaka University, Japan - OUSSEP Student of Technical University of Lodz, Poland - Mechatronics ---- Daddy, why is that man using Windows...? Do not stare at him son... - it`s impolite! ---- - join EU ! - are you jokeing?? check the facts!! then let us speak again! ---- end of message _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From jakob at unthought.net Mon Jun 2 06:15:58 2003 From: jakob at unthought.net (Jakob Oestergaard) Date: Mon, 2 Jun 2003 12:15:58 +0200 Subject: Gigabit performance issues and NFS In-Reply-To: <5.0.2.1.2.20030529092240.02aece48@pop.larc.nasa.gov> References: <5.0.2.1.2.20030529092240.02aece48@pop.larc.nasa.gov> Message-ID: <20030602101557.GB9276@unthought.net> On Thu, May 29, 2003 at 09:35:54AM -0400, Doug Farley wrote: > Fellow Wulfers, > > I know this isnt 100% wulf related, although it is part of my wulfs setup, > but this is the best forum where everyone has alot of good experience. NFS is 'wulf related, whether we like it or not :) > > Well Heres the deal, I have a nice 2TB Linux file server with an Intel > e1000 based nic in it. And I have an SGI O3 (master node) that is dumping > to it with a tigon series gigabit card. I've tuned both, and my ttcp and > netpipe performance average ~ 80-95MB/s which is more than reasonable for > me. Both the fibre channel on my SGI and the raid (3ware) on my Linux box > can write at 40MB/s sustained, read is a little faster for both maybe ~ > 50MB/s sustained. I can get ftp/http transfers between the two to go at > 39-40MB/s, which again i'm reasonably happy with. BUT, the part that is > killing me is nfs and scp. Both crawl in at around 8-11MB/s with no other > devices on the network. 11MB/sec with scp is quite good - considering everything is encrypted and what not... With NFS that's pretty poor though, I'd agree. > Any exports from the SGI i've exported with the > 32bitclients flag, and i've pumped my r&wsize windows up to 32K, and forced > nfs v3 on both Linux and Irix. After spending a week scouring the web I've > found nothing that has worked, and SGI support thinks its a Linux nfs > problem, which could be, but i'd like to get the opinion of this crowd in > hopes of some light! What does top and vmstat on your NFS server tell you? How many nfsd threads are busy (in R or D state), during the writes ? The default number of nfsd threads is 8, which may be a little low. I run 16 threads here, on a somewhat smaller NFS server (also with a Gbit NIC). If you only see one or two nfsd threads in R or D state, anywhere near the top of your "top", then this should not be the problem. Try specifying the "async" parameter for the given mount in your exports file on the NFS server. Just to see if this helps. There are some considerations you need to make here - if the client does a sync() and you use the async option on the server, you are not guaranteed that the data has reached the disk platters by the time the client sync() call returns. This may or may not matter for you. What does vmstat say during such a big write? Is the CPU idle or busy, is it spending all it's time in the kernel? How's the ping between the machines, when doing the write and when the network is more idle? You may have a switch in between that does store-and-forward instead of cut-through, when the network gets loaded. Latency hurts NFS. -- ................................................................ : jakob at unthought.net : And I see the elder races, : :.........................: putrid forms of man : : Jakob ?stergaard : See him rise and claim the earth, : : OZ9ABN : his downfall is at hand. : :.........................:............{Konkhra}...............: _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From d.l.farley at larc.nasa.gov Mon Jun 2 10:17:53 2003 From: d.l.farley at larc.nasa.gov (Doug Farley) Date: Mon, 02 Jun 2003 10:17:53 -0400 Subject: Gigabit performance issues and NFS In-Reply-To: <20030602101557.GB9276@unthought.net> References: <5.0.2.1.2.20030529092240.02aece48@pop.larc.nasa.gov> <5.0.2.1.2.20030529092240.02aece48@pop.larc.nasa.gov> Message-ID: <5.0.2.1.2.20030602080851.02ced0f0@pop.larc.nasa.gov> Jakob, For the switch I have a hp procurve 2708, from the netpipe results i'd say its a cut-through switch, however I may have not applied enough load to it to find out, and I have seen no data either way to determine otherwise. On writing from the SGI to my Linux NFS server I have 4 nfsd's running on the Linux box, with all 4 in either a R or D state. I've included copies of the top results for reference. In regards to the use of async instead of sync in the exports, the speed was 10-14MB/s, there were again 2 nfsd's in R state and 2 in D state as before. with a load average of ~ 4, each of the nfsd consuming ~ 3% cpu, and each of the biod's consuming about 6%. I hope that maybe some of these numbers/figures (while possibly excessive) might help the group shed some light onto my problem. Thank you all again, Doug Farley For SGI Writing to Linux: ping before: ----Linux-g.localdomain PING Statistics---- 5 packets transmitted, 5 packets received, 0.0% packet loss round-trip min/avg/max = 0.219/0.286/0.427 ms ----Linux-g.localdomain PING Statistics---- 22889 packets transmitted, 22888 packets received, 0.0% packet loss round-trip min/avg/max = 0.202/0.476/2.307 ms 2072.5 packets/sec sent, 2072.5 packets/sec received --- SGI-g.localdomain ping statistics --- 13 packets transmitted, 13 received, 0% loss, time 11993ms rtt min/avg/max/mdev = 0.108/0.178/0.230/0.036 ms /ping before ping during: ----Linux-g.localdomain PING Statistics---- 6 packets transmitted, 6 packets received, 0.0% packet loss round-trip min/avg/max = 0.240/0.333/0.380 ms ----Linux-g.localdomain PING Statistics---- 33140 packets transmitted, 33140 packets received, 0.0% packet loss round-trip min/avg/max = 0.149/0.264/2.670 ms 3687.1 packets/sec sent, 3687.1 packets/sec received --- SGI-g.localdomain ping statistics --- 9 packets transmitted, 9 received, 0% loss, time 7999ms rtt min/avg/max/mdev = 0.130/0.230/0.316/0.050 ms --- SGI-g.localdomain ping statistics --- 25450 packets transmitted, 25449 received, 0% loss, time 11371ms rtt min/avg/max/mdev = 0.076/0.243/2.221/0.059 ms, ipg/ewma 0.446/0.249 ms /ping during SGI: top header before: ---- IRIX64 SGI 6.5 IP35 load averages: 0.02 0.00 0.00 57 processes: 56 sleeping, 1 running 8 CPUs: 99.4% idle, 0.0% usr, 0.4% ker, 0.0% wait, 0.0% xbrk, 0.2% intr Memory: 8192M max, 7313M avail, 7264M free, 4096M swap, 4096M free swap /top header before top header during: IRIX64 SGI 6.5 IP35 load averages: 1.28 0.48 0.18 61 processes: 59 sleeping, 2 running 8 CPUs: 83.6% idle, 0.1% usr, 7.3% ker, 6.2% wait, 0.0% xbrk, 2.9% intr Memory: 8192M max, 7455M avail, 5703M free, 4096M swap, 4096M free swap PID PGRP USERNAME PRI SIZE RES STATE TIME WCPU% CPU% COMMAND 511570519 511570519 dfarley 20 416K 304K sleep 0:29 22.2 30.16 ln 511571530 511571530 root 21 0K 0K sleep 0:08 6.5 6.63 bio3d 8975393 8975393 root 21 0K 0K sleep 0:07 6.1 6.53 bio3d 511549378 511549378 root 21 0K 0K run/1 12:24 6.0 6.15 bio3d 511563810 511563810 root 21 0K 0K sleep 0:08 6.5 5.74 bio3d 511543776 511541950 root 20 0K 0K sleep 49:18 2.2 2.88 nfsd 511568500 511568500 dfarley 20 2208K 1536K run/3 0:00 0.2 0.24 top 511567378 511549203 dfarley 20 4208K 3104K sleep 0:00 0.0 0.02 sshd 8928 8928 root 20 1808K 1088K sleep 1:02 0.0 0.01 prngd 410 410 root 20 2512K 2512K sleep 1:34 0.0 0.01 ntpd 381 0 root 20 2816K 2064K sleep 0:32 0.0 0.00 ipmon /top header during /SGI Linux Box: vmstat before: procs memory swap io system cpu r b w swpd free buff cache si so bi bo in cs us sy id 0 0 0 18980 18012 516412 418384 0 0 22 40 44 41 0 1 47 /vmstat before vmstat during: procs memory swap io system cpu r b w swpd free buff cache si so bi bo in cs us sy id 0 0 5 18980 8552 518828 445648 0 0 22 43 45 44 0 1 47 /vmstat during top header before: up 9 days, 17:03, 2 users, load average: 0.33, 0.06, 0.00 72 processes: 70 sleeping, 2 running, 0 zombie, 0 stopped CPU states: 0.0% user, 1.0% system, 0.0% nice, 99.0% idle Mem: 1031204K av, 1015316K used, 15888K free, 0K shrd, 532128K buff Swap: 2048276K av, 18980K used, 2029296K free 422944K cached /top header before top header during: up 9 days, 17:27, 2 users, load average: 2.60, 0.76, 0.26 70 processes: 66 sleeping, 4 running, 0 zombie, 0 stopped CPU states: 0.0% user, 25.6% system, 0.0% nice, 74.4% idle Mem: 1031204K av, 1022200K used, 9004K free, 0K shrd, 517376K buff Swap: 2048276K av, 18980K used, 2029296K free 446512K cached PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME COMMAND 14732 root 15 0 0 0 0 RW 4.2 0.0 1:21 nfsd 14725 root 15 0 0 0 0 DW 3.8 0.0 0:55 nfsd 14726 root 14 0 0 0 0 RW 3.8 0.0 2:08 nfsd 14729 root 14 0 0 0 0 DW 3.6 0.0 2:52 nfsd 5 root 9 0 0 0 0 SW 0.6 0.0 1:12 kswapd 6 root 9 0 0 0 0 SW 0.6 0.0 32:40 kscand 14730 root 9 0 0 0 0 SW 0.2 0.0 1:23 nfsd 1 root 9 0 156 128 96 S 0.0 0.0 0:04 init 2 root 9 0 0 0 0 SW 0.0 0.0 0:00 keventd 3 root 9 0 0 0 0 SW 0.0 0.0 0:00 kapmd /top header during /Linux Box ============================== Doug Farley Data Analysis and Imaging Branch Systems Engineering Competency NASA Langley Research Center < D.L.FARLEY at LaRC.NASA.GOV > < Phone +1 757 864-8141 > At 12:15 PM 6/2/2003 +0200, Jakob Oestergaard wrote: >On Thu, May 29, 2003 at 09:35:54AM -0400, Doug Farley wrote: > > Fellow Wulfers, > > > > I know this isnt 100% wulf related, although it is part of my wulfs setup, > > but this is the best forum where everyone has alot of good experience. > >NFS is 'wulf related, whether we like it or not :) > > > > > Well Heres the deal, I have a nice 2TB Linux file server with an Intel > > e1000 based nic in it. And I have an SGI O3 (master node) that is dumping > > to it with a tigon series gigabit card. I've tuned both, and my ttcp and > > netpipe performance average ~ 80-95MB/s which is more than reasonable for > > me. Both the fibre channel on my SGI and the raid (3ware) on my Linux box > > can write at 40MB/s sustained, read is a little faster for both maybe ~ > > 50MB/s sustained. I can get ftp/http transfers between the two to go at > > 39-40MB/s, which again i'm reasonably happy with. BUT, the part that is > > killing me is nfs and scp. Both crawl in at around 8-11MB/s with no other > > devices on the network. > >11MB/sec with scp is quite good - considering everything is encrypted >and what not... > >With NFS that's pretty poor though, I'd agree. > > > Any exports from the SGI i've exported with the > > 32bitclients flag, and i've pumped my r&wsize windows up to 32K, and > forced > > nfs v3 on both Linux and Irix. After spending a week scouring the web > I've > > found nothing that has worked, and SGI support thinks its a Linux nfs > > problem, which could be, but i'd like to get the opinion of this crowd in > > hopes of some light! > >What does top and vmstat on your NFS server tell you? > >How many nfsd threads are busy (in R or D state), during the writes ? > >The default number of nfsd threads is 8, which may be a little low. I >run 16 threads here, on a somewhat smaller NFS server (also with a Gbit >NIC). If you only see one or two nfsd threads in R or D state, >anywhere near the top of your "top", then this should not be the >problem. > >Try specifying the "async" parameter for the given mount in your exports >file on the NFS server. Just to see if this helps. There are some >considerations you need to make here - if the client does a sync() and >you use the async option on the server, you are not guaranteed that the >data has reached the disk platters by the time the client sync() call >returns. This may or may not matter for you. > >What does vmstat say during such a big write? Is the CPU idle or busy, >is it spending all it's time in the kernel? > >How's the ping between the machines, when doing the write and when the >network is more idle? You may have a switch in between that does >store-and-forward instead of cut-through, when the network gets loaded. >Latency hurts NFS. > >-- >................................................................ >: jakob at unthought.net : And I see the elder races, : >:.........................: putrid forms of man : >: Jakob ?stergaard : See him rise and claim the earth, : >: OZ9ABN : his downfall is at hand. : >:.........................:............{Konkhra}...............: _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From gerry.creager at tamu.edu Sun Jun 1 23:47:25 2003 From: gerry.creager at tamu.edu (Gerry Creager N5JXS) Date: Sun, 01 Jun 2003 22:47:25 -0500 Subject: Cheap PCs from Wal-Mart In-Reply-To: <20030601132552.D11031@mikee.ath.cx> References: <5.2.0.9.2.20030529152525.02e8cb68@mailhost4.jpl.nasa.gov> <20030601094712.C11031@mikee.ath.cx> <3EDA426A.10102@tamu.edu> <20030601132552.D11031@mikee.ath.cx> Message-ID: <3EDAC8CD.1070109@tamu.edu> Via EPiA M-6000 motherboard/chipset combo (at least that's what I recall, but the EPiA V looks closer to what I installed...) with an SiS onboard audio chipset, 10/100 ethernet onboard VGA video chipset, and "tv-out" in both S-Video and composite (RCA jack). I'm running it with a 40 GB Maxtor (boought solely on price as there was nothing smaller available for less money at the time). I added a Hayes 56k V92 modem because what I'm planning to do is poll a bunch of automated weather sites for data and they are all dial-up... Check out: http://www.viavpsd.com/product/epia_v_spec.jsp?motherboardId=141 http://www.viavpsd.com/product/epia_m_spec.jsp?motherboardId=81 http://www.viavpsd.com/product/epia_mini_itx_spec.jsp?motherboardId=21 The case I found (at Fry's) was $79; short of going into the office tonight I'll have to look on outpost.com -- darn! Not there. I'll try to find the info on the case, but it comes with a wall wart for AC power, and there's a 12v coaxial DC jack on the case... I'm not at all sure I'd use it for a compute cluster, but it might do some low-end work and it is certainly is a low-heat/low power solution. Gerry Mike Eggleston wrote: > On Sun, 01 Jun 2003, Gerry Creager N5JXS wrote: > > >>I've got a Via Epia Mini-ITX board I've gotten for another project. We >>loaded RH 9 via PXE-Boot (with particular thanks to RGB!), as in, a >>normal install rather than diskless boot. Works great. I've got the >>slowest, equivalent to a 600 MHz Celeron, but that's all I needed for >>this project... in fact, overkill. >> >>Runs pretty cool, even with just the 2 puny little case fans, and can be >>driven off a 12v Wall Wart (?tm?; should I trade mark it? no...) at >>less than 5a. The wart I have is rated for 4.5a, but I've not measured >>it yet. >> >>Built-on sound, video, USB, 1394, ethernet, and video out. > > > Would you please give the specifics for your borard, processor, memory, > etc? I'm interested in your setup. > > Mike -- Gerry Creager -- gerry.creager at tamu.edu Network Engineering -- AATLT, Texas A&M University Cell: 979.229.5301 Office: 979.458.4020 FAX: 979.847.8578 Page: 979.228.0173 Office: 903A Eller Bldg, TAMU, College Station, TX 77843 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From jhearns at freesolutions.net Mon Jun 2 11:26:51 2003 From: jhearns at freesolutions.net (John Hearns) Date: Mon, 2 Jun 2003 16:26:51 +0100 Subject: Cheap PCs from Wal-Mart References: <5.2.0.9.2.20030529152525.02e8cb68@mailhost4.jpl.nasa.gov> <5.2.1.1.0.20030529163946.01d00a58@mail.toaster.net> <20030601094712.C11031@mikee.ath.cx> <3EDA426A.10102@tamu.edu> <3EDAFA58.5070907@nada.kth.se> Message-ID: <002401c3291b$67a52d80$8461cdc2@DREAD> I think the list knows that I'm a fan of the mini-ITX - I've got four of them at home. (I've had a paper accepted for the UKUUG Linux Developers Conference on mini-ITX also) re. the points on floating point performance, these are valid. But I think we should note the new Nehemiah boards which are just out. Well worth looking at I think - I intend to get one soon, and I'll report back if/when I do. http://www.mini-itx.com/reviews/nehemiah/?page=10#s21 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From tegner at nada.kth.se Mon Jun 2 03:18:48 2003 From: tegner at nada.kth.se (Jon Tegner) Date: Mon, 02 Jun 2003 09:18:48 +0200 Subject: Cheap PCs from Wal-Mart In-Reply-To: <3EDA426A.10102@tamu.edu> References: <5.2.0.9.2.20030529152525.02e8cb68@mailhost4.jpl.nasa.gov> <5.2.1.1.0.20030529163946.01d00a58@mail.toaster.net> <20030601094712.C11031@mikee.ath.cx> <3EDA426A.10102@tamu.edu> Message-ID: <3EDAFA58.5070907@nada.kth.se> I have used the Via Epia Mini-ITX board (EPIA M9000), and I would not recommend it on any application heavy on floating point operations. In my experience (CFD) it is closer to a 300 MHz Celeron than a 600 (but I guess it depends on the application...). /jon Gerry Creager N5JXS wrote: > I've got a Via Epia Mini-ITX board I've gotten for another project. > We loaded RH 9 via PXE-Boot (with particular thanks to RGB!), as in, a > normal install rather than diskless boot. Works great. I've got the > slowest, equivalent to a 600 MHz Celeron, but that's all I needed for > this project... in fact, overkill. > > Runs pretty cool, even with just the 2 puny little case fans, and can > be driven off a 12v Wall Wart (?tm?; should I trade mark it? no...) > at less than 5a. The wart I have is rated for 4.5a, but I've not > measured it yet. > > Built-on sound, video, USB, 1394, ethernet, and video out. > > gerry > > > Mike Eggleston wrote: > >> On Thu, 29 May 2003, Robert wrote: >> >> >>> Hello >>> >>> these guys make small foot print 1U rackmountable low powered units >>> and they can also be wall mounted, real cool units for clusters for >>> starter's. >>> >>> http://www.ironsystems.com/products/iservers/aclass/a110_low_power.htm >>> for $499.. >> >> >> >> I had not read of the Via chips and had been planning to use the AMD >> Athlon >> chips. Are the via chips ok? Will they run linux/mosix ok? Any specific >> thoughts about them? >> >> Mike >> _______________________________________________ >> Beowulf mailing list, Beowulf at beowulf.org >> To change your subscription (digest mode or unsubscribe) visit >> http://www.beowulf.org/mailman/listinfo/beowulf > > _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From gerry.creager at tamu.edu Sun Jun 1 23:59:31 2003 From: gerry.creager at tamu.edu (Gerry Creager N5JXS) Date: Sun, 01 Jun 2003 22:59:31 -0500 Subject: Cheap PCs from Wal-Mart In-Reply-To: <20030601221802.A16442@mikee.ath.cx> References: <20030601094712.C11031@mikee.ath.cx> <20030601181353.B25006@www2> <20030601221802.A16442@mikee.ath.cx> Message-ID: <3EDACBA3.6000501@tamu.edu> Mosix/web service, maybe. Not for MY database needs, though, although, now that you mention it, I may have to try! gerry Mike Eggleston wrote: > On Sun, 01 Jun 2003, Bob Drzyzgula wrote: > > >>On Sun, Jun 01, 2003 at 04:56:02PM -0400, Mark Hahn wrote: >> >>>>I had not read of the Via chips and had been planning to use the AMD Athlon >>>>chips. Are the via chips ok? Will they run linux/mosix ok? Any specific >>>>thoughts about them? >>> >>>they're fairly slow, from what I've read, and still use the P5 pinout >>>(which means you'll peak at around 500 MB/s). the chip is optimized >>>for area/power/price first, not performance - VIA's been pushing it for >>>use in embedded/appliance kinds of uses. >> >>Actually, the lastest Via Cyrix/Centaur C3 chips use a Socket 370 interface, >>and support SSE. One thing that made them particularly slow until >>recently was that they ran the FPU at half the core speed; the 1GHz >>unit now runs it at full speed. >> >>Still, the C3 is probably a bad choice for a computationally intensive >>application. > > > Ok, but what about something like a cluster with mosix in front > and some database servers in back? Or web servers in front, or email > servers.... this sort of thing? -- Gerry Creager -- gerry.creager at tamu.edu Network Engineering -- AATLT, Texas A&M University Cell: 979.229.5301 Office: 979.458.4020 FAX: 979.847.8578 Page: 979.228.0173 Office: 903A Eller Bldg, TAMU, College Station, TX 77843 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From samhdaniel at earthlink.net Mon Jun 2 11:10:29 2003 From: samhdaniel at earthlink.net (Sam Daniel) Date: 02 Jun 2003 11:10:29 -0400 Subject: Cheap PCs from Wal-Mart In-Reply-To: <5.2.1.1.0.20030529163946.01d00a58@mail.toaster.net> References: <5.2.0.9.2.20030529152525.02e8cb68@mailhost4.jpl.nasa.gov> <5.2.1.1.0.20030529163946.01d00a58@mail.toaster.net> Message-ID: <1054566629.4082.1.camel@wulf> The only problem with these is that the C3-933 runs all floating point at half the processor speed. If you're trying to really solve a problem, that could be a big drawback. For a home system, it wouldn't be so bad. Unless you really need the 1U form factor, you can beat the price by nearly 50% by putting together the same hardware in a mini-atx or mid-tower case of your own choosing. VIA recently released the C3-1GHz with a new core, called "Nehemiah", which is supposed to run the FPU at full speed, but I haven't seen confirmation of that in any tests yet. The cost will only be $10-$20 more than the C3-933. Using the new processor will make these systems a lot more attractive, with even lower power consumption than the C3-933. Sam On Thu, 2003-05-29 at 19:51, Robert wrote: > Hello > > these guys make small foot print 1U rackmountable low powered units and > they can also be wall mounted, real cool units for clusters for starter's. > http://www.ironsystems.com/products/iservers/aclass/a110_low_power.htm for > $499.. > > Rob > > > At 04:19 PM 5/29/2003 -0700, Alvin Oga wrote: > > >hi ya > > > >cheap PCs can be gotten almost anywhere ??? doesnt have to be > >walmart/circuit city/emachines/etc > > > >$ 30 cheap pc case ( that makes the PC their widget ) > >$ 70 generic motherboard w/ onboard nic, onboard svga > >$ 70 Celeron-1.7G 478pin fsb400 cpu > >$ 25 128MB pc-133 > >$ 25 50x cdrom > >$ 60 20GB ide disk > >---- ------------- > >$ 280 grand total > > > >$ 25 oem ms license > > > >mb, cpu, disks can be lot lower in $$ if you use p3 and pc-133 meory > > > >via series mb w/ p3-800 is about $85 total ( subtract ~ $60 from above ) > > > >same cost estimates for amd duron/athlon based systems > > > >you can save the shipping by bying locally... > >and might be zero sales tax in some states too > > > >stuff all that into a 1U chassis and add $100 - $250 extra ... > >and take out the cost of the "generic midtower case" > > > >and if there's a problem w/ the pc, i'd hate to worry about how to return > >it and get a better box back or is it, as typically the case, > >that they'd simply send out a different returned PC .. since its a > >warranty replacement, they dont have to send you a brand new pc > >like they would have to with a new order > > > >On Thu, 29 May 2003, Jim Lux wrote: > > > > > For those of you looking to build a cluster on the (real) cheap, Walmart > > > has mailorder PCs, with Lindows (a Linux variant) installed for $200 (plus > > > sales tax and shipping, of course). > > > > > > I just bought one of these for my daughter (with WinXP, for $300.. I guess > > > the MS license is $100) and while it's no ball of fire, and the keyboard > > > and mouse are what you'd expect for a $200 computer, it DOES work ok..at > > > >magic ! > > > >have fun > >alvin > > > >_______________________________________________ > >Beowulf mailing list, Beowulf at beowulf.org > >To change your subscription (digest mode or unsubscribe) visit > >http://www.beowulf.org/mailman/listinfo/beowulf > > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org > To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From hahn at physics.mcmaster.ca Mon Jun 2 13:08:40 2003 From: hahn at physics.mcmaster.ca (Mark Hahn) Date: Mon, 2 Jun 2003 13:08:40 -0400 (EDT) Subject: Gigabit performance issues and NFS In-Reply-To: <5.0.2.1.2.20030602080851.02ced0f0@pop.larc.nasa.gov> Message-ID: > ping before: > ----Linux-g.localdomain PING Statistics---- > 5 packets transmitted, 5 packets received, 0.0% packet loss > round-trip min/avg/max = 0.219/0.286/0.427 ms > > ----Linux-g.localdomain PING Statistics---- > 22889 packets transmitted, 22888 packets received, 0.0% packet loss > round-trip min/avg/max = 0.202/0.476/2.307 ms > 2072.5 packets/sec sent, 2072.5 packets/sec received > > --- SGI-g.localdomain ping statistics --- > 13 packets transmitted, 13 received, 0% loss, time 11993ms > rtt min/avg/max/mdev = 0.108/0.178/0.230/0.036 ms > /ping before uh, that sucks! here's what I see on a pair of gbe-connected nodes (dual-xeon/2400, i7500, e1000) on an SMC SMC8624T: | [hahn at cat48 hahn]$ ping -c100 -q cat47 | PING cat47 (10.0.0.47) from 10.0.0.48 : 56(84) bytes of data. | | --- cat47 ping statistics --- | 100 packets transmitted, 100 received, 0% loss, time 98998ms | rtt min/avg/max/mdev = 0.034/0.035/0.046/0.004 ms and here's two nodes on a (3com 48+2) 100bT switch: | [root at cat8 root]# ping -c100 -q cat2 | PING cat2 (10.0.0.2) from 10.0.0.8 : 56(84) bytes of data. | | --- cat2 ping statistics --- | 100 packets transmitted, 100 received, 0% loss, time 98996ms | rtt min/avg/max/mdev = 0.099/0.101/0.241/0.016 ms in other words, your network appears to be rather sluggish, unless I've misunderstood your ping numbers. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From josip at lanl.gov Mon Jun 2 13:21:02 2003 From: josip at lanl.gov (Josip Loncaric) Date: Mon, 02 Jun 2003 11:21:02 -0600 Subject: Gigabit performance issues and NFS In-Reply-To: <5.0.2.1.2.20030602080851.02ced0f0@pop.larc.nasa.gov> References: <5.0.2.1.2.20030529092240.02aece48@pop.larc.nasa.gov> <5.0.2.1.2.20030529092240.02aece48@pop.larc.nasa.gov> <5.0.2.1.2.20030602080851.02ced0f0@pop.larc.nasa.gov> Message-ID: <3EDB877E.7060308@lanl.gov> Doug, In addition to NFS tuning suggested by Jakob, you may want to check your gigabit ethernet driver parameters. This was the key problem we identified on Coral, our cluster at ICASE (my old LaRC home :-), and the fix was to increase the interrupt mitigation parameters for the acenic device driver. The reason is this: NFS in Linux uses UDP, and it takes 6 normal UDP frames (unless you are using jumbo frames) to transfer one 8KB NFS block (a typical NFS tuning choice). If the NFS server gets interrupted more than every 6th frame, chances are that some frames will be lost. If any of the 6 frames is lost, all 6 have to be re-sent, leading to further losses, etc. This can collapse NFS performance by 100:1. We found that the following line in /etc/modules.conf restored reasonable (but not great) NFS performance: options acenic tx_coal_tick=75 rx_coal_tick=75 max_tx_desc=32 max_rx_desc=6 The key bit is that last parameter "max_rx_desc=6" but the others help as well. Optimal choices may be hardware dependent, but the above is a good starting point for the acenic device driver. On the other hand, your NFS performance results are fairly typical. We found that acenic-based Gigabit Ethernet can deliver ~72% of its rated performance only under rare circumstances and with special effort. Normally, we got FTP performance of about 28 MB/s and NFS performance of about 13 MB/s. Sincerely, Josip _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From luis.licon at yakko.cimav.edu.mx Mon Jun 2 13:56:11 2003 From: luis.licon at yakko.cimav.edu.mx (Luis Fernando Licon Padilla) Date: Mon, 02 Jun 2003 11:56:11 -0600 Subject: ClusterKnoppix References: <1054534705.15377.8.camel@harwood.home> Message-ID: <3EDB8FBB.5060907@yakko.cimav.edu.mx> John Hearns wrote: >I saw this on Slashdot a few days ago. > >It is a customised Knoppix distribution, with an OpenMosix patched >kernel, and PXE/DHCP enabled etc. >I intend to try this out in the next few days. > >For those who haven't used Knoppix, it is a CD which contains a bootable >Debian-based distribution which installs itself in memory, not touching >the hard drive. Knoppix has excellent hardware detection, and sets up >a good desktop. It 'just works'. >Uses might include demonstrating a Linux desktop to friends/colleagues, >testing if a PC can run Linux before buying/accepting it, or as a rescue >disk for troubleshooting an already installed Linux system. > >John Hearns > >_______________________________________________ >Beowulf mailing list, Beowulf at beowulf.org >To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf > > > > Hi you all guys, I have a 12 dual nodes beowulf running on Red Hat Linux 8.0, but you know, I have the update problem since I do not want to pay the update service to Red Hat, so my question is, Does someone have experience about clusters running on both Debian and RedHat distributions, who can help me to decide on this issue? Thanks, Luis -- ISC Luis Fernando Licon Padilla Advanced Materials Research Center Miguel de Cervantes 120 Complejo Industrial Chihuahua C.P.31109 Chihuahua, Chih. Mexico Phone: 52 (614)4391154 Fax: 52 (614)4391112 alternative e-mail: lordsirion2002 at yahoo.com _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From hahn at physics.mcmaster.ca Mon Jun 2 14:42:32 2003 From: hahn at physics.mcmaster.ca (Mark Hahn) Date: Mon, 2 Jun 2003 14:42:32 -0400 (EDT) Subject: Cheap PCs from Wal-Mart In-Reply-To: <002401c3291b$67a52d80$8461cdc2@DREAD> Message-ID: > re. the points on floating point performance, these are valid. > But I think we should note the new Nehemiah boards which are just out. > Well worth looking at I think - I intend to get one soon, and I'll report > back if/when I do. > http://www.mini-itx.com/reviews/nehemiah/?page=10#s21 hmm. for rhetorical purposes, let's compare the numbers from this article to two high-end unis: dhry whet mmi mmf memi memf e800 1048 285 963 1588 194 208 e10k 1300 351 1193 1968 233 245 e10k-n 1591 366 2255 2285 664 389 p4 6809 9327 22170 13896 5050 5041 ath 3319 8855 13011 12217 2912 3080 I think it's pretty clear that you need to expect much lower performance from even the 'high-end' VIA chips. if your code more resembles dhrystone (mostly integer, cache friendly), then you can mostly expect to scale with clock speed, and the VIA chips might be attractive on a speed/(heat*cost) basis. for general clusters, where memory bandwidth and FP performance and integrated gigabit are big advantages, VIA doesn't compete. data is from the article above and http://www6.tomshardware.com/cpu/20030521/index.html _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rocky at atipa.com Mon Jun 2 14:09:34 2003 From: rocky at atipa.com (Rocky McGaugh) Date: Mon, 2 Jun 2003 13:09:34 -0500 (CDT) Subject: ClusterKnoppix In-Reply-To: <3EDB8FBB.5060907@yakko.cimav.edu.mx> Message-ID: On Mon, 2 Jun 2003, Luis Fernando Licon Padilla wrote: > Hi you all guys, > > I have a 12 dual nodes beowulf running on Red Hat Linux 8.0, but you > know, I have the update problem since I do not want to pay the update > service to Red Hat, so my question is, Does someone have experience > about clusters running on both Debian and RedHat distributions, who can > help me to decide on this issue? > > Thanks, > > Luis for redhat, yum is the best thing since sliced bread. http://linux.duke.edu/projects/yum/ -- Rocky McGaugh Atipa Technologies rocky at atipatechnologies.com rmcgaugh at atipa.com 1-785-841-9513 x3110 http://67.8450073/ perl -e 'print unpack(u, ".=W=W+F%T:7\!A+F-O;0H`");' _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From mikee at mikee.ath.cx Mon Jun 2 15:42:23 2003 From: mikee at mikee.ath.cx (Mike Eggleston) Date: Mon, 2 Jun 2003 14:42:23 -0500 Subject: ClusterKnoppix In-Reply-To: <3EDB8FBB.5060907@yakko.cimav.edu.mx>; from luis.licon@yakko.cimav.edu.mx on Mon, Jun 02, 2003 at 11:56:11AM -0600 References: <1054534705.15377.8.camel@harwood.home> <3EDB8FBB.5060907@yakko.cimav.edu.mx> Message-ID: <20030602144223.L17733@mikee.ath.cx> On Mon, 02 Jun 2003, Luis Fernando Licon Padilla wrote: > John Hearns wrote: > > >I saw this on Slashdot a few days ago. > > > >It is a customised Knoppix distribution, with an OpenMosix patched > >kernel, and PXE/DHCP enabled etc. > >I intend to try this out in the next few days. > > > >For those who haven't used Knoppix, it is a CD which contains a bootable > >Debian-based distribution which installs itself in memory, not touching > >the hard drive. Knoppix has excellent hardware detection, and sets up > >a good desktop. It 'just works'. > >Uses might include demonstrating a Linux desktop to friends/colleagues, > >testing if a PC can run Linux before buying/accepting it, or as a rescue > >disk for troubleshooting an already installed Linux system. > > > >John Hearns > > > >_______________________________________________ > >Beowulf mailing list, Beowulf at beowulf.org > >To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf > > > > > > > > > Hi you all guys, > > I have a 12 dual nodes beowulf running on Red Hat Linux 8.0, but you > know, I have the update problem since I do not want to pay the update > service to Red Hat, so my question is, Does someone have experience > about clusters running on both Debian and RedHat distributions, who can > help me to decide on this issue? Have you tried using apt? It seems to run a day or two behind the RedHat updates, but it gets all the updates. Try setting a nightly cron entry to execute 'apt-get update; apt-get update' for all your nodes. You can find apt at www.freshrpms.net. Mike _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From Robin.Laing at drdc-rddc.gc.ca Mon Jun 2 17:25:33 2003 From: Robin.Laing at drdc-rddc.gc.ca (Robin Laing) Date: Mon, 02 Jun 2003 15:25:33 -0600 Subject: Help: different kernel images for slaves. Message-ID: <3EDBC0CD.6090109@drdc-rddc.gc.ca> Hello, I am involved with adding some SMP machines to our small cluster and I am trying to find out how I tell the new machines to run the SMP kernel instead of the default in the /var/beowulf/config file. We are running RH 9 and BPpoc. Our present system is working with single processor boot image and the SMP machines will boot with only one processor working. I have compiled a SMP kernel to test but I cannot just change the boot kernel for all the other machines. Presently in /var/beowulf are: boot.img nodeboot.img nodeinfo I want to add boot_smp.img I have not seen any documentation on configuring /etc/beowulf/config to work with more than one boot image. Is it possible to define which nodes boot with which kernel? -- Robin Laing Instrumentation Technologist Voice: 1.403.544.4762 Military Engineering Section FAX: 1.403.544.4704 Defence R&D Canada - Suffield Email: Robin.Laing at DRDC-RDDC.gc.ca PO Box 4000, Station Main WWW: http://www.suffield.drdc-rddc.gc.ca Medicine Hat, AB, T1A 8K6 Canada _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From math at velocet.ca Mon Jun 2 18:22:34 2003 From: math at velocet.ca (Ken Chase) Date: Mon, 2 Jun 2003 18:22:34 -0400 Subject: Cheap PCs from Wal-Mart In-Reply-To: ; from alvin@Maggie.Linux-Consulting.com on Mon, Jun 02, 2003 at 02:18:06PM -0700 References: Message-ID: <20030602182234.E81856@velocet.ca> On Mon, Jun 02, 2003 at 02:18:06PM -0700, Alvin Oga's all... >> hmm. for rhetorical purposes, let's compare the numbers from this article >> to two high-end unis: >> >> dhry whet mmi mmf memi memf >> e800 1048 285 963 1588 194 208 >> e10k 1300 351 1193 1968 233 245 >> e10k-n 1591 366 2255 2285 664 389 >> p4 6809 9327 22170 13896 5050 5041 >> ath 3319 8855 13011 12217 2912 3080 > >I wonder what the specific details was for the aove p4/ath cpus. > >For the $$$ difference betwen a p4/amd and c3 ... it seems >like a non-issue for performance of spending the extra $50 - $100 >for better performance in the above benchmark numbers > >( but, its a bigger price/performance gap for the lastest FSB-800 cpus > and runs $300 - $500 or higher > >good stuff in either case ... Since power & cooling is almost always free for .edu's, yes, .edu cluster design and others' will be extremely different. Not to mention that .edu's cant stand losing nodes in a throwaway design that accomodates tolerable rates of failure. (Anyone operating a cluster in this mode? Im very curious as to how you surmounted the psychological objections to it!) When you say 'TCO' you REALLY have to define what you mean -- "total cost of OWNERSHIP" or "total cost of OPERATION"? You can own a really nice cluster for $x if you dont have to cut $x into slices to buy it AND pay for running power and cooling for it for n years. Anyone admit to being in the 'must pay for power and other operationals' camp and want to explain how their designs differ? (especially those that pay a premium for floor space - these edens start winning on two fronts. Low heat = low space requirements.) I suspect EDENs will be viable for people in the expensive power & floorspace areas (Bay area, etc?) /kc -- Ken Chase, math at velocet.ca * Velocet Communications Inc. * Toronto, Canada Wiznet Velocet DSL.ca Datavaults 24/7: 416-967-4414 tollfree: 1-866-353-0363 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From James.P.Lux at jpl.nasa.gov Mon Jun 2 18:25:47 2003 From: James.P.Lux at jpl.nasa.gov (Jim Lux) Date: Mon, 02 Jun 2003 15:25:47 -0700 Subject: Cheap PCs from Wal-Mart References: <002401c3291b$67a52d80$8461cdc2@DREAD> Message-ID: <5.2.0.9.2.20030602150917.02f2ba88@mailhost4.jpl.nasa.gov> Which P4 and Athlon were you comparing.. I went to the linked page at Toms Hardware and found reasonably close results Athlon XP 3200+, and P4 3GHz... Now, what's the power consumption of those two? I think the P4 dissipates a max of 85W (poking around on the same Tom's page) The C3 at 800 MHz dissipates 6.65W average and peaks at 12W The Nehemiah at 1 GHz dissipates 11.25/15 W ave/pk So, looking at the scaling and comparing P4 against Via C3 Nehemiah Speed 6809/1591 4.3:1 Power 85/15 5.7:1 So the Via looks like it does more computes/Joule... This is really a first order approximation. You'd have to look at peripheral power, memory power, and power supply conversion and power distribution efficiency. Peripherals and memory probably scale with memory speed fairly linearly. The higher powered device will have more I^2R losses in the power supply and distribution. (Supplying 50 or so Amps at 2-3V is no easy feat) Here's the upshot... if you're in an environment where ultimate performance is important, than you're probably best going with as few big fast processors as you can get (the original Cray paradigm). On the other hand, if you're in an environment where maximizing computation for a fixed amount of power/heat dissipation resources (i.e. spaceflight or portable operation), then things like the Via C3 start to look attractive, assuming your task scales well. Since I'm generally interested in resource(other than dollars) constrained computing, I'd like to see more attention paid to computing per joule or computing rate/watt. There are different tradeoffs whether you are concerned about total energy (i.e. you're running off a battery) or whether you have an instantaneous power dissipation limit (i.e. thermal radiators in space). (that is, you can decide, do I want to run the calculation in one hour at 200W, or over two hours at 100W) At 02:42 PM 6/2/2003 -0400, Mark Hahn wrote: > > re. the points on floating point performance, these are valid. > > But I think we should note the new Nehemiah boards which are just out. > > Well worth looking at I think - I intend to get one soon, and I'll report > > back if/when I do. > > http://www.mini-itx.com/reviews/nehemiah/?page=10#s21 > >hmm. for rhetorical purposes, let's compare the numbers from this article >to two high-end unis: > > dhry whet mmi mmf memi memf >e800 1048 285 963 1588 194 208 >e10k 1300 351 1193 1968 233 245 >e10k-n 1591 366 2255 2285 664 389 >p4 6809 9327 22170 13896 5050 5041 >ath 3319 8855 13011 12217 2912 3080 > >I think it's pretty clear that you need to expect much lower >performance from even the 'high-end' VIA chips. if your code >more resembles dhrystone (mostly integer, cache friendly), >then you can mostly expect to scale with clock speed, and the >VIA chips might be attractive on a speed/(heat*cost) basis. > >for general clusters, where memory bandwidth and FP performance >and integrated gigabit are big advantages, VIA doesn't compete. > >data is from the article above and > http://www6.tomshardware.com/cpu/20030521/index.html > >__ James Lux, P.E. Spacecraft Telecommunications Section Jet Propulsion Laboratory, Mail Stop 161-213 4800 Oak Grove Drive Pasadena CA 91109 tel: (818)354-2075 fax: (818)393-6875 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From josip at lanl.gov Mon Jun 2 18:39:54 2003 From: josip at lanl.gov (Josip Loncaric) Date: Mon, 02 Jun 2003 16:39:54 -0600 Subject: Cheap PCs from Wal-Mart In-Reply-To: References: Message-ID: <3EDBD23A.7020508@lanl.gov> Mark Hahn wrote: > > data is from the article above and > http://www6.tomshardware.com/cpu/20030521/index.html ... which leads to the following very interesting observation: "Our lab engineers made a curious discovery when they were examining the new 865PE board from MSI. By incorporating special logic circuitry, the manufacturer has succeeded in boosting the speed of the CPU through dynamic overclocking in such a way that it is not detectable with conventional benchmarking utilities like WCPUID, Intel Frequency Display, CPUZ, or SiSoft Sandra 2003. However, FSB and CPU speeds are only increased when applications are started or when benchmark programs have finished - subject to CPU usage reaching close to 100 percent." (see http://www6.tomshardware.com/cpu/20030521/800fsb-25.html) Neat... but beware of unintended consequences, e.g. variable OS event timing... Sincerely, Josip _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From alvin at Maggie.Linux-Consulting.com Mon Jun 2 18:54:37 2003 From: alvin at Maggie.Linux-Consulting.com (Alvin Oga) Date: Mon, 2 Jun 2003 15:54:37 -0700 (PDT) Subject: Cheap PCs from Wal-Mart - mini-itx - 12v In-Reply-To: <20030602182436.F81856@velocet.ca> Message-ID: hi ya ken On Mon, 2 Jun 2003, Ken Chase wrote: > >we did lots of testing on that 12v dc-dc input to the mini-itx mb > >( well, www.mini-box.com folks did all the testing and ordering of the > >( custom 12v dc-to-dc atx power supply these tests were for 1-z 2-z systems .. one dc-to-dc ps for each mini-itx motherboard in the stackable case-size of a standard cdrom > Any numbers on the PF of these ac-to-dv (then to dc) floor-brick-with- > power-cables xformers? I suspect this stuff isnt worth anything til you > run it off a site-wide 12V infrastructure to get a good PF rating. or one can operate like a colo ?? - take 110VAC in, convert to 12V dc batteries, generate 110VAC to powerup the colo'd servers or 48V dc ( like the phone systems ) have fun alvin _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From joelja at darkwing.uoregon.edu Mon Jun 2 18:48:22 2003 From: joelja at darkwing.uoregon.edu (Joel Jaeggli) Date: Mon, 2 Jun 2003 15:48:22 -0700 (PDT) Subject: Cheap PCs from Wal-Mart In-Reply-To: <20030602182234.E81856@velocet.ca> Message-ID: On Mon, 2 Jun 2003, Ken Chase wrote: > On Mon, Jun 02, 2003 at 02:18:06PM -0700, Alvin Oga's all... > >> hmm. for rhetorical purposes, let's compare the numbers from this article > >> to two high-end unis: > >> > >> dhry whet mmi mmf memi memf > >> e800 1048 285 963 1588 194 208 > >> e10k 1300 351 1193 1968 233 245 > >> e10k-n 1591 366 2255 2285 664 389 > >> p4 6809 9327 22170 13896 5050 5041 > >> ath 3319 8855 13011 12217 2912 3080 > > > >I wonder what the specific details was for the aove p4/ath cpus. > > > >For the $$$ difference betwen a p4/amd and c3 ... it seems > >like a non-issue for performance of spending the extra $50 - $100 > >for better performance in the above benchmark numbers > > > >( but, its a bigger price/performance gap for the lastest FSB-800 cpus > > and runs $300 - $500 or higher > > > >good stuff in either case ... > > Since power & cooling is almost always free for .edu's, yes, .edu cluster > design and others' will be extremely different. I'm not entirely clear on the compatibility of the phrase "Since power & cooling is almost always free for .edu's," and the phrase "massive chiller upgrade required". clusters especially large ones have real costs associated with power, space, and cooling that can't been hidden in your general budget... > Not to mention that .edu's cant stand losing nodes in a throwaway > design that accomodates tolerable rates of failure. (Anyone operating > a cluster in this mode? Im very curious as to how you surmounted the > psychological objections to it!) > > When you say 'TCO' you REALLY have to define what you mean -- "total cost > of OWNERSHIP" or "total cost of OPERATION"? You can own a really nice > cluster for $x if you dont have to cut $x into slices to buy it > AND pay for running power and cooling for it for n years. > > Anyone admit to being in the 'must pay for power and other operationals' > camp and want to explain how their designs differ? (especially those > that pay a premium for floor space - these edens start winning on two > fronts. Low heat = low space requirements.) > > I suspect EDENs will be viable for people in the expensive power & floorspace > areas (Bay area, etc?) > > /kc > -- -------------------------------------------------------------------------- Joel Jaeggli Academic User Services joelja at darkwing.uoregon.edu -- PGP Key Fingerprint: 1DE9 8FCA 51FB 4195 B42A 9C32 A30D 121E -- In Dr. Johnson's famous dictionary patriotism is defined as the last resort of the scoundrel. With all due respect to an enlightened but inferior lexicographer I beg to submit that it is the first. -- Ambrose Bierce, "The Devil's Dictionary" _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From math at velocet.ca Mon Jun 2 18:24:36 2003 From: math at velocet.ca (Ken Chase) Date: Mon, 2 Jun 2003 18:24:36 -0400 Subject: Cheap PCs from Wal-Mart - mini-itx In-Reply-To: ; from alvin@Maggie.Linux-Consulting.com on Mon, Jun 02, 2003 at 12:02:24AM -0700 References: <20030601132552.D11031@mikee.ath.cx> Message-ID: <20030602182436.F81856@velocet.ca> On Mon, Jun 02, 2003 at 12:02:24AM -0700, Alvin Oga's all... > >hi ya > >On Sun, 1 Jun 2003, Mike Eggleston wrote: > >> On Sun, 01 Jun 2003, Gerry Creager N5JXS wrote: >> >> > I've got a Via Epia Mini-ITX board I've gotten for another project. We >> > loaded RH 9 via PXE-Boot (with particular thanks to RGB!), as in, a >> > normal install rather than diskless boot. Works great. I've got the >> > slowest, equivalent to a 600 MHz Celeron, but that's all I needed for >> > this project... in fact, overkill. >> > >> > Runs pretty cool, even with just the 2 puny little case fans, and can be >> > driven off a 12v Wall Wart (?tm?; should I trade mark it? no...) at >> > less than 5a. The wart I have is rated for 4.5a, but I've not measured >> > it yet. > >we did lots of testing on that 12v dc-dc input to the mini-itx mb >( well, www.mini-box.com folks did all the testing and ordering of the >( custom 12v dc-to-dc atx power supply Any numbers on the PF of these ac-to-dv (then to dc) floor-brick-with- power-cables xformers? I suspect this stuff isnt worth anything til you run it off a site-wide 12V infrastructure to get a good PF rating. /kc -- Ken Chase, math at velocet.ca * Velocet Communications Inc. * Toronto, Canada Wiznet Velocet DSL.ca Datavaults 24/7: 416-967-4414 tollfree: 1-866-353-0363 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From alvin at Maggie.Linux-Consulting.com Mon Jun 2 17:18:06 2003 From: alvin at Maggie.Linux-Consulting.com (Alvin Oga) Date: Mon, 2 Jun 2003 14:18:06 -0700 (PDT) Subject: Cheap PCs from Wal-Mart In-Reply-To: Message-ID: hi ya On Mon, 2 Jun 2003, Mark Hahn wrote: > > re. the points on floating point performance, these are valid. > > But I think we should note the new Nehemiah boards which are just out. > > Well worth looking at I think - I intend to get one soon, and I'll report > > back if/when I do. > > http://www.mini-itx.com/reviews/nehemiah/?page=10#s21 > > hmm. for rhetorical purposes, let's compare the numbers from this article > to two high-end unis: > > dhry whet mmi mmf memi memf > e800 1048 285 963 1588 194 208 > e10k 1300 351 1193 1968 233 245 > e10k-n 1591 366 2255 2285 664 389 > p4 6809 9327 22170 13896 5050 5041 > ath 3319 8855 13011 12217 2912 3080 I wonder what the specific details was for the aove p4/ath cpus. For the $$$ difference betwen a p4/amd and c3 ... it seems like a non-issue for performance of spending the extra $50 - $100 for better performance in the above benchmark numbers ( but, its a bigger price/performance gap for the lastest FSB-800 cpus and runs $300 - $500 or higher good stuff in either case ... have fun alvin > I think it's pretty clear that you need to expect much lower > performance from even the 'high-end' VIA chips. if your code > more resembles dhrystone (mostly integer, cache friendly), > then you can mostly expect to scale with clock speed, and the > VIA chips might be attractive on a speed/(heat*cost) basis. > > for general clusters, where memory bandwidth and FP performance > and integrated gigabit are big advantages, VIA doesn't compete. > > data is from the article above and > http://www6.tomshardware.com/cpu/20030521/index.html > > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org > To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf > _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From hahn at physics.mcmaster.ca Mon Jun 2 19:15:12 2003 From: hahn at physics.mcmaster.ca (Mark Hahn) Date: Mon, 2 Jun 2003 19:15:12 -0400 (EDT) Subject: Cheap PCs from Wal-Mart In-Reply-To: Message-ID: > > hmm. for rhetorical purposes, let's compare the numbers from this article > > to two high-end unis: > > > > dhry whet mmi mmf memi memf > > e800 1048 285 963 1588 194 208 > > e10k 1300 351 1193 1968 233 245 > > e10k-n 1591 366 2255 2285 664 389 > > p4 6809 9327 22170 13896 5050 5041 > > ath 3319 8855 13011 12217 2912 3080 > > I wonder what the specific details was for the aove p4/ath cpus. it's not interesting or relevant. for the record, by "high-end", I meant "highest end currently available". that is, P4, FSB800, i875, 2-3 GHz. the highest-end Athlon is significantly lower-end (single ddr400 I think, PR3200). > For the $$$ difference betwen a p4/amd and c3 ... it seems sure, high-end desktops are more expensive than integrated appliance boards, no surprise there. the point is that they're also dramatically faster. being much faster means you can't just ignore how slow the VIA chips are, at least not in this context (compute clusters). remember that speed advantages are often *multiplicative*, so you might get 4x speedup from CPU, 7x from dram, 2x from disk, 8x from gigabit for a total of 448x! guaranteed not to exceed, of course, but the principle is sound... > like a non-issue for performance of spending the extra $50 - $100 > for better performance in the above benchmark numbers AMD is lagging in uniprocessors, no question, no surprise. it's astonishing that a non-fly-by-night company could simply forget to deal with that little issue of motherboards. afaikt, there are hundreds, possibly thousands of shops worldwide who could produce a basic 6-layer motherboard without breaking a sweat... _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From math at velocet.ca Mon Jun 2 19:55:24 2003 From: math at velocet.ca (Ken Chase) Date: Mon, 2 Jun 2003 19:55:24 -0400 Subject: Cheap PCs from Wal-Mart In-Reply-To: ; from joelja@darkwing.uoregon.edu on Mon, Jun 02, 2003 at 03:48:22PM -0700 References: <20030602182234.E81856@velocet.ca> Message-ID: <20030602195524.K81856@velocet.ca> On Mon, Jun 02, 2003 at 03:48:22PM -0700, Joel Jaeggli's all... >On Mon, 2 Jun 2003, Ken Chase wrote: >> Since power & cooling is almost always free for .edu's, yes, .edu cluster >> design and others' will be extremely different. > >I'm not entirely clear on the compatibility of the phrase "Since power & >cooling is almost always free for .edu's," and the phrase "massive chiller >upgrade required". clusters especially large ones have real costs >associated with power, space, and cooling that can't been hidden in your >general budget... I was generalizing a bit too harshly, but two of the designs we've shipped out to universities were free from any considerations of power, cooling or total cost of operation. :) Kinda nice to be free of the more difficult aspects of design! However, other designs of clusters and similar systems (server farms) are pathologically encumbered with such considerations to the point that they're they major concern. Both situations give rise to radically different designs, and whoever was questioning the use of the Eden must not have encountered these requirements. /kc > >> Not to mention that .edu's cant stand losing nodes in a throwaway >> design that accomodates tolerable rates of failure. (Anyone operating >> a cluster in this mode? Im very curious as to how you surmounted the >> psychological objections to it!) >> >> When you say 'TCO' you REALLY have to define what you mean -- "total cost >> of OWNERSHIP" or "total cost of OPERATION"? You can own a really nice >> cluster for $x if you dont have to cut $x into slices to buy it >> AND pay for running power and cooling for it for n years. >> >> Anyone admit to being in the 'must pay for power and other operationals' >> camp and want to explain how their designs differ? (especially those >> that pay a premium for floor space - these edens start winning on two >> fronts. Low heat = low space requirements.) >> >> I suspect EDENs will be viable for people in the expensive power & floorspace >> areas (Bay area, etc?) >> >> /kc >> > >-- >-------------------------------------------------------------------------- >Joel Jaeggli Academic User Services joelja at darkwing.uoregon.edu >-- PGP Key Fingerprint: 1DE9 8FCA 51FB 4195 B42A 9C32 A30D 121E -- > In Dr. Johnson's famous dictionary patriotism is defined as the last > resort of the scoundrel. With all due respect to an enlightened but > inferior lexicographer I beg to submit that it is the first. > -- Ambrose Bierce, "The Devil's Dictionary" > -- Ken Chase, math at velocet.ca * Velocet Communications Inc. * Toronto, Canada Wiznet Velocet DSL.ca Datavaults 24/7: 416-967-4414 tollfree: 1-866-353-0363 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From hahn at physics.mcmaster.ca Mon Jun 2 19:36:29 2003 From: hahn at physics.mcmaster.ca (Mark Hahn) Date: Mon, 2 Jun 2003 19:36:29 -0400 (EDT) Subject: Cheap PCs from Wal-Mart In-Reply-To: <5.2.0.9.2.20030602150917.02f2ba88@mailhost4.jpl.nasa.gov> Message-ID: > I went to the linked page at Toms Hardware and found reasonably close results > Athlon XP 3200+, and P4 3GHz... right, as I said "high-end". > Now, what's the power consumption of those two? > I think the P4 dissipates a max of 85W (poking around on the same Tom's page) that would be max TDP. > The C3 at 800 MHz dissipates 6.65W average and peaks at 12W > The Nehemiah at 1 GHz dissipates 11.25/15 W ave/pk I suspect VIA's "peak" is not equivalent (lower) than Intel's TDP. > So, looking at the scaling and comparing P4 against Via C3 Nehemiah > Speed 6809/1591 4.3:1 > Power 85/15 5.7:1 > > So the Via looks like it does more computes/Joule... well, for dhrystone at least. which isn't really surprising if you think about the fact that dhry is entirely on-chip, and much of the power dissipated by a high-end CPU is actually the external interface. if you consider a *real* workload which has some external load, you'd see much higher throughput on the thick system. > This is really a first order approximation. You'd have to look at > peripheral power, memory power, and power supply conversion and power > distribution efficiency. Peripherals and memory probably scale with memory > speed fairly linearly. huh? do you mean "the P4 drives ram much faster and so the ram will also dissipate more power"? > The higher powered device will have more I^2R > losses in the power supply and distribution. (Supplying 50 or so Amps at > 2-3V is no easy feat) well, the PS supplies 3.3V; the Vcc for the CPU only has to travel a couple of inches, and that's probably in a plane. I don't really thing that's an issue. > Here's the upshot... if you're in an environment where ultimate performance > is important, than you're probably best going with as few big fast > processors as you can get (the original Cray paradigm). On the other hand, > if you're in an environment where maximizing computation for a fixed amount > of power/heat dissipation resources (i.e. spaceflight or portable > operation), then things like the Via C3 start to look attractive, assuming > your task scales well. almost tautological, it's so unobjectionable ;) my whole point was that people must not even start to think that these low-power CPUs are close in performance. and even if they were, I don't think people really have an intuitive sense for what "scales well" means - I doubt *anyone* has a task that is as undemanding of the system as dhrystone is, for instance. > computing, I'd like to see more attention paid to computing per joule or > computing rate/watt. There are different tradeoffs whether you are sure, but please, let's use specFPrate per watt. even that's a bit dated, since it's so cache-friendly. but the real question is: can you afford to use a cluster node which has, say, 10-20% of the performance? you can stretch Amdahl's law a bit and see that the further you push wimpy nodes, the smaller a problem domain you can address (requires ever looser coupled programs, longer latency of individual work-units, etc). _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From lindahl at keyresearch.com Mon Jun 2 20:52:23 2003 From: lindahl at keyresearch.com (Greg Lindahl) Date: Mon, 2 Jun 2003 17:52:23 -0700 Subject: Cheap PCs from Wal-Mart In-Reply-To: <20030602182234.E81856@velocet.ca> References: <20030602182234.E81856@velocet.ca> Message-ID: <20030603005223.GA2037@greglaptop.internal.keyresearch.com> On Mon, Jun 02, 2003 at 06:22:34PM -0400, Ken Chase wrote: > I suspect EDENs will be viable for people in the expensive power & floorspace > areas (Bay area, etc?) Us? Floorspace is less than 80 cents per month per square foot, in a high-quality machineroom. There's a huge glut. Office space is cheap, too. Now 2 years ago, neither was true... -- greg _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rgb at phy.duke.edu Mon Jun 2 22:23:47 2003 From: rgb at phy.duke.edu (Robert G. Brown) Date: Mon, 2 Jun 2003 22:23:47 -0400 (EDT) Subject: Cheap PCs from Wal-Mart In-Reply-To: Message-ID: On Mon, 2 Jun 2003, Mark Hahn wrote: > but the real question is: can you afford to use a cluster node which > has, say, 10-20% of the performance? you can stretch Amdahl's law a bit > and see that the further you push wimpy nodes, the smaller a problem > domain you can address (requires ever looser coupled programs, longer > latency of individual work-units, etc). Ya. I've refrained from this discussion because I'm busy and y'all've been doing so very well anyway, but I think Mark has it dead right. Really, truly cheap-o nodes are a lovely thing for (as was proposed in the original post long ago) a tiny $1K hobby/play cluster, or even for a toy cluster for a high school cluster laboratory where $2-3K is "a lot of money" for them to raise. The purpose of such a cluster is NOT to get some piece of numerical work done as cost-effectively as possible. It is to teach a generation of kids about managing linux systems, writing programs, and the rudiments of cluster computing (Amdahl's law and so forth) in an inexpensive environment that they can "play" with. It is for cluster humans like me to have cheap miniclusters at home to play with to try things out in an environment they can "break the hell out of" without interfering with their production environment. For real numerical work, the following argue against using the cheapest possible nodes: a) MFLOPS/$$ are generally not peak for the cheapest hardware, even in raw aggregate. I "usually" find that I can get the most work done for the least money one or two clock generations back from peak/bleeding edge. b) Mark's repeated observations on "good benchmarks" in terms of which to find the cost-benefit winner are also dead on. The best benchmark is (of course) the application(s) you plan to run. Lacking numbers for that application, you'll have to do your best to guestimate it from e.g. specfp, stream, and other benchmarks. Most people on this list will find raw integer benchmarks to be somewhat irrelevant, as HPC performance tends to be dominated by floating point operations. Then there is the usual list of rate-limiting bottlenecks: cpu, memory, network. c) Cheap nodes are cheap, and in a tanstaafl world will likely break early, break often. For a few nodes in a toy cluster with some sort of warranty, your aggregate risk may be low and you may have the energy to deal with failures that occur. For 128 nodes in a production cluster, a failure a week or more will soon drive you mad. d) Cheap slow nodes probably draw MORE power per unit of work done than do faster newer more expensive nodes. I say "probably" because I haven't done all of the measurements to be able to say this for a fact, but note that a number of items in a "typical computer" chassis produce a more or less invariant draw. You have to power the power supply itself, a floppy, a NIC, a hdd, a motherboard, a video card. Some of those things actually run quieter and draw less in more expensive versions. Others draw at a rate that scales with clock (although not necessarily linearly). Cooling goes with heat. Both cost money. For example, consider an imaginary node where overhead is 40 watts and CPU/memory at speed "1" are 40 more watts. To get to speed 2 with perfect scaling you can buy two nodes and draw 160 watts total. Or you can buy one node at speed 2, forget scaling, and even if the 2x cpu draws 80 watts, end up drawing only 120 watts. If the faster CPU has a marginal cost of less than the second entire system, you end up winning on raw dollars as well (one of the things that favors fast nodes as in a). e) Finally, cheap nodes are almost obsolete when you buy them, so it should come as no surprise that in a production environment they'll be obsolete in a lot less than a typical three year lifetime. I've read cost benefit studies that suggest that from a TCO perspective even fairly MODERN nodes should be replaced every 12 to 18 months (although this particular timeframe depends a bit on a variety of overhead and management costs). Cheap nodes shouldn't even be brought home. Moore's Law is inexorable and unforgiving, and you'll find that your entire $1000 "cheap cluster" may be replaceable by a single $1000 desktop system that is twice as fast INSIDE a year from when you bought it if you're not careful. So for hobby/home, sure, wal-mart specials, cheapest homemade, web-special boxes are fine. Hell, I still run nodes at home as slow as 400 MHz Celerons -- but they aren't "production", they are for fun. At Duke I buy much more expensive, much more reliable, much faster systems, and curse the gods when even THEY break down from time to time. rgb -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb at phy.duke.edu _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From math at velocet.ca Mon Jun 2 22:40:09 2003 From: math at velocet.ca (Ken Chase) Date: Mon, 2 Jun 2003 22:40:09 -0400 Subject: Cheap PCs from Wal-Mart In-Reply-To: <5.2.0.9.2.20030602150917.02f2ba88@mailhost4.jpl.nasa.gov>; from James.P.Lux@jpl.nasa.gov on Mon, Jun 02, 2003 at 03:25:47PM -0700 References: <002401c3291b$67a52d80$8461cdc2@DREAD> <5.2.0.9.2.20030602150917.02f2ba88@mailhost4.jpl.nasa.gov> Message-ID: <20030602224009.L81856@velocet.ca> On Mon, Jun 02, 2003 at 03:25:47PM -0700, Jim Lux's all... >Which P4 and Athlon were you comparing.. >I went to the linked page at Toms Hardware and found reasonably close results >Athlon XP 3200+, and P4 3GHz... > >Now, what's the power consumption of those two? >I think the P4 dissipates a max of 85W (poking around on the same Tom's page) > >The C3 at 800 MHz dissipates 6.65W average and peaks at 12W >The Nehemiah at 1 GHz dissipates 11.25/15 W ave/pk > >So, looking at the scaling and comparing P4 against Via C3 Nehemiah >Speed 6809/1591 4.3:1 >Power 85/15 5.7:1 I found slightly different numbers for the C3-Ezra-T 800: (mind the spaces) http://www.via.com.tw/en/viac3/VIA C3 Ezra-T datasheet v1.0 .pdf taking this to its pathological conclusion: C3 Ezra T-800 C3 Nehemiah 1Ghz P4 ------------------------------------------------------------------------------ heat dissip. W 5.0/8.5pk 11.25av/15pk 85 (peak or avg?) dhrystones d 1048 1591 6809 d/W 209/123 141/106 80 ratio 2.6 / 1.5 1.8 / 1.3 1 Now I dont know which power number to choose for the C3's - the power ratings are listed as "average power dissipated while running Winstone 99 on Win98" in the PDF. And there's no numbers there for the E10k-n (the pdf link was off a June 2002 article from Tom's HW) for avg/peak power. These numbers dont really mean that much, except in a very general sense. Specific benchmarks and direct power usage readings would be much more useful, considering: - If you add a HD to the setup per CPU, you blow your total W/node figure (not to mention your price curve). - I dont know what dhrystones are an accurate measure of what everyone is doing with clusters these days - I dont know if what people do with clusters excercises average or peak power consumption for any CPU (or for some and not others). (I do know that burnMMX and burnK7 running together would blow our breakers for dual Tbird 1.3Ghz on Tyan 2460s with no HDs on 300W enermax power supplies and 512Mb ram at 8 boards/15 Amps, where as running G98 or gromacs code did not, so what's running is relevant.) Furthermore, it is to be noted that the E10K-n does require active cooling, changing the amount of space required ('volume' for airflow must be included) - though Im guessing that in a 'blade' type setup or with inline airflow over big heatsinks, you could avoid fans onboard. For computation power vs Watts, we should really be looking at the new low-power celerons, I bet they have some figures competitive to the C3. /kc >> >> dhry whet mmi mmf memi memf >>e800 1048 285 963 1588 194 208 >>e10k 1300 351 1193 1968 233 245 >>e10k-n 1591 366 2255 2285 664 389 >>p4 6809 9327 22170 13896 5050 5041 >>ath 3319 8855 13011 12217 2912 3080 >> >>I think it's pretty clear that you need to expect much lower >>performance from even the 'high-end' VIA chips. if your code >>more resembles dhrystone (mostly integer, cache friendly), >>then you can mostly expect to scale with clock speed, and the >>VIA chips might be attractive on a speed/(heat*cost) basis. >> >>for general clusters, where memory bandwidth and FP performance >>and integrated gigabit are big advantages, VIA doesn't compete. >> >>data is from the article above and >> http://www6.tomshardware.com/cpu/20030521/index.html >> >>__ > >James Lux, P.E. >Spacecraft Telecommunications Section >Jet Propulsion Laboratory, Mail Stop 161-213 >4800 Oak Grove Drive >Pasadena CA 91109 >tel: (818)354-2075 >fax: (818)393-6875 > >_______________________________________________ >Beowulf mailing list, Beowulf at beowulf.org >To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- Ken Chase, math at velocet.ca * Velocet Communications Inc. * Toronto, Canada Wiznet Velocet DSL.ca Datavaults 24/7: 416-967-4414 tollfree: 1-866-353-0363 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From math at velocet.ca Mon Jun 2 22:54:12 2003 From: math at velocet.ca (Ken Chase) Date: Mon, 2 Jun 2003 22:54:12 -0400 Subject: Cheap PCs from Wal-Mart - mini-itx - 12v In-Reply-To: ; from alvin@Maggie.Linux-Consulting.com on Mon, Jun 02, 2003 at 03:54:37PM -0700 References: <20030602182436.F81856@velocet.ca> Message-ID: <20030602225412.M81856@velocet.ca> On Mon, Jun 02, 2003 at 03:54:37PM -0700, Alvin Oga's all... > >hi ya ken > >On Mon, 2 Jun 2003, Ken Chase wrote: > >> >we did lots of testing on that 12v dc-dc input to the mini-itx mb >> >( well, www.mini-box.com folks did all the testing and ordering of the >> >( custom 12v dc-to-dc atx power supply > >these tests were for 1-z 2-z systems .. one dc-to-dc ps for each mini-itx >motherboard in the stackable case-size of a standard cdrom > >> Any numbers on the PF of these ac-to-dv (then to dc) floor-brick-with- >> power-cables xformers? I suspect this stuff isnt worth anything til you >> run it off a site-wide 12V infrastructure to get a good PF rating. > >or one can operate like a colo ?? > - take 110VAC in, convert to 12V dc batteries, generate 110VAC > to powerup the colo'd servers > >or 48V dc ( like the phone systems ) totally agreed. my point was that if you dont do this, you'll lose a fair bit in any power saved. I measured a couple laptops around the office with that horridly-named 'kill a watt' or whatever its called thing we ordered after someone on list suggested it - one shoddy old laptop xformer was running at 0.45pf! Most were in .55 to .6 range. Most PC atx ac to dc standard supplies were in .75 to .85 range. A few of our nicer servers had 0.98 to 1.0 pf. Furthermore, our UPS situation would be a fair bit more efficient I suspect were we backing up a fair bit more 12 and 48V than trying to rectify it back to AC, regardless of what losses the end systems incur (just think - ac to dc to charge the batteries, and then in an outtage, dc to ac back to dc. the horror!) Furthermore, it seems to be the only way to power these low-power systems, with DC to DC PS's - these things would probably require a 75 to 100W AC ATX PS for a diskless setup... Who makes those anymore - hard enough these days to find a 250. /kc > >have fun >alvin -- Ken Chase, math at velocet.ca * Velocet Communications Inc. * Toronto, Canada Wiznet Velocet DSL.ca Datavaults 24/7: 416-967-4414 tollfree: 1-866-353-0363 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From jhearns at freesolutions.net Tue Jun 3 00:00:35 2003 From: jhearns at freesolutions.net (John Hearns) Date: 03 Jun 2003 05:00:35 +0100 Subject: ClusterKnoppix In-Reply-To: <20030602144223.L17733@mikee.ath.cx> References: <1054534705.15377.8.camel@harwood.home> <3EDB8FBB.5060907@yakko.cimav.edu.mx> <20030602144223.L17733@mikee.ath.cx> Message-ID: <1054612836.4441.12.camel@harwood.home> On Mon, 2003-06-02 at 20:42, Mike Eggleston wrote: > Have you tried using apt? It seems to run a day or two behind the RedHat > updates, but it gets all the updates. Try setting a nightly cron entry to > execute 'apt-get update; apt-get update' for all your nodes. You can find > apt at www.freshrpms.net. > Mike, I use apt-for-rpm from Freshrpms also, and find it excellent. Last year I also did a lot of work with Current, which is an open source implementation of an up2date server: http://current.tigris.org/ This could also work well for Luis. We should also mention Yum, as Bob Brown is on the list! http://linux.duke.edu/projects/yum/ _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From math at velocet.ca Tue Jun 3 00:05:01 2003 From: math at velocet.ca (Ken Chase) Date: Tue, 3 Jun 2003 00:05:01 -0400 Subject: Cheap PCs from Wal-Mart In-Reply-To: ; from hahn@physics.mcmaster.ca on Mon, Jun 02, 2003 at 11:59:14PM -0400 References: <20030602224009.L81856@velocet.ca> Message-ID: <20030603000501.Q81856@velocet.ca> On Mon, Jun 02, 2003 at 11:59:14PM -0400, Mark Hahn's all... >> - If you add a HD to the setup per CPU, you blow your total W/node figure >> (not to mention your price curve). > >well, a desktop HD would be 5-10W, but I expect a laptop HD would be >a good match. Now you want to run this thing for years to recover the price diff between the two on power ;) What kind of data rates you get to laptop drives? /kc -- Ken Chase, math at velocet.ca * Velocet Communications Inc. * Toronto, Canada Wiznet Velocet DSL.ca Datavaults 24/7: 416-967-4414 tollfree: 1-866-353-0363 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From jhearns at freesolutions.net Mon Jun 2 23:56:23 2003 From: jhearns at freesolutions.net (John Hearns) Date: 03 Jun 2003 04:56:23 +0100 Subject: Linux Magazine article Message-ID: <1054612584.4445.6.camel@harwood.home> I just picked up the June edition of Linux Magazine here in the UK, and read it on the train back from Glasgow. This edition concerns Beowulf computing - and has an excellent article by our very own Bob Brown. Congratulations! For anyone in the UK, Linux Magazine is carried by the Borders bookshops chain. I got mine in the Buchanan Street branch. Oh, and that's the US magazine - not the excellent UK Linux Magazine, edited by John Southern. ps. Two small children sat opposite me on the train. I convinced the little boy that I lived on the train, and slept under the table at night. Wonder if he believedme, and is telling stories at school? _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From jhearns at freesolutions.net Tue Jun 3 00:15:50 2003 From: jhearns at freesolutions.net (John Hearns) Date: 03 Jun 2003 05:15:50 +0100 Subject: Cheap PCs from Wal-Mart In-Reply-To: References: Message-ID: <1054613751.4445.21.camel@harwood.home> On Tue, 2003-06-03 at 03:23, Robert G. Brown wrote: > > It is to teach a generation of kids about managing linux systems, > writing programs, and the rudiments of cluster computing (Amdahl's law > and so forth) in an inexpensive environment that they can "play" with. > It is for cluster humans like me to have cheap miniclusters at home to > play with to try things out in an environment they can "break the hell > out of" without interfering with their production environment. Good points. That's why I have my nodes - I live in a small apartment in London's Docklands. Not much spare space, and with the high temperatures in London at the moment, I'm glad of them! (Air conditioning is not that common in UK houses). Don't forget also Jim Lux's thoughts on portable/resource constrained computing. If anyone is interested, here is the abstract for my UKUUG Developers Talk. Maybe a good excuse to visit the Edinburgh Festival! http://www.ukuug.org/events/linux2003/prog/abstract-JHearns-1.shtml BTW, I'm to talk about C optimisations for the VIA processors during the talk. If anyone on the list is good with this sort of thing - specifically the latest gcc versions, or commercial compilers please drop me an email off list if I can pick your brains. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From Andrew.Cannon at nnc.co.uk Tue Jun 3 03:11:14 2003 From: Andrew.Cannon at nnc.co.uk (Cannon, Andrew) Date: Tue, 3 Jun 2003 08:11:14 +0100 Subject: Help: different kernel images for slaves. Message-ID: This might not help, but from what I've been told (I'm due to try it in our 16 node cluster soon) you can use the Kickstart program to distribute different versions of most of the software in RH across the network. I'm sure you should be able to do the same with the kernel too (I hope so...) Hope this helps a bit. Andrew -----Original Message----- From: Robin Laing [mailto:Robin.Laing at drdc-rddc.gc.ca] Sent: Monday, June 02, 2003 10:26 PM To: beowulf at beowulf.org Subject: Help: different kernel images for slaves. Hello, I am involved with adding some SMP machines to our small cluster and I am trying to find out how I tell the new machines to run the SMP kernel instead of the default in the /var/beowulf/config file. We are running RH 9 and BPpoc. Our present system is working with single processor boot image and the SMP machines will boot with only one processor working. I have compiled a SMP kernel to test but I cannot just change the boot kernel for all the other machines. Presently in /var/beowulf are: boot.img nodeboot.img nodeinfo I want to add boot_smp.img I have not seen any documentation on configuring /etc/beowulf/config to work with more than one boot image. Is it possible to define which nodes boot with which kernel? -- Robin Laing Instrumentation Technologist Voice: 1.403.544.4762 Military Engineering Section FAX: 1.403.544.4704 Defence R&D Canada - Suffield Email: Robin.Laing at DRDC-RDDC.gc.ca PO Box 4000, Station Main WWW: http://www.suffield.drdc-rddc.gc.ca Medicine Hat, AB, T1A 8K6 Canada _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf *********************************************************************************** NNC Limited Booths Hall Chelford Road Knutsford Cheshire WA16 8QZ Country of Registration: United Kingdom Registered Number: 1120437 This e-mail and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this e-mail in error please notify the NNC system manager by e-mail at eadm at nnc.co.uk. *********************************************************************************** _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From lange at informatik.Uni-Koeln.DE Tue Jun 3 05:14:59 2003 From: lange at informatik.Uni-Koeln.DE (Thomas Lange) Date: Tue, 3 Jun 2003 11:14:59 +0200 Subject: ClusterKnoppix In-Reply-To: <3EDB8FBB.5060907@yakko.cimav.edu.mx> References: <1054534705.15377.8.camel@harwood.home> <3EDB8FBB.5060907@yakko.cimav.edu.mx> Message-ID: <16092.26387.257370.530388@informatik.uni-koeln.de> >>>>> On Mon, 02 Jun 2003 11:56:11 -0600, Luis Fernando Licon Padilla said: > I have a 12 dual nodes beowulf running on Red Hat Linux 8.0, but > you know, I have the update problem since I do not want to pay > the update service to Red Hat, so my question is, Does someone > have experience about clusters running on both Debian and RedHat > distributions, who can help me to decide on this issue? Have a look at FAI, the fully automatic installation (and upgrade) for Debian. On the FAI web pages http://www.informatik.uni-koeln.de/fai/ you can find some success story of people how installed a beowulf cluster using my tool. Updates are very simple (and for free, of course) using apt-get. -- regards Thomas _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rgb at phy.duke.edu Tue Jun 3 07:05:33 2003 From: rgb at phy.duke.edu (Robert G. Brown) Date: Tue, 3 Jun 2003 07:05:33 -0400 (EDT) Subject: ClusterKnoppix In-Reply-To: <1054612836.4441.12.camel@harwood.home> Message-ID: On 3 Jun 2003, John Hearns wrote: > We should also mention Yum, as Bob Brown is on the list! > http://linux.duke.edu/projects/yum/ (Remembering that although I do love yum, it is the brainchild of Seth Vidal and Mike Stenner at Duke. I just use it, make a lot of yammering sounds on the list about making its config file xmlish, and sometimes have lunch with them and exchange a bit of banter about python, a language I've sworn not to learn so I CAN'T help with yum that I'll probably know by the end of summer...:-) rgb -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb at phy.duke.edu _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From James.P.Lux at jpl.nasa.gov Tue Jun 3 09:54:50 2003 From: James.P.Lux at jpl.nasa.gov (Jim Lux) Date: Tue, 3 Jun 2003 06:54:50 -0700 Subject: Cheap PCs from Wal-Mart - mini-itx References: <20030601132552.D11031@mikee.ath.cx> <20030602182436.F81856@velocet.ca> Message-ID: <003001c329d7$b3d9aac0$02a8a8c0@office1> > >> > Runs pretty cool, even with just the 2 puny little case fans, and can be > >> > driven off a 12v Wall Wart (?tm?; should I trade mark it? no...) at > >> > less than 5a. The wart I have is rated for 4.5a, but I've not measured > >> > it yet. > > > >we did lots of testing on that 12v dc-dc input to the mini-itx mb > >( well, www.mini-box.com folks did all the testing and ordering of the > >( custom 12v dc-to-dc atx power supply > > Any numbers on the PF of these ac-to-dv (then to dc) floor-brick-with- > power-cables xformers? I suspect this stuff isnt worth anything til you > run it off a site-wide 12V infrastructure to get a good PF rating. > > /kc Power factor, or efficiency? CE marking requires fairly good harmonic control and power factor, even for small wall warts or floor bricks. It's become fairly straightforward to design "good" power supplies, since there is huge consumer demand (via regulatory process) and the semiconductor mfrs have produced inexpensive chips to make PFC easier. Efficiency, though, isn't all that hot, there. Especially considered in the system context, because you have a linevoltage:12V conversion, and then another 12V:5 and 12V:3 conversion, each of which is probably 80-85% efficient. In a standard PC, you have only one conversion. However, you wouldn't want 12V distribution either. IR losses in the cabling would eat you alive, and maintaining regulation would be very difficult. One better solution would be to distribute unregulated 300VDC to all the mobos, and have them convert that to the required 12,5,and 3 in one step (essentially splitting the standard PC power supply down the middle). For safety reasons, you don't see many 300VDC distribution designs, though. Telco stuff uses 48VDC for distribution... high enough that IR losses aren't killing you, you can get it with batteries, and it's below the informal "50V is low voltage" shock hazard cutoff. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From James.P.Lux at jpl.nasa.gov Tue Jun 3 10:04:29 2003 From: James.P.Lux at jpl.nasa.gov (Jim Lux) Date: Tue, 3 Jun 2003 07:04:29 -0700 Subject: Cheap PCs from Wal-Mart References: Message-ID: <003d01c329d9$0cbf1fc0$02a8a8c0@office1> > > The C3 at 800 MHz dissipates 6.65W average and peaks at 12W > > The Nehemiah at 1 GHz dissipates 11.25/15 W ave/pk > > I suspect VIA's "peak" is not equivalent (lower) than Intel's TDP. Hard to know until someone sticks a current probe on the devices... > > > This is really a first order approximation. You'd have to look at > > peripheral power, memory power, and power supply conversion and power > > distribution efficiency. Peripherals and memory probably scale with memory > > speed fairly linearly. > > huh? do you mean "the P4 drives ram much faster and so the ram > will also dissipate more power"? Precisely.. CMOS, to a first order, has power dissipation proportional to clock frequency. Cycle the bus at 200 MHz and it draws twice as much power as cycling the bus at 100 MHz. I don't know if the P4 or C3 have the same bus width, too? Wider buses draw more power (for the line drivers/receivers). > > > The higher powered device will have more I^2R > > losses in the power supply and distribution. (Supplying 50 or so Amps at > > 2-3V is no easy feat) > > well, the PS supplies 3.3V; the Vcc for the CPU only has to travel > a couple of inches, and that's probably in a plane. I don't really > thing that's an issue. It's a huge issue... The efficiency of the PS is lower at 3.3V than at 5V or 12V, for instance. As far as the CPU core voltage regulator, the same applies... If you push power through anything, you're going to have more IR losses at 1.8V than at 2.5V. There's only so much copper available on the board to carry the current, and the pin or ball is only so big. On chip, there's the issue of the power carrying conductors in the metalization. The saving grace is that, for CMOS, the power dissipation also scales with voltage, so as the feature size goes down (increasing IR losses), the power consumed goes down too (reducing current). However, consider.. a contact/trace resistance of 5 milliohms, carrying a current of 20 amps, disspates 2 Watts... 5 mOhm is pretty darn low.. especially for a pin only 25 mils on a side. > > my whole point was that people must not even start to think that > these low-power CPUs are close in performance. and even if they were, > I don't think people really have an intuitive sense for what "scales well" > means - I doubt *anyone* has a task that is as undemanding of the system > as dhrystone is, for instance. > > > computing, I'd like to see more attention paid to computing per joule or > > computing rate/watt. There are different tradeoffs whether you are > > sure, but please, let's use specFPrate per watt. even that's a bit > dated, since it's so cache-friendly. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From Robin.Laing at drdc-rddc.gc.ca Tue Jun 3 10:51:44 2003 From: Robin.Laing at drdc-rddc.gc.ca (Robin Laing) Date: Tue, 03 Jun 2003 08:51:44 -0600 Subject: Help: different kernel images for slaves. Message-ID: <3EDCB600.4080500@drdc-rddc.gc.ca> Andrew, This would work but I forgot to mention that these are diskless workstations. Under bproc there is a setting in the /etc/beowulf/config for kernel image but I cannot find any documentation for assigning or allocating a different kernel for different machines. I will look at kickstart though to see if it would work and install on each system. > Message: 10 > From: "Cannon, Andrew" > To: beowulf at beowulf.org > Subject: RE: Help: different kernel images for slaves. > Date: Tue, 3 Jun 2003 08:11:14 +0100 > > This might not help, but from what I've been told (I'm due to try it in our > 16 node cluster soon) you can use the Kickstart program to distribute > different versions of most of the software in RH across the network. I'm > sure you should be able to do the same with the kernel too (I hope so...) > > Hope this helps a bit. > > Andrew > > -----Original Message----- > From: Robin Laing [mailto:Robin.Laing at drdc-rddc.gc.ca] > Sent: Monday, June 02, 2003 10:26 PM > To: beowulf at beowulf.org > Subject: Help: different kernel images for slaves. > > > Hello, > > I am involved with adding some SMP machines to our small cluster and I > am trying to find out how I tell the new machines to run the SMP > kernel instead of the default in the /var/beowulf/config file. > > We are running RH 9 and BPpoc. > > Our present system is working with single processor boot image and the > SMP machines will boot with only one processor working. I have > compiled a SMP kernel to test but I cannot just change the boot kernel > for all the other machines. > > Presently in /var/beowulf are: > boot.img nodeboot.img nodeinfo > > I want to add boot_smp.img > > I have not seen any documentation on configuring /etc/beowulf/config > to work with more than one boot image. > > Is it possible to define which nodes boot with which kernel? > > -- Robin Laing Instrumentation Technologist Voice: 1.403.544.4762 Military Engineering Section FAX: 1.403.544.4704 Defence R&D Canada - Suffield Email: Robin.Laing at DRDC-RDDC.gc.ca PO Box 4000, Station Main WWW: http://www.suffield.drdc-rddc.gc.ca Medicine Hat, AB, T1A 8K6 Canada _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- Robin Laing Instrumentation Technologist Voice: 1.403.544.4762 Military Engineering Section FAX: 1.403.544.4704 Defence R&D Canada - Suffield Email: Robin.Laing at DRDC-RDDC.gc.ca PO Box 4000, Station Main WWW: http://www.suffield.drdc-rddc.gc.ca Medicine Hat, AB, T1A 8K6 Canada _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From bryan at ayesha.phys.Virginia.EDU Tue Jun 3 10:36:24 2003 From: bryan at ayesha.phys.Virginia.EDU (Bryan K. Wright) Date: Tue, 03 Jun 2003 10:36:24 -0400 Subject: Data corruption w/ tg3 and bcm5700 drivers Message-ID: <200306031436.h53EaPJ29643@ayesha.phys.Virginia.EDU> Hi folks, We're having some problems with the tg3 and bcm5700 drivers, and I was hoping somebody out there might have a clue to share. We have several boxes based on the GA-7DPXDW+ motherboard, with 3c996 network cards. (According to the bcm5700 driver, these identify themselves as BCM95700A6.) We've tried both the bcm5700 driver and a couple of versions of the tg3 driver (1.2 and 1.5) with similar results. We've tried the tg3 drivers with both the "stock" Red Hat 2.4.18-5 kernel (from RH 7.2) and with the openMosix 2.4.20-openmosix2 kernel. We find that, transferring large (multi-gigabyte) files from one machine to another results in either an error that aborts the transfer (at a random time) or, if the transfer completes, we find that the transferred file has the same size as the original, but their checksums (from /usr/bin/sum) differ. If we try the transfer using scp, the transfer always aborts, often with a message about a corrupt Message Authentication Code. Using ftp, the transfer will sometimes complete. Interestingly, we find that if we do the experiment with a file full of zeros (e.g., dd if=/dev/zero of=junk.dat ...) the checksums match (on those occasions when the transfer completes, that is). Real data is always corrupted, though. We've tried this with nothing but a cross-connect cable between the boxes, with the same results. Any suggestions would be appreciated, either solutions or diagnostics we can try. Thanks in advance, Bryan Wright -- =============================================================================== Bryan Wright |"If you take cranberries and stew them like Physics Department | applesauce, they taste much more like prunes University of Virginia | than rhubarb does." -- Groucho Charlottesville, VA 22901 | (434) 924-7218 | bryan at virginia.edu =============================================================================== _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From jhearns at freesolutions.net Tue Jun 3 11:29:59 2003 From: jhearns at freesolutions.net (John Hearns) Date: Tue, 3 Jun 2003 16:29:59 +0100 Subject: Help: different kernel images for slaves. References: <3EDCB600.4080500@drdc-rddc.gc.ca> Message-ID: <01dd01c329e5$01ab29b0$8461cdc2@DREAD> ----- Original Message ----- From: "Robin Laing" To: Sent: Tuesday, June 03, 2003 3:51 PM Subject: RE: Help: different kernel images for slaves. > Andrew, > > This would work but I forgot to mention that these are diskless > workstations. Under bproc there is a setting in the > /etc/beowulf/config for kernel image but I cannot find any > documentation for assigning or allocating a different kernel for > different machines. > > I will look at kickstart though to see if it would work and install on > each system. You can do diskless installs using Linux Terminal Server Project http://www.ltsp.org Loading different kernels would be a matter of editing the /etc/dhcpd.conf and creating an entry for each workstation. The LTSP documentation gives an example, where 'filename' is the path to the kernel for that particular workstation: group { use-host-decl-names on; option log-servers 192.168.0.254; host ws001 { hardware ethernet 00:E0:18:E0:04:82; fixed-address 192.168.0.1; filename "/lts/vmlinuz.ltsp"; } } I stress that I haven't actually DONE this - though I have set up and demoed LTSP. The OSCAR people are also working on OSCAR-lite, though I don't know if you can distribute different kernels using it. LTSP might be a good way for you to go - I can help you getting it running, though the documentation is good. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rajesh at utdallas.edu Tue Jun 3 11:31:06 2003 From: rajesh at utdallas.edu (Rajesh Bhairampally) Date: Tue, 3 Jun 2003 10:31:06 -0500 Subject: Cheap PCs from Wal-Mart References: <1054613751.4445.21.camel@harwood.home> Message-ID: <020501c329e5$26ba77b0$c900a8c0@Rajesh> > > It is to teach a generation of kids about managing linux systems, > > writing programs, and the rudiments of cluster computing (Amdahl's law > > and so forth) in an inexpensive environment that they can "play" with. i am new to cluster computing environment and would like to learn with hand-on experience. i would like to start with two linux boxes (i recently placed an order for PCs from walmart) in my home environment. I am wondering if there are any tutorials/manuals for newbees to teach beowulf clustering concepts and hands-on tutorials. Any help is greately appreciated. Thanks, Rajesh _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From Robin.Laing at drdc-rddc.gc.ca Tue Jun 3 11:48:46 2003 From: Robin.Laing at drdc-rddc.gc.ca (Robin Laing) Date: Tue, 03 Jun 2003 09:48:46 -0600 Subject: Help: different kernel images for slaves. In-Reply-To: <01dd01c329e5$01ab29b0$8461cdc2@DREAD> References: <3EDCB600.4080500@drdc-rddc.gc.ca> <01dd01c329e5$01ab29b0$8461cdc2@DREAD> Message-ID: <3EDCC35E.8060301@drdc-rddc.gc.ca> Hello John, This is one of the things I am looking at. I just found out that the MB has DHCP or bootp built in. Our present cluster is using beoboot and the administrator (part time and I am going to be the other) wants to use beoboot. I am now reading documentation on this matter. John Hearns wrote: > ----- Original Message ----- > From: "Robin Laing" > To: > Sent: Tuesday, June 03, 2003 3:51 PM > Subject: RE: Help: different kernel images for slaves. > > > >>Andrew, >> >>This would work but I forgot to mention that these are diskless >>workstations. Under bproc there is a setting in the >>/etc/beowulf/config for kernel image but I cannot find any >>documentation for assigning or allocating a different kernel for >>different machines. >> >>I will look at kickstart though to see if it would work and install on >>each system. > > > You can do diskless installs using Linux Terminal Server Project > http://www.ltsp.org > > Loading different kernels would be a matter of editing the > /etc/dhcpd.conf and creating an entry for each workstation. > The LTSP documentation gives an example, > where 'filename' is the path to the kernel for that particular workstation: > > > group { > use-host-decl-names on; > option log-servers 192.168.0.254; > > host ws001 { > hardware ethernet 00:E0:18:E0:04:82; > fixed-address 192.168.0.1; > filename "/lts/vmlinuz.ltsp"; > } > } > > > > I stress that I haven't actually DONE this - though I have set up and demoed > LTSP. > > > The OSCAR people are also working on OSCAR-lite, though I don't know if you > can > distribute different kernels using it. > LTSP might be a good way for you to go - I can help you getting it running, > though the documentation > is good. > -- Robin Laing Instrumentation Technologist Voice: 1.403.544.4762 Military Engineering Section FAX: 1.403.544.4704 Defence R&D Canada - Suffield Email: Robin.Laing at DRDC-RDDC.gc.ca PO Box 4000, Station Main WWW: http://www.suffield.drdc-rddc.gc.ca Medicine Hat, AB, T1A 8K6 Canada _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rgb at phy.duke.edu Tue Jun 3 12:34:30 2003 From: rgb at phy.duke.edu (Robert G. Brown) Date: Tue, 3 Jun 2003 12:34:30 -0400 (EDT) Subject: Cheap PCs from Wal-Mart In-Reply-To: <020501c329e5$26ba77b0$c900a8c0@Rajesh> Message-ID: On Tue, 3 Jun 2003, Rajesh Bhairampally wrote: > > > It is to teach a generation of kids about managing linux systems, > > > writing programs, and the rudiments of cluster computing (Amdahl's law > > > and so forth) in an inexpensive environment that they can "play" with. > > i am new to cluster computing environment and would like to learn with > hand-on experience. > i would like to start with two linux boxes (i recently placed an order for > PCs from walmart) in > my home environment. I am wondering if there are any tutorials/manuals for > newbees to teach beowulf clustering concepts and hands-on tutorials. Any > help is greately appreciated. In Linux Magazine (already mentioned today) Forrest Hoffman (a long-time beowulfer) runs an "Extreme Linux" column that you might find very helpful. Forrest published material every month that ranages from newbie/tutorial to fairly advanced. On www.linux-mag.html his current online article appears to be "Message Passing for Master/Slave Programs". This article explores the use of MPI; previous columns have covered PVM and lots of other stuff. Both PVM and MPI have demo/examples in their installation kit(s). There is additional information on cluster setup and operation in many places -- so many I have a hard time keeping them straight myself. To help my own memory, I've collected a LOT of resources you might find useful on http://www.phy.duke.edu/brahma/index.php In particular, http://www.phy.duke.edu/brahma/Resources/links.php has close to everything clusteroid that I've been able to find -- FAQ, HOWTO, Underground, Scyld, bproc, FAI, online magazines devoted to clustering in whole or in part, newsletters, and more. This site is "almost" up to date -- I rebuilt it a short time ago, and noticed that it still has indentation problems while reviewing it a moment ago:-) BTW (to all the list cluster experts) I'd cherish suggestions for additions, deletions, or modifications to this page and its companion vendors.php page. Vendors should carefully read about the tee-shirt clause.;-) Tee shirts aren't really required, but (as I note) they can't hurt... rgb > > Thanks, > Rajesh > > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org > To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf > -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb at phy.duke.edu _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From luis.licon at yakko.cimav.edu.mx Tue Jun 3 14:28:14 2003 From: luis.licon at yakko.cimav.edu.mx (Luis Fernando Licon Padilla) Date: Tue, 03 Jun 2003 12:28:14 -0600 Subject: ClusterKnoppix References: <1054534705.15377.8.camel@harwood.home> <3EDB8FBB.5060907@yakko.cimav.edu.mx> Message-ID: <3EDCE8BE.1010502@yakko.cimav.edu.mx> Luis Fernando Licon Padilla wrote: > John Hearns wrote: > >> I saw this on Slashdot a few days ago. >> >> It is a customised Knoppix distribution, with an OpenMosix patched >> kernel, and PXE/DHCP enabled etc. >> I intend to try this out in the next few days. >> >> For those who haven't used Knoppix, it is a CD which contains a bootable >> Debian-based distribution which installs itself in memory, not touching >> the hard drive. Knoppix has excellent hardware detection, and sets up >> a good desktop. It 'just works'. >> Uses might include demonstrating a Linux desktop to friends/colleagues, >> testing if a PC can run Linux before buying/accepting it, or as a rescue >> disk for troubleshooting an already installed Linux system. >> >> John Hearns >> >> _______________________________________________ >> Beowulf mailing list, Beowulf at beowulf.org >> To change your subscription (digest mode or unsubscribe) visit >> http://www.beowulf.org/mailman/listinfo/beowulf >> >> >> >> > Hi you all guys, > > I have a 12 dual nodes beowulf running on Red Hat Linux 8.0, but you > know, I have the update problem since I do not want to pay the update > service to Red Hat, so my question is, Does someone have experience > about clusters running on both Debian and RedHat distributions, who > can help me to decide on this issue? > > Thanks, > > Luis > Thank you all guys for sharing your experiences with me, Sincerily, Luis -- ISC Luis Fernando Licon Padilla Advanced Materials Research Center Miguel de Cervantes 120 Complejo Industrial Chihuahua C.P.31109 Chihuahua, Chih. Mexico Phone: 52 (614)4391154 Fax: 52 (614)4391112 alternative e-mail: lordsirion2002 at yahoo.com _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From bogdan.costescu at iwr.uni-heidelberg.de Tue Jun 3 14:32:34 2003 From: bogdan.costescu at iwr.uni-heidelberg.de (Bogdan Costescu) Date: Tue, 3 Jun 2003 20:32:34 +0200 (CEST) Subject: Data corruption w/ tg3 and bcm5700 drivers In-Reply-To: <200306031436.h53EaPJ29643@ayesha.phys.Virginia.EDU> Message-ID: On Tue, 3 Jun 2003, Bryan K. Wright wrote: > We've tried both the bcm5700 driver and a couple of versions > of the tg3 driver (1.2 and 1.5) with similar results. We've tried the > tg3 drivers with both the "stock" Red Hat 2.4.18-5 kernel (from RH > 7.2) and with the openMosix 2.4.20-openmosix2 kernel. RedHat has released several kernels after 2.4.18-5 and IIRC some were related to bugs in tg3 driver. The current one just released is 2.4.20-18.7 (for 7.2). > We find that, transferring large (multi-gigabyte) files from > one machine to another results in either an error that aborts the > transfer (at a random time) or, if the transfer completes, we find > that the transferred file has the same size as the original, but > their checksums (from /usr/bin/sum) differ. This might also be a memory problem on one (or both) computer(s). Let memtest86 go through several cycles of its tests to make sure that this is not the case. -- Bogdan Costescu IWR - Interdisziplinaeres Zentrum fuer Wissenschaftliches Rechnen Universitaet Heidelberg, INF 368, D-69120 Heidelberg, GERMANY Telephone: +49 6221 54 8869, Telefax: +49 6221 54 8868 E-mail: Bogdan.Costescu at IWR.Uni-Heidelberg.De _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From wrp at alpha0.bioch.virginia.edu Tue Jun 3 15:55:11 2003 From: wrp at alpha0.bioch.virginia.edu (William R.Pearson) Date: Tue, 3 Jun 2003 15:55:11 -0400 Subject: Data corruption w/ tg3 and bcm5700 drivers Message-ID: <494D29CF-95FD-11D7-9C0C-003065F9A63A@alpha0.bioch.virginia.edu> We have had terrible problems with the tg3/bcm5700 drivers as well, but only on our head machines with dual 3Com996 boards. We have not tried a new kernel for the past 3 months, but before that we tried everything from 2.4.18-3 to 2.4.20, the ONLY two versions of the kernel that were stable for several hours were 2.4.18-3 (original with RH7.3) and 2.4.18-4. We have not had any problems with machines with a single 3Com996 NIC (our nodes). We are very frustrated with dual 3Com996 support, as there have been at least 4 kernel upgrades that specifically mentioned this problem, and none of them have fixed it. Our machines are dual 1 GHz PIII with supermicro MBs. Bill Pearson _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From xyzzy at speakeasy.org Tue Jun 3 16:11:39 2003 From: xyzzy at speakeasy.org (Trent Piepho) Date: Tue, 3 Jun 2003 13:11:39 -0700 (PDT) Subject: measuring power usage Message-ID: I seem to recall reading in a mailing list, I think this one, about a relatively inexpensive device that would measure the real wattage and power factor of electronics. I think that someone on the list bought one and posted a review. Does this ring a bell with anyone? I'd like to get some real numbers about what our cluster is drawing and how much we capacity we have for new hardware. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From selva at mailaps.org Tue Jun 3 16:26:12 2003 From: selva at mailaps.org (Selva Nair) Date: Tue, 3 Jun 2003 16:26:12 -0400 (EDT) Subject: Data corruption w/ tg3 and bcm5700 drivers In-Reply-To: <200306031436.h53EaPJ29643@ayesha.phys.Virginia.EDU> Message-ID: On Tue, 3 Jun 2003, Bryan K. Wright wrote: > Hi folks, > > We're having some problems with the tg3 and bcm5700 drivers, > and I was hoping somebody out there might have a clue to share. > > We have several boxes based on the GA-7DPXDW+ motherboard, > with 3c996 network cards. (According to the bcm5700 driver, these > identify themselves as BCM95700A6.) > > We've tried both the bcm5700 driver and a couple of versions > of the tg3 driver (1.2 and 1.5) with similar results. We've tried the > tg3 drivers with both the "stock" Red Hat 2.4.18-5 kernel (from RH > 7.2) and with the openMosix 2.4.20-openmosix2 kernel. I have had very similar issues with the 3com 996 cards using Redhat 7.0-&.2 kernels. I switched to the Broadcom BCM5700 driver distributed by Broadcom Corporation and the network is pretty stable under constant heavy load since then. I use their driver source version 2.0.32 (release date: 01/02/2002) rebuilt locally. I cant recall where I downloaded the sources from. Selva _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From v_454 at yahoo.com Tue Jun 3 16:19:57 2003 From: v_454 at yahoo.com (Steve Elliot) Date: Tue, 3 Jun 2003 13:19:57 -0700 (PDT) Subject: Large File Support for RedHat Linux 8.0 Kernal 2.4.18-14 Message-ID: <20030603201957.52444.qmail@web40506.mail.yahoo.com> Hello, I am trying to find out if there is a newer version of the LFS rpms. Currently I can only find the rpms for Kernal-2.2.12-20 and glibc-2.1 on sclyd's site. Any help would be appreciated and thanks in advance. Steve Elliot __________________________________ Do you Yahoo!? Yahoo! Calendar - Free online calendar with sync to Outlook(TM). http://calendar.yahoo.com _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From Robin.Laing at drdc-rddc.gc.ca Tue Jun 3 16:50:14 2003 From: Robin.Laing at drdc-rddc.gc.ca (Robin Laing) Date: Tue, 03 Jun 2003 14:50:14 -0600 Subject: Help: different kernel images for slaves - Solved Message-ID: <3EDD0A06.9060706@drdc-rddc.gc.ca> The administrator found the solution in the source tree Release Notes for beoboot-cm.1.4. In the /etc/beowulf/config [...] Support for multiple boot images has been added via the configuration file. For example: bootfile /var/beowulf/boot.img # this is the default boot image bootfile 5 /var/beowulf/boot2.img # Node 5 takes a different boot file. You can also supply comma separated node ranges like 5-10,100-120 [...] Now if only this would get put into the normal documentation. Isn't it strange how documentation in any field seems to be the last thing that gets completed. Of course this is a start in the process of getting the kernel to work. The process does find the correct kernel on boot and thus we are still working with only one procedure for the whole cluster. Now to get all the other little things to work. :) I hope that this info will be of use for others. -- Robin Laing Instrumentation Technologist Voice: 1.403.544.4762 Military Engineering Section FAX: 1.403.544.4704 Defence R&D Canada - Suffield Email: Robin.Laing at DRDC-RDDC.gc.ca PO Box 4000, Station Main WWW: http://www.suffield.drdc-rddc.gc.ca Medicine Hat, AB, T1A 8K6 Canada _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From becker at scyld.com Tue Jun 3 17:36:25 2003 From: becker at scyld.com (Donald Becker) Date: Tue, 3 Jun 2003 17:36:25 -0400 (EDT) Subject: Data corruption w/ tg3 and bcm5700 drivers In-Reply-To: <494D29CF-95FD-11D7-9C0C-003065F9A63A@alpha0.bioch.virginia.edu> Message-ID: On Tue, 3 Jun 2003, William R.Pearson wrote: > We have had terrible problems with the tg3/bcm5700 drivers as well, > but only on our head machines with dual 3Com996 boards. We use the bcm5700 driver with our distribution. Although we have to frequently update it to support new IDs, it has been much more reliable than the tg3 driver. The tg3 driver has had many new versions that claim to have fixed all of the serious problems with the previous version, and be even faster. But some of the new versions are closely tied to subtle interface changes in 2.5, which makes it risky to just back-port to production kernels without careful study of what the changes mean. [[ Of course, I have strong personal opinions about network driver API changes, backward compatible stable driver releases and the difficulty of keeping driver code working. I'm not above saying "not as easy as it looks from the outside, eh?". ]] -- Donald Becker becker at scyld.com Scyld Computing Corporation http://www.scyld.com 914 Bay Ridge Road, Suite 220 Scyld Beowulf cluster system Annapolis MD 21403 410-990-9993 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From lindahl at keyresearch.com Tue Jun 3 17:56:10 2003 From: lindahl at keyresearch.com (Greg Lindahl) Date: Tue, 3 Jun 2003 14:56:10 -0700 Subject: Data corruption w/ tg3 and bcm5700 drivers In-Reply-To: <494D29CF-95FD-11D7-9C0C-003065F9A63A@alpha0.bioch.virginia.edu> References: <494D29CF-95FD-11D7-9C0C-003065F9A63A@alpha0.bioch.virginia.edu> Message-ID: <20030603215610.GA1884@greglaptop.internal.keyresearch.com> On Tue, Jun 03, 2003 at 03:55:11PM -0400, William R.Pearson wrote: > We have had terrible problems with the tg3/bcm5700 drivers as well, > but only on our head machines with dual 3Com996 boards. Bill, Do you see the instability when you're only using 1 of the interfaces? Do you see the instability when you boot a uniprocessor kernel? These are the kinds of experiments you can do to narrow down the problem. greg _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From becker at scyld.com Tue Jun 3 17:43:10 2003 From: becker at scyld.com (Donald Becker) Date: Tue, 3 Jun 2003 17:43:10 -0400 (EDT) Subject: Large File Support for RedHat Linux 8.0 Kernal 2.4.18-14 In-Reply-To: <20030603201957.52444.qmail@web40506.mail.yahoo.com> Message-ID: On Tue, 3 Jun 2003, Steve Elliot wrote: > I am trying to find out if there is a newer version of > the LFS rpms. Currently I can only find the rpms for > Kernal-2.2.12-20 and glibc-2.1 on sclyd's site. Easy answer: we were the first commercial distribution with large file LFS support, way back in the days of 2.2 kernel (circa 2000). The 2.4 kernel and current C library have LFS support, thus new patch sets are not needed. -- Donald Becker becker at scyld.com Scyld Computing Corporation http://www.scyld.com 914 Bay Ridge Road, Suite 220 Scyld Beowulf cluster system Annapolis MD 21403 410-990-9993 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From selva at mailaps.org Tue Jun 3 18:11:06 2003 From: selva at mailaps.org (Selva Nair) Date: Tue, 3 Jun 2003 18:11:06 -0400 (EDT) Subject: measuring power usage In-Reply-To: Message-ID: On Tue, 3 Jun 2003, Trent Piepho wrote: > I seem to recall reading in a mailing list, I think this one, about a > relatively inexpensive device that would measure the real wattage and power > factor of electronics. I think that someone on the list bought one and posted > a review. Does this ring a bell with anyone? I'd like to get some real > numbers about what our cluster is drawing and how much we capacity we have for > new hardware. google "kill a watt" Selva _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rgb at phy.duke.edu Tue Jun 3 18:40:46 2003 From: rgb at phy.duke.edu (Robert G. Brown) Date: Tue, 3 Jun 2003 18:40:46 -0400 (EDT) Subject: measuring power usage In-Reply-To: Message-ID: On Tue, 3 Jun 2003, Selva Nair wrote: > On Tue, 3 Jun 2003, Trent Piepho wrote: > > > I seem to recall reading in a mailing list, I think this one, about a > > relatively inexpensive device that would measure the real wattage and power > > factor of electronics. I think that someone on the list bought one and posted > > a review. Does this ring a bell with anyone? I'd like to get some real > > numbers about what our cluster is drawing and how much we capacity we have for > > new hardware. > > google "kill a watt" Yes, very lovely thingie. There are links to at least one source on the brahma vendors page, and I >>have<< pictures of a kill-a-watt in action, if I ever overcome my innate inertia and post them on the brahma photo tour... rgb > > Selva > > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org > To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf > -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb at phy.duke.edu _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From dlane at ap.stmarys.ca Tue Jun 3 18:50:38 2003 From: dlane at ap.stmarys.ca (Dave Lane) Date: Tue, 03 Jun 2003 19:50:38 -0300 Subject: measuring power usage In-Reply-To: References: Message-ID: <5.2.0.9.0.20030603194551.023ddb08@crux.stmarys.ca> At 06:40 PM 6/3/2003 -0400, Robert G. Brown wrote: >On Tue, 3 Jun 2003, Selva Nair wrote: > > > On Tue, 3 Jun 2003, Trent Piepho wrote: > > > > > I seem to recall reading in a mailing list, I think this one, about a > > > relatively inexpensive device that would measure the real wattage and > power > > > factor of electronics. I think that someone on the list bought one > and posted > > > a review. Does this ring a bell with anyone? I'd like to get some real > > > numbers about what our cluster is drawing and how much we capacity we > have for > > > new hardware. > > > > google "kill a watt" > >Yes, very lovely thingie. There are links to at least one source on the >brahma vendors page, and I >>have<< pictures of a kill-a-watt in action, >if I ever overcome my innate inertia and post them on the brahma photo >tour... There is nothing in the flier that indicates that it measures rms amps and hence rms watts. Computers are not resistive loads and they draw current in a switching fashion. As a result they usually produce a lower current reading on a non-rms meter than they are actually using. From memory the measurements I did produced about a 20% difference in current measurements between Fluke rms and ordinary meters. ... Dave _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From alvin at Maggie.Linux-Consulting.com Tue Jun 3 19:19:38 2003 From: alvin at Maggie.Linux-Consulting.com (Alvin Oga) Date: Tue, 3 Jun 2003 16:19:38 -0700 (PDT) Subject: measuring power usage - meter In-Reply-To: <5.2.0.9.0.20030603194551.023ddb08@crux.stmarys.ca> Message-ID: On Tue, 3 Jun 2003, Dave Lane wrote: > At 06:40 PM 6/3/2003 -0400, Robert G. Brown wrote: .. > > > google "kill a watt" > > > >Yes, very lovely thingie. There are links to at least one source on the > >brahma vendors page, and I >>have<< pictures of a kill-a-watt in action, > >if I ever overcome my innate inertia and post them on the brahma photo > >tour... i hear a regular power meter is about $100 ... and i would assume it would be accurate down to the pennies since the local utilities supposed to be charging $0.xx per kwhr c ya alvin _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From xyzzy at speakeasy.org Tue Jun 3 19:35:56 2003 From: xyzzy at speakeasy.org (Trent Piepho) Date: Tue, 3 Jun 2003 16:35:56 -0700 (PDT) Subject: measuring power usage In-Reply-To: <5.2.0.9.0.20030603194551.023ddb08@crux.stmarys.ca> Message-ID: On Tue, 3 Jun 2003, Dave Lane wrote: > > > google "kill a watt" > > > >Yes, very lovely thingie. There are links to at least one source on the > >brahma vendors page, and I >>have<< pictures of a kill-a-watt in action, > >if I ever overcome my innate inertia and post them on the brahma photo > >tour... Thanks to all that responded. Cute pun for a name, but it completely thwarted my google search for terms like "power factor" and "meter". BTW, is there a real searchable list archive anywhere? google groups has lists.beowulf but it looks like they stopped archiving that in 2000. The archives on beowulf.org aren't searchable, unless you try to use a google web search to find pages that were indexed. > There is nothing in the flier that indicates that it measures rms amps and > hence rms watts. Computers are not resistive loads and they draw current in It measures power factor and frequency, so it should be doing what is necessary to find true rms. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From dlane at ap.stmarys.ca Tue Jun 3 20:00:21 2003 From: dlane at ap.stmarys.ca (Dave Lane) Date: Tue, 03 Jun 2003 21:00:21 -0300 Subject: measuring power usage In-Reply-To: References: <5.2.0.9.0.20030603194551.023ddb08@crux.stmarys.ca> Message-ID: <5.2.0.9.0.20030603205241.023df5a8@crux.stmarys.ca> At 04:35 PM 6/3/2003 -0700, Trent Piepho wrote: > > There is nothing in the flier that indicates that it measures rms amps and > > hence rms watts. Computers are not resistive loads and they draw > current in > >It measures power factor and frequency, so it should be doing what is >necessary to find true rms. Measuring true rms current depends on measuring in some way the area under the curve from the shape of the waveform. Non-rms meters usually rectify the voltage to all positive voltage and low pass filter it to convert to a DC value, then a fudge factor is used which assumes the waveform is a sine wave (which it isn't). ... Dave ps. what you really need for an accurate measurement of RMS power and power factor is something like: http://ca.fluke.com/caen/products/features.htm?cs_id=5179(FlukeProducts)&category=PHASE1(FlukeProducts) which unfortunely is about $1900US - ouch! There are likely other alternatives that are cheaper. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rgb at phy.duke.edu Tue Jun 3 21:09:23 2003 From: rgb at phy.duke.edu (Robert G. Brown) Date: Tue, 3 Jun 2003 21:09:23 -0400 (EDT) Subject: measuring power usage In-Reply-To: Message-ID: On Tue, 3 Jun 2003, Trent Piepho wrote: > > There is nothing in the flier that indicates that it measures rms amps and > > hence rms watts. Computers are not resistive loads and they draw current in > > It measures power factor and frequency, so it should be doing what is > necessary to find true rms. Yeah, it's hard to say whether it does or doesn't even from the brochure that comes with it. My recollection is that it "implies" that it does, since they go out of their way to indicate that appropriate usage is to measure draw of e.g. refrigerators and other loads that typically have PF<1. However, the (non-PFC) switching power supplies used in PC's only draw current for roughly the middle third of each half-cycle and have a significant harmonic load (and harmonic distortion) at 180 Hz; it isn't just a matter of evaluating _avg for w = 2\pi 60 and some small/measured delta. So it is anybody's guess if it does it right. Still, even with a 10% or so fudge on the power factor, I find the device to be very useful. Without it, it is fairly difficult to have ANY idea of the power draw of a given PC. Even if it is basically returning just (1/2)V_0 I_0 that's useful information. And all for less than $50 delivered... rgb -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb at phy.duke.edu _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From James.P.Lux at jpl.nasa.gov Wed Jun 4 00:25:19 2003 From: James.P.Lux at jpl.nasa.gov (Jim Lux) Date: Tue, 3 Jun 2003 21:25:19 -0700 Subject: Kill-A-Watt Re: measuring power usage References: <5.2.0.9.0.20030603194551.023ddb08@crux.stmarys.ca> Message-ID: <00b701c32a51$4e9dfc40$02a8a8c0@office1> I know someone who has reverse engineered the Kill-A-Watt, as well as done extensive testing at various power factors and waveforms. The thing really does measure actual instantaneous power (averaged over a few cycles), so weird current waveforms, etc. don't fool it. It definitely measures active power vs apparent power (RMS V * RMS I) as well as other parameters.. Accurate to a few percent, which is substantially better than most inexpensive DMMs. > >Yes, very lovely thingie. There are links to at least one source on the > >brahma vendors page, and I >>have<< pictures of a kill-a-watt in action, > >if I ever overcome my innate inertia and post them on the brahma photo > >tour... > > There is nothing in the flier that indicates that it measures rms amps and > hence rms watts. Computers are not resistive loads and they draw current in > a switching fashion. As a result they usually produce a lower current > reading on a non-rms meter than they are actually using. From memory the > measurements I did produced about a 20% difference in current measurements > between Fluke rms and ordinary meters. > _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From James.P.Lux at jpl.nasa.gov Wed Jun 4 00:41:19 2003 From: James.P.Lux at jpl.nasa.gov (Jim Lux) Date: Tue, 3 Jun 2003 21:41:19 -0700 Subject: Cheap PCs from Wal-Mart References: Message-ID: <00c301c32a53$8b3d2c00$02a8a8c0@office1> From: "Mark Hahn" To: "Jim Lux" Sent: Tuesday, June 03, 2003 4:32 PM Subject: Re: Cheap PCs from Wal-Mart > > > > This is really a first order approximation. You'd have to look at > > > > peripheral power, memory power, and power supply conversion and power > > > > distribution efficiency. Peripherals and memory probably scale with > > memory > > > > speed fairly linearly. > > > > > > huh? do you mean "the P4 drives ram much faster and so the ram > > > will also dissipate more power"? > > > > Precisely.. CMOS, to a first order, has power dissipation proportional to > > clock frequency. Cycle the bus at 200 MHz and it draws twice as much power > > as cycling the bus at 100 MHz. I don't know if the P4 or C3 have the same > > bus width, too? Wider buses draw more power (for the line > > drivers/receivers). > > your basic premise in all of this is that you have some kind of > computational task which simply doesn't need memory bandwidth > or FP power. yes, desktop/server CPUs are not at all optimized > for that kind of load, and never will be, so why aren't you using > a mips or arm chip? Because you want to take advantage of the millions (billions?) of dollars being spent on development for the consumer market. Sure, you could use a more highly optimized chipset, or you could even spend a few tens of millions of dollars and design your own. However, you'd not be able to reap the benefits of: 1) Extensive defacto testing of the consumer chip's internal logic... 2) Amortization of development costs across literally millions of units 3) Development tools at low cost 4) Large numbers of people familiar with the instruction set, and the development tools. One could, of course, acquire all these things with other approaches, however, at some point you have to decide, for a low volume application, whether you want to spend your money on one-off development, or just buying some number more processors. The latter approach has a graceful degradation path in the event of failures. Granted, this grossly oversimplifies what goes into those sorts of applications, but, the real point is that the fastest chip isn't always the best system solution. > > It's a huge issue... The efficiency of the PS is lower at 3.3V than at 5V or > > 12V, for instance. As far as the CPU core voltage regulator, the same > > applies... If you push power through anything, you're going to have more IR > > losses at 1.8V than at 2.5V. There's only so much copper available on the > > board to carry the current, and the pin or ball is only so big. On chip, > > except that we're talking about a whole *plane* devoted to VCC > (which can be modeled as a rather large wire), and literally hundreds > of pins. I'm guessing that loss is on the order of 2-5%. People are more than willing to spend hundreds of thousands of dollars to increase efficiency 5% on spaceflight applications. If all you got is 100W from solar panels, constrained by fundamental limits (how far you are from the sun, how much mass you're willing to devote to panels, etc.), then there are situations where 5% is a very big deal. Especially if it makes a big difference in thermal management. > > > However, consider.. a contact/trace resistance of 5 milliohms, carrying a > > current of 20 amps, disspates 2 Watts... 5 mOhm is pretty darn low.. > > especially for a pin only 25 mils on a side. > > except when there are 100 of them. P4, 478 pin package, 85 power pins, 180 ground pins.. who knows what sort of bond wires connect them internally, etc. Sure, you've got lots of pins, and lots of copper on the board, but, still, running 100W in small area is no trivial matter. > > as I said, I'm not denigrating the value of low-power chips in certain > domains. I just don't think there's a good argument for compute clusters > of low-power chips, at least not general-purpose ones. and for special > purpose, I'd suggest offloading all the hard stuff onto an FPGA or the like, > and using a CPU that doesn't even pretend to be competitive... Aha.. but isn't the whole point of cluster computing a'la Beowulf to accept some inefficiencies over a custom design in exchange for very attractive pricing of a commodity component? Otherwise, wouldn't everyone just go out and buy N-way Crays? > > regards, mark hahn. > _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From James.P.Lux at jpl.nasa.gov Wed Jun 4 00:43:17 2003 From: James.P.Lux at jpl.nasa.gov (Jim Lux) Date: Tue, 3 Jun 2003 21:43:17 -0700 Subject: measuring power usage References: <5.2.0.9.0.20030603194551.023ddb08@crux.stmarys.ca> <5.2.0.9.0.20030603205241.023df5a8@crux.stmarys.ca> Message-ID: <00ca01c32a53$d1433280$02a8a8c0@office1> Kill-A-Watt does just this.. measures actual waveforms. Quite a novel feat, really, to package it all up and sell it for $40.. What the Fluke gives you that the Kill-A-Watt doesn't is things like an external interface, calibration manuals, multiple ranges, etc. The Fluke's probably also potentially calibrateable to a NIST traceable standard, and is probably more reliable. ----- Original Message ----- From: "Dave Lane" To: "Beowulf Mailing list" Sent: Tuesday, June 03, 2003 5:00 PM Subject: Re: measuring power usage > At 04:35 PM 6/3/2003 -0700, Trent Piepho wrote: > > > > There is nothing in the flier that indicates that it measures rms amps and > > > hence rms watts. Computers are not resistive loads and they draw > > current in > > > >It measures power factor and frequency, so it should be doing what is > >necessary to find true rms. > > Measuring true rms current depends on measuring in some way the area under > the curve from the shape of the waveform. Non-rms meters usually rectify > the voltage to all positive voltage and low pass filter it to convert to a > DC value, then a fudge factor is used which assumes the waveform is a sine > wave (which it isn't). > > ... Dave > > ps. what you really need for an accurate measurement of RMS power and power > factor is something like: > http://ca.fluke.com/caen/products/features.htm?cs_id=5179(FlukeProducts)&cat egory=PHASE1(FlukeProducts) > which unfortunely is about $1900US - ouch! There are likely other > alternatives that are cheaper. > > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org > To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From dajelen at att.net Wed Jun 4 17:06:46 2003 From: dajelen at att.net (dajelen at att.net) Date: Wed, 04 Jun 2003 21:06:46 +0000 Subject: How to get started . . . Message-ID: <200306042100.h54L0K515113@NewBlue.Scyld.com> Hi all, I've been lurking on this list for a while and have finally gotten up the nerve to ask a question. How can I get started in the Beowulf arena? Ideally, I'd like to become active on a project somewhere and learn as I go. Does anyone have ideas on how this could be accomplished? Dave _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From shewa at inel.gov Wed Jun 4 18:06:51 2003 From: shewa at inel.gov (Andrew Shewmaker) Date: Wed, 04 Jun 2003 16:06:51 -0600 Subject: measuring power usage In-Reply-To: References: Message-ID: <3EDE6D7B.9030107@inel.gov> Trent Piepho wrote: >Thanks to all that responded. Cute pun for a name, but it completely thwarted >my google search for terms like "power factor" and "meter". > >BTW, is there a real searchable list archive anywhere? google groups has >lists.beowulf but it looks like they stopped archiving that in 2000. The >archives on beowulf.org aren't searchable, unless you try to use a google web >search to find pages that were indexed. > > Never mind my last email. I just remembered this one. http://marc.theaimsgroup.com/?l=beowulf&r=1&w=2 -Andrew _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From shewa at inel.gov Wed Jun 4 18:03:47 2003 From: shewa at inel.gov (Andrew Shewmaker) Date: Wed, 04 Jun 2003 16:03:47 -0600 Subject: measuring power usage In-Reply-To: References: Message-ID: <3EDE6CC3.3020608@inel.gov> Trent Piepho wrote: >Thanks to all that responded. Cute pun for a name, but it completely thwarted >my google search for terms like "power factor" and "meter". > >BTW, is there a real searchable list archive anywhere? google groups has >lists.beowulf but it looks like they stopped archiving that in 2000. The >archives on beowulf.org aren't searchable, unless you try to use a google web >search to find pages that were indexed. > I point google to geocrawler sometimes. Type "site:geocrawler.com beowulf" plus your query. I'm not sure how often they update their index of geocrawler.com though Andrew _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From Andrew.Cannon at nnc.co.uk Thu Jun 5 03:25:45 2003 From: Andrew.Cannon at nnc.co.uk (Cannon, Andrew) Date: Thu, 5 Jun 2003 08:25:45 +0100 Subject: How to get started . . . Message-ID: What sort of resources have you got? If you have a number of computers, I would recommend that you try networking them together and finding a task that they can all work on. Try using MPICH or PVM for a simple introduction to clustering. You will probably get lots of advice, some of it conflicting, some good and some bad. Search on Google etc too. The resources are out there. Have fun. Andy -----Original Message----- From: dajelen at att.net [mailto:dajelen at att.net] Sent: Wednesday, June 04, 2003 10:07 PM To: beowulf at beowulf.org Subject: How to get started . . . Hi all, I've been lurking on this list for a while and have finally gotten up the nerve to ask a question. How can I get started in the Beowulf arena? Ideally, I'd like to become active on a project somewhere and learn as I go. Does anyone have ideas on how this could be accomplished? Dave _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf *********************************************************************************** NNC Limited Booths Hall Chelford Road Knutsford Cheshire WA16 8QZ Country of Registration: United Kingdom Registered Number: 1120437 This e-mail and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this e-mail in error please notify the NNC system manager by e-mail at eadm at nnc.co.uk. *********************************************************************************** _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From mat.harris at genestate.com Thu Jun 5 05:38:20 2003 From: mat.harris at genestate.com (Mat Harris) Date: Thu, 5 Jun 2003 10:38:20 +0100 Subject: How to get started . . . In-Reply-To: References: Message-ID: <20030605093819.GA8024@genestate.com> hi, I am in the same boat. what is a good idea for a simple trial task? I am a 'beginner' c programmer so nothing to complex. I don't really know what i can do to test the small collection of machines I have. Will I have to program a certain way to get the 'cluster' effect? cheers On Thu, Jun 05, 2003 at 08:25:45 +0100, Cannon, Andrew wrote: > What sort of resources have you got? If you have a number of computers, I > would recommend that you try networking them together and finding a task > that they can all work on. Try using MPICH or PVM for a simple introduction > to clustering. You will probably get lots of advice, some of it conflicting, > some good and some bad. Search on Google etc too. The resources are out > there. > > Have fun. > > Andy > > -----Original Message----- > From: dajelen at att.net [mailto:dajelen at att.net] > Sent: Wednesday, June 04, 2003 10:07 PM > To: beowulf at beowulf.org > Subject: How to get started . . . > > > Hi all, > > I've been lurking on this list for a while and have finally gotten up > the nerve to ask a question. > > How can I get started in the Beowulf arena? > > Ideally, I'd like to become active on a project somewhere and learn > as I go. > > Does anyone have ideas on how this could be accomplished? > > Dave > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org > To change your subscription (digest mode or unsubscribe) visit > http://www.beowulf.org/mailman/listinfo/beowulf > > > *********************************************************************************** > NNC Limited > Booths Hall > Chelford Road > Knutsford > Cheshire > WA16 8QZ > > Country of Registration: United Kingdom > Registered Number: 1120437 > > This e-mail and any files transmitted with it are confidential and > intended solely for the use of the individual or entity to whom they > are addressed. If you have received this e-mail in error please notify > the NNC system manager by e-mail at eadm at nnc.co.uk. > *********************************************************************************** > > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org > To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- -- Mat Harris ( ) ASCII Ribbon Campaign mat.harris at genestate.com X against HTML on email / \ and Usenet posts * .* **. .**** .' *****. ** '. '*****. .*** **.. '****** .**** .***. '******' ****** .****. ****** ******* .****. '******' .*******. .***** ******' .*******. ******. .*****' .*******. ******. .*****' .******. .****** *****. .****** ..' *****. '**** .******. .****. .. .***** ***' .****. ..******' '***... '****. '** **** ..*******'' '*******.. **** ** ***. ..*******'' '*********. *** '* .** ..*******' '********.. ** * *' .*****'' ''*******. *** ***' ''******.. .***' ****. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 232 bytes Desc: not available URL: From dajelen at att.net Thu Jun 5 10:20:48 2003 From: dajelen at att.net (dajelen at att.net) Date: Thu, 05 Jun 2003 14:20:48 +0000 Subject: How to get started . . . Message-ID: <200306051414.h55EED514584@NewBlue.Scyld.com> Unfortunately (or fortunately), my experience is in programming, rather than physics or epidemiology or another area that had domain-specific knowledge. I've heard of some "hard" problems that would require the use of a Beowulf- type cluster, but don't know enough of the specifics to take it to programming the cluster at this point (optimization isn't even close to the table at this point.) I mentioned becoming active on a project with the idea that I could join something in process and learn how the tasks are identified and how they are solved - and what methods work best to solve problem X. How do I find a project and get to the point where I can begin to consider the programming questions? > > How can I get started in the Beowulf arena? > > find a project. something you want to compute. > > > Ideally, I'd like to become active on a project somewhere and learn > > as I go. > > cooperative projects really only work if participants > have a concrete goal in mind ("fix feature x"). > _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From dajelen at att.net Thu Jun 5 10:30:49 2003 From: dajelen at att.net (dajelen at att.net) Date: Thu, 05 Jun 2003 14:30:49 +0000 Subject: How to get started . . . Message-ID: <200306051424.h55EOG516251@NewBlue.Scyld.com> Hi Heidi, Thanks for the input, and I will check out your web page. I might have mis-stated my objective. Yes, I do have two old PCs that I can use to build a Beowulf cluster. (My resources are limited, so the pre-built cluster is not an option at the moment.) I understand the principles behind a Beowulf cluster and message-passing, problem partitioning, etc. My question was an attempt to determine/identify a project to join that could utilize the cluster (2-nodes) so that I could understand the programming better and learn more about the behavior of the cluster itself and how to optimize the cluster. I thought that joining a project that's already in process would give me a head-start - someone else would have already planned out the problem to be solved, determined the best way (or, at least, A way) to solve it, and was ready (or has been) implementing the solution. My background is programming, rather than a specific domain, so I feel that I can assist with any project that is out there. Which problem domains might be the most interesting? Are there projects that can use another programming hand? If I need to, how can I find a project and begin the process on my own? I do appreciate any and all direction you can provide, Dave > Hi Dave: > Do you have 2 old machines you could try and hook together? That's all > you need to get started.. I very detailed description of how to setup > is on my > Webpage.. > http://chaos.fullerton.edu/~heidi/heidi.html > > you can buy a ready made cluster online! An 8 node cluster will set you > back about $10k (depends on the components though.. that is a very good > one!) > heidi > > -----Original Message----- > From: dajelen at att.net [mailto:dajelen at att.net] > Sent: Wednesday, June 04, 2003 2:07 PM > To: beowulf at beowulf.org > Subject: How to get started . . . > > Hi all, > > I've been lurking on this list for a while and have finally gotten up > the nerve to ask a question. > > How can I get started in the Beowulf arena? > > Ideally, I'd like to become active on a project somewhere and learn > as I go. > > Does anyone have ideas on how this could be accomplished? > > Dave > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org > To change your subscription (digest mode or unsubscribe) visit > http://www.beowulf.org/mailman/listinfo/beowulf _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From beerli at csit.fsu.edu Wed Jun 4 23:53:29 2003 From: beerli at csit.fsu.edu (Peter Beerli) Date: Wed, 4 Jun 2003 23:53:29 -0400 (EDT) Subject: queuing systems and cpu usage? Message-ID: Hi all, I am a novice to all clustering and queueing systems. So far, my cluster needs were satisfied on ad hoc cluster to run embarrassingly parallel code (MCMC runs). After my move, I can use time on a IBM Regatta cluster (and I am impatiently waiting for my very own Linux Athlon cluster). The Regatta cluster is running loadleveler which seems to have an abysmal job-scheduling performance (it seems that currently out of the max=480 cpus only 288 are used [4 idle nodes out of 15] and many jobs (including mine) are in the queues, waiting). I would be glad to hear information what schedulers you prefer and (preferably) also get some numbers of how many nodes and cpus are idle under standard load** or other appropriate statistics (what is the appropriate statistics?). (this is not really a question for this list but some might be able to answer: is out regatta loadleveler misconfigured?) **standard load: too many users submit jobs for the queue with the longest time limit, very few medium, small length jobs, most of the jobs are obviously only using <32 cpus (a node on the regatta has 32 cpus) thanks, Peter ---- Peter Beerli, School of Computational Science and Information Technology (CSIT) Dirac Science Library, Florida State University Tallahassee, Florida 32306-4120 USA old webpage: http://evolution.genetics.washington.edu/PBhtmls/beerli.html _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From johnb at quadrics.com Thu Jun 5 12:18:03 2003 From: johnb at quadrics.com (John Brookes) Date: Thu, 5 Jun 2003 17:18:03 +0100 Subject: queuing systems and cpu usage? (Partly OT) Message-ID: <010C86D15E4D1247B9A5DD312B7F5AA7E5E1AF@stegosaurus.bristol.quadrics.com> Peter, I came across LoadLeveler on an SP3 in a former job and also found the scheduling to be pretty poor. To be fair to IBM, they readily admitted the fact. I quickly found the Maui scheduler from the Maui Supercomputing Centre (now defunct? its old url no longer works). At the time it was licensed (though free), but they were working on the legal issues to make it freely distributable. It's now a SourceForge project (so I assume they succeeded with the legal :) I'm not sure if/where a fully-supported version that plugs into LL can be found these days (or even whether the project retains the LL interface), but it did much better than either of the built-ins at that time and works well now (on other systems, at least). It's also highly configurable, so you can make it as nice (or nasty!) as you like. YM will almost certainly V, as my only experiences under IBM are from ~3yo versions, but if you can get it to work it'd probably be a Good Thing. Maui's often used as the scheduler for PBS and Sun GridEngine nowadays, so getting to know its foibles wouldn't be wasted once your Linux/Athlon cluster arrives. The project is at: http://mauischeduler.sourceforge.net/ Some information on Maui and the Maui/LL tie-in can be found at eg: http://supercluster.org/documentation/ Cheers, John Brookes Quadrics Ltd. > -----Original Message----- > From: Peter Beerli [mailto:beerli at csit.fsu.edu] > Sent: 05 June 2003 04:53 > To: Beowulf Mailing list > Subject: queuing systems and cpu usage? > > > Hi all, > I am a novice to all clustering and queueing systems. > So far, my cluster needs were satisfied on ad hoc cluster to run > embarrassingly parallel code (MCMC runs). After my move, I can use > time on a IBM Regatta cluster (and I am impatiently waiting > for my very > own Linux Athlon cluster). The Regatta cluster is running > loadleveler which seems to > have an abysmal job-scheduling performance (it seems that > currently out of the max=480 cpus > only 288 are used [4 idle nodes out of 15] and many jobs > (including mine) are in the queues, waiting). > > I would be glad to hear information what schedulers you > prefer and (preferably) > also get some numbers of how many nodes and cpus are idle > under standard load** > or other appropriate statistics (what is the appropriate statistics?). > > (this is not really a question for this list but some might > be able to answer: > is out regatta loadleveler misconfigured?) > > **standard load: too many users submit jobs for the queue > with the longest time limit, > very few medium, small length jobs, most of the jobs are > obviously only using <32 cpus > (a node on the regatta has 32 cpus) > > thanks, > Peter > ---- > Peter Beerli, > School of Computational Science and Information Technology (CSIT) > Dirac Science Library, Florida State University > Tallahassee, Florida 32306-4120 USA > old webpage: http://evolution.genetics.washington.edu/PBhtmls/beerli.html _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From sfrolov at accufo.vwh.net Thu Jun 5 10:20:47 2003 From: sfrolov at accufo.vwh.net (sfrolov at accufo.vwh.net) Date: Thu, 5 Jun 2003 08:20:47 -0600 (MDT) Subject: dual Xeon issue Message-ID: <1054822842.97498@accufo.vwh.net> Sorry for a noob-like question. We are in the process of buying a Xeon-based cluster for scientific computing (running MPI parallelized numerical models). We've benchmarked clasters offered by Microway and Aspen Systems and discovered that we are getting much better results using only one CPU per box (running the model on 6 boxes using one CPU in each box was about 50% faster than running it on 3 boxes using both CPUs). This was surprising to us since we've used to be limited by interprocessor communications, which should be a lot faster between CPUs in one box. Can anybody explain the reason for this and, more importantly, is there any way to improve this situation. Thanks, Sergei. P.S. Can anybody recommend the best performance/price ratio cluster? _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From shewa at inel.gov Thu Jun 5 12:45:54 2003 From: shewa at inel.gov (Andrew Shewmaker) Date: Thu, 05 Jun 2003 10:45:54 -0600 Subject: How to get started . . . In-Reply-To: <20030605093819.GA8024@genestate.com> References: <20030605093819.GA8024@genestate.com> Message-ID: <3EDF73C2.4080507@inel.gov> Mat Harris wrote: > hi, I am in the same boat. what is a good idea for a simple trial task? > > I am a 'beginner' c programmer so nothing to complex. I don't really know what i > can do to test the small collection of machines I have. Why, "hello world", of course =-) The Ohio Super Computer Center [1] has some good online courses and you should read Robert G. Brown's book on his site [2]. [1] http://oscinfo.osc.edu/training/ [2] http://www.phy.duke.edu/brahma/index.php > Will I have to program a certain way to get the 'cluster' effect? First, make sure you read as much as you can of the available material on the web. It sounds to me like you are still trying to understand what a beowulf cluster is. You are probably not distinguishing between high performance clusters running parallelized codes and High Availability (HA) clusters, which are concerned with failover and load balancing. The beowulf FAQ explains things like this (see www.beowulf.org) and www.lcic.org has links to information about all types of clusters. Beowulf clusters must be programmed in a certain way. They don't automatically make an application run faster. However, there are projects that are attempting to make a cluster of computers look more like one Single System Image. OpenMosix (openmosix.sf.net) and SSI for Linux (ssic-linux.sf.net) are two projects that are working on providing a single process space, distributed shared memory, and more. Both of these are more on the HA end of things, but you can still run parallel codes on them. Some of their features might increase the overhead and lower the performance of parallel applications. These systems may also have difficulty scaling up to several hundreds or thousands of nodes. Bproc (bproc.sf.net) was designed to create a second generation of beowulf clusters although first generation clusters, using rsh or ssh to start processes remotely, are still more common. Bproc provides a single process space for the cluster and scales up to at least 1024 nodes...Pink (www.lanl.gov/projects/pink/) is the largest one I know of. Only the master node has a full Linux installation while the slave nodes are totally dependent on the master. Scyld (www.scyld.com) and Clustermatic (www.clustermatic.org) can help you set up a bproc based cluster. -Andrew -- Andrew Shewmaker, Associate Engineer Phone: 1-208-526-1276 Idaho National Eng. and Environmental Lab. P.0. Box 1625, M.S. 3605 Idaho Falls, Idaho 83415-3605 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From timm at fnal.gov Thu Jun 5 14:45:11 2003 From: timm at fnal.gov (Steven Timm) Date: Thu, 5 Jun 2003 13:45:11 -0500 (CDT) Subject: dual Xeon issue In-Reply-To: <1054822842.97498@accufo.vwh.net> Message-ID: I'm not sure if I can explain the problem or not, but a crucial piece of information that anyone would need to know to explain it is--do you have the hyper-threading enabled or not? Up until now we have found that Linux tends to allocate two processes to the extra logical cpu's supplied with hyperthreading without much regard to whether they are different physical cpu's or not. This can slow performance down.. doubly so if you are dependent on memory bandwidth. Steve ------------------------------------------------------------------ Steven C. Timm (630) 840-8525 timm at fnal.gov http://home.fnal.gov/~timm/ Fermilab Computing Division/Core Support Services Dept. Assistant Group Leader, Scientific Computing Support Group Lead of Computing Farms Team On Thu, 5 Jun 2003 sfrolov at accufo.vwh.net wrote: > Sorry for a noob-like question. We are in the process of buying a Xeon-based cluster for scientific computing (running MPI parallelized numerical models). We've benchmarked clasters offered by Microway and Aspen Systems and discovered that we are getting much better results using only one CPU per box (running the model on 6 boxes using one CPU in each box was about 50% faster than running it on 3 boxes using both CPUs). This was surprising to us since we've used to be limited by interprocessor communications, which should be a lot faster between CPUs in one box. Can anybody explain the reason for this and, more importantly, is there any way to improve this situation. > > Thanks, > > Sergei. > > P.S. Can anybody recommend the best performance/price ratio cluster? > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org > To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf > _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From siegert at sfu.ca Thu Jun 5 15:17:03 2003 From: siegert at sfu.ca (Martin Siegert) Date: Thu, 5 Jun 2003 12:17:03 -0700 Subject: dual Xeon issue In-Reply-To: <1054822842.97498@accufo.vwh.net> References: <1054822842.97498@accufo.vwh.net> Message-ID: <20030605191703.GB6918@stikine.ucs.sfu.ca> On Thu, Jun 05, 2003 at 08:20:47AM -0600, sfrolov at accufo.vwh.net wrote: > Sorry for a noob-like question. We are in the process of buying a Xeon-based > cluster for scientific computing (running MPI parallelized numerical models). > We've benchmarked clasters offered by Microway and Aspen Systems and discovered > that we are getting much better results using only one CPU per box (running the > model on 6 boxes using one CPU in each box was about 50% faster than running it > on 3 boxes using both CPUs). This was surprising to us since we've used to be > limited by interprocessor communications, which should be a lot faster between > CPUs in one box. Can anybody explain the reason for this and, more importantly, > is there any way to improve this situation. Since you mentioned that you are limited by interprocessor communications the reason for the slowdown in the dual-processor configuration is probably that the communication between processors on different nodes becomes the bottleneck. Since the link between nodes is now shared between two processors on each node you effectively halve the per processor bandwidth. Remedy: either buy single processor boxes or double the bandwidth (cf channel bonding). -- Martin Siegert Manager, Research Services Academic Computing Services phone: (604) 291-4691 Simon Fraser University fax: (604) 291-4242 Burnaby, British Columbia email: siegert at sfu.ca Canada V5A 1S6 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From orion at cora.nwra.com Thu Jun 5 16:03:58 2003 From: orion at cora.nwra.com (Orion Poplawski) Date: Thu, 05 Jun 2003 14:03:58 -0600 Subject: HyperThreading in Cluster Message-ID: <3EDFA22E.8090902@cora.nwra.com> This is partly prompted by Steven Timm's recent comment about HyperThreading, as well as something I read in a magazine the other day stating the HPC clusters need to run with HyperThreading off. Can anyone shed more light on the current status of HyperThreading in parallel computing? I could see where allocating two compute threads to a single processor might bog things down (swapping data sets in/out of cache, etc.), but it would seem ideal to allocate one compute thread per processor, and let the other virtual processor help with OS/interrupt type operations. Does the linux scheduler have such control yet? -- Orion Poplawski System Administrator 303-415-9701 x222 Colorado Research Associates/NWRA FAX: 303-415-9702 3380 Mitchell Lane, Boulder CO 80301 www.co-ra.com _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From math at velocet.ca Thu Jun 5 15:06:32 2003 From: math at velocet.ca (Ken Chase) Date: Thu, 5 Jun 2003 15:06:32 -0400 Subject: dual Xeon issue In-Reply-To: <1054822842.97498@accufo.vwh.net>; from sfrolov@accufo.vwh.net on Thu, Jun 05, 2003 at 08:20:47AM -0600 References: <1054822842.97498@accufo.vwh.net> Message-ID: <20030605150632.J61960@velocet.ca> On Thu, Jun 05, 2003 at 08:20:47AM -0600, sfrolov at accufo.vwh.net's >P.S. Can anybody recommend the best performance/price ratio cluster? "DEPENDS" like always. Depends on what you're doing. Depends on if your code scales well, or superlinearly, or not at all. Depends on the cost of power, floor space, cooling, gear, gear being shipped to you, humans to assemble, install, test, and maintain it. Depends on whats free and what's not. (see my post on .edu vs .com clusters earlier this week). Even more generally, depends on what you mean by 'performance'. Does it have to sing and dance well for morons who like blinkenlights? Does it have to run one single job so freakin' fast that the organization can publish results as bragging rights, but makes it ass-slow running 30 jobs at the same time by researchers fighting over resources? Or did you mean 'best throughput in jobs per month'? What did you mean? 'depends'. /kc -- Ken Chase, math at velocet.ca * Velocet Communications Inc. * Toronto, Canada Wiznet Velocet DSL.ca Datavaults 24/7: 416-967-4414 tollfree: 1-866-353-0363 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From ctierney at hpti.com Thu Jun 5 15:35:16 2003 From: ctierney at hpti.com (Craig Tierney) Date: 05 Jun 2003 13:35:16 -0600 Subject: dual Xeon issue In-Reply-To: <1054822842.97498@accufo.vwh.net> References: <1054822842.97498@accufo.vwh.net> Message-ID: <1054841715.11421.136.camel@woody> On Thu, 2003-06-05 at 08:20, sfrolov at accufo.vwh.net wrote: > Sorry for a noob-like question. We are in the process of buying a Xeon-based cluster for scientific computing (running MPI parallelized numerical models). We've benchmarked clasters offered by Microway and Aspen Systems and discovered that we are getting much better results using only one CPU per box (running the model on 6 boxes using one CPU in each box was about 50% faster than running it on 3 boxes using both CPUs). This was surprising to us since we've used to be limited by interprocessor communications, which should be a lot faster between CPUs in one box. Can anybody explain the reason for this and, more importantly, is there any way to improve this situation. > You are probably pushing the memory bandwidth on the motherboard. On most Xeon motherboards, the memory bandwidth can be exhausted by 1 cpu. Rarely do you get a 2x speedup when using the 2nd cpu. Your results are typical. You could go with single CPU motherboards. The new 800Mhz FSB looks quite attractive. However, the odds are that when you include the extra interconnect and the other infrastructure that the dual Xeon system will provide better price/performance. It does all depend on your code. I doubt that a Microway or an Aspen Systems cluster is going to perform that differently if configured similarly. Make sure that Hyperthreading is off and you are using a good compiler (Intel). Look at what support you get for your money. Things are going to break, and knowing how the vendor is going fix your problems is important. Craig > Thanks, > > Sergei. > > P.S. Can anybody recommend the best performance/price ratio cluster? > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org > To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- Craig Tierney _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From glen at cert.ucr.edu Thu Jun 5 15:39:18 2003 From: glen at cert.ucr.edu (Glen Kaukola) Date: Thu, 05 Jun 2003 12:39:18 -0700 Subject: ups units Message-ID: <3EDF9C66.5090106@cert.ucr.edu> Hi everyone, I've got a question about UPS units. If they're rated at 1500VA, then how many computers with 400 watt power supplies should they be able to handle? And how many computers with 550 watt power supplies? If someone could let me know how you calculate the answer I'd really appricate it. The reason I ask is because we have a number of Cyberpower and APC UPS units rated at 1500VA. They each have 2 computers plugged into them, and seemed to be working fine, as me and my boss tested them by unplugging each while the computers plugged into them were running. But recently we had a power spike or whatever you'd call it, that lasted less than a second, and afterwards I noticed that many of the computers plugged into the Cyberpower units had rebooted. I contacted the company we purchased the Cyberpower units from, and he in turn contacted Cyberpower. Cyberpower informed him that there shouldn't be more than one computer plugged into each UPS unit, and said something which I didn't understand about 12 amps, 125 volts, and 400 X 2 watts being more than 1500AV. It all sounded rather fishy to me, so I was hoping that somebody on this list could clear things up for me. Thanks in advance, Glen _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From lindahl at keyresearch.com Thu Jun 5 16:13:37 2003 From: lindahl at keyresearch.com (Greg Lindahl) Date: Thu, 5 Jun 2003 13:13:37 -0700 Subject: dual Xeon issue In-Reply-To: <1054822842.97498@accufo.vwh.net> References: <1054822842.97498@accufo.vwh.net> Message-ID: <20030605201337.GB1513@greglaptop.internal.keyresearch.com> On Thu, Jun 05, 2003 at 08:20:47AM -0600, sfrolov at accufo.vwh.net wrote: > This was surprising to us since we've used to be limited by > interprocessor communications, which should be a lot faster between > CPUs in one box. How much memory bandwidth does your code want? Xeon systems usually have a memory bus which can be maxed out by a single cpu. As a by-the-way, Opteron systems, if you get the memory affinity going correctly, have memory bandwidth that scales as you add more processors. Also, your request for the "best performance/price ratio cluster" doesn't have an answer -- different people are going to see different performance depending on what their application is. That's doubly true for unusual apps like yours. -- greg _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From josip at lanl.gov Thu Jun 5 16:12:32 2003 From: josip at lanl.gov (Josip Loncaric) Date: Thu, 05 Jun 2003 14:12:32 -0600 Subject: dual Xeon issue In-Reply-To: <1054822842.97498@accufo.vwh.net> References: <1054822842.97498@accufo.vwh.net> Message-ID: <3EDFA430.9060102@lanl.gov> sfrolov at accufo.vwh.net wrote: > [...] We've benchmarked clasters offered by Microway and > Aspen Systems and discovered that we are getting much better results > using only one CPU per box (running the model on 6 boxes using one > CPU in each box was about 50% faster than running it on 3 boxes using > both CPUs). Your results are only slightly worse than expected. This is a generic situation with memory bandwidth limited codes, where a dual CPU box typically counts as only 1.5 single CPU boxes (approximately) due to shared resources (memory bandwidth, network card, certain OS functions). However, for a code which spins mostly in cache and whose processes to not communicate much, a dual CPU box can match two singles. Shared memory communication is much faster than network, and it can be a big win if used between tightly coupled processes within a dual CPU box, but there is a caveat. Polling two different MPI devices (shmem and net) can impose its own penalty. Moreover, since most problems do not partition neatly into loosely coupled pairs of tightly coupled processes, exploiting the higher bandwidth of shared memory communication is tricky. For all these reasons, some people prefer singles. Others like denser packaging so they buy duals. When a high end network is involved (Myriniet, Quadrics), duals are clearly more cost effective since they require fewer network ports for the same number of CPUs. Sincerely, Josip _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From vijayv at CSE.MsState.EDU Thu Jun 5 19:15:15 2003 From: vijayv at CSE.MsState.EDU (Vijay Velusamy) Date: Thu, 5 Jun 2003 18:15:15 -0500 (CDT) Subject: dual Xeon issue In-Reply-To: <1054822842.97498@accufo.vwh.net> Message-ID: Which MPI implementation are you using for the benchmarking. This would actually depend on the implementation specifics. The implementation needs to need to be aware of and take advantage of SMP's. Also are you using a SMP kernel for this? -Vijay On Thu, 5 Jun 2003 sfrolov at accufo.vwh.net wrote: > Sorry for a noob-like question. We are in the process of buying a Xeon-based cluster for scientific computing (running MPI parallelized numerical models). We've benchmarked clasters offered by Microway and Aspen Systems and discovered that we are getting much better results using only one CPU per box (running the model on 6 boxes using one CPU in each box was about 50% faster than running it on 3 boxes using both CPUs). This was surprising to us since we've used to be limited by interprocessor communications, which should be a lot faster between CPUs in one box. Can anybody explain the reason for this and, more importantly, is there any way to improve this situation. > > Thanks, > > Sergei. > > P.S. Can anybody recommend the best performance/price ratio cluster? > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org > To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf > _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From nuclearnine at prodigy.net Thu Jun 5 22:32:47 2003 From: nuclearnine at prodigy.net (Nuclear Nine) Date: Thu, 5 Jun 2003 22:32:47 -0400 Subject: How to get started . . . References: <200306051424.h55EOG516251@NewBlue.Scyld.com> Message-ID: <003e01c32bd3$ebbfdf60$88574b43@Homer> I am a newbie as well. What does everyone think about the Rocks cluster distribution? According to their website it seems like the easiest solution. Any comments. http://www.rocksclusters.org/Rocks/ ----- Original Message ----- From: To: "Fearn, Heidi" Cc: Sent: Thursday, June 05, 2003 10:30 AM Subject: RE: How to get started . . . Hi Heidi, Thanks for the input, and I will check out your web page. I might have mis-stated my objective. Yes, I do have two old PCs that I can use to build a Beowulf cluster. (My resources are limited, so the pre-built cluster is not an option at the moment.) I understand the principles behind a Beowulf cluster and message-passing, problem partitioning, etc. My question was an attempt to determine/identify a project to join that could utilize the cluster (2-nodes) so that I could understand the programming better and learn more about the behavior of the cluster itself and how to optimize the cluster. I thought that joining a project that's already in process would give me a head-start - someone else would have already planned out the problem to be solved, determined the best way (or, at least, A way) to solve it, and was ready (or has been) implementing the solution. My background is programming, rather than a specific domain, so I feel that I can assist with any project that is out there. Which problem domains might be the most interesting? Are there projects that can use another programming hand? If I need to, how can I find a project and begin the process on my own? I do appreciate any and all direction you can provide, Dave > Hi Dave: > Do you have 2 old machines you could try and hook together? That's all > you need to get started.. I very detailed description of how to setup > is on my > Webpage.. > http://chaos.fullerton.edu/~heidi/heidi.html > > you can buy a ready made cluster online! An 8 node cluster will set you > back about $10k (depends on the components though.. that is a very good > one!) > heidi > > -----Original Message----- > From: dajelen at att.net [mailto:dajelen at att.net] > Sent: Wednesday, June 04, 2003 2:07 PM > To: beowulf at beowulf.org > Subject: How to get started . . . > > Hi all, > > I've been lurking on this list for a while and have finally gotten up > the nerve to ask a question. > > How can I get started in the Beowulf arena? > > Ideally, I'd like to become active on a project somewhere and learn > as I go. > > Does anyone have ideas on how this could be accomplished? > > Dave > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org > To change your subscription (digest mode or unsubscribe) visit > http://www.beowulf.org/mailman/listinfo/beowulf _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From andrewxwang at yahoo.com.tw Thu Jun 5 23:18:16 2003 From: andrewxwang at yahoo.com.tw (=?big5?q?Andrew=20Wang?=) Date: Fri, 6 Jun 2003 11:18:16 +0800 (CST) Subject: queuing systems and cpu usage? In-Reply-To: Message-ID: <20030606031816.7347.qmail@web16804.mail.tpe.yahoo.com> If there are a lot of people submit jobs to Loadleveler, while Loadlevler is scheduling, then the performance is not good. To reduce the impact of job submition, make sure the spool directory is on a local file system. (a gridengine cluster was having problem because the spool directory was on NFS) -- I am assume Loadlevler is similar to gridengine -- spool directory access performance is important. Otherwise, it must be the scheduler performance, then use Maui. Lastly, is Loadlevler free? Thanks, Andrew. --- Peter Beerli ???? > Hi all, > I am a novice to all clustering and queueing > systems. > So far, my cluster needs were satisfied on ad hoc > cluster to run > embarrassingly parallel code (MCMC runs). After my > move, I can use > time on a IBM Regatta cluster (and I am impatiently > waiting for my very > own Linux Athlon cluster). The Regatta cluster is > running loadleveler which seems to > have an abysmal job-scheduling performance (it seems > that currently out of the max=480 cpus > only 288 are used [4 idle nodes out of 15] and many > jobs (including mine) are in the queues, waiting). > > I would be glad to hear information what schedulers > you prefer and (preferably) > also get some numbers of how many nodes and cpus are > idle under standard load** > or other appropriate statistics (what is the > appropriate statistics?). > > (this is not really a question for this list but > some might be able to answer: > is out regatta loadleveler misconfigured?) > > **standard load: too many users submit jobs for the > queue with the longest time limit, > very few medium, small length jobs, most of the > jobs are obviously only using <32 cpus > (a node on the regatta has 32 cpus) > > thanks, > Peter > ---- > Peter Beerli, > School of Computational Science and Information > Technology (CSIT) > Dirac Science Library, Florida State University > Tallahassee, Florida 32306-4120 USA > old webpage: > http://evolution.genetics.washington.edu/PBhtmls/beerli.html > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org > To change your subscription (digest mode or > unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf ----------------------------------------------------------------- ??? Yahoo!?? ??????? - ???????????? http://fate.yahoo.com.tw/ _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From smuelas at mecanica.upm.es Fri Jun 6 03:15:32 2003 From: smuelas at mecanica.upm.es (Santiago Muelas) Date: Fri, 6 Jun 2003 09:15:32 +0200 Subject: dual Xeon issue In-Reply-To: <1054822842.97498@accufo.vwh.net> References: <1054822842.97498@accufo.vwh.net> Message-ID: <20030606091532.6e0201fb.smuelas@mecanica.upm.es> I am also quite a "newby" to this beowulf-world. A couple of months ago I asked a question in this list because my program did not attained my expectations. I was convinced of an ethernet problem and asked for giga-ethernet... I receive many very nice answers, but one specially was the very right one. It came from "master" Brown. He told me to fine-tune my program before doing any other thing... and he was right. Now, I am learning with four Ahtlon XP2200 computers and sometime I tested them against a dual 2-Xeon-2200 Dell , expensive one, plus, sometime another dual with 2 Ahtlon 1800. So, as finally the results are --for me-- impressive and the size of my little "toy" could be similar to yours, and despite the fact that I am the perfect oppposite to an expert in clusters, here are my recomendations: - Do not expend money in "better" things. In this aspect, quantity is better than "theoretical-quality" - Normal ethernet is great and more than enough for 4,5 or 6 machines.(and an standard switch) - Choose the best price-quality possibility. - Use "xmpi" and work on your program until the "red" has almost disappeared. And here are my results, tested with a finite element program written by my friend Lee Margetts abd fine-tuned by him: - Time to run an iterative, 100.000 equations program on one Ahtlon: 58 sec. On 2 machines 37 sec.(I have limited the iterations to the minimum) - The same on one CPU with the Xeon: 62 sec. On 2 CPU 40 sec. The Xeon has 1Gb. memory for each processor. The Ahtlons 512Kb each. Now a bigger problem: 330.000 equations. In this case and due to memory requirements I was not able to run the program on just 2 Ahtlons, so I run it on the four. With the same total memory as the Dell-Xeon, I tested on this one with the two processors. Clearly, the results are not comparable but give an interesting result: - Time to run on the 2-Xeon: 2m56sc. - Time to run on the 4 Ahtlons: 1m24sc. So less than one half. Now, the cost: Each Athlon has 1 processor, 512Mb, one motherboard (standard with ethernet included) and one HD (chipest, but new). Total price in euros: 350-400. Cost of 4 Un. --> 1400 euros. Dell dual-Xeon with 2Gb. total ram, scsi disk and.... 4500-5000 euros. So the decission is simple. I will increase imediately the ram on the Ahtlons to attain 4Gb. (cost of 2 Gb= 200 euros. I use the standard cheapest OEM memory cards. Excellent, BTW) All that means that I can have a nice beowulf with 12 computers based on Ahtlons (now Ahtlon XP 2400 has the best quality/price), for the same cost as a prestigious dual computer. I think this is the reason of beowulf philosophy !! Thanks to all for your patience. On Thu, 5 Jun 2003 08:20:47 -0600 (MDT) sfrolov at accufo.vwh.net wrote: > Sorry for a noob-like question. We are in the process of buying a Xeon-based cluster for scientific computing (running MPI parallelized numerical models). We've benchmarked clasters offered by Microway and Aspen Systems and discovered that we are getting much better results using only one CPU per box (running the model on 6 boxes using one CPU in each box was about 50% faster than running it on 3 boxes using both CPUs). This was surprising to us since we've used to be limited by interprocessor communications, which should be a lot faster between CPUs in one box. Can anybody explain the reason for this and, more importantly, is there any way to improve this situation. > > Thanks, > > Sergei. > > P.S. Can anybody recommend the best performance/price ratio cluster? > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org > To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- Santiago Muelas Profesor de Resistencia de Materiales y C?lculo de Estructuras ETSI de Caminos, Canales y Puertos (U.P.M) smuelas at mecanica.upm.es http://w3.mecanica.upm.es/~smuelas _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From jhearns at freesolutions.net Fri Jun 6 05:23:38 2003 From: jhearns at freesolutions.net (John Hearns) Date: Fri, 6 Jun 2003 10:23:38 +0100 Subject: PS2 cluster Message-ID: <004d01c32c0d$6b442190$8461cdc2@DREAD> I remember when the PS2 first came out, ideas were kicked around on the Beowulf list about clustering them and using the vector units in them (and using Firewire networking) Looks like NCS are actually doing this, using the PS2 Linux kit and Ethernet: http://access.ncsa.uiuc.edu/Releases/05.27.03_Playing_th.html Wow! _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From Thomas.Alrutz at dlr.de Fri Jun 6 07:28:24 2003 From: Thomas.Alrutz at dlr.de (Thomas Alrutz) Date: Fri, 06 Jun 2003 13:28:24 +0200 Subject: dual Xeon issue References: <1054822842.97498@accufo.vwh.net> Message-ID: <3EE07AD8.70206@dlr.de> > Sorry for a noob-like question. We are in the process of buying a Xeon-based cluster > for scientific computing (running MPI parallelized numerical models). > We've benchmarked clasters offered by Microway and Aspen Systems and discovered that we > are getting much better results using only one CPU per box (running the model on 6 boxes > using one CPU in each box was about 50% faster than running it on 3 boxes using both CPUs). > This was surprising to us since we've used to be limited by interprocessor communications, > which should be a lot faster between CPUs in one box. Can anybody explain the reason for > this and, more importantly, is there any way to improve this situation. > > Thanks, > > Sergei. > The so called "poor performance" in parallel mode when using 2 CPUs per node (box), comes due to the fact that the memory bandwidth is limited. If you are employing 2 CPUs then the 2 jobs share there memory interface. Otherwise if you are employing only 1 CPU and no other job is running on the node, you have the full memory bandwidth exclusive. This yields to the performance gain of only 50% when both CPUs are working. Our benchmarks using the unstructured Navier-Stokes TAU-code solver for a numerical calculation had shown the same behavior. We used a wing-body-engine aircraft configuration with 2 million grid points for the benchmarks and employed a full multigrid cycle to test the communication (here MPI-calls for a domain decomposition model). The performance gain (faster main loop times for a iteration) that we got for the 2. CPU are : Athlon MP FSB 133 1.6 GHz 72% Xeon Rambus FSB 100 2.0 GHz 55% Xeon DDR FSB 100 2.4 GHz 47% Xeon DDR FSB 100 2.8 GHz 43% Xeon DDR FSB 133 2.4 GHz 50% These are the values for 1 node in use. We have watched a decrease of the performance gain for the 2. CPU, when more then 1 node were used for the calculation (e.g. Xeon 2.4 GHz FSB 100 on 8 nodes only 37% for the 2. CPU). So if you are using a code that needs a lot of memory transfers, then you have decide, if the gain of the performance is worth the cost of the 2. CPU! But if you are looking for good benchmark results you might try an Opteron system (NUMA instead of Bus). Thomas -- __/|__ | Dipl.-Math. Thomas Alrutz /_/_/_/ | DLR Institute of Aerodynamics and Flow Technology |/ | Numerical Methods DLR | Bunsenstr. 10 | D-37073 Goettingen/Germany _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rgb at phy.duke.edu Fri Jun 6 08:41:38 2003 From: rgb at phy.duke.edu (Robert G. Brown) Date: Fri, 6 Jun 2003 08:41:38 -0400 (EDT) Subject: dual Xeon issue In-Reply-To: <20030606091532.6e0201fb.smuelas@mecanica.upm.es> Message-ID: On Fri, 6 Jun 2003, Santiago Muelas wrote: > - Do not expend money in "better" things. In this aspect, quantity is > better than "theoretical-quality" > - Normal ethernet is great and more than enough for 4,5 or 6 > machines.(and an standard switch) > - Choose the best price-quality possibility. > - Use "xmpi" and work on your program until the "red" has almost > disappeared. Ah, chela, you have done well! Honorable Master beg to offer only one small correction. Suggest do not assume problem of others same as problem of self. Sometimes problem need better quality, expensive network, dual processors, sometimes not. YMMV is the True Tao. However, observation that True Tao is optimize price/performance make Master very happy, very happy. Chela is Enlightened. Master also pleased that chela discover superlinear speedup and more. Suggest chela investigate memory bound feature of program -- perhaps dual cpu, one memory bus a problem? rgb Da "master" feelin' a wee bit lunatic this morning:-) Curiously, I had just written the following poem and added it to Hot Tea...time to go look in the mirror and see if those bloodshot eyes are lookin' a little crazy once again. -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb at phy.duke.edu Brilliance and Madness True brilliance, my friend, on the edge of madness lies So that many a great master has lunatic eyes. 'Tis an irregular stair that mounts to the skies And thus we conclude: it's crazy to be wise! From: http://www.phy.duke.edu/~rgb/Poetry/hot_tea.php _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From canon at nersc.gov Fri Jun 6 11:53:48 2003 From: canon at nersc.gov (canon at nersc.gov) Date: Fri, 06 Jun 2003 08:53:48 -0700 Subject: queuing systems and cpu usage? In-Reply-To: Message from Peter Beerli of "Wed, 04 Jun 2003 23:53:29 EDT." Message-ID: <200306061553.h56FrmS4018724@pookie.nersc.gov> Peter, We use LL on our ~6000 cpu SP system (416 Nighthawk II with two plane colony interconnect). We are able to get > 90% scheduled utilization even with very big jobs (>2048). While LL is hardly perfect, you should be able to get more out of it than you stated. I don't handle the LL config for the system, so I can tell you any magic bullets. I'm sure our SP admins would be happy to give you some suggestions though. On our 200 node Linux cluster we currently use LSF which is quite good. However, we are looking hard at SGE since it is free, open source, and maturing rapidly. Let me know if you would like more info. --Shane ------------------------------------------------------------------------ Shane Canon voice: 510-486-6981 PSDF Project Lead fax: 510-486-7520 National Energy Research Scientific Computing Center 1 Cyclotron Road Mailstop 943-256 Berkeley, CA 94720 canon at nersc.gov ------------------------------------------------------------------------ _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From shewa at inel.gov Fri Jun 6 12:46:22 2003 From: shewa at inel.gov (Andrew Shewmaker) Date: Fri, 06 Jun 2003 10:46:22 -0600 Subject: How to get started . . . In-Reply-To: <003e01c32bd3$ebbfdf60$88574b43@Homer> References: <200306051424.h55EOG516251@NewBlue.Scyld.com> <003e01c32bd3$ebbfdf60$88574b43@Homer> Message-ID: <3EE0C55E.2040602@inel.gov> Nuclear Nine wrote: > I am a newbie as well. What does everyone think about the Rocks cluster > distribution? According to their website it seems like the easiest > solution. Any comments. Rocks is very easy to use. I found it a bit easier to set up than OSCAR, but I think OSCAR has some other advantages (e.g., it can be in can be installed on several distributions). Both Rocks and OSCAR are great ways to build first generation beowulf clusters, and I hear Mandrake's is nice too. However, I believe all of these use NFS for at least /home. This may not affect you much on a small clusters, but it depends on your application. I avoid using NFS whenever I can. When/if you encounter bottlenecks with NFS, you should consider looking at lustre (www.lustre.org), PVFS (www.parl.clemson.edu/pvfs/), and v9fs (www.clustermatic.org). In addition to using NFS, these first generation cluster environments use ssh or rsh to start processes on the compute nodes. These work okay, but bproc (bproc.sf.net) provides a better way IMO. -Andrew -- Andrew Shewmaker, Associate Engineer Phone: 1-208-526-1276 Idaho National Eng. and Environmental Lab. P.0. Box 1625, M.S. 3605 Idaho Falls, Idaho 83415-3605 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From wdinkel at atipa.com Fri Jun 6 12:39:20 2003 From: wdinkel at atipa.com (Will Dinkel) Date: 06 Jun 2003 11:39:20 -0500 Subject: HyperThreading in Cluster In-Reply-To: <3EDFA22E.8090902@cora.nwra.com> References: <3EDFA22E.8090902@cora.nwra.com> Message-ID: <1054917560.9645.265.camel@zappa> As far as I know, the majority of the kernels that you will find in today's Linux distributions are HyperThreading "aware", but not HyperThreading "intelligent". As you mention below, the scheduler doesn't know that allocating two compute threads to a single processor is a BAD thing, especially when it leaves the other physical processor idle. Ingo Molnar has a patch to the latest 2.5.X kernel releases that addresses this issue. He uses per-physical-cpu runqueues and load balancing to ensure that the system load is correctly balanced between physical and virtual cpus. It's experimental, updated fairly regularly, delivers promising results, and can be found here: http://people.redhat.com/mingo/O(1)-scheduler/ On Thu, 2003-06-05 at 15:03, Orion Poplawski wrote: > This is partly prompted by Steven Timm's recent comment about > HyperThreading, as well as something I read in a magazine the other day > stating the HPC clusters need to run with HyperThreading off. Can > anyone shed more light on the current status of HyperThreading in > parallel computing? > > I could see where allocating two compute threads to a single processor > might bog things down (swapping data sets in/out of cache, etc.), but > it would seem ideal to allocate one compute thread per processor, and > let the other virtual processor help with OS/interrupt type operations. > Does the linux scheduler have such control yet? -- Will Dinkel Atipa Technologies -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part URL: From vngalchin at yahoo.com Fri Jun 6 20:17:04 2003 From: vngalchin at yahoo.com (Galchin Vasili) Date: Fri, 6 Jun 2003 17:17:04 -0700 (PDT) Subject: How is fault detection handled in Beowulf? Message-ID: <20030607001704.38843.qmail@web12203.mail.yahoo.com> Hello, How is fault detection handled in Beowulf? Is any attempt made to take a node out of the pool if a fault is detected? Regards, Vasili Galchin __________________________________ Do you Yahoo!? Yahoo! Calendar - Free online calendar with sync to Outlook(TM). http://calendar.yahoo.com _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From andrewxwang at yahoo.com.tw Sat Jun 7 09:45:06 2003 From: andrewxwang at yahoo.com.tw (=?big5?q?Andrew=20Wang?=) Date: Sat, 7 Jun 2003 21:45:06 +0800 (CST) Subject: GridEngine WorkShop 2003 (SGE 6.0, Globus 2/3 integration, etc...) Message-ID: <20030607134506.60981.qmail@web16804.mail.tpe.yahoo.com> Is the workshop free? Is there a website to list the programs? Thanks, Andrew. --- Ron Chen wrote: > Last years presentation available at: > http://gridengine.sunsource.net/project/gridengine/workshop22-24.04.02/proceedings.html > > -Ron > > > --- Monika Grobecker wrote: > > Dear Grid Engine Partner > > > > You are cordially invited to our second Grid Engine > > Development Workshop, > > September 22 to 24, 2003, in Regensburg, Germany. > > > > The objectives of the workshop are: > > > > Get together with the SGE developers and our > > development partners - > > Give and get an indepth overview on the ongoing and > > next Grid Engine > > developments - > > Present and discuss the development planned by Sun > > for the next major Grid > > Engine release (6.0) - > > Exchange research and development plans and > > coordinate activities. > > > > See below for a rough tentative agenda and travel > > details. > > > > We need your workshop registration by June 20, 2003 > > to enable us making the > > necessary hotel and conference room arrangements. > > Please register by simply > > sending e-mail to monika.grobecker at sun.com. > > > > Tentative Agenda: > > > > Sept 22: Welcome and Grid Engine Status Update > > Welcome > > Grid Engine Project Update > > Project activity and site updates > > New subprojects (GEP, JAM, Jgrid) > > Grid Engine Roadmap > > Architecture changes and feature enhancements > > Timetable > > Sun development plan > > Project gaps and contribution opportunities > > > > Sept 23: Partner and Contributor's Day > > Partners and contributors present their work with or > > around Grid Engine, e.g. > > on: > > Development contributions > > Globus Toolkit 2 and 3 integration; OGSA/Web > > services relation > > Grid Engine Portal > > End user scenarios: problems solved and lessons > > learned > > > > Sept 24: SIG and Workshop Day > > Identify Special Interest Groups > > Parallel SIG workshops > > Presentation and discussion of workshop results > > Adjourn > > > > Travel is easiest through the airport of Munich and > > rental car. There is a > > choice of hotel rooms from EUR 62 to 109, including > > breakfast. Try to arrive > > Saturday. Oktoberfest in Munich starts September 20 > > this year. Regensburg is > > very much worthwile a visit too, as many of you > > know. > > Details will follow soon. > > > > Regards > > > > Wolfgang > > Fritz > > Monika > > > > ************************************************ > > Sun Microsystems Gridware GmbH > > Monika Grobecker > > Dr.-Leo-Ritter-Str. 7 > > D-93049 Regensburg Phone: +49-941-3075-100 > > Germany Fax: +49-941-3075-222 > > > > mailto:monika.grobecker at sun.com > > ************************************************ > > > > > > > > > ----------------------------------------------------------------- ??? Yahoo!?? ??????? - ???????????? http://fate.yahoo.com.tw/ _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From andrewxwang at yahoo.com.tw Sat Jun 7 09:45:08 2003 From: andrewxwang at yahoo.com.tw (=?big5?q?Andrew=20Wang?=) Date: Sat, 7 Jun 2003 21:45:08 +0800 (CST) Subject: GridEngine WorkShop 2003 (SGE 6.0, Globus 2/3 integration, etc...) Message-ID: <20030607134508.49866.qmail@web16802.mail.tpe.yahoo.com> Is the workshop free? Is there a website to list the programs? Thanks, Andrew. --- Ron Chen wrote: > Last years presentation available at: > http://gridengine.sunsource.net/project/gridengine/workshop22-24.04.02/proceedings.html > > -Ron > > > --- Monika Grobecker wrote: > > Dear Grid Engine Partner > > > > You are cordially invited to our second Grid Engine > > Development Workshop, > > September 22 to 24, 2003, in Regensburg, Germany. > > > > The objectives of the workshop are: > > > > Get together with the SGE developers and our > > development partners - > > Give and get an indepth overview on the ongoing and > > next Grid Engine > > developments - > > Present and discuss the development planned by Sun > > for the next major Grid > > Engine release (6.0) - > > Exchange research and development plans and > > coordinate activities. > > > > See below for a rough tentative agenda and travel > > details. > > > > We need your workshop registration by June 20, 2003 > > to enable us making the > > necessary hotel and conference room arrangements. > > Please register by simply > > sending e-mail to monika.grobecker at sun.com. > > > > Tentative Agenda: > > > > Sept 22: Welcome and Grid Engine Status Update > > Welcome > > Grid Engine Project Update > > Project activity and site updates > > New subprojects (GEP, JAM, Jgrid) > > Grid Engine Roadmap > > Architecture changes and feature enhancements > > Timetable > > Sun development plan > > Project gaps and contribution opportunities > > > > Sept 23: Partner and Contributor's Day > > Partners and contributors present their work with or > > around Grid Engine, e.g. > > on: > > Development contributions > > Globus Toolkit 2 and 3 integration; OGSA/Web > > services relation > > Grid Engine Portal > > End user scenarios: problems solved and lessons > > learned > > > > Sept 24: SIG and Workshop Day > > Identify Special Interest Groups > > Parallel SIG workshops > > Presentation and discussion of workshop results > > Adjourn > > > > Travel is easiest through the airport of Munich and > > rental car. There is a > > choice of hotel rooms from EUR 62 to 109, including > > breakfast. Try to arrive > > Saturday. Oktoberfest in Munich starts September 20 > > this year. Regensburg is > > very much worthwile a visit too, as many of you > > know. > > Details will follow soon. > > > > Regards > > > > Wolfgang > > Fritz > > Monika > > > > ************************************************ > > Sun Microsystems Gridware GmbH > > Monika Grobecker > > Dr.-Leo-Ritter-Str. 7 > > D-93049 Regensburg Phone: +49-941-3075-100 > > Germany Fax: +49-941-3075-222 > > > > mailto:monika.grobecker at sun.com > > ************************************************ > > > > > > > > > ----------------------------------------------------------------- ??? Yahoo!?? ??????? - ???????????? http://fate.yahoo.com.tw/ _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From andrewxwang at yahoo.com.tw Sat Jun 7 10:05:27 2003 From: andrewxwang at yahoo.com.tw (=?big5?q?Andrew=20Wang?=) Date: Sat, 7 Jun 2003 22:05:27 +0800 (CST) Subject: queuing systems and cpu usage? (Partly OT) In-Reply-To: <010C86D15E4D1247B9A5DD312B7F5AA7E5E1AF@stegosaurus.bristol.quadrics.com> Message-ID: <20030607140528.10165.qmail@web16812.mail.tpe.yahoo.com> Just one point to share. The latest version of Maui is available at: http://www.supercluster.org/maui/ The version on http://mauischeduler.sourceforge.net/ is the `Molokini Edition', from what I have learnt from other people, never use this version! This is what people on the xcat list told me, I did not try it, but seems like those are valid points: http://bohnsack.com/lists/archives/xcat-user/2385.html Thanks, Andrew. --- John Brookes >> Maui's often used as the scheduler for PBS and Sun > GridEngine nowadays, so > getting to know its foibles wouldn't be wasted once > your Linux/Athlon > cluster arrives. > > The project is at: > http://mauischeduler.sourceforge.net/ > > Some information on Maui and the Maui/LL tie-in can > be found at eg: > http://supercluster.org/documentation/ > > Cheers, > > John Brookes > Quadrics Ltd. > > > > -----Original Message----- > > From: Peter Beerli [mailto:beerli at csit.fsu.edu] > > Sent: 05 June 2003 04:53 > > To: Beowulf Mailing list > > Subject: queuing systems and cpu usage? > > > > > > Hi all, > > I am a novice to all clustering and queueing > systems. > > So far, my cluster needs were satisfied on ad hoc > cluster to run > > embarrassingly parallel code (MCMC runs). After my > move, I can use > > time on a IBM Regatta cluster (and I am > impatiently waiting > > for my very > > own Linux Athlon cluster). The Regatta cluster is > running > > loadleveler which seems to > > have an abysmal job-scheduling performance (it > seems that > > currently out of the max=480 cpus > > only 288 are used [4 idle nodes out of 15] and > many jobs > > (including mine) are in the queues, waiting). > > > > I would be glad to hear information what > schedulers you > > prefer and (preferably) > > also get some numbers of how many nodes and cpus > are idle > > under standard load** > > or other appropriate statistics (what is the > appropriate statistics?). > > > > (this is not really a question for this list but > some might > > be able to answer: > > is out regatta loadleveler misconfigured?) > > > > **standard load: too many users submit jobs for > the queue > > with the longest time limit, > > very few medium, small length jobs, most of the > jobs are > > obviously only using <32 cpus > > (a node on the regatta has 32 cpus) > > > > thanks, > > Peter > > ---- > > Peter Beerli, > > School of Computational Science and Information > Technology (CSIT) > > Dirac Science Library, Florida State University > > Tallahassee, Florida 32306-4120 USA > > old webpage: > http://evolution.genetics.washington.edu/PBhtmls/beerli.html > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org > To change your subscription (digest mode or > unsubscribe) visit > http://www.beowulf.org/mailman/listinfo/beowulf > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org > To change your subscription (digest mode or > unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf ----------------------------------------------------------------- ??? Yahoo!?? ??????? - ???????????? http://fate.yahoo.com.tw/ _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From James.P.Lux at jpl.nasa.gov Sat Jun 7 12:26:51 2003 From: James.P.Lux at jpl.nasa.gov (Jim Lux) Date: Sat, 7 Jun 2003 09:26:51 -0700 Subject: ups units References: <3EDF9C66.5090106@cert.ucr.edu> Message-ID: <005201c32d11$9aaf9cc0$02a8a8c0@office1> First off... VA is not watts... Watts is active power ( integral of instantaneous voltage * instantaneous current). VA is RMS volts * RMS amps. If you hook up an RMS ammeter and measure the current draw, multiply that by the RMS voltage, you'll actually be measuring VA (Volt-Amps) If, and only if, the load is resistive so the current and voltage waveforms are "in phase" and identical, will VA=Watts. For all other cases (where PC power supplies happen to be), the current isn't necessarily in phase with the voltage, nor is the current waveform a nice sinusoid. The ratio between the "watts" and the "VA" is called the power factor. So, if you have a device that runs at 120V, and draws 10 amps, it has a VA of 1200. But, maybe the current is out of phase a bit, or non-sinusoidal, so if you actually measure the power (i.e. how fast the wheel spins on the meter), (maybe you put the load in a sealed, insulated box and measure the temperature rise?)... you find that it dissipates, say 1000 Watts. The power factor, in this case, would be 1000/1200 or 0.83 (which, by the way, is a typical kind of pf for a switching power supply). The other thing to bear in mind is the "Watt" number advertised for a power supply (i.e. a 400W power supply) may not bear much connection to how much power is actually consumed or supplied. The nameplate must (in a regulatory sense) give you the RMS current and RMS voltage, so you can get VA. It will also usually give Watts. Depending on the marking requirements, it might give power factor, as well. (The accuracy and reliability of nameplate markings is whole 'nother story... Unless something catches fire, or gets potentially blamed for a disaster, it's unlikely that the stuff actually gets checked.. certainly not on a "per each unit manufactured" basis) --------- Now we come to the other big problem.. inrush current. The nameplate ratings are for the "steady state" situation. Many devices, particularly those with large capacitors inside (i.e. power supplies) draw s everal times their steady state current when power is first applied. (AC Motors also draw 3-4 times their running current when starting, too). The circuitbreaker in your panel doesn't trip because it has a "inverse time/current" characteristic.. A short overload won't trip it, but a sustained one will (remember that the purpose of the circuit breaker ("overcurrent protection" or OCP) is to keep the place from burning down; NOT to protect your equipment. The short inrush might lead to a "light blink" as the load pulls down the supply voltage a bit, but won't usually trip the breaker. In fuses, this is the difference between "fast blow" and "slow blow".. although there are literally dozens of curves available. For circuit breakers, they actually have ones with knobs to set the curve, used in industrial applications (where you're turning on 10 kW of lights at a shot, or starting a 50 HP compressor, etc.) Your UPS, on the other hand, probably does overcurrent protection with a very fast acting current sensor. The OCP on the load side of the UPS is designed to protect the UPS (i.e. don't fry the output transistors). I suspect that this is a difference between the inexpensive UPS's and the "big iron room full of batteries" UPSs.. The former is really designed to run one PC (and it's associated fairly small inrush), the latter is designed with slow trip and some overhead in the design. ----- Original Message ----- From: "Glen Kaukola" To: "Beowulf" Sent: Thursday, June 05, 2003 12:39 PM Subject: ups units > I contacted the company we purchased the Cyberpower units from, and he > in turn contacted Cyberpower. Cyberpower informed him that there > shouldn't be more than one computer plugged into each UPS unit, and said > something which I didn't understand about 12 amps, 125 volts, and 400 X > 2 watts being more than 1500AV. It all sounded rather fishy to me, so I > was hoping that somebody on this list could clear things up for me. 12A * 125V (which sounds like a worst case, to me.. 125 is really high for a line voltage) = 1500 VA... That's probably their max output spec.. Consider your 400W power supplies.. If they had 0.8 power factor , the current each will draw is (400/110) / 0.8 = 4.5 Amps A 2:1 inrush would be perfectly reasonable... _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rgb at phy.duke.edu Sat Jun 7 14:29:33 2003 From: rgb at phy.duke.edu (Robert G. Brown) Date: Sat, 7 Jun 2003 14:29:33 -0400 (EDT) Subject: HPC conference near Raleigh NC... Message-ID: Dear Raleigh area beowulfers, Just to let you know, there is a conference/workshop on HPC in the Raleigh-Durham-Chapel Hill area next week at Wake Tech Community College. Wake Tech is one of several area colleges that are participants in a major NSF project designed to foster training programs for cluster administrators and programmers to meet what are (I think correctly) perceived to be exponentially growing demand. I've been informally advising them through the grantseeking process, although I'm not an actual participant. They've got a nice list of speakers for Friday with tracks focussing on bioinformatics (one of the major interests in this area with all the genetics and drug projects and companies), infrastructure, and tools and programming. In addition some talks will focus on educational issues that cluster users don't ordinarily focus on. The conference is free, and will even provide complimentary meals to people who register (so they can get a head count ahead of time). Area students are especially encouraged to attend. The conference website is at: http://www.highperformancecomputing.org/wtcc/wtcc061203.html and as of a few minutes ago still doesn't have the actual agenda posted. I have a copy however (as I'm one of the speakers) and am putting a copy at http://www.phy.duke.edu/~rgb/agenda.pdf (as I don't think the list manager really likes 200K attachments:-). If you're in the area and are interested in grid/cluster/beowulf computing (or any of the specific topics covered by talks on the agenda), give it a look and consider attending -- it should be fun. Again, be sure to contact Michele Blackmon and register if you plan to attend so that they can be sure to have enough food and room and so forth; this is their first such conference and may be surprised at the number of people in the area that are interested. Duke people interested in going are welcome to contact me about hitching a ride -- I've got a big car;-) rgb -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb at phy.duke.edu _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rgb at phy.duke.edu Sat Jun 7 14:47:29 2003 From: rgb at phy.duke.edu (Robert G. Brown) Date: Sat, 7 Jun 2003 14:47:29 -0400 (EDT) Subject: ups units In-Reply-To: <005201c32d11$9aaf9cc0$02a8a8c0@office1> Message-ID: On Sat, 7 Jun 2003, Jim Lux wrote: > First off... > VA is not watts... Watts is active power ( integral of instantaneous voltage ^^^^^^^^ time average (to pick nits -- VA and Watts >>are<< identical units. But Jim knew that; he meant time integral divided by the time, or average:-) > * instantaneous current). VA is RMS volts * RMS amps. Everything else Jim said was just peachy. I'd amplify a couple of points about PF -- if all your systems have a PF of 0.8 to 0.9 (typical for a non-PFC switching power supply) then as he noted a system that draws 120 watts (average power) on a 120 volt line needs to draw perhaps 120 Amps, not the 100 Amps you might expect on the basis of the average. However, power companies have to be able to deliver PEAK currents, not average currents, and usually charge you on the basis of those peaks as it limits their ability to distribute power elsewhere on their grid. These higher, off phase currents also have negative implications for your primary supply transformers, which can heat and actually become damaged or have a shorted lifetime if continuously operated near their nominal peak capacity. Finally, voltage brownouts during the peak draw period can limit the systems' ability to draw current during precisely the period they need to be drawing current to provide sustained power. This, in turn, can cause systems to lose some of their innate ability to buffer and ride out line surges caused by (for example) lots of systems turning on and off at once. Which in turn means that there can be more downtime, more expense and lost work associated with downtime, more hassle. The moral of all which is: a) Consider using harmonic mitigating transformers and other line conditioning on new "large" facilities intended to hold lots of cluster nodes with switching power supplies and get the wiring done by competent people who understand the issues involved and not just joe electrician off of the street; b) Otherwise leave a healthy margin of surplus capacity on each line when considering what you can run on old circuits or even new circuits with no special power factor/harmonic correction measures taken. rgb -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb at phy.duke.edu _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From James.P.Lux at jpl.nasa.gov Sat Jun 7 19:23:51 2003 From: James.P.Lux at jpl.nasa.gov (Jim Lux) Date: Sat, 7 Jun 2003 16:23:51 -0700 Subject: ups units References: Message-ID: <000801c32d4b$dbb25340$02a8a8c0@office1> ----- Original Message ----- From: "Robert G. Brown" To: "Jim Lux" Cc: "Glen Kaukola" ; "Beowulf" Sent: Saturday, June 07, 2003 11:47 AM Subject: Re: ups units > On Sat, 7 Jun 2003, Jim Lux wrote: > > > First off... > > VA is not watts... Watts is active power ( integral of instantaneous voltage > ^^^^^^^^ > time average > > (to pick nits -- VA and Watts >>are<< identical units. But Jim knew > that; he meant time integral divided by the time, or average:-) Indeed.. from a dimensional analysis standpoint, both VA and Watts (and VARs for that matter) are all rate of energy.. Joules/sec And, yes, it's really the time average of the product of the instantaneous power (i.e. volts*amps) > > > * instantaneous current). VA is RMS volts * RMS amps. > > Everything else Jim said was just peachy. I'd amplify a couple of > points about PF -- if all your systems have a PF of 0.8 to 0.9 (typical > for a non-PFC switching power supply) then as he noted a system that > draws 120 watts (average power) on a 120 volt line needs to draw perhaps > 120 Amps, not the 100 Amps you might expect on the basis of the average. And, watch out for low line voltage... Switching supplies are kind of interesting.. they're "constant power" devices, so as the line voltage drops, the current increases. This is unlike most conventional resistive/inductive loads, where as voltage drops, so does the current. Gang up a raft o' PCs so that your line voltage gets sucked down to 110 or 105 volts, and the current goes up correspondingly.. increasing the IR losses in the distribution circuitry. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From crhea at mayo.edu Sun Jun 8 12:00:07 2003 From: crhea at mayo.edu (Cris Rhea) Date: Sun, 8 Jun 2003 11:00:07 -0500 (CDT) Subject: Oddball networking question... Message-ID: <200306081600.h58G07wT005701@sijer.mayo.edu> Currently, my small RedHat-based cluster is connected via 100Mb Ethernet (small Cisco switch). I'm adding another node to act as a dedicated NFS server and would like to get as much bandwidth as possible between that system and the switch. At this time, I'm not is a position to upgrade everything to GigE, but would like to use a pair of GigE ports between the NFS server and the switch. The Cisco switch is a 24 port 10/100 switch, but copper GigE cards are available for the "option slots". >From the reading I've done, Linux suports channel bonding; but all nodes would require multiple Ethernet cards and also be running the bonding drivers. I'm looking only to bond two GigE links between my NFS server and the switch. Is this possible? If so, what's it called under Linux so I can read up on it? TIA- --- Cris ---- Cristopher J. Rhea Mayo Foundation Research Computing Facility Pavilion 2-25 crhea at Mayo.EDU Rochester, MN 55905 Fax: (507) 266-4486 (507) 284-0587 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From jhearns at freesolutions.net Sun Jun 8 13:47:22 2003 From: jhearns at freesolutions.net (John Hearns) Date: 08 Jun 2003 18:47:22 +0100 Subject: Oddball networking question... In-Reply-To: <200306081600.h58G07wT005701@sijer.mayo.edu> References: <200306081600.h58G07wT005701@sijer.mayo.edu> Message-ID: <1055094442.1656.1.camel@harwood.home> On Sun, 2003-06-08 at 17:00, Cris Rhea wrote: > > The Cisco switch is a 24 port 10/100 switch, but copper GigE cards are available > for the "option slots". > > From the reading I've done, Linux suports channel bonding; but all nodes would > require multiple Ethernet cards and also be running the bonding drivers. Why not use a single GigE card and one GBIC in the switch. I don't understand why you think two might be needed. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From bryan at westernfire.com Sun Jun 8 14:25:16 2003 From: bryan at westernfire.com (Bryan Klein) Date: Sun, 8 Jun 2003 11:25:16 -0700 Subject: Adaptive MPI on Beowulf... Message-ID: <000101c32deb$5044da80$6401a8c0@Thunder> I was curious if anyone here has ever run jobs using Adaptive MPI. http://finesse.cs.uiuc.edu/research/ampi/ I run some fortran/c++ code using MPI and this Charm++ AMPI stuff caught my eye. Bryan Klein Western Fire Center, Inc. bryan at westernfire.com _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From nfalano at hotmail.com Sun Jun 8 13:32:41 2003 From: nfalano at hotmail.com (Norman Alano) Date: Mon, 09 Jun 2003 01:32:41 +0800 Subject: need help re: script Message-ID: greetings! i need help re: my scenario. how i can create a script that i will only create account on the server and it automatically creates account on the nodes. and how i can create a script that i will shutdown the server node and it will automatically shutdown the nodes. thank you very much for your help cheers norman alano _________________________________________________________________ MSN 8 with e-mail virus protection service: 2 months FREE* http://join.msn.com/?page=features/virus _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From jmdavis at mail2.vcu.edu Sun Jun 8 13:50:25 2003 From: jmdavis at mail2.vcu.edu (Mike Davis) Date: Sun, 08 Jun 2003 13:50:25 -0400 Subject: Oddball networking question... In-Reply-To: <200306081600.h58G07wT005701@sijer.mayo.edu> References: <200306081600.h58G07wT005701@sijer.mayo.edu> Message-ID: <3EE37761.5040605@mail2.vcu.edu> The cisco switches support trunking (which is what you're looking for), but I'm not sure if this ability is limited to only connections between cisco products (the only way that I've done it). I would think that for a small cluster, you wouldn't need to trunk the GigE, but your app may be so disk intensive that it could be necessary. My channel bonded clusters all use two switches rather than a virtual switch. But, I'm in the process of replacing the networking of these older clusters with GigE. My new clusters were purchased with GigE. Channel bonding even FastE offers good performance increases (up to 190Mb/sec) in netperf. The problems that I've had are occasional race conditions leading to inetd dying and nodes locking up on occasion. Good luck, Mike Cris Rhea wrote: >Currently, my small RedHat-based cluster is connected via 100Mb Ethernet >(small Cisco switch). > >I'm adding another node to act as a dedicated NFS server and would like to get >as much bandwidth as possible between that system and the switch. > >At this time, I'm not is a position to upgrade everything to GigE, but would >like to use a pair of GigE ports between the NFS server and the switch. > >The Cisco switch is a 24 port 10/100 switch, but copper GigE cards are available >for the "option slots". > >From the reading I've done, Linux suports channel bonding; but all nodes would >require multiple Ethernet cards and also be running the bonding drivers. > >I'm looking only to bond two GigE links between my NFS server and the switch. >Is this possible? If so, what's it called under Linux so I can read up on it? > >TIA- > >--- Cris > > > > >---- > Cristopher J. Rhea Mayo Foundation > Research Computing Facility Pavilion 2-25 > crhea at Mayo.EDU Rochester, MN 55905 > Fax: (507) 266-4486 (507) 284-0587 >_______________________________________________ >Beowulf mailing list, Beowulf at beowulf.org >To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf > > > _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From tekka99 at libero.it Mon Jun 9 05:10:00 2003 From: tekka99 at libero.it (=?iso-8859-1?Q?Gianluca_Cecchi?=) Date: Mon, 9 Jun 2003 11:10:00 +0200 Subject: =?iso-8859-1?Q?lm=5Fsensors_configuration_tips_for_dual_athlon_MB?= Message-ID: Hello, I have several dual athlon servers as comp. nodes that are not in CED rooms. To avoid too much noise I'm using fans with low rpm values (around 4200) for the cpus of those servers. The mainboards used are Chaintech 7KDD, based on AMD 762 + 768 chipset. The cpus used are MP2200+ (1800MHz). >From the bios I'm able to set as the highest shutdown temp 70 centigrad degrees and sometimes I have spontaneus shutdown due to these settings, I imagine (if I reboot immediately I see about 63 degrees for the Cpus and 55 degrees for the MB). I would like to have any suggestion based on the experience of this list's users. The poer supply I'm using is a 450wat extreme. The o.s. is Slackware 9.0 with 2.4.20 kernel (also tried some 2.4.21-rc kernels). On the systems there are four scsi drives, as they are used as head node also. I have installed lm_sensors, to disable bios check and replace with sw checks and alert messages. But I have some problems in configuring the sensors.conf file. Any hints for dual athlon based systems? Based on sensors-detect command, at boot I load these modules and run these commands: /sbin/modprobe i2c-proc # I2C adapter drivers /sbin/modprobe i2c-amd756 /sbin/modprobe i2c-isa # I2C chip drivers /sbin/modprobe it87 /sbin/modprobe eeprom /usr/local/bin/sensors -s but the output is quite strange. I would like also to have some feedback on maximum safe temp I can set for Athlon MP 2200+ cpus and for MB cpu, and if the bios settings available is actually too restrictive as I think, or if I'm tttally wrong and unwise... How can I gain, if I add a case fan? Any hint? Thanks in advance. Bye, Gianluca _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rmyers1400 at attbi.com Mon Jun 9 05:24:57 2003 From: rmyers1400 at attbi.com (Robert Myers) Date: Mon, 09 Jun 2003 05:24:57 -0400 Subject: lm_sensors configuration tips for dual athlon MB In-Reply-To: References: Message-ID: <3EE45269.4090209@attbi.com> Gianluca Cecchi wrote: >I have several dual athlon servers as comp. nodes that are not in CED rooms. >To avoid too much noise I'm using fans with low rpm values (around 4200) for >the cpus of those servers. > If the cabinet will permit it (and it is worth cutting some custom holes if you're up for that sort of thing), a ducted chassis fan is a good way to move alot of air quietly. Might not have to let your CPU's run so hot. RM _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From shewa at inel.gov Mon Jun 9 09:46:19 2003 From: shewa at inel.gov (Andrew Shewmaker) Date: Mon, 09 Jun 2003 07:46:19 -0600 Subject: need help re: script In-Reply-To: References: Message-ID: <3EE48FAB.6090308@inel.gov> Norman Alano wrote: > > greetings! > > i need help re: my scenario. how i can create a script that i will only > create account on the server and it automatically creates account on the > nodes. and how i can create a script that i will shutdown the server > node and it will automatically shutdown the nodes. You might want to take a look at the C3 tools. http://www.csm.ornl.gov/torc/C3/ Andrew -- Andrew Shewmaker, Associate Engineer Phone: 1-208-526-1276 Idaho National Eng. and Environmental Lab. P.0. Box 1625, M.S. 3605 Idaho Falls, Idaho 83415-3605 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From josip at lanl.gov Mon Jun 9 11:17:12 2003 From: josip at lanl.gov (Josip Loncaric) Date: Mon, 09 Jun 2003 09:17:12 -0600 Subject: ups units In-Reply-To: <005201c32d11$9aaf9cc0$02a8a8c0@office1> References: <3EDF9C66.5090106@cert.ucr.edu> <005201c32d11$9aaf9cc0$02a8a8c0@office1> Message-ID: <3EE4A4F8.2090405@lanl.gov> Jim Lux wrote: > First off... > VA is not watts... Watts is active power ( integral of instantaneous voltage > * instantaneous current). VA is RMS volts * RMS amps Exactly! VA rating is only an upper bound on Watts... > The ratio between the "watts" and the "VA" is called the power factor. So, > if you have a device that runs at 120V, and draws 10 amps, it has a VA of > 1200. But, maybe the current is out of phase a bit, or non-sinusoidal, so > if you actually measure the power (i.e. how fast the wheel spins on the > meter), (maybe you put the load in a sealed, insulated box and measure the > temperature rise?)... you find that it dissipates, say 1000 Watts. The > power factor, in this case, would be 1000/1200 or 0.83 (which, by the way, > is a typical kind of pf for a switching power supply). Kill-a-Watt (available at Radio Shack for $30) can measure VA, Watts, and power factor. At the input to my UPS unit (supporting a dual CPU server, a fast ethernet switch, etc.), it reports about 185 VA or 130 W or power factor 0.7 (somewhat lower than Jim's example). > Now we come to the other big problem.. inrush current. This server draws less than 1 A RMS, but when it is powered up it draws almost 2 A RMS briefly, which matches Glen's 2:1 inrush estimate. Since the power-on phase is brief, a slow acting 20 A circuit breaker can support 12 such machines. However, as Jim pointed out, one needs to be more careful in observing VA limits of commodity UPS units whose electronics are sensitive to instantaneous load. Finally, the power supply rating printed on your PC is only an upper bound. While your PC may require a 300-400 W power supply (mainly to supply sufficient inrush current to the PC's components), under normal circumstances the PC will dissipate less than half this maximum power rating. So, if you have a room with X PCs, here is an estimate of the sufficient air conditioner capacity: you will have to remove roughly X*150W of heat (i.e. roughly X*500 BTU/h) plus a safety margin. Sincerely, Josip P.S. This "half the power supply rating" power dissipation estimate is very crude, but can be useful when more accurate figures are hard to obtain. BTW, in sizing power supplies, the aggregate Watt rating does not tell the whole story, because a PS supplies multiple voltages, each with its own current limit. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From jhearns at freesolutions.net Mon Jun 9 12:08:26 2003 From: jhearns at freesolutions.net (John Hearns) Date: Mon, 9 Jun 2003 17:08:26 +0100 Subject: ups units References: <3EDF9C66.5090106@cert.ucr.edu> <005201c32d11$9aaf9cc0$02a8a8c0@office1> <3EE4A4F8.2090405@lanl.gov> Message-ID: <035101c32ea1$63364820$8461cdc2@DREAD> > > Kill-a-Watt (available at Radio Shack for $30) can measure VA, Watts, > and power factor. At the input to my UPS unit (supporting a dual CPU > server, a fast ethernet switch, etc.), it reports about 185 VA or 130 W > or power factor 0.7 (somewhat lower than Jim's example). > For us European types, a quick search of the Radio Shack UK site did not reveal this product. So I did some Googling, and found a european version at http://www.prodigit.com/e2000m.htm (referenced in a thread on Tesla coils, with Jim Lux as a contributor. Do you really have such a thing, Jim?) It looks as if it has a Euro-standard power plug/socket. A UK distributor is listed as: http://www.etps.co.uk/ They just told me there is no 'UK tooled' version - ie. no UK three pin socket. They are getting me some idea of pricings for the Euro model. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From gotero at linuxprophet.com Mon Jun 9 21:23:46 2003 From: gotero at linuxprophet.com (Glen Otero) Date: Mon, 9 Jun 2003 18:23:46 -0700 Subject: Linux clusters for bioinformatics Message-ID: <2EBBCCCA-9AE2-11D7-8F9E-000393911A90@linuxprophet.com> Beowulfers- Just wanted to announce my cluster distro project, BioBrew, to the list. BioBrew is an open source Linux cluster distribution that is enhanced for bioinformaticists and life scientists. BioBrew automates cluster installation, includes all the HPC software a cluster enthusiast needs, and contains the NCBI toolkit, BLAST, mpiBLAST, HMMER, ClustalW, GROMACS, WISE, and EMBOSS, and will be available free of charge at Bioinformatics.org. BioBrew is in late stage beta testing and will be released this month at ClusterWorld, June 24th-26th in San Jose, where I'll be giving a talk on BioBrew. Please drop by if you're in the neighborhood and contact me if you'd like a free pass to get into the exhibit hall. I'm conducting a free online presentation on BioBrew later this week. If you'd like information on the presentation, please contact me off list. Glen Otero, Ph.D. Linux Prophet gotero at linuxprophet.com _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From sgaudet at wildopensource.com Tue Jun 10 08:27:28 2003 From: sgaudet at wildopensource.com (Stephen Gaudet) Date: Tue, 10 Jun 2003 08:27:28 -0400 Subject: Two Linux server specialists combine Message-ID: <3EE5CEB0.40301@wildopensource.com> Penguin Computing, a server maker specializing in machines running Linux, has signed an agreement to acquire Scyld Computing, the company founded by a pioneer of "Beowulf" Linux supercomputers, the companies plan to announce Tuesday. more: http://msnbc-cnet.com.com/2100-1010_3-1014970.html?type=pt&part=msnbc&tag=alert&form=feed&subj=cnetnews -- Steve Gaudet Wild Open Source (home office) ---------------------- Bedford, NH 03110 pH:603-488-1599 cell:603-498-1600 http://www.wildopensource.com _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From mbasa at incas.ro Mon Jun 9 08:00:09 2003 From: mbasa at incas.ro (Mihai Basa) Date: Mon, 9 Jun 2003 21:00:09 +0900 Subject: need help re: script Message-ID: <20030609210009.M72045@incas.ro> Hi Norman, I'm curious what the others will respond to your question, but here's one answer: http://seti.home.ro/add_cluster_user.sh It's a script that we've used for a while (we're using ROCKS now). It uses C3 (Cluster Command and Control) which you can get from http://www.csm.ornl.gov/torc/C3/ - which is a great tool to have around a cluster anyhow, and pretty easy to install. The script works ok, but isn't 100% abuse-proof. It creates users on all nodes and sets them up for password-less ssh logins. You might actually want to write your own script, but I hope this could help you to a quick start. To shutdown the nodes (once you've got C3) all you need is to type: cexec poweroff ...and voila, instant silence! :) Mihai NA> greetings! NA> i need help re: my scenario. how i can create a script that i will only NA> create account on the server and it automatically creates account on the NA> nodes. and how i can create a script that i will shutdown the server node NA> and it will automatically shutdown the nodes. NA> thank you very much for your help NA> cheers NA> norman alano NA> _________________________________________________________________ NA> MSN 8 with e-mail virus protection service: 2 months FREE* NA> http://join.msn.com/?page=features/virus NA> _______________________________________________ NA> Beowulf mailing list, Beowulf at beowulf.org NA> To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rene.storm at emplics.com Tue Jun 10 10:37:47 2003 From: rene.storm at emplics.com (Rene Storm) Date: Tue, 10 Jun 2003 16:37:47 +0200 Subject: AW: need help re: script Message-ID: <29B376A04977B944A3D87D22C495FB23012746@vertrieb.emplics.com> Hi, use nis (ypserv) to manage your accounts. http://www.linux.org/docs/ldp/howto/NIS-HOWTO/index.html For your execution scripts you could do something like: #!/bin/bash # . /etc/profile for the environment, needs passwordless rsh NODES="node02 node01" for NODE in $NODES; do echo "---------- $NODE" rsh $NODE ". /etc/profile; $*" echo done If you halt or reboot your machines, this machine should be last, to execute the script on ;o) There are many tools for that out there. Take a look at sourceforge. Cu Ren? -----Urspr?ngliche Nachricht----- Von: Norman Alano [mailto:nfalano at hotmail.com] Gesendet: Sonntag, 8. Juni 2003 19:33 An: beowulf at beowulf.org Betreff: need help re: script greetings! i need help re: my scenario. how i can create a script that i will only create account on the server and it automatically creates account on the nodes. and how i can create a script that i will shutdown the server node and it will automatically shutdown the nodes. thank you very much for your help cheers norman alano _________________________________________________________________ MSN 8 with e-mail virus protection service: 2 months FREE* http://join.msn.com/?page=features/virus _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From andy at ideafix.litec.csic.es Tue Jun 10 12:40:23 2003 From: andy at ideafix.litec.csic.es (A.P.Manners) Date: Tue, 10 Jun 2003 17:40:23 +0100 Subject: NAS Parallel Benchmarks for Current Hardware Message-ID: <3EE609F7.BE430A1E@ideafix.litec.csic.es> I am looking to put together a small cluster for numerical simulation and have been surprised at how few NPB benchmark results using current hardware I can find via google. Pointers to either a repository or individual results gratefully received. Is there some other (reasonably widely used) suite of benchmarks that is relevant for large scale numerical simulation? Thanks for any info. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From 8nrf at qlink.queensu.ca Tue Jun 10 11:42:26 2003 From: 8nrf at qlink.queensu.ca (Nathan Fredrickson) Date: Tue, 10 Jun 2003 11:42:26 -0400 Subject: need help re: script In-Reply-To: <29B376A04977B944A3D87D22C495FB23012746@vertrieb.emplics.com> Message-ID: > For your execution scripts you could do something like: > > #!/bin/bash > # . /etc/profile for the environment, needs passwordless rsh > NODES="node02 node01" > > for NODE in $NODES; do > echo "---------- $NODE" > rsh $NODE ". /etc/profile; $*" > echo > done > >There are many tools for that out there. Take a look at sourceforge. The distributed shell is a perl script that is similar to above, but has more features. It has become my favorite cluster tool. It can be found at: http://dsh.sourceforge.net/ Nathan _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From M.Arndt at science-computing.de Tue Jun 10 12:30:36 2003 From: M.Arndt at science-computing.de (Michael Arndt) Date: Tue, 10 Jun 2003 18:30:36 +0200 Subject: Comparison of Cluster price ratios Message-ID: <20030610183036.A26812@blnsrv1.science-computing.de> Hello * some weeks ago someone mailed, that he collects on a website information concerning price/performance rations for different linux cluster types (CPU / Disk / Mem) i am looking for info about somewhat actual numbers what to calculate for management info as numbers for Dollars / Megabyte Diskspace for Megabucks / Gigabyte SCSI / IDE, dito Mem, and CPU --- i know about the limitations of those numbers, but if someone has done something in this direction for linux clusters, please remail me the pointer TIA Micha _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rgupta at cse.iitkgp.ernet.in Tue Jun 10 16:22:40 2003 From: rgupta at cse.iitkgp.ernet.in (Rakesh Gupta) Date: Wed, 11 Jun 2003 01:52:40 +0530 (IST) Subject: AW: need help re: script In-Reply-To: <29B376A04977B944A3D87D22C495FB23012746@vertrieb.emplics.com> Message-ID: Hi, A simple solution would be to create an account on the server node and then do a copy of /etc/passwd /etc/groups /etc/shadow ( use rcp or scp or ssh ) to the cleint node for creating and managing account. For shutdown write another script ( basically run shutdown using ssh on the cleint ) wait for sufficient time and the shutdown the server. I have the scripts for this and could mail you if you are interested. Regards rakesh On Tue, 10 Jun 2003, Rene Storm wrote: > Hi, > > > use nis (ypserv) to manage your accounts. > http://www.linux.org/docs/ldp/howto/NIS-HOWTO/index.html > > For your execution scripts you could do something like: > > #!/bin/bash > # . /etc/profile for the environment, needs passwordless rsh > NODES="node02 node01" > > for NODE in $NODES; do > echo "---------- $NODE" > rsh $NODE ". /etc/profile; $*" > echo > done > > If you halt or reboot your machines, this machine should be last, to execute the script on ;o) > > There are many tools for that out there. Take a look at sourceforge. > > Cu > Ren? > > > -----Urspr?ngliche Nachricht----- > Von: Norman Alano [mailto:nfalano at hotmail.com] > Gesendet: Sonntag, 8. Juni 2003 19:33 > An: beowulf at beowulf.org > Betreff: need help re: script > > > > greetings! > > i need help re: my scenario. how i can create a script that i will only > create account on the server and it automatically creates account on the > nodes. and how i can create a script that i will shutdown the server node > and it will automatically shutdown the nodes. > > thank you very much for your help > > cheers > > norman alano > > _________________________________________________________________ > MSN 8 with e-mail virus protection service: 2 months FREE* > http://join.msn.com/?page=features/virus > > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org > To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org > To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf > -- ---------------------------------------------------------------------- Rakesh Gupta Research Consultant Computer Science and Engineering Department IIT Kharagpur West Bengal India - 721302 URL: http://www.crx.iitkgp.ernet.in/~rakesh/ Phone: 09832117500 -------------------------------------------------------------------- _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rbw at ahpcrc.org Tue Jun 10 18:36:29 2003 From: rbw at ahpcrc.org (Richard Walsh) Date: Tue, 10 Jun 2003 17:36:29 -0500 Subject: Nice survey of higher end power supplies ... Message-ID: <200306102236.h5AMaTw31240@mycroft.ahpcrc.org> All, Intersecting the recent interesting discussion of power and such, Tom's Hardware has just put up a nice review of higher-end supplies rating them on effeciency, noise, etc. There is a nice summary table at the end. You can ask your supplier what they are using and then look at the table and say ... "I don't want that one." ;-) http://www17.tomshardware.com/howto/20030609/index.html Enjoy, rbw #--------------------------------------------------- # Richard Walsh # Project Manager, Cluster Computing, Computational # Chemistry and Finance # netASPx, Inc. # 1200 Washington Ave. So. # Minneapolis, MN 55415 # VOX: 612-337-3467 # FAX: 612-337-3400 # EMAIL: rbw at networkcs.com, richard.walsh at netaspx.com # rbw at ahpcrc.org # #--------------------------------------------------- # "Why waste time learning when ignornace is # instantaneous?" -Thomas Hobbes #--------------------------------------------------- _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From hahn at physics.mcmaster.ca Tue Jun 10 20:31:31 2003 From: hahn at physics.mcmaster.ca (Mark Hahn) Date: Tue, 10 Jun 2003 20:31:31 -0400 (EDT) Subject: Nice survey of higher end power supplies ... In-Reply-To: <200306102236.h5AMaTw31240@mycroft.ahpcrc.org> Message-ID: > etc. There is a nice summary table at the end. You can > ask your supplier what they are using and then look at > the table and say ... "I don't want that one." ;-) > > http://www17.tomshardware.com/howto/20030609/index.html it's interesting, but I don't think it's all that useful. for instance, how many of you have cluster nodes that need >400W PS's? as usual THG seems mainly focused on satisfying the gamer/overclocker that he's got Da B35t. one relevant and interesting factoid was that they claim the PS's all operate at about 70% efficiency. it's a little hard to tell how they measured that, but I think it's at peak load. it would be nice to know whether efficiency is better at sane/normal loads. I just received a pair of kill-a-watt's today, and am eager to plug things into them. a PIII/833/i815 256M ram, 40G + 120G SATA demo machine I have here was around 50W. but someone already made off with the single kill-a-watt I brought in ;) PF .74 on that machine, by the way. I was kind of surprised, since it was a compaq deskpro. pretty old, though, that's probably why. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From snkapadi at usc.edu Tue Jun 10 16:50:39 2003 From: snkapadi at usc.edu (shyam kapadia) Date: Tue, 10 Jun 2003 13:50:39 -0700 Subject: Call for Papers for S.C.A.L.E Message-ID: <000501c32f91$f36e2070$6502a8c0@mouse> Southern CAlifornia Linux Exposition (SCALE) 2003. The USC, Simi/Conejo, and UCLA Linux User Groups are proud to present the second annual Southern California Linux Expo scheduled for November 22nd, 2003 at the Los Angeles Convention Center. The Southern California Linux Expo will bring together Linux and Open Source Software companies, developers, and users. The conference is being held by Linux user groups from the local area: the USCLUG, SCLUG, and UCLALUG. Building on the tremendous success of last year's SCALE, the three LUGs will expand on previous LUG Fests to promote Linux and the Open Source Software community. We INVITE you to share your work on Linux and Open Source projects with the rest of the community. Presentations can be in the form of speeches, tutorials, or demonstrations. A speech on Linux's uses in multimedia, a tutorial on iptables (demonstrating routing and firewall applications in Linux), or a demonstration of projects such as games, are just some examples of appropriate material. Presentations are allotted time in increments of 30 minutes up to 2 hours. For speeches, please submit your paper along with a description of the speech format and intended audience. For tutorials and demonstrations, please submit your proposal and let us know of any special needs such as audio/video equipment. All proposals are to be sent to kapadia at socallinuxexpo.com. The deadline for submissions has been extended to Sunday, October 6th at midnight. Further details for last years expo can be found at http://www.socallinuxexpo.com. Thanks and Regards SoCAlLinuxExpo CFP Committee. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From jeff at aslab.com Tue Jun 10 21:49:25 2003 From: jeff at aslab.com (Jeff Nguyen) Date: Tue, 10 Jun 2003 18:49:25 -0700 Subject: Nice survey of higher end power supplies ... References: <200306102236.h5AMaTw31240@mycroft.ahpcrc.org> Message-ID: <151001c32fbb$b0156d90$6502a8c0@jeff> > Intersecting the recent interesting discussion of power > and such, Tom's Hardware has just put up a nice review > of higher-end supplies rating them on effeciency, noise, > etc. There is a nice summary table at the end. You can > ask your supplier what they are using and then look at > the table and say ... "I don't want that one." ;-) > > http://www17.tomshardware.com/howto/20030609/index.html > There is one important factor which is difficult for the author to provide in the review. That is the reliability factor of the power supply under normal operating environment. It the power supply performs well but does not last, it is not an ideal choice either. Jeff ASL Inc. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From exa at kablonet.com.tr Tue Jun 10 17:32:37 2003 From: exa at kablonet.com.tr (Eray Ozkural) Date: Wed, 11 Jun 2003 00:32:37 +0300 Subject: PS2 cluster In-Reply-To: <004d01c32c0d$6b442190$8461cdc2@DREAD> References: <004d01c32c0d$6b442190$8461cdc2@DREAD> Message-ID: <200306110032.38038.exa@kablonet.com.tr> On Friday 06 June 2003 12:23, John Hearns wrote: > I remember when the PS2 first came out, > ideas were kicked around on the Beowulf list about > clustering them and using the vector units in them > (and using Firewire networking) I could definitely build a really cool 2^6 hypercube with firewire hardware but last time I checked there weren't any free TCP/IP drivers for firewire that would support the kind of routing I'd like (could you have something like CT routing in software?). Anybody's got the latest scoop on firewire for linux networking? Cheers, -- Eray Ozkural (exa) Comp. Sci. Dept., Bilkent University, Ankara KDE Project: http://www.kde.org www: http://www.cs.bilkent.edu.tr/~erayo Malfunction: http://mp3.com/ariza GPG public key fingerprint: 360C 852F 88B0 A745 F31B EA0F 7C07 AE16 874D 539C _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From fryman at cc.gatech.edu Tue Jun 10 21:44:04 2003 From: fryman at cc.gatech.edu (Josh Fryman) Date: Tue, 10 Jun 2003 21:44:04 -0400 Subject: Nice survey of higher end power supplies ... In-Reply-To: References: <200306102236.h5AMaTw31240@mycroft.ahpcrc.org> Message-ID: <20030610214404.02a655e9.fryman@cc.gatech.edu> > I just received a pair of kill-a-watt's today, and am eager to > plug things into them. speaking of these... i bought one at radioshack for $20 or so, and am curious if anyone out there has reverse-engineered the insides? i'm getting ready to attempt such a task to get the Watts output tied directly to a little micro that can report the power consumption back into the network. after all, why look at the LCD in some random location when i could just display it on my desktop? :) -j _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From James.P.Lux at jpl.nasa.gov Wed Jun 11 00:35:38 2003 From: James.P.Lux at jpl.nasa.gov (Jim Lux) Date: Tue, 10 Jun 2003 21:35:38 -0700 Subject: Nice survey of higher end power supplies ... References: Message-ID: <000601c32fd2$e87f4b80$02a8a8c0@office1> > > > > http://www17.tomshardware.com/howto/20030609/index.html > > it's interesting, but I don't think it's all that useful. > for instance, how many of you have cluster nodes that need >400W PS's? > as usual THG seems mainly focused on satisfying the gamer/overclocker > that he's got Da B35t. > > one relevant and interesting factoid was that they claim the PS's > all operate at about 70% efficiency. it's a little hard to tell > how they measured that, but I think it's at peak load. it would > be nice to know whether efficiency is better at sane/normal loads. Probably worse efficiency at light loads. There's a fairly significant "overhead" power that is consumed at all times, as well as a "proportional to load" loss. Also, the power factor is probably much worse at light loads, unless the PS uses some form of harmonic/PF correction (e.g. it is CE marked, and actually complies with the mark). The usual input stage is a rectifier to a capacitor input filter, which has an input current waveform that is very "pulse like" at low loads. High pulse current >> high I^2*R losses > > PF .74 on that machine, by the way. I was kind of surprised, > since it was a compaq deskpro. pretty old, though, that's probably why. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rgb at phy.duke.edu Wed Jun 11 09:03:40 2003 From: rgb at phy.duke.edu (Robert G. Brown) Date: Wed, 11 Jun 2003 09:03:40 -0400 (EDT) Subject: Nice survey of higher end power supplies ... In-Reply-To: Message-ID: On Tue, 10 Jun 2003, Mark Hahn wrote: > one relevant and interesting factoid was that they claim the PS's > all operate at about 70% efficiency. it's a little hard to tell > how they measured that, but I think it's at peak load. it would > be nice to know whether efficiency is better at sane/normal loads. They could be confusing PF and efficiency (obviously NOT the same thing at all). To otherwise measure efficiency they'd have to put controlled loads (e.g. big resistors, light bulbs) on all the various DC out lines and measure power delivery to that load (relatively simple, VA = W for DC) while measuring average power in. The number is believable, though. > I just received a pair of kill-a-watt's today, and am eager to > plug things into them. a PIII/833/i815 256M ram, 40G + 120G SATA > demo machine I have here was around 50W. but someone already > made off with the single kill-a-watt I brought in ;) Tell them to go buy their own. For $40 or $50 delivered, why not? rgb -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb at phy.duke.edu _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From k_kolokoutsas at yahoo.co.uk Wed Jun 11 08:01:01 2003 From: k_kolokoutsas at yahoo.co.uk (=?iso-8859-1?q?kolokoutsas=20konstantinos?=) Date: Wed, 11 Jun 2003 13:01:01 +0100 (BST) Subject: ideal motherboard for diskless beowulf Message-ID: <20030611120101.60267.qmail@web9901.mail.yahoo.com> Hello, I am in the process of designing a 12-node diskless cluster. I am going with AMD CPUs, but not quite confident which mobo to use, for serving the following with a limited budget: - (must) onboard LAN - (must) proven Linux (RH7.2) compatibility - (ideal) micro-atx - (ideal) wide range of CPU support, i.e. from Duron sub-GHz up to XP 2600+ - (bonus) some overclocking capabilities In terms of chip thanks in advance, Dr. Kostas Kolokoutsas __________________________________________________ Yahoo! Plus - For a better Internet experience http://uk.promotions.yahoo.com/yplus/yoffer.html _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rgb at phy.duke.edu Wed Jun 11 09:10:49 2003 From: rgb at phy.duke.edu (Robert G. Brown) Date: Wed, 11 Jun 2003 09:10:49 -0400 (EDT) Subject: Nice survey of higher end power supplies ... In-Reply-To: <20030610214404.02a655e9.fryman@cc.gatech.edu> Message-ID: On Tue, 10 Jun 2003, Josh Fryman wrote: > > I just received a pair of kill-a-watt's today, and am eager to > > plug things into them. > > speaking of these... i bought one at radioshack for $20 or so, and > am curious if anyone out there has reverse-engineered the insides? > > i'm getting ready to attempt such a task to get the Watts output > tied directly to a little micro that can report the power consumption > back into the network. after all, why look at the LCD in some > random location when i could just display it on my desktop? :) Do it! The RS232 chips are pretty cheap, IIRC. Then sell the little mini-circuit back to KAW for their "KAWII" with a serial readout. At $20 retail they can't be making a lot of money any more, and I'd cheerily put one on every circuit line in our cluster for $40 or even $50 each, if I could read the total line input. In fact, being able to control the output via serial line would be lovely too. There are only a half-dozen buttons or so, so a three or four bit set of control codes would likely do fine... rgb -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb at phy.duke.edu _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rgb at phy.duke.edu Wed Jun 11 09:57:39 2003 From: rgb at phy.duke.edu (Robert G. Brown) Date: Wed, 11 Jun 2003 09:57:39 -0400 (EDT) Subject: ideal motherboard for diskless beowulf In-Reply-To: <20030611120101.60267.qmail@web9901.mail.yahoo.com> Message-ID: On Wed, 11 Jun 2003, kolokoutsas konstantinos wrote: > Hello, > > I am in the process of designing a 12-node diskless > cluster. I am going with AMD CPUs, but not quite > confident which mobo to use, for serving the following > with a limited budget: > > - (must) onboard LAN > - (must) proven Linux (RH7.2) compatibility > - (ideal) micro-atx > - (ideal) wide range of CPU support, i.e. from Duron > sub-GHz up to XP 2600+ > - (bonus) some overclocking capabilities UP or MP? MP can be a hair cheaper per CPU, if your tasks aren't resource bound. They won't fit in a micro-atx, they won't do Duron CPUs, but on AVERAGE they will take up even less than two micro-atx's, especially in a 1U form factor. I've used a variety of MSI boards and (except for the taiwan capacitor fiasco) been generally satisfied with them. I use Tyan's (e.g. Tyan 2466 or better) for duals -- they have onboard 100BT or better (current motherboards have dual GigE NICs onboard, as well as video). Just about any of these boards will work fine with any version of RHL from 7-9. Beware motherboards with onboard RTL 8139 NICs -- they are cheap, common, and suck. I'd plan on adding a decent NIC a la carte if I "had" to pick a motherboard with an onboard RTL. Finally, I'd recommend using RH 9, not 7.2, and 7.3 over 7.2 if it came to that. Linux really does get better, and an essential part of installing ANYTHING is arranging for regular updates to fix security problems and bugs. The older versions tend to get frozen out and no longer be aggressively updated, meaning that any vulnerabilities or bugs in the setup won't get fixed. rgb -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb at phy.duke.edu _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From josip at lanl.gov Wed Jun 11 10:16:35 2003 From: josip at lanl.gov (Josip Loncaric) Date: Wed, 11 Jun 2003 08:16:35 -0600 Subject: Nice survey of higher end power supplies ... In-Reply-To: References: Message-ID: <3EE739C3.5040308@lanl.gov> Mark Hahn wrote: > > PF .74 on that machine, by the way. I was kind of surprised, > since it was a compaq deskpro. pretty old, though, that's probably why. Power factor is about 0.65 on my own machine. I was kind of surprised too. According to Kill-a-watt, my 2.8 GHz P4 w/Asus motherboard, lots of other goodies, and a 2002-vintage Antec PS dissipates 130-180 Watts in operation depending on activity. Another disappointment was its "sleep" mode which dissipates 100 Watts (not much better than the 130 W idle state). When "off" it still uses 5 Watts... Sincerely, Josip P.S. PF 0.65 means that at 180 W the machine draws about 270 VA. This VA rating is important when sizing UPS units. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From jhearns at freesolutions.net Wed Jun 11 10:47:47 2003 From: jhearns at freesolutions.net (John Hearns) Date: Wed, 11 Jun 2003 15:47:47 +0100 Subject: ideal motherboard for diskless beowulf References: Message-ID: <03d501c33028$6fc89fb0$8461cdc2@DREAD> > > Finally, I'd recommend using RH 9, not 7.2, and 7.3 over 7.2 if it came > to that. Linux really does get better, and an essential part of > installing ANYTHING is arranging for regular updates to fix security > problems and bugs. The older versions tend to get frozen out and no > longer be aggressively updated, meaning that any vulnerabilities or bugs > in the setup won't get fixed. > Given that RH is at version 9, I think the current RH policy is that version 7.xx will be unsupported after one year. 7.2 and 7.3 have End of Life dates for errata updates of 31st December 2003 http://www.redhat.com/apps/support/errata/ I would guess you are specifying 7.2 or 7.3 due to your applications. You should consider testing/recompiling/certifying them on a more recent version. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rbw at ahpcrc.org Wed Jun 11 11:22:15 2003 From: rbw at ahpcrc.org (Richard Walsh) Date: Wed, 11 Jun 2003 10:22:15 -0500 Subject: Nice survey of higher end power supplies ... Message-ID: <200306111522.h5BFMFh14234@mycroft.ahpcrc.org> Wanting to make sure I have these power related terms straight, I wrote up definitions for myself based on recent discussions and would be grateful if folks would validate and/or correct them. 1. Idealized or VA Rated Power: An idealized measure of power delivered (or drawn) that assumes the voltage and amperage are in phase, sinusoidal, and of the magnitudes (RMS) used in the calculation. Most often used to rate/describe line power. 2. Active or Watt Rated Power: The power truly delivered (or drawn) and averaged over some period of time (integral divided by time). Typically less than 1 above (but measured in the same units of course) because most loads are not perfectly resistive and supplied power has non- sinusoidal waveforms, out of phase voltage and amperage, less than ideal magnitudes (RMS) for V and A. (The complicating effects of Bob's line harmonics, etc. would register here as well I suppose.) Often used to describe power drawn/used by a device (a PC power supply) and distinct from line power. The ratio of 2 to 1 is the Power Factor. 3. Power Efficiency: Not to be confused with the Power Factor, the ratio of the power delivered from a device to a given load to the power consumed on input by the device. In the case of a typical PC power supply, this would be the ratio of the power drawn (by lines [3.3, 5, 12V] in use for a particular load over a specific time) over the power consumed by the PC power supply which includes its own draw and heat related losses (my guess is that active power [2 above] should be used in the denominator here). Too wordy, but are these definitions accurate, sufficiently complete? rbw _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From mikee at mikee.ath.cx Wed Jun 11 11:24:29 2003 From: mikee at mikee.ath.cx (Mike Eggleston) Date: Wed, 11 Jun 2003 10:24:29 -0500 Subject: ideal motherboard for diskless beowulf In-Reply-To: <20030611120101.60267.qmail@web9901.mail.yahoo.com>; from k_kolokoutsas@yahoo.co.uk on Wed, Jun 11, 2003 at 01:01:01PM +0100 References: <20030611120101.60267.qmail@web9901.mail.yahoo.com> Message-ID: <20030611102429.A16195@mikee.ath.cx> On Wed, 11 Jun 2003, kolokoutsas konstantinos wrote: > Hello, > > I am in the process of designing a 12-node diskless > cluster. I am going with AMD CPUs, but not quite > confident which mobo to use, for serving the following > with a limited budget: > > - (must) onboard LAN > - (must) proven Linux (RH7.2) compatibility > - (ideal) micro-atx > - (ideal) wide range of CPU support, i.e. from Duron > sub-GHz up to XP 2600+ > - (bonus) some overclocking capabilities Don't know about ideal, but I am using the Biostar M7KVQ with an AMD XP 2100+ and 512MB. Mike _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rgb at phy.duke.edu Wed Jun 11 14:15:41 2003 From: rgb at phy.duke.edu (Robert G. Brown) Date: Wed, 11 Jun 2003 14:15:41 -0400 (EDT) Subject: Nice survey of higher end power supplies ... In-Reply-To: <200306111522.h5BFMFh14234@mycroft.ahpcrc.org> Message-ID: On Wed, 11 Jun 2003, Richard Walsh wrote: > Too wordy, but are these definitions accurate, sufficiently complete? Sure, as far as I can see. rgb -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb at phy.duke.edu _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From abignone at unige.it Tue Jun 10 17:02:35 2003 From: abignone at unige.it (Franco A. Bignone) Date: Tue, 10 Jun 2003 23:02:35 +0200 Subject: PS2 cluster References: <200306061127.h56BRl505710@NewBlue.Scyld.com> Message-ID: <3EE6476B.E12277B2@unige.it> > Reply-To: "John Hearns" > From: "John Hearns" > To: > Subject: PS2 cluster > Date: Fri, 6 Jun 2003 10:23:38 +0100 > > I remember when the PS2 first came out, > ideas were kicked around on the Beowulf list about > clustering them and using the vector units in them > (and using Firewire networking) > > Looks like NCS are actually doing this, using the PS2 Linux kit > and Ethernet: > > http://access.ncsa.uiuc.edu/Releases/05.27.03_Playing_th.html > > Wow! Right, if I have understood correctly they have set a 70 node cluster, plus some connecting hardware for a price tag of 20,000 $. Not bad. Is it effective in price/performance ? It seems nice route to follow if your software make use of the VU, even if I don't have clear how much is difficult to set it up correctly. FB. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From abignone at unige.it Tue Jun 10 17:03:41 2003 From: abignone at unige.it (Franco A. Bignone) Date: Tue, 10 Jun 2003 23:03:41 +0200 Subject: PS2 cluster References: <200306061127.h56BRl505710@NewBlue.Scyld.com> Message-ID: <3EE647AD.765F2189@unige.it> > Reply-To: "John Hearns" > From: "John Hearns" > To: > Subject: PS2 cluster > Date: Fri, 6 Jun 2003 10:23:38 +0100 > > I remember when the PS2 first came out, > ideas were kicked around on the Beowulf list about > clustering them and using the vector units in them > (and using Firewire networking) > > Looks like NCS are actually doing this, using the PS2 Linux kit > and Ethernet: > > http://access.ncsa.uiuc.edu/Releases/05.27.03_Playing_th.html > > Wow! Right, if I have understood correctly they have set a 70 node cluster, plus some connecting hardware for a price tag of 20,000 $. Not bad. Is it effective in price/performance ? It seems nice route to follow if your software make use of the VU, even if I don't have clear how much is difficult to set it up correctly. FB . _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From abignone at unige.it Tue Jun 10 17:02:59 2003 From: abignone at unige.it (Franco A. Bignone) Date: Tue, 10 Jun 2003 23:02:59 +0200 Subject: PS2 cluster References: <200306061127.h56BRl505710@NewBlue.Scyld.com> Message-ID: <3EE64783.38C39F3B@unige.it> > Reply-To: "John Hearns" > From: "John Hearns" > To: > Subject: PS2 cluster > Date: Fri, 6 Jun 2003 10:23:38 +0100 > > I remember when the PS2 first came out, > ideas were kicked around on the Beowulf list about > clustering them and using the vector units in them > (and using Firewire networking) > > Looks like NCS are actually doing this, using the PS2 Linux kit > and Ethernet: > > http://access.ncsa.uiuc.edu/Releases/05.27.03_Playing_th.html > > Wow! Right, if I have understood correctly they have set a 70 node cluster, plus some connecting hardware for a price tag of 20,000 $. Not bad. Is it effective in price/performance ? It seems nice route to follow if your software make use of the VU, even if I don't have clear how much is difficult to set it up correctly. FB . _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From abignone at unige.it Tue Jun 10 17:04:07 2003 From: abignone at unige.it (Franco A. Bignone) Date: Tue, 10 Jun 2003 23:04:07 +0200 Subject: PS2 cluster References: <200306061127.h56BRl505710@NewBlue.Scyld.com> Message-ID: <3EE647C7.6686E8F3@unige.it> > Reply-To: "John Hearns" > From: "John Hearns" > To: > Subject: PS2 cluster > Date: Fri, 6 Jun 2003 10:23:38 +0100 > > I remember when the PS2 first came out, > ideas were kicked around on the Beowulf list about > clustering them and using the vector units in them > (and using Firewire networking) > > Looks like NCS are actually doing this, using the PS2 Linux kit > and Ethernet: > > http://access.ncsa.uiuc.edu/Releases/05.27.03_Playing_th.html > > Wow! Right, if I have understood correctly they have set a 70 node cluster, plus some connecting hardware for a price tag of 20,000 $. Not bad. Is it effective in price/performance ? It seems nice route to follow if your software make use of the VU, even if I don't have clear how much is difficult to set it up correctly. FB. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From abignone at unige.it Tue Jun 10 17:04:25 2003 From: abignone at unige.it (Franco A. Bignone) Date: Tue, 10 Jun 2003 23:04:25 +0200 Subject: PS2 cluster. References: <200306061127.h56BRl505710@NewBlue.Scyld.com> Message-ID: <3EE647D9.4A18D7C7@unige.it> > Reply-To: "John Hearns" > From: "John Hearns" > To: > Subject: PS2 cluster > Date: Fri, 6 Jun 2003 10:23:38 +0100 > > I remember when the PS2 first came out, > ideas were kicked around on the Beowulf list about > clustering them and using the vector units in them > (and using Firewire networking) > > Looks like NCS are actually doing this, using the PS2 Linux kit > and Ethernet: > > http://access.ncsa.uiuc.edu/Releases/05.27.03_Playing_th.html > > Wow! Right, if I have understood correctly they have set a 70 node cluster, plus some connecting hardware for a price tag of 20,000 $. Not bad. Is it effective in price/performance ? It seems nice route to follow if your software make use of the VU, even if I don't have clear how much is difficult to set it up correctly. FB . _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From abignone at unige.it Tue Jun 10 17:02:22 2003 From: abignone at unige.it (Franco A. Bignone) Date: Tue, 10 Jun 2003 23:02:22 +0200 Subject: PS2 cluster References: <200306061127.h56BRl505710@NewBlue.Scyld.com> Message-ID: <3EE6475D.7E41876F@unige.it> > Reply-To: "John Hearns" > From: "John Hearns" > To: > Subject: PS2 cluster > Date: Fri, 6 Jun 2003 10:23:38 +0100 > > I remember when the PS2 first came out, > ideas were kicked around on the Beowulf list about > clustering them and using the vector units in them > (and using Firewire networking) > > Looks like NCS are actually doing this, using the PS2 Linux kit > and Ethernet: > > http://access.ncsa.uiuc.edu/Releases/05.27.03_Playing_th.html > > Wow! Right, if I have understood correctly they have set a 70 node cluster, plus some connecting hardware for a price tag of 20,000 $. Not bad. Is it effective in price/performance ? It seems nice route to follow if your software make use of the VU, even if I don't have clear how much is difficult to set it up correctly. FB _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From abignone at unige.it Tue Jun 10 17:04:15 2003 From: abignone at unige.it (Franco A. Bignone) Date: Tue, 10 Jun 2003 23:04:15 +0200 Subject: PS2 cluster References: <200306061127.h56BRl505710@NewBlue.Scyld.com> Message-ID: <3EE647CF.44C294D4@unige.it> > Reply-To: "John Hearns" > From: "John Hearns" > To: > Subject: PS2 cluster > Date: Fri, 6 Jun 2003 10:23:38 +0100 > > I remember when the PS2 first came out, > ideas were kicked around on the Beowulf list about > clustering them and using the vector units in them > (and using Firewire networking) > > Looks like NCS are actually doing this, using the PS2 Linux kit > and Ethernet: > > http://access.ncsa.uiuc.edu/Releases/05.27.03_Playing_th.html > > Wow! Right, if I have understood correctly they have set a 70 node cluster, plus some connecting hardware for a price tag of 20,000 $. Not bad. Is it effective in price/performance ? It seems nice route to follow if your software make use of the VU, even if I don't have clear how much is difficult to set it up correctly. FB . _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From abignone at unige.it Tue Jun 10 17:03:47 2003 From: abignone at unige.it (Franco A. Bignone) Date: Tue, 10 Jun 2003 23:03:47 +0200 Subject: PS2 cluster References: <200306061127.h56BRl505710@NewBlue.Scyld.com> Message-ID: <3EE647B3.E76EE35F@unige.it> > Reply-To: "John Hearns" > From: "John Hearns" > To: > Subject: PS2 cluster > Date: Fri, 6 Jun 2003 10:23:38 +0100 > > I remember when the PS2 first came out, > ideas were kicked around on the Beowulf list about > clustering them and using the vector units in them > (and using Firewire networking) > > Looks like NCS are actually doing this, using the PS2 Linux kit > and Ethernet: > > http://access.ncsa.uiuc.edu/Releases/05.27.03_Playing_th.html > > Wow! Right, if I have understood correctly they have set a 70 node cluster, plus some connecting hardware for a price tag of 20,000 $. Not bad. Is it effective in price/performance ? It seems nice route to follow if your software make use of the VU, even if I don't have clear how much is difficult to set it up correctly. FB _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From abignone at unige.it Tue Jun 10 17:03:59 2003 From: abignone at unige.it (Franco A. Bignone) Date: Tue, 10 Jun 2003 23:03:59 +0200 Subject: PS2 cluster References: <200306061127.h56BRl505710@NewBlue.Scyld.com> Message-ID: <3EE647BF.1E32F9E1@unige.it> > Reply-To: "John Hearns" > From: "John Hearns" > To: > Subject: PS2 cluster > Date: Fri, 6 Jun 2003 10:23:38 +0100 > > I remember when the PS2 first came out, > ideas were kicked around on the Beowulf list about > clustering them and using the vector units in them > (and using Firewire networking) > > Looks like NCS are actually doing this, using the PS2 Linux kit > and Ethernet: > > http://access.ncsa.uiuc.edu/Releases/05.27.03_Playing_th.html > > Wow! Right, if I have understood correctly they have set a 70 node cluster, plus some connecting hardware for a price tag of 20,000 $. Not bad. Is it effective in price/performance ? It seems nice route to follow if your software make use of the VU, even if I don't have clear how much is difficult to set it up correctly. FB _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From abignone at unige.it Tue Jun 10 17:03:08 2003 From: abignone at unige.it (Franco A. Bignone) Date: Tue, 10 Jun 2003 23:03:08 +0200 Subject: PS2 cluster References: <200306061127.h56BRl505710@NewBlue.Scyld.com> Message-ID: <3EE6478C.BCE8A42F@unige.it> > Reply-To: "John Hearns" > From: "John Hearns" > To: > Subject: PS2 cluster > Date: Fri, 6 Jun 2003 10:23:38 +0100 > > I remember when the PS2 first came out, > ideas were kicked around on the Beowulf list about > clustering them and using the vector units in them > (and using Firewire networking) > > Looks like NCS are actually doing this, using the PS2 Linux kit > and Ethernet: > > http://access.ncsa.uiuc.edu/Releases/05.27.03_Playing_th.html > > Wow! Right, if I have understood correctly they have set a 70 node cluster, plus some connecting hardware for a price tag of 20,000 $. Not bad. Is it effective in price/performance ? It seems nice route to follow if your software make use of the VU, even if I don't have clear how much is difficult to set it up correctly. FB . _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From JAI_RANGI at SDSTATE.EDU Wed Jun 11 16:19:17 2003 From: JAI_RANGI at SDSTATE.EDU (RANGI, JAI) Date: Wed, 11 Jun 2003 15:19:17 -0500 Subject: Octave on Beowulf Message-ID: Octave is Mat lab like program but free. Has any tried to run octave on Beowulf Cluster. Any hint or link will be appreciated. Thanks Jai Rangi _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From JAI_RANGI at SDSTATE.EDU Wed Jun 11 16:19:17 2003 From: JAI_RANGI at SDSTATE.EDU (RANGI, JAI) Date: Wed, 11 Jun 2003 15:19:17 -0500 Subject: Octave on Beowulf Message-ID: Octave is Mat lab like program but free. Has any tried to run octave on Beowulf Cluster. Any hint or link will be appreciated. Thanks Jai Rangi _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From math at velocet.ca Wed Jun 11 16:19:00 2003 From: math at velocet.ca (Ken Chase) Date: Wed, 11 Jun 2003 16:19:00 -0400 Subject: PS2 cluster In-Reply-To: <3EE647CF.44C294D4@unige.it>; from abignone@unige.it on Tue, Jun 10, 2003 at 11:04:15PM +0200 References: <200306061127.h56BRl505710@NewBlue.Scyld.com> <3EE647CF.44C294D4@unige.it> Message-ID: <20030611161859.T29733@velocet.ca> On Tue, Jun 10, 2003 at 11:04:15PM +0200, Franco A. Bignone's all... >> Reply-To: "John Hearns" >> From: "John Hearns" >> To: >> Subject: PS2 cluster >> Date: Fri, 6 Jun 2003 10:23:38 +0100 >> >> I remember when the PS2 first came out, >> ideas were kicked around on the Beowulf list about >> clustering them and using the vector units in them >> (and using Firewire networking) >> >> Looks like NCS are actually doing this, using the PS2 Linux kit >> and Ethernet: >> >> http://access.ncsa.uiuc.edu/Releases/05.27.03_Playing_th.html >> >> Wow! > >Right, if I have understood correctly they have set a 70 node cluster, >plus some connecting hardware for a price tag of 20,000 $. Not bad. >Is it effective in price/performance ? It seems nice route to follow >if your software make use of the VU, even if I don't have clear how >much is difficult to set it up correctly. > >FB >. Every msg is slightly different that you sent to the list, especially the . at the end. are you trying to do something here? Perhpas your have notMeToo set in your beowulf-l mailman prefs. Go visit the mailman URL for this list please. /kc > >_______________________________________________ >Beowulf mailing list, Beowulf at beowulf.org >To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- Ken Chase, math at velocet.ca * Velocet Communications Inc. * Toronto, Canada Wiznet Velocet DSL.ca Datavaults 24/7: 416-967-4414 tollfree: 1-866-353-0363 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From becker at scyld.com Wed Jun 11 16:40:31 2003 From: becker at scyld.com (Donald Becker) Date: Wed, 11 Jun 2003 16:40:31 -0400 (EDT) Subject: Nice survey of higher end power supplies ... In-Reply-To: <3EE739C3.5040308@lanl.gov> Message-ID: On Wed, 11 Jun 2003, Josip Loncaric wrote: > Mark Hahn wrote: > > > > PF .74 on that machine, by the way. I was kind of surprised, > > since it was a compaq deskpro. pretty old, though, that's probably why. > > Power factor is about 0.65 on my own machine. I was kind of surprised > too. I'm shocked at that low PF. I had believed that most modern power supplies had some PF correction. > According to Kill-a-watt, my 2.8 GHz P4 w/Asus motherboard, lots > of other goodies, and a 2002-vintage Antec PS dissipates 130-180 Watts > in operation depending on activity. Another disappointment was its > "sleep" mode which dissipates 100 Watts (not much better than the 130 W > idle state). When "off" it still uses 5 Watts... This doesn't surprise me at all. Sleep mode on a desktop is no better than executing a halt instruction. Unlike a laptop, where all devices are known and controlled, a desktop machine still has to leave most of the system active to support unknown PCI or ISA devices. This has the potential to change. When most devices implement PCI power management, the OS should be able to verify that there are unsupported devices and tell the ACPI firmware to really turn things off. But you won't find this fully implemented for currrent motherboards. -- Donald Becker becker at scyld.com Scyld Computing Corporation http://www.scyld.com 914 Bay Ridge Road, Suite 220 Scyld Beowulf cluster system Annapolis MD 21403 410-990-9993 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From michael.worsham at mci.com Wed Jun 11 16:03:57 2003 From: michael.worsham at mci.com (Michael Worsham) Date: Wed, 11 Jun 2003 16:03:57 -0400 Subject: Slackware Beowulf Cluster Message-ID: <003c01c33054$98d248d0$2a408c0a@Wcomnet.com> Has anyone attempted to create a beowulf cluster with Slackware? If so, what files are required for a base install? Thanks. -- M _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From laurenceliew at yahoo.com.sg Thu Jun 12 02:05:47 2003 From: laurenceliew at yahoo.com.sg (=?iso-8859-1?q?Laurence?=) Date: Thu, 12 Jun 2003 14:05:47 +0800 (CST) Subject: Octave on Beowulf In-Reply-To: Message-ID: <20030612060547.45199.qmail@web11904.mail.yahoo.com> Hi, Octave will run on any linux platform.. so that includes all Beowulf.... I guess the question really is will it run FASTER? I know SciLab has PVM bindings...., for Octave.. I am not sure if there are PVM or MPI bindings... but you may wish to try to use some of the freely available MPI bindings/libraries for MatLab: google Matlab MPI Else.. if you code is not MPI/PVM parallel, that is it is serial code, you can just use PBS/SGE/LSF to submit your Octave jobs to the cluster.... Hope this helps. Cheers! Laurence --- "RANGI, JAI" wrote: > Octave is Mat lab like program but free. > Has any tried to run octave on Beowulf Cluster. > Any hint or link will be appreciated. > Thanks > > Jai Rangi > > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org > To change your subscription (digest mode or > unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf ===== --- Cheers! Laurence __________________________________________________ Do You Yahoo!? Send free SMS from your PC! http://sg.sms.yahoo.com _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From smuelas at mecanica.upm.es Thu Jun 12 01:57:48 2003 From: smuelas at mecanica.upm.es (Santiago Muelas) Date: Thu, 12 Jun 2003 07:57:48 +0200 Subject: ideal motherboard for diskless beowulf In-Reply-To: <20030611120101.60267.qmail@web9901.mail.yahoo.com> References: <20030611120101.60267.qmail@web9901.mail.yahoo.com> Message-ID: <20030612075748.2e08bb37.smuelas@mecanica.upm.es> I have just done what you are going to do. Only with 8 PC's. I give you MY recommendations just in case they are of any interest for you as I am not at all an expert. So, remind please that these are the advice of a novice, but I will write freely to make things easier... - As far as I know, the best AMD in quality/price today in Europe is Ahtlon XP2400. Just 100 eur. - If you are going to work with the beast near you, buy coolers from "Artic Cooler" model Cooper Silent for the CPU: 18 eur. - If your budget allows it, buy aluminium cases. Mr. XP2400 will be happy. - I strongly recommend as I have tested it, the last micro-atx from Shuttle with the chipset from Nvidia: nforce2. It goes around 10% quicker in my installation than any other tested. Price: 100 eur. - Personally, I don't overclock. I have payed a good amount to risk anything (and I don't think it is worth). Those are my actual feelings. Please, once you have your "children" working, tell me what are your feelings. I am very interested in new, relatively small installations. Santiago Muelas On Wed, 11 Jun 2003 13:01:01 +0100 (BST) kolokoutsas konstantinos wrote: > Hello, > > I am in the process of designing a 12-node diskless > cluster. I am going with AMD CPUs, but not quite > confident which mobo to use, for serving the following > with a limited budget: > > - (must) onboard LAN > - (must) proven Linux (RH7.2) compatibility > - (ideal) micro-atx > - (ideal) wide range of CPU support, i.e. from Duron > sub-GHz up to XP 2600+ > - (bonus) some overclocking capabilities > > In terms of chip > > thanks in advance, > Dr. Kostas Kolokoutsas > > > __________________________________________________ > Yahoo! Plus - For a better Internet experience > http://uk.promotions.yahoo.com/yplus/yoffer.html > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org > To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- Santiago Muelas E.T.S. Ingenieros de Caminos, (U.P.M) Tf.: (34) 91 336 66 59 e-mail: smuelas at mecanica.upm.es Fax: (34) 91 336 67 61 www: http://w3.mecanica.upm.es/~smuelas _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From laurenceliew at yahoo.com.sg Thu Jun 12 02:05:47 2003 From: laurenceliew at yahoo.com.sg (=?iso-8859-1?q?Laurence?=) Date: Thu, 12 Jun 2003 14:05:47 +0800 (CST) Subject: Octave on Beowulf In-Reply-To: Message-ID: <20030612060547.45199.qmail@web11904.mail.yahoo.com> Hi, Octave will run on any linux platform.. so that includes all Beowulf.... I guess the question really is will it run FASTER? I know SciLab has PVM bindings...., for Octave.. I am not sure if there are PVM or MPI bindings... but you may wish to try to use some of the freely available MPI bindings/libraries for MatLab: google Matlab MPI Else.. if you code is not MPI/PVM parallel, that is it is serial code, you can just use PBS/SGE/LSF to submit your Octave jobs to the cluster.... Hope this helps. Cheers! Laurence --- "RANGI, JAI" wrote: > Octave is Mat lab like program but free. > Has any tried to run octave on Beowulf Cluster. > Any hint or link will be appreciated. > Thanks > > Jai Rangi > > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org > To change your subscription (digest mode or > unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf ===== --- Cheers! Laurence __________________________________________________ Do You Yahoo!? Send free SMS from your PC! http://sg.sms.yahoo.com _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From abignone at unige.it Thu Jun 12 05:47:11 2003 From: abignone at unige.it (Franco A. Bignone) Date: Thu, 12 Jun 2003 11:47:11 +0200 Subject: PS2 cluster References: <200306061127.h56BRl505710@NewBlue.Scyld.com> <3EE647CF.44C294D4@unige.it> <20030611161859.T29733@velocet.ca> Message-ID: <3EE84C1E.A7CCD4E4@unige.it> Sorry for the inconvenience, but my mail kept telling me that sending had failed, while sending the message out. So I sent by mistake multiple copies trying to figure out the problem, when I understood what was happening was too late. FB -- ************************************************************************* * Dr. Franco A. Bignone, I.S.T., National Cancer Institute, Lab. Exp. * * Oncology, Lr.go Rosanna Benzi, 10, 16132, Genova, Italy. * * e-mail: franco.bignone at istge.it, abignone at unige.it * * http://gendyn.ist.unige.it http://gendyn.istge.it * * ph. home: +39-010-247-3070 (answ.) * * job: +39-010-5600-213, +39-010-355839, +39-010-5600641, * * fax: +39-010-5600-217 * ************************************************************************* _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From andy at ideafix.litec.csic.es Thu Jun 12 06:21:33 2003 From: andy at ideafix.litec.csic.es (andy) Date: Thu, 12 Jun 2003 11:21:33 +0100 Subject: NAS Parallel Benchmarks for Current Hardware References: <200306111810.WAA01122@nocserv.free.net> Message-ID: <3EE8542D.93CE3B57@ideafix.litec.csic.es> Mikhail, > > I am looking to put together a small cluster for numerical simulation > > and have been surprised at how few NPB benchmark results using current > > hardware I can find via google. > > > It's common situation w/NPB (in opposition to Linpack, SPECcpu e.a.) :-( Unfortunately, I am not sure linpack and spec reliably measure what is important for numerical work on a beowulf. There is some good news. I have got in contact with the group at NASA and they have got some funding to employ someone next week for a 3 month period to get things rolling again with the NPB benchmarks. Hopefully, some useful numbers for current machines should soon start appearing at: www.nas.nasa.gov/Software/NPB _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From collier at cs.widener.edu Wed Jun 11 19:15:30 2003 From: collier at cs.widener.edu (Aaron Collier) Date: Wed, 11 Jun 2003 19:15:30 -0400 (EDT) Subject: Slackware Beowulf Cluster Message-ID: Hello, I have actually been working on a cluster-oriented Linux distribution based on Slackware (actually version 8.1) called WUBCOS (Widener University Beowulf Cluster OS). I have been running the basic distribution on an eight node cluster I built as part of my senior project in computer science and mathematics. Unfortunately, the distribution isn't very refined (but still stable) as I no longer have access to adequate resources to continue development because I just graduated. WUBCOS is currently optimized for my particular hardware but I can tell you the software I have installed at the moment. I compiled a custom kernel and actually enabled hyper-threading (simultaneous multithreading) which works fairly well despite what others may say. I upgraded the kernel from 2.4.18 to 2.4.20 to add proper DMA support and HT support, but Slackware 9 includes kernel 2.4.20 (no source though). Since the member nodes don't have CD-ROM drives I wrote a small network-based remote node installation script that mounts an NFS partition hosted on the main server simply containing a copy of the Slackware installation CD and custom packages I created. I actually had to create a custom root disk set which includes the install script and the necessary nic drivers. I also have been using channel bonding (the default round-robin packet distribution scheme) and haven't had any problems with stability. The cluster uses both MPICH-MPI and LAM-MPI although MPICH has been patched to work more efficiently with Open PBS which is used as the only batch queue system. Also included is a precompiled ATLAS package which has been optimized for the Pentium 4 Xeon with threaded libraries. I also installed PVFS which works well enough. The distro uses NIS to distribute user information, DNS for standard name resolution, and NFS to distribute user home directories and actually /usr/local as well (people will argue this isn't good but it works for me). I also configured NTP because time travel was becoming all too common on the nodes. I also went through all of the system initialization scripts (located in /etc/rc.d) and disabled all unnecessary system servies (like httpd) although these files differ for the head node and the regular compute nodes. Speaking in terms of a general installation, since I had a large hard drive I installed everything (about 2GB) on the head node (actually I removed the sudo package). For the installation of the member nodes I made custom tagfiles so a node installation only requires about 750MB. What you need to install will probably depend upon your anticipated use for the cluster. Configuring Slackware to operate efficiently in a cluster environment can be fairly time consuming depending upon your needs but it is certainly well worth the effort. I am glad someone else recognizes the merits of the Slackware distribution. Please feel free to contact me if you have any additional questions. Bye, Aaron Collier P.S. If anyone would like to hire me then please send me an e-mail :o) _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From josip at lanl.gov Thu Jun 12 12:48:37 2003 From: josip at lanl.gov (Josip Loncaric) Date: Thu, 12 Jun 2003 10:48:37 -0600 Subject: Nice survey of higher end power supplies ... In-Reply-To: References: Message-ID: <3EE8AEE5.2070607@lanl.gov> Donald Becker wrote: > On Wed, 11 Jun 2003, Josip Loncaric wrote: >> >>Power factor is about 0.65 on my own machine. I was kind of surprised >>too. > > > I'm shocked at that low PF. I had believed that most modern power > supplies had some PF correction. Apparently not this one (Antec True430). Power factor this low may be related to the fact that this power supply is a bit oversized. My computer uses only 30-40% of the power supply peak rating. The oversized 430W PS came with the Antec case, but a 350W unit would have been more than adequate. Sincerely, Josip _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From johnh at sjgeophysics.com Thu Jun 12 13:31:38 2003 From: johnh at sjgeophysics.com (John Harrop) Date: 12 Jun 2003 10:31:38 -0700 Subject: Configuring OpenSSH 3.5p1 Message-ID: <1055439098.27276.31.camel@orion-2> I'm currently building a new cluster which unlike the older, more isolated one will be using OpenSSH. How are others configuring their ssh? I've been trying to keep the system in the 'recommended' settings of the O'Reilly SSH book but I'm having trouble getting automatic password authentication. (The passphrase authentication is fine.) For example, ssh-agent handles the passphrase fine, but I can't seem to get /etc/shosts.equiv or .shosts to work with the password. Has the security tightened up on OpenSSH since the book? Assistance would be appreciated - I have a feeling I'm missing something simple in all the ssh options, files and permissions! Cheers, John Harrop Geologist, GIS Manager SJ Geophysics Ltd _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From chaimj at majorlinux.com Thu Jun 12 13:59:19 2003 From: chaimj at majorlinux.com (Major Chai Mee Joon) Date: Fri, 13 Jun 2003 01:59:19 +0800 Subject: ideal motherboard for diskless beowulf In-Reply-To: <200306111521.h5BFLQY28078@NewBlue.Scyld.com> References: <200306111521.h5BFLQY28078@NewBlue.Scyld.com> Message-ID: <20030612175919.GC2630@mail.majorlinux.com> Hi, > On Wed, 11 Jun 2003, kolokoutsas konstantinos wrote: > > > Hello, > > > > I am in the process of designing a 12-node diskless > > cluster. I am going with AMD CPUs, but not quite > > confident which mobo to use, for serving the following > > with a limited budget: > > > > - (must) onboard LAN > > - (must) proven Linux (RH7.2) compatibility > > - (ideal) micro-atx > > - (ideal) wide range of CPU support, i.e. from Duron > > sub-GHz up to XP 2600+ > > - (bonus) some overclocking capabilities You just have to be careful if you're using the the latest boards unless you like to get your hands dirty. eg. Asus A7N8X Deluxe Board based on nVidia's Nforce2-Chipset, as the onboard Nvidia Ethernet requires the driver supplied by nVidia. Regards, Major Chai MajorLinux.com _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rgb at phy.duke.edu Thu Jun 12 14:11:41 2003 From: rgb at phy.duke.edu (Robert G. Brown) Date: Thu, 12 Jun 2003 14:11:41 -0400 (EDT) Subject: Configuring OpenSSH 3.5p1 In-Reply-To: <1055439098.27276.31.camel@orion-2> Message-ID: On 12 Jun 2003, John Harrop wrote: > I'm currently building a new cluster which unlike the older, more > isolated one will be using OpenSSH. How are others configuring their > ssh? > > I've been trying to keep the system in the 'recommended' settings of the > O'Reilly SSH book but I'm having trouble getting automatic password > authentication. (The passphrase authentication is fine.) For example, > ssh-agent handles the passphrase fine, but I can't seem to get > /etc/shosts.equiv or .shosts to work with the password. > > Has the security tightened up on OpenSSH since the book? Assistance > would be appreciated - I have a feeling I'm missing something simple in > all the ssh options, files and permissions! Unfortunately there are several places where authentication can occur (or not) in an ssh connection. Running ssh -v when trying can often help figure out what fails. I'm not quite clear on what you mean by "getting automatic password authentication". Ssh/sshd usually come preconfigured in a mode that requires that you enter your password when ssh-connecting to any host running the sshd. To configure it so that you DON'T need a password to run a remote shell command (for example so you can do things like): rgb at lilith|T:104>ssh ganesh ls -ald .ssh drwxr-xr-x 3 rgb prof 4096 Nov 7 2002 .ssh you need to generate a key pair. The tool for this is ssh-keygen, and you can create either RSA (ssh 1) or DSA (ssh 2) keys. One place you COULD be failing is that sshd can be configured to permit logins using keypairs or not. Read man sshd_config. In most versions of ssh these days, I'm pretty sure .shosts and /etc/shosts.equiv is pretty much ignored in favor of the much stronger rsa/dsa host identification. I enclose below a trace of ssh -v in a successful remote login with no password. Note that it gets down to where it needs to authenticate, announces that it can do so via either publickey (a keypair) or a password (where it has already validated the host identity) and then uses the publickey successfully. HTH, rgb -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb at phy.duke.edu rgb at lilith|T:109>ssh -v ganesh OpenSSH_3.5p1, SSH protocols 1.5/2.0, OpenSSL 0x0090701f debug1: Reading configuration data /etc/ssh/ssh_config debug1: Applying options for * debug1: Rhosts Authentication disabled, originating port will not be trusted. debug1: ssh_connect: needpriv 0 debug1: Connecting to ganesh [152.3.182.51] port 22. debug1: Connection established. debug1: identity file /home/rgb/.ssh/identity type 0 debug1: identity file /home/rgb/.ssh/id_rsa type -1 debug1: identity file /home/rgb/.ssh/id_dsa type 2 debug1: Remote protocol version 1.99, remote software version OpenSSH_3.1p1 debug1: match: OpenSSH_3.1p1 pat OpenSSH_2.*,OpenSSH_3.0*,OpenSSH_3.1* debug1: Enabling compatibility mode for protocol 2.0 debug1: Local version string SSH-2.0-OpenSSH_3.5p1 debug1: SSH2_MSG_KEXINIT sent debug1: SSH2_MSG_KEXINIT received debug1: kex: server->client aes128-cbc hmac-md5 none debug1: kex: client->server aes128-cbc hmac-md5 none debug1: SSH2_MSG_KEX_DH_GEX_REQUEST sent debug1: expecting SSH2_MSG_KEX_DH_GEX_GROUP debug1: dh_gen_key: priv key bits set: 127/256 debug1: bits set: 1577/3191 debug1: SSH2_MSG_KEX_DH_GEX_INIT sent debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY debug1: Host 'ganesh' is known and matches the RSA host key. debug1: Found key in /home/rgb/.ssh/known_hosts2:1 debug1: bits set: 1617/3191 debug1: ssh_rsa_verify: signature correct debug1: kex_derive_keys debug1: newkeys: mode 1 debug1: SSH2_MSG_NEWKEYS sent debug1: waiting for SSH2_MSG_NEWKEYS debug1: newkeys: mode 0 debug1: SSH2_MSG_NEWKEYS received debug1: done: ssh_kex2. debug1: send SSH2_MSG_SERVICE_REQUEST debug1: service_accept: ssh-userauth debug1: got SSH2_MSG_SERVICE_ACCEPT debug1: authentications that can continue: publickey,password debug1: next auth method to try is publickey debug1: try privkey: /home/rgb/.ssh/id_rsa debug1: try pubkey: /home/rgb/.ssh/id_dsa debug1: input_userauth_pk_ok: pkalg ssh-dss blen 435 lastkey 0x808bb10 hint 2 debug1: read PEM private key done: type DSA debug1: ssh-userauth2 successful: method publickey debug1: channel 0: new [client-session] debug1: send channel open 0 debug1: Entering interactive session. debug1: ssh_session2_setup: id 0 debug1: channel request 0: pty-req debug1: Requesting X11 forwarding with authentication spoofing. debug1: channel request 0: x11-req debug1: channel request 0: shell debug1: fd 3 setting TCP_NODELAY debug1: channel 0: open confirm rwindow 0 rmax 32768 Last login: Thu Jun 12 13:56:58 2003 from rgb.adsl.duke.edu rgb at ganesh|T:101> _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From widyono at cis.upenn.edu Thu Jun 12 14:28:44 2003 From: widyono at cis.upenn.edu (Daniel Widyono) Date: Thu, 12 Jun 2003 14:28:44 -0400 Subject: Configuring OpenSSH 3.5p1 In-Reply-To: References: <1055439098.27276.31.camel@orion-2> Message-ID: <20030612182844.GA14762@central.cis.upenn.edu> .shosts and shosts.equiv are disabled for root (check the source, last I checked it was hard-coded with no option). Are you having trouble with root, and are able to use .shosts or shosts.equiv for any other account? Regards, Dan W. On Thu, Jun 12, 2003 at 02:11:41PM -0400, Robert G. Brown wrote: > On 12 Jun 2003, John Harrop wrote: > > ssh-agent handles the passphrase fine, but I can't seem to get > > /etc/shosts.equiv or .shosts to work with the password. > In most versions of ssh these days, I'm pretty sure .shosts and > /etc/shosts.equiv is pretty much ignored in favor of the much stronger > rsa/dsa host identification. -- -- Daniel Widyono http://www.cis.upenn.edu/~widyono -- Liniac Project, CIS Dept., SEAS, University of Pennsylvania -- Mail: CIS Dept, 302 Levine 3330 Walnut St Philadelphia, PA 19104 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From bryan at ayesha.phys.Virginia.EDU Thu Jun 12 16:20:43 2003 From: bryan at ayesha.phys.Virginia.EDU (Bryan K. Wright) Date: Thu, 12 Jun 2003 16:20:43 -0400 Subject: Data corruption w/ tg3 and bcm5700 drivers In-Reply-To: Message from "Bryan K. Wright" of "Tue, 03 Jun 2003 10:36:24 EDT." <200306031436.h53EaPJ29643@ayesha.phys.Virginia.EDU> Message-ID: <200306122020.h5CKKhx27905@ayesha.phys.Virginia.EDU> Hi folks, (A recap for those who weren't tuned in earlier: I've been having data-corruption problems with Broadcom-based 3com Gigabit cards.) Here's an interesting new datum: The cards were previously in a 64-bit PCI slot, but they'll also work in a 32-bit slot. Moving them to the 32-bit slot apparently gets rid of the data corruption problem (we're currently using the bcm5700 driver.) Bryan -- =============================================================================== Bryan Wright |"If you take cranberries and stew them like Physics Department | applesauce, they taste much more like prunes University of Virginia | than rhubarb does." -- Groucho Charlottesville, VA 22901 | (434) 924-7218 | bryan at virginia.edu =============================================================================== _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From hahn at physics.mcmaster.ca Thu Jun 12 18:28:00 2003 From: hahn at physics.mcmaster.ca (Mark Hahn) Date: Thu, 12 Jun 2003 18:28:00 -0400 (EDT) Subject: ideal motherboard for diskless beowulf In-Reply-To: Message-ID: > > I am in the process of designing a 12-node diskless > > cluster. I am going with AMD CPUs, but not quite why are you fixed on AMD? in my opinion, there are several very important features which are scarce in the Athlon world: - fast memory. the basic dual-Xeon now has 2xpc2100 ram, which is critically important to a lot of codes. Opterons are a whole different story, of course, but then again, uniprocessor 6.4 GB/s P4c's are, too. don't even get me started about quad Itanium2's ;) - onboard, PXE-enabled gigabit, preferably two. for quite a while, Intel didn't have the best smell in the Linux world when it came to NICs, but the e1000 seems quite fine now. - PCIX - fast implementations of PCI are critical for getting serious cluster interconnect to perform. course, "serious" interconnect means big bucks, which makes this relevant only for certain categories of cluster. > > confident which mobo to use, for serving the following > > with a limited budget: > > > > - (must) onboard LAN are there *any* athlon boards with gigabit onboard? some, I guess. > > - (must) proven Linux (RH7.2) compatibility that's not really saying much - I think you'd be fine with anything recent from VIA, and I think SiS support has gotten better recently. of course, AMD still makes the only dual-athlon chipset, right? choosing from among the top 5 motherboard vendors is a good way to get decent compatibility... > > - (ideal) micro-atx hmm. certainly possible, but that puts you into the small/cheap category of clusters (as opposed to the large and probably more expensive, also known as "server"). u-atx just so it fits in small (cheap) cases? > > - (ideal) wide range of CPU support, i.e. from Duron > > sub-GHz up to XP 2600+ that shouldn't be a problem. though I'm not sure it would make sense to build a cluster that mixed CPUs like that, or one that went with Durons (simply because Athlon's at their sweetspot offer quite a bit better price/performance.) > > - (bonus) some overclocking capabilities just say no. seriously. it's hard to imagine why you'd want a compute cluster if you couldn't trust the results. > UP or MP? MP can be a hair cheaper per CPU, if your tasks aren't far be it from be to disagree with rgb, but I'm not so sure about this. at the very least, you'd have to add that it depends on the number of duplicated components for UP. it's clear that MP boards tend to cost 2-4x as much as uni, and MP chips are noticably more expensive as well. so it boils down to an extra disk, PS, case, switch port, maybe video card. (assuming you have the same ram-per-cpu in either case.) > fiasco) been generally satisfied with them. I use Tyan's (e.g. Tyan > 2466 or better) for duals -- they have onboard 100BT or better (current > motherboards have dual GigE NICs onboard, as well as video). oh, cool, I hadn't looked at Tyan dual-athlon boards recently - they've got both 64x66 and gigabit. > from 7-9. Beware motherboards with onboard RTL 8139 NICs -- they are > cheap, common, and suck. I'd plan on adding a decent NIC a la carte if > I "had" to pick a motherboard with an onboard RTL. 8139's are fairly wimpy, but then again, 100bT is pretty miserable - do you really worry about whether your 8139 is consuming a few more cycles than a smarter NIC would, given that it's only doing ~9 MB/s? _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From Mark at markandrewsmith.co.uk Thu Jun 12 19:01:54 2003 From: Mark at markandrewsmith.co.uk (Mark Andrew Smith) Date: Fri, 13 Jun 2003 00:01:54 +0100 Subject: Data corruption w/ tg3 and bcm5700 drivers In-Reply-To: <200306122020.h5CKKhx27905@ayesha.phys.Virginia.EDU> Message-ID: May I ask what motherboards (and possibly BIOS) are you using? Cheers, Mark. -----Original Message----- From: beowulf-admin at scyld.com [mailto:beowulf-admin at scyld.com]On Behalf Of Bryan K. Wright Sent: 12 June 2003 21:21 To: beowulf at beowulf.org Cc: bryan at ayesha.phys.Virginia.EDU Subject: Re: Data corruption w/ tg3 and bcm5700 drivers Hi folks, (A recap for those who weren't tuned in earlier: I've been having data-corruption problems with Broadcom-based 3com Gigabit cards.) Here's an interesting new datum: The cards were previously in a 64-bit PCI slot, but they'll also work in a 32-bit slot. Moving them to the 32-bit slot apparently gets rid of the data corruption problem (we're currently using the bcm5700 driver.) Bryan -- ============================================================================ === Bryan Wright |"If you take cranberries and stew them like Physics Department | applesauce, they taste much more like prunes University of Virginia | than rhubarb does." -- Groucho Charlottesville, VA 22901 | (434) 924-7218 | bryan at virginia.edu ============================================================================ === _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf This email has been scanned for viruses by NetBenefit using Sophos anti-virus technology This email has been scanned for viruses by NetBenefit using Sophos anti-virus technology _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From smuelas at mecanica.upm.es Fri Jun 13 02:31:11 2003 From: smuelas at mecanica.upm.es (Santiago Muelas) Date: Fri, 13 Jun 2003 08:31:11 +0200 Subject: Configuring OpenSSH 3.5p1 In-Reply-To: References: <1055439098.27276.31.camel@orion-2> Message-ID: <20030613083111.1f0f6394.smuelas@mecanica.upm.es> I'm running RedHat-9. Everything works perfectly with the simple generation of the key (Using ssh-keygen and putting the result on directory .ssh). Very clear explained on "man ssh". Nothing more to be done. On Thu, 12 Jun 2003 14:11:41 -0400 (EDT) "Robert G. Brown" wrote: > On 12 Jun 2003, John Harrop wrote: > > > I'm currently building a new cluster which unlike the older, more > > isolated one will be using OpenSSH. How are others configuring their > > ssh? > > > > I've been trying to keep the system in the 'recommended' settings of the > > O'Reilly SSH book but I'm having trouble getting automatic password > > authentication. (The passphrase authentication is fine.) For example, > > ssh-agent handles the passphrase fine, but I can't seem to get > > /etc/shosts.equiv or .shosts to work with the password. > > > > Has the security tightened up on OpenSSH since the book? Assistance > > would be appreciated - I have a feeling I'm missing something simple in > > all the ssh options, files and permissions! > > Unfortunately there are several places where authentication can occur > (or not) in an ssh connection. Running ssh -v when trying can often > help figure out what fails. > > I'm not quite clear on what you mean by "getting automatic password > authentication". Ssh/sshd usually come preconfigured in a mode that > requires that you enter your password when ssh-connecting to any host > running the sshd. To configure it so that you DON'T need a password to > run a remote shell command (for example so you can do things like): > > rgb at lilith|T:104>ssh ganesh ls -ald .ssh > drwxr-xr-x 3 rgb prof 4096 Nov 7 2002 .ssh > > you need to generate a key pair. The tool for this is ssh-keygen, and > you can create either RSA (ssh 1) or DSA (ssh 2) keys. One place you > COULD be failing is that sshd can be configured to permit logins using > keypairs or not. Read man sshd_config. > > In most versions of ssh these days, I'm pretty sure .shosts and > /etc/shosts.equiv is pretty much ignored in favor of the much stronger > rsa/dsa host identification. > > I enclose below a trace of ssh -v in a successful remote login with no > password. Note that it gets down to where it needs to authenticate, > announces that it can do so via either publickey (a keypair) or a > password (where it has already validated the host identity) and then > uses the publickey successfully. > > HTH, > > rgb > > -- > Robert G. Brown http://www.phy.duke.edu/~rgb/ > Duke University Dept. of Physics, Box 90305 > Durham, N.C. 27708-0305 > Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb at phy.duke.edu > > > rgb at lilith|T:109>ssh -v ganesh > OpenSSH_3.5p1, SSH protocols 1.5/2.0, OpenSSL 0x0090701f > debug1: Reading configuration data /etc/ssh/ssh_config > debug1: Applying options for * > debug1: Rhosts Authentication disabled, originating port will not be > trusted. > debug1: ssh_connect: needpriv 0 > debug1: Connecting to ganesh [152.3.182.51] port 22. > debug1: Connection established. > debug1: identity file /home/rgb/.ssh/identity type 0 > debug1: identity file /home/rgb/.ssh/id_rsa type -1 > debug1: identity file /home/rgb/.ssh/id_dsa type 2 > debug1: Remote protocol version 1.99, remote software version > OpenSSH_3.1p1 > debug1: match: OpenSSH_3.1p1 pat OpenSSH_2.*,OpenSSH_3.0*,OpenSSH_3.1* > debug1: Enabling compatibility mode for protocol 2.0 > debug1: Local version string SSH-2.0-OpenSSH_3.5p1 > debug1: SSH2_MSG_KEXINIT sent > debug1: SSH2_MSG_KEXINIT received > debug1: kex: server->client aes128-cbc hmac-md5 none > debug1: kex: client->server aes128-cbc hmac-md5 none > debug1: SSH2_MSG_KEX_DH_GEX_REQUEST sent > debug1: expecting SSH2_MSG_KEX_DH_GEX_GROUP > debug1: dh_gen_key: priv key bits set: 127/256 > debug1: bits set: 1577/3191 > debug1: SSH2_MSG_KEX_DH_GEX_INIT sent > debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY > debug1: Host 'ganesh' is known and matches the RSA host key. > debug1: Found key in /home/rgb/.ssh/known_hosts2:1 > debug1: bits set: 1617/3191 > debug1: ssh_rsa_verify: signature correct > debug1: kex_derive_keys > debug1: newkeys: mode 1 > debug1: SSH2_MSG_NEWKEYS sent > debug1: waiting for SSH2_MSG_NEWKEYS > debug1: newkeys: mode 0 > debug1: SSH2_MSG_NEWKEYS received > debug1: done: ssh_kex2. > debug1: send SSH2_MSG_SERVICE_REQUEST > debug1: service_accept: ssh-userauth > debug1: got SSH2_MSG_SERVICE_ACCEPT > debug1: authentications that can continue: publickey,password > debug1: next auth method to try is publickey > debug1: try privkey: /home/rgb/.ssh/id_rsa > debug1: try pubkey: /home/rgb/.ssh/id_dsa > debug1: input_userauth_pk_ok: pkalg ssh-dss blen 435 lastkey 0x808bb10 > hint 2 > debug1: read PEM private key done: type DSA > debug1: ssh-userauth2 successful: method publickey > debug1: channel 0: new [client-session] > debug1: send channel open 0 > debug1: Entering interactive session. > debug1: ssh_session2_setup: id 0 > debug1: channel request 0: pty-req > debug1: Requesting X11 forwarding with authentication spoofing. > debug1: channel request 0: x11-req > debug1: channel request 0: shell > debug1: fd 3 setting TCP_NODELAY > debug1: channel 0: open confirm rwindow 0 rmax 32768 > Last login: Thu Jun 12 13:56:58 2003 from rgb.adsl.duke.edu > rgb at ganesh|T:101> > > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org > To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- Santiago Muelas Profesor de Resistencia de Materiales y C?lculo de Estructuras ETSI de Caminos, Canales y Puertos (U.P.M) smuelas at mecanica.upm.es http://w3.mecanica.upm.es/~smuelas _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From smuelas at mecanica.upm.es Fri Jun 13 02:32:27 2003 From: smuelas at mecanica.upm.es (Santiago Muelas) Date: Fri, 13 Jun 2003 08:32:27 +0200 Subject: ideal motherboard for diskless beowulf In-Reply-To: <20030612175919.GC2630@mail.majorlinux.com> References: <200306111521.h5BFLQY28078@NewBlue.Scyld.com> <20030612175919.GC2630@mail.majorlinux.com> Message-ID: <20030613083227.6088fc60.smuelas@mecanica.upm.es> Also the last Shuttle, but it is something very simple to install.... On Fri, 13 Jun 2003 01:59:19 +0800 Major Chai Mee Joon wrote: > Hi, > > > On Wed, 11 Jun 2003, kolokoutsas konstantinos wrote: > > > > > Hello, > > > > > > I am in the process of designing a 12-node diskless > > > cluster. I am going with AMD CPUs, but not quite > > > confident which mobo to use, for serving the following > > > with a limited budget: > > > > > > - (must) onboard LAN > > > - (must) proven Linux (RH7.2) compatibility > > > - (ideal) micro-atx > > > - (ideal) wide range of CPU support, i.e. from Duron > > > sub-GHz up to XP 2600+ > > > - (bonus) some overclocking capabilities > > You just have to be careful if you're using the the latest boards > unless you like to get your hands dirty. > > eg. Asus A7N8X Deluxe Board based on nVidia's Nforce2-Chipset, > as the onboard Nvidia Ethernet requires the driver supplied by nVidia. > > > Regards, > > Major Chai > > MajorLinux.com > > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org > To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- Santiago Muelas Profesor de Resistencia de Materiales y C?lculo de Estructuras ETSI de Caminos, Canales y Puertos (U.P.M) smuelas at mecanica.upm.es http://w3.mecanica.upm.es/~smuelas _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From julien.leduc at imag.fr Fri Jun 13 09:14:01 2003 From: julien.leduc at imag.fr (Julien Leduc) Date: Fri, 13 Jun 2003 13:14:01 +0000 Subject: Configuring OpenSSH 3.5p1 In-Reply-To: <1055439098.27276.31.camel@orion-2> References: <1055439098.27276.31.camel@orion-2> Message-ID: <20030613131401.4967e2d7.julien.leduc@imag.fr> On 12 Jun 2003 10:31:38 -0700 John Harrop wrote: > I'm currently building a new cluster which unlike the older, more > isolated one will be using OpenSSH. How are others configuring their > ssh? He we have a 121 nodes cluster. The main server, that handle open PBS server and NIS server, is due to perform some administration tasks through SSH (with rshp/mput tools that allow good perfs over ssh admin tasks such as running remote scripts or copying files efficienltly over the cluster), so here we need to ssh on every node without entering a password. To do this, we have 2 pairs of dsa keys (generated with ssh-keygen -t dsa) for root: one for the server and another that is common to all the computing nodes, and we allow the root ssh login from the server on all the nodes by copying his public dsa key in the authorized_keys file (so all you have to do is: scp /root/.ssh/id_dsa.pub root at node_without_pass:/root/.ssh/authorized_keys the first time, you will have to type the pass but then it works :) ). The main problem you will face is how to spread the authorized keys file on all the nodes (If you have a lot of nodes...). Julien Leduc Laboratoire ID-IMAG _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From chris_oubre at hotmail.com Fri Jun 13 15:17:01 2003 From: chris_oubre at hotmail.com (Chris Oubre) Date: Fri, 13 Jun 2003 14:17:01 -0500 Subject: Opteron Large Memory Compilers? Message-ID: <001e01c331e0$5e30fd10$25462a80@rice.edu> Hello, We are looking into the feasibility of using Opterons in on cluster. One major issue we had when we benchmarked on them was the lack of large memory support in the compilers we used. We tried to run large memory jobs on an Opteron using the Portland Compilers and found that it would not compile jobs larger than 2 Gigs. We contacted the Portland Group and they responded that it was a know bug. We haven't tried again in a while and we were wondering if anyone knows if this has been fixed? Has anyone run jobs > 2 Gigs on a Opteron? And if so which compiler did you use. And finally what is the current consensus on which Opteron compiler is best for large memory jobs. Thank you. **************************************************** Christopher D. Oubre * email: chris_oubre at hotmail.com * research: http://cmt.rice.edu/~coubre * Web: http://www.angelfire.com/la2/oubre * Hangout: http://pub44.ezboard.com/bsouthterrebonne * Phone:(713)348-3541 Fax: (713)348-4150 * Rice University * Department of Physics, M.S. 61 * 6100 Main St. ^-^ * Houston, Tx 77251-1892, USA (O O) * -= Phlax=- ( v ) * ************************************m*m************* _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From ctibirna at giref.ulaval.ca Fri Jun 13 16:06:15 2003 From: ctibirna at giref.ulaval.ca (Cristian Tibirna) Date: Fri, 13 Jun 2003 16:06:15 -0400 Subject: strange behavior. Hints? Message-ID: <200306131606.15570.ctibirna@giref.ulaval.ca> Hello I have a small (16x(2P-IV / 4G RAM / myri), redhat-7.3 with xCAT) cluster that we use with good results since a few months. I am about to prepare some reports about speedups of our in-house FEM code (parallelised with MPICH-GM and using PETSc for solvers) and I'm doing some tests which consist mostly of launching a same (rather big) FEM simulation on decreasing number of nodes: for n in `seq -f"%02d" 16 1`; do mpirun -np $n ./simulator; done (OK, the script is a bit more complicated than this, but you get the idea). A strange phenomenon started to appear a few weeks ago. The simulation works very well for all n, apart n=10 and n=11. For these, the program segfaults on 2 to 5 of the nodes and of course this locks the execution (MPI waits) and I have to kill it. This is reliably reproductible. I'm absolutely sure my code has no special code dealing with the number of nodes (highly generalised OOP C++ code). Now, I start to believe there's some strange bug in the Myrinet hardware/software. But I feel this is a really wild guess. I plan to start investigation by tearing all the components apart (mpi, petsc, myrinet drivers) and test them again. But this is a really big battle with little chances of success, given that I can positively see all working correctly most of the time (i.e., running on 16, 15, 4, 2 etc. nodes works OK). I wonder if anybody saw such a behavior before and has some (more valuable) hints (than my wild guesses) for where to look and how to do it, eventually. Thanks a lot for your attention. -- Cristian Tibirna (1-418-) 656-2131 / 4340 Laval University - Quebec, CAN ... http://www.giref.ulaval.ca/~ctibirna Research profesional at GIREF ... ctibirna at giref.ulaval.ca PhD Student - Chemical Engng ... tibirna at gch.ulaval.ca _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From patrick at myri.com Fri Jun 13 16:58:53 2003 From: patrick at myri.com (Patrick Geoffray) Date: 13 Jun 2003 16:58:53 -0400 Subject: strange behavior. Hints? In-Reply-To: <200306131606.15570.ctibirna@giref.ulaval.ca> References: <200306131606.15570.ctibirna@giref.ulaval.ca> Message-ID: <1055537934.573.184.camel@asterix> Hi Christian, On Fri, 2003-06-13 at 16:06, Cristian Tibirna wrote: > Now, I start to believe there's some strange bug in the Myrinet > hardware/software. But I feel this is a really wild guess. Open an help ticket with help at myri.com and we will investigate as soon as possible. If your application segfaults, you can recompile MPICH-GM and your application with "-O0 -g" and run it under a debugger (not a problem under gdb for 10 processes). Once we know where it segfaults (from the gdb trace), we can find why. Regards Patrick -- Patrick Geoffray Myricom, Inc. http://www.myri.com _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From v_454 at yahoo.com Fri Jun 13 15:51:44 2003 From: v_454 at yahoo.com (Steve Elliot) Date: Fri, 13 Jun 2003 12:51:44 -0700 (PDT) Subject: Large File Support for RedHat Linux 8.0 Kernal 2.4.18-14 Message-ID: <20030613195144.41888.qmail@web40505.mail.yahoo.com> Our RedHat 8.0 doesn't seem to support Large file system out of the box. We have a testing program running on an ext3 file system, with both _LARGEFILE_SOURCE and _FILE_OFFSET_BITS=64 defined, the lseek() function failed when the file pointer goes beyound 2G. What's the steps to turn the large file support on? and how do I know it is on? __________________________________ Do you Yahoo!? Yahoo! Calendar - Free online calendar with sync to Outlook(TM). http://calendar.yahoo.com _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From tlovie at pokey.mine.nu Fri Jun 13 17:19:41 2003 From: tlovie at pokey.mine.nu (Thomas Lovie) Date: Fri, 13 Jun 2003 17:19:41 -0400 Subject: Large File Support for RedHat Linux 8.0 Kernal 2.4.18-14 In-Reply-To: <20030613195144.41888.qmail@web40505.mail.yahoo.com> Message-ID: <000301c331f1$83f189f0$1806a8c0@fishnet.exigentsi.com> I might be mistaken here, but I didn't think that ext3 could support large files. I thought you have to go to XFS or reiserfs. Tom. -----Original Message----- From: beowulf-admin at scyld.com [mailto:beowulf-admin at scyld.com] On Behalf Of Steve Elliot Sent: Friday, June 13, 2003 3:52 PM To: beowulf at beowulf.org Subject: Large File Support for RedHat Linux 8.0 Kernal 2.4.18-14 Our RedHat 8.0 doesn't seem to support Large file system out of the box. We have a testing program running on an ext3 file system, with both _LARGEFILE_SOURCE and _FILE_OFFSET_BITS=64 defined, the lseek() function failed when the file pointer goes beyound 2G. What's the steps to turn the large file support on? and how do I know it is on? __________________________________ Do you Yahoo!? Yahoo! Calendar - Free online calendar with sync to Outlook(TM). http://calendar.yahoo.com _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From joe.griffin at mscsoftware.com Fri Jun 13 17:25:43 2003 From: joe.griffin at mscsoftware.com (Joe Griffin) Date: Fri, 13 Jun 2003 14:25:43 -0700 Subject: Large File Support for RedHat Linux 8.0 Kernal 2.4.18-14 In-Reply-To: <20030613195144.41888.qmail@web40505.mail.yahoo.com> References: <20030613195144.41888.qmail@web40505.mail.yahoo.com> Message-ID: <3EEA4157.8030101@mscsoftware.com> My RedHat 8.0 system writes files larger than 8 Gb. everest <106> cat bigwrite.f program bigwrite integer buf(1024) do i=1,1024 buf(i)=i enddo open( unit=21, access='direct', form='unformatted', 1 status='unknown', recl=4096, err=700 ) irec=1 c do i=1,524288 c do i=1,2*524288 do i=1,4*524288 irec=i c write(6,*) 'writing, irec =', irec write(21, rec=irec, err=800) buf enddo write(6,*) 'write done, irec =', irec close( unit=21 ) stop 700 continue write(6,*) 'open err' stop 800 continue write(6,*) 'write err, irec=', irec close( unit=21 ) end everest <107> ifc bigwrite.f program BIGWRITE 26 Lines Compiled everest <108> a.out write done, irec = 2097152 everest <109> ls -ltr fort.21 -rw-r--r-- 1 jjg develop 8589934592 Jun 13 14:27 fort.21 My file system is ext2. Regards, Joe Griffin Steve Elliot wrote: >Our RedHat 8.0 doesn't seem to support Large file >system out of the box. We have a testing program >running on an ext3 file system, with both >_LARGEFILE_SOURCE and _FILE_OFFSET_BITS=64 defined, >the lseek() function failed when the file pointer goes >beyound 2G. What's the steps to turn the large file >support on? and how do I know it is on? > >__________________________________ >Do you Yahoo!? >Yahoo! Calendar - Free online calendar with sync to Outlook(TM). >http://calendar.yahoo.com >_______________________________________________ >Beowulf mailing list, Beowulf at beowulf.org >To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf > > > > _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From jim at ks.uiuc.edu Fri Jun 13 17:49:21 2003 From: jim at ks.uiuc.edu (Jim Phillips) Date: Fri, 13 Jun 2003 16:49:21 -0500 (CDT) Subject: Large File Support for RedHat Linux 8.0 Kernal 2.4.18-14 In-Reply-To: <3EEA4157.8030101@mscsoftware.com> Message-ID: Hi, You also need to add the O_LARGEFILE flag to your open() call. -Jim > Steve Elliot wrote: > > >Our RedHat 8.0 doesn't seem to support Large file > >system out of the box. We have a testing program > >running on an ext3 file system, with both > >_LARGEFILE_SOURCE and _FILE_OFFSET_BITS=64 defined, > >the lseek() function failed when the file pointer goes > >beyound 2G. What's the steps to turn the large file > >support on? and how do I know it is on? _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From becker at scyld.com Fri Jun 13 16:56:28 2003 From: becker at scyld.com (Donald Becker) Date: Fri, 13 Jun 2003 16:56:28 -0400 (EDT) Subject: Opteron Large Memory Compilers? In-Reply-To: <001e01c331e0$5e30fd10$25462a80@rice.edu> Message-ID: On Fri, 13 Jun 2003, Chris Oubre wrote: > We are looking into the feasibility of using Opterons in on > cluster. One major issue we had when we benchmarked on them was the > lack of large memory support in the compilers we used. We tried to run > large memory jobs on an Opteron using the Portland Compilers and found > that it would not compile jobs larger than 2 Gigs. Could you please clarify? Does the compiler not compile and link an executable larger than 2GB, or Does the program fail when dynamic libraries put the total size >2GB, or Does the executable not work with a data set larger than 2GB? It must be the last, since even the most bloated programs don't go much over 100MB of compiled code. > Has anyone run jobs > 2 Gigs on a Opteron? And if so which > compiler did you use. And finally what is the current consensus on which > Opteron compiler is best for large memory jobs. -- Donald Becker becker at scyld.com Scyld Computing Corporation http://www.scyld.com 914 Bay Ridge Road, Suite 220 Scyld Beowulf cluster system Annapolis MD 21403 410-990-9993 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From glen at cert.ucr.edu Fri Jun 13 18:38:50 2003 From: glen at cert.ucr.edu (Glen Kaukola) Date: Fri, 13 Jun 2003 15:38:50 -0700 Subject: Large File Support for RedHat Linux 8.0 Kernal 2.4.18-14 In-Reply-To: <20030613195144.41888.qmail@web40505.mail.yahoo.com> References: <20030613195144.41888.qmail@web40505.mail.yahoo.com> Message-ID: <3EEA527A.1030303@cert.ucr.edu> Steve Elliot wrote: >Our RedHat 8.0 doesn't seem to support Large file >system out of the box. We have a testing program >running on an ext3 file system, with both >_LARGEFILE_SOURCE and _FILE_OFFSET_BITS=64 defined, >the lseek() function failed when the file pointer goes >beyound 2G. What's the steps to turn the large file >support on? and how do I know it is on? > We had a similar problem, and it turned to be tcsh. tcsh doesn't support files over 2GB it seems, so we ended up having to convert all of our scripts over to bash. Glen _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From landman at scalableinformatics.com Fri Jun 13 20:50:11 2003 From: landman at scalableinformatics.com (Joseph Landman) Date: 13 Jun 2003 20:50:11 -0400 Subject: Large File Support for RedHat Linux 8.0 Kernal 2.4.18-14 In-Reply-To: <3EEA527A.1030303@cert.ucr.edu> References: <20030613195144.41888.qmail@web40505.mail.yahoo.com> <3EEA527A.1030303@cert.ucr.edu> Message-ID: <1055551811.4661.67.camel@protein.scalableinformatics.com> On Fri, 2003-06-13 at 18:38, Glen Kaukola wrote: > We had a similar problem, and it turned to be tcsh. tcsh doesn't > support files over 2GB it seems, so we ended up having to convert all of > our scripts over to bash. I simply pulled down the latest TCSH sources, and recompiled for large file support. I replaced the installed tcsh with the compiled tcsh. See http://bioinformatics.org/pipermail/biodevelopers/2002-May/000026.html This should be a far simpler solution than converting all your shell scripts. It would introduce far fewer bugs I would think, than a port. Hopefully the distro vendors (RH and others) in question will catch the hint and make sure that half of their distro is not missing the right compilation options. Shells, editors, file managers, etc all need this. -- Joseph Landman, Ph.D Scalable Informatics LLC email: landman at scalableinformatics.com web: http://scalableinformatics.com phone: +1 734 612 4615 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From johnh at sjgeophysics.com Sat Jun 14 04:18:25 2003 From: johnh at sjgeophysics.com (John Harrop) Date: Sat, 14 Jun 2003 01:18:25 -0700 Subject: Configuring OpenSSH 3.5p1 In-Reply-To: <20030613083111.1f0f6394.smuelas@mecanica.upm.es> References: <1055439098.27276.31.camel@orion-2> <20030613083111.1f0f6394.smuelas@mecanica.upm.es> Message-ID: <1055578706.1991.14.camel@cr517045-a> Thanks for the suggestions and comments. You got me thinking on some new lines that solved the problem. Actually there were two problems: One of my keys had been damaged or changed so that caused some problems. I also found out that the $HOME/.ssh directory must have no greater than 744 permissions (and I think it installs as 766). I recall being warned about this before - but its not as obvious in the docs as permission requirements of some files. The new system I'm working on is also running on RedHat 9. I'm glad to hear I'm not the only one using it. Cheers, John Harrop Geologist GIS and Software Development Manager SJ Geophysics Ltd On Thu, 2003-06-12 at 23:31, Santiago Muelas wrote: > I'm running RedHat-9. Everything works perfectly with the simple generation of the key (Using ssh-keygen and putting the result on directory .ssh). Very clear explained on "man ssh". Nothing more to be done. > > > > > On Thu, 12 Jun 2003 14:11:41 -0400 (EDT) > "Robert G. Brown" wrote: > > > On 12 Jun 2003, John Harrop wrote: > > > > > I'm currently building a new cluster which unlike the older, more > > > isolated one will be using OpenSSH. How are others configuring their > > > ssh? > > > > > > I've been trying to keep the system in the 'recommended' settings of the > > > O'Reilly SSH book but I'm having trouble getting automatic password > > > authentication. (The passphrase authentication is fine.) For example, > > > ssh-agent handles the passphrase fine, but I can't seem to get > > > /etc/shosts.equiv or .shosts to work with the password. > > > > > > Has the security tightened up on OpenSSH since the book? Assistance > > > would be appreciated - I have a feeling I'm missing something simple in > > > all the ssh options, files and permissions! > > > > Unfortunately there are several places where authentication can occur > > (or not) in an ssh connection. Running ssh -v when trying can often > > help figure out what fails. > > > > I'm not quite clear on what you mean by "getting automatic password > > authentication". Ssh/sshd usually come preconfigured in a mode that > > requires that you enter your password when ssh-connecting to any host > > running the sshd. To configure it so that you DON'T need a password to > > run a remote shell command (for example so you can do things like): > > > > rgb at lilith|T:104>ssh ganesh ls -ald .ssh > > drwxr-xr-x 3 rgb prof 4096 Nov 7 2002 .ssh > > > > you need to generate a key pair. The tool for this is ssh-keygen, and > > you can create either RSA (ssh 1) or DSA (ssh 2) keys. One place you > > COULD be failing is that sshd can be configured to permit logins using > > keypairs or not. Read man sshd_config. > > > > In most versions of ssh these days, I'm pretty sure .shosts and > > /etc/shosts.equiv is pretty much ignored in favor of the much stronger > > rsa/dsa host identification. > > > > I enclose below a trace of ssh -v in a successful remote login with no > > password. Note that it gets down to where it needs to authenticate, > > announces that it can do so via either publickey (a keypair) or a > > password (where it has already validated the host identity) and then > > uses the publickey successfully. > > > > HTH, > > > > rgb > > > > -- > > Robert G. Brown http://www.phy.duke.edu/~rgb/ > > Duke University Dept. of Physics, Box 90305 > > Durham, N.C. 27708-0305 > > Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb at phy.duke.edu > > > > > > rgb at lilith|T:109>ssh -v ganesh > > OpenSSH_3.5p1, SSH protocols 1.5/2.0, OpenSSL 0x0090701f > > debug1: Reading configuration data /etc/ssh/ssh_config > > debug1: Applying options for * > > debug1: Rhosts Authentication disabled, originating port will not be > > trusted. > > debug1: ssh_connect: needpriv 0 > > debug1: Connecting to ganesh [152.3.182.51] port 22. > > debug1: Connection established. > > debug1: identity file /home/rgb/.ssh/identity type 0 > > debug1: identity file /home/rgb/.ssh/id_rsa type -1 > > debug1: identity file /home/rgb/.ssh/id_dsa type 2 > > debug1: Remote protocol version 1.99, remote software version > > OpenSSH_3.1p1 > > debug1: match: OpenSSH_3.1p1 pat OpenSSH_2.*,OpenSSH_3.0*,OpenSSH_3.1* > > debug1: Enabling compatibility mode for protocol 2.0 > > debug1: Local version string SSH-2.0-OpenSSH_3.5p1 > > debug1: SSH2_MSG_KEXINIT sent > > debug1: SSH2_MSG_KEXINIT received > > debug1: kex: server->client aes128-cbc hmac-md5 none > > debug1: kex: client->server aes128-cbc hmac-md5 none > > debug1: SSH2_MSG_KEX_DH_GEX_REQUEST sent > > debug1: expecting SSH2_MSG_KEX_DH_GEX_GROUP > > debug1: dh_gen_key: priv key bits set: 127/256 > > debug1: bits set: 1577/3191 > > debug1: SSH2_MSG_KEX_DH_GEX_INIT sent > > debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY > > debug1: Host 'ganesh' is known and matches the RSA host key. > > debug1: Found key in /home/rgb/.ssh/known_hosts2:1 > > debug1: bits set: 1617/3191 > > debug1: ssh_rsa_verify: signature correct > > debug1: kex_derive_keys > > debug1: newkeys: mode 1 > > debug1: SSH2_MSG_NEWKEYS sent > > debug1: waiting for SSH2_MSG_NEWKEYS > > debug1: newkeys: mode 0 > > debug1: SSH2_MSG_NEWKEYS received > > debug1: done: ssh_kex2. > > debug1: send SSH2_MSG_SERVICE_REQUEST > > debug1: service_accept: ssh-userauth > > debug1: got SSH2_MSG_SERVICE_ACCEPT > > debug1: authentications that can continue: publickey,password > > debug1: next auth method to try is publickey > > debug1: try privkey: /home/rgb/.ssh/id_rsa > > debug1: try pubkey: /home/rgb/.ssh/id_dsa > > debug1: input_userauth_pk_ok: pkalg ssh-dss blen 435 lastkey 0x808bb10 > > hint 2 > > debug1: read PEM private key done: type DSA > > debug1: ssh-userauth2 successful: method publickey > > debug1: channel 0: new [client-session] > > debug1: send channel open 0 > > debug1: Entering interactive session. > > debug1: ssh_session2_setup: id 0 > > debug1: channel request 0: pty-req > > debug1: Requesting X11 forwarding with authentication spoofing. > > debug1: channel request 0: x11-req > > debug1: channel request 0: shell > > debug1: fd 3 setting TCP_NODELAY > > debug1: channel 0: open confirm rwindow 0 rmax 32768 > > Last login: Thu Jun 12 13:56:58 2003 from rgb.adsl.duke.edu > > rgb at ganesh|T:101> > > > > _______________________________________________ > > Beowulf mailing list, Beowulf at beowulf.org > > To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf > > > -- > Santiago Muelas > Profesor de Resistencia de Materiales y C?lculo de Estructuras > ETSI de Caminos, Canales y Puertos (U.P.M) > smuelas at mecanica.upm.es http://w3.mecanica.upm.es/~smuelas > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org > To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From k_kolokoutsas at yahoo.co.uk Fri Jun 13 19:48:56 2003 From: k_kolokoutsas at yahoo.co.uk (=?iso-8859-1?q?kolokoutsas=20konstantinos?=) Date: Sat, 14 Jun 2003 00:48:56 +0100 (BST) Subject: ideal motherboard for diskless beowulf In-Reply-To: Message-ID: <20030613234856.5266.qmail@web9908.mail.yahoo.com> Thank you all for the input! This beowulf will be dedicated to running one particular Monte Carlo particle acceleration code already running within RH7.2 and quite dependant on it in many ways, thus the RH7.2 criterium. The 12-node config will serve as a test for a larger cluster, thus the very limited budget, and the choice of (the cheaper) AMD CPUs. The micro-atx form factor is of interest because "I was given the challenge..." of putting as many motherboards in one customized full tower box as possible. Dual/Quad CPU motherboards are not an option, while due to portability issues, racks are out of the question. Thanks once again, Kostas Kolokoutsas __________________________________________________ Yahoo! Plus - For a better Internet experience http://uk.promotions.yahoo.com/yplus/yoffer.html _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From brian at swbell.net Fri Jun 13 23:39:07 2003 From: brian at swbell.net (Brian Macy) Date: Fri, 13 Jun 2003 22:39:07 -0500 (CDT) Subject: Large File Support for RedHat Linux 8.0 Kernal 2.4.18-14 In-Reply-To: <3EEA527A.1030303@cert.ucr.edu> Message-ID: On Fri, 13 Jun 2003, Glen Kaukola wrote: > Steve Elliot wrote: > > >Our RedHat 8.0 doesn't seem to support Large file > >system out of the box. We have a testing program > >running on an ext3 file system, with both > >_LARGEFILE_SOURCE and _FILE_OFFSET_BITS=64 defined, > >the lseek() function failed when the file pointer goes > >beyound 2G. What's the steps to turn the large file > >support on? and how do I know it is on? > > > > We had a similar problem, and it turned to be tcsh. tcsh doesn't > support files over 2GB it seems, so we ended up having to convert all of > our scripts over to bash. > > > Glen > The other thing you need to check is that you're using an off_t type variable for the offset argument to lseek. If you're using an int or a long, you'll still have the 32-bit restriction. Brian -- Brian Macy bcmacy at swbell.net _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From becker at scyld.com Sat Jun 14 06:25:55 2003 From: becker at scyld.com (Donald Becker) Date: Sat, 14 Jun 2003 06:25:55 -0400 (EDT) Subject: Large File Support for RedHat Linux 8.0 Kernal 2.4.18-14 In-Reply-To: <3EEA527A.1030303@cert.ucr.edu> Message-ID: On Fri, 13 Jun 2003, Glen Kaukola wrote: > Steve Elliot wrote: > >Our RedHat 8.0 doesn't seem to support Large file > >system out of the box. We have a testing program > >running on an ext3 file system, with both > >_LARGEFILE_SOURCE and _FILE_OFFSET_BITS=64 defined, > >the lseek() function failed when the file pointer goes > >beyound 2G. What's the steps to turn the large file > >support on? and how do I know it is on? > > > > We had a similar problem, and it turned to be tcsh. tcsh doesn't > support files over 2GB it seems, so we ended up having to convert all of > our scripts over to bash. Use bash version 2.0 or later. Version 1 had LFS problems, some surprising. In some cases you couldn't pipe ("|") or redirect more than 2GB of data. You'll also want to verify your specific 'perl' binary, which is used like a shell. Other places we had to make modifications were the FTP servers and clients, most of which do not have LFS. -- Donald Becker becker at scyld.com Scyld Computing Corporation http://www.scyld.com 914 Bay Ridge Road, Suite 220 Scyld Beowulf cluster system Annapolis MD 21403 410-990-9993 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From douglas at shore.net Sat Jun 14 11:59:01 2003 From: douglas at shore.net (Douglas O'Flaherty) Date: Sat, 14 Jun 2003 11:59:01 -0400 Subject: Opteron Large Memory Compilers? Message-ID: <3EEB4645.4040405@shore.net> Chris: The GA release of the PGI Compiler which supports Opteron is set for 6/30. I don't know of a 'new' beta release since the Opteron launch. There is a 2GB limit on a single data object in the beta, but larger aggregate data sets are supported. The limit is described in the release notes. I certainly would expect it to be changed in the 6/30 release. Was that the issue? There was a new beta of the Absoft Opteron compiler this month, about which I know little. I assume you've already tried gcc. cheers, doug Douglas O'Flaherty _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From maurice at harddata.com Sat Jun 14 13:05:34 2003 From: maurice at harddata.com (Maurice Hilarius) Date: Sat, 14 Jun 2003 11:05:34 -0600 Subject: Opteron Large Memory Compilers? In-Reply-To: <200306141032.h5EAWTU28993@NewBlue.Scyld.com> Message-ID: <5.1.1.6.2.20030614105819.03f917c0@mail.harddata.com> >From: "Chris Oubre" >Subject: Opteron Large Memory Compilers? >Date: Fri, 13 Jun 2003 14:17:01 -0500 > >Hello, > We are looking into the feasibility of using Opterons in on >cluster. One major issue we had when we benchmarked on them was the >lack of large memory support in the compilers we used. We tried to run >large memory jobs on an Opteron using the Portland Compilers and found >that it would not compile jobs larger than 2 Gigs. We contacted the >Portland Group and they responded that it was a know bug. We haven't >tried again in a while and we were wondering if anyone knows if this has >been fixed? I suggest you try GCC. It is not saddled with this problem, and in our tests generally quite a bit better than PGI in performance. You also want to grab the Core Math Library from AMD: http://www.amd.com/us-en/Processors/DevelopWithAMD/0,,30_2252_2282,00.html With our best regards, Maurice W. Hilarius Telephone: 01-780-456-9771 Hard Data Ltd. FAX: 01-780-456-9772 11060 - 166 Avenue mailto:maurice at harddata.com Edmonton, AB, Canada http://www.harddata.com/ T5X 1Y3 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From maurice at harddata.com Mon Jun 16 01:23:11 2003 From: maurice at harddata.com (Maurice Hilarius) Date: Sun, 15 Jun 2003 23:23:11 -0600 Subject: Two CS undergraduates trying to build a prototype beowulf need help with bproc In-Reply-To: <200306151902.h5FJ2uU11623@NewBlue.Scyld.com> Message-ID: <5.1.1.6.2.20030615231836.03c746c0@mail.harddata.com> If you want a working and tested free bproc based distribution you may download it from our ftp server. It is based on RedHat 7.3 (8.0 is rather messy) ftp://ftp.harddata.com/pub/hddcs/3.2/2003-spring/ Docs are included. >Hello friends. >We tried searching the mailing list archive, but the link on the beowulf >site was off, so we decided to try our luck and consult the oracles. :) > <<>> >So, cutting down to our problem: > >We installed redhat 8.0 in two machines, named master and slave1. We >setup nfs and rsh/rexec/rlogin and everything is working fine at kernel >level. We can rsh from/to slave1/master without any password prompts, >and pam looks ok too. > >We installed the 3.2.4 bproc/beowulf kernel patch in kernel 2.4.20, >recompiled it and as it seems, everything looks fine. > >We sucessfully modprobed vmadump and bproc, configured our master's >/etc/beowulf/config and all looked fine to go. > >However, when we tried running bpmaster (as root) it started, then >segfaulted. <<>> >Any sugestions, ideas, flames (those in pvt, pls) and the like would be >very welcome. :) > >Thanks in Advance, > >-- >Marcos Hiroshi Umino With our best regards, Maurice W. Hilarius Telephone: 01-780-456-9771 Hard Data Ltd. FAX: 01-780-456-9772 11060 - 166 Avenue mailto:maurice at harddata.com Edmonton, AB, Canada http://www.harddata.com/ T5X 1Y3 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From egan at sense.net Sun Jun 15 17:59:13 2003 From: egan at sense.net (Egan Ford) Date: Sun, 15 Jun 2003 15:59:13 -0600 Subject: NAS Parallel Benchmarks for Current Hardware In-Reply-To: <3EE8542D.93CE3B57@ideafix.litec.csic.es> Message-ID: <033001c33389$5dde1f00$27b358c7@titan> > Unfortunately, I am not sure linpack and spec reliably measure what is > important for numerical work on a beowulf. I disagree. SPECCPU v1.2 FP benchmarks (http://www.spec.org/cpu2000/CFP2000/): 168.wupwise Fortran 77 Physics / Quantum Chromodynamics 171.swim Fortran 77 Shallow Water Modeling 172.mgrid Fortran 77 Multi-grid Solver: 3D Potential Field 173.applu Fortran 77 Parabolic / Elliptic Partial Differential Equations 177.mesa C 3-D Graphics Library 178.galgel Fortran 90 Computational Fluid Dynamics 179.art C Image Recognition / Neural Networks 183.equake C Seismic Wave Propagation Simulation 187.facerec Fortran 90 Image Processing: Face Recognition 188.ammp C Computational Chemistry 189.lucas Fortran 90 Number Theory / Primality Testing 191.fma3d Fortran 90 Finite-element Crash Simulation 200.sixtrack Fortran 77 High Energy Nuclear Physics Accelerator Design 301.apsi Fortran 77 Meteorology: Pollutant Distribution _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From japa at mhu.eti.br Sat Jun 14 16:12:28 2003 From: japa at mhu.eti.br (Marcos Hiroshi Umino) Date: Sat, 14 Jun 2003 17:12:28 -0300 Subject: Two CS undergraduates trying to build a prototype beowulf need help with bproc Message-ID: <20030614165721.DA2B.JAPA@mhu.eti.br> Hello friends. We tried searching the mailing list archive, but the link on the beowulf site was off, so we decided to try our luck and consult the oracles. :) We are brazilian computer sciences undergraduates from the contryside of Sao Paulo, in a city called Presidente Prudente. Our computer sciences university is new and we are students of the first class, so actually, we are making our lab. It is a great chance to play around, so we two decided to team up and try to build a small prototype beowulf. We are not linux veterans, unfortunately, but we're trying hard and doing some progress. So, cutting down to our problem: We installed redhat 8.0 in two machines, named master and slave1. We setup nfs and rsh/rexec/rlogin and everything is working fine at kernel level. We can rsh from/to slave1/master without any password prompts, and pam looks ok too. We installed the 3.2.4 bproc/beowulf kernel patch in kernel 2.4.20, recompiled it and as it seems, everything looks fine. We sucessfully modprobed vmadump and bproc, configured our master's /etc/beowulf/config and all looked fine to go. However, when we tried running bpmaster (as root) it started, then segfaulted. Unfortunately, our guidebook is a brazilian translation, and I must admit the translator wasn't very good at it. We are trying to search for complementary textbooks (we are students, so books sugestions are great, but unfortunately are out of question). We are a bit lost because the book we're following seems to use an outdated beowulf version, because bproc docs seems to point toward no need for rsh/rexec. Any sugestions, ideas, flames (those in pvt, pls) and the like would be very welcome. :) Thanks in Advance, -- Marcos Hiroshi Umino _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From BrianTJones at RealEdu.org Sat Jun 14 15:31:09 2003 From: BrianTJones at RealEdu.org (BrianTJones at RealEdu.org) Date: Sat, 14 Jun 2003 15:31:09 -0400 Subject: ups units In-Reply-To: <3EDF9C66.5090106@cert.ucr.edu> References: <3EDF9C66.5090106@cert.ucr.edu> Message-ID: Glen, 1 of the things you have to keep in mind is the watt rating & watt load are 2 separate animals; Two ways of calculating your load; 1. put the PC as the ONLY thing running on a DIGITAL power meter for your home or business 2. look up all your parts' active & inactive loads 2b or not. Watts=Volts x Amperage P=IV Brian Sidenote; If you have dead or surplus equipment Donate them! Don't let lead & other heavy metals go into landfills & the water tables!! We're organizing donations to schools & nonprofits throughout the US & helping with the repair work. DonateHW at RealEdu.org for more details _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From andy at ideafix.litec.csic.es Mon Jun 16 04:44:32 2003 From: andy at ideafix.litec.csic.es (andy) Date: Mon, 16 Jun 2003 09:44:32 +0100 Subject: NAS Parallel Benchmarks for Current Hardware References: <033001c33389$5dde1f00$27b358c7@titan> Message-ID: <3EED8370.ADD04801@ideafix.litec.csic.es> Egan Ford wrote: > > > Unfortunately, I am not sure linpack and spec reliably measure what is > > important for numerical work on a beowulf. > > I disagree. I admit I should have been more careful and inserted the word "our" befor "numerical work on beowulf". My concern is partly that they does not test what I want to know and partly that linpack and spec blob ratings shift a lot of hardware. What I need (and I supect many others but I will not speak for them this time) is: * a range of grid sizes so that I can see cache/memory effects * a range of common numercal solvers (covering most of those I use) with differing communication characteristics so that I can gauge the likely behaviour of a range of simulations. No single number can provide such information but something the like NAS parallel benchmarks do. 15-20 years ago linpack stopped being a measure of anything useful when manufacturers adapted their compilers/hardware to generate unrepresentatively large blob ratings. This was brought home when commissioning a small shared memory machine and we could not get near the quoted linpack blob rating using various combinations of optimisation flags (just over 10 blobs instead of nearly 30 blobs if I recall correctly). After a phone call we used the undocumented -linpack compiler flag and bingo. Evidence is growing that the spec blob ratings have suffered in a similar way (though to a lesser degree). See, for example, "Sun breaks another SPEC benchmark" in comp.benchmarks: http://groups.google.com/groups?hl=en&lr=&ie=UTF-8&safe=off&group=comp.benchmarks _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From dwarkanath.mb at vinciti.com Mon Jun 16 07:18:53 2003 From: dwarkanath.mb at vinciti.com (Dwarkanath) Date: Mon, 16 Jun 2003 16:48:53 +0530 Subject: Beowulf for MAC OSX Message-ID: Hello Group, I wish to know, whether Beowulf supports MAC OS X. If yes, can i know the details/pre-requisites/steps to accomplish the same. Thanking you in advance, Dwarkanath MB _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From way2hrishi at rediffmail.com Mon Jun 16 09:19:20 2003 From: way2hrishi at rediffmail.com (Hrishikesh Dewan) Date: 16 Jun 2003 13:19:20 -0000 Subject: Calculating Performance Bencmark Message-ID: <20030616131920.9262.qmail@webmail24.rediffmail.com> Dear Friends & Experts, We are interested in setting up a Beowulf in our Mathematics Department. But the problem that we are facing is calculating the Peak Performance of the cluster. Can anyone help us , point to any links that have such information. Any help will always be highly regarded. Regards, Hrishikesh Dewan thinking in colors -------------- next part -------------- An HTML attachment was scrubbed... URL: From egan at sense.net Mon Jun 16 12:02:49 2003 From: egan at sense.net (Egan Ford) Date: Mon, 16 Jun 2003 10:02:49 -0600 Subject: NAS Parallel Benchmarks for Current Hardware In-Reply-To: <3EED8370.ADD04801@ideafix.litec.csic.es> Message-ID: <03cc01c33420$bd0cb3f0$27b358c7@titan> Unfortunately benchmarking is a game. I tell my customers to benchmark their application. It is the only way to really know what will work best for any particular application. We offer a benchmark lab to assist with that. > -----Original Message----- > From: andy at ideafix.litec.csic.es [mailto:andy at ideafix.litec.csic.es] > Sent: Monday, June 16, 2003 2:45 AM > To: egan at sense.net; beowulf at beowulf.org > Subject: Re: NAS Parallel Benchmarks for Current Hardware > > > Egan Ford wrote: > > > > > Unfortunately, I am not sure linpack and spec reliably > measure what is > > > important for numerical work on a beowulf. > > > > I disagree. > > I admit I should have been more careful and inserted the word "our" > befor "numerical work on beowulf". My concern is partly that they does > not test what I want to know and partly that linpack and spec blob > ratings shift a lot of hardware. > > What I need (and I supect many others but I will not speak > for them this > time) is: > > * a range of grid sizes so that I can see cache/memory effects > > * a range of common numercal solvers (covering most of those > I use) with > differing communication characteristics so that I can gauge the likely > behaviour of a range of simulations. > > No single number can provide such information but something > the like NAS > parallel benchmarks do. > > 15-20 years ago linpack stopped being a measure of anything > useful when > manufacturers adapted their compilers/hardware to generate > unrepresentatively large blob ratings. This was brought home when > commissioning a small shared memory machine and we could not get near > the quoted linpack blob rating using various combinations of > optimisation flags (just over 10 blobs instead of nearly 30 blobs if I > recall correctly). After a phone call we used the > undocumented -linpack > compiler flag and bingo. > > Evidence is growing that the spec blob ratings have suffered in a > similar way (though to a lesser degree). See, for example, "Sun breaks > another SPEC benchmark" in comp.benchmarks: > http://groups.google.com/groups?hl=en&lr=&ie=UTF-8&safe=off&group=comp.bench marks _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rgb at phy.duke.edu Mon Jun 16 13:36:54 2003 From: rgb at phy.duke.edu (Robert G. Brown) Date: Mon, 16 Jun 2003 13:36:54 -0400 (EDT) Subject: ideal motherboard for diskless beowulf In-Reply-To: <20030613234856.5266.qmail@web9908.mail.yahoo.com> Message-ID: On Sat, 14 Jun 2003, kolokoutsas konstantinos wrote: > > Thank you all for the input! > > This beowulf will be dedicated to running one > particular Monte Carlo particle acceleration code > already running within RH7.2 and quite dependant on it > in many ways, thus the RH7.2 criterium. I'm sorry, how is that? a) Can't you just recompile under 7.3 or for that matter 9? b) I would expect most applications compiled for 7.2 to just plain "run" on 7.3, at least. Almost by definition, the major libraries don't change much between minor revision increments. It would certainly be worth it to try running it your application on 7.3. It would also be worth it to start porting your code into a source RPM format so that it can just be rebuilt in the future in five minutes whenever you want to run it on an RPM-supporting architecture.a > The 12-node config will serve as a test for a larger > cluster, thus the very limited budget, and the choice > of (the cheaper) AMD CPUs. > > The micro-atx form factor is of interest because "I > was given the challenge..." of putting as many > motherboards in one customized full tower box as > possible. Dual/Quad CPU motherboards are not an > option, while due to portability issues, racks are out > of the question. Hmm, I don't know what you mean by "portability issues either. I've built both tower/shelf clusters and rack clusters and set them up in e.g. Expo booths for a three day demo.. Tower/shelf clusters are a total PITA to transport. Rack-based clusters are often much easier and achieve a higher CPU density >>if<< you use a rolling rack. At Linux Expo three or four years ago we built a dual rack Netfinity cluster with dual P3 nodes, kindly loaned to us by IBM. The whole thing came off the truck and was set up and running (48 CPUs) in a matter of four hours or so. Taking it down was even faster -- a couple of hours start to finish. It took me almost as long to set up my much smaller tower cluster on a rolling shelf unit I brought from home, with all the cabling and carrying. Nowadays, there are some REALLY cool rolling four post racks. Check out the "department server rack" on http://www.phy.duke.edu/brahma/brahma_tour.php (right above "Seth, Looking Grumpy":-). I don't remember how many U it is, but at a guess perhaps 20U. One could EASILY fit 16 1U cases in it with room to spare, or 32 (or more) CPUs. Monitor and KVM (switch) on top, middlin' hefty UPS on the bottom, and you can literally roll it from room to room without even powering it down! And this is only one choice -- I'll bet there are a variety of options even here. When you are ready to scale up, just buy two post racks and move the nodes into a permanent home... If your money is REALLY tight perhaps you can't afford this, but if you are trying to "sell" the result a rolling rack is going to beat the pants off of a jury riggged tower setup in crowd appeal... rgb > > Thanks once again, > Kostas Kolokoutsas > > > > > __________________________________________________ > Yahoo! Plus - For a better Internet experience > http://uk.promotions.yahoo.com/yplus/yoffer.html > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org > To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf > -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb at phy.duke.edu _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From kpodesta at redbrick.dcu.ie Mon Jun 16 13:04:10 2003 From: kpodesta at redbrick.dcu.ie (Karl Podesta) Date: Mon, 16 Jun 2003 18:04:10 +0100 Subject: Beowulf for MAC OSX In-Reply-To: References: Message-ID: <20030616170410.GB17594@prodigy.Redbrick.DCU.IE> On Mon, Jun 16, 2003 at 04:48:53PM +0530, Dwarkanath wrote: > Hello Group, > > I wish to know, whether Beowulf supports MAC OS X. If yes, can i know the > details/pre-requisites/steps to accomplish the same. > > Thanking you in advance, > > Dwarkanath MB If you have machines which run OS X, then you could make a beowulf out of them. You just have to connect the machines together into a network, and then install the software you want to run (MPI/PVM, etc). Connecting the machines together should not be a problem - installing the software is where the bulk of your work is. You will need to compile MPI, PVM, or whatever software you want to run. There seems to be a short guide here: http://www.stat.ucla.edu/computing/clusters/deployment.php And you could also check out www.bioteam.net for a look at the Apple clusters they've put together, and the bioinformatics software they have available for Mac OS X. This is simplifying things, and assumes you just have a few spare Macs you want to use! If you're looking for a cluster to accomplish a specific task, obviously you need to think about your application, money you have to spare, etc. Kp -- Karl Podesta Dublin City University, Ireland _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From natorro at fisica.unam.mx Mon Jun 16 21:06:42 2003 From: natorro at fisica.unam.mx (Carlos Lopez Nataren) Date: 16 Jun 2003 20:06:42 -0500 Subject: SSH or RSH Message-ID: <1055812002.1628.6.camel@natorro.fisica.unam.mx> Hello, I was wondering if anybody has made a test on whether ssh can make a difference over rsh when using mpi (LAM/MPICH) and it is used for the slave nodes??? and one more, the same as before but now using NFS over IPSEC, can anyone tell??? Thanks a lot in advance. natorro -- Carlos Lopez Nataren Instituto de Fisica, UNAM _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From jhearns at freesolutions.net Tue Jun 17 04:08:22 2003 From: jhearns at freesolutions.net (John Hearns) Date: Tue, 17 Jun 2003 09:08:22 +0100 Subject: ideal motherboard for diskless beowulf References: Message-ID: <005b01c334a7$a1ef4060$8461cdc2@DREAD> ----- Original Message ----- From: "Robert G. Brown" To: "kolokoutsas konstantinos" Cc: Sent: Monday, June 16, 2003 6:36 PM Subject: Re: ideal motherboard for diskless beowulf > > > > This beowulf will be dedicated to running one > > particular Monte Carlo particle acceleration code > > already running within RH7.2 and quite dependant on it > > in many ways, thus the RH7.2 criterium. > > I'm sorry, how is that? > > a) Can't you just recompile under 7.3 or for that matter 9? You could, yes. However, HEP applications are linked with big libraries - such as the GEANT codes and the CERNLIBS. What happens is that there is a certification process, which checks out that the codes run on a certain version of (say) Redhat. However, the current certified version at CERN is 7.3, no? > > > > The micro-atx form factor is of interest because "I > > was given the challenge..." of putting as many > > motherboards in one customized full tower box as > > possible. Dual/Quad CPU motherboards are not an > > option, while due to portability issues, racks are out > > of the question. Kostas, if you are within reach of London you are welcome to have a look at the mini-ITX boards I have. I would be happy to help put the CERN environment on them and see how they run your codes. You certainly could make an easily portable cluster with these boards. (I am a paricle physicist, so do understand some of this stuff!) If you have a budget, there are boards using the same CPU which fit into the slots for a hard disk drive. That would fit your tower rack configuration - just get a full tower case, ordinary motherboard for the 'head' CPU, and populate all the drive bays with CPUs. You then use a small Ethernet switch to connect them. Again, I would be happy to help and advise on the boards, and helping get the Monte Carlo codes up and running. John Hearns _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From jhearns at freesolutions.net Tue Jun 17 04:41:47 2003 From: jhearns at freesolutions.net (John Hearns) Date: Tue, 17 Jun 2003 09:41:47 +0100 Subject: ideal motherboard for diskless beowulf References: Message-ID: <00f001c334ac$4d20f290$8461cdc2@DREAD> > > > > The micro-atx form factor is of interest because "I > > was given the challenge..." of putting as many > > motherboards in one customized full tower box as > > possible. Dual/Quad CPU motherboards are not an > > option, while due to portability issues, racks are out > > of the question. Kostas, you don't tell us your background. Are you a student or research fellow in one of the UK universities maybe? Which experiment are you working on? As I say, I did my PhD on ALEPH and have worked at CERN. I played around with ideas of putting lots of CPUs in a tower case last year. The discussion is on the Bewoulf list I think - the problem with that the little CPU boards were more expensive. I think that this configuration would be instructive to build, and would have a lot of applications. The boards I'm thinking of fit in a drive bay, and use power from a standard disk drive connector. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From phil.neumiller at convergys.com Mon Jun 16 17:49:35 2003 From: phil.neumiller at convergys.com (phil.neumiller at convergys.com) Date: Mon, 16 Jun 2003 17:49:35 -0400 Subject: Beowulf in Enterprise Environments Message-ID: Hi, Does anybody on the list know of folks using Beowulf clusters in what might be called traditional IT or Enterprise environments? Why/Why not? What about in cryptographic acceleration environments? Why/Why not? Perhaps for stuff even as benign as SSL/TLS/IPsec terminations??? Thanks, Phil -- "NOTICE: The information contained in this electronic mail transmission is intended by Convergys Corporation for the use of the named individual or entity to which it is directed and may contain information that is privileged or otherwise confidential. If you have received this electronic mail transmission in error, please delete it from your system without copying or forwarding it, and notify the sender of the error by reply email or by telephone (collect), so that the sender's address records can be corrected." _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From lathama at yahoo.com Tue Jun 17 00:52:35 2003 From: lathama at yahoo.com (Andrew Latham) Date: Mon, 16 Jun 2003 21:52:35 -0700 (PDT) Subject: new to clusters Message-ID: <20030617045235.93517.qmail@web80512.mail.yahoo.com> I have recently put together(just hardware) two small clusters. What would be a good simple project for a novice like myself to break myself and my clusters in. they are small and cheap but great for learning. 9 - Allen Bradley DataMytes - P133 - 64mb - Compaq NC3121 3 - IBM PC 365 6589-13U - Dual P-Pro200 - 128mb - Compaq NC3121 is there a project that would be good to run to warm these boys up and teach me about managing a cluster. ===== Andrew Latham LathamA.com lathama at lathama.com lathama at yahoo.com LathamA(lay-th-ham-eh) _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From jhearns at freesolutions.net Tue Jun 17 06:51:24 2003 From: jhearns at freesolutions.net (John Hearns) Date: Tue, 17 Jun 2003 11:51:24 +0100 Subject: Beowulf in Enterprise Environments References: Message-ID: <002301c334be$6837e4f0$8461cdc2@DREAD> ----- Original Message ----- From: To: Sent: Monday, June 16, 2003 10:49 PM Subject: Beowulf in Enterprise Environments > Hi, > > Does anybody on the list know of folks using Beowulf clusters in what might > be called > traditional IT or Enterprise environments? Why/Why not? > > What about in cryptographic acceleration environments? Why/Why not? > Perhaps for > stuff even as benign as SSL/TLS/IPsec terminations??? Depends what you mean by Beowulf of course :-) If we expand the definition to include Clusters of Workstations, or batch farms, then yes. People like car manufacturers, aerospace companies running simulations. Render farms for movie production. I guess financial institutions running Monte Carlos also. The Beowulf list however will (quite rightly) set the definition tighter, to mean a parallell cluster using MPI or PVM, and commodity hardware. Then I admit not to have much knowledge of uses in the Enterprise. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From cjereme at ucla.edu Tue Jun 17 08:39:56 2003 From: cjereme at ucla.edu (cjereme at ucla.edu) Date: Tue, 17 Jun 2003 4:39:56 PST Subject: rocks nacpi Message-ID: <200306171239.h5HCduH16662@webmail.my.ucla.edu> An embedded and charset-unspecified text was scrubbed... Name: not available URL: From rgb at phy.duke.edu Tue Jun 17 09:34:15 2003 From: rgb at phy.duke.edu (Robert G. Brown) Date: Tue, 17 Jun 2003 09:34:15 -0400 (EDT) Subject: SSH or RSH In-Reply-To: <1055812002.1628.6.camel@natorro.fisica.unam.mx> Message-ID: On 16 Jun 2003, Carlos Lopez Nataren wrote: > Hello, I was wondering if anybody has made a test on whether ssh can > make a difference over rsh when using mpi (LAM/MPICH) and it is used for > the slave nodes??? Make a difference of what sort? Both rsh and ssh are in the range of negligible overhead for most sensible job distribution schema. They are invoked once per remote process for the sole purpose of distributing the remote processes. AFAIK, they aren't used as any sort of meaningful (long term) communications pipeline by any of major parallel packages, which are designed to provide their own, highly efficient communications and job control. You can look back at the list archives -- I've benchmarked ssh and rsh several times over the last few years. I no longer can do rsh -- it isn't even installed on dulug systems by default, although I am yumming to see if an rsh rpm is still out there. However, over a rather noisy >>wireless<< connection from my laptop the time required to establish an ssh connection is 0.4 seconds, on average. The ssh-derived overhead in the time required to spawn 100 remote jobs would thus be less than a minute absolute; even allowing rsh to be "infinitely fast" by comparison I'd say that this is negligible. It is a bit more (marginally) expensive when it comes to scp. When copying files, its bidirectional encryption algorithm significantly adds time relative to e.g. rsh or a raw socket pipe, and is also significantly more CPU intensive. I honestly don't know if any of the MPI's use rcp to copy a binary image to the remote host to execute it -- pvm does not (relying on the binary already being available on the nodes on a predetermined path). For a large binary, this might not be negligible. Overall, I'd say that if you are running something where the rsh/ssh difference matters, you should be looking at Scyld or bproc anyway as they reduce this sort of overhead to the theoretical minimum, at the cost of providing virtually no node security (so you really have to run your cluster inside a head node/firewall). In all other cases, I'd recommend using ssh. It is time for rsh to just go away. ssh has lots of advantages besides just security, and as network and CPU speeds continue to increase the marginal cost of using it (which was negligible years ago) will continue to decrease. > and one more, the same as before but now using NFS over IPSEC, can > anyone tell??? No clue here, sorry. rgb -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb at phy.duke.edu _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From shewa at inel.gov Tue Jun 17 11:03:10 2003 From: shewa at inel.gov (Andrew Shewmaker) Date: Tue, 17 Jun 2003 09:03:10 -0600 Subject: Two CS undergraduates trying to build a prototype beowulf need help with bproc In-Reply-To: <20030614165721.DA2B.JAPA@mhu.eti.br> References: <20030614165721.DA2B.JAPA@mhu.eti.br> Message-ID: <3EEF2DAE.2080808@inel.gov> Marcos Hiroshi Umino wrote: > Hello friends. > > We tried searching the mailing list archive, but the link on the beowulf > site was off, so we decided to try our luck and consult the oracles. :) Try http://marc.theaimsgroup.com/?l=beowulf&r=1&w=2 > We installed redhat 8.0 in two machines, named master and slave1. We > setup nfs and rsh/rexec/rlogin and everything is working fine at kernel > level. We can rsh from/to slave1/master without any password prompts, > and pam looks ok too. > > We installed the 3.2.4 bproc/beowulf kernel patch in kernel 2.4.20, > recompiled it and as it seems, everything looks fine. > > We sucessfully modprobed vmadump and bproc, configured our master's > /etc/beowulf/config and all looked fine to go. > > However, when we tried running bpmaster (as root) it started, then > segfaulted. You know, bproc has a mailing list too. http://bproc.sf.net I remember running into a problem with bpmaster and my network interfaces. Do you have two interfaces? One for the public network and one for the beowulf? If your master node has only one NIC and you already configured the interface, then I think bpmaster will also try to initialize a network interface and you will have trouble. > Unfortunately, our guidebook is a brazilian translation, and I must > admit the translator wasn't very good at it. > > We are trying to search for complementary textbooks (we are students, so > books sugestions are great, but unfortunately are out of question). We > are a bit lost because the book we're following seems to use an outdated > beowulf version, because bproc docs seems to point toward no need for > rsh/rexec. Yes. With bproc you have no need of rsh/rexec. Although you can install bproc on a cluster with full Linux installs on every node, bproc based clusters usually only have a full install on the master. The slave nodes boot up a minimal kernel from the master. Andrew -- Andrew Shewmaker, Associate Engineer Phone: 1-208-526-1276 Idaho National Eng. and Environmental Lab. P.0. Box 1625, M.S. 3605 Idaho Falls, Idaho 83415-3605 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rgb at phy.duke.edu Tue Jun 17 11:06:42 2003 From: rgb at phy.duke.edu (Robert G. Brown) Date: Tue, 17 Jun 2003 11:06:42 -0400 (EDT) Subject: new to clusters In-Reply-To: <20030617045235.93517.qmail@web80512.mail.yahoo.com> Message-ID: On Mon, 16 Jun 2003, Andrew Latham wrote: > I have recently put together(just hardware) two small clusters. What would be a > good simple project for a novice like myself to break myself and my clusters > in. they are small and cheap but great for learning. > > 9 - Allen Bradley DataMytes - P133 - 64mb - Compaq NC3121 > 3 - IBM PC 365 6589-13U - Dual P-Pro200 - 128mb - Compaq NC3121 > > is there a project that would be good to run to warm these boys up and teach me > about managing a cluster. Check out e.g. Forrest Hoffman's "Extreme Linux" Column in Linux Magazine, www.linux-mag.com. Forrest has been running a series of tutorial articles on MPI (and in earlier issues, PVM). The June LM issue had a number of articles on cluster infrastructure and management. http://www.phy.duke.edu/brahma has links to lots of beowulf/cluster websites and other resources to help facilitate learning as well. Both PVM and MPI come with packages of "examples" of parallel programs, some of which are quite fun as demos (e.g. mandelbrot set toys, pvmpov) and others of which demonstrate at least some of the various parallel programming paradigms. Finally, there are a number of excellent books out there with tutorial sections on cluster programming. MIT press has a series of books on both MPI and PVM and a companion volume on beowulfs that is probably "the" beowulf book, again with numerous examples. There are a number of books on parallel algorithms: Amalsi and Gottlieb, Foster,... (I can't remember if I have a list linked to brahma or not -- I should probably put one there). That ought to be enough to get you going through the toy cluster stage to where you can be teaching and learning with a good appreciation for the basic toolset. rgb -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb at phy.duke.edu _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rgb at phy.duke.edu Tue Jun 17 11:17:24 2003 From: rgb at phy.duke.edu (Robert G. Brown) Date: Tue, 17 Jun 2003 11:17:24 -0400 (EDT) Subject: Beowulf in Enterprise Environments In-Reply-To: <002301c334be$6837e4f0$8461cdc2@DREAD> Message-ID: On Tue, 17 Jun 2003, John Hearns wrote: > > ----- Original Message ----- > From: > To: > Sent: Monday, June 16, 2003 10:49 PM > Subject: Beowulf in Enterprise Environments > > > > Hi, > > > > Does anybody on the list know of folks using Beowulf clusters in what > might > > be called > > traditional IT or Enterprise environments? Why/Why not? > > > > What about in cryptographic acceleration environments? Why/Why not? As Johns notes, "Beowulf" clusters per se, possibly not. Generalized clusters or LANs, they've been doing this longer than the term "beowulf" has been around -- really as long as COMPUTERS have been around. RC5 is cryptography spread out across the net in what is now a virtual architype of a "grid" computation. crack (the venerable Unix password cracking program) can be run embarrassingly parallel over a network and in fact has been programmed to do so on request for a rather long time now. The NSA has probably been doing this from back when the "cluster" they were using was a cluster of mainframes or a cluster of supercomputers. Indeed, the very first decryption "computers" used in WWII pretty much WERE clusters. ...etc. However, in the enterprise one has to carefully differentiate between "High Availability" (HA) cluster applications -- a distributed webserver, google, distributed databases -- and "High Performance Computing" (HPC) applications. HPC is where beowulfs and compute clusters live and is the primary topic of interest on this list. HA is where enterprise server farms of all flavors live, and is NOT generally discussed a lot on this list (although there are other lists where it is). "Extreme Linux" (once upon a time) referred to the union of HA and HPC and perhaps a bit of other linux automagic to makes it so that (as Sun used to so delicately and cleverly put it) "the network IS the computer". Corporations that need HPC, of course, do clusters all the time -- simulations, finite element analysis in engineering, etc. However, most of what one calls "enterprise" computing is likely to be a component of HA, or at best in a middle ground. The enterprise is a lot more about reliable delivery and load control and balancing than it is about screaming aggregate FLOPS. rgb -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb at phy.duke.edu _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From way2saurav at rediffmail.com Tue Jun 17 06:32:17 2003 From: way2saurav at rediffmail.com (Saurav Gohain) Date: 17 Jun 2003 10:32:17 -0000 Subject: what is a flop Message-ID: <20030617103217.2576.qmail@webmail28.rediffmail.com> Dear Sir/Friends I am new to cluster and hence asking this question ( may be a silly one for u all). In super computers and cluster, I find Peak Performance rated as TFLOPs, GFLOP's etc. Now, the question is what is a flop. From a beowulf pdf tutorial, i came to know that it is calculated as Total Peak Performance= No of Nodes * No of CPU's * Floating point operations per second * No of Cycles per second. Although, the first two and the last is easily derivable, what about the third one. I checked the specs of Opetron and P4 but there's isn't any mention about the Flops. How will i derive then ? In a comparison graph in Intel's website, I found a term SPEC*fp2000 ratings. Now this ratings are like 2.4 , 3.4 etc. Is it the number that I should input in the above formula to calculate the total peak performance in flops about a cluster. Kindly let me know the facts... Regards, Saurav -------------- next part -------------- An HTML attachment was scrubbed... URL: From jhearns at freesolutions.net Tue Jun 17 12:47:58 2003 From: jhearns at freesolutions.net (John Hearns) Date: Tue, 17 Jun 2003 17:47:58 +0100 Subject: ideal motherboard for diskless beowulf References: <005b01c334a7$a1ef4060$8461cdc2@DREAD> <3EEF6E4C.3698AB62@starpower.net> Message-ID: <011c01c334f0$384e5940$8461cdc2@DREAD> > [I'm a different responder..] > That idea has intrigued me for a while... I've been able to find a > number of 5.25 and even 3.5" form factor boards but no enclosures... > (well, only one but that appeared to be quite a bit too long to fit in a > standard bay...) > > The only integrated drive form factor device that I've been able to find > that is suitable to be a slave node is the one from TotalImpact. > Unfortunately that device has become a bit outdated and is VERY > expensive... (they also have an equally interesting/outdated/overpriced > quad G4 PCI board...) Repeated sessions with google has returned no > alternatives... > > Do you have any specific products in mind? The boards I looked at were from Nexcom. Their EBC series, for instance: http://www.nexcom.com/0330/nexweb/weben/ObjView.aspx?ObjID=Prod*10000083 I talked with the UK sales people, who put me onto a UK distributor. Sadly the price was about three times a mini-ITX board, so I couldn't afford them. Looking at their website now, there are 3.5" divices. Wow... http://www.nexcom.com/0330/nexweb/weben/ObjView.aspx?ObjID=Prod*10000088 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From roberto.ammendola at roma2.infn.it Tue Jun 17 14:34:48 2003 From: roberto.ammendola at roma2.infn.it (Roberto Ammendola) Date: Tue, 17 Jun 2003 20:34:48 +0200 Subject: what is a flop References: <20030617103217.2576.qmail@webmail28.rediffmail.com> Message-ID: <3EEF5F48.5020505@roma2.infn.it> Saurav Gohain wrote: > > From a beowulf pdf tutorial, i came to know that it is calculated as > Total Peak Performance= No of Nodes * No of CPU's * Floating point > operations per second * No of Cycles per second. > I would say: Total Peak Performance= No of Nodes * No of CPU's per Node * Floating point operations per clock cycle * No of Cycles per second. The "Floating point operations per clock cycle" depends on the processor, obviously, and on which instructions you use in your code. For example in a processor with the SSE instruction set you can perform 4 operations (on 32 bit register each) per clock cycle. One processor (Xeon or P4) running at 2.0 GHz can reach 8 GigaFlops. But this is only a peak performance. The real value is up to the cleverness of the programmer ;) cheers roberto _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From maurice at harddata.com Tue Jun 17 16:01:32 2003 From: maurice at harddata.com (Maurice Hilarius) Date: Tue, 17 Jun 2003 14:01:32 -0600 Subject: what is a flop (Roberto Ammendola) In-Reply-To: <200306171901.h5HJ1FU07223@NewBlue.Scyld.com> Message-ID: <5.1.1.6.2.20030617135556.03e7b9a0@mail.harddata.com> With regards to your message at 01:01 PM 6/17/03, beowulf-request at scyld.com. Where you stated: > Date: Tue, 17 Jun 2003 20:34:48 +0200 >From: Roberto Ammendola >Subject: Re: what is a flop > >Saurav Gohain wrote: > > > > > From a beowulf pdf tutorial, i came to know that it is calculated as > > Total Peak Performance= No of Nodes * No of CPU's * Floating point > > operations per second * No of Cycles per second. > > >I would say: > >Total Peak Performance= No of Nodes * No of CPU's per Node * Floating >point operations per clock cycle * No of Cycles per second. > >The "Floating point operations per clock cycle" depends on the >processor, obviously, and on which instructions you use in your code. >For example in a processor with the SSE instruction set you can perform >4 operations (on 32 bit register each) per clock cycle. One processor >(Xeon or P4) running at 2.0 GHz can reach 8 GigaFlops. But this is only >a peak performance. The real value is up to the cleverness of the >programmer ;) > >cheers >roberto And I would say dual CPU boards do not sale at a factor of 2:1 over singles. This is mainly ruled by 3 things: 1) efficiency of chipset for SMP operations, access to memory, buses, etc.. 2) efficiency of compiler at SMP/multithreaded code generation 3) efficiency of code design to complement the above factors. As a general ( really general as it changes a lot with code and compilers) the rule I know : Dual P3 ( VIA chipset): 1.5 : 1 Dual XEON P4 ( Intel 7501 chipset): 1.3 : 1 Dual AthlonMP ( AMD 760MPX chipset) 1.4 : 1 Dual Opteron (AMD 8xxx Chipset) 1.8 : 1 Dual Alpha (DEC Tsunami chipset) 1.8 : 1 With our best regards, Maurice W. Hilarius Telephone: 01-780-456-9771 Hard Data Ltd. FAX: 01-780-456-9772 11060 - 166 Avenue mailto:maurice at harddata.com Edmonton, AB, Canada http://www.harddata.com/ T5X 1Y3 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From M.Arndt at science-computing.de Tue Jun 17 12:21:27 2003 From: M.Arndt at science-computing.de (Michael Arndt) Date: Tue, 17 Jun 2003 18:21:27 +0200 Subject: Beowulf in Enterprise Environments In-Reply-To: ; from phil.neumiller@convergys.com on Mon, Jun 16, 2003 at 05:49:35PM -0400 References: Message-ID: <20030617182127.H21711@blnsrv1.science-computing.de> -you mix 3 questions ;-) we do a linux cluster concept ( like John before i would more define it as a "clustered peer to NOW" for complete german car industries and Chip development companies No Beowulf (in strict sense) there, reason being that main requirements favorise and priorize other strategies simple one: you have a application with around 100 000 $ license costs per user and the vendor guarantees function only for a certain unmodified ! distro ... there are other and more important nontechnical issues ... On the other hand: I assume there competent american companies able to do exactly the same on a beowulf base / concept when other needs and conditions are not the main issue ... If you take losse Beowulf == Linux Cluster then: Clusters are acepted and broad spread for productive and very important company issues ... e.g every german car manufacturer now does Crash Simulation on Linux > > Does anybody on the list know of folks using Beowulf clusters in what might > be called > traditional IT or Enterprise environments? Why/Why not? > > What about in cryptographic acceleration environments? Why/Why not? > Perhaps for > stuff even as benign as SSL/TLS/IPsec terminations??? there is a hardware company in germany called Clusterlabs ( active also in US) that does exactly this: Blades with Hardware SSL Cards for different uses i am sure there are others also .. Micha _________________________creating IT solutions Michael Arndt Bereichsleiter IT-Services Berlin phone +49(0)30 72 62 38-50 fax +49(0)30 72 62 38-59 m.arndt at science-computing.de science + computing ag Ehrenbergstrasse 19 D-10245 Berlin, Germany www.science-computing.de _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From phil.neumiller at convergys.com Tue Jun 17 16:23:56 2003 From: phil.neumiller at convergys.com (phil.neumiller at convergys.com) Date: Tue, 17 Jun 2003 16:23:56 -0400 Subject: Enterprise Beowulf and Beowulf on a Chip Message-ID: In my own search for finding out whether Beowulf makes sense for enterprise computing (specifically on-line transaction processing (OLTP)), I found a paper at Compaq research: http://research.compaq.com/wrl/projects/Database/ specifically the paper http://research.compaq.com/wrl/projects/Database/isca00.pdf This work emphasizes "large memory stall times" as being a primary culprit limiting OLTP performance. The solution promoted by Compaq researchers in the paper is chip multiprocessing (CMP). This makes me wonder if OLTP can benefit from simply lots of processors (with fast interconnect) to utilize more L1, L2 cache simultaneously (a might bit larger than a chip I might add!!)... Their Piranha system (a research prototype) integrates eight Alpha processor cores on a chip with a two level cache with cache coherence architecture (sort of a micro-sub-cluster). If this approach makes sense for OLTP doesn't a Beowulf make sense for OLTP work now? Also... If Beowulf makes sense on the macro level does it make sense in the micro-level or perhaps in the fractal sense of a self similar architecture (exploiting even more hierarchy)? -- "NOTICE: The information contained in this electronic mail transmission is intended by Convergys Corporation for the use of the named individual or entity to which it is directed and may contain information that is privileged or otherwise confidential. If you have received this electronic mail transmission in error, please delete it from your system without copying or forwarding it, and notify the sender of the error by reply email or by telephone (collect), so that the sender's address records can be corrected." _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From Daniel.Kidger at quadrics.com Tue Jun 17 12:28:54 2003 From: Daniel.Kidger at quadrics.com (Daniel Kidger) Date: Tue, 17 Jun 2003 17:28:54 +0100 Subject: what is a flop Message-ID: <010C86D15E4D1247B9A5DD312B7F5AA78DDF87@stegosaurus.bristol.quadrics.com> Dear Saurav, Actually the third term should be 'Floating point operations per cycle' Usualy further refined to mean specifically 64-bit floats ('double-precision') In the past most processor took many cycles to do each floating point operation. Now with pipelining most processors can issue at least one floating point operation every machine cycle (note though that very few can issue divides every cycle) The P4 can normally do one flop per cycle - but it can do two (using its SSE2 unit) but only if both are multiplies (or both adds) *and* both pairs of operands are contiguous in memory. Some architectures have two seperate floating point units: one just adds, the other multiplies (can add too) and hence also have a '2' here Some architectures most notably the vector machines have 16 or more floating point units. Many architectures have fused mutiply-add units (Alpha, Itanium, Mips, Power3, etc.) They can add the result of a multiply directly to another register. Thus these also have '2' flops per cycle. Some archtectures have 2 (or more) 'muladd' units. Hence iirc Power3/4 and Itanium2 can yield 4 flops per cycle. Yours, Daniel. -------------------------------------------------------------- Dr. Dan Kidger, Quadrics Ltd. daniel.kidger at quadrics.com One Bridewell St., Bristol, BS1 2AA, UK 0117 915 5505 ----------------------- www.quadrics.com -------------------- -----Original Message----- From: Saurav Gohain [mailto:way2saurav at rediffmail.com] Sent: 17 June 2003 11:32 To: beowulf at beowulf.org Subject: what is a flop Dear Sir/Friends I am new to cluster and hence asking this question ( may be a silly one for u all). In super computers and cluster, I find Peak Performance rated as TFLOPs, GFLOP's etc. Now, the question is what is a flop. >From a beowulf pdf tutorial, i came to know that it is calculated as Total Peak Performance= No of Nodes * No of CPU's * Floating point operations per second * No of Cycles per second. Although, the first two and the last is easily derivable, what about the third one. I checked the specs of Opetron and P4 but there's isn't any mention about the Flops. How will i derive then ? In a comparison graph in Intel's website, I found a term SPEC*fp2000 ratings. Now this ratings are like 2.4 , 3.4 etc. Is it the number that I should input in the above formula to calculate the total peak performance in flops about a cluster. Kindly let me know the facts... Regards, Saurav _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From xyzzy at speakeasy.org Tue Jun 17 16:25:05 2003 From: xyzzy at speakeasy.org (Trent Piepho) Date: Tue, 17 Jun 2003 13:25:05 -0700 (PDT) Subject: Rack or Cabinet? Message-ID: We're planning on buying a few rackmount systems, probably not so many that they take more than one rack, and I'm trying to decide between an open rack or a cabinet to house the systems+ups+switch+etc in. I don't have much experience with rackmount systems, so I'm wondering if anyone here has any experiences to share. The noise of the computer room is annoying the people near it, so I'd like to get a cabinet if they're quieter, but I'm concerned about the cooling issues. Does the enclosed space cause cabinets to overheat? I see some come with a few fans on the top, but is that really enough? We have a cooled computer room, but no fancy underfloor AC ducts. >From what I've gathered about the pro/cons of the two. Rack pros: cheaper, usually < $500 open design probably results in better cooling cons: noisy doesn't look as nice, with exposed rat's nests of wires, etc. Cabinet pros: Looks nice, with plexyglass doors and hidden wires, making non-techie types more impressed with how their computer dollars were spent dampens sound cons: expensive, typical > $1500 lack of airflow may result in poor cooling _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From James.P.Lux at jpl.nasa.gov Tue Jun 17 16:41:47 2003 From: James.P.Lux at jpl.nasa.gov (Jim Lux) Date: Tue, 17 Jun 2003 13:41:47 -0700 Subject: what is a flop References: <20030617103217.2576.qmail@webmail28.rediffmail.com> Message-ID: <001701c33510$df4e37e0$02a8a8c0@office1> FLOPS and the related MIPS are attempts to quantify the amount of computational work being done. FLOPS = Floating Point Operations per second, MIPS = Million Instructions per Second. The real problem with such measures is that they work fine for a classic, non-pipelined, one instruction per clock kind of processor. However, if a multiply takes 100 times as long as an add, how do you count up the ops? How do you count speed on a processor where the number of instructions per second is highly dependent on the sequence of instructions (virtually any modern pipelined processor) or on cache hits/misses. Even the venerable Z80 and 8080 microprocessors have variable numbers of clocks required for instructions (conditional jumps taking one of two different lengths of time). Only on processors designed for "lockstep" processing (like a lot of DSP cores) is the instruction rate reasonably constant (and there, things like interrupts can screw up the timing... a real problem if your code has to service an i/o device every N clocks, regardless) The best you can do with such simple measures is look at the time required to execute some fairly well defined mathematical operation, with a fairly well defined number and kind of operations required.. A floating point FFT with no fancy tricks might be a good example. The number of multiplies and adds is well known, and independent of the data being transformed. FFT performance compared with some other algorithm can tell you something about the caching and pipeline performance. As many will tell you, the true measure of performance is how fast it goes with YOUR application... ----- Original Message ----- From: Saurav Gohain To: beowulf at beowulf.org Sent: Tuesday, June 17, 2003 3:32 AM Subject: what is a flop Dear Sir/Friends I am new to cluster and hence asking this question ( may be a silly one for u all). In super computers and cluster, I find Peak Performance rated as TFLOPs, GFLOP's etc. Now, the question is what is a flop. >From a beowulf pdf tutorial, i came to know that it is calculated as Total Peak Performance= No of Nodes * No of CPU's * Floating point operations per second * No of Cycles per second. Although, the first two and the last is easily derivable, what about the third one. I checked the specs of Opetron and P4 but there's isn't any mention about the Flops. How will i derive then ? In a comparison graph in Intel's website, I found a term SPEC*fp2000 ratings. Now this ratings are like 2.4 , 3.4 etc. Is it the number that I should input in the above formula to calculate the total peak performance in flops about a cluster. Kindly let me know the facts... Regards, Saurav _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rbw at ahpcrc.org Tue Jun 17 17:40:25 2003 From: rbw at ahpcrc.org (Richard Walsh) Date: Tue, 17 Jun 2003 16:40:25 -0500 Subject: what is a flop Message-ID: <200306172140.h5HLePP05878@mycroft.ahpcrc.org> Dan Kidger wrote: >The P4 can normally do one flop per cycle - but it can do two (using its >SSE2 unit) but only if both are multiplies (or both adds) *and* both pairs >of operands are contiguous in memory. > >Some architectures have two seperate floating point units: one just adds, >the other multiplies (can add too) and hence also have a '2' here > >Some architectures most notably the vector machines have 16 or more floating >point units. > >Many architectures have fused mutiply-add units (Alpha, Itanium, Mips, >Power3, etc.) They can add the result of a multiply directly to another >register. Thus these also have '2' flops per cycle. > >Some archtectures have 2 (or more) 'muladd' units. Hence iirc Power3/4 and >Itanium2 can yield 4 flops per cycle. A glass ;-) menagerie ... if you will. Another point to be made of the this, is that peak performance per processor is slippery to define ... where does the boundary of the processor actually rest? At the edge of the chip, the module, ... a safe definition is perhaps a processing core consisting of a pairing of independent FPUs and dedicated registers. Cray quotes 12.8 Gflops for their X1 MSP ... but that is for a multi-chip module ... which includes 2 FPUs (mul and add) x 2 vector pipes x 4 vector cores ... this is gives you Daniel's 16 FPUs. As an exercise compute the vector clock of the X1, the peak performance of a Power 4 core running at 1.3 GHz? How about the whole Power 4 chip? ;-) Finally, the relevance of the peak performance ends right about ... here. Cheers, rbw _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From waitt at saic.com Tue Jun 17 19:30:37 2003 From: waitt at saic.com (Tim Wait) Date: Tue, 17 Jun 2003 19:30:37 -0400 Subject: what is a flop References: <200306172140.h5HLePP05878@mycroft.ahpcrc.org> Message-ID: <3EEFA49D.3070709@saic.com> > Finally, the relevance of the peak performance ends right about ... here. Hah! Very well put! May I add another? /home/me/bin/FLOPS_benchmark > /dev/null _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From joelja at darkwing.uoregon.edu Tue Jun 17 20:56:07 2003 From: joelja at darkwing.uoregon.edu (Joel Jaeggli) Date: Tue, 17 Jun 2003 17:56:07 -0700 (PDT) Subject: Rack or Cabinet? In-Reply-To: Message-ID: cabinets are really ideal when you have raised floors and forced-air cooling, in the absence of that, an open-rack will probably run cooler... joelja On Tue, 17 Jun 2003, Trent Piepho wrote: > We're planning on buying a few rackmount systems, probably not so many > that they take more than one rack, and I'm trying to decide between an open > rack or a cabinet to house the systems+ups+switch+etc in. > > I don't have much experience with rackmount systems, so I'm wondering if > anyone here has any experiences to share. The noise of the computer room is > annoying the people near it, so I'd like to get a cabinet if they're quieter, > but I'm concerned about the cooling issues. Does the enclosed space cause > cabinets to overheat? I see some come with a few fans on the top, but is that > really enough? We have a cooled computer room, but no fancy underfloor AC > ducts. > > >From what I've gathered about the pro/cons of the two. > > Rack > pros: > cheaper, usually < $500 > open design probably results in better cooling > > cons: > noisy > doesn't look as nice, with exposed rat's nests of wires, etc. > > Cabinet > pros: > Looks nice, with plexyglass doors and hidden wires, making non-techie types > more impressed with how their computer dollars were spent > dampens sound > cons: > expensive, typical > $1500 > lack of airflow may result in poor cooling > > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org > To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf > -- -------------------------------------------------------------------------- Joel Jaeggli Academic User Services joelja at darkwing.uoregon.edu -- PGP Key Fingerprint: 1DE9 8FCA 51FB 4195 B42A 9C32 A30D 121E -- In Dr. Johnson's famous dictionary patriotism is defined as the last resort of the scoundrel. With all due respect to an enlightened but inferior lexicographer I beg to submit that it is the first. -- Ambrose Bierce, "The Devil's Dictionary" _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From laurenceliew at yahoo.com.sg Tue Jun 17 21:44:17 2003 From: laurenceliew at yahoo.com.sg (=?iso-8859-1?q?Laurence?=) Date: Wed, 18 Jun 2003 09:44:17 +0800 (CST) Subject: Enterprise Beowulf and Beowulf on a Chip In-Reply-To: Message-ID: <20030618014417.99050.qmail@web11904.mail.yahoo.com> Hi, One of the accepted definition of a beowulf is the use of low-cost COMMODITY processor... mostly INTEL and AMD cpus. If and when INTEL/AMD have low-cost multicpu on a single chip.. and is available in your DIY shop next door.... yes.. I guess it would make for a faster beowulf... As for enterprise beowulf and OLTP if this is what you are interested in.... OLTP usually means a database... so unless the database is parallel eg like Oracle 9i RAC, that would still be your bottleneck... you could queue many requests in a farm of request/compute nodes... but if you have a large DB.. it may not make sense... Oracle 9i RAC runs on a cluster... but this is not the usual definition of a beowulf however... you may want to explore PVFS + MySQL or Oracle... using PVFS which gives you parallel IO in a beowulf cluster... you may be able to speed up the IO.. it would help maybe not so much in OLTP but I believe in data-mining type, an analogy would be the genomics/blast people... who blast there sequences against a flat file database... and they do experiment with such parallel IO techniques... when you think enterprise.. those that would use beowulf would be those with intensive computation needs... Volvo, Audi, Nissan for car simulation, Philips, Toshiba for chip design, multi-media design houses etc etc.... and others... for "normal" enterprise... a beowulf "MAY" not be very useful... until the day when that complex spreadsheets can be computed not on your computer.. but sent to a Spreadsheet farm in the data centre behind you..... Hope this helps. Cheers! laurence --- phil.neumiller at convergys.com wrote: > In my own search for finding out whether Beowulf > makes sense for > enterprise computing (specifically on-line > transaction processing > (OLTP)), I found a paper at Compaq research: > > http://research.compaq.com/wrl/projects/Database/ > > specifically the paper > > http://research.compaq.com/wrl/projects/Database/isca00.pdf > > This work emphasizes "large memory stall times" as > being > a primary culprit limiting OLTP performance. The > solution > promoted by Compaq researchers in the paper is chip > multiprocessing (CMP). This makes me wonder if OLTP > can benefit from simply lots of processors (with > fast interconnect) > to utilize more L1, L2 cache simultaneously (a might > bit larger > than a chip I might add!!)... > > Their Piranha system (a research prototype) > integrates eight > Alpha processor cores on a chip with a two level > cache with > cache coherence architecture (sort of a > micro-sub-cluster). > > If this approach makes sense for OLTP doesn't a > Beowulf > make sense for OLTP work now? > > Also... > > If Beowulf makes sense on the macro level does it > make > sense in the micro-level or perhaps in the fractal > sense of > a self similar architecture (exploiting even more > hierarchy)? > > > -- > "NOTICE: The information contained in this > electronic mail transmission is > intended by Convergys Corporation for the use of the > named individual or > entity to which it is directed and may contain > information that is > privileged or otherwise confidential. If you have > received this electronic > mail transmission in error, please delete it from > your system without > copying or forwarding it, and notify the sender of > the error by reply email > or by telephone (collect), so that the sender's > address records can be > corrected." > > > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org > To change your subscription (digest mode or > unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf ===== --- Cheers! Laurence __________________________________________________ Do You Yahoo!? Send free SMS from your PC! http://sg.sms.yahoo.com _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From franz.marini at mi.infn.it Wed Jun 18 04:53:17 2003 From: franz.marini at mi.infn.it (Franz Marini) Date: Wed, 18 Jun 2003 10:53:17 +0200 (CEST) Subject: SMP CPUs scaling factors (was "what is a flop") In-Reply-To: <5.1.1.6.2.20030617135556.03e7b9a0@mail.harddata.com> Message-ID: Hi, On Tue, 17 Jun 2003, Maurice Hilarius wrote: > And I would say dual CPU boards do not sale at a factor of 2:1 over singles. > ... > As a general ( really general as it changes a lot with code and compilers) > the rule I know : > > Dual P3 ( VIA chipset): 1.5 : 1 > Dual XEON P4 ( Intel 7501 chipset): 1.3 : 1 Hrm. Now, this is a figure I find hard to believe... This would mean that, for a "general" (and thus I think you mean "not-overly-optimized") parallel (threaded, I guess) code, you'd get only a 30% gain in a dual Xeon system over a single one... Don't know, it seems a somewhat conservative figure to me. > Dual AthlonMP ( AMD 760MPX chipset) 1.4 : 1 (The same considerations applied to the Xeon holds true here ;)) Does anyone have some real world application figures regarding the performance ratio between single and two-way (and maybe four-way) SMP systems based on the P4 Xeon processor ? I'm particularly interested in molecular dynamics code, like, e.g., Gromacs and CPMD. Have a nice day, Franz --------------------------------------------------------- Franz Marini Sys Admin and Software Analyst, Dept. of Physics, University of Milan, Italy. email : franz.marini at mi.infn.it --------------------------------------------------------- _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From Steve_L_Smith at dell.com Wed Jun 18 06:27:58 2003 From: Steve_L_Smith at dell.com (Steve_L_Smith at dell.com) Date: Wed, 18 Jun 2003 05:27:58 -0500 Subject: Rack or Cabinet Message-ID: <5D0C92E65C29D311A24D0090273C2168079973A6@BRKXMTCH02_b> Trent You should take the advice of the system vendor. At Dell (this is not an advert:-) we design our systems (as do all tier 1s) for mounting in enclosed racks (including side panels and doors). This is to ensure correct cooling. Running something like a Dell PowerEdge 1750 1U DP Xeon in an open cabinet is definitely not recommended. The systems are designed to have front to back cooling, allowing air to circulate randomly around the sides and tops upsets this airflow and will cause the heat within the server to rise, possibly leading to over-temperature situations. And if you do not fill a complete rack, you should also use blanking panels to maintain the airflow - also note that this also applies to e.g. a Myrinet switch - you should use blanking panels if you do not fully populate the switch chassis. Unfortunately noise is not something that we can do much about at the moment - putting hot processors in small spaces means we need big/fast (hence somewhat noisy) fans to push the air out of our box into your room! Hope this helps Steve ------------------------------------------------- Steve Smith HPC Business Manager Dell EMEA Dell Campus, Cain Road, Bracknell, RG12 1FA, UK Direct: +44 1344 372037 Switchboard: +44 1344 812000 Fax: +44 1344 372359 Mobile: +44 7802 594874 email: steve_l_smith at dell.com ------------------------------------------------------ --__--__-- Message: 2 Date: Tue, 17 Jun 2003 13:25:05 -0700 (PDT) From: Trent Piepho To: Beowulf Mailing list Subject: Rack or Cabinet? We're planning on buying a few rackmount systems, probably not so many that they take more than one rack, and I'm trying to decide between an open rack or a cabinet to house the systems+ups+switch+etc in. I don't have much experience with rackmount systems, so I'm wondering if anyone here has any experiences to share. The noise of the computer room is annoying the people near it, so I'd like to get a cabinet if they're quieter, but I'm concerned about the cooling issues. Does the enclosed space cause cabinets to overheat? I see some come with a few fans on the top, but is that really enough? We have a cooled computer room, but no fancy underfloor AC ducts. >From what I've gathered about the pro/cons of the two. Rack pros: cheaper, usually < $500 open design probably results in better cooling cons: noisy doesn't look as nice, with exposed rat's nests of wires, etc. Cabinet pros: Looks nice, with plexyglass doors and hidden wires, making non-techie types more impressed with how their computer dollars were spent dampens sound cons: expensive, typical > $1500 lack of airflow may result in poor cooling --__--__-- _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From alvin at Maggie.Linux-Consulting.com Wed Jun 18 06:53:47 2003 From: alvin at Maggie.Linux-Consulting.com (Alvin Oga) Date: Wed, 18 Jun 2003 03:53:47 -0700 (PDT) Subject: Rack or Cabinet In-Reply-To: <5D0C92E65C29D311A24D0090273C2168079973A6@BRKXMTCH02_b> Message-ID: hi ya steve On Wed, 18 Jun 2003 Steve_L_Smith at dell.com wrote: > Trent > > You should take the advice of the system vendor. At Dell (this is not an > advert:-) we design our systems (as do all tier 1s) for mounting in enclosed > racks (including side panels and doors). This is to ensure correct cooling. > Running something like a Dell PowerEdge 1750 1U DP Xeon in an open cabinet > is definitely not recommended. The systems are designed to have front to for the dual xeons or dual-p4s, what do your normal temperatures run at say using normal 25C or normal 50F computer room temps ??? > back cooling, allowing air to circulate randomly around the sides and tops > upsets this airflow and will cause the heat within the server to rise, > possibly leading to over-temperature situations. And if you do not fill a > complete rack, you should also use blanking panels to maintain the airflow - > also note that this also applies to e.g. a Myrinet switch - you should use > blanking panels if you do not fully populate the switch chassis. and if the air in the chassis is meant to go side to side ... the cabinet airflow should also be doing the same ... > Unfortunately noise is not something that we can do much about at the moment > - putting hot processors in small spaces means we need big/fast (hence > somewhat noisy) fans to push the air out of our box into your room! you can cut some noice by making bigger holes for fans instead of small holes ... holes as big as the diameter of the fan blades .. less interference from the air flow squeezing thru/pass the obstacle but than again, that's not possible/realistic for making 12" diameter holes if one is using 12" vent fans i have a p4-2.0G running at 28C ... am nicely cool and happy... in normal 25C room temp ... w/ normal fans (noisy) in a 1U mini-itx box.. -- am thinking the bios sensor must be whacky, but the air is in fact cool as is the heatsinks ... and its happily calculating away have fun alvin _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From Steve_L_Smith at dell.com Wed Jun 18 06:27:58 2003 From: Steve_L_Smith at dell.com (Steve_L_Smith at dell.com) Date: Wed, 18 Jun 2003 05:27:58 -0500 Subject: Rack or Cabinet Message-ID: <5D0C92E65C29D311A24D0090273C2168079973A6@BRKXMTCH02_b> Trent You should take the advice of the system vendor. At Dell (this is not an advert:-) we design our systems (as do all tier 1s) for mounting in enclosed racks (including side panels and doors). This is to ensure correct cooling. Running something like a Dell PowerEdge 1750 1U DP Xeon in an open cabinet is definitely not recommended. The systems are designed to have front to back cooling, allowing air to circulate randomly around the sides and tops upsets this airflow and will cause the heat within the server to rise, possibly leading to over-temperature situations. And if you do not fill a complete rack, you should also use blanking panels to maintain the airflow - also note that this also applies to e.g. a Myrinet switch - you should use blanking panels if you do not fully populate the switch chassis. Unfortunately noise is not something that we can do much about at the moment - putting hot processors in small spaces means we need big/fast (hence somewhat noisy) fans to push the air out of our box into your room! Hope this helps Steve ------------------------------------------------- Steve Smith HPC Business Manager Dell EMEA Dell Campus, Cain Road, Bracknell, RG12 1FA, UK Direct: +44 1344 372037 Switchboard: +44 1344 812000 Fax: +44 1344 372359 Mobile: +44 7802 594874 email: steve_l_smith at dell.com ------------------------------------------------------ --__--__-- Message: 2 Date: Tue, 17 Jun 2003 13:25:05 -0700 (PDT) From: Trent Piepho To: Beowulf Mailing list Subject: Rack or Cabinet? We're planning on buying a few rackmount systems, probably not so many that they take more than one rack, and I'm trying to decide between an open rack or a cabinet to house the systems+ups+switch+etc in. I don't have much experience with rackmount systems, so I'm wondering if anyone here has any experiences to share. The noise of the computer room is annoying the people near it, so I'd like to get a cabinet if they're quieter, but I'm concerned about the cooling issues. Does the enclosed space cause cabinets to overheat? I see some come with a few fans on the top, but is that really enough? We have a cooled computer room, but no fancy underfloor AC ducts. >From what I've gathered about the pro/cons of the two. Rack pros: cheaper, usually < $500 open design probably results in better cooling cons: noisy doesn't look as nice, with exposed rat's nests of wires, etc. Cabinet pros: Looks nice, with plexyglass doors and hidden wires, making non-techie types more impressed with how their computer dollars were spent dampens sound cons: expensive, typical > $1500 lack of airflow may result in poor cooling --__--__-- _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From alvin at Maggie.Linux-Consulting.com Wed Jun 18 06:53:47 2003 From: alvin at Maggie.Linux-Consulting.com (Alvin Oga) Date: Wed, 18 Jun 2003 03:53:47 -0700 (PDT) Subject: Rack or Cabinet In-Reply-To: <5D0C92E65C29D311A24D0090273C2168079973A6@BRKXMTCH02_b> Message-ID: hi ya steve On Wed, 18 Jun 2003 Steve_L_Smith at dell.com wrote: > Trent > > You should take the advice of the system vendor. At Dell (this is not an > advert:-) we design our systems (as do all tier 1s) for mounting in enclosed > racks (including side panels and doors). This is to ensure correct cooling. > Running something like a Dell PowerEdge 1750 1U DP Xeon in an open cabinet > is definitely not recommended. The systems are designed to have front to for the dual xeons or dual-p4s, what do your normal temperatures run at say using normal 25C or normal 50F computer room temps ??? > back cooling, allowing air to circulate randomly around the sides and tops > upsets this airflow and will cause the heat within the server to rise, > possibly leading to over-temperature situations. And if you do not fill a > complete rack, you should also use blanking panels to maintain the airflow - > also note that this also applies to e.g. a Myrinet switch - you should use > blanking panels if you do not fully populate the switch chassis. and if the air in the chassis is meant to go side to side ... the cabinet airflow should also be doing the same ... > Unfortunately noise is not something that we can do much about at the moment > - putting hot processors in small spaces means we need big/fast (hence > somewhat noisy) fans to push the air out of our box into your room! you can cut some noice by making bigger holes for fans instead of small holes ... holes as big as the diameter of the fan blades .. less interference from the air flow squeezing thru/pass the obstacle but than again, that's not possible/realistic for making 12" diameter holes if one is using 12" vent fans i have a p4-2.0G running at 28C ... am nicely cool and happy... in normal 25C room temp ... w/ normal fans (noisy) in a 1U mini-itx box.. -- am thinking the bios sensor must be whacky, but the air is in fact cool as is the heatsinks ... and its happily calculating away have fun alvin _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From k_kolokoutsas at yahoo.co.uk Wed Jun 18 06:09:37 2003 From: k_kolokoutsas at yahoo.co.uk (=?iso-8859-1?q?kolokoutsas=20konstantinos?=) Date: Wed, 18 Jun 2003 11:09:37 +0100 (BST) Subject: ideal motherboard for diskless beowulf In-Reply-To: <200306180855.h5I8tJU30389@NewBlue.Scyld.com> Message-ID: <20030618100937.9589.qmail@web9907.mail.yahoo.com> John, thanks for your comments. My background is in Stellar Astrophysics (Imperial), however the beowulf is for a friend postdoc doing relativistic shock particle acceleration at MPIfA. Not quite sure about the characteristics of her Monte Carlo code, so can't comment on library usage/dependencies etc. My interest in this particular exercise has more to do with hardware, and to identify what would be the best micro-atx Athlon/Athlon XP motherboard for a beowulf cluster within budget. Yes, the mini-ITX form factor is a quite interesting possibility. However, I have also come across the EBX form factor (5.25"). Such motherboards, primarily used in embedded systems are manufactured by www.ITOX.com, can take up to 1GHZ P3/Celeron CPUs, with various power supply configs and are prohibitely expensive(~$300 without CPU/memory), for the budget of this beowulf at least. Kostas _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From Steve_L_Smith at dell.com Wed Jun 18 08:34:29 2003 From: Steve_L_Smith at dell.com (Steve_L_Smith at dell.com) Date: Wed, 18 Jun 2003 07:34:29 -0500 Subject: Rack or Cabinet Message-ID: <5D0C92E65C29D311A24D0090273C2168079973A9@BRKXMTCH02_b> Hi Alvin Our systems are designed for a maximum inlet temperature of 35C, and a front to back rise of up to 5C - this is pretty extreme. Normally (good airflow through the system, not too many cables and a system that is not running at max - full of disks and memory and really working hard!) we'd expect a 2C rise, so your 28C in a 25C inlet room is about right. The other thing I didn't mention is that in a full rack (e.g. 42 x 2P Xeon systems) you would expect to see up to 11C rise in temperature from the bottom of the rack to the top - again extreme but worth bearing in mind when deploying full racks of hot boxes. Not sure I'd recommend people cutting big holes for the fans - unless it isn't a Dell of course:-) Cheers Steve -----Original Message----- From: Alvin Oga [mailto:alvin at Maggie.Linux-Consulting.com] Sent: 18 June 2003 11:54 To: Steve_L_Smith at exchange.dell.com Cc: beowulf at scyld.com; beowulf at beowulf.org Subject: Re:Rack or Cabinet hi ya steve On Wed, 18 Jun 2003 Steve_L_Smith at dell.com wrote: > Trent > > You should take the advice of the system vendor. At Dell (this is not an > advert:-) we design our systems (as do all tier 1s) for mounting in enclosed > racks (including side panels and doors). This is to ensure correct cooling. > Running something like a Dell PowerEdge 1750 1U DP Xeon in an open cabinet > is definitely not recommended. The systems are designed to have front to for the dual xeons or dual-p4s, what do your normal temperatures run at say using normal 25C or normal 50F computer room temps ??? > back cooling, allowing air to circulate randomly around the sides and tops > upsets this airflow and will cause the heat within the server to rise, > possibly leading to over-temperature situations. And if you do not fill a > complete rack, you should also use blanking panels to maintain the airflow - > also note that this also applies to e.g. a Myrinet switch - you should use > blanking panels if you do not fully populate the switch chassis. and if the air in the chassis is meant to go side to side ... the cabinet airflow should also be doing the same ... > Unfortunately noise is not something that we can do much about at the moment > - putting hot processors in small spaces means we need big/fast (hence > somewhat noisy) fans to push the air out of our box into your room! you can cut some noice by making bigger holes for fans instead of small holes ... holes as big as the diameter of the fan blades .. less interference from the air flow squeezing thru/pass the obstacle but than again, that's not possible/realistic for making 12" diameter holes if one is using 12" vent fans i have a p4-2.0G running at 28C ... am nicely cool and happy... in normal 25C room temp ... w/ normal fans (noisy) in a 1U mini-itx box.. -- am thinking the bios sensor must be whacky, but the air is in fact cool as is the heatsinks ... and its happily calculating away have fun alvin _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From Steve_L_Smith at dell.com Wed Jun 18 08:34:29 2003 From: Steve_L_Smith at dell.com (Steve_L_Smith at dell.com) Date: Wed, 18 Jun 2003 07:34:29 -0500 Subject: Rack or Cabinet Message-ID: <5D0C92E65C29D311A24D0090273C2168079973A9@BRKXMTCH02_b> Hi Alvin Our systems are designed for a maximum inlet temperature of 35C, and a front to back rise of up to 5C - this is pretty extreme. Normally (good airflow through the system, not too many cables and a system that is not running at max - full of disks and memory and really working hard!) we'd expect a 2C rise, so your 28C in a 25C inlet room is about right. The other thing I didn't mention is that in a full rack (e.g. 42 x 2P Xeon systems) you would expect to see up to 11C rise in temperature from the bottom of the rack to the top - again extreme but worth bearing in mind when deploying full racks of hot boxes. Not sure I'd recommend people cutting big holes for the fans - unless it isn't a Dell of course:-) Cheers Steve -----Original Message----- From: Alvin Oga [mailto:alvin at Maggie.Linux-Consulting.com] Sent: 18 June 2003 11:54 To: Steve_L_Smith at exchange.dell.com Cc: beowulf at scyld.com; beowulf at beowulf.org Subject: Re:Rack or Cabinet hi ya steve On Wed, 18 Jun 2003 Steve_L_Smith at dell.com wrote: > Trent > > You should take the advice of the system vendor. At Dell (this is not an > advert:-) we design our systems (as do all tier 1s) for mounting in enclosed > racks (including side panels and doors). This is to ensure correct cooling. > Running something like a Dell PowerEdge 1750 1U DP Xeon in an open cabinet > is definitely not recommended. The systems are designed to have front to for the dual xeons or dual-p4s, what do your normal temperatures run at say using normal 25C or normal 50F computer room temps ??? > back cooling, allowing air to circulate randomly around the sides and tops > upsets this airflow and will cause the heat within the server to rise, > possibly leading to over-temperature situations. And if you do not fill a > complete rack, you should also use blanking panels to maintain the airflow - > also note that this also applies to e.g. a Myrinet switch - you should use > blanking panels if you do not fully populate the switch chassis. and if the air in the chassis is meant to go side to side ... the cabinet airflow should also be doing the same ... > Unfortunately noise is not something that we can do much about at the moment > - putting hot processors in small spaces means we need big/fast (hence > somewhat noisy) fans to push the air out of our box into your room! you can cut some noice by making bigger holes for fans instead of small holes ... holes as big as the diameter of the fan blades .. less interference from the air flow squeezing thru/pass the obstacle but than again, that's not possible/realistic for making 12" diameter holes if one is using 12" vent fans i have a p4-2.0G running at 28C ... am nicely cool and happy... in normal 25C room temp ... w/ normal fans (noisy) in a 1U mini-itx box.. -- am thinking the bios sensor must be whacky, but the air is in fact cool as is the heatsinks ... and its happily calculating away have fun alvin _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From bogdan.costescu at iwr.uni-heidelberg.de Wed Jun 18 09:38:46 2003 From: bogdan.costescu at iwr.uni-heidelberg.de (Bogdan Costescu) Date: Wed, 18 Jun 2003 15:38:46 +0200 (CEST) Subject: SMP CPUs scaling factors (was "what is a flop") In-Reply-To: Message-ID: On Wed, 18 Jun 2003, Franz Marini wrote: > Does anyone have some real world application figures regarding the > performance ratio between single and two-way (and maybe four-way) SMP > systems based on the P4 Xeon processor ? I'm particularly interested in > molecular dynamics code, like, e.g., Gromacs and CPMD. If CHARMM figures are also valid: - when running 2 single-CPU jobs on a 2-way system, real (wall) time of each process is the same as if the process was running alone. - when running a parallel job with shmem/usysv the speed-up is close to 2 (1.98 or so). This is with "normal" MD. Using PME might make things different... -- Bogdan Costescu IWR - Interdisziplinaeres Zentrum fuer Wissenschaftliches Rechnen Universitaet Heidelberg, INF 368, D-69120 Heidelberg, GERMANY Telephone: +49 6221 54 8869, Telefax: +49 6221 54 8868 E-mail: Bogdan.Costescu at IWR.Uni-Heidelberg.De _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rgb at phy.duke.edu Wed Jun 18 11:20:03 2003 From: rgb at phy.duke.edu (Robert G. Brown) Date: Wed, 18 Jun 2003 11:20:03 -0400 (EDT) Subject: Rack or Cabinet? In-Reply-To: Message-ID: On Tue, 17 Jun 2003, Trent Piepho wrote: > Rack > pros: > cheaper, usually < $500 > open design probably results in better cooling > > cons: > noisy > doesn't look as nice, with exposed rat's nests of wires, etc. > > Cabinet > pros: > Looks nice, with plexyglass doors and hidden wires, making non-techie types > more impressed with how their computer dollars were spent > dampens sound > cons: > expensive, typical > $1500 > lack of airflow may result in poor cooling Yup, I think you've about got it. Except that I think that closed cabinets are often power-ventilated with directed AC so that cooling may not be that much of a problem, to be fair. To me, the more than $1000 difference in cost is pretty much everything. We keep all the racks in a room where sound doesn't matter and there are enough nodes that the impression factor doesn't matter much either. It's a workingperson's cluster vs a show cluster kind of thing...;-) rgb -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb at phy.duke.edu _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From lathama at yahoo.com Wed Jun 18 11:08:50 2003 From: lathama at yahoo.com (Andrew Latham) Date: Wed, 18 Jun 2003 08:08:50 -0700 (PDT) Subject: power supplies Message-ID: <20030618150850.70991.qmail@web80513.mail.yahoo.com> Has any one used or liked any AT/ATX external power supplies. My clusters are using minimal power. The power supplys are wasting energy converting(heat). I see the advantage to building a rackmount power converter and playing with independant ATX swithing. What are some ideas or comments about such a project. ===== Andrew Latham LathamA.com lathama at lathama.com lathama at yahoo.com LathamA(lay-th-ham-eh) _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From Daniel.Kidger at quadrics.com Wed Jun 18 13:54:10 2003 From: Daniel.Kidger at quadrics.com (Daniel Kidger) Date: Wed, 18 Jun 2003 18:54:10 +0100 Subject: what is a flop Message-ID: <010C86D15E4D1247B9A5DD312B7F5AA78DDF8E@stegosaurus.bristol.quadrics.com> -----Original Message----- >From: Mikhail Kuzminsky [mailto:kus at free.net] >Sent: None >To: roberto.ammendola at roma2.infn.it >Cc: beowulf at beowulf.org >Subject: Re: what is a flop > > >According to Roberto Ammendola >> The "Floating point operations per clock cycle" depends on the >> processor, obviously, and on which instructions you use in your code. >> For example in a processor with the SSE instruction set you can perform >> 4 operations (on 32 bit register each) per clock cycle. One processor >> (Xeon or P4) running at > Taking into account that throughput of FMUL and FADD units in >P4/Xeon is 2 cycles, i.e. FP result may be received on any 2nd sycle >only, the peak Performance of P4/2 Ghz must be 4 GFLOPS. IMHO You are both correct and also wrong at the same time. The P4/Xeon *can* do 8 Gflop/s but only in 'single-precision'. It can do this by issueing just one SSE2 instruction but that instruction does 4 muls (or adds) on a 128-bit load. (as 4*4byte consecutive floats). compare with doing 2 muls (or adds) on 2*16 consecutive 8byte 'doubles'). A Flop is usually defines as being on a number of at least 64bits. iirc the P4/Xeon can only issue one floating point instruction per cycle, and so outside of the SSE2 unit it can only achive clockspeed Gflops/s. Hence a 2.0 GHz P4/Xeon should be quoted as 4 GigaFlops peak As a side note SSE2 only works to the standard 64 bits, but the FPUs work to 80bits, hence you often get slightly different numerical results when comparing IEEE maths between the P4 FPU and the SSE2 or indeed between the P4 and say an Alpha. Yours, Daniel. -------------------------------------------------------------- Dr. Dan Kidger, Quadrics Ltd. daniel.kidger at quadrics.com One Bridewell St., Bristol, BS1 2AA, UK 0117 915 5505 ----------------------- www.quadrics.com -------------------- _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From joelja at darkwing.uoregon.edu Wed Jun 18 14:24:34 2003 From: joelja at darkwing.uoregon.edu (Joel Jaeggli) Date: Wed, 18 Jun 2003 11:24:34 -0700 (PDT) Subject: power supplies In-Reply-To: <20030618150850.70991.qmail@web80513.mail.yahoo.com> Message-ID: If you carry 3.3volts for any significant distance you'll have a substantial voltage drop the ammount of courrent you can will quickly result in larger conductors... you can get nebs compliant -48volt dc atx power supplies, then just use use a hanging huge rectifier (or two) to power the whole stack. but then you have the joy of working with large amounts of dc power... joelja On Wed, 18 Jun 2003, Andrew Latham wrote: > Has any one used or liked any AT/ATX external power supplies. My clusters are > using minimal power. The power supplys are wasting energy converting(heat). > I see the advantage to building a rackmount power converter and playing with > independant ATX swithing. What are some ideas or comments about such a project. > > > ===== > Andrew Latham > LathamA.com > lathama at lathama.com > lathama at yahoo.com > LathamA(lay-th-ham-eh) > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org > To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf > -- -------------------------------------------------------------------------- Joel Jaeggli Academic User Services joelja at darkwing.uoregon.edu -- PGP Key Fingerprint: 1DE9 8FCA 51FB 4195 B42A 9C32 A30D 121E -- In Dr. Johnson's famous dictionary patriotism is defined as the last resort of the scoundrel. With all due respect to an enlightened but inferior lexicographer I beg to submit that it is the first. -- Ambrose Bierce, "The Devil's Dictionary" _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From thakur at mcs.anl.gov Wed Jun 18 15:19:57 2003 From: thakur at mcs.anl.gov (Rajeev Thakur) Date: Wed, 18 Jun 2003 14:19:57 -0500 Subject: MPI-REDUCE-TO-ALL In-Reply-To: <200306181904.h5IJ4eU10906@NewBlue.Scyld.com> Message-ID: > Date: Wed, 18 Jun 2003 16:02:22 -0300 > From: Renato Silva > Cc: beowulf at beowulf.org > > Hi Folks > > Does anybody knows what are the communication algorithm used in the > mpich-2.5 > to do the Reduce-to-all ? > > Thanks > Renato Silva All the collective algorithms are described in the source code. See the file src/coll/intra_fns_new.c and look for MPI_Allreduce. The next release of MPICH will have a different algorithm for long-message allreduce. Rajeev _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rssr at lncc.br Wed Jun 18 15:02:22 2003 From: rssr at lncc.br (Renato Silva) Date: Wed, 18 Jun 2003 16:02:22 -0300 Subject: MPI-REDUCE-TO-ALL References: <033001c33389$5dde1f00$27b358c7@titan> <3EED8370.ADD04801@ideafix.litec.csic.es> Message-ID: <3EF0B73E.629EF049@lncc.br> Hi Folks Does anybody knows what are the communication algorithm used in the mpich-2.5 to do the Reduce-to-all ? Thanks Renato Silva _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From alvin at Maggie.Linux-Consulting.com Wed Jun 18 16:32:23 2003 From: alvin at Maggie.Linux-Consulting.com (Alvin Oga) Date: Wed, 18 Jun 2003 13:32:23 -0700 (PDT) Subject: power supplies In-Reply-To: <20030618150850.70991.qmail@web80513.mail.yahoo.com> Message-ID: hi ya andrew if you have access to the outside ... ( sending exhaust air outside ) than, i'd use 48v or 12 car batteries or sealed lead/acid batteries use a dc-dc converter to drive your +12vdc atx power supply or -48vdc atx power supply - but these +12vdc atx power supp is expensive compared to $20 110vac atx supply - we're using an $5 ac adaptor to give +12vdc out and plugs into a custom dc-dc convertor ( $50 ) to give us about 100W atx power supply capacity have fun alvin On Wed, 18 Jun 2003, Andrew Latham wrote: > Has any one used or liked any AT/ATX external power supplies. My clusters are > using minimal power. The power supplys are wasting energy converting(heat). > I see the advantage to building a rackmount power converter and playing with > independant ATX swithing. What are some ideas or comments about such a project. > _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From dinamica at webcable.com.br Wed Jun 18 15:41:02 2003 From: dinamica at webcable.com.br (Dinamica) Date: Wed, 18 Jun 2003 16:41:02 -0300 Subject: MPI and commons in FORTRAN Message-ID: <20030618194102.GA3233@webcable.com.br> Hi list, I am afraid this is a bit off-topic but i have searched the web and could not find any helpful pointer about this. Pls, let me know if you can help me or if there is another forum to post this: I would like to parallelize just one subroutine call (the func/error evaluation) of a large FORTRAN code (the genetic algorhythm by Carrol). I have tried to code this in a couple of ways: 1) MPI_INIT'ing in the main routine or 2) MPI_INIT'ing only in the routine that contains the loop (to be parallelized) and actually makes the call to the func/error evaluation. None of them work and i am afraid i am missing something regarding the way common blocks are handled in MPI/parallel codes. Since this is the first time i try something more elaborate (IMHO) with MPI and FORTRAN, I am a bit lost and would like to know if there is any documentation or examples i can read more about this. Sorry if this looks vague, i can give more details if you want. Thanks a lot for your help, Guilherme Menegon IQUSP, Brasil --------------------------- _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From mphil39 at hotmail.com Wed Jun 18 18:05:22 2003 From: mphil39 at hotmail.com (Matt Phillips) Date: Wed, 18 Jun 2003 18:05:22 -0400 Subject: Beowulf in Enterprise Environments Message-ID: On Tue, 17 Jun 2003, Robert G. Brown wrote: >crack (the venerable Unix password cracking program) can be run >embarrassingly parallel over a network and in fact has been programmed >to do so on request for a rather long time now. Sounds reasonable that it can be made embarrassingly parallel. However, its creator Alec Muffett continue s to deny that it can/should be parallelized. http://www.crypticide.org/users/alecm/security/c50-faq.html (Micsellany section Q 5&6 ) Matt _________________________________________________________________ Protect your PC - get McAfee.com VirusScan Online http://clinic.mcafee.com/clinic/ibuy/campaign.asp?cid=3963 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rgb at phy.duke.edu Wed Jun 18 21:05:59 2003 From: rgb at phy.duke.edu (Robert G. Brown) Date: Wed, 18 Jun 2003 21:05:59 -0400 (EDT) Subject: Beowulf in Enterprise Environments In-Reply-To: Message-ID: On Wed, 18 Jun 2003, Matt Phillips wrote: > > On Tue, 17 Jun 2003, Robert G. Brown wrote: > >crack (the venerable Unix password cracking program) can be run > >embarrassingly parallel over a network and in fact has been programmed > >to do so on request for a rather long time now. > > Sounds reasonable that it can be made embarrassingly parallel. However, its > creator Alec Muffett continue s to deny that it can/should be parallelized. > > http://www.crypticide.org/users/alecm/security/c50-faq.html (Micsellany > section Q 5&6 ) You mean the part of the README where network mode is described? As in: %< Snip ============== * conf/network.conf This is the file used to configure Crack for network running; this file contains lines, each of which has several fields: host:relpow:nfsbool:rshuser:crackdir Where: o host This is the name of the host to which Crack should "rsh", in order to despatch a job. There can be several instances of the same hostname in the file, if desired, in order to dispatch more than one job to a given host (if it has more than one CPU, for instance). %< Snip ============== Sounds like integrated, embarrassingly parallel operation to me... Works that way too, back when I used to run crack on passwd files;-) rgb -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb at phy.duke.edu _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From hahn at physics.mcmaster.ca Wed Jun 18 22:19:37 2003 From: hahn at physics.mcmaster.ca (Mark Hahn) Date: Wed, 18 Jun 2003 22:19:37 -0400 (EDT) Subject: power supplies In-Reply-To: Message-ID: > If you carry 3.3volts for any significant distance you'll have a > substantial voltage drop the ammount of courrent you can will quickly > result in larger conductors... I have this remarkable image in my mind of 20 or so lovely EATX dual-opteron motherboards arranged radially like fins around a couple of heavy-gauge copper cables running vertically through a cylindrical rack, with a monster fan at the bottom. it would be lovely, and perhaps even worthy of a cray-ish couch around it ;) heck, such an arrangement could even cut down on that murderous time-of-flight component in gigabit ethernet performance :) _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From hahn at physics.mcmaster.ca Wed Jun 18 23:03:09 2003 From: hahn at physics.mcmaster.ca (Mark Hahn) Date: Wed, 18 Jun 2003 23:03:09 -0400 (EDT) Subject: Enterprise Beowulf and Beowulf on a Chip In-Reply-To: Message-ID: > In my own search for finding out whether Beowulf makes sense for > enterprise computing (specifically on-line transaction processing > (OLTP)), OLTP has significant throughput-type parallelism, so can be done by clusters quite nicely. it could be that you'd want a low-latency interconnect for lockmanager-type synchronization, but wouldn't necessarily need much bandwidth. > This work emphasizes "large memory stall times" as being > a primary culprit limiting OLTP performance. The solution right: OLTP is computationally trivial. so you can either use CMP-type parallelism (where stalls still waste the CPU for the duration, but CPUs are cheap) or SMT (which they reject for a practical reason, namely that CMP is a lot easier to throw together.) > multiprocessing (CMP). This makes me wonder if OLTP > can benefit from simply lots of processors (with fast interconnect) > to utilize more L1, L2 cache simultaneously (a might bit larger > than a chip I might add!!)... it's not clear how much good cache does you for OLTP - if you look at figure 6, you see that miss rate decreases between P4 and P8 - that is, when you have 8 1M caches, you start to approach a working set. in figure 8, you see that misses are still something like 33% of execution time, though, so even 12M cache is not terribly effective. > If this approach makes sense for OLTP doesn't a Beowulf > make sense for OLTP work now? clusters are great for OLTP, but Beowulf is usually considered a fairly specific kind of cluster, which is tuned in ways that are not much of advantage for OLTP. > If Beowulf makes sense on the macro level does it make > sense in the micro-level or perhaps in the fractal sense of > a self similar architecture (exploiting even more hierarchy)? I do not believe there is any problem obtaining whatever OLTP performance you need. do you need 700K tpmC? I really don't think so. making OLTP cheap (even at modest performance) is an entirely different topic. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From auro at nietzsche.mit.edu Wed Jun 18 23:22:25 2003 From: auro at nietzsche.mit.edu (Stefano) Date: Wed, 18 Jun 2003 23:22:25 -0400 (EDT) Subject: cluster of AOD Opteron In-Reply-To: Message-ID: Hi guys, As I am going to receive some funding this fall, I was wondering of buying an opteron cluster for my research. Mainlym the cluster will run VASP (an ab-initio quantum program, written by a group in Wien), with myrinet. Is somebody who is using AMD opterons yet ? Any myrinet trouble ? Is somebody from AMD reading ? I think some fortran vendor has announced the port of their F90 to the opteron. Well, it would be nice to recompile VASP for 64bits and see how fast it goes. With the itanium2 (compiled in 2 version 32 and 64 bits), it not so fast to justify the HUGE cost of an itanium cluster. Maybe the opteron will shake high-performace scientific computing ! Regards, Stefano Curtarolo _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From kpodesta at redbrick.dcu.ie Thu Jun 19 07:20:49 2003 From: kpodesta at redbrick.dcu.ie (Karl Podesta) Date: Thu, 19 Jun 2003 12:20:49 +0100 Subject: Beowulf in Enterprise Environments In-Reply-To: References: Message-ID: <20030619112048.GE12882@prodigy.Redbrick.DCU.IE> On Wed, Jun 18, 2003 at 09:05:59PM -0400, Robert G. Brown wrote: > You mean the part of the README where network mode is described? As in: > - SNIP - > Sounds like integrated, embarrassingly parallel operation to me... > > Works that way too, back when I used to run crack on passwd files;-) > > rgb The author seems to be talking about parallelising the actual crypt() algorithm, to do a single crack accross many CPUs, and maintains it's like 9 women trying to produce 1 baby in 1 month :-) So perhaps a bit of mis-information on his side about parallelising operations? When he could simply run multiple instances of crack accross a network like you describe, a good oul bit of data/domain decomposition, maybe put an MPI wrapper round it to collate results or nicey it up for running it from a head node accross a cluster - that's maybe what people were requesting from him and he mis-understood. Kp -- Karl Podesta Dublin City University, Ireland _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rgb at phy.duke.edu Thu Jun 19 10:08:04 2003 From: rgb at phy.duke.edu (Robert G. Brown) Date: Thu, 19 Jun 2003 10:08:04 -0400 (EDT) Subject: Beowulf in Enterprise Environments In-Reply-To: <20030619112048.GE12882@prodigy.Redbrick.DCU.IE> Message-ID: On Thu, 19 Jun 2003, Karl Podesta wrote: > The author seems to be talking about parallelising the actual crypt() > algorithm, to do a single crack accross many CPUs, and maintains it's > like 9 women trying to produce 1 baby in 1 month :-) > > So perhaps a bit of mis-information on his side about parallelising > operations? When he could simply run multiple instances of crack accross > a network like you describe, a good oul bit of data/domain decomposition, > maybe put an MPI wrapper round it to collate results or nicey it up > for running it from a head node accross a cluster - that's maybe what > people were requesting from him and he mis-understood. You got me. As the crack README clearly states, it has been programmed to run (nearly EP) over a network from very early versions of crack. Crack works by applying rules and transformations to dictionaries to generate a trial password, crypting the trial password, and comparing the crypted result to the crypts in the passwd file, which is why the crypts were split off in public-unreadable /etc/shadow and made md5 to significantly increase the work required to generate the crypt. IIRC, the network version works by taking the ruleset to be tested and farming (chunks of) the ruleset out round-robin in a master-slave paradigm, in which the slaves return all the crypt hits they encounter to the master for cumulation. In some ways, it is a lovely model parallel application for relatively trivial parallelism -- a short message plus startup overhead to get a slave going, a short (well, one HOPES short:-) return from the slave, a whole lot of slave computation in between -- quite similar to the way a mandelbrot set generator is parallelized but without the baggage of PVM or MPI, scaling efficiency out to the high 90's for quite a few nodes in spite of the inefficiency of lots of rsh spawns. Parallelizing MD5 or libcrypt itself, well, that's just plain silly... like parallelizing a single threaded random number generator. Quite a lot like it, actually. Why, almost identical:-) Let's insert a really, really high latency network IPC hit in between ordinary arithmetic instructions in a single thread with innate sequential dependencies... hmmm. I don't know the timing because I can't remember when crack was first parallelized, but (to come back to the original thread about enterprise parallel applications) I'd bet that it preceded (and possibly inspired) even the venerable RC5 challenge etc. as one of the very first if not the first widely distributed and commonly used truly network parallel application programs, used in many an enterprise long before the "beowulf" per se was invented. I have no idea if the parallelization was done by black hats or white hats (or even both:-) -- the program was in pretty common use by both, back in the good old days when the ypx program could often as not provide you with a complete copy of any NIS domain's passwd file and crack could often as not give you a half dozen passwd hits from that file in the first four or five minutes of NON parallel operation on a 4 MFLOP Sun workstation. Or crackers would snoop a passwd from a telnet or rsh connection from some grad student logging back in to check mail over the summer and would then proceed to copy the local passwd file (pre-shadow) to work on at their liesure. Either you (as a white hat) ran crack on your passwd files and whomped users who thought their userid twice or a word like "unbreakable" was a "good password" upside the head or the black hats would cheerfully do it for you...:-) Needless to say, been there, coped with that. One of many reasons I'm a shameless advocate of eliminating rsh altogether from the world of programming. telnet can stay as a nice little daemon debugging tool, as long as telnetd is staked to an anthill along with rshd... rgb -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb at phy.duke.edu _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From edwardsa at plk.af.mil Thu Jun 19 12:02:31 2003 From: edwardsa at plk.af.mil (Arthur H. Edwards) Date: Thu, 19 Jun 2003 10:02:31 -0600 Subject: F95/F90 compilers Message-ID: <20030619160231.GA4799@plk.af.mil> I'm about to purchase a F95 compiler for a beowulf running Debian with mpich. I'm obviously looking for performance. I'm also looking for a compiler that doesn't choke or create garbage. Art Edwards -- Art Edwards Senior Research Physicist Air Force Research Laboratory Electronics Foundations Branch KAFB, New Mexico (505) 853-6042 (v) (505) 846-2290 (f) _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From gust_77ar at yahoo.com Thu Jun 19 12:16:58 2003 From: gust_77ar at yahoo.com (Gustavo Velardez) Date: Thu, 19 Jun 2003 09:16:58 -0700 (PDT) Subject: Cluster installation Message-ID: <20030619161658.44319.qmail@web14103.mail.yahoo.com> Hi everybody, We need to know the heat generated (that is, how much cooling is needed) by a beowulf cluster of 11 units. Each unit has 2 Pentium 4 processors. The master node has a power consumption of 338 W and each slave node 295 W. Where can I get this information? Thanx Gustavo __________________________________ Do you Yahoo!? SBC Yahoo! DSL - Now only $29.95 per month! http://sbc.yahoo.com _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From jhearns at freesolutions.net Thu Jun 19 12:36:22 2003 From: jhearns at freesolutions.net (John Hearns) Date: Thu, 19 Jun 2003 17:36:22 +0100 Subject: Network Booting References: <20030619090737.672f725f.torsten@howard.cc> Message-ID: <01a501c33680$ef5d5190$8461cdc2@DREAD> > 7. Does the Linux Terminal Server Project (LTSP) have anything useful > that I could/should be using? Yes indeed - I think there is an article linked from the LTSP page on installing a cluster using LTSP. And if you install LTSP, you will learn about PXE booting and DHCP _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From moor007 at bellsouth.net Thu Jun 19 12:56:02 2003 From: moor007 at bellsouth.net (moor007 at bellsouth.net) Date: Thu, 19 Jun 2003 11:56:02 -0500 Subject: cluster of AOD Opteron (Stefano) In-Reply-To: <200306191635.h5JGZGU30202@NewBlue.Scyld.com> Message-ID: <000401c33683$aaf403c0$0b01a8c0@redstorm> I just received my hardware yesterday for my opteron cluster. My tech will start putting it together today or tomorrow. I am building a 16 CPU cluster w/ the 240 processor onboard the Tyan 2880. I will be using the 2D wulfkit running SuSE enterprise server and Portland Group Server for the Opteron. I am hoping it will be fast. Of course, that is relative. Anyway, I said all that to say that I will begin posting performance benchmarks as they become available. Tim _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From egan at sense.net Thu Jun 19 12:54:49 2003 From: egan at sense.net (Egan Ford) Date: Thu, 19 Jun 2003 10:54:49 -0600 Subject: F95/F90 compilers In-Reply-To: <20030619160231.GA4799@plk.af.mil> Message-ID: <07c301c33683$805a19b0$27b358c7@titan> Both PGI and Intel have eval versions freely available. I recommend you try both and decide for yourself. I am assuming that you are using Intel/AMD-based clusters. My experience has been with both products is that price/performance is greater with the optimized compilers. > -----Original Message----- > From: beowulf-admin at scyld.com > [mailto:beowulf-admin at scyld.com] On Behalf Of Arthur H. Edwards > Sent: Thursday, June 19, 2003 10:03 AM > To: beowulf at beowulf.org > Subject: F95/F90 compilers > > > I'm about to purchase a F95 compiler for a beowulf running Debian with > mpich. I'm obviously looking for performance. I'm also looking for a > compiler that doesn't choke or create garbage. > > Art Edwards > > -- > Art Edwards > Senior Research Physicist > Air Force Research Laboratory > Electronics Foundations Branch > KAFB, New Mexico > > (505) 853-6042 (v) > (505) 846-2290 (f) > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org > To change your subscription (digest mode or unsubscribe) > visit http://www.beowulf.org/mailman/listinfo/beowulf > _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From torsten at howard.cc Thu Jun 19 09:07:37 2003 From: torsten at howard.cc (torsten) Date: Thu, 19 Jun 2003 09:07:37 -0400 Subject: Network Booting Message-ID: <20030619090737.672f725f.torsten@howard.cc> Good Morning: I've been struggling for days to make a single computer be able to network boot. I've read many documents, and I'm a little confused. Here are some questions, and I am very grateful for assistance, in advance. 1. I've seen references to bootp/dhcp, bootp or dhcp, bootp and dhcp, dhcp over bootp - Which one do I need. Do I need both bootp and dhcp? 2. My Lab is on the University subnet. Will my bootp/dhcp server conflict with the university system? Do I need to isolate my lab with IP masq first? 3. On which broadcast/subnet does PXE send bootp requests. Does it make a difference, with PXE, whether I use bootp or DHCP? 4. What are people using to setup/install partitions? I've seen bpbatch and ka-tools. I'm interested in GPL software, primarily. 5. Are there any known problems with Realtek PXE ROMS? My network cards are system integrated Realtek's. 6. Should I just download and setup the MOSIX stuff? 7. Does the Linux Terminal Server Project (LTSP) have anything useful that I could/should be using? Thanks, Torsten _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From lathama at yahoo.com Thu Jun 19 00:46:50 2003 From: lathama at yahoo.com (Andrew Latham) Date: Wed, 18 Jun 2003 21:46:50 -0700 (PDT) Subject: power supplies In-Reply-To: Message-ID: <20030619044650.61470.qmail@web80513.mail.yahoo.com> "... cray-ish couch ..." Now this was what I was thinking about :-) I understand the powerloss. However the heat produced by 9 AT powersupplies by using ~20 watts each is more than they are worth. I think that there is a more efficient method. Considering a cable run of 3 ft. a center 4U chasis with a sizable powersupply could carry nearly a full rack. These are ruff estimites mind you. Does this seem to be pos. or is the 48V method going to win. I would prefer to wire it myself and I found a great place for rackmount goodies (mouser.com) --- Mark Hahn wrote: > > If you carry 3.3volts for any significant distance you'll have a > > substantial voltage drop the ammount of courrent you can will quickly > > result in larger conductors... > > I have this remarkable image in my mind of 20 or so lovely EATX > dual-opteron motherboards arranged radially like fins around > a couple of heavy-gauge copper cables running vertically through > a cylindrical rack, with a monster fan at the bottom. it would > be lovely, and perhaps even worthy of a cray-ish couch around it ;) > > heck, such an arrangement could even cut down on that murderous > time-of-flight component in gigabit ethernet performance :) > > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org > To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf ===== Andrew Latham LathamA.com lathama at lathama.com lathama at yahoo.com LathamA(lay-th-ham-eh) _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From beerli at csit.fsu.edu Thu Jun 19 10:49:24 2003 From: beerli at csit.fsu.edu (Peter Beerli) Date: Thu, 19 Jun 2003 10:49:24 -0400 (EDT) Subject: seeking citation of EP Message-ID: Does someone know who came up with the term "Embarrassingly Parallel" and give the citation? If there are several contexts then the preferred context would be Markov chain Monte Carlo related. thanks, Peter ---- Peter Beerli, School of Computational Science and Information Technology (CSIT) Dirac Science Library, Florida State University Tallahassee, Florida 32306-4120 USA old webpage: http://evolution.genetics.washington.edu/PBhtmls/beerli.html _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From egan at sense.net Thu Jun 19 13:30:23 2003 From: egan at sense.net (Egan Ford) Date: Thu, 19 Jun 2003 11:30:23 -0600 Subject: Network Booting In-Reply-To: <20030619090737.672f725f.torsten@howard.cc> Message-ID: <07c501c33688$76ee0b70$27b358c7@titan> > 1. I've seen references to bootp/dhcp, bootp or dhcp, bootp > and dhcp, > dhcp over bootp - Which one do I need. Do I need both bootp and dhcp? Solutions like pxelinux.0, nbgrub, and etherboot all work with dhcp. I find dhcp easy to work with. > 2. My Lab is on the University subnet. Will my > bootp/dhcp server > conflict with the university system? Do I need to isolate > my lab with > IP masq first? Block dhcp/bootp traffic at the switch port that uplinks to the university system. If you have a multi homed management node running dhcp edit your dhcp startup script or config file to only bind on the eth device that is for your cluster. > > 3. On which broadcast/subnet does PXE send bootp requests. > Does it make > a difference, with PXE, whether I use bootp or DHCP? PXE uses DHCP to get an address. > 4. What are people using to setup/install partitions? I've > seen bpbatch > and ka-tools. I'm interested in GPL software, primarily. Use pxelinux.0 or nbgrub to network boot, then use the native network installer for your OS. e.g. kickstart for RH, autoyast for SuSE. Or use systemimager for any distro. > 6. Should I just download and setup the MOSIX stuff? I would recommend a cluster without MOSIX first, then later determine if MOSIX is the right solution for you. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From glen at cert.ucr.edu Thu Jun 19 12:44:34 2003 From: glen at cert.ucr.edu (Glen Kaukola) Date: Thu, 19 Jun 2003 09:44:34 -0700 Subject: F95/F90 compilers In-Reply-To: <20030619160231.GA4799@plk.af.mil> References: <20030619160231.GA4799@plk.af.mil> Message-ID: <3EF1E872.5070404@cert.ucr.edu> Arthur H. Edwards wrote: >I'm about to purchase a F95 compiler for a beowulf running Debian with >mpich. I'm obviously looking for performance. I'm also looking for a >compiler that doesn't choke or create garbage. > The Fortran compiler from Intel is free, as long as you're not going to be producing any commercial applications with it I think. And our experience with it, is that even on an AMD processor the code that it produces runs faster than the code produced by our portland group compiler. Glen _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From egan at sense.net Thu Jun 19 12:54:49 2003 From: egan at sense.net (Egan Ford) Date: Thu, 19 Jun 2003 10:54:49 -0600 Subject: seeking citation of EP In-Reply-To: Message-ID: <07c401c33683$80912f40$27b358c7@titan> Unrelated, but a HPC shop I visited preferred the term, "naturally parallel". Guess "embarrassingly" was negative. > -----Original Message----- > From: beowulf-admin at scyld.com > [mailto:beowulf-admin at scyld.com] On Behalf Of Peter Beerli > Sent: Thursday, June 19, 2003 8:49 AM > To: beowulf at beowulf.org > Subject: seeking citation of EP > > > Does someone know who came up with > the term > "Embarrassingly Parallel" > and give the citation? If there are several contexts then > the preferred context would be Markov chain Monte Carlo related. > > thanks, > Peter > ---- > Peter Beerli, > School of Computational Science and Information Technology (CSIT) > Dirac Science Library, Florida State University > Tallahassee, Florida 32306-4120 USA > old webpage: > http://evolution.genetics.washington.edu/PBhtm> ls/beerli.html > > > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org > To change your subscription (digest mode or unsubscribe) > visit http://www.beowulf.org/mailman/listinfo/beowulf > _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From hahn at physics.mcmaster.ca Thu Jun 19 14:01:58 2003 From: hahn at physics.mcmaster.ca (Mark Hahn) Date: Thu, 19 Jun 2003 14:01:58 -0400 (EDT) Subject: seeking citation of EP In-Reply-To: <07c401c33683$80912f40$27b358c7@titan> Message-ID: > Unrelated, but a HPC shop I visited preferred the term, "naturally > parallel". Guess "embarrassingly" was negative. euphemisms are usually dumb, but that's exceptionally so. EP is no more natural than tightly-coupled parallelism (ie, the opposite of EP.) if you care about honest, meaningful terms, use tight/loose coupling. personally, I like EP - an allusion to "embarassment of riches", I believe, and probably from Pfister's book. has that ring to it ;) _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rgb at phy.duke.edu Thu Jun 19 13:50:17 2003 From: rgb at phy.duke.edu (Robert G. Brown) Date: Thu, 19 Jun 2003 13:50:17 -0400 (EDT) Subject: Cluster installation In-Reply-To: <20030619161658.44319.qmail@web14103.mail.yahoo.com> Message-ID: On Thu, 19 Jun 2003, Gustavo Velardez wrote: > Hi everybody, > We need to know the heat generated (that is, how much > cooling is needed) by a beowulf cluster of 11 units. > Each unit has 2 Pentium 4 processors. The master node > has a power consumption of 338 W and each slave node > 295 W. > Where can I get this information? > Thanx > > Gustavo You mean, other than by adding 338+2950 = 3288 Watts (Joules per second)? This is a bit less than one ton of air conditioning (a "ton" of AC is the capacity to remove 3504 watts continuously). However, I'd be lavish and install a healthy overcapacity to allow for growth of your cluster and to keep the space >>cold<< and not just not hot. Also, human bodies add 100W or so each, lights ditto, and the walls or exterior ventilation can allow heat to diffuse in or out depending on whether the ambient space outside is warmer or cooler. Two or three tons wouldn't be out of the question, with adequate flow and delivery and warm air return. Note that a "ton" of AC doesn't necessarily mean a particularly large compressor/chiller. We're talking a bit more than a window unit (although there are some pretty big window units) but all the AC you need should sit nicely in less than a cubic meter of outdoor space, plus of course plumbing and the blower to actually deliver "cold" to the space to be cooled (or more properly, to remove the heat therefrom). rgb -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb at phy.duke.edu _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rgb at phy.duke.edu Thu Jun 19 13:40:47 2003 From: rgb at phy.duke.edu (Robert G. Brown) Date: Thu, 19 Jun 2003 13:40:47 -0400 (EDT) Subject: Network Booting In-Reply-To: <20030619090737.672f725f.torsten@howard.cc> Message-ID: On Thu, 19 Jun 2003, torsten wrote: > Good Morning: > > I've been struggling for days to make a single computer be able to > network boot. I've read many documents, and I'm a little confused. > Here are some questions, and I am very grateful for assistance, in > advance. Have you read in particular the HOWTOs on tldp.org? The DHCP and the Diskless Linux howtos: http://www.tldp.org/HOWTO/mini/DHCP/index.html http://www.tldp.org/HOWTO/Diskless-HOWTO.html http://www.tldp.org/HOWTO/Diskless-root-NFS-HOWTO.html http://www.tldp.org/HOWTO/Diskless-root-NFS-other-HOWTO.html are good, and the following aren't quite HOWTOs for PXE but they cross reference a lot of documents that can be used to help you out. http://www.kegel.com/linux/pxe.html http://www.stanford.edu/~alfw/PXE-Kickstart/PXE-Kickstart.html > 1. I've seen references to bootp/dhcp, bootp or dhcp, bootp and dhcp, > dhcp over bootp - Which one do I need. Do I need both bootp and dhcp? bootp is a very old network protocol supporting diskless booting. You send a bootp broadcast/request out on a network, and it returns your network identity (which it works out based on e.g. your MAC address) and a tftp (trivial file transfer protocol) address where you can find what you devoutly hope is the kernel you are supposed to boot. TFTP is a sucking wound from the point of view of security and hence must be used with caution -- in the old days it was pretty easy to set up tftp by accident so that anybody in the netverse could do a tftp connection to the server and ask nicely for /etc/passwd (see earlier comment on parallel crack:-) and it would just say uh, duh, sure... dhcp is a modern improvement (specified by various RFC's as is, for that matter, bootp itself) on bootp. In addition to giving the connecting host an IP number either from a pool or based on its explicit MAC address, it can send all kinds of other information back to the client. For example, mask information, nameserver(s), lease time, NIS information, NTP information, routing information -- enough to totally configure the interface. In addition, it can (like bootp) specify both server addresses and file paths on the servers. A lot of your conflict likely arises because for a while dhcp couldn't directly support pxe booting, so a pxe client was developed that could. We've used it here for a fairly long time and it works fine, and you can even find details of setting up pxelinux including sample configuration files on the brahma resources page (resources for beowulf builders). However, in recent linuces dhpc can directly handle PXE (bootp) requests. So all you SHOULD need is tftp and dhcp. Alas, the TLDP DHPC HOWTO seems not to have caught on to this yet, although the PXE HOWTO has gone away -- "interesting"... a TLDP gap. Still, as Kegel's lovely article makes clear, with just DHPC and TFTP you can give a system all kinds of things to boot: grub, pxelinux, etherboot, a network install image, a diskless boot image, or "whatever evil program you desire". The latter means that you need to look out for people running dhcp servers in your LAN, as in at least the old days the fastest server (when there is more than one) wins and gets to tell the booting host what to boot. [Ah, I remember well the days when SGI's came with bootparamd on by default and were always the fastest thing on any LAN and were never configured to support diskless boots of e.g. Suns and in fact were somewhat broken for that purpose. You could always tell when a new one was installed because all the diskless systems stopped booting until you went over and drilled a stake through the SGI's little daemonic heart.] Nowadays it's merely a security problem. Use whatever means necessary to ensure that no bootp/dhcp servers exist within the broadcast boundary of your LAN (up to the router/switch that filters broadcasts, if you've got one). Try not to kill anyone. Permanently that is. Shooting them in the kneecaps is generally held to be ok. > 2. My Lab is on the University subnet. Will my bootp/dhcp server > conflict with the university system? Do I need to isolate my lab with > IP masq first? See previous comments. I personally am fond of my kneecaps and imagine that you kinda like your own. Therefore, I would be very certain that your LAN is isolated from the University sytem if they indeed run a WAN dhpc server before setting up one of your own. Even if they aren't running a dhcp server on the WAN, you'll still want this isolation. Do YOU want to be giving out IP numbers and so forth to all the booting systems on campus? Of course not. You might do a really good job and find yourself saddled with it permamently. I think I'd rather get shot in the kneecaps than that. > 3. On which broadcast/subnet does PXE send bootp requests. Does it make > a difference, with PXE, whether I use bootp or DHCP? The broadcasts are ETHERNET broadcasts. That is, they aren't IP broadcasts at all as IP hasn't yet been configured on the host containing the device issuing the request (if that makes sense). The NIC shouts at the top of its metaphorical lungs IS THERE ANYBODY OUT THERE THAT HAS THE FAINTEST IDEA WHO I AM and hopefully the One True Server says "why certainly, you'd be 192.168.1.38, your netmask is 255.255.255.0, you can route through 192.168.1.250, your server is 192.168.1.1, and if you ask it nicely it will send you an image at /tftpboot/i386/kernel_RH9 so you can boot..." The security problem is obvious, as if my slightly faster server answers first, I can tell it to boot /tftpboot/i386/kernel_RH9_evil from my very own system, the one that behaves just like RH9's kernel except for the backdoor on port 47319 that I can get into if I enter the right 1337 password. Other reasons for limiting the range of the broadcast are similarly obvious. An ethernet broadcast is moderately expensive as it goes to every port on a switch and requires active processing by every host when it gets there. It actually eats a bit of CPU and net bandwidth from every host and branch of the LAN. This is why network segmentation was invented, in a way -- some old networks were notorious for generating truly incredible amounts of broadcast traffic, kind of like a perpetually running DoS attack that was perfectly "normal". It is pretty easy these days to create a boundary to a broadcast domain. Lots of (managed) switches permit you to turn off broadcast forwarding. Of course, a router or dual headed linux box used as a router accomplish the same thing even better. > 4. What are people using to setup/install partitions? I've seen bpbatch > and ka-tools. I'm interested in GPL software, primarily. I'm not sure what you mean here. Are you talking about e.g. fdisk/mkfs? Higher order front ends of same? Or something for setting up diskless systems specifically? We actually mostly use PXE/DHCP to fuel kickstart, which creates partitions and installs them flawlessly and even moderately intelligently. There are lots of ways to accomplish the same thing, though, including scripts that use fdisk and mkfs. Most of them are probably GPL -- I didn't realize that anybody was selling proprietary linux tools in this space (there's an oxymoron for you -- "proprietary linux tools";-). > 5. Are there any known problems with Realtek PXE ROMS? My network cards > are system integrated Realtek's. I will try to stifle any response, since I honestly don't know how their PXE cards behave. However, overall I loathe RTL NICs. I wouldn't be surprised if they behave badly. Then again, in the fullness of time even RealTek may get the humble NIC right... > 6. Should I just download and setup the MOSIX stuff? Depends on what you want to do. Generally speaking I'd answer "no" with a puzzled expression on my face, although for certain task mixes it isn't horribly unreasonable. One major problem is that it lags the major distributions pretty significantly, which can produce both security and user satisfaction gaps. However, it does work nicely as a user-transparent, moderately "expensive" job migration facility, when it works. > 7. Does the Linux Terminal Server Project (LTSP) have anything useful > that I could/should be using? See answer to 6. Conceivably, but it depends on what you're trying to do. So what are you trying to do? (Hope some of the above helps:-) rgb -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb at phy.duke.edu _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From josip at lanl.gov Thu Jun 19 16:26:04 2003 From: josip at lanl.gov (Josip Loncaric) Date: Thu, 19 Jun 2003 14:26:04 -0600 Subject: F95/F90 compilers In-Reply-To: <3EF1E872.5070404@cert.ucr.edu> References: <20030619160231.GA4799@plk.af.mil> <3EF1E872.5070404@cert.ucr.edu> Message-ID: <3EF21C5C.10207@lanl.gov> Glen Kaukola wrote: > Arthur H. Edwards wrote: > >> I'm about to purchase a F95 compiler for a beowulf running Debian with >> mpich. I'm obviously looking for performance. I'm also looking for a >> compiler that doesn't choke or create garbage. >> > > The Fortran compiler from Intel is free, as long as you're not going to > be producing any commercial applications with it I think. And our > experience with it, is that even on an AMD processor the code that it > produces runs faster than the code produced by our portland group compiler. Agreed -- but at my old institute, we had pretty good luck with the Portland Group Fortran. Intel's Fortran can be faster, but unfortunately, it sometimes generates incorrect code. Some ifc 7.0 versions incorrectly handled equivalence statements -- which are liberally used in many old Fortran packages. I'm not sure if this has been completely fixed even in the latest ifc 7.1 versions. Sincerely, Josip _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rgb at phy.duke.edu Thu Jun 19 15:53:05 2003 From: rgb at phy.duke.edu (Robert G. Brown) Date: Thu, 19 Jun 2003 15:53:05 -0400 (EDT) Subject: Network Booting In-Reply-To: <20030619144306.571533d6.torsten@howard.cc> Message-ID: On Thu, 19 Jun 2003, torsten wrote: > > >> 7. Does the Linux Terminal Server Project (LTSP) have anything useful > >> that I could/should be using? > > > >See answer to 6. Conceivably, but it depends on what you're trying to > >do. So what are you trying to do? > > > Thank you for the very complete answers. This solves a number of concerns > I have run into. > > I am trying setup a cluster to run MPICH. Some of the PC's are > headed workstations, the rest will be headless (for MPICH only). > > While we have manually (via CD-ROM) setup the workstations, I want to > automate the addition and maintenance of headless PC's. An excellent idea. There are a number of resources linked to the brahma page http://www.phy.duke.edu/brahma that should help you out. There are lots of ways to do it. We use DHCP+PXE+KICKSTART+YUM here (and there is moderate documentation of how to go about it on one of the brahma references) but there is also FAL (or is it FAI) for debian, there is Scyld (if you want a commercial solution that makes your cluster pretty much plug-n-play for MPI jobs), there is clustermatic and bproc, and Mandrake and SuSE have solutions of their own I'm less familiar with. > > I am trying to use DHCP with PXE booting to boot the PC's. Any advice > toward this general direction, I am grateful for. Hopefully some of the many links I've thrown at you will be of use. I don't think it is useful for me to write a HOWTO directly to the whole list, though... especially when there is so much excellent documentation now online. rgb P.S. -- I can now say that Mosix is not a good idea. I'd suggest picking a package-based distribution that supports automated installation AND an automated package maintenance tool and go with it OR pick Scyld, dicker out a reasonable license fee for your small cluster, and go that way. Your long term interests are not just automating installation but automating REinstallation, running maintenance package updates and upgrades, and LAN operation (accountspace, shared filespace and so forth). An e.g. RH based cluster is one approach that makes your cluster very much like a LAN of workstations, just some of those workstations have no heads and are used pretty much only for MPI or PVM or EP/script managed computations. The Scyld cluster makes your cluster nodes into "a supercomputer" -- a virtual MP machine -- typically with a single head, a custom task/file loader and clusterwide PID space. If I understand it correctly (I'm sure Don will correct me if I'm misrepresenting it:-). Your current model sounds more like a NOW cluster with multiple servers and points of access and nodes you can login to (you don't log into a "node" in a Scyld cluster any more than you log into a "processor" in an MP machine:-) but I don't know if that is by deliberate design or just what you knew how to do. -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb at phy.duke.edu _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From torsten at howard.cc Thu Jun 19 14:43:06 2003 From: torsten at howard.cc (torsten) Date: Thu, 19 Jun 2003 14:43:06 -0400 Subject: Network Booting In-Reply-To: References: <20030619090737.672f725f.torsten@howard.cc> Message-ID: <20030619144306.571533d6.torsten@howard.cc> >> 7. Does the Linux Terminal Server Project (LTSP) have anything useful >> that I could/should be using? > >See answer to 6. Conceivably, but it depends on what you're trying to >do. So what are you trying to do? Thank you for the very complete answers. This solves a number of concerns I have run into. I am trying setup a cluster to run MPICH. Some of the PC's are headed workstations, the rest will be headless (for MPICH only). While we have manually (via CD-ROM) setup the workstations, I want to automate the addition and maintenance of headless PC's. I am trying to use DHCP with PXE booting to boot the PC's. Any advice toward this general direction, I am grateful for. Torsten _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From James.P.Lux at jpl.nasa.gov Thu Jun 19 16:14:59 2003 From: James.P.Lux at jpl.nasa.gov (Jim Lux) Date: Thu, 19 Jun 2003 13:14:59 -0700 Subject: Cluster installation In-Reply-To: <20030619161658.44319.qmail@web14103.mail.yahoo.com> Message-ID: <5.2.0.9.2.20030619130907.02f5a960@mailhost4.jpl.nasa.gov> Simple.. all the electrical power is converted to heat. 1 unit @338W and 10 units at 295W gives you, to a first order, 2950+338 = 3288 Watts... So, in energy terms, that's 3.3 kJ/second. You can convert kJ to calories, BTU, tons of ice, etc. A comment: 295 and 338 appear to be overly precise numbers. I would venture to guess that the instantaneous power consumption of a PC varies somewhat, depending on instruction stream (fine scale) and environmental effects (line voltage, air temperature, etc.) You might just assume 350W per node and be done with it... Nit picky note: indeed, the information coming out of the cluster is somewhat more organized (i.e. lower entropy) than the information going in, and there is some small, but non-zero, amount of energy required to reduce the information entropy. Then again, one can also use the energy to randomize the data. At 09:16 AM 6/19/2003 -0700, you wrote: >Hi everybody, >We need to know the heat generated (that is, how much >cooling is needed) by a beowulf cluster of 11 units. >Each unit has 2 Pentium 4 processors. The master node >has a power consumption of 338 W and each slave node >295 W. >Where can I get this information? >Thanx > > Gustavo James Lux, P.E. Spacecraft Telecommunications Section Jet Propulsion Laboratory, Mail Stop 161-213 4800 Oak Grove Drive Pasadena CA 91109 tel: (818)354-2075 fax: (818)393-6875 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rbw at ahpcrc.org Thu Jun 19 17:13:09 2003 From: rbw at ahpcrc.org (Richard Walsh) Date: Thu, 19 Jun 2003 16:13:09 -0500 Subject: Cluster installation Message-ID: <200306192113.h5JLD9N24407@mycroft.ahpcrc.org> Jim Lux wrote: >Then again, one can also use the energy to randomize the data. Can one get paid to do this ... :-) ... ? Sounds like a case of turning lead into dross rather than gold. Nature is "nicht boshaft", but programs do this more often than we would like, especially when run as black boxen. May all your programming experiences be ... entropy reducing. Maxwell's Daemon (rbw) #--------------------------------------------------- # Richard Walsh # Project Manager, Cluster Computing, Computational # Chemistry and Finance # netASPx, Inc. # 1200 Washington Ave. So. # Minneapolis, MN 55415 # VOX: 612-337-3467 # FAX: 612-337-3400 # EMAIL: rbw at networkcs.com, richard.walsh at netaspx.com # rbw at ahpcrc.org # #--------------------------------------------------- # "Put a glass of wine in a barrel of sewage and you # have sewage. Put a glass of sewage in a barrel of # wine and you also have sewage. # -Schopenhauer #--------------------------------------------------- _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From mikee at mikee.ath.cx Thu Jun 19 17:25:06 2003 From: mikee at mikee.ath.cx (Mike Eggleston) Date: Thu, 19 Jun 2003 16:25:06 -0500 Subject: Network Booting In-Reply-To: <20030619144306.571533d6.torsten@howard.cc>; from torsten@howard.cc on Thu, Jun 19, 2003 at 02:43:06PM -0400 References: <20030619090737.672f725f.torsten@howard.cc> <20030619144306.571533d6.torsten@howard.cc> Message-ID: <20030619162506.A19914@mikee.ath.cx> On Thu, 19 Jun 2003, torsten wrote: > > >> 7. Does the Linux Terminal Server Project (LTSP) have anything useful > >> that I could/should be using? > > > >See answer to 6. Conceivably, but it depends on what you're trying to > >do. So what are you trying to do? > > > Thank you for the very complete answers. This solves a number of concerns > I have run into. > > I am trying setup a cluster to run MPICH. Some of the PC's are > headed workstations, the rest will be headless (for MPICH only). > > While we have manually (via CD-ROM) setup the workstations, I want to > automate the addition and maintenance of headless PC's. > > I am trying to use DHCP with PXE booting to boot the PC's. Any advice > toward this general direction, I am grateful for. I have used LTSP to setup an ad-hoc cluster for PVM. It worked well. Mike _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From ctierney at hpti.com Thu Jun 19 16:53:22 2003 From: ctierney at hpti.com (Craig Tierney) Date: 19 Jun 2003 14:53:22 -0600 Subject: F95/F90 compilers In-Reply-To: <3EF21C5C.10207@lanl.gov> References: <20030619160231.GA4799@plk.af.mil> <3EF1E872.5070404@cert.ucr.edu> <3EF21C5C.10207@lanl.gov> Message-ID: <1056056002.11801.134.camel@woody> On Thu, 2003-06-19 at 14:26, Josip Loncaric wrote: > Glen Kaukola wrote: > > Arthur H. Edwards wrote: > > > >> I'm about to purchase a F95 compiler for a beowulf running Debian with > >> mpich. I'm obviously looking for performance. I'm also looking for a > >> compiler that doesn't choke or create garbage. > >> > > > > The Fortran compiler from Intel is free, as long as you're not going to > > be producing any commercial applications with it I think. And our > > experience with it, is that even on an AMD processor the code that it > > produces runs faster than the code produced by our portland group compiler. > > Agreed -- but at my old institute, we had pretty good luck with the > Portland Group Fortran. Intel's Fortran can be faster, but > unfortunately, it sometimes generates incorrect code. Some ifc 7.0 > versions incorrectly handled equivalence statements -- which are > liberally used in many old Fortran packages. I'm not sure if this has > been completely fixed even in the latest ifc 7.1 versions. > And at times the Portland Group Fortran generates incorrect code as well. PG has a much longer history in high performance computing so they better support some features that users are used to having. Intel is trying to play catch up. They are doing a good job. I try and keep both compilers around just in case I have a problem with one or the other. Craig > Sincerely, > Josip > > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org > To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- Craig Tierney _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From jducom at nd.edu Thu Jun 19 17:43:12 2003 From: jducom at nd.edu (Jean-Christophe Ducom) Date: Thu, 19 Jun 2003 16:43:12 -0500 Subject: Channel bonding - CPU affinity Message-ID: <3EF22E70.6030800@nd.edu> Just wondering: is it possible to bind an ethernet interface to a specific cpu via cpu affinity mask? Or is it just simpler (and much better in terms of performance) to use channel bonding? Thanks for any comments. JC _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From yudong at lshp.gsfc.nasa.gov Thu Jun 19 16:44:00 2003 From: yudong at lshp.gsfc.nasa.gov (Yudong Tian) Date: Thu, 19 Jun 2003 16:44:00 -0400 Subject: Network Booting In-Reply-To: <20030619090737.672f725f.torsten@howard.cc> Message-ID: My linux nodes use RealTek chips and I have been doing network installation and booting fine. The procedure is a little involved, and I do have some step-by-step notes here: http://lis.gsfc.nasa.gov/yudong/notes/net-install.txt I hope this can help with your Q1,2,3. ------------------------------------------------------------ Falun Dafa: The Tao of Meditation (http://www.falundafa.org) ------------------------------------------------------------ Yudong Tian, Ph.D. NASA/GSFC (301) 286-2275 > -----Original Message----- > From: beowulf-admin at scyld.com [mailto:beowulf-admin at scyld.com]On Behalf > Of torsten > Sent: Thursday, June 19, 2003 9:08 AM > To: beowulf at beowulf.org > Subject: Network Booting > > > Good Morning: > > I've been struggling for days to make a single computer be able to > network boot. I've read many documents, and I'm a little confused. > Here are some questions, and I am very grateful for assistance, in > advance. > > 1. I've seen references to bootp/dhcp, bootp or dhcp, bootp and dhcp, > dhcp over bootp - Which one do I need. Do I need both bootp and dhcp? > > 2. My Lab is on the University subnet. Will my bootp/dhcp server > conflict with the university system? Do I need to isolate my lab with > IP masq first? > > 3. On which broadcast/subnet does PXE send bootp requests. Does it make > a difference, with PXE, whether I use bootp or DHCP? > > 4. What are people using to setup/install partitions? I've seen bpbatch > and ka-tools. I'm interested in GPL software, primarily. > > 5. Are there any known problems with Realtek PXE ROMS? My network cards > are system integrated Realtek's. > > 6. Should I just download and setup the MOSIX stuff? > > 7. Does the Linux Terminal Server Project (LTSP) have anything useful > that I could/should be using? > > Thanks, Torsten > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org > To change your subscription (digest mode or unsubscribe) visit > http://www.beowulf.org/mailman/listinfo/beowulf _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From nordquist at geosci.uchicago.edu Thu Jun 19 16:35:49 2003 From: nordquist at geosci.uchicago.edu (Russell Nordquist) Date: Thu, 19 Jun 2003 15:35:49 -0500 (CDT) Subject: Network Booting In-Reply-To: <20030619144306.571533d6.torsten@howard.cc> Message-ID: rgb mentioned kickstart and FAI. Also worth looking into is systemimager. it works quite well for relatively homogeneous machines and is distro agnostic. It is quite easy to use this with dhcp/pxe to clone/update machines. http://www.systemimager.org russell On Thu, 19 Jun 2003 at 14:43, torsten wrote: > > >> 7. Does the Linux Terminal Server Project (LTSP) have anything useful > >> that I could/should be using? > > > >See answer to 6. Conceivably, but it depends on what you're trying to > >do. So what are you trying to do? > > > Thank you for the very complete answers. This solves a number of concerns > I have run into. > > I am trying setup a cluster to run MPICH. Some of the PC's are > headed workstations, the rest will be headless (for MPICH only). > > While we have manually (via CD-ROM) setup the workstations, I want to > automate the addition and maintenance of headless PC's. > > I am trying to use DHCP with PXE booting to boot the PC's. Any advice > toward this general direction, I am grateful for. > > Torsten > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org > To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf > - - - - - - - - - - - - Russell Nordquist UNIX Systems Administrator Geophysical Sciences Computing http://geosci.uchicago.edu/computing NSIT, University of Chicago - - - - - - - - - - - _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From hahn at physics.mcmaster.ca Thu Jun 19 18:40:46 2003 From: hahn at physics.mcmaster.ca (Mark Hahn) Date: Thu, 19 Jun 2003 18:40:46 -0400 (EDT) Subject: Network Booting In-Reply-To: <20030619162506.A19914@mikee.ath.cx> Message-ID: > > I am trying to use DHCP with PXE booting to boot the PC's. Any advice > > toward this general direction, I am grateful for. > > I have used LTSP to setup an ad-hoc cluster for PVM. It worked well. I looked at LTSP before building my current cluster, since I wanted nodes to be completely ephemeral (nothing installed on local disk.) but there's not really anything unique in LTSP that you need - in fact, my system is basically pure RH8, with minor tweaks to mount /dev/shm in various places, populate a /var in one, start sshd, etc. I'm about to add 10-12 dual-opterons to the existing 48 dual-xeons, and it looks like I could actually get away with changing only the kernel. (one line in tftpd or pxelinux config files.) I don't know yet how much trouble it is to run opterons in extended mode (64b, extra regs). seems like the main benefit is fast dram, which you get either way... _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From douglas at shore.net Thu Jun 19 23:52:23 2003 From: douglas at shore.net (Douglas O'Flaherty) Date: Thu, 19 Jun 2003 23:52:23 -0400 Subject: 95/F90 compilers Message-ID: <3EF284F7.9090304@shore.net> Art: I cite Polyhedron as a good general source for comparative data and articles. The web site is http://www.polyhedron.co.uk/ Pay attention to the version numbers when looking at comparisons. The PGI compiler in the test matrix is 3.2-4 and PGI is releasing version 5.0 at the end of this month. I wish I knew of other broad sources. You can also search for others who run similar applications as you do and see if they have comparative compiler data. Don't assume that a compiler that does one application well will also optimize a different application. Also, don't forget to look at Math Library performance. They make a difference. doug _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From baaden at smplinux.de Fri Jun 20 05:35:08 2003 From: baaden at smplinux.de (Marc Baaden) Date: Fri, 20 Jun 2003 11:35:08 +0200 Subject: [OT] Maximum performance on single processor ? Message-ID: <200306200935.LAA17759@apex.ibpc.fr> Hi, I am sorry if this query is slightly off-topic for the list. But I guess that many people here have also experiences with respect to my enquiry. We are looking for the highest performance machine on single (or very few, let's say 4) processors, because we need to run unparallelized and/or poorly parallelized code. Our budget is of roughly 20000 $, and a linux/beowulf platform would be ideal. The current top Intel x86 or AMD processors are too slow, and we do not have access to other architectures. Thank you very much in advance, Marc Baaden -- Dr. Marc Baaden - Institut de Biologie Physico-Chimique, Paris mailto:baaden at smplinux.de - http://www.marc-baaden.de FAX: +49 697912 39550 - Tel: +33 15841 5176 ou +33 609 843217 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From mikee at mikee.ath.cx Fri Jun 20 08:07:36 2003 From: mikee at mikee.ath.cx (Mike Eggleston) Date: Fri, 20 Jun 2003 07:07:36 -0500 Subject: [OT] Maximum performance on single processor ? In-Reply-To: <200306200935.LAA17759@apex.ibpc.fr>; from baaden@smplinux.de on Fri, Jun 20, 2003 at 11:35:08AM +0200 References: <200306200935.LAA17759@apex.ibpc.fr> Message-ID: <20030620070736.A23866@mikee.ath.cx> On Fri, 20 Jun 2003, Marc Baaden wrote: > > Hi, > > I am sorry if this query is slightly off-topic for the list. > But I guess that many people here have also experiences with > respect to my enquiry. > > We are looking for the highest performance machine on single > (or very few, let's say 4) processors, because we need to run > unparallelized and/or poorly parallelized code. > > Our budget is of roughly 20000 $, and a linux/beowulf platform > would be ideal. > > The current top Intel x86 or AMD processors are too slow, and we > do not have access to other architectures. Do you have the source to what you're running? I once read of a study that showed how an Apple II could out-perform a Cray... the Apple II ran something like a quicksort where the Cray ran a bubblesort. Mike _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From joelja at darkwing.uoregon.edu Fri Jun 20 06:15:35 2003 From: joelja at darkwing.uoregon.edu (Joel Jaeggli) Date: Fri, 20 Jun 2003 03:15:35 -0700 (PDT) Subject: [OT] Maximum performance on single processor ? In-Reply-To: <200306200935.LAA17759@apex.ibpc.fr> Message-ID: On Fri, 20 Jun 2003, Marc Baaden wrote: > > Hi, > > I am sorry if this query is slightly off-topic for the list. > But I guess that many people here have also experiences with > respect to my enquiry. > > We are looking for the highest performance machine on single > (or very few, let's say 4) processors, because we need to run > unparallelized and/or poorly parallelized code. > > Our budget is of roughly 20000 $, and a linux/beowulf platform > would be ideal. actually by defintion if your code is can't be parallelized then a cluster of off-the-shelf hardware probably isn't appropriate. > The current top Intel x86 or AMD processors are too slow, and we > do not have access to other architectures. Then you're screwed. > Thank you very much in advance, > Marc Baaden > > -- -------------------------------------------------------------------------- Joel Jaeggli Academic User Services joelja at darkwing.uoregon.edu -- PGP Key Fingerprint: 1DE9 8FCA 51FB 4195 B42A 9C32 A30D 121E -- In Dr. Johnson's famous dictionary patriotism is defined as the last resort of the scoundrel. With all due respect to an enlightened but inferior lexicographer I beg to submit that it is the first. -- Ambrose Bierce, "The Devil's Dictionary" _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From baaden at smplinux.de Fri Jun 20 08:44:05 2003 From: baaden at smplinux.de (Marc Baaden) Date: Fri, 20 Jun 2003 14:44:05 +0200 Subject: [OT] Maximum performance on single processor ? In-Reply-To: Your message of "Fri, 20 Jun 2003 07:06:25 EDT." Message-ID: <200306201244.OAA22202@apex.ibpc.fr> Thanks for all your comments and suggestions. I realize (and some of you mentioned it) that you need more information to understand better what I am looking for. I have an existing application which is part of a project. I have the source code. It is Fortran. It *can* be parallelized, but we would rather spend our time on the other parts of the project which need to be written from scratch *first*. The application is to run in real time, that is the user does something and as a function of user input and the calculation with the fortran program that I described, there is a correponding feedback to the user on the screen (and in some Virtual Reality equipment). Right now, even on simple test cases, the "response time" (eg calculation time for a single step) of our program is on the order of the second. (this is for an athlon MP 2600+) We need to get that down to a fraction of seconds, best milli-seconds, in order to be usable in real time. (makes it a factor of roughly 1000) As I said the code can indeed be parallelized - maybe even simply cleaned up in some parts - but unfortunately there remains very much other important stuff to do. So we'd rather spend some money on a really fast CPU and not touch the code at the moment. So my question was more, what is the fastest CPU I can get for $20000 at the moment (without explicitly parallelizing, hyperthreading or vectorizing my code). Cheers, Marc Baaden -- Dr. Marc Baaden - Institut de Biologie Physico-Chimique, Paris mailto:baaden at smplinux.de - http://www.marc-baaden.de FAX: +49 697912 39550 - Tel: +33 15841 5176 ou +33 609 843217 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From jhearns at freesolutions.net Fri Jun 20 07:03:16 2003 From: jhearns at freesolutions.net (John Hearns) Date: Fri, 20 Jun 2003 12:03:16 +0100 Subject: [OT] Maximum performance on single processor ? References: <200306200935.LAA17759@apex.ibpc.fr> Message-ID: <019601c3371b$901cd840$8461cdc2@DREAD> > > I am sorry if this query is slightly off-topic for the list. > But I guess that many people here have also experiences with > respect to my enquiry. > > We are looking for the highest performance machine on single > (or very few, let's say 4) processors, because we need to run > unparallelized and/or poorly parallelized code. > > Our budget is of roughly 20000 $, and a linux/beowulf platform > would be ideal. > > The current top Intel x86 or AMD processors are too slow, and we > do not have access to other architectures. > Being mischevous, $20000 would probably buy you the cabinet for a http://www.sgi.com/servers/altix/ Seriously though, people on the list will ask for more information on your application. You say 'too slow' - exaclty what do you mean by this? If it is (say) an application like gene sequence matching, which takes a long time to match against databases, then you may be unable to speed up individual runs. But what you CAN do, and what will probably make your researchers happy, is to have a high throughput - so you get good utilisation of your machines, and results coming through the pipeline. The list really needs some specifics though - and some timings on the processors you have tried. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rgb at phy.duke.edu Fri Jun 20 07:06:25 2003 From: rgb at phy.duke.edu (Robert G. Brown) Date: Fri, 20 Jun 2003 07:06:25 -0400 (EDT) Subject: [OT] Maximum performance on single processor ? In-Reply-To: Message-ID: On Fri, 20 Jun 2003, Joel Jaeggli wrote: > > Our budget is of roughly 20000 $, and a linux/beowulf platform > > would be ideal. > > actually by defintion if your code is can't be parallelized then a cluster > of off-the-shelf hardware probably isn't appropriate. > > > The current top Intel x86 or AMD processors are too slow, and we > > do not have access to other architectures. > > Then you're screwed. Let me be a TEENY bit more optimistic than Joel. The one glimmer of hope I can hold out is that if your code is single threaded by highly vectorized, you might look at some exotic solutions (and some less exotic solutions) involving a vector co-processor, either integrated with the main CPU or as a programmable add-on. Some of the vector processors have extremely high floating point rates for the limited instruction set and data manipulations they are capable of. Three slightly off-the-beaten path solutions you might look into are: a) Power PC. Apple is actually pushing them for clusters these days (I just attended a talk where they were touted), and the upper end ones have both an L3 cache layer and an embedded Altivec vector unit. Although the CPU itself is around 1 GHz and hence "slow" in terms of cycles, the vector unit has some extremely high overall FLOPS rate for the right kind of code. I know nothing of the level of difficulty of programming for it, but Apple supports it with compilers and so forth. b) http://arrakis.ncsa.uiuc.edu/ps2/cluster.php. OK, yes, it's sort of a toy, sort of a joke, but there is at least one Sony Playstation 2 cluster out there. Why, you ask? It runs linux (the "emotion engine" core is a MIPS CPU) and it has two attached vector DSP's. Programming it so that it uses the VU's is PITA, and it may not be what you need, but you want exotica, and this is it. c) NEC has a strange one it does: http://www.silicon.com/news/500009/1/3016.html I know next to nothing about it, but maybe you can cut a deal with them to try it out. May or may not have linux support yet (but they might give you a couple if you were to offer to do the port:-). Again, 8 GFLOP vector processors. d) Vector units are available in a lot of contexts for Linux systems, few of them (alas) straightforward. DSP's, high end video cards. In most cases these units have staggering peak FLOPS (more then 10 GFLOPS in some cases IIRC) but in all cases programming them and optimizing to achieve a reasonable fraction of peak in real world throughput will be, um, a "challenge" unless your problem is shaped just right. But it might be, so give these a look. Otherwise, as Joel says, you're pretty much screwed, especially with a budget of only $20K. This assumes that you've already looked at Intel Itanium and AMD Opteron solutions, and already know about the SSE2 instructions available to those CPUs. In many cases, throughput is related to memory bandwidth in the limit, and these units are designed with fast wide memory. Make sure that you understand your application very well indeed and that it really isn't parallelizable. If it is vectorizable you may have hope. If it isn't vectorizable OR parallelizable then the best you will do anywhere is the fastest CPU du jour... however inadequate it may be. rgb -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb at phy.duke.edu _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From jhearns at freesolutions.net Fri Jun 20 09:01:45 2003 From: jhearns at freesolutions.net (John Hearns) Date: Fri, 20 Jun 2003 14:01:45 +0100 Subject: [OT] Maximum performance on single processor ? References: <200306201244.OAA22202@apex.ibpc.fr> Message-ID: <026701c3372c$1eb39480$8461cdc2@DREAD> > > Right now, even on simple test cases, the "response time" (eg calculation > time for a single step) of our program is on the order of the second. > (this is for an athlon MP 2600+) > We need to get that down to a fraction of seconds, best milli-seconds, > in order to be usable in real time. (makes it a factor of roughly 1000) > > As I said the code can indeed be parallelized - maybe even simply cleaned > up in some parts - but unfortunately there remains very much other important > stuff to do. So we'd rather spend some money on a really fast CPU and not > touch the code at the moment. I disagree. You really must do some code profiling. You may find some real big bottleneck, which is either unintentional or not necessary. Also, you should consider how the code is written - if you are manipulating large arrays then the order of access, and the stride length can make a big difference. You may be ranging all over a huge array, pulling things in from memory(*) when a different algorithm/different loop arrangement could keep things in cache. And Fortran, being 'closer to the machine' can expose these things (I may be wrong). Have you looked at books like: "High Performance Computing" http://www.oreilly.com/catalog/hpc2/ (*) Sorry to teach my granny to suck eggs, but your machine isn't going into swap I trust. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From landman at scalableinformatics.com Fri Jun 20 09:11:27 2003 From: landman at scalableinformatics.com (Joe Landman) Date: Fri, 20 Jun 2003 09:11:27 -0400 Subject: [OT] Maximum performance on single processor ? In-Reply-To: <200306201244.OAA22202@apex.ibpc.fr> References: <200306201244.OAA22202@apex.ibpc.fr> Message-ID: <3EF307FF.6010409@scalableinformatics.com> Marc Baaden wrote: > Right now, even on simple test cases, the "response time" (eg calculation > time for a single step) of our program is on the order of the second. > (this is for an athlon MP 2600+) > We need to get that down to a fraction of seconds, best milli-seconds, > in order to be usable in real time. (makes it a factor of roughly 1000) You are not going to get a 3 order of magnitude increase in performance by switching CPUs. You won't get an order of magnitude over the Athlon. > So my question was more, what is the fastest CPU I can get for $20000 > at the moment (without explicitly parallelizing, hyperthreading or > vectorizing my code). Before you pursue this any more, lets try to reset expectations to be more in line with what would need to occur. 1) the CPUs are generally within single digit factors of performance on a given code, and this is as much dominated by memory speed and/or latency for various codes, as it could be by CPU "speed" 2) As it does not appear that you have profiled the code (just a guess), I would strongly urge you do that as your next step. This will tell you (if you chose not to optimize your code) where the code is spending most of its time. If this is in fetching data from memory then a "faster" CPU will not help, as the CPU is not the bottleneck. 3) to get multiple orders of magnitude change in performance, you will need at minimum an algorithm shift. The algorithms used in your code now are taking time, and you need to know how much time. It would be terrible if your code takes 1/2 second to set up the calculation in the first place, and another 1/2 second to perform it. In this case, the first half second would impede any faster algorithm. If you comment out the computation, how long does the setup/teardown take for the code? Though you don't want to work on this code, I am not sure you will be able to get more than some percentage points with a new CPU without at minimum a profile of the code, and a hard look at what it is doing, and how it is doing it. > > > Cheers, > Marc Baaden > -- Joseph Landman, Ph.D Scalable Informatics LLC, email: landman at scalableinformatics.com web : http://scalableinformatics.com phone: +1 734 612 4615 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From seth at hogg.org Fri Jun 20 09:15:47 2003 From: seth at hogg.org (Simon Hogg) Date: Fri, 20 Jun 2003 14:15:47 +0100 Subject: [OT] Maximum performance on single processor ? In-Reply-To: <200306201244.OAA22202@apex.ibpc.fr> References: Message-ID: <4.3.2.7.2.20030620140207.00ae23a0@pop.freeuk.net> At 14:44 20/06/03 +0200, Marc Baaden wrote: >I have an existing application which is part of a project. I have >the source code. It is Fortran. It *can* be parallelized, but we >would rather spend our time on the other parts of the project >which need to be written from scratch *first*. > >The application is to run in real time, that is the user does something >and as a function of user input and the calculation with the fortran >program that I described, there is a correponding feedback to the >user on the screen (and in some Virtual Reality equipment). > >Right now, even on simple test cases, the "response time" (eg calculation >time for a single step) of our program is on the order of the second. >(this is for an athlon MP 2600+) >We need to get that down to a fraction of seconds, best milli-seconds, >in order to be usable in real time. (makes it a factor of roughly 1000) > >As I said the code can indeed be parallelized - maybe even simply cleaned >up in some parts - but unfortunately there remains very much other important >stuff to do. So we'd rather spend some money on a really fast CPU and not >touch the code at the moment. > >So my question was more, what is the fastest CPU I can get for $20000 >at the moment (without explicitly parallelizing, hyperthreading or >vectorizing my code). I'm sure some other people will give 'better' answers, but from having a look at your web pages, I would be tempted to go down the route of second-hand SGI equipment. For example (and no, I don't know how the performance stacks up, I'm looking partly at a general bio-informatics / SGI link if that makes sense) I can see for sale an Origin 2000 Quad 500MHz / 4GB RAM for UKP 15,725. Just another suggestion to add to the pile. -- Simon _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From hahn at physics.mcmaster.ca Fri Jun 20 10:29:57 2003 From: hahn at physics.mcmaster.ca (Mark Hahn) Date: Fri, 20 Jun 2003 10:29:57 -0400 (EDT) Subject: [OT] Maximum performance on single processor ? In-Reply-To: Message-ID: > Although the CPU itself is around 1 GHz and hence "slow" in terms of > cycles, the vector unit has some extremely high overall FLOPS rate for > the right kind of code. where "right kind of code" means "exquisitely cache-friendly". I suspect that some speed-hacked fortran is going to be very much not like that. as previous posts pointed out, unless the original code is remarkably stupid, hardware is simply not going to give you 1000x. finding a smart kid and setting him up with a $5k P4/i875 seems a lot more plausible to me. the real lateral-thinking approach is something like fpga's, but if you can't be bothered to sanity-check the original code, fpga's are out of the question... _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From jhearns at freesolutions.net Fri Jun 20 11:04:22 2003 From: jhearns at freesolutions.net (John Hearns) Date: Fri, 20 Jun 2003 16:04:22 +0100 Subject: [OT] Maximum performance on single processor ? References: Message-ID: <031901c3373d$3ea46600$8461cdc2@DREAD> My advice: Go find your local systems guru. This person is usually to be found in a dimly lit basement office. You can recognise it by the 'Nicht Fur Lookenpeepers' notices and the 1972 lineprinter ASCII art Snoopy calendar on the wall. The person will have disguised a spiffy top of the range monitor with carefully chosen junk. On that monitor, he/she will have many windows open. One will be tailing your mail file (joke). Bring gifts of beer. As him/her for advice: "Please help me to get my program running faster". _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From mikee at mikee.ath.cx Fri Jun 20 10:57:32 2003 From: mikee at mikee.ath.cx (Mike Eggleston) Date: Fri, 20 Jun 2003 09:57:32 -0500 Subject: [OT] Maximum performance on single processor ? In-Reply-To: <200306201244.OAA22202@apex.ibpc.fr>; from baaden@smplinux.de on Fri, Jun 20, 2003 at 02:44:05PM +0200 References: <200306201244.OAA22202@apex.ibpc.fr> Message-ID: <20030620095732.D23866@mikee.ath.cx> On Fri, 20 Jun 2003, Marc Baaden wrote: > > Thanks for all your comments and suggestions. I realize (and some > of you mentioned it) that you need more information to understand > better what I am looking for. > > I have an existing application which is part of a project. I have > the source code. It is Fortran. It *can* be parallelized, but we > would rather spend our time on the other parts of the project > which need to be written from scratch *first*. > > The application is to run in real time, that is the user does something > and as a function of user input and the calculation with the fortran > program that I described, there is a correponding feedback to the > user on the screen (and in some Virtual Reality equipment). > > Right now, even on simple test cases, the "response time" (eg calculation > time for a single step) of our program is on the order of the second. > (this is for an athlon MP 2600+) > We need to get that down to a fraction of seconds, best milli-seconds, > in order to be usable in real time. (makes it a factor of roughly 1000) > > As I said the code can indeed be parallelized - maybe even simply cleaned > up in some parts - but unfortunately there remains very much other important > stuff to do. So we'd rather spend some money on a really fast CPU and not > touch the code at the moment. > > So my question was more, what is the fastest CPU I can get for $20000 > at the moment (without explicitly parallelizing, hyperthreading or > vectorizing my code). Can the lisp technique of memoization help with your processing? Mike _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rmyers1400 at attbi.com Fri Jun 20 11:16:37 2003 From: rmyers1400 at attbi.com (Robert Myers) Date: Fri, 20 Jun 2003 11:16:37 -0400 Subject: [OT] Maximum performance on single processor ? In-Reply-To: <200306201244.OAA22202@apex.ibpc.fr> References: <200306201244.OAA22202@apex.ibpc.fr> Message-ID: <3EF32555.7040006@attbi.com> Marc Baaden wrote: >As I said the code can indeed be parallelized - maybe even simply cleaned >up in some parts - but unfortunately there remains very much other important >stuff to do. So we'd rather spend some money on a really fast CPU and not >touch the code at the moment. > >So my question was more, what is the fastest CPU I can get for $20000 >at the moment (without explicitly parallelizing, hyperthreading or >vectorizing my code). > > You may have done better service to the readers of this list than you have to yourself, since the readers of this list are naturally inclined to think about the sort of question you have asked, and the answers have been interesting to read. The fact that you are asking the question, though, begs an answer that others have partially given: no plausible hardware purchase you can make will be a substitute for you or someone you are working with to get smart about being clever with hardware and software. You don't want to invest in people, but people is what you must invest in. There is probably someone out there really hungry for work or just plain interested who could get you an awful lot of mileage for the money you want to spend. Without that sort of expertise, the chances of your getting much further than you already are through a hardware purchase are, to a very good approximation, zero. The application you describe sounds like something that might wind up in an ASIC or FPGA, but that involves serious people time. The only no-brains speedup strategy that could work is the most powerful speed-up strategy of all: wait a couple of years for off-the-shelf hardware to get faster. RM _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From Daniel.Kidger at quadrics.com Fri Jun 20 10:02:44 2003 From: Daniel.Kidger at quadrics.com (Daniel Kidger) Date: Fri, 20 Jun 2003 15:02:44 +0100 Subject: cluster of AMD Opteron Message-ID: <010C86D15E4D1247B9A5DD312B7F5AA78DDF91@stegosaurus.bristol.quadrics.com> From: Mikhail Kuzminsky [mailto:kus at free.net] >> Is somebody who is using AMD opterons yet ? > We tested 2-way SMP server based on RioWorks mobo. But I should >not recommend this motherboard for using: by default it has no >monitoring (temperature etc) chips on the board, it's necessary >to buy special additional card ! Unfortunately as a result I don't >have data about lm_sensors work. Moreover, the choice of SMP >boards is very restricted now: Tyan S2880 and MSI K8D. We have a small cluster of Newisys 2100 dual-Opteron 1U-nodes. our NDA prevents me from going into much praise, but I can say that they *do* have execellent environemental monitoring of just about everythinbg on the Mobo. You can even get a live output of the current power consumption :-) (which of course relates to another long running thread on the beowulf list). Yours, Daniel. -------------------------------------------------------------- Dr. Dan Kidger, Quadrics Ltd. daniel.kidger at quadrics.com One Bridewell St., Bristol, BS1 2AA, UK 0117 915 5505 ----------------------- www.quadrics.com -------------------- _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From maurice at harddata.com Fri Jun 20 12:09:54 2003 From: maurice at harddata.com (Maurice Hilarius) Date: Fri, 20 Jun 2003 10:09:54 -0600 Subject: [OT] Maximum performance on single processor ? In-Reply-To: <200306201502.h5KF2EU21424@NewBlue.Scyld.com> Message-ID: <5.1.1.6.2.20030620100637.03ca3450@mail.harddata.com> With regards to your message: >From: Marc Baaden <<>> >So my question was more, what is the fastest CPU I can get for $20000 >at the moment (without explicitly parallelizing, hyperthreading or >vectorizing my code). I believe the answer is an Opteron. Of course that much money will buy quite a lot of CPUs! With the memory bandwidth, floating point engine, and Hypertransport links on these, they are VERY fast for this type of thing. If you want access to a machine to run some code samples on, please contact me off-list and we can arrange some time for you on a machine. We have test/demo dual Opteron servers in place, available over SSH connections, for exactly this purpose. They are set up with compilers, libraries and tools suitable for this use. Thanks! With our best regards, Maurice W. Hilarius Telephone: 01-780-456-9771 Hard Data Ltd. FAX: 01-780-456-9772 11060 - 166 Avenue mailto:maurice at harddata.com Edmonton, AB, Canada http://www.harddata.com/ T5X 1Y3 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From jcmoore at atipa.com Fri Jun 20 12:24:20 2003 From: jcmoore at atipa.com (Curt Moore) Date: 20 Jun 2003 11:24:20 -0500 Subject: cluster of AOD Opteron In-Reply-To: <200306201342.RAA28782@nocserv.free.net> References: <200306201342.RAA28782@nocserv.free.net> Message-ID: <1056126260.9688.15.camel@picard.lab.atipa.com> The RioWorks HDAMA (Arima) motherboard does have on-board sensors, adm1026 based. We've recently deployed a 100 node, 200 CPU, cluster of these systems and all works very well. Arima does have planned both a mini BMC which does just management type functions and also a full BMC with will do other neat things, I believe, such as KVM over LAN. Below is a lm_sensors dump from an Arima HDAMA. adm1026-i2c-0-2c Adapter: SMBus AMD8111 adapter at 80e0 Algorithm: Non-I2C SMBus adapter in0: +1.15 V (min = +0.00 V, max = +2.99 V) in1: +1.59 V (min = +0.00 V, max = +2.99 V) in2: +1.57 V (min = +0.00 V, max = +2.99 V) in3: +1.19 V (min = +0.00 V, max = +2.99 V) in4: +1.18 V (min = +0.00 V, max = +2.99 V) in5: +1.14 V (min = +0.00 V, max = +2.99 V) in6: +1.24 V (min = +0.00 V, max = +2.49 V) in7: +1.59 V (min = +0.00 V, max = +2.49 V) in8: +0.00 V (min = +0.00 V, max = +2.49 V) in9: +0.45 V (min = +1.25 V, max = +0.98 V) in10: +2.70 V (min = +0.00 V, max = +3.98 V) in11: +3.33 V (min = +0.00 V, max = +4.42 V) in12: +3.38 V (min = +0.00 V, max = +4.42 V) in13: +5.12 V (min = +0.00 V, max = +6.63 V) in14: +1.57 V (min = +0.00 V, max = +2.99 V) in15: +11.88 V (min = +0.00 V, max = +15.94 V) in16: -12.03 V (min = +2.43 V, max = -16.00 V) fan0: 0 RPM (min = 0 RPM, div = 2) fan1: 0 RPM (min = 0 RPM, div = 2) fan2: 0 RPM (min = 0 RPM, div = 2) fan3: 0 RPM (min = 0 RPM, div = 2) fan4: 0 RPM (min = 0 RPM, div = 1) fan5: 0 RPM (min = 0 RPM, div = 1) fan6: -1 RPM (min = 0 RPM, div = 1) fan7: -1 RPM (min = 0 RPM, div = 1) temp1: +37?C (min = -128?C, max = +80?C) temp2: +46?C (min = -128?C, max = +100?C) temp3: +46?C (min = -128?C, max = +100?C) vid: +1.850 V (VRM Version 9.1) -Curt ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Curt Moore Systems Integration Engineer At?pa Technologies jcmoore at atipa.com (O) 785-813-0312 (Fax) 785-841-1809 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From alangrimes at starpower.net Fri Jun 20 15:26:01 2003 From: alangrimes at starpower.net (Alan Grimes) Date: Fri, 20 Jun 2003 12:26:01 -0700 Subject: [OT] Maximum performance on single processor ? References: <031901c3373d$3ea46600$8461cdc2@DREAD> Message-ID: <3EF35FC9.3084A9FF@starpower.net> om >From what little I've heard my advice for this rig is: 1. The athlon MP is limited by its 266mhz FSB... For truly single-threaded code, get an Athlon 3200 and a dual channel DDR board... This should earn you roughly a 1.6x speedup over your existing rig... I, myself, am using only 1.2ghz chips in my dual athlon because of the limited system bandwidth.. 2. The current benchmark king is the P4 3.0 ghz/ 4x200mhz FSB. You can profit from this even more if you have two threads going... (this would be roughly 2x over the MP you mentioned..)... 3. It still isn't clear what your simulation is (I might have missed it) but it _MIGHT_ be profitable to seperate out your front end.. If your front end requires quite a bit more than what can be farmed out to an OpenGL 1.3+ gfx accelerator, an the bandwidth to your engine is reasonable, consider putting your front end on the athlon mentioned above and your back-end on the P4 and link them over GBE... Then make sure there is as little overhead as possible on the P4... Take off all unnecessary software... Depending on how much CPU your front end takes and how well you can make your network work for you, this could earn you as much as a 2x speedup over the 1.6x above... -- Testimonial: I wasted 2,000 hours using Linux. http://users.rcn.com/alangrimes/ _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rgb at phy.duke.edu Fri Jun 20 12:32:10 2003 From: rgb at phy.duke.edu (Robert G. Brown) Date: Fri, 20 Jun 2003 12:32:10 -0400 (EDT) Subject: [OT] Maximum performance on single processor ? In-Reply-To: <200306201244.OAA22202@apex.ibpc.fr> Message-ID: On Fri, 20 Jun 2003, Marc Baaden wrote: > > Thanks for all your comments and suggestions. I realize (and some > of you mentioned it) that you need more information to understand > better what I am looking for. > > I have an existing application which is part of a project. I have > the source code. It is Fortran. It *can* be parallelized, but we > would rather spend our time on the other parts of the project > which need to be written from scratch *first*. > > The application is to run in real time, that is the user does something > and as a function of user input and the calculation with the fortran > program that I described, there is a correponding feedback to the > user on the screen (and in some Virtual Reality equipment). > > Right now, even on simple test cases, the "response time" (eg calculation > time for a single step) of our program is on the order of the second. > (this is for an athlon MP 2600+) > We need to get that down to a fraction of seconds, best milli-seconds, > in order to be usable in real time. (makes it a factor of roughly 1000) > > As I said the code can indeed be parallelized - maybe even simply cleaned > up in some parts - but unfortunately there remains very much other important > stuff to do. So we'd rather spend some money on a really fast CPU and not > touch the code at the moment. > > So my question was more, what is the fastest CPU I can get for $20000 > at the moment (without explicitly parallelizing, hyperthreading or > vectorizing my code). Ain't happenin'. The factor of 1000, I mean. Not no way, not no how, not without completely rewriting your code. >From this description, the ONLY thing you should be working on is rewriting the code from scratch, doing a serious job of algorithm analysis, optimization, vectorization, parallelization, and (IMHO) not in fortran. If it is graphics/visualization, consider studying things like computer games, render farms, openGL, and various libraries and hardware devices that can move the work required into custom processors. It could be that you are using Fortran to do various data transformations in poorly written software that are now done extremely rapidly in highly specialized hardware (not even on a "CPU" -- on an offboard coprocessor with its own memory and instruction set). Or not. However, you're not getting a factor of 1000 relative to an Athlon 2600 on any hardware in existence for $20K. I only wish...:-) rgb > > > Cheers, > Marc Baaden > > -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb at phy.duke.edu _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From Daniel.Kidger at quadrics.com Fri Jun 20 13:04:39 2003 From: Daniel.Kidger at quadrics.com (Daniel Kidger) Date: Fri, 20 Jun 2003 18:04:39 +0100 Subject: [OT] Maximum performance on single processor ? Message-ID: <010C86D15E4D1247B9A5DD312B7F5AA78DDF95@stegosaurus.bristol.quadrics.com> -----Original Message----- > I have an existing application which is part of a project. I have > the source code. It is Fortran. It *can* be parallelized, but we > would rather spend our time on the other parts of the project > which need to be written from scratch *first*. Fortran is not a problem - for numerical work it should be able to beat C 95% of the time IMHO > The application is to run in real time, that is the user does something > and as a function of user input and the calculation with the fortran > program that I described, there is a correponding feedback to the > user on the screen (and in some Virtual Reality equipment). A few years ago, I was involved in a project that involved doing Finite Elements within a VR environment. It can be done, but you must look closely at your application code to see *why* it is taking over 1000 milliseconds per step: Do you know which bit of code takes the most time? Can you save small arrays and such like to avoid recalculation? How are you getting new data to/from the Fortran - reading and writing files is easy to implement but far too slow for VR. Are you using the appropriate matrix maths libraries for your platform (eg mkl, atlas, libgoto, etc.)? I hope that you don't have any matrix multiplies as 3 nested Do-loops! and even.. Can you use any spare CPUs in a cluster to speculatively calculate parts of the next timestep(s) even if sometimes you have to throw these results away? Yours, Daniel. -------------------------------------------------------------- Dr. Dan Kidger, Quadrics Ltd. daniel.kidger at quadrics.com One Bridewell St., Bristol, BS1 2AA, UK 0117 915 5505 ----------------------- www.quadrics.com -------------------- _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rgb at phy.duke.edu Fri Jun 20 12:51:04 2003 From: rgb at phy.duke.edu (Robert G. Brown) Date: Fri, 20 Jun 2003 12:51:04 -0400 (EDT) Subject: [OT] Maximum performance on single processor ? In-Reply-To: <3EF32555.7040006@attbi.com> Message-ID: On Fri, 20 Jun 2003, Robert Myers wrote: > The only no-brains speedup strategy that could work is the most powerful > speed-up strategy of all: wait a couple of years for off-the-shelf > hardware to get faster. Hmm, Moore's law assuming a very optimistic annual doubling requires log_2(1024) = 10 years to get a kilospeedup. Your conclusions remains unchallenged -- this problem will only resolve with the investment of serious human energy redoing the application -- maybe. Or a (literal) "quantum leap" -- if quantum computers mature sooner than ten years. Assuming that QC's don't require novel, intrinsically parallel programming in their own right and won't just run off-the-shelf serial fortran all that fast. rgb > > RM > > > > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org > To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf > -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb at phy.duke.edu _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From abijur at craft-tech.com Fri Jun 20 13:07:54 2003 From: abijur at craft-tech.com (Ashwin Bijur) Date: Fri, 20 Jun 2003 13:07:54 -0400 Subject: diskless booting & channel bonding Message-ID: <3EF33F6A.60309@craft-tech.com> Can anyone point me to a resource that would give general (or step-by-step) instructions on booting a diskless cluster with channel bonding? Thanks, Ashwin Bijur. abijur at craft-tech.com _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From mikee at mikee.ath.cx Fri Jun 20 13:16:59 2003 From: mikee at mikee.ath.cx (Mike Eggleston) Date: Fri, 20 Jun 2003 12:16:59 -0500 Subject: [OT] Maximum performance on single processor ? In-Reply-To: <031901c3373d$3ea46600$8461cdc2@DREAD>; from jhearns@freesolutions.net on Fri, Jun 20, 2003 at 04:04:22PM +0100 References: <031901c3373d$3ea46600$8461cdc2@DREAD> Message-ID: <20030620121659.H23866@mikee.ath.cx> On Fri, 20 Jun 2003, John Hearns wrote: > My advice: > > Go find your local systems guru. > This person is usually to be found in a dimly lit basement office. You can > recognise it by the > 'Nicht Fur Lookenpeepers' notices and the 1972 lineprinter ASCII art Snoopy > calendar on the > wall. > The person will have disguised a spiffy top of the range monitor with > carefully chosen junk. > On that monitor, he/she will have many windows open. One will be tailing > your mail file (joke). > > Bring gifts of beer. > As him/her for advice: "Please help me to get my program running faster". I resemble that remark! Mike _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From baaden at smplinux.de Fri Jun 20 13:43:27 2003 From: baaden at smplinux.de (Marc Baaden) Date: Fri, 20 Jun 2003 19:43:27 +0200 Subject: [OT] Maximum performance on single processor ? In-Reply-To: Your message of "Fri, 20 Jun 2003 12:51:04 EDT." Message-ID: <200306201743.TAA27894@apex.ibpc.fr> Hi again, first thanks for all the helpful (though a bit discouraging) comments. Firstly I should probably say that the factor of 1000 was what we'd ideally need to be absolutely sure about the feasability. Maybe a factor 10-20 is sufficient, maybe less. Then, just out of pure curiosity, I'd still like to know what kind of speedup by investment I could get for 20k$, let's say eg with respect to the Athlon MB 2600 and which are currently the fastest processors in this pricerange. I think someone "estimated" a factor of 4-5 for a Power4+, is that right ? landman at scalableinformatics.com said: >> As it does not appear that you have profiled the code (just a guess), Half-right half-wrong guess. I have very rapidly profiled the code, just confirming what I already guessed, that the bottleneck (> 80% of the time) is the energy calculation subroutine, which calculates pairwise interactions between atoms in the system. There are indeed potential speedup factors in this code, like - introduction of a neighbour list - parallelization .. or maybe the use of a specialized board (MD-GRAPE) for just these calculations. Unfortunately I currently do lack the competence for parallelization or other optimizations (probably trying to use SSE2 code would be even better), and part of the more specialized methods seems like a complicated and particularly short-lived business. hahn at physics.mcmaster.ca said: >> do you mean you want uniprocessors? you also need to characterize the >> code MUCH better in order to choose a machine. for instance, power* >> and itanium chips are both very fast for some kinds of FP code, but >> miserable for integer-dominated code. We do use floating-point code. hahn at physics.mcmaster.ca said: >> have you actually done paper calculations to see whether the >> performance you observe is sensible/expected? ie, that you're not >> triggering some pathological behavior, even bugs? Yes. The code is tested (> 10 years). But we know it has not been developed with speed in mind (my contest for the understatement of the year) >> have you done the >> high-resolution measuring to know whether your user inputs are >> actually arriving without being blocked (for instance, on 20ms >> scheduler ticks)? I have measured the calculation separately for now. Not even doing the user part. But I do have an alternative program which already does run. So I know that the method in itself is working. It is just the calculation module which is in Fortran that is not yet adapted. hahn at physics.mcmaster.ca said: >> an athlon MP 2600+) >> a quite poor architecture for today; it was competitive a year ago. oops, we must have been very ill advised then, because the machine is just a month old, and was the "top" model from our vendor. Or is there such a gap between the US and the European market ? mikee at mikee.ath.cx said: >> Can the lisp technique of memoization help with your processing? Sorry, I don't know about this. What is it about ? beowulf-request at scyld.com said: >> Also, you should consider how the code is written - if you are >> manipulating large arrays then the order of access, and the stride >> length can make a big difference. You may be ranging all over a huge >> array, pulling things in from memory(*) when a different algorithm/ >> different loop arrangement could keep things in cache. So - if I interpret your and other people's statements right - depending on the code, memory might be as important/critical as CPU power ? Is there a way to check on what the code depends most ? >> And Fortran, >> being 'closer to the machine' can expose these things (I may be >> wrong). I don't see how (but I am not very experienced in this), any more direct ideas ? >> Have you looked at books like: "High Performance Computing" >> http://www.oreilly.com/catalog/hpc2/ No, not yet. I will definitely need some guidance into how to best optimize the code. beowulf-request at scyld.com said: >> (*) Sorry to teach my granny to suck eggs, but your machine isn't >> going into swap I trust. I was about to say no it is not .. but I shall give it a serious check. johnb at quadrics.com said: >> If it looks like it would be a major project, you could get a summer >> student to get on with doing it whilst the rest of the folks are doing >> the other stuff. It is (going to be) a major project. I would love having a summer student doing the job, but I certainly haven't been able to get hold of someone with the corresponding competences. (Also, in general the people I met who might be competent, did not program in Fortran :)) ) rgb at phy.duke.edu said: >> From this description, the ONLY thing you should be working on is >> rewriting the code from scratch, doing a serious job of algorithm >> analysis, optimization, vectorization, parallelization, and (IMHO) not >> in fortran. >> If it is graphics/visualization, consider studying things like >> computer games, render farms, openGL, and various libraries and >> hardware devices that can move the work required into custom >> processors. It could be that you are using Fortran to do various data >> transformations in poorly written software that are now done extremely >> rapidly in highly specialized hardware (not even on a "CPU" -- on an >> offboard coprocessor with its own memory and instruction set). Or >> not. These are all excellent remarks. But I do totally lack competence or knowledge in these fields, and it seems extremely difficult/dangerous to me to make the right choices in this context or to get an overview of what the best ways are to optimize the code/algorithm. Also I do not (yet) know the fortran code very well, as I am not developing/ maintaining it, and this does not really help in making those choices. Well, if you are still with me, then thanks for reading through all this. Cheers, Marc Baaden -- Dr. Marc Baaden - Institut de Biologie Physico-Chimique, Paris mailto:baaden at smplinux.de - http://www.marc-baaden.de FAX: +49 697912 39550 - Tel: +33 15841 5176 ou +33 609 843217 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From ctierney at hpti.com Fri Jun 20 13:43:39 2003 From: ctierney at hpti.com (Craig Tierney) Date: 20 Jun 2003 11:43:39 -0600 Subject: [OT] Maximum performance on single processor ? In-Reply-To: References: Message-ID: <1056131019.11801.1493.camel@woody> stuff deleted.... > Ain't happenin'. The factor of 1000, I mean. Not no way, not no how, > not without completely rewriting your code. > > >From this description, the ONLY thing you should be working on is > rewriting the code from scratch, doing a serious job of algorithm > analysis, optimization, vectorization, parallelization, and (IMHO) not > in fortran. If someone else had said this I would probably have had some snide comment and caused some sort of flame war. However after noticing the comment came from a helpful and knowledgeable list member I will have to hold back. However, I really want to know why 'not in fortran'? Not everyone is a programming god and can visualise how a particular piece of code is going to end up in assembler. You can get good performance out of C and C++ but you have to be careful, and I still don't know if you can beat Fortran in many cases. In a nutshell, without starting a flamewar (too late), what's wrong with Fortran for high performance numerical codes? PS. I still hate Fortran, never liked it. However I have found that it does the job. Craig > > If it is graphics/visualization, consider studying things like computer > games, render farms, openGL, and various libraries and hardware devices > that can move the work required into custom processors. It could be > that you are using Fortran to do various data transformations in poorly > written software that are now done extremely rapidly in highly > specialized hardware (not even on a "CPU" -- on an offboard coprocessor > with its own memory and instruction set). Or not. > > However, you're not getting a factor of 1000 relative to an Athlon 2600 > on any hardware in existence for $20K. I only wish...:-) > > rgb > > > > > > > Cheers, > > Marc Baaden > > > > -- Craig Tierney _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From waitt at saic.com Fri Jun 20 14:02:10 2003 From: waitt at saic.com (Tim Wait) Date: Fri, 20 Jun 2003 14:02:10 -0400 Subject: [OT] Maximum performance on single processor ? References: Message-ID: <3EF34C22.6090301@saic.com> As others have suggested, profile the code. You may be suffering from excessive cache misses. The Athlon 2600 only has 256k L2. A relatively cheap fix for that could be simply getting a proc with lots of cache. ie, alpha, MIPS, etc. You should be able to find a vendor to let you run some benchmarks. Tim _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From xyzzy at speakeasy.org Fri Jun 20 14:07:59 2003 From: xyzzy at speakeasy.org (Trent Piepho) Date: Fri, 20 Jun 2003 11:07:59 -0700 (PDT) Subject: [OT] Maximum performance on single processor ? In-Reply-To: <3EF32555.7040006@attbi.com> Message-ID: On Fri, 20 Jun 2003, Robert Myers wrote: > You may have done better service to the readers of this list than you > have to yourself, since the readers of this list are naturally inclined > to think about the sort of question you have asked, and the answers have > been interesting to read. > > The fact that you are asking the question, though, begs an answer that > others have partially given: no plausible hardware purchase you can make > will be a substitute for you or someone you are working with to get I'm in agreement with what everyone else has said, if you're using an AthlonMP 2600 now, no < $20k single cpu system will give you a factor of 1000 speedup. A factor of two is probably the best you can hope for. But, the original question is still unanswered. If you had a non-parallel scientific fortran code, what current system <$20k would run it the fastest? Opteron, Itanium, Xeon, Athlon, Power4? Of course, I realize that the correct answer is, "it depends." Things like memory access patterns, suitability for SIMD instructions, floating point mix, etc. all make a big difference. But without that knowledge, what would the best guess be? _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From mikee at mikee.ath.cx Fri Jun 20 14:28:51 2003 From: mikee at mikee.ath.cx (Mike Eggleston) Date: Fri, 20 Jun 2003 13:28:51 -0500 Subject: [OT] Maximum performance on single processor ? In-Reply-To: <5.1.1.6.2.20030620100637.03ca3450@mail.harddata.com>; from maurice@harddata.com on Fri, Jun 20, 2003 at 10:09:54AM -0600 References: <200306201502.h5KF2EU21424@NewBlue.Scyld.com> <5.1.1.6.2.20030620100637.03ca3450@mail.harddata.com> Message-ID: <20030620132851.J23866@mikee.ath.cx> On Fri, 20 Jun 2003, Maurice Hilarius wrote: > With regards to your message: > >From: Marc Baaden > > <<>> > > >So my question was more, what is the fastest CPU I can get for $20000 > >at the moment (without explicitly parallelizing, hyperthreading or > >vectorizing my code). > > > I believe the answer is an Opteron. > Of course that much money will buy quite a lot of CPUs! > > With the memory bandwidth, floating point engine, and Hypertransport links > on these, they are VERY fast for this type of thing. > If you want access to a machine to run some code samples on, please contact > me off-list and we can arrange some time for you on a machine. > We have test/demo dual Opteron servers in place, available over SSH > connections, for exactly this purpose. > They are set up with compilers, libraries and tools suitable for this use. If you have something like RH7+ on one of these boxes, would you please send me the output of 'cat /proc/cpuinfo'? Thanks. Mike _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From mikee at mikee.ath.cx Fri Jun 20 15:08:25 2003 From: mikee at mikee.ath.cx (Mike Eggleston) Date: Fri, 20 Jun 2003 14:08:25 -0500 Subject: [OT] Maximum performance on single processor ? In-Reply-To: ; from rgb@phy.duke.edu on Fri, Jun 20, 2003 at 12:51:04PM -0400 References: <3EF32555.7040006@attbi.com> Message-ID: <20030620140825.K23866@mikee.ath.cx> On Fri, 20 Jun 2003, Robert Myers wrote: > The only no-brains speedup strategy that could work is the most powerful > speed-up strategy of all: wait a couple of years for off-the-shelf > hardware to get faster. The code that executes the fastest is the code that does not exist. Mike _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From xyzzy at speakeasy.org Fri Jun 20 15:15:24 2003 From: xyzzy at speakeasy.org (Trent Piepho) Date: Fri, 20 Jun 2003 12:15:24 -0700 (PDT) Subject: [OT] Maximum performance on single processor ? In-Reply-To: <20030620132851.J23866@mikee.ath.cx> Message-ID: On Fri, 20 Jun 2003, Mike Eggleston wrote: > > With the memory bandwidth, floating point engine, and Hypertransport links > > on these, they are VERY fast for this type of thing. > > If you want access to a machine to run some code samples on, please contact > > me off-list and we can arrange some time for you on a machine. > > We have test/demo dual Opteron servers in place, available over SSH > > connections, for exactly this purpose. > > They are set up with compilers, libraries and tools suitable for this use. > > If you have something like RH7+ on one of these boxes, would you please send > me the output of 'cat /proc/cpuinfo'? Thanks. How about the kernel startup messages too? I want to a five digit bogomips value! It's like CPU pr0n..... _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rene at renestorm.de Fri Jun 20 18:01:35 2003 From: rene at renestorm.de (ReneStorm) Date: Sat, 21 Jun 2003 00:01:35 +0200 Subject: Pallas and LanaiXP, sporadic gaps Message-ID: Hi Folks, does anyone have experience with the Pallas Benchmark and the new Myrinet LanaiXP Cards ? Problem: I ve got some strang, sporadic gaps on our 32 Node (64 CPU) Cluster. Example: processes 32: (32 waiting in barrier) SendRecv byte Mb/sec 512 74.54 1024 129.78 2048 193.25 4096 252.89 8192 46.84 16384 70.79 32768 230.63 32768 256.37 Cluster Equipment: 32x Myrinet Lanai M3F-PCIXD-2 Myrinet-Fiber/PCI-X Interface, 64x 2.4 Ghz, 32 GB Mem All myrinet cards are running with 133 MHz, (PCI read/write above 800 MB/sec). Compiler: gcc, pgroup, intel The configuration looks fine. Cards are very fast. 8 byte PingPong below 7usec, Peak Bandwidth near 500 MB/sec Thanks in advance and have a nice weekend. Rene _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From patrick at myri.com Fri Jun 20 19:44:07 2003 From: patrick at myri.com (Patrick Geoffray) Date: 20 Jun 2003 19:44:07 -0400 Subject: Pallas and LanaiXP, sporadic gaps In-Reply-To: References: Message-ID: <1056152649.964.30.camel@asterix> Hi Rene, On Fri, 2003-06-20 at 18:01, ReneStorm wrote: > does anyone have experience with the Pallas Benchmark and the new Myrinet > LanaiXP Cards ? > > Problem: I ve got some strang, sporadic gaps on our 32 Node (64 CPU) > Cluster. I think this is a known bug that has been fixed recently by the GM-2 folks. They released an update (GM 2.0.1) yesterday. You should see the number of resends going up big time in the GM counters at the same time than the hiccups. Please let me know if you still see it with the latest GM. Patrick -- Patrick Geoffray Myricom, Inc. http://www.myri.com _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From bushnell at chem.ucsb.edu Fri Jun 20 20:19:44 2003 From: bushnell at chem.ucsb.edu (John Bushnell) Date: Fri, 20 Jun 2003 17:19:44 -0700 (PDT) Subject: [OT] Maximum performance on single processor ? In-Reply-To: <200306201743.TAA27894@apex.ibpc.fr> Message-ID: On Fri, 20 Jun 2003, Marc Baaden wrote: > landman at scalableinformatics.com said: > >> As it does not appear that you have profiled the code (just a guess), > > Half-right half-wrong guess. I have very rapidly profiled the code, just > confirming what I already guessed, that the bottleneck (> 80% of the time) > is the energy calculation subroutine, which calculates pairwise interactions > between atoms in the system. > There are indeed potential speedup factors in this code, like > - introduction of a neighbour list > - parallelization > .. or maybe the use of a specialized board (MD-GRAPE) for just these > calculations. Looking at some of the info on the Riken site who evidently makes these GRAPE cards, this sound like a good bet if pairwise energy calculations are really where most of the speedup is needed. They give a peak performance for the 4-cpu GRAPE-2 as 64GFLOPS (ouch). Very interesting. I've never heard of these cards. I wonder what the approx. cost is? They look very easy to implement, and even have a really simple piece of Fortran code showing how to use one on their web site. One of their benchmarks shows an almost 100x speedup using three cards on a Sun... - John _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From Serguei.Patchkovskii at sympatico.ca Fri Jun 20 21:16:44 2003 From: Serguei.Patchkovskii at sympatico.ca (Serguei Patchkovskii) Date: Fri, 20 Jun 2003 21:16:44 -0400 Subject: cluster of AOD Opteron References: <200306201342.RAA28782@nocserv.free.net> Message-ID: <005701c33792$c7c1ddf0$6501a8c0@sims.nrc.ca> > The price for Opteron- > based servers is high, and price/performance ratio in comparison > w/Xeon is not clear. Once you start populating your systems with "interesting" amounts of memory (i.e. anything above 2Gbytes), the price difference between dual Opterons and dual Xeons is really in the noise - at least at the places we buy. If your suppliers charge you a lot more for Opterons, may be you should look for another source? Serguei _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From hahn at physics.mcmaster.ca Sat Jun 21 03:01:32 2003 From: hahn at physics.mcmaster.ca (Mark Hahn) Date: Sat, 21 Jun 2003 03:01:32 -0400 (EDT) Subject: [OT] Maximum performance on single processor ? In-Reply-To: Message-ID: > > If you have something like RH7+ on one of these boxes, would you please send > > me the output of 'cat /proc/cpuinfo'? Thanks. > > How about the kernel startup messages too? I want to a five digit bogomips > value! It's like CPU pr0n..... they're not *that* interesting: Total of 2 processors activated (5570.56 BogoMIPS). if you're looking for bogo-max, you'd look for a larger machine (IBM ccNUMA? GS320 or Marvel?) _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From seth at hogg.org Sat Jun 21 06:23:46 2003 From: seth at hogg.org (Simon Hogg) Date: Sat, 21 Jun 2003 11:23:46 +0100 Subject: [OT] Maximum performance on single processor ? In-Reply-To: References: <200306201743.TAA27894@apex.ibpc.fr> Message-ID: <4.3.2.7.2.20030621111838.00aec9f0@pop.freeuk.net> At 17:19 20/06/03 -0700, John Bushnell wrote: > Looking at some of the info on the Riken site who evidently makes >these GRAPE cards, this sound like a good bet if pairwise energy >calculations are really where most of the speedup is needed. They >give a peak performance for the 4-cpu GRAPE-2 as 64GFLOPS (ouch). http://www.research.ibm.com/grape/ I hadn't heard of these either, but a quote from the IBM site; "These chips will be combined with other special chips at RIKEN later this year to make a Molecular Dynamics Machine, which will run at a whopping 100 Teraflops speed, the fastest for any computer in the world." There seems to be a fair bit of marketing speak at that site as well :-) -- Simon _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From christopher.yip at utoronto.ca Sat Jun 21 17:15:35 2003 From: christopher.yip at utoronto.ca (Christopher Yip) Date: Sat, 21 Jun 2003 17:15:35 -0400 Subject: MDGRAPE In-Reply-To: <200306211901.h5LJ17U15703@NewBlue.Scyld.com> Message-ID: <7F9D857E-A42D-11D7-A1EE-000A9587184C@utoronto.ca> Here's a link to that MD GRAPE info: http://www.peta.co.jp/ and a couple of their users: http://aci5.astronomy.pomona.edu/docs/grape/MD- GRAPE2_implementation.html Christopher M. Yip, Ph.D., P.Eng. Associate Professor - Canada Research Chair in Molecular Imaging Departments of Chemical Engineering and Applied Chemistry Department of Biochemistry Institute of Biomaterials and Biomedical Engineering 407 Rosebrugh Building Toronto, Ontario, CANADA M5S 3G9 (416) 978-7853 (416) 978-4317 (fax) christopher.yip at utoronto.ca http://bigten.ibme.utoronto.ca _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From iwao at rickey-net.com Sat Jun 21 23:55:47 2003 From: iwao at rickey-net.com (Iwao Makino) Date: Sun, 22 Jun 2003 12:55:47 +0900 Subject: [OT] Re:GRAPE In-Reply-To: References: Message-ID: For those of whom interested for Grape(s) ... There's another GRAPE(not MDGRAPE) This is for Astronomical simulation ONLY but... Is faster than earth simulator of NEC. Below site talk about both At 17:19 -0700 on 03.6.20, John Bushnell wrote regarding Re: [OT] Maximum performance on single processor ?: > Looking at some of the info on the Riken site who evidently makes >these GRAPE cards, this sound like a good bet if pairwise energy >calculations are really where most of the speedup is needed. They >give a peak performance for the 4-cpu GRAPE-2 as 64GFLOPS (ouch). >Very interesting. I've never heard of these cards. I wonder what >the approx. cost is? I see MDGRAPE-2 costs shown about $17,000 #Not sure they include booster card or not. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From landman at scalableinformatics.com Sat Jun 21 09:37:35 2003 From: landman at scalableinformatics.com (Joseph Landman) Date: 21 Jun 2003 09:37:35 -0400 Subject: [OT] Maximum performance on single processor ? In-Reply-To: <200306201743.TAA27894@apex.ibpc.fr> References: <200306201743.TAA27894@apex.ibpc.fr> Message-ID: <1056202654.4432.24.camel@protein.scalableinformatics.com> On Fri, 2003-06-20 at 13:43, Marc Baaden wrote: > landman at scalableinformatics.com said: > >> As it does not appear that you have profiled the code (just a guess), > > Half-right half-wrong guess. I have very rapidly profiled the code, just > confirming what I already guessed, that the bottleneck (> 80% of the time) > is the energy calculation subroutine, which calculates pairwise interactions > between atoms in the system. Excellent. I am glad I was half wrong :). > There are indeed potential speedup factors in this code, like > - introduction of a neighbour list > - parallelization > .. or maybe the use of a specialized board (MD-GRAPE) for just these > calculations. One of the fastest speedup methods I have used for someone to accelerate single processor performance of their code was to look at the pair interactions, which required a square-root operation, and recast that from force (which required the square-root operation) to potential energy, which did not. That gave me a factor of 5 in that loop which had been dominating the calculation. If you can live with 2 potential energy evaluations at a fixed interval, you might be able to get away from the square root. You also want to rewrite the innards to get away from division, even to the point of pre-calculating it and playing games there. Most x86 do not have a hardware accelerated 1/x, or 1/sqrt(x). Power4 ABI, MIPS n64 ABI, and the PA-RISC ABI do. The compilers are even "smart enough" in the case of the SGI compilers, or reasonable enough to help recast your calculations this way. Most of the pair-wise interactions I have encountered in research codes has been a high precision pair potential, or a high precision sum over all contributions from all interacting orbitals. If you are dealing wit MD (sounds like you are), you might look at whether or not you need as high a precision. Other codes I have worked on use cut-off radii which limit the number of neighbors (and the search through the system for neighbors with which to compute interactions). Square-roots tend to pop-up here as well, in the distance calculation. It is (usually) trivial (and still meaningful) to avoid distance, and do a distance squared, so you don't have to pay for the expensive square root. > Unfortunately I currently do lack the competence for parallelization or other > optimizations (probably trying to use SSE2 code would be even better), and > part of the more specialized methods seems like a complicated and particularly > short-lived business. Understood. You might explore with various groups whether or not they could help you. You might be able to work with a local computer science, or physics/chemistry department to locate an exceptional student who would gain from the experience of working with a company. You certainly could find a company to at least assess the situation for you and let you know whether or not it is feasable. If it were, I would suspect the changes would cost less than 20k$. [...] > hahn at physics.mcmaster.ca said: > >> an athlon MP 2600+) > >> a quite poor architecture for today; it was competitive a year ago. I disagree with the characterization of "quite poor". On certain classes of code, it is significantly better than its direct competition, though on other classes it is significantly worse. [...] > So - if I interpret your and other people's statements right - depending on > the code, memory might be as important/critical as CPU power ? > Is there a way to check on what the code depends most ? Yes. Using a combination of profiling techniques, and exploitation of CPU profiling counters, you should be able to track the most expensive portion of the calculation, and why it is most expensive. See the Oprofile bit on http://freshmeat.net (http://freshmeat.net/projects/oprofile/?topic_id=45) and Troy Baer's excellent lperfex (http://freshmeat.net/projects/lperfex/?topic_id=861). -- Joseph Landman, Ph.D Scalable Informatics LLC email: landman at scalableinformatics.com web: http://scalableinformatics.com phone: +1 734 612 4615 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From andrewxwang at yahoo.com.tw Mon Jun 23 23:22:59 2003 From: andrewxwang at yahoo.com.tw (=?big5?q?Andrew=20Wang?=) Date: Mon, 23 Jun 2003 20:22:59 -0700 (PDT) Subject: Opteron (x86-64) compute farms/clusters? Message-ID: <20030624032259.48447.qmail@web16809.mail.tpe.yahoo.com> How well the existing tools run on Opteron machines? Does LAM-MPI or MPICH run in 64-bit mode? Also, has anyone tried Gridengine or PBS on it? Lastly, is there an opensource Opteron compile farm that I can access? I would like to see if my code really runs correctly on them before buying! Andrew. __________________________________ Do you Yahoo!? SBC Yahoo! DSL - Now only $29.95 per month! http://sbc.yahoo.com _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From Thomas.Alrutz at dlr.de Tue Jun 24 04:19:48 2003 From: Thomas.Alrutz at dlr.de (Thomas Alrutz) Date: Tue, 24 Jun 2003 10:19:48 +0200 Subject: Opteron (x86-64) compute farms/clusters? References: <20030624032259.48447.qmail@web16809.mail.tpe.yahoo.com> Message-ID: <3EF809A4.1050802@dlr.de> I just made some benchmarks on a Opteron 240 (1.4 GHz) node running with Suse/United Linux Enterprise edition. I have sucessfully compiled mpich-1.2.4 in 64 bit without any problems (./configure -device=ch_p4 -commtype=shared). The default compiler is the gcc-3.2.2 (maybe a Suse patch) and is set to 64Bit, the Portland (5.0beta) compiler didn't worked at all ! I tried our CFD-code (TAU) to run 3 aerodynamik configurations on this machine with both CPUs and the results are better then estimated. We achieved in full multigrid (5 cycles, 1 equation turbulence model) a efficiency of about 97%, 92% and 101 % for the second CPU. Those results are much better as the results we get on the Intel Xeons (around 50%). Thomas -- __/|__ | Dipl.-Math. Thomas Alrutz /_/_/_/ | DLR Institut fuer Aerodynamik und Stroemungstechnik |/ | Numerische Verfahren DLR | Bunsenstr. 10 | D-37073 Goettingen/Germany _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rodmur at maybe.org Tue Jun 24 09:18:57 2003 From: rodmur at maybe.org (Dale Harris) Date: Tue, 24 Jun 2003 06:18:57 -0700 Subject: [OT] Omega Nomad OM-44 Message-ID: <20030624131857.GM7917@maybe.org> Hi, I realize this is kind of off topic. But don't exactly know where else to ask off hand. Just curious if anyone has played with one of the Omega Nomad OM-40 family of temperature sensors: http://www.omega.com/pptst/OM-40.html And got it running under Linux? I wasn't very successful in getting information out of Omega about it other than that they don't support Linux. It doesn't look like it should be too hard to do since it has a serial access. I thought it might be a fairly nice device for the machine room, it's really small, put it about it anywhere. Relatively cheap. -- Dale Harris rodmur at maybe.org /.-) _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From shewa at inel.gov Tue Jun 24 10:09:09 2003 From: shewa at inel.gov (Andrew Shewmaker) Date: Tue, 24 Jun 2003 08:09:09 -0600 Subject: Opteron (x86-64) compute farms/clusters? In-Reply-To: <20030624032259.48447.qmail@web16809.mail.tpe.yahoo.com> References: <20030624032259.48447.qmail@web16809.mail.tpe.yahoo.com> Message-ID: <3EF85B85.1090200@inel.gov> Andrew Wang wrote: > How well the existing tools run on Opteron machines? > > Does LAM-MPI or MPICH run in 64-bit mode? Also, has > anyone tried Gridengine or PBS on it? > > Lastly, is there an opensource Opteron compile farm > that I can access? I would like to see if my code > really runs correctly on them before buying! > > Andrew. Most vendors will give you a remote account or send you an evaluation unit. I imagine you'll probably be contacted off-list by several of them. I've compiled a 64-bit MPICH, GROMACS, and a few other codes with a GCC 3.3 prerelease. I have also used the beta PGI compiler with good results. Some build scripts require slight modification to recognize x86-64 as an architecture, but most porting is trivial. GROMACS has some optimized assembly that didn't come out quite right, but I bet they have it fixed by now. All my testing was a couple of weeks before the release, but I haven't gotten any in yet unfortunately. Andrew -- Andrew Shewmaker, Associate Engineer Phone: 1-208-526-1276 Idaho National Eng. and Environmental Lab. P.0. Box 1625, M.S. 3605 Idaho Falls, Idaho 83415-3605 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From Luc.Vereecken at chem.kuleuven.ac.be Tue Jun 24 13:22:12 2003 From: Luc.Vereecken at chem.kuleuven.ac.be (Luc Vereecken) Date: Tue, 24 Jun 2003 19:22:12 +0200 Subject: [OT] Omega Nomad OM-44 In-Reply-To: <20030624131857.GM7917@maybe.org> Message-ID: <3.0.6.32.20030624192212.008f1100@arrhenius.chem.kuleuven.ac.be> At 06:18 AM 6/24/03 -0700, Dale Harris wrote: >Just curious if anyone has played with one of the Omega Nomad >OM-40 family of temperature sensors: > >http://www.omega.com/pptst/OM-40.html > >And got it running under Linux? I wasn't very successful in getting >information out of Omega about it other than that they don't support >Linux. A nice alternative are the devices by picotech http://www.picotech.com which are supported under linux and windows with drivers and logging software. They have a linux driver that one can access easily through shared memory (C-example program can be downloaded). I use it to monitor the temperature in my computer room, log it, do temperature-related shutdowns, etc. all based on very simple perl scripts (I use the TH-03 with 3 thermistor-based temperature measurement channels for air-inlet, ambient air, and air-outlet). Nice equipment, works very well. Luc Vereecken _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From nixon at nsc.liu.se Tue Jun 24 15:39:30 2003 From: nixon at nsc.liu.se (nixon at nsc.liu.se) Date: Tue, 24 Jun 2003 21:39:30 +0200 Subject: [OT] Omega Nomad OM-44 In-Reply-To: <3.0.6.32.20030624192212.008f1100@arrhenius.chem.kuleuven.ac.be> (Luc Vereecken's message of "Tue, 24 Jun 2003 19:22:12 +0200") References: <3.0.6.32.20030624192212.008f1100@arrhenius.chem.kuleuven.ac.be> Message-ID: Luc Vereecken writes: > A nice alternative are the devices by picotech > http://www.picotech.com > which are supported under linux and windows with drivers and logging > software. They have a linux driver that one can access easily through > shared memory (C-example program can be downloaded). I use it to monitor > the temperature in my computer room, log it, do temperature-related > shutdowns, etc. all based on very simple perl scripts (I use the TH-03 with > 3 thermistor-based temperature measurement channels for air-inlet, ambient > air, and air-outlet). Nice equipment, works very well. I second that. I have a TH-03 and a RH-02 in our computer room, which have saved us a couple of times when the campus cooling system went down. And since the serial protocol is documented you can easily replace the supplied driver with your own. I've written a little Python daemon that polls the units and makes the measurements available via XMLRPC, which fits our monitoring setup nicely. -- Leif Nixon Systems expert ------------------------------------------------------------ National Supercomputer Centre Linkoping University ------------------------------------------------------------ _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From hahn at physics.mcmaster.ca Tue Jun 24 18:14:06 2003 From: hahn at physics.mcmaster.ca (Mark Hahn) Date: Tue, 24 Jun 2003 18:14:06 -0400 (EDT) Subject: [OT] Omega Nomad OM-44 In-Reply-To: <3.0.6.32.20030624192212.008f1100@arrhenius.chem.kuleuven.ac.be> Message-ID: > A nice alternative are the devices by picotech > http://www.picotech.com I prefer to use iButtons. these are button-sized devices that you hang off a two-wire bus, and plug into some computer's serial port. everything is utterly open, and it's cheap and flexible. I have 6 sensors in my machineroom: http://sharcnet.mcmaster.ca/~hahn/cgi-bin/temp.cgi which are very trivially collected and logged to the obvious kind of sql table. www.ibutton.com. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rodmur at maybe.org Tue Jun 24 19:21:28 2003 From: rodmur at maybe.org (Dale Harris) Date: Tue, 24 Jun 2003 16:21:28 -0700 Subject: [OT] Omega Nomad OM-44 In-Reply-To: References: <3.0.6.32.20030624192212.008f1100@arrhenius.chem.kuleuven.ac.be> Message-ID: <20030624232128.GR7917@maybe.org> On Tue, Jun 24, 2003 at 06:14:06PM -0400, Mark Hahn elucidated: > > http://www.picotech.com > > www.ibutton.com. > grr.. wish I had known about those before buying the Nomad. Dale (/me slaps forhead) _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rgb at phy.duke.edu Wed Jun 25 11:56:13 2003 From: rgb at phy.duke.edu (Robert G. Brown) Date: Wed, 25 Jun 2003 11:56:13 -0400 (EDT) Subject: [OT] Omega Nomad OM-44 In-Reply-To: <20030624232128.GR7917@maybe.org> Message-ID: On Tue, 24 Jun 2003, Dale Harris wrote: > On Tue, Jun 24, 2003 at 06:14:06PM -0400, Mark Hahn elucidated: > > > http://www.picotech.com > > > > www.ibutton.com. > > > > grr.. wish I had known about those before buying the Nomad. > Both of these are now on the brahma resources page, along with a third vendor's thermal sensor link. > (/me slaps forhead) You shouldn't feel too bad. It actually has taken years for there to be an acceptable crop of low-cost offerings in this arena. For a while there, one had to either DIY or use something as expensive as a Netbotz in order to get online environmental monitoring -- there was only one cheap device that would work in Linux and the tiny company that made/resold it bellied up. At least now there is choice, and the choices are all <$200! For an extra $100 or so, you can add a PC-TV card and an X10 and build yourself a netbotz out of any system with a bit of attention to spare for polling, or buy one of the legendary wal-mart specials and build a complete net monitoring appliance with video, thermal, humidity, door monitoring for <$600. rgb > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org > To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf > -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb at phy.duke.edu _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From philippe.blaise at cea.fr Wed Jun 25 05:28:26 2003 From: philippe.blaise at cea.fr (phblai) Date: Wed, 25 Jun 2003 11:28:26 +0200 Subject: Opteron (x86-64) compute farms/clusters? In-Reply-To: <200306241612.UAA09513@nocserv.free.net> References: <200306241612.UAA09513@nocserv.free.net> Message-ID: <3EF96B3A.70601@cea.fr> Mikhail Kuzminsky wrote: >According to Thomas Alrutz > > >>I just made some benchmarks on a Opteron 240 (1.4 GHz) node running with >>Suse/United Linux Enterprise edition. >>I have sucessfully compiled mpich-1.2.4 in 64 bit without any problems >>(./configure -device=ch_p4 -commtype=shared). The default compiler is >>the gcc-3.2.2 (maybe a Suse patch) and is set to 64Bit, the Portland >>(5.0beta) compiler didn't worked at all ! >> >>I tried our CFD-code (TAU) to run 3 aerodynamik configurations on this >>machine with both CPUs and the results are better then estimated. >>We achieved in full multigrid (5 cycles, 1 equation turbulence model) a >>efficiency of about 97%, 92% and 101 % for the second CPU. >>Those results are much better as the results we get on the Intel Xeons >>(around 50%). >> >> > It looks that this results are predictable: Xeon CPUs require high >memory bandwidth, but both CPUs share common system bus. Opteron CPUs >have own memory buses and scale in this sense excellent. Better SPECrate >results for Opteron (i.e. work on a mix of tasks) confirm (in particular) >this features. CFD codes, I beleive, require high memory throughput ... > >Mikhail Kuzminsky >Zelinsky Institute of Organic Chemistry >Moscow > > >_______________________________________________ >Beowulf mailing list, Beowulf at beowulf.org >To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf > > > > the 4 cpus (ccNUMA) AMD Opteron scales very well (each cpu has its own memory with a 5.3 GB/s bandwith access), compare to the (SMP) Intel / Itanium2 equivalent (I870 chipset), where the 4 cpus share a common 6.4GB/s memory access. The 4 cpus Opteron is cheap, but the Itanium2 can perform quite better on some benchs (the 3MB L3 cache has a 32 GB/s bandwith !). In fact, the AMD Opteron architecture is very close to the HP/Compaq/Digital EV7 Marvel one, that scales very well to 32 cpus (maybe 64 ?). But I don' know if AMD plans to make machines with more than 8 cpus ? Philippe Blaise _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From kuku at physik.rwth-aachen.de Wed Jun 25 09:46:38 2003 From: kuku at physik.rwth-aachen.de (Christoph Kukulies) Date: Wed, 25 Jun 2003 15:46:38 +0200 Subject: 3Com 905C and ASUS P4GX8 Message-ID: <200306251346.h5PDkcF19321@accms33.physik.rwth-aachen.de> Hi, I installed Linux RH 7.3 on a Dual 3GHz P4 CPU Asus P4GX8 MB and since the on board 100 MBit controller didn't seem to be supported by this Linux version (doubt whether any version does in the moment) I put in a 3Com 905c and run it with the 509 vortex driver. I'm seeing this in /var/log/messages and the machine is very slow on network copies and activity. Jun 25 10:24:34 accms07 kernel: 14: @c33a3580 length 800000be status 000100beJun 25 10:24:34 accms07 kernel: 15: @c33a35c0 length 800000be status 000100beJun 25 10:24:34 accms07 kernel: eth0: Transmit error, Tx status register 82. Jun 25 10:24:34 accms07 kernel: Probably a duplex mismatch. See Documentation/networking/vortex.txt Jun 25 10:24:34 accms07 kernel: Flags; bus-master 1, dirty 156838(6) current 156838(6) Jun 25 10:24:34 accms07 kernel: Transmit list 00000000 vs. c33a3380. Jun 25 10:24:34 accms07 kernel: 0: @c33a3200 length 800000be status 000100be Jun 25 10:24:34 accms07 kernel: 1: @c33a3240 length 800000be status 000100be Jun 25 10:24:34 accms07 kernel: 2: @c33a3280 length 800000be status 000100be Jun 25 10:24:34 accms07 kernel: 3: @c33a32c0 length 800000be status 000100be Jun 25 10:24:34 accms07 kernel: 4: @c33a3300 length 800000be status 000100be Jun 25 10:24:34 accms07 kernel: 5: @c33a3340 length 800000be status 800100be Jun 25 10:24:34 accms07 kernel: 6: @c33a3380 length 800000be status 000100be Jun 25 10:24:34 accms07 kernel: 7: @c33a33c0 length 800000be status 000100be Jun 25 10:24:34 accms07 kernel: 8: @c33a3400 length 800000be status 000100be Jun 25 10:24:34 accms07 kernel: 9: @c33a3440 length 800000be status 000100be Jun 25 10:24:34 accms07 kernel: 10: @c33a3480 length 800000be status 000100beJun 25 10:24:34 accms07 kernel: 11: @c33a34c0 length 800000be status 000100beJun 25 10:24:34 accms07 kernel: 12: @c33a3500 length 800000be status 000100beJun 25 10:24:34 accms07 kernel: 13: @c33a3540 length 800000be status 000100be Any clues? -- Chris Christoph P. U. Kukulies kukulies (at) rwth-aachen.de _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From luis.licon at yakko.cimav.edu.mx Wed Jun 25 17:53:21 2003 From: luis.licon at yakko.cimav.edu.mx (Luis Fernando Licon Padilla) Date: Wed, 25 Jun 2003 15:53:21 -0600 Subject: Off topic - G5 question Message-ID: <3EFA19D1.7010009@yakko.cimav.edu.mx> Hi you all guys, I've just checked the G5 processor out, what do you think about it, it seems like it's not good enough considering it's architecture and it's performance. http://www.apple.com/lae/g5 Cheerz, Luis -- ISC Luis Fernando Licon Padilla Telecommunications Department Advanced Materials Research Center Miguel de Cervantes 120 Complejo Industrial Chihuahua C.P.31109 Chihuahua, Chih. Mexico Phone: 52 (614)4391154 Fax: 52 (614)4391112 alternative e-mail: lordsirion2002 at yahoo.com _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From seth at hogg.org Wed Jun 25 18:42:59 2003 From: seth at hogg.org (Simon Hogg) Date: Wed, 25 Jun 2003 23:42:59 +0100 Subject: Off topic - G5 question In-Reply-To: <3EFA19D1.7010009@yakko.cimav.edu.mx> Message-ID: <4.3.2.7.2.20030625234133.00ac75a0@pop.freeuk.net> At 15:53 25/06/03 -0600, Luis Fernando Licon Padilla wrote: >Hi you all guys, I've just checked the G5 processor out, what do you think >about it, it seems like it's not good enough considering it's architecture >and it's performance. > >http://www.apple.com/lae/g5 There are some benchmarks linked from there which look impressive, but (obviously?) there are some question marks over the benchmarks; http://www.extremetech.com/article2/0,3973,1136018,00.asp has a discussion. Simon _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From hahn at physics.mcmaster.ca Wed Jun 25 20:20:00 2003 From: hahn at physics.mcmaster.ca (Mark Hahn) Date: Wed, 25 Jun 2003 20:20:00 -0400 (EDT) Subject: Off topic - G5 question In-Reply-To: <3EFA19D1.7010009@yakko.cimav.edu.mx> Message-ID: > Hi you all guys, I've just checked the G5 processor out, what do you > think about it, it seems like it's not good enough considering it's > architecture and it's performance. > > http://www.apple.com/lae/g5 it's a nice step forward for Apple - the first time they've actually taken dram seriously. but Apple drinks a little too much of their own koolaid, or at least gives their marketing weasels a little too much leash. if you haven't looked already, Apple paid one of those analyst companies (Veritest, nice name) to prove that the G5 spanks those unwashed, beige-boxed, Intel-using heathens. to accomplish this, they tuned the G5 quite nicely, and relentlessly de-tuned the PC - stopping just short of installing WinME on it ;) if you want to make your competition look bad, run their benchmarks for them. at a glance, it might seem sensible to use gcc 3.3 for both machines; as Veritest says: To be able to directly compare the performance of the hardware systems, the same compiler - GCC, with similar settings were used on both platforms. OK, well, first of all, even if you build gcc 3.3 from scratch on each system, it's not going to factor out compiler effects. gcc does slightly different things on each platform. next, they didn't actually use the same compiler - they used Apple's gcc 3.3 binary, and for the PC, one build from virgin sources. all of this begs the question of why they wouldn't use Intel's compilers, which for some old reason, do really well on SPEC sources... it gets worse. things like tuning the dual-G5 to avoid snoop traffic when running (uniprocessor) CPU2000 tests, but turning *on* HT on the (uniprocessor) P4, and running an SMP kernel. again, there's a nugget of plausibility there, since they presumably couldn't get a uniprocessor version of Mac OS X. sure enough: they turn HT off and install a uni kernel to test SPEC rate on the uniprocessor and disable HT on the SMP system. quite different compiler switches for the two platforms, as well. don't forget to install a special single-threaded malloc, but only for the uniprocessor G5 tests... they appear to be trying to make a same-to-same comparison, which is both difficult and strange. users want a best-to-best comparison. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rdn at uchicago.edu Thu Jun 26 02:45:18 2003 From: rdn at uchicago.edu (Russell Nordquist) Date: Thu, 26 Jun 2003 01:45:18 -0500 (CDT) Subject: is this a good candidate for bproc? Message-ID: I've been lurking on this list for a while and I blame this question on that. I keep seeing new stuff I want to play with. Yesterday it was the iButton and today it's bproc. Here is the situation....I have a four node "cluster" that will be running independent computational jobs. No mpi, etc just lots 'o non-parallel jobs. My initial inclination was to setup a traditional (ie ssh to nodes and start jobs) cluster with SGE (and hide the ssh to nodes part). That way I can limit one job per CPU and be done with it. But, then I got to thinking about single system images. One issue I have had with queue systems is jobs lingering after their queue slot is gone (admittedly this has only happened with mpich programs, but our clusters haven't done much else until now). Users don't know to check process running on the nodes (and shouldn't need to) so I get e-mails about slow jobs and find the non-scheduled run aways. bproc's ability to allow users to manage processes from the master node would take care of this (hey, it's like a twin tower origin 200 now :). and I won't have to deal with libraries, etc on the nodes. However, I've never admined bproc or even been on a bproc system. Does this sound like a good place for bproc? I would use bproc+SGE to keep the 1 job 1 cpu efficiency. Is it realistic to expect that any job that can be run using "ssh node1 -e ./job.exe" would run with "bpsh 1 ./job.exe", or are there code related gotchas (threads?)? I guess I could set up both and see how it each works, but I wanted to get a feel for how useful people found bproc and it's limitations. Although I really want to experiment with bproc, I need to keep the day to day admin time on this system low. thanks russell - - - - - - - - - - - - Russell Nordquist UNIX Systems Administrator Geophysical Sciences Computing http://geosci.uchicago.edu/computing NSIT, University of Chicago - - - - - - - - - - - _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From hanzl at noel.feld.cvut.cz Thu Jun 26 06:33:27 2003 From: hanzl at noel.feld.cvut.cz (hanzl at noel.feld.cvut.cz) Date: Thu, 26 Jun 2003 12:33:27 +0200 Subject: is this a good candidate for bproc? In-Reply-To: References: Message-ID: <20030626123327N.hanzl@unknown-domain> > independent computational jobs. No mpi, etc just lots 'o non-parallel jobs. > ... > I would use bproc+SGE to keep the 1 job 1 cpu efficiency. We have something like this, and I described it here: http://noel.feld.cvut.cz/magi/sge+bproc.html (It is a quick first draft yet uncorrected by any feedback, any opinions/additions/corrections from any list readers welcome). > Is it realistic to expect that any job that can be run using "ssh > node1 -e ./job.exe" would run with "bpsh 1 ./job.exe", or are there > code related gotchas (threads?)? Most compiled executables computing something are just fine. Running complex scripts or administrative programs and daemons on nodes often can be solved by using more NFS mounts than is usual in bproc world to get access to all the executables, libraries and config files. The only nasty problem I am aware of may be the absence of support for control terminal stuff on slave nodes. This stuff is one of the few little known dark corners of UNIX - it is related to kernel calls like setsid(2) and to delivery of signals to groups of processes. In userland this omission translates to non-working Ctrl-C and spurious error messages from some interactive programs. Best Regards Vaclav Hanzl _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rgb at phy.duke.edu Thu Jun 26 10:04:00 2003 From: rgb at phy.duke.edu (Robert G. Brown) Date: Thu, 26 Jun 2003 10:04:00 -0400 (EDT) Subject: Off topic - G5 question In-Reply-To: <3EFA19D1.7010009@yakko.cimav.edu.mx> Message-ID: On Wed, 25 Jun 2003, Luis Fernando Licon Padilla wrote: > Hi you all guys, I've just checked the G5 processor out, what do you > think about it, it seems like it's not good enough considering it's > architecture and it's performance. > > http://www.apple.com/lae/g5 Looks great (if you believe their white papers) if you're planning to run BLAST. Doesn't look horribly shabby at floating point performance overall, has a nice peppy FSB. I certainly wouldn't kick it out of my system room for chewing crackers. So to speak. Although I believe that it requires some fairly careful programming to get maximum speed out of the CPU and would be very curious about real-world benchmarks for code that is just slapped on it and compiled, no tuning or hand vectorization. However, the traditional litany requires that one examine price as well as raw performance, and comparing it to P4 (really P6) class CPUs is apples to oranges. A more interesting and fair comparison would be to opterons and itaniums, and including price. rgb > > > Cheerz, > > > Luis > > -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb at phy.duke.edu _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From seth at hogg.org Thu Jun 26 11:42:39 2003 From: seth at hogg.org (Simon Hogg) Date: Thu, 26 Jun 2003 16:42:39 +0100 Subject: Off topic - G5 question In-Reply-To: References: <3EFA19D1.7010009@yakko.cimav.edu.mx> Message-ID: <4.3.2.7.2.20030626164046.00aedb70@pop.freeuk.net> At 10:04 26/06/03 -0400, Robert G. Brown wrote: >On Wed, 25 Jun 2003, Luis Fernando Licon Padilla wrote: >> http://www.apple.com/lae/g5 >Looks great (if you believe their white papers) if you're planning to >run BLAST. [...] Although I believe that >it requires some fairly careful programming to get maximum speed out of >the CPU and would be very curious about real-world benchmarks for code >that is just slapped on it and compiled, no tuning or hand >vectorization. Aye, there's the rub ... If you believe their white papers ... Comparing A/G BLAST on the G5 with 'vanilla' BLAST on the Xeons. "Apple compared the performance of the dual 2GHz Power Mac G5 running A/G BLAST with a 3GHz Pentium 4?based system and a dual 3.06GHz Xeon-based system, both running Red Hat Linux 9.0 and NCBI BLAST." And what is A/G BLAST? "A/G BLAST is an optimized version of NCBI BLAST developed by Apple in collaboration with Genentech. Optimized for dual PowerPC G5 processors, the Velocity Engine, and the symmetric multiprocessing capabilities of Mac OS X[...]" So once again, it would be nice to see some 'real-world' comparisons of like-with-like. >However, the traditional litany requires that one examine price as well >as raw performance, and comparing it to P4 (really P6) class CPUs is >apples to oranges. A more interesting and fair comparison would be to >opterons and itaniums, and including price. Habsolutely agree on that! -- Simon _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From amitoj at cs.uh.edu Thu Jun 26 11:56:36 2003 From: amitoj at cs.uh.edu (Amitoj G. Singh) Date: Thu, 26 Jun 2003 10:56:36 -0500 (CDT) Subject: is this a good candidate for bproc? In-Reply-To: Message-ID: Worth a try. Look under /etc/security. There's some stuff that will allow you to put restrictions such as the 1 job 1 cpu limit you want. - Amitoj. On Thu, 26 Jun 2003, Russell Nordquist wrote: > I've been lurking on this list for a while and I blame this question on > that. I keep seeing new stuff I want to play with. Yesterday it was the > iButton and today it's bproc. > > Here is the situation....I have a four node "cluster" that will be running > independent computational jobs. No mpi, etc just lots 'o non-parallel jobs. > My initial inclination was to setup a traditional (ie ssh to nodes and > start jobs) cluster with SGE (and hide the ssh to nodes part). That way I > can limit one job per CPU and be done with it. > > But, then I got to thinking about single system images. One issue I have > had with queue systems is jobs lingering after their queue slot is gone > (admittedly this has only happened with mpich programs, but our clusters > haven't done much else until now). Users don't know to check process > running on the nodes (and shouldn't need to) so I get e-mails about slow > jobs and find the non-scheduled run aways. bproc's ability to allow users > to manage processes from the master node would take care of this (hey, > it's like a twin tower origin 200 now :). and I won't have to deal with > libraries, etc on the nodes. > > However, I've never admined bproc or even been on a bproc system. > Does this sound like a good place for bproc? I would use bproc+SGE to keep > the 1 job 1 cpu efficiency. Is it realistic to expect that any job that > can be run using "ssh node1 -e ./job.exe" would run with "bpsh 1 > ./job.exe", or are there code related gotchas (threads?)? > > I guess I could set up both and see how it each works, but I wanted to get > a feel for how useful people found bproc and it's limitations. Although I > really want to experiment with bproc, I need to keep the day to day admin > time on this system low. > > thanks > russell > > - - - - - - - - - - - - > Russell Nordquist > UNIX Systems Administrator > Geophysical Sciences Computing > http://geosci.uchicago.edu/computing > NSIT, University of Chicago > - - - - - - - - - - - > > > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org > To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf > _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rdn at uchicago.edu Thu Jun 26 13:19:46 2003 From: rdn at uchicago.edu (Russell Nordquist) Date: Thu, 26 Jun 2003 12:19:46 -0500 (CDT) Subject: is this a good candidate for bproc? In-Reply-To: <20030626123327N.hanzl@unknown-domain> Message-ID: On Thu, 26 Jun 2003 at 12:33, hanzl at noel.feld.cvut.cz wrote: > > independent computational jobs. No mpi, etc just lots 'o non-parallel jobs. > > ... > > I would use bproc+SGE to keep the 1 job 1 cpu efficiency. > > We have something like this, and I described it here: > > http://noel.feld.cvut.cz/magi/sge+bproc.html > > (It is a quick first draft yet uncorrected by any feedback, any > opinions/additions/corrections from any list readers welcome). looks good to me, thanks. I actually ran across this link when I was researching. Anyone got any opinions on the bproc vs the ssh/nfs approach? I've never used bproc, so I don't have a fell for it? russell > Best Regards > > Vaclav Hanzl > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org > To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf > - - - - - - - - - - - - Russell Nordquist UNIX Systems Administrator Geophysical Sciences Computing http://geosci.uchicago.edu/computing NSIT, University of Chicago - - - - - - - - - - - _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rgb at phy.duke.edu Thu Jun 26 13:55:25 2003 From: rgb at phy.duke.edu (Robert G. Brown) Date: Thu, 26 Jun 2003 13:55:25 -0400 (EDT) Subject: Off topic - G5 question In-Reply-To: <4.3.2.7.2.20030626164046.00aedb70@pop.freeuk.net> Message-ID: On Thu, 26 Jun 2003, Simon Hogg wrote: > And what is A/G BLAST? "A/G BLAST is an optimized version of NCBI BLAST > developed by Apple in collaboration with Genentech. Optimized for dual > PowerPC G5 processors, the Velocity Engine, and the symmetric > multiprocessing capabilities of Mac OS X[...]" Awwww, you-all is jes' so cynical...;-) Actually, I attended at least part of an HPC talk a week+ ago given by an Apple guy -- about half devoted to BLAST and genetic stuff and half devoted to marketing (as far as I could tell:-). From what I could dig out of him (and this is all from memory so don't shoot me if I'm wrong) the G5 has an embedded vector unit in it (the "Velocity Engine" in the marketspeak above?) that has a modestly spectacular peak throughput for instruction streams that match -- I want to say 12 GFLOPs or something like that. The CPU itself has a fairly hefty L3 cache outside of L2 and L1 and some cleverness designed to try to keep the gaping maw the vector unit represents filled. It actually looked like a thoughtful architecture at least at the block device/bus level. However, as is usually the case with vector sub- or co-processors, the real problem is getting code to use it at all, let alone effectively, as the compiler usually groks "CPU" and doesn't know offhand when or how to send code to the vector processor for a speedup or when to leave it alone and execute on the CPU itself. I would guess that code has to minimally be vectorized and instrumented to use the VP with pragma's or the like, or worse, require actual cross-compiled objects to inline in the code, but I really don't know or care enough to fine out. So I think that is what they are referring to as an "optimized version" -- one that somebody has taken the time to instrument for the VP and tune for their cache sizes and so forth. Apple was clearly very interested in targeting the genomics clustering people -- they may (correctly) see them as deep pockets backing an insatiable appetite for cluster power over the next decade or so as projects everywhere build up the lexicon of genetic expression of every living creature on the planet (let alone the "human genome") and be gunning for an inside edge with an architecture and application set deliberately tuned for good performance in this one market. Of course, this makes a LOT of things very difficult to properly benchmark. At a guess, if one hand-tuned almost any sort of vector-intensive floating point code for the VP architecture, one might see a fairly spectacular speedup, but note that the unit's integer performance (indicative of how well the CPU does everyday work) was distinctly unimpressive so a NON-vectorizable application might really suck on it. Also one needs to use SSE2 instrumented compilers and instructions to get fair comparisons from alternative architectures, for the same reasons -- if one compiles a patch of code that SSE2 would (possibly spectacularly) speed up with a non-SSE2 compiler, well, performance might well suck. Mark's observation about best-to-best comparisons are likely one approach, especially if you plan to examine vendor benchmarks. Let Apple tune the hell out of BLAST, but only compare results to an Intel-tuned BLAST on >>its<< favorite CPU/system du jour, etc. Alternatively, just compiling the same source with gcc with the same flags on both systems works for me, at least, as this is likely to be the extent of the effort I want to make tuning per architecture, ever. Maintaining architecture-specific optimizations in a changing world of hardware technology is very painful and expensive. > So once again, it would be nice to see some 'real-world' comparisons of > like-with-like. Amen, although the problem that has stumped philosophers over the ages is: just what IS the real world, anyway? Pardon me, I have to put a cat into a superposition state with a diabolical device...:-) rgb -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb at phy.duke.edu _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From henken at seas.upenn.edu Thu Jun 26 14:22:44 2003 From: henken at seas.upenn.edu (Nicholas Henke) Date: 26 Jun 2003 14:22:44 -0400 Subject: is this a good candidate for bproc? In-Reply-To: References: Message-ID: <1056651763.24158.17.camel@roughneck.liniac.upenn.edu> On Thu, 2003-06-26 at 02:45, Russell Nordquist wrote: > > However, I've never admined bproc or even been on a bproc system. > Does this sound like a good place for bproc? I would use bproc+SGE to keep > the 1 job 1 cpu efficiency. Is it realistic to expect that any job that > can be run using "ssh node1 -e ./job.exe" would run with "bpsh 1 > ./job.exe", or are there code related gotchas (threads?)? Java is really unhappy with bproc. As far as I can tell bproc does not support use of the clone system call yet, and java uses that to spawn its threads. Nic -- Nicholas Henke Penguin Herder & Linux Cluster System Programmer Liniac Project - Univ. of Pennsylvania _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From walkev at presearch.com Thu Jun 26 15:34:52 2003 From: walkev at presearch.com (Vann H. Walke) Date: 26 Jun 2003 15:34:52 -0400 Subject: Off topic - G5 question In-Reply-To: References: Message-ID: <1056656072.24962.47.camel@localhost.localdomain> > Alternatively, just compiling the same source with gcc with the same > flags on both systems works for me, at least, as this is likely to be > the extent of the effort I want to make tuning per architecture, ever. > Maintaining architecture-specific optimizations in a changing world of > hardware technology is very painful and expensive. I pretty much agree with all your points, but don't completely ignore optimizations. For some of our signal processing code, we get a ~2.5x speedup by using SSE primitives. It is isn't hard to justify a couple days of weeding through code to reduce your hardware requirements by half. Plus, at least for the SIMD type instructions porting to other SIMD architectures isn't excruciatingly painful. We were able to tune our code for both the Altivec and SSE instructions, thus giving us "real" performance comparisons. Vann -- Vann H. Walke Presearch, Inc. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From hahn at physics.mcmaster.ca Thu Jun 26 17:39:09 2003 From: hahn at physics.mcmaster.ca (Mark Hahn) Date: Thu, 26 Jun 2003 17:39:09 -0400 (EDT) Subject: Off topic - G5 question In-Reply-To: Message-ID: > unit represents filled. It actually looked like a thoughtful > architecture at least at the block device/bus level. it's dandy. the problem is Apple's marketing nonsense. SIMD is great, too. cache is great. but please don't claim that your in-cache SIMD FLOPS is a general/true FLOPS rate for the machine, since it's NOT, for anything other than Mandelbrot surfing. incidentally, I don't see *any* off-chip cache in the G5's. > Of course, this makes a LOT of things very difficult to properly > benchmark. I don't agree at all. I want to see precisely two sets of numbers when evaluating new hardware performance: - fairly clean microbenchmarks like Stream, streaming sequential file IO, zero-byte packet latency, etc. - best-case performance on real codes - for the compute cluster world, that means SpecFP and SpecFPrate. other fields may want tuned-BLAS or 45-filter photoshop. well, and price, but that's normally boringly predictable (I say "way too expensive!" a lot ;) > suck on it. Also one needs to use SSE2 instrumented compilers and > instructions to get fair comparisons from alternative architectures, for I find that many of my users spend most of their serious time inside tuned library functions, so the compiler is less relevant. though Intel's compilers *do* auto-vectorize onto SSE2. gcc can use SSE2, but doesn't currently autovectorize. incidentally, gcc 3.3 *does* also support intrinsics for Altivec vector operations; I'm not sure whether similar intrinsics are available for SSE2. > flags on both systems works for me, at least, as this is likely to be > the extent of the effort I want to make tuning per architecture, ever. I find this irresponsible (sorry rgb!). if you're going to consume more than a few CPU hours, and those cycles are coming from a shared resource, failing to optimize is just plain hostile. I still see users fling their code at a machine with g77 prog.f && ./a.out and wonder why it's so slow. usually, I stifle the obvious answer, and (hey, this is Canada) apologize for not providing the information they need to make supra-moronic use of the machine... regards, mark 'surly' hahn. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rgb at phy.duke.edu Thu Jun 26 18:12:22 2003 From: rgb at phy.duke.edu (Robert G. Brown) Date: Thu, 26 Jun 2003 18:12:22 -0400 (EDT) Subject: Off topic - G5 question In-Reply-To: Message-ID: On Thu, 26 Jun 2003, Mark Hahn wrote: > I find that many of my users spend most of their serious time inside > tuned library functions, so the compiler is less relevant. though Intel's > compilers *do* auto-vectorize onto SSE2. gcc can use SSE2, but doesn't > currently autovectorize. incidentally, gcc 3.3 *does* also support > intrinsics for Altivec vector operations; I'm not sure whether similar > intrinsics are available for SSE2. > > > flags on both systems works for me, at least, as this is likely to be > > the extent of the effort I want to make tuning per architecture, ever. > > I find this irresponsible (sorry rgb!). if you're going to consume > more than a few CPU hours, and those cycles are coming from a shared > resource, failing to optimize is just plain hostile. Oh, that's ok. I freely admit to utter irresponsibility. Laziness too! I was (of course) at least partly tongue in cheek, although the nugget of truth in what I was saying is what I'd call the "CM5 phenomenon". CM5's were a nifty architecture too -- basically (IIRC) a network of sparcstations each with an attached vector unit. Programming it was nightmarish -- it went beyond instrumenting code, one basically had to cross-compile code modules for the vector units and use the Sparcs as computers to run these modules and handle all the nitty gritty for them (like access to I/O and other resources). Then the CM5 bellied up (Duke sold its old one for a few thousand $$ to somebody interested in salvaging the gold in its gazillion contacts). All that effort totally wasted. We're talking years of a human life, not just a few weeks porting a simple application, and he was just one of Duke's many CM5 users...:-) Or if you prefer, the MPI phenomenon. Back when the CM5 had to be hand coded, so did nearly everything else. Code maintenance was nightmarish, as ever parallel supercomputer manufacturer had their own message passing API and so forth. So the government (major supercomputer purchaser that it is) finally said that enough was enough and that they were only going to use one API in the future and if machines didn't support it they weren't going to be bought, and lo, MPI was born so people didn't have to rewrite their code. Or related examples abound in all realms of computedom. The more you customize or optimize per architecture, the less portable your code becomes and the more expensive it becomes to change architectures until you find yourself in a golden trap, still buying IBM mainframes because you have all this custom stuff that your business relies on that (in its day) was heavily customized to take advantage of all of the nifty features of IBM mainframes. Curiously, a cluster of G5's with their embedded Altivecs, or a cluster of playstations with their embedded vector processor, bear a strong resemblance to the CM5 architecturally and (in the case of the PS2's) in how one has to program them with what amounts to a cross-compiler with instrumented calls to send code and data on to the vector unit for handling. This kind of architecture makes me very nervous. Optimizing is no longer just a matter of picking good compiler flags or writing core loops sensibly (things that can easily be changed or that are good ideas on nearly any architecture); it starts to become an issue of writing code "just for the Altivec" or "just for the PS2" -- hacking your application (a process that can take a LONG time, not just hours or days) irreversibly so that you have to spend hours or days again to change it back to run on a more normal architecture. It is this lack of portability that can, and I say can, be a killer to productivity and ultimately to one's ability to select freely from COTS architectures. Once the code is modified and optimized for (say) a G5, one has to take into account the ADDITIONAL cost of un/remodifying it and reoptimizing it for (say) an Opteron or Itanium or whateverisnextium. This makes Apple very happy, in precisely the way that Microsoft is made happy every time a businessman says "we have to run Windows because everybody is used to it and it would cost us too much time/money/stress to change" even when it is visibly less cost-effective head to head with alternatives. > I still see users fling their code at a machine with > g77 prog.f && ./a.out > and wonder why it's so slow. usually, I stifle the obvious answer, > and (hey, this is Canada) apologize for not providing the information > they need to make supra-moronic use of the machine... Right. They need to be running gcc -O3 -o prog prog.c prog instead. Obviously;-) (yoke, yoke, I was mayking a yoke to help you forget how screwed you are...:-) Still, you're right of course, especially where things like SSE2 or Altivec aware compilers can do a lot of the optimizing for you if your code isn't too complex, or the issue is using ATLAS-tuned blas instead of a really shitty off the shelf blas. Still, there is something to be said for writing just plain sound and efficient code (in C:-) and using mundane -O-level optimization as a basis for how good a machine is, especially if you don't plan to study the microarchitecture layout and figure out if you can get four flops in the pipeline per clock, but only if you rearrange your code in a completely nonintuitive way so it does a multiply/add in a certain way... > regards, mark 'surly' hahn. (and to cheer you up..:-) rg-let-me-show-you-my-hand-tuned-8087-library-some-day-b -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb at phy.duke.edu _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rrstump at sandia.gov Thu Jun 26 21:04:23 2003 From: rrstump at sandia.gov (Roland Stumpf) Date: Thu, 26 Jun 2003 18:04:23 -0700 Subject: Intel PRO/1000CT Gigabit ethernet with CSA Message-ID: <3EFB9817.6090001@sandia.gov> Hi everybody ! We are configuring a ~40 node cluster for parallel materials modeling applications (e.g. VASP code from Vienna). My guess is that the sweet spot in terms of communication speed versus cost is with gigabit ethernet. Does anybody have an opinion on the new Intel PRO/1000CT Gigabit NIC that travels through the CSA (Communications Streaming Architecture) bus ? The Intel fact sheet is, as often, short on quantitative detail but it looks promising: http://www.intel.com/design/network/products/lan/controllers/82547ei.htm How does this NIC and a switch compare to Myrinet and similar networks ? We would be especially interested in an improvement of the latency for MPI communication over other Gigabit or 100Mbit ethernet. Another concern is if the faster GigE cards could saturate a fully connected modern 24 port switch like the Dell PowerConnect 5224, leading to dropped packages and crashing jobs. Thanks, Roland _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From daniel.pfenniger at obs.unige.ch Fri Jun 27 02:54:33 2003 From: daniel.pfenniger at obs.unige.ch (Daniel Pfenniger) Date: Fri, 27 Jun 2003 08:54:33 +0200 Subject: Intel PRO/1000CT Gigabit ethernet with CSA In-Reply-To: <3EFB9817.6090001@sandia.gov> References: <3EFB9817.6090001@sandia.gov> Message-ID: <3EFBEA29.60602@obs.unige.ch> Hi, For a small experimental cluster (24 dual Xeon nodes) we decided to use InfiniBand technology, which from specs is 4 times faster (8Gb/s), 1.5 lower latency (~5musec) than Myrinet for approximately the same cost/port. Our estimate is that with current 3GHz class processors one needs larger bandwith and lower latency than 2xGigaEthernet in order to make efficient use of the rest of the hardware, this for typical MPI type runs. Of course there exist always sufficiently coarse grained parallel applications for which GigaEthernet would be the good choice. At the moment there are very few >32 port fat-tree switches for InfiniBand, but 96-128 port swicthes should be available in the next months. Dan Roland Stumpf wrote: > Hi everybody ! > > We are configuring a ~40 node cluster for parallel materials modeling > applications (e.g. VASP code from Vienna). My guess is that the sweet > spot in terms of communication speed versus cost is with gigabit > ethernet. Does anybody have an opinion on the new Intel PRO/1000CT > Gigabit NIC that travels through the CSA (Communications > Streaming Architecture) bus ? The Intel fact sheet is, as often, short > on quantitative detail but it looks promising: > > http://www.intel.com/design/network/products/lan/controllers/82547ei.htm > > How does this NIC and a switch compare to Myrinet and similar networks ? > We would be especially interested in an improvement of the latency for > MPI communication over other Gigabit or 100Mbit ethernet. Another > concern is if the faster GigE cards could saturate a fully connected > modern 24 port switch like the Dell PowerConnect 5224, leading to > dropped packages and crashing jobs. > > Thanks, > Roland _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From graham.mullier at syngenta.com Fri Jun 27 05:28:55 2003 From: graham.mullier at syngenta.com (graham.mullier at syngenta.com) Date: Fri, 27 Jun 2003 10:28:55 +0100 Subject: Off topic - G5 question Message-ID: <0B27450D68F1D511993E0001FA7ED2B3025E50F9@ukjhmbx12.ukjh.zeneca.com> I think Rob made an excellent point. I'd like to paraphrase: optimising for a very unusual architecture, with poor support from the compilation (etc) tools, should not be undertaken lightly. I've seen examples in my time of very promising (=hyped) add-on hardware units for Vaxes, IBM mainframes, plus complex specialised graphics systems (E&S PS390) that either didn't live up to the promised performance, or where the effort involved to make use of the specialised hardware made it difficult to justify the work. Vax - the extra units seemed to offer performance similar to the strange new (to me at the time) Unix machines. So we bought an SGI server and once we'd got over the shock found it very useful because it was so flexible, with good compilation tools. IBM - we never saw the supposed performance improvement, and the only way to get calculations (large ab initio molecular orbital calculations) onto the system was to hand-deliver a tape (1/4-inch reel...). We suggested that the vector unit wasn't worth it for our work. E&S PS390 - goodwhen someone else wrote the graphics code, nightmare when it was someone in our group. He produced an excellent general visualisation system for 3D scatter plots but when we stopped using the hardware we had to throw away the code. Not a good use of resources. I prefer good general-purpose systems where other people help me by working on the tools necessary to make best use of the hardware. Graham Graham Mullier Chemoinformatics Team Leader, Chemistry Design Group, Syngenta, Bracknell, RG42 6EY, UK. direct line: +44 (0) 1344 414163 mailto:Graham.Mullier at syngenta.com -----Original Message----- From: Robert G. Brown [mailto:rgb at phy.duke.edu] Sent: 26 June 2003 23:12 To: Mark Hahn Cc: Beowulf Mailing List Subject: Re: Off topic - G5 question [...] Oh, that's ok. I freely admit to utter irresponsibility. Laziness too! I was (of course) at least partly tongue in cheek, although the nugget of truth in what I was saying is what I'd call the "CM5 phenomenon". [...] This kind of architecture makes me very nervous. Optimizing is no longer just a matter of picking good compiler flags or writing core loops sensibly (things that can easily be changed or that are good ideas on nearly any architecture); [...] It is this lack of portability that can, and I say can, be a killer to productivity and ultimately to one's ability to select freely from COTS architectures. Once the code is modified and optimized for (say) a G5, one has to take into account the ADDITIONAL cost of un/remodifying it and reoptimizing it for (say) an Opteron or Itanium or whateverisnextium. [...] _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From wrankin at ee.duke.edu Fri Jun 27 07:57:15 2003 From: wrankin at ee.duke.edu (Bill Rankin) Date: 27 Jun 2003 07:57:15 -0400 Subject: Job Posting - cluster admin. Message-ID: <1056715034.2172.21.camel@rohgun.cse.duke.edu> FYI - we are seeking a Beowulf admin for our university cluster. If you know of anyone that is interested, please forward them this information. The web page for Duke HR: http://www.hr.duke.edu/jobs/main.html Go to "Job Postings" and enter the folllowing requisition number. CAM45496-061203 Or just search for "Beowulf" in the job descriptions. Thanks, -bill -- dr. bill rankin .................................... research scientist wrankin at ee.duke.edu .......................... center for computational duke university ...................... science engineering and medicine http://www.ee.duke.edu/~wrankin ................... http://cse.duke.edu _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From mikee at mikee.ath.cx Fri Jun 27 10:13:00 2003 From: mikee at mikee.ath.cx (Mike Eggleston) Date: Fri, 27 Jun 2003 09:13:00 -0500 Subject: Job Posting - cluster admin. In-Reply-To: <1056715034.2172.21.camel@rohgun.cse.duke.edu>; from wrankin@ee.duke.edu on Fri, Jun 27, 2003 at 07:57:15AM -0400 References: <1056715034.2172.21.camel@rohgun.cse.duke.edu> Message-ID: <20030627091300.A18502@mikee.ath.cx> On Fri, 27 Jun 2003, Bill Rankin wrote: > FYI - we are seeking a Beowulf admin for our university cluster. If you > know of anyone that is interested, please forward them this information. > > The web page for Duke HR: > > http://www.hr.duke.edu/jobs/main.html > > Go to "Job Postings" and enter the folllowing requisition number. > > CAM45496-061203 > > Or just search for "Beowulf" in the job descriptions. > > Thanks, > > -bill > > -- > dr. bill rankin .................................... research scientist > wrankin at ee.duke.edu .......................... center for computational > duke university ...................... science engineering and medicine > http://www.ee.duke.edu/~wrankin ................... http://cse.duke.edu I'd love to. Can I do it from Dallas? Mike _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From becker at scyld.com Thu Jun 26 03:08:36 2003 From: becker at scyld.com (Donald Becker) Date: Thu, 26 Jun 2003 00:08:36 -0700 (PDT) Subject: 3Com 905C and ASUS P4GX8 In-Reply-To: <200306251346.h5PDkcF19321@accms33.physik.rwth-aachen.de> Message-ID: On Wed, 25 Jun 2003, Christoph Kukulies wrote: > I installed Linux RH 7.3 on a Dual 3GHz P4 CPU Asus P4GX8 MB > and since the on board 100 MBit controller didn't seem to be > supported by this Linux version (doubt whether any version does > in the moment) The Scyld distribution ships with our driver package, and almost certainly supports the card. But your problem is likely a configuration issue, not a driver failure. > I'm seeing this in /var/log/messages and the machine > is very slow on network copies and activity. What is the detection message? > eth0: Transmit error, Tx status register 82. Read http://www.scyld.com/network/vortex.html Specifically the section that says 0x82 Out of window collision. This typically occurs when some other Ethernet host is incorrectly set to full duplex on a half duplex network. -- Donald Becker becker at scyld.com Scyld Computing Corporation http://www.scyld.com 914 Bay Ridge Road, Suite 220 Scyld Beowulf cluster system Annapolis MD 21403 410-990-9993 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From daniel.pfenniger at obs.unige.ch Fri Jun 27 13:46:28 2003 From: daniel.pfenniger at obs.unige.ch (Daniel Pfenniger) Date: Fri, 27 Jun 2003 19:46:28 +0200 Subject: Intel PRO/1000CT Gigabit ethernet with CSA In-Reply-To: <200306271701.VAA12659@nocserv.free.net> References: <200306271701.VAA12659@nocserv.free.net> Message-ID: <3EFC82F4.6040502@obs.unige.ch> Mikhail Kuzminsky wrote: > According to Daniel Pfenniger > >>For a small experimental cluster (24 dual Xeon nodes) >>we decided to use InfiniBand technology, which from specs is >>4 times faster (8Gb/s), 1.5 lower latency (~5musec) than >>Myrinet for approximately the same cost/port. > > Could you pls compare them a bit more detailed ? > Infiniband card costs (as I heard) about $1000-, (HCA-Net from > FabricNetworks, former InfiniSwitch ?), what is close to Myrinet. > But what is about switches (I heard about high prices) ? > > In particular, I'm interesting in very small switches; > FabricNetworks produce 8-port 800-series switch, but I don't > know about prices. May be there is 6 or 4 port switches ? I don't know about such small switches. FabricNetworks list prices are ~10K$ for a 12 port switch, ~15K$ for a 24 port switch. But these 2 switches are not fully fat. A fully fat tree 32 port switch list price is estimated by me around 30-40K$, but it is not official because very recent (such a switch has been shown at the Heidelberg HPC conference these days). The exact costs are presently not well fixed because several companies enter the market. The nice thing about IB is that it is an open standard, the components from different companies are compatible, which is good for pressing costs down. > BTW, is it possible to connect pair of nodes by means of > "cross-over" cable (as in Ethernet), i.e. w/o switch ? Yes. The same copper cable as for the switch should do (but I have not tested yet). Each HCA has *two* ports, so you should be able to build a fully connected no hop 3-node cluster without a switch. With two HCA per nodes you might raise this number to 5 and still have a reasonable cost/node with respect to the switched solution. The 3m cables cost something around 300$ though, not completely negligible. Daniel Pfenniger _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From patrick at myri.com Fri Jun 27 15:00:36 2003 From: patrick at myri.com (Patrick Geoffray) Date: 27 Jun 2003 15:00:36 -0400 Subject: Intel PRO/1000CT Gigabit ethernet with CSA In-Reply-To: <3EFC82F4.6040502@obs.unige.ch> References: <200306271701.VAA12659@nocserv.free.net> <3EFC82F4.6040502@obs.unige.ch> Message-ID: <1056740437.548.101.camel@asterix> On Fri, 2003-06-27 at 13:46, Daniel Pfenniger wrote: > The exact costs are presently not well fixed because several companies > enter the market. The nice thing about IB is that it is an open > standard, the components from different companies are compatible, > which is good for pressing costs down. With the slicon coming from one company (actually 2 but the second one does only switch chip), the price adjustment would mainly affect the reseller, where the margin are not that high. I don't expect much a price war in the Infiniband market, mainly because many IB shops are already just burning (limited) VC cash. The main point for price advantage of IB is if the volume goes up. It's a very different problem that the multiple-vendors-marketing-stuff. One can argue that HPC does not yield such high volumes, only a business market like the Databases one does. Remember Gigabit Ethernet. It was very expensive when the early adopters were the HPC crowd and the price didn't drop until it made its way to the desktop. It's the case for 10GE today. Patrick -- Patrick Geoffray Myricom, Inc. http://www.myri.com _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From daniel.pfenniger at obs.unige.ch Fri Jun 27 15:52:51 2003 From: daniel.pfenniger at obs.unige.ch (Daniel Pfenniger) Date: Fri, 27 Jun 2003 21:52:51 +0200 Subject: Intel PRO/1000CT Gigabit ethernet with CSA In-Reply-To: <1056740437.548.101.camel@asterix> References: <200306271701.VAA12659@nocserv.free.net> <3EFC82F4.6040502@obs.unige.ch> <1056740437.548.101.camel@asterix> Message-ID: <3EFCA093.4090006@obs.unige.ch> Patrick Geoffray wrote: > On Fri, 2003-06-27 at 13:46, Daniel Pfenniger wrote: >>The exact costs are presently not well fixed because several companies >>enter the market. The nice thing about IB is that it is an open >>standard, the components from different companies are compatible, >>which is good for pressing costs down. > > > With the slicon coming from one company (actually 2 but the second one > does only switch chip), the price adjustment would mainly affect the > reseller, where the margin are not that high. I don't expect much a > price war in the Infiniband market, mainly because many IB shops are > already just burning (limited) VC cash. > The main point for price advantage of IB is if the volume goes up. It's > a very different problem that the multiple-vendors-marketing-stuff. One > can argue that HPC does not yield such high volumes, only a business > market like the Databases one does. > > Remember Gigabit Ethernet. It was very expensive when the early adopters > were the HPC crowd and the price didn't drop until it made its way to > the desktop. It's the case for 10GE today. > > Patrick > > Patrick Geoffray > Myricom, Inc. Yes I mostly agree with your analysis, database is the only significant potential market for IB. However the problem with 1GBE or 10GBE is that the latency remains poor for HPC applications, while IB goes in the right direction. The real comparison to be made is not between GE and IB, but between IB and Myricom products, which belong to an especially protected niche. As a result for years the Myrinet products did hardly drop in price for a sub-Moore's-law increase in performance, because of a lack of competition (the price we paid for our Myricom cards and switch 18 months ago is today *exactly* the same). Now IB at least introduces a healthy stimulation for this low latency HPC sector. Daniel _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From keith.murphy at attglobal.net Fri Jun 27 17:36:37 2003 From: keith.murphy at attglobal.net (Keith Murphy) Date: Fri, 27 Jun 2003 14:36:37 -0700 Subject: Intel PRO/1000CT Gigabit ethernet with CSA References: <200306271701.VAA12659@nocserv.free.net> <3EFC82F4.6040502@obs.unige.ch> <1056740437.548.101.camel@asterix> <3EFCA093.4090006@obs.unige.ch> Message-ID: <01a401c33cf4$306ef250$02fea8c0@oemcomputer> There have been competitive priced low latency interconnect cards to Myricom for some years including products from Quadrix and Dolphin. The reason for the lack of movement on pricing, is the high performance cluster interconnect is still a very small market I am sure that even in the very best of years the total yearly quantity of low latency interconnect cards and switches sold, never exceeds the daily quantity of Ethernet cards and switches sold. Price reductions are not reflected in Moore's law, unless accompanied by a subsequent expotentitial leap in the size of the market. Keith Murphy Dolphin Interconnect T: 818-597-2114 F: 818-597-2119 C: 818-292-5100 www.dolphinics.com www.scali.com ----- Original Message ----- From: "Daniel Pfenniger" To: "Patrick Geoffray" Cc: "Beowulf cluster mailing list" Sent: Friday, June 27, 2003 12:52 PM Subject: Re: Intel PRO/1000CT Gigabit ethernet with CSA > Patrick Geoffray wrote: > > On Fri, 2003-06-27 at 13:46, Daniel Pfenniger wrote: > > >>The exact costs are presently not well fixed because several companies > >>enter the market. The nice thing about IB is that it is an open > >>standard, the components from different companies are compatible, > >>which is good for pressing costs down. > > > > > > With the slicon coming from one company (actually 2 but the second one > > does only switch chip), the price adjustment would mainly affect the > > reseller, where the margin are not that high. I don't expect much a > > price war in the Infiniband market, mainly because many IB shops are > > already just burning (limited) VC cash. > > The main point for price advantage of IB is if the volume goes up. It's > > a very different problem that the multiple-vendors-marketing-stuff. One > > can argue that HPC does not yield such high volumes, only a business > > market like the Databases one does. > > > > Remember Gigabit Ethernet. It was very expensive when the early adopters > > were the HPC crowd and the price didn't drop until it made its way to > > the desktop. It's the case for 10GE today. > > > > Patrick > > > > Patrick Geoffray > > Myricom, Inc. > > Yes I mostly agree with your analysis, database is the only significant > potential market for IB. > > However the problem with 1GBE or 10GBE is that the latency remains poor > for HPC applications, while IB goes in the right direction. > The real comparison to be made is not between GE and IB, but between > IB and Myricom products, which belong to an especially protected niche. > As a result for years the Myrinet products did hardly drop in price > for a sub-Moore's-law increase in performance, because of a lack of > competition (the price we paid for our Myricom cards and switch > 18 months ago is today *exactly* the same). > Now IB at least introduces a healthy stimulation for this low latency > HPC sector. > > Daniel > > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org > To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf > _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rrstump at sandia.gov Thu Jun 26 12:36:07 2003 From: rrstump at sandia.gov (Roland Stumpf) Date: Thu, 26 Jun 2003 09:36:07 -0700 Subject: Intel PRO/1000CT Gigabit ethernet with CSA Message-ID: <3EFB20F7.20007@sandia.gov> Hi everybody ! We are configuring a ~40 node cluster for parallel materials modeling applications (e.g. VASP code from Vienna). My guess is that the sweet spot in terms of communication speed versus cost is with gigabit ethernet. Does anybody have an opinion on the new Intel PRO/1000CT Gigabit LAN controller that travels through the CSA (Communications Streaming Architecture) bus ? The Intel fact sheet is, as often, short on quantitative detail but it looks promising: http://www.intel.com/design/network/products/lan/controllers/82547ei.htm We would be especially interested in an improvement of the latency for MPI communication over other Gigabit or 100Mbit ethernet. Another concern is if the faster GigE cards could saturate a fully connected modern 24 port switch like the Dell PowerConnect 5224, leading to dropped packages and crashing jobs. Thanks, Roland _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From toon at moene.indiv.nluug.nl Fri Jun 27 14:50:14 2003 From: toon at moene.indiv.nluug.nl (Toon Moene) Date: Fri, 27 Jun 2003 20:50:14 +0200 Subject: [Fwd: Re: Off topic - G5 question] Message-ID: <3EFC91E6.2080701@moene.indiv.nluug.nl> Oops, forgot to "reply all". -- Toon Moene - mailto:toon at moene.indiv.nluug.nl - phoneto: +31 346 214290 Saturnushof 14, 3738 XG Maartensdijk, The Netherlands Maintainer, GNU Fortran 77: http://gcc.gnu.org/onlinedocs/g77_news.html GNU Fortran 95: http://gcc-g95.sourceforge.net/ (under construction) -------------- next part -------------- An embedded message was scrubbed... From: unknown sender Subject: no subject Date: no date Size: 1227 URL: From mof at labf.org Thu Jun 26 12:13:34 2003 From: mof at labf.org (Mof) Date: Fri, 27 Jun 2003 01:43:34 +0930 Subject: Off topic - G5 question In-Reply-To: References: Message-ID: <200306270143.35048.mof@labf.org> On Thu, 26 Jun 2003 11:34 pm, Robert G. Brown wrote: > On Wed, 25 Jun 2003, Luis Fernando Licon Padilla wrote: > > Hi you all guys, I've just checked the G5 processor out, what do you > > think about it, it seems like it's not good enough considering it's > > architecture and it's performance. > > > > http://www.apple.com/lae/g5 > > Looks great (if you believe their white papers) if you're planning to > run BLAST. Doesn't look horribly shabby at floating point performance > overall, has a nice peppy FSB. I certainly wouldn't kick it out of my > system room for chewing crackers. So to speak. Although I believe that > it requires some fairly careful programming to get maximum speed out of > the CPU and would be very curious about real-world benchmarks for code > that is just slapped on it and compiled, no tuning or hand > vectorization. > > However, the traditional litany requires that one examine price as well > as raw performance, and comparing it to P4 (really P6) class CPUs is > apples to oranges. A more interesting and fair comparison would be to > opterons and itaniums, and including price. An article talking about some of the problems with the benchmarks : http://www.extremetech.com/article2/0,3973,1136018,00.asp Mof. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From widyono at cis.upenn.edu Wed Jun 25 21:11:58 2003 From: widyono at cis.upenn.edu (Daniel Widyono) Date: Wed, 25 Jun 2003 21:11:58 -0400 Subject: 3Com 905C and ASUS P4GX8 In-Reply-To: <200306251346.h5PDkcF19321@accms33.physik.rwth-aachen.de> References: <200306251346.h5PDkcF19321@accms33.physik.rwth-aachen.de> Message-ID: <20030626011158.GB27312@central.cis.upenn.edu> > Jun 25 10:24:34 accms07 kernel: Probably a duplex mismatch. See Documentation/networking/vortex.txt > Any clues? Yes. The clue is spelled out in the error message. :) Regards, Dan W. -- -- Daniel Widyono http://www.cis.upenn.edu/~widyono -- Liniac Project, CIS Dept., SEAS, University of Pennsylvania -- Mail: CIS Dept, 302 Levine 3330 Walnut St Philadelphia, PA 19104 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From rmyers1400 at attbi.com Fri Jun 27 21:26:22 2003 From: rmyers1400 at attbi.com (Robert Myers) Date: Fri, 27 Jun 2003 21:26:22 -0400 Subject: Intel PRO/1000CT Gigabit ethernet with CSA (fwd) In-Reply-To: References: Message-ID: <3EFCEEBE.2060107@attbi.com> Donald Becker wrote: >This appears to connect directly to the internal motherboard chipset >crossbar, giving high-bandwidth access without (directly) conflicting >with the PCI bus. > >This architecture does reduce the bus access latency compared PCI, but >there will be no measurable reduction in network communication latency. > >If there were a Infiniband adapter connected to the same crossbar port, >you *would* be able to notice a significant reduction in single-word >transfer latency. > > There doesn't seem to be enough documentation anywhere even to decide whether such an enterprise is possible. CSA appears to be proprietary, so, Intel being Intel, you probably need a license to hook up to that port with an infiniband adapter of your own creation. Anyone have a clue otherwise? I feel almost silly asking, because every bit of evidence I've seen indicates that Intel is trying to create a competitive advantage for its own network silicon by giving it unique access to the MCH. RM RM _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From robl at mcs.anl.gov Fri Jun 27 21:50:56 2003 From: robl at mcs.anl.gov (Robert Latham) Date: Fri, 27 Jun 2003 20:50:56 -0500 Subject: Off topic - G5 question In-Reply-To: <3EFA19D1.7010009@yakko.cimav.edu.mx> References: <3EFA19D1.7010009@yakko.cimav.edu.mx> Message-ID: <20030628015056.GM9353@mcs.anl.gov> On Wed, Jun 25, 2003 at 03:53:21PM -0600, Luis Fernando Licon Padilla wrote: > Hi you all guys, I've just checked the G5 processor out, what do you > think about it, it seems like it's not good enough considering it's > architecture and it's performance. > > http://www.apple.com/lae/g5 a fast cpu is ok, but really (and i'm biased here :> ) IO is a much more limiting factor for high performance computing. so forget about the chip speeds and which one is faster... the G4 had pretty low heat dissipation. The itaniums are notorius for their heat dissipation (i've heard reports in the ballpark of 120 watts) while the G5 is reported to throw off 47 watts. But who can say what the real number will be when don't forget to factor operational costs into the price/performance calculation. ==rob -- Rob Latham Mathematics and Computer Science Division A215 0178 EA2D B059 8CDF Argonne National Labs, IL USA B29D F333 664A 4280 315B _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From becker at scyld.com Fri Jun 27 20:57:34 2003 From: becker at scyld.com (Donald Becker) Date: Fri, 27 Jun 2003 17:57:34 -0700 (PDT) Subject: Intel PRO/1000CT Gigabit ethernet with CSA (fwd) In-Reply-To: Message-ID: On Fri, 27 Jun 2003, Mark Hahn wrote: > Hi Don. do you happen to have any experience > with this new CSA feature - specifically whether it > actually lowers gigabit latency? This appears to connect directly to the internal motherboard chipset crossbar, giving high-bandwidth access without (directly) conflicting with the PCI bus. This architecture does reduce the bus access latency compared PCI, but there will be no measurable reduction in network communication latency. If there were a Infiniband adapter connected to the same crossbar port, you *would* be able to notice a significant reduction in single-word transfer latency. > Streaming Architecture) bus ? The Intel fact sheet is, as often, short > on quantitative detail but it looks promising: > > http://www.intel.com/design/network/products/lan/controllers/82547ei.htm Yup, this is very similar to the way earlier Intel Fast Ethernet chips connected to the Intel chipsets. If you look at the motherboards you'll see that those chips had significantly fewer pins than the PCI NICS, yet they appear to software to be a regular PCI device. > How does this NIC and a switch compare to Myrinet and similar > networks? Almost exactly the same as a 66Mhz PCI adapter. > modern 24 port switch like the Dell PowerConnect 5224, leading to > dropped packages and crashing jobs. You'll have the same constraints: Don't saturate with small packets Minimize multicast and broadcast packets Limit to low-rate service establishment (i.e. not back-to-back) Don't use for data transport. -- Donald Becker becker at scyld.com Scyld Computing Corporation http://www.scyld.com 914 Bay Ridge Road, Suite 220 Scyld Beowulf cluster system Annapolis MD 21403 410-990-9993 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From toon at moene.indiv.nluug.nl Sat Jun 28 06:13:03 2003 From: toon at moene.indiv.nluug.nl (Toon Moene) Date: Sat, 28 Jun 2003 12:13:03 +0200 Subject: [Fwd: Re: Off topic - G5 question] References: <3EFC91E6.2080701@moene.indiv.nluug.nl> Message-ID: <3EFD6A2F.50709@moene.indiv.nluug.nl> I wrote: > Well, I could make -O2 -ffast-math -funroll-loops the default (but then, > I don't think I'll get this past the IEEE754 police @ gcc.gnu.org :-) and Mark Hahn asked me (offline) whether -ffast-math is important. Well, as always: it depends. However, without -ffast-math, gcc and g77 zealously will follow IEEE 754 restrictions on floating point operations. Perhaps it's illustrative to see what that means by giving an example (culled from a bug report by one of Apple's compiler engineers). Consider this loop: DO IZ = 1, NZ DO IY = 1, NY DO IX = 1, NX CMAP(IX,IY,IZ) = DCONJG(CMAP(IX,IY,IZ)) / DFLOAT(NX*NY*NZ) END DO END DO END DO with I[XYZ], N[XYZ] integers and CMAP a double precision complex rank-3 array. DCONJG is expanded inline no matter what, so that's a rather basic operation. However, without -ffast-math, g77 is required to convert the double precision real DFLOAT(NX*NY*NZ) - a loop invariant - to double complex and than _perform a double complex division_ inside the loop. With -ffast-math, all that remains is two multiplications with the inverse of DFLOAT(NX*NY*NZ) for the real and imaginary parts of CMAP(IX,IY,IZ). -- Toon Moene - mailto:toon at moene.indiv.nluug.nl - phoneto: +31 346 214290 Saturnushof 14, 3738 XG Maartensdijk, The Netherlands Maintainer, GNU Fortran 77: http://gcc.gnu.org/onlinedocs/g77_news.html GNU Fortran 95: http://gcc-g95.sourceforge.net/ (under construction) _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From kuku at physik.rwth-aachen.de Sat Jun 28 03:24:17 2003 From: kuku at physik.rwth-aachen.de (Christoph P. Kukulies) Date: Sat, 28 Jun 2003 09:24:17 +0200 Subject: 3Com 905C and ASUS P4GX8 In-Reply-To: References: <200306251346.h5PDkcF19321@accms33.physik.rwth-aachen.de> Message-ID: <20030628072417.GA8289@gilberto.physik.rwth-aachen.de> On Thu, Jun 26, 2003 at 12:08:36AM -0700, Donald Becker wrote: > On Wed, 25 Jun 2003, Christoph Kukulies wrote: > > > I installed Linux RH 7.3 on a Dual 3GHz P4 CPU Asus P4GX8 MB > > and since the on board 100 MBit controller didn't seem to be > > supported by this Linux version (doubt whether any version does > > in the moment) > > The Scyld distribution ships with our driver package, and almost > certainly supports the card. But your problem is likely a configuration > issue, not a driver failure. > > > I'm seeing this in /var/log/messages and the machine > > is very slow on network copies and activity. > > What is the detection message? Jun 23 08:46:11 accms07 kernel: PCI: Found IRQ 9 for device 02:00.0 Jun 23 08:46:11 accms07 kernel: 3c59x: Donald Becker and others. www.scyld.com/network/vortex.html Jun 23 08:46:11 accms07 kernel: See Documentation/networking/vortex.txt Jun 23 08:46:11 accms07 kernel: 02:00.0: 3Com PCI 3c905C Tornado at 0xa800. Vers LK1.1.18-ac Jun 23 08:46:11 accms07 kernel: 00:01:02:1e:db:74, IRQ 9 Jun 23 08:46:11 accms07 kernel: product code 4552 rev 00.13 date 05-06-00 Jun 23 08:46:11 accms07 kernel: 8K byte-wide RAM 5:3 Rx:Tx split, autoselect/Autonegotiate interface. Jun 23 08:46:11 accms07 kernel: MII transceiver found at address 24, status 782d. Jun 23 08:46:11 accms07 kernel: Enabling bus-master transmits and whole-frame receives. Jun 23 08:46:11 accms07 kernel: 02:00.0: scatter/gather enabled. h/w checksums enabled Jun 23 08:46:11 accms07 kernel: Linux agpgart interface v0.99 (c) Jeff Hartmann > > > eth0: Transmit error, Tx status register 82. > > Read http://www.scyld.com/network/vortex.html > Specifically the section that says > 0x82 Out of window collision. This typically occurs when > some other Ethernet host is incorrectly set to full duplex on a half duplex > network. > > Thanks. I will check if the switch might be set to HD fixed instead of FD Auto. -- Chris Christoph P. U. Kukulies kukulies (at) rwth-aachen.de _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From hahn at physics.mcmaster.ca Sun Jun 29 12:37:45 2003 From: hahn at physics.mcmaster.ca (Mark Hahn) Date: Sun, 29 Jun 2003 12:37:45 -0400 (EDT) Subject: interconnect latency, disected. In-Reply-To: <200306291414.SAA12281@nocserv.free.net> Message-ID: does anyone have references handy for recent work on interconnect latency? specifically, I'm noticing two things: - many of the papers I see that contrast IB with GBE seem to claim that GBE latency is much larger than I measure (20-30 us). - I haven't found any recent studies on where even 25 us of GBE latency comes from. I recall a really great study from years ago that broke a single tx/rx down into an explicit timeline, but that was for ~100 MIP CPUs and 100bT, I think. - are there current/active efforts to use something other than TCP for implementing the usual MPI primitives? I'd love to see something that used ethernet's broad/multicast support, for instance. thanks, mark. _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From lindahl at keyresearch.com Sun Jun 29 15:25:33 2003 From: lindahl at keyresearch.com (Greg Lindahl) Date: Sun, 29 Jun 2003 12:25:33 -0700 Subject: interconnect latency, disected. In-Reply-To: References: <200306291414.SAA12281@nocserv.free.net> Message-ID: <20030629192533.GA11418@greglaptop.greghome.keyresearch.com> On Sun, Jun 29, 2003 at 12:37:45PM -0400, Mark Hahn wrote: > - I haven't found any recent studies on where even 25 us of > GBE latency comes from. store-and-forward swithes? It's entertaining to measure back-to-back latency, but not very realistic. > - are there current/active efforts to use something other than > TCP for implementing the usual MPI primitives? I'd love to see > something that used ethernet's broad/multicast support, for instance. GAMMA and MVIA don't use TCP, but I don't think either has that. What kind of savings do you expect, given the mismatch between unreliable broadcast & multicast and MPI's semantics? I see a lot more enthusiasm for such features from people who haven't looked at the details... greg _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From fmahr at gmx.de Sun Jun 29 17:24:19 2003 From: fmahr at gmx.de (Ferdinand Mahr) Date: Sun, 29 Jun 2003 23:24:19 +0200 Subject: interconnect latency, disected. References: Message-ID: <3EFF5903.4E4E9D54@gmx.de> Hi everyone, Mark Hahn wrote: > > does anyone have references handy for recent work on interconnect latency? ...that's almost what I'm currently looking for. Does anyone have those references for a wider variety of interconnection technologies? I'm thinking of FE, GE, Myrinet, Dolphin, IB, Quadrics... I don't need the complete results from, say, a PALLAS MPI Benchmark run, but maybe the latency of a ping or something similar. It's not for a buying decision, but just to have a clue. Thanks, Ferdinand _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From sp at scali.com Sun Jun 29 16:22:33 2003 From: sp at scali.com (Steffen Persvold) Date: Sun, 29 Jun 2003 22:22:33 +0200 (CEST) Subject: Intel PRO/1000CT Gigabit ethernet with CSA In-Reply-To: <200306291414.SAA12281@nocserv.free.net> Message-ID: On Sun, 29 Jun 103, Mikhail Kuzminsky wrote: > According to Daniel Pfenniger > > Patrick Geoffray wrote: > > > On Fri, 2003-06-27 at 13:46, Daniel Pfenniger wrote: > > >>The exact costs are presently not well fixed because several companies > > >>enter the market. The nice thing about IB is that it is an open > > >>standard, the components from different companies are compatible, > > >>which is good for pressing costs down. > > > > > > With the slicon coming from one company (actually 2 but the second one > > > does only switch chip), the price adjustment would mainly affect the > > > reseller, where the margin are not that high. I don't expect much a > > > price war in the Infiniband market, mainly because many IB shops are > > > already just burning (limited) VC cash. > > > The main point for price advantage of IB is if the volume goes up. It's > > > a very different problem that the multiple-vendors-marketing-stuff. One > > > can argue that HPC does not yield such high volumes, only a business > > > market like the Databases one does. > > > > > > Remember Gigabit Ethernet. It was very expensive when the early adopters > > > were the HPC crowd and the price didn't drop until it made its way to > > > the desktop. It's the case for 10GE today. > > > ... > > > Patrick Geoffray > > > Myricom, Inc. > > > > Yes I mostly agree with your analysis, database is the only significant > > potential market for IB. > > > > However the problem with 1GBE or 10GBE is that the latency remains poor > > for HPC applications, while IB goes in the right direction. > > The real comparison to be made is not between GE and IB, but between > > IB and Myricom products, which belong to an especially protected niche. > > As a result for years the Myrinet products did hardly drop in price > > for a sub-Moore's-law increase in performance, because of a lack of > > competition (the price we paid for our Myricom cards and switch > > 18 months ago is today *exactly* the same). > > I agree with you both. From the viewpoint of HPC clusters the IB > competitor is Myrinet (and SCI etc). But there are many applications > w/coarse-grained parallelism, where bandwidth is the main thing, not the > latency (I think, quantum chemistry applications are bandwidth- > limited). In this case (i.e. if latnecy is less important) 10Gb Ethernet > is also IB competitor. Moreover, IB, I beleive, will be used for > TCP/IP connections also - in opposition to Myrinet etc. (I beleive > there is no TCP/IP drivers for Myrinet - am I correct ?) No, actually there is. The GM driver emulates an normal ethernet device the same way as IB does (and SCI). However this is not the strength of these high speed interconnects, and not the optimal way of using them (seen from our HPC/MPI perspective). > > Again, from the veiwpoint of some real appilications, there are some > applications which > use TCP/IP stack for parallelization (I agree that is bad, but ...) > - for example Linda tools (used in Gaussian) work over TCP/IP, Gamess-US > DDI "subsystem" works over TCP/IP. In the case of IB or 10Gb Ethernet > TCP/IP is possible. > I believe the current TCP/IP implementation in Linux is not capable of utilizing the possible bandwidth a 10Gb Ethernet connection can provide. Do do that you would need new protocols utilizing RDMA (doing true zerocopy as we do in MPI). And then of course you are into problems like userspace memory pinning etc... Regards, -- Steffen Persvold ,,, mailto: sp at scali.com Senior Software Engineer (o-o) http://www.scali.com -----------------------------oOO-(_)-OOo----------------------------- Scali AS, PObox 150, Oppsal, N-0619 Oslo, Norway, Tel: +4792484511 _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From mprinkey at aeolusresearch.com Sun Jun 29 18:59:06 2003 From: mprinkey at aeolusresearch.com (Michael T. Prinkey) Date: Sun, 29 Jun 2003 18:59:06 -0400 (EDT) Subject: interconnect latency, disected. In-Reply-To: References: Message-ID: <50772.192.168.1.3.1056927546.squirrel@ra.aeolustec.com> There is at least two MPI Ethernet implementations that try to use transfer layers other than TCP: http://www.nersc.gov/research/FTG/mvich/ http://www.disi.unige.it/project/gamma/mpigamma/ Both claim ~10-13 usec latencies for older Gbit NICs. MVICH seems to be dead. Gamma is still not SMP safe. Both are pretty short on hardware support. AFAIK, neither support either of the Gbit NICs I have been using (tg3 and e1000). I recently tested Netgear GA622Ts end to end with netpipe. Latencies were roughly 22 usecs for packets <100 bytes, bandwidth was respectable (540 Mbps for 8k packets, 850 Mbps for ~100k packets). Latencies for Intel e1000 though a cheap 16-port switch were ~63 usecs. Bandwidth was lower across the board through the switch. Although I haven't done thorough testing, my best guess is that most of the latency is coming from the switch, so TCP vs UDP vs raw ethernet frames might be moot. > > does anyone have references handy for recent work on interconnect > latency? > > specifically, I'm noticing two things: > > - many of the papers I see that contrast IB with GBE seem to > claim that GBE latency is much larger than I measure (20-30 us). > > - I haven't found any recent studies on where even 25 us of > GBE latency comes from. I recall a really great study from years ago > that broke a single tx/rx down into an explicit timeline, > but that was for ~100 MIP CPUs and 100bT, I think. > > - are there current/active efforts to use something other than > TCP for implementing the usual MPI primitives? I'd love to see > something that used ethernet's broad/multicast support, for instance. > > thanks, mark. > > _______________________________________________ > Beowulf mailing list, Beowulf at beowulf.org > To change your subscription (digest mode or unsubscribe) visit > http://www.beowulf.org/mailman/listinfo/beowulf _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From jcookeman at yahoo.com Sun Jun 29 23:20:16 2003 From: jcookeman at yahoo.com (Justin Cook) Date: Sun, 29 Jun 2003 20:20:16 -0700 (PDT) Subject: Hard link /etc/passwd Message-ID: <20030630032016.88507.qmail@web10607.mail.yahoo.com> Good day, I have an 11 node diskless cluster. All slave node roots are under /tftpboot/node1 ... /tftpboot/node2 ... so on. Is it safe to hard link the /etc/passwd and /etc/group file to the server nodes for consistency across the network? __________________________________ Do you Yahoo!? SBC Yahoo! DSL - Now only $29.95 per month! http://sbc.yahoo.com _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From jcownie at etnus.com Mon Jun 30 05:30:37 2003 From: jcownie at etnus.com (James Cownie) Date: Mon, 30 Jun 2003 10:30:37 +0100 Subject: [Fwd: Re: Off topic - G5 question] In-Reply-To: Message from Toon Moene of "Fri, 27 Jun 2003 20:50:14 +0200." <3EFC91E6.2080701@moene.indiv.nluug.nl> Message-ID: <19Wuzd-1Xe-00@etnus.com> Toon Moene wrote :- > Mark Hahn wrote: > > > I still see users fling their code at a machine with > > g77 prog.f && ./a.out > > and wonder why it's so slow. > > Well, I could make -O2 -ffast-math -funroll-loops the default (but then, > I don't think I'll get this past the IEEE754 police @ gcc.gnu.org :-) However there's nothing to stop Mark (assuming he has control of the machine) from setting the system up so that the "g77" you get by default is really a shell script which turns on appropriate optimisation flags, and also providing a command either to compile unoptimised, or to change the users $PATH so that they can get to a "raw" g77. I know that this is the approach used by Nick McLaren on the big Sun machines at Cambridge. http://www.hpcf.cam.ac.uk/prog_franklin_compilers.html Depending on the interests of your users (are they interested in their application domain, or in becoming compiler switch experts), they may prefer a system where someone has already defaulted the compiler to use an apporpiate set of magic optimisation flags. -- Jim James Cownie Etnus, LLC. +44 117 9071438 http://www.etnus.com _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From jhearns at freesolutions.net Mon Jun 30 05:39:33 2003 From: jhearns at freesolutions.net (John Hearns) Date: Mon, 30 Jun 2003 10:39:33 +0100 Subject: mpiBLAST article on The Register Message-ID: <3F000555.6000706@freesolutions.net> I think this has already been on the Beowulf list, so forgive me if that's so. An interesting article on MPI enabled BLAST on The Register: http://www.theregister.co.uk/content/61/31471.html "The 240 processor Linux cluster helped show that some scientific computing tasks will run with adequate performance and incredible reliability on a system that can fit in an average closest.(sic)" _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From tekka99 at libero.it Mon Jun 30 06:07:21 2003 From: tekka99 at libero.it (=?iso-8859-1?Q?Gianluca_Cecchi?=) Date: Mon, 30 Jun 2003 12:07:21 +0200 Subject: =?iso-8859-1?Q?Re:Hard_link_/etc/passwd?= Message-ID: > Good day, > I have an 11 node diskless cluster. All slave node > roots are under /tftpboot/node1 ... /tftpboot/node2 > ... so on. Is it safe to hard link the /etc/passwd > and /etc/group file to the server nodes for > consistency across the network? Apart from the beowulf related srategy, take in mind that hard links cannot traverse filesystems.. Bye, Gianluca _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From jcownie at etnus.com Mon Jun 30 05:00:16 2003 From: jcownie at etnus.com (James Cownie) Date: Mon, 30 Jun 2003 10:00:16 +0100 Subject: interconnect latency, dissected. In-Reply-To: Message from Ferdinand Mahr of "Sun, 29 Jun 2003 23:24:19 +0200." <3EFF5903.4E4E9D54@gmx.de> Message-ID: <19WuWG-1Uo-00@etnus.com> Mark Hahn wrote: > > does anyone have references handy for recent work on interconnect latency? Try http://www.cs.berkeley.edu/~bonachea/upc/netperf.pdf It doesn't have Inifinband, but does have Quadrics, Myrinet 2000, GigE and IBM. -- Jim James Cownie Etnus, LLC. +44 117 9071438 http://www.etnus.com _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From wrankin at ee.duke.edu Mon Jun 30 11:48:19 2003 From: wrankin at ee.duke.edu (Bill Rankin) Date: 30 Jun 2003 11:48:19 -0400 Subject: Job Posting - cluster admin. In-Reply-To: <20030627091300.A18502@mikee.ath.cx> References: <1056715034.2172.21.camel@rohgun.cse.duke.edu> <20030627091300.A18502@mikee.ath.cx> Message-ID: <1056988099.3706.20.camel@rohgun.cse.duke.edu> > I'd love to. Can I do it from Dallas? It would be a heck of a commute each morning. :-) -b _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From dieter at engr.uky.edu Mon Jun 30 17:15:21 2003 From: dieter at engr.uky.edu (William Dieter) Date: Mon, 30 Jun 2003 17:15:21 -0400 Subject: Hard link /etc/passwd In-Reply-To: <20030630032016.88507.qmail@web10607.mail.yahoo.com> Message-ID: You have to be careful when doing maintenance. For example, if you do: mv /etc/passwd /etc/passwd.bak cp /etc/passwd.bak /etc/passwd all of the copies will be linked to the backup copy. Normally you might not do this, but some text editors sometimes do similar things silently... A symbolic link might be safer. Bill. On Sunday, June 29, 2003, at 11:20 PM, Justin Cook wrote: > Good day, > I have an 11 node diskless cluster. All slave node > roots are under /tftpboot/node1 ... /tftpboot/node2 > ... so on. Is it safe to hard link the /etc/passwd > and /etc/group file to the server nodes for > consistency across the network? _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From clwang at csis.hku.hk Mon Jun 23 03:32:03 2003 From: clwang at csis.hku.hk (Cho-Li Wang) Date: Mon, 23 Jun 2003 15:32:03 +0800 Subject: Cluster2003: Final Call for Paper Message-ID: <3EF6ACF3.10D63009@csis.hku.hk> Dear Colleagues: Due to numerous requests from participants and especially the good news that Hong Kong is now removed from the SARS affected areas by WHO at 3:00pm today (June 23, Hong Kong time), we are pleased to further extend the submission deadline to June 30, 2003. If you have not submitted your paper to Cluster2003 yet, please consider to submit it before the new deadline. Best Regards, Cho-Li Wang and Daniel S. Katz Cluster2003 program Co-Chairs ----------------------------------------------------------------- ***************************************************** * Deadline further extended to June 30, 2003 (Firm) * ***************************************************** Cluster 2003 IEEE International Conference on Cluster Computing December 1-4, 2003, Hong Kong URL: http://www.csis.hku.hk/cluster2003/ Organized by Department of Computer Science and Information Systems, The University of Hong Kong Clusters today are an important element of mainstream computing. In less than a decade, "lowly" computer clusters have come to prominence as the most convenient and cost-effective tool for solving many complex computational problems. Being able to bring enough computing cycles together in a pool is one thing, but to be able to use these cycles efficiently so that the aggregate power of a cluster can be fully harnessed is another. Research and development efforts must continue to explore better strategies to make clusters an even greater tool when facing the grandest challenges of this age. Cluster 2003, to be held in the most dynamic city in the Orient, will provide an open forum for cluster researchers and system builders, as well as cluster users, to present and discuss new directions and results that could shape the future of cluster computing. Cluster 2003 welcomes paper submissions from engineers and scientists in academia, industry, and government describing their latest research findings in any aspects of cluster computing. Topics of interest include, but are not limited to: Cluster Middleware -- Single-System Image Services -- Software Environments and Tools -- Standard Software for Clusters -- I/O Libraries, File Systems, and Distributed RAID Cluster Networking -- High-Speed System Interconnects -- Lightweight Communication Protocols -- Fast Message Passing Libraries Managing Clusters -- Cluster Security and Reliability -- Tools for Managing Clusters -- Cluster Job and Resource Management -- High-Availability Cluster Solutions Applications -- Scientific and E-Commerce Applications -- Data Distribution and Load Balancing -- Innovative Cluster Applications Performance Analysis and Evaluation -- Benchmarking and Profiling Tools -- Performance Prediction and Modeling -- Analysis and Visualization Grid Computing and Clusters -- Grid Applications Integration -- Network-Based Distributed Computing -- Mobile Agents and Java for Clusters/Grids -- Middleware, Tools and Environments Paper Submission: Paper Format: Not to exceed 25 double-spaced pages (including figures, tables and references) using 10-12 point font on 8.5 x 11-inch pages. Show page number at each page. Include an abstract, five to ten keywords, and the corresponding author's e-mail address. Electronic Submission: Web-based submissions are strongly encouraged. The URL will be announced two weeks before the submission deadline at the conference web site. Authors should submit a PostScript (level 2) or PDF file. Postal submissions are not accepted. Journal Special Issue: Authors of the best papers from the conference will be invited to submit an expand version of their papers for possible publication in a special issue of the International Journal of High Performance Computing and Networking (IJHPCN), which will appear in 2004. Tutorials: For more information about proposals for organizing tutorials, please contact Tutorials Chair Ira Pramanick (ira.pramanick at sun.com) by June 30, 2003. Exhibitions: Companies interested in presenting exhibits at the meeting should contact one of the Exhibits Co-Chairs - Jim Ang (jaang at sandia.gov) or Nam Ng (Nng at cc.hku.hk) by August 31, 2003. Important Dates: -- Papers Submission Deadline: June 30, 2003 (Firm) -- Tutorial Proposals Due: June 30, 2003 -- Author Notification: (TBA) -- Exhibition Proposal Due: August 31, 2003 -- Final Manuscript Due: (TBA) -- Pre-Conference Tutorials: December 1, 2003 -- Conference: December 2-4, 2003 Proceedings of the conference and workshops will be available at the conference. ------------------ Chairs/Committees ------------------ General Co-Chairs Jack Dongarra (University of Tennessee) Lionel Ni (Hong Kong University of Science and Technology) General Vice Chair Francis C.M. Lau (The University of Hong Kong) Program Co-Chairs Daniel S. Katz (Jet Propulsion Laboratory) Cho-Li Wang (The University of Hong Kong) Program Vice Chairs Bill Gropp (Argonne National Laboratory) -- Middleware Wolfgang Rehm (Technische Universita"t Chemnitz) -- Hardware Zhiwei Xu (Chinese Academy of Sciences, China) -- Applications Tutorials Chair Ira Pramanick (Sun Microsystems) Workshops Chair Jiannong Cao (Hong Kong Polytechnic University) Exhibits/Sponsors Co-Chairs Jim Ang (Sandia National Lab) Nam Ng (The University of Hong Kong) Publications Chair Rajkumar Buyya (The University of Melbourne) Publicity Chair Arthur B. Maccabe (The University of New Mexico) Poster Chair Putchong Uthayopas (Kasetsart University) Finance/Registration Chair Alvin Chan (Hong Kong Polytechnic University) Local Arrangements Chair Anthony T.C. Tam (The University of Hong Kong) For further information please send e-mail to: cluster2003 at csis.hku.hk ************************************************************* To unsubscribe from srg-announce: send e-mail to: majordomo at csis.hku.hk, subject: anything, with content: unsubscribe srg-announce ************************************************************* _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From clwang at csis.hku.hk Mon Jun 23 03:32:03 2003 From: clwang at csis.hku.hk (Cho-Li Wang) Date: Mon, 23 Jun 2003 15:32:03 +0800 Subject: Cluster2003: Final Call for Paper Message-ID: <3EF6ACF3.10D63009@csis.hku.hk> Dear Colleagues: Due to numerous requests from participants and especially the good news that Hong Kong is now removed from the SARS affected areas by WHO at 3:00pm today (June 23, Hong Kong time), we are pleased to further extend the submission deadline to June 30, 2003. If you have not submitted your paper to Cluster2003 yet, please consider to submit it before the new deadline. Best Regards, Cho-Li Wang and Daniel S. Katz Cluster2003 program Co-Chairs ----------------------------------------------------------------- ***************************************************** * Deadline further extended to June 30, 2003 (Firm) * ***************************************************** Cluster 2003 IEEE International Conference on Cluster Computing December 1-4, 2003, Hong Kong URL: http://www.csis.hku.hk/cluster2003/ Organized by Department of Computer Science and Information Systems, The University of Hong Kong Clusters today are an important element of mainstream computing. In less than a decade, "lowly" computer clusters have come to prominence as the most convenient and cost-effective tool for solving many complex computational problems. Being able to bring enough computing cycles together in a pool is one thing, but to be able to use these cycles efficiently so that the aggregate power of a cluster can be fully harnessed is another. Research and development efforts must continue to explore better strategies to make clusters an even greater tool when facing the grandest challenges of this age. Cluster 2003, to be held in the most dynamic city in the Orient, will provide an open forum for cluster researchers and system builders, as well as cluster users, to present and discuss new directions and results that could shape the future of cluster computing. Cluster 2003 welcomes paper submissions from engineers and scientists in academia, industry, and government describing their latest research findings in any aspects of cluster computing. Topics of interest include, but are not limited to: Cluster Middleware -- Single-System Image Services -- Software Environments and Tools -- Standard Software for Clusters -- I/O Libraries, File Systems, and Distributed RAID Cluster Networking -- High-Speed System Interconnects -- Lightweight Communication Protocols -- Fast Message Passing Libraries Managing Clusters -- Cluster Security and Reliability -- Tools for Managing Clusters -- Cluster Job and Resource Management -- High-Availability Cluster Solutions Applications -- Scientific and E-Commerce Applications -- Data Distribution and Load Balancing -- Innovative Cluster Applications Performance Analysis and Evaluation -- Benchmarking and Profiling Tools -- Performance Prediction and Modeling -- Analysis and Visualization Grid Computing and Clusters -- Grid Applications Integration -- Network-Based Distributed Computing -- Mobile Agents and Java for Clusters/Grids -- Middleware, Tools and Environments Paper Submission: Paper Format: Not to exceed 25 double-spaced pages (including figures, tables and references) using 10-12 point font on 8.5 x 11-inch pages. Show page number at each page. Include an abstract, five to ten keywords, and the corresponding author's e-mail address. Electronic Submission: Web-based submissions are strongly encouraged. The URL will be announced two weeks before the submission deadline at the conference web site. Authors should submit a PostScript (level 2) or PDF file. Postal submissions are not accepted. Journal Special Issue: Authors of the best papers from the conference will be invited to submit an expand version of their papers for possible publication in a special issue of the International Journal of High Performance Computing and Networking (IJHPCN), which will appear in 2004. Tutorials: For more information about proposals for organizing tutorials, please contact Tutorials Chair Ira Pramanick (ira.pramanick at sun.com) by June 30, 2003. Exhibitions: Companies interested in presenting exhibits at the meeting should contact one of the Exhibits Co-Chairs - Jim Ang (jaang at sandia.gov) or Nam Ng (Nng at cc.hku.hk) by August 31, 2003. Important Dates: -- Papers Submission Deadline: June 30, 2003 (Firm) -- Tutorial Proposals Due: June 30, 2003 -- Author Notification: (TBA) -- Exhibition Proposal Due: August 31, 2003 -- Final Manuscript Due: (TBA) -- Pre-Conference Tutorials: December 1, 2003 -- Conference: December 2-4, 2003 Proceedings of the conference and workshops will be available at the conference. ------------------ Chairs/Committees ------------------ General Co-Chairs Jack Dongarra (University of Tennessee) Lionel Ni (Hong Kong University of Science and Technology) General Vice Chair Francis C.M. Lau (The University of Hong Kong) Program Co-Chairs Daniel S. Katz (Jet Propulsion Laboratory) Cho-Li Wang (The University of Hong Kong) Program Vice Chairs Bill Gropp (Argonne National Laboratory) -- Middleware Wolfgang Rehm (Technische Universita"t Chemnitz) -- Hardware Zhiwei Xu (Chinese Academy of Sciences, China) -- Applications Tutorials Chair Ira Pramanick (Sun Microsystems) Workshops Chair Jiannong Cao (Hong Kong Polytechnic University) Exhibits/Sponsors Co-Chairs Jim Ang (Sandia National Lab) Nam Ng (The University of Hong Kong) Publications Chair Rajkumar Buyya (The University of Melbourne) Publicity Chair Arthur B. Maccabe (The University of New Mexico) Poster Chair Putchong Uthayopas (Kasetsart University) Finance/Registration Chair Alvin Chan (Hong Kong Polytechnic University) Local Arrangements Chair Anthony T.C. Tam (The University of Hong Kong) For further information please send e-mail to: cluster2003 at csis.hku.hk ************************************************************* To unsubscribe from srg-announce: send e-mail to: majordomo at csis.hku.hk, subject: anything, with content: unsubscribe srg-announce ************************************************************* _______________________________________________ Beowulf mailing list, Beowulf at beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf