From deadline at plogic.com Mon Aug 7 16:54:08 2000 From: deadline at plogic.com (Douglas Eadline) Date: Mon, 7 Aug 2000 16:54:08 -0400 (EDT) Subject: parAgent (auto/assist parallelizing program) anyone? In-Reply-To: <398F16ED.8EA56CAC@systemsbiology.org> Message-ID: On Mon, 7 Aug 2000, Jimmy Eng wrote: > Sorry - the correct URL is > http://darwin.ee.iastate.edu/~srl/ > > Jimmy Eng wrote: > > > > Does anyone have any experience or comments/thoughts on > > Parallelization Agent (parAgent)? It's software to assist > > in parallelizing fortran77 code developed at Iowa State: > > http://darwin.ww.iastate.edu/~srl/ > > > > As much as I can tell from briefly flipping through the web > > pages, there are similarities to Bert77 from www.plogic.com > > mentioned in the Beowulf FAQ. I have looked at the parAgent pages and can give a quick summary of the differences. 1) BERT uses profile information about the target hardware. Profile information is used so BERT can make good decisions about what concurrent parts of the program SHOULD be run in parallel for a given hardware platform. In general, not all concurrent parts of a program produce speed-up when run in parallel. It is not clear how parAgent will decide (other than the user) what parts to parallelize. 2) BERT uses a thin layer to interface to either PVM or MPI. BERT currently can choose from two "parallelization" models (dataflow and dynamic) depending on the application and the profile information. BERT works on clusters. parAgent only supports the IBM SP and Cray T3 with an SPMD model. 3) parAgent seems to use some heuristic knowledge about the program (provided by the user) while BERT pretty much works "below the algorithm". 4) BERT can give pre-runtime estimates of parallel efficiencies. This can be very effective for both old codes and new ones (new algorithm development) with out having to run codes. We will be introducing a static (data sliced) model for BERT later this year. From our experience with, old codes (sorry legacy) the important thing is to determine where your parallelization efforts will pay-off. BERT allows this question to be answered first instead of investing several weeks of trial and error. I hope this helps. But I think it is difficult to compare these types of tools unless you play with them a bit. Doug ------------------------------------------------------------------- Paralogic, Inc. | PEAK | Voice:+610.814.2800 130 Webster Street | PARALLEL | Fax:+610.814.5844 Bethlehem, PA 18015 USA | PERFORMANCE | http://www.plogic.com ------------------------------------------------------------------- _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From auro at nietzsche.mit.edu Mon Aug 7 19:03:41 2000 From: auro at nietzsche.mit.edu (Stefano Curtarolo) Date: Mon, 7 Aug 2000 18:03:41 -0500 (EST) Subject: best motherboard In-Reply-To: Message-ID: Hi we are thinking to create a new beowulf with DUAL P3 cpus (as fast as possible). I know that there are a lot of motherboards but I am confused by all the different specifications. Which is the best motherboard ? Any advice is welcome ! Thanks Stefano Curtarolo _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From astroguy at bellsouth.net Mon Aug 7 22:50:56 2000 From: astroguy at bellsouth.net (Lucky) Date: Mon, 07 Aug 2000 22:50:56 -0400 Subject: best motherboard Message-ID: <398F7590.455AE87D@bellsouth.net> Hi Stefano, Stefano Curtarolo wrote: > Hi > > we are thinking to create a new beowulf with DUAL P3 cpus (as fast > as possible). I know that there are a lot of motherboards but I am > confused by all the different specifications. > Which is the best motherboard ? Any advice is welcome ! > > Thanks > > Stefano Curtarolo > > _______________________________________________ > Beowulf mailing list > Beowulf at beowulf.org > http://www.beowulf.org/mailman/listinfo/beowulf I know you asked for the best... And the fastest for P3. I can't really say but for the money it's hard to beat the Tyan Tiger 100. It's sweet and can take a gig of ram. 100mhz on your bus thru put and with 2 P2's it can deliver a processor speed of appox 450meghrtz. If you are just looking to get your feet wet with some of this stuff, it provides a lot of entertainment for a fraction of where the P3 are at the moment. The board itself is about a buck and a half and you can find the 233 (512K on L-2) for about 40 bucks a throw... So for less than you can buy one P3 (last time I check they were holding around 275)... You can have a bad ass dual 233 and a three ring day at the circus for a fraction of the ticket of admission. In fact, you could buy 3 Tigers loaded for what you would spend on just the two P3's, so if you are looking for cost to speed, it might be an interesting project to set those 3 Tiger nodes and you have performance that exceeds the returns of the single dual P3... Benching a Gigahzs speeds at a fraction of the cost. Of course that IS what makes all this beowufery stuff soooo much fun. Hope this helps, Chip _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From mark at rrsg.ee.uct.ac.za Tue Aug 8 11:05:05 2000 From: mark at rrsg.ee.uct.ac.za (Mark Gebhardt) Date: Tue, 08 Aug 2000 17:05:05 +0200 Subject: African Clusters Message-ID: <399021A1.343D5145@rrsg.ee.uct.ac.za> I am very interested to know how many Beowulf (or other) clusters are in Africa (especially South Africa). This technology, with its low cost of ownership is ideally suited to African economies where there is little money available in research or company budgets. At the University of Cape Town, we have an 8-node cluster which has been (and continues to be) used in research for industry in SA. There is also work being done at UCT in the field of weather systems analysis using a clustered architecture. I would appreciate any information or links which list subscribers may have. Thanks. Regards Mark Gebhardt _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From eugene.leitl at lrz.uni-muenchen.de Tue Aug 8 17:17:03 2000 From: eugene.leitl at lrz.uni-muenchen.de (Eugene Leitl) Date: Tue, 8 Aug 2000 14:17:03 -0700 (PDT) Subject: turbolinux.com Message-ID: <14736.30927.184222.422591@lrz.uni-muenchen.de> Sorry for nontechnical content, but... What is your opinion about http://www.turbolinux.com/ ? A flash in the pan, or worth investing in? Any other investment alternatives? _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From eugene.leitl at lrz.uni-muenchen.de Tue Aug 8 21:12:13 2000 From: eugene.leitl at lrz.uni-muenchen.de (Eugene Leitl) Date: Tue, 8 Aug 2000 18:12:13 -0700 (PDT) Subject: clustered Linux in rendering farms and global derivatives risk management Message-ID: <14736.45037.698498.775328@lrz.uni-muenchen.de> http://www.hoise.com/primeur/00/articles/monthly/AE-PR-08-00-29.html San Francisco 18 July 2000 TurboLinux Inc., the High-performance Linux company TurboLinux, announced that J.P. Morgan & Co. Incorporated has deployed TurboLinux's EnFuzion software to help power the firm's worldwide risk management system for fixed income derivatives on a cluster of more than 1,000 nodes. [...] http://www.linuxnews.com/stories.php?story=102 X-Men's Special Effects Team Moving to Maya--and Linux [...] _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From eugene.leitl at lrz.uni-muenchen.de Tue Aug 8 21:12:13 2000 From: eugene.leitl at lrz.uni-muenchen.de (Eugene Leitl) Date: Tue, 8 Aug 2000 18:12:13 -0700 (PDT) Subject: clustered Linux in rendering farms and global derivatives risk management Message-ID: <14736.45037.698498.775328@lrz.uni-muenchen.de> http://www.hoise.com/primeur/00/articles/monthly/AE-PR-08-00-29.html San Francisco 18 July 2000 TurboLinux Inc., the High-performance Linux company TurboLinux, announced that J.P. Morgan & Co. Incorporated has deployed TurboLinux's EnFuzion software to help power the firm's worldwide risk management system for fixed income derivatives on a cluster of more than 1,000 nodes. [...] http://www.linuxnews.com/stories.php?story=102 X-Men's Special Effects Team Moving to Maya--and Linux [...] _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From indraneel at www.cdfd.org.in Wed Aug 9 12:15:21 2000 From: indraneel at www.cdfd.org.in (Indraneel Majumdar) Date: Wed, 9 Aug 2000 09:15:21 -0700 (PDT) Subject: turbolinux.com In-Reply-To: <14736.30927.184222.422591@lrz.uni-muenchen.de> Message-ID: Can someone explain how parametric processing is done? On Tue, 8 Aug 2000, Eugene Leitl wrote: > > Sorry for nontechnical content, but... > > What is your opinion about http://www.turbolinux.com/ ? > > A flash in the pan, or worth investing in? Any other investment > alternatives? > > _______________________________________________ > Beowulf mailing list > Beowulf at beowulf.org > http://www.beowulf.org/mailman/listinfo/beowulf > /************************************************************************. # Indraneel Majumdar ? E-mail: indraneel at 123india.com # # Bioinformatics Unit (EMBNET node), ? URL: http://scorpius.iwarp.com # # Centre for DNA Fingerprinting and Diagnostics, # # Hyderabad, India - 500076 # `************************************************************************/ _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From zolia at lydys.sc-uni.ktu.lt Wed Aug 9 01:35:23 2000 From: zolia at lydys.sc-uni.ktu.lt (zolia) Date: Wed, 9 Aug 2000 08:35:23 +0300 (EEST) Subject: books Message-ID: hi, could anybody tell me what is must reed books for parallel programming and cluster theory? One i know for sure: "Designing and Building Parallel Programs" :) Wich are currently in book stores? ==================================================================== Antanas Masevicius Kaunas University of Technology Studentu 48a-101 Computer Center LT-3028 Kaunas LITNET NOC UNIX Systems Administrator Lithuania E-mail: zolia at sc.ktu.lt _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From indraneel at www.cdfd.org.in Thu Aug 10 12:33:11 2000 From: indraneel at www.cdfd.org.in (Indraneel Majumdar) Date: Thu, 10 Aug 2000 09:33:11 -0700 (PDT) Subject: turbolinux.com (product: enfusion) In-Reply-To: <200008092146.QAA11582@mail5.doit.wisc.edu> Message-ID: Hi, The reason I asked was that TurboLinux's enfusion is supposed to do that. They say you don't need to rewrite code. So if it is embarassingly parallel how is it different from pvm? (I'm a bio student with no maths background) Is it that the same program is run with different initial parameters on different proessors and then you guess an approximate result? If so, then how does one ensure that the individual runs are shorter than the whole? You'll probably need to chose the initial parameters carefully for that, so how is that done? I guess you'll have to know the program algorithm to do that, and I assume that every program has a different algorithm. Are there things like generic algorithm analysers? I'm into bioinformatics, and am trying to find out whether things like enfusion (which TurboLinux targets at protein modelling) might be created inhouse. So I need to know how it works. Can you explain or give me any links for detailed answers? Thanks very much, Indraneel On Wed, 9 Aug 2000, Erik Paulson wrote: > You run the same program a whole bunch of times, each time with different > inputs. (See also Monte Carlo simulation) > > It's a polite way of saying that you've got embarrassingly parallel work :) > > -Erik > > > /************************************************************************. # Indraneel Majumdar ? E-mail: indraneel at 123india.com # # Bioinformatics Unit (EMBNET node), ? URL: http://scorpius.iwarp.com # # Centre for DNA Fingerprinting and Diagnostics, # # Hyderabad, India - 500076 # `************************************************************************/ _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From epaulson at students.wisc.edu Wed Aug 9 17:46:25 2000 From: epaulson at students.wisc.edu (Erik Paulson) Date: Wed, 09 Aug 2000 16:46:25 CDT Subject: turbolinux.com In-Reply-To: References: Message-ID: <200008092146.QAA11582@mail5.doit.wisc.edu> On Wed, 9 Aug 2000, Indraneel Majumdar wrote: > Date: Wed, 9 Aug 2000 09:15:21 -0700 (PDT) > To: Eugene Leitl > From: Indraneel Majumdar > Subject: Re: turbolinux.com > > Can someone explain how parametric processing is done? > > You run the same program a whole bunch of times, each time with different inputs. (See also Monte Carlo simulation) It's a polite way of saying that you've got embarrassingly parallel work :) -Erik _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From deadline at plogic.com Thu Aug 10 11:16:33 2000 From: deadline at plogic.com (Douglas Eadline) Date: Thu, 10 Aug 2000 11:16:33 -0400 (EDT) Subject: turbolinux.com (product: enfusion) In-Reply-To: Message-ID: On Thu, 10 Aug 2000, Indraneel Majumdar wrote: > Hi, > > The reason I asked was that TurboLinux's enfusion is supposed to do that. > They say you don't need to rewrite code. If you want to use more than one processor for a single sequential program, you need to modify code. (or use code modified for multiple processors). There is no "magic bullet". (wish there was) > So if it is embarassingly > parallel how is it different from pvm? (I'm a bio student with no maths > background) Is it that the same program is run with different initial > parameters on different proessors and then you guess an approximate > result? If so, then how does one ensure that the individual runs are > shorter than the whole? You'll probably need to chose the initial > parameters carefully for that, so how is that done? I guess you'll have to > know the program algorithm to do that, and I assume that every program has > a different algorithm. Are there things like generic algorithm analysers? > Not yet. A generic algorithm analyser would be nice, but just think how hard it is to figure out what someones C program is trying to do. > I'm into bioinformatics, and am trying to find out whether things like > enfusion (which TurboLinux targets at protein modelling) might be created > inhouse. So I need to know how it works. Can you explain or give me any > links for detailed answers? Doug ------------------------------------------------------------------- Paralogic, Inc. | PEAK | Voice:+610.814.2800 130 Webster Street | PARALLEL | Fax:+610.814.5844 Bethlehem, PA 18015 USA | PERFORMANCE | http://www.plogic.com ------------------------------------------------------------------- _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From deadline at plogic.com Thu Aug 10 11:16:33 2000 From: deadline at plogic.com (Douglas Eadline) Date: Thu, 10 Aug 2000 11:16:33 -0400 (EDT) Subject: turbolinux.com (product: enfusion) In-Reply-To: Message-ID: On Thu, 10 Aug 2000, Indraneel Majumdar wrote: > Hi, > > The reason I asked was that TurboLinux's enfusion is supposed to do that. > They say you don't need to rewrite code. If you want to use more than one processor for a single sequential program, you need to modify code. (or use code modified for multiple processors). There is no "magic bullet". (wish there was) > So if it is embarassingly > parallel how is it different from pvm? (I'm a bio student with no maths > background) Is it that the same program is run with different initial > parameters on different proessors and then you guess an approximate > result? If so, then how does one ensure that the individual runs are > shorter than the whole? You'll probably need to chose the initial > parameters carefully for that, so how is that done? I guess you'll have to > know the program algorithm to do that, and I assume that every program has > a different algorithm. Are there things like generic algorithm analysers? > Not yet. A generic algorithm analyser would be nice, but just think how hard it is to figure out what someones C program is trying to do. > I'm into bioinformatics, and am trying to find out whether things like > enfusion (which TurboLinux targets at protein modelling) might be created > inhouse. So I need to know how it works. Can you explain or give me any > links for detailed answers? Doug ------------------------------------------------------------------- Paralogic, Inc. | PEAK | Voice:+610.814.2800 130 Webster Street | PARALLEL | Fax:+610.814.5844 Bethlehem, PA 18015 USA | PERFORMANCE | http://www.plogic.com ------------------------------------------------------------------- _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From epaulson at students.wisc.edu Thu Aug 10 12:31:27 2000 From: epaulson at students.wisc.edu (Erik Paulson) Date: Thu, 10 Aug 2000 11:31:27 CDT Subject: turbolinux.com (product: enfusion) In-Reply-To: References: Message-ID: <200008101631.LAA67506@mail5.doit.wisc.edu> On Thu, 10 Aug 2000, Indraneel Majumdar wrote: > Date: Thu, 10 Aug 2000 09:33:11 -0700 (PDT) > To: Erik Paulson > From: Indraneel Majumdar > Subject: Re: turbolinux.com (product: enfusion) > > Hi, > > The reason I asked was that TurboLinux's enfusion is supposed to do that. > I don't think that parametric processing is a term that has really caught on yet. We call it high-throughput computing.... > > They say you don't need to rewrite code. So if it is embarassingly > parallel how is it different from pvm? (I'm a bio student with no maths > background) PVM is a message-passing library. Embarassingly parallel is a class of problems. You can use PVM to solve embarassingly parallel problems, but you can oftentimes get away with a simpler solution. (Incidently, around here we're supposed to use "naturally parallel" instead of "embarassingly parallel" - I guess the funding sources aren't always so hip on anything with "embarassingly" in the description) > Is it that the same program is run with different initial > parameters on different proessors and then you guess an approximate > result? No, you run it multiple times with different parameters looking at all the answers. For example, one of our users does engine simulations. He's got a program that simulates one stroke of his engine that takes about an hour to run. What he needs to know is what mixture of fuel and air gives the best results, so he's going to experimentally try them all (I think he's going to try like 5,000 different combinations or something) enfusion (or in our case Condor) will run them all, and give him the answer in 5000/ hours. This all comes with no changes to his code, so it basically scales right up with the number of machines availble (something I'm sure a number of people on this list wish their problems did :) > If so, then how does one ensure that the individual runs are > shorter than the whole? You'll probably need to chose the initial > parameters carefully for that, so how is that done? I guess you'll have > to > know the program algorithm to do that. Yup. I would have no idea how to choose the parameters for the engine run. (I just turn the key and go in my car) > and I assume that every program > has > a different algorithm. Are there things like generic algorithm analysers? > No, and there are boring-cs-theory reasons why there aren't. > > I'm into bioinformatics, and am trying to find out whether things like > enfusion (which TurboLinux targets at protein modelling) might be created > inhouse. So I need to know how it works. Can you explain or give me any > links for detailed answers? > You can create a VERY basic system with {r,s}sh and perl in 20 minutes. It gets harder as you start adding features :) -Erik _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From indraneel at www.cdfd.org.in Fri Aug 11 03:18:56 2000 From: indraneel at www.cdfd.org.in (Indraneel Majumdar) Date: Fri, 11 Aug 2000 00:18:56 -0700 (PDT) Subject: turbolinux.com (product: enfusion) In-Reply-To: <3992C6F0.301BB167@mscsoftware.com> Message-ID: I have some more queries which I have inserted in between the text. (Please remember that I seem to have less maths background than you are assuming, but I'm trying to understand) Thanks for your replies. On Thu, 10 Aug 2000, David Lombard wrote: > > EnFuzion is a "design scan" program. That is, it runs tens, hundreds, > thousands, &etc of jobs to exhaustively search a design space. No > modifications are required to the program being run, as it's run under > control of a script. What exactly is a design scan? Is it the entire range of possible input and output values for an algorithm? > > But don't fool yourself into thinking you now have a "parallel" program > -- you don't. > > What you do have is a tool for trying many combinations of input > parameters as quickly as your resources will allow. You could run these > cases by hand (ouch) or with some scripts that use rsh(1) and rcp(1). > > The typical lowest level process for a design scan is: > > Specify the design variables and their ranges. How do I specify the variables and ranges without knowing the algorithm? Generally I'll have only one set of inputs. Do I have to run the program previously with different sets of inputs so that it can learn (similiar to neural networks) the possible algorithm (or derive a simplified relation between input and output)? > Create a script that takes as input one value for each design variable, > runs the program, and extracts whatever results are appropriate. How does it know what results are appropriate? The program then also needs an error checking routine (reminds me of genetic algorithms). > Run the script with every combination of design variables and save the > output in some form for post-processing. The parallel processing comes > from running the combinations on different systems. As stated in an > earlier post, this is the most embarrassingly parallel processing you > can do as each run is completely independent from every other run. > > Tools like EnFuzion, iSIGHT, and others (including my own company's) > offer varying levels of sophistication beyond this. Running the > combinations is the easy part; achieving the desired result *without* > running all the combinations, is the hard part. > > So, if you have a long-running program that you need to run faster, this > is not the right approach. If, on the other hand, you have a situation > that is suitable to a design scan, then the method, if not the products, > can help. > > -- > David N. Lombard > MSC.Software > Thanks, Indraneel /************************************************************************. # Indraneel Majumdar ? E-mail: indraneel at 123india.com # # Bioinformatics Unit (EMBNET node), ? URL: http://scorpius.iwarp.com # # Centre for DNA Fingerprinting and Diagnostics, # # Hyderabad, India - 500076 # `************************************************************************/ _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From troy at osc.edu Thu Aug 10 14:57:13 2000 From: troy at osc.edu (Troy Baer) Date: Thu, 10 Aug 2000 14:57:13 -0400 Subject: lperfex 0.5 available Message-ID: Hi all, I'm happy to announce that lperfex 0.5 is now available. Lperfex is a hardware performance monitoring tool for Linux/IA32 systems, based on Erik Hendriks' performance patch. If you've used Cray's hpm or SGI's perfex, then lperfex should seem fairly familiar. If not, think of lperfex as a variation on the time command which can also track low-level hardware events like floating point operations, cache misses, and so on. It is not intrusive into the code whose performance it measures and does not require special compilation or code instrumentation. Lperfex 0.5 fixes a number of bugs in the version 0.3 release, particularly in command line argument handling. Bug reports and suggestions for new features are welcomed. Documentation and code for lperfex is available from http://www.osc.edu/~troy/lperfex/. The code is licensed under the GPL. To use it, you need to be running Linux with Erik Hendriks' performance counters patch and library v0.7 (available from http://www.beowulf.org/software/) on an Intel P6 core processor (i.e. PPro/P2/P3/Xeon/Celeron). The libperf library and Linux kernel patches that are distributed with recent versions of the PerfAPI code should also be compatible, although I haven't had a chance to test that yet. --Troy -- Troy Baer email: troy at osc.edu Science & Technology Support phone: 614-292-9701 Ohio Supercomputer Center web: http://oscinfo.osc.edu _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From epaulson at students.wisc.edu Thu Aug 10 15:40:58 2000 From: epaulson at students.wisc.edu (Erik Paulson) Date: Thu, 10 Aug 2000 14:40:58 CDT Subject: turbolinux.com (product: enfusion) In-Reply-To: References: Message-ID: <200008101940.OAA05094@mail5.doit.wisc.edu> On Fri, 11 Aug 2000, Indraneel Majumdar wrote: > Date: Fri, 11 Aug 2000 00:18:56 -0700 (PDT) > To: David Lombard > From: Indraneel Majumdar > Subject: Re: turbolinux.com (product: enfusion) > > I have some more queries which I have inserted in between the text. > (Please remember that I seem to have less maths background than you are > assuming, but I'm trying to understand) Thanks for your replies. > > On Thu, 10 Aug 2000, David Lombard wrote: > > > > EnFuzion is a "design scan" program. That is, it runs tens, hundreds, > > thousands, &etc of jobs to exhaustively search a design space. No > > modifications are required to the program being run, as it's run under > > control of a script. > > What exactly is a design scan? Is it the entire range of possible input > and output values for an algorithm? > No. It's merely trying out as many different inputs as you've got the time/resources/interest in trying. > > > > But don't fool yourself into thinking you now have a "parallel" program > > -- you don't. > > > > What you do have is a tool for trying many combinations of input > > parameters as quickly as your resources will allow. You could run > these > > cases by hand (ouch) or with some scripts that use rsh(1) and rcp(1). > > > > The typical lowest level process for a design scan is: > > > > Specify the design variables and their ranges. > > How do I specify the variables and ranges without knowing the algorithm? You don't. > Generally I'll have only one set of inputs. Do I have to run the program > previously with different sets of inputs so that it can learn (similiar > to > neural networks) the possible algorithm (or derive a simplified relation > between input and output)? You're expecting too much from this program. All it does is run your program many many times. If you had it run "Hello, World" a million times, it would. Enfusion is not going to figure out that it's running "Hello, World" and optimize itself for doing that. > > > Create a script that takes as input one value for each design variable, > > runs the program, and extracts whatever results are appropriate. > > How does it know what results are appropriate? It doesn't. > The program then also needs > an error checking routine (reminds me of genetic algorithms). > That's totally beyond the scope of these sorts of systems. -Erik _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From indraneel at www.cdfd.org.in Fri Aug 11 04:12:28 2000 From: indraneel at www.cdfd.org.in (Indraneel Majumdar) Date: Fri, 11 Aug 2000 01:12:28 -0700 (PDT) Subject: turbolinux.com (product: enfusion) In-Reply-To: <200008101631.LAA67506@mail5.doit.wisc.edu> Message-ID: On Thu, 10 Aug 2000, Erik Paulson wrote: > > No, you run it multiple times with different parameters looking at all the > answers. > to try like 5,000 different combinations or something) > enfusion (or in our case Condor) will run them all, and give him the answer > in 5000/ hours. This all comes with no > So if a program has to be run multiple times with different values as inputs I use enfusion (or condor or whatever..) which runs the whole program on each node but with different input parameters (so parametric processing) and if a program has to run only once but can be made modular, then I use message passing and run each module on a different node. So a modular program that uses message passing, if required to run several times can also make use of enfusion (or condor etc.). Am I right or way off the mark? > > You can create a VERY basic system with {r,s}sh and perl in 20 minutes. > It gets harder as you start adding features :) > If I am right then this would mean the protein modelling program was putting only the accepted possible bond angle values at each point of the protein chain and checking for total free energy or whatever. So I can run the same program several times on different nodes with different initial bond angle values and this would be parametric processing. Am I right? > -Erik > > /************************************************************************. # Indraneel Majumdar ? E-mail: indraneel at 123india.com # # Bioinformatics Unit (EMBNET node), ? URL: http://scorpius.iwarp.com # # Centre for DNA Fingerprinting and Diagnostics, # # Hyderabad, India - 500076 # `************************************************************************/ _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From shi at physics.ucsd.edu Fri Aug 11 12:44:34 2000 From: shi at physics.ucsd.edu (xiangdong shi) Date: Fri, 11 Aug 2000 09:44:34 -0700 (PDT) Subject: Newbie need help on MPICH install Message-ID: I am testing MPICH on four machines running COREL LINUX 1.1 (2.2.14 kernel). I have set up the network such that they can do rsh without prompting for password. When I configure MPICH on the one the machine I plan to use as server, however, the log complains checking whether /usr/bin/rsh works ... no Errors while trying to run true on mpi0 with rsh mpi0: Connection refused mpi0 is the name of the machine. I also ran test on a system running Red Hat 6.2, getting the same error. Please help. Thanks. Xiangdong _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From Lechner at drs-esg.com Fri Aug 11 13:04:26 2000 From: Lechner at drs-esg.com (Lechner, David) Date: Fri, 11 Aug 2000 13:04:26 -0400 Subject: Intel performance with loops vs. vectors - Message-ID: Has anyone done any direct comparison of performance of modern Intel or AMD processors running floating point operations as both vector operations and as simple loops? Does it make a difference any more? Some of the results posted to this list seem to imply that loop calculations now run as fast as vector op.s on today's microprocessors. We are considering some tests but would appreciate any insight or comment people would be willing to provide. W/Thanks & Regards/ Dave Lechner. _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From david.lombard at mscsoftware.com Fri Aug 11 14:36:01 2000 From: david.lombard at mscsoftware.com (David Lombard) Date: Fri, 11 Aug 2000 11:36:01 -0700 Subject: Intel performance with loops vs. vectors - References: Message-ID: <39944791.DFD9A560@mscsoftware.com> "Lechner, David" wrote: > > Has anyone done any direct comparison of performance of modern Intel or AMD > processors running floating point operations as both vector operations and > as simple loops? Does it make a difference any more? Some of the results > posted to this list seem to imply that loop calculations now run as fast as > vector op.s on today's microprocessors. We are considering some tests but > would appreciate any insight or comment people would be willing to provide. > I'm not quite sure what your question is. A "vector operation" implies the hardware has some sort of vector instructions. Intel has MMX and KNI instructions that provide specific operations for very short vectors, i.e., 4x32-bit, &etc. But, neither Intel nor AMD have a general purpose vector capability such as found on Cray or NEC systems. By "vector operations", do you mean, for example, calling a BLAS operation vs simply coding the loop directly? If you do, then the first point is that you've badly abused the terminology. At any rate, there could be an advantage using BLAS or other library routines if the library routines have advanced coding, either by guiding the compiler, such as is done by Atlas, or by writing the function in assembly language (as we do for our MSC.Nastran product), or both. Another possible interpretation is: are today's Intel and AMD processors as fast as vector systems? For that, one must apply the standard answer, "it depends". If you have heavy integer and scalar operations or other "poor" vector situations, then Intel can beat both Cray and NEC. If you have something that vectorizes well, you could hit 97% peak theoretical on a T90 (as do we on a matrix-matrix multiply), and Intel is very much slower. -- David N. Lombard MSC.Software _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From levenson at valinux.com Fri Aug 11 16:56:37 2000 From: levenson at valinux.com (Howard Levenson) Date: Fri, 11 Aug 2000 16:56:37 -0400 Subject: BWBUG Meeting Aug 22nd. New Location! Message-ID: <39946885.3284770E@valinux.com> Hi, The next meeting of the Baltimore-Washington Beowulf User Group will be on August 22nd at 3:00 P.M. Please note the new meeting location at Logicon DPC in Laurel, MD. Directions can be found at: http://bwbug.tux.org Speaker: Greg Lindahl Topic: Bleeding Edge Linux Supercomputing Greg Lindahl and HPTi have been involved with several big Linux cluster efforts. One of their recent activities has been a partnership with Sandia National Lab to integrate CPlant, which is the fastest Linux cluster. Can CPlant's phase III break a teraflop? Details can be found at: http://www.hpti.com http://www.cs.sandia.gov/cplant A recent interview with Greg at Slashdot: http://slashdot.org/interviews/00/06/01/133232.shtml Hope to see you there! _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From morton at cs.umt.edu Sat Aug 12 20:32:33 2000 From: morton at cs.umt.edu (Don Morton) Date: Sat, 12 Aug 2000 18:32:33 -0600 Subject: Facilities Requirements for Beowulf Clusters? Message-ID: <3995ECA1.F6FBCA4A@cs.umt.edu> Howdy, for years I've been running a 9-node cluster of 100MHz Pentiums, and I've gotten away with stuffing them into a single room with no additional power or air conditioning considerations. I'm in a position now where I'll be getting 16 to 32 nodes, high-end, along with a high-end server, and I'm kind of worrying that a typical room's power system and, particularly, the air conditioning, might not be adequate. I realise I can calculate the power requirements and compare them to what's available in the room, but A/C is a little tougher. Have any of you tried jamming 16 to 32 nodes in a small room that's typically kept at around 22C (72F) with any success? I hear about a number of medium-sized systems like the one I'm getting ready to acquire, and I wonder if people are really going all out to spend extra bucks on the facilities? Thanks, Don Morton -- Don Morton http://www.cs.umt.edu/u/morton/ Department of Computer Science The University of Montana Missoula, MT 59812 | Voice (406) 243-4975 | Fax (406) 243-5139 _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From czarek at scheraga2.chem.cornell.edu Sun Aug 13 00:33:07 2000 From: czarek at scheraga2.chem.cornell.edu (Cezary Czaplewski) Date: Sun, 13 Aug 2000 00:33:07 -0400 (EDT) Subject: Facilities Requirements for Beowulf Clusters? In-Reply-To: <3995ECA1.F6FBCA4A@cs.umt.edu> Message-ID: Hi, We have 55 dual PIII 650 nodes in quite small room (to be precise part of bigger room which divided by two metal cabinets into small 'computer room' and a workplace with a few desks and desktop computers). While staying close to the rack you can feel heat produced by all these computer, maximum CPU's temperature as measured using lm-sensor is around 55C. As the limit in Intel's documents is 82C for 650 MHz CPU I think it is OK. Earlier this room had been kept around 22C, after setting up cluster I asked to increase air flow for A/C in this room and it was enough not to overheat CPU's. czarek > I realize I can calculate the power requirements and compare them > to what's available in the room, but A/C is a little tougher. Have > any of you tried jamming 16 to 32 nodes in a small room that's typically > kept at around 22C (72F) with any success? I hear about a number of > medium-sized systems like the one I'm getting ready to acquire, > and I wonder if people are really going all out to spend extra > bucks on the facilities? > > Thanks, > > Don Morton > -- > Don Morton http://www.cs.umt.edu/u/morton/ > Department of Computer Science The University of Montana > Missoula, MT 59812 | Voice (406) 243-4975 | Fax (406) 243-5139 > > _______________________________________________ > Beowulf mailing list > Beowulf at beowulf.org > http://www.beowulf.org/mailman/listinfo/beowulf > _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From zolia at lydys.sc-uni.ktu.lt Sun Aug 13 13:04:26 2000 From: zolia at lydys.sc-uni.ktu.lt (zolia) Date: Sun, 13 Aug 2000 20:04:26 +0300 (EEST) Subject: probs running mpqc Message-ID: hello, then i trying to run mpqc on MPI i get error like rcmd: node1: Success p0_8592: p4_error: Timeout in making connection to remote process on node1: 0 bm_list_8593: p4_error: interrupt SIGINT: 2 strace shows following: connect(8, {sin_family=AF_INET, sin_port=htons(753), ^^^^^^^^^ sin_addr=inet_addr("193.219.61.66")}}, 16) = -1 ECONNREFUSED (Connection refused) fork() = 8612 rt_sigaction(SIGALRM, NULL, {SIG_DFL}, 8) = 0 rt_sigaction(SIGALRM, {0x40023f04, [ALRM], 0x4000000}, NULL, 8) = 0 setitimer(ITIMER_REAL, {it_interval={0, 0}, it_value={300, 0}}, NULL) = 0 accept(6, rcmd: node1: Success 0xbfffdd1c, [16]) = ? ERESTARTSYS (To be restarted) --- SIGCHLD (Child exited) --- accept(6, and hangs until timeout. It seems that it tries to connect to 753 port, but that is neither rlogin, nor rexec or rsh port. .p4pg contains: host 0 node1 1 ==================================================================== Antanas Masevicius Kaunas University of Technology Studentu 48a-101 Computer Center LT-3028 Kaunas LITNET NOC UNIX Systems Administrator Lithuania E-mail: zolia at sc.ktu.lt _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From zolia at lydys.sc-uni.ktu.lt Mon Aug 14 00:30:13 2000 From: zolia at lydys.sc-uni.ktu.lt (zolia) Date: Mon, 14 Aug 2000 07:30:13 +0300 (EEST) Subject: probs running mpqc In-Reply-To: <20000814005042.18236.qmail@aros.ca.sandia.gov> Message-ID: hello, in this case i'm stuck not with calculations, but with establishing connection with other then localhost. As i mentioned in the earlier posting , sys call trace shows that 753 port is tried to be opened, but thera are nothing on that end. Maybe there is something very bad with my env configurationt and starting method ? My env for running mpqc on two nodes is : ----------- script to run----------- #!/bin/sh export Nproc=2, export MessageGrp=MPIMessageGrp export NPROC=2; export MESSAGEGRP=":( debug = 1 argv = [ -execer_id DQS -master_host lydys -my_hostname lydys -my_nodenum 0 -my_numprocs 2 -total_numnodes 2 # ???? -p4pg .p4pg # ???? -job_id 100 ] )"; export GIVENMESSAGEGRP="mpich"; export SCLIBDIR=/usr/src/mpqc/mpqc-1.2.3/lib /usr/src/mpqc/mpqc-1.2.3/src/bin/mpqc/mpqc $* ------------- end of script ------------ and run it to test smth like : lydys:/usr/src/mpqc/mpqc-1.2.3$ mpirun -np 2 ./runmpqc-mpipro -d -f src/bin/mpqc/sample/scfopt.in but this doesn't start any proces on other machine. When i create .p4pg file: lydys 0 perlas 1 [occures that problem with 753 port:] mpqc: unrecognized option -p4pg MPIMessageGrp::init: entered Calling MPI_Init with -execer_id DQS -master_host lydys -my_hostname lydys -my_nodenum 0 -my_numprocs 2 -total_numnodes 2 -p4pg .p4pg -job_id 100 rcmd: perlas.sc-uni.ktu.lt: Success p0_12456: p4_error: Timeout in making connection to remote process on perlas.sc-uni.ktu.lt: 0 bm_list_12457: p4_error: interrupt SIGINT: 2 [ and sys trace again: ] connect(8, {sin_family=AF_INET, sin_port=htons(753), sin_addr=inet_addr("193.219.61.66")}}, 16) = -1 ECONNREFUSED (Connection refused) fork() = 12654 rt_sigaction(SIGALRM, NULL, {SIG_DFL}, 8) = 0 rt_sigaction(SIGALRM, {0x40023f04, [ALRM], 0x4000000}, NULL, 8) = 0 setitimer(ITIMER_REAL, {it_interval={0, 0}, it_value={300, 0}}, NULL) = 0 accept(6, rcmd: perlas.sc-uni.ktu.lt: Success 0xbfffdb6c, [16]) = ? ERESTARTSYS (To be restarted) --- SIGCHLD (Child exited) --- accept(6, and hangs here, some SIG problems ??? any ideas? ==================================================================== Antanas Masevicius Kaunas University of Technology Studentu 48a-101 Computer Center LT-3028 Kaunas LITNET NOC UNIX Systems Administrator Lithuania E-mail: zolia at sc.ktu.lt On 14 Aug 2000, Curtis L. Janssen wrote: > I mainly use MPI/Pro on Linux machines since it is the only MPI > that will support the MP2 gradient algorithm in MPQC. However, > for Hartree-Fock, MPICH should work. It would help to know what > kind of calculation you are doing and how far it got. > > Curt > ----------------------------------------------------------------------- > Curtis Janssen > cljanss at ca.sandia.gov http://aros.ca.sandia.gov/~cljanss > Tel: +1 925-294-1509 Fax: +1 925-294-2234 > Sandia National Labs, MS 9217, PO Box 969, Livermore, CA 94551, USA > _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From mrmail at pp.phys.uvic.ca Mon Aug 14 14:59:54 2000 From: mrmail at pp.phys.uvic.ca (Michael J. Rensing) Date: Mon, 14 Aug 2000 11:59:54 -0700 (PDT) Subject: Tux viking image Message-ID: Hi all, I felt that it was time for a Viking theme'd Tux to appear. I'm using one as a logo for our Beowulf. (http://pingu.phys.uvic.ca/muse/) Please feel free to check out our efforts and perhaps download your own copy from http://pingu.phys.uvic.ca/muse/tuxtheviking.html. As always, acknowledgement of our work would be appreciated! Feel free to e-mail any polite comments to me (mailto:mrensing at uvic.ca). Rude comments can be e-mail'd to mailto:/dev/null Michael ------------------------------------------------------------ Dr. Michael J. Rensing | mailto:mrensing at uvic.ca Physics and Astronomy | phone: 250-721-7741 University of Victoria | fax: 250-721-7752 ------------------------------------------------------------ _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From eugene.leitl at lrz.uni-muenchen.de Mon Aug 14 19:37:02 2000 From: eugene.leitl at lrz.uni-muenchen.de (Eugene Leitl) Date: Mon, 14 Aug 2000 16:37:02 -0700 (PDT) Subject: NASA solicitation for proposals for algorithms in analysis and optimization in aerospace Message-ID: <14744.33438.330214.272492@lrz.uni-muenchen.de> http://nais.msfc.nasa.gov/cgi-bin/EPS/synopsis.cgi?acqid=74561 NASA RESEARCH ANNOUNCEMENT: INNOVATIVE ALGORITHMS FOR ENGINEERING ANALYSIS AND OPTIMIZATION General Information Solicitation Number: NRA-00-LaRC-1 NAIS Posted Date: Aug 10, 2000 CBDNet Posted Date: Aug 10, 2000 Response Date: Dec 01, 2000 Classification Code: A -- Research and Development Contracting Office Address NASA/Langley Research Center, Mail Stop 144, Industry Assistance Office, Hampton, VA 23681-0001 Description On or about Spetember 1, 2000 NASA Langley Research Center will solicit proposals for algorithms in analysis and optimization in aerospace engineering capable of effectively exploiting concurrently operating processors whose number may be very large; hundreds of thousands, even millions are expected to become available within two decades. The intent is to stimulate innovation, including new paradigms for intrisically concurrent solutions in analysis and optimization in major aerospace applications which are now stymied by too high cost and too long turn-around times. A previous NRA addressed the same topic in 1998 and instigated research that resulted in a number of publications. The next generation research on this topic called for herein should include an assessment of the method's strengths and shortcomings from the future user's point of view. Awards, single or multiple, will be granted within a total of $400K for the first year, with a continuation of the initial awards at NASA'S option for two more years at up to $400K per year. Awards instruments may include contracts, grants, and/or cooperative agreements. Cost sharing agreements may be proposed. Technical questions may be addressed to Dr. Jaroslaw Sobieski at j.sobieski at larc.nasa.gov, or to the Contracting Officer. Point of Contact Name: David H. Jones Title: Contracting Officer Phone: (757) 864-2421 Fax: (757) 864-8863 Email: D.H.JONES at larc.nasa.gov _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From dagekko at execpc.com Mon Aug 14 21:32:27 2000 From: dagekko at execpc.com (eric hill) Date: Mon, 14 Aug 2000 20:32:27 -0500 Subject: looking for 2-4 port ethernet cards Message-ID: we are looking for reccomendations on 2-4 port ethernet (100m) cards for a meshed cube to be built over the next 3 months any product advice on multiport ethernet cards would be appreciated thanx eric j hill dagekko at execpc.com _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From cgreer1 at midsouth.rr.com Tue Aug 15 01:46:02 2000 From: cgreer1 at midsouth.rr.com (Chris Greer) Date: Tue, 15 Aug 2000 00:46:02 -0500 Subject: looking for 2-4 port ethernet cards References: Message-ID: <3998D91A.A8A1DA34@midsouth.rr.com> We are currntly using a Quad Ethernetcard from ZnYX. It uses the Tulip driver and seems to work fomr for us. We aren't currently using this card in a cluster, we are using it in a firewall. Our specific card is the ZnYX ZX346. It's not the newest card they make, but we got them from another group at our company who couldn't get them to work under their preferred OS. I tried them and the RedHat 6.0 install auto-detected the card and all has been fine ever sense. We are not running this card in a "production" environment yet because we are still in a testing phase, but so far all is well. Chris G. eric hill wrote: > > we are looking for reccomendations on 2-4 port ethernet (100m) > cards for a meshed cube to be built over the next 3 months > > any product advice on multiport ethernet cards would be > appreciated > > thanx > > eric j hill > dagekko at execpc.com > > _______________________________________________ > Beowulf mailing list > Beowulf at beowulf.org > http://www.beowulf.org/mailman/listinfo/beowulf _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From deadline at plogic.com Wed Aug 16 18:09:31 2000 From: deadline at plogic.com (Douglas Eadline) Date: Wed, 16 Aug 2000 18:09:31 -0400 (EDT) Subject: LAM: Multiple NICs in SMP nodes In-Reply-To: Message-ID: On Wed, 16 Aug 2000, David van der Spoel wrote: > Hi, > > This may be a bit off-topic (OS / MPI) but anyway... > I have a couple (4, soon 8) of SMP machines (dual P3) running RH linux. To > improve network performance I bought extra network cards, so that all > machines have two. My main application (molecular simulation) communicates > in a ring topology most of the time. Dual network cards could be connected > in a point to point topology for optimal communication, but other > (non-ring) communication is sometimes necessary as well, so everything is > connected through a switch running at 100 Mbit full duplex. > > Now here are the questions: > Is there a way of binding processes to network cards (in general not in > Linux as far as I know...), or alternatively, is there a way to have a > persistant TCP connection within LAM? As a third alternative: is there > anyone that has experimented with channel bonding in recent (2.2.16+) > Linux kernels? Does one need a special switch to use channel bonding? The following is on PII-Linux RH6.2 system. I looked at this a while ago and found I could not get LAM to do it. My idea was that since there are two CPUs on the motherboard and two network cards (each running a separate network), I tried to see if I could boot LAM to use both networks. For instance, I had 4 systems called coyote1, coyote2, coyote3, coyote4 On the second network, there were called coyote1a, coyote2a, coyote3a, coyote4a. I tried a boot schema: coyote1 coyote2 coyote3 coyote4 coyote1a coyote2a coyote3a coyote4a So when LAM tried to start on coyote1a it complained about the daemon already running. My goal was to fool LAM and make 4 dual machines look like 8 singles. The best I came up with was to do this: coyote1 coyote2a coyote3 coyote4a This way if coyote1 needed to send to all the nodes, it could use two interfaces at the same time. I tried running the NASA parallel benchmarks with this, but found no big performance gain. I have tried channel bonding, it works. However, if you need better latency, channel bonding will provide the same a single network. Throughput however, can be almost double a single network card. Just some random thoughts Doug > > > Groeten, David. > ________________________________________________________________________ > Dr. David van der Spoel Biomedical center, Dept. of Biochemistry > s-mail: Husargatan 3, Box 576, 75123 Uppsala, Sweden > e-mail: spoel at xray.bmc.uu.se www: http://zorn.bmc.uu.se/~spoel > phone: 46 18 471 4205 fax: 46 18 511 755 > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > > _______________________________________________ > This list is archived at http://www.mpi.nd.edu/MailArchives/lam/ > -- ------------------------------------------------------------------- Paralogic, Inc. | PEAK | Voice:+610.814.2800 130 Webster Street | PARALLEL | Fax:+610.814.5844 Bethlehem, PA 18015 USA | PERFORMANCE | http://www.plogic.com ------------------------------------------------------------------- _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From eugene.leitl at lrz.uni-muenchen.de Thu Aug 17 02:52:13 2000 From: eugene.leitl at lrz.uni-muenchen.de (Eugene Leitl) Date: Wed, 16 Aug 2000 23:52:13 -0700 (PDT) Subject: [dsp-clusters] Question In-Reply-To: <001b01c00817$dcc45be0$95c41918@san.rr.com> References: <8nc1gi+57h9@eGroups.com> <14745.48273.669328.888911@lrz.uni-muenchen.de> <00ad01c00712$ccd9b780$25714bd1@computerparts.com> <14745.54987.281042.912350@lrz.uni-muenchen.de> <399B19C9.28A50C2A@scripps.edu> <14747.18899.79210.5196@lrz.uni-muenchen.de> <001b01c00817$dcc45be0$95c41918@san.rr.com> Message-ID: <14747.35741.701325.445358@lrz.uni-muenchen.de> Cody Lutsch writes: > A mix match of PCs is just out of the question for large Beowulf projects, > period. I can imagine the nightmare of managing a bunch of PCs lying on the > floor. When you move to a racked environment; all hubs, switches, KVMs, Lying on the floor? I thought it was clear from the URLs I provided that one shelves off-shelf (sorry) commodity (whether platinum-plated or not, depends on how your codes scale in respect to parallelism) PCs on cheap heavy-duty shelves as available from CostCo et al. The Stone Soupercomputer is clearly not a standard way of doing things, machines in a cluster are typically homogenous. And I don't get the "nightmare" bit. If you don't need physical access to the machine, managing is exactly the same. If you do, plucking a desktop off the shelf, and setting a spare in its place is, if anything, easier than unscrewing the gliding rails from a 19" rackmount. If you want to transport a 19" rackmounted cluster, you have to disassemble it anyway, as it is much too heavy for tipping safely. The only advantage is footprint, as you can pack more than 64 nodes into a single full height 19". > APCs, routers, servers, you name it, are available in a 19" flavor.... all > fitting in the same enclosure. While I don't have anything against rackmount switches (since they don't come at an extra cost), I'd probably go with a distributed UPS solution (one $100 UPS for each node), or use a dieser generator with a crossover circuit instead of an UPS for larger installations. Also, minimizing the length of cabling might want you to put the switches where the nodes are. > > Well, if it was my grant money, I'd build the thing with my students, > > because that way I'd have 3-4 times as many nodes for the same > > money. > > It no where near triples or quadruples the cost per node. If you are using > high end systems, the percent added due to Rackmount hardware becomes > increasingly small. Lets use $200 as the cost of a 19" Rackmount case (some > are a lot more, some are a bit less). Lets also say a typical desktop type > case is free (they are not free, but it makes the math easier). Now lets If you order a set of identical machines from a Taiwan company, or compare individual component prices, you'll notice that they don't add up. All I know that a COTS Taiwan-made desktop PC costs a lot less than the same hardware packed in a rackmount, because of economies of scale. It's both about the price of the rackmount case, as it is about having to switch the assembly line (COTS warez are being knocked out in 10-100 k quantities). > put in $2000 of hardware. The Rackmount comes to $2,200/node where the > 'mix-and-match' comes to $2000/node. This is a very small increase as far > as percents go. This dollar investment is quickly made up by the level of $2000/node? That's a lot, according to http://www.pricewatch.com/1/234/2187-1.htm I can get an Athlon 700 (admittedly, with only 32 MBytes RAM, but RAM is cheap nowadays) for $511. While in a real world case I'll be probably looking more at $1000, there's some definitive difference between $1000 and $1200 (plus the fractional cost of a 19", which can be up to $1000/nodes per rack). Typically, the more nodes you use, the more the "small" differences start to aggregate. $200 price difference per node results in $20000 price bill if we're talking 100 nodes. > organization and standardization provided. Look at a rack full of servers, > with a single monitor, keyboard/mouse tray, KVM to switch between them, and > a UPS, and tell me that's not a better way of doing things. I'd rather have a single (2-3 for redundancy in case of large ones) console for the entire cluster, thank you. As you noticed, shelved PCs in the links I mentioned do have a single console for the whole cluster. It has nothing to do with rackmount or not rackmount. > If you are collocating the system(s) somewhere, the SAVINGS of a high > density solution (1U servers, or a '3-in-1' 2U) is unquestionable. Look at > the price of colo's recently? By spending a few more bucks to get a higher > density solution, you save the cost of renting another rack/cage/room. In academic settings, space is usually not a problem. As to density, I'm not sure a 4U rackmount case can beat an industrial strength shelf. 2U or 1U might be another matter, but a room in a university is not a climatized ISP room with false floors and a set of admins. The costs for administration and location (plus power) do not usually figure in an academic setting. > > Notice that these are big installations. Anyone knows how Google > > houses their 6 kBoxes? > > Actually, I don't know how Google house their machines... I would be > interested to see their setup... 6,000? Wow. I've tried hawking for some photos on http://google.com , but very little meat there. Whoever can find some actual photos, please post them here. > Right now we are building a 1200 node setup for a company, I'm glad I'm not > the one that has to draw out the networking plan for that! :) Wow. FastEthernet (channel bonded), or Myrinet? Something else entirely? > Thanks for the links, very informative. > > Cody You're very welcome. I forwarded your last mail to the Beowulf list, to see what the folks there might want to say on housing issues. Those of you interested in the issue might want to peruse http://www.supercomputer.org/cgi-bin/search.cgi?ul=&ps=20&np=0&q=rack I've thought a bit about casing DSP clusters, both with only on-chip and off-chip memory, but I'd like to hear your ideas on the matter. Particularly, SHARC-type 6-link DSPs would require short interconnects, unlike to long-haul networking technologies like GBit Ethernet. _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From deadline at plogic.com Thu Aug 17 07:24:44 2000 From: deadline at plogic.com (Douglas Eadline) Date: Thu, 17 Aug 2000 07:24:44 -0400 (EDT) Subject: [dsp-clusters] Question In-Reply-To: <14747.35741.701325.445358@lrz.uni-muenchen.de> Message-ID: On Wed, 16 Aug 2000, Eugene Leitl wrote: ---snip--- > > $2000/node? That's a lot, according to > http://www.pricewatch.com/1/234/2187-1.htm > I can get an Athlon 700 (admittedly, with only 32 MBytes RAM, but RAM > is cheap nowadays) for $511. While in a real world case I'll be > probably looking more at $1000, there's some definitive difference > between $1000 and $1200 (plus the fractional cost of a 19", which can > be up to $1000/nodes per rack). I thought I would offer a bit of experience here for the "do it yourself builders". Unless you go direct to a DIMM manufacturer, RAM is a crap-shoot. Bargain PC Shacks (like those on pricewatch) often have no control of the quality or manufacturer of the DIMMs. Remember, "cheaply made RAM" is cheap. Running Windows is very different than running a floating point problem on a dual PIII system for several hours, but to many PC Shack vendors Windows is what determines if RAM works! Be careful of the cheap DRAM that has the 20% re-stocking fee. These can blow your budget if you need to replace RAM. Doug ------------------------------------------------------------------- Paralogic, Inc. | PEAK | Voice:+610.814.2800 130 Webster Street | PARALLEL | Fax:+610.814.5844 Bethlehem, PA 18015 USA | PERFORMANCE | http://www.plogic.com ------------------------------------------------------------------- _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From gerry at cs.tamu.edu Thu Aug 17 09:24:59 2000 From: gerry at cs.tamu.edu (Gerry Creager N5JXS) Date: Thu, 17 Aug 2000 08:24:59 -0500 Subject: [dsp-clusters] Question References: <8nc1gi+57h9@eGroups.com> <14745.48273.669328.888911@lrz.uni-muenchen.de> <00ad01c00712$ccd9b780$25714bd1@computerparts.com> <14745.54987.281042.912350@lrz.uni-muenchen.de> <399B19C9.28A50C2A@scripps.edu> <14747.18899.79210.5196@lrz.uni-muenchen.de> <001b01c00817$dcc45be0$95c41918@san.rr.com> <14747.35741.701325.445358@lrz.uni-muenchen.de> Message-ID: <399BE7AB.5560FCC5@cs.tamu.edu> For the cluster I've beenworking on at home (16 P133 nodes) I've gone rack-mount. For the cluster I'm building at the office (about 60 nodes, heterogeneous 486/33 thru PIII-667) I'm using racks with system shelves.... but only because my boss decided that wire racks were not nice-looking enough for the tours we lead thru the new network engineering lab. The home cluster is constructed of 4/chassis P-133 SBCs on an ISA bus with 64MB of memory and 4GB of local IDE disk. I got chassis at auction for $75 each, and was the benefactor of the largess of a telco vendor updating their P133s to PIIs last summer. Mechanical KVMs were $25/4 ports at the Dallas, TX "First Saturday" giant fleamarket. The rack was another $75 for a scavenged AS-400 cabinet. The home cluster was done this way primarily for space management in my shop. Aside from the 16-node cluster, I have several other boxes external to the cluster for ham radio, meteorology and realtime monitoring. The thought of scattering 20-odd tower cases around in there was not pretty. And, I got everything cheap. Note that I got enough cases to put all the CPUs in the rack. While they're on a dedicated and isolated 20-Amp circuit, power is still a concern. I'm max'd out for that one practically, if not theoretically. For the office, the issue is deployment of a number of tower-case systems. With the number I have to work with, I'd have preferred to stack 'em on tables, or wire shelves. More space-efficient. But, to maintain the aesthetics, we're using system shelves in open-frame relay racks. Also for aesthetics, my switches are not colocated with the systems although they're in the same room. They're in a seperate set of racks dedicated to network devices. Thus, the decision to chassis/rack mount or not is determined by a number of factors. For my work at home, the primary consideration was space. At the office, building a working cluster is secondary to the need of having a new laboratory that is also a showcase for our network engineering program. Were I building a cluster solely for for computing (soon, now, soon) I'll revisit the considerations, and suspect I'll fall on Eugene's side of the equation. I feel compelled to remind everyone that for the TCP/IP seven-layer model, 2 new layers have been identified to be added at the top (thus creating a 9-layer model...): Layer 8 is Financial, and Layer 9 is Political. I believe that Layers 8 & 9 are present inthe decision of how to configure a Cluster environment, as well. Regards, Gerry -- Gerry Creager gerry at cs.tamu.edu, gerry at page4.cs.tamu.edu Network Engineering |Research focusing on Computer Science Department |Satellite Geodesy and Texas A&M University |Geodetic Control 979.458.4020 (Phone) -- 979.847.8578 (Fax) _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From spoel at xray.bmc.uu.se Thu Aug 17 09:04:20 2000 From: spoel at xray.bmc.uu.se (David van der Spoel) Date: Thu, 17 Aug 2000 15:04:20 +0200 (CEST) Subject: switch for channel bonding Message-ID: Hi, Is there someone that knows of a list of switches that support channel bonding? Groeten, David. ________________________________________________________________________ Dr. David van der Spoel Biomedical center, Dept. of Biochemistry s-mail: Husargatan 3, Box 576, 75123 Uppsala, Sweden e-mail: spoel at xray.bmc.uu.se www: http://zorn.bmc.uu.se/~spoel phone: 46 18 471 4205 fax: 46 18 511 755 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From admin at elcomsoft.gr Thu Aug 17 13:56:29 2000 From: admin at elcomsoft.gr (Ioannis F Sotiriadis) Date: Thu, 17 Aug 2000 20:56:29 +0300 Subject: problem & questions Message-ID: <14872.000817@ath.forthnet.gr> i am putting a beowulf cluster together (nodes diskless) using the new 3com bootable lancard and have nothing but problems AND all the HOWTO's take to much for granted :-( I am using on the card BOOT METHOD: TCP/IP and PROTOCOL: BOOTP (available methods TCP/IP, NETWARE, BPL and PXE - available protocols (under TCP/IP) DHCP and BOOTP and (under netware) 802.2 802.3 and EthII) RESULTS: when the node boots the message is "UNDI initialization failed" and boots again - on the server the message (everytime the node boots) "eth0:BTL8139: Interrupt line blocked,status 1" What am i doing wrong and/or what is missing? -- Best regards, Ioannis mailto: admin at elcomsoft.gr _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From jdm2d at cs.virginia.edu Thu Aug 17 20:20:31 2000 From: jdm2d at cs.virginia.edu (Justin Moore) Date: Thu, 17 Aug 2000 20:20:31 -0400 (EDT) Subject: switch for channel bonding In-Reply-To: Message-ID: Hello, If by "channel bonding" you mean combining several NICs in the same host into a single I/O channel to the "outside world", then I don't think there are any that do this. Channel bonding as referred to in the Beowulf community involves using some of Don Becker's patches to kernel net modules to get this behavior. Some switches support something known as "trunking", in which several ports on the switch are combined into one logical I/O channel; i.e., if ports 3-6 are "trunked", data coming in for that trunk can be sent on ports 3, 4, 5 or 6. This won't work as you probably want it to, since each NIC (if those cards are in the same host) will just receive random fragments of packets. I know several 3Com switches support trunking, such as the 3900 series. For actual channel bonding, though, go to the Scyld web site (http://www.scyld.com) and check out the networking pages. -jdm ---------------------------- Justin Moore Assistant Centurion SysAdmin University of Virginia jdm2d at cs.virginia.edu ---------------------------- On Thu, 17 Aug 2000, David van der Spoel wrote: > Hi, > > Is there someone that knows of a list of switches that support channel > bonding? > > > Groeten, David. > ________________________________________________________________________ > Dr. David van der Spoel Biomedical center, Dept. of Biochemistry > s-mail: Husargatan 3, Box 576, 75123 Uppsala, Sweden > e-mail: spoel at xray.bmc.uu.se www: http://zorn.bmc.uu.se/~spoel > phone: 46 18 471 4205 fax: 46 18 511 755 > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > > > _______________________________________________ > Beowulf mailing list > Beowulf at beowulf.org > http://www.beowulf.org/mailman/listinfo/beowulf > _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From rgb at phy.duke.edu Thu Aug 17 22:09:19 2000 From: rgb at phy.duke.edu (Robert G. Brown) Date: Thu, 17 Aug 2000 22:09:19 -0400 (EDT) Subject: Facilities Requirements for Beowulf Clusters? In-Reply-To: <3995ECA1.F6FBCA4A@cs.umt.edu> Message-ID: On Sat, 12 Aug 2000, Don Morton wrote: > Howdy, for years I've been running a 9-node cluster of 100MHz Pentiums, > and I've gotten away with stuffing them into a single room with no > additional power or air conditioning considerations. > > I'm in a position now where I'll be getting 16 to 32 nodes, high-end, > along with a high-end server, and I'm kind of worrying that a typical > room's power system and, particularly, the air conditioning, might > not be adequate. > > I realise I can calculate the power requirements and compare them > to what's available in the room, but A/C is a little tougher. Have > any of you tried jamming 16 to 32 nodes in a small room that's typically > kept at around 22C (72F) with any success? I hear about a number of > medium-sized systems like the one I'm getting ready to acquire, > and I wonder if people are really going all out to spend extra > bucks on the facilities? > > Thanks, > > Don Morton > I do a bit of an analysis of the A/C requirements for beowulfs in the beowulf draft book available in pdf form from www.phy.duke.edu/brahma. Basically, the rule is that all the energy that goes into the room has to come out. Some will come out through the walls and ceiling, driven by the temperature difference between inside the room (hot, presumably) and outside. A bit might come out via convection patterns if the room isn't sealed. All the rest must be actively removed. This is a VERY REAL thing to worry about. 16 nodes produce as much heat as a good sized space heater, 32 nodes will damn near heat a small house. My home office (a largish 14x14 room) is close to ten degrees warmer than the rest of the house in spite of air conditioning and gets up in the 80-90 range very fast if the A/C goes off for any reason. I have only four nodes -- I don't think I could go to eight without installing a window unit A/C (or keeping the rest of the house VERY cold or installing some sort of fan to mix the room's air with house air faster than the standard A/C vents do). Our equipment room in the physics department is unfortunately small and has no windows, one (solid) door (kept locked) and typically has 12-16 systems running therein. If the A/C fails but the power doesn't, the temperature inside soars over 100F very quickly -- we've experienced hardware failures twice because of this and for a while had a thermal kill switch on the most important systems (back when they were very expensive and unique compared to our current stack of COTS boxes). If you are counting on non-driven diffusion processes to keep a 16+ node beowulf cool in a small room with no thermostat or significant air mixing in a larger 72F ambient space, I doubt that you will comfortably succeed unless you install e.g. a door exhaust fan that constantly mixes the air inside and outside and hence transfers the heat to the building A/C (presuming that it has the surplus capacity to deal with it). If you plan to go to 32 nodes (presuming 3200 watts, give or take a bit) I suspect you'll need a ton or so of dedicated A/C (a ton of A/C is defined in the book) with its own thermostat and possibly with a coupled kill switch for the room power. rgb -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb at phy.duke.edu _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From shi at physics.ucsd.edu Fri Aug 18 01:23:30 2000 From: shi at physics.ucsd.edu (xiangdong shi) Date: Thu, 17 Aug 2000 22:23:30 -0700 (PDT) Subject: Floppy booting or network bootrom Message-ID: I would like to hear opinion about the pro and cons of booting through floppy and booting through a bootable NIC for diskless nodes. The particular plan I have is building a 8 nodes cluster for myself. Then possibly goes larger if all goes well. Thanks. Xiangdong Shi _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From bala at jncasr.ac.in Fri Aug 18 01:26:11 2000 From: bala at jncasr.ac.in (Dr. Balasubramanian Sundaram) Date: Fri, 18 Aug 2000 10:56:11 +0530 Subject: problem & questions References: <14872.000817@ath.forthnet.gr> Message-ID: <399CC8F2.29457E46@jncasr.ac.in> dear ioannis, Using PXE enabled network card, to build a cluster is a rather painful exercise. Three months ago, we built a 8-node diskless cluster using Intel EtherExpressPro network card with PXE support (i.e., we did not have to burn the eeprom). You are using a 3COM card, but the underlying problems, I suppose, are similar. We have shared our experience in our homepage. You can view it at: http://www.jncasr.ac.in/kamadhenu Also, Intel has some rudimentary PXE support documentation for Linux. One of them is: ftp://download.intel.com/ial/wfm/pxesdklinux.pdf This will give some indication of things one needs to take care in mounting the root partition. A couple of simple things to use while building a diskless cluster (pardon me if i am telling things that you know already). 1) Work only with the master (disked machine), and ONE slave to start with. 2) Connect a spare monitor and spare keyboard on the slave also, so that you can view what is going on in the slave. 3) Run bootpd with the debug option. 4) Do "tail -f /var/log/messages" on the master, to keep track of any error messages. 5) As root, run "tcpdump -i eth0" (if eth0 is on the beowulf-private network) on the master, to look at communication between master & slave. I would consider myself a beginner in linux, but still the output of these commands can be interpreted with simple common sense. and, it can help you in solving simple routing problems, for example. good luck, bala Ioannis F Sotiriadis wrote: > > i am putting a beowulf cluster together (nodes diskless) > using the new 3com bootable lancard and have nothing but > problems AND all the HOWTO's take to much for granted :-( > > I am using on the card BOOT METHOD: TCP/IP and PROTOCOL: BOOTP > (available methods TCP/IP, NETWARE, BPL and PXE - available > protocols (under TCP/IP) DHCP and BOOTP and (under netware) > 802.2 802.3 and EthII) > > RESULTS: when the node boots the message is "UNDI > initialization failed" and boots again - on the server the > message (everytime the node boots) "eth0:BTL8139: Interrupt > line blocked,status 1" > > What am i doing wrong and/or what is missing? > > -- > Best regards, > Ioannis mailto: admin at elcomsoft.gr > > _______________________________________________ > Beowulf mailing list > Beowulf at beowulf.org > http://www.beowulf.org/mailman/listinfo/beowulf _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From Bogdan.Costescu at IWR.Uni-Heidelberg.De Fri Aug 18 10:44:03 2000 From: Bogdan.Costescu at IWR.Uni-Heidelberg.De (Bogdan Costescu) Date: Fri, 18 Aug 2000 16:44:03 +0200 (CEST) Subject: switch for channel bonding In-Reply-To: Message-ID: Hi, AFAIK Cisco calls "channel-bonding" what others call "trunking". I have used Cisco 5000 series' "channel-bonding" and BayNetworks 350-24T's "trunking" facilities and they worked just the same. Whatever it's called it's supposed to do this: allow bundling of several physical links into a logical one. To do this, they require that all the NICs at the other end have the same MAC (Ethernet address). It's not possible to split one packet among the physical links, but different packets can go through different links. The decision of which link to use for sending a packet is completely up to the device (switch or Linux host, not the NIC) and there may be different strategies. For example the Linux "channel-bonding" module presently uses a Round-Robin strategy, where the packet "i" is sent through the link i % n (i modulo n), where "n" is the number of physical links available. At the Rx end, the packets from "ethx" are all sent to the bonding module (in the order they were received), and the upper levels just see the packets coming from "bond0". The bonding facility was introduced in the 2.2 series earlier this year and was recently modified. It can be used to bond 2 or more NICs (the maximum number is not a limit of the bonding module AFAIK) - usually the switch limits the number of physical links that can be bonded (4 for some Cisco switches and for BayNetworks 350). You can bond different brands of NICs (e.g. a 3Com, a DEC-based and a EEPro), the bonding module doesn't care about this; however, for getting good results, the bonded NICs should be similar in performance. And no, you don't need any patches for the network drivers; I suppose that one of the things that was present in the patches that you mention was the abillity to set the Ethernet address; nowadays, all the network drivers that I know of allow this. To come back to the original question: there are quite a lot of switches that allow "channel-bonding"/"trunking"/whatever is called. I know of quite a lot of the Cisco switches, the 3900 switches from 3Com, BayNetworks 350 and 450, some Intel switches. Usually, this facility is mentioned in the white-paper of the switch, so just searching on the web sites of these (and maybe other) manufacturers might give you the answers; also usually they write a lot about how beneficial this facility can be to you, so it's quite hard to miss it. Mr. Thomas Davis (tadavis at lbl.gov) wrote in the README accompanying the bonding module that he would like to put together a list of switches reported to work with it (based on the feedback from the users...) However, I don't know if this list exists. Sincerely, Bogdan Costescu IWR - Interdisziplinaeres Zentrum fuer Wissenschaftliches Rechnen Universitaet Heidelberg, INF 368, D-69120 Heidelberg, GERMANY Telephone: +49 6221 54 8869, Telefax: +49 6221 54 8868 E-mail: Bogdan.Costescu at IWR.Uni-Heidelberg.De _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From mello at msg.ucsf.edu Fri Aug 18 10:37:31 2000 From: mello at msg.ucsf.edu (Mel Jones) Date: Fri, 18 Aug 2000 07:37:31 -0700 Subject: MOSIX for Alpha? Message-ID: <399D4A2B.439FF567@msg.ucsf.edu> Hi: Does anyone know of any MOSIX-esque software for alpha processors? We would like to explore dynamically re-configuring our cluster as some linear combination of beowulf / non-beowulf nodes depending on the current job mix. How are other people doing this? It may well be that MOSIX is an overkill for us I just want to use the stack as optimally as I can. We really can't afford for these machines to be idle. Any leads would be appreciated. Thanks... mel jones Department of Biochemistry UCSF ps - I appologize if this is a redundant request. I sent one very similar yesterday but assume it's lost in the ether since it neither bounced nor appeared again... _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From pauln at psc.edu Fri Aug 18 12:24:29 2000 From: pauln at psc.edu (Paul Nowoczynski) Date: Fri, 18 Aug 2000 12:24:29 -0400 (EDT) Subject: switch for channel bonding In-Reply-To: Message-ID: trunking, link aggregation, channel bonding, are all the same. i think the best method is to use 2 switches. the problem that i've seen on both cisco and intel switches is that the recv streams are not divided between the 2 recv'ing interfaces. so don't expect to see ~180 mbits/sec between a point-to-point sender/receiver application if you're using the switch's trunking capability, you'll need multiple streams to get that. but if you can wire your cluster so that interface0 goes to one switch and interface1 goes to another switch then you could get good point-to-point bandwidth. paul On Fri, 18 Aug 2000, Bogdan Costescu wrote: > > Hi, > > AFAIK Cisco calls "channel-bonding" what others call "trunking". I have > used Cisco 5000 series' "channel-bonding" and BayNetworks 350-24T's > "trunking" facilities and they worked just the same. Whatever it's called > it's supposed to do this: allow bundling of several physical links into a > logical one. To do this, they require that all the NICs at the other end > have the same MAC (Ethernet address). > > It's not possible to split one packet among the physical links, but > different packets can go through different links. The decision of which > link to use for sending a packet is completely up to the device (switch or > Linux host, not the NIC) and there may be different strategies. For > example the Linux "channel-bonding" module presently uses a Round-Robin > strategy, where the packet "i" is sent through the link i % n (i modulo > n), where "n" is the number of physical links available. At the Rx end, > the packets from "ethx" are all sent to the bonding module (in the order > they were received), and the upper levels just see the packets coming from > "bond0". > > The bonding facility was introduced in the 2.2 series earlier this year > and was recently modified. It can be used to bond 2 or more NICs (the > maximum number is not a limit of the bonding module AFAIK) - usually the > switch limits the number of physical links that can be bonded (4 for some > Cisco switches and for BayNetworks 350). You can bond different brands of > NICs (e.g. a 3Com, a DEC-based and a EEPro), the bonding module doesn't > care about this; however, for getting good results, the bonded NICs should > be similar in performance. And no, you don't need any patches for the > network drivers; I suppose that one of the things that was present in the > patches that you mention was the abillity to set the Ethernet address; > nowadays, all the network drivers that I know of allow this. > > To come back to the original question: there are quite a lot of switches > that allow "channel-bonding"/"trunking"/whatever is called. I know of > quite a lot of the Cisco switches, the 3900 switches from 3Com, > BayNetworks 350 and 450, some Intel switches. Usually, this facility is > mentioned in the white-paper of the switch, so just searching on the web > sites of these (and maybe other) manufacturers might give you the answers; > also usually they write a lot about how beneficial this facility can be to > you, so it's quite hard to miss it. > Mr. Thomas Davis (tadavis at lbl.gov) wrote in the README accompanying the > bonding module that he would like to put together a list of switches > reported to work with it (based on the feedback from the users...) > However, I don't know if this list exists. > > Sincerely, > > Bogdan Costescu > > IWR - Interdisziplinaeres Zentrum fuer Wissenschaftliches Rechnen > Universitaet Heidelberg, INF 368, D-69120 Heidelberg, GERMANY > Telephone: +49 6221 54 8869, Telefax: +49 6221 54 8868 > E-mail: Bogdan.Costescu at IWR.Uni-Heidelberg.De > > > > _______________________________________________ > Beowulf mailing list > Beowulf at beowulf.org > http://www.beowulf.org/mailman/listinfo/beowulf > _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From Bogdan.Costescu at IWR.Uni-Heidelberg.De Fri Aug 18 12:53:27 2000 From: Bogdan.Costescu at IWR.Uni-Heidelberg.De (Bogdan Costescu) Date: Fri, 18 Aug 2000 18:53:27 +0200 (CEST) Subject: switch for channel bonding In-Reply-To: Message-ID: On Fri, 18 Aug 2000, Paul Nowoczynski wrote: > trunking, link aggregation, channel bonding, are all the same. i think > the best method is to use 2 switches. the problem that i've seen on both > cisco and intel switches is that the recv streams are not divided between > the 2 recv'ing interfaces. so don't expect to see ~180 mbits/sec > between a point-to-point sender/receiver application if you're using the > switch's trunking capability, you'll need multiple streams to get that. Sorry to dissapoint you, but I got this working. My BayNetworks 350-24T sends packets in the Round-Robin fashion and I was able to get more than 180 Mbit/s (measured with ttcp) with 2 3C905C cards in each node. One Cisco switch (that I have used but I don't administer) gave me at the beginning strange results. ifconfig would show for my 2 bonded NICs the same number of Tx packets, but very unbalanced Rx packets. Empirically (by looking at the LEDs on the back of the cards) I found that only one link was used normally and only if this link was busy, the second would be used. However, after I talked to the admin, I got "normal" results from ifconfig, showing very similar numbers for Rx packets; I haven't asked him again, but I guess that he changed something in the switch config (as the traffic didn't change). However, I'm not sure if this can change the bandwidth results. > but if you can wire your cluster so that interface0 goes to one switch and > interface1 goes to another switch then you could get good point-to-point > bandwidth. Using 2 different switches is one approach, but only solves one class of problems. For example, if you need 3 NICs/nodes, you need 3 switches and so on; the wiring might become problematic. On the other hand, there are applications which require only one node (let's call it master) to send/receive large amounts of data, while the computing nodes handle amounts of data which can "fit" into normal (single NIC) bandwidth. This problem can easily be solved by having a switch which supports channel-bonding/trunking/link-aggregation to which the master will be linked through bonded links, while the compute nodes will be able to use just 1 NIC. (and this applies just as well to a NFS server). Sincerely, Bogdan Costescu IWR - Interdisziplinaeres Zentrum fuer Wissenschaftliches Rechnen Universitaet Heidelberg, INF 368, D-69120 Heidelberg, GERMANY Telephone: +49 6221 54 8869, Telefax: +49 6221 54 8868 E-mail: Bogdan.Costescu at IWR.Uni-Heidelberg.De _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From pauln at psc.edu Fri Aug 18 13:05:48 2000 From: pauln at psc.edu (Paul Nowoczynski) Date: Fri, 18 Aug 2000 13:05:48 -0400 (EDT) Subject: switch for channel bonding In-Reply-To: Message-ID: i'm not disappointed at all! it's great to hear that a switch manufacturer got their act together and did a proper implementation.. i guess the BayNetworks 350-24T will be my next switch :) paul On Fri, 18 Aug 2000, Bogdan Costescu wrote: > > On Fri, 18 Aug 2000, Paul Nowoczynski wrote: > > > trunking, link aggregation, channel bonding, are all the same. i think > > the best method is to use 2 switches. the problem that i've seen on both > > cisco and intel switches is that the recv streams are not divided between > > the 2 recv'ing interfaces. so don't expect to see ~180 mbits/sec > > between a point-to-point sender/receiver application if you're using the > > switch's trunking capability, you'll need multiple streams to get that. > > Sorry to dissapoint you, but I got this working. My BayNetworks 350-24T > sends packets in the Round-Robin fashion and I was able to get more than > 180 Mbit/s (measured with ttcp) with 2 3C905C cards in each node. > One Cisco switch (that I have used but I don't administer) gave me at the > beginning strange results. ifconfig would show for my 2 bonded NICs the > same number of Tx packets, but very unbalanced Rx packets. Empirically (by > looking at the LEDs on the back of the cards) I found that only one link > was used normally and only if this link was busy, the second would be > used. However, after I talked to the admin, I got "normal" results from > ifconfig, showing very similar numbers for Rx packets; I haven't asked > him again, but I guess that he changed something in the switch config (as > the traffic didn't change). However, I'm not sure if this can change the > bandwidth results. > > > but if you can wire your cluster so that interface0 goes to one switch and > > interface1 goes to another switch then you could get good point-to-point > > bandwidth. > > Using 2 different switches is one approach, but only solves one class of > problems. For example, if you need 3 NICs/nodes, you need 3 switches and > so on; the wiring might become problematic. On the other hand, there are > applications which require only one node (let's call it master) to > send/receive large amounts of data, while the computing nodes handle > amounts of data which can "fit" into normal (single NIC) bandwidth. This > problem can easily be solved by having a switch which supports > channel-bonding/trunking/link-aggregation to which the master will be > linked through bonded links, while the compute nodes will be able to use > just 1 NIC. (and this applies just as well to a NFS server). > > Sincerely, > > Bogdan Costescu > > IWR - Interdisziplinaeres Zentrum fuer Wissenschaftliches Rechnen > Universitaet Heidelberg, INF 368, D-69120 Heidelberg, GERMANY > Telephone: +49 6221 54 8869, Telefax: +49 6221 54 8868 > E-mail: Bogdan.Costescu at IWR.Uni-Heidelberg.De > > _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From josip at icase.edu Fri Aug 18 14:37:23 2000 From: josip at icase.edu (Josip Loncaric) Date: Fri, 18 Aug 2000 14:37:23 -0400 Subject: NFS 100:1 performance loss Message-ID: <399D8263.D2D981E@icase.edu> Has anyone seen this NFS performance problem? Our Beowulf has two servers w/RAID-0 arrays which deliver 45MB/s, connected via Gigabit Ethernet. The machines are dual PIII/500 systems with 512MB of RAM each. We are running Red Hat 6.2 updated to kernel 2.2.16-3 and recently we updated to nfs-utils-0.1.9.1-1. We are running 16 kernel nfsd threads on each machine. Here is how long it takes to copy a 28,955,860 byte file from machine 1 to machine 2: rcp: 1.04 seconds (27.8 MB/s, where 1MB=10^6B) ftp: 1.12 seconds (25.8 MB/s) NFS 1KB: 12.35 seconds ( 2.3 MB/s, using rsize=wsize=1024) NFS 8KB: 129.42 seconds ( 0.2 MB/s, using rsize=wsize=8192) Clearly, there is something very wrong with NFS, particularly with 8KB rsize/wsize (which should have improved performance!). Our system manager tells me that reduced Linux NFS performance with 8KB rsize/wsize is a known problem, but even at 1KB our NFS is getting less than 10% of the rcp or ftp performance. Any ideas? Josip -- Dr. Josip Loncaric, Senior Staff Scientist mailto:josip at icase.edu ICASE, Mail Stop 132C PGP key at http://www.icase.edu./~josip/ NASA Langley Research Center mailto:j.loncaric at larc.nasa.gov Hampton, VA 23681-2199, USA Tel. +1 757 864-2192 Fax +1 757 864-6134 _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From jakob at ostenfeld.dtu.dk Fri Aug 18 16:32:50 2000 From: jakob at ostenfeld.dtu.dk (=?iso-8859-1?Q?Jakob_=D8stergaard?=) Date: Fri, 18 Aug 2000 22:32:50 +0200 Subject: NFS 100:1 performance loss In-Reply-To: <399D8263.D2D981E@icase.edu> References: <399D8263.D2D981E@icase.edu> Message-ID: <20000818223249.A2415@ostenfeld.dtu.dk> On Fri, 18 Aug 2000, Josip Loncaric wrote: > Has anyone seen this NFS performance problem? > > Our Beowulf has two servers w/RAID-0 arrays which deliver 45MB/s, > connected via Gigabit Ethernet. The machines are dual PIII/500 systems > with 512MB of RAM each. We are running Red Hat 6.2 updated to kernel > 2.2.16-3 and recently we updated to nfs-utils-0.1.9.1-1. We are running > 16 kernel nfsd threads on each machine. Here is how long it takes to > copy a 28,955,860 byte file from machine 1 to machine 2: > > rcp: 1.04 seconds (27.8 MB/s, where 1MB=10^6B) > ftp: 1.12 seconds (25.8 MB/s) > NFS 1KB: 12.35 seconds ( 2.3 MB/s, using rsize=wsize=1024) > NFS 8KB: 129.42 seconds ( 0.2 MB/s, using rsize=wsize=8192) > > Clearly, there is something very wrong with NFS, particularly with 8KB > rsize/wsize (which should have improved performance!). Our system > manager tells me that reduced Linux NFS performance with 8KB rsize/wsize > is a known problem, but even at 1KB our NFS is getting less than 10% of > the rcp or ftp performance. > > Any ideas? > Josip You do use the kernel nfs server right ? knfsd, not the old userspace nfs daemon... Our NFS server gets us some 5 MB/s over 100MBit, which is pretty good I think. Linux NFS is not ``excellent'' yet, but with the kernel NFS server it should be better than ``horrible'' at least :) Eventually, you could try the NFS patches (for both client and server) available somewhere at sourceforge (don't have the URL, but a quick search should get you there). They should resolve most of the annying errors you see with Linux NFS, and they should improve performance too I think. We don't use them yet however, so I don't have any experience to go with that one. -- ................................................................ : jakob at ostenfeld.dtu.dk : And I see the elder races, : :.........................: putrid forms of man : : Jakob ?stergaard : See him rise and claim the earth, : : OZ9ABN : his downfall is at hand. : :.........................:............{Konkhra}...............: _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From jdavis at hess.com Fri Aug 18 16:47:14 2000 From: jdavis at hess.com (Jeff Davis (Work)) Date: Fri, 18 Aug 2000 15:47:14 -0500 Subject: NFS 100:1 performance loss References: <399D8263.D2D981E@icase.edu> <20000818223249.A2415@ostenfeld.dtu.dk> Message-ID: <007901c00955$81e96280$9a08010a@ep.hess.com> >From the NFS mailing list "The experimental kernel from VA Linux, kernel 2.2.16-8.ext3.3, are at: http://ftp.valinux.com/pub/support/hjl/redhat/6.2/ It is based on 2.2.16-8 from RedHat rawhide with NFS V3, ext3 0.0.2f, new pcmcia-cs, ....., many other changes. There is no support whatsoever. If you have problems, which cannot be reproduced under stock 2.2.16 nor 2.2.16-8 from RedHat, post them to the NFS mailing" Jeff Davis http://www.jrdavis.net/ jdavis at hess.com If you pick up a starving dog and make him prosperous, he will not bite you. This is the principal difference between a dog and a man. -- Mark Twain ----- Original Message ----- From: "Jakob ?stergaard" To: "Josip Loncaric" Cc: "Beowulf mailing list" Sent: Friday, August 18, 2000 3:32 PM Subject: Re: NFS 100:1 performance loss On Fri, 18 Aug 2000, Josip Loncaric wrote: > Has anyone seen this NFS performance problem? > > Our Beowulf has two servers w/RAID-0 arrays which deliver 45MB/s, > connected via Gigabit Ethernet. The machines are dual PIII/500 systems > with 512MB of RAM each. We are running Red Hat 6.2 updated to kernel > 2.2.16-3 and recently we updated to nfs-utils-0.1.9.1-1. We are running > 16 kernel nfsd threads on each machine. Here is how long it takes to > copy a 28,955,860 byte file from machine 1 to machine 2: > > rcp: 1.04 seconds (27.8 MB/s, where 1MB=10^6B) > ftp: 1.12 seconds (25.8 MB/s) > NFS 1KB: 12.35 seconds ( 2.3 MB/s, using rsize=wsize=1024) > NFS 8KB: 129.42 seconds ( 0.2 MB/s, using rsize=wsize=8192) > > Clearly, there is something very wrong with NFS, particularly with 8KB > rsize/wsize (which should have improved performance!). Our system > manager tells me that reduced Linux NFS performance with 8KB rsize/wsize > is a known problem, but even at 1KB our NFS is getting less than 10% of > the rcp or ftp performance. > > Any ideas? > Josip You do use the kernel nfs server right ? knfsd, not the old userspace nfs daemon... Our NFS server gets us some 5 MB/s over 100MBit, which is pretty good I think. Linux NFS is not ``excellent'' yet, but with the kernel NFS server it should be better than ``horrible'' at least :) Eventually, you could try the NFS patches (for both client and server) available somewhere at sourceforge (don't have the URL, but a quick search should get you there). They should resolve most of the annying errors you see with Linux NFS, and they should improve performance too I think. We don't use them yet however, so I don't have any experience to go with that one. -- ................................................................ : jakob at ostenfeld.dtu.dk : And I see the elder races, : :.........................: putrid forms of man : : Jakob ?stergaard : See him rise and claim the earth, : : OZ9ABN : his downfall is at hand. : :.........................:............{Konkhra}...............: _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From josip at icase.edu Fri Aug 18 17:56:49 2000 From: josip at icase.edu (Josip Loncaric) Date: Fri, 18 Aug 2000 17:56:49 -0400 Subject: NFS 100:1 performance loss References: <399D8263.D2D981E@icase.edu> <20000818223249.A2415@ostenfeld.dtu.dk> Message-ID: <399DB121.4D4131EE@icase.edu> Jakob ?stergaard wrote: > > On Fri, 18 Aug 2000, Josip Loncaric wrote: > > > We are running > > 16 kernel nfsd threads on each machine. Here is how long it takes to > > copy a 28,955,860 byte file from machine 1 to machine 2: > > > > rcp: 1.04 seconds (27.8 MB/s, where 1MB=10^6B) > > ftp: 1.12 seconds (25.8 MB/s) > > NFS 1KB: 12.35 seconds ( 2.3 MB/s, using rsize=wsize=1024) > > NFS 8KB: 129.42 seconds ( 0.2 MB/s, using rsize=wsize=8192) > > You do use the kernel nfs server right ? knfsd, not the old > userspace nfs daemon... We *do* use kernel nfsd -- but here is a hint of what may be wrong with Linux NFS: We did not have a serious NFS problem while we used jumbo frames (MTU=9000) on our Gigabit Ethernet interfaces. Recently, we switched to regular size frames (MTU=1500) so that the servers could communicate with machines which cannot handle jumbo frames. I assume that rsize=wsize=8192 means that NFS will send bursts of UDP packets to carry an 8KB message. Our MTU change means that an 8KB NFS message cannot be sent as a single UDP packet, but takes at least 6 UDP packets. On the theory that kernel nfsd cannot handle very high data rates and that some UDP packets were not being received correctly, I've changed the interrupt settings on our interface card and got much better results: 30 microseconds / 2 frames ------> NFS 8KB test takes 129.42 seconds 75 microseconds / 6 frames ------> NFS 8KB test takes 4.92 seconds Before, the NIC would interupt within 30 microseconds or on receiving 2 frames (whichever came first), thus delivering an incomplete NFS message in all cases. Now, it interrupts within 75 microseconds or on receiving 6 frames, which has a good chance of delivering the full NFS message. It's nice that we got back a factor of 25, but this NFS performance is still 4+ times slower than the plain 'rcp'. I'll experiment with this further and report what I learn. Sincerely, Josip -- Dr. Josip Loncaric, Senior Staff Scientist mailto:josip at icase.edu ICASE, Mail Stop 132C PGP key at http://www.icase.edu./~josip/ NASA Langley Research Center mailto:j.loncaric at larc.nasa.gov Hampton, VA 23681-2199, USA Tel. +1 757 864-2192 Fax +1 757 864-6134 _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From siegert at sfu.ca Fri Aug 18 18:03:10 2000 From: siegert at sfu.ca (Martin Siegert) Date: Fri, 18 Aug 2000 15:03:10 -0700 (PDT) Subject: NFS 100:1 performance loss In-Reply-To: <399D8263.D2D981E@icase.edu> Message-ID: On Fri, 18 Aug 2000, Josip Loncaric wrote: > Has anyone seen this NFS performance problem? > > Our Beowulf has two servers w/RAID-0 arrays which deliver 45MB/s, > connected via Gigabit Ethernet. The machines are dual PIII/500 systems > with 512MB of RAM each. We are running Red Hat 6.2 updated to kernel > 2.2.16-3 and recently we updated to nfs-utils-0.1.9.1-1. We are running > 16 kernel nfsd threads on each machine. Here is how long it takes to > copy a 28,955,860 byte file from machine 1 to machine 2: > > rcp: 1.04 seconds (27.8 MB/s, where 1MB=10^6B) > ftp: 1.12 seconds (25.8 MB/s) > NFS 1KB: 12.35 seconds ( 2.3 MB/s, using rsize=wsize=1024) > NFS 8KB: 129.42 seconds ( 0.2 MB/s, using rsize=wsize=8192) > > Clearly, there is something very wrong with NFS, particularly with 8KB > rsize/wsize (which should have improved performance!). Our system > manager tells me that reduced Linux NFS performance with 8KB rsize/wsize > is a known problem, but even at 1KB our NFS is getting less than 10% of > the rcp or ftp performance. > > Any ideas? No ideas. Sorry. But here are the numbers from our system: dual PII/400, 256MB RAM kernel-2.2.16-3 nfs-utils-0.1.9.1-1 No RAID, etc., just IDE drives, /home and /usr/local NFS exported from the master node. Contents of /etc/fstab: ... b01:/usr/local /usr/local nfs ro,hard,intr,bg,rsize=8192 0 0 b01:/home /home nfs rw,hard,intr,bg,rsize=8192,wsize=8192 0 0 # cd /usr/local/src/tar-files # ls -l linux-2.2.16bugaboo.tar -rw-r--r-- 1 root root 81674240 Jun 28 15:13 linux-2.2.16bugaboo.tar # time cp linux-2.2.16bugaboo.tar /tmp real 0m9.799s user 0m0.040s sys 0m2.150s That's about 63.59 Mbit/s. Second test (after "gzip linux-2.2.16bugaboo.tar" on master node): # ls -l linux-2.2.16bugaboo.tar.gz -rw-r--r-- 1 root root 20673033 Jun 28 15:13 linux-2.2.16bugaboo.tar.gz # time cp linux-2.2.16bugaboo.tar.gz /tmp real 0m2.383s user 0m0.000s sys 0m0.280s That's about 66.19 Mbit/s. In the other direction: # cd /tmp # time cp linux-2.2.16bugaboo.tar.gz /home/siegert real 0m1.854s user 0m0.000s sys 0m0.270s That's about 85.07 Mbit/s. This is with 100baseT and 3C905B NICs. So I doubt that it is NFS that is at fault here ... Cheers, Martin ======================================================================== Martin Siegert Academic Computing Services phone: (604) 291-4691 Simon Fraser University fax: (604) 291-4242 Burnaby, British Columbia email: siegert at sfu.ca Canada V5A 1S6 ======================================================================== _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From mello at msg.ucsf.edu Fri Aug 18 18:49:48 2000 From: mello at msg.ucsf.edu (Mel Jones) Date: Fri, 18 Aug 2000 15:49:48 -0700 Subject: NFS 100:1 performance loss References: <399D8263.D2D981E@icase.edu> Message-ID: <399DBD8C.C0EE9FA@msg.ucsf.edu> Josip Loncaric wrote: > Has anyone seen this NFS performance problem? > > Our Beowulf has two servers w/RAID-0 arrays which deliver 45MB/s, > connected via Gigabit Ethernet. The machines are dual PIII/500 systems > with 512MB of RAM each. We are running Red Hat 6.2 updated to kernel > 2.2.16-3 and recently we updated to nfs-utils-0.1.9.1-1. We are running > 16 kernel nfsd threads on each machine. Here is how long it takes to > copy a 28,955,860 byte file from machine 1 to machine 2: > > rcp: 1.04 seconds (27.8 MB/s, where 1MB=10^6B) > ftp: 1.12 seconds (25.8 MB/s) > NFS 1KB: 12.35 seconds ( 2.3 MB/s, using rsize=wsize=1024) > NFS 8KB: 129.42 seconds ( 0.2 MB/s, using rsize=wsize=8192) > > Clearly, there is something very wrong with NFS, particularly with 8KB > rsize/wsize (which should have improved performance!). Our system > manager tells me that reduced Linux NFS performance with 8KB rsize/wsize > is a known problem, but even at 1KB our NFS is getting less than 10% of > the rcp or ftp performance. > > Any ideas? > Josip > > -- > Dr. Josip Loncaric, Senior Staff Scientist mailto:josip at icase.edu > ICASE, Mail Stop 132C PGP key at http://www.icase.edu./~josip/ > NASA Langley Research Center mailto:j.loncaric at larc.nasa.gov > Hampton, VA 23681-2199, USA Tel. +1 757 864-2192 Fax +1 757 864-6134 > > _______________________________________________ > Beowulf mailing list > Beowulf at beowulf.org > http://www.beowulf.org/mailman/listinfo/beowulf I've seen a similar but nowhere near as bad result on our network. This may be completely irrelevant but I found NFS traffic between my SGI machines about 4x slower than FTP. It got much better when I set the transport to TCP rather than UDP. The problem turned out to be that my switches weren't necessarily forwarding the packets in order of receipt and this was taken care of much more efficiently on the receiving machine with TCP. ... mello _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From indraneel at www.cdfd.org.in Sat Aug 19 10:13:27 2000 From: indraneel at www.cdfd.org.in (Indraneel Majumdar) Date: Sat, 19 Aug 2000 07:13:27 -0700 (PDT) Subject: Floppy booting or network bootrom In-Reply-To: Message-ID: One good reason is: If your nodes are accessible to all, then someone might start using your boot floppy for storing data, and if there is a power failure... (at least you'll have to go round from node to node booting them) Another reason: Are you absolutely sure that your room is dust free, low in humidity, doesn't have fungal spores... (or you might invite headaches trying to discover booting problems, and making boot floppies every month) Another reason: Is your setup a high security system? Boot floppies can boot your brand of the kernel, or it can boot theirs... (I have used boot floppy for over a year since the bios of my node didn't support netwok booting, but then I had only two nodes and they were right beside me all the time.) NIC booting is just a single time pain. Floppy booting is a pang. \Indraneel On Thu, 17 Aug 2000, xiangdong shi wrote: > I would like to hear opinion about the pro and cons of booting through > floppy and booting through a bootable NIC for diskless nodes. The > particular plan I have is building a 8 nodes cluster for myself. Then > possibly goes larger if all goes well. Thanks. > > Xiangdong Shi > > > _______________________________________________ > Beowulf mailing list > Beowulf at beowulf.org > http://www.beowulf.org/mailman/listinfo/beowulf > /************************************************************************. # Indraneel Majumdar ? E-mail: indraneel at 123india.com # # Bioinformatics Unit (EMBNET node), ? URL: http://scorpius.iwarp.com # # Centre for DNA Fingerprinting and Diagnostics, # # Hyderabad, India - 500076 # `************************************************************************/ _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From itti at cco.caltech.edu Fri Aug 18 21:52:27 2000 From: itti at cco.caltech.edu (Laurent Itti) Date: Fri, 18 Aug 2000 18:52:27 -0700 (PDT) Subject: newbie: 16-node 500Mbps design Message-ID: Hi all - I will soon start a computational neuroscience lab at USC, and would like to install a small 16-node Beowulf system there. Applications will be either running multiple simulations of a same program with different parameters (each run taking a few days), or processing video sequences through the entire cluster, each node taking care of some aspect of visual processing (e.g., edge detection, color processing). For the latter application, high network throughput is needed, so I am thinking of channel-bonding 5 cheap ethernet boards per node, all connected through 5 switches. We have about $20k to spend on the baby, so we target about $1k/node plus switches and misc. For that price, we target 733MHz/256Mb/30Gb per node. To be open to upgrades and recycling of nodes as desktops when faster nodes are introduced, we take mobos with integrated video and sound. I would greatly appreciate comments on the following: - PIII/733 vs. Athlon/800 is a hard choice. The i815 chipset for PIII seems to be supported under Linux, delivering full ATA/100 speed (www.gentus.com). I have not found the equivalent for Athlon mobos. Is there any Athlon mobo with integrated video/sound? - has anybody ever benchmarked ethernet boards under Linux? Why pay $50+ for a 3com board when a good old RTL8139 sells for $8? Does anybody have an idea of possible differences in throughput or CPU use (using Linux drivers, not windoze or theoretical)? - 16 singles vs. 8 dual-cpu nodes is also a hard call. With 16 singles we can get better distribution of data flow over the net. But I am open at this point to suggestions regarding good dual-CPU boards with full Linux support. - I am unclear at this point as to whether the Abit SE6 mobo does have the integrated ethernet. Can't find those CNR riser boards for sale anywhere! In addition to the 16 compute nodes, we will have a desktop (in another room) to control the whole thing. The following is my draft starting point to begin negotiating with local retailers. Any comment also appreciated! Thanks a lot! -- laurent itti Seventeen (17) nodes consisting of: Abit SE6 motherboard (i815E+ICH2 chipset) $145 gogocity CNR riser board ??? Intel 733EB FC-PGA CPU (retail box version?) $240 egghead Qty. 2, 128Mb PC133 168-pin SDRAM $270 local 30Gb UATA/100 7200 RPM Hard drive $170 egghead Mitsumi 1.44 floppy drive $10 local Qty. 4, RTL8139 10/100 ethernet card $32 local Qty. 5, 3' enhanced CAT5 350MHz ethernet cable $2.50 kristamicro Qty. 1, Panasonic Panaflo fan, 500,000 hrs MTBF $3 dealdeal ATX desktop case $45 ??? Mandrake Linux 7.1 deluxe $56 linuxmall -------------------------------------------------------------------- Qty. 17 @ $973.50 each [the 17th node, aka the "control node" (to be used as a regular desktop PC) will actually have only one (on-board) ethernet, but two longer ether cables; see below. It will share keyboard/mouse/monitor with another PC already in place, using a 2-port KVM switch. Five switches will be used to connect the beowulf: 4x16-port and 1x24-port, as the latter will also connect the beowulf to the outside]. [I like Mandrake 7.1 and will patch the kernel (if necessary) using stuff from www.gentus.com for ATA/100 support. Intel has X servers]. 17 nodes $16,549.50 delete Qty. 4 RTL8139 and Qty. 5 ether cables -$34.50 7' ether cable, control node to wall $2 kristamicro 25' ether cable, wall to beowulf $5 kristamicro Qty. 4, Linksys Etherfast-II 16-port 10/100 switch $1,200 egghead 24-port Linksys Etherfast-II 24-port 10/100 switch $427 egghead Belkin Omnicube PS/2 2-port KVM switch $90 egghead Qty. 2, Belkin gold KVM cable kits $50 local Seagate Scorpion 4/8 internal IDE tape drive $190 local Qty 10, 4/8 tapes $250 local --------------------------------------------------------------------- $18,729 CA/LA County sales tax @ 8.25% $1,545.14 ------------- $20,274.14 [almost all disk space is /tmp, so no need for big tape drive] _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From itti at cco.caltech.edu Sat Aug 19 00:24:32 2000 From: itti at cco.caltech.edu (Laurent Itti) Date: Fri, 18 Aug 2000 21:24:32 -0700 (PDT) Subject: newbie: 16-node 500Mbps design In-Reply-To: Message-ID: Thanks for your reply! > I think 5 ethernet cards per node working is pushing it. Will you even be > able to stuff that many onto your motherboards? If you can, getting linux to > work properly with that many cards, generating lots and lots of interrupts, is > going to be a challenge, if not impossible. Even with dual fast ethernet, > getting maximal performance out of linux TCP is not easy. Check the list > archives about TCP stalls to see the problems people are having. great, exactly the kind of info I was looking for (the MBs have all peripherals on-board and 6 open slots) . I'll check it out! > With 16 singles you have 16 CPUs each connected to the others via a 100+mbs > ethernet link. With 8 duals you have 16 CPUs, with each CPU connected to one [...] yes, that's the way I had understood the dilemna so far. I will have to think of what type of communication flow we will mostly have. > > Qty. 2, 128Mb PC133 168-pin SDRAM $270 local > > No ECC? With over 2GB of total RAM, the probably that you will get a single excellent suggestion! thanks! > > Mandrake Linux 7.1 deluxe $56 linuxmall > > No need to buy a copy of linux for each machine. It's those people building > NT clusters who have to spend half their money on software. well, as pointed out by the previous email, it's only going to be $800 in total, i.e., less than a single Matlab license with signal & image toolboxes, and less than 5% of total cost. The idea was to say "thank you" for the great OS. If there is a better way to spend those 5% in helping the Linux community, I am open. But it must be something that I can put on a P.O. without the purchasing dept thinking that I am wasting money in donations. > You can't have a channel-bonded machine on the same network with a non-channel > bonded machine, like you have here. Your control node is connected to one > switch, while each node is connected to 5. This means that 4 out of 5 packets excellent point! so I guess 5 ethers to the cluster, and one more to the outside for that node ;-) but it looks like the interconnect will probably scale down after your first comment (maybe 2 or 3 instead of 5). thanks again for the great feedback! -- laurent _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From hahn at coffee.psychology.mcmaster.ca Sat Aug 19 12:49:10 2000 From: hahn at coffee.psychology.mcmaster.ca (Mark Hahn) Date: Sat, 19 Aug 2000 12:49:10 -0400 (EDT) Subject: newbie: 16-node 500Mbps design In-Reply-To: Message-ID: > > I think 5 ethernet cards per node working is pushing it. Will you even be > > able to stuff that many onto your motherboards? If you can, getting linux to > > work properly with that many cards, generating lots and lots of interrupts, is > > going to be a challenge, if not impossible. Even with dual fast ethernet, > > getting maximal performance out of linux TCP is not easy. Check the list > > archives about TCP stalls to see the problems people are having. there seems to be quite a lot of urban legendry here. I certainly don't see any "TCP stalls" or know anyone who does. perhaps on crappy old 2.2 kernels, but anyone who runs them deserves what they get. > toolboxes, and less than 5% of total cost. The idea was to say "thank > you" for the great OS. If there is a better way to spend those 5% in but saying "thank you" to mandrake is a bit silly, since they're just packagers. _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From admin at elcomsoft.gr Sat Aug 19 16:03:44 2000 From: admin at elcomsoft.gr (Ioannis F Sotiriadis) Date: Sat, 19 Aug 2000 23:03:44 +0300 Subject: newbie: 16-node 500Mbps design Message-ID: <2960.000819@ath.forthnet.gr> Just a remark on A-Bit dual MBoards. i am putting together a small Beowulf Cluster (12 nodes, dual) and my A-Bits are supposed to be the last in the market as they are going to get their new (coppermine supporting) MBoards on the market "pretty soon" whatever that means !! -- Best regards, Ioannis mailto: admin at elcomsoft.gr _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From omri at NMR.MGH.Harvard.EDU Sun Aug 20 09:51:18 2000 From: omri at NMR.MGH.Harvard.EDU (Omri Schwarz) Date: Sun, 20 Aug 2000 09:51:18 -0400 (EDT) Subject: AFS to be released as open source. Message-ID: So, are people planning to use it in their 'wulfs? Though I'm sure there are drawbacks, AFS +Kerberos for file service on the 'wulf means the gateway node isn't broadcasting "crack me, please!" to the world the way it does with NFS. Omri Schwarz --- omri at nmr.mgh.harvard.edu Timeless wisdom of biomedical engineering: "Noise is principally due to the presence of the patient." -- R.F. Farr _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From brua at paralline.com Sun Aug 20 10:24:52 2000 From: brua at paralline.com (Pierre Brua) Date: Sun, 20 Aug 2000 16:24:52 +0200 Subject: MOSIX for Alpha? References: <399D4A2B.439FF567@msg.ucsf.edu> Message-ID: <399FEA34.3E466454@paralline.com> Mel Jones wrote: > Does anyone know of any MOSIX-esque software for alpha > processors? Got money ? : #Subject: Re: mosix on alphalinux # Date: Wed, 11 Aug 1999 09:46:57 +0300 # From: "Prof. Amnon Barak" # To: mosix-list at cs.huji.ac.il # #We evaluated the alphalinux port and know what needs to be done. #We will start the project when/if we get funding to do it. > We would like to explore dynamically re-configuring our cluster > as some linear combination of beowulf / non-beowulf nodes > depending on the current job mix. How are other people doing > this? batch queuing systems with a sequential job queue and a parallel job queue ? Do I misunderstand the question ? If you want to know wether your beowulf can be divided in separate beowulf cluster through Mosix, the answer is yes : you only have to put a different description in /etc/mosix.map and restart the mosix daemons. You can also request a particular job to stay on a specific node, disabling Mosix load balancing policy for this job. > It may well be that > MOSIX is an overkill for us I just want to use the stack as optimally > as I can. We really can't afford for these machines to be idle. At first sight, I would say a queing system (like PBS or others) would suit your needs, since Mosix is not (yet) available on Alpha processors Linux systems. Hope it helps, Pierre -- PARALLINE Pierre BRUA Parallelism & Linux Solutions 71,av. des Vosges Phone:+33 388 141 740 mailto:brua at paralline.com F-67000 STRASBOURG Fax:+33 388 141 741 http://www.paralline.com _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From mlucas at imagelinks.com Sun Aug 20 11:10:11 2000 From: mlucas at imagelinks.com (Mark Lucas) Date: Sun, 20 Aug 2000 11:10:11 -0400 Subject: Image Processing on a BeoWulf Message-ID: Looking for some guidance on our strategy for satellite and aerial image processing on a BeoWulf cluster. We have successfully added PVM into our production software. Essentially, very large image processing jobs are sectioned into image tiles and the tiles are farmed out to the nodes for the mathematical transformations (image processing functions, image space to world space to map projection space transformations). On individual jobs this works very well as the process is extremely CPU bound. As a result we are seeing close to linear scaling on individual jobs. For each job we use all of the available processors in the cluster. Our desire is to allow many jobs to be initiated ad-hoc by the production operators or by web based clients and take advantage of the parallelism and scaling offered by the cluster. Whenever we have attempted that we seem to lock the system up. We see the same symptoms when we inadvertently schedule a single job and ask for more processors than are currently available. We currently have a work around in place - we implemented a queue that lines up the submitted jobs sequentially. This certainly is not optimal as small jobs have to wait behind large ones. We are learning as we go and are looking for advice and guidance. is this expected behavior from PVM? Will we need to look at MPI? Any help and advice would be greatly appreciated. BTW, We will be in Atlanta showing this off (hopefully on more than one job at a time :-) ) Thanks, Mark -- ********************** Mark R Lucas Chief Technical Officer ImageLinks Inc. 4450 W Eau Gallie Blvd Suite 164 Melbourne Fl 32934 321 253 0011 (work) 321 253 5559 (fax) mlucas at imagelinks.com ********************** _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From karsten.petersen at informatik.tu-chemnitz.de Sun Aug 20 16:27:02 2000 From: karsten.petersen at informatik.tu-chemnitz.de (Karsten Petersen) Date: Sun, 20 Aug 2000 22:27:02 +0200 (CEST) Subject: AFS to be released as open source. In-Reply-To: Message-ID: On Sun, 20 Aug 2000, Omri Schwarz wrote: > So, are people planning to use it in their 'wulfs? we are already using it in our 512 node Beowulf cluster. Have a look at: http://www.tu-chemnitz.de/urz/anwendungen/CLIC/index.php3?lang=en AFS is the most important central infrastructure of our whole campus network, our cfengine-based distributed administration mechanisms take heavy use on it. > Though I'm sure there are drawbacks, yes - the users must be careful. We have only one GBit/s line for all nodes as the clusters uplink, so working inside the AFS instead of the node's /tmp won't be too good ... Greets, Karsten -- ,-, Student of Computer Science at Chemnitz University of Technology ,-, | | EMail: Karsten at kapet.de WWW: http://www.kapet.de/ | | '-' Home: kapet at dollerup.csn V72 / 230 Phone: +49-177-82 35 136 '-' _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From glindahl at hpti.com Sun Aug 20 20:48:52 2000 From: glindahl at hpti.com (Greg Lindahl) Date: Sun, 20 Aug 2000 20:48:52 -0400 Subject: Image Processing on a BeoWulf In-Reply-To: Message-ID: <000401c00b09$9368bd20$d086fea9@hptilap.hpti.com> > Our desire is to allow many jobs to be initiated ad-hoc by the > production operators or by web based clients and take advantage of > the parallelism and scaling offered by the cluster. At this point you need a scheduler. You said you were using PVM. PVM isn't that smart at handing out nodes when you ask for a lot more than you have. In addition, it depends on the details when you have several jobs running on a node and talking. Sometimes (I know this is true for mpich) a job doing the wrong thing will basically spin in a busy-wait loop if the people it's trying to talk to happen to not be running. This results in such awful performance that you'd think that the job was stuck. I don't know about PVM; I'd think that the "non direct route" option should be OK, but I've never tried it. > We currently have a work > around in place - we implemented a queue that lines up the submitted > jobs sequentially. This certainly is not optimal as small jobs have > to wait behind large ones. That's a scheduler. Another scheduler you could use would be a queue system like PBS. However, what you really want is a queue system which provides "gang scheduling". With gang scheduling, only one program at a time is awake on a node, so you don't have any spin-wait problems. The RWCP guys have this for their SCORE operating system. But SCORE is pretty big, and I don't think it's easy to extract just that one feature. Now the T3E has a lovely gang scheduler... An alternate which might do better for you would be to have all your jobs use the same set of worker processes, one per cpu. Then multiple jobs would just send extra work and have to wait until the workers got around to handling them. Since PVM has full dynamic process creation, this is fairly easy to write -- N slaves, create a new master to hand out work each time you have a new job to do. -- greg _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From RSchilling at affiliatedhealth.org Mon Aug 21 12:41:20 2000 From: RSchilling at affiliatedhealth.org (Schilling, Richard) Date: Mon, 21 Aug 2000 09:41:20 -0700 Subject: NFS 100:1 performance loss Message-ID: <51FCCCF0C130D211BE550008C724149EBE0D79@mail1.affiliatedhealth.org> Slow NFS because of UDP jives with the spec behind UDP: the broadcasts are not guaranteed to arrive in a particular order, and are not guaranteed to arrive at all. TCP is definitely the way to go for fast and reliable. That's what FTP uses. Richard Schilling Web Integration Programmer Affiliated Health Services Mount Vernon, WA > -----Original Message----- > From: Mel Jones [mailto:mello at msg.ucsf.edu] > Sent: Friday, August 18, 2000 3:50 PM > To: josip at icase.edu > Cc: Beowulf mailing list > Subject: Re: NFS 100:1 performance loss > > > Josip Loncaric wrote: > > > Has anyone seen this NFS performance problem? > > > > Our Beowulf has two servers w/RAID-0 arrays which deliver 45MB/s, > > connected via Gigabit Ethernet. The machines are dual > PIII/500 systems > > with 512MB of RAM each. We are running Red Hat 6.2 updated > to kernel > > 2.2.16-3 and recently we updated to nfs-utils-0.1.9.1-1. > We are running > > 16 kernel nfsd threads on each machine. Here is how long > it takes to > > copy a 28,955,860 byte file from machine 1 to machine 2: > > > > rcp: 1.04 seconds (27.8 MB/s, where 1MB=10^6B) > > ftp: 1.12 seconds (25.8 MB/s) > > NFS 1KB: 12.35 seconds ( 2.3 MB/s, using rsize=wsize=1024) > > NFS 8KB: 129.42 seconds ( 0.2 MB/s, using rsize=wsize=8192) > > > > Clearly, there is something very wrong with NFS, > particularly with 8KB > > rsize/wsize (which should have improved performance!). Our system > > manager tells me that reduced Linux NFS performance with > 8KB rsize/wsize > > is a known problem, but even at 1KB our NFS is getting less > than 10% of > > the rcp or ftp performance. > > > > Any ideas? > > Josip > > > > -- > > Dr. Josip Loncaric, Senior Staff Scientist > mailto:josip at icase.edu > > ICASE, Mail Stop 132C PGP > key at http://www.icase.edu./~josip/ > > NASA Langley Research Center > mailto:j.loncaric at larc.nasa.gov > > Hampton, VA 23681-2199, USA > Tel. +1 757 864-2192 Fax +1 757 864-6134 > > > > _______________________________________________ > > Beowulf mailing list > > Beowulf at beowulf.org > > http://www.beowulf.org/mailman/listinfo/beowulf > > I've seen a similar but nowhere near as bad result on our > network. This may > be completely irrelevant but I found NFS traffic between my > SGI machines > about 4x slower than FTP. It got much better when I set the > transport to > TCP rather than UDP. The problem turned out to be that my > switches weren't > necessarily forwarding the packets in order of receipt and > this was taken > care of much more efficiently on the receiving machine with TCP. > > ... mello > > > > _______________________________________________ > Beowulf mailing list > Beowulf at beowulf.org > http://www.beowulf.org/mailman/listinfo/beowulf > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rmarshal at texas.net Mon Aug 21 15:53:46 2000 From: rmarshal at texas.net (Robert Marshall) Date: Mon, 21 Aug 2000 14:53:46 -0500 Subject: Facilities Requirements for Beowulf Clusters? In-Reply-To: Message-ID: > -----Original Message----- > From: beowulf-admin at beowulf.org [mailto:beowulf-admin at beowulf.org]On > Behalf Of Robert G. Brown > Sent: Thursday, August 17, 2000 9:09 PM > To: Don Morton > Cc: beowulf at beowulf.org; Ford, Ray; Robinson, Guy > Subject: Re: Facilities Requirements for Beowulf Clusters? > > > I realise I can calculate the power requirements and compare them > > to what's available in the room, but A/C is a little tougher. Have > > any of you tried jamming 16 to 32 nodes in a small room that's > typically > > kept at around 22C (72F) with any success? I hear about a number of > > medium-sized systems like the one I'm getting ready to acquire, > > and I wonder if people are really going all out to spend extra > > bucks on the facilities? > > I opened a home consulting business in December of 1998 in one bedroom (Southeast facing) of a four-bedroom two-story home in San Antonio. In December of 1999 I built and placed online a 10-CPU cluster. Here are my increases in energy costs for starting my business and for operating my cluster; Note that columns are cumulative for the year, and that the second two columns represent rising cost in dollars based on then current price of electricity, Ave usage ACFM elect. Cluster Computer before ACFM cost Electricity cost 1994-1997 1998-1999 2000 Month (kwh) $0.066/kwh $0.066/kwh January 2,305 $5.25 $21.09 February 4,597 $(1.45) $41.45 Mar 6,813 $(5.02) $84.68 Apr 9,254 $(4.01) $134.90 May 12,021 $53.49 $169.65 Jun 15,271 $119.15 $191.90 Jul 19,428 $141.22 $273.21 Aug 23,258 $173.04 Sept 26,555 $200.33 Oct 28,978 $228.71 Nov 31,364 $229.20 Dec 34,021 $238.29 > _______________________________________________ > Beowulf mailing list > Beowulf at beowulf.org > http://www.beowulf.org/mailman/listinfo/beowulf > Robert L Marshall A Computer For Me 23724 Up Mountain Road San Antonio TX 78255-2000 (210) 698-2133 rmarshal at texas.net _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From Phil.Williams at nottingham.ac.uk Mon Aug 21 17:08:51 2000 From: Phil.Williams at nottingham.ac.uk (Phil Williams) Date: Mon, 21 Aug 2000 22:08:51 +0100 Subject: PhD position available Message-ID: <4.3.2.7.2.20000821220559.00c35d50@pop.nottingham.ac.uk> Please pass this on to anyone who may be interested... We have a PhD position available in the UK from this October to develop computational chemistry code (molecular dynamics) to run over a 64-processor Beowulf on Myrinet. For details please see http://pharm6.pharm.nottingham.ac.uk/ Phil Phil Williams Lecturer in Biophysical Chemistry Laboratory of Biophysics and Surface Analysis School of Pharmaceutical Sciences The University of Nottingham, Nottingham NG7 2RD Tel: +44 (0)115 9515025 or +44 (0)7885 772666 Fax: +44 (0)115 9515110 Phil.Williams at nottingham.ac.uk http://pharm6.pharm.nottingham.ac.uk/ _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From levenson at valinux.com Mon Aug 21 17:27:17 2000 From: levenson at valinux.com (Howard Levenson) Date: Mon, 21 Aug 2000 17:27:17 -0400 Subject: BWBUG Meeting Change Message-ID: <39A19EB5.E1F9FA09@valinux.com> Hi, As a result of a scheduling conflict, Tarek El-Ghazawi, Associate Professor, at GMU will be presenting at this weeks BWBUG meeting. Tarek has graciously agreed to discuss the role of Beowulf computing in remote sensing and image processing. This meeting takes place, Tuesday, 8-22 at 3:00 P.M. Please check the web site for more background information. The location remains at LogiconDPC in Laurel, MD and other details can be found at http://bwbug.tux.org Please feel free to contact me with any questions. See you tomorrow. _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From itti at cco.caltech.edu Mon Aug 21 19:26:46 2000 From: itti at cco.caltech.edu (Laurent Itti) Date: Mon, 21 Aug 2000 16:26:46 -0700 (PDT) Subject: newbie: 16-node 500Mbps design In-Reply-To: Message-ID: Hi all: thanks again for all the very good comments. I'll summarize a few highlights: ethernet: --------- - "check out www.netperf.org for benchmarks" - "DEC Tulip chips seem to have a very good reputation re. throughput" - "the 3com (or any tulip) does full busmastering; the 8139 sort of does, but has limitations that require the OS to copy the packet much of the time. which is not necessarily a problem, since people have been doing copy-and-checksum in one step for years. the 3c9xx's, though, offload both the copy and the checksum." I see; that indeed can justify the price premium. chipsets: --------- - "in general, chipsets don't require "support" since they're usually quite back-compatible." But some chipsets do have explicit support in the kernel... and I guess it makes a big difference whether you access your disk via DMA ("supported chipset") or PIO ("fallback if chipset is unsupported"). - "ATA-100: why want it, current disks max out at 40MB/s?" then, when I add more drives in a year or two, I won't be limited by the bus speed? (2*40 > 66; but I agree that it's a bit overkill; just since the price premium is low, I'll give it a try). - "forget gentus; they're just hangers-on. you can get the official stuff from linux-ide.org or kernel.org" thanks! - "Abit will have a new dual-coppermine M.B. soon (whatever that means)." sounds good, I'll try it for the head of the system. buying linux: ------------- - "[buying one copy/node for retail Mandrake costs] less than 5% of total machine cost. The idea was to say "thank you" for the great OS. But saying "thank you" to mandrake is a bit silly, since they're just packagers." well, yes and no. Most of the time in trying to design the system I have spent in trying to match chipsets to drivers. Ultimately, I don't see why we should waste so much volunteer effort on hacking out sub-optimal linux drivers from partial docs. There are many more interesting programming challenges around. In the future, I want fully-optimized linux support for all hardware I buy -> only manufacturers can provide that. Packagers are the ones who increase general linux awareness so that manufacturers finally start realizing that they must provide Linux drivers... with so many boxes for RedHat and Mandrake in stores everywhere, it's getting more and more difficult to ignore them. just my 2 cents but I don't want to get too much off-topic. Thanks a lot to all, I'll post some benchmarks once we have the machine running ;-) -- laurent with contributions from Scott L. Delinger Gerry Creager N5JXS Mark Hahn Ioannis F Sotiriadis _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From deadline at plogic.com Tue Aug 22 08:18:31 2000 From: deadline at plogic.com (Douglas Eadline) Date: Tue, 22 Aug 2000 08:18:31 -0400 (EDT) Subject: Interesting package from IBM Message-ID: I just came across this at IBM. An interesting debugging method that may be useful for some debugging problems. I just glanced at the readme and it seems interesting (but not real simple to use) http://oss.software.ibm.com/developerworks/opensource/linux/projects/dprobes/ Doug ------------------------------------------------------------------- Paralogic, Inc. | PEAK | Voice:+610.814.2800 130 Webster Street | PARALLEL | Fax:+610.814.5844 Bethlehem, PA 18015 USA | PERFORMANCE | http://www.plogic.com ------------------------------------------------------------------- _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From rgb at phy.duke.edu Wed Aug 23 10:41:58 2000 From: rgb at phy.duke.edu (Robert G. Brown) Date: Wed, 23 Aug 2000 10:41:58 -0400 (EDT) Subject: Card/switch performance query Message-ID: Dear All, We are getting ready to design and fund an upgrade for our existing Intel-based environment. I need data/advice on two issues: a) I'm generally dissatisfied with the performance of the Netgear 10/100 card -- in most of my tests it peaks out (too) far below wire speed. Some years ago I gave up 3c905's in favor of tulips because the tulips outperformed them, were cheaper, and the 3c9xx cards had endless driver problems. I'm sure the world has moved on since then. SO my question: What 10/100 cards would you (any or all of you) recommend for use in a "standard design" beowulf today? Please address price, driver/card stability with current kernels, and if at all possible, quantitative performance. I can provide scripts to generate a "sweep" of tcp streaming throughput measurements for a range of message sizes from 1 to enough bytes to encompass a standard ethernet MTU using either netperf or (preferrably) the (new) bw_tcp tool in lmbench. I find that the full sweep provides invaluable information as well as reveals potentially serious performance problems. I'd also be interested in latency measurements (lat_tcp from lmbench or request/response measurements in netperf). Since latency and throughput definitely depends on switch type, please indicate what switch type you are using and whether it is cut-through or store-and-forward. I'm particularly interested in comparative performance/reliability/cost of the current 3c905, eepro100 and tulip-ish cards but would love to hear about any others that might have emerged as strong competitors as well. b) We'll very likely be upgrading/replacing our current aged Cisco Cat 5000 switch at the same time. I expect that we'll need to support as many as 64 nodes, although to connect to the department network proper we'll probably have to go through an uplink to another (similar?) switch. The old Cisco was very nice in its day but has somewhat broken Nway and we'd definitely like to get a switch that will last and give decent (100 base) performance for the next few years. On this list and in recent discussion, the HP ProCurve 4000M has been suggested several times by fairly smart folks (with the suggestion made by Don Becker, IIRC, that one just buy two and put the cards and power supply from the second back into the first to get an 80 port chassis, rather than either get the 80 ports all at once or try to upgrade). Another more recent suggestion (from Christopher Hogue) was to get a Foundry Networks Fastiron II, which is also the OEM of the HP 9000 series. I suspect that this is out of our budget range but I'd still be interested in price/reliability/performance reports. I know this is a perennial discussion item, but I have actually searched the archives already and although there is some anecdotal stuff there there aren't many true price/reliability/performance reports, especially with quantitative reporting of the numbers. In addition to your answers getting into the archives as reports in and of their own right, I'll try to take the time to post an executive summary when the thread peters out. rgb -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb at phy.duke.edu _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From t-hauptman at nwu.edu Wed Aug 23 10:54:47 2000 From: t-hauptman at nwu.edu (Traveler Hauptman) Date: Wed, 23 Aug 2000 09:54:47 -0500 Subject: OpenSSH problems Message-ID: <000801c00d12$15a262f0$8c456981@mech.nwu.edu> I am having trouble setting up OpenSSH to replace rsh. I would like eliminate the use of passwords for users going between machines on a guarded beowulf. What I *think* I want is to use RSA + Rhosts authentication. The docs say that all I need is a /etc/hosts.equiv and /etc/ssh_known_hosts. However, this doesn't seem to work. Is there anyone who could give me some pointers or better yet forward a copy of the config files and whatever other files I need? (feel free to munge the keys... :) Thanks, Traveler Hauptman Theoretical & Applied Mechanics Lab Northwestern University -------------- next part -------------- An HTML attachment was scrubbed... URL: From rmcgaugh at atipa.com Wed Aug 23 14:40:55 2000 From: rmcgaugh at atipa.com (Rocky McGaugh) Date: Wed, 23 Aug 2000 13:40:55 -0500 (CDT) Subject: OpenSSH problems In-Reply-To: <000801c00d12$15a262f0$8c456981@mech.nwu.edu> Message-ID: On Wed, 23 Aug 2000, Traveler Hauptman wrote: > I am having trouble setting up OpenSSH to replace rsh. I would like eliminate the use of passwords for users going between machines on a guarded beowulf. What I *think* I want is to use RSA + Rhosts authentication. The docs say that all I need is a /etc/hosts.equiv and /etc/ssh_known_hosts. However, this doesn't seem to work. Is there anyone who could give me some pointers or better yet forward a copy of the config files and whatever other files I need? (feel free to munge the keys... :) > > Thanks, > > Traveler Hauptman > Theoretical & Applied Mechanics Lab > Northwestern University > on the client, run ssh-keygen to create the users pub and private keys. copy the users key (~/.ssh/identity.pub) to the server you are attempting to login to into the file ~/.ssh/authorized_keys. good luck and have fun..:) -- Rocky McGaugh rmcgaugh at atipa.com _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From rajkumar at csse.monash.edu.au Wed Aug 23 22:30:55 2000 From: rajkumar at csse.monash.edu.au (Rajkumar Buyya) Date: Thu, 24 Aug 2000 12:30:55 +1000 Subject: A Report on Cluster Computing R&D in Australia References: <472E67987253@listmanager.infotech.monash.edu.au> Message-ID: <39A488DF.9C2C472E@csse.monash.edu.au> Dear All, A TFCC report on "Cluster Computing R&D in Australia" by M. Baker, R. Buyya, K. Hawick, H. James, and H. Jin, published as part of the: Asian Technology Information Program (ATIP), Japan/USA, April, 2000. is now made public and freely available for the benefit of community. ABSTRACT: This report provides overview of Cluster Computing R&D activities in the Asia-Pacific region with particular focus on Australian research activities. It emerged as an outcome of the first annual cluster computing meeting of the IEEE Task Force on Cluster Computing (http://www.ieeetfcc.org) held in Melbourne, Australia, Dec. 1999. We discuss details of R&D at ten Australian universities as well as highlights of research at several other sites in Asia. FULL REPORT: http://www.ieeetfcc.org/press.html OR http://www.csse.monash.edu.au/~rajkumar/papers/ClusterComputingAU.pdf Thank you very much. Sincerely Yours R. Buyya (On behalf of the IEEE Task Force on Cluster Computing) _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From t-hauptman at nwu.edu Thu Aug 24 09:45:56 2000 From: t-hauptman at nwu.edu (Traveler Hauptman) Date: Thu, 24 Aug 2000 08:45:56 -0500 Subject: SSH clarification Message-ID: Thank you all for your replies to my inquiry. I realized when I got them that I didn't ask the question quite right. I will be the system administrator for a small beowulf with a medium number of users who are as a rule not experts with unix when they start using the system. I would like to set up secure shell so that it works globaly for all users without every individual having to set up keys or any other files. The OpenSSH man page suggests that this is possible by using a /etc/ssh_known_hosts and /etc/hosts.equiv calling it RSA+Rhosts Authentication. I am using RPM's for RH6.2. Is there anyone out there who has set up OpenSSH to use this type of authentication? Otherwise I guess I will just write a script to set up the keys during new user adds. Thanks, Traveler Hauptman _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From lusk at mcs.anl.gov Thu Aug 24 10:18:38 2000 From: lusk at mcs.anl.gov (Rusty Lusk) Date: Thu, 24 Aug 2000 09:18:38 -0500 Subject: probs running mpqc In-Reply-To: Message from zolia of "Sun, 13 Aug 2000 20:04:26 +0300." Message-ID: <200008241418.JAA193796@mcs.anl.gov> It is best to send problems with mpich to mpi-bugs at mcs.anl.gov. | It seems that it tries to connect to 753 port, but that is neither rlogin, | nor rexec or rsh port. 753 is the default port for the secure server. Be sure to read the Installation and User Guides on this topic if you are intentionally using the secure server. It is not the default. | .p4pg contains: | host 0 | node1 1 This is not a valid procgroup file. The User's Guide describes the use of procgroup files if you choose to use them. You don't need to. Regards, Rusty Lusk _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From bropers at lsu.edu Thu Aug 24 12:32:09 2000 From: bropers at lsu.edu (bropers at lsu.edu) Date: Thu, 24 Aug 2000 11:32:09 -0500 (CDT) Subject: SSH clarification In-Reply-To: Message-ID: I guess I'm still confused. If you don't want the users to have to type in a password each time (ala rsh), the only thing I know to do is create a key for each user and stick it on each of the nodes they need access to. I've done this once, but only as a test though, it's too much of a security hole for me (that's why we've all turned off rsh in the first place, right?). Maybe you an describe how you want the system to be from a users perspective? On Thu, 24 Aug 2000, Traveler Hauptman wrote: > Thank you all for your replies to my inquiry. I realized when I got them > that I didn't ask the question quite right. I will be the system > administrator for a small beowulf with a medium number of users who are as > a > rule not experts with unix when they start using the system. I would like > to > set up secure shell so that it works globaly for all users without every > individual having to set up keys or any other files. > > The OpenSSH man page suggests that this is possible by using a > /etc/ssh_known_hosts and /etc/hosts.equiv calling it RSA+Rhosts > Authentication. I am using RPM's for RH6.2. > > Is there anyone out there who has set up OpenSSH to use this type of > authentication? > > Otherwise I guess I will just write a script to set up the keys during new > user adds. > > Thanks, > > Traveler Hauptman -- Brian D. Ropers-Huilman (225) 388-0461 (V) Systems Administrator (225) 388-6400 (F) Office of Computing Services bropers at lsu.edu High Performance Computing http://www.ropers-huilman.net/ Fred Frey Building, Rm. 201, E-1Q -- \o/ Louisiana State University -- __o --- | Baton Rouge, LA 70803-1900 --- `\<, `\\, O/ O O/ O _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From josip at icase.edu Thu Aug 24 15:03:32 2000 From: josip at icase.edu (Josip Loncaric) Date: Thu, 24 Aug 2000 15:03:32 -0400 Subject: Card/switch performance query References: Message-ID: <39A57184.6B50256D@icase.edu> "Robert G. Brown" wrote: > > What 10/100 cards would you (any or all of you) recommend for use in a > "standard design" beowulf today? My experience is limited to tulip cards (NetGear FA310TX and Kingston KNE100TX). We also had driver problems because the same network card model can actually contain umpteen varieties of chips and board revisions, each one with its own quirks. It is difficult for the driver to recover from so many different quirks when pushing the network to the bleeding edge. Perhaps some other cards have greater hardware uniformity, but 'hardware uniformity' and 'mass market' generally do not go together. Unless someone convinces me that the situation is truly better with another card, I'm inclined to stick with tulips. At least we learned how to deal with their quirks... > b) We'll very likely be upgrading/replacing our current aged Cisco Cat > 5000 switch at the same time. We've had good luck with 3com's SuperStack II Switch 3900 (the 36-port version, 3C39036). 3com changed its marketing strategy recently, and I'm not sure if 3C39036 was affected or not. Sincerely, Josip -- Dr. Josip Loncaric, Senior Staff Scientist mailto:josip at icase.edu ICASE, Mail Stop 132C PGP key at http://www.icase.edu./~josip/ NASA Langley Research Center mailto:j.loncaric at larc.nasa.gov Hampton, VA 23681-2199, USA Tel. +1 757 864-2192 Fax +1 757 864-6134 _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From cgreer1 at midsouth.rr.com Fri Aug 25 00:07:05 2000 From: cgreer1 at midsouth.rr.com (Chris Greer) Date: Thu, 24 Aug 2000 23:07:05 -0500 Subject: SSH clarification References: Message-ID: <39A5F0E9.A9F1DB0D@midsouth.rr.com> I haven't used OpenSSH, but I know the regular ssh has an shost.equiv for this purpose. Chris G. Traveler Hauptman wrote: > > Thank you all for your replies to my inquiry. I realized when I got them > that I didn't ask the question quite right. I will be the system > administrator for a small beowulf with a medium number of users who are as a > rule not experts with unix when they start using the system. I would like to > set up secure shell so that it works globaly for all users without every > individual having to set up keys or any other files. > > The OpenSSH man page suggests that this is possible by using a > /etc/ssh_known_hosts and /etc/hosts.equiv calling it RSA+Rhosts > Authentication. I am using RPM's for RH6.2. > > Is there anyone out there who has set up OpenSSH to use this type of > authentication? > > Otherwise I guess I will just write a script to set up the keys during new > user adds. > > Thanks, > > Traveler Hauptman > > _______________________________________________ > Beowulf mailing list > Beowulf at beowulf.org > http://www.beowulf.org/mailman/listinfo/beowulf _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From jalton at olsh.cx Fri Aug 25 18:02:33 2000 From: jalton at olsh.cx (James Alton) Date: Fri, 25 Aug 2000 15:02:33 -0700 Subject: Beowulf cluster Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I'm looking to build a cheap beowulf cluster. I need it to fit into a 19" rackmount enclosure. I was wondering if anyone had any good sources on a very simple way to mount a motherboard into the 19" form. I need a piece of metal (bare shelf) that has raised places with holes to mount a standard ATX motherboard, and on the same piece of metal, perhaps 2-4 3.5 inch devices. As well, there should be holes that make this piece of metal look like cheese. (This is for air flow.) Also, there has to be an area to mount an ATX power supply. I am willing to pay 20-40 bucks per piece. This seems to be way cheaper than an enclosed node system, each enclosure costs like $250? This seems outrageous for a piece of metal and power supply. If possible, I would also like the motherboard mount to be able to slide out. If no one has any information on this, I would even consider taking a metal shop class, or getting commercial production for such a bare motherboard rackmount. It seems like something there is a great need for. :James Alton jalton at olsh.cx -----BEGIN PGP SIGNATURE----- Version: PGPfreeware 6.5.3 for non-commercial use iQA/AwUBOabs+J/SF5seuRySEQLxtgCaA/WwBPFEKqPCIXy/XlPZ9A5HxkEAn1Mk elIcZyQiNlwuwymIOMlv4yfO =8A6r -----END PGP SIGNATURE----- _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From pratte at lincweb.com Fri Aug 25 20:51:17 2000 From: pratte at lincweb.com (Robert Pratte) Date: Fri, 25 Aug 2000 19:51:17 -0500 Subject: Beowulf cluster References: Message-ID: <39A71485.53ECF111@lincweb.com> Years ago, I mounted my old Atari 800 boards using wooden dowel cut to length for spacers. I transferred the mounting points onto paper by placing the main board on a sheet of paper and marking the holes with a pen, then taped the paper to the metal and drilled through. I cut a number of equal length wooden dowel rod, wrapped the ends with electrical tape (to help prevent splitting), and drilled the centers with an undersized bit. Screw everything into place, and you're done. Of course, swiss cheese can be acheived after you have marked your holes via a hole saw. It should be fairly easy to acheive what you are after using this technique. Another interesting idea might be to try to adapt old bread trays. The hard part here will be mounting your equipment, but I could think of a few designs. Not sure if this helps. James Alton wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > I'm looking to build a cheap beowulf cluster. I need it to fit into a > 19" rackmount enclosure. > I was wondering if anyone had any good sources on a very simple way > to mount a motherboard > into the 19" form. I need a piece of metal (bare shelf) that has > raised places with holes > to mount a standard ATX motherboard, and on the same piece of metal, > perhaps 2-4 3.5 inch devices. > As well, there should be holes that make this piece of metal look > like cheese. (This is for air > flow.) Also, there has to be an area to mount an ATX power supply. I > am willing to pay 20-40 bucks > per piece. This seems to be way cheaper than an enclosed node system, > each enclosure costs like > $250? This seems outrageous for a piece of metal and power supply. If > possible, I would also like > the motherboard mount to be able to slide out. If no one has any > information on this, > I would even consider taking a metal shop class, or getting > commercial production for such a > bare motherboard rackmount. It seems like something there is a great > need for. > > :James Alton > jalton at olsh.cx > > -----BEGIN PGP SIGNATURE----- > Version: PGPfreeware 6.5.3 for non-commercial use > > iQA/AwUBOabs+J/SF5seuRySEQLxtgCaA/WwBPFEKqPCIXy/XlPZ9A5HxkEAn1Mk > elIcZyQiNlwuwymIOMlv4yfO > =8A6r > -----END PGP SIGNATURE----- > > _______________________________________________ > Beowulf mailing list > Beowulf at beowulf.org > http://www.beowulf.org/mailman/listinfo/beowulf _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From webmaster at tgirouard.com Sat Aug 26 05:38:42 2000 From: webmaster at tgirouard.com (webmaster) Date: Sat, 26 Aug 2000 05:38:42 -0400 Subject: Beowulf cluster In-Reply-To: Message-ID: I have been thinking the exact same thing and have a number of resources to make up such a mounting system. How many boards do you want to put into it? I will also talk to an engineer friend so that one power supply can power n boards to keep down the heat as well as mounting some large fans. Terry Girouard webmaster at tgirouard.com PO Box 8803 Burlington, VT 05401 802-862-3553 -----Original Message----- From: beowulf-admin at beowulf.org [mailto:beowulf-admin at beowulf.org]On Behalf Of James Alton Sent: Friday, August 25, 2000 6:03 PM To: beowulf at beowulf.org Subject: Beowulf cluster -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I'm looking to build a cheap beowulf cluster. I need it to fit into a 19" rackmount enclosure. I was wondering if anyone had any good sources on a very simple way to mount a motherboard into the 19" form. I need a piece of metal (bare shelf) that has raised places with holes to mount a standard ATX motherboard, and on the same piece of metal, perhaps 2-4 3.5 inch devices. As well, there should be holes that make this piece of metal look like cheese. (This is for air flow.) Also, there has to be an area to mount an ATX power supply. I am willing to pay 20-40 bucks per piece. This seems to be way cheaper than an enclosed node system, each enclosure costs like $250? This seems outrageous for a piece of metal and power supply. If possible, I would also like the motherboard mount to be able to slide out. If no one has any information on this, I would even consider taking a metal shop class, or getting commercial production for such a bare motherboard rackmount. It seems like something there is a great need for. :James Alton jalton at olsh.cx -----BEGIN PGP SIGNATURE----- Version: PGPfreeware 6.5.3 for non-commercial use iQA/AwUBOabs+J/SF5seuRySEQLxtgCaA/WwBPFEKqPCIXy/XlPZ9A5HxkEAn1Mk elIcZyQiNlwuwymIOMlv4yfO =8A6r -----END PGP SIGNATURE----- _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From josip at icase.edu Mon Aug 28 10:58:25 2000 From: josip at icase.edu (Josip Loncaric) Date: Mon, 28 Aug 2000 10:58:25 -0400 Subject: Beowulf cluster References: Message-ID: <39AA7E11.50532A62@icase.edu> James Alton wrote: > > I was wondering if anyone had any good sources on a very simple way > to mount a motherboard > into the 19" form. A while back I saw an advertisement for a rack full of bare shelves where PC components could be mounted (ATX motherboard, power supply, disk, fan). Apparently, this is intended for server farms. The outfit which advertised this was http://www.kingstarusa.com/index_netscape.html but I did not pursue this further. I have no idea whatsoever if their stuff actually works, nor how much it costs. Sincerely, Josip -- Dr. Josip Loncaric, Senior Staff Scientist mailto:josip at icase.edu ICASE, Mail Stop 132C PGP key at http://www.icase.edu./~josip/ NASA Langley Research Center mailto:j.loncaric at larc.nasa.gov Hampton, VA 23681-2199, USA Tel. +1 757 864-2192 Fax +1 757 864-6134 _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From camm at enhanced.com Mon Aug 28 11:59:20 2000 From: camm at enhanced.com (Camm Maguire) Date: 28 Aug 2000 11:59:20 -0400 Subject: LAM: SCALAPACK users? In-Reply-To: "Horatio B. Bogbindero"'s message of "Mon, 28 Aug 2000 08:49:01 +0800 (PHT)" References: Message-ID: <54ya1hqsvb.fsf@intech9.enhanced.com> Greetings! "Horatio B. Bogbindero" writes: > are there any scalapack users in the list? i have a few questions to ask: > Here! > -is there s C/C++ implementation of SCALAPACK? if not, can i call the > fortran SCALAPACK/PBLAS functions from C/C++? > No c interface, to use routines: 1) add a '_' after routine name 2) Pass all arguments as pointers (i.e. an arg of '1' would have to be passed as &i, with i having been set to 1.) 3) Link with g77 > -are there any good scalapack documentation/manuals out there? the > scalapack site only feature some lawns but nothing like a users manual. > The faq and user's guide at netlib are both good. They're included in the new Debian scalapack-doc package. Take care, > > thanks. > > --------------------- > william.s.yu at ieee.org > > Misfortune, n.: > The kind of fortune that never misses. > -- Ambrose Bierce, "The Devil's Dictionary" > > > _______________________________________________ > This list is archived at http://www.mpi.nd.edu/MailArchives/lam/ > > -- Camm Maguire camm at enhanced.com ========================================================================== "The earth is but one country, and mankind its citizens." -- Baha'u'llah _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From jerosejr at cajunbro.com Mon Aug 28 15:58:15 2000 From: jerosejr at cajunbro.com (Joseph E. Rose, Jr.) Date: Mon, 28 Aug 2000 14:58:15 -0500 Subject: Building Beowulf cluster in 19" rack In-Reply-To: <39AA7E11.50532A62@icase.edu> Message-ID: <000701c0112a$4e972200$7d15a8c0@cajunbro.int> Well, I have an 18 node cluster built inside of one 19" rack enclosure. I made a series of front-to-back "L" rails, mounted them about 9" apart, and added two side-to-side rails for support. I then cut a 4x8 sheet of pegboard (actually several) in the dimensions to slide in and out on the rails. Mount the rails with the "L" pointed down, and against the side of the cabinet. I have two nodes per board, mounted with cable-ties for ease of maintenance. Add a couple of Tripplite power strips on both doors, the switch and a BIG UPS in the bottom, and the fans on the back door. It's been running almost a month, and no other changes have popped up. I don't have my webpage completed yet, but will post a picture and let you know when I do. Have a good day, Joseph E. Rose, Jr. BRISK System Analyst Illgen Simulation Technologies, Inc. (210) 348-6886/ Cell: (210) 710-9060 San Antonio, Texas, U.S.A "What you can do, or dream you can, begin it; Boldness has genius, power, and magic in it." --Goethe -----Original Message----- From: beowulf-admin at beowulf.org [mailto:beowulf-admin at beowulf.org]On Behalf Of Josip Loncaric Sent: Monday, August 28, 2000 9:58 AM Cc: beowulf at beowulf.org Subject: Re: Beowulf cluster James Alton wrote: > > I was wondering if anyone had any good sources on a very simple way > to mount a motherboard > into the 19" form. A while back I saw an advertisement for a rack full of bare shelves where PC components could be mounted (ATX motherboard, power supply, disk, fan). Apparently, this is intended for server farms. The outfit which advertised this was http://www.kingstarusa.com/index_netscape.html but I did not pursue this further. I have no idea whatsoever if their stuff actually works, nor how much it costs. Sincerely, Josip -- Dr. Josip Loncaric, Senior Staff Scientist mailto:josip at icase.edu ICASE, Mail Stop 132C PGP key at http://www.icase.edu./~josip/ NASA Langley Research Center mailto:j.loncaric at larc.nasa.gov Hampton, VA 23681-2199, USA Tel. +1 757 864-2192 Fax +1 757 864-6134 _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From hahn at coffee.psychology.mcmaster.ca Mon Aug 28 16:34:21 2000 From: hahn at coffee.psychology.mcmaster.ca (Mark Hahn) Date: Mon, 28 Aug 2000 16:34:21 -0400 (EDT) Subject: newbie: 16-node 500Mbps design In-Reply-To: Message-ID: > > there seems to be quite a lot of urban legendry here. I certainly > > don't see any "TCP stalls" or know anyone who does. perhaps on crappy > > Go read http://www.icase.edu/coral/LinuxTCP2.html, then you'll know about > someone who does. I've also had problems with various 2.2.x kernels. Getting no. Josip's (fine) works is a specific tuning for small-packet performance; it violates the standards, or at least accepted practice for TCP. that's fine for tweaking your cluster, but it does NOT show a general problem with stalls. it's a little unclear to me why he calls these events "deadlocks", since afaikt, they're simply retransmit timeouts in TCP terminology, part of TCP's congestion-avoidance heuristics. there's nothing wrong with breaking TCP for intra-cluster performance; TCP might even be the wrong basic design for a switched, never-congested, single-hop network. > the right driver for our tulip cards has been a pain too. Each version will > work with a different set of cards. Go check the archives of this list and I don't know of any card-specific tulip problems in modern (2.4) kernels. > > old 2.2 kernels, but anyone who runs them deserves what they get. > > I could also say anyone who runs experimental kernels on a production system > deserves what they get. oh, you mean excellent performance and improved stability? yes, you're right. if you're satisfied with 2.2, good for you! but it's just plain dishonest to pretend that 2.2 is better than 2.4 because it's code is, uh, "more mature". 2.4 has several improvements that are relevant to cluster computing, as well as fixes that result in better stability. regards, mark hahn. _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From glindahl at hpti.com Mon Aug 28 16:52:21 2000 From: glindahl at hpti.com (Greg Lindahl) Date: Mon, 28 Aug 2000 16:52:21 -0400 Subject: newbie: 16-node 500Mbps design In-Reply-To: Message-ID: <000f01c01131$dc5b1400$063efea9@hptilap.hpti.com> > > I could also say anyone who runs experimental kernels on a > production system > > deserves what they get. > > oh, you mean excellent performance and improved stability? > yes, you're right. if you're satisfied with 2.2, good for you! > > but it's just plain dishonest to pretend that 2.2 is better than 2.4 > because it's code is, uh, "more mature". Um, just in case anyone is wondering: 2.2 and 2.4 are both stable kernels. 2.4 happens to be brand new. 2.3 was experimental. This conversation makes me think that people are likely to be confused on this issue. And many, many people would not find it dishonest to prefer 2.2 over 2.4, because although 2.4 has received much more testing than any release of 2.3, 2.4 is still new. No pretending required; it's a simple fact. When 2.4 hits version 2.4.5 or so and starts shipping in major distributions, then it will be considered mature. -- g _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From xyzzy at speakeasy.org Mon Aug 28 17:16:17 2000 From: xyzzy at speakeasy.org (Trent Piepho) Date: Mon, 28 Aug 2000 14:16:17 -0700 (PDT) Subject: newbie: 16-node 500Mbps design In-Reply-To: Message-ID: On Mon, 28 Aug 2000, Mark Hahn wrote: > > > there seems to be quite a lot of urban legendry here. I certainly > > > don't see any "TCP stalls" or know anyone who does. perhaps on crappy > > > > Go read http://www.icase.edu/coral/LinuxTCP2.html, then you'll know about > > someone who does. I've also had problems with various 2.2.x kernels. Getting > > no. Josip's (fine) works is a specific tuning for small-packet performance; > it violates the standards, or at least accepted practice for TCP. So the people who have problems with mpich hanging are just imagining it? > > the right driver for our tulip cards has been a pain too. Each version will > > work with a different set of cards. Go check the archives of this list and > > I don't know of any card-specific tulip problems in modern (2.4) kernels. I've got 11 tulip cards that won't work with 2.4 kernels, but will work with the 0.90q version of the tulip driver. Just because you don't know of something, doesn't mean it doesn't exist. _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From xyzzy at speakeasy.org Mon Aug 28 17:29:31 2000 From: xyzzy at speakeasy.org (Trent Piepho) Date: Mon, 28 Aug 2000 14:29:31 -0700 (PDT) Subject: newbie: 16-node 500Mbps design In-Reply-To: <000f01c01131$dc5b1400$063efea9@hptilap.hpti.com> Message-ID: On Mon, 28 Aug 2000, Greg Lindahl wrote: > Um, just in case anyone is wondering: > > 2.2 and 2.4 are both stable kernels. 2.4 happens to be brand new. So new in fact, that, according to Linus, "It doesn't really exist yet." The 2.4 kernel is still at test7 and hasn't been released. _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From josip at icase.edu Mon Aug 28 19:00:31 2000 From: josip at icase.edu (Josip Loncaric) Date: Mon, 28 Aug 2000 19:00:31 -0400 Subject: newbie: 16-node 500Mbps design References: Message-ID: <39AAEF0F.D69E6A86@icase.edu> Mark Hahn wrote: > > no. Josip's (fine) works is a specific tuning for small-packet performance; > it violates the standards, or at least accepted practice for TCP. Using retransmit timeouts shorter than 200ms may break TCP connections to older BSD hosts. However, this fixed 200ms floor value of the retransmit interval is ridiculous on a closed Beowulf network (200ms at 100Mbit/s represents 2.5 MBytes). BTW, retransmit timeout is adaptively estimated by TCP, so limiting this estimate from below by using the floor of 20ms (which Linux can easily handle on Intels) is appropriate for Beowulf use. > that's fine for tweaking your cluster, but it does NOT show a general problem > with stalls. it's a little unclear to me why he calls these events > "deadlocks", since afaikt, they're simply retransmit timeouts in TCP > terminology, part of TCP's congestion-avoidance heuristics. TCP stalls happen often, but as long as there is a good reason (e.g. congestion) I would not call them 'deadlocks'. However, when both sender and receiver have the capacity to transfer more data, but are forced to wait for a timeout because of a deadlock in TCP logic, then the term is appropriate. One form of this deadlock is described in: "How a large ATM MTU causes deadlocks in TCP data transfers," by Kjersti Moldeklev and Per Gunningberg, IEEE/ACM Trans. on Networking, v3, No. 4, Aug. 1995, pp. 409-422. (see http://www2.comp.polyu.edu.hk/~comp555/INPSII/deadlock.pdf) A common feature of this deadlock and the one my patch addresses is the fact that delayed ACKs could be mistaken for network congestion. My simple fix reduces the probability of deadlocks by using immediate ACKs with (adjustable) probability (we use p=1/8). In seeking a deadlock-free TCP, others have proposed a more elaborate Adaptive Acknowledgment Algorithm: Adam Yeung and Rocky K. C. Chang, "Improving TCP Throughput Performance on High-Speed Networks with a Receiver-Side Adaptive Acknowledgment Algorithm." (see http://www2.comp.polyu.edu.hk/~comp555/INPSII/d2-5.pdf) Sincerely, Josip -- Dr. Josip Loncaric, Senior Staff Scientist mailto:josip at icase.edu ICASE, Mail Stop 132C PGP key at http://www.icase.edu./~josip/ NASA Langley Research Center mailto:j.loncaric at larc.nasa.gov Hampton, VA 23681-2199, USA Tel. +1 757 864-2192 Fax +1 757 864-6134 _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From hahn at coffee.psychology.mcmaster.ca Mon Aug 28 20:12:09 2000 From: hahn at coffee.psychology.mcmaster.ca (Mark Hahn) Date: Mon, 28 Aug 2000 20:12:09 -0400 (EDT) Subject: newbie: 16-node 500Mbps design In-Reply-To: <000f01c01131$dc5b1400$063efea9@hptilap.hpti.com> Message-ID: > And many, many people would not find it dishonest to prefer 2.2 over 2.4, > because although 2.4 has received much more testing than any release of 2.3, > 2.4 is still new. No pretending required; it's a simple fact. When 2.4 hits > version 2.4.5 or so and starts shipping in major distributions, then it will > be considered mature. the labels are problematic: 2.2 is called "stable", but is not more stable (robust). multiple revisions make a release "mature", but does not imply improvement of any kind; nor does more testing. the only truely relevant benchmark for your cluster is *your* code. similarly, other people's random testing (of 2.2 or 2.4) is only weak evidence of how it'll run on your cluster. if your application does little IO and makes few syscalls, and is running happily under 2.2, then don't bother looking at 2.4. reality is that 2.2 is merely the state of Linux circa late summer 1998. by definition, it lacks the 2 years of improvements that 2.3/2.4 benefits from. many of those improvements are directly relevant to clusters, especially the major changes to SMP locking, VM, networking. many of the changes during 2.3 were redesigns to solve problems inherent to 2.2, not added functionality. regards, mark hahn. _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From itti at cco.caltech.edu Mon Aug 28 23:22:32 2000 From: itti at cco.caltech.edu (Laurent Itti) Date: Mon, 28 Aug 2000 20:22:32 -0700 (PDT) Subject: Beowulf cluster In-Reply-To: Message-ID: Hi James: I have built many similar machines using plexiglass as a base, and a good drill press from the student machine shop. Typically, you make a template using a sheet of metal from home depot, and then mass-drill all your actual boards, by clamping the template onto a stack of boards (number will depend on plexiglass thickness; start with just one, as trying to do too many can be disastrous). Remember rule No. 1 with the drill press: "it brakes for nobody," so be sure to never hold your work yourself, but have it clamped to the workplan instead and keep your hands clear ;-) For the M.B. mounts, you can buy motherboard standoffs at good PC part stores (the kind of guys who specialize in connectors and adapters). there are two problems, though: 1) cooling. Recent motherboards will work much better if you have some air flow; you seem to suggest that having many holes in the sheet of metal will do the trick, but keep in mind that the motherboard does not have such holes, so that will kill any vertical air flow in your rack. In my experience, 2 fans pushing air into the front of the box and 2 pulling out at the back gives infinite uptime (i.e., for me, never reboot unless city power failure). If some of your chips run hot, I recommend gluing heatsinks on them with heat-conducting epoxy glue (use very little, otherwise it will flow over the pins of the chips as you press the heatsink into place; not a problem if your glue is not current-conductive, but looks messy). 2) sliding only the motherboard will be impractical because it will have lots of cabling connected to it -> you should slide the whole assembly out; that way, you just have to disconnect power and ether (if your cables are short) before sliding out. just my 2 cents... -- laurent _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From itti at cco.caltech.edu Tue Aug 29 02:32:35 2000 From: itti at cco.caltech.edu (Laurent Itti) Date: Mon, 28 Aug 2000 23:32:35 -0700 (PDT) Subject: USB floppy? Message-ID: Hi all: has anybody experience with installing Linux off a USB floppy drive? I did it on my slim laptop (no integrated floppy; just an external USB one) and it worked great as long as only one disk was needed (i.e., would not recognize the drive after the first floppy had been loaded). since modern motherboards all have USB, maybe we could forget about floppy drives in clusters and just have one external USB drive around for emergencies? I wonder whether that was a special feature of my laptop's BIOS of if it will also work with desktop motherboards? thanks! -- laurent _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From David_Walters at sra.com Tue Aug 29 08:34:10 2000 From: David_Walters at sra.com (Walters, David) Date: Tue, 29 Aug 2000 08:34:10 -0400 Subject: Beowulf cluster Message-ID: Check out audio and sound reinforcement rack equipment companies. I own a tray that is similar to what you describe, in my sound reinforcement rack. Standard 19", pull out tray, full of holes for mounting various OEM audio products, but easily modifiable. Start with a company called Raxxess (www.raxxess.com). -----Original Message----- From: James Alton [mailto:jalton at olsh.cx] Sent: Friday, August 25, 2000 6:03 PM To: beowulf at beowulf.org Subject: Beowulf cluster -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I'm looking to build a cheap beowulf cluster. I need it to fit into a 19" rackmount enclosure. I was wondering if anyone had any good sources on a very simple way to mount a motherboard into the 19" form. I need a piece of metal (bare shelf) that has raised places with holes to mount a standard ATX motherboard, and on the same piece of metal, perhaps 2-4 3.5 inch devices. As well, there should be holes that make this piece of metal look like cheese. (This is for air flow.) Also, there has to be an area to mount an ATX power supply. I am willing to pay 20-40 bucks per piece. This seems to be way cheaper than an enclosed node system, each enclosure costs like $250? This seems outrageous for a piece of metal and power supply. If possible, I would also like the motherboard mount to be able to slide out. If no one has any information on this, I would even consider taking a metal shop class, or getting commercial production for such a bare motherboard rackmount. It seems like something there is a great need for. :James Alton jalton at olsh.cx -----BEGIN PGP SIGNATURE----- Version: PGPfreeware 6.5.3 for non-commercial use iQA/AwUBOabs+J/SF5seuRySEQLxtgCaA/WwBPFEKqPCIXy/XlPZ9A5HxkEAn1Mk elIcZyQiNlwuwymIOMlv4yfO =8A6r -----END PGP SIGNATURE----- _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From bremner at unb.ca Tue Aug 29 12:34:45 2000 From: bremner at unb.ca (David Bremner) Date: Tue, 29 Aug 2000 13:34:45 -0300 (ADT) Subject: SCI vs Myrinet for Alpha Message-ID: <14763.58917.882076.378278@convex.cs.unb.ca> I looked through the archives as best I could (Metaquestion: Is there a better way to search the unified archives than google?) and found an informative posting from Chris Worley, and some older discussion (from 1997). Having noticed that Dolphin now has supposedly support for Alpha (look on www.alphanews.net for links), I wonder if there are compelling reasons to consider SCI instead of Myrinet. I am looking at about 8 2 proc UP2000 nodes (well, this is a separate flamewar;-) ) or 16 DS10L's The downsides as I see it are 1) The drivers are not open source. Well, this is not just an ideological issue for me; it seems like a bit of a maintenence nightmare. 2) At this size, the extra cost of the adapters more or less pays for a myrinet switch, so the switchless topology does not help much. With 16 nodes, the advantage seems to tilt further to myrinet. Also, I don't know about software prices, i.e. license fees for ScaliMPI 3) The support for Myrinet on alpha seems much more mature. Well, it sounds like my mind is made up, but I am really just starting to investigate. _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From brua at paralline.com Tue Aug 29 16:01:40 2000 From: brua at paralline.com (Pierre Brua) Date: Tue, 29 Aug 2000 22:01:40 +0200 Subject: SCI vs Myrinet for Alpha References: <14763.58917.882076.378278@convex.cs.unb.ca> Message-ID: <39AC16A4.78D56FF4@paralline.com> David Bremner wrote: > 1) The drivers are not open source. Well, this is not just an > ideological issue for me; it seems like a bit of a maintenence > nightmare. I asked Dolphinics about that, more than one year ago. They said it was not on their roadmap to open their driver or GPLize it because they want to "avoid code forking and protocols incompatibility problems between forked drivers". Well, that cleared the point for me... I don't know about their future plans. I don't know wether ScaliMPI is GPL or not. I suppose it's not since I never saw a download page of it. Paralline is currently a Myrinet reseller, in case you would happen to ask :-) Hope it helps, Pierre BRUA -- PARALLINE /// Parallelism & Linux /// 71,av. des Vosges Phone:+33 388 141 740 mailto:brua at paralline.com F-67000 STRASBOURG Fax:+33 388 141 741 http://www.paralline.com _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From Tom.Morris at alpha-processor.com Tue Aug 29 16:23:06 2000 From: Tom.Morris at alpha-processor.com (Tom Morris) Date: Tue, 29 Aug 2000 16:23:06 -0400 Subject: SCI vs Myrinet for Alpha Message-ID: <051DFF3BBA73D3119A5800A0C95BD0215875A5@barracuda.api-networks.com> David Bremner [SMTP:bremner at unb.ca] wrote: > Having noticed that Dolphin now has supposedly support for Alpha (look > on www.alphanews.net for links), I wonder if there are > compelling reasons to consider SCI instead of Myrinet. > > I am looking at about 8 2 proc UP2000 nodes What are your buying criteria? Bandwidth? Latency? Price? Each of the interconnects currently supporting Alpha (Myricom, Dolphin/Scali, Quadrics) has different strengths. I have clusters using all three. When do you plan to buy? Myricom has introduced interface cards with higher link speeds and has said that they will soon introduce matching switches. Scali has said that it will soon introduce a new generation of product with 64-bit PCI and higher speed links. > 1) The drivers are not open source. Well, this is not just an > ideological issue for me; it seems like a bit of a maintenence > nightmare. It's worth asking vendors this question specifically rather than depending on past history because; a) the answer tends to change over time and b) their behavior is often modified by customer requests. > Well, it sounds like my mind is made up, but I am really just starting > to investigate. Are these two things mutually exclusive? :-) Tom _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From yoon at bh.kyungpook.ac.kr Tue Aug 29 20:31:10 2000 From: yoon at bh.kyungpook.ac.kr (Yoon Jae Ho) Date: Wed, 30 Aug 2000 09:31:10 +0900 Subject: ITBL Project from Japan ? Message-ID: <001601c01219$c356c560$5f72f2cb@TEST> I read the news about Japan's ITBL Project. In the news, Japan's 62 supercomputers which is located in the 31 research Institute in the Japan will be linked with 2.4Gflops fast network. The Super-supercomptuers will be used for Genome research . Do you have any information about that Projects or Urls ? ----------------------------------------------------------------------------------------------- Yoon Jae Ho Economist POSCO Research Institute yoon at bh.kyungpook.ac.kr jhyoon at mail.posri.re.kr http://ie.korea.ac.kr/~supercom/ Korea Beowulf Imagination is more important than knowledge. A. Einstein "????? '???? ????? ??'??? ? ? ???, ??? ??? ????? ??? ????." ?? ??? "??? ?? ??? ??? ??? ??? ??? ??" ??? 2000.4.22 "???? ???? ?? ??? ??? ??? ????" ? ?? 2000.4.29 "???? ??? ??? ??? ??? ????" ? ?? 2000.4.24 -------------- next part -------------- An HTML attachment was scrubbed... URL: From becker at scyld.com Tue Aug 29 22:44:37 2000 From: becker at scyld.com (Donald Becker) Date: Tue, 29 Aug 2000 22:44:37 -0400 (EDT) Subject: USB floppy? In-Reply-To: Message-ID: On Mon, 28 Aug 2000, Laurent Itti wrote: > has anybody experience with installing Linux off a USB floppy drive? Yup. > I did > it on my slim laptop (no integrated floppy; just an external USB one) and Vaio? > it worked great as long as only one disk was needed (i.e., would not > recognize the drive after the first floppy had been loaded). And that one disk has to have an integrated ramdisk, so that the BIOS reads the whole disk. A boot disk that only loads the kernel and then expects to mount the floppy will not work without some major effort. > since modern motherboards all have USB, maybe we could forget about floppy > drives in clusters and just have one external USB drive around for > emergencies? > I wonder whether that was a special feature of my laptop's BIOS of if it > will also work with desktop motherboards? You identified the problem: the BIOS only supports USB keyboards and mice, and emulates PS/2 devices using hardware traps and "SMM" behind the OSes back. The BIOS cannot do the same emulation for standard floppy behavior. No chipset has the hardware support for trapping floppy controller accesses in the same way as keyboard controller accesses. Donald Becker becker at scyld.com Scyld Computing Corporation http://www.scyld.com 410 Severn Ave. Suite 210 Beowulf-II Cluster Distribution Annapolis MD 21403 _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From camm at enhanced.com Wed Aug 30 10:42:56 2000 From: camm at enhanced.com (Camm Maguire) Date: 30 Aug 2000 10:42:56 -0400 Subject: LAM: SCALAPACK users? In-Reply-To: "Horatio B. Bogbindero"'s message of "Tue, 29 Aug 2000 08:26:44 +0800 (PHT)" References: Message-ID: <544s42n72n.fsf@intech9.enhanced.com> Greetings! The following sample program (matrix definition ommitted), which implements an inverse iteration method to generate a particular eigenvector of a matrix, should clarify things, I hope. Please write back if I can be of more assistance! (P.S. One other thing I forgot to mention is that all fortran routines assume passed matrices are *column-major*. The code below should clarify.) Take care, ============================================================================= main.c: ============================================================================= #include #include #include #include #define TINY 1.0e-10 [snip].... static void mpiend(void) { int i; i=0; blacs_exit__(&i); } int main(int argc,char * argv[]) { int ic,nr=1,nc=1,mr,mc,N,nb=64,rsrc=0,mxlda,info,i,j,k,np,*iwork; int desca[9],descaf[9],descb[9],descv[9],*ipiv,ione=1,lr,lc; double **a,**af,**b,**v,*work,*x,*r,*c,rcond,ferr,berr,*q; double xi,pp,pa,ps,pc,bx,t,l,l1; /* struct timeval tv,tv1; */ int ch,debug=0,liwork=-1,lwork=-1,izero=0,itwo=2; char *f,eq; MPI_Init(&argc,&argv); MPI_Comm_size(MPI_COMM_WORLD,&np); if (atexit(mpiend)) error("Can't setup mpiabort on exit\n"); nc=nr=(int)sqrt(np); sl_init__(&ic,&nr,&nc); blacs_gridinfo__(&ic,&nr,&nc,&mr,&mc); /* Cblacs_gridinfo(ic,nr,nc,&mr,&mc); */ i=0; if (mr<0) blacs_exit__(&i); pc=1.0; while ((ch=getopt(argc,argv,"x:i:p:a:s:c:n:d:"))!=EOF) switch(ch) { case 'x': sscanf(optarg,"%lf",&bx); break; case 'i': sscanf(optarg,"%lf",&xi); break; case 'p': sscanf(optarg,"%lf",&pp); break; case 'a': sscanf(optarg,"%lf",&pa); break; case 's': sscanf(optarg,"%lf",&ps); break; case 'c': sscanf(optarg,"%lf",&pc); break; case 'n': sscanf(optarg,"%d",&N); break; case 'd': sscanf(optarg,"%d",&debug); break; default: break; } mxlda=N/nc + N%nc; if (mxlda%nb) mxlda+=(nb-mxlda%nb); lr=numroc_(&N,&nb,&mr,&rsrc,&nr); lc=numroc_(&N,&nb,&mc,&rsrc,&nc); /* lc=lr=mxlda; */ mem2(a,lc,lr); mem2(af,lc,lr); mem2(b,1,lr); mem2(v,1,lr); mem(x,N); mem(ipiv,lr+nb); mem(r,lr); mem(c,lc); mem(q,mxlda*mxlda); for (i=0;i writes: > > > > > are there any scalapack users in the list? i have a few questions to ask: > > > > > > > Here! > > > > > -is there s C/C++ implementation of SCALAPACK? if not, can i call the > > > fortran SCALAPACK/PBLAS functions from C/C++? > > > > > > > No c interface, to use routines: > > > > 1) add a '_' after routine name > > 2) Pass all arguments as pointers (i.e. an arg of '1' would have to > > be passed as &i, with i having been set to 1.) > > 3) Link with g77 > > > hmmm. let me get this right. is it... > > to call BLACS: > > CALL BLACS_PINFO( IAM, NPROCS) > > to c/c++ : > > blacs_pinfo_(iam, nprocs); > > to call a scalapack routine: > > CALL PDGETRF( N( I ), N( I ), MEM( IPA ), 1, 1, DESCA, MEM( IPPIV ), INFO) > > to c/c++: > > pdgetrf_(n,n,mem1,1,1,desca,mem2,info); > > ??? > > > > -are there any good scalapack documentation/manuals out there? the > > > scalapack site only feature some lawns but nothing like a users manual. > > > > > > > The faq and user's guide at netlib are both good. They're included in > > the new Debian scalapack-doc package. > > > how do i use the scalapack docs if they are a debian package and i use > redhat? is there a good printable version available like a pdf/ps version > of the scalapack-slug document at netlib. > > --------------------- > william.s.yu at ieee.org > > It's easier to fight for one's principles than to live up to them. > > > > -- Camm Maguire camm at enhanced.com ========================================================================== "The earth is but one country, and mankind its citizens." -- Baha'u'llah _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From vbeddo at ucla.edu Wed Aug 30 17:06:31 2000 From: vbeddo at ucla.edu (Vanessa Beddo) Date: Wed, 30 Aug 2000 14:06:31 -0700 Subject: MPICH-1.2.0: p4_error: Could not allocate memory for commandline argv: 251658240 Message-ID: <001301c012c6$2d037600$0600000a@atronx> Hi, I am having problems getting my Powerpc's and Intel X86 boxes to work together using MPICH-1.2.0. I have been able to succesfully compile and use MPICH on both the PowerPC and X86 machines separately. However when I try to run a program on both architectures at the same time I'm getting error messages. My configuration is as follows: 4 Pentium machines (P3-700, P166, P100, P75) running Redhat Linux 6.2 4 PowerPC's (6500/250, 8550/200, 6400/180, 7600/120) running Suse Linux 6.4 (powerpc edition) I have properly configured the hosts files, nfs, pam etc. When I use the following procgroup file which uses only one of the Intels and PowerPC's powerpc 0 /home/arno/CPI intel 1 /home/arno/CPI so I ran: root at dns: /home/arno # mpirun -p4pg procgroup -v CPI -running /home/arno/CPI on 1 LINUX ch_p4 processors -p1_1283: p4_error: Could not allocate memory for commandline argv: 251658240 -p0_3902: (1.661768) Trying to receive a message when there are no connections; Bailing out -rm_l_1_1284: p4_error: interrupt SIGINT: 2 -P4 procgroup file is procgroup. If I do this with this PROCGROUP file: intel 0 /home/arno/CPI powerpc 1 /home/arno/CPI root at gateway: /home/arno # mpirun -p4pg procgroup -v CPI -running /home/arno/CPI on 1 LINUX ch_p4 processors -p1_4089: p4_error: Could not allocate memory for commandline args: 251658240 -p0_9700: (2.792165) Trying to receive a message when there are no connections; Bailing out -rm_l_1_4090: p4_error: interrupt SIGINT: 2 -P4 procgroup file is procgroup. Please help, Arno Ouwehand -------------- next part -------------- An HTML attachment was scrubbed... URL: From yoon at bh.kyungpook.ac.kr Wed Aug 30 22:40:54 2000 From: yoon at bh.kyungpook.ac.kr (Yoon Jae Ho) Date: Thu, 31 Aug 2000 11:40:54 +0900 Subject: LAM: SCALAPACK users? References: <544s42n72n.fsf@intech9.enhanced.com> Message-ID: <009801c012f4$e8e64860$5f72f2cb@TEST> please refer http://www.netlib.org/scalapack/slug/node26.html In those example1 , please do it like this using mpich ( not LAM ) g77 -O -o example1 example1.f /root/SCALAPACK/scalapack_LINUX.a /root/SCALAPACK/pblas_LINUX.a /root/SCALAPACK/tools_LINUX.a /root/BLACS/LIB/blacsF77init_MPI-LINUX-0.a /usr/local/blaslapack/blas.a /usr/local/mpich/build/LINUX/ch_p4/lib/libmpich.a -lm -lnsl please refer the example1 from the above site . from Yoon Jae Ho http://ie.korea.ac.kr/~supercom/ Korea Beowulf jhyoon at mail.posri.re.kr yoon at bh.kyungpook.ac.kr ----- Original Message ----- From: Camm Maguire To: Cc: Sent: Wednesday, August 30, 2000 11:42 PM Subject: Re: LAM: SCALAPACK users? > Greetings! The following sample program (matrix definition ommitted), > which implements an inverse iteration method to generate a particular > eigenvector of a matrix, should clarify things, I hope. Please write > back if I can be of more assistance! > > (P.S. One other thing I forgot to mention is that all fortran routines > assume passed matrices are *column-major*. The code below should > clarify.) > > Take care, > > ============================================================================= > main.c: > ============================================================================= > #include > #include > #include > #include > > #define TINY 1.0e-10 > > [snip].... > > > static void > mpiend(void) { > > int i; > > i=0; > blacs_exit__(&i); > > > } > > int > main(int argc,char * argv[]) { > > int ic,nr=1,nc=1,mr,mc,N,nb=64,rsrc=0,mxlda,info,i,j,k,np,*iwork; > int desca[9],descaf[9],descb[9],descv[9],*ipiv,ione=1,lr,lc; > double **a,**af,**b,**v,*work,*x,*r,*c,rcond,ferr,berr,*q; > double xi,pp,pa,ps,pc,bx,t,l,l1; > /* struct timeval tv,tv1; */ > int ch,debug=0,liwork=-1,lwork=-1,izero=0,itwo=2; > char *f,eq; > > > > MPI_Init(&argc,&argv); > MPI_Comm_size(MPI_COMM_WORLD,&np); > if (atexit(mpiend)) > error("Can't setup mpiabort on exit\n"); > > nc=nr=(int)sqrt(np); > > sl_init__(&ic,&nr,&nc); > blacs_gridinfo__(&ic,&nr,&nc,&mr,&mc); > /* Cblacs_gridinfo(ic,nr,nc,&mr,&mc); */ > > i=0; > if (mr<0) > blacs_exit__(&i); > > pc=1.0; > > while ((ch=getopt(argc,argv,"x:i:p:a:s:c:n:d:"))!=EOF) > switch(ch) { > case 'x': > sscanf(optarg,"%lf",&bx); > break; > case 'i': > sscanf(optarg,"%lf",&xi); > break; > case 'p': > sscanf(optarg,"%lf",&pp); > break; > case 'a': > sscanf(optarg,"%lf",&pa); > break; > case 's': > sscanf(optarg,"%lf",&ps); > break; > case 'c': > sscanf(optarg,"%lf",&pc); > break; > case 'n': > sscanf(optarg,"%d",&N); > break; > case 'd': > sscanf(optarg,"%d",&debug); > break; > default: > break; > } > > mxlda=N/nc + N%nc; > if (mxlda%nb) mxlda+=(nb-mxlda%nb); > > lr=numroc_(&N,&nb,&mr,&rsrc,&nr); > lc=numroc_(&N,&nb,&mc,&rsrc,&nc); > /* lc=lr=mxlda; */ > mem2(a,lc,lr); > mem2(af,lc,lr); > mem2(b,1,lr); > mem2(v,1,lr); > mem(x,N); > mem(ipiv,lr+nb); > mem(r,lr); > mem(c,lc); > mem(q,mxlda*mxlda); > > for (i=0;i x[i]=bx+xi*i; > > l=1.0; > for (i=0;i for (j=0;j int ni,nj; > > ni=i; > ni/=nb; > if (ni%nr!=mr) > continue; > ni/=nr; > ni*=nb; > ni+=i%nb; > > nj=j; > nj/=nb; > if (nj%nc!=mc) > continue; > nj/=nc; > nj*=nb; > nj+=j%nb; > > a[nj][ni]=kernel(x[i],x[j],0.0,pc*pp,pa/pc,ps/pc)*xi; > if (i==j) > a[nj][ni]-=l; > > } > > descinit_(desca,&N,&N,&nb,&nb,&rsrc,&rsrc,&ic,&lr,&info); > descinit_(descaf,&N,&N,&nb,&nb,&rsrc,&rsrc,&ic,&lr,&info); > descinit_(descv,&N,&ione,&nb,&nb,&rsrc,&rsrc,&ic,&lr,&info); > descinit_(descb,&N,&ione,&nb,&nb,&rsrc,&rsrc,&ic,&lr,&info); > > if (!mc) { > randnor(b[0],lr,1.0); > > t=0.0; > for (i=0;i t+=b[0][i]*b[0][i]; > dgsum2d_(&ic,"C"," ",&ione,&ione,&t,&ione,&izero,&mc); > dgebr2d_(&ic,"C"," ",&ione,&ione,&t,&ione,&izero,&mc); > > t=1.0/sqrt(t); > for (i=0;i b[0][i]*=t; > } > > > if (!mr && !mc) { > fprintf(stderr,"Starting ev search\n"); > fflush(stderr); > } > > f="N"; > k=-1; > pdgesvx_(f,"N",&N,&ione,a[0],&ione,&ione,desca, > af[0],&ione,&ione,descaf,ipiv,&eq, > r,c,b[0],&ione,&ione,descb,v[0],&ione,&ione,descv, > &rcond,&ferr,&berr,&t,&k,&j,&k,&info); > lwork=t; > mem(work,lwork); > liwork=j; > mem(iwork,liwork); > pdgesvx_(f,"N",&N,&ione,a[0],&ione,&ione,desca, > af[0],&ione,&ione,descaf,ipiv,&eq, > r,c,b[0],&ione,&ione,descb,v[0],&ione,&ione,descv, > &rcond,&ferr,&berr,work,&lwork,iwork,&liwork,&info); > if (debug && !mr && !mc) > fprintf(stderr,"Factor done: info %d rc %e fe %e be %e eq %c\n", > info,rcond,ferr,berr,eq); > if (info) > if (!mr && !mc) > error("Factorization failed: info %d rc %e fe %e be %e eq %c\n", > info,rcond,ferr,berr,eq); > else > return -1; > > f="F"; > > for (i=0;i<200 && (!debug || i > double t,t1[2]; > int j; > > if (!mc) { > > for (j=0,t1[0]=t1[1]=0.0;j t1[0]+=v[0][j]*v[0][j]; > t1[1]+=v[0][j]; > } > dgsum2d_(&ic,"C"," ",&itwo,&ione,t1,&itwo,&izero,&mc); > dgebr2d_(&ic,"C"," ",&itwo,&ione,t1,&itwo,&izero,&mc); > > t=1.0/sqrt(t1[0]); > t=t1[1]<0.0 ? -t : t; > > for (j=0;j v[0][j]*=t; > > for (t=0.0,j=0;j t+=(b[0][j]-v[0][j])*(b[0][j]-v[0][j]); > > dgsum2d_(&ic,"C"," ",&ione,&ione,&t,&ione,&izero,&mc); > dgebr2d_(&ic,"C"," ",&ione,&ione,&t,&ione,&izero,&mc); > > t=sqrt(t); > } > if (!mc) > memcpy(b[0],v[0],lr*sizeof(*b[0])); > > if (debug && !mc) { > > if (!mr) > fprintf(stderr,"Result %d: diff %e\n",i,t); > > for (j=0;j if (j==mr) > memcpy(q+j*mxlda,b[0],lr*sizeof(*b[0])); > dgebr2d_(&ic,"C"," ",&mxlda,&ione,q+j*mxlda,&N,&j,&mc); > } > > if (!mr) { > for (j=0;j int nj,nbb; > > nj=j; > nj/=nb; > nbb=nj%nr; > nj/=nr; > nj*=nb; > nj+=j%nb; > printf("%e %e\n",x[j],q[nbb*mxlda+nj]); > } > printf("\n\n"); > } > } > > pdgesvx_(f,"N",&N,&ione,a[0],&ione,&ione,desca, > af[0],&ione,&ione,descaf,ipiv,&eq, > r,c,b[0],&ione,&ione,descb,v[0],&ione,&ione,descv, > &rcond,&ferr,&berr,work,&lwork,iwork,&liwork,&info); > if (debug && !mr && !mc) > fprintf(stderr,"Iteration %d: info %d rc %e fe %e be%e eq %c\n", > i,info,rcond,ferr,berr,eq); > > dgebr2d_(&ic,"R"," ",&ione,&ione,&t,&ione,&mr,&izero); > > if (t break; > > } > > if (!mc) { > > for (t=0.0,j=0;j t+=b[0][j]*v[0][j]; > > dgsum2d_(&ic,"C"," ",&ione,&ione,&t,&ione,&izero,&mc); > dgebr2d_(&ic,"C"," ",&ione,&ione,&t,&ione,&izero,&mc); > > l1=l+1.0/t; > > if (!mr) { > fprintf(stderr,"done, %d iterations, eigenv %f\n",i,l1); > fflush(stderr); > } > > for (j=0;j if (j==mr) > memcpy(q+j*mxlda,b[0],lr*sizeof(*b[0])); > dgebr2d_(&ic,"C"," ",&mxlda,&ione,q+j*mxlda,&N,&j,&mc); > } > > if (!mr) > for (i=0;i int ni,nbb; > > ni=i; > ni/=nb; > nbb=ni%nr; > ni/=nr; > ni*=nb; > ni+=i%nb; > printf("%e %e\n",x[i],q[nbb*mxlda+ni]); > } > } > > blacs_gridexit__(&ic); > > exit(0); > > } > ============================================================================= > Makefile: > ============================================================================= > SC=-lscalapack-lam -lpblas-lam -ltools-lam -lredist-lam > BL=-lblacsCinit-lam -lblacs-lam -lblacsCinit-lam > BLAS=-lblas > > LOADLIBES = $(SC) $(BL) $(BLAS) -lstd -lnum -lm -lmpi > LINK=g77 > CFLAGS+=-I /usr/include/mpi > > include $(HOME)/etc/Makefile > > ============================================================================= > > "Horatio B. Bogbindero" writes: > > > > > > > > are there any scalapack users in the list? i have a few questions to ask: > > > > > > > > > > Here! > > > > > > > -is there s C/C++ implementation of SCALAPACK? if not, can i call the > > > > fortran SCALAPACK/PBLAS functions from C/C++? > > > > > > > > > > No c interface, to use routines: > > > > > > 1) add a '_' after routine name > > > 2) Pass all arguments as pointers (i.e. an arg of '1' would have to > > > be passed as &i, with i having been set to 1.) > > > 3) Link with g77 > > > > > hmmm. let me get this right. is it... > > > > to call BLACS: > > > > CALL BLACS_PINFO( IAM, NPROCS) > > > > to c/c++ : > > > > blacs_pinfo_(iam, nprocs); > > > > to call a scalapack routine: > > > > CALL PDGETRF( N( I ), N( I ), MEM( IPA ), 1, 1, DESCA, MEM( IPPIV ), INFO) > > > > to c/c++: > > > > pdgetrf_(n,n,mem1,1,1,desca,mem2,info); > > > > ??? > > > > > > -are there any good scalapack documentation/manuals out there? the > > > > scalapack site only feature some lawns but nothing like a users manual. > > > > > > > > > > The faq and user's guide at netlib are both good. They're included in > > > the new Debian scalapack-doc package. > > > > > how do i use the scalapack docs if they are a debian package and i use > > redhat? is there a good printable version available like a pdf/ps version > > of the scalapack-slug document at netlib. > > > > --------------------- > > william.s.yu at ieee.org > > > > It's easier to fight for one's principles than to live up to them. > > > > > > > > > > -- > Camm Maguire camm at enhanced.com > ========================================================================== > "The earth is but one country, and mankind its citizens." -- Baha'u'llah > > _______________________________________________ > Beowulf mailing list > Beowulf at beowulf.org > http://www.beowulf.org/mailman/listinfo/beowulf _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From nilesham38 at usa.net Thu Aug 31 00:05:37 2000 From: nilesham38 at usa.net (nilesh modi) Date: 30 Aug 00 22:05:37 MDT Subject: what is mpich? Message-ID: <20000831040538.25317.qmail@www0a.netaddress.usa.net> i am working on a project in which we have used lam for mpi configuration , i would like to know what is mpich ? and does it differ from lam-mpi . and how? can we use directly mpich in our project ? thanx, nilesh. ____________________________________________________________________ Get free email and a permanent address at http://www.netaddress.com/?N=1 _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From tony at MPI-Softtech.Com Thu Aug 31 08:16:54 2000 From: tony at MPI-Softtech.Com (Tony Skjellum) Date: Thu, 31 Aug 2000 07:16:54 -0500 (CDT) Subject: what is mpich? In-Reply-To: <20000831040538.25317.qmail@www0a.netaddress.usa.net> Message-ID: MPICH is the Argonne/Mississippi State model implementation of MPI. http://www-unix.mcs.anl.gov/mpi/mpich/ ... Anthony Skjellum, PhD, President (tony at mpi-softtech.com) MPI Software Technology, Inc., Ste. 33, 101 S. Lafayette, Starkville, MS 39759 +1-(662)320-4300 x15; FAX: +1-(662)320-4301; http://www.mpi-softtech.com "Best-of-breed Software for Beowulf and Easy-to-Own Commercial Clusters." On 30 Aug 2000, nilesh modi wrote: > i am working on a project in which we have used lam for mpi configuration , > i would like to know what is mpich ? and does it differ from lam-mpi . > and how? > can we use directly mpich in our project ? > > > thanx, > nilesh. > > ____________________________________________________________________ > Get free email and a permanent address at http://www.netaddress.com/?N=1 > > _______________________________________________ > Beowulf mailing list > Beowulf at beowulf.org > http://www.beowulf.org/mailman/listinfo/beowulf > _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From bockmabe at plu.edu Thu Aug 31 14:38:20 2000 From: bockmabe at plu.edu (Bryce Bockman) Date: Thu, 31 Aug 2000 11:38:20 -0700 Subject: A kernel level message passing service Message-ID: <39AEA61C.75E2F885@plu.edu> I have recently become aware of work being done to incorporate certain types of services (ex: web servers) into the kernel of unix systems, so as to take away a level of abstraction, and eliminate the overhead in performing system calls from user space, thus creating a more efficient server. My question is this: Does is make sense that this sort of thing could be applied to a message passing interfaces for any sizable gain in performance? If so, is there any work already being done on this topic? My initial reaction to the thought is that there would be all kinds of security issues, but maybe some sort of "safe" API could be developed to address these issues. Cheers, Bryce Bockman _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From tony at MPI-Softtech.Com Thu Aug 31 14:43:15 2000 From: tony at MPI-Softtech.Com (Tony Skjellum) Date: Thu, 31 Aug 2000 13:43:15 -0500 (CDT) Subject: A kernel level message passing service In-Reply-To: <39AEA61C.75E2F885@plu.edu> Message-ID: Bryce, see www.viarch.org ... the experience here is that good VI Arch implementations appear to do what you are asking about (low overhead, high bandwidth, low-medium latency), for clusters of small to medium size. Other related pages (they don't have much public yet) would be www.rapidio.org www.infinibandta.org -Tony Anthony Skjellum, PhD, President (tony at mpi-softtech.com) MPI Software Technology, Inc., Ste. 33, 101 S. Lafayette, Starkville, MS 39759 +1-(662)320-4300 x15; FAX: +1-(662)320-4301; http://www.mpi-softtech.com "Best-of-breed Software for Beowulf and Easy-to-Own Commercial Clusters." On Thu, 31 Aug 2000, Bryce Bockman wrote: > I have recently become aware of work being done to incorporate certain > types of services (ex: web servers) into the kernel of unix systems, so > as to take away a level of abstraction, and eliminate the overhead in > performing system calls from user space, thus creating a more efficient > server. > > My question is this: > > Does is make sense that this sort of thing could be applied to a message > passing interfaces for any sizable gain in performance? If so, is there > any work already being done on this topic? > > My initial reaction to the thought is that there would be all kinds of > security issues, but maybe some sort of "safe" API could be developed to > address these issues. > > Cheers, > Bryce Bockman > > _______________________________________________ > Beowulf mailing list > Beowulf at beowulf.org > http://www.beowulf.org/mailman/listinfo/beowulf > _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From pgeoffra at cs.utk.edu Thu Aug 31 15:45:54 2000 From: pgeoffra at cs.utk.edu (Patrick Geoffray) Date: Thu, 31 Aug 2000 15:45:54 -0400 (EDT) Subject: A kernel level message passing service In-Reply-To: Message-ID: On Thu, 31 Aug 2000, Bryce Bockman wrote: > > Does is make sense that this sort of thing could be applied to a message > > passing interfaces for any sizable gain in performance? If so, is there > > any work already being done on this topic? > My initial reaction to the thought is that there would be all kinds of > security issues, but maybe some sort of "safe" API could be developed to > address these issues. On Thu, 31 Aug 2000, Tony Skjellum wrote: > Bryce, see www.viarch.org ... the experience here is that good VI Arch > implementations appear to do what you are asking about (low overhead, Hi Bryce, Actually, almost all of the driver for high performance interconnects provides OS-bypass functionnalities. It's true for Myrinet, SCI, Quadrics, VIA is finally one more flavour. But you are right : it's a tradeoff between security and performance. Usually the systeme calls overhead for security is very small and out of the critical path. The kernel is the only way to protect against other users in a multi-users environnment. Patrick Geoffray --- Aerospatiale MATRA - Sycomore Universite Lyon I - RESAM http://lhpca.univ-lyon1.fr _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From troy at osc.edu Thu Aug 31 16:30:22 2000 From: troy at osc.edu (Troy Baer) Date: Thu, 31 Aug 2000 16:30:22 -0400 Subject: lperfex 1.0 released Message-ID: Hi all, I'm happy to announce that lperfex 1.0 is now available. Lperfex is a hardware performance monitoring tool for Linux/IA32 systems, using the interface provided by Erik Hendriks' performance patch and its descendent in the PAPI 1.1 beta release. If you've used Cray's hpm or SGI's perfex, then lperfex should seem fairly familiar. If not, think of lperfex as a variation on the time command which can also track low-level hardware events like floating point operations, cache misses, and so on. It is not intrusive into the code whose performance it measures and does not require special compilation or code instrumentation. The 1.0 release changes how wallclock and CPU time used by the measured process are recovered from the get_rusage() and times() syscalls, now using a method that is (hopefully) more accurate. Several new performance metrics have also been added, and the online documentation has been improved to give examples of which event or events to specify on the command line to measure a given metric. Documentation and code for lperfex is available from http://www.osc.edu/~troy/lperfex/. The code is licensed under the GPL. To use it, you need to be running Linux with Erik Hendriks' performance counters patch and library v0.7 (available from http://www.beowulf.org/software/) on an Intel P6 core processor. The libperf library and Linux kernel patches that are distributed with PAPI 1.1 beta (http://icl.cs.utk.edu/projects/papi/) are also be compatible and provide a fairly nice low-level instrumentation library as well. Lperfex 1.0 will likely be the last version to rely directly on the perf patch from the Beowulf.org site. Future versions will probably be based on the PAPI low-level interface, as this should make lperfex portable to a number of non-Linux/x86 platforms. --Troy -- Troy Baer email: troy at osc.edu Science & Technology Support phone: 614-292-9701 Ohio Supercomputer Center web: http://oscinfo.osc.edu _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf From dmetz at imagelinks.com Wed Aug 23 13:11:20 2000 From: dmetz at imagelinks.com (Derald Metzger) Date: Wed, 23 Aug 2000 13:11:20 -0400 Subject: OpenSSH problems Message-ID: <200008231711.NAA20016@dante-e0.imagelinks.com> > > I am having trouble setting up OpenSSH to replace rsh. I would like = > eliminate the use of passwords for users going between machines on a = > guarded beowulf. What I *think* I want is to use RSA + Rhosts = > authentication. The docs say that all I need is a /etc/hosts.equiv and = > /etc/ssh_known_hosts. However, this doesn't seem to work. Is there = > anyone who could give me some pointers or better yet forward a copy of = > the config files and whatever other files I need? (feel free to munge = > the keys... :) >... > Traveler Hauptman Look for these lines in your sshd_config: # Don't read ~/.rhosts and ~/.shosts files IgnoreRhosts no # For this to work you will also need host keys in /etc/ssh_known_hosts RhostsRSAAuthentication yes Insure that the pub keys for each host are in ssh_known_hosts. Insure that the users' pub keys are in their ~/.ssh/authorized_keys The above will probably get things working tho there are other gotchas, eg protocol 1 rather than 2 (2 uses different files). Life is fairly easy for the users if their home dir is nfs mounted so they only have to deal with one authorized_keys file. The sysadmin needs to put each hosts pub keys in ssh_known_hosts. Derald _______________________________________________ Beowulf mailing list Beowulf at beowulf.org http://www.beowulf.org/mailman/listinfo/beowulf