From heiko.bauke at snafu.de  Tue Mar  1 08:30:22 2011
From: heiko.bauke at snafu.de (Heiko Bauke)
Date: Tue, 1 Mar 2011 14:30:22 +0100
Subject: [Beowulf] Random numbers (was Re: /dev/random entropy on
 stateless/headless nodes)
In-Reply-To: <alpine.LFD.2.02.1102281346210.3235@lilith>
References: <alpine.BSF.2.00.1102251708280.98544@freeman.4gh.net>
	<Pine.LNX.4.64.1102261715520.7392@coffee.psychology.mcmaster.ca>
	<AANLkTinrDKWde+2VYAYu0XdE9CTU=H-NcO=AippwC+_p@mail.gmail.com>
	<alpine.LFD.2.02.1102271131580.4648@lilith>
	<20110228104103.6e195d2b@keitel105.dhcp.mpi-hd.mpg.de>
	<alpine.LFD.2.02.1102281346210.3235@lilith>
Message-ID: <20110301143022.3a0d6e5d@keitel105.dhcp.mpi-hd.mpg.de>

Hi,

On Mon, 28 Feb 2011 15:03:33 -0500 (EST)
"Robert G. Brown" <rgb at phy.duke.edu> wrote:

> It would also be very interesting to try feeding one of your MRG or
> YARN-type generators into dieharder and running the full series of
> tests on it (there are actually several diehard tests in dieharder
> that look for hyperplanar correlations as well as the long tail
> "monkey" tests). If I ever have time I'll try to give it a shot.  Is
> the library and/or a cli available anyplace easy to find?

there is an open source C++ library implementing our YARN generators and
others, see http://trng.berlios.de/ and http://trng.berlios.de/trng.pdf .


	Heiko


-- 
-- Irrend lernt man.
-- (Johann Wolfgang von Goethe, 1749-1832)
-- Number Crunch Blog @ http://numbercrunch.de
--        Heiko Bauke @ http://www.mpi-hd.mpg.de/personalhomes/bauke
_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.


From rgb at phy.duke.edu  Tue Mar  1 10:25:30 2011
From: rgb at phy.duke.edu (Robert G. Brown)
Date: Tue, 1 Mar 2011 10:25:30 -0500 (EST)
Subject: [Beowulf] Random numbers (was Re: /dev/random entropy on
 stateless/headless nodes)
In-Reply-To: <20110301143022.3a0d6e5d@keitel105.dhcp.mpi-hd.mpg.de>
References: <alpine.BSF.2.00.1102251708280.98544@freeman.4gh.net>
	<Pine.LNX.4.64.1102261715520.7392@coffee.psychology.mcmaster.ca>
	<AANLkTinrDKWde+2VYAYu0XdE9CTU=H-NcO=AippwC+_p@mail.gmail.com>
	<alpine.LFD.2.02.1102271131580.4648@lilith>
	<20110228104103.6e195d2b@keitel105.dhcp.mpi-hd.mpg.de>
	<alpine.LFD.2.02.1102281346210.3235@lilith>
	<20110301143022.3a0d6e5d@keitel105.dhcp.mpi-hd.mpg.de>
Message-ID: <alpine.LFD.2.02.1103011020150.4012@lilith>

On Tue, 1 Mar 2011, Heiko Bauke wrote:

> Hi,
>
> On Mon, 28 Feb 2011 15:03:33 -0500 (EST)
> "Robert G. Brown" <rgb at phy.duke.edu> wrote:
>
>> It would also be very interesting to try feeding one of your MRG or
>> YARN-type generators into dieharder and running the full series of
>> tests on it (there are actually several diehard tests in dieharder
>> that look for hyperplanar correlations as well as the long tail
>> "monkey" tests). If I ever have time I'll try to give it a shot.  Is
>> the library and/or a cli available anyplace easy to find?
>
> there is an open source C++ library implementing our YARN generators and
> others, see http://trng.berlios.de/ and http://trng.berlios.de/trng.pdf .

Thanks, I grabbed it.  I don't usually use C++, but hopefully I can link
this back to a C interface or at worst write a simple CLI program to
spit out a binary stream.  dieharder can test a binary stream via a pipe
(it is set up to test "large" numbers of random numbers, much larger
than diehard, although practically speaking many of the tests are
asymtotic and fail if you try to test the 10^12 and up numbers of rands
used in many modern MC computations.  But it at least makes it easy (and
still fairly reliable) to test 10^9 or more.  (I'd love to make it work
for longer streams, but that proves to be remarkably difficult for
numerical reasons, for at least the most "interesting" tests...:-).

Might take a while to get around to this, but if/when I do I'll try to
let you know.

    rgb

>
>
> 	Heiko
>
>
> -- 
> -- Irrend lernt man.
> -- (Johann Wolfgang von Goethe, 1749-1832)
> -- Number Crunch Blog @ http://numbercrunch.de
> --        Heiko Bauke @ http://www.mpi-hd.mpg.de/personalhomes/bauke
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
>

Robert G. Brown	                       http://www.phy.duke.edu/~rgb/
Duke University Dept. of Physics, Box 90305
Durham, N.C. 27708-0305
Phone: 1-919-660-2567  Fax: 919-660-2525     email:rgb at phy.duke.edu


_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.


From john.hearns at mclaren.com  Tue Mar  1 10:50:41 2011
From: john.hearns at mclaren.com (Hearns, John)
Date: Tue, 1 Mar 2011 15:50:41 -0000
Subject: [Beowulf] Why do we need SANs
Message-ID: <68A57CCFD4005646957BD2D18E60667B1292EC74@milexchmb1.mil.tagmclarengroup.com>

This is a thought provoking article on storage:

http://www.theregister.co.uk/2011/03/01/why_do_we_need_sans/


In the light of the recent thread on Thunderbolt, note:
"Infiniband or ..... extermalised PCIe bus"
Do we see the HPC system of the future being a huge blade server farm,
running lots of virtual machines,
with some sort of object storage system spread across the machine
running on selected blades?




John Hearns | CFD Hardware Specialist | McLaren Racing Limited
McLaren Technology Centre, Chertsey Road, Woking, Surrey GU21 4YH, UK

T:  +44 (0) 1483 261000
D:  +44 (0) 1483 262352
F:  +44 (0) 1483 261010
E:  john.hearns at mclaren.com
W:  www.mclaren.com




The contents of this email are confidential and for the exclusive use of the intended recipient.  If you receive this email in error you should not copy it, retransmit it, use it or disclose its contents but should return it to the sender immediately and delete your copy.
_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.


From john.hearns at mclaren.com  Wed Mar  2 04:44:43 2011
From: john.hearns at mclaren.com (Hearns, John)
Date: Wed, 2 Mar 2011 09:44:43 -0000
Subject: [Beowulf] Random numbers (was Re: /dev/random entropy on
 stateless/headless nodes)
In-Reply-To: <alpine.LFD.2.02.1103011020150.4012@lilith>
References: <alpine.BSF.2.00.1102251708280.98544@freeman.4gh.net><Pine.LNX.4.64.1102261715520.7392@coffee.psychology.mcmaster.ca><AANLkTinrDKWde+2VYAYu0XdE9CTU=H-NcO=AippwC+_p@mail.gmail.com><alpine.LFD.2.02.1102271131580.4648@lilith><20110228104103.6e195d2b@keitel105.dhcp.mpi-hd.mpg.de><alpine.LFD.2.02.1102281346210.3235@lilith><20110301143022.3a0d6e5d@keitel105.dhcp.mpi-hd.mpg.de>
	<alpine.LFD.2.02.1103011020150.4012@lilith>
Message-ID: <68A57CCFD4005646957BD2D18E60667B1292ECDB@milexchmb1.mil.tagmclarengroup.com>

Talking about seeding random numbers from dates and network interfaces,
I happen to be looking at the Intel igb driver for a completely separate
reason today.

One of the parameters in the igb kernel module is:


parm:           entropy:Allow igb to populate the /dev/random entropy
pool (int)


I guess this isn't news to most people.
John Hearns

The contents of this email are confidential and for the exclusive use of the intended recipient.  If you receive this email in error you should not copy it, retransmit it, use it or disclose its contents but should return it to the sender immediately and delete your copy.
_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.


From hahn at mcmaster.ca  Fri Mar  4 00:15:16 2011
From: hahn at mcmaster.ca (Mark Hahn)
Date: Fri, 4 Mar 2011 00:15:16 -0500 (EST)
Subject: [Beowulf] Why do we need SANs
In-Reply-To: <68A57CCFD4005646957BD2D18E60667B1292EC74@milexchmb1.mil.tagmclarengroup.com>
References: <68A57CCFD4005646957BD2D18E60667B1292EC74@milexchmb1.mil.tagmclarengroup.com>
Message-ID: <Pine.LNX.4.64.1103032343240.24334@coffee.psychology.mcmaster.ca>

> This is a thought provoking article on storage:
>
> http://www.theregister.co.uk/2011/03/01/why_do_we_need_sans/

I didn't think it was particularly good: basically just the observation
that you can put lots of servers instances into a big box and do away 
with a lot of infrastructure.  call it "mainframe".

the reason this is not good and maybe even bad is that it pretends 
that there's no need for scalable systems.  and it seems to be based 
on the premise that infrastructure such as FC, which has always been
very complex and expensive, would somehow be necessary.

besides, who would even propose SANs these days?  they're inherently
hostile to sharing, and it's all about sharing today...

> In the light of the recent thread on Thunderbolt, note:
> "Infiniband or ..... extermalised PCIe bus"

as far as I can tell, thunderbolt doesn't introduce anything interesting
regarding addressing, switching fabrics or low-latency transactions between
hosts.  I can't tell whether Intel wants to push it in that direction,
or whether it's a short-term sop for Apple.

> Do we see the HPC system of the future being a huge blade server farm,
> running lots of virtual machines,

hell no.  first of all, blades are done.  they never made much sense,
and what little sense they had (higher efficiency PSUs and cooling)
has now become available in standard/commodity parts.

but more importantly, why the heck would we want VMs for HPC?  virtualization
is great for packing lots of different, low-duty-cycle server instances 
onto less hardware.  and it provides some help in shuffling them around.
HPC is high duty-cycle, and doesn't the inherent inefficiency of virt.
being able to move processes around would he helpful for HPC, but it doesn't
need to be whole OS instances.

> with some sort of object storage system spread across the machine
> running on selected blades?

SANs simply lack the expressive power necessary to support modern 
file/object/database/web sharing protocols.  and the SAN world, which 
considers 8Gb FC to be sexy, has been left in the dust by 40 Gb IB 
or even ethernet.  (if IB had built on ethernet protocols rather than 
badly reinventing the wheel, no one would even remember FC today...)

regards, mark hahn.
_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.


From hahn at mcmaster.ca  Tue Mar  8 00:26:26 2011
From: hahn at mcmaster.ca (Mark Hahn)
Date: Tue, 8 Mar 2011 00:26:26 -0500 (EST)
Subject: [Beowulf] Execution time measurements (fwd)
In-Reply-To: <E1PvvZz-0003ng-00.mikky_m-mail-ru@f121.mail.ru>
References: <68A57CCFD4005646957BD2D18E60667B1292EC74@milexchmb1.mil.tagmclarengroup.com>
	<Pine.LNX.4.64.1103032343240.24334@coffee.psychology.mcmaster.ca>
	<E1PvvZz-0003ng-00.mikky_m-mail-ru@f121.mail.ru>
Message-ID: <Pine.LNX.4.64.1103072315300.8438@coffee.psychology.mcmaster.ca>

Hi list,
I'm forwarding a message (took the liberty of inserting some of my own
comments...)

Mikhail Kuzminsky writes:
> I still don't have the possibility to send the messages directly to Beowulf
>maillist (although I receive all the mail messages). Could you pls forward
>my question to beowulf at beowulf.org ? 
>
> Thanks for your help !
> Mikhail
>
> Execution time measurements
>
> I  obtained strange results for Gaussian-09 (G09) execution times in
>sequential (!) run on dual socket Opteron-2350 based node under Open SuSE
>10.3 (kernel 2.6.22). The measurements were performed for  "pure cpu-bound"
>job (frequencies calculation at DFT level) where start-stop execution time
>is practically equal to cpu execution time (difference is lower than 1 min
>per 1 day of execution). G09 itself prints both cpu execution time, and
>start & stop dates/time information. 
>
> There is some job which execution time is about 1 day. But really it was
>measured two *DIFFERENT* execution times: 1st - 1 day (24h) and 2nd - 1 day
>3 hours (27h). Both results were reproduced few times and gives the same
>quantum-chemical results excluding execution time.  There was no other
>applications run simultaneously w/this measurements. Execution time
>difference isn't localized in any of G09 parts (links).

OK, if I understand: same tests, different timings.

> "Fast" execution  was reproduced 2 times: one - as usual sequential run and
>on - in simultaneous execution of 2 examples of this job (therefore there is
>no mutual influence of this jobs). These runs was not managed  manually in
>the sense of numa allocation. 
>
> "Slow" execution was reproduced minimum 5 times. The memory required for
>job execution is 4+ GB, and it's possible to allocate the necessary RAM from
>one node only (we have 8 GB per node, 16 GB per server). I forced (using
>numactl) to use both cpu (core) and memory from node 1, but it gives "slow"
>execution time. When I forced cpu 1 and memory 0, execution time was
>increased to 1h (up to 28h).

if I understand, the tests runs slower if the process is remote from memory.
that's not at all surprising, right?

> (BTW, g09 links are threads. Does numactl forcing for main module "extends"
>to child threads also ?)

numactl uses sched_setaffinity/etal to set process properties that _are_
inherited.  so unless g09 manipulates the numa settings, the "outer" numactl
settings will control threads and links as well.

> Then I checked G09 own timings via execution under time command. G09 and
>time results was the same, but I looked only "slow" execution time.

I haven't seen inaccuracies in g09's time-reporting.

> The frequency of cpus was fixed (there was no cpufreq kernel module loaded).
>
> I'll be very appreciate in any ideas about possible reason of two different
>execution time observations. 
>
>Mikhail Kuzminsky
>Computer Assistance to Chemical Research Center
>Zelinsky Institute of Organic Chemistry RAS
>Moscow Russia

I think you already have a grip on the answer: any program runs fastest 
when it's running with memory "local" to the CPU it's using.  those pages,
after all, are effectively slower.  if the app doesn't control this (or you
with numactl), then you should expect performance to lie somewhere between
the two extremes (fully local vs fully remote).  the kernel does make some
effort at keeping things local - and for that matter, avoiding moving a 
process among multiple cores/sockets.

how much this matters depends on the app.  anything cache-friendly won't
care - GUIs and a lot of servers would prefer to run sooner, rather than 
insisting on a particular cpu or bank of memory...

I'll confess something here: my organization hasn't bothered with any 
special affinity settings until recently.  I'm not sure how often HPC centers
do worry about this.  obviously, such settings complicate the scheduler's 
job and probably require more tuning by the user...

regards, mark hahn.


_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.


From deadline at eadline.org  Wed Mar  9 10:48:08 2011
From: deadline at eadline.org (Douglas Eadline)
Date: Wed, 9 Mar 2011 10:48:08 -0500 (EST)
Subject: [Beowulf] =?iso-8859-1?q?Chinese_supercomputers_to_use_=91homemad?=
	=?iso-8859-1?q?e=92_chips?=
In-Reply-To: <Pine.LNX.4.64.1103072315300.8438@coffee.psychology.mcmaster.ca>
References: <68A57CCFD4005646957BD2D18E60667B1292EC74@milexchmb1.mil.tagmclarengroup.com>
	<Pine.LNX.4.64.1103032343240.24334@coffee.psychology.mcmaster.ca>
	<E1PvvZz-0003ng-00.mikky_m-mail-ru@f121.mail.ru>
	<Pine.LNX.4.64.1103072315300.8438@coffee.psychology.mcmaster.ca>
Message-ID: <46317.192.168.93.213.1299685688.squirrel@mail.eadline.org>


Interesting:

Chinese supercomputers to use ?homemade? chips


http://timesofindia.indiatimes.com/tech/personal-tech/computing/Chinese-supercomputers-to-use-homemade-chips/articleshow/7655183.cms


--
Doug

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.

_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf


From vanallsburg at hope.edu  Thu Mar 10 11:49:51 2011
From: vanallsburg at hope.edu (Paul Van Allsburg)
Date: Thu, 10 Mar 2011 11:49:51 -0500
Subject: [Beowulf] ramdisk with tmpfs
Message-ID: <4D79012F.90709@hope.edu>

Hi All,
I want to make a ramdisk available to cluster users and I'm curious what  your experiences/suggestions might be.  I'm adding 40 
machines to a cluster that have 24 gig of ram.  I'd like to offer the option of allowing some users to be able to run a job on a 
machine with 20 gig carved out for a ram disk.

The cluster is running centos 5.5 with torque & maui.

I expect the user will have to request one machine for the job and have the prologue/eplogue scripts mount & unmount the 
ramdisk. Are there any success / horror stories that I might be enlightened by?

Thanks!
Paul

-- 
Paul Van Allsburg
Scientific Computing Specialist
Natural Sciences Division, Hope College
35 E. 12th St.  Holland, Michigan 49423
616-395-7292    vanallsburg at hope.edu
http://www.hope.edu/academic/csm/


_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.


From Craig.Tierney at noaa.gov  Thu Mar 10 12:16:51 2011
From: Craig.Tierney at noaa.gov (Craig Tierney)
Date: Thu, 10 Mar 2011 10:16:51 -0700
Subject: [Beowulf] ramdisk with tmpfs
In-Reply-To: <4D79012F.90709@hope.edu>
References: <4D79012F.90709@hope.edu>
Message-ID: <4D790783.4090002@noaa.gov>

On 3/10/11 9:49 AM, Paul Van Allsburg wrote:
> Hi All,
> I want to make a ramdisk available to cluster users and I'm curious what  your experiences/suggestions might be.  I'm adding 40
> machines to a cluster that have 24 gig of ram.  I'd like to offer the option of allowing some users to be able to run a job on a
> machine with 20 gig carved out for a ram disk.
>
> The cluster is running centos 5.5 with torque&  maui.
>
> I expect the user will have to request one machine for the job and have the prologue/eplogue scripts mount&  unmount the
> ramdisk. Are there any success / horror stories that I might be enlightened by?
>
> Thanks!
> Paul
>

As far as I recall, Centos creates a ramdisk by default at /dev/shm whose
maximum size is 1/2 of available memory.  The ramdisk uses available memory
as needed, and doesn't block an application from allocating
all of memory (as long as there is nothing in the ramdisk).

So you don't have to create one.  You can leave it bigger if you want,
but what is important is that the prologue/epilogue clears out the files
left there before the next run.  You could just mount/umount one as well
if you want, as you can have multiple ramdisks mounted at one time which
all size dynamically.

Craig
_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.


From shaeffer at neuralscape.com  Thu Mar 10 18:23:31 2011
From: shaeffer at neuralscape.com (Karen Shaeffer)
Date: Thu, 10 Mar 2011 15:23:31 -0800
Subject: [Beowulf] IBM's Watson on Jeopardy tonight
In-Reply-To: <alpine.LFD.2.02.1102161240390.3531@lilith>
References: <E1PpSsG-0006IP-Er@mendel.bio.caltech.edu>
	<4D5B8733.1010505@gmail.com>
	<50888AA2846A4ECF9332AFB80EFB87D5@USERPC>
	<0627DE5B-7CAD-4A72-8E8F-8D60167FC43B@jax.org>
	<ECE7A93BD093E1439C20020FBE87C47FEDE0A471A6@ALTPHYEMBEVSP20.RES.AD.JPL>
	<4D5BDF3B.6010104@pathscale.com>
	<alpine.LFD.2.02.1102161000050.3627@lilith>
	<4D5BEF7E.8060004@pathscale.com>
	<alpine.LFD.2.02.1102161240390.3531@lilith>
Message-ID: <20110310232331.GB5729@synapse.neuralscape.com>

On Wed, Feb 16, 2011 at 01:06:35PM -0500, Robert G. Brown wrote:
> On Wed, 16 Feb 2011, "C. Bergstr?m" wrote:
> 
> >>We are, but that problem is, well, "hard".  As in grand challenge hard.
> >I wonder how you'd really describe the human brain learning in terms of a 
> >programming model....
> 
> Well, AI researchers tend to have two answers.  One is semantic and the
> other is microscopic.  The semantic description is
> functional/operational, broken down in terms of e.g. conditionals and
> logical elements, and doesn't come close to explaining consciousness
> (see Searle's "Chinese Room" objection):
> 

Hi,
Well, IBM is presenting "Inside the mind of Watson" at Stanford University
on March 14. The speaker is Chris Welty of IBM. There are actually several
presentations. Here is one:

http://bmir.stanford.edu/events/view.php/inside_the_mind_of_watson

There is another one at 12:15 PM earlier that day associated with the
symbolic systems group. Those are often open to the public, if you
ask for permission to attend.

Karen
-- 
 Karen Shaeffer
 Neuralscape, Palo Alto, Ca. 94306
 shaeffer at neuralscape.com  http://www.neuralscape.com
_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.



From hahn at mcmaster.ca  Fri Mar 11 01:20:11 2011
From: hahn at mcmaster.ca (Mark Hahn)
Date: Fri, 11 Mar 2011 01:20:11 -0500 (EST)
Subject: [Beowulf]
 =?iso-8859-1?q?Chinese_supercomputers_to_use_=91homemad?=
 =?iso-8859-1?q?e=92_chips?=
In-Reply-To: <46317.192.168.93.213.1299685688.squirrel@mail.eadline.org>
References: <68A57CCFD4005646957BD2D18E60667B1292EC74@milexchmb1.mil.tagmclarengroup.com>
	<Pine.LNX.4.64.1103032343240.24334@coffee.psychology.mcmaster.ca>
	<E1PvvZz-0003ng-00.mikky_m-mail-ru@f121.mail.ru>
	<Pine.LNX.4.64.1103072315300.8438@coffee.psychology.mcmaster.ca>
	<46317.192.168.93.213.1299685688.squirrel@mail.eadline.org>
Message-ID: <Pine.LNX.4.64.1103110108590.19223@coffee.psychology.mcmaster.ca>

> Interesting:
> Chinese supercomputers to use ?homemade? chips
> http://timesofindia.indiatimes.com/tech/personal-tech/computing/Chinese-supercomputers-to-use-homemade-chips/articleshow/7655183.cms

it's important to remind ourselves that China is still a centrally-planned,
totalitarian dictatorship.  I mention this only because this announcement
is a bit like Putin et al announcing that they'll develop their own 
linux distro because Russia is big and important and mustn't allow 
itself to be vulnerable to foreign hegemony.

so far, the very shallow reporting I've seen has said that future 
generations will add wide FP vector units.  nothing wrong with that,
though it's a bit unclear to me why other companies haven't done it
if there is, in fact, lots of important vector codes that will run 
efficiently on such a configuration.  adding/widening vector FP is 
not breakthrough engineering afaikt.

has anyone heard anything juicy about the Tianhe interconnect?
-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.

-------------- next part --------------
_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

From eagles051387 at gmail.com  Fri Mar 11 03:47:10 2011
From: eagles051387 at gmail.com (Jonathan Aquilina)
Date: Fri, 11 Mar 2011 09:47:10 +0100
Subject: [Beowulf]
	=?windows-1252?q?Chinese_supercomputers_to_use_=91homem?=
	=?windows-1252?q?ade=92_chips?=
In-Reply-To: <Pine.LNX.4.64.1103110108590.19223@coffee.psychology.mcmaster.ca>
References: <68A57CCFD4005646957BD2D18E60667B1292EC74@milexchmb1.mil.tagmclarengroup.com>
	<Pine.LNX.4.64.1103032343240.24334@coffee.psychology.mcmaster.ca>
	<E1PvvZz-0003ng-00.mikky_m-mail-ru@f121.mail.ru>
	<Pine.LNX.4.64.1103072315300.8438@coffee.psychology.mcmaster.ca>
	<46317.192.168.93.213.1299685688.squirrel@mail.eadline.org>
	<Pine.LNX.4.64.1103110108590.19223@coffee.psychology.mcmaster.ca>
Message-ID: <AANLkTike3Nf9tf_p4q_q+Bc7FsudyenDUJVdYQWRoy3L@mail.gmail.com>

some more details about the chips.

http://www.physorg.com/news/2011-03-dawning-china-homemade-supercomputer.html

On Fri, Mar 11, 2011 at 7:20 AM, Mark Hahn <hahn at mcmaster.ca> wrote:

>  Interesting:
>> Chinese supercomputers to use ?homemade? chips
>>
>> http://timesofindia.indiatimes.com/tech/personal-tech/computing/Chinese-supercomputers-to-use-homemade-chips/articleshow/7655183.cms
>>
>
> it's important to remind ourselves that China is still a centrally-planned,
> totalitarian dictatorship.  I mention this only because this announcement
> is a bit like Putin et al announcing that they'll develop their own linux
> distro because Russia is big and important and mustn't allow itself to be
> vulnerable to foreign hegemony.
>
> so far, the very shallow reporting I've seen has said that future
> generations will add wide FP vector units.  nothing wrong with that,
> though it's a bit unclear to me why other companies haven't done it
> if there is, in fact, lots of important vector codes that will run
> efficiently on such a configuration.  adding/widening vector FP is not
> breakthrough engineering afaikt.
>
> has anyone heard anything juicy about the Tianhe interconnect?
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit
> http://www.beowulf.org/mailman/listinfo/beowulf
>
>


-- 
Jonathan Aquilina

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.clustermonkey.net/pipermail/beowulf/attachments/20110311/968a5bd4/attachment.html>
-------------- next part --------------
_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

From prentice at ias.edu  Fri Mar 11 09:27:04 2011
From: prentice at ias.edu (Prentice Bisbal)
Date: Fri, 11 Mar 2011 09:27:04 -0500
Subject: [Beowulf] IBM's Watson on Jeopardy tonight
In-Reply-To: <20110310232331.GB5729@synapse.neuralscape.com>
References: <E1PpSsG-0006IP-Er@mendel.bio.caltech.edu>	<4D5B8733.1010505@gmail.com>	<50888AA2846A4ECF9332AFB80EFB87D5@USERPC>	<0627DE5B-7CAD-4A72-8E8F-8D60167FC43B@jax.org>	<ECE7A93BD093E1439C20020FBE87C47FEDE0A471A6@ALTPHYEMBEVSP20.RES.AD.JPL>	<4D5BDF3B.6010104@pathscale.com>	<alpine.LFD.2.02.1102161000050.3627@lilith>	<4D5BEF7E.8060004@pathscale.com>	<alpine.LFD.2.02.1102161240390.3531@lilith>
	<20110310232331.GB5729@synapse.neuralscape.com>
Message-ID: <4D7A3138.9070304@ias.edu>


On 03/10/2011 06:23 PM, Karen Shaeffer wrote:
> On Wed, Feb 16, 2011 at 01:06:35PM -0500, Robert G. Brown wrote:
>> On Wed, 16 Feb 2011, "C. Bergstr?m" wrote:
>>
>>>> We are, but that problem is, well, "hard".  As in grand challenge hard.
>>> I wonder how you'd really describe the human brain learning in terms of a 
>>> programming model....
>>
>> Well, AI researchers tend to have two answers.  One is semantic and the
>> other is microscopic.  The semantic description is
>> functional/operational, broken down in terms of e.g. conditionals and
>> logical elements, and doesn't come close to explaining consciousness
>> (see Searle's "Chinese Room" objection):
>>
> 
> Hi,
> Well, IBM is presenting "Inside the mind of Watson" at Stanford University
> on March 14. The speaker is Chris Welty of IBM. There are actually several
> presentations. Here is one:
> 
> http://bmir.stanford.edu/events/view.php/inside_the_mind_of_watson
> 
> There is another one at 12:15 PM earlier that day associated with the
> symbolic systems group. Those are often open to the public, if you
> ask for permission to attend.
> 
> Karen

This reminded me: On Feb 9, 2011,  PBS aired a Nova episode about the
development of Watson. The entire episode is available online:

http://www.pbs.org/wgbh/nova/tech/smartest-machine-on-earth.html

There's other interesting links and videos available on that page, too.


-- 
Prentice
_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.



From hahn at mcmaster.ca  Sat Mar 12 14:48:36 2011
From: hahn at mcmaster.ca (Mark Hahn)
Date: Sat, 12 Mar 2011 14:48:36 -0500 (EST)
Subject: [Beowulf]
 =?windows-1252?q?Chinese_supercomputers_to_use_=91homem?=
 =?windows-1252?q?ade=92_chips?=
In-Reply-To: <AANLkTike3Nf9tf_p4q_q+Bc7FsudyenDUJVdYQWRoy3L@mail.gmail.com>
References: <68A57CCFD4005646957BD2D18E60667B1292EC74@milexchmb1.mil.tagmclarengroup.com>
	<Pine.LNX.4.64.1103032343240.24334@coffee.psychology.mcmaster.ca>
	<E1PvvZz-0003ng-00.mikky_m-mail-ru@f121.mail.ru>
	<Pine.LNX.4.64.1103072315300.8438@coffee.psychology.mcmaster.ca>
	<46317.192.168.93.213.1299685688.squirrel@mail.eadline.org>
	<Pine.LNX.4.64.1103110108590.19223@coffee.psychology.mcmaster.ca>
	<AANLkTike3Nf9tf_p4q_q+Bc7FsudyenDUJVdYQWRoy3L@mail.gmail.com>
Message-ID: <Pine.LNX.4.64.1103110948200.2209@coffee.psychology.mcmaster.ca>

> some more details about the chips.
>
> http://www.physorg.com/news/2011-03-dawning-china-homemade-supercomputer.html

seems like mostly an nth-gen rehash of the info here:
http://www.theregister.co.uk/2011/02/25/ict_godson_3b_chip/

which discusses the provenance of the basic architecture.
40W gets you 8 cores at 1 GHz.  each core appears to have 2x 256b FP SIMD,
but no mention is made of their rate.  if the rate were 1/cycle, that would
be 32 gflops per unit.  the article mentions "performance on those vector
units is 128 Gflops, so I'm not quite sure how the math goes.  later in 
the article it says 16x 8c chips go into a single 1U and yield 2 Tflops,
therefore 16 gflops per core.  so maybe it can start a new instr every other 
cycle.  in any case, 128 Gflops for 40W.

a similar comparison with a mainstream cluster chip might be closer to 1
gflop/watt (at the socket level, ignoring support chips, fans, PSU efficiency.)

current-gen Nvidia and ATI chips are in the 2-3 gflops/watt range,
though that's not a completely fair comparison.

regards, mark hahn.
_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.


From eugen at leitl.org  Sat Mar 12 14:59:50 2011
From: eugen at leitl.org (Eugen Leitl)
Date: Sat, 12 Mar 2011 20:59:50 +0100
Subject: [Beowulf]
	=?utf-8?q?Chinese_supercomputers_to_use_=E2=80=98homema?=
	=?utf-8?b?ZGXigJkgY2hpcHM=?=
In-Reply-To: <Pine.LNX.4.64.1103110948200.2209@coffee.psychology.mcmaster.ca>
References: <68A57CCFD4005646957BD2D18E60667B1292EC74@milexchmb1.mil.tagmclarengroup.com>
	<Pine.LNX.4.64.1103032343240.24334@coffee.psychology.mcmaster.ca>
	<E1PvvZz-0003ng-00.mikky_m-mail-ru@f121.mail.ru>
	<Pine.LNX.4.64.1103072315300.8438@coffee.psychology.mcmaster.ca>
	<46317.192.168.93.213.1299685688.squirrel@mail.eadline.org>
	<Pine.LNX.4.64.1103110108590.19223@coffee.psychology.mcmaster.ca>
	<AANLkTike3Nf9tf_p4q_q+Bc7FsudyenDUJVdYQWRoy3L@mail.gmail.com>
	<Pine.LNX.4.64.1103110948200.2209@coffee.psychology.mcmaster.ca>
Message-ID: <20110312195950.GZ23560@leitl.org>

On Sat, Mar 12, 2011 at 02:48:36PM -0500, Mark Hahn wrote:

> current-gen Nvidia and ATI chips are in the 2-3 gflops/watt range,
> though that's not a completely fair comparison.

Another vaporware shop is Calxeda
http://www.heise.de/newsticker/meldung/480-Core-Server-mit-ARM-Prozessoren-1206857.html
</kraut>

480 core/2U, 5 W/SoC node, 10x of x86 performance/Watt,
15-20x performance/USD.

-- 
Eugen* Leitl <a href="http://leitl.org">leitl</a> http://leitl.org
______________________________________________________________
ICBM: 48.07100, 11.36820 http://www.ativel.com http://postbiota.org
8B29F6BE: 099D 78BA 2FD3 B014 B08A  7779 75B0 2443 8B29 F6BE
_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.


From orion at cora.nwra.com  Thu Mar 17 13:42:14 2011
From: orion at cora.nwra.com (Orion Poplawski)
Date: Thu, 17 Mar 2011 11:42:14 -0600
Subject: [Beowulf] ramdisk with tmpfs
In-Reply-To: <4D790783.4090002@noaa.gov>
References: <4D79012F.90709@hope.edu> <4D790783.4090002@noaa.gov>
Message-ID: <4D8247F6.3090608@cora.nwra.com>

On 03/10/2011 10:16 AM, Craig Tierney wrote:
> On 3/10/11 9:49 AM, Paul Van Allsburg wrote:
>> Hi All,
>> I want to make a ramdisk available to cluster users and I'm curious what  your experiences/suggestions might be.  I'm adding 40
>> machines to a cluster that have 24 gig of ram.  I'd like to offer the option of allowing some users to be able to run a job on a
>> machine with 20 gig carved out for a ram disk.
>>
>> The cluster is running centos 5.5 with torque&   maui.
>>
>> I expect the user will have to request one machine for the job and have the prologue/eplogue scripts mount&   unmount the
>> ramdisk. Are there any success / horror stories that I might be enlightened by?
>>
>> Thanks!
>> Paul
>>
>
> As far as I recall, Centos creates a ramdisk by default at /dev/shm whose
> maximum size is 1/2 of available memory.  The ramdisk uses available memory
> as needed, and doesn't block an application from allocating
> all of memory (as long as there is nothing in the ramdisk).

No, don't use /dev/shm - this is a API mount for posix shared memory.  Mount 
another tmpfs somehwere else (we mount one at /tmp).  tmpfs is very nice, just 
remember that is competes with the RAM demands of your processes.

-- 
Orion Poplawski
Technical Manager                     303-415-9701 x222
NWRA/CoRA Division                    FAX: 303-415-9702
3380 Mitchell Lane                  orion at cora.nwra.com
Boulder, CO 80301              http://www.cora.nwra.com
_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.


From hahn at mcmaster.ca  Thu Mar 17 16:50:58 2011
From: hahn at mcmaster.ca (Mark Hahn)
Date: Thu, 17 Mar 2011 16:50:58 -0400 (EDT)
Subject: [Beowulf] ramdisk with tmpfs
In-Reply-To: <4D8247F6.3090608@cora.nwra.com>
References: <4D79012F.90709@hope.edu> <4D790783.4090002@noaa.gov>
	<4D8247F6.3090608@cora.nwra.com>
Message-ID: <Pine.LNX.4.64.1103171626270.7368@coffee.psychology.mcmaster.ca>

> No, don't use /dev/shm - this is a API mount for posix shared memory.  Mount
> another tmpfs somehwere else (we mount one at /tmp).  tmpfs is very nice, just
> remember that is competes with the RAM demands of your processes.

ramdisks are icky and tmfs is indeed much nicer.  tmpfs competes for ram,
but can be swapped, so is relatively polite.  tmpfs also has options
for limiting the size and number of inodes (set these for safety.)  and since
it's basically allocating ram for you, it also has NUMA parameters - 
see Documentation/filesystems/tmpfs.txt for details.  (I wonder whether 
allocating files mmaped from numa-tweaked tmpfs would provide the kind of
persistent numa binding to pages that some openmpi people want...)
_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.


From deadline at eadline.org  Mon Mar 21 08:51:06 2011
From: deadline at eadline.org (Douglas Eadline)
Date: Mon, 21 Mar 2011 08:51:06 -0400 (EDT)
Subject: [Beowulf] GP-GPU experience
In-Reply-To: <46317.192.168.93.213.1299685688.squirrel@mail.eadline.org>
References: <68A57CCFD4005646957BD2D18E60667B1292EC74@milexchmb1.mil.tagmclarengroup.com>
	<Pine.LNX.4.64.1103032343240.24334@coffee.psychology.mcmaster.ca>
	<E1PvvZz-0003ng-00.mikky_m-mail-ru@f121.mail.ru>
	<Pine.LNX.4.64.1103072315300.8438@coffee.psychology.mcmaster.ca>
	<46317.192.168.93.213.1299685688.squirrel@mail.eadline.org>
Message-ID: <45254.192.168.93.213.1300711866.squirrel@mail.eadline.org>


I was recently given a copy of "GPU Computing Gems"
to review. It is basically research quality NVidia success
stories, some of which are quite impressive.

I got to thinking about how others are fairing (or not)
with GP-GPU technology. I put up a simple poll on
ClusterMonkey to help get a general idea.
(you can find it on the front page right top)
If you have a moment, please provide
your experience (results are available as well).

  http://www.clustermonkey.net/

BTW: You can see all the previous polls
and links to other market data  here:

http://goo.gl/lDcUJ


--
Doug

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.

_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf


From eugen at leitl.org  Mon Mar 21 09:11:27 2011
From: eugen at leitl.org (Eugen Leitl)
Date: Mon, 21 Mar 2011 14:11:27 +0100
Subject: [Beowulf] GP-GPU experience
In-Reply-To: <45254.192.168.93.213.1300711866.squirrel@mail.eadline.org>
References: <68A57CCFD4005646957BD2D18E60667B1292EC74@milexchmb1.mil.tagmclarengroup.com>
	<Pine.LNX.4.64.1103032343240.24334@coffee.psychology.mcmaster.ca>
	<E1PvvZz-0003ng-00.mikky_m-mail-ru@f121.mail.ru>
	<Pine.LNX.4.64.1103072315300.8438@coffee.psychology.mcmaster.ca>
	<46317.192.168.93.213.1299685688.squirrel@mail.eadline.org>
	<45254.192.168.93.213.1300711866.squirrel@mail.eadline.org>
Message-ID: <20110321131127.GB23560@leitl.org>

On Mon, Mar 21, 2011 at 08:51:06AM -0400, Douglas Eadline wrote:
> 
> I was recently given a copy of "GPU Computing Gems"
> to review. It is basically research quality NVidia success
> stories, some of which are quite impressive.

This one?

http://www.amazon.com/GPU-Computing-Gems-Emerald-Applications/dp/0123849888

the reviewers didn't seem to think much of it.
 
> I got to thinking about how others are fairing (or not)
> with GP-GPU technology. I put up a simple poll on
> ClusterMonkey to help get a general idea.
> (you can find it on the front page right top)
> If you have a moment, please provide
> your experience (results are available as well).
> 
>   http://www.clustermonkey.net/
> 
> BTW: You can see all the previous polls
> and links to other market data  here:
> 
> http://goo.gl/lDcUJ

-- 
Eugen* Leitl <a href="http://leitl.org">leitl</a> http://leitl.org
______________________________________________________________
ICBM: 48.07100, 11.36820 http://www.ativel.com http://postbiota.org
8B29F6BE: 099D 78BA 2FD3 B014 B08A  7779 75B0 2443 8B29 F6BE
_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.


From john.hearns at mclaren.com  Mon Mar 21 11:33:40 2011
From: john.hearns at mclaren.com (Hearns, John)
Date: Mon, 21 Mar 2011 15:33:40 -0000
Subject: [Beowulf] Cray to build huge,
	grunting 20-petaflop 'Titan' for US gov labs
Message-ID: <207BB2F60743C34496BE41039233A80903C2B096@MRL-PWEXCHMB02.mil.tagmclarengroup.com>

http://www.theregister.co.uk/2011/03/21/ornl_cray_titan_super/


Looking at the final slide, guess we'd better start investing in
circular machine rooms.
(Actually..... I already work in a circular building.... Hmmm.....)

John Hearns | CFD Hardware Specialist | McLaren Racing Limited
McLaren Technology Centre, Chertsey Road, Woking, Surrey GU21 4YH, UK

T:  +44 (0) 1483 261000
D:  +44 (0) 1483 262352
F:  +44 (0) 1483 261010
E:  john.hearns at mclaren.com
W:  www.mclaren.com




The contents of this email are confidential and for the exclusive use of the intended recipient.  If you receive this email in error you should not copy it, retransmit it, use it or disclose its contents but should return it to the sender immediately and delete your copy.
_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.


From asabigue at fing.edu.uy  Mon Mar 21 12:12:55 2011
From: asabigue at fing.edu.uy (ariel sabiguero yawelak)
Date: Mon, 21 Mar 2011 13:12:55 -0300
Subject: [Beowulf] Cray to build huge,
 grunting 20-petaflop 'Titan' for US gov labs
In-Reply-To: <207BB2F60743C34496BE41039233A80903C2B096@MRL-PWEXCHMB02.mil.tagmclarengroup.com>
References: <207BB2F60743C34496BE41039233A80903C2B096@MRL-PWEXCHMB02.mil.tagmclarengroup.com>
Message-ID: <4D877907.7080806@fing.edu.uy>

So cool thinking about 100-250 petaflops, which is only 1 order of 
magnitude away from the first exaflop machine. It is interesting, on the 
other hand, the article recently published by IEEE Spectrum named 
"Next-Generation Supercomputers", which states that:/
/

    /"....The practical exaflops-class supercomputer DARPA was hoping
    for just wasn't going to be attainable by 2015. In fact, it might
    not be possible anytime in the foreseeable future....."
    /
    /http://spectrum.ieee.org/computing/hardware/nextgeneration-supercomputers/


There are some differences between different classes of exaflop 
supercomputers, but P. Kogge managed to convice me that "it might not be 
possible <..> in the <..> future".

Pretty interesting divergence.....

regards
ariel

El 21/03/11 12:33, Hearns, John escribi?:
> http://www.theregister.co.uk/2011/03/21/ornl_cray_titan_super/
>
>
> Looking at the final slide, guess we'd better start investing in
> circular machine rooms.
> (Actually..... I already work in a circular building.... Hmmm.....)
>
> John Hearns | CFD Hardware Specialist | McLaren Racing Limited
> McLaren Technology Centre, Chertsey Road, Woking, Surrey GU21 4YH, UK
>
> T:  +44 (0) 1483 261000
> D:  +44 (0) 1483 262352
> F:  +44 (0) 1483 261010
> E:  john.hearns at mclaren.com
> W:  www.mclaren.com
>
>
>
>
> The contents of this email are confidential and for the exclusive use of the intended recipient.  If you receive this email in error you should not copy it, retransmit it, use it or disclose its contents but should return it to the sender immediately and delete your copy.
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
>

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.clustermonkey.net/pipermail/beowulf/attachments/20110321/c9047ce4/attachment.html>
-------------- next part --------------
_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

From eugen at leitl.org  Wed Mar 23 09:25:23 2011
From: eugen at leitl.org (Eugen Leitl)
Date: Wed, 23 Mar 2011 14:25:23 +0100
Subject: [Beowulf] relativistic contraints for trading
Message-ID: <20110323132523.GG23560@leitl.org>


http://www.bbc.co.uk/news/science-environment-12827752

Stock trades to exploit speed of light, says researcher

By Jason Palmer Science and technology reporter, BBC News, Dallas

Map showing relation between exchanges and trading locations Optimal
high-frequency trading locations (blue) exist for pairs of major financial
exchanges (red)

Financial institutions may soon change what they trade or where they do their
trading because of the speed of light.

"High-frequency trading" carried out by computers often depends on differing
prices of a financial instrument in two geographically-separated markets.

Exactly how far the signals have to go can make a difference in such trades.

Alexander Wissner-Gross told the American Physical Society meeting that
financial institutions are looking at ways to exploit the light-speed trick.

Dr Wissner-Gross, of Harvard University, said that the latencies -
essentially, the time delay for a signal to wing its way from one global
financial centre to another - advantaged some locations for some trades and
different locations for others.

There is a vast market for ever-faster fibre-optic cables to try to
physically "get there faster" but Dr Wissner-Gross said that the purely
technological approach to gaining an advantage was reaching a limit.

Trades now travel at nearly 90% of the ultimate speed limit set by physics,
the speed of light in the cables.  Competitive advantage

His first solution, published in 2010, considered the various latencies in
global fibre-optic links and mapped out where the optimal points for
financial transactions to originate - midway between two major financial hubs
to maximise the chance of "buying low" in one place and "selling high" in
another.

That of course resulted in a number of ideal locations in all corners of the
globe, including the oceans. But wholesale relocation of operations does not
immediately appeal to many firms.

"I'm now working... with real companies on real deployments that don't
require you deploy a floating data centre in the middle of the ocean; we say,
'OK, you have your existing infrastructure, that's not moving - now, given
your location, which stocks in various locations are you best positioned to
trade?'"

"If you don't have the budget to put new data centres in the middle of the
ocean you can, for example, use existing data centres that are an
approximation to the optimal location in the ocean - say, Nova Scotia for New
York to London," Dr Wissner-Gross told BBC News.

Because there is a clear, physical advantage to the approach, Dr
Wissner-Gross said that the first firm to try to exploit the effect will be
at significant competitive advantage - until more firms follow suit.

That means that out-of-the-way places - at high latitudes or mid-ocean island
chains - could in time turn into global financial centres.

"It's instructive to start to think about latency correlations as a new sort
of resource," he explained.

"If you're positioned between two major financial hubs, you may be far out of
the way, rather far from population centres, maybe economically poor, but
because of your unique position, that could be a natural resource."
_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.


From eugen at leitl.org  Thu Mar 24 06:45:42 2011
From: eugen at leitl.org (Eugen Leitl)
Date: Thu, 24 Mar 2011 11:45:42 +0100
Subject: [Beowulf] Rome Lab's supercomputer is made up of 1,
	700 off-the-shelf	PlayStation 3 gaming consoles
Message-ID: <20110324104542.GE23560@leitl.org>


http://www.syracuse.com/news/index.ssf/2011/03/rome_labs_supercomputer_is_mad.html

Rome Lab's supercomputer is made up of 1,700 off-the-shelf PlayStation 3
gaming consoles

Published: Wednesday, March 23, 2011, 6:00 AM     Updated: Wednesday, March
23, 2011, 8:02 AM

Dave Tobin / The Post-Standard By Dave Tobin / The Post-Standard

Jhn Berry / The Post-Standard Morgan Bishop, a computer scientist at the Air
Force Research Lab, in Rome, is surrounded by a high-speed, image-processing
network of 1,716 PlayStation 3 systems linked to each other and other
components.

Rome, NY -- Computer scientists just up the Thruway at Rome?s Air Force
Research Lab have assembled one of the world?s largest, fastest and cheapest
supercomputers ? and it?s made from PlayStation 3s.

By linking together 1,716 PlayStation 3s, they?ve created a supercomputer
that?s very good at processing, manipulating and interpreting vast amounts of
imagery. This will provide analysts with new levels of detail from the
pictures gathered on long surveillance flights by spy planes.

The PlayStation 3 is a video gaming console that originally sold for about
$500. It was developed by Sony, released in 2006 and is known for its
sizzlingly clear video graphics.

The Air Force calls the souped-up PlayStations the Condor Supercomputer and
says it is among the 40 fastest computers in the world. The Condor went
online late last year, and it will likely change the way the Air Force and
the Air National Guard watch things on the ground.

The creation, while offbeat, illustrates the modern job for the operation
that began as Rome Air Development Center in 1951, researching radar. It has
survived the closing of Griffiss Air Force Base in 1995 to find a new niche.

These days, Rome Lab?s research focuses on information technology,
particularly cybersecurity and high-performance computing. The lab employs
789 people in military and civilian jobs, with a payroll of $82 million a
year. It oversees contracts worth nearly $3 billion.

Rome is helping the military face a growing problem: Great advances in
airborne surveillance systems have the military drowning in visual data.

Meanwhile, the Air Force and the Air National Guard, as well as federal
agencies such as U.S. Customs and Border Protection, which already gather
surveillance video from the air, are pursuing technologies to gather more.
The goal: constant surveillance over large areas.
2011-02-22-jb-romelab3.JPGView full sizeJohn Berry / The Post-Standard A
PlayStation 3 video game console, one of the 1,716 systems linked together at
Rome Lab to form a supercomputer.

The Condor helps meet that. The Air Force has begun using a new radar
technology dubbed ?Gotcha,? with far sharper resolution than previous radar.
To maximize Gotcha?s potential, the power of a computer such as the Condor is
needed.

The Condor will enable 24-hour, real-time surveillance over a roughly
15-mile-wide area, said Mark Barnell, director of high-performance computing
at the Rome research lab.

Video processed from the radar signals can be viewed in real time or played
back to investigate what led to an event ? an explosion, an uprising or an
ambush. As with a video game, a viewer can change perspectives, going from
air to ground to look around buildings.

?You can literally rewind or predict forward (in the future), based on the
information you have,? Barnell said.

Development of the Condor started nearly five years ago, shortly after Sony
put the PlayStation 3 on the market. Richard W. Linderman, then senior
scientist at Rome?s Air Force research lab, brought the new PlayStation 3
home and began experimenting. The PS3 can run Linux, a software operating
system used in most of the world?s supercomputers.

At Rome Lab, Linderman asked his research team to try linking eight
PlayStation 3s to see what they could do. Impressed, he increased the number
to 336. That worked even better. What could more than a thousand do?

Rome Lab asked the Department of Defense for $2.5 million to assemble its
supercomputer. By the time money to buy that many was approved in 2009,
PlayStation 3s were hard to find. Rome Lab bought as many as they could ?
1,700.

To custom-build a supercomputer without using commercial off-the-shelf
PlayStation 3s would likely have cost 10 times as much, Barnell said. In
addition, the Condor uses a fraction of the energy that comparably sized
supercomputers use. Portions of it ? say 300 machines ? can be turned on
while the rest are off, depending on a job?s needs.

Rome Lab plans to work with the New York Air National Guard?s 174th Fighter
Wing, Barnell said. The 174th is seeking FAA permission to fly MQ-9 Reapers
in Northern New York, starting this summer.  2011-02-22-jb-romelab1.JPGView
full sizeJohn Berry / The Post-Standard Mark Barnell, director of
high-performance computing at the Air Force Research Lab, in Rome, stands
with a data wall that displays the video output of the center's Condor
Supercomputer. This sample image was made by a radar system aboard an
airplane.

The Air Force is also using the Condor to process ground-based radar images
of space objects, again with extraordinary clarity. Barnell shows images of a
space shuttle orbiting Earth at 5 miles a second. Without Condor processing,
the shuttle image is a blurry black triangle. With Condor processing, it is
sharp and distinct. It?s clear that its payload doors are open.

?This is important because other countries are pursuing space missions, and
we don?t always know their intent,? Barnell said.

The Condor?s third major use is for computational intelligence ? a form of
computer reasoning and decision-making. One example is with words: Condor can
scan or process text in any language at 20 pages a second, fill in missing
sections it has never seen with 99.9 percent accuracy and tell the user
whether the information is important.

?Jobs that used to take hours or days now take seconds,? Barnell said.

Barnell cautions that the Condor is not the ?Holy Grail of computing.?

Rome Lab is sharing Condor access with researchers at other government
agencies, colleges and universities. Among them are Cornell University,
Dartmouth College and the universities of Florida, Maryland and Tennessee.
Researchers at Syracuse University and State University Institute of
Technology at Utica/Rome are also expected to have access, Barnell said.

As impressive as the Condor is, it won?t be for long. Barnell envisions
integrating smartphone processors into high-performance computing, putting
the power of a Condor into a small surveillance drone the size of your fist,
something weighing less than a pound and using the energy of a standard light
bulb.

?In a couple of years, this will fade away,? he said.

Contact Dave Tobin at dtobin at syracuse.com or 470-3277.

_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.



From trainor at presciencetrust.org  Sun Mar 27 21:48:45 2011
From: trainor at presciencetrust.org (Douglas J. Trainor)
Date: Sun, 27 Mar 2011 21:48:45 -0400
Subject: [Beowulf] Quanta crams 512 cores into pizza box server
Message-ID: <8A93BA65-6FBF-4020-9E79-E9D61D16CFB6@presciencetrust.org>

http://www.theregister.co.uk/2011/03/28/quanta_tilera_server_ships/print.html
_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.


From landman at scalableinformatics.com  Mon Mar 28 10:18:26 2011
From: landman at scalableinformatics.com (Joe Landman)
Date: Mon, 28 Mar 2011 10:18:26 -0400
Subject: [Beowulf] Virtualized Infiniband in KVM?
Message-ID: <4D9098B2.2080801@scalableinformatics.com>

Hi folks

   I know about the VT-d bits for virtualized 1 1GbE/10GbE and their 
ability to be leveraged for KVM stacks.  Is there a similar capability 
for IB?  I figured I'd ask here first before bugging the OFED lists. 
This is an HPC situation, and I am trying to see if there is any 
experience with IB in a KVM virtual machine.  Any latency/bandwidth 
measures/tests, or any data for that matter would be of interest.

   I know Xen had some capability with this for a while (virualized IB 
stack as a passthru).  I'd like to avoid Xen if possible, and focus upon 
KVM for a number of reasons.

   Thanks!

   Joe

-- 
Joseph Landman, Ph.D
Founder and CEO
Scalable Informatics Inc.
email: landman at scalableinformatics.com
web  : http://scalableinformatics.com
        http://scalableinformatics.com/sicluster
phone: +1 734 786 8423 x121
fax  : +1 866 888 3112
cell : +1 734 612 4615
_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.


From raq at cttc.upc.edu  Tue Mar 29 04:37:27 2011
From: raq at cttc.upc.edu (Ramiro Alba)
Date: Tue, 29 Mar 2011 10:37:27 +0200
Subject: [Beowulf] AMD 8 cores vs 12 cores CPUs and Infiniband
Message-ID: <1301387847.1995.144.camel@mundo>

Hi all,

We currently have an IB DDR cluster of 128 AMD QuadCore 'Barcelona' (4 *
2 cores) using DDR 'Flextronics' switches ( http://www.ibswitches.com)
with an over-subscription 2:1 (two 24 port switches/rack a one 144 port
modular 'Flextronics' Fat-Tree switch to link). Every IB card in nodes,
is Mellanox DDR InfiniHost III Lx PCI Express x8 (one port) and we use
IB both for calculations (MPI) and storage (Lustre: 80 TB using 1 MDS
and 2 OSSs on a DDN 9900 unit)

Now, we are planning to add new AMD 'Magny-Cours' nodes (16 or 24 cores)
using Infiniband QDR, but linking with the Flextronics 144 port DDR
switch, using 'Hybrid Pasive Copper QSFP to MicroGiGaCN cables', so as
we can reach the Lustre storage.

But there are two main issues that we are worried about:

1 - One port IB cards QP saturation

Using 16 cores per node (8 * 2) seem the 'safe' option, but the 24 cores
(12 * 2) is better in term of price per job. Our CFD applications using
MPI (OpenMPI) may need to do about 15 'MPI_allreduce' calls in one
seccond or less, and we may probably using a pool of 1500 cores. ??Is
anyone having this kind of 'message rate', using AMD 24 cores, and can
tell me about his/her solution/experience?

2 - I've heard that QLogic behavior is better in terms of QP creation, I
have also to think on linking IB DDR with QDR to reach the 'Lustre'
storage. I suppose the main issue is, which QDR switch or switches are
linking with the Flextronics 144 port DDR switch, but I do not know what
is the role of the node card (Mellanox/Qlogic). Again, can anyone tell
me about his/her solution/experience?

Any comment suggestion will be welcomed

Thanks in advance

Regards
-- 
Ramiro Alba

Centre Tecnol??gic de Tranfer??ncia de Calor
http://www.cttc.upc.edu


Escola T??cnica Superior d'Enginyeries
Industrial i Aeron??utica de Terrassa
Colom 11, E-08222, Terrassa, Barcelona, Spain
Tel: (+34) 93 739 86 46



-- 
Aquest missatge ha estat analitzat per MailScanner
a la cerca de virus i d'altres continguts perillosos,
i es considera que est? net.


-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.

-------------- next part --------------
_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

From Shainer at Mellanox.com  Tue Mar 29 18:24:19 2011
From: Shainer at Mellanox.com (Gilad Shainer)
Date: Tue, 29 Mar 2011 15:24:19 -0700
Subject: [Beowulf] AMD 8 cores vs 12 cores CPUs and Infiniband
References: <1301387847.1995.144.camel@mundo>
Message-ID: <9FA59C95FFCBB34EA5E42C1A8573784F037FD082@mtiexch01.mti.com>

I have been using single card on Magny-Cours with no issues at all. You can definitely go with the QDR adapters (latest and greatest). Feel free to contact me directly if you need more info.

On the switch side, switches build according to the spec will auto negotiate to the lower speed, so connecting QDR port to DDR port will cause the QDR port to go down to DDR, but it will work with no issues.

Gilad


-----Original Message-----
From: beowulf-bounces at beowulf.org [mailto:beowulf-bounces at beowulf.org] On Behalf Of Ramiro Alba
Sent: Tuesday, March 29, 2011 1:37 AM
To: beowulf at beowulf.org
Subject: [Beowulf] AMD 8 cores vs 12 cores CPUs and Infiniband

Hi all,

We currently have an IB DDR cluster of 128 AMD QuadCore 'Barcelona' (4 *
2 cores) using DDR 'Flextronics' switches ( http://www.ibswitches.com) with an over-subscription 2:1 (two 24 port switches/rack a one 144 port modular 'Flextronics' Fat-Tree switch to link). Every IB card in nodes, is Mellanox DDR InfiniHost III Lx PCI Express x8 (one port) and we use IB both for calculations (MPI) and storage (Lustre: 80 TB using 1 MDS and 2 OSSs on a DDN 9900 unit)

Now, we are planning to add new AMD 'Magny-Cours' nodes (16 or 24 cores) using Infiniband QDR, but linking with the Flextronics 144 port DDR switch, using 'Hybrid Pasive Copper QSFP to MicroGiGaCN cables', so as we can reach the Lustre storage.

But there are two main issues that we are worried about:

1 - One port IB cards QP saturation

Using 16 cores per node (8 * 2) seem the 'safe' option, but the 24 cores
(12 * 2) is better in term of price per job. Our CFD applications using MPI (OpenMPI) may need to do about 15 'MPI_allreduce' calls in one seccond or less, and we may probably using a pool of 1500 cores. ?Is anyone having this kind of 'message rate', using AMD 24 cores, and can tell me about his/her solution/experience?

2 - I've heard that QLogic behavior is better in terms of QP creation, I have also to think on linking IB DDR with QDR to reach the 'Lustre'
storage. I suppose the main issue is, which QDR switch or switches are linking with the Flextronics 144 port DDR switch, but I do not know what is the role of the node card (Mellanox/Qlogic). Again, can anyone tell me about his/her solution/experience?

Any comment suggestion will be welcomed

Thanks in advance

Regards
--
Ramiro Alba

Centre Tecnol?gic de Tranfer?ncia de Calor http://www.cttc.upc.edu


Escola T?cnica Superior d'Enginyeries
Industrial i Aeron?utica de Terrassa
Colom 11, E-08222, Terrassa, Barcelona, Spain
Tel: (+34) 93 739 86 46



--
Aquest missatge ha estat analitzat per MailScanner a la cerca de virus i d'altres continguts perillosos, i es considera que est net.

_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.



From hahn at mcmaster.ca  Wed Mar 30 00:02:33 2011
From: hahn at mcmaster.ca (Mark Hahn)
Date: Wed, 30 Mar 2011 00:02:33 -0400 (EDT)
Subject: [Beowulf] AMD 8 cores vs 12 cores CPUs and Infiniband
In-Reply-To: <9FA59C95FFCBB34EA5E42C1A8573784F037FD082@mtiexch01.mti.com>
References: <1301387847.1995.144.camel@mundo>
	<9FA59C95FFCBB34EA5E42C1A8573784F037FD082@mtiexch01.mti.com>
Message-ID: <Pine.LNX.4.64.1103292349500.16437@coffee.psychology.mcmaster.ca>

> I have been using single card on Magny-Cours with no issues at all. You can

interesting.  what adjustments have you made to the MPI stack to permit this?
we've had a variety of apps that fail intermittently on high-core nodes.
I have to say I was surprised such a thing came up - not sure whether it's 
inherent to IB or a result of the openmpi stack.  our usual way to test 
this is to gradually reduce the ranks-per-node for the job until it starts
to work.  an interesting cosmology code works at 1 pppn but not 3 ppn
on our recent 12c MC, mellanox QDR cluster.

> Using 16 cores per node (8 * 2) seem the 'safe' option, but the 24 cores
> (12 * 2) is better in term of price per job. Our CFD applications using MPI
> (OpenMPI) may need to do about 15 'MPI_allreduce' calls in one seccond or
> less, and we may probably using a pool of 1500 cores.

but will that allreduce be across 1500 cores?  I can get you a scaling curve
for the previously mentioned MC cluster (2.2 GHz).

> 2 - I've heard that QLogic behavior is better in terms of QP creation, I

well, they've often bragged about message rates - I'm not sure how relate
that is to QP creation.  I'd be interested to hear of some experiences, too.

regards, mark hahn.
_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.


From lindahl at pbm.com  Wed Mar 30 15:08:13 2011
From: lindahl at pbm.com (Greg Lindahl)
Date: Wed, 30 Mar 2011 12:08:13 -0700
Subject: [Beowulf] AMD 8 cores vs 12 cores CPUs and Infiniband
In-Reply-To: <Pine.LNX.4.64.1103292349500.16437@coffee.psychology.mcmaster.ca>
References: <1301387847.1995.144.camel@mundo>
	<9FA59C95FFCBB34EA5E42C1A8573784F037FD082@mtiexch01.mti.com>
	<Pine.LNX.4.64.1103292349500.16437@coffee.psychology.mcmaster.ca>
Message-ID: <20110330190813.GA16949@bx9.net>

On Wed, Mar 30, 2011 at 12:02:33AM -0400, Mark Hahn wrote:

> > 2 - I've heard that QLogic behavior is better in terms of QP creation, I
> 
> well, they've often bragged about message rates - I'm not sure how relate
> that is to QP creation.

They are two separate issues. PSM's equivalent of a QP is much
lighter-weight than the IB standard QP. With QLogic cards, if you are
talking to a lot of different QPs, your performance doesn't degrade
like it does with traditional IB cards.

It's yet another thing which isn't well measured by 2-core
microbenchmarks.

-- greg

(I no longer have a financial interest in QLogic. You should
try out the /hpc slashtag on blekko, though...)
_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.


From Shainer at Mellanox.com  Wed Mar 30 17:51:44 2011
From: Shainer at Mellanox.com (Gilad Shainer)
Date: Wed, 30 Mar 2011 14:51:44 -0700
Subject: [Beowulf] AMD 8 cores vs 12 cores CPUs and Infiniband
References: <1301387847.1995.144.camel@mundo><9FA59C95FFCBB34EA5E42C1A8573784F037FD082@mtiexch01.mti.com><Pine.LNX.4.64.1103292349500.16437@coffee.psychology.mcmaster.ca>
	<20110330190813.GA16949@bx9.net>
Message-ID: <9FA59C95FFCBB34EA5E42C1A8573784F037FD256@mtiexch01.mti.com>

> > well, they've often bragged about message rates - I'm not sure how
> relate
> > that is to QP creation.
> 
> They are two separate issues. PSM's equivalent of a QP is much
> lighter-weight than the IB standard QP. With QLogic cards, if you are
> talking to a lot of different QPs, your performance doesn't degrade
> like it does with traditional IB cards.
> 
> It's yet another thing which isn't well measured by 2-core
> microbenchmarks.


The performance does not degrade with standard IB solutions. 

Gilad
_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.


From orion at cora.nwra.com  Wed Mar 30 18:42:44 2011
From: orion at cora.nwra.com (Orion Poplawski)
Date: Wed, 30 Mar 2011 16:42:44 -0600
Subject: [Beowulf] GP-GPU experience
In-Reply-To: <45254.192.168.93.213.1300711866.squirrel@mail.eadline.org>
References: <68A57CCFD4005646957BD2D18E60667B1292EC74@milexchmb1.mil.tagmclarengroup.com>	<Pine.LNX.4.64.1103032343240.24334@coffee.psychology.mcmaster.ca>	<E1PvvZz-0003ng-00.mikky_m-mail-ru@f121.mail.ru>	<Pine.LNX.4.64.1103072315300.8438@coffee.psychology.mcmaster.ca>	<46317.192.168.93.213.1299685688.squirrel@mail.eadline.org>
	<45254.192.168.93.213.1300711866.squirrel@mail.eadline.org>
Message-ID: <4D93B1E4.3080407@cora.nwra.com>

On 03/21/2011 06:51 AM, Douglas Eadline wrote:
> I got to thinking about how others are fairing (or not)
> with GP-GPU technology. I put up a simple poll on
> ClusterMonkey to help get a general idea.
> (you can find it on the front page right top)
> If you have a moment, please provide
> your experience (results are available as well).

We've seen some reasonable speedup (12x) with some matlab code using Jacket. 
It required up-to-the-minute bugfixes/enhancements from Accelereyes to get it 
working though.  Ran into lots of limitations with some other code (sparse 
matrices) that prevented it from being usable.  Have some reports of success 
with gpulib and IDL.


-- 
Orion Poplawski
Technical Manager                     303-415-9701 x222
NWRA/CoRA Division                    FAX: 303-415-9702
3380 Mitchell Lane                  orion at cora.nwra.com
Boulder, CO 80301              http://www.cora.nwra.com
_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.


From hearnsj at googlemail.com  Thu Mar 31 00:43:07 2011
From: hearnsj at googlemail.com (John Hearns)
Date: Thu, 31 Mar 2011 05:43:07 +0100
Subject: [Beowulf] Job in the UK
Message-ID: <AANLkTimiJVnP5gwFuOAaa80mdj0PM5JMh_ojKuUTCxUJ@mail.gmail.com>

If anyone on the Beowulf list is looking for a job as an HPC engineer,
I spotted an agency advert for an HPC/Linux support engineer.
This is in a place called Banbury, so a quick google for 'HPC engineer
Banbury' will turn up the agencies.
Drop me an email off list if you're interested - it might be worth
looking at the F1 teams in the area.

I hasten to add this has nothing to do with my present employer.

John Hearns
_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.


From samuel at unimelb.edu.au  Thu Mar 31 06:56:25 2011
From: samuel at unimelb.edu.au (Christopher Samuel)
Date: Thu, 31 Mar 2011 21:56:25 +1100
Subject: [Beowulf] Virtualized Infiniband in KVM?
In-Reply-To: <4D9098B2.2080801@scalableinformatics.com>
References: <4D9098B2.2080801@scalableinformatics.com>
Message-ID: <4D945DD9.5010305@unimelb.edu.au>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 29/03/11 01:18, Joe Landman wrote:

> I know about the VT-d bits for virtualized 1 1GbE/10GbE
> and their  ability to be leveraged for KVM stacks.  Is
> there a similar capability for IB?

I believe so, but according to this presentation the
numbers don't look that good:

http://salsahpc.indiana.edu/CloudCom2010/slides/PDF/Recommendations%20for%20Virtualization%20Technologies%20in%20High%20Performance%20Computing.pdf

It was presented on 2nd December 2010 so probably
reasonably up to date although their slides provide
no information at all on which kernel version they
were using for this testing.

More info on the paper here:

http://salsahpc.indiana.edu/CloudCom2010/papers.html#00108

[...]
# This work benchmarks two virtual machine monitors, OpenVZ and KVM,
# specifically focusing on I/O throughput since CPU efficiency has
# been extensively studied [1]. OpenVZ offers near native I/O
# performance. Amazon?s EC2 ?Cluster Compute Node? product is also
# considered for comparative purposes and performs quite well. The
# EC2 ?Cluster Compute Node? product utilizes the Xen hypervisor in
# hvm mode and 10 Gbit/s Ethernet for high throughput communication.
# Therefore, we also briefly studied Xen on our hardware platform
# (in hvm mode) to determine if there are still areas of improvement
# in KVM that allow EC2 to outperform KVM (with InfiniBand host channel
# adapters operating at 20 Gbit/s) in MPI benchmarks. We conclude
# that KVM?s I/O performance is suboptimal, potentially due to memory
# management problems in the hypervisor. Amazon?s EC2 service is
# promising, although further investigation is necessary to understand
# the effects of network based storage on I/O throughput in compute
# nodes. Amazon?s offering may be attractive for users searching for
# "InfiniBand-like" performance without the upfront investment required
# to build an InfiniBand cluster or users wishing to dynamically expand
# their cluster during periods of high demand.

cheers!
Chris
- -- 
    Christopher Samuel - Senior Systems Administrator
 VLSCI - Victorian Life Sciences Computation Initiative
 Email: samuel at unimelb.edu.au Phone: +61 (0)3 903 55545
         http://www.vlsci.unimelb.edu.au/

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk2UXdkACgkQO2KABBYQAh+O+wCeKazajm8BAoVPKFtS3LrxF3BE
SxEAniyVrwxyQTzm+oLy8ny+aPqTilxh
=qByj
-----END PGP SIGNATURE-----
_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.



From samuel at unimelb.edu.au  Thu Mar 31 07:01:26 2011
From: samuel at unimelb.edu.au (Christopher Samuel)
Date: Thu, 31 Mar 2011 22:01:26 +1100
Subject: [Beowulf] Virtualized Infiniband in KVM?
In-Reply-To: <4D945DD9.5010305@unimelb.edu.au>
References: <4D9098B2.2080801@scalableinformatics.com>
	<4D945DD9.5010305@unimelb.edu.au>
Message-ID: <4D945F06.8060301@unimelb.edu.au>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 31/03/11 21:56, Christopher Samuel wrote:

> It was presented on 2nd December 2010 so probably
> reasonably up to date although their slides provide
> no information at all on which kernel version they
> were using for this testing.

Just found the PDF of the paper itself, they tested
Fedora 12 with 2.6.32 for KVM, so there might be
performance improvements in the last 6 kernel releases
since that one came out.

http://www.nd.edu/~nregola/files/cloudcom2010.pdf

cheers!
Chris
- -- 
    Christopher Samuel - Senior Systems Administrator
 VLSCI - Victorian Life Sciences Computation Initiative
 Email: samuel at unimelb.edu.au Phone: +61 (0)3 903 55545
         http://www.vlsci.unimelb.edu.au/

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk2UXwYACgkQO2KABBYQAh+qoACeMaQ4v021F9JwnFm3sm1ZFOOE
uE0AnjQXeioyClBvP9C5LlJrVLG9TiZt
=Dmnj
-----END PGP SIGNATURE-----
_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.


From samuel at unimelb.edu.au  Thu Mar 31 07:12:22 2011
From: samuel at unimelb.edu.au (Christopher Samuel)
Date: Thu, 31 Mar 2011 22:12:22 +1100
Subject: [Beowulf] Execution time measurements (fwd)
In-Reply-To: <Pine.LNX.4.64.1103072315300.8438@coffee.psychology.mcmaster.ca>
References: <68A57CCFD4005646957BD2D18E60667B1292EC74@milexchmb1.mil.tagmclarengroup.com>	<Pine.LNX.4.64.1103032343240.24334@coffee.psychology.mcmaster.ca>	<E1PvvZz-0003ng-00.mikky_m-mail-ru@f121.mail.ru>
	<Pine.LNX.4.64.1103072315300.8438@coffee.psychology.mcmaster.ca>
Message-ID: <4D946196.4090308@unimelb.edu.au>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 08/03/11 16:26, Mark Hahn wrote:

> if the app doesn't control this (or you with numactl),
> then you should expect performance to lie somewhere
> between the two extremes (fully local vs fully remote).
> the kernel does make some effort at keeping things local
> - and for that matter, avoiding moving a process among
> multiple cores/sockets.

It's worth also mentioning the issue of "NUMA diffusion"
through swapping made by David Singleton from ANU on the
hwloc-devel list:

http://www.open-mpi.org/community/lists/hwloc-devel/2011/02/2012.php

# Unless it has changed very recently, Linux swapin_readahead
# is the main culprit in messing with NUMA locality on that
# platform. Faulting a single page causes 8 or 16 or whatever
# contiguous pages to be read from swap. An arbitrary contiguous
# range of pages in swap may not even come from the same process
# far less the same NUMA node. My understanding is that since
# there is no NUMA info with the swap entry, the only policy
# that can be applied to is that of the faulting vma in the
# faulting process. The faulted page will have the desired NUMA
# placement but possibly not the rest. So swapping mixes
# different process' NUMA policies leading to a "NUMA diffusion
# process".

Keep in mind that the reason that ANU runs systems with swap
is so they can suspend jobs, page the entire thing out and
start a new higher priority job.  Running without swap isn't
really an option for them..

cheers,
Chris
- -- 
    Christopher Samuel - Senior Systems Administrator
 VLSCI - Victorian Life Sciences Computation Initiative
 Email: samuel at unimelb.edu.au Phone: +61 (0)3 903 55545
         http://www.vlsci.unimelb.edu.au/

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk2UYZYACgkQO2KABBYQAh/YxwCdFXz5yAdpKqz3G/Sk+AY73E2E
XmMAn0aws1puNktvnaHQbhycul7pMrRp
=jke/
-----END PGP SIGNATURE-----
_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.


From heiko.bauke at snafu.de  Tue Mar  1 08:30:22 2011
From: heiko.bauke at snafu.de (Heiko Bauke)
Date: Tue, 1 Mar 2011 14:30:22 +0100
Subject: [Beowulf] Random numbers (was Re: /dev/random entropy on
 stateless/headless nodes)
In-Reply-To: <alpine.LFD.2.02.1102281346210.3235@lilith>
References: <alpine.BSF.2.00.1102251708280.98544@freeman.4gh.net>
	<Pine.LNX.4.64.1102261715520.7392@coffee.psychology.mcmaster.ca>
	<AANLkTinrDKWde+2VYAYu0XdE9CTU=H-NcO=AippwC+_p@mail.gmail.com>
	<alpine.LFD.2.02.1102271131580.4648@lilith>
	<20110228104103.6e195d2b@keitel105.dhcp.mpi-hd.mpg.de>
	<alpine.LFD.2.02.1102281346210.3235@lilith>
Message-ID: <20110301143022.3a0d6e5d@keitel105.dhcp.mpi-hd.mpg.de>

Hi,

On Mon, 28 Feb 2011 15:03:33 -0500 (EST)
"Robert G. Brown" <rgb at phy.duke.edu> wrote:

> It would also be very interesting to try feeding one of your MRG or
> YARN-type generators into dieharder and running the full series of
> tests on it (there are actually several diehard tests in dieharder
> that look for hyperplanar correlations as well as the long tail
> "monkey" tests). If I ever have time I'll try to give it a shot.  Is
> the library and/or a cli available anyplace easy to find?

there is an open source C++ library implementing our YARN generators and
others, see http://trng.berlios.de/ and http://trng.berlios.de/trng.pdf .


	Heiko


-- 
-- Irrend lernt man.
-- (Johann Wolfgang von Goethe, 1749-1832)
-- Number Crunch Blog @ http://numbercrunch.de
--        Heiko Bauke @ http://www.mpi-hd.mpg.de/personalhomes/bauke
_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.


From rgb at phy.duke.edu  Tue Mar  1 10:25:30 2011
From: rgb at phy.duke.edu (Robert G. Brown)
Date: Tue, 1 Mar 2011 10:25:30 -0500 (EST)
Subject: [Beowulf] Random numbers (was Re: /dev/random entropy on
 stateless/headless nodes)
In-Reply-To: <20110301143022.3a0d6e5d@keitel105.dhcp.mpi-hd.mpg.de>
References: <alpine.BSF.2.00.1102251708280.98544@freeman.4gh.net>
	<Pine.LNX.4.64.1102261715520.7392@coffee.psychology.mcmaster.ca>
	<AANLkTinrDKWde+2VYAYu0XdE9CTU=H-NcO=AippwC+_p@mail.gmail.com>
	<alpine.LFD.2.02.1102271131580.4648@lilith>
	<20110228104103.6e195d2b@keitel105.dhcp.mpi-hd.mpg.de>
	<alpine.LFD.2.02.1102281346210.3235@lilith>
	<20110301143022.3a0d6e5d@keitel105.dhcp.mpi-hd.mpg.de>
Message-ID: <alpine.LFD.2.02.1103011020150.4012@lilith>

On Tue, 1 Mar 2011, Heiko Bauke wrote:

> Hi,
>
> On Mon, 28 Feb 2011 15:03:33 -0500 (EST)
> "Robert G. Brown" <rgb at phy.duke.edu> wrote:
>
>> It would also be very interesting to try feeding one of your MRG or
>> YARN-type generators into dieharder and running the full series of
>> tests on it (there are actually several diehard tests in dieharder
>> that look for hyperplanar correlations as well as the long tail
>> "monkey" tests). If I ever have time I'll try to give it a shot.  Is
>> the library and/or a cli available anyplace easy to find?
>
> there is an open source C++ library implementing our YARN generators and
> others, see http://trng.berlios.de/ and http://trng.berlios.de/trng.pdf .

Thanks, I grabbed it.  I don't usually use C++, but hopefully I can link
this back to a C interface or at worst write a simple CLI program to
spit out a binary stream.  dieharder can test a binary stream via a pipe
(it is set up to test "large" numbers of random numbers, much larger
than diehard, although practically speaking many of the tests are
asymtotic and fail if you try to test the 10^12 and up numbers of rands
used in many modern MC computations.  But it at least makes it easy (and
still fairly reliable) to test 10^9 or more.  (I'd love to make it work
for longer streams, but that proves to be remarkably difficult for
numerical reasons, for at least the most "interesting" tests...:-).

Might take a while to get around to this, but if/when I do I'll try to
let you know.

    rgb

>
>
> 	Heiko
>
>
> -- 
> -- Irrend lernt man.
> -- (Johann Wolfgang von Goethe, 1749-1832)
> -- Number Crunch Blog @ http://numbercrunch.de
> --        Heiko Bauke @ http://www.mpi-hd.mpg.de/personalhomes/bauke
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
>

Robert G. Brown	                       http://www.phy.duke.edu/~rgb/
Duke University Dept. of Physics, Box 90305
Durham, N.C. 27708-0305
Phone: 1-919-660-2567  Fax: 919-660-2525     email:rgb at phy.duke.edu


_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.


From john.hearns at mclaren.com  Tue Mar  1 10:50:41 2011
From: john.hearns at mclaren.com (Hearns, John)
Date: Tue, 1 Mar 2011 15:50:41 -0000
Subject: [Beowulf] Why do we need SANs
Message-ID: <68A57CCFD4005646957BD2D18E60667B1292EC74@milexchmb1.mil.tagmclarengroup.com>

This is a thought provoking article on storage:

http://www.theregister.co.uk/2011/03/01/why_do_we_need_sans/


In the light of the recent thread on Thunderbolt, note:
"Infiniband or ..... extermalised PCIe bus"
Do we see the HPC system of the future being a huge blade server farm,
running lots of virtual machines,
with some sort of object storage system spread across the machine
running on selected blades?




John Hearns | CFD Hardware Specialist | McLaren Racing Limited
McLaren Technology Centre, Chertsey Road, Woking, Surrey GU21 4YH, UK

T:  +44 (0) 1483 261000
D:  +44 (0) 1483 262352
F:  +44 (0) 1483 261010
E:  john.hearns at mclaren.com
W:  www.mclaren.com




The contents of this email are confidential and for the exclusive use of the intended recipient.  If you receive this email in error you should not copy it, retransmit it, use it or disclose its contents but should return it to the sender immediately and delete your copy.
_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.


From john.hearns at mclaren.com  Wed Mar  2 04:44:43 2011
From: john.hearns at mclaren.com (Hearns, John)
Date: Wed, 2 Mar 2011 09:44:43 -0000
Subject: [Beowulf] Random numbers (was Re: /dev/random entropy on
 stateless/headless nodes)
In-Reply-To: <alpine.LFD.2.02.1103011020150.4012@lilith>
References: <alpine.BSF.2.00.1102251708280.98544@freeman.4gh.net><Pine.LNX.4.64.1102261715520.7392@coffee.psychology.mcmaster.ca><AANLkTinrDKWde+2VYAYu0XdE9CTU=H-NcO=AippwC+_p@mail.gmail.com><alpine.LFD.2.02.1102271131580.4648@lilith><20110228104103.6e195d2b@keitel105.dhcp.mpi-hd.mpg.de><alpine.LFD.2.02.1102281346210.3235@lilith><20110301143022.3a0d6e5d@keitel105.dhcp.mpi-hd.mpg.de>
	<alpine.LFD.2.02.1103011020150.4012@lilith>
Message-ID: <68A57CCFD4005646957BD2D18E60667B1292ECDB@milexchmb1.mil.tagmclarengroup.com>

Talking about seeding random numbers from dates and network interfaces,
I happen to be looking at the Intel igb driver for a completely separate
reason today.

One of the parameters in the igb kernel module is:


parm:           entropy:Allow igb to populate the /dev/random entropy
pool (int)


I guess this isn't news to most people.
John Hearns

The contents of this email are confidential and for the exclusive use of the intended recipient.  If you receive this email in error you should not copy it, retransmit it, use it or disclose its contents but should return it to the sender immediately and delete your copy.
_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.


From hahn at mcmaster.ca  Fri Mar  4 00:15:16 2011
From: hahn at mcmaster.ca (Mark Hahn)
Date: Fri, 4 Mar 2011 00:15:16 -0500 (EST)
Subject: [Beowulf] Why do we need SANs
In-Reply-To: <68A57CCFD4005646957BD2D18E60667B1292EC74@milexchmb1.mil.tagmclarengroup.com>
References: <68A57CCFD4005646957BD2D18E60667B1292EC74@milexchmb1.mil.tagmclarengroup.com>
Message-ID: <Pine.LNX.4.64.1103032343240.24334@coffee.psychology.mcmaster.ca>

> This is a thought provoking article on storage:
>
> http://www.theregister.co.uk/2011/03/01/why_do_we_need_sans/

I didn't think it was particularly good: basically just the observation
that you can put lots of servers instances into a big box and do away 
with a lot of infrastructure.  call it "mainframe".

the reason this is not good and maybe even bad is that it pretends 
that there's no need for scalable systems.  and it seems to be based 
on the premise that infrastructure such as FC, which has always been
very complex and expensive, would somehow be necessary.

besides, who would even propose SANs these days?  they're inherently
hostile to sharing, and it's all about sharing today...

> In the light of the recent thread on Thunderbolt, note:
> "Infiniband or ..... extermalised PCIe bus"

as far as I can tell, thunderbolt doesn't introduce anything interesting
regarding addressing, switching fabrics or low-latency transactions between
hosts.  I can't tell whether Intel wants to push it in that direction,
or whether it's a short-term sop for Apple.

> Do we see the HPC system of the future being a huge blade server farm,
> running lots of virtual machines,

hell no.  first of all, blades are done.  they never made much sense,
and what little sense they had (higher efficiency PSUs and cooling)
has now become available in standard/commodity parts.

but more importantly, why the heck would we want VMs for HPC?  virtualization
is great for packing lots of different, low-duty-cycle server instances 
onto less hardware.  and it provides some help in shuffling them around.
HPC is high duty-cycle, and doesn't the inherent inefficiency of virt.
being able to move processes around would he helpful for HPC, but it doesn't
need to be whole OS instances.

> with some sort of object storage system spread across the machine
> running on selected blades?

SANs simply lack the expressive power necessary to support modern 
file/object/database/web sharing protocols.  and the SAN world, which 
considers 8Gb FC to be sexy, has been left in the dust by 40 Gb IB 
or even ethernet.  (if IB had built on ethernet protocols rather than 
badly reinventing the wheel, no one would even remember FC today...)

regards, mark hahn.
_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.


From hahn at mcmaster.ca  Tue Mar  8 00:26:26 2011
From: hahn at mcmaster.ca (Mark Hahn)
Date: Tue, 8 Mar 2011 00:26:26 -0500 (EST)
Subject: [Beowulf] Execution time measurements (fwd)
In-Reply-To: <E1PvvZz-0003ng-00.mikky_m-mail-ru@f121.mail.ru>
References: <68A57CCFD4005646957BD2D18E60667B1292EC74@milexchmb1.mil.tagmclarengroup.com>
	<Pine.LNX.4.64.1103032343240.24334@coffee.psychology.mcmaster.ca>
	<E1PvvZz-0003ng-00.mikky_m-mail-ru@f121.mail.ru>
Message-ID: <Pine.LNX.4.64.1103072315300.8438@coffee.psychology.mcmaster.ca>

Hi list,
I'm forwarding a message (took the liberty of inserting some of my own
comments...)

Mikhail Kuzminsky writes:
> I still don't have the possibility to send the messages directly to Beowulf
>maillist (although I receive all the mail messages). Could you pls forward
>my question to beowulf at beowulf.org ? 
>
> Thanks for your help !
> Mikhail
>
> Execution time measurements
>
> I  obtained strange results for Gaussian-09 (G09) execution times in
>sequential (!) run on dual socket Opteron-2350 based node under Open SuSE
>10.3 (kernel 2.6.22). The measurements were performed for  "pure cpu-bound"
>job (frequencies calculation at DFT level) where start-stop execution time
>is practically equal to cpu execution time (difference is lower than 1 min
>per 1 day of execution). G09 itself prints both cpu execution time, and
>start & stop dates/time information. 
>
> There is some job which execution time is about 1 day. But really it was
>measured two *DIFFERENT* execution times: 1st - 1 day (24h) and 2nd - 1 day
>3 hours (27h). Both results were reproduced few times and gives the same
>quantum-chemical results excluding execution time.  There was no other
>applications run simultaneously w/this measurements. Execution time
>difference isn't localized in any of G09 parts (links).

OK, if I understand: same tests, different timings.

> "Fast" execution  was reproduced 2 times: one - as usual sequential run and
>on - in simultaneous execution of 2 examples of this job (therefore there is
>no mutual influence of this jobs). These runs was not managed  manually in
>the sense of numa allocation. 
>
> "Slow" execution was reproduced minimum 5 times. The memory required for
>job execution is 4+ GB, and it's possible to allocate the necessary RAM from
>one node only (we have 8 GB per node, 16 GB per server). I forced (using
>numactl) to use both cpu (core) and memory from node 1, but it gives "slow"
>execution time. When I forced cpu 1 and memory 0, execution time was
>increased to 1h (up to 28h).

if I understand, the tests runs slower if the process is remote from memory.
that's not at all surprising, right?

> (BTW, g09 links are threads. Does numactl forcing for main module "extends"
>to child threads also ?)

numactl uses sched_setaffinity/etal to set process properties that _are_
inherited.  so unless g09 manipulates the numa settings, the "outer" numactl
settings will control threads and links as well.

> Then I checked G09 own timings via execution under time command. G09 and
>time results was the same, but I looked only "slow" execution time.

I haven't seen inaccuracies in g09's time-reporting.

> The frequency of cpus was fixed (there was no cpufreq kernel module loaded).
>
> I'll be very appreciate in any ideas about possible reason of two different
>execution time observations. 
>
>Mikhail Kuzminsky
>Computer Assistance to Chemical Research Center
>Zelinsky Institute of Organic Chemistry RAS
>Moscow Russia

I think you already have a grip on the answer: any program runs fastest 
when it's running with memory "local" to the CPU it's using.  those pages,
after all, are effectively slower.  if the app doesn't control this (or you
with numactl), then you should expect performance to lie somewhere between
the two extremes (fully local vs fully remote).  the kernel does make some
effort at keeping things local - and for that matter, avoiding moving a 
process among multiple cores/sockets.

how much this matters depends on the app.  anything cache-friendly won't
care - GUIs and a lot of servers would prefer to run sooner, rather than 
insisting on a particular cpu or bank of memory...

I'll confess something here: my organization hasn't bothered with any 
special affinity settings until recently.  I'm not sure how often HPC centers
do worry about this.  obviously, such settings complicate the scheduler's 
job and probably require more tuning by the user...

regards, mark hahn.


_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.


From deadline at eadline.org  Wed Mar  9 10:48:08 2011
From: deadline at eadline.org (Douglas Eadline)
Date: Wed, 9 Mar 2011 10:48:08 -0500 (EST)
Subject: [Beowulf] =?iso-8859-1?q?Chinese_supercomputers_to_use_=91homemad?=
	=?iso-8859-1?q?e=92_chips?=
In-Reply-To: <Pine.LNX.4.64.1103072315300.8438@coffee.psychology.mcmaster.ca>
References: <68A57CCFD4005646957BD2D18E60667B1292EC74@milexchmb1.mil.tagmclarengroup.com>
	<Pine.LNX.4.64.1103032343240.24334@coffee.psychology.mcmaster.ca>
	<E1PvvZz-0003ng-00.mikky_m-mail-ru@f121.mail.ru>
	<Pine.LNX.4.64.1103072315300.8438@coffee.psychology.mcmaster.ca>
Message-ID: <46317.192.168.93.213.1299685688.squirrel@mail.eadline.org>


Interesting:

Chinese supercomputers to use ?homemade? chips


http://timesofindia.indiatimes.com/tech/personal-tech/computing/Chinese-supercomputers-to-use-homemade-chips/articleshow/7655183.cms


--
Doug

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.

_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf


From vanallsburg at hope.edu  Thu Mar 10 11:49:51 2011
From: vanallsburg at hope.edu (Paul Van Allsburg)
Date: Thu, 10 Mar 2011 11:49:51 -0500
Subject: [Beowulf] ramdisk with tmpfs
Message-ID: <4D79012F.90709@hope.edu>

Hi All,
I want to make a ramdisk available to cluster users and I'm curious what  your experiences/suggestions might be.  I'm adding 40 
machines to a cluster that have 24 gig of ram.  I'd like to offer the option of allowing some users to be able to run a job on a 
machine with 20 gig carved out for a ram disk.

The cluster is running centos 5.5 with torque & maui.

I expect the user will have to request one machine for the job and have the prologue/eplogue scripts mount & unmount the 
ramdisk. Are there any success / horror stories that I might be enlightened by?

Thanks!
Paul

-- 
Paul Van Allsburg
Scientific Computing Specialist
Natural Sciences Division, Hope College
35 E. 12th St.  Holland, Michigan 49423
616-395-7292    vanallsburg at hope.edu
http://www.hope.edu/academic/csm/


_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.


From Craig.Tierney at noaa.gov  Thu Mar 10 12:16:51 2011
From: Craig.Tierney at noaa.gov (Craig Tierney)
Date: Thu, 10 Mar 2011 10:16:51 -0700
Subject: [Beowulf] ramdisk with tmpfs
In-Reply-To: <4D79012F.90709@hope.edu>
References: <4D79012F.90709@hope.edu>
Message-ID: <4D790783.4090002@noaa.gov>

On 3/10/11 9:49 AM, Paul Van Allsburg wrote:
> Hi All,
> I want to make a ramdisk available to cluster users and I'm curious what  your experiences/suggestions might be.  I'm adding 40
> machines to a cluster that have 24 gig of ram.  I'd like to offer the option of allowing some users to be able to run a job on a
> machine with 20 gig carved out for a ram disk.
>
> The cluster is running centos 5.5 with torque&  maui.
>
> I expect the user will have to request one machine for the job and have the prologue/eplogue scripts mount&  unmount the
> ramdisk. Are there any success / horror stories that I might be enlightened by?
>
> Thanks!
> Paul
>

As far as I recall, Centos creates a ramdisk by default at /dev/shm whose
maximum size is 1/2 of available memory.  The ramdisk uses available memory
as needed, and doesn't block an application from allocating
all of memory (as long as there is nothing in the ramdisk).

So you don't have to create one.  You can leave it bigger if you want,
but what is important is that the prologue/epilogue clears out the files
left there before the next run.  You could just mount/umount one as well
if you want, as you can have multiple ramdisks mounted at one time which
all size dynamically.

Craig
_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.


From shaeffer at neuralscape.com  Thu Mar 10 18:23:31 2011
From: shaeffer at neuralscape.com (Karen Shaeffer)
Date: Thu, 10 Mar 2011 15:23:31 -0800
Subject: [Beowulf] IBM's Watson on Jeopardy tonight
In-Reply-To: <alpine.LFD.2.02.1102161240390.3531@lilith>
References: <E1PpSsG-0006IP-Er@mendel.bio.caltech.edu>
	<4D5B8733.1010505@gmail.com>
	<50888AA2846A4ECF9332AFB80EFB87D5@USERPC>
	<0627DE5B-7CAD-4A72-8E8F-8D60167FC43B@jax.org>
	<ECE7A93BD093E1439C20020FBE87C47FEDE0A471A6@ALTPHYEMBEVSP20.RES.AD.JPL>
	<4D5BDF3B.6010104@pathscale.com>
	<alpine.LFD.2.02.1102161000050.3627@lilith>
	<4D5BEF7E.8060004@pathscale.com>
	<alpine.LFD.2.02.1102161240390.3531@lilith>
Message-ID: <20110310232331.GB5729@synapse.neuralscape.com>

On Wed, Feb 16, 2011 at 01:06:35PM -0500, Robert G. Brown wrote:
> On Wed, 16 Feb 2011, "C. Bergstr?m" wrote:
> 
> >>We are, but that problem is, well, "hard".  As in grand challenge hard.
> >I wonder how you'd really describe the human brain learning in terms of a 
> >programming model....
> 
> Well, AI researchers tend to have two answers.  One is semantic and the
> other is microscopic.  The semantic description is
> functional/operational, broken down in terms of e.g. conditionals and
> logical elements, and doesn't come close to explaining consciousness
> (see Searle's "Chinese Room" objection):
> 

Hi,
Well, IBM is presenting "Inside the mind of Watson" at Stanford University
on March 14. The speaker is Chris Welty of IBM. There are actually several
presentations. Here is one:

http://bmir.stanford.edu/events/view.php/inside_the_mind_of_watson

There is another one at 12:15 PM earlier that day associated with the
symbolic systems group. Those are often open to the public, if you
ask for permission to attend.

Karen
-- 
 Karen Shaeffer
 Neuralscape, Palo Alto, Ca. 94306
 shaeffer at neuralscape.com  http://www.neuralscape.com
_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.



From hahn at mcmaster.ca  Fri Mar 11 01:20:11 2011
From: hahn at mcmaster.ca (Mark Hahn)
Date: Fri, 11 Mar 2011 01:20:11 -0500 (EST)
Subject: [Beowulf]
 =?iso-8859-1?q?Chinese_supercomputers_to_use_=91homemad?=
 =?iso-8859-1?q?e=92_chips?=
In-Reply-To: <46317.192.168.93.213.1299685688.squirrel@mail.eadline.org>
References: <68A57CCFD4005646957BD2D18E60667B1292EC74@milexchmb1.mil.tagmclarengroup.com>
	<Pine.LNX.4.64.1103032343240.24334@coffee.psychology.mcmaster.ca>
	<E1PvvZz-0003ng-00.mikky_m-mail-ru@f121.mail.ru>
	<Pine.LNX.4.64.1103072315300.8438@coffee.psychology.mcmaster.ca>
	<46317.192.168.93.213.1299685688.squirrel@mail.eadline.org>
Message-ID: <Pine.LNX.4.64.1103110108590.19223@coffee.psychology.mcmaster.ca>

> Interesting:
> Chinese supercomputers to use ?homemade? chips
> http://timesofindia.indiatimes.com/tech/personal-tech/computing/Chinese-supercomputers-to-use-homemade-chips/articleshow/7655183.cms

it's important to remind ourselves that China is still a centrally-planned,
totalitarian dictatorship.  I mention this only because this announcement
is a bit like Putin et al announcing that they'll develop their own 
linux distro because Russia is big and important and mustn't allow 
itself to be vulnerable to foreign hegemony.

so far, the very shallow reporting I've seen has said that future 
generations will add wide FP vector units.  nothing wrong with that,
though it's a bit unclear to me why other companies haven't done it
if there is, in fact, lots of important vector codes that will run 
efficiently on such a configuration.  adding/widening vector FP is 
not breakthrough engineering afaikt.

has anyone heard anything juicy about the Tianhe interconnect?
-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.

-------------- next part --------------
_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

From eagles051387 at gmail.com  Fri Mar 11 03:47:10 2011
From: eagles051387 at gmail.com (Jonathan Aquilina)
Date: Fri, 11 Mar 2011 09:47:10 +0100
Subject: [Beowulf]
	=?windows-1252?q?Chinese_supercomputers_to_use_=91homem?=
	=?windows-1252?q?ade=92_chips?=
In-Reply-To: <Pine.LNX.4.64.1103110108590.19223@coffee.psychology.mcmaster.ca>
References: <68A57CCFD4005646957BD2D18E60667B1292EC74@milexchmb1.mil.tagmclarengroup.com>
	<Pine.LNX.4.64.1103032343240.24334@coffee.psychology.mcmaster.ca>
	<E1PvvZz-0003ng-00.mikky_m-mail-ru@f121.mail.ru>
	<Pine.LNX.4.64.1103072315300.8438@coffee.psychology.mcmaster.ca>
	<46317.192.168.93.213.1299685688.squirrel@mail.eadline.org>
	<Pine.LNX.4.64.1103110108590.19223@coffee.psychology.mcmaster.ca>
Message-ID: <AANLkTike3Nf9tf_p4q_q+Bc7FsudyenDUJVdYQWRoy3L@mail.gmail.com>

some more details about the chips.

http://www.physorg.com/news/2011-03-dawning-china-homemade-supercomputer.html

On Fri, Mar 11, 2011 at 7:20 AM, Mark Hahn <hahn at mcmaster.ca> wrote:

>  Interesting:
>> Chinese supercomputers to use ?homemade? chips
>>
>> http://timesofindia.indiatimes.com/tech/personal-tech/computing/Chinese-supercomputers-to-use-homemade-chips/articleshow/7655183.cms
>>
>
> it's important to remind ourselves that China is still a centrally-planned,
> totalitarian dictatorship.  I mention this only because this announcement
> is a bit like Putin et al announcing that they'll develop their own linux
> distro because Russia is big and important and mustn't allow itself to be
> vulnerable to foreign hegemony.
>
> so far, the very shallow reporting I've seen has said that future
> generations will add wide FP vector units.  nothing wrong with that,
> though it's a bit unclear to me why other companies haven't done it
> if there is, in fact, lots of important vector codes that will run
> efficiently on such a configuration.  adding/widening vector FP is not
> breakthrough engineering afaikt.
>
> has anyone heard anything juicy about the Tianhe interconnect?
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit
> http://www.beowulf.org/mailman/listinfo/beowulf
>
>


-- 
Jonathan Aquilina

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.clustermonkey.net/pipermail/beowulf/attachments/20110311/968a5bd4/attachment-0001.html>
-------------- next part --------------
_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

From prentice at ias.edu  Fri Mar 11 09:27:04 2011
From: prentice at ias.edu (Prentice Bisbal)
Date: Fri, 11 Mar 2011 09:27:04 -0500
Subject: [Beowulf] IBM's Watson on Jeopardy tonight
In-Reply-To: <20110310232331.GB5729@synapse.neuralscape.com>
References: <E1PpSsG-0006IP-Er@mendel.bio.caltech.edu>	<4D5B8733.1010505@gmail.com>	<50888AA2846A4ECF9332AFB80EFB87D5@USERPC>	<0627DE5B-7CAD-4A72-8E8F-8D60167FC43B@jax.org>	<ECE7A93BD093E1439C20020FBE87C47FEDE0A471A6@ALTPHYEMBEVSP20.RES.AD.JPL>	<4D5BDF3B.6010104@pathscale.com>	<alpine.LFD.2.02.1102161000050.3627@lilith>	<4D5BEF7E.8060004@pathscale.com>	<alpine.LFD.2.02.1102161240390.3531@lilith>
	<20110310232331.GB5729@synapse.neuralscape.com>
Message-ID: <4D7A3138.9070304@ias.edu>


On 03/10/2011 06:23 PM, Karen Shaeffer wrote:
> On Wed, Feb 16, 2011 at 01:06:35PM -0500, Robert G. Brown wrote:
>> On Wed, 16 Feb 2011, "C. Bergstr?m" wrote:
>>
>>>> We are, but that problem is, well, "hard".  As in grand challenge hard.
>>> I wonder how you'd really describe the human brain learning in terms of a 
>>> programming model....
>>
>> Well, AI researchers tend to have two answers.  One is semantic and the
>> other is microscopic.  The semantic description is
>> functional/operational, broken down in terms of e.g. conditionals and
>> logical elements, and doesn't come close to explaining consciousness
>> (see Searle's "Chinese Room" objection):
>>
> 
> Hi,
> Well, IBM is presenting "Inside the mind of Watson" at Stanford University
> on March 14. The speaker is Chris Welty of IBM. There are actually several
> presentations. Here is one:
> 
> http://bmir.stanford.edu/events/view.php/inside_the_mind_of_watson
> 
> There is another one at 12:15 PM earlier that day associated with the
> symbolic systems group. Those are often open to the public, if you
> ask for permission to attend.
> 
> Karen

This reminded me: On Feb 9, 2011,  PBS aired a Nova episode about the
development of Watson. The entire episode is available online:

http://www.pbs.org/wgbh/nova/tech/smartest-machine-on-earth.html

There's other interesting links and videos available on that page, too.


-- 
Prentice
_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.



From hahn at mcmaster.ca  Sat Mar 12 14:48:36 2011
From: hahn at mcmaster.ca (Mark Hahn)
Date: Sat, 12 Mar 2011 14:48:36 -0500 (EST)
Subject: [Beowulf]
 =?windows-1252?q?Chinese_supercomputers_to_use_=91homem?=
 =?windows-1252?q?ade=92_chips?=
In-Reply-To: <AANLkTike3Nf9tf_p4q_q+Bc7FsudyenDUJVdYQWRoy3L@mail.gmail.com>
References: <68A57CCFD4005646957BD2D18E60667B1292EC74@milexchmb1.mil.tagmclarengroup.com>
	<Pine.LNX.4.64.1103032343240.24334@coffee.psychology.mcmaster.ca>
	<E1PvvZz-0003ng-00.mikky_m-mail-ru@f121.mail.ru>
	<Pine.LNX.4.64.1103072315300.8438@coffee.psychology.mcmaster.ca>
	<46317.192.168.93.213.1299685688.squirrel@mail.eadline.org>
	<Pine.LNX.4.64.1103110108590.19223@coffee.psychology.mcmaster.ca>
	<AANLkTike3Nf9tf_p4q_q+Bc7FsudyenDUJVdYQWRoy3L@mail.gmail.com>
Message-ID: <Pine.LNX.4.64.1103110948200.2209@coffee.psychology.mcmaster.ca>

> some more details about the chips.
>
> http://www.physorg.com/news/2011-03-dawning-china-homemade-supercomputer.html

seems like mostly an nth-gen rehash of the info here:
http://www.theregister.co.uk/2011/02/25/ict_godson_3b_chip/

which discusses the provenance of the basic architecture.
40W gets you 8 cores at 1 GHz.  each core appears to have 2x 256b FP SIMD,
but no mention is made of their rate.  if the rate were 1/cycle, that would
be 32 gflops per unit.  the article mentions "performance on those vector
units is 128 Gflops, so I'm not quite sure how the math goes.  later in 
the article it says 16x 8c chips go into a single 1U and yield 2 Tflops,
therefore 16 gflops per core.  so maybe it can start a new instr every other 
cycle.  in any case, 128 Gflops for 40W.

a similar comparison with a mainstream cluster chip might be closer to 1
gflop/watt (at the socket level, ignoring support chips, fans, PSU efficiency.)

current-gen Nvidia and ATI chips are in the 2-3 gflops/watt range,
though that's not a completely fair comparison.

regards, mark hahn.
_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.


From eugen at leitl.org  Sat Mar 12 14:59:50 2011
From: eugen at leitl.org (Eugen Leitl)
Date: Sat, 12 Mar 2011 20:59:50 +0100
Subject: [Beowulf]
	=?utf-8?q?Chinese_supercomputers_to_use_=E2=80=98homema?=
	=?utf-8?b?ZGXigJkgY2hpcHM=?=
In-Reply-To: <Pine.LNX.4.64.1103110948200.2209@coffee.psychology.mcmaster.ca>
References: <68A57CCFD4005646957BD2D18E60667B1292EC74@milexchmb1.mil.tagmclarengroup.com>
	<Pine.LNX.4.64.1103032343240.24334@coffee.psychology.mcmaster.ca>
	<E1PvvZz-0003ng-00.mikky_m-mail-ru@f121.mail.ru>
	<Pine.LNX.4.64.1103072315300.8438@coffee.psychology.mcmaster.ca>
	<46317.192.168.93.213.1299685688.squirrel@mail.eadline.org>
	<Pine.LNX.4.64.1103110108590.19223@coffee.psychology.mcmaster.ca>
	<AANLkTike3Nf9tf_p4q_q+Bc7FsudyenDUJVdYQWRoy3L@mail.gmail.com>
	<Pine.LNX.4.64.1103110948200.2209@coffee.psychology.mcmaster.ca>
Message-ID: <20110312195950.GZ23560@leitl.org>

On Sat, Mar 12, 2011 at 02:48:36PM -0500, Mark Hahn wrote:

> current-gen Nvidia and ATI chips are in the 2-3 gflops/watt range,
> though that's not a completely fair comparison.

Another vaporware shop is Calxeda
http://www.heise.de/newsticker/meldung/480-Core-Server-mit-ARM-Prozessoren-1206857.html
</kraut>

480 core/2U, 5 W/SoC node, 10x of x86 performance/Watt,
15-20x performance/USD.

-- 
Eugen* Leitl <a href="http://leitl.org">leitl</a> http://leitl.org
______________________________________________________________
ICBM: 48.07100, 11.36820 http://www.ativel.com http://postbiota.org
8B29F6BE: 099D 78BA 2FD3 B014 B08A  7779 75B0 2443 8B29 F6BE
_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.


From orion at cora.nwra.com  Thu Mar 17 13:42:14 2011
From: orion at cora.nwra.com (Orion Poplawski)
Date: Thu, 17 Mar 2011 11:42:14 -0600
Subject: [Beowulf] ramdisk with tmpfs
In-Reply-To: <4D790783.4090002@noaa.gov>
References: <4D79012F.90709@hope.edu> <4D790783.4090002@noaa.gov>
Message-ID: <4D8247F6.3090608@cora.nwra.com>

On 03/10/2011 10:16 AM, Craig Tierney wrote:
> On 3/10/11 9:49 AM, Paul Van Allsburg wrote:
>> Hi All,
>> I want to make a ramdisk available to cluster users and I'm curious what  your experiences/suggestions might be.  I'm adding 40
>> machines to a cluster that have 24 gig of ram.  I'd like to offer the option of allowing some users to be able to run a job on a
>> machine with 20 gig carved out for a ram disk.
>>
>> The cluster is running centos 5.5 with torque&   maui.
>>
>> I expect the user will have to request one machine for the job and have the prologue/eplogue scripts mount&   unmount the
>> ramdisk. Are there any success / horror stories that I might be enlightened by?
>>
>> Thanks!
>> Paul
>>
>
> As far as I recall, Centos creates a ramdisk by default at /dev/shm whose
> maximum size is 1/2 of available memory.  The ramdisk uses available memory
> as needed, and doesn't block an application from allocating
> all of memory (as long as there is nothing in the ramdisk).

No, don't use /dev/shm - this is a API mount for posix shared memory.  Mount 
another tmpfs somehwere else (we mount one at /tmp).  tmpfs is very nice, just 
remember that is competes with the RAM demands of your processes.

-- 
Orion Poplawski
Technical Manager                     303-415-9701 x222
NWRA/CoRA Division                    FAX: 303-415-9702
3380 Mitchell Lane                  orion at cora.nwra.com
Boulder, CO 80301              http://www.cora.nwra.com
_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.


From hahn at mcmaster.ca  Thu Mar 17 16:50:58 2011
From: hahn at mcmaster.ca (Mark Hahn)
Date: Thu, 17 Mar 2011 16:50:58 -0400 (EDT)
Subject: [Beowulf] ramdisk with tmpfs
In-Reply-To: <4D8247F6.3090608@cora.nwra.com>
References: <4D79012F.90709@hope.edu> <4D790783.4090002@noaa.gov>
	<4D8247F6.3090608@cora.nwra.com>
Message-ID: <Pine.LNX.4.64.1103171626270.7368@coffee.psychology.mcmaster.ca>

> No, don't use /dev/shm - this is a API mount for posix shared memory.  Mount
> another tmpfs somehwere else (we mount one at /tmp).  tmpfs is very nice, just
> remember that is competes with the RAM demands of your processes.

ramdisks are icky and tmfs is indeed much nicer.  tmpfs competes for ram,
but can be swapped, so is relatively polite.  tmpfs also has options
for limiting the size and number of inodes (set these for safety.)  and since
it's basically allocating ram for you, it also has NUMA parameters - 
see Documentation/filesystems/tmpfs.txt for details.  (I wonder whether 
allocating files mmaped from numa-tweaked tmpfs would provide the kind of
persistent numa binding to pages that some openmpi people want...)
_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.


From deadline at eadline.org  Mon Mar 21 08:51:06 2011
From: deadline at eadline.org (Douglas Eadline)
Date: Mon, 21 Mar 2011 08:51:06 -0400 (EDT)
Subject: [Beowulf] GP-GPU experience
In-Reply-To: <46317.192.168.93.213.1299685688.squirrel@mail.eadline.org>
References: <68A57CCFD4005646957BD2D18E60667B1292EC74@milexchmb1.mil.tagmclarengroup.com>
	<Pine.LNX.4.64.1103032343240.24334@coffee.psychology.mcmaster.ca>
	<E1PvvZz-0003ng-00.mikky_m-mail-ru@f121.mail.ru>
	<Pine.LNX.4.64.1103072315300.8438@coffee.psychology.mcmaster.ca>
	<46317.192.168.93.213.1299685688.squirrel@mail.eadline.org>
Message-ID: <45254.192.168.93.213.1300711866.squirrel@mail.eadline.org>


I was recently given a copy of "GPU Computing Gems"
to review. It is basically research quality NVidia success
stories, some of which are quite impressive.

I got to thinking about how others are fairing (or not)
with GP-GPU technology. I put up a simple poll on
ClusterMonkey to help get a general idea.
(you can find it on the front page right top)
If you have a moment, please provide
your experience (results are available as well).

  http://www.clustermonkey.net/

BTW: You can see all the previous polls
and links to other market data  here:

http://goo.gl/lDcUJ


--
Doug

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.

_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf


From eugen at leitl.org  Mon Mar 21 09:11:27 2011
From: eugen at leitl.org (Eugen Leitl)
Date: Mon, 21 Mar 2011 14:11:27 +0100
Subject: [Beowulf] GP-GPU experience
In-Reply-To: <45254.192.168.93.213.1300711866.squirrel@mail.eadline.org>
References: <68A57CCFD4005646957BD2D18E60667B1292EC74@milexchmb1.mil.tagmclarengroup.com>
	<Pine.LNX.4.64.1103032343240.24334@coffee.psychology.mcmaster.ca>
	<E1PvvZz-0003ng-00.mikky_m-mail-ru@f121.mail.ru>
	<Pine.LNX.4.64.1103072315300.8438@coffee.psychology.mcmaster.ca>
	<46317.192.168.93.213.1299685688.squirrel@mail.eadline.org>
	<45254.192.168.93.213.1300711866.squirrel@mail.eadline.org>
Message-ID: <20110321131127.GB23560@leitl.org>

On Mon, Mar 21, 2011 at 08:51:06AM -0400, Douglas Eadline wrote:
> 
> I was recently given a copy of "GPU Computing Gems"
> to review. It is basically research quality NVidia success
> stories, some of which are quite impressive.

This one?

http://www.amazon.com/GPU-Computing-Gems-Emerald-Applications/dp/0123849888

the reviewers didn't seem to think much of it.
 
> I got to thinking about how others are fairing (or not)
> with GP-GPU technology. I put up a simple poll on
> ClusterMonkey to help get a general idea.
> (you can find it on the front page right top)
> If you have a moment, please provide
> your experience (results are available as well).
> 
>   http://www.clustermonkey.net/
> 
> BTW: You can see all the previous polls
> and links to other market data  here:
> 
> http://goo.gl/lDcUJ

-- 
Eugen* Leitl <a href="http://leitl.org">leitl</a> http://leitl.org
______________________________________________________________
ICBM: 48.07100, 11.36820 http://www.ativel.com http://postbiota.org
8B29F6BE: 099D 78BA 2FD3 B014 B08A  7779 75B0 2443 8B29 F6BE
_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.


From john.hearns at mclaren.com  Mon Mar 21 11:33:40 2011
From: john.hearns at mclaren.com (Hearns, John)
Date: Mon, 21 Mar 2011 15:33:40 -0000
Subject: [Beowulf] Cray to build huge,
	grunting 20-petaflop 'Titan' for US gov labs
Message-ID: <207BB2F60743C34496BE41039233A80903C2B096@MRL-PWEXCHMB02.mil.tagmclarengroup.com>

http://www.theregister.co.uk/2011/03/21/ornl_cray_titan_super/


Looking at the final slide, guess we'd better start investing in
circular machine rooms.
(Actually..... I already work in a circular building.... Hmmm.....)

John Hearns | CFD Hardware Specialist | McLaren Racing Limited
McLaren Technology Centre, Chertsey Road, Woking, Surrey GU21 4YH, UK

T:  +44 (0) 1483 261000
D:  +44 (0) 1483 262352
F:  +44 (0) 1483 261010
E:  john.hearns at mclaren.com
W:  www.mclaren.com




The contents of this email are confidential and for the exclusive use of the intended recipient.  If you receive this email in error you should not copy it, retransmit it, use it or disclose its contents but should return it to the sender immediately and delete your copy.
_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.


From asabigue at fing.edu.uy  Mon Mar 21 12:12:55 2011
From: asabigue at fing.edu.uy (ariel sabiguero yawelak)
Date: Mon, 21 Mar 2011 13:12:55 -0300
Subject: [Beowulf] Cray to build huge,
 grunting 20-petaflop 'Titan' for US gov labs
In-Reply-To: <207BB2F60743C34496BE41039233A80903C2B096@MRL-PWEXCHMB02.mil.tagmclarengroup.com>
References: <207BB2F60743C34496BE41039233A80903C2B096@MRL-PWEXCHMB02.mil.tagmclarengroup.com>
Message-ID: <4D877907.7080806@fing.edu.uy>

So cool thinking about 100-250 petaflops, which is only 1 order of 
magnitude away from the first exaflop machine. It is interesting, on the 
other hand, the article recently published by IEEE Spectrum named 
"Next-Generation Supercomputers", which states that:/
/

    /"....The practical exaflops-class supercomputer DARPA was hoping
    for just wasn't going to be attainable by 2015. In fact, it might
    not be possible anytime in the foreseeable future....."
    /
    /http://spectrum.ieee.org/computing/hardware/nextgeneration-supercomputers/


There are some differences between different classes of exaflop 
supercomputers, but P. Kogge managed to convice me that "it might not be 
possible <..> in the <..> future".

Pretty interesting divergence.....

regards
ariel

El 21/03/11 12:33, Hearns, John escribi?:
> http://www.theregister.co.uk/2011/03/21/ornl_cray_titan_super/
>
>
> Looking at the final slide, guess we'd better start investing in
> circular machine rooms.
> (Actually..... I already work in a circular building.... Hmmm.....)
>
> John Hearns | CFD Hardware Specialist | McLaren Racing Limited
> McLaren Technology Centre, Chertsey Road, Woking, Surrey GU21 4YH, UK
>
> T:  +44 (0) 1483 261000
> D:  +44 (0) 1483 262352
> F:  +44 (0) 1483 261010
> E:  john.hearns at mclaren.com
> W:  www.mclaren.com
>
>
>
>
> The contents of this email are confidential and for the exclusive use of the intended recipient.  If you receive this email in error you should not copy it, retransmit it, use it or disclose its contents but should return it to the sender immediately and delete your copy.
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
>

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.clustermonkey.net/pipermail/beowulf/attachments/20110321/c9047ce4/attachment-0001.html>
-------------- next part --------------
_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

From eugen at leitl.org  Wed Mar 23 09:25:23 2011
From: eugen at leitl.org (Eugen Leitl)
Date: Wed, 23 Mar 2011 14:25:23 +0100
Subject: [Beowulf] relativistic contraints for trading
Message-ID: <20110323132523.GG23560@leitl.org>


http://www.bbc.co.uk/news/science-environment-12827752

Stock trades to exploit speed of light, says researcher

By Jason Palmer Science and technology reporter, BBC News, Dallas

Map showing relation between exchanges and trading locations Optimal
high-frequency trading locations (blue) exist for pairs of major financial
exchanges (red)

Financial institutions may soon change what they trade or where they do their
trading because of the speed of light.

"High-frequency trading" carried out by computers often depends on differing
prices of a financial instrument in two geographically-separated markets.

Exactly how far the signals have to go can make a difference in such trades.

Alexander Wissner-Gross told the American Physical Society meeting that
financial institutions are looking at ways to exploit the light-speed trick.

Dr Wissner-Gross, of Harvard University, said that the latencies -
essentially, the time delay for a signal to wing its way from one global
financial centre to another - advantaged some locations for some trades and
different locations for others.

There is a vast market for ever-faster fibre-optic cables to try to
physically "get there faster" but Dr Wissner-Gross said that the purely
technological approach to gaining an advantage was reaching a limit.

Trades now travel at nearly 90% of the ultimate speed limit set by physics,
the speed of light in the cables.  Competitive advantage

His first solution, published in 2010, considered the various latencies in
global fibre-optic links and mapped out where the optimal points for
financial transactions to originate - midway between two major financial hubs
to maximise the chance of "buying low" in one place and "selling high" in
another.

That of course resulted in a number of ideal locations in all corners of the
globe, including the oceans. But wholesale relocation of operations does not
immediately appeal to many firms.

"I'm now working... with real companies on real deployments that don't
require you deploy a floating data centre in the middle of the ocean; we say,
'OK, you have your existing infrastructure, that's not moving - now, given
your location, which stocks in various locations are you best positioned to
trade?'"

"If you don't have the budget to put new data centres in the middle of the
ocean you can, for example, use existing data centres that are an
approximation to the optimal location in the ocean - say, Nova Scotia for New
York to London," Dr Wissner-Gross told BBC News.

Because there is a clear, physical advantage to the approach, Dr
Wissner-Gross said that the first firm to try to exploit the effect will be
at significant competitive advantage - until more firms follow suit.

That means that out-of-the-way places - at high latitudes or mid-ocean island
chains - could in time turn into global financial centres.

"It's instructive to start to think about latency correlations as a new sort
of resource," he explained.

"If you're positioned between two major financial hubs, you may be far out of
the way, rather far from population centres, maybe economically poor, but
because of your unique position, that could be a natural resource."
_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.


From eugen at leitl.org  Thu Mar 24 06:45:42 2011
From: eugen at leitl.org (Eugen Leitl)
Date: Thu, 24 Mar 2011 11:45:42 +0100
Subject: [Beowulf] Rome Lab's supercomputer is made up of 1,
	700 off-the-shelf	PlayStation 3 gaming consoles
Message-ID: <20110324104542.GE23560@leitl.org>


http://www.syracuse.com/news/index.ssf/2011/03/rome_labs_supercomputer_is_mad.html

Rome Lab's supercomputer is made up of 1,700 off-the-shelf PlayStation 3
gaming consoles

Published: Wednesday, March 23, 2011, 6:00 AM     Updated: Wednesday, March
23, 2011, 8:02 AM

Dave Tobin / The Post-Standard By Dave Tobin / The Post-Standard

Jhn Berry / The Post-Standard Morgan Bishop, a computer scientist at the Air
Force Research Lab, in Rome, is surrounded by a high-speed, image-processing
network of 1,716 PlayStation 3 systems linked to each other and other
components.

Rome, NY -- Computer scientists just up the Thruway at Rome?s Air Force
Research Lab have assembled one of the world?s largest, fastest and cheapest
supercomputers ? and it?s made from PlayStation 3s.

By linking together 1,716 PlayStation 3s, they?ve created a supercomputer
that?s very good at processing, manipulating and interpreting vast amounts of
imagery. This will provide analysts with new levels of detail from the
pictures gathered on long surveillance flights by spy planes.

The PlayStation 3 is a video gaming console that originally sold for about
$500. It was developed by Sony, released in 2006 and is known for its
sizzlingly clear video graphics.

The Air Force calls the souped-up PlayStations the Condor Supercomputer and
says it is among the 40 fastest computers in the world. The Condor went
online late last year, and it will likely change the way the Air Force and
the Air National Guard watch things on the ground.

The creation, while offbeat, illustrates the modern job for the operation
that began as Rome Air Development Center in 1951, researching radar. It has
survived the closing of Griffiss Air Force Base in 1995 to find a new niche.

These days, Rome Lab?s research focuses on information technology,
particularly cybersecurity and high-performance computing. The lab employs
789 people in military and civilian jobs, with a payroll of $82 million a
year. It oversees contracts worth nearly $3 billion.

Rome is helping the military face a growing problem: Great advances in
airborne surveillance systems have the military drowning in visual data.

Meanwhile, the Air Force and the Air National Guard, as well as federal
agencies such as U.S. Customs and Border Protection, which already gather
surveillance video from the air, are pursuing technologies to gather more.
The goal: constant surveillance over large areas.
2011-02-22-jb-romelab3.JPGView full sizeJohn Berry / The Post-Standard A
PlayStation 3 video game console, one of the 1,716 systems linked together at
Rome Lab to form a supercomputer.

The Condor helps meet that. The Air Force has begun using a new radar
technology dubbed ?Gotcha,? with far sharper resolution than previous radar.
To maximize Gotcha?s potential, the power of a computer such as the Condor is
needed.

The Condor will enable 24-hour, real-time surveillance over a roughly
15-mile-wide area, said Mark Barnell, director of high-performance computing
at the Rome research lab.

Video processed from the radar signals can be viewed in real time or played
back to investigate what led to an event ? an explosion, an uprising or an
ambush. As with a video game, a viewer can change perspectives, going from
air to ground to look around buildings.

?You can literally rewind or predict forward (in the future), based on the
information you have,? Barnell said.

Development of the Condor started nearly five years ago, shortly after Sony
put the PlayStation 3 on the market. Richard W. Linderman, then senior
scientist at Rome?s Air Force research lab, brought the new PlayStation 3
home and began experimenting. The PS3 can run Linux, a software operating
system used in most of the world?s supercomputers.

At Rome Lab, Linderman asked his research team to try linking eight
PlayStation 3s to see what they could do. Impressed, he increased the number
to 336. That worked even better. What could more than a thousand do?

Rome Lab asked the Department of Defense for $2.5 million to assemble its
supercomputer. By the time money to buy that many was approved in 2009,
PlayStation 3s were hard to find. Rome Lab bought as many as they could ?
1,700.

To custom-build a supercomputer without using commercial off-the-shelf
PlayStation 3s would likely have cost 10 times as much, Barnell said. In
addition, the Condor uses a fraction of the energy that comparably sized
supercomputers use. Portions of it ? say 300 machines ? can be turned on
while the rest are off, depending on a job?s needs.

Rome Lab plans to work with the New York Air National Guard?s 174th Fighter
Wing, Barnell said. The 174th is seeking FAA permission to fly MQ-9 Reapers
in Northern New York, starting this summer.  2011-02-22-jb-romelab1.JPGView
full sizeJohn Berry / The Post-Standard Mark Barnell, director of
high-performance computing at the Air Force Research Lab, in Rome, stands
with a data wall that displays the video output of the center's Condor
Supercomputer. This sample image was made by a radar system aboard an
airplane.

The Air Force is also using the Condor to process ground-based radar images
of space objects, again with extraordinary clarity. Barnell shows images of a
space shuttle orbiting Earth at 5 miles a second. Without Condor processing,
the shuttle image is a blurry black triangle. With Condor processing, it is
sharp and distinct. It?s clear that its payload doors are open.

?This is important because other countries are pursuing space missions, and
we don?t always know their intent,? Barnell said.

The Condor?s third major use is for computational intelligence ? a form of
computer reasoning and decision-making. One example is with words: Condor can
scan or process text in any language at 20 pages a second, fill in missing
sections it has never seen with 99.9 percent accuracy and tell the user
whether the information is important.

?Jobs that used to take hours or days now take seconds,? Barnell said.

Barnell cautions that the Condor is not the ?Holy Grail of computing.?

Rome Lab is sharing Condor access with researchers at other government
agencies, colleges and universities. Among them are Cornell University,
Dartmouth College and the universities of Florida, Maryland and Tennessee.
Researchers at Syracuse University and State University Institute of
Technology at Utica/Rome are also expected to have access, Barnell said.

As impressive as the Condor is, it won?t be for long. Barnell envisions
integrating smartphone processors into high-performance computing, putting
the power of a Condor into a small surveillance drone the size of your fist,
something weighing less than a pound and using the energy of a standard light
bulb.

?In a couple of years, this will fade away,? he said.

Contact Dave Tobin at dtobin at syracuse.com or 470-3277.

_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.



From trainor at presciencetrust.org  Sun Mar 27 21:48:45 2011
From: trainor at presciencetrust.org (Douglas J. Trainor)
Date: Sun, 27 Mar 2011 21:48:45 -0400
Subject: [Beowulf] Quanta crams 512 cores into pizza box server
Message-ID: <8A93BA65-6FBF-4020-9E79-E9D61D16CFB6@presciencetrust.org>

http://www.theregister.co.uk/2011/03/28/quanta_tilera_server_ships/print.html
_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.


From landman at scalableinformatics.com  Mon Mar 28 10:18:26 2011
From: landman at scalableinformatics.com (Joe Landman)
Date: Mon, 28 Mar 2011 10:18:26 -0400
Subject: [Beowulf] Virtualized Infiniband in KVM?
Message-ID: <4D9098B2.2080801@scalableinformatics.com>

Hi folks

   I know about the VT-d bits for virtualized 1 1GbE/10GbE and their 
ability to be leveraged for KVM stacks.  Is there a similar capability 
for IB?  I figured I'd ask here first before bugging the OFED lists. 
This is an HPC situation, and I am trying to see if there is any 
experience with IB in a KVM virtual machine.  Any latency/bandwidth 
measures/tests, or any data for that matter would be of interest.

   I know Xen had some capability with this for a while (virualized IB 
stack as a passthru).  I'd like to avoid Xen if possible, and focus upon 
KVM for a number of reasons.

   Thanks!

   Joe

-- 
Joseph Landman, Ph.D
Founder and CEO
Scalable Informatics Inc.
email: landman at scalableinformatics.com
web  : http://scalableinformatics.com
        http://scalableinformatics.com/sicluster
phone: +1 734 786 8423 x121
fax  : +1 866 888 3112
cell : +1 734 612 4615
_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.


From raq at cttc.upc.edu  Tue Mar 29 04:37:27 2011
From: raq at cttc.upc.edu (Ramiro Alba)
Date: Tue, 29 Mar 2011 10:37:27 +0200
Subject: [Beowulf] AMD 8 cores vs 12 cores CPUs and Infiniband
Message-ID: <1301387847.1995.144.camel@mundo>

Hi all,

We currently have an IB DDR cluster of 128 AMD QuadCore 'Barcelona' (4 *
2 cores) using DDR 'Flextronics' switches ( http://www.ibswitches.com)
with an over-subscription 2:1 (two 24 port switches/rack a one 144 port
modular 'Flextronics' Fat-Tree switch to link). Every IB card in nodes,
is Mellanox DDR InfiniHost III Lx PCI Express x8 (one port) and we use
IB both for calculations (MPI) and storage (Lustre: 80 TB using 1 MDS
and 2 OSSs on a DDN 9900 unit)

Now, we are planning to add new AMD 'Magny-Cours' nodes (16 or 24 cores)
using Infiniband QDR, but linking with the Flextronics 144 port DDR
switch, using 'Hybrid Pasive Copper QSFP to MicroGiGaCN cables', so as
we can reach the Lustre storage.

But there are two main issues that we are worried about:

1 - One port IB cards QP saturation

Using 16 cores per node (8 * 2) seem the 'safe' option, but the 24 cores
(12 * 2) is better in term of price per job. Our CFD applications using
MPI (OpenMPI) may need to do about 15 'MPI_allreduce' calls in one
seccond or less, and we may probably using a pool of 1500 cores. ??Is
anyone having this kind of 'message rate', using AMD 24 cores, and can
tell me about his/her solution/experience?

2 - I've heard that QLogic behavior is better in terms of QP creation, I
have also to think on linking IB DDR with QDR to reach the 'Lustre'
storage. I suppose the main issue is, which QDR switch or switches are
linking with the Flextronics 144 port DDR switch, but I do not know what
is the role of the node card (Mellanox/Qlogic). Again, can anyone tell
me about his/her solution/experience?

Any comment suggestion will be welcomed

Thanks in advance

Regards
-- 
Ramiro Alba

Centre Tecnol??gic de Tranfer??ncia de Calor
http://www.cttc.upc.edu


Escola T??cnica Superior d'Enginyeries
Industrial i Aeron??utica de Terrassa
Colom 11, E-08222, Terrassa, Barcelona, Spain
Tel: (+34) 93 739 86 46



-- 
Aquest missatge ha estat analitzat per MailScanner
a la cerca de virus i d'altres continguts perillosos,
i es considera que est? net.


-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.

-------------- next part --------------
_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

From Shainer at Mellanox.com  Tue Mar 29 18:24:19 2011
From: Shainer at Mellanox.com (Gilad Shainer)
Date: Tue, 29 Mar 2011 15:24:19 -0700
Subject: [Beowulf] AMD 8 cores vs 12 cores CPUs and Infiniband
References: <1301387847.1995.144.camel@mundo>
Message-ID: <9FA59C95FFCBB34EA5E42C1A8573784F037FD082@mtiexch01.mti.com>

I have been using single card on Magny-Cours with no issues at all. You can definitely go with the QDR adapters (latest and greatest). Feel free to contact me directly if you need more info.

On the switch side, switches build according to the spec will auto negotiate to the lower speed, so connecting QDR port to DDR port will cause the QDR port to go down to DDR, but it will work with no issues.

Gilad


-----Original Message-----
From: beowulf-bounces at beowulf.org [mailto:beowulf-bounces at beowulf.org] On Behalf Of Ramiro Alba
Sent: Tuesday, March 29, 2011 1:37 AM
To: beowulf at beowulf.org
Subject: [Beowulf] AMD 8 cores vs 12 cores CPUs and Infiniband

Hi all,

We currently have an IB DDR cluster of 128 AMD QuadCore 'Barcelona' (4 *
2 cores) using DDR 'Flextronics' switches ( http://www.ibswitches.com) with an over-subscription 2:1 (two 24 port switches/rack a one 144 port modular 'Flextronics' Fat-Tree switch to link). Every IB card in nodes, is Mellanox DDR InfiniHost III Lx PCI Express x8 (one port) and we use IB both for calculations (MPI) and storage (Lustre: 80 TB using 1 MDS and 2 OSSs on a DDN 9900 unit)

Now, we are planning to add new AMD 'Magny-Cours' nodes (16 or 24 cores) using Infiniband QDR, but linking with the Flextronics 144 port DDR switch, using 'Hybrid Pasive Copper QSFP to MicroGiGaCN cables', so as we can reach the Lustre storage.

But there are two main issues that we are worried about:

1 - One port IB cards QP saturation

Using 16 cores per node (8 * 2) seem the 'safe' option, but the 24 cores
(12 * 2) is better in term of price per job. Our CFD applications using MPI (OpenMPI) may need to do about 15 'MPI_allreduce' calls in one seccond or less, and we may probably using a pool of 1500 cores. ?Is anyone having this kind of 'message rate', using AMD 24 cores, and can tell me about his/her solution/experience?

2 - I've heard that QLogic behavior is better in terms of QP creation, I have also to think on linking IB DDR with QDR to reach the 'Lustre'
storage. I suppose the main issue is, which QDR switch or switches are linking with the Flextronics 144 port DDR switch, but I do not know what is the role of the node card (Mellanox/Qlogic). Again, can anyone tell me about his/her solution/experience?

Any comment suggestion will be welcomed

Thanks in advance

Regards
--
Ramiro Alba

Centre Tecnol?gic de Tranfer?ncia de Calor http://www.cttc.upc.edu


Escola T?cnica Superior d'Enginyeries
Industrial i Aeron?utica de Terrassa
Colom 11, E-08222, Terrassa, Barcelona, Spain
Tel: (+34) 93 739 86 46



--
Aquest missatge ha estat analitzat per MailScanner a la cerca de virus i d'altres continguts perillosos, i es considera que est net.

_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.



From hahn at mcmaster.ca  Wed Mar 30 00:02:33 2011
From: hahn at mcmaster.ca (Mark Hahn)
Date: Wed, 30 Mar 2011 00:02:33 -0400 (EDT)
Subject: [Beowulf] AMD 8 cores vs 12 cores CPUs and Infiniband
In-Reply-To: <9FA59C95FFCBB34EA5E42C1A8573784F037FD082@mtiexch01.mti.com>
References: <1301387847.1995.144.camel@mundo>
	<9FA59C95FFCBB34EA5E42C1A8573784F037FD082@mtiexch01.mti.com>
Message-ID: <Pine.LNX.4.64.1103292349500.16437@coffee.psychology.mcmaster.ca>

> I have been using single card on Magny-Cours with no issues at all. You can

interesting.  what adjustments have you made to the MPI stack to permit this?
we've had a variety of apps that fail intermittently on high-core nodes.
I have to say I was surprised such a thing came up - not sure whether it's 
inherent to IB or a result of the openmpi stack.  our usual way to test 
this is to gradually reduce the ranks-per-node for the job until it starts
to work.  an interesting cosmology code works at 1 pppn but not 3 ppn
on our recent 12c MC, mellanox QDR cluster.

> Using 16 cores per node (8 * 2) seem the 'safe' option, but the 24 cores
> (12 * 2) is better in term of price per job. Our CFD applications using MPI
> (OpenMPI) may need to do about 15 'MPI_allreduce' calls in one seccond or
> less, and we may probably using a pool of 1500 cores.

but will that allreduce be across 1500 cores?  I can get you a scaling curve
for the previously mentioned MC cluster (2.2 GHz).

> 2 - I've heard that QLogic behavior is better in terms of QP creation, I

well, they've often bragged about message rates - I'm not sure how relate
that is to QP creation.  I'd be interested to hear of some experiences, too.

regards, mark hahn.
_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.


From lindahl at pbm.com  Wed Mar 30 15:08:13 2011
From: lindahl at pbm.com (Greg Lindahl)
Date: Wed, 30 Mar 2011 12:08:13 -0700
Subject: [Beowulf] AMD 8 cores vs 12 cores CPUs and Infiniband
In-Reply-To: <Pine.LNX.4.64.1103292349500.16437@coffee.psychology.mcmaster.ca>
References: <1301387847.1995.144.camel@mundo>
	<9FA59C95FFCBB34EA5E42C1A8573784F037FD082@mtiexch01.mti.com>
	<Pine.LNX.4.64.1103292349500.16437@coffee.psychology.mcmaster.ca>
Message-ID: <20110330190813.GA16949@bx9.net>

On Wed, Mar 30, 2011 at 12:02:33AM -0400, Mark Hahn wrote:

> > 2 - I've heard that QLogic behavior is better in terms of QP creation, I
> 
> well, they've often bragged about message rates - I'm not sure how relate
> that is to QP creation.

They are two separate issues. PSM's equivalent of a QP is much
lighter-weight than the IB standard QP. With QLogic cards, if you are
talking to a lot of different QPs, your performance doesn't degrade
like it does with traditional IB cards.

It's yet another thing which isn't well measured by 2-core
microbenchmarks.

-- greg

(I no longer have a financial interest in QLogic. You should
try out the /hpc slashtag on blekko, though...)
_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.


From Shainer at Mellanox.com  Wed Mar 30 17:51:44 2011
From: Shainer at Mellanox.com (Gilad Shainer)
Date: Wed, 30 Mar 2011 14:51:44 -0700
Subject: [Beowulf] AMD 8 cores vs 12 cores CPUs and Infiniband
References: <1301387847.1995.144.camel@mundo><9FA59C95FFCBB34EA5E42C1A8573784F037FD082@mtiexch01.mti.com><Pine.LNX.4.64.1103292349500.16437@coffee.psychology.mcmaster.ca>
	<20110330190813.GA16949@bx9.net>
Message-ID: <9FA59C95FFCBB34EA5E42C1A8573784F037FD256@mtiexch01.mti.com>

> > well, they've often bragged about message rates - I'm not sure how
> relate
> > that is to QP creation.
> 
> They are two separate issues. PSM's equivalent of a QP is much
> lighter-weight than the IB standard QP. With QLogic cards, if you are
> talking to a lot of different QPs, your performance doesn't degrade
> like it does with traditional IB cards.
> 
> It's yet another thing which isn't well measured by 2-core
> microbenchmarks.


The performance does not degrade with standard IB solutions. 

Gilad
_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.


From orion at cora.nwra.com  Wed Mar 30 18:42:44 2011
From: orion at cora.nwra.com (Orion Poplawski)
Date: Wed, 30 Mar 2011 16:42:44 -0600
Subject: [Beowulf] GP-GPU experience
In-Reply-To: <45254.192.168.93.213.1300711866.squirrel@mail.eadline.org>
References: <68A57CCFD4005646957BD2D18E60667B1292EC74@milexchmb1.mil.tagmclarengroup.com>	<Pine.LNX.4.64.1103032343240.24334@coffee.psychology.mcmaster.ca>	<E1PvvZz-0003ng-00.mikky_m-mail-ru@f121.mail.ru>	<Pine.LNX.4.64.1103072315300.8438@coffee.psychology.mcmaster.ca>	<46317.192.168.93.213.1299685688.squirrel@mail.eadline.org>
	<45254.192.168.93.213.1300711866.squirrel@mail.eadline.org>
Message-ID: <4D93B1E4.3080407@cora.nwra.com>

On 03/21/2011 06:51 AM, Douglas Eadline wrote:
> I got to thinking about how others are fairing (or not)
> with GP-GPU technology. I put up a simple poll on
> ClusterMonkey to help get a general idea.
> (you can find it on the front page right top)
> If you have a moment, please provide
> your experience (results are available as well).

We've seen some reasonable speedup (12x) with some matlab code using Jacket. 
It required up-to-the-minute bugfixes/enhancements from Accelereyes to get it 
working though.  Ran into lots of limitations with some other code (sparse 
matrices) that prevented it from being usable.  Have some reports of success 
with gpulib and IDL.


-- 
Orion Poplawski
Technical Manager                     303-415-9701 x222
NWRA/CoRA Division                    FAX: 303-415-9702
3380 Mitchell Lane                  orion at cora.nwra.com
Boulder, CO 80301              http://www.cora.nwra.com
_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.


From hearnsj at googlemail.com  Thu Mar 31 00:43:07 2011
From: hearnsj at googlemail.com (John Hearns)
Date: Thu, 31 Mar 2011 05:43:07 +0100
Subject: [Beowulf] Job in the UK
Message-ID: <AANLkTimiJVnP5gwFuOAaa80mdj0PM5JMh_ojKuUTCxUJ@mail.gmail.com>

If anyone on the Beowulf list is looking for a job as an HPC engineer,
I spotted an agency advert for an HPC/Linux support engineer.
This is in a place called Banbury, so a quick google for 'HPC engineer
Banbury' will turn up the agencies.
Drop me an email off list if you're interested - it might be worth
looking at the F1 teams in the area.

I hasten to add this has nothing to do with my present employer.

John Hearns
_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.


From samuel at unimelb.edu.au  Thu Mar 31 06:56:25 2011
From: samuel at unimelb.edu.au (Christopher Samuel)
Date: Thu, 31 Mar 2011 21:56:25 +1100
Subject: [Beowulf] Virtualized Infiniband in KVM?
In-Reply-To: <4D9098B2.2080801@scalableinformatics.com>
References: <4D9098B2.2080801@scalableinformatics.com>
Message-ID: <4D945DD9.5010305@unimelb.edu.au>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 29/03/11 01:18, Joe Landman wrote:

> I know about the VT-d bits for virtualized 1 1GbE/10GbE
> and their  ability to be leveraged for KVM stacks.  Is
> there a similar capability for IB?

I believe so, but according to this presentation the
numbers don't look that good:

http://salsahpc.indiana.edu/CloudCom2010/slides/PDF/Recommendations%20for%20Virtualization%20Technologies%20in%20High%20Performance%20Computing.pdf

It was presented on 2nd December 2010 so probably
reasonably up to date although their slides provide
no information at all on which kernel version they
were using for this testing.

More info on the paper here:

http://salsahpc.indiana.edu/CloudCom2010/papers.html#00108

[...]
# This work benchmarks two virtual machine monitors, OpenVZ and KVM,
# specifically focusing on I/O throughput since CPU efficiency has
# been extensively studied [1]. OpenVZ offers near native I/O
# performance. Amazon?s EC2 ?Cluster Compute Node? product is also
# considered for comparative purposes and performs quite well. The
# EC2 ?Cluster Compute Node? product utilizes the Xen hypervisor in
# hvm mode and 10 Gbit/s Ethernet for high throughput communication.
# Therefore, we also briefly studied Xen on our hardware platform
# (in hvm mode) to determine if there are still areas of improvement
# in KVM that allow EC2 to outperform KVM (with InfiniBand host channel
# adapters operating at 20 Gbit/s) in MPI benchmarks. We conclude
# that KVM?s I/O performance is suboptimal, potentially due to memory
# management problems in the hypervisor. Amazon?s EC2 service is
# promising, although further investigation is necessary to understand
# the effects of network based storage on I/O throughput in compute
# nodes. Amazon?s offering may be attractive for users searching for
# "InfiniBand-like" performance without the upfront investment required
# to build an InfiniBand cluster or users wishing to dynamically expand
# their cluster during periods of high demand.

cheers!
Chris
- -- 
    Christopher Samuel - Senior Systems Administrator
 VLSCI - Victorian Life Sciences Computation Initiative
 Email: samuel at unimelb.edu.au Phone: +61 (0)3 903 55545
         http://www.vlsci.unimelb.edu.au/

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk2UXdkACgkQO2KABBYQAh+O+wCeKazajm8BAoVPKFtS3LrxF3BE
SxEAniyVrwxyQTzm+oLy8ny+aPqTilxh
=qByj
-----END PGP SIGNATURE-----
_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.



From samuel at unimelb.edu.au  Thu Mar 31 07:01:26 2011
From: samuel at unimelb.edu.au (Christopher Samuel)
Date: Thu, 31 Mar 2011 22:01:26 +1100
Subject: [Beowulf] Virtualized Infiniband in KVM?
In-Reply-To: <4D945DD9.5010305@unimelb.edu.au>
References: <4D9098B2.2080801@scalableinformatics.com>
	<4D945DD9.5010305@unimelb.edu.au>
Message-ID: <4D945F06.8060301@unimelb.edu.au>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 31/03/11 21:56, Christopher Samuel wrote:

> It was presented on 2nd December 2010 so probably
> reasonably up to date although their slides provide
> no information at all on which kernel version they
> were using for this testing.

Just found the PDF of the paper itself, they tested
Fedora 12 with 2.6.32 for KVM, so there might be
performance improvements in the last 6 kernel releases
since that one came out.

http://www.nd.edu/~nregola/files/cloudcom2010.pdf

cheers!
Chris
- -- 
    Christopher Samuel - Senior Systems Administrator
 VLSCI - Victorian Life Sciences Computation Initiative
 Email: samuel at unimelb.edu.au Phone: +61 (0)3 903 55545
         http://www.vlsci.unimelb.edu.au/

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk2UXwYACgkQO2KABBYQAh+qoACeMaQ4v021F9JwnFm3sm1ZFOOE
uE0AnjQXeioyClBvP9C5LlJrVLG9TiZt
=Dmnj
-----END PGP SIGNATURE-----
_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.


From samuel at unimelb.edu.au  Thu Mar 31 07:12:22 2011
From: samuel at unimelb.edu.au (Christopher Samuel)
Date: Thu, 31 Mar 2011 22:12:22 +1100
Subject: [Beowulf] Execution time measurements (fwd)
In-Reply-To: <Pine.LNX.4.64.1103072315300.8438@coffee.psychology.mcmaster.ca>
References: <68A57CCFD4005646957BD2D18E60667B1292EC74@milexchmb1.mil.tagmclarengroup.com>	<Pine.LNX.4.64.1103032343240.24334@coffee.psychology.mcmaster.ca>	<E1PvvZz-0003ng-00.mikky_m-mail-ru@f121.mail.ru>
	<Pine.LNX.4.64.1103072315300.8438@coffee.psychology.mcmaster.ca>
Message-ID: <4D946196.4090308@unimelb.edu.au>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 08/03/11 16:26, Mark Hahn wrote:

> if the app doesn't control this (or you with numactl),
> then you should expect performance to lie somewhere
> between the two extremes (fully local vs fully remote).
> the kernel does make some effort at keeping things local
> - and for that matter, avoiding moving a process among
> multiple cores/sockets.

It's worth also mentioning the issue of "NUMA diffusion"
through swapping made by David Singleton from ANU on the
hwloc-devel list:

http://www.open-mpi.org/community/lists/hwloc-devel/2011/02/2012.php

# Unless it has changed very recently, Linux swapin_readahead
# is the main culprit in messing with NUMA locality on that
# platform. Faulting a single page causes 8 or 16 or whatever
# contiguous pages to be read from swap. An arbitrary contiguous
# range of pages in swap may not even come from the same process
# far less the same NUMA node. My understanding is that since
# there is no NUMA info with the swap entry, the only policy
# that can be applied to is that of the faulting vma in the
# faulting process. The faulted page will have the desired NUMA
# placement but possibly not the rest. So swapping mixes
# different process' NUMA policies leading to a "NUMA diffusion
# process".

Keep in mind that the reason that ANU runs systems with swap
is so they can suspend jobs, page the entire thing out and
start a new higher priority job.  Running without swap isn't
really an option for them..

cheers,
Chris
- -- 
    Christopher Samuel - Senior Systems Administrator
 VLSCI - Victorian Life Sciences Computation Initiative
 Email: samuel at unimelb.edu.au Phone: +61 (0)3 903 55545
         http://www.vlsci.unimelb.edu.au/

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk2UYZYACgkQO2KABBYQAh/YxwCdFXz5yAdpKqz3G/Sk+AY73E2E
XmMAn0aws1puNktvnaHQbhycul7pMrRp
=jke/
-----END PGP SIGNATURE-----
_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.