<br><br><div class="gmail_quote">On Wed, Feb 9, 2011 at 2:37 PM, Prentice Bisbal <span dir="ltr"><<a href="mailto:prentice@ias.edu">prentice@ias.edu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
Craig Tierney wrote:<br>
> On 2/9/11 7:38 AM, Prentice Bisbal wrote:<br>
>> I don't think Numascale/ScaleMP has much of a cost advantage anymore.<br>
>><br>
>> About 6 months ago, I purchased a couple of Dell PowerEdge R815s with<br>
>> 128 GB of RAM and 32 cores. We looked at similar RAM configurations a<br>
>> couple of years ago. and the cost premium for that much RAM was<br>
>> prohibitive (the price for additional RAM seemed to go up exponentially<br>
>> with the amount of RAM) so we stayed at 32 GB. This time around, the<br>
>> premium for the additional RAM seemed marginal. Not sure how large you<br>
>> can go and keep that "marginal" relationship, but it's much larger than<br>
>> it was a couple of years ago.<br>
>><br>
><br>
> There is only so much memory you can put in a box, regardless of the<br>
> cost. ScaleMP/Numascale let you get around that issue. What if I need<br>
> a TB of RAM? Yes, I might be able to convert my algorithms, but if I can<br>
> add $3k per node to get cache-coherent memory, why not go that route?<br>
<br>
>Agreed. I was going to add the caveat that there are limits to how much<br>
>RAM you can add to a single box, or there eventually be a point the<br>
>amount of RAM in a single box vs. price is no longer nonlinear, but then<br>
>I got lazy. ;)<br></blockquote><div><br><br>Don't understand why you need TB of RAM? Don't forget the fact that $3K can almost buy one-node!!! <br><br><br></div><blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
<br>
><br>
><br>
>> And don't forget to figure in the cost of the interconnect. I don't know<br>
>> what Numascale requires, but ScaleMP require IB, which can add<br>
>> significant costs if you don't have an existing IB fabric to use.<br>
>><br>
><br>
> ScaleMP is a software solution, Numascale is a cache-coherent hardware solution.<br>
> The list cost of the 3D torus version Numascale card with 4GB Ram (highest end card).<br>
> Cables are similar to IB, about $120 each.<br>
><br>
> It would be really cool to build SMP system to sit next to my cluster. When users<br>
> need large memory, or want to do things with UPC or other similar languages,<br>
> an SMP system like this would be handy.<br>
><br>
> - I am not advocating for Numascale, or any other products here. I am just<br>
> trying to provide information to the community.<br>
><br>
> Craig<br>
><br>
><br>
><br>
>> Prentice<br>
>><br>
>><br>
>> Douglas Eadline wrote:<br>
>>> Although I wrote about the SMP and the Numascale hardware,<br>
>>> I have not yet had a chance to use it.<br>
>>><br>
>>> To me there are two issues worth considering. First,<br>
>>> if you need a big SMP this might be a low<br>
>>> cost solution. Second, should your application<br>
>>> not scale best in a node-based ccNuma system (ScaleMP<br>
>>> or Numascale), MPI is still an option. Indeed, no need<br>
>>> to rewrite the codes. And, management is probably<br>
>>> much easier.<br>
>>><br>
>>> Of course for large systems, clusters work best.<br>
>>><br>
>>> --<br>
>>> Doug<br>
>>><br>
>>><br>
>>>> On Thu, 2011-02-03 at 09:45 +0000, Hearns, John wrote:<br>
>>>>> Also look at ScaleMP and Numascale<br>
>>>>> Here's a damn good article from Doug Eadline:<br>
>>>>> <a href="http://www.linux-mag.com/id/7947" target="_blank">http://www.linux-mag.com/id/7947</a><br>
>>>>><br>
>>>>> I must admit though I don't know how far a budget of 30K takes you<br>
>>>>> there!<br>
>>>> My understanding is that the Numascale cards cost "a bit more" than IB<br>
>>>> cards but you don't need a switch (it's a 3D taurus I think) so it would<br>
>>>> be worth looking into.<br>
>>>><br>
>>>> Rob<br>
>>>><br>
>> _______________________________________________<br>
>> Beowulf mailing list, <a href="mailto:Beowulf@beowulf.org">Beowulf@beowulf.org</a> sponsored by Penguin Computing<br>
>> To change your subscription (digest mode or unsubscribe) visit <a href="http://www.beowulf.org/mailman/listinfo/beowulf" target="_blank">http://www.beowulf.org/mailman/listinfo/beowulf</a><br>
> _______________________________________________<br>
> Beowulf mailing list, <a href="mailto:Beowulf@beowulf.org">Beowulf@beowulf.org</a> sponsored by Penguin Computing<br>
> To change your subscription (digest mode or unsubscribe) visit <a href="http://www.beowulf.org/mailman/listinfo/beowulf" target="_blank">http://www.beowulf.org/mailman/listinfo/beowulf</a><br>
><br>
<br>
--<br>
<font color="#888888">Prentice Bisbal<br>
Linux Software Support Specialist/System Administrator<br>
School of Natural Sciences<br>
Institute for Advanced Study<br>
Princeton, NJ<br>
_______________________________________________<br>
Beowulf mailing list, <a href="mailto:Beowulf@beowulf.org">Beowulf@beowulf.org</a> sponsored by Penguin Computing<br>
To change your subscription (digest mode or unsubscribe) visit <a href="http://www.beowulf.org/mailman/listinfo/beowulf" target="_blank">http://www.beowulf.org/mailman/listinfo/beowulf</a><br>
</font></blockquote></div><br>
<br />--
<br />This message has been scanned for viruses and
<br />dangerous content by
<a href="http://www.mailscanner.info/"><b>MailScanner</b></a>, and is
<br />believed to be clean.