<html><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><br><div><div>On Sep 10, 2009, at 10:44 AM, Rahul Nabar wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><div>On Wed, Sep 9, 2009 at 3:38 PM, Greg Keller <<a href="mailto:Greg@keller.net">Greg@keller.net</a>> wrote:<br><blockquote type="cite">For example, Lot's of nodes reading and writing<br></blockquote><blockquote type="cite">different files in a generically staggered fashion,<br></blockquote><br>How do you enforce the staggering? Do people write staggered I/O codes<br>themselves? Or can on alliviate this problem by scheduler settings?<br></div></blockquote><div>Although there's probably a way to enforce it at the app level, or scheduler, all of that would require specific knowledge of what jobs (and nodes) are accessing what files how at what time. I was thinking that if it's largely embarassingly parallel jobs that start/stop independently and have somewhat randomized IO, then there is some natural staggering. If the app starts on all nodes simultanously and then they all start reading/writing the same files nearly simultaneously, then staggering is probably impossible and a parallel FS is worth investigating.</div><br><blockquote type="cite"><div><br><blockquote type="cite">Luster or eventually pNFS if things get ugly. But not all NFS servers are<br></blockquote><blockquote type="cite">created equal, and a solid purpose built appliance may handle loads a<br></blockquote><blockquote type="cite">general purpose linux NFS server won't.<br></blockquote><br>Disk array connected to generic Linux server? Or standalone<br>Fileserver? Reccomendations?<br></div></blockquote><blockquote type="cite"><div><font class="Apple-style-span" color="#000000"><br></font>What exactly does a "solid purpose built appliance" offer that a<br>Generic Linux server (well configured) connected to an array of disks<br>does not offer?<br></div></blockquote><div><div>Joe's post is spot on here. Don't let legend and lore scare you off, NFS can do great things on current generic and special purpose servers with the right config and software. There's nothing in your configuration and usage summary that screams NFS killer to me. If you use generic or special purpose servers, you can repurpose them as part of a parallel FS if you need to.</div><div><br></div>Purpose built *appliances* generally give you:</div><div>Simple setup and admin GUI</div><div>Replication and other fancy features HPCC doesn't normally care about</div><div>Zero flexibility if you change course and head towards a parallel FS.</div><div>A singular support channel to complain to if things go badly (YMMV)</div><div><br></div><div>None of those matter to me more than the money they cost, so I buy standard servers and run standard linux NFS on internal raid controllers with no HA, and have occasional crashes and issues I can't resolve cleanly. We are perpetually looking for a "next step" to get better support/stability, but it's good enough for our 300 and 600 node systems at the moment.</div><div><br></div><blockquote type="cite"><div><font class="Apple-style-span" color="#000000"><br></font>-- <br>Rahul<br></div></blockquote></div><div><br></div>Cheers!<div>Greg</div></body></html>