ajt at rri.sari.ac.uk
Wed Sep 24 06:49:46 EDT 2003
Eric R Johnson wrote:
> as possible on a system dedicated to this task. Now that you have a
> little more info, do you still recommend tmpfs? If so, is there a
> specific performance issue that leads us in that direction, because the
> search time is all that matters in this case.
On the basis of what you've told us, I think you would be better off
avoiding the filesystem overhead altogether and read your database
directly into memory within a program. Accessing RAM directly by memory
references in a program is as fast as you can get. Anything else you do
will incur an OS filesystem call overhead. This may not be practical, of
course, but it would give you the fastest solution. One-step (hash)
lookups of whatever your database contains directly from RAM is fastest.
My basic point about ramdisk/tmpfs is that I think you would be better
off allowing the memory manager to keep your most recently accessed data
in memory for you. However, if your entire job, including the database,
fits into physical memory (RAM) then no swapping to disk will occur with
tmpfs. I think you really should be looking for a solution that gives
you the best performance improvement for the least effort - i.e. tmpfs.
Forgive me if I'm teaching you to suck eggs, but if search time is all
that matters then your search algorithm is important. There is little
point accessing your database as fast as possible using a poor search
algorithm: Indeed, you may find that your concern about 'fast' access to
your data on ramdisk/tmpfs is not the main bottleneck after all. You
could benchmark the database problem on harddisk and tmpfs to find out.
Dr. A.J.Travis, | mailto:ajt at rri.sari.ac.uk
Rowett Research Institute, | http://www.rri.sari.ac.uk/~ajt
Greenburn Road, Bucksburn, | phone:+44 (0)1224 712751
Aberdeen AB21 9SB, Scotland, UK. | fax:+44 (0)1224 716687
Beowulf mailing list, Beowulf at beowulf.org
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
More information about the Beowulf