[Beowulf] dedupe filesystem

Rahul Nabar rpnabar at gmail.com
Fri Jun 26 12:14:55 EDT 2009

On Tue, Jun 2, 2009 at 11:56 PM, Matt Lawrence<matt at technoronin.com> wrote:
> I have found a great deaal of duplication in install trees, particularly
> when you just want to install the latest.  I've managed to get some massive
> savings with NIM on AIX and some lesser but stll very good savings with
> CentOS by building parallel trees and hard linking the files.

I'm jumping on this discussion late but I found a great number (but
not so much in size I suspect) of dupes when I ran fdupes on my drives
now. Some in quite unexpected places. eg. we have job submission shell
wrappers for PBS/torque. The number of dupes indicates that often
people are not removing these files at all after the job finishes. So,
more than an automated-dedup solution fdupes has helped me know what
are the filesystem cleanup opportunities I have!

The next time someone comes running for a disk quota increase I know
what to check! ;-)


Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

More information about the Beowulf mailing list