[Beowulf] Re: dedupe Filesystem

Kilian CAVALOTTI kilian.cavalotti.work at gmail.com
Fri Jun 5 08:39:24 EDT 2009

On Wednesday 03 June 2009 14:24:19 Lawrence Stewart wrote:
> I probably don't need to remind anyone here that deduplication on a
> live filesystem (as opposed to backups) can have really bad
> performance effects.  Imagine if you have to move the disk arms around
> for every file for every block of every file.  Modern filesystems do
> well at keeping files contiguous and often keep all the files of a
> directory nearly.  This locality gets trashed by deduplicaiton.  This
> won't matter if the problem is making backups smaller or making
> incrementals run faster, but it is not good for the performance of a
> live filesystem.

It's kind of new to me, but it looks like some vendors have block-level 
deduplication systems integrated in some of their product lines.
See http://www.theregister.co.uk/2009/06/05/dell_block_dedupe/ or 

> -Larry/thinking about what to do next


Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

More information about the Beowulf mailing list