[Lustre-discuss] [Lustre-devel] Integrity and corruption - can file systems be scalable?

[Nicolas Williams]

On Sun, Jul 04, 2010 at 11:53:29PM -0400, Dmitry Zogin wrote:
> What I really mean is the defragmentation issue and not the
> fragmentation itself. All file systems becomes fragmented, as it is
> unavoidable. But the defragmentation of the file system using hash
> trees really becomes a problem.

That is emphatically not true.

To defragment a ZFS-like filesystem all you need to do is traverse the
metadata looking for live blocks from old transaction groups, then
relocate those by writing them out again almost as if an application had
written to them (except with no mtime updates).

In ZFS we call this block pointer rewrite, or bp rewrite.

> >Everything we do involves trade-offs.
> >
> Yes, but if the performance drop becomes unacceptable, any gain in
> the integrity is miserable.

I believe ZFS has shown that unacceptable performance losses are not
required in order to get the additional integrity protection.

Nico
--