[Lustre-devel] Wide area use of Lustre and client caches

[Andreas Dilger]

On May 09, 2008  09:08 -0600, Peter J. Braam wrote:
> Nono - striping should only be used to get more bandwidth from servers.  The
> correct solution to the problem you point out is a lock conversion, planned
> long ago, still far away maybe (Nikita?).

This task is something that Oleg has been working on occasionally,
I think there are patches around but fairly old (though not as bad
as might be expected, because LDLM code changes relatively slowly).

> On 5/9/08 8:25 AM, "Brian J. Murrell" <Brian.Murrell at Sun.COM> wrote:
> > Even still it's useful during implementation to think of the use case of
> > that remote client having read a file and caching and holding a read
> > lock on that file, say 1GB in size, and then another client wanting to
> > update say, 1KB in the middle of the file.  It would be beneficial for
> > that 1GB file to have a small (but still practical) stripe size so that
> > the amount of cache that needs to be thrown away to accommodate the
> > write is relatively small.

Having multiple stripes would save invalidation of (nstripes - 1) / nstripes
of the file, but in general the "update in the middle" paradigm is very
rare in real life, so in practise I don't think this will help much.

Even the "add a byte in the middle of a text file" case always causes
the whole file to be rewritten because of backing up the old file.

The only common applications I'm aware of that do partial-file read/write
operations are databases and peer-to-peer file sharing.

Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.