[Lustre-devel] changelog for whole filesystem?
rread at bitsoflight.com
Fri Nov 12 15:41:48 PST 2010
It seems simpler to do the iteration on a snapshot, instead of a live filesystem, and allow post-snapshot changes to accumulate on the regular changelog for processing once the "from empty" iteration was complete.
On Nov 11, 2010, at 10:10 AM, Eric Barton wrote:
> Nathan wrote...
>> This same "initial population of a database from objects" problem
>> occurs when trying to replicate a Lustre filesystem using the
> That must be a great test case. If we can reliably reconstruct an
> exact replica using a "from empty" changelog, we must have got it
> right :)
>> The problem is actually more complicated even for a single changelog
>> consumer: since iterating through the virtual changelog takes non-0
>> time, you're not sure if a virtual record was created before or
>> after an actual changelog entry. E.g. you might try to resolve the
>> name of a file for a virtual record either before or after a rename,
>> and then later you would see the rename in the changelog, leading to
>> an inconsistent view of the namespace.
>> If you don't care about having the exact right name, then it's easy
>> enough to ignore inapplicable changelog records.
> I was trying to hint at the need to filter changes for individual
> consumers depending on how far through the initial "from empty"
> iteration they are in my original post. Andreas stated it more
> explicitly. I think you just need to enumerate the cases - e.g. for
> Iterated over yet? Record
> Source Inode Target Inode emitted
> no no none
> no yes delete target
> yes yes rename source + delete target
> yes no rename source
> The final case just looks like a rename where the target name didn't
> exist already. In any case, the only real requirement on the stream
> of changelog records constructed in the initial iteration is that
> consistent filesystem state be reconstructed after the last record
> is consumed.
> Eric Barton
> CTO Whamcloud, Inc.
> Tel: +44 117 330 1575
> Mob: +44 7920 797 273
> Lustre-devel mailing list
> Lustre-devel at lists.lustre.org
More information about the lustre-devel