[Lustre-devel] changelog for whole filesystem?

[Robert Read]

It seems simpler to do the iteration on  a snapshot, instead of a live filesystem, and allow post-snapshot changes to accumulate on the regular changelog for processing once the "from empty" iteration was complete.  

robert

On Nov 11, 2010, at 10:10 AM, Eric Barton wrote:

> Nathan wrote...
> 
>> This same "initial population of a database from objects" problem
>> occurs when trying to replicate a Lustre filesystem using the
>> changelog.
> 
> That must be a great test case.  If we can reliably reconstruct an
> exact replica using a "from empty" changelog, we must have got it
> right :)
> 
>> The problem is actually more complicated even for a single changelog
>> consumer: since iterating through the virtual changelog takes non-0
>> time, you're not sure if a virtual record was created before or
>> after an actual changelog entry.  E.g. you might try to resolve the
>> name of a file for a virtual record either before or after a rename,
>> and then later you would see the rename in the changelog, leading to
>> an inconsistent view of the namespace.
>> 
>> If you don't care about having the exact right name, then it's easy
>> enough to ignore inapplicable changelog records.
> 
> I was trying to hint at the need to filter changes for individual
> consumers depending on how far through the initial "from empty"
> iteration they are in my original post.  Andreas stated it more
> explicitly.  I think you just need to enumerate the cases - e.g. for
> rename...
> 
> Iterated over yet?               Record
> Source Inode   Target Inode      emitted
> no             no                none
> no             yes               delete target
> yes            yes               rename source + delete target
> yes            no                rename source
> 
> The final case just looks like a rename where the target name didn't
> exist already.  In any case, the only real requirement on the stream
> of changelog records constructed in the initial iteration is that
> consistent filesystem state be reconstructed after the last record
> is consumed.
> 
> -- 
> 
>                Cheers,
>                        Eric
> 
> Eric Barton
> CTO Whamcloud, Inc.
> Tel: +44 117 330 1575
> Mob: +44 7920 797 273
> 
> _______________________________________________
> Lustre-devel mailing list
> Lustre-devel at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-devel