[Lustre-devel] changelog for whole filesystem?
eeb at whamcloud.com
Thu Nov 11 10:10:32 PST 2010
> This same "initial population of a database from objects" problem
> occurs when trying to replicate a Lustre filesystem using the
That must be a great test case. If we can reliably reconstruct an
exact replica using a "from empty" changelog, we must have got it
> The problem is actually more complicated even for a single changelog
> consumer: since iterating through the virtual changelog takes non-0
> time, you're not sure if a virtual record was created before or
> after an actual changelog entry. E.g. you might try to resolve the
> name of a file for a virtual record either before or after a rename,
> and then later you would see the rename in the changelog, leading to
> an inconsistent view of the namespace.
> If you don't care about having the exact right name, then it's easy
> enough to ignore inapplicable changelog records.
I was trying to hint at the need to filter changes for individual
consumers depending on how far through the initial "from empty"
iteration they are in my original post. Andreas stated it more
explicitly. I think you just need to enumerate the cases - e.g. for
Iterated over yet? Record
Source Inode Target Inode emitted
no no none
no yes delete target
yes yes rename source + delete target
yes no rename source
The final case just looks like a rename where the target name didn't
exist already. In any case, the only real requirement on the stream
of changelog records constructed in the initial iteration is that
consistent filesystem state be reconstructed after the last record
CTO Whamcloud, Inc.
Tel: +44 117 330 1575
Mob: +44 7920 797 273
More information about the lustre-devel