[Lustre-devel] Replication/server logs
Peter.Braam at Sun.COM
Thu Jun 5 20:36:34 PDT 2008
Added Lustre devel
Jeff will schedule regular calls with us, just spoke with him yesterday.
On 6/5/08 9:54 AM, "Nathaniel Rutman" <Nathan.Rutman at Sun.COM> wrote:
> So far there are three problems with ZFS changelogs that I have come across:
> 1. potentially slow path lookup. There is an existing function
> zfs_obj_to_path that does the job by searching for the object number in
> the parent directory, so O(dir_entries) for each ancestor. Supposedly
> Jeff Bonwick has some ideas on this besides our two old ideas (obj,name
> database and storing name in an EA), but I have heard nothing from him
> and not much from the rest of the ZFS team. Mitigation: we could just
> ignore this problem for now and consider it an optimization.
Yes, but the optimization is a must have.
> 2. hardlinks - a single parent object is stored with each object (the
> last hardlink). But if that parent is unlinked, then an object may be
> orphaned with no valid parent (until a directory lookup of one of the
> other links.) So we would (eventually) need a solution dealing with
> multiple hardlinks; this should be addressed in the solution for #1
> above. Mitigation: we may choose not to consider this a critical use
> case in the initial implementation.
Indeed, ignore for now, but solve in #1.
> 3. directory content deltas to file names - the only change information
> we have is modified blocks, not a list of modified files. Files that
> have been modified are directly/easily identified by object number, but
> renames for instance can only be detected by a careful comparison
> between an old raw block from a directory and the new version of that
> block. And the directory structure can be quite complex (different
> versions are used depending on file name length, for instance). Really
> we should use the native directory parsing tools, but they require the
> user to start reading from the beginning and the current position in the
> directory is opaque, so there's no easy way to find just the objects
> associated with the changed directory block. So in the worst case, for
> renames (and maybe creates/unlinks), we would scan and compare the
> entire (top level) contents of the old and new directories.
We need new tools. Zapdiff should become a general function is what Bill
Moore said a few weeks ago.
More information about the lustre-devel