[Lustre-discuss] Looping in __d_lookup

[Andreas Dilger]

On Jun 13, 2008  10:16 +0200, Johann Lombardi wrote:
> On Wed, Jun 04, 2008 at 11:25:34AM +0200, Jakob Goldbach wrote:
> > > > > @@ -470,8 +470,8 @@ revalidate_finish:
> > > > >          spin_lock(&dcache_lock);
> > > > >          lock_dentry(de);
> > > > >          __d_drop(de);
> > > > > +        _d_rehash(de);
> > > > >          unlock_dentry(de);
> > > > > -        __d_rehash(de, 0);
> > > > >          spin_unlock(&dcache_lock);
> 
> Actually, _d_rehash() is not exported by the kernel, so I assume that you have
> patched the kernel. At least, that's what I understand from the OpenVZ ticket:
> http://bugzilla.openvz.org/show_bug.cgi?id=895#c27
> 
> Of course, we cannot do this for patchless clients :(

> > Im calling _d_rehash which Andres suggest
> > to be defined in lustre_pathcless_compat.h. Your cut is from kernel
> > pathces anyway. My race happens on _patchless_ client.
> 
> As Yangsheng pointed out, the problem is that d_hash() cannot be used
> by Lustre since the dentry_hashtable symbol is not exported.

Doh!  I looked at this code a couple of times, but didn't think that
d_hash() would be using an internal pointer.  What could be pretty easy
to do is to extract the value for the dentry_hashtable pointer during
the filesystem init and make our own pointer with the same value.

struct hlist_head *ll_dentry_hashtable;

struct hlist_head *ll_d_hash(struct dentry *parent, unsigned long hash)
{
	hash += ((unsigned long) parent ^ GOLDEN_RATIO_PRIME) / L1_CACHE_BYTES;
	hash = hash ^ ((hash ^ GOLDEN_RATIO_PRIME) >> D_HASHBITS);
	return ll_dentry_hashtable + (hash & D_HASHMASK);
}

static inline void ll_d_rehash(struct dentry *dentry)
{
	dentry->d_flags &= ~DCACHE_UNHASHED;
	hlist_add_head_rcu(&dentry->d_hash, ll_d_hash(dentry->d_parent,
						      dentry->d_name.hash));
}


static int init_lustre_lite()
{
	struct dentry dummy;
	unsigned int dummy_hash;
	struct hlist_node *pprev = NULL, *pprev2;

	/* The first time ll_d_hash() is called, ll_dentry_hashtable == NULL,
	 * so it returns only the hash table bucket offset.  We use this to
	 * figure out the un-exported dentry_hashtable pointer.  This might
	 * be racy (we aren't guaranteed to still be the first entry on the
	 * hash chain, so it should be called a few times to verify. */
	memset(&dummy, 0, sizeof(dummy));
	spin_lock_init(&dummy.d_lock);
	dummy_hash = ll_d_hash(&dummy, 0);

	while (pprev != pprev2) {
		d_rehash(&dummy);
		pprev2 = pprev;
		pprev = dummy.d_hash.pprev
		hlist_del_rcu(&dummy.d_hash);
	}

	ll_dentry_hashtable = pprev - dummy_hash;




Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.