From neilb at suse.com  Fri Jun  1 05:38:15 2018
From: neilb at suse.com (NeilBrown)
Date: Fri, 01 Jun 2018 15:38:15 +1000
Subject: [lustre-devel] [PATCH v2 6/6] staging: lustre: mdc: use large
 xattr buffers for old servers
In-Reply-To: <AE3B1D00-46A8-42FA-9025-2E680C185294@intel.com>
References: <1527603705-30450-1-git-send-email-jsimmons@infradead.org>
 <1527603705-30450-7-git-send-email-jsimmons@infradead.org>
 <20180531165426.GA23414@kroah.com>
 <AE3B1D00-46A8-42FA-9025-2E680C185294@intel.com>
Message-ID: <87po1brrqw.fsf@notabene.neil.brown.name>

On Thu, May 31 2018, Dilger, Andreas wrote:

> On May 31, 2018, at 18:54, Greg Kroah-Hartman <gregkh at linuxfoundation.org> wrote:
>> 
>> On Tue, May 29, 2018 at 10:21:45AM -0400, James Simmons wrote:
>>> From: "John L. Hammond" <john.hammond at intel.com>
>>> 
>>> Pre 2.10.1 MDTs will crash when they receive a listxattr (MDS_GETXATTR
>>> with OBD_MD_FLXATTRLS) RPC for an orphan or dead object. So for
>>> clients connected to these older MDTs, try to avoid sending listxattr
>>> RPCs by making the bulk getxattr (MDS_GETXATTR with OBD_MD_FLXATTRALL)
>>> more likely to succeed and thereby reducing the chances of falling
>>> back to listxattr.
>>> 
>>> +#if LUSTRE_VERSION_CODE < OBD_OCD_VERSION(3, 0, 53, 0)
>> 
>> Why are you adding pointless version checks to mainline?  Please don't
>> add new ones of these, you need to be working on removing the existing
>> ones.
>
> These are not Linux kernel version checks, but rather Lustre release version
> checks.  This allows us to remove workarounds like this in the future when
> they are no longer needed, rather than accumulating cruft forever.  It's like
> the separation of NFSv2 vs NFSv3 vs NFSv4.

It looks very different to the separation of NFSv{2,3,4}.  Those are
conditionally compiled on a whole-file basis.
If we ever want to remove this code it should be hard to search for
occurances of OBD_OCD_VERSION() in the code, we don't need the C
preprocessor to be able to remove them for us.
In this particular example:

+	if (exp->exp_connect_data.ocd_version < OBD_OCD_VERSION(2, 10, 1, 0))
+		min_buf_size = exp->exp_connect_data.ocd_max_easize;

if you want to be able to compile without that one test, you could arrange
that OBD_OSC_VERSION(2, 10, 1, 0) evaluates to 0.
As ocd_version is unsigned, the comparison will always be false, and
the compiler will optimize the code away.

As a general rule, you need a very good reason to have #if or #ifdef in
.c files.  They are usually OK in .h files.

Thanks,
NeilBrown

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180601/78892757/attachment.sig>

From gregkh at linuxfoundation.org  Fri Jun  1 08:12:11 2018
From: gregkh at linuxfoundation.org (Greg Kroah-Hartman)
Date: Fri, 1 Jun 2018 10:12:11 +0200
Subject: [lustre-devel] [PATCH v2 6/6] staging: lustre: mdc: use large
 xattr buffers for old servers
In-Reply-To: <AE3B1D00-46A8-42FA-9025-2E680C185294@intel.com>
References: <1527603705-30450-1-git-send-email-jsimmons@infradead.org>
 <1527603705-30450-7-git-send-email-jsimmons@infradead.org>
 <20180531165426.GA23414@kroah.com>
 <AE3B1D00-46A8-42FA-9025-2E680C185294@intel.com>
Message-ID: <20180601081211.GD12809@kroah.com>

On Thu, May 31, 2018 at 05:30:24PM +0000, Dilger, Andreas wrote:
> On May 31, 2018, at 18:54, Greg Kroah-Hartman <gregkh at linuxfoundation.org> wrote:
> > 
> > On Tue, May 29, 2018 at 10:21:45AM -0400, James Simmons wrote:
> >> From: "John L. Hammond" <john.hammond at intel.com>
> >> 
> >> Pre 2.10.1 MDTs will crash when they receive a listxattr (MDS_GETXATTR
> >> with OBD_MD_FLXATTRLS) RPC for an orphan or dead object. So for
> >> clients connected to these older MDTs, try to avoid sending listxattr
> >> RPCs by making the bulk getxattr (MDS_GETXATTR with OBD_MD_FLXATTRALL)
> >> more likely to succeed and thereby reducing the chances of falling
> >> back to listxattr.
> >> 
> >> +#if LUSTRE_VERSION_CODE < OBD_OCD_VERSION(3, 0, 53, 0)
> > 
> > Why are you adding pointless version checks to mainline?  Please don't
> > add new ones of these, you need to be working on removing the existing
> > ones.
> 
> These are not Linux kernel version checks, but rather Lustre release version
> checks.  This allows us to remove workarounds like this in the future when
> they are no longer needed, rather than accumulating cruft forever.  It's like
> the separation of NFSv2 vs NFSv3 vs NFSv4.

As Neil said, this is not like NFS at all.  Those are well-defined
Kconfig options that are used to compile whole new files and
include/exclude different functions entirely.

It is not used to check in the middle of code flow if something should,
or should not, happen.

As Neil also said, if you really have to do this, put it behind
functions that you properly define in a .h file, do not put #if code in
a .c file if at all possible (yeah, nfs does not do this everywhere, but
it is much better than the random checks you all have today in the
lustre code.)

So as-is, I am not taking this patch, sorry, it needs to be reworked.

greg k-h

From gregkh at linuxfoundation.org  Fri Jun  1 08:28:36 2018
From: gregkh at linuxfoundation.org (Greg Kroah-Hartman)
Date: Fri, 1 Jun 2018 10:28:36 +0200
Subject: [lustre-devel] [PATCH v2 00/25] staging: lustre: libcfs: SMP
 rework
In-Reply-To: <1527603725-30560-1-git-send-email-jsimmons@infradead.org>
References: <1527603725-30560-1-git-send-email-jsimmons@infradead.org>
Message-ID: <20180601082836.GA19242@kroah.com>

On Tue, May 29, 2018 at 10:21:40AM -0400, James Simmons wrote:
> From: James Simmons <uja.ornl at yahoo.com>
> 
> Recently lustre support has been expanded to extreme machines with as
> many as a 1000+ cores. On the other end lustre also has been ported
> to platforms like ARM and KNL which have uniquie NUMA and core setup.
> For example some devices exist that have NUMA nodes with no cores.
> With these new platforms the limitations of the Lustre's SMP code
> came to light so a lot of work was needed. This resulted in this
> patch set which has been tested on these platforms.

That's great work, but why is this happening instead of effort to get
this out of the staging tree?  I see a mix of "clean this up" combined
with "add this new feature" happening here, and I am getting tired of
it.

I keep saying, "no new features until this gets out of staging", so why
isn't anyone working directly on getting this out of staging?  I can
only assume the reason why is because I keep being "nice" and accepting
random new feature additions :(

So, no more, I'm really really really tired of dealing with this
filesystem for all of these years.  There is still loads to do to get
this cleaned up and moved out (as my simple debugfs cleanup patch series
showed).  So please do it.

Again, I am not going to take any more new feature additions or anything
that does not obviously look like an attempt to get this code cleaned up
into mergable shape.  Adding things like "now works on systems with
thousands of CPUs as well as an RPi" is not such work, unless you can
directly show me an end result of cleaner, smaller, and more easily
reviewable code.

This patch series is now dropped, if you think it meets the above
criteria, please feel free to resend it to be judged in this manner.

thanks,

greg k-h

From gregkh at linuxfoundation.org  Fri Jun  1 08:29:57 2018
From: gregkh at linuxfoundation.org (Greg Kroah-Hartman)
Date: Fri, 1 Jun 2018 10:29:57 +0200
Subject: [lustre-devel] [PATCH v2 v2 07/25] staging: lustre: libcfs:
 replace num_possible_cpus() with nr_cpu_ids
In-Reply-To: <1527603725-30560-8-git-send-email-jsimmons@infradead.org>
References: <1527603725-30560-1-git-send-email-jsimmons@infradead.org>
 <1527603725-30560-8-git-send-email-jsimmons@infradead.org>
Message-ID: <20180601082957.GB19242@kroah.com>

On Tue, May 29, 2018 at 10:21:47AM -0400, James Simmons wrote:
> From: Amir Shehata <amir.shehata at intel.com>
> 
> Move from num_possible_cpus() to nr_cpu_ids.

Nit, when doing a "v2" of a patch, in a series that is not all "v2", put
the moniker after the patch number.  Otherwise it does not sort properly
in anyone's email client and I have to manually edit it in order to get
it to apply in the correct order.

Remember, make things impossible for me to reject.  This is trivial for
me to reject on that issue alone, given that I deal with hundreds of
staging patches a week...

thanks,

greg k-h

From gregkh at linuxfoundation.org  Fri Jun  1 08:31:40 2018
From: gregkh at linuxfoundation.org (Greg Kroah-Hartman)
Date: Fri, 1 Jun 2018 10:31:40 +0200
Subject: [lustre-devel] [PATCH v2 v2 07/25] staging: lustre: libcfs:
 replace num_possible_cpus() with nr_cpu_ids
In-Reply-To: <1527603725-30560-8-git-send-email-jsimmons@infradead.org>
References: <1527603725-30560-1-git-send-email-jsimmons@infradead.org>
 <1527603725-30560-8-git-send-email-jsimmons@infradead.org>
Message-ID: <20180601083140.GC19242@kroah.com>

On Tue, May 29, 2018 at 10:21:47AM -0400, James Simmons wrote:
> From: Amir Shehata <amir.shehata at intel.com>
> 
> Move from num_possible_cpus() to nr_cpu_ids.

This says what you did, not _why_ you did it.  If these functions are
really identical, why do we have both of them?  :)

greg k-h

From gregkh at linuxfoundation.org  Fri Jun  1 08:33:59 2018
From: gregkh at linuxfoundation.org (Greg Kroah-Hartman)
Date: Fri, 1 Jun 2018 10:33:59 +0200
Subject: [lustre-devel] [PATCH v2 18/25] staging: lustre: libcfs: rename
 goto label in cfs_cpt_table_print
In-Reply-To: <1527603725-30560-19-git-send-email-jsimmons@infradead.org>
References: <1527603725-30560-1-git-send-email-jsimmons@infradead.org>
 <1527603725-30560-19-git-send-email-jsimmons@infradead.org>
Message-ID: <20180601083359.GD19242@kroah.com>

On Tue, May 29, 2018 at 10:21:58AM -0400, James Simmons wrote:
> From: Dmitry Eremin <dmitry.eremin at intel.com>
> 
> Change goto label out to err.

Again, this says what it does, not why.


From gregkh at linuxfoundation.org  Fri Jun  1 08:37:54 2018
From: gregkh at linuxfoundation.org (Greg Kroah-Hartman)
Date: Fri, 1 Jun 2018 10:37:54 +0200
Subject: [lustre-devel] [PATCH v2 08/25] staging: lustre: libcfs: NUMA
 support
In-Reply-To: <1527603725-30560-9-git-send-email-jsimmons@infradead.org>
References: <1527603725-30560-1-git-send-email-jsimmons@infradead.org>
 <1527603725-30560-9-git-send-email-jsimmons@infradead.org>
Message-ID: <20180601083754.GE19242@kroah.com>

On Tue, May 29, 2018 at 10:21:48AM -0400, James Simmons wrote:
> From: Amir Shehata <amir.shehata at intel.com>
> 
> This patch adds NUMA node support.

Really?  It looks like you just added an empty data structure pointer
that doesn't really do anything at all.

Where are you reading the host memory NUMA information from?

And why would a filesystem care about this type of thing?  Are you
going to now mirror what the scheduler does with regards to NUMA
topology issues?  How are you going to handle things when the topology
changes?  What systems did you test this on?  What performance
improvements were seen?  What downsides are there with all of this?

I need a whole lot more information here...

> NUMA node information is stored
> in the CPT table. A NUMA node mask is maintained for the entire
> table as well as for each CPT to track the NUMA nodes related to
> each of the CPTs. Add new function cfs_cpt_of_node() which returns
> the CPT of a particular NUMA node.

This doesn't really seem to match up with the code changes from what I
can tell...

thanks,

greg k-h

From gregkh at linuxfoundation.org  Fri Jun  1 08:39:32 2018
From: gregkh at linuxfoundation.org (Greg Kroah-Hartman)
Date: Fri, 1 Jun 2018 10:39:32 +0200
Subject: [lustre-devel] [PATCH v2 13/25] staging: lustre: libcfs: fix
 libcfs_cpu coding style
In-Reply-To: <1527603725-30560-14-git-send-email-jsimmons@infradead.org>
References: <1527603725-30560-1-git-send-email-jsimmons@infradead.org>
 <1527603725-30560-14-git-send-email-jsimmons@infradead.org>
Message-ID: <20180601083932.GF19242@kroah.com>

On Tue, May 29, 2018 at 10:21:53AM -0400, James Simmons wrote:
>  int cfs_cpt_set_nodemask(struct cfs_cpt_table *cptab,
> -			 int cpt, nodemask_t *mask);
> +			 int cpt, const nodemask_t *mask);

This is not a coding style change.  It is a semi-functional change.
Don't bury code changes in the middle of "cleanup code formatting"
patches, it makes maintainers very grumpy when we notice this as we then
feel like we might have missed other things being snuck by us in the
guise of these types of patches...

thanks,

greg k-h

From gregkh at linuxfoundation.org  Fri Jun  1 08:41:00 2018
From: gregkh at linuxfoundation.org (Greg Kroah-Hartman)
Date: Fri, 1 Jun 2018 10:41:00 +0200
Subject: [lustre-devel] [PATCH v2 11/25] staging: lustre: libcfs:
 provide debugfs files for distance handling
In-Reply-To: <1527603725-30560-12-git-send-email-jsimmons@infradead.org>
References: <1527603725-30560-1-git-send-email-jsimmons@infradead.org>
 <1527603725-30560-12-git-send-email-jsimmons@infradead.org>
Message-ID: <20180601084100.GG19242@kroah.com>

On Tue, May 29, 2018 at 10:21:51AM -0400, James Simmons wrote:
> From: Amir Shehata <amir.shehata at intel.com>
> 
> On systems with large number of NUMA nodes and cores it is easy
> to incorrectly configure their use with Lustre. Provide debugfs
> files which can help track down any issues.
> 
> Signed-off-by: Amir Shehata <amir.shehata at intel.com>
> Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-7734
> Reviewed-on: http://review.whamcloud.com/18916
> Reviewed-by: Olaf Weber <olaf at sgi.com>
> Reviewed-by: Doug Oucharek <dougso at me.com>
> Signed-off-by: James Simmons <jsimmons at infradead.org>
> ---
> Changelog:
> 
> v1) Initial patch
> v2) Rebased patch. No code changes from original patch
> 
>  drivers/staging/lustre/lnet/libcfs/module.c | 53 +++++++++++++++++++++++++++++
>  1 file changed, 53 insertions(+)
> 
> diff --git a/drivers/staging/lustre/lnet/libcfs/module.c b/drivers/staging/lustre/lnet/libcfs/module.c
> index b438d456..d2dfc29 100644
> --- a/drivers/staging/lustre/lnet/libcfs/module.c
> +++ b/drivers/staging/lustre/lnet/libcfs/module.c
> @@ -468,6 +468,53 @@ static int proc_cpt_table(struct ctl_table *table, int write,
>  				    __proc_cpt_table);
>  }
>  
> +static int __proc_cpt_distance(void *data, int write,
> +			       loff_t pos, void __user *buffer, int nob)
> +{
> +	char *buf = NULL;
> +	int len = 4096;
> +	int rc = 0;
> +
> +	if (write)
> +		return -EPERM;
> +
> +	LASSERT(cfs_cpt_tab);

What is this assert really checking?  Why is it needed?

thanks,

greg k-h

From gregkh at linuxfoundation.org  Fri Jun  1 08:43:47 2018
From: gregkh at linuxfoundation.org (Greg Kroah-Hartman)
Date: Fri, 1 Jun 2018 10:43:47 +0200
Subject: [lustre-devel] [PATCH v2 23/25] staging: lustre: libcfs: rework
 CPU pattern parsing code
In-Reply-To: <1527603725-30560-24-git-send-email-jsimmons@infradead.org>
References: <1527603725-30560-1-git-send-email-jsimmons@infradead.org>
 <1527603725-30560-24-git-send-email-jsimmons@infradead.org>
Message-ID: <20180601084347.GH19242@kroah.com>

On Tue, May 29, 2018 at 10:22:03AM -0400, James Simmons wrote:
> From: Dmitry Eremin <dmitry.eremin at intel.com>
> 
> Currently the module param string for CPU pattern can be
> modified which is wrong. Rewrite CPU pattern parsing code
> to avoid the passed buffer from being changed. This change
> also enables us to add real errors propogation to the caller
> functions.
> 
> Signed-off-by: Dmitry Eremin <dmitry.eremin at intel.com>
> Signed-off-by: Amir Shehata <amir.shehata at intel.com>
> Signed-off-by: Andreas Dilger <andreas.dilger at intel.com>
> Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-8703
> Reviewed-on: https://review.whamcloud.com/23306
> Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-9715
> Reviewed-on: https://review.whamcloud.com/27872
> Reviewed-by: James Simmons <uja.ornl at yahoo.com>
> Reviewed-by: Andreas Dilger <andreas.dilger at intel.com>
> Reviewed-by: Patrick Farrell <paf at cray.com>
> Reviewed-by: Olaf Weber <olaf.weber at hpe.com>
> Reviewed-by: Oleg Drokin <oleg.drokin at intel.com>
> Signed-off-by: James Simmons <jsimmons at infradead.org>
> ---
> Changelog:
> 
> v1) Initial patch
> v2) Rebased patch. No changes in code from earlier patch
> 
>  .../lustre/include/linux/libcfs/libcfs_cpu.h       |   2 +-
>  drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c    | 146 ++++++++++++---------
>  2 files changed, 87 insertions(+), 61 deletions(-)
> 
> diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h
> index c0aa0b3..12ed0a9 100644
> --- a/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h
> +++ b/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h
> @@ -393,7 +393,7 @@ static inline int cfs_cpu_init(void)
>  
>  static inline void cfs_cpu_fini(void)
>  {
> -	if (cfs_cpt_tab) {
> +	if (!IS_ERR_OR_NULL(cfs_cpt_tab)) {
>  		cfs_cpt_table_free(cfs_cpt_tab);
>  		cfs_cpt_tab = NULL;
>  	}
> diff --git a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
> index 649f7f9..aed48de 100644
> --- a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
> +++ b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
> @@ -692,11 +692,11 @@ int cfs_cpt_bind(struct cfs_cpt_table *cptab, int cpt)
>  		nodemask = cptab->ctb_parts[cpt].cpt_nodemask;
>  	}
>  
> -	if (cpumask_any_and(*cpumask, cpu_online_mask) >= nr_cpu_ids) {
> +	if (!cpumask_intersects(*cpumask, cpu_online_mask)) {
>  		CDEBUG(D_INFO,
>  		       "No online CPU found in CPU partition %d, did someone do CPU hotplug on system? You might need to reload Lustre modules to keep system working well.\n",
>  		       cpt);

This is the funniest error message I have seen in a while.

No one should have to reload all kernel modules just because the CPU
topology changed, that's crazy.  You have the ability to read all of
this at runtime, and react to changes that happen while the system is
running.  You should never need/rely on userspace passing in random
strings to pretend to match up with what the system really has at the
moment, that way lies madness.

All of this should be ripped out and use the proper apis instead.  No
special userspace api should be needed at all.

greg k-h

From gregkh at linuxfoundation.org  Fri Jun  1 09:11:33 2018
From: gregkh at linuxfoundation.org (Greg Kroah-Hartman)
Date: Fri, 1 Jun 2018 11:11:33 +0200
Subject: [lustre-devel] [PATCH] staging: lustre: delete the filesystem from
 the tree.
Message-ID: <20180601091133.GA27521@kroah.com>

The Lustre filesystem has been in the kernel tree for over 5 years now.
While it has been an endless source of enjoyment for new kernel
developers learning how to do basic codingstyle cleanups, as well as an
semi-entertaining source of bewilderment from the vfs developers any
time they have looked into the codebase to try to figure out how to port
their latest api changes to this filesystem, it has not really moved
forward into the "this is in shape to get out of staging" despite many
half-completed attempts.

And getting code out of staging is the main goal of that portion of the
kernel tree.  Code should not stagnate and it feels like having this
code in staging is only causing the development cycle of the filesystem
to take longer than it should.  There is a whole separate out-of-tree
copy of this codebase where the developers work on it, and then random
changes are thrown over the wall at staging at some later point in time.
This dual-tree development model has never worked, and the state of this
codebase is proof of that.

So, let's just delete the whole mess.  Now the lustre developers can go
off and work in their out-of-tree codebase and not have to worry about
providing valid changelog entries and breaking their patches up into
logical pieces.  They can take the time they have spend doing those
types of housekeeping chores and get the codebase into a much better
shape, and it can be submitted for inclusion into the real part of the
kernel tree when ready.

Signed-off-by: Greg Kroah-Hartman <gregkh at linuxfoundation.org>
---
 MAINTAINERS                                   |    9 -
 drivers/staging/Kconfig                       |    2 -
 drivers/staging/Makefile                      |    1 -
 drivers/staging/lustre/Kconfig                |    3 -
 drivers/staging/lustre/Makefile               |    2 -
 drivers/staging/lustre/README.txt             |   83 -
 drivers/staging/lustre/TODO                   |  302 --
 .../lustre/include/linux/libcfs/libcfs.h      |   76 -
 .../lustre/include/linux/libcfs/libcfs_cpu.h  |  434 --
 .../include/linux/libcfs/libcfs_crypto.h      |  208 -
 .../include/linux/libcfs/libcfs_debug.h       |  207 -
 .../lustre/include/linux/libcfs/libcfs_fail.h |  194 -
 .../lustre/include/linux/libcfs/libcfs_hash.h |  869 ----
 .../include/linux/libcfs/libcfs_private.h     |  200 -
 .../include/linux/libcfs/libcfs_string.h      |  102 -
 .../staging/lustre/include/linux/lnet/api.h   |  212 -
 .../lustre/include/linux/lnet/lib-lnet.h      |  652 ---
 .../lustre/include/linux/lnet/lib-types.h     |  666 ---
 .../lustre/include/linux/lnet/socklnd.h       |   87 -
 .../include/uapi/linux/lnet/libcfs_debug.h    |  149 -
 .../include/uapi/linux/lnet/libcfs_ioctl.h    |  141 -
 .../lustre/include/uapi/linux/lnet/lnet-dlc.h |  150 -
 .../include/uapi/linux/lnet/lnet-types.h      |  669 ---
 .../lustre/include/uapi/linux/lnet/lnetctl.h  |  123 -
 .../lustre/include/uapi/linux/lnet/lnetst.h   |  556 ---
 .../lustre/include/uapi/linux/lnet/nidstr.h   |  119 -
 .../lustre/include/uapi/linux/lnet/socklnd.h  |   44 -
 .../include/uapi/linux/lustre/lustre_cfg.h    |  261 -
 .../include/uapi/linux/lustre/lustre_fid.h    |  293 --
 .../include/uapi/linux/lustre/lustre_fiemap.h |   72 -
 .../include/uapi/linux/lustre/lustre_idl.h    | 2690 -----------
 .../include/uapi/linux/lustre/lustre_ioctl.h  |  229 -
 .../uapi/linux/lustre/lustre_kernelcomm.h     |   94 -
 .../include/uapi/linux/lustre/lustre_ostid.h  |  236 -
 .../include/uapi/linux/lustre/lustre_param.h  |   94 -
 .../include/uapi/linux/lustre/lustre_user.h   | 1327 ------
 .../include/uapi/linux/lustre/lustre_ver.h    |   27 -
 drivers/staging/lustre/lnet/Kconfig           |   46 -
 drivers/staging/lustre/lnet/Makefile          |    1 -
 drivers/staging/lustre/lnet/klnds/Makefile    |    1 -
 .../lustre/lnet/klnds/o2iblnd/Makefile        |    5 -
 .../lustre/lnet/klnds/o2iblnd/o2iblnd.c       | 2958 ------------
 .../lustre/lnet/klnds/o2iblnd/o2iblnd.h       | 1048 ----
 .../lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c    | 3763 ---------------
 .../lnet/klnds/o2iblnd/o2iblnd_modparams.c    |  296 --
 .../lustre/lnet/klnds/socklnd/Makefile        |    6 -
 .../lustre/lnet/klnds/socklnd/socklnd.c       | 2921 ------------
 .../lustre/lnet/klnds/socklnd/socklnd.h       |  704 ---
 .../lustre/lnet/klnds/socklnd/socklnd_cb.c    | 2586 ----------
 .../lustre/lnet/klnds/socklnd/socklnd_lib.c   |  534 ---
 .../lnet/klnds/socklnd/socklnd_modparams.c    |  184 -
 .../lustre/lnet/klnds/socklnd/socklnd_proto.c |  810 ----
 drivers/staging/lustre/lnet/libcfs/Makefile   |   16 -
 drivers/staging/lustre/lnet/libcfs/debug.c    |  461 --
 drivers/staging/lustre/lnet/libcfs/fail.c     |  146 -
 drivers/staging/lustre/lnet/libcfs/hash.c     | 2065 --------
 .../staging/lustre/lnet/libcfs/libcfs_cpu.c   | 1086 -----
 .../staging/lustre/lnet/libcfs/libcfs_lock.c  |  155 -
 .../staging/lustre/lnet/libcfs/libcfs_mem.c   |  171 -
 .../lustre/lnet/libcfs/libcfs_string.c        |  562 ---
 .../lustre/lnet/libcfs/linux-crypto-adler.c   |  139 -
 .../staging/lustre/lnet/libcfs/linux-crypto.c |  447 --
 .../staging/lustre/lnet/libcfs/linux-crypto.h |   30 -
 .../staging/lustre/lnet/libcfs/linux-debug.c  |  142 -
 .../lustre/lnet/libcfs/linux-tracefile.c      |  258 -
 drivers/staging/lustre/lnet/libcfs/module.c   |  758 ---
 .../staging/lustre/lnet/libcfs/tracefile.c    | 1198 -----
 .../staging/lustre/lnet/libcfs/tracefile.h    |  274 --
 drivers/staging/lustre/lnet/lnet/Makefile     |   10 -
 drivers/staging/lustre/lnet/lnet/acceptor.c   |  501 --
 drivers/staging/lustre/lnet/lnet/api-ni.c     | 2307 ---------
 drivers/staging/lustre/lnet/lnet/config.c     | 1235 -----
 drivers/staging/lustre/lnet/lnet/lib-eq.c     |  426 --
 drivers/staging/lustre/lnet/lnet/lib-md.c     |  463 --
 drivers/staging/lustre/lnet/lnet/lib-me.c     |  274 --
 drivers/staging/lustre/lnet/lnet/lib-move.c   | 2386 ----------
 drivers/staging/lustre/lnet/lnet/lib-msg.c    |  625 ---
 drivers/staging/lustre/lnet/lnet/lib-ptl.c    |  987 ----
 drivers/staging/lustre/lnet/lnet/lib-socket.c |  585 ---
 drivers/staging/lustre/lnet/lnet/lo.c         |  105 -
 drivers/staging/lustre/lnet/lnet/module.c     |  239 -
 drivers/staging/lustre/lnet/lnet/net_fault.c  | 1023 ----
 drivers/staging/lustre/lnet/lnet/nidstrings.c | 1261 -----
 drivers/staging/lustre/lnet/lnet/peer.c       |  456 --
 drivers/staging/lustre/lnet/lnet/router.c     | 1799 -------
 .../staging/lustre/lnet/lnet/router_proc.c    |  907 ----
 drivers/staging/lustre/lnet/selftest/Makefile |    7 -
 .../staging/lustre/lnet/selftest/brw_test.c   |  526 --
 drivers/staging/lustre/lnet/selftest/conctl.c |  801 ----
 drivers/staging/lustre/lnet/selftest/conrpc.c | 1396 ------
 drivers/staging/lustre/lnet/selftest/conrpc.h |  142 -
 .../staging/lustre/lnet/selftest/console.c    | 2104 --------
 .../staging/lustre/lnet/selftest/console.h    |  244 -
 .../staging/lustre/lnet/selftest/framework.c  | 1786 -------
 drivers/staging/lustre/lnet/selftest/module.c |  169 -
 .../staging/lustre/lnet/selftest/ping_test.c  |  228 -
 drivers/staging/lustre/lnet/selftest/rpc.c    | 1682 -------
 drivers/staging/lustre/lnet/selftest/rpc.h    |  295 --
 .../staging/lustre/lnet/selftest/selftest.h   |  622 ---
 drivers/staging/lustre/lnet/selftest/timer.c  |  244 -
 drivers/staging/lustre/lnet/selftest/timer.h  |   50 -
 drivers/staging/lustre/lustre/Kconfig         |   45 -
 drivers/staging/lustre/lustre/Makefile        |    2 -
 drivers/staging/lustre/lustre/fid/Makefile    |    5 -
 .../staging/lustre/lustre/fid/fid_internal.h  |   46 -
 drivers/staging/lustre/lustre/fid/fid_lib.c   |   87 -
 .../staging/lustre/lustre/fid/fid_request.c   |  410 --
 drivers/staging/lustre/lustre/fid/lproc_fid.c |  225 -
 drivers/staging/lustre/lustre/fld/Makefile    |    5 -
 drivers/staging/lustre/lustre/fld/fld_cache.c |  516 --
 .../staging/lustre/lustre/fld/fld_internal.h  |  170 -
 .../staging/lustre/lustre/fld/fld_request.c   |  446 --
 drivers/staging/lustre/lustre/fld/lproc_fld.c |  154 -
 .../staging/lustre/lustre/include/cl_object.h | 2463 ----------
 .../lustre/lustre/include/interval_tree.h     |  119 -
 .../staging/lustre/lustre/include/llog_swab.h |   67 -
 .../lustre/lustre/include/lprocfs_status.h    |  646 ---
 .../staging/lustre/lustre/include/lu_object.h | 1305 -----
 .../staging/lustre/lustre/include/lu_ref.h    |  178 -
 .../lustre/lustre/include/lustre_acl.h        |   51 -
 .../lustre/lustre/include/lustre_compat.h     |   82 -
 .../lustre/lustre/include/lustre_debug.h      |   52 -
 .../lustre/lustre/include/lustre_disk.h       |  152 -
 .../lustre/lustre/include/lustre_dlm.h        | 1346 ------
 .../lustre/lustre/include/lustre_dlm_flags.h  |  402 --
 .../lustre/lustre/include/lustre_errno.h      |  198 -
 .../lustre/lustre/include/lustre_export.h     |  250 -
 .../lustre/lustre/include/lustre_fid.h        |  676 ---
 .../lustre/lustre/include/lustre_fld.h        |  137 -
 .../staging/lustre/lustre/include/lustre_ha.h |   61 -
 .../lustre/lustre/include/lustre_handles.h    |   91 -
 .../lustre/lustre/include/lustre_import.h     |  369 --
 .../lustre/lustre/include/lustre_intent.h     |   71 -
 .../lustre/lustre/include/lustre_kernelcomm.h |   56 -
 .../lustre/lustre/include/lustre_lib.h        |  126 -
 .../lustre/lustre/include/lustre_linkea.h     |   93 -
 .../lustre/lustre/include/lustre_lmv.h        |  174 -
 .../lustre/lustre/include/lustre_log.h        |  382 --
 .../lustre/lustre/include/lustre_mdc.h        |  229 -
 .../lustre/lustre/include/lustre_mds.h        |   62 -
 .../lustre/lustre/include/lustre_net.h        | 2360 ---------
 .../lustre/lustre/include/lustre_nrs.h        |  718 ---
 .../lustre/lustre/include/lustre_nrs_fifo.h   |   71 -
 .../lustre/lustre/include/lustre_obdo.h       |   55 -
 .../lustre/include/lustre_patchless_compat.h  |   68 -
 .../lustre/lustre/include/lustre_req_layout.h |  307 --
 .../lustre/lustre/include/lustre_sec.h        | 1072 -----
 .../lustre/lustre/include/lustre_swab.h       |  109 -
 drivers/staging/lustre/lustre/include/obd.h   | 1114 -----
 .../staging/lustre/lustre/include/obd_cksum.h |  153 -
 .../staging/lustre/lustre/include/obd_class.h | 1603 -------
 .../lustre/lustre/include/obd_support.h       |  517 --
 .../staging/lustre/lustre/include/seq_range.h |  200 -
 .../lustre/lustre/ldlm/interval_tree.c        |  599 ---
 drivers/staging/lustre/lustre/ldlm/l_lock.c   |   73 -
 .../staging/lustre/lustre/ldlm/ldlm_extent.c  |  258 -
 .../staging/lustre/lustre/ldlm/ldlm_flock.c   |  486 --
 .../lustre/lustre/ldlm/ldlm_inodebits.c       |   69 -
 .../lustre/lustre/ldlm/ldlm_internal.h        |  342 --
 drivers/staging/lustre/lustre/ldlm/ldlm_lib.c |  842 ----
 .../staging/lustre/lustre/ldlm/ldlm_lock.c    | 2135 ---------
 .../staging/lustre/lustre/ldlm/ldlm_lockd.c   | 1163 -----
 .../staging/lustre/lustre/ldlm/ldlm_plain.c   |   68 -
 .../staging/lustre/lustre/ldlm/ldlm_pool.c    | 1013 ----
 .../staging/lustre/lustre/ldlm/ldlm_request.c | 2033 --------
 .../lustre/lustre/ldlm/ldlm_resource.c        | 1318 ------
 drivers/staging/lustre/lustre/llite/Makefile  |   13 -
 drivers/staging/lustre/lustre/llite/acl.c     |  108 -
 drivers/staging/lustre/lustre/llite/dcache.c  |  300 --
 drivers/staging/lustre/lustre/llite/dir.c     | 1708 -------
 drivers/staging/lustre/lustre/llite/file.c    | 3580 --------------
 drivers/staging/lustre/lustre/llite/glimpse.c |  205 -
 .../staging/lustre/lustre/llite/lcommon_cl.c  |  292 --
 .../lustre/lustre/llite/lcommon_misc.c        |  186 -
 .../lustre/lustre/llite/llite_internal.h      | 1344 ------
 .../staging/lustre/lustre/llite/llite_lib.c   | 2668 -----------
 .../staging/lustre/lustre/llite/llite_mmap.c  |  480 --
 .../staging/lustre/lustre/llite/llite_nfs.c   |  375 --
 .../staging/lustre/lustre/llite/lproc_llite.c | 1659 -------
 drivers/staging/lustre/lustre/llite/namei.c   | 1207 -----
 .../staging/lustre/lustre/llite/range_lock.c  |  241 -
 .../staging/lustre/lustre/llite/range_lock.h  |   83 -
 drivers/staging/lustre/lustre/llite/rw.c      | 1214 -----
 drivers/staging/lustre/lustre/llite/rw26.c    |  641 ---
 .../staging/lustre/lustre/llite/statahead.c   | 1577 ------
 drivers/staging/lustre/lustre/llite/super25.c |  189 -
 drivers/staging/lustre/lustre/llite/symlink.c |  159 -
 drivers/staging/lustre/lustre/llite/vvp_dev.c |  640 ---
 .../lustre/lustre/llite/vvp_internal.h        |  321 --
 drivers/staging/lustre/lustre/llite/vvp_io.c  | 1374 ------
 .../staging/lustre/lustre/llite/vvp_lock.c    |   87 -
 .../staging/lustre/lustre/llite/vvp_object.c  |  303 --
 .../staging/lustre/lustre/llite/vvp_page.c    |  523 --
 drivers/staging/lustre/lustre/llite/xattr.c   |  665 ---
 .../staging/lustre/lustre/llite/xattr_cache.c |  504 --
 .../lustre/lustre/llite/xattr_security.c      |   96 -
 drivers/staging/lustre/lustre/lmv/Makefile    |    5 -
 drivers/staging/lustre/lustre/lmv/lmv_fld.c   |   82 -
 .../staging/lustre/lustre/lmv/lmv_intent.c    |  521 --
 .../staging/lustre/lustre/lmv/lmv_internal.h  |  164 -
 drivers/staging/lustre/lustre/lmv/lmv_obd.c   | 3131 ------------
 drivers/staging/lustre/lustre/lmv/lproc_lmv.c |  173 -
 drivers/staging/lustre/lustre/lov/Makefile    |    9 -
 .../lustre/lustre/lov/lov_cl_internal.h       |  639 ---
 drivers/staging/lustre/lustre/lov/lov_dev.c   |  384 --
 drivers/staging/lustre/lustre/lov/lov_ea.c    |  331 --
 .../staging/lustre/lustre/lov/lov_internal.h  |  286 --
 drivers/staging/lustre/lustre/lov/lov_io.c    | 1023 ----
 drivers/staging/lustre/lustre/lov/lov_lock.c  |  348 --
 drivers/staging/lustre/lustre/lov/lov_merge.c |  105 -
 drivers/staging/lustre/lustre/lov/lov_obd.c   | 1444 ------
 .../staging/lustre/lustre/lov/lov_object.c    | 1625 -------
 .../staging/lustre/lustre/lov/lov_offset.c    |  269 --
 drivers/staging/lustre/lustre/lov/lov_pack.c  |  400 --
 drivers/staging/lustre/lustre/lov/lov_page.c  |  136 -
 drivers/staging/lustre/lustre/lov/lov_pool.c  |  546 ---
 .../staging/lustre/lustre/lov/lov_request.c   |  354 --
 .../staging/lustre/lustre/lov/lovsub_dev.c    |  147 -
 .../staging/lustre/lustre/lov/lovsub_lock.c   |   81 -
 .../staging/lustre/lustre/lov/lovsub_object.c |  180 -
 .../staging/lustre/lustre/lov/lovsub_page.c   |   68 -
 drivers/staging/lustre/lustre/lov/lproc_lov.c |  299 --
 drivers/staging/lustre/lustre/mdc/Makefile    |    5 -
 drivers/staging/lustre/lustre/mdc/lproc_mdc.c |  231 -
 .../staging/lustre/lustre/mdc/mdc_internal.h  |  144 -
 drivers/staging/lustre/lustre/mdc/mdc_lib.c   |  498 --
 drivers/staging/lustre/lustre/mdc/mdc_locks.c | 1239 -----
 drivers/staging/lustre/lustre/mdc/mdc_reint.c |  421 --
 .../staging/lustre/lustre/mdc/mdc_request.c   | 2770 -----------
 drivers/staging/lustre/lustre/mgc/Makefile    |    5 -
 drivers/staging/lustre/lustre/mgc/lproc_mgc.c |   69 -
 .../staging/lustre/lustre/mgc/mgc_internal.h  |   57 -
 .../staging/lustre/lustre/mgc/mgc_request.c   | 1851 --------
 .../staging/lustre/lustre/obdclass/Makefile   |   12 -
 .../lustre/lustre/obdclass/cl_internal.h      |   95 -
 .../staging/lustre/lustre/obdclass/cl_io.c    | 1151 -----
 .../staging/lustre/lustre/obdclass/cl_lock.c  |  275 --
 .../lustre/lustre/obdclass/cl_object.c        | 1059 -----
 .../staging/lustre/lustre/obdclass/cl_page.c  | 1045 ----
 .../lustre/lustre/obdclass/class_obd.c        |  544 ---
 .../staging/lustre/lustre/obdclass/debug.c    |   96 -
 .../staging/lustre/lustre/obdclass/genops.c   | 1480 ------
 .../lustre/lustre/obdclass/kernelcomm.c       |  240 -
 .../staging/lustre/lustre/obdclass/linkea.c   |  249 -
 .../lustre/obdclass/linux/linux-module.c      |  514 --
 .../lustre/obdclass/linux/linux-sysctl.c      |  162 -
 drivers/staging/lustre/lustre/obdclass/llog.c |  524 --
 .../staging/lustre/lustre/obdclass/llog_cat.c |  236 -
 .../lustre/lustre/obdclass/llog_internal.h    |   79 -
 .../staging/lustre/lustre/obdclass/llog_obd.c |  225 -
 .../lustre/lustre/obdclass/llog_swab.c        |  412 --
 .../lustre/lustre/obdclass/lprocfs_counters.c |  134 -
 .../lustre/lustre/obdclass/lprocfs_status.c   | 1698 -------
 .../lustre/lustre/obdclass/lu_object.c        | 2056 --------
 .../staging/lustre/lustre/obdclass/lu_ref.c   |   45 -
 .../lustre/lustre/obdclass/lustre_handles.c   |  241 -
 .../lustre/lustre/obdclass/lustre_peer.c      |  214 -
 .../lustre/lustre/obdclass/obd_config.c       | 1538 ------
 .../lustre/lustre/obdclass/obd_mount.c        | 1245 -----
 drivers/staging/lustre/lustre/obdclass/obdo.c |  181 -
 .../lustre/lustre/obdclass/statfs_pack.c      |   58 -
 drivers/staging/lustre/lustre/obdclass/uuid.c |   45 -
 .../staging/lustre/lustre/obdecho/Makefile    |    5 -
 .../lustre/lustre/obdecho/echo_client.c       | 1729 -------
 .../lustre/lustre/obdecho/echo_internal.h     |   42 -
 drivers/staging/lustre/lustre/osc/Makefile    |    6 -
 drivers/staging/lustre/lustre/osc/lproc_osc.c |  838 ----
 drivers/staging/lustre/lustre/osc/osc_cache.c | 3306 -------------
 .../lustre/lustre/osc/osc_cl_internal.h       |  681 ---
 drivers/staging/lustre/lustre/osc/osc_dev.c   |  246 -
 .../staging/lustre/lustre/osc/osc_internal.h  |  237 -
 drivers/staging/lustre/lustre/osc/osc_io.c    |  918 ----
 drivers/staging/lustre/lustre/osc/osc_lock.c  | 1230 -----
 .../staging/lustre/lustre/osc/osc_object.c    |  473 --
 drivers/staging/lustre/lustre/osc/osc_page.c  | 1094 -----
 drivers/staging/lustre/lustre/osc/osc_quota.c |  236 -
 .../staging/lustre/lustre/osc/osc_request.c   | 2907 ------------
 drivers/staging/lustre/lustre/ptlrpc/Makefile |   23 -
 drivers/staging/lustre/lustre/ptlrpc/client.c | 3271 -------------
 .../staging/lustre/lustre/ptlrpc/connection.c |  192 -
 drivers/staging/lustre/lustre/ptlrpc/errno.c  |  383 --
 drivers/staging/lustre/lustre/ptlrpc/events.c |  585 ---
 drivers/staging/lustre/lustre/ptlrpc/import.c | 1677 -------
 drivers/staging/lustre/lustre/ptlrpc/layout.c | 2232 ---------
 .../lustre/lustre/ptlrpc/llog_client.c        |  338 --
 .../staging/lustre/lustre/ptlrpc/llog_net.c   |   67 -
 .../lustre/lustre/ptlrpc/lproc_ptlrpc.c       | 1316 ------
 drivers/staging/lustre/lustre/ptlrpc/niobuf.c |  771 ---
 drivers/staging/lustre/lustre/ptlrpc/nrs.c    | 1613 -------
 .../staging/lustre/lustre/ptlrpc/nrs_fifo.c   |  270 --
 .../lustre/lustre/ptlrpc/pack_generic.c       | 2311 ---------
 drivers/staging/lustre/lustre/ptlrpc/pers.c   |   72 -
 drivers/staging/lustre/lustre/ptlrpc/pinger.c |  474 --
 .../lustre/lustre/ptlrpc/ptlrpc_internal.h    |  371 --
 .../lustre/lustre/ptlrpc/ptlrpc_module.c      |  186 -
 .../staging/lustre/lustre/ptlrpc/ptlrpcd.c    |  914 ----
 .../staging/lustre/lustre/ptlrpc/recover.c    |  374 --
 drivers/staging/lustre/lustre/ptlrpc/sec.c    | 2379 ----------
 .../staging/lustre/lustre/ptlrpc/sec_bulk.c   |  572 ---
 .../staging/lustre/lustre/ptlrpc/sec_config.c |  850 ----
 drivers/staging/lustre/lustre/ptlrpc/sec_gc.c |  190 -
 .../staging/lustre/lustre/ptlrpc/sec_lproc.c  |  170 -
 .../staging/lustre/lustre/ptlrpc/sec_null.c   |  459 --
 .../staging/lustre/lustre/ptlrpc/sec_plain.c  | 1023 ----
 .../staging/lustre/lustre/ptlrpc/service.c    | 2807 -----------
 .../staging/lustre/lustre/ptlrpc/wiretest.c   | 4210 -----------------
 drivers/staging/lustre/sysfs-fs-lustre        |  654 ---
 scripts/selinux/mdp/mdp.c                     |    1 -
 308 files changed, 195272 deletions(-)
 delete mode 100644 drivers/staging/lustre/Kconfig
 delete mode 100644 drivers/staging/lustre/Makefile
 delete mode 100644 drivers/staging/lustre/README.txt
 delete mode 100644 drivers/staging/lustre/TODO
 delete mode 100644 drivers/staging/lustre/include/linux/libcfs/libcfs.h
 delete mode 100644 drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h
 delete mode 100644 drivers/staging/lustre/include/linux/libcfs/libcfs_crypto.h
 delete mode 100644 drivers/staging/lustre/include/linux/libcfs/libcfs_debug.h
 delete mode 100644 drivers/staging/lustre/include/linux/libcfs/libcfs_fail.h
 delete mode 100644 drivers/staging/lustre/include/linux/libcfs/libcfs_hash.h
 delete mode 100644 drivers/staging/lustre/include/linux/libcfs/libcfs_private.h
 delete mode 100644 drivers/staging/lustre/include/linux/libcfs/libcfs_string.h
 delete mode 100644 drivers/staging/lustre/include/linux/lnet/api.h
 delete mode 100644 drivers/staging/lustre/include/linux/lnet/lib-lnet.h
 delete mode 100644 drivers/staging/lustre/include/linux/lnet/lib-types.h
 delete mode 100644 drivers/staging/lustre/include/linux/lnet/socklnd.h
 delete mode 100644 drivers/staging/lustre/include/uapi/linux/lnet/libcfs_debug.h
 delete mode 100644 drivers/staging/lustre/include/uapi/linux/lnet/libcfs_ioctl.h
 delete mode 100644 drivers/staging/lustre/include/uapi/linux/lnet/lnet-dlc.h
 delete mode 100644 drivers/staging/lustre/include/uapi/linux/lnet/lnet-types.h
 delete mode 100644 drivers/staging/lustre/include/uapi/linux/lnet/lnetctl.h
 delete mode 100644 drivers/staging/lustre/include/uapi/linux/lnet/lnetst.h
 delete mode 100644 drivers/staging/lustre/include/uapi/linux/lnet/nidstr.h
 delete mode 100644 drivers/staging/lustre/include/uapi/linux/lnet/socklnd.h
 delete mode 100644 drivers/staging/lustre/include/uapi/linux/lustre/lustre_cfg.h
 delete mode 100644 drivers/staging/lustre/include/uapi/linux/lustre/lustre_fid.h
 delete mode 100644 drivers/staging/lustre/include/uapi/linux/lustre/lustre_fiemap.h
 delete mode 100644 drivers/staging/lustre/include/uapi/linux/lustre/lustre_idl.h
 delete mode 100644 drivers/staging/lustre/include/uapi/linux/lustre/lustre_ioctl.h
 delete mode 100644 drivers/staging/lustre/include/uapi/linux/lustre/lustre_kernelcomm.h
 delete mode 100644 drivers/staging/lustre/include/uapi/linux/lustre/lustre_ostid.h
 delete mode 100644 drivers/staging/lustre/include/uapi/linux/lustre/lustre_param.h
 delete mode 100644 drivers/staging/lustre/include/uapi/linux/lustre/lustre_user.h
 delete mode 100644 drivers/staging/lustre/include/uapi/linux/lustre/lustre_ver.h
 delete mode 100644 drivers/staging/lustre/lnet/Kconfig
 delete mode 100644 drivers/staging/lustre/lnet/Makefile
 delete mode 100644 drivers/staging/lustre/lnet/klnds/Makefile
 delete mode 100644 drivers/staging/lustre/lnet/klnds/o2iblnd/Makefile
 delete mode 100644 drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c
 delete mode 100644 drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.h
 delete mode 100644 drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c
 delete mode 100644 drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd_modparams.c
 delete mode 100644 drivers/staging/lustre/lnet/klnds/socklnd/Makefile
 delete mode 100644 drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c
 delete mode 100644 drivers/staging/lustre/lnet/klnds/socklnd/socklnd.h
 delete mode 100644 drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c
 delete mode 100644 drivers/staging/lustre/lnet/klnds/socklnd/socklnd_lib.c
 delete mode 100644 drivers/staging/lustre/lnet/klnds/socklnd/socklnd_modparams.c
 delete mode 100644 drivers/staging/lustre/lnet/klnds/socklnd/socklnd_proto.c
 delete mode 100644 drivers/staging/lustre/lnet/libcfs/Makefile
 delete mode 100644 drivers/staging/lustre/lnet/libcfs/debug.c
 delete mode 100644 drivers/staging/lustre/lnet/libcfs/fail.c
 delete mode 100644 drivers/staging/lustre/lnet/libcfs/hash.c
 delete mode 100644 drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
 delete mode 100644 drivers/staging/lustre/lnet/libcfs/libcfs_lock.c
 delete mode 100644 drivers/staging/lustre/lnet/libcfs/libcfs_mem.c
 delete mode 100644 drivers/staging/lustre/lnet/libcfs/libcfs_string.c
 delete mode 100644 drivers/staging/lustre/lnet/libcfs/linux-crypto-adler.c
 delete mode 100644 drivers/staging/lustre/lnet/libcfs/linux-crypto.c
 delete mode 100644 drivers/staging/lustre/lnet/libcfs/linux-crypto.h
 delete mode 100644 drivers/staging/lustre/lnet/libcfs/linux-debug.c
 delete mode 100644 drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
 delete mode 100644 drivers/staging/lustre/lnet/libcfs/module.c
 delete mode 100644 drivers/staging/lustre/lnet/libcfs/tracefile.c
 delete mode 100644 drivers/staging/lustre/lnet/libcfs/tracefile.h
 delete mode 100644 drivers/staging/lustre/lnet/lnet/Makefile
 delete mode 100644 drivers/staging/lustre/lnet/lnet/acceptor.c
 delete mode 100644 drivers/staging/lustre/lnet/lnet/api-ni.c
 delete mode 100644 drivers/staging/lustre/lnet/lnet/config.c
 delete mode 100644 drivers/staging/lustre/lnet/lnet/lib-eq.c
 delete mode 100644 drivers/staging/lustre/lnet/lnet/lib-md.c
 delete mode 100644 drivers/staging/lustre/lnet/lnet/lib-me.c
 delete mode 100644 drivers/staging/lustre/lnet/lnet/lib-move.c
 delete mode 100644 drivers/staging/lustre/lnet/lnet/lib-msg.c
 delete mode 100644 drivers/staging/lustre/lnet/lnet/lib-ptl.c
 delete mode 100644 drivers/staging/lustre/lnet/lnet/lib-socket.c
 delete mode 100644 drivers/staging/lustre/lnet/lnet/lo.c
 delete mode 100644 drivers/staging/lustre/lnet/lnet/module.c
 delete mode 100644 drivers/staging/lustre/lnet/lnet/net_fault.c
 delete mode 100644 drivers/staging/lustre/lnet/lnet/nidstrings.c
 delete mode 100644 drivers/staging/lustre/lnet/lnet/peer.c
 delete mode 100644 drivers/staging/lustre/lnet/lnet/router.c
 delete mode 100644 drivers/staging/lustre/lnet/lnet/router_proc.c
 delete mode 100644 drivers/staging/lustre/lnet/selftest/Makefile
 delete mode 100644 drivers/staging/lustre/lnet/selftest/brw_test.c
 delete mode 100644 drivers/staging/lustre/lnet/selftest/conctl.c
 delete mode 100644 drivers/staging/lustre/lnet/selftest/conrpc.c
 delete mode 100644 drivers/staging/lustre/lnet/selftest/conrpc.h
 delete mode 100644 drivers/staging/lustre/lnet/selftest/console.c
 delete mode 100644 drivers/staging/lustre/lnet/selftest/console.h
 delete mode 100644 drivers/staging/lustre/lnet/selftest/framework.c
 delete mode 100644 drivers/staging/lustre/lnet/selftest/module.c
 delete mode 100644 drivers/staging/lustre/lnet/selftest/ping_test.c
 delete mode 100644 drivers/staging/lustre/lnet/selftest/rpc.c
 delete mode 100644 drivers/staging/lustre/lnet/selftest/rpc.h
 delete mode 100644 drivers/staging/lustre/lnet/selftest/selftest.h
 delete mode 100644 drivers/staging/lustre/lnet/selftest/timer.c
 delete mode 100644 drivers/staging/lustre/lnet/selftest/timer.h
 delete mode 100644 drivers/staging/lustre/lustre/Kconfig
 delete mode 100644 drivers/staging/lustre/lustre/Makefile
 delete mode 100644 drivers/staging/lustre/lustre/fid/Makefile
 delete mode 100644 drivers/staging/lustre/lustre/fid/fid_internal.h
 delete mode 100644 drivers/staging/lustre/lustre/fid/fid_lib.c
 delete mode 100644 drivers/staging/lustre/lustre/fid/fid_request.c
 delete mode 100644 drivers/staging/lustre/lustre/fid/lproc_fid.c
 delete mode 100644 drivers/staging/lustre/lustre/fld/Makefile
 delete mode 100644 drivers/staging/lustre/lustre/fld/fld_cache.c
 delete mode 100644 drivers/staging/lustre/lustre/fld/fld_internal.h
 delete mode 100644 drivers/staging/lustre/lustre/fld/fld_request.c
 delete mode 100644 drivers/staging/lustre/lustre/fld/lproc_fld.c
 delete mode 100644 drivers/staging/lustre/lustre/include/cl_object.h
 delete mode 100644 drivers/staging/lustre/lustre/include/interval_tree.h
 delete mode 100644 drivers/staging/lustre/lustre/include/llog_swab.h
 delete mode 100644 drivers/staging/lustre/lustre/include/lprocfs_status.h
 delete mode 100644 drivers/staging/lustre/lustre/include/lu_object.h
 delete mode 100644 drivers/staging/lustre/lustre/include/lu_ref.h
 delete mode 100644 drivers/staging/lustre/lustre/include/lustre_acl.h
 delete mode 100644 drivers/staging/lustre/lustre/include/lustre_compat.h
 delete mode 100644 drivers/staging/lustre/lustre/include/lustre_debug.h
 delete mode 100644 drivers/staging/lustre/lustre/include/lustre_disk.h
 delete mode 100644 drivers/staging/lustre/lustre/include/lustre_dlm.h
 delete mode 100644 drivers/staging/lustre/lustre/include/lustre_dlm_flags.h
 delete mode 100644 drivers/staging/lustre/lustre/include/lustre_errno.h
 delete mode 100644 drivers/staging/lustre/lustre/include/lustre_export.h
 delete mode 100644 drivers/staging/lustre/lustre/include/lustre_fid.h
 delete mode 100644 drivers/staging/lustre/lustre/include/lustre_fld.h
 delete mode 100644 drivers/staging/lustre/lustre/include/lustre_ha.h
 delete mode 100644 drivers/staging/lustre/lustre/include/lustre_handles.h
 delete mode 100644 drivers/staging/lustre/lustre/include/lustre_import.h
 delete mode 100644 drivers/staging/lustre/lustre/include/lustre_intent.h
 delete mode 100644 drivers/staging/lustre/lustre/include/lustre_kernelcomm.h
 delete mode 100644 drivers/staging/lustre/lustre/include/lustre_lib.h
 delete mode 100644 drivers/staging/lustre/lustre/include/lustre_linkea.h
 delete mode 100644 drivers/staging/lustre/lustre/include/lustre_lmv.h
 delete mode 100644 drivers/staging/lustre/lustre/include/lustre_log.h
 delete mode 100644 drivers/staging/lustre/lustre/include/lustre_mdc.h
 delete mode 100644 drivers/staging/lustre/lustre/include/lustre_mds.h
 delete mode 100644 drivers/staging/lustre/lustre/include/lustre_net.h
 delete mode 100644 drivers/staging/lustre/lustre/include/lustre_nrs.h
 delete mode 100644 drivers/staging/lustre/lustre/include/lustre_nrs_fifo.h
 delete mode 100644 drivers/staging/lustre/lustre/include/lustre_obdo.h
 delete mode 100644 drivers/staging/lustre/lustre/include/lustre_patchless_compat.h
 delete mode 100644 drivers/staging/lustre/lustre/include/lustre_req_layout.h
 delete mode 100644 drivers/staging/lustre/lustre/include/lustre_sec.h
 delete mode 100644 drivers/staging/lustre/lustre/include/lustre_swab.h
 delete mode 100644 drivers/staging/lustre/lustre/include/obd.h
 delete mode 100644 drivers/staging/lustre/lustre/include/obd_cksum.h
 delete mode 100644 drivers/staging/lustre/lustre/include/obd_class.h
 delete mode 100644 drivers/staging/lustre/lustre/include/obd_support.h
 delete mode 100644 drivers/staging/lustre/lustre/include/seq_range.h
 delete mode 100644 drivers/staging/lustre/lustre/ldlm/interval_tree.c
 delete mode 100644 drivers/staging/lustre/lustre/ldlm/l_lock.c
 delete mode 100644 drivers/staging/lustre/lustre/ldlm/ldlm_extent.c
 delete mode 100644 drivers/staging/lustre/lustre/ldlm/ldlm_flock.c
 delete mode 100644 drivers/staging/lustre/lustre/ldlm/ldlm_inodebits.c
 delete mode 100644 drivers/staging/lustre/lustre/ldlm/ldlm_internal.h
 delete mode 100644 drivers/staging/lustre/lustre/ldlm/ldlm_lib.c
 delete mode 100644 drivers/staging/lustre/lustre/ldlm/ldlm_lock.c
 delete mode 100644 drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c
 delete mode 100644 drivers/staging/lustre/lustre/ldlm/ldlm_plain.c
 delete mode 100644 drivers/staging/lustre/lustre/ldlm/ldlm_pool.c
 delete mode 100644 drivers/staging/lustre/lustre/ldlm/ldlm_request.c
 delete mode 100644 drivers/staging/lustre/lustre/ldlm/ldlm_resource.c
 delete mode 100644 drivers/staging/lustre/lustre/llite/Makefile
 delete mode 100644 drivers/staging/lustre/lustre/llite/acl.c
 delete mode 100644 drivers/staging/lustre/lustre/llite/dcache.c
 delete mode 100644 drivers/staging/lustre/lustre/llite/dir.c
 delete mode 100644 drivers/staging/lustre/lustre/llite/file.c
 delete mode 100644 drivers/staging/lustre/lustre/llite/glimpse.c
 delete mode 100644 drivers/staging/lustre/lustre/llite/lcommon_cl.c
 delete mode 100644 drivers/staging/lustre/lustre/llite/lcommon_misc.c
 delete mode 100644 drivers/staging/lustre/lustre/llite/llite_internal.h
 delete mode 100644 drivers/staging/lustre/lustre/llite/llite_lib.c
 delete mode 100644 drivers/staging/lustre/lustre/llite/llite_mmap.c
 delete mode 100644 drivers/staging/lustre/lustre/llite/llite_nfs.c
 delete mode 100644 drivers/staging/lustre/lustre/llite/lproc_llite.c
 delete mode 100644 drivers/staging/lustre/lustre/llite/namei.c
 delete mode 100644 drivers/staging/lustre/lustre/llite/range_lock.c
 delete mode 100644 drivers/staging/lustre/lustre/llite/range_lock.h
 delete mode 100644 drivers/staging/lustre/lustre/llite/rw.c
 delete mode 100644 drivers/staging/lustre/lustre/llite/rw26.c
 delete mode 100644 drivers/staging/lustre/lustre/llite/statahead.c
 delete mode 100644 drivers/staging/lustre/lustre/llite/super25.c
 delete mode 100644 drivers/staging/lustre/lustre/llite/symlink.c
 delete mode 100644 drivers/staging/lustre/lustre/llite/vvp_dev.c
 delete mode 100644 drivers/staging/lustre/lustre/llite/vvp_internal.h
 delete mode 100644 drivers/staging/lustre/lustre/llite/vvp_io.c
 delete mode 100644 drivers/staging/lustre/lustre/llite/vvp_lock.c
 delete mode 100644 drivers/staging/lustre/lustre/llite/vvp_object.c
 delete mode 100644 drivers/staging/lustre/lustre/llite/vvp_page.c
 delete mode 100644 drivers/staging/lustre/lustre/llite/xattr.c
 delete mode 100644 drivers/staging/lustre/lustre/llite/xattr_cache.c
 delete mode 100644 drivers/staging/lustre/lustre/llite/xattr_security.c
 delete mode 100644 drivers/staging/lustre/lustre/lmv/Makefile
 delete mode 100644 drivers/staging/lustre/lustre/lmv/lmv_fld.c
 delete mode 100644 drivers/staging/lustre/lustre/lmv/lmv_intent.c
 delete mode 100644 drivers/staging/lustre/lustre/lmv/lmv_internal.h
 delete mode 100644 drivers/staging/lustre/lustre/lmv/lmv_obd.c
 delete mode 100644 drivers/staging/lustre/lustre/lmv/lproc_lmv.c
 delete mode 100644 drivers/staging/lustre/lustre/lov/Makefile
 delete mode 100644 drivers/staging/lustre/lustre/lov/lov_cl_internal.h
 delete mode 100644 drivers/staging/lustre/lustre/lov/lov_dev.c
 delete mode 100644 drivers/staging/lustre/lustre/lov/lov_ea.c
 delete mode 100644 drivers/staging/lustre/lustre/lov/lov_internal.h
 delete mode 100644 drivers/staging/lustre/lustre/lov/lov_io.c
 delete mode 100644 drivers/staging/lustre/lustre/lov/lov_lock.c
 delete mode 100644 drivers/staging/lustre/lustre/lov/lov_merge.c
 delete mode 100644 drivers/staging/lustre/lustre/lov/lov_obd.c
 delete mode 100644 drivers/staging/lustre/lustre/lov/lov_object.c
 delete mode 100644 drivers/staging/lustre/lustre/lov/lov_offset.c
 delete mode 100644 drivers/staging/lustre/lustre/lov/lov_pack.c
 delete mode 100644 drivers/staging/lustre/lustre/lov/lov_page.c
 delete mode 100644 drivers/staging/lustre/lustre/lov/lov_pool.c
 delete mode 100644 drivers/staging/lustre/lustre/lov/lov_request.c
 delete mode 100644 drivers/staging/lustre/lustre/lov/lovsub_dev.c
 delete mode 100644 drivers/staging/lustre/lustre/lov/lovsub_lock.c
 delete mode 100644 drivers/staging/lustre/lustre/lov/lovsub_object.c
 delete mode 100644 drivers/staging/lustre/lustre/lov/lovsub_page.c
 delete mode 100644 drivers/staging/lustre/lustre/lov/lproc_lov.c
 delete mode 100644 drivers/staging/lustre/lustre/mdc/Makefile
 delete mode 100644 drivers/staging/lustre/lustre/mdc/lproc_mdc.c
 delete mode 100644 drivers/staging/lustre/lustre/mdc/mdc_internal.h
 delete mode 100644 drivers/staging/lustre/lustre/mdc/mdc_lib.c
 delete mode 100644 drivers/staging/lustre/lustre/mdc/mdc_locks.c
 delete mode 100644 drivers/staging/lustre/lustre/mdc/mdc_reint.c
 delete mode 100644 drivers/staging/lustre/lustre/mdc/mdc_request.c
 delete mode 100644 drivers/staging/lustre/lustre/mgc/Makefile
 delete mode 100644 drivers/staging/lustre/lustre/mgc/lproc_mgc.c
 delete mode 100644 drivers/staging/lustre/lustre/mgc/mgc_internal.h
 delete mode 100644 drivers/staging/lustre/lustre/mgc/mgc_request.c
 delete mode 100644 drivers/staging/lustre/lustre/obdclass/Makefile
 delete mode 100644 drivers/staging/lustre/lustre/obdclass/cl_internal.h
 delete mode 100644 drivers/staging/lustre/lustre/obdclass/cl_io.c
 delete mode 100644 drivers/staging/lustre/lustre/obdclass/cl_lock.c
 delete mode 100644 drivers/staging/lustre/lustre/obdclass/cl_object.c
 delete mode 100644 drivers/staging/lustre/lustre/obdclass/cl_page.c
 delete mode 100644 drivers/staging/lustre/lustre/obdclass/class_obd.c
 delete mode 100644 drivers/staging/lustre/lustre/obdclass/debug.c
 delete mode 100644 drivers/staging/lustre/lustre/obdclass/genops.c
 delete mode 100644 drivers/staging/lustre/lustre/obdclass/kernelcomm.c
 delete mode 100644 drivers/staging/lustre/lustre/obdclass/linkea.c
 delete mode 100644 drivers/staging/lustre/lustre/obdclass/linux/linux-module.c
 delete mode 100644 drivers/staging/lustre/lustre/obdclass/linux/linux-sysctl.c
 delete mode 100644 drivers/staging/lustre/lustre/obdclass/llog.c
 delete mode 100644 drivers/staging/lustre/lustre/obdclass/llog_cat.c
 delete mode 100644 drivers/staging/lustre/lustre/obdclass/llog_internal.h
 delete mode 100644 drivers/staging/lustre/lustre/obdclass/llog_obd.c
 delete mode 100644 drivers/staging/lustre/lustre/obdclass/llog_swab.c
 delete mode 100644 drivers/staging/lustre/lustre/obdclass/lprocfs_counters.c
 delete mode 100644 drivers/staging/lustre/lustre/obdclass/lprocfs_status.c
 delete mode 100644 drivers/staging/lustre/lustre/obdclass/lu_object.c
 delete mode 100644 drivers/staging/lustre/lustre/obdclass/lu_ref.c
 delete mode 100644 drivers/staging/lustre/lustre/obdclass/lustre_handles.c
 delete mode 100644 drivers/staging/lustre/lustre/obdclass/lustre_peer.c
 delete mode 100644 drivers/staging/lustre/lustre/obdclass/obd_config.c
 delete mode 100644 drivers/staging/lustre/lustre/obdclass/obd_mount.c
 delete mode 100644 drivers/staging/lustre/lustre/obdclass/obdo.c
 delete mode 100644 drivers/staging/lustre/lustre/obdclass/statfs_pack.c
 delete mode 100644 drivers/staging/lustre/lustre/obdclass/uuid.c
 delete mode 100644 drivers/staging/lustre/lustre/obdecho/Makefile
 delete mode 100644 drivers/staging/lustre/lustre/obdecho/echo_client.c
 delete mode 100644 drivers/staging/lustre/lustre/obdecho/echo_internal.h
 delete mode 100644 drivers/staging/lustre/lustre/osc/Makefile
 delete mode 100644 drivers/staging/lustre/lustre/osc/lproc_osc.c
 delete mode 100644 drivers/staging/lustre/lustre/osc/osc_cache.c
 delete mode 100644 drivers/staging/lustre/lustre/osc/osc_cl_internal.h
 delete mode 100644 drivers/staging/lustre/lustre/osc/osc_dev.c
 delete mode 100644 drivers/staging/lustre/lustre/osc/osc_internal.h
 delete mode 100644 drivers/staging/lustre/lustre/osc/osc_io.c
 delete mode 100644 drivers/staging/lustre/lustre/osc/osc_lock.c
 delete mode 100644 drivers/staging/lustre/lustre/osc/osc_object.c
 delete mode 100644 drivers/staging/lustre/lustre/osc/osc_page.c
 delete mode 100644 drivers/staging/lustre/lustre/osc/osc_quota.c
 delete mode 100644 drivers/staging/lustre/lustre/osc/osc_request.c
 delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/Makefile
 delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/client.c
 delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/connection.c
 delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/errno.c
 delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/events.c
 delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/import.c
 delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/layout.c
 delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/llog_client.c
 delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/llog_net.c
 delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/lproc_ptlrpc.c
 delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/niobuf.c
 delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/nrs.c
 delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/nrs_fifo.c
 delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/pack_generic.c
 delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/pers.c
 delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/pinger.c
 delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/ptlrpc_internal.h
 delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/ptlrpc_module.c
 delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c
 delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/recover.c
 delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/sec.c
 delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/sec_bulk.c
 delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/sec_config.c
 delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/sec_gc.c
 delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/sec_lproc.c
 delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/sec_null.c
 delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/sec_plain.c
 delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/service.c
 delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/wiretest.c
 delete mode 100644 drivers/staging/lustre/sysfs-fs-lustre

diff --git a/MAINTAINERS b/MAINTAINERS
index 4b65225d443a..db158767de20 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -13329,15 +13329,6 @@ S:	Odd Fixes
 F:	Documentation/devicetree/bindings/staging/iio/
 F:	drivers/staging/iio/
 
-STAGING - LUSTRE PARALLEL FILESYSTEM
-M:	Oleg Drokin <oleg.drokin at intel.com>
-M:	Andreas Dilger <andreas.dilger at intel.com>
-M:	James Simmons <jsimmons at infradead.org>
-L:	lustre-devel at lists.lustre.org (moderated for non-subscribers)
-W:	http://wiki.lustre.org/
-S:	Maintained
-F:	drivers/staging/lustre
-
 STAGING - NVIDIA COMPLIANT EMBEDDED CONTROLLER INTERFACE (nvec)
 M:	Marc Dietrich <marvin24 at gmx.de>
 L:	ac100 at lists.launchpad.net (moderated for non-subscribers)
diff --git a/drivers/staging/Kconfig b/drivers/staging/Kconfig
index d5926f0d3f6c..1c357ef669ae 100644
--- a/drivers/staging/Kconfig
+++ b/drivers/staging/Kconfig
@@ -84,8 +84,6 @@ source "drivers/staging/netlogic/Kconfig"
 
 source "drivers/staging/mt29f_spinand/Kconfig"
 
-source "drivers/staging/lustre/Kconfig"
-
 source "drivers/staging/dgnc/Kconfig"
 
 source "drivers/staging/gs_fpgaboot/Kconfig"
diff --git a/drivers/staging/Makefile b/drivers/staging/Makefile
index 919753c3d3f6..2edb9860931e 100644
--- a/drivers/staging/Makefile
+++ b/drivers/staging/Makefile
@@ -32,7 +32,6 @@ obj-$(CONFIG_STAGING_BOARD)	+= board/
 obj-$(CONFIG_LTE_GDM724X)	+= gdm724x/
 obj-$(CONFIG_FIREWIRE_SERIAL)	+= fwserial/
 obj-$(CONFIG_GOLDFISH)		+= goldfish/
-obj-$(CONFIG_LNET)		+= lustre/
 obj-$(CONFIG_DGNC)			+= dgnc/
 obj-$(CONFIG_MTD_SPINAND_MT29F)	+= mt29f_spinand/
 obj-$(CONFIG_GS_FPGABOOT)	+= gs_fpgaboot/
diff --git a/drivers/staging/lustre/Kconfig b/drivers/staging/lustre/Kconfig
deleted file mode 100644
index b7d81096eee9..000000000000
diff --git a/drivers/staging/lustre/Makefile b/drivers/staging/lustre/Makefile
deleted file mode 100644
index 95ffe337a80a..000000000000
diff --git a/drivers/staging/lustre/README.txt b/drivers/staging/lustre/README.txt
deleted file mode 100644
index 783959240490..000000000000
diff --git a/drivers/staging/lustre/TODO b/drivers/staging/lustre/TODO
deleted file mode 100644
index 5332cdb19bfa..000000000000
diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs.h b/drivers/staging/lustre/include/linux/libcfs/libcfs.h
deleted file mode 100644
index edc7ed0dcb94..000000000000
diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h
deleted file mode 100644
index 61641c41c492..000000000000
diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_crypto.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_crypto.h
deleted file mode 100644
index 176fae7319e3..000000000000
diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_debug.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_debug.h
deleted file mode 100644
index 17534a76362a..000000000000
diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_fail.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_fail.h
deleted file mode 100644
index 8074e390b4d1..000000000000
diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_hash.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_hash.h
deleted file mode 100644
index be315958a4b3..000000000000
diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_private.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_private.h
deleted file mode 100644
index 491d5971d199..000000000000
diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_string.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_string.h
deleted file mode 100644
index cd7c3ccb2dc0..000000000000
diff --git a/drivers/staging/lustre/include/linux/lnet/api.h b/drivers/staging/lustre/include/linux/lnet/api.h
deleted file mode 100644
index dae2e4f0056c..000000000000
diff --git a/drivers/staging/lustre/include/linux/lnet/lib-lnet.h b/drivers/staging/lustre/include/linux/lnet/lib-lnet.h
deleted file mode 100644
index 973c17a1c4a1..000000000000
diff --git a/drivers/staging/lustre/include/linux/lnet/lib-types.h b/drivers/staging/lustre/include/linux/lnet/lib-types.h
deleted file mode 100644
index cfe8ee424e94..000000000000
diff --git a/drivers/staging/lustre/include/linux/lnet/socklnd.h b/drivers/staging/lustre/include/linux/lnet/socklnd.h
deleted file mode 100644
index 6bd1bca190a3..000000000000
diff --git a/drivers/staging/lustre/include/uapi/linux/lnet/libcfs_debug.h b/drivers/staging/lustre/include/uapi/linux/lnet/libcfs_debug.h
deleted file mode 100644
index c4d9472b374f..000000000000
diff --git a/drivers/staging/lustre/include/uapi/linux/lnet/libcfs_ioctl.h b/drivers/staging/lustre/include/uapi/linux/lnet/libcfs_ioctl.h
deleted file mode 100644
index cce6b58e3682..000000000000
diff --git a/drivers/staging/lustre/include/uapi/linux/lnet/lnet-dlc.h b/drivers/staging/lustre/include/uapi/linux/lnet/lnet-dlc.h
deleted file mode 100644
index c1619f411d81..000000000000
diff --git a/drivers/staging/lustre/include/uapi/linux/lnet/lnet-types.h b/drivers/staging/lustre/include/uapi/linux/lnet/lnet-types.h
deleted file mode 100644
index 1be9b7aa7326..000000000000
diff --git a/drivers/staging/lustre/include/uapi/linux/lnet/lnetctl.h b/drivers/staging/lustre/include/uapi/linux/lnet/lnetctl.h
deleted file mode 100644
index cccb32dd28f2..000000000000
diff --git a/drivers/staging/lustre/include/uapi/linux/lnet/lnetst.h b/drivers/staging/lustre/include/uapi/linux/lnet/lnetst.h
deleted file mode 100644
index a4f9ff01d458..000000000000
diff --git a/drivers/staging/lustre/include/uapi/linux/lnet/nidstr.h b/drivers/staging/lustre/include/uapi/linux/lnet/nidstr.h
deleted file mode 100644
index 882074ed6021..000000000000
diff --git a/drivers/staging/lustre/include/uapi/linux/lnet/socklnd.h b/drivers/staging/lustre/include/uapi/linux/lnet/socklnd.h
deleted file mode 100644
index 6453e053fa99..000000000000
diff --git a/drivers/staging/lustre/include/uapi/linux/lustre/lustre_cfg.h b/drivers/staging/lustre/include/uapi/linux/lustre/lustre_cfg.h
deleted file mode 100644
index 11b51d93f64c..000000000000
diff --git a/drivers/staging/lustre/include/uapi/linux/lustre/lustre_fid.h b/drivers/staging/lustre/include/uapi/linux/lustre/lustre_fid.h
deleted file mode 100644
index 2e7a8d103777..000000000000
diff --git a/drivers/staging/lustre/include/uapi/linux/lustre/lustre_fiemap.h b/drivers/staging/lustre/include/uapi/linux/lustre/lustre_fiemap.h
deleted file mode 100644
index d375a476f5ea..000000000000
diff --git a/drivers/staging/lustre/include/uapi/linux/lustre/lustre_idl.h b/drivers/staging/lustre/include/uapi/linux/lustre/lustre_idl.h
deleted file mode 100644
index 6c7e3992d646..000000000000
diff --git a/drivers/staging/lustre/include/uapi/linux/lustre/lustre_ioctl.h b/drivers/staging/lustre/include/uapi/linux/lustre/lustre_ioctl.h
deleted file mode 100644
index 6e4e109fb874..000000000000
diff --git a/drivers/staging/lustre/include/uapi/linux/lustre/lustre_kernelcomm.h b/drivers/staging/lustre/include/uapi/linux/lustre/lustre_kernelcomm.h
deleted file mode 100644
index 94dadbe8e069..000000000000
diff --git a/drivers/staging/lustre/include/uapi/linux/lustre/lustre_ostid.h b/drivers/staging/lustre/include/uapi/linux/lustre/lustre_ostid.h
deleted file mode 100644
index 3343b602219b..000000000000
diff --git a/drivers/staging/lustre/include/uapi/linux/lustre/lustre_param.h b/drivers/staging/lustre/include/uapi/linux/lustre/lustre_param.h
deleted file mode 100644
index 1eab2ceca338..000000000000
diff --git a/drivers/staging/lustre/include/uapi/linux/lustre/lustre_user.h b/drivers/staging/lustre/include/uapi/linux/lustre/lustre_user.h
deleted file mode 100644
index 69387f36d1f1..000000000000
diff --git a/drivers/staging/lustre/include/uapi/linux/lustre/lustre_ver.h b/drivers/staging/lustre/include/uapi/linux/lustre/lustre_ver.h
deleted file mode 100644
index 19c9135e2273..000000000000
diff --git a/drivers/staging/lustre/lnet/Kconfig b/drivers/staging/lustre/lnet/Kconfig
deleted file mode 100644
index ad049e6f24e4..000000000000
diff --git a/drivers/staging/lustre/lnet/Makefile b/drivers/staging/lustre/lnet/Makefile
deleted file mode 100644
index 0a380fe88ce8..000000000000
diff --git a/drivers/staging/lustre/lnet/klnds/Makefile b/drivers/staging/lustre/lnet/klnds/Makefile
deleted file mode 100644
index c23e4f67f837..000000000000
diff --git a/drivers/staging/lustre/lnet/klnds/o2iblnd/Makefile b/drivers/staging/lustre/lnet/klnds/o2iblnd/Makefile
deleted file mode 100644
index 4affe1d79948..000000000000
diff --git a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c
deleted file mode 100644
index f0b4eb42bc1d..000000000000
diff --git a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.h b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.h
deleted file mode 100644
index 217503f125bc..000000000000
diff --git a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c
deleted file mode 100644
index 65b7a62943ad..000000000000
diff --git a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd_modparams.c b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd_modparams.c
deleted file mode 100644
index 39d07926d603..000000000000
diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/Makefile b/drivers/staging/lustre/lnet/klnds/socklnd/Makefile
deleted file mode 100644
index a7da1abfc804..000000000000
diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c
deleted file mode 100644
index f01b34ac1a53..000000000000
diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.h b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.h
deleted file mode 100644
index 4e5c89a692a3..000000000000
diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c
deleted file mode 100644
index 01b31a6bb588..000000000000
diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_lib.c b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_lib.c
deleted file mode 100644
index 93a02cd6b6b5..000000000000
diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_modparams.c b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_modparams.c
deleted file mode 100644
index 5663a4ca94d4..000000000000
diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_proto.c b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_proto.c
deleted file mode 100644
index 05982dac781c..000000000000
diff --git a/drivers/staging/lustre/lnet/libcfs/Makefile b/drivers/staging/lustre/lnet/libcfs/Makefile
deleted file mode 100644
index 6a1b232da495..000000000000
diff --git a/drivers/staging/lustre/lnet/libcfs/debug.c b/drivers/staging/lustre/lnet/libcfs/debug.c
deleted file mode 100644
index 06f694f6a28f..000000000000
diff --git a/drivers/staging/lustre/lnet/libcfs/fail.c b/drivers/staging/lustre/lnet/libcfs/fail.c
deleted file mode 100644
index bd86b3b5bc34..000000000000
diff --git a/drivers/staging/lustre/lnet/libcfs/hash.c b/drivers/staging/lustre/lnet/libcfs/hash.c
deleted file mode 100644
index 48be66f0d654..000000000000
diff --git a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
deleted file mode 100644
index 3d1cf457b286..000000000000
diff --git a/drivers/staging/lustre/lnet/libcfs/libcfs_lock.c b/drivers/staging/lustre/lnet/libcfs/libcfs_lock.c
deleted file mode 100644
index 223505c37545..000000000000
diff --git a/drivers/staging/lustre/lnet/libcfs/libcfs_mem.c b/drivers/staging/lustre/lnet/libcfs/libcfs_mem.c
deleted file mode 100644
index 2d533be9bb30..000000000000
diff --git a/drivers/staging/lustre/lnet/libcfs/libcfs_string.c b/drivers/staging/lustre/lnet/libcfs/libcfs_string.c
deleted file mode 100644
index e1fb1263e3ae..000000000000
diff --git a/drivers/staging/lustre/lnet/libcfs/linux-crypto-adler.c b/drivers/staging/lustre/lnet/libcfs/linux-crypto-adler.c
deleted file mode 100644
index db81ed527452..000000000000
diff --git a/drivers/staging/lustre/lnet/libcfs/linux-crypto.c b/drivers/staging/lustre/lnet/libcfs/linux-crypto.c
deleted file mode 100644
index 21ff9bf6da47..000000000000
diff --git a/drivers/staging/lustre/lnet/libcfs/linux-crypto.h b/drivers/staging/lustre/lnet/libcfs/linux-crypto.h
deleted file mode 100644
index 5616e9ea1450..000000000000
diff --git a/drivers/staging/lustre/lnet/libcfs/linux-debug.c b/drivers/staging/lustre/lnet/libcfs/linux-debug.c
deleted file mode 100644
index 15ab849374c2..000000000000
diff --git a/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c b/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
deleted file mode 100644
index 347138409eba..000000000000
diff --git a/drivers/staging/lustre/lnet/libcfs/module.c b/drivers/staging/lustre/lnet/libcfs/module.c
deleted file mode 100644
index 5dc7de9e6478..000000000000
diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.c b/drivers/staging/lustre/lnet/libcfs/tracefile.c
deleted file mode 100644
index 7ca562e156f0..000000000000
diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.h b/drivers/staging/lustre/lnet/libcfs/tracefile.h
deleted file mode 100644
index 0608240d897f..000000000000
diff --git a/drivers/staging/lustre/lnet/lnet/Makefile b/drivers/staging/lustre/lnet/lnet/Makefile
deleted file mode 100644
index 0a9d70924fe0..000000000000
diff --git a/drivers/staging/lustre/lnet/lnet/acceptor.c b/drivers/staging/lustre/lnet/lnet/acceptor.c
deleted file mode 100644
index 5648f17eddc0..000000000000
diff --git a/drivers/staging/lustre/lnet/lnet/api-ni.c b/drivers/staging/lustre/lnet/lnet/api-ni.c
deleted file mode 100644
index f9ed6977056c..000000000000
diff --git a/drivers/staging/lustre/lnet/lnet/config.c b/drivers/staging/lustre/lnet/lnet/config.c
deleted file mode 100644
index 55ecc1998b7e..000000000000
diff --git a/drivers/staging/lustre/lnet/lnet/lib-eq.c b/drivers/staging/lustre/lnet/lnet/lib-eq.c
deleted file mode 100644
index c78e70373ab4..000000000000
diff --git a/drivers/staging/lustre/lnet/lnet/lib-md.c b/drivers/staging/lustre/lnet/lnet/lib-md.c
deleted file mode 100644
index 8a22514aaf71..000000000000
diff --git a/drivers/staging/lustre/lnet/lnet/lib-me.c b/drivers/staging/lustre/lnet/lnet/lib-me.c
deleted file mode 100644
index 672e37bdd045..000000000000
diff --git a/drivers/staging/lustre/lnet/lnet/lib-move.c b/drivers/staging/lustre/lnet/lnet/lib-move.c
deleted file mode 100644
index f8eaf8ff8d8d..000000000000
diff --git a/drivers/staging/lustre/lnet/lnet/lib-msg.c b/drivers/staging/lustre/lnet/lnet/lib-msg.c
deleted file mode 100644
index 0091273c04b9..000000000000
diff --git a/drivers/staging/lustre/lnet/lnet/lib-ptl.c b/drivers/staging/lustre/lnet/lnet/lib-ptl.c
deleted file mode 100644
index fc47379c5938..000000000000
diff --git a/drivers/staging/lustre/lnet/lnet/lib-socket.c b/drivers/staging/lustre/lnet/lnet/lib-socket.c
deleted file mode 100644
index 9b61260155f2..000000000000
diff --git a/drivers/staging/lustre/lnet/lnet/lo.c b/drivers/staging/lustre/lnet/lnet/lo.c
deleted file mode 100644
index 7456b989e451..000000000000
diff --git a/drivers/staging/lustre/lnet/lnet/module.c b/drivers/staging/lustre/lnet/lnet/module.c
deleted file mode 100644
index 9d06664f0c17..000000000000
diff --git a/drivers/staging/lustre/lnet/lnet/net_fault.c b/drivers/staging/lustre/lnet/lnet/net_fault.c
deleted file mode 100644
index 0066394b0bb0..000000000000
diff --git a/drivers/staging/lustre/lnet/lnet/nidstrings.c b/drivers/staging/lustre/lnet/lnet/nidstrings.c
deleted file mode 100644
index 0f6c3fa16c65..000000000000
diff --git a/drivers/staging/lustre/lnet/lnet/peer.c b/drivers/staging/lustre/lnet/lnet/peer.c
deleted file mode 100644
index 58294149f7b2..000000000000
diff --git a/drivers/staging/lustre/lnet/lnet/router.c b/drivers/staging/lustre/lnet/lnet/router.c
deleted file mode 100644
index 6267d5e4bbd6..000000000000
diff --git a/drivers/staging/lustre/lnet/lnet/router_proc.c b/drivers/staging/lustre/lnet/lnet/router_proc.c
deleted file mode 100644
index ae4b7f5953a0..000000000000
diff --git a/drivers/staging/lustre/lnet/selftest/Makefile b/drivers/staging/lustre/lnet/selftest/Makefile
deleted file mode 100644
index 3ccc8966b566..000000000000
diff --git a/drivers/staging/lustre/lnet/selftest/brw_test.c b/drivers/staging/lustre/lnet/selftest/brw_test.c
deleted file mode 100644
index f1ee219bc8f3..000000000000
diff --git a/drivers/staging/lustre/lnet/selftest/conctl.c b/drivers/staging/lustre/lnet/selftest/conctl.c
deleted file mode 100644
index 906d82d90c0c..000000000000
diff --git a/drivers/staging/lustre/lnet/selftest/conrpc.c b/drivers/staging/lustre/lnet/selftest/conrpc.c
deleted file mode 100644
index 0dabade3d091..000000000000
diff --git a/drivers/staging/lustre/lnet/selftest/conrpc.h b/drivers/staging/lustre/lnet/selftest/conrpc.h
deleted file mode 100644
index ce2f92d04838..000000000000
diff --git a/drivers/staging/lustre/lnet/selftest/console.c b/drivers/staging/lustre/lnet/selftest/console.c
deleted file mode 100644
index 3c1c1b5997e0..000000000000
diff --git a/drivers/staging/lustre/lnet/selftest/console.h b/drivers/staging/lustre/lnet/selftest/console.h
deleted file mode 100644
index 2826205e36a1..000000000000
diff --git a/drivers/staging/lustre/lnet/selftest/framework.c b/drivers/staging/lustre/lnet/selftest/framework.c
deleted file mode 100644
index 741af10560ad..000000000000
diff --git a/drivers/staging/lustre/lnet/selftest/module.c b/drivers/staging/lustre/lnet/selftest/module.c
deleted file mode 100644
index 9ba65320f748..000000000000
diff --git a/drivers/staging/lustre/lnet/selftest/ping_test.c b/drivers/staging/lustre/lnet/selftest/ping_test.c
deleted file mode 100644
index f54bd630dbf8..000000000000
diff --git a/drivers/staging/lustre/lnet/selftest/rpc.c b/drivers/staging/lustre/lnet/selftest/rpc.c
deleted file mode 100644
index 9613b0a77007..000000000000
diff --git a/drivers/staging/lustre/lnet/selftest/rpc.h b/drivers/staging/lustre/lnet/selftest/rpc.h
deleted file mode 100644
index 465b5b534423..000000000000
diff --git a/drivers/staging/lustre/lnet/selftest/selftest.h b/drivers/staging/lustre/lnet/selftest/selftest.h
deleted file mode 100644
index 8737fa96b192..000000000000
diff --git a/drivers/staging/lustre/lnet/selftest/timer.c b/drivers/staging/lustre/lnet/selftest/timer.c
deleted file mode 100644
index 582f252b3e12..000000000000
diff --git a/drivers/staging/lustre/lnet/selftest/timer.h b/drivers/staging/lustre/lnet/selftest/timer.h
deleted file mode 100644
index 7f0ef9bd0cda..000000000000
diff --git a/drivers/staging/lustre/lustre/Kconfig b/drivers/staging/lustre/lustre/Kconfig
deleted file mode 100644
index ccb78a945995..000000000000
diff --git a/drivers/staging/lustre/lustre/Makefile b/drivers/staging/lustre/lustre/Makefile
deleted file mode 100644
index 331e4fcdd5a2..000000000000
diff --git a/drivers/staging/lustre/lustre/fid/Makefile b/drivers/staging/lustre/lustre/fid/Makefile
deleted file mode 100644
index 77b65b92667d..000000000000
diff --git a/drivers/staging/lustre/lustre/fid/fid_internal.h b/drivers/staging/lustre/lustre/fid/fid_internal.h
deleted file mode 100644
index 14569e969a31..000000000000
diff --git a/drivers/staging/lustre/lustre/fid/fid_lib.c b/drivers/staging/lustre/lustre/fid/fid_lib.c
deleted file mode 100644
index ac52b378c155..000000000000
diff --git a/drivers/staging/lustre/lustre/fid/fid_request.c b/drivers/staging/lustre/lustre/fid/fid_request.c
deleted file mode 100644
index a34fd90ca5e5..000000000000
diff --git a/drivers/staging/lustre/lustre/fid/lproc_fid.c b/drivers/staging/lustre/lustre/fid/lproc_fid.c
deleted file mode 100644
index 0aabf473c9bd..000000000000
diff --git a/drivers/staging/lustre/lustre/fld/Makefile b/drivers/staging/lustre/lustre/fld/Makefile
deleted file mode 100644
index 426deba8b815..000000000000
diff --git a/drivers/staging/lustre/lustre/fld/fld_cache.c b/drivers/staging/lustre/lustre/fld/fld_cache.c
deleted file mode 100644
index a7415c9a1c28..000000000000
diff --git a/drivers/staging/lustre/lustre/fld/fld_internal.h b/drivers/staging/lustre/lustre/fld/fld_internal.h
deleted file mode 100644
index e1d6aaa5c2b4..000000000000
diff --git a/drivers/staging/lustre/lustre/fld/fld_request.c b/drivers/staging/lustre/lustre/fld/fld_request.c
deleted file mode 100644
index 97f7ea632346..000000000000
diff --git a/drivers/staging/lustre/lustre/fld/lproc_fld.c b/drivers/staging/lustre/lustre/fld/lproc_fld.c
deleted file mode 100644
index 0bcfb26ef8aa..000000000000
diff --git a/drivers/staging/lustre/lustre/include/cl_object.h b/drivers/staging/lustre/lustre/include/cl_object.h
deleted file mode 100644
index 6f7b991be809..000000000000
diff --git a/drivers/staging/lustre/lustre/include/interval_tree.h b/drivers/staging/lustre/lustre/include/interval_tree.h
deleted file mode 100644
index 7d119c1a0469..000000000000
diff --git a/drivers/staging/lustre/lustre/include/llog_swab.h b/drivers/staging/lustre/lustre/include/llog_swab.h
deleted file mode 100644
index 0433b79efdcb..000000000000
diff --git a/drivers/staging/lustre/lustre/include/lprocfs_status.h b/drivers/staging/lustre/lustre/include/lprocfs_status.h
deleted file mode 100644
index 495e6f5f676b..000000000000
diff --git a/drivers/staging/lustre/lustre/include/lu_object.h b/drivers/staging/lustre/lustre/include/lu_object.h
deleted file mode 100644
index f29bbca5af65..000000000000
diff --git a/drivers/staging/lustre/lustre/include/lu_ref.h b/drivers/staging/lustre/lustre/include/lu_ref.h
deleted file mode 100644
index ad0c24d29ffa..000000000000
diff --git a/drivers/staging/lustre/lustre/include/lustre_acl.h b/drivers/staging/lustre/lustre/include/lustre_acl.h
deleted file mode 100644
index e7575a172b5f..000000000000
diff --git a/drivers/staging/lustre/lustre/include/lustre_compat.h b/drivers/staging/lustre/lustre/include/lustre_compat.h
deleted file mode 100644
index 3c6db0d632dc..000000000000
diff --git a/drivers/staging/lustre/lustre/include/lustre_debug.h b/drivers/staging/lustre/lustre/include/lustre_debug.h
deleted file mode 100644
index 721a81f923e3..000000000000
diff --git a/drivers/staging/lustre/lustre/include/lustre_disk.h b/drivers/staging/lustre/lustre/include/lustre_disk.h
deleted file mode 100644
index 886e817644d6..000000000000
diff --git a/drivers/staging/lustre/lustre/include/lustre_dlm.h b/drivers/staging/lustre/lustre/include/lustre_dlm.h
deleted file mode 100644
index 2c55241258cc..000000000000
diff --git a/drivers/staging/lustre/lustre/include/lustre_dlm_flags.h b/drivers/staging/lustre/lustre/include/lustre_dlm_flags.h
deleted file mode 100644
index 53db031c4c8c..000000000000
diff --git a/drivers/staging/lustre/lustre/include/lustre_errno.h b/drivers/staging/lustre/lustre/include/lustre_errno.h
deleted file mode 100644
index 59fbb9f47ff1..000000000000
diff --git a/drivers/staging/lustre/lustre/include/lustre_export.h b/drivers/staging/lustre/lustre/include/lustre_export.h
deleted file mode 100644
index 79ad5aae86b9..000000000000
diff --git a/drivers/staging/lustre/lustre/include/lustre_fid.h b/drivers/staging/lustre/lustre/include/lustre_fid.h
deleted file mode 100644
index 094ad282de2c..000000000000
diff --git a/drivers/staging/lustre/lustre/include/lustre_fld.h b/drivers/staging/lustre/lustre/include/lustre_fld.h
deleted file mode 100644
index f42122a4dfaa..000000000000
diff --git a/drivers/staging/lustre/lustre/include/lustre_ha.h b/drivers/staging/lustre/lustre/include/lustre_ha.h
deleted file mode 100644
index cbd68985ada9..000000000000
diff --git a/drivers/staging/lustre/lustre/include/lustre_handles.h b/drivers/staging/lustre/lustre/include/lustre_handles.h
deleted file mode 100644
index 3556ce8d94e8..000000000000
diff --git a/drivers/staging/lustre/lustre/include/lustre_import.h b/drivers/staging/lustre/lustre/include/lustre_import.h
deleted file mode 100644
index ac3805ead620..000000000000
diff --git a/drivers/staging/lustre/lustre/include/lustre_intent.h b/drivers/staging/lustre/lustre/include/lustre_intent.h
deleted file mode 100644
index 51e5c0e03872..000000000000
diff --git a/drivers/staging/lustre/lustre/include/lustre_kernelcomm.h b/drivers/staging/lustre/lustre/include/lustre_kernelcomm.h
deleted file mode 100644
index 2b3fa8430185..000000000000
diff --git a/drivers/staging/lustre/lustre/include/lustre_lib.h b/drivers/staging/lustre/lustre/include/lustre_lib.h
deleted file mode 100644
index 87748e9902a7..000000000000
diff --git a/drivers/staging/lustre/lustre/include/lustre_linkea.h b/drivers/staging/lustre/lustre/include/lustre_linkea.h
deleted file mode 100644
index 03db1511bfd3..000000000000
diff --git a/drivers/staging/lustre/lustre/include/lustre_lmv.h b/drivers/staging/lustre/lustre/include/lustre_lmv.h
deleted file mode 100644
index 080ec1f8e19f..000000000000
diff --git a/drivers/staging/lustre/lustre/include/lustre_log.h b/drivers/staging/lustre/lustre/include/lustre_log.h
deleted file mode 100644
index 07f4e600386b..000000000000
diff --git a/drivers/staging/lustre/lustre/include/lustre_mdc.h b/drivers/staging/lustre/lustre/include/lustre_mdc.h
deleted file mode 100644
index a9c9992a2502..000000000000
diff --git a/drivers/staging/lustre/lustre/include/lustre_mds.h b/drivers/staging/lustre/lustre/include/lustre_mds.h
deleted file mode 100644
index f665556556ec..000000000000
diff --git a/drivers/staging/lustre/lustre/include/lustre_net.h b/drivers/staging/lustre/lustre/include/lustre_net.h
deleted file mode 100644
index 35b43a77eb18..000000000000
diff --git a/drivers/staging/lustre/lustre/include/lustre_nrs.h b/drivers/staging/lustre/lustre/include/lustre_nrs.h
deleted file mode 100644
index ffa7317da35b..000000000000
diff --git a/drivers/staging/lustre/lustre/include/lustre_nrs_fifo.h b/drivers/staging/lustre/lustre/include/lustre_nrs_fifo.h
deleted file mode 100644
index b70d97d4acbb..000000000000
diff --git a/drivers/staging/lustre/lustre/include/lustre_obdo.h b/drivers/staging/lustre/lustre/include/lustre_obdo.h
deleted file mode 100644
index d67dcbb84f18..000000000000
diff --git a/drivers/staging/lustre/lustre/include/lustre_patchless_compat.h b/drivers/staging/lustre/lustre/include/lustre_patchless_compat.h
deleted file mode 100644
index 298476ea7557..000000000000
diff --git a/drivers/staging/lustre/lustre/include/lustre_req_layout.h b/drivers/staging/lustre/lustre/include/lustre_req_layout.h
deleted file mode 100644
index 213d0a01adcf..000000000000
diff --git a/drivers/staging/lustre/lustre/include/lustre_sec.h b/drivers/staging/lustre/lustre/include/lustre_sec.h
deleted file mode 100644
index d35bcbc98831..000000000000
diff --git a/drivers/staging/lustre/lustre/include/lustre_swab.h b/drivers/staging/lustre/lustre/include/lustre_swab.h
deleted file mode 100644
index 9d786bbe7f3f..000000000000
diff --git a/drivers/staging/lustre/lustre/include/obd.h b/drivers/staging/lustre/lustre/include/obd.h
deleted file mode 100644
index b1907bbffb19..000000000000
diff --git a/drivers/staging/lustre/lustre/include/obd_cksum.h b/drivers/staging/lustre/lustre/include/obd_cksum.h
deleted file mode 100644
index e5f7bb20415d..000000000000
diff --git a/drivers/staging/lustre/lustre/include/obd_class.h b/drivers/staging/lustre/lustre/include/obd_class.h
deleted file mode 100644
index fc9c7720fee0..000000000000
diff --git a/drivers/staging/lustre/lustre/include/obd_support.h b/drivers/staging/lustre/lustre/include/obd_support.h
deleted file mode 100644
index 9e41633823f7..000000000000
diff --git a/drivers/staging/lustre/lustre/include/seq_range.h b/drivers/staging/lustre/lustre/include/seq_range.h
deleted file mode 100644
index 9450da728160..000000000000
diff --git a/drivers/staging/lustre/lustre/ldlm/interval_tree.c b/drivers/staging/lustre/lustre/ldlm/interval_tree.c
deleted file mode 100644
index 8df7a4463c21..000000000000
diff --git a/drivers/staging/lustre/lustre/ldlm/l_lock.c b/drivers/staging/lustre/lustre/ldlm/l_lock.c
deleted file mode 100644
index 296259aa51e6..000000000000
diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_extent.c b/drivers/staging/lustre/lustre/ldlm/ldlm_extent.c
deleted file mode 100644
index 4da23ade2bb3..000000000000
diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_flock.c b/drivers/staging/lustre/lustre/ldlm/ldlm_flock.c
deleted file mode 100644
index 94f3b1e49896..000000000000
diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_inodebits.c b/drivers/staging/lustre/lustre/ldlm/ldlm_inodebits.c
deleted file mode 100644
index 2926208cdfa1..000000000000
diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_internal.h b/drivers/staging/lustre/lustre/ldlm/ldlm_internal.h
deleted file mode 100644
index bc33ca100620..000000000000
diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_lib.c b/drivers/staging/lustre/lustre/ldlm/ldlm_lib.c
deleted file mode 100644
index 0aa4f234a4f4..000000000000
diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c b/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c
deleted file mode 100644
index a644d133063b..000000000000
diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c b/drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c
deleted file mode 100644
index 5963e90d0938..000000000000
diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_plain.c b/drivers/staging/lustre/lustre/ldlm/ldlm_plain.c
deleted file mode 100644
index 33b5a3f96fcb..000000000000
diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c b/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c
deleted file mode 100644
index 36d14ee4e5b1..000000000000
diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_request.c b/drivers/staging/lustre/lustre/ldlm/ldlm_request.c
deleted file mode 100644
index cdc52eed6d85..000000000000
diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_resource.c b/drivers/staging/lustre/lustre/ldlm/ldlm_resource.c
deleted file mode 100644
index c93b019b8e37..000000000000
diff --git a/drivers/staging/lustre/lustre/llite/Makefile b/drivers/staging/lustre/lustre/llite/Makefile
deleted file mode 100644
index 5200924182ae..000000000000
diff --git a/drivers/staging/lustre/lustre/llite/acl.c b/drivers/staging/lustre/lustre/llite/acl.c
deleted file mode 100644
index 2ee9ff931236..000000000000
diff --git a/drivers/staging/lustre/lustre/llite/dcache.c b/drivers/staging/lustre/lustre/llite/dcache.c
deleted file mode 100644
index 11b82c639bfe..000000000000
diff --git a/drivers/staging/lustre/lustre/llite/dir.c b/drivers/staging/lustre/lustre/llite/dir.c
deleted file mode 100644
index 688dddf3ca47..000000000000
diff --git a/drivers/staging/lustre/lustre/llite/file.c b/drivers/staging/lustre/lustre/llite/file.c
deleted file mode 100644
index 02295931883b..000000000000
diff --git a/drivers/staging/lustre/lustre/llite/glimpse.c b/drivers/staging/lustre/lustre/llite/glimpse.c
deleted file mode 100644
index ce0d51767da3..000000000000
diff --git a/drivers/staging/lustre/lustre/llite/lcommon_cl.c b/drivers/staging/lustre/lustre/llite/lcommon_cl.c
deleted file mode 100644
index d7ea39ce0cb2..000000000000
diff --git a/drivers/staging/lustre/lustre/llite/lcommon_misc.c b/drivers/staging/lustre/lustre/llite/lcommon_misc.c
deleted file mode 100644
index a246b955306e..000000000000
diff --git a/drivers/staging/lustre/lustre/llite/llite_internal.h b/drivers/staging/lustre/lustre/llite/llite_internal.h
deleted file mode 100644
index c08a6e14b6d7..000000000000
diff --git a/drivers/staging/lustre/lustre/llite/llite_lib.c b/drivers/staging/lustre/lustre/llite/llite_lib.c
deleted file mode 100644
index 36066c839160..000000000000
diff --git a/drivers/staging/lustre/lustre/llite/llite_mmap.c b/drivers/staging/lustre/lustre/llite/llite_mmap.c
deleted file mode 100644
index d7fb5533f707..000000000000
diff --git a/drivers/staging/lustre/lustre/llite/llite_nfs.c b/drivers/staging/lustre/lustre/llite/llite_nfs.c
deleted file mode 100644
index 14172688d55f..000000000000
diff --git a/drivers/staging/lustre/lustre/llite/lproc_llite.c b/drivers/staging/lustre/lustre/llite/lproc_llite.c
deleted file mode 100644
index 49bf1b7ee311..000000000000
diff --git a/drivers/staging/lustre/lustre/llite/namei.c b/drivers/staging/lustre/lustre/llite/namei.c
deleted file mode 100644
index d5f6d20afe8c..000000000000
diff --git a/drivers/staging/lustre/lustre/llite/range_lock.c b/drivers/staging/lustre/lustre/llite/range_lock.c
deleted file mode 100644
index 008a8874118d..000000000000
diff --git a/drivers/staging/lustre/lustre/llite/range_lock.h b/drivers/staging/lustre/lustre/llite/range_lock.h
deleted file mode 100644
index 9ebac09160f2..000000000000
diff --git a/drivers/staging/lustre/lustre/llite/rw.c b/drivers/staging/lustre/lustre/llite/rw.c
deleted file mode 100644
index 3e008ce7275d..000000000000
diff --git a/drivers/staging/lustre/lustre/llite/rw26.c b/drivers/staging/lustre/lustre/llite/rw26.c
deleted file mode 100644
index 722e5ea1af5f..000000000000
diff --git a/drivers/staging/lustre/lustre/llite/statahead.c b/drivers/staging/lustre/lustre/llite/statahead.c
deleted file mode 100644
index d864f5f36d85..000000000000
diff --git a/drivers/staging/lustre/lustre/llite/super25.c b/drivers/staging/lustre/lustre/llite/super25.c
deleted file mode 100644
index d335f29556c2..000000000000
diff --git a/drivers/staging/lustre/lustre/llite/symlink.c b/drivers/staging/lustre/lustre/llite/symlink.c
deleted file mode 100644
index 0690fdbf49f5..000000000000
diff --git a/drivers/staging/lustre/lustre/llite/vvp_dev.c b/drivers/staging/lustre/lustre/llite/vvp_dev.c
deleted file mode 100644
index 31dc3c0ade01..000000000000
diff --git a/drivers/staging/lustre/lustre/llite/vvp_internal.h b/drivers/staging/lustre/lustre/llite/vvp_internal.h
deleted file mode 100644
index 7d3abb43584a..000000000000
diff --git a/drivers/staging/lustre/lustre/llite/vvp_io.c b/drivers/staging/lustre/lustre/llite/vvp_io.c
deleted file mode 100644
index e7a4778e02e4..000000000000
diff --git a/drivers/staging/lustre/lustre/llite/vvp_lock.c b/drivers/staging/lustre/lustre/llite/vvp_lock.c
deleted file mode 100644
index 4b6c7143bd2c..000000000000
diff --git a/drivers/staging/lustre/lustre/llite/vvp_object.c b/drivers/staging/lustre/lustre/llite/vvp_object.c
deleted file mode 100644
index b2cb51c8f7f4..000000000000
diff --git a/drivers/staging/lustre/lustre/llite/vvp_page.c b/drivers/staging/lustre/lustre/llite/vvp_page.c
deleted file mode 100644
index 6eb0565ddc22..000000000000
diff --git a/drivers/staging/lustre/lustre/llite/xattr.c b/drivers/staging/lustre/lustre/llite/xattr.c
deleted file mode 100644
index 7fa0a419c094..000000000000
diff --git a/drivers/staging/lustre/lustre/llite/xattr_cache.c b/drivers/staging/lustre/lustre/llite/xattr_cache.c
deleted file mode 100644
index 5da69ba088c4..000000000000
diff --git a/drivers/staging/lustre/lustre/llite/xattr_security.c b/drivers/staging/lustre/lustre/llite/xattr_security.c
deleted file mode 100644
index 93ec07531ac7..000000000000
diff --git a/drivers/staging/lustre/lustre/lmv/Makefile b/drivers/staging/lustre/lustre/lmv/Makefile
deleted file mode 100644
index 91c99114aa13..000000000000
diff --git a/drivers/staging/lustre/lustre/lmv/lmv_fld.c b/drivers/staging/lustre/lustre/lmv/lmv_fld.c
deleted file mode 100644
index 00dc858c10c9..000000000000
diff --git a/drivers/staging/lustre/lustre/lmv/lmv_intent.c b/drivers/staging/lustre/lustre/lmv/lmv_intent.c
deleted file mode 100644
index 1e850fdbc623..000000000000
diff --git a/drivers/staging/lustre/lustre/lmv/lmv_internal.h b/drivers/staging/lustre/lustre/lmv/lmv_internal.h
deleted file mode 100644
index 68a99170c424..000000000000
diff --git a/drivers/staging/lustre/lustre/lmv/lmv_obd.c b/drivers/staging/lustre/lustre/lmv/lmv_obd.c
deleted file mode 100644
index 65f94e6ecaad..000000000000
diff --git a/drivers/staging/lustre/lustre/lmv/lproc_lmv.c b/drivers/staging/lustre/lustre/lmv/lproc_lmv.c
deleted file mode 100644
index 30727b7acccc..000000000000
diff --git a/drivers/staging/lustre/lustre/lov/Makefile b/drivers/staging/lustre/lustre/lov/Makefile
deleted file mode 100644
index 1ebf0193f61a..000000000000
diff --git a/drivers/staging/lustre/lustre/lov/lov_cl_internal.h b/drivers/staging/lustre/lustre/lov/lov_cl_internal.h
deleted file mode 100644
index e4f762137a4a..000000000000
diff --git a/drivers/staging/lustre/lustre/lov/lov_dev.c b/drivers/staging/lustre/lustre/lov/lov_dev.c
deleted file mode 100644
index c7db23472346..000000000000
diff --git a/drivers/staging/lustre/lustre/lov/lov_ea.c b/drivers/staging/lustre/lustre/lov/lov_ea.c
deleted file mode 100644
index c80320ab0858..000000000000
diff --git a/drivers/staging/lustre/lustre/lov/lov_internal.h b/drivers/staging/lustre/lustre/lov/lov_internal.h
deleted file mode 100644
index 47042f27ca90..000000000000
diff --git a/drivers/staging/lustre/lustre/lov/lov_io.c b/drivers/staging/lustre/lustre/lov/lov_io.c
deleted file mode 100644
index b823f8a21856..000000000000
diff --git a/drivers/staging/lustre/lustre/lov/lov_lock.c b/drivers/staging/lustre/lustre/lov/lov_lock.c
deleted file mode 100644
index b0292100bf26..000000000000
diff --git a/drivers/staging/lustre/lustre/lov/lov_merge.c b/drivers/staging/lustre/lustre/lov/lov_merge.c
deleted file mode 100644
index 006717cf7a41..000000000000
diff --git a/drivers/staging/lustre/lustre/lov/lov_obd.c b/drivers/staging/lustre/lustre/lov/lov_obd.c
deleted file mode 100644
index 344ff4b20168..000000000000
diff --git a/drivers/staging/lustre/lustre/lov/lov_object.c b/drivers/staging/lustre/lustre/lov/lov_object.c
deleted file mode 100644
index adc90f310fd7..000000000000
diff --git a/drivers/staging/lustre/lustre/lov/lov_offset.c b/drivers/staging/lustre/lustre/lov/lov_offset.c
deleted file mode 100644
index a5f00f6ec347..000000000000
diff --git a/drivers/staging/lustre/lustre/lov/lov_pack.c b/drivers/staging/lustre/lustre/lov/lov_pack.c
deleted file mode 100644
index b1060d02a164..000000000000
diff --git a/drivers/staging/lustre/lustre/lov/lov_page.c b/drivers/staging/lustre/lustre/lov/lov_page.c
deleted file mode 100644
index cfae1294d77a..000000000000
diff --git a/drivers/staging/lustre/lustre/lov/lov_pool.c b/drivers/staging/lustre/lustre/lov/lov_pool.c
deleted file mode 100644
index b2a88ba72eb2..000000000000
diff --git a/drivers/staging/lustre/lustre/lov/lov_request.c b/drivers/staging/lustre/lustre/lov/lov_request.c
deleted file mode 100644
index cb8567f20ea7..000000000000
diff --git a/drivers/staging/lustre/lustre/lov/lovsub_dev.c b/drivers/staging/lustre/lustre/lov/lovsub_dev.c
deleted file mode 100644
index 7e89a2e485fc..000000000000
diff --git a/drivers/staging/lustre/lustre/lov/lovsub_lock.c b/drivers/staging/lustre/lustre/lov/lovsub_lock.c
deleted file mode 100644
index ea492be2eef3..000000000000
diff --git a/drivers/staging/lustre/lustre/lov/lovsub_object.c b/drivers/staging/lustre/lustre/lov/lovsub_object.c
deleted file mode 100644
index 13d452086b61..000000000000
diff --git a/drivers/staging/lustre/lustre/lov/lovsub_page.c b/drivers/staging/lustre/lustre/lov/lovsub_page.c
deleted file mode 100644
index 915520bcdd60..000000000000
diff --git a/drivers/staging/lustre/lustre/lov/lproc_lov.c b/drivers/staging/lustre/lustre/lov/lproc_lov.c
deleted file mode 100644
index 721440feef72..000000000000
diff --git a/drivers/staging/lustre/lustre/mdc/Makefile b/drivers/staging/lustre/lustre/mdc/Makefile
deleted file mode 100644
index c7bc3351ccb0..000000000000
diff --git a/drivers/staging/lustre/lustre/mdc/lproc_mdc.c b/drivers/staging/lustre/lustre/mdc/lproc_mdc.c
deleted file mode 100644
index 6cce32491eb5..000000000000
diff --git a/drivers/staging/lustre/lustre/mdc/mdc_internal.h b/drivers/staging/lustre/lustre/mdc/mdc_internal.h
deleted file mode 100644
index 28924e927b50..000000000000
diff --git a/drivers/staging/lustre/lustre/mdc/mdc_lib.c b/drivers/staging/lustre/lustre/mdc/mdc_lib.c
deleted file mode 100644
index d582968987ff..000000000000
diff --git a/drivers/staging/lustre/lustre/mdc/mdc_locks.c b/drivers/staging/lustre/lustre/mdc/mdc_locks.c
deleted file mode 100644
index a8aa0fa5e87a..000000000000
diff --git a/drivers/staging/lustre/lustre/mdc/mdc_reint.c b/drivers/staging/lustre/lustre/mdc/mdc_reint.c
deleted file mode 100644
index e77c00df0693..000000000000
diff --git a/drivers/staging/lustre/lustre/mdc/mdc_request.c b/drivers/staging/lustre/lustre/mdc/mdc_request.c
deleted file mode 100644
index cff31cb0a9ac..000000000000
diff --git a/drivers/staging/lustre/lustre/mgc/Makefile b/drivers/staging/lustre/lustre/mgc/Makefile
deleted file mode 100644
index 8abf108dbcf7..000000000000
diff --git a/drivers/staging/lustre/lustre/mgc/lproc_mgc.c b/drivers/staging/lustre/lustre/mgc/lproc_mgc.c
deleted file mode 100644
index 636770624e8f..000000000000
diff --git a/drivers/staging/lustre/lustre/mgc/mgc_internal.h b/drivers/staging/lustre/lustre/mgc/mgc_internal.h
deleted file mode 100644
index 9541892b67c7..000000000000
diff --git a/drivers/staging/lustre/lustre/mgc/mgc_request.c b/drivers/staging/lustre/lustre/mgc/mgc_request.c
deleted file mode 100644
index 32df804614d3..000000000000
diff --git a/drivers/staging/lustre/lustre/obdclass/Makefile b/drivers/staging/lustre/lustre/obdclass/Makefile
deleted file mode 100644
index e3fa9acff4c4..000000000000
diff --git a/drivers/staging/lustre/lustre/obdclass/cl_internal.h b/drivers/staging/lustre/lustre/obdclass/cl_internal.h
deleted file mode 100644
index a0db830ca841..000000000000
diff --git a/drivers/staging/lustre/lustre/obdclass/cl_io.c b/drivers/staging/lustre/lustre/obdclass/cl_io.c
deleted file mode 100644
index fcdae6029258..000000000000
diff --git a/drivers/staging/lustre/lustre/obdclass/cl_lock.c b/drivers/staging/lustre/lustre/obdclass/cl_lock.c
deleted file mode 100644
index 9ca29a26a38b..000000000000
diff --git a/drivers/staging/lustre/lustre/obdclass/cl_object.c b/drivers/staging/lustre/lustre/obdclass/cl_object.c
deleted file mode 100644
index 42cce2dc5a45..000000000000
diff --git a/drivers/staging/lustre/lustre/obdclass/cl_page.c b/drivers/staging/lustre/lustre/obdclass/cl_page.c
deleted file mode 100644
index 916cf81c5997..000000000000
diff --git a/drivers/staging/lustre/lustre/obdclass/class_obd.c b/drivers/staging/lustre/lustre/obdclass/class_obd.c
deleted file mode 100644
index d6c46858941b..000000000000
diff --git a/drivers/staging/lustre/lustre/obdclass/debug.c b/drivers/staging/lustre/lustre/obdclass/debug.c
deleted file mode 100644
index 2156a82a613a..000000000000
diff --git a/drivers/staging/lustre/lustre/obdclass/genops.c b/drivers/staging/lustre/lustre/obdclass/genops.c
deleted file mode 100644
index 234f383ce6d9..000000000000
diff --git a/drivers/staging/lustre/lustre/obdclass/kernelcomm.c b/drivers/staging/lustre/lustre/obdclass/kernelcomm.c
deleted file mode 100644
index 63067a7f1e19..000000000000
diff --git a/drivers/staging/lustre/lustre/obdclass/linkea.c b/drivers/staging/lustre/lustre/obdclass/linkea.c
deleted file mode 100644
index 74c99ee216bb..000000000000
diff --git a/drivers/staging/lustre/lustre/obdclass/linux/linux-module.c b/drivers/staging/lustre/lustre/obdclass/linux/linux-module.c
deleted file mode 100644
index 9c800580053b..000000000000
diff --git a/drivers/staging/lustre/lustre/obdclass/linux/linux-sysctl.c b/drivers/staging/lustre/lustre/obdclass/linux/linux-sysctl.c
deleted file mode 100644
index e5e8687784ee..000000000000
diff --git a/drivers/staging/lustre/lustre/obdclass/llog.c b/drivers/staging/lustre/lustre/obdclass/llog.c
deleted file mode 100644
index bba84eae1e19..000000000000
diff --git a/drivers/staging/lustre/lustre/obdclass/llog_cat.c b/drivers/staging/lustre/lustre/obdclass/llog_cat.c
deleted file mode 100644
index d9c63adff206..000000000000
diff --git a/drivers/staging/lustre/lustre/obdclass/llog_internal.h b/drivers/staging/lustre/lustre/obdclass/llog_internal.h
deleted file mode 100644
index 4991d4e589dc..000000000000
diff --git a/drivers/staging/lustre/lustre/obdclass/llog_obd.c b/drivers/staging/lustre/lustre/obdclass/llog_obd.c
deleted file mode 100644
index 26aea114a29b..000000000000
diff --git a/drivers/staging/lustre/lustre/obdclass/llog_swab.c b/drivers/staging/lustre/lustre/obdclass/llog_swab.c
deleted file mode 100644
index b431c3408fe4..000000000000
diff --git a/drivers/staging/lustre/lustre/obdclass/lprocfs_counters.c b/drivers/staging/lustre/lustre/obdclass/lprocfs_counters.c
deleted file mode 100644
index 85f09aff6e83..000000000000
diff --git a/drivers/staging/lustre/lustre/obdclass/lprocfs_status.c b/drivers/staging/lustre/lustre/obdclass/lprocfs_status.c
deleted file mode 100644
index bdbe6f52031a..000000000000
diff --git a/drivers/staging/lustre/lustre/obdclass/lu_object.c b/drivers/staging/lustre/lustre/obdclass/lu_object.c
deleted file mode 100644
index aa9d74e087f4..000000000000
diff --git a/drivers/staging/lustre/lustre/obdclass/lu_ref.c b/drivers/staging/lustre/lustre/obdclass/lu_ref.c
deleted file mode 100644
index f67cb89ea0ba..000000000000
diff --git a/drivers/staging/lustre/lustre/obdclass/lustre_handles.c b/drivers/staging/lustre/lustre/obdclass/lustre_handles.c
deleted file mode 100644
index cdc8dc10690d..000000000000
diff --git a/drivers/staging/lustre/lustre/obdclass/lustre_peer.c b/drivers/staging/lustre/lustre/obdclass/lustre_peer.c
deleted file mode 100644
index e286a2665423..000000000000
diff --git a/drivers/staging/lustre/lustre/obdclass/obd_config.c b/drivers/staging/lustre/lustre/obdclass/obd_config.c
deleted file mode 100644
index ffc1814398a5..000000000000
diff --git a/drivers/staging/lustre/lustre/obdclass/obd_mount.c b/drivers/staging/lustre/lustre/obdclass/obd_mount.c
deleted file mode 100644
index 06c38fdef7ba..000000000000
diff --git a/drivers/staging/lustre/lustre/obdclass/obdo.c b/drivers/staging/lustre/lustre/obdclass/obdo.c
deleted file mode 100644
index c4503bc36591..000000000000
diff --git a/drivers/staging/lustre/lustre/obdclass/statfs_pack.c b/drivers/staging/lustre/lustre/obdclass/statfs_pack.c
deleted file mode 100644
index 355e888885f4..000000000000
diff --git a/drivers/staging/lustre/lustre/obdclass/uuid.c b/drivers/staging/lustre/lustre/obdclass/uuid.c
deleted file mode 100644
index ec8c6dc5c9a7..000000000000
diff --git a/drivers/staging/lustre/lustre/obdecho/Makefile b/drivers/staging/lustre/lustre/obdecho/Makefile
deleted file mode 100644
index 6be66fbab872..000000000000
diff --git a/drivers/staging/lustre/lustre/obdecho/echo_client.c b/drivers/staging/lustre/lustre/obdecho/echo_client.c
deleted file mode 100644
index b692e76e7108..000000000000
diff --git a/drivers/staging/lustre/lustre/obdecho/echo_internal.h b/drivers/staging/lustre/lustre/obdecho/echo_internal.h
deleted file mode 100644
index 42faa164fabb..000000000000
diff --git a/drivers/staging/lustre/lustre/osc/Makefile b/drivers/staging/lustre/lustre/osc/Makefile
deleted file mode 100644
index 30dec90e64e8..000000000000
diff --git a/drivers/staging/lustre/lustre/osc/lproc_osc.c b/drivers/staging/lustre/lustre/osc/lproc_osc.c
deleted file mode 100644
index 6a705bc5420c..000000000000
diff --git a/drivers/staging/lustre/lustre/osc/osc_cache.c b/drivers/staging/lustre/lustre/osc/osc_cache.c
deleted file mode 100644
index f26983004843..000000000000
diff --git a/drivers/staging/lustre/lustre/osc/osc_cl_internal.h b/drivers/staging/lustre/lustre/osc/osc_cl_internal.h
deleted file mode 100644
index 2d3cba16ef34..000000000000
diff --git a/drivers/staging/lustre/lustre/osc/osc_dev.c b/drivers/staging/lustre/lustre/osc/osc_dev.c
deleted file mode 100644
index 2b5f324743e2..000000000000
diff --git a/drivers/staging/lustre/lustre/osc/osc_internal.h b/drivers/staging/lustre/lustre/osc/osc_internal.h
deleted file mode 100644
index 4ddba1354bef..000000000000
diff --git a/drivers/staging/lustre/lustre/osc/osc_io.c b/drivers/staging/lustre/lustre/osc/osc_io.c
deleted file mode 100644
index 67734a8ed331..000000000000
diff --git a/drivers/staging/lustre/lustre/osc/osc_lock.c b/drivers/staging/lustre/lustre/osc/osc_lock.c
deleted file mode 100644
index d93d33dc8dc4..000000000000
diff --git a/drivers/staging/lustre/lustre/osc/osc_object.c b/drivers/staging/lustre/lustre/osc/osc_object.c
deleted file mode 100644
index 84240181c7ea..000000000000
diff --git a/drivers/staging/lustre/lustre/osc/osc_page.c b/drivers/staging/lustre/lustre/osc/osc_page.c
deleted file mode 100644
index 20c553ef3a5e..000000000000
diff --git a/drivers/staging/lustre/lustre/osc/osc_quota.c b/drivers/staging/lustre/lustre/osc/osc_quota.c
deleted file mode 100644
index 723ec2fb18bf..000000000000
diff --git a/drivers/staging/lustre/lustre/osc/osc_request.c b/drivers/staging/lustre/lustre/osc/osc_request.c
deleted file mode 100644
index 61ef6c8d7a12..000000000000
diff --git a/drivers/staging/lustre/lustre/ptlrpc/Makefile b/drivers/staging/lustre/lustre/ptlrpc/Makefile
deleted file mode 100644
index 1deb1971b39e..000000000000
diff --git a/drivers/staging/lustre/lustre/ptlrpc/client.c b/drivers/staging/lustre/lustre/ptlrpc/client.c
deleted file mode 100644
index c1b82bf20f08..000000000000
diff --git a/drivers/staging/lustre/lustre/ptlrpc/connection.c b/drivers/staging/lustre/lustre/ptlrpc/connection.c
deleted file mode 100644
index fb35a89ca6c6..000000000000
diff --git a/drivers/staging/lustre/lustre/ptlrpc/errno.c b/drivers/staging/lustre/lustre/ptlrpc/errno.c
deleted file mode 100644
index b904524fc1c6..000000000000
diff --git a/drivers/staging/lustre/lustre/ptlrpc/events.c b/drivers/staging/lustre/lustre/ptlrpc/events.c
deleted file mode 100644
index 130bacc2c891..000000000000
diff --git a/drivers/staging/lustre/lustre/ptlrpc/import.c b/drivers/staging/lustre/lustre/ptlrpc/import.c
deleted file mode 100644
index 1a0f35dfab97..000000000000
diff --git a/drivers/staging/lustre/lustre/ptlrpc/layout.c b/drivers/staging/lustre/lustre/ptlrpc/layout.c
deleted file mode 100644
index 417d4a151433..000000000000
diff --git a/drivers/staging/lustre/lustre/ptlrpc/llog_client.c b/drivers/staging/lustre/lustre/ptlrpc/llog_client.c
deleted file mode 100644
index 946d538121de..000000000000
diff --git a/drivers/staging/lustre/lustre/ptlrpc/llog_net.c b/drivers/staging/lustre/lustre/ptlrpc/llog_net.c
deleted file mode 100644
index b871d9e40a9e..000000000000
diff --git a/drivers/staging/lustre/lustre/ptlrpc/lproc_ptlrpc.c b/drivers/staging/lustre/lustre/ptlrpc/lproc_ptlrpc.c
deleted file mode 100644
index 0b638837f88b..000000000000
diff --git a/drivers/staging/lustre/lustre/ptlrpc/niobuf.c b/drivers/staging/lustre/lustre/ptlrpc/niobuf.c
deleted file mode 100644
index 2897afb8806c..000000000000
diff --git a/drivers/staging/lustre/lustre/ptlrpc/nrs.c b/drivers/staging/lustre/lustre/ptlrpc/nrs.c
deleted file mode 100644
index e09b86529c5d..000000000000
diff --git a/drivers/staging/lustre/lustre/ptlrpc/nrs_fifo.c b/drivers/staging/lustre/lustre/ptlrpc/nrs_fifo.c
deleted file mode 100644
index ff630d94dd26..000000000000
diff --git a/drivers/staging/lustre/lustre/ptlrpc/pack_generic.c b/drivers/staging/lustre/lustre/ptlrpc/pack_generic.c
deleted file mode 100644
index 6ac9bb570663..000000000000
diff --git a/drivers/staging/lustre/lustre/ptlrpc/pers.c b/drivers/staging/lustre/lustre/ptlrpc/pers.c
deleted file mode 100644
index 2466868afb9c..000000000000
diff --git a/drivers/staging/lustre/lustre/ptlrpc/pinger.c b/drivers/staging/lustre/lustre/ptlrpc/pinger.c
deleted file mode 100644
index b3297b5ce395..000000000000
diff --git a/drivers/staging/lustre/lustre/ptlrpc/ptlrpc_internal.h b/drivers/staging/lustre/lustre/ptlrpc/ptlrpc_internal.h
deleted file mode 100644
index 134b74234519..000000000000
diff --git a/drivers/staging/lustre/lustre/ptlrpc/ptlrpc_module.c b/drivers/staging/lustre/lustre/ptlrpc/ptlrpc_module.c
deleted file mode 100644
index 5c32b657b3b5..000000000000
diff --git a/drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c b/drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c
deleted file mode 100644
index 531005411edf..000000000000
diff --git a/drivers/staging/lustre/lustre/ptlrpc/recover.c b/drivers/staging/lustre/lustre/ptlrpc/recover.c
deleted file mode 100644
index 2ea0a7ff87dd..000000000000
diff --git a/drivers/staging/lustre/lustre/ptlrpc/sec.c b/drivers/staging/lustre/lustre/ptlrpc/sec.c
deleted file mode 100644
index e193f3346e6f..000000000000
diff --git a/drivers/staging/lustre/lustre/ptlrpc/sec_bulk.c b/drivers/staging/lustre/lustre/ptlrpc/sec_bulk.c
deleted file mode 100644
index 625b9520d78f..000000000000
diff --git a/drivers/staging/lustre/lustre/ptlrpc/sec_config.c b/drivers/staging/lustre/lustre/ptlrpc/sec_config.c
deleted file mode 100644
index 2389f9a8f534..000000000000
diff --git a/drivers/staging/lustre/lustre/ptlrpc/sec_gc.c b/drivers/staging/lustre/lustre/ptlrpc/sec_gc.c
deleted file mode 100644
index 2c8bad7b7877..000000000000
diff --git a/drivers/staging/lustre/lustre/ptlrpc/sec_lproc.c b/drivers/staging/lustre/lustre/ptlrpc/sec_lproc.c
deleted file mode 100644
index 2bb75ebd5d98..000000000000
diff --git a/drivers/staging/lustre/lustre/ptlrpc/sec_null.c b/drivers/staging/lustre/lustre/ptlrpc/sec_null.c
deleted file mode 100644
index ecc387d1b9b4..000000000000
diff --git a/drivers/staging/lustre/lustre/ptlrpc/sec_plain.c b/drivers/staging/lustre/lustre/ptlrpc/sec_plain.c
deleted file mode 100644
index ec3d9af76b17..000000000000
diff --git a/drivers/staging/lustre/lustre/ptlrpc/service.c b/drivers/staging/lustre/lustre/ptlrpc/service.c
deleted file mode 100644
index 3fd8c746f460..000000000000
diff --git a/drivers/staging/lustre/lustre/ptlrpc/wiretest.c b/drivers/staging/lustre/lustre/ptlrpc/wiretest.c
deleted file mode 100644
index f9394c3e1ee2..000000000000
diff --git a/drivers/staging/lustre/sysfs-fs-lustre b/drivers/staging/lustre/sysfs-fs-lustre
deleted file mode 100644
index 8691c6543a9c..000000000000
diff --git a/scripts/selinux/mdp/mdp.c b/scripts/selinux/mdp/mdp.c
index ffe8179f5d41..073fe7537f6c 100644
--- a/scripts/selinux/mdp/mdp.c
+++ b/scripts/selinux/mdp/mdp.c
@@ -124,7 +124,6 @@ int main(int argc, char *argv[])
 	fprintf(fout, "fs_use_xattr reiserfs user_u:base_r:base_t;\n");
 	fprintf(fout, "fs_use_xattr jffs2 user_u:base_r:base_t;\n");
 	fprintf(fout, "fs_use_xattr gfs2 user_u:base_r:base_t;\n");
-	fprintf(fout, "fs_use_xattr lustre user_u:base_r:base_t;\n");
 
 	fprintf(fout, "fs_use_task eventpollfs user_u:base_r:base_t;\n");
 	fprintf(fout, "fs_use_task pipefs user_u:base_r:base_t;\n");
-- 
2.17.1



From hch at infradead.org  Fri Jun  1 11:41:51 2018
From: hch at infradead.org (Christoph Hellwig)
Date: Fri, 1 Jun 2018 04:41:51 -0700
Subject: [lustre-devel] [PATCH] staging: lustre: delete the filesystem
 from the tree.
In-Reply-To: <20180601091133.GA27521@kroah.com>
References: <20180601091133.GA27521@kroah.com>
Message-ID: <20180601114151.GA25225@infradead.org>

Thanks,

all that churn without much visible progress to a mergeable codebase
was really ennoying.

I'd recommend if people want to merge lustre they start with a managable
subset first, e.g. the fs client code with simple IP-only networking.

From adilger at dilger.ca  Fri Jun  1 18:20:27 2018
From: adilger at dilger.ca (Andreas Dilger)
Date: Fri, 1 Jun 2018 14:20:27 -0400
Subject: [lustre-devel] [PATCH] staging: lustre: delete the filesystem
 from the tree.
In-Reply-To: <20180601114151.GA25225@infradead.org>
References: <20180601091133.GA27521@kroah.com>
 <20180601114151.GA25225@infradead.org>
Message-ID: <E5BF8501-35EB-4DDF-ADBE-6016CF31A502@dilger.ca>

On Jun 1, 2018, at 7:41 AM, Christoph Hellwig <hch at infradead.org> wrote:
> 
> Thanks,
> 
> all that churn without much visible progress to a mergeable codebase
> was really ennoying.
> 
> I'd recommend if people want to merge lustre they start with a managable
> subset first, e.g. the fs client code with simple IP-only networking.

Adding or removing the IB networking makes basically no difference to the
code size.  This would also make the client much less useful, since a large
number of sites use Lustre with IB networks.

Cheers, Andreas





-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 873 bytes
Desc: Message signed with OpenPGP
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180601/7f63d49e/attachment.sig>

From doucharek at cray.com  Fri Jun  1 18:25:24 2018
From: doucharek at cray.com (Doug Oucharek)
Date: Fri, 1 Jun 2018 18:25:24 +0000
Subject: [lustre-devel] [PATCH] staging: lustre: delete the filesystem
 from the tree.
In-Reply-To: <E5BF8501-35EB-4DDF-ADBE-6016CF31A502@dilger.ca>
References: <20180601091133.GA27521@kroah.com>
 <20180601114151.GA25225@infradead.org>
 <E5BF8501-35EB-4DDF-ADBE-6016CF31A502@dilger.ca>
Message-ID: <29ACF5A8-7608-46BB-8191-E3FEB77D0F24@cray.com>

Would it makes sense to land LNet and LNDs on their own first?  Get the networking house in order first before layering on the file system?

Doug

> On Jun 1, 2018, at 11:20 AM, Andreas Dilger <adilger at dilger.ca> wrote:
> 
> On Jun 1, 2018, at 7:41 AM, Christoph Hellwig <hch at infradead.org> wrote:
>> 
>> Thanks,
>> 
>> all that churn without much visible progress to a mergeable codebase
>> was really ennoying.
>> 
>> I'd recommend if people want to merge lustre they start with a managable
>> subset first, e.g. the fs client code with simple IP-only networking.
> 
> Adding or removing the IB networking makes basically no difference to the
> code size.  This would also make the client much less useful, since a large
> number of sites use Lustre with IB networks.
> 
> Cheers, Andreas
> 
> 
> 
> 
> 
> _______________________________________________
> lustre-devel mailing list
> lustre-devel at lists.lustre.org
> http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org


From adilger at dilger.ca  Fri Jun  1 18:30:49 2018
From: adilger at dilger.ca (Andreas Dilger)
Date: Fri, 1 Jun 2018 14:30:49 -0400
Subject: [lustre-devel] [PATCH] staging: lustre: delete the filesystem
 from the tree.
In-Reply-To: <20180601091133.GA27521@kroah.com>
References: <20180601091133.GA27521@kroah.com>
Message-ID: <6A5E4E02-AE72-4068-BF93-38791BF38B4F@dilger.ca>

On Jun 1, 2018, at 5:11 AM, Greg Kroah-Hartman <gregkh at linuxfoundation.org> wrote:
> 
> The Lustre filesystem has been in the kernel tree for over 5 years now.
> While it has been an endless source of enjoyment for new kernel
> developers learning how to do basic codingstyle cleanups, as well as an
> semi-entertaining source of bewilderment from the vfs developers any
> time they have looked into the codebase to try to figure out how to port
> their latest api changes to this filesystem, it has not really moved
> forward into the "this is in shape to get out of staging" despite many
> half-completed attempts.

I am happy to submit a patch that moves Lustre out of staging and into
the mainline.  I'm just about to board a flight, but it could be done
later today.  Then we can avoid the constant churn of kernel newbies
submitting patches that break the code.

Lustre has been in use at large sites around the world for 18 years now.
Over 70% of the largest 100 systems in the world use Lustre.  It runs at
universities, oil companies, weather bureaus around the world, etc.

I know Andrew has long been a supporter of getting code into the kernel
that users are using.  This is code that thousands of large computers
are using with exabytes of storage, a lot more than orangefs, exofs, and
random other filesystems that seem to get into the kernel easily.

It's true the code is not as pretty as it could be, but the same is true
of code that has been in the kernel for ages.  We've spent years cleaning
it up in staging, and it has gotten a lot better.  Getting the client into
the mainline kernel will accelerate the development and cleanup, and
Christoph will no longer be able to submit patches that break the code
because it is only in staging.  This will also (finally) allow us to get
this code in sync with the out-of-tree code and converge on a single tree.

Cheers, Andreas

> And getting code out of staging is the main goal of that portion of the
> kernel tree.  Code should not stagnate and it feels like having this
> code in staging is only causing the development cycle of the filesystem
> to take longer than it should.  There is a whole separate out-of-tree
> copy of this codebase where the developers work on it, and then random
> changes are thrown over the wall at staging at some later point in time.
> This dual-tree development model has never worked, and the state of this
> codebase is proof of that.
> 
> So, let's just delete the whole mess.  Now the lustre developers can go
> off and work in their out-of-tree codebase and not have to worry about
> providing valid changelog entries and breaking their patches up into
> logical pieces.  They can take the time they have spend doing those
> types of housekeeping chores and get the codebase into a much better
> shape, and it can be submitted for inclusion into the real part of the
> kernel tree when ready.
> 
> Signed-off-by: Greg Kroah-Hartman <gregkh at linuxfoundation.org>
> ---
> MAINTAINERS                                   |    9 -
> drivers/staging/Kconfig                       |    2 -
> drivers/staging/Makefile                      |    1 -
> drivers/staging/lustre/Kconfig                |    3 -
> drivers/staging/lustre/Makefile               |    2 -
> drivers/staging/lustre/README.txt             |   83 -
> drivers/staging/lustre/TODO                   |  302 --
> .../lustre/include/linux/libcfs/libcfs.h      |   76 -
> .../lustre/include/linux/libcfs/libcfs_cpu.h  |  434 --
> .../include/linux/libcfs/libcfs_crypto.h      |  208 -
> .../include/linux/libcfs/libcfs_debug.h       |  207 -
> .../lustre/include/linux/libcfs/libcfs_fail.h |  194 -
> .../lustre/include/linux/libcfs/libcfs_hash.h |  869 ----
> .../include/linux/libcfs/libcfs_private.h     |  200 -
> .../include/linux/libcfs/libcfs_string.h      |  102 -
> .../staging/lustre/include/linux/lnet/api.h   |  212 -
> .../lustre/include/linux/lnet/lib-lnet.h      |  652 ---
> .../lustre/include/linux/lnet/lib-types.h     |  666 ---
> .../lustre/include/linux/lnet/socklnd.h       |   87 -
> .../include/uapi/linux/lnet/libcfs_debug.h    |  149 -
> .../include/uapi/linux/lnet/libcfs_ioctl.h    |  141 -
> .../lustre/include/uapi/linux/lnet/lnet-dlc.h |  150 -
> .../include/uapi/linux/lnet/lnet-types.h      |  669 ---
> .../lustre/include/uapi/linux/lnet/lnetctl.h  |  123 -
> .../lustre/include/uapi/linux/lnet/lnetst.h   |  556 ---
> .../lustre/include/uapi/linux/lnet/nidstr.h   |  119 -
> .../lustre/include/uapi/linux/lnet/socklnd.h  |   44 -
> .../include/uapi/linux/lustre/lustre_cfg.h    |  261 -
> .../include/uapi/linux/lustre/lustre_fid.h    |  293 --
> .../include/uapi/linux/lustre/lustre_fiemap.h |   72 -
> .../include/uapi/linux/lustre/lustre_idl.h    | 2690 -----------
> .../include/uapi/linux/lustre/lustre_ioctl.h  |  229 -
> .../uapi/linux/lustre/lustre_kernelcomm.h     |   94 -
> .../include/uapi/linux/lustre/lustre_ostid.h  |  236 -
> .../include/uapi/linux/lustre/lustre_param.h  |   94 -
> .../include/uapi/linux/lustre/lustre_user.h   | 1327 ------
> .../include/uapi/linux/lustre/lustre_ver.h    |   27 -
> drivers/staging/lustre/lnet/Kconfig           |   46 -
> drivers/staging/lustre/lnet/Makefile          |    1 -
> drivers/staging/lustre/lnet/klnds/Makefile    |    1 -
> .../lustre/lnet/klnds/o2iblnd/Makefile        |    5 -
> .../lustre/lnet/klnds/o2iblnd/o2iblnd.c       | 2958 ------------
> .../lustre/lnet/klnds/o2iblnd/o2iblnd.h       | 1048 ----
> .../lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c    | 3763 ---------------
> .../lnet/klnds/o2iblnd/o2iblnd_modparams.c    |  296 --
> .../lustre/lnet/klnds/socklnd/Makefile        |    6 -
> .../lustre/lnet/klnds/socklnd/socklnd.c       | 2921 ------------
> .../lustre/lnet/klnds/socklnd/socklnd.h       |  704 ---
> .../lustre/lnet/klnds/socklnd/socklnd_cb.c    | 2586 ----------
> .../lustre/lnet/klnds/socklnd/socklnd_lib.c   |  534 ---
> .../lnet/klnds/socklnd/socklnd_modparams.c    |  184 -
> .../lustre/lnet/klnds/socklnd/socklnd_proto.c |  810 ----
> drivers/staging/lustre/lnet/libcfs/Makefile   |   16 -
> drivers/staging/lustre/lnet/libcfs/debug.c    |  461 --
> drivers/staging/lustre/lnet/libcfs/fail.c     |  146 -
> drivers/staging/lustre/lnet/libcfs/hash.c     | 2065 --------
> .../staging/lustre/lnet/libcfs/libcfs_cpu.c   | 1086 -----
> .../staging/lustre/lnet/libcfs/libcfs_lock.c  |  155 -
> .../staging/lustre/lnet/libcfs/libcfs_mem.c   |  171 -
> .../lustre/lnet/libcfs/libcfs_string.c        |  562 ---
> .../lustre/lnet/libcfs/linux-crypto-adler.c   |  139 -
> .../staging/lustre/lnet/libcfs/linux-crypto.c |  447 --
> .../staging/lustre/lnet/libcfs/linux-crypto.h |   30 -
> .../staging/lustre/lnet/libcfs/linux-debug.c  |  142 -
> .../lustre/lnet/libcfs/linux-tracefile.c      |  258 -
> drivers/staging/lustre/lnet/libcfs/module.c   |  758 ---
> .../staging/lustre/lnet/libcfs/tracefile.c    | 1198 -----
> .../staging/lustre/lnet/libcfs/tracefile.h    |  274 --
> drivers/staging/lustre/lnet/lnet/Makefile     |   10 -
> drivers/staging/lustre/lnet/lnet/acceptor.c   |  501 --
> drivers/staging/lustre/lnet/lnet/api-ni.c     | 2307 ---------
> drivers/staging/lustre/lnet/lnet/config.c     | 1235 -----
> drivers/staging/lustre/lnet/lnet/lib-eq.c     |  426 --
> drivers/staging/lustre/lnet/lnet/lib-md.c     |  463 --
> drivers/staging/lustre/lnet/lnet/lib-me.c     |  274 --
> drivers/staging/lustre/lnet/lnet/lib-move.c   | 2386 ----------
> drivers/staging/lustre/lnet/lnet/lib-msg.c    |  625 ---
> drivers/staging/lustre/lnet/lnet/lib-ptl.c    |  987 ----
> drivers/staging/lustre/lnet/lnet/lib-socket.c |  585 ---
> drivers/staging/lustre/lnet/lnet/lo.c         |  105 -
> drivers/staging/lustre/lnet/lnet/module.c     |  239 -
> drivers/staging/lustre/lnet/lnet/net_fault.c  | 1023 ----
> drivers/staging/lustre/lnet/lnet/nidstrings.c | 1261 -----
> drivers/staging/lustre/lnet/lnet/peer.c       |  456 --
> drivers/staging/lustre/lnet/lnet/router.c     | 1799 -------
> .../staging/lustre/lnet/lnet/router_proc.c    |  907 ----
> drivers/staging/lustre/lnet/selftest/Makefile |    7 -
> .../staging/lustre/lnet/selftest/brw_test.c   |  526 --
> drivers/staging/lustre/lnet/selftest/conctl.c |  801 ----
> drivers/staging/lustre/lnet/selftest/conrpc.c | 1396 ------
> drivers/staging/lustre/lnet/selftest/conrpc.h |  142 -
> .../staging/lustre/lnet/selftest/console.c    | 2104 --------
> .../staging/lustre/lnet/selftest/console.h    |  244 -
> .../staging/lustre/lnet/selftest/framework.c  | 1786 -------
> drivers/staging/lustre/lnet/selftest/module.c |  169 -
> .../staging/lustre/lnet/selftest/ping_test.c  |  228 -
> drivers/staging/lustre/lnet/selftest/rpc.c    | 1682 -------
> drivers/staging/lustre/lnet/selftest/rpc.h    |  295 --
> .../staging/lustre/lnet/selftest/selftest.h   |  622 ---
> drivers/staging/lustre/lnet/selftest/timer.c  |  244 -
> drivers/staging/lustre/lnet/selftest/timer.h  |   50 -
> drivers/staging/lustre/lustre/Kconfig         |   45 -
> drivers/staging/lustre/lustre/Makefile        |    2 -
> drivers/staging/lustre/lustre/fid/Makefile    |    5 -
> .../staging/lustre/lustre/fid/fid_internal.h  |   46 -
> drivers/staging/lustre/lustre/fid/fid_lib.c   |   87 -
> .../staging/lustre/lustre/fid/fid_request.c   |  410 --
> drivers/staging/lustre/lustre/fid/lproc_fid.c |  225 -
> drivers/staging/lustre/lustre/fld/Makefile    |    5 -
> drivers/staging/lustre/lustre/fld/fld_cache.c |  516 --
> .../staging/lustre/lustre/fld/fld_internal.h  |  170 -
> .../staging/lustre/lustre/fld/fld_request.c   |  446 --
> drivers/staging/lustre/lustre/fld/lproc_fld.c |  154 -
> .../staging/lustre/lustre/include/cl_object.h | 2463 ----------
> .../lustre/lustre/include/interval_tree.h     |  119 -
> .../staging/lustre/lustre/include/llog_swab.h |   67 -
> .../lustre/lustre/include/lprocfs_status.h    |  646 ---
> .../staging/lustre/lustre/include/lu_object.h | 1305 -----
> .../staging/lustre/lustre/include/lu_ref.h    |  178 -
> .../lustre/lustre/include/lustre_acl.h        |   51 -
> .../lustre/lustre/include/lustre_compat.h     |   82 -
> .../lustre/lustre/include/lustre_debug.h      |   52 -
> .../lustre/lustre/include/lustre_disk.h       |  152 -
> .../lustre/lustre/include/lustre_dlm.h        | 1346 ------
> .../lustre/lustre/include/lustre_dlm_flags.h  |  402 --
> .../lustre/lustre/include/lustre_errno.h      |  198 -
> .../lustre/lustre/include/lustre_export.h     |  250 -
> .../lustre/lustre/include/lustre_fid.h        |  676 ---
> .../lustre/lustre/include/lustre_fld.h        |  137 -
> .../staging/lustre/lustre/include/lustre_ha.h |   61 -
> .../lustre/lustre/include/lustre_handles.h    |   91 -
> .../lustre/lustre/include/lustre_import.h     |  369 --
> .../lustre/lustre/include/lustre_intent.h     |   71 -
> .../lustre/lustre/include/lustre_kernelcomm.h |   56 -
> .../lustre/lustre/include/lustre_lib.h        |  126 -
> .../lustre/lustre/include/lustre_linkea.h     |   93 -
> .../lustre/lustre/include/lustre_lmv.h        |  174 -
> .../lustre/lustre/include/lustre_log.h        |  382 --
> .../lustre/lustre/include/lustre_mdc.h        |  229 -
> .../lustre/lustre/include/lustre_mds.h        |   62 -
> .../lustre/lustre/include/lustre_net.h        | 2360 ---------
> .../lustre/lustre/include/lustre_nrs.h        |  718 ---
> .../lustre/lustre/include/lustre_nrs_fifo.h   |   71 -
> .../lustre/lustre/include/lustre_obdo.h       |   55 -
> .../lustre/include/lustre_patchless_compat.h  |   68 -
> .../lustre/lustre/include/lustre_req_layout.h |  307 --
> .../lustre/lustre/include/lustre_sec.h        | 1072 -----
> .../lustre/lustre/include/lustre_swab.h       |  109 -
> drivers/staging/lustre/lustre/include/obd.h   | 1114 -----
> .../staging/lustre/lustre/include/obd_cksum.h |  153 -
> .../staging/lustre/lustre/include/obd_class.h | 1603 -------
> .../lustre/lustre/include/obd_support.h       |  517 --
> .../staging/lustre/lustre/include/seq_range.h |  200 -
> .../lustre/lustre/ldlm/interval_tree.c        |  599 ---
> drivers/staging/lustre/lustre/ldlm/l_lock.c   |   73 -
> .../staging/lustre/lustre/ldlm/ldlm_extent.c  |  258 -
> .../staging/lustre/lustre/ldlm/ldlm_flock.c   |  486 --
> .../lustre/lustre/ldlm/ldlm_inodebits.c       |   69 -
> .../lustre/lustre/ldlm/ldlm_internal.h        |  342 --
> drivers/staging/lustre/lustre/ldlm/ldlm_lib.c |  842 ----
> .../staging/lustre/lustre/ldlm/ldlm_lock.c    | 2135 ---------
> .../staging/lustre/lustre/ldlm/ldlm_lockd.c   | 1163 -----
> .../staging/lustre/lustre/ldlm/ldlm_plain.c   |   68 -
> .../staging/lustre/lustre/ldlm/ldlm_pool.c    | 1013 ----
> .../staging/lustre/lustre/ldlm/ldlm_request.c | 2033 --------
> .../lustre/lustre/ldlm/ldlm_resource.c        | 1318 ------
> drivers/staging/lustre/lustre/llite/Makefile  |   13 -
> drivers/staging/lustre/lustre/llite/acl.c     |  108 -
> drivers/staging/lustre/lustre/llite/dcache.c  |  300 --
> drivers/staging/lustre/lustre/llite/dir.c     | 1708 -------
> drivers/staging/lustre/lustre/llite/file.c    | 3580 --------------
> drivers/staging/lustre/lustre/llite/glimpse.c |  205 -
> .../staging/lustre/lustre/llite/lcommon_cl.c  |  292 --
> .../lustre/lustre/llite/lcommon_misc.c        |  186 -
> .../lustre/lustre/llite/llite_internal.h      | 1344 ------
> .../staging/lustre/lustre/llite/llite_lib.c   | 2668 -----------
> .../staging/lustre/lustre/llite/llite_mmap.c  |  480 --
> .../staging/lustre/lustre/llite/llite_nfs.c   |  375 --
> .../staging/lustre/lustre/llite/lproc_llite.c | 1659 -------
> drivers/staging/lustre/lustre/llite/namei.c   | 1207 -----
> .../staging/lustre/lustre/llite/range_lock.c  |  241 -
> .../staging/lustre/lustre/llite/range_lock.h  |   83 -
> drivers/staging/lustre/lustre/llite/rw.c      | 1214 -----
> drivers/staging/lustre/lustre/llite/rw26.c    |  641 ---
> .../staging/lustre/lustre/llite/statahead.c   | 1577 ------
> drivers/staging/lustre/lustre/llite/super25.c |  189 -
> drivers/staging/lustre/lustre/llite/symlink.c |  159 -
> drivers/staging/lustre/lustre/llite/vvp_dev.c |  640 ---
> .../lustre/lustre/llite/vvp_internal.h        |  321 --
> drivers/staging/lustre/lustre/llite/vvp_io.c  | 1374 ------
> .../staging/lustre/lustre/llite/vvp_lock.c    |   87 -
> .../staging/lustre/lustre/llite/vvp_object.c  |  303 --
> .../staging/lustre/lustre/llite/vvp_page.c    |  523 --
> drivers/staging/lustre/lustre/llite/xattr.c   |  665 ---
> .../staging/lustre/lustre/llite/xattr_cache.c |  504 --
> .../lustre/lustre/llite/xattr_security.c      |   96 -
> drivers/staging/lustre/lustre/lmv/Makefile    |    5 -
> drivers/staging/lustre/lustre/lmv/lmv_fld.c   |   82 -
> .../staging/lustre/lustre/lmv/lmv_intent.c    |  521 --
> .../staging/lustre/lustre/lmv/lmv_internal.h  |  164 -
> drivers/staging/lustre/lustre/lmv/lmv_obd.c   | 3131 ------------
> drivers/staging/lustre/lustre/lmv/lproc_lmv.c |  173 -
> drivers/staging/lustre/lustre/lov/Makefile    |    9 -
> .../lustre/lustre/lov/lov_cl_internal.h       |  639 ---
> drivers/staging/lustre/lustre/lov/lov_dev.c   |  384 --
> drivers/staging/lustre/lustre/lov/lov_ea.c    |  331 --
> .../staging/lustre/lustre/lov/lov_internal.h  |  286 --
> drivers/staging/lustre/lustre/lov/lov_io.c    | 1023 ----
> drivers/staging/lustre/lustre/lov/lov_lock.c  |  348 --
> drivers/staging/lustre/lustre/lov/lov_merge.c |  105 -
> drivers/staging/lustre/lustre/lov/lov_obd.c   | 1444 ------
> .../staging/lustre/lustre/lov/lov_object.c    | 1625 -------
> .../staging/lustre/lustre/lov/lov_offset.c    |  269 --
> drivers/staging/lustre/lustre/lov/lov_pack.c  |  400 --
> drivers/staging/lustre/lustre/lov/lov_page.c  |  136 -
> drivers/staging/lustre/lustre/lov/lov_pool.c  |  546 ---
> .../staging/lustre/lustre/lov/lov_request.c   |  354 --
> .../staging/lustre/lustre/lov/lovsub_dev.c    |  147 -
> .../staging/lustre/lustre/lov/lovsub_lock.c   |   81 -
> .../staging/lustre/lustre/lov/lovsub_object.c |  180 -
> .../staging/lustre/lustre/lov/lovsub_page.c   |   68 -
> drivers/staging/lustre/lustre/lov/lproc_lov.c |  299 --
> drivers/staging/lustre/lustre/mdc/Makefile    |    5 -
> drivers/staging/lustre/lustre/mdc/lproc_mdc.c |  231 -
> .../staging/lustre/lustre/mdc/mdc_internal.h  |  144 -
> drivers/staging/lustre/lustre/mdc/mdc_lib.c   |  498 --
> drivers/staging/lustre/lustre/mdc/mdc_locks.c | 1239 -----
> drivers/staging/lustre/lustre/mdc/mdc_reint.c |  421 --
> .../staging/lustre/lustre/mdc/mdc_request.c   | 2770 -----------
> drivers/staging/lustre/lustre/mgc/Makefile    |    5 -
> drivers/staging/lustre/lustre/mgc/lproc_mgc.c |   69 -
> .../staging/lustre/lustre/mgc/mgc_internal.h  |   57 -
> .../staging/lustre/lustre/mgc/mgc_request.c   | 1851 --------
> .../staging/lustre/lustre/obdclass/Makefile   |   12 -
> .../lustre/lustre/obdclass/cl_internal.h      |   95 -
> .../staging/lustre/lustre/obdclass/cl_io.c    | 1151 -----
> .../staging/lustre/lustre/obdclass/cl_lock.c  |  275 --
> .../lustre/lustre/obdclass/cl_object.c        | 1059 -----
> .../staging/lustre/lustre/obdclass/cl_page.c  | 1045 ----
> .../lustre/lustre/obdclass/class_obd.c        |  544 ---
> .../staging/lustre/lustre/obdclass/debug.c    |   96 -
> .../staging/lustre/lustre/obdclass/genops.c   | 1480 ------
> .../lustre/lustre/obdclass/kernelcomm.c       |  240 -
> .../staging/lustre/lustre/obdclass/linkea.c   |  249 -
> .../lustre/obdclass/linux/linux-module.c      |  514 --
> .../lustre/obdclass/linux/linux-sysctl.c      |  162 -
> drivers/staging/lustre/lustre/obdclass/llog.c |  524 --
> .../staging/lustre/lustre/obdclass/llog_cat.c |  236 -
> .../lustre/lustre/obdclass/llog_internal.h    |   79 -
> .../staging/lustre/lustre/obdclass/llog_obd.c |  225 -
> .../lustre/lustre/obdclass/llog_swab.c        |  412 --
> .../lustre/lustre/obdclass/lprocfs_counters.c |  134 -
> .../lustre/lustre/obdclass/lprocfs_status.c   | 1698 -------
> .../lustre/lustre/obdclass/lu_object.c        | 2056 --------
> .../staging/lustre/lustre/obdclass/lu_ref.c   |   45 -
> .../lustre/lustre/obdclass/lustre_handles.c   |  241 -
> .../lustre/lustre/obdclass/lustre_peer.c      |  214 -
> .../lustre/lustre/obdclass/obd_config.c       | 1538 ------
> .../lustre/lustre/obdclass/obd_mount.c        | 1245 -----
> drivers/staging/lustre/lustre/obdclass/obdo.c |  181 -
> .../lustre/lustre/obdclass/statfs_pack.c      |   58 -
> drivers/staging/lustre/lustre/obdclass/uuid.c |   45 -
> .../staging/lustre/lustre/obdecho/Makefile    |    5 -
> .../lustre/lustre/obdecho/echo_client.c       | 1729 -------
> .../lustre/lustre/obdecho/echo_internal.h     |   42 -
> drivers/staging/lustre/lustre/osc/Makefile    |    6 -
> drivers/staging/lustre/lustre/osc/lproc_osc.c |  838 ----
> drivers/staging/lustre/lustre/osc/osc_cache.c | 3306 -------------
> .../lustre/lustre/osc/osc_cl_internal.h       |  681 ---
> drivers/staging/lustre/lustre/osc/osc_dev.c   |  246 -
> .../staging/lustre/lustre/osc/osc_internal.h  |  237 -
> drivers/staging/lustre/lustre/osc/osc_io.c    |  918 ----
> drivers/staging/lustre/lustre/osc/osc_lock.c  | 1230 -----
> .../staging/lustre/lustre/osc/osc_object.c    |  473 --
> drivers/staging/lustre/lustre/osc/osc_page.c  | 1094 -----
> drivers/staging/lustre/lustre/osc/osc_quota.c |  236 -
> .../staging/lustre/lustre/osc/osc_request.c   | 2907 ------------
> drivers/staging/lustre/lustre/ptlrpc/Makefile |   23 -
> drivers/staging/lustre/lustre/ptlrpc/client.c | 3271 -------------
> .../staging/lustre/lustre/ptlrpc/connection.c |  192 -
> drivers/staging/lustre/lustre/ptlrpc/errno.c  |  383 --
> drivers/staging/lustre/lustre/ptlrpc/events.c |  585 ---
> drivers/staging/lustre/lustre/ptlrpc/import.c | 1677 -------
> drivers/staging/lustre/lustre/ptlrpc/layout.c | 2232 ---------
> .../lustre/lustre/ptlrpc/llog_client.c        |  338 --
> .../staging/lustre/lustre/ptlrpc/llog_net.c   |   67 -
> .../lustre/lustre/ptlrpc/lproc_ptlrpc.c       | 1316 ------
> drivers/staging/lustre/lustre/ptlrpc/niobuf.c |  771 ---
> drivers/staging/lustre/lustre/ptlrpc/nrs.c    | 1613 -------
> .../staging/lustre/lustre/ptlrpc/nrs_fifo.c   |  270 --
> .../lustre/lustre/ptlrpc/pack_generic.c       | 2311 ---------
> drivers/staging/lustre/lustre/ptlrpc/pers.c   |   72 -
> drivers/staging/lustre/lustre/ptlrpc/pinger.c |  474 --
> .../lustre/lustre/ptlrpc/ptlrpc_internal.h    |  371 --
> .../lustre/lustre/ptlrpc/ptlrpc_module.c      |  186 -
> .../staging/lustre/lustre/ptlrpc/ptlrpcd.c    |  914 ----
> .../staging/lustre/lustre/ptlrpc/recover.c    |  374 --
> drivers/staging/lustre/lustre/ptlrpc/sec.c    | 2379 ----------
> .../staging/lustre/lustre/ptlrpc/sec_bulk.c   |  572 ---
> .../staging/lustre/lustre/ptlrpc/sec_config.c |  850 ----
> drivers/staging/lustre/lustre/ptlrpc/sec_gc.c |  190 -
> .../staging/lustre/lustre/ptlrpc/sec_lproc.c  |  170 -
> .../staging/lustre/lustre/ptlrpc/sec_null.c   |  459 --
> .../staging/lustre/lustre/ptlrpc/sec_plain.c  | 1023 ----
> .../staging/lustre/lustre/ptlrpc/service.c    | 2807 -----------
> .../staging/lustre/lustre/ptlrpc/wiretest.c   | 4210 -----------------
> drivers/staging/lustre/sysfs-fs-lustre        |  654 ---
> scripts/selinux/mdp/mdp.c                     |    1 -
> 308 files changed, 195272 deletions(-)
> delete mode 100644 drivers/staging/lustre/Kconfig
> delete mode 100644 drivers/staging/lustre/Makefile
> delete mode 100644 drivers/staging/lustre/README.txt
> delete mode 100644 drivers/staging/lustre/TODO
> delete mode 100644 drivers/staging/lustre/include/linux/libcfs/libcfs.h
> delete mode 100644 drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h
> delete mode 100644 drivers/staging/lustre/include/linux/libcfs/libcfs_crypto.h
> delete mode 100644 drivers/staging/lustre/include/linux/libcfs/libcfs_debug.h
> delete mode 100644 drivers/staging/lustre/include/linux/libcfs/libcfs_fail.h
> delete mode 100644 drivers/staging/lustre/include/linux/libcfs/libcfs_hash.h
> delete mode 100644 drivers/staging/lustre/include/linux/libcfs/libcfs_private.h
> delete mode 100644 drivers/staging/lustre/include/linux/libcfs/libcfs_string.h
> delete mode 100644 drivers/staging/lustre/include/linux/lnet/api.h
> delete mode 100644 drivers/staging/lustre/include/linux/lnet/lib-lnet.h
> delete mode 100644 drivers/staging/lustre/include/linux/lnet/lib-types.h
> delete mode 100644 drivers/staging/lustre/include/linux/lnet/socklnd.h
> delete mode 100644 drivers/staging/lustre/include/uapi/linux/lnet/libcfs_debug.h
> delete mode 100644 drivers/staging/lustre/include/uapi/linux/lnet/libcfs_ioctl.h
> delete mode 100644 drivers/staging/lustre/include/uapi/linux/lnet/lnet-dlc.h
> delete mode 100644 drivers/staging/lustre/include/uapi/linux/lnet/lnet-types.h
> delete mode 100644 drivers/staging/lustre/include/uapi/linux/lnet/lnetctl.h
> delete mode 100644 drivers/staging/lustre/include/uapi/linux/lnet/lnetst.h
> delete mode 100644 drivers/staging/lustre/include/uapi/linux/lnet/nidstr.h
> delete mode 100644 drivers/staging/lustre/include/uapi/linux/lnet/socklnd.h
> delete mode 100644 drivers/staging/lustre/include/uapi/linux/lustre/lustre_cfg.h
> delete mode 100644 drivers/staging/lustre/include/uapi/linux/lustre/lustre_fid.h
> delete mode 100644 drivers/staging/lustre/include/uapi/linux/lustre/lustre_fiemap.h
> delete mode 100644 drivers/staging/lustre/include/uapi/linux/lustre/lustre_idl.h
> delete mode 100644 drivers/staging/lustre/include/uapi/linux/lustre/lustre_ioctl.h
> delete mode 100644 drivers/staging/lustre/include/uapi/linux/lustre/lustre_kernelcomm.h
> delete mode 100644 drivers/staging/lustre/include/uapi/linux/lustre/lustre_ostid.h
> delete mode 100644 drivers/staging/lustre/include/uapi/linux/lustre/lustre_param.h
> delete mode 100644 drivers/staging/lustre/include/uapi/linux/lustre/lustre_user.h
> delete mode 100644 drivers/staging/lustre/include/uapi/linux/lustre/lustre_ver.h
> delete mode 100644 drivers/staging/lustre/lnet/Kconfig
> delete mode 100644 drivers/staging/lustre/lnet/Makefile
> delete mode 100644 drivers/staging/lustre/lnet/klnds/Makefile
> delete mode 100644 drivers/staging/lustre/lnet/klnds/o2iblnd/Makefile
> delete mode 100644 drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c
> delete mode 100644 drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.h
> delete mode 100644 drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c
> delete mode 100644 drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd_modparams.c
> delete mode 100644 drivers/staging/lustre/lnet/klnds/socklnd/Makefile
> delete mode 100644 drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c
> delete mode 100644 drivers/staging/lustre/lnet/klnds/socklnd/socklnd.h
> delete mode 100644 drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c
> delete mode 100644 drivers/staging/lustre/lnet/klnds/socklnd/socklnd_lib.c
> delete mode 100644 drivers/staging/lustre/lnet/klnds/socklnd/socklnd_modparams.c
> delete mode 100644 drivers/staging/lustre/lnet/klnds/socklnd/socklnd_proto.c
> delete mode 100644 drivers/staging/lustre/lnet/libcfs/Makefile
> delete mode 100644 drivers/staging/lustre/lnet/libcfs/debug.c
> delete mode 100644 drivers/staging/lustre/lnet/libcfs/fail.c
> delete mode 100644 drivers/staging/lustre/lnet/libcfs/hash.c
> delete mode 100644 drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
> delete mode 100644 drivers/staging/lustre/lnet/libcfs/libcfs_lock.c
> delete mode 100644 drivers/staging/lustre/lnet/libcfs/libcfs_mem.c
> delete mode 100644 drivers/staging/lustre/lnet/libcfs/libcfs_string.c
> delete mode 100644 drivers/staging/lustre/lnet/libcfs/linux-crypto-adler.c
> delete mode 100644 drivers/staging/lustre/lnet/libcfs/linux-crypto.c
> delete mode 100644 drivers/staging/lustre/lnet/libcfs/linux-crypto.h
> delete mode 100644 drivers/staging/lustre/lnet/libcfs/linux-debug.c
> delete mode 100644 drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
> delete mode 100644 drivers/staging/lustre/lnet/libcfs/module.c
> delete mode 100644 drivers/staging/lustre/lnet/libcfs/tracefile.c
> delete mode 100644 drivers/staging/lustre/lnet/libcfs/tracefile.h
> delete mode 100644 drivers/staging/lustre/lnet/lnet/Makefile
> delete mode 100644 drivers/staging/lustre/lnet/lnet/acceptor.c
> delete mode 100644 drivers/staging/lustre/lnet/lnet/api-ni.c
> delete mode 100644 drivers/staging/lustre/lnet/lnet/config.c
> delete mode 100644 drivers/staging/lustre/lnet/lnet/lib-eq.c
> delete mode 100644 drivers/staging/lustre/lnet/lnet/lib-md.c
> delete mode 100644 drivers/staging/lustre/lnet/lnet/lib-me.c
> delete mode 100644 drivers/staging/lustre/lnet/lnet/lib-move.c
> delete mode 100644 drivers/staging/lustre/lnet/lnet/lib-msg.c
> delete mode 100644 drivers/staging/lustre/lnet/lnet/lib-ptl.c
> delete mode 100644 drivers/staging/lustre/lnet/lnet/lib-socket.c
> delete mode 100644 drivers/staging/lustre/lnet/lnet/lo.c
> delete mode 100644 drivers/staging/lustre/lnet/lnet/module.c
> delete mode 100644 drivers/staging/lustre/lnet/lnet/net_fault.c
> delete mode 100644 drivers/staging/lustre/lnet/lnet/nidstrings.c
> delete mode 100644 drivers/staging/lustre/lnet/lnet/peer.c
> delete mode 100644 drivers/staging/lustre/lnet/lnet/router.c
> delete mode 100644 drivers/staging/lustre/lnet/lnet/router_proc.c
> delete mode 100644 drivers/staging/lustre/lnet/selftest/Makefile
> delete mode 100644 drivers/staging/lustre/lnet/selftest/brw_test.c
> delete mode 100644 drivers/staging/lustre/lnet/selftest/conctl.c
> delete mode 100644 drivers/staging/lustre/lnet/selftest/conrpc.c
> delete mode 100644 drivers/staging/lustre/lnet/selftest/conrpc.h
> delete mode 100644 drivers/staging/lustre/lnet/selftest/console.c
> delete mode 100644 drivers/staging/lustre/lnet/selftest/console.h
> delete mode 100644 drivers/staging/lustre/lnet/selftest/framework.c
> delete mode 100644 drivers/staging/lustre/lnet/selftest/module.c
> delete mode 100644 drivers/staging/lustre/lnet/selftest/ping_test.c
> delete mode 100644 drivers/staging/lustre/lnet/selftest/rpc.c
> delete mode 100644 drivers/staging/lustre/lnet/selftest/rpc.h
> delete mode 100644 drivers/staging/lustre/lnet/selftest/selftest.h
> delete mode 100644 drivers/staging/lustre/lnet/selftest/timer.c
> delete mode 100644 drivers/staging/lustre/lnet/selftest/timer.h
> delete mode 100644 drivers/staging/lustre/lustre/Kconfig
> delete mode 100644 drivers/staging/lustre/lustre/Makefile
> delete mode 100644 drivers/staging/lustre/lustre/fid/Makefile
> delete mode 100644 drivers/staging/lustre/lustre/fid/fid_internal.h
> delete mode 100644 drivers/staging/lustre/lustre/fid/fid_lib.c
> delete mode 100644 drivers/staging/lustre/lustre/fid/fid_request.c
> delete mode 100644 drivers/staging/lustre/lustre/fid/lproc_fid.c
> delete mode 100644 drivers/staging/lustre/lustre/fld/Makefile
> delete mode 100644 drivers/staging/lustre/lustre/fld/fld_cache.c
> delete mode 100644 drivers/staging/lustre/lustre/fld/fld_internal.h
> delete mode 100644 drivers/staging/lustre/lustre/fld/fld_request.c
> delete mode 100644 drivers/staging/lustre/lustre/fld/lproc_fld.c
> delete mode 100644 drivers/staging/lustre/lustre/include/cl_object.h
> delete mode 100644 drivers/staging/lustre/lustre/include/interval_tree.h
> delete mode 100644 drivers/staging/lustre/lustre/include/llog_swab.h
> delete mode 100644 drivers/staging/lustre/lustre/include/lprocfs_status.h
> delete mode 100644 drivers/staging/lustre/lustre/include/lu_object.h
> delete mode 100644 drivers/staging/lustre/lustre/include/lu_ref.h
> delete mode 100644 drivers/staging/lustre/lustre/include/lustre_acl.h
> delete mode 100644 drivers/staging/lustre/lustre/include/lustre_compat.h
> delete mode 100644 drivers/staging/lustre/lustre/include/lustre_debug.h
> delete mode 100644 drivers/staging/lustre/lustre/include/lustre_disk.h
> delete mode 100644 drivers/staging/lustre/lustre/include/lustre_dlm.h
> delete mode 100644 drivers/staging/lustre/lustre/include/lustre_dlm_flags.h
> delete mode 100644 drivers/staging/lustre/lustre/include/lustre_errno.h
> delete mode 100644 drivers/staging/lustre/lustre/include/lustre_export.h
> delete mode 100644 drivers/staging/lustre/lustre/include/lustre_fid.h
> delete mode 100644 drivers/staging/lustre/lustre/include/lustre_fld.h
> delete mode 100644 drivers/staging/lustre/lustre/include/lustre_ha.h
> delete mode 100644 drivers/staging/lustre/lustre/include/lustre_handles.h
> delete mode 100644 drivers/staging/lustre/lustre/include/lustre_import.h
> delete mode 100644 drivers/staging/lustre/lustre/include/lustre_intent.h
> delete mode 100644 drivers/staging/lustre/lustre/include/lustre_kernelcomm.h
> delete mode 100644 drivers/staging/lustre/lustre/include/lustre_lib.h
> delete mode 100644 drivers/staging/lustre/lustre/include/lustre_linkea.h
> delete mode 100644 drivers/staging/lustre/lustre/include/lustre_lmv.h
> delete mode 100644 drivers/staging/lustre/lustre/include/lustre_log.h
> delete mode 100644 drivers/staging/lustre/lustre/include/lustre_mdc.h
> delete mode 100644 drivers/staging/lustre/lustre/include/lustre_mds.h
> delete mode 100644 drivers/staging/lustre/lustre/include/lustre_net.h
> delete mode 100644 drivers/staging/lustre/lustre/include/lustre_nrs.h
> delete mode 100644 drivers/staging/lustre/lustre/include/lustre_nrs_fifo.h
> delete mode 100644 drivers/staging/lustre/lustre/include/lustre_obdo.h
> delete mode 100644 drivers/staging/lustre/lustre/include/lustre_patchless_compat.h
> delete mode 100644 drivers/staging/lustre/lustre/include/lustre_req_layout.h
> delete mode 100644 drivers/staging/lustre/lustre/include/lustre_sec.h
> delete mode 100644 drivers/staging/lustre/lustre/include/lustre_swab.h
> delete mode 100644 drivers/staging/lustre/lustre/include/obd.h
> delete mode 100644 drivers/staging/lustre/lustre/include/obd_cksum.h
> delete mode 100644 drivers/staging/lustre/lustre/include/obd_class.h
> delete mode 100644 drivers/staging/lustre/lustre/include/obd_support.h
> delete mode 100644 drivers/staging/lustre/lustre/include/seq_range.h
> delete mode 100644 drivers/staging/lustre/lustre/ldlm/interval_tree.c
> delete mode 100644 drivers/staging/lustre/lustre/ldlm/l_lock.c
> delete mode 100644 drivers/staging/lustre/lustre/ldlm/ldlm_extent.c
> delete mode 100644 drivers/staging/lustre/lustre/ldlm/ldlm_flock.c
> delete mode 100644 drivers/staging/lustre/lustre/ldlm/ldlm_inodebits.c
> delete mode 100644 drivers/staging/lustre/lustre/ldlm/ldlm_internal.h
> delete mode 100644 drivers/staging/lustre/lustre/ldlm/ldlm_lib.c
> delete mode 100644 drivers/staging/lustre/lustre/ldlm/ldlm_lock.c
> delete mode 100644 drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c
> delete mode 100644 drivers/staging/lustre/lustre/ldlm/ldlm_plain.c
> delete mode 100644 drivers/staging/lustre/lustre/ldlm/ldlm_pool.c
> delete mode 100644 drivers/staging/lustre/lustre/ldlm/ldlm_request.c
> delete mode 100644 drivers/staging/lustre/lustre/ldlm/ldlm_resource.c
> delete mode 100644 drivers/staging/lustre/lustre/llite/Makefile
> delete mode 100644 drivers/staging/lustre/lustre/llite/acl.c
> delete mode 100644 drivers/staging/lustre/lustre/llite/dcache.c
> delete mode 100644 drivers/staging/lustre/lustre/llite/dir.c
> delete mode 100644 drivers/staging/lustre/lustre/llite/file.c
> delete mode 100644 drivers/staging/lustre/lustre/llite/glimpse.c
> delete mode 100644 drivers/staging/lustre/lustre/llite/lcommon_cl.c
> delete mode 100644 drivers/staging/lustre/lustre/llite/lcommon_misc.c
> delete mode 100644 drivers/staging/lustre/lustre/llite/llite_internal.h
> delete mode 100644 drivers/staging/lustre/lustre/llite/llite_lib.c
> delete mode 100644 drivers/staging/lustre/lustre/llite/llite_mmap.c
> delete mode 100644 drivers/staging/lustre/lustre/llite/llite_nfs.c
> delete mode 100644 drivers/staging/lustre/lustre/llite/lproc_llite.c
> delete mode 100644 drivers/staging/lustre/lustre/llite/namei.c
> delete mode 100644 drivers/staging/lustre/lustre/llite/range_lock.c
> delete mode 100644 drivers/staging/lustre/lustre/llite/range_lock.h
> delete mode 100644 drivers/staging/lustre/lustre/llite/rw.c
> delete mode 100644 drivers/staging/lustre/lustre/llite/rw26.c
> delete mode 100644 drivers/staging/lustre/lustre/llite/statahead.c
> delete mode 100644 drivers/staging/lustre/lustre/llite/super25.c
> delete mode 100644 drivers/staging/lustre/lustre/llite/symlink.c
> delete mode 100644 drivers/staging/lustre/lustre/llite/vvp_dev.c
> delete mode 100644 drivers/staging/lustre/lustre/llite/vvp_internal.h
> delete mode 100644 drivers/staging/lustre/lustre/llite/vvp_io.c
> delete mode 100644 drivers/staging/lustre/lustre/llite/vvp_lock.c
> delete mode 100644 drivers/staging/lustre/lustre/llite/vvp_object.c
> delete mode 100644 drivers/staging/lustre/lustre/llite/vvp_page.c
> delete mode 100644 drivers/staging/lustre/lustre/llite/xattr.c
> delete mode 100644 drivers/staging/lustre/lustre/llite/xattr_cache.c
> delete mode 100644 drivers/staging/lustre/lustre/llite/xattr_security.c
> delete mode 100644 drivers/staging/lustre/lustre/lmv/Makefile
> delete mode 100644 drivers/staging/lustre/lustre/lmv/lmv_fld.c
> delete mode 100644 drivers/staging/lustre/lustre/lmv/lmv_intent.c
> delete mode 100644 drivers/staging/lustre/lustre/lmv/lmv_internal.h
> delete mode 100644 drivers/staging/lustre/lustre/lmv/lmv_obd.c
> delete mode 100644 drivers/staging/lustre/lustre/lmv/lproc_lmv.c
> delete mode 100644 drivers/staging/lustre/lustre/lov/Makefile
> delete mode 100644 drivers/staging/lustre/lustre/lov/lov_cl_internal.h
> delete mode 100644 drivers/staging/lustre/lustre/lov/lov_dev.c
> delete mode 100644 drivers/staging/lustre/lustre/lov/lov_ea.c
> delete mode 100644 drivers/staging/lustre/lustre/lov/lov_internal.h
> delete mode 100644 drivers/staging/lustre/lustre/lov/lov_io.c
> delete mode 100644 drivers/staging/lustre/lustre/lov/lov_lock.c
> delete mode 100644 drivers/staging/lustre/lustre/lov/lov_merge.c
> delete mode 100644 drivers/staging/lustre/lustre/lov/lov_obd.c
> delete mode 100644 drivers/staging/lustre/lustre/lov/lov_object.c
> delete mode 100644 drivers/staging/lustre/lustre/lov/lov_offset.c
> delete mode 100644 drivers/staging/lustre/lustre/lov/lov_pack.c
> delete mode 100644 drivers/staging/lustre/lustre/lov/lov_page.c
> delete mode 100644 drivers/staging/lustre/lustre/lov/lov_pool.c
> delete mode 100644 drivers/staging/lustre/lustre/lov/lov_request.c
> delete mode 100644 drivers/staging/lustre/lustre/lov/lovsub_dev.c
> delete mode 100644 drivers/staging/lustre/lustre/lov/lovsub_lock.c
> delete mode 100644 drivers/staging/lustre/lustre/lov/lovsub_object.c
> delete mode 100644 drivers/staging/lustre/lustre/lov/lovsub_page.c
> delete mode 100644 drivers/staging/lustre/lustre/lov/lproc_lov.c
> delete mode 100644 drivers/staging/lustre/lustre/mdc/Makefile
> delete mode 100644 drivers/staging/lustre/lustre/mdc/lproc_mdc.c
> delete mode 100644 drivers/staging/lustre/lustre/mdc/mdc_internal.h
> delete mode 100644 drivers/staging/lustre/lustre/mdc/mdc_lib.c
> delete mode 100644 drivers/staging/lustre/lustre/mdc/mdc_locks.c
> delete mode 100644 drivers/staging/lustre/lustre/mdc/mdc_reint.c
> delete mode 100644 drivers/staging/lustre/lustre/mdc/mdc_request.c
> delete mode 100644 drivers/staging/lustre/lustre/mgc/Makefile
> delete mode 100644 drivers/staging/lustre/lustre/mgc/lproc_mgc.c
> delete mode 100644 drivers/staging/lustre/lustre/mgc/mgc_internal.h
> delete mode 100644 drivers/staging/lustre/lustre/mgc/mgc_request.c
> delete mode 100644 drivers/staging/lustre/lustre/obdclass/Makefile
> delete mode 100644 drivers/staging/lustre/lustre/obdclass/cl_internal.h
> delete mode 100644 drivers/staging/lustre/lustre/obdclass/cl_io.c
> delete mode 100644 drivers/staging/lustre/lustre/obdclass/cl_lock.c
> delete mode 100644 drivers/staging/lustre/lustre/obdclass/cl_object.c
> delete mode 100644 drivers/staging/lustre/lustre/obdclass/cl_page.c
> delete mode 100644 drivers/staging/lustre/lustre/obdclass/class_obd.c
> delete mode 100644 drivers/staging/lustre/lustre/obdclass/debug.c
> delete mode 100644 drivers/staging/lustre/lustre/obdclass/genops.c
> delete mode 100644 drivers/staging/lustre/lustre/obdclass/kernelcomm.c
> delete mode 100644 drivers/staging/lustre/lustre/obdclass/linkea.c
> delete mode 100644 drivers/staging/lustre/lustre/obdclass/linux/linux-module.c
> delete mode 100644 drivers/staging/lustre/lustre/obdclass/linux/linux-sysctl.c
> delete mode 100644 drivers/staging/lustre/lustre/obdclass/llog.c
> delete mode 100644 drivers/staging/lustre/lustre/obdclass/llog_cat.c
> delete mode 100644 drivers/staging/lustre/lustre/obdclass/llog_internal.h
> delete mode 100644 drivers/staging/lustre/lustre/obdclass/llog_obd.c
> delete mode 100644 drivers/staging/lustre/lustre/obdclass/llog_swab.c
> delete mode 100644 drivers/staging/lustre/lustre/obdclass/lprocfs_counters.c
> delete mode 100644 drivers/staging/lustre/lustre/obdclass/lprocfs_status.c
> delete mode 100644 drivers/staging/lustre/lustre/obdclass/lu_object.c
> delete mode 100644 drivers/staging/lustre/lustre/obdclass/lu_ref.c
> delete mode 100644 drivers/staging/lustre/lustre/obdclass/lustre_handles.c
> delete mode 100644 drivers/staging/lustre/lustre/obdclass/lustre_peer.c
> delete mode 100644 drivers/staging/lustre/lustre/obdclass/obd_config.c
> delete mode 100644 drivers/staging/lustre/lustre/obdclass/obd_mount.c
> delete mode 100644 drivers/staging/lustre/lustre/obdclass/obdo.c
> delete mode 100644 drivers/staging/lustre/lustre/obdclass/statfs_pack.c
> delete mode 100644 drivers/staging/lustre/lustre/obdclass/uuid.c
> delete mode 100644 drivers/staging/lustre/lustre/obdecho/Makefile
> delete mode 100644 drivers/staging/lustre/lustre/obdecho/echo_client.c
> delete mode 100644 drivers/staging/lustre/lustre/obdecho/echo_internal.h
> delete mode 100644 drivers/staging/lustre/lustre/osc/Makefile
> delete mode 100644 drivers/staging/lustre/lustre/osc/lproc_osc.c
> delete mode 100644 drivers/staging/lustre/lustre/osc/osc_cache.c
> delete mode 100644 drivers/staging/lustre/lustre/osc/osc_cl_internal.h
> delete mode 100644 drivers/staging/lustre/lustre/osc/osc_dev.c
> delete mode 100644 drivers/staging/lustre/lustre/osc/osc_internal.h
> delete mode 100644 drivers/staging/lustre/lustre/osc/osc_io.c
> delete mode 100644 drivers/staging/lustre/lustre/osc/osc_lock.c
> delete mode 100644 drivers/staging/lustre/lustre/osc/osc_object.c
> delete mode 100644 drivers/staging/lustre/lustre/osc/osc_page.c
> delete mode 100644 drivers/staging/lustre/lustre/osc/osc_quota.c
> delete mode 100644 drivers/staging/lustre/lustre/osc/osc_request.c
> delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/Makefile
> delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/client.c
> delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/connection.c
> delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/errno.c
> delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/events.c
> delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/import.c
> delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/layout.c
> delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/llog_client.c
> delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/llog_net.c
> delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/lproc_ptlrpc.c
> delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/niobuf.c
> delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/nrs.c
> delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/nrs_fifo.c
> delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/pack_generic.c
> delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/pers.c
> delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/pinger.c
> delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/ptlrpc_internal.h
> delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/ptlrpc_module.c
> delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c
> delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/recover.c
> delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/sec.c
> delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/sec_bulk.c
> delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/sec_config.c
> delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/sec_gc.c
> delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/sec_lproc.c
> delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/sec_null.c
> delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/sec_plain.c
> delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/service.c
> delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/wiretest.c
> delete mode 100644 drivers/staging/lustre/sysfs-fs-lustre
> 
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 4b65225d443a..db158767de20 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -13329,15 +13329,6 @@ S:	Odd Fixes
> F:	Documentation/devicetree/bindings/staging/iio/
> F:	drivers/staging/iio/
> 
> -STAGING - LUSTRE PARALLEL FILESYSTEM
> -M:	Oleg Drokin <oleg.drokin at intel.com>
> -M:	Andreas Dilger <andreas.dilger at intel.com>
> -M:	James Simmons <jsimmons at infradead.org>
> -L:	lustre-devel at lists.lustre.org (moderated for non-subscribers)
> -W:	http://wiki.lustre.org/
> -S:	Maintained
> -F:	drivers/staging/lustre
> -
> STAGING - NVIDIA COMPLIANT EMBEDDED CONTROLLER INTERFACE (nvec)
> M:	Marc Dietrich <marvin24 at gmx.de>
> L:	ac100 at lists.launchpad.net (moderated for non-subscribers)
> diff --git a/drivers/staging/Kconfig b/drivers/staging/Kconfig
> index d5926f0d3f6c..1c357ef669ae 100644
> --- a/drivers/staging/Kconfig
> +++ b/drivers/staging/Kconfig
> @@ -84,8 +84,6 @@ source "drivers/staging/netlogic/Kconfig"
> 
> source "drivers/staging/mt29f_spinand/Kconfig"
> 
> -source "drivers/staging/lustre/Kconfig"
> -
> source "drivers/staging/dgnc/Kconfig"
> 
> source "drivers/staging/gs_fpgaboot/Kconfig"
> diff --git a/drivers/staging/Makefile b/drivers/staging/Makefile
> index 919753c3d3f6..2edb9860931e 100644
> --- a/drivers/staging/Makefile
> +++ b/drivers/staging/Makefile
> @@ -32,7 +32,6 @@ obj-$(CONFIG_STAGING_BOARD)	+= board/
> obj-$(CONFIG_LTE_GDM724X)	+= gdm724x/
> obj-$(CONFIG_FIREWIRE_SERIAL)	+= fwserial/
> obj-$(CONFIG_GOLDFISH)		+= goldfish/
> -obj-$(CONFIG_LNET)		+= lustre/
> obj-$(CONFIG_DGNC)			+= dgnc/
> obj-$(CONFIG_MTD_SPINAND_MT29F)	+= mt29f_spinand/
> obj-$(CONFIG_GS_FPGABOOT)	+= gs_fpgaboot/
> diff --git a/drivers/staging/lustre/Kconfig b/drivers/staging/lustre/Kconfig
> deleted file mode 100644
> index b7d81096eee9..000000000000
> diff --git a/drivers/staging/lustre/Makefile b/drivers/staging/lustre/Makefile
> deleted file mode 100644
> index 95ffe337a80a..000000000000
> diff --git a/drivers/staging/lustre/README.txt b/drivers/staging/lustre/README.txt
> deleted file mode 100644
> index 783959240490..000000000000
> diff --git a/drivers/staging/lustre/TODO b/drivers/staging/lustre/TODO
> deleted file mode 100644
> index 5332cdb19bfa..000000000000
> diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs.h b/drivers/staging/lustre/include/linux/libcfs/libcfs.h
> deleted file mode 100644
> index edc7ed0dcb94..000000000000
> diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h
> deleted file mode 100644
> index 61641c41c492..000000000000
> diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_crypto.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_crypto.h
> deleted file mode 100644
> index 176fae7319e3..000000000000
> diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_debug.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_debug.h
> deleted file mode 100644
> index 17534a76362a..000000000000
> diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_fail.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_fail.h
> deleted file mode 100644
> index 8074e390b4d1..000000000000
> diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_hash.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_hash.h
> deleted file mode 100644
> index be315958a4b3..000000000000
> diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_private.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_private.h
> deleted file mode 100644
> index 491d5971d199..000000000000
> diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_string.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_string.h
> deleted file mode 100644
> index cd7c3ccb2dc0..000000000000
> diff --git a/drivers/staging/lustre/include/linux/lnet/api.h b/drivers/staging/lustre/include/linux/lnet/api.h
> deleted file mode 100644
> index dae2e4f0056c..000000000000
> diff --git a/drivers/staging/lustre/include/linux/lnet/lib-lnet.h b/drivers/staging/lustre/include/linux/lnet/lib-lnet.h
> deleted file mode 100644
> index 973c17a1c4a1..000000000000
> diff --git a/drivers/staging/lustre/include/linux/lnet/lib-types.h b/drivers/staging/lustre/include/linux/lnet/lib-types.h
> deleted file mode 100644
> index cfe8ee424e94..000000000000
> diff --git a/drivers/staging/lustre/include/linux/lnet/socklnd.h b/drivers/staging/lustre/include/linux/lnet/socklnd.h
> deleted file mode 100644
> index 6bd1bca190a3..000000000000
> diff --git a/drivers/staging/lustre/include/uapi/linux/lnet/libcfs_debug.h b/drivers/staging/lustre/include/uapi/linux/lnet/libcfs_debug.h
> deleted file mode 100644
> index c4d9472b374f..000000000000
> diff --git a/drivers/staging/lustre/include/uapi/linux/lnet/libcfs_ioctl.h b/drivers/staging/lustre/include/uapi/linux/lnet/libcfs_ioctl.h
> deleted file mode 100644
> index cce6b58e3682..000000000000
> diff --git a/drivers/staging/lustre/include/uapi/linux/lnet/lnet-dlc.h b/drivers/staging/lustre/include/uapi/linux/lnet/lnet-dlc.h
> deleted file mode 100644
> index c1619f411d81..000000000000
> diff --git a/drivers/staging/lustre/include/uapi/linux/lnet/lnet-types.h b/drivers/staging/lustre/include/uapi/linux/lnet/lnet-types.h
> deleted file mode 100644
> index 1be9b7aa7326..000000000000
> diff --git a/drivers/staging/lustre/include/uapi/linux/lnet/lnetctl.h b/drivers/staging/lustre/include/uapi/linux/lnet/lnetctl.h
> deleted file mode 100644
> index cccb32dd28f2..000000000000
> diff --git a/drivers/staging/lustre/include/uapi/linux/lnet/lnetst.h b/drivers/staging/lustre/include/uapi/linux/lnet/lnetst.h
> deleted file mode 100644
> index a4f9ff01d458..000000000000
> diff --git a/drivers/staging/lustre/include/uapi/linux/lnet/nidstr.h b/drivers/staging/lustre/include/uapi/linux/lnet/nidstr.h
> deleted file mode 100644
> index 882074ed6021..000000000000
> diff --git a/drivers/staging/lustre/include/uapi/linux/lnet/socklnd.h b/drivers/staging/lustre/include/uapi/linux/lnet/socklnd.h
> deleted file mode 100644
> index 6453e053fa99..000000000000
> diff --git a/drivers/staging/lustre/include/uapi/linux/lustre/lustre_cfg.h b/drivers/staging/lustre/include/uapi/linux/lustre/lustre_cfg.h
> deleted file mode 100644
> index 11b51d93f64c..000000000000
> diff --git a/drivers/staging/lustre/include/uapi/linux/lustre/lustre_fid.h b/drivers/staging/lustre/include/uapi/linux/lustre/lustre_fid.h
> deleted file mode 100644
> index 2e7a8d103777..000000000000
> diff --git a/drivers/staging/lustre/include/uapi/linux/lustre/lustre_fiemap.h b/drivers/staging/lustre/include/uapi/linux/lustre/lustre_fiemap.h
> deleted file mode 100644
> index d375a476f5ea..000000000000
> diff --git a/drivers/staging/lustre/include/uapi/linux/lustre/lustre_idl.h b/drivers/staging/lustre/include/uapi/linux/lustre/lustre_idl.h
> deleted file mode 100644
> index 6c7e3992d646..000000000000
> diff --git a/drivers/staging/lustre/include/uapi/linux/lustre/lustre_ioctl.h b/drivers/staging/lustre/include/uapi/linux/lustre/lustre_ioctl.h
> deleted file mode 100644
> index 6e4e109fb874..000000000000
> diff --git a/drivers/staging/lustre/include/uapi/linux/lustre/lustre_kernelcomm.h b/drivers/staging/lustre/include/uapi/linux/lustre/lustre_kernelcomm.h
> deleted file mode 100644
> index 94dadbe8e069..000000000000
> diff --git a/drivers/staging/lustre/include/uapi/linux/lustre/lustre_ostid.h b/drivers/staging/lustre/include/uapi/linux/lustre/lustre_ostid.h
> deleted file mode 100644
> index 3343b602219b..000000000000
> diff --git a/drivers/staging/lustre/include/uapi/linux/lustre/lustre_param.h b/drivers/staging/lustre/include/uapi/linux/lustre/lustre_param.h
> deleted file mode 100644
> index 1eab2ceca338..000000000000
> diff --git a/drivers/staging/lustre/include/uapi/linux/lustre/lustre_user.h b/drivers/staging/lustre/include/uapi/linux/lustre/lustre_user.h
> deleted file mode 100644
> index 69387f36d1f1..000000000000
> diff --git a/drivers/staging/lustre/include/uapi/linux/lustre/lustre_ver.h b/drivers/staging/lustre/include/uapi/linux/lustre/lustre_ver.h
> deleted file mode 100644
> index 19c9135e2273..000000000000
> diff --git a/drivers/staging/lustre/lnet/Kconfig b/drivers/staging/lustre/lnet/Kconfig
> deleted file mode 100644
> index ad049e6f24e4..000000000000
> diff --git a/drivers/staging/lustre/lnet/Makefile b/drivers/staging/lustre/lnet/Makefile
> deleted file mode 100644
> index 0a380fe88ce8..000000000000
> diff --git a/drivers/staging/lustre/lnet/klnds/Makefile b/drivers/staging/lustre/lnet/klnds/Makefile
> deleted file mode 100644
> index c23e4f67f837..000000000000
> diff --git a/drivers/staging/lustre/lnet/klnds/o2iblnd/Makefile b/drivers/staging/lustre/lnet/klnds/o2iblnd/Makefile
> deleted file mode 100644
> index 4affe1d79948..000000000000
> diff --git a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c
> deleted file mode 100644
> index f0b4eb42bc1d..000000000000
> diff --git a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.h b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.h
> deleted file mode 100644
> index 217503f125bc..000000000000
> diff --git a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c
> deleted file mode 100644
> index 65b7a62943ad..000000000000
> diff --git a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd_modparams.c b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd_modparams.c
> deleted file mode 100644
> index 39d07926d603..000000000000
> diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/Makefile b/drivers/staging/lustre/lnet/klnds/socklnd/Makefile
> deleted file mode 100644
> index a7da1abfc804..000000000000
> diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c
> deleted file mode 100644
> index f01b34ac1a53..000000000000
> diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.h b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.h
> deleted file mode 100644
> index 4e5c89a692a3..000000000000
> diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c
> deleted file mode 100644
> index 01b31a6bb588..000000000000
> diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_lib.c b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_lib.c
> deleted file mode 100644
> index 93a02cd6b6b5..000000000000
> diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_modparams.c b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_modparams.c
> deleted file mode 100644
> index 5663a4ca94d4..000000000000
> diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_proto.c b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_proto.c
> deleted file mode 100644
> index 05982dac781c..000000000000
> diff --git a/drivers/staging/lustre/lnet/libcfs/Makefile b/drivers/staging/lustre/lnet/libcfs/Makefile
> deleted file mode 100644
> index 6a1b232da495..000000000000
> diff --git a/drivers/staging/lustre/lnet/libcfs/debug.c b/drivers/staging/lustre/lnet/libcfs/debug.c
> deleted file mode 100644
> index 06f694f6a28f..000000000000
> diff --git a/drivers/staging/lustre/lnet/libcfs/fail.c b/drivers/staging/lustre/lnet/libcfs/fail.c
> deleted file mode 100644
> index bd86b3b5bc34..000000000000
> diff --git a/drivers/staging/lustre/lnet/libcfs/hash.c b/drivers/staging/lustre/lnet/libcfs/hash.c
> deleted file mode 100644
> index 48be66f0d654..000000000000
> diff --git a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
> deleted file mode 100644
> index 3d1cf457b286..000000000000
> diff --git a/drivers/staging/lustre/lnet/libcfs/libcfs_lock.c b/drivers/staging/lustre/lnet/libcfs/libcfs_lock.c
> deleted file mode 100644
> index 223505c37545..000000000000
> diff --git a/drivers/staging/lustre/lnet/libcfs/libcfs_mem.c b/drivers/staging/lustre/lnet/libcfs/libcfs_mem.c
> deleted file mode 100644
> index 2d533be9bb30..000000000000
> diff --git a/drivers/staging/lustre/lnet/libcfs/libcfs_string.c b/drivers/staging/lustre/lnet/libcfs/libcfs_string.c
> deleted file mode 100644
> index e1fb1263e3ae..000000000000
> diff --git a/drivers/staging/lustre/lnet/libcfs/linux-crypto-adler.c b/drivers/staging/lustre/lnet/libcfs/linux-crypto-adler.c
> deleted file mode 100644
> index db81ed527452..000000000000
> diff --git a/drivers/staging/lustre/lnet/libcfs/linux-crypto.c b/drivers/staging/lustre/lnet/libcfs/linux-crypto.c
> deleted file mode 100644
> index 21ff9bf6da47..000000000000
> diff --git a/drivers/staging/lustre/lnet/libcfs/linux-crypto.h b/drivers/staging/lustre/lnet/libcfs/linux-crypto.h
> deleted file mode 100644
> index 5616e9ea1450..000000000000
> diff --git a/drivers/staging/lustre/lnet/libcfs/linux-debug.c b/drivers/staging/lustre/lnet/libcfs/linux-debug.c
> deleted file mode 100644
> index 15ab849374c2..000000000000
> diff --git a/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c b/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
> deleted file mode 100644
> index 347138409eba..000000000000
> diff --git a/drivers/staging/lustre/lnet/libcfs/module.c b/drivers/staging/lustre/lnet/libcfs/module.c
> deleted file mode 100644
> index 5dc7de9e6478..000000000000
> diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.c b/drivers/staging/lustre/lnet/libcfs/tracefile.c
> deleted file mode 100644
> index 7ca562e156f0..000000000000
> diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.h b/drivers/staging/lustre/lnet/libcfs/tracefile.h
> deleted file mode 100644
> index 0608240d897f..000000000000
> diff --git a/drivers/staging/lustre/lnet/lnet/Makefile b/drivers/staging/lustre/lnet/lnet/Makefile
> deleted file mode 100644
> index 0a9d70924fe0..000000000000
> diff --git a/drivers/staging/lustre/lnet/lnet/acceptor.c b/drivers/staging/lustre/lnet/lnet/acceptor.c
> deleted file mode 100644
> index 5648f17eddc0..000000000000
> diff --git a/drivers/staging/lustre/lnet/lnet/api-ni.c b/drivers/staging/lustre/lnet/lnet/api-ni.c
> deleted file mode 100644
> index f9ed6977056c..000000000000
> diff --git a/drivers/staging/lustre/lnet/lnet/config.c b/drivers/staging/lustre/lnet/lnet/config.c
> deleted file mode 100644
> index 55ecc1998b7e..000000000000
> diff --git a/drivers/staging/lustre/lnet/lnet/lib-eq.c b/drivers/staging/lustre/lnet/lnet/lib-eq.c
> deleted file mode 100644
> index c78e70373ab4..000000000000
> diff --git a/drivers/staging/lustre/lnet/lnet/lib-md.c b/drivers/staging/lustre/lnet/lnet/lib-md.c
> deleted file mode 100644
> index 8a22514aaf71..000000000000
> diff --git a/drivers/staging/lustre/lnet/lnet/lib-me.c b/drivers/staging/lustre/lnet/lnet/lib-me.c
> deleted file mode 100644
> index 672e37bdd045..000000000000
> diff --git a/drivers/staging/lustre/lnet/lnet/lib-move.c b/drivers/staging/lustre/lnet/lnet/lib-move.c
> deleted file mode 100644
> index f8eaf8ff8d8d..000000000000
> diff --git a/drivers/staging/lustre/lnet/lnet/lib-msg.c b/drivers/staging/lustre/lnet/lnet/lib-msg.c
> deleted file mode 100644
> index 0091273c04b9..000000000000
> diff --git a/drivers/staging/lustre/lnet/lnet/lib-ptl.c b/drivers/staging/lustre/lnet/lnet/lib-ptl.c
> deleted file mode 100644
> index fc47379c5938..000000000000
> diff --git a/drivers/staging/lustre/lnet/lnet/lib-socket.c b/drivers/staging/lustre/lnet/lnet/lib-socket.c
> deleted file mode 100644
> index 9b61260155f2..000000000000
> diff --git a/drivers/staging/lustre/lnet/lnet/lo.c b/drivers/staging/lustre/lnet/lnet/lo.c
> deleted file mode 100644
> index 7456b989e451..000000000000
> diff --git a/drivers/staging/lustre/lnet/lnet/module.c b/drivers/staging/lustre/lnet/lnet/module.c
> deleted file mode 100644
> index 9d06664f0c17..000000000000
> diff --git a/drivers/staging/lustre/lnet/lnet/net_fault.c b/drivers/staging/lustre/lnet/lnet/net_fault.c
> deleted file mode 100644
> index 0066394b0bb0..000000000000
> diff --git a/drivers/staging/lustre/lnet/lnet/nidstrings.c b/drivers/staging/lustre/lnet/lnet/nidstrings.c
> deleted file mode 100644
> index 0f6c3fa16c65..000000000000
> diff --git a/drivers/staging/lustre/lnet/lnet/peer.c b/drivers/staging/lustre/lnet/lnet/peer.c
> deleted file mode 100644
> index 58294149f7b2..000000000000
> diff --git a/drivers/staging/lustre/lnet/lnet/router.c b/drivers/staging/lustre/lnet/lnet/router.c
> deleted file mode 100644
> index 6267d5e4bbd6..000000000000
> diff --git a/drivers/staging/lustre/lnet/lnet/router_proc.c b/drivers/staging/lustre/lnet/lnet/router_proc.c
> deleted file mode 100644
> index ae4b7f5953a0..000000000000
> diff --git a/drivers/staging/lustre/lnet/selftest/Makefile b/drivers/staging/lustre/lnet/selftest/Makefile
> deleted file mode 100644
> index 3ccc8966b566..000000000000
> diff --git a/drivers/staging/lustre/lnet/selftest/brw_test.c b/drivers/staging/lustre/lnet/selftest/brw_test.c
> deleted file mode 100644
> index f1ee219bc8f3..000000000000
> diff --git a/drivers/staging/lustre/lnet/selftest/conctl.c b/drivers/staging/lustre/lnet/selftest/conctl.c
> deleted file mode 100644
> index 906d82d90c0c..000000000000
> diff --git a/drivers/staging/lustre/lnet/selftest/conrpc.c b/drivers/staging/lustre/lnet/selftest/conrpc.c
> deleted file mode 100644
> index 0dabade3d091..000000000000
> diff --git a/drivers/staging/lustre/lnet/selftest/conrpc.h b/drivers/staging/lustre/lnet/selftest/conrpc.h
> deleted file mode 100644
> index ce2f92d04838..000000000000
> diff --git a/drivers/staging/lustre/lnet/selftest/console.c b/drivers/staging/lustre/lnet/selftest/console.c
> deleted file mode 100644
> index 3c1c1b5997e0..000000000000
> diff --git a/drivers/staging/lustre/lnet/selftest/console.h b/drivers/staging/lustre/lnet/selftest/console.h
> deleted file mode 100644
> index 2826205e36a1..000000000000
> diff --git a/drivers/staging/lustre/lnet/selftest/framework.c b/drivers/staging/lustre/lnet/selftest/framework.c
> deleted file mode 100644
> index 741af10560ad..000000000000
> diff --git a/drivers/staging/lustre/lnet/selftest/module.c b/drivers/staging/lustre/lnet/selftest/module.c
> deleted file mode 100644
> index 9ba65320f748..000000000000
> diff --git a/drivers/staging/lustre/lnet/selftest/ping_test.c b/drivers/staging/lustre/lnet/selftest/ping_test.c
> deleted file mode 100644
> index f54bd630dbf8..000000000000
> diff --git a/drivers/staging/lustre/lnet/selftest/rpc.c b/drivers/staging/lustre/lnet/selftest/rpc.c
> deleted file mode 100644
> index 9613b0a77007..000000000000
> diff --git a/drivers/staging/lustre/lnet/selftest/rpc.h b/drivers/staging/lustre/lnet/selftest/rpc.h
> deleted file mode 100644
> index 465b5b534423..000000000000
> diff --git a/drivers/staging/lustre/lnet/selftest/selftest.h b/drivers/staging/lustre/lnet/selftest/selftest.h
> deleted file mode 100644
> index 8737fa96b192..000000000000
> diff --git a/drivers/staging/lustre/lnet/selftest/timer.c b/drivers/staging/lustre/lnet/selftest/timer.c
> deleted file mode 100644
> index 582f252b3e12..000000000000
> diff --git a/drivers/staging/lustre/lnet/selftest/timer.h b/drivers/staging/lustre/lnet/selftest/timer.h
> deleted file mode 100644
> index 7f0ef9bd0cda..000000000000
> diff --git a/drivers/staging/lustre/lustre/Kconfig b/drivers/staging/lustre/lustre/Kconfig
> deleted file mode 100644
> index ccb78a945995..000000000000
> diff --git a/drivers/staging/lustre/lustre/Makefile b/drivers/staging/lustre/lustre/Makefile
> deleted file mode 100644
> index 331e4fcdd5a2..000000000000
> diff --git a/drivers/staging/lustre/lustre/fid/Makefile b/drivers/staging/lustre/lustre/fid/Makefile
> deleted file mode 100644
> index 77b65b92667d..000000000000
> diff --git a/drivers/staging/lustre/lustre/fid/fid_internal.h b/drivers/staging/lustre/lustre/fid/fid_internal.h
> deleted file mode 100644
> index 14569e969a31..000000000000
> diff --git a/drivers/staging/lustre/lustre/fid/fid_lib.c b/drivers/staging/lustre/lustre/fid/fid_lib.c
> deleted file mode 100644
> index ac52b378c155..000000000000
> diff --git a/drivers/staging/lustre/lustre/fid/fid_request.c b/drivers/staging/lustre/lustre/fid/fid_request.c
> deleted file mode 100644
> index a34fd90ca5e5..000000000000
> diff --git a/drivers/staging/lustre/lustre/fid/lproc_fid.c b/drivers/staging/lustre/lustre/fid/lproc_fid.c
> deleted file mode 100644
> index 0aabf473c9bd..000000000000
> diff --git a/drivers/staging/lustre/lustre/fld/Makefile b/drivers/staging/lustre/lustre/fld/Makefile
> deleted file mode 100644
> index 426deba8b815..000000000000
> diff --git a/drivers/staging/lustre/lustre/fld/fld_cache.c b/drivers/staging/lustre/lustre/fld/fld_cache.c
> deleted file mode 100644
> index a7415c9a1c28..000000000000
> diff --git a/drivers/staging/lustre/lustre/fld/fld_internal.h b/drivers/staging/lustre/lustre/fld/fld_internal.h
> deleted file mode 100644
> index e1d6aaa5c2b4..000000000000
> diff --git a/drivers/staging/lustre/lustre/fld/fld_request.c b/drivers/staging/lustre/lustre/fld/fld_request.c
> deleted file mode 100644
> index 97f7ea632346..000000000000
> diff --git a/drivers/staging/lustre/lustre/fld/lproc_fld.c b/drivers/staging/lustre/lustre/fld/lproc_fld.c
> deleted file mode 100644
> index 0bcfb26ef8aa..000000000000
> diff --git a/drivers/staging/lustre/lustre/include/cl_object.h b/drivers/staging/lustre/lustre/include/cl_object.h
> deleted file mode 100644
> index 6f7b991be809..000000000000
> diff --git a/drivers/staging/lustre/lustre/include/interval_tree.h b/drivers/staging/lustre/lustre/include/interval_tree.h
> deleted file mode 100644
> index 7d119c1a0469..000000000000
> diff --git a/drivers/staging/lustre/lustre/include/llog_swab.h b/drivers/staging/lustre/lustre/include/llog_swab.h
> deleted file mode 100644
> index 0433b79efdcb..000000000000
> diff --git a/drivers/staging/lustre/lustre/include/lprocfs_status.h b/drivers/staging/lustre/lustre/include/lprocfs_status.h
> deleted file mode 100644
> index 495e6f5f676b..000000000000
> diff --git a/drivers/staging/lustre/lustre/include/lu_object.h b/drivers/staging/lustre/lustre/include/lu_object.h
> deleted file mode 100644
> index f29bbca5af65..000000000000
> diff --git a/drivers/staging/lustre/lustre/include/lu_ref.h b/drivers/staging/lustre/lustre/include/lu_ref.h
> deleted file mode 100644
> index ad0c24d29ffa..000000000000
> diff --git a/drivers/staging/lustre/lustre/include/lustre_acl.h b/drivers/staging/lustre/lustre/include/lustre_acl.h
> deleted file mode 100644
> index e7575a172b5f..000000000000
> diff --git a/drivers/staging/lustre/lustre/include/lustre_compat.h b/drivers/staging/lustre/lustre/include/lustre_compat.h
> deleted file mode 100644
> index 3c6db0d632dc..000000000000
> diff --git a/drivers/staging/lustre/lustre/include/lustre_debug.h b/drivers/staging/lustre/lustre/include/lustre_debug.h
> deleted file mode 100644
> index 721a81f923e3..000000000000
> diff --git a/drivers/staging/lustre/lustre/include/lustre_disk.h b/drivers/staging/lustre/lustre/include/lustre_disk.h
> deleted file mode 100644
> index 886e817644d6..000000000000
> diff --git a/drivers/staging/lustre/lustre/include/lustre_dlm.h b/drivers/staging/lustre/lustre/include/lustre_dlm.h
> deleted file mode 100644
> index 2c55241258cc..000000000000
> diff --git a/drivers/staging/lustre/lustre/include/lustre_dlm_flags.h b/drivers/staging/lustre/lustre/include/lustre_dlm_flags.h
> deleted file mode 100644
> index 53db031c4c8c..000000000000
> diff --git a/drivers/staging/lustre/lustre/include/lustre_errno.h b/drivers/staging/lustre/lustre/include/lustre_errno.h
> deleted file mode 100644
> index 59fbb9f47ff1..000000000000
> diff --git a/drivers/staging/lustre/lustre/include/lustre_export.h b/drivers/staging/lustre/lustre/include/lustre_export.h
> deleted file mode 100644
> index 79ad5aae86b9..000000000000
> diff --git a/drivers/staging/lustre/lustre/include/lustre_fid.h b/drivers/staging/lustre/lustre/include/lustre_fid.h
> deleted file mode 100644
> index 094ad282de2c..000000000000
> diff --git a/drivers/staging/lustre/lustre/include/lustre_fld.h b/drivers/staging/lustre/lustre/include/lustre_fld.h
> deleted file mode 100644
> index f42122a4dfaa..000000000000
> diff --git a/drivers/staging/lustre/lustre/include/lustre_ha.h b/drivers/staging/lustre/lustre/include/lustre_ha.h
> deleted file mode 100644
> index cbd68985ada9..000000000000
> diff --git a/drivers/staging/lustre/lustre/include/lustre_handles.h b/drivers/staging/lustre/lustre/include/lustre_handles.h
> deleted file mode 100644
> index 3556ce8d94e8..000000000000
> diff --git a/drivers/staging/lustre/lustre/include/lustre_import.h b/drivers/staging/lustre/lustre/include/lustre_import.h
> deleted file mode 100644
> index ac3805ead620..000000000000
> diff --git a/drivers/staging/lustre/lustre/include/lustre_intent.h b/drivers/staging/lustre/lustre/include/lustre_intent.h
> deleted file mode 100644
> index 51e5c0e03872..000000000000
> diff --git a/drivers/staging/lustre/lustre/include/lustre_kernelcomm.h b/drivers/staging/lustre/lustre/include/lustre_kernelcomm.h
> deleted file mode 100644
> index 2b3fa8430185..000000000000
> diff --git a/drivers/staging/lustre/lustre/include/lustre_lib.h b/drivers/staging/lustre/lustre/include/lustre_lib.h
> deleted file mode 100644
> index 87748e9902a7..000000000000
> diff --git a/drivers/staging/lustre/lustre/include/lustre_linkea.h b/drivers/staging/lustre/lustre/include/lustre_linkea.h
> deleted file mode 100644
> index 03db1511bfd3..000000000000
> diff --git a/drivers/staging/lustre/lustre/include/lustre_lmv.h b/drivers/staging/lustre/lustre/include/lustre_lmv.h
> deleted file mode 100644
> index 080ec1f8e19f..000000000000
> diff --git a/drivers/staging/lustre/lustre/include/lustre_log.h b/drivers/staging/lustre/lustre/include/lustre_log.h
> deleted file mode 100644
> index 07f4e600386b..000000000000
> diff --git a/drivers/staging/lustre/lustre/include/lustre_mdc.h b/drivers/staging/lustre/lustre/include/lustre_mdc.h
> deleted file mode 100644
> index a9c9992a2502..000000000000
> diff --git a/drivers/staging/lustre/lustre/include/lustre_mds.h b/drivers/staging/lustre/lustre/include/lustre_mds.h
> deleted file mode 100644
> index f665556556ec..000000000000
> diff --git a/drivers/staging/lustre/lustre/include/lustre_net.h b/drivers/staging/lustre/lustre/include/lustre_net.h
> deleted file mode 100644
> index 35b43a77eb18..000000000000
> diff --git a/drivers/staging/lustre/lustre/include/lustre_nrs.h b/drivers/staging/lustre/lustre/include/lustre_nrs.h
> deleted file mode 100644
> index ffa7317da35b..000000000000
> diff --git a/drivers/staging/lustre/lustre/include/lustre_nrs_fifo.h b/drivers/staging/lustre/lustre/include/lustre_nrs_fifo.h
> deleted file mode 100644
> index b70d97d4acbb..000000000000
> diff --git a/drivers/staging/lustre/lustre/include/lustre_obdo.h b/drivers/staging/lustre/lustre/include/lustre_obdo.h
> deleted file mode 100644
> index d67dcbb84f18..000000000000
> diff --git a/drivers/staging/lustre/lustre/include/lustre_patchless_compat.h b/drivers/staging/lustre/lustre/include/lustre_patchless_compat.h
> deleted file mode 100644
> index 298476ea7557..000000000000
> diff --git a/drivers/staging/lustre/lustre/include/lustre_req_layout.h b/drivers/staging/lustre/lustre/include/lustre_req_layout.h
> deleted file mode 100644
> index 213d0a01adcf..000000000000
> diff --git a/drivers/staging/lustre/lustre/include/lustre_sec.h b/drivers/staging/lustre/lustre/include/lustre_sec.h
> deleted file mode 100644
> index d35bcbc98831..000000000000
> diff --git a/drivers/staging/lustre/lustre/include/lustre_swab.h b/drivers/staging/lustre/lustre/include/lustre_swab.h
> deleted file mode 100644
> index 9d786bbe7f3f..000000000000
> diff --git a/drivers/staging/lustre/lustre/include/obd.h b/drivers/staging/lustre/lustre/include/obd.h
> deleted file mode 100644
> index b1907bbffb19..000000000000
> diff --git a/drivers/staging/lustre/lustre/include/obd_cksum.h b/drivers/staging/lustre/lustre/include/obd_cksum.h
> deleted file mode 100644
> index e5f7bb20415d..000000000000
> diff --git a/drivers/staging/lustre/lustre/include/obd_class.h b/drivers/staging/lustre/lustre/include/obd_class.h
> deleted file mode 100644
> index fc9c7720fee0..000000000000
> diff --git a/drivers/staging/lustre/lustre/include/obd_support.h b/drivers/staging/lustre/lustre/include/obd_support.h
> deleted file mode 100644
> index 9e41633823f7..000000000000
> diff --git a/drivers/staging/lustre/lustre/include/seq_range.h b/drivers/staging/lustre/lustre/include/seq_range.h
> deleted file mode 100644
> index 9450da728160..000000000000
> diff --git a/drivers/staging/lustre/lustre/ldlm/interval_tree.c b/drivers/staging/lustre/lustre/ldlm/interval_tree.c
> deleted file mode 100644
> index 8df7a4463c21..000000000000
> diff --git a/drivers/staging/lustre/lustre/ldlm/l_lock.c b/drivers/staging/lustre/lustre/ldlm/l_lock.c
> deleted file mode 100644
> index 296259aa51e6..000000000000
> diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_extent.c b/drivers/staging/lustre/lustre/ldlm/ldlm_extent.c
> deleted file mode 100644
> index 4da23ade2bb3..000000000000
> diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_flock.c b/drivers/staging/lustre/lustre/ldlm/ldlm_flock.c
> deleted file mode 100644
> index 94f3b1e49896..000000000000
> diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_inodebits.c b/drivers/staging/lustre/lustre/ldlm/ldlm_inodebits.c
> deleted file mode 100644
> index 2926208cdfa1..000000000000
> diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_internal.h b/drivers/staging/lustre/lustre/ldlm/ldlm_internal.h
> deleted file mode 100644
> index bc33ca100620..000000000000
> diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_lib.c b/drivers/staging/lustre/lustre/ldlm/ldlm_lib.c
> deleted file mode 100644
> index 0aa4f234a4f4..000000000000
> diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c b/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c
> deleted file mode 100644
> index a644d133063b..000000000000
> diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c b/drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c
> deleted file mode 100644
> index 5963e90d0938..000000000000
> diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_plain.c b/drivers/staging/lustre/lustre/ldlm/ldlm_plain.c
> deleted file mode 100644
> index 33b5a3f96fcb..000000000000
> diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c b/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c
> deleted file mode 100644
> index 36d14ee4e5b1..000000000000
> diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_request.c b/drivers/staging/lustre/lustre/ldlm/ldlm_request.c
> deleted file mode 100644
> index cdc52eed6d85..000000000000
> diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_resource.c b/drivers/staging/lustre/lustre/ldlm/ldlm_resource.c
> deleted file mode 100644
> index c93b019b8e37..000000000000
> diff --git a/drivers/staging/lustre/lustre/llite/Makefile b/drivers/staging/lustre/lustre/llite/Makefile
> deleted file mode 100644
> index 5200924182ae..000000000000
> diff --git a/drivers/staging/lustre/lustre/llite/acl.c b/drivers/staging/lustre/lustre/llite/acl.c
> deleted file mode 100644
> index 2ee9ff931236..000000000000
> diff --git a/drivers/staging/lustre/lustre/llite/dcache.c b/drivers/staging/lustre/lustre/llite/dcache.c
> deleted file mode 100644
> index 11b82c639bfe..000000000000
> diff --git a/drivers/staging/lustre/lustre/llite/dir.c b/drivers/staging/lustre/lustre/llite/dir.c
> deleted file mode 100644
> index 688dddf3ca47..000000000000
> diff --git a/drivers/staging/lustre/lustre/llite/file.c b/drivers/staging/lustre/lustre/llite/file.c
> deleted file mode 100644
> index 02295931883b..000000000000
> diff --git a/drivers/staging/lustre/lustre/llite/glimpse.c b/drivers/staging/lustre/lustre/llite/glimpse.c
> deleted file mode 100644
> index ce0d51767da3..000000000000
> diff --git a/drivers/staging/lustre/lustre/llite/lcommon_cl.c b/drivers/staging/lustre/lustre/llite/lcommon_cl.c
> deleted file mode 100644
> index d7ea39ce0cb2..000000000000
> diff --git a/drivers/staging/lustre/lustre/llite/lcommon_misc.c b/drivers/staging/lustre/lustre/llite/lcommon_misc.c
> deleted file mode 100644
> index a246b955306e..000000000000
> diff --git a/drivers/staging/lustre/lustre/llite/llite_internal.h b/drivers/staging/lustre/lustre/llite/llite_internal.h
> deleted file mode 100644
> index c08a6e14b6d7..000000000000
> diff --git a/drivers/staging/lustre/lustre/llite/llite_lib.c b/drivers/staging/lustre/lustre/llite/llite_lib.c
> deleted file mode 100644
> index 36066c839160..000000000000
> diff --git a/drivers/staging/lustre/lustre/llite/llite_mmap.c b/drivers/staging/lustre/lustre/llite/llite_mmap.c
> deleted file mode 100644
> index d7fb5533f707..000000000000
> diff --git a/drivers/staging/lustre/lustre/llite/llite_nfs.c b/drivers/staging/lustre/lustre/llite/llite_nfs.c
> deleted file mode 100644
> index 14172688d55f..000000000000
> diff --git a/drivers/staging/lustre/lustre/llite/lproc_llite.c b/drivers/staging/lustre/lustre/llite/lproc_llite.c
> deleted file mode 100644
> index 49bf1b7ee311..000000000000
> diff --git a/drivers/staging/lustre/lustre/llite/namei.c b/drivers/staging/lustre/lustre/llite/namei.c
> deleted file mode 100644
> index d5f6d20afe8c..000000000000
> diff --git a/drivers/staging/lustre/lustre/llite/range_lock.c b/drivers/staging/lustre/lustre/llite/range_lock.c
> deleted file mode 100644
> index 008a8874118d..000000000000
> diff --git a/drivers/staging/lustre/lustre/llite/range_lock.h b/drivers/staging/lustre/lustre/llite/range_lock.h
> deleted file mode 100644
> index 9ebac09160f2..000000000000
> diff --git a/drivers/staging/lustre/lustre/llite/rw.c b/drivers/staging/lustre/lustre/llite/rw.c
> deleted file mode 100644
> index 3e008ce7275d..000000000000
> diff --git a/drivers/staging/lustre/lustre/llite/rw26.c b/drivers/staging/lustre/lustre/llite/rw26.c
> deleted file mode 100644
> index 722e5ea1af5f..000000000000
> diff --git a/drivers/staging/lustre/lustre/llite/statahead.c b/drivers/staging/lustre/lustre/llite/statahead.c
> deleted file mode 100644
> index d864f5f36d85..000000000000
> diff --git a/drivers/staging/lustre/lustre/llite/super25.c b/drivers/staging/lustre/lustre/llite/super25.c
> deleted file mode 100644
> index d335f29556c2..000000000000
> diff --git a/drivers/staging/lustre/lustre/llite/symlink.c b/drivers/staging/lustre/lustre/llite/symlink.c
> deleted file mode 100644
> index 0690fdbf49f5..000000000000
> diff --git a/drivers/staging/lustre/lustre/llite/vvp_dev.c b/drivers/staging/lustre/lustre/llite/vvp_dev.c
> deleted file mode 100644
> index 31dc3c0ade01..000000000000
> diff --git a/drivers/staging/lustre/lustre/llite/vvp_internal.h b/drivers/staging/lustre/lustre/llite/vvp_internal.h
> deleted file mode 100644
> index 7d3abb43584a..000000000000
> diff --git a/drivers/staging/lustre/lustre/llite/vvp_io.c b/drivers/staging/lustre/lustre/llite/vvp_io.c
> deleted file mode 100644
> index e7a4778e02e4..000000000000
> diff --git a/drivers/staging/lustre/lustre/llite/vvp_lock.c b/drivers/staging/lustre/lustre/llite/vvp_lock.c
> deleted file mode 100644
> index 4b6c7143bd2c..000000000000
> diff --git a/drivers/staging/lustre/lustre/llite/vvp_object.c b/drivers/staging/lustre/lustre/llite/vvp_object.c
> deleted file mode 100644
> index b2cb51c8f7f4..000000000000
> diff --git a/drivers/staging/lustre/lustre/llite/vvp_page.c b/drivers/staging/lustre/lustre/llite/vvp_page.c
> deleted file mode 100644
> index 6eb0565ddc22..000000000000
> diff --git a/drivers/staging/lustre/lustre/llite/xattr.c b/drivers/staging/lustre/lustre/llite/xattr.c
> deleted file mode 100644
> index 7fa0a419c094..000000000000
> diff --git a/drivers/staging/lustre/lustre/llite/xattr_cache.c b/drivers/staging/lustre/lustre/llite/xattr_cache.c
> deleted file mode 100644
> index 5da69ba088c4..000000000000
> diff --git a/drivers/staging/lustre/lustre/llite/xattr_security.c b/drivers/staging/lustre/lustre/llite/xattr_security.c
> deleted file mode 100644
> index 93ec07531ac7..000000000000
> diff --git a/drivers/staging/lustre/lustre/lmv/Makefile b/drivers/staging/lustre/lustre/lmv/Makefile
> deleted file mode 100644
> index 91c99114aa13..000000000000
> diff --git a/drivers/staging/lustre/lustre/lmv/lmv_fld.c b/drivers/staging/lustre/lustre/lmv/lmv_fld.c
> deleted file mode 100644
> index 00dc858c10c9..000000000000
> diff --git a/drivers/staging/lustre/lustre/lmv/lmv_intent.c b/drivers/staging/lustre/lustre/lmv/lmv_intent.c
> deleted file mode 100644
> index 1e850fdbc623..000000000000
> diff --git a/drivers/staging/lustre/lustre/lmv/lmv_internal.h b/drivers/staging/lustre/lustre/lmv/lmv_internal.h
> deleted file mode 100644
> index 68a99170c424..000000000000
> diff --git a/drivers/staging/lustre/lustre/lmv/lmv_obd.c b/drivers/staging/lustre/lustre/lmv/lmv_obd.c
> deleted file mode 100644
> index 65f94e6ecaad..000000000000
> diff --git a/drivers/staging/lustre/lustre/lmv/lproc_lmv.c b/drivers/staging/lustre/lustre/lmv/lproc_lmv.c
> deleted file mode 100644
> index 30727b7acccc..000000000000
> diff --git a/drivers/staging/lustre/lustre/lov/Makefile b/drivers/staging/lustre/lustre/lov/Makefile
> deleted file mode 100644
> index 1ebf0193f61a..000000000000
> diff --git a/drivers/staging/lustre/lustre/lov/lov_cl_internal.h b/drivers/staging/lustre/lustre/lov/lov_cl_internal.h
> deleted file mode 100644
> index e4f762137a4a..000000000000
> diff --git a/drivers/staging/lustre/lustre/lov/lov_dev.c b/drivers/staging/lustre/lustre/lov/lov_dev.c
> deleted file mode 100644
> index c7db23472346..000000000000
> diff --git a/drivers/staging/lustre/lustre/lov/lov_ea.c b/drivers/staging/lustre/lustre/lov/lov_ea.c
> deleted file mode 100644
> index c80320ab0858..000000000000
> diff --git a/drivers/staging/lustre/lustre/lov/lov_internal.h b/drivers/staging/lustre/lustre/lov/lov_internal.h
> deleted file mode 100644
> index 47042f27ca90..000000000000
> diff --git a/drivers/staging/lustre/lustre/lov/lov_io.c b/drivers/staging/lustre/lustre/lov/lov_io.c
> deleted file mode 100644
> index b823f8a21856..000000000000
> diff --git a/drivers/staging/lustre/lustre/lov/lov_lock.c b/drivers/staging/lustre/lustre/lov/lov_lock.c
> deleted file mode 100644
> index b0292100bf26..000000000000
> diff --git a/drivers/staging/lustre/lustre/lov/lov_merge.c b/drivers/staging/lustre/lustre/lov/lov_merge.c
> deleted file mode 100644
> index 006717cf7a41..000000000000
> diff --git a/drivers/staging/lustre/lustre/lov/lov_obd.c b/drivers/staging/lustre/lustre/lov/lov_obd.c
> deleted file mode 100644
> index 344ff4b20168..000000000000
> diff --git a/drivers/staging/lustre/lustre/lov/lov_object.c b/drivers/staging/lustre/lustre/lov/lov_object.c
> deleted file mode 100644
> index adc90f310fd7..000000000000
> diff --git a/drivers/staging/lustre/lustre/lov/lov_offset.c b/drivers/staging/lustre/lustre/lov/lov_offset.c
> deleted file mode 100644
> index a5f00f6ec347..000000000000
> diff --git a/drivers/staging/lustre/lustre/lov/lov_pack.c b/drivers/staging/lustre/lustre/lov/lov_pack.c
> deleted file mode 100644
> index b1060d02a164..000000000000
> diff --git a/drivers/staging/lustre/lustre/lov/lov_page.c b/drivers/staging/lustre/lustre/lov/lov_page.c
> deleted file mode 100644
> index cfae1294d77a..000000000000
> diff --git a/drivers/staging/lustre/lustre/lov/lov_pool.c b/drivers/staging/lustre/lustre/lov/lov_pool.c
> deleted file mode 100644
> index b2a88ba72eb2..000000000000
> diff --git a/drivers/staging/lustre/lustre/lov/lov_request.c b/drivers/staging/lustre/lustre/lov/lov_request.c
> deleted file mode 100644
> index cb8567f20ea7..000000000000
> diff --git a/drivers/staging/lustre/lustre/lov/lovsub_dev.c b/drivers/staging/lustre/lustre/lov/lovsub_dev.c
> deleted file mode 100644
> index 7e89a2e485fc..000000000000
> diff --git a/drivers/staging/lustre/lustre/lov/lovsub_lock.c b/drivers/staging/lustre/lustre/lov/lovsub_lock.c
> deleted file mode 100644
> index ea492be2eef3..000000000000
> diff --git a/drivers/staging/lustre/lustre/lov/lovsub_object.c b/drivers/staging/lustre/lustre/lov/lovsub_object.c
> deleted file mode 100644
> index 13d452086b61..000000000000
> diff --git a/drivers/staging/lustre/lustre/lov/lovsub_page.c b/drivers/staging/lustre/lustre/lov/lovsub_page.c
> deleted file mode 100644
> index 915520bcdd60..000000000000
> diff --git a/drivers/staging/lustre/lustre/lov/lproc_lov.c b/drivers/staging/lustre/lustre/lov/lproc_lov.c
> deleted file mode 100644
> index 721440feef72..000000000000
> diff --git a/drivers/staging/lustre/lustre/mdc/Makefile b/drivers/staging/lustre/lustre/mdc/Makefile
> deleted file mode 100644
> index c7bc3351ccb0..000000000000
> diff --git a/drivers/staging/lustre/lustre/mdc/lproc_mdc.c b/drivers/staging/lustre/lustre/mdc/lproc_mdc.c
> deleted file mode 100644
> index 6cce32491eb5..000000000000
> diff --git a/drivers/staging/lustre/lustre/mdc/mdc_internal.h b/drivers/staging/lustre/lustre/mdc/mdc_internal.h
> deleted file mode 100644
> index 28924e927b50..000000000000
> diff --git a/drivers/staging/lustre/lustre/mdc/mdc_lib.c b/drivers/staging/lustre/lustre/mdc/mdc_lib.c
> deleted file mode 100644
> index d582968987ff..000000000000
> diff --git a/drivers/staging/lustre/lustre/mdc/mdc_locks.c b/drivers/staging/lustre/lustre/mdc/mdc_locks.c
> deleted file mode 100644
> index a8aa0fa5e87a..000000000000
> diff --git a/drivers/staging/lustre/lustre/mdc/mdc_reint.c b/drivers/staging/lustre/lustre/mdc/mdc_reint.c
> deleted file mode 100644
> index e77c00df0693..000000000000
> diff --git a/drivers/staging/lustre/lustre/mdc/mdc_request.c b/drivers/staging/lustre/lustre/mdc/mdc_request.c
> deleted file mode 100644
> index cff31cb0a9ac..000000000000
> diff --git a/drivers/staging/lustre/lustre/mgc/Makefile b/drivers/staging/lustre/lustre/mgc/Makefile
> deleted file mode 100644
> index 8abf108dbcf7..000000000000
> diff --git a/drivers/staging/lustre/lustre/mgc/lproc_mgc.c b/drivers/staging/lustre/lustre/mgc/lproc_mgc.c
> deleted file mode 100644
> index 636770624e8f..000000000000
> diff --git a/drivers/staging/lustre/lustre/mgc/mgc_internal.h b/drivers/staging/lustre/lustre/mgc/mgc_internal.h
> deleted file mode 100644
> index 9541892b67c7..000000000000
> diff --git a/drivers/staging/lustre/lustre/mgc/mgc_request.c b/drivers/staging/lustre/lustre/mgc/mgc_request.c
> deleted file mode 100644
> index 32df804614d3..000000000000
> diff --git a/drivers/staging/lustre/lustre/obdclass/Makefile b/drivers/staging/lustre/lustre/obdclass/Makefile
> deleted file mode 100644
> index e3fa9acff4c4..000000000000
> diff --git a/drivers/staging/lustre/lustre/obdclass/cl_internal.h b/drivers/staging/lustre/lustre/obdclass/cl_internal.h
> deleted file mode 100644
> index a0db830ca841..000000000000
> diff --git a/drivers/staging/lustre/lustre/obdclass/cl_io.c b/drivers/staging/lustre/lustre/obdclass/cl_io.c
> deleted file mode 100644
> index fcdae6029258..000000000000
> diff --git a/drivers/staging/lustre/lustre/obdclass/cl_lock.c b/drivers/staging/lustre/lustre/obdclass/cl_lock.c
> deleted file mode 100644
> index 9ca29a26a38b..000000000000
> diff --git a/drivers/staging/lustre/lustre/obdclass/cl_object.c b/drivers/staging/lustre/lustre/obdclass/cl_object.c
> deleted file mode 100644
> index 42cce2dc5a45..000000000000
> diff --git a/drivers/staging/lustre/lustre/obdclass/cl_page.c b/drivers/staging/lustre/lustre/obdclass/cl_page.c
> deleted file mode 100644
> index 916cf81c5997..000000000000
> diff --git a/drivers/staging/lustre/lustre/obdclass/class_obd.c b/drivers/staging/lustre/lustre/obdclass/class_obd.c
> deleted file mode 100644
> index d6c46858941b..000000000000
> diff --git a/drivers/staging/lustre/lustre/obdclass/debug.c b/drivers/staging/lustre/lustre/obdclass/debug.c
> deleted file mode 100644
> index 2156a82a613a..000000000000
> diff --git a/drivers/staging/lustre/lustre/obdclass/genops.c b/drivers/staging/lustre/lustre/obdclass/genops.c
> deleted file mode 100644
> index 234f383ce6d9..000000000000
> diff --git a/drivers/staging/lustre/lustre/obdclass/kernelcomm.c b/drivers/staging/lustre/lustre/obdclass/kernelcomm.c
> deleted file mode 100644
> index 63067a7f1e19..000000000000
> diff --git a/drivers/staging/lustre/lustre/obdclass/linkea.c b/drivers/staging/lustre/lustre/obdclass/linkea.c
> deleted file mode 100644
> index 74c99ee216bb..000000000000
> diff --git a/drivers/staging/lustre/lustre/obdclass/linux/linux-module.c b/drivers/staging/lustre/lustre/obdclass/linux/linux-module.c
> deleted file mode 100644
> index 9c800580053b..000000000000
> diff --git a/drivers/staging/lustre/lustre/obdclass/linux/linux-sysctl.c b/drivers/staging/lustre/lustre/obdclass/linux/linux-sysctl.c
> deleted file mode 100644
> index e5e8687784ee..000000000000
> diff --git a/drivers/staging/lustre/lustre/obdclass/llog.c b/drivers/staging/lustre/lustre/obdclass/llog.c
> deleted file mode 100644
> index bba84eae1e19..000000000000
> diff --git a/drivers/staging/lustre/lustre/obdclass/llog_cat.c b/drivers/staging/lustre/lustre/obdclass/llog_cat.c
> deleted file mode 100644
> index d9c63adff206..000000000000
> diff --git a/drivers/staging/lustre/lustre/obdclass/llog_internal.h b/drivers/staging/lustre/lustre/obdclass/llog_internal.h
> deleted file mode 100644
> index 4991d4e589dc..000000000000
> diff --git a/drivers/staging/lustre/lustre/obdclass/llog_obd.c b/drivers/staging/lustre/lustre/obdclass/llog_obd.c
> deleted file mode 100644
> index 26aea114a29b..000000000000
> diff --git a/drivers/staging/lustre/lustre/obdclass/llog_swab.c b/drivers/staging/lustre/lustre/obdclass/llog_swab.c
> deleted file mode 100644
> index b431c3408fe4..000000000000
> diff --git a/drivers/staging/lustre/lustre/obdclass/lprocfs_counters.c b/drivers/staging/lustre/lustre/obdclass/lprocfs_counters.c
> deleted file mode 100644
> index 85f09aff6e83..000000000000
> diff --git a/drivers/staging/lustre/lustre/obdclass/lprocfs_status.c b/drivers/staging/lustre/lustre/obdclass/lprocfs_status.c
> deleted file mode 100644
> index bdbe6f52031a..000000000000
> diff --git a/drivers/staging/lustre/lustre/obdclass/lu_object.c b/drivers/staging/lustre/lustre/obdclass/lu_object.c
> deleted file mode 100644
> index aa9d74e087f4..000000000000
> diff --git a/drivers/staging/lustre/lustre/obdclass/lu_ref.c b/drivers/staging/lustre/lustre/obdclass/lu_ref.c
> deleted file mode 100644
> index f67cb89ea0ba..000000000000
> diff --git a/drivers/staging/lustre/lustre/obdclass/lustre_handles.c b/drivers/staging/lustre/lustre/obdclass/lustre_handles.c
> deleted file mode 100644
> index cdc8dc10690d..000000000000
> diff --git a/drivers/staging/lustre/lustre/obdclass/lustre_peer.c b/drivers/staging/lustre/lustre/obdclass/lustre_peer.c
> deleted file mode 100644
> index e286a2665423..000000000000
> diff --git a/drivers/staging/lustre/lustre/obdclass/obd_config.c b/drivers/staging/lustre/lustre/obdclass/obd_config.c
> deleted file mode 100644
> index ffc1814398a5..000000000000
> diff --git a/drivers/staging/lustre/lustre/obdclass/obd_mount.c b/drivers/staging/lustre/lustre/obdclass/obd_mount.c
> deleted file mode 100644
> index 06c38fdef7ba..000000000000
> diff --git a/drivers/staging/lustre/lustre/obdclass/obdo.c b/drivers/staging/lustre/lustre/obdclass/obdo.c
> deleted file mode 100644
> index c4503bc36591..000000000000
> diff --git a/drivers/staging/lustre/lustre/obdclass/statfs_pack.c b/drivers/staging/lustre/lustre/obdclass/statfs_pack.c
> deleted file mode 100644
> index 355e888885f4..000000000000
> diff --git a/drivers/staging/lustre/lustre/obdclass/uuid.c b/drivers/staging/lustre/lustre/obdclass/uuid.c
> deleted file mode 100644
> index ec8c6dc5c9a7..000000000000
> diff --git a/drivers/staging/lustre/lustre/obdecho/Makefile b/drivers/staging/lustre/lustre/obdecho/Makefile
> deleted file mode 100644
> index 6be66fbab872..000000000000
> diff --git a/drivers/staging/lustre/lustre/obdecho/echo_client.c b/drivers/staging/lustre/lustre/obdecho/echo_client.c
> deleted file mode 100644
> index b692e76e7108..000000000000
> diff --git a/drivers/staging/lustre/lustre/obdecho/echo_internal.h b/drivers/staging/lustre/lustre/obdecho/echo_internal.h
> deleted file mode 100644
> index 42faa164fabb..000000000000
> diff --git a/drivers/staging/lustre/lustre/osc/Makefile b/drivers/staging/lustre/lustre/osc/Makefile
> deleted file mode 100644
> index 30dec90e64e8..000000000000
> diff --git a/drivers/staging/lustre/lustre/osc/lproc_osc.c b/drivers/staging/lustre/lustre/osc/lproc_osc.c
> deleted file mode 100644
> index 6a705bc5420c..000000000000
> diff --git a/drivers/staging/lustre/lustre/osc/osc_cache.c b/drivers/staging/lustre/lustre/osc/osc_cache.c
> deleted file mode 100644
> index f26983004843..000000000000
> diff --git a/drivers/staging/lustre/lustre/osc/osc_cl_internal.h b/drivers/staging/lustre/lustre/osc/osc_cl_internal.h
> deleted file mode 100644
> index 2d3cba16ef34..000000000000
> diff --git a/drivers/staging/lustre/lustre/osc/osc_dev.c b/drivers/staging/lustre/lustre/osc/osc_dev.c
> deleted file mode 100644
> index 2b5f324743e2..000000000000
> diff --git a/drivers/staging/lustre/lustre/osc/osc_internal.h b/drivers/staging/lustre/lustre/osc/osc_internal.h
> deleted file mode 100644
> index 4ddba1354bef..000000000000
> diff --git a/drivers/staging/lustre/lustre/osc/osc_io.c b/drivers/staging/lustre/lustre/osc/osc_io.c
> deleted file mode 100644
> index 67734a8ed331..000000000000
> diff --git a/drivers/staging/lustre/lustre/osc/osc_lock.c b/drivers/staging/lustre/lustre/osc/osc_lock.c
> deleted file mode 100644
> index d93d33dc8dc4..000000000000
> diff --git a/drivers/staging/lustre/lustre/osc/osc_object.c b/drivers/staging/lustre/lustre/osc/osc_object.c
> deleted file mode 100644
> index 84240181c7ea..000000000000
> diff --git a/drivers/staging/lustre/lustre/osc/osc_page.c b/drivers/staging/lustre/lustre/osc/osc_page.c
> deleted file mode 100644
> index 20c553ef3a5e..000000000000
> diff --git a/drivers/staging/lustre/lustre/osc/osc_quota.c b/drivers/staging/lustre/lustre/osc/osc_quota.c
> deleted file mode 100644
> index 723ec2fb18bf..000000000000
> diff --git a/drivers/staging/lustre/lustre/osc/osc_request.c b/drivers/staging/lustre/lustre/osc/osc_request.c
> deleted file mode 100644
> index 61ef6c8d7a12..000000000000
> diff --git a/drivers/staging/lustre/lustre/ptlrpc/Makefile b/drivers/staging/lustre/lustre/ptlrpc/Makefile
> deleted file mode 100644
> index 1deb1971b39e..000000000000
> diff --git a/drivers/staging/lustre/lustre/ptlrpc/client.c b/drivers/staging/lustre/lustre/ptlrpc/client.c
> deleted file mode 100644
> index c1b82bf20f08..000000000000
> diff --git a/drivers/staging/lustre/lustre/ptlrpc/connection.c b/drivers/staging/lustre/lustre/ptlrpc/connection.c
> deleted file mode 100644
> index fb35a89ca6c6..000000000000
> diff --git a/drivers/staging/lustre/lustre/ptlrpc/errno.c b/drivers/staging/lustre/lustre/ptlrpc/errno.c
> deleted file mode 100644
> index b904524fc1c6..000000000000
> diff --git a/drivers/staging/lustre/lustre/ptlrpc/events.c b/drivers/staging/lustre/lustre/ptlrpc/events.c
> deleted file mode 100644
> index 130bacc2c891..000000000000
> diff --git a/drivers/staging/lustre/lustre/ptlrpc/import.c b/drivers/staging/lustre/lustre/ptlrpc/import.c
> deleted file mode 100644
> index 1a0f35dfab97..000000000000
> diff --git a/drivers/staging/lustre/lustre/ptlrpc/layout.c b/drivers/staging/lustre/lustre/ptlrpc/layout.c
> deleted file mode 100644
> index 417d4a151433..000000000000
> diff --git a/drivers/staging/lustre/lustre/ptlrpc/llog_client.c b/drivers/staging/lustre/lustre/ptlrpc/llog_client.c
> deleted file mode 100644
> index 946d538121de..000000000000
> diff --git a/drivers/staging/lustre/lustre/ptlrpc/llog_net.c b/drivers/staging/lustre/lustre/ptlrpc/llog_net.c
> deleted file mode 100644
> index b871d9e40a9e..000000000000
> diff --git a/drivers/staging/lustre/lustre/ptlrpc/lproc_ptlrpc.c b/drivers/staging/lustre/lustre/ptlrpc/lproc_ptlrpc.c
> deleted file mode 100644
> index 0b638837f88b..000000000000
> diff --git a/drivers/staging/lustre/lustre/ptlrpc/niobuf.c b/drivers/staging/lustre/lustre/ptlrpc/niobuf.c
> deleted file mode 100644
> index 2897afb8806c..000000000000
> diff --git a/drivers/staging/lustre/lustre/ptlrpc/nrs.c b/drivers/staging/lustre/lustre/ptlrpc/nrs.c
> deleted file mode 100644
> index e09b86529c5d..000000000000
> diff --git a/drivers/staging/lustre/lustre/ptlrpc/nrs_fifo.c b/drivers/staging/lustre/lustre/ptlrpc/nrs_fifo.c
> deleted file mode 100644
> index ff630d94dd26..000000000000
> diff --git a/drivers/staging/lustre/lustre/ptlrpc/pack_generic.c b/drivers/staging/lustre/lustre/ptlrpc/pack_generic.c
> deleted file mode 100644
> index 6ac9bb570663..000000000000
> diff --git a/drivers/staging/lustre/lustre/ptlrpc/pers.c b/drivers/staging/lustre/lustre/ptlrpc/pers.c
> deleted file mode 100644
> index 2466868afb9c..000000000000
> diff --git a/drivers/staging/lustre/lustre/ptlrpc/pinger.c b/drivers/staging/lustre/lustre/ptlrpc/pinger.c
> deleted file mode 100644
> index b3297b5ce395..000000000000
> diff --git a/drivers/staging/lustre/lustre/ptlrpc/ptlrpc_internal.h b/drivers/staging/lustre/lustre/ptlrpc/ptlrpc_internal.h
> deleted file mode 100644
> index 134b74234519..000000000000
> diff --git a/drivers/staging/lustre/lustre/ptlrpc/ptlrpc_module.c b/drivers/staging/lustre/lustre/ptlrpc/ptlrpc_module.c
> deleted file mode 100644
> index 5c32b657b3b5..000000000000
> diff --git a/drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c b/drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c
> deleted file mode 100644
> index 531005411edf..000000000000
> diff --git a/drivers/staging/lustre/lustre/ptlrpc/recover.c b/drivers/staging/lustre/lustre/ptlrpc/recover.c
> deleted file mode 100644
> index 2ea0a7ff87dd..000000000000
> diff --git a/drivers/staging/lustre/lustre/ptlrpc/sec.c b/drivers/staging/lustre/lustre/ptlrpc/sec.c
> deleted file mode 100644
> index e193f3346e6f..000000000000
> diff --git a/drivers/staging/lustre/lustre/ptlrpc/sec_bulk.c b/drivers/staging/lustre/lustre/ptlrpc/sec_bulk.c
> deleted file mode 100644
> index 625b9520d78f..000000000000
> diff --git a/drivers/staging/lustre/lustre/ptlrpc/sec_config.c b/drivers/staging/lustre/lustre/ptlrpc/sec_config.c
> deleted file mode 100644
> index 2389f9a8f534..000000000000
> diff --git a/drivers/staging/lustre/lustre/ptlrpc/sec_gc.c b/drivers/staging/lustre/lustre/ptlrpc/sec_gc.c
> deleted file mode 100644
> index 2c8bad7b7877..000000000000
> diff --git a/drivers/staging/lustre/lustre/ptlrpc/sec_lproc.c b/drivers/staging/lustre/lustre/ptlrpc/sec_lproc.c
> deleted file mode 100644
> index 2bb75ebd5d98..000000000000
> diff --git a/drivers/staging/lustre/lustre/ptlrpc/sec_null.c b/drivers/staging/lustre/lustre/ptlrpc/sec_null.c
> deleted file mode 100644
> index ecc387d1b9b4..000000000000
> diff --git a/drivers/staging/lustre/lustre/ptlrpc/sec_plain.c b/drivers/staging/lustre/lustre/ptlrpc/sec_plain.c
> deleted file mode 100644
> index ec3d9af76b17..000000000000
> diff --git a/drivers/staging/lustre/lustre/ptlrpc/service.c b/drivers/staging/lustre/lustre/ptlrpc/service.c
> deleted file mode 100644
> index 3fd8c746f460..000000000000
> diff --git a/drivers/staging/lustre/lustre/ptlrpc/wiretest.c b/drivers/staging/lustre/lustre/ptlrpc/wiretest.c
> deleted file mode 100644
> index f9394c3e1ee2..000000000000
> diff --git a/drivers/staging/lustre/sysfs-fs-lustre b/drivers/staging/lustre/sysfs-fs-lustre
> deleted file mode 100644
> index 8691c6543a9c..000000000000
> diff --git a/scripts/selinux/mdp/mdp.c b/scripts/selinux/mdp/mdp.c
> index ffe8179f5d41..073fe7537f6c 100644
> --- a/scripts/selinux/mdp/mdp.c
> +++ b/scripts/selinux/mdp/mdp.c
> @@ -124,7 +124,6 @@ int main(int argc, char *argv[])
> 	fprintf(fout, "fs_use_xattr reiserfs user_u:base_r:base_t;\n");
> 	fprintf(fout, "fs_use_xattr jffs2 user_u:base_r:base_t;\n");
> 	fprintf(fout, "fs_use_xattr gfs2 user_u:base_r:base_t;\n");
> -	fprintf(fout, "fs_use_xattr lustre user_u:base_r:base_t;\n");
> 
> 	fprintf(fout, "fs_use_task eventpollfs user_u:base_r:base_t;\n");
> 	fprintf(fout, "fs_use_task pipefs user_u:base_r:base_t;\n");
> --
> 2.17.1
> 
> 


Cheers, Andreas





-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 873 bytes
Desc: Message signed with OpenPGP
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180601/701f3c9e/attachment.sig>

From green at linuxhacker.ru  Fri Jun  1 18:41:00 2018
From: green at linuxhacker.ru (Oleg Drokin)
Date: Fri, 1 Jun 2018 14:41:00 -0400
Subject: [lustre-devel] [PATCH] staging: lustre: delete the filesystem
 from the tree.
In-Reply-To: <6A5E4E02-AE72-4068-BF93-38791BF38B4F@dilger.ca>
References: <20180601091133.GA27521@kroah.com>
 <6A5E4E02-AE72-4068-BF93-38791BF38B4F@dilger.ca>
Message-ID: <67482006-1A5C-48F0-A3B1-BBC2956302B1@linuxhacker.ru>


> On Jun 1, 2018, at 2:30 PM, Andreas Dilger <adilger at dilger.ca> wrote:
> 
> On Jun 1, 2018, at 5:11 AM, Greg Kroah-Hartman <gregkh at linuxfoundation.org> wrote:
>> 
>> The Lustre filesystem has been in the kernel tree for over 5 years now.
>> While it has been an endless source of enjoyment for new kernel
>> developers learning how to do basic codingstyle cleanups, as well as an
>> semi-entertaining source of bewilderment from the vfs developers any
>> time they have looked into the codebase to try to figure out how to port
>> their latest api changes to this filesystem, it has not really moved
>> forward into the "this is in shape to get out of staging" despite many
>> half-completed attempts.
> 
> I am happy to submit a patch that moves Lustre out of staging and into
> the mainline.  I'm just about to board a flight, but it could be done
> later today.  Then we can avoid the constant churn of kernel newbies
> submitting patches that break the code.

Yeah, I would also be in favor of this move, obviously.

While staging seems to be a great mechanism to jumpstart compatibility
of smaller projects into great (or not so great) compliance, a big enough
codebase seems to attract a lot of bickering about various minor
details (in addition to some not so minor) that detracts from everybody
in the end, unfortunately.

I know Christoph was unhappy about our o2ib code, if that’s his most
gating issue - it’s modular enough to drop that particular bit until
a suitable rework can happen and merged separately.

Similarly we can look into chopping some other highly controversial bits
if somebody comes up their objections, I guess?


> Lustre has been in use at large sites around the world for 18 years now.
> Over 70% of the largest 100 systems in the world use Lustre.  It runs at
> universities, oil companies, weather bureaus around the world, etc.
> 
> I know Andrew has long been a supporter of getting code into the kernel
> that users are using.  This is code that thousands of large computers
> are using with exabytes of storage, a lot more than orangefs, exofs, and
> random other filesystems that seem to get into the kernel easily.
> 
> It's true the code is not as pretty as it could be, but the same is true
> of code that has been in the kernel for ages.  We've spent years cleaning
> it up in staging, and it has gotten a lot better.  Getting the client into
> the mainline kernel will accelerate the development and cleanup, and
> Christoph will no longer be able to submit patches that break the code
> because it is only in staging.  This will also (finally) allow us to get
> this code in sync with the out-of-tree code and converge on a single tree.
> 
> Cheers, Andreas
> 
>> And getting code out of staging is the main goal of that portion of the
>> kernel tree.  Code should not stagnate and it feels like having this
>> code in staging is only causing the development cycle of the filesystem
>> to take longer than it should.  There is a whole separate out-of-tree
>> copy of this codebase where the developers work on it, and then random
>> changes are thrown over the wall at staging at some later point in time.
>> This dual-tree development model has never worked, and the state of this
>> codebase is proof of that.
>> 
>> So, let's just delete the whole mess.  Now the lustre developers can go
>> off and work in their out-of-tree codebase and not have to worry about
>> providing valid changelog entries and breaking their patches up into
>> logical pieces.  They can take the time they have spend doing those
>> types of housekeeping chores and get the codebase into a much better
>> shape, and it can be submitted for inclusion into the real part of the
>> kernel tree when ready.
>> 
>> Signed-off-by: Greg Kroah-Hartman <gregkh at linuxfoundation.org>
>> ---
>> MAINTAINERS                                   |    9 -
>> drivers/staging/Kconfig                       |    2 -
>> drivers/staging/Makefile                      |    1 -
>> drivers/staging/lustre/Kconfig                |    3 -
>> drivers/staging/lustre/Makefile               |    2 -
>> drivers/staging/lustre/README.txt             |   83 -
>> drivers/staging/lustre/TODO                   |  302 --
>> .../lustre/include/linux/libcfs/libcfs.h      |   76 -
>> .../lustre/include/linux/libcfs/libcfs_cpu.h  |  434 --
>> .../include/linux/libcfs/libcfs_crypto.h      |  208 -
>> .../include/linux/libcfs/libcfs_debug.h       |  207 -
>> .../lustre/include/linux/libcfs/libcfs_fail.h |  194 -
>> .../lustre/include/linux/libcfs/libcfs_hash.h |  869 ----
>> .../include/linux/libcfs/libcfs_private.h     |  200 -
>> .../include/linux/libcfs/libcfs_string.h      |  102 -
>> .../staging/lustre/include/linux/lnet/api.h   |  212 -
>> .../lustre/include/linux/lnet/lib-lnet.h      |  652 ---
>> .../lustre/include/linux/lnet/lib-types.h     |  666 ---
>> .../lustre/include/linux/lnet/socklnd.h       |   87 -
>> .../include/uapi/linux/lnet/libcfs_debug.h    |  149 -
>> .../include/uapi/linux/lnet/libcfs_ioctl.h    |  141 -
>> .../lustre/include/uapi/linux/lnet/lnet-dlc.h |  150 -
>> .../include/uapi/linux/lnet/lnet-types.h      |  669 ---
>> .../lustre/include/uapi/linux/lnet/lnetctl.h  |  123 -
>> .../lustre/include/uapi/linux/lnet/lnetst.h   |  556 ---
>> .../lustre/include/uapi/linux/lnet/nidstr.h   |  119 -
>> .../lustre/include/uapi/linux/lnet/socklnd.h  |   44 -
>> .../include/uapi/linux/lustre/lustre_cfg.h    |  261 -
>> .../include/uapi/linux/lustre/lustre_fid.h    |  293 --
>> .../include/uapi/linux/lustre/lustre_fiemap.h |   72 -
>> .../include/uapi/linux/lustre/lustre_idl.h    | 2690 -----------
>> .../include/uapi/linux/lustre/lustre_ioctl.h  |  229 -
>> .../uapi/linux/lustre/lustre_kernelcomm.h     |   94 -
>> .../include/uapi/linux/lustre/lustre_ostid.h  |  236 -
>> .../include/uapi/linux/lustre/lustre_param.h  |   94 -
>> .../include/uapi/linux/lustre/lustre_user.h   | 1327 ------
>> .../include/uapi/linux/lustre/lustre_ver.h    |   27 -
>> drivers/staging/lustre/lnet/Kconfig           |   46 -
>> drivers/staging/lustre/lnet/Makefile          |    1 -
>> drivers/staging/lustre/lnet/klnds/Makefile    |    1 -
>> .../lustre/lnet/klnds/o2iblnd/Makefile        |    5 -
>> .../lustre/lnet/klnds/o2iblnd/o2iblnd.c       | 2958 ------------
>> .../lustre/lnet/klnds/o2iblnd/o2iblnd.h       | 1048 ----
>> .../lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c    | 3763 ---------------
>> .../lnet/klnds/o2iblnd/o2iblnd_modparams.c    |  296 --
>> .../lustre/lnet/klnds/socklnd/Makefile        |    6 -
>> .../lustre/lnet/klnds/socklnd/socklnd.c       | 2921 ------------
>> .../lustre/lnet/klnds/socklnd/socklnd.h       |  704 ---
>> .../lustre/lnet/klnds/socklnd/socklnd_cb.c    | 2586 ----------
>> .../lustre/lnet/klnds/socklnd/socklnd_lib.c   |  534 ---
>> .../lnet/klnds/socklnd/socklnd_modparams.c    |  184 -
>> .../lustre/lnet/klnds/socklnd/socklnd_proto.c |  810 ----
>> drivers/staging/lustre/lnet/libcfs/Makefile   |   16 -
>> drivers/staging/lustre/lnet/libcfs/debug.c    |  461 --
>> drivers/staging/lustre/lnet/libcfs/fail.c     |  146 -
>> drivers/staging/lustre/lnet/libcfs/hash.c     | 2065 --------
>> .../staging/lustre/lnet/libcfs/libcfs_cpu.c   | 1086 -----
>> .../staging/lustre/lnet/libcfs/libcfs_lock.c  |  155 -
>> .../staging/lustre/lnet/libcfs/libcfs_mem.c   |  171 -
>> .../lustre/lnet/libcfs/libcfs_string.c        |  562 ---
>> .../lustre/lnet/libcfs/linux-crypto-adler.c   |  139 -
>> .../staging/lustre/lnet/libcfs/linux-crypto.c |  447 --
>> .../staging/lustre/lnet/libcfs/linux-crypto.h |   30 -
>> .../staging/lustre/lnet/libcfs/linux-debug.c  |  142 -
>> .../lustre/lnet/libcfs/linux-tracefile.c      |  258 -
>> drivers/staging/lustre/lnet/libcfs/module.c   |  758 ---
>> .../staging/lustre/lnet/libcfs/tracefile.c    | 1198 -----
>> .../staging/lustre/lnet/libcfs/tracefile.h    |  274 --
>> drivers/staging/lustre/lnet/lnet/Makefile     |   10 -
>> drivers/staging/lustre/lnet/lnet/acceptor.c   |  501 --
>> drivers/staging/lustre/lnet/lnet/api-ni.c     | 2307 ---------
>> drivers/staging/lustre/lnet/lnet/config.c     | 1235 -----
>> drivers/staging/lustre/lnet/lnet/lib-eq.c     |  426 --
>> drivers/staging/lustre/lnet/lnet/lib-md.c     |  463 --
>> drivers/staging/lustre/lnet/lnet/lib-me.c     |  274 --
>> drivers/staging/lustre/lnet/lnet/lib-move.c   | 2386 ----------
>> drivers/staging/lustre/lnet/lnet/lib-msg.c    |  625 ---
>> drivers/staging/lustre/lnet/lnet/lib-ptl.c    |  987 ----
>> drivers/staging/lustre/lnet/lnet/lib-socket.c |  585 ---
>> drivers/staging/lustre/lnet/lnet/lo.c         |  105 -
>> drivers/staging/lustre/lnet/lnet/module.c     |  239 -
>> drivers/staging/lustre/lnet/lnet/net_fault.c  | 1023 ----
>> drivers/staging/lustre/lnet/lnet/nidstrings.c | 1261 -----
>> drivers/staging/lustre/lnet/lnet/peer.c       |  456 --
>> drivers/staging/lustre/lnet/lnet/router.c     | 1799 -------
>> .../staging/lustre/lnet/lnet/router_proc.c    |  907 ----
>> drivers/staging/lustre/lnet/selftest/Makefile |    7 -
>> .../staging/lustre/lnet/selftest/brw_test.c   |  526 --
>> drivers/staging/lustre/lnet/selftest/conctl.c |  801 ----
>> drivers/staging/lustre/lnet/selftest/conrpc.c | 1396 ------
>> drivers/staging/lustre/lnet/selftest/conrpc.h |  142 -
>> .../staging/lustre/lnet/selftest/console.c    | 2104 --------
>> .../staging/lustre/lnet/selftest/console.h    |  244 -
>> .../staging/lustre/lnet/selftest/framework.c  | 1786 -------
>> drivers/staging/lustre/lnet/selftest/module.c |  169 -
>> .../staging/lustre/lnet/selftest/ping_test.c  |  228 -
>> drivers/staging/lustre/lnet/selftest/rpc.c    | 1682 -------
>> drivers/staging/lustre/lnet/selftest/rpc.h    |  295 --
>> .../staging/lustre/lnet/selftest/selftest.h   |  622 ---
>> drivers/staging/lustre/lnet/selftest/timer.c  |  244 -
>> drivers/staging/lustre/lnet/selftest/timer.h  |   50 -
>> drivers/staging/lustre/lustre/Kconfig         |   45 -
>> drivers/staging/lustre/lustre/Makefile        |    2 -
>> drivers/staging/lustre/lustre/fid/Makefile    |    5 -
>> .../staging/lustre/lustre/fid/fid_internal.h  |   46 -
>> drivers/staging/lustre/lustre/fid/fid_lib.c   |   87 -
>> .../staging/lustre/lustre/fid/fid_request.c   |  410 --
>> drivers/staging/lustre/lustre/fid/lproc_fid.c |  225 -
>> drivers/staging/lustre/lustre/fld/Makefile    |    5 -
>> drivers/staging/lustre/lustre/fld/fld_cache.c |  516 --
>> .../staging/lustre/lustre/fld/fld_internal.h  |  170 -
>> .../staging/lustre/lustre/fld/fld_request.c   |  446 --
>> drivers/staging/lustre/lustre/fld/lproc_fld.c |  154 -
>> .../staging/lustre/lustre/include/cl_object.h | 2463 ----------
>> .../lustre/lustre/include/interval_tree.h     |  119 -
>> .../staging/lustre/lustre/include/llog_swab.h |   67 -
>> .../lustre/lustre/include/lprocfs_status.h    |  646 ---
>> .../staging/lustre/lustre/include/lu_object.h | 1305 -----
>> .../staging/lustre/lustre/include/lu_ref.h    |  178 -
>> .../lustre/lustre/include/lustre_acl.h        |   51 -
>> .../lustre/lustre/include/lustre_compat.h     |   82 -
>> .../lustre/lustre/include/lustre_debug.h      |   52 -
>> .../lustre/lustre/include/lustre_disk.h       |  152 -
>> .../lustre/lustre/include/lustre_dlm.h        | 1346 ------
>> .../lustre/lustre/include/lustre_dlm_flags.h  |  402 --
>> .../lustre/lustre/include/lustre_errno.h      |  198 -
>> .../lustre/lustre/include/lustre_export.h     |  250 -
>> .../lustre/lustre/include/lustre_fid.h        |  676 ---
>> .../lustre/lustre/include/lustre_fld.h        |  137 -
>> .../staging/lustre/lustre/include/lustre_ha.h |   61 -
>> .../lustre/lustre/include/lustre_handles.h    |   91 -
>> .../lustre/lustre/include/lustre_import.h     |  369 --
>> .../lustre/lustre/include/lustre_intent.h     |   71 -
>> .../lustre/lustre/include/lustre_kernelcomm.h |   56 -
>> .../lustre/lustre/include/lustre_lib.h        |  126 -
>> .../lustre/lustre/include/lustre_linkea.h     |   93 -
>> .../lustre/lustre/include/lustre_lmv.h        |  174 -
>> .../lustre/lustre/include/lustre_log.h        |  382 --
>> .../lustre/lustre/include/lustre_mdc.h        |  229 -
>> .../lustre/lustre/include/lustre_mds.h        |   62 -
>> .../lustre/lustre/include/lustre_net.h        | 2360 ---------
>> .../lustre/lustre/include/lustre_nrs.h        |  718 ---
>> .../lustre/lustre/include/lustre_nrs_fifo.h   |   71 -
>> .../lustre/lustre/include/lustre_obdo.h       |   55 -
>> .../lustre/include/lustre_patchless_compat.h  |   68 -
>> .../lustre/lustre/include/lustre_req_layout.h |  307 --
>> .../lustre/lustre/include/lustre_sec.h        | 1072 -----
>> .../lustre/lustre/include/lustre_swab.h       |  109 -
>> drivers/staging/lustre/lustre/include/obd.h   | 1114 -----
>> .../staging/lustre/lustre/include/obd_cksum.h |  153 -
>> .../staging/lustre/lustre/include/obd_class.h | 1603 -------
>> .../lustre/lustre/include/obd_support.h       |  517 --
>> .../staging/lustre/lustre/include/seq_range.h |  200 -
>> .../lustre/lustre/ldlm/interval_tree.c        |  599 ---
>> drivers/staging/lustre/lustre/ldlm/l_lock.c   |   73 -
>> .../staging/lustre/lustre/ldlm/ldlm_extent.c  |  258 -
>> .../staging/lustre/lustre/ldlm/ldlm_flock.c   |  486 --
>> .../lustre/lustre/ldlm/ldlm_inodebits.c       |   69 -
>> .../lustre/lustre/ldlm/ldlm_internal.h        |  342 --
>> drivers/staging/lustre/lustre/ldlm/ldlm_lib.c |  842 ----
>> .../staging/lustre/lustre/ldlm/ldlm_lock.c    | 2135 ---------
>> .../staging/lustre/lustre/ldlm/ldlm_lockd.c   | 1163 -----
>> .../staging/lustre/lustre/ldlm/ldlm_plain.c   |   68 -
>> .../staging/lustre/lustre/ldlm/ldlm_pool.c    | 1013 ----
>> .../staging/lustre/lustre/ldlm/ldlm_request.c | 2033 --------
>> .../lustre/lustre/ldlm/ldlm_resource.c        | 1318 ------
>> drivers/staging/lustre/lustre/llite/Makefile  |   13 -
>> drivers/staging/lustre/lustre/llite/acl.c     |  108 -
>> drivers/staging/lustre/lustre/llite/dcache.c  |  300 --
>> drivers/staging/lustre/lustre/llite/dir.c     | 1708 -------
>> drivers/staging/lustre/lustre/llite/file.c    | 3580 --------------
>> drivers/staging/lustre/lustre/llite/glimpse.c |  205 -
>> .../staging/lustre/lustre/llite/lcommon_cl.c  |  292 --
>> .../lustre/lustre/llite/lcommon_misc.c        |  186 -
>> .../lustre/lustre/llite/llite_internal.h      | 1344 ------
>> .../staging/lustre/lustre/llite/llite_lib.c   | 2668 -----------
>> .../staging/lustre/lustre/llite/llite_mmap.c  |  480 --
>> .../staging/lustre/lustre/llite/llite_nfs.c   |  375 --
>> .../staging/lustre/lustre/llite/lproc_llite.c | 1659 -------
>> drivers/staging/lustre/lustre/llite/namei.c   | 1207 -----
>> .../staging/lustre/lustre/llite/range_lock.c  |  241 -
>> .../staging/lustre/lustre/llite/range_lock.h  |   83 -
>> drivers/staging/lustre/lustre/llite/rw.c      | 1214 -----
>> drivers/staging/lustre/lustre/llite/rw26.c    |  641 ---
>> .../staging/lustre/lustre/llite/statahead.c   | 1577 ------
>> drivers/staging/lustre/lustre/llite/super25.c |  189 -
>> drivers/staging/lustre/lustre/llite/symlink.c |  159 -
>> drivers/staging/lustre/lustre/llite/vvp_dev.c |  640 ---
>> .../lustre/lustre/llite/vvp_internal.h        |  321 --
>> drivers/staging/lustre/lustre/llite/vvp_io.c  | 1374 ------
>> .../staging/lustre/lustre/llite/vvp_lock.c    |   87 -
>> .../staging/lustre/lustre/llite/vvp_object.c  |  303 --
>> .../staging/lustre/lustre/llite/vvp_page.c    |  523 --
>> drivers/staging/lustre/lustre/llite/xattr.c   |  665 ---
>> .../staging/lustre/lustre/llite/xattr_cache.c |  504 --
>> .../lustre/lustre/llite/xattr_security.c      |   96 -
>> drivers/staging/lustre/lustre/lmv/Makefile    |    5 -
>> drivers/staging/lustre/lustre/lmv/lmv_fld.c   |   82 -
>> .../staging/lustre/lustre/lmv/lmv_intent.c    |  521 --
>> .../staging/lustre/lustre/lmv/lmv_internal.h  |  164 -
>> drivers/staging/lustre/lustre/lmv/lmv_obd.c   | 3131 ------------
>> drivers/staging/lustre/lustre/lmv/lproc_lmv.c |  173 -
>> drivers/staging/lustre/lustre/lov/Makefile    |    9 -
>> .../lustre/lustre/lov/lov_cl_internal.h       |  639 ---
>> drivers/staging/lustre/lustre/lov/lov_dev.c   |  384 --
>> drivers/staging/lustre/lustre/lov/lov_ea.c    |  331 --
>> .../staging/lustre/lustre/lov/lov_internal.h  |  286 --
>> drivers/staging/lustre/lustre/lov/lov_io.c    | 1023 ----
>> drivers/staging/lustre/lustre/lov/lov_lock.c  |  348 --
>> drivers/staging/lustre/lustre/lov/lov_merge.c |  105 -
>> drivers/staging/lustre/lustre/lov/lov_obd.c   | 1444 ------
>> .../staging/lustre/lustre/lov/lov_object.c    | 1625 -------
>> .../staging/lustre/lustre/lov/lov_offset.c    |  269 --
>> drivers/staging/lustre/lustre/lov/lov_pack.c  |  400 --
>> drivers/staging/lustre/lustre/lov/lov_page.c  |  136 -
>> drivers/staging/lustre/lustre/lov/lov_pool.c  |  546 ---
>> .../staging/lustre/lustre/lov/lov_request.c   |  354 --
>> .../staging/lustre/lustre/lov/lovsub_dev.c    |  147 -
>> .../staging/lustre/lustre/lov/lovsub_lock.c   |   81 -
>> .../staging/lustre/lustre/lov/lovsub_object.c |  180 -
>> .../staging/lustre/lustre/lov/lovsub_page.c   |   68 -
>> drivers/staging/lustre/lustre/lov/lproc_lov.c |  299 --
>> drivers/staging/lustre/lustre/mdc/Makefile    |    5 -
>> drivers/staging/lustre/lustre/mdc/lproc_mdc.c |  231 -
>> .../staging/lustre/lustre/mdc/mdc_internal.h  |  144 -
>> drivers/staging/lustre/lustre/mdc/mdc_lib.c   |  498 --
>> drivers/staging/lustre/lustre/mdc/mdc_locks.c | 1239 -----
>> drivers/staging/lustre/lustre/mdc/mdc_reint.c |  421 --
>> .../staging/lustre/lustre/mdc/mdc_request.c   | 2770 -----------
>> drivers/staging/lustre/lustre/mgc/Makefile    |    5 -
>> drivers/staging/lustre/lustre/mgc/lproc_mgc.c |   69 -
>> .../staging/lustre/lustre/mgc/mgc_internal.h  |   57 -
>> .../staging/lustre/lustre/mgc/mgc_request.c   | 1851 --------
>> .../staging/lustre/lustre/obdclass/Makefile   |   12 -
>> .../lustre/lustre/obdclass/cl_internal.h      |   95 -
>> .../staging/lustre/lustre/obdclass/cl_io.c    | 1151 -----
>> .../staging/lustre/lustre/obdclass/cl_lock.c  |  275 --
>> .../lustre/lustre/obdclass/cl_object.c        | 1059 -----
>> .../staging/lustre/lustre/obdclass/cl_page.c  | 1045 ----
>> .../lustre/lustre/obdclass/class_obd.c        |  544 ---
>> .../staging/lustre/lustre/obdclass/debug.c    |   96 -
>> .../staging/lustre/lustre/obdclass/genops.c   | 1480 ------
>> .../lustre/lustre/obdclass/kernelcomm.c       |  240 -
>> .../staging/lustre/lustre/obdclass/linkea.c   |  249 -
>> .../lustre/obdclass/linux/linux-module.c      |  514 --
>> .../lustre/obdclass/linux/linux-sysctl.c      |  162 -
>> drivers/staging/lustre/lustre/obdclass/llog.c |  524 --
>> .../staging/lustre/lustre/obdclass/llog_cat.c |  236 -
>> .../lustre/lustre/obdclass/llog_internal.h    |   79 -
>> .../staging/lustre/lustre/obdclass/llog_obd.c |  225 -
>> .../lustre/lustre/obdclass/llog_swab.c        |  412 --
>> .../lustre/lustre/obdclass/lprocfs_counters.c |  134 -
>> .../lustre/lustre/obdclass/lprocfs_status.c   | 1698 -------
>> .../lustre/lustre/obdclass/lu_object.c        | 2056 --------
>> .../staging/lustre/lustre/obdclass/lu_ref.c   |   45 -
>> .../lustre/lustre/obdclass/lustre_handles.c   |  241 -
>> .../lustre/lustre/obdclass/lustre_peer.c      |  214 -
>> .../lustre/lustre/obdclass/obd_config.c       | 1538 ------
>> .../lustre/lustre/obdclass/obd_mount.c        | 1245 -----
>> drivers/staging/lustre/lustre/obdclass/obdo.c |  181 -
>> .../lustre/lustre/obdclass/statfs_pack.c      |   58 -
>> drivers/staging/lustre/lustre/obdclass/uuid.c |   45 -
>> .../staging/lustre/lustre/obdecho/Makefile    |    5 -
>> .../lustre/lustre/obdecho/echo_client.c       | 1729 -------
>> .../lustre/lustre/obdecho/echo_internal.h     |   42 -
>> drivers/staging/lustre/lustre/osc/Makefile    |    6 -
>> drivers/staging/lustre/lustre/osc/lproc_osc.c |  838 ----
>> drivers/staging/lustre/lustre/osc/osc_cache.c | 3306 -------------
>> .../lustre/lustre/osc/osc_cl_internal.h       |  681 ---
>> drivers/staging/lustre/lustre/osc/osc_dev.c   |  246 -
>> .../staging/lustre/lustre/osc/osc_internal.h  |  237 -
>> drivers/staging/lustre/lustre/osc/osc_io.c    |  918 ----
>> drivers/staging/lustre/lustre/osc/osc_lock.c  | 1230 -----
>> .../staging/lustre/lustre/osc/osc_object.c    |  473 --
>> drivers/staging/lustre/lustre/osc/osc_page.c  | 1094 -----
>> drivers/staging/lustre/lustre/osc/osc_quota.c |  236 -
>> .../staging/lustre/lustre/osc/osc_request.c   | 2907 ------------
>> drivers/staging/lustre/lustre/ptlrpc/Makefile |   23 -
>> drivers/staging/lustre/lustre/ptlrpc/client.c | 3271 -------------
>> .../staging/lustre/lustre/ptlrpc/connection.c |  192 -
>> drivers/staging/lustre/lustre/ptlrpc/errno.c  |  383 --
>> drivers/staging/lustre/lustre/ptlrpc/events.c |  585 ---
>> drivers/staging/lustre/lustre/ptlrpc/import.c | 1677 -------
>> drivers/staging/lustre/lustre/ptlrpc/layout.c | 2232 ---------
>> .../lustre/lustre/ptlrpc/llog_client.c        |  338 --
>> .../staging/lustre/lustre/ptlrpc/llog_net.c   |   67 -
>> .../lustre/lustre/ptlrpc/lproc_ptlrpc.c       | 1316 ------
>> drivers/staging/lustre/lustre/ptlrpc/niobuf.c |  771 ---
>> drivers/staging/lustre/lustre/ptlrpc/nrs.c    | 1613 -------
>> .../staging/lustre/lustre/ptlrpc/nrs_fifo.c   |  270 --
>> .../lustre/lustre/ptlrpc/pack_generic.c       | 2311 ---------
>> drivers/staging/lustre/lustre/ptlrpc/pers.c   |   72 -
>> drivers/staging/lustre/lustre/ptlrpc/pinger.c |  474 --
>> .../lustre/lustre/ptlrpc/ptlrpc_internal.h    |  371 --
>> .../lustre/lustre/ptlrpc/ptlrpc_module.c      |  186 -
>> .../staging/lustre/lustre/ptlrpc/ptlrpcd.c    |  914 ----
>> .../staging/lustre/lustre/ptlrpc/recover.c    |  374 --
>> drivers/staging/lustre/lustre/ptlrpc/sec.c    | 2379 ----------
>> .../staging/lustre/lustre/ptlrpc/sec_bulk.c   |  572 ---
>> .../staging/lustre/lustre/ptlrpc/sec_config.c |  850 ----
>> drivers/staging/lustre/lustre/ptlrpc/sec_gc.c |  190 -
>> .../staging/lustre/lustre/ptlrpc/sec_lproc.c  |  170 -
>> .../staging/lustre/lustre/ptlrpc/sec_null.c   |  459 --
>> .../staging/lustre/lustre/ptlrpc/sec_plain.c  | 1023 ----
>> .../staging/lustre/lustre/ptlrpc/service.c    | 2807 -----------
>> .../staging/lustre/lustre/ptlrpc/wiretest.c   | 4210 -----------------
>> drivers/staging/lustre/sysfs-fs-lustre        |  654 ---
>> scripts/selinux/mdp/mdp.c                     |    1 -
>> 308 files changed, 195272 deletions(-)
>> delete mode 100644 drivers/staging/lustre/Kconfig
>> delete mode 100644 drivers/staging/lustre/Makefile
>> delete mode 100644 drivers/staging/lustre/README.txt
>> delete mode 100644 drivers/staging/lustre/TODO
>> delete mode 100644 drivers/staging/lustre/include/linux/libcfs/libcfs.h
>> delete mode 100644 drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h
>> delete mode 100644 drivers/staging/lustre/include/linux/libcfs/libcfs_crypto.h
>> delete mode 100644 drivers/staging/lustre/include/linux/libcfs/libcfs_debug.h
>> delete mode 100644 drivers/staging/lustre/include/linux/libcfs/libcfs_fail.h
>> delete mode 100644 drivers/staging/lustre/include/linux/libcfs/libcfs_hash.h
>> delete mode 100644 drivers/staging/lustre/include/linux/libcfs/libcfs_private.h
>> delete mode 100644 drivers/staging/lustre/include/linux/libcfs/libcfs_string.h
>> delete mode 100644 drivers/staging/lustre/include/linux/lnet/api.h
>> delete mode 100644 drivers/staging/lustre/include/linux/lnet/lib-lnet.h
>> delete mode 100644 drivers/staging/lustre/include/linux/lnet/lib-types.h
>> delete mode 100644 drivers/staging/lustre/include/linux/lnet/socklnd.h
>> delete mode 100644 drivers/staging/lustre/include/uapi/linux/lnet/libcfs_debug.h
>> delete mode 100644 drivers/staging/lustre/include/uapi/linux/lnet/libcfs_ioctl.h
>> delete mode 100644 drivers/staging/lustre/include/uapi/linux/lnet/lnet-dlc.h
>> delete mode 100644 drivers/staging/lustre/include/uapi/linux/lnet/lnet-types.h
>> delete mode 100644 drivers/staging/lustre/include/uapi/linux/lnet/lnetctl.h
>> delete mode 100644 drivers/staging/lustre/include/uapi/linux/lnet/lnetst.h
>> delete mode 100644 drivers/staging/lustre/include/uapi/linux/lnet/nidstr.h
>> delete mode 100644 drivers/staging/lustre/include/uapi/linux/lnet/socklnd.h
>> delete mode 100644 drivers/staging/lustre/include/uapi/linux/lustre/lustre_cfg.h
>> delete mode 100644 drivers/staging/lustre/include/uapi/linux/lustre/lustre_fid.h
>> delete mode 100644 drivers/staging/lustre/include/uapi/linux/lustre/lustre_fiemap.h
>> delete mode 100644 drivers/staging/lustre/include/uapi/linux/lustre/lustre_idl.h
>> delete mode 100644 drivers/staging/lustre/include/uapi/linux/lustre/lustre_ioctl.h
>> delete mode 100644 drivers/staging/lustre/include/uapi/linux/lustre/lustre_kernelcomm.h
>> delete mode 100644 drivers/staging/lustre/include/uapi/linux/lustre/lustre_ostid.h
>> delete mode 100644 drivers/staging/lustre/include/uapi/linux/lustre/lustre_param.h
>> delete mode 100644 drivers/staging/lustre/include/uapi/linux/lustre/lustre_user.h
>> delete mode 100644 drivers/staging/lustre/include/uapi/linux/lustre/lustre_ver.h
>> delete mode 100644 drivers/staging/lustre/lnet/Kconfig
>> delete mode 100644 drivers/staging/lustre/lnet/Makefile
>> delete mode 100644 drivers/staging/lustre/lnet/klnds/Makefile
>> delete mode 100644 drivers/staging/lustre/lnet/klnds/o2iblnd/Makefile
>> delete mode 100644 drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c
>> delete mode 100644 drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.h
>> delete mode 100644 drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c
>> delete mode 100644 drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd_modparams.c
>> delete mode 100644 drivers/staging/lustre/lnet/klnds/socklnd/Makefile
>> delete mode 100644 drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c
>> delete mode 100644 drivers/staging/lustre/lnet/klnds/socklnd/socklnd.h
>> delete mode 100644 drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c
>> delete mode 100644 drivers/staging/lustre/lnet/klnds/socklnd/socklnd_lib.c
>> delete mode 100644 drivers/staging/lustre/lnet/klnds/socklnd/socklnd_modparams.c
>> delete mode 100644 drivers/staging/lustre/lnet/klnds/socklnd/socklnd_proto.c
>> delete mode 100644 drivers/staging/lustre/lnet/libcfs/Makefile
>> delete mode 100644 drivers/staging/lustre/lnet/libcfs/debug.c
>> delete mode 100644 drivers/staging/lustre/lnet/libcfs/fail.c
>> delete mode 100644 drivers/staging/lustre/lnet/libcfs/hash.c
>> delete mode 100644 drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
>> delete mode 100644 drivers/staging/lustre/lnet/libcfs/libcfs_lock.c
>> delete mode 100644 drivers/staging/lustre/lnet/libcfs/libcfs_mem.c
>> delete mode 100644 drivers/staging/lustre/lnet/libcfs/libcfs_string.c
>> delete mode 100644 drivers/staging/lustre/lnet/libcfs/linux-crypto-adler.c
>> delete mode 100644 drivers/staging/lustre/lnet/libcfs/linux-crypto.c
>> delete mode 100644 drivers/staging/lustre/lnet/libcfs/linux-crypto.h
>> delete mode 100644 drivers/staging/lustre/lnet/libcfs/linux-debug.c
>> delete mode 100644 drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
>> delete mode 100644 drivers/staging/lustre/lnet/libcfs/module.c
>> delete mode 100644 drivers/staging/lustre/lnet/libcfs/tracefile.c
>> delete mode 100644 drivers/staging/lustre/lnet/libcfs/tracefile.h
>> delete mode 100644 drivers/staging/lustre/lnet/lnet/Makefile
>> delete mode 100644 drivers/staging/lustre/lnet/lnet/acceptor.c
>> delete mode 100644 drivers/staging/lustre/lnet/lnet/api-ni.c
>> delete mode 100644 drivers/staging/lustre/lnet/lnet/config.c
>> delete mode 100644 drivers/staging/lustre/lnet/lnet/lib-eq.c
>> delete mode 100644 drivers/staging/lustre/lnet/lnet/lib-md.c
>> delete mode 100644 drivers/staging/lustre/lnet/lnet/lib-me.c
>> delete mode 100644 drivers/staging/lustre/lnet/lnet/lib-move.c
>> delete mode 100644 drivers/staging/lustre/lnet/lnet/lib-msg.c
>> delete mode 100644 drivers/staging/lustre/lnet/lnet/lib-ptl.c
>> delete mode 100644 drivers/staging/lustre/lnet/lnet/lib-socket.c
>> delete mode 100644 drivers/staging/lustre/lnet/lnet/lo.c
>> delete mode 100644 drivers/staging/lustre/lnet/lnet/module.c
>> delete mode 100644 drivers/staging/lustre/lnet/lnet/net_fault.c
>> delete mode 100644 drivers/staging/lustre/lnet/lnet/nidstrings.c
>> delete mode 100644 drivers/staging/lustre/lnet/lnet/peer.c
>> delete mode 100644 drivers/staging/lustre/lnet/lnet/router.c
>> delete mode 100644 drivers/staging/lustre/lnet/lnet/router_proc.c
>> delete mode 100644 drivers/staging/lustre/lnet/selftest/Makefile
>> delete mode 100644 drivers/staging/lustre/lnet/selftest/brw_test.c
>> delete mode 100644 drivers/staging/lustre/lnet/selftest/conctl.c
>> delete mode 100644 drivers/staging/lustre/lnet/selftest/conrpc.c
>> delete mode 100644 drivers/staging/lustre/lnet/selftest/conrpc.h
>> delete mode 100644 drivers/staging/lustre/lnet/selftest/console.c
>> delete mode 100644 drivers/staging/lustre/lnet/selftest/console.h
>> delete mode 100644 drivers/staging/lustre/lnet/selftest/framework.c
>> delete mode 100644 drivers/staging/lustre/lnet/selftest/module.c
>> delete mode 100644 drivers/staging/lustre/lnet/selftest/ping_test.c
>> delete mode 100644 drivers/staging/lustre/lnet/selftest/rpc.c
>> delete mode 100644 drivers/staging/lustre/lnet/selftest/rpc.h
>> delete mode 100644 drivers/staging/lustre/lnet/selftest/selftest.h
>> delete mode 100644 drivers/staging/lustre/lnet/selftest/timer.c
>> delete mode 100644 drivers/staging/lustre/lnet/selftest/timer.h
>> delete mode 100644 drivers/staging/lustre/lustre/Kconfig
>> delete mode 100644 drivers/staging/lustre/lustre/Makefile
>> delete mode 100644 drivers/staging/lustre/lustre/fid/Makefile
>> delete mode 100644 drivers/staging/lustre/lustre/fid/fid_internal.h
>> delete mode 100644 drivers/staging/lustre/lustre/fid/fid_lib.c
>> delete mode 100644 drivers/staging/lustre/lustre/fid/fid_request.c
>> delete mode 100644 drivers/staging/lustre/lustre/fid/lproc_fid.c
>> delete mode 100644 drivers/staging/lustre/lustre/fld/Makefile
>> delete mode 100644 drivers/staging/lustre/lustre/fld/fld_cache.c
>> delete mode 100644 drivers/staging/lustre/lustre/fld/fld_internal.h
>> delete mode 100644 drivers/staging/lustre/lustre/fld/fld_request.c
>> delete mode 100644 drivers/staging/lustre/lustre/fld/lproc_fld.c
>> delete mode 100644 drivers/staging/lustre/lustre/include/cl_object.h
>> delete mode 100644 drivers/staging/lustre/lustre/include/interval_tree.h
>> delete mode 100644 drivers/staging/lustre/lustre/include/llog_swab.h
>> delete mode 100644 drivers/staging/lustre/lustre/include/lprocfs_status.h
>> delete mode 100644 drivers/staging/lustre/lustre/include/lu_object.h
>> delete mode 100644 drivers/staging/lustre/lustre/include/lu_ref.h
>> delete mode 100644 drivers/staging/lustre/lustre/include/lustre_acl.h
>> delete mode 100644 drivers/staging/lustre/lustre/include/lustre_compat.h
>> delete mode 100644 drivers/staging/lustre/lustre/include/lustre_debug.h
>> delete mode 100644 drivers/staging/lustre/lustre/include/lustre_disk.h
>> delete mode 100644 drivers/staging/lustre/lustre/include/lustre_dlm.h
>> delete mode 100644 drivers/staging/lustre/lustre/include/lustre_dlm_flags.h
>> delete mode 100644 drivers/staging/lustre/lustre/include/lustre_errno.h
>> delete mode 100644 drivers/staging/lustre/lustre/include/lustre_export.h
>> delete mode 100644 drivers/staging/lustre/lustre/include/lustre_fid.h
>> delete mode 100644 drivers/staging/lustre/lustre/include/lustre_fld.h
>> delete mode 100644 drivers/staging/lustre/lustre/include/lustre_ha.h
>> delete mode 100644 drivers/staging/lustre/lustre/include/lustre_handles.h
>> delete mode 100644 drivers/staging/lustre/lustre/include/lustre_import.h
>> delete mode 100644 drivers/staging/lustre/lustre/include/lustre_intent.h
>> delete mode 100644 drivers/staging/lustre/lustre/include/lustre_kernelcomm.h
>> delete mode 100644 drivers/staging/lustre/lustre/include/lustre_lib.h
>> delete mode 100644 drivers/staging/lustre/lustre/include/lustre_linkea.h
>> delete mode 100644 drivers/staging/lustre/lustre/include/lustre_lmv.h
>> delete mode 100644 drivers/staging/lustre/lustre/include/lustre_log.h
>> delete mode 100644 drivers/staging/lustre/lustre/include/lustre_mdc.h
>> delete mode 100644 drivers/staging/lustre/lustre/include/lustre_mds.h
>> delete mode 100644 drivers/staging/lustre/lustre/include/lustre_net.h
>> delete mode 100644 drivers/staging/lustre/lustre/include/lustre_nrs.h
>> delete mode 100644 drivers/staging/lustre/lustre/include/lustre_nrs_fifo.h
>> delete mode 100644 drivers/staging/lustre/lustre/include/lustre_obdo.h
>> delete mode 100644 drivers/staging/lustre/lustre/include/lustre_patchless_compat.h
>> delete mode 100644 drivers/staging/lustre/lustre/include/lustre_req_layout.h
>> delete mode 100644 drivers/staging/lustre/lustre/include/lustre_sec.h
>> delete mode 100644 drivers/staging/lustre/lustre/include/lustre_swab.h
>> delete mode 100644 drivers/staging/lustre/lustre/include/obd.h
>> delete mode 100644 drivers/staging/lustre/lustre/include/obd_cksum.h
>> delete mode 100644 drivers/staging/lustre/lustre/include/obd_class.h
>> delete mode 100644 drivers/staging/lustre/lustre/include/obd_support.h
>> delete mode 100644 drivers/staging/lustre/lustre/include/seq_range.h
>> delete mode 100644 drivers/staging/lustre/lustre/ldlm/interval_tree.c
>> delete mode 100644 drivers/staging/lustre/lustre/ldlm/l_lock.c
>> delete mode 100644 drivers/staging/lustre/lustre/ldlm/ldlm_extent.c
>> delete mode 100644 drivers/staging/lustre/lustre/ldlm/ldlm_flock.c
>> delete mode 100644 drivers/staging/lustre/lustre/ldlm/ldlm_inodebits.c
>> delete mode 100644 drivers/staging/lustre/lustre/ldlm/ldlm_internal.h
>> delete mode 100644 drivers/staging/lustre/lustre/ldlm/ldlm_lib.c
>> delete mode 100644 drivers/staging/lustre/lustre/ldlm/ldlm_lock.c
>> delete mode 100644 drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c
>> delete mode 100644 drivers/staging/lustre/lustre/ldlm/ldlm_plain.c
>> delete mode 100644 drivers/staging/lustre/lustre/ldlm/ldlm_pool.c
>> delete mode 100644 drivers/staging/lustre/lustre/ldlm/ldlm_request.c
>> delete mode 100644 drivers/staging/lustre/lustre/ldlm/ldlm_resource.c
>> delete mode 100644 drivers/staging/lustre/lustre/llite/Makefile
>> delete mode 100644 drivers/staging/lustre/lustre/llite/acl.c
>> delete mode 100644 drivers/staging/lustre/lustre/llite/dcache.c
>> delete mode 100644 drivers/staging/lustre/lustre/llite/dir.c
>> delete mode 100644 drivers/staging/lustre/lustre/llite/file.c
>> delete mode 100644 drivers/staging/lustre/lustre/llite/glimpse.c
>> delete mode 100644 drivers/staging/lustre/lustre/llite/lcommon_cl.c
>> delete mode 100644 drivers/staging/lustre/lustre/llite/lcommon_misc.c
>> delete mode 100644 drivers/staging/lustre/lustre/llite/llite_internal.h
>> delete mode 100644 drivers/staging/lustre/lustre/llite/llite_lib.c
>> delete mode 100644 drivers/staging/lustre/lustre/llite/llite_mmap.c
>> delete mode 100644 drivers/staging/lustre/lustre/llite/llite_nfs.c
>> delete mode 100644 drivers/staging/lustre/lustre/llite/lproc_llite.c
>> delete mode 100644 drivers/staging/lustre/lustre/llite/namei.c
>> delete mode 100644 drivers/staging/lustre/lustre/llite/range_lock.c
>> delete mode 100644 drivers/staging/lustre/lustre/llite/range_lock.h
>> delete mode 100644 drivers/staging/lustre/lustre/llite/rw.c
>> delete mode 100644 drivers/staging/lustre/lustre/llite/rw26.c
>> delete mode 100644 drivers/staging/lustre/lustre/llite/statahead.c
>> delete mode 100644 drivers/staging/lustre/lustre/llite/super25.c
>> delete mode 100644 drivers/staging/lustre/lustre/llite/symlink.c
>> delete mode 100644 drivers/staging/lustre/lustre/llite/vvp_dev.c
>> delete mode 100644 drivers/staging/lustre/lustre/llite/vvp_internal.h
>> delete mode 100644 drivers/staging/lustre/lustre/llite/vvp_io.c
>> delete mode 100644 drivers/staging/lustre/lustre/llite/vvp_lock.c
>> delete mode 100644 drivers/staging/lustre/lustre/llite/vvp_object.c
>> delete mode 100644 drivers/staging/lustre/lustre/llite/vvp_page.c
>> delete mode 100644 drivers/staging/lustre/lustre/llite/xattr.c
>> delete mode 100644 drivers/staging/lustre/lustre/llite/xattr_cache.c
>> delete mode 100644 drivers/staging/lustre/lustre/llite/xattr_security.c
>> delete mode 100644 drivers/staging/lustre/lustre/lmv/Makefile
>> delete mode 100644 drivers/staging/lustre/lustre/lmv/lmv_fld.c
>> delete mode 100644 drivers/staging/lustre/lustre/lmv/lmv_intent.c
>> delete mode 100644 drivers/staging/lustre/lustre/lmv/lmv_internal.h
>> delete mode 100644 drivers/staging/lustre/lustre/lmv/lmv_obd.c
>> delete mode 100644 drivers/staging/lustre/lustre/lmv/lproc_lmv.c
>> delete mode 100644 drivers/staging/lustre/lustre/lov/Makefile
>> delete mode 100644 drivers/staging/lustre/lustre/lov/lov_cl_internal.h
>> delete mode 100644 drivers/staging/lustre/lustre/lov/lov_dev.c
>> delete mode 100644 drivers/staging/lustre/lustre/lov/lov_ea.c
>> delete mode 100644 drivers/staging/lustre/lustre/lov/lov_internal.h
>> delete mode 100644 drivers/staging/lustre/lustre/lov/lov_io.c
>> delete mode 100644 drivers/staging/lustre/lustre/lov/lov_lock.c
>> delete mode 100644 drivers/staging/lustre/lustre/lov/lov_merge.c
>> delete mode 100644 drivers/staging/lustre/lustre/lov/lov_obd.c
>> delete mode 100644 drivers/staging/lustre/lustre/lov/lov_object.c
>> delete mode 100644 drivers/staging/lustre/lustre/lov/lov_offset.c
>> delete mode 100644 drivers/staging/lustre/lustre/lov/lov_pack.c
>> delete mode 100644 drivers/staging/lustre/lustre/lov/lov_page.c
>> delete mode 100644 drivers/staging/lustre/lustre/lov/lov_pool.c
>> delete mode 100644 drivers/staging/lustre/lustre/lov/lov_request.c
>> delete mode 100644 drivers/staging/lustre/lustre/lov/lovsub_dev.c
>> delete mode 100644 drivers/staging/lustre/lustre/lov/lovsub_lock.c
>> delete mode 100644 drivers/staging/lustre/lustre/lov/lovsub_object.c
>> delete mode 100644 drivers/staging/lustre/lustre/lov/lovsub_page.c
>> delete mode 100644 drivers/staging/lustre/lustre/lov/lproc_lov.c
>> delete mode 100644 drivers/staging/lustre/lustre/mdc/Makefile
>> delete mode 100644 drivers/staging/lustre/lustre/mdc/lproc_mdc.c
>> delete mode 100644 drivers/staging/lustre/lustre/mdc/mdc_internal.h
>> delete mode 100644 drivers/staging/lustre/lustre/mdc/mdc_lib.c
>> delete mode 100644 drivers/staging/lustre/lustre/mdc/mdc_locks.c
>> delete mode 100644 drivers/staging/lustre/lustre/mdc/mdc_reint.c
>> delete mode 100644 drivers/staging/lustre/lustre/mdc/mdc_request.c
>> delete mode 100644 drivers/staging/lustre/lustre/mgc/Makefile
>> delete mode 100644 drivers/staging/lustre/lustre/mgc/lproc_mgc.c
>> delete mode 100644 drivers/staging/lustre/lustre/mgc/mgc_internal.h
>> delete mode 100644 drivers/staging/lustre/lustre/mgc/mgc_request.c
>> delete mode 100644 drivers/staging/lustre/lustre/obdclass/Makefile
>> delete mode 100644 drivers/staging/lustre/lustre/obdclass/cl_internal.h
>> delete mode 100644 drivers/staging/lustre/lustre/obdclass/cl_io.c
>> delete mode 100644 drivers/staging/lustre/lustre/obdclass/cl_lock.c
>> delete mode 100644 drivers/staging/lustre/lustre/obdclass/cl_object.c
>> delete mode 100644 drivers/staging/lustre/lustre/obdclass/cl_page.c
>> delete mode 100644 drivers/staging/lustre/lustre/obdclass/class_obd.c
>> delete mode 100644 drivers/staging/lustre/lustre/obdclass/debug.c
>> delete mode 100644 drivers/staging/lustre/lustre/obdclass/genops.c
>> delete mode 100644 drivers/staging/lustre/lustre/obdclass/kernelcomm.c
>> delete mode 100644 drivers/staging/lustre/lustre/obdclass/linkea.c
>> delete mode 100644 drivers/staging/lustre/lustre/obdclass/linux/linux-module.c
>> delete mode 100644 drivers/staging/lustre/lustre/obdclass/linux/linux-sysctl.c
>> delete mode 100644 drivers/staging/lustre/lustre/obdclass/llog.c
>> delete mode 100644 drivers/staging/lustre/lustre/obdclass/llog_cat.c
>> delete mode 100644 drivers/staging/lustre/lustre/obdclass/llog_internal.h
>> delete mode 100644 drivers/staging/lustre/lustre/obdclass/llog_obd.c
>> delete mode 100644 drivers/staging/lustre/lustre/obdclass/llog_swab.c
>> delete mode 100644 drivers/staging/lustre/lustre/obdclass/lprocfs_counters.c
>> delete mode 100644 drivers/staging/lustre/lustre/obdclass/lprocfs_status.c
>> delete mode 100644 drivers/staging/lustre/lustre/obdclass/lu_object.c
>> delete mode 100644 drivers/staging/lustre/lustre/obdclass/lu_ref.c
>> delete mode 100644 drivers/staging/lustre/lustre/obdclass/lustre_handles.c
>> delete mode 100644 drivers/staging/lustre/lustre/obdclass/lustre_peer.c
>> delete mode 100644 drivers/staging/lustre/lustre/obdclass/obd_config.c
>> delete mode 100644 drivers/staging/lustre/lustre/obdclass/obd_mount.c
>> delete mode 100644 drivers/staging/lustre/lustre/obdclass/obdo.c
>> delete mode 100644 drivers/staging/lustre/lustre/obdclass/statfs_pack.c
>> delete mode 100644 drivers/staging/lustre/lustre/obdclass/uuid.c
>> delete mode 100644 drivers/staging/lustre/lustre/obdecho/Makefile
>> delete mode 100644 drivers/staging/lustre/lustre/obdecho/echo_client.c
>> delete mode 100644 drivers/staging/lustre/lustre/obdecho/echo_internal.h
>> delete mode 100644 drivers/staging/lustre/lustre/osc/Makefile
>> delete mode 100644 drivers/staging/lustre/lustre/osc/lproc_osc.c
>> delete mode 100644 drivers/staging/lustre/lustre/osc/osc_cache.c
>> delete mode 100644 drivers/staging/lustre/lustre/osc/osc_cl_internal.h
>> delete mode 100644 drivers/staging/lustre/lustre/osc/osc_dev.c
>> delete mode 100644 drivers/staging/lustre/lustre/osc/osc_internal.h
>> delete mode 100644 drivers/staging/lustre/lustre/osc/osc_io.c
>> delete mode 100644 drivers/staging/lustre/lustre/osc/osc_lock.c
>> delete mode 100644 drivers/staging/lustre/lustre/osc/osc_object.c
>> delete mode 100644 drivers/staging/lustre/lustre/osc/osc_page.c
>> delete mode 100644 drivers/staging/lustre/lustre/osc/osc_quota.c
>> delete mode 100644 drivers/staging/lustre/lustre/osc/osc_request.c
>> delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/Makefile
>> delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/client.c
>> delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/connection.c
>> delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/errno.c
>> delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/events.c
>> delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/import.c
>> delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/layout.c
>> delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/llog_client.c
>> delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/llog_net.c
>> delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/lproc_ptlrpc.c
>> delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/niobuf.c
>> delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/nrs.c
>> delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/nrs_fifo.c
>> delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/pack_generic.c
>> delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/pers.c
>> delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/pinger.c
>> delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/ptlrpc_internal.h
>> delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/ptlrpc_module.c
>> delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c
>> delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/recover.c
>> delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/sec.c
>> delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/sec_bulk.c
>> delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/sec_config.c
>> delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/sec_gc.c
>> delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/sec_lproc.c
>> delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/sec_null.c
>> delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/sec_plain.c
>> delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/service.c
>> delete mode 100644 drivers/staging/lustre/lustre/ptlrpc/wiretest.c
>> delete mode 100644 drivers/staging/lustre/sysfs-fs-lustre
>> 
>> diff --git a/MAINTAINERS b/MAINTAINERS
>> index 4b65225d443a..db158767de20 100644
>> --- a/MAINTAINERS
>> +++ b/MAINTAINERS
>> @@ -13329,15 +13329,6 @@ S:	Odd Fixes
>> F:	Documentation/devicetree/bindings/staging/iio/
>> F:	drivers/staging/iio/
>> 
>> -STAGING - LUSTRE PARALLEL FILESYSTEM
>> -M:	Oleg Drokin <oleg.drokin at intel.com>
>> -M:	Andreas Dilger <andreas.dilger at intel.com>
>> -M:	James Simmons <jsimmons at infradead.org>
>> -L:	lustre-devel at lists.lustre.org (moderated for non-subscribers)
>> -W:	http://wiki.lustre.org/
>> -S:	Maintained
>> -F:	drivers/staging/lustre
>> -
>> STAGING - NVIDIA COMPLIANT EMBEDDED CONTROLLER INTERFACE (nvec)
>> M:	Marc Dietrich <marvin24 at gmx.de>
>> L:	ac100 at lists.launchpad.net (moderated for non-subscribers)
>> diff --git a/drivers/staging/Kconfig b/drivers/staging/Kconfig
>> index d5926f0d3f6c..1c357ef669ae 100644
>> --- a/drivers/staging/Kconfig
>> +++ b/drivers/staging/Kconfig
>> @@ -84,8 +84,6 @@ source "drivers/staging/netlogic/Kconfig"
>> 
>> source "drivers/staging/mt29f_spinand/Kconfig"
>> 
>> -source "drivers/staging/lustre/Kconfig"
>> -
>> source "drivers/staging/dgnc/Kconfig"
>> 
>> source "drivers/staging/gs_fpgaboot/Kconfig"
>> diff --git a/drivers/staging/Makefile b/drivers/staging/Makefile
>> index 919753c3d3f6..2edb9860931e 100644
>> --- a/drivers/staging/Makefile
>> +++ b/drivers/staging/Makefile
>> @@ -32,7 +32,6 @@ obj-$(CONFIG_STAGING_BOARD)	+= board/
>> obj-$(CONFIG_LTE_GDM724X)	+= gdm724x/
>> obj-$(CONFIG_FIREWIRE_SERIAL)	+= fwserial/
>> obj-$(CONFIG_GOLDFISH)		+= goldfish/
>> -obj-$(CONFIG_LNET)		+= lustre/
>> obj-$(CONFIG_DGNC)			+= dgnc/
>> obj-$(CONFIG_MTD_SPINAND_MT29F)	+= mt29f_spinand/
>> obj-$(CONFIG_GS_FPGABOOT)	+= gs_fpgaboot/
>> diff --git a/drivers/staging/lustre/Kconfig b/drivers/staging/lustre/Kconfig
>> deleted file mode 100644
>> index b7d81096eee9..000000000000
>> diff --git a/drivers/staging/lustre/Makefile b/drivers/staging/lustre/Makefile
>> deleted file mode 100644
>> index 95ffe337a80a..000000000000
>> diff --git a/drivers/staging/lustre/README.txt b/drivers/staging/lustre/README.txt
>> deleted file mode 100644
>> index 783959240490..000000000000
>> diff --git a/drivers/staging/lustre/TODO b/drivers/staging/lustre/TODO
>> deleted file mode 100644
>> index 5332cdb19bfa..000000000000
>> diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs.h b/drivers/staging/lustre/include/linux/libcfs/libcfs.h
>> deleted file mode 100644
>> index edc7ed0dcb94..000000000000
>> diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h
>> deleted file mode 100644
>> index 61641c41c492..000000000000
>> diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_crypto.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_crypto.h
>> deleted file mode 100644
>> index 176fae7319e3..000000000000
>> diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_debug.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_debug.h
>> deleted file mode 100644
>> index 17534a76362a..000000000000
>> diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_fail.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_fail.h
>> deleted file mode 100644
>> index 8074e390b4d1..000000000000
>> diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_hash.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_hash.h
>> deleted file mode 100644
>> index be315958a4b3..000000000000
>> diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_private.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_private.h
>> deleted file mode 100644
>> index 491d5971d199..000000000000
>> diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_string.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_string.h
>> deleted file mode 100644
>> index cd7c3ccb2dc0..000000000000
>> diff --git a/drivers/staging/lustre/include/linux/lnet/api.h b/drivers/staging/lustre/include/linux/lnet/api.h
>> deleted file mode 100644
>> index dae2e4f0056c..000000000000
>> diff --git a/drivers/staging/lustre/include/linux/lnet/lib-lnet.h b/drivers/staging/lustre/include/linux/lnet/lib-lnet.h
>> deleted file mode 100644
>> index 973c17a1c4a1..000000000000
>> diff --git a/drivers/staging/lustre/include/linux/lnet/lib-types.h b/drivers/staging/lustre/include/linux/lnet/lib-types.h
>> deleted file mode 100644
>> index cfe8ee424e94..000000000000
>> diff --git a/drivers/staging/lustre/include/linux/lnet/socklnd.h b/drivers/staging/lustre/include/linux/lnet/socklnd.h
>> deleted file mode 100644
>> index 6bd1bca190a3..000000000000
>> diff --git a/drivers/staging/lustre/include/uapi/linux/lnet/libcfs_debug.h b/drivers/staging/lustre/include/uapi/linux/lnet/libcfs_debug.h
>> deleted file mode 100644
>> index c4d9472b374f..000000000000
>> diff --git a/drivers/staging/lustre/include/uapi/linux/lnet/libcfs_ioctl.h b/drivers/staging/lustre/include/uapi/linux/lnet/libcfs_ioctl.h
>> deleted file mode 100644
>> index cce6b58e3682..000000000000
>> diff --git a/drivers/staging/lustre/include/uapi/linux/lnet/lnet-dlc.h b/drivers/staging/lustre/include/uapi/linux/lnet/lnet-dlc.h
>> deleted file mode 100644
>> index c1619f411d81..000000000000
>> diff --git a/drivers/staging/lustre/include/uapi/linux/lnet/lnet-types.h b/drivers/staging/lustre/include/uapi/linux/lnet/lnet-types.h
>> deleted file mode 100644
>> index 1be9b7aa7326..000000000000
>> diff --git a/drivers/staging/lustre/include/uapi/linux/lnet/lnetctl.h b/drivers/staging/lustre/include/uapi/linux/lnet/lnetctl.h
>> deleted file mode 100644
>> index cccb32dd28f2..000000000000
>> diff --git a/drivers/staging/lustre/include/uapi/linux/lnet/lnetst.h b/drivers/staging/lustre/include/uapi/linux/lnet/lnetst.h
>> deleted file mode 100644
>> index a4f9ff01d458..000000000000
>> diff --git a/drivers/staging/lustre/include/uapi/linux/lnet/nidstr.h b/drivers/staging/lustre/include/uapi/linux/lnet/nidstr.h
>> deleted file mode 100644
>> index 882074ed6021..000000000000
>> diff --git a/drivers/staging/lustre/include/uapi/linux/lnet/socklnd.h b/drivers/staging/lustre/include/uapi/linux/lnet/socklnd.h
>> deleted file mode 100644
>> index 6453e053fa99..000000000000
>> diff --git a/drivers/staging/lustre/include/uapi/linux/lustre/lustre_cfg.h b/drivers/staging/lustre/include/uapi/linux/lustre/lustre_cfg.h
>> deleted file mode 100644
>> index 11b51d93f64c..000000000000
>> diff --git a/drivers/staging/lustre/include/uapi/linux/lustre/lustre_fid.h b/drivers/staging/lustre/include/uapi/linux/lustre/lustre_fid.h
>> deleted file mode 100644
>> index 2e7a8d103777..000000000000
>> diff --git a/drivers/staging/lustre/include/uapi/linux/lustre/lustre_fiemap.h b/drivers/staging/lustre/include/uapi/linux/lustre/lustre_fiemap.h
>> deleted file mode 100644
>> index d375a476f5ea..000000000000
>> diff --git a/drivers/staging/lustre/include/uapi/linux/lustre/lustre_idl.h b/drivers/staging/lustre/include/uapi/linux/lustre/lustre_idl.h
>> deleted file mode 100644
>> index 6c7e3992d646..000000000000
>> diff --git a/drivers/staging/lustre/include/uapi/linux/lustre/lustre_ioctl.h b/drivers/staging/lustre/include/uapi/linux/lustre/lustre_ioctl.h
>> deleted file mode 100644
>> index 6e4e109fb874..000000000000
>> diff --git a/drivers/staging/lustre/include/uapi/linux/lustre/lustre_kernelcomm.h b/drivers/staging/lustre/include/uapi/linux/lustre/lustre_kernelcomm.h
>> deleted file mode 100644
>> index 94dadbe8e069..000000000000
>> diff --git a/drivers/staging/lustre/include/uapi/linux/lustre/lustre_ostid.h b/drivers/staging/lustre/include/uapi/linux/lustre/lustre_ostid.h
>> deleted file mode 100644
>> index 3343b602219b..000000000000
>> diff --git a/drivers/staging/lustre/include/uapi/linux/lustre/lustre_param.h b/drivers/staging/lustre/include/uapi/linux/lustre/lustre_param.h
>> deleted file mode 100644
>> index 1eab2ceca338..000000000000
>> diff --git a/drivers/staging/lustre/include/uapi/linux/lustre/lustre_user.h b/drivers/staging/lustre/include/uapi/linux/lustre/lustre_user.h
>> deleted file mode 100644
>> index 69387f36d1f1..000000000000
>> diff --git a/drivers/staging/lustre/include/uapi/linux/lustre/lustre_ver.h b/drivers/staging/lustre/include/uapi/linux/lustre/lustre_ver.h
>> deleted file mode 100644
>> index 19c9135e2273..000000000000
>> diff --git a/drivers/staging/lustre/lnet/Kconfig b/drivers/staging/lustre/lnet/Kconfig
>> deleted file mode 100644
>> index ad049e6f24e4..000000000000
>> diff --git a/drivers/staging/lustre/lnet/Makefile b/drivers/staging/lustre/lnet/Makefile
>> deleted file mode 100644
>> index 0a380fe88ce8..000000000000
>> diff --git a/drivers/staging/lustre/lnet/klnds/Makefile b/drivers/staging/lustre/lnet/klnds/Makefile
>> deleted file mode 100644
>> index c23e4f67f837..000000000000
>> diff --git a/drivers/staging/lustre/lnet/klnds/o2iblnd/Makefile b/drivers/staging/lustre/lnet/klnds/o2iblnd/Makefile
>> deleted file mode 100644
>> index 4affe1d79948..000000000000
>> diff --git a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c
>> deleted file mode 100644
>> index f0b4eb42bc1d..000000000000
>> diff --git a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.h b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.h
>> deleted file mode 100644
>> index 217503f125bc..000000000000
>> diff --git a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c
>> deleted file mode 100644
>> index 65b7a62943ad..000000000000
>> diff --git a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd_modparams.c b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd_modparams.c
>> deleted file mode 100644
>> index 39d07926d603..000000000000
>> diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/Makefile b/drivers/staging/lustre/lnet/klnds/socklnd/Makefile
>> deleted file mode 100644
>> index a7da1abfc804..000000000000
>> diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c
>> deleted file mode 100644
>> index f01b34ac1a53..000000000000
>> diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.h b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.h
>> deleted file mode 100644
>> index 4e5c89a692a3..000000000000
>> diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c
>> deleted file mode 100644
>> index 01b31a6bb588..000000000000
>> diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_lib.c b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_lib.c
>> deleted file mode 100644
>> index 93a02cd6b6b5..000000000000
>> diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_modparams.c b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_modparams.c
>> deleted file mode 100644
>> index 5663a4ca94d4..000000000000
>> diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_proto.c b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_proto.c
>> deleted file mode 100644
>> index 05982dac781c..000000000000
>> diff --git a/drivers/staging/lustre/lnet/libcfs/Makefile b/drivers/staging/lustre/lnet/libcfs/Makefile
>> deleted file mode 100644
>> index 6a1b232da495..000000000000
>> diff --git a/drivers/staging/lustre/lnet/libcfs/debug.c b/drivers/staging/lustre/lnet/libcfs/debug.c
>> deleted file mode 100644
>> index 06f694f6a28f..000000000000
>> diff --git a/drivers/staging/lustre/lnet/libcfs/fail.c b/drivers/staging/lustre/lnet/libcfs/fail.c
>> deleted file mode 100644
>> index bd86b3b5bc34..000000000000
>> diff --git a/drivers/staging/lustre/lnet/libcfs/hash.c b/drivers/staging/lustre/lnet/libcfs/hash.c
>> deleted file mode 100644
>> index 48be66f0d654..000000000000
>> diff --git a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
>> deleted file mode 100644
>> index 3d1cf457b286..000000000000
>> diff --git a/drivers/staging/lustre/lnet/libcfs/libcfs_lock.c b/drivers/staging/lustre/lnet/libcfs/libcfs_lock.c
>> deleted file mode 100644
>> index 223505c37545..000000000000
>> diff --git a/drivers/staging/lustre/lnet/libcfs/libcfs_mem.c b/drivers/staging/lustre/lnet/libcfs/libcfs_mem.c
>> deleted file mode 100644
>> index 2d533be9bb30..000000000000
>> diff --git a/drivers/staging/lustre/lnet/libcfs/libcfs_string.c b/drivers/staging/lustre/lnet/libcfs/libcfs_string.c
>> deleted file mode 100644
>> index e1fb1263e3ae..000000000000
>> diff --git a/drivers/staging/lustre/lnet/libcfs/linux-crypto-adler.c b/drivers/staging/lustre/lnet/libcfs/linux-crypto-adler.c
>> deleted file mode 100644
>> index db81ed527452..000000000000
>> diff --git a/drivers/staging/lustre/lnet/libcfs/linux-crypto.c b/drivers/staging/lustre/lnet/libcfs/linux-crypto.c
>> deleted file mode 100644
>> index 21ff9bf6da47..000000000000
>> diff --git a/drivers/staging/lustre/lnet/libcfs/linux-crypto.h b/drivers/staging/lustre/lnet/libcfs/linux-crypto.h
>> deleted file mode 100644
>> index 5616e9ea1450..000000000000
>> diff --git a/drivers/staging/lustre/lnet/libcfs/linux-debug.c b/drivers/staging/lustre/lnet/libcfs/linux-debug.c
>> deleted file mode 100644
>> index 15ab849374c2..000000000000
>> diff --git a/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c b/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
>> deleted file mode 100644
>> index 347138409eba..000000000000
>> diff --git a/drivers/staging/lustre/lnet/libcfs/module.c b/drivers/staging/lustre/lnet/libcfs/module.c
>> deleted file mode 100644
>> index 5dc7de9e6478..000000000000
>> diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.c b/drivers/staging/lustre/lnet/libcfs/tracefile.c
>> deleted file mode 100644
>> index 7ca562e156f0..000000000000
>> diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.h b/drivers/staging/lustre/lnet/libcfs/tracefile.h
>> deleted file mode 100644
>> index 0608240d897f..000000000000
>> diff --git a/drivers/staging/lustre/lnet/lnet/Makefile b/drivers/staging/lustre/lnet/lnet/Makefile
>> deleted file mode 100644
>> index 0a9d70924fe0..000000000000
>> diff --git a/drivers/staging/lustre/lnet/lnet/acceptor.c b/drivers/staging/lustre/lnet/lnet/acceptor.c
>> deleted file mode 100644
>> index 5648f17eddc0..000000000000
>> diff --git a/drivers/staging/lustre/lnet/lnet/api-ni.c b/drivers/staging/lustre/lnet/lnet/api-ni.c
>> deleted file mode 100644
>> index f9ed6977056c..000000000000
>> diff --git a/drivers/staging/lustre/lnet/lnet/config.c b/drivers/staging/lustre/lnet/lnet/config.c
>> deleted file mode 100644
>> index 55ecc1998b7e..000000000000
>> diff --git a/drivers/staging/lustre/lnet/lnet/lib-eq.c b/drivers/staging/lustre/lnet/lnet/lib-eq.c
>> deleted file mode 100644
>> index c78e70373ab4..000000000000
>> diff --git a/drivers/staging/lustre/lnet/lnet/lib-md.c b/drivers/staging/lustre/lnet/lnet/lib-md.c
>> deleted file mode 100644
>> index 8a22514aaf71..000000000000
>> diff --git a/drivers/staging/lustre/lnet/lnet/lib-me.c b/drivers/staging/lustre/lnet/lnet/lib-me.c
>> deleted file mode 100644
>> index 672e37bdd045..000000000000
>> diff --git a/drivers/staging/lustre/lnet/lnet/lib-move.c b/drivers/staging/lustre/lnet/lnet/lib-move.c
>> deleted file mode 100644
>> index f8eaf8ff8d8d..000000000000
>> diff --git a/drivers/staging/lustre/lnet/lnet/lib-msg.c b/drivers/staging/lustre/lnet/lnet/lib-msg.c
>> deleted file mode 100644
>> index 0091273c04b9..000000000000
>> diff --git a/drivers/staging/lustre/lnet/lnet/lib-ptl.c b/drivers/staging/lustre/lnet/lnet/lib-ptl.c
>> deleted file mode 100644
>> index fc47379c5938..000000000000
>> diff --git a/drivers/staging/lustre/lnet/lnet/lib-socket.c b/drivers/staging/lustre/lnet/lnet/lib-socket.c
>> deleted file mode 100644
>> index 9b61260155f2..000000000000
>> diff --git a/drivers/staging/lustre/lnet/lnet/lo.c b/drivers/staging/lustre/lnet/lnet/lo.c
>> deleted file mode 100644
>> index 7456b989e451..000000000000
>> diff --git a/drivers/staging/lustre/lnet/lnet/module.c b/drivers/staging/lustre/lnet/lnet/module.c
>> deleted file mode 100644
>> index 9d06664f0c17..000000000000
>> diff --git a/drivers/staging/lustre/lnet/lnet/net_fault.c b/drivers/staging/lustre/lnet/lnet/net_fault.c
>> deleted file mode 100644
>> index 0066394b0bb0..000000000000
>> diff --git a/drivers/staging/lustre/lnet/lnet/nidstrings.c b/drivers/staging/lustre/lnet/lnet/nidstrings.c
>> deleted file mode 100644
>> index 0f6c3fa16c65..000000000000
>> diff --git a/drivers/staging/lustre/lnet/lnet/peer.c b/drivers/staging/lustre/lnet/lnet/peer.c
>> deleted file mode 100644
>> index 58294149f7b2..000000000000
>> diff --git a/drivers/staging/lustre/lnet/lnet/router.c b/drivers/staging/lustre/lnet/lnet/router.c
>> deleted file mode 100644
>> index 6267d5e4bbd6..000000000000
>> diff --git a/drivers/staging/lustre/lnet/lnet/router_proc.c b/drivers/staging/lustre/lnet/lnet/router_proc.c
>> deleted file mode 100644
>> index ae4b7f5953a0..000000000000
>> diff --git a/drivers/staging/lustre/lnet/selftest/Makefile b/drivers/staging/lustre/lnet/selftest/Makefile
>> deleted file mode 100644
>> index 3ccc8966b566..000000000000
>> diff --git a/drivers/staging/lustre/lnet/selftest/brw_test.c b/drivers/staging/lustre/lnet/selftest/brw_test.c
>> deleted file mode 100644
>> index f1ee219bc8f3..000000000000
>> diff --git a/drivers/staging/lustre/lnet/selftest/conctl.c b/drivers/staging/lustre/lnet/selftest/conctl.c
>> deleted file mode 100644
>> index 906d82d90c0c..000000000000
>> diff --git a/drivers/staging/lustre/lnet/selftest/conrpc.c b/drivers/staging/lustre/lnet/selftest/conrpc.c
>> deleted file mode 100644
>> index 0dabade3d091..000000000000
>> diff --git a/drivers/staging/lustre/lnet/selftest/conrpc.h b/drivers/staging/lustre/lnet/selftest/conrpc.h
>> deleted file mode 100644
>> index ce2f92d04838..000000000000
>> diff --git a/drivers/staging/lustre/lnet/selftest/console.c b/drivers/staging/lustre/lnet/selftest/console.c
>> deleted file mode 100644
>> index 3c1c1b5997e0..000000000000
>> diff --git a/drivers/staging/lustre/lnet/selftest/console.h b/drivers/staging/lustre/lnet/selftest/console.h
>> deleted file mode 100644
>> index 2826205e36a1..000000000000
>> diff --git a/drivers/staging/lustre/lnet/selftest/framework.c b/drivers/staging/lustre/lnet/selftest/framework.c
>> deleted file mode 100644
>> index 741af10560ad..000000000000
>> diff --git a/drivers/staging/lustre/lnet/selftest/module.c b/drivers/staging/lustre/lnet/selftest/module.c
>> deleted file mode 100644
>> index 9ba65320f748..000000000000
>> diff --git a/drivers/staging/lustre/lnet/selftest/ping_test.c b/drivers/staging/lustre/lnet/selftest/ping_test.c
>> deleted file mode 100644
>> index f54bd630dbf8..000000000000
>> diff --git a/drivers/staging/lustre/lnet/selftest/rpc.c b/drivers/staging/lustre/lnet/selftest/rpc.c
>> deleted file mode 100644
>> index 9613b0a77007..000000000000
>> diff --git a/drivers/staging/lustre/lnet/selftest/rpc.h b/drivers/staging/lustre/lnet/selftest/rpc.h
>> deleted file mode 100644
>> index 465b5b534423..000000000000
>> diff --git a/drivers/staging/lustre/lnet/selftest/selftest.h b/drivers/staging/lustre/lnet/selftest/selftest.h
>> deleted file mode 100644
>> index 8737fa96b192..000000000000
>> diff --git a/drivers/staging/lustre/lnet/selftest/timer.c b/drivers/staging/lustre/lnet/selftest/timer.c
>> deleted file mode 100644
>> index 582f252b3e12..000000000000
>> diff --git a/drivers/staging/lustre/lnet/selftest/timer.h b/drivers/staging/lustre/lnet/selftest/timer.h
>> deleted file mode 100644
>> index 7f0ef9bd0cda..000000000000
>> diff --git a/drivers/staging/lustre/lustre/Kconfig b/drivers/staging/lustre/lustre/Kconfig
>> deleted file mode 100644
>> index ccb78a945995..000000000000
>> diff --git a/drivers/staging/lustre/lustre/Makefile b/drivers/staging/lustre/lustre/Makefile
>> deleted file mode 100644
>> index 331e4fcdd5a2..000000000000
>> diff --git a/drivers/staging/lustre/lustre/fid/Makefile b/drivers/staging/lustre/lustre/fid/Makefile
>> deleted file mode 100644
>> index 77b65b92667d..000000000000
>> diff --git a/drivers/staging/lustre/lustre/fid/fid_internal.h b/drivers/staging/lustre/lustre/fid/fid_internal.h
>> deleted file mode 100644
>> index 14569e969a31..000000000000
>> diff --git a/drivers/staging/lustre/lustre/fid/fid_lib.c b/drivers/staging/lustre/lustre/fid/fid_lib.c
>> deleted file mode 100644
>> index ac52b378c155..000000000000
>> diff --git a/drivers/staging/lustre/lustre/fid/fid_request.c b/drivers/staging/lustre/lustre/fid/fid_request.c
>> deleted file mode 100644
>> index a34fd90ca5e5..000000000000
>> diff --git a/drivers/staging/lustre/lustre/fid/lproc_fid.c b/drivers/staging/lustre/lustre/fid/lproc_fid.c
>> deleted file mode 100644
>> index 0aabf473c9bd..000000000000
>> diff --git a/drivers/staging/lustre/lustre/fld/Makefile b/drivers/staging/lustre/lustre/fld/Makefile
>> deleted file mode 100644
>> index 426deba8b815..000000000000
>> diff --git a/drivers/staging/lustre/lustre/fld/fld_cache.c b/drivers/staging/lustre/lustre/fld/fld_cache.c
>> deleted file mode 100644
>> index a7415c9a1c28..000000000000
>> diff --git a/drivers/staging/lustre/lustre/fld/fld_internal.h b/drivers/staging/lustre/lustre/fld/fld_internal.h
>> deleted file mode 100644
>> index e1d6aaa5c2b4..000000000000
>> diff --git a/drivers/staging/lustre/lustre/fld/fld_request.c b/drivers/staging/lustre/lustre/fld/fld_request.c
>> deleted file mode 100644
>> index 97f7ea632346..000000000000
>> diff --git a/drivers/staging/lustre/lustre/fld/lproc_fld.c b/drivers/staging/lustre/lustre/fld/lproc_fld.c
>> deleted file mode 100644
>> index 0bcfb26ef8aa..000000000000
>> diff --git a/drivers/staging/lustre/lustre/include/cl_object.h b/drivers/staging/lustre/lustre/include/cl_object.h
>> deleted file mode 100644
>> index 6f7b991be809..000000000000
>> diff --git a/drivers/staging/lustre/lustre/include/interval_tree.h b/drivers/staging/lustre/lustre/include/interval_tree.h
>> deleted file mode 100644
>> index 7d119c1a0469..000000000000
>> diff --git a/drivers/staging/lustre/lustre/include/llog_swab.h b/drivers/staging/lustre/lustre/include/llog_swab.h
>> deleted file mode 100644
>> index 0433b79efdcb..000000000000
>> diff --git a/drivers/staging/lustre/lustre/include/lprocfs_status.h b/drivers/staging/lustre/lustre/include/lprocfs_status.h
>> deleted file mode 100644
>> index 495e6f5f676b..000000000000
>> diff --git a/drivers/staging/lustre/lustre/include/lu_object.h b/drivers/staging/lustre/lustre/include/lu_object.h
>> deleted file mode 100644
>> index f29bbca5af65..000000000000
>> diff --git a/drivers/staging/lustre/lustre/include/lu_ref.h b/drivers/staging/lustre/lustre/include/lu_ref.h
>> deleted file mode 100644
>> index ad0c24d29ffa..000000000000
>> diff --git a/drivers/staging/lustre/lustre/include/lustre_acl.h b/drivers/staging/lustre/lustre/include/lustre_acl.h
>> deleted file mode 100644
>> index e7575a172b5f..000000000000
>> diff --git a/drivers/staging/lustre/lustre/include/lustre_compat.h b/drivers/staging/lustre/lustre/include/lustre_compat.h
>> deleted file mode 100644
>> index 3c6db0d632dc..000000000000
>> diff --git a/drivers/staging/lustre/lustre/include/lustre_debug.h b/drivers/staging/lustre/lustre/include/lustre_debug.h
>> deleted file mode 100644
>> index 721a81f923e3..000000000000
>> diff --git a/drivers/staging/lustre/lustre/include/lustre_disk.h b/drivers/staging/lustre/lustre/include/lustre_disk.h
>> deleted file mode 100644
>> index 886e817644d6..000000000000
>> diff --git a/drivers/staging/lustre/lustre/include/lustre_dlm.h b/drivers/staging/lustre/lustre/include/lustre_dlm.h
>> deleted file mode 100644
>> index 2c55241258cc..000000000000
>> diff --git a/drivers/staging/lustre/lustre/include/lustre_dlm_flags.h b/drivers/staging/lustre/lustre/include/lustre_dlm_flags.h
>> deleted file mode 100644
>> index 53db031c4c8c..000000000000
>> diff --git a/drivers/staging/lustre/lustre/include/lustre_errno.h b/drivers/staging/lustre/lustre/include/lustre_errno.h
>> deleted file mode 100644
>> index 59fbb9f47ff1..000000000000
>> diff --git a/drivers/staging/lustre/lustre/include/lustre_export.h b/drivers/staging/lustre/lustre/include/lustre_export.h
>> deleted file mode 100644
>> index 79ad5aae86b9..000000000000
>> diff --git a/drivers/staging/lustre/lustre/include/lustre_fid.h b/drivers/staging/lustre/lustre/include/lustre_fid.h
>> deleted file mode 100644
>> index 094ad282de2c..000000000000
>> diff --git a/drivers/staging/lustre/lustre/include/lustre_fld.h b/drivers/staging/lustre/lustre/include/lustre_fld.h
>> deleted file mode 100644
>> index f42122a4dfaa..000000000000
>> diff --git a/drivers/staging/lustre/lustre/include/lustre_ha.h b/drivers/staging/lustre/lustre/include/lustre_ha.h
>> deleted file mode 100644
>> index cbd68985ada9..000000000000
>> diff --git a/drivers/staging/lustre/lustre/include/lustre_handles.h b/drivers/staging/lustre/lustre/include/lustre_handles.h
>> deleted file mode 100644
>> index 3556ce8d94e8..000000000000
>> diff --git a/drivers/staging/lustre/lustre/include/lustre_import.h b/drivers/staging/lustre/lustre/include/lustre_import.h
>> deleted file mode 100644
>> index ac3805ead620..000000000000
>> diff --git a/drivers/staging/lustre/lustre/include/lustre_intent.h b/drivers/staging/lustre/lustre/include/lustre_intent.h
>> deleted file mode 100644
>> index 51e5c0e03872..000000000000
>> diff --git a/drivers/staging/lustre/lustre/include/lustre_kernelcomm.h b/drivers/staging/lustre/lustre/include/lustre_kernelcomm.h
>> deleted file mode 100644
>> index 2b3fa8430185..000000000000
>> diff --git a/drivers/staging/lustre/lustre/include/lustre_lib.h b/drivers/staging/lustre/lustre/include/lustre_lib.h
>> deleted file mode 100644
>> index 87748e9902a7..000000000000
>> diff --git a/drivers/staging/lustre/lustre/include/lustre_linkea.h b/drivers/staging/lustre/lustre/include/lustre_linkea.h
>> deleted file mode 100644
>> index 03db1511bfd3..000000000000
>> diff --git a/drivers/staging/lustre/lustre/include/lustre_lmv.h b/drivers/staging/lustre/lustre/include/lustre_lmv.h
>> deleted file mode 100644
>> index 080ec1f8e19f..000000000000
>> diff --git a/drivers/staging/lustre/lustre/include/lustre_log.h b/drivers/staging/lustre/lustre/include/lustre_log.h
>> deleted file mode 100644
>> index 07f4e600386b..000000000000
>> diff --git a/drivers/staging/lustre/lustre/include/lustre_mdc.h b/drivers/staging/lustre/lustre/include/lustre_mdc.h
>> deleted file mode 100644
>> index a9c9992a2502..000000000000
>> diff --git a/drivers/staging/lustre/lustre/include/lustre_mds.h b/drivers/staging/lustre/lustre/include/lustre_mds.h
>> deleted file mode 100644
>> index f665556556ec..000000000000
>> diff --git a/drivers/staging/lustre/lustre/include/lustre_net.h b/drivers/staging/lustre/lustre/include/lustre_net.h
>> deleted file mode 100644
>> index 35b43a77eb18..000000000000
>> diff --git a/drivers/staging/lustre/lustre/include/lustre_nrs.h b/drivers/staging/lustre/lustre/include/lustre_nrs.h
>> deleted file mode 100644
>> index ffa7317da35b..000000000000
>> diff --git a/drivers/staging/lustre/lustre/include/lustre_nrs_fifo.h b/drivers/staging/lustre/lustre/include/lustre_nrs_fifo.h
>> deleted file mode 100644
>> index b70d97d4acbb..000000000000
>> diff --git a/drivers/staging/lustre/lustre/include/lustre_obdo.h b/drivers/staging/lustre/lustre/include/lustre_obdo.h
>> deleted file mode 100644
>> index d67dcbb84f18..000000000000
>> diff --git a/drivers/staging/lustre/lustre/include/lustre_patchless_compat.h b/drivers/staging/lustre/lustre/include/lustre_patchless_compat.h
>> deleted file mode 100644
>> index 298476ea7557..000000000000
>> diff --git a/drivers/staging/lustre/lustre/include/lustre_req_layout.h b/drivers/staging/lustre/lustre/include/lustre_req_layout.h
>> deleted file mode 100644
>> index 213d0a01adcf..000000000000
>> diff --git a/drivers/staging/lustre/lustre/include/lustre_sec.h b/drivers/staging/lustre/lustre/include/lustre_sec.h
>> deleted file mode 100644
>> index d35bcbc98831..000000000000
>> diff --git a/drivers/staging/lustre/lustre/include/lustre_swab.h b/drivers/staging/lustre/lustre/include/lustre_swab.h
>> deleted file mode 100644
>> index 9d786bbe7f3f..000000000000
>> diff --git a/drivers/staging/lustre/lustre/include/obd.h b/drivers/staging/lustre/lustre/include/obd.h
>> deleted file mode 100644
>> index b1907bbffb19..000000000000
>> diff --git a/drivers/staging/lustre/lustre/include/obd_cksum.h b/drivers/staging/lustre/lustre/include/obd_cksum.h
>> deleted file mode 100644
>> index e5f7bb20415d..000000000000
>> diff --git a/drivers/staging/lustre/lustre/include/obd_class.h b/drivers/staging/lustre/lustre/include/obd_class.h
>> deleted file mode 100644
>> index fc9c7720fee0..000000000000
>> diff --git a/drivers/staging/lustre/lustre/include/obd_support.h b/drivers/staging/lustre/lustre/include/obd_support.h
>> deleted file mode 100644
>> index 9e41633823f7..000000000000
>> diff --git a/drivers/staging/lustre/lustre/include/seq_range.h b/drivers/staging/lustre/lustre/include/seq_range.h
>> deleted file mode 100644
>> index 9450da728160..000000000000
>> diff --git a/drivers/staging/lustre/lustre/ldlm/interval_tree.c b/drivers/staging/lustre/lustre/ldlm/interval_tree.c
>> deleted file mode 100644
>> index 8df7a4463c21..000000000000
>> diff --git a/drivers/staging/lustre/lustre/ldlm/l_lock.c b/drivers/staging/lustre/lustre/ldlm/l_lock.c
>> deleted file mode 100644
>> index 296259aa51e6..000000000000
>> diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_extent.c b/drivers/staging/lustre/lustre/ldlm/ldlm_extent.c
>> deleted file mode 100644
>> index 4da23ade2bb3..000000000000
>> diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_flock.c b/drivers/staging/lustre/lustre/ldlm/ldlm_flock.c
>> deleted file mode 100644
>> index 94f3b1e49896..000000000000
>> diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_inodebits.c b/drivers/staging/lustre/lustre/ldlm/ldlm_inodebits.c
>> deleted file mode 100644
>> index 2926208cdfa1..000000000000
>> diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_internal.h b/drivers/staging/lustre/lustre/ldlm/ldlm_internal.h
>> deleted file mode 100644
>> index bc33ca100620..000000000000
>> diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_lib.c b/drivers/staging/lustre/lustre/ldlm/ldlm_lib.c
>> deleted file mode 100644
>> index 0aa4f234a4f4..000000000000
>> diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c b/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c
>> deleted file mode 100644
>> index a644d133063b..000000000000
>> diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c b/drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c
>> deleted file mode 100644
>> index 5963e90d0938..000000000000
>> diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_plain.c b/drivers/staging/lustre/lustre/ldlm/ldlm_plain.c
>> deleted file mode 100644
>> index 33b5a3f96fcb..000000000000
>> diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c b/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c
>> deleted file mode 100644
>> index 36d14ee4e5b1..000000000000
>> diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_request.c b/drivers/staging/lustre/lustre/ldlm/ldlm_request.c
>> deleted file mode 100644
>> index cdc52eed6d85..000000000000
>> diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_resource.c b/drivers/staging/lustre/lustre/ldlm/ldlm_resource.c
>> deleted file mode 100644
>> index c93b019b8e37..000000000000
>> diff --git a/drivers/staging/lustre/lustre/llite/Makefile b/drivers/staging/lustre/lustre/llite/Makefile
>> deleted file mode 100644
>> index 5200924182ae..000000000000
>> diff --git a/drivers/staging/lustre/lustre/llite/acl.c b/drivers/staging/lustre/lustre/llite/acl.c
>> deleted file mode 100644
>> index 2ee9ff931236..000000000000
>> diff --git a/drivers/staging/lustre/lustre/llite/dcache.c b/drivers/staging/lustre/lustre/llite/dcache.c
>> deleted file mode 100644
>> index 11b82c639bfe..000000000000
>> diff --git a/drivers/staging/lustre/lustre/llite/dir.c b/drivers/staging/lustre/lustre/llite/dir.c
>> deleted file mode 100644
>> index 688dddf3ca47..000000000000
>> diff --git a/drivers/staging/lustre/lustre/llite/file.c b/drivers/staging/lustre/lustre/llite/file.c
>> deleted file mode 100644
>> index 02295931883b..000000000000
>> diff --git a/drivers/staging/lustre/lustre/llite/glimpse.c b/drivers/staging/lustre/lustre/llite/glimpse.c
>> deleted file mode 100644
>> index ce0d51767da3..000000000000
>> diff --git a/drivers/staging/lustre/lustre/llite/lcommon_cl.c b/drivers/staging/lustre/lustre/llite/lcommon_cl.c
>> deleted file mode 100644
>> index d7ea39ce0cb2..000000000000
>> diff --git a/drivers/staging/lustre/lustre/llite/lcommon_misc.c b/drivers/staging/lustre/lustre/llite/lcommon_misc.c
>> deleted file mode 100644
>> index a246b955306e..000000000000
>> diff --git a/drivers/staging/lustre/lustre/llite/llite_internal.h b/drivers/staging/lustre/lustre/llite/llite_internal.h
>> deleted file mode 100644
>> index c08a6e14b6d7..000000000000
>> diff --git a/drivers/staging/lustre/lustre/llite/llite_lib.c b/drivers/staging/lustre/lustre/llite/llite_lib.c
>> deleted file mode 100644
>> index 36066c839160..000000000000
>> diff --git a/drivers/staging/lustre/lustre/llite/llite_mmap.c b/drivers/staging/lustre/lustre/llite/llite_mmap.c
>> deleted file mode 100644
>> index d7fb5533f707..000000000000
>> diff --git a/drivers/staging/lustre/lustre/llite/llite_nfs.c b/drivers/staging/lustre/lustre/llite/llite_nfs.c
>> deleted file mode 100644
>> index 14172688d55f..000000000000
>> diff --git a/drivers/staging/lustre/lustre/llite/lproc_llite.c b/drivers/staging/lustre/lustre/llite/lproc_llite.c
>> deleted file mode 100644
>> index 49bf1b7ee311..000000000000
>> diff --git a/drivers/staging/lustre/lustre/llite/namei.c b/drivers/staging/lustre/lustre/llite/namei.c
>> deleted file mode 100644
>> index d5f6d20afe8c..000000000000
>> diff --git a/drivers/staging/lustre/lustre/llite/range_lock.c b/drivers/staging/lustre/lustre/llite/range_lock.c
>> deleted file mode 100644
>> index 008a8874118d..000000000000
>> diff --git a/drivers/staging/lustre/lustre/llite/range_lock.h b/drivers/staging/lustre/lustre/llite/range_lock.h
>> deleted file mode 100644
>> index 9ebac09160f2..000000000000
>> diff --git a/drivers/staging/lustre/lustre/llite/rw.c b/drivers/staging/lustre/lustre/llite/rw.c
>> deleted file mode 100644
>> index 3e008ce7275d..000000000000
>> diff --git a/drivers/staging/lustre/lustre/llite/rw26.c b/drivers/staging/lustre/lustre/llite/rw26.c
>> deleted file mode 100644
>> index 722e5ea1af5f..000000000000
>> diff --git a/drivers/staging/lustre/lustre/llite/statahead.c b/drivers/staging/lustre/lustre/llite/statahead.c
>> deleted file mode 100644
>> index d864f5f36d85..000000000000
>> diff --git a/drivers/staging/lustre/lustre/llite/super25.c b/drivers/staging/lustre/lustre/llite/super25.c
>> deleted file mode 100644
>> index d335f29556c2..000000000000
>> diff --git a/drivers/staging/lustre/lustre/llite/symlink.c b/drivers/staging/lustre/lustre/llite/symlink.c
>> deleted file mode 100644
>> index 0690fdbf49f5..000000000000
>> diff --git a/drivers/staging/lustre/lustre/llite/vvp_dev.c b/drivers/staging/lustre/lustre/llite/vvp_dev.c
>> deleted file mode 100644
>> index 31dc3c0ade01..000000000000
>> diff --git a/drivers/staging/lustre/lustre/llite/vvp_internal.h b/drivers/staging/lustre/lustre/llite/vvp_internal.h
>> deleted file mode 100644
>> index 7d3abb43584a..000000000000
>> diff --git a/drivers/staging/lustre/lustre/llite/vvp_io.c b/drivers/staging/lustre/lustre/llite/vvp_io.c
>> deleted file mode 100644
>> index e7a4778e02e4..000000000000
>> diff --git a/drivers/staging/lustre/lustre/llite/vvp_lock.c b/drivers/staging/lustre/lustre/llite/vvp_lock.c
>> deleted file mode 100644
>> index 4b6c7143bd2c..000000000000
>> diff --git a/drivers/staging/lustre/lustre/llite/vvp_object.c b/drivers/staging/lustre/lustre/llite/vvp_object.c
>> deleted file mode 100644
>> index b2cb51c8f7f4..000000000000
>> diff --git a/drivers/staging/lustre/lustre/llite/vvp_page.c b/drivers/staging/lustre/lustre/llite/vvp_page.c
>> deleted file mode 100644
>> index 6eb0565ddc22..000000000000
>> diff --git a/drivers/staging/lustre/lustre/llite/xattr.c b/drivers/staging/lustre/lustre/llite/xattr.c
>> deleted file mode 100644
>> index 7fa0a419c094..000000000000
>> diff --git a/drivers/staging/lustre/lustre/llite/xattr_cache.c b/drivers/staging/lustre/lustre/llite/xattr_cache.c
>> deleted file mode 100644
>> index 5da69ba088c4..000000000000
>> diff --git a/drivers/staging/lustre/lustre/llite/xattr_security.c b/drivers/staging/lustre/lustre/llite/xattr_security.c
>> deleted file mode 100644
>> index 93ec07531ac7..000000000000
>> diff --git a/drivers/staging/lustre/lustre/lmv/Makefile b/drivers/staging/lustre/lustre/lmv/Makefile
>> deleted file mode 100644
>> index 91c99114aa13..000000000000
>> diff --git a/drivers/staging/lustre/lustre/lmv/lmv_fld.c b/drivers/staging/lustre/lustre/lmv/lmv_fld.c
>> deleted file mode 100644
>> index 00dc858c10c9..000000000000
>> diff --git a/drivers/staging/lustre/lustre/lmv/lmv_intent.c b/drivers/staging/lustre/lustre/lmv/lmv_intent.c
>> deleted file mode 100644
>> index 1e850fdbc623..000000000000
>> diff --git a/drivers/staging/lustre/lustre/lmv/lmv_internal.h b/drivers/staging/lustre/lustre/lmv/lmv_internal.h
>> deleted file mode 100644
>> index 68a99170c424..000000000000
>> diff --git a/drivers/staging/lustre/lustre/lmv/lmv_obd.c b/drivers/staging/lustre/lustre/lmv/lmv_obd.c
>> deleted file mode 100644
>> index 65f94e6ecaad..000000000000
>> diff --git a/drivers/staging/lustre/lustre/lmv/lproc_lmv.c b/drivers/staging/lustre/lustre/lmv/lproc_lmv.c
>> deleted file mode 100644
>> index 30727b7acccc..000000000000
>> diff --git a/drivers/staging/lustre/lustre/lov/Makefile b/drivers/staging/lustre/lustre/lov/Makefile
>> deleted file mode 100644
>> index 1ebf0193f61a..000000000000
>> diff --git a/drivers/staging/lustre/lustre/lov/lov_cl_internal.h b/drivers/staging/lustre/lustre/lov/lov_cl_internal.h
>> deleted file mode 100644
>> index e4f762137a4a..000000000000
>> diff --git a/drivers/staging/lustre/lustre/lov/lov_dev.c b/drivers/staging/lustre/lustre/lov/lov_dev.c
>> deleted file mode 100644
>> index c7db23472346..000000000000
>> diff --git a/drivers/staging/lustre/lustre/lov/lov_ea.c b/drivers/staging/lustre/lustre/lov/lov_ea.c
>> deleted file mode 100644
>> index c80320ab0858..000000000000
>> diff --git a/drivers/staging/lustre/lustre/lov/lov_internal.h b/drivers/staging/lustre/lustre/lov/lov_internal.h
>> deleted file mode 100644
>> index 47042f27ca90..000000000000
>> diff --git a/drivers/staging/lustre/lustre/lov/lov_io.c b/drivers/staging/lustre/lustre/lov/lov_io.c
>> deleted file mode 100644
>> index b823f8a21856..000000000000
>> diff --git a/drivers/staging/lustre/lustre/lov/lov_lock.c b/drivers/staging/lustre/lustre/lov/lov_lock.c
>> deleted file mode 100644
>> index b0292100bf26..000000000000
>> diff --git a/drivers/staging/lustre/lustre/lov/lov_merge.c b/drivers/staging/lustre/lustre/lov/lov_merge.c
>> deleted file mode 100644
>> index 006717cf7a41..000000000000
>> diff --git a/drivers/staging/lustre/lustre/lov/lov_obd.c b/drivers/staging/lustre/lustre/lov/lov_obd.c
>> deleted file mode 100644
>> index 344ff4b20168..000000000000
>> diff --git a/drivers/staging/lustre/lustre/lov/lov_object.c b/drivers/staging/lustre/lustre/lov/lov_object.c
>> deleted file mode 100644
>> index adc90f310fd7..000000000000
>> diff --git a/drivers/staging/lustre/lustre/lov/lov_offset.c b/drivers/staging/lustre/lustre/lov/lov_offset.c
>> deleted file mode 100644
>> index a5f00f6ec347..000000000000
>> diff --git a/drivers/staging/lustre/lustre/lov/lov_pack.c b/drivers/staging/lustre/lustre/lov/lov_pack.c
>> deleted file mode 100644
>> index b1060d02a164..000000000000
>> diff --git a/drivers/staging/lustre/lustre/lov/lov_page.c b/drivers/staging/lustre/lustre/lov/lov_page.c
>> deleted file mode 100644
>> index cfae1294d77a..000000000000
>> diff --git a/drivers/staging/lustre/lustre/lov/lov_pool.c b/drivers/staging/lustre/lustre/lov/lov_pool.c
>> deleted file mode 100644
>> index b2a88ba72eb2..000000000000
>> diff --git a/drivers/staging/lustre/lustre/lov/lov_request.c b/drivers/staging/lustre/lustre/lov/lov_request.c
>> deleted file mode 100644
>> index cb8567f20ea7..000000000000
>> diff --git a/drivers/staging/lustre/lustre/lov/lovsub_dev.c b/drivers/staging/lustre/lustre/lov/lovsub_dev.c
>> deleted file mode 100644
>> index 7e89a2e485fc..000000000000
>> diff --git a/drivers/staging/lustre/lustre/lov/lovsub_lock.c b/drivers/staging/lustre/lustre/lov/lovsub_lock.c
>> deleted file mode 100644
>> index ea492be2eef3..000000000000
>> diff --git a/drivers/staging/lustre/lustre/lov/lovsub_object.c b/drivers/staging/lustre/lustre/lov/lovsub_object.c
>> deleted file mode 100644
>> index 13d452086b61..000000000000
>> diff --git a/drivers/staging/lustre/lustre/lov/lovsub_page.c b/drivers/staging/lustre/lustre/lov/lovsub_page.c
>> deleted file mode 100644
>> index 915520bcdd60..000000000000
>> diff --git a/drivers/staging/lustre/lustre/lov/lproc_lov.c b/drivers/staging/lustre/lustre/lov/lproc_lov.c
>> deleted file mode 100644
>> index 721440feef72..000000000000
>> diff --git a/drivers/staging/lustre/lustre/mdc/Makefile b/drivers/staging/lustre/lustre/mdc/Makefile
>> deleted file mode 100644
>> index c7bc3351ccb0..000000000000
>> diff --git a/drivers/staging/lustre/lustre/mdc/lproc_mdc.c b/drivers/staging/lustre/lustre/mdc/lproc_mdc.c
>> deleted file mode 100644
>> index 6cce32491eb5..000000000000
>> diff --git a/drivers/staging/lustre/lustre/mdc/mdc_internal.h b/drivers/staging/lustre/lustre/mdc/mdc_internal.h
>> deleted file mode 100644
>> index 28924e927b50..000000000000
>> diff --git a/drivers/staging/lustre/lustre/mdc/mdc_lib.c b/drivers/staging/lustre/lustre/mdc/mdc_lib.c
>> deleted file mode 100644
>> index d582968987ff..000000000000
>> diff --git a/drivers/staging/lustre/lustre/mdc/mdc_locks.c b/drivers/staging/lustre/lustre/mdc/mdc_locks.c
>> deleted file mode 100644
>> index a8aa0fa5e87a..000000000000
>> diff --git a/drivers/staging/lustre/lustre/mdc/mdc_reint.c b/drivers/staging/lustre/lustre/mdc/mdc_reint.c
>> deleted file mode 100644
>> index e77c00df0693..000000000000
>> diff --git a/drivers/staging/lustre/lustre/mdc/mdc_request.c b/drivers/staging/lustre/lustre/mdc/mdc_request.c
>> deleted file mode 100644
>> index cff31cb0a9ac..000000000000
>> diff --git a/drivers/staging/lustre/lustre/mgc/Makefile b/drivers/staging/lustre/lustre/mgc/Makefile
>> deleted file mode 100644
>> index 8abf108dbcf7..000000000000
>> diff --git a/drivers/staging/lustre/lustre/mgc/lproc_mgc.c b/drivers/staging/lustre/lustre/mgc/lproc_mgc.c
>> deleted file mode 100644
>> index 636770624e8f..000000000000
>> diff --git a/drivers/staging/lustre/lustre/mgc/mgc_internal.h b/drivers/staging/lustre/lustre/mgc/mgc_internal.h
>> deleted file mode 100644
>> index 9541892b67c7..000000000000
>> diff --git a/drivers/staging/lustre/lustre/mgc/mgc_request.c b/drivers/staging/lustre/lustre/mgc/mgc_request.c
>> deleted file mode 100644
>> index 32df804614d3..000000000000
>> diff --git a/drivers/staging/lustre/lustre/obdclass/Makefile b/drivers/staging/lustre/lustre/obdclass/Makefile
>> deleted file mode 100644
>> index e3fa9acff4c4..000000000000
>> diff --git a/drivers/staging/lustre/lustre/obdclass/cl_internal.h b/drivers/staging/lustre/lustre/obdclass/cl_internal.h
>> deleted file mode 100644
>> index a0db830ca841..000000000000
>> diff --git a/drivers/staging/lustre/lustre/obdclass/cl_io.c b/drivers/staging/lustre/lustre/obdclass/cl_io.c
>> deleted file mode 100644
>> index fcdae6029258..000000000000
>> diff --git a/drivers/staging/lustre/lustre/obdclass/cl_lock.c b/drivers/staging/lustre/lustre/obdclass/cl_lock.c
>> deleted file mode 100644
>> index 9ca29a26a38b..000000000000
>> diff --git a/drivers/staging/lustre/lustre/obdclass/cl_object.c b/drivers/staging/lustre/lustre/obdclass/cl_object.c
>> deleted file mode 100644
>> index 42cce2dc5a45..000000000000
>> diff --git a/drivers/staging/lustre/lustre/obdclass/cl_page.c b/drivers/staging/lustre/lustre/obdclass/cl_page.c
>> deleted file mode 100644
>> index 916cf81c5997..000000000000
>> diff --git a/drivers/staging/lustre/lustre/obdclass/class_obd.c b/drivers/staging/lustre/lustre/obdclass/class_obd.c
>> deleted file mode 100644
>> index d6c46858941b..000000000000
>> diff --git a/drivers/staging/lustre/lustre/obdclass/debug.c b/drivers/staging/lustre/lustre/obdclass/debug.c
>> deleted file mode 100644
>> index 2156a82a613a..000000000000
>> diff --git a/drivers/staging/lustre/lustre/obdclass/genops.c b/drivers/staging/lustre/lustre/obdclass/genops.c
>> deleted file mode 100644
>> index 234f383ce6d9..000000000000
>> diff --git a/drivers/staging/lustre/lustre/obdclass/kernelcomm.c b/drivers/staging/lustre/lustre/obdclass/kernelcomm.c
>> deleted file mode 100644
>> index 63067a7f1e19..000000000000
>> diff --git a/drivers/staging/lustre/lustre/obdclass/linkea.c b/drivers/staging/lustre/lustre/obdclass/linkea.c
>> deleted file mode 100644
>> index 74c99ee216bb..000000000000
>> diff --git a/drivers/staging/lustre/lustre/obdclass/linux/linux-module.c b/drivers/staging/lustre/lustre/obdclass/linux/linux-module.c
>> deleted file mode 100644
>> index 9c800580053b..000000000000
>> diff --git a/drivers/staging/lustre/lustre/obdclass/linux/linux-sysctl.c b/drivers/staging/lustre/lustre/obdclass/linux/linux-sysctl.c
>> deleted file mode 100644
>> index e5e8687784ee..000000000000
>> diff --git a/drivers/staging/lustre/lustre/obdclass/llog.c b/drivers/staging/lustre/lustre/obdclass/llog.c
>> deleted file mode 100644
>> index bba84eae1e19..000000000000
>> diff --git a/drivers/staging/lustre/lustre/obdclass/llog_cat.c b/drivers/staging/lustre/lustre/obdclass/llog_cat.c
>> deleted file mode 100644
>> index d9c63adff206..000000000000
>> diff --git a/drivers/staging/lustre/lustre/obdclass/llog_internal.h b/drivers/staging/lustre/lustre/obdclass/llog_internal.h
>> deleted file mode 100644
>> index 4991d4e589dc..000000000000
>> diff --git a/drivers/staging/lustre/lustre/obdclass/llog_obd.c b/drivers/staging/lustre/lustre/obdclass/llog_obd.c
>> deleted file mode 100644
>> index 26aea114a29b..000000000000
>> diff --git a/drivers/staging/lustre/lustre/obdclass/llog_swab.c b/drivers/staging/lustre/lustre/obdclass/llog_swab.c
>> deleted file mode 100644
>> index b431c3408fe4..000000000000
>> diff --git a/drivers/staging/lustre/lustre/obdclass/lprocfs_counters.c b/drivers/staging/lustre/lustre/obdclass/lprocfs_counters.c
>> deleted file mode 100644
>> index 85f09aff6e83..000000000000
>> diff --git a/drivers/staging/lustre/lustre/obdclass/lprocfs_status.c b/drivers/staging/lustre/lustre/obdclass/lprocfs_status.c
>> deleted file mode 100644
>> index bdbe6f52031a..000000000000
>> diff --git a/drivers/staging/lustre/lustre/obdclass/lu_object.c b/drivers/staging/lustre/lustre/obdclass/lu_object.c
>> deleted file mode 100644
>> index aa9d74e087f4..000000000000
>> diff --git a/drivers/staging/lustre/lustre/obdclass/lu_ref.c b/drivers/staging/lustre/lustre/obdclass/lu_ref.c
>> deleted file mode 100644
>> index f67cb89ea0ba..000000000000
>> diff --git a/drivers/staging/lustre/lustre/obdclass/lustre_handles.c b/drivers/staging/lustre/lustre/obdclass/lustre_handles.c
>> deleted file mode 100644
>> index cdc8dc10690d..000000000000
>> diff --git a/drivers/staging/lustre/lustre/obdclass/lustre_peer.c b/drivers/staging/lustre/lustre/obdclass/lustre_peer.c
>> deleted file mode 100644
>> index e286a2665423..000000000000
>> diff --git a/drivers/staging/lustre/lustre/obdclass/obd_config.c b/drivers/staging/lustre/lustre/obdclass/obd_config.c
>> deleted file mode 100644
>> index ffc1814398a5..000000000000
>> diff --git a/drivers/staging/lustre/lustre/obdclass/obd_mount.c b/drivers/staging/lustre/lustre/obdclass/obd_mount.c
>> deleted file mode 100644
>> index 06c38fdef7ba..000000000000
>> diff --git a/drivers/staging/lustre/lustre/obdclass/obdo.c b/drivers/staging/lustre/lustre/obdclass/obdo.c
>> deleted file mode 100644
>> index c4503bc36591..000000000000
>> diff --git a/drivers/staging/lustre/lustre/obdclass/statfs_pack.c b/drivers/staging/lustre/lustre/obdclass/statfs_pack.c
>> deleted file mode 100644
>> index 355e888885f4..000000000000
>> diff --git a/drivers/staging/lustre/lustre/obdclass/uuid.c b/drivers/staging/lustre/lustre/obdclass/uuid.c
>> deleted file mode 100644
>> index ec8c6dc5c9a7..000000000000
>> diff --git a/drivers/staging/lustre/lustre/obdecho/Makefile b/drivers/staging/lustre/lustre/obdecho/Makefile
>> deleted file mode 100644
>> index 6be66fbab872..000000000000
>> diff --git a/drivers/staging/lustre/lustre/obdecho/echo_client.c b/drivers/staging/lustre/lustre/obdecho/echo_client.c
>> deleted file mode 100644
>> index b692e76e7108..000000000000
>> diff --git a/drivers/staging/lustre/lustre/obdecho/echo_internal.h b/drivers/staging/lustre/lustre/obdecho/echo_internal.h
>> deleted file mode 100644
>> index 42faa164fabb..000000000000
>> diff --git a/drivers/staging/lustre/lustre/osc/Makefile b/drivers/staging/lustre/lustre/osc/Makefile
>> deleted file mode 100644
>> index 30dec90e64e8..000000000000
>> diff --git a/drivers/staging/lustre/lustre/osc/lproc_osc.c b/drivers/staging/lustre/lustre/osc/lproc_osc.c
>> deleted file mode 100644
>> index 6a705bc5420c..000000000000
>> diff --git a/drivers/staging/lustre/lustre/osc/osc_cache.c b/drivers/staging/lustre/lustre/osc/osc_cache.c
>> deleted file mode 100644
>> index f26983004843..000000000000
>> diff --git a/drivers/staging/lustre/lustre/osc/osc_cl_internal.h b/drivers/staging/lustre/lustre/osc/osc_cl_internal.h
>> deleted file mode 100644
>> index 2d3cba16ef34..000000000000
>> diff --git a/drivers/staging/lustre/lustre/osc/osc_dev.c b/drivers/staging/lustre/lustre/osc/osc_dev.c
>> deleted file mode 100644
>> index 2b5f324743e2..000000000000
>> diff --git a/drivers/staging/lustre/lustre/osc/osc_internal.h b/drivers/staging/lustre/lustre/osc/osc_internal.h
>> deleted file mode 100644
>> index 4ddba1354bef..000000000000
>> diff --git a/drivers/staging/lustre/lustre/osc/osc_io.c b/drivers/staging/lustre/lustre/osc/osc_io.c
>> deleted file mode 100644
>> index 67734a8ed331..000000000000
>> diff --git a/drivers/staging/lustre/lustre/osc/osc_lock.c b/drivers/staging/lustre/lustre/osc/osc_lock.c
>> deleted file mode 100644
>> index d93d33dc8dc4..000000000000
>> diff --git a/drivers/staging/lustre/lustre/osc/osc_object.c b/drivers/staging/lustre/lustre/osc/osc_object.c
>> deleted file mode 100644
>> index 84240181c7ea..000000000000
>> diff --git a/drivers/staging/lustre/lustre/osc/osc_page.c b/drivers/staging/lustre/lustre/osc/osc_page.c
>> deleted file mode 100644
>> index 20c553ef3a5e..000000000000
>> diff --git a/drivers/staging/lustre/lustre/osc/osc_quota.c b/drivers/staging/lustre/lustre/osc/osc_quota.c
>> deleted file mode 100644
>> index 723ec2fb18bf..000000000000
>> diff --git a/drivers/staging/lustre/lustre/osc/osc_request.c b/drivers/staging/lustre/lustre/osc/osc_request.c
>> deleted file mode 100644
>> index 61ef6c8d7a12..000000000000
>> diff --git a/drivers/staging/lustre/lustre/ptlrpc/Makefile b/drivers/staging/lustre/lustre/ptlrpc/Makefile
>> deleted file mode 100644
>> index 1deb1971b39e..000000000000
>> diff --git a/drivers/staging/lustre/lustre/ptlrpc/client.c b/drivers/staging/lustre/lustre/ptlrpc/client.c
>> deleted file mode 100644
>> index c1b82bf20f08..000000000000
>> diff --git a/drivers/staging/lustre/lustre/ptlrpc/connection.c b/drivers/staging/lustre/lustre/ptlrpc/connection.c
>> deleted file mode 100644
>> index fb35a89ca6c6..000000000000
>> diff --git a/drivers/staging/lustre/lustre/ptlrpc/errno.c b/drivers/staging/lustre/lustre/ptlrpc/errno.c
>> deleted file mode 100644
>> index b904524fc1c6..000000000000
>> diff --git a/drivers/staging/lustre/lustre/ptlrpc/events.c b/drivers/staging/lustre/lustre/ptlrpc/events.c
>> deleted file mode 100644
>> index 130bacc2c891..000000000000
>> diff --git a/drivers/staging/lustre/lustre/ptlrpc/import.c b/drivers/staging/lustre/lustre/ptlrpc/import.c
>> deleted file mode 100644
>> index 1a0f35dfab97..000000000000
>> diff --git a/drivers/staging/lustre/lustre/ptlrpc/layout.c b/drivers/staging/lustre/lustre/ptlrpc/layout.c
>> deleted file mode 100644
>> index 417d4a151433..000000000000
>> diff --git a/drivers/staging/lustre/lustre/ptlrpc/llog_client.c b/drivers/staging/lustre/lustre/ptlrpc/llog_client.c
>> deleted file mode 100644
>> index 946d538121de..000000000000
>> diff --git a/drivers/staging/lustre/lustre/ptlrpc/llog_net.c b/drivers/staging/lustre/lustre/ptlrpc/llog_net.c
>> deleted file mode 100644
>> index b871d9e40a9e..000000000000
>> diff --git a/drivers/staging/lustre/lustre/ptlrpc/lproc_ptlrpc.c b/drivers/staging/lustre/lustre/ptlrpc/lproc_ptlrpc.c
>> deleted file mode 100644
>> index 0b638837f88b..000000000000
>> diff --git a/drivers/staging/lustre/lustre/ptlrpc/niobuf.c b/drivers/staging/lustre/lustre/ptlrpc/niobuf.c
>> deleted file mode 100644
>> index 2897afb8806c..000000000000
>> diff --git a/drivers/staging/lustre/lustre/ptlrpc/nrs.c b/drivers/staging/lustre/lustre/ptlrpc/nrs.c
>> deleted file mode 100644
>> index e09b86529c5d..000000000000
>> diff --git a/drivers/staging/lustre/lustre/ptlrpc/nrs_fifo.c b/drivers/staging/lustre/lustre/ptlrpc/nrs_fifo.c
>> deleted file mode 100644
>> index ff630d94dd26..000000000000
>> diff --git a/drivers/staging/lustre/lustre/ptlrpc/pack_generic.c b/drivers/staging/lustre/lustre/ptlrpc/pack_generic.c
>> deleted file mode 100644
>> index 6ac9bb570663..000000000000
>> diff --git a/drivers/staging/lustre/lustre/ptlrpc/pers.c b/drivers/staging/lustre/lustre/ptlrpc/pers.c
>> deleted file mode 100644
>> index 2466868afb9c..000000000000
>> diff --git a/drivers/staging/lustre/lustre/ptlrpc/pinger.c b/drivers/staging/lustre/lustre/ptlrpc/pinger.c
>> deleted file mode 100644
>> index b3297b5ce395..000000000000
>> diff --git a/drivers/staging/lustre/lustre/ptlrpc/ptlrpc_internal.h b/drivers/staging/lustre/lustre/ptlrpc/ptlrpc_internal.h
>> deleted file mode 100644
>> index 134b74234519..000000000000
>> diff --git a/drivers/staging/lustre/lustre/ptlrpc/ptlrpc_module.c b/drivers/staging/lustre/lustre/ptlrpc/ptlrpc_module.c
>> deleted file mode 100644
>> index 5c32b657b3b5..000000000000
>> diff --git a/drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c b/drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c
>> deleted file mode 100644
>> index 531005411edf..000000000000
>> diff --git a/drivers/staging/lustre/lustre/ptlrpc/recover.c b/drivers/staging/lustre/lustre/ptlrpc/recover.c
>> deleted file mode 100644
>> index 2ea0a7ff87dd..000000000000
>> diff --git a/drivers/staging/lustre/lustre/ptlrpc/sec.c b/drivers/staging/lustre/lustre/ptlrpc/sec.c
>> deleted file mode 100644
>> index e193f3346e6f..000000000000
>> diff --git a/drivers/staging/lustre/lustre/ptlrpc/sec_bulk.c b/drivers/staging/lustre/lustre/ptlrpc/sec_bulk.c
>> deleted file mode 100644
>> index 625b9520d78f..000000000000
>> diff --git a/drivers/staging/lustre/lustre/ptlrpc/sec_config.c b/drivers/staging/lustre/lustre/ptlrpc/sec_config.c
>> deleted file mode 100644
>> index 2389f9a8f534..000000000000
>> diff --git a/drivers/staging/lustre/lustre/ptlrpc/sec_gc.c b/drivers/staging/lustre/lustre/ptlrpc/sec_gc.c
>> deleted file mode 100644
>> index 2c8bad7b7877..000000000000
>> diff --git a/drivers/staging/lustre/lustre/ptlrpc/sec_lproc.c b/drivers/staging/lustre/lustre/ptlrpc/sec_lproc.c
>> deleted file mode 100644
>> index 2bb75ebd5d98..000000000000
>> diff --git a/drivers/staging/lustre/lustre/ptlrpc/sec_null.c b/drivers/staging/lustre/lustre/ptlrpc/sec_null.c
>> deleted file mode 100644
>> index ecc387d1b9b4..000000000000
>> diff --git a/drivers/staging/lustre/lustre/ptlrpc/sec_plain.c b/drivers/staging/lustre/lustre/ptlrpc/sec_plain.c
>> deleted file mode 100644
>> index ec3d9af76b17..000000000000
>> diff --git a/drivers/staging/lustre/lustre/ptlrpc/service.c b/drivers/staging/lustre/lustre/ptlrpc/service.c
>> deleted file mode 100644
>> index 3fd8c746f460..000000000000
>> diff --git a/drivers/staging/lustre/lustre/ptlrpc/wiretest.c b/drivers/staging/lustre/lustre/ptlrpc/wiretest.c
>> deleted file mode 100644
>> index f9394c3e1ee2..000000000000
>> diff --git a/drivers/staging/lustre/sysfs-fs-lustre b/drivers/staging/lustre/sysfs-fs-lustre
>> deleted file mode 100644
>> index 8691c6543a9c..000000000000
>> diff --git a/scripts/selinux/mdp/mdp.c b/scripts/selinux/mdp/mdp.c
>> index ffe8179f5d41..073fe7537f6c 100644
>> --- a/scripts/selinux/mdp/mdp.c
>> +++ b/scripts/selinux/mdp/mdp.c
>> @@ -124,7 +124,6 @@ int main(int argc, char *argv[])
>> 	fprintf(fout, "fs_use_xattr reiserfs user_u:base_r:base_t;\n");
>> 	fprintf(fout, "fs_use_xattr jffs2 user_u:base_r:base_t;\n");
>> 	fprintf(fout, "fs_use_xattr gfs2 user_u:base_r:base_t;\n");
>> -	fprintf(fout, "fs_use_xattr lustre user_u:base_r:base_t;\n");
>> 
>> 	fprintf(fout, "fs_use_task eventpollfs user_u:base_r:base_t;\n");
>> 	fprintf(fout, "fs_use_task pipefs user_u:base_r:base_t;\n");
>> --
>> 2.17.1
>> 
>> 
> 
> 
> Cheers, Andreas
> 
> 
> 
> 
> 


From gregkh at linuxfoundation.org  Fri Jun  1 19:08:39 2018
From: gregkh at linuxfoundation.org (Greg Kroah-Hartman)
Date: Fri, 1 Jun 2018 21:08:39 +0200
Subject: [lustre-devel] [PATCH] staging: lustre: delete the filesystem
 from the tree.
In-Reply-To: <6A5E4E02-AE72-4068-BF93-38791BF38B4F@dilger.ca>
References: <20180601091133.GA27521@kroah.com>
 <6A5E4E02-AE72-4068-BF93-38791BF38B4F@dilger.ca>
Message-ID: <20180601190839.GA20734@kroah.com>

On Fri, Jun 01, 2018 at 02:30:49PM -0400, Andreas Dilger wrote:
> On Jun 1, 2018, at 5:11 AM, Greg Kroah-Hartman <gregkh at linuxfoundation.org> wrote:
> > 
> > The Lustre filesystem has been in the kernel tree for over 5 years now.
> > While it has been an endless source of enjoyment for new kernel
> > developers learning how to do basic codingstyle cleanups, as well as an
> > semi-entertaining source of bewilderment from the vfs developers any
> > time they have looked into the codebase to try to figure out how to port
> > their latest api changes to this filesystem, it has not really moved
> > forward into the "this is in shape to get out of staging" despite many
> > half-completed attempts.
> 
> I am happy to submit a patch that moves Lustre out of staging and into
> the mainline.  I'm just about to board a flight, but it could be done
> later today.  Then we can avoid the constant churn of kernel newbies
> submitting patches that break the code.

It's not the kernel newbies that are your biggest problem.  It's your
development model, and lack of support from anyone to actually do real
cleanups.

Every few months, when I get really bored, I do a drive-by pass and have
never yet failed to find numerous problems that you all have been
totally ignoring.  Look at the debugfs patch series I sent out this
week.  Look at all of the wonderful work that NeilBrown has been doing
on core cleanups for your mess.  Neither he nor I should be the ones
that have to do this stuff, you all should be doing it yourself.

You all have had over 5 years to work on cleaning up your crazy shim
layers.  It's still not done.  And based on the recent patch series that
was sent to me, I don't even see anyone planning on working on that type
of thing.

I'm sorry, but I am tired of it.  If you want to submit this for the
filesystem maintainers to accept, wonderful, please do so.  But that
will not consist of a patch that just moves the directories.  You have
to create a real set of patches and send them to linux-fsdevel, just
like any other new filesystem does.

And I would like to point out the fact that while this whole thing sat
in staging, other filesystems and major driver subsystems entered
staging, got cleaned up, and moved on out.  It can be done.  But for
some reason you all don't want to do the real work to do it, which is
fine, I understand that you need management buy-in to make that happen,
and you all need to focus on your users and new features for them.

So if you really do want this out of staging, go off and do the real
work to do so and then merge it the "real" way.  I'm tired of dealing
with it in staging as-is and having to every few months start rejecting
new feature patches and tell people that "this is the last time I'm
going to tell you..."

You all were warned many times, this is not my fault, but rather, yours,
or your management for not allowing you to prioritize this work.

> Lustre has been in use at large sites around the world for 18 years now.
> Over 70% of the largest 100 systems in the world use Lustre.  It runs at
> universities, oil companies, weather bureaus around the world, etc.

And yet, I bet none of them run the in-kernel codebase, right?

I bet you could get all of this cleaned up and merge properly in 6
months if you really wanted to do so.  But it seems that no one really
wants to do that work, which is sad.  I should not have to be the one to
force you all, but it seems like I now have to be.

> I know Andrew has long been a supporter of getting code into the kernel
> that users are using.  This is code that thousands of large computers
> are using with exabytes of storage, a lot more than orangefs, exofs, and
> random other filesystems that seem to get into the kernel easily.

Because those developers took the time and did it right.

Please, compare yourself to orangefs.  That is the perfect example of
how to do everything right.  They got their code into staging, cleaned
it up, talked to us about what was needed to do to get the remaining
bits in proper shape, they assigned dedicated developers to do that
work, talked with all of us at different conferences around the world to
check up and constantly ensure that they were doing the right thing, and
most importantly, they asked for feedback and acted on it.  In the end,
their codebase is much smaller, works better, is in the "real" part of
the kernel, and available to every Linux user out there.

So yes, you are no orangefs, but you really should aspire to be just
like them, as they know how to do all of this correctly.

> It's true the code is not as pretty as it could be, but the same is true
> of code that has been in the kernel for ages.

If you know of major subsystems that look at bad as lustre does with the
numerous wrapper layers and other crazy things that you all do, please
let me know and I will be glad to go point people at that code to help
get it fixed up.

Also note that the attempt to defend your behavior with "but he's doing
the same thing over there!" is not a valid defense at all.  I'll gladly
go over and deal with your misbehaving sibling after dealing with you :)

> We've spent years cleaning it up in staging, and it has gotten a lot
> better.

Better yes.  Correct and mergable out of staging, no.  You all have had
plenty of time to get your act together.  Maybe this is what you all
really need to kick it into gear to do it correctly.  I certainly hope
so, because I think we all can agree that what we have been doing for
the past 5 years is NOT working at all.

sorry,

greg k-h

From neilb at suse.com  Fri Jun  1 23:19:40 2018
From: neilb at suse.com (NeilBrown)
Date: Sat, 02 Jun 2018 09:19:40 +1000
Subject: [lustre-devel] [PATCH] staging: lustre: delete the filesystem
 from the tree.
In-Reply-To: <29ACF5A8-7608-46BB-8191-E3FEB77D0F24@cray.com>
References: <20180601091133.GA27521@kroah.com>
 <20180601114151.GA25225@infradead.org>
 <E5BF8501-35EB-4DDF-ADBE-6016CF31A502@dilger.ca>
 <29ACF5A8-7608-46BB-8191-E3FEB77D0F24@cray.com>
Message-ID: <87h8mmrt6b.fsf@notabene.neil.brown.name>

On Fri, Jun 01 2018, Doug Oucharek wrote:

> Would it makes sense to land LNet and LNDs on their own first?  Get
> the networking house in order first before layering on the file
> system?

I'd like to turn that question on it's head:
  Do we need LNet and LNDs?  What value do they provide?
(this is a genuine question, not being sarcastic).

It is a while since I tried to understand LNet, and then it was a
fairly superficial look, but I think it is an abstraction layer
that provides packet-based send/receive with some numa-awareness
and routing functionality.  It sits over sockets (TCP) and IB and
provides a uniform interface.

That is almost a description of the xprt layer in sunrpc.  sunrpc
doesn't have routing, but it does have some numa awareness (for the
server side at least) and it definitely provides packet-based
send/receive over various transports - tcp, udp, local (unix domain),
and IB.
So: can we use sunrpc/xprt in place of LNet?  How much would we need to
enhance sunrpc/xprt for this to work?  What hooks would be needed to
implement the routing as a separate layer.

If LNet is, in some way, much better than sunrpc, then can we share that
superior functionality with our NFS friends by adding it to sunrpc?

Maybe the answer to this is "no", but I think LNet would be hard to sell
without a clear statement of why that was the answer.

One reason that I would like to see lustre stay in drivers/staging (so I
do not support Greg's patch) is that this sort of transition of Lustre
to using an improved sunrpc/xprt would be much easier if both were in
the same tree.  Certainly it would be easier for a larger community to
be participating in the work.

Thanks,
NeilBrown
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180602/953bfcbc/attachment.sig>

From neilb at suse.com  Sat Jun  2 00:28:38 2018
From: neilb at suse.com (NeilBrown)
Date: Sat, 02 Jun 2018 10:28:38 +1000
Subject: [lustre-devel] [PATCH] staging: lustre: delete the filesystem
 from the tree.
In-Reply-To: <20180601091133.GA27521@kroah.com>
References: <20180601091133.GA27521@kroah.com>
Message-ID: <87bmcurpzd.fsf@notabene.neil.brown.name>

On Fri, Jun 01 2018, Greg Kroah-Hartman wrote:

>
> So, let's just delete the whole mess.  Now the lustre developers can go
> off and work in their out-of-tree codebase and not have to worry about
> providing valid changelog entries and breaking their patches up into
> logical pieces.

I find it incredible that anyone would think that not having to "worry
about providing valid changelogs" and not "breaking their patches up
into logic pieces" could ever be seen as a good idea.  I hope that if
lustre development is excluded from mainline for a time, that we can
still maintain the practices that demonstrably work so well.

For the record: I'm not in favor of ejecting this code from mainline.  I
think that the long term result may be that it never comes back, and
will like at least delay the process.
But you must do what you think is best.

Thanks,
NeilBrown
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180602/30a54852/attachment.sig>

From andreas.dilger at intel.com  Sun Jun  3 20:34:52 2018
From: andreas.dilger at intel.com (Dilger, Andreas)
Date: Sun, 3 Jun 2018 20:34:52 +0000
Subject: [lustre-devel] [PATCH] staging: lustre: delete the filesystem
 from the tree.
In-Reply-To: <87h8mmrt6b.fsf@notabene.neil.brown.name>
References: <20180601091133.GA27521@kroah.com>
 <20180601114151.GA25225@infradead.org>
 <E5BF8501-35EB-4DDF-ADBE-6016CF31A502@dilger.ca>
 <29ACF5A8-7608-46BB-8191-E3FEB77D0F24@cray.com>
 <87h8mmrt6b.fsf@notabene.neil.brown.name>
Message-ID: <58123CDD-8424-4E1D-A11F-0F899970A49B@intel.com>

On Jun 1, 2018, at 17:19, NeilBrown <neilb at suse.com> wrote:
> 
> On Fri, Jun 01 2018, Doug Oucharek wrote:
> 
>> Would it makes sense to land LNet and LNDs on their own first?  Get
>> the networking house in order first before layering on the file
>> system?
> 
> I'd like to turn that question on it's head:
>  Do we need LNet and LNDs?  What value do they provide?
> (this is a genuine question, not being sarcastic).
> 
> It is a while since I tried to understand LNet, and then it was a
> fairly superficial look, but I think it is an abstraction layer
> that provides packet-based send/receive with some numa-awareness
> and routing functionality.  It sits over sockets (TCP) and IB and
> provides a uniform interface.

LNet is originally based on a high-performance networking stack called
Portals (v3, http://www.cs.sandia.gov/Portals/), with additions for LNet
routing to allow cross-network bridging.

A critical part of LNet is that it is for RDMA and not packet-based
messages.  Everything in Lustre is structured around RDMA.  Of course,
RDMA is not possible with TCP so it just does send/receive under the
covers, though it can do zero copy data sends (and at one time zero-copy
receives, but those changes were rejected by the kernel maintainers).
It definitely does RDMA with IB, RoCE, OPA in the kernel, and other RDMA
network types not in the kernel (e.g. Cray Gemini/Aries, Atos/Bull BXI,
and previously older network types no longer supported).

Even with TCP it has some improvements for performance, such as using
separate sockets for send and receive of large messages, as well as
a socket for small messages that has Nagle disabled so that it does
not delay those packets for aggregation.

In addition to the RDMA support, there is also multi-rail support in
the out-of-tree version that we haven't been allowed to land, which
can aggregate network bandwidth.  While there exists channel bonding
for TCP connections, that does not exist for IB or other RDMA networks.

> That is almost a description of the xprt layer in sunrpc.  sunrpc
> doesn't have routing, but it does have some numa awareness (for the
> server side at least) and it definitely provides packet-based
> send/receive over various transports - tcp, udp, local (unix domain),
> and IB.
> So: can we use sunrpc/xprt in place of LNet?

No, that would totally kill the performance of Lustre.

> How much would we need to enhance sunrpc/xprt for this to work?  What
> hooks would be needed to implement the routing as a separate layer.
> 
> If LNet is, in some way, much better than sunrpc, then can we share that
> superior functionality with our NFS friends by adding it to sunrpc?

There was some discussion at NetApp about adding a Lustre/LNet transport
for pNFS, but I don't think it ever got beyond the proposal stage:

https://tools.ietf.org/html/draft-faibish-nfsv4-pnfs-lustre-layout-07

> Maybe the answer to this is "no", but I think LNet would be hard to sell
> without a clear statement of why that was the answer.

There are other users outside of the kernel tree that use LNet in addition
to just Lustre.  The Cray "DVS" I/O forwarding service[*] uses LNet, and
another experimental filesystem named Zest[+] also used LNet.

[*] https://www.alcf.anl.gov/files/Sugiyama-Wallace-Thursday16B-slides.pdf
[+] https://www.psc.edu/images/zest/zest-sc07-paper.pdf

> One reason that I would like to see lustre stay in drivers/staging (so I
> do not support Greg's patch) is that this sort of transition of Lustre
> to using an improved sunrpc/xprt would be much easier if both were in
> the same tree.  Certainly it would be easier for a larger community to
> be participating in the work.

I don't think the proposal to encapsulate all of the Lustre protocol into
pNFS made a lot of sense, since this would have only really been available
on Linux, at which point it would be better to use the native Lustre client
rather than funnel everything through pNFS.

However, _just_ using the LNet transport for (p)NFS might make sense.  LNet
is largely independent from Lustre (it used to be a separate source tree)
and is very efficient over the network.

Cheers, Andreas
--
Andreas Dilger
Lustre Principal Architect
Intel Corporation








From neilb at suse.com  Mon Jun  4 03:54:55 2018
From: neilb at suse.com (NeilBrown)
Date: Mon, 04 Jun 2018 13:54:55 +1000
Subject: [lustre-devel] [PATCH] staging: lustre: delete the filesystem
 from the tree.
In-Reply-To: <58123CDD-8424-4E1D-A11F-0F899970A49B@intel.com>
References: <20180601091133.GA27521@kroah.com>
 <20180601114151.GA25225@infradead.org>
 <E5BF8501-35EB-4DDF-ADBE-6016CF31A502@dilger.ca>
 <29ACF5A8-7608-46BB-8191-E3FEB77D0F24@cray.com>
 <87h8mmrt6b.fsf@notabene.neil.brown.name>
 <58123CDD-8424-4E1D-A11F-0F899970A49B@intel.com>
Message-ID: <87h8mjp5o0.fsf@notabene.neil.brown.name>

On Sun, Jun 03 2018, Dilger, Andreas wrote:

> On Jun 1, 2018, at 17:19, NeilBrown <neilb at suse.com> wrote:
>> 
>> On Fri, Jun 01 2018, Doug Oucharek wrote:
>> 
>>> Would it makes sense to land LNet and LNDs on their own first?  Get
>>> the networking house in order first before layering on the file
>>> system?
>> 
>> I'd like to turn that question on it's head:
>>  Do we need LNet and LNDs?  What value do they provide?
>> (this is a genuine question, not being sarcastic).
>> 
>> It is a while since I tried to understand LNet, and then it was a
>> fairly superficial look, but I think it is an abstraction layer
>> that provides packet-based send/receive with some numa-awareness
>> and routing functionality.  It sits over sockets (TCP) and IB and
>> provides a uniform interface.
>
> LNet is originally based on a high-performance networking stack called
> Portals (v3, http://www.cs.sandia.gov/Portals/), with additions for LNet
> routing to allow cross-network bridging.
>
> A critical part of LNet is that it is for RDMA and not packet-based
> messages.  Everything in Lustre is structured around RDMA.  Of course,
> RDMA is not possible with TCP so it just does send/receive under the
> covers, though it can do zero copy data sends (and at one time zero-copy
> receives, but those changes were rejected by the kernel maintainers).
> It definitely does RDMA with IB, RoCE, OPA in the kernel, and other RDMA
> network types not in the kernel (e.g. Cray Gemini/Aries, Atos/Bull BXI,
> and previously older network types no longer supported).

Thanks!  That will probably help me understand it more easily next time
I dive in.

>
> Even with TCP it has some improvements for performance, such as using
> separate sockets for send and receive of large messages, as well as
> a socket for small messages that has Nagle disabled so that it does
> not delay those packets for aggregation.

That sounds like something that could benefit NFS...
pNFS already partially does this by virtue of the fact that data often
goes to a different server than control, so a different socket is
needed.  I wonder if it could benefit from more explicit separate of
message sizes.


Thanks a lot for this background info!
NeilBrown

>
> In addition to the RDMA support, there is also multi-rail support in
> the out-of-tree version that we haven't been allowed to land, which
> can aggregate network bandwidth.  While there exists channel bonding
> for TCP connections, that does not exist for IB or other RDMA networks.
>
>> That is almost a description of the xprt layer in sunrpc.  sunrpc
>> doesn't have routing, but it does have some numa awareness (for the
>> server side at least) and it definitely provides packet-based
>> send/receive over various transports - tcp, udp, local (unix domain),
>> and IB.
>> So: can we use sunrpc/xprt in place of LNet?
>
> No, that would totally kill the performance of Lustre.
>
>> How much would we need to enhance sunrpc/xprt for this to work?  What
>> hooks would be needed to implement the routing as a separate layer.
>> 
>> If LNet is, in some way, much better than sunrpc, then can we share that
>> superior functionality with our NFS friends by adding it to sunrpc?
>
> There was some discussion at NetApp about adding a Lustre/LNet transport
> for pNFS, but I don't think it ever got beyond the proposal stage:
>
> https://tools.ietf.org/html/draft-faibish-nfsv4-pnfs-lustre-layout-07
>
>> Maybe the answer to this is "no", but I think LNet would be hard to sell
>> without a clear statement of why that was the answer.
>
> There are other users outside of the kernel tree that use LNet in addition
> to just Lustre.  The Cray "DVS" I/O forwarding service[*] uses LNet, and
> another experimental filesystem named Zest[+] also used LNet.
>
> [*] https://www.alcf.anl.gov/files/Sugiyama-Wallace-Thursday16B-slides.pdf
> [+] https://www.psc.edu/images/zest/zest-sc07-paper.pdf
>
>> One reason that I would like to see lustre stay in drivers/staging (so I
>> do not support Greg's patch) is that this sort of transition of Lustre
>> to using an improved sunrpc/xprt would be much easier if both were in
>> the same tree.  Certainly it would be easier for a larger community to
>> be participating in the work.
>
> I don't think the proposal to encapsulate all of the Lustre protocol into
> pNFS made a lot of sense, since this would have only really been available
> on Linux, at which point it would be better to use the native Lustre client
> rather than funnel everything through pNFS.
>
> However, _just_ using the LNet transport for (p)NFS might make sense.  LNet
> is largely independent from Lustre (it used to be a separate source tree)
> and is very efficient over the network.
>
> Cheers, Andreas
> --
> Andreas Dilger
> Lustre Principal Architect
> Intel Corporation
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180604/f24aa49f/attachment.sig>

From alexey.lyashkov at gmail.com  Mon Jun  4 03:59:45 2018
From: alexey.lyashkov at gmail.com (Alexey Lyashkov)
Date: Mon, 4 Jun 2018 06:59:45 +0300
Subject: [lustre-devel] [PATCH] staging: lustre: delete the filesystem
 from the tree.
In-Reply-To: <87h8mjp5o0.fsf@notabene.neil.brown.name>
References: <20180601091133.GA27521@kroah.com>
 <20180601114151.GA25225@infradead.org>
 <E5BF8501-35EB-4DDF-ADBE-6016CF31A502@dilger.ca>
 <29ACF5A8-7608-46BB-8191-E3FEB77D0F24@cray.com>
 <87h8mmrt6b.fsf@notabene.neil.brown.name>
 <58123CDD-8424-4E1D-A11F-0F899970A49B@intel.com>
 <87h8mjp5o0.fsf@notabene.neil.brown.name>
Message-ID: <FE290581-D17D-4CEC-9AE2-B79DCBBD0823@gmail.com>


> 4 июня 2018 г., в 6:54, NeilBrown <neilb at suse.com> написал(а):
> 
> On Sun, Jun 03 2018, Dilger, Andreas wrote:
> 
>> On Jun 1, 2018, at 17:19, NeilBrown <neilb at suse.com> wrote:
>>> 
>>> On Fri, Jun 01 2018, Doug Oucharek wrote:
>>> 
>>>> Would it makes sense to land LNet and LNDs on their own first?  Get
>>>> the networking house in order first before layering on the file
>>>> system?
>>> 
>>> I'd like to turn that question on it's head:
>>> Do we need LNet and LNDs?  What value do they provide?
>>> (this is a genuine question, not being sarcastic).
>>> 
>>> It is a while since I tried to understand LNet, and then it was a
>>> fairly superficial look, but I think it is an abstraction layer
>>> that provides packet-based send/receive with some numa-awareness
>>> and routing functionality.  It sits over sockets (TCP) and IB and
>>> provides a uniform interface.
>> 
>> LNet is originally based on a high-performance networking stack called
>> Portals (v3, http://www.cs.sandia.gov/Portals/), with additions for LNet
>> routing to allow cross-network bridging.
>> 
>> A critical part of LNet is that it is for RDMA and not packet-based
>> messages.  Everything in Lustre is structured around RDMA.  Of course,
>> RDMA is not possible with TCP

To be clear. Soft IB (aka Soft RoCE) driver is part of OFED stack from 4.8(or 4.9).
So RDMA API now is possible with TCP networks.


Alex

From adilger at dilger.ca  Mon Jun  4 04:15:31 2018
From: adilger at dilger.ca (Andreas Dilger)
Date: Sun, 3 Jun 2018 22:15:31 -0600
Subject: [lustre-devel] [PATCH] staging: lustre: delete the filesystem
 from the tree.
In-Reply-To: <FE290581-D17D-4CEC-9AE2-B79DCBBD0823@gmail.com>
References: <20180601091133.GA27521@kroah.com>
 <20180601114151.GA25225@infradead.org>
 <E5BF8501-35EB-4DDF-ADBE-6016CF31A502@dilger.ca>
 <29ACF5A8-7608-46BB-8191-E3FEB77D0F24@cray.com>
 <87h8mmrt6b.fsf@notabene.neil.brown.name>
 <58123CDD-8424-4E1D-A11F-0F899970A49B@intel.com>
 <87h8mjp5o0.fsf@notabene.neil.brown.name>
 <FE290581-D17D-4CEC-9AE2-B79DCBBD0823@gmail.com>
Message-ID: <CF1B7A14-3C16-4CD1-8274-C6A3E0065624@dilger.ca>

On Jun 3, 2018, at 9:59 PM, Alexey Lyashkov <alexey.lyashkov at gmail.com> wrote:
> 
>> On Sun, Jun 03 2018, Dilger, Andreas wrote:
>> 
>>> LNet is originally based on a high-performance networking stack called
>>> Portals (v3, http://www.cs.sandia.gov/Portals/), with additions for LNet
>>> routing to allow cross-network bridging.
>>> 
>>> A critical part of LNet is that it is for RDMA and not packet-based
>>> messages.  Everything in Lustre is structured around RDMA.  Of course,
>>> RDMA is not possible with TCP
> 
> To be clear. Soft IB (aka Soft RoCE) driver is part of OFED stack from 4.8
> (or 4.9).  So RDMA API now is possible with TCP networks.

Well, strictly speaking RoCE still isn't possible with TCP networks.  RoCE v1
is an Ethernet layer protocol (not IP based), while RoCE v2 is UDP/IP based.

Cheers, Andreas





-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 873 bytes
Desc: Message signed with OpenPGP
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180603/b5789886/attachment.sig>

From alexey.lyashkov at gmail.com  Mon Jun  4 04:17:48 2018
From: alexey.lyashkov at gmail.com (Alexey Lyashkov)
Date: Mon, 4 Jun 2018 07:17:48 +0300
Subject: [lustre-devel] [PATCH] staging: lustre: delete the filesystem
 from the tree.
In-Reply-To: <CF1B7A14-3C16-4CD1-8274-C6A3E0065624@dilger.ca>
References: <20180601091133.GA27521@kroah.com>
 <20180601114151.GA25225@infradead.org>
 <E5BF8501-35EB-4DDF-ADBE-6016CF31A502@dilger.ca>
 <29ACF5A8-7608-46BB-8191-E3FEB77D0F24@cray.com>
 <87h8mmrt6b.fsf@notabene.neil.brown.name>
 <58123CDD-8424-4E1D-A11F-0F899970A49B@intel.com>
 <87h8mjp5o0.fsf@notabene.neil.brown.name>
 <FE290581-D17D-4CEC-9AE2-B79DCBBD0823@gmail.com>
 <CF1B7A14-3C16-4CD1-8274-C6A3E0065624@dilger.ca>
Message-ID: <271C72E6-27C0-425F-B43F-16A37C638DCB@gmail.com>


> 4 июня 2018 г., в 7:15, Andreas Dilger <adilger at dilger.ca> написал(а):
> 
> On Jun 3, 2018, at 9:59 PM, Alexey Lyashkov <alexey.lyashkov at gmail.com> wrote:
>> 
>>> On Sun, Jun 03 2018, Dilger, Andreas wrote:
>>> 
>>>> LNet is originally based on a high-performance networking stack called
>>>> Portals (v3, http://www.cs.sandia.gov/Portals/), with additions for LNet
>>>> routing to allow cross-network bridging.
>>>> 
>>>> A critical part of LNet is that it is for RDMA and not packet-based
>>>> messages.  Everything in Lustre is structured around RDMA.  Of course,
>>>> RDMA is not possible with TCP
>> 
>> To be clear. Soft IB (aka Soft RoCE) driver is part of OFED stack from 4.8
>> (or 4.9).  So RDMA API now is possible with TCP networks.
> 
> Well, strictly speaking RoCE still isn't possible with TCP networks.  RoCE v1
> is an Ethernet layer protocol (not IP based), while RoCE v2 is UDP/IP based.
> 
> 
Andreas,

I talk about RXE driver in OFED stack. It have works over TCP networks.
It likely to be RoCE v2 compatible as UDP tunnel use and exist for while.



Alex


From hch at infradead.org  Mon Jun  4 07:09:22 2018
From: hch at infradead.org (Christoph Hellwig)
Date: Mon, 4 Jun 2018 00:09:22 -0700
Subject: [lustre-devel] [PATCH] staging: lustre: delete the filesystem
 from the tree.
In-Reply-To: <20180601190839.GA20734@kroah.com>
References: <20180601091133.GA27521@kroah.com>
 <6A5E4E02-AE72-4068-BF93-38791BF38B4F@dilger.ca>
 <20180601190839.GA20734@kroah.com>
Message-ID: <20180604070922.GA13156@infradead.org>

On Fri, Jun 01, 2018 at 09:08:39PM +0200, Greg Kroah-Hartman wrote:
> Please, compare yourself to orangefs.  That is the perfect example of
> how to do everything right.  They got their code into staging, cleaned
> it up, talked to us about what was needed to do to get the remaining
> bits in proper shape, they assigned dedicated developers to do that
> work, talked with all of us at different conferences around the world to
> check up and constantly ensure that they were doing the right thing, and
> most importantly, they asked for feedback and acted on it.  In the end,
> their codebase is much smaller, works better, is in the "real" part of
> the kernel, and available to every Linux user out there.

FYI, orangefs never went through the statging tree.  Which might be
one reason why it got merged so quickly - allowing rapid iteration
without respect to merged windows, and doing all the trivial cleanups
either before or after (but not at the same time as) the feature
work really does help productivity.

From gregkh at linuxfoundation.org  Mon Jun  4 07:14:43 2018
From: gregkh at linuxfoundation.org (Greg Kroah-Hartman)
Date: Mon, 4 Jun 2018 09:14:43 +0200
Subject: [lustre-devel] [PATCH] staging: lustre: delete the filesystem
 from the tree.
In-Reply-To: <20180604070922.GA13156@infradead.org>
References: <20180601091133.GA27521@kroah.com>
 <6A5E4E02-AE72-4068-BF93-38791BF38B4F@dilger.ca>
 <20180601190839.GA20734@kroah.com>
 <20180604070922.GA13156@infradead.org>
Message-ID: <20180604071443.GA32335@kroah.com>

On Mon, Jun 04, 2018 at 12:09:22AM -0700, Christoph Hellwig wrote:
> On Fri, Jun 01, 2018 at 09:08:39PM +0200, Greg Kroah-Hartman wrote:
> > Please, compare yourself to orangefs.  That is the perfect example of
> > how to do everything right.  They got their code into staging, cleaned
> > it up, talked to us about what was needed to do to get the remaining
> > bits in proper shape, they assigned dedicated developers to do that
> > work, talked with all of us at different conferences around the world to
> > check up and constantly ensure that they were doing the right thing, and
> > most importantly, they asked for feedback and acted on it.  In the end,
> > their codebase is much smaller, works better, is in the "real" part of
> > the kernel, and available to every Linux user out there.
> 
> FYI, orangefs never went through the statging tree.  Which might be
> one reason why it got merged so quickly - allowing rapid iteration
> without respect to merged windows, and doing all the trivial cleanups
> either before or after (but not at the same time as) the feature
> work really does help productivity.

Ah, my mistake, for some reason I thought it did, I guess I had offered
to take it that way if the developers wanted it.

And yes, doing all of the needed cleanups and other changes outside of
the kernel tree should be much much faster, which is why I bet it would
only take 6 months max to get lustre merged "properly" if they really
wanted to do it, by working out-of-tree.

Heck, they already have an out-of-tree repo today, so it's not like
removing the in-kernel version is going to change their normal
development workflow :(

greg k-h

From neilb at suse.com  Wed Jun  6 06:05:18 2018
From: neilb at suse.com (NeilBrown)
Date: Wed, 06 Jun 2018 16:05:18 +1000
Subject: [lustre-devel] [md PATCH 00/11] staging: More lustre cleanup -
 particularly interval-trees
Message-ID: <152826510267.16761.14361003167157833896.stgit@noble>

Hi all,
 here is my next batch of cleanups for lustre.
 Note that this series conflicts with a patch that Greg posted recently
 but that has not yet landed in staging-next (it is in
 staging-testing).
 Up to Greg which gets into staging-next first of course.

 I started with the last 5 patches but noticed interval-tree and
 decided I should deal with that first.  So I added the
 interval-tree clean-up.  That set is more work to review,
 so I put it first.

 Comments welcome,

Thanks,
NeilBrown

---

NeilBrown (11):
      staging: lustre: simplify use of interval-tree.
      staging: lustre: change lock_matches() to return bool.
      staging: lustre: move interval_insert call from ldlm_lock to ldlm_extent
      staging: lustre: convert range_lock to linux interval_trees.
      staging: lustre: convert ldlm extent locks to linux extent-tree
      staging: lustre: remove interval_tree
      staging: lustre: fold lprocfs_call_handler functionality into lnet_debugfs_*
      staging: lustre: obdclass: move linux/linux-foo.c to foo.c
      staging: lustre: discard WIRE_ATTR
      staging: lustre: move ldlm into ptlrpc
      staging: lustre: centralize setting of subdir-ccflags-y


 drivers/staging/lustre/Makefile                    |    3 
 .../staging/lustre/include/linux/libcfs/libcfs.h   |    4 
 .../staging/lustre/include/linux/lnet/socklnd.h    |    8 
 .../lustre/include/uapi/linux/lnet/lnet-types.h    |   28 
 .../lustre/include/uapi/linux/lnet/lnetst.h        |    4 
 drivers/staging/lustre/lnet/klnds/o2iblnd/Makefile |    2 
 .../staging/lustre/lnet/klnds/o2iblnd/o2iblnd.h    |   22 
 drivers/staging/lustre/lnet/klnds/socklnd/Makefile |    2 
 drivers/staging/lustre/lnet/libcfs/Makefile        |    2 
 drivers/staging/lustre/lnet/libcfs/module.c        |   84 -
 drivers/staging/lustre/lnet/lnet/Makefile          |    2 
 drivers/staging/lustre/lnet/lnet/router_proc.c     |   41 
 drivers/staging/lustre/lnet/selftest/Makefile      |    2 
 drivers/staging/lustre/lnet/selftest/rpc.h         |   48 
 drivers/staging/lustre/lustre/fid/Makefile         |    2 
 drivers/staging/lustre/lustre/fld/Makefile         |    2 
 .../staging/lustre/lustre/include/interval_tree.h  |  119 -
 drivers/staging/lustre/lustre/include/lustre_dlm.h |   19 
 drivers/staging/lustre/lustre/ldlm/interval_tree.c |  599 ------
 drivers/staging/lustre/lustre/ldlm/l_lock.c        |   73 -
 drivers/staging/lustre/lustre/ldlm/ldlm_extent.c   |  258 --
 drivers/staging/lustre/lustre/ldlm/ldlm_flock.c    |  486 -----
 .../staging/lustre/lustre/ldlm/ldlm_inodebits.c    |   69 -
 drivers/staging/lustre/lustre/ldlm/ldlm_internal.h |  342 ---
 drivers/staging/lustre/lustre/ldlm/ldlm_lib.c      |  842 --------
 drivers/staging/lustre/lustre/ldlm/ldlm_lock.c     | 2135 --------------------
 drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c    | 1163 -----------
 drivers/staging/lustre/lustre/ldlm/ldlm_plain.c    |   68 -
 drivers/staging/lustre/lustre/ldlm/ldlm_pool.c     | 1013 ---------
 drivers/staging/lustre/lustre/ldlm/ldlm_request.c  | 2033 -------------------
 drivers/staging/lustre/lustre/ldlm/ldlm_resource.c | 1318 ------------
 drivers/staging/lustre/lustre/llite/Makefile       |    2 
 drivers/staging/lustre/lustre/llite/file.c         |    8 
 drivers/staging/lustre/lustre/llite/range_lock.c   |  141 -
 drivers/staging/lustre/lustre/llite/range_lock.h   |   25 
 drivers/staging/lustre/lustre/lmv/Makefile         |    2 
 drivers/staging/lustre/lustre/lov/Makefile         |    2 
 drivers/staging/lustre/lustre/mdc/Makefile         |    2 
 drivers/staging/lustre/lustre/mgc/Makefile         |    2 
 drivers/staging/lustre/lustre/obdclass/Makefile    |    4 
 .../lustre/lustre/obdclass/linux/linux-module.c    |  514 -----
 .../lustre/lustre/obdclass/linux/linux-sysctl.c    |  162 --
 drivers/staging/lustre/lustre/obdclass/module.c    |  514 +++++
 drivers/staging/lustre/lustre/obdclass/sysctl.c    |  162 ++
 drivers/staging/lustre/lustre/obdecho/Makefile     |    2 
 drivers/staging/lustre/lustre/osc/Makefile         |    2 
 drivers/staging/lustre/lustre/ptlrpc/Makefile      |   17 
 drivers/staging/lustre/lustre/ptlrpc/l_lock.c      |   73 +
 drivers/staging/lustre/lustre/ptlrpc/ldlm_extent.c |  206 ++
 drivers/staging/lustre/lustre/ptlrpc/ldlm_flock.c  |  486 +++++
 .../staging/lustre/lustre/ptlrpc/ldlm_inodebits.c  |   69 +
 .../staging/lustre/lustre/ptlrpc/ldlm_internal.h   |  329 +++
 drivers/staging/lustre/lustre/ptlrpc/ldlm_lib.c    |  842 ++++++++
 drivers/staging/lustre/lustre/ptlrpc/ldlm_lock.c   | 2103 ++++++++++++++++++++
 drivers/staging/lustre/lustre/ptlrpc/ldlm_lockd.c  | 1154 +++++++++++
 drivers/staging/lustre/lustre/ptlrpc/ldlm_plain.c  |   68 +
 drivers/staging/lustre/lustre/ptlrpc/ldlm_pool.c   | 1013 +++++++++
 .../staging/lustre/lustre/ptlrpc/ldlm_request.c    | 2033 +++++++++++++++++++
 .../staging/lustre/lustre/ptlrpc/ldlm_resource.c   | 1318 ++++++++++++
 .../staging/lustre/lustre/ptlrpc/ptlrpc_internal.h |    2 
 60 files changed, 10527 insertions(+), 11523 deletions(-)
 delete mode 100644 drivers/staging/lustre/lustre/include/interval_tree.h
 delete mode 100644 drivers/staging/lustre/lustre/ldlm/interval_tree.c
 delete mode 100644 drivers/staging/lustre/lustre/ldlm/l_lock.c
 delete mode 100644 drivers/staging/lustre/lustre/ldlm/ldlm_extent.c
 delete mode 100644 drivers/staging/lustre/lustre/ldlm/ldlm_flock.c
 delete mode 100644 drivers/staging/lustre/lustre/ldlm/ldlm_inodebits.c
 delete mode 100644 drivers/staging/lustre/lustre/ldlm/ldlm_internal.h
 delete mode 100644 drivers/staging/lustre/lustre/ldlm/ldlm_lib.c
 delete mode 100644 drivers/staging/lustre/lustre/ldlm/ldlm_lock.c
 delete mode 100644 drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c
 delete mode 100644 drivers/staging/lustre/lustre/ldlm/ldlm_plain.c
 delete mode 100644 drivers/staging/lustre/lustre/ldlm/ldlm_pool.c
 delete mode 100644 drivers/staging/lustre/lustre/ldlm/ldlm_request.c
 delete mode 100644 drivers/staging/lustre/lustre/ldlm/ldlm_resource.c
 delete mode 100644 drivers/staging/lustre/lustre/obdclass/linux/linux-module.c
 delete mode 100644 drivers/staging/lustre/lustre/obdclass/linux/linux-sysctl.c
 create mode 100644 drivers/staging/lustre/lustre/obdclass/module.c
 create mode 100644 drivers/staging/lustre/lustre/obdclass/sysctl.c
 create mode 100644 drivers/staging/lustre/lustre/ptlrpc/l_lock.c
 create mode 100644 drivers/staging/lustre/lustre/ptlrpc/ldlm_extent.c
 create mode 100644 drivers/staging/lustre/lustre/ptlrpc/ldlm_flock.c
 create mode 100644 drivers/staging/lustre/lustre/ptlrpc/ldlm_inodebits.c
 create mode 100644 drivers/staging/lustre/lustre/ptlrpc/ldlm_internal.h
 create mode 100644 drivers/staging/lustre/lustre/ptlrpc/ldlm_lib.c
 create mode 100644 drivers/staging/lustre/lustre/ptlrpc/ldlm_lock.c
 create mode 100644 drivers/staging/lustre/lustre/ptlrpc/ldlm_lockd.c
 create mode 100644 drivers/staging/lustre/lustre/ptlrpc/ldlm_plain.c
 create mode 100644 drivers/staging/lustre/lustre/ptlrpc/ldlm_pool.c
 create mode 100644 drivers/staging/lustre/lustre/ptlrpc/ldlm_request.c
 create mode 100644 drivers/staging/lustre/lustre/ptlrpc/ldlm_resource.c

--
Signature


From neilb at suse.com  Wed Jun  6 06:05:18 2018
From: neilb at suse.com (NeilBrown)
Date: Wed, 06 Jun 2018 16:05:18 +1000
Subject: [lustre-devel] [PATCH 01/11] staging: lustre: simplify use of
 interval-tree.
In-Reply-To: <152826510267.16761.14361003167157833896.stgit@noble>
References: <152826510267.16761.14361003167157833896.stgit@noble>
Message-ID: <152826511890.16761.16115276596203531205.stgit@noble>

Lustre has a private interval-tree implementation.  This
implementation (inexplicably) refuses to insert an interval if an
identical interval already exists.  It is OK with all sorts of
overlapping intervals, but identical intervals are rejected.

Both users of interval-tree in lustre would be simpler if this was not
the case.  They need to store all intervals, even if some are
identical.

llite/range_lock.c add a rl_next_lock list_head to each lock.
If it cannot insert a new lock because the range is in use, it
attached the new lock to the existing lock using rl_next_lock.
This requires extra code to iterate over the rl_next_lock lists when
iterating over locks, and to update the list when deleting a lock from
the tree.

ldlm_extend allocates a separate ldlm_interval which as a list of
ldlm_locks which share the same interval.  This is linked together
by over-loading the l_sl_policy which, for non-extent locks, is used
for linking together locks with the same policy.
This doesn't only require extra code, but also an extra memory
allocation.

This patch removes all that complexity.
- interval_insert() now never fails.
- consequently rl_next_lock is always empty and
  rl_lock_count is always zero. so they are removed
- every ldlm_lock has linked directly into the
  interval tree, so each has an embedded interval_node
  rather than a pointer to a 'struct ldlm_interval'
- ldlm_interval is now unused, so it is gone as it
  the kmemcache from which they were allocated.
- the various functions for allocating an ldlm_interval
  and attaching to a lock or detaching from a lock
  are also gone.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 .../staging/lustre/lustre/include/interval_tree.h  |    4 +
 drivers/staging/lustre/lustre/include/lustre_dlm.h |   12 ---
 drivers/staging/lustre/lustre/ldlm/interval_tree.c |   13 +--
 drivers/staging/lustre/lustre/ldlm/ldlm_extent.c   |   76 ++------------------
 drivers/staging/lustre/lustre/ldlm/ldlm_internal.h |   17 ----
 drivers/staging/lustre/lustre/ldlm/ldlm_lock.c     |   25 +------
 drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c    |    9 --
 drivers/staging/lustre/lustre/llite/range_lock.c   |   59 +---------------
 drivers/staging/lustre/lustre/llite/range_lock.h   |    8 --
 9 files changed, 17 insertions(+), 206 deletions(-)

diff --git a/drivers/staging/lustre/lustre/include/interval_tree.h b/drivers/staging/lustre/lustre/include/interval_tree.h
index 7d119c1a0469..bcda74fc7875 100644
--- a/drivers/staging/lustre/lustre/include/interval_tree.h
+++ b/drivers/staging/lustre/lustre/include/interval_tree.h
@@ -100,8 +100,8 @@ static inline int interval_set(struct interval_node *node,
 typedef enum interval_iter (*interval_callback_t)(struct interval_node *node,
 						  void *args);
 
-struct interval_node *interval_insert(struct interval_node *node,
-				      struct interval_node **root);
+void interval_insert(struct interval_node *node,
+		     struct interval_node **root);
 void interval_erase(struct interval_node *node, struct interval_node **root);
 
 /*
diff --git a/drivers/staging/lustre/lustre/include/lustre_dlm.h b/drivers/staging/lustre/lustre/include/lustre_dlm.h
index 2c55241258cc..baeb8c63352b 100644
--- a/drivers/staging/lustre/lustre/include/lustre_dlm.h
+++ b/drivers/staging/lustre/lustre/include/lustre_dlm.h
@@ -513,16 +513,6 @@ struct ldlm_glimpse_work {
 /** The ldlm_glimpse_work is allocated on the stack and should not be freed. */
 #define LDLM_GL_WORK_NOFREE 0x1
 
-/** Interval node data for each LDLM_EXTENT lock. */
-struct ldlm_interval {
-	struct interval_node	li_node;  /* node for tree management */
-	struct list_head	li_group; /* the locks which have the same
-					   * policy - group of the policy
-					   */
-};
-
-#define to_ldlm_interval(n) container_of(n, struct ldlm_interval, li_node)
-
 /**
  * Interval tree for extent locks.
  * The interval tree must be accessed under the resource lock.
@@ -631,7 +621,7 @@ struct ldlm_lock {
 	/**
 	 * Tree node for ldlm_extent.
 	 */
-	struct ldlm_interval	*l_tree_node;
+	struct interval_node	l_tree_node;
 	/**
 	 * Requested mode.
 	 * Protected by lr_lock.
diff --git a/drivers/staging/lustre/lustre/ldlm/interval_tree.c b/drivers/staging/lustre/lustre/ldlm/interval_tree.c
index 8df7a4463c21..f5232059d1b1 100644
--- a/drivers/staging/lustre/lustre/ldlm/interval_tree.c
+++ b/drivers/staging/lustre/lustre/ldlm/interval_tree.c
@@ -97,11 +97,6 @@ static inline int extent_overlapped(struct interval_node_extent *e1,
 	return (e1->start <= e2->end) && (e2->start <= e1->end);
 }
 
-static inline int node_equal(struct interval_node *n1, struct interval_node *n2)
-{
-	return extent_equal(&n1->in_extent, &n2->in_extent);
-}
-
 static struct interval_node *interval_first(struct interval_node *node)
 {
 	if (!node)
@@ -299,8 +294,8 @@ static void interval_insert_color(struct interval_node *node,
 	(*root)->in_color = INTERVAL_BLACK;
 }
 
-struct interval_node *interval_insert(struct interval_node *node,
-				      struct interval_node **root)
+void interval_insert(struct interval_node *node,
+		     struct interval_node **root)
 
 {
 	struct interval_node **p, *parent = NULL;
@@ -309,8 +304,6 @@ struct interval_node *interval_insert(struct interval_node *node,
 	p = root;
 	while (*p) {
 		parent = *p;
-		if (node_equal(parent, node))
-			return parent;
 
 		/* max_high field must be updated after each iteration */
 		if (parent->in_max_high < interval_high(node))
@@ -331,8 +324,6 @@ struct interval_node *interval_insert(struct interval_node *node,
 
 	interval_insert_color(node, root);
 	node->in_intree = 1;
-
-	return NULL;
 }
 EXPORT_SYMBOL(interval_insert);
 
diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_extent.c b/drivers/staging/lustre/lustre/ldlm/ldlm_extent.c
index 4da23ade2bb3..2f4c305bb340 100644
--- a/drivers/staging/lustre/lustre/ldlm/ldlm_extent.c
+++ b/drivers/staging/lustre/lustre/ldlm/ldlm_extent.c
@@ -92,55 +92,6 @@ __u64 ldlm_extent_shift_kms(struct ldlm_lock *lock, __u64 old_kms)
 }
 EXPORT_SYMBOL(ldlm_extent_shift_kms);
 
-struct kmem_cache *ldlm_interval_slab;
-
-/* interval tree, for LDLM_EXTENT. */
-static void ldlm_interval_attach(struct ldlm_interval *n, struct ldlm_lock *l)
-{
-	LASSERT(!l->l_tree_node);
-	LASSERT(l->l_resource->lr_type == LDLM_EXTENT);
-
-	list_add_tail(&l->l_sl_policy, &n->li_group);
-	l->l_tree_node = n;
-}
-
-struct ldlm_interval *ldlm_interval_alloc(struct ldlm_lock *lock)
-{
-	struct ldlm_interval *node;
-
-	LASSERT(lock->l_resource->lr_type == LDLM_EXTENT);
-	node = kmem_cache_zalloc(ldlm_interval_slab, GFP_NOFS);
-	if (!node)
-		return NULL;
-
-	INIT_LIST_HEAD(&node->li_group);
-	ldlm_interval_attach(node, lock);
-	return node;
-}
-
-void ldlm_interval_free(struct ldlm_interval *node)
-{
-	if (node) {
-		LASSERT(list_empty(&node->li_group));
-		LASSERT(!interval_is_intree(&node->li_node));
-		kmem_cache_free(ldlm_interval_slab, node);
-	}
-}
-
-struct ldlm_interval *ldlm_interval_detach(struct ldlm_lock *l)
-{
-	struct ldlm_interval *n = l->l_tree_node;
-
-	if (!n)
-		return NULL;
-
-	LASSERT(!list_empty(&n->li_group));
-	l->l_tree_node = NULL;
-	list_del_init(&l->l_sl_policy);
-
-	return list_empty(&n->li_group) ? n : NULL;
-}
-
 static inline int lock_mode_to_index(enum ldlm_mode mode)
 {
 	int index;
@@ -157,16 +108,13 @@ static inline int lock_mode_to_index(enum ldlm_mode mode)
 void ldlm_extent_add_lock(struct ldlm_resource *res,
 			  struct ldlm_lock *lock)
 {
-	struct interval_node *found, **root;
-	struct ldlm_interval *node;
+	struct interval_node **root;
 	struct ldlm_extent *extent;
 	int idx, rc;
 
 	LASSERT(lock->l_granted_mode == lock->l_req_mode);
 
-	node = lock->l_tree_node;
-	LASSERT(node);
-	LASSERT(!interval_is_intree(&node->li_node));
+	LASSERT(!interval_is_intree(&lock->l_tree_node));
 
 	idx = lock_mode_to_index(lock->l_granted_mode);
 	LASSERT(lock->l_granted_mode == 1 << idx);
@@ -174,18 +122,11 @@ void ldlm_extent_add_lock(struct ldlm_resource *res,
 
 	/* node extent initialize */
 	extent = &lock->l_policy_data.l_extent;
-	rc = interval_set(&node->li_node, extent->start, extent->end);
+	rc = interval_set(&lock->l_tree_node, extent->start, extent->end);
 	LASSERT(!rc);
 
 	root = &res->lr_itree[idx].lit_root;
-	found = interval_insert(&node->li_node, root);
-	if (found) { /* The policy group found. */
-		struct ldlm_interval *tmp;
-
-		tmp = ldlm_interval_detach(lock);
-		ldlm_interval_free(tmp);
-		ldlm_interval_attach(to_ldlm_interval(found), lock);
-	}
+	interval_insert(&lock->l_tree_node, root);
 	res->lr_itree[idx].lit_size++;
 
 	/* even though we use interval tree to manage the extent lock, we also
@@ -219,11 +160,10 @@ void ldlm_extent_add_lock(struct ldlm_resource *res,
 void ldlm_extent_unlink_lock(struct ldlm_lock *lock)
 {
 	struct ldlm_resource *res = lock->l_resource;
-	struct ldlm_interval *node = lock->l_tree_node;
 	struct ldlm_interval_tree *tree;
 	int idx;
 
-	if (!node || !interval_is_intree(&node->li_node)) /* duplicate unlink */
+	if (!interval_is_intree(&lock->l_tree_node)) /* duplicate unlink */
 		return;
 
 	idx = lock_mode_to_index(lock->l_granted_mode);
@@ -233,11 +173,7 @@ void ldlm_extent_unlink_lock(struct ldlm_lock *lock)
 	LASSERT(tree->lit_root); /* assure the tree is not null */
 
 	tree->lit_size--;
-	node = ldlm_interval_detach(lock);
-	if (node) {
-		interval_erase(&node->li_node, &tree->lit_root);
-		ldlm_interval_free(node);
-	}
+	interval_erase(&lock->l_tree_node, &tree->lit_root);
 }
 
 void ldlm_extent_policy_wire_to_local(const union ldlm_wire_policy_data *wpolicy,
diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_internal.h b/drivers/staging/lustre/lustre/ldlm/ldlm_internal.h
index bc33ca100620..159de8a59cbb 100644
--- a/drivers/staging/lustre/lustre/ldlm/ldlm_internal.h
+++ b/drivers/staging/lustre/lustre/ldlm/ldlm_internal.h
@@ -189,23 +189,6 @@ __u64 ldlm_pool_get_slv(struct ldlm_pool *pl);
 void ldlm_pool_set_clv(struct ldlm_pool *pl, __u64 clv);
 __u32 ldlm_pool_get_lvf(struct ldlm_pool *pl);
 
-/* interval tree, for LDLM_EXTENT. */
-extern struct kmem_cache *ldlm_interval_slab; /* slab cache for ldlm_interval */
-struct ldlm_interval *ldlm_interval_detach(struct ldlm_lock *l);
-struct ldlm_interval *ldlm_interval_alloc(struct ldlm_lock *lock);
-void ldlm_interval_free(struct ldlm_interval *node);
-/* this function must be called with res lock held */
-static inline struct ldlm_extent *
-ldlm_interval_extent(struct ldlm_interval *node)
-{
-	struct ldlm_lock *lock;
-
-	LASSERT(!list_empty(&node->li_group));
-
-	lock = list_entry(node->li_group.next, struct ldlm_lock, l_sl_policy);
-	return &lock->l_policy_data.l_extent;
-}
-
 int ldlm_init(void);
 void ldlm_exit(void);
 
diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c b/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c
index a644d133063b..13b1b5fdada9 100644
--- a/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c
+++ b/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c
@@ -185,7 +185,6 @@ void ldlm_lock_put(struct ldlm_lock *lock)
 
 		kfree(lock->l_lvb_data);
 
-		ldlm_interval_free(ldlm_interval_detach(lock));
 		lu_ref_fini(&lock->l_reference);
 		OBD_FREE_RCU(lock, sizeof(*lock), &lock->l_handle);
 	}
@@ -1138,17 +1137,10 @@ static int lock_matches(struct ldlm_lock *lock, struct lock_match_data *data)
 
 static enum interval_iter itree_overlap_cb(struct interval_node *in, void *args)
 {
-	struct ldlm_interval *node = to_ldlm_interval(in);
 	struct lock_match_data *data = args;
-	struct ldlm_lock *lock;
-	int rc;
+	struct ldlm_lock *lock = container_of(in, struct ldlm_lock, l_tree_node);
 
-	list_for_each_entry(lock, &node->li_group, l_sl_policy) {
-		rc = lock_matches(lock, data);
-		if (rc == INTERVAL_ITER_STOP)
-			return INTERVAL_ITER_STOP;
-	}
-	return INTERVAL_ITER_CONT;
+	return lock_matches(lock, data);
 }
 
 /**
@@ -1564,15 +1556,6 @@ struct ldlm_lock *ldlm_lock_create(struct ldlm_namespace *ns,
 		lock->l_glimpse_ast = cbs->lcs_glimpse;
 	}
 
-	lock->l_tree_node = NULL;
-	/* if this is the extent lock, allocate the interval tree node */
-	if (type == LDLM_EXTENT) {
-		if (!ldlm_interval_alloc(lock)) {
-			rc = -ENOMEM;
-			goto out;
-		}
-	}
-
 	if (lvb_len) {
 		lock->l_lvb_len = lvb_len;
 		lock->l_lvb_data = kzalloc(lvb_len, GFP_NOFS);
@@ -1625,10 +1608,6 @@ enum ldlm_error ldlm_lock_enqueue(struct ldlm_namespace *ns,
 
 	ldlm_resource_unlink_lock(lock);
 
-	/* Cannot happen unless on the server */
-	if (res->lr_type == LDLM_EXTENT && !lock->l_tree_node)
-		LBUG();
-
 	/* Some flags from the enqueue want to make it into the AST, via the
 	 * lock's l_flags.
 	 */
diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c b/drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c
index 5963e90d0938..f410ef6c02ef 100644
--- a/drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c
+++ b/drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c
@@ -1134,14 +1134,6 @@ int ldlm_init(void)
 		return -ENOMEM;
 	}
 
-	ldlm_interval_slab = kmem_cache_create("interval_node",
-					       sizeof(struct ldlm_interval),
-					       0, SLAB_HWCACHE_ALIGN, NULL);
-	if (!ldlm_interval_slab) {
-		kmem_cache_destroy(ldlm_resource_slab);
-		kmem_cache_destroy(ldlm_lock_slab);
-		return -ENOMEM;
-	}
 #if LUSTRE_TRACKS_LOCK_EXP_REFS
 	class_export_dump_hook = ldlm_dump_export_locks;
 #endif
@@ -1159,5 +1151,4 @@ void ldlm_exit(void)
 	 */
 	synchronize_rcu();
 	kmem_cache_destroy(ldlm_lock_slab);
-	kmem_cache_destroy(ldlm_interval_slab);
 }
diff --git a/drivers/staging/lustre/lustre/llite/range_lock.c b/drivers/staging/lustre/lustre/llite/range_lock.c
index 008a8874118d..eaa23f4c414e 100644
--- a/drivers/staging/lustre/lustre/llite/range_lock.c
+++ b/drivers/staging/lustre/lustre/llite/range_lock.c
@@ -74,19 +74,12 @@ int range_lock_init(struct range_lock *lock, __u64 start, __u64 end)
 	if (rc)
 		return rc;
 
-	INIT_LIST_HEAD(&lock->rl_next_lock);
 	lock->rl_task = NULL;
-	lock->rl_lock_count = 0;
 	lock->rl_blocking_ranges = 0;
 	lock->rl_sequence = 0;
 	return rc;
 }
 
-static inline struct range_lock *next_lock(struct range_lock *lock)
-{
-	return list_entry(lock->rl_next_lock.next, typeof(*lock), rl_next_lock);
-}
-
 /**
  * Helper function of range_unlock()
  *
@@ -102,14 +95,7 @@ static enum interval_iter range_unlock_cb(struct interval_node *node, void *arg)
 {
 	struct range_lock *lock = arg;
 	struct range_lock *overlap = node2rangelock(node);
-	struct range_lock *iter;
 
-	list_for_each_entry(iter, &overlap->rl_next_lock, rl_next_lock) {
-		if (iter->rl_sequence > lock->rl_sequence) {
-			--iter->rl_blocking_ranges;
-			LASSERT(iter->rl_blocking_ranges > 0);
-		}
-	}
 	if (overlap->rl_sequence > lock->rl_sequence) {
 		--overlap->rl_blocking_ranges;
 		if (overlap->rl_blocking_ranges == 0)
@@ -131,32 +117,8 @@ static enum interval_iter range_unlock_cb(struct interval_node *node, void *arg)
 void range_unlock(struct range_lock_tree *tree, struct range_lock *lock)
 {
 	spin_lock(&tree->rlt_lock);
-	if (!list_empty(&lock->rl_next_lock)) {
-		struct range_lock *next;
-
-		if (interval_is_intree(&lock->rl_node)) { /* first lock */
-			/* Insert the next same range lock into the tree */
-			next = next_lock(lock);
-			next->rl_lock_count = lock->rl_lock_count - 1;
-			interval_erase(&lock->rl_node, &tree->rlt_root);
-			interval_insert(&next->rl_node, &tree->rlt_root);
-		} else {
-			/* find the first lock in tree */
-			list_for_each_entry(next, &lock->rl_next_lock,
-					    rl_next_lock) {
-				if (!interval_is_intree(&next->rl_node))
-					continue;
-
-				LASSERT(next->rl_lock_count > 0);
-				next->rl_lock_count--;
-				break;
-			}
-		}
-		list_del_init(&lock->rl_next_lock);
-	} else {
-		LASSERT(interval_is_intree(&lock->rl_node));
-		interval_erase(&lock->rl_node, &tree->rlt_root);
-	}
+	LASSERT(interval_is_intree(&lock->rl_node));
+	interval_erase(&lock->rl_node, &tree->rlt_root);
 
 	interval_search(tree->rlt_root, &lock->rl_node.in_extent,
 			range_unlock_cb, lock);
@@ -177,9 +139,8 @@ void range_unlock(struct range_lock_tree *tree, struct range_lock *lock)
 static enum interval_iter range_lock_cb(struct interval_node *node, void *arg)
 {
 	struct range_lock *lock = arg;
-	struct range_lock *overlap = node2rangelock(node);
 
-	lock->rl_blocking_ranges += overlap->rl_lock_count + 1;
+	lock->rl_blocking_ranges++;
 	return INTERVAL_ITER_CONT;
 }
 
@@ -198,7 +159,6 @@ static enum interval_iter range_lock_cb(struct interval_node *node, void *arg)
  */
 int range_lock(struct range_lock_tree *tree, struct range_lock *lock)
 {
-	struct interval_node *node;
 	int rc = 0;
 
 	spin_lock(&tree->rlt_lock);
@@ -208,18 +168,7 @@ int range_lock(struct range_lock_tree *tree, struct range_lock *lock)
 	 */
 	interval_search(tree->rlt_root, &lock->rl_node.in_extent,
 			range_lock_cb, lock);
-	/*
-	 * Insert to the tree if I am unique, otherwise I've been linked to
-	 * the rl_next_lock of another lock which has the same range as mine
-	 * in range_lock_cb().
-	 */
-	node = interval_insert(&lock->rl_node, &tree->rlt_root);
-	if (node) {
-		struct range_lock *tmp = node2rangelock(node);
-
-		list_add_tail(&lock->rl_next_lock, &tmp->rl_next_lock);
-		tmp->rl_lock_count++;
-	}
+	interval_insert(&lock->rl_node, &tree->rlt_root);
 	lock->rl_sequence = ++tree->rlt_sequence;
 
 	while (lock->rl_blocking_ranges > 0) {
diff --git a/drivers/staging/lustre/lustre/llite/range_lock.h b/drivers/staging/lustre/lustre/llite/range_lock.h
index 9ebac09160f2..10ef1a995d26 100644
--- a/drivers/staging/lustre/lustre/llite/range_lock.h
+++ b/drivers/staging/lustre/lustre/llite/range_lock.h
@@ -46,14 +46,6 @@ struct range_lock {
 	 * Process to enqueue this lock.
 	 */
 	struct task_struct	*rl_task;
-	/**
-	 * List of locks with the same range.
-	 */
-	struct list_head	rl_next_lock;
-	/**
-	 * Number of locks in the list rl_next_lock
-	 */
-	unsigned int		rl_lock_count;
 	/**
 	 * Number of ranges which are blocking acquisition of the lock
 	 */



From neilb at suse.com  Wed Jun  6 06:05:18 2018
From: neilb at suse.com (NeilBrown)
Date: Wed, 06 Jun 2018 16:05:18 +1000
Subject: [lustre-devel] [PATCH 02/11] staging: lustre: change lock_matches()
 to return bool.
In-Reply-To: <152826510267.16761.14361003167157833896.stgit@noble>
References: <152826510267.16761.14361003167157833896.stgit@noble>
Message-ID: <152826511895.16761.11859115039331359491.stgit@noble>

The name of the function lock_matches() sounds like it
performs a test (it does) and so should return a bool.
Returning a bool gives a slight code simplification (in
search_queue) and more simplification in future patches.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 drivers/staging/lustre/lustre/ldlm/ldlm_lock.c |   37 ++++++++++++------------
 1 file changed, 18 insertions(+), 19 deletions(-)

diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c b/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c
index 13b1b5fdada9..034935e06393 100644
--- a/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c
+++ b/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c
@@ -1053,20 +1053,20 @@ struct lock_match_data {
  * \param lock	test-against this lock
  * \param data	parameters
  */
-static int lock_matches(struct ldlm_lock *lock, struct lock_match_data *data)
+static bool lock_matches(struct ldlm_lock *lock, struct lock_match_data *data)
 {
 	union ldlm_policy_data *lpol = &lock->l_policy_data;
 	enum ldlm_mode match;
 
 	if (lock == data->lmd_old)
-		return INTERVAL_ITER_STOP;
+		return true;
 
 	/*
 	 * Check if this lock can be matched.
 	 * Used by LU-2919(exclusive open) for open lease lock
 	 */
 	if (ldlm_is_excl(lock))
-		return INTERVAL_ITER_CONT;
+		return false;
 
 	/*
 	 * llite sometimes wants to match locks that will be
@@ -1078,26 +1078,26 @@ static int lock_matches(struct ldlm_lock *lock, struct lock_match_data *data)
 	 */
 	if (ldlm_is_cbpending(lock) &&
 	    !(data->lmd_flags & LDLM_FL_CBPENDING))
-		return INTERVAL_ITER_CONT;
+		return false;
 
 	if (!data->lmd_unref && ldlm_is_cbpending(lock) &&
 	    !lock->l_readers && !lock->l_writers)
-		return INTERVAL_ITER_CONT;
+		return false;
 
 	if (!(lock->l_req_mode & *data->lmd_mode))
-		return INTERVAL_ITER_CONT;
+		return false;
 	match = lock->l_req_mode;
 
 	switch (lock->l_resource->lr_type) {
 	case LDLM_EXTENT:
 		if (lpol->l_extent.start > data->lmd_policy->l_extent.start ||
 		    lpol->l_extent.end < data->lmd_policy->l_extent.end)
-			return INTERVAL_ITER_CONT;
+			return false;
 
 		if (unlikely(match == LCK_GROUP) &&
 		    data->lmd_policy->l_extent.gid != LDLM_GID_ANY &&
 		    lpol->l_extent.gid != data->lmd_policy->l_extent.gid)
-			return INTERVAL_ITER_CONT;
+			return false;
 		break;
 	case LDLM_IBITS:
 		/*
@@ -1107,7 +1107,7 @@ static int lock_matches(struct ldlm_lock *lock, struct lock_match_data *data)
 		if ((lpol->l_inodebits.bits &
 		     data->lmd_policy->l_inodebits.bits) !=
 		    data->lmd_policy->l_inodebits.bits)
-			return INTERVAL_ITER_CONT;
+			return false;
 		break;
 	default:
 		break;
@@ -1117,10 +1117,10 @@ static int lock_matches(struct ldlm_lock *lock, struct lock_match_data *data)
 	 * of bits.
 	 */
 	if (!data->lmd_unref && LDLM_HAVE_MASK(lock, GONE))
-		return INTERVAL_ITER_CONT;
+		return false;
 
 	if (!equi(data->lmd_flags & LDLM_FL_LOCAL_ONLY, ldlm_is_local(lock)))
-		return INTERVAL_ITER_CONT;
+		return false;
 
 	if (data->lmd_flags & LDLM_FL_TEST_LOCK) {
 		LDLM_LOCK_GET(lock);
@@ -1132,15 +1132,17 @@ static int lock_matches(struct ldlm_lock *lock, struct lock_match_data *data)
 	*data->lmd_mode = match;
 	data->lmd_lock = lock;
 
-	return INTERVAL_ITER_STOP;
+	return true;
 }
 
 static enum interval_iter itree_overlap_cb(struct interval_node *in, void *args)
 {
 	struct lock_match_data *data = args;
-	struct ldlm_lock *lock = container_of(in, struct ldlm_lock, l_tree_node);
+	struct ldlm_lock *lock = container_of(in, struct ldlm_lock,
+					      l_tree_node);
 
-	return lock_matches(lock, data);
+	return lock_matches(lock, data) ?
+		INTERVAL_ITER_STOP : INTERVAL_ITER_CONT;
 }
 
 /**
@@ -1187,13 +1189,10 @@ static struct ldlm_lock *search_queue(struct list_head *queue,
 				      struct lock_match_data *data)
 {
 	struct ldlm_lock *lock;
-	int rc;
 
-	list_for_each_entry(lock, queue, l_res_link) {
-		rc = lock_matches(lock, data);
-		if (rc == INTERVAL_ITER_STOP)
+	list_for_each_entry(lock, queue, l_res_link)
+		if (lock_matches(lock, data))
 			return data->lmd_lock;
-	}
 	return NULL;
 }
 



From neilb at suse.com  Wed Jun  6 06:05:19 2018
From: neilb at suse.com (NeilBrown)
Date: Wed, 06 Jun 2018 16:05:19 +1000
Subject: [lustre-devel] [PATCH 03/11] staging: lustre: move interval_insert
 call from ldlm_lock to ldlm_extent
In-Reply-To: <152826510267.16761.14361003167157833896.stgit@noble>
References: <152826510267.16761.14361003167157833896.stgit@noble>
Message-ID: <152826511899.16761.11963878471070889487.stgit@noble>

Moving this call results in all interval-tree handling code
being in the one file.  This will simplify conversion to
use Linux interval trees.

The addition of 'struct cb' is a little ugly, but will be gone
is a subsequent patch.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 drivers/staging/lustre/lustre/ldlm/ldlm_extent.c   |   27 ++++++++++++++++++++
 drivers/staging/lustre/lustre/ldlm/ldlm_internal.h |    4 +++
 drivers/staging/lustre/lustre/ldlm/ldlm_lock.c     |   17 +++----------
 3 files changed, 35 insertions(+), 13 deletions(-)

diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_extent.c b/drivers/staging/lustre/lustre/ldlm/ldlm_extent.c
index 2f4c305bb340..eb1a9077a514 100644
--- a/drivers/staging/lustre/lustre/ldlm/ldlm_extent.c
+++ b/drivers/staging/lustre/lustre/ldlm/ldlm_extent.c
@@ -192,3 +192,30 @@ void ldlm_extent_policy_local_to_wire(const union ldlm_policy_data *lpolicy,
 	wpolicy->l_extent.end = lpolicy->l_extent.end;
 	wpolicy->l_extent.gid = lpolicy->l_extent.gid;
 }
+
+struct cb {
+	void *arg;
+	bool (*found)(struct ldlm_lock *lock, void *arg);
+};
+
+static enum interval_iter itree_overlap_cb(struct interval_node *in, void *arg)
+{
+	struct cb *cb = arg;
+	struct ldlm_lock *lock = container_of(in, struct ldlm_lock,
+					      l_tree_node);
+
+	return cb->found(lock, cb->arg) ?
+		INTERVAL_ITER_STOP : INTERVAL_ITER_CONT;
+}
+
+void ldlm_extent_search(struct interval_node *root,
+			struct interval_node_extent *ext,
+			bool (*matches)(struct ldlm_lock *lock, void *data),
+			void *data)
+{
+	struct cb cb = {
+		.arg = data,
+		.found = matches,
+	};
+	interval_search(root, ext, itree_overlap_cb, &cb);
+}
diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_internal.h b/drivers/staging/lustre/lustre/ldlm/ldlm_internal.h
index 159de8a59cbb..756fa3d9db3c 100644
--- a/drivers/staging/lustre/lustre/ldlm/ldlm_internal.h
+++ b/drivers/staging/lustre/lustre/ldlm/ldlm_internal.h
@@ -169,6 +169,10 @@ extern struct kmem_cache *ldlm_lock_slab;
 /* ldlm_extent.c */
 void ldlm_extent_add_lock(struct ldlm_resource *res, struct ldlm_lock *lock);
 void ldlm_extent_unlink_lock(struct ldlm_lock *lock);
+void ldlm_extent_search(struct interval_node *root,
+			struct interval_node_extent *ext,
+			bool (*matches)(struct ldlm_lock *lock, void *data),
+			void *data);
 
 /* l_lock.c */
 void l_check_ns_lock(struct ldlm_namespace *ns);
diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c b/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c
index 034935e06393..4213fe047073 100644
--- a/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c
+++ b/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c
@@ -1053,8 +1053,9 @@ struct lock_match_data {
  * \param lock	test-against this lock
  * \param data	parameters
  */
-static bool lock_matches(struct ldlm_lock *lock, struct lock_match_data *data)
+static bool lock_matches(struct ldlm_lock *lock, void *vdata)
 {
+	struct lock_match_data *data = vdata;
 	union ldlm_policy_data *lpol = &lock->l_policy_data;
 	enum ldlm_mode match;
 
@@ -1135,16 +1136,6 @@ static bool lock_matches(struct ldlm_lock *lock, struct lock_match_data *data)
 	return true;
 }
 
-static enum interval_iter itree_overlap_cb(struct interval_node *in, void *args)
-{
-	struct lock_match_data *data = args;
-	struct ldlm_lock *lock = container_of(in, struct ldlm_lock,
-					      l_tree_node);
-
-	return lock_matches(lock, data) ?
-		INTERVAL_ITER_STOP : INTERVAL_ITER_CONT;
-}
-
 /**
  * Search for a lock with given parameters in interval trees.
  *
@@ -1171,8 +1162,8 @@ static struct ldlm_lock *search_itree(struct ldlm_resource *res,
 		if (!(tree->lit_mode & *data->lmd_mode))
 			continue;
 
-		interval_search(tree->lit_root, &ext,
-				itree_overlap_cb, data);
+		ldlm_extent_search(tree->lit_root, &ext,
+				   lock_matches, data);
 	}
 	return data->lmd_lock;
 }



From neilb at suse.com  Wed Jun  6 06:05:19 2018
From: neilb at suse.com (NeilBrown)
Date: Wed, 06 Jun 2018 16:05:19 +1000
Subject: [lustre-devel] [PATCH 04/11] staging: lustre: convert range_lock to
 linux interval_trees.
In-Reply-To: <152826510267.16761.14361003167157833896.stgit@noble>
References: <152826510267.16761.14361003167157833896.stgit@noble>
Message-ID: <152826511902.16761.12169723574817283239.stgit@noble>

Linux has a fully-generic interval tree implementation
which can be tailored to different use cases.
Use it for range_lock rather than the lustre version.
This allows us to get rid of some call-backs and generally
simplifies the code.

We cannot use the pre-built version in lib/interval_tree.c
as we need 64bit endpoints and it only provides "unsigned long".

Signed-off-by: NeilBrown <neilb at suse.com>
---
 drivers/staging/lustre/lustre/llite/file.c       |    8 +-
 drivers/staging/lustre/lustre/llite/range_lock.c |   94 ++++++++--------------
 drivers/staging/lustre/lustre/llite/range_lock.h |   17 ++--
 3 files changed, 47 insertions(+), 72 deletions(-)

diff --git a/drivers/staging/lustre/lustre/llite/file.c b/drivers/staging/lustre/lustre/llite/file.c
index 02295931883b..e888ed6e74bc 100644
--- a/drivers/staging/lustre/lustre/llite/file.c
+++ b/drivers/staging/lustre/lustre/llite/file.c
@@ -1086,8 +1086,8 @@ ll_file_io_generic(const struct lu_env *env, struct vvp_io_args *args,
 		     (iot == CIT_READ && (file->f_flags & O_DIRECT))) &&
 		    !(vio->vui_fd->fd_flags & LL_FILE_GROUP_LOCKED)) {
 			CDEBUG(D_VFSTRACE, "Range lock [%llu, %llu]\n",
-			       range.rl_node.in_extent.start,
-			       range.rl_node.in_extent.end);
+			       range.rl_start,
+			       range.rl_last);
 			rc = range_lock(&lli->lli_write_tree, &range);
 			if (rc < 0)
 				goto out;
@@ -1099,8 +1099,8 @@ ll_file_io_generic(const struct lu_env *env, struct vvp_io_args *args,
 		ll_cl_remove(file, env);
 		if (range_locked) {
 			CDEBUG(D_VFSTRACE, "Range unlock [%llu, %llu]\n",
-			       range.rl_node.in_extent.start,
-			       range.rl_node.in_extent.end);
+			       range.rl_start,
+			       range.rl_last);
 			range_unlock(&lli->lli_write_tree, &range);
 		}
 	} else {
diff --git a/drivers/staging/lustre/lustre/llite/range_lock.c b/drivers/staging/lustre/lustre/llite/range_lock.c
index eaa23f4c414e..acdb0dc00a89 100644
--- a/drivers/staging/lustre/lustre/llite/range_lock.c
+++ b/drivers/staging/lustre/lustre/llite/range_lock.c
@@ -37,7 +37,13 @@
 #include "range_lock.h"
 #include <uapi/linux/lustre/lustre_idl.h>
 #include <linux/libcfs/libcfs.h>
+#include <linux/interval_tree_generic.h>
 
+#define START(node) ((node)->rl_start)
+#define LAST(node)  ((node)->rl_last)
+
+INTERVAL_TREE_DEFINE(struct range_lock, rl_rb, __u64, __subtree_last,
+		     START, LAST, static, range);
 /**
  * Initialize a range lock tree
  *
@@ -48,7 +54,7 @@
  */
 void range_lock_tree_init(struct range_lock_tree *tree)
 {
-	tree->rlt_root = NULL;
+	tree->rlt_root = RB_ROOT_CACHED;
 	tree->rlt_sequence = 0;
 	spin_lock_init(&tree->rlt_lock);
 }
@@ -65,43 +71,19 @@ void range_lock_tree_init(struct range_lock_tree *tree)
  */
 int range_lock_init(struct range_lock *lock, __u64 start, __u64 end)
 {
-	int rc;
+	RB_CLEAR_NODE(&lock->rl_rb);
 
-	memset(&lock->rl_node, 0, sizeof(lock->rl_node));
 	if (end != LUSTRE_EOF)
 		end >>= PAGE_SHIFT;
-	rc = interval_set(&lock->rl_node, start >> PAGE_SHIFT, end);
-	if (rc)
-		return rc;
+	lock->rl_start = start >> PAGE_SHIFT;
+	lock->rl_last = end;
+	if (lock->rl_start > lock->rl_last)
+		return -ERANGE;
 
 	lock->rl_task = NULL;
 	lock->rl_blocking_ranges = 0;
 	lock->rl_sequence = 0;
-	return rc;
-}
-
-/**
- * Helper function of range_unlock()
- *
- * \param node [in]	a range lock found overlapped during interval node
- *			search
- * \param arg [in]	the range lock to be tested
- *
- * \retval INTERVAL_ITER_CONT	indicate to continue the search for next
- *				overlapping range node
- * \retval INTERVAL_ITER_STOP	indicate to stop the search
- */
-static enum interval_iter range_unlock_cb(struct interval_node *node, void *arg)
-{
-	struct range_lock *lock = arg;
-	struct range_lock *overlap = node2rangelock(node);
-
-	if (overlap->rl_sequence > lock->rl_sequence) {
-		--overlap->rl_blocking_ranges;
-		if (overlap->rl_blocking_ranges == 0)
-			wake_up_process(overlap->rl_task);
-	}
-	return INTERVAL_ITER_CONT;
+	return 0;
 }
 
 /**
@@ -112,36 +94,27 @@ static enum interval_iter range_unlock_cb(struct interval_node *node, void *arg)
  *
  * If this lock has been granted, relase it; if not, just delete it from
  * the tree or the same region lock list. Wake up those locks only blocked
- * by this lock through range_unlock_cb().
+ * by this lock.
  */
 void range_unlock(struct range_lock_tree *tree, struct range_lock *lock)
 {
-	spin_lock(&tree->rlt_lock);
-	LASSERT(interval_is_intree(&lock->rl_node));
-	interval_erase(&lock->rl_node, &tree->rlt_root);
+	struct range_lock *overlap;
 
-	interval_search(tree->rlt_root, &lock->rl_node.in_extent,
-			range_unlock_cb, lock);
-	spin_unlock(&tree->rlt_lock);
-}
+	spin_lock(&tree->rlt_lock);
+	LASSERT(!RB_EMPTY_NODE(&lock->rl_rb));
+	range_remove(lock, &tree->rlt_root);
 
-/**
- * Helper function of range_lock()
- *
- * \param node [in]	a range lock found overlapped during interval node
- *			search
- * \param arg [in]	the range lock to be tested
- *
- * \retval INTERVAL_ITER_CONT	indicate to continue the search for next
- *				overlapping range node
- * \retval INTERVAL_ITER_STOP	indicate to stop the search
- */
-static enum interval_iter range_lock_cb(struct interval_node *node, void *arg)
-{
-	struct range_lock *lock = arg;
+	for (overlap = range_iter_first(&tree->rlt_root,
+					lock->rl_start, lock->rl_last);
+	     overlap;
+	     overlap = range_iter_next(overlap, lock->rl_start, lock->rl_last))
+		if (overlap->rl_sequence > lock->rl_sequence) {
+			--overlap->rl_blocking_ranges;
+			if (overlap->rl_blocking_ranges == 0)
+				wake_up_process(overlap->rl_task);
+		}
 
-	lock->rl_blocking_ranges++;
-	return INTERVAL_ITER_CONT;
+	spin_unlock(&tree->rlt_lock);
 }
 
 /**
@@ -160,15 +133,20 @@ static enum interval_iter range_lock_cb(struct interval_node *node, void *arg)
 int range_lock(struct range_lock_tree *tree, struct range_lock *lock)
 {
 	int rc = 0;
+	struct range_lock *it;
 
 	spin_lock(&tree->rlt_lock);
 	/*
 	 * We need to check for all conflicting intervals
 	 * already in the tree.
 	 */
-	interval_search(tree->rlt_root, &lock->rl_node.in_extent,
-			range_lock_cb, lock);
-	interval_insert(&lock->rl_node, &tree->rlt_root);
+	for (it = range_iter_first(&tree->rlt_root,
+				   lock->rl_start, lock->rl_last);
+	     it;
+	     it = range_iter_next(it, lock->rl_start, lock->rl_last))
+		lock->rl_blocking_ranges++;
+
+	range_insert(lock, &tree->rlt_root);
 	lock->rl_sequence = ++tree->rlt_sequence;
 
 	while (lock->rl_blocking_ranges > 0) {
diff --git a/drivers/staging/lustre/lustre/llite/range_lock.h b/drivers/staging/lustre/lustre/llite/range_lock.h
index 10ef1a995d26..2a0704d21481 100644
--- a/drivers/staging/lustre/lustre/llite/range_lock.h
+++ b/drivers/staging/lustre/lustre/llite/range_lock.h
@@ -38,10 +38,12 @@
 #define _RANGE_LOCK_H
 
 #include <linux/spinlock.h>
-#include <interval_tree.h>
+#include <linux/rbtree.h>
 
 struct range_lock {
-	struct interval_node	rl_node;
+	struct rb_node		rl_rb;
+	__u64			rl_start, rl_last;
+	__u64			__subtree_last;
 	/**
 	 * Process to enqueue this lock.
 	 */
@@ -57,15 +59,10 @@ struct range_lock {
 	__u64			rl_sequence;
 };
 
-static inline struct range_lock *node2rangelock(const struct interval_node *n)
-{
-	return container_of(n, struct range_lock, rl_node);
-}
-
 struct range_lock_tree {
-	struct interval_node	*rlt_root;
-	spinlock_t		 rlt_lock;	/* protect range lock tree */
-	__u64			 rlt_sequence;
+	struct rb_root_cached	rlt_root;
+	spinlock_t		rlt_lock;	/* protect range lock tree */
+	__u64			rlt_sequence;
 };
 
 void range_lock_tree_init(struct range_lock_tree *tree);



From neilb at suse.com  Wed Jun  6 06:05:19 2018
From: neilb at suse.com (NeilBrown)
Date: Wed, 06 Jun 2018 16:05:19 +1000
Subject: [lustre-devel] [PATCH 05/11] staging: lustre: convert ldlm extent
 locks to linux extent-tree
In-Reply-To: <152826510267.16761.14361003167157833896.stgit@noble>
References: <152826510267.16761.14361003167157833896.stgit@noble>
Message-ID: <152826511905.16761.134377818890309985.stgit@noble>

As linux has a fully customizable extent tree implementation,
use that instead of the one in lustre.
This has a small benefit in that the start/end only need to
be stored in the ldlm_lock once instead of twice - in both
l_policy_data.l_exent and l_tree_node.
It also makes the code simpler.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 drivers/staging/lustre/lustre/include/lustre_dlm.h |    9 ++-
 drivers/staging/lustre/lustre/ldlm/ldlm_extent.c   |   61 ++++++++------------
 drivers/staging/lustre/lustre/ldlm/ldlm_internal.h |    4 +
 drivers/staging/lustre/lustre/ldlm/ldlm_lock.c     |   11 ++--
 drivers/staging/lustre/lustre/ldlm/ldlm_resource.c |    2 -
 5 files changed, 36 insertions(+), 51 deletions(-)

diff --git a/drivers/staging/lustre/lustre/include/lustre_dlm.h b/drivers/staging/lustre/lustre/include/lustre_dlm.h
index baeb8c63352b..4f196c27b76b 100644
--- a/drivers/staging/lustre/lustre/include/lustre_dlm.h
+++ b/drivers/staging/lustre/lustre/include/lustre_dlm.h
@@ -49,7 +49,6 @@
 #include <lustre_net.h>
 #include <lustre_import.h>
 #include <lustre_handles.h>
-#include <interval_tree.h>	/* for interval_node{}, ldlm_extent */
 #include <lu_ref.h>
 
 #include "lustre_dlm_flags.h"
@@ -523,7 +522,7 @@ struct ldlm_interval_tree {
 	/** Tree size. */
 	int			lit_size;
 	enum ldlm_mode		lit_mode;  /* lock mode */
-	struct interval_node	*lit_root; /* actual ldlm_interval */
+	struct rb_root_cached	lit_root; /* actual interval tree */
 };
 
 /** Whether to track references to exports by LDLM locks. */
@@ -619,9 +618,11 @@ struct ldlm_lock {
 	 */
 	struct list_head		l_res_link;
 	/**
-	 * Tree node for ldlm_extent.
+	 * Interval-tree node for ldlm_extent.
 	 */
-	struct interval_node	l_tree_node;
+	struct rb_node		l_rb;
+	__u64			__subtree_last;
+
 	/**
 	 * Requested mode.
 	 * Protected by lr_lock.
diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_extent.c b/drivers/staging/lustre/lustre/ldlm/ldlm_extent.c
index eb1a9077a514..225c023b0bba 100644
--- a/drivers/staging/lustre/lustre/ldlm/ldlm_extent.c
+++ b/drivers/staging/lustre/lustre/ldlm/ldlm_extent.c
@@ -53,6 +53,12 @@
 #include <obd_class.h>
 #include <lustre_lib.h>
 #include "ldlm_internal.h"
+#include <linux/interval_tree_generic.h>
+
+#define START(node) ((node)->l_policy_data.l_extent.start)
+#define LAST(node) ((node)->l_policy_data.l_extent.end)
+INTERVAL_TREE_DEFINE(struct ldlm_lock, l_rb, __u64, __subtree_last,
+		     START, LAST, static, extent);
 
 /* When a lock is cancelled by a client, the KMS may undergo change if this
  * is the "highest lock".  This function returns the new KMS value.
@@ -108,26 +114,20 @@ static inline int lock_mode_to_index(enum ldlm_mode mode)
 void ldlm_extent_add_lock(struct ldlm_resource *res,
 			  struct ldlm_lock *lock)
 {
-	struct interval_node **root;
-	struct ldlm_extent *extent;
-	int idx, rc;
+	struct ldlm_interval_tree *tree;
+	int idx;
 
 	LASSERT(lock->l_granted_mode == lock->l_req_mode);
 
-	LASSERT(!interval_is_intree(&lock->l_tree_node));
+	LASSERT(RB_EMPTY_NODE(&lock->l_rb));
 
 	idx = lock_mode_to_index(lock->l_granted_mode);
 	LASSERT(lock->l_granted_mode == 1 << idx);
 	LASSERT(lock->l_granted_mode == res->lr_itree[idx].lit_mode);
 
-	/* node extent initialize */
-	extent = &lock->l_policy_data.l_extent;
-	rc = interval_set(&lock->l_tree_node, extent->start, extent->end);
-	LASSERT(!rc);
-
-	root = &res->lr_itree[idx].lit_root;
-	interval_insert(&lock->l_tree_node, root);
-	res->lr_itree[idx].lit_size++;
+	tree = &res->lr_itree[idx];
+	extent_insert(lock, &tree->lit_root);
+	tree->lit_size++;
 
 	/* even though we use interval tree to manage the extent lock, we also
 	 * add the locks into grant list, for debug purpose, ..
@@ -163,17 +163,15 @@ void ldlm_extent_unlink_lock(struct ldlm_lock *lock)
 	struct ldlm_interval_tree *tree;
 	int idx;
 
-	if (!interval_is_intree(&lock->l_tree_node)) /* duplicate unlink */
+	if (RB_EMPTY_NODE(&lock->l_rb)) /* duplicate unlink */
 		return;
 
 	idx = lock_mode_to_index(lock->l_granted_mode);
 	LASSERT(lock->l_granted_mode == 1 << idx);
 	tree = &res->lr_itree[idx];
 
-	LASSERT(tree->lit_root); /* assure the tree is not null */
-
 	tree->lit_size--;
-	interval_erase(&lock->l_tree_node, &tree->lit_root);
+	extent_remove(lock, &tree->lit_root);
 }
 
 void ldlm_extent_policy_wire_to_local(const union ldlm_wire_policy_data *wpolicy,
@@ -193,29 +191,16 @@ void ldlm_extent_policy_local_to_wire(const union ldlm_policy_data *lpolicy,
 	wpolicy->l_extent.gid = lpolicy->l_extent.gid;
 }
 
-struct cb {
-	void *arg;
-	bool (*found)(struct ldlm_lock *lock, void *arg);
-};
-
-static enum interval_iter itree_overlap_cb(struct interval_node *in, void *arg)
-{
-	struct cb *cb = arg;
-	struct ldlm_lock *lock = container_of(in, struct ldlm_lock,
-					      l_tree_node);
-
-	return cb->found(lock, cb->arg) ?
-		INTERVAL_ITER_STOP : INTERVAL_ITER_CONT;
-}
-
-void ldlm_extent_search(struct interval_node *root,
-			struct interval_node_extent *ext,
+void ldlm_extent_search(struct rb_root_cached *root,
+			__u64 start, __u64 end,
 			bool (*matches)(struct ldlm_lock *lock, void *data),
 			void *data)
 {
-	struct cb cb = {
-		.arg = data,
-		.found = matches,
-	};
-	interval_search(root, ext, itree_overlap_cb, &cb);
+	struct ldlm_lock *lock;
+
+	for (lock = extent_iter_first(root, start, end);
+	     lock;
+	     lock = extent_iter_next(lock, start, end))
+		if (matches(lock, data))
+			break;
 }
diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_internal.h b/drivers/staging/lustre/lustre/ldlm/ldlm_internal.h
index 756fa3d9db3c..60a15b963c8a 100644
--- a/drivers/staging/lustre/lustre/ldlm/ldlm_internal.h
+++ b/drivers/staging/lustre/lustre/ldlm/ldlm_internal.h
@@ -169,8 +169,8 @@ extern struct kmem_cache *ldlm_lock_slab;
 /* ldlm_extent.c */
 void ldlm_extent_add_lock(struct ldlm_resource *res, struct ldlm_lock *lock);
 void ldlm_extent_unlink_lock(struct ldlm_lock *lock);
-void ldlm_extent_search(struct interval_node *root,
-			struct interval_node_extent *ext,
+void ldlm_extent_search(struct rb_root_cached *root,
+			__u64 start, __u64 end,
 			bool (*matches)(struct ldlm_lock *lock, void *data),
 			void *data);
 
diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c b/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c
index 4213fe047073..2fb2e088dc87 100644
--- a/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c
+++ b/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c
@@ -405,6 +405,7 @@ static struct ldlm_lock *ldlm_lock_new(struct ldlm_resource *resource)
 	lock->l_blocking_lock = NULL;
 	INIT_LIST_HEAD(&lock->l_sl_mode);
 	INIT_LIST_HEAD(&lock->l_sl_policy);
+	RB_CLEAR_NODE(&lock->l_rb);
 
 	lprocfs_counter_incr(ldlm_res_to_ns(resource)->ns_stats,
 			     LDLM_NSS_LOCKS);
@@ -1147,22 +1148,20 @@ static bool lock_matches(struct ldlm_lock *lock, void *vdata)
 static struct ldlm_lock *search_itree(struct ldlm_resource *res,
 				      struct lock_match_data *data)
 {
-	struct interval_node_extent ext = {
-		.start	= data->lmd_policy->l_extent.start,
-		.end	= data->lmd_policy->l_extent.end
-	};
 	int idx;
 
 	for (idx = 0; idx < LCK_MODE_NUM; idx++) {
 		struct ldlm_interval_tree *tree = &res->lr_itree[idx];
 
-		if (!tree->lit_root)
+		if (RB_EMPTY_ROOT(&tree->lit_root.rb_root))
 			continue;
 
 		if (!(tree->lit_mode & *data->lmd_mode))
 			continue;
 
-		ldlm_extent_search(tree->lit_root, &ext,
+		ldlm_extent_search(&tree->lit_root,
+				   data->lmd_policy->l_extent.start,
+				   data->lmd_policy->l_extent.end,
 				   lock_matches, data);
 	}
 	return data->lmd_lock;
diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_resource.c b/drivers/staging/lustre/lustre/ldlm/ldlm_resource.c
index c93b019b8e37..3946d62ff009 100644
--- a/drivers/staging/lustre/lustre/ldlm/ldlm_resource.c
+++ b/drivers/staging/lustre/lustre/ldlm/ldlm_resource.c
@@ -1017,7 +1017,7 @@ static struct ldlm_resource *ldlm_resource_new(void)
 	for (idx = 0; idx < LCK_MODE_NUM; idx++) {
 		res->lr_itree[idx].lit_size = 0;
 		res->lr_itree[idx].lit_mode = 1 << idx;
-		res->lr_itree[idx].lit_root = NULL;
+		res->lr_itree[idx].lit_root = RB_ROOT_CACHED;
 	}
 
 	atomic_set(&res->lr_refcount, 1);



From neilb at suse.com  Wed Jun  6 06:05:19 2018
From: neilb at suse.com (NeilBrown)
Date: Wed, 06 Jun 2018 16:05:19 +1000
Subject: [lustre-devel] [PATCH 06/11] staging: lustre: remove interval_tree
In-Reply-To: <152826510267.16761.14361003167157833896.stgit@noble>
References: <152826510267.16761.14361003167157833896.stgit@noble>
Message-ID: <152826511909.16761.11573648246807449778.stgit@noble>

There are no remaining users for this library,
so discard it.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 .../staging/lustre/lustre/include/interval_tree.h  |  119 ----
 drivers/staging/lustre/lustre/ldlm/interval_tree.c |  590 --------------------
 drivers/staging/lustre/lustre/ptlrpc/Makefile      |    1 
 3 files changed, 710 deletions(-)
 delete mode 100644 drivers/staging/lustre/lustre/include/interval_tree.h
 delete mode 100644 drivers/staging/lustre/lustre/ldlm/interval_tree.c

diff --git a/drivers/staging/lustre/lustre/include/interval_tree.h b/drivers/staging/lustre/lustre/include/interval_tree.h
deleted file mode 100644
index bcda74fc7875..000000000000
--- a/drivers/staging/lustre/lustre/include/interval_tree.h
+++ /dev/null
@@ -1,119 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-/*
- * GPL HEADER START
- *
- * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 only,
- * as published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful, but
- * WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
- * General Public License version 2 for more details (a copy is included
- * in the LICENSE file that accompanied this code).
- *
- * You should have received a copy of the GNU General Public License
- * version 2 along with this program; If not, see
- * http://www.gnu.org/licenses/gpl-2.0.html
- *
- * GPL HEADER END
- */
-/*
- * Copyright (c) 2008, 2010, Oracle and/or its affiliates. All rights reserved.
- * Use is subject to license terms.
- */
-/*
- * This file is part of Lustre, http://www.lustre.org/
- * Lustre is a trademark of Sun Microsystems, Inc.
- *
- * lustre/include/interval_tree.h
- *
- * Author: Huang Wei <huangwei at clusterfs.com>
- * Author: Jay Xiong <jinshan.xiong at sun.com>
- */
-
-#ifndef _INTERVAL_H__
-#define _INTERVAL_H__
-
-#include <linux/errno.h>
-#include <linux/string.h>
-#include <linux/types.h>
-
-struct interval_node {
-	struct interval_node   *in_left;
-	struct interval_node   *in_right;
-	struct interval_node   *in_parent;
-	unsigned		in_color:1,
-				in_intree:1, /** set if the node is in tree */
-				in_res1:30;
-	__u8		    in_res2[4];  /** tags, 8-bytes aligned */
-	__u64		   in_max_high;
-	struct interval_node_extent {
-		__u64 start;
-		__u64 end;
-	} in_extent;
-};
-
-enum interval_iter {
-	INTERVAL_ITER_CONT = 1,
-	INTERVAL_ITER_STOP = 2
-};
-
-static inline int interval_is_intree(struct interval_node *node)
-{
-	return node->in_intree == 1;
-}
-
-static inline __u64 interval_low(struct interval_node *node)
-{
-	return node->in_extent.start;
-}
-
-static inline __u64 interval_high(struct interval_node *node)
-{
-	return node->in_extent.end;
-}
-
-static inline int interval_set(struct interval_node *node,
-			       __u64 start, __u64 end)
-{
-	if (start > end)
-		return -ERANGE;
-	node->in_extent.start = start;
-	node->in_extent.end = end;
-	node->in_max_high = end;
-	return 0;
-}
-
-/*
- * Rules to write an interval callback.
- *  - the callback returns INTERVAL_ITER_STOP when it thinks the iteration
- *    should be stopped. It will then cause the iteration function to return
- *    immediately with return value INTERVAL_ITER_STOP.
- *  - callbacks for interval_iterate and interval_iterate_reverse: Every
- *    nodes in the tree will be set to @node before the callback being called
- *  - callback for interval_search: Only overlapped node will be set to @node
- *    before the callback being called.
- */
-typedef enum interval_iter (*interval_callback_t)(struct interval_node *node,
-						  void *args);
-
-void interval_insert(struct interval_node *node,
-		     struct interval_node **root);
-void interval_erase(struct interval_node *node, struct interval_node **root);
-
-/*
- * Search the extents in the tree and call @func for each overlapped
- * extents.
- */
-enum interval_iter interval_search(struct interval_node *root,
-				   struct interval_node_extent *ex,
-				   interval_callback_t func, void *data);
-
-enum interval_iter interval_iterate_reverse(struct interval_node *root,
-					    interval_callback_t func,
-					    void *data);
-
-#endif
diff --git a/drivers/staging/lustre/lustre/ldlm/interval_tree.c b/drivers/staging/lustre/lustre/ldlm/interval_tree.c
deleted file mode 100644
index f5232059d1b1..000000000000
--- a/drivers/staging/lustre/lustre/ldlm/interval_tree.c
+++ /dev/null
@@ -1,590 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-/*
- * GPL HEADER START
- *
- * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 only,
- * as published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful, but
- * WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
- * General Public License version 2 for more details (a copy is included
- * in the LICENSE file that accompanied this code).
- *
- * You should have received a copy of the GNU General Public License
- * version 2 along with this program; If not, see
- * http://www.gnu.org/licenses/gpl-2.0.html
- *
- * GPL HEADER END
- */
-/*
- * Copyright (c) 2008, 2010, Oracle and/or its affiliates. All rights reserved.
- * Use is subject to license terms.
- */
-/*
- * This file is part of Lustre, http://www.lustre.org/
- * Lustre is a trademark of Sun Microsystems, Inc.
- *
- * lustre/ldlm/interval_tree.c
- *
- * Interval tree library used by ldlm extent lock code
- *
- * Author: Huang Wei <huangwei at clusterfs.com>
- * Author: Jay Xiong <jinshan.xiong at sun.com>
- */
-#include <lustre_dlm.h>
-#include <obd_support.h>
-#include <interval_tree.h>
-
-enum {
-	INTERVAL_RED = 0,
-	INTERVAL_BLACK = 1
-};
-
-static inline int node_is_left_child(struct interval_node *node)
-{
-	return node == node->in_parent->in_left;
-}
-
-static inline int node_is_right_child(struct interval_node *node)
-{
-	return node == node->in_parent->in_right;
-}
-
-static inline int node_is_red(struct interval_node *node)
-{
-	return node->in_color == INTERVAL_RED;
-}
-
-static inline int node_is_black(struct interval_node *node)
-{
-	return node->in_color == INTERVAL_BLACK;
-}
-
-static inline int extent_compare(struct interval_node_extent *e1,
-				 struct interval_node_extent *e2)
-{
-	int rc;
-
-	if (e1->start == e2->start) {
-		if (e1->end < e2->end)
-			rc = -1;
-		else if (e1->end > e2->end)
-			rc = 1;
-		else
-			rc = 0;
-	} else {
-		if (e1->start < e2->start)
-			rc = -1;
-		else
-			rc = 1;
-	}
-	return rc;
-}
-
-static inline int extent_equal(struct interval_node_extent *e1,
-			       struct interval_node_extent *e2)
-{
-	return (e1->start == e2->start) && (e1->end == e2->end);
-}
-
-static inline int extent_overlapped(struct interval_node_extent *e1,
-				    struct interval_node_extent *e2)
-{
-	return (e1->start <= e2->end) && (e2->start <= e1->end);
-}
-
-static struct interval_node *interval_first(struct interval_node *node)
-{
-	if (!node)
-		return NULL;
-	while (node->in_left)
-		node = node->in_left;
-	return node;
-}
-
-static struct interval_node *interval_last(struct interval_node *node)
-{
-	if (!node)
-		return NULL;
-	while (node->in_right)
-		node = node->in_right;
-	return node;
-}
-
-static struct interval_node *interval_next(struct interval_node *node)
-{
-	if (!node)
-		return NULL;
-	if (node->in_right)
-		return interval_first(node->in_right);
-	while (node->in_parent && node_is_right_child(node))
-		node = node->in_parent;
-	return node->in_parent;
-}
-
-static struct interval_node *interval_prev(struct interval_node *node)
-{
-	if (!node)
-		return NULL;
-
-	if (node->in_left)
-		return interval_last(node->in_left);
-
-	while (node->in_parent && node_is_left_child(node))
-		node = node->in_parent;
-
-	return node->in_parent;
-}
-
-enum interval_iter interval_iterate_reverse(struct interval_node *root,
-					    interval_callback_t func,
-					    void *data)
-{
-	enum interval_iter rc = INTERVAL_ITER_CONT;
-	struct interval_node *node;
-
-	for (node = interval_last(root); node; node = interval_prev(node)) {
-		rc = func(node, data);
-		if (rc == INTERVAL_ITER_STOP)
-			break;
-	}
-
-	return rc;
-}
-EXPORT_SYMBOL(interval_iterate_reverse);
-
-static void __rotate_change_maxhigh(struct interval_node *node,
-				    struct interval_node *rotate)
-{
-	__u64 left_max, right_max;
-
-	rotate->in_max_high = node->in_max_high;
-	left_max = node->in_left ? node->in_left->in_max_high : 0;
-	right_max = node->in_right ? node->in_right->in_max_high : 0;
-	node->in_max_high  = max(interval_high(node),
-				 max(left_max, right_max));
-}
-
-/* The left rotation "pivots" around the link from node to node->right, and
- * - node will be linked to node->right's left child, and
- * - node->right's left child will be linked to node's right child.
- */
-static void __rotate_left(struct interval_node *node,
-			  struct interval_node **root)
-{
-	struct interval_node *right = node->in_right;
-	struct interval_node *parent = node->in_parent;
-
-	node->in_right = right->in_left;
-	if (node->in_right)
-		right->in_left->in_parent = node;
-
-	right->in_left = node;
-	right->in_parent = parent;
-	if (parent) {
-		if (node_is_left_child(node))
-			parent->in_left = right;
-		else
-			parent->in_right = right;
-	} else {
-		*root = right;
-	}
-	node->in_parent = right;
-
-	/* update max_high for node and right */
-	__rotate_change_maxhigh(node, right);
-}
-
-/* The right rotation "pivots" around the link from node to node->left, and
- * - node will be linked to node->left's right child, and
- * - node->left's right child will be linked to node's left child.
- */
-static void __rotate_right(struct interval_node *node,
-			   struct interval_node **root)
-{
-	struct interval_node *left = node->in_left;
-	struct interval_node *parent = node->in_parent;
-
-	node->in_left = left->in_right;
-	if (node->in_left)
-		left->in_right->in_parent = node;
-	left->in_right = node;
-
-	left->in_parent = parent;
-	if (parent) {
-		if (node_is_right_child(node))
-			parent->in_right = left;
-		else
-			parent->in_left = left;
-	} else {
-		*root = left;
-	}
-	node->in_parent = left;
-
-	/* update max_high for node and left */
-	__rotate_change_maxhigh(node, left);
-}
-
-#define interval_swap(a, b) do {			\
-	struct interval_node *c = a; a = b; b = c;      \
-} while (0)
-
-/*
- * Operations INSERT and DELETE, when run on a tree with n keys,
- * take O(logN) time.Because they modify the tree, the result
- * may violate the red-black properties.To restore these properties,
- * we must change the colors of some of the nodes in the tree
- * and also change the pointer structure.
- */
-static void interval_insert_color(struct interval_node *node,
-				  struct interval_node **root)
-{
-	struct interval_node *parent, *gparent;
-
-	while ((parent = node->in_parent) && node_is_red(parent)) {
-		gparent = parent->in_parent;
-		/* Parent is RED, so gparent must not be NULL */
-		if (node_is_left_child(parent)) {
-			struct interval_node *uncle;
-
-			uncle = gparent->in_right;
-			if (uncle && node_is_red(uncle)) {
-				uncle->in_color = INTERVAL_BLACK;
-				parent->in_color = INTERVAL_BLACK;
-				gparent->in_color = INTERVAL_RED;
-				node = gparent;
-				continue;
-			}
-
-			if (parent->in_right == node) {
-				__rotate_left(parent, root);
-				interval_swap(node, parent);
-			}
-
-			parent->in_color = INTERVAL_BLACK;
-			gparent->in_color = INTERVAL_RED;
-			__rotate_right(gparent, root);
-		} else {
-			struct interval_node *uncle;
-
-			uncle = gparent->in_left;
-			if (uncle && node_is_red(uncle)) {
-				uncle->in_color = INTERVAL_BLACK;
-				parent->in_color = INTERVAL_BLACK;
-				gparent->in_color = INTERVAL_RED;
-				node = gparent;
-				continue;
-			}
-
-			if (node_is_left_child(node)) {
-				__rotate_right(parent, root);
-				interval_swap(node, parent);
-			}
-
-			parent->in_color = INTERVAL_BLACK;
-			gparent->in_color = INTERVAL_RED;
-			__rotate_left(gparent, root);
-		}
-	}
-
-	(*root)->in_color = INTERVAL_BLACK;
-}
-
-void interval_insert(struct interval_node *node,
-		     struct interval_node **root)
-
-{
-	struct interval_node **p, *parent = NULL;
-
-	LASSERT(!interval_is_intree(node));
-	p = root;
-	while (*p) {
-		parent = *p;
-
-		/* max_high field must be updated after each iteration */
-		if (parent->in_max_high < interval_high(node))
-			parent->in_max_high = interval_high(node);
-
-		if (extent_compare(&node->in_extent, &parent->in_extent) < 0)
-			p = &parent->in_left;
-		else
-			p = &parent->in_right;
-	}
-
-	/* link node into the tree */
-	node->in_parent = parent;
-	node->in_color = INTERVAL_RED;
-	node->in_left = NULL;
-	node->in_right = NULL;
-	*p = node;
-
-	interval_insert_color(node, root);
-	node->in_intree = 1;
-}
-EXPORT_SYMBOL(interval_insert);
-
-static inline int node_is_black_or_0(struct interval_node *node)
-{
-	return !node || node_is_black(node);
-}
-
-static void interval_erase_color(struct interval_node *node,
-				 struct interval_node *parent,
-				 struct interval_node **root)
-{
-	struct interval_node *tmp;
-
-	while (node_is_black_or_0(node) && node != *root) {
-		if (parent->in_left == node) {
-			tmp = parent->in_right;
-			if (node_is_red(tmp)) {
-				tmp->in_color = INTERVAL_BLACK;
-				parent->in_color = INTERVAL_RED;
-				__rotate_left(parent, root);
-				tmp = parent->in_right;
-			}
-			if (node_is_black_or_0(tmp->in_left) &&
-			    node_is_black_or_0(tmp->in_right)) {
-				tmp->in_color = INTERVAL_RED;
-				node = parent;
-				parent = node->in_parent;
-			} else {
-				if (node_is_black_or_0(tmp->in_right)) {
-					struct interval_node *o_left;
-
-					o_left = tmp->in_left;
-					if (o_left)
-						o_left->in_color = INTERVAL_BLACK;
-					tmp->in_color = INTERVAL_RED;
-					__rotate_right(tmp, root);
-					tmp = parent->in_right;
-				}
-				tmp->in_color = parent->in_color;
-				parent->in_color = INTERVAL_BLACK;
-				if (tmp->in_right)
-					tmp->in_right->in_color = INTERVAL_BLACK;
-				__rotate_left(parent, root);
-				node = *root;
-				break;
-			}
-		} else {
-			tmp = parent->in_left;
-			if (node_is_red(tmp)) {
-				tmp->in_color = INTERVAL_BLACK;
-				parent->in_color = INTERVAL_RED;
-				__rotate_right(parent, root);
-				tmp = parent->in_left;
-			}
-			if (node_is_black_or_0(tmp->in_left) &&
-			    node_is_black_or_0(tmp->in_right)) {
-				tmp->in_color = INTERVAL_RED;
-				node = parent;
-				parent = node->in_parent;
-			} else {
-				if (node_is_black_or_0(tmp->in_left)) {
-					struct interval_node *o_right;
-
-					o_right = tmp->in_right;
-					if (o_right)
-						o_right->in_color = INTERVAL_BLACK;
-					tmp->in_color = INTERVAL_RED;
-					__rotate_left(tmp, root);
-					tmp = parent->in_left;
-				}
-				tmp->in_color = parent->in_color;
-				parent->in_color = INTERVAL_BLACK;
-				if (tmp->in_left)
-					tmp->in_left->in_color = INTERVAL_BLACK;
-				__rotate_right(parent, root);
-				node = *root;
-				break;
-			}
-		}
-	}
-	if (node)
-		node->in_color = INTERVAL_BLACK;
-}
-
-/*
- * if the @max_high value of @node is changed, this function traverse  a path
- * from node  up to the root to update max_high for the whole tree.
- */
-static void update_maxhigh(struct interval_node *node,
-			   __u64  old_maxhigh)
-{
-	__u64 left_max, right_max;
-
-	while (node) {
-		left_max = node->in_left ? node->in_left->in_max_high : 0;
-		right_max = node->in_right ? node->in_right->in_max_high : 0;
-		node->in_max_high = max(interval_high(node),
-					max(left_max, right_max));
-
-		if (node->in_max_high >= old_maxhigh)
-			break;
-		node = node->in_parent;
-	}
-}
-
-void interval_erase(struct interval_node *node,
-		    struct interval_node **root)
-{
-	struct interval_node *child, *parent;
-	int color;
-
-	LASSERT(interval_is_intree(node));
-	node->in_intree = 0;
-	if (!node->in_left) {
-		child = node->in_right;
-	} else if (!node->in_right) {
-		child = node->in_left;
-	} else { /* Both left and right child are not NULL */
-		struct interval_node *old = node;
-
-		node = interval_next(node);
-		child = node->in_right;
-		parent = node->in_parent;
-		color = node->in_color;
-
-		if (child)
-			child->in_parent = parent;
-		if (parent == old)
-			parent->in_right = child;
-		else
-			parent->in_left = child;
-
-		node->in_color = old->in_color;
-		node->in_right = old->in_right;
-		node->in_left = old->in_left;
-		node->in_parent = old->in_parent;
-
-		if (old->in_parent) {
-			if (node_is_left_child(old))
-				old->in_parent->in_left = node;
-			else
-				old->in_parent->in_right = node;
-		} else {
-			*root = node;
-		}
-
-		old->in_left->in_parent = node;
-		if (old->in_right)
-			old->in_right->in_parent = node;
-		update_maxhigh(child ? : parent, node->in_max_high);
-		update_maxhigh(node, old->in_max_high);
-		if (parent == old)
-			parent = node;
-		goto color;
-	}
-	parent = node->in_parent;
-	color = node->in_color;
-
-	if (child)
-		child->in_parent = parent;
-	if (parent) {
-		if (node_is_left_child(node))
-			parent->in_left = child;
-		else
-			parent->in_right = child;
-	} else {
-		*root = child;
-	}
-
-	update_maxhigh(child ? : parent, node->in_max_high);
-
-color:
-	if (color == INTERVAL_BLACK)
-		interval_erase_color(child, parent, root);
-}
-EXPORT_SYMBOL(interval_erase);
-
-static inline int interval_may_overlap(struct interval_node *node,
-				       struct interval_node_extent *ext)
-{
-	return (ext->start <= node->in_max_high &&
-		ext->end >= interval_low(node));
-}
-
-/*
- * This function finds all intervals that overlap interval ext,
- * and calls func to handle resulted intervals one by one.
- * in lustre, this function will find all conflicting locks in
- * the granted queue and add these locks to the ast work list.
- *
- * {
- *	if (!node)
- *		return 0;
- *	if (ext->end < interval_low(node)) {
- *		interval_search(node->in_left, ext, func, data);
- *	} else if (interval_may_overlap(node, ext)) {
- *		if (extent_overlapped(ext, &node->in_extent))
- *			func(node, data);
- *		interval_search(node->in_left, ext, func, data);
- *		interval_search(node->in_right, ext, func, data);
- *	}
- *	return 0;
- * }
- *
- */
-enum interval_iter interval_search(struct interval_node *node,
-				   struct interval_node_extent *ext,
-				   interval_callback_t func,
-				   void *data)
-{
-	enum interval_iter rc = INTERVAL_ITER_CONT;
-	struct interval_node *parent;
-
-	LASSERT(ext);
-	LASSERT(func);
-
-	while (node) {
-		if (ext->end < interval_low(node)) {
-			if (node->in_left) {
-				node = node->in_left;
-				continue;
-			}
-		} else if (interval_may_overlap(node, ext)) {
-			if (extent_overlapped(ext, &node->in_extent)) {
-				rc = func(node, data);
-				if (rc == INTERVAL_ITER_STOP)
-					break;
-			}
-
-			if (node->in_left) {
-				node = node->in_left;
-				continue;
-			}
-			if (node->in_right) {
-				node = node->in_right;
-				continue;
-			}
-		}
-
-		parent = node->in_parent;
-		while (parent) {
-			if (node_is_left_child(node) &&
-			    parent->in_right) {
-				/*
-				 * If we ever got the left, it means that the
-				 * parent met ext->end<interval_low(parent), or
-				 * may_overlap(parent). If the former is true,
-				 * we needn't go back. So stop early and check
-				 * may_overlap(parent) after this loop.
-				 */
-				node = parent->in_right;
-				break;
-			}
-			node = parent;
-			parent = parent->in_parent;
-		}
-		if (!parent || !interval_may_overlap(parent, ext))
-			break;
-	}
-
-	return rc;
-}
-EXPORT_SYMBOL(interval_search);
diff --git a/drivers/staging/lustre/lustre/ptlrpc/Makefile b/drivers/staging/lustre/lustre/ptlrpc/Makefile
index 1deb1971b39e..77f8eabb2e28 100644
--- a/drivers/staging/lustre/lustre/ptlrpc/Makefile
+++ b/drivers/staging/lustre/lustre/ptlrpc/Makefile
@@ -11,7 +11,6 @@ ldlm_objs += $(LDLM)ldlm_plain.o $(LDLM)ldlm_extent.o
 ldlm_objs += $(LDLM)ldlm_request.o $(LDLM)ldlm_lockd.o
 ldlm_objs += $(LDLM)ldlm_flock.o $(LDLM)ldlm_inodebits.o
 ldlm_objs += $(LDLM)ldlm_pool.o
-ldlm_objs += $(LDLM)interval_tree.o
 ptlrpc_objs := client.o recover.o connection.o niobuf.o pack_generic.o
 ptlrpc_objs += events.o ptlrpc_module.o service.o pinger.o
 ptlrpc_objs += llog_net.o llog_client.o import.o ptlrpcd.o



From neilb at suse.com  Wed Jun  6 06:05:19 2018
From: neilb at suse.com (NeilBrown)
Date: Wed, 06 Jun 2018 16:05:19 +1000
Subject: [lustre-devel] [PATCH 07/11] staging: lustre: fold
 lprocfs_call_handler functionality into lnet_debugfs_*
In-Reply-To: <152826510267.16761.14361003167157833896.stgit@noble>
References: <152826510267.16761.14361003167157833896.stgit@noble>
Message-ID: <152826511912.16761.6908134754944227444.stgit@noble>

The calling convention for ->proc_handler is rather clumsy,
as a comment in fs/procfs/proc_sysctl.c confirms.
lustre has copied this convention to lnet_debugfs_{read,write},
and then provided a wrapper for handlers - lprocfs_call_handler -
to work around the clumsiness.

It is cleaner to just fold the functionality of lprocfs_call_handler()
into lnet_debugfs_* and let them call the final handler directly.

If these files were ever moved to /proc/sys (which seems unlikely) the
handling in fs/procfs/proc_sysctl.c would need to be fixed to, but
that would not be a bad thing.

So modify all the functions that did use the wrapper to not need it
now that a more sane calling convention is available.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 .../staging/lustre/include/linux/libcfs/libcfs.h   |    4 -
 drivers/staging/lustre/lnet/libcfs/module.c        |   84 +++++++-------------
 drivers/staging/lustre/lnet/lnet/router_proc.c     |   41 +++-------
 3 files changed, 41 insertions(+), 88 deletions(-)

diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs.h b/drivers/staging/lustre/include/linux/libcfs/libcfs.h
index edc7ed0dcb94..7ac609328256 100644
--- a/drivers/staging/lustre/include/linux/libcfs/libcfs.h
+++ b/drivers/staging/lustre/include/linux/libcfs/libcfs.h
@@ -57,10 +57,6 @@ int libcfs_setup(void);
 extern struct workqueue_struct *cfs_rehash_wq;
 
 void lustre_insert_debugfs(struct ctl_table *table);
-int lprocfs_call_handler(void *data, int write, loff_t *ppos,
-			 void __user *buffer, size_t *lenp,
-			 int (*handler)(void *data, int write, loff_t pos,
-					void __user *buffer, int len));
 
 /*
  * Memory
diff --git a/drivers/staging/lustre/lnet/libcfs/module.c b/drivers/staging/lustre/lnet/libcfs/module.c
index 5dc7de9e6478..02c404c6738e 100644
--- a/drivers/staging/lustre/lnet/libcfs/module.c
+++ b/drivers/staging/lustre/lnet/libcfs/module.c
@@ -290,33 +290,15 @@ static struct miscdevice libcfs_dev = {
 
 static int libcfs_dev_registered;
 
-int lprocfs_call_handler(void *data, int write, loff_t *ppos,
-			 void __user *buffer, size_t *lenp,
-			 int (*handler)(void *data, int write, loff_t pos,
-					void __user *buffer, int len))
-{
-	int rc = handler(data, write, *ppos, buffer, *lenp);
-
-	if (rc < 0)
-		return rc;
-
-	if (write) {
-		*ppos += *lenp;
-	} else {
-		*lenp = rc;
-		*ppos += rc;
-	}
-	return 0;
-}
-EXPORT_SYMBOL(lprocfs_call_handler);
-
-static int __proc_dobitmasks(void *data, int write,
-			     loff_t pos, void __user *buffer, int nob)
+static int proc_dobitmasks(struct ctl_table *table, int write,
+			   void __user *buffer, size_t *lenp, loff_t *ppos)
 {
 	const int tmpstrlen = 512;
 	char *tmpstr;
 	int rc;
-	unsigned int *mask = data;
+	size_t nob = *lenp;
+	loff_t pos = *ppos;
+	unsigned int *mask = table->data;
 	int is_subsys = (mask == &libcfs_subsystem_debug) ? 1 : 0;
 	int is_printk = (mask == &libcfs_printk) ? 1 : 0;
 
@@ -351,32 +333,23 @@ static int __proc_dobitmasks(void *data, int write,
 	return rc;
 }
 
-static int proc_dobitmasks(struct ctl_table *table, int write,
-			   void __user *buffer, size_t *lenp, loff_t *ppos)
+static int proc_dump_kernel(struct ctl_table *table, int write,
+			    void __user *buffer, size_t *lenp, loff_t *ppos)
 {
-	return lprocfs_call_handler(table->data, write, ppos, buffer, lenp,
-				    __proc_dobitmasks);
-}
+	size_t nob = *lenp;
 
-static int __proc_dump_kernel(void *data, int write,
-			      loff_t pos, void __user *buffer, int nob)
-{
 	if (!write)
 		return 0;
 
 	return cfs_trace_dump_debug_buffer_usrstr(buffer, nob);
 }
 
-static int proc_dump_kernel(struct ctl_table *table, int write,
+static int proc_daemon_file(struct ctl_table *table, int write,
 			    void __user *buffer, size_t *lenp, loff_t *ppos)
 {
-	return lprocfs_call_handler(table->data, write, ppos, buffer, lenp,
-				    __proc_dump_kernel);
-}
+	size_t nob = *lenp;
+	loff_t pos = *ppos;
 
-static int __proc_daemon_file(void *data, int write,
-			      loff_t pos, void __user *buffer, int nob)
-{
 	if (!write) {
 		int len = strlen(cfs_tracefile);
 
@@ -390,13 +363,6 @@ static int __proc_daemon_file(void *data, int write,
 	return cfs_trace_daemon_command_usrstr(buffer, nob);
 }
 
-static int proc_daemon_file(struct ctl_table *table, int write,
-			    void __user *buffer, size_t *lenp, loff_t *ppos)
-{
-	return lprocfs_call_handler(table->data, write, ppos, buffer, lenp,
-				    __proc_daemon_file);
-}
-
 static int libcfs_force_lbug(struct ctl_table *table, int write,
 			     void __user *buffer,
 			     size_t *lenp, loff_t *ppos)
@@ -419,9 +385,11 @@ static int proc_fail_loc(struct ctl_table *table, int write,
 	return rc;
 }
 
-static int __proc_cpt_table(void *data, int write,
-			    loff_t pos, void __user *buffer, int nob)
+static int proc_cpt_table(struct ctl_table *table, int write,
+			  void __user *buffer, size_t *lenp, loff_t *ppos)
 {
+	size_t nob = *lenp;
+	loff_t pos = *ppos;
 	char *buf = NULL;
 	int len = 4096;
 	int rc  = 0;
@@ -457,13 +425,6 @@ static int __proc_cpt_table(void *data, int write,
 	return rc;
 }
 
-static int proc_cpt_table(struct ctl_table *table, int write,
-			  void __user *buffer, size_t *lenp, loff_t *ppos)
-{
-	return lprocfs_call_handler(table->data, write, ppos, buffer, lenp,
-				    __proc_cpt_table);
-}
-
 static struct ctl_table lnet_table[] = {
 	{
 		.procname = "debug",
@@ -573,10 +534,17 @@ static ssize_t lnet_debugfs_read(struct file *filp, char __user *buf,
 {
 	struct ctl_table *table = filp->private_data;
 	int error;
+	loff_t old_pos = *ppos;
 
 	error = table->proc_handler(table, 0, (void __user *)buf, &count, ppos);
-	if (!error)
+	/*
+	 * On success, the length read is either in error or in count.
+	 * If ppos changed, then use count, else use error
+	 */
+	if (!error && *ppos != old_pos)
 		error = count;
+	else if (error > 0)
+		*ppos += error;
 
 	return error;
 }
@@ -586,10 +554,14 @@ static ssize_t lnet_debugfs_write(struct file *filp, const char __user *buf,
 {
 	struct ctl_table *table = filp->private_data;
 	int error;
+	loff_t old_pos = *ppos;
 
 	error = table->proc_handler(table, 1, (void __user *)buf, &count, ppos);
-	if (!error)
+	if (!error) {
 		error = count;
+		if (*ppos == old_pos)
+			*ppos += count;
+	}
 
 	return error;
 }
diff --git a/drivers/staging/lustre/lnet/lnet/router_proc.c b/drivers/staging/lustre/lnet/lnet/router_proc.c
index ae4b7f5953a0..f135082fec5c 100644
--- a/drivers/staging/lustre/lnet/lnet/router_proc.c
+++ b/drivers/staging/lustre/lnet/lnet/router_proc.c
@@ -74,11 +74,13 @@
 
 #define LNET_PROC_VERSION(v)	((unsigned int)((v) & LNET_PROC_VER_MASK))
 
-static int __proc_lnet_stats(void *data, int write,
-			     loff_t pos, void __user *buffer, int nob)
+static int proc_lnet_stats(struct ctl_table *table, int write,
+			   void __user *buffer, size_t *lenp, loff_t *ppos)
 {
 	int rc;
 	struct lnet_counters *ctrs;
+	size_t nob = *lenp;
+	loff_t pos = *ppos;
 	int len;
 	char *tmpstr;
 	const int tmpsiz = 256; /* 7 %u and 4 %llu */
@@ -122,13 +124,6 @@ static int __proc_lnet_stats(void *data, int write,
 	return rc;
 }
 
-static int proc_lnet_stats(struct ctl_table *table, int write,
-			   void __user *buffer, size_t *lenp, loff_t *ppos)
-{
-	return lprocfs_call_handler(table->data, write, ppos, buffer, lenp,
-				    __proc_lnet_stats);
-}
-
 static int proc_lnet_routes(struct ctl_table *table, int write,
 			    void __user *buffer, size_t *lenp, loff_t *ppos)
 {
@@ -562,9 +557,11 @@ static int proc_lnet_peers(struct ctl_table *table, int write,
 	return rc;
 }
 
-static int __proc_lnet_buffers(void *data, int write,
-			       loff_t pos, void __user *buffer, int nob)
+static int proc_lnet_buffers(struct ctl_table *table, int write,
+			     void __user *buffer, size_t *lenp, loff_t *ppos)
 {
+	size_t nob = *lenp;
+	loff_t pos = *ppos;
 	char *s;
 	char *tmpstr;
 	int tmpsiz;
@@ -620,13 +617,6 @@ static int __proc_lnet_buffers(void *data, int write,
 	return rc;
 }
 
-static int proc_lnet_buffers(struct ctl_table *table, int write,
-			     void __user *buffer, size_t *lenp, loff_t *ppos)
-{
-	return lprocfs_call_handler(table->data, write, ppos, buffer, lenp,
-				    __proc_lnet_buffers);
-}
-
 static int proc_lnet_nis(struct ctl_table *table, int write,
 			 void __user *buffer, size_t *lenp, loff_t *ppos)
 {
@@ -784,10 +774,13 @@ static struct lnet_portal_rotors	portal_rotors[] = {
 	},
 };
 
-static int __proc_lnet_portal_rotor(void *data, int write,
-				    loff_t pos, void __user *buffer, int nob)
+static int proc_lnet_portal_rotor(struct ctl_table *table, int write,
+				  void __user *buffer, size_t *lenp,
+				  loff_t *ppos)
 {
 	const int buf_len = 128;
+	size_t nob = *lenp;
+	loff_t pos = *ppos;
 	char *buf;
 	char *tmp;
 	int rc;
@@ -845,14 +838,6 @@ static int __proc_lnet_portal_rotor(void *data, int write,
 	return rc;
 }
 
-static int proc_lnet_portal_rotor(struct ctl_table *table, int write,
-				  void __user *buffer, size_t *lenp,
-				  loff_t *ppos)
-{
-	return lprocfs_call_handler(table->data, write, ppos, buffer, lenp,
-				    __proc_lnet_portal_rotor);
-}
-
 static struct ctl_table lnet_table[] = {
 	/*
 	 * NB No .strategy entries have been provided since sysctl(8) prefers



From neilb at suse.com  Wed Jun  6 06:05:19 2018
From: neilb at suse.com (NeilBrown)
Date: Wed, 06 Jun 2018 16:05:19 +1000
Subject: [lustre-devel] [PATCH 08/11] staging: lustre: obdclass: move
 linux/linux-foo.c to foo.c
In-Reply-To: <152826510267.16761.14361003167157833896.stgit@noble>
References: <152826510267.16761.14361003167157833896.stgit@noble>
Message-ID: <152826511916.16761.16388038372000609798.stgit@noble>

As lustre is now linux-only, having this linux sub-directory
with files named "linux-something" is just noise.  Move them
to a more friendly name.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 drivers/staging/lustre/lustre/obdclass/Makefile    |    2 
 .../lustre/lustre/obdclass/linux/linux-module.c    |  514 --------------------
 .../lustre/lustre/obdclass/linux/linux-sysctl.c    |  162 ------
 drivers/staging/lustre/lustre/obdclass/module.c    |  514 ++++++++++++++++++++
 drivers/staging/lustre/lustre/obdclass/sysctl.c    |  162 ++++++
 5 files changed, 677 insertions(+), 677 deletions(-)
 delete mode 100644 drivers/staging/lustre/lustre/obdclass/linux/linux-module.c
 delete mode 100644 drivers/staging/lustre/lustre/obdclass/linux/linux-sysctl.c
 create mode 100644 drivers/staging/lustre/lustre/obdclass/module.c
 create mode 100644 drivers/staging/lustre/lustre/obdclass/sysctl.c

diff --git a/drivers/staging/lustre/lustre/obdclass/Makefile b/drivers/staging/lustre/lustre/obdclass/Makefile
index e3fa9acff4c4..e36ba2167d10 100644
--- a/drivers/staging/lustre/lustre/obdclass/Makefile
+++ b/drivers/staging/lustre/lustre/obdclass/Makefile
@@ -4,7 +4,7 @@ subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/lustre/include
 
 obj-$(CONFIG_LUSTRE_FS) += obdclass.o
 
-obdclass-y := linux/linux-module.o linux/linux-sysctl.o \
+obdclass-y := module.o sysctl.o \
 	      llog.o llog_cat.o llog_obd.o llog_swab.o class_obd.o debug.o \
 	      genops.o uuid.o lprocfs_status.o lprocfs_counters.o \
 	      lustre_handles.o lustre_peer.o statfs_pack.o linkea.o \
diff --git a/drivers/staging/lustre/lustre/obdclass/linux/linux-module.c b/drivers/staging/lustre/lustre/obdclass/linux/linux-module.c
deleted file mode 100644
index 9c800580053b..000000000000
--- a/drivers/staging/lustre/lustre/obdclass/linux/linux-module.c
+++ /dev/null
@@ -1,514 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-/*
- * GPL HEADER START
- *
- * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 only,
- * as published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful, but
- * WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
- * General Public License version 2 for more details (a copy is included
- * in the LICENSE file that accompanied this code).
- *
- * You should have received a copy of the GNU General Public License
- * version 2 along with this program; If not, see
- * http://www.gnu.org/licenses/gpl-2.0.html
- *
- * GPL HEADER END
- */
-/*
- * Copyright (c) 2007, 2010, Oracle and/or its affiliates. All rights reserved.
- * Use is subject to license terms.
- *
- * Copyright (c) 2011, 2012, Intel Corporation.
- */
-/*
- * This file is part of Lustre, http://www.lustre.org/
- * Lustre is a trademark of Sun Microsystems, Inc.
- *
- * lustre/obdclass/linux/linux-module.c
- *
- * Object Devices Class Driver
- * These are the only exported functions, they provide some generic
- * infrastructure for managing object devices
- */
-
-#define DEBUG_SUBSYSTEM S_CLASS
-
-#include <linux/module.h>
-#include <linux/errno.h>
-#include <linux/kernel.h>
-#include <linux/sched.h>
-#include <linux/lp.h>
-#include <linux/slab.h>
-#include <linux/ioport.h>
-#include <linux/fcntl.h>
-#include <linux/delay.h>
-#include <linux/skbuff.h>
-#include <linux/fs.h>
-#include <linux/poll.h>
-#include <linux/list.h>
-#include <linux/highmem.h>
-#include <linux/io.h>
-#include <asm/ioctls.h>
-#include <linux/uaccess.h>
-#include <linux/miscdevice.h>
-#include <linux/seq_file.h>
-#include <linux/kobject.h>
-
-#include <uapi/linux/lnet/lnetctl.h>
-#include <obd_support.h>
-#include <obd_class.h>
-#include <lprocfs_status.h>
-#include <uapi/linux/lustre/lustre_ioctl.h>
-#include <uapi/linux/lustre/lustre_ver.h>
-
-#define OBD_MAX_IOCTL_BUFFER	8192
-
-static int obd_ioctl_is_invalid(struct obd_ioctl_data *data)
-{
-	if (data->ioc_len > BIT(30)) {
-		CERROR("OBD ioctl: ioc_len larger than 1<<30\n");
-		return 1;
-	}
-
-	if (data->ioc_inllen1 > BIT(30)) {
-		CERROR("OBD ioctl: ioc_inllen1 larger than 1<<30\n");
-		return 1;
-	}
-
-	if (data->ioc_inllen2 > BIT(30)) {
-		CERROR("OBD ioctl: ioc_inllen2 larger than 1<<30\n");
-		return 1;
-	}
-
-	if (data->ioc_inllen3 > BIT(30)) {
-		CERROR("OBD ioctl: ioc_inllen3 larger than 1<<30\n");
-		return 1;
-	}
-
-	if (data->ioc_inllen4 > BIT(30)) {
-		CERROR("OBD ioctl: ioc_inllen4 larger than 1<<30\n");
-		return 1;
-	}
-
-	if (data->ioc_inlbuf1 && data->ioc_inllen1 == 0) {
-		CERROR("OBD ioctl: inlbuf1 pointer but 0 length\n");
-		return 1;
-	}
-
-	if (data->ioc_inlbuf2 && data->ioc_inllen2 == 0) {
-		CERROR("OBD ioctl: inlbuf2 pointer but 0 length\n");
-		return 1;
-	}
-
-	if (data->ioc_inlbuf3 && data->ioc_inllen3 == 0) {
-		CERROR("OBD ioctl: inlbuf3 pointer but 0 length\n");
-		return 1;
-	}
-
-	if (data->ioc_inlbuf4 && data->ioc_inllen4 == 0) {
-		CERROR("OBD ioctl: inlbuf4 pointer but 0 length\n");
-		return 1;
-	}
-
-	if (data->ioc_pbuf1 && data->ioc_plen1 == 0) {
-		CERROR("OBD ioctl: pbuf1 pointer but 0 length\n");
-		return 1;
-	}
-
-	if (data->ioc_pbuf2 && data->ioc_plen2 == 0) {
-		CERROR("OBD ioctl: pbuf2 pointer but 0 length\n");
-		return 1;
-	}
-
-	if (!data->ioc_pbuf1 && data->ioc_plen1 != 0) {
-		CERROR("OBD ioctl: plen1 set but NULL pointer\n");
-		return 1;
-	}
-
-	if (!data->ioc_pbuf2 && data->ioc_plen2 != 0) {
-		CERROR("OBD ioctl: plen2 set but NULL pointer\n");
-		return 1;
-	}
-
-	if (obd_ioctl_packlen(data) > data->ioc_len) {
-		CERROR("OBD ioctl: packlen exceeds ioc_len (%d > %d)\n",
-		obd_ioctl_packlen(data), data->ioc_len);
-		return 1;
-	}
-
-	return 0;
-}
-
-/* buffer MUST be at least the size of obd_ioctl_hdr */
-int obd_ioctl_getdata(char **buf, int *len, void __user *arg)
-{
-	struct obd_ioctl_hdr hdr;
-	struct obd_ioctl_data *data;
-	int err;
-	int offset = 0;
-
-	if (copy_from_user(&hdr, arg, sizeof(hdr)))
-		return -EFAULT;
-
-	if (hdr.ioc_version != OBD_IOCTL_VERSION) {
-		CERROR("Version mismatch kernel (%x) vs application (%x)\n",
-		       OBD_IOCTL_VERSION, hdr.ioc_version);
-		return -EINVAL;
-	}
-
-	if (hdr.ioc_len > OBD_MAX_IOCTL_BUFFER) {
-		CERROR("User buffer len %d exceeds %d max buffer\n",
-		       hdr.ioc_len, OBD_MAX_IOCTL_BUFFER);
-		return -EINVAL;
-	}
-
-	if (hdr.ioc_len < sizeof(struct obd_ioctl_data)) {
-		CERROR("User buffer too small for ioctl (%d)\n", hdr.ioc_len);
-		return -EINVAL;
-	}
-
-	/* When there are lots of processes calling vmalloc on multi-core
-	 * system, the high lock contention will hurt performance badly,
-	 * obdfilter-survey is an example, which relies on ioctl. So we'd
-	 * better avoid vmalloc on ioctl path. LU-66
-	 */
-	*buf = kvzalloc(hdr.ioc_len, GFP_KERNEL);
-	if (!*buf) {
-		CERROR("Cannot allocate control buffer of len %d\n",
-		       hdr.ioc_len);
-		return -EINVAL;
-	}
-	*len = hdr.ioc_len;
-	data = (struct obd_ioctl_data *)*buf;
-
-	if (copy_from_user(*buf, arg, hdr.ioc_len)) {
-		err = -EFAULT;
-		goto free_buf;
-	}
-	if (hdr.ioc_len != data->ioc_len) {
-		err = -EINVAL;
-		goto free_buf;
-	}
-
-	if (obd_ioctl_is_invalid(data)) {
-		CERROR("ioctl not correctly formatted\n");
-		err = -EINVAL;
-		goto free_buf;
-	}
-
-	if (data->ioc_inllen1) {
-		data->ioc_inlbuf1 = &data->ioc_bulk[0];
-		offset += cfs_size_round(data->ioc_inllen1);
-	}
-
-	if (data->ioc_inllen2) {
-		data->ioc_inlbuf2 = &data->ioc_bulk[0] + offset;
-		offset += cfs_size_round(data->ioc_inllen2);
-	}
-
-	if (data->ioc_inllen3) {
-		data->ioc_inlbuf3 = &data->ioc_bulk[0] + offset;
-		offset += cfs_size_round(data->ioc_inllen3);
-	}
-
-	if (data->ioc_inllen4)
-		data->ioc_inlbuf4 = &data->ioc_bulk[0] + offset;
-
-	return 0;
-
-free_buf:
-	kvfree(*buf);
-	return err;
-}
-EXPORT_SYMBOL(obd_ioctl_getdata);
-
-/*  opening /dev/obd */
-static int obd_class_open(struct inode *inode, struct file *file)
-{
-	try_module_get(THIS_MODULE);
-	return 0;
-}
-
-/*  closing /dev/obd */
-static int obd_class_release(struct inode *inode, struct file *file)
-{
-	module_put(THIS_MODULE);
-	return 0;
-}
-
-/* to control /dev/obd */
-static long obd_class_ioctl(struct file *filp, unsigned int cmd,
-			    unsigned long arg)
-{
-	int err = 0;
-
-	/* Allow non-root access for OBD_IOC_PING_TARGET - used by lfs check */
-	if (!capable(CAP_SYS_ADMIN) && (cmd != OBD_IOC_PING_TARGET))
-		return err = -EACCES;
-	if ((cmd & 0xffffff00) == ((int)'T') << 8) /* ignore all tty ioctls */
-		return err = -ENOTTY;
-
-	err = class_handle_ioctl(cmd, (unsigned long)arg);
-
-	return err;
-}
-
-/* declare character device */
-static const struct file_operations obd_psdev_fops = {
-	.owner	  = THIS_MODULE,
-	.unlocked_ioctl = obd_class_ioctl, /* unlocked_ioctl */
-	.open	   = obd_class_open,      /* open */
-	.release	= obd_class_release,   /* release */
-};
-
-/* modules setup */
-struct miscdevice obd_psdev = {
-	.minor = MISC_DYNAMIC_MINOR,
-	.name  = OBD_DEV_NAME,
-	.fops  = &obd_psdev_fops,
-};
-
-static ssize_t version_show(struct kobject *kobj, struct attribute *attr,
-			    char *buf)
-{
-	return sprintf(buf, "%s\n", LUSTRE_VERSION_STRING);
-}
-
-static ssize_t pinger_show(struct kobject *kobj, struct attribute *attr,
-			   char *buf)
-{
-	return sprintf(buf, "%s\n", "on");
-}
-
-static ssize_t
-health_check_show(struct kobject *kobj, struct attribute *attr, char *buf)
-{
-	bool healthy = true;
-	int i;
-	size_t len = 0;
-
-	if (libcfs_catastrophe)
-		return sprintf(buf, "LBUG\n");
-
-	read_lock(&obd_dev_lock);
-	for (i = 0; i < class_devno_max(); i++) {
-		struct obd_device *obd;
-
-		obd = class_num2obd(i);
-		if (!obd || !obd->obd_attached || !obd->obd_set_up)
-			continue;
-
-		LASSERT(obd->obd_magic == OBD_DEVICE_MAGIC);
-		if (obd->obd_stopping)
-			continue;
-
-		class_incref(obd, __func__, current);
-		read_unlock(&obd_dev_lock);
-
-		if (obd_health_check(NULL, obd))
-			healthy = false;
-		class_decref(obd, __func__, current);
-		read_lock(&obd_dev_lock);
-	}
-	read_unlock(&obd_dev_lock);
-
-	if (healthy)
-		len = sprintf(buf, "healthy\n");
-	else
-		len = sprintf(buf, "NOT HEALTHY\n");
-
-	return len;
-}
-
-static ssize_t jobid_var_show(struct kobject *kobj, struct attribute *attr,
-			      char *buf)
-{
-	return snprintf(buf, PAGE_SIZE, "%s\n", obd_jobid_var);
-}
-
-static ssize_t jobid_var_store(struct kobject *kobj, struct attribute *attr,
-			       const char *buffer,
-			       size_t count)
-{
-	if (!count || count > JOBSTATS_JOBID_VAR_MAX_LEN)
-		return -EINVAL;
-
-	memset(obd_jobid_var, 0, JOBSTATS_JOBID_VAR_MAX_LEN + 1);
-
-	memcpy(obd_jobid_var, buffer, count);
-
-	/* Trim the trailing '\n' if any */
-	if (obd_jobid_var[count - 1] == '\n')
-		obd_jobid_var[count - 1] = 0;
-
-	return count;
-}
-
-static ssize_t jobid_name_show(struct kobject *kobj, struct attribute *attr,
-			       char *buf)
-{
-	return snprintf(buf, PAGE_SIZE, "%s\n", obd_jobid_node);
-}
-
-static ssize_t jobid_name_store(struct kobject *kobj, struct attribute *attr,
-				const char *buffer,
-				size_t count)
-{
-	if (!count || count > LUSTRE_JOBID_SIZE)
-		return -EINVAL;
-
-	memcpy(obd_jobid_node, buffer, count);
-
-	obd_jobid_node[count] = 0;
-
-	/* Trim the trailing '\n' if any */
-	if (obd_jobid_node[count - 1] == '\n')
-		obd_jobid_node[count - 1] = 0;
-
-	return count;
-}
-
-/* Root for /sys/kernel/debug/lustre */
-struct dentry *debugfs_lustre_root;
-EXPORT_SYMBOL_GPL(debugfs_lustre_root);
-
-LUSTRE_RO_ATTR(version);
-LUSTRE_RO_ATTR(pinger);
-LUSTRE_RO_ATTR(health_check);
-LUSTRE_RW_ATTR(jobid_var);
-LUSTRE_RW_ATTR(jobid_name);
-
-static struct attribute *lustre_attrs[] = {
-	&lustre_attr_version.attr,
-	&lustre_attr_pinger.attr,
-	&lustre_attr_health_check.attr,
-	&lustre_attr_jobid_name.attr,
-	&lustre_attr_jobid_var.attr,
-	NULL,
-};
-
-static void *obd_device_list_seq_start(struct seq_file *p, loff_t *pos)
-{
-	if (*pos >= class_devno_max())
-		return NULL;
-
-	return pos;
-}
-
-static void obd_device_list_seq_stop(struct seq_file *p, void *v)
-{
-}
-
-static void *obd_device_list_seq_next(struct seq_file *p, void *v, loff_t *pos)
-{
-	++*pos;
-	if (*pos >= class_devno_max())
-		return NULL;
-
-	return pos;
-}
-
-static int obd_device_list_seq_show(struct seq_file *p, void *v)
-{
-	loff_t index = *(loff_t *)v;
-	struct obd_device *obd = class_num2obd((int)index);
-	char *status;
-
-	if (!obd)
-		return 0;
-
-	LASSERT(obd->obd_magic == OBD_DEVICE_MAGIC);
-	if (obd->obd_stopping)
-		status = "ST";
-	else if (obd->obd_inactive)
-		status = "IN";
-	else if (obd->obd_set_up)
-		status = "UP";
-	else if (obd->obd_attached)
-		status = "AT";
-	else
-		status = "--";
-
-	seq_printf(p, "%3d %s %s %s %s %d\n",
-		   (int)index, status, obd->obd_type->typ_name,
-		   obd->obd_name, obd->obd_uuid.uuid,
-		   atomic_read(&obd->obd_refcount));
-	return 0;
-}
-
-static const struct seq_operations obd_device_list_sops = {
-	.start = obd_device_list_seq_start,
-	.stop = obd_device_list_seq_stop,
-	.next = obd_device_list_seq_next,
-	.show = obd_device_list_seq_show,
-};
-
-static int obd_device_list_open(struct inode *inode, struct file *file)
-{
-	struct seq_file *seq;
-	int rc = seq_open(file, &obd_device_list_sops);
-
-	if (rc)
-		return rc;
-
-	seq = file->private_data;
-	seq->private = inode->i_private;
-
-	return 0;
-}
-
-static const struct file_operations obd_device_list_fops = {
-	.owner   = THIS_MODULE,
-	.open    = obd_device_list_open,
-	.read    = seq_read,
-	.llseek  = seq_lseek,
-	.release = seq_release,
-};
-
-struct kobject *lustre_kobj;
-EXPORT_SYMBOL_GPL(lustre_kobj);
-
-static const struct attribute_group lustre_attr_group = {
-	.attrs = lustre_attrs,
-};
-
-int class_procfs_init(void)
-{
-	int rc = -ENOMEM;
-
-	lustre_kobj = kobject_create_and_add("lustre", fs_kobj);
-	if (!lustre_kobj)
-		goto out;
-
-	/* Create the files associated with this kobject */
-	rc = sysfs_create_group(lustre_kobj, &lustre_attr_group);
-	if (rc) {
-		kobject_put(lustre_kobj);
-		goto out;
-	}
-
-	debugfs_lustre_root = debugfs_create_dir("lustre", NULL);
-
-	debugfs_create_file("devices", 0444, debugfs_lustre_root, NULL,
-			    &obd_device_list_fops);
-out:
-	return rc;
-}
-
-int class_procfs_clean(void)
-{
-	debugfs_remove_recursive(debugfs_lustre_root);
-
-	debugfs_lustre_root = NULL;
-
-	sysfs_remove_group(lustre_kobj, &lustre_attr_group);
-	kobject_put(lustre_kobj);
-
-	return 0;
-}
diff --git a/drivers/staging/lustre/lustre/obdclass/linux/linux-sysctl.c b/drivers/staging/lustre/lustre/obdclass/linux/linux-sysctl.c
deleted file mode 100644
index e5e8687784ee..000000000000
--- a/drivers/staging/lustre/lustre/obdclass/linux/linux-sysctl.c
+++ /dev/null
@@ -1,162 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-/*
- * GPL HEADER START
- *
- * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 only,
- * as published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful, but
- * WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
- * General Public License version 2 for more details (a copy is included
- * in the LICENSE file that accompanied this code).
- *
- * You should have received a copy of the GNU General Public License
- * version 2 along with this program; If not, see
- * http://www.gnu.org/licenses/gpl-2.0.html
- *
- * GPL HEADER END
- */
-/*
- * Copyright (c) 1999, 2010, Oracle and/or its affiliates. All rights reserved.
- * Use is subject to license terms.
- *
- * Copyright (c) 2011, 2015, Intel Corporation.
- */
-/*
- * This file is part of Lustre, http://www.lustre.org/
- * Lustre is a trademark of Sun Microsystems, Inc.
- */
-
-#include <linux/module.h>
-#include <linux/sysctl.h>
-#include <linux/sched.h>
-#include <linux/mm.h>
-#include <linux/slab.h>
-#include <linux/stat.h>
-#include <linux/ctype.h>
-#include <linux/bitops.h>
-#include <linux/uaccess.h>
-#include <linux/utsname.h>
-
-#define DEBUG_SUBSYSTEM S_CLASS
-
-#include <obd_support.h>
-#include <lprocfs_status.h>
-#include <obd_class.h>
-
-struct static_lustre_uintvalue_attr {
-	struct {
-		struct attribute attr;
-		ssize_t (*show)(struct kobject *kobj, struct attribute *attr,
-				char *buf);
-		ssize_t (*store)(struct kobject *kobj, struct attribute *attr,
-				 const char *buf, size_t len);
-	} u;
-	int *value;
-};
-
-static ssize_t static_uintvalue_show(struct kobject *kobj,
-				     struct attribute *attr,
-				     char *buf)
-{
-	struct static_lustre_uintvalue_attr *lattr = (void *)attr;
-
-	return sprintf(buf, "%d\n", *lattr->value);
-}
-
-static ssize_t static_uintvalue_store(struct kobject *kobj,
-				      struct attribute *attr,
-				      const char *buffer, size_t count)
-{
-	struct static_lustre_uintvalue_attr *lattr  = (void *)attr;
-	int rc;
-	unsigned int val;
-
-	rc = kstrtouint(buffer, 10, &val);
-	if (rc)
-		return rc;
-
-	*lattr->value = val;
-
-	return count;
-}
-
-#define LUSTRE_STATIC_UINT_ATTR(name, value) \
-static struct static_lustre_uintvalue_attr lustre_sattr_##name =	\
-					{__ATTR(name, 0644,		\
-						static_uintvalue_show,	\
-						static_uintvalue_store),\
-					  value }
-
-LUSTRE_STATIC_UINT_ATTR(timeout, &obd_timeout);
-
-static ssize_t max_dirty_mb_show(struct kobject *kobj, struct attribute *attr,
-				 char *buf)
-{
-	return sprintf(buf, "%lu\n",
-		       obd_max_dirty_pages / (1 << (20 - PAGE_SHIFT)));
-}
-
-static ssize_t max_dirty_mb_store(struct kobject *kobj, struct attribute *attr,
-				  const char *buffer, size_t count)
-{
-	int rc;
-	unsigned long val;
-
-	rc = kstrtoul(buffer, 10, &val);
-	if (rc)
-		return rc;
-
-	val *= 1 << (20 - PAGE_SHIFT); /* convert to pages */
-
-	if (val > ((totalram_pages / 10) * 9)) {
-		/* Somebody wants to assign too much memory to dirty pages */
-		return -EINVAL;
-	}
-
-	if (val < 4 << (20 - PAGE_SHIFT)) {
-		/* Less than 4 Mb for dirty cache is also bad */
-		return -EINVAL;
-	}
-
-	obd_max_dirty_pages = val;
-
-	return count;
-}
-LUSTRE_RW_ATTR(max_dirty_mb);
-
-LUSTRE_STATIC_UINT_ATTR(debug_peer_on_timeout, &obd_debug_peer_on_timeout);
-LUSTRE_STATIC_UINT_ATTR(dump_on_timeout, &obd_dump_on_timeout);
-LUSTRE_STATIC_UINT_ATTR(dump_on_eviction, &obd_dump_on_eviction);
-LUSTRE_STATIC_UINT_ATTR(at_min, &at_min);
-LUSTRE_STATIC_UINT_ATTR(at_max, &at_max);
-LUSTRE_STATIC_UINT_ATTR(at_extra, &at_extra);
-LUSTRE_STATIC_UINT_ATTR(at_early_margin, &at_early_margin);
-LUSTRE_STATIC_UINT_ATTR(at_history, &at_history);
-
-static struct attribute *lustre_attrs[] = {
-	&lustre_sattr_timeout.u.attr,
-	&lustre_attr_max_dirty_mb.attr,
-	&lustre_sattr_debug_peer_on_timeout.u.attr,
-	&lustre_sattr_dump_on_timeout.u.attr,
-	&lustre_sattr_dump_on_eviction.u.attr,
-	&lustre_sattr_at_min.u.attr,
-	&lustre_sattr_at_max.u.attr,
-	&lustre_sattr_at_extra.u.attr,
-	&lustre_sattr_at_early_margin.u.attr,
-	&lustre_sattr_at_history.u.attr,
-	NULL,
-};
-
-static const struct attribute_group lustre_attr_group = {
-	.attrs = lustre_attrs,
-};
-
-int obd_sysctl_init(void)
-{
-	return sysfs_create_group(lustre_kobj, &lustre_attr_group);
-}
diff --git a/drivers/staging/lustre/lustre/obdclass/module.c b/drivers/staging/lustre/lustre/obdclass/module.c
new file mode 100644
index 000000000000..9c800580053b
--- /dev/null
+++ b/drivers/staging/lustre/lustre/obdclass/module.c
@@ -0,0 +1,514 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * GPL HEADER START
+ *
+ * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 only,
+ * as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License version 2 for more details (a copy is included
+ * in the LICENSE file that accompanied this code).
+ *
+ * You should have received a copy of the GNU General Public License
+ * version 2 along with this program; If not, see
+ * http://www.gnu.org/licenses/gpl-2.0.html
+ *
+ * GPL HEADER END
+ */
+/*
+ * Copyright (c) 2007, 2010, Oracle and/or its affiliates. All rights reserved.
+ * Use is subject to license terms.
+ *
+ * Copyright (c) 2011, 2012, Intel Corporation.
+ */
+/*
+ * This file is part of Lustre, http://www.lustre.org/
+ * Lustre is a trademark of Sun Microsystems, Inc.
+ *
+ * lustre/obdclass/linux/linux-module.c
+ *
+ * Object Devices Class Driver
+ * These are the only exported functions, they provide some generic
+ * infrastructure for managing object devices
+ */
+
+#define DEBUG_SUBSYSTEM S_CLASS
+
+#include <linux/module.h>
+#include <linux/errno.h>
+#include <linux/kernel.h>
+#include <linux/sched.h>
+#include <linux/lp.h>
+#include <linux/slab.h>
+#include <linux/ioport.h>
+#include <linux/fcntl.h>
+#include <linux/delay.h>
+#include <linux/skbuff.h>
+#include <linux/fs.h>
+#include <linux/poll.h>
+#include <linux/list.h>
+#include <linux/highmem.h>
+#include <linux/io.h>
+#include <asm/ioctls.h>
+#include <linux/uaccess.h>
+#include <linux/miscdevice.h>
+#include <linux/seq_file.h>
+#include <linux/kobject.h>
+
+#include <uapi/linux/lnet/lnetctl.h>
+#include <obd_support.h>
+#include <obd_class.h>
+#include <lprocfs_status.h>
+#include <uapi/linux/lustre/lustre_ioctl.h>
+#include <uapi/linux/lustre/lustre_ver.h>
+
+#define OBD_MAX_IOCTL_BUFFER	8192
+
+static int obd_ioctl_is_invalid(struct obd_ioctl_data *data)
+{
+	if (data->ioc_len > BIT(30)) {
+		CERROR("OBD ioctl: ioc_len larger than 1<<30\n");
+		return 1;
+	}
+
+	if (data->ioc_inllen1 > BIT(30)) {
+		CERROR("OBD ioctl: ioc_inllen1 larger than 1<<30\n");
+		return 1;
+	}
+
+	if (data->ioc_inllen2 > BIT(30)) {
+		CERROR("OBD ioctl: ioc_inllen2 larger than 1<<30\n");
+		return 1;
+	}
+
+	if (data->ioc_inllen3 > BIT(30)) {
+		CERROR("OBD ioctl: ioc_inllen3 larger than 1<<30\n");
+		return 1;
+	}
+
+	if (data->ioc_inllen4 > BIT(30)) {
+		CERROR("OBD ioctl: ioc_inllen4 larger than 1<<30\n");
+		return 1;
+	}
+
+	if (data->ioc_inlbuf1 && data->ioc_inllen1 == 0) {
+		CERROR("OBD ioctl: inlbuf1 pointer but 0 length\n");
+		return 1;
+	}
+
+	if (data->ioc_inlbuf2 && data->ioc_inllen2 == 0) {
+		CERROR("OBD ioctl: inlbuf2 pointer but 0 length\n");
+		return 1;
+	}
+
+	if (data->ioc_inlbuf3 && data->ioc_inllen3 == 0) {
+		CERROR("OBD ioctl: inlbuf3 pointer but 0 length\n");
+		return 1;
+	}
+
+	if (data->ioc_inlbuf4 && data->ioc_inllen4 == 0) {
+		CERROR("OBD ioctl: inlbuf4 pointer but 0 length\n");
+		return 1;
+	}
+
+	if (data->ioc_pbuf1 && data->ioc_plen1 == 0) {
+		CERROR("OBD ioctl: pbuf1 pointer but 0 length\n");
+		return 1;
+	}
+
+	if (data->ioc_pbuf2 && data->ioc_plen2 == 0) {
+		CERROR("OBD ioctl: pbuf2 pointer but 0 length\n");
+		return 1;
+	}
+
+	if (!data->ioc_pbuf1 && data->ioc_plen1 != 0) {
+		CERROR("OBD ioctl: plen1 set but NULL pointer\n");
+		return 1;
+	}
+
+	if (!data->ioc_pbuf2 && data->ioc_plen2 != 0) {
+		CERROR("OBD ioctl: plen2 set but NULL pointer\n");
+		return 1;
+	}
+
+	if (obd_ioctl_packlen(data) > data->ioc_len) {
+		CERROR("OBD ioctl: packlen exceeds ioc_len (%d > %d)\n",
+		obd_ioctl_packlen(data), data->ioc_len);
+		return 1;
+	}
+
+	return 0;
+}
+
+/* buffer MUST be at least the size of obd_ioctl_hdr */
+int obd_ioctl_getdata(char **buf, int *len, void __user *arg)
+{
+	struct obd_ioctl_hdr hdr;
+	struct obd_ioctl_data *data;
+	int err;
+	int offset = 0;
+
+	if (copy_from_user(&hdr, arg, sizeof(hdr)))
+		return -EFAULT;
+
+	if (hdr.ioc_version != OBD_IOCTL_VERSION) {
+		CERROR("Version mismatch kernel (%x) vs application (%x)\n",
+		       OBD_IOCTL_VERSION, hdr.ioc_version);
+		return -EINVAL;
+	}
+
+	if (hdr.ioc_len > OBD_MAX_IOCTL_BUFFER) {
+		CERROR("User buffer len %d exceeds %d max buffer\n",
+		       hdr.ioc_len, OBD_MAX_IOCTL_BUFFER);
+		return -EINVAL;
+	}
+
+	if (hdr.ioc_len < sizeof(struct obd_ioctl_data)) {
+		CERROR("User buffer too small for ioctl (%d)\n", hdr.ioc_len);
+		return -EINVAL;
+	}
+
+	/* When there are lots of processes calling vmalloc on multi-core
+	 * system, the high lock contention will hurt performance badly,
+	 * obdfilter-survey is an example, which relies on ioctl. So we'd
+	 * better avoid vmalloc on ioctl path. LU-66
+	 */
+	*buf = kvzalloc(hdr.ioc_len, GFP_KERNEL);
+	if (!*buf) {
+		CERROR("Cannot allocate control buffer of len %d\n",
+		       hdr.ioc_len);
+		return -EINVAL;
+	}
+	*len = hdr.ioc_len;
+	data = (struct obd_ioctl_data *)*buf;
+
+	if (copy_from_user(*buf, arg, hdr.ioc_len)) {
+		err = -EFAULT;
+		goto free_buf;
+	}
+	if (hdr.ioc_len != data->ioc_len) {
+		err = -EINVAL;
+		goto free_buf;
+	}
+
+	if (obd_ioctl_is_invalid(data)) {
+		CERROR("ioctl not correctly formatted\n");
+		err = -EINVAL;
+		goto free_buf;
+	}
+
+	if (data->ioc_inllen1) {
+		data->ioc_inlbuf1 = &data->ioc_bulk[0];
+		offset += cfs_size_round(data->ioc_inllen1);
+	}
+
+	if (data->ioc_inllen2) {
+		data->ioc_inlbuf2 = &data->ioc_bulk[0] + offset;
+		offset += cfs_size_round(data->ioc_inllen2);
+	}
+
+	if (data->ioc_inllen3) {
+		data->ioc_inlbuf3 = &data->ioc_bulk[0] + offset;
+		offset += cfs_size_round(data->ioc_inllen3);
+	}
+
+	if (data->ioc_inllen4)
+		data->ioc_inlbuf4 = &data->ioc_bulk[0] + offset;
+
+	return 0;
+
+free_buf:
+	kvfree(*buf);
+	return err;
+}
+EXPORT_SYMBOL(obd_ioctl_getdata);
+
+/*  opening /dev/obd */
+static int obd_class_open(struct inode *inode, struct file *file)
+{
+	try_module_get(THIS_MODULE);
+	return 0;
+}
+
+/*  closing /dev/obd */
+static int obd_class_release(struct inode *inode, struct file *file)
+{
+	module_put(THIS_MODULE);
+	return 0;
+}
+
+/* to control /dev/obd */
+static long obd_class_ioctl(struct file *filp, unsigned int cmd,
+			    unsigned long arg)
+{
+	int err = 0;
+
+	/* Allow non-root access for OBD_IOC_PING_TARGET - used by lfs check */
+	if (!capable(CAP_SYS_ADMIN) && (cmd != OBD_IOC_PING_TARGET))
+		return err = -EACCES;
+	if ((cmd & 0xffffff00) == ((int)'T') << 8) /* ignore all tty ioctls */
+		return err = -ENOTTY;
+
+	err = class_handle_ioctl(cmd, (unsigned long)arg);
+
+	return err;
+}
+
+/* declare character device */
+static const struct file_operations obd_psdev_fops = {
+	.owner	  = THIS_MODULE,
+	.unlocked_ioctl = obd_class_ioctl, /* unlocked_ioctl */
+	.open	   = obd_class_open,      /* open */
+	.release	= obd_class_release,   /* release */
+};
+
+/* modules setup */
+struct miscdevice obd_psdev = {
+	.minor = MISC_DYNAMIC_MINOR,
+	.name  = OBD_DEV_NAME,
+	.fops  = &obd_psdev_fops,
+};
+
+static ssize_t version_show(struct kobject *kobj, struct attribute *attr,
+			    char *buf)
+{
+	return sprintf(buf, "%s\n", LUSTRE_VERSION_STRING);
+}
+
+static ssize_t pinger_show(struct kobject *kobj, struct attribute *attr,
+			   char *buf)
+{
+	return sprintf(buf, "%s\n", "on");
+}
+
+static ssize_t
+health_check_show(struct kobject *kobj, struct attribute *attr, char *buf)
+{
+	bool healthy = true;
+	int i;
+	size_t len = 0;
+
+	if (libcfs_catastrophe)
+		return sprintf(buf, "LBUG\n");
+
+	read_lock(&obd_dev_lock);
+	for (i = 0; i < class_devno_max(); i++) {
+		struct obd_device *obd;
+
+		obd = class_num2obd(i);
+		if (!obd || !obd->obd_attached || !obd->obd_set_up)
+			continue;
+
+		LASSERT(obd->obd_magic == OBD_DEVICE_MAGIC);
+		if (obd->obd_stopping)
+			continue;
+
+		class_incref(obd, __func__, current);
+		read_unlock(&obd_dev_lock);
+
+		if (obd_health_check(NULL, obd))
+			healthy = false;
+		class_decref(obd, __func__, current);
+		read_lock(&obd_dev_lock);
+	}
+	read_unlock(&obd_dev_lock);
+
+	if (healthy)
+		len = sprintf(buf, "healthy\n");
+	else
+		len = sprintf(buf, "NOT HEALTHY\n");
+
+	return len;
+}
+
+static ssize_t jobid_var_show(struct kobject *kobj, struct attribute *attr,
+			      char *buf)
+{
+	return snprintf(buf, PAGE_SIZE, "%s\n", obd_jobid_var);
+}
+
+static ssize_t jobid_var_store(struct kobject *kobj, struct attribute *attr,
+			       const char *buffer,
+			       size_t count)
+{
+	if (!count || count > JOBSTATS_JOBID_VAR_MAX_LEN)
+		return -EINVAL;
+
+	memset(obd_jobid_var, 0, JOBSTATS_JOBID_VAR_MAX_LEN + 1);
+
+	memcpy(obd_jobid_var, buffer, count);
+
+	/* Trim the trailing '\n' if any */
+	if (obd_jobid_var[count - 1] == '\n')
+		obd_jobid_var[count - 1] = 0;
+
+	return count;
+}
+
+static ssize_t jobid_name_show(struct kobject *kobj, struct attribute *attr,
+			       char *buf)
+{
+	return snprintf(buf, PAGE_SIZE, "%s\n", obd_jobid_node);
+}
+
+static ssize_t jobid_name_store(struct kobject *kobj, struct attribute *attr,
+				const char *buffer,
+				size_t count)
+{
+	if (!count || count > LUSTRE_JOBID_SIZE)
+		return -EINVAL;
+
+	memcpy(obd_jobid_node, buffer, count);
+
+	obd_jobid_node[count] = 0;
+
+	/* Trim the trailing '\n' if any */
+	if (obd_jobid_node[count - 1] == '\n')
+		obd_jobid_node[count - 1] = 0;
+
+	return count;
+}
+
+/* Root for /sys/kernel/debug/lustre */
+struct dentry *debugfs_lustre_root;
+EXPORT_SYMBOL_GPL(debugfs_lustre_root);
+
+LUSTRE_RO_ATTR(version);
+LUSTRE_RO_ATTR(pinger);
+LUSTRE_RO_ATTR(health_check);
+LUSTRE_RW_ATTR(jobid_var);
+LUSTRE_RW_ATTR(jobid_name);
+
+static struct attribute *lustre_attrs[] = {
+	&lustre_attr_version.attr,
+	&lustre_attr_pinger.attr,
+	&lustre_attr_health_check.attr,
+	&lustre_attr_jobid_name.attr,
+	&lustre_attr_jobid_var.attr,
+	NULL,
+};
+
+static void *obd_device_list_seq_start(struct seq_file *p, loff_t *pos)
+{
+	if (*pos >= class_devno_max())
+		return NULL;
+
+	return pos;
+}
+
+static void obd_device_list_seq_stop(struct seq_file *p, void *v)
+{
+}
+
+static void *obd_device_list_seq_next(struct seq_file *p, void *v, loff_t *pos)
+{
+	++*pos;
+	if (*pos >= class_devno_max())
+		return NULL;
+
+	return pos;
+}
+
+static int obd_device_list_seq_show(struct seq_file *p, void *v)
+{
+	loff_t index = *(loff_t *)v;
+	struct obd_device *obd = class_num2obd((int)index);
+	char *status;
+
+	if (!obd)
+		return 0;
+
+	LASSERT(obd->obd_magic == OBD_DEVICE_MAGIC);
+	if (obd->obd_stopping)
+		status = "ST";
+	else if (obd->obd_inactive)
+		status = "IN";
+	else if (obd->obd_set_up)
+		status = "UP";
+	else if (obd->obd_attached)
+		status = "AT";
+	else
+		status = "--";
+
+	seq_printf(p, "%3d %s %s %s %s %d\n",
+		   (int)index, status, obd->obd_type->typ_name,
+		   obd->obd_name, obd->obd_uuid.uuid,
+		   atomic_read(&obd->obd_refcount));
+	return 0;
+}
+
+static const struct seq_operations obd_device_list_sops = {
+	.start = obd_device_list_seq_start,
+	.stop = obd_device_list_seq_stop,
+	.next = obd_device_list_seq_next,
+	.show = obd_device_list_seq_show,
+};
+
+static int obd_device_list_open(struct inode *inode, struct file *file)
+{
+	struct seq_file *seq;
+	int rc = seq_open(file, &obd_device_list_sops);
+
+	if (rc)
+		return rc;
+
+	seq = file->private_data;
+	seq->private = inode->i_private;
+
+	return 0;
+}
+
+static const struct file_operations obd_device_list_fops = {
+	.owner   = THIS_MODULE,
+	.open    = obd_device_list_open,
+	.read    = seq_read,
+	.llseek  = seq_lseek,
+	.release = seq_release,
+};
+
+struct kobject *lustre_kobj;
+EXPORT_SYMBOL_GPL(lustre_kobj);
+
+static const struct attribute_group lustre_attr_group = {
+	.attrs = lustre_attrs,
+};
+
+int class_procfs_init(void)
+{
+	int rc = -ENOMEM;
+
+	lustre_kobj = kobject_create_and_add("lustre", fs_kobj);
+	if (!lustre_kobj)
+		goto out;
+
+	/* Create the files associated with this kobject */
+	rc = sysfs_create_group(lustre_kobj, &lustre_attr_group);
+	if (rc) {
+		kobject_put(lustre_kobj);
+		goto out;
+	}
+
+	debugfs_lustre_root = debugfs_create_dir("lustre", NULL);
+
+	debugfs_create_file("devices", 0444, debugfs_lustre_root, NULL,
+			    &obd_device_list_fops);
+out:
+	return rc;
+}
+
+int class_procfs_clean(void)
+{
+	debugfs_remove_recursive(debugfs_lustre_root);
+
+	debugfs_lustre_root = NULL;
+
+	sysfs_remove_group(lustre_kobj, &lustre_attr_group);
+	kobject_put(lustre_kobj);
+
+	return 0;
+}
diff --git a/drivers/staging/lustre/lustre/obdclass/sysctl.c b/drivers/staging/lustre/lustre/obdclass/sysctl.c
new file mode 100644
index 000000000000..e5e8687784ee
--- /dev/null
+++ b/drivers/staging/lustre/lustre/obdclass/sysctl.c
@@ -0,0 +1,162 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * GPL HEADER START
+ *
+ * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 only,
+ * as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License version 2 for more details (a copy is included
+ * in the LICENSE file that accompanied this code).
+ *
+ * You should have received a copy of the GNU General Public License
+ * version 2 along with this program; If not, see
+ * http://www.gnu.org/licenses/gpl-2.0.html
+ *
+ * GPL HEADER END
+ */
+/*
+ * Copyright (c) 1999, 2010, Oracle and/or its affiliates. All rights reserved.
+ * Use is subject to license terms.
+ *
+ * Copyright (c) 2011, 2015, Intel Corporation.
+ */
+/*
+ * This file is part of Lustre, http://www.lustre.org/
+ * Lustre is a trademark of Sun Microsystems, Inc.
+ */
+
+#include <linux/module.h>
+#include <linux/sysctl.h>
+#include <linux/sched.h>
+#include <linux/mm.h>
+#include <linux/slab.h>
+#include <linux/stat.h>
+#include <linux/ctype.h>
+#include <linux/bitops.h>
+#include <linux/uaccess.h>
+#include <linux/utsname.h>
+
+#define DEBUG_SUBSYSTEM S_CLASS
+
+#include <obd_support.h>
+#include <lprocfs_status.h>
+#include <obd_class.h>
+
+struct static_lustre_uintvalue_attr {
+	struct {
+		struct attribute attr;
+		ssize_t (*show)(struct kobject *kobj, struct attribute *attr,
+				char *buf);
+		ssize_t (*store)(struct kobject *kobj, struct attribute *attr,
+				 const char *buf, size_t len);
+	} u;
+	int *value;
+};
+
+static ssize_t static_uintvalue_show(struct kobject *kobj,
+				     struct attribute *attr,
+				     char *buf)
+{
+	struct static_lustre_uintvalue_attr *lattr = (void *)attr;
+
+	return sprintf(buf, "%d\n", *lattr->value);
+}
+
+static ssize_t static_uintvalue_store(struct kobject *kobj,
+				      struct attribute *attr,
+				      const char *buffer, size_t count)
+{
+	struct static_lustre_uintvalue_attr *lattr  = (void *)attr;
+	int rc;
+	unsigned int val;
+
+	rc = kstrtouint(buffer, 10, &val);
+	if (rc)
+		return rc;
+
+	*lattr->value = val;
+
+	return count;
+}
+
+#define LUSTRE_STATIC_UINT_ATTR(name, value) \
+static struct static_lustre_uintvalue_attr lustre_sattr_##name =	\
+					{__ATTR(name, 0644,		\
+						static_uintvalue_show,	\
+						static_uintvalue_store),\
+					  value }
+
+LUSTRE_STATIC_UINT_ATTR(timeout, &obd_timeout);
+
+static ssize_t max_dirty_mb_show(struct kobject *kobj, struct attribute *attr,
+				 char *buf)
+{
+	return sprintf(buf, "%lu\n",
+		       obd_max_dirty_pages / (1 << (20 - PAGE_SHIFT)));
+}
+
+static ssize_t max_dirty_mb_store(struct kobject *kobj, struct attribute *attr,
+				  const char *buffer, size_t count)
+{
+	int rc;
+	unsigned long val;
+
+	rc = kstrtoul(buffer, 10, &val);
+	if (rc)
+		return rc;
+
+	val *= 1 << (20 - PAGE_SHIFT); /* convert to pages */
+
+	if (val > ((totalram_pages / 10) * 9)) {
+		/* Somebody wants to assign too much memory to dirty pages */
+		return -EINVAL;
+	}
+
+	if (val < 4 << (20 - PAGE_SHIFT)) {
+		/* Less than 4 Mb for dirty cache is also bad */
+		return -EINVAL;
+	}
+
+	obd_max_dirty_pages = val;
+
+	return count;
+}
+LUSTRE_RW_ATTR(max_dirty_mb);
+
+LUSTRE_STATIC_UINT_ATTR(debug_peer_on_timeout, &obd_debug_peer_on_timeout);
+LUSTRE_STATIC_UINT_ATTR(dump_on_timeout, &obd_dump_on_timeout);
+LUSTRE_STATIC_UINT_ATTR(dump_on_eviction, &obd_dump_on_eviction);
+LUSTRE_STATIC_UINT_ATTR(at_min, &at_min);
+LUSTRE_STATIC_UINT_ATTR(at_max, &at_max);
+LUSTRE_STATIC_UINT_ATTR(at_extra, &at_extra);
+LUSTRE_STATIC_UINT_ATTR(at_early_margin, &at_early_margin);
+LUSTRE_STATIC_UINT_ATTR(at_history, &at_history);
+
+static struct attribute *lustre_attrs[] = {
+	&lustre_sattr_timeout.u.attr,
+	&lustre_attr_max_dirty_mb.attr,
+	&lustre_sattr_debug_peer_on_timeout.u.attr,
+	&lustre_sattr_dump_on_timeout.u.attr,
+	&lustre_sattr_dump_on_eviction.u.attr,
+	&lustre_sattr_at_min.u.attr,
+	&lustre_sattr_at_max.u.attr,
+	&lustre_sattr_at_extra.u.attr,
+	&lustre_sattr_at_early_margin.u.attr,
+	&lustre_sattr_at_history.u.attr,
+	NULL,
+};
+
+static const struct attribute_group lustre_attr_group = {
+	.attrs = lustre_attrs,
+};
+
+int obd_sysctl_init(void)
+{
+	return sysfs_create_group(lustre_kobj, &lustre_attr_group);
+}



From neilb at suse.com  Wed Jun  6 06:05:19 2018
From: neilb at suse.com (NeilBrown)
Date: Wed, 06 Jun 2018 16:05:19 +1000
Subject: [lustre-devel] [PATCH 09/11] staging: lustre: discard WIRE_ATTR
In-Reply-To: <152826510267.16761.14361003167157833896.stgit@noble>
References: <152826510267.16761.14361003167157833896.stgit@noble>
Message-ID: <152826511920.16761.2460891067961959676.stgit@noble>

This macro adds nothing of value, and make the code harder
to read for new readers.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 .../staging/lustre/include/linux/lnet/socklnd.h    |    8 ++-
 .../lustre/include/uapi/linux/lnet/lnet-types.h    |   28 +++++-------
 .../lustre/include/uapi/linux/lnet/lnetst.h        |    4 +-
 .../staging/lustre/lnet/klnds/o2iblnd/o2iblnd.h    |   22 +++++----
 drivers/staging/lustre/lnet/selftest/rpc.h         |   48 ++++++++++----------
 5 files changed, 54 insertions(+), 56 deletions(-)

diff --git a/drivers/staging/lustre/include/linux/lnet/socklnd.h b/drivers/staging/lustre/include/linux/lnet/socklnd.h
index 6bd1bca190a3..9f69257e000b 100644
--- a/drivers/staging/lustre/include/linux/lnet/socklnd.h
+++ b/drivers/staging/lustre/include/linux/lnet/socklnd.h
@@ -50,7 +50,7 @@ struct ksock_hello_msg {
 	__u32		kshm_ctype;	/* connection type */
 	__u32		kshm_nips;	/* # IP addrs */
 	__u32		kshm_ips[0];	/* IP addrs */
-} WIRE_ATTR;
+} __packed;
 
 struct ksock_lnet_msg {
 	struct lnet_hdr	ksnm_hdr;	/* lnet hdr */
@@ -61,7 +61,7 @@ struct ksock_lnet_msg {
 	 * structure definitions. lnet payload will be stored just after
 	 * the body of structure ksock_lnet_msg_t
 	 */
-} WIRE_ATTR;
+} __packed;
 
 struct ksock_msg {
 	__u32	ksm_type;		/* type of socklnd message */
@@ -71,8 +71,8 @@ struct ksock_msg {
 		struct ksock_lnet_msg lnetmsg; /* lnet message, it's empty if
 						* it's NOOP
 						*/
-	} WIRE_ATTR ksm_u;
-} WIRE_ATTR;
+	} __packed ksm_u;
+} __packed;
 
 #define KSOCK_MSG_NOOP	0xC0	/* ksm_u empty */
 #define KSOCK_MSG_LNET	0xC1	/* lnet msg */
diff --git a/drivers/staging/lustre/include/uapi/linux/lnet/lnet-types.h b/drivers/staging/lustre/include/uapi/linux/lnet/lnet-types.h
index 1be9b7aa7326..f97e7d9d881f 100644
--- a/drivers/staging/lustre/include/uapi/linux/lnet/lnet-types.h
+++ b/drivers/staging/lustre/include/uapi/linux/lnet/lnet-types.h
@@ -112,14 +112,12 @@ static inline __u32 LNET_MKNET(__u32 type, __u32 num)
 	return (type << 16) | num;
 }
 
-#define WIRE_ATTR	__packed
-
 /* Packed version of lnet_process_id to transfer via network */
 struct lnet_process_id_packed {
 	/* node id / process id */
 	lnet_nid_t	nid;
 	lnet_pid_t	pid;
-} WIRE_ATTR;
+} __packed;
 
 /*
  * The wire handle's interface cookie only matches one network interface in
@@ -130,7 +128,7 @@ struct lnet_process_id_packed {
 struct lnet_handle_wire {
 	__u64	wh_interface_cookie;
 	__u64	wh_object_cookie;
-} WIRE_ATTR;
+} __packed;
 
 enum lnet_msg_type {
 	LNET_MSG_ACK = 0,
@@ -150,7 +148,7 @@ struct lnet_ack {
 	struct lnet_handle_wire	dst_wmd;
 	__u64			match_bits;
 	__u32			mlength;
-} WIRE_ATTR;
+} __packed;
 
 struct lnet_put {
 	struct lnet_handle_wire	ack_wmd;
@@ -158,7 +156,7 @@ struct lnet_put {
 	__u64			hdr_data;
 	__u32			ptl_index;
 	__u32			offset;
-} WIRE_ATTR;
+} __packed;
 
 struct lnet_get {
 	struct lnet_handle_wire	return_wmd;
@@ -166,16 +164,16 @@ struct lnet_get {
 	__u32			ptl_index;
 	__u32			src_offset;
 	__u32			sink_length;
-} WIRE_ATTR;
+} __packed;
 
 struct lnet_reply {
 	struct lnet_handle_wire	dst_wmd;
-} WIRE_ATTR;
+} __packed;
 
 struct lnet_hello {
 	__u64			incarnation;
 	__u32			type;
-} WIRE_ATTR;
+} __packed;
 
 struct lnet_hdr {
 	lnet_nid_t	dest_nid;
@@ -192,7 +190,7 @@ struct lnet_hdr {
 		struct lnet_reply	reply;
 		struct lnet_hello	hello;
 	} msg;
-} WIRE_ATTR;
+} __packed;
 
 /*
  * A HELLO message contains a magic number and protocol version
@@ -208,7 +206,7 @@ struct lnet_magicversion {
 	__u32	magic;		/* LNET_PROTO_TCP_MAGIC */
 	__u16	version_major;	/* increment on incompatible change */
 	__u16	version_minor;	/* increment on compatible change */
-} WIRE_ATTR;
+} __packed;
 
 /* PROTO MAGIC for LNDs */
 #define LNET_PROTO_IB_MAGIC		0x0be91b91
@@ -232,7 +230,7 @@ struct lnet_acceptor_connreq {
 	__u32	acr_magic;		/* PTL_ACCEPTOR_PROTO_MAGIC */
 	__u32	acr_version;		/* protocol version */
 	__u64	acr_nid;		/* target NID */
-} WIRE_ATTR;
+} __packed;
 
 #define LNET_PROTO_ACCEPTOR_VERSION	1
 
@@ -240,7 +238,7 @@ struct lnet_ni_status {
 	lnet_nid_t	ns_nid;
 	__u32		ns_status;
 	__u32		ns_unused;
-} WIRE_ATTR;
+} __packed;
 
 struct lnet_ping_info {
 	__u32			pi_magic;
@@ -248,7 +246,7 @@ struct lnet_ping_info {
 	lnet_pid_t		pi_pid;
 	__u32			pi_nnis;
 	struct lnet_ni_status	pi_ni[0];
-} WIRE_ATTR;
+} __packed;
 
 struct lnet_counters {
 	__u32	msgs_alloc;
@@ -262,7 +260,7 @@ struct lnet_counters {
 	__u64	recv_length;
 	__u64	route_length;
 	__u64	drop_length;
-} WIRE_ATTR;
+} __packed;
 
 #define LNET_NI_STATUS_UP      0x15aac0de
 #define LNET_NI_STATUS_DOWN    0xdeadface
diff --git a/drivers/staging/lustre/include/uapi/linux/lnet/lnetst.h b/drivers/staging/lustre/include/uapi/linux/lnet/lnetst.h
index a4f9ff01d458..7edba2c5bb87 100644
--- a/drivers/staging/lustre/include/uapi/linux/lnet/lnetst.h
+++ b/drivers/staging/lustre/include/uapi/linux/lnet/lnetst.h
@@ -542,7 +542,7 @@ struct srpc_counters {
 	__u32 rpcs_expired;
 	__u64 bulk_get;
 	__u64 bulk_put;
-} WIRE_ATTR;
+} __packed;
 
 struct sfw_counters {
 	/** milliseconds since current session started */
@@ -551,6 +551,6 @@ struct sfw_counters {
 	__u32 zombie_sessions;
 	__u32 brw_errors;
 	__u32 ping_errors;
-} WIRE_ATTR;
+} __packed;
 
 #endif
diff --git a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.h b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.h
index 217503f125bc..7d8429672616 100644
--- a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.h
+++ b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.h
@@ -359,45 +359,45 @@ struct kib_connparams {
 	__u16        ibcp_queue_depth;
 	__u16        ibcp_max_frags;
 	__u32        ibcp_max_msg_size;
-} WIRE_ATTR;
+} __packed;
 
 struct kib_immediate_msg {
 	struct lnet_hdr	ibim_hdr;        /* portals header */
 	char         ibim_payload[0]; /* piggy-backed payload */
-} WIRE_ATTR;
+} __packed;
 
 struct kib_rdma_frag {
 	__u32        rf_nob;          /* # bytes this frag */
 	__u64        rf_addr;         /* CAVEAT EMPTOR: misaligned!! */
-} WIRE_ATTR;
+} __packed;
 
 struct kib_rdma_desc {
 	__u32           rd_key;       /* local/remote key */
 	__u32           rd_nfrags;    /* # fragments */
 	struct kib_rdma_frag	rd_frags[0];	/* buffer frags */
-} WIRE_ATTR;
+} __packed;
 
 struct kib_putreq_msg {
 	struct lnet_hdr	ibprm_hdr;    /* portals header */
 	__u64           ibprm_cookie; /* opaque completion cookie */
-} WIRE_ATTR;
+} __packed;
 
 struct kib_putack_msg {
 	__u64           ibpam_src_cookie; /* reflected completion cookie */
 	__u64           ibpam_dst_cookie; /* opaque completion cookie */
 	struct kib_rdma_desc ibpam_rd;         /* sender's sink buffer */
-} WIRE_ATTR;
+} __packed;
 
 struct kib_get_msg {
 	struct lnet_hdr ibgm_hdr;     /* portals header */
 	__u64           ibgm_cookie;  /* opaque completion cookie */
 	struct kib_rdma_desc ibgm_rd;      /* rdma descriptor */
-} WIRE_ATTR;
+} __packed;
 
 struct kib_completion_msg {
 	__u64           ibcm_cookie;  /* opaque completion cookie */
 	__s32           ibcm_status;  /* < 0 failure: >= 0 length */
-} WIRE_ATTR;
+} __packed;
 
 struct kib_msg {
 	/* First 2 fields fixed FOR ALL TIME */
@@ -420,8 +420,8 @@ struct kib_msg {
 		struct kib_putack_msg		putack;
 		struct kib_get_msg		get;
 		struct kib_completion_msg	completion;
-	} WIRE_ATTR ibm_u;
-} WIRE_ATTR;
+	} __packed ibm_u;
+} __packed;
 
 #define IBLND_MSG_MAGIC     LNET_PROTO_IB_MAGIC /* unique magic */
 
@@ -447,7 +447,7 @@ struct kib_rej {
 	__u8             ibr_padding;     /* padding */
 	__u64            ibr_incarnation; /* incarnation of peer */
 	struct kib_connparams ibr_cp;          /* connection parameters */
-} WIRE_ATTR;
+} __packed;
 
 /* connection rejection reasons */
 #define IBLND_REJECT_CONN_RACE      1 /* You lost connection race */
diff --git a/drivers/staging/lustre/lnet/selftest/rpc.h b/drivers/staging/lustre/lnet/selftest/rpc.h
index 465b5b534423..9ce336739449 100644
--- a/drivers/staging/lustre/lnet/selftest/rpc.h
+++ b/drivers/staging/lustre/lnet/selftest/rpc.h
@@ -72,12 +72,12 @@ enum srpc_msg_type {
 struct srpc_generic_reqst {
 	__u64			rpyid;		/* reply buffer matchbits */
 	__u64			bulkid;		/* bulk buffer matchbits */
-} WIRE_ATTR;
+} __packed;
 
 struct srpc_generic_reply {
 	__u32			status;
 	struct lst_sid		sid;
-} WIRE_ATTR;
+} __packed;
 
 /* FRAMEWORK RPCs */
 struct srpc_mksn_reqst {
@@ -85,30 +85,30 @@ struct srpc_mksn_reqst {
 	struct lst_sid		mksn_sid;	/* session id */
 	__u32			mksn_force;	/* use brute force */
 	char			mksn_name[LST_NAME_SIZE];
-} WIRE_ATTR; /* make session request */
+} __packed; /* make session request */
 
 struct srpc_mksn_reply {
 	__u32			mksn_status;	/* session status */
 	struct lst_sid		mksn_sid;	/* session id */
 	__u32			mksn_timeout;	/* session timeout */
 	char			mksn_name[LST_NAME_SIZE];
-} WIRE_ATTR; /* make session reply */
+} __packed; /* make session reply */
 
 struct srpc_rmsn_reqst {
 	__u64			rmsn_rpyid;	/* reply buffer matchbits */
 	struct lst_sid		rmsn_sid;	/* session id */
-} WIRE_ATTR; /* remove session request */
+} __packed; /* remove session request */
 
 struct srpc_rmsn_reply {
 	__u32			rmsn_status;
 	struct lst_sid		rmsn_sid;	/* session id */
-} WIRE_ATTR; /* remove session reply */
+} __packed; /* remove session reply */
 
 struct srpc_join_reqst {
 	__u64			join_rpyid;	/* reply buffer matchbits */
 	struct lst_sid		join_sid;	/* session id to join */
 	char			join_group[LST_NAME_SIZE]; /* group name */
-} WIRE_ATTR;
+} __packed;
 
 struct srpc_join_reply {
 	__u32			join_status;	/* returned status */
@@ -117,13 +117,13 @@ struct srpc_join_reply {
 						 * expire
 						 */
 	char			join_session[LST_NAME_SIZE]; /* session name */
-} WIRE_ATTR;
+} __packed;
 
 struct srpc_debug_reqst {
 	__u64			dbg_rpyid;	/* reply buffer matchbits */
 	struct lst_sid		dbg_sid;	/* session id */
 	__u32			dbg_flags;	/* bitmap of debug */
-} WIRE_ATTR;
+} __packed;
 
 struct srpc_debug_reply {
 	__u32			dbg_status;	/* returned code */
@@ -131,7 +131,7 @@ struct srpc_debug_reply {
 	__u32			dbg_timeout;	/* session timeout */
 	__u32			dbg_nbatch;	/* # of batches in the node */
 	char			dbg_name[LST_NAME_SIZE]; /* session name */
-} WIRE_ATTR;
+} __packed;
 
 #define SRPC_BATCH_OPC_RUN	1
 #define SRPC_BATCH_OPC_STOP	2
@@ -144,20 +144,20 @@ struct srpc_batch_reqst {
 	__u32		   bar_opc;	   /* create/start/stop batch */
 	__u32		   bar_testidx;    /* index of test */
 	__u32		   bar_arg;	   /* parameters */
-} WIRE_ATTR;
+} __packed;
 
 struct srpc_batch_reply {
 	__u32		   bar_status;	   /* status of request */
 	struct lst_sid	   bar_sid;	   /* session id */
 	__u32		   bar_active;	   /* # of active tests in batch/test */
 	__u32		   bar_time;	   /* remained time */
-} WIRE_ATTR;
+} __packed;
 
 struct srpc_stat_reqst {
 	__u64		   str_rpyid;	   /* reply buffer matchbits */
 	struct lst_sid	   str_sid;	   /* session id */
 	__u32		   str_type;	   /* type of stat */
-} WIRE_ATTR;
+} __packed;
 
 struct srpc_stat_reply {
 	__u32		   str_status;
@@ -165,25 +165,25 @@ struct srpc_stat_reply {
 	struct sfw_counters	str_fw;
 	struct srpc_counters	str_rpc;
 	struct lnet_counters    str_lnet;
-} WIRE_ATTR;
+} __packed;
 
 struct test_bulk_req {
 	__u32		   blk_opc;	   /* bulk operation code */
 	__u32		   blk_npg;	   /* # of pages */
 	__u32		   blk_flags;	   /* reserved flags */
-} WIRE_ATTR;
+} __packed;
 
 struct test_bulk_req_v1 {
 	__u16		   blk_opc;	   /* bulk operation code */
 	__u16		   blk_flags;	   /* data check flags */
 	__u32		   blk_len;	   /* data length */
 	__u32		   blk_offset;	   /* offset */
-} WIRE_ATTR;
+} __packed;
 
 struct test_ping_req {
 	__u32		   png_size;	   /* size of ping message */
 	__u32		   png_flags;	   /* reserved flags */
-} WIRE_ATTR;
+} __packed;
 
 struct srpc_test_reqst {
 	__u64			tsr_rpyid;	/* reply buffer matchbits */
@@ -204,12 +204,12 @@ struct srpc_test_reqst {
 		struct test_bulk_req	bulk_v0;
 		struct test_bulk_req_v1	bulk_v1;
 	} tsr_u;
-} WIRE_ATTR;
+} __packed;
 
 struct srpc_test_reply {
 	__u32			tsr_status;	/* returned code */
 	struct lst_sid		tsr_sid;
-} WIRE_ATTR;
+} __packed;
 
 /* TEST RPCs */
 struct srpc_ping_reqst {
@@ -218,13 +218,13 @@ struct srpc_ping_reqst {
 	__u32		   pnr_seq;
 	__u64		   pnr_time_sec;
 	__u64		   pnr_time_usec;
-} WIRE_ATTR;
+} __packed;
 
 struct srpc_ping_reply {
 	__u32		   pnr_status;
 	__u32		   pnr_magic;
 	__u32		   pnr_seq;
-} WIRE_ATTR;
+} __packed;
 
 struct srpc_brw_reqst {
 	__u64		   brw_rpyid;	   /* reply buffer matchbits */
@@ -232,11 +232,11 @@ struct srpc_brw_reqst {
 	__u32		   brw_rw;	   /* read or write */
 	__u32		   brw_len;	   /* bulk data len */
 	__u32		   brw_flags;	   /* bulk data patterns */
-} WIRE_ATTR; /* bulk r/w request */
+} __packed; /* bulk r/w request */
 
 struct srpc_brw_reply {
 	__u32		   brw_status;
-} WIRE_ATTR; /* bulk r/w reply */
+} __packed; /* bulk r/w reply */
 
 #define SRPC_MSG_MAGIC		0xeeb0f00d
 #define SRPC_MSG_VERSION	1
@@ -272,7 +272,7 @@ struct srpc_msg {
 		struct srpc_brw_reqst		brw_reqst;
 		struct srpc_brw_reply		brw_reply;
 	}     msg_body;
-} WIRE_ATTR;
+} __packed;
 
 static inline void
 srpc_unpack_msg_hdr(struct srpc_msg *msg)



From neilb at suse.com  Wed Jun  6 06:05:19 2018
From: neilb at suse.com (NeilBrown)
Date: Wed, 06 Jun 2018 16:05:19 +1000
Subject: [lustre-devel] [PATCH 11/11] staging: lustre: centralize setting of
 subdir-ccflags-y
In-Reply-To: <152826510267.16761.14361003167157833896.stgit@noble>
References: <152826510267.16761.14361003167157833896.stgit@noble>
Message-ID: <152826511930.16761.12770722873423628004.stgit@noble>

We don't need to set subdir-ccflags-y in every Makefile.
The whole point of the "subdir-" bit is that the setting
can go once in the top-level Makefile.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 drivers/staging/lustre/Makefile                    |    3 +++
 drivers/staging/lustre/lnet/klnds/o2iblnd/Makefile |    2 --
 drivers/staging/lustre/lnet/klnds/socklnd/Makefile |    2 --
 drivers/staging/lustre/lnet/libcfs/Makefile        |    2 --
 drivers/staging/lustre/lnet/lnet/Makefile          |    2 --
 drivers/staging/lustre/lnet/selftest/Makefile      |    2 --
 drivers/staging/lustre/lustre/fid/Makefile         |    2 --
 drivers/staging/lustre/lustre/fld/Makefile         |    2 --
 drivers/staging/lustre/lustre/llite/Makefile       |    2 --
 drivers/staging/lustre/lustre/lmv/Makefile         |    2 --
 drivers/staging/lustre/lustre/lov/Makefile         |    2 --
 drivers/staging/lustre/lustre/mdc/Makefile         |    2 --
 drivers/staging/lustre/lustre/mgc/Makefile         |    2 --
 drivers/staging/lustre/lustre/obdclass/Makefile    |    2 --
 drivers/staging/lustre/lustre/obdecho/Makefile     |    2 --
 drivers/staging/lustre/lustre/osc/Makefile         |    2 --
 drivers/staging/lustre/lustre/ptlrpc/Makefile      |    2 --
 17 files changed, 3 insertions(+), 32 deletions(-)

diff --git a/drivers/staging/lustre/Makefile b/drivers/staging/lustre/Makefile
index 95ffe337a80a..a44086fa8668 100644
--- a/drivers/staging/lustre/Makefile
+++ b/drivers/staging/lustre/Makefile
@@ -1,2 +1,5 @@
+subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/include
+subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/lustre/include
+
 obj-$(CONFIG_LNET)		+= lnet/
 obj-$(CONFIG_LUSTRE_FS)		+= lustre/
diff --git a/drivers/staging/lustre/lnet/klnds/o2iblnd/Makefile b/drivers/staging/lustre/lnet/klnds/o2iblnd/Makefile
index 4affe1d79948..e1a05ece130c 100644
--- a/drivers/staging/lustre/lnet/klnds/o2iblnd/Makefile
+++ b/drivers/staging/lustre/lnet/klnds/o2iblnd/Makefile
@@ -1,5 +1,3 @@
-subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/include
-subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/lustre/include
 
 obj-$(CONFIG_LNET_XPRT_IB) += ko2iblnd.o
 ko2iblnd-y := o2iblnd.o o2iblnd_cb.o o2iblnd_modparams.o
diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/Makefile b/drivers/staging/lustre/lnet/klnds/socklnd/Makefile
index a7da1abfc804..4d03cad997c1 100644
--- a/drivers/staging/lustre/lnet/klnds/socklnd/Makefile
+++ b/drivers/staging/lustre/lnet/klnds/socklnd/Makefile
@@ -1,5 +1,3 @@
-subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/include
-subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/lustre/include
 
 obj-$(CONFIG_LNET) += ksocklnd.o
 
diff --git a/drivers/staging/lustre/lnet/libcfs/Makefile b/drivers/staging/lustre/lnet/libcfs/Makefile
index 6a1b232da495..3d6b99c6e883 100644
--- a/drivers/staging/lustre/lnet/libcfs/Makefile
+++ b/drivers/staging/lustre/lnet/libcfs/Makefile
@@ -1,6 +1,4 @@
 # SPDX-License-Identifier: GPL-2.0
-subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/include
-subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/lustre/include
 
 obj-$(CONFIG_LNET) += libcfs.o
 
diff --git a/drivers/staging/lustre/lnet/lnet/Makefile b/drivers/staging/lustre/lnet/lnet/Makefile
index 0a9d70924fe0..ba33e90e47ec 100644
--- a/drivers/staging/lustre/lnet/lnet/Makefile
+++ b/drivers/staging/lustre/lnet/lnet/Makefile
@@ -1,6 +1,4 @@
 # SPDX-License-Identifier: GPL-2.0
-subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/include
-subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/lustre/include
 
 obj-$(CONFIG_LNET) += lnet.o
 
diff --git a/drivers/staging/lustre/lnet/selftest/Makefile b/drivers/staging/lustre/lnet/selftest/Makefile
index 3ccc8966b566..16f8efcd1531 100644
--- a/drivers/staging/lustre/lnet/selftest/Makefile
+++ b/drivers/staging/lustre/lnet/selftest/Makefile
@@ -1,5 +1,3 @@
-subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/include
-subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/lustre/include
 
 obj-$(CONFIG_LNET_SELFTEST) := lnet_selftest.o
 
diff --git a/drivers/staging/lustre/lustre/fid/Makefile b/drivers/staging/lustre/lustre/fid/Makefile
index 77b65b92667d..11f6e4046e61 100644
--- a/drivers/staging/lustre/lustre/fid/Makefile
+++ b/drivers/staging/lustre/lustre/fid/Makefile
@@ -1,5 +1,3 @@
-subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/include
-subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/lustre/include/
 
 obj-$(CONFIG_LUSTRE_FS) += fid.o
 fid-y := fid_request.o fid_lib.o lproc_fid.o
diff --git a/drivers/staging/lustre/lustre/fld/Makefile b/drivers/staging/lustre/lustre/fld/Makefile
index 426deba8b815..e7843d7aac9d 100644
--- a/drivers/staging/lustre/lustre/fld/Makefile
+++ b/drivers/staging/lustre/lustre/fld/Makefile
@@ -1,5 +1,3 @@
-subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/include
-subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/lustre/include/
 
 obj-$(CONFIG_LUSTRE_FS) += fld.o
 fld-y := fld_request.o fld_cache.o lproc_fld.o
diff --git a/drivers/staging/lustre/lustre/llite/Makefile b/drivers/staging/lustre/lustre/llite/Makefile
index 5200924182ae..f46b61770254 100644
--- a/drivers/staging/lustre/lustre/llite/Makefile
+++ b/drivers/staging/lustre/lustre/llite/Makefile
@@ -1,6 +1,4 @@
 # SPDX-License-Identifier: GPL-2.0
-subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/include
-subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/lustre/include
 
 obj-$(CONFIG_LUSTRE_FS) += lustre.o
 lustre-y := dcache.o dir.o file.o llite_lib.o llite_nfs.o \
diff --git a/drivers/staging/lustre/lustre/lmv/Makefile b/drivers/staging/lustre/lustre/lmv/Makefile
index 91c99114aa13..7a392c31c508 100644
--- a/drivers/staging/lustre/lustre/lmv/Makefile
+++ b/drivers/staging/lustre/lustre/lmv/Makefile
@@ -1,5 +1,3 @@
-subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/include
-subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/lustre/include
 
 obj-$(CONFIG_LUSTRE_FS) += lmv.o
 lmv-y := lmv_obd.o lmv_intent.o lmv_fld.o lproc_lmv.o
diff --git a/drivers/staging/lustre/lustre/lov/Makefile b/drivers/staging/lustre/lustre/lov/Makefile
index 1ebf0193f61a..6294769aca95 100644
--- a/drivers/staging/lustre/lustre/lov/Makefile
+++ b/drivers/staging/lustre/lustre/lov/Makefile
@@ -1,6 +1,4 @@
 # SPDX-License-Identifier: GPL-2.0
-subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/include
-subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/lustre/include
 
 obj-$(CONFIG_LUSTRE_FS) += lov.o
 lov-y := lov_obd.o lov_pack.o lov_offset.o lov_merge.o \
diff --git a/drivers/staging/lustre/lustre/mdc/Makefile b/drivers/staging/lustre/lustre/mdc/Makefile
index c7bc3351ccb0..c3bc71f6596d 100644
--- a/drivers/staging/lustre/lustre/mdc/Makefile
+++ b/drivers/staging/lustre/lustre/mdc/Makefile
@@ -1,5 +1,3 @@
-subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/include
-subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/lustre/include
 
 obj-$(CONFIG_LUSTRE_FS) += mdc.o
 mdc-y := mdc_request.o mdc_reint.o mdc_lib.o mdc_locks.o lproc_mdc.o
diff --git a/drivers/staging/lustre/lustre/mgc/Makefile b/drivers/staging/lustre/lustre/mgc/Makefile
index 8abf108dbcf7..301cd7a1fa25 100644
--- a/drivers/staging/lustre/lustre/mgc/Makefile
+++ b/drivers/staging/lustre/lustre/mgc/Makefile
@@ -1,5 +1,3 @@
-subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/include
-subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/lustre/include
 
 obj-$(CONFIG_LUSTRE_FS) += mgc.o
 mgc-y := mgc_request.o lproc_mgc.o
diff --git a/drivers/staging/lustre/lustre/obdclass/Makefile b/drivers/staging/lustre/lustre/obdclass/Makefile
index e36ba2167d10..be118391b402 100644
--- a/drivers/staging/lustre/lustre/obdclass/Makefile
+++ b/drivers/staging/lustre/lustre/obdclass/Makefile
@@ -1,6 +1,4 @@
 # SPDX-License-Identifier: GPL-2.0
-subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/include
-subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/lustre/include
 
 obj-$(CONFIG_LUSTRE_FS) += obdclass.o
 
diff --git a/drivers/staging/lustre/lustre/obdecho/Makefile b/drivers/staging/lustre/lustre/obdecho/Makefile
index 6be66fbab872..4ae346f5ddcd 100644
--- a/drivers/staging/lustre/lustre/obdecho/Makefile
+++ b/drivers/staging/lustre/lustre/obdecho/Makefile
@@ -1,5 +1,3 @@
-subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/include
-subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/lustre/include
 
 obj-$(CONFIG_LUSTRE_FS) += obdecho.o
 obdecho-y := echo_client.o
diff --git a/drivers/staging/lustre/lustre/osc/Makefile b/drivers/staging/lustre/lustre/osc/Makefile
index 30dec90e64e8..ecdf07e925c7 100644
--- a/drivers/staging/lustre/lustre/osc/Makefile
+++ b/drivers/staging/lustre/lustre/osc/Makefile
@@ -1,5 +1,3 @@
-subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/include
-subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/lustre/include
 
 obj-$(CONFIG_LUSTRE_FS) += osc.o
 osc-y := osc_request.o osc_dev.o osc_object.o \
diff --git a/drivers/staging/lustre/lustre/ptlrpc/Makefile b/drivers/staging/lustre/lustre/ptlrpc/Makefile
index aa152e652002..0fae7de09de2 100644
--- a/drivers/staging/lustre/lustre/ptlrpc/Makefile
+++ b/drivers/staging/lustre/lustre/ptlrpc/Makefile
@@ -1,6 +1,4 @@
 # SPDX-License-Identifier: GPL-2.0
-subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/include
-subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/lustre/include
 
 obj-$(CONFIG_LUSTRE_FS) += ptlrpc.o
 



From neilb at suse.com  Wed Jun  6 06:05:19 2018
From: neilb at suse.com (NeilBrown)
Date: Wed, 06 Jun 2018 16:05:19 +1000
Subject: [lustre-devel] [PATCH 10/11] staging: lustre: move ldlm into ptlrpc
In-Reply-To: <152826510267.16761.14361003167157833896.stgit@noble>
References: <152826510267.16761.14361003167157833896.stgit@noble>
Message-ID: <152826511923.16761.9237280635711887801.stgit@noble>

The ldlm code is built into the ptlrpc module, yet it lived in a
separate directory.  This requires filename editing in the Makefile
and make it difficult to e.g. build the .s file for code in ldlm.

All the ldlm files have distinctive names so confusion from having
ptlrpc and ldlm in the same directory is unlikely.  So move them all
into ptlrpc.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 drivers/staging/lustre/lustre/ldlm/l_lock.c        |   73 -
 drivers/staging/lustre/lustre/ldlm/ldlm_extent.c   |  206 --
 drivers/staging/lustre/lustre/ldlm/ldlm_flock.c    |  486 -----
 .../staging/lustre/lustre/ldlm/ldlm_inodebits.c    |   69 -
 drivers/staging/lustre/lustre/ldlm/ldlm_internal.h |  329 ---
 drivers/staging/lustre/lustre/ldlm/ldlm_lib.c      |  842 --------
 drivers/staging/lustre/lustre/ldlm/ldlm_lock.c     | 2103 --------------------
 drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c    | 1154 -----------
 drivers/staging/lustre/lustre/ldlm/ldlm_plain.c    |   68 -
 drivers/staging/lustre/lustre/ldlm/ldlm_pool.c     | 1013 ----------
 drivers/staging/lustre/lustre/ldlm/ldlm_request.c  | 2033 -------------------
 drivers/staging/lustre/lustre/ldlm/ldlm_resource.c | 1318 -------------
 drivers/staging/lustre/lustre/ptlrpc/Makefile      |   14 
 drivers/staging/lustre/lustre/ptlrpc/l_lock.c      |   73 +
 drivers/staging/lustre/lustre/ptlrpc/ldlm_extent.c |  206 ++
 drivers/staging/lustre/lustre/ptlrpc/ldlm_flock.c  |  486 +++++
 .../staging/lustre/lustre/ptlrpc/ldlm_inodebits.c  |   69 +
 .../staging/lustre/lustre/ptlrpc/ldlm_internal.h   |  329 +++
 drivers/staging/lustre/lustre/ptlrpc/ldlm_lib.c    |  842 ++++++++
 drivers/staging/lustre/lustre/ptlrpc/ldlm_lock.c   | 2103 ++++++++++++++++++++
 drivers/staging/lustre/lustre/ptlrpc/ldlm_lockd.c  | 1154 +++++++++++
 drivers/staging/lustre/lustre/ptlrpc/ldlm_plain.c  |   68 +
 drivers/staging/lustre/lustre/ptlrpc/ldlm_pool.c   | 1013 ++++++++++
 .../staging/lustre/lustre/ptlrpc/ldlm_request.c    | 2033 +++++++++++++++++++
 .../staging/lustre/lustre/ptlrpc/ldlm_resource.c   | 1318 +++++++++++++
 .../staging/lustre/lustre/ptlrpc/ptlrpc_internal.h |    2 
 26 files changed, 9702 insertions(+), 9702 deletions(-)
 delete mode 100644 drivers/staging/lustre/lustre/ldlm/l_lock.c
 delete mode 100644 drivers/staging/lustre/lustre/ldlm/ldlm_extent.c
 delete mode 100644 drivers/staging/lustre/lustre/ldlm/ldlm_flock.c
 delete mode 100644 drivers/staging/lustre/lustre/ldlm/ldlm_inodebits.c
 delete mode 100644 drivers/staging/lustre/lustre/ldlm/ldlm_internal.h
 delete mode 100644 drivers/staging/lustre/lustre/ldlm/ldlm_lib.c
 delete mode 100644 drivers/staging/lustre/lustre/ldlm/ldlm_lock.c
 delete mode 100644 drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c
 delete mode 100644 drivers/staging/lustre/lustre/ldlm/ldlm_plain.c
 delete mode 100644 drivers/staging/lustre/lustre/ldlm/ldlm_pool.c
 delete mode 100644 drivers/staging/lustre/lustre/ldlm/ldlm_request.c
 delete mode 100644 drivers/staging/lustre/lustre/ldlm/ldlm_resource.c
 create mode 100644 drivers/staging/lustre/lustre/ptlrpc/l_lock.c
 create mode 100644 drivers/staging/lustre/lustre/ptlrpc/ldlm_extent.c
 create mode 100644 drivers/staging/lustre/lustre/ptlrpc/ldlm_flock.c
 create mode 100644 drivers/staging/lustre/lustre/ptlrpc/ldlm_inodebits.c
 create mode 100644 drivers/staging/lustre/lustre/ptlrpc/ldlm_internal.h
 create mode 100644 drivers/staging/lustre/lustre/ptlrpc/ldlm_lib.c
 create mode 100644 drivers/staging/lustre/lustre/ptlrpc/ldlm_lock.c
 create mode 100644 drivers/staging/lustre/lustre/ptlrpc/ldlm_lockd.c
 create mode 100644 drivers/staging/lustre/lustre/ptlrpc/ldlm_plain.c
 create mode 100644 drivers/staging/lustre/lustre/ptlrpc/ldlm_pool.c
 create mode 100644 drivers/staging/lustre/lustre/ptlrpc/ldlm_request.c
 create mode 100644 drivers/staging/lustre/lustre/ptlrpc/ldlm_resource.c

diff --git a/drivers/staging/lustre/lustre/ldlm/l_lock.c b/drivers/staging/lustre/lustre/ldlm/l_lock.c
deleted file mode 100644
index 296259aa51e6..000000000000
--- a/drivers/staging/lustre/lustre/ldlm/l_lock.c
+++ /dev/null
@@ -1,73 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-/*
- * GPL HEADER START
- *
- * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 only,
- * as published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful, but
- * WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
- * General Public License version 2 for more details (a copy is included
- * in the LICENSE file that accompanied this code).
- *
- * You should have received a copy of the GNU General Public License
- * version 2 along with this program; If not, see
- * http://www.gnu.org/licenses/gpl-2.0.html
- *
- * GPL HEADER END
- */
-/*
- * Copyright (c) 2002, 2010, Oracle and/or its affiliates. All rights reserved.
- * Use is subject to license terms.
- *
- * Copyright (c) 2012, Intel Corporation.
- */
-/*
- * This file is part of Lustre, http://www.lustre.org/
- * Lustre is a trademark of Sun Microsystems, Inc.
- */
-
-#define DEBUG_SUBSYSTEM S_LDLM
-
-#include <lustre_dlm.h>
-#include <lustre_lib.h>
-
-/**
- * Lock a lock and its resource.
- *
- * LDLM locking uses resource to serialize access to locks
- * but there is a case when we change resource of lock upon
- * enqueue reply. We rely on lock->l_resource = new_res
- * being an atomic operation.
- */
-struct ldlm_resource *lock_res_and_lock(struct ldlm_lock *lock)
-				__acquires(&lock->l_lock)
-				__acquires(&lock->l_resource->lr_lock)
-{
-	spin_lock(&lock->l_lock);
-
-	lock_res(lock->l_resource);
-
-	ldlm_set_res_locked(lock);
-	return lock->l_resource;
-}
-EXPORT_SYMBOL(lock_res_and_lock);
-
-/**
- * Unlock a lock and its resource previously locked with lock_res_and_lock
- */
-void unlock_res_and_lock(struct ldlm_lock *lock)
-		__releases(&lock->l_resource->lr_lock)
-		__releases(&lock->l_lock)
-{
-	/* on server-side resource of lock doesn't change */
-	ldlm_clear_res_locked(lock);
-
-	unlock_res(lock->l_resource);
-	spin_unlock(&lock->l_lock);
-}
-EXPORT_SYMBOL(unlock_res_and_lock);
diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_extent.c b/drivers/staging/lustre/lustre/ldlm/ldlm_extent.c
deleted file mode 100644
index 225c023b0bba..000000000000
--- a/drivers/staging/lustre/lustre/ldlm/ldlm_extent.c
+++ /dev/null
@@ -1,206 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-/*
- * GPL HEADER START
- *
- * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 only,
- * as published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful, but
- * WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
- * General Public License version 2 for more details (a copy is included
- * in the LICENSE file that accompanied this code).
- *
- * You should have received a copy of the GNU General Public License
- * version 2 along with this program; If not, see
- * http://www.gnu.org/licenses/gpl-2.0.html
- *
- * GPL HEADER END
- */
-/*
- * Copyright (c) 2002, 2010, Oracle and/or its affiliates. All rights reserved.
- * Use is subject to license terms.
- *
- * Copyright (c) 2010, 2012, Intel Corporation.
- */
-/*
- * This file is part of Lustre, http://www.lustre.org/
- * Lustre is a trademark of Sun Microsystems, Inc.
- *
- * lustre/ldlm/ldlm_extent.c
- *
- * Author: Peter Braam <braam at clusterfs.com>
- * Author: Phil Schwan <phil at clusterfs.com>
- */
-
-/**
- * This file contains implementation of EXTENT lock type
- *
- * EXTENT lock type is for locking a contiguous range of values, represented
- * by 64-bit starting and ending offsets (inclusive). There are several extent
- * lock modes, some of which may be mutually incompatible. Extent locks are
- * considered incompatible if their modes are incompatible and their extents
- * intersect.  See the lock mode compatibility matrix in lustre_dlm.h.
- */
-
-#define DEBUG_SUBSYSTEM S_LDLM
-#include <lustre_dlm.h>
-#include <obd_support.h>
-#include <obd.h>
-#include <obd_class.h>
-#include <lustre_lib.h>
-#include "ldlm_internal.h"
-#include <linux/interval_tree_generic.h>
-
-#define START(node) ((node)->l_policy_data.l_extent.start)
-#define LAST(node) ((node)->l_policy_data.l_extent.end)
-INTERVAL_TREE_DEFINE(struct ldlm_lock, l_rb, __u64, __subtree_last,
-		     START, LAST, static, extent);
-
-/* When a lock is cancelled by a client, the KMS may undergo change if this
- * is the "highest lock".  This function returns the new KMS value.
- * Caller must hold lr_lock already.
- *
- * NB: A lock on [x,y] protects a KMS of up to y + 1 bytes!
- */
-__u64 ldlm_extent_shift_kms(struct ldlm_lock *lock, __u64 old_kms)
-{
-	struct ldlm_resource *res = lock->l_resource;
-	struct ldlm_lock *lck;
-	__u64 kms = 0;
-
-	/* don't let another thread in ldlm_extent_shift_kms race in
-	 * just after we finish and take our lock into account in its
-	 * calculation of the kms
-	 */
-	ldlm_set_kms_ignore(lock);
-
-	list_for_each_entry(lck, &res->lr_granted, l_res_link) {
-
-		if (ldlm_is_kms_ignore(lck))
-			continue;
-
-		if (lck->l_policy_data.l_extent.end >= old_kms)
-			return old_kms;
-
-		/* This extent _has_ to be smaller than old_kms (checked above)
-		 * so kms can only ever be smaller or the same as old_kms.
-		 */
-		if (lck->l_policy_data.l_extent.end + 1 > kms)
-			kms = lck->l_policy_data.l_extent.end + 1;
-	}
-	LASSERTF(kms <= old_kms, "kms %llu old_kms %llu\n", kms, old_kms);
-
-	return kms;
-}
-EXPORT_SYMBOL(ldlm_extent_shift_kms);
-
-static inline int lock_mode_to_index(enum ldlm_mode mode)
-{
-	int index;
-
-	LASSERT(mode != 0);
-	LASSERT(is_power_of_2(mode));
-	for (index = -1; mode; index++)
-		mode >>= 1;
-	LASSERT(index < LCK_MODE_NUM);
-	return index;
-}
-
-/** Add newly granted lock into interval tree for the resource. */
-void ldlm_extent_add_lock(struct ldlm_resource *res,
-			  struct ldlm_lock *lock)
-{
-	struct ldlm_interval_tree *tree;
-	int idx;
-
-	LASSERT(lock->l_granted_mode == lock->l_req_mode);
-
-	LASSERT(RB_EMPTY_NODE(&lock->l_rb));
-
-	idx = lock_mode_to_index(lock->l_granted_mode);
-	LASSERT(lock->l_granted_mode == 1 << idx);
-	LASSERT(lock->l_granted_mode == res->lr_itree[idx].lit_mode);
-
-	tree = &res->lr_itree[idx];
-	extent_insert(lock, &tree->lit_root);
-	tree->lit_size++;
-
-	/* even though we use interval tree to manage the extent lock, we also
-	 * add the locks into grant list, for debug purpose, ..
-	 */
-	ldlm_resource_add_lock(res, &res->lr_granted, lock);
-
-	if (OBD_FAIL_CHECK(OBD_FAIL_LDLM_GRANT_CHECK)) {
-		struct ldlm_lock *lck;
-
-		list_for_each_entry_reverse(lck, &res->lr_granted,
-					    l_res_link) {
-			if (lck == lock)
-				continue;
-			if (lockmode_compat(lck->l_granted_mode,
-					    lock->l_granted_mode))
-				continue;
-			if (ldlm_extent_overlap(&lck->l_req_extent,
-						&lock->l_req_extent)) {
-				CDEBUG(D_ERROR,
-				       "granting conflicting lock %p %p\n",
-				       lck, lock);
-				ldlm_resource_dump(D_ERROR, res);
-				LBUG();
-			}
-		}
-	}
-}
-
-/** Remove cancelled lock from resource interval tree. */
-void ldlm_extent_unlink_lock(struct ldlm_lock *lock)
-{
-	struct ldlm_resource *res = lock->l_resource;
-	struct ldlm_interval_tree *tree;
-	int idx;
-
-	if (RB_EMPTY_NODE(&lock->l_rb)) /* duplicate unlink */
-		return;
-
-	idx = lock_mode_to_index(lock->l_granted_mode);
-	LASSERT(lock->l_granted_mode == 1 << idx);
-	tree = &res->lr_itree[idx];
-
-	tree->lit_size--;
-	extent_remove(lock, &tree->lit_root);
-}
-
-void ldlm_extent_policy_wire_to_local(const union ldlm_wire_policy_data *wpolicy,
-				      union ldlm_policy_data *lpolicy)
-{
-	lpolicy->l_extent.start = wpolicy->l_extent.start;
-	lpolicy->l_extent.end = wpolicy->l_extent.end;
-	lpolicy->l_extent.gid = wpolicy->l_extent.gid;
-}
-
-void ldlm_extent_policy_local_to_wire(const union ldlm_policy_data *lpolicy,
-				      union ldlm_wire_policy_data *wpolicy)
-{
-	memset(wpolicy, 0, sizeof(*wpolicy));
-	wpolicy->l_extent.start = lpolicy->l_extent.start;
-	wpolicy->l_extent.end = lpolicy->l_extent.end;
-	wpolicy->l_extent.gid = lpolicy->l_extent.gid;
-}
-
-void ldlm_extent_search(struct rb_root_cached *root,
-			__u64 start, __u64 end,
-			bool (*matches)(struct ldlm_lock *lock, void *data),
-			void *data)
-{
-	struct ldlm_lock *lock;
-
-	for (lock = extent_iter_first(root, start, end);
-	     lock;
-	     lock = extent_iter_next(lock, start, end))
-		if (matches(lock, data))
-			break;
-}
diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_flock.c b/drivers/staging/lustre/lustre/ldlm/ldlm_flock.c
deleted file mode 100644
index 94f3b1e49896..000000000000
--- a/drivers/staging/lustre/lustre/ldlm/ldlm_flock.c
+++ /dev/null
@@ -1,486 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-/*
- * GPL HEADER START
- *
- * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 only,
- * as published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful, but
- * WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
- * General Public License version 2 for more details (a copy is included
- * in the LICENSE file that accompanied this code).
- *
- * You should have received a copy of the GNU General Public License
- * version 2 along with this program; If not, see
- * http://www.gnu.org/licenses/gpl-2.0.html
- *
- * GPL HEADER END
- */
-/*
- * Copyright (c) 2003 Hewlett-Packard Development Company LP.
- * Developed under the sponsorship of the US Government under
- * Subcontract No. B514193
- *
- * Copyright (c) 2003, 2010, Oracle and/or its affiliates. All rights reserved.
- * Use is subject to license terms.
- *
- * Copyright (c) 2010, 2012, Intel Corporation.
- */
-/*
- * This file is part of Lustre, http://www.lustre.org/
- * Lustre is a trademark of Sun Microsystems, Inc.
- */
-
-/**
- * This file implements POSIX lock type for Lustre.
- * Its policy properties are start and end of extent and PID.
- *
- * These locks are only done through MDS due to POSIX semantics requiring
- * e.g. that locks could be only partially released and as such split into
- * two parts, and also that two adjacent locks from the same process may be
- * merged into a single wider lock.
- *
- * Lock modes are mapped like this:
- * PR and PW for READ and WRITE locks
- * NL to request a releasing of a portion of the lock
- *
- * These flock locks never timeout.
- */
-
-#define DEBUG_SUBSYSTEM S_LDLM
-
-#include <lustre_dlm.h>
-#include <obd_support.h>
-#include <obd_class.h>
-#include <lustre_lib.h>
-#include <linux/list.h>
-#include "ldlm_internal.h"
-
-static inline int
-ldlm_same_flock_owner(struct ldlm_lock *lock, struct ldlm_lock *new)
-{
-	return((new->l_policy_data.l_flock.owner ==
-		lock->l_policy_data.l_flock.owner) &&
-	       (new->l_export == lock->l_export));
-}
-
-static inline int
-ldlm_flocks_overlap(struct ldlm_lock *lock, struct ldlm_lock *new)
-{
-	return((new->l_policy_data.l_flock.start <=
-		lock->l_policy_data.l_flock.end) &&
-	       (new->l_policy_data.l_flock.end >=
-		lock->l_policy_data.l_flock.start));
-}
-
-static inline void
-ldlm_flock_destroy(struct ldlm_lock *lock, enum ldlm_mode mode)
-{
-	LDLM_DEBUG(lock, "%s(mode: %d)",
-		   __func__, mode);
-
-	list_del_init(&lock->l_res_link);
-
-	/* client side - set a flag to prevent sending a CANCEL */
-	lock->l_flags |= LDLM_FL_LOCAL_ONLY | LDLM_FL_CBPENDING;
-
-	/* when reaching here, it is under lock_res_and_lock(). Thus,
-	 * need call the nolock version of ldlm_lock_decref_internal
-	 */
-	ldlm_lock_decref_internal_nolock(lock, mode);
-
-	ldlm_lock_destroy_nolock(lock);
-}
-
-/**
- * Process a granting attempt for flock lock.
- * Must be called under ns lock held.
- *
- * This function looks for any conflicts for \a lock in the granted or
- * waiting queues. The lock is granted if no conflicts are found in
- * either queue.
- *
- * It is also responsible for splitting a lock if a portion of the lock
- * is released.
- *
- */
-static int ldlm_process_flock_lock(struct ldlm_lock *req)
-{
-	struct ldlm_resource *res = req->l_resource;
-	struct ldlm_namespace *ns = ldlm_res_to_ns(res);
-	struct ldlm_lock *tmp;
-	struct ldlm_lock *lock;
-	struct ldlm_lock *new = req;
-	struct ldlm_lock *new2 = NULL;
-	enum ldlm_mode mode = req->l_req_mode;
-	int added = (mode == LCK_NL);
-	int splitted = 0;
-	const struct ldlm_callback_suite null_cbs = { };
-
-	CDEBUG(D_DLMTRACE,
-	       "owner %llu pid %u mode %u start %llu end %llu\n",
-	       new->l_policy_data.l_flock.owner,
-	       new->l_policy_data.l_flock.pid, mode,
-	       req->l_policy_data.l_flock.start,
-	       req->l_policy_data.l_flock.end);
-
-	/* No blocking ASTs are sent to the clients for
-	 * Posix file & record locks
-	 */
-	req->l_blocking_ast = NULL;
-
-reprocess:
-	/* This loop determines where this processes locks start
-	 * in the resource lr_granted list.
-	 */
-	list_for_each_entry(lock, &res->lr_granted, l_res_link)
-		if (ldlm_same_flock_owner(lock, req))
-			break;
-
-	/* Scan the locks owned by this process to find the insertion point
-	 * (as locks are ordered), and to handle overlaps.
-	 * We may have to merge or split existing locks.
-	 */
-	list_for_each_entry_safe_from(lock, tmp, &res->lr_granted, l_res_link) {
-
-		if (!ldlm_same_flock_owner(lock, new))
-			break;
-
-		if (lock->l_granted_mode == mode) {
-			/* If the modes are the same then we need to process
-			 * locks that overlap OR adjoin the new lock. The extra
-			 * logic condition is necessary to deal with arithmetic
-			 * overflow and underflow.
-			 */
-			if ((new->l_policy_data.l_flock.start >
-			     (lock->l_policy_data.l_flock.end + 1)) &&
-			    (lock->l_policy_data.l_flock.end != OBD_OBJECT_EOF))
-				continue;
-
-			if ((new->l_policy_data.l_flock.end <
-			     (lock->l_policy_data.l_flock.start - 1)) &&
-			    (lock->l_policy_data.l_flock.start != 0))
-				break;
-
-			if (new->l_policy_data.l_flock.start <
-			    lock->l_policy_data.l_flock.start) {
-				lock->l_policy_data.l_flock.start =
-					new->l_policy_data.l_flock.start;
-			} else {
-				new->l_policy_data.l_flock.start =
-					lock->l_policy_data.l_flock.start;
-			}
-
-			if (new->l_policy_data.l_flock.end >
-			    lock->l_policy_data.l_flock.end) {
-				lock->l_policy_data.l_flock.end =
-					new->l_policy_data.l_flock.end;
-			} else {
-				new->l_policy_data.l_flock.end =
-					lock->l_policy_data.l_flock.end;
-			}
-
-			if (added) {
-				ldlm_flock_destroy(lock, mode);
-			} else {
-				new = lock;
-				added = 1;
-			}
-			continue;
-		}
-
-		if (new->l_policy_data.l_flock.start >
-		    lock->l_policy_data.l_flock.end)
-			continue;
-
-		if (new->l_policy_data.l_flock.end <
-		    lock->l_policy_data.l_flock.start)
-			break;
-
-		if (new->l_policy_data.l_flock.start <=
-		    lock->l_policy_data.l_flock.start) {
-			if (new->l_policy_data.l_flock.end <
-			    lock->l_policy_data.l_flock.end) {
-				lock->l_policy_data.l_flock.start =
-					new->l_policy_data.l_flock.end + 1;
-				break;
-			}
-			ldlm_flock_destroy(lock, lock->l_req_mode);
-			continue;
-		}
-		if (new->l_policy_data.l_flock.end >=
-		    lock->l_policy_data.l_flock.end) {
-			lock->l_policy_data.l_flock.end =
-				new->l_policy_data.l_flock.start - 1;
-			continue;
-		}
-
-		/* split the existing lock into two locks */
-
-		/* if this is an F_UNLCK operation then we could avoid
-		 * allocating a new lock and use the req lock passed in
-		 * with the request but this would complicate the reply
-		 * processing since updates to req get reflected in the
-		 * reply. The client side replays the lock request so
-		 * it must see the original lock data in the reply.
-		 */
-
-		/* XXX - if ldlm_lock_new() can sleep we should
-		 * release the lr_lock, allocate the new lock,
-		 * and restart processing this lock.
-		 */
-		if (!new2) {
-			unlock_res_and_lock(req);
-			new2 = ldlm_lock_create(ns, &res->lr_name, LDLM_FLOCK,
-						lock->l_granted_mode, &null_cbs,
-						NULL, 0, LVB_T_NONE);
-			lock_res_and_lock(req);
-			if (IS_ERR(new2)) {
-				ldlm_flock_destroy(req, lock->l_granted_mode);
-				return LDLM_ITER_STOP;
-			}
-			goto reprocess;
-		}
-
-		splitted = 1;
-
-		new2->l_granted_mode = lock->l_granted_mode;
-		new2->l_policy_data.l_flock.pid =
-			new->l_policy_data.l_flock.pid;
-		new2->l_policy_data.l_flock.owner =
-			new->l_policy_data.l_flock.owner;
-		new2->l_policy_data.l_flock.start =
-			lock->l_policy_data.l_flock.start;
-		new2->l_policy_data.l_flock.end =
-			new->l_policy_data.l_flock.start - 1;
-		lock->l_policy_data.l_flock.start =
-			new->l_policy_data.l_flock.end + 1;
-		new2->l_conn_export = lock->l_conn_export;
-		if (lock->l_export)
-			new2->l_export = class_export_lock_get(lock->l_export,
-							       new2);
-		ldlm_lock_addref_internal_nolock(new2,
-						 lock->l_granted_mode);
-
-		/* insert new2 at lock */
-		ldlm_resource_add_lock(res, &lock->l_res_link, new2);
-		LDLM_LOCK_RELEASE(new2);
-		break;
-	}
-
-	/* if new2 is created but never used, destroy it*/
-	if (splitted == 0 && new2)
-		ldlm_lock_destroy_nolock(new2);
-
-	/* At this point we're granting the lock request. */
-	req->l_granted_mode = req->l_req_mode;
-
-	if (!added) {
-		list_del_init(&req->l_res_link);
-		/* insert new lock before "lock", which might be the
-		 * next lock for this owner, or might be the first
-		 * lock for the next owner, or might not be a lock at
-		 * all, but instead points at the head of the list
-		 */
-		ldlm_resource_add_lock(res, &lock->l_res_link, req);
-	}
-
-	/* In case we're reprocessing the requested lock we can't destroy
-	 * it until after calling ldlm_add_ast_work_item() above so that laawi()
-	 * can bump the reference count on \a req. Otherwise \a req
-	 * could be freed before the completion AST can be sent.
-	 */
-	if (added)
-		ldlm_flock_destroy(req, mode);
-
-	ldlm_resource_dump(D_INFO, res);
-	return LDLM_ITER_CONTINUE;
-}
-
-/**
- * Flock completion callback function.
- *
- * \param lock [in,out]: A lock to be handled
- * \param flags    [in]: flags
- * \param *data    [in]: ldlm_work_cp_ast_lock() will use ldlm_cb_set_arg
- *
- * \retval 0    : success
- * \retval <0   : failure
- */
-int
-ldlm_flock_completion_ast(struct ldlm_lock *lock, __u64 flags, void *data)
-{
-	struct file_lock		*getlk = lock->l_ast_data;
-	int				rc = 0;
-
-	OBD_FAIL_TIMEOUT(OBD_FAIL_LDLM_CP_CB_WAIT2, 4);
-	if (OBD_FAIL_PRECHECK(OBD_FAIL_LDLM_CP_CB_WAIT3)) {
-		lock_res_and_lock(lock);
-		lock->l_flags |= LDLM_FL_FAIL_LOC;
-		unlock_res_and_lock(lock);
-		OBD_FAIL_TIMEOUT(OBD_FAIL_LDLM_CP_CB_WAIT3, 4);
-	}
-	CDEBUG(D_DLMTRACE, "flags: 0x%llx data: %p getlk: %p\n",
-	       flags, data, getlk);
-
-	LASSERT(flags != LDLM_FL_WAIT_NOREPROC);
-
-	if (flags & LDLM_FL_FAILED)
-		goto granted;
-
-	if (!(flags & LDLM_FL_BLOCKED_MASK)) {
-		if (!data)
-			/* mds granted the lock in the reply */
-			goto granted;
-		/* CP AST RPC: lock get granted, wake it up */
-		wake_up(&lock->l_waitq);
-		return 0;
-	}
-
-	LDLM_DEBUG(lock,
-		   "client-side enqueue returned a blocked lock, sleeping");
-
-	/* Go to sleep until the lock is granted. */
-	rc = l_wait_event_abortable(lock->l_waitq, is_granted_or_cancelled(lock));
-
-	if (rc) {
-		lock_res_and_lock(lock);
-
-		/* client side - set flag to prevent lock from being put on LRU list */
-		ldlm_set_cbpending(lock);
-		unlock_res_and_lock(lock);
-
-		LDLM_DEBUG(lock, "client-side enqueue waking up: failed (%d)",
-			   rc);
-		return rc;
-	}
-
-granted:
-	OBD_FAIL_TIMEOUT(OBD_FAIL_LDLM_CP_CB_WAIT, 10);
-
-	if (OBD_FAIL_PRECHECK(OBD_FAIL_LDLM_CP_CB_WAIT4)) {
-		lock_res_and_lock(lock);
-		/* DEADLOCK is always set with CBPENDING */
-		lock->l_flags |= LDLM_FL_FLOCK_DEADLOCK | LDLM_FL_CBPENDING;
-		unlock_res_and_lock(lock);
-		OBD_FAIL_TIMEOUT(OBD_FAIL_LDLM_CP_CB_WAIT4, 4);
-	}
-	if (OBD_FAIL_PRECHECK(OBD_FAIL_LDLM_CP_CB_WAIT5)) {
-		lock_res_and_lock(lock);
-		/* DEADLOCK is always set with CBPENDING */
-		lock->l_flags |= LDLM_FL_FAIL_LOC |
-				 LDLM_FL_FLOCK_DEADLOCK | LDLM_FL_CBPENDING;
-		unlock_res_and_lock(lock);
-		OBD_FAIL_TIMEOUT(OBD_FAIL_LDLM_CP_CB_WAIT5, 4);
-	}
-
-	lock_res_and_lock(lock);
-
-	/*
-	 * Protect against race where lock could have been just destroyed
-	 * due to overlap in ldlm_process_flock_lock().
-	 */
-	if (ldlm_is_destroyed(lock)) {
-		unlock_res_and_lock(lock);
-		LDLM_DEBUG(lock, "client-side enqueue waking up: destroyed");
-		/*
-		 * An error is still to be returned, to propagate it up to
-		 * ldlm_cli_enqueue_fini() caller.
-		 */
-		return -EIO;
-	}
-
-	/* ldlm_lock_enqueue() has already placed lock on the granted list. */
-	ldlm_resource_unlink_lock(lock);
-
-	/*
-	 * Import invalidation. We need to actually release the lock
-	 * references being held, so that it can go away. No point in
-	 * holding the lock even if app still believes it has it, since
-	 * server already dropped it anyway. Only for granted locks too.
-	 */
-	/* Do the same for DEADLOCK'ed locks. */
-	if (ldlm_is_failed(lock) || ldlm_is_flock_deadlock(lock)) {
-		int mode;
-
-		if (flags & LDLM_FL_TEST_LOCK)
-			LASSERT(ldlm_is_test_lock(lock));
-
-		if (ldlm_is_test_lock(lock) || ldlm_is_flock_deadlock(lock))
-			mode = getlk->fl_type;
-		else
-			mode = lock->l_granted_mode;
-
-		if (ldlm_is_flock_deadlock(lock)) {
-			LDLM_DEBUG(lock,
-				   "client-side enqueue deadlock received");
-			rc = -EDEADLK;
-		}
-		ldlm_flock_destroy(lock, mode);
-		unlock_res_and_lock(lock);
-
-		/* Need to wake up the waiter if we were evicted */
-		wake_up(&lock->l_waitq);
-
-		/*
-		 * An error is still to be returned, to propagate it up to
-		 * ldlm_cli_enqueue_fini() caller.
-		 */
-		return rc ? : -EIO;
-	}
-
-	LDLM_DEBUG(lock, "client-side enqueue granted");
-
-	if (flags & LDLM_FL_TEST_LOCK) {
-		/* fcntl(F_GETLK) request */
-		/* The old mode was saved in getlk->fl_type so that if the mode
-		 * in the lock changes we can decref the appropriate refcount.
-		 */
-		LASSERT(ldlm_is_test_lock(lock));
-		ldlm_flock_destroy(lock, getlk->fl_type);
-		switch (lock->l_granted_mode) {
-		case LCK_PR:
-			getlk->fl_type = F_RDLCK;
-			break;
-		case LCK_PW:
-			getlk->fl_type = F_WRLCK;
-			break;
-		default:
-			getlk->fl_type = F_UNLCK;
-		}
-		getlk->fl_pid = -(pid_t)lock->l_policy_data.l_flock.pid;
-		getlk->fl_start = (loff_t)lock->l_policy_data.l_flock.start;
-		getlk->fl_end = (loff_t)lock->l_policy_data.l_flock.end;
-	} else {
-		/* We need to reprocess the lock to do merges or splits
-		 * with existing locks owned by this process.
-		 */
-		ldlm_process_flock_lock(lock);
-	}
-	unlock_res_and_lock(lock);
-	return rc;
-}
-EXPORT_SYMBOL(ldlm_flock_completion_ast);
-
-void ldlm_flock_policy_wire_to_local(const union ldlm_wire_policy_data *wpolicy,
-				     union ldlm_policy_data *lpolicy)
-{
-	lpolicy->l_flock.start = wpolicy->l_flock.lfw_start;
-	lpolicy->l_flock.end = wpolicy->l_flock.lfw_end;
-	lpolicy->l_flock.pid = wpolicy->l_flock.lfw_pid;
-	lpolicy->l_flock.owner = wpolicy->l_flock.lfw_owner;
-}
-
-void ldlm_flock_policy_local_to_wire(const union ldlm_policy_data *lpolicy,
-				     union ldlm_wire_policy_data *wpolicy)
-{
-	memset(wpolicy, 0, sizeof(*wpolicy));
-	wpolicy->l_flock.lfw_start = lpolicy->l_flock.start;
-	wpolicy->l_flock.lfw_end = lpolicy->l_flock.end;
-	wpolicy->l_flock.lfw_pid = lpolicy->l_flock.pid;
-	wpolicy->l_flock.lfw_owner = lpolicy->l_flock.owner;
-}
diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_inodebits.c b/drivers/staging/lustre/lustre/ldlm/ldlm_inodebits.c
deleted file mode 100644
index 2926208cdfa1..000000000000
--- a/drivers/staging/lustre/lustre/ldlm/ldlm_inodebits.c
+++ /dev/null
@@ -1,69 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-/*
- * GPL HEADER START
- *
- * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 only,
- * as published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful, but
- * WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
- * General Public License version 2 for more details (a copy is included
- * in the LICENSE file that accompanied this code).
- *
- * You should have received a copy of the GNU General Public License
- * version 2 along with this program; If not, see
- * http://www.gnu.org/licenses/gpl-2.0.html
- *
- * GPL HEADER END
- */
-/*
- * Copyright (c) 2004, 2010, Oracle and/or its affiliates. All rights reserved.
- * Use is subject to license terms.
- *
- * Copyright (c) 2011, Intel Corporation.
- */
-/*
- * This file is part of Lustre, http://www.lustre.org/
- * Lustre is a trademark of Sun Microsystems, Inc.
- *
- * lustre/ldlm/ldlm_inodebits.c
- *
- * Author: Peter Braam <braam at clusterfs.com>
- * Author: Phil Schwan <phil at clusterfs.com>
- */
-
-/**
- * This file contains implementation of IBITS lock type
- *
- * IBITS lock type contains a bit mask determining various properties of an
- * object. The meanings of specific bits are specific to the caller and are
- * opaque to LDLM code.
- *
- * Locks with intersecting bitmasks and conflicting lock modes (e.g.  LCK_PW)
- * are considered conflicting.  See the lock mode compatibility matrix
- * in lustre_dlm.h.
- */
-
-#define DEBUG_SUBSYSTEM S_LDLM
-
-#include <lustre_dlm.h>
-#include <obd_support.h>
-#include <lustre_lib.h>
-#include "ldlm_internal.h"
-
-void ldlm_ibits_policy_wire_to_local(const union ldlm_wire_policy_data *wpolicy,
-				     union ldlm_policy_data *lpolicy)
-{
-	lpolicy->l_inodebits.bits = wpolicy->l_inodebits.bits;
-}
-
-void ldlm_ibits_policy_local_to_wire(const union ldlm_policy_data *lpolicy,
-				     union ldlm_wire_policy_data *wpolicy)
-{
-	memset(wpolicy, 0, sizeof(*wpolicy));
-	wpolicy->l_inodebits.bits = lpolicy->l_inodebits.bits;
-}
diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_internal.h b/drivers/staging/lustre/lustre/ldlm/ldlm_internal.h
deleted file mode 100644
index 60a15b963c8a..000000000000
--- a/drivers/staging/lustre/lustre/ldlm/ldlm_internal.h
+++ /dev/null
@@ -1,329 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-/*
- * GPL HEADER START
- *
- * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 only,
- * as published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful, but
- * WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
- * General Public License version 2 for more details (a copy is included
- * in the LICENSE file that accompanied this code).
- *
- * You should have received a copy of the GNU General Public License
- * version 2 along with this program; If not, see
- * http://www.gnu.org/licenses/gpl-2.0.html
- *
- * GPL HEADER END
- */
-/*
- * Copyright (c) 2003, 2010, Oracle and/or its affiliates. All rights reserved.
- * Use is subject to license terms.
- *
- * Copyright (c) 2011, 2015, Intel Corporation.
- */
-/*
- * This file is part of Lustre, http://www.lustre.org/
- * Lustre is a trademark of Sun Microsystems, Inc.
- */
-
-#define MAX_STRING_SIZE 128
-
-extern int ldlm_srv_namespace_nr;
-extern int ldlm_cli_namespace_nr;
-extern struct mutex ldlm_srv_namespace_lock;
-extern struct list_head ldlm_srv_namespace_list;
-extern struct mutex ldlm_cli_namespace_lock;
-extern struct list_head ldlm_cli_active_namespace_list;
-
-static inline int ldlm_namespace_nr_read(enum ldlm_side client)
-{
-	return client == LDLM_NAMESPACE_SERVER ?
-		ldlm_srv_namespace_nr : ldlm_cli_namespace_nr;
-}
-
-static inline void ldlm_namespace_nr_inc(enum ldlm_side client)
-{
-	if (client == LDLM_NAMESPACE_SERVER)
-		ldlm_srv_namespace_nr++;
-	else
-		ldlm_cli_namespace_nr++;
-}
-
-static inline void ldlm_namespace_nr_dec(enum ldlm_side client)
-{
-	if (client == LDLM_NAMESPACE_SERVER)
-		ldlm_srv_namespace_nr--;
-	else
-		ldlm_cli_namespace_nr--;
-}
-
-static inline struct list_head *ldlm_namespace_list(enum ldlm_side client)
-{
-	return client == LDLM_NAMESPACE_SERVER ?
-		&ldlm_srv_namespace_list : &ldlm_cli_active_namespace_list;
-}
-
-static inline struct mutex *ldlm_namespace_lock(enum ldlm_side client)
-{
-	return client == LDLM_NAMESPACE_SERVER ?
-		&ldlm_srv_namespace_lock : &ldlm_cli_namespace_lock;
-}
-
-/* ns_bref is the number of resources in this namespace */
-static inline int ldlm_ns_empty(struct ldlm_namespace *ns)
-{
-	return atomic_read(&ns->ns_bref) == 0;
-}
-
-void ldlm_namespace_move_to_active_locked(struct ldlm_namespace *ns,
-					  enum ldlm_side client);
-void ldlm_namespace_move_to_inactive_locked(struct ldlm_namespace *ns,
-					    enum ldlm_side client);
-struct ldlm_namespace *ldlm_namespace_first_locked(enum ldlm_side client);
-
-/* ldlm_request.c */
-/* Cancel lru flag, it indicates we cancel aged locks. */
-enum {
-	LDLM_LRU_FLAG_AGED	= BIT(0), /* Cancel old non-LRU resize locks */
-	LDLM_LRU_FLAG_PASSED	= BIT(1), /* Cancel passed number of locks. */
-	LDLM_LRU_FLAG_SHRINK	= BIT(2), /* Cancel locks from shrinker. */
-	LDLM_LRU_FLAG_LRUR	= BIT(3), /* Cancel locks from lru resize. */
-	LDLM_LRU_FLAG_NO_WAIT	= BIT(4), /* Cancel locks w/o blocking (neither
-					   * sending nor waiting for any rpcs)
-					   */
-	LDLM_LRU_FLAG_LRUR_NO_WAIT = BIT(5), /* LRUR + NO_WAIT */
-};
-
-int ldlm_cancel_lru(struct ldlm_namespace *ns, int nr,
-		    enum ldlm_cancel_flags sync, int flags);
-int ldlm_cancel_lru_local(struct ldlm_namespace *ns,
-			  struct list_head *cancels, int count, int max,
-			  enum ldlm_cancel_flags cancel_flags, int flags);
-extern unsigned int ldlm_enqueue_min;
-extern unsigned int ldlm_cancel_unused_locks_before_replay;
-
-/* ldlm_lock.c */
-
-struct ldlm_cb_set_arg {
-	struct ptlrpc_request_set	*set;
-	int				 type; /* LDLM_{CP,BL,GL}_CALLBACK */
-	atomic_t			 restart;
-	struct list_head			*list;
-	union ldlm_gl_desc		*gl_desc; /* glimpse AST descriptor */
-};
-
-enum ldlm_desc_ast_t {
-	LDLM_WORK_BL_AST,
-	LDLM_WORK_CP_AST,
-	LDLM_WORK_REVOKE_AST,
-	LDLM_WORK_GL_AST
-};
-
-void ldlm_grant_lock(struct ldlm_lock *lock, struct list_head *work_list);
-int ldlm_fill_lvb(struct ldlm_lock *lock, struct req_capsule *pill,
-		  enum req_location loc, void *data, int size);
-struct ldlm_lock *
-ldlm_lock_create(struct ldlm_namespace *ns, const struct ldlm_res_id *id,
-		 enum ldlm_type type, enum ldlm_mode mode,
-		 const struct ldlm_callback_suite *cbs,
-		 void *data, __u32 lvb_len, enum lvb_type lvb_type);
-enum ldlm_error ldlm_lock_enqueue(struct ldlm_namespace *ns,
-				  struct ldlm_lock **lock, void *cookie,
-				  __u64 *flags);
-void ldlm_lock_addref_internal(struct ldlm_lock *lock, enum ldlm_mode mode);
-void ldlm_lock_addref_internal_nolock(struct ldlm_lock *lock,
-				      enum ldlm_mode mode);
-void ldlm_lock_decref_internal(struct ldlm_lock *lock, enum ldlm_mode mode);
-void ldlm_lock_decref_internal_nolock(struct ldlm_lock *lock,
-				      enum ldlm_mode mode);
-int ldlm_run_ast_work(struct ldlm_namespace *ns, struct list_head *rpc_list,
-		      enum ldlm_desc_ast_t ast_type);
-int ldlm_lock_remove_from_lru_check(struct ldlm_lock *lock, time_t last_use);
-#define ldlm_lock_remove_from_lru(lock) ldlm_lock_remove_from_lru_check(lock, 0)
-int ldlm_lock_remove_from_lru_nolock(struct ldlm_lock *lock);
-void ldlm_lock_destroy_nolock(struct ldlm_lock *lock);
-
-/* ldlm_lockd.c */
-int ldlm_bl_to_thread_lock(struct ldlm_namespace *ns, struct ldlm_lock_desc *ld,
-			   struct ldlm_lock *lock);
-int ldlm_bl_to_thread_list(struct ldlm_namespace *ns,
-			   struct ldlm_lock_desc *ld,
-			   struct list_head *cancels, int count,
-			   enum ldlm_cancel_flags cancel_flags);
-int ldlm_bl_thread_wakeup(void);
-
-void ldlm_handle_bl_callback(struct ldlm_namespace *ns,
-			     struct ldlm_lock_desc *ld, struct ldlm_lock *lock);
-
-extern struct kmem_cache *ldlm_resource_slab;
-extern struct kset *ldlm_ns_kset;
-
-/* ldlm_lockd.c & ldlm_lock.c */
-extern struct kmem_cache *ldlm_lock_slab;
-
-/* ldlm_extent.c */
-void ldlm_extent_add_lock(struct ldlm_resource *res, struct ldlm_lock *lock);
-void ldlm_extent_unlink_lock(struct ldlm_lock *lock);
-void ldlm_extent_search(struct rb_root_cached *root,
-			__u64 start, __u64 end,
-			bool (*matches)(struct ldlm_lock *lock, void *data),
-			void *data);
-
-/* l_lock.c */
-void l_check_ns_lock(struct ldlm_namespace *ns);
-void l_check_no_ns_lock(struct ldlm_namespace *ns);
-
-extern struct dentry *ldlm_svc_debugfs_dir;
-
-struct ldlm_state {
-	struct ptlrpc_service *ldlm_cb_service;
-	struct ptlrpc_service *ldlm_cancel_service;
-	struct ptlrpc_client *ldlm_client;
-	struct ptlrpc_connection *ldlm_server_conn;
-	struct ldlm_bl_pool *ldlm_bl_pool;
-};
-
-/* ldlm_pool.c */
-__u64 ldlm_pool_get_slv(struct ldlm_pool *pl);
-void ldlm_pool_set_clv(struct ldlm_pool *pl, __u64 clv);
-__u32 ldlm_pool_get_lvf(struct ldlm_pool *pl);
-
-int ldlm_init(void);
-void ldlm_exit(void);
-
-enum ldlm_policy_res {
-	LDLM_POLICY_CANCEL_LOCK,
-	LDLM_POLICY_KEEP_LOCK,
-	LDLM_POLICY_SKIP_LOCK
-};
-
-#define LDLM_POOL_SYSFS_PRINT_int(v) sprintf(buf, "%d\n", v)
-#define LDLM_POOL_SYSFS_SET_int(a, b) { a = b; }
-#define LDLM_POOL_SYSFS_PRINT_u64(v) sprintf(buf, "%lld\n", v)
-#define LDLM_POOL_SYSFS_SET_u64(a, b) { a = b; }
-#define LDLM_POOL_SYSFS_PRINT_atomic(v) sprintf(buf, "%d\n", atomic_read(&v))
-#define LDLM_POOL_SYSFS_SET_atomic(a, b) atomic_set(&a, b)
-
-#define LDLM_POOL_SYSFS_READER_SHOW(var, type)				    \
-	static ssize_t var##_show(struct kobject *kobj,			    \
-				  struct attribute *attr,		    \
-				  char *buf)				    \
-	{								    \
-		struct ldlm_pool *pl = container_of(kobj, struct ldlm_pool, \
-						    pl_kobj);		    \
-		type tmp;						    \
-									    \
-		spin_lock(&pl->pl_lock);				    \
-		tmp = pl->pl_##var;					    \
-		spin_unlock(&pl->pl_lock);				    \
-									    \
-		return LDLM_POOL_SYSFS_PRINT_##type(tmp);		    \
-	}								    \
-	struct __##var##__dummy_read {; } /* semicolon catcher */
-
-#define LDLM_POOL_SYSFS_WRITER_STORE(var, type)				    \
-	static ssize_t var##_store(struct kobject *kobj,		    \
-				     struct attribute *attr,		    \
-				     const char *buffer,		    \
-				     size_t count)			    \
-	{								    \
-		struct ldlm_pool *pl = container_of(kobj, struct ldlm_pool, \
-						    pl_kobj);		    \
-		unsigned long tmp;					    \
-		int rc;							    \
-									    \
-		rc = kstrtoul(buffer, 10, &tmp);			    \
-		if (rc < 0) {						    \
-			return rc;					    \
-		}							    \
-									    \
-		spin_lock(&pl->pl_lock);				    \
-		LDLM_POOL_SYSFS_SET_##type(pl->pl_##var, tmp);		    \
-		spin_unlock(&pl->pl_lock);				    \
-									    \
-		return count;						    \
-	}								    \
-	struct __##var##__dummy_write {; } /* semicolon catcher */
-
-#define LDLM_POOL_SYSFS_READER_NOLOCK_SHOW(var, type)			    \
-	static ssize_t var##_show(struct kobject *kobj,		    \
-				    struct attribute *attr,		    \
-				    char *buf)				    \
-	{								    \
-		struct ldlm_pool *pl = container_of(kobj, struct ldlm_pool, \
-						    pl_kobj);		    \
-									    \
-		return LDLM_POOL_SYSFS_PRINT_##type(pl->pl_##var);	    \
-	}								    \
-	struct __##var##__dummy_read {; } /* semicolon catcher */
-
-#define LDLM_POOL_SYSFS_WRITER_NOLOCK_STORE(var, type)			    \
-	static ssize_t var##_store(struct kobject *kobj,		    \
-				     struct attribute *attr,		    \
-				     const char *buffer,		    \
-				     size_t count)			    \
-	{								    \
-		struct ldlm_pool *pl = container_of(kobj, struct ldlm_pool, \
-						    pl_kobj);		    \
-		unsigned long tmp;					    \
-		int rc;							    \
-									    \
-		rc = kstrtoul(buffer, 10, &tmp);			    \
-		if (rc < 0) {						    \
-			return rc;					    \
-		}							    \
-									    \
-		LDLM_POOL_SYSFS_SET_##type(pl->pl_##var, tmp);		    \
-									    \
-		return count;						    \
-	}								    \
-	struct __##var##__dummy_write {; } /* semicolon catcher */
-
-static inline int is_granted_or_cancelled(struct ldlm_lock *lock)
-{
-	int ret = 0;
-
-	lock_res_and_lock(lock);
-	if ((lock->l_req_mode == lock->l_granted_mode) &&
-	    !ldlm_is_cp_reqd(lock))
-		ret = 1;
-	else if (ldlm_is_failed(lock) || ldlm_is_cancel(lock))
-		ret = 1;
-	unlock_res_and_lock(lock);
-
-	return ret;
-}
-
-typedef void (*ldlm_policy_wire_to_local_t)(const union ldlm_wire_policy_data *,
-					    union ldlm_policy_data *);
-
-typedef void (*ldlm_policy_local_to_wire_t)(const union ldlm_policy_data *,
-					    union ldlm_wire_policy_data *);
-
-void ldlm_plain_policy_wire_to_local(const union ldlm_wire_policy_data *wpolicy,
-				     union ldlm_policy_data *lpolicy);
-void ldlm_plain_policy_local_to_wire(const union ldlm_policy_data *lpolicy,
-				     union ldlm_wire_policy_data *wpolicy);
-void ldlm_ibits_policy_wire_to_local(const union ldlm_wire_policy_data *wpolicy,
-				     union ldlm_policy_data *lpolicy);
-void ldlm_ibits_policy_local_to_wire(const union ldlm_policy_data *lpolicy,
-				     union ldlm_wire_policy_data *wpolicy);
-void ldlm_extent_policy_wire_to_local(const union ldlm_wire_policy_data *wpolicy,
-				      union ldlm_policy_data *lpolicy);
-void ldlm_extent_policy_local_to_wire(const union ldlm_policy_data *lpolicy,
-				      union ldlm_wire_policy_data *wpolicy);
-void ldlm_flock_policy_wire_to_local(const union ldlm_wire_policy_data *wpolicy,
-				     union ldlm_policy_data *lpolicy);
-void ldlm_flock_policy_local_to_wire(const union ldlm_policy_data *lpolicy,
-				     union ldlm_wire_policy_data *wpolicy);
-
-static inline bool ldlm_res_eq(const struct ldlm_res_id *res0,
-			       const struct ldlm_res_id *res1)
-{
-	return memcmp(res0, res1, sizeof(*res0)) == 0;
-}
diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_lib.c b/drivers/staging/lustre/lustre/ldlm/ldlm_lib.c
deleted file mode 100644
index 0aa4f234a4f4..000000000000
--- a/drivers/staging/lustre/lustre/ldlm/ldlm_lib.c
+++ /dev/null
@@ -1,842 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-/*
- * GPL HEADER START
- *
- * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 only,
- * as published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful, but
- * WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
- * General Public License version 2 for more details (a copy is included
- * in the LICENSE file that accompanied this code).
- *
- * You should have received a copy of the GNU General Public License
- * version 2 along with this program; If not, see
- * http://www.gnu.org/licenses/gpl-2.0.html
- *
- * GPL HEADER END
- */
-/*
- * Copyright (c) 2003, 2010, Oracle and/or its affiliates. All rights reserved.
- * Use is subject to license terms.
- *
- * Copyright (c) 2010, 2015, Intel Corporation.
- */
-/*
- * This file is part of Lustre, http://www.lustre.org/
- * Lustre is a trademark of Sun Microsystems, Inc.
- */
-
-/**
- * This file deals with various client/target related logic including recovery.
- *
- * TODO: This code more logically belongs in the ptlrpc module than in ldlm and
- * should be moved.
- */
-
-#define DEBUG_SUBSYSTEM S_LDLM
-
-#include <obd.h>
-#include <obd_class.h>
-#include <lustre_dlm.h>
-#include <lustre_net.h>
-#include <lustre_sec.h>
-#include "ldlm_internal.h"
-
-/* @priority: If non-zero, move the selected connection to the list head.
- * @create: If zero, only search in existing connections.
- */
-static int import_set_conn(struct obd_import *imp, struct obd_uuid *uuid,
-			   int priority, int create)
-{
-	struct ptlrpc_connection *ptlrpc_conn;
-	struct obd_import_conn *imp_conn = NULL, *item;
-	int rc = 0;
-
-	if (!create && !priority) {
-		CDEBUG(D_HA, "Nothing to do\n");
-		return -EINVAL;
-	}
-
-	ptlrpc_conn = ptlrpc_uuid_to_connection(uuid);
-	if (!ptlrpc_conn) {
-		CDEBUG(D_HA, "can't find connection %s\n", uuid->uuid);
-		return -ENOENT;
-	}
-
-	if (create) {
-		imp_conn = kzalloc(sizeof(*imp_conn), GFP_NOFS);
-		if (!imp_conn) {
-			rc = -ENOMEM;
-			goto out_put;
-		}
-	}
-
-	spin_lock(&imp->imp_lock);
-	list_for_each_entry(item, &imp->imp_conn_list, oic_item) {
-		if (obd_uuid_equals(uuid, &item->oic_uuid)) {
-			if (priority) {
-				list_del(&item->oic_item);
-				list_add(&item->oic_item,
-					 &imp->imp_conn_list);
-				item->oic_last_attempt = 0;
-			}
-			CDEBUG(D_HA, "imp %p@%s: found existing conn %s%s\n",
-			       imp, imp->imp_obd->obd_name, uuid->uuid,
-			       (priority ? ", moved to head" : ""));
-			spin_unlock(&imp->imp_lock);
-			rc = 0;
-			goto out_free;
-		}
-	}
-	/* No existing import connection found for \a uuid. */
-	if (create) {
-		imp_conn->oic_conn = ptlrpc_conn;
-		imp_conn->oic_uuid = *uuid;
-		imp_conn->oic_last_attempt = 0;
-		if (priority)
-			list_add(&imp_conn->oic_item, &imp->imp_conn_list);
-		else
-			list_add_tail(&imp_conn->oic_item,
-				      &imp->imp_conn_list);
-		CDEBUG(D_HA, "imp %p@%s: add connection %s at %s\n",
-		       imp, imp->imp_obd->obd_name, uuid->uuid,
-		       (priority ? "head" : "tail"));
-	} else {
-		spin_unlock(&imp->imp_lock);
-		rc = -ENOENT;
-		goto out_free;
-	}
-
-	spin_unlock(&imp->imp_lock);
-	return 0;
-out_free:
-	kfree(imp_conn);
-out_put:
-	ptlrpc_connection_put(ptlrpc_conn);
-	return rc;
-}
-
-int import_set_conn_priority(struct obd_import *imp, struct obd_uuid *uuid)
-{
-	return import_set_conn(imp, uuid, 1, 0);
-}
-
-int client_import_add_conn(struct obd_import *imp, struct obd_uuid *uuid,
-			   int priority)
-{
-	return import_set_conn(imp, uuid, priority, 1);
-}
-EXPORT_SYMBOL(client_import_add_conn);
-
-int client_import_del_conn(struct obd_import *imp, struct obd_uuid *uuid)
-{
-	struct obd_import_conn *imp_conn;
-	struct obd_export *dlmexp;
-	int rc = -ENOENT;
-
-	spin_lock(&imp->imp_lock);
-	if (list_empty(&imp->imp_conn_list)) {
-		LASSERT(!imp->imp_connection);
-		goto out;
-	}
-
-	list_for_each_entry(imp_conn, &imp->imp_conn_list, oic_item) {
-		if (!obd_uuid_equals(uuid, &imp_conn->oic_uuid))
-			continue;
-		LASSERT(imp_conn->oic_conn);
-
-		if (imp_conn == imp->imp_conn_current) {
-			LASSERT(imp_conn->oic_conn == imp->imp_connection);
-
-			if (imp->imp_state != LUSTRE_IMP_CLOSED &&
-			    imp->imp_state != LUSTRE_IMP_DISCON) {
-				CERROR("can't remove current connection\n");
-				rc = -EBUSY;
-				goto out;
-			}
-
-			ptlrpc_connection_put(imp->imp_connection);
-			imp->imp_connection = NULL;
-
-			dlmexp = class_conn2export(&imp->imp_dlm_handle);
-			if (dlmexp && dlmexp->exp_connection) {
-				LASSERT(dlmexp->exp_connection ==
-					imp_conn->oic_conn);
-				ptlrpc_connection_put(dlmexp->exp_connection);
-				dlmexp->exp_connection = NULL;
-			}
-
-			if (dlmexp)
-				class_export_put(dlmexp);
-		}
-
-		list_del(&imp_conn->oic_item);
-		ptlrpc_connection_put(imp_conn->oic_conn);
-		kfree(imp_conn);
-		CDEBUG(D_HA, "imp %p@%s: remove connection %s\n",
-		       imp, imp->imp_obd->obd_name, uuid->uuid);
-		rc = 0;
-		break;
-	}
-out:
-	spin_unlock(&imp->imp_lock);
-	if (rc == -ENOENT)
-		CERROR("connection %s not found\n", uuid->uuid);
-	return rc;
-}
-EXPORT_SYMBOL(client_import_del_conn);
-
-/**
- * Find conn UUID by peer NID. \a peer is a server NID. This function is used
- * to find a conn uuid of \a imp which can reach \a peer.
- */
-int client_import_find_conn(struct obd_import *imp, lnet_nid_t peer,
-			    struct obd_uuid *uuid)
-{
-	struct obd_import_conn *conn;
-	int rc = -ENOENT;
-
-	spin_lock(&imp->imp_lock);
-	list_for_each_entry(conn, &imp->imp_conn_list, oic_item) {
-		/* Check if conn UUID does have this peer NID. */
-		if (class_check_uuid(&conn->oic_uuid, peer)) {
-			*uuid = conn->oic_uuid;
-			rc = 0;
-			break;
-		}
-	}
-	spin_unlock(&imp->imp_lock);
-	return rc;
-}
-EXPORT_SYMBOL(client_import_find_conn);
-
-void client_destroy_import(struct obd_import *imp)
-{
-	/* Drop security policy instance after all RPCs have finished/aborted
-	 * to let all busy contexts be released.
-	 */
-	class_import_get(imp);
-	class_destroy_import(imp);
-	sptlrpc_import_sec_put(imp);
-	class_import_put(imp);
-}
-EXPORT_SYMBOL(client_destroy_import);
-
-/* Configure an RPC client OBD device.
- *
- * lcfg parameters:
- * 1 - client UUID
- * 2 - server UUID
- * 3 - inactive-on-startup
- */
-int client_obd_setup(struct obd_device *obddev, struct lustre_cfg *lcfg)
-{
-	struct client_obd *cli = &obddev->u.cli;
-	struct obd_import *imp;
-	struct obd_uuid server_uuid;
-	int rq_portal, rp_portal, connect_op;
-	char *name = obddev->obd_type->typ_name;
-	enum ldlm_ns_type ns_type = LDLM_NS_TYPE_UNKNOWN;
-	int rc;
-
-	/* In a more perfect world, we would hang a ptlrpc_client off of
-	 * obd_type and just use the values from there.
-	 */
-	if (!strcmp(name, LUSTRE_OSC_NAME)) {
-		rq_portal = OST_REQUEST_PORTAL;
-		rp_portal = OSC_REPLY_PORTAL;
-		connect_op = OST_CONNECT;
-		cli->cl_sp_me = LUSTRE_SP_CLI;
-		cli->cl_sp_to = LUSTRE_SP_OST;
-		ns_type = LDLM_NS_TYPE_OSC;
-	} else if (!strcmp(name, LUSTRE_MDC_NAME) ||
-		   !strcmp(name, LUSTRE_LWP_NAME)) {
-		rq_portal = MDS_REQUEST_PORTAL;
-		rp_portal = MDC_REPLY_PORTAL;
-		connect_op = MDS_CONNECT;
-		cli->cl_sp_me = LUSTRE_SP_CLI;
-		cli->cl_sp_to = LUSTRE_SP_MDT;
-		ns_type = LDLM_NS_TYPE_MDC;
-	} else if (!strcmp(name, LUSTRE_MGC_NAME)) {
-		rq_portal = MGS_REQUEST_PORTAL;
-		rp_portal = MGC_REPLY_PORTAL;
-		connect_op = MGS_CONNECT;
-		cli->cl_sp_me = LUSTRE_SP_MGC;
-		cli->cl_sp_to = LUSTRE_SP_MGS;
-		cli->cl_flvr_mgc.sf_rpc = SPTLRPC_FLVR_INVALID;
-		ns_type = LDLM_NS_TYPE_MGC;
-	} else {
-		CERROR("unknown client OBD type \"%s\", can't setup\n",
-		       name);
-		return -EINVAL;
-	}
-
-	if (LUSTRE_CFG_BUFLEN(lcfg, 1) < 1) {
-		CERROR("requires a TARGET UUID\n");
-		return -EINVAL;
-	}
-
-	if (LUSTRE_CFG_BUFLEN(lcfg, 1) > 37) {
-		CERROR("client UUID must be less than 38 characters\n");
-		return -EINVAL;
-	}
-
-	if (LUSTRE_CFG_BUFLEN(lcfg, 2) < 1) {
-		CERROR("setup requires a SERVER UUID\n");
-		return -EINVAL;
-	}
-
-	if (LUSTRE_CFG_BUFLEN(lcfg, 2) > 37) {
-		CERROR("target UUID must be less than 38 characters\n");
-		return -EINVAL;
-	}
-
-	init_rwsem(&cli->cl_sem);
-	cli->cl_conn_count = 0;
-	memcpy(server_uuid.uuid, lustre_cfg_buf(lcfg, 2),
-	       min_t(unsigned int, LUSTRE_CFG_BUFLEN(lcfg, 2),
-		     sizeof(server_uuid)));
-
-	cli->cl_dirty_pages = 0;
-	cli->cl_avail_grant = 0;
-	/* FIXME: Should limit this for the sum of all cl_dirty_max_pages. */
-	/*
-	 * cl_dirty_max_pages may be changed at connect time in
-	 * ptlrpc_connect_interpret().
-	 */
-	client_adjust_max_dirty(cli);
-	INIT_LIST_HEAD(&cli->cl_cache_waiters);
-	INIT_LIST_HEAD(&cli->cl_loi_ready_list);
-	INIT_LIST_HEAD(&cli->cl_loi_hp_ready_list);
-	INIT_LIST_HEAD(&cli->cl_loi_write_list);
-	INIT_LIST_HEAD(&cli->cl_loi_read_list);
-	spin_lock_init(&cli->cl_loi_list_lock);
-	atomic_set(&cli->cl_pending_w_pages, 0);
-	atomic_set(&cli->cl_pending_r_pages, 0);
-	cli->cl_r_in_flight = 0;
-	cli->cl_w_in_flight = 0;
-
-	spin_lock_init(&cli->cl_read_rpc_hist.oh_lock);
-	spin_lock_init(&cli->cl_write_rpc_hist.oh_lock);
-	spin_lock_init(&cli->cl_read_page_hist.oh_lock);
-	spin_lock_init(&cli->cl_write_page_hist.oh_lock);
-	spin_lock_init(&cli->cl_read_offset_hist.oh_lock);
-	spin_lock_init(&cli->cl_write_offset_hist.oh_lock);
-
-	/* lru for osc. */
-	INIT_LIST_HEAD(&cli->cl_lru_osc);
-	atomic_set(&cli->cl_lru_shrinkers, 0);
-	atomic_long_set(&cli->cl_lru_busy, 0);
-	atomic_long_set(&cli->cl_lru_in_list, 0);
-	INIT_LIST_HEAD(&cli->cl_lru_list);
-	spin_lock_init(&cli->cl_lru_list_lock);
-	atomic_long_set(&cli->cl_unstable_count, 0);
-	INIT_LIST_HEAD(&cli->cl_shrink_list);
-
-	init_waitqueue_head(&cli->cl_destroy_waitq);
-	atomic_set(&cli->cl_destroy_in_flight, 0);
-	/* Turn on checksumming by default. */
-	cli->cl_checksum = 1;
-	/*
-	 * The supported checksum types will be worked out at connect time
-	 * Set cl_chksum* to CRC32 for now to avoid returning screwed info
-	 * through procfs.
-	 */
-	cli->cl_cksum_type = OBD_CKSUM_CRC32;
-	cli->cl_supp_cksum_types = OBD_CKSUM_CRC32;
-	atomic_set(&cli->cl_resends, OSC_DEFAULT_RESENDS);
-
-	/*
-	 * Set it to possible maximum size. It may be reduced by ocd_brw_size
-	 * from OFD after connecting.
-	 */
-	cli->cl_max_pages_per_rpc = PTLRPC_MAX_BRW_PAGES;
-
-	/*
-	 * set cl_chunkbits default value to PAGE_CACHE_SHIFT,
-	 * it will be updated at OSC connection time.
-	 */
-	cli->cl_chunkbits = PAGE_SHIFT;
-
-	if (!strcmp(name, LUSTRE_MDC_NAME))
-		cli->cl_max_rpcs_in_flight = OBD_MAX_RIF_DEFAULT;
-	else if (totalram_pages >> (20 - PAGE_SHIFT) <= 128 /* MB */)
-		cli->cl_max_rpcs_in_flight = 2;
-	else if (totalram_pages >> (20 - PAGE_SHIFT) <= 256 /* MB */)
-		cli->cl_max_rpcs_in_flight = 3;
-	else if (totalram_pages >> (20 - PAGE_SHIFT) <= 512 /* MB */)
-		cli->cl_max_rpcs_in_flight = 4;
-	else
-		cli->cl_max_rpcs_in_flight = OBD_MAX_RIF_DEFAULT;
-
-	spin_lock_init(&cli->cl_mod_rpcs_lock);
-	spin_lock_init(&cli->cl_mod_rpcs_hist.oh_lock);
-	cli->cl_max_mod_rpcs_in_flight = 0;
-	cli->cl_mod_rpcs_in_flight = 0;
-	cli->cl_close_rpcs_in_flight = 0;
-	init_waitqueue_head(&cli->cl_mod_rpcs_waitq);
-	cli->cl_mod_tag_bitmap = NULL;
-
-	if (connect_op == MDS_CONNECT) {
-		cli->cl_max_mod_rpcs_in_flight = cli->cl_max_rpcs_in_flight - 1;
-		cli->cl_mod_tag_bitmap = kcalloc(BITS_TO_LONGS(OBD_MAX_RIF_MAX),
-						 sizeof(long), GFP_NOFS);
-		if (!cli->cl_mod_tag_bitmap) {
-			rc = -ENOMEM;
-			goto err;
-		}
-	}
-
-	rc = ldlm_get_ref();
-	if (rc) {
-		CERROR("ldlm_get_ref failed: %d\n", rc);
-		goto err;
-	}
-
-	ptlrpc_init_client(rq_portal, rp_portal, name,
-			   &obddev->obd_ldlm_client);
-
-	imp = class_new_import(obddev);
-	if (!imp) {
-		rc = -ENOENT;
-		goto err_ldlm;
-	}
-	imp->imp_client = &obddev->obd_ldlm_client;
-	imp->imp_connect_op = connect_op;
-	memcpy(cli->cl_target_uuid.uuid, lustre_cfg_buf(lcfg, 1),
-	       LUSTRE_CFG_BUFLEN(lcfg, 1));
-	class_import_put(imp);
-
-	rc = client_import_add_conn(imp, &server_uuid, 1);
-	if (rc) {
-		CERROR("can't add initial connection\n");
-		goto err_import;
-	}
-
-	cli->cl_import = imp;
-	/* cli->cl_max_mds_easize updated by mdc_init_ea_size() */
-	cli->cl_max_mds_easize = sizeof(struct lov_mds_md_v3);
-
-	if (LUSTRE_CFG_BUFLEN(lcfg, 3) > 0) {
-		if (!strcmp(lustre_cfg_string(lcfg, 3), "inactive")) {
-			CDEBUG(D_HA, "marking %s %s->%s as inactive\n",
-			       name, obddev->obd_name,
-			       cli->cl_target_uuid.uuid);
-			spin_lock(&imp->imp_lock);
-			imp->imp_deactive = 1;
-			spin_unlock(&imp->imp_lock);
-		}
-	}
-
-	obddev->obd_namespace = ldlm_namespace_new(obddev, obddev->obd_name,
-						   LDLM_NAMESPACE_CLIENT,
-						   LDLM_NAMESPACE_GREEDY,
-						   ns_type);
-	if (!obddev->obd_namespace) {
-		CERROR("Unable to create client namespace - %s\n",
-		       obddev->obd_name);
-		rc = -ENOMEM;
-		goto err_import;
-	}
-
-	return rc;
-
-err_import:
-	class_destroy_import(imp);
-err_ldlm:
-	ldlm_put_ref();
-err:
-	kfree(cli->cl_mod_tag_bitmap);
-	cli->cl_mod_tag_bitmap = NULL;
-	return rc;
-}
-EXPORT_SYMBOL(client_obd_setup);
-
-int client_obd_cleanup(struct obd_device *obddev)
-{
-	struct client_obd *cli = &obddev->u.cli;
-
-	ldlm_namespace_free_post(obddev->obd_namespace);
-	obddev->obd_namespace = NULL;
-
-	obd_cleanup_client_import(obddev);
-	LASSERT(!obddev->u.cli.cl_import);
-
-	ldlm_put_ref();
-
-	kfree(cli->cl_mod_tag_bitmap);
-	cli->cl_mod_tag_bitmap = NULL;
-
-	return 0;
-}
-EXPORT_SYMBOL(client_obd_cleanup);
-
-/* ->o_connect() method for client side (OSC and MDC and MGC) */
-int client_connect_import(const struct lu_env *env,
-			  struct obd_export **exp,
-			  struct obd_device *obd, struct obd_uuid *cluuid,
-			  struct obd_connect_data *data, void *localdata)
-{
-	struct client_obd       *cli    = &obd->u.cli;
-	struct obd_import       *imp    = cli->cl_import;
-	struct obd_connect_data *ocd;
-	struct lustre_handle    conn    = { 0 };
-	bool is_mdc = false;
-	int		     rc;
-
-	*exp = NULL;
-	down_write(&cli->cl_sem);
-	if (cli->cl_conn_count > 0) {
-		rc = -EALREADY;
-		goto out_sem;
-	}
-
-	rc = class_connect(&conn, obd, cluuid);
-	if (rc)
-		goto out_sem;
-
-	cli->cl_conn_count++;
-	*exp = class_conn2export(&conn);
-
-	LASSERT(obd->obd_namespace);
-
-	imp->imp_dlm_handle = conn;
-	rc = ptlrpc_init_import(imp);
-	if (rc != 0)
-		goto out_ldlm;
-
-	ocd = &imp->imp_connect_data;
-	if (data) {
-		*ocd = *data;
-		is_mdc = !strncmp(imp->imp_obd->obd_type->typ_name,
-				  LUSTRE_MDC_NAME, 3);
-		if (is_mdc)
-			data->ocd_connect_flags |= OBD_CONNECT_MULTIMODRPCS;
-		imp->imp_connect_flags_orig = data->ocd_connect_flags;
-	}
-
-	rc = ptlrpc_connect_import(imp);
-	if (rc != 0) {
-		if (data && is_mdc)
-			data->ocd_connect_flags &= ~OBD_CONNECT_MULTIMODRPCS;
-		LASSERT(imp->imp_state == LUSTRE_IMP_DISCON);
-		goto out_ldlm;
-	}
-	LASSERT(*exp && (*exp)->exp_connection);
-
-	if (data) {
-		LASSERTF((ocd->ocd_connect_flags & data->ocd_connect_flags) ==
-			 ocd->ocd_connect_flags, "old %#llx, new %#llx\n",
-			 data->ocd_connect_flags, ocd->ocd_connect_flags);
-		data->ocd_connect_flags = ocd->ocd_connect_flags;
-		/* clear the flag as it was not set and is not known
-		 * by upper layers
-		 */
-		if (is_mdc)
-			data->ocd_connect_flags &= ~OBD_CONNECT_MULTIMODRPCS;
-	}
-
-	ptlrpc_pinger_add_import(imp);
-
-	if (rc) {
-out_ldlm:
-		cli->cl_conn_count--;
-		class_disconnect(*exp);
-		*exp = NULL;
-	}
-out_sem:
-	up_write(&cli->cl_sem);
-
-	return rc;
-}
-EXPORT_SYMBOL(client_connect_import);
-
-int client_disconnect_export(struct obd_export *exp)
-{
-	struct obd_device *obd = class_exp2obd(exp);
-	struct client_obd *cli;
-	struct obd_import *imp;
-	int rc = 0, err;
-
-	if (!obd) {
-		CERROR("invalid export for disconnect: exp %p cookie %#llx\n",
-		       exp, exp ? exp->exp_handle.h_cookie : -1);
-		return -EINVAL;
-	}
-
-	cli = &obd->u.cli;
-	imp = cli->cl_import;
-
-	down_write(&cli->cl_sem);
-	CDEBUG(D_INFO, "disconnect %s - %zu\n", obd->obd_name,
-	       cli->cl_conn_count);
-
-	if (!cli->cl_conn_count) {
-		CERROR("disconnecting disconnected device (%s)\n",
-		       obd->obd_name);
-		rc = -EINVAL;
-		goto out_disconnect;
-	}
-
-	cli->cl_conn_count--;
-	if (cli->cl_conn_count) {
-		rc = 0;
-		goto out_disconnect;
-	}
-
-	/* Mark import deactivated now, so we don't try to reconnect if any
-	 * of the cleanup RPCs fails (e.g. LDLM cancel, etc).  We don't
-	 * fully deactivate the import, or that would drop all requests.
-	 */
-	spin_lock(&imp->imp_lock);
-	imp->imp_deactive = 1;
-	spin_unlock(&imp->imp_lock);
-
-	/* Some non-replayable imports (MDS's OSCs) are pinged, so just
-	 * delete it regardless.  (It's safe to delete an import that was
-	 * never added.)
-	 */
-	(void)ptlrpc_pinger_del_import(imp);
-
-	if (obd->obd_namespace) {
-		/* obd_force == local only */
-		ldlm_cli_cancel_unused(obd->obd_namespace, NULL,
-				       obd->obd_force ? LCF_LOCAL : 0, NULL);
-		ldlm_namespace_free_prior(obd->obd_namespace, imp,
-					  obd->obd_force);
-	}
-
-	/* There's no need to hold sem while disconnecting an import,
-	 * and it may actually cause deadlock in GSS.
-	 */
-	up_write(&cli->cl_sem);
-	rc = ptlrpc_disconnect_import(imp, 0);
-	down_write(&cli->cl_sem);
-
-	ptlrpc_invalidate_import(imp);
-
-out_disconnect:
-	/* Use server style - class_disconnect should be always called for
-	 * o_disconnect.
-	 */
-	err = class_disconnect(exp);
-	if (!rc && err)
-		rc = err;
-
-	up_write(&cli->cl_sem);
-
-	return rc;
-}
-EXPORT_SYMBOL(client_disconnect_export);
-
-/**
- * Packs current SLV and Limit into \a req.
- */
-int target_pack_pool_reply(struct ptlrpc_request *req)
-{
-	struct obd_device *obd;
-
-	/* Check that we still have all structures alive as this may
-	 * be some late RPC at shutdown time.
-	 */
-	if (unlikely(!req->rq_export || !req->rq_export->exp_obd ||
-		     !exp_connect_lru_resize(req->rq_export))) {
-		lustre_msg_set_slv(req->rq_repmsg, 0);
-		lustre_msg_set_limit(req->rq_repmsg, 0);
-		return 0;
-	}
-
-	/* OBD is alive here as export is alive, which we checked above. */
-	obd = req->rq_export->exp_obd;
-
-	read_lock(&obd->obd_pool_lock);
-	lustre_msg_set_slv(req->rq_repmsg, obd->obd_pool_slv);
-	lustre_msg_set_limit(req->rq_repmsg, obd->obd_pool_limit);
-	read_unlock(&obd->obd_pool_lock);
-
-	return 0;
-}
-EXPORT_SYMBOL(target_pack_pool_reply);
-
-static int
-target_send_reply_msg(struct ptlrpc_request *req, int rc, int fail_id)
-{
-	if (OBD_FAIL_CHECK_ORSET(fail_id & ~OBD_FAIL_ONCE, OBD_FAIL_ONCE)) {
-		DEBUG_REQ(D_ERROR, req, "dropping reply");
-		return -ECOMM;
-	}
-
-	if (unlikely(rc)) {
-		DEBUG_REQ(D_NET, req, "processing error (%d)", rc);
-		req->rq_status = rc;
-		return ptlrpc_send_error(req, 1);
-	}
-
-	DEBUG_REQ(D_NET, req, "sending reply");
-	return ptlrpc_send_reply(req, PTLRPC_REPLY_MAYBE_DIFFICULT);
-}
-
-void target_send_reply(struct ptlrpc_request *req, int rc, int fail_id)
-{
-	struct ptlrpc_service_part *svcpt;
-	int			netrc;
-	struct ptlrpc_reply_state *rs;
-	struct obd_export	 *exp;
-
-	if (req->rq_no_reply)
-		return;
-
-	svcpt = req->rq_rqbd->rqbd_svcpt;
-	rs = req->rq_reply_state;
-	if (!rs || !rs->rs_difficult) {
-		/* no notifiers */
-		target_send_reply_msg(req, rc, fail_id);
-		return;
-	}
-
-	/* must be an export if locks saved */
-	LASSERT(req->rq_export);
-	/* req/reply consistent */
-	LASSERT(rs->rs_svcpt == svcpt);
-
-	/* "fresh" reply */
-	LASSERT(!rs->rs_scheduled);
-	LASSERT(!rs->rs_scheduled_ever);
-	LASSERT(!rs->rs_handled);
-	LASSERT(!rs->rs_on_net);
-	LASSERT(!rs->rs_export);
-	LASSERT(list_empty(&rs->rs_obd_list));
-	LASSERT(list_empty(&rs->rs_exp_list));
-
-	exp = class_export_get(req->rq_export);
-
-	/* disable reply scheduling while I'm setting up */
-	rs->rs_scheduled = 1;
-	rs->rs_on_net    = 1;
-	rs->rs_xid       = req->rq_xid;
-	rs->rs_transno   = req->rq_transno;
-	rs->rs_export    = exp;
-	rs->rs_opc       = lustre_msg_get_opc(req->rq_reqmsg);
-
-	spin_lock(&exp->exp_uncommitted_replies_lock);
-	CDEBUG(D_NET, "rs transno = %llu, last committed = %llu\n",
-	       rs->rs_transno, exp->exp_last_committed);
-	if (rs->rs_transno > exp->exp_last_committed) {
-		/* not committed already */
-		list_add_tail(&rs->rs_obd_list,
-			      &exp->exp_uncommitted_replies);
-	}
-	spin_unlock(&exp->exp_uncommitted_replies_lock);
-
-	spin_lock(&exp->exp_lock);
-	list_add_tail(&rs->rs_exp_list, &exp->exp_outstanding_replies);
-	spin_unlock(&exp->exp_lock);
-
-	netrc = target_send_reply_msg(req, rc, fail_id);
-
-	spin_lock(&svcpt->scp_rep_lock);
-
-	atomic_inc(&svcpt->scp_nreps_difficult);
-
-	if (netrc != 0) {
-		/* error sending: reply is off the net.  Also we need +1
-		 * reply ref until ptlrpc_handle_rs() is done
-		 * with the reply state (if the send was successful, there
-		 * would have been +1 ref for the net, which
-		 * reply_out_callback leaves alone)
-		 */
-		rs->rs_on_net = 0;
-		ptlrpc_rs_addref(rs);
-	}
-
-	spin_lock(&rs->rs_lock);
-	if (rs->rs_transno <= exp->exp_last_committed ||
-	    (!rs->rs_on_net && !rs->rs_no_ack) ||
-	    list_empty(&rs->rs_exp_list) ||     /* completed already */
-	    list_empty(&rs->rs_obd_list)) {
-		CDEBUG(D_HA, "Schedule reply immediately\n");
-		ptlrpc_dispatch_difficult_reply(rs);
-	} else {
-		list_add(&rs->rs_list, &svcpt->scp_rep_active);
-		rs->rs_scheduled = 0;	/* allow notifier to schedule */
-	}
-	spin_unlock(&rs->rs_lock);
-	spin_unlock(&svcpt->scp_rep_lock);
-}
-EXPORT_SYMBOL(target_send_reply);
-
-enum ldlm_mode lck_compat_array[] = {
-	[LCK_EX]	= LCK_COMPAT_EX,
-	[LCK_PW]	= LCK_COMPAT_PW,
-	[LCK_PR]	= LCK_COMPAT_PR,
-	[LCK_CW]	= LCK_COMPAT_CW,
-	[LCK_CR]	= LCK_COMPAT_CR,
-	[LCK_NL]	= LCK_COMPAT_NL,
-	[LCK_GROUP]	= LCK_COMPAT_GROUP,
-	[LCK_COS]	= LCK_COMPAT_COS,
-};
-
-/**
- * Rather arbitrary mapping from LDLM error codes to errno values. This should
- * not escape to the user level.
- */
-int ldlm_error2errno(enum ldlm_error error)
-{
-	int result;
-
-	switch (error) {
-	case ELDLM_OK:
-	case ELDLM_LOCK_MATCHED:
-		result = 0;
-		break;
-	case ELDLM_LOCK_CHANGED:
-		result = -ESTALE;
-		break;
-	case ELDLM_LOCK_ABORTED:
-		result = -ENAVAIL;
-		break;
-	case ELDLM_LOCK_REPLACED:
-		result = -ESRCH;
-		break;
-	case ELDLM_NO_LOCK_DATA:
-		result = -ENOENT;
-		break;
-	case ELDLM_NAMESPACE_EXISTS:
-		result = -EEXIST;
-		break;
-	case ELDLM_BAD_NAMESPACE:
-		result = -EBADF;
-		break;
-	default:
-		if (((int)error) < 0)  /* cast to signed type */
-			result = error; /* as enum ldlm_error can be unsigned */
-		else {
-			CERROR("Invalid DLM result code: %d\n", error);
-			result = -EPROTO;
-		}
-	}
-	return result;
-}
-EXPORT_SYMBOL(ldlm_error2errno);
-
-#if LUSTRE_TRACKS_LOCK_EXP_REFS
-void ldlm_dump_export_locks(struct obd_export *exp)
-{
-	spin_lock(&exp->exp_locks_list_guard);
-	if (!list_empty(&exp->exp_locks_list)) {
-		struct ldlm_lock *lock;
-
-		CERROR("dumping locks for export %p,ignore if the unmount doesn't hang\n",
-		       exp);
-		list_for_each_entry(lock, &exp->exp_locks_list,
-				    l_exp_refs_link)
-			LDLM_ERROR(lock, "lock:");
-	}
-	spin_unlock(&exp->exp_locks_list_guard);
-}
-#endif
diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c b/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c
deleted file mode 100644
index 2fb2e088dc87..000000000000
--- a/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c
+++ /dev/null
@@ -1,2103 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-/*
- * GPL HEADER START
- *
- * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 only,
- * as published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful, but
- * WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
- * General Public License version 2 for more details (a copy is included
- * in the LICENSE file that accompanied this code).
- *
- * You should have received a copy of the GNU General Public License
- * version 2 along with this program; If not, see
- * http://www.gnu.org/licenses/gpl-2.0.html
- *
- * GPL HEADER END
- */
-/*
- * Copyright (c) 2002, 2010, Oracle and/or its affiliates. All rights reserved.
- * Use is subject to license terms.
- *
- * Copyright (c) 2010, 2015, Intel Corporation.
- */
-/*
- * This file is part of Lustre, http://www.lustre.org/
- * Lustre is a trademark of Sun Microsystems, Inc.
- *
- * lustre/ldlm/ldlm_lock.c
- *
- * Author: Peter Braam <braam at clusterfs.com>
- * Author: Phil Schwan <phil at clusterfs.com>
- */
-
-#define DEBUG_SUBSYSTEM S_LDLM
-
-#include <lustre_intent.h>
-#include <lustre_swab.h>
-#include <obd_class.h>
-#include "ldlm_internal.h"
-
-/* lock types */
-char *ldlm_lockname[] = {
-	[0]		= "--",
-	[LCK_EX]	= "EX",
-	[LCK_PW]	= "PW",
-	[LCK_PR]	= "PR",
-	[LCK_CW]	= "CW",
-	[LCK_CR]	= "CR",
-	[LCK_NL]	= "NL",
-	[LCK_GROUP]	= "GROUP",
-	[LCK_COS]	= "COS",
-};
-EXPORT_SYMBOL(ldlm_lockname);
-
-static char *ldlm_typename[] = {
-	[LDLM_PLAIN]	= "PLN",
-	[LDLM_EXTENT]	= "EXT",
-	[LDLM_FLOCK]	= "FLK",
-	[LDLM_IBITS]	= "IBT",
-};
-
-static ldlm_policy_wire_to_local_t ldlm_policy_wire_to_local[] = {
-	[LDLM_PLAIN - LDLM_MIN_TYPE]	= ldlm_plain_policy_wire_to_local,
-	[LDLM_EXTENT - LDLM_MIN_TYPE]	= ldlm_extent_policy_wire_to_local,
-	[LDLM_FLOCK - LDLM_MIN_TYPE]	= ldlm_flock_policy_wire_to_local,
-	[LDLM_IBITS - LDLM_MIN_TYPE]	= ldlm_ibits_policy_wire_to_local,
-};
-
-static ldlm_policy_local_to_wire_t ldlm_policy_local_to_wire[] = {
-	[LDLM_PLAIN - LDLM_MIN_TYPE]	= ldlm_plain_policy_local_to_wire,
-	[LDLM_EXTENT - LDLM_MIN_TYPE]	= ldlm_extent_policy_local_to_wire,
-	[LDLM_FLOCK - LDLM_MIN_TYPE]	= ldlm_flock_policy_local_to_wire,
-	[LDLM_IBITS - LDLM_MIN_TYPE]	= ldlm_ibits_policy_local_to_wire,
-};
-
-/**
- * Converts lock policy from local format to on the wire lock_desc format
- */
-static void ldlm_convert_policy_to_wire(enum ldlm_type type,
-					const union ldlm_policy_data *lpolicy,
-					union ldlm_wire_policy_data *wpolicy)
-{
-	ldlm_policy_local_to_wire_t convert;
-
-	convert = ldlm_policy_local_to_wire[type - LDLM_MIN_TYPE];
-
-	convert(lpolicy, wpolicy);
-}
-
-/**
- * Converts lock policy from on the wire lock_desc format to local format
- */
-void ldlm_convert_policy_to_local(struct obd_export *exp, enum ldlm_type type,
-				  const union ldlm_wire_policy_data *wpolicy,
-				  union ldlm_policy_data *lpolicy)
-{
-	ldlm_policy_wire_to_local_t convert;
-
-	convert = ldlm_policy_wire_to_local[type - LDLM_MIN_TYPE];
-
-	convert(wpolicy, lpolicy);
-}
-
-const char *ldlm_it2str(enum ldlm_intent_flags it)
-{
-	switch (it) {
-	case IT_OPEN:
-		return "open";
-	case IT_CREAT:
-		return "creat";
-	case (IT_OPEN | IT_CREAT):
-		return "open|creat";
-	case IT_READDIR:
-		return "readdir";
-	case IT_GETATTR:
-		return "getattr";
-	case IT_LOOKUP:
-		return "lookup";
-	case IT_UNLINK:
-		return "unlink";
-	case IT_GETXATTR:
-		return "getxattr";
-	case IT_LAYOUT:
-		return "layout";
-	default:
-		CERROR("Unknown intent 0x%08x\n", it);
-		return "UNKNOWN";
-	}
-}
-EXPORT_SYMBOL(ldlm_it2str);
-
-/*
- * REFCOUNTED LOCK OBJECTS
- */
-
-/**
- * Get a reference on a lock.
- *
- * Lock refcounts, during creation:
- *   - one special one for allocation, dec'd only once in destroy
- *   - one for being a lock that's in-use
- *   - one for the addref associated with a new lock
- */
-struct ldlm_lock *ldlm_lock_get(struct ldlm_lock *lock)
-{
-	atomic_inc(&lock->l_refc);
-	return lock;
-}
-EXPORT_SYMBOL(ldlm_lock_get);
-
-/**
- * Release lock reference.
- *
- * Also frees the lock if it was last reference.
- */
-void ldlm_lock_put(struct ldlm_lock *lock)
-{
-	LASSERT(lock->l_resource != LP_POISON);
-	LASSERT(atomic_read(&lock->l_refc) > 0);
-	if (atomic_dec_and_test(&lock->l_refc)) {
-		struct ldlm_resource *res;
-
-		LDLM_DEBUG(lock,
-			   "final lock_put on destroyed lock, freeing it.");
-
-		res = lock->l_resource;
-		LASSERT(ldlm_is_destroyed(lock));
-		LASSERT(list_empty(&lock->l_res_link));
-		LASSERT(list_empty(&lock->l_pending_chain));
-
-		lprocfs_counter_decr(ldlm_res_to_ns(res)->ns_stats,
-				     LDLM_NSS_LOCKS);
-		lu_ref_del(&res->lr_reference, "lock", lock);
-		ldlm_resource_putref(res);
-		lock->l_resource = NULL;
-		if (lock->l_export) {
-			class_export_lock_put(lock->l_export, lock);
-			lock->l_export = NULL;
-		}
-
-		kfree(lock->l_lvb_data);
-
-		lu_ref_fini(&lock->l_reference);
-		OBD_FREE_RCU(lock, sizeof(*lock), &lock->l_handle);
-	}
-}
-EXPORT_SYMBOL(ldlm_lock_put);
-
-/**
- * Removes LDLM lock \a lock from LRU. Assumes LRU is already locked.
- */
-int ldlm_lock_remove_from_lru_nolock(struct ldlm_lock *lock)
-{
-	int rc = 0;
-
-	if (!list_empty(&lock->l_lru)) {
-		struct ldlm_namespace *ns = ldlm_lock_to_ns(lock);
-
-		LASSERT(lock->l_resource->lr_type != LDLM_FLOCK);
-		list_del_init(&lock->l_lru);
-		LASSERT(ns->ns_nr_unused > 0);
-		ns->ns_nr_unused--;
-		rc = 1;
-	}
-	return rc;
-}
-
-/**
- * Removes LDLM lock \a lock from LRU. Obtains the LRU lock first.
- *
- * If \a last_use is non-zero, it will remove the lock from LRU only if
- * it matches lock's l_last_used.
- *
- * \retval 0 if \a last_use is set, the lock is not in LRU list or \a last_use
- *           doesn't match lock's l_last_used;
- *           otherwise, the lock hasn't been in the LRU list.
- * \retval 1 the lock was in LRU list and removed.
- */
-int ldlm_lock_remove_from_lru_check(struct ldlm_lock *lock, time_t last_use)
-{
-	struct ldlm_namespace *ns = ldlm_lock_to_ns(lock);
-	int rc = 0;
-
-	spin_lock(&ns->ns_lock);
-	if (last_use == 0 || last_use == lock->l_last_used)
-		rc = ldlm_lock_remove_from_lru_nolock(lock);
-	spin_unlock(&ns->ns_lock);
-
-	return rc;
-}
-
-/**
- * Adds LDLM lock \a lock to namespace LRU. Assumes LRU is already locked.
- */
-static void ldlm_lock_add_to_lru_nolock(struct ldlm_lock *lock)
-{
-	struct ldlm_namespace *ns = ldlm_lock_to_ns(lock);
-
-	lock->l_last_used = jiffies;
-	LASSERT(list_empty(&lock->l_lru));
-	LASSERT(lock->l_resource->lr_type != LDLM_FLOCK);
-	list_add_tail(&lock->l_lru, &ns->ns_unused_list);
-	ldlm_clear_skipped(lock);
-	LASSERT(ns->ns_nr_unused >= 0);
-	ns->ns_nr_unused++;
-}
-
-/**
- * Adds LDLM lock \a lock to namespace LRU. Obtains necessary LRU locks
- * first.
- */
-static void ldlm_lock_add_to_lru(struct ldlm_lock *lock)
-{
-	struct ldlm_namespace *ns = ldlm_lock_to_ns(lock);
-
-	spin_lock(&ns->ns_lock);
-	ldlm_lock_add_to_lru_nolock(lock);
-	spin_unlock(&ns->ns_lock);
-}
-
-/**
- * Moves LDLM lock \a lock that is already in namespace LRU to the tail of
- * the LRU. Performs necessary LRU locking
- */
-static void ldlm_lock_touch_in_lru(struct ldlm_lock *lock)
-{
-	struct ldlm_namespace *ns = ldlm_lock_to_ns(lock);
-
-	spin_lock(&ns->ns_lock);
-	if (!list_empty(&lock->l_lru)) {
-		ldlm_lock_remove_from_lru_nolock(lock);
-		ldlm_lock_add_to_lru_nolock(lock);
-	}
-	spin_unlock(&ns->ns_lock);
-}
-
-/**
- * Helper to destroy a locked lock.
- *
- * Used by ldlm_lock_destroy and ldlm_lock_destroy_nolock
- * Must be called with l_lock and lr_lock held.
- *
- * Does not actually free the lock data, but rather marks the lock as
- * destroyed by setting l_destroyed field in the lock to 1.  Destroys a
- * handle->lock association too, so that the lock can no longer be found
- * and removes the lock from LRU list.  Actual lock freeing occurs when
- * last lock reference goes away.
- *
- * Original comment (of some historical value):
- * This used to have a 'strict' flag, which recovery would use to mark an
- * in-use lock as needing-to-die.  Lest I am ever tempted to put it back, I
- * shall explain why it's gone: with the new hash table scheme, once you call
- * ldlm_lock_destroy, you can never drop your final references on this lock.
- * Because it's not in the hash table anymore.  -phil
- */
-static int ldlm_lock_destroy_internal(struct ldlm_lock *lock)
-{
-	if (lock->l_readers || lock->l_writers) {
-		LDLM_ERROR(lock, "lock still has references");
-		LBUG();
-	}
-
-	if (!list_empty(&lock->l_res_link)) {
-		LDLM_ERROR(lock, "lock still on resource");
-		LBUG();
-	}
-
-	if (ldlm_is_destroyed(lock)) {
-		LASSERT(list_empty(&lock->l_lru));
-		return 0;
-	}
-	ldlm_set_destroyed(lock);
-
-	ldlm_lock_remove_from_lru(lock);
-	class_handle_unhash(&lock->l_handle);
-
-	return 1;
-}
-
-/**
- * Destroys a LDLM lock \a lock. Performs necessary locking first.
- */
-static void ldlm_lock_destroy(struct ldlm_lock *lock)
-{
-	int first;
-
-	lock_res_and_lock(lock);
-	first = ldlm_lock_destroy_internal(lock);
-	unlock_res_and_lock(lock);
-
-	/* drop reference from hashtable only for first destroy */
-	if (first) {
-		lu_ref_del(&lock->l_reference, "hash", lock);
-		LDLM_LOCK_RELEASE(lock);
-	}
-}
-
-/**
- * Destroys a LDLM lock \a lock that is already locked.
- */
-void ldlm_lock_destroy_nolock(struct ldlm_lock *lock)
-{
-	int first;
-
-	first = ldlm_lock_destroy_internal(lock);
-	/* drop reference from hashtable only for first destroy */
-	if (first) {
-		lu_ref_del(&lock->l_reference, "hash", lock);
-		LDLM_LOCK_RELEASE(lock);
-	}
-}
-
-/* this is called by portals_handle2object with the handle lock taken */
-static void lock_handle_addref(void *lock)
-{
-	LDLM_LOCK_GET((struct ldlm_lock *)lock);
-}
-
-static void lock_handle_free(void *lock, int size)
-{
-	LASSERT(size == sizeof(struct ldlm_lock));
-	kmem_cache_free(ldlm_lock_slab, lock);
-}
-
-static struct portals_handle_ops lock_handle_ops = {
-	.hop_addref = lock_handle_addref,
-	.hop_free   = lock_handle_free,
-};
-
-/**
- *
- * Allocate and initialize new lock structure.
- *
- * usage: pass in a resource on which you have done ldlm_resource_get
- *	new lock will take over the refcount.
- * returns: lock with refcount 2 - one for current caller and one for remote
- */
-static struct ldlm_lock *ldlm_lock_new(struct ldlm_resource *resource)
-{
-	struct ldlm_lock *lock;
-
-	LASSERT(resource);
-
-	lock = kmem_cache_zalloc(ldlm_lock_slab, GFP_NOFS);
-	if (!lock)
-		return NULL;
-
-	spin_lock_init(&lock->l_lock);
-	lock->l_resource = resource;
-	lu_ref_add(&resource->lr_reference, "lock", lock);
-
-	atomic_set(&lock->l_refc, 2);
-	INIT_LIST_HEAD(&lock->l_res_link);
-	INIT_LIST_HEAD(&lock->l_lru);
-	INIT_LIST_HEAD(&lock->l_pending_chain);
-	INIT_LIST_HEAD(&lock->l_bl_ast);
-	INIT_LIST_HEAD(&lock->l_cp_ast);
-	INIT_LIST_HEAD(&lock->l_rk_ast);
-	init_waitqueue_head(&lock->l_waitq);
-	lock->l_blocking_lock = NULL;
-	INIT_LIST_HEAD(&lock->l_sl_mode);
-	INIT_LIST_HEAD(&lock->l_sl_policy);
-	RB_CLEAR_NODE(&lock->l_rb);
-
-	lprocfs_counter_incr(ldlm_res_to_ns(resource)->ns_stats,
-			     LDLM_NSS_LOCKS);
-	INIT_LIST_HEAD(&lock->l_handle.h_link);
-	class_handle_hash(&lock->l_handle, &lock_handle_ops);
-
-	lu_ref_init(&lock->l_reference);
-	lu_ref_add(&lock->l_reference, "hash", lock);
-	lock->l_callback_timeout = 0;
-
-#if LUSTRE_TRACKS_LOCK_EXP_REFS
-	INIT_LIST_HEAD(&lock->l_exp_refs_link);
-	lock->l_exp_refs_nr = 0;
-	lock->l_exp_refs_target = NULL;
-#endif
-
-	return lock;
-}
-
-/**
- * Moves LDLM lock \a lock to another resource.
- * This is used on client when server returns some other lock than requested
- * (typically as a result of intent operation)
- */
-int ldlm_lock_change_resource(struct ldlm_namespace *ns, struct ldlm_lock *lock,
-			      const struct ldlm_res_id *new_resid)
-{
-	struct ldlm_resource *oldres = lock->l_resource;
-	struct ldlm_resource *newres;
-	int type;
-
-	lock_res_and_lock(lock);
-	if (memcmp(new_resid, &lock->l_resource->lr_name,
-		   sizeof(lock->l_resource->lr_name)) == 0) {
-		/* Nothing to do */
-		unlock_res_and_lock(lock);
-		return 0;
-	}
-
-	LASSERT(new_resid->name[0] != 0);
-
-	/* This function assumes that the lock isn't on any lists */
-	LASSERT(list_empty(&lock->l_res_link));
-
-	type = oldres->lr_type;
-	unlock_res_and_lock(lock);
-
-	newres = ldlm_resource_get(ns, NULL, new_resid, type, 1);
-	if (IS_ERR(newres))
-		return PTR_ERR(newres);
-
-	lu_ref_add(&newres->lr_reference, "lock", lock);
-	/*
-	 * To flip the lock from the old to the new resource, lock, oldres and
-	 * newres have to be locked. Resource spin-locks are nested within
-	 * lock->l_lock, and are taken in the memory address order to avoid
-	 * dead-locks.
-	 */
-	spin_lock(&lock->l_lock);
-	oldres = lock->l_resource;
-	if (oldres < newres) {
-		lock_res(oldres);
-		lock_res_nested(newres, LRT_NEW);
-	} else {
-		lock_res(newres);
-		lock_res_nested(oldres, LRT_NEW);
-	}
-	LASSERT(memcmp(new_resid, &oldres->lr_name,
-		       sizeof(oldres->lr_name)) != 0);
-	lock->l_resource = newres;
-	unlock_res(oldres);
-	unlock_res_and_lock(lock);
-
-	/* ...and the flowers are still standing! */
-	lu_ref_del(&oldres->lr_reference, "lock", lock);
-	ldlm_resource_putref(oldres);
-
-	return 0;
-}
-
-/** \defgroup ldlm_handles LDLM HANDLES
- * Ways to get hold of locks without any addresses.
- * @{
- */
-
-/**
- * Fills in handle for LDLM lock \a lock into supplied \a lockh
- * Does not take any references.
- */
-void ldlm_lock2handle(const struct ldlm_lock *lock, struct lustre_handle *lockh)
-{
-	lockh->cookie = lock->l_handle.h_cookie;
-}
-EXPORT_SYMBOL(ldlm_lock2handle);
-
-/**
- * Obtain a lock reference by handle.
- *
- * if \a flags: atomically get the lock and set the flags.
- *	      Return NULL if flag already set
- */
-struct ldlm_lock *__ldlm_handle2lock(const struct lustre_handle *handle,
-				     __u64 flags)
-{
-	struct ldlm_lock *lock;
-
-	LASSERT(handle);
-
-	lock = class_handle2object(handle->cookie, NULL);
-	if (!lock)
-		return NULL;
-
-	if (lock->l_export && lock->l_export->exp_failed) {
-		CDEBUG(D_INFO, "lock export failed: lock %p, exp %p\n",
-		       lock, lock->l_export);
-		LDLM_LOCK_PUT(lock);
-		return NULL;
-	}
-
-	/* It's unlikely but possible that someone marked the lock as
-	 * destroyed after we did handle2object on it
-	 */
-	if (flags == 0 && !ldlm_is_destroyed(lock)) {
-		lu_ref_add(&lock->l_reference, "handle", current);
-		return lock;
-	}
-
-	lock_res_and_lock(lock);
-
-	LASSERT(lock->l_resource);
-
-	lu_ref_add_atomic(&lock->l_reference, "handle", current);
-	if (unlikely(ldlm_is_destroyed(lock))) {
-		unlock_res_and_lock(lock);
-		CDEBUG(D_INFO, "lock already destroyed: lock %p\n", lock);
-		LDLM_LOCK_PUT(lock);
-		return NULL;
-	}
-
-	if (flags) {
-		if (lock->l_flags & flags) {
-			unlock_res_and_lock(lock);
-			LDLM_LOCK_PUT(lock);
-			return NULL;
-		}
-
-		lock->l_flags |= flags;
-	}
-
-	unlock_res_and_lock(lock);
-	return lock;
-}
-EXPORT_SYMBOL(__ldlm_handle2lock);
-/** @} ldlm_handles */
-
-/**
- * Fill in "on the wire" representation for given LDLM lock into supplied
- * lock descriptor \a desc structure.
- */
-void ldlm_lock2desc(struct ldlm_lock *lock, struct ldlm_lock_desc *desc)
-{
-	ldlm_res2desc(lock->l_resource, &desc->l_resource);
-	desc->l_req_mode = lock->l_req_mode;
-	desc->l_granted_mode = lock->l_granted_mode;
-	ldlm_convert_policy_to_wire(lock->l_resource->lr_type,
-				    &lock->l_policy_data,
-				    &desc->l_policy_data);
-}
-
-/**
- * Add a lock to list of conflicting locks to send AST to.
- *
- * Only add if we have not sent a blocking AST to the lock yet.
- */
-static void ldlm_add_bl_work_item(struct ldlm_lock *lock, struct ldlm_lock *new,
-				  struct list_head *work_list)
-{
-	if (!ldlm_is_ast_sent(lock)) {
-		LDLM_DEBUG(lock, "lock incompatible; sending blocking AST.");
-		ldlm_set_ast_sent(lock);
-		/* If the enqueuing client said so, tell the AST recipient to
-		 * discard dirty data, rather than writing back.
-		 */
-		if (ldlm_is_ast_discard_data(new))
-			ldlm_set_discard_data(lock);
-		LASSERT(list_empty(&lock->l_bl_ast));
-		list_add(&lock->l_bl_ast, work_list);
-		LDLM_LOCK_GET(lock);
-		LASSERT(!lock->l_blocking_lock);
-		lock->l_blocking_lock = LDLM_LOCK_GET(new);
-	}
-}
-
-/**
- * Add a lock to list of just granted locks to send completion AST to.
- */
-static void ldlm_add_cp_work_item(struct ldlm_lock *lock,
-				  struct list_head *work_list)
-{
-	if (!ldlm_is_cp_reqd(lock)) {
-		ldlm_set_cp_reqd(lock);
-		LDLM_DEBUG(lock, "lock granted; sending completion AST.");
-		LASSERT(list_empty(&lock->l_cp_ast));
-		list_add(&lock->l_cp_ast, work_list);
-		LDLM_LOCK_GET(lock);
-	}
-}
-
-/**
- * Aggregator function to add AST work items into a list. Determines
- * what sort of an AST work needs to be done and calls the proper
- * adding function.
- * Must be called with lr_lock held.
- */
-static void ldlm_add_ast_work_item(struct ldlm_lock *lock,
-				   struct ldlm_lock *new,
-				   struct list_head *work_list)
-{
-	check_res_locked(lock->l_resource);
-	if (new)
-		ldlm_add_bl_work_item(lock, new, work_list);
-	else
-		ldlm_add_cp_work_item(lock, work_list);
-}
-
-/**
- * Add specified reader/writer reference to LDLM lock with handle \a lockh.
- * r/w reference type is determined by \a mode
- * Calls ldlm_lock_addref_internal.
- */
-void ldlm_lock_addref(const struct lustre_handle *lockh, enum ldlm_mode mode)
-{
-	struct ldlm_lock *lock;
-
-	lock = ldlm_handle2lock(lockh);
-	LASSERTF(lock, "Non-existing lock: %llx\n", lockh->cookie);
-	ldlm_lock_addref_internal(lock, mode);
-	LDLM_LOCK_PUT(lock);
-}
-EXPORT_SYMBOL(ldlm_lock_addref);
-
-/**
- * Helper function.
- * Add specified reader/writer reference to LDLM lock \a lock.
- * r/w reference type is determined by \a mode
- * Removes lock from LRU if it is there.
- * Assumes the LDLM lock is already locked.
- */
-void ldlm_lock_addref_internal_nolock(struct ldlm_lock *lock,
-				      enum ldlm_mode mode)
-{
-	ldlm_lock_remove_from_lru(lock);
-	if (mode & (LCK_NL | LCK_CR | LCK_PR)) {
-		lock->l_readers++;
-		lu_ref_add_atomic(&lock->l_reference, "reader", lock);
-	}
-	if (mode & (LCK_EX | LCK_CW | LCK_PW | LCK_GROUP | LCK_COS)) {
-		lock->l_writers++;
-		lu_ref_add_atomic(&lock->l_reference, "writer", lock);
-	}
-	LDLM_LOCK_GET(lock);
-	lu_ref_add_atomic(&lock->l_reference, "user", lock);
-	LDLM_DEBUG(lock, "ldlm_lock_addref(%s)", ldlm_lockname[mode]);
-}
-
-/**
- * Attempts to add reader/writer reference to a lock with handle \a lockh, and
- * fails if lock is already LDLM_FL_CBPENDING or destroyed.
- *
- * \retval 0 success, lock was addref-ed
- *
- * \retval -EAGAIN lock is being canceled.
- */
-int ldlm_lock_addref_try(const struct lustre_handle *lockh, enum ldlm_mode mode)
-{
-	struct ldlm_lock *lock;
-	int	       result;
-
-	result = -EAGAIN;
-	lock = ldlm_handle2lock(lockh);
-	if (lock) {
-		lock_res_and_lock(lock);
-		if (lock->l_readers != 0 || lock->l_writers != 0 ||
-		    !ldlm_is_cbpending(lock)) {
-			ldlm_lock_addref_internal_nolock(lock, mode);
-			result = 0;
-		}
-		unlock_res_and_lock(lock);
-		LDLM_LOCK_PUT(lock);
-	}
-	return result;
-}
-EXPORT_SYMBOL(ldlm_lock_addref_try);
-
-/**
- * Add specified reader/writer reference to LDLM lock \a lock.
- * Locks LDLM lock and calls ldlm_lock_addref_internal_nolock to do the work.
- * Only called for local locks.
- */
-void ldlm_lock_addref_internal(struct ldlm_lock *lock, enum ldlm_mode mode)
-{
-	lock_res_and_lock(lock);
-	ldlm_lock_addref_internal_nolock(lock, mode);
-	unlock_res_and_lock(lock);
-}
-
-/**
- * Removes reader/writer reference for LDLM lock \a lock.
- * Assumes LDLM lock is already locked.
- * only called in ldlm_flock_destroy and for local locks.
- * Does NOT add lock to LRU if no r/w references left to accommodate flock locks
- * that cannot be placed in LRU.
- */
-void ldlm_lock_decref_internal_nolock(struct ldlm_lock *lock,
-				      enum ldlm_mode mode)
-{
-	LDLM_DEBUG(lock, "ldlm_lock_decref(%s)", ldlm_lockname[mode]);
-	if (mode & (LCK_NL | LCK_CR | LCK_PR)) {
-		LASSERT(lock->l_readers > 0);
-		lu_ref_del(&lock->l_reference, "reader", lock);
-		lock->l_readers--;
-	}
-	if (mode & (LCK_EX | LCK_CW | LCK_PW | LCK_GROUP | LCK_COS)) {
-		LASSERT(lock->l_writers > 0);
-		lu_ref_del(&lock->l_reference, "writer", lock);
-		lock->l_writers--;
-	}
-
-	lu_ref_del(&lock->l_reference, "user", lock);
-	LDLM_LOCK_RELEASE(lock);    /* matches the LDLM_LOCK_GET() in addref */
-}
-
-/**
- * Removes reader/writer reference for LDLM lock \a lock.
- * Locks LDLM lock first.
- * If the lock is determined to be client lock on a client and r/w refcount
- * drops to zero and the lock is not blocked, the lock is added to LRU lock
- * on the namespace.
- * For blocked LDLM locks if r/w count drops to zero, blocking_ast is called.
- */
-void ldlm_lock_decref_internal(struct ldlm_lock *lock, enum ldlm_mode mode)
-{
-	struct ldlm_namespace *ns;
-
-	lock_res_and_lock(lock);
-
-	ns = ldlm_lock_to_ns(lock);
-
-	ldlm_lock_decref_internal_nolock(lock, mode);
-
-	if ((ldlm_is_local(lock) || lock->l_req_mode == LCK_GROUP) &&
-	    !lock->l_readers && !lock->l_writers) {
-		/* If this is a local lock on a server namespace and this was
-		 * the last reference, cancel the lock.
-		 *
-		 * Group locks are special:
-		 * They must not go in LRU, but they are not called back
-		 * like non-group locks, instead they are manually released.
-		 * They have an l_writers reference which they keep until
-		 * they are manually released, so we remove them when they have
-		 * no more reader or writer references. - LU-6368
-		 */
-		ldlm_set_cbpending(lock);
-	}
-
-	if (!lock->l_readers && !lock->l_writers && ldlm_is_cbpending(lock)) {
-		/* If we received a blocked AST and this was the last reference,
-		 * run the callback.
-		 */
-		LDLM_DEBUG(lock, "final decref done on cbpending lock");
-
-		LDLM_LOCK_GET(lock); /* dropped by bl thread */
-		ldlm_lock_remove_from_lru(lock);
-		unlock_res_and_lock(lock);
-
-		if (ldlm_is_fail_loc(lock))
-			OBD_RACE(OBD_FAIL_LDLM_CP_BL_RACE);
-
-		if (ldlm_is_atomic_cb(lock) ||
-		    ldlm_bl_to_thread_lock(ns, NULL, lock) != 0)
-			ldlm_handle_bl_callback(ns, NULL, lock);
-	} else if (!lock->l_readers && !lock->l_writers &&
-		   !ldlm_is_no_lru(lock) && !ldlm_is_bl_ast(lock)) {
-		LDLM_DEBUG(lock, "add lock into lru list");
-
-		/* If this is a client-side namespace and this was the last
-		 * reference, put it on the LRU.
-		 */
-		ldlm_lock_add_to_lru(lock);
-		unlock_res_and_lock(lock);
-
-		if (ldlm_is_fail_loc(lock))
-			OBD_RACE(OBD_FAIL_LDLM_CP_BL_RACE);
-
-		/* Call ldlm_cancel_lru() only if EARLY_CANCEL and LRU RESIZE
-		 * are not supported by the server, otherwise, it is done on
-		 * enqueue.
-		 */
-		if (!exp_connect_cancelset(lock->l_conn_export) &&
-		    !ns_connect_lru_resize(ns))
-			ldlm_cancel_lru(ns, 0, LCF_ASYNC, 0);
-	} else {
-		LDLM_DEBUG(lock, "do not add lock into lru list");
-		unlock_res_and_lock(lock);
-	}
-}
-
-/**
- * Decrease reader/writer refcount for LDLM lock with handle \a lockh
- */
-void ldlm_lock_decref(const struct lustre_handle *lockh, enum ldlm_mode mode)
-{
-	struct ldlm_lock *lock = __ldlm_handle2lock(lockh, 0);
-
-	LASSERTF(lock, "Non-existing lock: %#llx\n", lockh->cookie);
-	ldlm_lock_decref_internal(lock, mode);
-	LDLM_LOCK_PUT(lock);
-}
-EXPORT_SYMBOL(ldlm_lock_decref);
-
-/**
- * Decrease reader/writer refcount for LDLM lock with handle
- * \a lockh and mark it for subsequent cancellation once r/w refcount
- * drops to zero instead of putting into LRU.
- */
-void ldlm_lock_decref_and_cancel(const struct lustre_handle *lockh,
-				 enum ldlm_mode mode)
-{
-	struct ldlm_lock *lock = __ldlm_handle2lock(lockh, 0);
-
-	LASSERT(lock);
-
-	LDLM_DEBUG(lock, "ldlm_lock_decref(%s)", ldlm_lockname[mode]);
-	lock_res_and_lock(lock);
-	ldlm_set_cbpending(lock);
-	unlock_res_and_lock(lock);
-	ldlm_lock_decref_internal(lock, mode);
-	LDLM_LOCK_PUT(lock);
-}
-EXPORT_SYMBOL(ldlm_lock_decref_and_cancel);
-
-struct sl_insert_point {
-	struct list_head *res_link;
-	struct list_head *mode_link;
-	struct list_head *policy_link;
-};
-
-/**
- * Finds a position to insert the new lock into granted lock list.
- *
- * Used for locks eligible for skiplist optimization.
- *
- * Parameters:
- *      queue [input]:  the granted list where search acts on;
- *      req [input]:    the lock whose position to be located;
- *      prev [output]:  positions within 3 lists to insert @req to
- * Return Value:
- *      filled @prev
- * NOTE: called by
- *  - ldlm_grant_lock_with_skiplist
- */
-static void search_granted_lock(struct list_head *queue,
-				struct ldlm_lock *req,
-				struct sl_insert_point *prev)
-{
-	struct ldlm_lock *lock, *mode_end, *policy_end;
-
-	list_for_each_entry(lock, queue, l_res_link) {
-
-		mode_end = list_prev_entry(lock, l_sl_mode);
-
-		if (lock->l_req_mode != req->l_req_mode) {
-			/* jump to last lock of mode group */
-			lock = mode_end;
-			continue;
-		}
-
-		/* suitable mode group is found */
-		if (lock->l_resource->lr_type == LDLM_PLAIN) {
-			/* insert point is last lock of the mode group */
-			prev->res_link = &mode_end->l_res_link;
-			prev->mode_link = &mode_end->l_sl_mode;
-			prev->policy_link = &req->l_sl_policy;
-			return;
-		}
-
-		if (lock->l_resource->lr_type == LDLM_IBITS) {
-			for (;;) {
-				policy_end =
-					list_prev_entry(lock, l_sl_policy);
-
-				if (lock->l_policy_data.l_inodebits.bits ==
-				    req->l_policy_data.l_inodebits.bits) {
-					/* insert point is last lock of
-					 * the policy group
-					 */
-					prev->res_link =
-						&policy_end->l_res_link;
-					prev->mode_link =
-						&policy_end->l_sl_mode;
-					prev->policy_link =
-						&policy_end->l_sl_policy;
-					return;
-				}
-
-				if (policy_end == mode_end)
-					/* done with mode group */
-					break;
-
-				/* go to next policy group within mode group */
-				lock = list_next_entry(policy_end, l_res_link);
-			}  /* loop over policy groups within the mode group */
-
-			/* insert point is last lock of the mode group,
-			 * new policy group is started
-			 */
-			prev->res_link = &mode_end->l_res_link;
-			prev->mode_link = &mode_end->l_sl_mode;
-			prev->policy_link = &req->l_sl_policy;
-			return;
-		}
-
-		LDLM_ERROR(lock, "is not LDLM_PLAIN or LDLM_IBITS lock");
-		LBUG();
-	}
-
-	/* insert point is last lock on the queue,
-	 * new mode group and new policy group are started
-	 */
-	prev->res_link = queue->prev;
-	prev->mode_link = &req->l_sl_mode;
-	prev->policy_link = &req->l_sl_policy;
-}
-
-/**
- * Add a lock into resource granted list after a position described by
- * \a prev.
- */
-static void ldlm_granted_list_add_lock(struct ldlm_lock *lock,
-				       struct sl_insert_point *prev)
-{
-	struct ldlm_resource *res = lock->l_resource;
-
-	check_res_locked(res);
-
-	ldlm_resource_dump(D_INFO, res);
-	LDLM_DEBUG(lock, "About to add lock:");
-
-	if (ldlm_is_destroyed(lock)) {
-		CDEBUG(D_OTHER, "Lock destroyed, not adding to resource\n");
-		return;
-	}
-
-	LASSERT(list_empty(&lock->l_res_link));
-	LASSERT(list_empty(&lock->l_sl_mode));
-	LASSERT(list_empty(&lock->l_sl_policy));
-
-	/*
-	 * lock->link == prev->link means lock is first starting the group.
-	 * Don't re-add to itself to suppress kernel warnings.
-	 */
-	if (&lock->l_res_link != prev->res_link)
-		list_add(&lock->l_res_link, prev->res_link);
-	if (&lock->l_sl_mode != prev->mode_link)
-		list_add(&lock->l_sl_mode, prev->mode_link);
-	if (&lock->l_sl_policy != prev->policy_link)
-		list_add(&lock->l_sl_policy, prev->policy_link);
-}
-
-/**
- * Add a lock to granted list on a resource maintaining skiplist
- * correctness.
- */
-static void ldlm_grant_lock_with_skiplist(struct ldlm_lock *lock)
-{
-	struct sl_insert_point prev;
-
-	LASSERT(lock->l_req_mode == lock->l_granted_mode);
-
-	search_granted_lock(&lock->l_resource->lr_granted, lock, &prev);
-	ldlm_granted_list_add_lock(lock, &prev);
-}
-
-/**
- * Perform lock granting bookkeeping.
- *
- * Includes putting the lock into granted list and updating lock mode.
- * NOTE: called by
- *  - ldlm_lock_enqueue
- *  - ldlm_reprocess_queue
- *  - ldlm_lock_convert
- *
- * must be called with lr_lock held
- */
-void ldlm_grant_lock(struct ldlm_lock *lock, struct list_head *work_list)
-{
-	struct ldlm_resource *res = lock->l_resource;
-
-	check_res_locked(res);
-
-	lock->l_granted_mode = lock->l_req_mode;
-
-	if (work_list && lock->l_completion_ast)
-		ldlm_add_ast_work_item(lock, NULL, work_list);
-
-	if (res->lr_type == LDLM_PLAIN || res->lr_type == LDLM_IBITS) {
-		ldlm_grant_lock_with_skiplist(lock);
-	} else if (res->lr_type == LDLM_EXTENT) {
-		ldlm_extent_add_lock(res, lock);
-	} else if (res->lr_type == LDLM_FLOCK) {
-		/*
-		 * We should not add locks to granted list in
-		 * the following cases:
-		 * - this is an UNLOCK but not a real lock;
-		 * - this is a TEST lock;
-		 * - this is a F_CANCELLK lock (async flock has req_mode == 0)
-		 * - this is a deadlock (flock cannot be granted)
-		 */
-		if (!lock->l_req_mode || lock->l_req_mode == LCK_NL ||
-		    ldlm_is_test_lock(lock) || ldlm_is_flock_deadlock(lock))
-			return;
-		ldlm_resource_add_lock(res, &res->lr_granted, lock);
-	} else {
-		LBUG();
-	}
-
-	ldlm_pool_add(&ldlm_res_to_ns(res)->ns_pool, lock);
-}
-
-/**
- * Describe the overlap between two locks.  itree_overlap_cb data.
- */
-struct lock_match_data {
-	struct ldlm_lock	*lmd_old;
-	struct ldlm_lock	*lmd_lock;
-	enum ldlm_mode		*lmd_mode;
-	union ldlm_policy_data	*lmd_policy;
-	__u64			 lmd_flags;
-	int			 lmd_unref;
-};
-
-/**
- * Check if the given @lock meets the criteria for a match.
- * A reference on the lock is taken if matched.
- *
- * \param lock	test-against this lock
- * \param data	parameters
- */
-static bool lock_matches(struct ldlm_lock *lock, void *vdata)
-{
-	struct lock_match_data *data = vdata;
-	union ldlm_policy_data *lpol = &lock->l_policy_data;
-	enum ldlm_mode match;
-
-	if (lock == data->lmd_old)
-		return true;
-
-	/*
-	 * Check if this lock can be matched.
-	 * Used by LU-2919(exclusive open) for open lease lock
-	 */
-	if (ldlm_is_excl(lock))
-		return false;
-
-	/*
-	 * llite sometimes wants to match locks that will be
-	 * canceled when their users drop, but we allow it to match
-	 * if it passes in CBPENDING and the lock still has users.
-	 * this is generally only going to be used by children
-	 * whose parents already hold a lock so forward progress
-	 * can still happen.
-	 */
-	if (ldlm_is_cbpending(lock) &&
-	    !(data->lmd_flags & LDLM_FL_CBPENDING))
-		return false;
-
-	if (!data->lmd_unref && ldlm_is_cbpending(lock) &&
-	    !lock->l_readers && !lock->l_writers)
-		return false;
-
-	if (!(lock->l_req_mode & *data->lmd_mode))
-		return false;
-	match = lock->l_req_mode;
-
-	switch (lock->l_resource->lr_type) {
-	case LDLM_EXTENT:
-		if (lpol->l_extent.start > data->lmd_policy->l_extent.start ||
-		    lpol->l_extent.end < data->lmd_policy->l_extent.end)
-			return false;
-
-		if (unlikely(match == LCK_GROUP) &&
-		    data->lmd_policy->l_extent.gid != LDLM_GID_ANY &&
-		    lpol->l_extent.gid != data->lmd_policy->l_extent.gid)
-			return false;
-		break;
-	case LDLM_IBITS:
-		/*
-		 * We match if we have existing lock with same or wider set
-		 * of bits.
-		 */
-		if ((lpol->l_inodebits.bits &
-		     data->lmd_policy->l_inodebits.bits) !=
-		    data->lmd_policy->l_inodebits.bits)
-			return false;
-		break;
-	default:
-		break;
-	}
-	/*
-	 * We match if we have existing lock with same or wider set
-	 * of bits.
-	 */
-	if (!data->lmd_unref && LDLM_HAVE_MASK(lock, GONE))
-		return false;
-
-	if (!equi(data->lmd_flags & LDLM_FL_LOCAL_ONLY, ldlm_is_local(lock)))
-		return false;
-
-	if (data->lmd_flags & LDLM_FL_TEST_LOCK) {
-		LDLM_LOCK_GET(lock);
-		ldlm_lock_touch_in_lru(lock);
-	} else {
-		ldlm_lock_addref_internal_nolock(lock, match);
-	}
-
-	*data->lmd_mode = match;
-	data->lmd_lock = lock;
-
-	return true;
-}
-
-/**
- * Search for a lock with given parameters in interval trees.
- *
- * \param res	search for a lock in this resource
- * \param data	parameters
- *
- * \retval	a referenced lock or NULL.
- */
-static struct ldlm_lock *search_itree(struct ldlm_resource *res,
-				      struct lock_match_data *data)
-{
-	int idx;
-
-	for (idx = 0; idx < LCK_MODE_NUM; idx++) {
-		struct ldlm_interval_tree *tree = &res->lr_itree[idx];
-
-		if (RB_EMPTY_ROOT(&tree->lit_root.rb_root))
-			continue;
-
-		if (!(tree->lit_mode & *data->lmd_mode))
-			continue;
-
-		ldlm_extent_search(&tree->lit_root,
-				   data->lmd_policy->l_extent.start,
-				   data->lmd_policy->l_extent.end,
-				   lock_matches, data);
-	}
-	return data->lmd_lock;
-}
-
-/**
- * Search for a lock with given properties in a queue.
- *
- * \param queue	search for a lock in this queue
- * \param data	parameters
- *
- * \retval	a referenced lock or NULL.
- */
-static struct ldlm_lock *search_queue(struct list_head *queue,
-				      struct lock_match_data *data)
-{
-	struct ldlm_lock *lock;
-
-	list_for_each_entry(lock, queue, l_res_link)
-		if (lock_matches(lock, data))
-			return data->lmd_lock;
-	return NULL;
-}
-
-void ldlm_lock_fail_match_locked(struct ldlm_lock *lock)
-{
-	if ((lock->l_flags & LDLM_FL_FAIL_NOTIFIED) == 0) {
-		lock->l_flags |= LDLM_FL_FAIL_NOTIFIED;
-		wake_up_all(&lock->l_waitq);
-	}
-}
-
-/**
- * Mark lock as "matchable" by OST.
- *
- * Used to prevent certain races in LOV/OSC where the lock is granted, but LVB
- * is not yet valid.
- * Assumes LDLM lock is already locked.
- */
-void ldlm_lock_allow_match_locked(struct ldlm_lock *lock)
-{
-	ldlm_set_lvb_ready(lock);
-	wake_up_all(&lock->l_waitq);
-}
-EXPORT_SYMBOL(ldlm_lock_allow_match_locked);
-
-/**
- * Mark lock as "matchable" by OST.
- * Locks the lock and then \see ldlm_lock_allow_match_locked
- */
-void ldlm_lock_allow_match(struct ldlm_lock *lock)
-{
-	lock_res_and_lock(lock);
-	ldlm_lock_allow_match_locked(lock);
-	unlock_res_and_lock(lock);
-}
-EXPORT_SYMBOL(ldlm_lock_allow_match);
-
-/**
- * Attempt to find a lock with specified properties.
- *
- * Typically returns a reference to matched lock unless LDLM_FL_TEST_LOCK is
- * set in \a flags
- *
- * Can be called in two ways:
- *
- * If 'ns' is NULL, then lockh describes an existing lock that we want to look
- * for a duplicate of.
- *
- * Otherwise, all of the fields must be filled in, to match against.
- *
- * If 'flags' contains LDLM_FL_LOCAL_ONLY, then only match local locks on the
- *     server (ie, connh is NULL)
- * If 'flags' contains LDLM_FL_BLOCK_GRANTED, then only locks on the granted
- *     list will be considered
- * If 'flags' contains LDLM_FL_CBPENDING, then locks that have been marked
- *     to be canceled can still be matched as long as they still have reader
- *     or writer referneces
- * If 'flags' contains LDLM_FL_TEST_LOCK, then don't actually reference a lock,
- *     just tell us if we would have matched.
- *
- * \retval 1 if it finds an already-existing lock that is compatible; in this
- * case, lockh is filled in with a addref()ed lock
- *
- * We also check security context, and if that fails we simply return 0 (to
- * keep caller code unchanged), the context failure will be discovered by
- * caller sometime later.
- */
-enum ldlm_mode ldlm_lock_match(struct ldlm_namespace *ns, __u64 flags,
-			       const struct ldlm_res_id *res_id,
-			       enum ldlm_type type,
-			       union ldlm_policy_data *policy,
-			       enum ldlm_mode mode,
-			       struct lustre_handle *lockh, int unref)
-{
-	struct lock_match_data data = {
-		.lmd_old	= NULL,
-		.lmd_lock	= NULL,
-		.lmd_mode	= &mode,
-		.lmd_policy	= policy,
-		.lmd_flags	= flags,
-		.lmd_unref	= unref,
-	};
-	struct ldlm_resource *res;
-	struct ldlm_lock *lock;
-	int rc = 0;
-
-	if (!ns) {
-		data.lmd_old = ldlm_handle2lock(lockh);
-		LASSERT(data.lmd_old);
-
-		ns = ldlm_lock_to_ns(data.lmd_old);
-		res_id = &data.lmd_old->l_resource->lr_name;
-		type = data.lmd_old->l_resource->lr_type;
-		*data.lmd_mode = data.lmd_old->l_req_mode;
-	}
-
-	res = ldlm_resource_get(ns, NULL, res_id, type, 0);
-	if (IS_ERR(res)) {
-		LASSERT(!data.lmd_old);
-		return 0;
-	}
-
-	LDLM_RESOURCE_ADDREF(res);
-	lock_res(res);
-
-	if (res->lr_type == LDLM_EXTENT)
-		lock = search_itree(res, &data);
-	else
-		lock = search_queue(&res->lr_granted, &data);
-	if (lock) {
-		rc = 1;
-		goto out;
-	}
-	if (flags & LDLM_FL_BLOCK_GRANTED) {
-		rc = 0;
-		goto out;
-	}
-	lock = search_queue(&res->lr_waiting, &data);
-	if (lock) {
-		rc = 1;
-		goto out;
-	}
-out:
-	unlock_res(res);
-	LDLM_RESOURCE_DELREF(res);
-	ldlm_resource_putref(res);
-
-	if (lock) {
-		ldlm_lock2handle(lock, lockh);
-		if ((flags & LDLM_FL_LVB_READY) && !ldlm_is_lvb_ready(lock)) {
-			__u64 wait_flags = LDLM_FL_LVB_READY |
-				LDLM_FL_DESTROYED | LDLM_FL_FAIL_NOTIFIED;
-
-			if (lock->l_completion_ast) {
-				int err = lock->l_completion_ast(lock,
-							  LDLM_FL_WAIT_NOREPROC,
-								 NULL);
-				if (err) {
-					if (flags & LDLM_FL_TEST_LOCK)
-						LDLM_LOCK_RELEASE(lock);
-					else
-						ldlm_lock_decref_internal(lock,
-									  mode);
-					rc = 0;
-					goto out2;
-				}
-			}
-
-			/* XXX FIXME see comment on CAN_MATCH in lustre_dlm.h */
-			wait_event_idle_timeout(lock->l_waitq,
-						lock->l_flags & wait_flags,
-						obd_timeout * HZ);
-			if (!ldlm_is_lvb_ready(lock)) {
-				if (flags & LDLM_FL_TEST_LOCK)
-					LDLM_LOCK_RELEASE(lock);
-				else
-					ldlm_lock_decref_internal(lock, mode);
-				rc = 0;
-			}
-		}
-	}
- out2:
-	if (rc) {
-		LDLM_DEBUG(lock, "matched (%llu %llu)",
-			   (type == LDLM_PLAIN || type == LDLM_IBITS) ?
-				res_id->name[2] : policy->l_extent.start,
-			   (type == LDLM_PLAIN || type == LDLM_IBITS) ?
-				res_id->name[3] : policy->l_extent.end);
-
-		/* check user's security context */
-		if (lock->l_conn_export &&
-		    sptlrpc_import_check_ctx(
-				class_exp2cliimp(lock->l_conn_export))) {
-			if (!(flags & LDLM_FL_TEST_LOCK))
-				ldlm_lock_decref_internal(lock, mode);
-			rc = 0;
-		}
-
-		if (flags & LDLM_FL_TEST_LOCK)
-			LDLM_LOCK_RELEASE(lock);
-
-	} else if (!(flags & LDLM_FL_TEST_LOCK)) {/*less verbose for test-only*/
-		LDLM_DEBUG_NOLOCK("not matched ns %p type %u mode %u res %llu/%llu (%llu %llu)",
-				  ns, type, mode, res_id->name[0],
-				  res_id->name[1],
-				  (type == LDLM_PLAIN || type == LDLM_IBITS) ?
-					res_id->name[2] : policy->l_extent.start,
-				  (type == LDLM_PLAIN || type == LDLM_IBITS) ?
-					res_id->name[3] : policy->l_extent.end);
-	}
-	if (data.lmd_old)
-		LDLM_LOCK_PUT(data.lmd_old);
-
-	return rc ? mode : 0;
-}
-EXPORT_SYMBOL(ldlm_lock_match);
-
-enum ldlm_mode ldlm_revalidate_lock_handle(const struct lustre_handle *lockh,
-					   __u64 *bits)
-{
-	struct ldlm_lock *lock;
-	enum ldlm_mode mode = 0;
-
-	lock = ldlm_handle2lock(lockh);
-	if (lock) {
-		lock_res_and_lock(lock);
-		if (LDLM_HAVE_MASK(lock, GONE))
-			goto out;
-
-		if (ldlm_is_cbpending(lock) &&
-		    lock->l_readers == 0 && lock->l_writers == 0)
-			goto out;
-
-		if (bits)
-			*bits = lock->l_policy_data.l_inodebits.bits;
-		mode = lock->l_granted_mode;
-		ldlm_lock_addref_internal_nolock(lock, mode);
-	}
-
-out:
-	if (lock) {
-		unlock_res_and_lock(lock);
-		LDLM_LOCK_PUT(lock);
-	}
-	return mode;
-}
-EXPORT_SYMBOL(ldlm_revalidate_lock_handle);
-
-/** The caller must guarantee that the buffer is large enough. */
-int ldlm_fill_lvb(struct ldlm_lock *lock, struct req_capsule *pill,
-		  enum req_location loc, void *data, int size)
-{
-	void *lvb;
-
-	LASSERT(data);
-	LASSERT(size >= 0);
-
-	switch (lock->l_lvb_type) {
-	case LVB_T_OST:
-		if (size == sizeof(struct ost_lvb)) {
-			if (loc == RCL_CLIENT)
-				lvb = req_capsule_client_swab_get(pill,
-								  &RMF_DLM_LVB,
-							lustre_swab_ost_lvb);
-			else
-				lvb = req_capsule_server_swab_get(pill,
-								  &RMF_DLM_LVB,
-							lustre_swab_ost_lvb);
-			if (unlikely(!lvb)) {
-				LDLM_ERROR(lock, "no LVB");
-				return -EPROTO;
-			}
-
-			memcpy(data, lvb, size);
-		} else if (size == sizeof(struct ost_lvb_v1)) {
-			struct ost_lvb *olvb = data;
-
-			if (loc == RCL_CLIENT)
-				lvb = req_capsule_client_swab_get(pill,
-								  &RMF_DLM_LVB,
-							lustre_swab_ost_lvb_v1);
-			else
-				lvb = req_capsule_server_sized_swab_get(pill,
-						&RMF_DLM_LVB, size,
-						lustre_swab_ost_lvb_v1);
-			if (unlikely(!lvb)) {
-				LDLM_ERROR(lock, "no LVB");
-				return -EPROTO;
-			}
-
-			memcpy(data, lvb, size);
-			olvb->lvb_mtime_ns = 0;
-			olvb->lvb_atime_ns = 0;
-			olvb->lvb_ctime_ns = 0;
-		} else {
-			LDLM_ERROR(lock, "Replied unexpected ost LVB size %d",
-				   size);
-			return -EINVAL;
-		}
-		break;
-	case LVB_T_LQUOTA:
-		if (size == sizeof(struct lquota_lvb)) {
-			if (loc == RCL_CLIENT)
-				lvb = req_capsule_client_swab_get(pill,
-								  &RMF_DLM_LVB,
-							lustre_swab_lquota_lvb);
-			else
-				lvb = req_capsule_server_swab_get(pill,
-								  &RMF_DLM_LVB,
-							lustre_swab_lquota_lvb);
-			if (unlikely(!lvb)) {
-				LDLM_ERROR(lock, "no LVB");
-				return -EPROTO;
-			}
-
-			memcpy(data, lvb, size);
-		} else {
-			LDLM_ERROR(lock,
-				   "Replied unexpected lquota LVB size %d",
-				   size);
-			return -EINVAL;
-		}
-		break;
-	case LVB_T_LAYOUT:
-		if (size == 0)
-			break;
-
-		if (loc == RCL_CLIENT)
-			lvb = req_capsule_client_get(pill, &RMF_DLM_LVB);
-		else
-			lvb = req_capsule_server_get(pill, &RMF_DLM_LVB);
-		if (unlikely(!lvb)) {
-			LDLM_ERROR(lock, "no LVB");
-			return -EPROTO;
-		}
-
-		memcpy(data, lvb, size);
-		break;
-	default:
-		LDLM_ERROR(lock, "Unknown LVB type: %d", lock->l_lvb_type);
-		dump_stack();
-		return -EINVAL;
-	}
-
-	return 0;
-}
-
-/**
- * Create and fill in new LDLM lock with specified properties.
- * Returns a referenced lock
- */
-struct ldlm_lock *ldlm_lock_create(struct ldlm_namespace *ns,
-				   const struct ldlm_res_id *res_id,
-				   enum ldlm_type type,
-				   enum ldlm_mode mode,
-				   const struct ldlm_callback_suite *cbs,
-				   void *data, __u32 lvb_len,
-				   enum lvb_type lvb_type)
-{
-	struct ldlm_lock *lock;
-	struct ldlm_resource *res;
-	int rc;
-
-	res = ldlm_resource_get(ns, NULL, res_id, type, 1);
-	if (IS_ERR(res))
-		return ERR_CAST(res);
-
-	lock = ldlm_lock_new(res);
-	if (!lock) {
-		ldlm_resource_putref(res);
-		return ERR_PTR(-ENOMEM);
-	}
-
-	lock->l_req_mode = mode;
-	lock->l_ast_data = data;
-	lock->l_pid = current->pid;
-	if (cbs) {
-		lock->l_blocking_ast = cbs->lcs_blocking;
-		lock->l_completion_ast = cbs->lcs_completion;
-		lock->l_glimpse_ast = cbs->lcs_glimpse;
-	}
-
-	if (lvb_len) {
-		lock->l_lvb_len = lvb_len;
-		lock->l_lvb_data = kzalloc(lvb_len, GFP_NOFS);
-		if (!lock->l_lvb_data) {
-			rc = -ENOMEM;
-			goto out;
-		}
-	}
-
-	lock->l_lvb_type = lvb_type;
-	if (OBD_FAIL_CHECK(OBD_FAIL_LDLM_NEW_LOCK)) {
-		rc = -ENOENT;
-		goto out;
-	}
-
-	return lock;
-
-out:
-	ldlm_lock_destroy(lock);
-	LDLM_LOCK_RELEASE(lock);
-	return ERR_PTR(rc);
-}
-
-
-
-/**
- * Enqueue (request) a lock.
- * On the client this is called from ldlm_cli_enqueue_fini
- * after we already got an initial reply from the server with some status.
- *
- * Does not block. As a result of enqueue the lock would be put
- * into granted or waiting list.
- */
-enum ldlm_error ldlm_lock_enqueue(struct ldlm_namespace *ns,
-				  struct ldlm_lock **lockp,
-				  void *cookie, __u64 *flags)
-{
-	struct ldlm_lock *lock = *lockp;
-	struct ldlm_resource *res = lock->l_resource;
-
-	lock_res_and_lock(lock);
-	if (lock->l_req_mode == lock->l_granted_mode) {
-		/* The server returned a blocked lock, but it was granted
-		 * before we got a chance to actually enqueue it.  We don't
-		 * need to do anything else.
-		 */
-		*flags &= ~LDLM_FL_BLOCKED_MASK;
-		goto out;
-	}
-
-	ldlm_resource_unlink_lock(lock);
-
-	/* Some flags from the enqueue want to make it into the AST, via the
-	 * lock's l_flags.
-	 */
-	if (*flags & LDLM_FL_AST_DISCARD_DATA)
-		ldlm_set_ast_discard_data(lock);
-	if (*flags & LDLM_FL_TEST_LOCK)
-		ldlm_set_test_lock(lock);
-
-	/*
-	 * This distinction between local lock trees is very important; a client
-	 * namespace only has information about locks taken by that client, and
-	 * thus doesn't have enough information to decide for itself if it can
-	 * be granted (below).  In this case, we do exactly what the server
-	 * tells us to do, as dictated by the 'flags'.
-	 */
-	if (*flags & (LDLM_FL_BLOCK_WAIT | LDLM_FL_BLOCK_GRANTED))
-		ldlm_resource_add_lock(res, &res->lr_waiting, lock);
-	else
-		ldlm_grant_lock(lock, NULL);
-
-out:
-	unlock_res_and_lock(lock);
-	return ELDLM_OK;
-}
-
-/**
- * Process a call to blocking AST callback for a lock in ast_work list
- */
-static int
-ldlm_work_bl_ast_lock(struct ptlrpc_request_set *rqset, void *opaq)
-{
-	struct ldlm_cb_set_arg *arg = opaq;
-	struct ldlm_lock_desc   d;
-	int		     rc;
-	struct ldlm_lock       *lock;
-
-	if (list_empty(arg->list))
-		return -ENOENT;
-
-	lock = list_first_entry(arg->list, struct ldlm_lock, l_bl_ast);
-
-	/* nobody should touch l_bl_ast */
-	lock_res_and_lock(lock);
-	list_del_init(&lock->l_bl_ast);
-
-	LASSERT(ldlm_is_ast_sent(lock));
-	LASSERT(lock->l_bl_ast_run == 0);
-	LASSERT(lock->l_blocking_lock);
-	lock->l_bl_ast_run++;
-	unlock_res_and_lock(lock);
-
-	ldlm_lock2desc(lock->l_blocking_lock, &d);
-
-	rc = lock->l_blocking_ast(lock, &d, (void *)arg, LDLM_CB_BLOCKING);
-	LDLM_LOCK_RELEASE(lock->l_blocking_lock);
-	lock->l_blocking_lock = NULL;
-	LDLM_LOCK_RELEASE(lock);
-
-	return rc;
-}
-
-/**
- * Process a call to completion AST callback for a lock in ast_work list
- */
-static int
-ldlm_work_cp_ast_lock(struct ptlrpc_request_set *rqset, void *opaq)
-{
-	struct ldlm_cb_set_arg  *arg = opaq;
-	int		      rc = 0;
-	struct ldlm_lock	*lock;
-	ldlm_completion_callback completion_callback;
-
-	if (list_empty(arg->list))
-		return -ENOENT;
-
-	lock = list_first_entry(arg->list, struct ldlm_lock, l_cp_ast);
-
-	/* It's possible to receive a completion AST before we've set
-	 * the l_completion_ast pointer: either because the AST arrived
-	 * before the reply, or simply because there's a small race
-	 * window between receiving the reply and finishing the local
-	 * enqueue. (bug 842)
-	 *
-	 * This can't happen with the blocking_ast, however, because we
-	 * will never call the local blocking_ast until we drop our
-	 * reader/writer reference, which we won't do until we get the
-	 * reply and finish enqueueing.
-	 */
-
-	/* nobody should touch l_cp_ast */
-	lock_res_and_lock(lock);
-	list_del_init(&lock->l_cp_ast);
-	LASSERT(ldlm_is_cp_reqd(lock));
-	/* save l_completion_ast since it can be changed by
-	 * mds_intent_policy(), see bug 14225
-	 */
-	completion_callback = lock->l_completion_ast;
-	ldlm_clear_cp_reqd(lock);
-	unlock_res_and_lock(lock);
-
-	if (completion_callback)
-		rc = completion_callback(lock, 0, (void *)arg);
-	LDLM_LOCK_RELEASE(lock);
-
-	return rc;
-}
-
-/**
- * Process a call to revocation AST callback for a lock in ast_work list
- */
-static int
-ldlm_work_revoke_ast_lock(struct ptlrpc_request_set *rqset, void *opaq)
-{
-	struct ldlm_cb_set_arg *arg = opaq;
-	struct ldlm_lock_desc   desc;
-	int		     rc;
-	struct ldlm_lock       *lock;
-
-	if (list_empty(arg->list))
-		return -ENOENT;
-
-	lock = list_first_entry(arg->list, struct ldlm_lock, l_rk_ast);
-	list_del_init(&lock->l_rk_ast);
-
-	/* the desc just pretend to exclusive */
-	ldlm_lock2desc(lock, &desc);
-	desc.l_req_mode = LCK_EX;
-	desc.l_granted_mode = 0;
-
-	rc = lock->l_blocking_ast(lock, &desc, (void *)arg, LDLM_CB_BLOCKING);
-	LDLM_LOCK_RELEASE(lock);
-
-	return rc;
-}
-
-/**
- * Process a call to glimpse AST callback for a lock in ast_work list
- */
-static int ldlm_work_gl_ast_lock(struct ptlrpc_request_set *rqset, void *opaq)
-{
-	struct ldlm_cb_set_arg		*arg = opaq;
-	struct ldlm_glimpse_work	*gl_work;
-	struct ldlm_lock		*lock;
-	int				 rc = 0;
-
-	if (list_empty(arg->list))
-		return -ENOENT;
-
-	gl_work = list_first_entry(arg->list, struct ldlm_glimpse_work,
-			     gl_list);
-	list_del_init(&gl_work->gl_list);
-
-	lock = gl_work->gl_lock;
-
-	/* transfer the glimpse descriptor to ldlm_cb_set_arg */
-	arg->gl_desc = gl_work->gl_desc;
-
-	/* invoke the actual glimpse callback */
-	if (lock->l_glimpse_ast(lock, (void *)arg) == 0)
-		rc = 1;
-
-	LDLM_LOCK_RELEASE(lock);
-
-	if ((gl_work->gl_flags & LDLM_GL_WORK_NOFREE) == 0)
-		kfree(gl_work);
-
-	return rc;
-}
-
-/**
- * Process list of locks in need of ASTs being sent.
- *
- * Used on server to send multiple ASTs together instead of sending one by
- * one.
- */
-int ldlm_run_ast_work(struct ldlm_namespace *ns, struct list_head *rpc_list,
-		      enum ldlm_desc_ast_t ast_type)
-{
-	struct ldlm_cb_set_arg *arg;
-	set_producer_func       work_ast_lock;
-	int		     rc;
-
-	if (list_empty(rpc_list))
-		return 0;
-
-	arg = kzalloc(sizeof(*arg), GFP_NOFS);
-	if (!arg)
-		return -ENOMEM;
-
-	atomic_set(&arg->restart, 0);
-	arg->list = rpc_list;
-
-	switch (ast_type) {
-	case LDLM_WORK_BL_AST:
-		arg->type = LDLM_BL_CALLBACK;
-		work_ast_lock = ldlm_work_bl_ast_lock;
-		break;
-	case LDLM_WORK_CP_AST:
-		arg->type = LDLM_CP_CALLBACK;
-		work_ast_lock = ldlm_work_cp_ast_lock;
-		break;
-	case LDLM_WORK_REVOKE_AST:
-		arg->type = LDLM_BL_CALLBACK;
-		work_ast_lock = ldlm_work_revoke_ast_lock;
-		break;
-	case LDLM_WORK_GL_AST:
-		arg->type = LDLM_GL_CALLBACK;
-		work_ast_lock = ldlm_work_gl_ast_lock;
-		break;
-	default:
-		LBUG();
-	}
-
-	/* We create a ptlrpc request set with flow control extension.
-	 * This request set will use the work_ast_lock function to produce new
-	 * requests and will send a new request each time one completes in order
-	 * to keep the number of requests in flight to ns_max_parallel_ast
-	 */
-	arg->set = ptlrpc_prep_fcset(ns->ns_max_parallel_ast ? : UINT_MAX,
-				     work_ast_lock, arg);
-	if (!arg->set) {
-		rc = -ENOMEM;
-		goto out;
-	}
-
-	ptlrpc_set_wait(arg->set);
-	ptlrpc_set_destroy(arg->set);
-
-	rc = atomic_read(&arg->restart) ? -ERESTART : 0;
-	goto out;
-out:
-	kfree(arg);
-	return rc;
-}
-
-static bool is_bl_done(struct ldlm_lock *lock)
-{
-	bool bl_done = true;
-
-	if (!ldlm_is_bl_done(lock)) {
-		lock_res_and_lock(lock);
-		bl_done = ldlm_is_bl_done(lock);
-		unlock_res_and_lock(lock);
-	}
-
-	return bl_done;
-}
-
-/**
- * Helper function to call blocking AST for LDLM lock \a lock in a
- * "cancelling" mode.
- */
-void ldlm_cancel_callback(struct ldlm_lock *lock)
-{
-	check_res_locked(lock->l_resource);
-	if (!ldlm_is_cancel(lock)) {
-		ldlm_set_cancel(lock);
-		if (lock->l_blocking_ast) {
-			unlock_res_and_lock(lock);
-			lock->l_blocking_ast(lock, NULL, lock->l_ast_data,
-					     LDLM_CB_CANCELING);
-			lock_res_and_lock(lock);
-		} else {
-			LDLM_DEBUG(lock, "no blocking ast");
-		}
-		/* only canceller can set bl_done bit */
-		ldlm_set_bl_done(lock);
-		wake_up_all(&lock->l_waitq);
-	} else if (!ldlm_is_bl_done(lock)) {
-		/*
-		 * The lock is guaranteed to have been canceled once
-		 * returning from this function.
-		 */
-		unlock_res_and_lock(lock);
-		wait_event_idle(lock->l_waitq, is_bl_done(lock));
-		lock_res_and_lock(lock);
-	}
-}
-
-/**
- * Remove skiplist-enabled LDLM lock \a req from granted list
- */
-void ldlm_unlink_lock_skiplist(struct ldlm_lock *req)
-{
-	if (req->l_resource->lr_type != LDLM_PLAIN &&
-	    req->l_resource->lr_type != LDLM_IBITS)
-		return;
-
-	list_del_init(&req->l_sl_policy);
-	list_del_init(&req->l_sl_mode);
-}
-
-/**
- * Attempts to cancel LDLM lock \a lock that has no reader/writer references.
- */
-void ldlm_lock_cancel(struct ldlm_lock *lock)
-{
-	struct ldlm_resource *res;
-	struct ldlm_namespace *ns;
-
-	lock_res_and_lock(lock);
-
-	res = lock->l_resource;
-	ns  = ldlm_res_to_ns(res);
-
-	/* Please do not, no matter how tempting, remove this LBUG without
-	 * talking to me first. -phik
-	 */
-	if (lock->l_readers || lock->l_writers) {
-		LDLM_ERROR(lock, "lock still has references");
-		LBUG();
-	}
-
-	/* Releases cancel callback. */
-	ldlm_cancel_callback(lock);
-
-	ldlm_resource_unlink_lock(lock);
-	ldlm_lock_destroy_nolock(lock);
-
-	if (lock->l_granted_mode == lock->l_req_mode)
-		ldlm_pool_del(&ns->ns_pool, lock);
-
-	/* Make sure we will not be called again for same lock what is possible
-	 * if not to zero out lock->l_granted_mode
-	 */
-	lock->l_granted_mode = LCK_MINMODE;
-	unlock_res_and_lock(lock);
-}
-EXPORT_SYMBOL(ldlm_lock_cancel);
-
-/**
- * Set opaque data into the lock that only makes sense to upper layer.
- */
-int ldlm_lock_set_data(const struct lustre_handle *lockh, void *data)
-{
-	struct ldlm_lock *lock = ldlm_handle2lock(lockh);
-	int rc = -EINVAL;
-
-	if (lock) {
-		if (!lock->l_ast_data)
-			lock->l_ast_data = data;
-		if (lock->l_ast_data == data)
-			rc = 0;
-		LDLM_LOCK_PUT(lock);
-	}
-	return rc;
-}
-EXPORT_SYMBOL(ldlm_lock_set_data);
-
-struct export_cl_data {
-	struct obd_export	*ecl_exp;
-	int			ecl_loop;
-};
-
-/**
- * Print lock with lock handle \a lockh description into debug log.
- *
- * Used when printing all locks on a resource for debug purposes.
- */
-void ldlm_lock_dump_handle(int level, const struct lustre_handle *lockh)
-{
-	struct ldlm_lock *lock;
-
-	if (!((libcfs_debug | D_ERROR) & level))
-		return;
-
-	lock = ldlm_handle2lock(lockh);
-	if (!lock)
-		return;
-
-	LDLM_DEBUG_LIMIT(level, lock, "###");
-
-	LDLM_LOCK_PUT(lock);
-}
-EXPORT_SYMBOL(ldlm_lock_dump_handle);
-
-/**
- * Print lock information with custom message into debug log.
- * Helper function.
- */
-void _ldlm_lock_debug(struct ldlm_lock *lock,
-		      struct libcfs_debug_msg_data *msgdata,
-		      const char *fmt, ...)
-{
-	va_list args;
-	struct obd_export *exp = lock->l_export;
-	struct ldlm_resource *resource = lock->l_resource;
-	char *nid = "local";
-
-	va_start(args, fmt);
-
-	if (exp && exp->exp_connection) {
-		nid = libcfs_nid2str(exp->exp_connection->c_peer.nid);
-	} else if (exp && exp->exp_obd) {
-		struct obd_import *imp = exp->exp_obd->u.cli.cl_import;
-
-		nid = libcfs_nid2str(imp->imp_connection->c_peer.nid);
-	}
-
-	if (!resource) {
-		libcfs_debug_vmsg2(msgdata, fmt, args,
-				   " ns: \?\? lock: %p/%#llx lrc: %d/%d,%d mode: %s/%s res: \?\? rrc=\?\? type: \?\?\? flags: %#llx nid: %s remote: %#llx expref: %d pid: %u timeout: %lu lvb_type: %d\n",
-				   lock,
-				   lock->l_handle.h_cookie,
-				   atomic_read(&lock->l_refc),
-				   lock->l_readers, lock->l_writers,
-				   ldlm_lockname[lock->l_granted_mode],
-				   ldlm_lockname[lock->l_req_mode],
-				   lock->l_flags, nid,
-				   lock->l_remote_handle.cookie,
-				   exp ? atomic_read(&exp->exp_refcount) : -99,
-				   lock->l_pid, lock->l_callback_timeout,
-				   lock->l_lvb_type);
-		va_end(args);
-		return;
-	}
-
-	switch (resource->lr_type) {
-	case LDLM_EXTENT:
-		libcfs_debug_vmsg2(msgdata, fmt, args,
-				   " ns: %s lock: %p/%#llx lrc: %d/%d,%d mode: %s/%s res: " DLDLMRES " rrc: %d type: %s [%llu->%llu] (req %llu->%llu) flags: %#llx nid: %s remote: %#llx expref: %d pid: %u timeout: %lu lvb_type: %d\n",
-				   ldlm_lock_to_ns_name(lock), lock,
-				   lock->l_handle.h_cookie,
-				   atomic_read(&lock->l_refc),
-				   lock->l_readers, lock->l_writers,
-				   ldlm_lockname[lock->l_granted_mode],
-				   ldlm_lockname[lock->l_req_mode],
-				   PLDLMRES(resource),
-				   atomic_read(&resource->lr_refcount),
-				   ldlm_typename[resource->lr_type],
-				   lock->l_policy_data.l_extent.start,
-				   lock->l_policy_data.l_extent.end,
-				   lock->l_req_extent.start,
-				   lock->l_req_extent.end,
-				   lock->l_flags, nid,
-				   lock->l_remote_handle.cookie,
-				   exp ? atomic_read(&exp->exp_refcount) : -99,
-				   lock->l_pid, lock->l_callback_timeout,
-				   lock->l_lvb_type);
-		break;
-
-	case LDLM_FLOCK:
-		libcfs_debug_vmsg2(msgdata, fmt, args,
-				   " ns: %s lock: %p/%#llx lrc: %d/%d,%d mode: %s/%s res: " DLDLMRES " rrc: %d type: %s pid: %d [%llu->%llu] flags: %#llx nid: %s remote: %#llx expref: %d pid: %u timeout: %lu\n",
-				   ldlm_lock_to_ns_name(lock), lock,
-				   lock->l_handle.h_cookie,
-				   atomic_read(&lock->l_refc),
-				   lock->l_readers, lock->l_writers,
-				   ldlm_lockname[lock->l_granted_mode],
-				   ldlm_lockname[lock->l_req_mode],
-				   PLDLMRES(resource),
-				   atomic_read(&resource->lr_refcount),
-				   ldlm_typename[resource->lr_type],
-				   lock->l_policy_data.l_flock.pid,
-				   lock->l_policy_data.l_flock.start,
-				   lock->l_policy_data.l_flock.end,
-				   lock->l_flags, nid,
-				   lock->l_remote_handle.cookie,
-				   exp ? atomic_read(&exp->exp_refcount) : -99,
-				   lock->l_pid, lock->l_callback_timeout);
-		break;
-
-	case LDLM_IBITS:
-		libcfs_debug_vmsg2(msgdata, fmt, args,
-				   " ns: %s lock: %p/%#llx lrc: %d/%d,%d mode: %s/%s res: " DLDLMRES " bits %#llx rrc: %d type: %s flags: %#llx nid: %s remote: %#llx expref: %d pid: %u timeout: %lu lvb_type: %d\n",
-				   ldlm_lock_to_ns_name(lock),
-				   lock, lock->l_handle.h_cookie,
-				   atomic_read(&lock->l_refc),
-				   lock->l_readers, lock->l_writers,
-				   ldlm_lockname[lock->l_granted_mode],
-				   ldlm_lockname[lock->l_req_mode],
-				   PLDLMRES(resource),
-				   lock->l_policy_data.l_inodebits.bits,
-				   atomic_read(&resource->lr_refcount),
-				   ldlm_typename[resource->lr_type],
-				   lock->l_flags, nid,
-				   lock->l_remote_handle.cookie,
-				   exp ? atomic_read(&exp->exp_refcount) : -99,
-				   lock->l_pid, lock->l_callback_timeout,
-				   lock->l_lvb_type);
-		break;
-
-	default:
-		libcfs_debug_vmsg2(msgdata, fmt, args,
-				   " ns: %s lock: %p/%#llx lrc: %d/%d,%d mode: %s/%s res: " DLDLMRES " rrc: %d type: %s flags: %#llx nid: %s remote: %#llx expref: %d pid: %u timeout: %lu lvb_type: %d\n",
-				   ldlm_lock_to_ns_name(lock),
-				   lock, lock->l_handle.h_cookie,
-				   atomic_read(&lock->l_refc),
-				   lock->l_readers, lock->l_writers,
-				   ldlm_lockname[lock->l_granted_mode],
-				   ldlm_lockname[lock->l_req_mode],
-				   PLDLMRES(resource),
-				   atomic_read(&resource->lr_refcount),
-				   ldlm_typename[resource->lr_type],
-				   lock->l_flags, nid,
-				   lock->l_remote_handle.cookie,
-				   exp ? atomic_read(&exp->exp_refcount) : -99,
-				   lock->l_pid, lock->l_callback_timeout,
-				   lock->l_lvb_type);
-		break;
-	}
-	va_end(args);
-}
-EXPORT_SYMBOL(_ldlm_lock_debug);
diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c b/drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c
deleted file mode 100644
index f410ef6c02ef..000000000000
--- a/drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c
+++ /dev/null
@@ -1,1154 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-/*
- * GPL HEADER START
- *
- * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 only,
- * as published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful, but
- * WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
- * General Public License version 2 for more details (a copy is included
- * in the LICENSE file that accompanied this code).
- *
- * You should have received a copy of the GNU General Public License
- * version 2 along with this program; If not, see
- * http://www.gnu.org/licenses/gpl-2.0.html
- *
- * GPL HEADER END
- */
-/*
- * Copyright (c) 2002, 2010, Oracle and/or its affiliates. All rights reserved.
- * Use is subject to license terms.
- *
- * Copyright (c) 2010, 2015, Intel Corporation.
- */
-/*
- * This file is part of Lustre, http://www.lustre.org/
- * Lustre is a trademark of Sun Microsystems, Inc.
- *
- * lustre/ldlm/ldlm_lockd.c
- *
- * Author: Peter Braam <braam at clusterfs.com>
- * Author: Phil Schwan <phil at clusterfs.com>
- */
-
-#define DEBUG_SUBSYSTEM S_LDLM
-
-#include <linux/kthread.h>
-#include <linux/sched/mm.h>
-#include <lustre_dlm.h>
-#include <obd_class.h>
-#include <linux/list.h>
-#include "ldlm_internal.h"
-
-static int ldlm_num_threads;
-module_param(ldlm_num_threads, int, 0444);
-MODULE_PARM_DESC(ldlm_num_threads, "number of DLM service threads to start");
-
-static char *ldlm_cpts;
-module_param(ldlm_cpts, charp, 0444);
-MODULE_PARM_DESC(ldlm_cpts, "CPU partitions ldlm threads should run on");
-
-static struct mutex	ldlm_ref_mutex;
-static int ldlm_refcount;
-
-static struct kobject *ldlm_kobj;
-struct kset *ldlm_ns_kset;
-static struct kset *ldlm_svc_kset;
-
-struct ldlm_cb_async_args {
-	struct ldlm_cb_set_arg *ca_set_arg;
-	struct ldlm_lock       *ca_lock;
-};
-
-/* LDLM state */
-
-static struct ldlm_state *ldlm_state;
-
-#define ELT_STOPPED   0
-#define ELT_READY     1
-#define ELT_TERMINATE 2
-
-struct ldlm_bl_pool {
-	spinlock_t		blp_lock;
-
-	/*
-	 * blp_prio_list is used for callbacks that should be handled
-	 * as a priority. It is used for LDLM_FL_DISCARD_DATA requests.
-	 * see bug 13843
-	 */
-	struct list_head	      blp_prio_list;
-
-	/*
-	 * blp_list is used for all other callbacks which are likely
-	 * to take longer to process.
-	 */
-	struct list_head	      blp_list;
-
-	wait_queue_head_t	     blp_waitq;
-	struct completion	blp_comp;
-	atomic_t	    blp_num_threads;
-	atomic_t	    blp_busy_threads;
-	int		     blp_min_threads;
-	int		     blp_max_threads;
-};
-
-struct ldlm_bl_work_item {
-	struct list_head	      blwi_entry;
-	struct ldlm_namespace  *blwi_ns;
-	struct ldlm_lock_desc   blwi_ld;
-	struct ldlm_lock       *blwi_lock;
-	struct list_head	      blwi_head;
-	int		     blwi_count;
-	struct completion	blwi_comp;
-	enum ldlm_cancel_flags  blwi_flags;
-	int		     blwi_mem_pressure;
-};
-
-/**
- * Callback handler for receiving incoming blocking ASTs.
- *
- * This can only happen on client side.
- */
-void ldlm_handle_bl_callback(struct ldlm_namespace *ns,
-			     struct ldlm_lock_desc *ld, struct ldlm_lock *lock)
-{
-	int do_ast;
-
-	LDLM_DEBUG(lock, "client blocking AST callback handler");
-
-	lock_res_and_lock(lock);
-	ldlm_set_cbpending(lock);
-
-	if (ldlm_is_cancel_on_block(lock))
-		ldlm_set_cancel(lock);
-
-	do_ast = !lock->l_readers && !lock->l_writers;
-	unlock_res_and_lock(lock);
-
-	if (do_ast) {
-		CDEBUG(D_DLMTRACE,
-		       "Lock %p already unused, calling callback (%p)\n", lock,
-		       lock->l_blocking_ast);
-		if (lock->l_blocking_ast)
-			lock->l_blocking_ast(lock, ld, lock->l_ast_data,
-					     LDLM_CB_BLOCKING);
-	} else {
-		CDEBUG(D_DLMTRACE,
-		       "Lock %p is referenced, will be cancelled later\n",
-		       lock);
-	}
-
-	LDLM_DEBUG(lock, "client blocking callback handler END");
-	LDLM_LOCK_RELEASE(lock);
-}
-
-/**
- * Callback handler for receiving incoming completion ASTs.
- *
- * This only can happen on client side.
- */
-static void ldlm_handle_cp_callback(struct ptlrpc_request *req,
-				    struct ldlm_namespace *ns,
-				    struct ldlm_request *dlm_req,
-				    struct ldlm_lock *lock)
-{
-	int lvb_len;
-	LIST_HEAD(ast_list);
-	int rc = 0;
-
-	LDLM_DEBUG(lock, "client completion callback handler START");
-
-	if (OBD_FAIL_CHECK(OBD_FAIL_LDLM_CANCEL_BL_CB_RACE)) {
-		int to = HZ;
-
-		while (to > 0) {
-			set_current_state(TASK_INTERRUPTIBLE);
-			schedule_timeout(to);
-			if (lock->l_granted_mode == lock->l_req_mode ||
-			    ldlm_is_destroyed(lock))
-				break;
-		}
-	}
-
-	lvb_len = req_capsule_get_size(&req->rq_pill, &RMF_DLM_LVB, RCL_CLIENT);
-	if (lvb_len < 0) {
-		LDLM_ERROR(lock, "Fail to get lvb_len, rc = %d", lvb_len);
-		rc = lvb_len;
-		goto out;
-	} else if (lvb_len > 0) {
-		if (lock->l_lvb_len > 0) {
-			/* for extent lock, lvb contains ost_lvb{}. */
-			LASSERT(lock->l_lvb_data);
-
-			if (unlikely(lock->l_lvb_len < lvb_len)) {
-				LDLM_ERROR(lock,
-					   "Replied LVB is larger than expectation, expected = %d, replied = %d",
-					   lock->l_lvb_len, lvb_len);
-				rc = -EINVAL;
-				goto out;
-			}
-		} else if (ldlm_has_layout(lock)) { /* for layout lock, lvb has
-						     * variable length
-						     */
-			void *lvb_data;
-
-			lvb_data = kzalloc(lvb_len, GFP_NOFS);
-			if (!lvb_data) {
-				LDLM_ERROR(lock, "No memory: %d.\n", lvb_len);
-				rc = -ENOMEM;
-				goto out;
-			}
-
-			lock_res_and_lock(lock);
-			LASSERT(!lock->l_lvb_data);
-			lock->l_lvb_type = LVB_T_LAYOUT;
-			lock->l_lvb_data = lvb_data;
-			lock->l_lvb_len = lvb_len;
-			unlock_res_and_lock(lock);
-		}
-	}
-
-	lock_res_and_lock(lock);
-	if (ldlm_is_destroyed(lock) ||
-	    lock->l_granted_mode == lock->l_req_mode) {
-		/* bug 11300: the lock has already been granted */
-		unlock_res_and_lock(lock);
-		LDLM_DEBUG(lock, "Double grant race happened");
-		rc = 0;
-		goto out;
-	}
-
-	/* If we receive the completion AST before the actual enqueue returned,
-	 * then we might need to switch lock modes, resources, or extents.
-	 */
-	if (dlm_req->lock_desc.l_granted_mode != lock->l_req_mode) {
-		lock->l_req_mode = dlm_req->lock_desc.l_granted_mode;
-		LDLM_DEBUG(lock, "completion AST, new lock mode");
-	}
-
-	if (lock->l_resource->lr_type != LDLM_PLAIN) {
-		ldlm_convert_policy_to_local(req->rq_export,
-					  dlm_req->lock_desc.l_resource.lr_type,
-					  &dlm_req->lock_desc.l_policy_data,
-					  &lock->l_policy_data);
-		LDLM_DEBUG(lock, "completion AST, new policy data");
-	}
-
-	ldlm_resource_unlink_lock(lock);
-	if (memcmp(&dlm_req->lock_desc.l_resource.lr_name,
-		   &lock->l_resource->lr_name,
-		   sizeof(lock->l_resource->lr_name)) != 0) {
-		unlock_res_and_lock(lock);
-		rc = ldlm_lock_change_resource(ns, lock,
-				&dlm_req->lock_desc.l_resource.lr_name);
-		if (rc < 0) {
-			LDLM_ERROR(lock, "Failed to allocate resource");
-			goto out;
-		}
-		LDLM_DEBUG(lock, "completion AST, new resource");
-		CERROR("change resource!\n");
-		lock_res_and_lock(lock);
-	}
-
-	if (dlm_req->lock_flags & LDLM_FL_AST_SENT) {
-		/* BL_AST locks are not needed in LRU.
-		 * Let ldlm_cancel_lru() be fast.
-		 */
-		ldlm_lock_remove_from_lru(lock);
-		lock->l_flags |= LDLM_FL_CBPENDING | LDLM_FL_BL_AST;
-		LDLM_DEBUG(lock, "completion AST includes blocking AST");
-	}
-
-	if (lock->l_lvb_len > 0) {
-		rc = ldlm_fill_lvb(lock, &req->rq_pill, RCL_CLIENT,
-				   lock->l_lvb_data, lvb_len);
-		if (rc < 0) {
-			unlock_res_and_lock(lock);
-			goto out;
-		}
-	}
-
-	ldlm_grant_lock(lock, &ast_list);
-	unlock_res_and_lock(lock);
-
-	LDLM_DEBUG(lock, "callback handler finished, about to run_ast_work");
-
-	/* Let Enqueue to call osc_lock_upcall() and initialize l_ast_data */
-	OBD_FAIL_TIMEOUT(OBD_FAIL_OSC_CP_ENQ_RACE, 2);
-
-	ldlm_run_ast_work(ns, &ast_list, LDLM_WORK_CP_AST);
-
-	LDLM_DEBUG_NOLOCK("client completion callback handler END (lock %p)",
-			  lock);
-	goto out;
-
-out:
-	if (rc < 0) {
-		lock_res_and_lock(lock);
-		ldlm_set_failed(lock);
-		unlock_res_and_lock(lock);
-		wake_up(&lock->l_waitq);
-	}
-	LDLM_LOCK_RELEASE(lock);
-}
-
-/**
- * Callback handler for receiving incoming glimpse ASTs.
- *
- * This only can happen on client side.  After handling the glimpse AST
- * we also consider dropping the lock here if it is unused locally for a
- * long time.
- */
-static void ldlm_handle_gl_callback(struct ptlrpc_request *req,
-				    struct ldlm_namespace *ns,
-				    struct ldlm_request *dlm_req,
-				    struct ldlm_lock *lock)
-{
-	int rc = -ENOSYS;
-
-	LDLM_DEBUG(lock, "client glimpse AST callback handler");
-
-	if (lock->l_glimpse_ast)
-		rc = lock->l_glimpse_ast(lock, req);
-
-	if (req->rq_repmsg) {
-		ptlrpc_reply(req);
-	} else {
-		req->rq_status = rc;
-		ptlrpc_error(req);
-	}
-
-	lock_res_and_lock(lock);
-	if (lock->l_granted_mode == LCK_PW &&
-	    !lock->l_readers && !lock->l_writers &&
-	    time_after(jiffies,
-		       lock->l_last_used + 10 * HZ)) {
-		unlock_res_and_lock(lock);
-		if (ldlm_bl_to_thread_lock(ns, NULL, lock))
-			ldlm_handle_bl_callback(ns, NULL, lock);
-
-		return;
-	}
-	unlock_res_and_lock(lock);
-	LDLM_LOCK_RELEASE(lock);
-}
-
-static int ldlm_callback_reply(struct ptlrpc_request *req, int rc)
-{
-	if (req->rq_no_reply)
-		return 0;
-
-	req->rq_status = rc;
-	if (!req->rq_packed_final) {
-		rc = lustre_pack_reply(req, 1, NULL, NULL);
-		if (rc)
-			return rc;
-	}
-	return ptlrpc_reply(req);
-}
-
-static int __ldlm_bl_to_thread(struct ldlm_bl_work_item *blwi,
-			       enum ldlm_cancel_flags cancel_flags)
-{
-	struct ldlm_bl_pool *blp = ldlm_state->ldlm_bl_pool;
-
-	spin_lock(&blp->blp_lock);
-	if (blwi->blwi_lock && ldlm_is_discard_data(blwi->blwi_lock)) {
-		/* add LDLM_FL_DISCARD_DATA requests to the priority list */
-		list_add_tail(&blwi->blwi_entry, &blp->blp_prio_list);
-	} else {
-		/* other blocking callbacks are added to the regular list */
-		list_add_tail(&blwi->blwi_entry, &blp->blp_list);
-	}
-	spin_unlock(&blp->blp_lock);
-
-	wake_up(&blp->blp_waitq);
-
-	/* can not check blwi->blwi_flags as blwi could be already freed in
-	 * LCF_ASYNC mode
-	 */
-	if (!(cancel_flags & LCF_ASYNC))
-		wait_for_completion(&blwi->blwi_comp);
-
-	return 0;
-}
-
-static inline void init_blwi(struct ldlm_bl_work_item *blwi,
-			     struct ldlm_namespace *ns,
-			     struct ldlm_lock_desc *ld,
-			     struct list_head *cancels, int count,
-			     struct ldlm_lock *lock,
-			     enum ldlm_cancel_flags cancel_flags)
-{
-	init_completion(&blwi->blwi_comp);
-	INIT_LIST_HEAD(&blwi->blwi_head);
-
-	if (current->flags & PF_MEMALLOC)
-		blwi->blwi_mem_pressure = 1;
-
-	blwi->blwi_ns = ns;
-	blwi->blwi_flags = cancel_flags;
-	if (ld)
-		blwi->blwi_ld = *ld;
-	if (count) {
-		list_add(&blwi->blwi_head, cancels);
-		list_del_init(cancels);
-		blwi->blwi_count = count;
-	} else {
-		blwi->blwi_lock = lock;
-	}
-}
-
-/**
- * Queues a list of locks \a cancels containing \a count locks
- * for later processing by a blocking thread.  If \a count is zero,
- * then the lock referenced as \a lock is queued instead.
- *
- * The blocking thread would then call ->l_blocking_ast callback in the lock.
- * If list addition fails an error is returned and caller is supposed to
- * call ->l_blocking_ast itself.
- */
-static int ldlm_bl_to_thread(struct ldlm_namespace *ns,
-			     struct ldlm_lock_desc *ld,
-			     struct ldlm_lock *lock,
-			     struct list_head *cancels, int count,
-			     enum ldlm_cancel_flags cancel_flags)
-{
-	if (cancels && count == 0)
-		return 0;
-
-	if (cancel_flags & LCF_ASYNC) {
-		struct ldlm_bl_work_item *blwi;
-
-		blwi = kzalloc(sizeof(*blwi), GFP_NOFS);
-		if (!blwi)
-			return -ENOMEM;
-		init_blwi(blwi, ns, ld, cancels, count, lock, cancel_flags);
-
-		return __ldlm_bl_to_thread(blwi, cancel_flags);
-	} else {
-		/* if it is synchronous call do minimum mem alloc, as it could
-		 * be triggered from kernel shrinker
-		 */
-		struct ldlm_bl_work_item blwi;
-
-		memset(&blwi, 0, sizeof(blwi));
-		init_blwi(&blwi, ns, ld, cancels, count, lock, cancel_flags);
-		return __ldlm_bl_to_thread(&blwi, cancel_flags);
-	}
-}
-
-int ldlm_bl_to_thread_lock(struct ldlm_namespace *ns, struct ldlm_lock_desc *ld,
-			   struct ldlm_lock *lock)
-{
-	return ldlm_bl_to_thread(ns, ld, lock, NULL, 0, LCF_ASYNC);
-}
-
-int ldlm_bl_to_thread_list(struct ldlm_namespace *ns, struct ldlm_lock_desc *ld,
-			   struct list_head *cancels, int count,
-			   enum ldlm_cancel_flags cancel_flags)
-{
-	return ldlm_bl_to_thread(ns, ld, NULL, cancels, count, cancel_flags);
-}
-
-int ldlm_bl_thread_wakeup(void)
-{
-	wake_up(&ldlm_state->ldlm_bl_pool->blp_waitq);
-	return 0;
-}
-
-/* Setinfo coming from Server (eg MDT) to Client (eg MDC)! */
-static int ldlm_handle_setinfo(struct ptlrpc_request *req)
-{
-	struct obd_device *obd = req->rq_export->exp_obd;
-	char *key;
-	void *val;
-	int keylen, vallen;
-	int rc = -ENOSYS;
-
-	DEBUG_REQ(D_HSM, req, "%s: handle setinfo\n", obd->obd_name);
-
-	req_capsule_set(&req->rq_pill, &RQF_OBD_SET_INFO);
-
-	key = req_capsule_client_get(&req->rq_pill, &RMF_SETINFO_KEY);
-	if (!key) {
-		DEBUG_REQ(D_IOCTL, req, "no set_info key");
-		return -EFAULT;
-	}
-	keylen = req_capsule_get_size(&req->rq_pill, &RMF_SETINFO_KEY,
-				      RCL_CLIENT);
-	val = req_capsule_client_get(&req->rq_pill, &RMF_SETINFO_VAL);
-	if (!val) {
-		DEBUG_REQ(D_IOCTL, req, "no set_info val");
-		return -EFAULT;
-	}
-	vallen = req_capsule_get_size(&req->rq_pill, &RMF_SETINFO_VAL,
-				      RCL_CLIENT);
-
-	/* We are responsible for swabbing contents of val */
-
-	if (KEY_IS(KEY_HSM_COPYTOOL_SEND))
-		/* Pass it on to mdc (the "export" in this case) */
-		rc = obd_set_info_async(req->rq_svc_thread->t_env,
-					req->rq_export,
-					sizeof(KEY_HSM_COPYTOOL_SEND),
-					KEY_HSM_COPYTOOL_SEND,
-					vallen, val, NULL);
-	else
-		DEBUG_REQ(D_WARNING, req, "ignoring unknown key %s", key);
-
-	return rc;
-}
-
-static inline void ldlm_callback_errmsg(struct ptlrpc_request *req,
-					const char *msg, int rc,
-					const struct lustre_handle *handle)
-{
-	DEBUG_REQ((req->rq_no_reply || rc) ? D_WARNING : D_DLMTRACE, req,
-		  "%s: [nid %s] [rc %d] [lock %#llx]",
-		  msg, libcfs_id2str(req->rq_peer), rc,
-		  handle ? handle->cookie : 0);
-	if (req->rq_no_reply)
-		CWARN("No reply was sent, maybe cause bug 21636.\n");
-	else if (rc)
-		CWARN("Send reply failed, maybe cause bug 21636.\n");
-}
-
-/* TODO: handle requests in a similar way as MDT: see mdt_handle_common() */
-static int ldlm_callback_handler(struct ptlrpc_request *req)
-{
-	struct ldlm_namespace *ns;
-	struct ldlm_request *dlm_req;
-	struct ldlm_lock *lock;
-	int rc;
-
-	/* Requests arrive in sender's byte order.  The ptlrpc service
-	 * handler has already checked and, if necessary, byte-swapped the
-	 * incoming request message body, but I am responsible for the
-	 * message buffers.
-	 */
-
-	/* do nothing for sec context finalize */
-	if (lustre_msg_get_opc(req->rq_reqmsg) == SEC_CTX_FINI)
-		return 0;
-
-	req_capsule_init(&req->rq_pill, req, RCL_SERVER);
-
-	if (!req->rq_export) {
-		rc = ldlm_callback_reply(req, -ENOTCONN);
-		ldlm_callback_errmsg(req, "Operate on unconnected server",
-				     rc, NULL);
-		return 0;
-	}
-
-	LASSERT(req->rq_export->exp_obd);
-
-	switch (lustre_msg_get_opc(req->rq_reqmsg)) {
-	case LDLM_BL_CALLBACK:
-		if (OBD_FAIL_CHECK(OBD_FAIL_LDLM_BL_CALLBACK_NET)) {
-			if (cfs_fail_err)
-				ldlm_callback_reply(req, -(int)cfs_fail_err);
-			return 0;
-		}
-		break;
-	case LDLM_CP_CALLBACK:
-		if (OBD_FAIL_CHECK(OBD_FAIL_LDLM_CP_CALLBACK_NET))
-			return 0;
-		break;
-	case LDLM_GL_CALLBACK:
-		if (OBD_FAIL_CHECK(OBD_FAIL_LDLM_GL_CALLBACK_NET))
-			return 0;
-		break;
-	case LDLM_SET_INFO:
-		rc = ldlm_handle_setinfo(req);
-		ldlm_callback_reply(req, rc);
-		return 0;
-	default:
-		CERROR("unknown opcode %u\n",
-		       lustre_msg_get_opc(req->rq_reqmsg));
-		ldlm_callback_reply(req, -EPROTO);
-		return 0;
-	}
-
-	ns = req->rq_export->exp_obd->obd_namespace;
-	LASSERT(ns);
-
-	req_capsule_set(&req->rq_pill, &RQF_LDLM_CALLBACK);
-
-	dlm_req = req_capsule_client_get(&req->rq_pill, &RMF_DLM_REQ);
-	if (!dlm_req) {
-		rc = ldlm_callback_reply(req, -EPROTO);
-		ldlm_callback_errmsg(req, "Operate without parameter", rc,
-				     NULL);
-		return 0;
-	}
-
-	/* Force a known safe race, send a cancel to the server for a lock
-	 * which the server has already started a blocking callback on.
-	 */
-	if (OBD_FAIL_CHECK(OBD_FAIL_LDLM_CANCEL_BL_CB_RACE) &&
-	    lustre_msg_get_opc(req->rq_reqmsg) == LDLM_BL_CALLBACK) {
-		rc = ldlm_cli_cancel(&dlm_req->lock_handle[0], 0);
-		if (rc < 0)
-			CERROR("ldlm_cli_cancel: %d\n", rc);
-	}
-
-	lock = ldlm_handle2lock_long(&dlm_req->lock_handle[0], 0);
-	if (!lock) {
-		CDEBUG(D_DLMTRACE,
-		       "callback on lock %#llx - lock disappeared\n",
-		       dlm_req->lock_handle[0].cookie);
-		rc = ldlm_callback_reply(req, -EINVAL);
-		ldlm_callback_errmsg(req, "Operate with invalid parameter", rc,
-				     &dlm_req->lock_handle[0]);
-		return 0;
-	}
-
-	if (ldlm_is_fail_loc(lock) &&
-	    lustre_msg_get_opc(req->rq_reqmsg) == LDLM_BL_CALLBACK)
-		OBD_RACE(OBD_FAIL_LDLM_CP_BL_RACE);
-
-	/* Copy hints/flags (e.g. LDLM_FL_DISCARD_DATA) from AST. */
-	lock_res_and_lock(lock);
-	lock->l_flags |= ldlm_flags_from_wire(dlm_req->lock_flags &
-					      LDLM_FL_AST_MASK);
-	if (lustre_msg_get_opc(req->rq_reqmsg) == LDLM_BL_CALLBACK) {
-		/* If somebody cancels lock and cache is already dropped,
-		 * or lock is failed before cp_ast received on client,
-		 * we can tell the server we have no lock. Otherwise, we
-		 * should send cancel after dropping the cache.
-		 */
-		if ((ldlm_is_canceling(lock) && ldlm_is_bl_done(lock)) ||
-		    ldlm_is_failed(lock)) {
-			LDLM_DEBUG(lock,
-				   "callback on lock %#llx - lock disappeared",
-				   dlm_req->lock_handle[0].cookie);
-			unlock_res_and_lock(lock);
-			LDLM_LOCK_RELEASE(lock);
-			rc = ldlm_callback_reply(req, -EINVAL);
-			ldlm_callback_errmsg(req, "Operate on stale lock", rc,
-					     &dlm_req->lock_handle[0]);
-			return 0;
-		}
-		/* BL_AST locks are not needed in LRU.
-		 * Let ldlm_cancel_lru() be fast.
-		 */
-		ldlm_lock_remove_from_lru(lock);
-		ldlm_set_bl_ast(lock);
-	}
-	unlock_res_and_lock(lock);
-
-	/* We want the ost thread to get this reply so that it can respond
-	 * to ost requests (write cache writeback) that might be triggered
-	 * in the callback.
-	 *
-	 * But we'd also like to be able to indicate in the reply that we're
-	 * cancelling right now, because it's unused, or have an intent result
-	 * in the reply, so we might have to push the responsibility for sending
-	 * the reply down into the AST handlers, alas.
-	 */
-
-	switch (lustre_msg_get_opc(req->rq_reqmsg)) {
-	case LDLM_BL_CALLBACK:
-		CDEBUG(D_INODE, "blocking ast\n");
-		req_capsule_extend(&req->rq_pill, &RQF_LDLM_BL_CALLBACK);
-		if (!ldlm_is_cancel_on_block(lock)) {
-			rc = ldlm_callback_reply(req, 0);
-			if (req->rq_no_reply || rc)
-				ldlm_callback_errmsg(req, "Normal process", rc,
-						     &dlm_req->lock_handle[0]);
-		}
-		if (ldlm_bl_to_thread_lock(ns, &dlm_req->lock_desc, lock))
-			ldlm_handle_bl_callback(ns, &dlm_req->lock_desc, lock);
-		break;
-	case LDLM_CP_CALLBACK:
-		CDEBUG(D_INODE, "completion ast\n");
-		req_capsule_extend(&req->rq_pill, &RQF_LDLM_CP_CALLBACK);
-		ldlm_callback_reply(req, 0);
-		ldlm_handle_cp_callback(req, ns, dlm_req, lock);
-		break;
-	case LDLM_GL_CALLBACK:
-		CDEBUG(D_INODE, "glimpse ast\n");
-		req_capsule_extend(&req->rq_pill, &RQF_LDLM_GL_CALLBACK);
-		ldlm_handle_gl_callback(req, ns, dlm_req, lock);
-		break;
-	default:
-		LBUG();			 /* checked above */
-	}
-
-	return 0;
-}
-
-static int ldlm_bl_get_work(struct ldlm_bl_pool *blp,
-			    struct ldlm_bl_work_item **p_blwi,
-			    struct obd_export **p_exp)
-{
-	int num_th = atomic_read(&blp->blp_num_threads);
-	struct ldlm_bl_work_item *blwi = NULL;
-	static unsigned int num_bl;
-
-	spin_lock(&blp->blp_lock);
-	/* process a request from the blp_list at least every blp_num_threads */
-	if (!list_empty(&blp->blp_list) &&
-	    (list_empty(&blp->blp_prio_list) || num_bl == 0))
-		blwi = list_first_entry(&blp->blp_list,
-					struct ldlm_bl_work_item, blwi_entry);
-	else
-		if (!list_empty(&blp->blp_prio_list))
-			blwi = list_first_entry(&blp->blp_prio_list,
-						struct ldlm_bl_work_item,
-						blwi_entry);
-
-	if (blwi) {
-		if (++num_bl >= num_th)
-			num_bl = 0;
-		list_del(&blwi->blwi_entry);
-	}
-	spin_unlock(&blp->blp_lock);
-	*p_blwi = blwi;
-
-	return (*p_blwi || *p_exp) ? 1 : 0;
-}
-
-/* This only contains temporary data until the thread starts */
-struct ldlm_bl_thread_data {
-	struct ldlm_bl_pool	*bltd_blp;
-	struct completion	bltd_comp;
-	int			bltd_num;
-};
-
-static int ldlm_bl_thread_main(void *arg);
-
-static int ldlm_bl_thread_start(struct ldlm_bl_pool *blp, bool check_busy)
-{
-	struct ldlm_bl_thread_data bltd = { .bltd_blp = blp };
-	struct task_struct *task;
-
-	init_completion(&bltd.bltd_comp);
-
-	bltd.bltd_num = atomic_inc_return(&blp->blp_num_threads);
-	if (bltd.bltd_num >= blp->blp_max_threads) {
-		atomic_dec(&blp->blp_num_threads);
-		return 0;
-	}
-
-	LASSERTF(bltd.bltd_num > 0, "thread num:%d\n", bltd.bltd_num);
-	if (check_busy &&
-	    atomic_read(&blp->blp_busy_threads) < (bltd.bltd_num - 1)) {
-		atomic_dec(&blp->blp_num_threads);
-		return 0;
-	}
-
-	task = kthread_run(ldlm_bl_thread_main, &bltd, "ldlm_bl_%02d",
-			   bltd.bltd_num);
-	if (IS_ERR(task)) {
-		CERROR("cannot start LDLM thread ldlm_bl_%02d: rc %ld\n",
-		       bltd.bltd_num, PTR_ERR(task));
-		atomic_dec(&blp->blp_num_threads);
-		return PTR_ERR(task);
-	}
-	wait_for_completion(&bltd.bltd_comp);
-
-	return 0;
-}
-
-/* Not fatal if racy and have a few too many threads */
-static int ldlm_bl_thread_need_create(struct ldlm_bl_pool *blp,
-				      struct ldlm_bl_work_item *blwi)
-{
-	if (atomic_read(&blp->blp_num_threads) >= blp->blp_max_threads)
-		return 0;
-
-	if (atomic_read(&blp->blp_busy_threads) <
-	    atomic_read(&blp->blp_num_threads))
-		return 0;
-
-	if (blwi && (!blwi->blwi_ns || blwi->blwi_mem_pressure))
-		return 0;
-
-	return 1;
-}
-
-static int ldlm_bl_thread_blwi(struct ldlm_bl_pool *blp,
-			       struct ldlm_bl_work_item *blwi)
-{
-	unsigned int flags = 0;
-
-	if (!blwi->blwi_ns)
-		/* added by ldlm_cleanup() */
-		return LDLM_ITER_STOP;
-
-	if (blwi->blwi_mem_pressure)
-		flags = memalloc_noreclaim_save();
-
-	OBD_FAIL_TIMEOUT(OBD_FAIL_LDLM_PAUSE_CANCEL2, 4);
-
-	if (blwi->blwi_count) {
-		int count;
-
-		/*
-		 * The special case when we cancel locks in lru
-		 * asynchronously, we pass the list of locks here.
-		 * Thus locks are marked LDLM_FL_CANCELING, but NOT
-		 * canceled locally yet.
-		 */
-		count = ldlm_cli_cancel_list_local(&blwi->blwi_head,
-						   blwi->blwi_count,
-						   LCF_BL_AST);
-		ldlm_cli_cancel_list(&blwi->blwi_head, count, NULL,
-				     blwi->blwi_flags);
-	} else {
-		ldlm_handle_bl_callback(blwi->blwi_ns, &blwi->blwi_ld,
-					blwi->blwi_lock);
-	}
-	if (blwi->blwi_mem_pressure)
-		memalloc_noreclaim_restore(flags);
-
-	if (blwi->blwi_flags & LCF_ASYNC)
-		kfree(blwi);
-	else
-		complete(&blwi->blwi_comp);
-
-	return 0;
-}
-
-/**
- * Main blocking requests processing thread.
- *
- * Callers put locks into its queue by calling ldlm_bl_to_thread.
- * This thread in the end ends up doing actual call to ->l_blocking_ast
- * for queued locks.
- */
-static int ldlm_bl_thread_main(void *arg)
-{
-	struct ldlm_bl_pool *blp;
-	struct ldlm_bl_thread_data *bltd = arg;
-
-	blp = bltd->bltd_blp;
-
-	complete(&bltd->bltd_comp);
-	/* cannot use bltd after this, it is only on caller's stack */
-
-	while (1) {
-		struct ldlm_bl_work_item *blwi = NULL;
-		struct obd_export *exp = NULL;
-		int rc;
-
-		rc = ldlm_bl_get_work(blp, &blwi, &exp);
-		if (!rc)
-			wait_event_idle_exclusive(blp->blp_waitq,
-						  ldlm_bl_get_work(blp, &blwi,
-								   &exp));
-		atomic_inc(&blp->blp_busy_threads);
-
-		if (ldlm_bl_thread_need_create(blp, blwi))
-			/* discard the return value, we tried */
-			ldlm_bl_thread_start(blp, true);
-
-		if (blwi)
-			rc = ldlm_bl_thread_blwi(blp, blwi);
-
-		atomic_dec(&blp->blp_busy_threads);
-
-		if (rc == LDLM_ITER_STOP)
-			break;
-	}
-
-	atomic_dec(&blp->blp_num_threads);
-	complete(&blp->blp_comp);
-	return 0;
-}
-
-static int ldlm_setup(void);
-static int ldlm_cleanup(void);
-
-int ldlm_get_ref(void)
-{
-	int rc = 0;
-
-	rc = ptlrpc_inc_ref();
-	if (rc)
-		return rc;
-
-	mutex_lock(&ldlm_ref_mutex);
-	if (++ldlm_refcount == 1) {
-		rc = ldlm_setup();
-		if (rc)
-			ldlm_refcount--;
-	}
-	mutex_unlock(&ldlm_ref_mutex);
-
-	if (rc)
-		ptlrpc_dec_ref();
-
-	return rc;
-}
-
-void ldlm_put_ref(void)
-{
-	int rc = 0;
-	mutex_lock(&ldlm_ref_mutex);
-	if (ldlm_refcount == 1) {
-		rc = ldlm_cleanup();
-
-		if (rc)
-			CERROR("ldlm_cleanup failed: %d\n", rc);
-		else
-			ldlm_refcount--;
-	} else {
-		ldlm_refcount--;
-	}
-	mutex_unlock(&ldlm_ref_mutex);
-	if (!rc)
-		ptlrpc_dec_ref();
-}
-
-static ssize_t cancel_unused_locks_before_replay_show(struct kobject *kobj,
-						      struct attribute *attr,
-						      char *buf)
-{
-	return sprintf(buf, "%d\n", ldlm_cancel_unused_locks_before_replay);
-}
-
-static ssize_t cancel_unused_locks_before_replay_store(struct kobject *kobj,
-						       struct attribute *attr,
-						       const char *buffer,
-						       size_t count)
-{
-	int rc;
-	unsigned long val;
-
-	rc = kstrtoul(buffer, 10, &val);
-	if (rc)
-		return rc;
-
-	ldlm_cancel_unused_locks_before_replay = val;
-
-	return count;
-}
-LUSTRE_RW_ATTR(cancel_unused_locks_before_replay);
-
-/* These are for root of /sys/fs/lustre/ldlm */
-static struct attribute *ldlm_attrs[] = {
-	&lustre_attr_cancel_unused_locks_before_replay.attr,
-	NULL,
-};
-
-static const struct attribute_group ldlm_attr_group = {
-	.attrs = ldlm_attrs,
-};
-
-static int ldlm_setup(void)
-{
-	static struct ptlrpc_service_conf	conf;
-	struct ldlm_bl_pool			*blp = NULL;
-	int rc = 0;
-	int i;
-
-	if (ldlm_state)
-		return -EALREADY;
-
-	ldlm_state = kzalloc(sizeof(*ldlm_state), GFP_NOFS);
-	if (!ldlm_state)
-		return -ENOMEM;
-
-	ldlm_kobj = kobject_create_and_add("ldlm", lustre_kobj);
-	if (!ldlm_kobj) {
-		rc = -ENOMEM;
-		goto out;
-	}
-
-	rc = sysfs_create_group(ldlm_kobj, &ldlm_attr_group);
-	if (rc)
-		goto out;
-
-	ldlm_ns_kset = kset_create_and_add("namespaces", NULL, ldlm_kobj);
-	if (!ldlm_ns_kset) {
-		rc = -ENOMEM;
-		goto out;
-	}
-
-	ldlm_svc_kset = kset_create_and_add("services", NULL, ldlm_kobj);
-	if (!ldlm_svc_kset) {
-		rc = -ENOMEM;
-		goto out;
-	}
-
-	ldlm_debugfs_setup();
-
-	memset(&conf, 0, sizeof(conf));
-	conf = (typeof(conf)) {
-		.psc_name		= "ldlm_cbd",
-		.psc_watchdog_factor	= 2,
-		.psc_buf		= {
-			.bc_nbufs		= LDLM_CLIENT_NBUFS,
-			.bc_buf_size		= LDLM_BUFSIZE,
-			.bc_req_max_size	= LDLM_MAXREQSIZE,
-			.bc_rep_max_size	= LDLM_MAXREPSIZE,
-			.bc_req_portal		= LDLM_CB_REQUEST_PORTAL,
-			.bc_rep_portal		= LDLM_CB_REPLY_PORTAL,
-		},
-		.psc_thr		= {
-			.tc_thr_name		= "ldlm_cb",
-			.tc_thr_factor		= LDLM_THR_FACTOR,
-			.tc_nthrs_init		= LDLM_NTHRS_INIT,
-			.tc_nthrs_base		= LDLM_NTHRS_BASE,
-			.tc_nthrs_max		= LDLM_NTHRS_MAX,
-			.tc_nthrs_user		= ldlm_num_threads,
-			.tc_cpu_affinity	= 1,
-			.tc_ctx_tags		= LCT_MD_THREAD | LCT_DT_THREAD,
-		},
-		.psc_cpt		= {
-			.cc_pattern		= ldlm_cpts,
-		},
-		.psc_ops		= {
-			.so_req_handler		= ldlm_callback_handler,
-		},
-	};
-	ldlm_state->ldlm_cb_service =
-			ptlrpc_register_service(&conf, ldlm_svc_kset,
-						ldlm_svc_debugfs_dir);
-	if (IS_ERR(ldlm_state->ldlm_cb_service)) {
-		CERROR("failed to start service\n");
-		rc = PTR_ERR(ldlm_state->ldlm_cb_service);
-		ldlm_state->ldlm_cb_service = NULL;
-		goto out;
-	}
-
-	blp = kzalloc(sizeof(*blp), GFP_NOFS);
-	if (!blp) {
-		rc = -ENOMEM;
-		goto out;
-	}
-	ldlm_state->ldlm_bl_pool = blp;
-
-	spin_lock_init(&blp->blp_lock);
-	INIT_LIST_HEAD(&blp->blp_list);
-	INIT_LIST_HEAD(&blp->blp_prio_list);
-	init_waitqueue_head(&blp->blp_waitq);
-	atomic_set(&blp->blp_num_threads, 0);
-	atomic_set(&blp->blp_busy_threads, 0);
-
-	if (ldlm_num_threads == 0) {
-		blp->blp_min_threads = LDLM_NTHRS_INIT;
-		blp->blp_max_threads = LDLM_NTHRS_MAX;
-	} else {
-		blp->blp_min_threads = min_t(int, LDLM_NTHRS_MAX,
-					     max_t(int, LDLM_NTHRS_INIT,
-						   ldlm_num_threads));
-
-		blp->blp_max_threads = blp->blp_min_threads;
-	}
-
-	for (i = 0; i < blp->blp_min_threads; i++) {
-		rc = ldlm_bl_thread_start(blp, false);
-		if (rc < 0)
-			goto out;
-	}
-
-	rc = ldlm_pools_init();
-	if (rc) {
-		CERROR("Failed to initialize LDLM pools: %d\n", rc);
-		goto out;
-	}
-	return 0;
-
- out:
-	ldlm_cleanup();
-	return rc;
-}
-
-static int ldlm_cleanup(void)
-{
-	if (!list_empty(ldlm_namespace_list(LDLM_NAMESPACE_SERVER)) ||
-	    !list_empty(ldlm_namespace_list(LDLM_NAMESPACE_CLIENT))) {
-		CERROR("ldlm still has namespaces; clean these up first.\n");
-		ldlm_dump_all_namespaces(LDLM_NAMESPACE_SERVER, D_DLMTRACE);
-		ldlm_dump_all_namespaces(LDLM_NAMESPACE_CLIENT, D_DLMTRACE);
-		return -EBUSY;
-	}
-
-	ldlm_pools_fini();
-
-	if (ldlm_state->ldlm_bl_pool) {
-		struct ldlm_bl_pool *blp = ldlm_state->ldlm_bl_pool;
-
-		while (atomic_read(&blp->blp_num_threads) > 0) {
-			struct ldlm_bl_work_item blwi = { .blwi_ns = NULL };
-
-			init_completion(&blp->blp_comp);
-
-			spin_lock(&blp->blp_lock);
-			list_add_tail(&blwi.blwi_entry, &blp->blp_list);
-			wake_up(&blp->blp_waitq);
-			spin_unlock(&blp->blp_lock);
-
-			wait_for_completion(&blp->blp_comp);
-		}
-
-		kfree(blp);
-	}
-
-	if (ldlm_state->ldlm_cb_service)
-		ptlrpc_unregister_service(ldlm_state->ldlm_cb_service);
-
-	if (ldlm_ns_kset)
-		kset_unregister(ldlm_ns_kset);
-	if (ldlm_svc_kset)
-		kset_unregister(ldlm_svc_kset);
-	if (ldlm_kobj) {
-		sysfs_remove_group(ldlm_kobj, &ldlm_attr_group);
-		kobject_put(ldlm_kobj);
-	}
-
-	ldlm_debugfs_cleanup();
-
-	kfree(ldlm_state);
-	ldlm_state = NULL;
-
-	return 0;
-}
-
-int ldlm_init(void)
-{
-	mutex_init(&ldlm_ref_mutex);
-	mutex_init(ldlm_namespace_lock(LDLM_NAMESPACE_SERVER));
-	mutex_init(ldlm_namespace_lock(LDLM_NAMESPACE_CLIENT));
-	ldlm_resource_slab = kmem_cache_create("ldlm_resources",
-					       sizeof(struct ldlm_resource), 0,
-					       SLAB_HWCACHE_ALIGN, NULL);
-	if (!ldlm_resource_slab)
-		return -ENOMEM;
-
-	ldlm_lock_slab = kmem_cache_create("ldlm_locks",
-					   sizeof(struct ldlm_lock), 0,
-					   SLAB_HWCACHE_ALIGN |
-					   SLAB_TYPESAFE_BY_RCU, NULL);
-	if (!ldlm_lock_slab) {
-		kmem_cache_destroy(ldlm_resource_slab);
-		return -ENOMEM;
-	}
-
-#if LUSTRE_TRACKS_LOCK_EXP_REFS
-	class_export_dump_hook = ldlm_dump_export_locks;
-#endif
-	return 0;
-}
-
-void ldlm_exit(void)
-{
-	if (ldlm_refcount)
-		CERROR("ldlm_refcount is %d in %s!\n", ldlm_refcount, __func__);
-	kmem_cache_destroy(ldlm_resource_slab);
-	/* ldlm_lock_put() use RCU to call ldlm_lock_free, so need call
-	 * synchronize_rcu() to wait a grace period elapsed, so that
-	 * ldlm_lock_free() get a chance to be called.
-	 */
-	synchronize_rcu();
-	kmem_cache_destroy(ldlm_lock_slab);
-}
diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_plain.c b/drivers/staging/lustre/lustre/ldlm/ldlm_plain.c
deleted file mode 100644
index 33b5a3f96fcb..000000000000
--- a/drivers/staging/lustre/lustre/ldlm/ldlm_plain.c
+++ /dev/null
@@ -1,68 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-/*
- * GPL HEADER START
- *
- * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 only,
- * as published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful, but
- * WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
- * General Public License version 2 for more details (a copy is included
- * in the LICENSE file that accompanied this code).
- *
- * You should have received a copy of the GNU General Public License
- * version 2 along with this program; If not, see
- * http://www.gnu.org/licenses/gpl-2.0.html
- *
- * GPL HEADER END
- */
-/*
- * Copyright (c) 2003, 2010, Oracle and/or its affiliates. All rights reserved.
- * Use is subject to license terms.
- *
- * Copyright (c) 2011, Intel Corporation.
- */
-/*
- * This file is part of Lustre, http://www.lustre.org/
- * Lustre is a trademark of Sun Microsystems, Inc.
- *
- * lustre/ldlm/ldlm_plain.c
- *
- * Author: Peter Braam <braam at clusterfs.com>
- * Author: Phil Schwan <phil at clusterfs.com>
- */
-
-/**
- * This file contains implementation of PLAIN lock type.
- *
- * PLAIN locks are the simplest form of LDLM locking, and are used when
- * there only needs to be a single lock on a resource. This avoids some
- * of the complexity of EXTENT and IBITS lock types, but doesn't allow
- * different "parts" of a resource to be locked concurrently.  Example
- * use cases for PLAIN locks include locking of MGS configuration logs
- * and (as of Lustre 2.4) quota records.
- */
-
-#define DEBUG_SUBSYSTEM S_LDLM
-
-#include <lustre_dlm.h>
-#include <obd_support.h>
-#include <lustre_lib.h>
-
-#include "ldlm_internal.h"
-
-void ldlm_plain_policy_wire_to_local(const union ldlm_wire_policy_data *wpolicy,
-				     union ldlm_policy_data *lpolicy)
-{
-	/* No policy for plain locks */
-}
-
-void ldlm_plain_policy_local_to_wire(const union ldlm_policy_data *lpolicy,
-				     union ldlm_wire_policy_data *wpolicy)
-{
-	/* No policy for plain locks */
-}
diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c b/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c
deleted file mode 100644
index 36d14ee4e5b1..000000000000
--- a/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c
+++ /dev/null
@@ -1,1013 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-/*
- * GPL HEADER START
- *
- * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 only,
- * as published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful, but
- * WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
- * General Public License version 2 for more details (a copy is included
- * in the LICENSE file that accompanied this code).
- *
- * You should have received a copy of the GNU General Public License
- * version 2 along with this program; If not, see
- * http://www.gnu.org/licenses/gpl-2.0.html
- *
- * GPL HEADER END
- */
-/*
- * Copyright (c) 2007, 2010, Oracle and/or its affiliates. All rights reserved.
- * Use is subject to license terms.
- *
- * Copyright (c) 2010, 2015, Intel Corporation.
- */
-/*
- * This file is part of Lustre, http://www.lustre.org/
- * Lustre is a trademark of Sun Microsystems, Inc.
- *
- * lustre/ldlm/ldlm_pool.c
- *
- * Author: Yury Umanets <umka at clusterfs.com>
- */
-
-/*
- * Idea of this code is rather simple. Each second, for each server namespace
- * we have SLV - server lock volume which is calculated on current number of
- * granted locks, grant speed for past period, etc - that is, locking load.
- * This SLV number may be thought as a flow definition for simplicity. It is
- * sent to clients with each occasion to let them know what is current load
- * situation on the server. By default, at the beginning, SLV on server is
- * set max value which is calculated as the following: allow to one client
- * have all locks of limit ->pl_limit for 10h.
- *
- * Next, on clients, number of cached locks is not limited artificially in any
- * way as it was before. Instead, client calculates CLV, that is, client lock
- * volume for each lock and compares it with last SLV from the server. CLV is
- * calculated as the number of locks in LRU * lock live time in seconds. If
- * CLV > SLV - lock is canceled.
- *
- * Client has LVF, that is, lock volume factor which regulates how much
- * sensitive client should be about last SLV from server. The higher LVF is the
- * more locks will be canceled on client. Default value for it is 1. Setting LVF
- * to 2 means that client will cancel locks 2 times faster.
- *
- * Locks on a client will be canceled more intensively in these cases:
- * (1) if SLV is smaller, that is, load is higher on the server;
- * (2) client has a lot of locks (the more locks are held by client, the bigger
- *     chances that some of them should be canceled);
- * (3) client has old locks (taken some time ago);
- *
- * Thus, according to flow paradigm that we use for better understanding SLV,
- * CLV is the volume of particle in flow described by SLV. According to this,
- * if flow is getting thinner, more and more particles become outside of it and
- * as particles are locks, they should be canceled.
- *
- * General idea of this belongs to Vitaly Fertman (vitaly at clusterfs.com).
- * Andreas Dilger (adilger at clusterfs.com) proposed few nice ideas like using
- * LVF and many cleanups. Flow definition to allow more easy understanding of
- * the logic belongs to Nikita Danilov (nikita at clusterfs.com) as well as many
- * cleanups and fixes. And design and implementation are done by Yury Umanets
- * (umka at clusterfs.com).
- *
- * Glossary for terms used:
- *
- * pl_limit - Number of allowed locks in pool. Applies to server and client
- * side (tunable);
- *
- * pl_granted - Number of granted locks (calculated);
- * pl_grant_rate - Number of granted locks for last T (calculated);
- * pl_cancel_rate - Number of canceled locks for last T (calculated);
- * pl_grant_speed - Grant speed (GR - CR) for last T (calculated);
- * pl_grant_plan - Planned number of granted locks for next T (calculated);
- * pl_server_lock_volume - Current server lock volume (calculated);
- *
- * As it may be seen from list above, we have few possible tunables which may
- * affect behavior much. They all may be modified via sysfs. However, they also
- * give a possibility for constructing few pre-defined behavior policies. If
- * none of predefines is suitable for a working pattern being used, new one may
- * be "constructed" via sysfs tunables.
- */
-
-#define DEBUG_SUBSYSTEM S_LDLM
-
-#include <lustre_dlm.h>
-#include <cl_object.h>
-#include <obd_class.h>
-#include <obd_support.h>
-#include "ldlm_internal.h"
-
-/*
- * 50 ldlm locks for 1MB of RAM.
- */
-#define LDLM_POOL_HOST_L ((NUM_CACHEPAGES >> (20 - PAGE_SHIFT)) * 50)
-
-/*
- * Maximal possible grant step plan in %.
- */
-#define LDLM_POOL_MAX_GSP (30)
-
-/*
- * Minimal possible grant step plan in %.
- */
-#define LDLM_POOL_MIN_GSP (1)
-
-/*
- * This controls the speed of reaching LDLM_POOL_MAX_GSP
- * with increasing thread period.
- */
-#define LDLM_POOL_GSP_STEP_SHIFT (2)
-
-/*
- * LDLM_POOL_GSP% of all locks is default GP.
- */
-#define LDLM_POOL_GP(L)   (((L) * LDLM_POOL_MAX_GSP) / 100)
-
-/*
- * Max age for locks on clients.
- */
-#define LDLM_POOL_MAX_AGE (36000)
-
-/*
- * The granularity of SLV calculation.
- */
-#define LDLM_POOL_SLV_SHIFT (10)
-
-static inline __u64 dru(__u64 val, __u32 shift, int round_up)
-{
-	return (val + (round_up ? (1 << shift) - 1 : 0)) >> shift;
-}
-
-static inline __u64 ldlm_pool_slv_max(__u32 L)
-{
-	/*
-	 * Allow to have all locks for 1 client for 10 hrs.
-	 * Formula is the following: limit * 10h / 1 client.
-	 */
-	__u64 lim = (__u64)L *  LDLM_POOL_MAX_AGE / 1;
-	return lim;
-}
-
-static inline __u64 ldlm_pool_slv_min(__u32 L)
-{
-	return 1;
-}
-
-enum {
-	LDLM_POOL_FIRST_STAT = 0,
-	LDLM_POOL_GRANTED_STAT = LDLM_POOL_FIRST_STAT,
-	LDLM_POOL_GRANT_STAT,
-	LDLM_POOL_CANCEL_STAT,
-	LDLM_POOL_GRANT_RATE_STAT,
-	LDLM_POOL_CANCEL_RATE_STAT,
-	LDLM_POOL_GRANT_PLAN_STAT,
-	LDLM_POOL_SLV_STAT,
-	LDLM_POOL_SHRINK_REQTD_STAT,
-	LDLM_POOL_SHRINK_FREED_STAT,
-	LDLM_POOL_RECALC_STAT,
-	LDLM_POOL_TIMING_STAT,
-	LDLM_POOL_LAST_STAT
-};
-
-/**
- * Calculates suggested grant_step in % of available locks for passed
- * \a period. This is later used in grant_plan calculations.
- */
-static inline int ldlm_pool_t2gsp(unsigned int t)
-{
-	/*
-	 * This yields 1% grant step for anything below LDLM_POOL_GSP_STEP
-	 * and up to 30% for anything higher than LDLM_POOL_GSP_STEP.
-	 *
-	 * How this will affect execution is the following:
-	 *
-	 * - for thread period 1s we will have grant_step 1% which good from
-	 * pov of taking some load off from server and push it out to clients.
-	 * This is like that because 1% for grant_step means that server will
-	 * not allow clients to get lots of locks in short period of time and
-	 * keep all old locks in their caches. Clients will always have to
-	 * get some locks back if they want to take some new;
-	 *
-	 * - for thread period 10s (which is default) we will have 23% which
-	 * means that clients will have enough of room to take some new locks
-	 * without getting some back. All locks from this 23% which were not
-	 * taken by clients in current period will contribute in SLV growing.
-	 * SLV growing means more locks cached on clients until limit or grant
-	 * plan is reached.
-	 */
-	return LDLM_POOL_MAX_GSP -
-		((LDLM_POOL_MAX_GSP - LDLM_POOL_MIN_GSP) >>
-		 (t >> LDLM_POOL_GSP_STEP_SHIFT));
-}
-
-/**
- * Recalculates next stats on passed \a pl.
- *
- * \pre ->pl_lock is locked.
- */
-static void ldlm_pool_recalc_stats(struct ldlm_pool *pl)
-{
-	int grant_plan = pl->pl_grant_plan;
-	__u64 slv = pl->pl_server_lock_volume;
-	int granted = atomic_read(&pl->pl_granted);
-	int grant_rate = atomic_read(&pl->pl_grant_rate);
-	int cancel_rate = atomic_read(&pl->pl_cancel_rate);
-
-	lprocfs_counter_add(pl->pl_stats, LDLM_POOL_SLV_STAT,
-			    slv);
-	lprocfs_counter_add(pl->pl_stats, LDLM_POOL_GRANTED_STAT,
-			    granted);
-	lprocfs_counter_add(pl->pl_stats, LDLM_POOL_GRANT_RATE_STAT,
-			    grant_rate);
-	lprocfs_counter_add(pl->pl_stats, LDLM_POOL_GRANT_PLAN_STAT,
-			    grant_plan);
-	lprocfs_counter_add(pl->pl_stats, LDLM_POOL_CANCEL_RATE_STAT,
-			    cancel_rate);
-}
-
-/**
- * Sets SLV and Limit from container_of(pl, struct ldlm_namespace,
- * ns_pool)->ns_obd tp passed \a pl.
- */
-static void ldlm_cli_pool_pop_slv(struct ldlm_pool *pl)
-{
-	struct obd_device *obd;
-
-	/*
-	 * Get new SLV and Limit from obd which is updated with coming
-	 * RPCs.
-	 */
-	obd = container_of(pl, struct ldlm_namespace,
-			   ns_pool)->ns_obd;
-	read_lock(&obd->obd_pool_lock);
-	pl->pl_server_lock_volume = obd->obd_pool_slv;
-	atomic_set(&pl->pl_limit, obd->obd_pool_limit);
-	read_unlock(&obd->obd_pool_lock);
-}
-
-/**
- * Recalculates client size pool \a pl according to current SLV and Limit.
- */
-static int ldlm_cli_pool_recalc(struct ldlm_pool *pl)
-{
-	time64_t recalc_interval_sec;
-	int ret;
-
-	recalc_interval_sec = ktime_get_real_seconds() - pl->pl_recalc_time;
-	if (recalc_interval_sec < pl->pl_recalc_period)
-		return 0;
-
-	spin_lock(&pl->pl_lock);
-	/*
-	 * Check if we need to recalc lists now.
-	 */
-	recalc_interval_sec = ktime_get_real_seconds() - pl->pl_recalc_time;
-	if (recalc_interval_sec < pl->pl_recalc_period) {
-		spin_unlock(&pl->pl_lock);
-		return 0;
-	}
-
-	/*
-	 * Make sure that pool knows last SLV and Limit from obd.
-	 */
-	ldlm_cli_pool_pop_slv(pl);
-
-	spin_unlock(&pl->pl_lock);
-
-	/*
-	 * Do not cancel locks in case lru resize is disabled for this ns.
-	 */
-	if (!ns_connect_lru_resize(container_of(pl, struct ldlm_namespace,
-						ns_pool))) {
-		ret = 0;
-		goto out;
-	}
-
-	/*
-	 * In the time of canceling locks on client we do not need to maintain
-	 * sharp timing, we only want to cancel locks asap according to new SLV.
-	 * It may be called when SLV has changed much, this is why we do not
-	 * take into account pl->pl_recalc_time here.
-	 */
-	ret = ldlm_cancel_lru(container_of(pl, struct ldlm_namespace, ns_pool),
-			      0, LCF_ASYNC, LDLM_LRU_FLAG_LRUR);
-
-out:
-	spin_lock(&pl->pl_lock);
-	/*
-	 * Time of LRU resizing might be longer than period,
-	 * so update after LRU resizing rather than before it.
-	 */
-	pl->pl_recalc_time = ktime_get_real_seconds();
-	lprocfs_counter_add(pl->pl_stats, LDLM_POOL_TIMING_STAT,
-			    recalc_interval_sec);
-	spin_unlock(&pl->pl_lock);
-	return ret;
-}
-
-/**
- * This function is main entry point for memory pressure handling on client
- * side.  Main goal of this function is to cancel some number of locks on
- * passed \a pl according to \a nr and \a gfp_mask.
- */
-static int ldlm_cli_pool_shrink(struct ldlm_pool *pl,
-				int nr, gfp_t gfp_mask)
-{
-	struct ldlm_namespace *ns;
-	int unused;
-
-	ns = container_of(pl, struct ldlm_namespace, ns_pool);
-
-	/*
-	 * Do not cancel locks in case lru resize is disabled for this ns.
-	 */
-	if (!ns_connect_lru_resize(ns))
-		return 0;
-
-	/*
-	 * Make sure that pool knows last SLV and Limit from obd.
-	 */
-	ldlm_cli_pool_pop_slv(pl);
-
-	spin_lock(&ns->ns_lock);
-	unused = ns->ns_nr_unused;
-	spin_unlock(&ns->ns_lock);
-
-	if (nr == 0)
-		return (unused / 100) * sysctl_vfs_cache_pressure;
-	else
-		return ldlm_cancel_lru(ns, nr, LCF_ASYNC, LDLM_LRU_FLAG_SHRINK);
-}
-
-static const struct ldlm_pool_ops ldlm_cli_pool_ops = {
-	.po_recalc = ldlm_cli_pool_recalc,
-	.po_shrink = ldlm_cli_pool_shrink
-};
-
-/**
- * Pool recalc wrapper. Will call either client or server pool recalc callback
- * depending what pool \a pl is used.
- */
-static int ldlm_pool_recalc(struct ldlm_pool *pl)
-{
-	u32 recalc_interval_sec;
-	int count;
-
-	recalc_interval_sec = ktime_get_real_seconds() - pl->pl_recalc_time;
-	if (recalc_interval_sec > 0) {
-		spin_lock(&pl->pl_lock);
-		recalc_interval_sec = ktime_get_real_seconds() - pl->pl_recalc_time;
-
-		if (recalc_interval_sec > 0) {
-			/*
-			 * Update pool statistics every 1s.
-			 */
-			ldlm_pool_recalc_stats(pl);
-
-			/*
-			 * Zero out all rates and speed for the last period.
-			 */
-			atomic_set(&pl->pl_grant_rate, 0);
-			atomic_set(&pl->pl_cancel_rate, 0);
-		}
-		spin_unlock(&pl->pl_lock);
-	}
-
-	if (pl->pl_ops->po_recalc) {
-		count = pl->pl_ops->po_recalc(pl);
-		lprocfs_counter_add(pl->pl_stats, LDLM_POOL_RECALC_STAT,
-				    count);
-	}
-
-	recalc_interval_sec = pl->pl_recalc_time - ktime_get_real_seconds() +
-			      pl->pl_recalc_period;
-	if (recalc_interval_sec <= 0) {
-		/* DEBUG: should be re-removed after LU-4536 is fixed */
-		CDEBUG(D_DLMTRACE,
-		       "%s: Negative interval(%ld), too short period(%ld)\n",
-		       pl->pl_name, (long)recalc_interval_sec,
-		       (long)pl->pl_recalc_period);
-
-		/* Prevent too frequent recalculation. */
-		recalc_interval_sec = 1;
-	}
-
-	return recalc_interval_sec;
-}
-
-/*
- * Pool shrink wrapper. Will call either client or server pool recalc callback
- * depending what pool pl is used. When nr == 0, just return the number of
- * freeable locks. Otherwise, return the number of canceled locks.
- */
-static int ldlm_pool_shrink(struct ldlm_pool *pl, int nr, gfp_t gfp_mask)
-{
-	int cancel = 0;
-
-	if (pl->pl_ops->po_shrink) {
-		cancel = pl->pl_ops->po_shrink(pl, nr, gfp_mask);
-		if (nr > 0) {
-			lprocfs_counter_add(pl->pl_stats,
-					    LDLM_POOL_SHRINK_REQTD_STAT,
-					    nr);
-			lprocfs_counter_add(pl->pl_stats,
-					    LDLM_POOL_SHRINK_FREED_STAT,
-					    cancel);
-			CDEBUG(D_DLMTRACE,
-			       "%s: request to shrink %d locks, shrunk %d\n",
-			       pl->pl_name, nr, cancel);
-		}
-	}
-	return cancel;
-}
-
-static int lprocfs_pool_state_seq_show(struct seq_file *m, void *unused)
-{
-	int granted, grant_rate, cancel_rate;
-	int grant_speed, lvf;
-	struct ldlm_pool *pl = m->private;
-	__u64 slv, clv;
-	__u32 limit;
-
-	spin_lock(&pl->pl_lock);
-	slv = pl->pl_server_lock_volume;
-	clv = pl->pl_client_lock_volume;
-	limit = atomic_read(&pl->pl_limit);
-	granted = atomic_read(&pl->pl_granted);
-	grant_rate = atomic_read(&pl->pl_grant_rate);
-	cancel_rate = atomic_read(&pl->pl_cancel_rate);
-	grant_speed = grant_rate - cancel_rate;
-	lvf = atomic_read(&pl->pl_lock_volume_factor);
-	spin_unlock(&pl->pl_lock);
-
-	seq_printf(m, "LDLM pool state (%s):\n"
-		      "  SLV: %llu\n"
-		      "  CLV: %llu\n"
-		      "  LVF: %d\n",
-		      pl->pl_name, slv, clv, lvf);
-
-	seq_printf(m, "  GR:  %d\n  CR:  %d\n  GS:  %d\n"
-		      "  G:   %d\n  L:   %d\n",
-		      grant_rate, cancel_rate, grant_speed,
-		      granted, limit);
-
-	return 0;
-}
-
-LPROC_SEQ_FOPS_RO(lprocfs_pool_state);
-
-static ssize_t grant_speed_show(struct kobject *kobj, struct attribute *attr,
-				char *buf)
-{
-	struct ldlm_pool *pl = container_of(kobj, struct ldlm_pool,
-					    pl_kobj);
-
-	int	       grant_speed;
-
-	spin_lock(&pl->pl_lock);
-	/* serialize with ldlm_pool_recalc */
-	grant_speed = atomic_read(&pl->pl_grant_rate) -
-			atomic_read(&pl->pl_cancel_rate);
-	spin_unlock(&pl->pl_lock);
-	return sprintf(buf, "%d\n", grant_speed);
-}
-LUSTRE_RO_ATTR(grant_speed);
-
-LDLM_POOL_SYSFS_READER_SHOW(grant_plan, int);
-LUSTRE_RO_ATTR(grant_plan);
-
-LDLM_POOL_SYSFS_READER_SHOW(recalc_period, int);
-LDLM_POOL_SYSFS_WRITER_STORE(recalc_period, int);
-LUSTRE_RW_ATTR(recalc_period);
-
-LDLM_POOL_SYSFS_READER_NOLOCK_SHOW(server_lock_volume, u64);
-LUSTRE_RO_ATTR(server_lock_volume);
-
-LDLM_POOL_SYSFS_READER_NOLOCK_SHOW(limit, atomic);
-LDLM_POOL_SYSFS_WRITER_NOLOCK_STORE(limit, atomic);
-LUSTRE_RW_ATTR(limit);
-
-LDLM_POOL_SYSFS_READER_NOLOCK_SHOW(granted, atomic);
-LUSTRE_RO_ATTR(granted);
-
-LDLM_POOL_SYSFS_READER_NOLOCK_SHOW(cancel_rate, atomic);
-LUSTRE_RO_ATTR(cancel_rate);
-
-LDLM_POOL_SYSFS_READER_NOLOCK_SHOW(grant_rate, atomic);
-LUSTRE_RO_ATTR(grant_rate);
-
-LDLM_POOL_SYSFS_READER_NOLOCK_SHOW(lock_volume_factor, atomic);
-LDLM_POOL_SYSFS_WRITER_NOLOCK_STORE(lock_volume_factor, atomic);
-LUSTRE_RW_ATTR(lock_volume_factor);
-
-#define LDLM_POOL_ADD_VAR(name, var, ops)			\
-	do {							\
-		snprintf(var_name, MAX_STRING_SIZE, #name);	\
-		pool_vars[0].data = var;			\
-		pool_vars[0].fops = ops;			\
-		ldebugfs_add_vars(pl->pl_debugfs_entry, pool_vars, NULL);\
-	} while (0)
-
-/* These are for pools in /sys/fs/lustre/ldlm/namespaces/.../pool */
-static struct attribute *ldlm_pl_attrs[] = {
-	&lustre_attr_grant_speed.attr,
-	&lustre_attr_grant_plan.attr,
-	&lustre_attr_recalc_period.attr,
-	&lustre_attr_server_lock_volume.attr,
-	&lustre_attr_limit.attr,
-	&lustre_attr_granted.attr,
-	&lustre_attr_cancel_rate.attr,
-	&lustre_attr_grant_rate.attr,
-	&lustre_attr_lock_volume_factor.attr,
-	NULL,
-};
-
-static void ldlm_pl_release(struct kobject *kobj)
-{
-	struct ldlm_pool *pl = container_of(kobj, struct ldlm_pool,
-					    pl_kobj);
-	complete(&pl->pl_kobj_unregister);
-}
-
-static struct kobj_type ldlm_pl_ktype = {
-	.default_attrs	= ldlm_pl_attrs,
-	.sysfs_ops	= &lustre_sysfs_ops,
-	.release	= ldlm_pl_release,
-};
-
-static int ldlm_pool_sysfs_init(struct ldlm_pool *pl)
-{
-	struct ldlm_namespace *ns = container_of(pl, struct ldlm_namespace,
-						 ns_pool);
-	int err;
-
-	init_completion(&pl->pl_kobj_unregister);
-	err = kobject_init_and_add(&pl->pl_kobj, &ldlm_pl_ktype, &ns->ns_kobj,
-				   "pool");
-
-	return err;
-}
-
-static int ldlm_pool_debugfs_init(struct ldlm_pool *pl)
-{
-	struct ldlm_namespace *ns = container_of(pl, struct ldlm_namespace,
-						 ns_pool);
-	struct dentry *debugfs_ns_parent;
-	struct lprocfs_vars pool_vars[2];
-	char *var_name = NULL;
-	int rc = 0;
-
-	var_name = kzalloc(MAX_STRING_SIZE + 1, GFP_NOFS);
-	if (!var_name)
-		return -ENOMEM;
-
-	debugfs_ns_parent = ns->ns_debugfs_entry;
-	if (IS_ERR_OR_NULL(debugfs_ns_parent)) {
-		CERROR("%s: debugfs entry is not initialized\n",
-		       ldlm_ns_name(ns));
-		rc = -EINVAL;
-		goto out_free_name;
-	}
-	pl->pl_debugfs_entry = debugfs_create_dir("pool", debugfs_ns_parent);
-
-	var_name[MAX_STRING_SIZE] = '\0';
-	memset(pool_vars, 0, sizeof(pool_vars));
-	pool_vars[0].name = var_name;
-
-	LDLM_POOL_ADD_VAR(state, pl, &lprocfs_pool_state_fops);
-
-	pl->pl_stats = lprocfs_alloc_stats(LDLM_POOL_LAST_STAT -
-					   LDLM_POOL_FIRST_STAT, 0);
-	if (!pl->pl_stats) {
-		rc = -ENOMEM;
-		goto out_free_name;
-	}
-
-	lprocfs_counter_init(pl->pl_stats, LDLM_POOL_GRANTED_STAT,
-			     LPROCFS_CNTR_AVGMINMAX | LPROCFS_CNTR_STDDEV,
-			     "granted", "locks");
-	lprocfs_counter_init(pl->pl_stats, LDLM_POOL_GRANT_STAT,
-			     LPROCFS_CNTR_AVGMINMAX | LPROCFS_CNTR_STDDEV,
-			     "grant", "locks");
-	lprocfs_counter_init(pl->pl_stats, LDLM_POOL_CANCEL_STAT,
-			     LPROCFS_CNTR_AVGMINMAX | LPROCFS_CNTR_STDDEV,
-			     "cancel", "locks");
-	lprocfs_counter_init(pl->pl_stats, LDLM_POOL_GRANT_RATE_STAT,
-			     LPROCFS_CNTR_AVGMINMAX | LPROCFS_CNTR_STDDEV,
-			     "grant_rate", "locks/s");
-	lprocfs_counter_init(pl->pl_stats, LDLM_POOL_CANCEL_RATE_STAT,
-			     LPROCFS_CNTR_AVGMINMAX | LPROCFS_CNTR_STDDEV,
-			     "cancel_rate", "locks/s");
-	lprocfs_counter_init(pl->pl_stats, LDLM_POOL_GRANT_PLAN_STAT,
-			     LPROCFS_CNTR_AVGMINMAX | LPROCFS_CNTR_STDDEV,
-			     "grant_plan", "locks/s");
-	lprocfs_counter_init(pl->pl_stats, LDLM_POOL_SLV_STAT,
-			     LPROCFS_CNTR_AVGMINMAX | LPROCFS_CNTR_STDDEV,
-			     "slv", "slv");
-	lprocfs_counter_init(pl->pl_stats, LDLM_POOL_SHRINK_REQTD_STAT,
-			     LPROCFS_CNTR_AVGMINMAX | LPROCFS_CNTR_STDDEV,
-			     "shrink_request", "locks");
-	lprocfs_counter_init(pl->pl_stats, LDLM_POOL_SHRINK_FREED_STAT,
-			     LPROCFS_CNTR_AVGMINMAX | LPROCFS_CNTR_STDDEV,
-			     "shrink_freed", "locks");
-	lprocfs_counter_init(pl->pl_stats, LDLM_POOL_RECALC_STAT,
-			     LPROCFS_CNTR_AVGMINMAX | LPROCFS_CNTR_STDDEV,
-			     "recalc_freed", "locks");
-	lprocfs_counter_init(pl->pl_stats, LDLM_POOL_TIMING_STAT,
-			     LPROCFS_CNTR_AVGMINMAX | LPROCFS_CNTR_STDDEV,
-			     "recalc_timing", "sec");
-	debugfs_create_file("stats", 0644, pl->pl_debugfs_entry, pl->pl_stats,
-			    &lprocfs_stats_seq_fops);
-
-out_free_name:
-	kfree(var_name);
-	return rc;
-}
-
-static void ldlm_pool_sysfs_fini(struct ldlm_pool *pl)
-{
-	kobject_put(&pl->pl_kobj);
-	wait_for_completion(&pl->pl_kobj_unregister);
-}
-
-static void ldlm_pool_debugfs_fini(struct ldlm_pool *pl)
-{
-	if (pl->pl_stats) {
-		lprocfs_free_stats(&pl->pl_stats);
-		pl->pl_stats = NULL;
-	}
-	debugfs_remove_recursive(pl->pl_debugfs_entry);
-}
-
-int ldlm_pool_init(struct ldlm_pool *pl, struct ldlm_namespace *ns,
-		   int idx, enum ldlm_side client)
-{
-	int rc;
-
-	spin_lock_init(&pl->pl_lock);
-	atomic_set(&pl->pl_granted, 0);
-	pl->pl_recalc_time = ktime_get_real_seconds();
-	atomic_set(&pl->pl_lock_volume_factor, 1);
-
-	atomic_set(&pl->pl_grant_rate, 0);
-	atomic_set(&pl->pl_cancel_rate, 0);
-	pl->pl_grant_plan = LDLM_POOL_GP(LDLM_POOL_HOST_L);
-
-	snprintf(pl->pl_name, sizeof(pl->pl_name), "ldlm-pool-%s-%d",
-		 ldlm_ns_name(ns), idx);
-
-	atomic_set(&pl->pl_limit, 1);
-	pl->pl_server_lock_volume = 0;
-	pl->pl_ops = &ldlm_cli_pool_ops;
-	pl->pl_recalc_period = LDLM_POOL_CLI_DEF_RECALC_PERIOD;
-	pl->pl_client_lock_volume = 0;
-	rc = ldlm_pool_debugfs_init(pl);
-	if (rc)
-		return rc;
-
-	rc = ldlm_pool_sysfs_init(pl);
-	if (rc)
-		return rc;
-
-	CDEBUG(D_DLMTRACE, "Lock pool %s is initialized\n", pl->pl_name);
-
-	return rc;
-}
-
-void ldlm_pool_fini(struct ldlm_pool *pl)
-{
-	ldlm_pool_sysfs_fini(pl);
-	ldlm_pool_debugfs_fini(pl);
-
-	/*
-	 * Pool should not be used after this point. We can't free it here as
-	 * it lives in struct ldlm_namespace, but still interested in catching
-	 * any abnormal using cases.
-	 */
-	POISON(pl, 0x5a, sizeof(*pl));
-}
-
-/**
- * Add new taken ldlm lock \a lock into pool \a pl accounting.
- */
-void ldlm_pool_add(struct ldlm_pool *pl, struct ldlm_lock *lock)
-{
-	/*
-	 * FLOCK locks are special in a sense that they are almost never
-	 * cancelled, instead special kind of lock is used to drop them.
-	 * also there is no LRU for flock locks, so no point in tracking
-	 * them anyway.
-	 */
-	if (lock->l_resource->lr_type == LDLM_FLOCK)
-		return;
-
-	atomic_inc(&pl->pl_granted);
-	atomic_inc(&pl->pl_grant_rate);
-	lprocfs_counter_incr(pl->pl_stats, LDLM_POOL_GRANT_STAT);
-	/*
-	 * Do not do pool recalc for client side as all locks which
-	 * potentially may be canceled has already been packed into
-	 * enqueue/cancel rpc. Also we do not want to run out of stack
-	 * with too long call paths.
-	 */
-}
-
-/**
- * Remove ldlm lock \a lock from pool \a pl accounting.
- */
-void ldlm_pool_del(struct ldlm_pool *pl, struct ldlm_lock *lock)
-{
-	/*
-	 * Filter out FLOCK locks. Read above comment in ldlm_pool_add().
-	 */
-	if (lock->l_resource->lr_type == LDLM_FLOCK)
-		return;
-
-	LASSERT(atomic_read(&pl->pl_granted) > 0);
-	atomic_dec(&pl->pl_granted);
-	atomic_inc(&pl->pl_cancel_rate);
-
-	lprocfs_counter_incr(pl->pl_stats, LDLM_POOL_CANCEL_STAT);
-}
-
-/**
- * Returns current \a pl SLV.
- *
- * \pre ->pl_lock is not locked.
- */
-__u64 ldlm_pool_get_slv(struct ldlm_pool *pl)
-{
-	__u64 slv;
-
-	spin_lock(&pl->pl_lock);
-	slv = pl->pl_server_lock_volume;
-	spin_unlock(&pl->pl_lock);
-	return slv;
-}
-
-/**
- * Sets passed \a clv to \a pl.
- *
- * \pre ->pl_lock is not locked.
- */
-void ldlm_pool_set_clv(struct ldlm_pool *pl, __u64 clv)
-{
-	spin_lock(&pl->pl_lock);
-	pl->pl_client_lock_volume = clv;
-	spin_unlock(&pl->pl_lock);
-}
-
-/**
- * Returns current LVF from \a pl.
- */
-__u32 ldlm_pool_get_lvf(struct ldlm_pool *pl)
-{
-	return atomic_read(&pl->pl_lock_volume_factor);
-}
-
-static int ldlm_pool_granted(struct ldlm_pool *pl)
-{
-	return atomic_read(&pl->pl_granted);
-}
-
-/*
- * count locks from all namespaces (if possible). Returns number of
- * cached locks.
- */
-static unsigned long ldlm_pools_count(enum ldlm_side client, gfp_t gfp_mask)
-{
-	unsigned long total = 0;
-	int nr_ns;
-	struct ldlm_namespace *ns;
-	struct ldlm_namespace *ns_old = NULL; /* loop detection */
-
-	if (client == LDLM_NAMESPACE_CLIENT && !(gfp_mask & __GFP_FS))
-		return 0;
-
-	CDEBUG(D_DLMTRACE, "Request to count %s locks from all pools\n",
-	       client == LDLM_NAMESPACE_CLIENT ? "client" : "server");
-
-	/*
-	 * Find out how many resources we may release.
-	 */
-	for (nr_ns = ldlm_namespace_nr_read(client);
-	     nr_ns > 0; nr_ns--) {
-		mutex_lock(ldlm_namespace_lock(client));
-		if (list_empty(ldlm_namespace_list(client))) {
-			mutex_unlock(ldlm_namespace_lock(client));
-			return 0;
-		}
-		ns = ldlm_namespace_first_locked(client);
-
-		if (ns == ns_old) {
-			mutex_unlock(ldlm_namespace_lock(client));
-			break;
-		}
-
-		if (ldlm_ns_empty(ns)) {
-			ldlm_namespace_move_to_inactive_locked(ns, client);
-			mutex_unlock(ldlm_namespace_lock(client));
-			continue;
-		}
-
-		if (!ns_old)
-			ns_old = ns;
-
-		ldlm_namespace_get(ns);
-		ldlm_namespace_move_to_active_locked(ns, client);
-		mutex_unlock(ldlm_namespace_lock(client));
-		total += ldlm_pool_shrink(&ns->ns_pool, 0, gfp_mask);
-		ldlm_namespace_put(ns);
-	}
-
-	return total;
-}
-
-static unsigned long ldlm_pools_scan(enum ldlm_side client, int nr,
-				     gfp_t gfp_mask)
-{
-	unsigned long freed = 0;
-	int tmp, nr_ns;
-	struct ldlm_namespace *ns;
-
-	if (client == LDLM_NAMESPACE_CLIENT && !(gfp_mask & __GFP_FS))
-		return -1;
-
-	/*
-	 * Shrink at least ldlm_namespace_nr_read(client) namespaces.
-	 */
-	for (tmp = nr_ns = ldlm_namespace_nr_read(client);
-	     tmp > 0; tmp--) {
-		int cancel, nr_locks;
-
-		/*
-		 * Do not call shrink under ldlm_namespace_lock(client)
-		 */
-		mutex_lock(ldlm_namespace_lock(client));
-		if (list_empty(ldlm_namespace_list(client))) {
-			mutex_unlock(ldlm_namespace_lock(client));
-			break;
-		}
-		ns = ldlm_namespace_first_locked(client);
-		ldlm_namespace_get(ns);
-		ldlm_namespace_move_to_active_locked(ns, client);
-		mutex_unlock(ldlm_namespace_lock(client));
-
-		nr_locks = ldlm_pool_granted(&ns->ns_pool);
-		/*
-		 * We use to shrink propotionally but with new shrinker API,
-		 * we lost the total number of freeable locks.
-		 */
-		cancel = 1 + min_t(int, nr_locks, nr / nr_ns);
-		freed += ldlm_pool_shrink(&ns->ns_pool, cancel, gfp_mask);
-		ldlm_namespace_put(ns);
-	}
-	/*
-	 * we only decrease the SLV in server pools shrinker, return
-	 * SHRINK_STOP to kernel to avoid needless loop. LU-1128
-	 */
-	return freed;
-}
-
-static unsigned long ldlm_pools_cli_count(struct shrinker *s,
-					  struct shrink_control *sc)
-{
-	return ldlm_pools_count(LDLM_NAMESPACE_CLIENT, sc->gfp_mask);
-}
-
-static unsigned long ldlm_pools_cli_scan(struct shrinker *s,
-					 struct shrink_control *sc)
-{
-	return ldlm_pools_scan(LDLM_NAMESPACE_CLIENT, sc->nr_to_scan,
-			       sc->gfp_mask);
-}
-
-static void ldlm_pools_recalc(struct work_struct *ws);
-static DECLARE_DELAYED_WORK(ldlm_recalc_pools, ldlm_pools_recalc);
-
-static void ldlm_pools_recalc(struct work_struct *ws)
-{
-	enum ldlm_side client = LDLM_NAMESPACE_CLIENT;
-	struct ldlm_namespace *ns;
-	struct ldlm_namespace *ns_old = NULL;
-	/* seconds of sleep if no active namespaces */
-	int time = LDLM_POOL_CLI_DEF_RECALC_PERIOD;
-	int nr;
-
-	/*
-	 * Recalc at least ldlm_namespace_nr_read(client) namespaces.
-	 */
-	for (nr = ldlm_namespace_nr_read(client); nr > 0; nr--) {
-		int     skip;
-		/*
-		 * Lock the list, get first @ns in the list, getref, move it
-		 * to the tail, unlock and call pool recalc. This way we avoid
-		 * calling recalc under @ns lock what is really good as we get
-		 * rid of potential deadlock on client nodes when canceling
-		 * locks synchronously.
-		 */
-		mutex_lock(ldlm_namespace_lock(client));
-		if (list_empty(ldlm_namespace_list(client))) {
-			mutex_unlock(ldlm_namespace_lock(client));
-			break;
-		}
-		ns = ldlm_namespace_first_locked(client);
-
-		if (ns_old == ns) { /* Full pass complete */
-			mutex_unlock(ldlm_namespace_lock(client));
-			break;
-		}
-
-		/* We got an empty namespace, need to move it back to inactive
-		 * list.
-		 * The race with parallel resource creation is fine:
-		 * - If they do namespace_get before our check, we fail the
-		 *   check and they move this item to the end of the list anyway
-		 * - If we do the check and then they do namespace_get, then
-		 *   we move the namespace to inactive and they will move
-		 *   it back to active (synchronised by the lock, so no clash
-		 *   there).
-		 */
-		if (ldlm_ns_empty(ns)) {
-			ldlm_namespace_move_to_inactive_locked(ns, client);
-			mutex_unlock(ldlm_namespace_lock(client));
-			continue;
-		}
-
-		if (!ns_old)
-			ns_old = ns;
-
-		spin_lock(&ns->ns_lock);
-		/*
-		 * skip ns which is being freed, and we don't want to increase
-		 * its refcount again, not even temporarily. bz21519 & LU-499.
-		 */
-		if (ns->ns_stopping) {
-			skip = 1;
-		} else {
-			skip = 0;
-			ldlm_namespace_get(ns);
-		}
-		spin_unlock(&ns->ns_lock);
-
-		ldlm_namespace_move_to_active_locked(ns, client);
-		mutex_unlock(ldlm_namespace_lock(client));
-
-		/*
-		 * After setup is done - recalc the pool.
-		 */
-		if (!skip) {
-			int ttime = ldlm_pool_recalc(&ns->ns_pool);
-
-			if (ttime < time)
-				time = ttime;
-
-			ldlm_namespace_put(ns);
-		}
-	}
-
-	/* Wake up the blocking threads from time to time. */
-	ldlm_bl_thread_wakeup();
-
-	schedule_delayed_work(&ldlm_recalc_pools, time * HZ);
-}
-
-static int ldlm_pools_thread_start(void)
-{
-	schedule_delayed_work(&ldlm_recalc_pools, 0);
-
-	return 0;
-}
-
-static void ldlm_pools_thread_stop(void)
-{
-	cancel_delayed_work_sync(&ldlm_recalc_pools);
-}
-
-static struct shrinker ldlm_pools_cli_shrinker = {
-	.count_objects	= ldlm_pools_cli_count,
-	.scan_objects	= ldlm_pools_cli_scan,
-	.seeks		= DEFAULT_SEEKS,
-};
-
-int ldlm_pools_init(void)
-{
-	int rc;
-
-	rc = ldlm_pools_thread_start();
-	if (!rc)
-		rc = register_shrinker(&ldlm_pools_cli_shrinker);
-
-	return rc;
-}
-
-void ldlm_pools_fini(void)
-{
-	unregister_shrinker(&ldlm_pools_cli_shrinker);
-
-	ldlm_pools_thread_stop();
-}
diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_request.c b/drivers/staging/lustre/lustre/ldlm/ldlm_request.c
deleted file mode 100644
index cdc52eed6d85..000000000000
--- a/drivers/staging/lustre/lustre/ldlm/ldlm_request.c
+++ /dev/null
@@ -1,2033 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-/*
- * GPL HEADER START
- *
- * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 only,
- * as published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful, but
- * WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
- * General Public License version 2 for more details (a copy is included
- * in the LICENSE file that accompanied this code).
- *
- * You should have received a copy of the GNU General Public License
- * version 2 along with this program; If not, see
- * http://www.gnu.org/licenses/gpl-2.0.html
- *
- * GPL HEADER END
- */
-/*
- * Copyright (c) 2002, 2010, Oracle and/or its affiliates. All rights reserved.
- * Use is subject to license terms.
- *
- * Copyright (c) 2010, 2015, Intel Corporation.
- */
-/*
- * This file is part of Lustre, http://www.lustre.org/
- * Lustre is a trademark of Sun Microsystems, Inc.
- */
-/**
- * This file contains Asynchronous System Trap (AST) handlers and related
- * LDLM request-processing routines.
- *
- * An AST is a callback issued on a lock when its state is changed. There are
- * several different types of ASTs (callbacks) registered for each lock:
- *
- * - completion AST: when a lock is enqueued by some process, but cannot be
- *   granted immediately due to other conflicting locks on the same resource,
- *   the completion AST is sent to notify the caller when the lock is
- *   eventually granted
- *
- * - blocking AST: when a lock is granted to some process, if another process
- *   enqueues a conflicting (blocking) lock on a resource, a blocking AST is
- *   sent to notify the holder(s) of the lock(s) of the conflicting lock
- *   request. The lock holder(s) must release their lock(s) on that resource in
- *   a timely manner or be evicted by the server.
- *
- * - glimpse AST: this is used when a process wants information about a lock
- *   (i.e. the lock value block (LVB)) but does not necessarily require holding
- *   the lock. If the resource is locked, the lock holder(s) are sent glimpse
- *   ASTs and the LVB is returned to the caller, and lock holder(s) may CANCEL
- *   their lock(s) if they are idle. If the resource is not locked, the server
- *   may grant the lock.
- */
-
-#define DEBUG_SUBSYSTEM S_LDLM
-
-#include <lustre_errno.h>
-#include <lustre_dlm.h>
-#include <obd_class.h>
-#include <obd.h>
-#include <linux/libcfs/libcfs_hash.h>
-
-#include "ldlm_internal.h"
-
-unsigned int ldlm_enqueue_min = OBD_TIMEOUT_DEFAULT;
-module_param(ldlm_enqueue_min, uint, 0644);
-MODULE_PARM_DESC(ldlm_enqueue_min, "lock enqueue timeout minimum");
-
-/* in client side, whether the cached locks will be canceled before replay */
-unsigned int ldlm_cancel_unused_locks_before_replay = 1;
-
-struct ldlm_async_args {
-	struct lustre_handle lock_handle;
-};
-
-/**
- * ldlm_request_bufsize
- *
- * @count:	number of ldlm handles
- * @type:	ldlm opcode
- *
- * If opcode=LDLM_ENQUEUE, 1 slot is already occupied,
- * LDLM_LOCKREQ_HANDLE -1 slots are available.
- * Otherwise, LDLM_LOCKREQ_HANDLE slots are available.
- *
- * Return:	size of the request buffer
- */
-static int ldlm_request_bufsize(int count, int type)
-{
-	int avail = LDLM_LOCKREQ_HANDLES;
-
-	if (type == LDLM_ENQUEUE)
-		avail -= LDLM_ENQUEUE_CANCEL_OFF;
-
-	if (count > avail)
-		avail = (count - avail) * sizeof(struct lustre_handle);
-	else
-		avail = 0;
-
-	return sizeof(struct ldlm_request) + avail;
-}
-
-static void ldlm_expired_completion_wait(struct ldlm_lock *lock, __u32 conn_cnt)
-{
-	struct obd_import *imp;
-	struct obd_device *obd;
-
-	if (!lock->l_conn_export) {
-		static unsigned long next_dump, last_dump;
-
-		LDLM_ERROR(lock,
-			   "lock timed out (enqueued at %lld, %llds ago); not entering recovery in server code, just going back to sleep",
-			   (s64)lock->l_last_activity,
-			   (s64)(ktime_get_real_seconds() -
-				 lock->l_last_activity));
-		if (time_after(jiffies, next_dump)) {
-			last_dump = next_dump;
-			next_dump = jiffies + 300 * HZ;
-			ldlm_namespace_dump(D_DLMTRACE,
-					    ldlm_lock_to_ns(lock));
-			if (last_dump == 0)
-				libcfs_debug_dumplog();
-		}
-		return;
-	}
-
-	obd = lock->l_conn_export->exp_obd;
-	imp = obd->u.cli.cl_import;
-	ptlrpc_fail_import(imp, conn_cnt);
-	LDLM_ERROR(lock,
-		   "lock timed out (enqueued at %lld, %llds ago), entering recovery for %s@%s",
-		   (s64)lock->l_last_activity,
-		   (s64)(ktime_get_real_seconds() - lock->l_last_activity),
-		   obd2cli_tgt(obd), imp->imp_connection->c_remote_uuid.uuid);
-}
-
-/**
- * Calculate the Completion timeout (covering enqueue, BL AST, data flush,
- * lock cancel, and their replies). Used for lock completion timeout on the
- * client side.
- *
- * \param[in] lock	lock which is waiting the completion callback
- *
- * \retval		timeout in seconds to wait for the server reply
- */
-/* We use the same basis for both server side and client side functions
- * from a single node.
- */
-static unsigned int ldlm_cp_timeout(struct ldlm_lock *lock)
-{
-	unsigned int timeout;
-
-	if (AT_OFF)
-		return obd_timeout;
-
-	/*
-	 * Wait a long time for enqueue - server may have to callback a
-	 * lock from another client.  Server will evict the other client if it
-	 * doesn't respond reasonably, and then give us the lock.
-	 */
-	timeout = at_get(ldlm_lock_to_ns_at(lock));
-	return max(3 * timeout, ldlm_enqueue_min);
-}
-
-/**
- * Helper function for ldlm_completion_ast(), updating timings when lock is
- * actually granted.
- */
-static int ldlm_completion_tail(struct ldlm_lock *lock, void *data)
-{
-	long delay;
-	int result = 0;
-
-	if (ldlm_is_destroyed(lock) || ldlm_is_failed(lock)) {
-		LDLM_DEBUG(lock, "client-side enqueue: destroyed");
-		result = -EIO;
-	} else if (!data) {
-		LDLM_DEBUG(lock, "client-side enqueue: granted");
-	} else {
-		/* Take into AT only CP RPC, not immediately granted locks */
-		delay = ktime_get_real_seconds() - lock->l_last_activity;
-		LDLM_DEBUG(lock, "client-side enqueue: granted after %lds",
-			   delay);
-
-		/* Update our time estimate */
-		at_measured(ldlm_lock_to_ns_at(lock), delay);
-	}
-	return result;
-}
-
-/**
- * Generic LDLM "completion" AST. This is called in several cases:
- *
- *     - when a reply to an ENQUEUE RPC is received from the server
- *       (ldlm_cli_enqueue_fini()). Lock might be granted or not granted at
- *       this point (determined by flags);
- *
- *     - when LDLM_CP_CALLBACK RPC comes to client to notify it that lock has
- *       been granted;
- *
- *     - when ldlm_lock_match(LDLM_FL_LVB_READY) is about to wait until lock
- *       gets correct lvb;
- *
- *     - to force all locks when resource is destroyed (cleanup_resource());
- *
- *     - during lock conversion (not used currently).
- *
- * If lock is not granted in the first case, this function waits until second
- * or penultimate cases happen in some other thread.
- *
- */
-int ldlm_completion_ast(struct ldlm_lock *lock, __u64 flags, void *data)
-{
-	/* XXX ALLOCATE - 160 bytes */
-	struct obd_device *obd;
-	struct obd_import *imp = NULL;
-	__u32 timeout;
-	__u32 conn_cnt = 0;
-	int rc = 0;
-
-	if (flags == LDLM_FL_WAIT_NOREPROC) {
-		LDLM_DEBUG(lock, "client-side enqueue waiting on pending lock");
-		goto noreproc;
-	}
-
-	if (!(flags & LDLM_FL_BLOCKED_MASK)) {
-		wake_up(&lock->l_waitq);
-		return 0;
-	}
-
-	LDLM_DEBUG(lock,
-		   "client-side enqueue returned a blocked lock, sleeping");
-
-noreproc:
-
-	obd = class_exp2obd(lock->l_conn_export);
-
-	/* if this is a local lock, then there is no import */
-	if (obd)
-		imp = obd->u.cli.cl_import;
-
-	timeout = ldlm_cp_timeout(lock);
-
-	lock->l_last_activity = ktime_get_real_seconds();
-
-	if (imp) {
-		spin_lock(&imp->imp_lock);
-		conn_cnt = imp->imp_conn_cnt;
-		spin_unlock(&imp->imp_lock);
-	}
-	if (OBD_FAIL_CHECK_RESET(OBD_FAIL_LDLM_INTR_CP_AST,
-				 OBD_FAIL_LDLM_CP_BL_RACE | OBD_FAIL_ONCE)) {
-		ldlm_set_fail_loc(lock);
-		rc = -EINTR;
-	} else {
-		/* Go to sleep until the lock is granted or canceled. */
-		if (!ldlm_is_no_timeout(lock)) {
-			/* Wait uninterruptible for a while first */
-			rc = wait_event_idle_timeout(lock->l_waitq,
-						     is_granted_or_cancelled(lock),
-						     timeout * HZ);
-			if (rc == 0)
-				ldlm_expired_completion_wait(lock, conn_cnt);
-		}
-		/* Now wait abortable */
-		if (rc == 0)
-			rc = l_wait_event_abortable(lock->l_waitq,
-						    is_granted_or_cancelled(lock));
-		else
-			rc = 0;
-	}
-
-	if (rc) {
-		LDLM_DEBUG(lock, "client-side enqueue waking up: failed (%d)",
-			   rc);
-		return rc;
-	}
-
-	return ldlm_completion_tail(lock, data);
-}
-EXPORT_SYMBOL(ldlm_completion_ast);
-
-static void failed_lock_cleanup(struct ldlm_namespace *ns,
-				struct ldlm_lock *lock, int mode)
-{
-	int need_cancel = 0;
-
-	/* Set a flag to prevent us from sending a CANCEL (bug 407) */
-	lock_res_and_lock(lock);
-	/* Check that lock is not granted or failed, we might race. */
-	if ((lock->l_req_mode != lock->l_granted_mode) &&
-	    !ldlm_is_failed(lock)) {
-		/* Make sure that this lock will not be found by raced
-		 * bl_ast and -EINVAL reply is sent to server anyways.
-		 * bug 17645
-		 */
-		lock->l_flags |= LDLM_FL_LOCAL_ONLY | LDLM_FL_FAILED |
-				 LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING;
-		need_cancel = 1;
-	}
-	unlock_res_and_lock(lock);
-
-	if (need_cancel)
-		LDLM_DEBUG(lock,
-			   "setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING");
-	else
-		LDLM_DEBUG(lock, "lock was granted or failed in race");
-
-	/* XXX - HACK because we shouldn't call ldlm_lock_destroy()
-	 *       from llite/file.c/ll_file_flock().
-	 */
-	/* This code makes for the fact that we do not have blocking handler on
-	 * a client for flock locks. As such this is the place where we must
-	 * completely kill failed locks. (interrupted and those that
-	 * were waiting to be granted when server evicted us.
-	 */
-	if (lock->l_resource->lr_type == LDLM_FLOCK) {
-		lock_res_and_lock(lock);
-		if (!ldlm_is_destroyed(lock)) {
-			ldlm_resource_unlink_lock(lock);
-			ldlm_lock_decref_internal_nolock(lock, mode);
-			ldlm_lock_destroy_nolock(lock);
-		}
-		unlock_res_and_lock(lock);
-	} else {
-		ldlm_lock_decref_internal(lock, mode);
-	}
-}
-
-/**
- * Finishing portion of client lock enqueue code.
- *
- * Called after receiving reply from server.
- */
-int ldlm_cli_enqueue_fini(struct obd_export *exp, struct ptlrpc_request *req,
-			  enum ldlm_type type, __u8 with_policy,
-			  enum ldlm_mode mode,
-			  __u64 *flags, void *lvb, __u32 lvb_len,
-			  const struct lustre_handle *lockh, int rc)
-{
-	struct ldlm_namespace *ns = exp->exp_obd->obd_namespace;
-	int is_replay = *flags & LDLM_FL_REPLAY;
-	struct ldlm_lock *lock;
-	struct ldlm_reply *reply;
-	int cleanup_phase = 1;
-
-	lock = ldlm_handle2lock(lockh);
-	/* ldlm_cli_enqueue is holding a reference on this lock. */
-	if (!lock) {
-		LASSERT(type == LDLM_FLOCK);
-		return -ENOLCK;
-	}
-
-	LASSERTF(ergo(lvb_len != 0, lvb_len == lock->l_lvb_len),
-		 "lvb_len = %d, l_lvb_len = %d\n", lvb_len, lock->l_lvb_len);
-
-	if (rc != ELDLM_OK) {
-		LASSERT(!is_replay);
-		LDLM_DEBUG(lock, "client-side enqueue END (%s)",
-			   rc == ELDLM_LOCK_ABORTED ? "ABORTED" : "FAILED");
-
-		if (rc != ELDLM_LOCK_ABORTED)
-			goto cleanup;
-	}
-
-	/* Before we return, swab the reply */
-	reply = req_capsule_server_get(&req->rq_pill, &RMF_DLM_REP);
-	if (!reply) {
-		rc = -EPROTO;
-		goto cleanup;
-	}
-
-	if (lvb_len > 0) {
-		int size = 0;
-
-		size = req_capsule_get_size(&req->rq_pill, &RMF_DLM_LVB,
-					    RCL_SERVER);
-		if (size < 0) {
-			LDLM_ERROR(lock, "Fail to get lvb_len, rc = %d", size);
-			rc = size;
-			goto cleanup;
-		} else if (unlikely(size > lvb_len)) {
-			LDLM_ERROR(lock,
-				   "Replied LVB is larger than expectation, expected = %d, replied = %d",
-				   lvb_len, size);
-			rc = -EINVAL;
-			goto cleanup;
-		}
-		lvb_len = size;
-	}
-
-	if (rc == ELDLM_LOCK_ABORTED) {
-		if (lvb_len > 0 && lvb)
-			rc = ldlm_fill_lvb(lock, &req->rq_pill, RCL_SERVER,
-					   lvb, lvb_len);
-		if (rc == 0)
-			rc = ELDLM_LOCK_ABORTED;
-		goto cleanup;
-	}
-
-	/* lock enqueued on the server */
-	cleanup_phase = 0;
-
-	lock_res_and_lock(lock);
-	lock->l_remote_handle = reply->lock_handle;
-
-	*flags = ldlm_flags_from_wire(reply->lock_flags);
-	lock->l_flags |= ldlm_flags_from_wire(reply->lock_flags &
-					      LDLM_FL_INHERIT_MASK);
-	unlock_res_and_lock(lock);
-
-	CDEBUG(D_INFO, "local: %p, remote cookie: %#llx, flags: 0x%llx\n",
-	       lock, reply->lock_handle.cookie, *flags);
-
-	/* If enqueue returned a blocked lock but the completion handler has
-	 * already run, then it fixed up the resource and we don't need to do it
-	 * again.
-	 */
-	if ((*flags) & LDLM_FL_LOCK_CHANGED) {
-		int newmode = reply->lock_desc.l_req_mode;
-
-		LASSERT(!is_replay);
-		if (newmode && newmode != lock->l_req_mode) {
-			LDLM_DEBUG(lock, "server returned different mode %s",
-				   ldlm_lockname[newmode]);
-			lock->l_req_mode = newmode;
-		}
-
-		if (!ldlm_res_eq(&reply->lock_desc.l_resource.lr_name,
-				 &lock->l_resource->lr_name)) {
-			CDEBUG(D_INFO,
-			       "remote intent success, locking " DLDLMRES " instead of " DLDLMRES "\n",
-			       PLDLMRES(&reply->lock_desc.l_resource),
-			       PLDLMRES(lock->l_resource));
-
-			rc = ldlm_lock_change_resource(ns, lock,
-					&reply->lock_desc.l_resource.lr_name);
-			if (rc || !lock->l_resource) {
-				rc = -ENOMEM;
-				goto cleanup;
-			}
-			LDLM_DEBUG(lock, "client-side enqueue, new resource");
-		}
-		if (with_policy)
-			if (!(type == LDLM_IBITS &&
-			      !(exp_connect_flags(exp) & OBD_CONNECT_IBITS)))
-				/* We assume lock type cannot change on server*/
-				ldlm_convert_policy_to_local(exp,
-						lock->l_resource->lr_type,
-						&reply->lock_desc.l_policy_data,
-						&lock->l_policy_data);
-		if (type != LDLM_PLAIN)
-			LDLM_DEBUG(lock,
-				   "client-side enqueue, new policy data");
-	}
-
-	if ((*flags) & LDLM_FL_AST_SENT) {
-		lock_res_and_lock(lock);
-		lock->l_flags |= LDLM_FL_CBPENDING |  LDLM_FL_BL_AST;
-		unlock_res_and_lock(lock);
-		LDLM_DEBUG(lock, "enqueue reply includes blocking AST");
-	}
-
-	/* If the lock has already been granted by a completion AST, don't
-	 * clobber the LVB with an older one.
-	 */
-	if (lvb_len > 0) {
-		/* We must lock or a racing completion might update lvb without
-		 * letting us know and we'll clobber the correct value.
-		 * Cannot unlock after the check either, as that still leaves
-		 * a tiny window for completion to get in
-		 */
-		lock_res_and_lock(lock);
-		if (lock->l_req_mode != lock->l_granted_mode)
-			rc = ldlm_fill_lvb(lock, &req->rq_pill, RCL_SERVER,
-					   lock->l_lvb_data, lvb_len);
-		unlock_res_and_lock(lock);
-		if (rc < 0) {
-			cleanup_phase = 1;
-			goto cleanup;
-		}
-	}
-
-	if (!is_replay) {
-		rc = ldlm_lock_enqueue(ns, &lock, NULL, flags);
-		if (lock->l_completion_ast) {
-			int err = lock->l_completion_ast(lock, *flags, NULL);
-
-			if (!rc)
-				rc = err;
-			if (rc)
-				cleanup_phase = 1;
-		}
-	}
-
-	if (lvb_len > 0 && lvb) {
-		/* Copy the LVB here, and not earlier, because the completion
-		 * AST (if any) can override what we got in the reply
-		 */
-		memcpy(lvb, lock->l_lvb_data, lvb_len);
-	}
-
-	LDLM_DEBUG(lock, "client-side enqueue END");
-cleanup:
-	if (cleanup_phase == 1 && rc)
-		failed_lock_cleanup(ns, lock, mode);
-	/* Put lock 2 times, the second reference is held by ldlm_cli_enqueue */
-	LDLM_LOCK_PUT(lock);
-	LDLM_LOCK_RELEASE(lock);
-	return rc;
-}
-EXPORT_SYMBOL(ldlm_cli_enqueue_fini);
-
-/**
- * Estimate number of lock handles that would fit into request of given
- * size.  PAGE_SIZE-512 is to allow TCP/IP and LNET headers to fit into
- * a single page on the send/receive side. XXX: 512 should be changed to
- * more adequate value.
- */
-static inline int ldlm_req_handles_avail(int req_size, int off)
-{
-	int avail;
-
-	avail = min_t(int, LDLM_MAXREQSIZE, PAGE_SIZE - 512) - req_size;
-	if (likely(avail >= 0))
-		avail /= (int)sizeof(struct lustre_handle);
-	else
-		avail = 0;
-	avail += LDLM_LOCKREQ_HANDLES - off;
-
-	return avail;
-}
-
-static inline int ldlm_capsule_handles_avail(struct req_capsule *pill,
-					     enum req_location loc,
-					     int off)
-{
-	u32 size = req_capsule_msg_size(pill, loc);
-
-	return ldlm_req_handles_avail(size, off);
-}
-
-static inline int ldlm_format_handles_avail(struct obd_import *imp,
-					    const struct req_format *fmt,
-					    enum req_location loc, int off)
-{
-	u32 size = req_capsule_fmt_size(imp->imp_msg_magic, fmt, loc);
-
-	return ldlm_req_handles_avail(size, off);
-}
-
-/**
- * Cancel LRU locks and pack them into the enqueue request. Pack there the given
- * \a count locks in \a cancels.
- *
- * This is to be called by functions preparing their own requests that
- * might contain lists of locks to cancel in addition to actual operation
- * that needs to be performed.
- */
-int ldlm_prep_elc_req(struct obd_export *exp, struct ptlrpc_request *req,
-		      int version, int opc, int canceloff,
-		      struct list_head *cancels, int count)
-{
-	struct ldlm_namespace   *ns = exp->exp_obd->obd_namespace;
-	struct req_capsule      *pill = &req->rq_pill;
-	struct ldlm_request     *dlm = NULL;
-	int flags, avail, to_free, pack = 0;
-	LIST_HEAD(head);
-	int rc;
-
-	if (!cancels)
-		cancels = &head;
-	if (ns_connect_cancelset(ns)) {
-		/* Estimate the amount of available space in the request. */
-		req_capsule_filled_sizes(pill, RCL_CLIENT);
-		avail = ldlm_capsule_handles_avail(pill, RCL_CLIENT, canceloff);
-
-		flags = ns_connect_lru_resize(ns) ?
-			LDLM_LRU_FLAG_LRUR_NO_WAIT : LDLM_LRU_FLAG_AGED;
-		to_free = !ns_connect_lru_resize(ns) &&
-			  opc == LDLM_ENQUEUE ? 1 : 0;
-
-		/* Cancel LRU locks here _only_ if the server supports
-		 * EARLY_CANCEL. Otherwise we have to send extra CANCEL
-		 * RPC, which will make us slower.
-		 */
-		if (avail > count)
-			count += ldlm_cancel_lru_local(ns, cancels, to_free,
-						       avail - count, 0, flags);
-		if (avail > count)
-			pack = count;
-		else
-			pack = avail;
-		req_capsule_set_size(pill, &RMF_DLM_REQ, RCL_CLIENT,
-				     ldlm_request_bufsize(pack, opc));
-	}
-
-	rc = ptlrpc_request_pack(req, version, opc);
-	if (rc) {
-		ldlm_lock_list_put(cancels, l_bl_ast, count);
-		return rc;
-	}
-
-	if (ns_connect_cancelset(ns)) {
-		if (canceloff) {
-			dlm = req_capsule_client_get(pill, &RMF_DLM_REQ);
-			LASSERT(dlm);
-			/* Skip first lock handler in ldlm_request_pack(),
-			 * this method will increment @lock_count according
-			 * to the lock handle amount actually written to
-			 * the buffer.
-			 */
-			dlm->lock_count = canceloff;
-		}
-		/* Pack into the request @pack lock handles. */
-		ldlm_cli_cancel_list(cancels, pack, req, 0);
-		/* Prepare and send separate cancel RPC for others. */
-		ldlm_cli_cancel_list(cancels, count - pack, NULL, 0);
-	} else {
-		ldlm_lock_list_put(cancels, l_bl_ast, count);
-	}
-	return 0;
-}
-EXPORT_SYMBOL(ldlm_prep_elc_req);
-
-int ldlm_prep_enqueue_req(struct obd_export *exp, struct ptlrpc_request *req,
-			  struct list_head *cancels, int count)
-{
-	return ldlm_prep_elc_req(exp, req, LUSTRE_DLM_VERSION, LDLM_ENQUEUE,
-				 LDLM_ENQUEUE_CANCEL_OFF, cancels, count);
-}
-EXPORT_SYMBOL(ldlm_prep_enqueue_req);
-
-static struct ptlrpc_request *ldlm_enqueue_pack(struct obd_export *exp,
-						int lvb_len)
-{
-	struct ptlrpc_request *req;
-	int rc;
-
-	req = ptlrpc_request_alloc(class_exp2cliimp(exp), &RQF_LDLM_ENQUEUE);
-	if (!req)
-		return ERR_PTR(-ENOMEM);
-
-	rc = ldlm_prep_enqueue_req(exp, req, NULL, 0);
-	if (rc) {
-		ptlrpc_request_free(req);
-		return ERR_PTR(rc);
-	}
-
-	req_capsule_set_size(&req->rq_pill, &RMF_DLM_LVB, RCL_SERVER, lvb_len);
-	ptlrpc_request_set_replen(req);
-	return req;
-}
-
-/**
- * Client-side lock enqueue.
- *
- * If a request has some specific initialisation it is passed in \a reqp,
- * otherwise it is created in ldlm_cli_enqueue.
- *
- * Supports sync and async requests, pass \a async flag accordingly. If a
- * request was created in ldlm_cli_enqueue and it is the async request,
- * pass it to the caller in \a reqp.
- */
-int ldlm_cli_enqueue(struct obd_export *exp, struct ptlrpc_request **reqp,
-		     struct ldlm_enqueue_info *einfo,
-		     const struct ldlm_res_id *res_id,
-		     union ldlm_policy_data const *policy, __u64 *flags,
-		     void *lvb, __u32 lvb_len, enum lvb_type lvb_type,
-		     struct lustre_handle *lockh, int async)
-{
-	struct ldlm_namespace *ns;
-	struct ldlm_lock      *lock;
-	struct ldlm_request   *body;
-	int		    is_replay = *flags & LDLM_FL_REPLAY;
-	int		    req_passed_in = 1;
-	int		    rc, err;
-	struct ptlrpc_request *req;
-
-	ns = exp->exp_obd->obd_namespace;
-
-	/* If we're replaying this lock, just check some invariants.
-	 * If we're creating a new lock, get everything all setup nicely.
-	 */
-	if (is_replay) {
-		lock = ldlm_handle2lock_long(lockh, 0);
-		LASSERT(lock);
-		LDLM_DEBUG(lock, "client-side enqueue START");
-		LASSERT(exp == lock->l_conn_export);
-	} else {
-		const struct ldlm_callback_suite cbs = {
-			.lcs_completion = einfo->ei_cb_cp,
-			.lcs_blocking	= einfo->ei_cb_bl,
-			.lcs_glimpse	= einfo->ei_cb_gl
-		};
-		lock = ldlm_lock_create(ns, res_id, einfo->ei_type,
-					einfo->ei_mode, &cbs, einfo->ei_cbdata,
-					lvb_len, lvb_type);
-		if (IS_ERR(lock))
-			return PTR_ERR(lock);
-		/* for the local lock, add the reference */
-		ldlm_lock_addref_internal(lock, einfo->ei_mode);
-		ldlm_lock2handle(lock, lockh);
-		if (policy)
-			lock->l_policy_data = *policy;
-
-		if (einfo->ei_type == LDLM_EXTENT) {
-			/* extent lock without policy is a bug */
-			if (!policy)
-				LBUG();
-
-			lock->l_req_extent = policy->l_extent;
-		}
-		LDLM_DEBUG(lock, "client-side enqueue START, flags %llx",
-			   *flags);
-	}
-
-	lock->l_conn_export = exp;
-	lock->l_export = NULL;
-	lock->l_blocking_ast = einfo->ei_cb_bl;
-	lock->l_flags |= (*flags & (LDLM_FL_NO_LRU | LDLM_FL_EXCL));
-	lock->l_last_activity = ktime_get_real_seconds();
-
-	/* lock not sent to server yet */
-	if (!reqp || !*reqp) {
-		req = ldlm_enqueue_pack(exp, lvb_len);
-		if (IS_ERR(req)) {
-			failed_lock_cleanup(ns, lock, einfo->ei_mode);
-			LDLM_LOCK_RELEASE(lock);
-			return PTR_ERR(req);
-		}
-
-		req_passed_in = 0;
-		if (reqp)
-			*reqp = req;
-	} else {
-		int len;
-
-		req = *reqp;
-		len = req_capsule_get_size(&req->rq_pill, &RMF_DLM_REQ,
-					   RCL_CLIENT);
-		LASSERTF(len >= sizeof(*body), "buflen[%d] = %d, not %d\n",
-			 DLM_LOCKREQ_OFF, len, (int)sizeof(*body));
-	}
-
-	/* Dump lock data into the request buffer */
-	body = req_capsule_client_get(&req->rq_pill, &RMF_DLM_REQ);
-	ldlm_lock2desc(lock, &body->lock_desc);
-	body->lock_flags = ldlm_flags_to_wire(*flags);
-	body->lock_handle[0] = *lockh;
-
-	if (async) {
-		LASSERT(reqp);
-		return 0;
-	}
-
-	LDLM_DEBUG(lock, "sending request");
-
-	rc = ptlrpc_queue_wait(req);
-
-	err = ldlm_cli_enqueue_fini(exp, req, einfo->ei_type, policy ? 1 : 0,
-				    einfo->ei_mode, flags, lvb, lvb_len,
-				    lockh, rc);
-
-	/* If ldlm_cli_enqueue_fini did not find the lock, we need to free
-	 * one reference that we took
-	 */
-	if (err == -ENOLCK)
-		LDLM_LOCK_RELEASE(lock);
-	else
-		rc = err;
-
-	if (!req_passed_in && req) {
-		ptlrpc_req_finished(req);
-		if (reqp)
-			*reqp = NULL;
-	}
-
-	return rc;
-}
-EXPORT_SYMBOL(ldlm_cli_enqueue);
-
-/**
- * Cancel locks locally.
- * Returns:
- * \retval LDLM_FL_LOCAL_ONLY if there is no need for a CANCEL RPC to the server
- * \retval LDLM_FL_CANCELING otherwise;
- * \retval LDLM_FL_BL_AST if there is a need for a separate CANCEL RPC.
- */
-static __u64 ldlm_cli_cancel_local(struct ldlm_lock *lock)
-{
-	__u64 rc = LDLM_FL_LOCAL_ONLY;
-
-	if (lock->l_conn_export) {
-		bool local_only;
-
-		LDLM_DEBUG(lock, "client-side cancel");
-		/* Set this flag to prevent others from getting new references*/
-		lock_res_and_lock(lock);
-		ldlm_set_cbpending(lock);
-		local_only = !!(lock->l_flags &
-				(LDLM_FL_LOCAL_ONLY | LDLM_FL_CANCEL_ON_BLOCK));
-		ldlm_cancel_callback(lock);
-		rc = ldlm_is_bl_ast(lock) ? LDLM_FL_BL_AST : LDLM_FL_CANCELING;
-		unlock_res_and_lock(lock);
-
-		if (local_only) {
-			CDEBUG(D_DLMTRACE,
-			       "not sending request (at caller's instruction)\n");
-			rc = LDLM_FL_LOCAL_ONLY;
-		}
-		ldlm_lock_cancel(lock);
-	} else {
-		LDLM_ERROR(lock, "Trying to cancel local lock");
-		LBUG();
-	}
-
-	return rc;
-}
-
-/**
- * Pack \a count locks in \a head into ldlm_request buffer of request \a req.
- */
-static void ldlm_cancel_pack(struct ptlrpc_request *req,
-			     struct list_head *head, int count)
-{
-	struct ldlm_request *dlm;
-	struct ldlm_lock *lock;
-	int max, packed = 0;
-
-	dlm = req_capsule_client_get(&req->rq_pill, &RMF_DLM_REQ);
-	LASSERT(dlm);
-
-	/* Check the room in the request buffer. */
-	max = req_capsule_get_size(&req->rq_pill, &RMF_DLM_REQ, RCL_CLIENT) -
-		sizeof(struct ldlm_request);
-	max /= sizeof(struct lustre_handle);
-	max += LDLM_LOCKREQ_HANDLES;
-	LASSERT(max >= dlm->lock_count + count);
-
-	/* XXX: it would be better to pack lock handles grouped by resource.
-	 * so that the server cancel would call filter_lvbo_update() less
-	 * frequently.
-	 */
-	list_for_each_entry(lock, head, l_bl_ast) {
-		if (!count--)
-			break;
-		LASSERT(lock->l_conn_export);
-		/* Pack the lock handle to the given request buffer. */
-		LDLM_DEBUG(lock, "packing");
-		dlm->lock_handle[dlm->lock_count++] = lock->l_remote_handle;
-		packed++;
-	}
-	CDEBUG(D_DLMTRACE, "%d locks packed\n", packed);
-}
-
-/**
- * Prepare and send a batched cancel RPC. It will include \a count lock
- * handles of locks given in \a cancels list.
- */
-static int ldlm_cli_cancel_req(struct obd_export *exp,
-			       struct list_head *cancels,
-			       int count, enum ldlm_cancel_flags flags)
-{
-	struct ptlrpc_request *req = NULL;
-	struct obd_import *imp;
-	int free, sent = 0;
-	int rc = 0;
-
-	LASSERT(exp);
-	LASSERT(count > 0);
-
-	CFS_FAIL_TIMEOUT(OBD_FAIL_LDLM_PAUSE_CANCEL, cfs_fail_val);
-
-	if (CFS_FAIL_CHECK(OBD_FAIL_LDLM_CANCEL_RACE))
-		return count;
-
-	free = ldlm_format_handles_avail(class_exp2cliimp(exp),
-					 &RQF_LDLM_CANCEL, RCL_CLIENT, 0);
-	if (count > free)
-		count = free;
-
-	while (1) {
-		imp = class_exp2cliimp(exp);
-		if (!imp || imp->imp_invalid) {
-			CDEBUG(D_DLMTRACE,
-			       "skipping cancel on invalid import %p\n", imp);
-			return count;
-		}
-
-		req = ptlrpc_request_alloc(imp, &RQF_LDLM_CANCEL);
-		if (!req) {
-			rc = -ENOMEM;
-			goto out;
-		}
-
-		req_capsule_filled_sizes(&req->rq_pill, RCL_CLIENT);
-		req_capsule_set_size(&req->rq_pill, &RMF_DLM_REQ, RCL_CLIENT,
-				     ldlm_request_bufsize(count, LDLM_CANCEL));
-
-		rc = ptlrpc_request_pack(req, LUSTRE_DLM_VERSION, LDLM_CANCEL);
-		if (rc) {
-			ptlrpc_request_free(req);
-			goto out;
-		}
-
-		req->rq_request_portal = LDLM_CANCEL_REQUEST_PORTAL;
-		req->rq_reply_portal = LDLM_CANCEL_REPLY_PORTAL;
-		ptlrpc_at_set_req_timeout(req);
-
-		ldlm_cancel_pack(req, cancels, count);
-
-		ptlrpc_request_set_replen(req);
-		if (flags & LCF_ASYNC) {
-			ptlrpcd_add_req(req);
-			sent = count;
-			goto out;
-		}
-
-		rc = ptlrpc_queue_wait(req);
-		if (rc == LUSTRE_ESTALE) {
-			CDEBUG(D_DLMTRACE,
-			       "client/server (nid %s) out of sync -- not fatal\n",
-			       libcfs_nid2str(req->rq_import->
-					      imp_connection->c_peer.nid));
-			rc = 0;
-		} else if (rc == -ETIMEDOUT && /* check there was no reconnect*/
-			   req->rq_import_generation == imp->imp_generation) {
-			ptlrpc_req_finished(req);
-			continue;
-		} else if (rc != ELDLM_OK) {
-			/* -ESHUTDOWN is common on umount */
-			CDEBUG_LIMIT(rc == -ESHUTDOWN ? D_DLMTRACE : D_ERROR,
-				     "Got rc %d from cancel RPC: canceling anyway\n",
-				     rc);
-			break;
-		}
-		sent = count;
-		break;
-	}
-
-	ptlrpc_req_finished(req);
-out:
-	return sent ? sent : rc;
-}
-
-static inline struct ldlm_pool *ldlm_imp2pl(struct obd_import *imp)
-{
-	return &imp->imp_obd->obd_namespace->ns_pool;
-}
-
-/**
- * Update client's OBD pool related fields with new SLV and Limit from \a req.
- */
-int ldlm_cli_update_pool(struct ptlrpc_request *req)
-{
-	struct obd_device *obd;
-	__u64 new_slv;
-	__u32 new_limit;
-
-	if (unlikely(!req->rq_import || !req->rq_import->imp_obd ||
-		     !imp_connect_lru_resize(req->rq_import))) {
-		/*
-		 * Do nothing for corner cases.
-		 */
-		return 0;
-	}
-
-	/* In some cases RPC may contain SLV and limit zeroed out. This
-	 * is the case when server does not support LRU resize feature.
-	 * This is also possible in some recovery cases when server-side
-	 * reqs have no reference to the OBD export and thus access to
-	 * server-side namespace is not possible.
-	 */
-	if (lustre_msg_get_slv(req->rq_repmsg) == 0 ||
-	    lustre_msg_get_limit(req->rq_repmsg) == 0) {
-		DEBUG_REQ(D_HA, req,
-			  "Zero SLV or Limit found (SLV: %llu, Limit: %u)",
-			  lustre_msg_get_slv(req->rq_repmsg),
-			  lustre_msg_get_limit(req->rq_repmsg));
-		return 0;
-	}
-
-	new_limit = lustre_msg_get_limit(req->rq_repmsg);
-	new_slv = lustre_msg_get_slv(req->rq_repmsg);
-	obd = req->rq_import->imp_obd;
-
-	/* Set new SLV and limit in OBD fields to make them accessible
-	 * to the pool thread. We do not access obd_namespace and pool
-	 * directly here as there is no reliable way to make sure that
-	 * they are still alive at cleanup time. Evil races are possible
-	 * which may cause Oops at that time.
-	 */
-	write_lock(&obd->obd_pool_lock);
-	obd->obd_pool_slv = new_slv;
-	obd->obd_pool_limit = new_limit;
-	write_unlock(&obd->obd_pool_lock);
-
-	return 0;
-}
-
-/**
- * Client side lock cancel.
- *
- * Lock must not have any readers or writers by this time.
- */
-int ldlm_cli_cancel(const struct lustre_handle *lockh,
-		    enum ldlm_cancel_flags cancel_flags)
-{
-	struct obd_export *exp;
-	int avail, flags, count = 1;
-	__u64 rc = 0;
-	struct ldlm_namespace *ns;
-	struct ldlm_lock *lock;
-	LIST_HEAD(cancels);
-
-	lock = ldlm_handle2lock_long(lockh, 0);
-	if (!lock) {
-		LDLM_DEBUG_NOLOCK("lock is already being destroyed");
-		return 0;
-	}
-
-	lock_res_and_lock(lock);
-	/* Lock is being canceled and the caller doesn't want to wait */
-	if (ldlm_is_canceling(lock) && (cancel_flags & LCF_ASYNC)) {
-		unlock_res_and_lock(lock);
-		LDLM_LOCK_RELEASE(lock);
-		return 0;
-	}
-
-	ldlm_set_canceling(lock);
-	unlock_res_and_lock(lock);
-
-	rc = ldlm_cli_cancel_local(lock);
-	if (rc == LDLM_FL_LOCAL_ONLY || cancel_flags & LCF_LOCAL) {
-		LDLM_LOCK_RELEASE(lock);
-		return 0;
-	}
-	/* Even if the lock is marked as LDLM_FL_BL_AST, this is a LDLM_CANCEL
-	 * RPC which goes to canceld portal, so we can cancel other LRU locks
-	 * here and send them all as one LDLM_CANCEL RPC.
-	 */
-	LASSERT(list_empty(&lock->l_bl_ast));
-	list_add(&lock->l_bl_ast, &cancels);
-
-	exp = lock->l_conn_export;
-	if (exp_connect_cancelset(exp)) {
-		avail = ldlm_format_handles_avail(class_exp2cliimp(exp),
-						  &RQF_LDLM_CANCEL,
-						  RCL_CLIENT, 0);
-		LASSERT(avail > 0);
-
-		ns = ldlm_lock_to_ns(lock);
-		flags = ns_connect_lru_resize(ns) ?
-			LDLM_LRU_FLAG_LRUR : LDLM_LRU_FLAG_AGED;
-		count += ldlm_cancel_lru_local(ns, &cancels, 0, avail - 1,
-					       LCF_BL_AST, flags);
-	}
-	ldlm_cli_cancel_list(&cancels, count, NULL, cancel_flags);
-	return 0;
-}
-EXPORT_SYMBOL(ldlm_cli_cancel);
-
-/**
- * Locally cancel up to \a count locks in list \a cancels.
- * Return the number of cancelled locks.
- */
-int ldlm_cli_cancel_list_local(struct list_head *cancels, int count,
-			       enum ldlm_cancel_flags flags)
-{
-	LIST_HEAD(head);
-	struct ldlm_lock *lock, *next;
-	int left = 0, bl_ast = 0;
-	__u64 rc;
-
-	left = count;
-	list_for_each_entry_safe(lock, next, cancels, l_bl_ast) {
-		if (left-- == 0)
-			break;
-
-		if (flags & LCF_LOCAL) {
-			rc = LDLM_FL_LOCAL_ONLY;
-			ldlm_lock_cancel(lock);
-		} else {
-			rc = ldlm_cli_cancel_local(lock);
-		}
-		/* Until we have compound requests and can send LDLM_CANCEL
-		 * requests batched with generic RPCs, we need to send cancels
-		 * with the LDLM_FL_BL_AST flag in a separate RPC from
-		 * the one being generated now.
-		 */
-		if (!(flags & LCF_BL_AST) && (rc == LDLM_FL_BL_AST)) {
-			LDLM_DEBUG(lock, "Cancel lock separately");
-			list_del_init(&lock->l_bl_ast);
-			list_add(&lock->l_bl_ast, &head);
-			bl_ast++;
-			continue;
-		}
-		if (rc == LDLM_FL_LOCAL_ONLY) {
-			/* CANCEL RPC should not be sent to server. */
-			list_del_init(&lock->l_bl_ast);
-			LDLM_LOCK_RELEASE(lock);
-			count--;
-		}
-	}
-	if (bl_ast > 0) {
-		count -= bl_ast;
-		ldlm_cli_cancel_list(&head, bl_ast, NULL, 0);
-	}
-
-	return count;
-}
-
-/**
- * Cancel as many locks as possible w/o sending any RPCs (e.g. to write back
- * dirty data, to close a file, ...) or waiting for any RPCs in-flight (e.g.
- * readahead requests, ...)
- */
-static enum ldlm_policy_res
-ldlm_cancel_no_wait_policy(struct ldlm_namespace *ns, struct ldlm_lock *lock,
-			   int unused, int added, int count)
-{
-	enum ldlm_policy_res result = LDLM_POLICY_CANCEL_LOCK;
-
-	/* don't check added & count since we want to process all locks
-	 * from unused list.
-	 * It's fine to not take lock to access lock->l_resource since
-	 * the lock has already been granted so it won't change.
-	 */
-	switch (lock->l_resource->lr_type) {
-	case LDLM_EXTENT:
-	case LDLM_IBITS:
-		if (ns->ns_cancel && ns->ns_cancel(lock) != 0)
-			break;
-		/* fall through */
-	default:
-		result = LDLM_POLICY_SKIP_LOCK;
-		lock_res_and_lock(lock);
-		ldlm_set_skipped(lock);
-		unlock_res_and_lock(lock);
-		break;
-	}
-
-	return result;
-}
-
-/**
- * Callback function for LRU-resize policy. Decides whether to keep
- * \a lock in LRU for current \a LRU size \a unused, added in current
- * scan \a added and number of locks to be preferably canceled \a count.
- *
- * \retval LDLM_POLICY_KEEP_LOCK keep lock in LRU in stop scanning
- *
- * \retval LDLM_POLICY_CANCEL_LOCK cancel lock from LRU
- */
-static enum ldlm_policy_res ldlm_cancel_lrur_policy(struct ldlm_namespace *ns,
-						    struct ldlm_lock *lock,
-						    int unused, int added,
-						    int count)
-{
-	unsigned long cur = jiffies;
-	struct ldlm_pool *pl = &ns->ns_pool;
-	__u64 slv, lvf, lv;
-	unsigned long la;
-
-	/* Stop LRU processing when we reach past @count or have checked all
-	 * locks in LRU.
-	 */
-	if (count && added >= count)
-		return LDLM_POLICY_KEEP_LOCK;
-
-	/*
-	 * Despite of the LV, It doesn't make sense to keep the lock which
-	 * is unused for ns_max_age time.
-	 */
-	if (time_after(jiffies, lock->l_last_used + ns->ns_max_age))
-		return LDLM_POLICY_CANCEL_LOCK;
-
-	slv = ldlm_pool_get_slv(pl);
-	lvf = ldlm_pool_get_lvf(pl);
-	la = (cur - lock->l_last_used) / HZ;
-	lv = lvf * la * unused;
-
-	/* Inform pool about current CLV to see it via debugfs. */
-	ldlm_pool_set_clv(pl, lv);
-
-	/* Stop when SLV is not yet come from server or lv is smaller than
-	 * it is.
-	 */
-	if (slv == 0 || lv < slv)
-		return LDLM_POLICY_KEEP_LOCK;
-
-	return LDLM_POLICY_CANCEL_LOCK;
-}
-
-/**
- * Callback function for debugfs used policy. Makes decision whether to keep
- * \a lock in LRU for current \a LRU size \a unused, added in current scan \a
- * added and number of locks to be preferably canceled \a count.
- *
- * \retval LDLM_POLICY_KEEP_LOCK keep lock in LRU in stop scanning
- *
- * \retval LDLM_POLICY_CANCEL_LOCK cancel lock from LRU
- */
-static enum ldlm_policy_res ldlm_cancel_passed_policy(struct ldlm_namespace *ns,
-						      struct ldlm_lock *lock,
-						      int unused, int added,
-						      int count)
-{
-	/* Stop LRU processing when we reach past @count or have checked all
-	 * locks in LRU.
-	 */
-	return (added >= count) ?
-		LDLM_POLICY_KEEP_LOCK : LDLM_POLICY_CANCEL_LOCK;
-}
-
-/**
- * Callback function for aged policy. Makes decision whether to keep \a lock in
- * LRU for current LRU size \a unused, added in current scan \a added and
- * number of locks to be preferably canceled \a count.
- *
- * \retval LDLM_POLICY_KEEP_LOCK keep lock in LRU in stop scanning
- *
- * \retval LDLM_POLICY_CANCEL_LOCK cancel lock from LRU
- */
-static enum ldlm_policy_res ldlm_cancel_aged_policy(struct ldlm_namespace *ns,
-						    struct ldlm_lock *lock,
-						    int unused, int added,
-						    int count)
-{
-	if ((added >= count) &&
-	    time_before(jiffies, lock->l_last_used + ns->ns_max_age))
-		return LDLM_POLICY_KEEP_LOCK;
-
-	return LDLM_POLICY_CANCEL_LOCK;
-}
-
-static enum ldlm_policy_res
-ldlm_cancel_lrur_no_wait_policy(struct ldlm_namespace *ns,
-				struct ldlm_lock *lock,
-				int unused, int added,
-				int count)
-{
-	enum ldlm_policy_res result;
-
-	result = ldlm_cancel_lrur_policy(ns, lock, unused, added, count);
-	if (result == LDLM_POLICY_KEEP_LOCK)
-		return result;
-
-	return ldlm_cancel_no_wait_policy(ns, lock, unused, added, count);
-}
-
-/**
- * Callback function for default policy. Makes decision whether to keep \a lock
- * in LRU for current LRU size \a unused, added in current scan \a added and
- * number of locks to be preferably canceled \a count.
- *
- * \retval LDLM_POLICY_KEEP_LOCK keep lock in LRU in stop scanning
- *
- * \retval LDLM_POLICY_CANCEL_LOCK cancel lock from LRU
- */
-static enum ldlm_policy_res
-ldlm_cancel_default_policy(struct ldlm_namespace *ns, struct ldlm_lock *lock,
-			   int unused, int added, int count)
-{
-	/* Stop LRU processing when we reach past count or have checked all
-	 * locks in LRU.
-	 */
-	return (added >= count) ?
-		LDLM_POLICY_KEEP_LOCK : LDLM_POLICY_CANCEL_LOCK;
-}
-
-typedef enum ldlm_policy_res (*ldlm_cancel_lru_policy_t)(
-						      struct ldlm_namespace *,
-						      struct ldlm_lock *, int,
-						      int, int);
-
-static ldlm_cancel_lru_policy_t
-ldlm_cancel_lru_policy(struct ldlm_namespace *ns, int flags)
-{
-	if (flags & LDLM_LRU_FLAG_NO_WAIT)
-		return ldlm_cancel_no_wait_policy;
-
-	if (ns_connect_lru_resize(ns)) {
-		if (flags & LDLM_LRU_FLAG_SHRINK)
-			/* We kill passed number of old locks. */
-			return ldlm_cancel_passed_policy;
-		else if (flags & LDLM_LRU_FLAG_LRUR)
-			return ldlm_cancel_lrur_policy;
-		else if (flags & LDLM_LRU_FLAG_PASSED)
-			return ldlm_cancel_passed_policy;
-		else if (flags & LDLM_LRU_FLAG_LRUR_NO_WAIT)
-			return ldlm_cancel_lrur_no_wait_policy;
-	} else {
-		if (flags & LDLM_LRU_FLAG_AGED)
-			return ldlm_cancel_aged_policy;
-	}
-
-	return ldlm_cancel_default_policy;
-}
-
-/**
- * - Free space in LRU for \a count new locks,
- *   redundant unused locks are canceled locally;
- * - also cancel locally unused aged locks;
- * - do not cancel more than \a max locks;
- * - GET the found locks and add them into the \a cancels list.
- *
- * A client lock can be added to the l_bl_ast list only when it is
- * marked LDLM_FL_CANCELING. Otherwise, somebody is already doing
- * CANCEL.  There are the following use cases:
- * ldlm_cancel_resource_local(), ldlm_cancel_lru_local() and
- * ldlm_cli_cancel(), which check and set this flag properly. As any
- * attempt to cancel a lock rely on this flag, l_bl_ast list is accessed
- * later without any special locking.
- *
- * Calling policies for enabled LRU resize:
- * ----------------------------------------
- * flags & LDLM_LRU_FLAG_LRUR	- use LRU resize policy (SLV from server) to
- *				  cancel not more than \a count locks;
- *
- * flags & LDLM_LRU_FLAG_PASSED - cancel \a count number of old locks (located
- *				  at the beginning of LRU list);
- *
- * flags & LDLM_LRU_FLAG_SHRINK - cancel not more than \a count locks according
- *				  to memory pressure policy function;
- *
- * flags & LDLM_LRU_FLAG_AGED   - cancel \a count locks according to
- *				  "aged policy".
- *
- * flags & LDLM_LRU_FLAG_NO_WAIT - cancel as many unused locks as possible
- *				   (typically before replaying locks) w/o
- *				   sending any RPCs or waiting for any
- *				   outstanding RPC to complete.
- */
-static int ldlm_prepare_lru_list(struct ldlm_namespace *ns,
-				 struct list_head *cancels, int count, int max,
-				 int flags)
-{
-	ldlm_cancel_lru_policy_t pf;
-	struct ldlm_lock *lock, *next;
-	int added = 0, unused, remained;
-	int no_wait = flags &
-		(LDLM_LRU_FLAG_NO_WAIT | LDLM_LRU_FLAG_LRUR_NO_WAIT);
-
-	spin_lock(&ns->ns_lock);
-	unused = ns->ns_nr_unused;
-	remained = unused;
-
-	if (!ns_connect_lru_resize(ns))
-		count += unused - ns->ns_max_unused;
-
-	pf = ldlm_cancel_lru_policy(ns, flags);
-	LASSERT(pf);
-
-	while (!list_empty(&ns->ns_unused_list)) {
-		enum ldlm_policy_res result;
-		time_t last_use = 0;
-
-		/* all unused locks */
-		if (remained-- <= 0)
-			break;
-
-		/* For any flags, stop scanning if @max is reached. */
-		if (max && added >= max)
-			break;
-
-		list_for_each_entry_safe(lock, next, &ns->ns_unused_list,
-					 l_lru) {
-			/* No locks which got blocking requests. */
-			LASSERT(!ldlm_is_bl_ast(lock));
-
-			if (no_wait && ldlm_is_skipped(lock))
-				/* already processed */
-				continue;
-
-			last_use = lock->l_last_used;
-			if (last_use == jiffies)
-				continue;
-
-			/* Somebody is already doing CANCEL. No need for this
-			 * lock in LRU, do not traverse it again.
-			 */
-			if (!ldlm_is_canceling(lock))
-				break;
-
-			ldlm_lock_remove_from_lru_nolock(lock);
-		}
-		if (&lock->l_lru == &ns->ns_unused_list)
-			break;
-
-		LDLM_LOCK_GET(lock);
-		spin_unlock(&ns->ns_lock);
-		lu_ref_add(&lock->l_reference, __func__, current);
-
-		/* Pass the lock through the policy filter and see if it
-		 * should stay in LRU.
-		 *
-		 * Even for shrinker policy we stop scanning if
-		 * we find a lock that should stay in the cache.
-		 * We should take into account lock age anyway
-		 * as a new lock is a valuable resource even if
-		 * it has a low weight.
-		 *
-		 * That is, for shrinker policy we drop only
-		 * old locks, but additionally choose them by
-		 * their weight. Big extent locks will stay in
-		 * the cache.
-		 */
-		result = pf(ns, lock, unused, added, count);
-		if (result == LDLM_POLICY_KEEP_LOCK) {
-			lu_ref_del(&lock->l_reference,
-				   __func__, current);
-			LDLM_LOCK_RELEASE(lock);
-			spin_lock(&ns->ns_lock);
-			break;
-		}
-		if (result == LDLM_POLICY_SKIP_LOCK) {
-			lu_ref_del(&lock->l_reference,
-				   __func__, current);
-			LDLM_LOCK_RELEASE(lock);
-			spin_lock(&ns->ns_lock);
-			continue;
-		}
-
-		lock_res_and_lock(lock);
-		/* Check flags again under the lock. */
-		if (ldlm_is_canceling(lock) ||
-		    (ldlm_lock_remove_from_lru_check(lock, last_use) == 0)) {
-			/* Another thread is removing lock from LRU, or
-			 * somebody is already doing CANCEL, or there
-			 * is a blocking request which will send cancel
-			 * by itself, or the lock is no longer unused or
-			 * the lock has been used since the pf() call and
-			 * pages could be put under it.
-			 */
-			unlock_res_and_lock(lock);
-			lu_ref_del(&lock->l_reference,
-				   __func__, current);
-			LDLM_LOCK_RELEASE(lock);
-			spin_lock(&ns->ns_lock);
-			continue;
-		}
-		LASSERT(!lock->l_readers && !lock->l_writers);
-
-		/* If we have chosen to cancel this lock voluntarily, we
-		 * better send cancel notification to server, so that it
-		 * frees appropriate state. This might lead to a race
-		 * where while we are doing cancel here, server is also
-		 * silently cancelling this lock.
-		 */
-		ldlm_clear_cancel_on_block(lock);
-
-		/* Setting the CBPENDING flag is a little misleading,
-		 * but prevents an important race; namely, once
-		 * CBPENDING is set, the lock can accumulate no more
-		 * readers/writers. Since readers and writers are
-		 * already zero here, ldlm_lock_decref() won't see
-		 * this flag and call l_blocking_ast
-		 */
-		lock->l_flags |= LDLM_FL_CBPENDING | LDLM_FL_CANCELING;
-
-		/* We can't re-add to l_lru as it confuses the
-		 * refcounting in ldlm_lock_remove_from_lru() if an AST
-		 * arrives after we drop lr_lock below. We use l_bl_ast
-		 * and can't use l_pending_chain as it is used both on
-		 * server and client nevertheless bug 5666 says it is
-		 * used only on server
-		 */
-		LASSERT(list_empty(&lock->l_bl_ast));
-		list_add(&lock->l_bl_ast, cancels);
-		unlock_res_and_lock(lock);
-		lu_ref_del(&lock->l_reference, __func__, current);
-		spin_lock(&ns->ns_lock);
-		added++;
-		unused--;
-	}
-	spin_unlock(&ns->ns_lock);
-	return added;
-}
-
-int ldlm_cancel_lru_local(struct ldlm_namespace *ns,
-			  struct list_head *cancels, int count, int max,
-			  enum ldlm_cancel_flags cancel_flags, int flags)
-{
-	int added;
-
-	added = ldlm_prepare_lru_list(ns, cancels, count, max, flags);
-	if (added <= 0)
-		return added;
-	return ldlm_cli_cancel_list_local(cancels, added, cancel_flags);
-}
-
-/**
- * Cancel at least \a nr locks from given namespace LRU.
- *
- * When called with LCF_ASYNC the blocking callback will be handled
- * in a thread and this function will return after the thread has been
- * asked to call the callback.  When called with LCF_ASYNC the blocking
- * callback will be performed in this function.
- */
-int ldlm_cancel_lru(struct ldlm_namespace *ns, int nr,
-		    enum ldlm_cancel_flags cancel_flags,
-		    int flags)
-{
-	LIST_HEAD(cancels);
-	int count, rc;
-
-	/* Just prepare the list of locks, do not actually cancel them yet.
-	 * Locks are cancelled later in a separate thread.
-	 */
-	count = ldlm_prepare_lru_list(ns, &cancels, nr, 0, flags);
-	rc = ldlm_bl_to_thread_list(ns, NULL, &cancels, count, cancel_flags);
-	if (rc == 0)
-		return count;
-
-	return 0;
-}
-
-/**
- * Find and cancel locally unused locks found on resource, matched to the
- * given policy, mode. GET the found locks and add them into the \a cancels
- * list.
- */
-int ldlm_cancel_resource_local(struct ldlm_resource *res,
-			       struct list_head *cancels,
-			       union ldlm_policy_data *policy,
-			       enum ldlm_mode mode, __u64 lock_flags,
-			       enum ldlm_cancel_flags cancel_flags,
-			       void *opaque)
-{
-	struct ldlm_lock *lock;
-	int count = 0;
-
-	lock_res(res);
-	list_for_each_entry(lock, &res->lr_granted, l_res_link) {
-		if (opaque && lock->l_ast_data != opaque) {
-			LDLM_ERROR(lock, "data %p doesn't match opaque %p",
-				   lock->l_ast_data, opaque);
-			continue;
-		}
-
-		if (lock->l_readers || lock->l_writers)
-			continue;
-
-		/* If somebody is already doing CANCEL, or blocking AST came,
-		 * skip this lock.
-		 */
-		if (ldlm_is_bl_ast(lock) || ldlm_is_canceling(lock))
-			continue;
-
-		if (lockmode_compat(lock->l_granted_mode, mode))
-			continue;
-
-		/* If policy is given and this is IBITS lock, add to list only
-		 * those locks that match by policy.
-		 */
-		if (policy && (lock->l_resource->lr_type == LDLM_IBITS) &&
-		    !(lock->l_policy_data.l_inodebits.bits &
-		      policy->l_inodebits.bits))
-			continue;
-
-		/* See CBPENDING comment in ldlm_cancel_lru */
-		lock->l_flags |= LDLM_FL_CBPENDING | LDLM_FL_CANCELING |
-				 lock_flags;
-
-		LASSERT(list_empty(&lock->l_bl_ast));
-		list_add(&lock->l_bl_ast, cancels);
-		LDLM_LOCK_GET(lock);
-		count++;
-	}
-	unlock_res(res);
-
-	return ldlm_cli_cancel_list_local(cancels, count, cancel_flags);
-}
-EXPORT_SYMBOL(ldlm_cancel_resource_local);
-
-/**
- * Cancel client-side locks from a list and send/prepare cancel RPCs to the
- * server.
- * If \a req is NULL, send CANCEL request to server with handles of locks
- * in the \a cancels. If EARLY_CANCEL is not supported, send CANCEL requests
- * separately per lock.
- * If \a req is not NULL, put handles of locks in \a cancels into the request
- * buffer at the offset \a off.
- * Destroy \a cancels at the end.
- */
-int ldlm_cli_cancel_list(struct list_head *cancels, int count,
-			 struct ptlrpc_request *req,
-			 enum ldlm_cancel_flags flags)
-{
-	struct ldlm_lock *lock;
-	int res = 0;
-
-	if (list_empty(cancels) || count == 0)
-		return 0;
-
-	/* XXX: requests (both batched and not) could be sent in parallel.
-	 * Usually it is enough to have just 1 RPC, but it is possible that
-	 * there are too many locks to be cancelled in LRU or on a resource.
-	 * It would also speed up the case when the server does not support
-	 * the feature.
-	 */
-	while (count > 0) {
-		LASSERT(!list_empty(cancels));
-		lock = list_first_entry(cancels, struct ldlm_lock, l_bl_ast);
-		LASSERT(lock->l_conn_export);
-
-		if (exp_connect_cancelset(lock->l_conn_export)) {
-			res = count;
-			if (req)
-				ldlm_cancel_pack(req, cancels, count);
-			else
-				res = ldlm_cli_cancel_req(lock->l_conn_export,
-							  cancels, count,
-							  flags);
-		} else {
-			res = ldlm_cli_cancel_req(lock->l_conn_export,
-						  cancels, 1, flags);
-		}
-
-		if (res < 0) {
-			CDEBUG_LIMIT(res == -ESHUTDOWN ? D_DLMTRACE : D_ERROR,
-				     "%s: %d\n", __func__, res);
-			res = count;
-		}
-
-		count -= res;
-		ldlm_lock_list_put(cancels, l_bl_ast, res);
-	}
-	LASSERT(count == 0);
-	return 0;
-}
-EXPORT_SYMBOL(ldlm_cli_cancel_list);
-
-/**
- * Cancel all locks on a resource that have 0 readers/writers.
- *
- * If flags & LDLM_FL_LOCAL_ONLY, throw the locks away without trying
- * to notify the server.
- */
-int ldlm_cli_cancel_unused_resource(struct ldlm_namespace *ns,
-				    const struct ldlm_res_id *res_id,
-				    union ldlm_policy_data *policy,
-				    enum ldlm_mode mode,
-				    enum ldlm_cancel_flags flags,
-				    void *opaque)
-{
-	struct ldlm_resource *res;
-	LIST_HEAD(cancels);
-	int count;
-	int rc;
-
-	res = ldlm_resource_get(ns, NULL, res_id, 0, 0);
-	if (IS_ERR(res)) {
-		/* This is not a problem. */
-		CDEBUG(D_INFO, "No resource %llu\n", res_id->name[0]);
-		return 0;
-	}
-
-	LDLM_RESOURCE_ADDREF(res);
-	count = ldlm_cancel_resource_local(res, &cancels, policy, mode,
-					   0, flags | LCF_BL_AST, opaque);
-	rc = ldlm_cli_cancel_list(&cancels, count, NULL, flags);
-	if (rc != ELDLM_OK)
-		CERROR("canceling unused lock " DLDLMRES ": rc = %d\n",
-		       PLDLMRES(res), rc);
-
-	LDLM_RESOURCE_DELREF(res);
-	ldlm_resource_putref(res);
-	return 0;
-}
-EXPORT_SYMBOL(ldlm_cli_cancel_unused_resource);
-
-struct ldlm_cli_cancel_arg {
-	int     lc_flags;
-	void   *lc_opaque;
-};
-
-static int ldlm_cli_hash_cancel_unused(struct cfs_hash *hs,
-				       struct cfs_hash_bd *bd,
-				       struct hlist_node *hnode, void *arg)
-{
-	struct ldlm_resource	   *res = cfs_hash_object(hs, hnode);
-	struct ldlm_cli_cancel_arg     *lc = arg;
-
-	ldlm_cli_cancel_unused_resource(ldlm_res_to_ns(res), &res->lr_name,
-					NULL, LCK_MINMODE,
-					lc->lc_flags, lc->lc_opaque);
-	/* must return 0 for hash iteration */
-	return 0;
-}
-
-/**
- * Cancel all locks on a namespace (or a specific resource, if given)
- * that have 0 readers/writers.
- *
- * If flags & LCF_LOCAL, throw the locks away without trying
- * to notify the server.
- */
-int ldlm_cli_cancel_unused(struct ldlm_namespace *ns,
-			   const struct ldlm_res_id *res_id,
-			   enum ldlm_cancel_flags flags, void *opaque)
-{
-	struct ldlm_cli_cancel_arg arg = {
-		.lc_flags       = flags,
-		.lc_opaque      = opaque,
-	};
-
-	if (!ns)
-		return ELDLM_OK;
-
-	if (res_id) {
-		return ldlm_cli_cancel_unused_resource(ns, res_id, NULL,
-						       LCK_MINMODE, flags,
-						       opaque);
-	} else {
-		cfs_hash_for_each_nolock(ns->ns_rs_hash,
-					 ldlm_cli_hash_cancel_unused, &arg, 0);
-		return ELDLM_OK;
-	}
-}
-EXPORT_SYMBOL(ldlm_cli_cancel_unused);
-
-/* Lock iterators. */
-
-static int ldlm_resource_foreach(struct ldlm_resource *res,
-				 ldlm_iterator_t iter, void *closure)
-{
-	struct ldlm_lock *tmp;
-	struct ldlm_lock *lock;
-	int rc = LDLM_ITER_CONTINUE;
-
-	if (!res)
-		return LDLM_ITER_CONTINUE;
-
-	lock_res(res);
-	list_for_each_entry_safe(lock, tmp, &res->lr_granted, l_res_link) {
-		if (iter(lock, closure) == LDLM_ITER_STOP) {
-			rc = LDLM_ITER_STOP;
-			goto out;
-		}
-	}
-
-	list_for_each_entry_safe(lock, tmp, &res->lr_waiting, l_res_link) {
-		if (iter(lock, closure) == LDLM_ITER_STOP) {
-			rc = LDLM_ITER_STOP;
-			goto out;
-		}
-	}
- out:
-	unlock_res(res);
-	return rc;
-}
-
-struct iter_helper_data {
-	ldlm_iterator_t iter;
-	void *closure;
-};
-
-static int ldlm_iter_helper(struct ldlm_lock *lock, void *closure)
-{
-	struct iter_helper_data *helper = closure;
-
-	return helper->iter(lock, helper->closure);
-}
-
-static int ldlm_res_iter_helper(struct cfs_hash *hs, struct cfs_hash_bd *bd,
-				struct hlist_node *hnode, void *arg)
-
-{
-	struct ldlm_resource *res = cfs_hash_object(hs, hnode);
-
-	return ldlm_resource_foreach(res, ldlm_iter_helper, arg) ==
-	       LDLM_ITER_STOP;
-}
-
-static void ldlm_namespace_foreach(struct ldlm_namespace *ns,
-				   ldlm_iterator_t iter, void *closure)
-
-{
-	struct iter_helper_data helper = {
-		.iter		= iter,
-		.closure	= closure,
-	};
-
-	cfs_hash_for_each_nolock(ns->ns_rs_hash,
-				 ldlm_res_iter_helper, &helper, 0);
-}
-
-/* non-blocking function to manipulate a lock whose cb_data is being put away.
- * return  0:  find no resource
- *       > 0:  must be LDLM_ITER_STOP/LDLM_ITER_CONTINUE.
- *       < 0:  errors
- */
-int ldlm_resource_iterate(struct ldlm_namespace *ns,
-			  const struct ldlm_res_id *res_id,
-			  ldlm_iterator_t iter, void *data)
-{
-	struct ldlm_resource *res;
-	int rc;
-
-	LASSERTF(ns, "must pass in namespace\n");
-
-	res = ldlm_resource_get(ns, NULL, res_id, 0, 0);
-	if (IS_ERR(res))
-		return 0;
-
-	LDLM_RESOURCE_ADDREF(res);
-	rc = ldlm_resource_foreach(res, iter, data);
-	LDLM_RESOURCE_DELREF(res);
-	ldlm_resource_putref(res);
-	return rc;
-}
-EXPORT_SYMBOL(ldlm_resource_iterate);
-
-/* Lock replay */
-
-static int ldlm_chain_lock_for_replay(struct ldlm_lock *lock, void *closure)
-{
-	struct list_head *list = closure;
-
-	/* we use l_pending_chain here, because it's unused on clients. */
-	LASSERTF(list_empty(&lock->l_pending_chain),
-		 "lock %p next %p prev %p\n",
-		 lock, &lock->l_pending_chain.next,
-		 &lock->l_pending_chain.prev);
-	/* bug 9573: don't replay locks left after eviction, or
-	 * bug 17614: locks being actively cancelled. Get a reference
-	 * on a lock so that it does not disappear under us (e.g. due to cancel)
-	 */
-	if (!(lock->l_flags & (LDLM_FL_FAILED | LDLM_FL_BL_DONE))) {
-		list_add(&lock->l_pending_chain, list);
-		LDLM_LOCK_GET(lock);
-	}
-
-	return LDLM_ITER_CONTINUE;
-}
-
-static int replay_lock_interpret(const struct lu_env *env,
-				 struct ptlrpc_request *req,
-				 struct ldlm_async_args *aa, int rc)
-{
-	struct ldlm_lock     *lock;
-	struct ldlm_reply    *reply;
-	struct obd_export    *exp;
-
-	atomic_dec(&req->rq_import->imp_replay_inflight);
-	if (rc != ELDLM_OK)
-		goto out;
-
-	reply = req_capsule_server_get(&req->rq_pill, &RMF_DLM_REP);
-	if (!reply) {
-		rc = -EPROTO;
-		goto out;
-	}
-
-	lock = ldlm_handle2lock(&aa->lock_handle);
-	if (!lock) {
-		CERROR("received replay ack for unknown local cookie %#llx remote cookie %#llx from server %s id %s\n",
-		       aa->lock_handle.cookie, reply->lock_handle.cookie,
-		       req->rq_export->exp_client_uuid.uuid,
-		       libcfs_id2str(req->rq_peer));
-		rc = -ESTALE;
-		goto out;
-	}
-
-	/* Key change rehash lock in per-export hash with new key */
-	exp = req->rq_export;
-	lock->l_remote_handle = reply->lock_handle;
-
-	LDLM_DEBUG(lock, "replayed lock:");
-	ptlrpc_import_recovery_state_machine(req->rq_import);
-	LDLM_LOCK_PUT(lock);
-out:
-	if (rc != ELDLM_OK)
-		ptlrpc_connect_import(req->rq_import);
-
-	return rc;
-}
-
-static int replay_one_lock(struct obd_import *imp, struct ldlm_lock *lock)
-{
-	struct ptlrpc_request *req;
-	struct ldlm_async_args *aa;
-	struct ldlm_request   *body;
-	int flags;
-
-	/* Bug 11974: Do not replay a lock which is actively being canceled */
-	if (ldlm_is_bl_done(lock)) {
-		LDLM_DEBUG(lock, "Not replaying canceled lock:");
-		return 0;
-	}
-
-	/* If this is reply-less callback lock, we cannot replay it, since
-	 * server might have long dropped it, but notification of that event was
-	 * lost by network. (and server granted conflicting lock already)
-	 */
-	if (ldlm_is_cancel_on_block(lock)) {
-		LDLM_DEBUG(lock, "Not replaying reply-less lock:");
-		ldlm_lock_cancel(lock);
-		return 0;
-	}
-
-	/*
-	 * If granted mode matches the requested mode, this lock is granted.
-	 *
-	 * If they differ, but we have a granted mode, then we were granted
-	 * one mode and now want another: ergo, converting.
-	 *
-	 * If we haven't been granted anything and are on a resource list,
-	 * then we're blocked/waiting.
-	 *
-	 * If we haven't been granted anything and we're NOT on a resource list,
-	 * then we haven't got a reply yet and don't have a known disposition.
-	 * This happens whenever a lock enqueue is the request that triggers
-	 * recovery.
-	 */
-	if (lock->l_granted_mode == lock->l_req_mode)
-		flags = LDLM_FL_REPLAY | LDLM_FL_BLOCK_GRANTED;
-	else if (lock->l_granted_mode)
-		flags = LDLM_FL_REPLAY | LDLM_FL_BLOCK_CONV;
-	else if (!list_empty(&lock->l_res_link))
-		flags = LDLM_FL_REPLAY | LDLM_FL_BLOCK_WAIT;
-	else
-		flags = LDLM_FL_REPLAY;
-
-	req = ptlrpc_request_alloc_pack(imp, &RQF_LDLM_ENQUEUE,
-					LUSTRE_DLM_VERSION, LDLM_ENQUEUE);
-	if (!req)
-		return -ENOMEM;
-
-	/* We're part of recovery, so don't wait for it. */
-	req->rq_send_state = LUSTRE_IMP_REPLAY_LOCKS;
-
-	body = req_capsule_client_get(&req->rq_pill, &RMF_DLM_REQ);
-	ldlm_lock2desc(lock, &body->lock_desc);
-	body->lock_flags = ldlm_flags_to_wire(flags);
-
-	ldlm_lock2handle(lock, &body->lock_handle[0]);
-	if (lock->l_lvb_len > 0)
-		req_capsule_extend(&req->rq_pill, &RQF_LDLM_ENQUEUE_LVB);
-	req_capsule_set_size(&req->rq_pill, &RMF_DLM_LVB, RCL_SERVER,
-			     lock->l_lvb_len);
-	ptlrpc_request_set_replen(req);
-	/* notify the server we've replayed all requests.
-	 * also, we mark the request to be put on a dedicated
-	 * queue to be processed after all request replayes.
-	 * bug 6063
-	 */
-	lustre_msg_set_flags(req->rq_reqmsg, MSG_REQ_REPLAY_DONE);
-
-	LDLM_DEBUG(lock, "replaying lock:");
-
-	atomic_inc(&req->rq_import->imp_replay_inflight);
-	BUILD_BUG_ON(sizeof(*aa) > sizeof(req->rq_async_args));
-	aa = ptlrpc_req_async_args(req);
-	aa->lock_handle = body->lock_handle[0];
-	req->rq_interpret_reply = (ptlrpc_interpterer_t)replay_lock_interpret;
-	ptlrpcd_add_req(req);
-
-	return 0;
-}
-
-/**
- * Cancel as many unused locks as possible before replay. since we are
- * in recovery, we can't wait for any outstanding RPCs to send any RPC
- * to the server.
- *
- * Called only in recovery before replaying locks. there is no need to
- * replay locks that are unused. since the clients may hold thousands of
- * cached unused locks, dropping the unused locks can greatly reduce the
- * load on the servers at recovery time.
- */
-static void ldlm_cancel_unused_locks_for_replay(struct ldlm_namespace *ns)
-{
-	int canceled;
-	LIST_HEAD(cancels);
-
-	CDEBUG(D_DLMTRACE,
-	       "Dropping as many unused locks as possible before replay for namespace %s (%d)\n",
-	       ldlm_ns_name(ns), ns->ns_nr_unused);
-
-	/* We don't need to care whether or not LRU resize is enabled
-	 * because the LDLM_LRU_FLAG_NO_WAIT policy doesn't use the
-	 * count parameter
-	 */
-	canceled = ldlm_cancel_lru_local(ns, &cancels, ns->ns_nr_unused, 0,
-					 LCF_LOCAL, LDLM_LRU_FLAG_NO_WAIT);
-
-	CDEBUG(D_DLMTRACE, "Canceled %d unused locks from namespace %s\n",
-	       canceled, ldlm_ns_name(ns));
-}
-
-int ldlm_replay_locks(struct obd_import *imp)
-{
-	struct ldlm_namespace *ns = imp->imp_obd->obd_namespace;
-	LIST_HEAD(list);
-	struct ldlm_lock *lock, *next;
-	int rc = 0;
-
-	LASSERT(atomic_read(&imp->imp_replay_inflight) == 0);
-
-	/* don't replay locks if import failed recovery */
-	if (imp->imp_vbr_failed)
-		return 0;
-
-	/* ensure this doesn't fall to 0 before all have been queued */
-	atomic_inc(&imp->imp_replay_inflight);
-
-	if (ldlm_cancel_unused_locks_before_replay)
-		ldlm_cancel_unused_locks_for_replay(ns);
-
-	ldlm_namespace_foreach(ns, ldlm_chain_lock_for_replay, &list);
-
-	list_for_each_entry_safe(lock, next, &list, l_pending_chain) {
-		list_del_init(&lock->l_pending_chain);
-		if (rc) {
-			LDLM_LOCK_RELEASE(lock);
-			continue; /* or try to do the rest? */
-		}
-		rc = replay_one_lock(imp, lock);
-		LDLM_LOCK_RELEASE(lock);
-	}
-
-	atomic_dec(&imp->imp_replay_inflight);
-
-	return rc;
-}
diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_resource.c b/drivers/staging/lustre/lustre/ldlm/ldlm_resource.c
deleted file mode 100644
index 3946d62ff009..000000000000
--- a/drivers/staging/lustre/lustre/ldlm/ldlm_resource.c
+++ /dev/null
@@ -1,1318 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-/*
- * GPL HEADER START
- *
- * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 only,
- * as published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful, but
- * WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
- * General Public License version 2 for more details (a copy is included
- * in the LICENSE file that accompanied this code).
- *
- * You should have received a copy of the GNU General Public License
- * version 2 along with this program; If not, see
- * http://www.gnu.org/licenses/gpl-2.0.html
- *
- * GPL HEADER END
- */
-/*
- * Copyright (c) 2002, 2010, Oracle and/or its affiliates. All rights reserved.
- * Use is subject to license terms.
- *
- * Copyright (c) 2010, 2015, Intel Corporation.
- */
-/*
- * This file is part of Lustre, http://www.lustre.org/
- * Lustre is a trademark of Sun Microsystems, Inc.
- *
- * lustre/ldlm/ldlm_resource.c
- *
- * Author: Phil Schwan <phil at clusterfs.com>
- * Author: Peter Braam <braam at clusterfs.com>
- */
-
-#define DEBUG_SUBSYSTEM S_LDLM
-#include <lustre_dlm.h>
-#include <lustre_fid.h>
-#include <obd_class.h>
-#include "ldlm_internal.h"
-#include <linux/libcfs/libcfs_hash.h>
-
-struct kmem_cache *ldlm_resource_slab, *ldlm_lock_slab;
-
-int ldlm_srv_namespace_nr;
-int ldlm_cli_namespace_nr;
-
-struct mutex ldlm_srv_namespace_lock;
-LIST_HEAD(ldlm_srv_namespace_list);
-
-struct mutex ldlm_cli_namespace_lock;
-/* Client Namespaces that have active resources in them.
- * Once all resources go away, ldlm_poold moves such namespaces to the
- * inactive list
- */
-LIST_HEAD(ldlm_cli_active_namespace_list);
-/* Client namespaces that don't have any locks in them */
-static LIST_HEAD(ldlm_cli_inactive_namespace_list);
-
-static struct dentry *ldlm_debugfs_dir;
-static struct dentry *ldlm_ns_debugfs_dir;
-struct dentry *ldlm_svc_debugfs_dir;
-
-/* during debug dump certain amount of granted locks for one resource to avoid
- * DDOS.
- */
-static unsigned int ldlm_dump_granted_max = 256;
-
-static ssize_t
-lprocfs_wr_dump_ns(struct file *file, const char __user *buffer,
-		   size_t count, loff_t *off)
-{
-	ldlm_dump_all_namespaces(LDLM_NAMESPACE_SERVER, D_DLMTRACE);
-	ldlm_dump_all_namespaces(LDLM_NAMESPACE_CLIENT, D_DLMTRACE);
-	return count;
-}
-
-LPROC_SEQ_FOPS_WR_ONLY(ldlm, dump_ns);
-
-static int ldlm_rw_uint_seq_show(struct seq_file *m, void *v)
-{
-	seq_printf(m, "%u\n", *(unsigned int *)m->private);
-	return 0;
-}
-
-static ssize_t
-ldlm_rw_uint_seq_write(struct file *file, const char __user *buffer,
-		       size_t count, loff_t *off)
-{
-	struct seq_file *seq = file->private_data;
-
-	if (count == 0)
-		return 0;
-	return kstrtouint_from_user(buffer, count, 0,
-				    (unsigned int *)seq->private);
-}
-
-LPROC_SEQ_FOPS(ldlm_rw_uint);
-
-static struct lprocfs_vars ldlm_debugfs_list[] = {
-	{ "dump_namespaces", &ldlm_dump_ns_fops, NULL, 0222 },
-	{ "dump_granted_max", &ldlm_rw_uint_fops, &ldlm_dump_granted_max },
-	{ NULL }
-};
-
-void ldlm_debugfs_setup(void)
-{
-	ldlm_debugfs_dir = debugfs_create_dir(OBD_LDLM_DEVICENAME,
-					      debugfs_lustre_root);
-
-	ldlm_ns_debugfs_dir = debugfs_create_dir("namespaces",
-						 ldlm_debugfs_dir);
-
-	ldlm_svc_debugfs_dir = debugfs_create_dir("services", ldlm_debugfs_dir);
-
-	ldebugfs_add_vars(ldlm_debugfs_dir, ldlm_debugfs_list, NULL);
-}
-
-void ldlm_debugfs_cleanup(void)
-{
-	debugfs_remove_recursive(ldlm_svc_debugfs_dir);
-	debugfs_remove_recursive(ldlm_ns_debugfs_dir);
-	debugfs_remove_recursive(ldlm_debugfs_dir);
-}
-
-static ssize_t resource_count_show(struct kobject *kobj, struct attribute *attr,
-				   char *buf)
-{
-	struct ldlm_namespace *ns = container_of(kobj, struct ldlm_namespace,
-						 ns_kobj);
-	__u64		  res = 0;
-	struct cfs_hash_bd	  bd;
-	int		    i;
-
-	/* result is not strictly consistent */
-	cfs_hash_for_each_bucket(ns->ns_rs_hash, &bd, i)
-		res += cfs_hash_bd_count_get(&bd);
-	return sprintf(buf, "%lld\n", res);
-}
-LUSTRE_RO_ATTR(resource_count);
-
-static ssize_t lock_count_show(struct kobject *kobj, struct attribute *attr,
-			       char *buf)
-{
-	struct ldlm_namespace *ns = container_of(kobj, struct ldlm_namespace,
-						 ns_kobj);
-	__u64		  locks;
-
-	locks = lprocfs_stats_collector(ns->ns_stats, LDLM_NSS_LOCKS,
-					LPROCFS_FIELDS_FLAGS_SUM);
-	return sprintf(buf, "%lld\n", locks);
-}
-LUSTRE_RO_ATTR(lock_count);
-
-static ssize_t lock_unused_count_show(struct kobject *kobj,
-				      struct attribute *attr,
-				      char *buf)
-{
-	struct ldlm_namespace *ns = container_of(kobj, struct ldlm_namespace,
-						 ns_kobj);
-
-	return sprintf(buf, "%d\n", ns->ns_nr_unused);
-}
-LUSTRE_RO_ATTR(lock_unused_count);
-
-static ssize_t lru_size_show(struct kobject *kobj, struct attribute *attr,
-			     char *buf)
-{
-	struct ldlm_namespace *ns = container_of(kobj, struct ldlm_namespace,
-						 ns_kobj);
-	__u32 *nr = &ns->ns_max_unused;
-
-	if (ns_connect_lru_resize(ns))
-		nr = &ns->ns_nr_unused;
-	return sprintf(buf, "%u\n", *nr);
-}
-
-static ssize_t lru_size_store(struct kobject *kobj, struct attribute *attr,
-			      const char *buffer, size_t count)
-{
-	struct ldlm_namespace *ns = container_of(kobj, struct ldlm_namespace,
-						 ns_kobj);
-	unsigned long tmp;
-	int lru_resize;
-	int err;
-
-	if (strncmp(buffer, "clear", 5) == 0) {
-		CDEBUG(D_DLMTRACE,
-		       "dropping all unused locks from namespace %s\n",
-		       ldlm_ns_name(ns));
-		if (ns_connect_lru_resize(ns)) {
-			int canceled, unused  = ns->ns_nr_unused;
-
-			/* Try to cancel all @ns_nr_unused locks. */
-			canceled = ldlm_cancel_lru(ns, unused, 0,
-						   LDLM_LRU_FLAG_PASSED);
-			if (canceled < unused) {
-				CDEBUG(D_DLMTRACE,
-				       "not all requested locks are canceled, requested: %d, canceled: %d\n",
-				       unused,
-				       canceled);
-				return -EINVAL;
-			}
-		} else {
-			tmp = ns->ns_max_unused;
-			ns->ns_max_unused = 0;
-			ldlm_cancel_lru(ns, 0, 0, LDLM_LRU_FLAG_PASSED);
-			ns->ns_max_unused = tmp;
-		}
-		return count;
-	}
-
-	err = kstrtoul(buffer, 10, &tmp);
-	if (err != 0) {
-		CERROR("lru_size: invalid value written\n");
-		return -EINVAL;
-	}
-	lru_resize = (tmp == 0);
-
-	if (ns_connect_lru_resize(ns)) {
-		if (!lru_resize)
-			ns->ns_max_unused = (unsigned int)tmp;
-
-		if (tmp > ns->ns_nr_unused)
-			tmp = ns->ns_nr_unused;
-		tmp = ns->ns_nr_unused - tmp;
-
-		CDEBUG(D_DLMTRACE,
-		       "changing namespace %s unused locks from %u to %u\n",
-		       ldlm_ns_name(ns), ns->ns_nr_unused,
-		       (unsigned int)tmp);
-		ldlm_cancel_lru(ns, tmp, LCF_ASYNC, LDLM_LRU_FLAG_PASSED);
-
-		if (!lru_resize) {
-			CDEBUG(D_DLMTRACE,
-			       "disable lru_resize for namespace %s\n",
-			       ldlm_ns_name(ns));
-			ns->ns_connect_flags &= ~OBD_CONNECT_LRU_RESIZE;
-		}
-	} else {
-		CDEBUG(D_DLMTRACE,
-		       "changing namespace %s max_unused from %u to %u\n",
-		       ldlm_ns_name(ns), ns->ns_max_unused,
-		       (unsigned int)tmp);
-		ns->ns_max_unused = (unsigned int)tmp;
-		ldlm_cancel_lru(ns, 0, LCF_ASYNC, LDLM_LRU_FLAG_PASSED);
-
-		/* Make sure that LRU resize was originally supported before
-		 * turning it on here.
-		 */
-		if (lru_resize &&
-		    (ns->ns_orig_connect_flags & OBD_CONNECT_LRU_RESIZE)) {
-			CDEBUG(D_DLMTRACE,
-			       "enable lru_resize for namespace %s\n",
-			       ldlm_ns_name(ns));
-			ns->ns_connect_flags |= OBD_CONNECT_LRU_RESIZE;
-		}
-	}
-
-	return count;
-}
-LUSTRE_RW_ATTR(lru_size);
-
-static ssize_t lru_max_age_show(struct kobject *kobj, struct attribute *attr,
-				char *buf)
-{
-	struct ldlm_namespace *ns = container_of(kobj, struct ldlm_namespace,
-						 ns_kobj);
-
-	return sprintf(buf, "%u\n", ns->ns_max_age);
-}
-
-static ssize_t lru_max_age_store(struct kobject *kobj, struct attribute *attr,
-				 const char *buffer, size_t count)
-{
-	struct ldlm_namespace *ns = container_of(kobj, struct ldlm_namespace,
-						 ns_kobj);
-	unsigned long tmp;
-	int err;
-
-	err = kstrtoul(buffer, 10, &tmp);
-	if (err != 0)
-		return -EINVAL;
-
-	ns->ns_max_age = tmp;
-
-	return count;
-}
-LUSTRE_RW_ATTR(lru_max_age);
-
-static ssize_t early_lock_cancel_show(struct kobject *kobj,
-				      struct attribute *attr,
-				      char *buf)
-{
-	struct ldlm_namespace *ns = container_of(kobj, struct ldlm_namespace,
-						 ns_kobj);
-
-	return sprintf(buf, "%d\n", ns_connect_cancelset(ns));
-}
-
-static ssize_t early_lock_cancel_store(struct kobject *kobj,
-				       struct attribute *attr,
-				       const char *buffer,
-				       size_t count)
-{
-	struct ldlm_namespace *ns = container_of(kobj, struct ldlm_namespace,
-						 ns_kobj);
-	unsigned long supp = -1;
-	int rc;
-
-	rc = kstrtoul(buffer, 10, &supp);
-	if (rc < 0)
-		return rc;
-
-	if (supp == 0)
-		ns->ns_connect_flags &= ~OBD_CONNECT_CANCELSET;
-	else if (ns->ns_orig_connect_flags & OBD_CONNECT_CANCELSET)
-		ns->ns_connect_flags |= OBD_CONNECT_CANCELSET;
-	return count;
-}
-LUSTRE_RW_ATTR(early_lock_cancel);
-
-/* These are for namespaces in /sys/fs/lustre/ldlm/namespaces/ */
-static struct attribute *ldlm_ns_attrs[] = {
-	&lustre_attr_resource_count.attr,
-	&lustre_attr_lock_count.attr,
-	&lustre_attr_lock_unused_count.attr,
-	&lustre_attr_lru_size.attr,
-	&lustre_attr_lru_max_age.attr,
-	&lustre_attr_early_lock_cancel.attr,
-	NULL,
-};
-
-static void ldlm_ns_release(struct kobject *kobj)
-{
-	struct ldlm_namespace *ns = container_of(kobj, struct ldlm_namespace,
-						 ns_kobj);
-	complete(&ns->ns_kobj_unregister);
-}
-
-static struct kobj_type ldlm_ns_ktype = {
-	.default_attrs  = ldlm_ns_attrs,
-	.sysfs_ops      = &lustre_sysfs_ops,
-	.release        = ldlm_ns_release,
-};
-
-static void ldlm_namespace_debugfs_unregister(struct ldlm_namespace *ns)
-{
-	debugfs_remove_recursive(ns->ns_debugfs_entry);
-
-	if (ns->ns_stats)
-		lprocfs_free_stats(&ns->ns_stats);
-}
-
-static void ldlm_namespace_sysfs_unregister(struct ldlm_namespace *ns)
-{
-	kobject_put(&ns->ns_kobj);
-	wait_for_completion(&ns->ns_kobj_unregister);
-}
-
-static int ldlm_namespace_sysfs_register(struct ldlm_namespace *ns)
-{
-	int err;
-
-	ns->ns_kobj.kset = ldlm_ns_kset;
-	init_completion(&ns->ns_kobj_unregister);
-	err = kobject_init_and_add(&ns->ns_kobj, &ldlm_ns_ktype, NULL,
-				   "%s", ldlm_ns_name(ns));
-
-	ns->ns_stats = lprocfs_alloc_stats(LDLM_NSS_LAST, 0);
-	if (!ns->ns_stats) {
-		kobject_put(&ns->ns_kobj);
-		return -ENOMEM;
-	}
-
-	lprocfs_counter_init(ns->ns_stats, LDLM_NSS_LOCKS,
-			     LPROCFS_CNTR_AVGMINMAX, "locks", "locks");
-
-	return err;
-}
-
-static int ldlm_namespace_debugfs_register(struct ldlm_namespace *ns)
-{
-	struct dentry *ns_entry;
-
-	if (!IS_ERR_OR_NULL(ns->ns_debugfs_entry)) {
-		ns_entry = ns->ns_debugfs_entry;
-	} else {
-		ns_entry = debugfs_create_dir(ldlm_ns_name(ns),
-					      ldlm_ns_debugfs_dir);
-		if (!ns_entry)
-			return -ENOMEM;
-		ns->ns_debugfs_entry = ns_entry;
-	}
-
-	return 0;
-}
-
-#undef MAX_STRING_SIZE
-
-static struct ldlm_resource *ldlm_resource_getref(struct ldlm_resource *res)
-{
-	LASSERT(res);
-	LASSERT(res != LP_POISON);
-	atomic_inc(&res->lr_refcount);
-	CDEBUG(D_INFO, "getref res: %p count: %d\n", res,
-	       atomic_read(&res->lr_refcount));
-	return res;
-}
-
-static unsigned int ldlm_res_hop_hash(struct cfs_hash *hs,
-				      const void *key, unsigned int mask)
-{
-	const struct ldlm_res_id     *id  = key;
-	unsigned int		val = 0;
-	unsigned int		i;
-
-	for (i = 0; i < RES_NAME_SIZE; i++)
-		val += id->name[i];
-	return val & mask;
-}
-
-static unsigned int ldlm_res_hop_fid_hash(struct cfs_hash *hs,
-					  const void *key, unsigned int mask)
-{
-	const struct ldlm_res_id *id = key;
-	struct lu_fid       fid;
-	__u32	       hash;
-	__u32	       val;
-
-	fid.f_seq = id->name[LUSTRE_RES_ID_SEQ_OFF];
-	fid.f_oid = (__u32)id->name[LUSTRE_RES_ID_VER_OID_OFF];
-	fid.f_ver = (__u32)(id->name[LUSTRE_RES_ID_VER_OID_OFF] >> 32);
-
-	hash = fid_flatten32(&fid);
-	hash += (hash >> 4) + (hash << 12); /* mixing oid and seq */
-	if (id->name[LUSTRE_RES_ID_HSH_OFF] != 0) {
-		val = id->name[LUSTRE_RES_ID_HSH_OFF];
-		hash += (val >> 5) + (val << 11);
-	} else {
-		val = fid_oid(&fid);
-	}
-	hash = hash_long(hash, hs->hs_bkt_bits);
-	/* give me another random factor */
-	hash -= hash_long((unsigned long)hs, val % 11 + 3);
-
-	hash <<= hs->hs_cur_bits - hs->hs_bkt_bits;
-	hash |= ldlm_res_hop_hash(hs, key, CFS_HASH_NBKT(hs) - 1);
-
-	return hash & mask;
-}
-
-static void *ldlm_res_hop_key(struct hlist_node *hnode)
-{
-	struct ldlm_resource   *res;
-
-	res = hlist_entry(hnode, struct ldlm_resource, lr_hash);
-	return &res->lr_name;
-}
-
-static int ldlm_res_hop_keycmp(const void *key, struct hlist_node *hnode)
-{
-	struct ldlm_resource   *res;
-
-	res = hlist_entry(hnode, struct ldlm_resource, lr_hash);
-	return ldlm_res_eq((const struct ldlm_res_id *)key,
-			   (const struct ldlm_res_id *)&res->lr_name);
-}
-
-static void *ldlm_res_hop_object(struct hlist_node *hnode)
-{
-	return hlist_entry(hnode, struct ldlm_resource, lr_hash);
-}
-
-static void ldlm_res_hop_get_locked(struct cfs_hash *hs,
-				    struct hlist_node *hnode)
-{
-	struct ldlm_resource *res;
-
-	res = hlist_entry(hnode, struct ldlm_resource, lr_hash);
-	ldlm_resource_getref(res);
-}
-
-static void ldlm_res_hop_put(struct cfs_hash *hs, struct hlist_node *hnode)
-{
-	struct ldlm_resource *res;
-
-	res = hlist_entry(hnode, struct ldlm_resource, lr_hash);
-	ldlm_resource_putref(res);
-}
-
-static struct cfs_hash_ops ldlm_ns_hash_ops = {
-	.hs_hash	= ldlm_res_hop_hash,
-	.hs_key		= ldlm_res_hop_key,
-	.hs_keycmp      = ldlm_res_hop_keycmp,
-	.hs_keycpy      = NULL,
-	.hs_object      = ldlm_res_hop_object,
-	.hs_get		= ldlm_res_hop_get_locked,
-	.hs_put		= ldlm_res_hop_put
-};
-
-static struct cfs_hash_ops ldlm_ns_fid_hash_ops = {
-	.hs_hash	= ldlm_res_hop_fid_hash,
-	.hs_key		= ldlm_res_hop_key,
-	.hs_keycmp      = ldlm_res_hop_keycmp,
-	.hs_keycpy      = NULL,
-	.hs_object      = ldlm_res_hop_object,
-	.hs_get		= ldlm_res_hop_get_locked,
-	.hs_put		= ldlm_res_hop_put
-};
-
-struct ldlm_ns_hash_def {
-	enum ldlm_ns_type nsd_type;
-	/** hash bucket bits */
-	unsigned int	nsd_bkt_bits;
-	/** hash bits */
-	unsigned int	nsd_all_bits;
-	/** hash operations */
-	struct cfs_hash_ops *nsd_hops;
-};
-
-static struct ldlm_ns_hash_def ldlm_ns_hash_defs[] = {
-	{
-		.nsd_type       = LDLM_NS_TYPE_MDC,
-		.nsd_bkt_bits   = 11,
-		.nsd_all_bits   = 16,
-		.nsd_hops       = &ldlm_ns_fid_hash_ops,
-	},
-	{
-		.nsd_type       = LDLM_NS_TYPE_MDT,
-		.nsd_bkt_bits   = 14,
-		.nsd_all_bits   = 21,
-		.nsd_hops       = &ldlm_ns_fid_hash_ops,
-	},
-	{
-		.nsd_type       = LDLM_NS_TYPE_OSC,
-		.nsd_bkt_bits   = 8,
-		.nsd_all_bits   = 12,
-		.nsd_hops       = &ldlm_ns_hash_ops,
-	},
-	{
-		.nsd_type       = LDLM_NS_TYPE_OST,
-		.nsd_bkt_bits   = 11,
-		.nsd_all_bits   = 17,
-		.nsd_hops       = &ldlm_ns_hash_ops,
-	},
-	{
-		.nsd_type       = LDLM_NS_TYPE_MGC,
-		.nsd_bkt_bits   = 4,
-		.nsd_all_bits   = 4,
-		.nsd_hops       = &ldlm_ns_hash_ops,
-	},
-	{
-		.nsd_type       = LDLM_NS_TYPE_MGT,
-		.nsd_bkt_bits   = 4,
-		.nsd_all_bits   = 4,
-		.nsd_hops       = &ldlm_ns_hash_ops,
-	},
-	{
-		.nsd_type       = LDLM_NS_TYPE_UNKNOWN,
-	},
-};
-
-/** Register \a ns in the list of namespaces */
-static void ldlm_namespace_register(struct ldlm_namespace *ns,
-				    enum ldlm_side client)
-{
-	mutex_lock(ldlm_namespace_lock(client));
-	LASSERT(list_empty(&ns->ns_list_chain));
-	list_add(&ns->ns_list_chain, &ldlm_cli_inactive_namespace_list);
-	ldlm_namespace_nr_inc(client);
-	mutex_unlock(ldlm_namespace_lock(client));
-}
-
-/**
- * Create and initialize new empty namespace.
- */
-struct ldlm_namespace *ldlm_namespace_new(struct obd_device *obd, char *name,
-					  enum ldlm_side client,
-					  enum ldlm_appetite apt,
-					  enum ldlm_ns_type ns_type)
-{
-	struct ldlm_namespace *ns = NULL;
-	struct ldlm_ns_bucket *nsb;
-	struct ldlm_ns_hash_def    *nsd;
-	struct cfs_hash_bd	  bd;
-	int		    idx;
-	int		    rc;
-
-	LASSERT(obd);
-
-	rc = ldlm_get_ref();
-	if (rc) {
-		CERROR("ldlm_get_ref failed: %d\n", rc);
-		return NULL;
-	}
-
-	for (idx = 0;; idx++) {
-		nsd = &ldlm_ns_hash_defs[idx];
-		if (nsd->nsd_type == LDLM_NS_TYPE_UNKNOWN) {
-			CERROR("Unknown type %d for ns %s\n", ns_type, name);
-			goto out_ref;
-		}
-
-		if (nsd->nsd_type == ns_type)
-			break;
-	}
-
-	ns = kzalloc(sizeof(*ns), GFP_NOFS);
-	if (!ns)
-		goto out_ref;
-
-	ns->ns_rs_hash = cfs_hash_create(name,
-					 nsd->nsd_all_bits, nsd->nsd_all_bits,
-					 nsd->nsd_bkt_bits, sizeof(*nsb),
-					 CFS_HASH_MIN_THETA,
-					 CFS_HASH_MAX_THETA,
-					 nsd->nsd_hops,
-					 CFS_HASH_DEPTH |
-					 CFS_HASH_BIGNAME |
-					 CFS_HASH_SPIN_BKTLOCK |
-					 CFS_HASH_NO_ITEMREF);
-	if (!ns->ns_rs_hash)
-		goto out_ns;
-
-	cfs_hash_for_each_bucket(ns->ns_rs_hash, &bd, idx) {
-		nsb = cfs_hash_bd_extra_get(ns->ns_rs_hash, &bd);
-		at_init(&nsb->nsb_at_estimate, ldlm_enqueue_min, 0);
-		nsb->nsb_namespace = ns;
-	}
-
-	ns->ns_obd      = obd;
-	ns->ns_appetite = apt;
-	ns->ns_client   = client;
-	ns->ns_name     = kstrdup(name, GFP_KERNEL);
-	if (!ns->ns_name)
-		goto out_hash;
-
-	INIT_LIST_HEAD(&ns->ns_list_chain);
-	INIT_LIST_HEAD(&ns->ns_unused_list);
-	spin_lock_init(&ns->ns_lock);
-	atomic_set(&ns->ns_bref, 0);
-	init_waitqueue_head(&ns->ns_waitq);
-
-	ns->ns_max_parallel_ast   = LDLM_DEFAULT_PARALLEL_AST_LIMIT;
-	ns->ns_nr_unused	  = 0;
-	ns->ns_max_unused	 = LDLM_DEFAULT_LRU_SIZE;
-	ns->ns_max_age	    = LDLM_DEFAULT_MAX_ALIVE;
-	ns->ns_orig_connect_flags = 0;
-	ns->ns_connect_flags      = 0;
-	ns->ns_stopping	   = 0;
-
-	rc = ldlm_namespace_sysfs_register(ns);
-	if (rc != 0) {
-		CERROR("Can't initialize ns sysfs, rc %d\n", rc);
-		goto out_hash;
-	}
-
-	rc = ldlm_namespace_debugfs_register(ns);
-	if (rc != 0) {
-		CERROR("Can't initialize ns proc, rc %d\n", rc);
-		goto out_sysfs;
-	}
-
-	idx = ldlm_namespace_nr_read(client);
-	rc = ldlm_pool_init(&ns->ns_pool, ns, idx, client);
-	if (rc) {
-		CERROR("Can't initialize lock pool, rc %d\n", rc);
-		goto out_proc;
-	}
-
-	ldlm_namespace_register(ns, client);
-	return ns;
-out_proc:
-	ldlm_namespace_debugfs_unregister(ns);
-out_sysfs:
-	ldlm_namespace_sysfs_unregister(ns);
-	ldlm_namespace_cleanup(ns, 0);
-out_hash:
-	kfree(ns->ns_name);
-	cfs_hash_putref(ns->ns_rs_hash);
-out_ns:
-	kfree(ns);
-out_ref:
-	ldlm_put_ref();
-	return NULL;
-}
-EXPORT_SYMBOL(ldlm_namespace_new);
-
-extern struct ldlm_lock *ldlm_lock_get(struct ldlm_lock *lock);
-
-/**
- * Cancel and destroy all locks on a resource.
- *
- * If flags contains FL_LOCAL_ONLY, don't try to tell the server, just
- * clean up.  This is currently only used for recovery, and we make
- * certain assumptions as a result--notably, that we shouldn't cancel
- * locks with refs.
- */
-static void cleanup_resource(struct ldlm_resource *res, struct list_head *q,
-			     __u64 flags)
-{
-	int rc = 0;
-	bool local_only = !!(flags & LDLM_FL_LOCAL_ONLY);
-
-	do {
-		struct ldlm_lock *lock = NULL, *tmp;
-		struct lustre_handle lockh;
-
-		/* First, we look for non-cleaned-yet lock
-		 * all cleaned locks are marked by CLEANED flag.
-		 */
-		lock_res(res);
-		list_for_each_entry(tmp, q, l_res_link) {
-			if (ldlm_is_cleaned(tmp))
-				continue;
-
-			lock = tmp;
-			LDLM_LOCK_GET(lock);
-			ldlm_set_cleaned(lock);
-			break;
-		}
-
-		if (!lock) {
-			unlock_res(res);
-			break;
-		}
-
-		/* Set CBPENDING so nothing in the cancellation path
-		 * can match this lock.
-		 */
-		ldlm_set_cbpending(lock);
-		ldlm_set_failed(lock);
-		lock->l_flags |= flags;
-
-		/* ... without sending a CANCEL message for local_only. */
-		if (local_only)
-			ldlm_set_local_only(lock);
-
-		if (local_only && (lock->l_readers || lock->l_writers)) {
-			/* This is a little bit gross, but much better than the
-			 * alternative: pretend that we got a blocking AST from
-			 * the server, so that when the lock is decref'd, it
-			 * will go away ...
-			 */
-			unlock_res(res);
-			LDLM_DEBUG(lock, "setting FL_LOCAL_ONLY");
-			if (lock->l_flags & LDLM_FL_FAIL_LOC) {
-				set_current_state(TASK_UNINTERRUPTIBLE);
-				schedule_timeout(4 * HZ);
-				set_current_state(TASK_RUNNING);
-			}
-			if (lock->l_completion_ast)
-				lock->l_completion_ast(lock, LDLM_FL_FAILED,
-						       NULL);
-			LDLM_LOCK_RELEASE(lock);
-			continue;
-		}
-
-		unlock_res(res);
-		ldlm_lock2handle(lock, &lockh);
-		rc = ldlm_cli_cancel(&lockh, LCF_LOCAL);
-		if (rc)
-			CERROR("ldlm_cli_cancel: %d\n", rc);
-		LDLM_LOCK_RELEASE(lock);
-	} while (1);
-}
-
-static int ldlm_resource_clean(struct cfs_hash *hs, struct cfs_hash_bd *bd,
-			       struct hlist_node *hnode, void *arg)
-{
-	struct ldlm_resource *res = cfs_hash_object(hs, hnode);
-	__u64 flags = *(__u64 *)arg;
-
-	cleanup_resource(res, &res->lr_granted, flags);
-	cleanup_resource(res, &res->lr_waiting, flags);
-
-	return 0;
-}
-
-static int ldlm_resource_complain(struct cfs_hash *hs, struct cfs_hash_bd *bd,
-				  struct hlist_node *hnode, void *arg)
-{
-	struct ldlm_resource  *res = cfs_hash_object(hs, hnode);
-
-	lock_res(res);
-	CERROR("%s: namespace resource " DLDLMRES
-	       " (%p) refcount nonzero (%d) after lock cleanup; forcing cleanup.\n",
-	       ldlm_ns_name(ldlm_res_to_ns(res)), PLDLMRES(res), res,
-	       atomic_read(&res->lr_refcount) - 1);
-
-	ldlm_resource_dump(D_ERROR, res);
-	unlock_res(res);
-	return 0;
-}
-
-/**
- * Cancel and destroy all locks in the namespace.
- *
- * Typically used during evictions when server notified client that it was
- * evicted and all of its state needs to be destroyed.
- * Also used during shutdown.
- */
-int ldlm_namespace_cleanup(struct ldlm_namespace *ns, __u64 flags)
-{
-	if (!ns) {
-		CDEBUG(D_INFO, "NULL ns, skipping cleanup\n");
-		return ELDLM_OK;
-	}
-
-	cfs_hash_for_each_nolock(ns->ns_rs_hash, ldlm_resource_clean,
-				 &flags, 0);
-	cfs_hash_for_each_nolock(ns->ns_rs_hash, ldlm_resource_complain,
-				 NULL, 0);
-	return ELDLM_OK;
-}
-EXPORT_SYMBOL(ldlm_namespace_cleanup);
-
-/**
- * Attempts to free namespace.
- *
- * Only used when namespace goes away, like during an unmount.
- */
-static int __ldlm_namespace_free(struct ldlm_namespace *ns, int force)
-{
-	/* At shutdown time, don't call the cancellation callback */
-	ldlm_namespace_cleanup(ns, force ? LDLM_FL_LOCAL_ONLY : 0);
-
-	if (atomic_read(&ns->ns_bref) > 0) {
-		int rc;
-
-		CDEBUG(D_DLMTRACE,
-		       "dlm namespace %s free waiting on refcount %d\n",
-		       ldlm_ns_name(ns), atomic_read(&ns->ns_bref));
-force_wait:
-		if (force)
-			rc = wait_event_idle_timeout(ns->ns_waitq,
-						     atomic_read(&ns->ns_bref) == 0,
-						     obd_timeout * HZ / 4) ? 0 : -ETIMEDOUT;
-		else
-			rc = l_wait_event_abortable(ns->ns_waitq,
-						    atomic_read(&ns->ns_bref) == 0);
-
-		/* Forced cleanups should be able to reclaim all references,
-		 * so it's safe to wait forever... we can't leak locks...
-		 */
-		if (force && rc == -ETIMEDOUT) {
-			LCONSOLE_ERROR("Forced cleanup waiting for %s namespace with %d resources in use, (rc=%d)\n",
-				       ldlm_ns_name(ns),
-				       atomic_read(&ns->ns_bref), rc);
-			goto force_wait;
-		}
-
-		if (atomic_read(&ns->ns_bref)) {
-			LCONSOLE_ERROR("Cleanup waiting for %s namespace with %d resources in use, (rc=%d)\n",
-				       ldlm_ns_name(ns),
-				       atomic_read(&ns->ns_bref), rc);
-			return ELDLM_NAMESPACE_EXISTS;
-		}
-		CDEBUG(D_DLMTRACE, "dlm namespace %s free done waiting\n",
-		       ldlm_ns_name(ns));
-	}
-
-	return ELDLM_OK;
-}
-
-/**
- * Performs various cleanups for passed \a ns to make it drop refc and be
- * ready for freeing. Waits for refc == 0.
- *
- * The following is done:
- * (0) Unregister \a ns from its list to make inaccessible for potential
- * users like pools thread and others;
- * (1) Clear all locks in \a ns.
- */
-void ldlm_namespace_free_prior(struct ldlm_namespace *ns,
-			       struct obd_import *imp,
-			       int force)
-{
-	int rc;
-
-	if (!ns)
-		return;
-
-	spin_lock(&ns->ns_lock);
-	ns->ns_stopping = 1;
-	spin_unlock(&ns->ns_lock);
-
-	/*
-	 * Can fail with -EINTR when force == 0 in which case try harder.
-	 */
-	rc = __ldlm_namespace_free(ns, force);
-	if (rc != ELDLM_OK) {
-		if (imp) {
-			ptlrpc_disconnect_import(imp, 0);
-			ptlrpc_invalidate_import(imp);
-		}
-
-		/*
-		 * With all requests dropped and the import inactive
-		 * we are guaranteed all reference will be dropped.
-		 */
-		rc = __ldlm_namespace_free(ns, 1);
-		LASSERT(rc == 0);
-	}
-}
-
-/** Unregister \a ns from the list of namespaces. */
-static void ldlm_namespace_unregister(struct ldlm_namespace *ns,
-				      enum ldlm_side client)
-{
-	mutex_lock(ldlm_namespace_lock(client));
-	LASSERT(!list_empty(&ns->ns_list_chain));
-	/* Some asserts and possibly other parts of the code are still
-	 * using list_empty(&ns->ns_list_chain). This is why it is
-	 * important to use list_del_init() here.
-	 */
-	list_del_init(&ns->ns_list_chain);
-	ldlm_namespace_nr_dec(client);
-	mutex_unlock(ldlm_namespace_lock(client));
-}
-
-/**
- * Performs freeing memory structures related to \a ns. This is only done
- * when ldlm_namespce_free_prior() successfully removed all resources
- * referencing \a ns and its refc == 0.
- */
-void ldlm_namespace_free_post(struct ldlm_namespace *ns)
-{
-	if (!ns)
-		return;
-
-	/* Make sure that nobody can find this ns in its list. */
-	ldlm_namespace_unregister(ns, ns->ns_client);
-	/* Fini pool _before_ parent proc dir is removed. This is important as
-	 * ldlm_pool_fini() removes own proc dir which is child to @dir.
-	 * Removing it after @dir may cause oops.
-	 */
-	ldlm_pool_fini(&ns->ns_pool);
-
-	ldlm_namespace_debugfs_unregister(ns);
-	ldlm_namespace_sysfs_unregister(ns);
-	cfs_hash_putref(ns->ns_rs_hash);
-	kfree(ns->ns_name);
-	/* Namespace \a ns should be not on list at this time, otherwise
-	 * this will cause issues related to using freed \a ns in poold
-	 * thread.
-	 */
-	LASSERT(list_empty(&ns->ns_list_chain));
-	kfree(ns);
-	ldlm_put_ref();
-}
-
-void ldlm_namespace_get(struct ldlm_namespace *ns)
-{
-	atomic_inc(&ns->ns_bref);
-}
-
-/* This is only for callers that care about refcount */
-static int ldlm_namespace_get_return(struct ldlm_namespace *ns)
-{
-	return atomic_inc_return(&ns->ns_bref);
-}
-
-void ldlm_namespace_put(struct ldlm_namespace *ns)
-{
-	if (atomic_dec_and_lock(&ns->ns_bref, &ns->ns_lock)) {
-		wake_up(&ns->ns_waitq);
-		spin_unlock(&ns->ns_lock);
-	}
-}
-
-/** Should be called with ldlm_namespace_lock(client) taken. */
-void ldlm_namespace_move_to_active_locked(struct ldlm_namespace *ns,
-					  enum ldlm_side client)
-{
-	LASSERT(!list_empty(&ns->ns_list_chain));
-	LASSERT(mutex_is_locked(ldlm_namespace_lock(client)));
-	list_move_tail(&ns->ns_list_chain, ldlm_namespace_list(client));
-}
-
-/** Should be called with ldlm_namespace_lock(client) taken. */
-void ldlm_namespace_move_to_inactive_locked(struct ldlm_namespace *ns,
-					    enum ldlm_side client)
-{
-	LASSERT(!list_empty(&ns->ns_list_chain));
-	LASSERT(mutex_is_locked(ldlm_namespace_lock(client)));
-	list_move_tail(&ns->ns_list_chain, &ldlm_cli_inactive_namespace_list);
-}
-
-/** Should be called with ldlm_namespace_lock(client) taken. */
-struct ldlm_namespace *ldlm_namespace_first_locked(enum ldlm_side client)
-{
-	LASSERT(mutex_is_locked(ldlm_namespace_lock(client)));
-	LASSERT(!list_empty(ldlm_namespace_list(client)));
-	return container_of(ldlm_namespace_list(client)->next,
-		struct ldlm_namespace, ns_list_chain);
-}
-
-/** Create and initialize new resource. */
-static struct ldlm_resource *ldlm_resource_new(void)
-{
-	struct ldlm_resource *res;
-	int idx;
-
-	res = kmem_cache_zalloc(ldlm_resource_slab, GFP_NOFS);
-	if (!res)
-		return NULL;
-
-	INIT_LIST_HEAD(&res->lr_granted);
-	INIT_LIST_HEAD(&res->lr_waiting);
-
-	/* Initialize interval trees for each lock mode. */
-	for (idx = 0; idx < LCK_MODE_NUM; idx++) {
-		res->lr_itree[idx].lit_size = 0;
-		res->lr_itree[idx].lit_mode = 1 << idx;
-		res->lr_itree[idx].lit_root = RB_ROOT_CACHED;
-	}
-
-	atomic_set(&res->lr_refcount, 1);
-	spin_lock_init(&res->lr_lock);
-	lu_ref_init(&res->lr_reference);
-
-	/* The creator of the resource must unlock the mutex after LVB
-	 * initialization.
-	 */
-	mutex_init(&res->lr_lvb_mutex);
-	mutex_lock(&res->lr_lvb_mutex);
-
-	return res;
-}
-
-/**
- * Return a reference to resource with given name, creating it if necessary.
- * Args: namespace with ns_lock unlocked
- * Locks: takes and releases NS hash-lock and res->lr_lock
- * Returns: referenced, unlocked ldlm_resource or NULL
- */
-struct ldlm_resource *
-ldlm_resource_get(struct ldlm_namespace *ns, struct ldlm_resource *parent,
-		  const struct ldlm_res_id *name, enum ldlm_type type,
-		  int create)
-{
-	struct hlist_node     *hnode;
-	struct ldlm_resource *res = NULL;
-	struct cfs_hash_bd	 bd;
-	__u64		 version;
-	int		      ns_refcount = 0;
-	int rc;
-
-	LASSERT(!parent);
-	LASSERT(ns->ns_rs_hash);
-	LASSERT(name->name[0] != 0);
-
-	cfs_hash_bd_get_and_lock(ns->ns_rs_hash, (void *)name, &bd, 0);
-	hnode = cfs_hash_bd_lookup_locked(ns->ns_rs_hash, &bd, (void *)name);
-	if (hnode) {
-		cfs_hash_bd_unlock(ns->ns_rs_hash, &bd, 0);
-		goto lvbo_init;
-	}
-
-	version = cfs_hash_bd_version_get(&bd);
-	cfs_hash_bd_unlock(ns->ns_rs_hash, &bd, 0);
-
-	if (create == 0)
-		return ERR_PTR(-ENOENT);
-
-	LASSERTF(type >= LDLM_MIN_TYPE && type < LDLM_MAX_TYPE,
-		 "type: %d\n", type);
-	res = ldlm_resource_new();
-	if (!res)
-		return ERR_PTR(-ENOMEM);
-
-	res->lr_ns_bucket  = cfs_hash_bd_extra_get(ns->ns_rs_hash, &bd);
-	res->lr_name       = *name;
-	res->lr_type       = type;
-
-	cfs_hash_bd_lock(ns->ns_rs_hash, &bd, 1);
-	hnode = (version == cfs_hash_bd_version_get(&bd)) ?  NULL :
-		cfs_hash_bd_lookup_locked(ns->ns_rs_hash, &bd, (void *)name);
-
-	if (hnode) {
-		/* Someone won the race and already added the resource. */
-		cfs_hash_bd_unlock(ns->ns_rs_hash, &bd, 1);
-		/* Clean lu_ref for failed resource. */
-		lu_ref_fini(&res->lr_reference);
-		/* We have taken lr_lvb_mutex. Drop it. */
-		mutex_unlock(&res->lr_lvb_mutex);
-		kmem_cache_free(ldlm_resource_slab, res);
-lvbo_init:
-		res = hlist_entry(hnode, struct ldlm_resource, lr_hash);
-		/* Synchronize with regard to resource creation. */
-		if (ns->ns_lvbo && ns->ns_lvbo->lvbo_init) {
-			mutex_lock(&res->lr_lvb_mutex);
-			mutex_unlock(&res->lr_lvb_mutex);
-		}
-
-		if (unlikely(res->lr_lvb_len < 0)) {
-			rc = res->lr_lvb_len;
-			ldlm_resource_putref(res);
-			res = ERR_PTR(rc);
-		}
-		return res;
-	}
-	/* We won! Let's add the resource. */
-	cfs_hash_bd_add_locked(ns->ns_rs_hash, &bd, &res->lr_hash);
-	if (cfs_hash_bd_count_get(&bd) == 1)
-		ns_refcount = ldlm_namespace_get_return(ns);
-
-	cfs_hash_bd_unlock(ns->ns_rs_hash, &bd, 1);
-	if (ns->ns_lvbo && ns->ns_lvbo->lvbo_init) {
-		OBD_FAIL_TIMEOUT(OBD_FAIL_LDLM_CREATE_RESOURCE, 2);
-		rc = ns->ns_lvbo->lvbo_init(res);
-		if (rc < 0) {
-			CERROR("%s: lvbo_init failed for resource %#llx:%#llx: rc = %d\n",
-			       ns->ns_obd->obd_name, name->name[0],
-			       name->name[1], rc);
-			res->lr_lvb_len = rc;
-			mutex_unlock(&res->lr_lvb_mutex);
-			ldlm_resource_putref(res);
-			return ERR_PTR(rc);
-		}
-	}
-
-	/* We create resource with locked lr_lvb_mutex. */
-	mutex_unlock(&res->lr_lvb_mutex);
-
-	/* Let's see if we happened to be the very first resource in this
-	 * namespace. If so, and this is a client namespace, we need to move
-	 * the namespace into the active namespaces list to be patrolled by
-	 * the ldlm_poold.
-	 */
-	if (ns_refcount == 1) {
-		mutex_lock(ldlm_namespace_lock(LDLM_NAMESPACE_CLIENT));
-		ldlm_namespace_move_to_active_locked(ns, LDLM_NAMESPACE_CLIENT);
-		mutex_unlock(ldlm_namespace_lock(LDLM_NAMESPACE_CLIENT));
-	}
-
-	return res;
-}
-EXPORT_SYMBOL(ldlm_resource_get);
-
-static void __ldlm_resource_putref_final(struct cfs_hash_bd *bd,
-					 struct ldlm_resource *res)
-{
-	struct ldlm_ns_bucket *nsb = res->lr_ns_bucket;
-	struct ldlm_namespace *ns = nsb->nsb_namespace;
-
-	if (!list_empty(&res->lr_granted)) {
-		ldlm_resource_dump(D_ERROR, res);
-		LBUG();
-	}
-
-	if (!list_empty(&res->lr_waiting)) {
-		ldlm_resource_dump(D_ERROR, res);
-		LBUG();
-	}
-
-	cfs_hash_bd_del_locked(ns->ns_rs_hash,
-			       bd, &res->lr_hash);
-	lu_ref_fini(&res->lr_reference);
-	cfs_hash_bd_unlock(ns->ns_rs_hash, bd, 1);
-	if (ns->ns_lvbo && ns->ns_lvbo->lvbo_free)
-		ns->ns_lvbo->lvbo_free(res);
-	if (cfs_hash_bd_count_get(bd) == 0)
-		ldlm_namespace_put(ns);
-	kmem_cache_free(ldlm_resource_slab, res);
-}
-
-void ldlm_resource_putref(struct ldlm_resource *res)
-{
-	struct ldlm_namespace *ns = ldlm_res_to_ns(res);
-	struct cfs_hash_bd   bd;
-
-	LASSERT_ATOMIC_GT_LT(&res->lr_refcount, 0, LI_POISON);
-	CDEBUG(D_INFO, "putref res: %p count: %d\n",
-	       res, atomic_read(&res->lr_refcount) - 1);
-
-	cfs_hash_bd_get(ns->ns_rs_hash, &res->lr_name, &bd);
-	if (cfs_hash_bd_dec_and_lock(ns->ns_rs_hash, &bd, &res->lr_refcount))
-		__ldlm_resource_putref_final(&bd, res);
-}
-EXPORT_SYMBOL(ldlm_resource_putref);
-
-/**
- * Add a lock into a given resource into specified lock list.
- */
-void ldlm_resource_add_lock(struct ldlm_resource *res, struct list_head *head,
-			    struct ldlm_lock *lock)
-{
-	check_res_locked(res);
-
-	LDLM_DEBUG(lock, "About to add this lock:");
-
-	if (ldlm_is_destroyed(lock)) {
-		CDEBUG(D_OTHER, "Lock destroyed, not adding to resource\n");
-		return;
-	}
-
-	LASSERT(list_empty(&lock->l_res_link));
-
-	list_add_tail(&lock->l_res_link, head);
-}
-
-void ldlm_resource_unlink_lock(struct ldlm_lock *lock)
-{
-	int type = lock->l_resource->lr_type;
-
-	check_res_locked(lock->l_resource);
-	if (type == LDLM_IBITS || type == LDLM_PLAIN)
-		ldlm_unlink_lock_skiplist(lock);
-	else if (type == LDLM_EXTENT)
-		ldlm_extent_unlink_lock(lock);
-	list_del_init(&lock->l_res_link);
-}
-EXPORT_SYMBOL(ldlm_resource_unlink_lock);
-
-void ldlm_res2desc(struct ldlm_resource *res, struct ldlm_resource_desc *desc)
-{
-	desc->lr_type = res->lr_type;
-	desc->lr_name = res->lr_name;
-}
-
-/**
- * Print information about all locks in all namespaces on this node to debug
- * log.
- */
-void ldlm_dump_all_namespaces(enum ldlm_side client, int level)
-{
-	struct ldlm_namespace *ns;
-
-	if (!((libcfs_debug | D_ERROR) & level))
-		return;
-
-	mutex_lock(ldlm_namespace_lock(client));
-
-	list_for_each_entry(ns, ldlm_namespace_list(client), ns_list_chain)
-		ldlm_namespace_dump(level, ns);
-
-	mutex_unlock(ldlm_namespace_lock(client));
-}
-
-static int ldlm_res_hash_dump(struct cfs_hash *hs, struct cfs_hash_bd *bd,
-			      struct hlist_node *hnode, void *arg)
-{
-	struct ldlm_resource *res = cfs_hash_object(hs, hnode);
-	int    level = (int)(unsigned long)arg;
-
-	lock_res(res);
-	ldlm_resource_dump(level, res);
-	unlock_res(res);
-
-	return 0;
-}
-
-/**
- * Print information about all locks in this namespace on this node to debug
- * log.
- */
-void ldlm_namespace_dump(int level, struct ldlm_namespace *ns)
-{
-	if (!((libcfs_debug | D_ERROR) & level))
-		return;
-
-	CDEBUG(level, "--- Namespace: %s (rc: %d, side: client)\n",
-	       ldlm_ns_name(ns), atomic_read(&ns->ns_bref));
-
-	if (time_before(jiffies, ns->ns_next_dump))
-		return;
-
-	cfs_hash_for_each_nolock(ns->ns_rs_hash,
-				 ldlm_res_hash_dump,
-				 (void *)(unsigned long)level, 0);
-	spin_lock(&ns->ns_lock);
-	ns->ns_next_dump = jiffies + 10 * HZ;
-	spin_unlock(&ns->ns_lock);
-}
-
-/**
- * Print information about all locks in this resource to debug log.
- */
-void ldlm_resource_dump(int level, struct ldlm_resource *res)
-{
-	struct ldlm_lock *lock;
-	unsigned int granted = 0;
-
-	BUILD_BUG_ON(RES_NAME_SIZE != 4);
-
-	if (!((libcfs_debug | D_ERROR) & level))
-		return;
-
-	CDEBUG(level, "--- Resource: " DLDLMRES " (%p) refcount = %d\n",
-	       PLDLMRES(res), res, atomic_read(&res->lr_refcount));
-
-	if (!list_empty(&res->lr_granted)) {
-		CDEBUG(level, "Granted locks (in reverse order):\n");
-		list_for_each_entry_reverse(lock, &res->lr_granted,
-					    l_res_link) {
-			LDLM_DEBUG_LIMIT(level, lock, "###");
-			if (!(level & D_CANTMASK) &&
-			    ++granted > ldlm_dump_granted_max) {
-				CDEBUG(level,
-				       "only dump %d granted locks to avoid DDOS.\n",
-				       granted);
-				break;
-			}
-		}
-	}
-	if (!list_empty(&res->lr_waiting)) {
-		CDEBUG(level, "Waiting locks:\n");
-		list_for_each_entry(lock, &res->lr_waiting, l_res_link)
-			LDLM_DEBUG_LIMIT(level, lock, "###");
-	}
-}
-EXPORT_SYMBOL(ldlm_resource_dump);
diff --git a/drivers/staging/lustre/lustre/ptlrpc/Makefile b/drivers/staging/lustre/lustre/ptlrpc/Makefile
index 77f8eabb2e28..aa152e652002 100644
--- a/drivers/staging/lustre/lustre/ptlrpc/Makefile
+++ b/drivers/staging/lustre/lustre/ptlrpc/Makefile
@@ -3,14 +3,14 @@ subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/include
 subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/lustre/include
 
 obj-$(CONFIG_LUSTRE_FS) += ptlrpc.o
-LDLM := ../../lustre/ldlm/
 
-ldlm_objs := $(LDLM)l_lock.o $(LDLM)ldlm_lock.o
-ldlm_objs += $(LDLM)ldlm_resource.o $(LDLM)ldlm_lib.o
-ldlm_objs += $(LDLM)ldlm_plain.o $(LDLM)ldlm_extent.o
-ldlm_objs += $(LDLM)ldlm_request.o $(LDLM)ldlm_lockd.o
-ldlm_objs += $(LDLM)ldlm_flock.o $(LDLM)ldlm_inodebits.o
-ldlm_objs += $(LDLM)ldlm_pool.o
+ldlm_objs := l_lock.o ldlm_lock.o
+ldlm_objs += ldlm_resource.o ldlm_lib.o
+ldlm_objs += ldlm_plain.o ldlm_extent.o
+ldlm_objs += ldlm_request.o ldlm_lockd.o
+ldlm_objs += ldlm_flock.o ldlm_inodebits.o
+ldlm_objs += ldlm_pool.o
+
 ptlrpc_objs := client.o recover.o connection.o niobuf.o pack_generic.o
 ptlrpc_objs += events.o ptlrpc_module.o service.o pinger.o
 ptlrpc_objs += llog_net.o llog_client.o import.o ptlrpcd.o
diff --git a/drivers/staging/lustre/lustre/ptlrpc/l_lock.c b/drivers/staging/lustre/lustre/ptlrpc/l_lock.c
new file mode 100644
index 000000000000..296259aa51e6
--- /dev/null
+++ b/drivers/staging/lustre/lustre/ptlrpc/l_lock.c
@@ -0,0 +1,73 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * GPL HEADER START
+ *
+ * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 only,
+ * as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License version 2 for more details (a copy is included
+ * in the LICENSE file that accompanied this code).
+ *
+ * You should have received a copy of the GNU General Public License
+ * version 2 along with this program; If not, see
+ * http://www.gnu.org/licenses/gpl-2.0.html
+ *
+ * GPL HEADER END
+ */
+/*
+ * Copyright (c) 2002, 2010, Oracle and/or its affiliates. All rights reserved.
+ * Use is subject to license terms.
+ *
+ * Copyright (c) 2012, Intel Corporation.
+ */
+/*
+ * This file is part of Lustre, http://www.lustre.org/
+ * Lustre is a trademark of Sun Microsystems, Inc.
+ */
+
+#define DEBUG_SUBSYSTEM S_LDLM
+
+#include <lustre_dlm.h>
+#include <lustre_lib.h>
+
+/**
+ * Lock a lock and its resource.
+ *
+ * LDLM locking uses resource to serialize access to locks
+ * but there is a case when we change resource of lock upon
+ * enqueue reply. We rely on lock->l_resource = new_res
+ * being an atomic operation.
+ */
+struct ldlm_resource *lock_res_and_lock(struct ldlm_lock *lock)
+				__acquires(&lock->l_lock)
+				__acquires(&lock->l_resource->lr_lock)
+{
+	spin_lock(&lock->l_lock);
+
+	lock_res(lock->l_resource);
+
+	ldlm_set_res_locked(lock);
+	return lock->l_resource;
+}
+EXPORT_SYMBOL(lock_res_and_lock);
+
+/**
+ * Unlock a lock and its resource previously locked with lock_res_and_lock
+ */
+void unlock_res_and_lock(struct ldlm_lock *lock)
+		__releases(&lock->l_resource->lr_lock)
+		__releases(&lock->l_lock)
+{
+	/* on server-side resource of lock doesn't change */
+	ldlm_clear_res_locked(lock);
+
+	unlock_res(lock->l_resource);
+	spin_unlock(&lock->l_lock);
+}
+EXPORT_SYMBOL(unlock_res_and_lock);
diff --git a/drivers/staging/lustre/lustre/ptlrpc/ldlm_extent.c b/drivers/staging/lustre/lustre/ptlrpc/ldlm_extent.c
new file mode 100644
index 000000000000..225c023b0bba
--- /dev/null
+++ b/drivers/staging/lustre/lustre/ptlrpc/ldlm_extent.c
@@ -0,0 +1,206 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * GPL HEADER START
+ *
+ * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 only,
+ * as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License version 2 for more details (a copy is included
+ * in the LICENSE file that accompanied this code).
+ *
+ * You should have received a copy of the GNU General Public License
+ * version 2 along with this program; If not, see
+ * http://www.gnu.org/licenses/gpl-2.0.html
+ *
+ * GPL HEADER END
+ */
+/*
+ * Copyright (c) 2002, 2010, Oracle and/or its affiliates. All rights reserved.
+ * Use is subject to license terms.
+ *
+ * Copyright (c) 2010, 2012, Intel Corporation.
+ */
+/*
+ * This file is part of Lustre, http://www.lustre.org/
+ * Lustre is a trademark of Sun Microsystems, Inc.
+ *
+ * lustre/ldlm/ldlm_extent.c
+ *
+ * Author: Peter Braam <braam at clusterfs.com>
+ * Author: Phil Schwan <phil at clusterfs.com>
+ */
+
+/**
+ * This file contains implementation of EXTENT lock type
+ *
+ * EXTENT lock type is for locking a contiguous range of values, represented
+ * by 64-bit starting and ending offsets (inclusive). There are several extent
+ * lock modes, some of which may be mutually incompatible. Extent locks are
+ * considered incompatible if their modes are incompatible and their extents
+ * intersect.  See the lock mode compatibility matrix in lustre_dlm.h.
+ */
+
+#define DEBUG_SUBSYSTEM S_LDLM
+#include <lustre_dlm.h>
+#include <obd_support.h>
+#include <obd.h>
+#include <obd_class.h>
+#include <lustre_lib.h>
+#include "ldlm_internal.h"
+#include <linux/interval_tree_generic.h>
+
+#define START(node) ((node)->l_policy_data.l_extent.start)
+#define LAST(node) ((node)->l_policy_data.l_extent.end)
+INTERVAL_TREE_DEFINE(struct ldlm_lock, l_rb, __u64, __subtree_last,
+		     START, LAST, static, extent);
+
+/* When a lock is cancelled by a client, the KMS may undergo change if this
+ * is the "highest lock".  This function returns the new KMS value.
+ * Caller must hold lr_lock already.
+ *
+ * NB: A lock on [x,y] protects a KMS of up to y + 1 bytes!
+ */
+__u64 ldlm_extent_shift_kms(struct ldlm_lock *lock, __u64 old_kms)
+{
+	struct ldlm_resource *res = lock->l_resource;
+	struct ldlm_lock *lck;
+	__u64 kms = 0;
+
+	/* don't let another thread in ldlm_extent_shift_kms race in
+	 * just after we finish and take our lock into account in its
+	 * calculation of the kms
+	 */
+	ldlm_set_kms_ignore(lock);
+
+	list_for_each_entry(lck, &res->lr_granted, l_res_link) {
+
+		if (ldlm_is_kms_ignore(lck))
+			continue;
+
+		if (lck->l_policy_data.l_extent.end >= old_kms)
+			return old_kms;
+
+		/* This extent _has_ to be smaller than old_kms (checked above)
+		 * so kms can only ever be smaller or the same as old_kms.
+		 */
+		if (lck->l_policy_data.l_extent.end + 1 > kms)
+			kms = lck->l_policy_data.l_extent.end + 1;
+	}
+	LASSERTF(kms <= old_kms, "kms %llu old_kms %llu\n", kms, old_kms);
+
+	return kms;
+}
+EXPORT_SYMBOL(ldlm_extent_shift_kms);
+
+static inline int lock_mode_to_index(enum ldlm_mode mode)
+{
+	int index;
+
+	LASSERT(mode != 0);
+	LASSERT(is_power_of_2(mode));
+	for (index = -1; mode; index++)
+		mode >>= 1;
+	LASSERT(index < LCK_MODE_NUM);
+	return index;
+}
+
+/** Add newly granted lock into interval tree for the resource. */
+void ldlm_extent_add_lock(struct ldlm_resource *res,
+			  struct ldlm_lock *lock)
+{
+	struct ldlm_interval_tree *tree;
+	int idx;
+
+	LASSERT(lock->l_granted_mode == lock->l_req_mode);
+
+	LASSERT(RB_EMPTY_NODE(&lock->l_rb));
+
+	idx = lock_mode_to_index(lock->l_granted_mode);
+	LASSERT(lock->l_granted_mode == 1 << idx);
+	LASSERT(lock->l_granted_mode == res->lr_itree[idx].lit_mode);
+
+	tree = &res->lr_itree[idx];
+	extent_insert(lock, &tree->lit_root);
+	tree->lit_size++;
+
+	/* even though we use interval tree to manage the extent lock, we also
+	 * add the locks into grant list, for debug purpose, ..
+	 */
+	ldlm_resource_add_lock(res, &res->lr_granted, lock);
+
+	if (OBD_FAIL_CHECK(OBD_FAIL_LDLM_GRANT_CHECK)) {
+		struct ldlm_lock *lck;
+
+		list_for_each_entry_reverse(lck, &res->lr_granted,
+					    l_res_link) {
+			if (lck == lock)
+				continue;
+			if (lockmode_compat(lck->l_granted_mode,
+					    lock->l_granted_mode))
+				continue;
+			if (ldlm_extent_overlap(&lck->l_req_extent,
+						&lock->l_req_extent)) {
+				CDEBUG(D_ERROR,
+				       "granting conflicting lock %p %p\n",
+				       lck, lock);
+				ldlm_resource_dump(D_ERROR, res);
+				LBUG();
+			}
+		}
+	}
+}
+
+/** Remove cancelled lock from resource interval tree. */
+void ldlm_extent_unlink_lock(struct ldlm_lock *lock)
+{
+	struct ldlm_resource *res = lock->l_resource;
+	struct ldlm_interval_tree *tree;
+	int idx;
+
+	if (RB_EMPTY_NODE(&lock->l_rb)) /* duplicate unlink */
+		return;
+
+	idx = lock_mode_to_index(lock->l_granted_mode);
+	LASSERT(lock->l_granted_mode == 1 << idx);
+	tree = &res->lr_itree[idx];
+
+	tree->lit_size--;
+	extent_remove(lock, &tree->lit_root);
+}
+
+void ldlm_extent_policy_wire_to_local(const union ldlm_wire_policy_data *wpolicy,
+				      union ldlm_policy_data *lpolicy)
+{
+	lpolicy->l_extent.start = wpolicy->l_extent.start;
+	lpolicy->l_extent.end = wpolicy->l_extent.end;
+	lpolicy->l_extent.gid = wpolicy->l_extent.gid;
+}
+
+void ldlm_extent_policy_local_to_wire(const union ldlm_policy_data *lpolicy,
+				      union ldlm_wire_policy_data *wpolicy)
+{
+	memset(wpolicy, 0, sizeof(*wpolicy));
+	wpolicy->l_extent.start = lpolicy->l_extent.start;
+	wpolicy->l_extent.end = lpolicy->l_extent.end;
+	wpolicy->l_extent.gid = lpolicy->l_extent.gid;
+}
+
+void ldlm_extent_search(struct rb_root_cached *root,
+			__u64 start, __u64 end,
+			bool (*matches)(struct ldlm_lock *lock, void *data),
+			void *data)
+{
+	struct ldlm_lock *lock;
+
+	for (lock = extent_iter_first(root, start, end);
+	     lock;
+	     lock = extent_iter_next(lock, start, end))
+		if (matches(lock, data))
+			break;
+}
diff --git a/drivers/staging/lustre/lustre/ptlrpc/ldlm_flock.c b/drivers/staging/lustre/lustre/ptlrpc/ldlm_flock.c
new file mode 100644
index 000000000000..94f3b1e49896
--- /dev/null
+++ b/drivers/staging/lustre/lustre/ptlrpc/ldlm_flock.c
@@ -0,0 +1,486 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * GPL HEADER START
+ *
+ * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 only,
+ * as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License version 2 for more details (a copy is included
+ * in the LICENSE file that accompanied this code).
+ *
+ * You should have received a copy of the GNU General Public License
+ * version 2 along with this program; If not, see
+ * http://www.gnu.org/licenses/gpl-2.0.html
+ *
+ * GPL HEADER END
+ */
+/*
+ * Copyright (c) 2003 Hewlett-Packard Development Company LP.
+ * Developed under the sponsorship of the US Government under
+ * Subcontract No. B514193
+ *
+ * Copyright (c) 2003, 2010, Oracle and/or its affiliates. All rights reserved.
+ * Use is subject to license terms.
+ *
+ * Copyright (c) 2010, 2012, Intel Corporation.
+ */
+/*
+ * This file is part of Lustre, http://www.lustre.org/
+ * Lustre is a trademark of Sun Microsystems, Inc.
+ */
+
+/**
+ * This file implements POSIX lock type for Lustre.
+ * Its policy properties are start and end of extent and PID.
+ *
+ * These locks are only done through MDS due to POSIX semantics requiring
+ * e.g. that locks could be only partially released and as such split into
+ * two parts, and also that two adjacent locks from the same process may be
+ * merged into a single wider lock.
+ *
+ * Lock modes are mapped like this:
+ * PR and PW for READ and WRITE locks
+ * NL to request a releasing of a portion of the lock
+ *
+ * These flock locks never timeout.
+ */
+
+#define DEBUG_SUBSYSTEM S_LDLM
+
+#include <lustre_dlm.h>
+#include <obd_support.h>
+#include <obd_class.h>
+#include <lustre_lib.h>
+#include <linux/list.h>
+#include "ldlm_internal.h"
+
+static inline int
+ldlm_same_flock_owner(struct ldlm_lock *lock, struct ldlm_lock *new)
+{
+	return((new->l_policy_data.l_flock.owner ==
+		lock->l_policy_data.l_flock.owner) &&
+	       (new->l_export == lock->l_export));
+}
+
+static inline int
+ldlm_flocks_overlap(struct ldlm_lock *lock, struct ldlm_lock *new)
+{
+	return((new->l_policy_data.l_flock.start <=
+		lock->l_policy_data.l_flock.end) &&
+	       (new->l_policy_data.l_flock.end >=
+		lock->l_policy_data.l_flock.start));
+}
+
+static inline void
+ldlm_flock_destroy(struct ldlm_lock *lock, enum ldlm_mode mode)
+{
+	LDLM_DEBUG(lock, "%s(mode: %d)",
+		   __func__, mode);
+
+	list_del_init(&lock->l_res_link);
+
+	/* client side - set a flag to prevent sending a CANCEL */
+	lock->l_flags |= LDLM_FL_LOCAL_ONLY | LDLM_FL_CBPENDING;
+
+	/* when reaching here, it is under lock_res_and_lock(). Thus,
+	 * need call the nolock version of ldlm_lock_decref_internal
+	 */
+	ldlm_lock_decref_internal_nolock(lock, mode);
+
+	ldlm_lock_destroy_nolock(lock);
+}
+
+/**
+ * Process a granting attempt for flock lock.
+ * Must be called under ns lock held.
+ *
+ * This function looks for any conflicts for \a lock in the granted or
+ * waiting queues. The lock is granted if no conflicts are found in
+ * either queue.
+ *
+ * It is also responsible for splitting a lock if a portion of the lock
+ * is released.
+ *
+ */
+static int ldlm_process_flock_lock(struct ldlm_lock *req)
+{
+	struct ldlm_resource *res = req->l_resource;
+	struct ldlm_namespace *ns = ldlm_res_to_ns(res);
+	struct ldlm_lock *tmp;
+	struct ldlm_lock *lock;
+	struct ldlm_lock *new = req;
+	struct ldlm_lock *new2 = NULL;
+	enum ldlm_mode mode = req->l_req_mode;
+	int added = (mode == LCK_NL);
+	int splitted = 0;
+	const struct ldlm_callback_suite null_cbs = { };
+
+	CDEBUG(D_DLMTRACE,
+	       "owner %llu pid %u mode %u start %llu end %llu\n",
+	       new->l_policy_data.l_flock.owner,
+	       new->l_policy_data.l_flock.pid, mode,
+	       req->l_policy_data.l_flock.start,
+	       req->l_policy_data.l_flock.end);
+
+	/* No blocking ASTs are sent to the clients for
+	 * Posix file & record locks
+	 */
+	req->l_blocking_ast = NULL;
+
+reprocess:
+	/* This loop determines where this processes locks start
+	 * in the resource lr_granted list.
+	 */
+	list_for_each_entry(lock, &res->lr_granted, l_res_link)
+		if (ldlm_same_flock_owner(lock, req))
+			break;
+
+	/* Scan the locks owned by this process to find the insertion point
+	 * (as locks are ordered), and to handle overlaps.
+	 * We may have to merge or split existing locks.
+	 */
+	list_for_each_entry_safe_from(lock, tmp, &res->lr_granted, l_res_link) {
+
+		if (!ldlm_same_flock_owner(lock, new))
+			break;
+
+		if (lock->l_granted_mode == mode) {
+			/* If the modes are the same then we need to process
+			 * locks that overlap OR adjoin the new lock. The extra
+			 * logic condition is necessary to deal with arithmetic
+			 * overflow and underflow.
+			 */
+			if ((new->l_policy_data.l_flock.start >
+			     (lock->l_policy_data.l_flock.end + 1)) &&
+			    (lock->l_policy_data.l_flock.end != OBD_OBJECT_EOF))
+				continue;
+
+			if ((new->l_policy_data.l_flock.end <
+			     (lock->l_policy_data.l_flock.start - 1)) &&
+			    (lock->l_policy_data.l_flock.start != 0))
+				break;
+
+			if (new->l_policy_data.l_flock.start <
+			    lock->l_policy_data.l_flock.start) {
+				lock->l_policy_data.l_flock.start =
+					new->l_policy_data.l_flock.start;
+			} else {
+				new->l_policy_data.l_flock.start =
+					lock->l_policy_data.l_flock.start;
+			}
+
+			if (new->l_policy_data.l_flock.end >
+			    lock->l_policy_data.l_flock.end) {
+				lock->l_policy_data.l_flock.end =
+					new->l_policy_data.l_flock.end;
+			} else {
+				new->l_policy_data.l_flock.end =
+					lock->l_policy_data.l_flock.end;
+			}
+
+			if (added) {
+				ldlm_flock_destroy(lock, mode);
+			} else {
+				new = lock;
+				added = 1;
+			}
+			continue;
+		}
+
+		if (new->l_policy_data.l_flock.start >
+		    lock->l_policy_data.l_flock.end)
+			continue;
+
+		if (new->l_policy_data.l_flock.end <
+		    lock->l_policy_data.l_flock.start)
+			break;
+
+		if (new->l_policy_data.l_flock.start <=
+		    lock->l_policy_data.l_flock.start) {
+			if (new->l_policy_data.l_flock.end <
+			    lock->l_policy_data.l_flock.end) {
+				lock->l_policy_data.l_flock.start =
+					new->l_policy_data.l_flock.end + 1;
+				break;
+			}
+			ldlm_flock_destroy(lock, lock->l_req_mode);
+			continue;
+		}
+		if (new->l_policy_data.l_flock.end >=
+		    lock->l_policy_data.l_flock.end) {
+			lock->l_policy_data.l_flock.end =
+				new->l_policy_data.l_flock.start - 1;
+			continue;
+		}
+
+		/* split the existing lock into two locks */
+
+		/* if this is an F_UNLCK operation then we could avoid
+		 * allocating a new lock and use the req lock passed in
+		 * with the request but this would complicate the reply
+		 * processing since updates to req get reflected in the
+		 * reply. The client side replays the lock request so
+		 * it must see the original lock data in the reply.
+		 */
+
+		/* XXX - if ldlm_lock_new() can sleep we should
+		 * release the lr_lock, allocate the new lock,
+		 * and restart processing this lock.
+		 */
+		if (!new2) {
+			unlock_res_and_lock(req);
+			new2 = ldlm_lock_create(ns, &res->lr_name, LDLM_FLOCK,
+						lock->l_granted_mode, &null_cbs,
+						NULL, 0, LVB_T_NONE);
+			lock_res_and_lock(req);
+			if (IS_ERR(new2)) {
+				ldlm_flock_destroy(req, lock->l_granted_mode);
+				return LDLM_ITER_STOP;
+			}
+			goto reprocess;
+		}
+
+		splitted = 1;
+
+		new2->l_granted_mode = lock->l_granted_mode;
+		new2->l_policy_data.l_flock.pid =
+			new->l_policy_data.l_flock.pid;
+		new2->l_policy_data.l_flock.owner =
+			new->l_policy_data.l_flock.owner;
+		new2->l_policy_data.l_flock.start =
+			lock->l_policy_data.l_flock.start;
+		new2->l_policy_data.l_flock.end =
+			new->l_policy_data.l_flock.start - 1;
+		lock->l_policy_data.l_flock.start =
+			new->l_policy_data.l_flock.end + 1;
+		new2->l_conn_export = lock->l_conn_export;
+		if (lock->l_export)
+			new2->l_export = class_export_lock_get(lock->l_export,
+							       new2);
+		ldlm_lock_addref_internal_nolock(new2,
+						 lock->l_granted_mode);
+
+		/* insert new2 at lock */
+		ldlm_resource_add_lock(res, &lock->l_res_link, new2);
+		LDLM_LOCK_RELEASE(new2);
+		break;
+	}
+
+	/* if new2 is created but never used, destroy it*/
+	if (splitted == 0 && new2)
+		ldlm_lock_destroy_nolock(new2);
+
+	/* At this point we're granting the lock request. */
+	req->l_granted_mode = req->l_req_mode;
+
+	if (!added) {
+		list_del_init(&req->l_res_link);
+		/* insert new lock before "lock", which might be the
+		 * next lock for this owner, or might be the first
+		 * lock for the next owner, or might not be a lock at
+		 * all, but instead points at the head of the list
+		 */
+		ldlm_resource_add_lock(res, &lock->l_res_link, req);
+	}
+
+	/* In case we're reprocessing the requested lock we can't destroy
+	 * it until after calling ldlm_add_ast_work_item() above so that laawi()
+	 * can bump the reference count on \a req. Otherwise \a req
+	 * could be freed before the completion AST can be sent.
+	 */
+	if (added)
+		ldlm_flock_destroy(req, mode);
+
+	ldlm_resource_dump(D_INFO, res);
+	return LDLM_ITER_CONTINUE;
+}
+
+/**
+ * Flock completion callback function.
+ *
+ * \param lock [in,out]: A lock to be handled
+ * \param flags    [in]: flags
+ * \param *data    [in]: ldlm_work_cp_ast_lock() will use ldlm_cb_set_arg
+ *
+ * \retval 0    : success
+ * \retval <0   : failure
+ */
+int
+ldlm_flock_completion_ast(struct ldlm_lock *lock, __u64 flags, void *data)
+{
+	struct file_lock		*getlk = lock->l_ast_data;
+	int				rc = 0;
+
+	OBD_FAIL_TIMEOUT(OBD_FAIL_LDLM_CP_CB_WAIT2, 4);
+	if (OBD_FAIL_PRECHECK(OBD_FAIL_LDLM_CP_CB_WAIT3)) {
+		lock_res_and_lock(lock);
+		lock->l_flags |= LDLM_FL_FAIL_LOC;
+		unlock_res_and_lock(lock);
+		OBD_FAIL_TIMEOUT(OBD_FAIL_LDLM_CP_CB_WAIT3, 4);
+	}
+	CDEBUG(D_DLMTRACE, "flags: 0x%llx data: %p getlk: %p\n",
+	       flags, data, getlk);
+
+	LASSERT(flags != LDLM_FL_WAIT_NOREPROC);
+
+	if (flags & LDLM_FL_FAILED)
+		goto granted;
+
+	if (!(flags & LDLM_FL_BLOCKED_MASK)) {
+		if (!data)
+			/* mds granted the lock in the reply */
+			goto granted;
+		/* CP AST RPC: lock get granted, wake it up */
+		wake_up(&lock->l_waitq);
+		return 0;
+	}
+
+	LDLM_DEBUG(lock,
+		   "client-side enqueue returned a blocked lock, sleeping");
+
+	/* Go to sleep until the lock is granted. */
+	rc = l_wait_event_abortable(lock->l_waitq, is_granted_or_cancelled(lock));
+
+	if (rc) {
+		lock_res_and_lock(lock);
+
+		/* client side - set flag to prevent lock from being put on LRU list */
+		ldlm_set_cbpending(lock);
+		unlock_res_and_lock(lock);
+
+		LDLM_DEBUG(lock, "client-side enqueue waking up: failed (%d)",
+			   rc);
+		return rc;
+	}
+
+granted:
+	OBD_FAIL_TIMEOUT(OBD_FAIL_LDLM_CP_CB_WAIT, 10);
+
+	if (OBD_FAIL_PRECHECK(OBD_FAIL_LDLM_CP_CB_WAIT4)) {
+		lock_res_and_lock(lock);
+		/* DEADLOCK is always set with CBPENDING */
+		lock->l_flags |= LDLM_FL_FLOCK_DEADLOCK | LDLM_FL_CBPENDING;
+		unlock_res_and_lock(lock);
+		OBD_FAIL_TIMEOUT(OBD_FAIL_LDLM_CP_CB_WAIT4, 4);
+	}
+	if (OBD_FAIL_PRECHECK(OBD_FAIL_LDLM_CP_CB_WAIT5)) {
+		lock_res_and_lock(lock);
+		/* DEADLOCK is always set with CBPENDING */
+		lock->l_flags |= LDLM_FL_FAIL_LOC |
+				 LDLM_FL_FLOCK_DEADLOCK | LDLM_FL_CBPENDING;
+		unlock_res_and_lock(lock);
+		OBD_FAIL_TIMEOUT(OBD_FAIL_LDLM_CP_CB_WAIT5, 4);
+	}
+
+	lock_res_and_lock(lock);
+
+	/*
+	 * Protect against race where lock could have been just destroyed
+	 * due to overlap in ldlm_process_flock_lock().
+	 */
+	if (ldlm_is_destroyed(lock)) {
+		unlock_res_and_lock(lock);
+		LDLM_DEBUG(lock, "client-side enqueue waking up: destroyed");
+		/*
+		 * An error is still to be returned, to propagate it up to
+		 * ldlm_cli_enqueue_fini() caller.
+		 */
+		return -EIO;
+	}
+
+	/* ldlm_lock_enqueue() has already placed lock on the granted list. */
+	ldlm_resource_unlink_lock(lock);
+
+	/*
+	 * Import invalidation. We need to actually release the lock
+	 * references being held, so that it can go away. No point in
+	 * holding the lock even if app still believes it has it, since
+	 * server already dropped it anyway. Only for granted locks too.
+	 */
+	/* Do the same for DEADLOCK'ed locks. */
+	if (ldlm_is_failed(lock) || ldlm_is_flock_deadlock(lock)) {
+		int mode;
+
+		if (flags & LDLM_FL_TEST_LOCK)
+			LASSERT(ldlm_is_test_lock(lock));
+
+		if (ldlm_is_test_lock(lock) || ldlm_is_flock_deadlock(lock))
+			mode = getlk->fl_type;
+		else
+			mode = lock->l_granted_mode;
+
+		if (ldlm_is_flock_deadlock(lock)) {
+			LDLM_DEBUG(lock,
+				   "client-side enqueue deadlock received");
+			rc = -EDEADLK;
+		}
+		ldlm_flock_destroy(lock, mode);
+		unlock_res_and_lock(lock);
+
+		/* Need to wake up the waiter if we were evicted */
+		wake_up(&lock->l_waitq);
+
+		/*
+		 * An error is still to be returned, to propagate it up to
+		 * ldlm_cli_enqueue_fini() caller.
+		 */
+		return rc ? : -EIO;
+	}
+
+	LDLM_DEBUG(lock, "client-side enqueue granted");
+
+	if (flags & LDLM_FL_TEST_LOCK) {
+		/* fcntl(F_GETLK) request */
+		/* The old mode was saved in getlk->fl_type so that if the mode
+		 * in the lock changes we can decref the appropriate refcount.
+		 */
+		LASSERT(ldlm_is_test_lock(lock));
+		ldlm_flock_destroy(lock, getlk->fl_type);
+		switch (lock->l_granted_mode) {
+		case LCK_PR:
+			getlk->fl_type = F_RDLCK;
+			break;
+		case LCK_PW:
+			getlk->fl_type = F_WRLCK;
+			break;
+		default:
+			getlk->fl_type = F_UNLCK;
+		}
+		getlk->fl_pid = -(pid_t)lock->l_policy_data.l_flock.pid;
+		getlk->fl_start = (loff_t)lock->l_policy_data.l_flock.start;
+		getlk->fl_end = (loff_t)lock->l_policy_data.l_flock.end;
+	} else {
+		/* We need to reprocess the lock to do merges or splits
+		 * with existing locks owned by this process.
+		 */
+		ldlm_process_flock_lock(lock);
+	}
+	unlock_res_and_lock(lock);
+	return rc;
+}
+EXPORT_SYMBOL(ldlm_flock_completion_ast);
+
+void ldlm_flock_policy_wire_to_local(const union ldlm_wire_policy_data *wpolicy,
+				     union ldlm_policy_data *lpolicy)
+{
+	lpolicy->l_flock.start = wpolicy->l_flock.lfw_start;
+	lpolicy->l_flock.end = wpolicy->l_flock.lfw_end;
+	lpolicy->l_flock.pid = wpolicy->l_flock.lfw_pid;
+	lpolicy->l_flock.owner = wpolicy->l_flock.lfw_owner;
+}
+
+void ldlm_flock_policy_local_to_wire(const union ldlm_policy_data *lpolicy,
+				     union ldlm_wire_policy_data *wpolicy)
+{
+	memset(wpolicy, 0, sizeof(*wpolicy));
+	wpolicy->l_flock.lfw_start = lpolicy->l_flock.start;
+	wpolicy->l_flock.lfw_end = lpolicy->l_flock.end;
+	wpolicy->l_flock.lfw_pid = lpolicy->l_flock.pid;
+	wpolicy->l_flock.lfw_owner = lpolicy->l_flock.owner;
+}
diff --git a/drivers/staging/lustre/lustre/ptlrpc/ldlm_inodebits.c b/drivers/staging/lustre/lustre/ptlrpc/ldlm_inodebits.c
new file mode 100644
index 000000000000..2926208cdfa1
--- /dev/null
+++ b/drivers/staging/lustre/lustre/ptlrpc/ldlm_inodebits.c
@@ -0,0 +1,69 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * GPL HEADER START
+ *
+ * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 only,
+ * as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License version 2 for more details (a copy is included
+ * in the LICENSE file that accompanied this code).
+ *
+ * You should have received a copy of the GNU General Public License
+ * version 2 along with this program; If not, see
+ * http://www.gnu.org/licenses/gpl-2.0.html
+ *
+ * GPL HEADER END
+ */
+/*
+ * Copyright (c) 2004, 2010, Oracle and/or its affiliates. All rights reserved.
+ * Use is subject to license terms.
+ *
+ * Copyright (c) 2011, Intel Corporation.
+ */
+/*
+ * This file is part of Lustre, http://www.lustre.org/
+ * Lustre is a trademark of Sun Microsystems, Inc.
+ *
+ * lustre/ldlm/ldlm_inodebits.c
+ *
+ * Author: Peter Braam <braam at clusterfs.com>
+ * Author: Phil Schwan <phil at clusterfs.com>
+ */
+
+/**
+ * This file contains implementation of IBITS lock type
+ *
+ * IBITS lock type contains a bit mask determining various properties of an
+ * object. The meanings of specific bits are specific to the caller and are
+ * opaque to LDLM code.
+ *
+ * Locks with intersecting bitmasks and conflicting lock modes (e.g.  LCK_PW)
+ * are considered conflicting.  See the lock mode compatibility matrix
+ * in lustre_dlm.h.
+ */
+
+#define DEBUG_SUBSYSTEM S_LDLM
+
+#include <lustre_dlm.h>
+#include <obd_support.h>
+#include <lustre_lib.h>
+#include "ldlm_internal.h"
+
+void ldlm_ibits_policy_wire_to_local(const union ldlm_wire_policy_data *wpolicy,
+				     union ldlm_policy_data *lpolicy)
+{
+	lpolicy->l_inodebits.bits = wpolicy->l_inodebits.bits;
+}
+
+void ldlm_ibits_policy_local_to_wire(const union ldlm_policy_data *lpolicy,
+				     union ldlm_wire_policy_data *wpolicy)
+{
+	memset(wpolicy, 0, sizeof(*wpolicy));
+	wpolicy->l_inodebits.bits = lpolicy->l_inodebits.bits;
+}
diff --git a/drivers/staging/lustre/lustre/ptlrpc/ldlm_internal.h b/drivers/staging/lustre/lustre/ptlrpc/ldlm_internal.h
new file mode 100644
index 000000000000..60a15b963c8a
--- /dev/null
+++ b/drivers/staging/lustre/lustre/ptlrpc/ldlm_internal.h
@@ -0,0 +1,329 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * GPL HEADER START
+ *
+ * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 only,
+ * as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License version 2 for more details (a copy is included
+ * in the LICENSE file that accompanied this code).
+ *
+ * You should have received a copy of the GNU General Public License
+ * version 2 along with this program; If not, see
+ * http://www.gnu.org/licenses/gpl-2.0.html
+ *
+ * GPL HEADER END
+ */
+/*
+ * Copyright (c) 2003, 2010, Oracle and/or its affiliates. All rights reserved.
+ * Use is subject to license terms.
+ *
+ * Copyright (c) 2011, 2015, Intel Corporation.
+ */
+/*
+ * This file is part of Lustre, http://www.lustre.org/
+ * Lustre is a trademark of Sun Microsystems, Inc.
+ */
+
+#define MAX_STRING_SIZE 128
+
+extern int ldlm_srv_namespace_nr;
+extern int ldlm_cli_namespace_nr;
+extern struct mutex ldlm_srv_namespace_lock;
+extern struct list_head ldlm_srv_namespace_list;
+extern struct mutex ldlm_cli_namespace_lock;
+extern struct list_head ldlm_cli_active_namespace_list;
+
+static inline int ldlm_namespace_nr_read(enum ldlm_side client)
+{
+	return client == LDLM_NAMESPACE_SERVER ?
+		ldlm_srv_namespace_nr : ldlm_cli_namespace_nr;
+}
+
+static inline void ldlm_namespace_nr_inc(enum ldlm_side client)
+{
+	if (client == LDLM_NAMESPACE_SERVER)
+		ldlm_srv_namespace_nr++;
+	else
+		ldlm_cli_namespace_nr++;
+}
+
+static inline void ldlm_namespace_nr_dec(enum ldlm_side client)
+{
+	if (client == LDLM_NAMESPACE_SERVER)
+		ldlm_srv_namespace_nr--;
+	else
+		ldlm_cli_namespace_nr--;
+}
+
+static inline struct list_head *ldlm_namespace_list(enum ldlm_side client)
+{
+	return client == LDLM_NAMESPACE_SERVER ?
+		&ldlm_srv_namespace_list : &ldlm_cli_active_namespace_list;
+}
+
+static inline struct mutex *ldlm_namespace_lock(enum ldlm_side client)
+{
+	return client == LDLM_NAMESPACE_SERVER ?
+		&ldlm_srv_namespace_lock : &ldlm_cli_namespace_lock;
+}
+
+/* ns_bref is the number of resources in this namespace */
+static inline int ldlm_ns_empty(struct ldlm_namespace *ns)
+{
+	return atomic_read(&ns->ns_bref) == 0;
+}
+
+void ldlm_namespace_move_to_active_locked(struct ldlm_namespace *ns,
+					  enum ldlm_side client);
+void ldlm_namespace_move_to_inactive_locked(struct ldlm_namespace *ns,
+					    enum ldlm_side client);
+struct ldlm_namespace *ldlm_namespace_first_locked(enum ldlm_side client);
+
+/* ldlm_request.c */
+/* Cancel lru flag, it indicates we cancel aged locks. */
+enum {
+	LDLM_LRU_FLAG_AGED	= BIT(0), /* Cancel old non-LRU resize locks */
+	LDLM_LRU_FLAG_PASSED	= BIT(1), /* Cancel passed number of locks. */
+	LDLM_LRU_FLAG_SHRINK	= BIT(2), /* Cancel locks from shrinker. */
+	LDLM_LRU_FLAG_LRUR	= BIT(3), /* Cancel locks from lru resize. */
+	LDLM_LRU_FLAG_NO_WAIT	= BIT(4), /* Cancel locks w/o blocking (neither
+					   * sending nor waiting for any rpcs)
+					   */
+	LDLM_LRU_FLAG_LRUR_NO_WAIT = BIT(5), /* LRUR + NO_WAIT */
+};
+
+int ldlm_cancel_lru(struct ldlm_namespace *ns, int nr,
+		    enum ldlm_cancel_flags sync, int flags);
+int ldlm_cancel_lru_local(struct ldlm_namespace *ns,
+			  struct list_head *cancels, int count, int max,
+			  enum ldlm_cancel_flags cancel_flags, int flags);
+extern unsigned int ldlm_enqueue_min;
+extern unsigned int ldlm_cancel_unused_locks_before_replay;
+
+/* ldlm_lock.c */
+
+struct ldlm_cb_set_arg {
+	struct ptlrpc_request_set	*set;
+	int				 type; /* LDLM_{CP,BL,GL}_CALLBACK */
+	atomic_t			 restart;
+	struct list_head			*list;
+	union ldlm_gl_desc		*gl_desc; /* glimpse AST descriptor */
+};
+
+enum ldlm_desc_ast_t {
+	LDLM_WORK_BL_AST,
+	LDLM_WORK_CP_AST,
+	LDLM_WORK_REVOKE_AST,
+	LDLM_WORK_GL_AST
+};
+
+void ldlm_grant_lock(struct ldlm_lock *lock, struct list_head *work_list);
+int ldlm_fill_lvb(struct ldlm_lock *lock, struct req_capsule *pill,
+		  enum req_location loc, void *data, int size);
+struct ldlm_lock *
+ldlm_lock_create(struct ldlm_namespace *ns, const struct ldlm_res_id *id,
+		 enum ldlm_type type, enum ldlm_mode mode,
+		 const struct ldlm_callback_suite *cbs,
+		 void *data, __u32 lvb_len, enum lvb_type lvb_type);
+enum ldlm_error ldlm_lock_enqueue(struct ldlm_namespace *ns,
+				  struct ldlm_lock **lock, void *cookie,
+				  __u64 *flags);
+void ldlm_lock_addref_internal(struct ldlm_lock *lock, enum ldlm_mode mode);
+void ldlm_lock_addref_internal_nolock(struct ldlm_lock *lock,
+				      enum ldlm_mode mode);
+void ldlm_lock_decref_internal(struct ldlm_lock *lock, enum ldlm_mode mode);
+void ldlm_lock_decref_internal_nolock(struct ldlm_lock *lock,
+				      enum ldlm_mode mode);
+int ldlm_run_ast_work(struct ldlm_namespace *ns, struct list_head *rpc_list,
+		      enum ldlm_desc_ast_t ast_type);
+int ldlm_lock_remove_from_lru_check(struct ldlm_lock *lock, time_t last_use);
+#define ldlm_lock_remove_from_lru(lock) ldlm_lock_remove_from_lru_check(lock, 0)
+int ldlm_lock_remove_from_lru_nolock(struct ldlm_lock *lock);
+void ldlm_lock_destroy_nolock(struct ldlm_lock *lock);
+
+/* ldlm_lockd.c */
+int ldlm_bl_to_thread_lock(struct ldlm_namespace *ns, struct ldlm_lock_desc *ld,
+			   struct ldlm_lock *lock);
+int ldlm_bl_to_thread_list(struct ldlm_namespace *ns,
+			   struct ldlm_lock_desc *ld,
+			   struct list_head *cancels, int count,
+			   enum ldlm_cancel_flags cancel_flags);
+int ldlm_bl_thread_wakeup(void);
+
+void ldlm_handle_bl_callback(struct ldlm_namespace *ns,
+			     struct ldlm_lock_desc *ld, struct ldlm_lock *lock);
+
+extern struct kmem_cache *ldlm_resource_slab;
+extern struct kset *ldlm_ns_kset;
+
+/* ldlm_lockd.c & ldlm_lock.c */
+extern struct kmem_cache *ldlm_lock_slab;
+
+/* ldlm_extent.c */
+void ldlm_extent_add_lock(struct ldlm_resource *res, struct ldlm_lock *lock);
+void ldlm_extent_unlink_lock(struct ldlm_lock *lock);
+void ldlm_extent_search(struct rb_root_cached *root,
+			__u64 start, __u64 end,
+			bool (*matches)(struct ldlm_lock *lock, void *data),
+			void *data);
+
+/* l_lock.c */
+void l_check_ns_lock(struct ldlm_namespace *ns);
+void l_check_no_ns_lock(struct ldlm_namespace *ns);
+
+extern struct dentry *ldlm_svc_debugfs_dir;
+
+struct ldlm_state {
+	struct ptlrpc_service *ldlm_cb_service;
+	struct ptlrpc_service *ldlm_cancel_service;
+	struct ptlrpc_client *ldlm_client;
+	struct ptlrpc_connection *ldlm_server_conn;
+	struct ldlm_bl_pool *ldlm_bl_pool;
+};
+
+/* ldlm_pool.c */
+__u64 ldlm_pool_get_slv(struct ldlm_pool *pl);
+void ldlm_pool_set_clv(struct ldlm_pool *pl, __u64 clv);
+__u32 ldlm_pool_get_lvf(struct ldlm_pool *pl);
+
+int ldlm_init(void);
+void ldlm_exit(void);
+
+enum ldlm_policy_res {
+	LDLM_POLICY_CANCEL_LOCK,
+	LDLM_POLICY_KEEP_LOCK,
+	LDLM_POLICY_SKIP_LOCK
+};
+
+#define LDLM_POOL_SYSFS_PRINT_int(v) sprintf(buf, "%d\n", v)
+#define LDLM_POOL_SYSFS_SET_int(a, b) { a = b; }
+#define LDLM_POOL_SYSFS_PRINT_u64(v) sprintf(buf, "%lld\n", v)
+#define LDLM_POOL_SYSFS_SET_u64(a, b) { a = b; }
+#define LDLM_POOL_SYSFS_PRINT_atomic(v) sprintf(buf, "%d\n", atomic_read(&v))
+#define LDLM_POOL_SYSFS_SET_atomic(a, b) atomic_set(&a, b)
+
+#define LDLM_POOL_SYSFS_READER_SHOW(var, type)				    \
+	static ssize_t var##_show(struct kobject *kobj,			    \
+				  struct attribute *attr,		    \
+				  char *buf)				    \
+	{								    \
+		struct ldlm_pool *pl = container_of(kobj, struct ldlm_pool, \
+						    pl_kobj);		    \
+		type tmp;						    \
+									    \
+		spin_lock(&pl->pl_lock);				    \
+		tmp = pl->pl_##var;					    \
+		spin_unlock(&pl->pl_lock);				    \
+									    \
+		return LDLM_POOL_SYSFS_PRINT_##type(tmp);		    \
+	}								    \
+	struct __##var##__dummy_read {; } /* semicolon catcher */
+
+#define LDLM_POOL_SYSFS_WRITER_STORE(var, type)				    \
+	static ssize_t var##_store(struct kobject *kobj,		    \
+				     struct attribute *attr,		    \
+				     const char *buffer,		    \
+				     size_t count)			    \
+	{								    \
+		struct ldlm_pool *pl = container_of(kobj, struct ldlm_pool, \
+						    pl_kobj);		    \
+		unsigned long tmp;					    \
+		int rc;							    \
+									    \
+		rc = kstrtoul(buffer, 10, &tmp);			    \
+		if (rc < 0) {						    \
+			return rc;					    \
+		}							    \
+									    \
+		spin_lock(&pl->pl_lock);				    \
+		LDLM_POOL_SYSFS_SET_##type(pl->pl_##var, tmp);		    \
+		spin_unlock(&pl->pl_lock);				    \
+									    \
+		return count;						    \
+	}								    \
+	struct __##var##__dummy_write {; } /* semicolon catcher */
+
+#define LDLM_POOL_SYSFS_READER_NOLOCK_SHOW(var, type)			    \
+	static ssize_t var##_show(struct kobject *kobj,		    \
+				    struct attribute *attr,		    \
+				    char *buf)				    \
+	{								    \
+		struct ldlm_pool *pl = container_of(kobj, struct ldlm_pool, \
+						    pl_kobj);		    \
+									    \
+		return LDLM_POOL_SYSFS_PRINT_##type(pl->pl_##var);	    \
+	}								    \
+	struct __##var##__dummy_read {; } /* semicolon catcher */
+
+#define LDLM_POOL_SYSFS_WRITER_NOLOCK_STORE(var, type)			    \
+	static ssize_t var##_store(struct kobject *kobj,		    \
+				     struct attribute *attr,		    \
+				     const char *buffer,		    \
+				     size_t count)			    \
+	{								    \
+		struct ldlm_pool *pl = container_of(kobj, struct ldlm_pool, \
+						    pl_kobj);		    \
+		unsigned long tmp;					    \
+		int rc;							    \
+									    \
+		rc = kstrtoul(buffer, 10, &tmp);			    \
+		if (rc < 0) {						    \
+			return rc;					    \
+		}							    \
+									    \
+		LDLM_POOL_SYSFS_SET_##type(pl->pl_##var, tmp);		    \
+									    \
+		return count;						    \
+	}								    \
+	struct __##var##__dummy_write {; } /* semicolon catcher */
+
+static inline int is_granted_or_cancelled(struct ldlm_lock *lock)
+{
+	int ret = 0;
+
+	lock_res_and_lock(lock);
+	if ((lock->l_req_mode == lock->l_granted_mode) &&
+	    !ldlm_is_cp_reqd(lock))
+		ret = 1;
+	else if (ldlm_is_failed(lock) || ldlm_is_cancel(lock))
+		ret = 1;
+	unlock_res_and_lock(lock);
+
+	return ret;
+}
+
+typedef void (*ldlm_policy_wire_to_local_t)(const union ldlm_wire_policy_data *,
+					    union ldlm_policy_data *);
+
+typedef void (*ldlm_policy_local_to_wire_t)(const union ldlm_policy_data *,
+					    union ldlm_wire_policy_data *);
+
+void ldlm_plain_policy_wire_to_local(const union ldlm_wire_policy_data *wpolicy,
+				     union ldlm_policy_data *lpolicy);
+void ldlm_plain_policy_local_to_wire(const union ldlm_policy_data *lpolicy,
+				     union ldlm_wire_policy_data *wpolicy);
+void ldlm_ibits_policy_wire_to_local(const union ldlm_wire_policy_data *wpolicy,
+				     union ldlm_policy_data *lpolicy);
+void ldlm_ibits_policy_local_to_wire(const union ldlm_policy_data *lpolicy,
+				     union ldlm_wire_policy_data *wpolicy);
+void ldlm_extent_policy_wire_to_local(const union ldlm_wire_policy_data *wpolicy,
+				      union ldlm_policy_data *lpolicy);
+void ldlm_extent_policy_local_to_wire(const union ldlm_policy_data *lpolicy,
+				      union ldlm_wire_policy_data *wpolicy);
+void ldlm_flock_policy_wire_to_local(const union ldlm_wire_policy_data *wpolicy,
+				     union ldlm_policy_data *lpolicy);
+void ldlm_flock_policy_local_to_wire(const union ldlm_policy_data *lpolicy,
+				     union ldlm_wire_policy_data *wpolicy);
+
+static inline bool ldlm_res_eq(const struct ldlm_res_id *res0,
+			       const struct ldlm_res_id *res1)
+{
+	return memcmp(res0, res1, sizeof(*res0)) == 0;
+}
diff --git a/drivers/staging/lustre/lustre/ptlrpc/ldlm_lib.c b/drivers/staging/lustre/lustre/ptlrpc/ldlm_lib.c
new file mode 100644
index 000000000000..0aa4f234a4f4
--- /dev/null
+++ b/drivers/staging/lustre/lustre/ptlrpc/ldlm_lib.c
@@ -0,0 +1,842 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * GPL HEADER START
+ *
+ * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 only,
+ * as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License version 2 for more details (a copy is included
+ * in the LICENSE file that accompanied this code).
+ *
+ * You should have received a copy of the GNU General Public License
+ * version 2 along with this program; If not, see
+ * http://www.gnu.org/licenses/gpl-2.0.html
+ *
+ * GPL HEADER END
+ */
+/*
+ * Copyright (c) 2003, 2010, Oracle and/or its affiliates. All rights reserved.
+ * Use is subject to license terms.
+ *
+ * Copyright (c) 2010, 2015, Intel Corporation.
+ */
+/*
+ * This file is part of Lustre, http://www.lustre.org/
+ * Lustre is a trademark of Sun Microsystems, Inc.
+ */
+
+/**
+ * This file deals with various client/target related logic including recovery.
+ *
+ * TODO: This code more logically belongs in the ptlrpc module than in ldlm and
+ * should be moved.
+ */
+
+#define DEBUG_SUBSYSTEM S_LDLM
+
+#include <obd.h>
+#include <obd_class.h>
+#include <lustre_dlm.h>
+#include <lustre_net.h>
+#include <lustre_sec.h>
+#include "ldlm_internal.h"
+
+/* @priority: If non-zero, move the selected connection to the list head.
+ * @create: If zero, only search in existing connections.
+ */
+static int import_set_conn(struct obd_import *imp, struct obd_uuid *uuid,
+			   int priority, int create)
+{
+	struct ptlrpc_connection *ptlrpc_conn;
+	struct obd_import_conn *imp_conn = NULL, *item;
+	int rc = 0;
+
+	if (!create && !priority) {
+		CDEBUG(D_HA, "Nothing to do\n");
+		return -EINVAL;
+	}
+
+	ptlrpc_conn = ptlrpc_uuid_to_connection(uuid);
+	if (!ptlrpc_conn) {
+		CDEBUG(D_HA, "can't find connection %s\n", uuid->uuid);
+		return -ENOENT;
+	}
+
+	if (create) {
+		imp_conn = kzalloc(sizeof(*imp_conn), GFP_NOFS);
+		if (!imp_conn) {
+			rc = -ENOMEM;
+			goto out_put;
+		}
+	}
+
+	spin_lock(&imp->imp_lock);
+	list_for_each_entry(item, &imp->imp_conn_list, oic_item) {
+		if (obd_uuid_equals(uuid, &item->oic_uuid)) {
+			if (priority) {
+				list_del(&item->oic_item);
+				list_add(&item->oic_item,
+					 &imp->imp_conn_list);
+				item->oic_last_attempt = 0;
+			}
+			CDEBUG(D_HA, "imp %p@%s: found existing conn %s%s\n",
+			       imp, imp->imp_obd->obd_name, uuid->uuid,
+			       (priority ? ", moved to head" : ""));
+			spin_unlock(&imp->imp_lock);
+			rc = 0;
+			goto out_free;
+		}
+	}
+	/* No existing import connection found for \a uuid. */
+	if (create) {
+		imp_conn->oic_conn = ptlrpc_conn;
+		imp_conn->oic_uuid = *uuid;
+		imp_conn->oic_last_attempt = 0;
+		if (priority)
+			list_add(&imp_conn->oic_item, &imp->imp_conn_list);
+		else
+			list_add_tail(&imp_conn->oic_item,
+				      &imp->imp_conn_list);
+		CDEBUG(D_HA, "imp %p@%s: add connection %s at %s\n",
+		       imp, imp->imp_obd->obd_name, uuid->uuid,
+		       (priority ? "head" : "tail"));
+	} else {
+		spin_unlock(&imp->imp_lock);
+		rc = -ENOENT;
+		goto out_free;
+	}
+
+	spin_unlock(&imp->imp_lock);
+	return 0;
+out_free:
+	kfree(imp_conn);
+out_put:
+	ptlrpc_connection_put(ptlrpc_conn);
+	return rc;
+}
+
+int import_set_conn_priority(struct obd_import *imp, struct obd_uuid *uuid)
+{
+	return import_set_conn(imp, uuid, 1, 0);
+}
+
+int client_import_add_conn(struct obd_import *imp, struct obd_uuid *uuid,
+			   int priority)
+{
+	return import_set_conn(imp, uuid, priority, 1);
+}
+EXPORT_SYMBOL(client_import_add_conn);
+
+int client_import_del_conn(struct obd_import *imp, struct obd_uuid *uuid)
+{
+	struct obd_import_conn *imp_conn;
+	struct obd_export *dlmexp;
+	int rc = -ENOENT;
+
+	spin_lock(&imp->imp_lock);
+	if (list_empty(&imp->imp_conn_list)) {
+		LASSERT(!imp->imp_connection);
+		goto out;
+	}
+
+	list_for_each_entry(imp_conn, &imp->imp_conn_list, oic_item) {
+		if (!obd_uuid_equals(uuid, &imp_conn->oic_uuid))
+			continue;
+		LASSERT(imp_conn->oic_conn);
+
+		if (imp_conn == imp->imp_conn_current) {
+			LASSERT(imp_conn->oic_conn == imp->imp_connection);
+
+			if (imp->imp_state != LUSTRE_IMP_CLOSED &&
+			    imp->imp_state != LUSTRE_IMP_DISCON) {
+				CERROR("can't remove current connection\n");
+				rc = -EBUSY;
+				goto out;
+			}
+
+			ptlrpc_connection_put(imp->imp_connection);
+			imp->imp_connection = NULL;
+
+			dlmexp = class_conn2export(&imp->imp_dlm_handle);
+			if (dlmexp && dlmexp->exp_connection) {
+				LASSERT(dlmexp->exp_connection ==
+					imp_conn->oic_conn);
+				ptlrpc_connection_put(dlmexp->exp_connection);
+				dlmexp->exp_connection = NULL;
+			}
+
+			if (dlmexp)
+				class_export_put(dlmexp);
+		}
+
+		list_del(&imp_conn->oic_item);
+		ptlrpc_connection_put(imp_conn->oic_conn);
+		kfree(imp_conn);
+		CDEBUG(D_HA, "imp %p@%s: remove connection %s\n",
+		       imp, imp->imp_obd->obd_name, uuid->uuid);
+		rc = 0;
+		break;
+	}
+out:
+	spin_unlock(&imp->imp_lock);
+	if (rc == -ENOENT)
+		CERROR("connection %s not found\n", uuid->uuid);
+	return rc;
+}
+EXPORT_SYMBOL(client_import_del_conn);
+
+/**
+ * Find conn UUID by peer NID. \a peer is a server NID. This function is used
+ * to find a conn uuid of \a imp which can reach \a peer.
+ */
+int client_import_find_conn(struct obd_import *imp, lnet_nid_t peer,
+			    struct obd_uuid *uuid)
+{
+	struct obd_import_conn *conn;
+	int rc = -ENOENT;
+
+	spin_lock(&imp->imp_lock);
+	list_for_each_entry(conn, &imp->imp_conn_list, oic_item) {
+		/* Check if conn UUID does have this peer NID. */
+		if (class_check_uuid(&conn->oic_uuid, peer)) {
+			*uuid = conn->oic_uuid;
+			rc = 0;
+			break;
+		}
+	}
+	spin_unlock(&imp->imp_lock);
+	return rc;
+}
+EXPORT_SYMBOL(client_import_find_conn);
+
+void client_destroy_import(struct obd_import *imp)
+{
+	/* Drop security policy instance after all RPCs have finished/aborted
+	 * to let all busy contexts be released.
+	 */
+	class_import_get(imp);
+	class_destroy_import(imp);
+	sptlrpc_import_sec_put(imp);
+	class_import_put(imp);
+}
+EXPORT_SYMBOL(client_destroy_import);
+
+/* Configure an RPC client OBD device.
+ *
+ * lcfg parameters:
+ * 1 - client UUID
+ * 2 - server UUID
+ * 3 - inactive-on-startup
+ */
+int client_obd_setup(struct obd_device *obddev, struct lustre_cfg *lcfg)
+{
+	struct client_obd *cli = &obddev->u.cli;
+	struct obd_import *imp;
+	struct obd_uuid server_uuid;
+	int rq_portal, rp_portal, connect_op;
+	char *name = obddev->obd_type->typ_name;
+	enum ldlm_ns_type ns_type = LDLM_NS_TYPE_UNKNOWN;
+	int rc;
+
+	/* In a more perfect world, we would hang a ptlrpc_client off of
+	 * obd_type and just use the values from there.
+	 */
+	if (!strcmp(name, LUSTRE_OSC_NAME)) {
+		rq_portal = OST_REQUEST_PORTAL;
+		rp_portal = OSC_REPLY_PORTAL;
+		connect_op = OST_CONNECT;
+		cli->cl_sp_me = LUSTRE_SP_CLI;
+		cli->cl_sp_to = LUSTRE_SP_OST;
+		ns_type = LDLM_NS_TYPE_OSC;
+	} else if (!strcmp(name, LUSTRE_MDC_NAME) ||
+		   !strcmp(name, LUSTRE_LWP_NAME)) {
+		rq_portal = MDS_REQUEST_PORTAL;
+		rp_portal = MDC_REPLY_PORTAL;
+		connect_op = MDS_CONNECT;
+		cli->cl_sp_me = LUSTRE_SP_CLI;
+		cli->cl_sp_to = LUSTRE_SP_MDT;
+		ns_type = LDLM_NS_TYPE_MDC;
+	} else if (!strcmp(name, LUSTRE_MGC_NAME)) {
+		rq_portal = MGS_REQUEST_PORTAL;
+		rp_portal = MGC_REPLY_PORTAL;
+		connect_op = MGS_CONNECT;
+		cli->cl_sp_me = LUSTRE_SP_MGC;
+		cli->cl_sp_to = LUSTRE_SP_MGS;
+		cli->cl_flvr_mgc.sf_rpc = SPTLRPC_FLVR_INVALID;
+		ns_type = LDLM_NS_TYPE_MGC;
+	} else {
+		CERROR("unknown client OBD type \"%s\", can't setup\n",
+		       name);
+		return -EINVAL;
+	}
+
+	if (LUSTRE_CFG_BUFLEN(lcfg, 1) < 1) {
+		CERROR("requires a TARGET UUID\n");
+		return -EINVAL;
+	}
+
+	if (LUSTRE_CFG_BUFLEN(lcfg, 1) > 37) {
+		CERROR("client UUID must be less than 38 characters\n");
+		return -EINVAL;
+	}
+
+	if (LUSTRE_CFG_BUFLEN(lcfg, 2) < 1) {
+		CERROR("setup requires a SERVER UUID\n");
+		return -EINVAL;
+	}
+
+	if (LUSTRE_CFG_BUFLEN(lcfg, 2) > 37) {
+		CERROR("target UUID must be less than 38 characters\n");
+		return -EINVAL;
+	}
+
+	init_rwsem(&cli->cl_sem);
+	cli->cl_conn_count = 0;
+	memcpy(server_uuid.uuid, lustre_cfg_buf(lcfg, 2),
+	       min_t(unsigned int, LUSTRE_CFG_BUFLEN(lcfg, 2),
+		     sizeof(server_uuid)));
+
+	cli->cl_dirty_pages = 0;
+	cli->cl_avail_grant = 0;
+	/* FIXME: Should limit this for the sum of all cl_dirty_max_pages. */
+	/*
+	 * cl_dirty_max_pages may be changed at connect time in
+	 * ptlrpc_connect_interpret().
+	 */
+	client_adjust_max_dirty(cli);
+	INIT_LIST_HEAD(&cli->cl_cache_waiters);
+	INIT_LIST_HEAD(&cli->cl_loi_ready_list);
+	INIT_LIST_HEAD(&cli->cl_loi_hp_ready_list);
+	INIT_LIST_HEAD(&cli->cl_loi_write_list);
+	INIT_LIST_HEAD(&cli->cl_loi_read_list);
+	spin_lock_init(&cli->cl_loi_list_lock);
+	atomic_set(&cli->cl_pending_w_pages, 0);
+	atomic_set(&cli->cl_pending_r_pages, 0);
+	cli->cl_r_in_flight = 0;
+	cli->cl_w_in_flight = 0;
+
+	spin_lock_init(&cli->cl_read_rpc_hist.oh_lock);
+	spin_lock_init(&cli->cl_write_rpc_hist.oh_lock);
+	spin_lock_init(&cli->cl_read_page_hist.oh_lock);
+	spin_lock_init(&cli->cl_write_page_hist.oh_lock);
+	spin_lock_init(&cli->cl_read_offset_hist.oh_lock);
+	spin_lock_init(&cli->cl_write_offset_hist.oh_lock);
+
+	/* lru for osc. */
+	INIT_LIST_HEAD(&cli->cl_lru_osc);
+	atomic_set(&cli->cl_lru_shrinkers, 0);
+	atomic_long_set(&cli->cl_lru_busy, 0);
+	atomic_long_set(&cli->cl_lru_in_list, 0);
+	INIT_LIST_HEAD(&cli->cl_lru_list);
+	spin_lock_init(&cli->cl_lru_list_lock);
+	atomic_long_set(&cli->cl_unstable_count, 0);
+	INIT_LIST_HEAD(&cli->cl_shrink_list);
+
+	init_waitqueue_head(&cli->cl_destroy_waitq);
+	atomic_set(&cli->cl_destroy_in_flight, 0);
+	/* Turn on checksumming by default. */
+	cli->cl_checksum = 1;
+	/*
+	 * The supported checksum types will be worked out at connect time
+	 * Set cl_chksum* to CRC32 for now to avoid returning screwed info
+	 * through procfs.
+	 */
+	cli->cl_cksum_type = OBD_CKSUM_CRC32;
+	cli->cl_supp_cksum_types = OBD_CKSUM_CRC32;
+	atomic_set(&cli->cl_resends, OSC_DEFAULT_RESENDS);
+
+	/*
+	 * Set it to possible maximum size. It may be reduced by ocd_brw_size
+	 * from OFD after connecting.
+	 */
+	cli->cl_max_pages_per_rpc = PTLRPC_MAX_BRW_PAGES;
+
+	/*
+	 * set cl_chunkbits default value to PAGE_CACHE_SHIFT,
+	 * it will be updated at OSC connection time.
+	 */
+	cli->cl_chunkbits = PAGE_SHIFT;
+
+	if (!strcmp(name, LUSTRE_MDC_NAME))
+		cli->cl_max_rpcs_in_flight = OBD_MAX_RIF_DEFAULT;
+	else if (totalram_pages >> (20 - PAGE_SHIFT) <= 128 /* MB */)
+		cli->cl_max_rpcs_in_flight = 2;
+	else if (totalram_pages >> (20 - PAGE_SHIFT) <= 256 /* MB */)
+		cli->cl_max_rpcs_in_flight = 3;
+	else if (totalram_pages >> (20 - PAGE_SHIFT) <= 512 /* MB */)
+		cli->cl_max_rpcs_in_flight = 4;
+	else
+		cli->cl_max_rpcs_in_flight = OBD_MAX_RIF_DEFAULT;
+
+	spin_lock_init(&cli->cl_mod_rpcs_lock);
+	spin_lock_init(&cli->cl_mod_rpcs_hist.oh_lock);
+	cli->cl_max_mod_rpcs_in_flight = 0;
+	cli->cl_mod_rpcs_in_flight = 0;
+	cli->cl_close_rpcs_in_flight = 0;
+	init_waitqueue_head(&cli->cl_mod_rpcs_waitq);
+	cli->cl_mod_tag_bitmap = NULL;
+
+	if (connect_op == MDS_CONNECT) {
+		cli->cl_max_mod_rpcs_in_flight = cli->cl_max_rpcs_in_flight - 1;
+		cli->cl_mod_tag_bitmap = kcalloc(BITS_TO_LONGS(OBD_MAX_RIF_MAX),
+						 sizeof(long), GFP_NOFS);
+		if (!cli->cl_mod_tag_bitmap) {
+			rc = -ENOMEM;
+			goto err;
+		}
+	}
+
+	rc = ldlm_get_ref();
+	if (rc) {
+		CERROR("ldlm_get_ref failed: %d\n", rc);
+		goto err;
+	}
+
+	ptlrpc_init_client(rq_portal, rp_portal, name,
+			   &obddev->obd_ldlm_client);
+
+	imp = class_new_import(obddev);
+	if (!imp) {
+		rc = -ENOENT;
+		goto err_ldlm;
+	}
+	imp->imp_client = &obddev->obd_ldlm_client;
+	imp->imp_connect_op = connect_op;
+	memcpy(cli->cl_target_uuid.uuid, lustre_cfg_buf(lcfg, 1),
+	       LUSTRE_CFG_BUFLEN(lcfg, 1));
+	class_import_put(imp);
+
+	rc = client_import_add_conn(imp, &server_uuid, 1);
+	if (rc) {
+		CERROR("can't add initial connection\n");
+		goto err_import;
+	}
+
+	cli->cl_import = imp;
+	/* cli->cl_max_mds_easize updated by mdc_init_ea_size() */
+	cli->cl_max_mds_easize = sizeof(struct lov_mds_md_v3);
+
+	if (LUSTRE_CFG_BUFLEN(lcfg, 3) > 0) {
+		if (!strcmp(lustre_cfg_string(lcfg, 3), "inactive")) {
+			CDEBUG(D_HA, "marking %s %s->%s as inactive\n",
+			       name, obddev->obd_name,
+			       cli->cl_target_uuid.uuid);
+			spin_lock(&imp->imp_lock);
+			imp->imp_deactive = 1;
+			spin_unlock(&imp->imp_lock);
+		}
+	}
+
+	obddev->obd_namespace = ldlm_namespace_new(obddev, obddev->obd_name,
+						   LDLM_NAMESPACE_CLIENT,
+						   LDLM_NAMESPACE_GREEDY,
+						   ns_type);
+	if (!obddev->obd_namespace) {
+		CERROR("Unable to create client namespace - %s\n",
+		       obddev->obd_name);
+		rc = -ENOMEM;
+		goto err_import;
+	}
+
+	return rc;
+
+err_import:
+	class_destroy_import(imp);
+err_ldlm:
+	ldlm_put_ref();
+err:
+	kfree(cli->cl_mod_tag_bitmap);
+	cli->cl_mod_tag_bitmap = NULL;
+	return rc;
+}
+EXPORT_SYMBOL(client_obd_setup);
+
+int client_obd_cleanup(struct obd_device *obddev)
+{
+	struct client_obd *cli = &obddev->u.cli;
+
+	ldlm_namespace_free_post(obddev->obd_namespace);
+	obddev->obd_namespace = NULL;
+
+	obd_cleanup_client_import(obddev);
+	LASSERT(!obddev->u.cli.cl_import);
+
+	ldlm_put_ref();
+
+	kfree(cli->cl_mod_tag_bitmap);
+	cli->cl_mod_tag_bitmap = NULL;
+
+	return 0;
+}
+EXPORT_SYMBOL(client_obd_cleanup);
+
+/* ->o_connect() method for client side (OSC and MDC and MGC) */
+int client_connect_import(const struct lu_env *env,
+			  struct obd_export **exp,
+			  struct obd_device *obd, struct obd_uuid *cluuid,
+			  struct obd_connect_data *data, void *localdata)
+{
+	struct client_obd       *cli    = &obd->u.cli;
+	struct obd_import       *imp    = cli->cl_import;
+	struct obd_connect_data *ocd;
+	struct lustre_handle    conn    = { 0 };
+	bool is_mdc = false;
+	int		     rc;
+
+	*exp = NULL;
+	down_write(&cli->cl_sem);
+	if (cli->cl_conn_count > 0) {
+		rc = -EALREADY;
+		goto out_sem;
+	}
+
+	rc = class_connect(&conn, obd, cluuid);
+	if (rc)
+		goto out_sem;
+
+	cli->cl_conn_count++;
+	*exp = class_conn2export(&conn);
+
+	LASSERT(obd->obd_namespace);
+
+	imp->imp_dlm_handle = conn;
+	rc = ptlrpc_init_import(imp);
+	if (rc != 0)
+		goto out_ldlm;
+
+	ocd = &imp->imp_connect_data;
+	if (data) {
+		*ocd = *data;
+		is_mdc = !strncmp(imp->imp_obd->obd_type->typ_name,
+				  LUSTRE_MDC_NAME, 3);
+		if (is_mdc)
+			data->ocd_connect_flags |= OBD_CONNECT_MULTIMODRPCS;
+		imp->imp_connect_flags_orig = data->ocd_connect_flags;
+	}
+
+	rc = ptlrpc_connect_import(imp);
+	if (rc != 0) {
+		if (data && is_mdc)
+			data->ocd_connect_flags &= ~OBD_CONNECT_MULTIMODRPCS;
+		LASSERT(imp->imp_state == LUSTRE_IMP_DISCON);
+		goto out_ldlm;
+	}
+	LASSERT(*exp && (*exp)->exp_connection);
+
+	if (data) {
+		LASSERTF((ocd->ocd_connect_flags & data->ocd_connect_flags) ==
+			 ocd->ocd_connect_flags, "old %#llx, new %#llx\n",
+			 data->ocd_connect_flags, ocd->ocd_connect_flags);
+		data->ocd_connect_flags = ocd->ocd_connect_flags;
+		/* clear the flag as it was not set and is not known
+		 * by upper layers
+		 */
+		if (is_mdc)
+			data->ocd_connect_flags &= ~OBD_CONNECT_MULTIMODRPCS;
+	}
+
+	ptlrpc_pinger_add_import(imp);
+
+	if (rc) {
+out_ldlm:
+		cli->cl_conn_count--;
+		class_disconnect(*exp);
+		*exp = NULL;
+	}
+out_sem:
+	up_write(&cli->cl_sem);
+
+	return rc;
+}
+EXPORT_SYMBOL(client_connect_import);
+
+int client_disconnect_export(struct obd_export *exp)
+{
+	struct obd_device *obd = class_exp2obd(exp);
+	struct client_obd *cli;
+	struct obd_import *imp;
+	int rc = 0, err;
+
+	if (!obd) {
+		CERROR("invalid export for disconnect: exp %p cookie %#llx\n",
+		       exp, exp ? exp->exp_handle.h_cookie : -1);
+		return -EINVAL;
+	}
+
+	cli = &obd->u.cli;
+	imp = cli->cl_import;
+
+	down_write(&cli->cl_sem);
+	CDEBUG(D_INFO, "disconnect %s - %zu\n", obd->obd_name,
+	       cli->cl_conn_count);
+
+	if (!cli->cl_conn_count) {
+		CERROR("disconnecting disconnected device (%s)\n",
+		       obd->obd_name);
+		rc = -EINVAL;
+		goto out_disconnect;
+	}
+
+	cli->cl_conn_count--;
+	if (cli->cl_conn_count) {
+		rc = 0;
+		goto out_disconnect;
+	}
+
+	/* Mark import deactivated now, so we don't try to reconnect if any
+	 * of the cleanup RPCs fails (e.g. LDLM cancel, etc).  We don't
+	 * fully deactivate the import, or that would drop all requests.
+	 */
+	spin_lock(&imp->imp_lock);
+	imp->imp_deactive = 1;
+	spin_unlock(&imp->imp_lock);
+
+	/* Some non-replayable imports (MDS's OSCs) are pinged, so just
+	 * delete it regardless.  (It's safe to delete an import that was
+	 * never added.)
+	 */
+	(void)ptlrpc_pinger_del_import(imp);
+
+	if (obd->obd_namespace) {
+		/* obd_force == local only */
+		ldlm_cli_cancel_unused(obd->obd_namespace, NULL,
+				       obd->obd_force ? LCF_LOCAL : 0, NULL);
+		ldlm_namespace_free_prior(obd->obd_namespace, imp,
+					  obd->obd_force);
+	}
+
+	/* There's no need to hold sem while disconnecting an import,
+	 * and it may actually cause deadlock in GSS.
+	 */
+	up_write(&cli->cl_sem);
+	rc = ptlrpc_disconnect_import(imp, 0);
+	down_write(&cli->cl_sem);
+
+	ptlrpc_invalidate_import(imp);
+
+out_disconnect:
+	/* Use server style - class_disconnect should be always called for
+	 * o_disconnect.
+	 */
+	err = class_disconnect(exp);
+	if (!rc && err)
+		rc = err;
+
+	up_write(&cli->cl_sem);
+
+	return rc;
+}
+EXPORT_SYMBOL(client_disconnect_export);
+
+/**
+ * Packs current SLV and Limit into \a req.
+ */
+int target_pack_pool_reply(struct ptlrpc_request *req)
+{
+	struct obd_device *obd;
+
+	/* Check that we still have all structures alive as this may
+	 * be some late RPC at shutdown time.
+	 */
+	if (unlikely(!req->rq_export || !req->rq_export->exp_obd ||
+		     !exp_connect_lru_resize(req->rq_export))) {
+		lustre_msg_set_slv(req->rq_repmsg, 0);
+		lustre_msg_set_limit(req->rq_repmsg, 0);
+		return 0;
+	}
+
+	/* OBD is alive here as export is alive, which we checked above. */
+	obd = req->rq_export->exp_obd;
+
+	read_lock(&obd->obd_pool_lock);
+	lustre_msg_set_slv(req->rq_repmsg, obd->obd_pool_slv);
+	lustre_msg_set_limit(req->rq_repmsg, obd->obd_pool_limit);
+	read_unlock(&obd->obd_pool_lock);
+
+	return 0;
+}
+EXPORT_SYMBOL(target_pack_pool_reply);
+
+static int
+target_send_reply_msg(struct ptlrpc_request *req, int rc, int fail_id)
+{
+	if (OBD_FAIL_CHECK_ORSET(fail_id & ~OBD_FAIL_ONCE, OBD_FAIL_ONCE)) {
+		DEBUG_REQ(D_ERROR, req, "dropping reply");
+		return -ECOMM;
+	}
+
+	if (unlikely(rc)) {
+		DEBUG_REQ(D_NET, req, "processing error (%d)", rc);
+		req->rq_status = rc;
+		return ptlrpc_send_error(req, 1);
+	}
+
+	DEBUG_REQ(D_NET, req, "sending reply");
+	return ptlrpc_send_reply(req, PTLRPC_REPLY_MAYBE_DIFFICULT);
+}
+
+void target_send_reply(struct ptlrpc_request *req, int rc, int fail_id)
+{
+	struct ptlrpc_service_part *svcpt;
+	int			netrc;
+	struct ptlrpc_reply_state *rs;
+	struct obd_export	 *exp;
+
+	if (req->rq_no_reply)
+		return;
+
+	svcpt = req->rq_rqbd->rqbd_svcpt;
+	rs = req->rq_reply_state;
+	if (!rs || !rs->rs_difficult) {
+		/* no notifiers */
+		target_send_reply_msg(req, rc, fail_id);
+		return;
+	}
+
+	/* must be an export if locks saved */
+	LASSERT(req->rq_export);
+	/* req/reply consistent */
+	LASSERT(rs->rs_svcpt == svcpt);
+
+	/* "fresh" reply */
+	LASSERT(!rs->rs_scheduled);
+	LASSERT(!rs->rs_scheduled_ever);
+	LASSERT(!rs->rs_handled);
+	LASSERT(!rs->rs_on_net);
+	LASSERT(!rs->rs_export);
+	LASSERT(list_empty(&rs->rs_obd_list));
+	LASSERT(list_empty(&rs->rs_exp_list));
+
+	exp = class_export_get(req->rq_export);
+
+	/* disable reply scheduling while I'm setting up */
+	rs->rs_scheduled = 1;
+	rs->rs_on_net    = 1;
+	rs->rs_xid       = req->rq_xid;
+	rs->rs_transno   = req->rq_transno;
+	rs->rs_export    = exp;
+	rs->rs_opc       = lustre_msg_get_opc(req->rq_reqmsg);
+
+	spin_lock(&exp->exp_uncommitted_replies_lock);
+	CDEBUG(D_NET, "rs transno = %llu, last committed = %llu\n",
+	       rs->rs_transno, exp->exp_last_committed);
+	if (rs->rs_transno > exp->exp_last_committed) {
+		/* not committed already */
+		list_add_tail(&rs->rs_obd_list,
+			      &exp->exp_uncommitted_replies);
+	}
+	spin_unlock(&exp->exp_uncommitted_replies_lock);
+
+	spin_lock(&exp->exp_lock);
+	list_add_tail(&rs->rs_exp_list, &exp->exp_outstanding_replies);
+	spin_unlock(&exp->exp_lock);
+
+	netrc = target_send_reply_msg(req, rc, fail_id);
+
+	spin_lock(&svcpt->scp_rep_lock);
+
+	atomic_inc(&svcpt->scp_nreps_difficult);
+
+	if (netrc != 0) {
+		/* error sending: reply is off the net.  Also we need +1
+		 * reply ref until ptlrpc_handle_rs() is done
+		 * with the reply state (if the send was successful, there
+		 * would have been +1 ref for the net, which
+		 * reply_out_callback leaves alone)
+		 */
+		rs->rs_on_net = 0;
+		ptlrpc_rs_addref(rs);
+	}
+
+	spin_lock(&rs->rs_lock);
+	if (rs->rs_transno <= exp->exp_last_committed ||
+	    (!rs->rs_on_net && !rs->rs_no_ack) ||
+	    list_empty(&rs->rs_exp_list) ||     /* completed already */
+	    list_empty(&rs->rs_obd_list)) {
+		CDEBUG(D_HA, "Schedule reply immediately\n");
+		ptlrpc_dispatch_difficult_reply(rs);
+	} else {
+		list_add(&rs->rs_list, &svcpt->scp_rep_active);
+		rs->rs_scheduled = 0;	/* allow notifier to schedule */
+	}
+	spin_unlock(&rs->rs_lock);
+	spin_unlock(&svcpt->scp_rep_lock);
+}
+EXPORT_SYMBOL(target_send_reply);
+
+enum ldlm_mode lck_compat_array[] = {
+	[LCK_EX]	= LCK_COMPAT_EX,
+	[LCK_PW]	= LCK_COMPAT_PW,
+	[LCK_PR]	= LCK_COMPAT_PR,
+	[LCK_CW]	= LCK_COMPAT_CW,
+	[LCK_CR]	= LCK_COMPAT_CR,
+	[LCK_NL]	= LCK_COMPAT_NL,
+	[LCK_GROUP]	= LCK_COMPAT_GROUP,
+	[LCK_COS]	= LCK_COMPAT_COS,
+};
+
+/**
+ * Rather arbitrary mapping from LDLM error codes to errno values. This should
+ * not escape to the user level.
+ */
+int ldlm_error2errno(enum ldlm_error error)
+{
+	int result;
+
+	switch (error) {
+	case ELDLM_OK:
+	case ELDLM_LOCK_MATCHED:
+		result = 0;
+		break;
+	case ELDLM_LOCK_CHANGED:
+		result = -ESTALE;
+		break;
+	case ELDLM_LOCK_ABORTED:
+		result = -ENAVAIL;
+		break;
+	case ELDLM_LOCK_REPLACED:
+		result = -ESRCH;
+		break;
+	case ELDLM_NO_LOCK_DATA:
+		result = -ENOENT;
+		break;
+	case ELDLM_NAMESPACE_EXISTS:
+		result = -EEXIST;
+		break;
+	case ELDLM_BAD_NAMESPACE:
+		result = -EBADF;
+		break;
+	default:
+		if (((int)error) < 0)  /* cast to signed type */
+			result = error; /* as enum ldlm_error can be unsigned */
+		else {
+			CERROR("Invalid DLM result code: %d\n", error);
+			result = -EPROTO;
+		}
+	}
+	return result;
+}
+EXPORT_SYMBOL(ldlm_error2errno);
+
+#if LUSTRE_TRACKS_LOCK_EXP_REFS
+void ldlm_dump_export_locks(struct obd_export *exp)
+{
+	spin_lock(&exp->exp_locks_list_guard);
+	if (!list_empty(&exp->exp_locks_list)) {
+		struct ldlm_lock *lock;
+
+		CERROR("dumping locks for export %p,ignore if the unmount doesn't hang\n",
+		       exp);
+		list_for_each_entry(lock, &exp->exp_locks_list,
+				    l_exp_refs_link)
+			LDLM_ERROR(lock, "lock:");
+	}
+	spin_unlock(&exp->exp_locks_list_guard);
+}
+#endif
diff --git a/drivers/staging/lustre/lustre/ptlrpc/ldlm_lock.c b/drivers/staging/lustre/lustre/ptlrpc/ldlm_lock.c
new file mode 100644
index 000000000000..2fb2e088dc87
--- /dev/null
+++ b/drivers/staging/lustre/lustre/ptlrpc/ldlm_lock.c
@@ -0,0 +1,2103 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * GPL HEADER START
+ *
+ * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 only,
+ * as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License version 2 for more details (a copy is included
+ * in the LICENSE file that accompanied this code).
+ *
+ * You should have received a copy of the GNU General Public License
+ * version 2 along with this program; If not, see
+ * http://www.gnu.org/licenses/gpl-2.0.html
+ *
+ * GPL HEADER END
+ */
+/*
+ * Copyright (c) 2002, 2010, Oracle and/or its affiliates. All rights reserved.
+ * Use is subject to license terms.
+ *
+ * Copyright (c) 2010, 2015, Intel Corporation.
+ */
+/*
+ * This file is part of Lustre, http://www.lustre.org/
+ * Lustre is a trademark of Sun Microsystems, Inc.
+ *
+ * lustre/ldlm/ldlm_lock.c
+ *
+ * Author: Peter Braam <braam at clusterfs.com>
+ * Author: Phil Schwan <phil at clusterfs.com>
+ */
+
+#define DEBUG_SUBSYSTEM S_LDLM
+
+#include <lustre_intent.h>
+#include <lustre_swab.h>
+#include <obd_class.h>
+#include "ldlm_internal.h"
+
+/* lock types */
+char *ldlm_lockname[] = {
+	[0]		= "--",
+	[LCK_EX]	= "EX",
+	[LCK_PW]	= "PW",
+	[LCK_PR]	= "PR",
+	[LCK_CW]	= "CW",
+	[LCK_CR]	= "CR",
+	[LCK_NL]	= "NL",
+	[LCK_GROUP]	= "GROUP",
+	[LCK_COS]	= "COS",
+};
+EXPORT_SYMBOL(ldlm_lockname);
+
+static char *ldlm_typename[] = {
+	[LDLM_PLAIN]	= "PLN",
+	[LDLM_EXTENT]	= "EXT",
+	[LDLM_FLOCK]	= "FLK",
+	[LDLM_IBITS]	= "IBT",
+};
+
+static ldlm_policy_wire_to_local_t ldlm_policy_wire_to_local[] = {
+	[LDLM_PLAIN - LDLM_MIN_TYPE]	= ldlm_plain_policy_wire_to_local,
+	[LDLM_EXTENT - LDLM_MIN_TYPE]	= ldlm_extent_policy_wire_to_local,
+	[LDLM_FLOCK - LDLM_MIN_TYPE]	= ldlm_flock_policy_wire_to_local,
+	[LDLM_IBITS - LDLM_MIN_TYPE]	= ldlm_ibits_policy_wire_to_local,
+};
+
+static ldlm_policy_local_to_wire_t ldlm_policy_local_to_wire[] = {
+	[LDLM_PLAIN - LDLM_MIN_TYPE]	= ldlm_plain_policy_local_to_wire,
+	[LDLM_EXTENT - LDLM_MIN_TYPE]	= ldlm_extent_policy_local_to_wire,
+	[LDLM_FLOCK - LDLM_MIN_TYPE]	= ldlm_flock_policy_local_to_wire,
+	[LDLM_IBITS - LDLM_MIN_TYPE]	= ldlm_ibits_policy_local_to_wire,
+};
+
+/**
+ * Converts lock policy from local format to on the wire lock_desc format
+ */
+static void ldlm_convert_policy_to_wire(enum ldlm_type type,
+					const union ldlm_policy_data *lpolicy,
+					union ldlm_wire_policy_data *wpolicy)
+{
+	ldlm_policy_local_to_wire_t convert;
+
+	convert = ldlm_policy_local_to_wire[type - LDLM_MIN_TYPE];
+
+	convert(lpolicy, wpolicy);
+}
+
+/**
+ * Converts lock policy from on the wire lock_desc format to local format
+ */
+void ldlm_convert_policy_to_local(struct obd_export *exp, enum ldlm_type type,
+				  const union ldlm_wire_policy_data *wpolicy,
+				  union ldlm_policy_data *lpolicy)
+{
+	ldlm_policy_wire_to_local_t convert;
+
+	convert = ldlm_policy_wire_to_local[type - LDLM_MIN_TYPE];
+
+	convert(wpolicy, lpolicy);
+}
+
+const char *ldlm_it2str(enum ldlm_intent_flags it)
+{
+	switch (it) {
+	case IT_OPEN:
+		return "open";
+	case IT_CREAT:
+		return "creat";
+	case (IT_OPEN | IT_CREAT):
+		return "open|creat";
+	case IT_READDIR:
+		return "readdir";
+	case IT_GETATTR:
+		return "getattr";
+	case IT_LOOKUP:
+		return "lookup";
+	case IT_UNLINK:
+		return "unlink";
+	case IT_GETXATTR:
+		return "getxattr";
+	case IT_LAYOUT:
+		return "layout";
+	default:
+		CERROR("Unknown intent 0x%08x\n", it);
+		return "UNKNOWN";
+	}
+}
+EXPORT_SYMBOL(ldlm_it2str);
+
+/*
+ * REFCOUNTED LOCK OBJECTS
+ */
+
+/**
+ * Get a reference on a lock.
+ *
+ * Lock refcounts, during creation:
+ *   - one special one for allocation, dec'd only once in destroy
+ *   - one for being a lock that's in-use
+ *   - one for the addref associated with a new lock
+ */
+struct ldlm_lock *ldlm_lock_get(struct ldlm_lock *lock)
+{
+	atomic_inc(&lock->l_refc);
+	return lock;
+}
+EXPORT_SYMBOL(ldlm_lock_get);
+
+/**
+ * Release lock reference.
+ *
+ * Also frees the lock if it was last reference.
+ */
+void ldlm_lock_put(struct ldlm_lock *lock)
+{
+	LASSERT(lock->l_resource != LP_POISON);
+	LASSERT(atomic_read(&lock->l_refc) > 0);
+	if (atomic_dec_and_test(&lock->l_refc)) {
+		struct ldlm_resource *res;
+
+		LDLM_DEBUG(lock,
+			   "final lock_put on destroyed lock, freeing it.");
+
+		res = lock->l_resource;
+		LASSERT(ldlm_is_destroyed(lock));
+		LASSERT(list_empty(&lock->l_res_link));
+		LASSERT(list_empty(&lock->l_pending_chain));
+
+		lprocfs_counter_decr(ldlm_res_to_ns(res)->ns_stats,
+				     LDLM_NSS_LOCKS);
+		lu_ref_del(&res->lr_reference, "lock", lock);
+		ldlm_resource_putref(res);
+		lock->l_resource = NULL;
+		if (lock->l_export) {
+			class_export_lock_put(lock->l_export, lock);
+			lock->l_export = NULL;
+		}
+
+		kfree(lock->l_lvb_data);
+
+		lu_ref_fini(&lock->l_reference);
+		OBD_FREE_RCU(lock, sizeof(*lock), &lock->l_handle);
+	}
+}
+EXPORT_SYMBOL(ldlm_lock_put);
+
+/**
+ * Removes LDLM lock \a lock from LRU. Assumes LRU is already locked.
+ */
+int ldlm_lock_remove_from_lru_nolock(struct ldlm_lock *lock)
+{
+	int rc = 0;
+
+	if (!list_empty(&lock->l_lru)) {
+		struct ldlm_namespace *ns = ldlm_lock_to_ns(lock);
+
+		LASSERT(lock->l_resource->lr_type != LDLM_FLOCK);
+		list_del_init(&lock->l_lru);
+		LASSERT(ns->ns_nr_unused > 0);
+		ns->ns_nr_unused--;
+		rc = 1;
+	}
+	return rc;
+}
+
+/**
+ * Removes LDLM lock \a lock from LRU. Obtains the LRU lock first.
+ *
+ * If \a last_use is non-zero, it will remove the lock from LRU only if
+ * it matches lock's l_last_used.
+ *
+ * \retval 0 if \a last_use is set, the lock is not in LRU list or \a last_use
+ *           doesn't match lock's l_last_used;
+ *           otherwise, the lock hasn't been in the LRU list.
+ * \retval 1 the lock was in LRU list and removed.
+ */
+int ldlm_lock_remove_from_lru_check(struct ldlm_lock *lock, time_t last_use)
+{
+	struct ldlm_namespace *ns = ldlm_lock_to_ns(lock);
+	int rc = 0;
+
+	spin_lock(&ns->ns_lock);
+	if (last_use == 0 || last_use == lock->l_last_used)
+		rc = ldlm_lock_remove_from_lru_nolock(lock);
+	spin_unlock(&ns->ns_lock);
+
+	return rc;
+}
+
+/**
+ * Adds LDLM lock \a lock to namespace LRU. Assumes LRU is already locked.
+ */
+static void ldlm_lock_add_to_lru_nolock(struct ldlm_lock *lock)
+{
+	struct ldlm_namespace *ns = ldlm_lock_to_ns(lock);
+
+	lock->l_last_used = jiffies;
+	LASSERT(list_empty(&lock->l_lru));
+	LASSERT(lock->l_resource->lr_type != LDLM_FLOCK);
+	list_add_tail(&lock->l_lru, &ns->ns_unused_list);
+	ldlm_clear_skipped(lock);
+	LASSERT(ns->ns_nr_unused >= 0);
+	ns->ns_nr_unused++;
+}
+
+/**
+ * Adds LDLM lock \a lock to namespace LRU. Obtains necessary LRU locks
+ * first.
+ */
+static void ldlm_lock_add_to_lru(struct ldlm_lock *lock)
+{
+	struct ldlm_namespace *ns = ldlm_lock_to_ns(lock);
+
+	spin_lock(&ns->ns_lock);
+	ldlm_lock_add_to_lru_nolock(lock);
+	spin_unlock(&ns->ns_lock);
+}
+
+/**
+ * Moves LDLM lock \a lock that is already in namespace LRU to the tail of
+ * the LRU. Performs necessary LRU locking
+ */
+static void ldlm_lock_touch_in_lru(struct ldlm_lock *lock)
+{
+	struct ldlm_namespace *ns = ldlm_lock_to_ns(lock);
+
+	spin_lock(&ns->ns_lock);
+	if (!list_empty(&lock->l_lru)) {
+		ldlm_lock_remove_from_lru_nolock(lock);
+		ldlm_lock_add_to_lru_nolock(lock);
+	}
+	spin_unlock(&ns->ns_lock);
+}
+
+/**
+ * Helper to destroy a locked lock.
+ *
+ * Used by ldlm_lock_destroy and ldlm_lock_destroy_nolock
+ * Must be called with l_lock and lr_lock held.
+ *
+ * Does not actually free the lock data, but rather marks the lock as
+ * destroyed by setting l_destroyed field in the lock to 1.  Destroys a
+ * handle->lock association too, so that the lock can no longer be found
+ * and removes the lock from LRU list.  Actual lock freeing occurs when
+ * last lock reference goes away.
+ *
+ * Original comment (of some historical value):
+ * This used to have a 'strict' flag, which recovery would use to mark an
+ * in-use lock as needing-to-die.  Lest I am ever tempted to put it back, I
+ * shall explain why it's gone: with the new hash table scheme, once you call
+ * ldlm_lock_destroy, you can never drop your final references on this lock.
+ * Because it's not in the hash table anymore.  -phil
+ */
+static int ldlm_lock_destroy_internal(struct ldlm_lock *lock)
+{
+	if (lock->l_readers || lock->l_writers) {
+		LDLM_ERROR(lock, "lock still has references");
+		LBUG();
+	}
+
+	if (!list_empty(&lock->l_res_link)) {
+		LDLM_ERROR(lock, "lock still on resource");
+		LBUG();
+	}
+
+	if (ldlm_is_destroyed(lock)) {
+		LASSERT(list_empty(&lock->l_lru));
+		return 0;
+	}
+	ldlm_set_destroyed(lock);
+
+	ldlm_lock_remove_from_lru(lock);
+	class_handle_unhash(&lock->l_handle);
+
+	return 1;
+}
+
+/**
+ * Destroys a LDLM lock \a lock. Performs necessary locking first.
+ */
+static void ldlm_lock_destroy(struct ldlm_lock *lock)
+{
+	int first;
+
+	lock_res_and_lock(lock);
+	first = ldlm_lock_destroy_internal(lock);
+	unlock_res_and_lock(lock);
+
+	/* drop reference from hashtable only for first destroy */
+	if (first) {
+		lu_ref_del(&lock->l_reference, "hash", lock);
+		LDLM_LOCK_RELEASE(lock);
+	}
+}
+
+/**
+ * Destroys a LDLM lock \a lock that is already locked.
+ */
+void ldlm_lock_destroy_nolock(struct ldlm_lock *lock)
+{
+	int first;
+
+	first = ldlm_lock_destroy_internal(lock);
+	/* drop reference from hashtable only for first destroy */
+	if (first) {
+		lu_ref_del(&lock->l_reference, "hash", lock);
+		LDLM_LOCK_RELEASE(lock);
+	}
+}
+
+/* this is called by portals_handle2object with the handle lock taken */
+static void lock_handle_addref(void *lock)
+{
+	LDLM_LOCK_GET((struct ldlm_lock *)lock);
+}
+
+static void lock_handle_free(void *lock, int size)
+{
+	LASSERT(size == sizeof(struct ldlm_lock));
+	kmem_cache_free(ldlm_lock_slab, lock);
+}
+
+static struct portals_handle_ops lock_handle_ops = {
+	.hop_addref = lock_handle_addref,
+	.hop_free   = lock_handle_free,
+};
+
+/**
+ *
+ * Allocate and initialize new lock structure.
+ *
+ * usage: pass in a resource on which you have done ldlm_resource_get
+ *	new lock will take over the refcount.
+ * returns: lock with refcount 2 - one for current caller and one for remote
+ */
+static struct ldlm_lock *ldlm_lock_new(struct ldlm_resource *resource)
+{
+	struct ldlm_lock *lock;
+
+	LASSERT(resource);
+
+	lock = kmem_cache_zalloc(ldlm_lock_slab, GFP_NOFS);
+	if (!lock)
+		return NULL;
+
+	spin_lock_init(&lock->l_lock);
+	lock->l_resource = resource;
+	lu_ref_add(&resource->lr_reference, "lock", lock);
+
+	atomic_set(&lock->l_refc, 2);
+	INIT_LIST_HEAD(&lock->l_res_link);
+	INIT_LIST_HEAD(&lock->l_lru);
+	INIT_LIST_HEAD(&lock->l_pending_chain);
+	INIT_LIST_HEAD(&lock->l_bl_ast);
+	INIT_LIST_HEAD(&lock->l_cp_ast);
+	INIT_LIST_HEAD(&lock->l_rk_ast);
+	init_waitqueue_head(&lock->l_waitq);
+	lock->l_blocking_lock = NULL;
+	INIT_LIST_HEAD(&lock->l_sl_mode);
+	INIT_LIST_HEAD(&lock->l_sl_policy);
+	RB_CLEAR_NODE(&lock->l_rb);
+
+	lprocfs_counter_incr(ldlm_res_to_ns(resource)->ns_stats,
+			     LDLM_NSS_LOCKS);
+	INIT_LIST_HEAD(&lock->l_handle.h_link);
+	class_handle_hash(&lock->l_handle, &lock_handle_ops);
+
+	lu_ref_init(&lock->l_reference);
+	lu_ref_add(&lock->l_reference, "hash", lock);
+	lock->l_callback_timeout = 0;
+
+#if LUSTRE_TRACKS_LOCK_EXP_REFS
+	INIT_LIST_HEAD(&lock->l_exp_refs_link);
+	lock->l_exp_refs_nr = 0;
+	lock->l_exp_refs_target = NULL;
+#endif
+
+	return lock;
+}
+
+/**
+ * Moves LDLM lock \a lock to another resource.
+ * This is used on client when server returns some other lock than requested
+ * (typically as a result of intent operation)
+ */
+int ldlm_lock_change_resource(struct ldlm_namespace *ns, struct ldlm_lock *lock,
+			      const struct ldlm_res_id *new_resid)
+{
+	struct ldlm_resource *oldres = lock->l_resource;
+	struct ldlm_resource *newres;
+	int type;
+
+	lock_res_and_lock(lock);
+	if (memcmp(new_resid, &lock->l_resource->lr_name,
+		   sizeof(lock->l_resource->lr_name)) == 0) {
+		/* Nothing to do */
+		unlock_res_and_lock(lock);
+		return 0;
+	}
+
+	LASSERT(new_resid->name[0] != 0);
+
+	/* This function assumes that the lock isn't on any lists */
+	LASSERT(list_empty(&lock->l_res_link));
+
+	type = oldres->lr_type;
+	unlock_res_and_lock(lock);
+
+	newres = ldlm_resource_get(ns, NULL, new_resid, type, 1);
+	if (IS_ERR(newres))
+		return PTR_ERR(newres);
+
+	lu_ref_add(&newres->lr_reference, "lock", lock);
+	/*
+	 * To flip the lock from the old to the new resource, lock, oldres and
+	 * newres have to be locked. Resource spin-locks are nested within
+	 * lock->l_lock, and are taken in the memory address order to avoid
+	 * dead-locks.
+	 */
+	spin_lock(&lock->l_lock);
+	oldres = lock->l_resource;
+	if (oldres < newres) {
+		lock_res(oldres);
+		lock_res_nested(newres, LRT_NEW);
+	} else {
+		lock_res(newres);
+		lock_res_nested(oldres, LRT_NEW);
+	}
+	LASSERT(memcmp(new_resid, &oldres->lr_name,
+		       sizeof(oldres->lr_name)) != 0);
+	lock->l_resource = newres;
+	unlock_res(oldres);
+	unlock_res_and_lock(lock);
+
+	/* ...and the flowers are still standing! */
+	lu_ref_del(&oldres->lr_reference, "lock", lock);
+	ldlm_resource_putref(oldres);
+
+	return 0;
+}
+
+/** \defgroup ldlm_handles LDLM HANDLES
+ * Ways to get hold of locks without any addresses.
+ * @{
+ */
+
+/**
+ * Fills in handle for LDLM lock \a lock into supplied \a lockh
+ * Does not take any references.
+ */
+void ldlm_lock2handle(const struct ldlm_lock *lock, struct lustre_handle *lockh)
+{
+	lockh->cookie = lock->l_handle.h_cookie;
+}
+EXPORT_SYMBOL(ldlm_lock2handle);
+
+/**
+ * Obtain a lock reference by handle.
+ *
+ * if \a flags: atomically get the lock and set the flags.
+ *	      Return NULL if flag already set
+ */
+struct ldlm_lock *__ldlm_handle2lock(const struct lustre_handle *handle,
+				     __u64 flags)
+{
+	struct ldlm_lock *lock;
+
+	LASSERT(handle);
+
+	lock = class_handle2object(handle->cookie, NULL);
+	if (!lock)
+		return NULL;
+
+	if (lock->l_export && lock->l_export->exp_failed) {
+		CDEBUG(D_INFO, "lock export failed: lock %p, exp %p\n",
+		       lock, lock->l_export);
+		LDLM_LOCK_PUT(lock);
+		return NULL;
+	}
+
+	/* It's unlikely but possible that someone marked the lock as
+	 * destroyed after we did handle2object on it
+	 */
+	if (flags == 0 && !ldlm_is_destroyed(lock)) {
+		lu_ref_add(&lock->l_reference, "handle", current);
+		return lock;
+	}
+
+	lock_res_and_lock(lock);
+
+	LASSERT(lock->l_resource);
+
+	lu_ref_add_atomic(&lock->l_reference, "handle", current);
+	if (unlikely(ldlm_is_destroyed(lock))) {
+		unlock_res_and_lock(lock);
+		CDEBUG(D_INFO, "lock already destroyed: lock %p\n", lock);
+		LDLM_LOCK_PUT(lock);
+		return NULL;
+	}
+
+	if (flags) {
+		if (lock->l_flags & flags) {
+			unlock_res_and_lock(lock);
+			LDLM_LOCK_PUT(lock);
+			return NULL;
+		}
+
+		lock->l_flags |= flags;
+	}
+
+	unlock_res_and_lock(lock);
+	return lock;
+}
+EXPORT_SYMBOL(__ldlm_handle2lock);
+/** @} ldlm_handles */
+
+/**
+ * Fill in "on the wire" representation for given LDLM lock into supplied
+ * lock descriptor \a desc structure.
+ */
+void ldlm_lock2desc(struct ldlm_lock *lock, struct ldlm_lock_desc *desc)
+{
+	ldlm_res2desc(lock->l_resource, &desc->l_resource);
+	desc->l_req_mode = lock->l_req_mode;
+	desc->l_granted_mode = lock->l_granted_mode;
+	ldlm_convert_policy_to_wire(lock->l_resource->lr_type,
+				    &lock->l_policy_data,
+				    &desc->l_policy_data);
+}
+
+/**
+ * Add a lock to list of conflicting locks to send AST to.
+ *
+ * Only add if we have not sent a blocking AST to the lock yet.
+ */
+static void ldlm_add_bl_work_item(struct ldlm_lock *lock, struct ldlm_lock *new,
+				  struct list_head *work_list)
+{
+	if (!ldlm_is_ast_sent(lock)) {
+		LDLM_DEBUG(lock, "lock incompatible; sending blocking AST.");
+		ldlm_set_ast_sent(lock);
+		/* If the enqueuing client said so, tell the AST recipient to
+		 * discard dirty data, rather than writing back.
+		 */
+		if (ldlm_is_ast_discard_data(new))
+			ldlm_set_discard_data(lock);
+		LASSERT(list_empty(&lock->l_bl_ast));
+		list_add(&lock->l_bl_ast, work_list);
+		LDLM_LOCK_GET(lock);
+		LASSERT(!lock->l_blocking_lock);
+		lock->l_blocking_lock = LDLM_LOCK_GET(new);
+	}
+}
+
+/**
+ * Add a lock to list of just granted locks to send completion AST to.
+ */
+static void ldlm_add_cp_work_item(struct ldlm_lock *lock,
+				  struct list_head *work_list)
+{
+	if (!ldlm_is_cp_reqd(lock)) {
+		ldlm_set_cp_reqd(lock);
+		LDLM_DEBUG(lock, "lock granted; sending completion AST.");
+		LASSERT(list_empty(&lock->l_cp_ast));
+		list_add(&lock->l_cp_ast, work_list);
+		LDLM_LOCK_GET(lock);
+	}
+}
+
+/**
+ * Aggregator function to add AST work items into a list. Determines
+ * what sort of an AST work needs to be done and calls the proper
+ * adding function.
+ * Must be called with lr_lock held.
+ */
+static void ldlm_add_ast_work_item(struct ldlm_lock *lock,
+				   struct ldlm_lock *new,
+				   struct list_head *work_list)
+{
+	check_res_locked(lock->l_resource);
+	if (new)
+		ldlm_add_bl_work_item(lock, new, work_list);
+	else
+		ldlm_add_cp_work_item(lock, work_list);
+}
+
+/**
+ * Add specified reader/writer reference to LDLM lock with handle \a lockh.
+ * r/w reference type is determined by \a mode
+ * Calls ldlm_lock_addref_internal.
+ */
+void ldlm_lock_addref(const struct lustre_handle *lockh, enum ldlm_mode mode)
+{
+	struct ldlm_lock *lock;
+
+	lock = ldlm_handle2lock(lockh);
+	LASSERTF(lock, "Non-existing lock: %llx\n", lockh->cookie);
+	ldlm_lock_addref_internal(lock, mode);
+	LDLM_LOCK_PUT(lock);
+}
+EXPORT_SYMBOL(ldlm_lock_addref);
+
+/**
+ * Helper function.
+ * Add specified reader/writer reference to LDLM lock \a lock.
+ * r/w reference type is determined by \a mode
+ * Removes lock from LRU if it is there.
+ * Assumes the LDLM lock is already locked.
+ */
+void ldlm_lock_addref_internal_nolock(struct ldlm_lock *lock,
+				      enum ldlm_mode mode)
+{
+	ldlm_lock_remove_from_lru(lock);
+	if (mode & (LCK_NL | LCK_CR | LCK_PR)) {
+		lock->l_readers++;
+		lu_ref_add_atomic(&lock->l_reference, "reader", lock);
+	}
+	if (mode & (LCK_EX | LCK_CW | LCK_PW | LCK_GROUP | LCK_COS)) {
+		lock->l_writers++;
+		lu_ref_add_atomic(&lock->l_reference, "writer", lock);
+	}
+	LDLM_LOCK_GET(lock);
+	lu_ref_add_atomic(&lock->l_reference, "user", lock);
+	LDLM_DEBUG(lock, "ldlm_lock_addref(%s)", ldlm_lockname[mode]);
+}
+
+/**
+ * Attempts to add reader/writer reference to a lock with handle \a lockh, and
+ * fails if lock is already LDLM_FL_CBPENDING or destroyed.
+ *
+ * \retval 0 success, lock was addref-ed
+ *
+ * \retval -EAGAIN lock is being canceled.
+ */
+int ldlm_lock_addref_try(const struct lustre_handle *lockh, enum ldlm_mode mode)
+{
+	struct ldlm_lock *lock;
+	int	       result;
+
+	result = -EAGAIN;
+	lock = ldlm_handle2lock(lockh);
+	if (lock) {
+		lock_res_and_lock(lock);
+		if (lock->l_readers != 0 || lock->l_writers != 0 ||
+		    !ldlm_is_cbpending(lock)) {
+			ldlm_lock_addref_internal_nolock(lock, mode);
+			result = 0;
+		}
+		unlock_res_and_lock(lock);
+		LDLM_LOCK_PUT(lock);
+	}
+	return result;
+}
+EXPORT_SYMBOL(ldlm_lock_addref_try);
+
+/**
+ * Add specified reader/writer reference to LDLM lock \a lock.
+ * Locks LDLM lock and calls ldlm_lock_addref_internal_nolock to do the work.
+ * Only called for local locks.
+ */
+void ldlm_lock_addref_internal(struct ldlm_lock *lock, enum ldlm_mode mode)
+{
+	lock_res_and_lock(lock);
+	ldlm_lock_addref_internal_nolock(lock, mode);
+	unlock_res_and_lock(lock);
+}
+
+/**
+ * Removes reader/writer reference for LDLM lock \a lock.
+ * Assumes LDLM lock is already locked.
+ * only called in ldlm_flock_destroy and for local locks.
+ * Does NOT add lock to LRU if no r/w references left to accommodate flock locks
+ * that cannot be placed in LRU.
+ */
+void ldlm_lock_decref_internal_nolock(struct ldlm_lock *lock,
+				      enum ldlm_mode mode)
+{
+	LDLM_DEBUG(lock, "ldlm_lock_decref(%s)", ldlm_lockname[mode]);
+	if (mode & (LCK_NL | LCK_CR | LCK_PR)) {
+		LASSERT(lock->l_readers > 0);
+		lu_ref_del(&lock->l_reference, "reader", lock);
+		lock->l_readers--;
+	}
+	if (mode & (LCK_EX | LCK_CW | LCK_PW | LCK_GROUP | LCK_COS)) {
+		LASSERT(lock->l_writers > 0);
+		lu_ref_del(&lock->l_reference, "writer", lock);
+		lock->l_writers--;
+	}
+
+	lu_ref_del(&lock->l_reference, "user", lock);
+	LDLM_LOCK_RELEASE(lock);    /* matches the LDLM_LOCK_GET() in addref */
+}
+
+/**
+ * Removes reader/writer reference for LDLM lock \a lock.
+ * Locks LDLM lock first.
+ * If the lock is determined to be client lock on a client and r/w refcount
+ * drops to zero and the lock is not blocked, the lock is added to LRU lock
+ * on the namespace.
+ * For blocked LDLM locks if r/w count drops to zero, blocking_ast is called.
+ */
+void ldlm_lock_decref_internal(struct ldlm_lock *lock, enum ldlm_mode mode)
+{
+	struct ldlm_namespace *ns;
+
+	lock_res_and_lock(lock);
+
+	ns = ldlm_lock_to_ns(lock);
+
+	ldlm_lock_decref_internal_nolock(lock, mode);
+
+	if ((ldlm_is_local(lock) || lock->l_req_mode == LCK_GROUP) &&
+	    !lock->l_readers && !lock->l_writers) {
+		/* If this is a local lock on a server namespace and this was
+		 * the last reference, cancel the lock.
+		 *
+		 * Group locks are special:
+		 * They must not go in LRU, but they are not called back
+		 * like non-group locks, instead they are manually released.
+		 * They have an l_writers reference which they keep until
+		 * they are manually released, so we remove them when they have
+		 * no more reader or writer references. - LU-6368
+		 */
+		ldlm_set_cbpending(lock);
+	}
+
+	if (!lock->l_readers && !lock->l_writers && ldlm_is_cbpending(lock)) {
+		/* If we received a blocked AST and this was the last reference,
+		 * run the callback.
+		 */
+		LDLM_DEBUG(lock, "final decref done on cbpending lock");
+
+		LDLM_LOCK_GET(lock); /* dropped by bl thread */
+		ldlm_lock_remove_from_lru(lock);
+		unlock_res_and_lock(lock);
+
+		if (ldlm_is_fail_loc(lock))
+			OBD_RACE(OBD_FAIL_LDLM_CP_BL_RACE);
+
+		if (ldlm_is_atomic_cb(lock) ||
+		    ldlm_bl_to_thread_lock(ns, NULL, lock) != 0)
+			ldlm_handle_bl_callback(ns, NULL, lock);
+	} else if (!lock->l_readers && !lock->l_writers &&
+		   !ldlm_is_no_lru(lock) && !ldlm_is_bl_ast(lock)) {
+		LDLM_DEBUG(lock, "add lock into lru list");
+
+		/* If this is a client-side namespace and this was the last
+		 * reference, put it on the LRU.
+		 */
+		ldlm_lock_add_to_lru(lock);
+		unlock_res_and_lock(lock);
+
+		if (ldlm_is_fail_loc(lock))
+			OBD_RACE(OBD_FAIL_LDLM_CP_BL_RACE);
+
+		/* Call ldlm_cancel_lru() only if EARLY_CANCEL and LRU RESIZE
+		 * are not supported by the server, otherwise, it is done on
+		 * enqueue.
+		 */
+		if (!exp_connect_cancelset(lock->l_conn_export) &&
+		    !ns_connect_lru_resize(ns))
+			ldlm_cancel_lru(ns, 0, LCF_ASYNC, 0);
+	} else {
+		LDLM_DEBUG(lock, "do not add lock into lru list");
+		unlock_res_and_lock(lock);
+	}
+}
+
+/**
+ * Decrease reader/writer refcount for LDLM lock with handle \a lockh
+ */
+void ldlm_lock_decref(const struct lustre_handle *lockh, enum ldlm_mode mode)
+{
+	struct ldlm_lock *lock = __ldlm_handle2lock(lockh, 0);
+
+	LASSERTF(lock, "Non-existing lock: %#llx\n", lockh->cookie);
+	ldlm_lock_decref_internal(lock, mode);
+	LDLM_LOCK_PUT(lock);
+}
+EXPORT_SYMBOL(ldlm_lock_decref);
+
+/**
+ * Decrease reader/writer refcount for LDLM lock with handle
+ * \a lockh and mark it for subsequent cancellation once r/w refcount
+ * drops to zero instead of putting into LRU.
+ */
+void ldlm_lock_decref_and_cancel(const struct lustre_handle *lockh,
+				 enum ldlm_mode mode)
+{
+	struct ldlm_lock *lock = __ldlm_handle2lock(lockh, 0);
+
+	LASSERT(lock);
+
+	LDLM_DEBUG(lock, "ldlm_lock_decref(%s)", ldlm_lockname[mode]);
+	lock_res_and_lock(lock);
+	ldlm_set_cbpending(lock);
+	unlock_res_and_lock(lock);
+	ldlm_lock_decref_internal(lock, mode);
+	LDLM_LOCK_PUT(lock);
+}
+EXPORT_SYMBOL(ldlm_lock_decref_and_cancel);
+
+struct sl_insert_point {
+	struct list_head *res_link;
+	struct list_head *mode_link;
+	struct list_head *policy_link;
+};
+
+/**
+ * Finds a position to insert the new lock into granted lock list.
+ *
+ * Used for locks eligible for skiplist optimization.
+ *
+ * Parameters:
+ *      queue [input]:  the granted list where search acts on;
+ *      req [input]:    the lock whose position to be located;
+ *      prev [output]:  positions within 3 lists to insert @req to
+ * Return Value:
+ *      filled @prev
+ * NOTE: called by
+ *  - ldlm_grant_lock_with_skiplist
+ */
+static void search_granted_lock(struct list_head *queue,
+				struct ldlm_lock *req,
+				struct sl_insert_point *prev)
+{
+	struct ldlm_lock *lock, *mode_end, *policy_end;
+
+	list_for_each_entry(lock, queue, l_res_link) {
+
+		mode_end = list_prev_entry(lock, l_sl_mode);
+
+		if (lock->l_req_mode != req->l_req_mode) {
+			/* jump to last lock of mode group */
+			lock = mode_end;
+			continue;
+		}
+
+		/* suitable mode group is found */
+		if (lock->l_resource->lr_type == LDLM_PLAIN) {
+			/* insert point is last lock of the mode group */
+			prev->res_link = &mode_end->l_res_link;
+			prev->mode_link = &mode_end->l_sl_mode;
+			prev->policy_link = &req->l_sl_policy;
+			return;
+		}
+
+		if (lock->l_resource->lr_type == LDLM_IBITS) {
+			for (;;) {
+				policy_end =
+					list_prev_entry(lock, l_sl_policy);
+
+				if (lock->l_policy_data.l_inodebits.bits ==
+				    req->l_policy_data.l_inodebits.bits) {
+					/* insert point is last lock of
+					 * the policy group
+					 */
+					prev->res_link =
+						&policy_end->l_res_link;
+					prev->mode_link =
+						&policy_end->l_sl_mode;
+					prev->policy_link =
+						&policy_end->l_sl_policy;
+					return;
+				}
+
+				if (policy_end == mode_end)
+					/* done with mode group */
+					break;
+
+				/* go to next policy group within mode group */
+				lock = list_next_entry(policy_end, l_res_link);
+			}  /* loop over policy groups within the mode group */
+
+			/* insert point is last lock of the mode group,
+			 * new policy group is started
+			 */
+			prev->res_link = &mode_end->l_res_link;
+			prev->mode_link = &mode_end->l_sl_mode;
+			prev->policy_link = &req->l_sl_policy;
+			return;
+		}
+
+		LDLM_ERROR(lock, "is not LDLM_PLAIN or LDLM_IBITS lock");
+		LBUG();
+	}
+
+	/* insert point is last lock on the queue,
+	 * new mode group and new policy group are started
+	 */
+	prev->res_link = queue->prev;
+	prev->mode_link = &req->l_sl_mode;
+	prev->policy_link = &req->l_sl_policy;
+}
+
+/**
+ * Add a lock into resource granted list after a position described by
+ * \a prev.
+ */
+static void ldlm_granted_list_add_lock(struct ldlm_lock *lock,
+				       struct sl_insert_point *prev)
+{
+	struct ldlm_resource *res = lock->l_resource;
+
+	check_res_locked(res);
+
+	ldlm_resource_dump(D_INFO, res);
+	LDLM_DEBUG(lock, "About to add lock:");
+
+	if (ldlm_is_destroyed(lock)) {
+		CDEBUG(D_OTHER, "Lock destroyed, not adding to resource\n");
+		return;
+	}
+
+	LASSERT(list_empty(&lock->l_res_link));
+	LASSERT(list_empty(&lock->l_sl_mode));
+	LASSERT(list_empty(&lock->l_sl_policy));
+
+	/*
+	 * lock->link == prev->link means lock is first starting the group.
+	 * Don't re-add to itself to suppress kernel warnings.
+	 */
+	if (&lock->l_res_link != prev->res_link)
+		list_add(&lock->l_res_link, prev->res_link);
+	if (&lock->l_sl_mode != prev->mode_link)
+		list_add(&lock->l_sl_mode, prev->mode_link);
+	if (&lock->l_sl_policy != prev->policy_link)
+		list_add(&lock->l_sl_policy, prev->policy_link);
+}
+
+/**
+ * Add a lock to granted list on a resource maintaining skiplist
+ * correctness.
+ */
+static void ldlm_grant_lock_with_skiplist(struct ldlm_lock *lock)
+{
+	struct sl_insert_point prev;
+
+	LASSERT(lock->l_req_mode == lock->l_granted_mode);
+
+	search_granted_lock(&lock->l_resource->lr_granted, lock, &prev);
+	ldlm_granted_list_add_lock(lock, &prev);
+}
+
+/**
+ * Perform lock granting bookkeeping.
+ *
+ * Includes putting the lock into granted list and updating lock mode.
+ * NOTE: called by
+ *  - ldlm_lock_enqueue
+ *  - ldlm_reprocess_queue
+ *  - ldlm_lock_convert
+ *
+ * must be called with lr_lock held
+ */
+void ldlm_grant_lock(struct ldlm_lock *lock, struct list_head *work_list)
+{
+	struct ldlm_resource *res = lock->l_resource;
+
+	check_res_locked(res);
+
+	lock->l_granted_mode = lock->l_req_mode;
+
+	if (work_list && lock->l_completion_ast)
+		ldlm_add_ast_work_item(lock, NULL, work_list);
+
+	if (res->lr_type == LDLM_PLAIN || res->lr_type == LDLM_IBITS) {
+		ldlm_grant_lock_with_skiplist(lock);
+	} else if (res->lr_type == LDLM_EXTENT) {
+		ldlm_extent_add_lock(res, lock);
+	} else if (res->lr_type == LDLM_FLOCK) {
+		/*
+		 * We should not add locks to granted list in
+		 * the following cases:
+		 * - this is an UNLOCK but not a real lock;
+		 * - this is a TEST lock;
+		 * - this is a F_CANCELLK lock (async flock has req_mode == 0)
+		 * - this is a deadlock (flock cannot be granted)
+		 */
+		if (!lock->l_req_mode || lock->l_req_mode == LCK_NL ||
+		    ldlm_is_test_lock(lock) || ldlm_is_flock_deadlock(lock))
+			return;
+		ldlm_resource_add_lock(res, &res->lr_granted, lock);
+	} else {
+		LBUG();
+	}
+
+	ldlm_pool_add(&ldlm_res_to_ns(res)->ns_pool, lock);
+}
+
+/**
+ * Describe the overlap between two locks.  itree_overlap_cb data.
+ */
+struct lock_match_data {
+	struct ldlm_lock	*lmd_old;
+	struct ldlm_lock	*lmd_lock;
+	enum ldlm_mode		*lmd_mode;
+	union ldlm_policy_data	*lmd_policy;
+	__u64			 lmd_flags;
+	int			 lmd_unref;
+};
+
+/**
+ * Check if the given @lock meets the criteria for a match.
+ * A reference on the lock is taken if matched.
+ *
+ * \param lock	test-against this lock
+ * \param data	parameters
+ */
+static bool lock_matches(struct ldlm_lock *lock, void *vdata)
+{
+	struct lock_match_data *data = vdata;
+	union ldlm_policy_data *lpol = &lock->l_policy_data;
+	enum ldlm_mode match;
+
+	if (lock == data->lmd_old)
+		return true;
+
+	/*
+	 * Check if this lock can be matched.
+	 * Used by LU-2919(exclusive open) for open lease lock
+	 */
+	if (ldlm_is_excl(lock))
+		return false;
+
+	/*
+	 * llite sometimes wants to match locks that will be
+	 * canceled when their users drop, but we allow it to match
+	 * if it passes in CBPENDING and the lock still has users.
+	 * this is generally only going to be used by children
+	 * whose parents already hold a lock so forward progress
+	 * can still happen.
+	 */
+	if (ldlm_is_cbpending(lock) &&
+	    !(data->lmd_flags & LDLM_FL_CBPENDING))
+		return false;
+
+	if (!data->lmd_unref && ldlm_is_cbpending(lock) &&
+	    !lock->l_readers && !lock->l_writers)
+		return false;
+
+	if (!(lock->l_req_mode & *data->lmd_mode))
+		return false;
+	match = lock->l_req_mode;
+
+	switch (lock->l_resource->lr_type) {
+	case LDLM_EXTENT:
+		if (lpol->l_extent.start > data->lmd_policy->l_extent.start ||
+		    lpol->l_extent.end < data->lmd_policy->l_extent.end)
+			return false;
+
+		if (unlikely(match == LCK_GROUP) &&
+		    data->lmd_policy->l_extent.gid != LDLM_GID_ANY &&
+		    lpol->l_extent.gid != data->lmd_policy->l_extent.gid)
+			return false;
+		break;
+	case LDLM_IBITS:
+		/*
+		 * We match if we have existing lock with same or wider set
+		 * of bits.
+		 */
+		if ((lpol->l_inodebits.bits &
+		     data->lmd_policy->l_inodebits.bits) !=
+		    data->lmd_policy->l_inodebits.bits)
+			return false;
+		break;
+	default:
+		break;
+	}
+	/*
+	 * We match if we have existing lock with same or wider set
+	 * of bits.
+	 */
+	if (!data->lmd_unref && LDLM_HAVE_MASK(lock, GONE))
+		return false;
+
+	if (!equi(data->lmd_flags & LDLM_FL_LOCAL_ONLY, ldlm_is_local(lock)))
+		return false;
+
+	if (data->lmd_flags & LDLM_FL_TEST_LOCK) {
+		LDLM_LOCK_GET(lock);
+		ldlm_lock_touch_in_lru(lock);
+	} else {
+		ldlm_lock_addref_internal_nolock(lock, match);
+	}
+
+	*data->lmd_mode = match;
+	data->lmd_lock = lock;
+
+	return true;
+}
+
+/**
+ * Search for a lock with given parameters in interval trees.
+ *
+ * \param res	search for a lock in this resource
+ * \param data	parameters
+ *
+ * \retval	a referenced lock or NULL.
+ */
+static struct ldlm_lock *search_itree(struct ldlm_resource *res,
+				      struct lock_match_data *data)
+{
+	int idx;
+
+	for (idx = 0; idx < LCK_MODE_NUM; idx++) {
+		struct ldlm_interval_tree *tree = &res->lr_itree[idx];
+
+		if (RB_EMPTY_ROOT(&tree->lit_root.rb_root))
+			continue;
+
+		if (!(tree->lit_mode & *data->lmd_mode))
+			continue;
+
+		ldlm_extent_search(&tree->lit_root,
+				   data->lmd_policy->l_extent.start,
+				   data->lmd_policy->l_extent.end,
+				   lock_matches, data);
+	}
+	return data->lmd_lock;
+}
+
+/**
+ * Search for a lock with given properties in a queue.
+ *
+ * \param queue	search for a lock in this queue
+ * \param data	parameters
+ *
+ * \retval	a referenced lock or NULL.
+ */
+static struct ldlm_lock *search_queue(struct list_head *queue,
+				      struct lock_match_data *data)
+{
+	struct ldlm_lock *lock;
+
+	list_for_each_entry(lock, queue, l_res_link)
+		if (lock_matches(lock, data))
+			return data->lmd_lock;
+	return NULL;
+}
+
+void ldlm_lock_fail_match_locked(struct ldlm_lock *lock)
+{
+	if ((lock->l_flags & LDLM_FL_FAIL_NOTIFIED) == 0) {
+		lock->l_flags |= LDLM_FL_FAIL_NOTIFIED;
+		wake_up_all(&lock->l_waitq);
+	}
+}
+
+/**
+ * Mark lock as "matchable" by OST.
+ *
+ * Used to prevent certain races in LOV/OSC where the lock is granted, but LVB
+ * is not yet valid.
+ * Assumes LDLM lock is already locked.
+ */
+void ldlm_lock_allow_match_locked(struct ldlm_lock *lock)
+{
+	ldlm_set_lvb_ready(lock);
+	wake_up_all(&lock->l_waitq);
+}
+EXPORT_SYMBOL(ldlm_lock_allow_match_locked);
+
+/**
+ * Mark lock as "matchable" by OST.
+ * Locks the lock and then \see ldlm_lock_allow_match_locked
+ */
+void ldlm_lock_allow_match(struct ldlm_lock *lock)
+{
+	lock_res_and_lock(lock);
+	ldlm_lock_allow_match_locked(lock);
+	unlock_res_and_lock(lock);
+}
+EXPORT_SYMBOL(ldlm_lock_allow_match);
+
+/**
+ * Attempt to find a lock with specified properties.
+ *
+ * Typically returns a reference to matched lock unless LDLM_FL_TEST_LOCK is
+ * set in \a flags
+ *
+ * Can be called in two ways:
+ *
+ * If 'ns' is NULL, then lockh describes an existing lock that we want to look
+ * for a duplicate of.
+ *
+ * Otherwise, all of the fields must be filled in, to match against.
+ *
+ * If 'flags' contains LDLM_FL_LOCAL_ONLY, then only match local locks on the
+ *     server (ie, connh is NULL)
+ * If 'flags' contains LDLM_FL_BLOCK_GRANTED, then only locks on the granted
+ *     list will be considered
+ * If 'flags' contains LDLM_FL_CBPENDING, then locks that have been marked
+ *     to be canceled can still be matched as long as they still have reader
+ *     or writer referneces
+ * If 'flags' contains LDLM_FL_TEST_LOCK, then don't actually reference a lock,
+ *     just tell us if we would have matched.
+ *
+ * \retval 1 if it finds an already-existing lock that is compatible; in this
+ * case, lockh is filled in with a addref()ed lock
+ *
+ * We also check security context, and if that fails we simply return 0 (to
+ * keep caller code unchanged), the context failure will be discovered by
+ * caller sometime later.
+ */
+enum ldlm_mode ldlm_lock_match(struct ldlm_namespace *ns, __u64 flags,
+			       const struct ldlm_res_id *res_id,
+			       enum ldlm_type type,
+			       union ldlm_policy_data *policy,
+			       enum ldlm_mode mode,
+			       struct lustre_handle *lockh, int unref)
+{
+	struct lock_match_data data = {
+		.lmd_old	= NULL,
+		.lmd_lock	= NULL,
+		.lmd_mode	= &mode,
+		.lmd_policy	= policy,
+		.lmd_flags	= flags,
+		.lmd_unref	= unref,
+	};
+	struct ldlm_resource *res;
+	struct ldlm_lock *lock;
+	int rc = 0;
+
+	if (!ns) {
+		data.lmd_old = ldlm_handle2lock(lockh);
+		LASSERT(data.lmd_old);
+
+		ns = ldlm_lock_to_ns(data.lmd_old);
+		res_id = &data.lmd_old->l_resource->lr_name;
+		type = data.lmd_old->l_resource->lr_type;
+		*data.lmd_mode = data.lmd_old->l_req_mode;
+	}
+
+	res = ldlm_resource_get(ns, NULL, res_id, type, 0);
+	if (IS_ERR(res)) {
+		LASSERT(!data.lmd_old);
+		return 0;
+	}
+
+	LDLM_RESOURCE_ADDREF(res);
+	lock_res(res);
+
+	if (res->lr_type == LDLM_EXTENT)
+		lock = search_itree(res, &data);
+	else
+		lock = search_queue(&res->lr_granted, &data);
+	if (lock) {
+		rc = 1;
+		goto out;
+	}
+	if (flags & LDLM_FL_BLOCK_GRANTED) {
+		rc = 0;
+		goto out;
+	}
+	lock = search_queue(&res->lr_waiting, &data);
+	if (lock) {
+		rc = 1;
+		goto out;
+	}
+out:
+	unlock_res(res);
+	LDLM_RESOURCE_DELREF(res);
+	ldlm_resource_putref(res);
+
+	if (lock) {
+		ldlm_lock2handle(lock, lockh);
+		if ((flags & LDLM_FL_LVB_READY) && !ldlm_is_lvb_ready(lock)) {
+			__u64 wait_flags = LDLM_FL_LVB_READY |
+				LDLM_FL_DESTROYED | LDLM_FL_FAIL_NOTIFIED;
+
+			if (lock->l_completion_ast) {
+				int err = lock->l_completion_ast(lock,
+							  LDLM_FL_WAIT_NOREPROC,
+								 NULL);
+				if (err) {
+					if (flags & LDLM_FL_TEST_LOCK)
+						LDLM_LOCK_RELEASE(lock);
+					else
+						ldlm_lock_decref_internal(lock,
+									  mode);
+					rc = 0;
+					goto out2;
+				}
+			}
+
+			/* XXX FIXME see comment on CAN_MATCH in lustre_dlm.h */
+			wait_event_idle_timeout(lock->l_waitq,
+						lock->l_flags & wait_flags,
+						obd_timeout * HZ);
+			if (!ldlm_is_lvb_ready(lock)) {
+				if (flags & LDLM_FL_TEST_LOCK)
+					LDLM_LOCK_RELEASE(lock);
+				else
+					ldlm_lock_decref_internal(lock, mode);
+				rc = 0;
+			}
+		}
+	}
+ out2:
+	if (rc) {
+		LDLM_DEBUG(lock, "matched (%llu %llu)",
+			   (type == LDLM_PLAIN || type == LDLM_IBITS) ?
+				res_id->name[2] : policy->l_extent.start,
+			   (type == LDLM_PLAIN || type == LDLM_IBITS) ?
+				res_id->name[3] : policy->l_extent.end);
+
+		/* check user's security context */
+		if (lock->l_conn_export &&
+		    sptlrpc_import_check_ctx(
+				class_exp2cliimp(lock->l_conn_export))) {
+			if (!(flags & LDLM_FL_TEST_LOCK))
+				ldlm_lock_decref_internal(lock, mode);
+			rc = 0;
+		}
+
+		if (flags & LDLM_FL_TEST_LOCK)
+			LDLM_LOCK_RELEASE(lock);
+
+	} else if (!(flags & LDLM_FL_TEST_LOCK)) {/*less verbose for test-only*/
+		LDLM_DEBUG_NOLOCK("not matched ns %p type %u mode %u res %llu/%llu (%llu %llu)",
+				  ns, type, mode, res_id->name[0],
+				  res_id->name[1],
+				  (type == LDLM_PLAIN || type == LDLM_IBITS) ?
+					res_id->name[2] : policy->l_extent.start,
+				  (type == LDLM_PLAIN || type == LDLM_IBITS) ?
+					res_id->name[3] : policy->l_extent.end);
+	}
+	if (data.lmd_old)
+		LDLM_LOCK_PUT(data.lmd_old);
+
+	return rc ? mode : 0;
+}
+EXPORT_SYMBOL(ldlm_lock_match);
+
+enum ldlm_mode ldlm_revalidate_lock_handle(const struct lustre_handle *lockh,
+					   __u64 *bits)
+{
+	struct ldlm_lock *lock;
+	enum ldlm_mode mode = 0;
+
+	lock = ldlm_handle2lock(lockh);
+	if (lock) {
+		lock_res_and_lock(lock);
+		if (LDLM_HAVE_MASK(lock, GONE))
+			goto out;
+
+		if (ldlm_is_cbpending(lock) &&
+		    lock->l_readers == 0 && lock->l_writers == 0)
+			goto out;
+
+		if (bits)
+			*bits = lock->l_policy_data.l_inodebits.bits;
+		mode = lock->l_granted_mode;
+		ldlm_lock_addref_internal_nolock(lock, mode);
+	}
+
+out:
+	if (lock) {
+		unlock_res_and_lock(lock);
+		LDLM_LOCK_PUT(lock);
+	}
+	return mode;
+}
+EXPORT_SYMBOL(ldlm_revalidate_lock_handle);
+
+/** The caller must guarantee that the buffer is large enough. */
+int ldlm_fill_lvb(struct ldlm_lock *lock, struct req_capsule *pill,
+		  enum req_location loc, void *data, int size)
+{
+	void *lvb;
+
+	LASSERT(data);
+	LASSERT(size >= 0);
+
+	switch (lock->l_lvb_type) {
+	case LVB_T_OST:
+		if (size == sizeof(struct ost_lvb)) {
+			if (loc == RCL_CLIENT)
+				lvb = req_capsule_client_swab_get(pill,
+								  &RMF_DLM_LVB,
+							lustre_swab_ost_lvb);
+			else
+				lvb = req_capsule_server_swab_get(pill,
+								  &RMF_DLM_LVB,
+							lustre_swab_ost_lvb);
+			if (unlikely(!lvb)) {
+				LDLM_ERROR(lock, "no LVB");
+				return -EPROTO;
+			}
+
+			memcpy(data, lvb, size);
+		} else if (size == sizeof(struct ost_lvb_v1)) {
+			struct ost_lvb *olvb = data;
+
+			if (loc == RCL_CLIENT)
+				lvb = req_capsule_client_swab_get(pill,
+								  &RMF_DLM_LVB,
+							lustre_swab_ost_lvb_v1);
+			else
+				lvb = req_capsule_server_sized_swab_get(pill,
+						&RMF_DLM_LVB, size,
+						lustre_swab_ost_lvb_v1);
+			if (unlikely(!lvb)) {
+				LDLM_ERROR(lock, "no LVB");
+				return -EPROTO;
+			}
+
+			memcpy(data, lvb, size);
+			olvb->lvb_mtime_ns = 0;
+			olvb->lvb_atime_ns = 0;
+			olvb->lvb_ctime_ns = 0;
+		} else {
+			LDLM_ERROR(lock, "Replied unexpected ost LVB size %d",
+				   size);
+			return -EINVAL;
+		}
+		break;
+	case LVB_T_LQUOTA:
+		if (size == sizeof(struct lquota_lvb)) {
+			if (loc == RCL_CLIENT)
+				lvb = req_capsule_client_swab_get(pill,
+								  &RMF_DLM_LVB,
+							lustre_swab_lquota_lvb);
+			else
+				lvb = req_capsule_server_swab_get(pill,
+								  &RMF_DLM_LVB,
+							lustre_swab_lquota_lvb);
+			if (unlikely(!lvb)) {
+				LDLM_ERROR(lock, "no LVB");
+				return -EPROTO;
+			}
+
+			memcpy(data, lvb, size);
+		} else {
+			LDLM_ERROR(lock,
+				   "Replied unexpected lquota LVB size %d",
+				   size);
+			return -EINVAL;
+		}
+		break;
+	case LVB_T_LAYOUT:
+		if (size == 0)
+			break;
+
+		if (loc == RCL_CLIENT)
+			lvb = req_capsule_client_get(pill, &RMF_DLM_LVB);
+		else
+			lvb = req_capsule_server_get(pill, &RMF_DLM_LVB);
+		if (unlikely(!lvb)) {
+			LDLM_ERROR(lock, "no LVB");
+			return -EPROTO;
+		}
+
+		memcpy(data, lvb, size);
+		break;
+	default:
+		LDLM_ERROR(lock, "Unknown LVB type: %d", lock->l_lvb_type);
+		dump_stack();
+		return -EINVAL;
+	}
+
+	return 0;
+}
+
+/**
+ * Create and fill in new LDLM lock with specified properties.
+ * Returns a referenced lock
+ */
+struct ldlm_lock *ldlm_lock_create(struct ldlm_namespace *ns,
+				   const struct ldlm_res_id *res_id,
+				   enum ldlm_type type,
+				   enum ldlm_mode mode,
+				   const struct ldlm_callback_suite *cbs,
+				   void *data, __u32 lvb_len,
+				   enum lvb_type lvb_type)
+{
+	struct ldlm_lock *lock;
+	struct ldlm_resource *res;
+	int rc;
+
+	res = ldlm_resource_get(ns, NULL, res_id, type, 1);
+	if (IS_ERR(res))
+		return ERR_CAST(res);
+
+	lock = ldlm_lock_new(res);
+	if (!lock) {
+		ldlm_resource_putref(res);
+		return ERR_PTR(-ENOMEM);
+	}
+
+	lock->l_req_mode = mode;
+	lock->l_ast_data = data;
+	lock->l_pid = current->pid;
+	if (cbs) {
+		lock->l_blocking_ast = cbs->lcs_blocking;
+		lock->l_completion_ast = cbs->lcs_completion;
+		lock->l_glimpse_ast = cbs->lcs_glimpse;
+	}
+
+	if (lvb_len) {
+		lock->l_lvb_len = lvb_len;
+		lock->l_lvb_data = kzalloc(lvb_len, GFP_NOFS);
+		if (!lock->l_lvb_data) {
+			rc = -ENOMEM;
+			goto out;
+		}
+	}
+
+	lock->l_lvb_type = lvb_type;
+	if (OBD_FAIL_CHECK(OBD_FAIL_LDLM_NEW_LOCK)) {
+		rc = -ENOENT;
+		goto out;
+	}
+
+	return lock;
+
+out:
+	ldlm_lock_destroy(lock);
+	LDLM_LOCK_RELEASE(lock);
+	return ERR_PTR(rc);
+}
+
+
+
+/**
+ * Enqueue (request) a lock.
+ * On the client this is called from ldlm_cli_enqueue_fini
+ * after we already got an initial reply from the server with some status.
+ *
+ * Does not block. As a result of enqueue the lock would be put
+ * into granted or waiting list.
+ */
+enum ldlm_error ldlm_lock_enqueue(struct ldlm_namespace *ns,
+				  struct ldlm_lock **lockp,
+				  void *cookie, __u64 *flags)
+{
+	struct ldlm_lock *lock = *lockp;
+	struct ldlm_resource *res = lock->l_resource;
+
+	lock_res_and_lock(lock);
+	if (lock->l_req_mode == lock->l_granted_mode) {
+		/* The server returned a blocked lock, but it was granted
+		 * before we got a chance to actually enqueue it.  We don't
+		 * need to do anything else.
+		 */
+		*flags &= ~LDLM_FL_BLOCKED_MASK;
+		goto out;
+	}
+
+	ldlm_resource_unlink_lock(lock);
+
+	/* Some flags from the enqueue want to make it into the AST, via the
+	 * lock's l_flags.
+	 */
+	if (*flags & LDLM_FL_AST_DISCARD_DATA)
+		ldlm_set_ast_discard_data(lock);
+	if (*flags & LDLM_FL_TEST_LOCK)
+		ldlm_set_test_lock(lock);
+
+	/*
+	 * This distinction between local lock trees is very important; a client
+	 * namespace only has information about locks taken by that client, and
+	 * thus doesn't have enough information to decide for itself if it can
+	 * be granted (below).  In this case, we do exactly what the server
+	 * tells us to do, as dictated by the 'flags'.
+	 */
+	if (*flags & (LDLM_FL_BLOCK_WAIT | LDLM_FL_BLOCK_GRANTED))
+		ldlm_resource_add_lock(res, &res->lr_waiting, lock);
+	else
+		ldlm_grant_lock(lock, NULL);
+
+out:
+	unlock_res_and_lock(lock);
+	return ELDLM_OK;
+}
+
+/**
+ * Process a call to blocking AST callback for a lock in ast_work list
+ */
+static int
+ldlm_work_bl_ast_lock(struct ptlrpc_request_set *rqset, void *opaq)
+{
+	struct ldlm_cb_set_arg *arg = opaq;
+	struct ldlm_lock_desc   d;
+	int		     rc;
+	struct ldlm_lock       *lock;
+
+	if (list_empty(arg->list))
+		return -ENOENT;
+
+	lock = list_first_entry(arg->list, struct ldlm_lock, l_bl_ast);
+
+	/* nobody should touch l_bl_ast */
+	lock_res_and_lock(lock);
+	list_del_init(&lock->l_bl_ast);
+
+	LASSERT(ldlm_is_ast_sent(lock));
+	LASSERT(lock->l_bl_ast_run == 0);
+	LASSERT(lock->l_blocking_lock);
+	lock->l_bl_ast_run++;
+	unlock_res_and_lock(lock);
+
+	ldlm_lock2desc(lock->l_blocking_lock, &d);
+
+	rc = lock->l_blocking_ast(lock, &d, (void *)arg, LDLM_CB_BLOCKING);
+	LDLM_LOCK_RELEASE(lock->l_blocking_lock);
+	lock->l_blocking_lock = NULL;
+	LDLM_LOCK_RELEASE(lock);
+
+	return rc;
+}
+
+/**
+ * Process a call to completion AST callback for a lock in ast_work list
+ */
+static int
+ldlm_work_cp_ast_lock(struct ptlrpc_request_set *rqset, void *opaq)
+{
+	struct ldlm_cb_set_arg  *arg = opaq;
+	int		      rc = 0;
+	struct ldlm_lock	*lock;
+	ldlm_completion_callback completion_callback;
+
+	if (list_empty(arg->list))
+		return -ENOENT;
+
+	lock = list_first_entry(arg->list, struct ldlm_lock, l_cp_ast);
+
+	/* It's possible to receive a completion AST before we've set
+	 * the l_completion_ast pointer: either because the AST arrived
+	 * before the reply, or simply because there's a small race
+	 * window between receiving the reply and finishing the local
+	 * enqueue. (bug 842)
+	 *
+	 * This can't happen with the blocking_ast, however, because we
+	 * will never call the local blocking_ast until we drop our
+	 * reader/writer reference, which we won't do until we get the
+	 * reply and finish enqueueing.
+	 */
+
+	/* nobody should touch l_cp_ast */
+	lock_res_and_lock(lock);
+	list_del_init(&lock->l_cp_ast);
+	LASSERT(ldlm_is_cp_reqd(lock));
+	/* save l_completion_ast since it can be changed by
+	 * mds_intent_policy(), see bug 14225
+	 */
+	completion_callback = lock->l_completion_ast;
+	ldlm_clear_cp_reqd(lock);
+	unlock_res_and_lock(lock);
+
+	if (completion_callback)
+		rc = completion_callback(lock, 0, (void *)arg);
+	LDLM_LOCK_RELEASE(lock);
+
+	return rc;
+}
+
+/**
+ * Process a call to revocation AST callback for a lock in ast_work list
+ */
+static int
+ldlm_work_revoke_ast_lock(struct ptlrpc_request_set *rqset, void *opaq)
+{
+	struct ldlm_cb_set_arg *arg = opaq;
+	struct ldlm_lock_desc   desc;
+	int		     rc;
+	struct ldlm_lock       *lock;
+
+	if (list_empty(arg->list))
+		return -ENOENT;
+
+	lock = list_first_entry(arg->list, struct ldlm_lock, l_rk_ast);
+	list_del_init(&lock->l_rk_ast);
+
+	/* the desc just pretend to exclusive */
+	ldlm_lock2desc(lock, &desc);
+	desc.l_req_mode = LCK_EX;
+	desc.l_granted_mode = 0;
+
+	rc = lock->l_blocking_ast(lock, &desc, (void *)arg, LDLM_CB_BLOCKING);
+	LDLM_LOCK_RELEASE(lock);
+
+	return rc;
+}
+
+/**
+ * Process a call to glimpse AST callback for a lock in ast_work list
+ */
+static int ldlm_work_gl_ast_lock(struct ptlrpc_request_set *rqset, void *opaq)
+{
+	struct ldlm_cb_set_arg		*arg = opaq;
+	struct ldlm_glimpse_work	*gl_work;
+	struct ldlm_lock		*lock;
+	int				 rc = 0;
+
+	if (list_empty(arg->list))
+		return -ENOENT;
+
+	gl_work = list_first_entry(arg->list, struct ldlm_glimpse_work,
+			     gl_list);
+	list_del_init(&gl_work->gl_list);
+
+	lock = gl_work->gl_lock;
+
+	/* transfer the glimpse descriptor to ldlm_cb_set_arg */
+	arg->gl_desc = gl_work->gl_desc;
+
+	/* invoke the actual glimpse callback */
+	if (lock->l_glimpse_ast(lock, (void *)arg) == 0)
+		rc = 1;
+
+	LDLM_LOCK_RELEASE(lock);
+
+	if ((gl_work->gl_flags & LDLM_GL_WORK_NOFREE) == 0)
+		kfree(gl_work);
+
+	return rc;
+}
+
+/**
+ * Process list of locks in need of ASTs being sent.
+ *
+ * Used on server to send multiple ASTs together instead of sending one by
+ * one.
+ */
+int ldlm_run_ast_work(struct ldlm_namespace *ns, struct list_head *rpc_list,
+		      enum ldlm_desc_ast_t ast_type)
+{
+	struct ldlm_cb_set_arg *arg;
+	set_producer_func       work_ast_lock;
+	int		     rc;
+
+	if (list_empty(rpc_list))
+		return 0;
+
+	arg = kzalloc(sizeof(*arg), GFP_NOFS);
+	if (!arg)
+		return -ENOMEM;
+
+	atomic_set(&arg->restart, 0);
+	arg->list = rpc_list;
+
+	switch (ast_type) {
+	case LDLM_WORK_BL_AST:
+		arg->type = LDLM_BL_CALLBACK;
+		work_ast_lock = ldlm_work_bl_ast_lock;
+		break;
+	case LDLM_WORK_CP_AST:
+		arg->type = LDLM_CP_CALLBACK;
+		work_ast_lock = ldlm_work_cp_ast_lock;
+		break;
+	case LDLM_WORK_REVOKE_AST:
+		arg->type = LDLM_BL_CALLBACK;
+		work_ast_lock = ldlm_work_revoke_ast_lock;
+		break;
+	case LDLM_WORK_GL_AST:
+		arg->type = LDLM_GL_CALLBACK;
+		work_ast_lock = ldlm_work_gl_ast_lock;
+		break;
+	default:
+		LBUG();
+	}
+
+	/* We create a ptlrpc request set with flow control extension.
+	 * This request set will use the work_ast_lock function to produce new
+	 * requests and will send a new request each time one completes in order
+	 * to keep the number of requests in flight to ns_max_parallel_ast
+	 */
+	arg->set = ptlrpc_prep_fcset(ns->ns_max_parallel_ast ? : UINT_MAX,
+				     work_ast_lock, arg);
+	if (!arg->set) {
+		rc = -ENOMEM;
+		goto out;
+	}
+
+	ptlrpc_set_wait(arg->set);
+	ptlrpc_set_destroy(arg->set);
+
+	rc = atomic_read(&arg->restart) ? -ERESTART : 0;
+	goto out;
+out:
+	kfree(arg);
+	return rc;
+}
+
+static bool is_bl_done(struct ldlm_lock *lock)
+{
+	bool bl_done = true;
+
+	if (!ldlm_is_bl_done(lock)) {
+		lock_res_and_lock(lock);
+		bl_done = ldlm_is_bl_done(lock);
+		unlock_res_and_lock(lock);
+	}
+
+	return bl_done;
+}
+
+/**
+ * Helper function to call blocking AST for LDLM lock \a lock in a
+ * "cancelling" mode.
+ */
+void ldlm_cancel_callback(struct ldlm_lock *lock)
+{
+	check_res_locked(lock->l_resource);
+	if (!ldlm_is_cancel(lock)) {
+		ldlm_set_cancel(lock);
+		if (lock->l_blocking_ast) {
+			unlock_res_and_lock(lock);
+			lock->l_blocking_ast(lock, NULL, lock->l_ast_data,
+					     LDLM_CB_CANCELING);
+			lock_res_and_lock(lock);
+		} else {
+			LDLM_DEBUG(lock, "no blocking ast");
+		}
+		/* only canceller can set bl_done bit */
+		ldlm_set_bl_done(lock);
+		wake_up_all(&lock->l_waitq);
+	} else if (!ldlm_is_bl_done(lock)) {
+		/*
+		 * The lock is guaranteed to have been canceled once
+		 * returning from this function.
+		 */
+		unlock_res_and_lock(lock);
+		wait_event_idle(lock->l_waitq, is_bl_done(lock));
+		lock_res_and_lock(lock);
+	}
+}
+
+/**
+ * Remove skiplist-enabled LDLM lock \a req from granted list
+ */
+void ldlm_unlink_lock_skiplist(struct ldlm_lock *req)
+{
+	if (req->l_resource->lr_type != LDLM_PLAIN &&
+	    req->l_resource->lr_type != LDLM_IBITS)
+		return;
+
+	list_del_init(&req->l_sl_policy);
+	list_del_init(&req->l_sl_mode);
+}
+
+/**
+ * Attempts to cancel LDLM lock \a lock that has no reader/writer references.
+ */
+void ldlm_lock_cancel(struct ldlm_lock *lock)
+{
+	struct ldlm_resource *res;
+	struct ldlm_namespace *ns;
+
+	lock_res_and_lock(lock);
+
+	res = lock->l_resource;
+	ns  = ldlm_res_to_ns(res);
+
+	/* Please do not, no matter how tempting, remove this LBUG without
+	 * talking to me first. -phik
+	 */
+	if (lock->l_readers || lock->l_writers) {
+		LDLM_ERROR(lock, "lock still has references");
+		LBUG();
+	}
+
+	/* Releases cancel callback. */
+	ldlm_cancel_callback(lock);
+
+	ldlm_resource_unlink_lock(lock);
+	ldlm_lock_destroy_nolock(lock);
+
+	if (lock->l_granted_mode == lock->l_req_mode)
+		ldlm_pool_del(&ns->ns_pool, lock);
+
+	/* Make sure we will not be called again for same lock what is possible
+	 * if not to zero out lock->l_granted_mode
+	 */
+	lock->l_granted_mode = LCK_MINMODE;
+	unlock_res_and_lock(lock);
+}
+EXPORT_SYMBOL(ldlm_lock_cancel);
+
+/**
+ * Set opaque data into the lock that only makes sense to upper layer.
+ */
+int ldlm_lock_set_data(const struct lustre_handle *lockh, void *data)
+{
+	struct ldlm_lock *lock = ldlm_handle2lock(lockh);
+	int rc = -EINVAL;
+
+	if (lock) {
+		if (!lock->l_ast_data)
+			lock->l_ast_data = data;
+		if (lock->l_ast_data == data)
+			rc = 0;
+		LDLM_LOCK_PUT(lock);
+	}
+	return rc;
+}
+EXPORT_SYMBOL(ldlm_lock_set_data);
+
+struct export_cl_data {
+	struct obd_export	*ecl_exp;
+	int			ecl_loop;
+};
+
+/**
+ * Print lock with lock handle \a lockh description into debug log.
+ *
+ * Used when printing all locks on a resource for debug purposes.
+ */
+void ldlm_lock_dump_handle(int level, const struct lustre_handle *lockh)
+{
+	struct ldlm_lock *lock;
+
+	if (!((libcfs_debug | D_ERROR) & level))
+		return;
+
+	lock = ldlm_handle2lock(lockh);
+	if (!lock)
+		return;
+
+	LDLM_DEBUG_LIMIT(level, lock, "###");
+
+	LDLM_LOCK_PUT(lock);
+}
+EXPORT_SYMBOL(ldlm_lock_dump_handle);
+
+/**
+ * Print lock information with custom message into debug log.
+ * Helper function.
+ */
+void _ldlm_lock_debug(struct ldlm_lock *lock,
+		      struct libcfs_debug_msg_data *msgdata,
+		      const char *fmt, ...)
+{
+	va_list args;
+	struct obd_export *exp = lock->l_export;
+	struct ldlm_resource *resource = lock->l_resource;
+	char *nid = "local";
+
+	va_start(args, fmt);
+
+	if (exp && exp->exp_connection) {
+		nid = libcfs_nid2str(exp->exp_connection->c_peer.nid);
+	} else if (exp && exp->exp_obd) {
+		struct obd_import *imp = exp->exp_obd->u.cli.cl_import;
+
+		nid = libcfs_nid2str(imp->imp_connection->c_peer.nid);
+	}
+
+	if (!resource) {
+		libcfs_debug_vmsg2(msgdata, fmt, args,
+				   " ns: \?\? lock: %p/%#llx lrc: %d/%d,%d mode: %s/%s res: \?\? rrc=\?\? type: \?\?\? flags: %#llx nid: %s remote: %#llx expref: %d pid: %u timeout: %lu lvb_type: %d\n",
+				   lock,
+				   lock->l_handle.h_cookie,
+				   atomic_read(&lock->l_refc),
+				   lock->l_readers, lock->l_writers,
+				   ldlm_lockname[lock->l_granted_mode],
+				   ldlm_lockname[lock->l_req_mode],
+				   lock->l_flags, nid,
+				   lock->l_remote_handle.cookie,
+				   exp ? atomic_read(&exp->exp_refcount) : -99,
+				   lock->l_pid, lock->l_callback_timeout,
+				   lock->l_lvb_type);
+		va_end(args);
+		return;
+	}
+
+	switch (resource->lr_type) {
+	case LDLM_EXTENT:
+		libcfs_debug_vmsg2(msgdata, fmt, args,
+				   " ns: %s lock: %p/%#llx lrc: %d/%d,%d mode: %s/%s res: " DLDLMRES " rrc: %d type: %s [%llu->%llu] (req %llu->%llu) flags: %#llx nid: %s remote: %#llx expref: %d pid: %u timeout: %lu lvb_type: %d\n",
+				   ldlm_lock_to_ns_name(lock), lock,
+				   lock->l_handle.h_cookie,
+				   atomic_read(&lock->l_refc),
+				   lock->l_readers, lock->l_writers,
+				   ldlm_lockname[lock->l_granted_mode],
+				   ldlm_lockname[lock->l_req_mode],
+				   PLDLMRES(resource),
+				   atomic_read(&resource->lr_refcount),
+				   ldlm_typename[resource->lr_type],
+				   lock->l_policy_data.l_extent.start,
+				   lock->l_policy_data.l_extent.end,
+				   lock->l_req_extent.start,
+				   lock->l_req_extent.end,
+				   lock->l_flags, nid,
+				   lock->l_remote_handle.cookie,
+				   exp ? atomic_read(&exp->exp_refcount) : -99,
+				   lock->l_pid, lock->l_callback_timeout,
+				   lock->l_lvb_type);
+		break;
+
+	case LDLM_FLOCK:
+		libcfs_debug_vmsg2(msgdata, fmt, args,
+				   " ns: %s lock: %p/%#llx lrc: %d/%d,%d mode: %s/%s res: " DLDLMRES " rrc: %d type: %s pid: %d [%llu->%llu] flags: %#llx nid: %s remote: %#llx expref: %d pid: %u timeout: %lu\n",
+				   ldlm_lock_to_ns_name(lock), lock,
+				   lock->l_handle.h_cookie,
+				   atomic_read(&lock->l_refc),
+				   lock->l_readers, lock->l_writers,
+				   ldlm_lockname[lock->l_granted_mode],
+				   ldlm_lockname[lock->l_req_mode],
+				   PLDLMRES(resource),
+				   atomic_read(&resource->lr_refcount),
+				   ldlm_typename[resource->lr_type],
+				   lock->l_policy_data.l_flock.pid,
+				   lock->l_policy_data.l_flock.start,
+				   lock->l_policy_data.l_flock.end,
+				   lock->l_flags, nid,
+				   lock->l_remote_handle.cookie,
+				   exp ? atomic_read(&exp->exp_refcount) : -99,
+				   lock->l_pid, lock->l_callback_timeout);
+		break;
+
+	case LDLM_IBITS:
+		libcfs_debug_vmsg2(msgdata, fmt, args,
+				   " ns: %s lock: %p/%#llx lrc: %d/%d,%d mode: %s/%s res: " DLDLMRES " bits %#llx rrc: %d type: %s flags: %#llx nid: %s remote: %#llx expref: %d pid: %u timeout: %lu lvb_type: %d\n",
+				   ldlm_lock_to_ns_name(lock),
+				   lock, lock->l_handle.h_cookie,
+				   atomic_read(&lock->l_refc),
+				   lock->l_readers, lock->l_writers,
+				   ldlm_lockname[lock->l_granted_mode],
+				   ldlm_lockname[lock->l_req_mode],
+				   PLDLMRES(resource),
+				   lock->l_policy_data.l_inodebits.bits,
+				   atomic_read(&resource->lr_refcount),
+				   ldlm_typename[resource->lr_type],
+				   lock->l_flags, nid,
+				   lock->l_remote_handle.cookie,
+				   exp ? atomic_read(&exp->exp_refcount) : -99,
+				   lock->l_pid, lock->l_callback_timeout,
+				   lock->l_lvb_type);
+		break;
+
+	default:
+		libcfs_debug_vmsg2(msgdata, fmt, args,
+				   " ns: %s lock: %p/%#llx lrc: %d/%d,%d mode: %s/%s res: " DLDLMRES " rrc: %d type: %s flags: %#llx nid: %s remote: %#llx expref: %d pid: %u timeout: %lu lvb_type: %d\n",
+				   ldlm_lock_to_ns_name(lock),
+				   lock, lock->l_handle.h_cookie,
+				   atomic_read(&lock->l_refc),
+				   lock->l_readers, lock->l_writers,
+				   ldlm_lockname[lock->l_granted_mode],
+				   ldlm_lockname[lock->l_req_mode],
+				   PLDLMRES(resource),
+				   atomic_read(&resource->lr_refcount),
+				   ldlm_typename[resource->lr_type],
+				   lock->l_flags, nid,
+				   lock->l_remote_handle.cookie,
+				   exp ? atomic_read(&exp->exp_refcount) : -99,
+				   lock->l_pid, lock->l_callback_timeout,
+				   lock->l_lvb_type);
+		break;
+	}
+	va_end(args);
+}
+EXPORT_SYMBOL(_ldlm_lock_debug);
diff --git a/drivers/staging/lustre/lustre/ptlrpc/ldlm_lockd.c b/drivers/staging/lustre/lustre/ptlrpc/ldlm_lockd.c
new file mode 100644
index 000000000000..f410ef6c02ef
--- /dev/null
+++ b/drivers/staging/lustre/lustre/ptlrpc/ldlm_lockd.c
@@ -0,0 +1,1154 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * GPL HEADER START
+ *
+ * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 only,
+ * as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License version 2 for more details (a copy is included
+ * in the LICENSE file that accompanied this code).
+ *
+ * You should have received a copy of the GNU General Public License
+ * version 2 along with this program; If not, see
+ * http://www.gnu.org/licenses/gpl-2.0.html
+ *
+ * GPL HEADER END
+ */
+/*
+ * Copyright (c) 2002, 2010, Oracle and/or its affiliates. All rights reserved.
+ * Use is subject to license terms.
+ *
+ * Copyright (c) 2010, 2015, Intel Corporation.
+ */
+/*
+ * This file is part of Lustre, http://www.lustre.org/
+ * Lustre is a trademark of Sun Microsystems, Inc.
+ *
+ * lustre/ldlm/ldlm_lockd.c
+ *
+ * Author: Peter Braam <braam at clusterfs.com>
+ * Author: Phil Schwan <phil at clusterfs.com>
+ */
+
+#define DEBUG_SUBSYSTEM S_LDLM
+
+#include <linux/kthread.h>
+#include <linux/sched/mm.h>
+#include <lustre_dlm.h>
+#include <obd_class.h>
+#include <linux/list.h>
+#include "ldlm_internal.h"
+
+static int ldlm_num_threads;
+module_param(ldlm_num_threads, int, 0444);
+MODULE_PARM_DESC(ldlm_num_threads, "number of DLM service threads to start");
+
+static char *ldlm_cpts;
+module_param(ldlm_cpts, charp, 0444);
+MODULE_PARM_DESC(ldlm_cpts, "CPU partitions ldlm threads should run on");
+
+static struct mutex	ldlm_ref_mutex;
+static int ldlm_refcount;
+
+static struct kobject *ldlm_kobj;
+struct kset *ldlm_ns_kset;
+static struct kset *ldlm_svc_kset;
+
+struct ldlm_cb_async_args {
+	struct ldlm_cb_set_arg *ca_set_arg;
+	struct ldlm_lock       *ca_lock;
+};
+
+/* LDLM state */
+
+static struct ldlm_state *ldlm_state;
+
+#define ELT_STOPPED   0
+#define ELT_READY     1
+#define ELT_TERMINATE 2
+
+struct ldlm_bl_pool {
+	spinlock_t		blp_lock;
+
+	/*
+	 * blp_prio_list is used for callbacks that should be handled
+	 * as a priority. It is used for LDLM_FL_DISCARD_DATA requests.
+	 * see bug 13843
+	 */
+	struct list_head	      blp_prio_list;
+
+	/*
+	 * blp_list is used for all other callbacks which are likely
+	 * to take longer to process.
+	 */
+	struct list_head	      blp_list;
+
+	wait_queue_head_t	     blp_waitq;
+	struct completion	blp_comp;
+	atomic_t	    blp_num_threads;
+	atomic_t	    blp_busy_threads;
+	int		     blp_min_threads;
+	int		     blp_max_threads;
+};
+
+struct ldlm_bl_work_item {
+	struct list_head	      blwi_entry;
+	struct ldlm_namespace  *blwi_ns;
+	struct ldlm_lock_desc   blwi_ld;
+	struct ldlm_lock       *blwi_lock;
+	struct list_head	      blwi_head;
+	int		     blwi_count;
+	struct completion	blwi_comp;
+	enum ldlm_cancel_flags  blwi_flags;
+	int		     blwi_mem_pressure;
+};
+
+/**
+ * Callback handler for receiving incoming blocking ASTs.
+ *
+ * This can only happen on client side.
+ */
+void ldlm_handle_bl_callback(struct ldlm_namespace *ns,
+			     struct ldlm_lock_desc *ld, struct ldlm_lock *lock)
+{
+	int do_ast;
+
+	LDLM_DEBUG(lock, "client blocking AST callback handler");
+
+	lock_res_and_lock(lock);
+	ldlm_set_cbpending(lock);
+
+	if (ldlm_is_cancel_on_block(lock))
+		ldlm_set_cancel(lock);
+
+	do_ast = !lock->l_readers && !lock->l_writers;
+	unlock_res_and_lock(lock);
+
+	if (do_ast) {
+		CDEBUG(D_DLMTRACE,
+		       "Lock %p already unused, calling callback (%p)\n", lock,
+		       lock->l_blocking_ast);
+		if (lock->l_blocking_ast)
+			lock->l_blocking_ast(lock, ld, lock->l_ast_data,
+					     LDLM_CB_BLOCKING);
+	} else {
+		CDEBUG(D_DLMTRACE,
+		       "Lock %p is referenced, will be cancelled later\n",
+		       lock);
+	}
+
+	LDLM_DEBUG(lock, "client blocking callback handler END");
+	LDLM_LOCK_RELEASE(lock);
+}
+
+/**
+ * Callback handler for receiving incoming completion ASTs.
+ *
+ * This only can happen on client side.
+ */
+static void ldlm_handle_cp_callback(struct ptlrpc_request *req,
+				    struct ldlm_namespace *ns,
+				    struct ldlm_request *dlm_req,
+				    struct ldlm_lock *lock)
+{
+	int lvb_len;
+	LIST_HEAD(ast_list);
+	int rc = 0;
+
+	LDLM_DEBUG(lock, "client completion callback handler START");
+
+	if (OBD_FAIL_CHECK(OBD_FAIL_LDLM_CANCEL_BL_CB_RACE)) {
+		int to = HZ;
+
+		while (to > 0) {
+			set_current_state(TASK_INTERRUPTIBLE);
+			schedule_timeout(to);
+			if (lock->l_granted_mode == lock->l_req_mode ||
+			    ldlm_is_destroyed(lock))
+				break;
+		}
+	}
+
+	lvb_len = req_capsule_get_size(&req->rq_pill, &RMF_DLM_LVB, RCL_CLIENT);
+	if (lvb_len < 0) {
+		LDLM_ERROR(lock, "Fail to get lvb_len, rc = %d", lvb_len);
+		rc = lvb_len;
+		goto out;
+	} else if (lvb_len > 0) {
+		if (lock->l_lvb_len > 0) {
+			/* for extent lock, lvb contains ost_lvb{}. */
+			LASSERT(lock->l_lvb_data);
+
+			if (unlikely(lock->l_lvb_len < lvb_len)) {
+				LDLM_ERROR(lock,
+					   "Replied LVB is larger than expectation, expected = %d, replied = %d",
+					   lock->l_lvb_len, lvb_len);
+				rc = -EINVAL;
+				goto out;
+			}
+		} else if (ldlm_has_layout(lock)) { /* for layout lock, lvb has
+						     * variable length
+						     */
+			void *lvb_data;
+
+			lvb_data = kzalloc(lvb_len, GFP_NOFS);
+			if (!lvb_data) {
+				LDLM_ERROR(lock, "No memory: %d.\n", lvb_len);
+				rc = -ENOMEM;
+				goto out;
+			}
+
+			lock_res_and_lock(lock);
+			LASSERT(!lock->l_lvb_data);
+			lock->l_lvb_type = LVB_T_LAYOUT;
+			lock->l_lvb_data = lvb_data;
+			lock->l_lvb_len = lvb_len;
+			unlock_res_and_lock(lock);
+		}
+	}
+
+	lock_res_and_lock(lock);
+	if (ldlm_is_destroyed(lock) ||
+	    lock->l_granted_mode == lock->l_req_mode) {
+		/* bug 11300: the lock has already been granted */
+		unlock_res_and_lock(lock);
+		LDLM_DEBUG(lock, "Double grant race happened");
+		rc = 0;
+		goto out;
+	}
+
+	/* If we receive the completion AST before the actual enqueue returned,
+	 * then we might need to switch lock modes, resources, or extents.
+	 */
+	if (dlm_req->lock_desc.l_granted_mode != lock->l_req_mode) {
+		lock->l_req_mode = dlm_req->lock_desc.l_granted_mode;
+		LDLM_DEBUG(lock, "completion AST, new lock mode");
+	}
+
+	if (lock->l_resource->lr_type != LDLM_PLAIN) {
+		ldlm_convert_policy_to_local(req->rq_export,
+					  dlm_req->lock_desc.l_resource.lr_type,
+					  &dlm_req->lock_desc.l_policy_data,
+					  &lock->l_policy_data);
+		LDLM_DEBUG(lock, "completion AST, new policy data");
+	}
+
+	ldlm_resource_unlink_lock(lock);
+	if (memcmp(&dlm_req->lock_desc.l_resource.lr_name,
+		   &lock->l_resource->lr_name,
+		   sizeof(lock->l_resource->lr_name)) != 0) {
+		unlock_res_and_lock(lock);
+		rc = ldlm_lock_change_resource(ns, lock,
+				&dlm_req->lock_desc.l_resource.lr_name);
+		if (rc < 0) {
+			LDLM_ERROR(lock, "Failed to allocate resource");
+			goto out;
+		}
+		LDLM_DEBUG(lock, "completion AST, new resource");
+		CERROR("change resource!\n");
+		lock_res_and_lock(lock);
+	}
+
+	if (dlm_req->lock_flags & LDLM_FL_AST_SENT) {
+		/* BL_AST locks are not needed in LRU.
+		 * Let ldlm_cancel_lru() be fast.
+		 */
+		ldlm_lock_remove_from_lru(lock);
+		lock->l_flags |= LDLM_FL_CBPENDING | LDLM_FL_BL_AST;
+		LDLM_DEBUG(lock, "completion AST includes blocking AST");
+	}
+
+	if (lock->l_lvb_len > 0) {
+		rc = ldlm_fill_lvb(lock, &req->rq_pill, RCL_CLIENT,
+				   lock->l_lvb_data, lvb_len);
+		if (rc < 0) {
+			unlock_res_and_lock(lock);
+			goto out;
+		}
+	}
+
+	ldlm_grant_lock(lock, &ast_list);
+	unlock_res_and_lock(lock);
+
+	LDLM_DEBUG(lock, "callback handler finished, about to run_ast_work");
+
+	/* Let Enqueue to call osc_lock_upcall() and initialize l_ast_data */
+	OBD_FAIL_TIMEOUT(OBD_FAIL_OSC_CP_ENQ_RACE, 2);
+
+	ldlm_run_ast_work(ns, &ast_list, LDLM_WORK_CP_AST);
+
+	LDLM_DEBUG_NOLOCK("client completion callback handler END (lock %p)",
+			  lock);
+	goto out;
+
+out:
+	if (rc < 0) {
+		lock_res_and_lock(lock);
+		ldlm_set_failed(lock);
+		unlock_res_and_lock(lock);
+		wake_up(&lock->l_waitq);
+	}
+	LDLM_LOCK_RELEASE(lock);
+}
+
+/**
+ * Callback handler for receiving incoming glimpse ASTs.
+ *
+ * This only can happen on client side.  After handling the glimpse AST
+ * we also consider dropping the lock here if it is unused locally for a
+ * long time.
+ */
+static void ldlm_handle_gl_callback(struct ptlrpc_request *req,
+				    struct ldlm_namespace *ns,
+				    struct ldlm_request *dlm_req,
+				    struct ldlm_lock *lock)
+{
+	int rc = -ENOSYS;
+
+	LDLM_DEBUG(lock, "client glimpse AST callback handler");
+
+	if (lock->l_glimpse_ast)
+		rc = lock->l_glimpse_ast(lock, req);
+
+	if (req->rq_repmsg) {
+		ptlrpc_reply(req);
+	} else {
+		req->rq_status = rc;
+		ptlrpc_error(req);
+	}
+
+	lock_res_and_lock(lock);
+	if (lock->l_granted_mode == LCK_PW &&
+	    !lock->l_readers && !lock->l_writers &&
+	    time_after(jiffies,
+		       lock->l_last_used + 10 * HZ)) {
+		unlock_res_and_lock(lock);
+		if (ldlm_bl_to_thread_lock(ns, NULL, lock))
+			ldlm_handle_bl_callback(ns, NULL, lock);
+
+		return;
+	}
+	unlock_res_and_lock(lock);
+	LDLM_LOCK_RELEASE(lock);
+}
+
+static int ldlm_callback_reply(struct ptlrpc_request *req, int rc)
+{
+	if (req->rq_no_reply)
+		return 0;
+
+	req->rq_status = rc;
+	if (!req->rq_packed_final) {
+		rc = lustre_pack_reply(req, 1, NULL, NULL);
+		if (rc)
+			return rc;
+	}
+	return ptlrpc_reply(req);
+}
+
+static int __ldlm_bl_to_thread(struct ldlm_bl_work_item *blwi,
+			       enum ldlm_cancel_flags cancel_flags)
+{
+	struct ldlm_bl_pool *blp = ldlm_state->ldlm_bl_pool;
+
+	spin_lock(&blp->blp_lock);
+	if (blwi->blwi_lock && ldlm_is_discard_data(blwi->blwi_lock)) {
+		/* add LDLM_FL_DISCARD_DATA requests to the priority list */
+		list_add_tail(&blwi->blwi_entry, &blp->blp_prio_list);
+	} else {
+		/* other blocking callbacks are added to the regular list */
+		list_add_tail(&blwi->blwi_entry, &blp->blp_list);
+	}
+	spin_unlock(&blp->blp_lock);
+
+	wake_up(&blp->blp_waitq);
+
+	/* can not check blwi->blwi_flags as blwi could be already freed in
+	 * LCF_ASYNC mode
+	 */
+	if (!(cancel_flags & LCF_ASYNC))
+		wait_for_completion(&blwi->blwi_comp);
+
+	return 0;
+}
+
+static inline void init_blwi(struct ldlm_bl_work_item *blwi,
+			     struct ldlm_namespace *ns,
+			     struct ldlm_lock_desc *ld,
+			     struct list_head *cancels, int count,
+			     struct ldlm_lock *lock,
+			     enum ldlm_cancel_flags cancel_flags)
+{
+	init_completion(&blwi->blwi_comp);
+	INIT_LIST_HEAD(&blwi->blwi_head);
+
+	if (current->flags & PF_MEMALLOC)
+		blwi->blwi_mem_pressure = 1;
+
+	blwi->blwi_ns = ns;
+	blwi->blwi_flags = cancel_flags;
+	if (ld)
+		blwi->blwi_ld = *ld;
+	if (count) {
+		list_add(&blwi->blwi_head, cancels);
+		list_del_init(cancels);
+		blwi->blwi_count = count;
+	} else {
+		blwi->blwi_lock = lock;
+	}
+}
+
+/**
+ * Queues a list of locks \a cancels containing \a count locks
+ * for later processing by a blocking thread.  If \a count is zero,
+ * then the lock referenced as \a lock is queued instead.
+ *
+ * The blocking thread would then call ->l_blocking_ast callback in the lock.
+ * If list addition fails an error is returned and caller is supposed to
+ * call ->l_blocking_ast itself.
+ */
+static int ldlm_bl_to_thread(struct ldlm_namespace *ns,
+			     struct ldlm_lock_desc *ld,
+			     struct ldlm_lock *lock,
+			     struct list_head *cancels, int count,
+			     enum ldlm_cancel_flags cancel_flags)
+{
+	if (cancels && count == 0)
+		return 0;
+
+	if (cancel_flags & LCF_ASYNC) {
+		struct ldlm_bl_work_item *blwi;
+
+		blwi = kzalloc(sizeof(*blwi), GFP_NOFS);
+		if (!blwi)
+			return -ENOMEM;
+		init_blwi(blwi, ns, ld, cancels, count, lock, cancel_flags);
+
+		return __ldlm_bl_to_thread(blwi, cancel_flags);
+	} else {
+		/* if it is synchronous call do minimum mem alloc, as it could
+		 * be triggered from kernel shrinker
+		 */
+		struct ldlm_bl_work_item blwi;
+
+		memset(&blwi, 0, sizeof(blwi));
+		init_blwi(&blwi, ns, ld, cancels, count, lock, cancel_flags);
+		return __ldlm_bl_to_thread(&blwi, cancel_flags);
+	}
+}
+
+int ldlm_bl_to_thread_lock(struct ldlm_namespace *ns, struct ldlm_lock_desc *ld,
+			   struct ldlm_lock *lock)
+{
+	return ldlm_bl_to_thread(ns, ld, lock, NULL, 0, LCF_ASYNC);
+}
+
+int ldlm_bl_to_thread_list(struct ldlm_namespace *ns, struct ldlm_lock_desc *ld,
+			   struct list_head *cancels, int count,
+			   enum ldlm_cancel_flags cancel_flags)
+{
+	return ldlm_bl_to_thread(ns, ld, NULL, cancels, count, cancel_flags);
+}
+
+int ldlm_bl_thread_wakeup(void)
+{
+	wake_up(&ldlm_state->ldlm_bl_pool->blp_waitq);
+	return 0;
+}
+
+/* Setinfo coming from Server (eg MDT) to Client (eg MDC)! */
+static int ldlm_handle_setinfo(struct ptlrpc_request *req)
+{
+	struct obd_device *obd = req->rq_export->exp_obd;
+	char *key;
+	void *val;
+	int keylen, vallen;
+	int rc = -ENOSYS;
+
+	DEBUG_REQ(D_HSM, req, "%s: handle setinfo\n", obd->obd_name);
+
+	req_capsule_set(&req->rq_pill, &RQF_OBD_SET_INFO);
+
+	key = req_capsule_client_get(&req->rq_pill, &RMF_SETINFO_KEY);
+	if (!key) {
+		DEBUG_REQ(D_IOCTL, req, "no set_info key");
+		return -EFAULT;
+	}
+	keylen = req_capsule_get_size(&req->rq_pill, &RMF_SETINFO_KEY,
+				      RCL_CLIENT);
+	val = req_capsule_client_get(&req->rq_pill, &RMF_SETINFO_VAL);
+	if (!val) {
+		DEBUG_REQ(D_IOCTL, req, "no set_info val");
+		return -EFAULT;
+	}
+	vallen = req_capsule_get_size(&req->rq_pill, &RMF_SETINFO_VAL,
+				      RCL_CLIENT);
+
+	/* We are responsible for swabbing contents of val */
+
+	if (KEY_IS(KEY_HSM_COPYTOOL_SEND))
+		/* Pass it on to mdc (the "export" in this case) */
+		rc = obd_set_info_async(req->rq_svc_thread->t_env,
+					req->rq_export,
+					sizeof(KEY_HSM_COPYTOOL_SEND),
+					KEY_HSM_COPYTOOL_SEND,
+					vallen, val, NULL);
+	else
+		DEBUG_REQ(D_WARNING, req, "ignoring unknown key %s", key);
+
+	return rc;
+}
+
+static inline void ldlm_callback_errmsg(struct ptlrpc_request *req,
+					const char *msg, int rc,
+					const struct lustre_handle *handle)
+{
+	DEBUG_REQ((req->rq_no_reply || rc) ? D_WARNING : D_DLMTRACE, req,
+		  "%s: [nid %s] [rc %d] [lock %#llx]",
+		  msg, libcfs_id2str(req->rq_peer), rc,
+		  handle ? handle->cookie : 0);
+	if (req->rq_no_reply)
+		CWARN("No reply was sent, maybe cause bug 21636.\n");
+	else if (rc)
+		CWARN("Send reply failed, maybe cause bug 21636.\n");
+}
+
+/* TODO: handle requests in a similar way as MDT: see mdt_handle_common() */
+static int ldlm_callback_handler(struct ptlrpc_request *req)
+{
+	struct ldlm_namespace *ns;
+	struct ldlm_request *dlm_req;
+	struct ldlm_lock *lock;
+	int rc;
+
+	/* Requests arrive in sender's byte order.  The ptlrpc service
+	 * handler has already checked and, if necessary, byte-swapped the
+	 * incoming request message body, but I am responsible for the
+	 * message buffers.
+	 */
+
+	/* do nothing for sec context finalize */
+	if (lustre_msg_get_opc(req->rq_reqmsg) == SEC_CTX_FINI)
+		return 0;
+
+	req_capsule_init(&req->rq_pill, req, RCL_SERVER);
+
+	if (!req->rq_export) {
+		rc = ldlm_callback_reply(req, -ENOTCONN);
+		ldlm_callback_errmsg(req, "Operate on unconnected server",
+				     rc, NULL);
+		return 0;
+	}
+
+	LASSERT(req->rq_export->exp_obd);
+
+	switch (lustre_msg_get_opc(req->rq_reqmsg)) {
+	case LDLM_BL_CALLBACK:
+		if (OBD_FAIL_CHECK(OBD_FAIL_LDLM_BL_CALLBACK_NET)) {
+			if (cfs_fail_err)
+				ldlm_callback_reply(req, -(int)cfs_fail_err);
+			return 0;
+		}
+		break;
+	case LDLM_CP_CALLBACK:
+		if (OBD_FAIL_CHECK(OBD_FAIL_LDLM_CP_CALLBACK_NET))
+			return 0;
+		break;
+	case LDLM_GL_CALLBACK:
+		if (OBD_FAIL_CHECK(OBD_FAIL_LDLM_GL_CALLBACK_NET))
+			return 0;
+		break;
+	case LDLM_SET_INFO:
+		rc = ldlm_handle_setinfo(req);
+		ldlm_callback_reply(req, rc);
+		return 0;
+	default:
+		CERROR("unknown opcode %u\n",
+		       lustre_msg_get_opc(req->rq_reqmsg));
+		ldlm_callback_reply(req, -EPROTO);
+		return 0;
+	}
+
+	ns = req->rq_export->exp_obd->obd_namespace;
+	LASSERT(ns);
+
+	req_capsule_set(&req->rq_pill, &RQF_LDLM_CALLBACK);
+
+	dlm_req = req_capsule_client_get(&req->rq_pill, &RMF_DLM_REQ);
+	if (!dlm_req) {
+		rc = ldlm_callback_reply(req, -EPROTO);
+		ldlm_callback_errmsg(req, "Operate without parameter", rc,
+				     NULL);
+		return 0;
+	}
+
+	/* Force a known safe race, send a cancel to the server for a lock
+	 * which the server has already started a blocking callback on.
+	 */
+	if (OBD_FAIL_CHECK(OBD_FAIL_LDLM_CANCEL_BL_CB_RACE) &&
+	    lustre_msg_get_opc(req->rq_reqmsg) == LDLM_BL_CALLBACK) {
+		rc = ldlm_cli_cancel(&dlm_req->lock_handle[0], 0);
+		if (rc < 0)
+			CERROR("ldlm_cli_cancel: %d\n", rc);
+	}
+
+	lock = ldlm_handle2lock_long(&dlm_req->lock_handle[0], 0);
+	if (!lock) {
+		CDEBUG(D_DLMTRACE,
+		       "callback on lock %#llx - lock disappeared\n",
+		       dlm_req->lock_handle[0].cookie);
+		rc = ldlm_callback_reply(req, -EINVAL);
+		ldlm_callback_errmsg(req, "Operate with invalid parameter", rc,
+				     &dlm_req->lock_handle[0]);
+		return 0;
+	}
+
+	if (ldlm_is_fail_loc(lock) &&
+	    lustre_msg_get_opc(req->rq_reqmsg) == LDLM_BL_CALLBACK)
+		OBD_RACE(OBD_FAIL_LDLM_CP_BL_RACE);
+
+	/* Copy hints/flags (e.g. LDLM_FL_DISCARD_DATA) from AST. */
+	lock_res_and_lock(lock);
+	lock->l_flags |= ldlm_flags_from_wire(dlm_req->lock_flags &
+					      LDLM_FL_AST_MASK);
+	if (lustre_msg_get_opc(req->rq_reqmsg) == LDLM_BL_CALLBACK) {
+		/* If somebody cancels lock and cache is already dropped,
+		 * or lock is failed before cp_ast received on client,
+		 * we can tell the server we have no lock. Otherwise, we
+		 * should send cancel after dropping the cache.
+		 */
+		if ((ldlm_is_canceling(lock) && ldlm_is_bl_done(lock)) ||
+		    ldlm_is_failed(lock)) {
+			LDLM_DEBUG(lock,
+				   "callback on lock %#llx - lock disappeared",
+				   dlm_req->lock_handle[0].cookie);
+			unlock_res_and_lock(lock);
+			LDLM_LOCK_RELEASE(lock);
+			rc = ldlm_callback_reply(req, -EINVAL);
+			ldlm_callback_errmsg(req, "Operate on stale lock", rc,
+					     &dlm_req->lock_handle[0]);
+			return 0;
+		}
+		/* BL_AST locks are not needed in LRU.
+		 * Let ldlm_cancel_lru() be fast.
+		 */
+		ldlm_lock_remove_from_lru(lock);
+		ldlm_set_bl_ast(lock);
+	}
+	unlock_res_and_lock(lock);
+
+	/* We want the ost thread to get this reply so that it can respond
+	 * to ost requests (write cache writeback) that might be triggered
+	 * in the callback.
+	 *
+	 * But we'd also like to be able to indicate in the reply that we're
+	 * cancelling right now, because it's unused, or have an intent result
+	 * in the reply, so we might have to push the responsibility for sending
+	 * the reply down into the AST handlers, alas.
+	 */
+
+	switch (lustre_msg_get_opc(req->rq_reqmsg)) {
+	case LDLM_BL_CALLBACK:
+		CDEBUG(D_INODE, "blocking ast\n");
+		req_capsule_extend(&req->rq_pill, &RQF_LDLM_BL_CALLBACK);
+		if (!ldlm_is_cancel_on_block(lock)) {
+			rc = ldlm_callback_reply(req, 0);
+			if (req->rq_no_reply || rc)
+				ldlm_callback_errmsg(req, "Normal process", rc,
+						     &dlm_req->lock_handle[0]);
+		}
+		if (ldlm_bl_to_thread_lock(ns, &dlm_req->lock_desc, lock))
+			ldlm_handle_bl_callback(ns, &dlm_req->lock_desc, lock);
+		break;
+	case LDLM_CP_CALLBACK:
+		CDEBUG(D_INODE, "completion ast\n");
+		req_capsule_extend(&req->rq_pill, &RQF_LDLM_CP_CALLBACK);
+		ldlm_callback_reply(req, 0);
+		ldlm_handle_cp_callback(req, ns, dlm_req, lock);
+		break;
+	case LDLM_GL_CALLBACK:
+		CDEBUG(D_INODE, "glimpse ast\n");
+		req_capsule_extend(&req->rq_pill, &RQF_LDLM_GL_CALLBACK);
+		ldlm_handle_gl_callback(req, ns, dlm_req, lock);
+		break;
+	default:
+		LBUG();			 /* checked above */
+	}
+
+	return 0;
+}
+
+static int ldlm_bl_get_work(struct ldlm_bl_pool *blp,
+			    struct ldlm_bl_work_item **p_blwi,
+			    struct obd_export **p_exp)
+{
+	int num_th = atomic_read(&blp->blp_num_threads);
+	struct ldlm_bl_work_item *blwi = NULL;
+	static unsigned int num_bl;
+
+	spin_lock(&blp->blp_lock);
+	/* process a request from the blp_list at least every blp_num_threads */
+	if (!list_empty(&blp->blp_list) &&
+	    (list_empty(&blp->blp_prio_list) || num_bl == 0))
+		blwi = list_first_entry(&blp->blp_list,
+					struct ldlm_bl_work_item, blwi_entry);
+	else
+		if (!list_empty(&blp->blp_prio_list))
+			blwi = list_first_entry(&blp->blp_prio_list,
+						struct ldlm_bl_work_item,
+						blwi_entry);
+
+	if (blwi) {
+		if (++num_bl >= num_th)
+			num_bl = 0;
+		list_del(&blwi->blwi_entry);
+	}
+	spin_unlock(&blp->blp_lock);
+	*p_blwi = blwi;
+
+	return (*p_blwi || *p_exp) ? 1 : 0;
+}
+
+/* This only contains temporary data until the thread starts */
+struct ldlm_bl_thread_data {
+	struct ldlm_bl_pool	*bltd_blp;
+	struct completion	bltd_comp;
+	int			bltd_num;
+};
+
+static int ldlm_bl_thread_main(void *arg);
+
+static int ldlm_bl_thread_start(struct ldlm_bl_pool *blp, bool check_busy)
+{
+	struct ldlm_bl_thread_data bltd = { .bltd_blp = blp };
+	struct task_struct *task;
+
+	init_completion(&bltd.bltd_comp);
+
+	bltd.bltd_num = atomic_inc_return(&blp->blp_num_threads);
+	if (bltd.bltd_num >= blp->blp_max_threads) {
+		atomic_dec(&blp->blp_num_threads);
+		return 0;
+	}
+
+	LASSERTF(bltd.bltd_num > 0, "thread num:%d\n", bltd.bltd_num);
+	if (check_busy &&
+	    atomic_read(&blp->blp_busy_threads) < (bltd.bltd_num - 1)) {
+		atomic_dec(&blp->blp_num_threads);
+		return 0;
+	}
+
+	task = kthread_run(ldlm_bl_thread_main, &bltd, "ldlm_bl_%02d",
+			   bltd.bltd_num);
+	if (IS_ERR(task)) {
+		CERROR("cannot start LDLM thread ldlm_bl_%02d: rc %ld\n",
+		       bltd.bltd_num, PTR_ERR(task));
+		atomic_dec(&blp->blp_num_threads);
+		return PTR_ERR(task);
+	}
+	wait_for_completion(&bltd.bltd_comp);
+
+	return 0;
+}
+
+/* Not fatal if racy and have a few too many threads */
+static int ldlm_bl_thread_need_create(struct ldlm_bl_pool *blp,
+				      struct ldlm_bl_work_item *blwi)
+{
+	if (atomic_read(&blp->blp_num_threads) >= blp->blp_max_threads)
+		return 0;
+
+	if (atomic_read(&blp->blp_busy_threads) <
+	    atomic_read(&blp->blp_num_threads))
+		return 0;
+
+	if (blwi && (!blwi->blwi_ns || blwi->blwi_mem_pressure))
+		return 0;
+
+	return 1;
+}
+
+static int ldlm_bl_thread_blwi(struct ldlm_bl_pool *blp,
+			       struct ldlm_bl_work_item *blwi)
+{
+	unsigned int flags = 0;
+
+	if (!blwi->blwi_ns)
+		/* added by ldlm_cleanup() */
+		return LDLM_ITER_STOP;
+
+	if (blwi->blwi_mem_pressure)
+		flags = memalloc_noreclaim_save();
+
+	OBD_FAIL_TIMEOUT(OBD_FAIL_LDLM_PAUSE_CANCEL2, 4);
+
+	if (blwi->blwi_count) {
+		int count;
+
+		/*
+		 * The special case when we cancel locks in lru
+		 * asynchronously, we pass the list of locks here.
+		 * Thus locks are marked LDLM_FL_CANCELING, but NOT
+		 * canceled locally yet.
+		 */
+		count = ldlm_cli_cancel_list_local(&blwi->blwi_head,
+						   blwi->blwi_count,
+						   LCF_BL_AST);
+		ldlm_cli_cancel_list(&blwi->blwi_head, count, NULL,
+				     blwi->blwi_flags);
+	} else {
+		ldlm_handle_bl_callback(blwi->blwi_ns, &blwi->blwi_ld,
+					blwi->blwi_lock);
+	}
+	if (blwi->blwi_mem_pressure)
+		memalloc_noreclaim_restore(flags);
+
+	if (blwi->blwi_flags & LCF_ASYNC)
+		kfree(blwi);
+	else
+		complete(&blwi->blwi_comp);
+
+	return 0;
+}
+
+/**
+ * Main blocking requests processing thread.
+ *
+ * Callers put locks into its queue by calling ldlm_bl_to_thread.
+ * This thread in the end ends up doing actual call to ->l_blocking_ast
+ * for queued locks.
+ */
+static int ldlm_bl_thread_main(void *arg)
+{
+	struct ldlm_bl_pool *blp;
+	struct ldlm_bl_thread_data *bltd = arg;
+
+	blp = bltd->bltd_blp;
+
+	complete(&bltd->bltd_comp);
+	/* cannot use bltd after this, it is only on caller's stack */
+
+	while (1) {
+		struct ldlm_bl_work_item *blwi = NULL;
+		struct obd_export *exp = NULL;
+		int rc;
+
+		rc = ldlm_bl_get_work(blp, &blwi, &exp);
+		if (!rc)
+			wait_event_idle_exclusive(blp->blp_waitq,
+						  ldlm_bl_get_work(blp, &blwi,
+								   &exp));
+		atomic_inc(&blp->blp_busy_threads);
+
+		if (ldlm_bl_thread_need_create(blp, blwi))
+			/* discard the return value, we tried */
+			ldlm_bl_thread_start(blp, true);
+
+		if (blwi)
+			rc = ldlm_bl_thread_blwi(blp, blwi);
+
+		atomic_dec(&blp->blp_busy_threads);
+
+		if (rc == LDLM_ITER_STOP)
+			break;
+	}
+
+	atomic_dec(&blp->blp_num_threads);
+	complete(&blp->blp_comp);
+	return 0;
+}
+
+static int ldlm_setup(void);
+static int ldlm_cleanup(void);
+
+int ldlm_get_ref(void)
+{
+	int rc = 0;
+
+	rc = ptlrpc_inc_ref();
+	if (rc)
+		return rc;
+
+	mutex_lock(&ldlm_ref_mutex);
+	if (++ldlm_refcount == 1) {
+		rc = ldlm_setup();
+		if (rc)
+			ldlm_refcount--;
+	}
+	mutex_unlock(&ldlm_ref_mutex);
+
+	if (rc)
+		ptlrpc_dec_ref();
+
+	return rc;
+}
+
+void ldlm_put_ref(void)
+{
+	int rc = 0;
+	mutex_lock(&ldlm_ref_mutex);
+	if (ldlm_refcount == 1) {
+		rc = ldlm_cleanup();
+
+		if (rc)
+			CERROR("ldlm_cleanup failed: %d\n", rc);
+		else
+			ldlm_refcount--;
+	} else {
+		ldlm_refcount--;
+	}
+	mutex_unlock(&ldlm_ref_mutex);
+	if (!rc)
+		ptlrpc_dec_ref();
+}
+
+static ssize_t cancel_unused_locks_before_replay_show(struct kobject *kobj,
+						      struct attribute *attr,
+						      char *buf)
+{
+	return sprintf(buf, "%d\n", ldlm_cancel_unused_locks_before_replay);
+}
+
+static ssize_t cancel_unused_locks_before_replay_store(struct kobject *kobj,
+						       struct attribute *attr,
+						       const char *buffer,
+						       size_t count)
+{
+	int rc;
+	unsigned long val;
+
+	rc = kstrtoul(buffer, 10, &val);
+	if (rc)
+		return rc;
+
+	ldlm_cancel_unused_locks_before_replay = val;
+
+	return count;
+}
+LUSTRE_RW_ATTR(cancel_unused_locks_before_replay);
+
+/* These are for root of /sys/fs/lustre/ldlm */
+static struct attribute *ldlm_attrs[] = {
+	&lustre_attr_cancel_unused_locks_before_replay.attr,
+	NULL,
+};
+
+static const struct attribute_group ldlm_attr_group = {
+	.attrs = ldlm_attrs,
+};
+
+static int ldlm_setup(void)
+{
+	static struct ptlrpc_service_conf	conf;
+	struct ldlm_bl_pool			*blp = NULL;
+	int rc = 0;
+	int i;
+
+	if (ldlm_state)
+		return -EALREADY;
+
+	ldlm_state = kzalloc(sizeof(*ldlm_state), GFP_NOFS);
+	if (!ldlm_state)
+		return -ENOMEM;
+
+	ldlm_kobj = kobject_create_and_add("ldlm", lustre_kobj);
+	if (!ldlm_kobj) {
+		rc = -ENOMEM;
+		goto out;
+	}
+
+	rc = sysfs_create_group(ldlm_kobj, &ldlm_attr_group);
+	if (rc)
+		goto out;
+
+	ldlm_ns_kset = kset_create_and_add("namespaces", NULL, ldlm_kobj);
+	if (!ldlm_ns_kset) {
+		rc = -ENOMEM;
+		goto out;
+	}
+
+	ldlm_svc_kset = kset_create_and_add("services", NULL, ldlm_kobj);
+	if (!ldlm_svc_kset) {
+		rc = -ENOMEM;
+		goto out;
+	}
+
+	ldlm_debugfs_setup();
+
+	memset(&conf, 0, sizeof(conf));
+	conf = (typeof(conf)) {
+		.psc_name		= "ldlm_cbd",
+		.psc_watchdog_factor	= 2,
+		.psc_buf		= {
+			.bc_nbufs		= LDLM_CLIENT_NBUFS,
+			.bc_buf_size		= LDLM_BUFSIZE,
+			.bc_req_max_size	= LDLM_MAXREQSIZE,
+			.bc_rep_max_size	= LDLM_MAXREPSIZE,
+			.bc_req_portal		= LDLM_CB_REQUEST_PORTAL,
+			.bc_rep_portal		= LDLM_CB_REPLY_PORTAL,
+		},
+		.psc_thr		= {
+			.tc_thr_name		= "ldlm_cb",
+			.tc_thr_factor		= LDLM_THR_FACTOR,
+			.tc_nthrs_init		= LDLM_NTHRS_INIT,
+			.tc_nthrs_base		= LDLM_NTHRS_BASE,
+			.tc_nthrs_max		= LDLM_NTHRS_MAX,
+			.tc_nthrs_user		= ldlm_num_threads,
+			.tc_cpu_affinity	= 1,
+			.tc_ctx_tags		= LCT_MD_THREAD | LCT_DT_THREAD,
+		},
+		.psc_cpt		= {
+			.cc_pattern		= ldlm_cpts,
+		},
+		.psc_ops		= {
+			.so_req_handler		= ldlm_callback_handler,
+		},
+	};
+	ldlm_state->ldlm_cb_service =
+			ptlrpc_register_service(&conf, ldlm_svc_kset,
+						ldlm_svc_debugfs_dir);
+	if (IS_ERR(ldlm_state->ldlm_cb_service)) {
+		CERROR("failed to start service\n");
+		rc = PTR_ERR(ldlm_state->ldlm_cb_service);
+		ldlm_state->ldlm_cb_service = NULL;
+		goto out;
+	}
+
+	blp = kzalloc(sizeof(*blp), GFP_NOFS);
+	if (!blp) {
+		rc = -ENOMEM;
+		goto out;
+	}
+	ldlm_state->ldlm_bl_pool = blp;
+
+	spin_lock_init(&blp->blp_lock);
+	INIT_LIST_HEAD(&blp->blp_list);
+	INIT_LIST_HEAD(&blp->blp_prio_list);
+	init_waitqueue_head(&blp->blp_waitq);
+	atomic_set(&blp->blp_num_threads, 0);
+	atomic_set(&blp->blp_busy_threads, 0);
+
+	if (ldlm_num_threads == 0) {
+		blp->blp_min_threads = LDLM_NTHRS_INIT;
+		blp->blp_max_threads = LDLM_NTHRS_MAX;
+	} else {
+		blp->blp_min_threads = min_t(int, LDLM_NTHRS_MAX,
+					     max_t(int, LDLM_NTHRS_INIT,
+						   ldlm_num_threads));
+
+		blp->blp_max_threads = blp->blp_min_threads;
+	}
+
+	for (i = 0; i < blp->blp_min_threads; i++) {
+		rc = ldlm_bl_thread_start(blp, false);
+		if (rc < 0)
+			goto out;
+	}
+
+	rc = ldlm_pools_init();
+	if (rc) {
+		CERROR("Failed to initialize LDLM pools: %d\n", rc);
+		goto out;
+	}
+	return 0;
+
+ out:
+	ldlm_cleanup();
+	return rc;
+}
+
+static int ldlm_cleanup(void)
+{
+	if (!list_empty(ldlm_namespace_list(LDLM_NAMESPACE_SERVER)) ||
+	    !list_empty(ldlm_namespace_list(LDLM_NAMESPACE_CLIENT))) {
+		CERROR("ldlm still has namespaces; clean these up first.\n");
+		ldlm_dump_all_namespaces(LDLM_NAMESPACE_SERVER, D_DLMTRACE);
+		ldlm_dump_all_namespaces(LDLM_NAMESPACE_CLIENT, D_DLMTRACE);
+		return -EBUSY;
+	}
+
+	ldlm_pools_fini();
+
+	if (ldlm_state->ldlm_bl_pool) {
+		struct ldlm_bl_pool *blp = ldlm_state->ldlm_bl_pool;
+
+		while (atomic_read(&blp->blp_num_threads) > 0) {
+			struct ldlm_bl_work_item blwi = { .blwi_ns = NULL };
+
+			init_completion(&blp->blp_comp);
+
+			spin_lock(&blp->blp_lock);
+			list_add_tail(&blwi.blwi_entry, &blp->blp_list);
+			wake_up(&blp->blp_waitq);
+			spin_unlock(&blp->blp_lock);
+
+			wait_for_completion(&blp->blp_comp);
+		}
+
+		kfree(blp);
+	}
+
+	if (ldlm_state->ldlm_cb_service)
+		ptlrpc_unregister_service(ldlm_state->ldlm_cb_service);
+
+	if (ldlm_ns_kset)
+		kset_unregister(ldlm_ns_kset);
+	if (ldlm_svc_kset)
+		kset_unregister(ldlm_svc_kset);
+	if (ldlm_kobj) {
+		sysfs_remove_group(ldlm_kobj, &ldlm_attr_group);
+		kobject_put(ldlm_kobj);
+	}
+
+	ldlm_debugfs_cleanup();
+
+	kfree(ldlm_state);
+	ldlm_state = NULL;
+
+	return 0;
+}
+
+int ldlm_init(void)
+{
+	mutex_init(&ldlm_ref_mutex);
+	mutex_init(ldlm_namespace_lock(LDLM_NAMESPACE_SERVER));
+	mutex_init(ldlm_namespace_lock(LDLM_NAMESPACE_CLIENT));
+	ldlm_resource_slab = kmem_cache_create("ldlm_resources",
+					       sizeof(struct ldlm_resource), 0,
+					       SLAB_HWCACHE_ALIGN, NULL);
+	if (!ldlm_resource_slab)
+		return -ENOMEM;
+
+	ldlm_lock_slab = kmem_cache_create("ldlm_locks",
+					   sizeof(struct ldlm_lock), 0,
+					   SLAB_HWCACHE_ALIGN |
+					   SLAB_TYPESAFE_BY_RCU, NULL);
+	if (!ldlm_lock_slab) {
+		kmem_cache_destroy(ldlm_resource_slab);
+		return -ENOMEM;
+	}
+
+#if LUSTRE_TRACKS_LOCK_EXP_REFS
+	class_export_dump_hook = ldlm_dump_export_locks;
+#endif
+	return 0;
+}
+
+void ldlm_exit(void)
+{
+	if (ldlm_refcount)
+		CERROR("ldlm_refcount is %d in %s!\n", ldlm_refcount, __func__);
+	kmem_cache_destroy(ldlm_resource_slab);
+	/* ldlm_lock_put() use RCU to call ldlm_lock_free, so need call
+	 * synchronize_rcu() to wait a grace period elapsed, so that
+	 * ldlm_lock_free() get a chance to be called.
+	 */
+	synchronize_rcu();
+	kmem_cache_destroy(ldlm_lock_slab);
+}
diff --git a/drivers/staging/lustre/lustre/ptlrpc/ldlm_plain.c b/drivers/staging/lustre/lustre/ptlrpc/ldlm_plain.c
new file mode 100644
index 000000000000..33b5a3f96fcb
--- /dev/null
+++ b/drivers/staging/lustre/lustre/ptlrpc/ldlm_plain.c
@@ -0,0 +1,68 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * GPL HEADER START
+ *
+ * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 only,
+ * as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License version 2 for more details (a copy is included
+ * in the LICENSE file that accompanied this code).
+ *
+ * You should have received a copy of the GNU General Public License
+ * version 2 along with this program; If not, see
+ * http://www.gnu.org/licenses/gpl-2.0.html
+ *
+ * GPL HEADER END
+ */
+/*
+ * Copyright (c) 2003, 2010, Oracle and/or its affiliates. All rights reserved.
+ * Use is subject to license terms.
+ *
+ * Copyright (c) 2011, Intel Corporation.
+ */
+/*
+ * This file is part of Lustre, http://www.lustre.org/
+ * Lustre is a trademark of Sun Microsystems, Inc.
+ *
+ * lustre/ldlm/ldlm_plain.c
+ *
+ * Author: Peter Braam <braam at clusterfs.com>
+ * Author: Phil Schwan <phil at clusterfs.com>
+ */
+
+/**
+ * This file contains implementation of PLAIN lock type.
+ *
+ * PLAIN locks are the simplest form of LDLM locking, and are used when
+ * there only needs to be a single lock on a resource. This avoids some
+ * of the complexity of EXTENT and IBITS lock types, but doesn't allow
+ * different "parts" of a resource to be locked concurrently.  Example
+ * use cases for PLAIN locks include locking of MGS configuration logs
+ * and (as of Lustre 2.4) quota records.
+ */
+
+#define DEBUG_SUBSYSTEM S_LDLM
+
+#include <lustre_dlm.h>
+#include <obd_support.h>
+#include <lustre_lib.h>
+
+#include "ldlm_internal.h"
+
+void ldlm_plain_policy_wire_to_local(const union ldlm_wire_policy_data *wpolicy,
+				     union ldlm_policy_data *lpolicy)
+{
+	/* No policy for plain locks */
+}
+
+void ldlm_plain_policy_local_to_wire(const union ldlm_policy_data *lpolicy,
+				     union ldlm_wire_policy_data *wpolicy)
+{
+	/* No policy for plain locks */
+}
diff --git a/drivers/staging/lustre/lustre/ptlrpc/ldlm_pool.c b/drivers/staging/lustre/lustre/ptlrpc/ldlm_pool.c
new file mode 100644
index 000000000000..36d14ee4e5b1
--- /dev/null
+++ b/drivers/staging/lustre/lustre/ptlrpc/ldlm_pool.c
@@ -0,0 +1,1013 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * GPL HEADER START
+ *
+ * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 only,
+ * as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License version 2 for more details (a copy is included
+ * in the LICENSE file that accompanied this code).
+ *
+ * You should have received a copy of the GNU General Public License
+ * version 2 along with this program; If not, see
+ * http://www.gnu.org/licenses/gpl-2.0.html
+ *
+ * GPL HEADER END
+ */
+/*
+ * Copyright (c) 2007, 2010, Oracle and/or its affiliates. All rights reserved.
+ * Use is subject to license terms.
+ *
+ * Copyright (c) 2010, 2015, Intel Corporation.
+ */
+/*
+ * This file is part of Lustre, http://www.lustre.org/
+ * Lustre is a trademark of Sun Microsystems, Inc.
+ *
+ * lustre/ldlm/ldlm_pool.c
+ *
+ * Author: Yury Umanets <umka at clusterfs.com>
+ */
+
+/*
+ * Idea of this code is rather simple. Each second, for each server namespace
+ * we have SLV - server lock volume which is calculated on current number of
+ * granted locks, grant speed for past period, etc - that is, locking load.
+ * This SLV number may be thought as a flow definition for simplicity. It is
+ * sent to clients with each occasion to let them know what is current load
+ * situation on the server. By default, at the beginning, SLV on server is
+ * set max value which is calculated as the following: allow to one client
+ * have all locks of limit ->pl_limit for 10h.
+ *
+ * Next, on clients, number of cached locks is not limited artificially in any
+ * way as it was before. Instead, client calculates CLV, that is, client lock
+ * volume for each lock and compares it with last SLV from the server. CLV is
+ * calculated as the number of locks in LRU * lock live time in seconds. If
+ * CLV > SLV - lock is canceled.
+ *
+ * Client has LVF, that is, lock volume factor which regulates how much
+ * sensitive client should be about last SLV from server. The higher LVF is the
+ * more locks will be canceled on client. Default value for it is 1. Setting LVF
+ * to 2 means that client will cancel locks 2 times faster.
+ *
+ * Locks on a client will be canceled more intensively in these cases:
+ * (1) if SLV is smaller, that is, load is higher on the server;
+ * (2) client has a lot of locks (the more locks are held by client, the bigger
+ *     chances that some of them should be canceled);
+ * (3) client has old locks (taken some time ago);
+ *
+ * Thus, according to flow paradigm that we use for better understanding SLV,
+ * CLV is the volume of particle in flow described by SLV. According to this,
+ * if flow is getting thinner, more and more particles become outside of it and
+ * as particles are locks, they should be canceled.
+ *
+ * General idea of this belongs to Vitaly Fertman (vitaly at clusterfs.com).
+ * Andreas Dilger (adilger at clusterfs.com) proposed few nice ideas like using
+ * LVF and many cleanups. Flow definition to allow more easy understanding of
+ * the logic belongs to Nikita Danilov (nikita at clusterfs.com) as well as many
+ * cleanups and fixes. And design and implementation are done by Yury Umanets
+ * (umka at clusterfs.com).
+ *
+ * Glossary for terms used:
+ *
+ * pl_limit - Number of allowed locks in pool. Applies to server and client
+ * side (tunable);
+ *
+ * pl_granted - Number of granted locks (calculated);
+ * pl_grant_rate - Number of granted locks for last T (calculated);
+ * pl_cancel_rate - Number of canceled locks for last T (calculated);
+ * pl_grant_speed - Grant speed (GR - CR) for last T (calculated);
+ * pl_grant_plan - Planned number of granted locks for next T (calculated);
+ * pl_server_lock_volume - Current server lock volume (calculated);
+ *
+ * As it may be seen from list above, we have few possible tunables which may
+ * affect behavior much. They all may be modified via sysfs. However, they also
+ * give a possibility for constructing few pre-defined behavior policies. If
+ * none of predefines is suitable for a working pattern being used, new one may
+ * be "constructed" via sysfs tunables.
+ */
+
+#define DEBUG_SUBSYSTEM S_LDLM
+
+#include <lustre_dlm.h>
+#include <cl_object.h>
+#include <obd_class.h>
+#include <obd_support.h>
+#include "ldlm_internal.h"
+
+/*
+ * 50 ldlm locks for 1MB of RAM.
+ */
+#define LDLM_POOL_HOST_L ((NUM_CACHEPAGES >> (20 - PAGE_SHIFT)) * 50)
+
+/*
+ * Maximal possible grant step plan in %.
+ */
+#define LDLM_POOL_MAX_GSP (30)
+
+/*
+ * Minimal possible grant step plan in %.
+ */
+#define LDLM_POOL_MIN_GSP (1)
+
+/*
+ * This controls the speed of reaching LDLM_POOL_MAX_GSP
+ * with increasing thread period.
+ */
+#define LDLM_POOL_GSP_STEP_SHIFT (2)
+
+/*
+ * LDLM_POOL_GSP% of all locks is default GP.
+ */
+#define LDLM_POOL_GP(L)   (((L) * LDLM_POOL_MAX_GSP) / 100)
+
+/*
+ * Max age for locks on clients.
+ */
+#define LDLM_POOL_MAX_AGE (36000)
+
+/*
+ * The granularity of SLV calculation.
+ */
+#define LDLM_POOL_SLV_SHIFT (10)
+
+static inline __u64 dru(__u64 val, __u32 shift, int round_up)
+{
+	return (val + (round_up ? (1 << shift) - 1 : 0)) >> shift;
+}
+
+static inline __u64 ldlm_pool_slv_max(__u32 L)
+{
+	/*
+	 * Allow to have all locks for 1 client for 10 hrs.
+	 * Formula is the following: limit * 10h / 1 client.
+	 */
+	__u64 lim = (__u64)L *  LDLM_POOL_MAX_AGE / 1;
+	return lim;
+}
+
+static inline __u64 ldlm_pool_slv_min(__u32 L)
+{
+	return 1;
+}
+
+enum {
+	LDLM_POOL_FIRST_STAT = 0,
+	LDLM_POOL_GRANTED_STAT = LDLM_POOL_FIRST_STAT,
+	LDLM_POOL_GRANT_STAT,
+	LDLM_POOL_CANCEL_STAT,
+	LDLM_POOL_GRANT_RATE_STAT,
+	LDLM_POOL_CANCEL_RATE_STAT,
+	LDLM_POOL_GRANT_PLAN_STAT,
+	LDLM_POOL_SLV_STAT,
+	LDLM_POOL_SHRINK_REQTD_STAT,
+	LDLM_POOL_SHRINK_FREED_STAT,
+	LDLM_POOL_RECALC_STAT,
+	LDLM_POOL_TIMING_STAT,
+	LDLM_POOL_LAST_STAT
+};
+
+/**
+ * Calculates suggested grant_step in % of available locks for passed
+ * \a period. This is later used in grant_plan calculations.
+ */
+static inline int ldlm_pool_t2gsp(unsigned int t)
+{
+	/*
+	 * This yields 1% grant step for anything below LDLM_POOL_GSP_STEP
+	 * and up to 30% for anything higher than LDLM_POOL_GSP_STEP.
+	 *
+	 * How this will affect execution is the following:
+	 *
+	 * - for thread period 1s we will have grant_step 1% which good from
+	 * pov of taking some load off from server and push it out to clients.
+	 * This is like that because 1% for grant_step means that server will
+	 * not allow clients to get lots of locks in short period of time and
+	 * keep all old locks in their caches. Clients will always have to
+	 * get some locks back if they want to take some new;
+	 *
+	 * - for thread period 10s (which is default) we will have 23% which
+	 * means that clients will have enough of room to take some new locks
+	 * without getting some back. All locks from this 23% which were not
+	 * taken by clients in current period will contribute in SLV growing.
+	 * SLV growing means more locks cached on clients until limit or grant
+	 * plan is reached.
+	 */
+	return LDLM_POOL_MAX_GSP -
+		((LDLM_POOL_MAX_GSP - LDLM_POOL_MIN_GSP) >>
+		 (t >> LDLM_POOL_GSP_STEP_SHIFT));
+}
+
+/**
+ * Recalculates next stats on passed \a pl.
+ *
+ * \pre ->pl_lock is locked.
+ */
+static void ldlm_pool_recalc_stats(struct ldlm_pool *pl)
+{
+	int grant_plan = pl->pl_grant_plan;
+	__u64 slv = pl->pl_server_lock_volume;
+	int granted = atomic_read(&pl->pl_granted);
+	int grant_rate = atomic_read(&pl->pl_grant_rate);
+	int cancel_rate = atomic_read(&pl->pl_cancel_rate);
+
+	lprocfs_counter_add(pl->pl_stats, LDLM_POOL_SLV_STAT,
+			    slv);
+	lprocfs_counter_add(pl->pl_stats, LDLM_POOL_GRANTED_STAT,
+			    granted);
+	lprocfs_counter_add(pl->pl_stats, LDLM_POOL_GRANT_RATE_STAT,
+			    grant_rate);
+	lprocfs_counter_add(pl->pl_stats, LDLM_POOL_GRANT_PLAN_STAT,
+			    grant_plan);
+	lprocfs_counter_add(pl->pl_stats, LDLM_POOL_CANCEL_RATE_STAT,
+			    cancel_rate);
+}
+
+/**
+ * Sets SLV and Limit from container_of(pl, struct ldlm_namespace,
+ * ns_pool)->ns_obd tp passed \a pl.
+ */
+static void ldlm_cli_pool_pop_slv(struct ldlm_pool *pl)
+{
+	struct obd_device *obd;
+
+	/*
+	 * Get new SLV and Limit from obd which is updated with coming
+	 * RPCs.
+	 */
+	obd = container_of(pl, struct ldlm_namespace,
+			   ns_pool)->ns_obd;
+	read_lock(&obd->obd_pool_lock);
+	pl->pl_server_lock_volume = obd->obd_pool_slv;
+	atomic_set(&pl->pl_limit, obd->obd_pool_limit);
+	read_unlock(&obd->obd_pool_lock);
+}
+
+/**
+ * Recalculates client size pool \a pl according to current SLV and Limit.
+ */
+static int ldlm_cli_pool_recalc(struct ldlm_pool *pl)
+{
+	time64_t recalc_interval_sec;
+	int ret;
+
+	recalc_interval_sec = ktime_get_real_seconds() - pl->pl_recalc_time;
+	if (recalc_interval_sec < pl->pl_recalc_period)
+		return 0;
+
+	spin_lock(&pl->pl_lock);
+	/*
+	 * Check if we need to recalc lists now.
+	 */
+	recalc_interval_sec = ktime_get_real_seconds() - pl->pl_recalc_time;
+	if (recalc_interval_sec < pl->pl_recalc_period) {
+		spin_unlock(&pl->pl_lock);
+		return 0;
+	}
+
+	/*
+	 * Make sure that pool knows last SLV and Limit from obd.
+	 */
+	ldlm_cli_pool_pop_slv(pl);
+
+	spin_unlock(&pl->pl_lock);
+
+	/*
+	 * Do not cancel locks in case lru resize is disabled for this ns.
+	 */
+	if (!ns_connect_lru_resize(container_of(pl, struct ldlm_namespace,
+						ns_pool))) {
+		ret = 0;
+		goto out;
+	}
+
+	/*
+	 * In the time of canceling locks on client we do not need to maintain
+	 * sharp timing, we only want to cancel locks asap according to new SLV.
+	 * It may be called when SLV has changed much, this is why we do not
+	 * take into account pl->pl_recalc_time here.
+	 */
+	ret = ldlm_cancel_lru(container_of(pl, struct ldlm_namespace, ns_pool),
+			      0, LCF_ASYNC, LDLM_LRU_FLAG_LRUR);
+
+out:
+	spin_lock(&pl->pl_lock);
+	/*
+	 * Time of LRU resizing might be longer than period,
+	 * so update after LRU resizing rather than before it.
+	 */
+	pl->pl_recalc_time = ktime_get_real_seconds();
+	lprocfs_counter_add(pl->pl_stats, LDLM_POOL_TIMING_STAT,
+			    recalc_interval_sec);
+	spin_unlock(&pl->pl_lock);
+	return ret;
+}
+
+/**
+ * This function is main entry point for memory pressure handling on client
+ * side.  Main goal of this function is to cancel some number of locks on
+ * passed \a pl according to \a nr and \a gfp_mask.
+ */
+static int ldlm_cli_pool_shrink(struct ldlm_pool *pl,
+				int nr, gfp_t gfp_mask)
+{
+	struct ldlm_namespace *ns;
+	int unused;
+
+	ns = container_of(pl, struct ldlm_namespace, ns_pool);
+
+	/*
+	 * Do not cancel locks in case lru resize is disabled for this ns.
+	 */
+	if (!ns_connect_lru_resize(ns))
+		return 0;
+
+	/*
+	 * Make sure that pool knows last SLV and Limit from obd.
+	 */
+	ldlm_cli_pool_pop_slv(pl);
+
+	spin_lock(&ns->ns_lock);
+	unused = ns->ns_nr_unused;
+	spin_unlock(&ns->ns_lock);
+
+	if (nr == 0)
+		return (unused / 100) * sysctl_vfs_cache_pressure;
+	else
+		return ldlm_cancel_lru(ns, nr, LCF_ASYNC, LDLM_LRU_FLAG_SHRINK);
+}
+
+static const struct ldlm_pool_ops ldlm_cli_pool_ops = {
+	.po_recalc = ldlm_cli_pool_recalc,
+	.po_shrink = ldlm_cli_pool_shrink
+};
+
+/**
+ * Pool recalc wrapper. Will call either client or server pool recalc callback
+ * depending what pool \a pl is used.
+ */
+static int ldlm_pool_recalc(struct ldlm_pool *pl)
+{
+	u32 recalc_interval_sec;
+	int count;
+
+	recalc_interval_sec = ktime_get_real_seconds() - pl->pl_recalc_time;
+	if (recalc_interval_sec > 0) {
+		spin_lock(&pl->pl_lock);
+		recalc_interval_sec = ktime_get_real_seconds() - pl->pl_recalc_time;
+
+		if (recalc_interval_sec > 0) {
+			/*
+			 * Update pool statistics every 1s.
+			 */
+			ldlm_pool_recalc_stats(pl);
+
+			/*
+			 * Zero out all rates and speed for the last period.
+			 */
+			atomic_set(&pl->pl_grant_rate, 0);
+			atomic_set(&pl->pl_cancel_rate, 0);
+		}
+		spin_unlock(&pl->pl_lock);
+	}
+
+	if (pl->pl_ops->po_recalc) {
+		count = pl->pl_ops->po_recalc(pl);
+		lprocfs_counter_add(pl->pl_stats, LDLM_POOL_RECALC_STAT,
+				    count);
+	}
+
+	recalc_interval_sec = pl->pl_recalc_time - ktime_get_real_seconds() +
+			      pl->pl_recalc_period;
+	if (recalc_interval_sec <= 0) {
+		/* DEBUG: should be re-removed after LU-4536 is fixed */
+		CDEBUG(D_DLMTRACE,
+		       "%s: Negative interval(%ld), too short period(%ld)\n",
+		       pl->pl_name, (long)recalc_interval_sec,
+		       (long)pl->pl_recalc_period);
+
+		/* Prevent too frequent recalculation. */
+		recalc_interval_sec = 1;
+	}
+
+	return recalc_interval_sec;
+}
+
+/*
+ * Pool shrink wrapper. Will call either client or server pool recalc callback
+ * depending what pool pl is used. When nr == 0, just return the number of
+ * freeable locks. Otherwise, return the number of canceled locks.
+ */
+static int ldlm_pool_shrink(struct ldlm_pool *pl, int nr, gfp_t gfp_mask)
+{
+	int cancel = 0;
+
+	if (pl->pl_ops->po_shrink) {
+		cancel = pl->pl_ops->po_shrink(pl, nr, gfp_mask);
+		if (nr > 0) {
+			lprocfs_counter_add(pl->pl_stats,
+					    LDLM_POOL_SHRINK_REQTD_STAT,
+					    nr);
+			lprocfs_counter_add(pl->pl_stats,
+					    LDLM_POOL_SHRINK_FREED_STAT,
+					    cancel);
+			CDEBUG(D_DLMTRACE,
+			       "%s: request to shrink %d locks, shrunk %d\n",
+			       pl->pl_name, nr, cancel);
+		}
+	}
+	return cancel;
+}
+
+static int lprocfs_pool_state_seq_show(struct seq_file *m, void *unused)
+{
+	int granted, grant_rate, cancel_rate;
+	int grant_speed, lvf;
+	struct ldlm_pool *pl = m->private;
+	__u64 slv, clv;
+	__u32 limit;
+
+	spin_lock(&pl->pl_lock);
+	slv = pl->pl_server_lock_volume;
+	clv = pl->pl_client_lock_volume;
+	limit = atomic_read(&pl->pl_limit);
+	granted = atomic_read(&pl->pl_granted);
+	grant_rate = atomic_read(&pl->pl_grant_rate);
+	cancel_rate = atomic_read(&pl->pl_cancel_rate);
+	grant_speed = grant_rate - cancel_rate;
+	lvf = atomic_read(&pl->pl_lock_volume_factor);
+	spin_unlock(&pl->pl_lock);
+
+	seq_printf(m, "LDLM pool state (%s):\n"
+		      "  SLV: %llu\n"
+		      "  CLV: %llu\n"
+		      "  LVF: %d\n",
+		      pl->pl_name, slv, clv, lvf);
+
+	seq_printf(m, "  GR:  %d\n  CR:  %d\n  GS:  %d\n"
+		      "  G:   %d\n  L:   %d\n",
+		      grant_rate, cancel_rate, grant_speed,
+		      granted, limit);
+
+	return 0;
+}
+
+LPROC_SEQ_FOPS_RO(lprocfs_pool_state);
+
+static ssize_t grant_speed_show(struct kobject *kobj, struct attribute *attr,
+				char *buf)
+{
+	struct ldlm_pool *pl = container_of(kobj, struct ldlm_pool,
+					    pl_kobj);
+
+	int	       grant_speed;
+
+	spin_lock(&pl->pl_lock);
+	/* serialize with ldlm_pool_recalc */
+	grant_speed = atomic_read(&pl->pl_grant_rate) -
+			atomic_read(&pl->pl_cancel_rate);
+	spin_unlock(&pl->pl_lock);
+	return sprintf(buf, "%d\n", grant_speed);
+}
+LUSTRE_RO_ATTR(grant_speed);
+
+LDLM_POOL_SYSFS_READER_SHOW(grant_plan, int);
+LUSTRE_RO_ATTR(grant_plan);
+
+LDLM_POOL_SYSFS_READER_SHOW(recalc_period, int);
+LDLM_POOL_SYSFS_WRITER_STORE(recalc_period, int);
+LUSTRE_RW_ATTR(recalc_period);
+
+LDLM_POOL_SYSFS_READER_NOLOCK_SHOW(server_lock_volume, u64);
+LUSTRE_RO_ATTR(server_lock_volume);
+
+LDLM_POOL_SYSFS_READER_NOLOCK_SHOW(limit, atomic);
+LDLM_POOL_SYSFS_WRITER_NOLOCK_STORE(limit, atomic);
+LUSTRE_RW_ATTR(limit);
+
+LDLM_POOL_SYSFS_READER_NOLOCK_SHOW(granted, atomic);
+LUSTRE_RO_ATTR(granted);
+
+LDLM_POOL_SYSFS_READER_NOLOCK_SHOW(cancel_rate, atomic);
+LUSTRE_RO_ATTR(cancel_rate);
+
+LDLM_POOL_SYSFS_READER_NOLOCK_SHOW(grant_rate, atomic);
+LUSTRE_RO_ATTR(grant_rate);
+
+LDLM_POOL_SYSFS_READER_NOLOCK_SHOW(lock_volume_factor, atomic);
+LDLM_POOL_SYSFS_WRITER_NOLOCK_STORE(lock_volume_factor, atomic);
+LUSTRE_RW_ATTR(lock_volume_factor);
+
+#define LDLM_POOL_ADD_VAR(name, var, ops)			\
+	do {							\
+		snprintf(var_name, MAX_STRING_SIZE, #name);	\
+		pool_vars[0].data = var;			\
+		pool_vars[0].fops = ops;			\
+		ldebugfs_add_vars(pl->pl_debugfs_entry, pool_vars, NULL);\
+	} while (0)
+
+/* These are for pools in /sys/fs/lustre/ldlm/namespaces/.../pool */
+static struct attribute *ldlm_pl_attrs[] = {
+	&lustre_attr_grant_speed.attr,
+	&lustre_attr_grant_plan.attr,
+	&lustre_attr_recalc_period.attr,
+	&lustre_attr_server_lock_volume.attr,
+	&lustre_attr_limit.attr,
+	&lustre_attr_granted.attr,
+	&lustre_attr_cancel_rate.attr,
+	&lustre_attr_grant_rate.attr,
+	&lustre_attr_lock_volume_factor.attr,
+	NULL,
+};
+
+static void ldlm_pl_release(struct kobject *kobj)
+{
+	struct ldlm_pool *pl = container_of(kobj, struct ldlm_pool,
+					    pl_kobj);
+	complete(&pl->pl_kobj_unregister);
+}
+
+static struct kobj_type ldlm_pl_ktype = {
+	.default_attrs	= ldlm_pl_attrs,
+	.sysfs_ops	= &lustre_sysfs_ops,
+	.release	= ldlm_pl_release,
+};
+
+static int ldlm_pool_sysfs_init(struct ldlm_pool *pl)
+{
+	struct ldlm_namespace *ns = container_of(pl, struct ldlm_namespace,
+						 ns_pool);
+	int err;
+
+	init_completion(&pl->pl_kobj_unregister);
+	err = kobject_init_and_add(&pl->pl_kobj, &ldlm_pl_ktype, &ns->ns_kobj,
+				   "pool");
+
+	return err;
+}
+
+static int ldlm_pool_debugfs_init(struct ldlm_pool *pl)
+{
+	struct ldlm_namespace *ns = container_of(pl, struct ldlm_namespace,
+						 ns_pool);
+	struct dentry *debugfs_ns_parent;
+	struct lprocfs_vars pool_vars[2];
+	char *var_name = NULL;
+	int rc = 0;
+
+	var_name = kzalloc(MAX_STRING_SIZE + 1, GFP_NOFS);
+	if (!var_name)
+		return -ENOMEM;
+
+	debugfs_ns_parent = ns->ns_debugfs_entry;
+	if (IS_ERR_OR_NULL(debugfs_ns_parent)) {
+		CERROR("%s: debugfs entry is not initialized\n",
+		       ldlm_ns_name(ns));
+		rc = -EINVAL;
+		goto out_free_name;
+	}
+	pl->pl_debugfs_entry = debugfs_create_dir("pool", debugfs_ns_parent);
+
+	var_name[MAX_STRING_SIZE] = '\0';
+	memset(pool_vars, 0, sizeof(pool_vars));
+	pool_vars[0].name = var_name;
+
+	LDLM_POOL_ADD_VAR(state, pl, &lprocfs_pool_state_fops);
+
+	pl->pl_stats = lprocfs_alloc_stats(LDLM_POOL_LAST_STAT -
+					   LDLM_POOL_FIRST_STAT, 0);
+	if (!pl->pl_stats) {
+		rc = -ENOMEM;
+		goto out_free_name;
+	}
+
+	lprocfs_counter_init(pl->pl_stats, LDLM_POOL_GRANTED_STAT,
+			     LPROCFS_CNTR_AVGMINMAX | LPROCFS_CNTR_STDDEV,
+			     "granted", "locks");
+	lprocfs_counter_init(pl->pl_stats, LDLM_POOL_GRANT_STAT,
+			     LPROCFS_CNTR_AVGMINMAX | LPROCFS_CNTR_STDDEV,
+			     "grant", "locks");
+	lprocfs_counter_init(pl->pl_stats, LDLM_POOL_CANCEL_STAT,
+			     LPROCFS_CNTR_AVGMINMAX | LPROCFS_CNTR_STDDEV,
+			     "cancel", "locks");
+	lprocfs_counter_init(pl->pl_stats, LDLM_POOL_GRANT_RATE_STAT,
+			     LPROCFS_CNTR_AVGMINMAX | LPROCFS_CNTR_STDDEV,
+			     "grant_rate", "locks/s");
+	lprocfs_counter_init(pl->pl_stats, LDLM_POOL_CANCEL_RATE_STAT,
+			     LPROCFS_CNTR_AVGMINMAX | LPROCFS_CNTR_STDDEV,
+			     "cancel_rate", "locks/s");
+	lprocfs_counter_init(pl->pl_stats, LDLM_POOL_GRANT_PLAN_STAT,
+			     LPROCFS_CNTR_AVGMINMAX | LPROCFS_CNTR_STDDEV,
+			     "grant_plan", "locks/s");
+	lprocfs_counter_init(pl->pl_stats, LDLM_POOL_SLV_STAT,
+			     LPROCFS_CNTR_AVGMINMAX | LPROCFS_CNTR_STDDEV,
+			     "slv", "slv");
+	lprocfs_counter_init(pl->pl_stats, LDLM_POOL_SHRINK_REQTD_STAT,
+			     LPROCFS_CNTR_AVGMINMAX | LPROCFS_CNTR_STDDEV,
+			     "shrink_request", "locks");
+	lprocfs_counter_init(pl->pl_stats, LDLM_POOL_SHRINK_FREED_STAT,
+			     LPROCFS_CNTR_AVGMINMAX | LPROCFS_CNTR_STDDEV,
+			     "shrink_freed", "locks");
+	lprocfs_counter_init(pl->pl_stats, LDLM_POOL_RECALC_STAT,
+			     LPROCFS_CNTR_AVGMINMAX | LPROCFS_CNTR_STDDEV,
+			     "recalc_freed", "locks");
+	lprocfs_counter_init(pl->pl_stats, LDLM_POOL_TIMING_STAT,
+			     LPROCFS_CNTR_AVGMINMAX | LPROCFS_CNTR_STDDEV,
+			     "recalc_timing", "sec");
+	debugfs_create_file("stats", 0644, pl->pl_debugfs_entry, pl->pl_stats,
+			    &lprocfs_stats_seq_fops);
+
+out_free_name:
+	kfree(var_name);
+	return rc;
+}
+
+static void ldlm_pool_sysfs_fini(struct ldlm_pool *pl)
+{
+	kobject_put(&pl->pl_kobj);
+	wait_for_completion(&pl->pl_kobj_unregister);
+}
+
+static void ldlm_pool_debugfs_fini(struct ldlm_pool *pl)
+{
+	if (pl->pl_stats) {
+		lprocfs_free_stats(&pl->pl_stats);
+		pl->pl_stats = NULL;
+	}
+	debugfs_remove_recursive(pl->pl_debugfs_entry);
+}
+
+int ldlm_pool_init(struct ldlm_pool *pl, struct ldlm_namespace *ns,
+		   int idx, enum ldlm_side client)
+{
+	int rc;
+
+	spin_lock_init(&pl->pl_lock);
+	atomic_set(&pl->pl_granted, 0);
+	pl->pl_recalc_time = ktime_get_real_seconds();
+	atomic_set(&pl->pl_lock_volume_factor, 1);
+
+	atomic_set(&pl->pl_grant_rate, 0);
+	atomic_set(&pl->pl_cancel_rate, 0);
+	pl->pl_grant_plan = LDLM_POOL_GP(LDLM_POOL_HOST_L);
+
+	snprintf(pl->pl_name, sizeof(pl->pl_name), "ldlm-pool-%s-%d",
+		 ldlm_ns_name(ns), idx);
+
+	atomic_set(&pl->pl_limit, 1);
+	pl->pl_server_lock_volume = 0;
+	pl->pl_ops = &ldlm_cli_pool_ops;
+	pl->pl_recalc_period = LDLM_POOL_CLI_DEF_RECALC_PERIOD;
+	pl->pl_client_lock_volume = 0;
+	rc = ldlm_pool_debugfs_init(pl);
+	if (rc)
+		return rc;
+
+	rc = ldlm_pool_sysfs_init(pl);
+	if (rc)
+		return rc;
+
+	CDEBUG(D_DLMTRACE, "Lock pool %s is initialized\n", pl->pl_name);
+
+	return rc;
+}
+
+void ldlm_pool_fini(struct ldlm_pool *pl)
+{
+	ldlm_pool_sysfs_fini(pl);
+	ldlm_pool_debugfs_fini(pl);
+
+	/*
+	 * Pool should not be used after this point. We can't free it here as
+	 * it lives in struct ldlm_namespace, but still interested in catching
+	 * any abnormal using cases.
+	 */
+	POISON(pl, 0x5a, sizeof(*pl));
+}
+
+/**
+ * Add new taken ldlm lock \a lock into pool \a pl accounting.
+ */
+void ldlm_pool_add(struct ldlm_pool *pl, struct ldlm_lock *lock)
+{
+	/*
+	 * FLOCK locks are special in a sense that they are almost never
+	 * cancelled, instead special kind of lock is used to drop them.
+	 * also there is no LRU for flock locks, so no point in tracking
+	 * them anyway.
+	 */
+	if (lock->l_resource->lr_type == LDLM_FLOCK)
+		return;
+
+	atomic_inc(&pl->pl_granted);
+	atomic_inc(&pl->pl_grant_rate);
+	lprocfs_counter_incr(pl->pl_stats, LDLM_POOL_GRANT_STAT);
+	/*
+	 * Do not do pool recalc for client side as all locks which
+	 * potentially may be canceled has already been packed into
+	 * enqueue/cancel rpc. Also we do not want to run out of stack
+	 * with too long call paths.
+	 */
+}
+
+/**
+ * Remove ldlm lock \a lock from pool \a pl accounting.
+ */
+void ldlm_pool_del(struct ldlm_pool *pl, struct ldlm_lock *lock)
+{
+	/*
+	 * Filter out FLOCK locks. Read above comment in ldlm_pool_add().
+	 */
+	if (lock->l_resource->lr_type == LDLM_FLOCK)
+		return;
+
+	LASSERT(atomic_read(&pl->pl_granted) > 0);
+	atomic_dec(&pl->pl_granted);
+	atomic_inc(&pl->pl_cancel_rate);
+
+	lprocfs_counter_incr(pl->pl_stats, LDLM_POOL_CANCEL_STAT);
+}
+
+/**
+ * Returns current \a pl SLV.
+ *
+ * \pre ->pl_lock is not locked.
+ */
+__u64 ldlm_pool_get_slv(struct ldlm_pool *pl)
+{
+	__u64 slv;
+
+	spin_lock(&pl->pl_lock);
+	slv = pl->pl_server_lock_volume;
+	spin_unlock(&pl->pl_lock);
+	return slv;
+}
+
+/**
+ * Sets passed \a clv to \a pl.
+ *
+ * \pre ->pl_lock is not locked.
+ */
+void ldlm_pool_set_clv(struct ldlm_pool *pl, __u64 clv)
+{
+	spin_lock(&pl->pl_lock);
+	pl->pl_client_lock_volume = clv;
+	spin_unlock(&pl->pl_lock);
+}
+
+/**
+ * Returns current LVF from \a pl.
+ */
+__u32 ldlm_pool_get_lvf(struct ldlm_pool *pl)
+{
+	return atomic_read(&pl->pl_lock_volume_factor);
+}
+
+static int ldlm_pool_granted(struct ldlm_pool *pl)
+{
+	return atomic_read(&pl->pl_granted);
+}
+
+/*
+ * count locks from all namespaces (if possible). Returns number of
+ * cached locks.
+ */
+static unsigned long ldlm_pools_count(enum ldlm_side client, gfp_t gfp_mask)
+{
+	unsigned long total = 0;
+	int nr_ns;
+	struct ldlm_namespace *ns;
+	struct ldlm_namespace *ns_old = NULL; /* loop detection */
+
+	if (client == LDLM_NAMESPACE_CLIENT && !(gfp_mask & __GFP_FS))
+		return 0;
+
+	CDEBUG(D_DLMTRACE, "Request to count %s locks from all pools\n",
+	       client == LDLM_NAMESPACE_CLIENT ? "client" : "server");
+
+	/*
+	 * Find out how many resources we may release.
+	 */
+	for (nr_ns = ldlm_namespace_nr_read(client);
+	     nr_ns > 0; nr_ns--) {
+		mutex_lock(ldlm_namespace_lock(client));
+		if (list_empty(ldlm_namespace_list(client))) {
+			mutex_unlock(ldlm_namespace_lock(client));
+			return 0;
+		}
+		ns = ldlm_namespace_first_locked(client);
+
+		if (ns == ns_old) {
+			mutex_unlock(ldlm_namespace_lock(client));
+			break;
+		}
+
+		if (ldlm_ns_empty(ns)) {
+			ldlm_namespace_move_to_inactive_locked(ns, client);
+			mutex_unlock(ldlm_namespace_lock(client));
+			continue;
+		}
+
+		if (!ns_old)
+			ns_old = ns;
+
+		ldlm_namespace_get(ns);
+		ldlm_namespace_move_to_active_locked(ns, client);
+		mutex_unlock(ldlm_namespace_lock(client));
+		total += ldlm_pool_shrink(&ns->ns_pool, 0, gfp_mask);
+		ldlm_namespace_put(ns);
+	}
+
+	return total;
+}
+
+static unsigned long ldlm_pools_scan(enum ldlm_side client, int nr,
+				     gfp_t gfp_mask)
+{
+	unsigned long freed = 0;
+	int tmp, nr_ns;
+	struct ldlm_namespace *ns;
+
+	if (client == LDLM_NAMESPACE_CLIENT && !(gfp_mask & __GFP_FS))
+		return -1;
+
+	/*
+	 * Shrink at least ldlm_namespace_nr_read(client) namespaces.
+	 */
+	for (tmp = nr_ns = ldlm_namespace_nr_read(client);
+	     tmp > 0; tmp--) {
+		int cancel, nr_locks;
+
+		/*
+		 * Do not call shrink under ldlm_namespace_lock(client)
+		 */
+		mutex_lock(ldlm_namespace_lock(client));
+		if (list_empty(ldlm_namespace_list(client))) {
+			mutex_unlock(ldlm_namespace_lock(client));
+			break;
+		}
+		ns = ldlm_namespace_first_locked(client);
+		ldlm_namespace_get(ns);
+		ldlm_namespace_move_to_active_locked(ns, client);
+		mutex_unlock(ldlm_namespace_lock(client));
+
+		nr_locks = ldlm_pool_granted(&ns->ns_pool);
+		/*
+		 * We use to shrink propotionally but with new shrinker API,
+		 * we lost the total number of freeable locks.
+		 */
+		cancel = 1 + min_t(int, nr_locks, nr / nr_ns);
+		freed += ldlm_pool_shrink(&ns->ns_pool, cancel, gfp_mask);
+		ldlm_namespace_put(ns);
+	}
+	/*
+	 * we only decrease the SLV in server pools shrinker, return
+	 * SHRINK_STOP to kernel to avoid needless loop. LU-1128
+	 */
+	return freed;
+}
+
+static unsigned long ldlm_pools_cli_count(struct shrinker *s,
+					  struct shrink_control *sc)
+{
+	return ldlm_pools_count(LDLM_NAMESPACE_CLIENT, sc->gfp_mask);
+}
+
+static unsigned long ldlm_pools_cli_scan(struct shrinker *s,
+					 struct shrink_control *sc)
+{
+	return ldlm_pools_scan(LDLM_NAMESPACE_CLIENT, sc->nr_to_scan,
+			       sc->gfp_mask);
+}
+
+static void ldlm_pools_recalc(struct work_struct *ws);
+static DECLARE_DELAYED_WORK(ldlm_recalc_pools, ldlm_pools_recalc);
+
+static void ldlm_pools_recalc(struct work_struct *ws)
+{
+	enum ldlm_side client = LDLM_NAMESPACE_CLIENT;
+	struct ldlm_namespace *ns;
+	struct ldlm_namespace *ns_old = NULL;
+	/* seconds of sleep if no active namespaces */
+	int time = LDLM_POOL_CLI_DEF_RECALC_PERIOD;
+	int nr;
+
+	/*
+	 * Recalc at least ldlm_namespace_nr_read(client) namespaces.
+	 */
+	for (nr = ldlm_namespace_nr_read(client); nr > 0; nr--) {
+		int     skip;
+		/*
+		 * Lock the list, get first @ns in the list, getref, move it
+		 * to the tail, unlock and call pool recalc. This way we avoid
+		 * calling recalc under @ns lock what is really good as we get
+		 * rid of potential deadlock on client nodes when canceling
+		 * locks synchronously.
+		 */
+		mutex_lock(ldlm_namespace_lock(client));
+		if (list_empty(ldlm_namespace_list(client))) {
+			mutex_unlock(ldlm_namespace_lock(client));
+			break;
+		}
+		ns = ldlm_namespace_first_locked(client);
+
+		if (ns_old == ns) { /* Full pass complete */
+			mutex_unlock(ldlm_namespace_lock(client));
+			break;
+		}
+
+		/* We got an empty namespace, need to move it back to inactive
+		 * list.
+		 * The race with parallel resource creation is fine:
+		 * - If they do namespace_get before our check, we fail the
+		 *   check and they move this item to the end of the list anyway
+		 * - If we do the check and then they do namespace_get, then
+		 *   we move the namespace to inactive and they will move
+		 *   it back to active (synchronised by the lock, so no clash
+		 *   there).
+		 */
+		if (ldlm_ns_empty(ns)) {
+			ldlm_namespace_move_to_inactive_locked(ns, client);
+			mutex_unlock(ldlm_namespace_lock(client));
+			continue;
+		}
+
+		if (!ns_old)
+			ns_old = ns;
+
+		spin_lock(&ns->ns_lock);
+		/*
+		 * skip ns which is being freed, and we don't want to increase
+		 * its refcount again, not even temporarily. bz21519 & LU-499.
+		 */
+		if (ns->ns_stopping) {
+			skip = 1;
+		} else {
+			skip = 0;
+			ldlm_namespace_get(ns);
+		}
+		spin_unlock(&ns->ns_lock);
+
+		ldlm_namespace_move_to_active_locked(ns, client);
+		mutex_unlock(ldlm_namespace_lock(client));
+
+		/*
+		 * After setup is done - recalc the pool.
+		 */
+		if (!skip) {
+			int ttime = ldlm_pool_recalc(&ns->ns_pool);
+
+			if (ttime < time)
+				time = ttime;
+
+			ldlm_namespace_put(ns);
+		}
+	}
+
+	/* Wake up the blocking threads from time to time. */
+	ldlm_bl_thread_wakeup();
+
+	schedule_delayed_work(&ldlm_recalc_pools, time * HZ);
+}
+
+static int ldlm_pools_thread_start(void)
+{
+	schedule_delayed_work(&ldlm_recalc_pools, 0);
+
+	return 0;
+}
+
+static void ldlm_pools_thread_stop(void)
+{
+	cancel_delayed_work_sync(&ldlm_recalc_pools);
+}
+
+static struct shrinker ldlm_pools_cli_shrinker = {
+	.count_objects	= ldlm_pools_cli_count,
+	.scan_objects	= ldlm_pools_cli_scan,
+	.seeks		= DEFAULT_SEEKS,
+};
+
+int ldlm_pools_init(void)
+{
+	int rc;
+
+	rc = ldlm_pools_thread_start();
+	if (!rc)
+		rc = register_shrinker(&ldlm_pools_cli_shrinker);
+
+	return rc;
+}
+
+void ldlm_pools_fini(void)
+{
+	unregister_shrinker(&ldlm_pools_cli_shrinker);
+
+	ldlm_pools_thread_stop();
+}
diff --git a/drivers/staging/lustre/lustre/ptlrpc/ldlm_request.c b/drivers/staging/lustre/lustre/ptlrpc/ldlm_request.c
new file mode 100644
index 000000000000..cdc52eed6d85
--- /dev/null
+++ b/drivers/staging/lustre/lustre/ptlrpc/ldlm_request.c
@@ -0,0 +1,2033 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * GPL HEADER START
+ *
+ * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 only,
+ * as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License version 2 for more details (a copy is included
+ * in the LICENSE file that accompanied this code).
+ *
+ * You should have received a copy of the GNU General Public License
+ * version 2 along with this program; If not, see
+ * http://www.gnu.org/licenses/gpl-2.0.html
+ *
+ * GPL HEADER END
+ */
+/*
+ * Copyright (c) 2002, 2010, Oracle and/or its affiliates. All rights reserved.
+ * Use is subject to license terms.
+ *
+ * Copyright (c) 2010, 2015, Intel Corporation.
+ */
+/*
+ * This file is part of Lustre, http://www.lustre.org/
+ * Lustre is a trademark of Sun Microsystems, Inc.
+ */
+/**
+ * This file contains Asynchronous System Trap (AST) handlers and related
+ * LDLM request-processing routines.
+ *
+ * An AST is a callback issued on a lock when its state is changed. There are
+ * several different types of ASTs (callbacks) registered for each lock:
+ *
+ * - completion AST: when a lock is enqueued by some process, but cannot be
+ *   granted immediately due to other conflicting locks on the same resource,
+ *   the completion AST is sent to notify the caller when the lock is
+ *   eventually granted
+ *
+ * - blocking AST: when a lock is granted to some process, if another process
+ *   enqueues a conflicting (blocking) lock on a resource, a blocking AST is
+ *   sent to notify the holder(s) of the lock(s) of the conflicting lock
+ *   request. The lock holder(s) must release their lock(s) on that resource in
+ *   a timely manner or be evicted by the server.
+ *
+ * - glimpse AST: this is used when a process wants information about a lock
+ *   (i.e. the lock value block (LVB)) but does not necessarily require holding
+ *   the lock. If the resource is locked, the lock holder(s) are sent glimpse
+ *   ASTs and the LVB is returned to the caller, and lock holder(s) may CANCEL
+ *   their lock(s) if they are idle. If the resource is not locked, the server
+ *   may grant the lock.
+ */
+
+#define DEBUG_SUBSYSTEM S_LDLM
+
+#include <lustre_errno.h>
+#include <lustre_dlm.h>
+#include <obd_class.h>
+#include <obd.h>
+#include <linux/libcfs/libcfs_hash.h>
+
+#include "ldlm_internal.h"
+
+unsigned int ldlm_enqueue_min = OBD_TIMEOUT_DEFAULT;
+module_param(ldlm_enqueue_min, uint, 0644);
+MODULE_PARM_DESC(ldlm_enqueue_min, "lock enqueue timeout minimum");
+
+/* in client side, whether the cached locks will be canceled before replay */
+unsigned int ldlm_cancel_unused_locks_before_replay = 1;
+
+struct ldlm_async_args {
+	struct lustre_handle lock_handle;
+};
+
+/**
+ * ldlm_request_bufsize
+ *
+ * @count:	number of ldlm handles
+ * @type:	ldlm opcode
+ *
+ * If opcode=LDLM_ENQUEUE, 1 slot is already occupied,
+ * LDLM_LOCKREQ_HANDLE -1 slots are available.
+ * Otherwise, LDLM_LOCKREQ_HANDLE slots are available.
+ *
+ * Return:	size of the request buffer
+ */
+static int ldlm_request_bufsize(int count, int type)
+{
+	int avail = LDLM_LOCKREQ_HANDLES;
+
+	if (type == LDLM_ENQUEUE)
+		avail -= LDLM_ENQUEUE_CANCEL_OFF;
+
+	if (count > avail)
+		avail = (count - avail) * sizeof(struct lustre_handle);
+	else
+		avail = 0;
+
+	return sizeof(struct ldlm_request) + avail;
+}
+
+static void ldlm_expired_completion_wait(struct ldlm_lock *lock, __u32 conn_cnt)
+{
+	struct obd_import *imp;
+	struct obd_device *obd;
+
+	if (!lock->l_conn_export) {
+		static unsigned long next_dump, last_dump;
+
+		LDLM_ERROR(lock,
+			   "lock timed out (enqueued at %lld, %llds ago); not entering recovery in server code, just going back to sleep",
+			   (s64)lock->l_last_activity,
+			   (s64)(ktime_get_real_seconds() -
+				 lock->l_last_activity));
+		if (time_after(jiffies, next_dump)) {
+			last_dump = next_dump;
+			next_dump = jiffies + 300 * HZ;
+			ldlm_namespace_dump(D_DLMTRACE,
+					    ldlm_lock_to_ns(lock));
+			if (last_dump == 0)
+				libcfs_debug_dumplog();
+		}
+		return;
+	}
+
+	obd = lock->l_conn_export->exp_obd;
+	imp = obd->u.cli.cl_import;
+	ptlrpc_fail_import(imp, conn_cnt);
+	LDLM_ERROR(lock,
+		   "lock timed out (enqueued at %lld, %llds ago), entering recovery for %s@%s",
+		   (s64)lock->l_last_activity,
+		   (s64)(ktime_get_real_seconds() - lock->l_last_activity),
+		   obd2cli_tgt(obd), imp->imp_connection->c_remote_uuid.uuid);
+}
+
+/**
+ * Calculate the Completion timeout (covering enqueue, BL AST, data flush,
+ * lock cancel, and their replies). Used for lock completion timeout on the
+ * client side.
+ *
+ * \param[in] lock	lock which is waiting the completion callback
+ *
+ * \retval		timeout in seconds to wait for the server reply
+ */
+/* We use the same basis for both server side and client side functions
+ * from a single node.
+ */
+static unsigned int ldlm_cp_timeout(struct ldlm_lock *lock)
+{
+	unsigned int timeout;
+
+	if (AT_OFF)
+		return obd_timeout;
+
+	/*
+	 * Wait a long time for enqueue - server may have to callback a
+	 * lock from another client.  Server will evict the other client if it
+	 * doesn't respond reasonably, and then give us the lock.
+	 */
+	timeout = at_get(ldlm_lock_to_ns_at(lock));
+	return max(3 * timeout, ldlm_enqueue_min);
+}
+
+/**
+ * Helper function for ldlm_completion_ast(), updating timings when lock is
+ * actually granted.
+ */
+static int ldlm_completion_tail(struct ldlm_lock *lock, void *data)
+{
+	long delay;
+	int result = 0;
+
+	if (ldlm_is_destroyed(lock) || ldlm_is_failed(lock)) {
+		LDLM_DEBUG(lock, "client-side enqueue: destroyed");
+		result = -EIO;
+	} else if (!data) {
+		LDLM_DEBUG(lock, "client-side enqueue: granted");
+	} else {
+		/* Take into AT only CP RPC, not immediately granted locks */
+		delay = ktime_get_real_seconds() - lock->l_last_activity;
+		LDLM_DEBUG(lock, "client-side enqueue: granted after %lds",
+			   delay);
+
+		/* Update our time estimate */
+		at_measured(ldlm_lock_to_ns_at(lock), delay);
+	}
+	return result;
+}
+
+/**
+ * Generic LDLM "completion" AST. This is called in several cases:
+ *
+ *     - when a reply to an ENQUEUE RPC is received from the server
+ *       (ldlm_cli_enqueue_fini()). Lock might be granted or not granted at
+ *       this point (determined by flags);
+ *
+ *     - when LDLM_CP_CALLBACK RPC comes to client to notify it that lock has
+ *       been granted;
+ *
+ *     - when ldlm_lock_match(LDLM_FL_LVB_READY) is about to wait until lock
+ *       gets correct lvb;
+ *
+ *     - to force all locks when resource is destroyed (cleanup_resource());
+ *
+ *     - during lock conversion (not used currently).
+ *
+ * If lock is not granted in the first case, this function waits until second
+ * or penultimate cases happen in some other thread.
+ *
+ */
+int ldlm_completion_ast(struct ldlm_lock *lock, __u64 flags, void *data)
+{
+	/* XXX ALLOCATE - 160 bytes */
+	struct obd_device *obd;
+	struct obd_import *imp = NULL;
+	__u32 timeout;
+	__u32 conn_cnt = 0;
+	int rc = 0;
+
+	if (flags == LDLM_FL_WAIT_NOREPROC) {
+		LDLM_DEBUG(lock, "client-side enqueue waiting on pending lock");
+		goto noreproc;
+	}
+
+	if (!(flags & LDLM_FL_BLOCKED_MASK)) {
+		wake_up(&lock->l_waitq);
+		return 0;
+	}
+
+	LDLM_DEBUG(lock,
+		   "client-side enqueue returned a blocked lock, sleeping");
+
+noreproc:
+
+	obd = class_exp2obd(lock->l_conn_export);
+
+	/* if this is a local lock, then there is no import */
+	if (obd)
+		imp = obd->u.cli.cl_import;
+
+	timeout = ldlm_cp_timeout(lock);
+
+	lock->l_last_activity = ktime_get_real_seconds();
+
+	if (imp) {
+		spin_lock(&imp->imp_lock);
+		conn_cnt = imp->imp_conn_cnt;
+		spin_unlock(&imp->imp_lock);
+	}
+	if (OBD_FAIL_CHECK_RESET(OBD_FAIL_LDLM_INTR_CP_AST,
+				 OBD_FAIL_LDLM_CP_BL_RACE | OBD_FAIL_ONCE)) {
+		ldlm_set_fail_loc(lock);
+		rc = -EINTR;
+	} else {
+		/* Go to sleep until the lock is granted or canceled. */
+		if (!ldlm_is_no_timeout(lock)) {
+			/* Wait uninterruptible for a while first */
+			rc = wait_event_idle_timeout(lock->l_waitq,
+						     is_granted_or_cancelled(lock),
+						     timeout * HZ);
+			if (rc == 0)
+				ldlm_expired_completion_wait(lock, conn_cnt);
+		}
+		/* Now wait abortable */
+		if (rc == 0)
+			rc = l_wait_event_abortable(lock->l_waitq,
+						    is_granted_or_cancelled(lock));
+		else
+			rc = 0;
+	}
+
+	if (rc) {
+		LDLM_DEBUG(lock, "client-side enqueue waking up: failed (%d)",
+			   rc);
+		return rc;
+	}
+
+	return ldlm_completion_tail(lock, data);
+}
+EXPORT_SYMBOL(ldlm_completion_ast);
+
+static void failed_lock_cleanup(struct ldlm_namespace *ns,
+				struct ldlm_lock *lock, int mode)
+{
+	int need_cancel = 0;
+
+	/* Set a flag to prevent us from sending a CANCEL (bug 407) */
+	lock_res_and_lock(lock);
+	/* Check that lock is not granted or failed, we might race. */
+	if ((lock->l_req_mode != lock->l_granted_mode) &&
+	    !ldlm_is_failed(lock)) {
+		/* Make sure that this lock will not be found by raced
+		 * bl_ast and -EINVAL reply is sent to server anyways.
+		 * bug 17645
+		 */
+		lock->l_flags |= LDLM_FL_LOCAL_ONLY | LDLM_FL_FAILED |
+				 LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING;
+		need_cancel = 1;
+	}
+	unlock_res_and_lock(lock);
+
+	if (need_cancel)
+		LDLM_DEBUG(lock,
+			   "setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING");
+	else
+		LDLM_DEBUG(lock, "lock was granted or failed in race");
+
+	/* XXX - HACK because we shouldn't call ldlm_lock_destroy()
+	 *       from llite/file.c/ll_file_flock().
+	 */
+	/* This code makes for the fact that we do not have blocking handler on
+	 * a client for flock locks. As such this is the place where we must
+	 * completely kill failed locks. (interrupted and those that
+	 * were waiting to be granted when server evicted us.
+	 */
+	if (lock->l_resource->lr_type == LDLM_FLOCK) {
+		lock_res_and_lock(lock);
+		if (!ldlm_is_destroyed(lock)) {
+			ldlm_resource_unlink_lock(lock);
+			ldlm_lock_decref_internal_nolock(lock, mode);
+			ldlm_lock_destroy_nolock(lock);
+		}
+		unlock_res_and_lock(lock);
+	} else {
+		ldlm_lock_decref_internal(lock, mode);
+	}
+}
+
+/**
+ * Finishing portion of client lock enqueue code.
+ *
+ * Called after receiving reply from server.
+ */
+int ldlm_cli_enqueue_fini(struct obd_export *exp, struct ptlrpc_request *req,
+			  enum ldlm_type type, __u8 with_policy,
+			  enum ldlm_mode mode,
+			  __u64 *flags, void *lvb, __u32 lvb_len,
+			  const struct lustre_handle *lockh, int rc)
+{
+	struct ldlm_namespace *ns = exp->exp_obd->obd_namespace;
+	int is_replay = *flags & LDLM_FL_REPLAY;
+	struct ldlm_lock *lock;
+	struct ldlm_reply *reply;
+	int cleanup_phase = 1;
+
+	lock = ldlm_handle2lock(lockh);
+	/* ldlm_cli_enqueue is holding a reference on this lock. */
+	if (!lock) {
+		LASSERT(type == LDLM_FLOCK);
+		return -ENOLCK;
+	}
+
+	LASSERTF(ergo(lvb_len != 0, lvb_len == lock->l_lvb_len),
+		 "lvb_len = %d, l_lvb_len = %d\n", lvb_len, lock->l_lvb_len);
+
+	if (rc != ELDLM_OK) {
+		LASSERT(!is_replay);
+		LDLM_DEBUG(lock, "client-side enqueue END (%s)",
+			   rc == ELDLM_LOCK_ABORTED ? "ABORTED" : "FAILED");
+
+		if (rc != ELDLM_LOCK_ABORTED)
+			goto cleanup;
+	}
+
+	/* Before we return, swab the reply */
+	reply = req_capsule_server_get(&req->rq_pill, &RMF_DLM_REP);
+	if (!reply) {
+		rc = -EPROTO;
+		goto cleanup;
+	}
+
+	if (lvb_len > 0) {
+		int size = 0;
+
+		size = req_capsule_get_size(&req->rq_pill, &RMF_DLM_LVB,
+					    RCL_SERVER);
+		if (size < 0) {
+			LDLM_ERROR(lock, "Fail to get lvb_len, rc = %d", size);
+			rc = size;
+			goto cleanup;
+		} else if (unlikely(size > lvb_len)) {
+			LDLM_ERROR(lock,
+				   "Replied LVB is larger than expectation, expected = %d, replied = %d",
+				   lvb_len, size);
+			rc = -EINVAL;
+			goto cleanup;
+		}
+		lvb_len = size;
+	}
+
+	if (rc == ELDLM_LOCK_ABORTED) {
+		if (lvb_len > 0 && lvb)
+			rc = ldlm_fill_lvb(lock, &req->rq_pill, RCL_SERVER,
+					   lvb, lvb_len);
+		if (rc == 0)
+			rc = ELDLM_LOCK_ABORTED;
+		goto cleanup;
+	}
+
+	/* lock enqueued on the server */
+	cleanup_phase = 0;
+
+	lock_res_and_lock(lock);
+	lock->l_remote_handle = reply->lock_handle;
+
+	*flags = ldlm_flags_from_wire(reply->lock_flags);
+	lock->l_flags |= ldlm_flags_from_wire(reply->lock_flags &
+					      LDLM_FL_INHERIT_MASK);
+	unlock_res_and_lock(lock);
+
+	CDEBUG(D_INFO, "local: %p, remote cookie: %#llx, flags: 0x%llx\n",
+	       lock, reply->lock_handle.cookie, *flags);
+
+	/* If enqueue returned a blocked lock but the completion handler has
+	 * already run, then it fixed up the resource and we don't need to do it
+	 * again.
+	 */
+	if ((*flags) & LDLM_FL_LOCK_CHANGED) {
+		int newmode = reply->lock_desc.l_req_mode;
+
+		LASSERT(!is_replay);
+		if (newmode && newmode != lock->l_req_mode) {
+			LDLM_DEBUG(lock, "server returned different mode %s",
+				   ldlm_lockname[newmode]);
+			lock->l_req_mode = newmode;
+		}
+
+		if (!ldlm_res_eq(&reply->lock_desc.l_resource.lr_name,
+				 &lock->l_resource->lr_name)) {
+			CDEBUG(D_INFO,
+			       "remote intent success, locking " DLDLMRES " instead of " DLDLMRES "\n",
+			       PLDLMRES(&reply->lock_desc.l_resource),
+			       PLDLMRES(lock->l_resource));
+
+			rc = ldlm_lock_change_resource(ns, lock,
+					&reply->lock_desc.l_resource.lr_name);
+			if (rc || !lock->l_resource) {
+				rc = -ENOMEM;
+				goto cleanup;
+			}
+			LDLM_DEBUG(lock, "client-side enqueue, new resource");
+		}
+		if (with_policy)
+			if (!(type == LDLM_IBITS &&
+			      !(exp_connect_flags(exp) & OBD_CONNECT_IBITS)))
+				/* We assume lock type cannot change on server*/
+				ldlm_convert_policy_to_local(exp,
+						lock->l_resource->lr_type,
+						&reply->lock_desc.l_policy_data,
+						&lock->l_policy_data);
+		if (type != LDLM_PLAIN)
+			LDLM_DEBUG(lock,
+				   "client-side enqueue, new policy data");
+	}
+
+	if ((*flags) & LDLM_FL_AST_SENT) {
+		lock_res_and_lock(lock);
+		lock->l_flags |= LDLM_FL_CBPENDING |  LDLM_FL_BL_AST;
+		unlock_res_and_lock(lock);
+		LDLM_DEBUG(lock, "enqueue reply includes blocking AST");
+	}
+
+	/* If the lock has already been granted by a completion AST, don't
+	 * clobber the LVB with an older one.
+	 */
+	if (lvb_len > 0) {
+		/* We must lock or a racing completion might update lvb without
+		 * letting us know and we'll clobber the correct value.
+		 * Cannot unlock after the check either, as that still leaves
+		 * a tiny window for completion to get in
+		 */
+		lock_res_and_lock(lock);
+		if (lock->l_req_mode != lock->l_granted_mode)
+			rc = ldlm_fill_lvb(lock, &req->rq_pill, RCL_SERVER,
+					   lock->l_lvb_data, lvb_len);
+		unlock_res_and_lock(lock);
+		if (rc < 0) {
+			cleanup_phase = 1;
+			goto cleanup;
+		}
+	}
+
+	if (!is_replay) {
+		rc = ldlm_lock_enqueue(ns, &lock, NULL, flags);
+		if (lock->l_completion_ast) {
+			int err = lock->l_completion_ast(lock, *flags, NULL);
+
+			if (!rc)
+				rc = err;
+			if (rc)
+				cleanup_phase = 1;
+		}
+	}
+
+	if (lvb_len > 0 && lvb) {
+		/* Copy the LVB here, and not earlier, because the completion
+		 * AST (if any) can override what we got in the reply
+		 */
+		memcpy(lvb, lock->l_lvb_data, lvb_len);
+	}
+
+	LDLM_DEBUG(lock, "client-side enqueue END");
+cleanup:
+	if (cleanup_phase == 1 && rc)
+		failed_lock_cleanup(ns, lock, mode);
+	/* Put lock 2 times, the second reference is held by ldlm_cli_enqueue */
+	LDLM_LOCK_PUT(lock);
+	LDLM_LOCK_RELEASE(lock);
+	return rc;
+}
+EXPORT_SYMBOL(ldlm_cli_enqueue_fini);
+
+/**
+ * Estimate number of lock handles that would fit into request of given
+ * size.  PAGE_SIZE-512 is to allow TCP/IP and LNET headers to fit into
+ * a single page on the send/receive side. XXX: 512 should be changed to
+ * more adequate value.
+ */
+static inline int ldlm_req_handles_avail(int req_size, int off)
+{
+	int avail;
+
+	avail = min_t(int, LDLM_MAXREQSIZE, PAGE_SIZE - 512) - req_size;
+	if (likely(avail >= 0))
+		avail /= (int)sizeof(struct lustre_handle);
+	else
+		avail = 0;
+	avail += LDLM_LOCKREQ_HANDLES - off;
+
+	return avail;
+}
+
+static inline int ldlm_capsule_handles_avail(struct req_capsule *pill,
+					     enum req_location loc,
+					     int off)
+{
+	u32 size = req_capsule_msg_size(pill, loc);
+
+	return ldlm_req_handles_avail(size, off);
+}
+
+static inline int ldlm_format_handles_avail(struct obd_import *imp,
+					    const struct req_format *fmt,
+					    enum req_location loc, int off)
+{
+	u32 size = req_capsule_fmt_size(imp->imp_msg_magic, fmt, loc);
+
+	return ldlm_req_handles_avail(size, off);
+}
+
+/**
+ * Cancel LRU locks and pack them into the enqueue request. Pack there the given
+ * \a count locks in \a cancels.
+ *
+ * This is to be called by functions preparing their own requests that
+ * might contain lists of locks to cancel in addition to actual operation
+ * that needs to be performed.
+ */
+int ldlm_prep_elc_req(struct obd_export *exp, struct ptlrpc_request *req,
+		      int version, int opc, int canceloff,
+		      struct list_head *cancels, int count)
+{
+	struct ldlm_namespace   *ns = exp->exp_obd->obd_namespace;
+	struct req_capsule      *pill = &req->rq_pill;
+	struct ldlm_request     *dlm = NULL;
+	int flags, avail, to_free, pack = 0;
+	LIST_HEAD(head);
+	int rc;
+
+	if (!cancels)
+		cancels = &head;
+	if (ns_connect_cancelset(ns)) {
+		/* Estimate the amount of available space in the request. */
+		req_capsule_filled_sizes(pill, RCL_CLIENT);
+		avail = ldlm_capsule_handles_avail(pill, RCL_CLIENT, canceloff);
+
+		flags = ns_connect_lru_resize(ns) ?
+			LDLM_LRU_FLAG_LRUR_NO_WAIT : LDLM_LRU_FLAG_AGED;
+		to_free = !ns_connect_lru_resize(ns) &&
+			  opc == LDLM_ENQUEUE ? 1 : 0;
+
+		/* Cancel LRU locks here _only_ if the server supports
+		 * EARLY_CANCEL. Otherwise we have to send extra CANCEL
+		 * RPC, which will make us slower.
+		 */
+		if (avail > count)
+			count += ldlm_cancel_lru_local(ns, cancels, to_free,
+						       avail - count, 0, flags);
+		if (avail > count)
+			pack = count;
+		else
+			pack = avail;
+		req_capsule_set_size(pill, &RMF_DLM_REQ, RCL_CLIENT,
+				     ldlm_request_bufsize(pack, opc));
+	}
+
+	rc = ptlrpc_request_pack(req, version, opc);
+	if (rc) {
+		ldlm_lock_list_put(cancels, l_bl_ast, count);
+		return rc;
+	}
+
+	if (ns_connect_cancelset(ns)) {
+		if (canceloff) {
+			dlm = req_capsule_client_get(pill, &RMF_DLM_REQ);
+			LASSERT(dlm);
+			/* Skip first lock handler in ldlm_request_pack(),
+			 * this method will increment @lock_count according
+			 * to the lock handle amount actually written to
+			 * the buffer.
+			 */
+			dlm->lock_count = canceloff;
+		}
+		/* Pack into the request @pack lock handles. */
+		ldlm_cli_cancel_list(cancels, pack, req, 0);
+		/* Prepare and send separate cancel RPC for others. */
+		ldlm_cli_cancel_list(cancels, count - pack, NULL, 0);
+	} else {
+		ldlm_lock_list_put(cancels, l_bl_ast, count);
+	}
+	return 0;
+}
+EXPORT_SYMBOL(ldlm_prep_elc_req);
+
+int ldlm_prep_enqueue_req(struct obd_export *exp, struct ptlrpc_request *req,
+			  struct list_head *cancels, int count)
+{
+	return ldlm_prep_elc_req(exp, req, LUSTRE_DLM_VERSION, LDLM_ENQUEUE,
+				 LDLM_ENQUEUE_CANCEL_OFF, cancels, count);
+}
+EXPORT_SYMBOL(ldlm_prep_enqueue_req);
+
+static struct ptlrpc_request *ldlm_enqueue_pack(struct obd_export *exp,
+						int lvb_len)
+{
+	struct ptlrpc_request *req;
+	int rc;
+
+	req = ptlrpc_request_alloc(class_exp2cliimp(exp), &RQF_LDLM_ENQUEUE);
+	if (!req)
+		return ERR_PTR(-ENOMEM);
+
+	rc = ldlm_prep_enqueue_req(exp, req, NULL, 0);
+	if (rc) {
+		ptlrpc_request_free(req);
+		return ERR_PTR(rc);
+	}
+
+	req_capsule_set_size(&req->rq_pill, &RMF_DLM_LVB, RCL_SERVER, lvb_len);
+	ptlrpc_request_set_replen(req);
+	return req;
+}
+
+/**
+ * Client-side lock enqueue.
+ *
+ * If a request has some specific initialisation it is passed in \a reqp,
+ * otherwise it is created in ldlm_cli_enqueue.
+ *
+ * Supports sync and async requests, pass \a async flag accordingly. If a
+ * request was created in ldlm_cli_enqueue and it is the async request,
+ * pass it to the caller in \a reqp.
+ */
+int ldlm_cli_enqueue(struct obd_export *exp, struct ptlrpc_request **reqp,
+		     struct ldlm_enqueue_info *einfo,
+		     const struct ldlm_res_id *res_id,
+		     union ldlm_policy_data const *policy, __u64 *flags,
+		     void *lvb, __u32 lvb_len, enum lvb_type lvb_type,
+		     struct lustre_handle *lockh, int async)
+{
+	struct ldlm_namespace *ns;
+	struct ldlm_lock      *lock;
+	struct ldlm_request   *body;
+	int		    is_replay = *flags & LDLM_FL_REPLAY;
+	int		    req_passed_in = 1;
+	int		    rc, err;
+	struct ptlrpc_request *req;
+
+	ns = exp->exp_obd->obd_namespace;
+
+	/* If we're replaying this lock, just check some invariants.
+	 * If we're creating a new lock, get everything all setup nicely.
+	 */
+	if (is_replay) {
+		lock = ldlm_handle2lock_long(lockh, 0);
+		LASSERT(lock);
+		LDLM_DEBUG(lock, "client-side enqueue START");
+		LASSERT(exp == lock->l_conn_export);
+	} else {
+		const struct ldlm_callback_suite cbs = {
+			.lcs_completion = einfo->ei_cb_cp,
+			.lcs_blocking	= einfo->ei_cb_bl,
+			.lcs_glimpse	= einfo->ei_cb_gl
+		};
+		lock = ldlm_lock_create(ns, res_id, einfo->ei_type,
+					einfo->ei_mode, &cbs, einfo->ei_cbdata,
+					lvb_len, lvb_type);
+		if (IS_ERR(lock))
+			return PTR_ERR(lock);
+		/* for the local lock, add the reference */
+		ldlm_lock_addref_internal(lock, einfo->ei_mode);
+		ldlm_lock2handle(lock, lockh);
+		if (policy)
+			lock->l_policy_data = *policy;
+
+		if (einfo->ei_type == LDLM_EXTENT) {
+			/* extent lock without policy is a bug */
+			if (!policy)
+				LBUG();
+
+			lock->l_req_extent = policy->l_extent;
+		}
+		LDLM_DEBUG(lock, "client-side enqueue START, flags %llx",
+			   *flags);
+	}
+
+	lock->l_conn_export = exp;
+	lock->l_export = NULL;
+	lock->l_blocking_ast = einfo->ei_cb_bl;
+	lock->l_flags |= (*flags & (LDLM_FL_NO_LRU | LDLM_FL_EXCL));
+	lock->l_last_activity = ktime_get_real_seconds();
+
+	/* lock not sent to server yet */
+	if (!reqp || !*reqp) {
+		req = ldlm_enqueue_pack(exp, lvb_len);
+		if (IS_ERR(req)) {
+			failed_lock_cleanup(ns, lock, einfo->ei_mode);
+			LDLM_LOCK_RELEASE(lock);
+			return PTR_ERR(req);
+		}
+
+		req_passed_in = 0;
+		if (reqp)
+			*reqp = req;
+	} else {
+		int len;
+
+		req = *reqp;
+		len = req_capsule_get_size(&req->rq_pill, &RMF_DLM_REQ,
+					   RCL_CLIENT);
+		LASSERTF(len >= sizeof(*body), "buflen[%d] = %d, not %d\n",
+			 DLM_LOCKREQ_OFF, len, (int)sizeof(*body));
+	}
+
+	/* Dump lock data into the request buffer */
+	body = req_capsule_client_get(&req->rq_pill, &RMF_DLM_REQ);
+	ldlm_lock2desc(lock, &body->lock_desc);
+	body->lock_flags = ldlm_flags_to_wire(*flags);
+	body->lock_handle[0] = *lockh;
+
+	if (async) {
+		LASSERT(reqp);
+		return 0;
+	}
+
+	LDLM_DEBUG(lock, "sending request");
+
+	rc = ptlrpc_queue_wait(req);
+
+	err = ldlm_cli_enqueue_fini(exp, req, einfo->ei_type, policy ? 1 : 0,
+				    einfo->ei_mode, flags, lvb, lvb_len,
+				    lockh, rc);
+
+	/* If ldlm_cli_enqueue_fini did not find the lock, we need to free
+	 * one reference that we took
+	 */
+	if (err == -ENOLCK)
+		LDLM_LOCK_RELEASE(lock);
+	else
+		rc = err;
+
+	if (!req_passed_in && req) {
+		ptlrpc_req_finished(req);
+		if (reqp)
+			*reqp = NULL;
+	}
+
+	return rc;
+}
+EXPORT_SYMBOL(ldlm_cli_enqueue);
+
+/**
+ * Cancel locks locally.
+ * Returns:
+ * \retval LDLM_FL_LOCAL_ONLY if there is no need for a CANCEL RPC to the server
+ * \retval LDLM_FL_CANCELING otherwise;
+ * \retval LDLM_FL_BL_AST if there is a need for a separate CANCEL RPC.
+ */
+static __u64 ldlm_cli_cancel_local(struct ldlm_lock *lock)
+{
+	__u64 rc = LDLM_FL_LOCAL_ONLY;
+
+	if (lock->l_conn_export) {
+		bool local_only;
+
+		LDLM_DEBUG(lock, "client-side cancel");
+		/* Set this flag to prevent others from getting new references*/
+		lock_res_and_lock(lock);
+		ldlm_set_cbpending(lock);
+		local_only = !!(lock->l_flags &
+				(LDLM_FL_LOCAL_ONLY | LDLM_FL_CANCEL_ON_BLOCK));
+		ldlm_cancel_callback(lock);
+		rc = ldlm_is_bl_ast(lock) ? LDLM_FL_BL_AST : LDLM_FL_CANCELING;
+		unlock_res_and_lock(lock);
+
+		if (local_only) {
+			CDEBUG(D_DLMTRACE,
+			       "not sending request (at caller's instruction)\n");
+			rc = LDLM_FL_LOCAL_ONLY;
+		}
+		ldlm_lock_cancel(lock);
+	} else {
+		LDLM_ERROR(lock, "Trying to cancel local lock");
+		LBUG();
+	}
+
+	return rc;
+}
+
+/**
+ * Pack \a count locks in \a head into ldlm_request buffer of request \a req.
+ */
+static void ldlm_cancel_pack(struct ptlrpc_request *req,
+			     struct list_head *head, int count)
+{
+	struct ldlm_request *dlm;
+	struct ldlm_lock *lock;
+	int max, packed = 0;
+
+	dlm = req_capsule_client_get(&req->rq_pill, &RMF_DLM_REQ);
+	LASSERT(dlm);
+
+	/* Check the room in the request buffer. */
+	max = req_capsule_get_size(&req->rq_pill, &RMF_DLM_REQ, RCL_CLIENT) -
+		sizeof(struct ldlm_request);
+	max /= sizeof(struct lustre_handle);
+	max += LDLM_LOCKREQ_HANDLES;
+	LASSERT(max >= dlm->lock_count + count);
+
+	/* XXX: it would be better to pack lock handles grouped by resource.
+	 * so that the server cancel would call filter_lvbo_update() less
+	 * frequently.
+	 */
+	list_for_each_entry(lock, head, l_bl_ast) {
+		if (!count--)
+			break;
+		LASSERT(lock->l_conn_export);
+		/* Pack the lock handle to the given request buffer. */
+		LDLM_DEBUG(lock, "packing");
+		dlm->lock_handle[dlm->lock_count++] = lock->l_remote_handle;
+		packed++;
+	}
+	CDEBUG(D_DLMTRACE, "%d locks packed\n", packed);
+}
+
+/**
+ * Prepare and send a batched cancel RPC. It will include \a count lock
+ * handles of locks given in \a cancels list.
+ */
+static int ldlm_cli_cancel_req(struct obd_export *exp,
+			       struct list_head *cancels,
+			       int count, enum ldlm_cancel_flags flags)
+{
+	struct ptlrpc_request *req = NULL;
+	struct obd_import *imp;
+	int free, sent = 0;
+	int rc = 0;
+
+	LASSERT(exp);
+	LASSERT(count > 0);
+
+	CFS_FAIL_TIMEOUT(OBD_FAIL_LDLM_PAUSE_CANCEL, cfs_fail_val);
+
+	if (CFS_FAIL_CHECK(OBD_FAIL_LDLM_CANCEL_RACE))
+		return count;
+
+	free = ldlm_format_handles_avail(class_exp2cliimp(exp),
+					 &RQF_LDLM_CANCEL, RCL_CLIENT, 0);
+	if (count > free)
+		count = free;
+
+	while (1) {
+		imp = class_exp2cliimp(exp);
+		if (!imp || imp->imp_invalid) {
+			CDEBUG(D_DLMTRACE,
+			       "skipping cancel on invalid import %p\n", imp);
+			return count;
+		}
+
+		req = ptlrpc_request_alloc(imp, &RQF_LDLM_CANCEL);
+		if (!req) {
+			rc = -ENOMEM;
+			goto out;
+		}
+
+		req_capsule_filled_sizes(&req->rq_pill, RCL_CLIENT);
+		req_capsule_set_size(&req->rq_pill, &RMF_DLM_REQ, RCL_CLIENT,
+				     ldlm_request_bufsize(count, LDLM_CANCEL));
+
+		rc = ptlrpc_request_pack(req, LUSTRE_DLM_VERSION, LDLM_CANCEL);
+		if (rc) {
+			ptlrpc_request_free(req);
+			goto out;
+		}
+
+		req->rq_request_portal = LDLM_CANCEL_REQUEST_PORTAL;
+		req->rq_reply_portal = LDLM_CANCEL_REPLY_PORTAL;
+		ptlrpc_at_set_req_timeout(req);
+
+		ldlm_cancel_pack(req, cancels, count);
+
+		ptlrpc_request_set_replen(req);
+		if (flags & LCF_ASYNC) {
+			ptlrpcd_add_req(req);
+			sent = count;
+			goto out;
+		}
+
+		rc = ptlrpc_queue_wait(req);
+		if (rc == LUSTRE_ESTALE) {
+			CDEBUG(D_DLMTRACE,
+			       "client/server (nid %s) out of sync -- not fatal\n",
+			       libcfs_nid2str(req->rq_import->
+					      imp_connection->c_peer.nid));
+			rc = 0;
+		} else if (rc == -ETIMEDOUT && /* check there was no reconnect*/
+			   req->rq_import_generation == imp->imp_generation) {
+			ptlrpc_req_finished(req);
+			continue;
+		} else if (rc != ELDLM_OK) {
+			/* -ESHUTDOWN is common on umount */
+			CDEBUG_LIMIT(rc == -ESHUTDOWN ? D_DLMTRACE : D_ERROR,
+				     "Got rc %d from cancel RPC: canceling anyway\n",
+				     rc);
+			break;
+		}
+		sent = count;
+		break;
+	}
+
+	ptlrpc_req_finished(req);
+out:
+	return sent ? sent : rc;
+}
+
+static inline struct ldlm_pool *ldlm_imp2pl(struct obd_import *imp)
+{
+	return &imp->imp_obd->obd_namespace->ns_pool;
+}
+
+/**
+ * Update client's OBD pool related fields with new SLV and Limit from \a req.
+ */
+int ldlm_cli_update_pool(struct ptlrpc_request *req)
+{
+	struct obd_device *obd;
+	__u64 new_slv;
+	__u32 new_limit;
+
+	if (unlikely(!req->rq_import || !req->rq_import->imp_obd ||
+		     !imp_connect_lru_resize(req->rq_import))) {
+		/*
+		 * Do nothing for corner cases.
+		 */
+		return 0;
+	}
+
+	/* In some cases RPC may contain SLV and limit zeroed out. This
+	 * is the case when server does not support LRU resize feature.
+	 * This is also possible in some recovery cases when server-side
+	 * reqs have no reference to the OBD export and thus access to
+	 * server-side namespace is not possible.
+	 */
+	if (lustre_msg_get_slv(req->rq_repmsg) == 0 ||
+	    lustre_msg_get_limit(req->rq_repmsg) == 0) {
+		DEBUG_REQ(D_HA, req,
+			  "Zero SLV or Limit found (SLV: %llu, Limit: %u)",
+			  lustre_msg_get_slv(req->rq_repmsg),
+			  lustre_msg_get_limit(req->rq_repmsg));
+		return 0;
+	}
+
+	new_limit = lustre_msg_get_limit(req->rq_repmsg);
+	new_slv = lustre_msg_get_slv(req->rq_repmsg);
+	obd = req->rq_import->imp_obd;
+
+	/* Set new SLV and limit in OBD fields to make them accessible
+	 * to the pool thread. We do not access obd_namespace and pool
+	 * directly here as there is no reliable way to make sure that
+	 * they are still alive at cleanup time. Evil races are possible
+	 * which may cause Oops at that time.
+	 */
+	write_lock(&obd->obd_pool_lock);
+	obd->obd_pool_slv = new_slv;
+	obd->obd_pool_limit = new_limit;
+	write_unlock(&obd->obd_pool_lock);
+
+	return 0;
+}
+
+/**
+ * Client side lock cancel.
+ *
+ * Lock must not have any readers or writers by this time.
+ */
+int ldlm_cli_cancel(const struct lustre_handle *lockh,
+		    enum ldlm_cancel_flags cancel_flags)
+{
+	struct obd_export *exp;
+	int avail, flags, count = 1;
+	__u64 rc = 0;
+	struct ldlm_namespace *ns;
+	struct ldlm_lock *lock;
+	LIST_HEAD(cancels);
+
+	lock = ldlm_handle2lock_long(lockh, 0);
+	if (!lock) {
+		LDLM_DEBUG_NOLOCK("lock is already being destroyed");
+		return 0;
+	}
+
+	lock_res_and_lock(lock);
+	/* Lock is being canceled and the caller doesn't want to wait */
+	if (ldlm_is_canceling(lock) && (cancel_flags & LCF_ASYNC)) {
+		unlock_res_and_lock(lock);
+		LDLM_LOCK_RELEASE(lock);
+		return 0;
+	}
+
+	ldlm_set_canceling(lock);
+	unlock_res_and_lock(lock);
+
+	rc = ldlm_cli_cancel_local(lock);
+	if (rc == LDLM_FL_LOCAL_ONLY || cancel_flags & LCF_LOCAL) {
+		LDLM_LOCK_RELEASE(lock);
+		return 0;
+	}
+	/* Even if the lock is marked as LDLM_FL_BL_AST, this is a LDLM_CANCEL
+	 * RPC which goes to canceld portal, so we can cancel other LRU locks
+	 * here and send them all as one LDLM_CANCEL RPC.
+	 */
+	LASSERT(list_empty(&lock->l_bl_ast));
+	list_add(&lock->l_bl_ast, &cancels);
+
+	exp = lock->l_conn_export;
+	if (exp_connect_cancelset(exp)) {
+		avail = ldlm_format_handles_avail(class_exp2cliimp(exp),
+						  &RQF_LDLM_CANCEL,
+						  RCL_CLIENT, 0);
+		LASSERT(avail > 0);
+
+		ns = ldlm_lock_to_ns(lock);
+		flags = ns_connect_lru_resize(ns) ?
+			LDLM_LRU_FLAG_LRUR : LDLM_LRU_FLAG_AGED;
+		count += ldlm_cancel_lru_local(ns, &cancels, 0, avail - 1,
+					       LCF_BL_AST, flags);
+	}
+	ldlm_cli_cancel_list(&cancels, count, NULL, cancel_flags);
+	return 0;
+}
+EXPORT_SYMBOL(ldlm_cli_cancel);
+
+/**
+ * Locally cancel up to \a count locks in list \a cancels.
+ * Return the number of cancelled locks.
+ */
+int ldlm_cli_cancel_list_local(struct list_head *cancels, int count,
+			       enum ldlm_cancel_flags flags)
+{
+	LIST_HEAD(head);
+	struct ldlm_lock *lock, *next;
+	int left = 0, bl_ast = 0;
+	__u64 rc;
+
+	left = count;
+	list_for_each_entry_safe(lock, next, cancels, l_bl_ast) {
+		if (left-- == 0)
+			break;
+
+		if (flags & LCF_LOCAL) {
+			rc = LDLM_FL_LOCAL_ONLY;
+			ldlm_lock_cancel(lock);
+		} else {
+			rc = ldlm_cli_cancel_local(lock);
+		}
+		/* Until we have compound requests and can send LDLM_CANCEL
+		 * requests batched with generic RPCs, we need to send cancels
+		 * with the LDLM_FL_BL_AST flag in a separate RPC from
+		 * the one being generated now.
+		 */
+		if (!(flags & LCF_BL_AST) && (rc == LDLM_FL_BL_AST)) {
+			LDLM_DEBUG(lock, "Cancel lock separately");
+			list_del_init(&lock->l_bl_ast);
+			list_add(&lock->l_bl_ast, &head);
+			bl_ast++;
+			continue;
+		}
+		if (rc == LDLM_FL_LOCAL_ONLY) {
+			/* CANCEL RPC should not be sent to server. */
+			list_del_init(&lock->l_bl_ast);
+			LDLM_LOCK_RELEASE(lock);
+			count--;
+		}
+	}
+	if (bl_ast > 0) {
+		count -= bl_ast;
+		ldlm_cli_cancel_list(&head, bl_ast, NULL, 0);
+	}
+
+	return count;
+}
+
+/**
+ * Cancel as many locks as possible w/o sending any RPCs (e.g. to write back
+ * dirty data, to close a file, ...) or waiting for any RPCs in-flight (e.g.
+ * readahead requests, ...)
+ */
+static enum ldlm_policy_res
+ldlm_cancel_no_wait_policy(struct ldlm_namespace *ns, struct ldlm_lock *lock,
+			   int unused, int added, int count)
+{
+	enum ldlm_policy_res result = LDLM_POLICY_CANCEL_LOCK;
+
+	/* don't check added & count since we want to process all locks
+	 * from unused list.
+	 * It's fine to not take lock to access lock->l_resource since
+	 * the lock has already been granted so it won't change.
+	 */
+	switch (lock->l_resource->lr_type) {
+	case LDLM_EXTENT:
+	case LDLM_IBITS:
+		if (ns->ns_cancel && ns->ns_cancel(lock) != 0)
+			break;
+		/* fall through */
+	default:
+		result = LDLM_POLICY_SKIP_LOCK;
+		lock_res_and_lock(lock);
+		ldlm_set_skipped(lock);
+		unlock_res_and_lock(lock);
+		break;
+	}
+
+	return result;
+}
+
+/**
+ * Callback function for LRU-resize policy. Decides whether to keep
+ * \a lock in LRU for current \a LRU size \a unused, added in current
+ * scan \a added and number of locks to be preferably canceled \a count.
+ *
+ * \retval LDLM_POLICY_KEEP_LOCK keep lock in LRU in stop scanning
+ *
+ * \retval LDLM_POLICY_CANCEL_LOCK cancel lock from LRU
+ */
+static enum ldlm_policy_res ldlm_cancel_lrur_policy(struct ldlm_namespace *ns,
+						    struct ldlm_lock *lock,
+						    int unused, int added,
+						    int count)
+{
+	unsigned long cur = jiffies;
+	struct ldlm_pool *pl = &ns->ns_pool;
+	__u64 slv, lvf, lv;
+	unsigned long la;
+
+	/* Stop LRU processing when we reach past @count or have checked all
+	 * locks in LRU.
+	 */
+	if (count && added >= count)
+		return LDLM_POLICY_KEEP_LOCK;
+
+	/*
+	 * Despite of the LV, It doesn't make sense to keep the lock which
+	 * is unused for ns_max_age time.
+	 */
+	if (time_after(jiffies, lock->l_last_used + ns->ns_max_age))
+		return LDLM_POLICY_CANCEL_LOCK;
+
+	slv = ldlm_pool_get_slv(pl);
+	lvf = ldlm_pool_get_lvf(pl);
+	la = (cur - lock->l_last_used) / HZ;
+	lv = lvf * la * unused;
+
+	/* Inform pool about current CLV to see it via debugfs. */
+	ldlm_pool_set_clv(pl, lv);
+
+	/* Stop when SLV is not yet come from server or lv is smaller than
+	 * it is.
+	 */
+	if (slv == 0 || lv < slv)
+		return LDLM_POLICY_KEEP_LOCK;
+
+	return LDLM_POLICY_CANCEL_LOCK;
+}
+
+/**
+ * Callback function for debugfs used policy. Makes decision whether to keep
+ * \a lock in LRU for current \a LRU size \a unused, added in current scan \a
+ * added and number of locks to be preferably canceled \a count.
+ *
+ * \retval LDLM_POLICY_KEEP_LOCK keep lock in LRU in stop scanning
+ *
+ * \retval LDLM_POLICY_CANCEL_LOCK cancel lock from LRU
+ */
+static enum ldlm_policy_res ldlm_cancel_passed_policy(struct ldlm_namespace *ns,
+						      struct ldlm_lock *lock,
+						      int unused, int added,
+						      int count)
+{
+	/* Stop LRU processing when we reach past @count or have checked all
+	 * locks in LRU.
+	 */
+	return (added >= count) ?
+		LDLM_POLICY_KEEP_LOCK : LDLM_POLICY_CANCEL_LOCK;
+}
+
+/**
+ * Callback function for aged policy. Makes decision whether to keep \a lock in
+ * LRU for current LRU size \a unused, added in current scan \a added and
+ * number of locks to be preferably canceled \a count.
+ *
+ * \retval LDLM_POLICY_KEEP_LOCK keep lock in LRU in stop scanning
+ *
+ * \retval LDLM_POLICY_CANCEL_LOCK cancel lock from LRU
+ */
+static enum ldlm_policy_res ldlm_cancel_aged_policy(struct ldlm_namespace *ns,
+						    struct ldlm_lock *lock,
+						    int unused, int added,
+						    int count)
+{
+	if ((added >= count) &&
+	    time_before(jiffies, lock->l_last_used + ns->ns_max_age))
+		return LDLM_POLICY_KEEP_LOCK;
+
+	return LDLM_POLICY_CANCEL_LOCK;
+}
+
+static enum ldlm_policy_res
+ldlm_cancel_lrur_no_wait_policy(struct ldlm_namespace *ns,
+				struct ldlm_lock *lock,
+				int unused, int added,
+				int count)
+{
+	enum ldlm_policy_res result;
+
+	result = ldlm_cancel_lrur_policy(ns, lock, unused, added, count);
+	if (result == LDLM_POLICY_KEEP_LOCK)
+		return result;
+
+	return ldlm_cancel_no_wait_policy(ns, lock, unused, added, count);
+}
+
+/**
+ * Callback function for default policy. Makes decision whether to keep \a lock
+ * in LRU for current LRU size \a unused, added in current scan \a added and
+ * number of locks to be preferably canceled \a count.
+ *
+ * \retval LDLM_POLICY_KEEP_LOCK keep lock in LRU in stop scanning
+ *
+ * \retval LDLM_POLICY_CANCEL_LOCK cancel lock from LRU
+ */
+static enum ldlm_policy_res
+ldlm_cancel_default_policy(struct ldlm_namespace *ns, struct ldlm_lock *lock,
+			   int unused, int added, int count)
+{
+	/* Stop LRU processing when we reach past count or have checked all
+	 * locks in LRU.
+	 */
+	return (added >= count) ?
+		LDLM_POLICY_KEEP_LOCK : LDLM_POLICY_CANCEL_LOCK;
+}
+
+typedef enum ldlm_policy_res (*ldlm_cancel_lru_policy_t)(
+						      struct ldlm_namespace *,
+						      struct ldlm_lock *, int,
+						      int, int);
+
+static ldlm_cancel_lru_policy_t
+ldlm_cancel_lru_policy(struct ldlm_namespace *ns, int flags)
+{
+	if (flags & LDLM_LRU_FLAG_NO_WAIT)
+		return ldlm_cancel_no_wait_policy;
+
+	if (ns_connect_lru_resize(ns)) {
+		if (flags & LDLM_LRU_FLAG_SHRINK)
+			/* We kill passed number of old locks. */
+			return ldlm_cancel_passed_policy;
+		else if (flags & LDLM_LRU_FLAG_LRUR)
+			return ldlm_cancel_lrur_policy;
+		else if (flags & LDLM_LRU_FLAG_PASSED)
+			return ldlm_cancel_passed_policy;
+		else if (flags & LDLM_LRU_FLAG_LRUR_NO_WAIT)
+			return ldlm_cancel_lrur_no_wait_policy;
+	} else {
+		if (flags & LDLM_LRU_FLAG_AGED)
+			return ldlm_cancel_aged_policy;
+	}
+
+	return ldlm_cancel_default_policy;
+}
+
+/**
+ * - Free space in LRU for \a count new locks,
+ *   redundant unused locks are canceled locally;
+ * - also cancel locally unused aged locks;
+ * - do not cancel more than \a max locks;
+ * - GET the found locks and add them into the \a cancels list.
+ *
+ * A client lock can be added to the l_bl_ast list only when it is
+ * marked LDLM_FL_CANCELING. Otherwise, somebody is already doing
+ * CANCEL.  There are the following use cases:
+ * ldlm_cancel_resource_local(), ldlm_cancel_lru_local() and
+ * ldlm_cli_cancel(), which check and set this flag properly. As any
+ * attempt to cancel a lock rely on this flag, l_bl_ast list is accessed
+ * later without any special locking.
+ *
+ * Calling policies for enabled LRU resize:
+ * ----------------------------------------
+ * flags & LDLM_LRU_FLAG_LRUR	- use LRU resize policy (SLV from server) to
+ *				  cancel not more than \a count locks;
+ *
+ * flags & LDLM_LRU_FLAG_PASSED - cancel \a count number of old locks (located
+ *				  at the beginning of LRU list);
+ *
+ * flags & LDLM_LRU_FLAG_SHRINK - cancel not more than \a count locks according
+ *				  to memory pressure policy function;
+ *
+ * flags & LDLM_LRU_FLAG_AGED   - cancel \a count locks according to
+ *				  "aged policy".
+ *
+ * flags & LDLM_LRU_FLAG_NO_WAIT - cancel as many unused locks as possible
+ *				   (typically before replaying locks) w/o
+ *				   sending any RPCs or waiting for any
+ *				   outstanding RPC to complete.
+ */
+static int ldlm_prepare_lru_list(struct ldlm_namespace *ns,
+				 struct list_head *cancels, int count, int max,
+				 int flags)
+{
+	ldlm_cancel_lru_policy_t pf;
+	struct ldlm_lock *lock, *next;
+	int added = 0, unused, remained;
+	int no_wait = flags &
+		(LDLM_LRU_FLAG_NO_WAIT | LDLM_LRU_FLAG_LRUR_NO_WAIT);
+
+	spin_lock(&ns->ns_lock);
+	unused = ns->ns_nr_unused;
+	remained = unused;
+
+	if (!ns_connect_lru_resize(ns))
+		count += unused - ns->ns_max_unused;
+
+	pf = ldlm_cancel_lru_policy(ns, flags);
+	LASSERT(pf);
+
+	while (!list_empty(&ns->ns_unused_list)) {
+		enum ldlm_policy_res result;
+		time_t last_use = 0;
+
+		/* all unused locks */
+		if (remained-- <= 0)
+			break;
+
+		/* For any flags, stop scanning if @max is reached. */
+		if (max && added >= max)
+			break;
+
+		list_for_each_entry_safe(lock, next, &ns->ns_unused_list,
+					 l_lru) {
+			/* No locks which got blocking requests. */
+			LASSERT(!ldlm_is_bl_ast(lock));
+
+			if (no_wait && ldlm_is_skipped(lock))
+				/* already processed */
+				continue;
+
+			last_use = lock->l_last_used;
+			if (last_use == jiffies)
+				continue;
+
+			/* Somebody is already doing CANCEL. No need for this
+			 * lock in LRU, do not traverse it again.
+			 */
+			if (!ldlm_is_canceling(lock))
+				break;
+
+			ldlm_lock_remove_from_lru_nolock(lock);
+		}
+		if (&lock->l_lru == &ns->ns_unused_list)
+			break;
+
+		LDLM_LOCK_GET(lock);
+		spin_unlock(&ns->ns_lock);
+		lu_ref_add(&lock->l_reference, __func__, current);
+
+		/* Pass the lock through the policy filter and see if it
+		 * should stay in LRU.
+		 *
+		 * Even for shrinker policy we stop scanning if
+		 * we find a lock that should stay in the cache.
+		 * We should take into account lock age anyway
+		 * as a new lock is a valuable resource even if
+		 * it has a low weight.
+		 *
+		 * That is, for shrinker policy we drop only
+		 * old locks, but additionally choose them by
+		 * their weight. Big extent locks will stay in
+		 * the cache.
+		 */
+		result = pf(ns, lock, unused, added, count);
+		if (result == LDLM_POLICY_KEEP_LOCK) {
+			lu_ref_del(&lock->l_reference,
+				   __func__, current);
+			LDLM_LOCK_RELEASE(lock);
+			spin_lock(&ns->ns_lock);
+			break;
+		}
+		if (result == LDLM_POLICY_SKIP_LOCK) {
+			lu_ref_del(&lock->l_reference,
+				   __func__, current);
+			LDLM_LOCK_RELEASE(lock);
+			spin_lock(&ns->ns_lock);
+			continue;
+		}
+
+		lock_res_and_lock(lock);
+		/* Check flags again under the lock. */
+		if (ldlm_is_canceling(lock) ||
+		    (ldlm_lock_remove_from_lru_check(lock, last_use) == 0)) {
+			/* Another thread is removing lock from LRU, or
+			 * somebody is already doing CANCEL, or there
+			 * is a blocking request which will send cancel
+			 * by itself, or the lock is no longer unused or
+			 * the lock has been used since the pf() call and
+			 * pages could be put under it.
+			 */
+			unlock_res_and_lock(lock);
+			lu_ref_del(&lock->l_reference,
+				   __func__, current);
+			LDLM_LOCK_RELEASE(lock);
+			spin_lock(&ns->ns_lock);
+			continue;
+		}
+		LASSERT(!lock->l_readers && !lock->l_writers);
+
+		/* If we have chosen to cancel this lock voluntarily, we
+		 * better send cancel notification to server, so that it
+		 * frees appropriate state. This might lead to a race
+		 * where while we are doing cancel here, server is also
+		 * silently cancelling this lock.
+		 */
+		ldlm_clear_cancel_on_block(lock);
+
+		/* Setting the CBPENDING flag is a little misleading,
+		 * but prevents an important race; namely, once
+		 * CBPENDING is set, the lock can accumulate no more
+		 * readers/writers. Since readers and writers are
+		 * already zero here, ldlm_lock_decref() won't see
+		 * this flag and call l_blocking_ast
+		 */
+		lock->l_flags |= LDLM_FL_CBPENDING | LDLM_FL_CANCELING;
+
+		/* We can't re-add to l_lru as it confuses the
+		 * refcounting in ldlm_lock_remove_from_lru() if an AST
+		 * arrives after we drop lr_lock below. We use l_bl_ast
+		 * and can't use l_pending_chain as it is used both on
+		 * server and client nevertheless bug 5666 says it is
+		 * used only on server
+		 */
+		LASSERT(list_empty(&lock->l_bl_ast));
+		list_add(&lock->l_bl_ast, cancels);
+		unlock_res_and_lock(lock);
+		lu_ref_del(&lock->l_reference, __func__, current);
+		spin_lock(&ns->ns_lock);
+		added++;
+		unused--;
+	}
+	spin_unlock(&ns->ns_lock);
+	return added;
+}
+
+int ldlm_cancel_lru_local(struct ldlm_namespace *ns,
+			  struct list_head *cancels, int count, int max,
+			  enum ldlm_cancel_flags cancel_flags, int flags)
+{
+	int added;
+
+	added = ldlm_prepare_lru_list(ns, cancels, count, max, flags);
+	if (added <= 0)
+		return added;
+	return ldlm_cli_cancel_list_local(cancels, added, cancel_flags);
+}
+
+/**
+ * Cancel at least \a nr locks from given namespace LRU.
+ *
+ * When called with LCF_ASYNC the blocking callback will be handled
+ * in a thread and this function will return after the thread has been
+ * asked to call the callback.  When called with LCF_ASYNC the blocking
+ * callback will be performed in this function.
+ */
+int ldlm_cancel_lru(struct ldlm_namespace *ns, int nr,
+		    enum ldlm_cancel_flags cancel_flags,
+		    int flags)
+{
+	LIST_HEAD(cancels);
+	int count, rc;
+
+	/* Just prepare the list of locks, do not actually cancel them yet.
+	 * Locks are cancelled later in a separate thread.
+	 */
+	count = ldlm_prepare_lru_list(ns, &cancels, nr, 0, flags);
+	rc = ldlm_bl_to_thread_list(ns, NULL, &cancels, count, cancel_flags);
+	if (rc == 0)
+		return count;
+
+	return 0;
+}
+
+/**
+ * Find and cancel locally unused locks found on resource, matched to the
+ * given policy, mode. GET the found locks and add them into the \a cancels
+ * list.
+ */
+int ldlm_cancel_resource_local(struct ldlm_resource *res,
+			       struct list_head *cancels,
+			       union ldlm_policy_data *policy,
+			       enum ldlm_mode mode, __u64 lock_flags,
+			       enum ldlm_cancel_flags cancel_flags,
+			       void *opaque)
+{
+	struct ldlm_lock *lock;
+	int count = 0;
+
+	lock_res(res);
+	list_for_each_entry(lock, &res->lr_granted, l_res_link) {
+		if (opaque && lock->l_ast_data != opaque) {
+			LDLM_ERROR(lock, "data %p doesn't match opaque %p",
+				   lock->l_ast_data, opaque);
+			continue;
+		}
+
+		if (lock->l_readers || lock->l_writers)
+			continue;
+
+		/* If somebody is already doing CANCEL, or blocking AST came,
+		 * skip this lock.
+		 */
+		if (ldlm_is_bl_ast(lock) || ldlm_is_canceling(lock))
+			continue;
+
+		if (lockmode_compat(lock->l_granted_mode, mode))
+			continue;
+
+		/* If policy is given and this is IBITS lock, add to list only
+		 * those locks that match by policy.
+		 */
+		if (policy && (lock->l_resource->lr_type == LDLM_IBITS) &&
+		    !(lock->l_policy_data.l_inodebits.bits &
+		      policy->l_inodebits.bits))
+			continue;
+
+		/* See CBPENDING comment in ldlm_cancel_lru */
+		lock->l_flags |= LDLM_FL_CBPENDING | LDLM_FL_CANCELING |
+				 lock_flags;
+
+		LASSERT(list_empty(&lock->l_bl_ast));
+		list_add(&lock->l_bl_ast, cancels);
+		LDLM_LOCK_GET(lock);
+		count++;
+	}
+	unlock_res(res);
+
+	return ldlm_cli_cancel_list_local(cancels, count, cancel_flags);
+}
+EXPORT_SYMBOL(ldlm_cancel_resource_local);
+
+/**
+ * Cancel client-side locks from a list and send/prepare cancel RPCs to the
+ * server.
+ * If \a req is NULL, send CANCEL request to server with handles of locks
+ * in the \a cancels. If EARLY_CANCEL is not supported, send CANCEL requests
+ * separately per lock.
+ * If \a req is not NULL, put handles of locks in \a cancels into the request
+ * buffer at the offset \a off.
+ * Destroy \a cancels at the end.
+ */
+int ldlm_cli_cancel_list(struct list_head *cancels, int count,
+			 struct ptlrpc_request *req,
+			 enum ldlm_cancel_flags flags)
+{
+	struct ldlm_lock *lock;
+	int res = 0;
+
+	if (list_empty(cancels) || count == 0)
+		return 0;
+
+	/* XXX: requests (both batched and not) could be sent in parallel.
+	 * Usually it is enough to have just 1 RPC, but it is possible that
+	 * there are too many locks to be cancelled in LRU or on a resource.
+	 * It would also speed up the case when the server does not support
+	 * the feature.
+	 */
+	while (count > 0) {
+		LASSERT(!list_empty(cancels));
+		lock = list_first_entry(cancels, struct ldlm_lock, l_bl_ast);
+		LASSERT(lock->l_conn_export);
+
+		if (exp_connect_cancelset(lock->l_conn_export)) {
+			res = count;
+			if (req)
+				ldlm_cancel_pack(req, cancels, count);
+			else
+				res = ldlm_cli_cancel_req(lock->l_conn_export,
+							  cancels, count,
+							  flags);
+		} else {
+			res = ldlm_cli_cancel_req(lock->l_conn_export,
+						  cancels, 1, flags);
+		}
+
+		if (res < 0) {
+			CDEBUG_LIMIT(res == -ESHUTDOWN ? D_DLMTRACE : D_ERROR,
+				     "%s: %d\n", __func__, res);
+			res = count;
+		}
+
+		count -= res;
+		ldlm_lock_list_put(cancels, l_bl_ast, res);
+	}
+	LASSERT(count == 0);
+	return 0;
+}
+EXPORT_SYMBOL(ldlm_cli_cancel_list);
+
+/**
+ * Cancel all locks on a resource that have 0 readers/writers.
+ *
+ * If flags & LDLM_FL_LOCAL_ONLY, throw the locks away without trying
+ * to notify the server.
+ */
+int ldlm_cli_cancel_unused_resource(struct ldlm_namespace *ns,
+				    const struct ldlm_res_id *res_id,
+				    union ldlm_policy_data *policy,
+				    enum ldlm_mode mode,
+				    enum ldlm_cancel_flags flags,
+				    void *opaque)
+{
+	struct ldlm_resource *res;
+	LIST_HEAD(cancels);
+	int count;
+	int rc;
+
+	res = ldlm_resource_get(ns, NULL, res_id, 0, 0);
+	if (IS_ERR(res)) {
+		/* This is not a problem. */
+		CDEBUG(D_INFO, "No resource %llu\n", res_id->name[0]);
+		return 0;
+	}
+
+	LDLM_RESOURCE_ADDREF(res);
+	count = ldlm_cancel_resource_local(res, &cancels, policy, mode,
+					   0, flags | LCF_BL_AST, opaque);
+	rc = ldlm_cli_cancel_list(&cancels, count, NULL, flags);
+	if (rc != ELDLM_OK)
+		CERROR("canceling unused lock " DLDLMRES ": rc = %d\n",
+		       PLDLMRES(res), rc);
+
+	LDLM_RESOURCE_DELREF(res);
+	ldlm_resource_putref(res);
+	return 0;
+}
+EXPORT_SYMBOL(ldlm_cli_cancel_unused_resource);
+
+struct ldlm_cli_cancel_arg {
+	int     lc_flags;
+	void   *lc_opaque;
+};
+
+static int ldlm_cli_hash_cancel_unused(struct cfs_hash *hs,
+				       struct cfs_hash_bd *bd,
+				       struct hlist_node *hnode, void *arg)
+{
+	struct ldlm_resource	   *res = cfs_hash_object(hs, hnode);
+	struct ldlm_cli_cancel_arg     *lc = arg;
+
+	ldlm_cli_cancel_unused_resource(ldlm_res_to_ns(res), &res->lr_name,
+					NULL, LCK_MINMODE,
+					lc->lc_flags, lc->lc_opaque);
+	/* must return 0 for hash iteration */
+	return 0;
+}
+
+/**
+ * Cancel all locks on a namespace (or a specific resource, if given)
+ * that have 0 readers/writers.
+ *
+ * If flags & LCF_LOCAL, throw the locks away without trying
+ * to notify the server.
+ */
+int ldlm_cli_cancel_unused(struct ldlm_namespace *ns,
+			   const struct ldlm_res_id *res_id,
+			   enum ldlm_cancel_flags flags, void *opaque)
+{
+	struct ldlm_cli_cancel_arg arg = {
+		.lc_flags       = flags,
+		.lc_opaque      = opaque,
+	};
+
+	if (!ns)
+		return ELDLM_OK;
+
+	if (res_id) {
+		return ldlm_cli_cancel_unused_resource(ns, res_id, NULL,
+						       LCK_MINMODE, flags,
+						       opaque);
+	} else {
+		cfs_hash_for_each_nolock(ns->ns_rs_hash,
+					 ldlm_cli_hash_cancel_unused, &arg, 0);
+		return ELDLM_OK;
+	}
+}
+EXPORT_SYMBOL(ldlm_cli_cancel_unused);
+
+/* Lock iterators. */
+
+static int ldlm_resource_foreach(struct ldlm_resource *res,
+				 ldlm_iterator_t iter, void *closure)
+{
+	struct ldlm_lock *tmp;
+	struct ldlm_lock *lock;
+	int rc = LDLM_ITER_CONTINUE;
+
+	if (!res)
+		return LDLM_ITER_CONTINUE;
+
+	lock_res(res);
+	list_for_each_entry_safe(lock, tmp, &res->lr_granted, l_res_link) {
+		if (iter(lock, closure) == LDLM_ITER_STOP) {
+			rc = LDLM_ITER_STOP;
+			goto out;
+		}
+	}
+
+	list_for_each_entry_safe(lock, tmp, &res->lr_waiting, l_res_link) {
+		if (iter(lock, closure) == LDLM_ITER_STOP) {
+			rc = LDLM_ITER_STOP;
+			goto out;
+		}
+	}
+ out:
+	unlock_res(res);
+	return rc;
+}
+
+struct iter_helper_data {
+	ldlm_iterator_t iter;
+	void *closure;
+};
+
+static int ldlm_iter_helper(struct ldlm_lock *lock, void *closure)
+{
+	struct iter_helper_data *helper = closure;
+
+	return helper->iter(lock, helper->closure);
+}
+
+static int ldlm_res_iter_helper(struct cfs_hash *hs, struct cfs_hash_bd *bd,
+				struct hlist_node *hnode, void *arg)
+
+{
+	struct ldlm_resource *res = cfs_hash_object(hs, hnode);
+
+	return ldlm_resource_foreach(res, ldlm_iter_helper, arg) ==
+	       LDLM_ITER_STOP;
+}
+
+static void ldlm_namespace_foreach(struct ldlm_namespace *ns,
+				   ldlm_iterator_t iter, void *closure)
+
+{
+	struct iter_helper_data helper = {
+		.iter		= iter,
+		.closure	= closure,
+	};
+
+	cfs_hash_for_each_nolock(ns->ns_rs_hash,
+				 ldlm_res_iter_helper, &helper, 0);
+}
+
+/* non-blocking function to manipulate a lock whose cb_data is being put away.
+ * return  0:  find no resource
+ *       > 0:  must be LDLM_ITER_STOP/LDLM_ITER_CONTINUE.
+ *       < 0:  errors
+ */
+int ldlm_resource_iterate(struct ldlm_namespace *ns,
+			  const struct ldlm_res_id *res_id,
+			  ldlm_iterator_t iter, void *data)
+{
+	struct ldlm_resource *res;
+	int rc;
+
+	LASSERTF(ns, "must pass in namespace\n");
+
+	res = ldlm_resource_get(ns, NULL, res_id, 0, 0);
+	if (IS_ERR(res))
+		return 0;
+
+	LDLM_RESOURCE_ADDREF(res);
+	rc = ldlm_resource_foreach(res, iter, data);
+	LDLM_RESOURCE_DELREF(res);
+	ldlm_resource_putref(res);
+	return rc;
+}
+EXPORT_SYMBOL(ldlm_resource_iterate);
+
+/* Lock replay */
+
+static int ldlm_chain_lock_for_replay(struct ldlm_lock *lock, void *closure)
+{
+	struct list_head *list = closure;
+
+	/* we use l_pending_chain here, because it's unused on clients. */
+	LASSERTF(list_empty(&lock->l_pending_chain),
+		 "lock %p next %p prev %p\n",
+		 lock, &lock->l_pending_chain.next,
+		 &lock->l_pending_chain.prev);
+	/* bug 9573: don't replay locks left after eviction, or
+	 * bug 17614: locks being actively cancelled. Get a reference
+	 * on a lock so that it does not disappear under us (e.g. due to cancel)
+	 */
+	if (!(lock->l_flags & (LDLM_FL_FAILED | LDLM_FL_BL_DONE))) {
+		list_add(&lock->l_pending_chain, list);
+		LDLM_LOCK_GET(lock);
+	}
+
+	return LDLM_ITER_CONTINUE;
+}
+
+static int replay_lock_interpret(const struct lu_env *env,
+				 struct ptlrpc_request *req,
+				 struct ldlm_async_args *aa, int rc)
+{
+	struct ldlm_lock     *lock;
+	struct ldlm_reply    *reply;
+	struct obd_export    *exp;
+
+	atomic_dec(&req->rq_import->imp_replay_inflight);
+	if (rc != ELDLM_OK)
+		goto out;
+
+	reply = req_capsule_server_get(&req->rq_pill, &RMF_DLM_REP);
+	if (!reply) {
+		rc = -EPROTO;
+		goto out;
+	}
+
+	lock = ldlm_handle2lock(&aa->lock_handle);
+	if (!lock) {
+		CERROR("received replay ack for unknown local cookie %#llx remote cookie %#llx from server %s id %s\n",
+		       aa->lock_handle.cookie, reply->lock_handle.cookie,
+		       req->rq_export->exp_client_uuid.uuid,
+		       libcfs_id2str(req->rq_peer));
+		rc = -ESTALE;
+		goto out;
+	}
+
+	/* Key change rehash lock in per-export hash with new key */
+	exp = req->rq_export;
+	lock->l_remote_handle = reply->lock_handle;
+
+	LDLM_DEBUG(lock, "replayed lock:");
+	ptlrpc_import_recovery_state_machine(req->rq_import);
+	LDLM_LOCK_PUT(lock);
+out:
+	if (rc != ELDLM_OK)
+		ptlrpc_connect_import(req->rq_import);
+
+	return rc;
+}
+
+static int replay_one_lock(struct obd_import *imp, struct ldlm_lock *lock)
+{
+	struct ptlrpc_request *req;
+	struct ldlm_async_args *aa;
+	struct ldlm_request   *body;
+	int flags;
+
+	/* Bug 11974: Do not replay a lock which is actively being canceled */
+	if (ldlm_is_bl_done(lock)) {
+		LDLM_DEBUG(lock, "Not replaying canceled lock:");
+		return 0;
+	}
+
+	/* If this is reply-less callback lock, we cannot replay it, since
+	 * server might have long dropped it, but notification of that event was
+	 * lost by network. (and server granted conflicting lock already)
+	 */
+	if (ldlm_is_cancel_on_block(lock)) {
+		LDLM_DEBUG(lock, "Not replaying reply-less lock:");
+		ldlm_lock_cancel(lock);
+		return 0;
+	}
+
+	/*
+	 * If granted mode matches the requested mode, this lock is granted.
+	 *
+	 * If they differ, but we have a granted mode, then we were granted
+	 * one mode and now want another: ergo, converting.
+	 *
+	 * If we haven't been granted anything and are on a resource list,
+	 * then we're blocked/waiting.
+	 *
+	 * If we haven't been granted anything and we're NOT on a resource list,
+	 * then we haven't got a reply yet and don't have a known disposition.
+	 * This happens whenever a lock enqueue is the request that triggers
+	 * recovery.
+	 */
+	if (lock->l_granted_mode == lock->l_req_mode)
+		flags = LDLM_FL_REPLAY | LDLM_FL_BLOCK_GRANTED;
+	else if (lock->l_granted_mode)
+		flags = LDLM_FL_REPLAY | LDLM_FL_BLOCK_CONV;
+	else if (!list_empty(&lock->l_res_link))
+		flags = LDLM_FL_REPLAY | LDLM_FL_BLOCK_WAIT;
+	else
+		flags = LDLM_FL_REPLAY;
+
+	req = ptlrpc_request_alloc_pack(imp, &RQF_LDLM_ENQUEUE,
+					LUSTRE_DLM_VERSION, LDLM_ENQUEUE);
+	if (!req)
+		return -ENOMEM;
+
+	/* We're part of recovery, so don't wait for it. */
+	req->rq_send_state = LUSTRE_IMP_REPLAY_LOCKS;
+
+	body = req_capsule_client_get(&req->rq_pill, &RMF_DLM_REQ);
+	ldlm_lock2desc(lock, &body->lock_desc);
+	body->lock_flags = ldlm_flags_to_wire(flags);
+
+	ldlm_lock2handle(lock, &body->lock_handle[0]);
+	if (lock->l_lvb_len > 0)
+		req_capsule_extend(&req->rq_pill, &RQF_LDLM_ENQUEUE_LVB);
+	req_capsule_set_size(&req->rq_pill, &RMF_DLM_LVB, RCL_SERVER,
+			     lock->l_lvb_len);
+	ptlrpc_request_set_replen(req);
+	/* notify the server we've replayed all requests.
+	 * also, we mark the request to be put on a dedicated
+	 * queue to be processed after all request replayes.
+	 * bug 6063
+	 */
+	lustre_msg_set_flags(req->rq_reqmsg, MSG_REQ_REPLAY_DONE);
+
+	LDLM_DEBUG(lock, "replaying lock:");
+
+	atomic_inc(&req->rq_import->imp_replay_inflight);
+	BUILD_BUG_ON(sizeof(*aa) > sizeof(req->rq_async_args));
+	aa = ptlrpc_req_async_args(req);
+	aa->lock_handle = body->lock_handle[0];
+	req->rq_interpret_reply = (ptlrpc_interpterer_t)replay_lock_interpret;
+	ptlrpcd_add_req(req);
+
+	return 0;
+}
+
+/**
+ * Cancel as many unused locks as possible before replay. since we are
+ * in recovery, we can't wait for any outstanding RPCs to send any RPC
+ * to the server.
+ *
+ * Called only in recovery before replaying locks. there is no need to
+ * replay locks that are unused. since the clients may hold thousands of
+ * cached unused locks, dropping the unused locks can greatly reduce the
+ * load on the servers at recovery time.
+ */
+static void ldlm_cancel_unused_locks_for_replay(struct ldlm_namespace *ns)
+{
+	int canceled;
+	LIST_HEAD(cancels);
+
+	CDEBUG(D_DLMTRACE,
+	       "Dropping as many unused locks as possible before replay for namespace %s (%d)\n",
+	       ldlm_ns_name(ns), ns->ns_nr_unused);
+
+	/* We don't need to care whether or not LRU resize is enabled
+	 * because the LDLM_LRU_FLAG_NO_WAIT policy doesn't use the
+	 * count parameter
+	 */
+	canceled = ldlm_cancel_lru_local(ns, &cancels, ns->ns_nr_unused, 0,
+					 LCF_LOCAL, LDLM_LRU_FLAG_NO_WAIT);
+
+	CDEBUG(D_DLMTRACE, "Canceled %d unused locks from namespace %s\n",
+	       canceled, ldlm_ns_name(ns));
+}
+
+int ldlm_replay_locks(struct obd_import *imp)
+{
+	struct ldlm_namespace *ns = imp->imp_obd->obd_namespace;
+	LIST_HEAD(list);
+	struct ldlm_lock *lock, *next;
+	int rc = 0;
+
+	LASSERT(atomic_read(&imp->imp_replay_inflight) == 0);
+
+	/* don't replay locks if import failed recovery */
+	if (imp->imp_vbr_failed)
+		return 0;
+
+	/* ensure this doesn't fall to 0 before all have been queued */
+	atomic_inc(&imp->imp_replay_inflight);
+
+	if (ldlm_cancel_unused_locks_before_replay)
+		ldlm_cancel_unused_locks_for_replay(ns);
+
+	ldlm_namespace_foreach(ns, ldlm_chain_lock_for_replay, &list);
+
+	list_for_each_entry_safe(lock, next, &list, l_pending_chain) {
+		list_del_init(&lock->l_pending_chain);
+		if (rc) {
+			LDLM_LOCK_RELEASE(lock);
+			continue; /* or try to do the rest? */
+		}
+		rc = replay_one_lock(imp, lock);
+		LDLM_LOCK_RELEASE(lock);
+	}
+
+	atomic_dec(&imp->imp_replay_inflight);
+
+	return rc;
+}
diff --git a/drivers/staging/lustre/lustre/ptlrpc/ldlm_resource.c b/drivers/staging/lustre/lustre/ptlrpc/ldlm_resource.c
new file mode 100644
index 000000000000..3946d62ff009
--- /dev/null
+++ b/drivers/staging/lustre/lustre/ptlrpc/ldlm_resource.c
@@ -0,0 +1,1318 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * GPL HEADER START
+ *
+ * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 only,
+ * as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License version 2 for more details (a copy is included
+ * in the LICENSE file that accompanied this code).
+ *
+ * You should have received a copy of the GNU General Public License
+ * version 2 along with this program; If not, see
+ * http://www.gnu.org/licenses/gpl-2.0.html
+ *
+ * GPL HEADER END
+ */
+/*
+ * Copyright (c) 2002, 2010, Oracle and/or its affiliates. All rights reserved.
+ * Use is subject to license terms.
+ *
+ * Copyright (c) 2010, 2015, Intel Corporation.
+ */
+/*
+ * This file is part of Lustre, http://www.lustre.org/
+ * Lustre is a trademark of Sun Microsystems, Inc.
+ *
+ * lustre/ldlm/ldlm_resource.c
+ *
+ * Author: Phil Schwan <phil at clusterfs.com>
+ * Author: Peter Braam <braam at clusterfs.com>
+ */
+
+#define DEBUG_SUBSYSTEM S_LDLM
+#include <lustre_dlm.h>
+#include <lustre_fid.h>
+#include <obd_class.h>
+#include "ldlm_internal.h"
+#include <linux/libcfs/libcfs_hash.h>
+
+struct kmem_cache *ldlm_resource_slab, *ldlm_lock_slab;
+
+int ldlm_srv_namespace_nr;
+int ldlm_cli_namespace_nr;
+
+struct mutex ldlm_srv_namespace_lock;
+LIST_HEAD(ldlm_srv_namespace_list);
+
+struct mutex ldlm_cli_namespace_lock;
+/* Client Namespaces that have active resources in them.
+ * Once all resources go away, ldlm_poold moves such namespaces to the
+ * inactive list
+ */
+LIST_HEAD(ldlm_cli_active_namespace_list);
+/* Client namespaces that don't have any locks in them */
+static LIST_HEAD(ldlm_cli_inactive_namespace_list);
+
+static struct dentry *ldlm_debugfs_dir;
+static struct dentry *ldlm_ns_debugfs_dir;
+struct dentry *ldlm_svc_debugfs_dir;
+
+/* during debug dump certain amount of granted locks for one resource to avoid
+ * DDOS.
+ */
+static unsigned int ldlm_dump_granted_max = 256;
+
+static ssize_t
+lprocfs_wr_dump_ns(struct file *file, const char __user *buffer,
+		   size_t count, loff_t *off)
+{
+	ldlm_dump_all_namespaces(LDLM_NAMESPACE_SERVER, D_DLMTRACE);
+	ldlm_dump_all_namespaces(LDLM_NAMESPACE_CLIENT, D_DLMTRACE);
+	return count;
+}
+
+LPROC_SEQ_FOPS_WR_ONLY(ldlm, dump_ns);
+
+static int ldlm_rw_uint_seq_show(struct seq_file *m, void *v)
+{
+	seq_printf(m, "%u\n", *(unsigned int *)m->private);
+	return 0;
+}
+
+static ssize_t
+ldlm_rw_uint_seq_write(struct file *file, const char __user *buffer,
+		       size_t count, loff_t *off)
+{
+	struct seq_file *seq = file->private_data;
+
+	if (count == 0)
+		return 0;
+	return kstrtouint_from_user(buffer, count, 0,
+				    (unsigned int *)seq->private);
+}
+
+LPROC_SEQ_FOPS(ldlm_rw_uint);
+
+static struct lprocfs_vars ldlm_debugfs_list[] = {
+	{ "dump_namespaces", &ldlm_dump_ns_fops, NULL, 0222 },
+	{ "dump_granted_max", &ldlm_rw_uint_fops, &ldlm_dump_granted_max },
+	{ NULL }
+};
+
+void ldlm_debugfs_setup(void)
+{
+	ldlm_debugfs_dir = debugfs_create_dir(OBD_LDLM_DEVICENAME,
+					      debugfs_lustre_root);
+
+	ldlm_ns_debugfs_dir = debugfs_create_dir("namespaces",
+						 ldlm_debugfs_dir);
+
+	ldlm_svc_debugfs_dir = debugfs_create_dir("services", ldlm_debugfs_dir);
+
+	ldebugfs_add_vars(ldlm_debugfs_dir, ldlm_debugfs_list, NULL);
+}
+
+void ldlm_debugfs_cleanup(void)
+{
+	debugfs_remove_recursive(ldlm_svc_debugfs_dir);
+	debugfs_remove_recursive(ldlm_ns_debugfs_dir);
+	debugfs_remove_recursive(ldlm_debugfs_dir);
+}
+
+static ssize_t resource_count_show(struct kobject *kobj, struct attribute *attr,
+				   char *buf)
+{
+	struct ldlm_namespace *ns = container_of(kobj, struct ldlm_namespace,
+						 ns_kobj);
+	__u64		  res = 0;
+	struct cfs_hash_bd	  bd;
+	int		    i;
+
+	/* result is not strictly consistent */
+	cfs_hash_for_each_bucket(ns->ns_rs_hash, &bd, i)
+		res += cfs_hash_bd_count_get(&bd);
+	return sprintf(buf, "%lld\n", res);
+}
+LUSTRE_RO_ATTR(resource_count);
+
+static ssize_t lock_count_show(struct kobject *kobj, struct attribute *attr,
+			       char *buf)
+{
+	struct ldlm_namespace *ns = container_of(kobj, struct ldlm_namespace,
+						 ns_kobj);
+	__u64		  locks;
+
+	locks = lprocfs_stats_collector(ns->ns_stats, LDLM_NSS_LOCKS,
+					LPROCFS_FIELDS_FLAGS_SUM);
+	return sprintf(buf, "%lld\n", locks);
+}
+LUSTRE_RO_ATTR(lock_count);
+
+static ssize_t lock_unused_count_show(struct kobject *kobj,
+				      struct attribute *attr,
+				      char *buf)
+{
+	struct ldlm_namespace *ns = container_of(kobj, struct ldlm_namespace,
+						 ns_kobj);
+
+	return sprintf(buf, "%d\n", ns->ns_nr_unused);
+}
+LUSTRE_RO_ATTR(lock_unused_count);
+
+static ssize_t lru_size_show(struct kobject *kobj, struct attribute *attr,
+			     char *buf)
+{
+	struct ldlm_namespace *ns = container_of(kobj, struct ldlm_namespace,
+						 ns_kobj);
+	__u32 *nr = &ns->ns_max_unused;
+
+	if (ns_connect_lru_resize(ns))
+		nr = &ns->ns_nr_unused;
+	return sprintf(buf, "%u\n", *nr);
+}
+
+static ssize_t lru_size_store(struct kobject *kobj, struct attribute *attr,
+			      const char *buffer, size_t count)
+{
+	struct ldlm_namespace *ns = container_of(kobj, struct ldlm_namespace,
+						 ns_kobj);
+	unsigned long tmp;
+	int lru_resize;
+	int err;
+
+	if (strncmp(buffer, "clear", 5) == 0) {
+		CDEBUG(D_DLMTRACE,
+		       "dropping all unused locks from namespace %s\n",
+		       ldlm_ns_name(ns));
+		if (ns_connect_lru_resize(ns)) {
+			int canceled, unused  = ns->ns_nr_unused;
+
+			/* Try to cancel all @ns_nr_unused locks. */
+			canceled = ldlm_cancel_lru(ns, unused, 0,
+						   LDLM_LRU_FLAG_PASSED);
+			if (canceled < unused) {
+				CDEBUG(D_DLMTRACE,
+				       "not all requested locks are canceled, requested: %d, canceled: %d\n",
+				       unused,
+				       canceled);
+				return -EINVAL;
+			}
+		} else {
+			tmp = ns->ns_max_unused;
+			ns->ns_max_unused = 0;
+			ldlm_cancel_lru(ns, 0, 0, LDLM_LRU_FLAG_PASSED);
+			ns->ns_max_unused = tmp;
+		}
+		return count;
+	}
+
+	err = kstrtoul(buffer, 10, &tmp);
+	if (err != 0) {
+		CERROR("lru_size: invalid value written\n");
+		return -EINVAL;
+	}
+	lru_resize = (tmp == 0);
+
+	if (ns_connect_lru_resize(ns)) {
+		if (!lru_resize)
+			ns->ns_max_unused = (unsigned int)tmp;
+
+		if (tmp > ns->ns_nr_unused)
+			tmp = ns->ns_nr_unused;
+		tmp = ns->ns_nr_unused - tmp;
+
+		CDEBUG(D_DLMTRACE,
+		       "changing namespace %s unused locks from %u to %u\n",
+		       ldlm_ns_name(ns), ns->ns_nr_unused,
+		       (unsigned int)tmp);
+		ldlm_cancel_lru(ns, tmp, LCF_ASYNC, LDLM_LRU_FLAG_PASSED);
+
+		if (!lru_resize) {
+			CDEBUG(D_DLMTRACE,
+			       "disable lru_resize for namespace %s\n",
+			       ldlm_ns_name(ns));
+			ns->ns_connect_flags &= ~OBD_CONNECT_LRU_RESIZE;
+		}
+	} else {
+		CDEBUG(D_DLMTRACE,
+		       "changing namespace %s max_unused from %u to %u\n",
+		       ldlm_ns_name(ns), ns->ns_max_unused,
+		       (unsigned int)tmp);
+		ns->ns_max_unused = (unsigned int)tmp;
+		ldlm_cancel_lru(ns, 0, LCF_ASYNC, LDLM_LRU_FLAG_PASSED);
+
+		/* Make sure that LRU resize was originally supported before
+		 * turning it on here.
+		 */
+		if (lru_resize &&
+		    (ns->ns_orig_connect_flags & OBD_CONNECT_LRU_RESIZE)) {
+			CDEBUG(D_DLMTRACE,
+			       "enable lru_resize for namespace %s\n",
+			       ldlm_ns_name(ns));
+			ns->ns_connect_flags |= OBD_CONNECT_LRU_RESIZE;
+		}
+	}
+
+	return count;
+}
+LUSTRE_RW_ATTR(lru_size);
+
+static ssize_t lru_max_age_show(struct kobject *kobj, struct attribute *attr,
+				char *buf)
+{
+	struct ldlm_namespace *ns = container_of(kobj, struct ldlm_namespace,
+						 ns_kobj);
+
+	return sprintf(buf, "%u\n", ns->ns_max_age);
+}
+
+static ssize_t lru_max_age_store(struct kobject *kobj, struct attribute *attr,
+				 const char *buffer, size_t count)
+{
+	struct ldlm_namespace *ns = container_of(kobj, struct ldlm_namespace,
+						 ns_kobj);
+	unsigned long tmp;
+	int err;
+
+	err = kstrtoul(buffer, 10, &tmp);
+	if (err != 0)
+		return -EINVAL;
+
+	ns->ns_max_age = tmp;
+
+	return count;
+}
+LUSTRE_RW_ATTR(lru_max_age);
+
+static ssize_t early_lock_cancel_show(struct kobject *kobj,
+				      struct attribute *attr,
+				      char *buf)
+{
+	struct ldlm_namespace *ns = container_of(kobj, struct ldlm_namespace,
+						 ns_kobj);
+
+	return sprintf(buf, "%d\n", ns_connect_cancelset(ns));
+}
+
+static ssize_t early_lock_cancel_store(struct kobject *kobj,
+				       struct attribute *attr,
+				       const char *buffer,
+				       size_t count)
+{
+	struct ldlm_namespace *ns = container_of(kobj, struct ldlm_namespace,
+						 ns_kobj);
+	unsigned long supp = -1;
+	int rc;
+
+	rc = kstrtoul(buffer, 10, &supp);
+	if (rc < 0)
+		return rc;
+
+	if (supp == 0)
+		ns->ns_connect_flags &= ~OBD_CONNECT_CANCELSET;
+	else if (ns->ns_orig_connect_flags & OBD_CONNECT_CANCELSET)
+		ns->ns_connect_flags |= OBD_CONNECT_CANCELSET;
+	return count;
+}
+LUSTRE_RW_ATTR(early_lock_cancel);
+
+/* These are for namespaces in /sys/fs/lustre/ldlm/namespaces/ */
+static struct attribute *ldlm_ns_attrs[] = {
+	&lustre_attr_resource_count.attr,
+	&lustre_attr_lock_count.attr,
+	&lustre_attr_lock_unused_count.attr,
+	&lustre_attr_lru_size.attr,
+	&lustre_attr_lru_max_age.attr,
+	&lustre_attr_early_lock_cancel.attr,
+	NULL,
+};
+
+static void ldlm_ns_release(struct kobject *kobj)
+{
+	struct ldlm_namespace *ns = container_of(kobj, struct ldlm_namespace,
+						 ns_kobj);
+	complete(&ns->ns_kobj_unregister);
+}
+
+static struct kobj_type ldlm_ns_ktype = {
+	.default_attrs  = ldlm_ns_attrs,
+	.sysfs_ops      = &lustre_sysfs_ops,
+	.release        = ldlm_ns_release,
+};
+
+static void ldlm_namespace_debugfs_unregister(struct ldlm_namespace *ns)
+{
+	debugfs_remove_recursive(ns->ns_debugfs_entry);
+
+	if (ns->ns_stats)
+		lprocfs_free_stats(&ns->ns_stats);
+}
+
+static void ldlm_namespace_sysfs_unregister(struct ldlm_namespace *ns)
+{
+	kobject_put(&ns->ns_kobj);
+	wait_for_completion(&ns->ns_kobj_unregister);
+}
+
+static int ldlm_namespace_sysfs_register(struct ldlm_namespace *ns)
+{
+	int err;
+
+	ns->ns_kobj.kset = ldlm_ns_kset;
+	init_completion(&ns->ns_kobj_unregister);
+	err = kobject_init_and_add(&ns->ns_kobj, &ldlm_ns_ktype, NULL,
+				   "%s", ldlm_ns_name(ns));
+
+	ns->ns_stats = lprocfs_alloc_stats(LDLM_NSS_LAST, 0);
+	if (!ns->ns_stats) {
+		kobject_put(&ns->ns_kobj);
+		return -ENOMEM;
+	}
+
+	lprocfs_counter_init(ns->ns_stats, LDLM_NSS_LOCKS,
+			     LPROCFS_CNTR_AVGMINMAX, "locks", "locks");
+
+	return err;
+}
+
+static int ldlm_namespace_debugfs_register(struct ldlm_namespace *ns)
+{
+	struct dentry *ns_entry;
+
+	if (!IS_ERR_OR_NULL(ns->ns_debugfs_entry)) {
+		ns_entry = ns->ns_debugfs_entry;
+	} else {
+		ns_entry = debugfs_create_dir(ldlm_ns_name(ns),
+					      ldlm_ns_debugfs_dir);
+		if (!ns_entry)
+			return -ENOMEM;
+		ns->ns_debugfs_entry = ns_entry;
+	}
+
+	return 0;
+}
+
+#undef MAX_STRING_SIZE
+
+static struct ldlm_resource *ldlm_resource_getref(struct ldlm_resource *res)
+{
+	LASSERT(res);
+	LASSERT(res != LP_POISON);
+	atomic_inc(&res->lr_refcount);
+	CDEBUG(D_INFO, "getref res: %p count: %d\n", res,
+	       atomic_read(&res->lr_refcount));
+	return res;
+}
+
+static unsigned int ldlm_res_hop_hash(struct cfs_hash *hs,
+				      const void *key, unsigned int mask)
+{
+	const struct ldlm_res_id     *id  = key;
+	unsigned int		val = 0;
+	unsigned int		i;
+
+	for (i = 0; i < RES_NAME_SIZE; i++)
+		val += id->name[i];
+	return val & mask;
+}
+
+static unsigned int ldlm_res_hop_fid_hash(struct cfs_hash *hs,
+					  const void *key, unsigned int mask)
+{
+	const struct ldlm_res_id *id = key;
+	struct lu_fid       fid;
+	__u32	       hash;
+	__u32	       val;
+
+	fid.f_seq = id->name[LUSTRE_RES_ID_SEQ_OFF];
+	fid.f_oid = (__u32)id->name[LUSTRE_RES_ID_VER_OID_OFF];
+	fid.f_ver = (__u32)(id->name[LUSTRE_RES_ID_VER_OID_OFF] >> 32);
+
+	hash = fid_flatten32(&fid);
+	hash += (hash >> 4) + (hash << 12); /* mixing oid and seq */
+	if (id->name[LUSTRE_RES_ID_HSH_OFF] != 0) {
+		val = id->name[LUSTRE_RES_ID_HSH_OFF];
+		hash += (val >> 5) + (val << 11);
+	} else {
+		val = fid_oid(&fid);
+	}
+	hash = hash_long(hash, hs->hs_bkt_bits);
+	/* give me another random factor */
+	hash -= hash_long((unsigned long)hs, val % 11 + 3);
+
+	hash <<= hs->hs_cur_bits - hs->hs_bkt_bits;
+	hash |= ldlm_res_hop_hash(hs, key, CFS_HASH_NBKT(hs) - 1);
+
+	return hash & mask;
+}
+
+static void *ldlm_res_hop_key(struct hlist_node *hnode)
+{
+	struct ldlm_resource   *res;
+
+	res = hlist_entry(hnode, struct ldlm_resource, lr_hash);
+	return &res->lr_name;
+}
+
+static int ldlm_res_hop_keycmp(const void *key, struct hlist_node *hnode)
+{
+	struct ldlm_resource   *res;
+
+	res = hlist_entry(hnode, struct ldlm_resource, lr_hash);
+	return ldlm_res_eq((const struct ldlm_res_id *)key,
+			   (const struct ldlm_res_id *)&res->lr_name);
+}
+
+static void *ldlm_res_hop_object(struct hlist_node *hnode)
+{
+	return hlist_entry(hnode, struct ldlm_resource, lr_hash);
+}
+
+static void ldlm_res_hop_get_locked(struct cfs_hash *hs,
+				    struct hlist_node *hnode)
+{
+	struct ldlm_resource *res;
+
+	res = hlist_entry(hnode, struct ldlm_resource, lr_hash);
+	ldlm_resource_getref(res);
+}
+
+static void ldlm_res_hop_put(struct cfs_hash *hs, struct hlist_node *hnode)
+{
+	struct ldlm_resource *res;
+
+	res = hlist_entry(hnode, struct ldlm_resource, lr_hash);
+	ldlm_resource_putref(res);
+}
+
+static struct cfs_hash_ops ldlm_ns_hash_ops = {
+	.hs_hash	= ldlm_res_hop_hash,
+	.hs_key		= ldlm_res_hop_key,
+	.hs_keycmp      = ldlm_res_hop_keycmp,
+	.hs_keycpy      = NULL,
+	.hs_object      = ldlm_res_hop_object,
+	.hs_get		= ldlm_res_hop_get_locked,
+	.hs_put		= ldlm_res_hop_put
+};
+
+static struct cfs_hash_ops ldlm_ns_fid_hash_ops = {
+	.hs_hash	= ldlm_res_hop_fid_hash,
+	.hs_key		= ldlm_res_hop_key,
+	.hs_keycmp      = ldlm_res_hop_keycmp,
+	.hs_keycpy      = NULL,
+	.hs_object      = ldlm_res_hop_object,
+	.hs_get		= ldlm_res_hop_get_locked,
+	.hs_put		= ldlm_res_hop_put
+};
+
+struct ldlm_ns_hash_def {
+	enum ldlm_ns_type nsd_type;
+	/** hash bucket bits */
+	unsigned int	nsd_bkt_bits;
+	/** hash bits */
+	unsigned int	nsd_all_bits;
+	/** hash operations */
+	struct cfs_hash_ops *nsd_hops;
+};
+
+static struct ldlm_ns_hash_def ldlm_ns_hash_defs[] = {
+	{
+		.nsd_type       = LDLM_NS_TYPE_MDC,
+		.nsd_bkt_bits   = 11,
+		.nsd_all_bits   = 16,
+		.nsd_hops       = &ldlm_ns_fid_hash_ops,
+	},
+	{
+		.nsd_type       = LDLM_NS_TYPE_MDT,
+		.nsd_bkt_bits   = 14,
+		.nsd_all_bits   = 21,
+		.nsd_hops       = &ldlm_ns_fid_hash_ops,
+	},
+	{
+		.nsd_type       = LDLM_NS_TYPE_OSC,
+		.nsd_bkt_bits   = 8,
+		.nsd_all_bits   = 12,
+		.nsd_hops       = &ldlm_ns_hash_ops,
+	},
+	{
+		.nsd_type       = LDLM_NS_TYPE_OST,
+		.nsd_bkt_bits   = 11,
+		.nsd_all_bits   = 17,
+		.nsd_hops       = &ldlm_ns_hash_ops,
+	},
+	{
+		.nsd_type       = LDLM_NS_TYPE_MGC,
+		.nsd_bkt_bits   = 4,
+		.nsd_all_bits   = 4,
+		.nsd_hops       = &ldlm_ns_hash_ops,
+	},
+	{
+		.nsd_type       = LDLM_NS_TYPE_MGT,
+		.nsd_bkt_bits   = 4,
+		.nsd_all_bits   = 4,
+		.nsd_hops       = &ldlm_ns_hash_ops,
+	},
+	{
+		.nsd_type       = LDLM_NS_TYPE_UNKNOWN,
+	},
+};
+
+/** Register \a ns in the list of namespaces */
+static void ldlm_namespace_register(struct ldlm_namespace *ns,
+				    enum ldlm_side client)
+{
+	mutex_lock(ldlm_namespace_lock(client));
+	LASSERT(list_empty(&ns->ns_list_chain));
+	list_add(&ns->ns_list_chain, &ldlm_cli_inactive_namespace_list);
+	ldlm_namespace_nr_inc(client);
+	mutex_unlock(ldlm_namespace_lock(client));
+}
+
+/**
+ * Create and initialize new empty namespace.
+ */
+struct ldlm_namespace *ldlm_namespace_new(struct obd_device *obd, char *name,
+					  enum ldlm_side client,
+					  enum ldlm_appetite apt,
+					  enum ldlm_ns_type ns_type)
+{
+	struct ldlm_namespace *ns = NULL;
+	struct ldlm_ns_bucket *nsb;
+	struct ldlm_ns_hash_def    *nsd;
+	struct cfs_hash_bd	  bd;
+	int		    idx;
+	int		    rc;
+
+	LASSERT(obd);
+
+	rc = ldlm_get_ref();
+	if (rc) {
+		CERROR("ldlm_get_ref failed: %d\n", rc);
+		return NULL;
+	}
+
+	for (idx = 0;; idx++) {
+		nsd = &ldlm_ns_hash_defs[idx];
+		if (nsd->nsd_type == LDLM_NS_TYPE_UNKNOWN) {
+			CERROR("Unknown type %d for ns %s\n", ns_type, name);
+			goto out_ref;
+		}
+
+		if (nsd->nsd_type == ns_type)
+			break;
+	}
+
+	ns = kzalloc(sizeof(*ns), GFP_NOFS);
+	if (!ns)
+		goto out_ref;
+
+	ns->ns_rs_hash = cfs_hash_create(name,
+					 nsd->nsd_all_bits, nsd->nsd_all_bits,
+					 nsd->nsd_bkt_bits, sizeof(*nsb),
+					 CFS_HASH_MIN_THETA,
+					 CFS_HASH_MAX_THETA,
+					 nsd->nsd_hops,
+					 CFS_HASH_DEPTH |
+					 CFS_HASH_BIGNAME |
+					 CFS_HASH_SPIN_BKTLOCK |
+					 CFS_HASH_NO_ITEMREF);
+	if (!ns->ns_rs_hash)
+		goto out_ns;
+
+	cfs_hash_for_each_bucket(ns->ns_rs_hash, &bd, idx) {
+		nsb = cfs_hash_bd_extra_get(ns->ns_rs_hash, &bd);
+		at_init(&nsb->nsb_at_estimate, ldlm_enqueue_min, 0);
+		nsb->nsb_namespace = ns;
+	}
+
+	ns->ns_obd      = obd;
+	ns->ns_appetite = apt;
+	ns->ns_client   = client;
+	ns->ns_name     = kstrdup(name, GFP_KERNEL);
+	if (!ns->ns_name)
+		goto out_hash;
+
+	INIT_LIST_HEAD(&ns->ns_list_chain);
+	INIT_LIST_HEAD(&ns->ns_unused_list);
+	spin_lock_init(&ns->ns_lock);
+	atomic_set(&ns->ns_bref, 0);
+	init_waitqueue_head(&ns->ns_waitq);
+
+	ns->ns_max_parallel_ast   = LDLM_DEFAULT_PARALLEL_AST_LIMIT;
+	ns->ns_nr_unused	  = 0;
+	ns->ns_max_unused	 = LDLM_DEFAULT_LRU_SIZE;
+	ns->ns_max_age	    = LDLM_DEFAULT_MAX_ALIVE;
+	ns->ns_orig_connect_flags = 0;
+	ns->ns_connect_flags      = 0;
+	ns->ns_stopping	   = 0;
+
+	rc = ldlm_namespace_sysfs_register(ns);
+	if (rc != 0) {
+		CERROR("Can't initialize ns sysfs, rc %d\n", rc);
+		goto out_hash;
+	}
+
+	rc = ldlm_namespace_debugfs_register(ns);
+	if (rc != 0) {
+		CERROR("Can't initialize ns proc, rc %d\n", rc);
+		goto out_sysfs;
+	}
+
+	idx = ldlm_namespace_nr_read(client);
+	rc = ldlm_pool_init(&ns->ns_pool, ns, idx, client);
+	if (rc) {
+		CERROR("Can't initialize lock pool, rc %d\n", rc);
+		goto out_proc;
+	}
+
+	ldlm_namespace_register(ns, client);
+	return ns;
+out_proc:
+	ldlm_namespace_debugfs_unregister(ns);
+out_sysfs:
+	ldlm_namespace_sysfs_unregister(ns);
+	ldlm_namespace_cleanup(ns, 0);
+out_hash:
+	kfree(ns->ns_name);
+	cfs_hash_putref(ns->ns_rs_hash);
+out_ns:
+	kfree(ns);
+out_ref:
+	ldlm_put_ref();
+	return NULL;
+}
+EXPORT_SYMBOL(ldlm_namespace_new);
+
+extern struct ldlm_lock *ldlm_lock_get(struct ldlm_lock *lock);
+
+/**
+ * Cancel and destroy all locks on a resource.
+ *
+ * If flags contains FL_LOCAL_ONLY, don't try to tell the server, just
+ * clean up.  This is currently only used for recovery, and we make
+ * certain assumptions as a result--notably, that we shouldn't cancel
+ * locks with refs.
+ */
+static void cleanup_resource(struct ldlm_resource *res, struct list_head *q,
+			     __u64 flags)
+{
+	int rc = 0;
+	bool local_only = !!(flags & LDLM_FL_LOCAL_ONLY);
+
+	do {
+		struct ldlm_lock *lock = NULL, *tmp;
+		struct lustre_handle lockh;
+
+		/* First, we look for non-cleaned-yet lock
+		 * all cleaned locks are marked by CLEANED flag.
+		 */
+		lock_res(res);
+		list_for_each_entry(tmp, q, l_res_link) {
+			if (ldlm_is_cleaned(tmp))
+				continue;
+
+			lock = tmp;
+			LDLM_LOCK_GET(lock);
+			ldlm_set_cleaned(lock);
+			break;
+		}
+
+		if (!lock) {
+			unlock_res(res);
+			break;
+		}
+
+		/* Set CBPENDING so nothing in the cancellation path
+		 * can match this lock.
+		 */
+		ldlm_set_cbpending(lock);
+		ldlm_set_failed(lock);
+		lock->l_flags |= flags;
+
+		/* ... without sending a CANCEL message for local_only. */
+		if (local_only)
+			ldlm_set_local_only(lock);
+
+		if (local_only && (lock->l_readers || lock->l_writers)) {
+			/* This is a little bit gross, but much better than the
+			 * alternative: pretend that we got a blocking AST from
+			 * the server, so that when the lock is decref'd, it
+			 * will go away ...
+			 */
+			unlock_res(res);
+			LDLM_DEBUG(lock, "setting FL_LOCAL_ONLY");
+			if (lock->l_flags & LDLM_FL_FAIL_LOC) {
+				set_current_state(TASK_UNINTERRUPTIBLE);
+				schedule_timeout(4 * HZ);
+				set_current_state(TASK_RUNNING);
+			}
+			if (lock->l_completion_ast)
+				lock->l_completion_ast(lock, LDLM_FL_FAILED,
+						       NULL);
+			LDLM_LOCK_RELEASE(lock);
+			continue;
+		}
+
+		unlock_res(res);
+		ldlm_lock2handle(lock, &lockh);
+		rc = ldlm_cli_cancel(&lockh, LCF_LOCAL);
+		if (rc)
+			CERROR("ldlm_cli_cancel: %d\n", rc);
+		LDLM_LOCK_RELEASE(lock);
+	} while (1);
+}
+
+static int ldlm_resource_clean(struct cfs_hash *hs, struct cfs_hash_bd *bd,
+			       struct hlist_node *hnode, void *arg)
+{
+	struct ldlm_resource *res = cfs_hash_object(hs, hnode);
+	__u64 flags = *(__u64 *)arg;
+
+	cleanup_resource(res, &res->lr_granted, flags);
+	cleanup_resource(res, &res->lr_waiting, flags);
+
+	return 0;
+}
+
+static int ldlm_resource_complain(struct cfs_hash *hs, struct cfs_hash_bd *bd,
+				  struct hlist_node *hnode, void *arg)
+{
+	struct ldlm_resource  *res = cfs_hash_object(hs, hnode);
+
+	lock_res(res);
+	CERROR("%s: namespace resource " DLDLMRES
+	       " (%p) refcount nonzero (%d) after lock cleanup; forcing cleanup.\n",
+	       ldlm_ns_name(ldlm_res_to_ns(res)), PLDLMRES(res), res,
+	       atomic_read(&res->lr_refcount) - 1);
+
+	ldlm_resource_dump(D_ERROR, res);
+	unlock_res(res);
+	return 0;
+}
+
+/**
+ * Cancel and destroy all locks in the namespace.
+ *
+ * Typically used during evictions when server notified client that it was
+ * evicted and all of its state needs to be destroyed.
+ * Also used during shutdown.
+ */
+int ldlm_namespace_cleanup(struct ldlm_namespace *ns, __u64 flags)
+{
+	if (!ns) {
+		CDEBUG(D_INFO, "NULL ns, skipping cleanup\n");
+		return ELDLM_OK;
+	}
+
+	cfs_hash_for_each_nolock(ns->ns_rs_hash, ldlm_resource_clean,
+				 &flags, 0);
+	cfs_hash_for_each_nolock(ns->ns_rs_hash, ldlm_resource_complain,
+				 NULL, 0);
+	return ELDLM_OK;
+}
+EXPORT_SYMBOL(ldlm_namespace_cleanup);
+
+/**
+ * Attempts to free namespace.
+ *
+ * Only used when namespace goes away, like during an unmount.
+ */
+static int __ldlm_namespace_free(struct ldlm_namespace *ns, int force)
+{
+	/* At shutdown time, don't call the cancellation callback */
+	ldlm_namespace_cleanup(ns, force ? LDLM_FL_LOCAL_ONLY : 0);
+
+	if (atomic_read(&ns->ns_bref) > 0) {
+		int rc;
+
+		CDEBUG(D_DLMTRACE,
+		       "dlm namespace %s free waiting on refcount %d\n",
+		       ldlm_ns_name(ns), atomic_read(&ns->ns_bref));
+force_wait:
+		if (force)
+			rc = wait_event_idle_timeout(ns->ns_waitq,
+						     atomic_read(&ns->ns_bref) == 0,
+						     obd_timeout * HZ / 4) ? 0 : -ETIMEDOUT;
+		else
+			rc = l_wait_event_abortable(ns->ns_waitq,
+						    atomic_read(&ns->ns_bref) == 0);
+
+		/* Forced cleanups should be able to reclaim all references,
+		 * so it's safe to wait forever... we can't leak locks...
+		 */
+		if (force && rc == -ETIMEDOUT) {
+			LCONSOLE_ERROR("Forced cleanup waiting for %s namespace with %d resources in use, (rc=%d)\n",
+				       ldlm_ns_name(ns),
+				       atomic_read(&ns->ns_bref), rc);
+			goto force_wait;
+		}
+
+		if (atomic_read(&ns->ns_bref)) {
+			LCONSOLE_ERROR("Cleanup waiting for %s namespace with %d resources in use, (rc=%d)\n",
+				       ldlm_ns_name(ns),
+				       atomic_read(&ns->ns_bref), rc);
+			return ELDLM_NAMESPACE_EXISTS;
+		}
+		CDEBUG(D_DLMTRACE, "dlm namespace %s free done waiting\n",
+		       ldlm_ns_name(ns));
+	}
+
+	return ELDLM_OK;
+}
+
+/**
+ * Performs various cleanups for passed \a ns to make it drop refc and be
+ * ready for freeing. Waits for refc == 0.
+ *
+ * The following is done:
+ * (0) Unregister \a ns from its list to make inaccessible for potential
+ * users like pools thread and others;
+ * (1) Clear all locks in \a ns.
+ */
+void ldlm_namespace_free_prior(struct ldlm_namespace *ns,
+			       struct obd_import *imp,
+			       int force)
+{
+	int rc;
+
+	if (!ns)
+		return;
+
+	spin_lock(&ns->ns_lock);
+	ns->ns_stopping = 1;
+	spin_unlock(&ns->ns_lock);
+
+	/*
+	 * Can fail with -EINTR when force == 0 in which case try harder.
+	 */
+	rc = __ldlm_namespace_free(ns, force);
+	if (rc != ELDLM_OK) {
+		if (imp) {
+			ptlrpc_disconnect_import(imp, 0);
+			ptlrpc_invalidate_import(imp);
+		}
+
+		/*
+		 * With all requests dropped and the import inactive
+		 * we are guaranteed all reference will be dropped.
+		 */
+		rc = __ldlm_namespace_free(ns, 1);
+		LASSERT(rc == 0);
+	}
+}
+
+/** Unregister \a ns from the list of namespaces. */
+static void ldlm_namespace_unregister(struct ldlm_namespace *ns,
+				      enum ldlm_side client)
+{
+	mutex_lock(ldlm_namespace_lock(client));
+	LASSERT(!list_empty(&ns->ns_list_chain));
+	/* Some asserts and possibly other parts of the code are still
+	 * using list_empty(&ns->ns_list_chain). This is why it is
+	 * important to use list_del_init() here.
+	 */
+	list_del_init(&ns->ns_list_chain);
+	ldlm_namespace_nr_dec(client);
+	mutex_unlock(ldlm_namespace_lock(client));
+}
+
+/**
+ * Performs freeing memory structures related to \a ns. This is only done
+ * when ldlm_namespce_free_prior() successfully removed all resources
+ * referencing \a ns and its refc == 0.
+ */
+void ldlm_namespace_free_post(struct ldlm_namespace *ns)
+{
+	if (!ns)
+		return;
+
+	/* Make sure that nobody can find this ns in its list. */
+	ldlm_namespace_unregister(ns, ns->ns_client);
+	/* Fini pool _before_ parent proc dir is removed. This is important as
+	 * ldlm_pool_fini() removes own proc dir which is child to @dir.
+	 * Removing it after @dir may cause oops.
+	 */
+	ldlm_pool_fini(&ns->ns_pool);
+
+	ldlm_namespace_debugfs_unregister(ns);
+	ldlm_namespace_sysfs_unregister(ns);
+	cfs_hash_putref(ns->ns_rs_hash);
+	kfree(ns->ns_name);
+	/* Namespace \a ns should be not on list at this time, otherwise
+	 * this will cause issues related to using freed \a ns in poold
+	 * thread.
+	 */
+	LASSERT(list_empty(&ns->ns_list_chain));
+	kfree(ns);
+	ldlm_put_ref();
+}
+
+void ldlm_namespace_get(struct ldlm_namespace *ns)
+{
+	atomic_inc(&ns->ns_bref);
+}
+
+/* This is only for callers that care about refcount */
+static int ldlm_namespace_get_return(struct ldlm_namespace *ns)
+{
+	return atomic_inc_return(&ns->ns_bref);
+}
+
+void ldlm_namespace_put(struct ldlm_namespace *ns)
+{
+	if (atomic_dec_and_lock(&ns->ns_bref, &ns->ns_lock)) {
+		wake_up(&ns->ns_waitq);
+		spin_unlock(&ns->ns_lock);
+	}
+}
+
+/** Should be called with ldlm_namespace_lock(client) taken. */
+void ldlm_namespace_move_to_active_locked(struct ldlm_namespace *ns,
+					  enum ldlm_side client)
+{
+	LASSERT(!list_empty(&ns->ns_list_chain));
+	LASSERT(mutex_is_locked(ldlm_namespace_lock(client)));
+	list_move_tail(&ns->ns_list_chain, ldlm_namespace_list(client));
+}
+
+/** Should be called with ldlm_namespace_lock(client) taken. */
+void ldlm_namespace_move_to_inactive_locked(struct ldlm_namespace *ns,
+					    enum ldlm_side client)
+{
+	LASSERT(!list_empty(&ns->ns_list_chain));
+	LASSERT(mutex_is_locked(ldlm_namespace_lock(client)));
+	list_move_tail(&ns->ns_list_chain, &ldlm_cli_inactive_namespace_list);
+}
+
+/** Should be called with ldlm_namespace_lock(client) taken. */
+struct ldlm_namespace *ldlm_namespace_first_locked(enum ldlm_side client)
+{
+	LASSERT(mutex_is_locked(ldlm_namespace_lock(client)));
+	LASSERT(!list_empty(ldlm_namespace_list(client)));
+	return container_of(ldlm_namespace_list(client)->next,
+		struct ldlm_namespace, ns_list_chain);
+}
+
+/** Create and initialize new resource. */
+static struct ldlm_resource *ldlm_resource_new(void)
+{
+	struct ldlm_resource *res;
+	int idx;
+
+	res = kmem_cache_zalloc(ldlm_resource_slab, GFP_NOFS);
+	if (!res)
+		return NULL;
+
+	INIT_LIST_HEAD(&res->lr_granted);
+	INIT_LIST_HEAD(&res->lr_waiting);
+
+	/* Initialize interval trees for each lock mode. */
+	for (idx = 0; idx < LCK_MODE_NUM; idx++) {
+		res->lr_itree[idx].lit_size = 0;
+		res->lr_itree[idx].lit_mode = 1 << idx;
+		res->lr_itree[idx].lit_root = RB_ROOT_CACHED;
+	}
+
+	atomic_set(&res->lr_refcount, 1);
+	spin_lock_init(&res->lr_lock);
+	lu_ref_init(&res->lr_reference);
+
+	/* The creator of the resource must unlock the mutex after LVB
+	 * initialization.
+	 */
+	mutex_init(&res->lr_lvb_mutex);
+	mutex_lock(&res->lr_lvb_mutex);
+
+	return res;
+}
+
+/**
+ * Return a reference to resource with given name, creating it if necessary.
+ * Args: namespace with ns_lock unlocked
+ * Locks: takes and releases NS hash-lock and res->lr_lock
+ * Returns: referenced, unlocked ldlm_resource or NULL
+ */
+struct ldlm_resource *
+ldlm_resource_get(struct ldlm_namespace *ns, struct ldlm_resource *parent,
+		  const struct ldlm_res_id *name, enum ldlm_type type,
+		  int create)
+{
+	struct hlist_node     *hnode;
+	struct ldlm_resource *res = NULL;
+	struct cfs_hash_bd	 bd;
+	__u64		 version;
+	int		      ns_refcount = 0;
+	int rc;
+
+	LASSERT(!parent);
+	LASSERT(ns->ns_rs_hash);
+	LASSERT(name->name[0] != 0);
+
+	cfs_hash_bd_get_and_lock(ns->ns_rs_hash, (void *)name, &bd, 0);
+	hnode = cfs_hash_bd_lookup_locked(ns->ns_rs_hash, &bd, (void *)name);
+	if (hnode) {
+		cfs_hash_bd_unlock(ns->ns_rs_hash, &bd, 0);
+		goto lvbo_init;
+	}
+
+	version = cfs_hash_bd_version_get(&bd);
+	cfs_hash_bd_unlock(ns->ns_rs_hash, &bd, 0);
+
+	if (create == 0)
+		return ERR_PTR(-ENOENT);
+
+	LASSERTF(type >= LDLM_MIN_TYPE && type < LDLM_MAX_TYPE,
+		 "type: %d\n", type);
+	res = ldlm_resource_new();
+	if (!res)
+		return ERR_PTR(-ENOMEM);
+
+	res->lr_ns_bucket  = cfs_hash_bd_extra_get(ns->ns_rs_hash, &bd);
+	res->lr_name       = *name;
+	res->lr_type       = type;
+
+	cfs_hash_bd_lock(ns->ns_rs_hash, &bd, 1);
+	hnode = (version == cfs_hash_bd_version_get(&bd)) ?  NULL :
+		cfs_hash_bd_lookup_locked(ns->ns_rs_hash, &bd, (void *)name);
+
+	if (hnode) {
+		/* Someone won the race and already added the resource. */
+		cfs_hash_bd_unlock(ns->ns_rs_hash, &bd, 1);
+		/* Clean lu_ref for failed resource. */
+		lu_ref_fini(&res->lr_reference);
+		/* We have taken lr_lvb_mutex. Drop it. */
+		mutex_unlock(&res->lr_lvb_mutex);
+		kmem_cache_free(ldlm_resource_slab, res);
+lvbo_init:
+		res = hlist_entry(hnode, struct ldlm_resource, lr_hash);
+		/* Synchronize with regard to resource creation. */
+		if (ns->ns_lvbo && ns->ns_lvbo->lvbo_init) {
+			mutex_lock(&res->lr_lvb_mutex);
+			mutex_unlock(&res->lr_lvb_mutex);
+		}
+
+		if (unlikely(res->lr_lvb_len < 0)) {
+			rc = res->lr_lvb_len;
+			ldlm_resource_putref(res);
+			res = ERR_PTR(rc);
+		}
+		return res;
+	}
+	/* We won! Let's add the resource. */
+	cfs_hash_bd_add_locked(ns->ns_rs_hash, &bd, &res->lr_hash);
+	if (cfs_hash_bd_count_get(&bd) == 1)
+		ns_refcount = ldlm_namespace_get_return(ns);
+
+	cfs_hash_bd_unlock(ns->ns_rs_hash, &bd, 1);
+	if (ns->ns_lvbo && ns->ns_lvbo->lvbo_init) {
+		OBD_FAIL_TIMEOUT(OBD_FAIL_LDLM_CREATE_RESOURCE, 2);
+		rc = ns->ns_lvbo->lvbo_init(res);
+		if (rc < 0) {
+			CERROR("%s: lvbo_init failed for resource %#llx:%#llx: rc = %d\n",
+			       ns->ns_obd->obd_name, name->name[0],
+			       name->name[1], rc);
+			res->lr_lvb_len = rc;
+			mutex_unlock(&res->lr_lvb_mutex);
+			ldlm_resource_putref(res);
+			return ERR_PTR(rc);
+		}
+	}
+
+	/* We create resource with locked lr_lvb_mutex. */
+	mutex_unlock(&res->lr_lvb_mutex);
+
+	/* Let's see if we happened to be the very first resource in this
+	 * namespace. If so, and this is a client namespace, we need to move
+	 * the namespace into the active namespaces list to be patrolled by
+	 * the ldlm_poold.
+	 */
+	if (ns_refcount == 1) {
+		mutex_lock(ldlm_namespace_lock(LDLM_NAMESPACE_CLIENT));
+		ldlm_namespace_move_to_active_locked(ns, LDLM_NAMESPACE_CLIENT);
+		mutex_unlock(ldlm_namespace_lock(LDLM_NAMESPACE_CLIENT));
+	}
+
+	return res;
+}
+EXPORT_SYMBOL(ldlm_resource_get);
+
+static void __ldlm_resource_putref_final(struct cfs_hash_bd *bd,
+					 struct ldlm_resource *res)
+{
+	struct ldlm_ns_bucket *nsb = res->lr_ns_bucket;
+	struct ldlm_namespace *ns = nsb->nsb_namespace;
+
+	if (!list_empty(&res->lr_granted)) {
+		ldlm_resource_dump(D_ERROR, res);
+		LBUG();
+	}
+
+	if (!list_empty(&res->lr_waiting)) {
+		ldlm_resource_dump(D_ERROR, res);
+		LBUG();
+	}
+
+	cfs_hash_bd_del_locked(ns->ns_rs_hash,
+			       bd, &res->lr_hash);
+	lu_ref_fini(&res->lr_reference);
+	cfs_hash_bd_unlock(ns->ns_rs_hash, bd, 1);
+	if (ns->ns_lvbo && ns->ns_lvbo->lvbo_free)
+		ns->ns_lvbo->lvbo_free(res);
+	if (cfs_hash_bd_count_get(bd) == 0)
+		ldlm_namespace_put(ns);
+	kmem_cache_free(ldlm_resource_slab, res);
+}
+
+void ldlm_resource_putref(struct ldlm_resource *res)
+{
+	struct ldlm_namespace *ns = ldlm_res_to_ns(res);
+	struct cfs_hash_bd   bd;
+
+	LASSERT_ATOMIC_GT_LT(&res->lr_refcount, 0, LI_POISON);
+	CDEBUG(D_INFO, "putref res: %p count: %d\n",
+	       res, atomic_read(&res->lr_refcount) - 1);
+
+	cfs_hash_bd_get(ns->ns_rs_hash, &res->lr_name, &bd);
+	if (cfs_hash_bd_dec_and_lock(ns->ns_rs_hash, &bd, &res->lr_refcount))
+		__ldlm_resource_putref_final(&bd, res);
+}
+EXPORT_SYMBOL(ldlm_resource_putref);
+
+/**
+ * Add a lock into a given resource into specified lock list.
+ */
+void ldlm_resource_add_lock(struct ldlm_resource *res, struct list_head *head,
+			    struct ldlm_lock *lock)
+{
+	check_res_locked(res);
+
+	LDLM_DEBUG(lock, "About to add this lock:");
+
+	if (ldlm_is_destroyed(lock)) {
+		CDEBUG(D_OTHER, "Lock destroyed, not adding to resource\n");
+		return;
+	}
+
+	LASSERT(list_empty(&lock->l_res_link));
+
+	list_add_tail(&lock->l_res_link, head);
+}
+
+void ldlm_resource_unlink_lock(struct ldlm_lock *lock)
+{
+	int type = lock->l_resource->lr_type;
+
+	check_res_locked(lock->l_resource);
+	if (type == LDLM_IBITS || type == LDLM_PLAIN)
+		ldlm_unlink_lock_skiplist(lock);
+	else if (type == LDLM_EXTENT)
+		ldlm_extent_unlink_lock(lock);
+	list_del_init(&lock->l_res_link);
+}
+EXPORT_SYMBOL(ldlm_resource_unlink_lock);
+
+void ldlm_res2desc(struct ldlm_resource *res, struct ldlm_resource_desc *desc)
+{
+	desc->lr_type = res->lr_type;
+	desc->lr_name = res->lr_name;
+}
+
+/**
+ * Print information about all locks in all namespaces on this node to debug
+ * log.
+ */
+void ldlm_dump_all_namespaces(enum ldlm_side client, int level)
+{
+	struct ldlm_namespace *ns;
+
+	if (!((libcfs_debug | D_ERROR) & level))
+		return;
+
+	mutex_lock(ldlm_namespace_lock(client));
+
+	list_for_each_entry(ns, ldlm_namespace_list(client), ns_list_chain)
+		ldlm_namespace_dump(level, ns);
+
+	mutex_unlock(ldlm_namespace_lock(client));
+}
+
+static int ldlm_res_hash_dump(struct cfs_hash *hs, struct cfs_hash_bd *bd,
+			      struct hlist_node *hnode, void *arg)
+{
+	struct ldlm_resource *res = cfs_hash_object(hs, hnode);
+	int    level = (int)(unsigned long)arg;
+
+	lock_res(res);
+	ldlm_resource_dump(level, res);
+	unlock_res(res);
+
+	return 0;
+}
+
+/**
+ * Print information about all locks in this namespace on this node to debug
+ * log.
+ */
+void ldlm_namespace_dump(int level, struct ldlm_namespace *ns)
+{
+	if (!((libcfs_debug | D_ERROR) & level))
+		return;
+
+	CDEBUG(level, "--- Namespace: %s (rc: %d, side: client)\n",
+	       ldlm_ns_name(ns), atomic_read(&ns->ns_bref));
+
+	if (time_before(jiffies, ns->ns_next_dump))
+		return;
+
+	cfs_hash_for_each_nolock(ns->ns_rs_hash,
+				 ldlm_res_hash_dump,
+				 (void *)(unsigned long)level, 0);
+	spin_lock(&ns->ns_lock);
+	ns->ns_next_dump = jiffies + 10 * HZ;
+	spin_unlock(&ns->ns_lock);
+}
+
+/**
+ * Print information about all locks in this resource to debug log.
+ */
+void ldlm_resource_dump(int level, struct ldlm_resource *res)
+{
+	struct ldlm_lock *lock;
+	unsigned int granted = 0;
+
+	BUILD_BUG_ON(RES_NAME_SIZE != 4);
+
+	if (!((libcfs_debug | D_ERROR) & level))
+		return;
+
+	CDEBUG(level, "--- Resource: " DLDLMRES " (%p) refcount = %d\n",
+	       PLDLMRES(res), res, atomic_read(&res->lr_refcount));
+
+	if (!list_empty(&res->lr_granted)) {
+		CDEBUG(level, "Granted locks (in reverse order):\n");
+		list_for_each_entry_reverse(lock, &res->lr_granted,
+					    l_res_link) {
+			LDLM_DEBUG_LIMIT(level, lock, "###");
+			if (!(level & D_CANTMASK) &&
+			    ++granted > ldlm_dump_granted_max) {
+				CDEBUG(level,
+				       "only dump %d granted locks to avoid DDOS.\n",
+				       granted);
+				break;
+			}
+		}
+	}
+	if (!list_empty(&res->lr_waiting)) {
+		CDEBUG(level, "Waiting locks:\n");
+		list_for_each_entry(lock, &res->lr_waiting, l_res_link)
+			LDLM_DEBUG_LIMIT(level, lock, "###");
+	}
+}
+EXPORT_SYMBOL(ldlm_resource_dump);
diff --git a/drivers/staging/lustre/lustre/ptlrpc/ptlrpc_internal.h b/drivers/staging/lustre/lustre/ptlrpc/ptlrpc_internal.h
index 134b74234519..09ccb3fdabba 100644
--- a/drivers/staging/lustre/lustre/ptlrpc/ptlrpc_internal.h
+++ b/drivers/staging/lustre/lustre/ptlrpc/ptlrpc_internal.h
@@ -36,7 +36,7 @@
 #ifndef PTLRPC_INTERNAL_H
 #define PTLRPC_INTERNAL_H
 
-#include "../ldlm/ldlm_internal.h"
+#include "ldlm_internal.h"
 
 struct ldlm_namespace;
 struct obd_import;



From neilb at suse.com  Thu Jun  7 02:29:30 2018
From: neilb at suse.com (NeilBrown)
Date: Thu, 07 Jun 2018 12:29:30 +1000
Subject: [lustre-devel] A new drivers/staging/lustre
Message-ID: <874lifnxbp.fsf@notabene.neil.brown.name>


Greg's patch to remove lustre has now landed in this staging-next tree,
so I suspect it will get to Linus before too long.  So I have to find a
new place to work on lustre.

I've added 2 branches to
   git://git.neil.brown.name/linux

lustre:
   is based on Greg's patch that removes lustre, and starts with a
   revert of the patch, followed by a merge of v4.17.
   I plan to merge each release and RC from Linus, and also
   add lustre patches that I think are "ready".  That will usually mean
   they have been posted to this list at least a week earlier, and
   have not had a credible negative response (Acks and Reviewed-by
   would be nice).
   I plan to update this branch about once a week, and to never rebase
   it.

lustre-testing:
   is based on 'lustre' and has most of my current lustre-related work.
   It includes assorted patches that are not specifically for lustre
   (rhashtables mostly at the moment).  Patches will move from here
   to 'lustre' or to mainline when they are ready.
   I plan to update this branch on most days that I work on Lustre,
   and expect it to rebase frequently.

I'm happy to review and, if acceptable, apply patches from other
developers.  I have fairly high standards, but if I don't accept your
patch I'll explain why and possible help fix it.
I'm happy to accept enhancements and new features, but these need
to be of a quality that would be accepted upstream.
I'm only interested in client-side code at present - nothing that is
only used on the server.  I do want to include server-side eventually,
but I need some focus for now.

I hope to get to a stage where the code is of suitable quality that I
can submit it to Linux as a new filesystem.  I hope that will happen
this year.

I hope we can continue to work together to make all this happen.
(That's enough hope for now, time to get back to code).

Thanks,
NeilBrown
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180607/cee8d602/attachment.sig>

From jsimmons at infradead.org  Thu Jun  7 05:25:17 2018
From: jsimmons at infradead.org (James Simmons)
Date: Thu, 7 Jun 2018 06:25:17 +0100 (BST)
Subject: [lustre-devel] A new drivers/staging/lustre
In-Reply-To: <874lifnxbp.fsf@notabene.neil.brown.name>
References: <874lifnxbp.fsf@notabene.neil.brown.name>
Message-ID: <alpine.LFD.2.21.1806070557300.23188@casper.infradead.org>


> Greg's patch to remove lustre has now landed in this staging-next tree,
> so I suspect it will get to Linus before too long.  So I have to find a
> new place to work on lustre.
> 
> I've added 2 branches to
>    git://git.neil.brown.name/linux
> 
> lustre:
>    is based on Greg's patch that removes lustre, and starts with a
>    revert of the patch, followed by a merge of v4.17.
>    I plan to merge each release and RC from Linus, and also
>    add lustre patches that I think are "ready".  That will usually mean
>    they have been posted to this list at least a week earlier, and
>    have not had a credible negative response (Acks and Reviewed-by
>    would be nice).
>    I plan to update this branch about once a week, and to never rebase
>    it.

I know Oleg also started to play with a tree but I don't know if he can
keep it up like you can. I added the parties interested so they can bless
this tree if they want. Mainly Oleg wanted to see what breaks when moving
to fs directory and the proper UAPI headers directory as well.

Please be patience with me. I normally do this work on the weekend. I put
it into my test bed and try it out. Getting reviews can at times be 
challenging to get. Hopefully people at Cray and Intel are willing to
step in for this as well. Patrick from Cray has been most helpful.

> lustre-testing:
>    is based on 'lustre' and has most of my current lustre-related work.
>    It includes assorted patches that are not specifically for lustre
>    (rhashtables mostly at the moment).  Patches will move from here
>    to 'lustre' or to mainline when they are ready.
>    I plan to update this branch on most days that I work on Lustre,
>    and expect it to rebase frequently.

I had question about that. Some things in Lustre could in theory be merged
into the linux kernel proper. Can that be done still?
 
> I'm happy to review and, if acceptable, apply patches from other
> developers.  I have fairly high standards, but if I don't accept your
> patch I'll explain why and possible help fix it.

Also long as you talk to me :-) I'm an easy person to work with. If I 
refuse a patch with do the same. I might sometimes seem irrational 
but I have valid reasons. Well at least in my head.

We need to really layout the roadmap.

> I'm happy to accept enhancements and new features, but these need
> to be of a quality that would be accepted upstream.

Absolutely. This should be music to some peoples ears. 

> I'm only interested in client-side code at present - nothing that is
> only used on the server.  I do want to include server-side eventually,
> but I need some focus for now.

Make sense. Anyways the backend file systems used are ldiskfs which is
a heavily modified ext4 filesystem and ZFS on the server side. I doubt
the kernel would accept ZFS backend suppport and the changes Lustre
does to ext4 have been mostly merged but a few pieces are missing.
So pushing the server code at this point wouldn't benefit us.

> I hope to get to a stage where the code is of suitable quality that I
> can submit it to Linux as a new filesystem.  I hope that will happen
> this year.

Are you thinking going back into staging or straight to the fs tree

> I hope we can continue to work together to make all this happen.
> (That's enough hope for now, time to get back to code).

From neilb at suse.com  Thu Jun  7 09:48:47 2018
From: neilb at suse.com (NeilBrown)
Date: Thu, 07 Jun 2018 19:48:47 +1000
Subject: [lustre-devel] [PATCH 10/11] staging: lustre: move ldlm into
 ptlrpc
In-Reply-To: <alpine.LFD.2.21.1806070546210.23188@casper.infradead.org>
References: <152826510267.16761.14361003167157833896.stgit@noble>
 <152826511923.16761.9237280635711887801.stgit@noble>
 <alpine.LFD.2.21.1806070546210.23188@casper.infradead.org>
Message-ID: <87sh5zlyf4.fsf@notabene.neil.brown.name>

On Thu, Jun 07 2018, James Simmons wrote:

>> The ldlm code is built into the ptlrpc module, yet it lived in a
>> separate directory.  This requires filename editing in the Makefile
>> and make it difficult to e.g. build the .s file for code in ldlm.
>> 
>> All the ldlm files have distinctive names so confusion from having
>> ptlrpc and ldlm in the same directory is unlikely.  So move them all
>> into ptlrpc.
>
> Nak. The reason is it would be nice to keep the directory structure.
> What really needs to be done and Oleg has looked into it is to reduced
> the number of modules created down to two, one for LNet and the other
> lustre.ko. This also is a step in the right direction to remove the
> create struct obd_ops and struct md_ops pointer madness. Well their
> is the issue with obd echo client but we can deal with this at a later 
> date. Also the number of EXPORT_SYMBOLS and things will greatly reduce.

Yeah, you are probably right.
I had a bit of a look at how to build everything into a
single module.  You can do with by having a single make
file that lists parts from other directories - the same way
that ptlrpc includes files from ldlm - but that is rather ugly.

I've very nearly got it working using the lib-y infrastructure.
I can build lnet as a single module, but the dependency calc isn't
quite right so things happen in the wrong order.  The build
fails the first time because some files don't exist, then
succeeds on the second run.
Hopefully I'll figure out how to make it work tomorrow.

Thanks for the review,
NeilBrown
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180607/ec5325e0/attachment.sig>

From neilb at suse.com  Thu Jun  7 09:58:12 2018
From: neilb at suse.com (NeilBrown)
Date: Thu, 07 Jun 2018 19:58:12 +1000
Subject: [lustre-devel] A new drivers/staging/lustre
In-Reply-To: <alpine.LFD.2.21.1806070557300.23188@casper.infradead.org>
References: <874lifnxbp.fsf@notabene.neil.brown.name>
 <alpine.LFD.2.21.1806070557300.23188@casper.infradead.org>
Message-ID: <87po12ncjv.fsf@notabene.neil.brown.name>

On Thu, Jun 07 2018, James Simmons wrote:

>> Greg's patch to remove lustre has now landed in this staging-next tree,
>> so I suspect it will get to Linus before too long.  So I have to find a
>> new place to work on lustre.
>> 
>> I've added 2 branches to
>>    git://git.neil.brown.name/linux
>> 
>> lustre:
>>    is based on Greg's patch that removes lustre, and starts with a
>>    revert of the patch, followed by a merge of v4.17.
>>    I plan to merge each release and RC from Linus, and also
>>    add lustre patches that I think are "ready".  That will usually mean
>>    they have been posted to this list at least a week earlier, and
>>    have not had a credible negative response (Acks and Reviewed-by
>>    would be nice).
>>    I plan to update this branch about once a week, and to never rebase
>>    it.
>
> I know Oleg also started to play with a tree but I don't know if he can
> keep it up like you can. I added the parties interested so they can bless
> this tree if they want. Mainly Oleg wanted to see what breaks when moving
> to fs directory and the proper UAPI headers directory as well.

I'm certainly happy to contribute to a different tree instead, if
someone else is working towards getting lustre code suitable for
upstream.  I created a tree myself because I find that saying "we should"
is not nearly as effective as saying "I have".

>
> Please be patience with me. I normally do this work on the weekend. I put
> it into my test bed and try it out. Getting reviews can at times be 
> challenging to get. Hopefully people at Cray and Intel are willing to
> step in for this as well. Patrick from Cray has been most helpful.
>
>> lustre-testing:
>>    is based on 'lustre' and has most of my current lustre-related work.
>>    It includes assorted patches that are not specifically for lustre
>>    (rhashtables mostly at the moment).  Patches will move from here
>>    to 'lustre' or to mainline when they are ready.
>>    I plan to update this branch on most days that I work on Lustre,
>>    and expect it to rebase frequently.
>
> I had question about that. Some things in Lustre could in theory be merged
> into the linux kernel proper. Can that be done still?

What things?

If it measurably benefits the kernel proper, then I suspect it might be
worth submitting.  Things can go direct without going though staging -
they just have to be of good quality with good justification (and
sometimes lots of patience).

>  
>> I'm happy to review and, if acceptable, apply patches from other
>> developers.  I have fairly high standards, but if I don't accept your
>> patch I'll explain why and possible help fix it.
>
> Also long as you talk to me :-) I'm an easy person to work with. If I 
> refuse a patch with do the same. I might sometimes seem irrational 
> but I have valid reasons. Well at least in my head.
>
> We need to really layout the roadmap.

I have very little faith in road maps - I prefer to make steps.  Once we
have made all the steps, we can look back and see what the map looked
like in retrospect.

The most I'm interested in is "client first, then server".
But feel free to propose something - it is helps you then it could be
useful.

>
>> I'm happy to accept enhancements and new features, but these need
>> to be of a quality that would be accepted upstream.
>
> Absolutely. This should be music to some peoples ears. 
>
>> I'm only interested in client-side code at present - nothing that is
>> only used on the server.  I do want to include server-side eventually,
>> but I need some focus for now.
>
> Make sense. Anyways the backend file systems used are ldiskfs which is
> a heavily modified ext4 filesystem and ZFS on the server side. I doubt
> the kernel would accept ZFS backend suppport and the changes Lustre
> does to ext4 have been mostly merged but a few pieces are missing.
> So pushing the server code at this point wouldn't benefit us.
>
>> I hope to get to a stage where the code is of suitable quality that I
>> can submit it to Linux as a new filesystem.  I hope that will happen
>> this year.
>
> Are you thinking going back into staging or straight to the fs tree

Greg has said he doesn't want it in staging.  So no, I'm not thinking of
anything going to staging.  I'm thinking of getting enough of a client
in reasonable shape that people can review it without feeling sick or
getting angry.

Thanks,
NeilBrown


>
>> I hope we can continue to work together to make all this happen.
>> (That's enough hope for now, time to get back to code).
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180607/b3f1a5b3/attachment.sig>

From doucharek at cray.com  Thu Jun  7 13:07:44 2018
From: doucharek at cray.com (Doug Oucharek)
Date: Thu, 7 Jun 2018 13:07:44 +0000
Subject: [lustre-devel] A new drivers/staging/lustre
In-Reply-To: <87po12ncjv.fsf@notabene.neil.brown.name>
References: <874lifnxbp.fsf@notabene.neil.brown.name>
 <alpine.LFD.2.21.1806070557300.23188@casper.infradead.org>
 <87po12ncjv.fsf@notabene.neil.brown.name>
Message-ID: <A139C7E5-0406-4078-B579-8DCF5B85F10F@cray.com>

What is the focus of landings in this tree?  There are two things needing to be done for an upstream Lustre:


  *   Get the source code to meet the Linux guidelines so it is acceptable to be in mainline.
  *   Get the binary product to have all the features and bug fixes that are in the Intel community tree so end users are interested in using the upstream version (users are unlikely to use a version of Lustre which is not current).

For the now-deleted staging area, we were supposed to be focusing on the first item but were submitting patches for the second item (syncing with Intel tree).  In my opinion, this is the core reason for never being able to get out of staging and getting deleted.

There are some very big (as in code size) features missing from upstream.  For example, Multi-Rail.  When should that be pushed relative to code cleanups?

Doug

On Jun 7, 2018, at 4:58 AM, NeilBrown <neilb at suse.com<mailto:neilb at suse.com>> wrote:

On Thu, Jun 07 2018, James Simmons wrote:

Greg's patch to remove lustre has now landed in this staging-next tree,
so I suspect it will get to Linus before too long.  So I have to find a
new place to work on lustre.

I've added 2 branches to
  git://git.neil.brown.name/linux

lustre:
  is based on Greg's patch that removes lustre, and starts with a
  revert of the patch, followed by a merge of v4.17.
  I plan to merge each release and RC from Linus, and also
  add lustre patches that I think are "ready".  That will usually mean
  they have been posted to this list at least a week earlier, and
  have not had a credible negative response (Acks and Reviewed-by
  would be nice).
  I plan to update this branch about once a week, and to never rebase
  it.

I know Oleg also started to play with a tree but I don't know if he can
keep it up like you can. I added the parties interested so they can bless
this tree if they want. Mainly Oleg wanted to see what breaks when moving
to fs directory and the proper UAPI headers directory as well.

I'm certainly happy to contribute to a different tree instead, if
someone else is working towards getting lustre code suitable for
upstream.  I created a tree myself because I find that saying "we should"
is not nearly as effective as saying "I have".


Please be patience with me. I normally do this work on the weekend. I put
it into my test bed and try it out. Getting reviews can at times be
challenging to get. Hopefully people at Cray and Intel are willing to
step in for this as well. Patrick from Cray has been most helpful.

lustre-testing:
  is based on 'lustre' and has most of my current lustre-related work.
  It includes assorted patches that are not specifically for lustre
  (rhashtables mostly at the moment).  Patches will move from here
  to 'lustre' or to mainline when they are ready.
  I plan to update this branch on most days that I work on Lustre,
  and expect it to rebase frequently.

I had question about that. Some things in Lustre could in theory be merged
into the linux kernel proper. Can that be done still?

What things?

If it measurably benefits the kernel proper, then I suspect it might be
worth submitting.  Things can go direct without going though staging -
they just have to be of good quality with good justification (and
sometimes lots of patience).


I'm happy to review and, if acceptable, apply patches from other
developers.  I have fairly high standards, but if I don't accept your
patch I'll explain why and possible help fix it.

Also long as you talk to me :-) I'm an easy person to work with. If I
refuse a patch with do the same. I might sometimes seem irrational
but I have valid reasons. Well at least in my head.

We need to really layout the roadmap.

I have very little faith in road maps - I prefer to make steps.  Once we
have made all the steps, we can look back and see what the map looked
like in retrospect.

The most I'm interested in is "client first, then server".
But feel free to propose something - it is helps you then it could be
useful.


I'm happy to accept enhancements and new features, but these need
to be of a quality that would be accepted upstream.

Absolutely. This should be music to some peoples ears.

I'm only interested in client-side code at present - nothing that is
only used on the server.  I do want to include server-side eventually,
but I need some focus for now.

Make sense. Anyways the backend file systems used are ldiskfs which is
a heavily modified ext4 filesystem and ZFS on the server side. I doubt
the kernel would accept ZFS backend suppport and the changes Lustre
does to ext4 have been mostly merged but a few pieces are missing.
So pushing the server code at this point wouldn't benefit us.

I hope to get to a stage where the code is of suitable quality that I
can submit it to Linux as a new filesystem.  I hope that will happen
this year.

Are you thinking going back into staging or straight to the fs tree

Greg has said he doesn't want it in staging.  So no, I'm not thinking of
anything going to staging.  I'm thinking of getting enough of a client
in reasonable shape that people can review it without feeling sick or
getting angry.

Thanks,
NeilBrown



I hope we can continue to work together to make all this happen.
(That's enough hope for now, time to get back to code).
_______________________________________________
lustre-devel mailing list
lustre-devel at lists.lustre.org<mailto:lustre-devel at lists.lustre.org>
http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180607/dee00f7d/attachment.html>

From paf at cray.com  Thu Jun  7 13:25:03 2018
From: paf at cray.com (Patrick Farrell)
Date: Thu, 7 Jun 2018 13:25:03 +0000
Subject: [lustre-devel] A new drivers/staging/lustre
In-Reply-To: <A139C7E5-0406-4078-B579-8DCF5B85F10F@cray.com>
References: <874lifnxbp.fsf@notabene.neil.brown.name>
 <alpine.LFD.2.21.1806070557300.23188@casper.infradead.org>
 <87po12ncjv.fsf@notabene.neil.brown.name>,
 <A139C7E5-0406-4078-B579-8DCF5B85F10F@cray.com>
Message-ID: <DM3PR1101MB1213BB514F8D6E25F725BB1DCB640@DM3PR1101MB1213.namprd11.prod.outlook.com>

Doug,

Another thought about the core reason.

Commitment to this.  The existing code state (weird, abandoned 2.4.something code) and merge rules seemingly made it impossible to shift development in this direction in any meaningful way, so perhaps it's chicken and egg...  but as long as Lustre is released and developed primarily out of tree, I can't see this working.  Would it just be a "sync everything but still do releases" approach?  Is that viable?  Etc.

Thoughts appreciated.

- Patrick


________________________________
From: lustre-devel <lustre-devel-bounces at lists.lustre.org> on behalf of Doug Oucharek <doucharek at cray.com>
Sent: Thursday, June 7, 2018 8:07:44 AM
To: NeilBrown
Cc: Oleg Drokin; lustre-devel
Subject: Re: [lustre-devel] A new drivers/staging/lustre

What is the focus of landings in this tree?  There are two things needing to be done for an upstream Lustre:


  *   Get the source code to meet the Linux guidelines so it is acceptable to be in mainline.
  *   Get the binary product to have all the features and bug fixes that are in the Intel community tree so end users are interested in using the upstream version (users are unlikely to use a version of Lustre which is not current).

For the now-deleted staging area, we were supposed to be focusing on the first item but were submitting patches for the second item (syncing with Intel tree).  In my opinion, this is the core reason for never being able to get out of staging and getting deleted.

There are some very big (as in code size) features missing from upstream.  For example, Multi-Rail.  When should that be pushed relative to code cleanups?

Doug

On Jun 7, 2018, at 4:58 AM, NeilBrown <neilb at suse.com<mailto:neilb at suse.com>> wrote:

On Thu, Jun 07 2018, James Simmons wrote:

Greg's patch to remove lustre has now landed in this staging-next tree,
so I suspect it will get to Linus before too long.  So I have to find a
new place to work on lustre.

I've added 2 branches to
  git://git.neil.brown.name/linux

lustre:
  is based on Greg's patch that removes lustre, and starts with a
  revert of the patch, followed by a merge of v4.17.
  I plan to merge each release and RC from Linus, and also
  add lustre patches that I think are "ready".  That will usually mean
  they have been posted to this list at least a week earlier, and
  have not had a credible negative response (Acks and Reviewed-by
  would be nice).
  I plan to update this branch about once a week, and to never rebase
  it.

I know Oleg also started to play with a tree but I don't know if he can
keep it up like you can. I added the parties interested so they can bless
this tree if they want. Mainly Oleg wanted to see what breaks when moving
to fs directory and the proper UAPI headers directory as well.

I'm certainly happy to contribute to a different tree instead, if
someone else is working towards getting lustre code suitable for
upstream.  I created a tree myself because I find that saying "we should"
is not nearly as effective as saying "I have".


Please be patience with me. I normally do this work on the weekend. I put
it into my test bed and try it out. Getting reviews can at times be
challenging to get. Hopefully people at Cray and Intel are willing to
step in for this as well. Patrick from Cray has been most helpful.

lustre-testing:
  is based on 'lustre' and has most of my current lustre-related work.
  It includes assorted patches that are not specifically for lustre
  (rhashtables mostly at the moment).  Patches will move from here
  to 'lustre' or to mainline when they are ready.
  I plan to update this branch on most days that I work on Lustre,
  and expect it to rebase frequently.

I had question about that. Some things in Lustre could in theory be merged
into the linux kernel proper. Can that be done still?

What things?

If it measurably benefits the kernel proper, then I suspect it might be
worth submitting.  Things can go direct without going though staging -
they just have to be of good quality with good justification (and
sometimes lots of patience).


I'm happy to review and, if acceptable, apply patches from other
developers.  I have fairly high standards, but if I don't accept your
patch I'll explain why and possible help fix it.

Also long as you talk to me :-) I'm an easy person to work with. If I
refuse a patch with do the same. I might sometimes seem irrational
but I have valid reasons. Well at least in my head.

We need to really layout the roadmap.

I have very little faith in road maps - I prefer to make steps.  Once we
have made all the steps, we can look back and see what the map looked
like in retrospect.

The most I'm interested in is "client first, then server".
But feel free to propose something - it is helps you then it could be
useful.


I'm happy to accept enhancements and new features, but these need
to be of a quality that would be accepted upstream.

Absolutely. This should be music to some peoples ears.

I'm only interested in client-side code at present - nothing that is
only used on the server.  I do want to include server-side eventually,
but I need some focus for now.

Make sense. Anyways the backend file systems used are ldiskfs which is
a heavily modified ext4 filesystem and ZFS on the server side. I doubt
the kernel would accept ZFS backend suppport and the changes Lustre
does to ext4 have been mostly merged but a few pieces are missing.
So pushing the server code at this point wouldn't benefit us.

I hope to get to a stage where the code is of suitable quality that I
can submit it to Linux as a new filesystem.  I hope that will happen
this year.

Are you thinking going back into staging or straight to the fs tree

Greg has said he doesn't want it in staging.  So no, I'm not thinking of
anything going to staging.  I'm thinking of getting enough of a client
in reasonable shape that people can review it without feeling sick or
getting angry.

Thanks,
NeilBrown



I hope we can continue to work together to make all this happen.
(That's enough hope for now, time to get back to code).
_______________________________________________
lustre-devel mailing list
lustre-devel at lists.lustre.org<mailto:lustre-devel at lists.lustre.org>
http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180607/8f0662bb/attachment.html>

From jsimmons at infradead.org  Thu Jun  7 04:51:47 2018
From: jsimmons at infradead.org (James Simmons)
Date: Thu, 7 Jun 2018 05:51:47 +0100 (BST)
Subject: [lustre-devel] [PATCH 10/11] staging: lustre: move ldlm into
 ptlrpc
In-Reply-To: <152826511923.16761.9237280635711887801.stgit@noble>
References: <152826510267.16761.14361003167157833896.stgit@noble>
 <152826511923.16761.9237280635711887801.stgit@noble>
Message-ID: <alpine.LFD.2.21.1806070546210.23188@casper.infradead.org>


> The ldlm code is built into the ptlrpc module, yet it lived in a
> separate directory.  This requires filename editing in the Makefile
> and make it difficult to e.g. build the .s file for code in ldlm.
> 
> All the ldlm files have distinctive names so confusion from having
> ptlrpc and ldlm in the same directory is unlikely.  So move them all
> into ptlrpc.

Nak. The reason is it would be nice to keep the directory structure.
What really needs to be done and Oleg has looked into it is to reduced
the number of modules created down to two, one for LNet and the other
lustre.ko. This also is a step in the right direction to remove the
create struct obd_ops and struct md_ops pointer madness. Well their
is the issue with obd echo client but we can deal with this at a later 
date. Also the number of EXPORT_SYMBOLS and things will greatly reduce.

 
> Signed-off-by: NeilBrown <neilb at suse.com>
> ---
>  drivers/staging/lustre/lustre/ldlm/l_lock.c        |   73 -
>  drivers/staging/lustre/lustre/ldlm/ldlm_extent.c   |  206 --
>  drivers/staging/lustre/lustre/ldlm/ldlm_flock.c    |  486 -----
>  .../staging/lustre/lustre/ldlm/ldlm_inodebits.c    |   69 -
>  drivers/staging/lustre/lustre/ldlm/ldlm_internal.h |  329 ---
>  drivers/staging/lustre/lustre/ldlm/ldlm_lib.c      |  842 --------
>  drivers/staging/lustre/lustre/ldlm/ldlm_lock.c     | 2103 --------------------
>  drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c    | 1154 -----------
>  drivers/staging/lustre/lustre/ldlm/ldlm_plain.c    |   68 -
>  drivers/staging/lustre/lustre/ldlm/ldlm_pool.c     | 1013 ----------
>  drivers/staging/lustre/lustre/ldlm/ldlm_request.c  | 2033 -------------------
>  drivers/staging/lustre/lustre/ldlm/ldlm_resource.c | 1318 -------------
>  drivers/staging/lustre/lustre/ptlrpc/Makefile      |   14 
>  drivers/staging/lustre/lustre/ptlrpc/l_lock.c      |   73 +
>  drivers/staging/lustre/lustre/ptlrpc/ldlm_extent.c |  206 ++
>  drivers/staging/lustre/lustre/ptlrpc/ldlm_flock.c  |  486 +++++
>  .../staging/lustre/lustre/ptlrpc/ldlm_inodebits.c  |   69 +
>  .../staging/lustre/lustre/ptlrpc/ldlm_internal.h   |  329 +++
>  drivers/staging/lustre/lustre/ptlrpc/ldlm_lib.c    |  842 ++++++++
>  drivers/staging/lustre/lustre/ptlrpc/ldlm_lock.c   | 2103 ++++++++++++++++++++
>  drivers/staging/lustre/lustre/ptlrpc/ldlm_lockd.c  | 1154 +++++++++++
>  drivers/staging/lustre/lustre/ptlrpc/ldlm_plain.c  |   68 +
>  drivers/staging/lustre/lustre/ptlrpc/ldlm_pool.c   | 1013 ++++++++++
>  .../staging/lustre/lustre/ptlrpc/ldlm_request.c    | 2033 +++++++++++++++++++
>  .../staging/lustre/lustre/ptlrpc/ldlm_resource.c   | 1318 +++++++++++++
>  .../staging/lustre/lustre/ptlrpc/ptlrpc_internal.h |    2 
>  26 files changed, 9702 insertions(+), 9702 deletions(-)
>  delete mode 100644 drivers/staging/lustre/lustre/ldlm/l_lock.c
>  delete mode 100644 drivers/staging/lustre/lustre/ldlm/ldlm_extent.c
>  delete mode 100644 drivers/staging/lustre/lustre/ldlm/ldlm_flock.c
>  delete mode 100644 drivers/staging/lustre/lustre/ldlm/ldlm_inodebits.c
>  delete mode 100644 drivers/staging/lustre/lustre/ldlm/ldlm_internal.h
>  delete mode 100644 drivers/staging/lustre/lustre/ldlm/ldlm_lib.c
>  delete mode 100644 drivers/staging/lustre/lustre/ldlm/ldlm_lock.c
>  delete mode 100644 drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c
>  delete mode 100644 drivers/staging/lustre/lustre/ldlm/ldlm_plain.c
>  delete mode 100644 drivers/staging/lustre/lustre/ldlm/ldlm_pool.c
>  delete mode 100644 drivers/staging/lustre/lustre/ldlm/ldlm_request.c
>  delete mode 100644 drivers/staging/lustre/lustre/ldlm/ldlm_resource.c
>  create mode 100644 drivers/staging/lustre/lustre/ptlrpc/l_lock.c
>  create mode 100644 drivers/staging/lustre/lustre/ptlrpc/ldlm_extent.c
>  create mode 100644 drivers/staging/lustre/lustre/ptlrpc/ldlm_flock.c
>  create mode 100644 drivers/staging/lustre/lustre/ptlrpc/ldlm_inodebits.c
>  create mode 100644 drivers/staging/lustre/lustre/ptlrpc/ldlm_internal.h
>  create mode 100644 drivers/staging/lustre/lustre/ptlrpc/ldlm_lib.c
>  create mode 100644 drivers/staging/lustre/lustre/ptlrpc/ldlm_lock.c
>  create mode 100644 drivers/staging/lustre/lustre/ptlrpc/ldlm_lockd.c
>  create mode 100644 drivers/staging/lustre/lustre/ptlrpc/ldlm_plain.c
>  create mode 100644 drivers/staging/lustre/lustre/ptlrpc/ldlm_pool.c
>  create mode 100644 drivers/staging/lustre/lustre/ptlrpc/ldlm_request.c
>  create mode 100644 drivers/staging/lustre/lustre/ptlrpc/ldlm_resource.c
> 
> diff --git a/drivers/staging/lustre/lustre/ldlm/l_lock.c b/drivers/staging/lustre/lustre/ldlm/l_lock.c
> deleted file mode 100644
> index 296259aa51e6..000000000000
> --- a/drivers/staging/lustre/lustre/ldlm/l_lock.c
> +++ /dev/null
> @@ -1,73 +0,0 @@
> -// SPDX-License-Identifier: GPL-2.0
> -/*
> - * GPL HEADER START
> - *
> - * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
> - *
> - * This program is free software; you can redistribute it and/or modify
> - * it under the terms of the GNU General Public License version 2 only,
> - * as published by the Free Software Foundation.
> - *
> - * This program is distributed in the hope that it will be useful, but
> - * WITHOUT ANY WARRANTY; without even the implied warranty of
> - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> - * General Public License version 2 for more details (a copy is included
> - * in the LICENSE file that accompanied this code).
> - *
> - * You should have received a copy of the GNU General Public License
> - * version 2 along with this program; If not, see
> - * http://www.gnu.org/licenses/gpl-2.0.html
> - *
> - * GPL HEADER END
> - */
> -/*
> - * Copyright (c) 2002, 2010, Oracle and/or its affiliates. All rights reserved.
> - * Use is subject to license terms.
> - *
> - * Copyright (c) 2012, Intel Corporation.
> - */
> -/*
> - * This file is part of Lustre, http://www.lustre.org/
> - * Lustre is a trademark of Sun Microsystems, Inc.
> - */
> -
> -#define DEBUG_SUBSYSTEM S_LDLM
> -
> -#include <lustre_dlm.h>
> -#include <lustre_lib.h>
> -
> -/**
> - * Lock a lock and its resource.
> - *
> - * LDLM locking uses resource to serialize access to locks
> - * but there is a case when we change resource of lock upon
> - * enqueue reply. We rely on lock->l_resource = new_res
> - * being an atomic operation.
> - */
> -struct ldlm_resource *lock_res_and_lock(struct ldlm_lock *lock)
> -				__acquires(&lock->l_lock)
> -				__acquires(&lock->l_resource->lr_lock)
> -{
> -	spin_lock(&lock->l_lock);
> -
> -	lock_res(lock->l_resource);
> -
> -	ldlm_set_res_locked(lock);
> -	return lock->l_resource;
> -}
> -EXPORT_SYMBOL(lock_res_and_lock);
> -
> -/**
> - * Unlock a lock and its resource previously locked with lock_res_and_lock
> - */
> -void unlock_res_and_lock(struct ldlm_lock *lock)
> -		__releases(&lock->l_resource->lr_lock)
> -		__releases(&lock->l_lock)
> -{
> -	/* on server-side resource of lock doesn't change */
> -	ldlm_clear_res_locked(lock);
> -
> -	unlock_res(lock->l_resource);
> -	spin_unlock(&lock->l_lock);
> -}
> -EXPORT_SYMBOL(unlock_res_and_lock);
> diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_extent.c b/drivers/staging/lustre/lustre/ldlm/ldlm_extent.c
> deleted file mode 100644
> index 225c023b0bba..000000000000
> --- a/drivers/staging/lustre/lustre/ldlm/ldlm_extent.c
> +++ /dev/null
> @@ -1,206 +0,0 @@
> -// SPDX-License-Identifier: GPL-2.0
> -/*
> - * GPL HEADER START
> - *
> - * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
> - *
> - * This program is free software; you can redistribute it and/or modify
> - * it under the terms of the GNU General Public License version 2 only,
> - * as published by the Free Software Foundation.
> - *
> - * This program is distributed in the hope that it will be useful, but
> - * WITHOUT ANY WARRANTY; without even the implied warranty of
> - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> - * General Public License version 2 for more details (a copy is included
> - * in the LICENSE file that accompanied this code).
> - *
> - * You should have received a copy of the GNU General Public License
> - * version 2 along with this program; If not, see
> - * http://www.gnu.org/licenses/gpl-2.0.html
> - *
> - * GPL HEADER END
> - */
> -/*
> - * Copyright (c) 2002, 2010, Oracle and/or its affiliates. All rights reserved.
> - * Use is subject to license terms.
> - *
> - * Copyright (c) 2010, 2012, Intel Corporation.
> - */
> -/*
> - * This file is part of Lustre, http://www.lustre.org/
> - * Lustre is a trademark of Sun Microsystems, Inc.
> - *
> - * lustre/ldlm/ldlm_extent.c
> - *
> - * Author: Peter Braam <braam at clusterfs.com>
> - * Author: Phil Schwan <phil at clusterfs.com>
> - */
> -
> -/**
> - * This file contains implementation of EXTENT lock type
> - *
> - * EXTENT lock type is for locking a contiguous range of values, represented
> - * by 64-bit starting and ending offsets (inclusive). There are several extent
> - * lock modes, some of which may be mutually incompatible. Extent locks are
> - * considered incompatible if their modes are incompatible and their extents
> - * intersect.  See the lock mode compatibility matrix in lustre_dlm.h.
> - */
> -
> -#define DEBUG_SUBSYSTEM S_LDLM
> -#include <lustre_dlm.h>
> -#include <obd_support.h>
> -#include <obd.h>
> -#include <obd_class.h>
> -#include <lustre_lib.h>
> -#include "ldlm_internal.h"
> -#include <linux/interval_tree_generic.h>
> -
> -#define START(node) ((node)->l_policy_data.l_extent.start)
> -#define LAST(node) ((node)->l_policy_data.l_extent.end)
> -INTERVAL_TREE_DEFINE(struct ldlm_lock, l_rb, __u64, __subtree_last,
> -		     START, LAST, static, extent);
> -
> -/* When a lock is cancelled by a client, the KMS may undergo change if this
> - * is the "highest lock".  This function returns the new KMS value.
> - * Caller must hold lr_lock already.
> - *
> - * NB: A lock on [x,y] protects a KMS of up to y + 1 bytes!
> - */
> -__u64 ldlm_extent_shift_kms(struct ldlm_lock *lock, __u64 old_kms)
> -{
> -	struct ldlm_resource *res = lock->l_resource;
> -	struct ldlm_lock *lck;
> -	__u64 kms = 0;
> -
> -	/* don't let another thread in ldlm_extent_shift_kms race in
> -	 * just after we finish and take our lock into account in its
> -	 * calculation of the kms
> -	 */
> -	ldlm_set_kms_ignore(lock);
> -
> -	list_for_each_entry(lck, &res->lr_granted, l_res_link) {
> -
> -		if (ldlm_is_kms_ignore(lck))
> -			continue;
> -
> -		if (lck->l_policy_data.l_extent.end >= old_kms)
> -			return old_kms;
> -
> -		/* This extent _has_ to be smaller than old_kms (checked above)
> -		 * so kms can only ever be smaller or the same as old_kms.
> -		 */
> -		if (lck->l_policy_data.l_extent.end + 1 > kms)
> -			kms = lck->l_policy_data.l_extent.end + 1;
> -	}
> -	LASSERTF(kms <= old_kms, "kms %llu old_kms %llu\n", kms, old_kms);
> -
> -	return kms;
> -}
> -EXPORT_SYMBOL(ldlm_extent_shift_kms);
> -
> -static inline int lock_mode_to_index(enum ldlm_mode mode)
> -{
> -	int index;
> -
> -	LASSERT(mode != 0);
> -	LASSERT(is_power_of_2(mode));
> -	for (index = -1; mode; index++)
> -		mode >>= 1;
> -	LASSERT(index < LCK_MODE_NUM);
> -	return index;
> -}
> -
> -/** Add newly granted lock into interval tree for the resource. */
> -void ldlm_extent_add_lock(struct ldlm_resource *res,
> -			  struct ldlm_lock *lock)
> -{
> -	struct ldlm_interval_tree *tree;
> -	int idx;
> -
> -	LASSERT(lock->l_granted_mode == lock->l_req_mode);
> -
> -	LASSERT(RB_EMPTY_NODE(&lock->l_rb));
> -
> -	idx = lock_mode_to_index(lock->l_granted_mode);
> -	LASSERT(lock->l_granted_mode == 1 << idx);
> -	LASSERT(lock->l_granted_mode == res->lr_itree[idx].lit_mode);
> -
> -	tree = &res->lr_itree[idx];
> -	extent_insert(lock, &tree->lit_root);
> -	tree->lit_size++;
> -
> -	/* even though we use interval tree to manage the extent lock, we also
> -	 * add the locks into grant list, for debug purpose, ..
> -	 */
> -	ldlm_resource_add_lock(res, &res->lr_granted, lock);
> -
> -	if (OBD_FAIL_CHECK(OBD_FAIL_LDLM_GRANT_CHECK)) {
> -		struct ldlm_lock *lck;
> -
> -		list_for_each_entry_reverse(lck, &res->lr_granted,
> -					    l_res_link) {
> -			if (lck == lock)
> -				continue;
> -			if (lockmode_compat(lck->l_granted_mode,
> -					    lock->l_granted_mode))
> -				continue;
> -			if (ldlm_extent_overlap(&lck->l_req_extent,
> -						&lock->l_req_extent)) {
> -				CDEBUG(D_ERROR,
> -				       "granting conflicting lock %p %p\n",
> -				       lck, lock);
> -				ldlm_resource_dump(D_ERROR, res);
> -				LBUG();
> -			}
> -		}
> -	}
> -}
> -
> -/** Remove cancelled lock from resource interval tree. */
> -void ldlm_extent_unlink_lock(struct ldlm_lock *lock)
> -{
> -	struct ldlm_resource *res = lock->l_resource;
> -	struct ldlm_interval_tree *tree;
> -	int idx;
> -
> -	if (RB_EMPTY_NODE(&lock->l_rb)) /* duplicate unlink */
> -		return;
> -
> -	idx = lock_mode_to_index(lock->l_granted_mode);
> -	LASSERT(lock->l_granted_mode == 1 << idx);
> -	tree = &res->lr_itree[idx];
> -
> -	tree->lit_size--;
> -	extent_remove(lock, &tree->lit_root);
> -}
> -
> -void ldlm_extent_policy_wire_to_local(const union ldlm_wire_policy_data *wpolicy,
> -				      union ldlm_policy_data *lpolicy)
> -{
> -	lpolicy->l_extent.start = wpolicy->l_extent.start;
> -	lpolicy->l_extent.end = wpolicy->l_extent.end;
> -	lpolicy->l_extent.gid = wpolicy->l_extent.gid;
> -}
> -
> -void ldlm_extent_policy_local_to_wire(const union ldlm_policy_data *lpolicy,
> -				      union ldlm_wire_policy_data *wpolicy)
> -{
> -	memset(wpolicy, 0, sizeof(*wpolicy));
> -	wpolicy->l_extent.start = lpolicy->l_extent.start;
> -	wpolicy->l_extent.end = lpolicy->l_extent.end;
> -	wpolicy->l_extent.gid = lpolicy->l_extent.gid;
> -}
> -
> -void ldlm_extent_search(struct rb_root_cached *root,
> -			__u64 start, __u64 end,
> -			bool (*matches)(struct ldlm_lock *lock, void *data),
> -			void *data)
> -{
> -	struct ldlm_lock *lock;
> -
> -	for (lock = extent_iter_first(root, start, end);
> -	     lock;
> -	     lock = extent_iter_next(lock, start, end))
> -		if (matches(lock, data))
> -			break;
> -}
> diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_flock.c b/drivers/staging/lustre/lustre/ldlm/ldlm_flock.c
> deleted file mode 100644
> index 94f3b1e49896..000000000000
> --- a/drivers/staging/lustre/lustre/ldlm/ldlm_flock.c
> +++ /dev/null
> @@ -1,486 +0,0 @@
> -// SPDX-License-Identifier: GPL-2.0
> -/*
> - * GPL HEADER START
> - *
> - * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
> - *
> - * This program is free software; you can redistribute it and/or modify
> - * it under the terms of the GNU General Public License version 2 only,
> - * as published by the Free Software Foundation.
> - *
> - * This program is distributed in the hope that it will be useful, but
> - * WITHOUT ANY WARRANTY; without even the implied warranty of
> - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> - * General Public License version 2 for more details (a copy is included
> - * in the LICENSE file that accompanied this code).
> - *
> - * You should have received a copy of the GNU General Public License
> - * version 2 along with this program; If not, see
> - * http://www.gnu.org/licenses/gpl-2.0.html
> - *
> - * GPL HEADER END
> - */
> -/*
> - * Copyright (c) 2003 Hewlett-Packard Development Company LP.
> - * Developed under the sponsorship of the US Government under
> - * Subcontract No. B514193
> - *
> - * Copyright (c) 2003, 2010, Oracle and/or its affiliates. All rights reserved.
> - * Use is subject to license terms.
> - *
> - * Copyright (c) 2010, 2012, Intel Corporation.
> - */
> -/*
> - * This file is part of Lustre, http://www.lustre.org/
> - * Lustre is a trademark of Sun Microsystems, Inc.
> - */
> -
> -/**
> - * This file implements POSIX lock type for Lustre.
> - * Its policy properties are start and end of extent and PID.
> - *
> - * These locks are only done through MDS due to POSIX semantics requiring
> - * e.g. that locks could be only partially released and as such split into
> - * two parts, and also that two adjacent locks from the same process may be
> - * merged into a single wider lock.
> - *
> - * Lock modes are mapped like this:
> - * PR and PW for READ and WRITE locks
> - * NL to request a releasing of a portion of the lock
> - *
> - * These flock locks never timeout.
> - */
> -
> -#define DEBUG_SUBSYSTEM S_LDLM
> -
> -#include <lustre_dlm.h>
> -#include <obd_support.h>
> -#include <obd_class.h>
> -#include <lustre_lib.h>
> -#include <linux/list.h>
> -#include "ldlm_internal.h"
> -
> -static inline int
> -ldlm_same_flock_owner(struct ldlm_lock *lock, struct ldlm_lock *new)
> -{
> -	return((new->l_policy_data.l_flock.owner ==
> -		lock->l_policy_data.l_flock.owner) &&
> -	       (new->l_export == lock->l_export));
> -}
> -
> -static inline int
> -ldlm_flocks_overlap(struct ldlm_lock *lock, struct ldlm_lock *new)
> -{
> -	return((new->l_policy_data.l_flock.start <=
> -		lock->l_policy_data.l_flock.end) &&
> -	       (new->l_policy_data.l_flock.end >=
> -		lock->l_policy_data.l_flock.start));
> -}
> -
> -static inline void
> -ldlm_flock_destroy(struct ldlm_lock *lock, enum ldlm_mode mode)
> -{
> -	LDLM_DEBUG(lock, "%s(mode: %d)",
> -		   __func__, mode);
> -
> -	list_del_init(&lock->l_res_link);
> -
> -	/* client side - set a flag to prevent sending a CANCEL */
> -	lock->l_flags |= LDLM_FL_LOCAL_ONLY | LDLM_FL_CBPENDING;
> -
> -	/* when reaching here, it is under lock_res_and_lock(). Thus,
> -	 * need call the nolock version of ldlm_lock_decref_internal
> -	 */
> -	ldlm_lock_decref_internal_nolock(lock, mode);
> -
> -	ldlm_lock_destroy_nolock(lock);
> -}
> -
> -/**
> - * Process a granting attempt for flock lock.
> - * Must be called under ns lock held.
> - *
> - * This function looks for any conflicts for \a lock in the granted or
> - * waiting queues. The lock is granted if no conflicts are found in
> - * either queue.
> - *
> - * It is also responsible for splitting a lock if a portion of the lock
> - * is released.
> - *
> - */
> -static int ldlm_process_flock_lock(struct ldlm_lock *req)
> -{
> -	struct ldlm_resource *res = req->l_resource;
> -	struct ldlm_namespace *ns = ldlm_res_to_ns(res);
> -	struct ldlm_lock *tmp;
> -	struct ldlm_lock *lock;
> -	struct ldlm_lock *new = req;
> -	struct ldlm_lock *new2 = NULL;
> -	enum ldlm_mode mode = req->l_req_mode;
> -	int added = (mode == LCK_NL);
> -	int splitted = 0;
> -	const struct ldlm_callback_suite null_cbs = { };
> -
> -	CDEBUG(D_DLMTRACE,
> -	       "owner %llu pid %u mode %u start %llu end %llu\n",
> -	       new->l_policy_data.l_flock.owner,
> -	       new->l_policy_data.l_flock.pid, mode,
> -	       req->l_policy_data.l_flock.start,
> -	       req->l_policy_data.l_flock.end);
> -
> -	/* No blocking ASTs are sent to the clients for
> -	 * Posix file & record locks
> -	 */
> -	req->l_blocking_ast = NULL;
> -
> -reprocess:
> -	/* This loop determines where this processes locks start
> -	 * in the resource lr_granted list.
> -	 */
> -	list_for_each_entry(lock, &res->lr_granted, l_res_link)
> -		if (ldlm_same_flock_owner(lock, req))
> -			break;
> -
> -	/* Scan the locks owned by this process to find the insertion point
> -	 * (as locks are ordered), and to handle overlaps.
> -	 * We may have to merge or split existing locks.
> -	 */
> -	list_for_each_entry_safe_from(lock, tmp, &res->lr_granted, l_res_link) {
> -
> -		if (!ldlm_same_flock_owner(lock, new))
> -			break;
> -
> -		if (lock->l_granted_mode == mode) {
> -			/* If the modes are the same then we need to process
> -			 * locks that overlap OR adjoin the new lock. The extra
> -			 * logic condition is necessary to deal with arithmetic
> -			 * overflow and underflow.
> -			 */
> -			if ((new->l_policy_data.l_flock.start >
> -			     (lock->l_policy_data.l_flock.end + 1)) &&
> -			    (lock->l_policy_data.l_flock.end != OBD_OBJECT_EOF))
> -				continue;
> -
> -			if ((new->l_policy_data.l_flock.end <
> -			     (lock->l_policy_data.l_flock.start - 1)) &&
> -			    (lock->l_policy_data.l_flock.start != 0))
> -				break;
> -
> -			if (new->l_policy_data.l_flock.start <
> -			    lock->l_policy_data.l_flock.start) {
> -				lock->l_policy_data.l_flock.start =
> -					new->l_policy_data.l_flock.start;
> -			} else {
> -				new->l_policy_data.l_flock.start =
> -					lock->l_policy_data.l_flock.start;
> -			}
> -
> -			if (new->l_policy_data.l_flock.end >
> -			    lock->l_policy_data.l_flock.end) {
> -				lock->l_policy_data.l_flock.end =
> -					new->l_policy_data.l_flock.end;
> -			} else {
> -				new->l_policy_data.l_flock.end =
> -					lock->l_policy_data.l_flock.end;
> -			}
> -
> -			if (added) {
> -				ldlm_flock_destroy(lock, mode);
> -			} else {
> -				new = lock;
> -				added = 1;
> -			}
> -			continue;
> -		}
> -
> -		if (new->l_policy_data.l_flock.start >
> -		    lock->l_policy_data.l_flock.end)
> -			continue;
> -
> -		if (new->l_policy_data.l_flock.end <
> -		    lock->l_policy_data.l_flock.start)
> -			break;
> -
> -		if (new->l_policy_data.l_flock.start <=
> -		    lock->l_policy_data.l_flock.start) {
> -			if (new->l_policy_data.l_flock.end <
> -			    lock->l_policy_data.l_flock.end) {
> -				lock->l_policy_data.l_flock.start =
> -					new->l_policy_data.l_flock.end + 1;
> -				break;
> -			}
> -			ldlm_flock_destroy(lock, lock->l_req_mode);
> -			continue;
> -		}
> -		if (new->l_policy_data.l_flock.end >=
> -		    lock->l_policy_data.l_flock.end) {
> -			lock->l_policy_data.l_flock.end =
> -				new->l_policy_data.l_flock.start - 1;
> -			continue;
> -		}
> -
> -		/* split the existing lock into two locks */
> -
> -		/* if this is an F_UNLCK operation then we could avoid
> -		 * allocating a new lock and use the req lock passed in
> -		 * with the request but this would complicate the reply
> -		 * processing since updates to req get reflected in the
> -		 * reply. The client side replays the lock request so
> -		 * it must see the original lock data in the reply.
> -		 */
> -
> -		/* XXX - if ldlm_lock_new() can sleep we should
> -		 * release the lr_lock, allocate the new lock,
> -		 * and restart processing this lock.
> -		 */
> -		if (!new2) {
> -			unlock_res_and_lock(req);
> -			new2 = ldlm_lock_create(ns, &res->lr_name, LDLM_FLOCK,
> -						lock->l_granted_mode, &null_cbs,
> -						NULL, 0, LVB_T_NONE);
> -			lock_res_and_lock(req);
> -			if (IS_ERR(new2)) {
> -				ldlm_flock_destroy(req, lock->l_granted_mode);
> -				return LDLM_ITER_STOP;
> -			}
> -			goto reprocess;
> -		}
> -
> -		splitted = 1;
> -
> -		new2->l_granted_mode = lock->l_granted_mode;
> -		new2->l_policy_data.l_flock.pid =
> -			new->l_policy_data.l_flock.pid;
> -		new2->l_policy_data.l_flock.owner =
> -			new->l_policy_data.l_flock.owner;
> -		new2->l_policy_data.l_flock.start =
> -			lock->l_policy_data.l_flock.start;
> -		new2->l_policy_data.l_flock.end =
> -			new->l_policy_data.l_flock.start - 1;
> -		lock->l_policy_data.l_flock.start =
> -			new->l_policy_data.l_flock.end + 1;
> -		new2->l_conn_export = lock->l_conn_export;
> -		if (lock->l_export)
> -			new2->l_export = class_export_lock_get(lock->l_export,
> -							       new2);
> -		ldlm_lock_addref_internal_nolock(new2,
> -						 lock->l_granted_mode);
> -
> -		/* insert new2 at lock */
> -		ldlm_resource_add_lock(res, &lock->l_res_link, new2);
> -		LDLM_LOCK_RELEASE(new2);
> -		break;
> -	}
> -
> -	/* if new2 is created but never used, destroy it*/
> -	if (splitted == 0 && new2)
> -		ldlm_lock_destroy_nolock(new2);
> -
> -	/* At this point we're granting the lock request. */
> -	req->l_granted_mode = req->l_req_mode;
> -
> -	if (!added) {
> -		list_del_init(&req->l_res_link);
> -		/* insert new lock before "lock", which might be the
> -		 * next lock for this owner, or might be the first
> -		 * lock for the next owner, or might not be a lock at
> -		 * all, but instead points at the head of the list
> -		 */
> -		ldlm_resource_add_lock(res, &lock->l_res_link, req);
> -	}
> -
> -	/* In case we're reprocessing the requested lock we can't destroy
> -	 * it until after calling ldlm_add_ast_work_item() above so that laawi()
> -	 * can bump the reference count on \a req. Otherwise \a req
> -	 * could be freed before the completion AST can be sent.
> -	 */
> -	if (added)
> -		ldlm_flock_destroy(req, mode);
> -
> -	ldlm_resource_dump(D_INFO, res);
> -	return LDLM_ITER_CONTINUE;
> -}
> -
> -/**
> - * Flock completion callback function.
> - *
> - * \param lock [in,out]: A lock to be handled
> - * \param flags    [in]: flags
> - * \param *data    [in]: ldlm_work_cp_ast_lock() will use ldlm_cb_set_arg
> - *
> - * \retval 0    : success
> - * \retval <0   : failure
> - */
> -int
> -ldlm_flock_completion_ast(struct ldlm_lock *lock, __u64 flags, void *data)
> -{
> -	struct file_lock		*getlk = lock->l_ast_data;
> -	int				rc = 0;
> -
> -	OBD_FAIL_TIMEOUT(OBD_FAIL_LDLM_CP_CB_WAIT2, 4);
> -	if (OBD_FAIL_PRECHECK(OBD_FAIL_LDLM_CP_CB_WAIT3)) {
> -		lock_res_and_lock(lock);
> -		lock->l_flags |= LDLM_FL_FAIL_LOC;
> -		unlock_res_and_lock(lock);
> -		OBD_FAIL_TIMEOUT(OBD_FAIL_LDLM_CP_CB_WAIT3, 4);
> -	}
> -	CDEBUG(D_DLMTRACE, "flags: 0x%llx data: %p getlk: %p\n",
> -	       flags, data, getlk);
> -
> -	LASSERT(flags != LDLM_FL_WAIT_NOREPROC);
> -
> -	if (flags & LDLM_FL_FAILED)
> -		goto granted;
> -
> -	if (!(flags & LDLM_FL_BLOCKED_MASK)) {
> -		if (!data)
> -			/* mds granted the lock in the reply */
> -			goto granted;
> -		/* CP AST RPC: lock get granted, wake it up */
> -		wake_up(&lock->l_waitq);
> -		return 0;
> -	}
> -
> -	LDLM_DEBUG(lock,
> -		   "client-side enqueue returned a blocked lock, sleeping");
> -
> -	/* Go to sleep until the lock is granted. */
> -	rc = l_wait_event_abortable(lock->l_waitq, is_granted_or_cancelled(lock));
> -
> -	if (rc) {
> -		lock_res_and_lock(lock);
> -
> -		/* client side - set flag to prevent lock from being put on LRU list */
> -		ldlm_set_cbpending(lock);
> -		unlock_res_and_lock(lock);
> -
> -		LDLM_DEBUG(lock, "client-side enqueue waking up: failed (%d)",
> -			   rc);
> -		return rc;
> -	}
> -
> -granted:
> -	OBD_FAIL_TIMEOUT(OBD_FAIL_LDLM_CP_CB_WAIT, 10);
> -
> -	if (OBD_FAIL_PRECHECK(OBD_FAIL_LDLM_CP_CB_WAIT4)) {
> -		lock_res_and_lock(lock);
> -		/* DEADLOCK is always set with CBPENDING */
> -		lock->l_flags |= LDLM_FL_FLOCK_DEADLOCK | LDLM_FL_CBPENDING;
> -		unlock_res_and_lock(lock);
> -		OBD_FAIL_TIMEOUT(OBD_FAIL_LDLM_CP_CB_WAIT4, 4);
> -	}
> -	if (OBD_FAIL_PRECHECK(OBD_FAIL_LDLM_CP_CB_WAIT5)) {
> -		lock_res_and_lock(lock);
> -		/* DEADLOCK is always set with CBPENDING */
> -		lock->l_flags |= LDLM_FL_FAIL_LOC |
> -				 LDLM_FL_FLOCK_DEADLOCK | LDLM_FL_CBPENDING;
> -		unlock_res_and_lock(lock);
> -		OBD_FAIL_TIMEOUT(OBD_FAIL_LDLM_CP_CB_WAIT5, 4);
> -	}
> -
> -	lock_res_and_lock(lock);
> -
> -	/*
> -	 * Protect against race where lock could have been just destroyed
> -	 * due to overlap in ldlm_process_flock_lock().
> -	 */
> -	if (ldlm_is_destroyed(lock)) {
> -		unlock_res_and_lock(lock);
> -		LDLM_DEBUG(lock, "client-side enqueue waking up: destroyed");
> -		/*
> -		 * An error is still to be returned, to propagate it up to
> -		 * ldlm_cli_enqueue_fini() caller.
> -		 */
> -		return -EIO;
> -	}
> -
> -	/* ldlm_lock_enqueue() has already placed lock on the granted list. */
> -	ldlm_resource_unlink_lock(lock);
> -
> -	/*
> -	 * Import invalidation. We need to actually release the lock
> -	 * references being held, so that it can go away. No point in
> -	 * holding the lock even if app still believes it has it, since
> -	 * server already dropped it anyway. Only for granted locks too.
> -	 */
> -	/* Do the same for DEADLOCK'ed locks. */
> -	if (ldlm_is_failed(lock) || ldlm_is_flock_deadlock(lock)) {
> -		int mode;
> -
> -		if (flags & LDLM_FL_TEST_LOCK)
> -			LASSERT(ldlm_is_test_lock(lock));
> -
> -		if (ldlm_is_test_lock(lock) || ldlm_is_flock_deadlock(lock))
> -			mode = getlk->fl_type;
> -		else
> -			mode = lock->l_granted_mode;
> -
> -		if (ldlm_is_flock_deadlock(lock)) {
> -			LDLM_DEBUG(lock,
> -				   "client-side enqueue deadlock received");
> -			rc = -EDEADLK;
> -		}
> -		ldlm_flock_destroy(lock, mode);
> -		unlock_res_and_lock(lock);
> -
> -		/* Need to wake up the waiter if we were evicted */
> -		wake_up(&lock->l_waitq);
> -
> -		/*
> -		 * An error is still to be returned, to propagate it up to
> -		 * ldlm_cli_enqueue_fini() caller.
> -		 */
> -		return rc ? : -EIO;
> -	}
> -
> -	LDLM_DEBUG(lock, "client-side enqueue granted");
> -
> -	if (flags & LDLM_FL_TEST_LOCK) {
> -		/* fcntl(F_GETLK) request */
> -		/* The old mode was saved in getlk->fl_type so that if the mode
> -		 * in the lock changes we can decref the appropriate refcount.
> -		 */
> -		LASSERT(ldlm_is_test_lock(lock));
> -		ldlm_flock_destroy(lock, getlk->fl_type);
> -		switch (lock->l_granted_mode) {
> -		case LCK_PR:
> -			getlk->fl_type = F_RDLCK;
> -			break;
> -		case LCK_PW:
> -			getlk->fl_type = F_WRLCK;
> -			break;
> -		default:
> -			getlk->fl_type = F_UNLCK;
> -		}
> -		getlk->fl_pid = -(pid_t)lock->l_policy_data.l_flock.pid;
> -		getlk->fl_start = (loff_t)lock->l_policy_data.l_flock.start;
> -		getlk->fl_end = (loff_t)lock->l_policy_data.l_flock.end;
> -	} else {
> -		/* We need to reprocess the lock to do merges or splits
> -		 * with existing locks owned by this process.
> -		 */
> -		ldlm_process_flock_lock(lock);
> -	}
> -	unlock_res_and_lock(lock);
> -	return rc;
> -}
> -EXPORT_SYMBOL(ldlm_flock_completion_ast);
> -
> -void ldlm_flock_policy_wire_to_local(const union ldlm_wire_policy_data *wpolicy,
> -				     union ldlm_policy_data *lpolicy)
> -{
> -	lpolicy->l_flock.start = wpolicy->l_flock.lfw_start;
> -	lpolicy->l_flock.end = wpolicy->l_flock.lfw_end;
> -	lpolicy->l_flock.pid = wpolicy->l_flock.lfw_pid;
> -	lpolicy->l_flock.owner = wpolicy->l_flock.lfw_owner;
> -}
> -
> -void ldlm_flock_policy_local_to_wire(const union ldlm_policy_data *lpolicy,
> -				     union ldlm_wire_policy_data *wpolicy)
> -{
> -	memset(wpolicy, 0, sizeof(*wpolicy));
> -	wpolicy->l_flock.lfw_start = lpolicy->l_flock.start;
> -	wpolicy->l_flock.lfw_end = lpolicy->l_flock.end;
> -	wpolicy->l_flock.lfw_pid = lpolicy->l_flock.pid;
> -	wpolicy->l_flock.lfw_owner = lpolicy->l_flock.owner;
> -}
> diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_inodebits.c b/drivers/staging/lustre/lustre/ldlm/ldlm_inodebits.c
> deleted file mode 100644
> index 2926208cdfa1..000000000000
> --- a/drivers/staging/lustre/lustre/ldlm/ldlm_inodebits.c
> +++ /dev/null
> @@ -1,69 +0,0 @@
> -// SPDX-License-Identifier: GPL-2.0
> -/*
> - * GPL HEADER START
> - *
> - * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
> - *
> - * This program is free software; you can redistribute it and/or modify
> - * it under the terms of the GNU General Public License version 2 only,
> - * as published by the Free Software Foundation.
> - *
> - * This program is distributed in the hope that it will be useful, but
> - * WITHOUT ANY WARRANTY; without even the implied warranty of
> - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> - * General Public License version 2 for more details (a copy is included
> - * in the LICENSE file that accompanied this code).
> - *
> - * You should have received a copy of the GNU General Public License
> - * version 2 along with this program; If not, see
> - * http://www.gnu.org/licenses/gpl-2.0.html
> - *
> - * GPL HEADER END
> - */
> -/*
> - * Copyright (c) 2004, 2010, Oracle and/or its affiliates. All rights reserved.
> - * Use is subject to license terms.
> - *
> - * Copyright (c) 2011, Intel Corporation.
> - */
> -/*
> - * This file is part of Lustre, http://www.lustre.org/
> - * Lustre is a trademark of Sun Microsystems, Inc.
> - *
> - * lustre/ldlm/ldlm_inodebits.c
> - *
> - * Author: Peter Braam <braam at clusterfs.com>
> - * Author: Phil Schwan <phil at clusterfs.com>
> - */
> -
> -/**
> - * This file contains implementation of IBITS lock type
> - *
> - * IBITS lock type contains a bit mask determining various properties of an
> - * object. The meanings of specific bits are specific to the caller and are
> - * opaque to LDLM code.
> - *
> - * Locks with intersecting bitmasks and conflicting lock modes (e.g.  LCK_PW)
> - * are considered conflicting.  See the lock mode compatibility matrix
> - * in lustre_dlm.h.
> - */
> -
> -#define DEBUG_SUBSYSTEM S_LDLM
> -
> -#include <lustre_dlm.h>
> -#include <obd_support.h>
> -#include <lustre_lib.h>
> -#include "ldlm_internal.h"
> -
> -void ldlm_ibits_policy_wire_to_local(const union ldlm_wire_policy_data *wpolicy,
> -				     union ldlm_policy_data *lpolicy)
> -{
> -	lpolicy->l_inodebits.bits = wpolicy->l_inodebits.bits;
> -}
> -
> -void ldlm_ibits_policy_local_to_wire(const union ldlm_policy_data *lpolicy,
> -				     union ldlm_wire_policy_data *wpolicy)
> -{
> -	memset(wpolicy, 0, sizeof(*wpolicy));
> -	wpolicy->l_inodebits.bits = lpolicy->l_inodebits.bits;
> -}
> diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_internal.h b/drivers/staging/lustre/lustre/ldlm/ldlm_internal.h
> deleted file mode 100644
> index 60a15b963c8a..000000000000
> --- a/drivers/staging/lustre/lustre/ldlm/ldlm_internal.h
> +++ /dev/null
> @@ -1,329 +0,0 @@
> -// SPDX-License-Identifier: GPL-2.0
> -/*
> - * GPL HEADER START
> - *
> - * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
> - *
> - * This program is free software; you can redistribute it and/or modify
> - * it under the terms of the GNU General Public License version 2 only,
> - * as published by the Free Software Foundation.
> - *
> - * This program is distributed in the hope that it will be useful, but
> - * WITHOUT ANY WARRANTY; without even the implied warranty of
> - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> - * General Public License version 2 for more details (a copy is included
> - * in the LICENSE file that accompanied this code).
> - *
> - * You should have received a copy of the GNU General Public License
> - * version 2 along with this program; If not, see
> - * http://www.gnu.org/licenses/gpl-2.0.html
> - *
> - * GPL HEADER END
> - */
> -/*
> - * Copyright (c) 2003, 2010, Oracle and/or its affiliates. All rights reserved.
> - * Use is subject to license terms.
> - *
> - * Copyright (c) 2011, 2015, Intel Corporation.
> - */
> -/*
> - * This file is part of Lustre, http://www.lustre.org/
> - * Lustre is a trademark of Sun Microsystems, Inc.
> - */
> -
> -#define MAX_STRING_SIZE 128
> -
> -extern int ldlm_srv_namespace_nr;
> -extern int ldlm_cli_namespace_nr;
> -extern struct mutex ldlm_srv_namespace_lock;
> -extern struct list_head ldlm_srv_namespace_list;
> -extern struct mutex ldlm_cli_namespace_lock;
> -extern struct list_head ldlm_cli_active_namespace_list;
> -
> -static inline int ldlm_namespace_nr_read(enum ldlm_side client)
> -{
> -	return client == LDLM_NAMESPACE_SERVER ?
> -		ldlm_srv_namespace_nr : ldlm_cli_namespace_nr;
> -}
> -
> -static inline void ldlm_namespace_nr_inc(enum ldlm_side client)
> -{
> -	if (client == LDLM_NAMESPACE_SERVER)
> -		ldlm_srv_namespace_nr++;
> -	else
> -		ldlm_cli_namespace_nr++;
> -}
> -
> -static inline void ldlm_namespace_nr_dec(enum ldlm_side client)
> -{
> -	if (client == LDLM_NAMESPACE_SERVER)
> -		ldlm_srv_namespace_nr--;
> -	else
> -		ldlm_cli_namespace_nr--;
> -}
> -
> -static inline struct list_head *ldlm_namespace_list(enum ldlm_side client)
> -{
> -	return client == LDLM_NAMESPACE_SERVER ?
> -		&ldlm_srv_namespace_list : &ldlm_cli_active_namespace_list;
> -}
> -
> -static inline struct mutex *ldlm_namespace_lock(enum ldlm_side client)
> -{
> -	return client == LDLM_NAMESPACE_SERVER ?
> -		&ldlm_srv_namespace_lock : &ldlm_cli_namespace_lock;
> -}
> -
> -/* ns_bref is the number of resources in this namespace */
> -static inline int ldlm_ns_empty(struct ldlm_namespace *ns)
> -{
> -	return atomic_read(&ns->ns_bref) == 0;
> -}
> -
> -void ldlm_namespace_move_to_active_locked(struct ldlm_namespace *ns,
> -					  enum ldlm_side client);
> -void ldlm_namespace_move_to_inactive_locked(struct ldlm_namespace *ns,
> -					    enum ldlm_side client);
> -struct ldlm_namespace *ldlm_namespace_first_locked(enum ldlm_side client);
> -
> -/* ldlm_request.c */
> -/* Cancel lru flag, it indicates we cancel aged locks. */
> -enum {
> -	LDLM_LRU_FLAG_AGED	= BIT(0), /* Cancel old non-LRU resize locks */
> -	LDLM_LRU_FLAG_PASSED	= BIT(1), /* Cancel passed number of locks. */
> -	LDLM_LRU_FLAG_SHRINK	= BIT(2), /* Cancel locks from shrinker. */
> -	LDLM_LRU_FLAG_LRUR	= BIT(3), /* Cancel locks from lru resize. */
> -	LDLM_LRU_FLAG_NO_WAIT	= BIT(4), /* Cancel locks w/o blocking (neither
> -					   * sending nor waiting for any rpcs)
> -					   */
> -	LDLM_LRU_FLAG_LRUR_NO_WAIT = BIT(5), /* LRUR + NO_WAIT */
> -};
> -
> -int ldlm_cancel_lru(struct ldlm_namespace *ns, int nr,
> -		    enum ldlm_cancel_flags sync, int flags);
> -int ldlm_cancel_lru_local(struct ldlm_namespace *ns,
> -			  struct list_head *cancels, int count, int max,
> -			  enum ldlm_cancel_flags cancel_flags, int flags);
> -extern unsigned int ldlm_enqueue_min;
> -extern unsigned int ldlm_cancel_unused_locks_before_replay;
> -
> -/* ldlm_lock.c */
> -
> -struct ldlm_cb_set_arg {
> -	struct ptlrpc_request_set	*set;
> -	int				 type; /* LDLM_{CP,BL,GL}_CALLBACK */
> -	atomic_t			 restart;
> -	struct list_head			*list;
> -	union ldlm_gl_desc		*gl_desc; /* glimpse AST descriptor */
> -};
> -
> -enum ldlm_desc_ast_t {
> -	LDLM_WORK_BL_AST,
> -	LDLM_WORK_CP_AST,
> -	LDLM_WORK_REVOKE_AST,
> -	LDLM_WORK_GL_AST
> -};
> -
> -void ldlm_grant_lock(struct ldlm_lock *lock, struct list_head *work_list);
> -int ldlm_fill_lvb(struct ldlm_lock *lock, struct req_capsule *pill,
> -		  enum req_location loc, void *data, int size);
> -struct ldlm_lock *
> -ldlm_lock_create(struct ldlm_namespace *ns, const struct ldlm_res_id *id,
> -		 enum ldlm_type type, enum ldlm_mode mode,
> -		 const struct ldlm_callback_suite *cbs,
> -		 void *data, __u32 lvb_len, enum lvb_type lvb_type);
> -enum ldlm_error ldlm_lock_enqueue(struct ldlm_namespace *ns,
> -				  struct ldlm_lock **lock, void *cookie,
> -				  __u64 *flags);
> -void ldlm_lock_addref_internal(struct ldlm_lock *lock, enum ldlm_mode mode);
> -void ldlm_lock_addref_internal_nolock(struct ldlm_lock *lock,
> -				      enum ldlm_mode mode);
> -void ldlm_lock_decref_internal(struct ldlm_lock *lock, enum ldlm_mode mode);
> -void ldlm_lock_decref_internal_nolock(struct ldlm_lock *lock,
> -				      enum ldlm_mode mode);
> -int ldlm_run_ast_work(struct ldlm_namespace *ns, struct list_head *rpc_list,
> -		      enum ldlm_desc_ast_t ast_type);
> -int ldlm_lock_remove_from_lru_check(struct ldlm_lock *lock, time_t last_use);
> -#define ldlm_lock_remove_from_lru(lock) ldlm_lock_remove_from_lru_check(lock, 0)
> -int ldlm_lock_remove_from_lru_nolock(struct ldlm_lock *lock);
> -void ldlm_lock_destroy_nolock(struct ldlm_lock *lock);
> -
> -/* ldlm_lockd.c */
> -int ldlm_bl_to_thread_lock(struct ldlm_namespace *ns, struct ldlm_lock_desc *ld,
> -			   struct ldlm_lock *lock);
> -int ldlm_bl_to_thread_list(struct ldlm_namespace *ns,
> -			   struct ldlm_lock_desc *ld,
> -			   struct list_head *cancels, int count,
> -			   enum ldlm_cancel_flags cancel_flags);
> -int ldlm_bl_thread_wakeup(void);
> -
> -void ldlm_handle_bl_callback(struct ldlm_namespace *ns,
> -			     struct ldlm_lock_desc *ld, struct ldlm_lock *lock);
> -
> -extern struct kmem_cache *ldlm_resource_slab;
> -extern struct kset *ldlm_ns_kset;
> -
> -/* ldlm_lockd.c & ldlm_lock.c */
> -extern struct kmem_cache *ldlm_lock_slab;
> -
> -/* ldlm_extent.c */
> -void ldlm_extent_add_lock(struct ldlm_resource *res, struct ldlm_lock *lock);
> -void ldlm_extent_unlink_lock(struct ldlm_lock *lock);
> -void ldlm_extent_search(struct rb_root_cached *root,
> -			__u64 start, __u64 end,
> -			bool (*matches)(struct ldlm_lock *lock, void *data),
> -			void *data);
> -
> -/* l_lock.c */
> -void l_check_ns_lock(struct ldlm_namespace *ns);
> -void l_check_no_ns_lock(struct ldlm_namespace *ns);
> -
> -extern struct dentry *ldlm_svc_debugfs_dir;
> -
> -struct ldlm_state {
> -	struct ptlrpc_service *ldlm_cb_service;
> -	struct ptlrpc_service *ldlm_cancel_service;
> -	struct ptlrpc_client *ldlm_client;
> -	struct ptlrpc_connection *ldlm_server_conn;
> -	struct ldlm_bl_pool *ldlm_bl_pool;
> -};
> -
> -/* ldlm_pool.c */
> -__u64 ldlm_pool_get_slv(struct ldlm_pool *pl);
> -void ldlm_pool_set_clv(struct ldlm_pool *pl, __u64 clv);
> -__u32 ldlm_pool_get_lvf(struct ldlm_pool *pl);
> -
> -int ldlm_init(void);
> -void ldlm_exit(void);
> -
> -enum ldlm_policy_res {
> -	LDLM_POLICY_CANCEL_LOCK,
> -	LDLM_POLICY_KEEP_LOCK,
> -	LDLM_POLICY_SKIP_LOCK
> -};
> -
> -#define LDLM_POOL_SYSFS_PRINT_int(v) sprintf(buf, "%d\n", v)
> -#define LDLM_POOL_SYSFS_SET_int(a, b) { a = b; }
> -#define LDLM_POOL_SYSFS_PRINT_u64(v) sprintf(buf, "%lld\n", v)
> -#define LDLM_POOL_SYSFS_SET_u64(a, b) { a = b; }
> -#define LDLM_POOL_SYSFS_PRINT_atomic(v) sprintf(buf, "%d\n", atomic_read(&v))
> -#define LDLM_POOL_SYSFS_SET_atomic(a, b) atomic_set(&a, b)
> -
> -#define LDLM_POOL_SYSFS_READER_SHOW(var, type)				    \
> -	static ssize_t var##_show(struct kobject *kobj,			    \
> -				  struct attribute *attr,		    \
> -				  char *buf)				    \
> -	{								    \
> -		struct ldlm_pool *pl = container_of(kobj, struct ldlm_pool, \
> -						    pl_kobj);		    \
> -		type tmp;						    \
> -									    \
> -		spin_lock(&pl->pl_lock);				    \
> -		tmp = pl->pl_##var;					    \
> -		spin_unlock(&pl->pl_lock);				    \
> -									    \
> -		return LDLM_POOL_SYSFS_PRINT_##type(tmp);		    \
> -	}								    \
> -	struct __##var##__dummy_read {; } /* semicolon catcher */
> -
> -#define LDLM_POOL_SYSFS_WRITER_STORE(var, type)				    \
> -	static ssize_t var##_store(struct kobject *kobj,		    \
> -				     struct attribute *attr,		    \
> -				     const char *buffer,		    \
> -				     size_t count)			    \
> -	{								    \
> -		struct ldlm_pool *pl = container_of(kobj, struct ldlm_pool, \
> -						    pl_kobj);		    \
> -		unsigned long tmp;					    \
> -		int rc;							    \
> -									    \
> -		rc = kstrtoul(buffer, 10, &tmp);			    \
> -		if (rc < 0) {						    \
> -			return rc;					    \
> -		}							    \
> -									    \
> -		spin_lock(&pl->pl_lock);				    \
> -		LDLM_POOL_SYSFS_SET_##type(pl->pl_##var, tmp);		    \
> -		spin_unlock(&pl->pl_lock);				    \
> -									    \
> -		return count;						    \
> -	}								    \
> -	struct __##var##__dummy_write {; } /* semicolon catcher */
> -
> -#define LDLM_POOL_SYSFS_READER_NOLOCK_SHOW(var, type)			    \
> -	static ssize_t var##_show(struct kobject *kobj,		    \
> -				    struct attribute *attr,		    \
> -				    char *buf)				    \
> -	{								    \
> -		struct ldlm_pool *pl = container_of(kobj, struct ldlm_pool, \
> -						    pl_kobj);		    \
> -									    \
> -		return LDLM_POOL_SYSFS_PRINT_##type(pl->pl_##var);	    \
> -	}								    \
> -	struct __##var##__dummy_read {; } /* semicolon catcher */
> -
> -#define LDLM_POOL_SYSFS_WRITER_NOLOCK_STORE(var, type)			    \
> -	static ssize_t var##_store(struct kobject *kobj,		    \
> -				     struct attribute *attr,		    \
> -				     const char *buffer,		    \
> -				     size_t count)			    \
> -	{								    \
> -		struct ldlm_pool *pl = container_of(kobj, struct ldlm_pool, \
> -						    pl_kobj);		    \
> -		unsigned long tmp;					    \
> -		int rc;							    \
> -									    \
> -		rc = kstrtoul(buffer, 10, &tmp);			    \
> -		if (rc < 0) {						    \
> -			return rc;					    \
> -		}							    \
> -									    \
> -		LDLM_POOL_SYSFS_SET_##type(pl->pl_##var, tmp);		    \
> -									    \
> -		return count;						    \
> -	}								    \
> -	struct __##var##__dummy_write {; } /* semicolon catcher */
> -
> -static inline int is_granted_or_cancelled(struct ldlm_lock *lock)
> -{
> -	int ret = 0;
> -
> -	lock_res_and_lock(lock);
> -	if ((lock->l_req_mode == lock->l_granted_mode) &&
> -	    !ldlm_is_cp_reqd(lock))
> -		ret = 1;
> -	else if (ldlm_is_failed(lock) || ldlm_is_cancel(lock))
> -		ret = 1;
> -	unlock_res_and_lock(lock);
> -
> -	return ret;
> -}
> -
> -typedef void (*ldlm_policy_wire_to_local_t)(const union ldlm_wire_policy_data *,
> -					    union ldlm_policy_data *);
> -
> -typedef void (*ldlm_policy_local_to_wire_t)(const union ldlm_policy_data *,
> -					    union ldlm_wire_policy_data *);
> -
> -void ldlm_plain_policy_wire_to_local(const union ldlm_wire_policy_data *wpolicy,
> -				     union ldlm_policy_data *lpolicy);
> -void ldlm_plain_policy_local_to_wire(const union ldlm_policy_data *lpolicy,
> -				     union ldlm_wire_policy_data *wpolicy);
> -void ldlm_ibits_policy_wire_to_local(const union ldlm_wire_policy_data *wpolicy,
> -				     union ldlm_policy_data *lpolicy);
> -void ldlm_ibits_policy_local_to_wire(const union ldlm_policy_data *lpolicy,
> -				     union ldlm_wire_policy_data *wpolicy);
> -void ldlm_extent_policy_wire_to_local(const union ldlm_wire_policy_data *wpolicy,
> -				      union ldlm_policy_data *lpolicy);
> -void ldlm_extent_policy_local_to_wire(const union ldlm_policy_data *lpolicy,
> -				      union ldlm_wire_policy_data *wpolicy);
> -void ldlm_flock_policy_wire_to_local(const union ldlm_wire_policy_data *wpolicy,
> -				     union ldlm_policy_data *lpolicy);
> -void ldlm_flock_policy_local_to_wire(const union ldlm_policy_data *lpolicy,
> -				     union ldlm_wire_policy_data *wpolicy);
> -
> -static inline bool ldlm_res_eq(const struct ldlm_res_id *res0,
> -			       const struct ldlm_res_id *res1)
> -{
> -	return memcmp(res0, res1, sizeof(*res0)) == 0;
> -}
> diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_lib.c b/drivers/staging/lustre/lustre/ldlm/ldlm_lib.c
> deleted file mode 100644
> index 0aa4f234a4f4..000000000000
> --- a/drivers/staging/lustre/lustre/ldlm/ldlm_lib.c
> +++ /dev/null
> @@ -1,842 +0,0 @@
> -// SPDX-License-Identifier: GPL-2.0
> -/*
> - * GPL HEADER START
> - *
> - * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
> - *
> - * This program is free software; you can redistribute it and/or modify
> - * it under the terms of the GNU General Public License version 2 only,
> - * as published by the Free Software Foundation.
> - *
> - * This program is distributed in the hope that it will be useful, but
> - * WITHOUT ANY WARRANTY; without even the implied warranty of
> - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> - * General Public License version 2 for more details (a copy is included
> - * in the LICENSE file that accompanied this code).
> - *
> - * You should have received a copy of the GNU General Public License
> - * version 2 along with this program; If not, see
> - * http://www.gnu.org/licenses/gpl-2.0.html
> - *
> - * GPL HEADER END
> - */
> -/*
> - * Copyright (c) 2003, 2010, Oracle and/or its affiliates. All rights reserved.
> - * Use is subject to license terms.
> - *
> - * Copyright (c) 2010, 2015, Intel Corporation.
> - */
> -/*
> - * This file is part of Lustre, http://www.lustre.org/
> - * Lustre is a trademark of Sun Microsystems, Inc.
> - */
> -
> -/**
> - * This file deals with various client/target related logic including recovery.
> - *
> - * TODO: This code more logically belongs in the ptlrpc module than in ldlm and
> - * should be moved.
> - */
> -
> -#define DEBUG_SUBSYSTEM S_LDLM
> -
> -#include <obd.h>
> -#include <obd_class.h>
> -#include <lustre_dlm.h>
> -#include <lustre_net.h>
> -#include <lustre_sec.h>
> -#include "ldlm_internal.h"
> -
> -/* @priority: If non-zero, move the selected connection to the list head.
> - * @create: If zero, only search in existing connections.
> - */
> -static int import_set_conn(struct obd_import *imp, struct obd_uuid *uuid,
> -			   int priority, int create)
> -{
> -	struct ptlrpc_connection *ptlrpc_conn;
> -	struct obd_import_conn *imp_conn = NULL, *item;
> -	int rc = 0;
> -
> -	if (!create && !priority) {
> -		CDEBUG(D_HA, "Nothing to do\n");
> -		return -EINVAL;
> -	}
> -
> -	ptlrpc_conn = ptlrpc_uuid_to_connection(uuid);
> -	if (!ptlrpc_conn) {
> -		CDEBUG(D_HA, "can't find connection %s\n", uuid->uuid);
> -		return -ENOENT;
> -	}
> -
> -	if (create) {
> -		imp_conn = kzalloc(sizeof(*imp_conn), GFP_NOFS);
> -		if (!imp_conn) {
> -			rc = -ENOMEM;
> -			goto out_put;
> -		}
> -	}
> -
> -	spin_lock(&imp->imp_lock);
> -	list_for_each_entry(item, &imp->imp_conn_list, oic_item) {
> -		if (obd_uuid_equals(uuid, &item->oic_uuid)) {
> -			if (priority) {
> -				list_del(&item->oic_item);
> -				list_add(&item->oic_item,
> -					 &imp->imp_conn_list);
> -				item->oic_last_attempt = 0;
> -			}
> -			CDEBUG(D_HA, "imp %p@%s: found existing conn %s%s\n",
> -			       imp, imp->imp_obd->obd_name, uuid->uuid,
> -			       (priority ? ", moved to head" : ""));
> -			spin_unlock(&imp->imp_lock);
> -			rc = 0;
> -			goto out_free;
> -		}
> -	}
> -	/* No existing import connection found for \a uuid. */
> -	if (create) {
> -		imp_conn->oic_conn = ptlrpc_conn;
> -		imp_conn->oic_uuid = *uuid;
> -		imp_conn->oic_last_attempt = 0;
> -		if (priority)
> -			list_add(&imp_conn->oic_item, &imp->imp_conn_list);
> -		else
> -			list_add_tail(&imp_conn->oic_item,
> -				      &imp->imp_conn_list);
> -		CDEBUG(D_HA, "imp %p@%s: add connection %s at %s\n",
> -		       imp, imp->imp_obd->obd_name, uuid->uuid,
> -		       (priority ? "head" : "tail"));
> -	} else {
> -		spin_unlock(&imp->imp_lock);
> -		rc = -ENOENT;
> -		goto out_free;
> -	}
> -
> -	spin_unlock(&imp->imp_lock);
> -	return 0;
> -out_free:
> -	kfree(imp_conn);
> -out_put:
> -	ptlrpc_connection_put(ptlrpc_conn);
> -	return rc;
> -}
> -
> -int import_set_conn_priority(struct obd_import *imp, struct obd_uuid *uuid)
> -{
> -	return import_set_conn(imp, uuid, 1, 0);
> -}
> -
> -int client_import_add_conn(struct obd_import *imp, struct obd_uuid *uuid,
> -			   int priority)
> -{
> -	return import_set_conn(imp, uuid, priority, 1);
> -}
> -EXPORT_SYMBOL(client_import_add_conn);
> -
> -int client_import_del_conn(struct obd_import *imp, struct obd_uuid *uuid)
> -{
> -	struct obd_import_conn *imp_conn;
> -	struct obd_export *dlmexp;
> -	int rc = -ENOENT;
> -
> -	spin_lock(&imp->imp_lock);
> -	if (list_empty(&imp->imp_conn_list)) {
> -		LASSERT(!imp->imp_connection);
> -		goto out;
> -	}
> -
> -	list_for_each_entry(imp_conn, &imp->imp_conn_list, oic_item) {
> -		if (!obd_uuid_equals(uuid, &imp_conn->oic_uuid))
> -			continue;
> -		LASSERT(imp_conn->oic_conn);
> -
> -		if (imp_conn == imp->imp_conn_current) {
> -			LASSERT(imp_conn->oic_conn == imp->imp_connection);
> -
> -			if (imp->imp_state != LUSTRE_IMP_CLOSED &&
> -			    imp->imp_state != LUSTRE_IMP_DISCON) {
> -				CERROR("can't remove current connection\n");
> -				rc = -EBUSY;
> -				goto out;
> -			}
> -
> -			ptlrpc_connection_put(imp->imp_connection);
> -			imp->imp_connection = NULL;
> -
> -			dlmexp = class_conn2export(&imp->imp_dlm_handle);
> -			if (dlmexp && dlmexp->exp_connection) {
> -				LASSERT(dlmexp->exp_connection ==
> -					imp_conn->oic_conn);
> -				ptlrpc_connection_put(dlmexp->exp_connection);
> -				dlmexp->exp_connection = NULL;
> -			}
> -
> -			if (dlmexp)
> -				class_export_put(dlmexp);
> -		}
> -
> -		list_del(&imp_conn->oic_item);
> -		ptlrpc_connection_put(imp_conn->oic_conn);
> -		kfree(imp_conn);
> -		CDEBUG(D_HA, "imp %p@%s: remove connection %s\n",
> -		       imp, imp->imp_obd->obd_name, uuid->uuid);
> -		rc = 0;
> -		break;
> -	}
> -out:
> -	spin_unlock(&imp->imp_lock);
> -	if (rc == -ENOENT)
> -		CERROR("connection %s not found\n", uuid->uuid);
> -	return rc;
> -}
> -EXPORT_SYMBOL(client_import_del_conn);
> -
> -/**
> - * Find conn UUID by peer NID. \a peer is a server NID. This function is used
> - * to find a conn uuid of \a imp which can reach \a peer.
> - */
> -int client_import_find_conn(struct obd_import *imp, lnet_nid_t peer,
> -			    struct obd_uuid *uuid)
> -{
> -	struct obd_import_conn *conn;
> -	int rc = -ENOENT;
> -
> -	spin_lock(&imp->imp_lock);
> -	list_for_each_entry(conn, &imp->imp_conn_list, oic_item) {
> -		/* Check if conn UUID does have this peer NID. */
> -		if (class_check_uuid(&conn->oic_uuid, peer)) {
> -			*uuid = conn->oic_uuid;
> -			rc = 0;
> -			break;
> -		}
> -	}
> -	spin_unlock(&imp->imp_lock);
> -	return rc;
> -}
> -EXPORT_SYMBOL(client_import_find_conn);
> -
> -void client_destroy_import(struct obd_import *imp)
> -{
> -	/* Drop security policy instance after all RPCs have finished/aborted
> -	 * to let all busy contexts be released.
> -	 */
> -	class_import_get(imp);
> -	class_destroy_import(imp);
> -	sptlrpc_import_sec_put(imp);
> -	class_import_put(imp);
> -}
> -EXPORT_SYMBOL(client_destroy_import);
> -
> -/* Configure an RPC client OBD device.
> - *
> - * lcfg parameters:
> - * 1 - client UUID
> - * 2 - server UUID
> - * 3 - inactive-on-startup
> - */
> -int client_obd_setup(struct obd_device *obddev, struct lustre_cfg *lcfg)
> -{
> -	struct client_obd *cli = &obddev->u.cli;
> -	struct obd_import *imp;
> -	struct obd_uuid server_uuid;
> -	int rq_portal, rp_portal, connect_op;
> -	char *name = obddev->obd_type->typ_name;
> -	enum ldlm_ns_type ns_type = LDLM_NS_TYPE_UNKNOWN;
> -	int rc;
> -
> -	/* In a more perfect world, we would hang a ptlrpc_client off of
> -	 * obd_type and just use the values from there.
> -	 */
> -	if (!strcmp(name, LUSTRE_OSC_NAME)) {
> -		rq_portal = OST_REQUEST_PORTAL;
> -		rp_portal = OSC_REPLY_PORTAL;
> -		connect_op = OST_CONNECT;
> -		cli->cl_sp_me = LUSTRE_SP_CLI;
> -		cli->cl_sp_to = LUSTRE_SP_OST;
> -		ns_type = LDLM_NS_TYPE_OSC;
> -	} else if (!strcmp(name, LUSTRE_MDC_NAME) ||
> -		   !strcmp(name, LUSTRE_LWP_NAME)) {
> -		rq_portal = MDS_REQUEST_PORTAL;
> -		rp_portal = MDC_REPLY_PORTAL;
> -		connect_op = MDS_CONNECT;
> -		cli->cl_sp_me = LUSTRE_SP_CLI;
> -		cli->cl_sp_to = LUSTRE_SP_MDT;
> -		ns_type = LDLM_NS_TYPE_MDC;
> -	} else if (!strcmp(name, LUSTRE_MGC_NAME)) {
> -		rq_portal = MGS_REQUEST_PORTAL;
> -		rp_portal = MGC_REPLY_PORTAL;
> -		connect_op = MGS_CONNECT;
> -		cli->cl_sp_me = LUSTRE_SP_MGC;
> -		cli->cl_sp_to = LUSTRE_SP_MGS;
> -		cli->cl_flvr_mgc.sf_rpc = SPTLRPC_FLVR_INVALID;
> -		ns_type = LDLM_NS_TYPE_MGC;
> -	} else {
> -		CERROR("unknown client OBD type \"%s\", can't setup\n",
> -		       name);
> -		return -EINVAL;
> -	}
> -
> -	if (LUSTRE_CFG_BUFLEN(lcfg, 1) < 1) {
> -		CERROR("requires a TARGET UUID\n");
> -		return -EINVAL;
> -	}
> -
> -	if (LUSTRE_CFG_BUFLEN(lcfg, 1) > 37) {
> -		CERROR("client UUID must be less than 38 characters\n");
> -		return -EINVAL;
> -	}
> -
> -	if (LUSTRE_CFG_BUFLEN(lcfg, 2) < 1) {
> -		CERROR("setup requires a SERVER UUID\n");
> -		return -EINVAL;
> -	}
> -
> -	if (LUSTRE_CFG_BUFLEN(lcfg, 2) > 37) {
> -		CERROR("target UUID must be less than 38 characters\n");
> -		return -EINVAL;
> -	}
> -
> -	init_rwsem(&cli->cl_sem);
> -	cli->cl_conn_count = 0;
> -	memcpy(server_uuid.uuid, lustre_cfg_buf(lcfg, 2),
> -	       min_t(unsigned int, LUSTRE_CFG_BUFLEN(lcfg, 2),
> -		     sizeof(server_uuid)));
> -
> -	cli->cl_dirty_pages = 0;
> -	cli->cl_avail_grant = 0;
> -	/* FIXME: Should limit this for the sum of all cl_dirty_max_pages. */
> -	/*
> -	 * cl_dirty_max_pages may be changed at connect time in
> -	 * ptlrpc_connect_interpret().
> -	 */
> -	client_adjust_max_dirty(cli);
> -	INIT_LIST_HEAD(&cli->cl_cache_waiters);
> -	INIT_LIST_HEAD(&cli->cl_loi_ready_list);
> -	INIT_LIST_HEAD(&cli->cl_loi_hp_ready_list);
> -	INIT_LIST_HEAD(&cli->cl_loi_write_list);
> -	INIT_LIST_HEAD(&cli->cl_loi_read_list);
> -	spin_lock_init(&cli->cl_loi_list_lock);
> -	atomic_set(&cli->cl_pending_w_pages, 0);
> -	atomic_set(&cli->cl_pending_r_pages, 0);
> -	cli->cl_r_in_flight = 0;
> -	cli->cl_w_in_flight = 0;
> -
> -	spin_lock_init(&cli->cl_read_rpc_hist.oh_lock);
> -	spin_lock_init(&cli->cl_write_rpc_hist.oh_lock);
> -	spin_lock_init(&cli->cl_read_page_hist.oh_lock);
> -	spin_lock_init(&cli->cl_write_page_hist.oh_lock);
> -	spin_lock_init(&cli->cl_read_offset_hist.oh_lock);
> -	spin_lock_init(&cli->cl_write_offset_hist.oh_lock);
> -
> -	/* lru for osc. */
> -	INIT_LIST_HEAD(&cli->cl_lru_osc);
> -	atomic_set(&cli->cl_lru_shrinkers, 0);
> -	atomic_long_set(&cli->cl_lru_busy, 0);
> -	atomic_long_set(&cli->cl_lru_in_list, 0);
> -	INIT_LIST_HEAD(&cli->cl_lru_list);
> -	spin_lock_init(&cli->cl_lru_list_lock);
> -	atomic_long_set(&cli->cl_unstable_count, 0);
> -	INIT_LIST_HEAD(&cli->cl_shrink_list);
> -
> -	init_waitqueue_head(&cli->cl_destroy_waitq);
> -	atomic_set(&cli->cl_destroy_in_flight, 0);
> -	/* Turn on checksumming by default. */
> -	cli->cl_checksum = 1;
> -	/*
> -	 * The supported checksum types will be worked out at connect time
> -	 * Set cl_chksum* to CRC32 for now to avoid returning screwed info
> -	 * through procfs.
> -	 */
> -	cli->cl_cksum_type = OBD_CKSUM_CRC32;
> -	cli->cl_supp_cksum_types = OBD_CKSUM_CRC32;
> -	atomic_set(&cli->cl_resends, OSC_DEFAULT_RESENDS);
> -
> -	/*
> -	 * Set it to possible maximum size. It may be reduced by ocd_brw_size
> -	 * from OFD after connecting.
> -	 */
> -	cli->cl_max_pages_per_rpc = PTLRPC_MAX_BRW_PAGES;
> -
> -	/*
> -	 * set cl_chunkbits default value to PAGE_CACHE_SHIFT,
> -	 * it will be updated at OSC connection time.
> -	 */
> -	cli->cl_chunkbits = PAGE_SHIFT;
> -
> -	if (!strcmp(name, LUSTRE_MDC_NAME))
> -		cli->cl_max_rpcs_in_flight = OBD_MAX_RIF_DEFAULT;
> -	else if (totalram_pages >> (20 - PAGE_SHIFT) <= 128 /* MB */)
> -		cli->cl_max_rpcs_in_flight = 2;
> -	else if (totalram_pages >> (20 - PAGE_SHIFT) <= 256 /* MB */)
> -		cli->cl_max_rpcs_in_flight = 3;
> -	else if (totalram_pages >> (20 - PAGE_SHIFT) <= 512 /* MB */)
> -		cli->cl_max_rpcs_in_flight = 4;
> -	else
> -		cli->cl_max_rpcs_in_flight = OBD_MAX_RIF_DEFAULT;
> -
> -	spin_lock_init(&cli->cl_mod_rpcs_lock);
> -	spin_lock_init(&cli->cl_mod_rpcs_hist.oh_lock);
> -	cli->cl_max_mod_rpcs_in_flight = 0;
> -	cli->cl_mod_rpcs_in_flight = 0;
> -	cli->cl_close_rpcs_in_flight = 0;
> -	init_waitqueue_head(&cli->cl_mod_rpcs_waitq);
> -	cli->cl_mod_tag_bitmap = NULL;
> -
> -	if (connect_op == MDS_CONNECT) {
> -		cli->cl_max_mod_rpcs_in_flight = cli->cl_max_rpcs_in_flight - 1;
> -		cli->cl_mod_tag_bitmap = kcalloc(BITS_TO_LONGS(OBD_MAX_RIF_MAX),
> -						 sizeof(long), GFP_NOFS);
> -		if (!cli->cl_mod_tag_bitmap) {
> -			rc = -ENOMEM;
> -			goto err;
> -		}
> -	}
> -
> -	rc = ldlm_get_ref();
> -	if (rc) {
> -		CERROR("ldlm_get_ref failed: %d\n", rc);
> -		goto err;
> -	}
> -
> -	ptlrpc_init_client(rq_portal, rp_portal, name,
> -			   &obddev->obd_ldlm_client);
> -
> -	imp = class_new_import(obddev);
> -	if (!imp) {
> -		rc = -ENOENT;
> -		goto err_ldlm;
> -	}
> -	imp->imp_client = &obddev->obd_ldlm_client;
> -	imp->imp_connect_op = connect_op;
> -	memcpy(cli->cl_target_uuid.uuid, lustre_cfg_buf(lcfg, 1),
> -	       LUSTRE_CFG_BUFLEN(lcfg, 1));
> -	class_import_put(imp);
> -
> -	rc = client_import_add_conn(imp, &server_uuid, 1);
> -	if (rc) {
> -		CERROR("can't add initial connection\n");
> -		goto err_import;
> -	}
> -
> -	cli->cl_import = imp;
> -	/* cli->cl_max_mds_easize updated by mdc_init_ea_size() */
> -	cli->cl_max_mds_easize = sizeof(struct lov_mds_md_v3);
> -
> -	if (LUSTRE_CFG_BUFLEN(lcfg, 3) > 0) {
> -		if (!strcmp(lustre_cfg_string(lcfg, 3), "inactive")) {
> -			CDEBUG(D_HA, "marking %s %s->%s as inactive\n",
> -			       name, obddev->obd_name,
> -			       cli->cl_target_uuid.uuid);
> -			spin_lock(&imp->imp_lock);
> -			imp->imp_deactive = 1;
> -			spin_unlock(&imp->imp_lock);
> -		}
> -	}
> -
> -	obddev->obd_namespace = ldlm_namespace_new(obddev, obddev->obd_name,
> -						   LDLM_NAMESPACE_CLIENT,
> -						   LDLM_NAMESPACE_GREEDY,
> -						   ns_type);
> -	if (!obddev->obd_namespace) {
> -		CERROR("Unable to create client namespace - %s\n",
> -		       obddev->obd_name);
> -		rc = -ENOMEM;
> -		goto err_import;
> -	}
> -
> -	return rc;
> -
> -err_import:
> -	class_destroy_import(imp);
> -err_ldlm:
> -	ldlm_put_ref();
> -err:
> -	kfree(cli->cl_mod_tag_bitmap);
> -	cli->cl_mod_tag_bitmap = NULL;
> -	return rc;
> -}
> -EXPORT_SYMBOL(client_obd_setup);
> -
> -int client_obd_cleanup(struct obd_device *obddev)
> -{
> -	struct client_obd *cli = &obddev->u.cli;
> -
> -	ldlm_namespace_free_post(obddev->obd_namespace);
> -	obddev->obd_namespace = NULL;
> -
> -	obd_cleanup_client_import(obddev);
> -	LASSERT(!obddev->u.cli.cl_import);
> -
> -	ldlm_put_ref();
> -
> -	kfree(cli->cl_mod_tag_bitmap);
> -	cli->cl_mod_tag_bitmap = NULL;
> -
> -	return 0;
> -}
> -EXPORT_SYMBOL(client_obd_cleanup);
> -
> -/* ->o_connect() method for client side (OSC and MDC and MGC) */
> -int client_connect_import(const struct lu_env *env,
> -			  struct obd_export **exp,
> -			  struct obd_device *obd, struct obd_uuid *cluuid,
> -			  struct obd_connect_data *data, void *localdata)
> -{
> -	struct client_obd       *cli    = &obd->u.cli;
> -	struct obd_import       *imp    = cli->cl_import;
> -	struct obd_connect_data *ocd;
> -	struct lustre_handle    conn    = { 0 };
> -	bool is_mdc = false;
> -	int		     rc;
> -
> -	*exp = NULL;
> -	down_write(&cli->cl_sem);
> -	if (cli->cl_conn_count > 0) {
> -		rc = -EALREADY;
> -		goto out_sem;
> -	}
> -
> -	rc = class_connect(&conn, obd, cluuid);
> -	if (rc)
> -		goto out_sem;
> -
> -	cli->cl_conn_count++;
> -	*exp = class_conn2export(&conn);
> -
> -	LASSERT(obd->obd_namespace);
> -
> -	imp->imp_dlm_handle = conn;
> -	rc = ptlrpc_init_import(imp);
> -	if (rc != 0)
> -		goto out_ldlm;
> -
> -	ocd = &imp->imp_connect_data;
> -	if (data) {
> -		*ocd = *data;
> -		is_mdc = !strncmp(imp->imp_obd->obd_type->typ_name,
> -				  LUSTRE_MDC_NAME, 3);
> -		if (is_mdc)
> -			data->ocd_connect_flags |= OBD_CONNECT_MULTIMODRPCS;
> -		imp->imp_connect_flags_orig = data->ocd_connect_flags;
> -	}
> -
> -	rc = ptlrpc_connect_import(imp);
> -	if (rc != 0) {
> -		if (data && is_mdc)
> -			data->ocd_connect_flags &= ~OBD_CONNECT_MULTIMODRPCS;
> -		LASSERT(imp->imp_state == LUSTRE_IMP_DISCON);
> -		goto out_ldlm;
> -	}
> -	LASSERT(*exp && (*exp)->exp_connection);
> -
> -	if (data) {
> -		LASSERTF((ocd->ocd_connect_flags & data->ocd_connect_flags) ==
> -			 ocd->ocd_connect_flags, "old %#llx, new %#llx\n",
> -			 data->ocd_connect_flags, ocd->ocd_connect_flags);
> -		data->ocd_connect_flags = ocd->ocd_connect_flags;
> -		/* clear the flag as it was not set and is not known
> -		 * by upper layers
> -		 */
> -		if (is_mdc)
> -			data->ocd_connect_flags &= ~OBD_CONNECT_MULTIMODRPCS;
> -	}
> -
> -	ptlrpc_pinger_add_import(imp);
> -
> -	if (rc) {
> -out_ldlm:
> -		cli->cl_conn_count--;
> -		class_disconnect(*exp);
> -		*exp = NULL;
> -	}
> -out_sem:
> -	up_write(&cli->cl_sem);
> -
> -	return rc;
> -}
> -EXPORT_SYMBOL(client_connect_import);
> -
> -int client_disconnect_export(struct obd_export *exp)
> -{
> -	struct obd_device *obd = class_exp2obd(exp);
> -	struct client_obd *cli;
> -	struct obd_import *imp;
> -	int rc = 0, err;
> -
> -	if (!obd) {
> -		CERROR("invalid export for disconnect: exp %p cookie %#llx\n",
> -		       exp, exp ? exp->exp_handle.h_cookie : -1);
> -		return -EINVAL;
> -	}
> -
> -	cli = &obd->u.cli;
> -	imp = cli->cl_import;
> -
> -	down_write(&cli->cl_sem);
> -	CDEBUG(D_INFO, "disconnect %s - %zu\n", obd->obd_name,
> -	       cli->cl_conn_count);
> -
> -	if (!cli->cl_conn_count) {
> -		CERROR("disconnecting disconnected device (%s)\n",
> -		       obd->obd_name);
> -		rc = -EINVAL;
> -		goto out_disconnect;
> -	}
> -
> -	cli->cl_conn_count--;
> -	if (cli->cl_conn_count) {
> -		rc = 0;
> -		goto out_disconnect;
> -	}
> -
> -	/* Mark import deactivated now, so we don't try to reconnect if any
> -	 * of the cleanup RPCs fails (e.g. LDLM cancel, etc).  We don't
> -	 * fully deactivate the import, or that would drop all requests.
> -	 */
> -	spin_lock(&imp->imp_lock);
> -	imp->imp_deactive = 1;
> -	spin_unlock(&imp->imp_lock);
> -
> -	/* Some non-replayable imports (MDS's OSCs) are pinged, so just
> -	 * delete it regardless.  (It's safe to delete an import that was
> -	 * never added.)
> -	 */
> -	(void)ptlrpc_pinger_del_import(imp);
> -
> -	if (obd->obd_namespace) {
> -		/* obd_force == local only */
> -		ldlm_cli_cancel_unused(obd->obd_namespace, NULL,
> -				       obd->obd_force ? LCF_LOCAL : 0, NULL);
> -		ldlm_namespace_free_prior(obd->obd_namespace, imp,
> -					  obd->obd_force);
> -	}
> -
> -	/* There's no need to hold sem while disconnecting an import,
> -	 * and it may actually cause deadlock in GSS.
> -	 */
> -	up_write(&cli->cl_sem);
> -	rc = ptlrpc_disconnect_import(imp, 0);
> -	down_write(&cli->cl_sem);
> -
> -	ptlrpc_invalidate_import(imp);
> -
> -out_disconnect:
> -	/* Use server style - class_disconnect should be always called for
> -	 * o_disconnect.
> -	 */
> -	err = class_disconnect(exp);
> -	if (!rc && err)
> -		rc = err;
> -
> -	up_write(&cli->cl_sem);
> -
> -	return rc;
> -}
> -EXPORT_SYMBOL(client_disconnect_export);
> -
> -/**
> - * Packs current SLV and Limit into \a req.
> - */
> -int target_pack_pool_reply(struct ptlrpc_request *req)
> -{
> -	struct obd_device *obd;
> -
> -	/* Check that we still have all structures alive as this may
> -	 * be some late RPC at shutdown time.
> -	 */
> -	if (unlikely(!req->rq_export || !req->rq_export->exp_obd ||
> -		     !exp_connect_lru_resize(req->rq_export))) {
> -		lustre_msg_set_slv(req->rq_repmsg, 0);
> -		lustre_msg_set_limit(req->rq_repmsg, 0);
> -		return 0;
> -	}
> -
> -	/* OBD is alive here as export is alive, which we checked above. */
> -	obd = req->rq_export->exp_obd;
> -
> -	read_lock(&obd->obd_pool_lock);
> -	lustre_msg_set_slv(req->rq_repmsg, obd->obd_pool_slv);
> -	lustre_msg_set_limit(req->rq_repmsg, obd->obd_pool_limit);
> -	read_unlock(&obd->obd_pool_lock);
> -
> -	return 0;
> -}
> -EXPORT_SYMBOL(target_pack_pool_reply);
> -
> -static int
> -target_send_reply_msg(struct ptlrpc_request *req, int rc, int fail_id)
> -{
> -	if (OBD_FAIL_CHECK_ORSET(fail_id & ~OBD_FAIL_ONCE, OBD_FAIL_ONCE)) {
> -		DEBUG_REQ(D_ERROR, req, "dropping reply");
> -		return -ECOMM;
> -	}
> -
> -	if (unlikely(rc)) {
> -		DEBUG_REQ(D_NET, req, "processing error (%d)", rc);
> -		req->rq_status = rc;
> -		return ptlrpc_send_error(req, 1);
> -	}
> -
> -	DEBUG_REQ(D_NET, req, "sending reply");
> -	return ptlrpc_send_reply(req, PTLRPC_REPLY_MAYBE_DIFFICULT);
> -}
> -
> -void target_send_reply(struct ptlrpc_request *req, int rc, int fail_id)
> -{
> -	struct ptlrpc_service_part *svcpt;
> -	int			netrc;
> -	struct ptlrpc_reply_state *rs;
> -	struct obd_export	 *exp;
> -
> -	if (req->rq_no_reply)
> -		return;
> -
> -	svcpt = req->rq_rqbd->rqbd_svcpt;
> -	rs = req->rq_reply_state;
> -	if (!rs || !rs->rs_difficult) {
> -		/* no notifiers */
> -		target_send_reply_msg(req, rc, fail_id);
> -		return;
> -	}
> -
> -	/* must be an export if locks saved */
> -	LASSERT(req->rq_export);
> -	/* req/reply consistent */
> -	LASSERT(rs->rs_svcpt == svcpt);
> -
> -	/* "fresh" reply */
> -	LASSERT(!rs->rs_scheduled);
> -	LASSERT(!rs->rs_scheduled_ever);
> -	LASSERT(!rs->rs_handled);
> -	LASSERT(!rs->rs_on_net);
> -	LASSERT(!rs->rs_export);
> -	LASSERT(list_empty(&rs->rs_obd_list));
> -	LASSERT(list_empty(&rs->rs_exp_list));
> -
> -	exp = class_export_get(req->rq_export);
> -
> -	/* disable reply scheduling while I'm setting up */
> -	rs->rs_scheduled = 1;
> -	rs->rs_on_net    = 1;
> -	rs->rs_xid       = req->rq_xid;
> -	rs->rs_transno   = req->rq_transno;
> -	rs->rs_export    = exp;
> -	rs->rs_opc       = lustre_msg_get_opc(req->rq_reqmsg);
> -
> -	spin_lock(&exp->exp_uncommitted_replies_lock);
> -	CDEBUG(D_NET, "rs transno = %llu, last committed = %llu\n",
> -	       rs->rs_transno, exp->exp_last_committed);
> -	if (rs->rs_transno > exp->exp_last_committed) {
> -		/* not committed already */
> -		list_add_tail(&rs->rs_obd_list,
> -			      &exp->exp_uncommitted_replies);
> -	}
> -	spin_unlock(&exp->exp_uncommitted_replies_lock);
> -
> -	spin_lock(&exp->exp_lock);
> -	list_add_tail(&rs->rs_exp_list, &exp->exp_outstanding_replies);
> -	spin_unlock(&exp->exp_lock);
> -
> -	netrc = target_send_reply_msg(req, rc, fail_id);
> -
> -	spin_lock(&svcpt->scp_rep_lock);
> -
> -	atomic_inc(&svcpt->scp_nreps_difficult);
> -
> -	if (netrc != 0) {
> -		/* error sending: reply is off the net.  Also we need +1
> -		 * reply ref until ptlrpc_handle_rs() is done
> -		 * with the reply state (if the send was successful, there
> -		 * would have been +1 ref for the net, which
> -		 * reply_out_callback leaves alone)
> -		 */
> -		rs->rs_on_net = 0;
> -		ptlrpc_rs_addref(rs);
> -	}
> -
> -	spin_lock(&rs->rs_lock);
> -	if (rs->rs_transno <= exp->exp_last_committed ||
> -	    (!rs->rs_on_net && !rs->rs_no_ack) ||
> -	    list_empty(&rs->rs_exp_list) ||     /* completed already */
> -	    list_empty(&rs->rs_obd_list)) {
> -		CDEBUG(D_HA, "Schedule reply immediately\n");
> -		ptlrpc_dispatch_difficult_reply(rs);
> -	} else {
> -		list_add(&rs->rs_list, &svcpt->scp_rep_active);
> -		rs->rs_scheduled = 0;	/* allow notifier to schedule */
> -	}
> -	spin_unlock(&rs->rs_lock);
> -	spin_unlock(&svcpt->scp_rep_lock);
> -}
> -EXPORT_SYMBOL(target_send_reply);
> -
> -enum ldlm_mode lck_compat_array[] = {
> -	[LCK_EX]	= LCK_COMPAT_EX,
> -	[LCK_PW]	= LCK_COMPAT_PW,
> -	[LCK_PR]	= LCK_COMPAT_PR,
> -	[LCK_CW]	= LCK_COMPAT_CW,
> -	[LCK_CR]	= LCK_COMPAT_CR,
> -	[LCK_NL]	= LCK_COMPAT_NL,
> -	[LCK_GROUP]	= LCK_COMPAT_GROUP,
> -	[LCK_COS]	= LCK_COMPAT_COS,
> -};
> -
> -/**
> - * Rather arbitrary mapping from LDLM error codes to errno values. This should
> - * not escape to the user level.
> - */
> -int ldlm_error2errno(enum ldlm_error error)
> -{
> -	int result;
> -
> -	switch (error) {
> -	case ELDLM_OK:
> -	case ELDLM_LOCK_MATCHED:
> -		result = 0;
> -		break;
> -	case ELDLM_LOCK_CHANGED:
> -		result = -ESTALE;
> -		break;
> -	case ELDLM_LOCK_ABORTED:
> -		result = -ENAVAIL;
> -		break;
> -	case ELDLM_LOCK_REPLACED:
> -		result = -ESRCH;
> -		break;
> -	case ELDLM_NO_LOCK_DATA:
> -		result = -ENOENT;
> -		break;
> -	case ELDLM_NAMESPACE_EXISTS:
> -		result = -EEXIST;
> -		break;
> -	case ELDLM_BAD_NAMESPACE:
> -		result = -EBADF;
> -		break;
> -	default:
> -		if (((int)error) < 0)  /* cast to signed type */
> -			result = error; /* as enum ldlm_error can be unsigned */
> -		else {
> -			CERROR("Invalid DLM result code: %d\n", error);
> -			result = -EPROTO;
> -		}
> -	}
> -	return result;
> -}
> -EXPORT_SYMBOL(ldlm_error2errno);
> -
> -#if LUSTRE_TRACKS_LOCK_EXP_REFS
> -void ldlm_dump_export_locks(struct obd_export *exp)
> -{
> -	spin_lock(&exp->exp_locks_list_guard);
> -	if (!list_empty(&exp->exp_locks_list)) {
> -		struct ldlm_lock *lock;
> -
> -		CERROR("dumping locks for export %p,ignore if the unmount doesn't hang\n",
> -		       exp);
> -		list_for_each_entry(lock, &exp->exp_locks_list,
> -				    l_exp_refs_link)
> -			LDLM_ERROR(lock, "lock:");
> -	}
> -	spin_unlock(&exp->exp_locks_list_guard);
> -}
> -#endif
> diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c b/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c
> deleted file mode 100644
> index 2fb2e088dc87..000000000000
> --- a/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c
> +++ /dev/null
> @@ -1,2103 +0,0 @@
> -// SPDX-License-Identifier: GPL-2.0
> -/*
> - * GPL HEADER START
> - *
> - * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
> - *
> - * This program is free software; you can redistribute it and/or modify
> - * it under the terms of the GNU General Public License version 2 only,
> - * as published by the Free Software Foundation.
> - *
> - * This program is distributed in the hope that it will be useful, but
> - * WITHOUT ANY WARRANTY; without even the implied warranty of
> - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> - * General Public License version 2 for more details (a copy is included
> - * in the LICENSE file that accompanied this code).
> - *
> - * You should have received a copy of the GNU General Public License
> - * version 2 along with this program; If not, see
> - * http://www.gnu.org/licenses/gpl-2.0.html
> - *
> - * GPL HEADER END
> - */
> -/*
> - * Copyright (c) 2002, 2010, Oracle and/or its affiliates. All rights reserved.
> - * Use is subject to license terms.
> - *
> - * Copyright (c) 2010, 2015, Intel Corporation.
> - */
> -/*
> - * This file is part of Lustre, http://www.lustre.org/
> - * Lustre is a trademark of Sun Microsystems, Inc.
> - *
> - * lustre/ldlm/ldlm_lock.c
> - *
> - * Author: Peter Braam <braam at clusterfs.com>
> - * Author: Phil Schwan <phil at clusterfs.com>
> - */
> -
> -#define DEBUG_SUBSYSTEM S_LDLM
> -
> -#include <lustre_intent.h>
> -#include <lustre_swab.h>
> -#include <obd_class.h>
> -#include "ldlm_internal.h"
> -
> -/* lock types */
> -char *ldlm_lockname[] = {
> -	[0]		= "--",
> -	[LCK_EX]	= "EX",
> -	[LCK_PW]	= "PW",
> -	[LCK_PR]	= "PR",
> -	[LCK_CW]	= "CW",
> -	[LCK_CR]	= "CR",
> -	[LCK_NL]	= "NL",
> -	[LCK_GROUP]	= "GROUP",
> -	[LCK_COS]	= "COS",
> -};
> -EXPORT_SYMBOL(ldlm_lockname);
> -
> -static char *ldlm_typename[] = {
> -	[LDLM_PLAIN]	= "PLN",
> -	[LDLM_EXTENT]	= "EXT",
> -	[LDLM_FLOCK]	= "FLK",
> -	[LDLM_IBITS]	= "IBT",
> -};
> -
> -static ldlm_policy_wire_to_local_t ldlm_policy_wire_to_local[] = {
> -	[LDLM_PLAIN - LDLM_MIN_TYPE]	= ldlm_plain_policy_wire_to_local,
> -	[LDLM_EXTENT - LDLM_MIN_TYPE]	= ldlm_extent_policy_wire_to_local,
> -	[LDLM_FLOCK - LDLM_MIN_TYPE]	= ldlm_flock_policy_wire_to_local,
> -	[LDLM_IBITS - LDLM_MIN_TYPE]	= ldlm_ibits_policy_wire_to_local,
> -};
> -
> -static ldlm_policy_local_to_wire_t ldlm_policy_local_to_wire[] = {
> -	[LDLM_PLAIN - LDLM_MIN_TYPE]	= ldlm_plain_policy_local_to_wire,
> -	[LDLM_EXTENT - LDLM_MIN_TYPE]	= ldlm_extent_policy_local_to_wire,
> -	[LDLM_FLOCK - LDLM_MIN_TYPE]	= ldlm_flock_policy_local_to_wire,
> -	[LDLM_IBITS - LDLM_MIN_TYPE]	= ldlm_ibits_policy_local_to_wire,
> -};
> -
> -/**
> - * Converts lock policy from local format to on the wire lock_desc format
> - */
> -static void ldlm_convert_policy_to_wire(enum ldlm_type type,
> -					const union ldlm_policy_data *lpolicy,
> -					union ldlm_wire_policy_data *wpolicy)
> -{
> -	ldlm_policy_local_to_wire_t convert;
> -
> -	convert = ldlm_policy_local_to_wire[type - LDLM_MIN_TYPE];
> -
> -	convert(lpolicy, wpolicy);
> -}
> -
> -/**
> - * Converts lock policy from on the wire lock_desc format to local format
> - */
> -void ldlm_convert_policy_to_local(struct obd_export *exp, enum ldlm_type type,
> -				  const union ldlm_wire_policy_data *wpolicy,
> -				  union ldlm_policy_data *lpolicy)
> -{
> -	ldlm_policy_wire_to_local_t convert;
> -
> -	convert = ldlm_policy_wire_to_local[type - LDLM_MIN_TYPE];
> -
> -	convert(wpolicy, lpolicy);
> -}
> -
> -const char *ldlm_it2str(enum ldlm_intent_flags it)
> -{
> -	switch (it) {
> -	case IT_OPEN:
> -		return "open";
> -	case IT_CREAT:
> -		return "creat";
> -	case (IT_OPEN | IT_CREAT):
> -		return "open|creat";
> -	case IT_READDIR:
> -		return "readdir";
> -	case IT_GETATTR:
> -		return "getattr";
> -	case IT_LOOKUP:
> -		return "lookup";
> -	case IT_UNLINK:
> -		return "unlink";
> -	case IT_GETXATTR:
> -		return "getxattr";
> -	case IT_LAYOUT:
> -		return "layout";
> -	default:
> -		CERROR("Unknown intent 0x%08x\n", it);
> -		return "UNKNOWN";
> -	}
> -}
> -EXPORT_SYMBOL(ldlm_it2str);
> -
> -/*
> - * REFCOUNTED LOCK OBJECTS
> - */
> -
> -/**
> - * Get a reference on a lock.
> - *
> - * Lock refcounts, during creation:
> - *   - one special one for allocation, dec'd only once in destroy
> - *   - one for being a lock that's in-use
> - *   - one for the addref associated with a new lock
> - */
> -struct ldlm_lock *ldlm_lock_get(struct ldlm_lock *lock)
> -{
> -	atomic_inc(&lock->l_refc);
> -	return lock;
> -}
> -EXPORT_SYMBOL(ldlm_lock_get);
> -
> -/**
> - * Release lock reference.
> - *
> - * Also frees the lock if it was last reference.
> - */
> -void ldlm_lock_put(struct ldlm_lock *lock)
> -{
> -	LASSERT(lock->l_resource != LP_POISON);
> -	LASSERT(atomic_read(&lock->l_refc) > 0);
> -	if (atomic_dec_and_test(&lock->l_refc)) {
> -		struct ldlm_resource *res;
> -
> -		LDLM_DEBUG(lock,
> -			   "final lock_put on destroyed lock, freeing it.");
> -
> -		res = lock->l_resource;
> -		LASSERT(ldlm_is_destroyed(lock));
> -		LASSERT(list_empty(&lock->l_res_link));
> -		LASSERT(list_empty(&lock->l_pending_chain));
> -
> -		lprocfs_counter_decr(ldlm_res_to_ns(res)->ns_stats,
> -				     LDLM_NSS_LOCKS);
> -		lu_ref_del(&res->lr_reference, "lock", lock);
> -		ldlm_resource_putref(res);
> -		lock->l_resource = NULL;
> -		if (lock->l_export) {
> -			class_export_lock_put(lock->l_export, lock);
> -			lock->l_export = NULL;
> -		}
> -
> -		kfree(lock->l_lvb_data);
> -
> -		lu_ref_fini(&lock->l_reference);
> -		OBD_FREE_RCU(lock, sizeof(*lock), &lock->l_handle);
> -	}
> -}
> -EXPORT_SYMBOL(ldlm_lock_put);
> -
> -/**
> - * Removes LDLM lock \a lock from LRU. Assumes LRU is already locked.
> - */
> -int ldlm_lock_remove_from_lru_nolock(struct ldlm_lock *lock)
> -{
> -	int rc = 0;
> -
> -	if (!list_empty(&lock->l_lru)) {
> -		struct ldlm_namespace *ns = ldlm_lock_to_ns(lock);
> -
> -		LASSERT(lock->l_resource->lr_type != LDLM_FLOCK);
> -		list_del_init(&lock->l_lru);
> -		LASSERT(ns->ns_nr_unused > 0);
> -		ns->ns_nr_unused--;
> -		rc = 1;
> -	}
> -	return rc;
> -}
> -
> -/**
> - * Removes LDLM lock \a lock from LRU. Obtains the LRU lock first.
> - *
> - * If \a last_use is non-zero, it will remove the lock from LRU only if
> - * it matches lock's l_last_used.
> - *
> - * \retval 0 if \a last_use is set, the lock is not in LRU list or \a last_use
> - *           doesn't match lock's l_last_used;
> - *           otherwise, the lock hasn't been in the LRU list.
> - * \retval 1 the lock was in LRU list and removed.
> - */
> -int ldlm_lock_remove_from_lru_check(struct ldlm_lock *lock, time_t last_use)
> -{
> -	struct ldlm_namespace *ns = ldlm_lock_to_ns(lock);
> -	int rc = 0;
> -
> -	spin_lock(&ns->ns_lock);
> -	if (last_use == 0 || last_use == lock->l_last_used)
> -		rc = ldlm_lock_remove_from_lru_nolock(lock);
> -	spin_unlock(&ns->ns_lock);
> -
> -	return rc;
> -}
> -
> -/**
> - * Adds LDLM lock \a lock to namespace LRU. Assumes LRU is already locked.
> - */
> -static void ldlm_lock_add_to_lru_nolock(struct ldlm_lock *lock)
> -{
> -	struct ldlm_namespace *ns = ldlm_lock_to_ns(lock);
> -
> -	lock->l_last_used = jiffies;
> -	LASSERT(list_empty(&lock->l_lru));
> -	LASSERT(lock->l_resource->lr_type != LDLM_FLOCK);
> -	list_add_tail(&lock->l_lru, &ns->ns_unused_list);
> -	ldlm_clear_skipped(lock);
> -	LASSERT(ns->ns_nr_unused >= 0);
> -	ns->ns_nr_unused++;
> -}
> -
> -/**
> - * Adds LDLM lock \a lock to namespace LRU. Obtains necessary LRU locks
> - * first.
> - */
> -static void ldlm_lock_add_to_lru(struct ldlm_lock *lock)
> -{
> -	struct ldlm_namespace *ns = ldlm_lock_to_ns(lock);
> -
> -	spin_lock(&ns->ns_lock);
> -	ldlm_lock_add_to_lru_nolock(lock);
> -	spin_unlock(&ns->ns_lock);
> -}
> -
> -/**
> - * Moves LDLM lock \a lock that is already in namespace LRU to the tail of
> - * the LRU. Performs necessary LRU locking
> - */
> -static void ldlm_lock_touch_in_lru(struct ldlm_lock *lock)
> -{
> -	struct ldlm_namespace *ns = ldlm_lock_to_ns(lock);
> -
> -	spin_lock(&ns->ns_lock);
> -	if (!list_empty(&lock->l_lru)) {
> -		ldlm_lock_remove_from_lru_nolock(lock);
> -		ldlm_lock_add_to_lru_nolock(lock);
> -	}
> -	spin_unlock(&ns->ns_lock);
> -}
> -
> -/**
> - * Helper to destroy a locked lock.
> - *
> - * Used by ldlm_lock_destroy and ldlm_lock_destroy_nolock
> - * Must be called with l_lock and lr_lock held.
> - *
> - * Does not actually free the lock data, but rather marks the lock as
> - * destroyed by setting l_destroyed field in the lock to 1.  Destroys a
> - * handle->lock association too, so that the lock can no longer be found
> - * and removes the lock from LRU list.  Actual lock freeing occurs when
> - * last lock reference goes away.
> - *
> - * Original comment (of some historical value):
> - * This used to have a 'strict' flag, which recovery would use to mark an
> - * in-use lock as needing-to-die.  Lest I am ever tempted to put it back, I
> - * shall explain why it's gone: with the new hash table scheme, once you call
> - * ldlm_lock_destroy, you can never drop your final references on this lock.
> - * Because it's not in the hash table anymore.  -phil
> - */
> -static int ldlm_lock_destroy_internal(struct ldlm_lock *lock)
> -{
> -	if (lock->l_readers || lock->l_writers) {
> -		LDLM_ERROR(lock, "lock still has references");
> -		LBUG();
> -	}
> -
> -	if (!list_empty(&lock->l_res_link)) {
> -		LDLM_ERROR(lock, "lock still on resource");
> -		LBUG();
> -	}
> -
> -	if (ldlm_is_destroyed(lock)) {
> -		LASSERT(list_empty(&lock->l_lru));
> -		return 0;
> -	}
> -	ldlm_set_destroyed(lock);
> -
> -	ldlm_lock_remove_from_lru(lock);
> -	class_handle_unhash(&lock->l_handle);
> -
> -	return 1;
> -}
> -
> -/**
> - * Destroys a LDLM lock \a lock. Performs necessary locking first.
> - */
> -static void ldlm_lock_destroy(struct ldlm_lock *lock)
> -{
> -	int first;
> -
> -	lock_res_and_lock(lock);
> -	first = ldlm_lock_destroy_internal(lock);
> -	unlock_res_and_lock(lock);
> -
> -	/* drop reference from hashtable only for first destroy */
> -	if (first) {
> -		lu_ref_del(&lock->l_reference, "hash", lock);
> -		LDLM_LOCK_RELEASE(lock);
> -	}
> -}
> -
> -/**
> - * Destroys a LDLM lock \a lock that is already locked.
> - */
> -void ldlm_lock_destroy_nolock(struct ldlm_lock *lock)
> -{
> -	int first;
> -
> -	first = ldlm_lock_destroy_internal(lock);
> -	/* drop reference from hashtable only for first destroy */
> -	if (first) {
> -		lu_ref_del(&lock->l_reference, "hash", lock);
> -		LDLM_LOCK_RELEASE(lock);
> -	}
> -}
> -
> -/* this is called by portals_handle2object with the handle lock taken */
> -static void lock_handle_addref(void *lock)
> -{
> -	LDLM_LOCK_GET((struct ldlm_lock *)lock);
> -}
> -
> -static void lock_handle_free(void *lock, int size)
> -{
> -	LASSERT(size == sizeof(struct ldlm_lock));
> -	kmem_cache_free(ldlm_lock_slab, lock);
> -}
> -
> -static struct portals_handle_ops lock_handle_ops = {
> -	.hop_addref = lock_handle_addref,
> -	.hop_free   = lock_handle_free,
> -};
> -
> -/**
> - *
> - * Allocate and initialize new lock structure.
> - *
> - * usage: pass in a resource on which you have done ldlm_resource_get
> - *	new lock will take over the refcount.
> - * returns: lock with refcount 2 - one for current caller and one for remote
> - */
> -static struct ldlm_lock *ldlm_lock_new(struct ldlm_resource *resource)
> -{
> -	struct ldlm_lock *lock;
> -
> -	LASSERT(resource);
> -
> -	lock = kmem_cache_zalloc(ldlm_lock_slab, GFP_NOFS);
> -	if (!lock)
> -		return NULL;
> -
> -	spin_lock_init(&lock->l_lock);
> -	lock->l_resource = resource;
> -	lu_ref_add(&resource->lr_reference, "lock", lock);
> -
> -	atomic_set(&lock->l_refc, 2);
> -	INIT_LIST_HEAD(&lock->l_res_link);
> -	INIT_LIST_HEAD(&lock->l_lru);
> -	INIT_LIST_HEAD(&lock->l_pending_chain);
> -	INIT_LIST_HEAD(&lock->l_bl_ast);
> -	INIT_LIST_HEAD(&lock->l_cp_ast);
> -	INIT_LIST_HEAD(&lock->l_rk_ast);
> -	init_waitqueue_head(&lock->l_waitq);
> -	lock->l_blocking_lock = NULL;
> -	INIT_LIST_HEAD(&lock->l_sl_mode);
> -	INIT_LIST_HEAD(&lock->l_sl_policy);
> -	RB_CLEAR_NODE(&lock->l_rb);
> -
> -	lprocfs_counter_incr(ldlm_res_to_ns(resource)->ns_stats,
> -			     LDLM_NSS_LOCKS);
> -	INIT_LIST_HEAD(&lock->l_handle.h_link);
> -	class_handle_hash(&lock->l_handle, &lock_handle_ops);
> -
> -	lu_ref_init(&lock->l_reference);
> -	lu_ref_add(&lock->l_reference, "hash", lock);
> -	lock->l_callback_timeout = 0;
> -
> -#if LUSTRE_TRACKS_LOCK_EXP_REFS
> -	INIT_LIST_HEAD(&lock->l_exp_refs_link);
> -	lock->l_exp_refs_nr = 0;
> -	lock->l_exp_refs_target = NULL;
> -#endif
> -
> -	return lock;
> -}
> -
> -/**
> - * Moves LDLM lock \a lock to another resource.
> - * This is used on client when server returns some other lock than requested
> - * (typically as a result of intent operation)
> - */
> -int ldlm_lock_change_resource(struct ldlm_namespace *ns, struct ldlm_lock *lock,
> -			      const struct ldlm_res_id *new_resid)
> -{
> -	struct ldlm_resource *oldres = lock->l_resource;
> -	struct ldlm_resource *newres;
> -	int type;
> -
> -	lock_res_and_lock(lock);
> -	if (memcmp(new_resid, &lock->l_resource->lr_name,
> -		   sizeof(lock->l_resource->lr_name)) == 0) {
> -		/* Nothing to do */
> -		unlock_res_and_lock(lock);
> -		return 0;
> -	}
> -
> -	LASSERT(new_resid->name[0] != 0);
> -
> -	/* This function assumes that the lock isn't on any lists */
> -	LASSERT(list_empty(&lock->l_res_link));
> -
> -	type = oldres->lr_type;
> -	unlock_res_and_lock(lock);
> -
> -	newres = ldlm_resource_get(ns, NULL, new_resid, type, 1);
> -	if (IS_ERR(newres))
> -		return PTR_ERR(newres);
> -
> -	lu_ref_add(&newres->lr_reference, "lock", lock);
> -	/*
> -	 * To flip the lock from the old to the new resource, lock, oldres and
> -	 * newres have to be locked. Resource spin-locks are nested within
> -	 * lock->l_lock, and are taken in the memory address order to avoid
> -	 * dead-locks.
> -	 */
> -	spin_lock(&lock->l_lock);
> -	oldres = lock->l_resource;
> -	if (oldres < newres) {
> -		lock_res(oldres);
> -		lock_res_nested(newres, LRT_NEW);
> -	} else {
> -		lock_res(newres);
> -		lock_res_nested(oldres, LRT_NEW);
> -	}
> -	LASSERT(memcmp(new_resid, &oldres->lr_name,
> -		       sizeof(oldres->lr_name)) != 0);
> -	lock->l_resource = newres;
> -	unlock_res(oldres);
> -	unlock_res_and_lock(lock);
> -
> -	/* ...and the flowers are still standing! */
> -	lu_ref_del(&oldres->lr_reference, "lock", lock);
> -	ldlm_resource_putref(oldres);
> -
> -	return 0;
> -}
> -
> -/** \defgroup ldlm_handles LDLM HANDLES
> - * Ways to get hold of locks without any addresses.
> - * @{
> - */
> -
> -/**
> - * Fills in handle for LDLM lock \a lock into supplied \a lockh
> - * Does not take any references.
> - */
> -void ldlm_lock2handle(const struct ldlm_lock *lock, struct lustre_handle *lockh)
> -{
> -	lockh->cookie = lock->l_handle.h_cookie;
> -}
> -EXPORT_SYMBOL(ldlm_lock2handle);
> -
> -/**
> - * Obtain a lock reference by handle.
> - *
> - * if \a flags: atomically get the lock and set the flags.
> - *	      Return NULL if flag already set
> - */
> -struct ldlm_lock *__ldlm_handle2lock(const struct lustre_handle *handle,
> -				     __u64 flags)
> -{
> -	struct ldlm_lock *lock;
> -
> -	LASSERT(handle);
> -
> -	lock = class_handle2object(handle->cookie, NULL);
> -	if (!lock)
> -		return NULL;
> -
> -	if (lock->l_export && lock->l_export->exp_failed) {
> -		CDEBUG(D_INFO, "lock export failed: lock %p, exp %p\n",
> -		       lock, lock->l_export);
> -		LDLM_LOCK_PUT(lock);
> -		return NULL;
> -	}
> -
> -	/* It's unlikely but possible that someone marked the lock as
> -	 * destroyed after we did handle2object on it
> -	 */
> -	if (flags == 0 && !ldlm_is_destroyed(lock)) {
> -		lu_ref_add(&lock->l_reference, "handle", current);
> -		return lock;
> -	}
> -
> -	lock_res_and_lock(lock);
> -
> -	LASSERT(lock->l_resource);
> -
> -	lu_ref_add_atomic(&lock->l_reference, "handle", current);
> -	if (unlikely(ldlm_is_destroyed(lock))) {
> -		unlock_res_and_lock(lock);
> -		CDEBUG(D_INFO, "lock already destroyed: lock %p\n", lock);
> -		LDLM_LOCK_PUT(lock);
> -		return NULL;
> -	}
> -
> -	if (flags) {
> -		if (lock->l_flags & flags) {
> -			unlock_res_and_lock(lock);
> -			LDLM_LOCK_PUT(lock);
> -			return NULL;
> -		}
> -
> -		lock->l_flags |= flags;
> -	}
> -
> -	unlock_res_and_lock(lock);
> -	return lock;
> -}
> -EXPORT_SYMBOL(__ldlm_handle2lock);
> -/** @} ldlm_handles */
> -
> -/**
> - * Fill in "on the wire" representation for given LDLM lock into supplied
> - * lock descriptor \a desc structure.
> - */
> -void ldlm_lock2desc(struct ldlm_lock *lock, struct ldlm_lock_desc *desc)
> -{
> -	ldlm_res2desc(lock->l_resource, &desc->l_resource);
> -	desc->l_req_mode = lock->l_req_mode;
> -	desc->l_granted_mode = lock->l_granted_mode;
> -	ldlm_convert_policy_to_wire(lock->l_resource->lr_type,
> -				    &lock->l_policy_data,
> -				    &desc->l_policy_data);
> -}
> -
> -/**
> - * Add a lock to list of conflicting locks to send AST to.
> - *
> - * Only add if we have not sent a blocking AST to the lock yet.
> - */
> -static void ldlm_add_bl_work_item(struct ldlm_lock *lock, struct ldlm_lock *new,
> -				  struct list_head *work_list)
> -{
> -	if (!ldlm_is_ast_sent(lock)) {
> -		LDLM_DEBUG(lock, "lock incompatible; sending blocking AST.");
> -		ldlm_set_ast_sent(lock);
> -		/* If the enqueuing client said so, tell the AST recipient to
> -		 * discard dirty data, rather than writing back.
> -		 */
> -		if (ldlm_is_ast_discard_data(new))
> -			ldlm_set_discard_data(lock);
> -		LASSERT(list_empty(&lock->l_bl_ast));
> -		list_add(&lock->l_bl_ast, work_list);
> -		LDLM_LOCK_GET(lock);
> -		LASSERT(!lock->l_blocking_lock);
> -		lock->l_blocking_lock = LDLM_LOCK_GET(new);
> -	}
> -}
> -
> -/**
> - * Add a lock to list of just granted locks to send completion AST to.
> - */
> -static void ldlm_add_cp_work_item(struct ldlm_lock *lock,
> -				  struct list_head *work_list)
> -{
> -	if (!ldlm_is_cp_reqd(lock)) {
> -		ldlm_set_cp_reqd(lock);
> -		LDLM_DEBUG(lock, "lock granted; sending completion AST.");
> -		LASSERT(list_empty(&lock->l_cp_ast));
> -		list_add(&lock->l_cp_ast, work_list);
> -		LDLM_LOCK_GET(lock);
> -	}
> -}
> -
> -/**
> - * Aggregator function to add AST work items into a list. Determines
> - * what sort of an AST work needs to be done and calls the proper
> - * adding function.
> - * Must be called with lr_lock held.
> - */
> -static void ldlm_add_ast_work_item(struct ldlm_lock *lock,
> -				   struct ldlm_lock *new,
> -				   struct list_head *work_list)
> -{
> -	check_res_locked(lock->l_resource);
> -	if (new)
> -		ldlm_add_bl_work_item(lock, new, work_list);
> -	else
> -		ldlm_add_cp_work_item(lock, work_list);
> -}
> -
> -/**
> - * Add specified reader/writer reference to LDLM lock with handle \a lockh.
> - * r/w reference type is determined by \a mode
> - * Calls ldlm_lock_addref_internal.
> - */
> -void ldlm_lock_addref(const struct lustre_handle *lockh, enum ldlm_mode mode)
> -{
> -	struct ldlm_lock *lock;
> -
> -	lock = ldlm_handle2lock(lockh);
> -	LASSERTF(lock, "Non-existing lock: %llx\n", lockh->cookie);
> -	ldlm_lock_addref_internal(lock, mode);
> -	LDLM_LOCK_PUT(lock);
> -}
> -EXPORT_SYMBOL(ldlm_lock_addref);
> -
> -/**
> - * Helper function.
> - * Add specified reader/writer reference to LDLM lock \a lock.
> - * r/w reference type is determined by \a mode
> - * Removes lock from LRU if it is there.
> - * Assumes the LDLM lock is already locked.
> - */
> -void ldlm_lock_addref_internal_nolock(struct ldlm_lock *lock,
> -				      enum ldlm_mode mode)
> -{
> -	ldlm_lock_remove_from_lru(lock);
> -	if (mode & (LCK_NL | LCK_CR | LCK_PR)) {
> -		lock->l_readers++;
> -		lu_ref_add_atomic(&lock->l_reference, "reader", lock);
> -	}
> -	if (mode & (LCK_EX | LCK_CW | LCK_PW | LCK_GROUP | LCK_COS)) {
> -		lock->l_writers++;
> -		lu_ref_add_atomic(&lock->l_reference, "writer", lock);
> -	}
> -	LDLM_LOCK_GET(lock);
> -	lu_ref_add_atomic(&lock->l_reference, "user", lock);
> -	LDLM_DEBUG(lock, "ldlm_lock_addref(%s)", ldlm_lockname[mode]);
> -}
> -
> -/**
> - * Attempts to add reader/writer reference to a lock with handle \a lockh, and
> - * fails if lock is already LDLM_FL_CBPENDING or destroyed.
> - *
> - * \retval 0 success, lock was addref-ed
> - *
> - * \retval -EAGAIN lock is being canceled.
> - */
> -int ldlm_lock_addref_try(const struct lustre_handle *lockh, enum ldlm_mode mode)
> -{
> -	struct ldlm_lock *lock;
> -	int	       result;
> -
> -	result = -EAGAIN;
> -	lock = ldlm_handle2lock(lockh);
> -	if (lock) {
> -		lock_res_and_lock(lock);
> -		if (lock->l_readers != 0 || lock->l_writers != 0 ||
> -		    !ldlm_is_cbpending(lock)) {
> -			ldlm_lock_addref_internal_nolock(lock, mode);
> -			result = 0;
> -		}
> -		unlock_res_and_lock(lock);
> -		LDLM_LOCK_PUT(lock);
> -	}
> -	return result;
> -}
> -EXPORT_SYMBOL(ldlm_lock_addref_try);
> -
> -/**
> - * Add specified reader/writer reference to LDLM lock \a lock.
> - * Locks LDLM lock and calls ldlm_lock_addref_internal_nolock to do the work.
> - * Only called for local locks.
> - */
> -void ldlm_lock_addref_internal(struct ldlm_lock *lock, enum ldlm_mode mode)
> -{
> -	lock_res_and_lock(lock);
> -	ldlm_lock_addref_internal_nolock(lock, mode);
> -	unlock_res_and_lock(lock);
> -}
> -
> -/**
> - * Removes reader/writer reference for LDLM lock \a lock.
> - * Assumes LDLM lock is already locked.
> - * only called in ldlm_flock_destroy and for local locks.
> - * Does NOT add lock to LRU if no r/w references left to accommodate flock locks
> - * that cannot be placed in LRU.
> - */
> -void ldlm_lock_decref_internal_nolock(struct ldlm_lock *lock,
> -				      enum ldlm_mode mode)
> -{
> -	LDLM_DEBUG(lock, "ldlm_lock_decref(%s)", ldlm_lockname[mode]);
> -	if (mode & (LCK_NL | LCK_CR | LCK_PR)) {
> -		LASSERT(lock->l_readers > 0);
> -		lu_ref_del(&lock->l_reference, "reader", lock);
> -		lock->l_readers--;
> -	}
> -	if (mode & (LCK_EX | LCK_CW | LCK_PW | LCK_GROUP | LCK_COS)) {
> -		LASSERT(lock->l_writers > 0);
> -		lu_ref_del(&lock->l_reference, "writer", lock);
> -		lock->l_writers--;
> -	}
> -
> -	lu_ref_del(&lock->l_reference, "user", lock);
> -	LDLM_LOCK_RELEASE(lock);    /* matches the LDLM_LOCK_GET() in addref */
> -}
> -
> -/**
> - * Removes reader/writer reference for LDLM lock \a lock.
> - * Locks LDLM lock first.
> - * If the lock is determined to be client lock on a client and r/w refcount
> - * drops to zero and the lock is not blocked, the lock is added to LRU lock
> - * on the namespace.
> - * For blocked LDLM locks if r/w count drops to zero, blocking_ast is called.
> - */
> -void ldlm_lock_decref_internal(struct ldlm_lock *lock, enum ldlm_mode mode)
> -{
> -	struct ldlm_namespace *ns;
> -
> -	lock_res_and_lock(lock);
> -
> -	ns = ldlm_lock_to_ns(lock);
> -
> -	ldlm_lock_decref_internal_nolock(lock, mode);
> -
> -	if ((ldlm_is_local(lock) || lock->l_req_mode == LCK_GROUP) &&
> -	    !lock->l_readers && !lock->l_writers) {
> -		/* If this is a local lock on a server namespace and this was
> -		 * the last reference, cancel the lock.
> -		 *
> -		 * Group locks are special:
> -		 * They must not go in LRU, but they are not called back
> -		 * like non-group locks, instead they are manually released.
> -		 * They have an l_writers reference which they keep until
> -		 * they are manually released, so we remove them when they have
> -		 * no more reader or writer references. - LU-6368
> -		 */
> -		ldlm_set_cbpending(lock);
> -	}
> -
> -	if (!lock->l_readers && !lock->l_writers && ldlm_is_cbpending(lock)) {
> -		/* If we received a blocked AST and this was the last reference,
> -		 * run the callback.
> -		 */
> -		LDLM_DEBUG(lock, "final decref done on cbpending lock");
> -
> -		LDLM_LOCK_GET(lock); /* dropped by bl thread */
> -		ldlm_lock_remove_from_lru(lock);
> -		unlock_res_and_lock(lock);
> -
> -		if (ldlm_is_fail_loc(lock))
> -			OBD_RACE(OBD_FAIL_LDLM_CP_BL_RACE);
> -
> -		if (ldlm_is_atomic_cb(lock) ||
> -		    ldlm_bl_to_thread_lock(ns, NULL, lock) != 0)
> -			ldlm_handle_bl_callback(ns, NULL, lock);
> -	} else if (!lock->l_readers && !lock->l_writers &&
> -		   !ldlm_is_no_lru(lock) && !ldlm_is_bl_ast(lock)) {
> -		LDLM_DEBUG(lock, "add lock into lru list");
> -
> -		/* If this is a client-side namespace and this was the last
> -		 * reference, put it on the LRU.
> -		 */
> -		ldlm_lock_add_to_lru(lock);
> -		unlock_res_and_lock(lock);
> -
> -		if (ldlm_is_fail_loc(lock))
> -			OBD_RACE(OBD_FAIL_LDLM_CP_BL_RACE);
> -
> -		/* Call ldlm_cancel_lru() only if EARLY_CANCEL and LRU RESIZE
> -		 * are not supported by the server, otherwise, it is done on
> -		 * enqueue.
> -		 */
> -		if (!exp_connect_cancelset(lock->l_conn_export) &&
> -		    !ns_connect_lru_resize(ns))
> -			ldlm_cancel_lru(ns, 0, LCF_ASYNC, 0);
> -	} else {
> -		LDLM_DEBUG(lock, "do not add lock into lru list");
> -		unlock_res_and_lock(lock);
> -	}
> -}
> -
> -/**
> - * Decrease reader/writer refcount for LDLM lock with handle \a lockh
> - */
> -void ldlm_lock_decref(const struct lustre_handle *lockh, enum ldlm_mode mode)
> -{
> -	struct ldlm_lock *lock = __ldlm_handle2lock(lockh, 0);
> -
> -	LASSERTF(lock, "Non-existing lock: %#llx\n", lockh->cookie);
> -	ldlm_lock_decref_internal(lock, mode);
> -	LDLM_LOCK_PUT(lock);
> -}
> -EXPORT_SYMBOL(ldlm_lock_decref);
> -
> -/**
> - * Decrease reader/writer refcount for LDLM lock with handle
> - * \a lockh and mark it for subsequent cancellation once r/w refcount
> - * drops to zero instead of putting into LRU.
> - */
> -void ldlm_lock_decref_and_cancel(const struct lustre_handle *lockh,
> -				 enum ldlm_mode mode)
> -{
> -	struct ldlm_lock *lock = __ldlm_handle2lock(lockh, 0);
> -
> -	LASSERT(lock);
> -
> -	LDLM_DEBUG(lock, "ldlm_lock_decref(%s)", ldlm_lockname[mode]);
> -	lock_res_and_lock(lock);
> -	ldlm_set_cbpending(lock);
> -	unlock_res_and_lock(lock);
> -	ldlm_lock_decref_internal(lock, mode);
> -	LDLM_LOCK_PUT(lock);
> -}
> -EXPORT_SYMBOL(ldlm_lock_decref_and_cancel);
> -
> -struct sl_insert_point {
> -	struct list_head *res_link;
> -	struct list_head *mode_link;
> -	struct list_head *policy_link;
> -};
> -
> -/**
> - * Finds a position to insert the new lock into granted lock list.
> - *
> - * Used for locks eligible for skiplist optimization.
> - *
> - * Parameters:
> - *      queue [input]:  the granted list where search acts on;
> - *      req [input]:    the lock whose position to be located;
> - *      prev [output]:  positions within 3 lists to insert @req to
> - * Return Value:
> - *      filled @prev
> - * NOTE: called by
> - *  - ldlm_grant_lock_with_skiplist
> - */
> -static void search_granted_lock(struct list_head *queue,
> -				struct ldlm_lock *req,
> -				struct sl_insert_point *prev)
> -{
> -	struct ldlm_lock *lock, *mode_end, *policy_end;
> -
> -	list_for_each_entry(lock, queue, l_res_link) {
> -
> -		mode_end = list_prev_entry(lock, l_sl_mode);
> -
> -		if (lock->l_req_mode != req->l_req_mode) {
> -			/* jump to last lock of mode group */
> -			lock = mode_end;
> -			continue;
> -		}
> -
> -		/* suitable mode group is found */
> -		if (lock->l_resource->lr_type == LDLM_PLAIN) {
> -			/* insert point is last lock of the mode group */
> -			prev->res_link = &mode_end->l_res_link;
> -			prev->mode_link = &mode_end->l_sl_mode;
> -			prev->policy_link = &req->l_sl_policy;
> -			return;
> -		}
> -
> -		if (lock->l_resource->lr_type == LDLM_IBITS) {
> -			for (;;) {
> -				policy_end =
> -					list_prev_entry(lock, l_sl_policy);
> -
> -				if (lock->l_policy_data.l_inodebits.bits ==
> -				    req->l_policy_data.l_inodebits.bits) {
> -					/* insert point is last lock of
> -					 * the policy group
> -					 */
> -					prev->res_link =
> -						&policy_end->l_res_link;
> -					prev->mode_link =
> -						&policy_end->l_sl_mode;
> -					prev->policy_link =
> -						&policy_end->l_sl_policy;
> -					return;
> -				}
> -
> -				if (policy_end == mode_end)
> -					/* done with mode group */
> -					break;
> -
> -				/* go to next policy group within mode group */
> -				lock = list_next_entry(policy_end, l_res_link);
> -			}  /* loop over policy groups within the mode group */
> -
> -			/* insert point is last lock of the mode group,
> -			 * new policy group is started
> -			 */
> -			prev->res_link = &mode_end->l_res_link;
> -			prev->mode_link = &mode_end->l_sl_mode;
> -			prev->policy_link = &req->l_sl_policy;
> -			return;
> -		}
> -
> -		LDLM_ERROR(lock, "is not LDLM_PLAIN or LDLM_IBITS lock");
> -		LBUG();
> -	}
> -
> -	/* insert point is last lock on the queue,
> -	 * new mode group and new policy group are started
> -	 */
> -	prev->res_link = queue->prev;
> -	prev->mode_link = &req->l_sl_mode;
> -	prev->policy_link = &req->l_sl_policy;
> -}
> -
> -/**
> - * Add a lock into resource granted list after a position described by
> - * \a prev.
> - */
> -static void ldlm_granted_list_add_lock(struct ldlm_lock *lock,
> -				       struct sl_insert_point *prev)
> -{
> -	struct ldlm_resource *res = lock->l_resource;
> -
> -	check_res_locked(res);
> -
> -	ldlm_resource_dump(D_INFO, res);
> -	LDLM_DEBUG(lock, "About to add lock:");
> -
> -	if (ldlm_is_destroyed(lock)) {
> -		CDEBUG(D_OTHER, "Lock destroyed, not adding to resource\n");
> -		return;
> -	}
> -
> -	LASSERT(list_empty(&lock->l_res_link));
> -	LASSERT(list_empty(&lock->l_sl_mode));
> -	LASSERT(list_empty(&lock->l_sl_policy));
> -
> -	/*
> -	 * lock->link == prev->link means lock is first starting the group.
> -	 * Don't re-add to itself to suppress kernel warnings.
> -	 */
> -	if (&lock->l_res_link != prev->res_link)
> -		list_add(&lock->l_res_link, prev->res_link);
> -	if (&lock->l_sl_mode != prev->mode_link)
> -		list_add(&lock->l_sl_mode, prev->mode_link);
> -	if (&lock->l_sl_policy != prev->policy_link)
> -		list_add(&lock->l_sl_policy, prev->policy_link);
> -}
> -
> -/**
> - * Add a lock to granted list on a resource maintaining skiplist
> - * correctness.
> - */
> -static void ldlm_grant_lock_with_skiplist(struct ldlm_lock *lock)
> -{
> -	struct sl_insert_point prev;
> -
> -	LASSERT(lock->l_req_mode == lock->l_granted_mode);
> -
> -	search_granted_lock(&lock->l_resource->lr_granted, lock, &prev);
> -	ldlm_granted_list_add_lock(lock, &prev);
> -}
> -
> -/**
> - * Perform lock granting bookkeeping.
> - *
> - * Includes putting the lock into granted list and updating lock mode.
> - * NOTE: called by
> - *  - ldlm_lock_enqueue
> - *  - ldlm_reprocess_queue
> - *  - ldlm_lock_convert
> - *
> - * must be called with lr_lock held
> - */
> -void ldlm_grant_lock(struct ldlm_lock *lock, struct list_head *work_list)
> -{
> -	struct ldlm_resource *res = lock->l_resource;
> -
> -	check_res_locked(res);
> -
> -	lock->l_granted_mode = lock->l_req_mode;
> -
> -	if (work_list && lock->l_completion_ast)
> -		ldlm_add_ast_work_item(lock, NULL, work_list);
> -
> -	if (res->lr_type == LDLM_PLAIN || res->lr_type == LDLM_IBITS) {
> -		ldlm_grant_lock_with_skiplist(lock);
> -	} else if (res->lr_type == LDLM_EXTENT) {
> -		ldlm_extent_add_lock(res, lock);
> -	} else if (res->lr_type == LDLM_FLOCK) {
> -		/*
> -		 * We should not add locks to granted list in
> -		 * the following cases:
> -		 * - this is an UNLOCK but not a real lock;
> -		 * - this is a TEST lock;
> -		 * - this is a F_CANCELLK lock (async flock has req_mode == 0)
> -		 * - this is a deadlock (flock cannot be granted)
> -		 */
> -		if (!lock->l_req_mode || lock->l_req_mode == LCK_NL ||
> -		    ldlm_is_test_lock(lock) || ldlm_is_flock_deadlock(lock))
> -			return;
> -		ldlm_resource_add_lock(res, &res->lr_granted, lock);
> -	} else {
> -		LBUG();
> -	}
> -
> -	ldlm_pool_add(&ldlm_res_to_ns(res)->ns_pool, lock);
> -}
> -
> -/**
> - * Describe the overlap between two locks.  itree_overlap_cb data.
> - */
> -struct lock_match_data {
> -	struct ldlm_lock	*lmd_old;
> -	struct ldlm_lock	*lmd_lock;
> -	enum ldlm_mode		*lmd_mode;
> -	union ldlm_policy_data	*lmd_policy;
> -	__u64			 lmd_flags;
> -	int			 lmd_unref;
> -};
> -
> -/**
> - * Check if the given @lock meets the criteria for a match.
> - * A reference on the lock is taken if matched.
> - *
> - * \param lock	test-against this lock
> - * \param data	parameters
> - */
> -static bool lock_matches(struct ldlm_lock *lock, void *vdata)
> -{
> -	struct lock_match_data *data = vdata;
> -	union ldlm_policy_data *lpol = &lock->l_policy_data;
> -	enum ldlm_mode match;
> -
> -	if (lock == data->lmd_old)
> -		return true;
> -
> -	/*
> -	 * Check if this lock can be matched.
> -	 * Used by LU-2919(exclusive open) for open lease lock
> -	 */
> -	if (ldlm_is_excl(lock))
> -		return false;
> -
> -	/*
> -	 * llite sometimes wants to match locks that will be
> -	 * canceled when their users drop, but we allow it to match
> -	 * if it passes in CBPENDING and the lock still has users.
> -	 * this is generally only going to be used by children
> -	 * whose parents already hold a lock so forward progress
> -	 * can still happen.
> -	 */
> -	if (ldlm_is_cbpending(lock) &&
> -	    !(data->lmd_flags & LDLM_FL_CBPENDING))
> -		return false;
> -
> -	if (!data->lmd_unref && ldlm_is_cbpending(lock) &&
> -	    !lock->l_readers && !lock->l_writers)
> -		return false;
> -
> -	if (!(lock->l_req_mode & *data->lmd_mode))
> -		return false;
> -	match = lock->l_req_mode;
> -
> -	switch (lock->l_resource->lr_type) {
> -	case LDLM_EXTENT:
> -		if (lpol->l_extent.start > data->lmd_policy->l_extent.start ||
> -		    lpol->l_extent.end < data->lmd_policy->l_extent.end)
> -			return false;
> -
> -		if (unlikely(match == LCK_GROUP) &&
> -		    data->lmd_policy->l_extent.gid != LDLM_GID_ANY &&
> -		    lpol->l_extent.gid != data->lmd_policy->l_extent.gid)
> -			return false;
> -		break;
> -	case LDLM_IBITS:
> -		/*
> -		 * We match if we have existing lock with same or wider set
> -		 * of bits.
> -		 */
> -		if ((lpol->l_inodebits.bits &
> -		     data->lmd_policy->l_inodebits.bits) !=
> -		    data->lmd_policy->l_inodebits.bits)
> -			return false;
> -		break;
> -	default:
> -		break;
> -	}
> -	/*
> -	 * We match if we have existing lock with same or wider set
> -	 * of bits.
> -	 */
> -	if (!data->lmd_unref && LDLM_HAVE_MASK(lock, GONE))
> -		return false;
> -
> -	if (!equi(data->lmd_flags & LDLM_FL_LOCAL_ONLY, ldlm_is_local(lock)))
> -		return false;
> -
> -	if (data->lmd_flags & LDLM_FL_TEST_LOCK) {
> -		LDLM_LOCK_GET(lock);
> -		ldlm_lock_touch_in_lru(lock);
> -	} else {
> -		ldlm_lock_addref_internal_nolock(lock, match);
> -	}
> -
> -	*data->lmd_mode = match;
> -	data->lmd_lock = lock;
> -
> -	return true;
> -}
> -
> -/**
> - * Search for a lock with given parameters in interval trees.
> - *
> - * \param res	search for a lock in this resource
> - * \param data	parameters
> - *
> - * \retval	a referenced lock or NULL.
> - */
> -static struct ldlm_lock *search_itree(struct ldlm_resource *res,
> -				      struct lock_match_data *data)
> -{
> -	int idx;
> -
> -	for (idx = 0; idx < LCK_MODE_NUM; idx++) {
> -		struct ldlm_interval_tree *tree = &res->lr_itree[idx];
> -
> -		if (RB_EMPTY_ROOT(&tree->lit_root.rb_root))
> -			continue;
> -
> -		if (!(tree->lit_mode & *data->lmd_mode))
> -			continue;
> -
> -		ldlm_extent_search(&tree->lit_root,
> -				   data->lmd_policy->l_extent.start,
> -				   data->lmd_policy->l_extent.end,
> -				   lock_matches, data);
> -	}
> -	return data->lmd_lock;
> -}
> -
> -/**
> - * Search for a lock with given properties in a queue.
> - *
> - * \param queue	search for a lock in this queue
> - * \param data	parameters
> - *
> - * \retval	a referenced lock or NULL.
> - */
> -static struct ldlm_lock *search_queue(struct list_head *queue,
> -				      struct lock_match_data *data)
> -{
> -	struct ldlm_lock *lock;
> -
> -	list_for_each_entry(lock, queue, l_res_link)
> -		if (lock_matches(lock, data))
> -			return data->lmd_lock;
> -	return NULL;
> -}
> -
> -void ldlm_lock_fail_match_locked(struct ldlm_lock *lock)
> -{
> -	if ((lock->l_flags & LDLM_FL_FAIL_NOTIFIED) == 0) {
> -		lock->l_flags |= LDLM_FL_FAIL_NOTIFIED;
> -		wake_up_all(&lock->l_waitq);
> -	}
> -}
> -
> -/**
> - * Mark lock as "matchable" by OST.
> - *
> - * Used to prevent certain races in LOV/OSC where the lock is granted, but LVB
> - * is not yet valid.
> - * Assumes LDLM lock is already locked.
> - */
> -void ldlm_lock_allow_match_locked(struct ldlm_lock *lock)
> -{
> -	ldlm_set_lvb_ready(lock);
> -	wake_up_all(&lock->l_waitq);
> -}
> -EXPORT_SYMBOL(ldlm_lock_allow_match_locked);
> -
> -/**
> - * Mark lock as "matchable" by OST.
> - * Locks the lock and then \see ldlm_lock_allow_match_locked
> - */
> -void ldlm_lock_allow_match(struct ldlm_lock *lock)
> -{
> -	lock_res_and_lock(lock);
> -	ldlm_lock_allow_match_locked(lock);
> -	unlock_res_and_lock(lock);
> -}
> -EXPORT_SYMBOL(ldlm_lock_allow_match);
> -
> -/**
> - * Attempt to find a lock with specified properties.
> - *
> - * Typically returns a reference to matched lock unless LDLM_FL_TEST_LOCK is
> - * set in \a flags
> - *
> - * Can be called in two ways:
> - *
> - * If 'ns' is NULL, then lockh describes an existing lock that we want to look
> - * for a duplicate of.
> - *
> - * Otherwise, all of the fields must be filled in, to match against.
> - *
> - * If 'flags' contains LDLM_FL_LOCAL_ONLY, then only match local locks on the
> - *     server (ie, connh is NULL)
> - * If 'flags' contains LDLM_FL_BLOCK_GRANTED, then only locks on the granted
> - *     list will be considered
> - * If 'flags' contains LDLM_FL_CBPENDING, then locks that have been marked
> - *     to be canceled can still be matched as long as they still have reader
> - *     or writer referneces
> - * If 'flags' contains LDLM_FL_TEST_LOCK, then don't actually reference a lock,
> - *     just tell us if we would have matched.
> - *
> - * \retval 1 if it finds an already-existing lock that is compatible; in this
> - * case, lockh is filled in with a addref()ed lock
> - *
> - * We also check security context, and if that fails we simply return 0 (to
> - * keep caller code unchanged), the context failure will be discovered by
> - * caller sometime later.
> - */
> -enum ldlm_mode ldlm_lock_match(struct ldlm_namespace *ns, __u64 flags,
> -			       const struct ldlm_res_id *res_id,
> -			       enum ldlm_type type,
> -			       union ldlm_policy_data *policy,
> -			       enum ldlm_mode mode,
> -			       struct lustre_handle *lockh, int unref)
> -{
> -	struct lock_match_data data = {
> -		.lmd_old	= NULL,
> -		.lmd_lock	= NULL,
> -		.lmd_mode	= &mode,
> -		.lmd_policy	= policy,
> -		.lmd_flags	= flags,
> -		.lmd_unref	= unref,
> -	};
> -	struct ldlm_resource *res;
> -	struct ldlm_lock *lock;
> -	int rc = 0;
> -
> -	if (!ns) {
> -		data.lmd_old = ldlm_handle2lock(lockh);
> -		LASSERT(data.lmd_old);
> -
> -		ns = ldlm_lock_to_ns(data.lmd_old);
> -		res_id = &data.lmd_old->l_resource->lr_name;
> -		type = data.lmd_old->l_resource->lr_type;
> -		*data.lmd_mode = data.lmd_old->l_req_mode;
> -	}
> -
> -	res = ldlm_resource_get(ns, NULL, res_id, type, 0);
> -	if (IS_ERR(res)) {
> -		LASSERT(!data.lmd_old);
> -		return 0;
> -	}
> -
> -	LDLM_RESOURCE_ADDREF(res);
> -	lock_res(res);
> -
> -	if (res->lr_type == LDLM_EXTENT)
> -		lock = search_itree(res, &data);
> -	else
> -		lock = search_queue(&res->lr_granted, &data);
> -	if (lock) {
> -		rc = 1;
> -		goto out;
> -	}
> -	if (flags & LDLM_FL_BLOCK_GRANTED) {
> -		rc = 0;
> -		goto out;
> -	}
> -	lock = search_queue(&res->lr_waiting, &data);
> -	if (lock) {
> -		rc = 1;
> -		goto out;
> -	}
> -out:
> -	unlock_res(res);
> -	LDLM_RESOURCE_DELREF(res);
> -	ldlm_resource_putref(res);
> -
> -	if (lock) {
> -		ldlm_lock2handle(lock, lockh);
> -		if ((flags & LDLM_FL_LVB_READY) && !ldlm_is_lvb_ready(lock)) {
> -			__u64 wait_flags = LDLM_FL_LVB_READY |
> -				LDLM_FL_DESTROYED | LDLM_FL_FAIL_NOTIFIED;
> -
> -			if (lock->l_completion_ast) {
> -				int err = lock->l_completion_ast(lock,
> -							  LDLM_FL_WAIT_NOREPROC,
> -								 NULL);
> -				if (err) {
> -					if (flags & LDLM_FL_TEST_LOCK)
> -						LDLM_LOCK_RELEASE(lock);
> -					else
> -						ldlm_lock_decref_internal(lock,
> -									  mode);
> -					rc = 0;
> -					goto out2;
> -				}
> -			}
> -
> -			/* XXX FIXME see comment on CAN_MATCH in lustre_dlm.h */
> -			wait_event_idle_timeout(lock->l_waitq,
> -						lock->l_flags & wait_flags,
> -						obd_timeout * HZ);
> -			if (!ldlm_is_lvb_ready(lock)) {
> -				if (flags & LDLM_FL_TEST_LOCK)
> -					LDLM_LOCK_RELEASE(lock);
> -				else
> -					ldlm_lock_decref_internal(lock, mode);
> -				rc = 0;
> -			}
> -		}
> -	}
> - out2:
> -	if (rc) {
> -		LDLM_DEBUG(lock, "matched (%llu %llu)",
> -			   (type == LDLM_PLAIN || type == LDLM_IBITS) ?
> -				res_id->name[2] : policy->l_extent.start,
> -			   (type == LDLM_PLAIN || type == LDLM_IBITS) ?
> -				res_id->name[3] : policy->l_extent.end);
> -
> -		/* check user's security context */
> -		if (lock->l_conn_export &&
> -		    sptlrpc_import_check_ctx(
> -				class_exp2cliimp(lock->l_conn_export))) {
> -			if (!(flags & LDLM_FL_TEST_LOCK))
> -				ldlm_lock_decref_internal(lock, mode);
> -			rc = 0;
> -		}
> -
> -		if (flags & LDLM_FL_TEST_LOCK)
> -			LDLM_LOCK_RELEASE(lock);
> -
> -	} else if (!(flags & LDLM_FL_TEST_LOCK)) {/*less verbose for test-only*/
> -		LDLM_DEBUG_NOLOCK("not matched ns %p type %u mode %u res %llu/%llu (%llu %llu)",
> -				  ns, type, mode, res_id->name[0],
> -				  res_id->name[1],
> -				  (type == LDLM_PLAIN || type == LDLM_IBITS) ?
> -					res_id->name[2] : policy->l_extent.start,
> -				  (type == LDLM_PLAIN || type == LDLM_IBITS) ?
> -					res_id->name[3] : policy->l_extent.end);
> -	}
> -	if (data.lmd_old)
> -		LDLM_LOCK_PUT(data.lmd_old);
> -
> -	return rc ? mode : 0;
> -}
> -EXPORT_SYMBOL(ldlm_lock_match);
> -
> -enum ldlm_mode ldlm_revalidate_lock_handle(const struct lustre_handle *lockh,
> -					   __u64 *bits)
> -{
> -	struct ldlm_lock *lock;
> -	enum ldlm_mode mode = 0;
> -
> -	lock = ldlm_handle2lock(lockh);
> -	if (lock) {
> -		lock_res_and_lock(lock);
> -		if (LDLM_HAVE_MASK(lock, GONE))
> -			goto out;
> -
> -		if (ldlm_is_cbpending(lock) &&
> -		    lock->l_readers == 0 && lock->l_writers == 0)
> -			goto out;
> -
> -		if (bits)
> -			*bits = lock->l_policy_data.l_inodebits.bits;
> -		mode = lock->l_granted_mode;
> -		ldlm_lock_addref_internal_nolock(lock, mode);
> -	}
> -
> -out:
> -	if (lock) {
> -		unlock_res_and_lock(lock);
> -		LDLM_LOCK_PUT(lock);
> -	}
> -	return mode;
> -}
> -EXPORT_SYMBOL(ldlm_revalidate_lock_handle);
> -
> -/** The caller must guarantee that the buffer is large enough. */
> -int ldlm_fill_lvb(struct ldlm_lock *lock, struct req_capsule *pill,
> -		  enum req_location loc, void *data, int size)
> -{
> -	void *lvb;
> -
> -	LASSERT(data);
> -	LASSERT(size >= 0);
> -
> -	switch (lock->l_lvb_type) {
> -	case LVB_T_OST:
> -		if (size == sizeof(struct ost_lvb)) {
> -			if (loc == RCL_CLIENT)
> -				lvb = req_capsule_client_swab_get(pill,
> -								  &RMF_DLM_LVB,
> -							lustre_swab_ost_lvb);
> -			else
> -				lvb = req_capsule_server_swab_get(pill,
> -								  &RMF_DLM_LVB,
> -							lustre_swab_ost_lvb);
> -			if (unlikely(!lvb)) {
> -				LDLM_ERROR(lock, "no LVB");
> -				return -EPROTO;
> -			}
> -
> -			memcpy(data, lvb, size);
> -		} else if (size == sizeof(struct ost_lvb_v1)) {
> -			struct ost_lvb *olvb = data;
> -
> -			if (loc == RCL_CLIENT)
> -				lvb = req_capsule_client_swab_get(pill,
> -								  &RMF_DLM_LVB,
> -							lustre_swab_ost_lvb_v1);
> -			else
> -				lvb = req_capsule_server_sized_swab_get(pill,
> -						&RMF_DLM_LVB, size,
> -						lustre_swab_ost_lvb_v1);
> -			if (unlikely(!lvb)) {
> -				LDLM_ERROR(lock, "no LVB");
> -				return -EPROTO;
> -			}
> -
> -			memcpy(data, lvb, size);
> -			olvb->lvb_mtime_ns = 0;
> -			olvb->lvb_atime_ns = 0;
> -			olvb->lvb_ctime_ns = 0;
> -		} else {
> -			LDLM_ERROR(lock, "Replied unexpected ost LVB size %d",
> -				   size);
> -			return -EINVAL;
> -		}
> -		break;
> -	case LVB_T_LQUOTA:
> -		if (size == sizeof(struct lquota_lvb)) {
> -			if (loc == RCL_CLIENT)
> -				lvb = req_capsule_client_swab_get(pill,
> -								  &RMF_DLM_LVB,
> -							lustre_swab_lquota_lvb);
> -			else
> -				lvb = req_capsule_server_swab_get(pill,
> -								  &RMF_DLM_LVB,
> -							lustre_swab_lquota_lvb);
> -			if (unlikely(!lvb)) {
> -				LDLM_ERROR(lock, "no LVB");
> -				return -EPROTO;
> -			}
> -
> -			memcpy(data, lvb, size);
> -		} else {
> -			LDLM_ERROR(lock,
> -				   "Replied unexpected lquota LVB size %d",
> -				   size);
> -			return -EINVAL;
> -		}
> -		break;
> -	case LVB_T_LAYOUT:
> -		if (size == 0)
> -			break;
> -
> -		if (loc == RCL_CLIENT)
> -			lvb = req_capsule_client_get(pill, &RMF_DLM_LVB);
> -		else
> -			lvb = req_capsule_server_get(pill, &RMF_DLM_LVB);
> -		if (unlikely(!lvb)) {
> -			LDLM_ERROR(lock, "no LVB");
> -			return -EPROTO;
> -		}
> -
> -		memcpy(data, lvb, size);
> -		break;
> -	default:
> -		LDLM_ERROR(lock, "Unknown LVB type: %d", lock->l_lvb_type);
> -		dump_stack();
> -		return -EINVAL;
> -	}
> -
> -	return 0;
> -}
> -
> -/**
> - * Create and fill in new LDLM lock with specified properties.
> - * Returns a referenced lock
> - */
> -struct ldlm_lock *ldlm_lock_create(struct ldlm_namespace *ns,
> -				   const struct ldlm_res_id *res_id,
> -				   enum ldlm_type type,
> -				   enum ldlm_mode mode,
> -				   const struct ldlm_callback_suite *cbs,
> -				   void *data, __u32 lvb_len,
> -				   enum lvb_type lvb_type)
> -{
> -	struct ldlm_lock *lock;
> -	struct ldlm_resource *res;
> -	int rc;
> -
> -	res = ldlm_resource_get(ns, NULL, res_id, type, 1);
> -	if (IS_ERR(res))
> -		return ERR_CAST(res);
> -
> -	lock = ldlm_lock_new(res);
> -	if (!lock) {
> -		ldlm_resource_putref(res);
> -		return ERR_PTR(-ENOMEM);
> -	}
> -
> -	lock->l_req_mode = mode;
> -	lock->l_ast_data = data;
> -	lock->l_pid = current->pid;
> -	if (cbs) {
> -		lock->l_blocking_ast = cbs->lcs_blocking;
> -		lock->l_completion_ast = cbs->lcs_completion;
> -		lock->l_glimpse_ast = cbs->lcs_glimpse;
> -	}
> -
> -	if (lvb_len) {
> -		lock->l_lvb_len = lvb_len;
> -		lock->l_lvb_data = kzalloc(lvb_len, GFP_NOFS);
> -		if (!lock->l_lvb_data) {
> -			rc = -ENOMEM;
> -			goto out;
> -		}
> -	}
> -
> -	lock->l_lvb_type = lvb_type;
> -	if (OBD_FAIL_CHECK(OBD_FAIL_LDLM_NEW_LOCK)) {
> -		rc = -ENOENT;
> -		goto out;
> -	}
> -
> -	return lock;
> -
> -out:
> -	ldlm_lock_destroy(lock);
> -	LDLM_LOCK_RELEASE(lock);
> -	return ERR_PTR(rc);
> -}
> -
> -
> -
> -/**
> - * Enqueue (request) a lock.
> - * On the client this is called from ldlm_cli_enqueue_fini
> - * after we already got an initial reply from the server with some status.
> - *
> - * Does not block. As a result of enqueue the lock would be put
> - * into granted or waiting list.
> - */
> -enum ldlm_error ldlm_lock_enqueue(struct ldlm_namespace *ns,
> -				  struct ldlm_lock **lockp,
> -				  void *cookie, __u64 *flags)
> -{
> -	struct ldlm_lock *lock = *lockp;
> -	struct ldlm_resource *res = lock->l_resource;
> -
> -	lock_res_and_lock(lock);
> -	if (lock->l_req_mode == lock->l_granted_mode) {
> -		/* The server returned a blocked lock, but it was granted
> -		 * before we got a chance to actually enqueue it.  We don't
> -		 * need to do anything else.
> -		 */
> -		*flags &= ~LDLM_FL_BLOCKED_MASK;
> -		goto out;
> -	}
> -
> -	ldlm_resource_unlink_lock(lock);
> -
> -	/* Some flags from the enqueue want to make it into the AST, via the
> -	 * lock's l_flags.
> -	 */
> -	if (*flags & LDLM_FL_AST_DISCARD_DATA)
> -		ldlm_set_ast_discard_data(lock);
> -	if (*flags & LDLM_FL_TEST_LOCK)
> -		ldlm_set_test_lock(lock);
> -
> -	/*
> -	 * This distinction between local lock trees is very important; a client
> -	 * namespace only has information about locks taken by that client, and
> -	 * thus doesn't have enough information to decide for itself if it can
> -	 * be granted (below).  In this case, we do exactly what the server
> -	 * tells us to do, as dictated by the 'flags'.
> -	 */
> -	if (*flags & (LDLM_FL_BLOCK_WAIT | LDLM_FL_BLOCK_GRANTED))
> -		ldlm_resource_add_lock(res, &res->lr_waiting, lock);
> -	else
> -		ldlm_grant_lock(lock, NULL);
> -
> -out:
> -	unlock_res_and_lock(lock);
> -	return ELDLM_OK;
> -}
> -
> -/**
> - * Process a call to blocking AST callback for a lock in ast_work list
> - */
> -static int
> -ldlm_work_bl_ast_lock(struct ptlrpc_request_set *rqset, void *opaq)
> -{
> -	struct ldlm_cb_set_arg *arg = opaq;
> -	struct ldlm_lock_desc   d;
> -	int		     rc;
> -	struct ldlm_lock       *lock;
> -
> -	if (list_empty(arg->list))
> -		return -ENOENT;
> -
> -	lock = list_first_entry(arg->list, struct ldlm_lock, l_bl_ast);
> -
> -	/* nobody should touch l_bl_ast */
> -	lock_res_and_lock(lock);
> -	list_del_init(&lock->l_bl_ast);
> -
> -	LASSERT(ldlm_is_ast_sent(lock));
> -	LASSERT(lock->l_bl_ast_run == 0);
> -	LASSERT(lock->l_blocking_lock);
> -	lock->l_bl_ast_run++;
> -	unlock_res_and_lock(lock);
> -
> -	ldlm_lock2desc(lock->l_blocking_lock, &d);
> -
> -	rc = lock->l_blocking_ast(lock, &d, (void *)arg, LDLM_CB_BLOCKING);
> -	LDLM_LOCK_RELEASE(lock->l_blocking_lock);
> -	lock->l_blocking_lock = NULL;
> -	LDLM_LOCK_RELEASE(lock);
> -
> -	return rc;
> -}
> -
> -/**
> - * Process a call to completion AST callback for a lock in ast_work list
> - */
> -static int
> -ldlm_work_cp_ast_lock(struct ptlrpc_request_set *rqset, void *opaq)
> -{
> -	struct ldlm_cb_set_arg  *arg = opaq;
> -	int		      rc = 0;
> -	struct ldlm_lock	*lock;
> -	ldlm_completion_callback completion_callback;
> -
> -	if (list_empty(arg->list))
> -		return -ENOENT;
> -
> -	lock = list_first_entry(arg->list, struct ldlm_lock, l_cp_ast);
> -
> -	/* It's possible to receive a completion AST before we've set
> -	 * the l_completion_ast pointer: either because the AST arrived
> -	 * before the reply, or simply because there's a small race
> -	 * window between receiving the reply and finishing the local
> -	 * enqueue. (bug 842)
> -	 *
> -	 * This can't happen with the blocking_ast, however, because we
> -	 * will never call the local blocking_ast until we drop our
> -	 * reader/writer reference, which we won't do until we get the
> -	 * reply and finish enqueueing.
> -	 */
> -
> -	/* nobody should touch l_cp_ast */
> -	lock_res_and_lock(lock);
> -	list_del_init(&lock->l_cp_ast);
> -	LASSERT(ldlm_is_cp_reqd(lock));
> -	/* save l_completion_ast since it can be changed by
> -	 * mds_intent_policy(), see bug 14225
> -	 */
> -	completion_callback = lock->l_completion_ast;
> -	ldlm_clear_cp_reqd(lock);
> -	unlock_res_and_lock(lock);
> -
> -	if (completion_callback)
> -		rc = completion_callback(lock, 0, (void *)arg);
> -	LDLM_LOCK_RELEASE(lock);
> -
> -	return rc;
> -}
> -
> -/**
> - * Process a call to revocation AST callback for a lock in ast_work list
> - */
> -static int
> -ldlm_work_revoke_ast_lock(struct ptlrpc_request_set *rqset, void *opaq)
> -{
> -	struct ldlm_cb_set_arg *arg = opaq;
> -	struct ldlm_lock_desc   desc;
> -	int		     rc;
> -	struct ldlm_lock       *lock;
> -
> -	if (list_empty(arg->list))
> -		return -ENOENT;
> -
> -	lock = list_first_entry(arg->list, struct ldlm_lock, l_rk_ast);
> -	list_del_init(&lock->l_rk_ast);
> -
> -	/* the desc just pretend to exclusive */
> -	ldlm_lock2desc(lock, &desc);
> -	desc.l_req_mode = LCK_EX;
> -	desc.l_granted_mode = 0;
> -
> -	rc = lock->l_blocking_ast(lock, &desc, (void *)arg, LDLM_CB_BLOCKING);
> -	LDLM_LOCK_RELEASE(lock);
> -
> -	return rc;
> -}
> -
> -/**
> - * Process a call to glimpse AST callback for a lock in ast_work list
> - */
> -static int ldlm_work_gl_ast_lock(struct ptlrpc_request_set *rqset, void *opaq)
> -{
> -	struct ldlm_cb_set_arg		*arg = opaq;
> -	struct ldlm_glimpse_work	*gl_work;
> -	struct ldlm_lock		*lock;
> -	int				 rc = 0;
> -
> -	if (list_empty(arg->list))
> -		return -ENOENT;
> -
> -	gl_work = list_first_entry(arg->list, struct ldlm_glimpse_work,
> -			     gl_list);
> -	list_del_init(&gl_work->gl_list);
> -
> -	lock = gl_work->gl_lock;
> -
> -	/* transfer the glimpse descriptor to ldlm_cb_set_arg */
> -	arg->gl_desc = gl_work->gl_desc;
> -
> -	/* invoke the actual glimpse callback */
> -	if (lock->l_glimpse_ast(lock, (void *)arg) == 0)
> -		rc = 1;
> -
> -	LDLM_LOCK_RELEASE(lock);
> -
> -	if ((gl_work->gl_flags & LDLM_GL_WORK_NOFREE) == 0)
> -		kfree(gl_work);
> -
> -	return rc;
> -}
> -
> -/**
> - * Process list of locks in need of ASTs being sent.
> - *
> - * Used on server to send multiple ASTs together instead of sending one by
> - * one.
> - */
> -int ldlm_run_ast_work(struct ldlm_namespace *ns, struct list_head *rpc_list,
> -		      enum ldlm_desc_ast_t ast_type)
> -{
> -	struct ldlm_cb_set_arg *arg;
> -	set_producer_func       work_ast_lock;
> -	int		     rc;
> -
> -	if (list_empty(rpc_list))
> -		return 0;
> -
> -	arg = kzalloc(sizeof(*arg), GFP_NOFS);
> -	if (!arg)
> -		return -ENOMEM;
> -
> -	atomic_set(&arg->restart, 0);
> -	arg->list = rpc_list;
> -
> -	switch (ast_type) {
> -	case LDLM_WORK_BL_AST:
> -		arg->type = LDLM_BL_CALLBACK;
> -		work_ast_lock = ldlm_work_bl_ast_lock;
> -		break;
> -	case LDLM_WORK_CP_AST:
> -		arg->type = LDLM_CP_CALLBACK;
> -		work_ast_lock = ldlm_work_cp_ast_lock;
> -		break;
> -	case LDLM_WORK_REVOKE_AST:
> -		arg->type = LDLM_BL_CALLBACK;
> -		work_ast_lock = ldlm_work_revoke_ast_lock;
> -		break;
> -	case LDLM_WORK_GL_AST:
> -		arg->type = LDLM_GL_CALLBACK;
> -		work_ast_lock = ldlm_work_gl_ast_lock;
> -		break;
> -	default:
> -		LBUG();
> -	}
> -
> -	/* We create a ptlrpc request set with flow control extension.
> -	 * This request set will use the work_ast_lock function to produce new
> -	 * requests and will send a new request each time one completes in order
> -	 * to keep the number of requests in flight to ns_max_parallel_ast
> -	 */
> -	arg->set = ptlrpc_prep_fcset(ns->ns_max_parallel_ast ? : UINT_MAX,
> -				     work_ast_lock, arg);
> -	if (!arg->set) {
> -		rc = -ENOMEM;
> -		goto out;
> -	}
> -
> -	ptlrpc_set_wait(arg->set);
> -	ptlrpc_set_destroy(arg->set);
> -
> -	rc = atomic_read(&arg->restart) ? -ERESTART : 0;
> -	goto out;
> -out:
> -	kfree(arg);
> -	return rc;
> -}
> -
> -static bool is_bl_done(struct ldlm_lock *lock)
> -{
> -	bool bl_done = true;
> -
> -	if (!ldlm_is_bl_done(lock)) {
> -		lock_res_and_lock(lock);
> -		bl_done = ldlm_is_bl_done(lock);
> -		unlock_res_and_lock(lock);
> -	}
> -
> -	return bl_done;
> -}
> -
> -/**
> - * Helper function to call blocking AST for LDLM lock \a lock in a
> - * "cancelling" mode.
> - */
> -void ldlm_cancel_callback(struct ldlm_lock *lock)
> -{
> -	check_res_locked(lock->l_resource);
> -	if (!ldlm_is_cancel(lock)) {
> -		ldlm_set_cancel(lock);
> -		if (lock->l_blocking_ast) {
> -			unlock_res_and_lock(lock);
> -			lock->l_blocking_ast(lock, NULL, lock->l_ast_data,
> -					     LDLM_CB_CANCELING);
> -			lock_res_and_lock(lock);
> -		} else {
> -			LDLM_DEBUG(lock, "no blocking ast");
> -		}
> -		/* only canceller can set bl_done bit */
> -		ldlm_set_bl_done(lock);
> -		wake_up_all(&lock->l_waitq);
> -	} else if (!ldlm_is_bl_done(lock)) {
> -		/*
> -		 * The lock is guaranteed to have been canceled once
> -		 * returning from this function.
> -		 */
> -		unlock_res_and_lock(lock);
> -		wait_event_idle(lock->l_waitq, is_bl_done(lock));
> -		lock_res_and_lock(lock);
> -	}
> -}
> -
> -/**
> - * Remove skiplist-enabled LDLM lock \a req from granted list
> - */
> -void ldlm_unlink_lock_skiplist(struct ldlm_lock *req)
> -{
> -	if (req->l_resource->lr_type != LDLM_PLAIN &&
> -	    req->l_resource->lr_type != LDLM_IBITS)
> -		return;
> -
> -	list_del_init(&req->l_sl_policy);
> -	list_del_init(&req->l_sl_mode);
> -}
> -
> -/**
> - * Attempts to cancel LDLM lock \a lock that has no reader/writer references.
> - */
> -void ldlm_lock_cancel(struct ldlm_lock *lock)
> -{
> -	struct ldlm_resource *res;
> -	struct ldlm_namespace *ns;
> -
> -	lock_res_and_lock(lock);
> -
> -	res = lock->l_resource;
> -	ns  = ldlm_res_to_ns(res);
> -
> -	/* Please do not, no matter how tempting, remove this LBUG without
> -	 * talking to me first. -phik
> -	 */
> -	if (lock->l_readers || lock->l_writers) {
> -		LDLM_ERROR(lock, "lock still has references");
> -		LBUG();
> -	}
> -
> -	/* Releases cancel callback. */
> -	ldlm_cancel_callback(lock);
> -
> -	ldlm_resource_unlink_lock(lock);
> -	ldlm_lock_destroy_nolock(lock);
> -
> -	if (lock->l_granted_mode == lock->l_req_mode)
> -		ldlm_pool_del(&ns->ns_pool, lock);
> -
> -	/* Make sure we will not be called again for same lock what is possible
> -	 * if not to zero out lock->l_granted_mode
> -	 */
> -	lock->l_granted_mode = LCK_MINMODE;
> -	unlock_res_and_lock(lock);
> -}
> -EXPORT_SYMBOL(ldlm_lock_cancel);
> -
> -/**
> - * Set opaque data into the lock that only makes sense to upper layer.
> - */
> -int ldlm_lock_set_data(const struct lustre_handle *lockh, void *data)
> -{
> -	struct ldlm_lock *lock = ldlm_handle2lock(lockh);
> -	int rc = -EINVAL;
> -
> -	if (lock) {
> -		if (!lock->l_ast_data)
> -			lock->l_ast_data = data;
> -		if (lock->l_ast_data == data)
> -			rc = 0;
> -		LDLM_LOCK_PUT(lock);
> -	}
> -	return rc;
> -}
> -EXPORT_SYMBOL(ldlm_lock_set_data);
> -
> -struct export_cl_data {
> -	struct obd_export	*ecl_exp;
> -	int			ecl_loop;
> -};
> -
> -/**
> - * Print lock with lock handle \a lockh description into debug log.
> - *
> - * Used when printing all locks on a resource for debug purposes.
> - */
> -void ldlm_lock_dump_handle(int level, const struct lustre_handle *lockh)
> -{
> -	struct ldlm_lock *lock;
> -
> -	if (!((libcfs_debug | D_ERROR) & level))
> -		return;
> -
> -	lock = ldlm_handle2lock(lockh);
> -	if (!lock)
> -		return;
> -
> -	LDLM_DEBUG_LIMIT(level, lock, "###");
> -
> -	LDLM_LOCK_PUT(lock);
> -}
> -EXPORT_SYMBOL(ldlm_lock_dump_handle);
> -
> -/**
> - * Print lock information with custom message into debug log.
> - * Helper function.
> - */
> -void _ldlm_lock_debug(struct ldlm_lock *lock,
> -		      struct libcfs_debug_msg_data *msgdata,
> -		      const char *fmt, ...)
> -{
> -	va_list args;
> -	struct obd_export *exp = lock->l_export;
> -	struct ldlm_resource *resource = lock->l_resource;
> -	char *nid = "local";
> -
> -	va_start(args, fmt);
> -
> -	if (exp && exp->exp_connection) {
> -		nid = libcfs_nid2str(exp->exp_connection->c_peer.nid);
> -	} else if (exp && exp->exp_obd) {
> -		struct obd_import *imp = exp->exp_obd->u.cli.cl_import;
> -
> -		nid = libcfs_nid2str(imp->imp_connection->c_peer.nid);
> -	}
> -
> -	if (!resource) {
> -		libcfs_debug_vmsg2(msgdata, fmt, args,
> -				   " ns: \?\? lock: %p/%#llx lrc: %d/%d,%d mode: %s/%s res: \?\? rrc=\?\? type: \?\?\? flags: %#llx nid: %s remote: %#llx expref: %d pid: %u timeout: %lu lvb_type: %d\n",
> -				   lock,
> -				   lock->l_handle.h_cookie,
> -				   atomic_read(&lock->l_refc),
> -				   lock->l_readers, lock->l_writers,
> -				   ldlm_lockname[lock->l_granted_mode],
> -				   ldlm_lockname[lock->l_req_mode],
> -				   lock->l_flags, nid,
> -				   lock->l_remote_handle.cookie,
> -				   exp ? atomic_read(&exp->exp_refcount) : -99,
> -				   lock->l_pid, lock->l_callback_timeout,
> -				   lock->l_lvb_type);
> -		va_end(args);
> -		return;
> -	}
> -
> -	switch (resource->lr_type) {
> -	case LDLM_EXTENT:
> -		libcfs_debug_vmsg2(msgdata, fmt, args,
> -				   " ns: %s lock: %p/%#llx lrc: %d/%d,%d mode: %s/%s res: " DLDLMRES " rrc: %d type: %s [%llu->%llu] (req %llu->%llu) flags: %#llx nid: %s remote: %#llx expref: %d pid: %u timeout: %lu lvb_type: %d\n",
> -				   ldlm_lock_to_ns_name(lock), lock,
> -				   lock->l_handle.h_cookie,
> -				   atomic_read(&lock->l_refc),
> -				   lock->l_readers, lock->l_writers,
> -				   ldlm_lockname[lock->l_granted_mode],
> -				   ldlm_lockname[lock->l_req_mode],
> -				   PLDLMRES(resource),
> -				   atomic_read(&resource->lr_refcount),
> -				   ldlm_typename[resource->lr_type],
> -				   lock->l_policy_data.l_extent.start,
> -				   lock->l_policy_data.l_extent.end,
> -				   lock->l_req_extent.start,
> -				   lock->l_req_extent.end,
> -				   lock->l_flags, nid,
> -				   lock->l_remote_handle.cookie,
> -				   exp ? atomic_read(&exp->exp_refcount) : -99,
> -				   lock->l_pid, lock->l_callback_timeout,
> -				   lock->l_lvb_type);
> -		break;
> -
> -	case LDLM_FLOCK:
> -		libcfs_debug_vmsg2(msgdata, fmt, args,
> -				   " ns: %s lock: %p/%#llx lrc: %d/%d,%d mode: %s/%s res: " DLDLMRES " rrc: %d type: %s pid: %d [%llu->%llu] flags: %#llx nid: %s remote: %#llx expref: %d pid: %u timeout: %lu\n",
> -				   ldlm_lock_to_ns_name(lock), lock,
> -				   lock->l_handle.h_cookie,
> -				   atomic_read(&lock->l_refc),
> -				   lock->l_readers, lock->l_writers,
> -				   ldlm_lockname[lock->l_granted_mode],
> -				   ldlm_lockname[lock->l_req_mode],
> -				   PLDLMRES(resource),
> -				   atomic_read(&resource->lr_refcount),
> -				   ldlm_typename[resource->lr_type],
> -				   lock->l_policy_data.l_flock.pid,
> -				   lock->l_policy_data.l_flock.start,
> -				   lock->l_policy_data.l_flock.end,
> -				   lock->l_flags, nid,
> -				   lock->l_remote_handle.cookie,
> -				   exp ? atomic_read(&exp->exp_refcount) : -99,
> -				   lock->l_pid, lock->l_callback_timeout);
> -		break;
> -
> -	case LDLM_IBITS:
> -		libcfs_debug_vmsg2(msgdata, fmt, args,
> -				   " ns: %s lock: %p/%#llx lrc: %d/%d,%d mode: %s/%s res: " DLDLMRES " bits %#llx rrc: %d type: %s flags: %#llx nid: %s remote: %#llx expref: %d pid: %u timeout: %lu lvb_type: %d\n",
> -				   ldlm_lock_to_ns_name(lock),
> -				   lock, lock->l_handle.h_cookie,
> -				   atomic_read(&lock->l_refc),
> -				   lock->l_readers, lock->l_writers,
> -				   ldlm_lockname[lock->l_granted_mode],
> -				   ldlm_lockname[lock->l_req_mode],
> -				   PLDLMRES(resource),
> -				   lock->l_policy_data.l_inodebits.bits,
> -				   atomic_read(&resource->lr_refcount),
> -				   ldlm_typename[resource->lr_type],
> -				   lock->l_flags, nid,
> -				   lock->l_remote_handle.cookie,
> -				   exp ? atomic_read(&exp->exp_refcount) : -99,
> -				   lock->l_pid, lock->l_callback_timeout,
> -				   lock->l_lvb_type);
> -		break;
> -
> -	default:
> -		libcfs_debug_vmsg2(msgdata, fmt, args,
> -				   " ns: %s lock: %p/%#llx lrc: %d/%d,%d mode: %s/%s res: " DLDLMRES " rrc: %d type: %s flags: %#llx nid: %s remote: %#llx expref: %d pid: %u timeout: %lu lvb_type: %d\n",
> -				   ldlm_lock_to_ns_name(lock),
> -				   lock, lock->l_handle.h_cookie,
> -				   atomic_read(&lock->l_refc),
> -				   lock->l_readers, lock->l_writers,
> -				   ldlm_lockname[lock->l_granted_mode],
> -				   ldlm_lockname[lock->l_req_mode],
> -				   PLDLMRES(resource),
> -				   atomic_read(&resource->lr_refcount),
> -				   ldlm_typename[resource->lr_type],
> -				   lock->l_flags, nid,
> -				   lock->l_remote_handle.cookie,
> -				   exp ? atomic_read(&exp->exp_refcount) : -99,
> -				   lock->l_pid, lock->l_callback_timeout,
> -				   lock->l_lvb_type);
> -		break;
> -	}
> -	va_end(args);
> -}
> -EXPORT_SYMBOL(_ldlm_lock_debug);
> diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c b/drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c
> deleted file mode 100644
> index f410ef6c02ef..000000000000
> --- a/drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c
> +++ /dev/null
> @@ -1,1154 +0,0 @@
> -// SPDX-License-Identifier: GPL-2.0
> -/*
> - * GPL HEADER START
> - *
> - * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
> - *
> - * This program is free software; you can redistribute it and/or modify
> - * it under the terms of the GNU General Public License version 2 only,
> - * as published by the Free Software Foundation.
> - *
> - * This program is distributed in the hope that it will be useful, but
> - * WITHOUT ANY WARRANTY; without even the implied warranty of
> - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> - * General Public License version 2 for more details (a copy is included
> - * in the LICENSE file that accompanied this code).
> - *
> - * You should have received a copy of the GNU General Public License
> - * version 2 along with this program; If not, see
> - * http://www.gnu.org/licenses/gpl-2.0.html
> - *
> - * GPL HEADER END
> - */
> -/*
> - * Copyright (c) 2002, 2010, Oracle and/or its affiliates. All rights reserved.
> - * Use is subject to license terms.
> - *
> - * Copyright (c) 2010, 2015, Intel Corporation.
> - */
> -/*
> - * This file is part of Lustre, http://www.lustre.org/
> - * Lustre is a trademark of Sun Microsystems, Inc.
> - *
> - * lustre/ldlm/ldlm_lockd.c
> - *
> - * Author: Peter Braam <braam at clusterfs.com>
> - * Author: Phil Schwan <phil at clusterfs.com>
> - */
> -
> -#define DEBUG_SUBSYSTEM S_LDLM
> -
> -#include <linux/kthread.h>
> -#include <linux/sched/mm.h>
> -#include <lustre_dlm.h>
> -#include <obd_class.h>
> -#include <linux/list.h>
> -#include "ldlm_internal.h"
> -
> -static int ldlm_num_threads;
> -module_param(ldlm_num_threads, int, 0444);
> -MODULE_PARM_DESC(ldlm_num_threads, "number of DLM service threads to start");
> -
> -static char *ldlm_cpts;
> -module_param(ldlm_cpts, charp, 0444);
> -MODULE_PARM_DESC(ldlm_cpts, "CPU partitions ldlm threads should run on");
> -
> -static struct mutex	ldlm_ref_mutex;
> -static int ldlm_refcount;
> -
> -static struct kobject *ldlm_kobj;
> -struct kset *ldlm_ns_kset;
> -static struct kset *ldlm_svc_kset;
> -
> -struct ldlm_cb_async_args {
> -	struct ldlm_cb_set_arg *ca_set_arg;
> -	struct ldlm_lock       *ca_lock;
> -};
> -
> -/* LDLM state */
> -
> -static struct ldlm_state *ldlm_state;
> -
> -#define ELT_STOPPED   0
> -#define ELT_READY     1
> -#define ELT_TERMINATE 2
> -
> -struct ldlm_bl_pool {
> -	spinlock_t		blp_lock;
> -
> -	/*
> -	 * blp_prio_list is used for callbacks that should be handled
> -	 * as a priority. It is used for LDLM_FL_DISCARD_DATA requests.
> -	 * see bug 13843
> -	 */
> -	struct list_head	      blp_prio_list;
> -
> -	/*
> -	 * blp_list is used for all other callbacks which are likely
> -	 * to take longer to process.
> -	 */
> -	struct list_head	      blp_list;
> -
> -	wait_queue_head_t	     blp_waitq;
> -	struct completion	blp_comp;
> -	atomic_t	    blp_num_threads;
> -	atomic_t	    blp_busy_threads;
> -	int		     blp_min_threads;
> -	int		     blp_max_threads;
> -};
> -
> -struct ldlm_bl_work_item {
> -	struct list_head	      blwi_entry;
> -	struct ldlm_namespace  *blwi_ns;
> -	struct ldlm_lock_desc   blwi_ld;
> -	struct ldlm_lock       *blwi_lock;
> -	struct list_head	      blwi_head;
> -	int		     blwi_count;
> -	struct completion	blwi_comp;
> -	enum ldlm_cancel_flags  blwi_flags;
> -	int		     blwi_mem_pressure;
> -};
> -
> -/**
> - * Callback handler for receiving incoming blocking ASTs.
> - *
> - * This can only happen on client side.
> - */
> -void ldlm_handle_bl_callback(struct ldlm_namespace *ns,
> -			     struct ldlm_lock_desc *ld, struct ldlm_lock *lock)
> -{
> -	int do_ast;
> -
> -	LDLM_DEBUG(lock, "client blocking AST callback handler");
> -
> -	lock_res_and_lock(lock);
> -	ldlm_set_cbpending(lock);
> -
> -	if (ldlm_is_cancel_on_block(lock))
> -		ldlm_set_cancel(lock);
> -
> -	do_ast = !lock->l_readers && !lock->l_writers;
> -	unlock_res_and_lock(lock);
> -
> -	if (do_ast) {
> -		CDEBUG(D_DLMTRACE,
> -		       "Lock %p already unused, calling callback (%p)\n", lock,
> -		       lock->l_blocking_ast);
> -		if (lock->l_blocking_ast)
> -			lock->l_blocking_ast(lock, ld, lock->l_ast_data,
> -					     LDLM_CB_BLOCKING);
> -	} else {
> -		CDEBUG(D_DLMTRACE,
> -		       "Lock %p is referenced, will be cancelled later\n",
> -		       lock);
> -	}
> -
> -	LDLM_DEBUG(lock, "client blocking callback handler END");
> -	LDLM_LOCK_RELEASE(lock);
> -}
> -
> -/**
> - * Callback handler for receiving incoming completion ASTs.
> - *
> - * This only can happen on client side.
> - */
> -static void ldlm_handle_cp_callback(struct ptlrpc_request *req,
> -				    struct ldlm_namespace *ns,
> -				    struct ldlm_request *dlm_req,
> -				    struct ldlm_lock *lock)
> -{
> -	int lvb_len;
> -	LIST_HEAD(ast_list);
> -	int rc = 0;
> -
> -	LDLM_DEBUG(lock, "client completion callback handler START");
> -
> -	if (OBD_FAIL_CHECK(OBD_FAIL_LDLM_CANCEL_BL_CB_RACE)) {
> -		int to = HZ;
> -
> -		while (to > 0) {
> -			set_current_state(TASK_INTERRUPTIBLE);
> -			schedule_timeout(to);
> -			if (lock->l_granted_mode == lock->l_req_mode ||
> -			    ldlm_is_destroyed(lock))
> -				break;
> -		}
> -	}
> -
> -	lvb_len = req_capsule_get_size(&req->rq_pill, &RMF_DLM_LVB, RCL_CLIENT);
> -	if (lvb_len < 0) {
> -		LDLM_ERROR(lock, "Fail to get lvb_len, rc = %d", lvb_len);
> -		rc = lvb_len;
> -		goto out;
> -	} else if (lvb_len > 0) {
> -		if (lock->l_lvb_len > 0) {
> -			/* for extent lock, lvb contains ost_lvb{}. */
> -			LASSERT(lock->l_lvb_data);
> -
> -			if (unlikely(lock->l_lvb_len < lvb_len)) {
> -				LDLM_ERROR(lock,
> -					   "Replied LVB is larger than expectation, expected = %d, replied = %d",
> -					   lock->l_lvb_len, lvb_len);
> -				rc = -EINVAL;
> -				goto out;
> -			}
> -		} else if (ldlm_has_layout(lock)) { /* for layout lock, lvb has
> -						     * variable length
> -						     */
> -			void *lvb_data;
> -
> -			lvb_data = kzalloc(lvb_len, GFP_NOFS);
> -			if (!lvb_data) {
> -				LDLM_ERROR(lock, "No memory: %d.\n", lvb_len);
> -				rc = -ENOMEM;
> -				goto out;
> -			}
> -
> -			lock_res_and_lock(lock);
> -			LASSERT(!lock->l_lvb_data);
> -			lock->l_lvb_type = LVB_T_LAYOUT;
> -			lock->l_lvb_data = lvb_data;
> -			lock->l_lvb_len = lvb_len;
> -			unlock_res_and_lock(lock);
> -		}
> -	}
> -
> -	lock_res_and_lock(lock);
> -	if (ldlm_is_destroyed(lock) ||
> -	    lock->l_granted_mode == lock->l_req_mode) {
> -		/* bug 11300: the lock has already been granted */
> -		unlock_res_and_lock(lock);
> -		LDLM_DEBUG(lock, "Double grant race happened");
> -		rc = 0;
> -		goto out;
> -	}
> -
> -	/* If we receive the completion AST before the actual enqueue returned,
> -	 * then we might need to switch lock modes, resources, or extents.
> -	 */
> -	if (dlm_req->lock_desc.l_granted_mode != lock->l_req_mode) {
> -		lock->l_req_mode = dlm_req->lock_desc.l_granted_mode;
> -		LDLM_DEBUG(lock, "completion AST, new lock mode");
> -	}
> -
> -	if (lock->l_resource->lr_type != LDLM_PLAIN) {
> -		ldlm_convert_policy_to_local(req->rq_export,
> -					  dlm_req->lock_desc.l_resource.lr_type,
> -					  &dlm_req->lock_desc.l_policy_data,
> -					  &lock->l_policy_data);
> -		LDLM_DEBUG(lock, "completion AST, new policy data");
> -	}
> -
> -	ldlm_resource_unlink_lock(lock);
> -	if (memcmp(&dlm_req->lock_desc.l_resource.lr_name,
> -		   &lock->l_resource->lr_name,
> -		   sizeof(lock->l_resource->lr_name)) != 0) {
> -		unlock_res_and_lock(lock);
> -		rc = ldlm_lock_change_resource(ns, lock,
> -				&dlm_req->lock_desc.l_resource.lr_name);
> -		if (rc < 0) {
> -			LDLM_ERROR(lock, "Failed to allocate resource");
> -			goto out;
> -		}
> -		LDLM_DEBUG(lock, "completion AST, new resource");
> -		CERROR("change resource!\n");
> -		lock_res_and_lock(lock);
> -	}
> -
> -	if (dlm_req->lock_flags & LDLM_FL_AST_SENT) {
> -		/* BL_AST locks are not needed in LRU.
> -		 * Let ldlm_cancel_lru() be fast.
> -		 */
> -		ldlm_lock_remove_from_lru(lock);
> -		lock->l_flags |= LDLM_FL_CBPENDING | LDLM_FL_BL_AST;
> -		LDLM_DEBUG(lock, "completion AST includes blocking AST");
> -	}
> -
> -	if (lock->l_lvb_len > 0) {
> -		rc = ldlm_fill_lvb(lock, &req->rq_pill, RCL_CLIENT,
> -				   lock->l_lvb_data, lvb_len);
> -		if (rc < 0) {
> -			unlock_res_and_lock(lock);
> -			goto out;
> -		}
> -	}
> -
> -	ldlm_grant_lock(lock, &ast_list);
> -	unlock_res_and_lock(lock);
> -
> -	LDLM_DEBUG(lock, "callback handler finished, about to run_ast_work");
> -
> -	/* Let Enqueue to call osc_lock_upcall() and initialize l_ast_data */
> -	OBD_FAIL_TIMEOUT(OBD_FAIL_OSC_CP_ENQ_RACE, 2);
> -
> -	ldlm_run_ast_work(ns, &ast_list, LDLM_WORK_CP_AST);
> -
> -	LDLM_DEBUG_NOLOCK("client completion callback handler END (lock %p)",
> -			  lock);
> -	goto out;
> -
> -out:
> -	if (rc < 0) {
> -		lock_res_and_lock(lock);
> -		ldlm_set_failed(lock);
> -		unlock_res_and_lock(lock);
> -		wake_up(&lock->l_waitq);
> -	}
> -	LDLM_LOCK_RELEASE(lock);
> -}
> -
> -/**
> - * Callback handler for receiving incoming glimpse ASTs.
> - *
> - * This only can happen on client side.  After handling the glimpse AST
> - * we also consider dropping the lock here if it is unused locally for a
> - * long time.
> - */
> -static void ldlm_handle_gl_callback(struct ptlrpc_request *req,
> -				    struct ldlm_namespace *ns,
> -				    struct ldlm_request *dlm_req,
> -				    struct ldlm_lock *lock)
> -{
> -	int rc = -ENOSYS;
> -
> -	LDLM_DEBUG(lock, "client glimpse AST callback handler");
> -
> -	if (lock->l_glimpse_ast)
> -		rc = lock->l_glimpse_ast(lock, req);
> -
> -	if (req->rq_repmsg) {
> -		ptlrpc_reply(req);
> -	} else {
> -		req->rq_status = rc;
> -		ptlrpc_error(req);
> -	}
> -
> -	lock_res_and_lock(lock);
> -	if (lock->l_granted_mode == LCK_PW &&
> -	    !lock->l_readers && !lock->l_writers &&
> -	    time_after(jiffies,
> -		       lock->l_last_used + 10 * HZ)) {
> -		unlock_res_and_lock(lock);
> -		if (ldlm_bl_to_thread_lock(ns, NULL, lock))
> -			ldlm_handle_bl_callback(ns, NULL, lock);
> -
> -		return;
> -	}
> -	unlock_res_and_lock(lock);
> -	LDLM_LOCK_RELEASE(lock);
> -}
> -
> -static int ldlm_callback_reply(struct ptlrpc_request *req, int rc)
> -{
> -	if (req->rq_no_reply)
> -		return 0;
> -
> -	req->rq_status = rc;
> -	if (!req->rq_packed_final) {
> -		rc = lustre_pack_reply(req, 1, NULL, NULL);
> -		if (rc)
> -			return rc;
> -	}
> -	return ptlrpc_reply(req);
> -}
> -
> -static int __ldlm_bl_to_thread(struct ldlm_bl_work_item *blwi,
> -			       enum ldlm_cancel_flags cancel_flags)
> -{
> -	struct ldlm_bl_pool *blp = ldlm_state->ldlm_bl_pool;
> -
> -	spin_lock(&blp->blp_lock);
> -	if (blwi->blwi_lock && ldlm_is_discard_data(blwi->blwi_lock)) {
> -		/* add LDLM_FL_DISCARD_DATA requests to the priority list */
> -		list_add_tail(&blwi->blwi_entry, &blp->blp_prio_list);
> -	} else {
> -		/* other blocking callbacks are added to the regular list */
> -		list_add_tail(&blwi->blwi_entry, &blp->blp_list);
> -	}
> -	spin_unlock(&blp->blp_lock);
> -
> -	wake_up(&blp->blp_waitq);
> -
> -	/* can not check blwi->blwi_flags as blwi could be already freed in
> -	 * LCF_ASYNC mode
> -	 */
> -	if (!(cancel_flags & LCF_ASYNC))
> -		wait_for_completion(&blwi->blwi_comp);
> -
> -	return 0;
> -}
> -
> -static inline void init_blwi(struct ldlm_bl_work_item *blwi,
> -			     struct ldlm_namespace *ns,
> -			     struct ldlm_lock_desc *ld,
> -			     struct list_head *cancels, int count,
> -			     struct ldlm_lock *lock,
> -			     enum ldlm_cancel_flags cancel_flags)
> -{
> -	init_completion(&blwi->blwi_comp);
> -	INIT_LIST_HEAD(&blwi->blwi_head);
> -
> -	if (current->flags & PF_MEMALLOC)
> -		blwi->blwi_mem_pressure = 1;
> -
> -	blwi->blwi_ns = ns;
> -	blwi->blwi_flags = cancel_flags;
> -	if (ld)
> -		blwi->blwi_ld = *ld;
> -	if (count) {
> -		list_add(&blwi->blwi_head, cancels);
> -		list_del_init(cancels);
> -		blwi->blwi_count = count;
> -	} else {
> -		blwi->blwi_lock = lock;
> -	}
> -}
> -
> -/**
> - * Queues a list of locks \a cancels containing \a count locks
> - * for later processing by a blocking thread.  If \a count is zero,
> - * then the lock referenced as \a lock is queued instead.
> - *
> - * The blocking thread would then call ->l_blocking_ast callback in the lock.
> - * If list addition fails an error is returned and caller is supposed to
> - * call ->l_blocking_ast itself.
> - */
> -static int ldlm_bl_to_thread(struct ldlm_namespace *ns,
> -			     struct ldlm_lock_desc *ld,
> -			     struct ldlm_lock *lock,
> -			     struct list_head *cancels, int count,
> -			     enum ldlm_cancel_flags cancel_flags)
> -{
> -	if (cancels && count == 0)
> -		return 0;
> -
> -	if (cancel_flags & LCF_ASYNC) {
> -		struct ldlm_bl_work_item *blwi;
> -
> -		blwi = kzalloc(sizeof(*blwi), GFP_NOFS);
> -		if (!blwi)
> -			return -ENOMEM;
> -		init_blwi(blwi, ns, ld, cancels, count, lock, cancel_flags);
> -
> -		return __ldlm_bl_to_thread(blwi, cancel_flags);
> -	} else {
> -		/* if it is synchronous call do minimum mem alloc, as it could
> -		 * be triggered from kernel shrinker
> -		 */
> -		struct ldlm_bl_work_item blwi;
> -
> -		memset(&blwi, 0, sizeof(blwi));
> -		init_blwi(&blwi, ns, ld, cancels, count, lock, cancel_flags);
> -		return __ldlm_bl_to_thread(&blwi, cancel_flags);
> -	}
> -}
> -
> -int ldlm_bl_to_thread_lock(struct ldlm_namespace *ns, struct ldlm_lock_desc *ld,
> -			   struct ldlm_lock *lock)
> -{
> -	return ldlm_bl_to_thread(ns, ld, lock, NULL, 0, LCF_ASYNC);
> -}
> -
> -int ldlm_bl_to_thread_list(struct ldlm_namespace *ns, struct ldlm_lock_desc *ld,
> -			   struct list_head *cancels, int count,
> -			   enum ldlm_cancel_flags cancel_flags)
> -{
> -	return ldlm_bl_to_thread(ns, ld, NULL, cancels, count, cancel_flags);
> -}
> -
> -int ldlm_bl_thread_wakeup(void)
> -{
> -	wake_up(&ldlm_state->ldlm_bl_pool->blp_waitq);
> -	return 0;
> -}
> -
> -/* Setinfo coming from Server (eg MDT) to Client (eg MDC)! */
> -static int ldlm_handle_setinfo(struct ptlrpc_request *req)
> -{
> -	struct obd_device *obd = req->rq_export->exp_obd;
> -	char *key;
> -	void *val;
> -	int keylen, vallen;
> -	int rc = -ENOSYS;
> -
> -	DEBUG_REQ(D_HSM, req, "%s: handle setinfo\n", obd->obd_name);
> -
> -	req_capsule_set(&req->rq_pill, &RQF_OBD_SET_INFO);
> -
> -	key = req_capsule_client_get(&req->rq_pill, &RMF_SETINFO_KEY);
> -	if (!key) {
> -		DEBUG_REQ(D_IOCTL, req, "no set_info key");
> -		return -EFAULT;
> -	}
> -	keylen = req_capsule_get_size(&req->rq_pill, &RMF_SETINFO_KEY,
> -				      RCL_CLIENT);
> -	val = req_capsule_client_get(&req->rq_pill, &RMF_SETINFO_VAL);
> -	if (!val) {
> -		DEBUG_REQ(D_IOCTL, req, "no set_info val");
> -		return -EFAULT;
> -	}
> -	vallen = req_capsule_get_size(&req->rq_pill, &RMF_SETINFO_VAL,
> -				      RCL_CLIENT);
> -
> -	/* We are responsible for swabbing contents of val */
> -
> -	if (KEY_IS(KEY_HSM_COPYTOOL_SEND))
> -		/* Pass it on to mdc (the "export" in this case) */
> -		rc = obd_set_info_async(req->rq_svc_thread->t_env,
> -					req->rq_export,
> -					sizeof(KEY_HSM_COPYTOOL_SEND),
> -					KEY_HSM_COPYTOOL_SEND,
> -					vallen, val, NULL);
> -	else
> -		DEBUG_REQ(D_WARNING, req, "ignoring unknown key %s", key);
> -
> -	return rc;
> -}
> -
> -static inline void ldlm_callback_errmsg(struct ptlrpc_request *req,
> -					const char *msg, int rc,
> -					const struct lustre_handle *handle)
> -{
> -	DEBUG_REQ((req->rq_no_reply || rc) ? D_WARNING : D_DLMTRACE, req,
> -		  "%s: [nid %s] [rc %d] [lock %#llx]",
> -		  msg, libcfs_id2str(req->rq_peer), rc,
> -		  handle ? handle->cookie : 0);
> -	if (req->rq_no_reply)
> -		CWARN("No reply was sent, maybe cause bug 21636.\n");
> -	else if (rc)
> -		CWARN("Send reply failed, maybe cause bug 21636.\n");
> -}
> -
> -/* TODO: handle requests in a similar way as MDT: see mdt_handle_common() */
> -static int ldlm_callback_handler(struct ptlrpc_request *req)
> -{
> -	struct ldlm_namespace *ns;
> -	struct ldlm_request *dlm_req;
> -	struct ldlm_lock *lock;
> -	int rc;
> -
> -	/* Requests arrive in sender's byte order.  The ptlrpc service
> -	 * handler has already checked and, if necessary, byte-swapped the
> -	 * incoming request message body, but I am responsible for the
> -	 * message buffers.
> -	 */
> -
> -	/* do nothing for sec context finalize */
> -	if (lustre_msg_get_opc(req->rq_reqmsg) == SEC_CTX_FINI)
> -		return 0;
> -
> -	req_capsule_init(&req->rq_pill, req, RCL_SERVER);
> -
> -	if (!req->rq_export) {
> -		rc = ldlm_callback_reply(req, -ENOTCONN);
> -		ldlm_callback_errmsg(req, "Operate on unconnected server",
> -				     rc, NULL);
> -		return 0;
> -	}
> -
> -	LASSERT(req->rq_export->exp_obd);
> -
> -	switch (lustre_msg_get_opc(req->rq_reqmsg)) {
> -	case LDLM_BL_CALLBACK:
> -		if (OBD_FAIL_CHECK(OBD_FAIL_LDLM_BL_CALLBACK_NET)) {
> -			if (cfs_fail_err)
> -				ldlm_callback_reply(req, -(int)cfs_fail_err);
> -			return 0;
> -		}
> -		break;
> -	case LDLM_CP_CALLBACK:
> -		if (OBD_FAIL_CHECK(OBD_FAIL_LDLM_CP_CALLBACK_NET))
> -			return 0;
> -		break;
> -	case LDLM_GL_CALLBACK:
> -		if (OBD_FAIL_CHECK(OBD_FAIL_LDLM_GL_CALLBACK_NET))
> -			return 0;
> -		break;
> -	case LDLM_SET_INFO:
> -		rc = ldlm_handle_setinfo(req);
> -		ldlm_callback_reply(req, rc);
> -		return 0;
> -	default:
> -		CERROR("unknown opcode %u\n",
> -		       lustre_msg_get_opc(req->rq_reqmsg));
> -		ldlm_callback_reply(req, -EPROTO);
> -		return 0;
> -	}
> -
> -	ns = req->rq_export->exp_obd->obd_namespace;
> -	LASSERT(ns);
> -
> -	req_capsule_set(&req->rq_pill, &RQF_LDLM_CALLBACK);
> -
> -	dlm_req = req_capsule_client_get(&req->rq_pill, &RMF_DLM_REQ);
> -	if (!dlm_req) {
> -		rc = ldlm_callback_reply(req, -EPROTO);
> -		ldlm_callback_errmsg(req, "Operate without parameter", rc,
> -				     NULL);
> -		return 0;
> -	}
> -
> -	/* Force a known safe race, send a cancel to the server for a lock
> -	 * which the server has already started a blocking callback on.
> -	 */
> -	if (OBD_FAIL_CHECK(OBD_FAIL_LDLM_CANCEL_BL_CB_RACE) &&
> -	    lustre_msg_get_opc(req->rq_reqmsg) == LDLM_BL_CALLBACK) {
> -		rc = ldlm_cli_cancel(&dlm_req->lock_handle[0], 0);
> -		if (rc < 0)
> -			CERROR("ldlm_cli_cancel: %d\n", rc);
> -	}
> -
> -	lock = ldlm_handle2lock_long(&dlm_req->lock_handle[0], 0);
> -	if (!lock) {
> -		CDEBUG(D_DLMTRACE,
> -		       "callback on lock %#llx - lock disappeared\n",
> -		       dlm_req->lock_handle[0].cookie);
> -		rc = ldlm_callback_reply(req, -EINVAL);
> -		ldlm_callback_errmsg(req, "Operate with invalid parameter", rc,
> -				     &dlm_req->lock_handle[0]);
> -		return 0;
> -	}
> -
> -	if (ldlm_is_fail_loc(lock) &&
> -	    lustre_msg_get_opc(req->rq_reqmsg) == LDLM_BL_CALLBACK)
> -		OBD_RACE(OBD_FAIL_LDLM_CP_BL_RACE);
> -
> -	/* Copy hints/flags (e.g. LDLM_FL_DISCARD_DATA) from AST. */
> -	lock_res_and_lock(lock);
> -	lock->l_flags |= ldlm_flags_from_wire(dlm_req->lock_flags &
> -					      LDLM_FL_AST_MASK);
> -	if (lustre_msg_get_opc(req->rq_reqmsg) == LDLM_BL_CALLBACK) {
> -		/* If somebody cancels lock and cache is already dropped,
> -		 * or lock is failed before cp_ast received on client,
> -		 * we can tell the server we have no lock. Otherwise, we
> -		 * should send cancel after dropping the cache.
> -		 */
> -		if ((ldlm_is_canceling(lock) && ldlm_is_bl_done(lock)) ||
> -		    ldlm_is_failed(lock)) {
> -			LDLM_DEBUG(lock,
> -				   "callback on lock %#llx - lock disappeared",
> -				   dlm_req->lock_handle[0].cookie);
> -			unlock_res_and_lock(lock);
> -			LDLM_LOCK_RELEASE(lock);
> -			rc = ldlm_callback_reply(req, -EINVAL);
> -			ldlm_callback_errmsg(req, "Operate on stale lock", rc,
> -					     &dlm_req->lock_handle[0]);
> -			return 0;
> -		}
> -		/* BL_AST locks are not needed in LRU.
> -		 * Let ldlm_cancel_lru() be fast.
> -		 */
> -		ldlm_lock_remove_from_lru(lock);
> -		ldlm_set_bl_ast(lock);
> -	}
> -	unlock_res_and_lock(lock);
> -
> -	/* We want the ost thread to get this reply so that it can respond
> -	 * to ost requests (write cache writeback) that might be triggered
> -	 * in the callback.
> -	 *
> -	 * But we'd also like to be able to indicate in the reply that we're
> -	 * cancelling right now, because it's unused, or have an intent result
> -	 * in the reply, so we might have to push the responsibility for sending
> -	 * the reply down into the AST handlers, alas.
> -	 */
> -
> -	switch (lustre_msg_get_opc(req->rq_reqmsg)) {
> -	case LDLM_BL_CALLBACK:
> -		CDEBUG(D_INODE, "blocking ast\n");
> -		req_capsule_extend(&req->rq_pill, &RQF_LDLM_BL_CALLBACK);
> -		if (!ldlm_is_cancel_on_block(lock)) {
> -			rc = ldlm_callback_reply(req, 0);
> -			if (req->rq_no_reply || rc)
> -				ldlm_callback_errmsg(req, "Normal process", rc,
> -						     &dlm_req->lock_handle[0]);
> -		}
> -		if (ldlm_bl_to_thread_lock(ns, &dlm_req->lock_desc, lock))
> -			ldlm_handle_bl_callback(ns, &dlm_req->lock_desc, lock);
> -		break;
> -	case LDLM_CP_CALLBACK:
> -		CDEBUG(D_INODE, "completion ast\n");
> -		req_capsule_extend(&req->rq_pill, &RQF_LDLM_CP_CALLBACK);
> -		ldlm_callback_reply(req, 0);
> -		ldlm_handle_cp_callback(req, ns, dlm_req, lock);
> -		break;
> -	case LDLM_GL_CALLBACK:
> -		CDEBUG(D_INODE, "glimpse ast\n");
> -		req_capsule_extend(&req->rq_pill, &RQF_LDLM_GL_CALLBACK);
> -		ldlm_handle_gl_callback(req, ns, dlm_req, lock);
> -		break;
> -	default:
> -		LBUG();			 /* checked above */
> -	}
> -
> -	return 0;
> -}
> -
> -static int ldlm_bl_get_work(struct ldlm_bl_pool *blp,
> -			    struct ldlm_bl_work_item **p_blwi,
> -			    struct obd_export **p_exp)
> -{
> -	int num_th = atomic_read(&blp->blp_num_threads);
> -	struct ldlm_bl_work_item *blwi = NULL;
> -	static unsigned int num_bl;
> -
> -	spin_lock(&blp->blp_lock);
> -	/* process a request from the blp_list at least every blp_num_threads */
> -	if (!list_empty(&blp->blp_list) &&
> -	    (list_empty(&blp->blp_prio_list) || num_bl == 0))
> -		blwi = list_first_entry(&blp->blp_list,
> -					struct ldlm_bl_work_item, blwi_entry);
> -	else
> -		if (!list_empty(&blp->blp_prio_list))
> -			blwi = list_first_entry(&blp->blp_prio_list,
> -						struct ldlm_bl_work_item,
> -						blwi_entry);
> -
> -	if (blwi) {
> -		if (++num_bl >= num_th)
> -			num_bl = 0;
> -		list_del(&blwi->blwi_entry);
> -	}
> -	spin_unlock(&blp->blp_lock);
> -	*p_blwi = blwi;
> -
> -	return (*p_blwi || *p_exp) ? 1 : 0;
> -}
> -
> -/* This only contains temporary data until the thread starts */
> -struct ldlm_bl_thread_data {
> -	struct ldlm_bl_pool	*bltd_blp;
> -	struct completion	bltd_comp;
> -	int			bltd_num;
> -};
> -
> -static int ldlm_bl_thread_main(void *arg);
> -
> -static int ldlm_bl_thread_start(struct ldlm_bl_pool *blp, bool check_busy)
> -{
> -	struct ldlm_bl_thread_data bltd = { .bltd_blp = blp };
> -	struct task_struct *task;
> -
> -	init_completion(&bltd.bltd_comp);
> -
> -	bltd.bltd_num = atomic_inc_return(&blp->blp_num_threads);
> -	if (bltd.bltd_num >= blp->blp_max_threads) {
> -		atomic_dec(&blp->blp_num_threads);
> -		return 0;
> -	}
> -
> -	LASSERTF(bltd.bltd_num > 0, "thread num:%d\n", bltd.bltd_num);
> -	if (check_busy &&
> -	    atomic_read(&blp->blp_busy_threads) < (bltd.bltd_num - 1)) {
> -		atomic_dec(&blp->blp_num_threads);
> -		return 0;
> -	}
> -
> -	task = kthread_run(ldlm_bl_thread_main, &bltd, "ldlm_bl_%02d",
> -			   bltd.bltd_num);
> -	if (IS_ERR(task)) {
> -		CERROR("cannot start LDLM thread ldlm_bl_%02d: rc %ld\n",
> -		       bltd.bltd_num, PTR_ERR(task));
> -		atomic_dec(&blp->blp_num_threads);
> -		return PTR_ERR(task);
> -	}
> -	wait_for_completion(&bltd.bltd_comp);
> -
> -	return 0;
> -}
> -
> -/* Not fatal if racy and have a few too many threads */
> -static int ldlm_bl_thread_need_create(struct ldlm_bl_pool *blp,
> -				      struct ldlm_bl_work_item *blwi)
> -{
> -	if (atomic_read(&blp->blp_num_threads) >= blp->blp_max_threads)
> -		return 0;
> -
> -	if (atomic_read(&blp->blp_busy_threads) <
> -	    atomic_read(&blp->blp_num_threads))
> -		return 0;
> -
> -	if (blwi && (!blwi->blwi_ns || blwi->blwi_mem_pressure))
> -		return 0;
> -
> -	return 1;
> -}
> -
> -static int ldlm_bl_thread_blwi(struct ldlm_bl_pool *blp,
> -			       struct ldlm_bl_work_item *blwi)
> -{
> -	unsigned int flags = 0;
> -
> -	if (!blwi->blwi_ns)
> -		/* added by ldlm_cleanup() */
> -		return LDLM_ITER_STOP;
> -
> -	if (blwi->blwi_mem_pressure)
> -		flags = memalloc_noreclaim_save();
> -
> -	OBD_FAIL_TIMEOUT(OBD_FAIL_LDLM_PAUSE_CANCEL2, 4);
> -
> -	if (blwi->blwi_count) {
> -		int count;
> -
> -		/*
> -		 * The special case when we cancel locks in lru
> -		 * asynchronously, we pass the list of locks here.
> -		 * Thus locks are marked LDLM_FL_CANCELING, but NOT
> -		 * canceled locally yet.
> -		 */
> -		count = ldlm_cli_cancel_list_local(&blwi->blwi_head,
> -						   blwi->blwi_count,
> -						   LCF_BL_AST);
> -		ldlm_cli_cancel_list(&blwi->blwi_head, count, NULL,
> -				     blwi->blwi_flags);
> -	} else {
> -		ldlm_handle_bl_callback(blwi->blwi_ns, &blwi->blwi_ld,
> -					blwi->blwi_lock);
> -	}
> -	if (blwi->blwi_mem_pressure)
> -		memalloc_noreclaim_restore(flags);
> -
> -	if (blwi->blwi_flags & LCF_ASYNC)
> -		kfree(blwi);
> -	else
> -		complete(&blwi->blwi_comp);
> -
> -	return 0;
> -}
> -
> -/**
> - * Main blocking requests processing thread.
> - *
> - * Callers put locks into its queue by calling ldlm_bl_to_thread.
> - * This thread in the end ends up doing actual call to ->l_blocking_ast
> - * for queued locks.
> - */
> -static int ldlm_bl_thread_main(void *arg)
> -{
> -	struct ldlm_bl_pool *blp;
> -	struct ldlm_bl_thread_data *bltd = arg;
> -
> -	blp = bltd->bltd_blp;
> -
> -	complete(&bltd->bltd_comp);
> -	/* cannot use bltd after this, it is only on caller's stack */
> -
> -	while (1) {
> -		struct ldlm_bl_work_item *blwi = NULL;
> -		struct obd_export *exp = NULL;
> -		int rc;
> -
> -		rc = ldlm_bl_get_work(blp, &blwi, &exp);
> -		if (!rc)
> -			wait_event_idle_exclusive(blp->blp_waitq,
> -						  ldlm_bl_get_work(blp, &blwi,
> -								   &exp));
> -		atomic_inc(&blp->blp_busy_threads);
> -
> -		if (ldlm_bl_thread_need_create(blp, blwi))
> -			/* discard the return value, we tried */
> -			ldlm_bl_thread_start(blp, true);
> -
> -		if (blwi)
> -			rc = ldlm_bl_thread_blwi(blp, blwi);
> -
> -		atomic_dec(&blp->blp_busy_threads);
> -
> -		if (rc == LDLM_ITER_STOP)
> -			break;
> -	}
> -
> -	atomic_dec(&blp->blp_num_threads);
> -	complete(&blp->blp_comp);
> -	return 0;
> -}
> -
> -static int ldlm_setup(void);
> -static int ldlm_cleanup(void);
> -
> -int ldlm_get_ref(void)
> -{
> -	int rc = 0;
> -
> -	rc = ptlrpc_inc_ref();
> -	if (rc)
> -		return rc;
> -
> -	mutex_lock(&ldlm_ref_mutex);
> -	if (++ldlm_refcount == 1) {
> -		rc = ldlm_setup();
> -		if (rc)
> -			ldlm_refcount--;
> -	}
> -	mutex_unlock(&ldlm_ref_mutex);
> -
> -	if (rc)
> -		ptlrpc_dec_ref();
> -
> -	return rc;
> -}
> -
> -void ldlm_put_ref(void)
> -{
> -	int rc = 0;
> -	mutex_lock(&ldlm_ref_mutex);
> -	if (ldlm_refcount == 1) {
> -		rc = ldlm_cleanup();
> -
> -		if (rc)
> -			CERROR("ldlm_cleanup failed: %d\n", rc);
> -		else
> -			ldlm_refcount--;
> -	} else {
> -		ldlm_refcount--;
> -	}
> -	mutex_unlock(&ldlm_ref_mutex);
> -	if (!rc)
> -		ptlrpc_dec_ref();
> -}
> -
> -static ssize_t cancel_unused_locks_before_replay_show(struct kobject *kobj,
> -						      struct attribute *attr,
> -						      char *buf)
> -{
> -	return sprintf(buf, "%d\n", ldlm_cancel_unused_locks_before_replay);
> -}
> -
> -static ssize_t cancel_unused_locks_before_replay_store(struct kobject *kobj,
> -						       struct attribute *attr,
> -						       const char *buffer,
> -						       size_t count)
> -{
> -	int rc;
> -	unsigned long val;
> -
> -	rc = kstrtoul(buffer, 10, &val);
> -	if (rc)
> -		return rc;
> -
> -	ldlm_cancel_unused_locks_before_replay = val;
> -
> -	return count;
> -}
> -LUSTRE_RW_ATTR(cancel_unused_locks_before_replay);
> -
> -/* These are for root of /sys/fs/lustre/ldlm */
> -static struct attribute *ldlm_attrs[] = {
> -	&lustre_attr_cancel_unused_locks_before_replay.attr,
> -	NULL,
> -};
> -
> -static const struct attribute_group ldlm_attr_group = {
> -	.attrs = ldlm_attrs,
> -};
> -
> -static int ldlm_setup(void)
> -{
> -	static struct ptlrpc_service_conf	conf;
> -	struct ldlm_bl_pool			*blp = NULL;
> -	int rc = 0;
> -	int i;
> -
> -	if (ldlm_state)
> -		return -EALREADY;
> -
> -	ldlm_state = kzalloc(sizeof(*ldlm_state), GFP_NOFS);
> -	if (!ldlm_state)
> -		return -ENOMEM;
> -
> -	ldlm_kobj = kobject_create_and_add("ldlm", lustre_kobj);
> -	if (!ldlm_kobj) {
> -		rc = -ENOMEM;
> -		goto out;
> -	}
> -
> -	rc = sysfs_create_group(ldlm_kobj, &ldlm_attr_group);
> -	if (rc)
> -		goto out;
> -
> -	ldlm_ns_kset = kset_create_and_add("namespaces", NULL, ldlm_kobj);
> -	if (!ldlm_ns_kset) {
> -		rc = -ENOMEM;
> -		goto out;
> -	}
> -
> -	ldlm_svc_kset = kset_create_and_add("services", NULL, ldlm_kobj);
> -	if (!ldlm_svc_kset) {
> -		rc = -ENOMEM;
> -		goto out;
> -	}
> -
> -	ldlm_debugfs_setup();
> -
> -	memset(&conf, 0, sizeof(conf));
> -	conf = (typeof(conf)) {
> -		.psc_name		= "ldlm_cbd",
> -		.psc_watchdog_factor	= 2,
> -		.psc_buf		= {
> -			.bc_nbufs		= LDLM_CLIENT_NBUFS,
> -			.bc_buf_size		= LDLM_BUFSIZE,
> -			.bc_req_max_size	= LDLM_MAXREQSIZE,
> -			.bc_rep_max_size	= LDLM_MAXREPSIZE,
> -			.bc_req_portal		= LDLM_CB_REQUEST_PORTAL,
> -			.bc_rep_portal		= LDLM_CB_REPLY_PORTAL,
> -		},
> -		.psc_thr		= {
> -			.tc_thr_name		= "ldlm_cb",
> -			.tc_thr_factor		= LDLM_THR_FACTOR,
> -			.tc_nthrs_init		= LDLM_NTHRS_INIT,
> -			.tc_nthrs_base		= LDLM_NTHRS_BASE,
> -			.tc_nthrs_max		= LDLM_NTHRS_MAX,
> -			.tc_nthrs_user		= ldlm_num_threads,
> -			.tc_cpu_affinity	= 1,
> -			.tc_ctx_tags		= LCT_MD_THREAD | LCT_DT_THREAD,
> -		},
> -		.psc_cpt		= {
> -			.cc_pattern		= ldlm_cpts,
> -		},
> -		.psc_ops		= {
> -			.so_req_handler		= ldlm_callback_handler,
> -		},
> -	};
> -	ldlm_state->ldlm_cb_service =
> -			ptlrpc_register_service(&conf, ldlm_svc_kset,
> -						ldlm_svc_debugfs_dir);
> -	if (IS_ERR(ldlm_state->ldlm_cb_service)) {
> -		CERROR("failed to start service\n");
> -		rc = PTR_ERR(ldlm_state->ldlm_cb_service);
> -		ldlm_state->ldlm_cb_service = NULL;
> -		goto out;
> -	}
> -
> -	blp = kzalloc(sizeof(*blp), GFP_NOFS);
> -	if (!blp) {
> -		rc = -ENOMEM;
> -		goto out;
> -	}
> -	ldlm_state->ldlm_bl_pool = blp;
> -
> -	spin_lock_init(&blp->blp_lock);
> -	INIT_LIST_HEAD(&blp->blp_list);
> -	INIT_LIST_HEAD(&blp->blp_prio_list);
> -	init_waitqueue_head(&blp->blp_waitq);
> -	atomic_set(&blp->blp_num_threads, 0);
> -	atomic_set(&blp->blp_busy_threads, 0);
> -
> -	if (ldlm_num_threads == 0) {
> -		blp->blp_min_threads = LDLM_NTHRS_INIT;
> -		blp->blp_max_threads = LDLM_NTHRS_MAX;
> -	} else {
> -		blp->blp_min_threads = min_t(int, LDLM_NTHRS_MAX,
> -					     max_t(int, LDLM_NTHRS_INIT,
> -						   ldlm_num_threads));
> -
> -		blp->blp_max_threads = blp->blp_min_threads;
> -	}
> -
> -	for (i = 0; i < blp->blp_min_threads; i++) {
> -		rc = ldlm_bl_thread_start(blp, false);
> -		if (rc < 0)
> -			goto out;
> -	}
> -
> -	rc = ldlm_pools_init();
> -	if (rc) {
> -		CERROR("Failed to initialize LDLM pools: %d\n", rc);
> -		goto out;
> -	}
> -	return 0;
> -
> - out:
> -	ldlm_cleanup();
> -	return rc;
> -}
> -
> -static int ldlm_cleanup(void)
> -{
> -	if (!list_empty(ldlm_namespace_list(LDLM_NAMESPACE_SERVER)) ||
> -	    !list_empty(ldlm_namespace_list(LDLM_NAMESPACE_CLIENT))) {
> -		CERROR("ldlm still has namespaces; clean these up first.\n");
> -		ldlm_dump_all_namespaces(LDLM_NAMESPACE_SERVER, D_DLMTRACE);
> -		ldlm_dump_all_namespaces(LDLM_NAMESPACE_CLIENT, D_DLMTRACE);
> -		return -EBUSY;
> -	}
> -
> -	ldlm_pools_fini();
> -
> -	if (ldlm_state->ldlm_bl_pool) {
> -		struct ldlm_bl_pool *blp = ldlm_state->ldlm_bl_pool;
> -
> -		while (atomic_read(&blp->blp_num_threads) > 0) {
> -			struct ldlm_bl_work_item blwi = { .blwi_ns = NULL };
> -
> -			init_completion(&blp->blp_comp);
> -
> -			spin_lock(&blp->blp_lock);
> -			list_add_tail(&blwi.blwi_entry, &blp->blp_list);
> -			wake_up(&blp->blp_waitq);
> -			spin_unlock(&blp->blp_lock);
> -
> -			wait_for_completion(&blp->blp_comp);
> -		}
> -
> -		kfree(blp);
> -	}
> -
> -	if (ldlm_state->ldlm_cb_service)
> -		ptlrpc_unregister_service(ldlm_state->ldlm_cb_service);
> -
> -	if (ldlm_ns_kset)
> -		kset_unregister(ldlm_ns_kset);
> -	if (ldlm_svc_kset)
> -		kset_unregister(ldlm_svc_kset);
> -	if (ldlm_kobj) {
> -		sysfs_remove_group(ldlm_kobj, &ldlm_attr_group);
> -		kobject_put(ldlm_kobj);
> -	}
> -
> -	ldlm_debugfs_cleanup();
> -
> -	kfree(ldlm_state);
> -	ldlm_state = NULL;
> -
> -	return 0;
> -}
> -
> -int ldlm_init(void)
> -{
> -	mutex_init(&ldlm_ref_mutex);
> -	mutex_init(ldlm_namespace_lock(LDLM_NAMESPACE_SERVER));
> -	mutex_init(ldlm_namespace_lock(LDLM_NAMESPACE_CLIENT));
> -	ldlm_resource_slab = kmem_cache_create("ldlm_resources",
> -					       sizeof(struct ldlm_resource), 0,
> -					       SLAB_HWCACHE_ALIGN, NULL);
> -	if (!ldlm_resource_slab)
> -		return -ENOMEM;
> -
> -	ldlm_lock_slab = kmem_cache_create("ldlm_locks",
> -					   sizeof(struct ldlm_lock), 0,
> -					   SLAB_HWCACHE_ALIGN |
> -					   SLAB_TYPESAFE_BY_RCU, NULL);
> -	if (!ldlm_lock_slab) {
> -		kmem_cache_destroy(ldlm_resource_slab);
> -		return -ENOMEM;
> -	}
> -
> -#if LUSTRE_TRACKS_LOCK_EXP_REFS
> -	class_export_dump_hook = ldlm_dump_export_locks;
> -#endif
> -	return 0;
> -}
> -
> -void ldlm_exit(void)
> -{
> -	if (ldlm_refcount)
> -		CERROR("ldlm_refcount is %d in %s!\n", ldlm_refcount, __func__);
> -	kmem_cache_destroy(ldlm_resource_slab);
> -	/* ldlm_lock_put() use RCU to call ldlm_lock_free, so need call
> -	 * synchronize_rcu() to wait a grace period elapsed, so that
> -	 * ldlm_lock_free() get a chance to be called.
> -	 */
> -	synchronize_rcu();
> -	kmem_cache_destroy(ldlm_lock_slab);
> -}
> diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_plain.c b/drivers/staging/lustre/lustre/ldlm/ldlm_plain.c
> deleted file mode 100644
> index 33b5a3f96fcb..000000000000
> --- a/drivers/staging/lustre/lustre/ldlm/ldlm_plain.c
> +++ /dev/null
> @@ -1,68 +0,0 @@
> -// SPDX-License-Identifier: GPL-2.0
> -/*
> - * GPL HEADER START
> - *
> - * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
> - *
> - * This program is free software; you can redistribute it and/or modify
> - * it under the terms of the GNU General Public License version 2 only,
> - * as published by the Free Software Foundation.
> - *
> - * This program is distributed in the hope that it will be useful, but
> - * WITHOUT ANY WARRANTY; without even the implied warranty of
> - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> - * General Public License version 2 for more details (a copy is included
> - * in the LICENSE file that accompanied this code).
> - *
> - * You should have received a copy of the GNU General Public License
> - * version 2 along with this program; If not, see
> - * http://www.gnu.org/licenses/gpl-2.0.html
> - *
> - * GPL HEADER END
> - */
> -/*
> - * Copyright (c) 2003, 2010, Oracle and/or its affiliates. All rights reserved.
> - * Use is subject to license terms.
> - *
> - * Copyright (c) 2011, Intel Corporation.
> - */
> -/*
> - * This file is part of Lustre, http://www.lustre.org/
> - * Lustre is a trademark of Sun Microsystems, Inc.
> - *
> - * lustre/ldlm/ldlm_plain.c
> - *
> - * Author: Peter Braam <braam at clusterfs.com>
> - * Author: Phil Schwan <phil at clusterfs.com>
> - */
> -
> -/**
> - * This file contains implementation of PLAIN lock type.
> - *
> - * PLAIN locks are the simplest form of LDLM locking, and are used when
> - * there only needs to be a single lock on a resource. This avoids some
> - * of the complexity of EXTENT and IBITS lock types, but doesn't allow
> - * different "parts" of a resource to be locked concurrently.  Example
> - * use cases for PLAIN locks include locking of MGS configuration logs
> - * and (as of Lustre 2.4) quota records.
> - */
> -
> -#define DEBUG_SUBSYSTEM S_LDLM
> -
> -#include <lustre_dlm.h>
> -#include <obd_support.h>
> -#include <lustre_lib.h>
> -
> -#include "ldlm_internal.h"
> -
> -void ldlm_plain_policy_wire_to_local(const union ldlm_wire_policy_data *wpolicy,
> -				     union ldlm_policy_data *lpolicy)
> -{
> -	/* No policy for plain locks */
> -}
> -
> -void ldlm_plain_policy_local_to_wire(const union ldlm_policy_data *lpolicy,
> -				     union ldlm_wire_policy_data *wpolicy)
> -{
> -	/* No policy for plain locks */
> -}
> diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c b/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c
> deleted file mode 100644
> index 36d14ee4e5b1..000000000000
> --- a/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c
> +++ /dev/null
> @@ -1,1013 +0,0 @@
> -// SPDX-License-Identifier: GPL-2.0
> -/*
> - * GPL HEADER START
> - *
> - * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
> - *
> - * This program is free software; you can redistribute it and/or modify
> - * it under the terms of the GNU General Public License version 2 only,
> - * as published by the Free Software Foundation.
> - *
> - * This program is distributed in the hope that it will be useful, but
> - * WITHOUT ANY WARRANTY; without even the implied warranty of
> - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> - * General Public License version 2 for more details (a copy is included
> - * in the LICENSE file that accompanied this code).
> - *
> - * You should have received a copy of the GNU General Public License
> - * version 2 along with this program; If not, see
> - * http://www.gnu.org/licenses/gpl-2.0.html
> - *
> - * GPL HEADER END
> - */
> -/*
> - * Copyright (c) 2007, 2010, Oracle and/or its affiliates. All rights reserved.
> - * Use is subject to license terms.
> - *
> - * Copyright (c) 2010, 2015, Intel Corporation.
> - */
> -/*
> - * This file is part of Lustre, http://www.lustre.org/
> - * Lustre is a trademark of Sun Microsystems, Inc.
> - *
> - * lustre/ldlm/ldlm_pool.c
> - *
> - * Author: Yury Umanets <umka at clusterfs.com>
> - */
> -
> -/*
> - * Idea of this code is rather simple. Each second, for each server namespace
> - * we have SLV - server lock volume which is calculated on current number of
> - * granted locks, grant speed for past period, etc - that is, locking load.
> - * This SLV number may be thought as a flow definition for simplicity. It is
> - * sent to clients with each occasion to let them know what is current load
> - * situation on the server. By default, at the beginning, SLV on server is
> - * set max value which is calculated as the following: allow to one client
> - * have all locks of limit ->pl_limit for 10h.
> - *
> - * Next, on clients, number of cached locks is not limited artificially in any
> - * way as it was before. Instead, client calculates CLV, that is, client lock
> - * volume for each lock and compares it with last SLV from the server. CLV is
> - * calculated as the number of locks in LRU * lock live time in seconds. If
> - * CLV > SLV - lock is canceled.
> - *
> - * Client has LVF, that is, lock volume factor which regulates how much
> - * sensitive client should be about last SLV from server. The higher LVF is the
> - * more locks will be canceled on client. Default value for it is 1. Setting LVF
> - * to 2 means that client will cancel locks 2 times faster.
> - *
> - * Locks on a client will be canceled more intensively in these cases:
> - * (1) if SLV is smaller, that is, load is higher on the server;
> - * (2) client has a lot of locks (the more locks are held by client, the bigger
> - *     chances that some of them should be canceled);
> - * (3) client has old locks (taken some time ago);
> - *
> - * Thus, according to flow paradigm that we use for better understanding SLV,
> - * CLV is the volume of particle in flow described by SLV. According to this,
> - * if flow is getting thinner, more and more particles become outside of it and
> - * as particles are locks, they should be canceled.
> - *
> - * General idea of this belongs to Vitaly Fertman (vitaly at clusterfs.com).
> - * Andreas Dilger (adilger at clusterfs.com) proposed few nice ideas like using
> - * LVF and many cleanups. Flow definition to allow more easy understanding of
> - * the logic belongs to Nikita Danilov (nikita at clusterfs.com) as well as many
> - * cleanups and fixes. And design and implementation are done by Yury Umanets
> - * (umka at clusterfs.com).
> - *
> - * Glossary for terms used:
> - *
> - * pl_limit - Number of allowed locks in pool. Applies to server and client
> - * side (tunable);
> - *
> - * pl_granted - Number of granted locks (calculated);
> - * pl_grant_rate - Number of granted locks for last T (calculated);
> - * pl_cancel_rate - Number of canceled locks for last T (calculated);
> - * pl_grant_speed - Grant speed (GR - CR) for last T (calculated);
> - * pl_grant_plan - Planned number of granted locks for next T (calculated);
> - * pl_server_lock_volume - Current server lock volume (calculated);
> - *
> - * As it may be seen from list above, we have few possible tunables which may
> - * affect behavior much. They all may be modified via sysfs. However, they also
> - * give a possibility for constructing few pre-defined behavior policies. If
> - * none of predefines is suitable for a working pattern being used, new one may
> - * be "constructed" via sysfs tunables.
> - */
> -
> -#define DEBUG_SUBSYSTEM S_LDLM
> -
> -#include <lustre_dlm.h>
> -#include <cl_object.h>
> -#include <obd_class.h>
> -#include <obd_support.h>
> -#include "ldlm_internal.h"
> -
> -/*
> - * 50 ldlm locks for 1MB of RAM.
> - */
> -#define LDLM_POOL_HOST_L ((NUM_CACHEPAGES >> (20 - PAGE_SHIFT)) * 50)
> -
> -/*
> - * Maximal possible grant step plan in %.
> - */
> -#define LDLM_POOL_MAX_GSP (30)
> -
> -/*
> - * Minimal possible grant step plan in %.
> - */
> -#define LDLM_POOL_MIN_GSP (1)
> -
> -/*
> - * This controls the speed of reaching LDLM_POOL_MAX_GSP
> - * with increasing thread period.
> - */
> -#define LDLM_POOL_GSP_STEP_SHIFT (2)
> -
> -/*
> - * LDLM_POOL_GSP% of all locks is default GP.
> - */
> -#define LDLM_POOL_GP(L)   (((L) * LDLM_POOL_MAX_GSP) / 100)
> -
> -/*
> - * Max age for locks on clients.
> - */
> -#define LDLM_POOL_MAX_AGE (36000)
> -
> -/*
> - * The granularity of SLV calculation.
> - */
> -#define LDLM_POOL_SLV_SHIFT (10)
> -
> -static inline __u64 dru(__u64 val, __u32 shift, int round_up)
> -{
> -	return (val + (round_up ? (1 << shift) - 1 : 0)) >> shift;
> -}
> -
> -static inline __u64 ldlm_pool_slv_max(__u32 L)
> -{
> -	/*
> -	 * Allow to have all locks for 1 client for 10 hrs.
> -	 * Formula is the following: limit * 10h / 1 client.
> -	 */
> -	__u64 lim = (__u64)L *  LDLM_POOL_MAX_AGE / 1;
> -	return lim;
> -}
> -
> -static inline __u64 ldlm_pool_slv_min(__u32 L)
> -{
> -	return 1;
> -}
> -
> -enum {
> -	LDLM_POOL_FIRST_STAT = 0,
> -	LDLM_POOL_GRANTED_STAT = LDLM_POOL_FIRST_STAT,
> -	LDLM_POOL_GRANT_STAT,
> -	LDLM_POOL_CANCEL_STAT,
> -	LDLM_POOL_GRANT_RATE_STAT,
> -	LDLM_POOL_CANCEL_RATE_STAT,
> -	LDLM_POOL_GRANT_PLAN_STAT,
> -	LDLM_POOL_SLV_STAT,
> -	LDLM_POOL_SHRINK_REQTD_STAT,
> -	LDLM_POOL_SHRINK_FREED_STAT,
> -	LDLM_POOL_RECALC_STAT,
> -	LDLM_POOL_TIMING_STAT,
> -	LDLM_POOL_LAST_STAT
> -};
> -
> -/**
> - * Calculates suggested grant_step in % of available locks for passed
> - * \a period. This is later used in grant_plan calculations.
> - */
> -static inline int ldlm_pool_t2gsp(unsigned int t)
> -{
> -	/*
> -	 * This yields 1% grant step for anything below LDLM_POOL_GSP_STEP
> -	 * and up to 30% for anything higher than LDLM_POOL_GSP_STEP.
> -	 *
> -	 * How this will affect execution is the following:
> -	 *
> -	 * - for thread period 1s we will have grant_step 1% which good from
> -	 * pov of taking some load off from server and push it out to clients.
> -	 * This is like that because 1% for grant_step means that server will
> -	 * not allow clients to get lots of locks in short period of time and
> -	 * keep all old locks in their caches. Clients will always have to
> -	 * get some locks back if they want to take some new;
> -	 *
> -	 * - for thread period 10s (which is default) we will have 23% which
> -	 * means that clients will have enough of room to take some new locks
> -	 * without getting some back. All locks from this 23% which were not
> -	 * taken by clients in current period will contribute in SLV growing.
> -	 * SLV growing means more locks cached on clients until limit or grant
> -	 * plan is reached.
> -	 */
> -	return LDLM_POOL_MAX_GSP -
> -		((LDLM_POOL_MAX_GSP - LDLM_POOL_MIN_GSP) >>
> -		 (t >> LDLM_POOL_GSP_STEP_SHIFT));
> -}
> -
> -/**
> - * Recalculates next stats on passed \a pl.
> - *
> - * \pre ->pl_lock is locked.
> - */
> -static void ldlm_pool_recalc_stats(struct ldlm_pool *pl)
> -{
> -	int grant_plan = pl->pl_grant_plan;
> -	__u64 slv = pl->pl_server_lock_volume;
> -	int granted = atomic_read(&pl->pl_granted);
> -	int grant_rate = atomic_read(&pl->pl_grant_rate);
> -	int cancel_rate = atomic_read(&pl->pl_cancel_rate);
> -
> -	lprocfs_counter_add(pl->pl_stats, LDLM_POOL_SLV_STAT,
> -			    slv);
> -	lprocfs_counter_add(pl->pl_stats, LDLM_POOL_GRANTED_STAT,
> -			    granted);
> -	lprocfs_counter_add(pl->pl_stats, LDLM_POOL_GRANT_RATE_STAT,
> -			    grant_rate);
> -	lprocfs_counter_add(pl->pl_stats, LDLM_POOL_GRANT_PLAN_STAT,
> -			    grant_plan);
> -	lprocfs_counter_add(pl->pl_stats, LDLM_POOL_CANCEL_RATE_STAT,
> -			    cancel_rate);
> -}
> -
> -/**
> - * Sets SLV and Limit from container_of(pl, struct ldlm_namespace,
> - * ns_pool)->ns_obd tp passed \a pl.
> - */
> -static void ldlm_cli_pool_pop_slv(struct ldlm_pool *pl)
> -{
> -	struct obd_device *obd;
> -
> -	/*
> -	 * Get new SLV and Limit from obd which is updated with coming
> -	 * RPCs.
> -	 */
> -	obd = container_of(pl, struct ldlm_namespace,
> -			   ns_pool)->ns_obd;
> -	read_lock(&obd->obd_pool_lock);
> -	pl->pl_server_lock_volume = obd->obd_pool_slv;
> -	atomic_set(&pl->pl_limit, obd->obd_pool_limit);
> -	read_unlock(&obd->obd_pool_lock);
> -}
> -
> -/**
> - * Recalculates client size pool \a pl according to current SLV and Limit.
> - */
> -static int ldlm_cli_pool_recalc(struct ldlm_pool *pl)
> -{
> -	time64_t recalc_interval_sec;
> -	int ret;
> -
> -	recalc_interval_sec = ktime_get_real_seconds() - pl->pl_recalc_time;
> -	if (recalc_interval_sec < pl->pl_recalc_period)
> -		return 0;
> -
> -	spin_lock(&pl->pl_lock);
> -	/*
> -	 * Check if we need to recalc lists now.
> -	 */
> -	recalc_interval_sec = ktime_get_real_seconds() - pl->pl_recalc_time;
> -	if (recalc_interval_sec < pl->pl_recalc_period) {
> -		spin_unlock(&pl->pl_lock);
> -		return 0;
> -	}
> -
> -	/*
> -	 * Make sure that pool knows last SLV and Limit from obd.
> -	 */
> -	ldlm_cli_pool_pop_slv(pl);
> -
> -	spin_unlock(&pl->pl_lock);
> -
> -	/*
> -	 * Do not cancel locks in case lru resize is disabled for this ns.
> -	 */
> -	if (!ns_connect_lru_resize(container_of(pl, struct ldlm_namespace,
> -						ns_pool))) {
> -		ret = 0;
> -		goto out;
> -	}
> -
> -	/*
> -	 * In the time of canceling locks on client we do not need to maintain
> -	 * sharp timing, we only want to cancel locks asap according to new SLV.
> -	 * It may be called when SLV has changed much, this is why we do not
> -	 * take into account pl->pl_recalc_time here.
> -	 */
> -	ret = ldlm_cancel_lru(container_of(pl, struct ldlm_namespace, ns_pool),
> -			      0, LCF_ASYNC, LDLM_LRU_FLAG_LRUR);
> -
> -out:
> -	spin_lock(&pl->pl_lock);
> -	/*
> -	 * Time of LRU resizing might be longer than period,
> -	 * so update after LRU resizing rather than before it.
> -	 */
> -	pl->pl_recalc_time = ktime_get_real_seconds();
> -	lprocfs_counter_add(pl->pl_stats, LDLM_POOL_TIMING_STAT,
> -			    recalc_interval_sec);
> -	spin_unlock(&pl->pl_lock);
> -	return ret;
> -}
> -
> -/**
> - * This function is main entry point for memory pressure handling on client
> - * side.  Main goal of this function is to cancel some number of locks on
> - * passed \a pl according to \a nr and \a gfp_mask.
> - */
> -static int ldlm_cli_pool_shrink(struct ldlm_pool *pl,
> -				int nr, gfp_t gfp_mask)
> -{
> -	struct ldlm_namespace *ns;
> -	int unused;
> -
> -	ns = container_of(pl, struct ldlm_namespace, ns_pool);
> -
> -	/*
> -	 * Do not cancel locks in case lru resize is disabled for this ns.
> -	 */
> -	if (!ns_connect_lru_resize(ns))
> -		return 0;
> -
> -	/*
> -	 * Make sure that pool knows last SLV and Limit from obd.
> -	 */
> -	ldlm_cli_pool_pop_slv(pl);
> -
> -	spin_lock(&ns->ns_lock);
> -	unused = ns->ns_nr_unused;
> -	spin_unlock(&ns->ns_lock);
> -
> -	if (nr == 0)
> -		return (unused / 100) * sysctl_vfs_cache_pressure;
> -	else
> -		return ldlm_cancel_lru(ns, nr, LCF_ASYNC, LDLM_LRU_FLAG_SHRINK);
> -}
> -
> -static const struct ldlm_pool_ops ldlm_cli_pool_ops = {
> -	.po_recalc = ldlm_cli_pool_recalc,
> -	.po_shrink = ldlm_cli_pool_shrink
> -};
> -
> -/**
> - * Pool recalc wrapper. Will call either client or server pool recalc callback
> - * depending what pool \a pl is used.
> - */
> -static int ldlm_pool_recalc(struct ldlm_pool *pl)
> -{
> -	u32 recalc_interval_sec;
> -	int count;
> -
> -	recalc_interval_sec = ktime_get_real_seconds() - pl->pl_recalc_time;
> -	if (recalc_interval_sec > 0) {
> -		spin_lock(&pl->pl_lock);
> -		recalc_interval_sec = ktime_get_real_seconds() - pl->pl_recalc_time;
> -
> -		if (recalc_interval_sec > 0) {
> -			/*
> -			 * Update pool statistics every 1s.
> -			 */
> -			ldlm_pool_recalc_stats(pl);
> -
> -			/*
> -			 * Zero out all rates and speed for the last period.
> -			 */
> -			atomic_set(&pl->pl_grant_rate, 0);
> -			atomic_set(&pl->pl_cancel_rate, 0);
> -		}
> -		spin_unlock(&pl->pl_lock);
> -	}
> -
> -	if (pl->pl_ops->po_recalc) {
> -		count = pl->pl_ops->po_recalc(pl);
> -		lprocfs_counter_add(pl->pl_stats, LDLM_POOL_RECALC_STAT,
> -				    count);
> -	}
> -
> -	recalc_interval_sec = pl->pl_recalc_time - ktime_get_real_seconds() +
> -			      pl->pl_recalc_period;
> -	if (recalc_interval_sec <= 0) {
> -		/* DEBUG: should be re-removed after LU-4536 is fixed */
> -		CDEBUG(D_DLMTRACE,
> -		       "%s: Negative interval(%ld), too short period(%ld)\n",
> -		       pl->pl_name, (long)recalc_interval_sec,
> -		       (long)pl->pl_recalc_period);
> -
> -		/* Prevent too frequent recalculation. */
> -		recalc_interval_sec = 1;
> -	}
> -
> -	return recalc_interval_sec;
> -}
> -
> -/*
> - * Pool shrink wrapper. Will call either client or server pool recalc callback
> - * depending what pool pl is used. When nr == 0, just return the number of
> - * freeable locks. Otherwise, return the number of canceled locks.
> - */
> -static int ldlm_pool_shrink(struct ldlm_pool *pl, int nr, gfp_t gfp_mask)
> -{
> -	int cancel = 0;
> -
> -	if (pl->pl_ops->po_shrink) {
> -		cancel = pl->pl_ops->po_shrink(pl, nr, gfp_mask);
> -		if (nr > 0) {
> -			lprocfs_counter_add(pl->pl_stats,
> -					    LDLM_POOL_SHRINK_REQTD_STAT,
> -					    nr);
> -			lprocfs_counter_add(pl->pl_stats,
> -					    LDLM_POOL_SHRINK_FREED_STAT,
> -					    cancel);
> -			CDEBUG(D_DLMTRACE,
> -			       "%s: request to shrink %d locks, shrunk %d\n",
> -			       pl->pl_name, nr, cancel);
> -		}
> -	}
> -	return cancel;
> -}
> -
> -static int lprocfs_pool_state_seq_show(struct seq_file *m, void *unused)
> -{
> -	int granted, grant_rate, cancel_rate;
> -	int grant_speed, lvf;
> -	struct ldlm_pool *pl = m->private;
> -	__u64 slv, clv;
> -	__u32 limit;
> -
> -	spin_lock(&pl->pl_lock);
> -	slv = pl->pl_server_lock_volume;
> -	clv = pl->pl_client_lock_volume;
> -	limit = atomic_read(&pl->pl_limit);
> -	granted = atomic_read(&pl->pl_granted);
> -	grant_rate = atomic_read(&pl->pl_grant_rate);
> -	cancel_rate = atomic_read(&pl->pl_cancel_rate);
> -	grant_speed = grant_rate - cancel_rate;
> -	lvf = atomic_read(&pl->pl_lock_volume_factor);
> -	spin_unlock(&pl->pl_lock);
> -
> -	seq_printf(m, "LDLM pool state (%s):\n"
> -		      "  SLV: %llu\n"
> -		      "  CLV: %llu\n"
> -		      "  LVF: %d\n",
> -		      pl->pl_name, slv, clv, lvf);
> -
> -	seq_printf(m, "  GR:  %d\n  CR:  %d\n  GS:  %d\n"
> -		      "  G:   %d\n  L:   %d\n",
> -		      grant_rate, cancel_rate, grant_speed,
> -		      granted, limit);
> -
> -	return 0;
> -}
> -
> -LPROC_SEQ_FOPS_RO(lprocfs_pool_state);
> -
> -static ssize_t grant_speed_show(struct kobject *kobj, struct attribute *attr,
> -				char *buf)
> -{
> -	struct ldlm_pool *pl = container_of(kobj, struct ldlm_pool,
> -					    pl_kobj);
> -
> -	int	       grant_speed;
> -
> -	spin_lock(&pl->pl_lock);
> -	/* serialize with ldlm_pool_recalc */
> -	grant_speed = atomic_read(&pl->pl_grant_rate) -
> -			atomic_read(&pl->pl_cancel_rate);
> -	spin_unlock(&pl->pl_lock);
> -	return sprintf(buf, "%d\n", grant_speed);
> -}
> -LUSTRE_RO_ATTR(grant_speed);
> -
> -LDLM_POOL_SYSFS_READER_SHOW(grant_plan, int);
> -LUSTRE_RO_ATTR(grant_plan);
> -
> -LDLM_POOL_SYSFS_READER_SHOW(recalc_period, int);
> -LDLM_POOL_SYSFS_WRITER_STORE(recalc_period, int);
> -LUSTRE_RW_ATTR(recalc_period);
> -
> -LDLM_POOL_SYSFS_READER_NOLOCK_SHOW(server_lock_volume, u64);
> -LUSTRE_RO_ATTR(server_lock_volume);
> -
> -LDLM_POOL_SYSFS_READER_NOLOCK_SHOW(limit, atomic);
> -LDLM_POOL_SYSFS_WRITER_NOLOCK_STORE(limit, atomic);
> -LUSTRE_RW_ATTR(limit);
> -
> -LDLM_POOL_SYSFS_READER_NOLOCK_SHOW(granted, atomic);
> -LUSTRE_RO_ATTR(granted);
> -
> -LDLM_POOL_SYSFS_READER_NOLOCK_SHOW(cancel_rate, atomic);
> -LUSTRE_RO_ATTR(cancel_rate);
> -
> -LDLM_POOL_SYSFS_READER_NOLOCK_SHOW(grant_rate, atomic);
> -LUSTRE_RO_ATTR(grant_rate);
> -
> -LDLM_POOL_SYSFS_READER_NOLOCK_SHOW(lock_volume_factor, atomic);
> -LDLM_POOL_SYSFS_WRITER_NOLOCK_STORE(lock_volume_factor, atomic);
> -LUSTRE_RW_ATTR(lock_volume_factor);
> -
> -#define LDLM_POOL_ADD_VAR(name, var, ops)			\
> -	do {							\
> -		snprintf(var_name, MAX_STRING_SIZE, #name);	\
> -		pool_vars[0].data = var;			\
> -		pool_vars[0].fops = ops;			\
> -		ldebugfs_add_vars(pl->pl_debugfs_entry, pool_vars, NULL);\
> -	} while (0)
> -
> -/* These are for pools in /sys/fs/lustre/ldlm/namespaces/.../pool */
> -static struct attribute *ldlm_pl_attrs[] = {
> -	&lustre_attr_grant_speed.attr,
> -	&lustre_attr_grant_plan.attr,
> -	&lustre_attr_recalc_period.attr,
> -	&lustre_attr_server_lock_volume.attr,
> -	&lustre_attr_limit.attr,
> -	&lustre_attr_granted.attr,
> -	&lustre_attr_cancel_rate.attr,
> -	&lustre_attr_grant_rate.attr,
> -	&lustre_attr_lock_volume_factor.attr,
> -	NULL,
> -};
> -
> -static void ldlm_pl_release(struct kobject *kobj)
> -{
> -	struct ldlm_pool *pl = container_of(kobj, struct ldlm_pool,
> -					    pl_kobj);
> -	complete(&pl->pl_kobj_unregister);
> -}
> -
> -static struct kobj_type ldlm_pl_ktype = {
> -	.default_attrs	= ldlm_pl_attrs,
> -	.sysfs_ops	= &lustre_sysfs_ops,
> -	.release	= ldlm_pl_release,
> -};
> -
> -static int ldlm_pool_sysfs_init(struct ldlm_pool *pl)
> -{
> -	struct ldlm_namespace *ns = container_of(pl, struct ldlm_namespace,
> -						 ns_pool);
> -	int err;
> -
> -	init_completion(&pl->pl_kobj_unregister);
> -	err = kobject_init_and_add(&pl->pl_kobj, &ldlm_pl_ktype, &ns->ns_kobj,
> -				   "pool");
> -
> -	return err;
> -}
> -
> -static int ldlm_pool_debugfs_init(struct ldlm_pool *pl)
> -{
> -	struct ldlm_namespace *ns = container_of(pl, struct ldlm_namespace,
> -						 ns_pool);
> -	struct dentry *debugfs_ns_parent;
> -	struct lprocfs_vars pool_vars[2];
> -	char *var_name = NULL;
> -	int rc = 0;
> -
> -	var_name = kzalloc(MAX_STRING_SIZE + 1, GFP_NOFS);
> -	if (!var_name)
> -		return -ENOMEM;
> -
> -	debugfs_ns_parent = ns->ns_debugfs_entry;
> -	if (IS_ERR_OR_NULL(debugfs_ns_parent)) {
> -		CERROR("%s: debugfs entry is not initialized\n",
> -		       ldlm_ns_name(ns));
> -		rc = -EINVAL;
> -		goto out_free_name;
> -	}
> -	pl->pl_debugfs_entry = debugfs_create_dir("pool", debugfs_ns_parent);
> -
> -	var_name[MAX_STRING_SIZE] = '\0';
> -	memset(pool_vars, 0, sizeof(pool_vars));
> -	pool_vars[0].name = var_name;
> -
> -	LDLM_POOL_ADD_VAR(state, pl, &lprocfs_pool_state_fops);
> -
> -	pl->pl_stats = lprocfs_alloc_stats(LDLM_POOL_LAST_STAT -
> -					   LDLM_POOL_FIRST_STAT, 0);
> -	if (!pl->pl_stats) {
> -		rc = -ENOMEM;
> -		goto out_free_name;
> -	}
> -
> -	lprocfs_counter_init(pl->pl_stats, LDLM_POOL_GRANTED_STAT,
> -			     LPROCFS_CNTR_AVGMINMAX | LPROCFS_CNTR_STDDEV,
> -			     "granted", "locks");
> -	lprocfs_counter_init(pl->pl_stats, LDLM_POOL_GRANT_STAT,
> -			     LPROCFS_CNTR_AVGMINMAX | LPROCFS_CNTR_STDDEV,
> -			     "grant", "locks");
> -	lprocfs_counter_init(pl->pl_stats, LDLM_POOL_CANCEL_STAT,
> -			     LPROCFS_CNTR_AVGMINMAX | LPROCFS_CNTR_STDDEV,
> -			     "cancel", "locks");
> -	lprocfs_counter_init(pl->pl_stats, LDLM_POOL_GRANT_RATE_STAT,
> -			     LPROCFS_CNTR_AVGMINMAX | LPROCFS_CNTR_STDDEV,
> -			     "grant_rate", "locks/s");
> -	lprocfs_counter_init(pl->pl_stats, LDLM_POOL_CANCEL_RATE_STAT,
> -			     LPROCFS_CNTR_AVGMINMAX | LPROCFS_CNTR_STDDEV,
> -			     "cancel_rate", "locks/s");
> -	lprocfs_counter_init(pl->pl_stats, LDLM_POOL_GRANT_PLAN_STAT,
> -			     LPROCFS_CNTR_AVGMINMAX | LPROCFS_CNTR_STDDEV,
> -			     "grant_plan", "locks/s");
> -	lprocfs_counter_init(pl->pl_stats, LDLM_POOL_SLV_STAT,
> -			     LPROCFS_CNTR_AVGMINMAX | LPROCFS_CNTR_STDDEV,
> -			     "slv", "slv");
> -	lprocfs_counter_init(pl->pl_stats, LDLM_POOL_SHRINK_REQTD_STAT,
> -			     LPROCFS_CNTR_AVGMINMAX | LPROCFS_CNTR_STDDEV,
> -			     "shrink_request", "locks");
> -	lprocfs_counter_init(pl->pl_stats, LDLM_POOL_SHRINK_FREED_STAT,
> -			     LPROCFS_CNTR_AVGMINMAX | LPROCFS_CNTR_STDDEV,
> -			     "shrink_freed", "locks");
> -	lprocfs_counter_init(pl->pl_stats, LDLM_POOL_RECALC_STAT,
> -			     LPROCFS_CNTR_AVGMINMAX | LPROCFS_CNTR_STDDEV,
> -			     "recalc_freed", "locks");
> -	lprocfs_counter_init(pl->pl_stats, LDLM_POOL_TIMING_STAT,
> -			     LPROCFS_CNTR_AVGMINMAX | LPROCFS_CNTR_STDDEV,
> -			     "recalc_timing", "sec");
> -	debugfs_create_file("stats", 0644, pl->pl_debugfs_entry, pl->pl_stats,
> -			    &lprocfs_stats_seq_fops);
> -
> -out_free_name:
> -	kfree(var_name);
> -	return rc;
> -}
> -
> -static void ldlm_pool_sysfs_fini(struct ldlm_pool *pl)
> -{
> -	kobject_put(&pl->pl_kobj);
> -	wait_for_completion(&pl->pl_kobj_unregister);
> -}
> -
> -static void ldlm_pool_debugfs_fini(struct ldlm_pool *pl)
> -{
> -	if (pl->pl_stats) {
> -		lprocfs_free_stats(&pl->pl_stats);
> -		pl->pl_stats = NULL;
> -	}
> -	debugfs_remove_recursive(pl->pl_debugfs_entry);
> -}
> -
> -int ldlm_pool_init(struct ldlm_pool *pl, struct ldlm_namespace *ns,
> -		   int idx, enum ldlm_side client)
> -{
> -	int rc;
> -
> -	spin_lock_init(&pl->pl_lock);
> -	atomic_set(&pl->pl_granted, 0);
> -	pl->pl_recalc_time = ktime_get_real_seconds();
> -	atomic_set(&pl->pl_lock_volume_factor, 1);
> -
> -	atomic_set(&pl->pl_grant_rate, 0);
> -	atomic_set(&pl->pl_cancel_rate, 0);
> -	pl->pl_grant_plan = LDLM_POOL_GP(LDLM_POOL_HOST_L);
> -
> -	snprintf(pl->pl_name, sizeof(pl->pl_name), "ldlm-pool-%s-%d",
> -		 ldlm_ns_name(ns), idx);
> -
> -	atomic_set(&pl->pl_limit, 1);
> -	pl->pl_server_lock_volume = 0;
> -	pl->pl_ops = &ldlm_cli_pool_ops;
> -	pl->pl_recalc_period = LDLM_POOL_CLI_DEF_RECALC_PERIOD;
> -	pl->pl_client_lock_volume = 0;
> -	rc = ldlm_pool_debugfs_init(pl);
> -	if (rc)
> -		return rc;
> -
> -	rc = ldlm_pool_sysfs_init(pl);
> -	if (rc)
> -		return rc;
> -
> -	CDEBUG(D_DLMTRACE, "Lock pool %s is initialized\n", pl->pl_name);
> -
> -	return rc;
> -}
> -
> -void ldlm_pool_fini(struct ldlm_pool *pl)
> -{
> -	ldlm_pool_sysfs_fini(pl);
> -	ldlm_pool_debugfs_fini(pl);
> -
> -	/*
> -	 * Pool should not be used after this point. We can't free it here as
> -	 * it lives in struct ldlm_namespace, but still interested in catching
> -	 * any abnormal using cases.
> -	 */
> -	POISON(pl, 0x5a, sizeof(*pl));
> -}
> -
> -/**
> - * Add new taken ldlm lock \a lock into pool \a pl accounting.
> - */
> -void ldlm_pool_add(struct ldlm_pool *pl, struct ldlm_lock *lock)
> -{
> -	/*
> -	 * FLOCK locks are special in a sense that they are almost never
> -	 * cancelled, instead special kind of lock is used to drop them.
> -	 * also there is no LRU for flock locks, so no point in tracking
> -	 * them anyway.
> -	 */
> -	if (lock->l_resource->lr_type == LDLM_FLOCK)
> -		return;
> -
> -	atomic_inc(&pl->pl_granted);
> -	atomic_inc(&pl->pl_grant_rate);
> -	lprocfs_counter_incr(pl->pl_stats, LDLM_POOL_GRANT_STAT);
> -	/*
> -	 * Do not do pool recalc for client side as all locks which
> -	 * potentially may be canceled has already been packed into
> -	 * enqueue/cancel rpc. Also we do not want to run out of stack
> -	 * with too long call paths.
> -	 */
> -}
> -
> -/**
> - * Remove ldlm lock \a lock from pool \a pl accounting.
> - */
> -void ldlm_pool_del(struct ldlm_pool *pl, struct ldlm_lock *lock)
> -{
> -	/*
> -	 * Filter out FLOCK locks. Read above comment in ldlm_pool_add().
> -	 */
> -	if (lock->l_resource->lr_type == LDLM_FLOCK)
> -		return;
> -
> -	LASSERT(atomic_read(&pl->pl_granted) > 0);
> -	atomic_dec(&pl->pl_granted);
> -	atomic_inc(&pl->pl_cancel_rate);
> -
> -	lprocfs_counter_incr(pl->pl_stats, LDLM_POOL_CANCEL_STAT);
> -}
> -
> -/**
> - * Returns current \a pl SLV.
> - *
> - * \pre ->pl_lock is not locked.
> - */
> -__u64 ldlm_pool_get_slv(struct ldlm_pool *pl)
> -{
> -	__u64 slv;
> -
> -	spin_lock(&pl->pl_lock);
> -	slv = pl->pl_server_lock_volume;
> -	spin_unlock(&pl->pl_lock);
> -	return slv;
> -}
> -
> -/**
> - * Sets passed \a clv to \a pl.
> - *
> - * \pre ->pl_lock is not locked.
> - */
> -void ldlm_pool_set_clv(struct ldlm_pool *pl, __u64 clv)
> -{
> -	spin_lock(&pl->pl_lock);
> -	pl->pl_client_lock_volume = clv;
> -	spin_unlock(&pl->pl_lock);
> -}
> -
> -/**
> - * Returns current LVF from \a pl.
> - */
> -__u32 ldlm_pool_get_lvf(struct ldlm_pool *pl)
> -{
> -	return atomic_read(&pl->pl_lock_volume_factor);
> -}
> -
> -static int ldlm_pool_granted(struct ldlm_pool *pl)
> -{
> -	return atomic_read(&pl->pl_granted);
> -}
> -
> -/*
> - * count locks from all namespaces (if possible). Returns number of
> - * cached locks.
> - */
> -static unsigned long ldlm_pools_count(enum ldlm_side client, gfp_t gfp_mask)
> -{
> -	unsigned long total = 0;
> -	int nr_ns;
> -	struct ldlm_namespace *ns;
> -	struct ldlm_namespace *ns_old = NULL; /* loop detection */
> -
> -	if (client == LDLM_NAMESPACE_CLIENT && !(gfp_mask & __GFP_FS))
> -		return 0;
> -
> -	CDEBUG(D_DLMTRACE, "Request to count %s locks from all pools\n",
> -	       client == LDLM_NAMESPACE_CLIENT ? "client" : "server");
> -
> -	/*
> -	 * Find out how many resources we may release.
> -	 */
> -	for (nr_ns = ldlm_namespace_nr_read(client);
> -	     nr_ns > 0; nr_ns--) {
> -		mutex_lock(ldlm_namespace_lock(client));
> -		if (list_empty(ldlm_namespace_list(client))) {
> -			mutex_unlock(ldlm_namespace_lock(client));
> -			return 0;
> -		}
> -		ns = ldlm_namespace_first_locked(client);
> -
> -		if (ns == ns_old) {
> -			mutex_unlock(ldlm_namespace_lock(client));
> -			break;
> -		}
> -
> -		if (ldlm_ns_empty(ns)) {
> -			ldlm_namespace_move_to_inactive_locked(ns, client);
> -			mutex_unlock(ldlm_namespace_lock(client));
> -			continue;
> -		}
> -
> -		if (!ns_old)
> -			ns_old = ns;
> -
> -		ldlm_namespace_get(ns);
> -		ldlm_namespace_move_to_active_locked(ns, client);
> -		mutex_unlock(ldlm_namespace_lock(client));
> -		total += ldlm_pool_shrink(&ns->ns_pool, 0, gfp_mask);
> -		ldlm_namespace_put(ns);
> -	}
> -
> -	return total;
> -}
> -
> -static unsigned long ldlm_pools_scan(enum ldlm_side client, int nr,
> -				     gfp_t gfp_mask)
> -{
> -	unsigned long freed = 0;
> -	int tmp, nr_ns;
> -	struct ldlm_namespace *ns;
> -
> -	if (client == LDLM_NAMESPACE_CLIENT && !(gfp_mask & __GFP_FS))
> -		return -1;
> -
> -	/*
> -	 * Shrink at least ldlm_namespace_nr_read(client) namespaces.
> -	 */
> -	for (tmp = nr_ns = ldlm_namespace_nr_read(client);
> -	     tmp > 0; tmp--) {
> -		int cancel, nr_locks;
> -
> -		/*
> -		 * Do not call shrink under ldlm_namespace_lock(client)
> -		 */
> -		mutex_lock(ldlm_namespace_lock(client));
> -		if (list_empty(ldlm_namespace_list(client))) {
> -			mutex_unlock(ldlm_namespace_lock(client));
> -			break;
> -		}
> -		ns = ldlm_namespace_first_locked(client);
> -		ldlm_namespace_get(ns);
> -		ldlm_namespace_move_to_active_locked(ns, client);
> -		mutex_unlock(ldlm_namespace_lock(client));
> -
> -		nr_locks = ldlm_pool_granted(&ns->ns_pool);
> -		/*
> -		 * We use to shrink propotionally but with new shrinker API,
> -		 * we lost the total number of freeable locks.
> -		 */
> -		cancel = 1 + min_t(int, nr_locks, nr / nr_ns);
> -		freed += ldlm_pool_shrink(&ns->ns_pool, cancel, gfp_mask);
> -		ldlm_namespace_put(ns);
> -	}
> -	/*
> -	 * we only decrease the SLV in server pools shrinker, return
> -	 * SHRINK_STOP to kernel to avoid needless loop. LU-1128
> -	 */
> -	return freed;
> -}
> -
> -static unsigned long ldlm_pools_cli_count(struct shrinker *s,
> -					  struct shrink_control *sc)
> -{
> -	return ldlm_pools_count(LDLM_NAMESPACE_CLIENT, sc->gfp_mask);
> -}
> -
> -static unsigned long ldlm_pools_cli_scan(struct shrinker *s,
> -					 struct shrink_control *sc)
> -{
> -	return ldlm_pools_scan(LDLM_NAMESPACE_CLIENT, sc->nr_to_scan,
> -			       sc->gfp_mask);
> -}
> -
> -static void ldlm_pools_recalc(struct work_struct *ws);
> -static DECLARE_DELAYED_WORK(ldlm_recalc_pools, ldlm_pools_recalc);
> -
> -static void ldlm_pools_recalc(struct work_struct *ws)
> -{
> -	enum ldlm_side client = LDLM_NAMESPACE_CLIENT;
> -	struct ldlm_namespace *ns;
> -	struct ldlm_namespace *ns_old = NULL;
> -	/* seconds of sleep if no active namespaces */
> -	int time = LDLM_POOL_CLI_DEF_RECALC_PERIOD;
> -	int nr;
> -
> -	/*
> -	 * Recalc at least ldlm_namespace_nr_read(client) namespaces.
> -	 */
> -	for (nr = ldlm_namespace_nr_read(client); nr > 0; nr--) {
> -		int     skip;
> -		/*
> -		 * Lock the list, get first @ns in the list, getref, move it
> -		 * to the tail, unlock and call pool recalc. This way we avoid
> -		 * calling recalc under @ns lock what is really good as we get
> -		 * rid of potential deadlock on client nodes when canceling
> -		 * locks synchronously.
> -		 */
> -		mutex_lock(ldlm_namespace_lock(client));
> -		if (list_empty(ldlm_namespace_list(client))) {
> -			mutex_unlock(ldlm_namespace_lock(client));
> -			break;
> -		}
> -		ns = ldlm_namespace_first_locked(client);
> -
> -		if (ns_old == ns) { /* Full pass complete */
> -			mutex_unlock(ldlm_namespace_lock(client));
> -			break;
> -		}
> -
> -		/* We got an empty namespace, need to move it back to inactive
> -		 * list.
> -		 * The race with parallel resource creation is fine:
> -		 * - If they do namespace_get before our check, we fail the
> -		 *   check and they move this item to the end of the list anyway
> -		 * - If we do the check and then they do namespace_get, then
> -		 *   we move the namespace to inactive and they will move
> -		 *   it back to active (synchronised by the lock, so no clash
> -		 *   there).
> -		 */
> -		if (ldlm_ns_empty(ns)) {
> -			ldlm_namespace_move_to_inactive_locked(ns, client);
> -			mutex_unlock(ldlm_namespace_lock(client));
> -			continue;
> -		}
> -
> -		if (!ns_old)
> -			ns_old = ns;
> -
> -		spin_lock(&ns->ns_lock);
> -		/*
> -		 * skip ns which is being freed, and we don't want to increase
> -		 * its refcount again, not even temporarily. bz21519 & LU-499.
> -		 */
> -		if (ns->ns_stopping) {
> -			skip = 1;
> -		} else {
> -			skip = 0;
> -			ldlm_namespace_get(ns);
> -		}
> -		spin_unlock(&ns->ns_lock);
> -
> -		ldlm_namespace_move_to_active_locked(ns, client);
> -		mutex_unlock(ldlm_namespace_lock(client));
> -
> -		/*
> -		 * After setup is done - recalc the pool.
> -		 */
> -		if (!skip) {
> -			int ttime = ldlm_pool_recalc(&ns->ns_pool);
> -
> -			if (ttime < time)
> -				time = ttime;
> -
> -			ldlm_namespace_put(ns);
> -		}
> -	}
> -
> -	/* Wake up the blocking threads from time to time. */
> -	ldlm_bl_thread_wakeup();
> -
> -	schedule_delayed_work(&ldlm_recalc_pools, time * HZ);
> -}
> -
> -static int ldlm_pools_thread_start(void)
> -{
> -	schedule_delayed_work(&ldlm_recalc_pools, 0);
> -
> -	return 0;
> -}
> -
> -static void ldlm_pools_thread_stop(void)
> -{
> -	cancel_delayed_work_sync(&ldlm_recalc_pools);
> -}
> -
> -static struct shrinker ldlm_pools_cli_shrinker = {
> -	.count_objects	= ldlm_pools_cli_count,
> -	.scan_objects	= ldlm_pools_cli_scan,
> -	.seeks		= DEFAULT_SEEKS,
> -};
> -
> -int ldlm_pools_init(void)
> -{
> -	int rc;
> -
> -	rc = ldlm_pools_thread_start();
> -	if (!rc)
> -		rc = register_shrinker(&ldlm_pools_cli_shrinker);
> -
> -	return rc;
> -}
> -
> -void ldlm_pools_fini(void)
> -{
> -	unregister_shrinker(&ldlm_pools_cli_shrinker);
> -
> -	ldlm_pools_thread_stop();
> -}
> diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_request.c b/drivers/staging/lustre/lustre/ldlm/ldlm_request.c
> deleted file mode 100644
> index cdc52eed6d85..000000000000
> --- a/drivers/staging/lustre/lustre/ldlm/ldlm_request.c
> +++ /dev/null
> @@ -1,2033 +0,0 @@
> -// SPDX-License-Identifier: GPL-2.0
> -/*
> - * GPL HEADER START
> - *
> - * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
> - *
> - * This program is free software; you can redistribute it and/or modify
> - * it under the terms of the GNU General Public License version 2 only,
> - * as published by the Free Software Foundation.
> - *
> - * This program is distributed in the hope that it will be useful, but
> - * WITHOUT ANY WARRANTY; without even the implied warranty of
> - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> - * General Public License version 2 for more details (a copy is included
> - * in the LICENSE file that accompanied this code).
> - *
> - * You should have received a copy of the GNU General Public License
> - * version 2 along with this program; If not, see
> - * http://www.gnu.org/licenses/gpl-2.0.html
> - *
> - * GPL HEADER END
> - */
> -/*
> - * Copyright (c) 2002, 2010, Oracle and/or its affiliates. All rights reserved.
> - * Use is subject to license terms.
> - *
> - * Copyright (c) 2010, 2015, Intel Corporation.
> - */
> -/*
> - * This file is part of Lustre, http://www.lustre.org/
> - * Lustre is a trademark of Sun Microsystems, Inc.
> - */
> -/**
> - * This file contains Asynchronous System Trap (AST) handlers and related
> - * LDLM request-processing routines.
> - *
> - * An AST is a callback issued on a lock when its state is changed. There are
> - * several different types of ASTs (callbacks) registered for each lock:
> - *
> - * - completion AST: when a lock is enqueued by some process, but cannot be
> - *   granted immediately due to other conflicting locks on the same resource,
> - *   the completion AST is sent to notify the caller when the lock is
> - *   eventually granted
> - *
> - * - blocking AST: when a lock is granted to some process, if another process
> - *   enqueues a conflicting (blocking) lock on a resource, a blocking AST is
> - *   sent to notify the holder(s) of the lock(s) of the conflicting lock
> - *   request. The lock holder(s) must release their lock(s) on that resource in
> - *   a timely manner or be evicted by the server.
> - *
> - * - glimpse AST: this is used when a process wants information about a lock
> - *   (i.e. the lock value block (LVB)) but does not necessarily require holding
> - *   the lock. If the resource is locked, the lock holder(s) are sent glimpse
> - *   ASTs and the LVB is returned to the caller, and lock holder(s) may CANCEL
> - *   their lock(s) if they are idle. If the resource is not locked, the server
> - *   may grant the lock.
> - */
> -
> -#define DEBUG_SUBSYSTEM S_LDLM
> -
> -#include <lustre_errno.h>
> -#include <lustre_dlm.h>
> -#include <obd_class.h>
> -#include <obd.h>
> -#include <linux/libcfs/libcfs_hash.h>
> -
> -#include "ldlm_internal.h"
> -
> -unsigned int ldlm_enqueue_min = OBD_TIMEOUT_DEFAULT;
> -module_param(ldlm_enqueue_min, uint, 0644);
> -MODULE_PARM_DESC(ldlm_enqueue_min, "lock enqueue timeout minimum");
> -
> -/* in client side, whether the cached locks will be canceled before replay */
> -unsigned int ldlm_cancel_unused_locks_before_replay = 1;
> -
> -struct ldlm_async_args {
> -	struct lustre_handle lock_handle;
> -};
> -
> -/**
> - * ldlm_request_bufsize
> - *
> - * @count:	number of ldlm handles
> - * @type:	ldlm opcode
> - *
> - * If opcode=LDLM_ENQUEUE, 1 slot is already occupied,
> - * LDLM_LOCKREQ_HANDLE -1 slots are available.
> - * Otherwise, LDLM_LOCKREQ_HANDLE slots are available.
> - *
> - * Return:	size of the request buffer
> - */
> -static int ldlm_request_bufsize(int count, int type)
> -{
> -	int avail = LDLM_LOCKREQ_HANDLES;
> -
> -	if (type == LDLM_ENQUEUE)
> -		avail -= LDLM_ENQUEUE_CANCEL_OFF;
> -
> -	if (count > avail)
> -		avail = (count - avail) * sizeof(struct lustre_handle);
> -	else
> -		avail = 0;
> -
> -	return sizeof(struct ldlm_request) + avail;
> -}
> -
> -static void ldlm_expired_completion_wait(struct ldlm_lock *lock, __u32 conn_cnt)
> -{
> -	struct obd_import *imp;
> -	struct obd_device *obd;
> -
> -	if (!lock->l_conn_export) {
> -		static unsigned long next_dump, last_dump;
> -
> -		LDLM_ERROR(lock,
> -			   "lock timed out (enqueued at %lld, %llds ago); not entering recovery in server code, just going back to sleep",
> -			   (s64)lock->l_last_activity,
> -			   (s64)(ktime_get_real_seconds() -
> -				 lock->l_last_activity));
> -		if (time_after(jiffies, next_dump)) {
> -			last_dump = next_dump;
> -			next_dump = jiffies + 300 * HZ;
> -			ldlm_namespace_dump(D_DLMTRACE,
> -					    ldlm_lock_to_ns(lock));
> -			if (last_dump == 0)
> -				libcfs_debug_dumplog();
> -		}
> -		return;
> -	}
> -
> -	obd = lock->l_conn_export->exp_obd;
> -	imp = obd->u.cli.cl_import;
> -	ptlrpc_fail_import(imp, conn_cnt);
> -	LDLM_ERROR(lock,
> -		   "lock timed out (enqueued at %lld, %llds ago), entering recovery for %s@%s",
> -		   (s64)lock->l_last_activity,
> -		   (s64)(ktime_get_real_seconds() - lock->l_last_activity),
> -		   obd2cli_tgt(obd), imp->imp_connection->c_remote_uuid.uuid);
> -}
> -
> -/**
> - * Calculate the Completion timeout (covering enqueue, BL AST, data flush,
> - * lock cancel, and their replies). Used for lock completion timeout on the
> - * client side.
> - *
> - * \param[in] lock	lock which is waiting the completion callback
> - *
> - * \retval		timeout in seconds to wait for the server reply
> - */
> -/* We use the same basis for both server side and client side functions
> - * from a single node.
> - */
> -static unsigned int ldlm_cp_timeout(struct ldlm_lock *lock)
> -{
> -	unsigned int timeout;
> -
> -	if (AT_OFF)
> -		return obd_timeout;
> -
> -	/*
> -	 * Wait a long time for enqueue - server may have to callback a
> -	 * lock from another client.  Server will evict the other client if it
> -	 * doesn't respond reasonably, and then give us the lock.
> -	 */
> -	timeout = at_get(ldlm_lock_to_ns_at(lock));
> -	return max(3 * timeout, ldlm_enqueue_min);
> -}
> -
> -/**
> - * Helper function for ldlm_completion_ast(), updating timings when lock is
> - * actually granted.
> - */
> -static int ldlm_completion_tail(struct ldlm_lock *lock, void *data)
> -{
> -	long delay;
> -	int result = 0;
> -
> -	if (ldlm_is_destroyed(lock) || ldlm_is_failed(lock)) {
> -		LDLM_DEBUG(lock, "client-side enqueue: destroyed");
> -		result = -EIO;
> -	} else if (!data) {
> -		LDLM_DEBUG(lock, "client-side enqueue: granted");
> -	} else {
> -		/* Take into AT only CP RPC, not immediately granted locks */
> -		delay = ktime_get_real_seconds() - lock->l_last_activity;
> -		LDLM_DEBUG(lock, "client-side enqueue: granted after %lds",
> -			   delay);
> -
> -		/* Update our time estimate */
> -		at_measured(ldlm_lock_to_ns_at(lock), delay);
> -	}
> -	return result;
> -}
> -
> -/**
> - * Generic LDLM "completion" AST. This is called in several cases:
> - *
> - *     - when a reply to an ENQUEUE RPC is received from the server
> - *       (ldlm_cli_enqueue_fini()). Lock might be granted or not granted at
> - *       this point (determined by flags);
> - *
> - *     - when LDLM_CP_CALLBACK RPC comes to client to notify it that lock has
> - *       been granted;
> - *
> - *     - when ldlm_lock_match(LDLM_FL_LVB_READY) is about to wait until lock
> - *       gets correct lvb;
> - *
> - *     - to force all locks when resource is destroyed (cleanup_resource());
> - *
> - *     - during lock conversion (not used currently).
> - *
> - * If lock is not granted in the first case, this function waits until second
> - * or penultimate cases happen in some other thread.
> - *
> - */
> -int ldlm_completion_ast(struct ldlm_lock *lock, __u64 flags, void *data)
> -{
> -	/* XXX ALLOCATE - 160 bytes */
> -	struct obd_device *obd;
> -	struct obd_import *imp = NULL;
> -	__u32 timeout;
> -	__u32 conn_cnt = 0;
> -	int rc = 0;
> -
> -	if (flags == LDLM_FL_WAIT_NOREPROC) {
> -		LDLM_DEBUG(lock, "client-side enqueue waiting on pending lock");
> -		goto noreproc;
> -	}
> -
> -	if (!(flags & LDLM_FL_BLOCKED_MASK)) {
> -		wake_up(&lock->l_waitq);
> -		return 0;
> -	}
> -
> -	LDLM_DEBUG(lock,
> -		   "client-side enqueue returned a blocked lock, sleeping");
> -
> -noreproc:
> -
> -	obd = class_exp2obd(lock->l_conn_export);
> -
> -	/* if this is a local lock, then there is no import */
> -	if (obd)
> -		imp = obd->u.cli.cl_import;
> -
> -	timeout = ldlm_cp_timeout(lock);
> -
> -	lock->l_last_activity = ktime_get_real_seconds();
> -
> -	if (imp) {
> -		spin_lock(&imp->imp_lock);
> -		conn_cnt = imp->imp_conn_cnt;
> -		spin_unlock(&imp->imp_lock);
> -	}
> -	if (OBD_FAIL_CHECK_RESET(OBD_FAIL_LDLM_INTR_CP_AST,
> -				 OBD_FAIL_LDLM_CP_BL_RACE | OBD_FAIL_ONCE)) {
> -		ldlm_set_fail_loc(lock);
> -		rc = -EINTR;
> -	} else {
> -		/* Go to sleep until the lock is granted or canceled. */
> -		if (!ldlm_is_no_timeout(lock)) {
> -			/* Wait uninterruptible for a while first */
> -			rc = wait_event_idle_timeout(lock->l_waitq,
> -						     is_granted_or_cancelled(lock),
> -						     timeout * HZ);
> -			if (rc == 0)
> -				ldlm_expired_completion_wait(lock, conn_cnt);
> -		}
> -		/* Now wait abortable */
> -		if (rc == 0)
> -			rc = l_wait_event_abortable(lock->l_waitq,
> -						    is_granted_or_cancelled(lock));
> -		else
> -			rc = 0;
> -	}
> -
> -	if (rc) {
> -		LDLM_DEBUG(lock, "client-side enqueue waking up: failed (%d)",
> -			   rc);
> -		return rc;
> -	}
> -
> -	return ldlm_completion_tail(lock, data);
> -}
> -EXPORT_SYMBOL(ldlm_completion_ast);
> -
> -static void failed_lock_cleanup(struct ldlm_namespace *ns,
> -				struct ldlm_lock *lock, int mode)
> -{
> -	int need_cancel = 0;
> -
> -	/* Set a flag to prevent us from sending a CANCEL (bug 407) */
> -	lock_res_and_lock(lock);
> -	/* Check that lock is not granted or failed, we might race. */
> -	if ((lock->l_req_mode != lock->l_granted_mode) &&
> -	    !ldlm_is_failed(lock)) {
> -		/* Make sure that this lock will not be found by raced
> -		 * bl_ast and -EINVAL reply is sent to server anyways.
> -		 * bug 17645
> -		 */
> -		lock->l_flags |= LDLM_FL_LOCAL_ONLY | LDLM_FL_FAILED |
> -				 LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING;
> -		need_cancel = 1;
> -	}
> -	unlock_res_and_lock(lock);
> -
> -	if (need_cancel)
> -		LDLM_DEBUG(lock,
> -			   "setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING");
> -	else
> -		LDLM_DEBUG(lock, "lock was granted or failed in race");
> -
> -	/* XXX - HACK because we shouldn't call ldlm_lock_destroy()
> -	 *       from llite/file.c/ll_file_flock().
> -	 */
> -	/* This code makes for the fact that we do not have blocking handler on
> -	 * a client for flock locks. As such this is the place where we must
> -	 * completely kill failed locks. (interrupted and those that
> -	 * were waiting to be granted when server evicted us.
> -	 */
> -	if (lock->l_resource->lr_type == LDLM_FLOCK) {
> -		lock_res_and_lock(lock);
> -		if (!ldlm_is_destroyed(lock)) {
> -			ldlm_resource_unlink_lock(lock);
> -			ldlm_lock_decref_internal_nolock(lock, mode);
> -			ldlm_lock_destroy_nolock(lock);
> -		}
> -		unlock_res_and_lock(lock);
> -	} else {
> -		ldlm_lock_decref_internal(lock, mode);
> -	}
> -}
> -
> -/**
> - * Finishing portion of client lock enqueue code.
> - *
> - * Called after receiving reply from server.
> - */
> -int ldlm_cli_enqueue_fini(struct obd_export *exp, struct ptlrpc_request *req,
> -			  enum ldlm_type type, __u8 with_policy,
> -			  enum ldlm_mode mode,
> -			  __u64 *flags, void *lvb, __u32 lvb_len,
> -			  const struct lustre_handle *lockh, int rc)
> -{
> -	struct ldlm_namespace *ns = exp->exp_obd->obd_namespace;
> -	int is_replay = *flags & LDLM_FL_REPLAY;
> -	struct ldlm_lock *lock;
> -	struct ldlm_reply *reply;
> -	int cleanup_phase = 1;
> -
> -	lock = ldlm_handle2lock(lockh);
> -	/* ldlm_cli_enqueue is holding a reference on this lock. */
> -	if (!lock) {
> -		LASSERT(type == LDLM_FLOCK);
> -		return -ENOLCK;
> -	}
> -
> -	LASSERTF(ergo(lvb_len != 0, lvb_len == lock->l_lvb_len),
> -		 "lvb_len = %d, l_lvb_len = %d\n", lvb_len, lock->l_lvb_len);
> -
> -	if (rc != ELDLM_OK) {
> -		LASSERT(!is_replay);
> -		LDLM_DEBUG(lock, "client-side enqueue END (%s)",
> -			   rc == ELDLM_LOCK_ABORTED ? "ABORTED" : "FAILED");
> -
> -		if (rc != ELDLM_LOCK_ABORTED)
> -			goto cleanup;
> -	}
> -
> -	/* Before we return, swab the reply */
> -	reply = req_capsule_server_get(&req->rq_pill, &RMF_DLM_REP);
> -	if (!reply) {
> -		rc = -EPROTO;
> -		goto cleanup;
> -	}
> -
> -	if (lvb_len > 0) {
> -		int size = 0;
> -
> -		size = req_capsule_get_size(&req->rq_pill, &RMF_DLM_LVB,
> -					    RCL_SERVER);
> -		if (size < 0) {
> -			LDLM_ERROR(lock, "Fail to get lvb_len, rc = %d", size);
> -			rc = size;
> -			goto cleanup;
> -		} else if (unlikely(size > lvb_len)) {
> -			LDLM_ERROR(lock,
> -				   "Replied LVB is larger than expectation, expected = %d, replied = %d",
> -				   lvb_len, size);
> -			rc = -EINVAL;
> -			goto cleanup;
> -		}
> -		lvb_len = size;
> -	}
> -
> -	if (rc == ELDLM_LOCK_ABORTED) {
> -		if (lvb_len > 0 && lvb)
> -			rc = ldlm_fill_lvb(lock, &req->rq_pill, RCL_SERVER,
> -					   lvb, lvb_len);
> -		if (rc == 0)
> -			rc = ELDLM_LOCK_ABORTED;
> -		goto cleanup;
> -	}
> -
> -	/* lock enqueued on the server */
> -	cleanup_phase = 0;
> -
> -	lock_res_and_lock(lock);
> -	lock->l_remote_handle = reply->lock_handle;
> -
> -	*flags = ldlm_flags_from_wire(reply->lock_flags);
> -	lock->l_flags |= ldlm_flags_from_wire(reply->lock_flags &
> -					      LDLM_FL_INHERIT_MASK);
> -	unlock_res_and_lock(lock);
> -
> -	CDEBUG(D_INFO, "local: %p, remote cookie: %#llx, flags: 0x%llx\n",
> -	       lock, reply->lock_handle.cookie, *flags);
> -
> -	/* If enqueue returned a blocked lock but the completion handler has
> -	 * already run, then it fixed up the resource and we don't need to do it
> -	 * again.
> -	 */
> -	if ((*flags) & LDLM_FL_LOCK_CHANGED) {
> -		int newmode = reply->lock_desc.l_req_mode;
> -
> -		LASSERT(!is_replay);
> -		if (newmode && newmode != lock->l_req_mode) {
> -			LDLM_DEBUG(lock, "server returned different mode %s",
> -				   ldlm_lockname[newmode]);
> -			lock->l_req_mode = newmode;
> -		}
> -
> -		if (!ldlm_res_eq(&reply->lock_desc.l_resource.lr_name,
> -				 &lock->l_resource->lr_name)) {
> -			CDEBUG(D_INFO,
> -			       "remote intent success, locking " DLDLMRES " instead of " DLDLMRES "\n",
> -			       PLDLMRES(&reply->lock_desc.l_resource),
> -			       PLDLMRES(lock->l_resource));
> -
> -			rc = ldlm_lock_change_resource(ns, lock,
> -					&reply->lock_desc.l_resource.lr_name);
> -			if (rc || !lock->l_resource) {
> -				rc = -ENOMEM;
> -				goto cleanup;
> -			}
> -			LDLM_DEBUG(lock, "client-side enqueue, new resource");
> -		}
> -		if (with_policy)
> -			if (!(type == LDLM_IBITS &&
> -			      !(exp_connect_flags(exp) & OBD_CONNECT_IBITS)))
> -				/* We assume lock type cannot change on server*/
> -				ldlm_convert_policy_to_local(exp,
> -						lock->l_resource->lr_type,
> -						&reply->lock_desc.l_policy_data,
> -						&lock->l_policy_data);
> -		if (type != LDLM_PLAIN)
> -			LDLM_DEBUG(lock,
> -				   "client-side enqueue, new policy data");
> -	}
> -
> -	if ((*flags) & LDLM_FL_AST_SENT) {
> -		lock_res_and_lock(lock);
> -		lock->l_flags |= LDLM_FL_CBPENDING |  LDLM_FL_BL_AST;
> -		unlock_res_and_lock(lock);
> -		LDLM_DEBUG(lock, "enqueue reply includes blocking AST");
> -	}
> -
> -	/* If the lock has already been granted by a completion AST, don't
> -	 * clobber the LVB with an older one.
> -	 */
> -	if (lvb_len > 0) {
> -		/* We must lock or a racing completion might update lvb without
> -		 * letting us know and we'll clobber the correct value.
> -		 * Cannot unlock after the check either, as that still leaves
> -		 * a tiny window for completion to get in
> -		 */
> -		lock_res_and_lock(lock);
> -		if (lock->l_req_mode != lock->l_granted_mode)
> -			rc = ldlm_fill_lvb(lock, &req->rq_pill, RCL_SERVER,
> -					   lock->l_lvb_data, lvb_len);
> -		unlock_res_and_lock(lock);
> -		if (rc < 0) {
> -			cleanup_phase = 1;
> -			goto cleanup;
> -		}
> -	}
> -
> -	if (!is_replay) {
> -		rc = ldlm_lock_enqueue(ns, &lock, NULL, flags);
> -		if (lock->l_completion_ast) {
> -			int err = lock->l_completion_ast(lock, *flags, NULL);
> -
> -			if (!rc)
> -				rc = err;
> -			if (rc)
> -				cleanup_phase = 1;
> -		}
> -	}
> -
> -	if (lvb_len > 0 && lvb) {
> -		/* Copy the LVB here, and not earlier, because the completion
> -		 * AST (if any) can override what we got in the reply
> -		 */
> -		memcpy(lvb, lock->l_lvb_data, lvb_len);
> -	}
> -
> -	LDLM_DEBUG(lock, "client-side enqueue END");
> -cleanup:
> -	if (cleanup_phase == 1 && rc)
> -		failed_lock_cleanup(ns, lock, mode);
> -	/* Put lock 2 times, the second reference is held by ldlm_cli_enqueue */
> -	LDLM_LOCK_PUT(lock);
> -	LDLM_LOCK_RELEASE(lock);
> -	return rc;
> -}
> -EXPORT_SYMBOL(ldlm_cli_enqueue_fini);
> -
> -/**
> - * Estimate number of lock handles that would fit into request of given
> - * size.  PAGE_SIZE-512 is to allow TCP/IP and LNET headers to fit into
> - * a single page on the send/receive side. XXX: 512 should be changed to
> - * more adequate value.
> - */
> -static inline int ldlm_req_handles_avail(int req_size, int off)
> -{
> -	int avail;
> -
> -	avail = min_t(int, LDLM_MAXREQSIZE, PAGE_SIZE - 512) - req_size;
> -	if (likely(avail >= 0))
> -		avail /= (int)sizeof(struct lustre_handle);
> -	else
> -		avail = 0;
> -	avail += LDLM_LOCKREQ_HANDLES - off;
> -
> -	return avail;
> -}
> -
> -static inline int ldlm_capsule_handles_avail(struct req_capsule *pill,
> -					     enum req_location loc,
> -					     int off)
> -{
> -	u32 size = req_capsule_msg_size(pill, loc);
> -
> -	return ldlm_req_handles_avail(size, off);
> -}
> -
> -static inline int ldlm_format_handles_avail(struct obd_import *imp,
> -					    const struct req_format *fmt,
> -					    enum req_location loc, int off)
> -{
> -	u32 size = req_capsule_fmt_size(imp->imp_msg_magic, fmt, loc);
> -
> -	return ldlm_req_handles_avail(size, off);
> -}
> -
> -/**
> - * Cancel LRU locks and pack them into the enqueue request. Pack there the given
> - * \a count locks in \a cancels.
> - *
> - * This is to be called by functions preparing their own requests that
> - * might contain lists of locks to cancel in addition to actual operation
> - * that needs to be performed.
> - */
> -int ldlm_prep_elc_req(struct obd_export *exp, struct ptlrpc_request *req,
> -		      int version, int opc, int canceloff,
> -		      struct list_head *cancels, int count)
> -{
> -	struct ldlm_namespace   *ns = exp->exp_obd->obd_namespace;
> -	struct req_capsule      *pill = &req->rq_pill;
> -	struct ldlm_request     *dlm = NULL;
> -	int flags, avail, to_free, pack = 0;
> -	LIST_HEAD(head);
> -	int rc;
> -
> -	if (!cancels)
> -		cancels = &head;
> -	if (ns_connect_cancelset(ns)) {
> -		/* Estimate the amount of available space in the request. */
> -		req_capsule_filled_sizes(pill, RCL_CLIENT);
> -		avail = ldlm_capsule_handles_avail(pill, RCL_CLIENT, canceloff);
> -
> -		flags = ns_connect_lru_resize(ns) ?
> -			LDLM_LRU_FLAG_LRUR_NO_WAIT : LDLM_LRU_FLAG_AGED;
> -		to_free = !ns_connect_lru_resize(ns) &&
> -			  opc == LDLM_ENQUEUE ? 1 : 0;
> -
> -		/* Cancel LRU locks here _only_ if the server supports
> -		 * EARLY_CANCEL. Otherwise we have to send extra CANCEL
> -		 * RPC, which will make us slower.
> -		 */
> -		if (avail > count)
> -			count += ldlm_cancel_lru_local(ns, cancels, to_free,
> -						       avail - count, 0, flags);
> -		if (avail > count)
> -			pack = count;
> -		else
> -			pack = avail;
> -		req_capsule_set_size(pill, &RMF_DLM_REQ, RCL_CLIENT,
> -				     ldlm_request_bufsize(pack, opc));
> -	}
> -
> -	rc = ptlrpc_request_pack(req, version, opc);
> -	if (rc) {
> -		ldlm_lock_list_put(cancels, l_bl_ast, count);
> -		return rc;
> -	}
> -
> -	if (ns_connect_cancelset(ns)) {
> -		if (canceloff) {
> -			dlm = req_capsule_client_get(pill, &RMF_DLM_REQ);
> -			LASSERT(dlm);
> -			/* Skip first lock handler in ldlm_request_pack(),
> -			 * this method will increment @lock_count according
> -			 * to the lock handle amount actually written to
> -			 * the buffer.
> -			 */
> -			dlm->lock_count = canceloff;
> -		}
> -		/* Pack into the request @pack lock handles. */
> -		ldlm_cli_cancel_list(cancels, pack, req, 0);
> -		/* Prepare and send separate cancel RPC for others. */
> -		ldlm_cli_cancel_list(cancels, count - pack, NULL, 0);
> -	} else {
> -		ldlm_lock_list_put(cancels, l_bl_ast, count);
> -	}
> -	return 0;
> -}
> -EXPORT_SYMBOL(ldlm_prep_elc_req);
> -
> -int ldlm_prep_enqueue_req(struct obd_export *exp, struct ptlrpc_request *req,
> -			  struct list_head *cancels, int count)
> -{
> -	return ldlm_prep_elc_req(exp, req, LUSTRE_DLM_VERSION, LDLM_ENQUEUE,
> -				 LDLM_ENQUEUE_CANCEL_OFF, cancels, count);
> -}
> -EXPORT_SYMBOL(ldlm_prep_enqueue_req);
> -
> -static struct ptlrpc_request *ldlm_enqueue_pack(struct obd_export *exp,
> -						int lvb_len)
> -{
> -	struct ptlrpc_request *req;
> -	int rc;
> -
> -	req = ptlrpc_request_alloc(class_exp2cliimp(exp), &RQF_LDLM_ENQUEUE);
> -	if (!req)
> -		return ERR_PTR(-ENOMEM);
> -
> -	rc = ldlm_prep_enqueue_req(exp, req, NULL, 0);
> -	if (rc) {
> -		ptlrpc_request_free(req);
> -		return ERR_PTR(rc);
> -	}
> -
> -	req_capsule_set_size(&req->rq_pill, &RMF_DLM_LVB, RCL_SERVER, lvb_len);
> -	ptlrpc_request_set_replen(req);
> -	return req;
> -}
> -
> -/**
> - * Client-side lock enqueue.
> - *
> - * If a request has some specific initialisation it is passed in \a reqp,
> - * otherwise it is created in ldlm_cli_enqueue.
> - *
> - * Supports sync and async requests, pass \a async flag accordingly. If a
> - * request was created in ldlm_cli_enqueue and it is the async request,
> - * pass it to the caller in \a reqp.
> - */
> -int ldlm_cli_enqueue(struct obd_export *exp, struct ptlrpc_request **reqp,
> -		     struct ldlm_enqueue_info *einfo,
> -		     const struct ldlm_res_id *res_id,
> -		     union ldlm_policy_data const *policy, __u64 *flags,
> -		     void *lvb, __u32 lvb_len, enum lvb_type lvb_type,
> -		     struct lustre_handle *lockh, int async)
> -{
> -	struct ldlm_namespace *ns;
> -	struct ldlm_lock      *lock;
> -	struct ldlm_request   *body;
> -	int		    is_replay = *flags & LDLM_FL_REPLAY;
> -	int		    req_passed_in = 1;
> -	int		    rc, err;
> -	struct ptlrpc_request *req;
> -
> -	ns = exp->exp_obd->obd_namespace;
> -
> -	/* If we're replaying this lock, just check some invariants.
> -	 * If we're creating a new lock, get everything all setup nicely.
> -	 */
> -	if (is_replay) {
> -		lock = ldlm_handle2lock_long(lockh, 0);
> -		LASSERT(lock);
> -		LDLM_DEBUG(lock, "client-side enqueue START");
> -		LASSERT(exp == lock->l_conn_export);
> -	} else {
> -		const struct ldlm_callback_suite cbs = {
> -			.lcs_completion = einfo->ei_cb_cp,
> -			.lcs_blocking	= einfo->ei_cb_bl,
> -			.lcs_glimpse	= einfo->ei_cb_gl
> -		};
> -		lock = ldlm_lock_create(ns, res_id, einfo->ei_type,
> -					einfo->ei_mode, &cbs, einfo->ei_cbdata,
> -					lvb_len, lvb_type);
> -		if (IS_ERR(lock))
> -			return PTR_ERR(lock);
> -		/* for the local lock, add the reference */
> -		ldlm_lock_addref_internal(lock, einfo->ei_mode);
> -		ldlm_lock2handle(lock, lockh);
> -		if (policy)
> -			lock->l_policy_data = *policy;
> -
> -		if (einfo->ei_type == LDLM_EXTENT) {
> -			/* extent lock without policy is a bug */
> -			if (!policy)
> -				LBUG();
> -
> -			lock->l_req_extent = policy->l_extent;
> -		}
> -		LDLM_DEBUG(lock, "client-side enqueue START, flags %llx",
> -			   *flags);
> -	}
> -
> -	lock->l_conn_export = exp;
> -	lock->l_export = NULL;
> -	lock->l_blocking_ast = einfo->ei_cb_bl;
> -	lock->l_flags |= (*flags & (LDLM_FL_NO_LRU | LDLM_FL_EXCL));
> -	lock->l_last_activity = ktime_get_real_seconds();
> -
> -	/* lock not sent to server yet */
> -	if (!reqp || !*reqp) {
> -		req = ldlm_enqueue_pack(exp, lvb_len);
> -		if (IS_ERR(req)) {
> -			failed_lock_cleanup(ns, lock, einfo->ei_mode);
> -			LDLM_LOCK_RELEASE(lock);
> -			return PTR_ERR(req);
> -		}
> -
> -		req_passed_in = 0;
> -		if (reqp)
> -			*reqp = req;
> -	} else {
> -		int len;
> -
> -		req = *reqp;
> -		len = req_capsule_get_size(&req->rq_pill, &RMF_DLM_REQ,
> -					   RCL_CLIENT);
> -		LASSERTF(len >= sizeof(*body), "buflen[%d] = %d, not %d\n",
> -			 DLM_LOCKREQ_OFF, len, (int)sizeof(*body));
> -	}
> -
> -	/* Dump lock data into the request buffer */
> -	body = req_capsule_client_get(&req->rq_pill, &RMF_DLM_REQ);
> -	ldlm_lock2desc(lock, &body->lock_desc);
> -	body->lock_flags = ldlm_flags_to_wire(*flags);
> -	body->lock_handle[0] = *lockh;
> -
> -	if (async) {
> -		LASSERT(reqp);
> -		return 0;
> -	}
> -
> -	LDLM_DEBUG(lock, "sending request");
> -
> -	rc = ptlrpc_queue_wait(req);
> -
> -	err = ldlm_cli_enqueue_fini(exp, req, einfo->ei_type, policy ? 1 : 0,
> -				    einfo->ei_mode, flags, lvb, lvb_len,
> -				    lockh, rc);
> -
> -	/* If ldlm_cli_enqueue_fini did not find the lock, we need to free
> -	 * one reference that we took
> -	 */
> -	if (err == -ENOLCK)
> -		LDLM_LOCK_RELEASE(lock);
> -	else
> -		rc = err;
> -
> -	if (!req_passed_in && req) {
> -		ptlrpc_req_finished(req);
> -		if (reqp)
> -			*reqp = NULL;
> -	}
> -
> -	return rc;
> -}
> -EXPORT_SYMBOL(ldlm_cli_enqueue);
> -
> -/**
> - * Cancel locks locally.
> - * Returns:
> - * \retval LDLM_FL_LOCAL_ONLY if there is no need for a CANCEL RPC to the server
> - * \retval LDLM_FL_CANCELING otherwise;
> - * \retval LDLM_FL_BL_AST if there is a need for a separate CANCEL RPC.
> - */
> -static __u64 ldlm_cli_cancel_local(struct ldlm_lock *lock)
> -{
> -	__u64 rc = LDLM_FL_LOCAL_ONLY;
> -
> -	if (lock->l_conn_export) {
> -		bool local_only;
> -
> -		LDLM_DEBUG(lock, "client-side cancel");
> -		/* Set this flag to prevent others from getting new references*/
> -		lock_res_and_lock(lock);
> -		ldlm_set_cbpending(lock);
> -		local_only = !!(lock->l_flags &
> -				(LDLM_FL_LOCAL_ONLY | LDLM_FL_CANCEL_ON_BLOCK));
> -		ldlm_cancel_callback(lock);
> -		rc = ldlm_is_bl_ast(lock) ? LDLM_FL_BL_AST : LDLM_FL_CANCELING;
> -		unlock_res_and_lock(lock);
> -
> -		if (local_only) {
> -			CDEBUG(D_DLMTRACE,
> -			       "not sending request (at caller's instruction)\n");
> -			rc = LDLM_FL_LOCAL_ONLY;
> -		}
> -		ldlm_lock_cancel(lock);
> -	} else {
> -		LDLM_ERROR(lock, "Trying to cancel local lock");
> -		LBUG();
> -	}
> -
> -	return rc;
> -}
> -
> -/**
> - * Pack \a count locks in \a head into ldlm_request buffer of request \a req.
> - */
> -static void ldlm_cancel_pack(struct ptlrpc_request *req,
> -			     struct list_head *head, int count)
> -{
> -	struct ldlm_request *dlm;
> -	struct ldlm_lock *lock;
> -	int max, packed = 0;
> -
> -	dlm = req_capsule_client_get(&req->rq_pill, &RMF_DLM_REQ);
> -	LASSERT(dlm);
> -
> -	/* Check the room in the request buffer. */
> -	max = req_capsule_get_size(&req->rq_pill, &RMF_DLM_REQ, RCL_CLIENT) -
> -		sizeof(struct ldlm_request);
> -	max /= sizeof(struct lustre_handle);
> -	max += LDLM_LOCKREQ_HANDLES;
> -	LASSERT(max >= dlm->lock_count + count);
> -
> -	/* XXX: it would be better to pack lock handles grouped by resource.
> -	 * so that the server cancel would call filter_lvbo_update() less
> -	 * frequently.
> -	 */
> -	list_for_each_entry(lock, head, l_bl_ast) {
> -		if (!count--)
> -			break;
> -		LASSERT(lock->l_conn_export);
> -		/* Pack the lock handle to the given request buffer. */
> -		LDLM_DEBUG(lock, "packing");
> -		dlm->lock_handle[dlm->lock_count++] = lock->l_remote_handle;
> -		packed++;
> -	}
> -	CDEBUG(D_DLMTRACE, "%d locks packed\n", packed);
> -}
> -
> -/**
> - * Prepare and send a batched cancel RPC. It will include \a count lock
> - * handles of locks given in \a cancels list.
> - */
> -static int ldlm_cli_cancel_req(struct obd_export *exp,
> -			       struct list_head *cancels,
> -			       int count, enum ldlm_cancel_flags flags)
> -{
> -	struct ptlrpc_request *req = NULL;
> -	struct obd_import *imp;
> -	int free, sent = 0;
> -	int rc = 0;
> -
> -	LASSERT(exp);
> -	LASSERT(count > 0);
> -
> -	CFS_FAIL_TIMEOUT(OBD_FAIL_LDLM_PAUSE_CANCEL, cfs_fail_val);
> -
> -	if (CFS_FAIL_CHECK(OBD_FAIL_LDLM_CANCEL_RACE))
> -		return count;
> -
> -	free = ldlm_format_handles_avail(class_exp2cliimp(exp),
> -					 &RQF_LDLM_CANCEL, RCL_CLIENT, 0);
> -	if (count > free)
> -		count = free;
> -
> -	while (1) {
> -		imp = class_exp2cliimp(exp);
> -		if (!imp || imp->imp_invalid) {
> -			CDEBUG(D_DLMTRACE,
> -			       "skipping cancel on invalid import %p\n", imp);
> -			return count;
> -		}
> -
> -		req = ptlrpc_request_alloc(imp, &RQF_LDLM_CANCEL);
> -		if (!req) {
> -			rc = -ENOMEM;
> -			goto out;
> -		}
> -
> -		req_capsule_filled_sizes(&req->rq_pill, RCL_CLIENT);
> -		req_capsule_set_size(&req->rq_pill, &RMF_DLM_REQ, RCL_CLIENT,
> -				     ldlm_request_bufsize(count, LDLM_CANCEL));
> -
> -		rc = ptlrpc_request_pack(req, LUSTRE_DLM_VERSION, LDLM_CANCEL);
> -		if (rc) {
> -			ptlrpc_request_free(req);
> -			goto out;
> -		}
> -
> -		req->rq_request_portal = LDLM_CANCEL_REQUEST_PORTAL;
> -		req->rq_reply_portal = LDLM_CANCEL_REPLY_PORTAL;
> -		ptlrpc_at_set_req_timeout(req);
> -
> -		ldlm_cancel_pack(req, cancels, count);
> -
> -		ptlrpc_request_set_replen(req);
> -		if (flags & LCF_ASYNC) {
> -			ptlrpcd_add_req(req);
> -			sent = count;
> -			goto out;
> -		}
> -
> -		rc = ptlrpc_queue_wait(req);
> -		if (rc == LUSTRE_ESTALE) {
> -			CDEBUG(D_DLMTRACE,
> -			       "client/server (nid %s) out of sync -- not fatal\n",
> -			       libcfs_nid2str(req->rq_import->
> -					      imp_connection->c_peer.nid));
> -			rc = 0;
> -		} else if (rc == -ETIMEDOUT && /* check there was no reconnect*/
> -			   req->rq_import_generation == imp->imp_generation) {
> -			ptlrpc_req_finished(req);
> -			continue;
> -		} else if (rc != ELDLM_OK) {
> -			/* -ESHUTDOWN is common on umount */
> -			CDEBUG_LIMIT(rc == -ESHUTDOWN ? D_DLMTRACE : D_ERROR,
> -				     "Got rc %d from cancel RPC: canceling anyway\n",
> -				     rc);
> -			break;
> -		}
> -		sent = count;
> -		break;
> -	}
> -
> -	ptlrpc_req_finished(req);
> -out:
> -	return sent ? sent : rc;
> -}
> -
> -static inline struct ldlm_pool *ldlm_imp2pl(struct obd_import *imp)
> -{
> -	return &imp->imp_obd->obd_namespace->ns_pool;
> -}
> -
> -/**
> - * Update client's OBD pool related fields with new SLV and Limit from \a req.
> - */
> -int ldlm_cli_update_pool(struct ptlrpc_request *req)
> -{
> -	struct obd_device *obd;
> -	__u64 new_slv;
> -	__u32 new_limit;
> -
> -	if (unlikely(!req->rq_import || !req->rq_import->imp_obd ||
> -		     !imp_connect_lru_resize(req->rq_import))) {
> -		/*
> -		 * Do nothing for corner cases.
> -		 */
> -		return 0;
> -	}
> -
> -	/* In some cases RPC may contain SLV and limit zeroed out. This
> -	 * is the case when server does not support LRU resize feature.
> -	 * This is also possible in some recovery cases when server-side
> -	 * reqs have no reference to the OBD export and thus access to
> -	 * server-side namespace is not possible.
> -	 */
> -	if (lustre_msg_get_slv(req->rq_repmsg) == 0 ||
> -	    lustre_msg_get_limit(req->rq_repmsg) == 0) {
> -		DEBUG_REQ(D_HA, req,
> -			  "Zero SLV or Limit found (SLV: %llu, Limit: %u)",
> -			  lustre_msg_get_slv(req->rq_repmsg),
> -			  lustre_msg_get_limit(req->rq_repmsg));
> -		return 0;
> -	}
> -
> -	new_limit = lustre_msg_get_limit(req->rq_repmsg);
> -	new_slv = lustre_msg_get_slv(req->rq_repmsg);
> -	obd = req->rq_import->imp_obd;
> -
> -	/* Set new SLV and limit in OBD fields to make them accessible
> -	 * to the pool thread. We do not access obd_namespace and pool
> -	 * directly here as there is no reliable way to make sure that
> -	 * they are still alive at cleanup time. Evil races are possible
> -	 * which may cause Oops at that time.
> -	 */
> -	write_lock(&obd->obd_pool_lock);
> -	obd->obd_pool_slv = new_slv;
> -	obd->obd_pool_limit = new_limit;
> -	write_unlock(&obd->obd_pool_lock);
> -
> -	return 0;
> -}
> -
> -/**
> - * Client side lock cancel.
> - *
> - * Lock must not have any readers or writers by this time.
> - */
> -int ldlm_cli_cancel(const struct lustre_handle *lockh,
> -		    enum ldlm_cancel_flags cancel_flags)
> -{
> -	struct obd_export *exp;
> -	int avail, flags, count = 1;
> -	__u64 rc = 0;
> -	struct ldlm_namespace *ns;
> -	struct ldlm_lock *lock;
> -	LIST_HEAD(cancels);
> -
> -	lock = ldlm_handle2lock_long(lockh, 0);
> -	if (!lock) {
> -		LDLM_DEBUG_NOLOCK("lock is already being destroyed");
> -		return 0;
> -	}
> -
> -	lock_res_and_lock(lock);
> -	/* Lock is being canceled and the caller doesn't want to wait */
> -	if (ldlm_is_canceling(lock) && (cancel_flags & LCF_ASYNC)) {
> -		unlock_res_and_lock(lock);
> -		LDLM_LOCK_RELEASE(lock);
> -		return 0;
> -	}
> -
> -	ldlm_set_canceling(lock);
> -	unlock_res_and_lock(lock);
> -
> -	rc = ldlm_cli_cancel_local(lock);
> -	if (rc == LDLM_FL_LOCAL_ONLY || cancel_flags & LCF_LOCAL) {
> -		LDLM_LOCK_RELEASE(lock);
> -		return 0;
> -	}
> -	/* Even if the lock is marked as LDLM_FL_BL_AST, this is a LDLM_CANCEL
> -	 * RPC which goes to canceld portal, so we can cancel other LRU locks
> -	 * here and send them all as one LDLM_CANCEL RPC.
> -	 */
> -	LASSERT(list_empty(&lock->l_bl_ast));
> -	list_add(&lock->l_bl_ast, &cancels);
> -
> -	exp = lock->l_conn_export;
> -	if (exp_connect_cancelset(exp)) {
> -		avail = ldlm_format_handles_avail(class_exp2cliimp(exp),
> -						  &RQF_LDLM_CANCEL,
> -						  RCL_CLIENT, 0);
> -		LASSERT(avail > 0);
> -
> -		ns = ldlm_lock_to_ns(lock);
> -		flags = ns_connect_lru_resize(ns) ?
> -			LDLM_LRU_FLAG_LRUR : LDLM_LRU_FLAG_AGED;
> -		count += ldlm_cancel_lru_local(ns, &cancels, 0, avail - 1,
> -					       LCF_BL_AST, flags);
> -	}
> -	ldlm_cli_cancel_list(&cancels, count, NULL, cancel_flags);
> -	return 0;
> -}
> -EXPORT_SYMBOL(ldlm_cli_cancel);
> -
> -/**
> - * Locally cancel up to \a count locks in list \a cancels.
> - * Return the number of cancelled locks.
> - */
> -int ldlm_cli_cancel_list_local(struct list_head *cancels, int count,
> -			       enum ldlm_cancel_flags flags)
> -{
> -	LIST_HEAD(head);
> -	struct ldlm_lock *lock, *next;
> -	int left = 0, bl_ast = 0;
> -	__u64 rc;
> -
> -	left = count;
> -	list_for_each_entry_safe(lock, next, cancels, l_bl_ast) {
> -		if (left-- == 0)
> -			break;
> -
> -		if (flags & LCF_LOCAL) {
> -			rc = LDLM_FL_LOCAL_ONLY;
> -			ldlm_lock_cancel(lock);
> -		} else {
> -			rc = ldlm_cli_cancel_local(lock);
> -		}
> -		/* Until we have compound requests and can send LDLM_CANCEL
> -		 * requests batched with generic RPCs, we need to send cancels
> -		 * with the LDLM_FL_BL_AST flag in a separate RPC from
> -		 * the one being generated now.
> -		 */
> -		if (!(flags & LCF_BL_AST) && (rc == LDLM_FL_BL_AST)) {
> -			LDLM_DEBUG(lock, "Cancel lock separately");
> -			list_del_init(&lock->l_bl_ast);
> -			list_add(&lock->l_bl_ast, &head);
> -			bl_ast++;
> -			continue;
> -		}
> -		if (rc == LDLM_FL_LOCAL_ONLY) {
> -			/* CANCEL RPC should not be sent to server. */
> -			list_del_init(&lock->l_bl_ast);
> -			LDLM_LOCK_RELEASE(lock);
> -			count--;
> -		}
> -	}
> -	if (bl_ast > 0) {
> -		count -= bl_ast;
> -		ldlm_cli_cancel_list(&head, bl_ast, NULL, 0);
> -	}
> -
> -	return count;
> -}
> -
> -/**
> - * Cancel as many locks as possible w/o sending any RPCs (e.g. to write back
> - * dirty data, to close a file, ...) or waiting for any RPCs in-flight (e.g.
> - * readahead requests, ...)
> - */
> -static enum ldlm_policy_res
> -ldlm_cancel_no_wait_policy(struct ldlm_namespace *ns, struct ldlm_lock *lock,
> -			   int unused, int added, int count)
> -{
> -	enum ldlm_policy_res result = LDLM_POLICY_CANCEL_LOCK;
> -
> -	/* don't check added & count since we want to process all locks
> -	 * from unused list.
> -	 * It's fine to not take lock to access lock->l_resource since
> -	 * the lock has already been granted so it won't change.
> -	 */
> -	switch (lock->l_resource->lr_type) {
> -	case LDLM_EXTENT:
> -	case LDLM_IBITS:
> -		if (ns->ns_cancel && ns->ns_cancel(lock) != 0)
> -			break;
> -		/* fall through */
> -	default:
> -		result = LDLM_POLICY_SKIP_LOCK;
> -		lock_res_and_lock(lock);
> -		ldlm_set_skipped(lock);
> -		unlock_res_and_lock(lock);
> -		break;
> -	}
> -
> -	return result;
> -}
> -
> -/**
> - * Callback function for LRU-resize policy. Decides whether to keep
> - * \a lock in LRU for current \a LRU size \a unused, added in current
> - * scan \a added and number of locks to be preferably canceled \a count.
> - *
> - * \retval LDLM_POLICY_KEEP_LOCK keep lock in LRU in stop scanning
> - *
> - * \retval LDLM_POLICY_CANCEL_LOCK cancel lock from LRU
> - */
> -static enum ldlm_policy_res ldlm_cancel_lrur_policy(struct ldlm_namespace *ns,
> -						    struct ldlm_lock *lock,
> -						    int unused, int added,
> -						    int count)
> -{
> -	unsigned long cur = jiffies;
> -	struct ldlm_pool *pl = &ns->ns_pool;
> -	__u64 slv, lvf, lv;
> -	unsigned long la;
> -
> -	/* Stop LRU processing when we reach past @count or have checked all
> -	 * locks in LRU.
> -	 */
> -	if (count && added >= count)
> -		return LDLM_POLICY_KEEP_LOCK;
> -
> -	/*
> -	 * Despite of the LV, It doesn't make sense to keep the lock which
> -	 * is unused for ns_max_age time.
> -	 */
> -	if (time_after(jiffies, lock->l_last_used + ns->ns_max_age))
> -		return LDLM_POLICY_CANCEL_LOCK;
> -
> -	slv = ldlm_pool_get_slv(pl);
> -	lvf = ldlm_pool_get_lvf(pl);
> -	la = (cur - lock->l_last_used) / HZ;
> -	lv = lvf * la * unused;
> -
> -	/* Inform pool about current CLV to see it via debugfs. */
> -	ldlm_pool_set_clv(pl, lv);
> -
> -	/* Stop when SLV is not yet come from server or lv is smaller than
> -	 * it is.
> -	 */
> -	if (slv == 0 || lv < slv)
> -		return LDLM_POLICY_KEEP_LOCK;
> -
> -	return LDLM_POLICY_CANCEL_LOCK;
> -}
> -
> -/**
> - * Callback function for debugfs used policy. Makes decision whether to keep
> - * \a lock in LRU for current \a LRU size \a unused, added in current scan \a
> - * added and number of locks to be preferably canceled \a count.
> - *
> - * \retval LDLM_POLICY_KEEP_LOCK keep lock in LRU in stop scanning
> - *
> - * \retval LDLM_POLICY_CANCEL_LOCK cancel lock from LRU
> - */
> -static enum ldlm_policy_res ldlm_cancel_passed_policy(struct ldlm_namespace *ns,
> -						      struct ldlm_lock *lock,
> -						      int unused, int added,
> -						      int count)
> -{
> -	/* Stop LRU processing when we reach past @count or have checked all
> -	 * locks in LRU.
> -	 */
> -	return (added >= count) ?
> -		LDLM_POLICY_KEEP_LOCK : LDLM_POLICY_CANCEL_LOCK;
> -}
> -
> -/**
> - * Callback function for aged policy. Makes decision whether to keep \a lock in
> - * LRU for current LRU size \a unused, added in current scan \a added and
> - * number of locks to be preferably canceled \a count.
> - *
> - * \retval LDLM_POLICY_KEEP_LOCK keep lock in LRU in stop scanning
> - *
> - * \retval LDLM_POLICY_CANCEL_LOCK cancel lock from LRU
> - */
> -static enum ldlm_policy_res ldlm_cancel_aged_policy(struct ldlm_namespace *ns,
> -						    struct ldlm_lock *lock,
> -						    int unused, int added,
> -						    int count)
> -{
> -	if ((added >= count) &&
> -	    time_before(jiffies, lock->l_last_used + ns->ns_max_age))
> -		return LDLM_POLICY_KEEP_LOCK;
> -
> -	return LDLM_POLICY_CANCEL_LOCK;
> -}
> -
> -static enum ldlm_policy_res
> -ldlm_cancel_lrur_no_wait_policy(struct ldlm_namespace *ns,
> -				struct ldlm_lock *lock,
> -				int unused, int added,
> -				int count)
> -{
> -	enum ldlm_policy_res result;
> -
> -	result = ldlm_cancel_lrur_policy(ns, lock, unused, added, count);
> -	if (result == LDLM_POLICY_KEEP_LOCK)
> -		return result;
> -
> -	return ldlm_cancel_no_wait_policy(ns, lock, unused, added, count);
> -}
> -
> -/**
> - * Callback function for default policy. Makes decision whether to keep \a lock
> - * in LRU for current LRU size \a unused, added in current scan \a added and
> - * number of locks to be preferably canceled \a count.
> - *
> - * \retval LDLM_POLICY_KEEP_LOCK keep lock in LRU in stop scanning
> - *
> - * \retval LDLM_POLICY_CANCEL_LOCK cancel lock from LRU
> - */
> -static enum ldlm_policy_res
> -ldlm_cancel_default_policy(struct ldlm_namespace *ns, struct ldlm_lock *lock,
> -			   int unused, int added, int count)
> -{
> -	/* Stop LRU processing when we reach past count or have checked all
> -	 * locks in LRU.
> -	 */
> -	return (added >= count) ?
> -		LDLM_POLICY_KEEP_LOCK : LDLM_POLICY_CANCEL_LOCK;
> -}
> -
> -typedef enum ldlm_policy_res (*ldlm_cancel_lru_policy_t)(
> -						      struct ldlm_namespace *,
> -						      struct ldlm_lock *, int,
> -						      int, int);
> -
> -static ldlm_cancel_lru_policy_t
> -ldlm_cancel_lru_policy(struct ldlm_namespace *ns, int flags)
> -{
> -	if (flags & LDLM_LRU_FLAG_NO_WAIT)
> -		return ldlm_cancel_no_wait_policy;
> -
> -	if (ns_connect_lru_resize(ns)) {
> -		if (flags & LDLM_LRU_FLAG_SHRINK)
> -			/* We kill passed number of old locks. */
> -			return ldlm_cancel_passed_policy;
> -		else if (flags & LDLM_LRU_FLAG_LRUR)
> -			return ldlm_cancel_lrur_policy;
> -		else if (flags & LDLM_LRU_FLAG_PASSED)
> -			return ldlm_cancel_passed_policy;
> -		else if (flags & LDLM_LRU_FLAG_LRUR_NO_WAIT)
> -			return ldlm_cancel_lrur_no_wait_policy;
> -	} else {
> -		if (flags & LDLM_LRU_FLAG_AGED)
> -			return ldlm_cancel_aged_policy;
> -	}
> -
> -	return ldlm_cancel_default_policy;
> -}
> -
> -/**
> - * - Free space in LRU for \a count new locks,
> - *   redundant unused locks are canceled locally;
> - * - also cancel locally unused aged locks;
> - * - do not cancel more than \a max locks;
> - * - GET the found locks and add them into the \a cancels list.
> - *
> - * A client lock can be added to the l_bl_ast list only when it is
> - * marked LDLM_FL_CANCELING. Otherwise, somebody is already doing
> - * CANCEL.  There are the following use cases:
> - * ldlm_cancel_resource_local(), ldlm_cancel_lru_local() and
> - * ldlm_cli_cancel(), which check and set this flag properly. As any
> - * attempt to cancel a lock rely on this flag, l_bl_ast list is accessed
> - * later without any special locking.
> - *
> - * Calling policies for enabled LRU resize:
> - * ----------------------------------------
> - * flags & LDLM_LRU_FLAG_LRUR	- use LRU resize policy (SLV from server) to
> - *				  cancel not more than \a count locks;
> - *
> - * flags & LDLM_LRU_FLAG_PASSED - cancel \a count number of old locks (located
> - *				  at the beginning of LRU list);
> - *
> - * flags & LDLM_LRU_FLAG_SHRINK - cancel not more than \a count locks according
> - *				  to memory pressure policy function;
> - *
> - * flags & LDLM_LRU_FLAG_AGED   - cancel \a count locks according to
> - *				  "aged policy".
> - *
> - * flags & LDLM_LRU_FLAG_NO_WAIT - cancel as many unused locks as possible
> - *				   (typically before replaying locks) w/o
> - *				   sending any RPCs or waiting for any
> - *				   outstanding RPC to complete.
> - */
> -static int ldlm_prepare_lru_list(struct ldlm_namespace *ns,
> -				 struct list_head *cancels, int count, int max,
> -				 int flags)
> -{
> -	ldlm_cancel_lru_policy_t pf;
> -	struct ldlm_lock *lock, *next;
> -	int added = 0, unused, remained;
> -	int no_wait = flags &
> -		(LDLM_LRU_FLAG_NO_WAIT | LDLM_LRU_FLAG_LRUR_NO_WAIT);
> -
> -	spin_lock(&ns->ns_lock);
> -	unused = ns->ns_nr_unused;
> -	remained = unused;
> -
> -	if (!ns_connect_lru_resize(ns))
> -		count += unused - ns->ns_max_unused;
> -
> -	pf = ldlm_cancel_lru_policy(ns, flags);
> -	LASSERT(pf);
> -
> -	while (!list_empty(&ns->ns_unused_list)) {
> -		enum ldlm_policy_res result;
> -		time_t last_use = 0;
> -
> -		/* all unused locks */
> -		if (remained-- <= 0)
> -			break;
> -
> -		/* For any flags, stop scanning if @max is reached. */
> -		if (max && added >= max)
> -			break;
> -
> -		list_for_each_entry_safe(lock, next, &ns->ns_unused_list,
> -					 l_lru) {
> -			/* No locks which got blocking requests. */
> -			LASSERT(!ldlm_is_bl_ast(lock));
> -
> -			if (no_wait && ldlm_is_skipped(lock))
> -				/* already processed */
> -				continue;
> -
> -			last_use = lock->l_last_used;
> -			if (last_use == jiffies)
> -				continue;
> -
> -			/* Somebody is already doing CANCEL. No need for this
> -			 * lock in LRU, do not traverse it again.
> -			 */
> -			if (!ldlm_is_canceling(lock))
> -				break;
> -
> -			ldlm_lock_remove_from_lru_nolock(lock);
> -		}
> -		if (&lock->l_lru == &ns->ns_unused_list)
> -			break;
> -
> -		LDLM_LOCK_GET(lock);
> -		spin_unlock(&ns->ns_lock);
> -		lu_ref_add(&lock->l_reference, __func__, current);
> -
> -		/* Pass the lock through the policy filter and see if it
> -		 * should stay in LRU.
> -		 *
> -		 * Even for shrinker policy we stop scanning if
> -		 * we find a lock that should stay in the cache.
> -		 * We should take into account lock age anyway
> -		 * as a new lock is a valuable resource even if
> -		 * it has a low weight.
> -		 *
> -		 * That is, for shrinker policy we drop only
> -		 * old locks, but additionally choose them by
> -		 * their weight. Big extent locks will stay in
> -		 * the cache.
> -		 */
> -		result = pf(ns, lock, unused, added, count);
> -		if (result == LDLM_POLICY_KEEP_LOCK) {
> -			lu_ref_del(&lock->l_reference,
> -				   __func__, current);
> -			LDLM_LOCK_RELEASE(lock);
> -			spin_lock(&ns->ns_lock);
> -			break;
> -		}
> -		if (result == LDLM_POLICY_SKIP_LOCK) {
> -			lu_ref_del(&lock->l_reference,
> -				   __func__, current);
> -			LDLM_LOCK_RELEASE(lock);
> -			spin_lock(&ns->ns_lock);
> -			continue;
> -		}
> -
> -		lock_res_and_lock(lock);
> -		/* Check flags again under the lock. */
> -		if (ldlm_is_canceling(lock) ||
> -		    (ldlm_lock_remove_from_lru_check(lock, last_use) == 0)) {
> -			/* Another thread is removing lock from LRU, or
> -			 * somebody is already doing CANCEL, or there
> -			 * is a blocking request which will send cancel
> -			 * by itself, or the lock is no longer unused or
> -			 * the lock has been used since the pf() call and
> -			 * pages could be put under it.
> -			 */
> -			unlock_res_and_lock(lock);
> -			lu_ref_del(&lock->l_reference,
> -				   __func__, current);
> -			LDLM_LOCK_RELEASE(lock);
> -			spin_lock(&ns->ns_lock);
> -			continue;
> -		}
> -		LASSERT(!lock->l_readers && !lock->l_writers);
> -
> -		/* If we have chosen to cancel this lock voluntarily, we
> -		 * better send cancel notification to server, so that it
> -		 * frees appropriate state. This might lead to a race
> -		 * where while we are doing cancel here, server is also
> -		 * silently cancelling this lock.
> -		 */
> -		ldlm_clear_cancel_on_block(lock);
> -
> -		/* Setting the CBPENDING flag is a little misleading,
> -		 * but prevents an important race; namely, once
> -		 * CBPENDING is set, the lock can accumulate no more
> -		 * readers/writers. Since readers and writers are
> -		 * already zero here, ldlm_lock_decref() won't see
> -		 * this flag and call l_blocking_ast
> -		 */
> -		lock->l_flags |= LDLM_FL_CBPENDING | LDLM_FL_CANCELING;
> -
> -		/* We can't re-add to l_lru as it confuses the
> -		 * refcounting in ldlm_lock_remove_from_lru() if an AST
> -		 * arrives after we drop lr_lock below. We use l_bl_ast
> -		 * and can't use l_pending_chain as it is used both on
> -		 * server and client nevertheless bug 5666 says it is
> -		 * used only on server
> -		 */
> -		LASSERT(list_empty(&lock->l_bl_ast));
> -		list_add(&lock->l_bl_ast, cancels);
> -		unlock_res_and_lock(lock);
> -		lu_ref_del(&lock->l_reference, __func__, current);
> -		spin_lock(&ns->ns_lock);
> -		added++;
> -		unused--;
> -	}
> -	spin_unlock(&ns->ns_lock);
> -	return added;
> -}
> -
> -int ldlm_cancel_lru_local(struct ldlm_namespace *ns,
> -			  struct list_head *cancels, int count, int max,
> -			  enum ldlm_cancel_flags cancel_flags, int flags)
> -{
> -	int added;
> -
> -	added = ldlm_prepare_lru_list(ns, cancels, count, max, flags);
> -	if (added <= 0)
> -		return added;
> -	return ldlm_cli_cancel_list_local(cancels, added, cancel_flags);
> -}
> -
> -/**
> - * Cancel at least \a nr locks from given namespace LRU.
> - *
> - * When called with LCF_ASYNC the blocking callback will be handled
> - * in a thread and this function will return after the thread has been
> - * asked to call the callback.  When called with LCF_ASYNC the blocking
> - * callback will be performed in this function.
> - */
> -int ldlm_cancel_lru(struct ldlm_namespace *ns, int nr,
> -		    enum ldlm_cancel_flags cancel_flags,
> -		    int flags)
> -{
> -	LIST_HEAD(cancels);
> -	int count, rc;
> -
> -	/* Just prepare the list of locks, do not actually cancel them yet.
> -	 * Locks are cancelled later in a separate thread.
> -	 */
> -	count = ldlm_prepare_lru_list(ns, &cancels, nr, 0, flags);
> -	rc = ldlm_bl_to_thread_list(ns, NULL, &cancels, count, cancel_flags);
> -	if (rc == 0)
> -		return count;
> -
> -	return 0;
> -}
> -
> -/**
> - * Find and cancel locally unused locks found on resource, matched to the
> - * given policy, mode. GET the found locks and add them into the \a cancels
> - * list.
> - */
> -int ldlm_cancel_resource_local(struct ldlm_resource *res,
> -			       struct list_head *cancels,
> -			       union ldlm_policy_data *policy,
> -			       enum ldlm_mode mode, __u64 lock_flags,
> -			       enum ldlm_cancel_flags cancel_flags,
> -			       void *opaque)
> -{
> -	struct ldlm_lock *lock;
> -	int count = 0;
> -
> -	lock_res(res);
> -	list_for_each_entry(lock, &res->lr_granted, l_res_link) {
> -		if (opaque && lock->l_ast_data != opaque) {
> -			LDLM_ERROR(lock, "data %p doesn't match opaque %p",
> -				   lock->l_ast_data, opaque);
> -			continue;
> -		}
> -
> -		if (lock->l_readers || lock->l_writers)
> -			continue;
> -
> -		/* If somebody is already doing CANCEL, or blocking AST came,
> -		 * skip this lock.
> -		 */
> -		if (ldlm_is_bl_ast(lock) || ldlm_is_canceling(lock))
> -			continue;
> -
> -		if (lockmode_compat(lock->l_granted_mode, mode))
> -			continue;
> -
> -		/* If policy is given and this is IBITS lock, add to list only
> -		 * those locks that match by policy.
> -		 */
> -		if (policy && (lock->l_resource->lr_type == LDLM_IBITS) &&
> -		    !(lock->l_policy_data.l_inodebits.bits &
> -		      policy->l_inodebits.bits))
> -			continue;
> -
> -		/* See CBPENDING comment in ldlm_cancel_lru */
> -		lock->l_flags |= LDLM_FL_CBPENDING | LDLM_FL_CANCELING |
> -				 lock_flags;
> -
> -		LASSERT(list_empty(&lock->l_bl_ast));
> -		list_add(&lock->l_bl_ast, cancels);
> -		LDLM_LOCK_GET(lock);
> -		count++;
> -	}
> -	unlock_res(res);
> -
> -	return ldlm_cli_cancel_list_local(cancels, count, cancel_flags);
> -}
> -EXPORT_SYMBOL(ldlm_cancel_resource_local);
> -
> -/**
> - * Cancel client-side locks from a list and send/prepare cancel RPCs to the
> - * server.
> - * If \a req is NULL, send CANCEL request to server with handles of locks
> - * in the \a cancels. If EARLY_CANCEL is not supported, send CANCEL requests
> - * separately per lock.
> - * If \a req is not NULL, put handles of locks in \a cancels into the request
> - * buffer at the offset \a off.
> - * Destroy \a cancels at the end.
> - */
> -int ldlm_cli_cancel_list(struct list_head *cancels, int count,
> -			 struct ptlrpc_request *req,
> -			 enum ldlm_cancel_flags flags)
> -{
> -	struct ldlm_lock *lock;
> -	int res = 0;
> -
> -	if (list_empty(cancels) || count == 0)
> -		return 0;
> -
> -	/* XXX: requests (both batched and not) could be sent in parallel.
> -	 * Usually it is enough to have just 1 RPC, but it is possible that
> -	 * there are too many locks to be cancelled in LRU or on a resource.
> -	 * It would also speed up the case when the server does not support
> -	 * the feature.
> -	 */
> -	while (count > 0) {
> -		LASSERT(!list_empty(cancels));
> -		lock = list_first_entry(cancels, struct ldlm_lock, l_bl_ast);
> -		LASSERT(lock->l_conn_export);
> -
> -		if (exp_connect_cancelset(lock->l_conn_export)) {
> -			res = count;
> -			if (req)
> -				ldlm_cancel_pack(req, cancels, count);
> -			else
> -				res = ldlm_cli_cancel_req(lock->l_conn_export,
> -							  cancels, count,
> -							  flags);
> -		} else {
> -			res = ldlm_cli_cancel_req(lock->l_conn_export,
> -						  cancels, 1, flags);
> -		}
> -
> -		if (res < 0) {
> -			CDEBUG_LIMIT(res == -ESHUTDOWN ? D_DLMTRACE : D_ERROR,
> -				     "%s: %d\n", __func__, res);
> -			res = count;
> -		}
> -
> -		count -= res;
> -		ldlm_lock_list_put(cancels, l_bl_ast, res);
> -	}
> -	LASSERT(count == 0);
> -	return 0;
> -}
> -EXPORT_SYMBOL(ldlm_cli_cancel_list);
> -
> -/**
> - * Cancel all locks on a resource that have 0 readers/writers.
> - *
> - * If flags & LDLM_FL_LOCAL_ONLY, throw the locks away without trying
> - * to notify the server.
> - */
> -int ldlm_cli_cancel_unused_resource(struct ldlm_namespace *ns,
> -				    const struct ldlm_res_id *res_id,
> -				    union ldlm_policy_data *policy,
> -				    enum ldlm_mode mode,
> -				    enum ldlm_cancel_flags flags,
> -				    void *opaque)
> -{
> -	struct ldlm_resource *res;
> -	LIST_HEAD(cancels);
> -	int count;
> -	int rc;
> -
> -	res = ldlm_resource_get(ns, NULL, res_id, 0, 0);
> -	if (IS_ERR(res)) {
> -		/* This is not a problem. */
> -		CDEBUG(D_INFO, "No resource %llu\n", res_id->name[0]);
> -		return 0;
> -	}
> -
> -	LDLM_RESOURCE_ADDREF(res);
> -	count = ldlm_cancel_resource_local(res, &cancels, policy, mode,
> -					   0, flags | LCF_BL_AST, opaque);
> -	rc = ldlm_cli_cancel_list(&cancels, count, NULL, flags);
> -	if (rc != ELDLM_OK)
> -		CERROR("canceling unused lock " DLDLMRES ": rc = %d\n",
> -		       PLDLMRES(res), rc);
> -
> -	LDLM_RESOURCE_DELREF(res);
> -	ldlm_resource_putref(res);
> -	return 0;
> -}
> -EXPORT_SYMBOL(ldlm_cli_cancel_unused_resource);
> -
> -struct ldlm_cli_cancel_arg {
> -	int     lc_flags;
> -	void   *lc_opaque;
> -};
> -
> -static int ldlm_cli_hash_cancel_unused(struct cfs_hash *hs,
> -				       struct cfs_hash_bd *bd,
> -				       struct hlist_node *hnode, void *arg)
> -{
> -	struct ldlm_resource	   *res = cfs_hash_object(hs, hnode);
> -	struct ldlm_cli_cancel_arg     *lc = arg;
> -
> -	ldlm_cli_cancel_unused_resource(ldlm_res_to_ns(res), &res->lr_name,
> -					NULL, LCK_MINMODE,
> -					lc->lc_flags, lc->lc_opaque);
> -	/* must return 0 for hash iteration */
> -	return 0;
> -}
> -
> -/**
> - * Cancel all locks on a namespace (or a specific resource, if given)
> - * that have 0 readers/writers.
> - *
> - * If flags & LCF_LOCAL, throw the locks away without trying
> - * to notify the server.
> - */
> -int ldlm_cli_cancel_unused(struct ldlm_namespace *ns,
> -			   const struct ldlm_res_id *res_id,
> -			   enum ldlm_cancel_flags flags, void *opaque)
> -{
> -	struct ldlm_cli_cancel_arg arg = {
> -		.lc_flags       = flags,
> -		.lc_opaque      = opaque,
> -	};
> -
> -	if (!ns)
> -		return ELDLM_OK;
> -
> -	if (res_id) {
> -		return ldlm_cli_cancel_unused_resource(ns, res_id, NULL,
> -						       LCK_MINMODE, flags,
> -						       opaque);
> -	} else {
> -		cfs_hash_for_each_nolock(ns->ns_rs_hash,
> -					 ldlm_cli_hash_cancel_unused, &arg, 0);
> -		return ELDLM_OK;
> -	}
> -}
> -EXPORT_SYMBOL(ldlm_cli_cancel_unused);
> -
> -/* Lock iterators. */
> -
> -static int ldlm_resource_foreach(struct ldlm_resource *res,
> -				 ldlm_iterator_t iter, void *closure)
> -{
> -	struct ldlm_lock *tmp;
> -	struct ldlm_lock *lock;
> -	int rc = LDLM_ITER_CONTINUE;
> -
> -	if (!res)
> -		return LDLM_ITER_CONTINUE;
> -
> -	lock_res(res);
> -	list_for_each_entry_safe(lock, tmp, &res->lr_granted, l_res_link) {
> -		if (iter(lock, closure) == LDLM_ITER_STOP) {
> -			rc = LDLM_ITER_STOP;
> -			goto out;
> -		}
> -	}
> -
> -	list_for_each_entry_safe(lock, tmp, &res->lr_waiting, l_res_link) {
> -		if (iter(lock, closure) == LDLM_ITER_STOP) {
> -			rc = LDLM_ITER_STOP;
> -			goto out;
> -		}
> -	}
> - out:
> -	unlock_res(res);
> -	return rc;
> -}
> -
> -struct iter_helper_data {
> -	ldlm_iterator_t iter;
> -	void *closure;
> -};
> -
> -static int ldlm_iter_helper(struct ldlm_lock *lock, void *closure)
> -{
> -	struct iter_helper_data *helper = closure;
> -
> -	return helper->iter(lock, helper->closure);
> -}
> -
> -static int ldlm_res_iter_helper(struct cfs_hash *hs, struct cfs_hash_bd *bd,
> -				struct hlist_node *hnode, void *arg)
> -
> -{
> -	struct ldlm_resource *res = cfs_hash_object(hs, hnode);
> -
> -	return ldlm_resource_foreach(res, ldlm_iter_helper, arg) ==
> -	       LDLM_ITER_STOP;
> -}
> -
> -static void ldlm_namespace_foreach(struct ldlm_namespace *ns,
> -				   ldlm_iterator_t iter, void *closure)
> -
> -{
> -	struct iter_helper_data helper = {
> -		.iter		= iter,
> -		.closure	= closure,
> -	};
> -
> -	cfs_hash_for_each_nolock(ns->ns_rs_hash,
> -				 ldlm_res_iter_helper, &helper, 0);
> -}
> -
> -/* non-blocking function to manipulate a lock whose cb_data is being put away.
> - * return  0:  find no resource
> - *       > 0:  must be LDLM_ITER_STOP/LDLM_ITER_CONTINUE.
> - *       < 0:  errors
> - */
> -int ldlm_resource_iterate(struct ldlm_namespace *ns,
> -			  const struct ldlm_res_id *res_id,
> -			  ldlm_iterator_t iter, void *data)
> -{
> -	struct ldlm_resource *res;
> -	int rc;
> -
> -	LASSERTF(ns, "must pass in namespace\n");
> -
> -	res = ldlm_resource_get(ns, NULL, res_id, 0, 0);
> -	if (IS_ERR(res))
> -		return 0;
> -
> -	LDLM_RESOURCE_ADDREF(res);
> -	rc = ldlm_resource_foreach(res, iter, data);
> -	LDLM_RESOURCE_DELREF(res);
> -	ldlm_resource_putref(res);
> -	return rc;
> -}
> -EXPORT_SYMBOL(ldlm_resource_iterate);
> -
> -/* Lock replay */
> -
> -static int ldlm_chain_lock_for_replay(struct ldlm_lock *lock, void *closure)
> -{
> -	struct list_head *list = closure;
> -
> -	/* we use l_pending_chain here, because it's unused on clients. */
> -	LASSERTF(list_empty(&lock->l_pending_chain),
> -		 "lock %p next %p prev %p\n",
> -		 lock, &lock->l_pending_chain.next,
> -		 &lock->l_pending_chain.prev);
> -	/* bug 9573: don't replay locks left after eviction, or
> -	 * bug 17614: locks being actively cancelled. Get a reference
> -	 * on a lock so that it does not disappear under us (e.g. due to cancel)
> -	 */
> -	if (!(lock->l_flags & (LDLM_FL_FAILED | LDLM_FL_BL_DONE))) {
> -		list_add(&lock->l_pending_chain, list);
> -		LDLM_LOCK_GET(lock);
> -	}
> -
> -	return LDLM_ITER_CONTINUE;
> -}
> -
> -static int replay_lock_interpret(const struct lu_env *env,
> -				 struct ptlrpc_request *req,
> -				 struct ldlm_async_args *aa, int rc)
> -{
> -	struct ldlm_lock     *lock;
> -	struct ldlm_reply    *reply;
> -	struct obd_export    *exp;
> -
> -	atomic_dec(&req->rq_import->imp_replay_inflight);
> -	if (rc != ELDLM_OK)
> -		goto out;
> -
> -	reply = req_capsule_server_get(&req->rq_pill, &RMF_DLM_REP);
> -	if (!reply) {
> -		rc = -EPROTO;
> -		goto out;
> -	}
> -
> -	lock = ldlm_handle2lock(&aa->lock_handle);
> -	if (!lock) {
> -		CERROR("received replay ack for unknown local cookie %#llx remote cookie %#llx from server %s id %s\n",
> -		       aa->lock_handle.cookie, reply->lock_handle.cookie,
> -		       req->rq_export->exp_client_uuid.uuid,
> -		       libcfs_id2str(req->rq_peer));
> -		rc = -ESTALE;
> -		goto out;
> -	}
> -
> -	/* Key change rehash lock in per-export hash with new key */
> -	exp = req->rq_export;
> -	lock->l_remote_handle = reply->lock_handle;
> -
> -	LDLM_DEBUG(lock, "replayed lock:");
> -	ptlrpc_import_recovery_state_machine(req->rq_import);
> -	LDLM_LOCK_PUT(lock);
> -out:
> -	if (rc != ELDLM_OK)
> -		ptlrpc_connect_import(req->rq_import);
> -
> -	return rc;
> -}
> -
> -static int replay_one_lock(struct obd_import *imp, struct ldlm_lock *lock)
> -{
> -	struct ptlrpc_request *req;
> -	struct ldlm_async_args *aa;
> -	struct ldlm_request   *body;
> -	int flags;
> -
> -	/* Bug 11974: Do not replay a lock which is actively being canceled */
> -	if (ldlm_is_bl_done(lock)) {
> -		LDLM_DEBUG(lock, "Not replaying canceled lock:");
> -		return 0;
> -	}
> -
> -	/* If this is reply-less callback lock, we cannot replay it, since
> -	 * server might have long dropped it, but notification of that event was
> -	 * lost by network. (and server granted conflicting lock already)
> -	 */
> -	if (ldlm_is_cancel_on_block(lock)) {
> -		LDLM_DEBUG(lock, "Not replaying reply-less lock:");
> -		ldlm_lock_cancel(lock);
> -		return 0;
> -	}
> -
> -	/*
> -	 * If granted mode matches the requested mode, this lock is granted.
> -	 *
> -	 * If they differ, but we have a granted mode, then we were granted
> -	 * one mode and now want another: ergo, converting.
> -	 *
> -	 * If we haven't been granted anything and are on a resource list,
> -	 * then we're blocked/waiting.
> -	 *
> -	 * If we haven't been granted anything and we're NOT on a resource list,
> -	 * then we haven't got a reply yet and don't have a known disposition.
> -	 * This happens whenever a lock enqueue is the request that triggers
> -	 * recovery.
> -	 */
> -	if (lock->l_granted_mode == lock->l_req_mode)
> -		flags = LDLM_FL_REPLAY | LDLM_FL_BLOCK_GRANTED;
> -	else if (lock->l_granted_mode)
> -		flags = LDLM_FL_REPLAY | LDLM_FL_BLOCK_CONV;
> -	else if (!list_empty(&lock->l_res_link))
> -		flags = LDLM_FL_REPLAY | LDLM_FL_BLOCK_WAIT;
> -	else
> -		flags = LDLM_FL_REPLAY;
> -
> -	req = ptlrpc_request_alloc_pack(imp, &RQF_LDLM_ENQUEUE,
> -					LUSTRE_DLM_VERSION, LDLM_ENQUEUE);
> -	if (!req)
> -		return -ENOMEM;
> -
> -	/* We're part of recovery, so don't wait for it. */
> -	req->rq_send_state = LUSTRE_IMP_REPLAY_LOCKS;
> -
> -	body = req_capsule_client_get(&req->rq_pill, &RMF_DLM_REQ);
> -	ldlm_lock2desc(lock, &body->lock_desc);
> -	body->lock_flags = ldlm_flags_to_wire(flags);
> -
> -	ldlm_lock2handle(lock, &body->lock_handle[0]);
> -	if (lock->l_lvb_len > 0)
> -		req_capsule_extend(&req->rq_pill, &RQF_LDLM_ENQUEUE_LVB);
> -	req_capsule_set_size(&req->rq_pill, &RMF_DLM_LVB, RCL_SERVER,
> -			     lock->l_lvb_len);
> -	ptlrpc_request_set_replen(req);
> -	/* notify the server we've replayed all requests.
> -	 * also, we mark the request to be put on a dedicated
> -	 * queue to be processed after all request replayes.
> -	 * bug 6063
> -	 */
> -	lustre_msg_set_flags(req->rq_reqmsg, MSG_REQ_REPLAY_DONE);
> -
> -	LDLM_DEBUG(lock, "replaying lock:");
> -
> -	atomic_inc(&req->rq_import->imp_replay_inflight);
> -	BUILD_BUG_ON(sizeof(*aa) > sizeof(req->rq_async_args));
> -	aa = ptlrpc_req_async_args(req);
> -	aa->lock_handle = body->lock_handle[0];
> -	req->rq_interpret_reply = (ptlrpc_interpterer_t)replay_lock_interpret;
> -	ptlrpcd_add_req(req);
> -
> -	return 0;
> -}
> -
> -/**
> - * Cancel as many unused locks as possible before replay. since we are
> - * in recovery, we can't wait for any outstanding RPCs to send any RPC
> - * to the server.
> - *
> - * Called only in recovery before replaying locks. there is no need to
> - * replay locks that are unused. since the clients may hold thousands of
> - * cached unused locks, dropping the unused locks can greatly reduce the
> - * load on the servers at recovery time.
> - */
> -static void ldlm_cancel_unused_locks_for_replay(struct ldlm_namespace *ns)
> -{
> -	int canceled;
> -	LIST_HEAD(cancels);
> -
> -	CDEBUG(D_DLMTRACE,
> -	       "Dropping as many unused locks as possible before replay for namespace %s (%d)\n",
> -	       ldlm_ns_name(ns), ns->ns_nr_unused);
> -
> -	/* We don't need to care whether or not LRU resize is enabled
> -	 * because the LDLM_LRU_FLAG_NO_WAIT policy doesn't use the
> -	 * count parameter
> -	 */
> -	canceled = ldlm_cancel_lru_local(ns, &cancels, ns->ns_nr_unused, 0,
> -					 LCF_LOCAL, LDLM_LRU_FLAG_NO_WAIT);
> -
> -	CDEBUG(D_DLMTRACE, "Canceled %d unused locks from namespace %s\n",
> -	       canceled, ldlm_ns_name(ns));
> -}
> -
> -int ldlm_replay_locks(struct obd_import *imp)
> -{
> -	struct ldlm_namespace *ns = imp->imp_obd->obd_namespace;
> -	LIST_HEAD(list);
> -	struct ldlm_lock *lock, *next;
> -	int rc = 0;
> -
> -	LASSERT(atomic_read(&imp->imp_replay_inflight) == 0);
> -
> -	/* don't replay locks if import failed recovery */
> -	if (imp->imp_vbr_failed)
> -		return 0;
> -
> -	/* ensure this doesn't fall to 0 before all have been queued */
> -	atomic_inc(&imp->imp_replay_inflight);
> -
> -	if (ldlm_cancel_unused_locks_before_replay)
> -		ldlm_cancel_unused_locks_for_replay(ns);
> -
> -	ldlm_namespace_foreach(ns, ldlm_chain_lock_for_replay, &list);
> -
> -	list_for_each_entry_safe(lock, next, &list, l_pending_chain) {
> -		list_del_init(&lock->l_pending_chain);
> -		if (rc) {
> -			LDLM_LOCK_RELEASE(lock);
> -			continue; /* or try to do the rest? */
> -		}
> -		rc = replay_one_lock(imp, lock);
> -		LDLM_LOCK_RELEASE(lock);
> -	}
> -
> -	atomic_dec(&imp->imp_replay_inflight);
> -
> -	return rc;
> -}
> diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_resource.c b/drivers/staging/lustre/lustre/ldlm/ldlm_resource.c
> deleted file mode 100644
> index 3946d62ff009..000000000000
> --- a/drivers/staging/lustre/lustre/ldlm/ldlm_resource.c
> +++ /dev/null
> @@ -1,1318 +0,0 @@
> -// SPDX-License-Identifier: GPL-2.0
> -/*
> - * GPL HEADER START
> - *
> - * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
> - *
> - * This program is free software; you can redistribute it and/or modify
> - * it under the terms of the GNU General Public License version 2 only,
> - * as published by the Free Software Foundation.
> - *
> - * This program is distributed in the hope that it will be useful, but
> - * WITHOUT ANY WARRANTY; without even the implied warranty of
> - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> - * General Public License version 2 for more details (a copy is included
> - * in the LICENSE file that accompanied this code).
> - *
> - * You should have received a copy of the GNU General Public License
> - * version 2 along with this program; If not, see
> - * http://www.gnu.org/licenses/gpl-2.0.html
> - *
> - * GPL HEADER END
> - */
> -/*
> - * Copyright (c) 2002, 2010, Oracle and/or its affiliates. All rights reserved.
> - * Use is subject to license terms.
> - *
> - * Copyright (c) 2010, 2015, Intel Corporation.
> - */
> -/*
> - * This file is part of Lustre, http://www.lustre.org/
> - * Lustre is a trademark of Sun Microsystems, Inc.
> - *
> - * lustre/ldlm/ldlm_resource.c
> - *
> - * Author: Phil Schwan <phil at clusterfs.com>
> - * Author: Peter Braam <braam at clusterfs.com>
> - */
> -
> -#define DEBUG_SUBSYSTEM S_LDLM
> -#include <lustre_dlm.h>
> -#include <lustre_fid.h>
> -#include <obd_class.h>
> -#include "ldlm_internal.h"
> -#include <linux/libcfs/libcfs_hash.h>
> -
> -struct kmem_cache *ldlm_resource_slab, *ldlm_lock_slab;
> -
> -int ldlm_srv_namespace_nr;
> -int ldlm_cli_namespace_nr;
> -
> -struct mutex ldlm_srv_namespace_lock;
> -LIST_HEAD(ldlm_srv_namespace_list);
> -
> -struct mutex ldlm_cli_namespace_lock;
> -/* Client Namespaces that have active resources in them.
> - * Once all resources go away, ldlm_poold moves such namespaces to the
> - * inactive list
> - */
> -LIST_HEAD(ldlm_cli_active_namespace_list);
> -/* Client namespaces that don't have any locks in them */
> -static LIST_HEAD(ldlm_cli_inactive_namespace_list);
> -
> -static struct dentry *ldlm_debugfs_dir;
> -static struct dentry *ldlm_ns_debugfs_dir;
> -struct dentry *ldlm_svc_debugfs_dir;
> -
> -/* during debug dump certain amount of granted locks for one resource to avoid
> - * DDOS.
> - */
> -static unsigned int ldlm_dump_granted_max = 256;
> -
> -static ssize_t
> -lprocfs_wr_dump_ns(struct file *file, const char __user *buffer,
> -		   size_t count, loff_t *off)
> -{
> -	ldlm_dump_all_namespaces(LDLM_NAMESPACE_SERVER, D_DLMTRACE);
> -	ldlm_dump_all_namespaces(LDLM_NAMESPACE_CLIENT, D_DLMTRACE);
> -	return count;
> -}
> -
> -LPROC_SEQ_FOPS_WR_ONLY(ldlm, dump_ns);
> -
> -static int ldlm_rw_uint_seq_show(struct seq_file *m, void *v)
> -{
> -	seq_printf(m, "%u\n", *(unsigned int *)m->private);
> -	return 0;
> -}
> -
> -static ssize_t
> -ldlm_rw_uint_seq_write(struct file *file, const char __user *buffer,
> -		       size_t count, loff_t *off)
> -{
> -	struct seq_file *seq = file->private_data;
> -
> -	if (count == 0)
> -		return 0;
> -	return kstrtouint_from_user(buffer, count, 0,
> -				    (unsigned int *)seq->private);
> -}
> -
> -LPROC_SEQ_FOPS(ldlm_rw_uint);
> -
> -static struct lprocfs_vars ldlm_debugfs_list[] = {
> -	{ "dump_namespaces", &ldlm_dump_ns_fops, NULL, 0222 },
> -	{ "dump_granted_max", &ldlm_rw_uint_fops, &ldlm_dump_granted_max },
> -	{ NULL }
> -};
> -
> -void ldlm_debugfs_setup(void)
> -{
> -	ldlm_debugfs_dir = debugfs_create_dir(OBD_LDLM_DEVICENAME,
> -					      debugfs_lustre_root);
> -
> -	ldlm_ns_debugfs_dir = debugfs_create_dir("namespaces",
> -						 ldlm_debugfs_dir);
> -
> -	ldlm_svc_debugfs_dir = debugfs_create_dir("services", ldlm_debugfs_dir);
> -
> -	ldebugfs_add_vars(ldlm_debugfs_dir, ldlm_debugfs_list, NULL);
> -}
> -
> -void ldlm_debugfs_cleanup(void)
> -{
> -	debugfs_remove_recursive(ldlm_svc_debugfs_dir);
> -	debugfs_remove_recursive(ldlm_ns_debugfs_dir);
> -	debugfs_remove_recursive(ldlm_debugfs_dir);
> -}
> -
> -static ssize_t resource_count_show(struct kobject *kobj, struct attribute *attr,
> -				   char *buf)
> -{
> -	struct ldlm_namespace *ns = container_of(kobj, struct ldlm_namespace,
> -						 ns_kobj);
> -	__u64		  res = 0;
> -	struct cfs_hash_bd	  bd;
> -	int		    i;
> -
> -	/* result is not strictly consistent */
> -	cfs_hash_for_each_bucket(ns->ns_rs_hash, &bd, i)
> -		res += cfs_hash_bd_count_get(&bd);
> -	return sprintf(buf, "%lld\n", res);
> -}
> -LUSTRE_RO_ATTR(resource_count);
> -
> -static ssize_t lock_count_show(struct kobject *kobj, struct attribute *attr,
> -			       char *buf)
> -{
> -	struct ldlm_namespace *ns = container_of(kobj, struct ldlm_namespace,
> -						 ns_kobj);
> -	__u64		  locks;
> -
> -	locks = lprocfs_stats_collector(ns->ns_stats, LDLM_NSS_LOCKS,
> -					LPROCFS_FIELDS_FLAGS_SUM);
> -	return sprintf(buf, "%lld\n", locks);
> -}
> -LUSTRE_RO_ATTR(lock_count);
> -
> -static ssize_t lock_unused_count_show(struct kobject *kobj,
> -				      struct attribute *attr,
> -				      char *buf)
> -{
> -	struct ldlm_namespace *ns = container_of(kobj, struct ldlm_namespace,
> -						 ns_kobj);
> -
> -	return sprintf(buf, "%d\n", ns->ns_nr_unused);
> -}
> -LUSTRE_RO_ATTR(lock_unused_count);
> -
> -static ssize_t lru_size_show(struct kobject *kobj, struct attribute *attr,
> -			     char *buf)
> -{
> -	struct ldlm_namespace *ns = container_of(kobj, struct ldlm_namespace,
> -						 ns_kobj);
> -	__u32 *nr = &ns->ns_max_unused;
> -
> -	if (ns_connect_lru_resize(ns))
> -		nr = &ns->ns_nr_unused;
> -	return sprintf(buf, "%u\n", *nr);
> -}
> -
> -static ssize_t lru_size_store(struct kobject *kobj, struct attribute *attr,
> -			      const char *buffer, size_t count)
> -{
> -	struct ldlm_namespace *ns = container_of(kobj, struct ldlm_namespace,
> -						 ns_kobj);
> -	unsigned long tmp;
> -	int lru_resize;
> -	int err;
> -
> -	if (strncmp(buffer, "clear", 5) == 0) {
> -		CDEBUG(D_DLMTRACE,
> -		       "dropping all unused locks from namespace %s\n",
> -		       ldlm_ns_name(ns));
> -		if (ns_connect_lru_resize(ns)) {
> -			int canceled, unused  = ns->ns_nr_unused;
> -
> -			/* Try to cancel all @ns_nr_unused locks. */
> -			canceled = ldlm_cancel_lru(ns, unused, 0,
> -						   LDLM_LRU_FLAG_PASSED);
> -			if (canceled < unused) {
> -				CDEBUG(D_DLMTRACE,
> -				       "not all requested locks are canceled, requested: %d, canceled: %d\n",
> -				       unused,
> -				       canceled);
> -				return -EINVAL;
> -			}
> -		} else {
> -			tmp = ns->ns_max_unused;
> -			ns->ns_max_unused = 0;
> -			ldlm_cancel_lru(ns, 0, 0, LDLM_LRU_FLAG_PASSED);
> -			ns->ns_max_unused = tmp;
> -		}
> -		return count;
> -	}
> -
> -	err = kstrtoul(buffer, 10, &tmp);
> -	if (err != 0) {
> -		CERROR("lru_size: invalid value written\n");
> -		return -EINVAL;
> -	}
> -	lru_resize = (tmp == 0);
> -
> -	if (ns_connect_lru_resize(ns)) {
> -		if (!lru_resize)
> -			ns->ns_max_unused = (unsigned int)tmp;
> -
> -		if (tmp > ns->ns_nr_unused)
> -			tmp = ns->ns_nr_unused;
> -		tmp = ns->ns_nr_unused - tmp;
> -
> -		CDEBUG(D_DLMTRACE,
> -		       "changing namespace %s unused locks from %u to %u\n",
> -		       ldlm_ns_name(ns), ns->ns_nr_unused,
> -		       (unsigned int)tmp);
> -		ldlm_cancel_lru(ns, tmp, LCF_ASYNC, LDLM_LRU_FLAG_PASSED);
> -
> -		if (!lru_resize) {
> -			CDEBUG(D_DLMTRACE,
> -			       "disable lru_resize for namespace %s\n",
> -			       ldlm_ns_name(ns));
> -			ns->ns_connect_flags &= ~OBD_CONNECT_LRU_RESIZE;
> -		}
> -	} else {
> -		CDEBUG(D_DLMTRACE,
> -		       "changing namespace %s max_unused from %u to %u\n",
> -		       ldlm_ns_name(ns), ns->ns_max_unused,
> -		       (unsigned int)tmp);
> -		ns->ns_max_unused = (unsigned int)tmp;
> -		ldlm_cancel_lru(ns, 0, LCF_ASYNC, LDLM_LRU_FLAG_PASSED);
> -
> -		/* Make sure that LRU resize was originally supported before
> -		 * turning it on here.
> -		 */
> -		if (lru_resize &&
> -		    (ns->ns_orig_connect_flags & OBD_CONNECT_LRU_RESIZE)) {
> -			CDEBUG(D_DLMTRACE,
> -			       "enable lru_resize for namespace %s\n",
> -			       ldlm_ns_name(ns));
> -			ns->ns_connect_flags |= OBD_CONNECT_LRU_RESIZE;
> -		}
> -	}
> -
> -	return count;
> -}
> -LUSTRE_RW_ATTR(lru_size);
> -
> -static ssize_t lru_max_age_show(struct kobject *kobj, struct attribute *attr,
> -				char *buf)
> -{
> -	struct ldlm_namespace *ns = container_of(kobj, struct ldlm_namespace,
> -						 ns_kobj);
> -
> -	return sprintf(buf, "%u\n", ns->ns_max_age);
> -}
> -
> -static ssize_t lru_max_age_store(struct kobject *kobj, struct attribute *attr,
> -				 const char *buffer, size_t count)
> -{
> -	struct ldlm_namespace *ns = container_of(kobj, struct ldlm_namespace,
> -						 ns_kobj);
> -	unsigned long tmp;
> -	int err;
> -
> -	err = kstrtoul(buffer, 10, &tmp);
> -	if (err != 0)
> -		return -EINVAL;
> -
> -	ns->ns_max_age = tmp;
> -
> -	return count;
> -}
> -LUSTRE_RW_ATTR(lru_max_age);
> -
> -static ssize_t early_lock_cancel_show(struct kobject *kobj,
> -				      struct attribute *attr,
> -				      char *buf)
> -{
> -	struct ldlm_namespace *ns = container_of(kobj, struct ldlm_namespace,
> -						 ns_kobj);
> -
> -	return sprintf(buf, "%d\n", ns_connect_cancelset(ns));
> -}
> -
> -static ssize_t early_lock_cancel_store(struct kobject *kobj,
> -				       struct attribute *attr,
> -				       const char *buffer,
> -				       size_t count)
> -{
> -	struct ldlm_namespace *ns = container_of(kobj, struct ldlm_namespace,
> -						 ns_kobj);
> -	unsigned long supp = -1;
> -	int rc;
> -
> -	rc = kstrtoul(buffer, 10, &supp);
> -	if (rc < 0)
> -		return rc;
> -
> -	if (supp == 0)
> -		ns->ns_connect_flags &= ~OBD_CONNECT_CANCELSET;
> -	else if (ns->ns_orig_connect_flags & OBD_CONNECT_CANCELSET)
> -		ns->ns_connect_flags |= OBD_CONNECT_CANCELSET;
> -	return count;
> -}
> -LUSTRE_RW_ATTR(early_lock_cancel);
> -
> -/* These are for namespaces in /sys/fs/lustre/ldlm/namespaces/ */
> -static struct attribute *ldlm_ns_attrs[] = {
> -	&lustre_attr_resource_count.attr,
> -	&lustre_attr_lock_count.attr,
> -	&lustre_attr_lock_unused_count.attr,
> -	&lustre_attr_lru_size.attr,
> -	&lustre_attr_lru_max_age.attr,
> -	&lustre_attr_early_lock_cancel.attr,
> -	NULL,
> -};
> -
> -static void ldlm_ns_release(struct kobject *kobj)
> -{
> -	struct ldlm_namespace *ns = container_of(kobj, struct ldlm_namespace,
> -						 ns_kobj);
> -	complete(&ns->ns_kobj_unregister);
> -}
> -
> -static struct kobj_type ldlm_ns_ktype = {
> -	.default_attrs  = ldlm_ns_attrs,
> -	.sysfs_ops      = &lustre_sysfs_ops,
> -	.release        = ldlm_ns_release,
> -};
> -
> -static void ldlm_namespace_debugfs_unregister(struct ldlm_namespace *ns)
> -{
> -	debugfs_remove_recursive(ns->ns_debugfs_entry);
> -
> -	if (ns->ns_stats)
> -		lprocfs_free_stats(&ns->ns_stats);
> -}
> -
> -static void ldlm_namespace_sysfs_unregister(struct ldlm_namespace *ns)
> -{
> -	kobject_put(&ns->ns_kobj);
> -	wait_for_completion(&ns->ns_kobj_unregister);
> -}
> -
> -static int ldlm_namespace_sysfs_register(struct ldlm_namespace *ns)
> -{
> -	int err;
> -
> -	ns->ns_kobj.kset = ldlm_ns_kset;
> -	init_completion(&ns->ns_kobj_unregister);
> -	err = kobject_init_and_add(&ns->ns_kobj, &ldlm_ns_ktype, NULL,
> -				   "%s", ldlm_ns_name(ns));
> -
> -	ns->ns_stats = lprocfs_alloc_stats(LDLM_NSS_LAST, 0);
> -	if (!ns->ns_stats) {
> -		kobject_put(&ns->ns_kobj);
> -		return -ENOMEM;
> -	}
> -
> -	lprocfs_counter_init(ns->ns_stats, LDLM_NSS_LOCKS,
> -			     LPROCFS_CNTR_AVGMINMAX, "locks", "locks");
> -
> -	return err;
> -}
> -
> -static int ldlm_namespace_debugfs_register(struct ldlm_namespace *ns)
> -{
> -	struct dentry *ns_entry;
> -
> -	if (!IS_ERR_OR_NULL(ns->ns_debugfs_entry)) {
> -		ns_entry = ns->ns_debugfs_entry;
> -	} else {
> -		ns_entry = debugfs_create_dir(ldlm_ns_name(ns),
> -					      ldlm_ns_debugfs_dir);
> -		if (!ns_entry)
> -			return -ENOMEM;
> -		ns->ns_debugfs_entry = ns_entry;
> -	}
> -
> -	return 0;
> -}
> -
> -#undef MAX_STRING_SIZE
> -
> -static struct ldlm_resource *ldlm_resource_getref(struct ldlm_resource *res)
> -{
> -	LASSERT(res);
> -	LASSERT(res != LP_POISON);
> -	atomic_inc(&res->lr_refcount);
> -	CDEBUG(D_INFO, "getref res: %p count: %d\n", res,
> -	       atomic_read(&res->lr_refcount));
> -	return res;
> -}
> -
> -static unsigned int ldlm_res_hop_hash(struct cfs_hash *hs,
> -				      const void *key, unsigned int mask)
> -{
> -	const struct ldlm_res_id     *id  = key;
> -	unsigned int		val = 0;
> -	unsigned int		i;
> -
> -	for (i = 0; i < RES_NAME_SIZE; i++)
> -		val += id->name[i];
> -	return val & mask;
> -}
> -
> -static unsigned int ldlm_res_hop_fid_hash(struct cfs_hash *hs,
> -					  const void *key, unsigned int mask)
> -{
> -	const struct ldlm_res_id *id = key;
> -	struct lu_fid       fid;
> -	__u32	       hash;
> -	__u32	       val;
> -
> -	fid.f_seq = id->name[LUSTRE_RES_ID_SEQ_OFF];
> -	fid.f_oid = (__u32)id->name[LUSTRE_RES_ID_VER_OID_OFF];
> -	fid.f_ver = (__u32)(id->name[LUSTRE_RES_ID_VER_OID_OFF] >> 32);
> -
> -	hash = fid_flatten32(&fid);
> -	hash += (hash >> 4) + (hash << 12); /* mixing oid and seq */
> -	if (id->name[LUSTRE_RES_ID_HSH_OFF] != 0) {
> -		val = id->name[LUSTRE_RES_ID_HSH_OFF];
> -		hash += (val >> 5) + (val << 11);
> -	} else {
> -		val = fid_oid(&fid);
> -	}
> -	hash = hash_long(hash, hs->hs_bkt_bits);
> -	/* give me another random factor */
> -	hash -= hash_long((unsigned long)hs, val % 11 + 3);
> -
> -	hash <<= hs->hs_cur_bits - hs->hs_bkt_bits;
> -	hash |= ldlm_res_hop_hash(hs, key, CFS_HASH_NBKT(hs) - 1);
> -
> -	return hash & mask;
> -}
> -
> -static void *ldlm_res_hop_key(struct hlist_node *hnode)
> -{
> -	struct ldlm_resource   *res;
> -
> -	res = hlist_entry(hnode, struct ldlm_resource, lr_hash);
> -	return &res->lr_name;
> -}
> -
> -static int ldlm_res_hop_keycmp(const void *key, struct hlist_node *hnode)
> -{
> -	struct ldlm_resource   *res;
> -
> -	res = hlist_entry(hnode, struct ldlm_resource, lr_hash);
> -	return ldlm_res_eq((const struct ldlm_res_id *)key,
> -			   (const struct ldlm_res_id *)&res->lr_name);
> -}
> -
> -static void *ldlm_res_hop_object(struct hlist_node *hnode)
> -{
> -	return hlist_entry(hnode, struct ldlm_resource, lr_hash);
> -}
> -
> -static void ldlm_res_hop_get_locked(struct cfs_hash *hs,
> -				    struct hlist_node *hnode)
> -{
> -	struct ldlm_resource *res;
> -
> -	res = hlist_entry(hnode, struct ldlm_resource, lr_hash);
> -	ldlm_resource_getref(res);
> -}
> -
> -static void ldlm_res_hop_put(struct cfs_hash *hs, struct hlist_node *hnode)
> -{
> -	struct ldlm_resource *res;
> -
> -	res = hlist_entry(hnode, struct ldlm_resource, lr_hash);
> -	ldlm_resource_putref(res);
> -}
> -
> -static struct cfs_hash_ops ldlm_ns_hash_ops = {
> -	.hs_hash	= ldlm_res_hop_hash,
> -	.hs_key		= ldlm_res_hop_key,
> -	.hs_keycmp      = ldlm_res_hop_keycmp,
> -	.hs_keycpy      = NULL,
> -	.hs_object      = ldlm_res_hop_object,
> -	.hs_get		= ldlm_res_hop_get_locked,
> -	.hs_put		= ldlm_res_hop_put
> -};
> -
> -static struct cfs_hash_ops ldlm_ns_fid_hash_ops = {
> -	.hs_hash	= ldlm_res_hop_fid_hash,
> -	.hs_key		= ldlm_res_hop_key,
> -	.hs_keycmp      = ldlm_res_hop_keycmp,
> -	.hs_keycpy      = NULL,
> -	.hs_object      = ldlm_res_hop_object,
> -	.hs_get		= ldlm_res_hop_get_locked,
> -	.hs_put		= ldlm_res_hop_put
> -};
> -
> -struct ldlm_ns_hash_def {
> -	enum ldlm_ns_type nsd_type;
> -	/** hash bucket bits */
> -	unsigned int	nsd_bkt_bits;
> -	/** hash bits */
> -	unsigned int	nsd_all_bits;
> -	/** hash operations */
> -	struct cfs_hash_ops *nsd_hops;
> -};
> -
> -static struct ldlm_ns_hash_def ldlm_ns_hash_defs[] = {
> -	{
> -		.nsd_type       = LDLM_NS_TYPE_MDC,
> -		.nsd_bkt_bits   = 11,
> -		.nsd_all_bits   = 16,
> -		.nsd_hops       = &ldlm_ns_fid_hash_ops,
> -	},
> -	{
> -		.nsd_type       = LDLM_NS_TYPE_MDT,
> -		.nsd_bkt_bits   = 14,
> -		.nsd_all_bits   = 21,
> -		.nsd_hops       = &ldlm_ns_fid_hash_ops,
> -	},
> -	{
> -		.nsd_type       = LDLM_NS_TYPE_OSC,
> -		.nsd_bkt_bits   = 8,
> -		.nsd_all_bits   = 12,
> -		.nsd_hops       = &ldlm_ns_hash_ops,
> -	},
> -	{
> -		.nsd_type       = LDLM_NS_TYPE_OST,
> -		.nsd_bkt_bits   = 11,
> -		.nsd_all_bits   = 17,
> -		.nsd_hops       = &ldlm_ns_hash_ops,
> -	},
> -	{
> -		.nsd_type       = LDLM_NS_TYPE_MGC,
> -		.nsd_bkt_bits   = 4,
> -		.nsd_all_bits   = 4,
> -		.nsd_hops       = &ldlm_ns_hash_ops,
> -	},
> -	{
> -		.nsd_type       = LDLM_NS_TYPE_MGT,
> -		.nsd_bkt_bits   = 4,
> -		.nsd_all_bits   = 4,
> -		.nsd_hops       = &ldlm_ns_hash_ops,
> -	},
> -	{
> -		.nsd_type       = LDLM_NS_TYPE_UNKNOWN,
> -	},
> -};
> -
> -/** Register \a ns in the list of namespaces */
> -static void ldlm_namespace_register(struct ldlm_namespace *ns,
> -				    enum ldlm_side client)
> -{
> -	mutex_lock(ldlm_namespace_lock(client));
> -	LASSERT(list_empty(&ns->ns_list_chain));
> -	list_add(&ns->ns_list_chain, &ldlm_cli_inactive_namespace_list);
> -	ldlm_namespace_nr_inc(client);
> -	mutex_unlock(ldlm_namespace_lock(client));
> -}
> -
> -/**
> - * Create and initialize new empty namespace.
> - */
> -struct ldlm_namespace *ldlm_namespace_new(struct obd_device *obd, char *name,
> -					  enum ldlm_side client,
> -					  enum ldlm_appetite apt,
> -					  enum ldlm_ns_type ns_type)
> -{
> -	struct ldlm_namespace *ns = NULL;
> -	struct ldlm_ns_bucket *nsb;
> -	struct ldlm_ns_hash_def    *nsd;
> -	struct cfs_hash_bd	  bd;
> -	int		    idx;
> -	int		    rc;
> -
> -	LASSERT(obd);
> -
> -	rc = ldlm_get_ref();
> -	if (rc) {
> -		CERROR("ldlm_get_ref failed: %d\n", rc);
> -		return NULL;
> -	}
> -
> -	for (idx = 0;; idx++) {
> -		nsd = &ldlm_ns_hash_defs[idx];
> -		if (nsd->nsd_type == LDLM_NS_TYPE_UNKNOWN) {
> -			CERROR("Unknown type %d for ns %s\n", ns_type, name);
> -			goto out_ref;
> -		}
> -
> -		if (nsd->nsd_type == ns_type)
> -			break;
> -	}
> -
> -	ns = kzalloc(sizeof(*ns), GFP_NOFS);
> -	if (!ns)
> -		goto out_ref;
> -
> -	ns->ns_rs_hash = cfs_hash_create(name,
> -					 nsd->nsd_all_bits, nsd->nsd_all_bits,
> -					 nsd->nsd_bkt_bits, sizeof(*nsb),
> -					 CFS_HASH_MIN_THETA,
> -					 CFS_HASH_MAX_THETA,
> -					 nsd->nsd_hops,
> -					 CFS_HASH_DEPTH |
> -					 CFS_HASH_BIGNAME |
> -					 CFS_HASH_SPIN_BKTLOCK |
> -					 CFS_HASH_NO_ITEMREF);
> -	if (!ns->ns_rs_hash)
> -		goto out_ns;
> -
> -	cfs_hash_for_each_bucket(ns->ns_rs_hash, &bd, idx) {
> -		nsb = cfs_hash_bd_extra_get(ns->ns_rs_hash, &bd);
> -		at_init(&nsb->nsb_at_estimate, ldlm_enqueue_min, 0);
> -		nsb->nsb_namespace = ns;
> -	}
> -
> -	ns->ns_obd      = obd;
> -	ns->ns_appetite = apt;
> -	ns->ns_client   = client;
> -	ns->ns_name     = kstrdup(name, GFP_KERNEL);
> -	if (!ns->ns_name)
> -		goto out_hash;
> -
> -	INIT_LIST_HEAD(&ns->ns_list_chain);
> -	INIT_LIST_HEAD(&ns->ns_unused_list);
> -	spin_lock_init(&ns->ns_lock);
> -	atomic_set(&ns->ns_bref, 0);
> -	init_waitqueue_head(&ns->ns_waitq);
> -
> -	ns->ns_max_parallel_ast   = LDLM_DEFAULT_PARALLEL_AST_LIMIT;
> -	ns->ns_nr_unused	  = 0;
> -	ns->ns_max_unused	 = LDLM_DEFAULT_LRU_SIZE;
> -	ns->ns_max_age	    = LDLM_DEFAULT_MAX_ALIVE;
> -	ns->ns_orig_connect_flags = 0;
> -	ns->ns_connect_flags      = 0;
> -	ns->ns_stopping	   = 0;
> -
> -	rc = ldlm_namespace_sysfs_register(ns);
> -	if (rc != 0) {
> -		CERROR("Can't initialize ns sysfs, rc %d\n", rc);
> -		goto out_hash;
> -	}
> -
> -	rc = ldlm_namespace_debugfs_register(ns);
> -	if (rc != 0) {
> -		CERROR("Can't initialize ns proc, rc %d\n", rc);
> -		goto out_sysfs;
> -	}
> -
> -	idx = ldlm_namespace_nr_read(client);
> -	rc = ldlm_pool_init(&ns->ns_pool, ns, idx, client);
> -	if (rc) {
> -		CERROR("Can't initialize lock pool, rc %d\n", rc);
> -		goto out_proc;
> -	}
> -
> -	ldlm_namespace_register(ns, client);
> -	return ns;
> -out_proc:
> -	ldlm_namespace_debugfs_unregister(ns);
> -out_sysfs:
> -	ldlm_namespace_sysfs_unregister(ns);
> -	ldlm_namespace_cleanup(ns, 0);
> -out_hash:
> -	kfree(ns->ns_name);
> -	cfs_hash_putref(ns->ns_rs_hash);
> -out_ns:
> -	kfree(ns);
> -out_ref:
> -	ldlm_put_ref();
> -	return NULL;
> -}
> -EXPORT_SYMBOL(ldlm_namespace_new);
> -
> -extern struct ldlm_lock *ldlm_lock_get(struct ldlm_lock *lock);
> -
> -/**
> - * Cancel and destroy all locks on a resource.
> - *
> - * If flags contains FL_LOCAL_ONLY, don't try to tell the server, just
> - * clean up.  This is currently only used for recovery, and we make
> - * certain assumptions as a result--notably, that we shouldn't cancel
> - * locks with refs.
> - */
> -static void cleanup_resource(struct ldlm_resource *res, struct list_head *q,
> -			     __u64 flags)
> -{
> -	int rc = 0;
> -	bool local_only = !!(flags & LDLM_FL_LOCAL_ONLY);
> -
> -	do {
> -		struct ldlm_lock *lock = NULL, *tmp;
> -		struct lustre_handle lockh;
> -
> -		/* First, we look for non-cleaned-yet lock
> -		 * all cleaned locks are marked by CLEANED flag.
> -		 */
> -		lock_res(res);
> -		list_for_each_entry(tmp, q, l_res_link) {
> -			if (ldlm_is_cleaned(tmp))
> -				continue;
> -
> -			lock = tmp;
> -			LDLM_LOCK_GET(lock);
> -			ldlm_set_cleaned(lock);
> -			break;
> -		}
> -
> -		if (!lock) {
> -			unlock_res(res);
> -			break;
> -		}
> -
> -		/* Set CBPENDING so nothing in the cancellation path
> -		 * can match this lock.
> -		 */
> -		ldlm_set_cbpending(lock);
> -		ldlm_set_failed(lock);
> -		lock->l_flags |= flags;
> -
> -		/* ... without sending a CANCEL message for local_only. */
> -		if (local_only)
> -			ldlm_set_local_only(lock);
> -
> -		if (local_only && (lock->l_readers || lock->l_writers)) {
> -			/* This is a little bit gross, but much better than the
> -			 * alternative: pretend that we got a blocking AST from
> -			 * the server, so that when the lock is decref'd, it
> -			 * will go away ...
> -			 */
> -			unlock_res(res);
> -			LDLM_DEBUG(lock, "setting FL_LOCAL_ONLY");
> -			if (lock->l_flags & LDLM_FL_FAIL_LOC) {
> -				set_current_state(TASK_UNINTERRUPTIBLE);
> -				schedule_timeout(4 * HZ);
> -				set_current_state(TASK_RUNNING);
> -			}
> -			if (lock->l_completion_ast)
> -				lock->l_completion_ast(lock, LDLM_FL_FAILED,
> -						       NULL);
> -			LDLM_LOCK_RELEASE(lock);
> -			continue;
> -		}
> -
> -		unlock_res(res);
> -		ldlm_lock2handle(lock, &lockh);
> -		rc = ldlm_cli_cancel(&lockh, LCF_LOCAL);
> -		if (rc)
> -			CERROR("ldlm_cli_cancel: %d\n", rc);
> -		LDLM_LOCK_RELEASE(lock);
> -	} while (1);
> -}
> -
> -static int ldlm_resource_clean(struct cfs_hash *hs, struct cfs_hash_bd *bd,
> -			       struct hlist_node *hnode, void *arg)
> -{
> -	struct ldlm_resource *res = cfs_hash_object(hs, hnode);
> -	__u64 flags = *(__u64 *)arg;
> -
> -	cleanup_resource(res, &res->lr_granted, flags);
> -	cleanup_resource(res, &res->lr_waiting, flags);
> -
> -	return 0;
> -}
> -
> -static int ldlm_resource_complain(struct cfs_hash *hs, struct cfs_hash_bd *bd,
> -				  struct hlist_node *hnode, void *arg)
> -{
> -	struct ldlm_resource  *res = cfs_hash_object(hs, hnode);
> -
> -	lock_res(res);
> -	CERROR("%s: namespace resource " DLDLMRES
> -	       " (%p) refcount nonzero (%d) after lock cleanup; forcing cleanup.\n",
> -	       ldlm_ns_name(ldlm_res_to_ns(res)), PLDLMRES(res), res,
> -	       atomic_read(&res->lr_refcount) - 1);
> -
> -	ldlm_resource_dump(D_ERROR, res);
> -	unlock_res(res);
> -	return 0;
> -}
> -
> -/**
> - * Cancel and destroy all locks in the namespace.
> - *
> - * Typically used during evictions when server notified client that it was
> - * evicted and all of its state needs to be destroyed.
> - * Also used during shutdown.
> - */
> -int ldlm_namespace_cleanup(struct ldlm_namespace *ns, __u64 flags)
> -{
> -	if (!ns) {
> -		CDEBUG(D_INFO, "NULL ns, skipping cleanup\n");
> -		return ELDLM_OK;
> -	}
> -
> -	cfs_hash_for_each_nolock(ns->ns_rs_hash, ldlm_resource_clean,
> -				 &flags, 0);
> -	cfs_hash_for_each_nolock(ns->ns_rs_hash, ldlm_resource_complain,
> -				 NULL, 0);
> -	return ELDLM_OK;
> -}
> -EXPORT_SYMBOL(ldlm_namespace_cleanup);
> -
> -/**
> - * Attempts to free namespace.
> - *
> - * Only used when namespace goes away, like during an unmount.
> - */
> -static int __ldlm_namespace_free(struct ldlm_namespace *ns, int force)
> -{
> -	/* At shutdown time, don't call the cancellation callback */
> -	ldlm_namespace_cleanup(ns, force ? LDLM_FL_LOCAL_ONLY : 0);
> -
> -	if (atomic_read(&ns->ns_bref) > 0) {
> -		int rc;
> -
> -		CDEBUG(D_DLMTRACE,
> -		       "dlm namespace %s free waiting on refcount %d\n",
> -		       ldlm_ns_name(ns), atomic_read(&ns->ns_bref));
> -force_wait:
> -		if (force)
> -			rc = wait_event_idle_timeout(ns->ns_waitq,
> -						     atomic_read(&ns->ns_bref) == 0,
> -						     obd_timeout * HZ / 4) ? 0 : -ETIMEDOUT;
> -		else
> -			rc = l_wait_event_abortable(ns->ns_waitq,
> -						    atomic_read(&ns->ns_bref) == 0);
> -
> -		/* Forced cleanups should be able to reclaim all references,
> -		 * so it's safe to wait forever... we can't leak locks...
> -		 */
> -		if (force && rc == -ETIMEDOUT) {
> -			LCONSOLE_ERROR("Forced cleanup waiting for %s namespace with %d resources in use, (rc=%d)\n",
> -				       ldlm_ns_name(ns),
> -				       atomic_read(&ns->ns_bref), rc);
> -			goto force_wait;
> -		}
> -
> -		if (atomic_read(&ns->ns_bref)) {
> -			LCONSOLE_ERROR("Cleanup waiting for %s namespace with %d resources in use, (rc=%d)\n",
> -				       ldlm_ns_name(ns),
> -				       atomic_read(&ns->ns_bref), rc);
> -			return ELDLM_NAMESPACE_EXISTS;
> -		}
> -		CDEBUG(D_DLMTRACE, "dlm namespace %s free done waiting\n",
> -		       ldlm_ns_name(ns));
> -	}
> -
> -	return ELDLM_OK;
> -}
> -
> -/**
> - * Performs various cleanups for passed \a ns to make it drop refc and be
> - * ready for freeing. Waits for refc == 0.
> - *
> - * The following is done:
> - * (0) Unregister \a ns from its list to make inaccessible for potential
> - * users like pools thread and others;
> - * (1) Clear all locks in \a ns.
> - */
> -void ldlm_namespace_free_prior(struct ldlm_namespace *ns,
> -			       struct obd_import *imp,
> -			       int force)
> -{
> -	int rc;
> -
> -	if (!ns)
> -		return;
> -
> -	spin_lock(&ns->ns_lock);
> -	ns->ns_stopping = 1;
> -	spin_unlock(&ns->ns_lock);
> -
> -	/*
> -	 * Can fail with -EINTR when force == 0 in which case try harder.
> -	 */
> -	rc = __ldlm_namespace_free(ns, force);
> -	if (rc != ELDLM_OK) {
> -		if (imp) {
> -			ptlrpc_disconnect_import(imp, 0);
> -			ptlrpc_invalidate_import(imp);
> -		}
> -
> -		/*
> -		 * With all requests dropped and the import inactive
> -		 * we are guaranteed all reference will be dropped.
> -		 */
> -		rc = __ldlm_namespace_free(ns, 1);
> -		LASSERT(rc == 0);
> -	}
> -}
> -
> -/** Unregister \a ns from the list of namespaces. */
> -static void ldlm_namespace_unregister(struct ldlm_namespace *ns,
> -				      enum ldlm_side client)
> -{
> -	mutex_lock(ldlm_namespace_lock(client));
> -	LASSERT(!list_empty(&ns->ns_list_chain));
> -	/* Some asserts and possibly other parts of the code are still
> -	 * using list_empty(&ns->ns_list_chain). This is why it is
> -	 * important to use list_del_init() here.
> -	 */
> -	list_del_init(&ns->ns_list_chain);
> -	ldlm_namespace_nr_dec(client);
> -	mutex_unlock(ldlm_namespace_lock(client));
> -}
> -
> -/**
> - * Performs freeing memory structures related to \a ns. This is only done
> - * when ldlm_namespce_free_prior() successfully removed all resources
> - * referencing \a ns and its refc == 0.
> - */
> -void ldlm_namespace_free_post(struct ldlm_namespace *ns)
> -{
> -	if (!ns)
> -		return;
> -
> -	/* Make sure that nobody can find this ns in its list. */
> -	ldlm_namespace_unregister(ns, ns->ns_client);
> -	/* Fini pool _before_ parent proc dir is removed. This is important as
> -	 * ldlm_pool_fini() removes own proc dir which is child to @dir.
> -	 * Removing it after @dir may cause oops.
> -	 */
> -	ldlm_pool_fini(&ns->ns_pool);
> -
> -	ldlm_namespace_debugfs_unregister(ns);
> -	ldlm_namespace_sysfs_unregister(ns);
> -	cfs_hash_putref(ns->ns_rs_hash);
> -	kfree(ns->ns_name);
> -	/* Namespace \a ns should be not on list at this time, otherwise
> -	 * this will cause issues related to using freed \a ns in poold
> -	 * thread.
> -	 */
> -	LASSERT(list_empty(&ns->ns_list_chain));
> -	kfree(ns);
> -	ldlm_put_ref();
> -}
> -
> -void ldlm_namespace_get(struct ldlm_namespace *ns)
> -{
> -	atomic_inc(&ns->ns_bref);
> -}
> -
> -/* This is only for callers that care about refcount */
> -static int ldlm_namespace_get_return(struct ldlm_namespace *ns)
> -{
> -	return atomic_inc_return(&ns->ns_bref);
> -}
> -
> -void ldlm_namespace_put(struct ldlm_namespace *ns)
> -{
> -	if (atomic_dec_and_lock(&ns->ns_bref, &ns->ns_lock)) {
> -		wake_up(&ns->ns_waitq);
> -		spin_unlock(&ns->ns_lock);
> -	}
> -}
> -
> -/** Should be called with ldlm_namespace_lock(client) taken. */
> -void ldlm_namespace_move_to_active_locked(struct ldlm_namespace *ns,
> -					  enum ldlm_side client)
> -{
> -	LASSERT(!list_empty(&ns->ns_list_chain));
> -	LASSERT(mutex_is_locked(ldlm_namespace_lock(client)));
> -	list_move_tail(&ns->ns_list_chain, ldlm_namespace_list(client));
> -}
> -
> -/** Should be called with ldlm_namespace_lock(client) taken. */
> -void ldlm_namespace_move_to_inactive_locked(struct ldlm_namespace *ns,
> -					    enum ldlm_side client)
> -{
> -	LASSERT(!list_empty(&ns->ns_list_chain));
> -	LASSERT(mutex_is_locked(ldlm_namespace_lock(client)));
> -	list_move_tail(&ns->ns_list_chain, &ldlm_cli_inactive_namespace_list);
> -}
> -
> -/** Should be called with ldlm_namespace_lock(client) taken. */
> -struct ldlm_namespace *ldlm_namespace_first_locked(enum ldlm_side client)
> -{
> -	LASSERT(mutex_is_locked(ldlm_namespace_lock(client)));
> -	LASSERT(!list_empty(ldlm_namespace_list(client)));
> -	return container_of(ldlm_namespace_list(client)->next,
> -		struct ldlm_namespace, ns_list_chain);
> -}
> -
> -/** Create and initialize new resource. */
> -static struct ldlm_resource *ldlm_resource_new(void)
> -{
> -	struct ldlm_resource *res;
> -	int idx;
> -
> -	res = kmem_cache_zalloc(ldlm_resource_slab, GFP_NOFS);
> -	if (!res)
> -		return NULL;
> -
> -	INIT_LIST_HEAD(&res->lr_granted);
> -	INIT_LIST_HEAD(&res->lr_waiting);
> -
> -	/* Initialize interval trees for each lock mode. */
> -	for (idx = 0; idx < LCK_MODE_NUM; idx++) {
> -		res->lr_itree[idx].lit_size = 0;
> -		res->lr_itree[idx].lit_mode = 1 << idx;
> -		res->lr_itree[idx].lit_root = RB_ROOT_CACHED;
> -	}
> -
> -	atomic_set(&res->lr_refcount, 1);
> -	spin_lock_init(&res->lr_lock);
> -	lu_ref_init(&res->lr_reference);
> -
> -	/* The creator of the resource must unlock the mutex after LVB
> -	 * initialization.
> -	 */
> -	mutex_init(&res->lr_lvb_mutex);
> -	mutex_lock(&res->lr_lvb_mutex);
> -
> -	return res;
> -}
> -
> -/**
> - * Return a reference to resource with given name, creating it if necessary.
> - * Args: namespace with ns_lock unlocked
> - * Locks: takes and releases NS hash-lock and res->lr_lock
> - * Returns: referenced, unlocked ldlm_resource or NULL
> - */
> -struct ldlm_resource *
> -ldlm_resource_get(struct ldlm_namespace *ns, struct ldlm_resource *parent,
> -		  const struct ldlm_res_id *name, enum ldlm_type type,
> -		  int create)
> -{
> -	struct hlist_node     *hnode;
> -	struct ldlm_resource *res = NULL;
> -	struct cfs_hash_bd	 bd;
> -	__u64		 version;
> -	int		      ns_refcount = 0;
> -	int rc;
> -
> -	LASSERT(!parent);
> -	LASSERT(ns->ns_rs_hash);
> -	LASSERT(name->name[0] != 0);
> -
> -	cfs_hash_bd_get_and_lock(ns->ns_rs_hash, (void *)name, &bd, 0);
> -	hnode = cfs_hash_bd_lookup_locked(ns->ns_rs_hash, &bd, (void *)name);
> -	if (hnode) {
> -		cfs_hash_bd_unlock(ns->ns_rs_hash, &bd, 0);
> -		goto lvbo_init;
> -	}
> -
> -	version = cfs_hash_bd_version_get(&bd);
> -	cfs_hash_bd_unlock(ns->ns_rs_hash, &bd, 0);
> -
> -	if (create == 0)
> -		return ERR_PTR(-ENOENT);
> -
> -	LASSERTF(type >= LDLM_MIN_TYPE && type < LDLM_MAX_TYPE,
> -		 "type: %d\n", type);
> -	res = ldlm_resource_new();
> -	if (!res)
> -		return ERR_PTR(-ENOMEM);
> -
> -	res->lr_ns_bucket  = cfs_hash_bd_extra_get(ns->ns_rs_hash, &bd);
> -	res->lr_name       = *name;
> -	res->lr_type       = type;
> -
> -	cfs_hash_bd_lock(ns->ns_rs_hash, &bd, 1);
> -	hnode = (version == cfs_hash_bd_version_get(&bd)) ?  NULL :
> -		cfs_hash_bd_lookup_locked(ns->ns_rs_hash, &bd, (void *)name);
> -
> -	if (hnode) {
> -		/* Someone won the race and already added the resource. */
> -		cfs_hash_bd_unlock(ns->ns_rs_hash, &bd, 1);
> -		/* Clean lu_ref for failed resource. */
> -		lu_ref_fini(&res->lr_reference);
> -		/* We have taken lr_lvb_mutex. Drop it. */
> -		mutex_unlock(&res->lr_lvb_mutex);
> -		kmem_cache_free(ldlm_resource_slab, res);
> -lvbo_init:
> -		res = hlist_entry(hnode, struct ldlm_resource, lr_hash);
> -		/* Synchronize with regard to resource creation. */
> -		if (ns->ns_lvbo && ns->ns_lvbo->lvbo_init) {
> -			mutex_lock(&res->lr_lvb_mutex);
> -			mutex_unlock(&res->lr_lvb_mutex);
> -		}
> -
> -		if (unlikely(res->lr_lvb_len < 0)) {
> -			rc = res->lr_lvb_len;
> -			ldlm_resource_putref(res);
> -			res = ERR_PTR(rc);
> -		}
> -		return res;
> -	}
> -	/* We won! Let's add the resource. */
> -	cfs_hash_bd_add_locked(ns->ns_rs_hash, &bd, &res->lr_hash);
> -	if (cfs_hash_bd_count_get(&bd) == 1)
> -		ns_refcount = ldlm_namespace_get_return(ns);
> -
> -	cfs_hash_bd_unlock(ns->ns_rs_hash, &bd, 1);
> -	if (ns->ns_lvbo && ns->ns_lvbo->lvbo_init) {
> -		OBD_FAIL_TIMEOUT(OBD_FAIL_LDLM_CREATE_RESOURCE, 2);
> -		rc = ns->ns_lvbo->lvbo_init(res);
> -		if (rc < 0) {
> -			CERROR("%s: lvbo_init failed for resource %#llx:%#llx: rc = %d\n",
> -			       ns->ns_obd->obd_name, name->name[0],
> -			       name->name[1], rc);
> -			res->lr_lvb_len = rc;
> -			mutex_unlock(&res->lr_lvb_mutex);
> -			ldlm_resource_putref(res);
> -			return ERR_PTR(rc);
> -		}
> -	}
> -
> -	/* We create resource with locked lr_lvb_mutex. */
> -	mutex_unlock(&res->lr_lvb_mutex);
> -
> -	/* Let's see if we happened to be the very first resource in this
> -	 * namespace. If so, and this is a client namespace, we need to move
> -	 * the namespace into the active namespaces list to be patrolled by
> -	 * the ldlm_poold.
> -	 */
> -	if (ns_refcount == 1) {
> -		mutex_lock(ldlm_namespace_lock(LDLM_NAMESPACE_CLIENT));
> -		ldlm_namespace_move_to_active_locked(ns, LDLM_NAMESPACE_CLIENT);
> -		mutex_unlock(ldlm_namespace_lock(LDLM_NAMESPACE_CLIENT));
> -	}
> -
> -	return res;
> -}
> -EXPORT_SYMBOL(ldlm_resource_get);
> -
> -static void __ldlm_resource_putref_final(struct cfs_hash_bd *bd,
> -					 struct ldlm_resource *res)
> -{
> -	struct ldlm_ns_bucket *nsb = res->lr_ns_bucket;
> -	struct ldlm_namespace *ns = nsb->nsb_namespace;
> -
> -	if (!list_empty(&res->lr_granted)) {
> -		ldlm_resource_dump(D_ERROR, res);
> -		LBUG();
> -	}
> -
> -	if (!list_empty(&res->lr_waiting)) {
> -		ldlm_resource_dump(D_ERROR, res);
> -		LBUG();
> -	}
> -
> -	cfs_hash_bd_del_locked(ns->ns_rs_hash,
> -			       bd, &res->lr_hash);
> -	lu_ref_fini(&res->lr_reference);
> -	cfs_hash_bd_unlock(ns->ns_rs_hash, bd, 1);
> -	if (ns->ns_lvbo && ns->ns_lvbo->lvbo_free)
> -		ns->ns_lvbo->lvbo_free(res);
> -	if (cfs_hash_bd_count_get(bd) == 0)
> -		ldlm_namespace_put(ns);
> -	kmem_cache_free(ldlm_resource_slab, res);
> -}
> -
> -void ldlm_resource_putref(struct ldlm_resource *res)
> -{
> -	struct ldlm_namespace *ns = ldlm_res_to_ns(res);
> -	struct cfs_hash_bd   bd;
> -
> -	LASSERT_ATOMIC_GT_LT(&res->lr_refcount, 0, LI_POISON);
> -	CDEBUG(D_INFO, "putref res: %p count: %d\n",
> -	       res, atomic_read(&res->lr_refcount) - 1);
> -
> -	cfs_hash_bd_get(ns->ns_rs_hash, &res->lr_name, &bd);
> -	if (cfs_hash_bd_dec_and_lock(ns->ns_rs_hash, &bd, &res->lr_refcount))
> -		__ldlm_resource_putref_final(&bd, res);
> -}
> -EXPORT_SYMBOL(ldlm_resource_putref);
> -
> -/**
> - * Add a lock into a given resource into specified lock list.
> - */
> -void ldlm_resource_add_lock(struct ldlm_resource *res, struct list_head *head,
> -			    struct ldlm_lock *lock)
> -{
> -	check_res_locked(res);
> -
> -	LDLM_DEBUG(lock, "About to add this lock:");
> -
> -	if (ldlm_is_destroyed(lock)) {
> -		CDEBUG(D_OTHER, "Lock destroyed, not adding to resource\n");
> -		return;
> -	}
> -
> -	LASSERT(list_empty(&lock->l_res_link));
> -
> -	list_add_tail(&lock->l_res_link, head);
> -}
> -
> -void ldlm_resource_unlink_lock(struct ldlm_lock *lock)
> -{
> -	int type = lock->l_resource->lr_type;
> -
> -	check_res_locked(lock->l_resource);
> -	if (type == LDLM_IBITS || type == LDLM_PLAIN)
> -		ldlm_unlink_lock_skiplist(lock);
> -	else if (type == LDLM_EXTENT)
> -		ldlm_extent_unlink_lock(lock);
> -	list_del_init(&lock->l_res_link);
> -}
> -EXPORT_SYMBOL(ldlm_resource_unlink_lock);
> -
> -void ldlm_res2desc(struct ldlm_resource *res, struct ldlm_resource_desc *desc)
> -{
> -	desc->lr_type = res->lr_type;
> -	desc->lr_name = res->lr_name;
> -}
> -
> -/**
> - * Print information about all locks in all namespaces on this node to debug
> - * log.
> - */
> -void ldlm_dump_all_namespaces(enum ldlm_side client, int level)
> -{
> -	struct ldlm_namespace *ns;
> -
> -	if (!((libcfs_debug | D_ERROR) & level))
> -		return;
> -
> -	mutex_lock(ldlm_namespace_lock(client));
> -
> -	list_for_each_entry(ns, ldlm_namespace_list(client), ns_list_chain)
> -		ldlm_namespace_dump(level, ns);
> -
> -	mutex_unlock(ldlm_namespace_lock(client));
> -}
> -
> -static int ldlm_res_hash_dump(struct cfs_hash *hs, struct cfs_hash_bd *bd,
> -			      struct hlist_node *hnode, void *arg)
> -{
> -	struct ldlm_resource *res = cfs_hash_object(hs, hnode);
> -	int    level = (int)(unsigned long)arg;
> -
> -	lock_res(res);
> -	ldlm_resource_dump(level, res);
> -	unlock_res(res);
> -
> -	return 0;
> -}
> -
> -/**
> - * Print information about all locks in this namespace on this node to debug
> - * log.
> - */
> -void ldlm_namespace_dump(int level, struct ldlm_namespace *ns)
> -{
> -	if (!((libcfs_debug | D_ERROR) & level))
> -		return;
> -
> -	CDEBUG(level, "--- Namespace: %s (rc: %d, side: client)\n",
> -	       ldlm_ns_name(ns), atomic_read(&ns->ns_bref));
> -
> -	if (time_before(jiffies, ns->ns_next_dump))
> -		return;
> -
> -	cfs_hash_for_each_nolock(ns->ns_rs_hash,
> -				 ldlm_res_hash_dump,
> -				 (void *)(unsigned long)level, 0);
> -	spin_lock(&ns->ns_lock);
> -	ns->ns_next_dump = jiffies + 10 * HZ;
> -	spin_unlock(&ns->ns_lock);
> -}
> -
> -/**
> - * Print information about all locks in this resource to debug log.
> - */
> -void ldlm_resource_dump(int level, struct ldlm_resource *res)
> -{
> -	struct ldlm_lock *lock;
> -	unsigned int granted = 0;
> -
> -	BUILD_BUG_ON(RES_NAME_SIZE != 4);
> -
> -	if (!((libcfs_debug | D_ERROR) & level))
> -		return;
> -
> -	CDEBUG(level, "--- Resource: " DLDLMRES " (%p) refcount = %d\n",
> -	       PLDLMRES(res), res, atomic_read(&res->lr_refcount));
> -
> -	if (!list_empty(&res->lr_granted)) {
> -		CDEBUG(level, "Granted locks (in reverse order):\n");
> -		list_for_each_entry_reverse(lock, &res->lr_granted,
> -					    l_res_link) {
> -			LDLM_DEBUG_LIMIT(level, lock, "###");
> -			if (!(level & D_CANTMASK) &&
> -			    ++granted > ldlm_dump_granted_max) {
> -				CDEBUG(level,
> -				       "only dump %d granted locks to avoid DDOS.\n",
> -				       granted);
> -				break;
> -			}
> -		}
> -	}
> -	if (!list_empty(&res->lr_waiting)) {
> -		CDEBUG(level, "Waiting locks:\n");
> -		list_for_each_entry(lock, &res->lr_waiting, l_res_link)
> -			LDLM_DEBUG_LIMIT(level, lock, "###");
> -	}
> -}
> -EXPORT_SYMBOL(ldlm_resource_dump);
> diff --git a/drivers/staging/lustre/lustre/ptlrpc/Makefile b/drivers/staging/lustre/lustre/ptlrpc/Makefile
> index 77f8eabb2e28..aa152e652002 100644
> --- a/drivers/staging/lustre/lustre/ptlrpc/Makefile
> +++ b/drivers/staging/lustre/lustre/ptlrpc/Makefile
> @@ -3,14 +3,14 @@ subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/include
>  subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/lustre/include
>  
>  obj-$(CONFIG_LUSTRE_FS) += ptlrpc.o
> -LDLM := ../../lustre/ldlm/
>  
> -ldlm_objs := $(LDLM)l_lock.o $(LDLM)ldlm_lock.o
> -ldlm_objs += $(LDLM)ldlm_resource.o $(LDLM)ldlm_lib.o
> -ldlm_objs += $(LDLM)ldlm_plain.o $(LDLM)ldlm_extent.o
> -ldlm_objs += $(LDLM)ldlm_request.o $(LDLM)ldlm_lockd.o
> -ldlm_objs += $(LDLM)ldlm_flock.o $(LDLM)ldlm_inodebits.o
> -ldlm_objs += $(LDLM)ldlm_pool.o
> +ldlm_objs := l_lock.o ldlm_lock.o
> +ldlm_objs += ldlm_resource.o ldlm_lib.o
> +ldlm_objs += ldlm_plain.o ldlm_extent.o
> +ldlm_objs += ldlm_request.o ldlm_lockd.o
> +ldlm_objs += ldlm_flock.o ldlm_inodebits.o
> +ldlm_objs += ldlm_pool.o
> +
>  ptlrpc_objs := client.o recover.o connection.o niobuf.o pack_generic.o
>  ptlrpc_objs += events.o ptlrpc_module.o service.o pinger.o
>  ptlrpc_objs += llog_net.o llog_client.o import.o ptlrpcd.o
> diff --git a/drivers/staging/lustre/lustre/ptlrpc/l_lock.c b/drivers/staging/lustre/lustre/ptlrpc/l_lock.c
> new file mode 100644
> index 000000000000..296259aa51e6
> --- /dev/null
> +++ b/drivers/staging/lustre/lustre/ptlrpc/l_lock.c
> @@ -0,0 +1,73 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * GPL HEADER START
> + *
> + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 only,
> + * as published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful, but
> + * WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> + * General Public License version 2 for more details (a copy is included
> + * in the LICENSE file that accompanied this code).
> + *
> + * You should have received a copy of the GNU General Public License
> + * version 2 along with this program; If not, see
> + * http://www.gnu.org/licenses/gpl-2.0.html
> + *
> + * GPL HEADER END
> + */
> +/*
> + * Copyright (c) 2002, 2010, Oracle and/or its affiliates. All rights reserved.
> + * Use is subject to license terms.
> + *
> + * Copyright (c) 2012, Intel Corporation.
> + */
> +/*
> + * This file is part of Lustre, http://www.lustre.org/
> + * Lustre is a trademark of Sun Microsystems, Inc.
> + */
> +
> +#define DEBUG_SUBSYSTEM S_LDLM
> +
> +#include <lustre_dlm.h>
> +#include <lustre_lib.h>
> +
> +/**
> + * Lock a lock and its resource.
> + *
> + * LDLM locking uses resource to serialize access to locks
> + * but there is a case when we change resource of lock upon
> + * enqueue reply. We rely on lock->l_resource = new_res
> + * being an atomic operation.
> + */
> +struct ldlm_resource *lock_res_and_lock(struct ldlm_lock *lock)
> +				__acquires(&lock->l_lock)
> +				__acquires(&lock->l_resource->lr_lock)
> +{
> +	spin_lock(&lock->l_lock);
> +
> +	lock_res(lock->l_resource);
> +
> +	ldlm_set_res_locked(lock);
> +	return lock->l_resource;
> +}
> +EXPORT_SYMBOL(lock_res_and_lock);
> +
> +/**
> + * Unlock a lock and its resource previously locked with lock_res_and_lock
> + */
> +void unlock_res_and_lock(struct ldlm_lock *lock)
> +		__releases(&lock->l_resource->lr_lock)
> +		__releases(&lock->l_lock)
> +{
> +	/* on server-side resource of lock doesn't change */
> +	ldlm_clear_res_locked(lock);
> +
> +	unlock_res(lock->l_resource);
> +	spin_unlock(&lock->l_lock);
> +}
> +EXPORT_SYMBOL(unlock_res_and_lock);
> diff --git a/drivers/staging/lustre/lustre/ptlrpc/ldlm_extent.c b/drivers/staging/lustre/lustre/ptlrpc/ldlm_extent.c
> new file mode 100644
> index 000000000000..225c023b0bba
> --- /dev/null
> +++ b/drivers/staging/lustre/lustre/ptlrpc/ldlm_extent.c
> @@ -0,0 +1,206 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * GPL HEADER START
> + *
> + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 only,
> + * as published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful, but
> + * WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> + * General Public License version 2 for more details (a copy is included
> + * in the LICENSE file that accompanied this code).
> + *
> + * You should have received a copy of the GNU General Public License
> + * version 2 along with this program; If not, see
> + * http://www.gnu.org/licenses/gpl-2.0.html
> + *
> + * GPL HEADER END
> + */
> +/*
> + * Copyright (c) 2002, 2010, Oracle and/or its affiliates. All rights reserved.
> + * Use is subject to license terms.
> + *
> + * Copyright (c) 2010, 2012, Intel Corporation.
> + */
> +/*
> + * This file is part of Lustre, http://www.lustre.org/
> + * Lustre is a trademark of Sun Microsystems, Inc.
> + *
> + * lustre/ldlm/ldlm_extent.c
> + *
> + * Author: Peter Braam <braam at clusterfs.com>
> + * Author: Phil Schwan <phil at clusterfs.com>
> + */
> +
> +/**
> + * This file contains implementation of EXTENT lock type
> + *
> + * EXTENT lock type is for locking a contiguous range of values, represented
> + * by 64-bit starting and ending offsets (inclusive). There are several extent
> + * lock modes, some of which may be mutually incompatible. Extent locks are
> + * considered incompatible if their modes are incompatible and their extents
> + * intersect.  See the lock mode compatibility matrix in lustre_dlm.h.
> + */
> +
> +#define DEBUG_SUBSYSTEM S_LDLM
> +#include <lustre_dlm.h>
> +#include <obd_support.h>
> +#include <obd.h>
> +#include <obd_class.h>
> +#include <lustre_lib.h>
> +#include "ldlm_internal.h"
> +#include <linux/interval_tree_generic.h>
> +
> +#define START(node) ((node)->l_policy_data.l_extent.start)
> +#define LAST(node) ((node)->l_policy_data.l_extent.end)
> +INTERVAL_TREE_DEFINE(struct ldlm_lock, l_rb, __u64, __subtree_last,
> +		     START, LAST, static, extent);
> +
> +/* When a lock is cancelled by a client, the KMS may undergo change if this
> + * is the "highest lock".  This function returns the new KMS value.
> + * Caller must hold lr_lock already.
> + *
> + * NB: A lock on [x,y] protects a KMS of up to y + 1 bytes!
> + */
> +__u64 ldlm_extent_shift_kms(struct ldlm_lock *lock, __u64 old_kms)
> +{
> +	struct ldlm_resource *res = lock->l_resource;
> +	struct ldlm_lock *lck;
> +	__u64 kms = 0;
> +
> +	/* don't let another thread in ldlm_extent_shift_kms race in
> +	 * just after we finish and take our lock into account in its
> +	 * calculation of the kms
> +	 */
> +	ldlm_set_kms_ignore(lock);
> +
> +	list_for_each_entry(lck, &res->lr_granted, l_res_link) {
> +
> +		if (ldlm_is_kms_ignore(lck))
> +			continue;
> +
> +		if (lck->l_policy_data.l_extent.end >= old_kms)
> +			return old_kms;
> +
> +		/* This extent _has_ to be smaller than old_kms (checked above)
> +		 * so kms can only ever be smaller or the same as old_kms.
> +		 */
> +		if (lck->l_policy_data.l_extent.end + 1 > kms)
> +			kms = lck->l_policy_data.l_extent.end + 1;
> +	}
> +	LASSERTF(kms <= old_kms, "kms %llu old_kms %llu\n", kms, old_kms);
> +
> +	return kms;
> +}
> +EXPORT_SYMBOL(ldlm_extent_shift_kms);
> +
> +static inline int lock_mode_to_index(enum ldlm_mode mode)
> +{
> +	int index;
> +
> +	LASSERT(mode != 0);
> +	LASSERT(is_power_of_2(mode));
> +	for (index = -1; mode; index++)
> +		mode >>= 1;
> +	LASSERT(index < LCK_MODE_NUM);
> +	return index;
> +}
> +
> +/** Add newly granted lock into interval tree for the resource. */
> +void ldlm_extent_add_lock(struct ldlm_resource *res,
> +			  struct ldlm_lock *lock)
> +{
> +	struct ldlm_interval_tree *tree;
> +	int idx;
> +
> +	LASSERT(lock->l_granted_mode == lock->l_req_mode);
> +
> +	LASSERT(RB_EMPTY_NODE(&lock->l_rb));
> +
> +	idx = lock_mode_to_index(lock->l_granted_mode);
> +	LASSERT(lock->l_granted_mode == 1 << idx);
> +	LASSERT(lock->l_granted_mode == res->lr_itree[idx].lit_mode);
> +
> +	tree = &res->lr_itree[idx];
> +	extent_insert(lock, &tree->lit_root);
> +	tree->lit_size++;
> +
> +	/* even though we use interval tree to manage the extent lock, we also
> +	 * add the locks into grant list, for debug purpose, ..
> +	 */
> +	ldlm_resource_add_lock(res, &res->lr_granted, lock);
> +
> +	if (OBD_FAIL_CHECK(OBD_FAIL_LDLM_GRANT_CHECK)) {
> +		struct ldlm_lock *lck;
> +
> +		list_for_each_entry_reverse(lck, &res->lr_granted,
> +					    l_res_link) {
> +			if (lck == lock)
> +				continue;
> +			if (lockmode_compat(lck->l_granted_mode,
> +					    lock->l_granted_mode))
> +				continue;
> +			if (ldlm_extent_overlap(&lck->l_req_extent,
> +						&lock->l_req_extent)) {
> +				CDEBUG(D_ERROR,
> +				       "granting conflicting lock %p %p\n",
> +				       lck, lock);
> +				ldlm_resource_dump(D_ERROR, res);
> +				LBUG();
> +			}
> +		}
> +	}
> +}
> +
> +/** Remove cancelled lock from resource interval tree. */
> +void ldlm_extent_unlink_lock(struct ldlm_lock *lock)
> +{
> +	struct ldlm_resource *res = lock->l_resource;
> +	struct ldlm_interval_tree *tree;
> +	int idx;
> +
> +	if (RB_EMPTY_NODE(&lock->l_rb)) /* duplicate unlink */
> +		return;
> +
> +	idx = lock_mode_to_index(lock->l_granted_mode);
> +	LASSERT(lock->l_granted_mode == 1 << idx);
> +	tree = &res->lr_itree[idx];
> +
> +	tree->lit_size--;
> +	extent_remove(lock, &tree->lit_root);
> +}
> +
> +void ldlm_extent_policy_wire_to_local(const union ldlm_wire_policy_data *wpolicy,
> +				      union ldlm_policy_data *lpolicy)
> +{
> +	lpolicy->l_extent.start = wpolicy->l_extent.start;
> +	lpolicy->l_extent.end = wpolicy->l_extent.end;
> +	lpolicy->l_extent.gid = wpolicy->l_extent.gid;
> +}
> +
> +void ldlm_extent_policy_local_to_wire(const union ldlm_policy_data *lpolicy,
> +				      union ldlm_wire_policy_data *wpolicy)
> +{
> +	memset(wpolicy, 0, sizeof(*wpolicy));
> +	wpolicy->l_extent.start = lpolicy->l_extent.start;
> +	wpolicy->l_extent.end = lpolicy->l_extent.end;
> +	wpolicy->l_extent.gid = lpolicy->l_extent.gid;
> +}
> +
> +void ldlm_extent_search(struct rb_root_cached *root,
> +			__u64 start, __u64 end,
> +			bool (*matches)(struct ldlm_lock *lock, void *data),
> +			void *data)
> +{
> +	struct ldlm_lock *lock;
> +
> +	for (lock = extent_iter_first(root, start, end);
> +	     lock;
> +	     lock = extent_iter_next(lock, start, end))
> +		if (matches(lock, data))
> +			break;
> +}
> diff --git a/drivers/staging/lustre/lustre/ptlrpc/ldlm_flock.c b/drivers/staging/lustre/lustre/ptlrpc/ldlm_flock.c
> new file mode 100644
> index 000000000000..94f3b1e49896
> --- /dev/null
> +++ b/drivers/staging/lustre/lustre/ptlrpc/ldlm_flock.c
> @@ -0,0 +1,486 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * GPL HEADER START
> + *
> + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 only,
> + * as published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful, but
> + * WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> + * General Public License version 2 for more details (a copy is included
> + * in the LICENSE file that accompanied this code).
> + *
> + * You should have received a copy of the GNU General Public License
> + * version 2 along with this program; If not, see
> + * http://www.gnu.org/licenses/gpl-2.0.html
> + *
> + * GPL HEADER END
> + */
> +/*
> + * Copyright (c) 2003 Hewlett-Packard Development Company LP.
> + * Developed under the sponsorship of the US Government under
> + * Subcontract No. B514193
> + *
> + * Copyright (c) 2003, 2010, Oracle and/or its affiliates. All rights reserved.
> + * Use is subject to license terms.
> + *
> + * Copyright (c) 2010, 2012, Intel Corporation.
> + */
> +/*
> + * This file is part of Lustre, http://www.lustre.org/
> + * Lustre is a trademark of Sun Microsystems, Inc.
> + */
> +
> +/**
> + * This file implements POSIX lock type for Lustre.
> + * Its policy properties are start and end of extent and PID.
> + *
> + * These locks are only done through MDS due to POSIX semantics requiring
> + * e.g. that locks could be only partially released and as such split into
> + * two parts, and also that two adjacent locks from the same process may be
> + * merged into a single wider lock.
> + *
> + * Lock modes are mapped like this:
> + * PR and PW for READ and WRITE locks
> + * NL to request a releasing of a portion of the lock
> + *
> + * These flock locks never timeout.
> + */
> +
> +#define DEBUG_SUBSYSTEM S_LDLM
> +
> +#include <lustre_dlm.h>
> +#include <obd_support.h>
> +#include <obd_class.h>
> +#include <lustre_lib.h>
> +#include <linux/list.h>
> +#include "ldlm_internal.h"
> +
> +static inline int
> +ldlm_same_flock_owner(struct ldlm_lock *lock, struct ldlm_lock *new)
> +{
> +	return((new->l_policy_data.l_flock.owner ==
> +		lock->l_policy_data.l_flock.owner) &&
> +	       (new->l_export == lock->l_export));
> +}
> +
> +static inline int
> +ldlm_flocks_overlap(struct ldlm_lock *lock, struct ldlm_lock *new)
> +{
> +	return((new->l_policy_data.l_flock.start <=
> +		lock->l_policy_data.l_flock.end) &&
> +	       (new->l_policy_data.l_flock.end >=
> +		lock->l_policy_data.l_flock.start));
> +}
> +
> +static inline void
> +ldlm_flock_destroy(struct ldlm_lock *lock, enum ldlm_mode mode)
> +{
> +	LDLM_DEBUG(lock, "%s(mode: %d)",
> +		   __func__, mode);
> +
> +	list_del_init(&lock->l_res_link);
> +
> +	/* client side - set a flag to prevent sending a CANCEL */
> +	lock->l_flags |= LDLM_FL_LOCAL_ONLY | LDLM_FL_CBPENDING;
> +
> +	/* when reaching here, it is under lock_res_and_lock(). Thus,
> +	 * need call the nolock version of ldlm_lock_decref_internal
> +	 */
> +	ldlm_lock_decref_internal_nolock(lock, mode);
> +
> +	ldlm_lock_destroy_nolock(lock);
> +}
> +
> +/**
> + * Process a granting attempt for flock lock.
> + * Must be called under ns lock held.
> + *
> + * This function looks for any conflicts for \a lock in the granted or
> + * waiting queues. The lock is granted if no conflicts are found in
> + * either queue.
> + *
> + * It is also responsible for splitting a lock if a portion of the lock
> + * is released.
> + *
> + */
> +static int ldlm_process_flock_lock(struct ldlm_lock *req)
> +{
> +	struct ldlm_resource *res = req->l_resource;
> +	struct ldlm_namespace *ns = ldlm_res_to_ns(res);
> +	struct ldlm_lock *tmp;
> +	struct ldlm_lock *lock;
> +	struct ldlm_lock *new = req;
> +	struct ldlm_lock *new2 = NULL;
> +	enum ldlm_mode mode = req->l_req_mode;
> +	int added = (mode == LCK_NL);
> +	int splitted = 0;
> +	const struct ldlm_callback_suite null_cbs = { };
> +
> +	CDEBUG(D_DLMTRACE,
> +	       "owner %llu pid %u mode %u start %llu end %llu\n",
> +	       new->l_policy_data.l_flock.owner,
> +	       new->l_policy_data.l_flock.pid, mode,
> +	       req->l_policy_data.l_flock.start,
> +	       req->l_policy_data.l_flock.end);
> +
> +	/* No blocking ASTs are sent to the clients for
> +	 * Posix file & record locks
> +	 */
> +	req->l_blocking_ast = NULL;
> +
> +reprocess:
> +	/* This loop determines where this processes locks start
> +	 * in the resource lr_granted list.
> +	 */
> +	list_for_each_entry(lock, &res->lr_granted, l_res_link)
> +		if (ldlm_same_flock_owner(lock, req))
> +			break;
> +
> +	/* Scan the locks owned by this process to find the insertion point
> +	 * (as locks are ordered), and to handle overlaps.
> +	 * We may have to merge or split existing locks.
> +	 */
> +	list_for_each_entry_safe_from(lock, tmp, &res->lr_granted, l_res_link) {
> +
> +		if (!ldlm_same_flock_owner(lock, new))
> +			break;
> +
> +		if (lock->l_granted_mode == mode) {
> +			/* If the modes are the same then we need to process
> +			 * locks that overlap OR adjoin the new lock. The extra
> +			 * logic condition is necessary to deal with arithmetic
> +			 * overflow and underflow.
> +			 */
> +			if ((new->l_policy_data.l_flock.start >
> +			     (lock->l_policy_data.l_flock.end + 1)) &&
> +			    (lock->l_policy_data.l_flock.end != OBD_OBJECT_EOF))
> +				continue;
> +
> +			if ((new->l_policy_data.l_flock.end <
> +			     (lock->l_policy_data.l_flock.start - 1)) &&
> +			    (lock->l_policy_data.l_flock.start != 0))
> +				break;
> +
> +			if (new->l_policy_data.l_flock.start <
> +			    lock->l_policy_data.l_flock.start) {
> +				lock->l_policy_data.l_flock.start =
> +					new->l_policy_data.l_flock.start;
> +			} else {
> +				new->l_policy_data.l_flock.start =
> +					lock->l_policy_data.l_flock.start;
> +			}
> +
> +			if (new->l_policy_data.l_flock.end >
> +			    lock->l_policy_data.l_flock.end) {
> +				lock->l_policy_data.l_flock.end =
> +					new->l_policy_data.l_flock.end;
> +			} else {
> +				new->l_policy_data.l_flock.end =
> +					lock->l_policy_data.l_flock.end;
> +			}
> +
> +			if (added) {
> +				ldlm_flock_destroy(lock, mode);
> +			} else {
> +				new = lock;
> +				added = 1;
> +			}
> +			continue;
> +		}
> +
> +		if (new->l_policy_data.l_flock.start >
> +		    lock->l_policy_data.l_flock.end)
> +			continue;
> +
> +		if (new->l_policy_data.l_flock.end <
> +		    lock->l_policy_data.l_flock.start)
> +			break;
> +
> +		if (new->l_policy_data.l_flock.start <=
> +		    lock->l_policy_data.l_flock.start) {
> +			if (new->l_policy_data.l_flock.end <
> +			    lock->l_policy_data.l_flock.end) {
> +				lock->l_policy_data.l_flock.start =
> +					new->l_policy_data.l_flock.end + 1;
> +				break;
> +			}
> +			ldlm_flock_destroy(lock, lock->l_req_mode);
> +			continue;
> +		}
> +		if (new->l_policy_data.l_flock.end >=
> +		    lock->l_policy_data.l_flock.end) {
> +			lock->l_policy_data.l_flock.end =
> +				new->l_policy_data.l_flock.start - 1;
> +			continue;
> +		}
> +
> +		/* split the existing lock into two locks */
> +
> +		/* if this is an F_UNLCK operation then we could avoid
> +		 * allocating a new lock and use the req lock passed in
> +		 * with the request but this would complicate the reply
> +		 * processing since updates to req get reflected in the
> +		 * reply. The client side replays the lock request so
> +		 * it must see the original lock data in the reply.
> +		 */
> +
> +		/* XXX - if ldlm_lock_new() can sleep we should
> +		 * release the lr_lock, allocate the new lock,
> +		 * and restart processing this lock.
> +		 */
> +		if (!new2) {
> +			unlock_res_and_lock(req);
> +			new2 = ldlm_lock_create(ns, &res->lr_name, LDLM_FLOCK,
> +						lock->l_granted_mode, &null_cbs,
> +						NULL, 0, LVB_T_NONE);
> +			lock_res_and_lock(req);
> +			if (IS_ERR(new2)) {
> +				ldlm_flock_destroy(req, lock->l_granted_mode);
> +				return LDLM_ITER_STOP;
> +			}
> +			goto reprocess;
> +		}
> +
> +		splitted = 1;
> +
> +		new2->l_granted_mode = lock->l_granted_mode;
> +		new2->l_policy_data.l_flock.pid =
> +			new->l_policy_data.l_flock.pid;
> +		new2->l_policy_data.l_flock.owner =
> +			new->l_policy_data.l_flock.owner;
> +		new2->l_policy_data.l_flock.start =
> +			lock->l_policy_data.l_flock.start;
> +		new2->l_policy_data.l_flock.end =
> +			new->l_policy_data.l_flock.start - 1;
> +		lock->l_policy_data.l_flock.start =
> +			new->l_policy_data.l_flock.end + 1;
> +		new2->l_conn_export = lock->l_conn_export;
> +		if (lock->l_export)
> +			new2->l_export = class_export_lock_get(lock->l_export,
> +							       new2);
> +		ldlm_lock_addref_internal_nolock(new2,
> +						 lock->l_granted_mode);
> +
> +		/* insert new2 at lock */
> +		ldlm_resource_add_lock(res, &lock->l_res_link, new2);
> +		LDLM_LOCK_RELEASE(new2);
> +		break;
> +	}
> +
> +	/* if new2 is created but never used, destroy it*/
> +	if (splitted == 0 && new2)
> +		ldlm_lock_destroy_nolock(new2);
> +
> +	/* At this point we're granting the lock request. */
> +	req->l_granted_mode = req->l_req_mode;
> +
> +	if (!added) {
> +		list_del_init(&req->l_res_link);
> +		/* insert new lock before "lock", which might be the
> +		 * next lock for this owner, or might be the first
> +		 * lock for the next owner, or might not be a lock at
> +		 * all, but instead points at the head of the list
> +		 */
> +		ldlm_resource_add_lock(res, &lock->l_res_link, req);
> +	}
> +
> +	/* In case we're reprocessing the requested lock we can't destroy
> +	 * it until after calling ldlm_add_ast_work_item() above so that laawi()
> +	 * can bump the reference count on \a req. Otherwise \a req
> +	 * could be freed before the completion AST can be sent.
> +	 */
> +	if (added)
> +		ldlm_flock_destroy(req, mode);
> +
> +	ldlm_resource_dump(D_INFO, res);
> +	return LDLM_ITER_CONTINUE;
> +}
> +
> +/**
> + * Flock completion callback function.
> + *
> + * \param lock [in,out]: A lock to be handled
> + * \param flags    [in]: flags
> + * \param *data    [in]: ldlm_work_cp_ast_lock() will use ldlm_cb_set_arg
> + *
> + * \retval 0    : success
> + * \retval <0   : failure
> + */
> +int
> +ldlm_flock_completion_ast(struct ldlm_lock *lock, __u64 flags, void *data)
> +{
> +	struct file_lock		*getlk = lock->l_ast_data;
> +	int				rc = 0;
> +
> +	OBD_FAIL_TIMEOUT(OBD_FAIL_LDLM_CP_CB_WAIT2, 4);
> +	if (OBD_FAIL_PRECHECK(OBD_FAIL_LDLM_CP_CB_WAIT3)) {
> +		lock_res_and_lock(lock);
> +		lock->l_flags |= LDLM_FL_FAIL_LOC;
> +		unlock_res_and_lock(lock);
> +		OBD_FAIL_TIMEOUT(OBD_FAIL_LDLM_CP_CB_WAIT3, 4);
> +	}
> +	CDEBUG(D_DLMTRACE, "flags: 0x%llx data: %p getlk: %p\n",
> +	       flags, data, getlk);
> +
> +	LASSERT(flags != LDLM_FL_WAIT_NOREPROC);
> +
> +	if (flags & LDLM_FL_FAILED)
> +		goto granted;
> +
> +	if (!(flags & LDLM_FL_BLOCKED_MASK)) {
> +		if (!data)
> +			/* mds granted the lock in the reply */
> +			goto granted;
> +		/* CP AST RPC: lock get granted, wake it up */
> +		wake_up(&lock->l_waitq);
> +		return 0;
> +	}
> +
> +	LDLM_DEBUG(lock,
> +		   "client-side enqueue returned a blocked lock, sleeping");
> +
> +	/* Go to sleep until the lock is granted. */
> +	rc = l_wait_event_abortable(lock->l_waitq, is_granted_or_cancelled(lock));
> +
> +	if (rc) {
> +		lock_res_and_lock(lock);
> +
> +		/* client side - set flag to prevent lock from being put on LRU list */
> +		ldlm_set_cbpending(lock);
> +		unlock_res_and_lock(lock);
> +
> +		LDLM_DEBUG(lock, "client-side enqueue waking up: failed (%d)",
> +			   rc);
> +		return rc;
> +	}
> +
> +granted:
> +	OBD_FAIL_TIMEOUT(OBD_FAIL_LDLM_CP_CB_WAIT, 10);
> +
> +	if (OBD_FAIL_PRECHECK(OBD_FAIL_LDLM_CP_CB_WAIT4)) {
> +		lock_res_and_lock(lock);
> +		/* DEADLOCK is always set with CBPENDING */
> +		lock->l_flags |= LDLM_FL_FLOCK_DEADLOCK | LDLM_FL_CBPENDING;
> +		unlock_res_and_lock(lock);
> +		OBD_FAIL_TIMEOUT(OBD_FAIL_LDLM_CP_CB_WAIT4, 4);
> +	}
> +	if (OBD_FAIL_PRECHECK(OBD_FAIL_LDLM_CP_CB_WAIT5)) {
> +		lock_res_and_lock(lock);
> +		/* DEADLOCK is always set with CBPENDING */
> +		lock->l_flags |= LDLM_FL_FAIL_LOC |
> +				 LDLM_FL_FLOCK_DEADLOCK | LDLM_FL_CBPENDING;
> +		unlock_res_and_lock(lock);
> +		OBD_FAIL_TIMEOUT(OBD_FAIL_LDLM_CP_CB_WAIT5, 4);
> +	}
> +
> +	lock_res_and_lock(lock);
> +
> +	/*
> +	 * Protect against race where lock could have been just destroyed
> +	 * due to overlap in ldlm_process_flock_lock().
> +	 */
> +	if (ldlm_is_destroyed(lock)) {
> +		unlock_res_and_lock(lock);
> +		LDLM_DEBUG(lock, "client-side enqueue waking up: destroyed");
> +		/*
> +		 * An error is still to be returned, to propagate it up to
> +		 * ldlm_cli_enqueue_fini() caller.
> +		 */
> +		return -EIO;
> +	}
> +
> +	/* ldlm_lock_enqueue() has already placed lock on the granted list. */
> +	ldlm_resource_unlink_lock(lock);
> +
> +	/*
> +	 * Import invalidation. We need to actually release the lock
> +	 * references being held, so that it can go away. No point in
> +	 * holding the lock even if app still believes it has it, since
> +	 * server already dropped it anyway. Only for granted locks too.
> +	 */
> +	/* Do the same for DEADLOCK'ed locks. */
> +	if (ldlm_is_failed(lock) || ldlm_is_flock_deadlock(lock)) {
> +		int mode;
> +
> +		if (flags & LDLM_FL_TEST_LOCK)
> +			LASSERT(ldlm_is_test_lock(lock));
> +
> +		if (ldlm_is_test_lock(lock) || ldlm_is_flock_deadlock(lock))
> +			mode = getlk->fl_type;
> +		else
> +			mode = lock->l_granted_mode;
> +
> +		if (ldlm_is_flock_deadlock(lock)) {
> +			LDLM_DEBUG(lock,
> +				   "client-side enqueue deadlock received");
> +			rc = -EDEADLK;
> +		}
> +		ldlm_flock_destroy(lock, mode);
> +		unlock_res_and_lock(lock);
> +
> +		/* Need to wake up the waiter if we were evicted */
> +		wake_up(&lock->l_waitq);
> +
> +		/*
> +		 * An error is still to be returned, to propagate it up to
> +		 * ldlm_cli_enqueue_fini() caller.
> +		 */
> +		return rc ? : -EIO;
> +	}
> +
> +	LDLM_DEBUG(lock, "client-side enqueue granted");
> +
> +	if (flags & LDLM_FL_TEST_LOCK) {
> +		/* fcntl(F_GETLK) request */
> +		/* The old mode was saved in getlk->fl_type so that if the mode
> +		 * in the lock changes we can decref the appropriate refcount.
> +		 */
> +		LASSERT(ldlm_is_test_lock(lock));
> +		ldlm_flock_destroy(lock, getlk->fl_type);
> +		switch (lock->l_granted_mode) {
> +		case LCK_PR:
> +			getlk->fl_type = F_RDLCK;
> +			break;
> +		case LCK_PW:
> +			getlk->fl_type = F_WRLCK;
> +			break;
> +		default:
> +			getlk->fl_type = F_UNLCK;
> +		}
> +		getlk->fl_pid = -(pid_t)lock->l_policy_data.l_flock.pid;
> +		getlk->fl_start = (loff_t)lock->l_policy_data.l_flock.start;
> +		getlk->fl_end = (loff_t)lock->l_policy_data.l_flock.end;
> +	} else {
> +		/* We need to reprocess the lock to do merges or splits
> +		 * with existing locks owned by this process.
> +		 */
> +		ldlm_process_flock_lock(lock);
> +	}
> +	unlock_res_and_lock(lock);
> +	return rc;
> +}
> +EXPORT_SYMBOL(ldlm_flock_completion_ast);
> +
> +void ldlm_flock_policy_wire_to_local(const union ldlm_wire_policy_data *wpolicy,
> +				     union ldlm_policy_data *lpolicy)
> +{
> +	lpolicy->l_flock.start = wpolicy->l_flock.lfw_start;
> +	lpolicy->l_flock.end = wpolicy->l_flock.lfw_end;
> +	lpolicy->l_flock.pid = wpolicy->l_flock.lfw_pid;
> +	lpolicy->l_flock.owner = wpolicy->l_flock.lfw_owner;
> +}
> +
> +void ldlm_flock_policy_local_to_wire(const union ldlm_policy_data *lpolicy,
> +				     union ldlm_wire_policy_data *wpolicy)
> +{
> +	memset(wpolicy, 0, sizeof(*wpolicy));
> +	wpolicy->l_flock.lfw_start = lpolicy->l_flock.start;
> +	wpolicy->l_flock.lfw_end = lpolicy->l_flock.end;
> +	wpolicy->l_flock.lfw_pid = lpolicy->l_flock.pid;
> +	wpolicy->l_flock.lfw_owner = lpolicy->l_flock.owner;
> +}
> diff --git a/drivers/staging/lustre/lustre/ptlrpc/ldlm_inodebits.c b/drivers/staging/lustre/lustre/ptlrpc/ldlm_inodebits.c
> new file mode 100644
> index 000000000000..2926208cdfa1
> --- /dev/null
> +++ b/drivers/staging/lustre/lustre/ptlrpc/ldlm_inodebits.c
> @@ -0,0 +1,69 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * GPL HEADER START
> + *
> + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 only,
> + * as published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful, but
> + * WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> + * General Public License version 2 for more details (a copy is included
> + * in the LICENSE file that accompanied this code).
> + *
> + * You should have received a copy of the GNU General Public License
> + * version 2 along with this program; If not, see
> + * http://www.gnu.org/licenses/gpl-2.0.html
> + *
> + * GPL HEADER END
> + */
> +/*
> + * Copyright (c) 2004, 2010, Oracle and/or its affiliates. All rights reserved.
> + * Use is subject to license terms.
> + *
> + * Copyright (c) 2011, Intel Corporation.
> + */
> +/*
> + * This file is part of Lustre, http://www.lustre.org/
> + * Lustre is a trademark of Sun Microsystems, Inc.
> + *
> + * lustre/ldlm/ldlm_inodebits.c
> + *
> + * Author: Peter Braam <braam at clusterfs.com>
> + * Author: Phil Schwan <phil at clusterfs.com>
> + */
> +
> +/**
> + * This file contains implementation of IBITS lock type
> + *
> + * IBITS lock type contains a bit mask determining various properties of an
> + * object. The meanings of specific bits are specific to the caller and are
> + * opaque to LDLM code.
> + *
> + * Locks with intersecting bitmasks and conflicting lock modes (e.g.  LCK_PW)
> + * are considered conflicting.  See the lock mode compatibility matrix
> + * in lustre_dlm.h.
> + */
> +
> +#define DEBUG_SUBSYSTEM S_LDLM
> +
> +#include <lustre_dlm.h>
> +#include <obd_support.h>
> +#include <lustre_lib.h>
> +#include "ldlm_internal.h"
> +
> +void ldlm_ibits_policy_wire_to_local(const union ldlm_wire_policy_data *wpolicy,
> +				     union ldlm_policy_data *lpolicy)
> +{
> +	lpolicy->l_inodebits.bits = wpolicy->l_inodebits.bits;
> +}
> +
> +void ldlm_ibits_policy_local_to_wire(const union ldlm_policy_data *lpolicy,
> +				     union ldlm_wire_policy_data *wpolicy)
> +{
> +	memset(wpolicy, 0, sizeof(*wpolicy));
> +	wpolicy->l_inodebits.bits = lpolicy->l_inodebits.bits;
> +}
> diff --git a/drivers/staging/lustre/lustre/ptlrpc/ldlm_internal.h b/drivers/staging/lustre/lustre/ptlrpc/ldlm_internal.h
> new file mode 100644
> index 000000000000..60a15b963c8a
> --- /dev/null
> +++ b/drivers/staging/lustre/lustre/ptlrpc/ldlm_internal.h
> @@ -0,0 +1,329 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * GPL HEADER START
> + *
> + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 only,
> + * as published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful, but
> + * WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> + * General Public License version 2 for more details (a copy is included
> + * in the LICENSE file that accompanied this code).
> + *
> + * You should have received a copy of the GNU General Public License
> + * version 2 along with this program; If not, see
> + * http://www.gnu.org/licenses/gpl-2.0.html
> + *
> + * GPL HEADER END
> + */
> +/*
> + * Copyright (c) 2003, 2010, Oracle and/or its affiliates. All rights reserved.
> + * Use is subject to license terms.
> + *
> + * Copyright (c) 2011, 2015, Intel Corporation.
> + */
> +/*
> + * This file is part of Lustre, http://www.lustre.org/
> + * Lustre is a trademark of Sun Microsystems, Inc.
> + */
> +
> +#define MAX_STRING_SIZE 128
> +
> +extern int ldlm_srv_namespace_nr;
> +extern int ldlm_cli_namespace_nr;
> +extern struct mutex ldlm_srv_namespace_lock;
> +extern struct list_head ldlm_srv_namespace_list;
> +extern struct mutex ldlm_cli_namespace_lock;
> +extern struct list_head ldlm_cli_active_namespace_list;
> +
> +static inline int ldlm_namespace_nr_read(enum ldlm_side client)
> +{
> +	return client == LDLM_NAMESPACE_SERVER ?
> +		ldlm_srv_namespace_nr : ldlm_cli_namespace_nr;
> +}
> +
> +static inline void ldlm_namespace_nr_inc(enum ldlm_side client)
> +{
> +	if (client == LDLM_NAMESPACE_SERVER)
> +		ldlm_srv_namespace_nr++;
> +	else
> +		ldlm_cli_namespace_nr++;
> +}
> +
> +static inline void ldlm_namespace_nr_dec(enum ldlm_side client)
> +{
> +	if (client == LDLM_NAMESPACE_SERVER)
> +		ldlm_srv_namespace_nr--;
> +	else
> +		ldlm_cli_namespace_nr--;
> +}
> +
> +static inline struct list_head *ldlm_namespace_list(enum ldlm_side client)
> +{
> +	return client == LDLM_NAMESPACE_SERVER ?
> +		&ldlm_srv_namespace_list : &ldlm_cli_active_namespace_list;
> +}
> +
> +static inline struct mutex *ldlm_namespace_lock(enum ldlm_side client)
> +{
> +	return client == LDLM_NAMESPACE_SERVER ?
> +		&ldlm_srv_namespace_lock : &ldlm_cli_namespace_lock;
> +}
> +
> +/* ns_bref is the number of resources in this namespace */
> +static inline int ldlm_ns_empty(struct ldlm_namespace *ns)
> +{
> +	return atomic_read(&ns->ns_bref) == 0;
> +}
> +
> +void ldlm_namespace_move_to_active_locked(struct ldlm_namespace *ns,
> +					  enum ldlm_side client);
> +void ldlm_namespace_move_to_inactive_locked(struct ldlm_namespace *ns,
> +					    enum ldlm_side client);
> +struct ldlm_namespace *ldlm_namespace_first_locked(enum ldlm_side client);
> +
> +/* ldlm_request.c */
> +/* Cancel lru flag, it indicates we cancel aged locks. */
> +enum {
> +	LDLM_LRU_FLAG_AGED	= BIT(0), /* Cancel old non-LRU resize locks */
> +	LDLM_LRU_FLAG_PASSED	= BIT(1), /* Cancel passed number of locks. */
> +	LDLM_LRU_FLAG_SHRINK	= BIT(2), /* Cancel locks from shrinker. */
> +	LDLM_LRU_FLAG_LRUR	= BIT(3), /* Cancel locks from lru resize. */
> +	LDLM_LRU_FLAG_NO_WAIT	= BIT(4), /* Cancel locks w/o blocking (neither
> +					   * sending nor waiting for any rpcs)
> +					   */
> +	LDLM_LRU_FLAG_LRUR_NO_WAIT = BIT(5), /* LRUR + NO_WAIT */
> +};
> +
> +int ldlm_cancel_lru(struct ldlm_namespace *ns, int nr,
> +		    enum ldlm_cancel_flags sync, int flags);
> +int ldlm_cancel_lru_local(struct ldlm_namespace *ns,
> +			  struct list_head *cancels, int count, int max,
> +			  enum ldlm_cancel_flags cancel_flags, int flags);
> +extern unsigned int ldlm_enqueue_min;
> +extern unsigned int ldlm_cancel_unused_locks_before_replay;
> +
> +/* ldlm_lock.c */
> +
> +struct ldlm_cb_set_arg {
> +	struct ptlrpc_request_set	*set;
> +	int				 type; /* LDLM_{CP,BL,GL}_CALLBACK */
> +	atomic_t			 restart;
> +	struct list_head			*list;
> +	union ldlm_gl_desc		*gl_desc; /* glimpse AST descriptor */
> +};
> +
> +enum ldlm_desc_ast_t {
> +	LDLM_WORK_BL_AST,
> +	LDLM_WORK_CP_AST,
> +	LDLM_WORK_REVOKE_AST,
> +	LDLM_WORK_GL_AST
> +};
> +
> +void ldlm_grant_lock(struct ldlm_lock *lock, struct list_head *work_list);
> +int ldlm_fill_lvb(struct ldlm_lock *lock, struct req_capsule *pill,
> +		  enum req_location loc, void *data, int size);
> +struct ldlm_lock *
> +ldlm_lock_create(struct ldlm_namespace *ns, const struct ldlm_res_id *id,
> +		 enum ldlm_type type, enum ldlm_mode mode,
> +		 const struct ldlm_callback_suite *cbs,
> +		 void *data, __u32 lvb_len, enum lvb_type lvb_type);
> +enum ldlm_error ldlm_lock_enqueue(struct ldlm_namespace *ns,
> +				  struct ldlm_lock **lock, void *cookie,
> +				  __u64 *flags);
> +void ldlm_lock_addref_internal(struct ldlm_lock *lock, enum ldlm_mode mode);
> +void ldlm_lock_addref_internal_nolock(struct ldlm_lock *lock,
> +				      enum ldlm_mode mode);
> +void ldlm_lock_decref_internal(struct ldlm_lock *lock, enum ldlm_mode mode);
> +void ldlm_lock_decref_internal_nolock(struct ldlm_lock *lock,
> +				      enum ldlm_mode mode);
> +int ldlm_run_ast_work(struct ldlm_namespace *ns, struct list_head *rpc_list,
> +		      enum ldlm_desc_ast_t ast_type);
> +int ldlm_lock_remove_from_lru_check(struct ldlm_lock *lock, time_t last_use);
> +#define ldlm_lock_remove_from_lru(lock) ldlm_lock_remove_from_lru_check(lock, 0)
> +int ldlm_lock_remove_from_lru_nolock(struct ldlm_lock *lock);
> +void ldlm_lock_destroy_nolock(struct ldlm_lock *lock);
> +
> +/* ldlm_lockd.c */
> +int ldlm_bl_to_thread_lock(struct ldlm_namespace *ns, struct ldlm_lock_desc *ld,
> +			   struct ldlm_lock *lock);
> +int ldlm_bl_to_thread_list(struct ldlm_namespace *ns,
> +			   struct ldlm_lock_desc *ld,
> +			   struct list_head *cancels, int count,
> +			   enum ldlm_cancel_flags cancel_flags);
> +int ldlm_bl_thread_wakeup(void);
> +
> +void ldlm_handle_bl_callback(struct ldlm_namespace *ns,
> +			     struct ldlm_lock_desc *ld, struct ldlm_lock *lock);
> +
> +extern struct kmem_cache *ldlm_resource_slab;
> +extern struct kset *ldlm_ns_kset;
> +
> +/* ldlm_lockd.c & ldlm_lock.c */
> +extern struct kmem_cache *ldlm_lock_slab;
> +
> +/* ldlm_extent.c */
> +void ldlm_extent_add_lock(struct ldlm_resource *res, struct ldlm_lock *lock);
> +void ldlm_extent_unlink_lock(struct ldlm_lock *lock);
> +void ldlm_extent_search(struct rb_root_cached *root,
> +			__u64 start, __u64 end,
> +			bool (*matches)(struct ldlm_lock *lock, void *data),
> +			void *data);
> +
> +/* l_lock.c */
> +void l_check_ns_lock(struct ldlm_namespace *ns);
> +void l_check_no_ns_lock(struct ldlm_namespace *ns);
> +
> +extern struct dentry *ldlm_svc_debugfs_dir;
> +
> +struct ldlm_state {
> +	struct ptlrpc_service *ldlm_cb_service;
> +	struct ptlrpc_service *ldlm_cancel_service;
> +	struct ptlrpc_client *ldlm_client;
> +	struct ptlrpc_connection *ldlm_server_conn;
> +	struct ldlm_bl_pool *ldlm_bl_pool;
> +};
> +
> +/* ldlm_pool.c */
> +__u64 ldlm_pool_get_slv(struct ldlm_pool *pl);
> +void ldlm_pool_set_clv(struct ldlm_pool *pl, __u64 clv);
> +__u32 ldlm_pool_get_lvf(struct ldlm_pool *pl);
> +
> +int ldlm_init(void);
> +void ldlm_exit(void);
> +
> +enum ldlm_policy_res {
> +	LDLM_POLICY_CANCEL_LOCK,
> +	LDLM_POLICY_KEEP_LOCK,
> +	LDLM_POLICY_SKIP_LOCK
> +};
> +
> +#define LDLM_POOL_SYSFS_PRINT_int(v) sprintf(buf, "%d\n", v)
> +#define LDLM_POOL_SYSFS_SET_int(a, b) { a = b; }
> +#define LDLM_POOL_SYSFS_PRINT_u64(v) sprintf(buf, "%lld\n", v)
> +#define LDLM_POOL_SYSFS_SET_u64(a, b) { a = b; }
> +#define LDLM_POOL_SYSFS_PRINT_atomic(v) sprintf(buf, "%d\n", atomic_read(&v))
> +#define LDLM_POOL_SYSFS_SET_atomic(a, b) atomic_set(&a, b)
> +
> +#define LDLM_POOL_SYSFS_READER_SHOW(var, type)				    \
> +	static ssize_t var##_show(struct kobject *kobj,			    \
> +				  struct attribute *attr,		    \
> +				  char *buf)				    \
> +	{								    \
> +		struct ldlm_pool *pl = container_of(kobj, struct ldlm_pool, \
> +						    pl_kobj);		    \
> +		type tmp;						    \
> +									    \
> +		spin_lock(&pl->pl_lock);				    \
> +		tmp = pl->pl_##var;					    \
> +		spin_unlock(&pl->pl_lock);				    \
> +									    \
> +		return LDLM_POOL_SYSFS_PRINT_##type(tmp);		    \
> +	}								    \
> +	struct __##var##__dummy_read {; } /* semicolon catcher */
> +
> +#define LDLM_POOL_SYSFS_WRITER_STORE(var, type)				    \
> +	static ssize_t var##_store(struct kobject *kobj,		    \
> +				     struct attribute *attr,		    \
> +				     const char *buffer,		    \
> +				     size_t count)			    \
> +	{								    \
> +		struct ldlm_pool *pl = container_of(kobj, struct ldlm_pool, \
> +						    pl_kobj);		    \
> +		unsigned long tmp;					    \
> +		int rc;							    \
> +									    \
> +		rc = kstrtoul(buffer, 10, &tmp);			    \
> +		if (rc < 0) {						    \
> +			return rc;					    \
> +		}							    \
> +									    \
> +		spin_lock(&pl->pl_lock);				    \
> +		LDLM_POOL_SYSFS_SET_##type(pl->pl_##var, tmp);		    \
> +		spin_unlock(&pl->pl_lock);				    \
> +									    \
> +		return count;						    \
> +	}								    \
> +	struct __##var##__dummy_write {; } /* semicolon catcher */
> +
> +#define LDLM_POOL_SYSFS_READER_NOLOCK_SHOW(var, type)			    \
> +	static ssize_t var##_show(struct kobject *kobj,		    \
> +				    struct attribute *attr,		    \
> +				    char *buf)				    \
> +	{								    \
> +		struct ldlm_pool *pl = container_of(kobj, struct ldlm_pool, \
> +						    pl_kobj);		    \
> +									    \
> +		return LDLM_POOL_SYSFS_PRINT_##type(pl->pl_##var);	    \
> +	}								    \
> +	struct __##var##__dummy_read {; } /* semicolon catcher */
> +
> +#define LDLM_POOL_SYSFS_WRITER_NOLOCK_STORE(var, type)			    \
> +	static ssize_t var##_store(struct kobject *kobj,		    \
> +				     struct attribute *attr,		    \
> +				     const char *buffer,		    \
> +				     size_t count)			    \
> +	{								    \
> +		struct ldlm_pool *pl = container_of(kobj, struct ldlm_pool, \
> +						    pl_kobj);		    \
> +		unsigned long tmp;					    \
> +		int rc;							    \
> +									    \
> +		rc = kstrtoul(buffer, 10, &tmp);			    \
> +		if (rc < 0) {						    \
> +			return rc;					    \
> +		}							    \
> +									    \
> +		LDLM_POOL_SYSFS_SET_##type(pl->pl_##var, tmp);		    \
> +									    \
> +		return count;						    \
> +	}								    \
> +	struct __##var##__dummy_write {; } /* semicolon catcher */
> +
> +static inline int is_granted_or_cancelled(struct ldlm_lock *lock)
> +{
> +	int ret = 0;
> +
> +	lock_res_and_lock(lock);
> +	if ((lock->l_req_mode == lock->l_granted_mode) &&
> +	    !ldlm_is_cp_reqd(lock))
> +		ret = 1;
> +	else if (ldlm_is_failed(lock) || ldlm_is_cancel(lock))
> +		ret = 1;
> +	unlock_res_and_lock(lock);
> +
> +	return ret;
> +}
> +
> +typedef void (*ldlm_policy_wire_to_local_t)(const union ldlm_wire_policy_data *,
> +					    union ldlm_policy_data *);
> +
> +typedef void (*ldlm_policy_local_to_wire_t)(const union ldlm_policy_data *,
> +					    union ldlm_wire_policy_data *);
> +
> +void ldlm_plain_policy_wire_to_local(const union ldlm_wire_policy_data *wpolicy,
> +				     union ldlm_policy_data *lpolicy);
> +void ldlm_plain_policy_local_to_wire(const union ldlm_policy_data *lpolicy,
> +				     union ldlm_wire_policy_data *wpolicy);
> +void ldlm_ibits_policy_wire_to_local(const union ldlm_wire_policy_data *wpolicy,
> +				     union ldlm_policy_data *lpolicy);
> +void ldlm_ibits_policy_local_to_wire(const union ldlm_policy_data *lpolicy,
> +				     union ldlm_wire_policy_data *wpolicy);
> +void ldlm_extent_policy_wire_to_local(const union ldlm_wire_policy_data *wpolicy,
> +				      union ldlm_policy_data *lpolicy);
> +void ldlm_extent_policy_local_to_wire(const union ldlm_policy_data *lpolicy,
> +				      union ldlm_wire_policy_data *wpolicy);
> +void ldlm_flock_policy_wire_to_local(const union ldlm_wire_policy_data *wpolicy,
> +				     union ldlm_policy_data *lpolicy);
> +void ldlm_flock_policy_local_to_wire(const union ldlm_policy_data *lpolicy,
> +				     union ldlm_wire_policy_data *wpolicy);
> +
> +static inline bool ldlm_res_eq(const struct ldlm_res_id *res0,
> +			       const struct ldlm_res_id *res1)
> +{
> +	return memcmp(res0, res1, sizeof(*res0)) == 0;
> +}
> diff --git a/drivers/staging/lustre/lustre/ptlrpc/ldlm_lib.c b/drivers/staging/lustre/lustre/ptlrpc/ldlm_lib.c
> new file mode 100644
> index 000000000000..0aa4f234a4f4
> --- /dev/null
> +++ b/drivers/staging/lustre/lustre/ptlrpc/ldlm_lib.c
> @@ -0,0 +1,842 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * GPL HEADER START
> + *
> + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 only,
> + * as published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful, but
> + * WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> + * General Public License version 2 for more details (a copy is included
> + * in the LICENSE file that accompanied this code).
> + *
> + * You should have received a copy of the GNU General Public License
> + * version 2 along with this program; If not, see
> + * http://www.gnu.org/licenses/gpl-2.0.html
> + *
> + * GPL HEADER END
> + */
> +/*
> + * Copyright (c) 2003, 2010, Oracle and/or its affiliates. All rights reserved.
> + * Use is subject to license terms.
> + *
> + * Copyright (c) 2010, 2015, Intel Corporation.
> + */
> +/*
> + * This file is part of Lustre, http://www.lustre.org/
> + * Lustre is a trademark of Sun Microsystems, Inc.
> + */
> +
> +/**
> + * This file deals with various client/target related logic including recovery.
> + *
> + * TODO: This code more logically belongs in the ptlrpc module than in ldlm and
> + * should be moved.
> + */
> +
> +#define DEBUG_SUBSYSTEM S_LDLM
> +
> +#include <obd.h>
> +#include <obd_class.h>
> +#include <lustre_dlm.h>
> +#include <lustre_net.h>
> +#include <lustre_sec.h>
> +#include "ldlm_internal.h"
> +
> +/* @priority: If non-zero, move the selected connection to the list head.
> + * @create: If zero, only search in existing connections.
> + */
> +static int import_set_conn(struct obd_import *imp, struct obd_uuid *uuid,
> +			   int priority, int create)
> +{
> +	struct ptlrpc_connection *ptlrpc_conn;
> +	struct obd_import_conn *imp_conn = NULL, *item;
> +	int rc = 0;
> +
> +	if (!create && !priority) {
> +		CDEBUG(D_HA, "Nothing to do\n");
> +		return -EINVAL;
> +	}
> +
> +	ptlrpc_conn = ptlrpc_uuid_to_connection(uuid);
> +	if (!ptlrpc_conn) {
> +		CDEBUG(D_HA, "can't find connection %s\n", uuid->uuid);
> +		return -ENOENT;
> +	}
> +
> +	if (create) {
> +		imp_conn = kzalloc(sizeof(*imp_conn), GFP_NOFS);
> +		if (!imp_conn) {
> +			rc = -ENOMEM;
> +			goto out_put;
> +		}
> +	}
> +
> +	spin_lock(&imp->imp_lock);
> +	list_for_each_entry(item, &imp->imp_conn_list, oic_item) {
> +		if (obd_uuid_equals(uuid, &item->oic_uuid)) {
> +			if (priority) {
> +				list_del(&item->oic_item);
> +				list_add(&item->oic_item,
> +					 &imp->imp_conn_list);
> +				item->oic_last_attempt = 0;
> +			}
> +			CDEBUG(D_HA, "imp %p@%s: found existing conn %s%s\n",
> +			       imp, imp->imp_obd->obd_name, uuid->uuid,
> +			       (priority ? ", moved to head" : ""));
> +			spin_unlock(&imp->imp_lock);
> +			rc = 0;
> +			goto out_free;
> +		}
> +	}
> +	/* No existing import connection found for \a uuid. */
> +	if (create) {
> +		imp_conn->oic_conn = ptlrpc_conn;
> +		imp_conn->oic_uuid = *uuid;
> +		imp_conn->oic_last_attempt = 0;
> +		if (priority)
> +			list_add(&imp_conn->oic_item, &imp->imp_conn_list);
> +		else
> +			list_add_tail(&imp_conn->oic_item,
> +				      &imp->imp_conn_list);
> +		CDEBUG(D_HA, "imp %p@%s: add connection %s at %s\n",
> +		       imp, imp->imp_obd->obd_name, uuid->uuid,
> +		       (priority ? "head" : "tail"));
> +	} else {
> +		spin_unlock(&imp->imp_lock);
> +		rc = -ENOENT;
> +		goto out_free;
> +	}
> +
> +	spin_unlock(&imp->imp_lock);
> +	return 0;
> +out_free:
> +	kfree(imp_conn);
> +out_put:
> +	ptlrpc_connection_put(ptlrpc_conn);
> +	return rc;
> +}
> +
> +int import_set_conn_priority(struct obd_import *imp, struct obd_uuid *uuid)
> +{
> +	return import_set_conn(imp, uuid, 1, 0);
> +}
> +
> +int client_import_add_conn(struct obd_import *imp, struct obd_uuid *uuid,
> +			   int priority)
> +{
> +	return import_set_conn(imp, uuid, priority, 1);
> +}
> +EXPORT_SYMBOL(client_import_add_conn);
> +
> +int client_import_del_conn(struct obd_import *imp, struct obd_uuid *uuid)
> +{
> +	struct obd_import_conn *imp_conn;
> +	struct obd_export *dlmexp;
> +	int rc = -ENOENT;
> +
> +	spin_lock(&imp->imp_lock);
> +	if (list_empty(&imp->imp_conn_list)) {
> +		LASSERT(!imp->imp_connection);
> +		goto out;
> +	}
> +
> +	list_for_each_entry(imp_conn, &imp->imp_conn_list, oic_item) {
> +		if (!obd_uuid_equals(uuid, &imp_conn->oic_uuid))
> +			continue;
> +		LASSERT(imp_conn->oic_conn);
> +
> +		if (imp_conn == imp->imp_conn_current) {
> +			LASSERT(imp_conn->oic_conn == imp->imp_connection);
> +
> +			if (imp->imp_state != LUSTRE_IMP_CLOSED &&
> +			    imp->imp_state != LUSTRE_IMP_DISCON) {
> +				CERROR("can't remove current connection\n");
> +				rc = -EBUSY;
> +				goto out;
> +			}
> +
> +			ptlrpc_connection_put(imp->imp_connection);
> +			imp->imp_connection = NULL;
> +
> +			dlmexp = class_conn2export(&imp->imp_dlm_handle);
> +			if (dlmexp && dlmexp->exp_connection) {
> +				LASSERT(dlmexp->exp_connection ==
> +					imp_conn->oic_conn);
> +				ptlrpc_connection_put(dlmexp->exp_connection);
> +				dlmexp->exp_connection = NULL;
> +			}
> +
> +			if (dlmexp)
> +				class_export_put(dlmexp);
> +		}
> +
> +		list_del(&imp_conn->oic_item);
> +		ptlrpc_connection_put(imp_conn->oic_conn);
> +		kfree(imp_conn);
> +		CDEBUG(D_HA, "imp %p@%s: remove connection %s\n",
> +		       imp, imp->imp_obd->obd_name, uuid->uuid);
> +		rc = 0;
> +		break;
> +	}
> +out:
> +	spin_unlock(&imp->imp_lock);
> +	if (rc == -ENOENT)
> +		CERROR("connection %s not found\n", uuid->uuid);
> +	return rc;
> +}
> +EXPORT_SYMBOL(client_import_del_conn);
> +
> +/**
> + * Find conn UUID by peer NID. \a peer is a server NID. This function is used
> + * to find a conn uuid of \a imp which can reach \a peer.
> + */
> +int client_import_find_conn(struct obd_import *imp, lnet_nid_t peer,
> +			    struct obd_uuid *uuid)
> +{
> +	struct obd_import_conn *conn;
> +	int rc = -ENOENT;
> +
> +	spin_lock(&imp->imp_lock);
> +	list_for_each_entry(conn, &imp->imp_conn_list, oic_item) {
> +		/* Check if conn UUID does have this peer NID. */
> +		if (class_check_uuid(&conn->oic_uuid, peer)) {
> +			*uuid = conn->oic_uuid;
> +			rc = 0;
> +			break;
> +		}
> +	}
> +	spin_unlock(&imp->imp_lock);
> +	return rc;
> +}
> +EXPORT_SYMBOL(client_import_find_conn);
> +
> +void client_destroy_import(struct obd_import *imp)
> +{
> +	/* Drop security policy instance after all RPCs have finished/aborted
> +	 * to let all busy contexts be released.
> +	 */
> +	class_import_get(imp);
> +	class_destroy_import(imp);
> +	sptlrpc_import_sec_put(imp);
> +	class_import_put(imp);
> +}
> +EXPORT_SYMBOL(client_destroy_import);
> +
> +/* Configure an RPC client OBD device.
> + *
> + * lcfg parameters:
> + * 1 - client UUID
> + * 2 - server UUID
> + * 3 - inactive-on-startup
> + */
> +int client_obd_setup(struct obd_device *obddev, struct lustre_cfg *lcfg)
> +{
> +	struct client_obd *cli = &obddev->u.cli;
> +	struct obd_import *imp;
> +	struct obd_uuid server_uuid;
> +	int rq_portal, rp_portal, connect_op;
> +	char *name = obddev->obd_type->typ_name;
> +	enum ldlm_ns_type ns_type = LDLM_NS_TYPE_UNKNOWN;
> +	int rc;
> +
> +	/* In a more perfect world, we would hang a ptlrpc_client off of
> +	 * obd_type and just use the values from there.
> +	 */
> +	if (!strcmp(name, LUSTRE_OSC_NAME)) {
> +		rq_portal = OST_REQUEST_PORTAL;
> +		rp_portal = OSC_REPLY_PORTAL;
> +		connect_op = OST_CONNECT;
> +		cli->cl_sp_me = LUSTRE_SP_CLI;
> +		cli->cl_sp_to = LUSTRE_SP_OST;
> +		ns_type = LDLM_NS_TYPE_OSC;
> +	} else if (!strcmp(name, LUSTRE_MDC_NAME) ||
> +		   !strcmp(name, LUSTRE_LWP_NAME)) {
> +		rq_portal = MDS_REQUEST_PORTAL;
> +		rp_portal = MDC_REPLY_PORTAL;
> +		connect_op = MDS_CONNECT;
> +		cli->cl_sp_me = LUSTRE_SP_CLI;
> +		cli->cl_sp_to = LUSTRE_SP_MDT;
> +		ns_type = LDLM_NS_TYPE_MDC;
> +	} else if (!strcmp(name, LUSTRE_MGC_NAME)) {
> +		rq_portal = MGS_REQUEST_PORTAL;
> +		rp_portal = MGC_REPLY_PORTAL;
> +		connect_op = MGS_CONNECT;
> +		cli->cl_sp_me = LUSTRE_SP_MGC;
> +		cli->cl_sp_to = LUSTRE_SP_MGS;
> +		cli->cl_flvr_mgc.sf_rpc = SPTLRPC_FLVR_INVALID;
> +		ns_type = LDLM_NS_TYPE_MGC;
> +	} else {
> +		CERROR("unknown client OBD type \"%s\", can't setup\n",
> +		       name);
> +		return -EINVAL;
> +	}
> +
> +	if (LUSTRE_CFG_BUFLEN(lcfg, 1) < 1) {
> +		CERROR("requires a TARGET UUID\n");
> +		return -EINVAL;
> +	}
> +
> +	if (LUSTRE_CFG_BUFLEN(lcfg, 1) > 37) {
> +		CERROR("client UUID must be less than 38 characters\n");
> +		return -EINVAL;
> +	}
> +
> +	if (LUSTRE_CFG_BUFLEN(lcfg, 2) < 1) {
> +		CERROR("setup requires a SERVER UUID\n");
> +		return -EINVAL;
> +	}
> +
> +	if (LUSTRE_CFG_BUFLEN(lcfg, 2) > 37) {
> +		CERROR("target UUID must be less than 38 characters\n");
> +		return -EINVAL;
> +	}
> +
> +	init_rwsem(&cli->cl_sem);
> +	cli->cl_conn_count = 0;
> +	memcpy(server_uuid.uuid, lustre_cfg_buf(lcfg, 2),
> +	       min_t(unsigned int, LUSTRE_CFG_BUFLEN(lcfg, 2),
> +		     sizeof(server_uuid)));
> +
> +	cli->cl_dirty_pages = 0;
> +	cli->cl_avail_grant = 0;
> +	/* FIXME: Should limit this for the sum of all cl_dirty_max_pages. */
> +	/*
> +	 * cl_dirty_max_pages may be changed at connect time in
> +	 * ptlrpc_connect_interpret().
> +	 */
> +	client_adjust_max_dirty(cli);
> +	INIT_LIST_HEAD(&cli->cl_cache_waiters);
> +	INIT_LIST_HEAD(&cli->cl_loi_ready_list);
> +	INIT_LIST_HEAD(&cli->cl_loi_hp_ready_list);
> +	INIT_LIST_HEAD(&cli->cl_loi_write_list);
> +	INIT_LIST_HEAD(&cli->cl_loi_read_list);
> +	spin_lock_init(&cli->cl_loi_list_lock);
> +	atomic_set(&cli->cl_pending_w_pages, 0);
> +	atomic_set(&cli->cl_pending_r_pages, 0);
> +	cli->cl_r_in_flight = 0;
> +	cli->cl_w_in_flight = 0;
> +
> +	spin_lock_init(&cli->cl_read_rpc_hist.oh_lock);
> +	spin_lock_init(&cli->cl_write_rpc_hist.oh_lock);
> +	spin_lock_init(&cli->cl_read_page_hist.oh_lock);
> +	spin_lock_init(&cli->cl_write_page_hist.oh_lock);
> +	spin_lock_init(&cli->cl_read_offset_hist.oh_lock);
> +	spin_lock_init(&cli->cl_write_offset_hist.oh_lock);
> +
> +	/* lru for osc. */
> +	INIT_LIST_HEAD(&cli->cl_lru_osc);
> +	atomic_set(&cli->cl_lru_shrinkers, 0);
> +	atomic_long_set(&cli->cl_lru_busy, 0);
> +	atomic_long_set(&cli->cl_lru_in_list, 0);
> +	INIT_LIST_HEAD(&cli->cl_lru_list);
> +	spin_lock_init(&cli->cl_lru_list_lock);
> +	atomic_long_set(&cli->cl_unstable_count, 0);
> +	INIT_LIST_HEAD(&cli->cl_shrink_list);
> +
> +	init_waitqueue_head(&cli->cl_destroy_waitq);
> +	atomic_set(&cli->cl_destroy_in_flight, 0);
> +	/* Turn on checksumming by default. */
> +	cli->cl_checksum = 1;
> +	/*
> +	 * The supported checksum types will be worked out at connect time
> +	 * Set cl_chksum* to CRC32 for now to avoid returning screwed info
> +	 * through procfs.
> +	 */
> +	cli->cl_cksum_type = OBD_CKSUM_CRC32;
> +	cli->cl_supp_cksum_types = OBD_CKSUM_CRC32;
> +	atomic_set(&cli->cl_resends, OSC_DEFAULT_RESENDS);
> +
> +	/*
> +	 * Set it to possible maximum size. It may be reduced by ocd_brw_size
> +	 * from OFD after connecting.
> +	 */
> +	cli->cl_max_pages_per_rpc = PTLRPC_MAX_BRW_PAGES;
> +
> +	/*
> +	 * set cl_chunkbits default value to PAGE_CACHE_SHIFT,
> +	 * it will be updated at OSC connection time.
> +	 */
> +	cli->cl_chunkbits = PAGE_SHIFT;
> +
> +	if (!strcmp(name, LUSTRE_MDC_NAME))
> +		cli->cl_max_rpcs_in_flight = OBD_MAX_RIF_DEFAULT;
> +	else if (totalram_pages >> (20 - PAGE_SHIFT) <= 128 /* MB */)
> +		cli->cl_max_rpcs_in_flight = 2;
> +	else if (totalram_pages >> (20 - PAGE_SHIFT) <= 256 /* MB */)
> +		cli->cl_max_rpcs_in_flight = 3;
> +	else if (totalram_pages >> (20 - PAGE_SHIFT) <= 512 /* MB */)
> +		cli->cl_max_rpcs_in_flight = 4;
> +	else
> +		cli->cl_max_rpcs_in_flight = OBD_MAX_RIF_DEFAULT;
> +
> +	spin_lock_init(&cli->cl_mod_rpcs_lock);
> +	spin_lock_init(&cli->cl_mod_rpcs_hist.oh_lock);
> +	cli->cl_max_mod_rpcs_in_flight = 0;
> +	cli->cl_mod_rpcs_in_flight = 0;
> +	cli->cl_close_rpcs_in_flight = 0;
> +	init_waitqueue_head(&cli->cl_mod_rpcs_waitq);
> +	cli->cl_mod_tag_bitmap = NULL;
> +
> +	if (connect_op == MDS_CONNECT) {
> +		cli->cl_max_mod_rpcs_in_flight = cli->cl_max_rpcs_in_flight - 1;
> +		cli->cl_mod_tag_bitmap = kcalloc(BITS_TO_LONGS(OBD_MAX_RIF_MAX),
> +						 sizeof(long), GFP_NOFS);
> +		if (!cli->cl_mod_tag_bitmap) {
> +			rc = -ENOMEM;
> +			goto err;
> +		}
> +	}
> +
> +	rc = ldlm_get_ref();
> +	if (rc) {
> +		CERROR("ldlm_get_ref failed: %d\n", rc);
> +		goto err;
> +	}
> +
> +	ptlrpc_init_client(rq_portal, rp_portal, name,
> +			   &obddev->obd_ldlm_client);
> +
> +	imp = class_new_import(obddev);
> +	if (!imp) {
> +		rc = -ENOENT;
> +		goto err_ldlm;
> +	}
> +	imp->imp_client = &obddev->obd_ldlm_client;
> +	imp->imp_connect_op = connect_op;
> +	memcpy(cli->cl_target_uuid.uuid, lustre_cfg_buf(lcfg, 1),
> +	       LUSTRE_CFG_BUFLEN(lcfg, 1));
> +	class_import_put(imp);
> +
> +	rc = client_import_add_conn(imp, &server_uuid, 1);
> +	if (rc) {
> +		CERROR("can't add initial connection\n");
> +		goto err_import;
> +	}
> +
> +	cli->cl_import = imp;
> +	/* cli->cl_max_mds_easize updated by mdc_init_ea_size() */
> +	cli->cl_max_mds_easize = sizeof(struct lov_mds_md_v3);
> +
> +	if (LUSTRE_CFG_BUFLEN(lcfg, 3) > 0) {
> +		if (!strcmp(lustre_cfg_string(lcfg, 3), "inactive")) {
> +			CDEBUG(D_HA, "marking %s %s->%s as inactive\n",
> +			       name, obddev->obd_name,
> +			       cli->cl_target_uuid.uuid);
> +			spin_lock(&imp->imp_lock);
> +			imp->imp_deactive = 1;
> +			spin_unlock(&imp->imp_lock);
> +		}
> +	}
> +
> +	obddev->obd_namespace = ldlm_namespace_new(obddev, obddev->obd_name,
> +						   LDLM_NAMESPACE_CLIENT,
> +						   LDLM_NAMESPACE_GREEDY,
> +						   ns_type);
> +	if (!obddev->obd_namespace) {
> +		CERROR("Unable to create client namespace - %s\n",
> +		       obddev->obd_name);
> +		rc = -ENOMEM;
> +		goto err_import;
> +	}
> +
> +	return rc;
> +
> +err_import:
> +	class_destroy_import(imp);
> +err_ldlm:
> +	ldlm_put_ref();
> +err:
> +	kfree(cli->cl_mod_tag_bitmap);
> +	cli->cl_mod_tag_bitmap = NULL;
> +	return rc;
> +}
> +EXPORT_SYMBOL(client_obd_setup);
> +
> +int client_obd_cleanup(struct obd_device *obddev)
> +{
> +	struct client_obd *cli = &obddev->u.cli;
> +
> +	ldlm_namespace_free_post(obddev->obd_namespace);
> +	obddev->obd_namespace = NULL;
> +
> +	obd_cleanup_client_import(obddev);
> +	LASSERT(!obddev->u.cli.cl_import);
> +
> +	ldlm_put_ref();
> +
> +	kfree(cli->cl_mod_tag_bitmap);
> +	cli->cl_mod_tag_bitmap = NULL;
> +
> +	return 0;
> +}
> +EXPORT_SYMBOL(client_obd_cleanup);
> +
> +/* ->o_connect() method for client side (OSC and MDC and MGC) */
> +int client_connect_import(const struct lu_env *env,
> +			  struct obd_export **exp,
> +			  struct obd_device *obd, struct obd_uuid *cluuid,
> +			  struct obd_connect_data *data, void *localdata)
> +{
> +	struct client_obd       *cli    = &obd->u.cli;
> +	struct obd_import       *imp    = cli->cl_import;
> +	struct obd_connect_data *ocd;
> +	struct lustre_handle    conn    = { 0 };
> +	bool is_mdc = false;
> +	int		     rc;
> +
> +	*exp = NULL;
> +	down_write(&cli->cl_sem);
> +	if (cli->cl_conn_count > 0) {
> +		rc = -EALREADY;
> +		goto out_sem;
> +	}
> +
> +	rc = class_connect(&conn, obd, cluuid);
> +	if (rc)
> +		goto out_sem;
> +
> +	cli->cl_conn_count++;
> +	*exp = class_conn2export(&conn);
> +
> +	LASSERT(obd->obd_namespace);
> +
> +	imp->imp_dlm_handle = conn;
> +	rc = ptlrpc_init_import(imp);
> +	if (rc != 0)
> +		goto out_ldlm;
> +
> +	ocd = &imp->imp_connect_data;
> +	if (data) {
> +		*ocd = *data;
> +		is_mdc = !strncmp(imp->imp_obd->obd_type->typ_name,
> +				  LUSTRE_MDC_NAME, 3);
> +		if (is_mdc)
> +			data->ocd_connect_flags |= OBD_CONNECT_MULTIMODRPCS;
> +		imp->imp_connect_flags_orig = data->ocd_connect_flags;
> +	}
> +
> +	rc = ptlrpc_connect_import(imp);
> +	if (rc != 0) {
> +		if (data && is_mdc)
> +			data->ocd_connect_flags &= ~OBD_CONNECT_MULTIMODRPCS;
> +		LASSERT(imp->imp_state == LUSTRE_IMP_DISCON);
> +		goto out_ldlm;
> +	}
> +	LASSERT(*exp && (*exp)->exp_connection);
> +
> +	if (data) {
> +		LASSERTF((ocd->ocd_connect_flags & data->ocd_connect_flags) ==
> +			 ocd->ocd_connect_flags, "old %#llx, new %#llx\n",
> +			 data->ocd_connect_flags, ocd->ocd_connect_flags);
> +		data->ocd_connect_flags = ocd->ocd_connect_flags;
> +		/* clear the flag as it was not set and is not known
> +		 * by upper layers
> +		 */
> +		if (is_mdc)
> +			data->ocd_connect_flags &= ~OBD_CONNECT_MULTIMODRPCS;
> +	}
> +
> +	ptlrpc_pinger_add_import(imp);
> +
> +	if (rc) {
> +out_ldlm:
> +		cli->cl_conn_count--;
> +		class_disconnect(*exp);
> +		*exp = NULL;
> +	}
> +out_sem:
> +	up_write(&cli->cl_sem);
> +
> +	return rc;
> +}
> +EXPORT_SYMBOL(client_connect_import);
> +
> +int client_disconnect_export(struct obd_export *exp)
> +{
> +	struct obd_device *obd = class_exp2obd(exp);
> +	struct client_obd *cli;
> +	struct obd_import *imp;
> +	int rc = 0, err;
> +
> +	if (!obd) {
> +		CERROR("invalid export for disconnect: exp %p cookie %#llx\n",
> +		       exp, exp ? exp->exp_handle.h_cookie : -1);
> +		return -EINVAL;
> +	}
> +
> +	cli = &obd->u.cli;
> +	imp = cli->cl_import;
> +
> +	down_write(&cli->cl_sem);
> +	CDEBUG(D_INFO, "disconnect %s - %zu\n", obd->obd_name,
> +	       cli->cl_conn_count);
> +
> +	if (!cli->cl_conn_count) {
> +		CERROR("disconnecting disconnected device (%s)\n",
> +		       obd->obd_name);
> +		rc = -EINVAL;
> +		goto out_disconnect;
> +	}
> +
> +	cli->cl_conn_count--;
> +	if (cli->cl_conn_count) {
> +		rc = 0;
> +		goto out_disconnect;
> +	}
> +
> +	/* Mark import deactivated now, so we don't try to reconnect if any
> +	 * of the cleanup RPCs fails (e.g. LDLM cancel, etc).  We don't
> +	 * fully deactivate the import, or that would drop all requests.
> +	 */
> +	spin_lock(&imp->imp_lock);
> +	imp->imp_deactive = 1;
> +	spin_unlock(&imp->imp_lock);
> +
> +	/* Some non-replayable imports (MDS's OSCs) are pinged, so just
> +	 * delete it regardless.  (It's safe to delete an import that was
> +	 * never added.)
> +	 */
> +	(void)ptlrpc_pinger_del_import(imp);
> +
> +	if (obd->obd_namespace) {
> +		/* obd_force == local only */
> +		ldlm_cli_cancel_unused(obd->obd_namespace, NULL,
> +				       obd->obd_force ? LCF_LOCAL : 0, NULL);
> +		ldlm_namespace_free_prior(obd->obd_namespace, imp,
> +					  obd->obd_force);
> +	}
> +
> +	/* There's no need to hold sem while disconnecting an import,
> +	 * and it may actually cause deadlock in GSS.
> +	 */
> +	up_write(&cli->cl_sem);
> +	rc = ptlrpc_disconnect_import(imp, 0);
> +	down_write(&cli->cl_sem);
> +
> +	ptlrpc_invalidate_import(imp);
> +
> +out_disconnect:
> +	/* Use server style - class_disconnect should be always called for
> +	 * o_disconnect.
> +	 */
> +	err = class_disconnect(exp);
> +	if (!rc && err)
> +		rc = err;
> +
> +	up_write(&cli->cl_sem);
> +
> +	return rc;
> +}
> +EXPORT_SYMBOL(client_disconnect_export);
> +
> +/**
> + * Packs current SLV and Limit into \a req.
> + */
> +int target_pack_pool_reply(struct ptlrpc_request *req)
> +{
> +	struct obd_device *obd;
> +
> +	/* Check that we still have all structures alive as this may
> +	 * be some late RPC at shutdown time.
> +	 */
> +	if (unlikely(!req->rq_export || !req->rq_export->exp_obd ||
> +		     !exp_connect_lru_resize(req->rq_export))) {
> +		lustre_msg_set_slv(req->rq_repmsg, 0);
> +		lustre_msg_set_limit(req->rq_repmsg, 0);
> +		return 0;
> +	}
> +
> +	/* OBD is alive here as export is alive, which we checked above. */
> +	obd = req->rq_export->exp_obd;
> +
> +	read_lock(&obd->obd_pool_lock);
> +	lustre_msg_set_slv(req->rq_repmsg, obd->obd_pool_slv);
> +	lustre_msg_set_limit(req->rq_repmsg, obd->obd_pool_limit);
> +	read_unlock(&obd->obd_pool_lock);
> +
> +	return 0;
> +}
> +EXPORT_SYMBOL(target_pack_pool_reply);
> +
> +static int
> +target_send_reply_msg(struct ptlrpc_request *req, int rc, int fail_id)
> +{
> +	if (OBD_FAIL_CHECK_ORSET(fail_id & ~OBD_FAIL_ONCE, OBD_FAIL_ONCE)) {
> +		DEBUG_REQ(D_ERROR, req, "dropping reply");
> +		return -ECOMM;
> +	}
> +
> +	if (unlikely(rc)) {
> +		DEBUG_REQ(D_NET, req, "processing error (%d)", rc);
> +		req->rq_status = rc;
> +		return ptlrpc_send_error(req, 1);
> +	}
> +
> +	DEBUG_REQ(D_NET, req, "sending reply");
> +	return ptlrpc_send_reply(req, PTLRPC_REPLY_MAYBE_DIFFICULT);
> +}
> +
> +void target_send_reply(struct ptlrpc_request *req, int rc, int fail_id)
> +{
> +	struct ptlrpc_service_part *svcpt;
> +	int			netrc;
> +	struct ptlrpc_reply_state *rs;
> +	struct obd_export	 *exp;
> +
> +	if (req->rq_no_reply)
> +		return;
> +
> +	svcpt = req->rq_rqbd->rqbd_svcpt;
> +	rs = req->rq_reply_state;
> +	if (!rs || !rs->rs_difficult) {
> +		/* no notifiers */
> +		target_send_reply_msg(req, rc, fail_id);
> +		return;
> +	}
> +
> +	/* must be an export if locks saved */
> +	LASSERT(req->rq_export);
> +	/* req/reply consistent */
> +	LASSERT(rs->rs_svcpt == svcpt);
> +
> +	/* "fresh" reply */
> +	LASSERT(!rs->rs_scheduled);
> +	LASSERT(!rs->rs_scheduled_ever);
> +	LASSERT(!rs->rs_handled);
> +	LASSERT(!rs->rs_on_net);
> +	LASSERT(!rs->rs_export);
> +	LASSERT(list_empty(&rs->rs_obd_list));
> +	LASSERT(list_empty(&rs->rs_exp_list));
> +
> +	exp = class_export_get(req->rq_export);
> +
> +	/* disable reply scheduling while I'm setting up */
> +	rs->rs_scheduled = 1;
> +	rs->rs_on_net    = 1;
> +	rs->rs_xid       = req->rq_xid;
> +	rs->rs_transno   = req->rq_transno;
> +	rs->rs_export    = exp;
> +	rs->rs_opc       = lustre_msg_get_opc(req->rq_reqmsg);
> +
> +	spin_lock(&exp->exp_uncommitted_replies_lock);
> +	CDEBUG(D_NET, "rs transno = %llu, last committed = %llu\n",
> +	       rs->rs_transno, exp->exp_last_committed);
> +	if (rs->rs_transno > exp->exp_last_committed) {
> +		/* not committed already */
> +		list_add_tail(&rs->rs_obd_list,
> +			      &exp->exp_uncommitted_replies);
> +	}
> +	spin_unlock(&exp->exp_uncommitted_replies_lock);
> +
> +	spin_lock(&exp->exp_lock);
> +	list_add_tail(&rs->rs_exp_list, &exp->exp_outstanding_replies);
> +	spin_unlock(&exp->exp_lock);
> +
> +	netrc = target_send_reply_msg(req, rc, fail_id);
> +
> +	spin_lock(&svcpt->scp_rep_lock);
> +
> +	atomic_inc(&svcpt->scp_nreps_difficult);
> +
> +	if (netrc != 0) {
> +		/* error sending: reply is off the net.  Also we need +1
> +		 * reply ref until ptlrpc_handle_rs() is done
> +		 * with the reply state (if the send was successful, there
> +		 * would have been +1 ref for the net, which
> +		 * reply_out_callback leaves alone)
> +		 */
> +		rs->rs_on_net = 0;
> +		ptlrpc_rs_addref(rs);
> +	}
> +
> +	spin_lock(&rs->rs_lock);
> +	if (rs->rs_transno <= exp->exp_last_committed ||
> +	    (!rs->rs_on_net && !rs->rs_no_ack) ||
> +	    list_empty(&rs->rs_exp_list) ||     /* completed already */
> +	    list_empty(&rs->rs_obd_list)) {
> +		CDEBUG(D_HA, "Schedule reply immediately\n");
> +		ptlrpc_dispatch_difficult_reply(rs);
> +	} else {
> +		list_add(&rs->rs_list, &svcpt->scp_rep_active);
> +		rs->rs_scheduled = 0;	/* allow notifier to schedule */
> +	}
> +	spin_unlock(&rs->rs_lock);
> +	spin_unlock(&svcpt->scp_rep_lock);
> +}
> +EXPORT_SYMBOL(target_send_reply);
> +
> +enum ldlm_mode lck_compat_array[] = {
> +	[LCK_EX]	= LCK_COMPAT_EX,
> +	[LCK_PW]	= LCK_COMPAT_PW,
> +	[LCK_PR]	= LCK_COMPAT_PR,
> +	[LCK_CW]	= LCK_COMPAT_CW,
> +	[LCK_CR]	= LCK_COMPAT_CR,
> +	[LCK_NL]	= LCK_COMPAT_NL,
> +	[LCK_GROUP]	= LCK_COMPAT_GROUP,
> +	[LCK_COS]	= LCK_COMPAT_COS,
> +};
> +
> +/**
> + * Rather arbitrary mapping from LDLM error codes to errno values. This should
> + * not escape to the user level.
> + */
> +int ldlm_error2errno(enum ldlm_error error)
> +{
> +	int result;
> +
> +	switch (error) {
> +	case ELDLM_OK:
> +	case ELDLM_LOCK_MATCHED:
> +		result = 0;
> +		break;
> +	case ELDLM_LOCK_CHANGED:
> +		result = -ESTALE;
> +		break;
> +	case ELDLM_LOCK_ABORTED:
> +		result = -ENAVAIL;
> +		break;
> +	case ELDLM_LOCK_REPLACED:
> +		result = -ESRCH;
> +		break;
> +	case ELDLM_NO_LOCK_DATA:
> +		result = -ENOENT;
> +		break;
> +	case ELDLM_NAMESPACE_EXISTS:
> +		result = -EEXIST;
> +		break;
> +	case ELDLM_BAD_NAMESPACE:
> +		result = -EBADF;
> +		break;
> +	default:
> +		if (((int)error) < 0)  /* cast to signed type */
> +			result = error; /* as enum ldlm_error can be unsigned */
> +		else {
> +			CERROR("Invalid DLM result code: %d\n", error);
> +			result = -EPROTO;
> +		}
> +	}
> +	return result;
> +}
> +EXPORT_SYMBOL(ldlm_error2errno);
> +
> +#if LUSTRE_TRACKS_LOCK_EXP_REFS
> +void ldlm_dump_export_locks(struct obd_export *exp)
> +{
> +	spin_lock(&exp->exp_locks_list_guard);
> +	if (!list_empty(&exp->exp_locks_list)) {
> +		struct ldlm_lock *lock;
> +
> +		CERROR("dumping locks for export %p,ignore if the unmount doesn't hang\n",
> +		       exp);
> +		list_for_each_entry(lock, &exp->exp_locks_list,
> +				    l_exp_refs_link)
> +			LDLM_ERROR(lock, "lock:");
> +	}
> +	spin_unlock(&exp->exp_locks_list_guard);
> +}
> +#endif
> diff --git a/drivers/staging/lustre/lustre/ptlrpc/ldlm_lock.c b/drivers/staging/lustre/lustre/ptlrpc/ldlm_lock.c
> new file mode 100644
> index 000000000000..2fb2e088dc87
> --- /dev/null
> +++ b/drivers/staging/lustre/lustre/ptlrpc/ldlm_lock.c
> @@ -0,0 +1,2103 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * GPL HEADER START
> + *
> + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 only,
> + * as published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful, but
> + * WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> + * General Public License version 2 for more details (a copy is included
> + * in the LICENSE file that accompanied this code).
> + *
> + * You should have received a copy of the GNU General Public License
> + * version 2 along with this program; If not, see
> + * http://www.gnu.org/licenses/gpl-2.0.html
> + *
> + * GPL HEADER END
> + */
> +/*
> + * Copyright (c) 2002, 2010, Oracle and/or its affiliates. All rights reserved.
> + * Use is subject to license terms.
> + *
> + * Copyright (c) 2010, 2015, Intel Corporation.
> + */
> +/*
> + * This file is part of Lustre, http://www.lustre.org/
> + * Lustre is a trademark of Sun Microsystems, Inc.
> + *
> + * lustre/ldlm/ldlm_lock.c
> + *
> + * Author: Peter Braam <braam at clusterfs.com>
> + * Author: Phil Schwan <phil at clusterfs.com>
> + */
> +
> +#define DEBUG_SUBSYSTEM S_LDLM
> +
> +#include <lustre_intent.h>
> +#include <lustre_swab.h>
> +#include <obd_class.h>
> +#include "ldlm_internal.h"
> +
> +/* lock types */
> +char *ldlm_lockname[] = {
> +	[0]		= "--",
> +	[LCK_EX]	= "EX",
> +	[LCK_PW]	= "PW",
> +	[LCK_PR]	= "PR",
> +	[LCK_CW]	= "CW",
> +	[LCK_CR]	= "CR",
> +	[LCK_NL]	= "NL",
> +	[LCK_GROUP]	= "GROUP",
> +	[LCK_COS]	= "COS",
> +};
> +EXPORT_SYMBOL(ldlm_lockname);
> +
> +static char *ldlm_typename[] = {
> +	[LDLM_PLAIN]	= "PLN",
> +	[LDLM_EXTENT]	= "EXT",
> +	[LDLM_FLOCK]	= "FLK",
> +	[LDLM_IBITS]	= "IBT",
> +};
> +
> +static ldlm_policy_wire_to_local_t ldlm_policy_wire_to_local[] = {
> +	[LDLM_PLAIN - LDLM_MIN_TYPE]	= ldlm_plain_policy_wire_to_local,
> +	[LDLM_EXTENT - LDLM_MIN_TYPE]	= ldlm_extent_policy_wire_to_local,
> +	[LDLM_FLOCK - LDLM_MIN_TYPE]	= ldlm_flock_policy_wire_to_local,
> +	[LDLM_IBITS - LDLM_MIN_TYPE]	= ldlm_ibits_policy_wire_to_local,
> +};
> +
> +static ldlm_policy_local_to_wire_t ldlm_policy_local_to_wire[] = {
> +	[LDLM_PLAIN - LDLM_MIN_TYPE]	= ldlm_plain_policy_local_to_wire,
> +	[LDLM_EXTENT - LDLM_MIN_TYPE]	= ldlm_extent_policy_local_to_wire,
> +	[LDLM_FLOCK - LDLM_MIN_TYPE]	= ldlm_flock_policy_local_to_wire,
> +	[LDLM_IBITS - LDLM_MIN_TYPE]	= ldlm_ibits_policy_local_to_wire,
> +};
> +
> +/**
> + * Converts lock policy from local format to on the wire lock_desc format
> + */
> +static void ldlm_convert_policy_to_wire(enum ldlm_type type,
> +					const union ldlm_policy_data *lpolicy,
> +					union ldlm_wire_policy_data *wpolicy)
> +{
> +	ldlm_policy_local_to_wire_t convert;
> +
> +	convert = ldlm_policy_local_to_wire[type - LDLM_MIN_TYPE];
> +
> +	convert(lpolicy, wpolicy);
> +}
> +
> +/**
> + * Converts lock policy from on the wire lock_desc format to local format
> + */
> +void ldlm_convert_policy_to_local(struct obd_export *exp, enum ldlm_type type,
> +				  const union ldlm_wire_policy_data *wpolicy,
> +				  union ldlm_policy_data *lpolicy)
> +{
> +	ldlm_policy_wire_to_local_t convert;
> +
> +	convert = ldlm_policy_wire_to_local[type - LDLM_MIN_TYPE];
> +
> +	convert(wpolicy, lpolicy);
> +}
> +
> +const char *ldlm_it2str(enum ldlm_intent_flags it)
> +{
> +	switch (it) {
> +	case IT_OPEN:
> +		return "open";
> +	case IT_CREAT:
> +		return "creat";
> +	case (IT_OPEN | IT_CREAT):
> +		return "open|creat";
> +	case IT_READDIR:
> +		return "readdir";
> +	case IT_GETATTR:
> +		return "getattr";
> +	case IT_LOOKUP:
> +		return "lookup";
> +	case IT_UNLINK:
> +		return "unlink";
> +	case IT_GETXATTR:
> +		return "getxattr";
> +	case IT_LAYOUT:
> +		return "layout";
> +	default:
> +		CERROR("Unknown intent 0x%08x\n", it);
> +		return "UNKNOWN";
> +	}
> +}
> +EXPORT_SYMBOL(ldlm_it2str);
> +
> +/*
> + * REFCOUNTED LOCK OBJECTS
> + */
> +
> +/**
> + * Get a reference on a lock.
> + *
> + * Lock refcounts, during creation:
> + *   - one special one for allocation, dec'd only once in destroy
> + *   - one for being a lock that's in-use
> + *   - one for the addref associated with a new lock
> + */
> +struct ldlm_lock *ldlm_lock_get(struct ldlm_lock *lock)
> +{
> +	atomic_inc(&lock->l_refc);
> +	return lock;
> +}
> +EXPORT_SYMBOL(ldlm_lock_get);
> +
> +/**
> + * Release lock reference.
> + *
> + * Also frees the lock if it was last reference.
> + */
> +void ldlm_lock_put(struct ldlm_lock *lock)
> +{
> +	LASSERT(lock->l_resource != LP_POISON);
> +	LASSERT(atomic_read(&lock->l_refc) > 0);
> +	if (atomic_dec_and_test(&lock->l_refc)) {
> +		struct ldlm_resource *res;
> +
> +		LDLM_DEBUG(lock,
> +			   "final lock_put on destroyed lock, freeing it.");
> +
> +		res = lock->l_resource;
> +		LASSERT(ldlm_is_destroyed(lock));
> +		LASSERT(list_empty(&lock->l_res_link));
> +		LASSERT(list_empty(&lock->l_pending_chain));
> +
> +		lprocfs_counter_decr(ldlm_res_to_ns(res)->ns_stats,
> +				     LDLM_NSS_LOCKS);
> +		lu_ref_del(&res->lr_reference, "lock", lock);
> +		ldlm_resource_putref(res);
> +		lock->l_resource = NULL;
> +		if (lock->l_export) {
> +			class_export_lock_put(lock->l_export, lock);
> +			lock->l_export = NULL;
> +		}
> +
> +		kfree(lock->l_lvb_data);
> +
> +		lu_ref_fini(&lock->l_reference);
> +		OBD_FREE_RCU(lock, sizeof(*lock), &lock->l_handle);
> +	}
> +}
> +EXPORT_SYMBOL(ldlm_lock_put);
> +
> +/**
> + * Removes LDLM lock \a lock from LRU. Assumes LRU is already locked.
> + */
> +int ldlm_lock_remove_from_lru_nolock(struct ldlm_lock *lock)
> +{
> +	int rc = 0;
> +
> +	if (!list_empty(&lock->l_lru)) {
> +		struct ldlm_namespace *ns = ldlm_lock_to_ns(lock);
> +
> +		LASSERT(lock->l_resource->lr_type != LDLM_FLOCK);
> +		list_del_init(&lock->l_lru);
> +		LASSERT(ns->ns_nr_unused > 0);
> +		ns->ns_nr_unused--;
> +		rc = 1;
> +	}
> +	return rc;
> +}
> +
> +/**
> + * Removes LDLM lock \a lock from LRU. Obtains the LRU lock first.
> + *
> + * If \a last_use is non-zero, it will remove the lock from LRU only if
> + * it matches lock's l_last_used.
> + *
> + * \retval 0 if \a last_use is set, the lock is not in LRU list or \a last_use
> + *           doesn't match lock's l_last_used;
> + *           otherwise, the lock hasn't been in the LRU list.
> + * \retval 1 the lock was in LRU list and removed.
> + */
> +int ldlm_lock_remove_from_lru_check(struct ldlm_lock *lock, time_t last_use)
> +{
> +	struct ldlm_namespace *ns = ldlm_lock_to_ns(lock);
> +	int rc = 0;
> +
> +	spin_lock(&ns->ns_lock);
> +	if (last_use == 0 || last_use == lock->l_last_used)
> +		rc = ldlm_lock_remove_from_lru_nolock(lock);
> +	spin_unlock(&ns->ns_lock);
> +
> +	return rc;
> +}
> +
> +/**
> + * Adds LDLM lock \a lock to namespace LRU. Assumes LRU is already locked.
> + */
> +static void ldlm_lock_add_to_lru_nolock(struct ldlm_lock *lock)
> +{
> +	struct ldlm_namespace *ns = ldlm_lock_to_ns(lock);
> +
> +	lock->l_last_used = jiffies;
> +	LASSERT(list_empty(&lock->l_lru));
> +	LASSERT(lock->l_resource->lr_type != LDLM_FLOCK);
> +	list_add_tail(&lock->l_lru, &ns->ns_unused_list);
> +	ldlm_clear_skipped(lock);
> +	LASSERT(ns->ns_nr_unused >= 0);
> +	ns->ns_nr_unused++;
> +}
> +
> +/**
> + * Adds LDLM lock \a lock to namespace LRU. Obtains necessary LRU locks
> + * first.
> + */
> +static void ldlm_lock_add_to_lru(struct ldlm_lock *lock)
> +{
> +	struct ldlm_namespace *ns = ldlm_lock_to_ns(lock);
> +
> +	spin_lock(&ns->ns_lock);
> +	ldlm_lock_add_to_lru_nolock(lock);
> +	spin_unlock(&ns->ns_lock);
> +}
> +
> +/**
> + * Moves LDLM lock \a lock that is already in namespace LRU to the tail of
> + * the LRU. Performs necessary LRU locking
> + */
> +static void ldlm_lock_touch_in_lru(struct ldlm_lock *lock)
> +{
> +	struct ldlm_namespace *ns = ldlm_lock_to_ns(lock);
> +
> +	spin_lock(&ns->ns_lock);
> +	if (!list_empty(&lock->l_lru)) {
> +		ldlm_lock_remove_from_lru_nolock(lock);
> +		ldlm_lock_add_to_lru_nolock(lock);
> +	}
> +	spin_unlock(&ns->ns_lock);
> +}
> +
> +/**
> + * Helper to destroy a locked lock.
> + *
> + * Used by ldlm_lock_destroy and ldlm_lock_destroy_nolock
> + * Must be called with l_lock and lr_lock held.
> + *
> + * Does not actually free the lock data, but rather marks the lock as
> + * destroyed by setting l_destroyed field in the lock to 1.  Destroys a
> + * handle->lock association too, so that the lock can no longer be found
> + * and removes the lock from LRU list.  Actual lock freeing occurs when
> + * last lock reference goes away.
> + *
> + * Original comment (of some historical value):
> + * This used to have a 'strict' flag, which recovery would use to mark an
> + * in-use lock as needing-to-die.  Lest I am ever tempted to put it back, I
> + * shall explain why it's gone: with the new hash table scheme, once you call
> + * ldlm_lock_destroy, you can never drop your final references on this lock.
> + * Because it's not in the hash table anymore.  -phil
> + */
> +static int ldlm_lock_destroy_internal(struct ldlm_lock *lock)
> +{
> +	if (lock->l_readers || lock->l_writers) {
> +		LDLM_ERROR(lock, "lock still has references");
> +		LBUG();
> +	}
> +
> +	if (!list_empty(&lock->l_res_link)) {
> +		LDLM_ERROR(lock, "lock still on resource");
> +		LBUG();
> +	}
> +
> +	if (ldlm_is_destroyed(lock)) {
> +		LASSERT(list_empty(&lock->l_lru));
> +		return 0;
> +	}
> +	ldlm_set_destroyed(lock);
> +
> +	ldlm_lock_remove_from_lru(lock);
> +	class_handle_unhash(&lock->l_handle);
> +
> +	return 1;
> +}
> +
> +/**
> + * Destroys a LDLM lock \a lock. Performs necessary locking first.
> + */
> +static void ldlm_lock_destroy(struct ldlm_lock *lock)
> +{
> +	int first;
> +
> +	lock_res_and_lock(lock);
> +	first = ldlm_lock_destroy_internal(lock);
> +	unlock_res_and_lock(lock);
> +
> +	/* drop reference from hashtable only for first destroy */
> +	if (first) {
> +		lu_ref_del(&lock->l_reference, "hash", lock);
> +		LDLM_LOCK_RELEASE(lock);
> +	}
> +}
> +
> +/**
> + * Destroys a LDLM lock \a lock that is already locked.
> + */
> +void ldlm_lock_destroy_nolock(struct ldlm_lock *lock)
> +{
> +	int first;
> +
> +	first = ldlm_lock_destroy_internal(lock);
> +	/* drop reference from hashtable only for first destroy */
> +	if (first) {
> +		lu_ref_del(&lock->l_reference, "hash", lock);
> +		LDLM_LOCK_RELEASE(lock);
> +	}
> +}
> +
> +/* this is called by portals_handle2object with the handle lock taken */
> +static void lock_handle_addref(void *lock)
> +{
> +	LDLM_LOCK_GET((struct ldlm_lock *)lock);
> +}
> +
> +static void lock_handle_free(void *lock, int size)
> +{
> +	LASSERT(size == sizeof(struct ldlm_lock));
> +	kmem_cache_free(ldlm_lock_slab, lock);
> +}
> +
> +static struct portals_handle_ops lock_handle_ops = {
> +	.hop_addref = lock_handle_addref,
> +	.hop_free   = lock_handle_free,
> +};
> +
> +/**
> + *
> + * Allocate and initialize new lock structure.
> + *
> + * usage: pass in a resource on which you have done ldlm_resource_get
> + *	new lock will take over the refcount.
> + * returns: lock with refcount 2 - one for current caller and one for remote
> + */
> +static struct ldlm_lock *ldlm_lock_new(struct ldlm_resource *resource)
> +{
> +	struct ldlm_lock *lock;
> +
> +	LASSERT(resource);
> +
> +	lock = kmem_cache_zalloc(ldlm_lock_slab, GFP_NOFS);
> +	if (!lock)
> +		return NULL;
> +
> +	spin_lock_init(&lock->l_lock);
> +	lock->l_resource = resource;
> +	lu_ref_add(&resource->lr_reference, "lock", lock);
> +
> +	atomic_set(&lock->l_refc, 2);
> +	INIT_LIST_HEAD(&lock->l_res_link);
> +	INIT_LIST_HEAD(&lock->l_lru);
> +	INIT_LIST_HEAD(&lock->l_pending_chain);
> +	INIT_LIST_HEAD(&lock->l_bl_ast);
> +	INIT_LIST_HEAD(&lock->l_cp_ast);
> +	INIT_LIST_HEAD(&lock->l_rk_ast);
> +	init_waitqueue_head(&lock->l_waitq);
> +	lock->l_blocking_lock = NULL;
> +	INIT_LIST_HEAD(&lock->l_sl_mode);
> +	INIT_LIST_HEAD(&lock->l_sl_policy);
> +	RB_CLEAR_NODE(&lock->l_rb);
> +
> +	lprocfs_counter_incr(ldlm_res_to_ns(resource)->ns_stats,
> +			     LDLM_NSS_LOCKS);
> +	INIT_LIST_HEAD(&lock->l_handle.h_link);
> +	class_handle_hash(&lock->l_handle, &lock_handle_ops);
> +
> +	lu_ref_init(&lock->l_reference);
> +	lu_ref_add(&lock->l_reference, "hash", lock);
> +	lock->l_callback_timeout = 0;
> +
> +#if LUSTRE_TRACKS_LOCK_EXP_REFS
> +	INIT_LIST_HEAD(&lock->l_exp_refs_link);
> +	lock->l_exp_refs_nr = 0;
> +	lock->l_exp_refs_target = NULL;
> +#endif
> +
> +	return lock;
> +}
> +
> +/**
> + * Moves LDLM lock \a lock to another resource.
> + * This is used on client when server returns some other lock than requested
> + * (typically as a result of intent operation)
> + */
> +int ldlm_lock_change_resource(struct ldlm_namespace *ns, struct ldlm_lock *lock,
> +			      const struct ldlm_res_id *new_resid)
> +{
> +	struct ldlm_resource *oldres = lock->l_resource;
> +	struct ldlm_resource *newres;
> +	int type;
> +
> +	lock_res_and_lock(lock);
> +	if (memcmp(new_resid, &lock->l_resource->lr_name,
> +		   sizeof(lock->l_resource->lr_name)) == 0) {
> +		/* Nothing to do */
> +		unlock_res_and_lock(lock);
> +		return 0;
> +	}
> +
> +	LASSERT(new_resid->name[0] != 0);
> +
> +	/* This function assumes that the lock isn't on any lists */
> +	LASSERT(list_empty(&lock->l_res_link));
> +
> +	type = oldres->lr_type;
> +	unlock_res_and_lock(lock);
> +
> +	newres = ldlm_resource_get(ns, NULL, new_resid, type, 1);
> +	if (IS_ERR(newres))
> +		return PTR_ERR(newres);
> +
> +	lu_ref_add(&newres->lr_reference, "lock", lock);
> +	/*
> +	 * To flip the lock from the old to the new resource, lock, oldres and
> +	 * newres have to be locked. Resource spin-locks are nested within
> +	 * lock->l_lock, and are taken in the memory address order to avoid
> +	 * dead-locks.
> +	 */
> +	spin_lock(&lock->l_lock);
> +	oldres = lock->l_resource;
> +	if (oldres < newres) {
> +		lock_res(oldres);
> +		lock_res_nested(newres, LRT_NEW);
> +	} else {
> +		lock_res(newres);
> +		lock_res_nested(oldres, LRT_NEW);
> +	}
> +	LASSERT(memcmp(new_resid, &oldres->lr_name,
> +		       sizeof(oldres->lr_name)) != 0);
> +	lock->l_resource = newres;
> +	unlock_res(oldres);
> +	unlock_res_and_lock(lock);
> +
> +	/* ...and the flowers are still standing! */
> +	lu_ref_del(&oldres->lr_reference, "lock", lock);
> +	ldlm_resource_putref(oldres);
> +
> +	return 0;
> +}
> +
> +/** \defgroup ldlm_handles LDLM HANDLES
> + * Ways to get hold of locks without any addresses.
> + * @{
> + */
> +
> +/**
> + * Fills in handle for LDLM lock \a lock into supplied \a lockh
> + * Does not take any references.
> + */
> +void ldlm_lock2handle(const struct ldlm_lock *lock, struct lustre_handle *lockh)
> +{
> +	lockh->cookie = lock->l_handle.h_cookie;
> +}
> +EXPORT_SYMBOL(ldlm_lock2handle);
> +
> +/**
> + * Obtain a lock reference by handle.
> + *
> + * if \a flags: atomically get the lock and set the flags.
> + *	      Return NULL if flag already set
> + */
> +struct ldlm_lock *__ldlm_handle2lock(const struct lustre_handle *handle,
> +				     __u64 flags)
> +{
> +	struct ldlm_lock *lock;
> +
> +	LASSERT(handle);
> +
> +	lock = class_handle2object(handle->cookie, NULL);
> +	if (!lock)
> +		return NULL;
> +
> +	if (lock->l_export && lock->l_export->exp_failed) {
> +		CDEBUG(D_INFO, "lock export failed: lock %p, exp %p\n",
> +		       lock, lock->l_export);
> +		LDLM_LOCK_PUT(lock);
> +		return NULL;
> +	}
> +
> +	/* It's unlikely but possible that someone marked the lock as
> +	 * destroyed after we did handle2object on it
> +	 */
> +	if (flags == 0 && !ldlm_is_destroyed(lock)) {
> +		lu_ref_add(&lock->l_reference, "handle", current);
> +		return lock;
> +	}
> +
> +	lock_res_and_lock(lock);
> +
> +	LASSERT(lock->l_resource);
> +
> +	lu_ref_add_atomic(&lock->l_reference, "handle", current);
> +	if (unlikely(ldlm_is_destroyed(lock))) {
> +		unlock_res_and_lock(lock);
> +		CDEBUG(D_INFO, "lock already destroyed: lock %p\n", lock);
> +		LDLM_LOCK_PUT(lock);
> +		return NULL;
> +	}
> +
> +	if (flags) {
> +		if (lock->l_flags & flags) {
> +			unlock_res_and_lock(lock);
> +			LDLM_LOCK_PUT(lock);
> +			return NULL;
> +		}
> +
> +		lock->l_flags |= flags;
> +	}
> +
> +	unlock_res_and_lock(lock);
> +	return lock;
> +}
> +EXPORT_SYMBOL(__ldlm_handle2lock);
> +/** @} ldlm_handles */
> +
> +/**
> + * Fill in "on the wire" representation for given LDLM lock into supplied
> + * lock descriptor \a desc structure.
> + */
> +void ldlm_lock2desc(struct ldlm_lock *lock, struct ldlm_lock_desc *desc)
> +{
> +	ldlm_res2desc(lock->l_resource, &desc->l_resource);
> +	desc->l_req_mode = lock->l_req_mode;
> +	desc->l_granted_mode = lock->l_granted_mode;
> +	ldlm_convert_policy_to_wire(lock->l_resource->lr_type,
> +				    &lock->l_policy_data,
> +				    &desc->l_policy_data);
> +}
> +
> +/**
> + * Add a lock to list of conflicting locks to send AST to.
> + *
> + * Only add if we have not sent a blocking AST to the lock yet.
> + */
> +static void ldlm_add_bl_work_item(struct ldlm_lock *lock, struct ldlm_lock *new,
> +				  struct list_head *work_list)
> +{
> +	if (!ldlm_is_ast_sent(lock)) {
> +		LDLM_DEBUG(lock, "lock incompatible; sending blocking AST.");
> +		ldlm_set_ast_sent(lock);
> +		/* If the enqueuing client said so, tell the AST recipient to
> +		 * discard dirty data, rather than writing back.
> +		 */
> +		if (ldlm_is_ast_discard_data(new))
> +			ldlm_set_discard_data(lock);
> +		LASSERT(list_empty(&lock->l_bl_ast));
> +		list_add(&lock->l_bl_ast, work_list);
> +		LDLM_LOCK_GET(lock);
> +		LASSERT(!lock->l_blocking_lock);
> +		lock->l_blocking_lock = LDLM_LOCK_GET(new);
> +	}
> +}
> +
> +/**
> + * Add a lock to list of just granted locks to send completion AST to.
> + */
> +static void ldlm_add_cp_work_item(struct ldlm_lock *lock,
> +				  struct list_head *work_list)
> +{
> +	if (!ldlm_is_cp_reqd(lock)) {
> +		ldlm_set_cp_reqd(lock);
> +		LDLM_DEBUG(lock, "lock granted; sending completion AST.");
> +		LASSERT(list_empty(&lock->l_cp_ast));
> +		list_add(&lock->l_cp_ast, work_list);
> +		LDLM_LOCK_GET(lock);
> +	}
> +}
> +
> +/**
> + * Aggregator function to add AST work items into a list. Determines
> + * what sort of an AST work needs to be done and calls the proper
> + * adding function.
> + * Must be called with lr_lock held.
> + */
> +static void ldlm_add_ast_work_item(struct ldlm_lock *lock,
> +				   struct ldlm_lock *new,
> +				   struct list_head *work_list)
> +{
> +	check_res_locked(lock->l_resource);
> +	if (new)
> +		ldlm_add_bl_work_item(lock, new, work_list);
> +	else
> +		ldlm_add_cp_work_item(lock, work_list);
> +}
> +
> +/**
> + * Add specified reader/writer reference to LDLM lock with handle \a lockh.
> + * r/w reference type is determined by \a mode
> + * Calls ldlm_lock_addref_internal.
> + */
> +void ldlm_lock_addref(const struct lustre_handle *lockh, enum ldlm_mode mode)
> +{
> +	struct ldlm_lock *lock;
> +
> +	lock = ldlm_handle2lock(lockh);
> +	LASSERTF(lock, "Non-existing lock: %llx\n", lockh->cookie);
> +	ldlm_lock_addref_internal(lock, mode);
> +	LDLM_LOCK_PUT(lock);
> +}
> +EXPORT_SYMBOL(ldlm_lock_addref);
> +
> +/**
> + * Helper function.
> + * Add specified reader/writer reference to LDLM lock \a lock.
> + * r/w reference type is determined by \a mode
> + * Removes lock from LRU if it is there.
> + * Assumes the LDLM lock is already locked.
> + */
> +void ldlm_lock_addref_internal_nolock(struct ldlm_lock *lock,
> +				      enum ldlm_mode mode)
> +{
> +	ldlm_lock_remove_from_lru(lock);
> +	if (mode & (LCK_NL | LCK_CR | LCK_PR)) {
> +		lock->l_readers++;
> +		lu_ref_add_atomic(&lock->l_reference, "reader", lock);
> +	}
> +	if (mode & (LCK_EX | LCK_CW | LCK_PW | LCK_GROUP | LCK_COS)) {
> +		lock->l_writers++;
> +		lu_ref_add_atomic(&lock->l_reference, "writer", lock);
> +	}
> +	LDLM_LOCK_GET(lock);
> +	lu_ref_add_atomic(&lock->l_reference, "user", lock);
> +	LDLM_DEBUG(lock, "ldlm_lock_addref(%s)", ldlm_lockname[mode]);
> +}
> +
> +/**
> + * Attempts to add reader/writer reference to a lock with handle \a lockh, and
> + * fails if lock is already LDLM_FL_CBPENDING or destroyed.
> + *
> + * \retval 0 success, lock was addref-ed
> + *
> + * \retval -EAGAIN lock is being canceled.
> + */
> +int ldlm_lock_addref_try(const struct lustre_handle *lockh, enum ldlm_mode mode)
> +{
> +	struct ldlm_lock *lock;
> +	int	       result;
> +
> +	result = -EAGAIN;
> +	lock = ldlm_handle2lock(lockh);
> +	if (lock) {
> +		lock_res_and_lock(lock);
> +		if (lock->l_readers != 0 || lock->l_writers != 0 ||
> +		    !ldlm_is_cbpending(lock)) {
> +			ldlm_lock_addref_internal_nolock(lock, mode);
> +			result = 0;
> +		}
> +		unlock_res_and_lock(lock);
> +		LDLM_LOCK_PUT(lock);
> +	}
> +	return result;
> +}
> +EXPORT_SYMBOL(ldlm_lock_addref_try);
> +
> +/**
> + * Add specified reader/writer reference to LDLM lock \a lock.
> + * Locks LDLM lock and calls ldlm_lock_addref_internal_nolock to do the work.
> + * Only called for local locks.
> + */
> +void ldlm_lock_addref_internal(struct ldlm_lock *lock, enum ldlm_mode mode)
> +{
> +	lock_res_and_lock(lock);
> +	ldlm_lock_addref_internal_nolock(lock, mode);
> +	unlock_res_and_lock(lock);
> +}
> +
> +/**
> + * Removes reader/writer reference for LDLM lock \a lock.
> + * Assumes LDLM lock is already locked.
> + * only called in ldlm_flock_destroy and for local locks.
> + * Does NOT add lock to LRU if no r/w references left to accommodate flock locks
> + * that cannot be placed in LRU.
> + */
> +void ldlm_lock_decref_internal_nolock(struct ldlm_lock *lock,
> +				      enum ldlm_mode mode)
> +{
> +	LDLM_DEBUG(lock, "ldlm_lock_decref(%s)", ldlm_lockname[mode]);
> +	if (mode & (LCK_NL | LCK_CR | LCK_PR)) {
> +		LASSERT(lock->l_readers > 0);
> +		lu_ref_del(&lock->l_reference, "reader", lock);
> +		lock->l_readers--;
> +	}
> +	if (mode & (LCK_EX | LCK_CW | LCK_PW | LCK_GROUP | LCK_COS)) {
> +		LASSERT(lock->l_writers > 0);
> +		lu_ref_del(&lock->l_reference, "writer", lock);
> +		lock->l_writers--;
> +	}
> +
> +	lu_ref_del(&lock->l_reference, "user", lock);
> +	LDLM_LOCK_RELEASE(lock);    /* matches the LDLM_LOCK_GET() in addref */
> +}
> +
> +/**
> + * Removes reader/writer reference for LDLM lock \a lock.
> + * Locks LDLM lock first.
> + * If the lock is determined to be client lock on a client and r/w refcount
> + * drops to zero and the lock is not blocked, the lock is added to LRU lock
> + * on the namespace.
> + * For blocked LDLM locks if r/w count drops to zero, blocking_ast is called.
> + */
> +void ldlm_lock_decref_internal(struct ldlm_lock *lock, enum ldlm_mode mode)
> +{
> +	struct ldlm_namespace *ns;
> +
> +	lock_res_and_lock(lock);
> +
> +	ns = ldlm_lock_to_ns(lock);
> +
> +	ldlm_lock_decref_internal_nolock(lock, mode);
> +
> +	if ((ldlm_is_local(lock) || lock->l_req_mode == LCK_GROUP) &&
> +	    !lock->l_readers && !lock->l_writers) {
> +		/* If this is a local lock on a server namespace and this was
> +		 * the last reference, cancel the lock.
> +		 *
> +		 * Group locks are special:
> +		 * They must not go in LRU, but they are not called back
> +		 * like non-group locks, instead they are manually released.
> +		 * They have an l_writers reference which they keep until
> +		 * they are manually released, so we remove them when they have
> +		 * no more reader or writer references. - LU-6368
> +		 */
> +		ldlm_set_cbpending(lock);
> +	}
> +
> +	if (!lock->l_readers && !lock->l_writers && ldlm_is_cbpending(lock)) {
> +		/* If we received a blocked AST and this was the last reference,
> +		 * run the callback.
> +		 */
> +		LDLM_DEBUG(lock, "final decref done on cbpending lock");
> +
> +		LDLM_LOCK_GET(lock); /* dropped by bl thread */
> +		ldlm_lock_remove_from_lru(lock);
> +		unlock_res_and_lock(lock);
> +
> +		if (ldlm_is_fail_loc(lock))
> +			OBD_RACE(OBD_FAIL_LDLM_CP_BL_RACE);
> +
> +		if (ldlm_is_atomic_cb(lock) ||
> +		    ldlm_bl_to_thread_lock(ns, NULL, lock) != 0)
> +			ldlm_handle_bl_callback(ns, NULL, lock);
> +	} else if (!lock->l_readers && !lock->l_writers &&
> +		   !ldlm_is_no_lru(lock) && !ldlm_is_bl_ast(lock)) {
> +		LDLM_DEBUG(lock, "add lock into lru list");
> +
> +		/* If this is a client-side namespace and this was the last
> +		 * reference, put it on the LRU.
> +		 */
> +		ldlm_lock_add_to_lru(lock);
> +		unlock_res_and_lock(lock);
> +
> +		if (ldlm_is_fail_loc(lock))
> +			OBD_RACE(OBD_FAIL_LDLM_CP_BL_RACE);
> +
> +		/* Call ldlm_cancel_lru() only if EARLY_CANCEL and LRU RESIZE
> +		 * are not supported by the server, otherwise, it is done on
> +		 * enqueue.
> +		 */
> +		if (!exp_connect_cancelset(lock->l_conn_export) &&
> +		    !ns_connect_lru_resize(ns))
> +			ldlm_cancel_lru(ns, 0, LCF_ASYNC, 0);
> +	} else {
> +		LDLM_DEBUG(lock, "do not add lock into lru list");
> +		unlock_res_and_lock(lock);
> +	}
> +}
> +
> +/**
> + * Decrease reader/writer refcount for LDLM lock with handle \a lockh
> + */
> +void ldlm_lock_decref(const struct lustre_handle *lockh, enum ldlm_mode mode)
> +{
> +	struct ldlm_lock *lock = __ldlm_handle2lock(lockh, 0);
> +
> +	LASSERTF(lock, "Non-existing lock: %#llx\n", lockh->cookie);
> +	ldlm_lock_decref_internal(lock, mode);
> +	LDLM_LOCK_PUT(lock);
> +}
> +EXPORT_SYMBOL(ldlm_lock_decref);
> +
> +/**
> + * Decrease reader/writer refcount for LDLM lock with handle
> + * \a lockh and mark it for subsequent cancellation once r/w refcount
> + * drops to zero instead of putting into LRU.
> + */
> +void ldlm_lock_decref_and_cancel(const struct lustre_handle *lockh,
> +				 enum ldlm_mode mode)
> +{
> +	struct ldlm_lock *lock = __ldlm_handle2lock(lockh, 0);
> +
> +	LASSERT(lock);
> +
> +	LDLM_DEBUG(lock, "ldlm_lock_decref(%s)", ldlm_lockname[mode]);
> +	lock_res_and_lock(lock);
> +	ldlm_set_cbpending(lock);
> +	unlock_res_and_lock(lock);
> +	ldlm_lock_decref_internal(lock, mode);
> +	LDLM_LOCK_PUT(lock);
> +}
> +EXPORT_SYMBOL(ldlm_lock_decref_and_cancel);
> +
> +struct sl_insert_point {
> +	struct list_head *res_link;
> +	struct list_head *mode_link;
> +	struct list_head *policy_link;
> +};
> +
> +/**
> + * Finds a position to insert the new lock into granted lock list.
> + *
> + * Used for locks eligible for skiplist optimization.
> + *
> + * Parameters:
> + *      queue [input]:  the granted list where search acts on;
> + *      req [input]:    the lock whose position to be located;
> + *      prev [output]:  positions within 3 lists to insert @req to
> + * Return Value:
> + *      filled @prev
> + * NOTE: called by
> + *  - ldlm_grant_lock_with_skiplist
> + */
> +static void search_granted_lock(struct list_head *queue,
> +				struct ldlm_lock *req,
> +				struct sl_insert_point *prev)
> +{
> +	struct ldlm_lock *lock, *mode_end, *policy_end;
> +
> +	list_for_each_entry(lock, queue, l_res_link) {
> +
> +		mode_end = list_prev_entry(lock, l_sl_mode);
> +
> +		if (lock->l_req_mode != req->l_req_mode) {
> +			/* jump to last lock of mode group */
> +			lock = mode_end;
> +			continue;
> +		}
> +
> +		/* suitable mode group is found */
> +		if (lock->l_resource->lr_type == LDLM_PLAIN) {
> +			/* insert point is last lock of the mode group */
> +			prev->res_link = &mode_end->l_res_link;
> +			prev->mode_link = &mode_end->l_sl_mode;
> +			prev->policy_link = &req->l_sl_policy;
> +			return;
> +		}
> +
> +		if (lock->l_resource->lr_type == LDLM_IBITS) {
> +			for (;;) {
> +				policy_end =
> +					list_prev_entry(lock, l_sl_policy);
> +
> +				if (lock->l_policy_data.l_inodebits.bits ==
> +				    req->l_policy_data.l_inodebits.bits) {
> +					/* insert point is last lock of
> +					 * the policy group
> +					 */
> +					prev->res_link =
> +						&policy_end->l_res_link;
> +					prev->mode_link =
> +						&policy_end->l_sl_mode;
> +					prev->policy_link =
> +						&policy_end->l_sl_policy;
> +					return;
> +				}
> +
> +				if (policy_end == mode_end)
> +					/* done with mode group */
> +					break;
> +
> +				/* go to next policy group within mode group */
> +				lock = list_next_entry(policy_end, l_res_link);
> +			}  /* loop over policy groups within the mode group */
> +
> +			/* insert point is last lock of the mode group,
> +			 * new policy group is started
> +			 */
> +			prev->res_link = &mode_end->l_res_link;
> +			prev->mode_link = &mode_end->l_sl_mode;
> +			prev->policy_link = &req->l_sl_policy;
> +			return;
> +		}
> +
> +		LDLM_ERROR(lock, "is not LDLM_PLAIN or LDLM_IBITS lock");
> +		LBUG();
> +	}
> +
> +	/* insert point is last lock on the queue,
> +	 * new mode group and new policy group are started
> +	 */
> +	prev->res_link = queue->prev;
> +	prev->mode_link = &req->l_sl_mode;
> +	prev->policy_link = &req->l_sl_policy;
> +}
> +
> +/**
> + * Add a lock into resource granted list after a position described by
> + * \a prev.
> + */
> +static void ldlm_granted_list_add_lock(struct ldlm_lock *lock,
> +				       struct sl_insert_point *prev)
> +{
> +	struct ldlm_resource *res = lock->l_resource;
> +
> +	check_res_locked(res);
> +
> +	ldlm_resource_dump(D_INFO, res);
> +	LDLM_DEBUG(lock, "About to add lock:");
> +
> +	if (ldlm_is_destroyed(lock)) {
> +		CDEBUG(D_OTHER, "Lock destroyed, not adding to resource\n");
> +		return;
> +	}
> +
> +	LASSERT(list_empty(&lock->l_res_link));
> +	LASSERT(list_empty(&lock->l_sl_mode));
> +	LASSERT(list_empty(&lock->l_sl_policy));
> +
> +	/*
> +	 * lock->link == prev->link means lock is first starting the group.
> +	 * Don't re-add to itself to suppress kernel warnings.
> +	 */
> +	if (&lock->l_res_link != prev->res_link)
> +		list_add(&lock->l_res_link, prev->res_link);
> +	if (&lock->l_sl_mode != prev->mode_link)
> +		list_add(&lock->l_sl_mode, prev->mode_link);
> +	if (&lock->l_sl_policy != prev->policy_link)
> +		list_add(&lock->l_sl_policy, prev->policy_link);
> +}
> +
> +/**
> + * Add a lock to granted list on a resource maintaining skiplist
> + * correctness.
> + */
> +static void ldlm_grant_lock_with_skiplist(struct ldlm_lock *lock)
> +{
> +	struct sl_insert_point prev;
> +
> +	LASSERT(lock->l_req_mode == lock->l_granted_mode);
> +
> +	search_granted_lock(&lock->l_resource->lr_granted, lock, &prev);
> +	ldlm_granted_list_add_lock(lock, &prev);
> +}
> +
> +/**
> + * Perform lock granting bookkeeping.
> + *
> + * Includes putting the lock into granted list and updating lock mode.
> + * NOTE: called by
> + *  - ldlm_lock_enqueue
> + *  - ldlm_reprocess_queue
> + *  - ldlm_lock_convert
> + *
> + * must be called with lr_lock held
> + */
> +void ldlm_grant_lock(struct ldlm_lock *lock, struct list_head *work_list)
> +{
> +	struct ldlm_resource *res = lock->l_resource;
> +
> +	check_res_locked(res);
> +
> +	lock->l_granted_mode = lock->l_req_mode;
> +
> +	if (work_list && lock->l_completion_ast)
> +		ldlm_add_ast_work_item(lock, NULL, work_list);
> +
> +	if (res->lr_type == LDLM_PLAIN || res->lr_type == LDLM_IBITS) {
> +		ldlm_grant_lock_with_skiplist(lock);
> +	} else if (res->lr_type == LDLM_EXTENT) {
> +		ldlm_extent_add_lock(res, lock);
> +	} else if (res->lr_type == LDLM_FLOCK) {
> +		/*
> +		 * We should not add locks to granted list in
> +		 * the following cases:
> +		 * - this is an UNLOCK but not a real lock;
> +		 * - this is a TEST lock;
> +		 * - this is a F_CANCELLK lock (async flock has req_mode == 0)
> +		 * - this is a deadlock (flock cannot be granted)
> +		 */
> +		if (!lock->l_req_mode || lock->l_req_mode == LCK_NL ||
> +		    ldlm_is_test_lock(lock) || ldlm_is_flock_deadlock(lock))
> +			return;
> +		ldlm_resource_add_lock(res, &res->lr_granted, lock);
> +	} else {
> +		LBUG();
> +	}
> +
> +	ldlm_pool_add(&ldlm_res_to_ns(res)->ns_pool, lock);
> +}
> +
> +/**
> + * Describe the overlap between two locks.  itree_overlap_cb data.
> + */
> +struct lock_match_data {
> +	struct ldlm_lock	*lmd_old;
> +	struct ldlm_lock	*lmd_lock;
> +	enum ldlm_mode		*lmd_mode;
> +	union ldlm_policy_data	*lmd_policy;
> +	__u64			 lmd_flags;
> +	int			 lmd_unref;
> +};
> +
> +/**
> + * Check if the given @lock meets the criteria for a match.
> + * A reference on the lock is taken if matched.
> + *
> + * \param lock	test-against this lock
> + * \param data	parameters
> + */
> +static bool lock_matches(struct ldlm_lock *lock, void *vdata)
> +{
> +	struct lock_match_data *data = vdata;
> +	union ldlm_policy_data *lpol = &lock->l_policy_data;
> +	enum ldlm_mode match;
> +
> +	if (lock == data->lmd_old)
> +		return true;
> +
> +	/*
> +	 * Check if this lock can be matched.
> +	 * Used by LU-2919(exclusive open) for open lease lock
> +	 */
> +	if (ldlm_is_excl(lock))
> +		return false;
> +
> +	/*
> +	 * llite sometimes wants to match locks that will be
> +	 * canceled when their users drop, but we allow it to match
> +	 * if it passes in CBPENDING and the lock still has users.
> +	 * this is generally only going to be used by children
> +	 * whose parents already hold a lock so forward progress
> +	 * can still happen.
> +	 */
> +	if (ldlm_is_cbpending(lock) &&
> +	    !(data->lmd_flags & LDLM_FL_CBPENDING))
> +		return false;
> +
> +	if (!data->lmd_unref && ldlm_is_cbpending(lock) &&
> +	    !lock->l_readers && !lock->l_writers)
> +		return false;
> +
> +	if (!(lock->l_req_mode & *data->lmd_mode))
> +		return false;
> +	match = lock->l_req_mode;
> +
> +	switch (lock->l_resource->lr_type) {
> +	case LDLM_EXTENT:
> +		if (lpol->l_extent.start > data->lmd_policy->l_extent.start ||
> +		    lpol->l_extent.end < data->lmd_policy->l_extent.end)
> +			return false;
> +
> +		if (unlikely(match == LCK_GROUP) &&
> +		    data->lmd_policy->l_extent.gid != LDLM_GID_ANY &&
> +		    lpol->l_extent.gid != data->lmd_policy->l_extent.gid)
> +			return false;
> +		break;
> +	case LDLM_IBITS:
> +		/*
> +		 * We match if we have existing lock with same or wider set
> +		 * of bits.
> +		 */
> +		if ((lpol->l_inodebits.bits &
> +		     data->lmd_policy->l_inodebits.bits) !=
> +		    data->lmd_policy->l_inodebits.bits)
> +			return false;
> +		break;
> +	default:
> +		break;
> +	}
> +	/*
> +	 * We match if we have existing lock with same or wider set
> +	 * of bits.
> +	 */
> +	if (!data->lmd_unref && LDLM_HAVE_MASK(lock, GONE))
> +		return false;
> +
> +	if (!equi(data->lmd_flags & LDLM_FL_LOCAL_ONLY, ldlm_is_local(lock)))
> +		return false;
> +
> +	if (data->lmd_flags & LDLM_FL_TEST_LOCK) {
> +		LDLM_LOCK_GET(lock);
> +		ldlm_lock_touch_in_lru(lock);
> +	} else {
> +		ldlm_lock_addref_internal_nolock(lock, match);
> +	}
> +
> +	*data->lmd_mode = match;
> +	data->lmd_lock = lock;
> +
> +	return true;
> +}
> +
> +/**
> + * Search for a lock with given parameters in interval trees.
> + *
> + * \param res	search for a lock in this resource
> + * \param data	parameters
> + *
> + * \retval	a referenced lock or NULL.
> + */
> +static struct ldlm_lock *search_itree(struct ldlm_resource *res,
> +				      struct lock_match_data *data)
> +{
> +	int idx;
> +
> +	for (idx = 0; idx < LCK_MODE_NUM; idx++) {
> +		struct ldlm_interval_tree *tree = &res->lr_itree[idx];
> +
> +		if (RB_EMPTY_ROOT(&tree->lit_root.rb_root))
> +			continue;
> +
> +		if (!(tree->lit_mode & *data->lmd_mode))
> +			continue;
> +
> +		ldlm_extent_search(&tree->lit_root,
> +				   data->lmd_policy->l_extent.start,
> +				   data->lmd_policy->l_extent.end,
> +				   lock_matches, data);
> +	}
> +	return data->lmd_lock;
> +}
> +
> +/**
> + * Search for a lock with given properties in a queue.
> + *
> + * \param queue	search for a lock in this queue
> + * \param data	parameters
> + *
> + * \retval	a referenced lock or NULL.
> + */
> +static struct ldlm_lock *search_queue(struct list_head *queue,
> +				      struct lock_match_data *data)
> +{
> +	struct ldlm_lock *lock;
> +
> +	list_for_each_entry(lock, queue, l_res_link)
> +		if (lock_matches(lock, data))
> +			return data->lmd_lock;
> +	return NULL;
> +}
> +
> +void ldlm_lock_fail_match_locked(struct ldlm_lock *lock)
> +{
> +	if ((lock->l_flags & LDLM_FL_FAIL_NOTIFIED) == 0) {
> +		lock->l_flags |= LDLM_FL_FAIL_NOTIFIED;
> +		wake_up_all(&lock->l_waitq);
> +	}
> +}
> +
> +/**
> + * Mark lock as "matchable" by OST.
> + *
> + * Used to prevent certain races in LOV/OSC where the lock is granted, but LVB
> + * is not yet valid.
> + * Assumes LDLM lock is already locked.
> + */
> +void ldlm_lock_allow_match_locked(struct ldlm_lock *lock)
> +{
> +	ldlm_set_lvb_ready(lock);
> +	wake_up_all(&lock->l_waitq);
> +}
> +EXPORT_SYMBOL(ldlm_lock_allow_match_locked);
> +
> +/**
> + * Mark lock as "matchable" by OST.
> + * Locks the lock and then \see ldlm_lock_allow_match_locked
> + */
> +void ldlm_lock_allow_match(struct ldlm_lock *lock)
> +{
> +	lock_res_and_lock(lock);
> +	ldlm_lock_allow_match_locked(lock);
> +	unlock_res_and_lock(lock);
> +}
> +EXPORT_SYMBOL(ldlm_lock_allow_match);
> +
> +/**
> + * Attempt to find a lock with specified properties.
> + *
> + * Typically returns a reference to matched lock unless LDLM_FL_TEST_LOCK is
> + * set in \a flags
> + *
> + * Can be called in two ways:
> + *
> + * If 'ns' is NULL, then lockh describes an existing lock that we want to look
> + * for a duplicate of.
> + *
> + * Otherwise, all of the fields must be filled in, to match against.
> + *
> + * If 'flags' contains LDLM_FL_LOCAL_ONLY, then only match local locks on the
> + *     server (ie, connh is NULL)
> + * If 'flags' contains LDLM_FL_BLOCK_GRANTED, then only locks on the granted
> + *     list will be considered
> + * If 'flags' contains LDLM_FL_CBPENDING, then locks that have been marked
> + *     to be canceled can still be matched as long as they still have reader
> + *     or writer referneces
> + * If 'flags' contains LDLM_FL_TEST_LOCK, then don't actually reference a lock,
> + *     just tell us if we would have matched.
> + *
> + * \retval 1 if it finds an already-existing lock that is compatible; in this
> + * case, lockh is filled in with a addref()ed lock
> + *
> + * We also check security context, and if that fails we simply return 0 (to
> + * keep caller code unchanged), the context failure will be discovered by
> + * caller sometime later.
> + */
> +enum ldlm_mode ldlm_lock_match(struct ldlm_namespace *ns, __u64 flags,
> +			       const struct ldlm_res_id *res_id,
> +			       enum ldlm_type type,
> +			       union ldlm_policy_data *policy,
> +			       enum ldlm_mode mode,
> +			       struct lustre_handle *lockh, int unref)
> +{
> +	struct lock_match_data data = {
> +		.lmd_old	= NULL,
> +		.lmd_lock	= NULL,
> +		.lmd_mode	= &mode,
> +		.lmd_policy	= policy,
> +		.lmd_flags	= flags,
> +		.lmd_unref	= unref,
> +	};
> +	struct ldlm_resource *res;
> +	struct ldlm_lock *lock;
> +	int rc = 0;
> +
> +	if (!ns) {
> +		data.lmd_old = ldlm_handle2lock(lockh);
> +		LASSERT(data.lmd_old);
> +
> +		ns = ldlm_lock_to_ns(data.lmd_old);
> +		res_id = &data.lmd_old->l_resource->lr_name;
> +		type = data.lmd_old->l_resource->lr_type;
> +		*data.lmd_mode = data.lmd_old->l_req_mode;
> +	}
> +
> +	res = ldlm_resource_get(ns, NULL, res_id, type, 0);
> +	if (IS_ERR(res)) {
> +		LASSERT(!data.lmd_old);
> +		return 0;
> +	}
> +
> +	LDLM_RESOURCE_ADDREF(res);
> +	lock_res(res);
> +
> +	if (res->lr_type == LDLM_EXTENT)
> +		lock = search_itree(res, &data);
> +	else
> +		lock = search_queue(&res->lr_granted, &data);
> +	if (lock) {
> +		rc = 1;
> +		goto out;
> +	}
> +	if (flags & LDLM_FL_BLOCK_GRANTED) {
> +		rc = 0;
> +		goto out;
> +	}
> +	lock = search_queue(&res->lr_waiting, &data);
> +	if (lock) {
> +		rc = 1;
> +		goto out;
> +	}
> +out:
> +	unlock_res(res);
> +	LDLM_RESOURCE_DELREF(res);
> +	ldlm_resource_putref(res);
> +
> +	if (lock) {
> +		ldlm_lock2handle(lock, lockh);
> +		if ((flags & LDLM_FL_LVB_READY) && !ldlm_is_lvb_ready(lock)) {
> +			__u64 wait_flags = LDLM_FL_LVB_READY |
> +				LDLM_FL_DESTROYED | LDLM_FL_FAIL_NOTIFIED;
> +
> +			if (lock->l_completion_ast) {
> +				int err = lock->l_completion_ast(lock,
> +							  LDLM_FL_WAIT_NOREPROC,
> +								 NULL);
> +				if (err) {
> +					if (flags & LDLM_FL_TEST_LOCK)
> +						LDLM_LOCK_RELEASE(lock);
> +					else
> +						ldlm_lock_decref_internal(lock,
> +									  mode);
> +					rc = 0;
> +					goto out2;
> +				}
> +			}
> +
> +			/* XXX FIXME see comment on CAN_MATCH in lustre_dlm.h */
> +			wait_event_idle_timeout(lock->l_waitq,
> +						lock->l_flags & wait_flags,
> +						obd_timeout * HZ);
> +			if (!ldlm_is_lvb_ready(lock)) {
> +				if (flags & LDLM_FL_TEST_LOCK)
> +					LDLM_LOCK_RELEASE(lock);
> +				else
> +					ldlm_lock_decref_internal(lock, mode);
> +				rc = 0;
> +			}
> +		}
> +	}
> + out2:
> +	if (rc) {
> +		LDLM_DEBUG(lock, "matched (%llu %llu)",
> +			   (type == LDLM_PLAIN || type == LDLM_IBITS) ?
> +				res_id->name[2] : policy->l_extent.start,
> +			   (type == LDLM_PLAIN || type == LDLM_IBITS) ?
> +				res_id->name[3] : policy->l_extent.end);
> +
> +		/* check user's security context */
> +		if (lock->l_conn_export &&
> +		    sptlrpc_import_check_ctx(
> +				class_exp2cliimp(lock->l_conn_export))) {
> +			if (!(flags & LDLM_FL_TEST_LOCK))
> +				ldlm_lock_decref_internal(lock, mode);
> +			rc = 0;
> +		}
> +
> +		if (flags & LDLM_FL_TEST_LOCK)
> +			LDLM_LOCK_RELEASE(lock);
> +
> +	} else if (!(flags & LDLM_FL_TEST_LOCK)) {/*less verbose for test-only*/
> +		LDLM_DEBUG_NOLOCK("not matched ns %p type %u mode %u res %llu/%llu (%llu %llu)",
> +				  ns, type, mode, res_id->name[0],
> +				  res_id->name[1],
> +				  (type == LDLM_PLAIN || type == LDLM_IBITS) ?
> +					res_id->name[2] : policy->l_extent.start,
> +				  (type == LDLM_PLAIN || type == LDLM_IBITS) ?
> +					res_id->name[3] : policy->l_extent.end);
> +	}
> +	if (data.lmd_old)
> +		LDLM_LOCK_PUT(data.lmd_old);
> +
> +	return rc ? mode : 0;
> +}
> +EXPORT_SYMBOL(ldlm_lock_match);
> +
> +enum ldlm_mode ldlm_revalidate_lock_handle(const struct lustre_handle *lockh,
> +					   __u64 *bits)
> +{
> +	struct ldlm_lock *lock;
> +	enum ldlm_mode mode = 0;
> +
> +	lock = ldlm_handle2lock(lockh);
> +	if (lock) {
> +		lock_res_and_lock(lock);
> +		if (LDLM_HAVE_MASK(lock, GONE))
> +			goto out;
> +
> +		if (ldlm_is_cbpending(lock) &&
> +		    lock->l_readers == 0 && lock->l_writers == 0)
> +			goto out;
> +
> +		if (bits)
> +			*bits = lock->l_policy_data.l_inodebits.bits;
> +		mode = lock->l_granted_mode;
> +		ldlm_lock_addref_internal_nolock(lock, mode);
> +	}
> +
> +out:
> +	if (lock) {
> +		unlock_res_and_lock(lock);
> +		LDLM_LOCK_PUT(lock);
> +	}
> +	return mode;
> +}
> +EXPORT_SYMBOL(ldlm_revalidate_lock_handle);
> +
> +/** The caller must guarantee that the buffer is large enough. */
> +int ldlm_fill_lvb(struct ldlm_lock *lock, struct req_capsule *pill,
> +		  enum req_location loc, void *data, int size)
> +{
> +	void *lvb;
> +
> +	LASSERT(data);
> +	LASSERT(size >= 0);
> +
> +	switch (lock->l_lvb_type) {
> +	case LVB_T_OST:
> +		if (size == sizeof(struct ost_lvb)) {
> +			if (loc == RCL_CLIENT)
> +				lvb = req_capsule_client_swab_get(pill,
> +								  &RMF_DLM_LVB,
> +							lustre_swab_ost_lvb);
> +			else
> +				lvb = req_capsule_server_swab_get(pill,
> +								  &RMF_DLM_LVB,
> +							lustre_swab_ost_lvb);
> +			if (unlikely(!lvb)) {
> +				LDLM_ERROR(lock, "no LVB");
> +				return -EPROTO;
> +			}
> +
> +			memcpy(data, lvb, size);
> +		} else if (size == sizeof(struct ost_lvb_v1)) {
> +			struct ost_lvb *olvb = data;
> +
> +			if (loc == RCL_CLIENT)
> +				lvb = req_capsule_client_swab_get(pill,
> +								  &RMF_DLM_LVB,
> +							lustre_swab_ost_lvb_v1);
> +			else
> +				lvb = req_capsule_server_sized_swab_get(pill,
> +						&RMF_DLM_LVB, size,
> +						lustre_swab_ost_lvb_v1);
> +			if (unlikely(!lvb)) {
> +				LDLM_ERROR(lock, "no LVB");
> +				return -EPROTO;
> +			}
> +
> +			memcpy(data, lvb, size);
> +			olvb->lvb_mtime_ns = 0;
> +			olvb->lvb_atime_ns = 0;
> +			olvb->lvb_ctime_ns = 0;
> +		} else {
> +			LDLM_ERROR(lock, "Replied unexpected ost LVB size %d",
> +				   size);
> +			return -EINVAL;
> +		}
> +		break;
> +	case LVB_T_LQUOTA:
> +		if (size == sizeof(struct lquota_lvb)) {
> +			if (loc == RCL_CLIENT)
> +				lvb = req_capsule_client_swab_get(pill,
> +								  &RMF_DLM_LVB,
> +							lustre_swab_lquota_lvb);
> +			else
> +				lvb = req_capsule_server_swab_get(pill,
> +								  &RMF_DLM_LVB,
> +							lustre_swab_lquota_lvb);
> +			if (unlikely(!lvb)) {
> +				LDLM_ERROR(lock, "no LVB");
> +				return -EPROTO;
> +			}
> +
> +			memcpy(data, lvb, size);
> +		} else {
> +			LDLM_ERROR(lock,
> +				   "Replied unexpected lquota LVB size %d",
> +				   size);
> +			return -EINVAL;
> +		}
> +		break;
> +	case LVB_T_LAYOUT:
> +		if (size == 0)
> +			break;
> +
> +		if (loc == RCL_CLIENT)
> +			lvb = req_capsule_client_get(pill, &RMF_DLM_LVB);
> +		else
> +			lvb = req_capsule_server_get(pill, &RMF_DLM_LVB);
> +		if (unlikely(!lvb)) {
> +			LDLM_ERROR(lock, "no LVB");
> +			return -EPROTO;
> +		}
> +
> +		memcpy(data, lvb, size);
> +		break;
> +	default:
> +		LDLM_ERROR(lock, "Unknown LVB type: %d", lock->l_lvb_type);
> +		dump_stack();
> +		return -EINVAL;
> +	}
> +
> +	return 0;
> +}
> +
> +/**
> + * Create and fill in new LDLM lock with specified properties.
> + * Returns a referenced lock
> + */
> +struct ldlm_lock *ldlm_lock_create(struct ldlm_namespace *ns,
> +				   const struct ldlm_res_id *res_id,
> +				   enum ldlm_type type,
> +				   enum ldlm_mode mode,
> +				   const struct ldlm_callback_suite *cbs,
> +				   void *data, __u32 lvb_len,
> +				   enum lvb_type lvb_type)
> +{
> +	struct ldlm_lock *lock;
> +	struct ldlm_resource *res;
> +	int rc;
> +
> +	res = ldlm_resource_get(ns, NULL, res_id, type, 1);
> +	if (IS_ERR(res))
> +		return ERR_CAST(res);
> +
> +	lock = ldlm_lock_new(res);
> +	if (!lock) {
> +		ldlm_resource_putref(res);
> +		return ERR_PTR(-ENOMEM);
> +	}
> +
> +	lock->l_req_mode = mode;
> +	lock->l_ast_data = data;
> +	lock->l_pid = current->pid;
> +	if (cbs) {
> +		lock->l_blocking_ast = cbs->lcs_blocking;
> +		lock->l_completion_ast = cbs->lcs_completion;
> +		lock->l_glimpse_ast = cbs->lcs_glimpse;
> +	}
> +
> +	if (lvb_len) {
> +		lock->l_lvb_len = lvb_len;
> +		lock->l_lvb_data = kzalloc(lvb_len, GFP_NOFS);
> +		if (!lock->l_lvb_data) {
> +			rc = -ENOMEM;
> +			goto out;
> +		}
> +	}
> +
> +	lock->l_lvb_type = lvb_type;
> +	if (OBD_FAIL_CHECK(OBD_FAIL_LDLM_NEW_LOCK)) {
> +		rc = -ENOENT;
> +		goto out;
> +	}
> +
> +	return lock;
> +
> +out:
> +	ldlm_lock_destroy(lock);
> +	LDLM_LOCK_RELEASE(lock);
> +	return ERR_PTR(rc);
> +}
> +
> +
> +
> +/**
> + * Enqueue (request) a lock.
> + * On the client this is called from ldlm_cli_enqueue_fini
> + * after we already got an initial reply from the server with some status.
> + *
> + * Does not block. As a result of enqueue the lock would be put
> + * into granted or waiting list.
> + */
> +enum ldlm_error ldlm_lock_enqueue(struct ldlm_namespace *ns,
> +				  struct ldlm_lock **lockp,
> +				  void *cookie, __u64 *flags)
> +{
> +	struct ldlm_lock *lock = *lockp;
> +	struct ldlm_resource *res = lock->l_resource;
> +
> +	lock_res_and_lock(lock);
> +	if (lock->l_req_mode == lock->l_granted_mode) {
> +		/* The server returned a blocked lock, but it was granted
> +		 * before we got a chance to actually enqueue it.  We don't
> +		 * need to do anything else.
> +		 */
> +		*flags &= ~LDLM_FL_BLOCKED_MASK;
> +		goto out;
> +	}
> +
> +	ldlm_resource_unlink_lock(lock);
> +
> +	/* Some flags from the enqueue want to make it into the AST, via the
> +	 * lock's l_flags.
> +	 */
> +	if (*flags & LDLM_FL_AST_DISCARD_DATA)
> +		ldlm_set_ast_discard_data(lock);
> +	if (*flags & LDLM_FL_TEST_LOCK)
> +		ldlm_set_test_lock(lock);
> +
> +	/*
> +	 * This distinction between local lock trees is very important; a client
> +	 * namespace only has information about locks taken by that client, and
> +	 * thus doesn't have enough information to decide for itself if it can
> +	 * be granted (below).  In this case, we do exactly what the server
> +	 * tells us to do, as dictated by the 'flags'.
> +	 */
> +	if (*flags & (LDLM_FL_BLOCK_WAIT | LDLM_FL_BLOCK_GRANTED))
> +		ldlm_resource_add_lock(res, &res->lr_waiting, lock);
> +	else
> +		ldlm_grant_lock(lock, NULL);
> +
> +out:
> +	unlock_res_and_lock(lock);
> +	return ELDLM_OK;
> +}
> +
> +/**
> + * Process a call to blocking AST callback for a lock in ast_work list
> + */
> +static int
> +ldlm_work_bl_ast_lock(struct ptlrpc_request_set *rqset, void *opaq)
> +{
> +	struct ldlm_cb_set_arg *arg = opaq;
> +	struct ldlm_lock_desc   d;
> +	int		     rc;
> +	struct ldlm_lock       *lock;
> +
> +	if (list_empty(arg->list))
> +		return -ENOENT;
> +
> +	lock = list_first_entry(arg->list, struct ldlm_lock, l_bl_ast);
> +
> +	/* nobody should touch l_bl_ast */
> +	lock_res_and_lock(lock);
> +	list_del_init(&lock->l_bl_ast);
> +
> +	LASSERT(ldlm_is_ast_sent(lock));
> +	LASSERT(lock->l_bl_ast_run == 0);
> +	LASSERT(lock->l_blocking_lock);
> +	lock->l_bl_ast_run++;
> +	unlock_res_and_lock(lock);
> +
> +	ldlm_lock2desc(lock->l_blocking_lock, &d);
> +
> +	rc = lock->l_blocking_ast(lock, &d, (void *)arg, LDLM_CB_BLOCKING);
> +	LDLM_LOCK_RELEASE(lock->l_blocking_lock);
> +	lock->l_blocking_lock = NULL;
> +	LDLM_LOCK_RELEASE(lock);
> +
> +	return rc;
> +}
> +
> +/**
> + * Process a call to completion AST callback for a lock in ast_work list
> + */
> +static int
> +ldlm_work_cp_ast_lock(struct ptlrpc_request_set *rqset, void *opaq)
> +{
> +	struct ldlm_cb_set_arg  *arg = opaq;
> +	int		      rc = 0;
> +	struct ldlm_lock	*lock;
> +	ldlm_completion_callback completion_callback;
> +
> +	if (list_empty(arg->list))
> +		return -ENOENT;
> +
> +	lock = list_first_entry(arg->list, struct ldlm_lock, l_cp_ast);
> +
> +	/* It's possible to receive a completion AST before we've set
> +	 * the l_completion_ast pointer: either because the AST arrived
> +	 * before the reply, or simply because there's a small race
> +	 * window between receiving the reply and finishing the local
> +	 * enqueue. (bug 842)
> +	 *
> +	 * This can't happen with the blocking_ast, however, because we
> +	 * will never call the local blocking_ast until we drop our
> +	 * reader/writer reference, which we won't do until we get the
> +	 * reply and finish enqueueing.
> +	 */
> +
> +	/* nobody should touch l_cp_ast */
> +	lock_res_and_lock(lock);
> +	list_del_init(&lock->l_cp_ast);
> +	LASSERT(ldlm_is_cp_reqd(lock));
> +	/* save l_completion_ast since it can be changed by
> +	 * mds_intent_policy(), see bug 14225
> +	 */
> +	completion_callback = lock->l_completion_ast;
> +	ldlm_clear_cp_reqd(lock);
> +	unlock_res_and_lock(lock);
> +
> +	if (completion_callback)
> +		rc = completion_callback(lock, 0, (void *)arg);
> +	LDLM_LOCK_RELEASE(lock);
> +
> +	return rc;
> +}
> +
> +/**
> + * Process a call to revocation AST callback for a lock in ast_work list
> + */
> +static int
> +ldlm_work_revoke_ast_lock(struct ptlrpc_request_set *rqset, void *opaq)
> +{
> +	struct ldlm_cb_set_arg *arg = opaq;
> +	struct ldlm_lock_desc   desc;
> +	int		     rc;
> +	struct ldlm_lock       *lock;
> +
> +	if (list_empty(arg->list))
> +		return -ENOENT;
> +
> +	lock = list_first_entry(arg->list, struct ldlm_lock, l_rk_ast);
> +	list_del_init(&lock->l_rk_ast);
> +
> +	/* the desc just pretend to exclusive */
> +	ldlm_lock2desc(lock, &desc);
> +	desc.l_req_mode = LCK_EX;
> +	desc.l_granted_mode = 0;
> +
> +	rc = lock->l_blocking_ast(lock, &desc, (void *)arg, LDLM_CB_BLOCKING);
> +	LDLM_LOCK_RELEASE(lock);
> +
> +	return rc;
> +}
> +
> +/**
> + * Process a call to glimpse AST callback for a lock in ast_work list
> + */
> +static int ldlm_work_gl_ast_lock(struct ptlrpc_request_set *rqset, void *opaq)
> +{
> +	struct ldlm_cb_set_arg		*arg = opaq;
> +	struct ldlm_glimpse_work	*gl_work;
> +	struct ldlm_lock		*lock;
> +	int				 rc = 0;
> +
> +	if (list_empty(arg->list))
> +		return -ENOENT;
> +
> +	gl_work = list_first_entry(arg->list, struct ldlm_glimpse_work,
> +			     gl_list);
> +	list_del_init(&gl_work->gl_list);
> +
> +	lock = gl_work->gl_lock;
> +
> +	/* transfer the glimpse descriptor to ldlm_cb_set_arg */
> +	arg->gl_desc = gl_work->gl_desc;
> +
> +	/* invoke the actual glimpse callback */
> +	if (lock->l_glimpse_ast(lock, (void *)arg) == 0)
> +		rc = 1;
> +
> +	LDLM_LOCK_RELEASE(lock);
> +
> +	if ((gl_work->gl_flags & LDLM_GL_WORK_NOFREE) == 0)
> +		kfree(gl_work);
> +
> +	return rc;
> +}
> +
> +/**
> + * Process list of locks in need of ASTs being sent.
> + *
> + * Used on server to send multiple ASTs together instead of sending one by
> + * one.
> + */
> +int ldlm_run_ast_work(struct ldlm_namespace *ns, struct list_head *rpc_list,
> +		      enum ldlm_desc_ast_t ast_type)
> +{
> +	struct ldlm_cb_set_arg *arg;
> +	set_producer_func       work_ast_lock;
> +	int		     rc;
> +
> +	if (list_empty(rpc_list))
> +		return 0;
> +
> +	arg = kzalloc(sizeof(*arg), GFP_NOFS);
> +	if (!arg)
> +		return -ENOMEM;
> +
> +	atomic_set(&arg->restart, 0);
> +	arg->list = rpc_list;
> +
> +	switch (ast_type) {
> +	case LDLM_WORK_BL_AST:
> +		arg->type = LDLM_BL_CALLBACK;
> +		work_ast_lock = ldlm_work_bl_ast_lock;
> +		break;
> +	case LDLM_WORK_CP_AST:
> +		arg->type = LDLM_CP_CALLBACK;
> +		work_ast_lock = ldlm_work_cp_ast_lock;
> +		break;
> +	case LDLM_WORK_REVOKE_AST:
> +		arg->type = LDLM_BL_CALLBACK;
> +		work_ast_lock = ldlm_work_revoke_ast_lock;
> +		break;
> +	case LDLM_WORK_GL_AST:
> +		arg->type = LDLM_GL_CALLBACK;
> +		work_ast_lock = ldlm_work_gl_ast_lock;
> +		break;
> +	default:
> +		LBUG();
> +	}
> +
> +	/* We create a ptlrpc request set with flow control extension.
> +	 * This request set will use the work_ast_lock function to produce new
> +	 * requests and will send a new request each time one completes in order
> +	 * to keep the number of requests in flight to ns_max_parallel_ast
> +	 */
> +	arg->set = ptlrpc_prep_fcset(ns->ns_max_parallel_ast ? : UINT_MAX,
> +				     work_ast_lock, arg);
> +	if (!arg->set) {
> +		rc = -ENOMEM;
> +		goto out;
> +	}
> +
> +	ptlrpc_set_wait(arg->set);
> +	ptlrpc_set_destroy(arg->set);
> +
> +	rc = atomic_read(&arg->restart) ? -ERESTART : 0;
> +	goto out;
> +out:
> +	kfree(arg);
> +	return rc;
> +}
> +
> +static bool is_bl_done(struct ldlm_lock *lock)
> +{
> +	bool bl_done = true;
> +
> +	if (!ldlm_is_bl_done(lock)) {
> +		lock_res_and_lock(lock);
> +		bl_done = ldlm_is_bl_done(lock);
> +		unlock_res_and_lock(lock);
> +	}
> +
> +	return bl_done;
> +}
> +
> +/**
> + * Helper function to call blocking AST for LDLM lock \a lock in a
> + * "cancelling" mode.
> + */
> +void ldlm_cancel_callback(struct ldlm_lock *lock)
> +{
> +	check_res_locked(lock->l_resource);
> +	if (!ldlm_is_cancel(lock)) {
> +		ldlm_set_cancel(lock);
> +		if (lock->l_blocking_ast) {
> +			unlock_res_and_lock(lock);
> +			lock->l_blocking_ast(lock, NULL, lock->l_ast_data,
> +					     LDLM_CB_CANCELING);
> +			lock_res_and_lock(lock);
> +		} else {
> +			LDLM_DEBUG(lock, "no blocking ast");
> +		}
> +		/* only canceller can set bl_done bit */
> +		ldlm_set_bl_done(lock);
> +		wake_up_all(&lock->l_waitq);
> +	} else if (!ldlm_is_bl_done(lock)) {
> +		/*
> +		 * The lock is guaranteed to have been canceled once
> +		 * returning from this function.
> +		 */
> +		unlock_res_and_lock(lock);
> +		wait_event_idle(lock->l_waitq, is_bl_done(lock));
> +		lock_res_and_lock(lock);
> +	}
> +}
> +
> +/**
> + * Remove skiplist-enabled LDLM lock \a req from granted list
> + */
> +void ldlm_unlink_lock_skiplist(struct ldlm_lock *req)
> +{
> +	if (req->l_resource->lr_type != LDLM_PLAIN &&
> +	    req->l_resource->lr_type != LDLM_IBITS)
> +		return;
> +
> +	list_del_init(&req->l_sl_policy);
> +	list_del_init(&req->l_sl_mode);
> +}
> +
> +/**
> + * Attempts to cancel LDLM lock \a lock that has no reader/writer references.
> + */
> +void ldlm_lock_cancel(struct ldlm_lock *lock)
> +{
> +	struct ldlm_resource *res;
> +	struct ldlm_namespace *ns;
> +
> +	lock_res_and_lock(lock);
> +
> +	res = lock->l_resource;
> +	ns  = ldlm_res_to_ns(res);
> +
> +	/* Please do not, no matter how tempting, remove this LBUG without
> +	 * talking to me first. -phik
> +	 */
> +	if (lock->l_readers || lock->l_writers) {
> +		LDLM_ERROR(lock, "lock still has references");
> +		LBUG();
> +	}
> +
> +	/* Releases cancel callback. */
> +	ldlm_cancel_callback(lock);
> +
> +	ldlm_resource_unlink_lock(lock);
> +	ldlm_lock_destroy_nolock(lock);
> +
> +	if (lock->l_granted_mode == lock->l_req_mode)
> +		ldlm_pool_del(&ns->ns_pool, lock);
> +
> +	/* Make sure we will not be called again for same lock what is possible
> +	 * if not to zero out lock->l_granted_mode
> +	 */
> +	lock->l_granted_mode = LCK_MINMODE;
> +	unlock_res_and_lock(lock);
> +}
> +EXPORT_SYMBOL(ldlm_lock_cancel);
> +
> +/**
> + * Set opaque data into the lock that only makes sense to upper layer.
> + */
> +int ldlm_lock_set_data(const struct lustre_handle *lockh, void *data)
> +{
> +	struct ldlm_lock *lock = ldlm_handle2lock(lockh);
> +	int rc = -EINVAL;
> +
> +	if (lock) {
> +		if (!lock->l_ast_data)
> +			lock->l_ast_data = data;
> +		if (lock->l_ast_data == data)
> +			rc = 0;
> +		LDLM_LOCK_PUT(lock);
> +	}
> +	return rc;
> +}
> +EXPORT_SYMBOL(ldlm_lock_set_data);
> +
> +struct export_cl_data {
> +	struct obd_export	*ecl_exp;
> +	int			ecl_loop;
> +};
> +
> +/**
> + * Print lock with lock handle \a lockh description into debug log.
> + *
> + * Used when printing all locks on a resource for debug purposes.
> + */
> +void ldlm_lock_dump_handle(int level, const struct lustre_handle *lockh)
> +{
> +	struct ldlm_lock *lock;
> +
> +	if (!((libcfs_debug | D_ERROR) & level))
> +		return;
> +
> +	lock = ldlm_handle2lock(lockh);
> +	if (!lock)
> +		return;
> +
> +	LDLM_DEBUG_LIMIT(level, lock, "###");
> +
> +	LDLM_LOCK_PUT(lock);
> +}
> +EXPORT_SYMBOL(ldlm_lock_dump_handle);
> +
> +/**
> + * Print lock information with custom message into debug log.
> + * Helper function.
> + */
> +void _ldlm_lock_debug(struct ldlm_lock *lock,
> +		      struct libcfs_debug_msg_data *msgdata,
> +		      const char *fmt, ...)
> +{
> +	va_list args;
> +	struct obd_export *exp = lock->l_export;
> +	struct ldlm_resource *resource = lock->l_resource;
> +	char *nid = "local";
> +
> +	va_start(args, fmt);
> +
> +	if (exp && exp->exp_connection) {
> +		nid = libcfs_nid2str(exp->exp_connection->c_peer.nid);
> +	} else if (exp && exp->exp_obd) {
> +		struct obd_import *imp = exp->exp_obd->u.cli.cl_import;
> +
> +		nid = libcfs_nid2str(imp->imp_connection->c_peer.nid);
> +	}
> +
> +	if (!resource) {
> +		libcfs_debug_vmsg2(msgdata, fmt, args,
> +				   " ns: \?\? lock: %p/%#llx lrc: %d/%d,%d mode: %s/%s res: \?\? rrc=\?\? type: \?\?\? flags: %#llx nid: %s remote: %#llx expref: %d pid: %u timeout: %lu lvb_type: %d\n",
> +				   lock,
> +				   lock->l_handle.h_cookie,
> +				   atomic_read(&lock->l_refc),
> +				   lock->l_readers, lock->l_writers,
> +				   ldlm_lockname[lock->l_granted_mode],
> +				   ldlm_lockname[lock->l_req_mode],
> +				   lock->l_flags, nid,
> +				   lock->l_remote_handle.cookie,
> +				   exp ? atomic_read(&exp->exp_refcount) : -99,
> +				   lock->l_pid, lock->l_callback_timeout,
> +				   lock->l_lvb_type);
> +		va_end(args);
> +		return;
> +	}
> +
> +	switch (resource->lr_type) {
> +	case LDLM_EXTENT:
> +		libcfs_debug_vmsg2(msgdata, fmt, args,
> +				   " ns: %s lock: %p/%#llx lrc: %d/%d,%d mode: %s/%s res: " DLDLMRES " rrc: %d type: %s [%llu->%llu] (req %llu->%llu) flags: %#llx nid: %s remote: %#llx expref: %d pid: %u timeout: %lu lvb_type: %d\n",
> +				   ldlm_lock_to_ns_name(lock), lock,
> +				   lock->l_handle.h_cookie,
> +				   atomic_read(&lock->l_refc),
> +				   lock->l_readers, lock->l_writers,
> +				   ldlm_lockname[lock->l_granted_mode],
> +				   ldlm_lockname[lock->l_req_mode],
> +				   PLDLMRES(resource),
> +				   atomic_read(&resource->lr_refcount),
> +				   ldlm_typename[resource->lr_type],
> +				   lock->l_policy_data.l_extent.start,
> +				   lock->l_policy_data.l_extent.end,
> +				   lock->l_req_extent.start,
> +				   lock->l_req_extent.end,
> +				   lock->l_flags, nid,
> +				   lock->l_remote_handle.cookie,
> +				   exp ? atomic_read(&exp->exp_refcount) : -99,
> +				   lock->l_pid, lock->l_callback_timeout,
> +				   lock->l_lvb_type);
> +		break;
> +
> +	case LDLM_FLOCK:
> +		libcfs_debug_vmsg2(msgdata, fmt, args,
> +				   " ns: %s lock: %p/%#llx lrc: %d/%d,%d mode: %s/%s res: " DLDLMRES " rrc: %d type: %s pid: %d [%llu->%llu] flags: %#llx nid: %s remote: %#llx expref: %d pid: %u timeout: %lu\n",
> +				   ldlm_lock_to_ns_name(lock), lock,
> +				   lock->l_handle.h_cookie,
> +				   atomic_read(&lock->l_refc),
> +				   lock->l_readers, lock->l_writers,
> +				   ldlm_lockname[lock->l_granted_mode],
> +				   ldlm_lockname[lock->l_req_mode],
> +				   PLDLMRES(resource),
> +				   atomic_read(&resource->lr_refcount),
> +				   ldlm_typename[resource->lr_type],
> +				   lock->l_policy_data.l_flock.pid,
> +				   lock->l_policy_data.l_flock.start,
> +				   lock->l_policy_data.l_flock.end,
> +				   lock->l_flags, nid,
> +				   lock->l_remote_handle.cookie,
> +				   exp ? atomic_read(&exp->exp_refcount) : -99,
> +				   lock->l_pid, lock->l_callback_timeout);
> +		break;
> +
> +	case LDLM_IBITS:
> +		libcfs_debug_vmsg2(msgdata, fmt, args,
> +				   " ns: %s lock: %p/%#llx lrc: %d/%d,%d mode: %s/%s res: " DLDLMRES " bits %#llx rrc: %d type: %s flags: %#llx nid: %s remote: %#llx expref: %d pid: %u timeout: %lu lvb_type: %d\n",
> +				   ldlm_lock_to_ns_name(lock),
> +				   lock, lock->l_handle.h_cookie,
> +				   atomic_read(&lock->l_refc),
> +				   lock->l_readers, lock->l_writers,
> +				   ldlm_lockname[lock->l_granted_mode],
> +				   ldlm_lockname[lock->l_req_mode],
> +				   PLDLMRES(resource),
> +				   lock->l_policy_data.l_inodebits.bits,
> +				   atomic_read(&resource->lr_refcount),
> +				   ldlm_typename[resource->lr_type],
> +				   lock->l_flags, nid,
> +				   lock->l_remote_handle.cookie,
> +				   exp ? atomic_read(&exp->exp_refcount) : -99,
> +				   lock->l_pid, lock->l_callback_timeout,
> +				   lock->l_lvb_type);
> +		break;
> +
> +	default:
> +		libcfs_debug_vmsg2(msgdata, fmt, args,
> +				   " ns: %s lock: %p/%#llx lrc: %d/%d,%d mode: %s/%s res: " DLDLMRES " rrc: %d type: %s flags: %#llx nid: %s remote: %#llx expref: %d pid: %u timeout: %lu lvb_type: %d\n",
> +				   ldlm_lock_to_ns_name(lock),
> +				   lock, lock->l_handle.h_cookie,
> +				   atomic_read(&lock->l_refc),
> +				   lock->l_readers, lock->l_writers,
> +				   ldlm_lockname[lock->l_granted_mode],
> +				   ldlm_lockname[lock->l_req_mode],
> +				   PLDLMRES(resource),
> +				   atomic_read(&resource->lr_refcount),
> +				   ldlm_typename[resource->lr_type],
> +				   lock->l_flags, nid,
> +				   lock->l_remote_handle.cookie,
> +				   exp ? atomic_read(&exp->exp_refcount) : -99,
> +				   lock->l_pid, lock->l_callback_timeout,
> +				   lock->l_lvb_type);
> +		break;
> +	}
> +	va_end(args);
> +}
> +EXPORT_SYMBOL(_ldlm_lock_debug);
> diff --git a/drivers/staging/lustre/lustre/ptlrpc/ldlm_lockd.c b/drivers/staging/lustre/lustre/ptlrpc/ldlm_lockd.c
> new file mode 100644
> index 000000000000..f410ef6c02ef
> --- /dev/null
> +++ b/drivers/staging/lustre/lustre/ptlrpc/ldlm_lockd.c
> @@ -0,0 +1,1154 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * GPL HEADER START
> + *
> + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 only,
> + * as published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful, but
> + * WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> + * General Public License version 2 for more details (a copy is included
> + * in the LICENSE file that accompanied this code).
> + *
> + * You should have received a copy of the GNU General Public License
> + * version 2 along with this program; If not, see
> + * http://www.gnu.org/licenses/gpl-2.0.html
> + *
> + * GPL HEADER END
> + */
> +/*
> + * Copyright (c) 2002, 2010, Oracle and/or its affiliates. All rights reserved.
> + * Use is subject to license terms.
> + *
> + * Copyright (c) 2010, 2015, Intel Corporation.
> + */
> +/*
> + * This file is part of Lustre, http://www.lustre.org/
> + * Lustre is a trademark of Sun Microsystems, Inc.
> + *
> + * lustre/ldlm/ldlm_lockd.c
> + *
> + * Author: Peter Braam <braam at clusterfs.com>
> + * Author: Phil Schwan <phil at clusterfs.com>
> + */
> +
> +#define DEBUG_SUBSYSTEM S_LDLM
> +
> +#include <linux/kthread.h>
> +#include <linux/sched/mm.h>
> +#include <lustre_dlm.h>
> +#include <obd_class.h>
> +#include <linux/list.h>
> +#include "ldlm_internal.h"
> +
> +static int ldlm_num_threads;
> +module_param(ldlm_num_threads, int, 0444);
> +MODULE_PARM_DESC(ldlm_num_threads, "number of DLM service threads to start");
> +
> +static char *ldlm_cpts;
> +module_param(ldlm_cpts, charp, 0444);
> +MODULE_PARM_DESC(ldlm_cpts, "CPU partitions ldlm threads should run on");
> +
> +static struct mutex	ldlm_ref_mutex;
> +static int ldlm_refcount;
> +
> +static struct kobject *ldlm_kobj;
> +struct kset *ldlm_ns_kset;
> +static struct kset *ldlm_svc_kset;
> +
> +struct ldlm_cb_async_args {
> +	struct ldlm_cb_set_arg *ca_set_arg;
> +	struct ldlm_lock       *ca_lock;
> +};
> +
> +/* LDLM state */
> +
> +static struct ldlm_state *ldlm_state;
> +
> +#define ELT_STOPPED   0
> +#define ELT_READY     1
> +#define ELT_TERMINATE 2
> +
> +struct ldlm_bl_pool {
> +	spinlock_t		blp_lock;
> +
> +	/*
> +	 * blp_prio_list is used for callbacks that should be handled
> +	 * as a priority. It is used for LDLM_FL_DISCARD_DATA requests.
> +	 * see bug 13843
> +	 */
> +	struct list_head	      blp_prio_list;
> +
> +	/*
> +	 * blp_list is used for all other callbacks which are likely
> +	 * to take longer to process.
> +	 */
> +	struct list_head	      blp_list;
> +
> +	wait_queue_head_t	     blp_waitq;
> +	struct completion	blp_comp;
> +	atomic_t	    blp_num_threads;
> +	atomic_t	    blp_busy_threads;
> +	int		     blp_min_threads;
> +	int		     blp_max_threads;
> +};
> +
> +struct ldlm_bl_work_item {
> +	struct list_head	      blwi_entry;
> +	struct ldlm_namespace  *blwi_ns;
> +	struct ldlm_lock_desc   blwi_ld;
> +	struct ldlm_lock       *blwi_lock;
> +	struct list_head	      blwi_head;
> +	int		     blwi_count;
> +	struct completion	blwi_comp;
> +	enum ldlm_cancel_flags  blwi_flags;
> +	int		     blwi_mem_pressure;
> +};
> +
> +/**
> + * Callback handler for receiving incoming blocking ASTs.
> + *
> + * This can only happen on client side.
> + */
> +void ldlm_handle_bl_callback(struct ldlm_namespace *ns,
> +			     struct ldlm_lock_desc *ld, struct ldlm_lock *lock)
> +{
> +	int do_ast;
> +
> +	LDLM_DEBUG(lock, "client blocking AST callback handler");
> +
> +	lock_res_and_lock(lock);
> +	ldlm_set_cbpending(lock);
> +
> +	if (ldlm_is_cancel_on_block(lock))
> +		ldlm_set_cancel(lock);
> +
> +	do_ast = !lock->l_readers && !lock->l_writers;
> +	unlock_res_and_lock(lock);
> +
> +	if (do_ast) {
> +		CDEBUG(D_DLMTRACE,
> +		       "Lock %p already unused, calling callback (%p)\n", lock,
> +		       lock->l_blocking_ast);
> +		if (lock->l_blocking_ast)
> +			lock->l_blocking_ast(lock, ld, lock->l_ast_data,
> +					     LDLM_CB_BLOCKING);
> +	} else {
> +		CDEBUG(D_DLMTRACE,
> +		       "Lock %p is referenced, will be cancelled later\n",
> +		       lock);
> +	}
> +
> +	LDLM_DEBUG(lock, "client blocking callback handler END");
> +	LDLM_LOCK_RELEASE(lock);
> +}
> +
> +/**
> + * Callback handler for receiving incoming completion ASTs.
> + *
> + * This only can happen on client side.
> + */
> +static void ldlm_handle_cp_callback(struct ptlrpc_request *req,
> +				    struct ldlm_namespace *ns,
> +				    struct ldlm_request *dlm_req,
> +				    struct ldlm_lock *lock)
> +{
> +	int lvb_len;
> +	LIST_HEAD(ast_list);
> +	int rc = 0;
> +
> +	LDLM_DEBUG(lock, "client completion callback handler START");
> +
> +	if (OBD_FAIL_CHECK(OBD_FAIL_LDLM_CANCEL_BL_CB_RACE)) {
> +		int to = HZ;
> +
> +		while (to > 0) {
> +			set_current_state(TASK_INTERRUPTIBLE);
> +			schedule_timeout(to);
> +			if (lock->l_granted_mode == lock->l_req_mode ||
> +			    ldlm_is_destroyed(lock))
> +				break;
> +		}
> +	}
> +
> +	lvb_len = req_capsule_get_size(&req->rq_pill, &RMF_DLM_LVB, RCL_CLIENT);
> +	if (lvb_len < 0) {
> +		LDLM_ERROR(lock, "Fail to get lvb_len, rc = %d", lvb_len);
> +		rc = lvb_len;
> +		goto out;
> +	} else if (lvb_len > 0) {
> +		if (lock->l_lvb_len > 0) {
> +			/* for extent lock, lvb contains ost_lvb{}. */
> +			LASSERT(lock->l_lvb_data);
> +
> +			if (unlikely(lock->l_lvb_len < lvb_len)) {
> +				LDLM_ERROR(lock,
> +					   "Replied LVB is larger than expectation, expected = %d, replied = %d",
> +					   lock->l_lvb_len, lvb_len);
> +				rc = -EINVAL;
> +				goto out;
> +			}
> +		} else if (ldlm_has_layout(lock)) { /* for layout lock, lvb has
> +						     * variable length
> +						     */
> +			void *lvb_data;
> +
> +			lvb_data = kzalloc(lvb_len, GFP_NOFS);
> +			if (!lvb_data) {
> +				LDLM_ERROR(lock, "No memory: %d.\n", lvb_len);
> +				rc = -ENOMEM;
> +				goto out;
> +			}
> +
> +			lock_res_and_lock(lock);
> +			LASSERT(!lock->l_lvb_data);
> +			lock->l_lvb_type = LVB_T_LAYOUT;
> +			lock->l_lvb_data = lvb_data;
> +			lock->l_lvb_len = lvb_len;
> +			unlock_res_and_lock(lock);
> +		}
> +	}
> +
> +	lock_res_and_lock(lock);
> +	if (ldlm_is_destroyed(lock) ||
> +	    lock->l_granted_mode == lock->l_req_mode) {
> +		/* bug 11300: the lock has already been granted */
> +		unlock_res_and_lock(lock);
> +		LDLM_DEBUG(lock, "Double grant race happened");
> +		rc = 0;
> +		goto out;
> +	}
> +
> +	/* If we receive the completion AST before the actual enqueue returned,
> +	 * then we might need to switch lock modes, resources, or extents.
> +	 */
> +	if (dlm_req->lock_desc.l_granted_mode != lock->l_req_mode) {
> +		lock->l_req_mode = dlm_req->lock_desc.l_granted_mode;
> +		LDLM_DEBUG(lock, "completion AST, new lock mode");
> +	}
> +
> +	if (lock->l_resource->lr_type != LDLM_PLAIN) {
> +		ldlm_convert_policy_to_local(req->rq_export,
> +					  dlm_req->lock_desc.l_resource.lr_type,
> +					  &dlm_req->lock_desc.l_policy_data,
> +					  &lock->l_policy_data);
> +		LDLM_DEBUG(lock, "completion AST, new policy data");
> +	}
> +
> +	ldlm_resource_unlink_lock(lock);
> +	if (memcmp(&dlm_req->lock_desc.l_resource.lr_name,
> +		   &lock->l_resource->lr_name,
> +		   sizeof(lock->l_resource->lr_name)) != 0) {
> +		unlock_res_and_lock(lock);
> +		rc = ldlm_lock_change_resource(ns, lock,
> +				&dlm_req->lock_desc.l_resource.lr_name);
> +		if (rc < 0) {
> +			LDLM_ERROR(lock, "Failed to allocate resource");
> +			goto out;
> +		}
> +		LDLM_DEBUG(lock, "completion AST, new resource");
> +		CERROR("change resource!\n");
> +		lock_res_and_lock(lock);
> +	}
> +
> +	if (dlm_req->lock_flags & LDLM_FL_AST_SENT) {
> +		/* BL_AST locks are not needed in LRU.
> +		 * Let ldlm_cancel_lru() be fast.
> +		 */
> +		ldlm_lock_remove_from_lru(lock);
> +		lock->l_flags |= LDLM_FL_CBPENDING | LDLM_FL_BL_AST;
> +		LDLM_DEBUG(lock, "completion AST includes blocking AST");
> +	}
> +
> +	if (lock->l_lvb_len > 0) {
> +		rc = ldlm_fill_lvb(lock, &req->rq_pill, RCL_CLIENT,
> +				   lock->l_lvb_data, lvb_len);
> +		if (rc < 0) {
> +			unlock_res_and_lock(lock);
> +			goto out;
> +		}
> +	}
> +
> +	ldlm_grant_lock(lock, &ast_list);
> +	unlock_res_and_lock(lock);
> +
> +	LDLM_DEBUG(lock, "callback handler finished, about to run_ast_work");
> +
> +	/* Let Enqueue to call osc_lock_upcall() and initialize l_ast_data */
> +	OBD_FAIL_TIMEOUT(OBD_FAIL_OSC_CP_ENQ_RACE, 2);
> +
> +	ldlm_run_ast_work(ns, &ast_list, LDLM_WORK_CP_AST);
> +
> +	LDLM_DEBUG_NOLOCK("client completion callback handler END (lock %p)",
> +			  lock);
> +	goto out;
> +
> +out:
> +	if (rc < 0) {
> +		lock_res_and_lock(lock);
> +		ldlm_set_failed(lock);
> +		unlock_res_and_lock(lock);
> +		wake_up(&lock->l_waitq);
> +	}
> +	LDLM_LOCK_RELEASE(lock);
> +}
> +
> +/**
> + * Callback handler for receiving incoming glimpse ASTs.
> + *
> + * This only can happen on client side.  After handling the glimpse AST
> + * we also consider dropping the lock here if it is unused locally for a
> + * long time.
> + */
> +static void ldlm_handle_gl_callback(struct ptlrpc_request *req,
> +				    struct ldlm_namespace *ns,
> +				    struct ldlm_request *dlm_req,
> +				    struct ldlm_lock *lock)
> +{
> +	int rc = -ENOSYS;
> +
> +	LDLM_DEBUG(lock, "client glimpse AST callback handler");
> +
> +	if (lock->l_glimpse_ast)
> +		rc = lock->l_glimpse_ast(lock, req);
> +
> +	if (req->rq_repmsg) {
> +		ptlrpc_reply(req);
> +	} else {
> +		req->rq_status = rc;
> +		ptlrpc_error(req);
> +	}
> +
> +	lock_res_and_lock(lock);
> +	if (lock->l_granted_mode == LCK_PW &&
> +	    !lock->l_readers && !lock->l_writers &&
> +	    time_after(jiffies,
> +		       lock->l_last_used + 10 * HZ)) {
> +		unlock_res_and_lock(lock);
> +		if (ldlm_bl_to_thread_lock(ns, NULL, lock))
> +			ldlm_handle_bl_callback(ns, NULL, lock);
> +
> +		return;
> +	}
> +	unlock_res_and_lock(lock);
> +	LDLM_LOCK_RELEASE(lock);
> +}
> +
> +static int ldlm_callback_reply(struct ptlrpc_request *req, int rc)
> +{
> +	if (req->rq_no_reply)
> +		return 0;
> +
> +	req->rq_status = rc;
> +	if (!req->rq_packed_final) {
> +		rc = lustre_pack_reply(req, 1, NULL, NULL);
> +		if (rc)
> +			return rc;
> +	}
> +	return ptlrpc_reply(req);
> +}
> +
> +static int __ldlm_bl_to_thread(struct ldlm_bl_work_item *blwi,
> +			       enum ldlm_cancel_flags cancel_flags)
> +{
> +	struct ldlm_bl_pool *blp = ldlm_state->ldlm_bl_pool;
> +
> +	spin_lock(&blp->blp_lock);
> +	if (blwi->blwi_lock && ldlm_is_discard_data(blwi->blwi_lock)) {
> +		/* add LDLM_FL_DISCARD_DATA requests to the priority list */
> +		list_add_tail(&blwi->blwi_entry, &blp->blp_prio_list);
> +	} else {
> +		/* other blocking callbacks are added to the regular list */
> +		list_add_tail(&blwi->blwi_entry, &blp->blp_list);
> +	}
> +	spin_unlock(&blp->blp_lock);
> +
> +	wake_up(&blp->blp_waitq);
> +
> +	/* can not check blwi->blwi_flags as blwi could be already freed in
> +	 * LCF_ASYNC mode
> +	 */
> +	if (!(cancel_flags & LCF_ASYNC))
> +		wait_for_completion(&blwi->blwi_comp);
> +
> +	return 0;
> +}
> +
> +static inline void init_blwi(struct ldlm_bl_work_item *blwi,
> +			     struct ldlm_namespace *ns,
> +			     struct ldlm_lock_desc *ld,
> +			     struct list_head *cancels, int count,
> +			     struct ldlm_lock *lock,
> +			     enum ldlm_cancel_flags cancel_flags)
> +{
> +	init_completion(&blwi->blwi_comp);
> +	INIT_LIST_HEAD(&blwi->blwi_head);
> +
> +	if (current->flags & PF_MEMALLOC)
> +		blwi->blwi_mem_pressure = 1;
> +
> +	blwi->blwi_ns = ns;
> +	blwi->blwi_flags = cancel_flags;
> +	if (ld)
> +		blwi->blwi_ld = *ld;
> +	if (count) {
> +		list_add(&blwi->blwi_head, cancels);
> +		list_del_init(cancels);
> +		blwi->blwi_count = count;
> +	} else {
> +		blwi->blwi_lock = lock;
> +	}
> +}
> +
> +/**
> + * Queues a list of locks \a cancels containing \a count locks
> + * for later processing by a blocking thread.  If \a count is zero,
> + * then the lock referenced as \a lock is queued instead.
> + *
> + * The blocking thread would then call ->l_blocking_ast callback in the lock.
> + * If list addition fails an error is returned and caller is supposed to
> + * call ->l_blocking_ast itself.
> + */
> +static int ldlm_bl_to_thread(struct ldlm_namespace *ns,
> +			     struct ldlm_lock_desc *ld,
> +			     struct ldlm_lock *lock,
> +			     struct list_head *cancels, int count,
> +			     enum ldlm_cancel_flags cancel_flags)
> +{
> +	if (cancels && count == 0)
> +		return 0;
> +
> +	if (cancel_flags & LCF_ASYNC) {
> +		struct ldlm_bl_work_item *blwi;
> +
> +		blwi = kzalloc(sizeof(*blwi), GFP_NOFS);
> +		if (!blwi)
> +			return -ENOMEM;
> +		init_blwi(blwi, ns, ld, cancels, count, lock, cancel_flags);
> +
> +		return __ldlm_bl_to_thread(blwi, cancel_flags);
> +	} else {
> +		/* if it is synchronous call do minimum mem alloc, as it could
> +		 * be triggered from kernel shrinker
> +		 */
> +		struct ldlm_bl_work_item blwi;
> +
> +		memset(&blwi, 0, sizeof(blwi));
> +		init_blwi(&blwi, ns, ld, cancels, count, lock, cancel_flags);
> +		return __ldlm_bl_to_thread(&blwi, cancel_flags);
> +	}
> +}
> +
> +int ldlm_bl_to_thread_lock(struct ldlm_namespace *ns, struct ldlm_lock_desc *ld,
> +			   struct ldlm_lock *lock)
> +{
> +	return ldlm_bl_to_thread(ns, ld, lock, NULL, 0, LCF_ASYNC);
> +}
> +
> +int ldlm_bl_to_thread_list(struct ldlm_namespace *ns, struct ldlm_lock_desc *ld,
> +			   struct list_head *cancels, int count,
> +			   enum ldlm_cancel_flags cancel_flags)
> +{
> +	return ldlm_bl_to_thread(ns, ld, NULL, cancels, count, cancel_flags);
> +}
> +
> +int ldlm_bl_thread_wakeup(void)
> +{
> +	wake_up(&ldlm_state->ldlm_bl_pool->blp_waitq);
> +	return 0;
> +}
> +
> +/* Setinfo coming from Server (eg MDT) to Client (eg MDC)! */
> +static int ldlm_handle_setinfo(struct ptlrpc_request *req)
> +{
> +	struct obd_device *obd = req->rq_export->exp_obd;
> +	char *key;
> +	void *val;
> +	int keylen, vallen;
> +	int rc = -ENOSYS;
> +
> +	DEBUG_REQ(D_HSM, req, "%s: handle setinfo\n", obd->obd_name);
> +
> +	req_capsule_set(&req->rq_pill, &RQF_OBD_SET_INFO);
> +
> +	key = req_capsule_client_get(&req->rq_pill, &RMF_SETINFO_KEY);
> +	if (!key) {
> +		DEBUG_REQ(D_IOCTL, req, "no set_info key");
> +		return -EFAULT;
> +	}
> +	keylen = req_capsule_get_size(&req->rq_pill, &RMF_SETINFO_KEY,
> +				      RCL_CLIENT);
> +	val = req_capsule_client_get(&req->rq_pill, &RMF_SETINFO_VAL);
> +	if (!val) {
> +		DEBUG_REQ(D_IOCTL, req, "no set_info val");
> +		return -EFAULT;
> +	}
> +	vallen = req_capsule_get_size(&req->rq_pill, &RMF_SETINFO_VAL,
> +				      RCL_CLIENT);
> +
> +	/* We are responsible for swabbing contents of val */
> +
> +	if (KEY_IS(KEY_HSM_COPYTOOL_SEND))
> +		/* Pass it on to mdc (the "export" in this case) */
> +		rc = obd_set_info_async(req->rq_svc_thread->t_env,
> +					req->rq_export,
> +					sizeof(KEY_HSM_COPYTOOL_SEND),
> +					KEY_HSM_COPYTOOL_SEND,
> +					vallen, val, NULL);
> +	else
> +		DEBUG_REQ(D_WARNING, req, "ignoring unknown key %s", key);
> +
> +	return rc;
> +}
> +
> +static inline void ldlm_callback_errmsg(struct ptlrpc_request *req,
> +					const char *msg, int rc,
> +					const struct lustre_handle *handle)
> +{
> +	DEBUG_REQ((req->rq_no_reply || rc) ? D_WARNING : D_DLMTRACE, req,
> +		  "%s: [nid %s] [rc %d] [lock %#llx]",
> +		  msg, libcfs_id2str(req->rq_peer), rc,
> +		  handle ? handle->cookie : 0);
> +	if (req->rq_no_reply)
> +		CWARN("No reply was sent, maybe cause bug 21636.\n");
> +	else if (rc)
> +		CWARN("Send reply failed, maybe cause bug 21636.\n");
> +}
> +
> +/* TODO: handle requests in a similar way as MDT: see mdt_handle_common() */
> +static int ldlm_callback_handler(struct ptlrpc_request *req)
> +{
> +	struct ldlm_namespace *ns;
> +	struct ldlm_request *dlm_req;
> +	struct ldlm_lock *lock;
> +	int rc;
> +
> +	/* Requests arrive in sender's byte order.  The ptlrpc service
> +	 * handler has already checked and, if necessary, byte-swapped the
> +	 * incoming request message body, but I am responsible for the
> +	 * message buffers.
> +	 */
> +
> +	/* do nothing for sec context finalize */
> +	if (lustre_msg_get_opc(req->rq_reqmsg) == SEC_CTX_FINI)
> +		return 0;
> +
> +	req_capsule_init(&req->rq_pill, req, RCL_SERVER);
> +
> +	if (!req->rq_export) {
> +		rc = ldlm_callback_reply(req, -ENOTCONN);
> +		ldlm_callback_errmsg(req, "Operate on unconnected server",
> +				     rc, NULL);
> +		return 0;
> +	}
> +
> +	LASSERT(req->rq_export->exp_obd);
> +
> +	switch (lustre_msg_get_opc(req->rq_reqmsg)) {
> +	case LDLM_BL_CALLBACK:
> +		if (OBD_FAIL_CHECK(OBD_FAIL_LDLM_BL_CALLBACK_NET)) {
> +			if (cfs_fail_err)
> +				ldlm_callback_reply(req, -(int)cfs_fail_err);
> +			return 0;
> +		}
> +		break;
> +	case LDLM_CP_CALLBACK:
> +		if (OBD_FAIL_CHECK(OBD_FAIL_LDLM_CP_CALLBACK_NET))
> +			return 0;
> +		break;
> +	case LDLM_GL_CALLBACK:
> +		if (OBD_FAIL_CHECK(OBD_FAIL_LDLM_GL_CALLBACK_NET))
> +			return 0;
> +		break;
> +	case LDLM_SET_INFO:
> +		rc = ldlm_handle_setinfo(req);
> +		ldlm_callback_reply(req, rc);
> +		return 0;
> +	default:
> +		CERROR("unknown opcode %u\n",
> +		       lustre_msg_get_opc(req->rq_reqmsg));
> +		ldlm_callback_reply(req, -EPROTO);
> +		return 0;
> +	}
> +
> +	ns = req->rq_export->exp_obd->obd_namespace;
> +	LASSERT(ns);
> +
> +	req_capsule_set(&req->rq_pill, &RQF_LDLM_CALLBACK);
> +
> +	dlm_req = req_capsule_client_get(&req->rq_pill, &RMF_DLM_REQ);
> +	if (!dlm_req) {
> +		rc = ldlm_callback_reply(req, -EPROTO);
> +		ldlm_callback_errmsg(req, "Operate without parameter", rc,
> +				     NULL);
> +		return 0;
> +	}
> +
> +	/* Force a known safe race, send a cancel to the server for a lock
> +	 * which the server has already started a blocking callback on.
> +	 */
> +	if (OBD_FAIL_CHECK(OBD_FAIL_LDLM_CANCEL_BL_CB_RACE) &&
> +	    lustre_msg_get_opc(req->rq_reqmsg) == LDLM_BL_CALLBACK) {
> +		rc = ldlm_cli_cancel(&dlm_req->lock_handle[0], 0);
> +		if (rc < 0)
> +			CERROR("ldlm_cli_cancel: %d\n", rc);
> +	}
> +
> +	lock = ldlm_handle2lock_long(&dlm_req->lock_handle[0], 0);
> +	if (!lock) {
> +		CDEBUG(D_DLMTRACE,
> +		       "callback on lock %#llx - lock disappeared\n",
> +		       dlm_req->lock_handle[0].cookie);
> +		rc = ldlm_callback_reply(req, -EINVAL);
> +		ldlm_callback_errmsg(req, "Operate with invalid parameter", rc,
> +				     &dlm_req->lock_handle[0]);
> +		return 0;
> +	}
> +
> +	if (ldlm_is_fail_loc(lock) &&
> +	    lustre_msg_get_opc(req->rq_reqmsg) == LDLM_BL_CALLBACK)
> +		OBD_RACE(OBD_FAIL_LDLM_CP_BL_RACE);
> +
> +	/* Copy hints/flags (e.g. LDLM_FL_DISCARD_DATA) from AST. */
> +	lock_res_and_lock(lock);
> +	lock->l_flags |= ldlm_flags_from_wire(dlm_req->lock_flags &
> +					      LDLM_FL_AST_MASK);
> +	if (lustre_msg_get_opc(req->rq_reqmsg) == LDLM_BL_CALLBACK) {
> +		/* If somebody cancels lock and cache is already dropped,
> +		 * or lock is failed before cp_ast received on client,
> +		 * we can tell the server we have no lock. Otherwise, we
> +		 * should send cancel after dropping the cache.
> +		 */
> +		if ((ldlm_is_canceling(lock) && ldlm_is_bl_done(lock)) ||
> +		    ldlm_is_failed(lock)) {
> +			LDLM_DEBUG(lock,
> +				   "callback on lock %#llx - lock disappeared",
> +				   dlm_req->lock_handle[0].cookie);
> +			unlock_res_and_lock(lock);
> +			LDLM_LOCK_RELEASE(lock);
> +			rc = ldlm_callback_reply(req, -EINVAL);
> +			ldlm_callback_errmsg(req, "Operate on stale lock", rc,
> +					     &dlm_req->lock_handle[0]);
> +			return 0;
> +		}
> +		/* BL_AST locks are not needed in LRU.
> +		 * Let ldlm_cancel_lru() be fast.
> +		 */
> +		ldlm_lock_remove_from_lru(lock);
> +		ldlm_set_bl_ast(lock);
> +	}
> +	unlock_res_and_lock(lock);
> +
> +	/* We want the ost thread to get this reply so that it can respond
> +	 * to ost requests (write cache writeback) that might be triggered
> +	 * in the callback.
> +	 *
> +	 * But we'd also like to be able to indicate in the reply that we're
> +	 * cancelling right now, because it's unused, or have an intent result
> +	 * in the reply, so we might have to push the responsibility for sending
> +	 * the reply down into the AST handlers, alas.
> +	 */
> +
> +	switch (lustre_msg_get_opc(req->rq_reqmsg)) {
> +	case LDLM_BL_CALLBACK:
> +		CDEBUG(D_INODE, "blocking ast\n");
> +		req_capsule_extend(&req->rq_pill, &RQF_LDLM_BL_CALLBACK);
> +		if (!ldlm_is_cancel_on_block(lock)) {
> +			rc = ldlm_callback_reply(req, 0);
> +			if (req->rq_no_reply || rc)
> +				ldlm_callback_errmsg(req, "Normal process", rc,
> +						     &dlm_req->lock_handle[0]);
> +		}
> +		if (ldlm_bl_to_thread_lock(ns, &dlm_req->lock_desc, lock))
> +			ldlm_handle_bl_callback(ns, &dlm_req->lock_desc, lock);
> +		break;
> +	case LDLM_CP_CALLBACK:
> +		CDEBUG(D_INODE, "completion ast\n");
> +		req_capsule_extend(&req->rq_pill, &RQF_LDLM_CP_CALLBACK);
> +		ldlm_callback_reply(req, 0);
> +		ldlm_handle_cp_callback(req, ns, dlm_req, lock);
> +		break;
> +	case LDLM_GL_CALLBACK:
> +		CDEBUG(D_INODE, "glimpse ast\n");
> +		req_capsule_extend(&req->rq_pill, &RQF_LDLM_GL_CALLBACK);
> +		ldlm_handle_gl_callback(req, ns, dlm_req, lock);
> +		break;
> +	default:
> +		LBUG();			 /* checked above */
> +	}
> +
> +	return 0;
> +}
> +
> +static int ldlm_bl_get_work(struct ldlm_bl_pool *blp,
> +			    struct ldlm_bl_work_item **p_blwi,
> +			    struct obd_export **p_exp)
> +{
> +	int num_th = atomic_read(&blp->blp_num_threads);
> +	struct ldlm_bl_work_item *blwi = NULL;
> +	static unsigned int num_bl;
> +
> +	spin_lock(&blp->blp_lock);
> +	/* process a request from the blp_list at least every blp_num_threads */
> +	if (!list_empty(&blp->blp_list) &&
> +	    (list_empty(&blp->blp_prio_list) || num_bl == 0))
> +		blwi = list_first_entry(&blp->blp_list,
> +					struct ldlm_bl_work_item, blwi_entry);
> +	else
> +		if (!list_empty(&blp->blp_prio_list))
> +			blwi = list_first_entry(&blp->blp_prio_list,
> +						struct ldlm_bl_work_item,
> +						blwi_entry);
> +
> +	if (blwi) {
> +		if (++num_bl >= num_th)
> +			num_bl = 0;
> +		list_del(&blwi->blwi_entry);
> +	}
> +	spin_unlock(&blp->blp_lock);
> +	*p_blwi = blwi;
> +
> +	return (*p_blwi || *p_exp) ? 1 : 0;
> +}
> +
> +/* This only contains temporary data until the thread starts */
> +struct ldlm_bl_thread_data {
> +	struct ldlm_bl_pool	*bltd_blp;
> +	struct completion	bltd_comp;
> +	int			bltd_num;
> +};
> +
> +static int ldlm_bl_thread_main(void *arg);
> +
> +static int ldlm_bl_thread_start(struct ldlm_bl_pool *blp, bool check_busy)
> +{
> +	struct ldlm_bl_thread_data bltd = { .bltd_blp = blp };
> +	struct task_struct *task;
> +
> +	init_completion(&bltd.bltd_comp);
> +
> +	bltd.bltd_num = atomic_inc_return(&blp->blp_num_threads);
> +	if (bltd.bltd_num >= blp->blp_max_threads) {
> +		atomic_dec(&blp->blp_num_threads);
> +		return 0;
> +	}
> +
> +	LASSERTF(bltd.bltd_num > 0, "thread num:%d\n", bltd.bltd_num);
> +	if (check_busy &&
> +	    atomic_read(&blp->blp_busy_threads) < (bltd.bltd_num - 1)) {
> +		atomic_dec(&blp->blp_num_threads);
> +		return 0;
> +	}
> +
> +	task = kthread_run(ldlm_bl_thread_main, &bltd, "ldlm_bl_%02d",
> +			   bltd.bltd_num);
> +	if (IS_ERR(task)) {
> +		CERROR("cannot start LDLM thread ldlm_bl_%02d: rc %ld\n",
> +		       bltd.bltd_num, PTR_ERR(task));
> +		atomic_dec(&blp->blp_num_threads);
> +		return PTR_ERR(task);
> +	}
> +	wait_for_completion(&bltd.bltd_comp);
> +
> +	return 0;
> +}
> +
> +/* Not fatal if racy and have a few too many threads */
> +static int ldlm_bl_thread_need_create(struct ldlm_bl_pool *blp,
> +				      struct ldlm_bl_work_item *blwi)
> +{
> +	if (atomic_read(&blp->blp_num_threads) >= blp->blp_max_threads)
> +		return 0;
> +
> +	if (atomic_read(&blp->blp_busy_threads) <
> +	    atomic_read(&blp->blp_num_threads))
> +		return 0;
> +
> +	if (blwi && (!blwi->blwi_ns || blwi->blwi_mem_pressure))
> +		return 0;
> +
> +	return 1;
> +}
> +
> +static int ldlm_bl_thread_blwi(struct ldlm_bl_pool *blp,
> +			       struct ldlm_bl_work_item *blwi)
> +{
> +	unsigned int flags = 0;
> +
> +	if (!blwi->blwi_ns)
> +		/* added by ldlm_cleanup() */
> +		return LDLM_ITER_STOP;
> +
> +	if (blwi->blwi_mem_pressure)
> +		flags = memalloc_noreclaim_save();
> +
> +	OBD_FAIL_TIMEOUT(OBD_FAIL_LDLM_PAUSE_CANCEL2, 4);
> +
> +	if (blwi->blwi_count) {
> +		int count;
> +
> +		/*
> +		 * The special case when we cancel locks in lru
> +		 * asynchronously, we pass the list of locks here.
> +		 * Thus locks are marked LDLM_FL_CANCELING, but NOT
> +		 * canceled locally yet.
> +		 */
> +		count = ldlm_cli_cancel_list_local(&blwi->blwi_head,
> +						   blwi->blwi_count,
> +						   LCF_BL_AST);
> +		ldlm_cli_cancel_list(&blwi->blwi_head, count, NULL,
> +				     blwi->blwi_flags);
> +	} else {
> +		ldlm_handle_bl_callback(blwi->blwi_ns, &blwi->blwi_ld,
> +					blwi->blwi_lock);
> +	}
> +	if (blwi->blwi_mem_pressure)
> +		memalloc_noreclaim_restore(flags);
> +
> +	if (blwi->blwi_flags & LCF_ASYNC)
> +		kfree(blwi);
> +	else
> +		complete(&blwi->blwi_comp);
> +
> +	return 0;
> +}
> +
> +/**
> + * Main blocking requests processing thread.
> + *
> + * Callers put locks into its queue by calling ldlm_bl_to_thread.
> + * This thread in the end ends up doing actual call to ->l_blocking_ast
> + * for queued locks.
> + */
> +static int ldlm_bl_thread_main(void *arg)
> +{
> +	struct ldlm_bl_pool *blp;
> +	struct ldlm_bl_thread_data *bltd = arg;
> +
> +	blp = bltd->bltd_blp;
> +
> +	complete(&bltd->bltd_comp);
> +	/* cannot use bltd after this, it is only on caller's stack */
> +
> +	while (1) {
> +		struct ldlm_bl_work_item *blwi = NULL;
> +		struct obd_export *exp = NULL;
> +		int rc;
> +
> +		rc = ldlm_bl_get_work(blp, &blwi, &exp);
> +		if (!rc)
> +			wait_event_idle_exclusive(blp->blp_waitq,
> +						  ldlm_bl_get_work(blp, &blwi,
> +								   &exp));
> +		atomic_inc(&blp->blp_busy_threads);
> +
> +		if (ldlm_bl_thread_need_create(blp, blwi))
> +			/* discard the return value, we tried */
> +			ldlm_bl_thread_start(blp, true);
> +
> +		if (blwi)
> +			rc = ldlm_bl_thread_blwi(blp, blwi);
> +
> +		atomic_dec(&blp->blp_busy_threads);
> +
> +		if (rc == LDLM_ITER_STOP)
> +			break;
> +	}
> +
> +	atomic_dec(&blp->blp_num_threads);
> +	complete(&blp->blp_comp);
> +	return 0;
> +}
> +
> +static int ldlm_setup(void);
> +static int ldlm_cleanup(void);
> +
> +int ldlm_get_ref(void)
> +{
> +	int rc = 0;
> +
> +	rc = ptlrpc_inc_ref();
> +	if (rc)
> +		return rc;
> +
> +	mutex_lock(&ldlm_ref_mutex);
> +	if (++ldlm_refcount == 1) {
> +		rc = ldlm_setup();
> +		if (rc)
> +			ldlm_refcount--;
> +	}
> +	mutex_unlock(&ldlm_ref_mutex);
> +
> +	if (rc)
> +		ptlrpc_dec_ref();
> +
> +	return rc;
> +}
> +
> +void ldlm_put_ref(void)
> +{
> +	int rc = 0;
> +	mutex_lock(&ldlm_ref_mutex);
> +	if (ldlm_refcount == 1) {
> +		rc = ldlm_cleanup();
> +
> +		if (rc)
> +			CERROR("ldlm_cleanup failed: %d\n", rc);
> +		else
> +			ldlm_refcount--;
> +	} else {
> +		ldlm_refcount--;
> +	}
> +	mutex_unlock(&ldlm_ref_mutex);
> +	if (!rc)
> +		ptlrpc_dec_ref();
> +}
> +
> +static ssize_t cancel_unused_locks_before_replay_show(struct kobject *kobj,
> +						      struct attribute *attr,
> +						      char *buf)
> +{
> +	return sprintf(buf, "%d\n", ldlm_cancel_unused_locks_before_replay);
> +}
> +
> +static ssize_t cancel_unused_locks_before_replay_store(struct kobject *kobj,
> +						       struct attribute *attr,
> +						       const char *buffer,
> +						       size_t count)
> +{
> +	int rc;
> +	unsigned long val;
> +
> +	rc = kstrtoul(buffer, 10, &val);
> +	if (rc)
> +		return rc;
> +
> +	ldlm_cancel_unused_locks_before_replay = val;
> +
> +	return count;
> +}
> +LUSTRE_RW_ATTR(cancel_unused_locks_before_replay);
> +
> +/* These are for root of /sys/fs/lustre/ldlm */
> +static struct attribute *ldlm_attrs[] = {
> +	&lustre_attr_cancel_unused_locks_before_replay.attr,
> +	NULL,
> +};
> +
> +static const struct attribute_group ldlm_attr_group = {
> +	.attrs = ldlm_attrs,
> +};
> +
> +static int ldlm_setup(void)
> +{
> +	static struct ptlrpc_service_conf	conf;
> +	struct ldlm_bl_pool			*blp = NULL;
> +	int rc = 0;
> +	int i;
> +
> +	if (ldlm_state)
> +		return -EALREADY;
> +
> +	ldlm_state = kzalloc(sizeof(*ldlm_state), GFP_NOFS);
> +	if (!ldlm_state)
> +		return -ENOMEM;
> +
> +	ldlm_kobj = kobject_create_and_add("ldlm", lustre_kobj);
> +	if (!ldlm_kobj) {
> +		rc = -ENOMEM;
> +		goto out;
> +	}
> +
> +	rc = sysfs_create_group(ldlm_kobj, &ldlm_attr_group);
> +	if (rc)
> +		goto out;
> +
> +	ldlm_ns_kset = kset_create_and_add("namespaces", NULL, ldlm_kobj);
> +	if (!ldlm_ns_kset) {
> +		rc = -ENOMEM;
> +		goto out;
> +	}
> +
> +	ldlm_svc_kset = kset_create_and_add("services", NULL, ldlm_kobj);
> +	if (!ldlm_svc_kset) {
> +		rc = -ENOMEM;
> +		goto out;
> +	}
> +
> +	ldlm_debugfs_setup();
> +
> +	memset(&conf, 0, sizeof(conf));
> +	conf = (typeof(conf)) {
> +		.psc_name		= "ldlm_cbd",
> +		.psc_watchdog_factor	= 2,
> +		.psc_buf		= {
> +			.bc_nbufs		= LDLM_CLIENT_NBUFS,
> +			.bc_buf_size		= LDLM_BUFSIZE,
> +			.bc_req_max_size	= LDLM_MAXREQSIZE,
> +			.bc_rep_max_size	= LDLM_MAXREPSIZE,
> +			.bc_req_portal		= LDLM_CB_REQUEST_PORTAL,
> +			.bc_rep_portal		= LDLM_CB_REPLY_PORTAL,
> +		},
> +		.psc_thr		= {
> +			.tc_thr_name		= "ldlm_cb",
> +			.tc_thr_factor		= LDLM_THR_FACTOR,
> +			.tc_nthrs_init		= LDLM_NTHRS_INIT,
> +			.tc_nthrs_base		= LDLM_NTHRS_BASE,
> +			.tc_nthrs_max		= LDLM_NTHRS_MAX,
> +			.tc_nthrs_user		= ldlm_num_threads,
> +			.tc_cpu_affinity	= 1,
> +			.tc_ctx_tags		= LCT_MD_THREAD | LCT_DT_THREAD,
> +		},
> +		.psc_cpt		= {
> +			.cc_pattern		= ldlm_cpts,
> +		},
> +		.psc_ops		= {
> +			.so_req_handler		= ldlm_callback_handler,
> +		},
> +	};
> +	ldlm_state->ldlm_cb_service =
> +			ptlrpc_register_service(&conf, ldlm_svc_kset,
> +						ldlm_svc_debugfs_dir);
> +	if (IS_ERR(ldlm_state->ldlm_cb_service)) {
> +		CERROR("failed to start service\n");
> +		rc = PTR_ERR(ldlm_state->ldlm_cb_service);
> +		ldlm_state->ldlm_cb_service = NULL;
> +		goto out;
> +	}
> +
> +	blp = kzalloc(sizeof(*blp), GFP_NOFS);
> +	if (!blp) {
> +		rc = -ENOMEM;
> +		goto out;
> +	}
> +	ldlm_state->ldlm_bl_pool = blp;
> +
> +	spin_lock_init(&blp->blp_lock);
> +	INIT_LIST_HEAD(&blp->blp_list);
> +	INIT_LIST_HEAD(&blp->blp_prio_list);
> +	init_waitqueue_head(&blp->blp_waitq);
> +	atomic_set(&blp->blp_num_threads, 0);
> +	atomic_set(&blp->blp_busy_threads, 0);
> +
> +	if (ldlm_num_threads == 0) {
> +		blp->blp_min_threads = LDLM_NTHRS_INIT;
> +		blp->blp_max_threads = LDLM_NTHRS_MAX;
> +	} else {
> +		blp->blp_min_threads = min_t(int, LDLM_NTHRS_MAX,
> +					     max_t(int, LDLM_NTHRS_INIT,
> +						   ldlm_num_threads));
> +
> +		blp->blp_max_threads = blp->blp_min_threads;
> +	}
> +
> +	for (i = 0; i < blp->blp_min_threads; i++) {
> +		rc = ldlm_bl_thread_start(blp, false);
> +		if (rc < 0)
> +			goto out;
> +	}
> +
> +	rc = ldlm_pools_init();
> +	if (rc) {
> +		CERROR("Failed to initialize LDLM pools: %d\n", rc);
> +		goto out;
> +	}
> +	return 0;
> +
> + out:
> +	ldlm_cleanup();
> +	return rc;
> +}
> +
> +static int ldlm_cleanup(void)
> +{
> +	if (!list_empty(ldlm_namespace_list(LDLM_NAMESPACE_SERVER)) ||
> +	    !list_empty(ldlm_namespace_list(LDLM_NAMESPACE_CLIENT))) {
> +		CERROR("ldlm still has namespaces; clean these up first.\n");
> +		ldlm_dump_all_namespaces(LDLM_NAMESPACE_SERVER, D_DLMTRACE);
> +		ldlm_dump_all_namespaces(LDLM_NAMESPACE_CLIENT, D_DLMTRACE);
> +		return -EBUSY;
> +	}
> +
> +	ldlm_pools_fini();
> +
> +	if (ldlm_state->ldlm_bl_pool) {
> +		struct ldlm_bl_pool *blp = ldlm_state->ldlm_bl_pool;
> +
> +		while (atomic_read(&blp->blp_num_threads) > 0) {
> +			struct ldlm_bl_work_item blwi = { .blwi_ns = NULL };
> +
> +			init_completion(&blp->blp_comp);
> +
> +			spin_lock(&blp->blp_lock);
> +			list_add_tail(&blwi.blwi_entry, &blp->blp_list);
> +			wake_up(&blp->blp_waitq);
> +			spin_unlock(&blp->blp_lock);
> +
> +			wait_for_completion(&blp->blp_comp);
> +		}
> +
> +		kfree(blp);
> +	}
> +
> +	if (ldlm_state->ldlm_cb_service)
> +		ptlrpc_unregister_service(ldlm_state->ldlm_cb_service);
> +
> +	if (ldlm_ns_kset)
> +		kset_unregister(ldlm_ns_kset);
> +	if (ldlm_svc_kset)
> +		kset_unregister(ldlm_svc_kset);
> +	if (ldlm_kobj) {
> +		sysfs_remove_group(ldlm_kobj, &ldlm_attr_group);
> +		kobject_put(ldlm_kobj);
> +	}
> +
> +	ldlm_debugfs_cleanup();
> +
> +	kfree(ldlm_state);
> +	ldlm_state = NULL;
> +
> +	return 0;
> +}
> +
> +int ldlm_init(void)
> +{
> +	mutex_init(&ldlm_ref_mutex);
> +	mutex_init(ldlm_namespace_lock(LDLM_NAMESPACE_SERVER));
> +	mutex_init(ldlm_namespace_lock(LDLM_NAMESPACE_CLIENT));
> +	ldlm_resource_slab = kmem_cache_create("ldlm_resources",
> +					       sizeof(struct ldlm_resource), 0,
> +					       SLAB_HWCACHE_ALIGN, NULL);
> +	if (!ldlm_resource_slab)
> +		return -ENOMEM;
> +
> +	ldlm_lock_slab = kmem_cache_create("ldlm_locks",
> +					   sizeof(struct ldlm_lock), 0,
> +					   SLAB_HWCACHE_ALIGN |
> +					   SLAB_TYPESAFE_BY_RCU, NULL);
> +	if (!ldlm_lock_slab) {
> +		kmem_cache_destroy(ldlm_resource_slab);
> +		return -ENOMEM;
> +	}
> +
> +#if LUSTRE_TRACKS_LOCK_EXP_REFS
> +	class_export_dump_hook = ldlm_dump_export_locks;
> +#endif
> +	return 0;
> +}
> +
> +void ldlm_exit(void)
> +{
> +	if (ldlm_refcount)
> +		CERROR("ldlm_refcount is %d in %s!\n", ldlm_refcount, __func__);
> +	kmem_cache_destroy(ldlm_resource_slab);
> +	/* ldlm_lock_put() use RCU to call ldlm_lock_free, so need call
> +	 * synchronize_rcu() to wait a grace period elapsed, so that
> +	 * ldlm_lock_free() get a chance to be called.
> +	 */
> +	synchronize_rcu();
> +	kmem_cache_destroy(ldlm_lock_slab);
> +}
> diff --git a/drivers/staging/lustre/lustre/ptlrpc/ldlm_plain.c b/drivers/staging/lustre/lustre/ptlrpc/ldlm_plain.c
> new file mode 100644
> index 000000000000..33b5a3f96fcb
> --- /dev/null
> +++ b/drivers/staging/lustre/lustre/ptlrpc/ldlm_plain.c
> @@ -0,0 +1,68 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * GPL HEADER START
> + *
> + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 only,
> + * as published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful, but
> + * WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> + * General Public License version 2 for more details (a copy is included
> + * in the LICENSE file that accompanied this code).
> + *
> + * You should have received a copy of the GNU General Public License
> + * version 2 along with this program; If not, see
> + * http://www.gnu.org/licenses/gpl-2.0.html
> + *
> + * GPL HEADER END
> + */
> +/*
> + * Copyright (c) 2003, 2010, Oracle and/or its affiliates. All rights reserved.
> + * Use is subject to license terms.
> + *
> + * Copyright (c) 2011, Intel Corporation.
> + */
> +/*
> + * This file is part of Lustre, http://www.lustre.org/
> + * Lustre is a trademark of Sun Microsystems, Inc.
> + *
> + * lustre/ldlm/ldlm_plain.c
> + *
> + * Author: Peter Braam <braam at clusterfs.com>
> + * Author: Phil Schwan <phil at clusterfs.com>
> + */
> +
> +/**
> + * This file contains implementation of PLAIN lock type.
> + *
> + * PLAIN locks are the simplest form of LDLM locking, and are used when
> + * there only needs to be a single lock on a resource. This avoids some
> + * of the complexity of EXTENT and IBITS lock types, but doesn't allow
> + * different "parts" of a resource to be locked concurrently.  Example
> + * use cases for PLAIN locks include locking of MGS configuration logs
> + * and (as of Lustre 2.4) quota records.
> + */
> +
> +#define DEBUG_SUBSYSTEM S_LDLM
> +
> +#include <lustre_dlm.h>
> +#include <obd_support.h>
> +#include <lustre_lib.h>
> +
> +#include "ldlm_internal.h"
> +
> +void ldlm_plain_policy_wire_to_local(const union ldlm_wire_policy_data *wpolicy,
> +				     union ldlm_policy_data *lpolicy)
> +{
> +	/* No policy for plain locks */
> +}
> +
> +void ldlm_plain_policy_local_to_wire(const union ldlm_policy_data *lpolicy,
> +				     union ldlm_wire_policy_data *wpolicy)
> +{
> +	/* No policy for plain locks */
> +}
> diff --git a/drivers/staging/lustre/lustre/ptlrpc/ldlm_pool.c b/drivers/staging/lustre/lustre/ptlrpc/ldlm_pool.c
> new file mode 100644
> index 000000000000..36d14ee4e5b1
> --- /dev/null
> +++ b/drivers/staging/lustre/lustre/ptlrpc/ldlm_pool.c
> @@ -0,0 +1,1013 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * GPL HEADER START
> + *
> + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 only,
> + * as published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful, but
> + * WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> + * General Public License version 2 for more details (a copy is included
> + * in the LICENSE file that accompanied this code).
> + *
> + * You should have received a copy of the GNU General Public License
> + * version 2 along with this program; If not, see
> + * http://www.gnu.org/licenses/gpl-2.0.html
> + *
> + * GPL HEADER END
> + */
> +/*
> + * Copyright (c) 2007, 2010, Oracle and/or its affiliates. All rights reserved.
> + * Use is subject to license terms.
> + *
> + * Copyright (c) 2010, 2015, Intel Corporation.
> + */
> +/*
> + * This file is part of Lustre, http://www.lustre.org/
> + * Lustre is a trademark of Sun Microsystems, Inc.
> + *
> + * lustre/ldlm/ldlm_pool.c
> + *
> + * Author: Yury Umanets <umka at clusterfs.com>
> + */
> +
> +/*
> + * Idea of this code is rather simple. Each second, for each server namespace
> + * we have SLV - server lock volume which is calculated on current number of
> + * granted locks, grant speed for past period, etc - that is, locking load.
> + * This SLV number may be thought as a flow definition for simplicity. It is
> + * sent to clients with each occasion to let them know what is current load
> + * situation on the server. By default, at the beginning, SLV on server is
> + * set max value which is calculated as the following: allow to one client
> + * have all locks of limit ->pl_limit for 10h.
> + *
> + * Next, on clients, number of cached locks is not limited artificially in any
> + * way as it was before. Instead, client calculates CLV, that is, client lock
> + * volume for each lock and compares it with last SLV from the server. CLV is
> + * calculated as the number of locks in LRU * lock live time in seconds. If
> + * CLV > SLV - lock is canceled.
> + *
> + * Client has LVF, that is, lock volume factor which regulates how much
> + * sensitive client should be about last SLV from server. The higher LVF is the
> + * more locks will be canceled on client. Default value for it is 1. Setting LVF
> + * to 2 means that client will cancel locks 2 times faster.
> + *
> + * Locks on a client will be canceled more intensively in these cases:
> + * (1) if SLV is smaller, that is, load is higher on the server;
> + * (2) client has a lot of locks (the more locks are held by client, the bigger
> + *     chances that some of them should be canceled);
> + * (3) client has old locks (taken some time ago);
> + *
> + * Thus, according to flow paradigm that we use for better understanding SLV,
> + * CLV is the volume of particle in flow described by SLV. According to this,
> + * if flow is getting thinner, more and more particles become outside of it and
> + * as particles are locks, they should be canceled.
> + *
> + * General idea of this belongs to Vitaly Fertman (vitaly at clusterfs.com).
> + * Andreas Dilger (adilger at clusterfs.com) proposed few nice ideas like using
> + * LVF and many cleanups. Flow definition to allow more easy understanding of
> + * the logic belongs to Nikita Danilov (nikita at clusterfs.com) as well as many
> + * cleanups and fixes. And design and implementation are done by Yury Umanets
> + * (umka at clusterfs.com).
> + *
> + * Glossary for terms used:
> + *
> + * pl_limit - Number of allowed locks in pool. Applies to server and client
> + * side (tunable);
> + *
> + * pl_granted - Number of granted locks (calculated);
> + * pl_grant_rate - Number of granted locks for last T (calculated);
> + * pl_cancel_rate - Number of canceled locks for last T (calculated);
> + * pl_grant_speed - Grant speed (GR - CR) for last T (calculated);
> + * pl_grant_plan - Planned number of granted locks for next T (calculated);
> + * pl_server_lock_volume - Current server lock volume (calculated);
> + *
> + * As it may be seen from list above, we have few possible tunables which may
> + * affect behavior much. They all may be modified via sysfs. However, they also
> + * give a possibility for constructing few pre-defined behavior policies. If
> + * none of predefines is suitable for a working pattern being used, new one may
> + * be "constructed" via sysfs tunables.
> + */
> +
> +#define DEBUG_SUBSYSTEM S_LDLM
> +
> +#include <lustre_dlm.h>
> +#include <cl_object.h>
> +#include <obd_class.h>
> +#include <obd_support.h>
> +#include "ldlm_internal.h"
> +
> +/*
> + * 50 ldlm locks for 1MB of RAM.
> + */
> +#define LDLM_POOL_HOST_L ((NUM_CACHEPAGES >> (20 - PAGE_SHIFT)) * 50)
> +
> +/*
> + * Maximal possible grant step plan in %.
> + */
> +#define LDLM_POOL_MAX_GSP (30)
> +
> +/*
> + * Minimal possible grant step plan in %.
> + */
> +#define LDLM_POOL_MIN_GSP (1)
> +
> +/*
> + * This controls the speed of reaching LDLM_POOL_MAX_GSP
> + * with increasing thread period.
> + */
> +#define LDLM_POOL_GSP_STEP_SHIFT (2)
> +
> +/*
> + * LDLM_POOL_GSP% of all locks is default GP.
> + */
> +#define LDLM_POOL_GP(L)   (((L) * LDLM_POOL_MAX_GSP) / 100)
> +
> +/*
> + * Max age for locks on clients.
> + */
> +#define LDLM_POOL_MAX_AGE (36000)
> +
> +/*
> + * The granularity of SLV calculation.
> + */
> +#define LDLM_POOL_SLV_SHIFT (10)
> +
> +static inline __u64 dru(__u64 val, __u32 shift, int round_up)
> +{
> +	return (val + (round_up ? (1 << shift) - 1 : 0)) >> shift;
> +}
> +
> +static inline __u64 ldlm_pool_slv_max(__u32 L)
> +{
> +	/*
> +	 * Allow to have all locks for 1 client for 10 hrs.
> +	 * Formula is the following: limit * 10h / 1 client.
> +	 */
> +	__u64 lim = (__u64)L *  LDLM_POOL_MAX_AGE / 1;
> +	return lim;
> +}
> +
> +static inline __u64 ldlm_pool_slv_min(__u32 L)
> +{
> +	return 1;
> +}
> +
> +enum {
> +	LDLM_POOL_FIRST_STAT = 0,
> +	LDLM_POOL_GRANTED_STAT = LDLM_POOL_FIRST_STAT,
> +	LDLM_POOL_GRANT_STAT,
> +	LDLM_POOL_CANCEL_STAT,
> +	LDLM_POOL_GRANT_RATE_STAT,
> +	LDLM_POOL_CANCEL_RATE_STAT,
> +	LDLM_POOL_GRANT_PLAN_STAT,
> +	LDLM_POOL_SLV_STAT,
> +	LDLM_POOL_SHRINK_REQTD_STAT,
> +	LDLM_POOL_SHRINK_FREED_STAT,
> +	LDLM_POOL_RECALC_STAT,
> +	LDLM_POOL_TIMING_STAT,
> +	LDLM_POOL_LAST_STAT
> +};
> +
> +/**
> + * Calculates suggested grant_step in % of available locks for passed
> + * \a period. This is later used in grant_plan calculations.
> + */
> +static inline int ldlm_pool_t2gsp(unsigned int t)
> +{
> +	/*
> +	 * This yields 1% grant step for anything below LDLM_POOL_GSP_STEP
> +	 * and up to 30% for anything higher than LDLM_POOL_GSP_STEP.
> +	 *
> +	 * How this will affect execution is the following:
> +	 *
> +	 * - for thread period 1s we will have grant_step 1% which good from
> +	 * pov of taking some load off from server and push it out to clients.
> +	 * This is like that because 1% for grant_step means that server will
> +	 * not allow clients to get lots of locks in short period of time and
> +	 * keep all old locks in their caches. Clients will always have to
> +	 * get some locks back if they want to take some new;
> +	 *
> +	 * - for thread period 10s (which is default) we will have 23% which
> +	 * means that clients will have enough of room to take some new locks
> +	 * without getting some back. All locks from this 23% which were not
> +	 * taken by clients in current period will contribute in SLV growing.
> +	 * SLV growing means more locks cached on clients until limit or grant
> +	 * plan is reached.
> +	 */
> +	return LDLM_POOL_MAX_GSP -
> +		((LDLM_POOL_MAX_GSP - LDLM_POOL_MIN_GSP) >>
> +		 (t >> LDLM_POOL_GSP_STEP_SHIFT));
> +}
> +
> +/**
> + * Recalculates next stats on passed \a pl.
> + *
> + * \pre ->pl_lock is locked.
> + */
> +static void ldlm_pool_recalc_stats(struct ldlm_pool *pl)
> +{
> +	int grant_plan = pl->pl_grant_plan;
> +	__u64 slv = pl->pl_server_lock_volume;
> +	int granted = atomic_read(&pl->pl_granted);
> +	int grant_rate = atomic_read(&pl->pl_grant_rate);
> +	int cancel_rate = atomic_read(&pl->pl_cancel_rate);
> +
> +	lprocfs_counter_add(pl->pl_stats, LDLM_POOL_SLV_STAT,
> +			    slv);
> +	lprocfs_counter_add(pl->pl_stats, LDLM_POOL_GRANTED_STAT,
> +			    granted);
> +	lprocfs_counter_add(pl->pl_stats, LDLM_POOL_GRANT_RATE_STAT,
> +			    grant_rate);
> +	lprocfs_counter_add(pl->pl_stats, LDLM_POOL_GRANT_PLAN_STAT,
> +			    grant_plan);
> +	lprocfs_counter_add(pl->pl_stats, LDLM_POOL_CANCEL_RATE_STAT,
> +			    cancel_rate);
> +}
> +
> +/**
> + * Sets SLV and Limit from container_of(pl, struct ldlm_namespace,
> + * ns_pool)->ns_obd tp passed \a pl.
> + */
> +static void ldlm_cli_pool_pop_slv(struct ldlm_pool *pl)
> +{
> +	struct obd_device *obd;
> +
> +	/*
> +	 * Get new SLV and Limit from obd which is updated with coming
> +	 * RPCs.
> +	 */
> +	obd = container_of(pl, struct ldlm_namespace,
> +			   ns_pool)->ns_obd;
> +	read_lock(&obd->obd_pool_lock);
> +	pl->pl_server_lock_volume = obd->obd_pool_slv;
> +	atomic_set(&pl->pl_limit, obd->obd_pool_limit);
> +	read_unlock(&obd->obd_pool_lock);
> +}
> +
> +/**
> + * Recalculates client size pool \a pl according to current SLV and Limit.
> + */
> +static int ldlm_cli_pool_recalc(struct ldlm_pool *pl)
> +{
> +	time64_t recalc_interval_sec;
> +	int ret;
> +
> +	recalc_interval_sec = ktime_get_real_seconds() - pl->pl_recalc_time;
> +	if (recalc_interval_sec < pl->pl_recalc_period)
> +		return 0;
> +
> +	spin_lock(&pl->pl_lock);
> +	/*
> +	 * Check if we need to recalc lists now.
> +	 */
> +	recalc_interval_sec = ktime_get_real_seconds() - pl->pl_recalc_time;
> +	if (recalc_interval_sec < pl->pl_recalc_period) {
> +		spin_unlock(&pl->pl_lock);
> +		return 0;
> +	}
> +
> +	/*
> +	 * Make sure that pool knows last SLV and Limit from obd.
> +	 */
> +	ldlm_cli_pool_pop_slv(pl);
> +
> +	spin_unlock(&pl->pl_lock);
> +
> +	/*
> +	 * Do not cancel locks in case lru resize is disabled for this ns.
> +	 */
> +	if (!ns_connect_lru_resize(container_of(pl, struct ldlm_namespace,
> +						ns_pool))) {
> +		ret = 0;
> +		goto out;
> +	}
> +
> +	/*
> +	 * In the time of canceling locks on client we do not need to maintain
> +	 * sharp timing, we only want to cancel locks asap according to new SLV.
> +	 * It may be called when SLV has changed much, this is why we do not
> +	 * take into account pl->pl_recalc_time here.
> +	 */
> +	ret = ldlm_cancel_lru(container_of(pl, struct ldlm_namespace, ns_pool),
> +			      0, LCF_ASYNC, LDLM_LRU_FLAG_LRUR);
> +
> +out:
> +	spin_lock(&pl->pl_lock);
> +	/*
> +	 * Time of LRU resizing might be longer than period,
> +	 * so update after LRU resizing rather than before it.
> +	 */
> +	pl->pl_recalc_time = ktime_get_real_seconds();
> +	lprocfs_counter_add(pl->pl_stats, LDLM_POOL_TIMING_STAT,
> +			    recalc_interval_sec);
> +	spin_unlock(&pl->pl_lock);
> +	return ret;
> +}
> +
> +/**
> + * This function is main entry point for memory pressure handling on client
> + * side.  Main goal of this function is to cancel some number of locks on
> + * passed \a pl according to \a nr and \a gfp_mask.
> + */
> +static int ldlm_cli_pool_shrink(struct ldlm_pool *pl,
> +				int nr, gfp_t gfp_mask)
> +{
> +	struct ldlm_namespace *ns;
> +	int unused;
> +
> +	ns = container_of(pl, struct ldlm_namespace, ns_pool);
> +
> +	/*
> +	 * Do not cancel locks in case lru resize is disabled for this ns.
> +	 */
> +	if (!ns_connect_lru_resize(ns))
> +		return 0;
> +
> +	/*
> +	 * Make sure that pool knows last SLV and Limit from obd.
> +	 */
> +	ldlm_cli_pool_pop_slv(pl);
> +
> +	spin_lock(&ns->ns_lock);
> +	unused = ns->ns_nr_unused;
> +	spin_unlock(&ns->ns_lock);
> +
> +	if (nr == 0)
> +		return (unused / 100) * sysctl_vfs_cache_pressure;
> +	else
> +		return ldlm_cancel_lru(ns, nr, LCF_ASYNC, LDLM_LRU_FLAG_SHRINK);
> +}
> +
> +static const struct ldlm_pool_ops ldlm_cli_pool_ops = {
> +	.po_recalc = ldlm_cli_pool_recalc,
> +	.po_shrink = ldlm_cli_pool_shrink
> +};
> +
> +/**
> + * Pool recalc wrapper. Will call either client or server pool recalc callback
> + * depending what pool \a pl is used.
> + */
> +static int ldlm_pool_recalc(struct ldlm_pool *pl)
> +{
> +	u32 recalc_interval_sec;
> +	int count;
> +
> +	recalc_interval_sec = ktime_get_real_seconds() - pl->pl_recalc_time;
> +	if (recalc_interval_sec > 0) {
> +		spin_lock(&pl->pl_lock);
> +		recalc_interval_sec = ktime_get_real_seconds() - pl->pl_recalc_time;
> +
> +		if (recalc_interval_sec > 0) {
> +			/*
> +			 * Update pool statistics every 1s.
> +			 */
> +			ldlm_pool_recalc_stats(pl);
> +
> +			/*
> +			 * Zero out all rates and speed for the last period.
> +			 */
> +			atomic_set(&pl->pl_grant_rate, 0);
> +			atomic_set(&pl->pl_cancel_rate, 0);
> +		}
> +		spin_unlock(&pl->pl_lock);
> +	}
> +
> +	if (pl->pl_ops->po_recalc) {
> +		count = pl->pl_ops->po_recalc(pl);
> +		lprocfs_counter_add(pl->pl_stats, LDLM_POOL_RECALC_STAT,
> +				    count);
> +	}
> +
> +	recalc_interval_sec = pl->pl_recalc_time - ktime_get_real_seconds() +
> +			      pl->pl_recalc_period;
> +	if (recalc_interval_sec <= 0) {
> +		/* DEBUG: should be re-removed after LU-4536 is fixed */
> +		CDEBUG(D_DLMTRACE,
> +		       "%s: Negative interval(%ld), too short period(%ld)\n",
> +		       pl->pl_name, (long)recalc_interval_sec,
> +		       (long)pl->pl_recalc_period);
> +
> +		/* Prevent too frequent recalculation. */
> +		recalc_interval_sec = 1;
> +	}
> +
> +	return recalc_interval_sec;
> +}
> +
> +/*
> + * Pool shrink wrapper. Will call either client or server pool recalc callback
> + * depending what pool pl is used. When nr == 0, just return the number of
> + * freeable locks. Otherwise, return the number of canceled locks.
> + */
> +static int ldlm_pool_shrink(struct ldlm_pool *pl, int nr, gfp_t gfp_mask)
> +{
> +	int cancel = 0;
> +
> +	if (pl->pl_ops->po_shrink) {
> +		cancel = pl->pl_ops->po_shrink(pl, nr, gfp_mask);
> +		if (nr > 0) {
> +			lprocfs_counter_add(pl->pl_stats,
> +					    LDLM_POOL_SHRINK_REQTD_STAT,
> +					    nr);
> +			lprocfs_counter_add(pl->pl_stats,
> +					    LDLM_POOL_SHRINK_FREED_STAT,
> +					    cancel);
> +			CDEBUG(D_DLMTRACE,
> +			       "%s: request to shrink %d locks, shrunk %d\n",
> +			       pl->pl_name, nr, cancel);
> +		}
> +	}
> +	return cancel;
> +}
> +
> +static int lprocfs_pool_state_seq_show(struct seq_file *m, void *unused)
> +{
> +	int granted, grant_rate, cancel_rate;
> +	int grant_speed, lvf;
> +	struct ldlm_pool *pl = m->private;
> +	__u64 slv, clv;
> +	__u32 limit;
> +
> +	spin_lock(&pl->pl_lock);
> +	slv = pl->pl_server_lock_volume;
> +	clv = pl->pl_client_lock_volume;
> +	limit = atomic_read(&pl->pl_limit);
> +	granted = atomic_read(&pl->pl_granted);
> +	grant_rate = atomic_read(&pl->pl_grant_rate);
> +	cancel_rate = atomic_read(&pl->pl_cancel_rate);
> +	grant_speed = grant_rate - cancel_rate;
> +	lvf = atomic_read(&pl->pl_lock_volume_factor);
> +	spin_unlock(&pl->pl_lock);
> +
> +	seq_printf(m, "LDLM pool state (%s):\n"
> +		      "  SLV: %llu\n"
> +		      "  CLV: %llu\n"
> +		      "  LVF: %d\n",
> +		      pl->pl_name, slv, clv, lvf);
> +
> +	seq_printf(m, "  GR:  %d\n  CR:  %d\n  GS:  %d\n"
> +		      "  G:   %d\n  L:   %d\n",
> +		      grant_rate, cancel_rate, grant_speed,
> +		      granted, limit);
> +
> +	return 0;
> +}
> +
> +LPROC_SEQ_FOPS_RO(lprocfs_pool_state);
> +
> +static ssize_t grant_speed_show(struct kobject *kobj, struct attribute *attr,
> +				char *buf)
> +{
> +	struct ldlm_pool *pl = container_of(kobj, struct ldlm_pool,
> +					    pl_kobj);
> +
> +	int	       grant_speed;
> +
> +	spin_lock(&pl->pl_lock);
> +	/* serialize with ldlm_pool_recalc */
> +	grant_speed = atomic_read(&pl->pl_grant_rate) -
> +			atomic_read(&pl->pl_cancel_rate);
> +	spin_unlock(&pl->pl_lock);
> +	return sprintf(buf, "%d\n", grant_speed);
> +}
> +LUSTRE_RO_ATTR(grant_speed);
> +
> +LDLM_POOL_SYSFS_READER_SHOW(grant_plan, int);
> +LUSTRE_RO_ATTR(grant_plan);
> +
> +LDLM_POOL_SYSFS_READER_SHOW(recalc_period, int);
> +LDLM_POOL_SYSFS_WRITER_STORE(recalc_period, int);
> +LUSTRE_RW_ATTR(recalc_period);
> +
> +LDLM_POOL_SYSFS_READER_NOLOCK_SHOW(server_lock_volume, u64);
> +LUSTRE_RO_ATTR(server_lock_volume);
> +
> +LDLM_POOL_SYSFS_READER_NOLOCK_SHOW(limit, atomic);
> +LDLM_POOL_SYSFS_WRITER_NOLOCK_STORE(limit, atomic);
> +LUSTRE_RW_ATTR(limit);
> +
> +LDLM_POOL_SYSFS_READER_NOLOCK_SHOW(granted, atomic);
> +LUSTRE_RO_ATTR(granted);
> +
> +LDLM_POOL_SYSFS_READER_NOLOCK_SHOW(cancel_rate, atomic);
> +LUSTRE_RO_ATTR(cancel_rate);
> +
> +LDLM_POOL_SYSFS_READER_NOLOCK_SHOW(grant_rate, atomic);
> +LUSTRE_RO_ATTR(grant_rate);
> +
> +LDLM_POOL_SYSFS_READER_NOLOCK_SHOW(lock_volume_factor, atomic);
> +LDLM_POOL_SYSFS_WRITER_NOLOCK_STORE(lock_volume_factor, atomic);
> +LUSTRE_RW_ATTR(lock_volume_factor);
> +
> +#define LDLM_POOL_ADD_VAR(name, var, ops)			\
> +	do {							\
> +		snprintf(var_name, MAX_STRING_SIZE, #name);	\
> +		pool_vars[0].data = var;			\
> +		pool_vars[0].fops = ops;			\
> +		ldebugfs_add_vars(pl->pl_debugfs_entry, pool_vars, NULL);\
> +	} while (0)
> +
> +/* These are for pools in /sys/fs/lustre/ldlm/namespaces/.../pool */
> +static struct attribute *ldlm_pl_attrs[] = {
> +	&lustre_attr_grant_speed.attr,
> +	&lustre_attr_grant_plan.attr,
> +	&lustre_attr_recalc_period.attr,
> +	&lustre_attr_server_lock_volume.attr,
> +	&lustre_attr_limit.attr,
> +	&lustre_attr_granted.attr,
> +	&lustre_attr_cancel_rate.attr,
> +	&lustre_attr_grant_rate.attr,
> +	&lustre_attr_lock_volume_factor.attr,
> +	NULL,
> +};
> +
> +static void ldlm_pl_release(struct kobject *kobj)
> +{
> +	struct ldlm_pool *pl = container_of(kobj, struct ldlm_pool,
> +					    pl_kobj);
> +	complete(&pl->pl_kobj_unregister);
> +}
> +
> +static struct kobj_type ldlm_pl_ktype = {
> +	.default_attrs	= ldlm_pl_attrs,
> +	.sysfs_ops	= &lustre_sysfs_ops,
> +	.release	= ldlm_pl_release,
> +};
> +
> +static int ldlm_pool_sysfs_init(struct ldlm_pool *pl)
> +{
> +	struct ldlm_namespace *ns = container_of(pl, struct ldlm_namespace,
> +						 ns_pool);
> +	int err;
> +
> +	init_completion(&pl->pl_kobj_unregister);
> +	err = kobject_init_and_add(&pl->pl_kobj, &ldlm_pl_ktype, &ns->ns_kobj,
> +				   "pool");
> +
> +	return err;
> +}
> +
> +static int ldlm_pool_debugfs_init(struct ldlm_pool *pl)
> +{
> +	struct ldlm_namespace *ns = container_of(pl, struct ldlm_namespace,
> +						 ns_pool);
> +	struct dentry *debugfs_ns_parent;
> +	struct lprocfs_vars pool_vars[2];
> +	char *var_name = NULL;
> +	int rc = 0;
> +
> +	var_name = kzalloc(MAX_STRING_SIZE + 1, GFP_NOFS);
> +	if (!var_name)
> +		return -ENOMEM;
> +
> +	debugfs_ns_parent = ns->ns_debugfs_entry;
> +	if (IS_ERR_OR_NULL(debugfs_ns_parent)) {
> +		CERROR("%s: debugfs entry is not initialized\n",
> +		       ldlm_ns_name(ns));
> +		rc = -EINVAL;
> +		goto out_free_name;
> +	}
> +	pl->pl_debugfs_entry = debugfs_create_dir("pool", debugfs_ns_parent);
> +
> +	var_name[MAX_STRING_SIZE] = '\0';
> +	memset(pool_vars, 0, sizeof(pool_vars));
> +	pool_vars[0].name = var_name;
> +
> +	LDLM_POOL_ADD_VAR(state, pl, &lprocfs_pool_state_fops);
> +
> +	pl->pl_stats = lprocfs_alloc_stats(LDLM_POOL_LAST_STAT -
> +					   LDLM_POOL_FIRST_STAT, 0);
> +	if (!pl->pl_stats) {
> +		rc = -ENOMEM;
> +		goto out_free_name;
> +	}
> +
> +	lprocfs_counter_init(pl->pl_stats, LDLM_POOL_GRANTED_STAT,
> +			     LPROCFS_CNTR_AVGMINMAX | LPROCFS_CNTR_STDDEV,
> +			     "granted", "locks");
> +	lprocfs_counter_init(pl->pl_stats, LDLM_POOL_GRANT_STAT,
> +			     LPROCFS_CNTR_AVGMINMAX | LPROCFS_CNTR_STDDEV,
> +			     "grant", "locks");
> +	lprocfs_counter_init(pl->pl_stats, LDLM_POOL_CANCEL_STAT,
> +			     LPROCFS_CNTR_AVGMINMAX | LPROCFS_CNTR_STDDEV,
> +			     "cancel", "locks");
> +	lprocfs_counter_init(pl->pl_stats, LDLM_POOL_GRANT_RATE_STAT,
> +			     LPROCFS_CNTR_AVGMINMAX | LPROCFS_CNTR_STDDEV,
> +			     "grant_rate", "locks/s");
> +	lprocfs_counter_init(pl->pl_stats, LDLM_POOL_CANCEL_RATE_STAT,
> +			     LPROCFS_CNTR_AVGMINMAX | LPROCFS_CNTR_STDDEV,
> +			     "cancel_rate", "locks/s");
> +	lprocfs_counter_init(pl->pl_stats, LDLM_POOL_GRANT_PLAN_STAT,
> +			     LPROCFS_CNTR_AVGMINMAX | LPROCFS_CNTR_STDDEV,
> +			     "grant_plan", "locks/s");
> +	lprocfs_counter_init(pl->pl_stats, LDLM_POOL_SLV_STAT,
> +			     LPROCFS_CNTR_AVGMINMAX | LPROCFS_CNTR_STDDEV,
> +			     "slv", "slv");
> +	lprocfs_counter_init(pl->pl_stats, LDLM_POOL_SHRINK_REQTD_STAT,
> +			     LPROCFS_CNTR_AVGMINMAX | LPROCFS_CNTR_STDDEV,
> +			     "shrink_request", "locks");
> +	lprocfs_counter_init(pl->pl_stats, LDLM_POOL_SHRINK_FREED_STAT,
> +			     LPROCFS_CNTR_AVGMINMAX | LPROCFS_CNTR_STDDEV,
> +			     "shrink_freed", "locks");
> +	lprocfs_counter_init(pl->pl_stats, LDLM_POOL_RECALC_STAT,
> +			     LPROCFS_CNTR_AVGMINMAX | LPROCFS_CNTR_STDDEV,
> +			     "recalc_freed", "locks");
> +	lprocfs_counter_init(pl->pl_stats, LDLM_POOL_TIMING_STAT,
> +			     LPROCFS_CNTR_AVGMINMAX | LPROCFS_CNTR_STDDEV,
> +			     "recalc_timing", "sec");
> +	debugfs_create_file("stats", 0644, pl->pl_debugfs_entry, pl->pl_stats,
> +			    &lprocfs_stats_seq_fops);
> +
> +out_free_name:
> +	kfree(var_name);
> +	return rc;
> +}
> +
> +static void ldlm_pool_sysfs_fini(struct ldlm_pool *pl)
> +{
> +	kobject_put(&pl->pl_kobj);
> +	wait_for_completion(&pl->pl_kobj_unregister);
> +}
> +
> +static void ldlm_pool_debugfs_fini(struct ldlm_pool *pl)
> +{
> +	if (pl->pl_stats) {
> +		lprocfs_free_stats(&pl->pl_stats);
> +		pl->pl_stats = NULL;
> +	}
> +	debugfs_remove_recursive(pl->pl_debugfs_entry);
> +}
> +
> +int ldlm_pool_init(struct ldlm_pool *pl, struct ldlm_namespace *ns,
> +		   int idx, enum ldlm_side client)
> +{
> +	int rc;
> +
> +	spin_lock_init(&pl->pl_lock);
> +	atomic_set(&pl->pl_granted, 0);
> +	pl->pl_recalc_time = ktime_get_real_seconds();
> +	atomic_set(&pl->pl_lock_volume_factor, 1);
> +
> +	atomic_set(&pl->pl_grant_rate, 0);
> +	atomic_set(&pl->pl_cancel_rate, 0);
> +	pl->pl_grant_plan = LDLM_POOL_GP(LDLM_POOL_HOST_L);
> +
> +	snprintf(pl->pl_name, sizeof(pl->pl_name), "ldlm-pool-%s-%d",
> +		 ldlm_ns_name(ns), idx);
> +
> +	atomic_set(&pl->pl_limit, 1);
> +	pl->pl_server_lock_volume = 0;
> +	pl->pl_ops = &ldlm_cli_pool_ops;
> +	pl->pl_recalc_period = LDLM_POOL_CLI_DEF_RECALC_PERIOD;
> +	pl->pl_client_lock_volume = 0;
> +	rc = ldlm_pool_debugfs_init(pl);
> +	if (rc)
> +		return rc;
> +
> +	rc = ldlm_pool_sysfs_init(pl);
> +	if (rc)
> +		return rc;
> +
> +	CDEBUG(D_DLMTRACE, "Lock pool %s is initialized\n", pl->pl_name);
> +
> +	return rc;
> +}
> +
> +void ldlm_pool_fini(struct ldlm_pool *pl)
> +{
> +	ldlm_pool_sysfs_fini(pl);
> +	ldlm_pool_debugfs_fini(pl);
> +
> +	/*
> +	 * Pool should not be used after this point. We can't free it here as
> +	 * it lives in struct ldlm_namespace, but still interested in catching
> +	 * any abnormal using cases.
> +	 */
> +	POISON(pl, 0x5a, sizeof(*pl));
> +}
> +
> +/**
> + * Add new taken ldlm lock \a lock into pool \a pl accounting.
> + */
> +void ldlm_pool_add(struct ldlm_pool *pl, struct ldlm_lock *lock)
> +{
> +	/*
> +	 * FLOCK locks are special in a sense that they are almost never
> +	 * cancelled, instead special kind of lock is used to drop them.
> +	 * also there is no LRU for flock locks, so no point in tracking
> +	 * them anyway.
> +	 */
> +	if (lock->l_resource->lr_type == LDLM_FLOCK)
> +		return;
> +
> +	atomic_inc(&pl->pl_granted);
> +	atomic_inc(&pl->pl_grant_rate);
> +	lprocfs_counter_incr(pl->pl_stats, LDLM_POOL_GRANT_STAT);
> +	/*
> +	 * Do not do pool recalc for client side as all locks which
> +	 * potentially may be canceled has already been packed into
> +	 * enqueue/cancel rpc. Also we do not want to run out of stack
> +	 * with too long call paths.
> +	 */
> +}
> +
> +/**
> + * Remove ldlm lock \a lock from pool \a pl accounting.
> + */
> +void ldlm_pool_del(struct ldlm_pool *pl, struct ldlm_lock *lock)
> +{
> +	/*
> +	 * Filter out FLOCK locks. Read above comment in ldlm_pool_add().
> +	 */
> +	if (lock->l_resource->lr_type == LDLM_FLOCK)
> +		return;
> +
> +	LASSERT(atomic_read(&pl->pl_granted) > 0);
> +	atomic_dec(&pl->pl_granted);
> +	atomic_inc(&pl->pl_cancel_rate);
> +
> +	lprocfs_counter_incr(pl->pl_stats, LDLM_POOL_CANCEL_STAT);
> +}
> +
> +/**
> + * Returns current \a pl SLV.
> + *
> + * \pre ->pl_lock is not locked.
> + */
> +__u64 ldlm_pool_get_slv(struct ldlm_pool *pl)
> +{
> +	__u64 slv;
> +
> +	spin_lock(&pl->pl_lock);
> +	slv = pl->pl_server_lock_volume;
> +	spin_unlock(&pl->pl_lock);
> +	return slv;
> +}
> +
> +/**
> + * Sets passed \a clv to \a pl.
> + *
> + * \pre ->pl_lock is not locked.
> + */
> +void ldlm_pool_set_clv(struct ldlm_pool *pl, __u64 clv)
> +{
> +	spin_lock(&pl->pl_lock);
> +	pl->pl_client_lock_volume = clv;
> +	spin_unlock(&pl->pl_lock);
> +}
> +
> +/**
> + * Returns current LVF from \a pl.
> + */
> +__u32 ldlm_pool_get_lvf(struct ldlm_pool *pl)
> +{
> +	return atomic_read(&pl->pl_lock_volume_factor);
> +}
> +
> +static int ldlm_pool_granted(struct ldlm_pool *pl)
> +{
> +	return atomic_read(&pl->pl_granted);
> +}
> +
> +/*
> + * count locks from all namespaces (if possible). Returns number of
> + * cached locks.
> + */
> +static unsigned long ldlm_pools_count(enum ldlm_side client, gfp_t gfp_mask)
> +{
> +	unsigned long total = 0;
> +	int nr_ns;
> +	struct ldlm_namespace *ns;
> +	struct ldlm_namespace *ns_old = NULL; /* loop detection */
> +
> +	if (client == LDLM_NAMESPACE_CLIENT && !(gfp_mask & __GFP_FS))
> +		return 0;
> +
> +	CDEBUG(D_DLMTRACE, "Request to count %s locks from all pools\n",
> +	       client == LDLM_NAMESPACE_CLIENT ? "client" : "server");
> +
> +	/*
> +	 * Find out how many resources we may release.
> +	 */
> +	for (nr_ns = ldlm_namespace_nr_read(client);
> +	     nr_ns > 0; nr_ns--) {
> +		mutex_lock(ldlm_namespace_lock(client));
> +		if (list_empty(ldlm_namespace_list(client))) {
> +			mutex_unlock(ldlm_namespace_lock(client));
> +			return 0;
> +		}
> +		ns = ldlm_namespace_first_locked(client);
> +
> +		if (ns == ns_old) {
> +			mutex_unlock(ldlm_namespace_lock(client));
> +			break;
> +		}
> +
> +		if (ldlm_ns_empty(ns)) {
> +			ldlm_namespace_move_to_inactive_locked(ns, client);
> +			mutex_unlock(ldlm_namespace_lock(client));
> +			continue;
> +		}
> +
> +		if (!ns_old)
> +			ns_old = ns;
> +
> +		ldlm_namespace_get(ns);
> +		ldlm_namespace_move_to_active_locked(ns, client);
> +		mutex_unlock(ldlm_namespace_lock(client));
> +		total += ldlm_pool_shrink(&ns->ns_pool, 0, gfp_mask);
> +		ldlm_namespace_put(ns);
> +	}
> +
> +	return total;
> +}
> +
> +static unsigned long ldlm_pools_scan(enum ldlm_side client, int nr,
> +				     gfp_t gfp_mask)
> +{
> +	unsigned long freed = 0;
> +	int tmp, nr_ns;
> +	struct ldlm_namespace *ns;
> +
> +	if (client == LDLM_NAMESPACE_CLIENT && !(gfp_mask & __GFP_FS))
> +		return -1;
> +
> +	/*
> +	 * Shrink at least ldlm_namespace_nr_read(client) namespaces.
> +	 */
> +	for (tmp = nr_ns = ldlm_namespace_nr_read(client);
> +	     tmp > 0; tmp--) {
> +		int cancel, nr_locks;
> +
> +		/*
> +		 * Do not call shrink under ldlm_namespace_lock(client)
> +		 */
> +		mutex_lock(ldlm_namespace_lock(client));
> +		if (list_empty(ldlm_namespace_list(client))) {
> +			mutex_unlock(ldlm_namespace_lock(client));
> +			break;
> +		}
> +		ns = ldlm_namespace_first_locked(client);
> +		ldlm_namespace_get(ns);
> +		ldlm_namespace_move_to_active_locked(ns, client);
> +		mutex_unlock(ldlm_namespace_lock(client));
> +
> +		nr_locks = ldlm_pool_granted(&ns->ns_pool);
> +		/*
> +		 * We use to shrink propotionally but with new shrinker API,
> +		 * we lost the total number of freeable locks.
> +		 */
> +		cancel = 1 + min_t(int, nr_locks, nr / nr_ns);
> +		freed += ldlm_pool_shrink(&ns->ns_pool, cancel, gfp_mask);
> +		ldlm_namespace_put(ns);
> +	}
> +	/*
> +	 * we only decrease the SLV in server pools shrinker, return
> +	 * SHRINK_STOP to kernel to avoid needless loop. LU-1128
> +	 */
> +	return freed;
> +}
> +
> +static unsigned long ldlm_pools_cli_count(struct shrinker *s,
> +					  struct shrink_control *sc)
> +{
> +	return ldlm_pools_count(LDLM_NAMESPACE_CLIENT, sc->gfp_mask);
> +}
> +
> +static unsigned long ldlm_pools_cli_scan(struct shrinker *s,
> +					 struct shrink_control *sc)
> +{
> +	return ldlm_pools_scan(LDLM_NAMESPACE_CLIENT, sc->nr_to_scan,
> +			       sc->gfp_mask);
> +}
> +
> +static void ldlm_pools_recalc(struct work_struct *ws);
> +static DECLARE_DELAYED_WORK(ldlm_recalc_pools, ldlm_pools_recalc);
> +
> +static void ldlm_pools_recalc(struct work_struct *ws)
> +{
> +	enum ldlm_side client = LDLM_NAMESPACE_CLIENT;
> +	struct ldlm_namespace *ns;
> +	struct ldlm_namespace *ns_old = NULL;
> +	/* seconds of sleep if no active namespaces */
> +	int time = LDLM_POOL_CLI_DEF_RECALC_PERIOD;
> +	int nr;
> +
> +	/*
> +	 * Recalc at least ldlm_namespace_nr_read(client) namespaces.
> +	 */
> +	for (nr = ldlm_namespace_nr_read(client); nr > 0; nr--) {
> +		int     skip;
> +		/*
> +		 * Lock the list, get first @ns in the list, getref, move it
> +		 * to the tail, unlock and call pool recalc. This way we avoid
> +		 * calling recalc under @ns lock what is really good as we get
> +		 * rid of potential deadlock on client nodes when canceling
> +		 * locks synchronously.
> +		 */
> +		mutex_lock(ldlm_namespace_lock(client));
> +		if (list_empty(ldlm_namespace_list(client))) {
> +			mutex_unlock(ldlm_namespace_lock(client));
> +			break;
> +		}
> +		ns = ldlm_namespace_first_locked(client);
> +
> +		if (ns_old == ns) { /* Full pass complete */
> +			mutex_unlock(ldlm_namespace_lock(client));
> +			break;
> +		}
> +
> +		/* We got an empty namespace, need to move it back to inactive
> +		 * list.
> +		 * The race with parallel resource creation is fine:
> +		 * - If they do namespace_get before our check, we fail the
> +		 *   check and they move this item to the end of the list anyway
> +		 * - If we do the check and then they do namespace_get, then
> +		 *   we move the namespace to inactive and they will move
> +		 *   it back to active (synchronised by the lock, so no clash
> +		 *   there).
> +		 */
> +		if (ldlm_ns_empty(ns)) {
> +			ldlm_namespace_move_to_inactive_locked(ns, client);
> +			mutex_unlock(ldlm_namespace_lock(client));
> +			continue;
> +		}
> +
> +		if (!ns_old)
> +			ns_old = ns;
> +
> +		spin_lock(&ns->ns_lock);
> +		/*
> +		 * skip ns which is being freed, and we don't want to increase
> +		 * its refcount again, not even temporarily. bz21519 & LU-499.
> +		 */
> +		if (ns->ns_stopping) {
> +			skip = 1;
> +		} else {
> +			skip = 0;
> +			ldlm_namespace_get(ns);
> +		}
> +		spin_unlock(&ns->ns_lock);
> +
> +		ldlm_namespace_move_to_active_locked(ns, client);
> +		mutex_unlock(ldlm_namespace_lock(client));
> +
> +		/*
> +		 * After setup is done - recalc the pool.
> +		 */
> +		if (!skip) {
> +			int ttime = ldlm_pool_recalc(&ns->ns_pool);
> +
> +			if (ttime < time)
> +				time = ttime;
> +
> +			ldlm_namespace_put(ns);
> +		}
> +	}
> +
> +	/* Wake up the blocking threads from time to time. */
> +	ldlm_bl_thread_wakeup();
> +
> +	schedule_delayed_work(&ldlm_recalc_pools, time * HZ);
> +}
> +
> +static int ldlm_pools_thread_start(void)
> +{
> +	schedule_delayed_work(&ldlm_recalc_pools, 0);
> +
> +	return 0;
> +}
> +
> +static void ldlm_pools_thread_stop(void)
> +{
> +	cancel_delayed_work_sync(&ldlm_recalc_pools);
> +}
> +
> +static struct shrinker ldlm_pools_cli_shrinker = {
> +	.count_objects	= ldlm_pools_cli_count,
> +	.scan_objects	= ldlm_pools_cli_scan,
> +	.seeks		= DEFAULT_SEEKS,
> +};
> +
> +int ldlm_pools_init(void)
> +{
> +	int rc;
> +
> +	rc = ldlm_pools_thread_start();
> +	if (!rc)
> +		rc = register_shrinker(&ldlm_pools_cli_shrinker);
> +
> +	return rc;
> +}
> +
> +void ldlm_pools_fini(void)
> +{
> +	unregister_shrinker(&ldlm_pools_cli_shrinker);
> +
> +	ldlm_pools_thread_stop();
> +}
> diff --git a/drivers/staging/lustre/lustre/ptlrpc/ldlm_request.c b/drivers/staging/lustre/lustre/ptlrpc/ldlm_request.c
> new file mode 100644
> index 000000000000..cdc52eed6d85
> --- /dev/null
> +++ b/drivers/staging/lustre/lustre/ptlrpc/ldlm_request.c
> @@ -0,0 +1,2033 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * GPL HEADER START
> + *
> + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 only,
> + * as published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful, but
> + * WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> + * General Public License version 2 for more details (a copy is included
> + * in the LICENSE file that accompanied this code).
> + *
> + * You should have received a copy of the GNU General Public License
> + * version 2 along with this program; If not, see
> + * http://www.gnu.org/licenses/gpl-2.0.html
> + *
> + * GPL HEADER END
> + */
> +/*
> + * Copyright (c) 2002, 2010, Oracle and/or its affiliates. All rights reserved.
> + * Use is subject to license terms.
> + *
> + * Copyright (c) 2010, 2015, Intel Corporation.
> + */
> +/*
> + * This file is part of Lustre, http://www.lustre.org/
> + * Lustre is a trademark of Sun Microsystems, Inc.
> + */
> +/**
> + * This file contains Asynchronous System Trap (AST) handlers and related
> + * LDLM request-processing routines.
> + *
> + * An AST is a callback issued on a lock when its state is changed. There are
> + * several different types of ASTs (callbacks) registered for each lock:
> + *
> + * - completion AST: when a lock is enqueued by some process, but cannot be
> + *   granted immediately due to other conflicting locks on the same resource,
> + *   the completion AST is sent to notify the caller when the lock is
> + *   eventually granted
> + *
> + * - blocking AST: when a lock is granted to some process, if another process
> + *   enqueues a conflicting (blocking) lock on a resource, a blocking AST is
> + *   sent to notify the holder(s) of the lock(s) of the conflicting lock
> + *   request. The lock holder(s) must release their lock(s) on that resource in
> + *   a timely manner or be evicted by the server.
> + *
> + * - glimpse AST: this is used when a process wants information about a lock
> + *   (i.e. the lock value block (LVB)) but does not necessarily require holding
> + *   the lock. If the resource is locked, the lock holder(s) are sent glimpse
> + *   ASTs and the LVB is returned to the caller, and lock holder(s) may CANCEL
> + *   their lock(s) if they are idle. If the resource is not locked, the server
> + *   may grant the lock.
> + */
> +
> +#define DEBUG_SUBSYSTEM S_LDLM
> +
> +#include <lustre_errno.h>
> +#include <lustre_dlm.h>
> +#include <obd_class.h>
> +#include <obd.h>
> +#include <linux/libcfs/libcfs_hash.h>
> +
> +#include "ldlm_internal.h"
> +
> +unsigned int ldlm_enqueue_min = OBD_TIMEOUT_DEFAULT;
> +module_param(ldlm_enqueue_min, uint, 0644);
> +MODULE_PARM_DESC(ldlm_enqueue_min, "lock enqueue timeout minimum");
> +
> +/* in client side, whether the cached locks will be canceled before replay */
> +unsigned int ldlm_cancel_unused_locks_before_replay = 1;
> +
> +struct ldlm_async_args {
> +	struct lustre_handle lock_handle;
> +};
> +
> +/**
> + * ldlm_request_bufsize
> + *
> + * @count:	number of ldlm handles
> + * @type:	ldlm opcode
> + *
> + * If opcode=LDLM_ENQUEUE, 1 slot is already occupied,
> + * LDLM_LOCKREQ_HANDLE -1 slots are available.
> + * Otherwise, LDLM_LOCKREQ_HANDLE slots are available.
> + *
> + * Return:	size of the request buffer
> + */
> +static int ldlm_request_bufsize(int count, int type)
> +{
> +	int avail = LDLM_LOCKREQ_HANDLES;
> +
> +	if (type == LDLM_ENQUEUE)
> +		avail -= LDLM_ENQUEUE_CANCEL_OFF;
> +
> +	if (count > avail)
> +		avail = (count - avail) * sizeof(struct lustre_handle);
> +	else
> +		avail = 0;
> +
> +	return sizeof(struct ldlm_request) + avail;
> +}
> +
> +static void ldlm_expired_completion_wait(struct ldlm_lock *lock, __u32 conn_cnt)
> +{
> +	struct obd_import *imp;
> +	struct obd_device *obd;
> +
> +	if (!lock->l_conn_export) {
> +		static unsigned long next_dump, last_dump;
> +
> +		LDLM_ERROR(lock,
> +			   "lock timed out (enqueued at %lld, %llds ago); not entering recovery in server code, just going back to sleep",
> +			   (s64)lock->l_last_activity,
> +			   (s64)(ktime_get_real_seconds() -
> +				 lock->l_last_activity));
> +		if (time_after(jiffies, next_dump)) {
> +			last_dump = next_dump;
> +			next_dump = jiffies + 300 * HZ;
> +			ldlm_namespace_dump(D_DLMTRACE,
> +					    ldlm_lock_to_ns(lock));
> +			if (last_dump == 0)
> +				libcfs_debug_dumplog();
> +		}
> +		return;
> +	}
> +
> +	obd = lock->l_conn_export->exp_obd;
> +	imp = obd->u.cli.cl_import;
> +	ptlrpc_fail_import(imp, conn_cnt);
> +	LDLM_ERROR(lock,
> +		   "lock timed out (enqueued at %lld, %llds ago), entering recovery for %s@%s",
> +		   (s64)lock->l_last_activity,
> +		   (s64)(ktime_get_real_seconds() - lock->l_last_activity),
> +		   obd2cli_tgt(obd), imp->imp_connection->c_remote_uuid.uuid);
> +}
> +
> +/**
> + * Calculate the Completion timeout (covering enqueue, BL AST, data flush,
> + * lock cancel, and their replies). Used for lock completion timeout on the
> + * client side.
> + *
> + * \param[in] lock	lock which is waiting the completion callback
> + *
> + * \retval		timeout in seconds to wait for the server reply
> + */
> +/* We use the same basis for both server side and client side functions
> + * from a single node.
> + */
> +static unsigned int ldlm_cp_timeout(struct ldlm_lock *lock)
> +{
> +	unsigned int timeout;
> +
> +	if (AT_OFF)
> +		return obd_timeout;
> +
> +	/*
> +	 * Wait a long time for enqueue - server may have to callback a
> +	 * lock from another client.  Server will evict the other client if it
> +	 * doesn't respond reasonably, and then give us the lock.
> +	 */
> +	timeout = at_get(ldlm_lock_to_ns_at(lock));
> +	return max(3 * timeout, ldlm_enqueue_min);
> +}
> +
> +/**
> + * Helper function for ldlm_completion_ast(), updating timings when lock is
> + * actually granted.
> + */
> +static int ldlm_completion_tail(struct ldlm_lock *lock, void *data)
> +{
> +	long delay;
> +	int result = 0;
> +
> +	if (ldlm_is_destroyed(lock) || ldlm_is_failed(lock)) {
> +		LDLM_DEBUG(lock, "client-side enqueue: destroyed");
> +		result = -EIO;
> +	} else if (!data) {
> +		LDLM_DEBUG(lock, "client-side enqueue: granted");
> +	} else {
> +		/* Take into AT only CP RPC, not immediately granted locks */
> +		delay = ktime_get_real_seconds() - lock->l_last_activity;
> +		LDLM_DEBUG(lock, "client-side enqueue: granted after %lds",
> +			   delay);
> +
> +		/* Update our time estimate */
> +		at_measured(ldlm_lock_to_ns_at(lock), delay);
> +	}
> +	return result;
> +}
> +
> +/**
> + * Generic LDLM "completion" AST. This is called in several cases:
> + *
> + *     - when a reply to an ENQUEUE RPC is received from the server
> + *       (ldlm_cli_enqueue_fini()). Lock might be granted or not granted at
> + *       this point (determined by flags);
> + *
> + *     - when LDLM_CP_CALLBACK RPC comes to client to notify it that lock has
> + *       been granted;
> + *
> + *     - when ldlm_lock_match(LDLM_FL_LVB_READY) is about to wait until lock
> + *       gets correct lvb;
> + *
> + *     - to force all locks when resource is destroyed (cleanup_resource());
> + *
> + *     - during lock conversion (not used currently).
> + *
> + * If lock is not granted in the first case, this function waits until second
> + * or penultimate cases happen in some other thread.
> + *
> + */
> +int ldlm_completion_ast(struct ldlm_lock *lock, __u64 flags, void *data)
> +{
> +	/* XXX ALLOCATE - 160 bytes */
> +	struct obd_device *obd;
> +	struct obd_import *imp = NULL;
> +	__u32 timeout;
> +	__u32 conn_cnt = 0;
> +	int rc = 0;
> +
> +	if (flags == LDLM_FL_WAIT_NOREPROC) {
> +		LDLM_DEBUG(lock, "client-side enqueue waiting on pending lock");
> +		goto noreproc;
> +	}
> +
> +	if (!(flags & LDLM_FL_BLOCKED_MASK)) {
> +		wake_up(&lock->l_waitq);
> +		return 0;
> +	}
> +
> +	LDLM_DEBUG(lock,
> +		   "client-side enqueue returned a blocked lock, sleeping");
> +
> +noreproc:
> +
> +	obd = class_exp2obd(lock->l_conn_export);
> +
> +	/* if this is a local lock, then there is no import */
> +	if (obd)
> +		imp = obd->u.cli.cl_import;
> +
> +	timeout = ldlm_cp_timeout(lock);
> +
> +	lock->l_last_activity = ktime_get_real_seconds();
> +
> +	if (imp) {
> +		spin_lock(&imp->imp_lock);
> +		conn_cnt = imp->imp_conn_cnt;
> +		spin_unlock(&imp->imp_lock);
> +	}
> +	if (OBD_FAIL_CHECK_RESET(OBD_FAIL_LDLM_INTR_CP_AST,
> +				 OBD_FAIL_LDLM_CP_BL_RACE | OBD_FAIL_ONCE)) {
> +		ldlm_set_fail_loc(lock);
> +		rc = -EINTR;
> +	} else {
> +		/* Go to sleep until the lock is granted or canceled. */
> +		if (!ldlm_is_no_timeout(lock)) {
> +			/* Wait uninterruptible for a while first */
> +			rc = wait_event_idle_timeout(lock->l_waitq,
> +						     is_granted_or_cancelled(lock),
> +						     timeout * HZ);
> +			if (rc == 0)
> +				ldlm_expired_completion_wait(lock, conn_cnt);
> +		}
> +		/* Now wait abortable */
> +		if (rc == 0)
> +			rc = l_wait_event_abortable(lock->l_waitq,
> +						    is_granted_or_cancelled(lock));
> +		else
> +			rc = 0;
> +	}
> +
> +	if (rc) {
> +		LDLM_DEBUG(lock, "client-side enqueue waking up: failed (%d)",
> +			   rc);
> +		return rc;
> +	}
> +
> +	return ldlm_completion_tail(lock, data);
> +}
> +EXPORT_SYMBOL(ldlm_completion_ast);
> +
> +static void failed_lock_cleanup(struct ldlm_namespace *ns,
> +				struct ldlm_lock *lock, int mode)
> +{
> +	int need_cancel = 0;
> +
> +	/* Set a flag to prevent us from sending a CANCEL (bug 407) */
> +	lock_res_and_lock(lock);
> +	/* Check that lock is not granted or failed, we might race. */
> +	if ((lock->l_req_mode != lock->l_granted_mode) &&
> +	    !ldlm_is_failed(lock)) {
> +		/* Make sure that this lock will not be found by raced
> +		 * bl_ast and -EINVAL reply is sent to server anyways.
> +		 * bug 17645
> +		 */
> +		lock->l_flags |= LDLM_FL_LOCAL_ONLY | LDLM_FL_FAILED |
> +				 LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING;
> +		need_cancel = 1;
> +	}
> +	unlock_res_and_lock(lock);
> +
> +	if (need_cancel)
> +		LDLM_DEBUG(lock,
> +			   "setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING");
> +	else
> +		LDLM_DEBUG(lock, "lock was granted or failed in race");
> +
> +	/* XXX - HACK because we shouldn't call ldlm_lock_destroy()
> +	 *       from llite/file.c/ll_file_flock().
> +	 */
> +	/* This code makes for the fact that we do not have blocking handler on
> +	 * a client for flock locks. As such this is the place where we must
> +	 * completely kill failed locks. (interrupted and those that
> +	 * were waiting to be granted when server evicted us.
> +	 */
> +	if (lock->l_resource->lr_type == LDLM_FLOCK) {
> +		lock_res_and_lock(lock);
> +		if (!ldlm_is_destroyed(lock)) {
> +			ldlm_resource_unlink_lock(lock);
> +			ldlm_lock_decref_internal_nolock(lock, mode);
> +			ldlm_lock_destroy_nolock(lock);
> +		}
> +		unlock_res_and_lock(lock);
> +	} else {
> +		ldlm_lock_decref_internal(lock, mode);
> +	}
> +}
> +
> +/**
> + * Finishing portion of client lock enqueue code.
> + *
> + * Called after receiving reply from server.
> + */
> +int ldlm_cli_enqueue_fini(struct obd_export *exp, struct ptlrpc_request *req,
> +			  enum ldlm_type type, __u8 with_policy,
> +			  enum ldlm_mode mode,
> +			  __u64 *flags, void *lvb, __u32 lvb_len,
> +			  const struct lustre_handle *lockh, int rc)
> +{
> +	struct ldlm_namespace *ns = exp->exp_obd->obd_namespace;
> +	int is_replay = *flags & LDLM_FL_REPLAY;
> +	struct ldlm_lock *lock;
> +	struct ldlm_reply *reply;
> +	int cleanup_phase = 1;
> +
> +	lock = ldlm_handle2lock(lockh);
> +	/* ldlm_cli_enqueue is holding a reference on this lock. */
> +	if (!lock) {
> +		LASSERT(type == LDLM_FLOCK);
> +		return -ENOLCK;
> +	}
> +
> +	LASSERTF(ergo(lvb_len != 0, lvb_len == lock->l_lvb_len),
> +		 "lvb_len = %d, l_lvb_len = %d\n", lvb_len, lock->l_lvb_len);
> +
> +	if (rc != ELDLM_OK) {
> +		LASSERT(!is_replay);
> +		LDLM_DEBUG(lock, "client-side enqueue END (%s)",
> +			   rc == ELDLM_LOCK_ABORTED ? "ABORTED" : "FAILED");
> +
> +		if (rc != ELDLM_LOCK_ABORTED)
> +			goto cleanup;
> +	}
> +
> +	/* Before we return, swab the reply */
> +	reply = req_capsule_server_get(&req->rq_pill, &RMF_DLM_REP);
> +	if (!reply) {
> +		rc = -EPROTO;
> +		goto cleanup;
> +	}
> +
> +	if (lvb_len > 0) {
> +		int size = 0;
> +
> +		size = req_capsule_get_size(&req->rq_pill, &RMF_DLM_LVB,
> +					    RCL_SERVER);
> +		if (size < 0) {
> +			LDLM_ERROR(lock, "Fail to get lvb_len, rc = %d", size);
> +			rc = size;
> +			goto cleanup;
> +		} else if (unlikely(size > lvb_len)) {
> +			LDLM_ERROR(lock,
> +				   "Replied LVB is larger than expectation, expected = %d, replied = %d",
> +				   lvb_len, size);
> +			rc = -EINVAL;
> +			goto cleanup;
> +		}
> +		lvb_len = size;
> +	}
> +
> +	if (rc == ELDLM_LOCK_ABORTED) {
> +		if (lvb_len > 0 && lvb)
> +			rc = ldlm_fill_lvb(lock, &req->rq_pill, RCL_SERVER,
> +					   lvb, lvb_len);
> +		if (rc == 0)
> +			rc = ELDLM_LOCK_ABORTED;
> +		goto cleanup;
> +	}
> +
> +	/* lock enqueued on the server */
> +	cleanup_phase = 0;
> +
> +	lock_res_and_lock(lock);
> +	lock->l_remote_handle = reply->lock_handle;
> +
> +	*flags = ldlm_flags_from_wire(reply->lock_flags);
> +	lock->l_flags |= ldlm_flags_from_wire(reply->lock_flags &
> +					      LDLM_FL_INHERIT_MASK);
> +	unlock_res_and_lock(lock);
> +
> +	CDEBUG(D_INFO, "local: %p, remote cookie: %#llx, flags: 0x%llx\n",
> +	       lock, reply->lock_handle.cookie, *flags);
> +
> +	/* If enqueue returned a blocked lock but the completion handler has
> +	 * already run, then it fixed up the resource and we don't need to do it
> +	 * again.
> +	 */
> +	if ((*flags) & LDLM_FL_LOCK_CHANGED) {
> +		int newmode = reply->lock_desc.l_req_mode;
> +
> +		LASSERT(!is_replay);
> +		if (newmode && newmode != lock->l_req_mode) {
> +			LDLM_DEBUG(lock, "server returned different mode %s",
> +				   ldlm_lockname[newmode]);
> +			lock->l_req_mode = newmode;
> +		}
> +
> +		if (!ldlm_res_eq(&reply->lock_desc.l_resource.lr_name,
> +				 &lock->l_resource->lr_name)) {
> +			CDEBUG(D_INFO,
> +			       "remote intent success, locking " DLDLMRES " instead of " DLDLMRES "\n",
> +			       PLDLMRES(&reply->lock_desc.l_resource),
> +			       PLDLMRES(lock->l_resource));
> +
> +			rc = ldlm_lock_change_resource(ns, lock,
> +					&reply->lock_desc.l_resource.lr_name);
> +			if (rc || !lock->l_resource) {
> +				rc = -ENOMEM;
> +				goto cleanup;
> +			}
> +			LDLM_DEBUG(lock, "client-side enqueue, new resource");
> +		}
> +		if (with_policy)
> +			if (!(type == LDLM_IBITS &&
> +			      !(exp_connect_flags(exp) & OBD_CONNECT_IBITS)))
> +				/* We assume lock type cannot change on server*/
> +				ldlm_convert_policy_to_local(exp,
> +						lock->l_resource->lr_type,
> +						&reply->lock_desc.l_policy_data,
> +						&lock->l_policy_data);
> +		if (type != LDLM_PLAIN)
> +			LDLM_DEBUG(lock,
> +				   "client-side enqueue, new policy data");
> +	}
> +
> +	if ((*flags) & LDLM_FL_AST_SENT) {
> +		lock_res_and_lock(lock);
> +		lock->l_flags |= LDLM_FL_CBPENDING |  LDLM_FL_BL_AST;
> +		unlock_res_and_lock(lock);
> +		LDLM_DEBUG(lock, "enqueue reply includes blocking AST");
> +	}
> +
> +	/* If the lock has already been granted by a completion AST, don't
> +	 * clobber the LVB with an older one.
> +	 */
> +	if (lvb_len > 0) {
> +		/* We must lock or a racing completion might update lvb without
> +		 * letting us know and we'll clobber the correct value.
> +		 * Cannot unlock after the check either, as that still leaves
> +		 * a tiny window for completion to get in
> +		 */
> +		lock_res_and_lock(lock);
> +		if (lock->l_req_mode != lock->l_granted_mode)
> +			rc = ldlm_fill_lvb(lock, &req->rq_pill, RCL_SERVER,
> +					   lock->l_lvb_data, lvb_len);
> +		unlock_res_and_lock(lock);
> +		if (rc < 0) {
> +			cleanup_phase = 1;
> +			goto cleanup;
> +		}
> +	}
> +
> +	if (!is_replay) {
> +		rc = ldlm_lock_enqueue(ns, &lock, NULL, flags);
> +		if (lock->l_completion_ast) {
> +			int err = lock->l_completion_ast(lock, *flags, NULL);
> +
> +			if (!rc)
> +				rc = err;
> +			if (rc)
> +				cleanup_phase = 1;
> +		}
> +	}
> +
> +	if (lvb_len > 0 && lvb) {
> +		/* Copy the LVB here, and not earlier, because the completion
> +		 * AST (if any) can override what we got in the reply
> +		 */
> +		memcpy(lvb, lock->l_lvb_data, lvb_len);
> +	}
> +
> +	LDLM_DEBUG(lock, "client-side enqueue END");
> +cleanup:
> +	if (cleanup_phase == 1 && rc)
> +		failed_lock_cleanup(ns, lock, mode);
> +	/* Put lock 2 times, the second reference is held by ldlm_cli_enqueue */
> +	LDLM_LOCK_PUT(lock);
> +	LDLM_LOCK_RELEASE(lock);
> +	return rc;
> +}
> +EXPORT_SYMBOL(ldlm_cli_enqueue_fini);
> +
> +/**
> + * Estimate number of lock handles that would fit into request of given
> + * size.  PAGE_SIZE-512 is to allow TCP/IP and LNET headers to fit into
> + * a single page on the send/receive side. XXX: 512 should be changed to
> + * more adequate value.
> + */
> +static inline int ldlm_req_handles_avail(int req_size, int off)
> +{
> +	int avail;
> +
> +	avail = min_t(int, LDLM_MAXREQSIZE, PAGE_SIZE - 512) - req_size;
> +	if (likely(avail >= 0))
> +		avail /= (int)sizeof(struct lustre_handle);
> +	else
> +		avail = 0;
> +	avail += LDLM_LOCKREQ_HANDLES - off;
> +
> +	return avail;
> +}
> +
> +static inline int ldlm_capsule_handles_avail(struct req_capsule *pill,
> +					     enum req_location loc,
> +					     int off)
> +{
> +	u32 size = req_capsule_msg_size(pill, loc);
> +
> +	return ldlm_req_handles_avail(size, off);
> +}
> +
> +static inline int ldlm_format_handles_avail(struct obd_import *imp,
> +					    const struct req_format *fmt,
> +					    enum req_location loc, int off)
> +{
> +	u32 size = req_capsule_fmt_size(imp->imp_msg_magic, fmt, loc);
> +
> +	return ldlm_req_handles_avail(size, off);
> +}
> +
> +/**
> + * Cancel LRU locks and pack them into the enqueue request. Pack there the given
> + * \a count locks in \a cancels.
> + *
> + * This is to be called by functions preparing their own requests that
> + * might contain lists of locks to cancel in addition to actual operation
> + * that needs to be performed.
> + */
> +int ldlm_prep_elc_req(struct obd_export *exp, struct ptlrpc_request *req,
> +		      int version, int opc, int canceloff,
> +		      struct list_head *cancels, int count)
> +{
> +	struct ldlm_namespace   *ns = exp->exp_obd->obd_namespace;
> +	struct req_capsule      *pill = &req->rq_pill;
> +	struct ldlm_request     *dlm = NULL;
> +	int flags, avail, to_free, pack = 0;
> +	LIST_HEAD(head);
> +	int rc;
> +
> +	if (!cancels)
> +		cancels = &head;
> +	if (ns_connect_cancelset(ns)) {
> +		/* Estimate the amount of available space in the request. */
> +		req_capsule_filled_sizes(pill, RCL_CLIENT);
> +		avail = ldlm_capsule_handles_avail(pill, RCL_CLIENT, canceloff);
> +
> +		flags = ns_connect_lru_resize(ns) ?
> +			LDLM_LRU_FLAG_LRUR_NO_WAIT : LDLM_LRU_FLAG_AGED;
> +		to_free = !ns_connect_lru_resize(ns) &&
> +			  opc == LDLM_ENQUEUE ? 1 : 0;
> +
> +		/* Cancel LRU locks here _only_ if the server supports
> +		 * EARLY_CANCEL. Otherwise we have to send extra CANCEL
> +		 * RPC, which will make us slower.
> +		 */
> +		if (avail > count)
> +			count += ldlm_cancel_lru_local(ns, cancels, to_free,
> +						       avail - count, 0, flags);
> +		if (avail > count)
> +			pack = count;
> +		else
> +			pack = avail;
> +		req_capsule_set_size(pill, &RMF_DLM_REQ, RCL_CLIENT,
> +				     ldlm_request_bufsize(pack, opc));
> +	}
> +
> +	rc = ptlrpc_request_pack(req, version, opc);
> +	if (rc) {
> +		ldlm_lock_list_put(cancels, l_bl_ast, count);
> +		return rc;
> +	}
> +
> +	if (ns_connect_cancelset(ns)) {
> +		if (canceloff) {
> +			dlm = req_capsule_client_get(pill, &RMF_DLM_REQ);
> +			LASSERT(dlm);
> +			/* Skip first lock handler in ldlm_request_pack(),
> +			 * this method will increment @lock_count according
> +			 * to the lock handle amount actually written to
> +			 * the buffer.
> +			 */
> +			dlm->lock_count = canceloff;
> +		}
> +		/* Pack into the request @pack lock handles. */
> +		ldlm_cli_cancel_list(cancels, pack, req, 0);
> +		/* Prepare and send separate cancel RPC for others. */
> +		ldlm_cli_cancel_list(cancels, count - pack, NULL, 0);
> +	} else {
> +		ldlm_lock_list_put(cancels, l_bl_ast, count);
> +	}
> +	return 0;
> +}
> +EXPORT_SYMBOL(ldlm_prep_elc_req);
> +
> +int ldlm_prep_enqueue_req(struct obd_export *exp, struct ptlrpc_request *req,
> +			  struct list_head *cancels, int count)
> +{
> +	return ldlm_prep_elc_req(exp, req, LUSTRE_DLM_VERSION, LDLM_ENQUEUE,
> +				 LDLM_ENQUEUE_CANCEL_OFF, cancels, count);
> +}
> +EXPORT_SYMBOL(ldlm_prep_enqueue_req);
> +
> +static struct ptlrpc_request *ldlm_enqueue_pack(struct obd_export *exp,
> +						int lvb_len)
> +{
> +	struct ptlrpc_request *req;
> +	int rc;
> +
> +	req = ptlrpc_request_alloc(class_exp2cliimp(exp), &RQF_LDLM_ENQUEUE);
> +	if (!req)
> +		return ERR_PTR(-ENOMEM);
> +
> +	rc = ldlm_prep_enqueue_req(exp, req, NULL, 0);
> +	if (rc) {
> +		ptlrpc_request_free(req);
> +		return ERR_PTR(rc);
> +	}
> +
> +	req_capsule_set_size(&req->rq_pill, &RMF_DLM_LVB, RCL_SERVER, lvb_len);
> +	ptlrpc_request_set_replen(req);
> +	return req;
> +}
> +
> +/**
> + * Client-side lock enqueue.
> + *
> + * If a request has some specific initialisation it is passed in \a reqp,
> + * otherwise it is created in ldlm_cli_enqueue.
> + *
> + * Supports sync and async requests, pass \a async flag accordingly. If a
> + * request was created in ldlm_cli_enqueue and it is the async request,
> + * pass it to the caller in \a reqp.
> + */
> +int ldlm_cli_enqueue(struct obd_export *exp, struct ptlrpc_request **reqp,
> +		     struct ldlm_enqueue_info *einfo,
> +		     const struct ldlm_res_id *res_id,
> +		     union ldlm_policy_data const *policy, __u64 *flags,
> +		     void *lvb, __u32 lvb_len, enum lvb_type lvb_type,
> +		     struct lustre_handle *lockh, int async)
> +{
> +	struct ldlm_namespace *ns;
> +	struct ldlm_lock      *lock;
> +	struct ldlm_request   *body;
> +	int		    is_replay = *flags & LDLM_FL_REPLAY;
> +	int		    req_passed_in = 1;
> +	int		    rc, err;
> +	struct ptlrpc_request *req;
> +
> +	ns = exp->exp_obd->obd_namespace;
> +
> +	/* If we're replaying this lock, just check some invariants.
> +	 * If we're creating a new lock, get everything all setup nicely.
> +	 */
> +	if (is_replay) {
> +		lock = ldlm_handle2lock_long(lockh, 0);
> +		LASSERT(lock);
> +		LDLM_DEBUG(lock, "client-side enqueue START");
> +		LASSERT(exp == lock->l_conn_export);
> +	} else {
> +		const struct ldlm_callback_suite cbs = {
> +			.lcs_completion = einfo->ei_cb_cp,
> +			.lcs_blocking	= einfo->ei_cb_bl,
> +			.lcs_glimpse	= einfo->ei_cb_gl
> +		};
> +		lock = ldlm_lock_create(ns, res_id, einfo->ei_type,
> +					einfo->ei_mode, &cbs, einfo->ei_cbdata,
> +					lvb_len, lvb_type);
> +		if (IS_ERR(lock))
> +			return PTR_ERR(lock);
> +		/* for the local lock, add the reference */
> +		ldlm_lock_addref_internal(lock, einfo->ei_mode);
> +		ldlm_lock2handle(lock, lockh);
> +		if (policy)
> +			lock->l_policy_data = *policy;
> +
> +		if (einfo->ei_type == LDLM_EXTENT) {
> +			/* extent lock without policy is a bug */
> +			if (!policy)
> +				LBUG();
> +
> +			lock->l_req_extent = policy->l_extent;
> +		}
> +		LDLM_DEBUG(lock, "client-side enqueue START, flags %llx",
> +			   *flags);
> +	}
> +
> +	lock->l_conn_export = exp;
> +	lock->l_export = NULL;
> +	lock->l_blocking_ast = einfo->ei_cb_bl;
> +	lock->l_flags |= (*flags & (LDLM_FL_NO_LRU | LDLM_FL_EXCL));
> +	lock->l_last_activity = ktime_get_real_seconds();
> +
> +	/* lock not sent to server yet */
> +	if (!reqp || !*reqp) {
> +		req = ldlm_enqueue_pack(exp, lvb_len);
> +		if (IS_ERR(req)) {
> +			failed_lock_cleanup(ns, lock, einfo->ei_mode);
> +			LDLM_LOCK_RELEASE(lock);
> +			return PTR_ERR(req);
> +		}
> +
> +		req_passed_in = 0;
> +		if (reqp)
> +			*reqp = req;
> +	} else {
> +		int len;
> +
> +		req = *reqp;
> +		len = req_capsule_get_size(&req->rq_pill, &RMF_DLM_REQ,
> +					   RCL_CLIENT);
> +		LASSERTF(len >= sizeof(*body), "buflen[%d] = %d, not %d\n",
> +			 DLM_LOCKREQ_OFF, len, (int)sizeof(*body));
> +	}
> +
> +	/* Dump lock data into the request buffer */
> +	body = req_capsule_client_get(&req->rq_pill, &RMF_DLM_REQ);
> +	ldlm_lock2desc(lock, &body->lock_desc);
> +	body->lock_flags = ldlm_flags_to_wire(*flags);
> +	body->lock_handle[0] = *lockh;
> +
> +	if (async) {
> +		LASSERT(reqp);
> +		return 0;
> +	}
> +
> +	LDLM_DEBUG(lock, "sending request");
> +
> +	rc = ptlrpc_queue_wait(req);
> +
> +	err = ldlm_cli_enqueue_fini(exp, req, einfo->ei_type, policy ? 1 : 0,
> +				    einfo->ei_mode, flags, lvb, lvb_len,
> +				    lockh, rc);
> +
> +	/* If ldlm_cli_enqueue_fini did not find the lock, we need to free
> +	 * one reference that we took
> +	 */
> +	if (err == -ENOLCK)
> +		LDLM_LOCK_RELEASE(lock);
> +	else
> +		rc = err;
> +
> +	if (!req_passed_in && req) {
> +		ptlrpc_req_finished(req);
> +		if (reqp)
> +			*reqp = NULL;
> +	}
> +
> +	return rc;
> +}
> +EXPORT_SYMBOL(ldlm_cli_enqueue);
> +
> +/**
> + * Cancel locks locally.
> + * Returns:
> + * \retval LDLM_FL_LOCAL_ONLY if there is no need for a CANCEL RPC to the server
> + * \retval LDLM_FL_CANCELING otherwise;
> + * \retval LDLM_FL_BL_AST if there is a need for a separate CANCEL RPC.
> + */
> +static __u64 ldlm_cli_cancel_local(struct ldlm_lock *lock)
> +{
> +	__u64 rc = LDLM_FL_LOCAL_ONLY;
> +
> +	if (lock->l_conn_export) {
> +		bool local_only;
> +
> +		LDLM_DEBUG(lock, "client-side cancel");
> +		/* Set this flag to prevent others from getting new references*/
> +		lock_res_and_lock(lock);
> +		ldlm_set_cbpending(lock);
> +		local_only = !!(lock->l_flags &
> +				(LDLM_FL_LOCAL_ONLY | LDLM_FL_CANCEL_ON_BLOCK));
> +		ldlm_cancel_callback(lock);
> +		rc = ldlm_is_bl_ast(lock) ? LDLM_FL_BL_AST : LDLM_FL_CANCELING;
> +		unlock_res_and_lock(lock);
> +
> +		if (local_only) {
> +			CDEBUG(D_DLMTRACE,
> +			       "not sending request (at caller's instruction)\n");
> +			rc = LDLM_FL_LOCAL_ONLY;
> +		}
> +		ldlm_lock_cancel(lock);
> +	} else {
> +		LDLM_ERROR(lock, "Trying to cancel local lock");
> +		LBUG();
> +	}
> +
> +	return rc;
> +}
> +
> +/**
> + * Pack \a count locks in \a head into ldlm_request buffer of request \a req.
> + */
> +static void ldlm_cancel_pack(struct ptlrpc_request *req,
> +			     struct list_head *head, int count)
> +{
> +	struct ldlm_request *dlm;
> +	struct ldlm_lock *lock;
> +	int max, packed = 0;
> +
> +	dlm = req_capsule_client_get(&req->rq_pill, &RMF_DLM_REQ);
> +	LASSERT(dlm);
> +
> +	/* Check the room in the request buffer. */
> +	max = req_capsule_get_size(&req->rq_pill, &RMF_DLM_REQ, RCL_CLIENT) -
> +		sizeof(struct ldlm_request);
> +	max /= sizeof(struct lustre_handle);
> +	max += LDLM_LOCKREQ_HANDLES;
> +	LASSERT(max >= dlm->lock_count + count);
> +
> +	/* XXX: it would be better to pack lock handles grouped by resource.
> +	 * so that the server cancel would call filter_lvbo_update() less
> +	 * frequently.
> +	 */
> +	list_for_each_entry(lock, head, l_bl_ast) {
> +		if (!count--)
> +			break;
> +		LASSERT(lock->l_conn_export);
> +		/* Pack the lock handle to the given request buffer. */
> +		LDLM_DEBUG(lock, "packing");
> +		dlm->lock_handle[dlm->lock_count++] = lock->l_remote_handle;
> +		packed++;
> +	}
> +	CDEBUG(D_DLMTRACE, "%d locks packed\n", packed);
> +}
> +
> +/**
> + * Prepare and send a batched cancel RPC. It will include \a count lock
> + * handles of locks given in \a cancels list.
> + */
> +static int ldlm_cli_cancel_req(struct obd_export *exp,
> +			       struct list_head *cancels,
> +			       int count, enum ldlm_cancel_flags flags)
> +{
> +	struct ptlrpc_request *req = NULL;
> +	struct obd_import *imp;
> +	int free, sent = 0;
> +	int rc = 0;
> +
> +	LASSERT(exp);
> +	LASSERT(count > 0);
> +
> +	CFS_FAIL_TIMEOUT(OBD_FAIL_LDLM_PAUSE_CANCEL, cfs_fail_val);
> +
> +	if (CFS_FAIL_CHECK(OBD_FAIL_LDLM_CANCEL_RACE))
> +		return count;
> +
> +	free = ldlm_format_handles_avail(class_exp2cliimp(exp),
> +					 &RQF_LDLM_CANCEL, RCL_CLIENT, 0);
> +	if (count > free)
> +		count = free;
> +
> +	while (1) {
> +		imp = class_exp2cliimp(exp);
> +		if (!imp || imp->imp_invalid) {
> +			CDEBUG(D_DLMTRACE,
> +			       "skipping cancel on invalid import %p\n", imp);
> +			return count;
> +		}
> +
> +		req = ptlrpc_request_alloc(imp, &RQF_LDLM_CANCEL);
> +		if (!req) {
> +			rc = -ENOMEM;
> +			goto out;
> +		}
> +
> +		req_capsule_filled_sizes(&req->rq_pill, RCL_CLIENT);
> +		req_capsule_set_size(&req->rq_pill, &RMF_DLM_REQ, RCL_CLIENT,
> +				     ldlm_request_bufsize(count, LDLM_CANCEL));
> +
> +		rc = ptlrpc_request_pack(req, LUSTRE_DLM_VERSION, LDLM_CANCEL);
> +		if (rc) {
> +			ptlrpc_request_free(req);
> +			goto out;
> +		}
> +
> +		req->rq_request_portal = LDLM_CANCEL_REQUEST_PORTAL;
> +		req->rq_reply_portal = LDLM_CANCEL_REPLY_PORTAL;
> +		ptlrpc_at_set_req_timeout(req);
> +
> +		ldlm_cancel_pack(req, cancels, count);
> +
> +		ptlrpc_request_set_replen(req);
> +		if (flags & LCF_ASYNC) {
> +			ptlrpcd_add_req(req);
> +			sent = count;
> +			goto out;
> +		}
> +
> +		rc = ptlrpc_queue_wait(req);
> +		if (rc == LUSTRE_ESTALE) {
> +			CDEBUG(D_DLMTRACE,
> +			       "client/server (nid %s) out of sync -- not fatal\n",
> +			       libcfs_nid2str(req->rq_import->
> +					      imp_connection->c_peer.nid));
> +			rc = 0;
> +		} else if (rc == -ETIMEDOUT && /* check there was no reconnect*/
> +			   req->rq_import_generation == imp->imp_generation) {
> +			ptlrpc_req_finished(req);
> +			continue;
> +		} else if (rc != ELDLM_OK) {
> +			/* -ESHUTDOWN is common on umount */
> +			CDEBUG_LIMIT(rc == -ESHUTDOWN ? D_DLMTRACE : D_ERROR,
> +				     "Got rc %d from cancel RPC: canceling anyway\n",
> +				     rc);
> +			break;
> +		}
> +		sent = count;
> +		break;
> +	}
> +
> +	ptlrpc_req_finished(req);
> +out:
> +	return sent ? sent : rc;
> +}
> +
> +static inline struct ldlm_pool *ldlm_imp2pl(struct obd_import *imp)
> +{
> +	return &imp->imp_obd->obd_namespace->ns_pool;
> +}
> +
> +/**
> + * Update client's OBD pool related fields with new SLV and Limit from \a req.
> + */
> +int ldlm_cli_update_pool(struct ptlrpc_request *req)
> +{
> +	struct obd_device *obd;
> +	__u64 new_slv;
> +	__u32 new_limit;
> +
> +	if (unlikely(!req->rq_import || !req->rq_import->imp_obd ||
> +		     !imp_connect_lru_resize(req->rq_import))) {
> +		/*
> +		 * Do nothing for corner cases.
> +		 */
> +		return 0;
> +	}
> +
> +	/* In some cases RPC may contain SLV and limit zeroed out. This
> +	 * is the case when server does not support LRU resize feature.
> +	 * This is also possible in some recovery cases when server-side
> +	 * reqs have no reference to the OBD export and thus access to
> +	 * server-side namespace is not possible.
> +	 */
> +	if (lustre_msg_get_slv(req->rq_repmsg) == 0 ||
> +	    lustre_msg_get_limit(req->rq_repmsg) == 0) {
> +		DEBUG_REQ(D_HA, req,
> +			  "Zero SLV or Limit found (SLV: %llu, Limit: %u)",
> +			  lustre_msg_get_slv(req->rq_repmsg),
> +			  lustre_msg_get_limit(req->rq_repmsg));
> +		return 0;
> +	}
> +
> +	new_limit = lustre_msg_get_limit(req->rq_repmsg);
> +	new_slv = lustre_msg_get_slv(req->rq_repmsg);
> +	obd = req->rq_import->imp_obd;
> +
> +	/* Set new SLV and limit in OBD fields to make them accessible
> +	 * to the pool thread. We do not access obd_namespace and pool
> +	 * directly here as there is no reliable way to make sure that
> +	 * they are still alive at cleanup time. Evil races are possible
> +	 * which may cause Oops at that time.
> +	 */
> +	write_lock(&obd->obd_pool_lock);
> +	obd->obd_pool_slv = new_slv;
> +	obd->obd_pool_limit = new_limit;
> +	write_unlock(&obd->obd_pool_lock);
> +
> +	return 0;
> +}
> +
> +/**
> + * Client side lock cancel.
> + *
> + * Lock must not have any readers or writers by this time.
> + */
> +int ldlm_cli_cancel(const struct lustre_handle *lockh,
> +		    enum ldlm_cancel_flags cancel_flags)
> +{
> +	struct obd_export *exp;
> +	int avail, flags, count = 1;
> +	__u64 rc = 0;
> +	struct ldlm_namespace *ns;
> +	struct ldlm_lock *lock;
> +	LIST_HEAD(cancels);
> +
> +	lock = ldlm_handle2lock_long(lockh, 0);
> +	if (!lock) {
> +		LDLM_DEBUG_NOLOCK("lock is already being destroyed");
> +		return 0;
> +	}
> +
> +	lock_res_and_lock(lock);
> +	/* Lock is being canceled and the caller doesn't want to wait */
> +	if (ldlm_is_canceling(lock) && (cancel_flags & LCF_ASYNC)) {
> +		unlock_res_and_lock(lock);
> +		LDLM_LOCK_RELEASE(lock);
> +		return 0;
> +	}
> +
> +	ldlm_set_canceling(lock);
> +	unlock_res_and_lock(lock);
> +
> +	rc = ldlm_cli_cancel_local(lock);
> +	if (rc == LDLM_FL_LOCAL_ONLY || cancel_flags & LCF_LOCAL) {
> +		LDLM_LOCK_RELEASE(lock);
> +		return 0;
> +	}
> +	/* Even if the lock is marked as LDLM_FL_BL_AST, this is a LDLM_CANCEL
> +	 * RPC which goes to canceld portal, so we can cancel other LRU locks
> +	 * here and send them all as one LDLM_CANCEL RPC.
> +	 */
> +	LASSERT(list_empty(&lock->l_bl_ast));
> +	list_add(&lock->l_bl_ast, &cancels);
> +
> +	exp = lock->l_conn_export;
> +	if (exp_connect_cancelset(exp)) {
> +		avail = ldlm_format_handles_avail(class_exp2cliimp(exp),
> +						  &RQF_LDLM_CANCEL,
> +						  RCL_CLIENT, 0);
> +		LASSERT(avail > 0);
> +
> +		ns = ldlm_lock_to_ns(lock);
> +		flags = ns_connect_lru_resize(ns) ?
> +			LDLM_LRU_FLAG_LRUR : LDLM_LRU_FLAG_AGED;
> +		count += ldlm_cancel_lru_local(ns, &cancels, 0, avail - 1,
> +					       LCF_BL_AST, flags);
> +	}
> +	ldlm_cli_cancel_list(&cancels, count, NULL, cancel_flags);
> +	return 0;
> +}
> +EXPORT_SYMBOL(ldlm_cli_cancel);
> +
> +/**
> + * Locally cancel up to \a count locks in list \a cancels.
> + * Return the number of cancelled locks.
> + */
> +int ldlm_cli_cancel_list_local(struct list_head *cancels, int count,
> +			       enum ldlm_cancel_flags flags)
> +{
> +	LIST_HEAD(head);
> +	struct ldlm_lock *lock, *next;
> +	int left = 0, bl_ast = 0;
> +	__u64 rc;
> +
> +	left = count;
> +	list_for_each_entry_safe(lock, next, cancels, l_bl_ast) {
> +		if (left-- == 0)
> +			break;
> +
> +		if (flags & LCF_LOCAL) {
> +			rc = LDLM_FL_LOCAL_ONLY;
> +			ldlm_lock_cancel(lock);
> +		} else {
> +			rc = ldlm_cli_cancel_local(lock);
> +		}
> +		/* Until we have compound requests and can send LDLM_CANCEL
> +		 * requests batched with generic RPCs, we need to send cancels
> +		 * with the LDLM_FL_BL_AST flag in a separate RPC from
> +		 * the one being generated now.
> +		 */
> +		if (!(flags & LCF_BL_AST) && (rc == LDLM_FL_BL_AST)) {
> +			LDLM_DEBUG(lock, "Cancel lock separately");
> +			list_del_init(&lock->l_bl_ast);
> +			list_add(&lock->l_bl_ast, &head);
> +			bl_ast++;
> +			continue;
> +		}
> +		if (rc == LDLM_FL_LOCAL_ONLY) {
> +			/* CANCEL RPC should not be sent to server. */
> +			list_del_init(&lock->l_bl_ast);
> +			LDLM_LOCK_RELEASE(lock);
> +			count--;
> +		}
> +	}
> +	if (bl_ast > 0) {
> +		count -= bl_ast;
> +		ldlm_cli_cancel_list(&head, bl_ast, NULL, 0);
> +	}
> +
> +	return count;
> +}
> +
> +/**
> + * Cancel as many locks as possible w/o sending any RPCs (e.g. to write back
> + * dirty data, to close a file, ...) or waiting for any RPCs in-flight (e.g.
> + * readahead requests, ...)
> + */
> +static enum ldlm_policy_res
> +ldlm_cancel_no_wait_policy(struct ldlm_namespace *ns, struct ldlm_lock *lock,
> +			   int unused, int added, int count)
> +{
> +	enum ldlm_policy_res result = LDLM_POLICY_CANCEL_LOCK;
> +
> +	/* don't check added & count since we want to process all locks
> +	 * from unused list.
> +	 * It's fine to not take lock to access lock->l_resource since
> +	 * the lock has already been granted so it won't change.
> +	 */
> +	switch (lock->l_resource->lr_type) {
> +	case LDLM_EXTENT:
> +	case LDLM_IBITS:
> +		if (ns->ns_cancel && ns->ns_cancel(lock) != 0)
> +			break;
> +		/* fall through */
> +	default:
> +		result = LDLM_POLICY_SKIP_LOCK;
> +		lock_res_and_lock(lock);
> +		ldlm_set_skipped(lock);
> +		unlock_res_and_lock(lock);
> +		break;
> +	}
> +
> +	return result;
> +}
> +
> +/**
> + * Callback function for LRU-resize policy. Decides whether to keep
> + * \a lock in LRU for current \a LRU size \a unused, added in current
> + * scan \a added and number of locks to be preferably canceled \a count.
> + *
> + * \retval LDLM_POLICY_KEEP_LOCK keep lock in LRU in stop scanning
> + *
> + * \retval LDLM_POLICY_CANCEL_LOCK cancel lock from LRU
> + */
> +static enum ldlm_policy_res ldlm_cancel_lrur_policy(struct ldlm_namespace *ns,
> +						    struct ldlm_lock *lock,
> +						    int unused, int added,
> +						    int count)
> +{
> +	unsigned long cur = jiffies;
> +	struct ldlm_pool *pl = &ns->ns_pool;
> +	__u64 slv, lvf, lv;
> +	unsigned long la;
> +
> +	/* Stop LRU processing when we reach past @count or have checked all
> +	 * locks in LRU.
> +	 */
> +	if (count && added >= count)
> +		return LDLM_POLICY_KEEP_LOCK;
> +
> +	/*
> +	 * Despite of the LV, It doesn't make sense to keep the lock which
> +	 * is unused for ns_max_age time.
> +	 */
> +	if (time_after(jiffies, lock->l_last_used + ns->ns_max_age))
> +		return LDLM_POLICY_CANCEL_LOCK;
> +
> +	slv = ldlm_pool_get_slv(pl);
> +	lvf = ldlm_pool_get_lvf(pl);
> +	la = (cur - lock->l_last_used) / HZ;
> +	lv = lvf * la * unused;
> +
> +	/* Inform pool about current CLV to see it via debugfs. */
> +	ldlm_pool_set_clv(pl, lv);
> +
> +	/* Stop when SLV is not yet come from server or lv is smaller than
> +	 * it is.
> +	 */
> +	if (slv == 0 || lv < slv)
> +		return LDLM_POLICY_KEEP_LOCK;
> +
> +	return LDLM_POLICY_CANCEL_LOCK;
> +}
> +
> +/**
> + * Callback function for debugfs used policy. Makes decision whether to keep
> + * \a lock in LRU for current \a LRU size \a unused, added in current scan \a
> + * added and number of locks to be preferably canceled \a count.
> + *
> + * \retval LDLM_POLICY_KEEP_LOCK keep lock in LRU in stop scanning
> + *
> + * \retval LDLM_POLICY_CANCEL_LOCK cancel lock from LRU
> + */
> +static enum ldlm_policy_res ldlm_cancel_passed_policy(struct ldlm_namespace *ns,
> +						      struct ldlm_lock *lock,
> +						      int unused, int added,
> +						      int count)
> +{
> +	/* Stop LRU processing when we reach past @count or have checked all
> +	 * locks in LRU.
> +	 */
> +	return (added >= count) ?
> +		LDLM_POLICY_KEEP_LOCK : LDLM_POLICY_CANCEL_LOCK;
> +}
> +
> +/**
> + * Callback function for aged policy. Makes decision whether to keep \a lock in
> + * LRU for current LRU size \a unused, added in current scan \a added and
> + * number of locks to be preferably canceled \a count.
> + *
> + * \retval LDLM_POLICY_KEEP_LOCK keep lock in LRU in stop scanning
> + *
> + * \retval LDLM_POLICY_CANCEL_LOCK cancel lock from LRU
> + */
> +static enum ldlm_policy_res ldlm_cancel_aged_policy(struct ldlm_namespace *ns,
> +						    struct ldlm_lock *lock,
> +						    int unused, int added,
> +						    int count)
> +{
> +	if ((added >= count) &&
> +	    time_before(jiffies, lock->l_last_used + ns->ns_max_age))
> +		return LDLM_POLICY_KEEP_LOCK;
> +
> +	return LDLM_POLICY_CANCEL_LOCK;
> +}
> +
> +static enum ldlm_policy_res
> +ldlm_cancel_lrur_no_wait_policy(struct ldlm_namespace *ns,
> +				struct ldlm_lock *lock,
> +				int unused, int added,
> +				int count)
> +{
> +	enum ldlm_policy_res result;
> +
> +	result = ldlm_cancel_lrur_policy(ns, lock, unused, added, count);
> +	if (result == LDLM_POLICY_KEEP_LOCK)
> +		return result;
> +
> +	return ldlm_cancel_no_wait_policy(ns, lock, unused, added, count);
> +}
> +
> +/**
> + * Callback function for default policy. Makes decision whether to keep \a lock
> + * in LRU for current LRU size \a unused, added in current scan \a added and
> + * number of locks to be preferably canceled \a count.
> + *
> + * \retval LDLM_POLICY_KEEP_LOCK keep lock in LRU in stop scanning
> + *
> + * \retval LDLM_POLICY_CANCEL_LOCK cancel lock from LRU
> + */
> +static enum ldlm_policy_res
> +ldlm_cancel_default_policy(struct ldlm_namespace *ns, struct ldlm_lock *lock,
> +			   int unused, int added, int count)
> +{
> +	/* Stop LRU processing when we reach past count or have checked all
> +	 * locks in LRU.
> +	 */
> +	return (added >= count) ?
> +		LDLM_POLICY_KEEP_LOCK : LDLM_POLICY_CANCEL_LOCK;
> +}
> +
> +typedef enum ldlm_policy_res (*ldlm_cancel_lru_policy_t)(
> +						      struct ldlm_namespace *,
> +						      struct ldlm_lock *, int,
> +						      int, int);
> +
> +static ldlm_cancel_lru_policy_t
> +ldlm_cancel_lru_policy(struct ldlm_namespace *ns, int flags)
> +{
> +	if (flags & LDLM_LRU_FLAG_NO_WAIT)
> +		return ldlm_cancel_no_wait_policy;
> +
> +	if (ns_connect_lru_resize(ns)) {
> +		if (flags & LDLM_LRU_FLAG_SHRINK)
> +			/* We kill passed number of old locks. */
> +			return ldlm_cancel_passed_policy;
> +		else if (flags & LDLM_LRU_FLAG_LRUR)
> +			return ldlm_cancel_lrur_policy;
> +		else if (flags & LDLM_LRU_FLAG_PASSED)
> +			return ldlm_cancel_passed_policy;
> +		else if (flags & LDLM_LRU_FLAG_LRUR_NO_WAIT)
> +			return ldlm_cancel_lrur_no_wait_policy;
> +	} else {
> +		if (flags & LDLM_LRU_FLAG_AGED)
> +			return ldlm_cancel_aged_policy;
> +	}
> +
> +	return ldlm_cancel_default_policy;
> +}
> +
> +/**
> + * - Free space in LRU for \a count new locks,
> + *   redundant unused locks are canceled locally;
> + * - also cancel locally unused aged locks;
> + * - do not cancel more than \a max locks;
> + * - GET the found locks and add them into the \a cancels list.
> + *
> + * A client lock can be added to the l_bl_ast list only when it is
> + * marked LDLM_FL_CANCELING. Otherwise, somebody is already doing
> + * CANCEL.  There are the following use cases:
> + * ldlm_cancel_resource_local(), ldlm_cancel_lru_local() and
> + * ldlm_cli_cancel(), which check and set this flag properly. As any
> + * attempt to cancel a lock rely on this flag, l_bl_ast list is accessed
> + * later without any special locking.
> + *
> + * Calling policies for enabled LRU resize:
> + * ----------------------------------------
> + * flags & LDLM_LRU_FLAG_LRUR	- use LRU resize policy (SLV from server) to
> + *				  cancel not more than \a count locks;
> + *
> + * flags & LDLM_LRU_FLAG_PASSED - cancel \a count number of old locks (located
> + *				  at the beginning of LRU list);
> + *
> + * flags & LDLM_LRU_FLAG_SHRINK - cancel not more than \a count locks according
> + *				  to memory pressure policy function;
> + *
> + * flags & LDLM_LRU_FLAG_AGED   - cancel \a count locks according to
> + *				  "aged policy".
> + *
> + * flags & LDLM_LRU_FLAG_NO_WAIT - cancel as many unused locks as possible
> + *				   (typically before replaying locks) w/o
> + *				   sending any RPCs or waiting for any
> + *				   outstanding RPC to complete.
> + */
> +static int ldlm_prepare_lru_list(struct ldlm_namespace *ns,
> +				 struct list_head *cancels, int count, int max,
> +				 int flags)
> +{
> +	ldlm_cancel_lru_policy_t pf;
> +	struct ldlm_lock *lock, *next;
> +	int added = 0, unused, remained;
> +	int no_wait = flags &
> +		(LDLM_LRU_FLAG_NO_WAIT | LDLM_LRU_FLAG_LRUR_NO_WAIT);
> +
> +	spin_lock(&ns->ns_lock);
> +	unused = ns->ns_nr_unused;
> +	remained = unused;
> +
> +	if (!ns_connect_lru_resize(ns))
> +		count += unused - ns->ns_max_unused;
> +
> +	pf = ldlm_cancel_lru_policy(ns, flags);
> +	LASSERT(pf);
> +
> +	while (!list_empty(&ns->ns_unused_list)) {
> +		enum ldlm_policy_res result;
> +		time_t last_use = 0;
> +
> +		/* all unused locks */
> +		if (remained-- <= 0)
> +			break;
> +
> +		/* For any flags, stop scanning if @max is reached. */
> +		if (max && added >= max)
> +			break;
> +
> +		list_for_each_entry_safe(lock, next, &ns->ns_unused_list,
> +					 l_lru) {
> +			/* No locks which got blocking requests. */
> +			LASSERT(!ldlm_is_bl_ast(lock));
> +
> +			if (no_wait && ldlm_is_skipped(lock))
> +				/* already processed */
> +				continue;
> +
> +			last_use = lock->l_last_used;
> +			if (last_use == jiffies)
> +				continue;
> +
> +			/* Somebody is already doing CANCEL. No need for this
> +			 * lock in LRU, do not traverse it again.
> +			 */
> +			if (!ldlm_is_canceling(lock))
> +				break;
> +
> +			ldlm_lock_remove_from_lru_nolock(lock);
> +		}
> +		if (&lock->l_lru == &ns->ns_unused_list)
> +			break;
> +
> +		LDLM_LOCK_GET(lock);
> +		spin_unlock(&ns->ns_lock);
> +		lu_ref_add(&lock->l_reference, __func__, current);
> +
> +		/* Pass the lock through the policy filter and see if it
> +		 * should stay in LRU.
> +		 *
> +		 * Even for shrinker policy we stop scanning if
> +		 * we find a lock that should stay in the cache.
> +		 * We should take into account lock age anyway
> +		 * as a new lock is a valuable resource even if
> +		 * it has a low weight.
> +		 *
> +		 * That is, for shrinker policy we drop only
> +		 * old locks, but additionally choose them by
> +		 * their weight. Big extent locks will stay in
> +		 * the cache.
> +		 */
> +		result = pf(ns, lock, unused, added, count);
> +		if (result == LDLM_POLICY_KEEP_LOCK) {
> +			lu_ref_del(&lock->l_reference,
> +				   __func__, current);
> +			LDLM_LOCK_RELEASE(lock);
> +			spin_lock(&ns->ns_lock);
> +			break;
> +		}
> +		if (result == LDLM_POLICY_SKIP_LOCK) {
> +			lu_ref_del(&lock->l_reference,
> +				   __func__, current);
> +			LDLM_LOCK_RELEASE(lock);
> +			spin_lock(&ns->ns_lock);
> +			continue;
> +		}
> +
> +		lock_res_and_lock(lock);
> +		/* Check flags again under the lock. */
> +		if (ldlm_is_canceling(lock) ||
> +		    (ldlm_lock_remove_from_lru_check(lock, last_use) == 0)) {
> +			/* Another thread is removing lock from LRU, or
> +			 * somebody is already doing CANCEL, or there
> +			 * is a blocking request which will send cancel
> +			 * by itself, or the lock is no longer unused or
> +			 * the lock has been used since the pf() call and
> +			 * pages could be put under it.
> +			 */
> +			unlock_res_and_lock(lock);
> +			lu_ref_del(&lock->l_reference,
> +				   __func__, current);
> +			LDLM_LOCK_RELEASE(lock);
> +			spin_lock(&ns->ns_lock);
> +			continue;
> +		}
> +		LASSERT(!lock->l_readers && !lock->l_writers);
> +
> +		/* If we have chosen to cancel this lock voluntarily, we
> +		 * better send cancel notification to server, so that it
> +		 * frees appropriate state. This might lead to a race
> +		 * where while we are doing cancel here, server is also
> +		 * silently cancelling this lock.
> +		 */
> +		ldlm_clear_cancel_on_block(lock);
> +
> +		/* Setting the CBPENDING flag is a little misleading,
> +		 * but prevents an important race; namely, once
> +		 * CBPENDING is set, the lock can accumulate no more
> +		 * readers/writers. Since readers and writers are
> +		 * already zero here, ldlm_lock_decref() won't see
> +		 * this flag and call l_blocking_ast
> +		 */
> +		lock->l_flags |= LDLM_FL_CBPENDING | LDLM_FL_CANCELING;
> +
> +		/* We can't re-add to l_lru as it confuses the
> +		 * refcounting in ldlm_lock_remove_from_lru() if an AST
> +		 * arrives after we drop lr_lock below. We use l_bl_ast
> +		 * and can't use l_pending_chain as it is used both on
> +		 * server and client nevertheless bug 5666 says it is
> +		 * used only on server
> +		 */
> +		LASSERT(list_empty(&lock->l_bl_ast));
> +		list_add(&lock->l_bl_ast, cancels);
> +		unlock_res_and_lock(lock);
> +		lu_ref_del(&lock->l_reference, __func__, current);
> +		spin_lock(&ns->ns_lock);
> +		added++;
> +		unused--;
> +	}
> +	spin_unlock(&ns->ns_lock);
> +	return added;
> +}
> +
> +int ldlm_cancel_lru_local(struct ldlm_namespace *ns,
> +			  struct list_head *cancels, int count, int max,
> +			  enum ldlm_cancel_flags cancel_flags, int flags)
> +{
> +	int added;
> +
> +	added = ldlm_prepare_lru_list(ns, cancels, count, max, flags);
> +	if (added <= 0)
> +		return added;
> +	return ldlm_cli_cancel_list_local(cancels, added, cancel_flags);
> +}
> +
> +/**
> + * Cancel at least \a nr locks from given namespace LRU.
> + *
> + * When called with LCF_ASYNC the blocking callback will be handled
> + * in a thread and this function will return after the thread has been
> + * asked to call the callback.  When called with LCF_ASYNC the blocking
> + * callback will be performed in this function.
> + */
> +int ldlm_cancel_lru(struct ldlm_namespace *ns, int nr,
> +		    enum ldlm_cancel_flags cancel_flags,
> +		    int flags)
> +{
> +	LIST_HEAD(cancels);
> +	int count, rc;
> +
> +	/* Just prepare the list of locks, do not actually cancel them yet.
> +	 * Locks are cancelled later in a separate thread.
> +	 */
> +	count = ldlm_prepare_lru_list(ns, &cancels, nr, 0, flags);
> +	rc = ldlm_bl_to_thread_list(ns, NULL, &cancels, count, cancel_flags);
> +	if (rc == 0)
> +		return count;
> +
> +	return 0;
> +}
> +
> +/**
> + * Find and cancel locally unused locks found on resource, matched to the
> + * given policy, mode. GET the found locks and add them into the \a cancels
> + * list.
> + */
> +int ldlm_cancel_resource_local(struct ldlm_resource *res,
> +			       struct list_head *cancels,
> +			       union ldlm_policy_data *policy,
> +			       enum ldlm_mode mode, __u64 lock_flags,
> +			       enum ldlm_cancel_flags cancel_flags,
> +			       void *opaque)
> +{
> +	struct ldlm_lock *lock;
> +	int count = 0;
> +
> +	lock_res(res);
> +	list_for_each_entry(lock, &res->lr_granted, l_res_link) {
> +		if (opaque && lock->l_ast_data != opaque) {
> +			LDLM_ERROR(lock, "data %p doesn't match opaque %p",
> +				   lock->l_ast_data, opaque);
> +			continue;
> +		}
> +
> +		if (lock->l_readers || lock->l_writers)
> +			continue;
> +
> +		/* If somebody is already doing CANCEL, or blocking AST came,
> +		 * skip this lock.
> +		 */
> +		if (ldlm_is_bl_ast(lock) || ldlm_is_canceling(lock))
> +			continue;
> +
> +		if (lockmode_compat(lock->l_granted_mode, mode))
> +			continue;
> +
> +		/* If policy is given and this is IBITS lock, add to list only
> +		 * those locks that match by policy.
> +		 */
> +		if (policy && (lock->l_resource->lr_type == LDLM_IBITS) &&
> +		    !(lock->l_policy_data.l_inodebits.bits &
> +		      policy->l_inodebits.bits))
> +			continue;
> +
> +		/* See CBPENDING comment in ldlm_cancel_lru */
> +		lock->l_flags |= LDLM_FL_CBPENDING | LDLM_FL_CANCELING |
> +				 lock_flags;
> +
> +		LASSERT(list_empty(&lock->l_bl_ast));
> +		list_add(&lock->l_bl_ast, cancels);
> +		LDLM_LOCK_GET(lock);
> +		count++;
> +	}
> +	unlock_res(res);
> +
> +	return ldlm_cli_cancel_list_local(cancels, count, cancel_flags);
> +}
> +EXPORT_SYMBOL(ldlm_cancel_resource_local);
> +
> +/**
> + * Cancel client-side locks from a list and send/prepare cancel RPCs to the
> + * server.
> + * If \a req is NULL, send CANCEL request to server with handles of locks
> + * in the \a cancels. If EARLY_CANCEL is not supported, send CANCEL requests
> + * separately per lock.
> + * If \a req is not NULL, put handles of locks in \a cancels into the request
> + * buffer at the offset \a off.
> + * Destroy \a cancels at the end.
> + */
> +int ldlm_cli_cancel_list(struct list_head *cancels, int count,
> +			 struct ptlrpc_request *req,
> +			 enum ldlm_cancel_flags flags)
> +{
> +	struct ldlm_lock *lock;
> +	int res = 0;
> +
> +	if (list_empty(cancels) || count == 0)
> +		return 0;
> +
> +	/* XXX: requests (both batched and not) could be sent in parallel.
> +	 * Usually it is enough to have just 1 RPC, but it is possible that
> +	 * there are too many locks to be cancelled in LRU or on a resource.
> +	 * It would also speed up the case when the server does not support
> +	 * the feature.
> +	 */
> +	while (count > 0) {
> +		LASSERT(!list_empty(cancels));
> +		lock = list_first_entry(cancels, struct ldlm_lock, l_bl_ast);
> +		LASSERT(lock->l_conn_export);
> +
> +		if (exp_connect_cancelset(lock->l_conn_export)) {
> +			res = count;
> +			if (req)
> +				ldlm_cancel_pack(req, cancels, count);
> +			else
> +				res = ldlm_cli_cancel_req(lock->l_conn_export,
> +							  cancels, count,
> +							  flags);
> +		} else {
> +			res = ldlm_cli_cancel_req(lock->l_conn_export,
> +						  cancels, 1, flags);
> +		}
> +
> +		if (res < 0) {
> +			CDEBUG_LIMIT(res == -ESHUTDOWN ? D_DLMTRACE : D_ERROR,
> +				     "%s: %d\n", __func__, res);
> +			res = count;
> +		}
> +
> +		count -= res;
> +		ldlm_lock_list_put(cancels, l_bl_ast, res);
> +	}
> +	LASSERT(count == 0);
> +	return 0;
> +}
> +EXPORT_SYMBOL(ldlm_cli_cancel_list);
> +
> +/**
> + * Cancel all locks on a resource that have 0 readers/writers.
> + *
> + * If flags & LDLM_FL_LOCAL_ONLY, throw the locks away without trying
> + * to notify the server.
> + */
> +int ldlm_cli_cancel_unused_resource(struct ldlm_namespace *ns,
> +				    const struct ldlm_res_id *res_id,
> +				    union ldlm_policy_data *policy,
> +				    enum ldlm_mode mode,
> +				    enum ldlm_cancel_flags flags,
> +				    void *opaque)
> +{
> +	struct ldlm_resource *res;
> +	LIST_HEAD(cancels);
> +	int count;
> +	int rc;
> +
> +	res = ldlm_resource_get(ns, NULL, res_id, 0, 0);
> +	if (IS_ERR(res)) {
> +		/* This is not a problem. */
> +		CDEBUG(D_INFO, "No resource %llu\n", res_id->name[0]);
> +		return 0;
> +	}
> +
> +	LDLM_RESOURCE_ADDREF(res);
> +	count = ldlm_cancel_resource_local(res, &cancels, policy, mode,
> +					   0, flags | LCF_BL_AST, opaque);
> +	rc = ldlm_cli_cancel_list(&cancels, count, NULL, flags);
> +	if (rc != ELDLM_OK)
> +		CERROR("canceling unused lock " DLDLMRES ": rc = %d\n",
> +		       PLDLMRES(res), rc);
> +
> +	LDLM_RESOURCE_DELREF(res);
> +	ldlm_resource_putref(res);
> +	return 0;
> +}
> +EXPORT_SYMBOL(ldlm_cli_cancel_unused_resource);
> +
> +struct ldlm_cli_cancel_arg {
> +	int     lc_flags;
> +	void   *lc_opaque;
> +};
> +
> +static int ldlm_cli_hash_cancel_unused(struct cfs_hash *hs,
> +				       struct cfs_hash_bd *bd,
> +				       struct hlist_node *hnode, void *arg)
> +{
> +	struct ldlm_resource	   *res = cfs_hash_object(hs, hnode);
> +	struct ldlm_cli_cancel_arg     *lc = arg;
> +
> +	ldlm_cli_cancel_unused_resource(ldlm_res_to_ns(res), &res->lr_name,
> +					NULL, LCK_MINMODE,
> +					lc->lc_flags, lc->lc_opaque);
> +	/* must return 0 for hash iteration */
> +	return 0;
> +}
> +
> +/**
> + * Cancel all locks on a namespace (or a specific resource, if given)
> + * that have 0 readers/writers.
> + *
> + * If flags & LCF_LOCAL, throw the locks away without trying
> + * to notify the server.
> + */
> +int ldlm_cli_cancel_unused(struct ldlm_namespace *ns,
> +			   const struct ldlm_res_id *res_id,
> +			   enum ldlm_cancel_flags flags, void *opaque)
> +{
> +	struct ldlm_cli_cancel_arg arg = {
> +		.lc_flags       = flags,
> +		.lc_opaque      = opaque,
> +	};
> +
> +	if (!ns)
> +		return ELDLM_OK;
> +
> +	if (res_id) {
> +		return ldlm_cli_cancel_unused_resource(ns, res_id, NULL,
> +						       LCK_MINMODE, flags,
> +						       opaque);
> +	} else {
> +		cfs_hash_for_each_nolock(ns->ns_rs_hash,
> +					 ldlm_cli_hash_cancel_unused, &arg, 0);
> +		return ELDLM_OK;
> +	}
> +}
> +EXPORT_SYMBOL(ldlm_cli_cancel_unused);
> +
> +/* Lock iterators. */
> +
> +static int ldlm_resource_foreach(struct ldlm_resource *res,
> +				 ldlm_iterator_t iter, void *closure)
> +{
> +	struct ldlm_lock *tmp;
> +	struct ldlm_lock *lock;
> +	int rc = LDLM_ITER_CONTINUE;
> +
> +	if (!res)
> +		return LDLM_ITER_CONTINUE;
> +
> +	lock_res(res);
> +	list_for_each_entry_safe(lock, tmp, &res->lr_granted, l_res_link) {
> +		if (iter(lock, closure) == LDLM_ITER_STOP) {
> +			rc = LDLM_ITER_STOP;
> +			goto out;
> +		}
> +	}
> +
> +	list_for_each_entry_safe(lock, tmp, &res->lr_waiting, l_res_link) {
> +		if (iter(lock, closure) == LDLM_ITER_STOP) {
> +			rc = LDLM_ITER_STOP;
> +			goto out;
> +		}
> +	}
> + out:
> +	unlock_res(res);
> +	return rc;
> +}
> +
> +struct iter_helper_data {
> +	ldlm_iterator_t iter;
> +	void *closure;
> +};
> +
> +static int ldlm_iter_helper(struct ldlm_lock *lock, void *closure)
> +{
> +	struct iter_helper_data *helper = closure;
> +
> +	return helper->iter(lock, helper->closure);
> +}
> +
> +static int ldlm_res_iter_helper(struct cfs_hash *hs, struct cfs_hash_bd *bd,
> +				struct hlist_node *hnode, void *arg)
> +
> +{
> +	struct ldlm_resource *res = cfs_hash_object(hs, hnode);
> +
> +	return ldlm_resource_foreach(res, ldlm_iter_helper, arg) ==
> +	       LDLM_ITER_STOP;
> +}
> +
> +static void ldlm_namespace_foreach(struct ldlm_namespace *ns,
> +				   ldlm_iterator_t iter, void *closure)
> +
> +{
> +	struct iter_helper_data helper = {
> +		.iter		= iter,
> +		.closure	= closure,
> +	};
> +
> +	cfs_hash_for_each_nolock(ns->ns_rs_hash,
> +				 ldlm_res_iter_helper, &helper, 0);
> +}
> +
> +/* non-blocking function to manipulate a lock whose cb_data is being put away.
> + * return  0:  find no resource
> + *       > 0:  must be LDLM_ITER_STOP/LDLM_ITER_CONTINUE.
> + *       < 0:  errors
> + */
> +int ldlm_resource_iterate(struct ldlm_namespace *ns,
> +			  const struct ldlm_res_id *res_id,
> +			  ldlm_iterator_t iter, void *data)
> +{
> +	struct ldlm_resource *res;
> +	int rc;
> +
> +	LASSERTF(ns, "must pass in namespace\n");
> +
> +	res = ldlm_resource_get(ns, NULL, res_id, 0, 0);
> +	if (IS_ERR(res))
> +		return 0;
> +
> +	LDLM_RESOURCE_ADDREF(res);
> +	rc = ldlm_resource_foreach(res, iter, data);
> +	LDLM_RESOURCE_DELREF(res);
> +	ldlm_resource_putref(res);
> +	return rc;
> +}
> +EXPORT_SYMBOL(ldlm_resource_iterate);
> +
> +/* Lock replay */
> +
> +static int ldlm_chain_lock_for_replay(struct ldlm_lock *lock, void *closure)
> +{
> +	struct list_head *list = closure;
> +
> +	/* we use l_pending_chain here, because it's unused on clients. */
> +	LASSERTF(list_empty(&lock->l_pending_chain),
> +		 "lock %p next %p prev %p\n",
> +		 lock, &lock->l_pending_chain.next,
> +		 &lock->l_pending_chain.prev);
> +	/* bug 9573: don't replay locks left after eviction, or
> +	 * bug 17614: locks being actively cancelled. Get a reference
> +	 * on a lock so that it does not disappear under us (e.g. due to cancel)
> +	 */
> +	if (!(lock->l_flags & (LDLM_FL_FAILED | LDLM_FL_BL_DONE))) {
> +		list_add(&lock->l_pending_chain, list);
> +		LDLM_LOCK_GET(lock);
> +	}
> +
> +	return LDLM_ITER_CONTINUE;
> +}
> +
> +static int replay_lock_interpret(const struct lu_env *env,
> +				 struct ptlrpc_request *req,
> +				 struct ldlm_async_args *aa, int rc)
> +{
> +	struct ldlm_lock     *lock;
> +	struct ldlm_reply    *reply;
> +	struct obd_export    *exp;
> +
> +	atomic_dec(&req->rq_import->imp_replay_inflight);
> +	if (rc != ELDLM_OK)
> +		goto out;
> +
> +	reply = req_capsule_server_get(&req->rq_pill, &RMF_DLM_REP);
> +	if (!reply) {
> +		rc = -EPROTO;
> +		goto out;
> +	}
> +
> +	lock = ldlm_handle2lock(&aa->lock_handle);
> +	if (!lock) {
> +		CERROR("received replay ack for unknown local cookie %#llx remote cookie %#llx from server %s id %s\n",
> +		       aa->lock_handle.cookie, reply->lock_handle.cookie,
> +		       req->rq_export->exp_client_uuid.uuid,
> +		       libcfs_id2str(req->rq_peer));
> +		rc = -ESTALE;
> +		goto out;
> +	}
> +
> +	/* Key change rehash lock in per-export hash with new key */
> +	exp = req->rq_export;
> +	lock->l_remote_handle = reply->lock_handle;
> +
> +	LDLM_DEBUG(lock, "replayed lock:");
> +	ptlrpc_import_recovery_state_machine(req->rq_import);
> +	LDLM_LOCK_PUT(lock);
> +out:
> +	if (rc != ELDLM_OK)
> +		ptlrpc_connect_import(req->rq_import);
> +
> +	return rc;
> +}
> +
> +static int replay_one_lock(struct obd_import *imp, struct ldlm_lock *lock)
> +{
> +	struct ptlrpc_request *req;
> +	struct ldlm_async_args *aa;
> +	struct ldlm_request   *body;
> +	int flags;
> +
> +	/* Bug 11974: Do not replay a lock which is actively being canceled */
> +	if (ldlm_is_bl_done(lock)) {
> +		LDLM_DEBUG(lock, "Not replaying canceled lock:");
> +		return 0;
> +	}
> +
> +	/* If this is reply-less callback lock, we cannot replay it, since
> +	 * server might have long dropped it, but notification of that event was
> +	 * lost by network. (and server granted conflicting lock already)
> +	 */
> +	if (ldlm_is_cancel_on_block(lock)) {
> +		LDLM_DEBUG(lock, "Not replaying reply-less lock:");
> +		ldlm_lock_cancel(lock);
> +		return 0;
> +	}
> +
> +	/*
> +	 * If granted mode matches the requested mode, this lock is granted.
> +	 *
> +	 * If they differ, but we have a granted mode, then we were granted
> +	 * one mode and now want another: ergo, converting.
> +	 *
> +	 * If we haven't been granted anything and are on a resource list,
> +	 * then we're blocked/waiting.
> +	 *
> +	 * If we haven't been granted anything and we're NOT on a resource list,
> +	 * then we haven't got a reply yet and don't have a known disposition.
> +	 * This happens whenever a lock enqueue is the request that triggers
> +	 * recovery.
> +	 */
> +	if (lock->l_granted_mode == lock->l_req_mode)
> +		flags = LDLM_FL_REPLAY | LDLM_FL_BLOCK_GRANTED;
> +	else if (lock->l_granted_mode)
> +		flags = LDLM_FL_REPLAY | LDLM_FL_BLOCK_CONV;
> +	else if (!list_empty(&lock->l_res_link))
> +		flags = LDLM_FL_REPLAY | LDLM_FL_BLOCK_WAIT;
> +	else
> +		flags = LDLM_FL_REPLAY;
> +
> +	req = ptlrpc_request_alloc_pack(imp, &RQF_LDLM_ENQUEUE,
> +					LUSTRE_DLM_VERSION, LDLM_ENQUEUE);
> +	if (!req)
> +		return -ENOMEM;
> +
> +	/* We're part of recovery, so don't wait for it. */
> +	req->rq_send_state = LUSTRE_IMP_REPLAY_LOCKS;
> +
> +	body = req_capsule_client_get(&req->rq_pill, &RMF_DLM_REQ);
> +	ldlm_lock2desc(lock, &body->lock_desc);
> +	body->lock_flags = ldlm_flags_to_wire(flags);
> +
> +	ldlm_lock2handle(lock, &body->lock_handle[0]);
> +	if (lock->l_lvb_len > 0)
> +		req_capsule_extend(&req->rq_pill, &RQF_LDLM_ENQUEUE_LVB);
> +	req_capsule_set_size(&req->rq_pill, &RMF_DLM_LVB, RCL_SERVER,
> +			     lock->l_lvb_len);
> +	ptlrpc_request_set_replen(req);
> +	/* notify the server we've replayed all requests.
> +	 * also, we mark the request to be put on a dedicated
> +	 * queue to be processed after all request replayes.
> +	 * bug 6063
> +	 */
> +	lustre_msg_set_flags(req->rq_reqmsg, MSG_REQ_REPLAY_DONE);
> +
> +	LDLM_DEBUG(lock, "replaying lock:");
> +
> +	atomic_inc(&req->rq_import->imp_replay_inflight);
> +	BUILD_BUG_ON(sizeof(*aa) > sizeof(req->rq_async_args));
> +	aa = ptlrpc_req_async_args(req);
> +	aa->lock_handle = body->lock_handle[0];
> +	req->rq_interpret_reply = (ptlrpc_interpterer_t)replay_lock_interpret;
> +	ptlrpcd_add_req(req);
> +
> +	return 0;
> +}
> +
> +/**
> + * Cancel as many unused locks as possible before replay. since we are
> + * in recovery, we can't wait for any outstanding RPCs to send any RPC
> + * to the server.
> + *
> + * Called only in recovery before replaying locks. there is no need to
> + * replay locks that are unused. since the clients may hold thousands of
> + * cached unused locks, dropping the unused locks can greatly reduce the
> + * load on the servers at recovery time.
> + */
> +static void ldlm_cancel_unused_locks_for_replay(struct ldlm_namespace *ns)
> +{
> +	int canceled;
> +	LIST_HEAD(cancels);
> +
> +	CDEBUG(D_DLMTRACE,
> +	       "Dropping as many unused locks as possible before replay for namespace %s (%d)\n",
> +	       ldlm_ns_name(ns), ns->ns_nr_unused);
> +
> +	/* We don't need to care whether or not LRU resize is enabled
> +	 * because the LDLM_LRU_FLAG_NO_WAIT policy doesn't use the
> +	 * count parameter
> +	 */
> +	canceled = ldlm_cancel_lru_local(ns, &cancels, ns->ns_nr_unused, 0,
> +					 LCF_LOCAL, LDLM_LRU_FLAG_NO_WAIT);
> +
> +	CDEBUG(D_DLMTRACE, "Canceled %d unused locks from namespace %s\n",
> +	       canceled, ldlm_ns_name(ns));
> +}
> +
> +int ldlm_replay_locks(struct obd_import *imp)
> +{
> +	struct ldlm_namespace *ns = imp->imp_obd->obd_namespace;
> +	LIST_HEAD(list);
> +	struct ldlm_lock *lock, *next;
> +	int rc = 0;
> +
> +	LASSERT(atomic_read(&imp->imp_replay_inflight) == 0);
> +
> +	/* don't replay locks if import failed recovery */
> +	if (imp->imp_vbr_failed)
> +		return 0;
> +
> +	/* ensure this doesn't fall to 0 before all have been queued */
> +	atomic_inc(&imp->imp_replay_inflight);
> +
> +	if (ldlm_cancel_unused_locks_before_replay)
> +		ldlm_cancel_unused_locks_for_replay(ns);
> +
> +	ldlm_namespace_foreach(ns, ldlm_chain_lock_for_replay, &list);
> +
> +	list_for_each_entry_safe(lock, next, &list, l_pending_chain) {
> +		list_del_init(&lock->l_pending_chain);
> +		if (rc) {
> +			LDLM_LOCK_RELEASE(lock);
> +			continue; /* or try to do the rest? */
> +		}
> +		rc = replay_one_lock(imp, lock);
> +		LDLM_LOCK_RELEASE(lock);
> +	}
> +
> +	atomic_dec(&imp->imp_replay_inflight);
> +
> +	return rc;
> +}
> diff --git a/drivers/staging/lustre/lustre/ptlrpc/ldlm_resource.c b/drivers/staging/lustre/lustre/ptlrpc/ldlm_resource.c
> new file mode 100644
> index 000000000000..3946d62ff009
> --- /dev/null
> +++ b/drivers/staging/lustre/lustre/ptlrpc/ldlm_resource.c
> @@ -0,0 +1,1318 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * GPL HEADER START
> + *
> + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 only,
> + * as published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful, but
> + * WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> + * General Public License version 2 for more details (a copy is included
> + * in the LICENSE file that accompanied this code).
> + *
> + * You should have received a copy of the GNU General Public License
> + * version 2 along with this program; If not, see
> + * http://www.gnu.org/licenses/gpl-2.0.html
> + *
> + * GPL HEADER END
> + */
> +/*
> + * Copyright (c) 2002, 2010, Oracle and/or its affiliates. All rights reserved.
> + * Use is subject to license terms.
> + *
> + * Copyright (c) 2010, 2015, Intel Corporation.
> + */
> +/*
> + * This file is part of Lustre, http://www.lustre.org/
> + * Lustre is a trademark of Sun Microsystems, Inc.
> + *
> + * lustre/ldlm/ldlm_resource.c
> + *
> + * Author: Phil Schwan <phil at clusterfs.com>
> + * Author: Peter Braam <braam at clusterfs.com>
> + */
> +
> +#define DEBUG_SUBSYSTEM S_LDLM
> +#include <lustre_dlm.h>
> +#include <lustre_fid.h>
> +#include <obd_class.h>
> +#include "ldlm_internal.h"
> +#include <linux/libcfs/libcfs_hash.h>
> +
> +struct kmem_cache *ldlm_resource_slab, *ldlm_lock_slab;
> +
> +int ldlm_srv_namespace_nr;
> +int ldlm_cli_namespace_nr;
> +
> +struct mutex ldlm_srv_namespace_lock;
> +LIST_HEAD(ldlm_srv_namespace_list);
> +
> +struct mutex ldlm_cli_namespace_lock;
> +/* Client Namespaces that have active resources in them.
> + * Once all resources go away, ldlm_poold moves such namespaces to the
> + * inactive list
> + */
> +LIST_HEAD(ldlm_cli_active_namespace_list);
> +/* Client namespaces that don't have any locks in them */
> +static LIST_HEAD(ldlm_cli_inactive_namespace_list);
> +
> +static struct dentry *ldlm_debugfs_dir;
> +static struct dentry *ldlm_ns_debugfs_dir;
> +struct dentry *ldlm_svc_debugfs_dir;
> +
> +/* during debug dump certain amount of granted locks for one resource to avoid
> + * DDOS.
> + */
> +static unsigned int ldlm_dump_granted_max = 256;
> +
> +static ssize_t
> +lprocfs_wr_dump_ns(struct file *file, const char __user *buffer,
> +		   size_t count, loff_t *off)
> +{
> +	ldlm_dump_all_namespaces(LDLM_NAMESPACE_SERVER, D_DLMTRACE);
> +	ldlm_dump_all_namespaces(LDLM_NAMESPACE_CLIENT, D_DLMTRACE);
> +	return count;
> +}
> +
> +LPROC_SEQ_FOPS_WR_ONLY(ldlm, dump_ns);
> +
> +static int ldlm_rw_uint_seq_show(struct seq_file *m, void *v)
> +{
> +	seq_printf(m, "%u\n", *(unsigned int *)m->private);
> +	return 0;
> +}
> +
> +static ssize_t
> +ldlm_rw_uint_seq_write(struct file *file, const char __user *buffer,
> +		       size_t count, loff_t *off)
> +{
> +	struct seq_file *seq = file->private_data;
> +
> +	if (count == 0)
> +		return 0;
> +	return kstrtouint_from_user(buffer, count, 0,
> +				    (unsigned int *)seq->private);
> +}
> +
> +LPROC_SEQ_FOPS(ldlm_rw_uint);
> +
> +static struct lprocfs_vars ldlm_debugfs_list[] = {
> +	{ "dump_namespaces", &ldlm_dump_ns_fops, NULL, 0222 },
> +	{ "dump_granted_max", &ldlm_rw_uint_fops, &ldlm_dump_granted_max },
> +	{ NULL }
> +};
> +
> +void ldlm_debugfs_setup(void)
> +{
> +	ldlm_debugfs_dir = debugfs_create_dir(OBD_LDLM_DEVICENAME,
> +					      debugfs_lustre_root);
> +
> +	ldlm_ns_debugfs_dir = debugfs_create_dir("namespaces",
> +						 ldlm_debugfs_dir);
> +
> +	ldlm_svc_debugfs_dir = debugfs_create_dir("services", ldlm_debugfs_dir);
> +
> +	ldebugfs_add_vars(ldlm_debugfs_dir, ldlm_debugfs_list, NULL);
> +}
> +
> +void ldlm_debugfs_cleanup(void)
> +{
> +	debugfs_remove_recursive(ldlm_svc_debugfs_dir);
> +	debugfs_remove_recursive(ldlm_ns_debugfs_dir);
> +	debugfs_remove_recursive(ldlm_debugfs_dir);
> +}
> +
> +static ssize_t resource_count_show(struct kobject *kobj, struct attribute *attr,
> +				   char *buf)
> +{
> +	struct ldlm_namespace *ns = container_of(kobj, struct ldlm_namespace,
> +						 ns_kobj);
> +	__u64		  res = 0;
> +	struct cfs_hash_bd	  bd;
> +	int		    i;
> +
> +	/* result is not strictly consistent */
> +	cfs_hash_for_each_bucket(ns->ns_rs_hash, &bd, i)
> +		res += cfs_hash_bd_count_get(&bd);
> +	return sprintf(buf, "%lld\n", res);
> +}
> +LUSTRE_RO_ATTR(resource_count);
> +
> +static ssize_t lock_count_show(struct kobject *kobj, struct attribute *attr,
> +			       char *buf)
> +{
> +	struct ldlm_namespace *ns = container_of(kobj, struct ldlm_namespace,
> +						 ns_kobj);
> +	__u64		  locks;
> +
> +	locks = lprocfs_stats_collector(ns->ns_stats, LDLM_NSS_LOCKS,
> +					LPROCFS_FIELDS_FLAGS_SUM);
> +	return sprintf(buf, "%lld\n", locks);
> +}
> +LUSTRE_RO_ATTR(lock_count);
> +
> +static ssize_t lock_unused_count_show(struct kobject *kobj,
> +				      struct attribute *attr,
> +				      char *buf)
> +{
> +	struct ldlm_namespace *ns = container_of(kobj, struct ldlm_namespace,
> +						 ns_kobj);
> +
> +	return sprintf(buf, "%d\n", ns->ns_nr_unused);
> +}
> +LUSTRE_RO_ATTR(lock_unused_count);
> +
> +static ssize_t lru_size_show(struct kobject *kobj, struct attribute *attr,
> +			     char *buf)
> +{
> +	struct ldlm_namespace *ns = container_of(kobj, struct ldlm_namespace,
> +						 ns_kobj);
> +	__u32 *nr = &ns->ns_max_unused;
> +
> +	if (ns_connect_lru_resize(ns))
> +		nr = &ns->ns_nr_unused;
> +	return sprintf(buf, "%u\n", *nr);
> +}
> +
> +static ssize_t lru_size_store(struct kobject *kobj, struct attribute *attr,
> +			      const char *buffer, size_t count)
> +{
> +	struct ldlm_namespace *ns = container_of(kobj, struct ldlm_namespace,
> +						 ns_kobj);
> +	unsigned long tmp;
> +	int lru_resize;
> +	int err;
> +
> +	if (strncmp(buffer, "clear", 5) == 0) {
> +		CDEBUG(D_DLMTRACE,
> +		       "dropping all unused locks from namespace %s\n",
> +		       ldlm_ns_name(ns));
> +		if (ns_connect_lru_resize(ns)) {
> +			int canceled, unused  = ns->ns_nr_unused;
> +
> +			/* Try to cancel all @ns_nr_unused locks. */
> +			canceled = ldlm_cancel_lru(ns, unused, 0,
> +						   LDLM_LRU_FLAG_PASSED);
> +			if (canceled < unused) {
> +				CDEBUG(D_DLMTRACE,
> +				       "not all requested locks are canceled, requested: %d, canceled: %d\n",
> +				       unused,
> +				       canceled);
> +				return -EINVAL;
> +			}
> +		} else {
> +			tmp = ns->ns_max_unused;
> +			ns->ns_max_unused = 0;
> +			ldlm_cancel_lru(ns, 0, 0, LDLM_LRU_FLAG_PASSED);
> +			ns->ns_max_unused = tmp;
> +		}
> +		return count;
> +	}
> +
> +	err = kstrtoul(buffer, 10, &tmp);
> +	if (err != 0) {
> +		CERROR("lru_size: invalid value written\n");
> +		return -EINVAL;
> +	}
> +	lru_resize = (tmp == 0);
> +
> +	if (ns_connect_lru_resize(ns)) {
> +		if (!lru_resize)
> +			ns->ns_max_unused = (unsigned int)tmp;
> +
> +		if (tmp > ns->ns_nr_unused)
> +			tmp = ns->ns_nr_unused;
> +		tmp = ns->ns_nr_unused - tmp;
> +
> +		CDEBUG(D_DLMTRACE,
> +		       "changing namespace %s unused locks from %u to %u\n",
> +		       ldlm_ns_name(ns), ns->ns_nr_unused,
> +		       (unsigned int)tmp);
> +		ldlm_cancel_lru(ns, tmp, LCF_ASYNC, LDLM_LRU_FLAG_PASSED);
> +
> +		if (!lru_resize) {
> +			CDEBUG(D_DLMTRACE,
> +			       "disable lru_resize for namespace %s\n",
> +			       ldlm_ns_name(ns));
> +			ns->ns_connect_flags &= ~OBD_CONNECT_LRU_RESIZE;
> +		}
> +	} else {
> +		CDEBUG(D_DLMTRACE,
> +		       "changing namespace %s max_unused from %u to %u\n",
> +		       ldlm_ns_name(ns), ns->ns_max_unused,
> +		       (unsigned int)tmp);
> +		ns->ns_max_unused = (unsigned int)tmp;
> +		ldlm_cancel_lru(ns, 0, LCF_ASYNC, LDLM_LRU_FLAG_PASSED);
> +
> +		/* Make sure that LRU resize was originally supported before
> +		 * turning it on here.
> +		 */
> +		if (lru_resize &&
> +		    (ns->ns_orig_connect_flags & OBD_CONNECT_LRU_RESIZE)) {
> +			CDEBUG(D_DLMTRACE,
> +			       "enable lru_resize for namespace %s\n",
> +			       ldlm_ns_name(ns));
> +			ns->ns_connect_flags |= OBD_CONNECT_LRU_RESIZE;
> +		}
> +	}
> +
> +	return count;
> +}
> +LUSTRE_RW_ATTR(lru_size);
> +
> +static ssize_t lru_max_age_show(struct kobject *kobj, struct attribute *attr,
> +				char *buf)
> +{
> +	struct ldlm_namespace *ns = container_of(kobj, struct ldlm_namespace,
> +						 ns_kobj);
> +
> +	return sprintf(buf, "%u\n", ns->ns_max_age);
> +}
> +
> +static ssize_t lru_max_age_store(struct kobject *kobj, struct attribute *attr,
> +				 const char *buffer, size_t count)
> +{
> +	struct ldlm_namespace *ns = container_of(kobj, struct ldlm_namespace,
> +						 ns_kobj);
> +	unsigned long tmp;
> +	int err;
> +
> +	err = kstrtoul(buffer, 10, &tmp);
> +	if (err != 0)
> +		return -EINVAL;
> +
> +	ns->ns_max_age = tmp;
> +
> +	return count;
> +}
> +LUSTRE_RW_ATTR(lru_max_age);
> +
> +static ssize_t early_lock_cancel_show(struct kobject *kobj,
> +				      struct attribute *attr,
> +				      char *buf)
> +{
> +	struct ldlm_namespace *ns = container_of(kobj, struct ldlm_namespace,
> +						 ns_kobj);
> +
> +	return sprintf(buf, "%d\n", ns_connect_cancelset(ns));
> +}
> +
> +static ssize_t early_lock_cancel_store(struct kobject *kobj,
> +				       struct attribute *attr,
> +				       const char *buffer,
> +				       size_t count)
> +{
> +	struct ldlm_namespace *ns = container_of(kobj, struct ldlm_namespace,
> +						 ns_kobj);
> +	unsigned long supp = -1;
> +	int rc;
> +
> +	rc = kstrtoul(buffer, 10, &supp);
> +	if (rc < 0)
> +		return rc;
> +
> +	if (supp == 0)
> +		ns->ns_connect_flags &= ~OBD_CONNECT_CANCELSET;
> +	else if (ns->ns_orig_connect_flags & OBD_CONNECT_CANCELSET)
> +		ns->ns_connect_flags |= OBD_CONNECT_CANCELSET;
> +	return count;
> +}
> +LUSTRE_RW_ATTR(early_lock_cancel);
> +
> +/* These are for namespaces in /sys/fs/lustre/ldlm/namespaces/ */
> +static struct attribute *ldlm_ns_attrs[] = {
> +	&lustre_attr_resource_count.attr,
> +	&lustre_attr_lock_count.attr,
> +	&lustre_attr_lock_unused_count.attr,
> +	&lustre_attr_lru_size.attr,
> +	&lustre_attr_lru_max_age.attr,
> +	&lustre_attr_early_lock_cancel.attr,
> +	NULL,
> +};
> +
> +static void ldlm_ns_release(struct kobject *kobj)
> +{
> +	struct ldlm_namespace *ns = container_of(kobj, struct ldlm_namespace,
> +						 ns_kobj);
> +	complete(&ns->ns_kobj_unregister);
> +}
> +
> +static struct kobj_type ldlm_ns_ktype = {
> +	.default_attrs  = ldlm_ns_attrs,
> +	.sysfs_ops      = &lustre_sysfs_ops,
> +	.release        = ldlm_ns_release,
> +};
> +
> +static void ldlm_namespace_debugfs_unregister(struct ldlm_namespace *ns)
> +{
> +	debugfs_remove_recursive(ns->ns_debugfs_entry);
> +
> +	if (ns->ns_stats)
> +		lprocfs_free_stats(&ns->ns_stats);
> +}
> +
> +static void ldlm_namespace_sysfs_unregister(struct ldlm_namespace *ns)
> +{
> +	kobject_put(&ns->ns_kobj);
> +	wait_for_completion(&ns->ns_kobj_unregister);
> +}
> +
> +static int ldlm_namespace_sysfs_register(struct ldlm_namespace *ns)
> +{
> +	int err;
> +
> +	ns->ns_kobj.kset = ldlm_ns_kset;
> +	init_completion(&ns->ns_kobj_unregister);
> +	err = kobject_init_and_add(&ns->ns_kobj, &ldlm_ns_ktype, NULL,
> +				   "%s", ldlm_ns_name(ns));
> +
> +	ns->ns_stats = lprocfs_alloc_stats(LDLM_NSS_LAST, 0);
> +	if (!ns->ns_stats) {
> +		kobject_put(&ns->ns_kobj);
> +		return -ENOMEM;
> +	}
> +
> +	lprocfs_counter_init(ns->ns_stats, LDLM_NSS_LOCKS,
> +			     LPROCFS_CNTR_AVGMINMAX, "locks", "locks");
> +
> +	return err;
> +}
> +
> +static int ldlm_namespace_debugfs_register(struct ldlm_namespace *ns)
> +{
> +	struct dentry *ns_entry;
> +
> +	if (!IS_ERR_OR_NULL(ns->ns_debugfs_entry)) {
> +		ns_entry = ns->ns_debugfs_entry;
> +	} else {
> +		ns_entry = debugfs_create_dir(ldlm_ns_name(ns),
> +					      ldlm_ns_debugfs_dir);
> +		if (!ns_entry)
> +			return -ENOMEM;
> +		ns->ns_debugfs_entry = ns_entry;
> +	}
> +
> +	return 0;
> +}
> +
> +#undef MAX_STRING_SIZE
> +
> +static struct ldlm_resource *ldlm_resource_getref(struct ldlm_resource *res)
> +{
> +	LASSERT(res);
> +	LASSERT(res != LP_POISON);
> +	atomic_inc(&res->lr_refcount);
> +	CDEBUG(D_INFO, "getref res: %p count: %d\n", res,
> +	       atomic_read(&res->lr_refcount));
> +	return res;
> +}
> +
> +static unsigned int ldlm_res_hop_hash(struct cfs_hash *hs,
> +				      const void *key, unsigned int mask)
> +{
> +	const struct ldlm_res_id     *id  = key;
> +	unsigned int		val = 0;
> +	unsigned int		i;
> +
> +	for (i = 0; i < RES_NAME_SIZE; i++)
> +		val += id->name[i];
> +	return val & mask;
> +}
> +
> +static unsigned int ldlm_res_hop_fid_hash(struct cfs_hash *hs,
> +					  const void *key, unsigned int mask)
> +{
> +	const struct ldlm_res_id *id = key;
> +	struct lu_fid       fid;
> +	__u32	       hash;
> +	__u32	       val;
> +
> +	fid.f_seq = id->name[LUSTRE_RES_ID_SEQ_OFF];
> +	fid.f_oid = (__u32)id->name[LUSTRE_RES_ID_VER_OID_OFF];
> +	fid.f_ver = (__u32)(id->name[LUSTRE_RES_ID_VER_OID_OFF] >> 32);
> +
> +	hash = fid_flatten32(&fid);
> +	hash += (hash >> 4) + (hash << 12); /* mixing oid and seq */
> +	if (id->name[LUSTRE_RES_ID_HSH_OFF] != 0) {
> +		val = id->name[LUSTRE_RES_ID_HSH_OFF];
> +		hash += (val >> 5) + (val << 11);
> +	} else {
> +		val = fid_oid(&fid);
> +	}
> +	hash = hash_long(hash, hs->hs_bkt_bits);
> +	/* give me another random factor */
> +	hash -= hash_long((unsigned long)hs, val % 11 + 3);
> +
> +	hash <<= hs->hs_cur_bits - hs->hs_bkt_bits;
> +	hash |= ldlm_res_hop_hash(hs, key, CFS_HASH_NBKT(hs) - 1);
> +
> +	return hash & mask;
> +}
> +
> +static void *ldlm_res_hop_key(struct hlist_node *hnode)
> +{
> +	struct ldlm_resource   *res;
> +
> +	res = hlist_entry(hnode, struct ldlm_resource, lr_hash);
> +	return &res->lr_name;
> +}
> +
> +static int ldlm_res_hop_keycmp(const void *key, struct hlist_node *hnode)
> +{
> +	struct ldlm_resource   *res;
> +
> +	res = hlist_entry(hnode, struct ldlm_resource, lr_hash);
> +	return ldlm_res_eq((const struct ldlm_res_id *)key,
> +			   (const struct ldlm_res_id *)&res->lr_name);
> +}
> +
> +static void *ldlm_res_hop_object(struct hlist_node *hnode)
> +{
> +	return hlist_entry(hnode, struct ldlm_resource, lr_hash);
> +}
> +
> +static void ldlm_res_hop_get_locked(struct cfs_hash *hs,
> +				    struct hlist_node *hnode)
> +{
> +	struct ldlm_resource *res;
> +
> +	res = hlist_entry(hnode, struct ldlm_resource, lr_hash);
> +	ldlm_resource_getref(res);
> +}
> +
> +static void ldlm_res_hop_put(struct cfs_hash *hs, struct hlist_node *hnode)
> +{
> +	struct ldlm_resource *res;
> +
> +	res = hlist_entry(hnode, struct ldlm_resource, lr_hash);
> +	ldlm_resource_putref(res);
> +}
> +
> +static struct cfs_hash_ops ldlm_ns_hash_ops = {
> +	.hs_hash	= ldlm_res_hop_hash,
> +	.hs_key		= ldlm_res_hop_key,
> +	.hs_keycmp      = ldlm_res_hop_keycmp,
> +	.hs_keycpy      = NULL,
> +	.hs_object      = ldlm_res_hop_object,
> +	.hs_get		= ldlm_res_hop_get_locked,
> +	.hs_put		= ldlm_res_hop_put
> +};
> +
> +static struct cfs_hash_ops ldlm_ns_fid_hash_ops = {
> +	.hs_hash	= ldlm_res_hop_fid_hash,
> +	.hs_key		= ldlm_res_hop_key,
> +	.hs_keycmp      = ldlm_res_hop_keycmp,
> +	.hs_keycpy      = NULL,
> +	.hs_object      = ldlm_res_hop_object,
> +	.hs_get		= ldlm_res_hop_get_locked,
> +	.hs_put		= ldlm_res_hop_put
> +};
> +
> +struct ldlm_ns_hash_def {
> +	enum ldlm_ns_type nsd_type;
> +	/** hash bucket bits */
> +	unsigned int	nsd_bkt_bits;
> +	/** hash bits */
> +	unsigned int	nsd_all_bits;
> +	/** hash operations */
> +	struct cfs_hash_ops *nsd_hops;
> +};
> +
> +static struct ldlm_ns_hash_def ldlm_ns_hash_defs[] = {
> +	{
> +		.nsd_type       = LDLM_NS_TYPE_MDC,
> +		.nsd_bkt_bits   = 11,
> +		.nsd_all_bits   = 16,
> +		.nsd_hops       = &ldlm_ns_fid_hash_ops,
> +	},
> +	{
> +		.nsd_type       = LDLM_NS_TYPE_MDT,
> +		.nsd_bkt_bits   = 14,
> +		.nsd_all_bits   = 21,
> +		.nsd_hops       = &ldlm_ns_fid_hash_ops,
> +	},
> +	{
> +		.nsd_type       = LDLM_NS_TYPE_OSC,
> +		.nsd_bkt_bits   = 8,
> +		.nsd_all_bits   = 12,
> +		.nsd_hops       = &ldlm_ns_hash_ops,
> +	},
> +	{
> +		.nsd_type       = LDLM_NS_TYPE_OST,
> +		.nsd_bkt_bits   = 11,
> +		.nsd_all_bits   = 17,
> +		.nsd_hops       = &ldlm_ns_hash_ops,
> +	},
> +	{
> +		.nsd_type       = LDLM_NS_TYPE_MGC,
> +		.nsd_bkt_bits   = 4,
> +		.nsd_all_bits   = 4,
> +		.nsd_hops       = &ldlm_ns_hash_ops,
> +	},
> +	{
> +		.nsd_type       = LDLM_NS_TYPE_MGT,
> +		.nsd_bkt_bits   = 4,
> +		.nsd_all_bits   = 4,
> +		.nsd_hops       = &ldlm_ns_hash_ops,
> +	},
> +	{
> +		.nsd_type       = LDLM_NS_TYPE_UNKNOWN,
> +	},
> +};
> +
> +/** Register \a ns in the list of namespaces */
> +static void ldlm_namespace_register(struct ldlm_namespace *ns,
> +				    enum ldlm_side client)
> +{
> +	mutex_lock(ldlm_namespace_lock(client));
> +	LASSERT(list_empty(&ns->ns_list_chain));
> +	list_add(&ns->ns_list_chain, &ldlm_cli_inactive_namespace_list);
> +	ldlm_namespace_nr_inc(client);
> +	mutex_unlock(ldlm_namespace_lock(client));
> +}
> +
> +/**
> + * Create and initialize new empty namespace.
> + */
> +struct ldlm_namespace *ldlm_namespace_new(struct obd_device *obd, char *name,
> +					  enum ldlm_side client,
> +					  enum ldlm_appetite apt,
> +					  enum ldlm_ns_type ns_type)
> +{
> +	struct ldlm_namespace *ns = NULL;
> +	struct ldlm_ns_bucket *nsb;
> +	struct ldlm_ns_hash_def    *nsd;
> +	struct cfs_hash_bd	  bd;
> +	int		    idx;
> +	int		    rc;
> +
> +	LASSERT(obd);
> +
> +	rc = ldlm_get_ref();
> +	if (rc) {
> +		CERROR("ldlm_get_ref failed: %d\n", rc);
> +		return NULL;
> +	}
> +
> +	for (idx = 0;; idx++) {
> +		nsd = &ldlm_ns_hash_defs[idx];
> +		if (nsd->nsd_type == LDLM_NS_TYPE_UNKNOWN) {
> +			CERROR("Unknown type %d for ns %s\n", ns_type, name);
> +			goto out_ref;
> +		}
> +
> +		if (nsd->nsd_type == ns_type)
> +			break;
> +	}
> +
> +	ns = kzalloc(sizeof(*ns), GFP_NOFS);
> +	if (!ns)
> +		goto out_ref;
> +
> +	ns->ns_rs_hash = cfs_hash_create(name,
> +					 nsd->nsd_all_bits, nsd->nsd_all_bits,
> +					 nsd->nsd_bkt_bits, sizeof(*nsb),
> +					 CFS_HASH_MIN_THETA,
> +					 CFS_HASH_MAX_THETA,
> +					 nsd->nsd_hops,
> +					 CFS_HASH_DEPTH |
> +					 CFS_HASH_BIGNAME |
> +					 CFS_HASH_SPIN_BKTLOCK |
> +					 CFS_HASH_NO_ITEMREF);
> +	if (!ns->ns_rs_hash)
> +		goto out_ns;
> +
> +	cfs_hash_for_each_bucket(ns->ns_rs_hash, &bd, idx) {
> +		nsb = cfs_hash_bd_extra_get(ns->ns_rs_hash, &bd);
> +		at_init(&nsb->nsb_at_estimate, ldlm_enqueue_min, 0);
> +		nsb->nsb_namespace = ns;
> +	}
> +
> +	ns->ns_obd      = obd;
> +	ns->ns_appetite = apt;
> +	ns->ns_client   = client;
> +	ns->ns_name     = kstrdup(name, GFP_KERNEL);
> +	if (!ns->ns_name)
> +		goto out_hash;
> +
> +	INIT_LIST_HEAD(&ns->ns_list_chain);
> +	INIT_LIST_HEAD(&ns->ns_unused_list);
> +	spin_lock_init(&ns->ns_lock);
> +	atomic_set(&ns->ns_bref, 0);
> +	init_waitqueue_head(&ns->ns_waitq);
> +
> +	ns->ns_max_parallel_ast   = LDLM_DEFAULT_PARALLEL_AST_LIMIT;
> +	ns->ns_nr_unused	  = 0;
> +	ns->ns_max_unused	 = LDLM_DEFAULT_LRU_SIZE;
> +	ns->ns_max_age	    = LDLM_DEFAULT_MAX_ALIVE;
> +	ns->ns_orig_connect_flags = 0;
> +	ns->ns_connect_flags      = 0;
> +	ns->ns_stopping	   = 0;
> +
> +	rc = ldlm_namespace_sysfs_register(ns);
> +	if (rc != 0) {
> +		CERROR("Can't initialize ns sysfs, rc %d\n", rc);
> +		goto out_hash;
> +	}
> +
> +	rc = ldlm_namespace_debugfs_register(ns);
> +	if (rc != 0) {
> +		CERROR("Can't initialize ns proc, rc %d\n", rc);
> +		goto out_sysfs;
> +	}
> +
> +	idx = ldlm_namespace_nr_read(client);
> +	rc = ldlm_pool_init(&ns->ns_pool, ns, idx, client);
> +	if (rc) {
> +		CERROR("Can't initialize lock pool, rc %d\n", rc);
> +		goto out_proc;
> +	}
> +
> +	ldlm_namespace_register(ns, client);
> +	return ns;
> +out_proc:
> +	ldlm_namespace_debugfs_unregister(ns);
> +out_sysfs:
> +	ldlm_namespace_sysfs_unregister(ns);
> +	ldlm_namespace_cleanup(ns, 0);
> +out_hash:
> +	kfree(ns->ns_name);
> +	cfs_hash_putref(ns->ns_rs_hash);
> +out_ns:
> +	kfree(ns);
> +out_ref:
> +	ldlm_put_ref();
> +	return NULL;
> +}
> +EXPORT_SYMBOL(ldlm_namespace_new);
> +
> +extern struct ldlm_lock *ldlm_lock_get(struct ldlm_lock *lock);
> +
> +/**
> + * Cancel and destroy all locks on a resource.
> + *
> + * If flags contains FL_LOCAL_ONLY, don't try to tell the server, just
> + * clean up.  This is currently only used for recovery, and we make
> + * certain assumptions as a result--notably, that we shouldn't cancel
> + * locks with refs.
> + */
> +static void cleanup_resource(struct ldlm_resource *res, struct list_head *q,
> +			     __u64 flags)
> +{
> +	int rc = 0;
> +	bool local_only = !!(flags & LDLM_FL_LOCAL_ONLY);
> +
> +	do {
> +		struct ldlm_lock *lock = NULL, *tmp;
> +		struct lustre_handle lockh;
> +
> +		/* First, we look for non-cleaned-yet lock
> +		 * all cleaned locks are marked by CLEANED flag.
> +		 */
> +		lock_res(res);
> +		list_for_each_entry(tmp, q, l_res_link) {
> +			if (ldlm_is_cleaned(tmp))
> +				continue;
> +
> +			lock = tmp;
> +			LDLM_LOCK_GET(lock);
> +			ldlm_set_cleaned(lock);
> +			break;
> +		}
> +
> +		if (!lock) {
> +			unlock_res(res);
> +			break;
> +		}
> +
> +		/* Set CBPENDING so nothing in the cancellation path
> +		 * can match this lock.
> +		 */
> +		ldlm_set_cbpending(lock);
> +		ldlm_set_failed(lock);
> +		lock->l_flags |= flags;
> +
> +		/* ... without sending a CANCEL message for local_only. */
> +		if (local_only)
> +			ldlm_set_local_only(lock);
> +
> +		if (local_only && (lock->l_readers || lock->l_writers)) {
> +			/* This is a little bit gross, but much better than the
> +			 * alternative: pretend that we got a blocking AST from
> +			 * the server, so that when the lock is decref'd, it
> +			 * will go away ...
> +			 */
> +			unlock_res(res);
> +			LDLM_DEBUG(lock, "setting FL_LOCAL_ONLY");
> +			if (lock->l_flags & LDLM_FL_FAIL_LOC) {
> +				set_current_state(TASK_UNINTERRUPTIBLE);
> +				schedule_timeout(4 * HZ);
> +				set_current_state(TASK_RUNNING);
> +			}
> +			if (lock->l_completion_ast)
> +				lock->l_completion_ast(lock, LDLM_FL_FAILED,
> +						       NULL);
> +			LDLM_LOCK_RELEASE(lock);
> +			continue;
> +		}
> +
> +		unlock_res(res);
> +		ldlm_lock2handle(lock, &lockh);
> +		rc = ldlm_cli_cancel(&lockh, LCF_LOCAL);
> +		if (rc)
> +			CERROR("ldlm_cli_cancel: %d\n", rc);
> +		LDLM_LOCK_RELEASE(lock);
> +	} while (1);
> +}
> +
> +static int ldlm_resource_clean(struct cfs_hash *hs, struct cfs_hash_bd *bd,
> +			       struct hlist_node *hnode, void *arg)
> +{
> +	struct ldlm_resource *res = cfs_hash_object(hs, hnode);
> +	__u64 flags = *(__u64 *)arg;
> +
> +	cleanup_resource(res, &res->lr_granted, flags);
> +	cleanup_resource(res, &res->lr_waiting, flags);
> +
> +	return 0;
> +}
> +
> +static int ldlm_resource_complain(struct cfs_hash *hs, struct cfs_hash_bd *bd,
> +				  struct hlist_node *hnode, void *arg)
> +{
> +	struct ldlm_resource  *res = cfs_hash_object(hs, hnode);
> +
> +	lock_res(res);
> +	CERROR("%s: namespace resource " DLDLMRES
> +	       " (%p) refcount nonzero (%d) after lock cleanup; forcing cleanup.\n",
> +	       ldlm_ns_name(ldlm_res_to_ns(res)), PLDLMRES(res), res,
> +	       atomic_read(&res->lr_refcount) - 1);
> +
> +	ldlm_resource_dump(D_ERROR, res);
> +	unlock_res(res);
> +	return 0;
> +}
> +
> +/**
> + * Cancel and destroy all locks in the namespace.
> + *
> + * Typically used during evictions when server notified client that it was
> + * evicted and all of its state needs to be destroyed.
> + * Also used during shutdown.
> + */
> +int ldlm_namespace_cleanup(struct ldlm_namespace *ns, __u64 flags)
> +{
> +	if (!ns) {
> +		CDEBUG(D_INFO, "NULL ns, skipping cleanup\n");
> +		return ELDLM_OK;
> +	}
> +
> +	cfs_hash_for_each_nolock(ns->ns_rs_hash, ldlm_resource_clean,
> +				 &flags, 0);
> +	cfs_hash_for_each_nolock(ns->ns_rs_hash, ldlm_resource_complain,
> +				 NULL, 0);
> +	return ELDLM_OK;
> +}
> +EXPORT_SYMBOL(ldlm_namespace_cleanup);
> +
> +/**
> + * Attempts to free namespace.
> + *
> + * Only used when namespace goes away, like during an unmount.
> + */
> +static int __ldlm_namespace_free(struct ldlm_namespace *ns, int force)
> +{
> +	/* At shutdown time, don't call the cancellation callback */
> +	ldlm_namespace_cleanup(ns, force ? LDLM_FL_LOCAL_ONLY : 0);
> +
> +	if (atomic_read(&ns->ns_bref) > 0) {
> +		int rc;
> +
> +		CDEBUG(D_DLMTRACE,
> +		       "dlm namespace %s free waiting on refcount %d\n",
> +		       ldlm_ns_name(ns), atomic_read(&ns->ns_bref));
> +force_wait:
> +		if (force)
> +			rc = wait_event_idle_timeout(ns->ns_waitq,
> +						     atomic_read(&ns->ns_bref) == 0,
> +						     obd_timeout * HZ / 4) ? 0 : -ETIMEDOUT;
> +		else
> +			rc = l_wait_event_abortable(ns->ns_waitq,
> +						    atomic_read(&ns->ns_bref) == 0);
> +
> +		/* Forced cleanups should be able to reclaim all references,
> +		 * so it's safe to wait forever... we can't leak locks...
> +		 */
> +		if (force && rc == -ETIMEDOUT) {
> +			LCONSOLE_ERROR("Forced cleanup waiting for %s namespace with %d resources in use, (rc=%d)\n",
> +				       ldlm_ns_name(ns),
> +				       atomic_read(&ns->ns_bref), rc);
> +			goto force_wait;
> +		}
> +
> +		if (atomic_read(&ns->ns_bref)) {
> +			LCONSOLE_ERROR("Cleanup waiting for %s namespace with %d resources in use, (rc=%d)\n",
> +				       ldlm_ns_name(ns),
> +				       atomic_read(&ns->ns_bref), rc);
> +			return ELDLM_NAMESPACE_EXISTS;
> +		}
> +		CDEBUG(D_DLMTRACE, "dlm namespace %s free done waiting\n",
> +		       ldlm_ns_name(ns));
> +	}
> +
> +	return ELDLM_OK;
> +}
> +
> +/**
> + * Performs various cleanups for passed \a ns to make it drop refc and be
> + * ready for freeing. Waits for refc == 0.
> + *
> + * The following is done:
> + * (0) Unregister \a ns from its list to make inaccessible for potential
> + * users like pools thread and others;
> + * (1) Clear all locks in \a ns.
> + */
> +void ldlm_namespace_free_prior(struct ldlm_namespace *ns,
> +			       struct obd_import *imp,
> +			       int force)
> +{
> +	int rc;
> +
> +	if (!ns)
> +		return;
> +
> +	spin_lock(&ns->ns_lock);
> +	ns->ns_stopping = 1;
> +	spin_unlock(&ns->ns_lock);
> +
> +	/*
> +	 * Can fail with -EINTR when force == 0 in which case try harder.
> +	 */
> +	rc = __ldlm_namespace_free(ns, force);
> +	if (rc != ELDLM_OK) {
> +		if (imp) {
> +			ptlrpc_disconnect_import(imp, 0);
> +			ptlrpc_invalidate_import(imp);
> +		}
> +
> +		/*
> +		 * With all requests dropped and the import inactive
> +		 * we are guaranteed all reference will be dropped.
> +		 */
> +		rc = __ldlm_namespace_free(ns, 1);
> +		LASSERT(rc == 0);
> +	}
> +}
> +
> +/** Unregister \a ns from the list of namespaces. */
> +static void ldlm_namespace_unregister(struct ldlm_namespace *ns,
> +				      enum ldlm_side client)
> +{
> +	mutex_lock(ldlm_namespace_lock(client));
> +	LASSERT(!list_empty(&ns->ns_list_chain));
> +	/* Some asserts and possibly other parts of the code are still
> +	 * using list_empty(&ns->ns_list_chain). This is why it is
> +	 * important to use list_del_init() here.
> +	 */
> +	list_del_init(&ns->ns_list_chain);
> +	ldlm_namespace_nr_dec(client);
> +	mutex_unlock(ldlm_namespace_lock(client));
> +}
> +
> +/**
> + * Performs freeing memory structures related to \a ns. This is only done
> + * when ldlm_namespce_free_prior() successfully removed all resources
> + * referencing \a ns and its refc == 0.
> + */
> +void ldlm_namespace_free_post(struct ldlm_namespace *ns)
> +{
> +	if (!ns)
> +		return;
> +
> +	/* Make sure that nobody can find this ns in its list. */
> +	ldlm_namespace_unregister(ns, ns->ns_client);
> +	/* Fini pool _before_ parent proc dir is removed. This is important as
> +	 * ldlm_pool_fini() removes own proc dir which is child to @dir.
> +	 * Removing it after @dir may cause oops.
> +	 */
> +	ldlm_pool_fini(&ns->ns_pool);
> +
> +	ldlm_namespace_debugfs_unregister(ns);
> +	ldlm_namespace_sysfs_unregister(ns);
> +	cfs_hash_putref(ns->ns_rs_hash);
> +	kfree(ns->ns_name);
> +	/* Namespace \a ns should be not on list at this time, otherwise
> +	 * this will cause issues related to using freed \a ns in poold
> +	 * thread.
> +	 */
> +	LASSERT(list_empty(&ns->ns_list_chain));
> +	kfree(ns);
> +	ldlm_put_ref();
> +}
> +
> +void ldlm_namespace_get(struct ldlm_namespace *ns)
> +{
> +	atomic_inc(&ns->ns_bref);
> +}
> +
> +/* This is only for callers that care about refcount */
> +static int ldlm_namespace_get_return(struct ldlm_namespace *ns)
> +{
> +	return atomic_inc_return(&ns->ns_bref);
> +}
> +
> +void ldlm_namespace_put(struct ldlm_namespace *ns)
> +{
> +	if (atomic_dec_and_lock(&ns->ns_bref, &ns->ns_lock)) {
> +		wake_up(&ns->ns_waitq);
> +		spin_unlock(&ns->ns_lock);
> +	}
> +}
> +
> +/** Should be called with ldlm_namespace_lock(client) taken. */
> +void ldlm_namespace_move_to_active_locked(struct ldlm_namespace *ns,
> +					  enum ldlm_side client)
> +{
> +	LASSERT(!list_empty(&ns->ns_list_chain));
> +	LASSERT(mutex_is_locked(ldlm_namespace_lock(client)));
> +	list_move_tail(&ns->ns_list_chain, ldlm_namespace_list(client));
> +}
> +
> +/** Should be called with ldlm_namespace_lock(client) taken. */
> +void ldlm_namespace_move_to_inactive_locked(struct ldlm_namespace *ns,
> +					    enum ldlm_side client)
> +{
> +	LASSERT(!list_empty(&ns->ns_list_chain));
> +	LASSERT(mutex_is_locked(ldlm_namespace_lock(client)));
> +	list_move_tail(&ns->ns_list_chain, &ldlm_cli_inactive_namespace_list);
> +}
> +
> +/** Should be called with ldlm_namespace_lock(client) taken. */
> +struct ldlm_namespace *ldlm_namespace_first_locked(enum ldlm_side client)
> +{
> +	LASSERT(mutex_is_locked(ldlm_namespace_lock(client)));
> +	LASSERT(!list_empty(ldlm_namespace_list(client)));
> +	return container_of(ldlm_namespace_list(client)->next,
> +		struct ldlm_namespace, ns_list_chain);
> +}
> +
> +/** Create and initialize new resource. */
> +static struct ldlm_resource *ldlm_resource_new(void)
> +{
> +	struct ldlm_resource *res;
> +	int idx;
> +
> +	res = kmem_cache_zalloc(ldlm_resource_slab, GFP_NOFS);
> +	if (!res)
> +		return NULL;
> +
> +	INIT_LIST_HEAD(&res->lr_granted);
> +	INIT_LIST_HEAD(&res->lr_waiting);
> +
> +	/* Initialize interval trees for each lock mode. */
> +	for (idx = 0; idx < LCK_MODE_NUM; idx++) {
> +		res->lr_itree[idx].lit_size = 0;
> +		res->lr_itree[idx].lit_mode = 1 << idx;
> +		res->lr_itree[idx].lit_root = RB_ROOT_CACHED;
> +	}
> +
> +	atomic_set(&res->lr_refcount, 1);
> +	spin_lock_init(&res->lr_lock);
> +	lu_ref_init(&res->lr_reference);
> +
> +	/* The creator of the resource must unlock the mutex after LVB
> +	 * initialization.
> +	 */
> +	mutex_init(&res->lr_lvb_mutex);
> +	mutex_lock(&res->lr_lvb_mutex);
> +
> +	return res;
> +}
> +
> +/**
> + * Return a reference to resource with given name, creating it if necessary.
> + * Args: namespace with ns_lock unlocked
> + * Locks: takes and releases NS hash-lock and res->lr_lock
> + * Returns: referenced, unlocked ldlm_resource or NULL
> + */
> +struct ldlm_resource *
> +ldlm_resource_get(struct ldlm_namespace *ns, struct ldlm_resource *parent,
> +		  const struct ldlm_res_id *name, enum ldlm_type type,
> +		  int create)
> +{
> +	struct hlist_node     *hnode;
> +	struct ldlm_resource *res = NULL;
> +	struct cfs_hash_bd	 bd;
> +	__u64		 version;
> +	int		      ns_refcount = 0;
> +	int rc;
> +
> +	LASSERT(!parent);
> +	LASSERT(ns->ns_rs_hash);
> +	LASSERT(name->name[0] != 0);
> +
> +	cfs_hash_bd_get_and_lock(ns->ns_rs_hash, (void *)name, &bd, 0);
> +	hnode = cfs_hash_bd_lookup_locked(ns->ns_rs_hash, &bd, (void *)name);
> +	if (hnode) {
> +		cfs_hash_bd_unlock(ns->ns_rs_hash, &bd, 0);
> +		goto lvbo_init;
> +	}
> +
> +	version = cfs_hash_bd_version_get(&bd);
> +	cfs_hash_bd_unlock(ns->ns_rs_hash, &bd, 0);
> +
> +	if (create == 0)
> +		return ERR_PTR(-ENOENT);
> +
> +	LASSERTF(type >= LDLM_MIN_TYPE && type < LDLM_MAX_TYPE,
> +		 "type: %d\n", type);
> +	res = ldlm_resource_new();
> +	if (!res)
> +		return ERR_PTR(-ENOMEM);
> +
> +	res->lr_ns_bucket  = cfs_hash_bd_extra_get(ns->ns_rs_hash, &bd);
> +	res->lr_name       = *name;
> +	res->lr_type       = type;
> +
> +	cfs_hash_bd_lock(ns->ns_rs_hash, &bd, 1);
> +	hnode = (version == cfs_hash_bd_version_get(&bd)) ?  NULL :
> +		cfs_hash_bd_lookup_locked(ns->ns_rs_hash, &bd, (void *)name);
> +
> +	if (hnode) {
> +		/* Someone won the race and already added the resource. */
> +		cfs_hash_bd_unlock(ns->ns_rs_hash, &bd, 1);
> +		/* Clean lu_ref for failed resource. */
> +		lu_ref_fini(&res->lr_reference);
> +		/* We have taken lr_lvb_mutex. Drop it. */
> +		mutex_unlock(&res->lr_lvb_mutex);
> +		kmem_cache_free(ldlm_resource_slab, res);
> +lvbo_init:
> +		res = hlist_entry(hnode, struct ldlm_resource, lr_hash);
> +		/* Synchronize with regard to resource creation. */
> +		if (ns->ns_lvbo && ns->ns_lvbo->lvbo_init) {
> +			mutex_lock(&res->lr_lvb_mutex);
> +			mutex_unlock(&res->lr_lvb_mutex);
> +		}
> +
> +		if (unlikely(res->lr_lvb_len < 0)) {
> +			rc = res->lr_lvb_len;
> +			ldlm_resource_putref(res);
> +			res = ERR_PTR(rc);
> +		}
> +		return res;
> +	}
> +	/* We won! Let's add the resource. */
> +	cfs_hash_bd_add_locked(ns->ns_rs_hash, &bd, &res->lr_hash);
> +	if (cfs_hash_bd_count_get(&bd) == 1)
> +		ns_refcount = ldlm_namespace_get_return(ns);
> +
> +	cfs_hash_bd_unlock(ns->ns_rs_hash, &bd, 1);
> +	if (ns->ns_lvbo && ns->ns_lvbo->lvbo_init) {
> +		OBD_FAIL_TIMEOUT(OBD_FAIL_LDLM_CREATE_RESOURCE, 2);
> +		rc = ns->ns_lvbo->lvbo_init(res);
> +		if (rc < 0) {
> +			CERROR("%s: lvbo_init failed for resource %#llx:%#llx: rc = %d\n",
> +			       ns->ns_obd->obd_name, name->name[0],
> +			       name->name[1], rc);
> +			res->lr_lvb_len = rc;
> +			mutex_unlock(&res->lr_lvb_mutex);
> +			ldlm_resource_putref(res);
> +			return ERR_PTR(rc);
> +		}
> +	}
> +
> +	/* We create resource with locked lr_lvb_mutex. */
> +	mutex_unlock(&res->lr_lvb_mutex);
> +
> +	/* Let's see if we happened to be the very first resource in this
> +	 * namespace. If so, and this is a client namespace, we need to move
> +	 * the namespace into the active namespaces list to be patrolled by
> +	 * the ldlm_poold.
> +	 */
> +	if (ns_refcount == 1) {
> +		mutex_lock(ldlm_namespace_lock(LDLM_NAMESPACE_CLIENT));
> +		ldlm_namespace_move_to_active_locked(ns, LDLM_NAMESPACE_CLIENT);
> +		mutex_unlock(ldlm_namespace_lock(LDLM_NAMESPACE_CLIENT));
> +	}
> +
> +	return res;
> +}
> +EXPORT_SYMBOL(ldlm_resource_get);
> +
> +static void __ldlm_resource_putref_final(struct cfs_hash_bd *bd,
> +					 struct ldlm_resource *res)
> +{
> +	struct ldlm_ns_bucket *nsb = res->lr_ns_bucket;
> +	struct ldlm_namespace *ns = nsb->nsb_namespace;
> +
> +	if (!list_empty(&res->lr_granted)) {
> +		ldlm_resource_dump(D_ERROR, res);
> +		LBUG();
> +	}
> +
> +	if (!list_empty(&res->lr_waiting)) {
> +		ldlm_resource_dump(D_ERROR, res);
> +		LBUG();
> +	}
> +
> +	cfs_hash_bd_del_locked(ns->ns_rs_hash,
> +			       bd, &res->lr_hash);
> +	lu_ref_fini(&res->lr_reference);
> +	cfs_hash_bd_unlock(ns->ns_rs_hash, bd, 1);
> +	if (ns->ns_lvbo && ns->ns_lvbo->lvbo_free)
> +		ns->ns_lvbo->lvbo_free(res);
> +	if (cfs_hash_bd_count_get(bd) == 0)
> +		ldlm_namespace_put(ns);
> +	kmem_cache_free(ldlm_resource_slab, res);
> +}
> +
> +void ldlm_resource_putref(struct ldlm_resource *res)
> +{
> +	struct ldlm_namespace *ns = ldlm_res_to_ns(res);
> +	struct cfs_hash_bd   bd;
> +
> +	LASSERT_ATOMIC_GT_LT(&res->lr_refcount, 0, LI_POISON);
> +	CDEBUG(D_INFO, "putref res: %p count: %d\n",
> +	       res, atomic_read(&res->lr_refcount) - 1);
> +
> +	cfs_hash_bd_get(ns->ns_rs_hash, &res->lr_name, &bd);
> +	if (cfs_hash_bd_dec_and_lock(ns->ns_rs_hash, &bd, &res->lr_refcount))
> +		__ldlm_resource_putref_final(&bd, res);
> +}
> +EXPORT_SYMBOL(ldlm_resource_putref);
> +
> +/**
> + * Add a lock into a given resource into specified lock list.
> + */
> +void ldlm_resource_add_lock(struct ldlm_resource *res, struct list_head *head,
> +			    struct ldlm_lock *lock)
> +{
> +	check_res_locked(res);
> +
> +	LDLM_DEBUG(lock, "About to add this lock:");
> +
> +	if (ldlm_is_destroyed(lock)) {
> +		CDEBUG(D_OTHER, "Lock destroyed, not adding to resource\n");
> +		return;
> +	}
> +
> +	LASSERT(list_empty(&lock->l_res_link));
> +
> +	list_add_tail(&lock->l_res_link, head);
> +}
> +
> +void ldlm_resource_unlink_lock(struct ldlm_lock *lock)
> +{
> +	int type = lock->l_resource->lr_type;
> +
> +	check_res_locked(lock->l_resource);
> +	if (type == LDLM_IBITS || type == LDLM_PLAIN)
> +		ldlm_unlink_lock_skiplist(lock);
> +	else if (type == LDLM_EXTENT)
> +		ldlm_extent_unlink_lock(lock);
> +	list_del_init(&lock->l_res_link);
> +}
> +EXPORT_SYMBOL(ldlm_resource_unlink_lock);
> +
> +void ldlm_res2desc(struct ldlm_resource *res, struct ldlm_resource_desc *desc)
> +{
> +	desc->lr_type = res->lr_type;
> +	desc->lr_name = res->lr_name;
> +}
> +
> +/**
> + * Print information about all locks in all namespaces on this node to debug
> + * log.
> + */
> +void ldlm_dump_all_namespaces(enum ldlm_side client, int level)
> +{
> +	struct ldlm_namespace *ns;
> +
> +	if (!((libcfs_debug | D_ERROR) & level))
> +		return;
> +
> +	mutex_lock(ldlm_namespace_lock(client));
> +
> +	list_for_each_entry(ns, ldlm_namespace_list(client), ns_list_chain)
> +		ldlm_namespace_dump(level, ns);
> +
> +	mutex_unlock(ldlm_namespace_lock(client));
> +}
> +
> +static int ldlm_res_hash_dump(struct cfs_hash *hs, struct cfs_hash_bd *bd,
> +			      struct hlist_node *hnode, void *arg)
> +{
> +	struct ldlm_resource *res = cfs_hash_object(hs, hnode);
> +	int    level = (int)(unsigned long)arg;
> +
> +	lock_res(res);
> +	ldlm_resource_dump(level, res);
> +	unlock_res(res);
> +
> +	return 0;
> +}
> +
> +/**
> + * Print information about all locks in this namespace on this node to debug
> + * log.
> + */
> +void ldlm_namespace_dump(int level, struct ldlm_namespace *ns)
> +{
> +	if (!((libcfs_debug | D_ERROR) & level))
> +		return;
> +
> +	CDEBUG(level, "--- Namespace: %s (rc: %d, side: client)\n",
> +	       ldlm_ns_name(ns), atomic_read(&ns->ns_bref));
> +
> +	if (time_before(jiffies, ns->ns_next_dump))
> +		return;
> +
> +	cfs_hash_for_each_nolock(ns->ns_rs_hash,
> +				 ldlm_res_hash_dump,
> +				 (void *)(unsigned long)level, 0);
> +	spin_lock(&ns->ns_lock);
> +	ns->ns_next_dump = jiffies + 10 * HZ;
> +	spin_unlock(&ns->ns_lock);
> +}
> +
> +/**
> + * Print information about all locks in this resource to debug log.
> + */
> +void ldlm_resource_dump(int level, struct ldlm_resource *res)
> +{
> +	struct ldlm_lock *lock;
> +	unsigned int granted = 0;
> +
> +	BUILD_BUG_ON(RES_NAME_SIZE != 4);
> +
> +	if (!((libcfs_debug | D_ERROR) & level))
> +		return;
> +
> +	CDEBUG(level, "--- Resource: " DLDLMRES " (%p) refcount = %d\n",
> +	       PLDLMRES(res), res, atomic_read(&res->lr_refcount));
> +
> +	if (!list_empty(&res->lr_granted)) {
> +		CDEBUG(level, "Granted locks (in reverse order):\n");
> +		list_for_each_entry_reverse(lock, &res->lr_granted,
> +					    l_res_link) {
> +			LDLM_DEBUG_LIMIT(level, lock, "###");
> +			if (!(level & D_CANTMASK) &&
> +			    ++granted > ldlm_dump_granted_max) {
> +				CDEBUG(level,
> +				       "only dump %d granted locks to avoid DDOS.\n",
> +				       granted);
> +				break;
> +			}
> +		}
> +	}
> +	if (!list_empty(&res->lr_waiting)) {
> +		CDEBUG(level, "Waiting locks:\n");
> +		list_for_each_entry(lock, &res->lr_waiting, l_res_link)
> +			LDLM_DEBUG_LIMIT(level, lock, "###");
> +	}
> +}
> +EXPORT_SYMBOL(ldlm_resource_dump);
> diff --git a/drivers/staging/lustre/lustre/ptlrpc/ptlrpc_internal.h b/drivers/staging/lustre/lustre/ptlrpc/ptlrpc_internal.h
> index 134b74234519..09ccb3fdabba 100644
> --- a/drivers/staging/lustre/lustre/ptlrpc/ptlrpc_internal.h
> +++ b/drivers/staging/lustre/lustre/ptlrpc/ptlrpc_internal.h
> @@ -36,7 +36,7 @@
>  #ifndef PTLRPC_INTERNAL_H
>  #define PTLRPC_INTERNAL_H
>  
> -#include "../ldlm/ldlm_internal.h"
> +#include "ldlm_internal.h"
>  
>  struct ldlm_namespace;
>  struct obd_import;
> 
> 
> 

From bevans at cray.com  Thu Jun  7 18:21:42 2018
From: bevans at cray.com (Ben Evans)
Date: Thu, 7 Jun 2018 18:21:42 +0000
Subject: [lustre-devel] [PATCH 10/11] staging: lustre: move ldlm into
 ptlrpc
In-Reply-To: <87sh5zlyf4.fsf@notabene.neil.brown.name>
References: <152826510267.16761.14361003167157833896.stgit@noble>
 <152826511923.16761.9237280635711887801.stgit@noble>
 <alpine.LFD.2.21.1806070546210.23188@casper.infradead.org>
 <87sh5zlyf4.fsf@notabene.neil.brown.name>
Message-ID: <D73EEFA7.FCA6%jevans@cray.com>



On 6/7/18, 5:48 AM, "lustre-devel on behalf of NeilBrown"
<lustre-devel-bounces at lists.lustre.org on behalf of neilb at suse.com> wrote:

>On Thu, Jun 07 2018, James Simmons wrote:
>
>>> The ldlm code is built into the ptlrpc module, yet it lived in a
>>> separate directory.  This requires filename editing in the Makefile
>>> and make it difficult to e.g. build the .s file for code in ldlm.
>>> 
>>> All the ldlm files have distinctive names so confusion from having
>>> ptlrpc and ldlm in the same directory is unlikely.  So move them all
>>> into ptlrpc.
>>
>> Nak. The reason is it would be nice to keep the directory structure.
>> What really needs to be done and Oleg has looked into it is to reduced
>> the number of modules created down to two, one for LNet and the other
>> lustre.ko. This also is a step in the right direction to remove the
>> create struct obd_ops and struct md_ops pointer madness. Well their
>> is the issue with obd echo client but we can deal with this at a later
>> date. Also the number of EXPORT_SYMBOLS and things will greatly reduce.
>
>Yeah, you are probably right.
>I had a bit of a look at how to build everything into a
>single module.  You can do with by having a single make
>file that lists parts from other directories - the same way
>that ptlrpc includes files from ldlm - but that is rather ugly.
>
>I've very nearly got it working using the lib-y infrastructure.
>I can build lnet as a single module, but the dependency calc isn't
>quite right so things happen in the wrong order.  The build
>fails the first time because some files don't exist, then
>succeeds on the second run.
>Hopefully I'll figure out how to make it work tomorrow.
>
>Thanks for the review,
>NeilBrown

Would this be client-only, or could the server code be added as well with
an ldiskfs/zfs module?


From neilb at suse.com  Thu Jun  7 20:50:29 2018
From: neilb at suse.com (NeilBrown)
Date: Fri, 08 Jun 2018 06:50:29 +1000
Subject: [lustre-devel] [PATCH 10/11] staging: lustre: move ldlm into
 ptlrpc
In-Reply-To: <D73EEFA7.FCA6%jevans@cray.com>
References: <152826510267.16761.14361003167157833896.stgit@noble>
 <152826511923.16761.9237280635711887801.stgit@noble>
 <alpine.LFD.2.21.1806070546210.23188@casper.infradead.org>
 <87sh5zlyf4.fsf@notabene.neil.brown.name> <D73EEFA7.FCA6%jevans@cray.com>
Message-ID: <87k1ramicq.fsf@notabene.neil.brown.name>

On Thu, Jun 07 2018, Ben Evans wrote:

> On 6/7/18, 5:48 AM, "lustre-devel on behalf of NeilBrown"
> <lustre-devel-bounces at lists.lustre.org on behalf of neilb at suse.com> wrote:
>
>>On Thu, Jun 07 2018, James Simmons wrote:
>>
>>>> The ldlm code is built into the ptlrpc module, yet it lived in a
>>>> separate directory.  This requires filename editing in the Makefile
>>>> and make it difficult to e.g. build the .s file for code in ldlm.
>>>> 
>>>> All the ldlm files have distinctive names so confusion from having
>>>> ptlrpc and ldlm in the same directory is unlikely.  So move them all
>>>> into ptlrpc.
>>>
>>> Nak. The reason is it would be nice to keep the directory structure.
>>> What really needs to be done and Oleg has looked into it is to reduced
>>> the number of modules created down to two, one for LNet and the other
>>> lustre.ko. This also is a step in the right direction to remove the
>>> create struct obd_ops and struct md_ops pointer madness. Well their
>>> is the issue with obd echo client but we can deal with this at a later
>>> date. Also the number of EXPORT_SYMBOLS and things will greatly reduce.
>>
>>Yeah, you are probably right.
>>I had a bit of a look at how to build everything into a
>>single module.  You can do with by having a single make
>>file that lists parts from other directories - the same way
>>that ptlrpc includes files from ldlm - but that is rather ugly.
>>
>>I've very nearly got it working using the lib-y infrastructure.
>>I can build lnet as a single module, but the dependency calc isn't
>>quite right so things happen in the wrong order.  The build
>>fails the first time because some files don't exist, then
>>succeeds on the second run.
>>Hopefully I'll figure out how to make it work tomorrow.
>>
>>Thanks for the review,
>>NeilBrown
>
> Would this be client-only, or could the server code be added as well with
> an ldiskfs/zfs module?

The important step is creating the infrastructure so that choice can
be easily made, and changed, with just a single line in a Makefile.
Fine-tuning decisions can follow.

I doubt it would make sense for ldiskfs and/or zfs to be in the same
module as lustre.  I also doubt that either will ever land upstream so
it hardly matters (to me).
When the server lands upstream (I think it will), it will use (at least)
the upstream ext4.  This might require changes to upstream ext4
(I understand a lot of work has already happened in that direction).  It
might require sacrificing some functionality (hopefully only
temporarily).  It might require having an incompatible on-disk format
(which is unfortunate but quite manageable).

Thanks,
NeilBrown
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180608/3cd44b19/attachment.sig>

From neilb at suse.com  Thu Jun  7 21:38:35 2018
From: neilb at suse.com (NeilBrown)
Date: Fri, 08 Jun 2018 07:38:35 +1000
Subject: [lustre-devel] A new drivers/staging/lustre
In-Reply-To: <A139C7E5-0406-4078-B579-8DCF5B85F10F@cray.com>
References: <874lifnxbp.fsf@notabene.neil.brown.name>
 <alpine.LFD.2.21.1806070557300.23188@casper.infradead.org>
 <87po12ncjv.fsf@notabene.neil.brown.name>
 <A139C7E5-0406-4078-B579-8DCF5B85F10F@cray.com>
Message-ID: <87h8memg4k.fsf@notabene.neil.brown.name>

On Thu, Jun 07 2018, Doug Oucharek wrote:

> What is the focus of landings in this tree?  There are two things needing to be done for an upstream Lustre:
>
>
>   *   Get the source code to meet the Linux guidelines so it is acceptable to be in mainline.
>   *   Get the binary product to have all the features and bug fixes that are in the Intel community tree so end users are interested in using the upstream version (users are unlikely to use a version of Lustre which is not current).
>
> For the now-deleted staging area, we were supposed to be focusing on the first item but were submitting patches for the second item (syncing with Intel tree).  In my opinion, this is the core reason for never being able to get out of staging and getting deleted.

My (undoubtedly biased) perspective on the history of lustre in staging
goes like this:
There are two things needed for some out-of-tree code to get into
mainline Linux:  the code needs to be integrated and the community needs
to be integrated (or a new sub-community needs to form).
In the case of lustre, the code was never really integrated because the
community never really tried to integrate.  Integrating and becoming
part of the Linux community takes time and effort, and it is quite
possible that management for various developers didn't allocate enough
time over a long enough period.  Integrating also requires a change in
attitude and I don't see much evidence of that.  I see clear evidence of
an "us and them" attitude among (some) lustre developers - almost as
though upstream linux is hostile territory full of unfriendly developers
who always reject our excellent code (even though they have lots of
horrible code themselves).  *We* need to see ourselves as part of the
Linux community, and we need to care about all of Linux as though it was
all ours (it *is* all ours, but *we* are a much larger group now).

Yes, the current code needs to be improved, bugs need to be fixed, and
features need to be added.  The order in which these is done is not the
most important things - if it were, Greg would have never accepted any
new features.  However he *did* accept them, but tried to remind the
lustre developers that there was other work to do.

Working together in one (single) community requires give-and-take.
Greg's behaviour as just described seems to be evidence of
give-and-take.  I think he kicked lustre out of staging because he
concluded that he was never going to get the matching give-and-take in
return.

So to answer your opening question, my focus for this tree is to train
any lustre developers who wish to engage about how to be part of the
Linux community.  As I've already said - I will accept features but I
prefer cleanups first.  I don't want to try to explain further than that
because it will be too hypothetical and unhelpful.  We - the Linux
community - don't work in hypotheticals.  We work with concrete objects
like patches.  So send me a patch and I will tell you what I think of
that specific patch.  It is up to you to generalise what I say to other
patches.  It might also be up to you to argue your case and tell me why
I'm wrong.  I'll be patient (because good upstream maintainers are) but
patience doesn't last forever (for Greg, it lasted about 5 years - I
hope mine won't be tried to that extent).

>
> There are some very big (as in code size) features missing from
> upstream.  For example, Multi-Rail.  When should that be pushed
> relative to code cleanups?

Never add features to ugly code - fix the code first.
The doesn't mean you cannot add any feature to lustre until all of
lustre is beautify.  But it does mean that if I can see in a patch some
ugly code and a new feature, then I won't be happy.  First clean up just
enough of the ugliness so that it won't be visible in the patch that
adds the feature.
But again, this is getting a bit too hypothetical.   If you care about a
feature, then post a patch.  We can take it from there.  The fact that
you care enough to post a patch cares significant weight - a lot more
weight than just asking about some feature.

Thanks,
NeilBrown
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180608/61bd6254/attachment.sig>

From neilb at suse.com  Thu Jun  7 21:50:37 2018
From: neilb at suse.com (NeilBrown)
Date: Fri, 08 Jun 2018 07:50:37 +1000
Subject: [lustre-devel] A new drivers/staging/lustre
In-Reply-To: <DM3PR1101MB1213BB514F8D6E25F725BB1DCB640@DM3PR1101MB1213.namprd11.prod.outlook.com>
References: <874lifnxbp.fsf@notabene.neil.brown.name>
 <alpine.LFD.2.21.1806070557300.23188@casper.infradead.org>
 <87po12ncjv.fsf@notabene.neil.brown.name>
 <A139C7E5-0406-4078-B579-8DCF5B85F10F@cray.com>
 <DM3PR1101MB1213BB514F8D6E25F725BB1DCB640@DM3PR1101MB1213.namprd11.prod.outlook.com>
Message-ID: <87efhimfki.fsf@notabene.neil.brown.name>

On Thu, Jun 07 2018, Patrick Farrell wrote:

> Doug,
>
> Another thought about the core reason.
>
> Commitment to this.  The existing code state (weird, abandoned 2.4.something code) and merge rules seemingly made it impossible to shift development in this direction in any meaningful way, so perhaps it's chicken and egg...  but as long as Lustre is released and developed primarily out of tree, I can't see this working.  Would it just be a "sync everything but still do releases" approach?  Is that viable?  Etc.
>
> Thoughts appreciated.

Yes, commitment is important - from management was well as developers.
If you think getting lustre upstream is important then make sure you
manager understands why.

Having development in two separate tree  - one heading for upstream
inclusion, the other used by customers - could be a problem and
certainly needs an end date, but I think it is our best option at the
moment (even though Greg claimed it was part of his reason for ejecting
lustre from staging).

I think that imposing an upstream development module on (what I've
decided to call) legacy-lustre would be a constant battle with a low
success rate.  Conversely moving code from legacy-lustre into an
upstream work-flow should be quite manageable providing time and skills
are available.

I would really like to be able to point to a location in the
legacy-lustre git history and be able to say "linux-lustre has all
features up to this point", and then to progressively copy features
across and move that point forward.  As a feature is copied, we make
sure it conforms with upstream standards.  I won't be doing any of this
until I'm really happy with what is already in linux-lustre, but I won't
try to set priorities for other people.

Once we get to the point where linux-lustre has feature-parity with
legacy-lustre, then we can discuss doing development in linux-lustre
first, and copying it across to legacy-lustre only as long as people
care.

Thanks,
NeilBrown
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180608/7da00f5f/attachment.sig>

From andreas.dilger at intel.com  Thu Jun  7 22:46:26 2018
From: andreas.dilger at intel.com (Dilger, Andreas)
Date: Thu, 7 Jun 2018 22:46:26 +0000
Subject: [lustre-devel] A new drivers/staging/lustre
In-Reply-To: <874lifnxbp.fsf@notabene.neil.brown.name>
References: <874lifnxbp.fsf@notabene.neil.brown.name>
Message-ID: <16129B94-CF52-4E5E-9A5A-2EC0E01BEEE0@intel.com>

On Jun 6, 2018, at 20:29, NeilBrown <neilb at suse.com> wrote:
> Greg's patch to remove lustre has now landed in this staging-next tree,
> so I suspect it will get to Linus before too long.  So I have to find a
> new place to work on lustre.
> 
> I've added 2 branches to
>   git://git.neil.brown.name/linux
> 
> lustre:
>   is based on Greg's patch that removes lustre, and starts with a
>   revert of the patch, followed by a merge of v4.17.
>   I plan to merge each release and RC from Linus, and also
>   add lustre patches that I think are "ready".  That will usually mean
>   they have been posted to this list at least a week earlier, and
>   have not had a credible negative response (Acks and Reviewed-by
>   would be nice).
>   I plan to update this branch about once a week, and to never rebase
>   it.
> 
> lustre-testing:
>   is based on 'lustre' and has most of my current lustre-related work.
>   It includes assorted patches that are not specifically for lustre
>   (rhashtables mostly at the moment).  Patches will move from here
>   to 'lustre' or to mainline when they are ready.
>   I plan to update this branch on most days that I work on Lustre,
>   and expect it to rebase frequently.

We also have a clone of Greg's staging branch on our Gerrit server,
which could be replaced with a "vanilla" upstream kernel clone that
holds a copy of the for-upstream Lustre tree.

One benefit of hosting the tree on the Gerrit server is that it is
much easier to get automated testing of the patches (a lot or a little)
to ensure that the code doesn't break anything obvious.

> I'm happy to review and, if acceptable, apply patches from other
> developers.  I have fairly high standards, but if I don't accept your
> patch I'll explain why and possible help fix it.
> I'm happy to accept enhancements and new features, but these need
> to be of a quality that would be accepted upstream.
> I'm only interested in client-side code at present - nothing that is
> only used on the server.  I do want to include server-side eventually,
> but I need some focus for now.
> 
> I hope to get to a stage where the code is of suitable quality that I
> can submit it to Linux as a new filesystem.  I hope that will happen
> this year.

I think one major drawback of starting with the from-staging version of
the code is that this is significantly behind the out-of-tree master
code and would need a lot of effort to catch up.

I think the only way we are going to make this work is if the cleanups
go into the same repo as the ongoing development.  Otherwise, we'll be
back in the same situation as we are now where there are two different
versions of the code and we need to move patches back and forth between
them.

On the plus side, this will avoid code drift, and the extra efforts of
porting patches back and forth.  On the minus side, it means (on some
fronts) that the code will regress from what is upstream, because we
don't have all of the trivial whitespace cleanups and other drive-by
kernel newbie patches that went into the upstream client.  However, it
does mean that those cleanups would now become part of the single code
tree and not be lost again in the future.

This would also mean some restructuring of the current Lustre tree so
that it could build as a drop-in at linux/fs/lustre but that is probably
a good idea for the long term in any case.

Cheers, Andreas
--
Andreas Dilger
Lustre Principal Architect
Intel Corporation








From doucharek at cray.com  Thu Jun  7 22:53:37 2018
From: doucharek at cray.com (Doug Oucharek)
Date: Thu, 7 Jun 2018 22:53:37 +0000
Subject: [lustre-devel] A new drivers/staging/lustre
In-Reply-To: <16129B94-CF52-4E5E-9A5A-2EC0E01BEEE0@intel.com>
References: <874lifnxbp.fsf@notabene.neil.brown.name>,
 <16129B94-CF52-4E5E-9A5A-2EC0E01BEEE0@intel.com>
Message-ID: <MWHPR11MB2029D262448ACF23E0275C11D4640@MWHPR11MB2029.namprd11.prod.outlook.com>

Does this mean we would be pushing patches to Gerrit rather than having to email them out?  I believe for the test system to work automatically, the answer is yes.

Doug

________________________________
From: lustre-devel <lustre-devel-bounces at lists.lustre.org> on behalf of Dilger, Andreas <andreas.dilger at intel.com>
Sent: Thursday, June 7, 2018 5:46:26 PM
To: NeilBrown
Cc: lustre-devel
Subject: Re: [lustre-devel] A new drivers/staging/lustre

On Jun 6, 2018, at 20:29, NeilBrown <neilb at suse.com> wrote:
> Greg's patch to remove lustre has now landed in this staging-next tree,
> so I suspect it will get to Linus before too long.  So I have to find a
> new place to work on lustre.
>
> I've added 2 branches to
>   git://git.neil.brown.name/linux
>
> lustre:
>   is based on Greg's patch that removes lustre, and starts with a
>   revert of the patch, followed by a merge of v4.17.
>   I plan to merge each release and RC from Linus, and also
>   add lustre patches that I think are "ready".  That will usually mean
>   they have been posted to this list at least a week earlier, and
>   have not had a credible negative response (Acks and Reviewed-by
>   would be nice).
>   I plan to update this branch about once a week, and to never rebase
>   it.
>
> lustre-testing:
>   is based on 'lustre' and has most of my current lustre-related work.
>   It includes assorted patches that are not specifically for lustre
>   (rhashtables mostly at the moment).  Patches will move from here
>   to 'lustre' or to mainline when they are ready.
>   I plan to update this branch on most days that I work on Lustre,
>   and expect it to rebase frequently.

We also have a clone of Greg's staging branch on our Gerrit server,
which could be replaced with a "vanilla" upstream kernel clone that
holds a copy of the for-upstream Lustre tree.

One benefit of hosting the tree on the Gerrit server is that it is
much easier to get automated testing of the patches (a lot or a little)
to ensure that the code doesn't break anything obvious.

> I'm happy to review and, if acceptable, apply patches from other
> developers.  I have fairly high standards, but if I don't accept your
> patch I'll explain why and possible help fix it.
> I'm happy to accept enhancements and new features, but these need
> to be of a quality that would be accepted upstream.
> I'm only interested in client-side code at present - nothing that is
> only used on the server.  I do want to include server-side eventually,
> but I need some focus for now.
>
> I hope to get to a stage where the code is of suitable quality that I
> can submit it to Linux as a new filesystem.  I hope that will happen
> this year.

I think one major drawback of starting with the from-staging version of
the code is that this is significantly behind the out-of-tree master
code and would need a lot of effort to catch up.

I think the only way we are going to make this work is if the cleanups
go into the same repo as the ongoing development.  Otherwise, we'll be
back in the same situation as we are now where there are two different
versions of the code and we need to move patches back and forth between
them.

On the plus side, this will avoid code drift, and the extra efforts of
porting patches back and forth.  On the minus side, it means (on some
fronts) that the code will regress from what is upstream, because we
don't have all of the trivial whitespace cleanups and other drive-by
kernel newbie patches that went into the upstream client.  However, it
does mean that those cleanups would now become part of the single code
tree and not be lost again in the future.

This would also mean some restructuring of the current Lustre tree so
that it could build as a drop-in at linux/fs/lustre but that is probably
a good idea for the long term in any case.

Cheers, Andreas
--
Andreas Dilger
Lustre Principal Architect
Intel Corporation







_______________________________________________
lustre-devel mailing list
lustre-devel at lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180607/e8ac9ba2/attachment.html>

From andreas.dilger at intel.com  Thu Jun  7 23:06:23 2018
From: andreas.dilger at intel.com (Dilger, Andreas)
Date: Thu, 7 Jun 2018 23:06:23 +0000
Subject: [lustre-devel] A new drivers/staging/lustre
In-Reply-To: <87h8memg4k.fsf@notabene.neil.brown.name>
References: <874lifnxbp.fsf@notabene.neil.brown.name>
 <alpine.LFD.2.21.1806070557300.23188@casper.infradead.org>
 <87po12ncjv.fsf@notabene.neil.brown.name>
 <A139C7E5-0406-4078-B579-8DCF5B85F10F@cray.com>
 <87h8memg4k.fsf@notabene.neil.brown.name>
Message-ID: <C50F2EA9-AD25-480A-8A05-2541206BA17F@intel.com>

On Jun 7, 2018, at 15:38, NeilBrown <neilb at suse.com> wrote:
> 
> On Thu, Jun 07 2018, Doug Oucharek wrote:
> 
>> What is the focus of landings in this tree?  There are two things needing to be done for an upstream Lustre:
>> 
>> 
>>  *   Get the source code to meet the Linux guidelines so it is acceptable to be in mainline.
>>  *   Get the binary product to have all the features and bug fixes that are in the Intel community tree so end users are interested in using the upstream version (users are unlikely to use a version of Lustre which is not current).
>> 
>> For the now-deleted staging area, we were supposed to be focusing on the first item but were submitting patches for the second item (syncing with Intel tree).  In my opinion, this is the core reason for never being able to get out of staging and getting deleted.
> 
> My (undoubtedly biased) perspective on the history of lustre in staging
> goes like this:
> There are two things needed for some out-of-tree code to get into
> mainline Linux:  the code needs to be integrated and the community needs
> to be integrated (or a new sub-community needs to form).
> In the case of lustre, the code was never really integrated because the
> community never really tried to integrate.

One of the issues here was that the group (not Intel) that submitted the
Lustre code to the staging tree promptly abandoned it for a couple of
years after they submitted it upstream, after promising the community
that they were in it for the long run.  That put the upstream integration
behind the eight-ball from the start.

>  Integrating and becoming
> part of the Linux community takes time and effort, and it is quite
> possible that management for various developers didn't allocate enough
> time over a long enough period.  Integrating also requires a change in
> attitude and I don't see much evidence of that.  I see clear evidence of
> an "us and them" attitude among (some) lustre developers - almost as
> though upstream linux is hostile territory full of unfriendly developers

Ah, but it *is* hostile territory, if you are not among the "in crowd".
Christoph can get any change he wants to be accepted, but if someone else
tries to push something similar it can be rejected outright or ignored
for months or years.

> who always reject our excellent code (even though they have lots of
> horrible code themselves).  *We* need to see ourselves as part of the
> Linux community, and we need to care about all of Linux as though it was
> all ours (it *is* all ours, but *we* are a much larger group now).
> 
> Yes, the current code needs to be improved, bugs need to be fixed, and
> features need to be added.  The order in which these is done is not the
> most important things - if it were, Greg would have never accepted any
> new features.  However he *did* accept them, but tried to remind the
> lustre developers that there was other work to do.
> 
> Working together in one (single) community requires give-and-take.
> Greg's behaviour as just described seems to be evidence of
> give-and-take.  I think he kicked lustre out of staging because he
> concluded that he was never going to get the matching give-and-take in
> return.
> 
> So to answer your opening question, my focus for this tree is to train
> any lustre developers who wish to engage about how to be part of the
> Linux community.  As I've already said - I will accept features but I
> prefer cleanups first.  I don't want to try to explain further than that
> because it will be too hypothetical and unhelpful.  We - the Linux
> community - don't work in hypotheticals.  We work with concrete objects
> like patches.  So send me a patch and I will tell you what I think of
> that specific patch.  It is up to you to generalise what I say to other
> patches.  It might also be up to you to argue your case and tell me why
> I'm wrong.  I'll be patient (because good upstream maintainers are) but
> patience doesn't last forever (for Greg, it lasted about 5 years - I
> hope mine won't be tried to that extent).

Like I said in my other email, I think having another fork of the Lustre
tree, especially one that is starting from two-year-old code is likely to
fail, because there will be twice as much effort spent to maintain the two
trees.  I'd rather see the cleanups and features go hand-in-hand into the
same tree.  I'd be thrilled to have more reviews done on the features before
they are landed, but we can't just stop all feature development for a year
or two (or five) while the code is merged into the upstream kernel.

>> There are some very big (as in code size) features missing from
>> upstream.  For example, Multi-Rail.  When should that be pushed
>> relative to code cleanups?
> 
> Never add features to ugly code - fix the code first.
> The doesn't mean you cannot add any feature to lustre until all of
> lustre is beautiful.  But it does mean that if I can see in a patch some
> ugly code and a new feature, then I won't be happy.  First clean up just
> enough of the ugliness so that it won't be visible in the patch that
> adds the feature.

The issue is that we _can't_ just stop the development of new code/features
for such a long time.  There are huge supercomputers being deployed or in
planning that depend on these new features, or they wouldn't have been
developed in the first place.

Consider if the NFSv4 spec was written and the code was developed, and you
were told you needed to go back to NFSv2 and start again?

> But again, this is getting a bit too hypothetical.   If you care about a
> feature, then post a patch.  We can take it from there.  The fact that
> you care enough to post a patch cares significant weight - a lot more
> weight than just asking about some feature.

We definitely aren't at the point of "asking for some feature to be developed".
At a minimum the starting point of the new upstream code needs to be the
current release, or any resources that could possibly be put towards improving
the Lustre code would be squandered on porting all of those patches to the
upstream tree.  I'm fine with spending time to improve the code that exists
today, but lets not start with a huge deficit from the outset.

Cheers, Andreas
--
Andreas Dilger
Lustre Principal Architect
Intel Corporation








From andreas.dilger at intel.com  Thu Jun  7 23:24:56 2018
From: andreas.dilger at intel.com (Dilger, Andreas)
Date: Thu, 7 Jun 2018 23:24:56 +0000
Subject: [lustre-devel] A new drivers/staging/lustre
In-Reply-To: <MWHPR11MB2029D262448ACF23E0275C11D4640@MWHPR11MB2029.namprd11.prod.outlook.com>
References: <874lifnxbp.fsf@notabene.neil.brown.name>
 <16129B94-CF52-4E5E-9A5A-2EC0E01BEEE0@intel.com>
 <MWHPR11MB2029D262448ACF23E0275C11D4640@MWHPR11MB2029.namprd11.prod.outlook.com>
Message-ID: <51E547E3-5876-4AE0-9226-FDE082213650@intel.com>

On Jun 7, 2018, at 16:53, Doug Oucharek <doucharek at cray.com> wrote:
> 
> Does this mean we would be pushing patches to Gerrit rather than having to email them out?  I believe for the test system to work automatically, the answer is yes.

I'm not an expert with Gerrit configuration, so I'm not sure if there is an
email-to-change gateway plugin for it or not.  I believe it can be configured
to send the full patch in emails.  I get an email for every patch submitted,
but I configure it to only deliver the commit message and not the whole patch.

Since everyone is using Git anyway, using "git push" to submit patches to Gerrit
for review (and forwards them to the list, if that is what people want) isn't
more effort than "git send-email" to send them to the list directly.

I think any branch which just takes patches without adequate testing in a
variety of configurations (e.g. DNE, ZFS/ldiskfs, TCP/IB, quota, recovery,
version interop, failover, etc.) is just going to accumulate small or large
breakages and will not be usable in production.

James can probably tell you how many times "simple" patches were landed into
the upstream kernel that ended up breaking something because they weren't
tested.  I don't expect everyone to be able to test all of those configurations
themselves, which is why it is good to have automated testing to do this for
every patch that is submitted.

I'd be *thrilled* if more organizations/developers provided automated testing
of patches (e.g. PPC, ARM, Cray networking, etc.), so it isn't like I'm trying
to monopolize the testing (far from it).  However, the reality is that there
*is* no other automated testing environment beyond what we have with the
existing Jenkins/Gerrit/autotest/Maloo infrastructure we have today.  It is
imperfect for sure, but a gigantic leap beyond not having anything at all.

Cheers, Andreas

> From: Dilger, Andreas <andreas.dilger at intel.com>
> Sent: Thursday, June 7, 2018 5:46:26 PM
> To: NeilBrown
> Cc: lustre-devel
> Subject: Re: [lustre-devel] A new drivers/staging/lustre
>  
> On Jun 6, 2018, at 20:29, NeilBrown <neilb at suse.com> wrote:
> > Greg's patch to remove lustre has now landed in this staging-next tree,
> > so I suspect it will get to Linus before too long.  So I have to find a
> > new place to work on lustre.
> > 
> > I've added 2 branches to
> >   git://git.neil.brown.name/linux
> > 
> > lustre:
> >   is based on Greg's patch that removes lustre, and starts with a
> >   revert of the patch, followed by a merge of v4.17.
> >   I plan to merge each release and RC from Linus, and also
> >   add lustre patches that I think are "ready".  That will usually mean
> >   they have been posted to this list at least a week earlier, and
> >   have not had a credible negative response (Acks and Reviewed-by
> >   would be nice).
> >   I plan to update this branch about once a week, and to never rebase
> >   it.
> > 
> > lustre-testing:
> >   is based on 'lustre' and has most of my current lustre-related work.
> >   It includes assorted patches that are not specifically for lustre
> >   (rhashtables mostly at the moment).  Patches will move from here
> >   to 'lustre' or to mainline when they are ready.
> >   I plan to update this branch on most days that I work on Lustre,
> >   and expect it to rebase frequently.
> 
> We also have a clone of Greg's staging branch on our Gerrit server,
> which could be replaced with a "vanilla" upstream kernel clone that
> holds a copy of the for-upstream Lustre tree.
> 
> One benefit of hosting the tree on the Gerrit server is that it is
> much easier to get automated testing of the patches (a lot or a little)
> to ensure that the code doesn't break anything obvious.
> 
> > I'm happy to review and, if acceptable, apply patches from other
> > developers.  I have fairly high standards, but if I don't accept your
> > patch I'll explain why and possible help fix it.
> > I'm happy to accept enhancements and new features, but these need
> > to be of a quality that would be accepted upstream.
> > I'm only interested in client-side code at present - nothing that is
> > only used on the server.  I do want to include server-side eventually,
> > but I need some focus for now.
> > 
> > I hope to get to a stage where the code is of suitable quality that I
> > can submit it to Linux as a new filesystem.  I hope that will happen
> > this year.
> 
> I think one major drawback of starting with the from-staging version of
> the code is that this is significantly behind the out-of-tree master
> code and would need a lot of effort to catch up.
> 
> I think the only way we are going to make this work is if the cleanups
> go into the same repo as the ongoing development.  Otherwise, we'll be
> back in the same situation as we are now where there are two different
> versions of the code and we need to move patches back and forth between
> them.
> 
> On the plus side, this will avoid code drift, and the extra efforts of
> porting patches back and forth.  On the minus side, it means (on some
> fronts) that the code will regress from what is upstream, because we
> don't have all of the trivial whitespace cleanups and other drive-by
> kernel newbie patches that went into the upstream client.  However, it
> does mean that those cleanups would now become part of the single code
> tree and not be lost again in the future.
> 
> This would also mean some restructuring of the current Lustre tree so
> that it could build as a drop-in at linux/fs/lustre but that is probably
> a good idea for the long term in any case.
> 
> Cheers, Andreas
> --
> Andreas Dilger
> Lustre Principal Architect
> Intel Corporation
> 
> 
> 
> 
> 
> 
> 
> _______________________________________________
> lustre-devel mailing list
> lustre-devel at lists.lustre.org
> http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org

Cheers, Andreas
--
Andreas Dilger
Lustre Principal Architect
Intel Corporation








From paf at cray.com  Thu Jun  7 23:27:01 2018
From: paf at cray.com (Patrick Farrell)
Date: Thu, 7 Jun 2018 23:27:01 +0000
Subject: [lustre-devel] A new drivers/staging/lustre
In-Reply-To: <C50F2EA9-AD25-480A-8A05-2541206BA17F@intel.com>
References: <874lifnxbp.fsf@notabene.neil.brown.name>
 <alpine.LFD.2.21.1806070557300.23188@casper.infradead.org>
 <87po12ncjv.fsf@notabene.neil.brown.name>
 <A139C7E5-0406-4078-B579-8DCF5B85F10F@cray.com>
 <87h8memg4k.fsf@notabene.neil.brown.name>,
 <C50F2EA9-AD25-480A-8A05-2541206BA17F@intel.com>
Message-ID: <DM3PR1101MB1213C5B93F78BD84A103EACFCB640@DM3PR1101MB1213.namprd11.prod.outlook.com>


For what it’s worth, I agree strongly with Andreas about getting this in sync with the current Intel branch FIRST.  There is far more additional code/history there in the form of already complete features and fixes than here in cleanups.

________________________________
From: lustre-devel <lustre-devel-bounces at lists.lustre.org> on behalf of Dilger, Andreas <andreas.dilger at intel.com>
Sent: Thursday, June 7, 2018 6:06:23 PM
To: NeilBrown
Cc: Drokin, Oleg; lustre-devel
Subject: Re: [lustre-devel] A new drivers/staging/lustre

On Jun 7, 2018, at 15:38, NeilBrown <neilb at suse.com> wrote:
>
> On Thu, Jun 07 2018, Doug Oucharek wrote:
>
>> What is the focus of landings in this tree?  There are two things needing to be done for an upstream Lustre:
>>
>>
>>  *   Get the source code to meet the Linux guidelines so it is acceptable to be in mainline.
>>  *   Get the binary product to have all the features and bug fixes that are in the Intel community tree so end users are interested in using the upstream version (users are unlikely to use a version of Lustre which is not current).
>>
>> For the now-deleted staging area, we were supposed to be focusing on the first item but were submitting patches for the second item (syncing with Intel tree).  In my opinion, this is the core reason for never being able to get out of staging and getting deleted.
>
> My (undoubtedly biased) perspective on the history of lustre in staging
> goes like this:
> There are two things needed for some out-of-tree code to get into
> mainline Linux:  the code needs to be integrated and the community needs
> to be integrated (or a new sub-community needs to form).
> In the case of lustre, the code was never really integrated because the
> community never really tried to integrate.

One of the issues here was that the group (not Intel) that submitted the
Lustre code to the staging tree promptly abandoned it for a couple of
years after they submitted it upstream, after promising the community
that they were in it for the long run.  That put the upstream integration
behind the eight-ball from the start.

>  Integrating and becoming
> part of the Linux community takes time and effort, and it is quite
> possible that management for various developers didn't allocate enough
> time over a long enough period.  Integrating also requires a change in
> attitude and I don't see much evidence of that.  I see clear evidence of
> an "us and them" attitude among (some) lustre developers - almost as
> though upstream linux is hostile territory full of unfriendly developers

Ah, but it *is* hostile territory, if you are not among the "in crowd".
Christoph can get any change he wants to be accepted, but if someone else
tries to push something similar it can be rejected outright or ignored
for months or years.

> who always reject our excellent code (even though they have lots of
> horrible code themselves).  *We* need to see ourselves as part of the
> Linux community, and we need to care about all of Linux as though it was
> all ours (it *is* all ours, but *we* are a much larger group now).
>
> Yes, the current code needs to be improved, bugs need to be fixed, and
> features need to be added.  The order in which these is done is not the
> most important things - if it were, Greg would have never accepted any
> new features.  However he *did* accept them, but tried to remind the
> lustre developers that there was other work to do.
>
> Working together in one (single) community requires give-and-take.
> Greg's behaviour as just described seems to be evidence of
> give-and-take.  I think he kicked lustre out of staging because he
> concluded that he was never going to get the matching give-and-take in
> return.
>
> So to answer your opening question, my focus for this tree is to train
> any lustre developers who wish to engage about how to be part of the
> Linux community.  As I've already said - I will accept features but I
> prefer cleanups first.  I don't want to try to explain further than that
> because it will be too hypothetical and unhelpful.  We - the Linux
> community - don't work in hypotheticals.  We work with concrete objects
> like patches.  So send me a patch and I will tell you what I think of
> that specific patch.  It is up to you to generalise what I say to other
> patches.  It might also be up to you to argue your case and tell me why
> I'm wrong.  I'll be patient (because good upstream maintainers are) but
> patience doesn't last forever (for Greg, it lasted about 5 years - I
> hope mine won't be tried to that extent).

Like I said in my other email, I think having another fork of the Lustre
tree, especially one that is starting from two-year-old code is likely to
fail, because there will be twice as much effort spent to maintain the two
trees.  I'd rather see the cleanups and features go hand-in-hand into the
same tree.  I'd be thrilled to have more reviews done on the features before
they are landed, but we can't just stop all feature development for a year
or two (or five) while the code is merged into the upstream kernel.

>> There are some very big (as in code size) features missing from
>> upstream.  For example, Multi-Rail.  When should that be pushed
>> relative to code cleanups?
>
> Never add features to ugly code - fix the code first.
> The doesn't mean you cannot add any feature to lustre until all of
> lustre is beautiful.  But it does mean that if I can see in a patch some
> ugly code and a new feature, then I won't be happy.  First clean up just
> enough of the ugliness so that it won't be visible in the patch that
> adds the feature.

The issue is that we _can't_ just stop the development of new code/features
for such a long time.  There are huge supercomputers being deployed or in
planning that depend on these new features, or they wouldn't have been
developed in the first place.

Consider if the NFSv4 spec was written and the code was developed, and you
were told you needed to go back to NFSv2 and start again?

> But again, this is getting a bit too hypothetical.   If you care about a
> feature, then post a patch.  We can take it from there.  The fact that
> you care enough to post a patch cares significant weight - a lot more
> weight than just asking about some feature.

We definitely aren't at the point of "asking for some feature to be developed".
At a minimum the starting point of the new upstream code needs to be the
current release, or any resources that could possibly be put towards improving
the Lustre code would be squandered on porting all of those patches to the
upstream tree.  I'm fine with spending time to improve the code that exists
today, but lets not start with a huge deficit from the outset.

Cheers, Andreas
--
Andreas Dilger
Lustre Principal Architect
Intel Corporation







_______________________________________________
lustre-devel mailing list
lustre-devel at lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180607/9e4d04a7/attachment.html>

From neilb at suse.com  Fri Jun  8 05:45:35 2018
From: neilb at suse.com (NeilBrown)
Date: Fri, 08 Jun 2018 15:45:35 +1000
Subject: [lustre-devel] A new drivers/staging/lustre
In-Reply-To: <C50F2EA9-AD25-480A-8A05-2541206BA17F@intel.com>
References: <874lifnxbp.fsf@notabene.neil.brown.name>
 <alpine.LFD.2.21.1806070557300.23188@casper.infradead.org>
 <87po12ncjv.fsf@notabene.neil.brown.name>
 <A139C7E5-0406-4078-B579-8DCF5B85F10F@cray.com>
 <87h8memg4k.fsf@notabene.neil.brown.name>
 <C50F2EA9-AD25-480A-8A05-2541206BA17F@intel.com>
Message-ID: <8736xxn85c.fsf@notabene.neil.brown.name>

On Thu, Jun 07 2018, Dilger, Andreas wrote:

> On Jun 7, 2018, at 15:38, NeilBrown <neilb at suse.com> wrote:
>> 
>> On Thu, Jun 07 2018, Doug Oucharek wrote:
>> 
>>> What is the focus of landings in this tree?  There are two things needing to be done for an upstream Lustre:
>>> 
>>> 
>>>  *   Get the source code to meet the Linux guidelines so it is acceptable to be in mainline.
>>>  *   Get the binary product to have all the features and bug fixes that are in the Intel community tree so end users are interested in using the upstream version (users are unlikely to use a version of Lustre which is not current).
>>> 
>>> For the now-deleted staging area, we were supposed to be focusing on the first item but were submitting patches for the second item (syncing with Intel tree).  In my opinion, this is the core reason for never being able to get out of staging and getting deleted.
>> 
>> My (undoubtedly biased) perspective on the history of lustre in staging
>> goes like this:
>> There are two things needed for some out-of-tree code to get into
>> mainline Linux:  the code needs to be integrated and the community needs
>> to be integrated (or a new sub-community needs to form).
>> In the case of lustre, the code was never really integrated because the
>> community never really tried to integrate.
>
> One of the issues here was that the group (not Intel) that submitted the
> Lustre code to the staging tree promptly abandoned it for a couple of
> years after they submitted it upstream, after promising the community
> that they were in it for the long run.  That put the upstream integration
> behind the eight-ball from the start.

Yes, we have a lot of "technical debt" to recover from, or repay, or
what ever analogy works best.

>
>>  Integrating and becoming
>> part of the Linux community takes time and effort, and it is quite
>> possible that management for various developers didn't allocate enough
>> time over a long enough period.  Integrating also requires a change in
>> attitude and I don't see much evidence of that.  I see clear evidence of
>> an "us and them" attitude among (some) lustre developers - almost as
>> though upstream linux is hostile territory full of unfriendly developers
>
> Ah, but it *is* hostile territory, if you are not among the "in crowd".
> Christoph can get any change he wants to be accepted, but if someone else
> tries to push something similar it can be rejected outright or ignored
> for months or years.

Christoph is taken more seriously than others primarily because he is
more consistently right.  He has built up trust with other key people
over years.  You cannot expect to get the same returns without the same
investment.
If he has interests that overlap with your, then it is certainly worth
trying to understand his point of view, and worth trying to work with
him.  That can be a challenge as his style can be a bit abrasive, but
I'm sure it can yield good returns.

Christoph may seem like the enemy, but he is really just another
developer like you or me.  He has his priorities and his hang-ups and he
wants to get his work done and to keep Linux generally in a healthy
state.
If you think of him (or anyone else) as the enemy, it will poison your
relationship before you even start.
If you think of him as a future friend who speaks a different language
which it will take you a while to understand, I think you will make more
progress.
Importantly, if you appear to be trying to work together you will create
a good impression on others and start you build your social capital, and
start to earn trust.

>
>> who always reject our excellent code (even though they have lots of
>> horrible code themselves).  *We* need to see ourselves as part of the
>> Linux community, and we need to care about all of Linux as though it was
>> all ours (it *is* all ours, but *we* are a much larger group now).
>> 
>> Yes, the current code needs to be improved, bugs need to be fixed, and
>> features need to be added.  The order in which these is done is not the
>> most important things - if it were, Greg would have never accepted any
>> new features.  However he *did* accept them, but tried to remind the
>> lustre developers that there was other work to do.
>> 
>> Working together in one (single) community requires give-and-take.
>> Greg's behaviour as just described seems to be evidence of
>> give-and-take.  I think he kicked lustre out of staging because he
>> concluded that he was never going to get the matching give-and-take in
>> return.
>> 
>> So to answer your opening question, my focus for this tree is to train
>> any lustre developers who wish to engage about how to be part of the
>> Linux community.  As I've already said - I will accept features but I
>> prefer cleanups first.  I don't want to try to explain further than that
>> because it will be too hypothetical and unhelpful.  We - the Linux
>> community - don't work in hypotheticals.  We work with concrete objects
>> like patches.  So send me a patch and I will tell you what I think of
>> that specific patch.  It is up to you to generalise what I say to other
>> patches.  It might also be up to you to argue your case and tell me why
>> I'm wrong.  I'll be patient (because good upstream maintainers are) but
>> patience doesn't last forever (for Greg, it lasted about 5 years - I
>> hope mine won't be tried to that extent).
>
> Like I said in my other email, I think having another fork of the Lustre
> tree, especially one that is starting from two-year-old code is likely to
> fail, because there will be twice as much effort spent to maintain the two
> trees.  I'd rather see the cleanups and features go hand-in-hand into the
> same tree.  I'd be thrilled to have more reviews done on the features before
> they are landed, but we can't just stop all feature development for a year
> or two (or five) while the code is merged into the upstream kernel.

Where did this idea of stopping all feature development come from?  It
is a ridiculous idea.
Of course, getting the code into shape for upstream inclusion will
require effort and time and someone will have to do that.  While they
are doing it they won't be working on features.
That either means new developers will need to get involved (like me) or
current developers will need to dedicate some of their time to the
upstream work.
So feature development doesn't have to stop, but it might slow unless
extra developers are added.

There seem to be two options:
1/ code clean up and continuing feature development happen in the one
   tree.
2/ feature development happens in one tree while code cleanup and
   integration of newly developed feature happens in another tree.

There are two variation on option 2.
2a/ the code-cleanup tree is based on what is being removed from
    drivers/staging.
2b/ the code-cleanup tree is created as a new fork of the current
    devel tree.

Each of 2a and 2b would have costs and benefits that the other doesn't
have.
I like 2a because a lot of cleanup has already been done, and because
porting patches across from the devel tree will provide a good
opportunity to review those patches.  Reviewing a large body of existing code
is really hard.  Reviewing patches one at a time is easier as they tend
to be conceptually coherent.

I like 2a and have already made steps in that direction.  But the
decision needs to be made by the people who will be doing the work.
Hopefully that isn't just me.

So: who has time to commit to creating an upstream-able version of
  lustre?  If everyone who will be committing time could make a clear
  statement of how they would like to proceed, then I'm sure we can come
  to agreement.

>
>>> There are some very big (as in code size) features missing from
>>> upstream.  For example, Multi-Rail.  When should that be pushed
>>> relative to code cleanups?
>> 
>> Never add features to ugly code - fix the code first.
>> The doesn't mean you cannot add any feature to lustre until all of
>> lustre is beautiful.  But it does mean that if I can see in a patch some
>> ugly code and a new feature, then I won't be happy.  First clean up just
>> enough of the ugliness so that it won't be visible in the patch that
>> adds the feature.
>
> The issue is that we _can't_ just stop the development of new code/features
> for such a long time.  There are huge supercomputers being deployed or in
> planning that depend on these new features, or they wouldn't have been
> developed in the first place.
>
> Consider if the NFSv4 spec was written and the code was developed, and you
> were told you needed to go back to NFSv2 and start again?
>
>> But again, this is getting a bit too hypothetical.   If you care about a
>> feature, then post a patch.  We can take it from there.  The fact that
>> you care enough to post a patch cares significant weight - a lot more
>> weight than just asking about some feature.
>
> We definitely aren't at the point of "asking for some feature to be developed".
> At a minimum the starting point of the new upstream code needs to be the
> current release, or any resources that could possibly be put towards improving
> the Lustre code would be squandered on porting all of those patches to the
> upstream tree.  I'm fine with spending time to improve the code that exists
> today, but lets not start with a huge deficit from the outset.

We have a huge deficit either way.  One way we have a deficit of
functionality, the other way we have a deficit of quality.

But let's not spend too much time talking about the problems.  I'd much
rather hear what you are doing about it.  If other people are working,
I'll align with them.

Thanks,
NeilBrown
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180608/8b8fe4ad/attachment.sig>

From neilb at suse.com  Fri Jun  8 06:00:17 2018
From: neilb at suse.com (NeilBrown)
Date: Fri, 08 Jun 2018 16:00:17 +1000
Subject: [lustre-devel] A new drivers/staging/lustre
In-Reply-To: <51E547E3-5876-4AE0-9226-FDE082213650@intel.com>
References: <874lifnxbp.fsf@notabene.neil.brown.name>
 <16129B94-CF52-4E5E-9A5A-2EC0E01BEEE0@intel.com>
 <MWHPR11MB2029D262448ACF23E0275C11D4640@MWHPR11MB2029.namprd11.prod.outlook.com>
 <51E547E3-5876-4AE0-9226-FDE082213650@intel.com>
Message-ID: <87zi05lswe.fsf@notabene.neil.brown.name>

On Thu, Jun 07 2018, Dilger, Andreas wrote:

> On Jun 7, 2018, at 16:53, Doug Oucharek <doucharek at cray.com> wrote:
>> 
>> Does this mean we would be pushing patches to Gerrit rather than having to email them out?  I believe for the test system to work automatically, the answer is yes.
>
> I'm not an expert with Gerrit configuration, so I'm not sure if there is an
> email-to-change gateway plugin for it or not.  I believe it can be configured
> to send the full patch in emails.  I get an email for every patch submitted,
> but I configure it to only deliver the commit message and not the whole patch.
>
> Since everyone is using Git anyway, using "git push" to submit patches to Gerrit
> for review (and forwards them to the list, if that is what people want) isn't
> more effort than "git send-email" to send them to the list directly.

I think that pushing patches to Gerrit for testing is essential.  I'm
less keen on using it for review because it doesn't fit my current
workflow, but I could probably adjust if necessary.  My preference would
be for people to develop patches doing their own local testing to avoid
the really embarrassing errors (I have a 4-node virtual cluster on my
desktop) and then post them to a list for wider review.  If that passes
then someone (maybe one of a small team other something) adds them to a
git tree (either with "git am" or "git pull", whatever works for people)
which gerrit runs tests on.  We would try to get this tree incorporated
into the 01.org testing so that we get even more coverage.  They might
not be able to run lustre-fs tests, but they do static analysis testing
which should cover lustre.

Code wouldn't migrate to the "main" tree until it had passed both review
and testing.

Thanks,
NeilBrown
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180608/a6a71e21/attachment.sig>

From neilb at suse.com  Fri Jun  8 06:07:21 2018
From: neilb at suse.com (NeilBrown)
Date: Fri, 08 Jun 2018 16:07:21 +1000
Subject: [lustre-devel] A new drivers/staging/lustre
In-Reply-To: <16129B94-CF52-4E5E-9A5A-2EC0E01BEEE0@intel.com>
References: <874lifnxbp.fsf@notabene.neil.brown.name>
 <16129B94-CF52-4E5E-9A5A-2EC0E01BEEE0@intel.com>
Message-ID: <87wov9lskm.fsf@notabene.neil.brown.name>

On Thu, Jun 07 2018, Dilger, Andreas wrote:

> On Jun 6, 2018, at 20:29, NeilBrown <neilb at suse.com> wrote:
>> Greg's patch to remove lustre has now landed in this staging-next tree,
>> so I suspect it will get to Linus before too long.  So I have to find a
>> new place to work on lustre.
>> 
>> I've added 2 branches to
>>   git://git.neil.brown.name/linux
>> 
>> lustre:
>>   is based on Greg's patch that removes lustre, and starts with a
>>   revert of the patch, followed by a merge of v4.17.
>>   I plan to merge each release and RC from Linus, and also
>>   add lustre patches that I think are "ready".  That will usually mean
>>   they have been posted to this list at least a week earlier, and
>>   have not had a credible negative response (Acks and Reviewed-by
>>   would be nice).
>>   I plan to update this branch about once a week, and to never rebase
>>   it.
>> 
>> lustre-testing:
>>   is based on 'lustre' and has most of my current lustre-related work.
>>   It includes assorted patches that are not specifically for lustre
>>   (rhashtables mostly at the moment).  Patches will move from here
>>   to 'lustre' or to mainline when they are ready.
>>   I plan to update this branch on most days that I work on Lustre,
>>   and expect it to rebase frequently.
>
> We also have a clone of Greg's staging branch on our Gerrit server,
> which could be replaced with a "vanilla" upstream kernel clone that
> holds a copy of the for-upstream Lustre tree.
>
> One benefit of hosting the tree on the Gerrit server is that it is
> much easier to get automated testing of the patches (a lot or a little)
> to ensure that the code doesn't break anything obvious.
>
>> I'm happy to review and, if acceptable, apply patches from other
>> developers.  I have fairly high standards, but if I don't accept your
>> patch I'll explain why and possible help fix it.
>> I'm happy to accept enhancements and new features, but these need
>> to be of a quality that would be accepted upstream.
>> I'm only interested in client-side code at present - nothing that is
>> only used on the server.  I do want to include server-side eventually,
>> but I need some focus for now.
>> 
>> I hope to get to a stage where the code is of suitable quality that I
>> can submit it to Linux as a new filesystem.  I hope that will happen
>> this year.
>
> I think one major drawback of starting with the from-staging version of
> the code is that this is significantly behind the out-of-tree master
> code and would need a lot of effort to catch up.
>
> I think the only way we are going to make this work is if the cleanups
> go into the same repo as the ongoing development.  Otherwise, we'll be
> back in the same situation as we are now where there are two different
> versions of the code and we need to move patches back and forth between
> them.
>
> On the plus side, this will avoid code drift, and the extra efforts of
> porting patches back and forth.  On the minus side, it means (on some
> fronts) that the code will regress from what is upstream, because we
> don't have all of the trivial whitespace cleanups and other drive-by
> kernel newbie patches that went into the upstream client.  However, it
> does mean that those cleanups would now become part of the single code
> tree and not be lost again in the future.
>
> This would also mean some restructuring of the current Lustre tree so
> that it could build as a drop-in at linux/fs/lustre but that is probably
> a good idea for the long term in any case.

One of the difficulties I imagine with a single tree is that the current
tree has lots of backward compatibility support to allow the code to
compile on old kernels.  Having to maintain that would seem to interfere
with getting the code into an upstreamable state.
If the lustre development model could change to be working primarily in
a tree what was kept in-sync with upstream and which didn't carry
back-compat code, then I think we might be able to do development and
cleanup in the same tree.
Every other linux project (that I know of) works this way.  Patches are
developed in an upstream kernel, then back-ported to other kernels as
needed.  Enterprise distros do this all the time and are quite good at
it.

Can we move all the kernel code out of the lustre tree and into a clone
of Linux??

Thanks,
NeilBrown


>
> Cheers, Andreas
> --
> Andreas Dilger
> Lustre Principal Architect
> Intel Corporation
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180608/da30e556/attachment.sig>

From neilb at suse.com  Fri Jun  8 06:59:47 2018
From: neilb at suse.com (NeilBrown)
Date: Fri, 08 Jun 2018 16:59:47 +1000
Subject: [lustre-devel] [PATCH 10/11] staging: lustre: move ldlm into
 ptlrpc
In-Reply-To: <87sh5zlyf4.fsf@notabene.neil.brown.name>
References: <152826510267.16761.14361003167157833896.stgit@noble>
 <152826511923.16761.9237280635711887801.stgit@noble>
 <alpine.LFD.2.21.1806070546210.23188@casper.infradead.org>
 <87sh5zlyf4.fsf@notabene.neil.brown.name>
Message-ID: <87tvqdlq58.fsf@notabene.neil.brown.name>

On Thu, Jun 07 2018, NeilBrown wrote:

> On Thu, Jun 07 2018, James Simmons wrote:
>
>>> The ldlm code is built into the ptlrpc module, yet it lived in a
>>> separate directory.  This requires filename editing in the Makefile
>>> and make it difficult to e.g. build the .s file for code in ldlm.
>>> 
>>> All the ldlm files have distinctive names so confusion from having
>>> ptlrpc and ldlm in the same directory is unlikely.  So move them all
>>> into ptlrpc.
>>
>> Nak. The reason is it would be nice to keep the directory structure.
>> What really needs to be done and Oleg has looked into it is to reduced
>> the number of modules created down to two, one for LNet and the other
>> lustre.ko. This also is a step in the right direction to remove the
>> create struct obd_ops and struct md_ops pointer madness. Well their
>> is the issue with obd echo client but we can deal with this at a later 
>> date. Also the number of EXPORT_SYMBOLS and things will greatly reduce.
>
> Yeah, you are probably right.
> I had a bit of a look at how to build everything into a
> single module.  You can do with by having a single make
> file that lists parts from other directories - the same way
> that ptlrpc includes files from ldlm - but that is rather ugly.
>
> I've very nearly got it working using the lib-y infrastructure.
> I can build lnet as a single module, but the dependency calc isn't
> quite right so things happen in the wrong order.  The build
> fails the first time because some files don't exist, then
> succeeds on the second run.
> Hopefully I'll figure out how to make it work tomorrow.

I needed lots of changes over my original hack, but I think I have
it quite close now.
This is just a dump of what I have. It needs to be properly split up
and documented but I thought it would be useful for people do see how
I think this should be done.

Obviously there is a big change to non-lustre code here.  That might
take a bit of selling upstream, but I think it is worth trying.  We
aren't the only project that would benefit from this.

What I have done is created infrastructure so that a Makefile
can declare that some .o files are built for use in a module and
placed in "mod.a".  Then another Makefile can declare that a module
includes mod.a from some other directory.
So now lustre/lnet and lustre/lustre each have a module.c and build a
mod.a from various subdirectories, and combine these into a single
module.

As yet this is only compile tested.  I suspect it will work, but I won't
find out until next week.

NeilBrown


diff --git a/scripts/Makefile.build b/scripts/Makefile.build
index 8bdb1dc4072c..9ae14ecb1e32 100644
--- a/scripts/Makefile.build
+++ b/scripts/Makefile.build
@@ -15,6 +15,8 @@ obj-y :=
 obj-m :=
 lib-y :=
 lib-m :=
+modobj-y :=
+modobj-m :=
 always :=
 targets :=
 subdir-y :=
@@ -80,12 +82,16 @@ ifneq ($(strip $(real-obj-y) $(need-builtin)),)
 builtin-target := $(obj)/built-in.a
 endif
 
+ifneq ($(strip $(modobj-m)),)
+modobj-target := $(obj)/mod.a
+endif
+
 modorder-target := $(obj)/modules.order
 
 # We keep a list of all modules in $(MODVERDIR)
 
 __build: $(if $(KBUILD_BUILTIN),$(builtin-target) $(lib-target) $(extra-y)) \
-	 $(if $(KBUILD_MODULES),$(obj-m) $(modorder-target)) \
+	 $(if $(KBUILD_MODULES),$(obj-m) $(modobj-target) $(modorder-target)) \
 	 $(subdir-ym) $(always)
 	@:
 
@@ -119,17 +125,19 @@ modkern_cflags =                                          \
 		$(KBUILD_CFLAGS_KERNEL) $(CFLAGS_KERNEL))
 quiet_modtag := $(empty)   $(empty)
 
-$(real-obj-m)        : part-of-module := y
-$(real-obj-m:.o=.i)  : part-of-module := y
-$(real-obj-m:.o=.s)  : part-of-module := y
-$(real-obj-m:.o=.lst): part-of-module := y
+_mod_obj = $(real-obj-m) $(real-modobj-m)
+
+$(_mod_obj)        : part-of-module := y
+$(_mod_obj:.o=.i)  : part-of-module := y
+$(_mod_obj:.o=.s)  : part-of-module := y
+$(_mod_obj:.o=.lst): part-of-module := y
 
-$(real-obj-m)        : quiet_modtag := [M]
-$(real-obj-m:.o=.i)  : quiet_modtag := [M]
-$(real-obj-m:.o=.s)  : quiet_modtag := [M]
-$(real-obj-m:.o=.lst): quiet_modtag := [M]
+$(_mod_obj)        : quiet_modtag := [M]
+$(_mod_obj:.o=.i)  : quiet_modtag := [M]
+$(_mod_obj:.o=.s)  : quiet_modtag := [M]
+$(_mod_obj:.o=.lst): quiet_modtag := [M]
 
-$(obj-m)             : quiet_modtag := [M]
+$(obj-m)           : quiet_modtag := [M]
 
 quiet_cmd_cc_s_c = CC $(quiet_modtag)  $@
 cmd_cc_s_c       = $(CC) $(c_flags) $(DISABLE_LTO) -fverbose-asm -S -o $@ $<
@@ -508,6 +516,22 @@ targets += $(obj)/lib-ksyms.o
 
 endif
 
+ifdef modobj-target
+
+quiet_cmd_ar_modobj = AR      $@
+      cmd_ar_modobj = rm -f $@; \
+                    $(AR) rcTP$(KBUILD_ARFLAGS) $@ $(filter $(real-modobj-m), $^)
+
+$(modobj-target): $(real-modobj-m) FORCE
+	$(call if_changed,ar_modobj)
+
+targets += $(modobj-target)
+
+# to build a mod.a, build the directory
+$(filter %/mod.a, $(modobj-m)) : %/mod.a : %
+
+endif # modobj-target
+
 #
 # Rule to link composite objects
 #
@@ -525,7 +549,7 @@ $($(subst $(obj)/,,$(@:.o=-y)))       \
 $($(subst $(obj)/,,$(@:.o=-m)))), $^)
 
 quiet_cmd_link_multi-m = LD [M]  $@
-cmd_link_multi-m = $(LD) $(ld_flags) -r -o $@ $(link_multi_deps) $(cmd_secanalysis)
+cmd_link_multi-m = $(LD) $(ld_flags) -r -o $@ $(patsubst %/mod.a, --whole-archive %/mod.a --no-whole-archive , $(link_multi_deps)) $(cmd_secanalysis)
 
 $(multi-used-m): FORCE
 	$(call if_changed,link_multi-m)
diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
index 5af34a2b0cd9..315590b841b6 100644
--- a/scripts/Makefile.lib
+++ b/scripts/Makefile.lib
@@ -12,9 +12,13 @@ KBUILD_CFLAGS += $(subdir-ccflags-y)
 # Figure out what we need to build from the various variables
 # ===========================================================================
 
+# objects in modobj-y are treated identically to obj-y
+obj-y := $(obj-y) $(filter-out $(modobj-y), $(obj-y))
+
 # When an object is listed to be built compiled-in and modular,
 # only build the compiled-in version
 obj-m := $(filter-out $(obj-y),$(obj-m))
+modobj-m := $(filter-out $(obj-m), $(modobj-m))
 
 # Libraries are always collected in one lib file.
 # Filter out objects already built-in
@@ -31,12 +35,17 @@ modorder	:= $(patsubst %/,%/modules.order, $(filter %/, $(obj-y)) $(obj-m:.o=.ko
 #   and add the directory to the list of dirs to descend into: $(subdir-y)
 # o if we encounter foo/ in $(obj-m), remove it from $(obj-m)
 #   and add the directory to the list of dirs to descend into: $(subdir-m)
+# o if we encounter foo/ in $(modobj-m), replace it by foo/mod.a
+#   and add the directory to the list of dirs to descend into: $(subdir-m)
 __subdir-y	:= $(patsubst %/,%,$(filter %/, $(obj-y)))
 subdir-y	+= $(__subdir-y)
 __subdir-m	:= $(patsubst %/,%,$(filter %/, $(obj-m)))
 subdir-m	+= $(__subdir-m)
+__subdir-m	:= $(patsubst %/,%,$(filter %/, $(modobj-m)))
+subdir-m	+= $(__subdir-m)
 obj-y		:= $(patsubst %/, %/built-in.a, $(obj-y))
 obj-m		:= $(filter-out %/, $(obj-m))
+modobj-m	:= $(patsubst %/, %/mod.a, $(modobj-m))
 
 # Subdirectories we need to descend into
 subdir-ym	:= $(sort $(subdir-y) $(subdir-m))
@@ -48,13 +57,15 @@ multi-used   := $(multi-used-y) $(multi-used-m)
 single-used-m := $(sort $(filter-out $(multi-used-m),$(obj-m)))
 
 # $(subdir-obj-y) is the list of objects in $(obj-y) which uses dir/ to
-# tell kbuild to descend
+# tell kbuild to descend.  Similarly $(subdir-modobj-m) for $(modobj-m)
 subdir-obj-y := $(filter %/built-in.a, $(obj-y))
+subdir-modobj-m := $(filter %/mod.a, $(modobj-m))
 
 # Replace multi-part objects by their individual parts,
 # including built-in.a from subdirectories
 real-obj-y := $(foreach m, $(obj-y), $(if $(strip $($(m:.o=-objs)) $($(m:.o=-y))),$($(m:.o=-objs)) $($(m:.o=-y)),$(m)))
 real-obj-m := $(foreach m, $(obj-m), $(if $(strip $($(m:.o=-objs)) $($(m:.o=-y)) $($(m:.o=-m))),$($(m:.o=-objs)) $($(m:.o=-y)) $($(m:.o=-m)),$(m)))
+real-modobj-m := $(foreach m, $(modobj-m), $(if $(strip $($(m:.o=-objs)) $($(m:.o=-y)) $($(m:.o=-m))),$($(m:.o=-objs)) $($(m:.o=-y)) $($(m:.o=-m)),$(m)))
 
 # DTB
 # If CONFIG_OF_ALL_DTBS is enabled, all DT blobs are built
@@ -68,10 +79,12 @@ always		:= $(addprefix $(obj)/,$(always))
 targets		:= $(addprefix $(obj)/,$(targets))
 modorder	:= $(addprefix $(obj)/,$(modorder))
 obj-m		:= $(addprefix $(obj)/,$(obj-m))
+modobj-m	:= $(addprefix $(obj)/,$(modobj-m))
 lib-y		:= $(addprefix $(obj)/,$(lib-y))
 subdir-obj-y	:= $(addprefix $(obj)/,$(subdir-obj-y))
 real-obj-y	:= $(addprefix $(obj)/,$(real-obj-y))
 real-obj-m	:= $(addprefix $(obj)/,$(real-obj-m))
+real-modobj-m	:= $(addprefix $(obj)/,$(real-modobj-m))
 single-used-m	:= $(addprefix $(obj)/,$(single-used-m))
 multi-used-m	:= $(addprefix $(obj)/,$(multi-used-m))
 subdir-ym	:= $(addprefix $(obj)/,$(subdir-ym))
diff --git a/drivers/staging/lustre/include/linux/lnet/lib-lnet.h b/drivers/staging/lustre/include/linux/lnet/lib-lnet.h
index 973c17a1c4a1..916fb78fd833 100644
--- a/drivers/staging/lustre/include/linux/lnet/lib-lnet.h
+++ b/drivers/staging/lustre/include/linux/lnet/lib-lnet.h
@@ -76,6 +76,14 @@ extern struct lnet the_lnet;	/* THE network */
 #define LNET_ACCEPTOR_MIN_RESERVED_PORT    512
 #define LNET_ACCEPTOR_MAX_RESERVED_PORT    1023
 
+int libcfs_init(void);
+int ko2iblnd_init(void);
+int ksocklnd_init(void);
+
+void libcfs_exit(void);
+void ko2iblnd_exit(void);
+void ksocklnd_exit(void);
+
 static inline int lnet_is_route_alive(struct lnet_route *route)
 {
 	/* gateway is down */
diff --git a/drivers/staging/lustre/lnet/Makefile b/drivers/staging/lustre/lnet/Makefile
index 0a380fe88ce8..058243c7a080 100644
--- a/drivers/staging/lustre/lnet/Makefile
+++ b/drivers/staging/lustre/lnet/Makefile
@@ -1 +1,10 @@
-obj-$(CONFIG_LNET) += libcfs/ lnet/ klnds/ selftest/
+# SPDX-License-Identifier: GPL-2.0
+
+subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/include
+subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/lustre/include
+
+obj-$(CONFIG_LNET) += selftest/
+modobj-$(CONFIG_LNET) += libcfs/ lnet/ klnds/
+
+obj-$(CONFIG_LNET) += lnet.o
+lnet-objs = module.o mod.a
diff --git a/drivers/staging/lustre/lnet/klnds/Makefile b/drivers/staging/lustre/lnet/klnds/Makefile
index c23e4f67f837..79dd66a6a7a1 100644
--- a/drivers/staging/lustre/lnet/klnds/Makefile
+++ b/drivers/staging/lustre/lnet/klnds/Makefile
@@ -1 +1 @@
-obj-$(CONFIG_LNET) += o2iblnd/  socklnd/
+modobj-$(CONFIG_LNET) += o2iblnd/  socklnd/
diff --git a/drivers/staging/lustre/lnet/klnds/o2iblnd/Makefile b/drivers/staging/lustre/lnet/klnds/o2iblnd/Makefile
index 4affe1d79948..6a0eb81cccfe 100644
--- a/drivers/staging/lustre/lnet/klnds/o2iblnd/Makefile
+++ b/drivers/staging/lustre/lnet/klnds/o2iblnd/Makefile
@@ -1,5 +1,4 @@
 subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/include
 subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/lustre/include
 
-obj-$(CONFIG_LNET_XPRT_IB) += ko2iblnd.o
-ko2iblnd-y := o2iblnd.o o2iblnd_cb.o o2iblnd_modparams.o
+modobj-$(CONFIG_LNET) := o2iblnd.o o2iblnd_cb.o o2iblnd_modparams.o
diff --git a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c
index f0b4eb42bc1d..4098c4566fce 100644
--- a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c
+++ b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c
@@ -2921,15 +2921,13 @@ static struct lnet_lnd the_o2iblnd = {
 	.lnd_recv	= kiblnd_recv,
 };
 
-static void __exit ko2iblnd_exit(void)
+void __exit ko2iblnd_exit(void)
 {
 	lnet_unregister_lnd(&the_o2iblnd);
 }
 
-static int __init ko2iblnd_init(void)
+int __init ko2iblnd_init(void)
 {
-	int rc;
-
 	BUILD_BUG_ON(sizeof(struct kib_msg) > IBLND_MSG_SIZE);
 	BUILD_BUG_ON(offsetof(struct kib_msg,
 			  ibm_u.get.ibgm_rd.rd_frags[IBLND_MAX_RDMA_FRAGS])
@@ -2940,19 +2938,7 @@ static int __init ko2iblnd_init(void)
 
 	kiblnd_tunables_init();
 
-	rc = libcfs_setup();
-	if (rc)
-		return rc;
-
 	lnet_register_lnd(&the_o2iblnd);
 
 	return 0;
 }
-
-MODULE_AUTHOR("OpenSFS, Inc. <http://www.lustre.org/>");
-MODULE_DESCRIPTION("OpenIB gen2 LNet Network Driver");
-MODULE_VERSION("2.7.0");
-MODULE_LICENSE("GPL");
-
-module_init(ko2iblnd_init);
-module_exit(ko2iblnd_exit);
diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/Makefile b/drivers/staging/lustre/lnet/klnds/socklnd/Makefile
index a7da1abfc804..231e1ab5dcf8 100644
--- a/drivers/staging/lustre/lnet/klnds/socklnd/Makefile
+++ b/drivers/staging/lustre/lnet/klnds/socklnd/Makefile
@@ -1,6 +1,4 @@
 subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/include
 subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/lustre/include
 
-obj-$(CONFIG_LNET) += ksocklnd.o
-
-ksocklnd-y := socklnd.o socklnd_cb.o socklnd_proto.o socklnd_modparams.o socklnd_lib.o
+modobj-$(CONFIG_LNET) := socklnd.o socklnd_cb.o socklnd_proto.o socklnd_modparams.o socklnd_lib.o
diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c
index f01b34ac1a53..d892bdef7133 100644
--- a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c
+++ b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c
@@ -2875,12 +2875,12 @@ ksocknal_startup(struct lnet_ni *ni)
 	return -ENETDOWN;
 }
 
-static void __exit ksocklnd_exit(void)
+void __exit ksocklnd_exit(void)
 {
 	lnet_unregister_lnd(&the_ksocklnd);
 }
 
-static int __init ksocklnd_init(void)
+int __init ksocklnd_init(void)
 {
 	int rc;
 
@@ -2903,19 +2903,7 @@ static int __init ksocklnd_init(void)
 	if (rc)
 		return rc;
 
-	rc = libcfs_setup();
-	if (rc)
-		return rc;
-
 	lnet_register_lnd(&the_ksocklnd);
 
 	return 0;
 }
-
-MODULE_AUTHOR("OpenSFS, Inc. <http://www.lustre.org/>");
-MODULE_DESCRIPTION("TCP Socket LNet Network Driver");
-MODULE_VERSION("2.7.0");
-MODULE_LICENSE("GPL");
-
-module_init(ksocklnd_init);
-module_exit(ksocklnd_exit);
diff --git a/drivers/staging/lustre/lnet/libcfs/Makefile b/drivers/staging/lustre/lnet/libcfs/Makefile
index 6a1b232da495..6f169a7b35ae 100644
--- a/drivers/staging/lustre/lnet/libcfs/Makefile
+++ b/drivers/staging/lustre/lnet/libcfs/Makefile
@@ -2,8 +2,6 @@
 subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/include
 subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/lustre/include
 
-obj-$(CONFIG_LNET) += libcfs.o
-
 libcfs-obj-y += linux-tracefile.o linux-debug.o
 libcfs-obj-y += linux-crypto.o
 libcfs-obj-y += linux-crypto-adler.o
@@ -13,4 +11,4 @@ libcfs-obj-y += libcfs_string.o hash.o
 libcfs-obj-$(CONFIG_SMP) += libcfs_cpu.o
 libcfs-obj-y += libcfs_mem.o libcfs_lock.o
 
-libcfs-objs := $(libcfs-obj-y)
+modobj-$(CONFIG_LNET) += $(libcfs-obj-y)
diff --git a/drivers/staging/lustre/lnet/libcfs/module.c b/drivers/staging/lustre/lnet/libcfs/module.c
index 02c404c6738e..b4ec1ebabc6d 100644
--- a/drivers/staging/lustre/lnet/libcfs/module.c
+++ b/drivers/staging/lustre/lnet/libcfs/module.c
@@ -688,7 +688,7 @@ int libcfs_setup(void)
 }
 EXPORT_SYMBOL(libcfs_setup);
 
-static int libcfs_init(void)
+int libcfs_init(void)
 {
 	int rc;
 
@@ -700,7 +700,7 @@ static int libcfs_init(void)
 	return rc;
 }
 
-static void libcfs_exit(void)
+void libcfs_exit(void)
 {
 	int rc;
 
@@ -720,11 +720,3 @@ static void libcfs_exit(void)
 	if (rc)
 		pr_err("LustreError: libcfs_debug_cleanup: %d\n", rc);
 }
-
-MODULE_AUTHOR("OpenSFS, Inc. <http://www.lustre.org/>");
-MODULE_DESCRIPTION("Lustre helper library");
-MODULE_VERSION(LIBCFS_VERSION);
-MODULE_LICENSE("GPL");
-
-module_init(libcfs_init);
-module_exit(libcfs_exit);
diff --git a/drivers/staging/lustre/lnet/lnet/Makefile b/drivers/staging/lustre/lnet/lnet/Makefile
index 0a9d70924fe0..f9f4eb125f6d 100644
--- a/drivers/staging/lustre/lnet/lnet/Makefile
+++ b/drivers/staging/lustre/lnet/lnet/Makefile
@@ -2,9 +2,7 @@
 subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/include
 subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/lustre/include
 
-obj-$(CONFIG_LNET) += lnet.o
-
-lnet-y := api-ni.o config.o nidstrings.o net_fault.o		\
+modobj-$(CONFIG_LNET) := api-ni.o config.o nidstrings.o net_fault.o		\
 	  lib-me.o lib-msg.o lib-eq.o lib-md.o lib-ptl.o	\
-	  lib-socket.o lib-move.o module.o lo.o			\
+	  lib-socket.o lib-move.o lo.o			\
 	  router.o router_proc.o acceptor.o peer.o
diff --git a/drivers/staging/lustre/lnet/lnet/module.c b/drivers/staging/lustre/lnet/lnet/module.c
deleted file mode 100644
index 9d06664f0c17..000000000000
--- a/drivers/staging/lustre/lnet/lnet/module.c
+++ /dev/null
@@ -1,239 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-/*
- * GPL HEADER START
- *
- * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 only,
- * as published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful, but
- * WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
- * General Public License version 2 for more details (a copy is included
- * in the LICENSE file that accompanied this code).
- *
- * You should have received a copy of the GNU General Public License
- * version 2 along with this program; If not, see
- * http://www.gnu.org/licenses/gpl-2.0.html
- *
- * GPL HEADER END
- */
-/*
- * Copyright (c) 2004, 2010, Oracle and/or its affiliates. All rights reserved.
- * Use is subject to license terms.
- *
- * Copyright (c) 2012, 2015, Intel Corporation.
- */
-/*
- * This file is part of Lustre, http://www.lustre.org/
- * Lustre is a trademark of Sun Microsystems, Inc.
- */
-
-#define DEBUG_SUBSYSTEM S_LNET
-
-#include <linux/lnet/lib-lnet.h>
-#include <uapi/linux/lnet/lnet-dlc.h>
-
-static int config_on_load;
-module_param(config_on_load, int, 0444);
-MODULE_PARM_DESC(config_on_load, "configure network at module load");
-
-static struct mutex lnet_config_mutex;
-
-static int
-lnet_configure(void *arg)
-{
-	/* 'arg' only there so I can be passed to cfs_create_thread() */
-	int rc = 0;
-
-	mutex_lock(&lnet_config_mutex);
-
-	if (!the_lnet.ln_niinit_self) {
-		rc = try_module_get(THIS_MODULE);
-
-		if (rc != 1)
-			goto out;
-
-		rc = LNetNIInit(LNET_PID_LUSTRE);
-		if (rc >= 0) {
-			the_lnet.ln_niinit_self = 1;
-			rc = 0;
-		} else {
-			module_put(THIS_MODULE);
-		}
-	}
-
-out:
-	mutex_unlock(&lnet_config_mutex);
-	return rc;
-}
-
-static int
-lnet_unconfigure(void)
-{
-	int refcount;
-
-	mutex_lock(&lnet_config_mutex);
-
-	if (the_lnet.ln_niinit_self) {
-		the_lnet.ln_niinit_self = 0;
-		LNetNIFini();
-		module_put(THIS_MODULE);
-	}
-
-	mutex_lock(&the_lnet.ln_api_mutex);
-	refcount = the_lnet.ln_refcount;
-	mutex_unlock(&the_lnet.ln_api_mutex);
-
-	mutex_unlock(&lnet_config_mutex);
-	return !refcount ? 0 : -EBUSY;
-}
-
-static int
-lnet_dyn_configure(struct libcfs_ioctl_hdr *hdr)
-{
-	struct lnet_ioctl_config_data *conf =
-		(struct lnet_ioctl_config_data *)hdr;
-	int rc;
-
-	if (conf->cfg_hdr.ioc_len < sizeof(*conf))
-		return -EINVAL;
-
-	mutex_lock(&lnet_config_mutex);
-	if (!the_lnet.ln_niinit_self) {
-		rc = -EINVAL;
-		goto out_unlock;
-	}
-	rc = lnet_dyn_add_ni(LNET_PID_LUSTRE, conf);
-out_unlock:
-	mutex_unlock(&lnet_config_mutex);
-
-	return rc;
-}
-
-static int
-lnet_dyn_unconfigure(struct libcfs_ioctl_hdr *hdr)
-{
-	struct lnet_ioctl_config_data *conf =
-		(struct lnet_ioctl_config_data *)hdr;
-	int rc;
-
-	if (conf->cfg_hdr.ioc_len < sizeof(*conf))
-		return -EINVAL;
-
-	mutex_lock(&lnet_config_mutex);
-	if (!the_lnet.ln_niinit_self) {
-		rc = -EINVAL;
-		goto out_unlock;
-	}
-	rc = lnet_dyn_del_ni(conf->cfg_net);
-out_unlock:
-	mutex_unlock(&lnet_config_mutex);
-
-	return rc;
-}
-
-static int
-lnet_ioctl(struct notifier_block *nb,
-	   unsigned long cmd, void *vdata)
-{
-	int rc;
-	struct libcfs_ioctl_hdr *hdr = vdata;
-
-	switch (cmd) {
-	case IOC_LIBCFS_CONFIGURE: {
-		struct libcfs_ioctl_data *data =
-			(struct libcfs_ioctl_data *)hdr;
-
-		if (data->ioc_hdr.ioc_len < sizeof(*data)) {
-			rc = -EINVAL;
-		} else {
-			the_lnet.ln_nis_from_mod_params = data->ioc_flags;
-			rc = lnet_configure(NULL);
-		}
-		break;
-	}
-
-	case IOC_LIBCFS_UNCONFIGURE:
-		rc = lnet_unconfigure();
-		break;
-
-	case IOC_LIBCFS_ADD_NET:
-		rc = lnet_dyn_configure(hdr);
-		break;
-
-	case IOC_LIBCFS_DEL_NET:
-		rc = lnet_dyn_unconfigure(hdr);
-		break;
-
-	default:
-		/*
-		 * Passing LNET_PID_ANY only gives me a ref if the net is up
-		 * already; I'll need it to ensure the net can't go down while
-		 * I'm called into it
-		 */
-		rc = LNetNIInit(LNET_PID_ANY);
-		if (rc >= 0) {
-			rc = LNetCtl(cmd, hdr);
-			LNetNIFini();
-		}
-		break;
-	}
-	return notifier_from_ioctl_errno(rc);
-}
-
-static struct notifier_block lnet_ioctl_handler = {
-	.notifier_call = lnet_ioctl,
-};
-
-static int __init lnet_init(void)
-{
-	int rc;
-
-	mutex_init(&lnet_config_mutex);
-
-	rc = libcfs_setup();
-	if (rc)
-		return rc;
-
-	rc = lnet_lib_init();
-	if (rc) {
-		CERROR("lnet_lib_init: error %d\n", rc);
-		return rc;
-	}
-
-	rc = blocking_notifier_chain_register(&libcfs_ioctl_list,
-					      &lnet_ioctl_handler);
-	LASSERT(!rc);
-
-	if (config_on_load) {
-		/*
-		 * Have to schedule a separate thread to avoid deadlocking
-		 * in modload
-		 */
-		(void)kthread_run(lnet_configure, NULL, "lnet_initd");
-	}
-
-	return 0;
-}
-
-static void __exit lnet_exit(void)
-{
-	int rc;
-
-	rc = blocking_notifier_chain_unregister(&libcfs_ioctl_list,
-						&lnet_ioctl_handler);
-	LASSERT(!rc);
-
-	lnet_lib_exit();
-}
-
-MODULE_AUTHOR("OpenSFS, Inc. <http://www.lustre.org/>");
-MODULE_DESCRIPTION("Lustre Networking layer");
-MODULE_VERSION(LNET_VERSION);
-MODULE_LICENSE("GPL");
-
-module_init(lnet_init);
-module_exit(lnet_exit);
diff --git a/drivers/staging/lustre/lnet/module.c b/drivers/staging/lustre/lnet/module.c
new file mode 100644
index 000000000000..2a51ae6542fb
--- /dev/null
+++ b/drivers/staging/lustre/lnet/module.c
@@ -0,0 +1,250 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * GPL HEADER START
+ *
+ * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 only,
+ * as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License version 2 for more details (a copy is included
+ * in the LICENSE file that accompanied this code).
+ *
+ * You should have received a copy of the GNU General Public License
+ * version 2 along with this program; If not, see
+ * http://www.gnu.org/licenses/gpl-2.0.html
+ *
+ * GPL HEADER END
+ */
+/*
+ * Copyright (c) 2004, 2010, Oracle and/or its affiliates. All rights reserved.
+ * Use is subject to license terms.
+ *
+ * Copyright (c) 2012, 2015, Intel Corporation.
+ */
+/*
+ * This file is part of Lustre, http://www.lustre.org/
+ * Lustre is a trademark of Sun Microsystems, Inc.
+ */
+
+#define DEBUG_SUBSYSTEM S_LNET
+
+#include <linux/lnet/lib-lnet.h>
+#include <uapi/linux/lnet/lnet-dlc.h>
+
+static int config_on_load;
+module_param(config_on_load, int, 0444);
+MODULE_PARM_DESC(config_on_load, "configure network at module load");
+
+static struct mutex lnet_config_mutex;
+
+static int
+lnet_configure(void *arg)
+{
+	/* 'arg' only there so I can be passed to cfs_create_thread() */
+	int rc = 0;
+
+	mutex_lock(&lnet_config_mutex);
+
+	if (!the_lnet.ln_niinit_self) {
+		rc = try_module_get(THIS_MODULE);
+
+		if (rc != 1)
+			goto out;
+
+		rc = LNetNIInit(LNET_PID_LUSTRE);
+		if (rc >= 0) {
+			the_lnet.ln_niinit_self = 1;
+			rc = 0;
+		} else {
+			module_put(THIS_MODULE);
+		}
+	}
+
+out:
+	mutex_unlock(&lnet_config_mutex);
+	return rc;
+}
+
+static int
+lnet_unconfigure(void)
+{
+	int refcount;
+
+	mutex_lock(&lnet_config_mutex);
+
+	if (the_lnet.ln_niinit_self) {
+		the_lnet.ln_niinit_self = 0;
+		LNetNIFini();
+		module_put(THIS_MODULE);
+	}
+
+	mutex_lock(&the_lnet.ln_api_mutex);
+	refcount = the_lnet.ln_refcount;
+	mutex_unlock(&the_lnet.ln_api_mutex);
+
+	mutex_unlock(&lnet_config_mutex);
+	return !refcount ? 0 : -EBUSY;
+}
+
+static int
+lnet_dyn_configure(struct libcfs_ioctl_hdr *hdr)
+{
+	struct lnet_ioctl_config_data *conf =
+		(struct lnet_ioctl_config_data *)hdr;
+	int rc;
+
+	if (conf->cfg_hdr.ioc_len < sizeof(*conf))
+		return -EINVAL;
+
+	mutex_lock(&lnet_config_mutex);
+	if (!the_lnet.ln_niinit_self) {
+		rc = -EINVAL;
+		goto out_unlock;
+	}
+	rc = lnet_dyn_add_ni(LNET_PID_LUSTRE, conf);
+out_unlock:
+	mutex_unlock(&lnet_config_mutex);
+
+	return rc;
+}
+
+static int
+lnet_dyn_unconfigure(struct libcfs_ioctl_hdr *hdr)
+{
+	struct lnet_ioctl_config_data *conf =
+		(struct lnet_ioctl_config_data *)hdr;
+	int rc;
+
+	if (conf->cfg_hdr.ioc_len < sizeof(*conf))
+		return -EINVAL;
+
+	mutex_lock(&lnet_config_mutex);
+	if (!the_lnet.ln_niinit_self) {
+		rc = -EINVAL;
+		goto out_unlock;
+	}
+	rc = lnet_dyn_del_ni(conf->cfg_net);
+out_unlock:
+	mutex_unlock(&lnet_config_mutex);
+
+	return rc;
+}
+
+static int
+lnet_ioctl(struct notifier_block *nb,
+	   unsigned long cmd, void *vdata)
+{
+	int rc;
+	struct libcfs_ioctl_hdr *hdr = vdata;
+
+	switch (cmd) {
+	case IOC_LIBCFS_CONFIGURE: {
+		struct libcfs_ioctl_data *data =
+			(struct libcfs_ioctl_data *)hdr;
+
+		if (data->ioc_hdr.ioc_len < sizeof(*data)) {
+			rc = -EINVAL;
+		} else {
+			the_lnet.ln_nis_from_mod_params = data->ioc_flags;
+			rc = lnet_configure(NULL);
+		}
+		break;
+	}
+
+	case IOC_LIBCFS_UNCONFIGURE:
+		rc = lnet_unconfigure();
+		break;
+
+	case IOC_LIBCFS_ADD_NET:
+		rc = lnet_dyn_configure(hdr);
+		break;
+
+	case IOC_LIBCFS_DEL_NET:
+		rc = lnet_dyn_unconfigure(hdr);
+		break;
+
+	default:
+		/*
+		 * Passing LNET_PID_ANY only gives me a ref if the net is up
+		 * already; I'll need it to ensure the net can't go down while
+		 * I'm called into it
+		 */
+		rc = LNetNIInit(LNET_PID_ANY);
+		if (rc >= 0) {
+			rc = LNetCtl(cmd, hdr);
+			LNetNIFini();
+		}
+		break;
+	}
+	return notifier_from_ioctl_errno(rc);
+}
+
+static struct notifier_block lnet_ioctl_handler = {
+	.notifier_call = lnet_ioctl,
+};
+
+static int __init lnet_init(void)
+{
+	int rc;
+
+	rc = libcfs_init();
+	if (rc)
+		return rc;
+
+	mutex_init(&lnet_config_mutex);
+
+	rc = libcfs_setup();
+	if (rc)
+		return rc;
+
+	rc = lnet_lib_init();
+	if (rc) {
+		CERROR("lnet_lib_init: error %d\n", rc);
+		return rc;
+	}
+
+	rc = blocking_notifier_chain_register(&libcfs_ioctl_list,
+					      &lnet_ioctl_handler);
+	LASSERT(!rc);
+
+	if (config_on_load) {
+		/*
+		 * Have to schedule a separate thread to avoid deadlocking
+		 * in modload
+		 */
+		(void)kthread_run(lnet_configure, NULL, "lnet_initd");
+	}
+
+	ko2iblnd_init();
+	ksocklnd_init();
+	return 0;
+}
+
+static void __exit lnet_exit(void)
+{
+	int rc;
+
+	ksocklnd_exit();
+	ko2iblnd_exit();
+
+	rc = blocking_notifier_chain_unregister(&libcfs_ioctl_list,
+						&lnet_ioctl_handler);
+	LASSERT(!rc);
+
+	lnet_lib_exit();
+
+	libcfs_exit();
+}
+
+MODULE_AUTHOR("OpenSFS, Inc. <http://www.lustre.org/>");
+MODULE_DESCRIPTION("Lustre Networking layer");
+MODULE_VERSION(LNET_VERSION);
+MODULE_LICENSE("GPL");
+
+module_init(lnet_init);
+module_exit(lnet_exit);
diff --git a/drivers/staging/lustre/lustre/Makefile b/drivers/staging/lustre/lustre/Makefile
index 331e4fcdd5a2..81eb4243afd3 100644
--- a/drivers/staging/lustre/lustre/Makefile
+++ b/drivers/staging/lustre/lustre/Makefile
@@ -1,2 +1,11 @@
-obj-$(CONFIG_LUSTRE_FS) += obdclass/ ptlrpc/ fld/ osc/ mgc/ \
-			   fid/ lov/ mdc/ lmv/ llite/ obdecho/
+# SPDX-License-Identifier: GPL-2.0
+
+subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/include
+subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/lustre/include
+
+modobj-$(CONFIG_LUSTRE_FS) += obdclass/ ldlm/ ptlrpc/ fld/ osc/ mgc/ \
+			   fid/ lov/ mdc/ lmv/ llite/
+obj-$(CONFIG_LUSTRE_FS) += obdecho/
+
+obj-$(CONFIG_LUSTRE_FS) += lustre.o
+lustre-objs = module.o mod.a
diff --git a/drivers/staging/lustre/lustre/fid/Makefile b/drivers/staging/lustre/lustre/fid/Makefile
index 77b65b92667d..c3f2c683c06e 100644
--- a/drivers/staging/lustre/lustre/fid/Makefile
+++ b/drivers/staging/lustre/lustre/fid/Makefile
@@ -1,5 +1,4 @@
 subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/include
 subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/lustre/include/
 
-obj-$(CONFIG_LUSTRE_FS) += fid.o
-fid-y := fid_request.o fid_lib.o lproc_fid.o
+modobj-$(CONFIG_LUSTRE_FS) += fid_request.o fid_lib.o lproc_fid.o
diff --git a/drivers/staging/lustre/lustre/fid/fid_request.c b/drivers/staging/lustre/lustre/fid/fid_request.c
index a34fd90ca5e5..0e8d92701204 100644
--- a/drivers/staging/lustre/lustre/fid/fid_request.c
+++ b/drivers/staging/lustre/lustre/fid/fid_request.c
@@ -383,7 +383,7 @@ int client_fid_fini(struct obd_device *obd)
 }
 EXPORT_SYMBOL(client_fid_fini);
 
-static int __init fid_init(void)
+int __init fid_init(void)
 {
 	int rc;
 
@@ -396,15 +396,7 @@ static int __init fid_init(void)
 	return 0;
 }
 
-static void __exit fid_exit(void)
+void __exit fid_exit(void)
 {
 	debugfs_remove_recursive(seq_debugfs_dir);
 }
-
-MODULE_AUTHOR("OpenSFS, Inc. <http://www.lustre.org/>");
-MODULE_DESCRIPTION("Lustre File IDentifier");
-MODULE_VERSION(LUSTRE_VERSION_STRING);
-MODULE_LICENSE("GPL");
-
-module_init(fid_init);
-module_exit(fid_exit);
diff --git a/drivers/staging/lustre/lustre/fld/Makefile b/drivers/staging/lustre/lustre/fld/Makefile
index 426deba8b815..37097a4421a0 100644
--- a/drivers/staging/lustre/lustre/fld/Makefile
+++ b/drivers/staging/lustre/lustre/fld/Makefile
@@ -1,5 +1,4 @@
 subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/include
 subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/lustre/include/
 
-obj-$(CONFIG_LUSTRE_FS) += fld.o
-fld-y := fld_request.o fld_cache.o lproc_fld.o
+modobj-$(CONFIG_LUSTRE_FS) += fld_request.o fld_cache.o lproc_fld.o
diff --git a/drivers/staging/lustre/lustre/fld/fld_request.c b/drivers/staging/lustre/lustre/fld/fld_request.c
index 97f7ea632346..21e7dbc65cd3 100644
--- a/drivers/staging/lustre/lustre/fld/fld_request.c
+++ b/drivers/staging/lustre/lustre/fld/fld_request.c
@@ -419,7 +419,7 @@ void fld_client_flush(struct lu_client_fld *fld)
 	fld_cache_flush(fld->lcf_cache);
 }
 
-static int __init fld_init(void)
+int __init fld_init(void)
 {
 	int rc;
 
@@ -432,15 +432,7 @@ static int __init fld_init(void)
 	return 0;
 }
 
-static void __exit fld_exit(void)
+void __exit fld_exit(void)
 {
 	debugfs_remove_recursive(fld_debugfs_dir);
 }
-
-MODULE_AUTHOR("OpenSFS, Inc. <http://www.lustre.org/>");
-MODULE_DESCRIPTION("Lustre FID Location Database");
-MODULE_VERSION(LUSTRE_VERSION_STRING);
-MODULE_LICENSE("GPL");
-
-module_init(fld_init)
-module_exit(fld_exit)
diff --git a/drivers/staging/lustre/lustre/include/lustre_compat.h b/drivers/staging/lustre/lustre/include/lustre_compat.h
index 3c6db0d632dc..3e4343e4740a 100644
--- a/drivers/staging/lustre/lustre/include/lustre_compat.h
+++ b/drivers/staging/lustre/lustre/include/lustre_compat.h
@@ -50,16 +50,6 @@
 #define current_ngroups current_cred()->group_info->ngroups
 #define current_groups current_cred()->group_info->small_block
 
-/*
- * OBD need working random driver, thus all our
- * initialization routines must be called after device
- * driver initialization
- */
-#ifndef MODULE
-#undef module_init
-#define module_init(a)     late_initcall(a)
-#endif
-
 #define LTIME_S(time)		   (time.tv_sec)
 
 #ifndef QUOTA_OK
diff --git a/drivers/staging/lustre/lustre/include/lustre_lib.h b/drivers/staging/lustre/lustre/include/lustre_lib.h
index 87748e9902a7..16fa74caed37 100644
--- a/drivers/staging/lustre/lustre/include/lustre_lib.h
+++ b/drivers/staging/lustre/lustre/include/lustre_lib.h
@@ -85,6 +85,25 @@ static inline int l_fatal_signal_pending(struct task_struct *p)
 
 /** @} lib */
 
+int fid_init(void);
+int fld_init(void);
+int lmv_init(void);
+int lov_init(void);
+int mdc_init(void);
+int mgc_init(void);
+int obdclass_init(void);
+int osc_init(void);
+int ptlrpc_init(void);
+
+void fid_exit(void);
+void fld_exit(void);
+void lmv_exit(void);
+void lov_exit(void);
+void mdc_exit(void);
+void mgc_exit(void);
+void obdclass_exit(void);
+void osc_exit(void);
+void ptlrpc_exit(void);
 
 
 /* l_wait_event_abortable() is a bit like wait_event_killable()
diff --git a/drivers/staging/lustre/lustre/ldlm/Makefile b/drivers/staging/lustre/lustre/ldlm/Makefile
new file mode 100644
index 000000000000..7dc83d1f3416
--- /dev/null
+++ b/drivers/staging/lustre/lustre/ldlm/Makefile
@@ -0,0 +1,12 @@
+# SPDX-License-Identifier: GPL-2.0
+subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/include
+subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/lustre/include
+
+ldlm_objs := l_lock.o ldlm_lock.o
+ldlm_objs += ldlm_resource.o ldlm_lib.o
+ldlm_objs += ldlm_plain.o ldlm_extent.o
+ldlm_objs += ldlm_request.o ldlm_lockd.o
+ldlm_objs += ldlm_flock.o ldlm_inodebits.o
+ldlm_objs += ldlm_pool.o
+
+modobj-$(CONFIG_LUSTRE_FS) += $(ldlm_objs)
diff --git a/drivers/staging/lustre/lustre/llite/Makefile b/drivers/staging/lustre/lustre/llite/Makefile
index 5200924182ae..ec0877d8f65e 100644
--- a/drivers/staging/lustre/lustre/llite/Makefile
+++ b/drivers/staging/lustre/lustre/llite/Makefile
@@ -2,7 +2,6 @@
 subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/include
 subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/lustre/include
 
-obj-$(CONFIG_LUSTRE_FS) += lustre.o
 lustre-y := dcache.o dir.o file.o llite_lib.o llite_nfs.o \
 	    rw.o rw26.o namei.o symlink.o llite_mmap.o range_lock.o \
 	    xattr.o xattr_cache.o xattr_security.o \
@@ -11,3 +10,5 @@ lustre-y := dcache.o dir.o file.o llite_lib.o llite_nfs.o \
 	    lproc_llite.o
 
 lustre-$(CONFIG_FS_POSIX_ACL) += acl.o
+
+modobj-$(CONFIG_LUSTRE_FS) += $(lustre-y)
diff --git a/drivers/staging/lustre/lustre/llite/super25.c b/drivers/staging/lustre/lustre/llite/super25.c
index d335f29556c2..8f81ae4e6998 100644
--- a/drivers/staging/lustre/lustre/llite/super25.c
+++ b/drivers/staging/lustre/lustre/llite/super25.c
@@ -81,9 +81,8 @@ struct super_operations lustre_super_operations = {
 	.remount_fs    = ll_remount_fs,
 	.show_options  = ll_show_options,
 };
-MODULE_ALIAS_FS("lustre");
 
-static int __init lustre_init(void)
+int __init lustre_init(void)
 {
 	int rc;
 
@@ -164,7 +163,7 @@ static int __init lustre_init(void)
 	return rc;
 }
 
-static void __exit lustre_exit(void)
+void __exit lustre_exit(void)
 {
 	lustre_register_super_ops(NULL, NULL, NULL);
 	lustre_register_client_process_config(NULL);
@@ -179,11 +178,3 @@ static void __exit lustre_exit(void)
 	kmem_cache_destroy(ll_inode_cachep);
 	kmem_cache_destroy(ll_file_data_slab);
 }
-
-MODULE_AUTHOR("OpenSFS, Inc. <http://www.lustre.org/>");
-MODULE_DESCRIPTION("Lustre Client File System");
-MODULE_VERSION(LUSTRE_VERSION_STRING);
-MODULE_LICENSE("GPL");
-
-module_init(lustre_init);
-module_exit(lustre_exit);
diff --git a/drivers/staging/lustre/lustre/lmv/Makefile b/drivers/staging/lustre/lustre/lmv/Makefile
index 91c99114aa13..4fa01f085f2a 100644
--- a/drivers/staging/lustre/lustre/lmv/Makefile
+++ b/drivers/staging/lustre/lustre/lmv/Makefile
@@ -1,5 +1,4 @@
 subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/include
 subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/lustre/include
 
-obj-$(CONFIG_LUSTRE_FS) += lmv.o
-lmv-y := lmv_obd.o lmv_intent.o lmv_fld.o lproc_lmv.o
+modobj-$(CONFIG_LUSTRE_FS) += lmv_obd.o lmv_intent.o lmv_fld.o lproc_lmv.o
diff --git a/drivers/staging/lustre/lustre/lmv/lmv_obd.c b/drivers/staging/lustre/lustre/lmv/lmv_obd.c
index 65f94e6ecaad..17e825f54668 100644
--- a/drivers/staging/lustre/lustre/lmv/lmv_obd.c
+++ b/drivers/staging/lustre/lustre/lmv/lmv_obd.c
@@ -3102,7 +3102,7 @@ static struct md_ops lmv_md_ops = {
 	.unpackmd		= lmv_unpackmd,
 };
 
-static int __init lmv_init(void)
+int __init lmv_init(void)
 {
 	struct lprocfs_static_vars lvars;
 	int rc;
@@ -3117,15 +3117,7 @@ static int __init lmv_init(void)
 				 LUSTRE_LMV_NAME, NULL);
 }
 
-static void lmv_exit(void)
+void lmv_exit(void)
 {
 	class_unregister_type(LUSTRE_LMV_NAME);
 }
-
-MODULE_AUTHOR("OpenSFS, Inc. <http://www.lustre.org/>");
-MODULE_DESCRIPTION("Lustre Logical Metadata Volume");
-MODULE_VERSION(LUSTRE_VERSION_STRING);
-MODULE_LICENSE("GPL");
-
-module_init(lmv_init);
-module_exit(lmv_exit);
diff --git a/drivers/staging/lustre/lustre/lov/Makefile b/drivers/staging/lustre/lustre/lov/Makefile
index 1ebf0193f61a..a4167566cee6 100644
--- a/drivers/staging/lustre/lustre/lov/Makefile
+++ b/drivers/staging/lustre/lustre/lov/Makefile
@@ -2,8 +2,7 @@
 subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/include
 subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/lustre/include
 
-obj-$(CONFIG_LUSTRE_FS) += lov.o
-lov-y := lov_obd.o lov_pack.o lov_offset.o lov_merge.o \
+modobj-$(CONFIG_LUSTRE_FS) += lov_obd.o lov_pack.o lov_offset.o lov_merge.o \
 	 lov_request.o lov_ea.o lov_dev.o lov_object.o lov_page.o  \
 	 lov_lock.o lov_io.o lovsub_dev.o lovsub_object.o lovsub_page.o      \
 	 lovsub_lock.o lov_pool.o lproc_lov.o
diff --git a/drivers/staging/lustre/lustre/lov/lov_obd.c b/drivers/staging/lustre/lustre/lov/lov_obd.c
index 344ff4b20168..4b2ed026ccf1 100644
--- a/drivers/staging/lustre/lustre/lov/lov_obd.c
+++ b/drivers/staging/lustre/lustre/lov/lov_obd.c
@@ -1388,7 +1388,7 @@ static struct obd_ops lov_obd_ops = {
 
 struct kmem_cache *lov_oinfo_slab;
 
-static int __init lov_init(void)
+int __init lov_init(void)
 {
 	struct lprocfs_static_vars lvars = { NULL };
 	int rc;
@@ -1427,18 +1427,10 @@ static int __init lov_init(void)
 	return rc;
 }
 
-static void /*__exit*/ lov_exit(void)
+void /*__exit*/ lov_exit(void)
 {
 	class_unregister_type(LUSTRE_LOV_NAME);
 	kmem_cache_destroy(lov_oinfo_slab);
 
 	lu_kmem_fini(lov_caches);
 }
-
-MODULE_AUTHOR("OpenSFS, Inc. <http://www.lustre.org/>");
-MODULE_DESCRIPTION("Lustre Logical Object Volume");
-MODULE_LICENSE("GPL");
-MODULE_VERSION(LUSTRE_VERSION_STRING);
-
-module_init(lov_init);
-module_exit(lov_exit);
diff --git a/drivers/staging/lustre/lustre/mdc/Makefile b/drivers/staging/lustre/lustre/mdc/Makefile
index c7bc3351ccb0..6ee93aa9e911 100644
--- a/drivers/staging/lustre/lustre/mdc/Makefile
+++ b/drivers/staging/lustre/lustre/mdc/Makefile
@@ -1,5 +1,4 @@
 subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/include
 subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/lustre/include
 
-obj-$(CONFIG_LUSTRE_FS) += mdc.o
-mdc-y := mdc_request.o mdc_reint.o mdc_lib.o mdc_locks.o lproc_mdc.o
+modobj-$(CONFIG_LUSTRE_FS) += mdc_request.o mdc_reint.o mdc_lib.o mdc_locks.o lproc_mdc.o
diff --git a/drivers/staging/lustre/lustre/mdc/mdc_request.c b/drivers/staging/lustre/lustre/mdc/mdc_request.c
index cff31cb0a9ac..57225bf2bb72 100644
--- a/drivers/staging/lustre/lustre/mdc/mdc_request.c
+++ b/drivers/staging/lustre/lustre/mdc/mdc_request.c
@@ -2741,7 +2741,7 @@ static struct md_ops mdc_md_ops = {
 	.revalidate_lock	= mdc_revalidate_lock
 };
 
-static int __init mdc_init(void)
+int __init mdc_init(void)
 {
 	struct lprocfs_static_vars lvars = { NULL };
 	int rc;
@@ -2756,15 +2756,8 @@ static int __init mdc_init(void)
 				 LUSTRE_MDC_NAME, NULL);
 }
 
-static void /*__exit*/ mdc_exit(void)
+void /*__exit*/ mdc_exit(void)
 {
 	class_unregister_type(LUSTRE_MDC_NAME);
 }
 
-MODULE_AUTHOR("OpenSFS, Inc. <http://www.lustre.org/>");
-MODULE_DESCRIPTION("Lustre Metadata Client");
-MODULE_VERSION(LUSTRE_VERSION_STRING);
-MODULE_LICENSE("GPL");
-
-module_init(mdc_init);
-module_exit(mdc_exit);
diff --git a/drivers/staging/lustre/lustre/mgc/Makefile b/drivers/staging/lustre/lustre/mgc/Makefile
index 8abf108dbcf7..b1555e33fe49 100644
--- a/drivers/staging/lustre/lustre/mgc/Makefile
+++ b/drivers/staging/lustre/lustre/mgc/Makefile
@@ -1,5 +1,4 @@
 subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/include
 subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/lustre/include
 
-obj-$(CONFIG_LUSTRE_FS) += mgc.o
-mgc-y := mgc_request.o lproc_mgc.o
+modobj-$(CONFIG_LUSTRE_FS) += mgc_request.o lproc_mgc.o
diff --git a/drivers/staging/lustre/lustre/mgc/mgc_request.c b/drivers/staging/lustre/lustre/mgc/mgc_request.c
index 32df804614d3..68ef04eddfd2 100644
--- a/drivers/staging/lustre/lustre/mgc/mgc_request.c
+++ b/drivers/staging/lustre/lustre/mgc/mgc_request.c
@@ -1825,7 +1825,7 @@ static struct obd_ops mgc_obd_ops = {
 	.process_config = mgc_process_config,
 };
 
-static int __init mgc_init(void)
+int __init mgc_init(void)
 {
 	int rc;
 
@@ -1837,15 +1837,7 @@ static int __init mgc_init(void)
 				   LUSTRE_MGC_NAME, NULL);
 }
 
-static void /*__exit*/ mgc_exit(void)
+void /*__exit*/ mgc_exit(void)
 {
 	class_unregister_type(LUSTRE_MGC_NAME);
 }
-
-MODULE_AUTHOR("OpenSFS, Inc. <http://www.lustre.org/>");
-MODULE_DESCRIPTION("Lustre Management Client");
-MODULE_VERSION(LUSTRE_VERSION_STRING);
-MODULE_LICENSE("GPL");
-
-module_init(mgc_init);
-module_exit(mgc_exit);
diff --git a/drivers/staging/lustre/lustre/module.c b/drivers/staging/lustre/lustre/module.c
new file mode 100644
index 000000000000..a8a8c9687c95
--- /dev/null
+++ b/drivers/staging/lustre/lustre/module.c
@@ -0,0 +1,61 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * GPL HEADER START
+ *
+ * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 only,
+ * as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License version 2 for more details (a copy is included
+ * in the LICENSE file that accompanied this code).
+ *
+ * You should have received a copy of the GNU General Public License
+ * version 2 along with this program; If not, see
+ * http://www.gnu.org/licenses/gpl-2.0.html
+ *
+ * GPL HEADER END
+ */
+
+#include <linux/module.h>
+#include <lustre_lib.h>
+
+static int __init  lustre_init(void)
+{
+	return obdclass_init() ||
+		ptlrpc_init() ||
+		fld_init() ||
+		osc_init() ||
+		mgc_init() ||
+		fid_init() ||
+		lov_init() ||
+		mdc_init() ||
+		lmv_init();
+}
+
+
+static void __exit lustre_exit(void)
+{
+	lmv_exit();
+	mdc_exit();
+	lov_exit();
+	fid_exit();
+	mgc_exit();
+	osc_exit();
+	fld_exit();
+	ptlrpc_exit();
+	obdclass_exit();
+}
+
+MODULE_ALIAS_FS("lustre");
+MODULE_AUTHOR("OpenSFS, Inc. <http://www.lustre.org/>");
+MODULE_DESCRIPTION("Lustre Client File System");
+MODULE_LICENSE("GPL");
+MODULE_VERSION(LUSTRE_VERSION_STRING);
+
+module_init(lustre_init);
+module_exit(lustre_exit);
diff --git a/drivers/staging/lustre/lustre/obdclass/Makefile b/drivers/staging/lustre/lustre/obdclass/Makefile
index e36ba2167d10..606647bde6a9 100644
--- a/drivers/staging/lustre/lustre/obdclass/Makefile
+++ b/drivers/staging/lustre/lustre/obdclass/Makefile
@@ -2,9 +2,7 @@
 subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/include
 subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/lustre/include
 
-obj-$(CONFIG_LUSTRE_FS) += obdclass.o
-
-obdclass-y := module.o sysctl.o \
+modobj-$(CONFIG_LUSTRE_FS) += module.o sysctl.o \
 	      llog.o llog_cat.o llog_obd.o llog_swab.o class_obd.o debug.o \
 	      genops.o uuid.o lprocfs_status.o lprocfs_counters.o \
 	      lustre_handles.o lustre_peer.o statfs_pack.o linkea.o \
diff --git a/drivers/staging/lustre/lustre/obdclass/class_obd.c b/drivers/staging/lustre/lustre/obdclass/class_obd.c
index d6c46858941b..6f07837b56f5 100644
--- a/drivers/staging/lustre/lustre/obdclass/class_obd.c
+++ b/drivers/staging/lustre/lustre/obdclass/class_obd.c
@@ -444,7 +444,7 @@ static int obd_init_checks(void)
 	return ret;
 }
 
-static int __init obdclass_init(void)
+int __init obdclass_init(void)
 {
 	int i, err;
 
@@ -517,7 +517,7 @@ static int __init obdclass_init(void)
 	return err;
 }
 
-static void obdclass_exit(void)
+void obdclass_exit(void)
 {
 	lustre_unregister_fs();
 
@@ -535,10 +535,3 @@ static void obdclass_exit(void)
 	obd_zombie_impexp_stop();
 }
 
-MODULE_AUTHOR("OpenSFS, Inc. <http://www.lustre.org/>");
-MODULE_DESCRIPTION("Lustre Class Driver");
-MODULE_VERSION(LUSTRE_VERSION_STRING);
-MODULE_LICENSE("GPL");
-
-module_init(obdclass_init);
-module_exit(obdclass_exit);
diff --git a/drivers/staging/lustre/lustre/obdclass/obd_mount.c b/drivers/staging/lustre/lustre/obdclass/obd_mount.c
index 06c38fdef7ba..30a5e34d5ae8 100644
--- a/drivers/staging/lustre/lustre/obdclass/obd_mount.c
+++ b/drivers/staging/lustre/lustre/obdclass/obd_mount.c
@@ -1232,7 +1232,6 @@ static struct file_system_type lustre_fs_type = {
 	.kill_sb	= lustre_kill_super,
 	.fs_flags	= FS_RENAME_DOES_D_MOVE,
 };
-MODULE_ALIAS_FS("lustre");
 
 int lustre_register_fs(void)
 {
diff --git a/drivers/staging/lustre/lustre/osc/Makefile b/drivers/staging/lustre/lustre/osc/Makefile
index 30dec90e64e8..5706a988c471 100644
--- a/drivers/staging/lustre/lustre/osc/Makefile
+++ b/drivers/staging/lustre/lustre/osc/Makefile
@@ -1,6 +1,5 @@
 subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/include
 subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/lustre/include
 
-obj-$(CONFIG_LUSTRE_FS) += osc.o
-osc-y := osc_request.o osc_dev.o osc_object.o \
+modobj-$(CONFIG_LUSTRE_FS) += osc_request.o osc_dev.o osc_object.o \
 	 osc_page.o osc_lock.o osc_io.o osc_quota.o osc_cache.o lproc_osc.o
diff --git a/drivers/staging/lustre/lustre/osc/osc_request.c b/drivers/staging/lustre/lustre/osc/osc_request.c
index 61ef6c8d7a12..46af1ad36f18 100644
--- a/drivers/staging/lustre/lustre/osc/osc_request.c
+++ b/drivers/staging/lustre/lustre/osc/osc_request.c
@@ -2819,7 +2819,7 @@ static struct shrinker osc_cache_shrinker = {
 	.seeks		= DEFAULT_SEEKS,
 };
 
-static int __init osc_init(void)
+int __init osc_init(void)
 {
 	struct lprocfs_static_vars lvars = { NULL };
 	unsigned int reqpool_size;
@@ -2890,7 +2890,7 @@ static int __init osc_init(void)
 	return rc;
 }
 
-static void /*__exit*/ osc_exit(void)
+void /*__exit*/ osc_exit(void)
 {
 	unregister_shrinker(&osc_cache_shrinker);
 	class_unregister_type(LUSTRE_OSC_NAME);
@@ -2898,10 +2898,3 @@ static void /*__exit*/ osc_exit(void)
 	ptlrpc_free_rq_pool(osc_rq_pool);
 }
 
-MODULE_AUTHOR("OpenSFS, Inc. <http://www.lustre.org/>");
-MODULE_DESCRIPTION("Lustre Object Storage Client (OSC)");
-MODULE_LICENSE("GPL");
-MODULE_VERSION(LUSTRE_VERSION_STRING);
-
-module_init(osc_init);
-module_exit(osc_exit);
diff --git a/drivers/staging/lustre/lustre/ptlrpc/Makefile b/drivers/staging/lustre/lustre/ptlrpc/Makefile
index 77f8eabb2e28..0dffec552511 100644
--- a/drivers/staging/lustre/lustre/ptlrpc/Makefile
+++ b/drivers/staging/lustre/lustre/ptlrpc/Makefile
@@ -2,15 +2,6 @@
 subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/include
 subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/lustre/include
 
-obj-$(CONFIG_LUSTRE_FS) += ptlrpc.o
-LDLM := ../../lustre/ldlm/
-
-ldlm_objs := $(LDLM)l_lock.o $(LDLM)ldlm_lock.o
-ldlm_objs += $(LDLM)ldlm_resource.o $(LDLM)ldlm_lib.o
-ldlm_objs += $(LDLM)ldlm_plain.o $(LDLM)ldlm_extent.o
-ldlm_objs += $(LDLM)ldlm_request.o $(LDLM)ldlm_lockd.o
-ldlm_objs += $(LDLM)ldlm_flock.o $(LDLM)ldlm_inodebits.o
-ldlm_objs += $(LDLM)ldlm_pool.o
 ptlrpc_objs := client.o recover.o connection.o niobuf.o pack_generic.o
 ptlrpc_objs += events.o ptlrpc_module.o service.o pinger.o
 ptlrpc_objs += llog_net.o llog_client.o import.o ptlrpcd.o
@@ -18,5 +9,7 @@ ptlrpc_objs += pers.o lproc_ptlrpc.o wiretest.o layout.o
 ptlrpc_objs += sec.o sec_bulk.o sec_gc.o sec_config.o
 ptlrpc_objs += sec_null.o sec_plain.o nrs.o nrs_fifo.o
 
-ptlrpc-y := $(ldlm_objs) $(ptlrpc_objs) sec_lproc.o
+ptlrpc-y := $(ptlrpc_objs) sec_lproc.o
 ptlrpc-$(CONFIG_LUSTRE_TRANSLATE_ERRNOS) += errno.o
+
+modobj-$(CONFIG_LUSTRE_FS) += $(ptlrpc-y)
diff --git a/drivers/staging/lustre/lustre/ptlrpc/ptlrpc_module.c b/drivers/staging/lustre/lustre/ptlrpc/ptlrpc_module.c
index 5c32b657b3b5..1260a7b9cae0 100644
--- a/drivers/staging/lustre/lustre/ptlrpc/ptlrpc_module.c
+++ b/drivers/staging/lustre/lustre/ptlrpc/ptlrpc_module.c
@@ -81,7 +81,7 @@ void ptlrpc_dec_ref(void)
 }
 EXPORT_SYMBOL(ptlrpc_dec_ref);
 
-static int __init ptlrpc_init(void)
+int __init ptlrpc_init(void)
 {
 	int rc, cleanup_phase = 0;
 
@@ -166,7 +166,7 @@ static int __init ptlrpc_init(void)
 	return rc;
 }
 
-static void __exit ptlrpc_exit(void)
+void __exit ptlrpc_exit(void)
 {
 	tgt_mod_exit();
 	ptlrpc_nrs_fini();
@@ -176,11 +176,3 @@ static void __exit ptlrpc_exit(void)
 	ptlrpc_hr_fini();
 	ptlrpc_connection_fini();
 }
-
-MODULE_AUTHOR("OpenSFS, Inc. <http://www.lustre.org/>");
-MODULE_DESCRIPTION("Lustre Request Processor and Lock Management");
-MODULE_VERSION(LUSTRE_VERSION_STRING);
-MODULE_LICENSE("GPL");
-
-module_init(ptlrpc_init);
-module_exit(ptlrpc_exit);
-- 
2.14.0.rc0.dirty

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180608/20bc371a/attachment.sig>

From gregkh at linuxfoundation.org  Mon Jun 11 20:50:58 2018
From: gregkh at linuxfoundation.org (Greg KH)
Date: Mon, 11 Jun 2018 22:50:58 +0200
Subject: [lustre-devel] [PATCH v3] staging: lustre: Change return type
 to vm_fault_t
In-Reply-To: <CAFqt6zamp=+adkyM+_4z405hN-9qOn1CsY_BgVtCrAfscBHhWg@mail.gmail.com>
References: <20180521180937.GA19605@jordon-HP-15-Notebook-PC>
 <CAFqt6zamp=+adkyM+_4z405hN-9qOn1CsY_BgVtCrAfscBHhWg@mail.gmail.com>
Message-ID: <20180611205058.GA30123@kroah.com>

On Tue, Jun 12, 2018 at 02:00:47AM +0530, Souptick Joarder wrote:
> On Mon, May 21, 2018 at 11:39 PM, Souptick Joarder <jrdr.linux at gmail.com> wrote:
> > Use new return type vm_fault_t for fault handler. For
> > now, this is just documenting that the function returns
> > a VM_FAULT value rather than an errno. Once all instances
> > are converted, vm_fault_t will become a distinct type.
> >
> > Ref-> commit 1c8f422059ae ("mm: change return type to
> > vm_fault_t") was added in 4.17-rc1 to introduce the new
> > typedef vm_fault_t. Currently we are making change to all
> > drivers to return vm_fault_t for page fault handlers. As
> > part of that lustre driver is also getting changed to
> > return vm_fault_t type.
> >
> > Signed-off-by: Souptick Joarder <jrdr.linux at gmail.com>
> > ---
> > v2: updated the change log
> >
> > v3: updated the change log
> >
> >  drivers/staging/lustre/lustre/llite/llite_mmap.c | 35 ++++++++++++------------
> >  1 file changed, 18 insertions(+), 17 deletions(-)
> >
> > diff --git a/drivers/staging/lustre/lustre/llite/llite_mmap.c b/drivers/staging/lustre/lustre/llite/llite_mmap.c
> > index c0533bd..5b8fd10 100644
> > --- a/drivers/staging/lustre/lustre/llite/llite_mmap.c
> > +++ b/drivers/staging/lustre/lustre/llite/llite_mmap.c
> > @@ -231,7 +231,7 @@ static int ll_page_mkwrite0(struct vm_area_struct *vma, struct page *vmpage,
> >         return result;
> >  }
> >
> > -static inline int to_fault_error(int result)
> > +static inline vm_fault_t to_fault_error(int result)
> >  {
> >         switch (result) {
> >         case 0:
> > @@ -261,7 +261,7 @@ static inline int to_fault_error(int result)
> >   * \retval VM_FAULT_ERROR on general error
> >   * \retval NOPAGE_OOM not have memory for allocate new page
> >   */
> > -static int ll_fault0(struct vm_area_struct *vma, struct vm_fault *vmf)
> > +static vm_fault_t ll_fault0(struct vm_area_struct *vma, struct vm_fault *vmf)
> >  {
> >         struct lu_env      *env;
> >         struct cl_io        *io;
> > @@ -269,16 +269,16 @@ static int ll_fault0(struct vm_area_struct *vma, struct vm_fault *vmf)
> >         struct page          *vmpage;
> >         unsigned long       ra_flags;
> >         int                   result = 0;
> > -       int                   fault_ret = 0;
> > +       vm_fault_t              fault_ret = 0;
> >         u16 refcheck;
> >
> >         env = cl_env_get(&refcheck);
> >         if (IS_ERR(env))
> > -               return PTR_ERR(env);
> > +               return VM_FAULT_ERROR;
> >
> >         io = ll_fault_io_init(env, vma, vmf->pgoff, &ra_flags);
> >         if (IS_ERR(io)) {
> > -               result = to_fault_error(PTR_ERR(io));
> > +               fault_ret = to_fault_error(PTR_ERR(io));
> >                 goto out;
> >         }
> >
> > @@ -319,15 +319,15 @@ static int ll_fault0(struct vm_area_struct *vma, struct vm_fault *vmf)
> >         if (result != 0 && !(fault_ret & VM_FAULT_RETRY))
> >                 fault_ret |= to_fault_error(result);
> >
> > -       CDEBUG(D_MMAP, "%s fault %d/%d\n", current->comm, fault_ret, result);
> > +       CDEBUG(D_MMAP, "%s fault %x/%d\n", current->comm, fault_ret, result);
> >         return fault_ret;
> >  }
> >
> > -static int ll_fault(struct vm_fault *vmf)
> > +static vm_fault_t ll_fault(struct vm_fault *vmf)
> >  {
> >         int count = 0;
> >         bool printed = false;
> > -       int result;
> > +       vm_fault_t result;
> >         sigset_t set;
> >
> >         /* Only SIGKILL and SIGTERM are allowed for fault/nopage/mkwrite
> > @@ -364,18 +364,19 @@ static int ll_fault(struct vm_fault *vmf)
> >         return result;
> >  }
> >
> > -static int ll_page_mkwrite(struct vm_fault *vmf)
> > +static vm_fault_t ll_page_mkwrite(struct vm_fault *vmf)
> >  {
> >         struct vm_area_struct *vma = vmf->vma;
> >         int count = 0;
> >         bool printed = false;
> >         bool retry;
> > -       int result;
> > +       int err;
> > +       vm_fault_t ret;
> >
> >         file_update_time(vma->vm_file);
> >         do {
> >                 retry = false;
> > -               result = ll_page_mkwrite0(vma, vmf->page, &retry);
> > +               err = ll_page_mkwrite0(vma, vmf->page, &retry);
> >
> >                 if (!printed && ++count > 16) {
> >                         const struct dentry *de = vma->vm_file->f_path.dentry;
> > @@ -387,25 +388,25 @@ static int ll_page_mkwrite(struct vm_fault *vmf)
> >                 }
> >         } while (retry);
> >
> > -       switch (result) {
> > +       switch (err) {
> >         case 0:
> >                 LASSERT(PageLocked(vmf->page));
> > -               result = VM_FAULT_LOCKED;
> > +               ret = VM_FAULT_LOCKED;
> >                 break;
> >         case -ENODATA:
> >         case -EAGAIN:
> >         case -EFAULT:
> > -               result = VM_FAULT_NOPAGE;
> > +               ret = VM_FAULT_NOPAGE;
> >                 break;
> >         case -ENOMEM:
> > -               result = VM_FAULT_OOM;
> > +               ret = VM_FAULT_OOM;
> >                 break;
> >         default:
> > -               result = VM_FAULT_SIGBUS;
> > +               ret = VM_FAULT_SIGBUS;
> >                 break;
> >         }
> >
> > -       return result;
> > +       return ret;
> >  }
> >
> >  /**
> > --
> > 1.9.1
> >
> 
> If no further comment, we would like to get this patch in 4.18-rc-X.

Why?  Is it a regression fix?  That's all that is allowed after -rc1.

And have you tried applying it to Linus's current tree?  :)

thanks,

greg k-h

From gregkh at linuxfoundation.org  Mon Jun 11 21:07:14 2018
From: gregkh at linuxfoundation.org (Greg KH)
Date: Mon, 11 Jun 2018 23:07:14 +0200
Subject: [lustre-devel] [PATCH v3] staging: lustre: Change return type
 to vm_fault_t
In-Reply-To: <CAFqt6zZJcN4=FDQSqiSznoEogavV9GBpqxx-onWG5Q3wQEM9cQ@mail.gmail.com>
References: <20180521180937.GA19605@jordon-HP-15-Notebook-PC>
 <CAFqt6zamp=+adkyM+_4z405hN-9qOn1CsY_BgVtCrAfscBHhWg@mail.gmail.com>
 <20180611205058.GA30123@kroah.com>
 <CAFqt6zZJcN4=FDQSqiSznoEogavV9GBpqxx-onWG5Q3wQEM9cQ@mail.gmail.com>
Message-ID: <20180611210714.GB4023@kroah.com>

On Tue, Jun 12, 2018 at 02:30:27AM +0530, Souptick Joarder wrote:
> > >
> > > If no further comment, we would like to get this patch in 4.18-rc-X.
> >
> > Why?  Is it a regression fix?  That's all that is allowed after -rc1.
> 
> No, this is not regression fix. We need to get this into 4.18-rc-1.  But
> mostly it can't make into linus tree in rc-1 :)

Why does it _have_ to get into 4.18-rc1?  My tree is long-closed and
Linus already has all of my patches in his tree for the staging section
of the kernel.

> > And have you tried applying it to Linus's current tree?  :)
> 
> Last tested on 4.17-rc-6 and it worked fine. Let me verify in current tree.

Try it, you might be surprised :)

greg k-h

From jsimmons at infradead.org  Tue Jun 12 00:12:15 2018
From: jsimmons at infradead.org (James Simmons)
Date: Tue, 12 Jun 2018 01:12:15 +0100 (BST)
Subject: [lustre-devel] A new drivers/staging/lustre
In-Reply-To: <C50F2EA9-AD25-480A-8A05-2541206BA17F@intel.com>
References: <874lifnxbp.fsf@notabene.neil.brown.name>
 <alpine.LFD.2.21.1806070557300.23188@casper.infradead.org>
 <87po12ncjv.fsf@notabene.neil.brown.name>
 <A139C7E5-0406-4078-B579-8DCF5B85F10F@cray.com>
 <87h8memg4k.fsf@notabene.neil.brown.name>
 <C50F2EA9-AD25-480A-8A05-2541206BA17F@intel.com>
Message-ID: <alpine.LFD.2.21.1806081606001.13085@casper.infradead.org>


> > On Thu, Jun 07 2018, Doug Oucharek wrote:
> > 
> >> What is the focus of landings in this tree?  There are two things needing to be done for an upstream Lustre:
> >> 
> >> 
> >>  *   Get the source code to meet the Linux guidelines so it is acceptable to be in mainline.
> >>  *   Get the binary product to have all the features and bug fixes that are in the Intel community tree so end users are interested in using the upstream version (users are unlikely to use a version of Lustre which is not current).
> >> 
> >> For the now-deleted staging area, we were supposed to be focusing on the first item but were submitting patches for the second item (syncing with Intel tree).  In my opinion, this is the core reason for never being able to get out of staging and getting deleted.
> > 
> > My (undoubtedly biased) perspective on the history of lustre in staging
> > goes like this:
> > There are two things needed for some out-of-tree code to get into
> > mainline Linux:  the code needs to be integrated and the community needs
> > to be integrated (or a new sub-community needs to form).
> > In the case of lustre, the code was never really integrated because the
> > community never really tried to integrate.
> 
> One of the issues here was that the group (not Intel) that submitted the
> Lustre code to the staging tree promptly abandoned it for a couple of
> years after they submitted it upstream, after promising the community
> that they were in it for the long run.  That put the upstream integration
> behind the eight-ball from the start.

The leason from this is that the upstream efforts can not be placed on an
single organization's efforts. It has to be a group effort. Before Neil
showed up I was the only person working on the upstream efforts for some
time. I came under pressure to abandon this effort but since I do this
on the weekends in my free time it doesn't impact my normal job duties.
So I managed to get away with this work. It does help that actually
useful by products from this effort shows up in the general lustre
community which in turn benefits my employeer. I bet for EMC the burden
was to much and since no one supported them they abandon this work.
The main reason people abandon things in life is feeling unsupported.
Even if all you do is act as a cheerleader it can have a big impact
on the people doing the work. At one time I felt the same way with the 
upstream client which I shared with Peter Jones. Peter pointed out that he 
knows sites that do use the upstream client. Since then my focus has been 
aimed at supporting those users. With my focus is on the users so if 
companies join or don't join in is not important to me.

> > Yes, the current code needs to be improved, bugs need to be fixed, and
> > features need to be added.  The order in which these is done is not the
> > most important things - if it were, Greg would have never accepted any
> > new features.  However he *did* accept them, but tried to remind the
> > lustre developers that there was other work to do.
> > 
> > Working together in one (single) community requires give-and-take.
> > Greg's behaviour as just described seems to be evidence of
> > give-and-take.  I think he kicked lustre out of staging because he
> > concluded that he was never going to get the matching give-and-take in
> > return.

Based on the emails sent I see no one understood the work I was doing for 
the last 2 years. I guess that is my fault for not having general 
discussion emails on the efforts. Mind you at first I tried to send 
non patch emails to the staging mailing list but they always were 
filtered out so I gave up. Also no one every talked to me about what the
cleanups should be. So I have always worked on what I know is a problem.
Mainly things reported by users and also things reported back to Oleg
from the VFS developers. That is how the roadmap came into being that
I discussed at developer's day for LUG.

First of all the upstream client is NOT at a 2.4 version. Its close to 
a 2.9 version. In fact its newer then the default lustre client Cray ships
with (2.7). Second I haven't pushed features for over a year. The first
year I did spend syncing the tree up to 2.8+ version which got us to a
point where people were willing to try it. 

After that it became this constant smash the bugs for the last year. The 
port to sysfs and the 64 bit time work was incorrect and lead to all kinds 
of regressions. Also the xattr which lustre heavly depends on was broken. 
Nearly all xattr bugs are fixed. I nearly have resolved most of the 64
bit time and sysfs bugs. Still haven't push them yet but that is next. 
So my focus is reduce all the regression to a manageable level. I really
didn't want to see a lustre client out of staging that was a complete 
piece of 'bleep'.

The client feature wise is nearly 2.9 but many bug fixes from 2.10 and
above have been merged to the upstream code. The gap is much smaller than
you realize. So the upstream client is strange hybrid. 

> > So to answer your opening question, my focus for this tree is to train
> > any lustre developers who wish to engage about how to be part of the
> > Linux community.  As I've already said - I will accept features but I
> > prefer cleanups first.  I don't want to try to explain further than that
> > because it will be too hypothetical and unhelpful.  We - the Linux
> > community - don't work in hypotheticals.  We work with concrete objects
> > like patches.  So send me a patch and I will tell you what I think of
> > that specific patch.  It is up to you to generalise what I say to other
> > patches.  It might also be up to you to argue your case and tell me why
> > I'm wrong.  I'll be patient (because good upstream maintainers are) but
> > patience doesn't last forever (for Greg, it lasted about 5 years - I
> > hope mine won't be tried to that extent).
> 
> Like I said in my other email, I think having another fork of the Lustre
> tree, especially one that is starting from two-year-old code is likely to
> fail, because there will be twice as much effort spent to maintain the two
> trees.  I'd rather see the cleanups and features go hand-in-hand into the
> same tree.  I'd be thrilled to have more reviews done on the features before
> they are landed, but we can't just stop all feature development for a year
> or two (or five) while the code is merged into the upstream kernel.

I don't think having another tree is such a big issue. As I pointed out
earlier Cray carries their own lustre client. Also Livermore Labs 
has their own special Choas lustre tree. Lastly we do backports to LTS
version of lustre all the time.

I'm really against using the out of tree for this kernel cleanup for a
few reasons. The gap between upstream and master is less than you think.
Outside the features landed the majority of the patches landed to the 
OpenSFS/Intel branch is my work to sync both the upstream client and the 
Intel community branch. Next from personal experinece it takes a very very
long time to land cleanup patches for the Intel branch. It normally takes
3 release cycles to complete any work which means we are looking at least
1.5 years to finish meeting upstream requirements. Now going in the other
direction in a few months we could reach parity with the out of tree
source. Especially since the gap is much smaller now. I ready to close
the gap.


From jsimmons at infradead.org  Tue Jun 12 00:13:41 2018
From: jsimmons at infradead.org (James Simmons)
Date: Tue, 12 Jun 2018 01:13:41 +0100 (BST)
Subject: [lustre-devel] A new drivers/staging/lustre
In-Reply-To: <874lifnxbp.fsf@notabene.neil.brown.name>
References: <874lifnxbp.fsf@notabene.neil.brown.name>
Message-ID: <alpine.LFD.2.21.1806120112530.15735@casper.infradead.org>


> Greg's patch to remove lustre has now landed in this staging-next tree,
> so I suspect it will get to Linus before too long.  So I have to find a
> new place to work on lustre.
> 
> I've added 2 branches to
>    git://git.neil.brown.name/linux

Its seems to be down. Do you plan to move the tree somewhere else. Perhaps
your github page?

From jsimmons at infradead.org  Tue Jun 12 00:21:53 2018
From: jsimmons at infradead.org (James Simmons)
Date: Tue, 12 Jun 2018 01:21:53 +0100 (BST)
Subject: [lustre-devel] A new drivers/staging/lustre
In-Reply-To: <87po12ncjv.fsf@notabene.neil.brown.name>
References: <874lifnxbp.fsf@notabene.neil.brown.name>
 <alpine.LFD.2.21.1806070557300.23188@casper.infradead.org>
 <87po12ncjv.fsf@notabene.neil.brown.name>
Message-ID: <alpine.LFD.2.21.1806120114020.15735@casper.infradead.org>


> >> lustre-testing:
> >>    is based on 'lustre' and has most of my current lustre-related work.
> >>    It includes assorted patches that are not specifically for lustre
> >>    (rhashtables mostly at the moment).  Patches will move from here
> >>    to 'lustre' or to mainline when they are ready.
> >>    I plan to update this branch on most days that I work on Lustre,
> >>    and expect it to rebase frequently.
> >
> > I had question about that. Some things in Lustre could in theory be merged
> > into the linux kernel proper. Can that be done still?
> 
> What things?
> 
> If it measurably benefits the kernel proper, then I suspect it might be
> worth submitting.  Things can go direct without going though staging -
> they just have to be of good quality with good justification (and
> sometimes lots of patience).

One piece of work done for Lustre was the ability to submit "units" like
K, M, G for sizes to sysfs files. This was developed before string helpers
was created and the code for Lustre is well very ugly. So I rewrote it
in the style of string helpers and it can be handly for other things as 
well such as module parameters setting. Lustre does have certain features
in it debugging system that can applied to the linux kernel debugging
system. That is more down the road. Their are few others like the crypto
wrapper for alder. I don't see why that can't be mainstream.
 
> >  
> >> I'm happy to review and, if acceptable, apply patches from other
> >> developers.  I have fairly high standards, but if I don't accept your
> >> patch I'll explain why and possible help fix it.
> >
> > Also long as you talk to me :-) I'm an easy person to work with. If I 
> > refuse a patch with do the same. I might sometimes seem irrational 
> > but I have valid reasons. Well at least in my head.
> >
> > We need to really layout the roadmap.
> 
> I have very little faith in road maps - I prefer to make steps.  Once we
> have made all the steps, we can look back and see what the map looked
> like in retrospect.

I was looking to see what you plan to work on. I wanted to avoid 
duplicating work. Some things I know of done already are listed under:

https://jira.hpdd.intel.com/browse/LU-9855
https://jira.hpdd.intel.com/browse/LU-10257
https://jira.hpdd.intel.com/browse/LU-10994

Their are a few others I have to hunt down but these are the major ones.

From neilb at suse.com  Tue Jun 12 00:55:13 2018
From: neilb at suse.com (NeilBrown)
Date: Tue, 12 Jun 2018 10:55:13 +1000
Subject: [lustre-devel] A new drivers/staging/lustre
In-Reply-To: <alpine.LFD.2.21.1806120112530.15735@casper.infradead.org>
References: <874lifnxbp.fsf@notabene.neil.brown.name>
 <alpine.LFD.2.21.1806120112530.15735@casper.infradead.org>
Message-ID: <87lgbkkemm.fsf@notabene.neil.brown.name>

On Tue, Jun 12 2018, James Simmons wrote:

>> Greg's patch to remove lustre has now landed in this staging-next tree,
>> so I suspect it will get to Linus before too long.  So I have to find a
>> new place to work on lustre.
>> 
>> I've added 2 branches to
>>    git://git.neil.brown.name/linux
>
> Its seems to be down. Do you plan to move the tree somewhere else. Perhaps
> your github page?

Bother - that's what you get for using Linux to host things (my VM
doesn't have heaps of memory and git likes to use a lot, and doesn't
always clean up well after itself).

git.neil.brown.name is back, and github.com/neilbrown/linux now
has two new branches: lustre/lutre and lustre/lustre-testing

Thanks,
NeilBrown
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180612/2c8ebe9b/attachment.sig>

From neilb at suse.com  Tue Jun 12 01:16:26 2018
From: neilb at suse.com (NeilBrown)
Date: Tue, 12 Jun 2018 11:16:26 +1000
Subject: [lustre-devel] A new drivers/staging/lustre
In-Reply-To: <alpine.LFD.2.21.1806120114020.15735@casper.infradead.org>
References: <874lifnxbp.fsf@notabene.neil.brown.name>
 <alpine.LFD.2.21.1806070557300.23188@casper.infradead.org>
 <87po12ncjv.fsf@notabene.neil.brown.name>
 <alpine.LFD.2.21.1806120114020.15735@casper.infradead.org>
Message-ID: <87in6okdn9.fsf@notabene.neil.brown.name>

On Tue, Jun 12 2018, James Simmons wrote:

>> >> lustre-testing:
>> >>    is based on 'lustre' and has most of my current lustre-related work.
>> >>    It includes assorted patches that are not specifically for lustre
>> >>    (rhashtables mostly at the moment).  Patches will move from here
>> >>    to 'lustre' or to mainline when they are ready.
>> >>    I plan to update this branch on most days that I work on Lustre,
>> >>    and expect it to rebase frequently.
>> >
>> > I had question about that. Some things in Lustre could in theory be merged
>> > into the linux kernel proper. Can that be done still?
>> 
>> What things?
>> 
>> If it measurably benefits the kernel proper, then I suspect it might be
>> worth submitting.  Things can go direct without going though staging -
>> they just have to be of good quality with good justification (and
>> sometimes lots of patience).
>
> One piece of work done for Lustre was the ability to submit "units" like
> K, M, G for sizes to sysfs files. This was developed before string helpers
> was created and the code for Lustre is well very ugly. So I rewrote it
> in the style of string helpers and it can be handly for other things as 
> well such as module parameters setting.

I cannot work out what we might need beyond string_helpers, and I cannot
find any code in lustre that might be what you are referring to.  But I
agree that making it easy to use unit in sysfs is probably a generally
good idea.

>                                         Lustre does have certain features
> in it debugging system that can applied to the linux kernel debugging
> system. That is more down the road. Their are few others like the crypto
> wrapper for alder. I don't see why that can't be mainstream.

I have wondered why crypto-adler isn't in the main crypto library.  I
understand that it is weak and deprecated, but I doubt that is a reason
to keep the code out.  Maybe we should just try to submit it.

>  
>> >  
>> >> I'm happy to review and, if acceptable, apply patches from other
>> >> developers.  I have fairly high standards, but if I don't accept your
>> >> patch I'll explain why and possible help fix it.
>> >
>> > Also long as you talk to me :-) I'm an easy person to work with. If I 
>> > refuse a patch with do the same. I might sometimes seem irrational 
>> > but I have valid reasons. Well at least in my head.
>> >
>> > We need to really layout the roadmap.
>> 
>> I have very little faith in road maps - I prefer to make steps.  Once we
>> have made all the steps, we can look back and see what the map looked
>> like in retrospect.
>
> I was looking to see what you plan to work on. I wanted to avoid 
> duplicating work. Some things I know of done already are listed under:

Providing that we regularly talk about what we are doing I think the
chance of duplication is fairly low, and I don't think the cost of
duplication is very high.  If we've both worked of doing X, then we
should each be able to review the work of the other quite quickly.

My current focus is cleaning up the build (reducing modules and
exports), renaming files (removing 'linux' from various names),
and removing unused cruft from include files (files named *compat.h are
next).
I then want to gain a more complete understanding of LNet.

>
> https://jira.hpdd.intel.com/browse/LU-9855

I had thought about attacking this, but it isn't on my current
shortlist.  I'll let you know if it gets near the top.

> https://jira.hpdd.intel.com/browse/LU-10257

I don't really know what this is about.  I think it is a much deeper
clean-up than the superficial stuff I've been focusing on.
If I read correctly, this has already been addressed in legacy-lustre.
In that case I probably would only get to it when I start going through
the git log locking for things to migrate to linux-lustre.

> https://jira.hpdd.intel.com/browse/LU-10994

This is even less on my radar than the above.

I suggest you feel free to do whatever seems suitable without worrying
if I'm already working there - but do report results and request review
often, even if you aren't completely happy yet.
If I ever find you working on something that I've started, I'll quietly
hide my work and just use the knowledge gained to provide insightful
review of you work.

Thanks,
NeilBrown
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180612/25c68dd0/attachment.sig>

From neilb at suse.com  Tue Jun 12 01:24:22 2018
From: neilb at suse.com (NeilBrown)
Date: Tue, 12 Jun 2018 11:24:22 +1000
Subject: [lustre-devel] A new drivers/staging/lustre
In-Reply-To: <alpine.LFD.2.21.1806081606001.13085@casper.infradead.org>
References: <874lifnxbp.fsf@notabene.neil.brown.name>
 <alpine.LFD.2.21.1806070557300.23188@casper.infradead.org>
 <87po12ncjv.fsf@notabene.neil.brown.name>
 <A139C7E5-0406-4078-B579-8DCF5B85F10F@cray.com>
 <87h8memg4k.fsf@notabene.neil.brown.name>
 <C50F2EA9-AD25-480A-8A05-2541206BA17F@intel.com>
 <alpine.LFD.2.21.1806081606001.13085@casper.infradead.org>
Message-ID: <87fu1skda1.fsf@notabene.neil.brown.name>

On Tue, Jun 12 2018, James Simmons wrote:

> The main reason people abandon things in life is feeling unsupported.
> Even if all you do is act as a cheerleader it can have a big impact
> on the people doing the work.

So much "yes".
Receiving feed-back, encouragement, affirmation that the work is valued,
is a REALLY big thing.  I do this because I'm paid, but also because I
enjoy the challenge and I value the community involvement.
A colleague in SUSE recently forwarded a line from an IRC conversation
that he was part of where someone commended me on my work on Lustre.  It
was really nice to receive that (though maybe a shame that it didn't
come more directly).
Who-ever said that - thanks!  You don't have to be writing patches to be
supporting this work.

Thanks,
NeilBrown
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180612/2d5cd6e0/attachment.sig>

From neilb at suse.com  Tue Jun 12 05:26:38 2018
From: neilb at suse.com (NeilBrown)
Date: Tue, 12 Jun 2018 15:26:38 +1000
Subject: [lustre-devel] [PATCH] staging: lustre: add error handling for
 try_module_get
In-Reply-To: <1528778968-42225-1-git-send-email-jiazhouyang09@gmail.com>
References: <1528778968-42225-1-git-send-email-jiazhouyang09@gmail.com>
Message-ID: <87y3fkinht.fsf@notabene.neil.brown.name>

On Tue, Jun 12 2018, Zhouyang Jia wrote:

> When try_module_get fails, the lack of error-handling code may
> cause unexpected results.
>
> This patch adds error-handling code after calling try_module_get.
>
> Signed-off-by: Zhouyang Jia <jiazhouyang09 at gmail.com>
> ---
>  drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c | 5 ++++-
>  1 file changed, 4 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c
> index 7086678..72a42bd 100644
> --- a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c
> +++ b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c
> @@ -2422,7 +2422,10 @@ ksocknal_base_startup(void)
>  
>  	/* flag lists/ptrs/locks initialised */
>  	ksocknal_data.ksnd_init = SOCKNAL_INIT_DATA;
> -	try_module_get(THIS_MODULE);
> +	if (!try_module_get(THIS_MODULE)) {
> +		CERROR("%s: cannot get module\n", __func__);
> +		goto failed;
> +	}
>  
>  	ksocknal_data.ksnd_sched_info = cfs_percpt_alloc(lnet_cpt_table(),
>  							 sizeof(*info));

Thanks for the patch....
I agree that this is probably a bug, but the code is still buggy after
you patch, just in a different way.
Try following through the code and see what happens when you 'goto
failed'.

NeilBrown


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180612/9407950d/attachment.sig>

From gregkh at linuxfoundation.org  Tue Jun 12 06:31:11 2018
From: gregkh at linuxfoundation.org (Greg Kroah-Hartman)
Date: Tue, 12 Jun 2018 08:31:11 +0200
Subject: [lustre-devel] [PATCH] staging: lustre: add error handling for
 try_module_get
In-Reply-To: <1528778968-42225-1-git-send-email-jiazhouyang09@gmail.com>
References: <1528778968-42225-1-git-send-email-jiazhouyang09@gmail.com>
Message-ID: <20180612063111.GC23073@kroah.com>

On Tue, Jun 12, 2018 at 12:49:26PM +0800, Zhouyang Jia wrote:
> When try_module_get fails, the lack of error-handling code may
> cause unexpected results.
> 
> This patch adds error-handling code after calling try_module_get.
> 
> Signed-off-by: Zhouyang Jia <jiazhouyang09 at gmail.com>
> ---
>  drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c | 5 ++++-

This patch does not apply to Linus's tree.  Always be sure to work
against linux-next to catch things like this.

thanks,

greg k-h

From jrdr.linux at gmail.com  Mon Jun 11 20:30:47 2018
From: jrdr.linux at gmail.com (Souptick Joarder)
Date: Tue, 12 Jun 2018 02:00:47 +0530
Subject: [lustre-devel] [PATCH v3] staging: lustre: Change return type
 to vm_fault_t
In-Reply-To: <20180521180937.GA19605@jordon-HP-15-Notebook-PC>
References: <20180521180937.GA19605@jordon-HP-15-Notebook-PC>
Message-ID: <CAFqt6zamp=+adkyM+_4z405hN-9qOn1CsY_BgVtCrAfscBHhWg@mail.gmail.com>

On Mon, May 21, 2018 at 11:39 PM, Souptick Joarder <jrdr.linux at gmail.com> wrote:
> Use new return type vm_fault_t for fault handler. For
> now, this is just documenting that the function returns
> a VM_FAULT value rather than an errno. Once all instances
> are converted, vm_fault_t will become a distinct type.
>
> Ref-> commit 1c8f422059ae ("mm: change return type to
> vm_fault_t") was added in 4.17-rc1 to introduce the new
> typedef vm_fault_t. Currently we are making change to all
> drivers to return vm_fault_t for page fault handlers. As
> part of that lustre driver is also getting changed to
> return vm_fault_t type.
>
> Signed-off-by: Souptick Joarder <jrdr.linux at gmail.com>
> ---
> v2: updated the change log
>
> v3: updated the change log
>
>  drivers/staging/lustre/lustre/llite/llite_mmap.c | 35 ++++++++++++------------
>  1 file changed, 18 insertions(+), 17 deletions(-)
>
> diff --git a/drivers/staging/lustre/lustre/llite/llite_mmap.c b/drivers/staging/lustre/lustre/llite/llite_mmap.c
> index c0533bd..5b8fd10 100644
> --- a/drivers/staging/lustre/lustre/llite/llite_mmap.c
> +++ b/drivers/staging/lustre/lustre/llite/llite_mmap.c
> @@ -231,7 +231,7 @@ static int ll_page_mkwrite0(struct vm_area_struct *vma, struct page *vmpage,
>         return result;
>  }
>
> -static inline int to_fault_error(int result)
> +static inline vm_fault_t to_fault_error(int result)
>  {
>         switch (result) {
>         case 0:
> @@ -261,7 +261,7 @@ static inline int to_fault_error(int result)
>   * \retval VM_FAULT_ERROR on general error
>   * \retval NOPAGE_OOM not have memory for allocate new page
>   */
> -static int ll_fault0(struct vm_area_struct *vma, struct vm_fault *vmf)
> +static vm_fault_t ll_fault0(struct vm_area_struct *vma, struct vm_fault *vmf)
>  {
>         struct lu_env      *env;
>         struct cl_io        *io;
> @@ -269,16 +269,16 @@ static int ll_fault0(struct vm_area_struct *vma, struct vm_fault *vmf)
>         struct page          *vmpage;
>         unsigned long       ra_flags;
>         int                   result = 0;
> -       int                   fault_ret = 0;
> +       vm_fault_t              fault_ret = 0;
>         u16 refcheck;
>
>         env = cl_env_get(&refcheck);
>         if (IS_ERR(env))
> -               return PTR_ERR(env);
> +               return VM_FAULT_ERROR;
>
>         io = ll_fault_io_init(env, vma, vmf->pgoff, &ra_flags);
>         if (IS_ERR(io)) {
> -               result = to_fault_error(PTR_ERR(io));
> +               fault_ret = to_fault_error(PTR_ERR(io));
>                 goto out;
>         }
>
> @@ -319,15 +319,15 @@ static int ll_fault0(struct vm_area_struct *vma, struct vm_fault *vmf)
>         if (result != 0 && !(fault_ret & VM_FAULT_RETRY))
>                 fault_ret |= to_fault_error(result);
>
> -       CDEBUG(D_MMAP, "%s fault %d/%d\n", current->comm, fault_ret, result);
> +       CDEBUG(D_MMAP, "%s fault %x/%d\n", current->comm, fault_ret, result);
>         return fault_ret;
>  }
>
> -static int ll_fault(struct vm_fault *vmf)
> +static vm_fault_t ll_fault(struct vm_fault *vmf)
>  {
>         int count = 0;
>         bool printed = false;
> -       int result;
> +       vm_fault_t result;
>         sigset_t set;
>
>         /* Only SIGKILL and SIGTERM are allowed for fault/nopage/mkwrite
> @@ -364,18 +364,19 @@ static int ll_fault(struct vm_fault *vmf)
>         return result;
>  }
>
> -static int ll_page_mkwrite(struct vm_fault *vmf)
> +static vm_fault_t ll_page_mkwrite(struct vm_fault *vmf)
>  {
>         struct vm_area_struct *vma = vmf->vma;
>         int count = 0;
>         bool printed = false;
>         bool retry;
> -       int result;
> +       int err;
> +       vm_fault_t ret;
>
>         file_update_time(vma->vm_file);
>         do {
>                 retry = false;
> -               result = ll_page_mkwrite0(vma, vmf->page, &retry);
> +               err = ll_page_mkwrite0(vma, vmf->page, &retry);
>
>                 if (!printed && ++count > 16) {
>                         const struct dentry *de = vma->vm_file->f_path.dentry;
> @@ -387,25 +388,25 @@ static int ll_page_mkwrite(struct vm_fault *vmf)
>                 }
>         } while (retry);
>
> -       switch (result) {
> +       switch (err) {
>         case 0:
>                 LASSERT(PageLocked(vmf->page));
> -               result = VM_FAULT_LOCKED;
> +               ret = VM_FAULT_LOCKED;
>                 break;
>         case -ENODATA:
>         case -EAGAIN:
>         case -EFAULT:
> -               result = VM_FAULT_NOPAGE;
> +               ret = VM_FAULT_NOPAGE;
>                 break;
>         case -ENOMEM:
> -               result = VM_FAULT_OOM;
> +               ret = VM_FAULT_OOM;
>                 break;
>         default:
> -               result = VM_FAULT_SIGBUS;
> +               ret = VM_FAULT_SIGBUS;
>                 break;
>         }
>
> -       return result;
> +       return ret;
>  }
>
>  /**
> --
> 1.9.1
>

If no further comment, we would like to get this patch in 4.18-rc-X.

From jrdr.linux at gmail.com  Mon Jun 11 21:00:27 2018
From: jrdr.linux at gmail.com (Souptick Joarder)
Date: Tue, 12 Jun 2018 02:30:27 +0530
Subject: [lustre-devel] [PATCH v3] staging: lustre: Change return type
 to vm_fault_t
In-Reply-To: <20180611205058.GA30123@kroah.com>
References: <20180521180937.GA19605@jordon-HP-15-Notebook-PC>
 <CAFqt6zamp=+adkyM+_4z405hN-9qOn1CsY_BgVtCrAfscBHhWg@mail.gmail.com>
 <20180611205058.GA30123@kroah.com>
Message-ID: <CAFqt6zZJcN4=FDQSqiSznoEogavV9GBpqxx-onWG5Q3wQEM9cQ@mail.gmail.com>

On 12-Jun-2018 2:21 AM, "Greg KH" <gregkh at linuxfoundation.org> wrote:
>
> On Tue, Jun 12, 2018 at 02:00:47AM +0530, Souptick Joarder wrote:
> > On Mon, May 21, 2018 at 11:39 PM, Souptick Joarder <jrdr.linux at gmail.com>
wrote:
> > > Use new return type vm_fault_t for fault handler. For
> > > now, this is just documenting that the function returns
> > > a VM_FAULT value rather than an errno. Once all instances
> > > are converted, vm_fault_t will become a distinct type.
> > >
> > > Ref-> commit 1c8f422059ae ("mm: change return type to
> > > vm_fault_t") was added in 4.17-rc1 to introduce the new
> > > typedef vm_fault_t. Currently we are making change to all
> > > drivers to return vm_fault_t for page fault handlers. As
> > > part of that lustre driver is also getting changed to
> > > return vm_fault_t type.
> > >
> > > Signed-off-by: Souptick Joarder <jrdr.linux at gmail.com>
> > > ---
> > > v2: updated the change log
> > >
> > > v3: updated the change log
> > >
> > >  drivers/staging/lustre/lustre/llite/llite_mmap.c | 35
++++++++++++------------
> > >  1 file changed, 18 insertions(+), 17 deletions(-)
> > >
> > > diff --git a/drivers/staging/lustre/lustre/llite/llite_mmap.c
b/drivers/staging/lustre/lustre/llite/llite_mmap.c
> > > index c0533bd..5b8fd10 100644
> > > --- a/drivers/staging/lustre/lustre/llite/llite_mmap.c
> > > +++ b/drivers/staging/lustre/lustre/llite/llite_mmap.c
> > > @@ -231,7 +231,7 @@ static int ll_page_mkwrite0(struct vm_area_struct
*vma, struct page *vmpage,
> > >         return result;
> > >  }
> > >
> > > -static inline int to_fault_error(int result)
> > > +static inline vm_fault_t to_fault_error(int result)
> > >  {
> > >         switch (result) {
> > >         case 0:
> > > @@ -261,7 +261,7 @@ static inline int to_fault_error(int result)
> > >   * \retval VM_FAULT_ERROR on general error
> > >   * \retval NOPAGE_OOM not have memory for allocate new page
> > >   */
> > > -static int ll_fault0(struct vm_area_struct *vma, struct vm_fault
*vmf)
> > > +static vm_fault_t ll_fault0(struct vm_area_struct *vma, struct
vm_fault *vmf)
> > >  {
> > >         struct lu_env      *env;
> > >         struct cl_io        *io;
> > > @@ -269,16 +269,16 @@ static int ll_fault0(struct vm_area_struct
*vma, struct vm_fault *vmf)
> > >         struct page          *vmpage;
> > >         unsigned long       ra_flags;
> > >         int                   result = 0;
> > > -       int                   fault_ret = 0;
> > > +       vm_fault_t              fault_ret = 0;
> > >         u16 refcheck;
> > >
> > >         env = cl_env_get(&refcheck);
> > >         if (IS_ERR(env))
> > > -               return PTR_ERR(env);
> > > +               return VM_FAULT_ERROR;
> > >
> > >         io = ll_fault_io_init(env, vma, vmf->pgoff, &ra_flags);
> > >         if (IS_ERR(io)) {
> > > -               result = to_fault_error(PTR_ERR(io));
> > > +               fault_ret = to_fault_error(PTR_ERR(io));
> > >                 goto out;
> > >         }
> > >
> > > @@ -319,15 +319,15 @@ static int ll_fault0(struct vm_area_struct
*vma, struct vm_fault *vmf)
> > >         if (result != 0 && !(fault_ret & VM_FAULT_RETRY))
> > >                 fault_ret |= to_fault_error(result);
> > >
> > > -       CDEBUG(D_MMAP, "%s fault %d/%d\n", current->comm, fault_ret,
result);
> > > +       CDEBUG(D_MMAP, "%s fault %x/%d\n", current->comm, fault_ret,
result);
> > >         return fault_ret;
> > >  }
> > >
> > > -static int ll_fault(struct vm_fault *vmf)
> > > +static vm_fault_t ll_fault(struct vm_fault *vmf)
> > >  {
> > >         int count = 0;
> > >         bool printed = false;
> > > -       int result;
> > > +       vm_fault_t result;
> > >         sigset_t set;
> > >
> > >         /* Only SIGKILL and SIGTERM are allowed for
fault/nopage/mkwrite
> > > @@ -364,18 +364,19 @@ static int ll_fault(struct vm_fault *vmf)
> > >         return result;
> > >  }
> > >
> > > -static int ll_page_mkwrite(struct vm_fault *vmf)
> > > +static vm_fault_t ll_page_mkwrite(struct vm_fault *vmf)
> > >  {
> > >         struct vm_area_struct *vma = vmf->vma;
> > >         int count = 0;
> > >         bool printed = false;
> > >         bool retry;
> > > -       int result;
> > > +       int err;
> > > +       vm_fault_t ret;
> > >
> > >         file_update_time(vma->vm_file);
> > >         do {
> > >                 retry = false;
> > > -               result = ll_page_mkwrite0(vma, vmf->page, &retry);
> > > +               err = ll_page_mkwrite0(vma, vmf->page, &retry);
> > >
> > >                 if (!printed && ++count > 16) {
> > >                         const struct dentry *de =
vma->vm_file->f_path.dentry;
> > > @@ -387,25 +388,25 @@ static int ll_page_mkwrite(struct vm_fault *vmf)
> > >                 }
> > >         } while (retry);
> > >
> > > -       switch (result) {
> > > +       switch (err) {
> > >         case 0:
> > >                 LASSERT(PageLocked(vmf->page));
> > > -               result = VM_FAULT_LOCKED;
> > > +               ret = VM_FAULT_LOCKED;
> > >                 break;
> > >         case -ENODATA:
> > >         case -EAGAIN:
> > >         case -EFAULT:
> > > -               result = VM_FAULT_NOPAGE;
> > > +               ret = VM_FAULT_NOPAGE;
> > >                 break;
> > >         case -ENOMEM:
> > > -               result = VM_FAULT_OOM;
> > > +               ret = VM_FAULT_OOM;
> > >                 break;
> > >         default:
> > > -               result = VM_FAULT_SIGBUS;
> > > +               ret = VM_FAULT_SIGBUS;
> > >                 break;
> > >         }
> > >
> > > -       return result;
> > > +       return ret;
> > >  }
> > >
> > >  /**
> > > --
> > > 1.9.1
> > >
> >
> > If no further comment, we would like to get this patch in 4.18-rc-X.
>
> Why?  Is it a regression fix?  That's all that is allowed after -rc1.

No, this is not regression fix. We need to get this into 4.18-rc-1.  But
mostly it can't make into linus tree in rc-1 :)
>
> And have you tried applying it to Linus's current tree?  :)

Last tested on 4.17-rc-6 and it worked fine. Let me verify in current tree.

>
> thanks,
>
> greg k-h
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180612/65d779be/attachment.html>

From jiazhouyang09 at gmail.com  Tue Jun 12 04:49:26 2018
From: jiazhouyang09 at gmail.com (Zhouyang Jia)
Date: Tue, 12 Jun 2018 12:49:26 +0800
Subject: [lustre-devel] [PATCH] staging: lustre: add error handling for
 try_module_get
Message-ID: <1528778968-42225-1-git-send-email-jiazhouyang09@gmail.com>

When try_module_get fails, the lack of error-handling code may
cause unexpected results.

This patch adds error-handling code after calling try_module_get.

Signed-off-by: Zhouyang Jia <jiazhouyang09 at gmail.com>
---
 drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c
index 7086678..72a42bd 100644
--- a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c
+++ b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c
@@ -2422,7 +2422,10 @@ ksocknal_base_startup(void)
 
 	/* flag lists/ptrs/locks initialised */
 	ksocknal_data.ksnd_init = SOCKNAL_INIT_DATA;
-	try_module_get(THIS_MODULE);
+	if (!try_module_get(THIS_MODULE)) {
+		CERROR("%s: cannot get module\n", __func__);
+		goto failed;
+	}
 
 	ksocknal_data.ksnd_sched_info = cfs_percpt_alloc(lnet_cpt_table(),
 							 sizeof(*info));
-- 
2.7.4


From neilb at suse.com  Wed Jun 13 12:02:55 2018
From: neilb at suse.com (NeilBrown)
Date: Wed, 13 Jun 2018 22:02:55 +1000
Subject: [lustre-devel] [PATCH] staging: lustre: add error handling for
 try_module_get
In-Reply-To: <ed5d91f54b5a4776a66df87b645b0ee9@AcuMS.aculab.com>
References: <1528778968-42225-1-git-send-email-jiazhouyang09@gmail.com>
 <ed5d91f54b5a4776a66df87b645b0ee9@AcuMS.aculab.com>
Message-ID: <87k1r2j3m8.fsf@notabene.neil.brown.name>

On Wed, Jun 13 2018, David Laight wrote:

> From: Zhouyang Jia
>> Sent: 12 June 2018 05:49
>> 
>> When try_module_get fails, the lack of error-handling code may
>> cause unexpected results.
>> 
>> This patch adds error-handling code after calling try_module_get.
> ...
>> +++ b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c
>> @@ -2422,7 +2422,10 @@ ksocknal_base_startup(void)
>> 
>>  	/* flag lists/ptrs/locks initialised */
>>  	ksocknal_data.ksnd_init = SOCKNAL_INIT_DATA;
>> -	try_module_get(THIS_MODULE);
>> +	if (!try_module_get(THIS_MODULE)) {
>> +		CERROR("%s: cannot get module\n", __func__);
>> +		goto failed;
>> +	}
>
>
> Can try_module_get(THIS_MODULE) ever fail?

Yes.

> Since you are running code in 'THIS_MODULE' the caller must have a
> reference that can't go away.

Not necessarily, though it does usually work that way.

try_module_get() can fail while the exit function is running, but it is
safe to run code in the module until the exit function completes.
So if the exit function takes a lock, then other code can safely run
code in the module while holding the lock, but not holding a reference
to the module.  If this code calls try_module_get(), it could fail.

That is exactly what is happening here.
ksoclnd_exit() calls lnet_unregister_lnd() which takes
the_lnet.ln_lnd_mutex.

ksocknal_base_startup() is called from ksocknal_startup()
which is the_ksocklnd.lnd_startup and is called, from
lnet_startup_lndni(), with that lock held.

> So try_module_get() just increments the count that is already greater
> than zero.
>
> Similarly module_put(THIS_MODULE) must never be able to release the
> last reference.

It can if a suitable lock is held.

> Any such calls that aren't in error paths after try_module_get() are
> probably buggy.
Being in an error path doesn't make it safe.
module_put(THIS_MODULE) can only be safe if a lock is held which
prevents the exit function from completing.  Some code outside the
module must release the lock.

Having said that, I don't really like this approach.  I much prefer for
the module reference to be taken and put outside of the module - it
seems less error-prone.

NeilBrown
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180613/d4ecae9b/attachment.sig>

From jsimmons at infradead.org  Wed Jun 13 21:38:16 2018
From: jsimmons at infradead.org (James Simmons)
Date: Wed, 13 Jun 2018 22:38:16 +0100 (BST)
Subject: [lustre-devel] [PATCH 11/11] staging: lustre: centralize
 setting of subdir-ccflags-y
In-Reply-To: <152826511930.16761.12770722873423628004.stgit@noble>
References: <152826510267.16761.14361003167157833896.stgit@noble>
 <152826511930.16761.12770722873423628004.stgit@noble>
Message-ID: <alpine.LFD.2.21.1806132237140.12664@casper.infradead.org>


> We don't need to set subdir-ccflags-y in every Makefile.
> The whole point of the "subdir-" bit is that the setting
> can go once in the top-level Makefile.
> 

Nak: When attempting to build individual components I get:

~/lustre-upstream$ make SUBDIRS=drivers/staging/lustre/lustre/lmv modules 
-j 16

  WARNING: Symbol version dump ./Module.symvers
           is missing; modules will have no dependencies and modversions.

  CC [M]  drivers/staging/lustre/lustre/lmv/lmv_obd.o
  CC [M]  drivers/staging/lustre/lustre/lmv/lmv_intent.o
  CC [M]  drivers/staging/lustre/lustre/lmv/lmv_fld.o
  CC [M]  drivers/staging/lustre/lustre/lmv/lproc_lmv.o
drivers/staging/lustre/lustre/lmv/lproc_lmv.c:38:28: fatal error: 
lprocfs_status.h: No such file or directory
 #include <lprocfs_status.h>
                            ^
compilation terminated.


> Signed-off-by: NeilBrown <neilb at suse.com>
> ---
>  drivers/staging/lustre/Makefile                    |    3 +++
>  drivers/staging/lustre/lnet/klnds/o2iblnd/Makefile |    2 --
>  drivers/staging/lustre/lnet/klnds/socklnd/Makefile |    2 --
>  drivers/staging/lustre/lnet/libcfs/Makefile        |    2 --
>  drivers/staging/lustre/lnet/lnet/Makefile          |    2 --
>  drivers/staging/lustre/lnet/selftest/Makefile      |    2 --
>  drivers/staging/lustre/lustre/fid/Makefile         |    2 --
>  drivers/staging/lustre/lustre/fld/Makefile         |    2 --
>  drivers/staging/lustre/lustre/llite/Makefile       |    2 --
>  drivers/staging/lustre/lustre/lmv/Makefile         |    2 --
>  drivers/staging/lustre/lustre/lov/Makefile         |    2 --
>  drivers/staging/lustre/lustre/mdc/Makefile         |    2 --
>  drivers/staging/lustre/lustre/mgc/Makefile         |    2 --
>  drivers/staging/lustre/lustre/obdclass/Makefile    |    2 --
>  drivers/staging/lustre/lustre/obdecho/Makefile     |    2 --
>  drivers/staging/lustre/lustre/osc/Makefile         |    2 --
>  drivers/staging/lustre/lustre/ptlrpc/Makefile      |    2 --
>  17 files changed, 3 insertions(+), 32 deletions(-)
> 
> diff --git a/drivers/staging/lustre/Makefile b/drivers/staging/lustre/Makefile
> index 95ffe337a80a..a44086fa8668 100644
> --- a/drivers/staging/lustre/Makefile
> +++ b/drivers/staging/lustre/Makefile
> @@ -1,2 +1,5 @@
> +subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/include
> +subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/lustre/include
> +
>  obj-$(CONFIG_LNET)		+= lnet/
>  obj-$(CONFIG_LUSTRE_FS)		+= lustre/
> diff --git a/drivers/staging/lustre/lnet/klnds/o2iblnd/Makefile b/drivers/staging/lustre/lnet/klnds/o2iblnd/Makefile
> index 4affe1d79948..e1a05ece130c 100644
> --- a/drivers/staging/lustre/lnet/klnds/o2iblnd/Makefile
> +++ b/drivers/staging/lustre/lnet/klnds/o2iblnd/Makefile
> @@ -1,5 +1,3 @@
> -subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/include
> -subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/lustre/include
>  
>  obj-$(CONFIG_LNET_XPRT_IB) += ko2iblnd.o
>  ko2iblnd-y := o2iblnd.o o2iblnd_cb.o o2iblnd_modparams.o
> diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/Makefile b/drivers/staging/lustre/lnet/klnds/socklnd/Makefile
> index a7da1abfc804..4d03cad997c1 100644
> --- a/drivers/staging/lustre/lnet/klnds/socklnd/Makefile
> +++ b/drivers/staging/lustre/lnet/klnds/socklnd/Makefile
> @@ -1,5 +1,3 @@
> -subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/include
> -subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/lustre/include
>  
>  obj-$(CONFIG_LNET) += ksocklnd.o
>  
> diff --git a/drivers/staging/lustre/lnet/libcfs/Makefile b/drivers/staging/lustre/lnet/libcfs/Makefile
> index 6a1b232da495..3d6b99c6e883 100644
> --- a/drivers/staging/lustre/lnet/libcfs/Makefile
> +++ b/drivers/staging/lustre/lnet/libcfs/Makefile
> @@ -1,6 +1,4 @@
>  # SPDX-License-Identifier: GPL-2.0
> -subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/include
> -subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/lustre/include
>  
>  obj-$(CONFIG_LNET) += libcfs.o
>  
> diff --git a/drivers/staging/lustre/lnet/lnet/Makefile b/drivers/staging/lustre/lnet/lnet/Makefile
> index 0a9d70924fe0..ba33e90e47ec 100644
> --- a/drivers/staging/lustre/lnet/lnet/Makefile
> +++ b/drivers/staging/lustre/lnet/lnet/Makefile
> @@ -1,6 +1,4 @@
>  # SPDX-License-Identifier: GPL-2.0
> -subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/include
> -subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/lustre/include
>  
>  obj-$(CONFIG_LNET) += lnet.o
>  
> diff --git a/drivers/staging/lustre/lnet/selftest/Makefile b/drivers/staging/lustre/lnet/selftest/Makefile
> index 3ccc8966b566..16f8efcd1531 100644
> --- a/drivers/staging/lustre/lnet/selftest/Makefile
> +++ b/drivers/staging/lustre/lnet/selftest/Makefile
> @@ -1,5 +1,3 @@
> -subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/include
> -subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/lustre/include
>  
>  obj-$(CONFIG_LNET_SELFTEST) := lnet_selftest.o
>  
> diff --git a/drivers/staging/lustre/lustre/fid/Makefile b/drivers/staging/lustre/lustre/fid/Makefile
> index 77b65b92667d..11f6e4046e61 100644
> --- a/drivers/staging/lustre/lustre/fid/Makefile
> +++ b/drivers/staging/lustre/lustre/fid/Makefile
> @@ -1,5 +1,3 @@
> -subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/include
> -subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/lustre/include/
>  
>  obj-$(CONFIG_LUSTRE_FS) += fid.o
>  fid-y := fid_request.o fid_lib.o lproc_fid.o
> diff --git a/drivers/staging/lustre/lustre/fld/Makefile b/drivers/staging/lustre/lustre/fld/Makefile
> index 426deba8b815..e7843d7aac9d 100644
> --- a/drivers/staging/lustre/lustre/fld/Makefile
> +++ b/drivers/staging/lustre/lustre/fld/Makefile
> @@ -1,5 +1,3 @@
> -subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/include
> -subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/lustre/include/
>  
>  obj-$(CONFIG_LUSTRE_FS) += fld.o
>  fld-y := fld_request.o fld_cache.o lproc_fld.o
> diff --git a/drivers/staging/lustre/lustre/llite/Makefile b/drivers/staging/lustre/lustre/llite/Makefile
> index 5200924182ae..f46b61770254 100644
> --- a/drivers/staging/lustre/lustre/llite/Makefile
> +++ b/drivers/staging/lustre/lustre/llite/Makefile
> @@ -1,6 +1,4 @@
>  # SPDX-License-Identifier: GPL-2.0
> -subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/include
> -subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/lustre/include
>  
>  obj-$(CONFIG_LUSTRE_FS) += lustre.o
>  lustre-y := dcache.o dir.o file.o llite_lib.o llite_nfs.o \
> diff --git a/drivers/staging/lustre/lustre/lmv/Makefile b/drivers/staging/lustre/lustre/lmv/Makefile
> index 91c99114aa13..7a392c31c508 100644
> --- a/drivers/staging/lustre/lustre/lmv/Makefile
> +++ b/drivers/staging/lustre/lustre/lmv/Makefile
> @@ -1,5 +1,3 @@
> -subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/include
> -subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/lustre/include
>  
>  obj-$(CONFIG_LUSTRE_FS) += lmv.o
>  lmv-y := lmv_obd.o lmv_intent.o lmv_fld.o lproc_lmv.o
> diff --git a/drivers/staging/lustre/lustre/lov/Makefile b/drivers/staging/lustre/lustre/lov/Makefile
> index 1ebf0193f61a..6294769aca95 100644
> --- a/drivers/staging/lustre/lustre/lov/Makefile
> +++ b/drivers/staging/lustre/lustre/lov/Makefile
> @@ -1,6 +1,4 @@
>  # SPDX-License-Identifier: GPL-2.0
> -subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/include
> -subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/lustre/include
>  
>  obj-$(CONFIG_LUSTRE_FS) += lov.o
>  lov-y := lov_obd.o lov_pack.o lov_offset.o lov_merge.o \
> diff --git a/drivers/staging/lustre/lustre/mdc/Makefile b/drivers/staging/lustre/lustre/mdc/Makefile
> index c7bc3351ccb0..c3bc71f6596d 100644
> --- a/drivers/staging/lustre/lustre/mdc/Makefile
> +++ b/drivers/staging/lustre/lustre/mdc/Makefile
> @@ -1,5 +1,3 @@
> -subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/include
> -subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/lustre/include
>  
>  obj-$(CONFIG_LUSTRE_FS) += mdc.o
>  mdc-y := mdc_request.o mdc_reint.o mdc_lib.o mdc_locks.o lproc_mdc.o
> diff --git a/drivers/staging/lustre/lustre/mgc/Makefile b/drivers/staging/lustre/lustre/mgc/Makefile
> index 8abf108dbcf7..301cd7a1fa25 100644
> --- a/drivers/staging/lustre/lustre/mgc/Makefile
> +++ b/drivers/staging/lustre/lustre/mgc/Makefile
> @@ -1,5 +1,3 @@
> -subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/include
> -subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/lustre/include
>  
>  obj-$(CONFIG_LUSTRE_FS) += mgc.o
>  mgc-y := mgc_request.o lproc_mgc.o
> diff --git a/drivers/staging/lustre/lustre/obdclass/Makefile b/drivers/staging/lustre/lustre/obdclass/Makefile
> index e36ba2167d10..be118391b402 100644
> --- a/drivers/staging/lustre/lustre/obdclass/Makefile
> +++ b/drivers/staging/lustre/lustre/obdclass/Makefile
> @@ -1,6 +1,4 @@
>  # SPDX-License-Identifier: GPL-2.0
> -subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/include
> -subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/lustre/include
>  
>  obj-$(CONFIG_LUSTRE_FS) += obdclass.o
>  
> diff --git a/drivers/staging/lustre/lustre/obdecho/Makefile b/drivers/staging/lustre/lustre/obdecho/Makefile
> index 6be66fbab872..4ae346f5ddcd 100644
> --- a/drivers/staging/lustre/lustre/obdecho/Makefile
> +++ b/drivers/staging/lustre/lustre/obdecho/Makefile
> @@ -1,5 +1,3 @@
> -subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/include
> -subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/lustre/include
>  
>  obj-$(CONFIG_LUSTRE_FS) += obdecho.o
>  obdecho-y := echo_client.o
> diff --git a/drivers/staging/lustre/lustre/osc/Makefile b/drivers/staging/lustre/lustre/osc/Makefile
> index 30dec90e64e8..ecdf07e925c7 100644
> --- a/drivers/staging/lustre/lustre/osc/Makefile
> +++ b/drivers/staging/lustre/lustre/osc/Makefile
> @@ -1,5 +1,3 @@
> -subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/include
> -subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/lustre/include
>  
>  obj-$(CONFIG_LUSTRE_FS) += osc.o
>  osc-y := osc_request.o osc_dev.o osc_object.o \
> diff --git a/drivers/staging/lustre/lustre/ptlrpc/Makefile b/drivers/staging/lustre/lustre/ptlrpc/Makefile
> index aa152e652002..0fae7de09de2 100644
> --- a/drivers/staging/lustre/lustre/ptlrpc/Makefile
> +++ b/drivers/staging/lustre/lustre/ptlrpc/Makefile
> @@ -1,6 +1,4 @@
>  # SPDX-License-Identifier: GPL-2.0
> -subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/include
> -subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/lustre/include
>  
>  obj-$(CONFIG_LUSTRE_FS) += ptlrpc.o
>  
> 
> 
> 

From jsimmons at infradead.org  Wed Jun 13 21:46:33 2018
From: jsimmons at infradead.org (James Simmons)
Date: Wed, 13 Jun 2018 22:46:33 +0100 (BST)
Subject: [lustre-devel] A new drivers/staging/lustre
In-Reply-To: <87lgbkkemm.fsf@notabene.neil.brown.name>
References: <874lifnxbp.fsf@notabene.neil.brown.name>
 <alpine.LFD.2.21.1806120112530.15735@casper.infradead.org>
 <87lgbkkemm.fsf@notabene.neil.brown.name>
Message-ID: <alpine.LFD.2.21.1806132244190.12664@casper.infradead.org>


> 
> >> Greg's patch to remove lustre has now landed in this staging-next tree,
> >> so I suspect it will get to Linus before too long.  So I have to find a
> >> new place to work on lustre.
> >> 
> >> I've added 2 branches to
> >>    git://git.neil.brown.name/linux
> >
> > Its seems to be down. Do you plan to move the tree somewhere else. Perhaps
> > your github page?
> 
> Bother - that's what you get for using Linux to host things (my VM
> doesn't have heaps of memory and git likes to use a lot, and doesn't
> always clean up well after itself).
> 
> git.neil.brown.name is back, and github.com/neilbrown/linux now
> has two new branches: lustre/lutre and lustre/lustre-testing

It lives!!! Trying to grab the tree from git.neil.brown.name and still 
chokes but the github tree works works like a charm. I built and tested
the tree and it works.

From jsimmons at infradead.org  Wed Jun 13 22:02:16 2018
From: jsimmons at infradead.org (James Simmons)
Date: Wed, 13 Jun 2018 23:02:16 +0100 (BST)
Subject: [lustre-devel] [PATCH v2 01/25] staging: lustre: libcfs:
 restore UMP handling
In-Reply-To: <87muwiuiuz.fsf@notabene.neil.brown.name>
References: <1527603725-30560-1-git-send-email-jsimmons@infradead.org>
 <1527603725-30560-2-git-send-email-jsimmons@infradead.org>
 <87muwiuiuz.fsf@notabene.neil.brown.name>
Message-ID: <alpine.LFD.2.21.1806132248500.12664@casper.infradead.org>


> > With the cleanup of the libcfs SMP handling all UMP handling
> > was removed. In the process now various NULL pointers and
> > empty fields are return in the UMP case which causes lustre
> > to crash hard. Restore the proper UMP handling so Lustre can
> > properly function.
> 
> Can't we just get lustre to handle the NULL pointer?
> Is most cases, the pointer is accessed through an accessor function, and
> on !CONFIG_SMP, that can be a static inline that doesn't even look at
> the pointer.

Lots of NULL pointer checks for a structure allocated at libcfs module 
start and only cleaned up at libcfs removal is not a clean approach.
So I have thought about it and I have to ask why allocate a global
struct cfs_cpu_table. It could be made static and fill it in which would
avoid the whole NULL pointer issue. Plus for the UMP case why allocate
a new cfs_cpu_table with cfs_cpt_table_alloc() which is exactly like
the default UMP cfs_cpu_table. Instead we could just return the pointer
to the static default cfs_cpt_tab every time. We still have the NULL
ctb_cpumask field to deal with. Does that sound like a better solution
to you? Doug what do you think?
 
> I really think this is a step backwards.  If you can identify specific
> problems caused by the current code, I'm sure we can fix them.
> 
> >
> > Signed-off-by: James Simmons <uja.ornl at yahoo.com>
> > Signed-off-by: Amir Shehata <amir.shehata at intel.com>
> > Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-7734
> 
> This bug doesn't seem to mention this patch at all
> 
> > Reviewed-on: http://review.whamcloud.com/18916
> 
> Nor does this review.

Yeah its mutated so much from what is in the Intel tree.
I do believe it was the last patch to touch this.

From doucharek at cray.com  Wed Jun 13 22:18:03 2018
From: doucharek at cray.com (Doug Oucharek)
Date: Wed, 13 Jun 2018 22:18:03 +0000
Subject: [lustre-devel] [PATCH v2 01/25] staging: lustre: libcfs:
 restore UMP handling
In-Reply-To: <alpine.LFD.2.21.1806132248500.12664@casper.infradead.org>
References: <1527603725-30560-1-git-send-email-jsimmons@infradead.org>
 <1527603725-30560-2-git-send-email-jsimmons@infradead.org>
 <87muwiuiuz.fsf@notabene.neil.brown.name>
 <alpine.LFD.2.21.1806132248500.12664@casper.infradead.org>
Message-ID: <17F46A71-4A75-499D-9220-A8503C6C26F3@cray.com>


> On Jun 13, 2018, at 3:02 PM, James Simmons <jsimmons at infradead.org> wrote:
> 
> 
>>> With the cleanup of the libcfs SMP handling all UMP handling
>>> was removed. In the process now various NULL pointers and
>>> empty fields are return in the UMP case which causes lustre
>>> to crash hard. Restore the proper UMP handling so Lustre can
>>> properly function.
>> 
>> Can't we just get lustre to handle the NULL pointer?
>> Is most cases, the pointer is accessed through an accessor function, and
>> on !CONFIG_SMP, that can be a static inline that doesn't even look at
>> the pointer.
> 
> Lots of NULL pointer checks for a structure allocated at libcfs module 
> start and only cleaned up at libcfs removal is not a clean approach.
> So I have thought about it and I have to ask why allocate a global
> struct cfs_cpu_table. It could be made static and fill it in which would
> avoid the whole NULL pointer issue. Plus for the UMP case why allocate
> a new cfs_cpu_table with cfs_cpt_table_alloc() which is exactly like
> the default UMP cfs_cpu_table. Instead we could just return the pointer
> to the static default cfs_cpt_tab every time. We still have the NULL
> ctb_cpumask field to deal with. Does that sound like a better solution
> to you? Doug what do you think?

I like your suggestion, James.  Always having a cfs_cpu_table so we don’t have to worry about a NULL pointer would be good.  I think we can track down the code which relies on the cpumask field to make it safe for the NULL case.  Is there possibility of having an empty cpumask rather than it being NULL?

> 
>> I really think this is a step backwards.  If you can identify specific
>> problems caused by the current code, I'm sure we can fix them.
>> 
>>> 
>>> Signed-off-by: James Simmons <uja.ornl at yahoo.com>
>>> Signed-off-by: Amir Shehata <amir.shehata at intel.com>
>>> Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-7734
>> 
>> This bug doesn't seem to mention this patch at all
>> 
>>> Reviewed-on: http://review.whamcloud.com/18916
>> 
>> Nor does this review.
> 
> Yeah its mutated so much from what is in the Intel tree.
> I do believe it was the last patch to touch this.
> _______________________________________________
> lustre-devel mailing list
> lustre-devel at lists.lustre.org
> http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org


From neilb at suse.com  Wed Jun 13 22:29:28 2018
From: neilb at suse.com (NeilBrown)
Date: Thu, 14 Jun 2018 08:29:28 +1000
Subject: [lustre-devel] [PATCH v2 01/25] staging: lustre: libcfs:
 restore UMP handling
In-Reply-To: <alpine.LFD.2.21.1806132248500.12664@casper.infradead.org>
References: <1527603725-30560-1-git-send-email-jsimmons@infradead.org>
 <1527603725-30560-2-git-send-email-jsimmons@infradead.org>
 <87muwiuiuz.fsf@notabene.neil.brown.name>
 <alpine.LFD.2.21.1806132248500.12664@casper.infradead.org>
Message-ID: <87h8m6ialz.fsf@notabene.neil.brown.name>

On Wed, Jun 13 2018, James Simmons wrote:

>> > With the cleanup of the libcfs SMP handling all UMP handling
>> > was removed. In the process now various NULL pointers and
>> > empty fields are return in the UMP case which causes lustre
>> > to crash hard. Restore the proper UMP handling so Lustre can
>> > properly function.
>> 
>> Can't we just get lustre to handle the NULL pointer?
>> Is most cases, the pointer is accessed through an accessor function, and
>> on !CONFIG_SMP, that can be a static inline that doesn't even look at
>> the pointer.
>
> Lots of NULL pointer checks for a structure allocated at libcfs module 
> start and only cleaned up at libcfs removal is not a clean approach.
> So I have thought about it and I have to ask why allocate a global
> struct cfs_cpu_table. It could be made static and fill it in which would
> avoid the whole NULL pointer issue. Plus for the UMP case why allocate
> a new cfs_cpu_table with cfs_cpt_table_alloc() which is exactly like
> the default UMP cfs_cpu_table. Instead we could just return the pointer
> to the static default cfs_cpt_tab every time. We still have the NULL
> ctb_cpumask field to deal with. Does that sound like a better solution
> to you? Doug what do you think?

I'm not convinced there will be lots of NULL pointer checks - maybe one
or two.  Most the accesses to the structure are inside helper
functions which are empty inlines in the UP build.

However I don't object to a static cfs_cpt_tab if that turns out to make
some code simpler.   I would want it to be clear up-front which code was
simplified so that an informed decision could be made.

Thanks,
NeilBrown


>  
>> I really think this is a step backwards.  If you can identify specific
>> problems caused by the current code, I'm sure we can fix them.
>> 
>> >
>> > Signed-off-by: James Simmons <uja.ornl at yahoo.com>
>> > Signed-off-by: Amir Shehata <amir.shehata at intel.com>
>> > Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-7734
>> 
>> This bug doesn't seem to mention this patch at all
>> 
>> > Reviewed-on: http://review.whamcloud.com/18916
>> 
>> Nor does this review.
>
> Yeah its mutated so much from what is in the Intel tree.
> I do believe it was the last patch to touch this.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180614/0fb53af7/attachment.sig>

From neilb at suse.com  Wed Jun 13 22:31:43 2018
From: neilb at suse.com (NeilBrown)
Date: Thu, 14 Jun 2018 08:31:43 +1000
Subject: [lustre-devel] A new drivers/staging/lustre
In-Reply-To: <alpine.LFD.2.21.1806132244190.12664@casper.infradead.org>
References: <874lifnxbp.fsf@notabene.neil.brown.name>
 <alpine.LFD.2.21.1806120112530.15735@casper.infradead.org>
 <87lgbkkemm.fsf@notabene.neil.brown.name>
 <alpine.LFD.2.21.1806132244190.12664@casper.infradead.org>
Message-ID: <87efhaiai8.fsf@notabene.neil.brown.name>

On Wed, Jun 13 2018, James Simmons wrote:

>> 
>> >> Greg's patch to remove lustre has now landed in this staging-next tree,
>> >> so I suspect it will get to Linus before too long.  So I have to find a
>> >> new place to work on lustre.
>> >> 
>> >> I've added 2 branches to
>> >>    git://git.neil.brown.name/linux
>> >
>> > Its seems to be down. Do you plan to move the tree somewhere else. Perhaps
>> > your github page?
>> 
>> Bother - that's what you get for using Linux to host things (my VM
>> doesn't have heaps of memory and git likes to use a lot, and doesn't
>> always clean up well after itself).
>> 
>> git.neil.brown.name is back, and github.com/neilbrown/linux now
>> has two new branches: lustre/lutre and lustre/lustre-testing
>
> It lives!!! Trying to grab the tree from git.neil.brown.name and still 
> chokes but the github tree works works like a charm. I built and tested
> the tree and it works.

I guess I'm going to have to deprecate using neil.brown.name for git, or
else arrange for a lot more memory. linux.git is getting too big.
It probably works OK if you "git clone --reference=/some/local/linux/tree/.git"
but people don't think to do that.
I'll make sure to keep github up to date.

Thanks,
NeilBrown
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180614/11f91e72/attachment.sig>

From neilb at suse.com  Wed Jun 13 23:21:37 2018
From: neilb at suse.com (NeilBrown)
Date: Thu, 14 Jun 2018 09:21:37 +1000
Subject: [lustre-devel] [PATCH 11/11] staging: lustre: centralize
 setting of subdir-ccflags-y
In-Reply-To: <alpine.LFD.2.21.1806132237140.12664@casper.infradead.org>
References: <152826510267.16761.14361003167157833896.stgit@noble>
 <152826511930.16761.12770722873423628004.stgit@noble>
 <alpine.LFD.2.21.1806132237140.12664@casper.infradead.org>
Message-ID: <871sdai872.fsf@notabene.neil.brown.name>

On Wed, Jun 13 2018, James Simmons wrote:

>> We don't need to set subdir-ccflags-y in every Makefile.
>> The whole point of the "subdir-" bit is that the setting
>> can go once in the top-level Makefile.
>> 
>
> Nak: When attempting to build individual components I get:
>
> ~/lustre-upstream$ make SUBDIRS=drivers/staging/lustre/lustre/lmv modules 
> -j 16
>
>   WARNING: Symbol version dump ./Module.symvers
>            is missing; modules will have no dependencies and modversions.
>
>   CC [M]  drivers/staging/lustre/lustre/lmv/lmv_obd.o
>   CC [M]  drivers/staging/lustre/lustre/lmv/lmv_intent.o
>   CC [M]  drivers/staging/lustre/lustre/lmv/lmv_fld.o
>   CC [M]  drivers/staging/lustre/lustre/lmv/lproc_lmv.o
> drivers/staging/lustre/lustre/lmv/lproc_lmv.c:38:28: fatal error: 
> lprocfs_status.h: No such file or directory
>  #include <lprocfs_status.h>
>                             ^
> compilation terminated.

Yes, I noticed that too.  Thanks.
It seems to suggest that subdir-ccflags-y isn't really a useful macro.
Maybe we should stick with cc-flags-y.  It is much more widely used.

Thanks,
NeilBrown

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180614/68f87a5a/attachment.sig>

From jsimmons at infradead.org  Thu Jun 14 02:38:03 2018
From: jsimmons at infradead.org (James Simmons)
Date: Thu, 14 Jun 2018 03:38:03 +0100 (BST)
Subject: [lustre-devel] [PATCH 07/11] staging: lustre: fold
 lprocfs_call_handler functionality into lnet_debugfs_*
In-Reply-To: <152826511912.16761.6908134754944227444.stgit@noble>
References: <152826510267.16761.14361003167157833896.stgit@noble>
 <152826511912.16761.6908134754944227444.stgit@noble>
Message-ID: <alpine.LFD.2.21.1806140336320.6734@casper.infradead.org>


> The calling convention for ->proc_handler is rather clumsy,
> as a comment in fs/procfs/proc_sysctl.c confirms.
> lustre has copied this convention to lnet_debugfs_{read,write},
> and then provided a wrapper for handlers - lprocfs_call_handler -
> to work around the clumsiness.
> 
> It is cleaner to just fold the functionality of lprocfs_call_handler()
> into lnet_debugfs_* and let them call the final handler directly.
> 
> If these files were ever moved to /proc/sys (which seems unlikely) the
> handling in fs/procfs/proc_sysctl.c would need to be fixed to, but
> that would not be a bad thing.
> 
> So modify all the functions that did use the wrapper to not need it
> now that a more sane calling convention is available.

Reviewed-by: James Simmons <jsimmons at infradead.org>
 
> Signed-off-by: NeilBrown <neilb at suse.com>
> ---
>  .../staging/lustre/include/linux/libcfs/libcfs.h   |    4 -
>  drivers/staging/lustre/lnet/libcfs/module.c        |   84 +++++++-------------
>  drivers/staging/lustre/lnet/lnet/router_proc.c     |   41 +++-------
>  3 files changed, 41 insertions(+), 88 deletions(-)
> 
> diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs.h b/drivers/staging/lustre/include/linux/libcfs/libcfs.h
> index edc7ed0dcb94..7ac609328256 100644
> --- a/drivers/staging/lustre/include/linux/libcfs/libcfs.h
> +++ b/drivers/staging/lustre/include/linux/libcfs/libcfs.h
> @@ -57,10 +57,6 @@ int libcfs_setup(void);
>  extern struct workqueue_struct *cfs_rehash_wq;
>  
>  void lustre_insert_debugfs(struct ctl_table *table);
> -int lprocfs_call_handler(void *data, int write, loff_t *ppos,
> -			 void __user *buffer, size_t *lenp,
> -			 int (*handler)(void *data, int write, loff_t pos,
> -					void __user *buffer, int len));
>  
>  /*
>   * Memory
> diff --git a/drivers/staging/lustre/lnet/libcfs/module.c b/drivers/staging/lustre/lnet/libcfs/module.c
> index 5dc7de9e6478..02c404c6738e 100644
> --- a/drivers/staging/lustre/lnet/libcfs/module.c
> +++ b/drivers/staging/lustre/lnet/libcfs/module.c
> @@ -290,33 +290,15 @@ static struct miscdevice libcfs_dev = {
>  
>  static int libcfs_dev_registered;
>  
> -int lprocfs_call_handler(void *data, int write, loff_t *ppos,
> -			 void __user *buffer, size_t *lenp,
> -			 int (*handler)(void *data, int write, loff_t pos,
> -					void __user *buffer, int len))
> -{
> -	int rc = handler(data, write, *ppos, buffer, *lenp);
> -
> -	if (rc < 0)
> -		return rc;
> -
> -	if (write) {
> -		*ppos += *lenp;
> -	} else {
> -		*lenp = rc;
> -		*ppos += rc;
> -	}
> -	return 0;
> -}
> -EXPORT_SYMBOL(lprocfs_call_handler);
> -
> -static int __proc_dobitmasks(void *data, int write,
> -			     loff_t pos, void __user *buffer, int nob)
> +static int proc_dobitmasks(struct ctl_table *table, int write,
> +			   void __user *buffer, size_t *lenp, loff_t *ppos)
>  {
>  	const int tmpstrlen = 512;
>  	char *tmpstr;
>  	int rc;
> -	unsigned int *mask = data;
> +	size_t nob = *lenp;
> +	loff_t pos = *ppos;
> +	unsigned int *mask = table->data;
>  	int is_subsys = (mask == &libcfs_subsystem_debug) ? 1 : 0;
>  	int is_printk = (mask == &libcfs_printk) ? 1 : 0;
>  
> @@ -351,32 +333,23 @@ static int __proc_dobitmasks(void *data, int write,
>  	return rc;
>  }
>  
> -static int proc_dobitmasks(struct ctl_table *table, int write,
> -			   void __user *buffer, size_t *lenp, loff_t *ppos)
> +static int proc_dump_kernel(struct ctl_table *table, int write,
> +			    void __user *buffer, size_t *lenp, loff_t *ppos)
>  {
> -	return lprocfs_call_handler(table->data, write, ppos, buffer, lenp,
> -				    __proc_dobitmasks);
> -}
> +	size_t nob = *lenp;
>  
> -static int __proc_dump_kernel(void *data, int write,
> -			      loff_t pos, void __user *buffer, int nob)
> -{
>  	if (!write)
>  		return 0;
>  
>  	return cfs_trace_dump_debug_buffer_usrstr(buffer, nob);
>  }
>  
> -static int proc_dump_kernel(struct ctl_table *table, int write,
> +static int proc_daemon_file(struct ctl_table *table, int write,
>  			    void __user *buffer, size_t *lenp, loff_t *ppos)
>  {
> -	return lprocfs_call_handler(table->data, write, ppos, buffer, lenp,
> -				    __proc_dump_kernel);
> -}
> +	size_t nob = *lenp;
> +	loff_t pos = *ppos;
>  
> -static int __proc_daemon_file(void *data, int write,
> -			      loff_t pos, void __user *buffer, int nob)
> -{
>  	if (!write) {
>  		int len = strlen(cfs_tracefile);
>  
> @@ -390,13 +363,6 @@ static int __proc_daemon_file(void *data, int write,
>  	return cfs_trace_daemon_command_usrstr(buffer, nob);
>  }
>  
> -static int proc_daemon_file(struct ctl_table *table, int write,
> -			    void __user *buffer, size_t *lenp, loff_t *ppos)
> -{
> -	return lprocfs_call_handler(table->data, write, ppos, buffer, lenp,
> -				    __proc_daemon_file);
> -}
> -
>  static int libcfs_force_lbug(struct ctl_table *table, int write,
>  			     void __user *buffer,
>  			     size_t *lenp, loff_t *ppos)
> @@ -419,9 +385,11 @@ static int proc_fail_loc(struct ctl_table *table, int write,
>  	return rc;
>  }
>  
> -static int __proc_cpt_table(void *data, int write,
> -			    loff_t pos, void __user *buffer, int nob)
> +static int proc_cpt_table(struct ctl_table *table, int write,
> +			  void __user *buffer, size_t *lenp, loff_t *ppos)
>  {
> +	size_t nob = *lenp;
> +	loff_t pos = *ppos;
>  	char *buf = NULL;
>  	int len = 4096;
>  	int rc  = 0;
> @@ -457,13 +425,6 @@ static int __proc_cpt_table(void *data, int write,
>  	return rc;
>  }
>  
> -static int proc_cpt_table(struct ctl_table *table, int write,
> -			  void __user *buffer, size_t *lenp, loff_t *ppos)
> -{
> -	return lprocfs_call_handler(table->data, write, ppos, buffer, lenp,
> -				    __proc_cpt_table);
> -}
> -
>  static struct ctl_table lnet_table[] = {
>  	{
>  		.procname = "debug",
> @@ -573,10 +534,17 @@ static ssize_t lnet_debugfs_read(struct file *filp, char __user *buf,
>  {
>  	struct ctl_table *table = filp->private_data;
>  	int error;
> +	loff_t old_pos = *ppos;
>  
>  	error = table->proc_handler(table, 0, (void __user *)buf, &count, ppos);
> -	if (!error)
> +	/*
> +	 * On success, the length read is either in error or in count.
> +	 * If ppos changed, then use count, else use error
> +	 */
> +	if (!error && *ppos != old_pos)
>  		error = count;
> +	else if (error > 0)
> +		*ppos += error;
>  
>  	return error;
>  }
> @@ -586,10 +554,14 @@ static ssize_t lnet_debugfs_write(struct file *filp, const char __user *buf,
>  {
>  	struct ctl_table *table = filp->private_data;
>  	int error;
> +	loff_t old_pos = *ppos;
>  
>  	error = table->proc_handler(table, 1, (void __user *)buf, &count, ppos);
> -	if (!error)
> +	if (!error) {
>  		error = count;
> +		if (*ppos == old_pos)
> +			*ppos += count;
> +	}
>  
>  	return error;
>  }
> diff --git a/drivers/staging/lustre/lnet/lnet/router_proc.c b/drivers/staging/lustre/lnet/lnet/router_proc.c
> index ae4b7f5953a0..f135082fec5c 100644
> --- a/drivers/staging/lustre/lnet/lnet/router_proc.c
> +++ b/drivers/staging/lustre/lnet/lnet/router_proc.c
> @@ -74,11 +74,13 @@
>  
>  #define LNET_PROC_VERSION(v)	((unsigned int)((v) & LNET_PROC_VER_MASK))
>  
> -static int __proc_lnet_stats(void *data, int write,
> -			     loff_t pos, void __user *buffer, int nob)
> +static int proc_lnet_stats(struct ctl_table *table, int write,
> +			   void __user *buffer, size_t *lenp, loff_t *ppos)
>  {
>  	int rc;
>  	struct lnet_counters *ctrs;
> +	size_t nob = *lenp;
> +	loff_t pos = *ppos;
>  	int len;
>  	char *tmpstr;
>  	const int tmpsiz = 256; /* 7 %u and 4 %llu */
> @@ -122,13 +124,6 @@ static int __proc_lnet_stats(void *data, int write,
>  	return rc;
>  }
>  
> -static int proc_lnet_stats(struct ctl_table *table, int write,
> -			   void __user *buffer, size_t *lenp, loff_t *ppos)
> -{
> -	return lprocfs_call_handler(table->data, write, ppos, buffer, lenp,
> -				    __proc_lnet_stats);
> -}
> -
>  static int proc_lnet_routes(struct ctl_table *table, int write,
>  			    void __user *buffer, size_t *lenp, loff_t *ppos)
>  {
> @@ -562,9 +557,11 @@ static int proc_lnet_peers(struct ctl_table *table, int write,
>  	return rc;
>  }
>  
> -static int __proc_lnet_buffers(void *data, int write,
> -			       loff_t pos, void __user *buffer, int nob)
> +static int proc_lnet_buffers(struct ctl_table *table, int write,
> +			     void __user *buffer, size_t *lenp, loff_t *ppos)
>  {
> +	size_t nob = *lenp;
> +	loff_t pos = *ppos;
>  	char *s;
>  	char *tmpstr;
>  	int tmpsiz;
> @@ -620,13 +617,6 @@ static int __proc_lnet_buffers(void *data, int write,
>  	return rc;
>  }
>  
> -static int proc_lnet_buffers(struct ctl_table *table, int write,
> -			     void __user *buffer, size_t *lenp, loff_t *ppos)
> -{
> -	return lprocfs_call_handler(table->data, write, ppos, buffer, lenp,
> -				    __proc_lnet_buffers);
> -}
> -
>  static int proc_lnet_nis(struct ctl_table *table, int write,
>  			 void __user *buffer, size_t *lenp, loff_t *ppos)
>  {
> @@ -784,10 +774,13 @@ static struct lnet_portal_rotors	portal_rotors[] = {
>  	},
>  };
>  
> -static int __proc_lnet_portal_rotor(void *data, int write,
> -				    loff_t pos, void __user *buffer, int nob)
> +static int proc_lnet_portal_rotor(struct ctl_table *table, int write,
> +				  void __user *buffer, size_t *lenp,
> +				  loff_t *ppos)
>  {
>  	const int buf_len = 128;
> +	size_t nob = *lenp;
> +	loff_t pos = *ppos;
>  	char *buf;
>  	char *tmp;
>  	int rc;
> @@ -845,14 +838,6 @@ static int __proc_lnet_portal_rotor(void *data, int write,
>  	return rc;
>  }
>  
> -static int proc_lnet_portal_rotor(struct ctl_table *table, int write,
> -				  void __user *buffer, size_t *lenp,
> -				  loff_t *ppos)
> -{
> -	return lprocfs_call_handler(table->data, write, ppos, buffer, lenp,
> -				    __proc_lnet_portal_rotor);
> -}
> -
>  static struct ctl_table lnet_table[] = {
>  	/*
>  	 * NB No .strategy entries have been provided since sysctl(8) prefers
> 
> 
> 

From jsimmons at infradead.org  Thu Jun 14 02:38:58 2018
From: jsimmons at infradead.org (James Simmons)
Date: Thu, 14 Jun 2018 03:38:58 +0100 (BST)
Subject: [lustre-devel] [PATCH 09/11] staging: lustre: discard WIRE_ATTR
In-Reply-To: <152826511920.16761.2460891067961959676.stgit@noble>
References: <152826510267.16761.14361003167157833896.stgit@noble>
 <152826511920.16761.2460891067961959676.stgit@noble>
Message-ID: <alpine.LFD.2.21.1806140338150.6734@casper.infradead.org>


> This macro adds nothing of value, and make the code harder
> to read for new readers.

Reviewed-by: James Simmons <jsimmons at infradead.org>
 
> Signed-off-by: NeilBrown <neilb at suse.com>
> ---
>  .../staging/lustre/include/linux/lnet/socklnd.h    |    8 ++-
>  .../lustre/include/uapi/linux/lnet/lnet-types.h    |   28 +++++-------
>  .../lustre/include/uapi/linux/lnet/lnetst.h        |    4 +-
>  .../staging/lustre/lnet/klnds/o2iblnd/o2iblnd.h    |   22 +++++----
>  drivers/staging/lustre/lnet/selftest/rpc.h         |   48 ++++++++++----------
>  5 files changed, 54 insertions(+), 56 deletions(-)
> 
> diff --git a/drivers/staging/lustre/include/linux/lnet/socklnd.h b/drivers/staging/lustre/include/linux/lnet/socklnd.h
> index 6bd1bca190a3..9f69257e000b 100644
> --- a/drivers/staging/lustre/include/linux/lnet/socklnd.h
> +++ b/drivers/staging/lustre/include/linux/lnet/socklnd.h
> @@ -50,7 +50,7 @@ struct ksock_hello_msg {
>  	__u32		kshm_ctype;	/* connection type */
>  	__u32		kshm_nips;	/* # IP addrs */
>  	__u32		kshm_ips[0];	/* IP addrs */
> -} WIRE_ATTR;
> +} __packed;
>  
>  struct ksock_lnet_msg {
>  	struct lnet_hdr	ksnm_hdr;	/* lnet hdr */
> @@ -61,7 +61,7 @@ struct ksock_lnet_msg {
>  	 * structure definitions. lnet payload will be stored just after
>  	 * the body of structure ksock_lnet_msg_t
>  	 */
> -} WIRE_ATTR;
> +} __packed;
>  
>  struct ksock_msg {
>  	__u32	ksm_type;		/* type of socklnd message */
> @@ -71,8 +71,8 @@ struct ksock_msg {
>  		struct ksock_lnet_msg lnetmsg; /* lnet message, it's empty if
>  						* it's NOOP
>  						*/
> -	} WIRE_ATTR ksm_u;
> -} WIRE_ATTR;
> +	} __packed ksm_u;
> +} __packed;
>  
>  #define KSOCK_MSG_NOOP	0xC0	/* ksm_u empty */
>  #define KSOCK_MSG_LNET	0xC1	/* lnet msg */
> diff --git a/drivers/staging/lustre/include/uapi/linux/lnet/lnet-types.h b/drivers/staging/lustre/include/uapi/linux/lnet/lnet-types.h
> index 1be9b7aa7326..f97e7d9d881f 100644
> --- a/drivers/staging/lustre/include/uapi/linux/lnet/lnet-types.h
> +++ b/drivers/staging/lustre/include/uapi/linux/lnet/lnet-types.h
> @@ -112,14 +112,12 @@ static inline __u32 LNET_MKNET(__u32 type, __u32 num)
>  	return (type << 16) | num;
>  }
>  
> -#define WIRE_ATTR	__packed
> -
>  /* Packed version of lnet_process_id to transfer via network */
>  struct lnet_process_id_packed {
>  	/* node id / process id */
>  	lnet_nid_t	nid;
>  	lnet_pid_t	pid;
> -} WIRE_ATTR;
> +} __packed;
>  
>  /*
>   * The wire handle's interface cookie only matches one network interface in
> @@ -130,7 +128,7 @@ struct lnet_process_id_packed {
>  struct lnet_handle_wire {
>  	__u64	wh_interface_cookie;
>  	__u64	wh_object_cookie;
> -} WIRE_ATTR;
> +} __packed;
>  
>  enum lnet_msg_type {
>  	LNET_MSG_ACK = 0,
> @@ -150,7 +148,7 @@ struct lnet_ack {
>  	struct lnet_handle_wire	dst_wmd;
>  	__u64			match_bits;
>  	__u32			mlength;
> -} WIRE_ATTR;
> +} __packed;
>  
>  struct lnet_put {
>  	struct lnet_handle_wire	ack_wmd;
> @@ -158,7 +156,7 @@ struct lnet_put {
>  	__u64			hdr_data;
>  	__u32			ptl_index;
>  	__u32			offset;
> -} WIRE_ATTR;
> +} __packed;
>  
>  struct lnet_get {
>  	struct lnet_handle_wire	return_wmd;
> @@ -166,16 +164,16 @@ struct lnet_get {
>  	__u32			ptl_index;
>  	__u32			src_offset;
>  	__u32			sink_length;
> -} WIRE_ATTR;
> +} __packed;
>  
>  struct lnet_reply {
>  	struct lnet_handle_wire	dst_wmd;
> -} WIRE_ATTR;
> +} __packed;
>  
>  struct lnet_hello {
>  	__u64			incarnation;
>  	__u32			type;
> -} WIRE_ATTR;
> +} __packed;
>  
>  struct lnet_hdr {
>  	lnet_nid_t	dest_nid;
> @@ -192,7 +190,7 @@ struct lnet_hdr {
>  		struct lnet_reply	reply;
>  		struct lnet_hello	hello;
>  	} msg;
> -} WIRE_ATTR;
> +} __packed;
>  
>  /*
>   * A HELLO message contains a magic number and protocol version
> @@ -208,7 +206,7 @@ struct lnet_magicversion {
>  	__u32	magic;		/* LNET_PROTO_TCP_MAGIC */
>  	__u16	version_major;	/* increment on incompatible change */
>  	__u16	version_minor;	/* increment on compatible change */
> -} WIRE_ATTR;
> +} __packed;
>  
>  /* PROTO MAGIC for LNDs */
>  #define LNET_PROTO_IB_MAGIC		0x0be91b91
> @@ -232,7 +230,7 @@ struct lnet_acceptor_connreq {
>  	__u32	acr_magic;		/* PTL_ACCEPTOR_PROTO_MAGIC */
>  	__u32	acr_version;		/* protocol version */
>  	__u64	acr_nid;		/* target NID */
> -} WIRE_ATTR;
> +} __packed;
>  
>  #define LNET_PROTO_ACCEPTOR_VERSION	1
>  
> @@ -240,7 +238,7 @@ struct lnet_ni_status {
>  	lnet_nid_t	ns_nid;
>  	__u32		ns_status;
>  	__u32		ns_unused;
> -} WIRE_ATTR;
> +} __packed;
>  
>  struct lnet_ping_info {
>  	__u32			pi_magic;
> @@ -248,7 +246,7 @@ struct lnet_ping_info {
>  	lnet_pid_t		pi_pid;
>  	__u32			pi_nnis;
>  	struct lnet_ni_status	pi_ni[0];
> -} WIRE_ATTR;
> +} __packed;
>  
>  struct lnet_counters {
>  	__u32	msgs_alloc;
> @@ -262,7 +260,7 @@ struct lnet_counters {
>  	__u64	recv_length;
>  	__u64	route_length;
>  	__u64	drop_length;
> -} WIRE_ATTR;
> +} __packed;
>  
>  #define LNET_NI_STATUS_UP      0x15aac0de
>  #define LNET_NI_STATUS_DOWN    0xdeadface
> diff --git a/drivers/staging/lustre/include/uapi/linux/lnet/lnetst.h b/drivers/staging/lustre/include/uapi/linux/lnet/lnetst.h
> index a4f9ff01d458..7edba2c5bb87 100644
> --- a/drivers/staging/lustre/include/uapi/linux/lnet/lnetst.h
> +++ b/drivers/staging/lustre/include/uapi/linux/lnet/lnetst.h
> @@ -542,7 +542,7 @@ struct srpc_counters {
>  	__u32 rpcs_expired;
>  	__u64 bulk_get;
>  	__u64 bulk_put;
> -} WIRE_ATTR;
> +} __packed;
>  
>  struct sfw_counters {
>  	/** milliseconds since current session started */
> @@ -551,6 +551,6 @@ struct sfw_counters {
>  	__u32 zombie_sessions;
>  	__u32 brw_errors;
>  	__u32 ping_errors;
> -} WIRE_ATTR;
> +} __packed;
>  
>  #endif
> diff --git a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.h b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.h
> index 217503f125bc..7d8429672616 100644
> --- a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.h
> +++ b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.h
> @@ -359,45 +359,45 @@ struct kib_connparams {
>  	__u16        ibcp_queue_depth;
>  	__u16        ibcp_max_frags;
>  	__u32        ibcp_max_msg_size;
> -} WIRE_ATTR;
> +} __packed;
>  
>  struct kib_immediate_msg {
>  	struct lnet_hdr	ibim_hdr;        /* portals header */
>  	char         ibim_payload[0]; /* piggy-backed payload */
> -} WIRE_ATTR;
> +} __packed;
>  
>  struct kib_rdma_frag {
>  	__u32        rf_nob;          /* # bytes this frag */
>  	__u64        rf_addr;         /* CAVEAT EMPTOR: misaligned!! */
> -} WIRE_ATTR;
> +} __packed;
>  
>  struct kib_rdma_desc {
>  	__u32           rd_key;       /* local/remote key */
>  	__u32           rd_nfrags;    /* # fragments */
>  	struct kib_rdma_frag	rd_frags[0];	/* buffer frags */
> -} WIRE_ATTR;
> +} __packed;
>  
>  struct kib_putreq_msg {
>  	struct lnet_hdr	ibprm_hdr;    /* portals header */
>  	__u64           ibprm_cookie; /* opaque completion cookie */
> -} WIRE_ATTR;
> +} __packed;
>  
>  struct kib_putack_msg {
>  	__u64           ibpam_src_cookie; /* reflected completion cookie */
>  	__u64           ibpam_dst_cookie; /* opaque completion cookie */
>  	struct kib_rdma_desc ibpam_rd;         /* sender's sink buffer */
> -} WIRE_ATTR;
> +} __packed;
>  
>  struct kib_get_msg {
>  	struct lnet_hdr ibgm_hdr;     /* portals header */
>  	__u64           ibgm_cookie;  /* opaque completion cookie */
>  	struct kib_rdma_desc ibgm_rd;      /* rdma descriptor */
> -} WIRE_ATTR;
> +} __packed;
>  
>  struct kib_completion_msg {
>  	__u64           ibcm_cookie;  /* opaque completion cookie */
>  	__s32           ibcm_status;  /* < 0 failure: >= 0 length */
> -} WIRE_ATTR;
> +} __packed;
>  
>  struct kib_msg {
>  	/* First 2 fields fixed FOR ALL TIME */
> @@ -420,8 +420,8 @@ struct kib_msg {
>  		struct kib_putack_msg		putack;
>  		struct kib_get_msg		get;
>  		struct kib_completion_msg	completion;
> -	} WIRE_ATTR ibm_u;
> -} WIRE_ATTR;
> +	} __packed ibm_u;
> +} __packed;
>  
>  #define IBLND_MSG_MAGIC     LNET_PROTO_IB_MAGIC /* unique magic */
>  
> @@ -447,7 +447,7 @@ struct kib_rej {
>  	__u8             ibr_padding;     /* padding */
>  	__u64            ibr_incarnation; /* incarnation of peer */
>  	struct kib_connparams ibr_cp;          /* connection parameters */
> -} WIRE_ATTR;
> +} __packed;
>  
>  /* connection rejection reasons */
>  #define IBLND_REJECT_CONN_RACE      1 /* You lost connection race */
> diff --git a/drivers/staging/lustre/lnet/selftest/rpc.h b/drivers/staging/lustre/lnet/selftest/rpc.h
> index 465b5b534423..9ce336739449 100644
> --- a/drivers/staging/lustre/lnet/selftest/rpc.h
> +++ b/drivers/staging/lustre/lnet/selftest/rpc.h
> @@ -72,12 +72,12 @@ enum srpc_msg_type {
>  struct srpc_generic_reqst {
>  	__u64			rpyid;		/* reply buffer matchbits */
>  	__u64			bulkid;		/* bulk buffer matchbits */
> -} WIRE_ATTR;
> +} __packed;
>  
>  struct srpc_generic_reply {
>  	__u32			status;
>  	struct lst_sid		sid;
> -} WIRE_ATTR;
> +} __packed;
>  
>  /* FRAMEWORK RPCs */
>  struct srpc_mksn_reqst {
> @@ -85,30 +85,30 @@ struct srpc_mksn_reqst {
>  	struct lst_sid		mksn_sid;	/* session id */
>  	__u32			mksn_force;	/* use brute force */
>  	char			mksn_name[LST_NAME_SIZE];
> -} WIRE_ATTR; /* make session request */
> +} __packed; /* make session request */
>  
>  struct srpc_mksn_reply {
>  	__u32			mksn_status;	/* session status */
>  	struct lst_sid		mksn_sid;	/* session id */
>  	__u32			mksn_timeout;	/* session timeout */
>  	char			mksn_name[LST_NAME_SIZE];
> -} WIRE_ATTR; /* make session reply */
> +} __packed; /* make session reply */
>  
>  struct srpc_rmsn_reqst {
>  	__u64			rmsn_rpyid;	/* reply buffer matchbits */
>  	struct lst_sid		rmsn_sid;	/* session id */
> -} WIRE_ATTR; /* remove session request */
> +} __packed; /* remove session request */
>  
>  struct srpc_rmsn_reply {
>  	__u32			rmsn_status;
>  	struct lst_sid		rmsn_sid;	/* session id */
> -} WIRE_ATTR; /* remove session reply */
> +} __packed; /* remove session reply */
>  
>  struct srpc_join_reqst {
>  	__u64			join_rpyid;	/* reply buffer matchbits */
>  	struct lst_sid		join_sid;	/* session id to join */
>  	char			join_group[LST_NAME_SIZE]; /* group name */
> -} WIRE_ATTR;
> +} __packed;
>  
>  struct srpc_join_reply {
>  	__u32			join_status;	/* returned status */
> @@ -117,13 +117,13 @@ struct srpc_join_reply {
>  						 * expire
>  						 */
>  	char			join_session[LST_NAME_SIZE]; /* session name */
> -} WIRE_ATTR;
> +} __packed;
>  
>  struct srpc_debug_reqst {
>  	__u64			dbg_rpyid;	/* reply buffer matchbits */
>  	struct lst_sid		dbg_sid;	/* session id */
>  	__u32			dbg_flags;	/* bitmap of debug */
> -} WIRE_ATTR;
> +} __packed;
>  
>  struct srpc_debug_reply {
>  	__u32			dbg_status;	/* returned code */
> @@ -131,7 +131,7 @@ struct srpc_debug_reply {
>  	__u32			dbg_timeout;	/* session timeout */
>  	__u32			dbg_nbatch;	/* # of batches in the node */
>  	char			dbg_name[LST_NAME_SIZE]; /* session name */
> -} WIRE_ATTR;
> +} __packed;
>  
>  #define SRPC_BATCH_OPC_RUN	1
>  #define SRPC_BATCH_OPC_STOP	2
> @@ -144,20 +144,20 @@ struct srpc_batch_reqst {
>  	__u32		   bar_opc;	   /* create/start/stop batch */
>  	__u32		   bar_testidx;    /* index of test */
>  	__u32		   bar_arg;	   /* parameters */
> -} WIRE_ATTR;
> +} __packed;
>  
>  struct srpc_batch_reply {
>  	__u32		   bar_status;	   /* status of request */
>  	struct lst_sid	   bar_sid;	   /* session id */
>  	__u32		   bar_active;	   /* # of active tests in batch/test */
>  	__u32		   bar_time;	   /* remained time */
> -} WIRE_ATTR;
> +} __packed;
>  
>  struct srpc_stat_reqst {
>  	__u64		   str_rpyid;	   /* reply buffer matchbits */
>  	struct lst_sid	   str_sid;	   /* session id */
>  	__u32		   str_type;	   /* type of stat */
> -} WIRE_ATTR;
> +} __packed;
>  
>  struct srpc_stat_reply {
>  	__u32		   str_status;
> @@ -165,25 +165,25 @@ struct srpc_stat_reply {
>  	struct sfw_counters	str_fw;
>  	struct srpc_counters	str_rpc;
>  	struct lnet_counters    str_lnet;
> -} WIRE_ATTR;
> +} __packed;
>  
>  struct test_bulk_req {
>  	__u32		   blk_opc;	   /* bulk operation code */
>  	__u32		   blk_npg;	   /* # of pages */
>  	__u32		   blk_flags;	   /* reserved flags */
> -} WIRE_ATTR;
> +} __packed;
>  
>  struct test_bulk_req_v1 {
>  	__u16		   blk_opc;	   /* bulk operation code */
>  	__u16		   blk_flags;	   /* data check flags */
>  	__u32		   blk_len;	   /* data length */
>  	__u32		   blk_offset;	   /* offset */
> -} WIRE_ATTR;
> +} __packed;
>  
>  struct test_ping_req {
>  	__u32		   png_size;	   /* size of ping message */
>  	__u32		   png_flags;	   /* reserved flags */
> -} WIRE_ATTR;
> +} __packed;
>  
>  struct srpc_test_reqst {
>  	__u64			tsr_rpyid;	/* reply buffer matchbits */
> @@ -204,12 +204,12 @@ struct srpc_test_reqst {
>  		struct test_bulk_req	bulk_v0;
>  		struct test_bulk_req_v1	bulk_v1;
>  	} tsr_u;
> -} WIRE_ATTR;
> +} __packed;
>  
>  struct srpc_test_reply {
>  	__u32			tsr_status;	/* returned code */
>  	struct lst_sid		tsr_sid;
> -} WIRE_ATTR;
> +} __packed;
>  
>  /* TEST RPCs */
>  struct srpc_ping_reqst {
> @@ -218,13 +218,13 @@ struct srpc_ping_reqst {
>  	__u32		   pnr_seq;
>  	__u64		   pnr_time_sec;
>  	__u64		   pnr_time_usec;
> -} WIRE_ATTR;
> +} __packed;
>  
>  struct srpc_ping_reply {
>  	__u32		   pnr_status;
>  	__u32		   pnr_magic;
>  	__u32		   pnr_seq;
> -} WIRE_ATTR;
> +} __packed;
>  
>  struct srpc_brw_reqst {
>  	__u64		   brw_rpyid;	   /* reply buffer matchbits */
> @@ -232,11 +232,11 @@ struct srpc_brw_reqst {
>  	__u32		   brw_rw;	   /* read or write */
>  	__u32		   brw_len;	   /* bulk data len */
>  	__u32		   brw_flags;	   /* bulk data patterns */
> -} WIRE_ATTR; /* bulk r/w request */
> +} __packed; /* bulk r/w request */
>  
>  struct srpc_brw_reply {
>  	__u32		   brw_status;
> -} WIRE_ATTR; /* bulk r/w reply */
> +} __packed; /* bulk r/w reply */
>  
>  #define SRPC_MSG_MAGIC		0xeeb0f00d
>  #define SRPC_MSG_VERSION	1
> @@ -272,7 +272,7 @@ struct srpc_msg {
>  		struct srpc_brw_reqst		brw_reqst;
>  		struct srpc_brw_reply		brw_reply;
>  	}     msg_body;
> -} WIRE_ATTR;
> +} __packed;
>  
>  static inline void
>  srpc_unpack_msg_hdr(struct srpc_msg *msg)
> 
> 
> 

From jsimmons at infradead.org  Thu Jun 14 02:40:08 2018
From: jsimmons at infradead.org (James Simmons)
Date: Thu, 14 Jun 2018 03:40:08 +0100 (BST)
Subject: [lustre-devel] [PATCH 08/11] staging: lustre: obdclass: move
 linux/linux-foo.c to foo.c
In-Reply-To: <152826511916.16761.16388038372000609798.stgit@noble>
References: <152826510267.16761.14361003167157833896.stgit@noble>
 <152826511916.16761.16388038372000609798.stgit@noble>
Message-ID: <alpine.LFD.2.21.1806140339130.6734@casper.infradead.org>


> As lustre is now linux-only, having this linux sub-directory
> with files named "linux-something" is just noise.  Move them
> to a more friendly name.

Reviewed-by: James Simmons <jsimmons at infradead.org>
 
> Signed-off-by: NeilBrown <neilb at suse.com>
> ---
>  drivers/staging/lustre/lustre/obdclass/Makefile    |    2 
>  .../lustre/lustre/obdclass/linux/linux-module.c    |  514 --------------------
>  .../lustre/lustre/obdclass/linux/linux-sysctl.c    |  162 ------
>  drivers/staging/lustre/lustre/obdclass/module.c    |  514 ++++++++++++++++++++
>  drivers/staging/lustre/lustre/obdclass/sysctl.c    |  162 ++++++
>  5 files changed, 677 insertions(+), 677 deletions(-)
>  delete mode 100644 drivers/staging/lustre/lustre/obdclass/linux/linux-module.c
>  delete mode 100644 drivers/staging/lustre/lustre/obdclass/linux/linux-sysctl.c
>  create mode 100644 drivers/staging/lustre/lustre/obdclass/module.c
>  create mode 100644 drivers/staging/lustre/lustre/obdclass/sysctl.c
> 
> diff --git a/drivers/staging/lustre/lustre/obdclass/Makefile b/drivers/staging/lustre/lustre/obdclass/Makefile
> index e3fa9acff4c4..e36ba2167d10 100644
> --- a/drivers/staging/lustre/lustre/obdclass/Makefile
> +++ b/drivers/staging/lustre/lustre/obdclass/Makefile
> @@ -4,7 +4,7 @@ subdir-ccflags-y += -I$(srctree)/drivers/staging/lustre/lustre/include
>  
>  obj-$(CONFIG_LUSTRE_FS) += obdclass.o
>  
> -obdclass-y := linux/linux-module.o linux/linux-sysctl.o \
> +obdclass-y := module.o sysctl.o \
>  	      llog.o llog_cat.o llog_obd.o llog_swab.o class_obd.o debug.o \
>  	      genops.o uuid.o lprocfs_status.o lprocfs_counters.o \
>  	      lustre_handles.o lustre_peer.o statfs_pack.o linkea.o \
> diff --git a/drivers/staging/lustre/lustre/obdclass/linux/linux-module.c b/drivers/staging/lustre/lustre/obdclass/linux/linux-module.c
> deleted file mode 100644
> index 9c800580053b..000000000000
> --- a/drivers/staging/lustre/lustre/obdclass/linux/linux-module.c
> +++ /dev/null
> @@ -1,514 +0,0 @@
> -// SPDX-License-Identifier: GPL-2.0
> -/*
> - * GPL HEADER START
> - *
> - * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
> - *
> - * This program is free software; you can redistribute it and/or modify
> - * it under the terms of the GNU General Public License version 2 only,
> - * as published by the Free Software Foundation.
> - *
> - * This program is distributed in the hope that it will be useful, but
> - * WITHOUT ANY WARRANTY; without even the implied warranty of
> - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> - * General Public License version 2 for more details (a copy is included
> - * in the LICENSE file that accompanied this code).
> - *
> - * You should have received a copy of the GNU General Public License
> - * version 2 along with this program; If not, see
> - * http://www.gnu.org/licenses/gpl-2.0.html
> - *
> - * GPL HEADER END
> - */
> -/*
> - * Copyright (c) 2007, 2010, Oracle and/or its affiliates. All rights reserved.
> - * Use is subject to license terms.
> - *
> - * Copyright (c) 2011, 2012, Intel Corporation.
> - */
> -/*
> - * This file is part of Lustre, http://www.lustre.org/
> - * Lustre is a trademark of Sun Microsystems, Inc.
> - *
> - * lustre/obdclass/linux/linux-module.c
> - *
> - * Object Devices Class Driver
> - * These are the only exported functions, they provide some generic
> - * infrastructure for managing object devices
> - */
> -
> -#define DEBUG_SUBSYSTEM S_CLASS
> -
> -#include <linux/module.h>
> -#include <linux/errno.h>
> -#include <linux/kernel.h>
> -#include <linux/sched.h>
> -#include <linux/lp.h>
> -#include <linux/slab.h>
> -#include <linux/ioport.h>
> -#include <linux/fcntl.h>
> -#include <linux/delay.h>
> -#include <linux/skbuff.h>
> -#include <linux/fs.h>
> -#include <linux/poll.h>
> -#include <linux/list.h>
> -#include <linux/highmem.h>
> -#include <linux/io.h>
> -#include <asm/ioctls.h>
> -#include <linux/uaccess.h>
> -#include <linux/miscdevice.h>
> -#include <linux/seq_file.h>
> -#include <linux/kobject.h>
> -
> -#include <uapi/linux/lnet/lnetctl.h>
> -#include <obd_support.h>
> -#include <obd_class.h>
> -#include <lprocfs_status.h>
> -#include <uapi/linux/lustre/lustre_ioctl.h>
> -#include <uapi/linux/lustre/lustre_ver.h>
> -
> -#define OBD_MAX_IOCTL_BUFFER	8192
> -
> -static int obd_ioctl_is_invalid(struct obd_ioctl_data *data)
> -{
> -	if (data->ioc_len > BIT(30)) {
> -		CERROR("OBD ioctl: ioc_len larger than 1<<30\n");
> -		return 1;
> -	}
> -
> -	if (data->ioc_inllen1 > BIT(30)) {
> -		CERROR("OBD ioctl: ioc_inllen1 larger than 1<<30\n");
> -		return 1;
> -	}
> -
> -	if (data->ioc_inllen2 > BIT(30)) {
> -		CERROR("OBD ioctl: ioc_inllen2 larger than 1<<30\n");
> -		return 1;
> -	}
> -
> -	if (data->ioc_inllen3 > BIT(30)) {
> -		CERROR("OBD ioctl: ioc_inllen3 larger than 1<<30\n");
> -		return 1;
> -	}
> -
> -	if (data->ioc_inllen4 > BIT(30)) {
> -		CERROR("OBD ioctl: ioc_inllen4 larger than 1<<30\n");
> -		return 1;
> -	}
> -
> -	if (data->ioc_inlbuf1 && data->ioc_inllen1 == 0) {
> -		CERROR("OBD ioctl: inlbuf1 pointer but 0 length\n");
> -		return 1;
> -	}
> -
> -	if (data->ioc_inlbuf2 && data->ioc_inllen2 == 0) {
> -		CERROR("OBD ioctl: inlbuf2 pointer but 0 length\n");
> -		return 1;
> -	}
> -
> -	if (data->ioc_inlbuf3 && data->ioc_inllen3 == 0) {
> -		CERROR("OBD ioctl: inlbuf3 pointer but 0 length\n");
> -		return 1;
> -	}
> -
> -	if (data->ioc_inlbuf4 && data->ioc_inllen4 == 0) {
> -		CERROR("OBD ioctl: inlbuf4 pointer but 0 length\n");
> -		return 1;
> -	}
> -
> -	if (data->ioc_pbuf1 && data->ioc_plen1 == 0) {
> -		CERROR("OBD ioctl: pbuf1 pointer but 0 length\n");
> -		return 1;
> -	}
> -
> -	if (data->ioc_pbuf2 && data->ioc_plen2 == 0) {
> -		CERROR("OBD ioctl: pbuf2 pointer but 0 length\n");
> -		return 1;
> -	}
> -
> -	if (!data->ioc_pbuf1 && data->ioc_plen1 != 0) {
> -		CERROR("OBD ioctl: plen1 set but NULL pointer\n");
> -		return 1;
> -	}
> -
> -	if (!data->ioc_pbuf2 && data->ioc_plen2 != 0) {
> -		CERROR("OBD ioctl: plen2 set but NULL pointer\n");
> -		return 1;
> -	}
> -
> -	if (obd_ioctl_packlen(data) > data->ioc_len) {
> -		CERROR("OBD ioctl: packlen exceeds ioc_len (%d > %d)\n",
> -		obd_ioctl_packlen(data), data->ioc_len);
> -		return 1;
> -	}
> -
> -	return 0;
> -}
> -
> -/* buffer MUST be at least the size of obd_ioctl_hdr */
> -int obd_ioctl_getdata(char **buf, int *len, void __user *arg)
> -{
> -	struct obd_ioctl_hdr hdr;
> -	struct obd_ioctl_data *data;
> -	int err;
> -	int offset = 0;
> -
> -	if (copy_from_user(&hdr, arg, sizeof(hdr)))
> -		return -EFAULT;
> -
> -	if (hdr.ioc_version != OBD_IOCTL_VERSION) {
> -		CERROR("Version mismatch kernel (%x) vs application (%x)\n",
> -		       OBD_IOCTL_VERSION, hdr.ioc_version);
> -		return -EINVAL;
> -	}
> -
> -	if (hdr.ioc_len > OBD_MAX_IOCTL_BUFFER) {
> -		CERROR("User buffer len %d exceeds %d max buffer\n",
> -		       hdr.ioc_len, OBD_MAX_IOCTL_BUFFER);
> -		return -EINVAL;
> -	}
> -
> -	if (hdr.ioc_len < sizeof(struct obd_ioctl_data)) {
> -		CERROR("User buffer too small for ioctl (%d)\n", hdr.ioc_len);
> -		return -EINVAL;
> -	}
> -
> -	/* When there are lots of processes calling vmalloc on multi-core
> -	 * system, the high lock contention will hurt performance badly,
> -	 * obdfilter-survey is an example, which relies on ioctl. So we'd
> -	 * better avoid vmalloc on ioctl path. LU-66
> -	 */
> -	*buf = kvzalloc(hdr.ioc_len, GFP_KERNEL);
> -	if (!*buf) {
> -		CERROR("Cannot allocate control buffer of len %d\n",
> -		       hdr.ioc_len);
> -		return -EINVAL;
> -	}
> -	*len = hdr.ioc_len;
> -	data = (struct obd_ioctl_data *)*buf;
> -
> -	if (copy_from_user(*buf, arg, hdr.ioc_len)) {
> -		err = -EFAULT;
> -		goto free_buf;
> -	}
> -	if (hdr.ioc_len != data->ioc_len) {
> -		err = -EINVAL;
> -		goto free_buf;
> -	}
> -
> -	if (obd_ioctl_is_invalid(data)) {
> -		CERROR("ioctl not correctly formatted\n");
> -		err = -EINVAL;
> -		goto free_buf;
> -	}
> -
> -	if (data->ioc_inllen1) {
> -		data->ioc_inlbuf1 = &data->ioc_bulk[0];
> -		offset += cfs_size_round(data->ioc_inllen1);
> -	}
> -
> -	if (data->ioc_inllen2) {
> -		data->ioc_inlbuf2 = &data->ioc_bulk[0] + offset;
> -		offset += cfs_size_round(data->ioc_inllen2);
> -	}
> -
> -	if (data->ioc_inllen3) {
> -		data->ioc_inlbuf3 = &data->ioc_bulk[0] + offset;
> -		offset += cfs_size_round(data->ioc_inllen3);
> -	}
> -
> -	if (data->ioc_inllen4)
> -		data->ioc_inlbuf4 = &data->ioc_bulk[0] + offset;
> -
> -	return 0;
> -
> -free_buf:
> -	kvfree(*buf);
> -	return err;
> -}
> -EXPORT_SYMBOL(obd_ioctl_getdata);
> -
> -/*  opening /dev/obd */
> -static int obd_class_open(struct inode *inode, struct file *file)
> -{
> -	try_module_get(THIS_MODULE);
> -	return 0;
> -}
> -
> -/*  closing /dev/obd */
> -static int obd_class_release(struct inode *inode, struct file *file)
> -{
> -	module_put(THIS_MODULE);
> -	return 0;
> -}
> -
> -/* to control /dev/obd */
> -static long obd_class_ioctl(struct file *filp, unsigned int cmd,
> -			    unsigned long arg)
> -{
> -	int err = 0;
> -
> -	/* Allow non-root access for OBD_IOC_PING_TARGET - used by lfs check */
> -	if (!capable(CAP_SYS_ADMIN) && (cmd != OBD_IOC_PING_TARGET))
> -		return err = -EACCES;
> -	if ((cmd & 0xffffff00) == ((int)'T') << 8) /* ignore all tty ioctls */
> -		return err = -ENOTTY;
> -
> -	err = class_handle_ioctl(cmd, (unsigned long)arg);
> -
> -	return err;
> -}
> -
> -/* declare character device */
> -static const struct file_operations obd_psdev_fops = {
> -	.owner	  = THIS_MODULE,
> -	.unlocked_ioctl = obd_class_ioctl, /* unlocked_ioctl */
> -	.open	   = obd_class_open,      /* open */
> -	.release	= obd_class_release,   /* release */
> -};
> -
> -/* modules setup */
> -struct miscdevice obd_psdev = {
> -	.minor = MISC_DYNAMIC_MINOR,
> -	.name  = OBD_DEV_NAME,
> -	.fops  = &obd_psdev_fops,
> -};
> -
> -static ssize_t version_show(struct kobject *kobj, struct attribute *attr,
> -			    char *buf)
> -{
> -	return sprintf(buf, "%s\n", LUSTRE_VERSION_STRING);
> -}
> -
> -static ssize_t pinger_show(struct kobject *kobj, struct attribute *attr,
> -			   char *buf)
> -{
> -	return sprintf(buf, "%s\n", "on");
> -}
> -
> -static ssize_t
> -health_check_show(struct kobject *kobj, struct attribute *attr, char *buf)
> -{
> -	bool healthy = true;
> -	int i;
> -	size_t len = 0;
> -
> -	if (libcfs_catastrophe)
> -		return sprintf(buf, "LBUG\n");
> -
> -	read_lock(&obd_dev_lock);
> -	for (i = 0; i < class_devno_max(); i++) {
> -		struct obd_device *obd;
> -
> -		obd = class_num2obd(i);
> -		if (!obd || !obd->obd_attached || !obd->obd_set_up)
> -			continue;
> -
> -		LASSERT(obd->obd_magic == OBD_DEVICE_MAGIC);
> -		if (obd->obd_stopping)
> -			continue;
> -
> -		class_incref(obd, __func__, current);
> -		read_unlock(&obd_dev_lock);
> -
> -		if (obd_health_check(NULL, obd))
> -			healthy = false;
> -		class_decref(obd, __func__, current);
> -		read_lock(&obd_dev_lock);
> -	}
> -	read_unlock(&obd_dev_lock);
> -
> -	if (healthy)
> -		len = sprintf(buf, "healthy\n");
> -	else
> -		len = sprintf(buf, "NOT HEALTHY\n");
> -
> -	return len;
> -}
> -
> -static ssize_t jobid_var_show(struct kobject *kobj, struct attribute *attr,
> -			      char *buf)
> -{
> -	return snprintf(buf, PAGE_SIZE, "%s\n", obd_jobid_var);
> -}
> -
> -static ssize_t jobid_var_store(struct kobject *kobj, struct attribute *attr,
> -			       const char *buffer,
> -			       size_t count)
> -{
> -	if (!count || count > JOBSTATS_JOBID_VAR_MAX_LEN)
> -		return -EINVAL;
> -
> -	memset(obd_jobid_var, 0, JOBSTATS_JOBID_VAR_MAX_LEN + 1);
> -
> -	memcpy(obd_jobid_var, buffer, count);
> -
> -	/* Trim the trailing '\n' if any */
> -	if (obd_jobid_var[count - 1] == '\n')
> -		obd_jobid_var[count - 1] = 0;
> -
> -	return count;
> -}
> -
> -static ssize_t jobid_name_show(struct kobject *kobj, struct attribute *attr,
> -			       char *buf)
> -{
> -	return snprintf(buf, PAGE_SIZE, "%s\n", obd_jobid_node);
> -}
> -
> -static ssize_t jobid_name_store(struct kobject *kobj, struct attribute *attr,
> -				const char *buffer,
> -				size_t count)
> -{
> -	if (!count || count > LUSTRE_JOBID_SIZE)
> -		return -EINVAL;
> -
> -	memcpy(obd_jobid_node, buffer, count);
> -
> -	obd_jobid_node[count] = 0;
> -
> -	/* Trim the trailing '\n' if any */
> -	if (obd_jobid_node[count - 1] == '\n')
> -		obd_jobid_node[count - 1] = 0;
> -
> -	return count;
> -}
> -
> -/* Root for /sys/kernel/debug/lustre */
> -struct dentry *debugfs_lustre_root;
> -EXPORT_SYMBOL_GPL(debugfs_lustre_root);
> -
> -LUSTRE_RO_ATTR(version);
> -LUSTRE_RO_ATTR(pinger);
> -LUSTRE_RO_ATTR(health_check);
> -LUSTRE_RW_ATTR(jobid_var);
> -LUSTRE_RW_ATTR(jobid_name);
> -
> -static struct attribute *lustre_attrs[] = {
> -	&lustre_attr_version.attr,
> -	&lustre_attr_pinger.attr,
> -	&lustre_attr_health_check.attr,
> -	&lustre_attr_jobid_name.attr,
> -	&lustre_attr_jobid_var.attr,
> -	NULL,
> -};
> -
> -static void *obd_device_list_seq_start(struct seq_file *p, loff_t *pos)
> -{
> -	if (*pos >= class_devno_max())
> -		return NULL;
> -
> -	return pos;
> -}
> -
> -static void obd_device_list_seq_stop(struct seq_file *p, void *v)
> -{
> -}
> -
> -static void *obd_device_list_seq_next(struct seq_file *p, void *v, loff_t *pos)
> -{
> -	++*pos;
> -	if (*pos >= class_devno_max())
> -		return NULL;
> -
> -	return pos;
> -}
> -
> -static int obd_device_list_seq_show(struct seq_file *p, void *v)
> -{
> -	loff_t index = *(loff_t *)v;
> -	struct obd_device *obd = class_num2obd((int)index);
> -	char *status;
> -
> -	if (!obd)
> -		return 0;
> -
> -	LASSERT(obd->obd_magic == OBD_DEVICE_MAGIC);
> -	if (obd->obd_stopping)
> -		status = "ST";
> -	else if (obd->obd_inactive)
> -		status = "IN";
> -	else if (obd->obd_set_up)
> -		status = "UP";
> -	else if (obd->obd_attached)
> -		status = "AT";
> -	else
> -		status = "--";
> -
> -	seq_printf(p, "%3d %s %s %s %s %d\n",
> -		   (int)index, status, obd->obd_type->typ_name,
> -		   obd->obd_name, obd->obd_uuid.uuid,
> -		   atomic_read(&obd->obd_refcount));
> -	return 0;
> -}
> -
> -static const struct seq_operations obd_device_list_sops = {
> -	.start = obd_device_list_seq_start,
> -	.stop = obd_device_list_seq_stop,
> -	.next = obd_device_list_seq_next,
> -	.show = obd_device_list_seq_show,
> -};
> -
> -static int obd_device_list_open(struct inode *inode, struct file *file)
> -{
> -	struct seq_file *seq;
> -	int rc = seq_open(file, &obd_device_list_sops);
> -
> -	if (rc)
> -		return rc;
> -
> -	seq = file->private_data;
> -	seq->private = inode->i_private;
> -
> -	return 0;
> -}
> -
> -static const struct file_operations obd_device_list_fops = {
> -	.owner   = THIS_MODULE,
> -	.open    = obd_device_list_open,
> -	.read    = seq_read,
> -	.llseek  = seq_lseek,
> -	.release = seq_release,
> -};
> -
> -struct kobject *lustre_kobj;
> -EXPORT_SYMBOL_GPL(lustre_kobj);
> -
> -static const struct attribute_group lustre_attr_group = {
> -	.attrs = lustre_attrs,
> -};
> -
> -int class_procfs_init(void)
> -{
> -	int rc = -ENOMEM;
> -
> -	lustre_kobj = kobject_create_and_add("lustre", fs_kobj);
> -	if (!lustre_kobj)
> -		goto out;
> -
> -	/* Create the files associated with this kobject */
> -	rc = sysfs_create_group(lustre_kobj, &lustre_attr_group);
> -	if (rc) {
> -		kobject_put(lustre_kobj);
> -		goto out;
> -	}
> -
> -	debugfs_lustre_root = debugfs_create_dir("lustre", NULL);
> -
> -	debugfs_create_file("devices", 0444, debugfs_lustre_root, NULL,
> -			    &obd_device_list_fops);
> -out:
> -	return rc;
> -}
> -
> -int class_procfs_clean(void)
> -{
> -	debugfs_remove_recursive(debugfs_lustre_root);
> -
> -	debugfs_lustre_root = NULL;
> -
> -	sysfs_remove_group(lustre_kobj, &lustre_attr_group);
> -	kobject_put(lustre_kobj);
> -
> -	return 0;
> -}
> diff --git a/drivers/staging/lustre/lustre/obdclass/linux/linux-sysctl.c b/drivers/staging/lustre/lustre/obdclass/linux/linux-sysctl.c
> deleted file mode 100644
> index e5e8687784ee..000000000000
> --- a/drivers/staging/lustre/lustre/obdclass/linux/linux-sysctl.c
> +++ /dev/null
> @@ -1,162 +0,0 @@
> -// SPDX-License-Identifier: GPL-2.0
> -/*
> - * GPL HEADER START
> - *
> - * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
> - *
> - * This program is free software; you can redistribute it and/or modify
> - * it under the terms of the GNU General Public License version 2 only,
> - * as published by the Free Software Foundation.
> - *
> - * This program is distributed in the hope that it will be useful, but
> - * WITHOUT ANY WARRANTY; without even the implied warranty of
> - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> - * General Public License version 2 for more details (a copy is included
> - * in the LICENSE file that accompanied this code).
> - *
> - * You should have received a copy of the GNU General Public License
> - * version 2 along with this program; If not, see
> - * http://www.gnu.org/licenses/gpl-2.0.html
> - *
> - * GPL HEADER END
> - */
> -/*
> - * Copyright (c) 1999, 2010, Oracle and/or its affiliates. All rights reserved.
> - * Use is subject to license terms.
> - *
> - * Copyright (c) 2011, 2015, Intel Corporation.
> - */
> -/*
> - * This file is part of Lustre, http://www.lustre.org/
> - * Lustre is a trademark of Sun Microsystems, Inc.
> - */
> -
> -#include <linux/module.h>
> -#include <linux/sysctl.h>
> -#include <linux/sched.h>
> -#include <linux/mm.h>
> -#include <linux/slab.h>
> -#include <linux/stat.h>
> -#include <linux/ctype.h>
> -#include <linux/bitops.h>
> -#include <linux/uaccess.h>
> -#include <linux/utsname.h>
> -
> -#define DEBUG_SUBSYSTEM S_CLASS
> -
> -#include <obd_support.h>
> -#include <lprocfs_status.h>
> -#include <obd_class.h>
> -
> -struct static_lustre_uintvalue_attr {
> -	struct {
> -		struct attribute attr;
> -		ssize_t (*show)(struct kobject *kobj, struct attribute *attr,
> -				char *buf);
> -		ssize_t (*store)(struct kobject *kobj, struct attribute *attr,
> -				 const char *buf, size_t len);
> -	} u;
> -	int *value;
> -};
> -
> -static ssize_t static_uintvalue_show(struct kobject *kobj,
> -				     struct attribute *attr,
> -				     char *buf)
> -{
> -	struct static_lustre_uintvalue_attr *lattr = (void *)attr;
> -
> -	return sprintf(buf, "%d\n", *lattr->value);
> -}
> -
> -static ssize_t static_uintvalue_store(struct kobject *kobj,
> -				      struct attribute *attr,
> -				      const char *buffer, size_t count)
> -{
> -	struct static_lustre_uintvalue_attr *lattr  = (void *)attr;
> -	int rc;
> -	unsigned int val;
> -
> -	rc = kstrtouint(buffer, 10, &val);
> -	if (rc)
> -		return rc;
> -
> -	*lattr->value = val;
> -
> -	return count;
> -}
> -
> -#define LUSTRE_STATIC_UINT_ATTR(name, value) \
> -static struct static_lustre_uintvalue_attr lustre_sattr_##name =	\
> -					{__ATTR(name, 0644,		\
> -						static_uintvalue_show,	\
> -						static_uintvalue_store),\
> -					  value }
> -
> -LUSTRE_STATIC_UINT_ATTR(timeout, &obd_timeout);
> -
> -static ssize_t max_dirty_mb_show(struct kobject *kobj, struct attribute *attr,
> -				 char *buf)
> -{
> -	return sprintf(buf, "%lu\n",
> -		       obd_max_dirty_pages / (1 << (20 - PAGE_SHIFT)));
> -}
> -
> -static ssize_t max_dirty_mb_store(struct kobject *kobj, struct attribute *attr,
> -				  const char *buffer, size_t count)
> -{
> -	int rc;
> -	unsigned long val;
> -
> -	rc = kstrtoul(buffer, 10, &val);
> -	if (rc)
> -		return rc;
> -
> -	val *= 1 << (20 - PAGE_SHIFT); /* convert to pages */
> -
> -	if (val > ((totalram_pages / 10) * 9)) {
> -		/* Somebody wants to assign too much memory to dirty pages */
> -		return -EINVAL;
> -	}
> -
> -	if (val < 4 << (20 - PAGE_SHIFT)) {
> -		/* Less than 4 Mb for dirty cache is also bad */
> -		return -EINVAL;
> -	}
> -
> -	obd_max_dirty_pages = val;
> -
> -	return count;
> -}
> -LUSTRE_RW_ATTR(max_dirty_mb);
> -
> -LUSTRE_STATIC_UINT_ATTR(debug_peer_on_timeout, &obd_debug_peer_on_timeout);
> -LUSTRE_STATIC_UINT_ATTR(dump_on_timeout, &obd_dump_on_timeout);
> -LUSTRE_STATIC_UINT_ATTR(dump_on_eviction, &obd_dump_on_eviction);
> -LUSTRE_STATIC_UINT_ATTR(at_min, &at_min);
> -LUSTRE_STATIC_UINT_ATTR(at_max, &at_max);
> -LUSTRE_STATIC_UINT_ATTR(at_extra, &at_extra);
> -LUSTRE_STATIC_UINT_ATTR(at_early_margin, &at_early_margin);
> -LUSTRE_STATIC_UINT_ATTR(at_history, &at_history);
> -
> -static struct attribute *lustre_attrs[] = {
> -	&lustre_sattr_timeout.u.attr,
> -	&lustre_attr_max_dirty_mb.attr,
> -	&lustre_sattr_debug_peer_on_timeout.u.attr,
> -	&lustre_sattr_dump_on_timeout.u.attr,
> -	&lustre_sattr_dump_on_eviction.u.attr,
> -	&lustre_sattr_at_min.u.attr,
> -	&lustre_sattr_at_max.u.attr,
> -	&lustre_sattr_at_extra.u.attr,
> -	&lustre_sattr_at_early_margin.u.attr,
> -	&lustre_sattr_at_history.u.attr,
> -	NULL,
> -};
> -
> -static const struct attribute_group lustre_attr_group = {
> -	.attrs = lustre_attrs,
> -};
> -
> -int obd_sysctl_init(void)
> -{
> -	return sysfs_create_group(lustre_kobj, &lustre_attr_group);
> -}
> diff --git a/drivers/staging/lustre/lustre/obdclass/module.c b/drivers/staging/lustre/lustre/obdclass/module.c
> new file mode 100644
> index 000000000000..9c800580053b
> --- /dev/null
> +++ b/drivers/staging/lustre/lustre/obdclass/module.c
> @@ -0,0 +1,514 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * GPL HEADER START
> + *
> + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 only,
> + * as published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful, but
> + * WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> + * General Public License version 2 for more details (a copy is included
> + * in the LICENSE file that accompanied this code).
> + *
> + * You should have received a copy of the GNU General Public License
> + * version 2 along with this program; If not, see
> + * http://www.gnu.org/licenses/gpl-2.0.html
> + *
> + * GPL HEADER END
> + */
> +/*
> + * Copyright (c) 2007, 2010, Oracle and/or its affiliates. All rights reserved.
> + * Use is subject to license terms.
> + *
> + * Copyright (c) 2011, 2012, Intel Corporation.
> + */
> +/*
> + * This file is part of Lustre, http://www.lustre.org/
> + * Lustre is a trademark of Sun Microsystems, Inc.
> + *
> + * lustre/obdclass/linux/linux-module.c
> + *
> + * Object Devices Class Driver
> + * These are the only exported functions, they provide some generic
> + * infrastructure for managing object devices
> + */
> +
> +#define DEBUG_SUBSYSTEM S_CLASS
> +
> +#include <linux/module.h>
> +#include <linux/errno.h>
> +#include <linux/kernel.h>
> +#include <linux/sched.h>
> +#include <linux/lp.h>
> +#include <linux/slab.h>
> +#include <linux/ioport.h>
> +#include <linux/fcntl.h>
> +#include <linux/delay.h>
> +#include <linux/skbuff.h>
> +#include <linux/fs.h>
> +#include <linux/poll.h>
> +#include <linux/list.h>
> +#include <linux/highmem.h>
> +#include <linux/io.h>
> +#include <asm/ioctls.h>
> +#include <linux/uaccess.h>
> +#include <linux/miscdevice.h>
> +#include <linux/seq_file.h>
> +#include <linux/kobject.h>
> +
> +#include <uapi/linux/lnet/lnetctl.h>
> +#include <obd_support.h>
> +#include <obd_class.h>
> +#include <lprocfs_status.h>
> +#include <uapi/linux/lustre/lustre_ioctl.h>
> +#include <uapi/linux/lustre/lustre_ver.h>
> +
> +#define OBD_MAX_IOCTL_BUFFER	8192
> +
> +static int obd_ioctl_is_invalid(struct obd_ioctl_data *data)
> +{
> +	if (data->ioc_len > BIT(30)) {
> +		CERROR("OBD ioctl: ioc_len larger than 1<<30\n");
> +		return 1;
> +	}
> +
> +	if (data->ioc_inllen1 > BIT(30)) {
> +		CERROR("OBD ioctl: ioc_inllen1 larger than 1<<30\n");
> +		return 1;
> +	}
> +
> +	if (data->ioc_inllen2 > BIT(30)) {
> +		CERROR("OBD ioctl: ioc_inllen2 larger than 1<<30\n");
> +		return 1;
> +	}
> +
> +	if (data->ioc_inllen3 > BIT(30)) {
> +		CERROR("OBD ioctl: ioc_inllen3 larger than 1<<30\n");
> +		return 1;
> +	}
> +
> +	if (data->ioc_inllen4 > BIT(30)) {
> +		CERROR("OBD ioctl: ioc_inllen4 larger than 1<<30\n");
> +		return 1;
> +	}
> +
> +	if (data->ioc_inlbuf1 && data->ioc_inllen1 == 0) {
> +		CERROR("OBD ioctl: inlbuf1 pointer but 0 length\n");
> +		return 1;
> +	}
> +
> +	if (data->ioc_inlbuf2 && data->ioc_inllen2 == 0) {
> +		CERROR("OBD ioctl: inlbuf2 pointer but 0 length\n");
> +		return 1;
> +	}
> +
> +	if (data->ioc_inlbuf3 && data->ioc_inllen3 == 0) {
> +		CERROR("OBD ioctl: inlbuf3 pointer but 0 length\n");
> +		return 1;
> +	}
> +
> +	if (data->ioc_inlbuf4 && data->ioc_inllen4 == 0) {
> +		CERROR("OBD ioctl: inlbuf4 pointer but 0 length\n");
> +		return 1;
> +	}
> +
> +	if (data->ioc_pbuf1 && data->ioc_plen1 == 0) {
> +		CERROR("OBD ioctl: pbuf1 pointer but 0 length\n");
> +		return 1;
> +	}
> +
> +	if (data->ioc_pbuf2 && data->ioc_plen2 == 0) {
> +		CERROR("OBD ioctl: pbuf2 pointer but 0 length\n");
> +		return 1;
> +	}
> +
> +	if (!data->ioc_pbuf1 && data->ioc_plen1 != 0) {
> +		CERROR("OBD ioctl: plen1 set but NULL pointer\n");
> +		return 1;
> +	}
> +
> +	if (!data->ioc_pbuf2 && data->ioc_plen2 != 0) {
> +		CERROR("OBD ioctl: plen2 set but NULL pointer\n");
> +		return 1;
> +	}
> +
> +	if (obd_ioctl_packlen(data) > data->ioc_len) {
> +		CERROR("OBD ioctl: packlen exceeds ioc_len (%d > %d)\n",
> +		obd_ioctl_packlen(data), data->ioc_len);
> +		return 1;
> +	}
> +
> +	return 0;
> +}
> +
> +/* buffer MUST be at least the size of obd_ioctl_hdr */
> +int obd_ioctl_getdata(char **buf, int *len, void __user *arg)
> +{
> +	struct obd_ioctl_hdr hdr;
> +	struct obd_ioctl_data *data;
> +	int err;
> +	int offset = 0;
> +
> +	if (copy_from_user(&hdr, arg, sizeof(hdr)))
> +		return -EFAULT;
> +
> +	if (hdr.ioc_version != OBD_IOCTL_VERSION) {
> +		CERROR("Version mismatch kernel (%x) vs application (%x)\n",
> +		       OBD_IOCTL_VERSION, hdr.ioc_version);
> +		return -EINVAL;
> +	}
> +
> +	if (hdr.ioc_len > OBD_MAX_IOCTL_BUFFER) {
> +		CERROR("User buffer len %d exceeds %d max buffer\n",
> +		       hdr.ioc_len, OBD_MAX_IOCTL_BUFFER);
> +		return -EINVAL;
> +	}
> +
> +	if (hdr.ioc_len < sizeof(struct obd_ioctl_data)) {
> +		CERROR("User buffer too small for ioctl (%d)\n", hdr.ioc_len);
> +		return -EINVAL;
> +	}
> +
> +	/* When there are lots of processes calling vmalloc on multi-core
> +	 * system, the high lock contention will hurt performance badly,
> +	 * obdfilter-survey is an example, which relies on ioctl. So we'd
> +	 * better avoid vmalloc on ioctl path. LU-66
> +	 */
> +	*buf = kvzalloc(hdr.ioc_len, GFP_KERNEL);
> +	if (!*buf) {
> +		CERROR("Cannot allocate control buffer of len %d\n",
> +		       hdr.ioc_len);
> +		return -EINVAL;
> +	}
> +	*len = hdr.ioc_len;
> +	data = (struct obd_ioctl_data *)*buf;
> +
> +	if (copy_from_user(*buf, arg, hdr.ioc_len)) {
> +		err = -EFAULT;
> +		goto free_buf;
> +	}
> +	if (hdr.ioc_len != data->ioc_len) {
> +		err = -EINVAL;
> +		goto free_buf;
> +	}
> +
> +	if (obd_ioctl_is_invalid(data)) {
> +		CERROR("ioctl not correctly formatted\n");
> +		err = -EINVAL;
> +		goto free_buf;
> +	}
> +
> +	if (data->ioc_inllen1) {
> +		data->ioc_inlbuf1 = &data->ioc_bulk[0];
> +		offset += cfs_size_round(data->ioc_inllen1);
> +	}
> +
> +	if (data->ioc_inllen2) {
> +		data->ioc_inlbuf2 = &data->ioc_bulk[0] + offset;
> +		offset += cfs_size_round(data->ioc_inllen2);
> +	}
> +
> +	if (data->ioc_inllen3) {
> +		data->ioc_inlbuf3 = &data->ioc_bulk[0] + offset;
> +		offset += cfs_size_round(data->ioc_inllen3);
> +	}
> +
> +	if (data->ioc_inllen4)
> +		data->ioc_inlbuf4 = &data->ioc_bulk[0] + offset;
> +
> +	return 0;
> +
> +free_buf:
> +	kvfree(*buf);
> +	return err;
> +}
> +EXPORT_SYMBOL(obd_ioctl_getdata);
> +
> +/*  opening /dev/obd */
> +static int obd_class_open(struct inode *inode, struct file *file)
> +{
> +	try_module_get(THIS_MODULE);
> +	return 0;
> +}
> +
> +/*  closing /dev/obd */
> +static int obd_class_release(struct inode *inode, struct file *file)
> +{
> +	module_put(THIS_MODULE);
> +	return 0;
> +}
> +
> +/* to control /dev/obd */
> +static long obd_class_ioctl(struct file *filp, unsigned int cmd,
> +			    unsigned long arg)
> +{
> +	int err = 0;
> +
> +	/* Allow non-root access for OBD_IOC_PING_TARGET - used by lfs check */
> +	if (!capable(CAP_SYS_ADMIN) && (cmd != OBD_IOC_PING_TARGET))
> +		return err = -EACCES;
> +	if ((cmd & 0xffffff00) == ((int)'T') << 8) /* ignore all tty ioctls */
> +		return err = -ENOTTY;
> +
> +	err = class_handle_ioctl(cmd, (unsigned long)arg);
> +
> +	return err;
> +}
> +
> +/* declare character device */
> +static const struct file_operations obd_psdev_fops = {
> +	.owner	  = THIS_MODULE,
> +	.unlocked_ioctl = obd_class_ioctl, /* unlocked_ioctl */
> +	.open	   = obd_class_open,      /* open */
> +	.release	= obd_class_release,   /* release */
> +};
> +
> +/* modules setup */
> +struct miscdevice obd_psdev = {
> +	.minor = MISC_DYNAMIC_MINOR,
> +	.name  = OBD_DEV_NAME,
> +	.fops  = &obd_psdev_fops,
> +};
> +
> +static ssize_t version_show(struct kobject *kobj, struct attribute *attr,
> +			    char *buf)
> +{
> +	return sprintf(buf, "%s\n", LUSTRE_VERSION_STRING);
> +}
> +
> +static ssize_t pinger_show(struct kobject *kobj, struct attribute *attr,
> +			   char *buf)
> +{
> +	return sprintf(buf, "%s\n", "on");
> +}
> +
> +static ssize_t
> +health_check_show(struct kobject *kobj, struct attribute *attr, char *buf)
> +{
> +	bool healthy = true;
> +	int i;
> +	size_t len = 0;
> +
> +	if (libcfs_catastrophe)
> +		return sprintf(buf, "LBUG\n");
> +
> +	read_lock(&obd_dev_lock);
> +	for (i = 0; i < class_devno_max(); i++) {
> +		struct obd_device *obd;
> +
> +		obd = class_num2obd(i);
> +		if (!obd || !obd->obd_attached || !obd->obd_set_up)
> +			continue;
> +
> +		LASSERT(obd->obd_magic == OBD_DEVICE_MAGIC);
> +		if (obd->obd_stopping)
> +			continue;
> +
> +		class_incref(obd, __func__, current);
> +		read_unlock(&obd_dev_lock);
> +
> +		if (obd_health_check(NULL, obd))
> +			healthy = false;
> +		class_decref(obd, __func__, current);
> +		read_lock(&obd_dev_lock);
> +	}
> +	read_unlock(&obd_dev_lock);
> +
> +	if (healthy)
> +		len = sprintf(buf, "healthy\n");
> +	else
> +		len = sprintf(buf, "NOT HEALTHY\n");
> +
> +	return len;
> +}
> +
> +static ssize_t jobid_var_show(struct kobject *kobj, struct attribute *attr,
> +			      char *buf)
> +{
> +	return snprintf(buf, PAGE_SIZE, "%s\n", obd_jobid_var);
> +}
> +
> +static ssize_t jobid_var_store(struct kobject *kobj, struct attribute *attr,
> +			       const char *buffer,
> +			       size_t count)
> +{
> +	if (!count || count > JOBSTATS_JOBID_VAR_MAX_LEN)
> +		return -EINVAL;
> +
> +	memset(obd_jobid_var, 0, JOBSTATS_JOBID_VAR_MAX_LEN + 1);
> +
> +	memcpy(obd_jobid_var, buffer, count);
> +
> +	/* Trim the trailing '\n' if any */
> +	if (obd_jobid_var[count - 1] == '\n')
> +		obd_jobid_var[count - 1] = 0;
> +
> +	return count;
> +}
> +
> +static ssize_t jobid_name_show(struct kobject *kobj, struct attribute *attr,
> +			       char *buf)
> +{
> +	return snprintf(buf, PAGE_SIZE, "%s\n", obd_jobid_node);
> +}
> +
> +static ssize_t jobid_name_store(struct kobject *kobj, struct attribute *attr,
> +				const char *buffer,
> +				size_t count)
> +{
> +	if (!count || count > LUSTRE_JOBID_SIZE)
> +		return -EINVAL;
> +
> +	memcpy(obd_jobid_node, buffer, count);
> +
> +	obd_jobid_node[count] = 0;
> +
> +	/* Trim the trailing '\n' if any */
> +	if (obd_jobid_node[count - 1] == '\n')
> +		obd_jobid_node[count - 1] = 0;
> +
> +	return count;
> +}
> +
> +/* Root for /sys/kernel/debug/lustre */
> +struct dentry *debugfs_lustre_root;
> +EXPORT_SYMBOL_GPL(debugfs_lustre_root);
> +
> +LUSTRE_RO_ATTR(version);
> +LUSTRE_RO_ATTR(pinger);
> +LUSTRE_RO_ATTR(health_check);
> +LUSTRE_RW_ATTR(jobid_var);
> +LUSTRE_RW_ATTR(jobid_name);
> +
> +static struct attribute *lustre_attrs[] = {
> +	&lustre_attr_version.attr,
> +	&lustre_attr_pinger.attr,
> +	&lustre_attr_health_check.attr,
> +	&lustre_attr_jobid_name.attr,
> +	&lustre_attr_jobid_var.attr,
> +	NULL,
> +};
> +
> +static void *obd_device_list_seq_start(struct seq_file *p, loff_t *pos)
> +{
> +	if (*pos >= class_devno_max())
> +		return NULL;
> +
> +	return pos;
> +}
> +
> +static void obd_device_list_seq_stop(struct seq_file *p, void *v)
> +{
> +}
> +
> +static void *obd_device_list_seq_next(struct seq_file *p, void *v, loff_t *pos)
> +{
> +	++*pos;
> +	if (*pos >= class_devno_max())
> +		return NULL;
> +
> +	return pos;
> +}
> +
> +static int obd_device_list_seq_show(struct seq_file *p, void *v)
> +{
> +	loff_t index = *(loff_t *)v;
> +	struct obd_device *obd = class_num2obd((int)index);
> +	char *status;
> +
> +	if (!obd)
> +		return 0;
> +
> +	LASSERT(obd->obd_magic == OBD_DEVICE_MAGIC);
> +	if (obd->obd_stopping)
> +		status = "ST";
> +	else if (obd->obd_inactive)
> +		status = "IN";
> +	else if (obd->obd_set_up)
> +		status = "UP";
> +	else if (obd->obd_attached)
> +		status = "AT";
> +	else
> +		status = "--";
> +
> +	seq_printf(p, "%3d %s %s %s %s %d\n",
> +		   (int)index, status, obd->obd_type->typ_name,
> +		   obd->obd_name, obd->obd_uuid.uuid,
> +		   atomic_read(&obd->obd_refcount));
> +	return 0;
> +}
> +
> +static const struct seq_operations obd_device_list_sops = {
> +	.start = obd_device_list_seq_start,
> +	.stop = obd_device_list_seq_stop,
> +	.next = obd_device_list_seq_next,
> +	.show = obd_device_list_seq_show,
> +};
> +
> +static int obd_device_list_open(struct inode *inode, struct file *file)
> +{
> +	struct seq_file *seq;
> +	int rc = seq_open(file, &obd_device_list_sops);
> +
> +	if (rc)
> +		return rc;
> +
> +	seq = file->private_data;
> +	seq->private = inode->i_private;
> +
> +	return 0;
> +}
> +
> +static const struct file_operations obd_device_list_fops = {
> +	.owner   = THIS_MODULE,
> +	.open    = obd_device_list_open,
> +	.read    = seq_read,
> +	.llseek  = seq_lseek,
> +	.release = seq_release,
> +};
> +
> +struct kobject *lustre_kobj;
> +EXPORT_SYMBOL_GPL(lustre_kobj);
> +
> +static const struct attribute_group lustre_attr_group = {
> +	.attrs = lustre_attrs,
> +};
> +
> +int class_procfs_init(void)
> +{
> +	int rc = -ENOMEM;
> +
> +	lustre_kobj = kobject_create_and_add("lustre", fs_kobj);
> +	if (!lustre_kobj)
> +		goto out;
> +
> +	/* Create the files associated with this kobject */
> +	rc = sysfs_create_group(lustre_kobj, &lustre_attr_group);
> +	if (rc) {
> +		kobject_put(lustre_kobj);
> +		goto out;
> +	}
> +
> +	debugfs_lustre_root = debugfs_create_dir("lustre", NULL);
> +
> +	debugfs_create_file("devices", 0444, debugfs_lustre_root, NULL,
> +			    &obd_device_list_fops);
> +out:
> +	return rc;
> +}
> +
> +int class_procfs_clean(void)
> +{
> +	debugfs_remove_recursive(debugfs_lustre_root);
> +
> +	debugfs_lustre_root = NULL;
> +
> +	sysfs_remove_group(lustre_kobj, &lustre_attr_group);
> +	kobject_put(lustre_kobj);
> +
> +	return 0;
> +}
> diff --git a/drivers/staging/lustre/lustre/obdclass/sysctl.c b/drivers/staging/lustre/lustre/obdclass/sysctl.c
> new file mode 100644
> index 000000000000..e5e8687784ee
> --- /dev/null
> +++ b/drivers/staging/lustre/lustre/obdclass/sysctl.c
> @@ -0,0 +1,162 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * GPL HEADER START
> + *
> + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 only,
> + * as published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful, but
> + * WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> + * General Public License version 2 for more details (a copy is included
> + * in the LICENSE file that accompanied this code).
> + *
> + * You should have received a copy of the GNU General Public License
> + * version 2 along with this program; If not, see
> + * http://www.gnu.org/licenses/gpl-2.0.html
> + *
> + * GPL HEADER END
> + */
> +/*
> + * Copyright (c) 1999, 2010, Oracle and/or its affiliates. All rights reserved.
> + * Use is subject to license terms.
> + *
> + * Copyright (c) 2011, 2015, Intel Corporation.
> + */
> +/*
> + * This file is part of Lustre, http://www.lustre.org/
> + * Lustre is a trademark of Sun Microsystems, Inc.
> + */
> +
> +#include <linux/module.h>
> +#include <linux/sysctl.h>
> +#include <linux/sched.h>
> +#include <linux/mm.h>
> +#include <linux/slab.h>
> +#include <linux/stat.h>
> +#include <linux/ctype.h>
> +#include <linux/bitops.h>
> +#include <linux/uaccess.h>
> +#include <linux/utsname.h>
> +
> +#define DEBUG_SUBSYSTEM S_CLASS
> +
> +#include <obd_support.h>
> +#include <lprocfs_status.h>
> +#include <obd_class.h>
> +
> +struct static_lustre_uintvalue_attr {
> +	struct {
> +		struct attribute attr;
> +		ssize_t (*show)(struct kobject *kobj, struct attribute *attr,
> +				char *buf);
> +		ssize_t (*store)(struct kobject *kobj, struct attribute *attr,
> +				 const char *buf, size_t len);
> +	} u;
> +	int *value;
> +};
> +
> +static ssize_t static_uintvalue_show(struct kobject *kobj,
> +				     struct attribute *attr,
> +				     char *buf)
> +{
> +	struct static_lustre_uintvalue_attr *lattr = (void *)attr;
> +
> +	return sprintf(buf, "%d\n", *lattr->value);
> +}
> +
> +static ssize_t static_uintvalue_store(struct kobject *kobj,
> +				      struct attribute *attr,
> +				      const char *buffer, size_t count)
> +{
> +	struct static_lustre_uintvalue_attr *lattr  = (void *)attr;
> +	int rc;
> +	unsigned int val;
> +
> +	rc = kstrtouint(buffer, 10, &val);
> +	if (rc)
> +		return rc;
> +
> +	*lattr->value = val;
> +
> +	return count;
> +}
> +
> +#define LUSTRE_STATIC_UINT_ATTR(name, value) \
> +static struct static_lustre_uintvalue_attr lustre_sattr_##name =	\
> +					{__ATTR(name, 0644,		\
> +						static_uintvalue_show,	\
> +						static_uintvalue_store),\
> +					  value }
> +
> +LUSTRE_STATIC_UINT_ATTR(timeout, &obd_timeout);
> +
> +static ssize_t max_dirty_mb_show(struct kobject *kobj, struct attribute *attr,
> +				 char *buf)
> +{
> +	return sprintf(buf, "%lu\n",
> +		       obd_max_dirty_pages / (1 << (20 - PAGE_SHIFT)));
> +}
> +
> +static ssize_t max_dirty_mb_store(struct kobject *kobj, struct attribute *attr,
> +				  const char *buffer, size_t count)
> +{
> +	int rc;
> +	unsigned long val;
> +
> +	rc = kstrtoul(buffer, 10, &val);
> +	if (rc)
> +		return rc;
> +
> +	val *= 1 << (20 - PAGE_SHIFT); /* convert to pages */
> +
> +	if (val > ((totalram_pages / 10) * 9)) {
> +		/* Somebody wants to assign too much memory to dirty pages */
> +		return -EINVAL;
> +	}
> +
> +	if (val < 4 << (20 - PAGE_SHIFT)) {
> +		/* Less than 4 Mb for dirty cache is also bad */
> +		return -EINVAL;
> +	}
> +
> +	obd_max_dirty_pages = val;
> +
> +	return count;
> +}
> +LUSTRE_RW_ATTR(max_dirty_mb);
> +
> +LUSTRE_STATIC_UINT_ATTR(debug_peer_on_timeout, &obd_debug_peer_on_timeout);
> +LUSTRE_STATIC_UINT_ATTR(dump_on_timeout, &obd_dump_on_timeout);
> +LUSTRE_STATIC_UINT_ATTR(dump_on_eviction, &obd_dump_on_eviction);
> +LUSTRE_STATIC_UINT_ATTR(at_min, &at_min);
> +LUSTRE_STATIC_UINT_ATTR(at_max, &at_max);
> +LUSTRE_STATIC_UINT_ATTR(at_extra, &at_extra);
> +LUSTRE_STATIC_UINT_ATTR(at_early_margin, &at_early_margin);
> +LUSTRE_STATIC_UINT_ATTR(at_history, &at_history);
> +
> +static struct attribute *lustre_attrs[] = {
> +	&lustre_sattr_timeout.u.attr,
> +	&lustre_attr_max_dirty_mb.attr,
> +	&lustre_sattr_debug_peer_on_timeout.u.attr,
> +	&lustre_sattr_dump_on_timeout.u.attr,
> +	&lustre_sattr_dump_on_eviction.u.attr,
> +	&lustre_sattr_at_min.u.attr,
> +	&lustre_sattr_at_max.u.attr,
> +	&lustre_sattr_at_extra.u.attr,
> +	&lustre_sattr_at_early_margin.u.attr,
> +	&lustre_sattr_at_history.u.attr,
> +	NULL,
> +};
> +
> +static const struct attribute_group lustre_attr_group = {
> +	.attrs = lustre_attrs,
> +};
> +
> +int obd_sysctl_init(void)
> +{
> +	return sysfs_create_group(lustre_kobj, &lustre_attr_group);
> +}
> 
> 
> 

From neilb at suse.com  Thu Jun 14 07:29:13 2018
From: neilb at suse.com (NeilBrown)
Date: Thu, 14 Jun 2018 17:29:13 +1000
Subject: [lustre-devel] [PATCH v3] staging: lustre: Change return type
 to vm_fault_t
In-Reply-To: <CAFqt6zZJcN4=FDQSqiSznoEogavV9GBpqxx-onWG5Q3wQEM9cQ@mail.gmail.com>
References: <20180521180937.GA19605@jordon-HP-15-Notebook-PC>
 <CAFqt6zamp=+adkyM+_4z405hN-9qOn1CsY_BgVtCrAfscBHhWg@mail.gmail.com>
 <20180611205058.GA30123@kroah.com>
 <CAFqt6zZJcN4=FDQSqiSznoEogavV9GBpqxx-onWG5Q3wQEM9cQ@mail.gmail.com>
Message-ID: <87lgbhhlme.fsf@notabene.neil.brown.name>

On Tue, Jun 12 2018, Souptick Joarder wrote:

> On 12-Jun-2018 2:21 AM, "Greg KH" <gregkh at linuxfoundation.org> wrote:
>>
>> On Tue, Jun 12, 2018 at 02:00:47AM +0530, Souptick Joarder wrote:
>> > On Mon, May 21, 2018 at 11:39 PM, Souptick Joarder <jrdr.linux at gmail.com>
>> >
>> > If no further comment, we would like to get this patch in 4.18-rc-X.
>>
>> Why?  Is it a regression fix?  That's all that is allowed after -rc1.
>
> No, this is not regression fix. We need to get this into 4.18-rc-1.  But
> mostly it can't make into linus tree in rc-1 :)
>>
>> And have you tried applying it to Linus's current tree?  :)
>
> Last tested on 4.17-rc-6 and it worked fine. Let me verify in current tree.
>

As you have undoubtedly noticed, lustre is no longer in Linus' tree.
I'm experimenting with maintaining a branch which retains the code
(lustre/* in github.com/neilbrown/linux) so we can get it ready for
merging properly.
I've added you patch to my tree.

Thanks,
NeilBrown
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180614/940a8594/attachment.sig>

From neilb at suse.com  Fri Jun 15 00:00:23 2018
From: neilb at suse.com (NeilBrown)
Date: Fri, 15 Jun 2018 10:00:23 +1000
Subject: [lustre-devel] [PATCH v3] staging: lustre: Change return type
 to vm_fault_t
In-Reply-To: <CAFqt6zYn2FjX2Sy5CFmMqvwPxys=SamMy+7=14dVNCR0Kt6R7w@mail.gmail.com>
References: <20180521180937.GA19605@jordon-HP-15-Notebook-PC>
 <CAFqt6zamp=+adkyM+_4z405hN-9qOn1CsY_BgVtCrAfscBHhWg@mail.gmail.com>
 <20180611205058.GA30123@kroah.com>
 <CAFqt6zZJcN4=FDQSqiSznoEogavV9GBpqxx-onWG5Q3wQEM9cQ@mail.gmail.com>
 <87lgbhhlme.fsf@notabene.neil.brown.name>
 <CAFqt6zYn2FjX2Sy5CFmMqvwPxys=SamMy+7=14dVNCR0Kt6R7w@mail.gmail.com>
Message-ID: <87d0wshqaw.fsf@notabene.neil.brown.name>

On Thu, Jun 14 2018, Souptick Joarder wrote:

> On Thu, Jun 14, 2018 at 12:59 PM, NeilBrown <neilb at suse.com> wrote:
>> On Tue, Jun 12 2018, Souptick Joarder wrote:
>>
>>> On 12-Jun-2018 2:21 AM, "Greg KH" <gregkh at linuxfoundation.org> wrote:
>>>>
>>>> On Tue, Jun 12, 2018 at 02:00:47AM +0530, Souptick Joarder wrote:
>>>> > On Mon, May 21, 2018 at 11:39 PM, Souptick Joarder <jrdr.linux at gmail.com>
>>>> >
>>>> > If no further comment, we would like to get this patch in 4.18-rc-X.
>>>>
>>>> Why?  Is it a regression fix?  That's all that is allowed after -rc1.
>>>
>>> No, this is not regression fix. We need to get this into 4.18-rc-1.  But
>>> mostly it can't make into linus tree in rc-1 :)
>>>>
>>>> And have you tried applying it to Linus's current tree?  :)
>>>
>>> Last tested on 4.17-rc-6 and it worked fine. Let me verify in current tree.
>>>
>>
>> As you have undoubtedly noticed, lustre is no longer in Linus' tree.
>> I'm experimenting with maintaining a branch which retains the code
>> (lustre/* in github.com/neilbrown/linux) so we can get it ready for
>> merging properly.
>> I've added you patch to my tree.
>
> You need to add this patch in your tree as well.
> https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/commit/?h=next-20180614&id=1c8f422059ae5da07db7406ab916203f9417e396
>
> This patch appears to be missing in your github branch.
??
That patch is in 4.17-rc1 (v4.17-rc1~99^2~6) and my lustre trees are
based on 4.17 (at least).  My 'master' might be a bit behind - maybe
that confused you.

Thanks,
NeilBrown
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180615/de7952d1/attachment.sig>

From neilb at suse.com  Fri Jun 15 07:11:30 2018
From: neilb at suse.com (NeilBrown)
Date: Fri, 15 Jun 2018 17:11:30 +1000
Subject: [lustre-devel] [PATCH 00/24] lustre - more cleanups including
 module reduction.
Message-ID: <152904663333.10587.10934053155404014785.stgit@noble>

Here are a bunch more cleanups that will appear in my lustre-testing
tree shortly.

The interesting stuff is some enhancements to kbuild to let us
build modules from multiple directories.
I've used that to clean up the build of ptlrpc+ldlm and to
merge libcfs and lnet into the one module - I've left the lnds
separate.

I've haven't progressed further in lustre because I'm not sure which -
if any - modules are also used by the server.
I'd rather shared code was in a separate module to client-only.

Which parts of lustre/lustre/* are also used by server?

After that, it is all fairly simple cleanups.

I plan to submit the kbuild stuff upstream next week.

Thanks,
NeilBrown

---

NeilBrown (24):
      kbuild: detect directories in components of a module.
      kbuild: treat a directory listed in a composite object as foo/mod.a
      kbuild: support building of per-directory mod.a
      kbuild: disable KBUILD_MODNAME when building for mod.a
      kbuild: Add documentation for modobj-m
      lustre: build ldlm in the ldlm directory.
      lustre: merge libcfs module into lnet.
      lustre: use truncate_inode_page in place of truncate_complete_page
      lustre: don't use spare bits in iattr.ia_valid
      lustre: remove lustre_patchless_compat.h
      lustre: discard current_n*groups macros.
      lustre: discard LTIME_S macro
      lustre: discard NO_QUOTA and QUOTA_OK
      lustre: discard ext2* bit operations
      lustre: centralize TIMES_SET_FLAGS
      lustre: remove redefinition of module_init().
      lustre: remove lustre_compat.h
      lustre/lnet: move tracefile locking from linux-tracefile.c to tracefile.c
      lustre: open code cfs_trace_max_debug_mb() into cfs_trace_set_debug_mb()
      lustre: move tcd locking across to tracefile.c
      lustre: move remainder of linux-tracefile.c to tracefile.c
      lustre: fold cfs_tracefile_*_arch into their only callers.
      lustre: renamed CFS_TCD_TYPE_MAX to CFS_TCD_TYPE_CNT
      lustre: discard TCD_MAX_TYPES


 Documentation/kbuild/makefiles.txt                 |   65 +++++
 drivers/staging/lustre/lnet/Makefile               |    7 -
 drivers/staging/lustre/lnet/libcfs/Makefile        |    7 -
 drivers/staging/lustre/lnet/libcfs/debug.c         |    6 
 .../staging/lustre/lnet/libcfs/linux-tracefile.c   |  258 --------------------
 drivers/staging/lustre/lnet/libcfs/module.c        |   26 +-
 drivers/staging/lustre/lnet/libcfs/tracefile.c     |  241 +++++++++++++++++--
 drivers/staging/lustre/lnet/libcfs/tracefile.h     |   40 ---
 drivers/staging/lustre/lnet/lnet/Makefile          |   11 -
 drivers/staging/lustre/lnet/lnet/module.c          |    9 +
 drivers/staging/lustre/lustre/fid/lproc_fid.c      |    1 
 drivers/staging/lustre/lustre/fld/lproc_fld.c      |    1 
 drivers/staging/lustre/lustre/include/cl_object.h  |    6 
 .../staging/lustre/lustre/include/lustre_compat.h  |   82 ------
 drivers/staging/lustre/lustre/include/lustre_net.h |    1 
 .../lustre/include/lustre_patchless_compat.h       |   68 -----
 drivers/staging/lustre/lustre/include/obd.h        |    7 +
 .../staging/lustre/lustre/include/obd_support.h    |    1 
 drivers/staging/lustre/lustre/ldlm/Makefile        |   11 +
 drivers/staging/lustre/lustre/llite/dir.c          |    3 
 drivers/staging/lustre/lustre/llite/file.c         |   36 +--
 drivers/staging/lustre/lustre/llite/lcommon_cl.c   |   11 -
 .../staging/lustre/lustre/llite/llite_internal.h   |   13 +
 drivers/staging/lustre/lustre/llite/llite_lib.c    |   52 ++--
 drivers/staging/lustre/lustre/llite/namei.c        |   16 +
 drivers/staging/lustre/lustre/llite/vvp_io.c       |    8 -
 drivers/staging/lustre/lustre/llite/vvp_object.c   |    6 
 drivers/staging/lustre/lustre/lmv/lmv_intent.c     |    6 
 drivers/staging/lustre/lustre/lmv/lmv_obd.c        |   21 +-
 drivers/staging/lustre/lustre/lov/lov_io.c         |    3 
 drivers/staging/lustre/lustre/lov/lov_obd.c        |    2 
 drivers/staging/lustre/lustre/lov/lov_pack.c       |    1 
 drivers/staging/lustre/lustre/lov/lov_page.c       |    1 
 drivers/staging/lustre/lustre/mdc/mdc_lib.c        |   19 +
 drivers/staging/lustre/lustre/mdc/mdc_reint.c      |    4 
 drivers/staging/lustre/lustre/mdc/mdc_request.c    |   12 -
 drivers/staging/lustre/lustre/obdclass/class_obd.c |    1 
 drivers/staging/lustre/lustre/obdclass/llog.c      |    7 -
 .../lustre/lustre/obdclass/lprocfs_status.c        |    1 
 .../staging/lustre/lustre/obdclass/obd_config.c    |    1 
 drivers/staging/lustre/lustre/obdclass/obd_mount.c |    1 
 drivers/staging/lustre/lustre/obdclass/obdo.c      |   10 -
 .../staging/lustre/lustre/obdecho/echo_client.c    |    1 
 drivers/staging/lustre/lustre/osc/lproc_osc.c      |    1 
 drivers/staging/lustre/lustre/osc/osc_cache.c      |    4 
 drivers/staging/lustre/lustre/osc/osc_io.c         |   23 +-
 drivers/staging/lustre/lustre/osc/osc_quota.c      |    4 
 drivers/staging/lustre/lustre/osc/osc_request.c    |    1 
 drivers/staging/lustre/lustre/ptlrpc/Makefile      |   12 -
 drivers/staging/lustre/lustre/ptlrpc/import.c      |    1 
 .../staging/lustre/lustre/ptlrpc/lproc_ptlrpc.c    |    1 
 drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c     |    1 
 drivers/staging/lustre/lustre/ptlrpc/sec.c         |   15 +
 drivers/staging/lustre/lustre/ptlrpc/sec_plain.c   |    1 
 drivers/staging/lustre/lustre/ptlrpc/service.c     |    1 
 scripts/Makefile.build                             |   57 +++-
 scripts/Makefile.lib                               |   63 +++--
 57 files changed, 584 insertions(+), 685 deletions(-)
 delete mode 100644 drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
 delete mode 100644 drivers/staging/lustre/lustre/include/lustre_compat.h
 delete mode 100644 drivers/staging/lustre/lustre/include/lustre_patchless_compat.h
 create mode 100644 drivers/staging/lustre/lustre/ldlm/Makefile

--
Signature


From neilb at suse.com  Fri Jun 15 07:11:30 2018
From: neilb at suse.com (NeilBrown)
Date: Fri, 15 Jun 2018 17:11:30 +1000
Subject: [lustre-devel] [PATCH 01/24] kbuild: detect directories in
 components of a module.
In-Reply-To: <152904663333.10587.10934053155404014785.stgit@noble>
References: <152904663333.10587.10934053155404014785.stgit@noble>
Message-ID: <152904669010.10587.15227086408644911106.stgit@noble>

This is a first step in a larger change and so can
only be fully understood in the larger context.

This patch changes the code for extracting directories
from a list of objects to extract them from real-obj-X
instead of obj-X.
This should not cause any change in behaviour yet
as listing directories as components of an object is
not currently supported and will cause an error.
A future patch will give a useful meaning to directories
listed in componsite objects.

A consequence of this change is that any subsequent use of obj-y or
obj-m will still have directories listed in it.
There are no subsequent uses of obj-y and only 2 of obj-m.

1/ obj-m is included as a dependency of __build.  subdir-ym, which
  contains all the directories mentioned in obj-m, is also a
  dependency, so this won't change the set of final dependencies.

2/ Any rule that builds a directory listed in obj-m will find that
   quite_modtag has the value '[M]'.  As quiet_modtag is not used
   when descending into directories, this is of no consequence.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 scripts/Makefile.lib |   38 +++++++++++++++++++-------------------
 1 file changed, 19 insertions(+), 19 deletions(-)

diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
index 5af34a2b0cd9..2f37262b9333 100644
--- a/scripts/Makefile.lib
+++ b/scripts/Makefile.lib
@@ -25,36 +25,36 @@ lib-y := $(filter-out $(obj-y), $(sort $(lib-y) $(lib-m)))
 # and -m subdirs.  Just put -y's first.
 modorder	:= $(patsubst %/,%/modules.order, $(filter %/, $(obj-y)) $(obj-m:.o=.ko))
 
+# if $(foo-objs), $(foo-y), or $(foo-m) exists, foo.o is a composite object
+multi-used-y := $(sort $(foreach m,$(obj-y), $(if $(strip $($(m:.o=-objs)) $($(m:.o=-y))), $(m))))
+multi-used-m := $(sort $(foreach m,$(obj-m), $(if $(strip $($(m:.o=-objs)) $($(m:.o=-y)) $($(m:.o=-m))), $(m))))
+multi-used   := $(multi-used-y) $(multi-used-m)
+single-used-m := $(sort $(filter-out $(multi-used-m),$(obj-m)))
+
+# Replace multi-part objects by their individual parts,
+# including built-in.a from subdirectories
+real-obj-y := $(foreach m, $(obj-y), $(if $(strip $($(m:.o=-objs)) $($(m:.o=-y))),$($(m:.o=-objs)) $($(m:.o=-y)),$(m)))
+real-obj-m := $(foreach m, $(obj-m), $(if $(strip $($(m:.o=-objs)) $($(m:.o=-y)) $($(m:.o=-m))),$($(m:.o=-objs)) $($(m:.o=-y)) $($(m:.o=-m)),$(m)))
+
 # Handle objects in subdirs
 # ---------------------------------------------------------------------------
-# o if we encounter foo/ in $(obj-y), replace it by foo/built-in.a
+# o if we encounter foo/ in $(real-obj-y), replace it by foo/built-in.a
 #   and add the directory to the list of dirs to descend into: $(subdir-y)
-# o if we encounter foo/ in $(obj-m), remove it from $(obj-m)
+# o if we encounter foo/ in $(real-obj-m), remove it from $(real-obj-m)
 #   and add the directory to the list of dirs to descend into: $(subdir-m)
-__subdir-y	:= $(patsubst %/,%,$(filter %/, $(obj-y)))
+__subdir-y	:= $(patsubst %/,%,$(filter %/, $(real-obj-y)))
 subdir-y	+= $(__subdir-y)
-__subdir-m	:= $(patsubst %/,%,$(filter %/, $(obj-m)))
+__subdir-m	:= $(patsubst %/,%,$(filter %/, $(real-obj-m)))
 subdir-m	+= $(__subdir-m)
-obj-y		:= $(patsubst %/, %/built-in.a, $(obj-y))
-obj-m		:= $(filter-out %/, $(obj-m))
+real-obj-y	:= $(patsubst %/, %/built-in.a, $(real-obj-y))
+real-obj-m	:= $(filter-out %/, $(real-obj-m))
 
 # Subdirectories we need to descend into
 subdir-ym	:= $(sort $(subdir-y) $(subdir-m))
 
-# if $(foo-objs), $(foo-y), or $(foo-m) exists, foo.o is a composite object
-multi-used-y := $(sort $(foreach m,$(obj-y), $(if $(strip $($(m:.o=-objs)) $($(m:.o=-y))), $(m))))
-multi-used-m := $(sort $(foreach m,$(obj-m), $(if $(strip $($(m:.o=-objs)) $($(m:.o=-y)) $($(m:.o=-m))), $(m))))
-multi-used   := $(multi-used-y) $(multi-used-m)
-single-used-m := $(sort $(filter-out $(multi-used-m),$(obj-m)))
-
-# $(subdir-obj-y) is the list of objects in $(obj-y) which uses dir/ to
+# $(subdir-obj-y) is the list of objects in $(real-obj-y) which uses dir/ to
 # tell kbuild to descend
-subdir-obj-y := $(filter %/built-in.a, $(obj-y))
-
-# Replace multi-part objects by their individual parts,
-# including built-in.a from subdirectories
-real-obj-y := $(foreach m, $(obj-y), $(if $(strip $($(m:.o=-objs)) $($(m:.o=-y))),$($(m:.o=-objs)) $($(m:.o=-y)),$(m)))
-real-obj-m := $(foreach m, $(obj-m), $(if $(strip $($(m:.o=-objs)) $($(m:.o=-y)) $($(m:.o=-m))),$($(m:.o=-objs)) $($(m:.o=-y)) $($(m:.o=-m)),$(m)))
+subdir-obj-y := $(filter %/built-in.a, $(real-obj-y))
 
 # DTB
 # If CONFIG_OF_ALL_DTBS is enabled, all DT blobs are built



From neilb at suse.com  Fri Jun 15 07:11:30 2018
From: neilb at suse.com (NeilBrown)
Date: Fri, 15 Jun 2018 17:11:30 +1000
Subject: [lustre-devel] [PATCH 02/24] kbuild: treat a directory listed in a
 composite object as foo/mod.a
In-Reply-To: <152904663333.10587.10934053155404014785.stgit@noble>
References: <152904663333.10587.10934053155404014785.stgit@noble>
Message-ID: <152904669013.10587.6424693677513069548.stgit@noble>

Currently a directory is not permitted in a composite
object.
This patch changes kbuild so that a directory it assumed
to mean the file "mod.a" in that directory.
The file cannot, yet, be created, so this does not yet
affect behaviour.

There are several parts to this.

1/ strip out all the directories from obj-m so that
  the directories that appear in real-obj-m must be
  parts of composite objects
2/ translate those directories from foo/ to foo/mod.a
  at the same time that obj-y directories becomes foo/built-in.a
3/ hold list of directories needed for modules in subdir-obj-m
  so that we can descend into them as required.
4/ We need a little "dance" in "Rule to link composite objects"
  where we strip the mod.a back off - so we can filter against
  the foo-{objs,y,m} macros, then add it back on for
  declaring dependencies.
  As part of this, multi_depend gains an extra argument
  being the name to append to any directory.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 scripts/Makefile.build |   15 ++++++++-------
 scripts/Makefile.lib   |   19 +++++++++++++++----
 2 files changed, 23 insertions(+), 11 deletions(-)

diff --git a/scripts/Makefile.build b/scripts/Makefile.build
index 8bdb1dc4072c..d42091532021 100644
--- a/scripts/Makefile.build
+++ b/scripts/Makefile.build
@@ -412,8 +412,9 @@ endif
 $(obj)/%.o: $(src)/%.S $(objtool_dep) FORCE
 	$(call if_changed_rule,as_o_S)
 
-targets += $(filter-out $(subdir-obj-y), $(real-obj-y)) $(real-obj-m) $(lib-y)
-targets += $(extra-y) $(MAKECMDGOALS) $(always)
+targets += $(filter-out $(subdir-obj-y), $(real-obj-y))
+targets += $(filter-out $(subdir-obj-m), $(real-obj-m))
+targets += $(lib-y) $(extra-y) $(MAKECMDGOALS) $(always)
 
 # Linker scripts preprocessor (.lds.S -> .lds)
 # ---------------------------------------------------------------------------
@@ -437,7 +438,7 @@ $(obj)/%.asn1.c $(obj)/%.asn1.h: $(src)/%.asn1 $(objtree)/scripts/asn1_compiler
 # ---------------------------------------------------------------------------
 
 # To build objects in subdirs, we need to descend into the directories
-$(sort $(subdir-obj-y)): $(subdir-ym) ;
+$(sort $(subdir-obj-y) $(subdir-obj-m)): $(subdir-ym) ;
 
 #
 # Rule to compile a set of .o files into one .o file
@@ -522,16 +523,16 @@ link_multi_deps =                     \
 $(filter $(addprefix $(obj)/,         \
 $($(subst $(obj)/,,$(@:.o=-objs)))    \
 $($(subst $(obj)/,,$(@:.o=-y)))       \
-$($(subst $(obj)/,,$(@:.o=-m)))), $^)
+$($(subst $(obj)/,,$(@:.o=-m)))), $(patsubst %/mod.a,%/,$^))
 
 quiet_cmd_link_multi-m = LD [M]  $@
-cmd_link_multi-m = $(LD) $(ld_flags) -r -o $@ $(link_multi_deps) $(cmd_secanalysis)
+cmd_link_multi-m = $(LD) $(ld_flags) -r -o $@ $(patsubst %/, --whole-archive %/mod.a --no-whole-archive ,$(link_multi_deps)) $(cmd_secanalysis)
 
 $(multi-used-m): FORCE
 	$(call if_changed,link_multi-m)
-	@{ echo $(@:.o=.ko); echo $(link_multi_deps); \
+	@{ echo $(@:.o=.ko); echo $(patsubst %/,%/mod.a,$(link_multi_deps)); \
 	   $(cmd_undef_syms); } > $(MODVERDIR)/$(@F:.o=.mod)
-$(call multi_depend, $(multi-used-m), .o, -objs -y -m)
+$(call multi_depend, $(multi-used-m), .o, -objs -y -m,mod.a)
 
 targets += $(multi-used-m)
 targets := $(filter-out $(PHONY), $(targets))
diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
index 2f37262b9333..c8f643144cd6 100644
--- a/scripts/Makefile.lib
+++ b/scripts/Makefile.lib
@@ -12,6 +12,13 @@ KBUILD_CFLAGS += $(subdir-ccflags-y)
 # Figure out what we need to build from the various variables
 # ===========================================================================
 
+# Directories in obj-m only cause that directory to be descended, which
+# is exactly what happens for directories in obj-y. So move all
+# directories from obj-m to obj-y.  Then we will know that any directory
+# in real-obj-m is a component of some other object.
+obj-y := $(obj-y) $(filter %/, $(obj-m))
+obj-m := $(filter-out %/, $(obj-m))
+
 # When an object is listed to be built compiled-in and modular,
 # only build the compiled-in version
 obj-m := $(filter-out $(obj-y),$(obj-m))
@@ -40,14 +47,14 @@ real-obj-m := $(foreach m, $(obj-m), $(if $(strip $($(m:.o=-objs)) $($(m:.o=-y))
 # ---------------------------------------------------------------------------
 # o if we encounter foo/ in $(real-obj-y), replace it by foo/built-in.a
 #   and add the directory to the list of dirs to descend into: $(subdir-y)
-# o if we encounter foo/ in $(real-obj-m), remove it from $(real-obj-m)
+# o if we encounter foo/ in $(real-obj-m), replace it by foo/mod.a
 #   and add the directory to the list of dirs to descend into: $(subdir-m)
 __subdir-y	:= $(patsubst %/,%,$(filter %/, $(real-obj-y)))
 subdir-y	+= $(__subdir-y)
 __subdir-m	:= $(patsubst %/,%,$(filter %/, $(real-obj-m)))
 subdir-m	+= $(__subdir-m)
 real-obj-y	:= $(patsubst %/, %/built-in.a, $(real-obj-y))
-real-obj-m	:= $(filter-out %/, $(real-obj-m))
+real-obj-m	:= $(patsubst %/, %/mod.a, $(real-obj-m))
 
 # Subdirectories we need to descend into
 subdir-ym	:= $(sort $(subdir-y) $(subdir-m))
@@ -55,6 +62,9 @@ subdir-ym	:= $(sort $(subdir-y) $(subdir-m))
 # $(subdir-obj-y) is the list of objects in $(real-obj-y) which uses dir/ to
 # tell kbuild to descend
 subdir-obj-y := $(filter %/built-in.a, $(real-obj-y))
+# $(subdir-obj-m) is the list of objects in $(real-obj-m) which uses dir/ to
+# tell kbuild to descend
+subdir-obj-m := $(filter %/mod.a, $(real-obj-m))
 
 # DTB
 # If CONFIG_OF_ALL_DTBS is enabled, all DT blobs are built
@@ -70,6 +80,7 @@ modorder	:= $(addprefix $(obj)/,$(modorder))
 obj-m		:= $(addprefix $(obj)/,$(obj-m))
 lib-y		:= $(addprefix $(obj)/,$(lib-y))
 subdir-obj-y	:= $(addprefix $(obj)/,$(subdir-obj-y))
+subdir-obj-m	:= $(addprefix $(obj)/,$(subdir-obj-m))
 real-obj-y	:= $(addprefix $(obj)/,$(real-obj-y))
 real-obj-m	:= $(addprefix $(obj)/,$(real-obj-m))
 single-used-m	:= $(addprefix $(obj)/,$(single-used-m))
@@ -172,11 +183,11 @@ dtc_cpp_flags  = -Wp,-MD,$(depfile).pre.tmp -nostdinc                    \
 
 # Useful for describing the dependency of composite objects
 # Usage:
-#   $(call multi_depend, multi_used_targets, suffix_to_remove, suffix_to_add)
+#   $(call multi_depend, multi_used_targets, suffix_to_remove, suffix_to_add, object_in_directories)
 define multi_depend
 $(foreach m, $(notdir $1), \
 	$(eval $(obj)/$m: \
-	$(addprefix $(obj)/, $(foreach s, $3, $($(m:%$(strip $2)=%$(s)))))))
+	$(addprefix $(obj)/, $(patsubst %/,%/$4,$(foreach s, $3, $($(m:%$(strip $2)=%$(s))))))))
 endef
 
 # LEX



From neilb at suse.com  Fri Jun 15 07:11:30 2018
From: neilb at suse.com (NeilBrown)
Date: Fri, 15 Jun 2018 17:11:30 +1000
Subject: [lustre-devel] [PATCH 03/24] kbuild: support building of
 per-directory mod.a
In-Reply-To: <152904663333.10587.10934053155404014785.stgit@noble>
References: <152904663333.10587.10934053155404014785.stgit@noble>
Message-ID: <152904669016.10587.15454151232253137450.stgit@noble>

This patch allows a "mod.a" to be built in any
directory.  A previous patch allows that mod.a
to be included in any module or another mod.a.

This is achieved via a new pair of macros: modobj-y and modobj-m.

Anything in modobj-y is added to obj-y and is otherwise
ignored.
Anything listed in modobj-m is built, almost as though "modobj.o" was
a requested target.  The objects are then combined
into mod.a.
These objects are always built with part-of-module=y.

This is sufficient to build a module from source in
multiple directories.  Each "other" directory lists
something like
 modobj-$(CONFIG_FOO) += bar.o bar.o bat.o

and the main directory lists

 obj-$(CONFIG-FOO) = foo.o
 foo-y = other1/ ../friend/other2/ module.o

Signed-off-by: NeilBrown <neilb at suse.com>
---
 scripts/Makefile.build |   44 +++++++++++++++++++++++++++++++++-----------
 scripts/Makefile.lib   |   14 ++++++++++----
 2 files changed, 43 insertions(+), 15 deletions(-)

diff --git a/scripts/Makefile.build b/scripts/Makefile.build
index d42091532021..703bc210f71f 100644
--- a/scripts/Makefile.build
+++ b/scripts/Makefile.build
@@ -15,6 +15,8 @@ obj-y :=
 obj-m :=
 lib-y :=
 lib-m :=
+modobj-y :=
+modobj-m :=
 always :=
 targets :=
 subdir-y :=
@@ -80,12 +82,16 @@ ifneq ($(strip $(real-obj-y) $(need-builtin)),)
 builtin-target := $(obj)/built-in.a
 endif
 
+ifneq ($(strip $(modobj-m)),)
+modobj-target := $(obj)/mod.a
+endif
+
 modorder-target := $(obj)/modules.order
 
 # We keep a list of all modules in $(MODVERDIR)
 
 __build: $(if $(KBUILD_BUILTIN),$(builtin-target) $(lib-target) $(extra-y)) \
-	 $(if $(KBUILD_MODULES),$(obj-m) $(modorder-target)) \
+	 $(if $(KBUILD_MODULES),$(obj-m) $(modorder-target) $(modobj-target))  \
 	 $(subdir-ym) $(always)
 	@:
 
@@ -119,17 +125,19 @@ modkern_cflags =                                          \
 		$(KBUILD_CFLAGS_KERNEL) $(CFLAGS_KERNEL))
 quiet_modtag := $(empty)   $(empty)
 
-$(real-obj-m)        : part-of-module := y
-$(real-obj-m:.o=.i)  : part-of-module := y
-$(real-obj-m:.o=.s)  : part-of-module := y
-$(real-obj-m:.o=.lst): part-of-module := y
+_mod_obj = $(real-obj-m) $(real-modobj-m)
+
+$(_mod_obj)        : part-of-module := y
+$(_mod_obj:.o=.i)  : part-of-module := y
+$(_mod_obj:.o=.s)  : part-of-module := y
+$(_mod_obj:.o=.lst): part-of-module := y
 
-$(real-obj-m)        : quiet_modtag := [M]
-$(real-obj-m:.o=.i)  : quiet_modtag := [M]
-$(real-obj-m:.o=.s)  : quiet_modtag := [M]
-$(real-obj-m:.o=.lst): quiet_modtag := [M]
+$(_mod_obj)        : quiet_modtag := [M]
+$(_mod_obj:.o=.i)  : quiet_modtag := [M]
+$(_mod_obj:.o=.s)  : quiet_modtag := [M]
+$(_mod_obj:.o=.lst): quiet_modtag := [M]
 
-$(obj-m)             : quiet_modtag := [M]
+$(obj-m)           : quiet_modtag := [M]
 
 quiet_cmd_cc_s_c = CC $(quiet_modtag)  $@
 cmd_cc_s_c       = $(CC) $(c_flags) $(DISABLE_LTO) -fverbose-asm -S -o $@ $<
@@ -413,7 +421,7 @@ $(obj)/%.o: $(src)/%.S $(objtool_dep) FORCE
 	$(call if_changed_rule,as_o_S)
 
 targets += $(filter-out $(subdir-obj-y), $(real-obj-y))
-targets += $(filter-out $(subdir-obj-m), $(real-obj-m))
+targets += $(filter-out $(subdir-obj-m) $(real-modobj-m), $(real-obj-m))
 targets += $(lib-y) $(extra-y) $(MAKECMDGOALS) $(always)
 
 # Linker scripts preprocessor (.lds.S -> .lds)
@@ -509,6 +517,20 @@ targets += $(obj)/lib-ksyms.o
 
 endif
 
+ifdef modobj-target
+
+quiet_cmd_ar_modobj = AR      $@
+      cmd_ar_modobj = rm -f $@; \
+                    $(AR) rcTP$(KBUILD_ARFLAGS) $@ $(filter $(real-modobj-m), $^)
+
+$(modobj-target): $(real-modobj-m) FORCE
+	$(call if_changed,ar_modobj)
+
+targets += $(modobj-target)
+
+endif # modobj-target
+
+
 #
 # Rule to link composite objects
 #
diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
index c8f643144cd6..90ee6e8c910f 100644
--- a/scripts/Makefile.lib
+++ b/scripts/Makefile.lib
@@ -16,12 +16,13 @@ KBUILD_CFLAGS += $(subdir-ccflags-y)
 # is exactly what happens for directories in obj-y. So move all
 # directories from obj-m to obj-y.  Then we will know that any directory
 # in real-obj-m is a component of some other object.
-obj-y := $(obj-y) $(filter %/, $(obj-m))
+obj-y := $(obj-y) $(filter %/, $(obj-m)) $(filter-out $(obj-y), $(modobj-y))
 obj-m := $(filter-out %/, $(obj-m))
 
 # When an object is listed to be built compiled-in and modular,
 # only build the compiled-in version
 obj-m := $(filter-out $(obj-y),$(obj-m))
+modobj-m := $(filter-out $(obj-y), $(modobj-m))
 
 # Libraries are always collected in one lib file.
 # Filter out objects already built-in
@@ -42,6 +43,7 @@ single-used-m := $(sort $(filter-out $(multi-used-m),$(obj-m)))
 # including built-in.a from subdirectories
 real-obj-y := $(foreach m, $(obj-y), $(if $(strip $($(m:.o=-objs)) $($(m:.o=-y))),$($(m:.o=-objs)) $($(m:.o=-y)),$(m)))
 real-obj-m := $(foreach m, $(obj-m), $(if $(strip $($(m:.o=-objs)) $($(m:.o=-y)) $($(m:.o=-m))),$($(m:.o=-objs)) $($(m:.o=-y)) $($(m:.o=-m)),$(m)))
+real-modobj-m := $(foreach m, $(modobj-m), $(if $(strip $($(m:.o=-objs)) $($(m:.o=-y)) $($(m:.o=-m))),$($(m:.o=-objs)) $($(m:.o=-y)) $($(m:.o=-m)),$(m)))
 
 # Handle objects in subdirs
 # ---------------------------------------------------------------------------
@@ -53,8 +55,11 @@ __subdir-y	:= $(patsubst %/,%,$(filter %/, $(real-obj-y)))
 subdir-y	+= $(__subdir-y)
 __subdir-m	:= $(patsubst %/,%,$(filter %/, $(real-obj-m)))
 subdir-m	+= $(__subdir-m)
+__subdir-mo	:= $(patsubst %/,%,$(filter %/, $(real-modobj-m)))
+subdir-m	+= $(__subdir-mo)
 real-obj-y	:= $(patsubst %/, %/built-in.a, $(real-obj-y))
 real-obj-m	:= $(patsubst %/, %/mod.a, $(real-obj-m))
+real-modobj-m	:= $(patsubst %/, %/mod.a, $(real-modobj-m))
 
 # Subdirectories we need to descend into
 subdir-ym	:= $(sort $(subdir-y) $(subdir-m))
@@ -62,9 +67,9 @@ subdir-ym	:= $(sort $(subdir-y) $(subdir-m))
 # $(subdir-obj-y) is the list of objects in $(real-obj-y) which uses dir/ to
 # tell kbuild to descend
 subdir-obj-y := $(filter %/built-in.a, $(real-obj-y))
-# $(subdir-obj-m) is the list of objects in $(real-obj-m) which uses dir/ to
-# tell kbuild to descend
-subdir-obj-m := $(filter %/mod.a, $(real-obj-m))
+# $(subdir-obj-m) is the list of objects in $(real-obj-m) and
+# $(real-modobj-m) which use dir/ to tell kbuild to descend
+subdir-obj-m := $(filter %/mod.a, $(real-obj-m) $(real-modobj-m))
 
 # DTB
 # If CONFIG_OF_ALL_DTBS is enabled, all DT blobs are built
@@ -83,6 +88,7 @@ subdir-obj-y	:= $(addprefix $(obj)/,$(subdir-obj-y))
 subdir-obj-m	:= $(addprefix $(obj)/,$(subdir-obj-m))
 real-obj-y	:= $(addprefix $(obj)/,$(real-obj-y))
 real-obj-m	:= $(addprefix $(obj)/,$(real-obj-m))
+real-modobj-m	:= $(addprefix $(obj)/,$(real-modobj-m))
 single-used-m	:= $(addprefix $(obj)/,$(single-used-m))
 multi-used-m	:= $(addprefix $(obj)/,$(multi-used-m))
 subdir-ym	:= $(addprefix $(obj)/,$(subdir-ym))



From neilb at suse.com  Fri Jun 15 07:11:30 2018
From: neilb at suse.com (NeilBrown)
Date: Fri, 15 Jun 2018 17:11:30 +1000
Subject: [lustre-devel] [PATCH 04/24] kbuild: disable KBUILD_MODNAME when
 building for mod.a
In-Reply-To: <152904663333.10587.10934053155404014785.stgit@noble>
References: <152904663333.10587.10934053155404014785.stgit@noble>
Message-ID: <152904669020.10587.8392749843886680541.stgit@noble>

When building an object to be included in mod.a we
cannot know the name of the module.  So don't define
KBUILD_MODNAME.  This will ensure attempt to use
that macro when the module name isn't know will
trigger an error.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 scripts/Makefile.lib |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
index 90ee6e8c910f..3b2edf767efa 100644
--- a/scripts/Makefile.lib
+++ b/scripts/Makefile.lib
@@ -96,7 +96,7 @@ subdir-ym	:= $(addprefix $(obj)/,$(subdir-ym))
 # Finds the multi-part object the current object will be linked into.
 # If the object belongs to two or more multi-part objects, all of them are
 # concatenated with a colon separator.
-modname-multi = $(subst $(space),:,$(sort $(foreach m,$(multi-used),\
+modname-multi = $(subst $(space),:,$(sort $(foreach m,$(multi-used) modobj.o,\
 		$(if $(filter $*.o, $($(m:.o=-objs)) $($(m:.o=-y)) $($(m:.o=-m))),$(m:.o=)))))
 
 modname = $(if $(modname-multi),$(modname-multi),$(basetarget))
@@ -106,7 +106,7 @@ modname = $(if $(modname-multi),$(modname-multi),$(basetarget))
 # end up in (or would, if it gets compiled in)
 name-fix = $(squote)$(quote)$(subst $(comma),_,$(subst -,_,$1))$(quote)$(squote)
 basename_flags = -DKBUILD_BASENAME=$(call name-fix,$(basetarget))
-modname_flags  = -DKBUILD_MODNAME=$(call name-fix,$(modname))
+modname_flags  = $(if $(filter-out modobj,$(modname)),-DKBUILD_MODNAME=$(call name-fix,$(modname)))
 
 orig_c_flags   = $(KBUILD_CPPFLAGS) $(KBUILD_CFLAGS) \
                  $(ccflags-y) $(CFLAGS_$(basetarget).o)



From neilb at suse.com  Fri Jun 15 07:11:30 2018
From: neilb at suse.com (NeilBrown)
Date: Fri, 15 Jun 2018 17:11:30 +1000
Subject: [lustre-devel] [PATCH 05/24] kbuild: Add documentation for modobj-m
In-Reply-To: <152904663333.10587.10934053155404014785.stgit@noble>
References: <152904663333.10587.10934053155404014785.stgit@noble>
Message-ID: <152904669023.10587.4124629758082370663.stgit@noble>

Add documentation for building modules
from multiple directories using modobj-m.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 Documentation/kbuild/makefiles.txt |   65 ++++++++++++++++++++++++++++++++++--
 1 file changed, 61 insertions(+), 4 deletions(-)

diff --git a/Documentation/kbuild/makefiles.txt b/Documentation/kbuild/makefiles.txt
index 048fc39a6b91..985a60cf0663 100644
--- a/Documentation/kbuild/makefiles.txt
+++ b/Documentation/kbuild/makefiles.txt
@@ -225,17 +225,74 @@ more details, with real examples.
 	part of the composite object ext2.o if $(CONFIG_EXT2_FS_XATTR)
 	evaluates to 'y'.
 
+	If you wish to include code in some other directory into a
+	module, that directory should be configured to create a
+	mod.a target from the relevant code, as described below.
+	The list of components for the module (e.g. foo-y) should
+	then contain the name of the directory (with trailing /).
+	The mod.a from that directory will be built and included
+	into this module.
+
 	Note: Of course, when you are building objects into the kernel,
 	the syntax above will also work. So, if you have CONFIG_EXT2_FS=y,
 	kbuild will build an ext2.o file for you out of the individual
 	parts and then link this into built-in.a, as you would expect.
 
---- 3.4 Objects which export symbols
+--- 3.4 Loadable modules from multiple directories - modobj-m
+
+	One way to build modules with source from multiple directories
+	is to list relative path names of all source files in a
+	single Makefile.
+
+	Example:
+		btrfs-$(CONFIG_BTRFS_FS_RUN_SANITY_TESTS) += tests/free-space-tests.o \
+		        tests/extent-buffer-tests.o tests/btrfs-tests.o \
+		....
+
+	This works, but defeats the ability to build individual
+	directories, or individual target files such as assembly
+	language targets.
+
+	Example:
+		make btrfs/tests/free-space-tests.s
+
+	A more general way is to create a mod.a file using modobj-m
+
+	Example:
+		modobj-$(CONFIG_LUSTRE_FS) += fid_request.o fid_lib.o lproc_fid.o
+
+	modobj-y is treated identically to obj-y so if CONFIG_LUSTRE_FS=y
+	then these objects are built in to the kernel.  If
+	CONFIG_LUSTRE_FS=m then these objects are gathered into an archive
+	called mod.a.  Archives from other directories can be included
+	by listing the directories in modobj-m, so
+
+	Example:
+		modobj-$(CONFIG_LUSTRE_FS) += fid/ fld/
+
+	This will create a mod.a in the same directory as the
+	Makefile, combining the contents of mod.a from the listed
+	directories.  The mod.a can instead be included into a module
+	by listing the directory with any other local object files.
+
+	Example:
+		obj-$(CONFIG_LUSTRE_FS) += lustre.o
+		lustre-y := module.o
+		lustre-y += obdclass/ ldlm/ ptlrpc/ fld/ osc/ mgc/
+
+	When CONFIG_LUSTRE_FS=m, kbuild will build lustre.ko from module.o and
+	the mod.a archives created in the listed directories.
+
+	When components of a module are built in a different directory
+	kbuild cannot determine the name of the module so the
+	KBUILD_MODNAME macro is left undefined.
+
+--- 3.5 Objects which export symbols
 
 	No special notation is required in the makefiles for
 	modules exporting symbols.
 
---- 3.5 Library file goals - lib-y
+--- 3.6 Library file goals - lib-y
 
 	Objects listed with obj-* are used for modules, or
 	combined in a built-in.a for that specific directory.
@@ -263,7 +320,7 @@ more details, with real examples.
 
 	Use of lib-y is normally restricted to lib/ and arch/*/lib.
 
---- 3.6 Descending down in directories
+--- 3.7 Descending down in directories
 
 	A Makefile is only responsible for building objects in its own
 	directory. Files in subdirectories should be taken care of by
@@ -290,7 +347,7 @@ more details, with real examples.
 	names. This allows kbuild to totally skip the directory if the
 	corresponding CONFIG_ option is neither 'y' nor 'm'.
 
---- 3.7 Compilation flags
+--- 3.8 Compilation flags
 
     ccflags-y, asflags-y and ldflags-y
 	These three flags apply only to the kbuild makefile in which they



From neilb at suse.com  Fri Jun 15 07:11:30 2018
From: neilb at suse.com (NeilBrown)
Date: Fri, 15 Jun 2018 17:11:30 +1000
Subject: [lustre-devel] [PATCH 06/24] lustre: build ldlm in the ldlm
 directory.
In-Reply-To: <152904663333.10587.10934053155404014785.stgit@noble>
References: <152904663333.10587.10934053155404014785.stgit@noble>
Message-ID: <152904669026.10587.12104937101821160060.stgit@noble>

ldlm code is currently build from the ptlrpc directory,
with files listed in the Makefile of that directory.
This is because both need to be in the same module.
This make partial builds such as
  make drivers/staging/lustre/lustre/ldlm/

ineffective.
kbuild recently gained the ability to build modules
from multiple directories.  Use that ability to
allow ldlm code to be built in the correct directory.

Note that we need to set KBUILD_MODNAME for ldlm to
build correctly.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 drivers/staging/lustre/lustre/ldlm/Makefile   |   11 +++++++++++
 drivers/staging/lustre/lustre/ptlrpc/Makefile |   12 +++---------
 2 files changed, 14 insertions(+), 9 deletions(-)
 create mode 100644 drivers/staging/lustre/lustre/ldlm/Makefile

diff --git a/drivers/staging/lustre/lustre/ldlm/Makefile b/drivers/staging/lustre/lustre/ldlm/Makefile
new file mode 100644
index 000000000000..413943230572
--- /dev/null
+++ b/drivers/staging/lustre/lustre/ldlm/Makefile
@@ -0,0 +1,11 @@
+# SPDX-License-Identifier: GPL-2.0
+ccflags-y += -I$(srctree)/drivers/staging/lustre/include
+ccflags-y += -I$(srctree)/drivers/staging/lustre/lustre/include
+ccflags-y += -DKBUILD_MODNAME='"ptlrpc"'
+
+modobj-$(CONFIG_LUSTRE_FS) += l_lock.o ldlm_lock.o
+modobj-$(CONFIG_LUSTRE_FS) += ldlm_resource.o ldlm_lib.o
+modobj-$(CONFIG_LUSTRE_FS) += ldlm_plain.o ldlm_extent.o
+modobj-$(CONFIG_LUSTRE_FS) += ldlm_request.o ldlm_lockd.o
+modobj-$(CONFIG_LUSTRE_FS) += ldlm_flock.o ldlm_inodebits.o
+modobj-$(CONFIG_LUSTRE_FS) += ldlm_pool.o
diff --git a/drivers/staging/lustre/lustre/ptlrpc/Makefile b/drivers/staging/lustre/lustre/ptlrpc/Makefile
index e0a482712d8e..9c27b850c52d 100644
--- a/drivers/staging/lustre/lustre/ptlrpc/Makefile
+++ b/drivers/staging/lustre/lustre/ptlrpc/Makefile
@@ -3,20 +3,14 @@ ccflags-y += -I$(srctree)/drivers/staging/lustre/include
 ccflags-y += -I$(srctree)/drivers/staging/lustre/lustre/include
 
 obj-$(CONFIG_LUSTRE_FS) += ptlrpc.o
-LDLM := ../../lustre/ldlm/
 
-ldlm_objs := $(LDLM)l_lock.o $(LDLM)ldlm_lock.o
-ldlm_objs += $(LDLM)ldlm_resource.o $(LDLM)ldlm_lib.o
-ldlm_objs += $(LDLM)ldlm_plain.o $(LDLM)ldlm_extent.o
-ldlm_objs += $(LDLM)ldlm_request.o $(LDLM)ldlm_lockd.o
-ldlm_objs += $(LDLM)ldlm_flock.o $(LDLM)ldlm_inodebits.o
-ldlm_objs += $(LDLM)ldlm_pool.o
-ptlrpc_objs := client.o recover.o connection.o niobuf.o pack_generic.o
+ptlrpc_objs += ../ldlm/
+ptlrpc_objs += client.o recover.o connection.o niobuf.o pack_generic.o
 ptlrpc_objs += events.o ptlrpc_module.o service.o pinger.o
 ptlrpc_objs += llog_net.o llog_client.o import.o ptlrpcd.o
 ptlrpc_objs += pers.o lproc_ptlrpc.o wiretest.o layout.o
 ptlrpc_objs += sec.o sec_bulk.o sec_gc.o sec_config.o
 ptlrpc_objs += sec_null.o sec_plain.o nrs.o nrs_fifo.o
 
-ptlrpc-y := $(ldlm_objs) $(ptlrpc_objs) sec_lproc.o
+ptlrpc-y := $(ptlrpc_objs) sec_lproc.o
 ptlrpc-$(CONFIG_LUSTRE_TRANSLATE_ERRNOS) += errno.o



From neilb at suse.com  Fri Jun 15 07:11:30 2018
From: neilb at suse.com (NeilBrown)
Date: Fri, 15 Jun 2018 17:11:30 +1000
Subject: [lustre-devel] [PATCH 07/24] lustre: merge libcfs module into lnet.
In-Reply-To: <152904663333.10587.10934053155404014785.stgit@noble>
References: <152904663333.10587.10934053155404014785.stgit@noble>
Message-ID: <152904669029.10587.7251524855971260363.stgit@noble>

We want to ultimately make libcfs disappear.  This step
makes the module disappear - all libcfs code is now included
in the lnet module.
This mean the lnet module_init function needs to call lnet_init().
Also the module parameters have changed names so we need to update
the links from debugs.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 drivers/staging/lustre/lnet/Makefile        |    7 ++++++-
 drivers/staging/lustre/lnet/libcfs/Makefile |    5 ++---
 drivers/staging/lustre/lnet/libcfs/module.c |   26 +++++++++-----------------
 drivers/staging/lustre/lnet/lnet/Makefile   |   11 +++++------
 drivers/staging/lustre/lnet/lnet/module.c   |    9 +++++++++
 5 files changed, 31 insertions(+), 27 deletions(-)

diff --git a/drivers/staging/lustre/lnet/Makefile b/drivers/staging/lustre/lnet/Makefile
index 0a380fe88ce8..e28be81e36cf 100644
--- a/drivers/staging/lustre/lnet/Makefile
+++ b/drivers/staging/lustre/lnet/Makefile
@@ -1 +1,6 @@
-obj-$(CONFIG_LNET) += libcfs/ lnet/ klnds/ selftest/
+# SPDX-License-Identifier: GPL-2.0
+
+obj-$(CONFIG_LNET) += klnds/ selftest/
+obj-$(CONFIG_LNET) += lnet.o
+
+lnet-y = libcfs/ lnet/
diff --git a/drivers/staging/lustre/lnet/libcfs/Makefile b/drivers/staging/lustre/lnet/libcfs/Makefile
index 5b13edc72837..1b6937c3deee 100644
--- a/drivers/staging/lustre/lnet/libcfs/Makefile
+++ b/drivers/staging/lustre/lnet/libcfs/Makefile
@@ -1,8 +1,7 @@
 # SPDX-License-Identifier: GPL-2.0
 ccflags-y += -I$(srctree)/drivers/staging/lustre/include
 ccflags-y += -I$(srctree)/drivers/staging/lustre/lustre/include
-
-obj-$(CONFIG_LNET) += libcfs.o
+ccflags-y += -DKBUILD_MODNAME='"lnet"'
 
 libcfs-obj-y += linux-tracefile.o linux-debug.o
 libcfs-obj-y += linux-crypto.o
@@ -13,4 +12,4 @@ libcfs-obj-y += libcfs_string.o hash.o
 libcfs-obj-$(CONFIG_SMP) += libcfs_cpu.o
 libcfs-obj-y += libcfs_mem.o libcfs_lock.o
 
-libcfs-objs := $(libcfs-obj-y)
+modobj-$(CONFIG_LNET) += $(libcfs-obj-y)
diff --git a/drivers/staging/lustre/lnet/libcfs/module.c b/drivers/staging/lustre/lnet/libcfs/module.c
index 02c404c6738e..b3eb0789425b 100644
--- a/drivers/staging/lustre/lnet/libcfs/module.c
+++ b/drivers/staging/lustre/lnet/libcfs/module.c
@@ -513,19 +513,19 @@ static struct ctl_table lnet_table[] = {
 
 static const struct lnet_debugfs_symlink_def lnet_debugfs_symlinks[] = {
 	{ "console_ratelimit",
-	  "/sys/module/libcfs/parameters/libcfs_console_ratelimit"},
+	  "/sys/module/lnet/parameters/libcfs_console_ratelimit"},
 	{ "debug_path",
-	  "/sys/module/libcfs/parameters/libcfs_debug_file_path"},
+	  "/sys/module/lnet/parameters/libcfs_debug_file_path"},
 	{ "panic_on_lbug",
-	  "/sys/module/libcfs/parameters/libcfs_panic_on_lbug"},
+	  "/sys/module/lnet/parameters/libcfs_panic_on_lbug"},
 	{ "libcfs_console_backoff",
-	  "/sys/module/libcfs/parameters/libcfs_console_backoff"},
+	  "/sys/module/lnet/parameters/libcfs_console_backoff"},
 	{ "debug_mb",
-	  "/sys/module/libcfs/parameters/libcfs_debug_mb"},
+	  "/sys/module/lnet/parameters/libcfs_debug_mb"},
 	{ "console_min_delay_centisecs",
-	  "/sys/module/libcfs/parameters/libcfs_console_min_delay"},
+	  "/sys/module/lnet/parameters/libcfs_console_min_delay"},
 	{ "console_max_delay_centisecs",
-	  "/sys/module/libcfs/parameters/libcfs_console_max_delay"},
+	  "/sys/module/lnet/parameters/libcfs_console_max_delay"},
 	{},
 };
 
@@ -688,7 +688,7 @@ int libcfs_setup(void)
 }
 EXPORT_SYMBOL(libcfs_setup);
 
-static int libcfs_init(void)
+int libcfs_init(void)
 {
 	int rc;
 
@@ -700,7 +700,7 @@ static int libcfs_init(void)
 	return rc;
 }
 
-static void libcfs_exit(void)
+void libcfs_exit(void)
 {
 	int rc;
 
@@ -720,11 +720,3 @@ static void libcfs_exit(void)
 	if (rc)
 		pr_err("LustreError: libcfs_debug_cleanup: %d\n", rc);
 }
-
-MODULE_AUTHOR("OpenSFS, Inc. <http://www.lustre.org/>");
-MODULE_DESCRIPTION("Lustre helper library");
-MODULE_VERSION(LIBCFS_VERSION);
-MODULE_LICENSE("GPL");
-
-module_init(libcfs_init);
-module_exit(libcfs_exit);
diff --git a/drivers/staging/lustre/lnet/lnet/Makefile b/drivers/staging/lustre/lnet/lnet/Makefile
index cbbb52133591..8323d98a9a1a 100644
--- a/drivers/staging/lustre/lnet/lnet/Makefile
+++ b/drivers/staging/lustre/lnet/lnet/Makefile
@@ -1,10 +1,9 @@
 # SPDX-License-Identifier: GPL-2.0
 ccflags-y += -I$(srctree)/drivers/staging/lustre/include
 ccflags-y += -I$(srctree)/drivers/staging/lustre/lustre/include
+ccflags-y += -DKBUILD_MODNAME='"lnet"'
 
-obj-$(CONFIG_LNET) += lnet.o
-
-lnet-y := api-ni.o config.o nidstrings.o net_fault.o		\
-	  lib-me.o lib-msg.o lib-eq.o lib-md.o lib-ptl.o	\
-	  lib-socket.o lib-move.o module.o lo.o			\
-	  router.o router_proc.o acceptor.o peer.o
+modobj-$(CONFIG_LNET) += api-ni.o config.o nidstrings.o net_fault.o
+modobj-$(CONFIG_LNET) += lib-me.o lib-msg.o lib-eq.o lib-md.o lib-ptl.o
+modobj-$(CONFIG_LNET) += lib-socket.o lib-move.o module.o lo.o
+modobj-$(CONFIG_LNET) += router.o router_proc.o acceptor.o peer.o
diff --git a/drivers/staging/lustre/lnet/lnet/module.c b/drivers/staging/lustre/lnet/lnet/module.c
index 9d06664f0c17..52bbe4524bb2 100644
--- a/drivers/staging/lustre/lnet/lnet/module.c
+++ b/drivers/staging/lustre/lnet/lnet/module.c
@@ -36,6 +36,9 @@
 #include <linux/lnet/lib-lnet.h>
 #include <uapi/linux/lnet/lnet-dlc.h>
 
+int libcfs_init(void);
+void libcfs_exit(void);
+
 static int config_on_load;
 module_param(config_on_load, int, 0444);
 MODULE_PARM_DESC(config_on_load, "configure network at module load");
@@ -192,6 +195,10 @@ static int __init lnet_init(void)
 {
 	int rc;
 
+	rc = libcfs_init();
+	if (rc)
+		return rc;
+
 	mutex_init(&lnet_config_mutex);
 
 	rc = libcfs_setup();
@@ -228,12 +235,14 @@ static void __exit lnet_exit(void)
 	LASSERT(!rc);
 
 	lnet_lib_exit();
+	libcfs_exit();
 }
 
 MODULE_AUTHOR("OpenSFS, Inc. <http://www.lustre.org/>");
 MODULE_DESCRIPTION("Lustre Networking layer");
 MODULE_VERSION(LNET_VERSION);
 MODULE_LICENSE("GPL");
+MODULE_ALIAS("libcfs");
 
 module_init(lnet_init);
 module_exit(lnet_exit);



From neilb at suse.com  Fri Jun 15 07:11:30 2018
From: neilb at suse.com (NeilBrown)
Date: Fri, 15 Jun 2018 17:11:30 +1000
Subject: [lustre-devel] [PATCH 08/24] lustre: use truncate_inode_page in
 place of truncate_complete_page
In-Reply-To: <152904663333.10587.10934053155404014785.stgit@noble>
References: <152904663333.10587.10934053155404014785.stgit@noble>
Message-ID: <152904669032.10587.11474850109829193046.stgit@noble>

Carrying a local truncate_complete_page() is not good
for maintainability.  Linux now used truncate_inode_page,
so use that instead.
For correct use, we need to pass the mapping to mdc_release_page()
rather than trusting page->mapping.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 .../lustre/include/lustre_patchless_compat.h       |   16 ----------------
 drivers/staging/lustre/lustre/llite/dir.c          |    3 +--
 .../staging/lustre/lustre/llite/llite_internal.h   |    4 ++--
 drivers/staging/lustre/lustre/mdc/mdc_request.c    |   12 ++++++------
 4 files changed, 9 insertions(+), 26 deletions(-)

diff --git a/drivers/staging/lustre/lustre/include/lustre_patchless_compat.h b/drivers/staging/lustre/lustre/include/lustre_patchless_compat.h
index 298476ea7557..dca1e3dbd183 100644
--- a/drivers/staging/lustre/lustre/include/lustre_patchless_compat.h
+++ b/drivers/staging/lustre/lustre/include/lustre_patchless_compat.h
@@ -41,22 +41,6 @@
 #include <linux/hash.h>
 #include <linux/pagemap.h>
 
-#define ll_delete_from_page_cache(page) delete_from_page_cache(page)
-
-static inline void
-truncate_complete_page(struct address_space *mapping, struct page *page)
-{
-	if (page->mapping != mapping)
-		return;
-
-	if (PagePrivate(page))
-		page->mapping->a_ops->invalidatepage(page, 0, PAGE_SIZE);
-
-	cancel_dirty_page(page);
-	ClearPageMappedToDisk(page);
-	ll_delete_from_page_cache(page);
-}
-
 #ifndef ATTR_CTIME_SET
 /*
  * set ATTR_CTIME_SET to a high value to avoid any risk of collision with other
diff --git a/drivers/staging/lustre/lustre/llite/dir.c b/drivers/staging/lustre/lustre/llite/dir.c
index 688dddf3ca47..cc5869cf9f93 100644
--- a/drivers/staging/lustre/lustre/llite/dir.c
+++ b/drivers/staging/lustre/lustre/llite/dir.c
@@ -167,8 +167,7 @@ void ll_release_page(struct inode *inode, struct page *page, bool remove)
 
 	if (remove) {
 		lock_page(page);
-		if (likely(page->mapping))
-			truncate_complete_page(page->mapping, page);
+		truncate_inode_page(inode->i_mapping, page);
 		unlock_page(page);
 	}
 	put_page(page);
diff --git a/drivers/staging/lustre/lustre/llite/llite_internal.h b/drivers/staging/lustre/lustre/llite/llite_internal.h
index c08a6e14b6d7..28cff58690d6 100644
--- a/drivers/staging/lustre/lustre/llite/llite_internal.h
+++ b/drivers/staging/lustre/lustre/llite/llite_internal.h
@@ -938,11 +938,11 @@ static inline void ll_invalidate_page(struct page *vmpage)
 		return;
 
 	/*
-	 * truncate_complete_page() calls
+	 * truncate_inode_page() calls
 	 * a_ops->invalidatepage()->cl_page_delete()->vvp_page_delete().
 	 */
 	ll_teardown_mmaps(mapping, offset, offset + PAGE_SIZE);
-	truncate_complete_page(mapping, vmpage);
+	truncate_inode_page(mapping, vmpage);
 }
 
 #define    ll_s2sbi(sb)	(s2lsi(sb)->lsi_llsbi)
diff --git a/drivers/staging/lustre/lustre/mdc/mdc_request.c b/drivers/staging/lustre/lustre/mdc/mdc_request.c
index cff31cb0a9ac..856e6a376852 100644
--- a/drivers/staging/lustre/lustre/mdc/mdc_request.c
+++ b/drivers/staging/lustre/lustre/mdc/mdc_request.c
@@ -922,12 +922,12 @@ static int mdc_getpage(struct obd_export *exp, const struct lu_fid *fid,
 	return 0;
 }
 
-static void mdc_release_page(struct page *page, int remove)
+static void mdc_release_page(struct address_space *mapping,
+			     struct page *page, int remove)
 {
 	if (remove) {
 		lock_page(page);
-		if (likely(page->mapping))
-			truncate_complete_page(page->mapping, page);
+		truncate_inode_page(mapping, page);
 		unlock_page(page);
 	}
 	put_page(page);
@@ -981,7 +981,7 @@ static struct page *mdc_page_locate(struct address_space *mapping, __u64 *hash,
 			       offset, *start, *end, *hash);
 			if (*hash > *end) {
 				kunmap(page);
-				mdc_release_page(page, 0);
+				mdc_release_page(mapping, page, 0);
 				page = NULL;
 			} else if (*end != *start && *hash == *end) {
 				/*
@@ -991,7 +991,7 @@ static struct page *mdc_page_locate(struct address_space *mapping, __u64 *hash,
 				 * fetch the page we want.
 				 */
 				kunmap(page);
-				mdc_release_page(page,
+				mdc_release_page(mapping, page,
 						 le32_to_cpu(dp->ldp_flags) & LDF_COLLIDE);
 				page = NULL;
 			}
@@ -1365,7 +1365,7 @@ static int mdc_read_page(struct obd_export *exp, struct md_op_data *op_data,
 	return rc;
 fail:
 	kunmap(page);
-	mdc_release_page(page, 1);
+	mdc_release_page(mapping, page, 1);
 	rc = -EIO;
 	goto out_unlock;
 }



From neilb at suse.com  Fri Jun 15 07:11:30 2018
From: neilb at suse.com (NeilBrown)
Date: Fri, 15 Jun 2018 17:11:30 +1000
Subject: [lustre-devel] [PATCH 09/24] lustre: don't use spare bits in
 iattr.ia_valid
In-Reply-To: <152904663333.10587.10934053155404014785.stgit@noble>
References: <152904663333.10587.10934053155404014785.stgit@noble>
Message-ID: <152904669036.10587.17899565981649604892.stgit@noble>

Using spare bits in iattr.ia_valid is not safe as the
bits might get used for some other purpose without
us noticing.
lustre currently used 4 spare bit, but they are all
defined in different places so this isn't immediately
obvious.

This patch brings all those bit definitions together
and defined a new op_data field (op_xvalid) to hold
the extra validity bits.

It also replaces sa_valid in struct cl_setattr_io
with sa_avalid and sa_xvalid.  Changing the name is
helpful as sa_valid already has another use within
lustre.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 drivers/staging/lustre/lustre/include/cl_object.h  |    5 +++-
 .../staging/lustre/lustre/include/lustre_compat.h  |    6 -----
 .../lustre/include/lustre_patchless_compat.h       |    8 -------
 drivers/staging/lustre/lustre/include/obd.h        |    7 ++++++
 drivers/staging/lustre/lustre/llite/file.c         |   12 ++++++----
 drivers/staging/lustre/lustre/llite/lcommon_cl.c   |    5 +++-
 .../staging/lustre/lustre/llite/llite_internal.h   |    5 +++-
 drivers/staging/lustre/lustre/llite/llite_lib.c    |   21 ++++++++++--------
 drivers/staging/lustre/lustre/llite/vvp_io.c       |    8 +++----
 drivers/staging/lustre/lustre/lmv/lmv_obd.c        |    5 +++-
 drivers/staging/lustre/lustre/lov/lov_io.c         |    3 ++-
 drivers/staging/lustre/lustre/mdc/mdc_lib.c        |   13 ++++++-----
 drivers/staging/lustre/lustre/osc/osc_io.c         |   23 ++++++++++----------
 13 files changed, 63 insertions(+), 58 deletions(-)

diff --git a/drivers/staging/lustre/lustre/include/cl_object.h b/drivers/staging/lustre/lustre/include/cl_object.h
index 6f7b991be809..caa66763c70d 100644
--- a/drivers/staging/lustre/lustre/include/cl_object.h
+++ b/drivers/staging/lustre/lustre/include/cl_object.h
@@ -1773,7 +1773,8 @@ struct cl_io {
 		struct cl_setattr_io {
 			struct ost_lvb   sa_attr;
 			unsigned int		 sa_attr_flags;
-			unsigned int     sa_valid;
+			unsigned int     sa_avalid;
+			unsigned int     sa_xvalid;
 			int		sa_stripe_index;
 			const struct lu_fid	*sa_parent_fid;
 		} ci_setattr;
@@ -2304,7 +2305,7 @@ static inline int cl_io_is_mkwrite(const struct cl_io *io)
 static inline int cl_io_is_trunc(const struct cl_io *io)
 {
 	return io->ci_type == CIT_SETATTR &&
-		(io->u.ci_setattr.sa_valid & ATTR_SIZE);
+		(io->u.ci_setattr.sa_avalid & ATTR_SIZE);
 }
 
 struct cl_io *cl_io_top(struct cl_io *io);
diff --git a/drivers/staging/lustre/lustre/include/lustre_compat.h b/drivers/staging/lustre/lustre/include/lustre_compat.h
index 3c6db0d632dc..6d1a433e32c5 100644
--- a/drivers/staging/lustre/lustre/include/lustre_compat.h
+++ b/drivers/staging/lustre/lustre/include/lustre_compat.h
@@ -41,12 +41,6 @@
 
 #include <lustre_patchless_compat.h>
 
-/*
- * set ATTR_BLOCKS to a high value to avoid any risk of collision with other
- * ATTR_* attributes (see bug 13828)
- */
-#define ATTR_BLOCKS    (1 << 27)
-
 #define current_ngroups current_cred()->group_info->ngroups
 #define current_groups current_cred()->group_info->small_block
 
diff --git a/drivers/staging/lustre/lustre/include/lustre_patchless_compat.h b/drivers/staging/lustre/lustre/include/lustre_patchless_compat.h
index dca1e3dbd183..1bbe1a779ae9 100644
--- a/drivers/staging/lustre/lustre/include/lustre_patchless_compat.h
+++ b/drivers/staging/lustre/lustre/include/lustre_patchless_compat.h
@@ -41,12 +41,4 @@
 #include <linux/hash.h>
 #include <linux/pagemap.h>
 
-#ifndef ATTR_CTIME_SET
-/*
- * set ATTR_CTIME_SET to a high value to avoid any risk of collision with other
- * ATTR_* attributes (see bug 13828)
- */
-#define ATTR_CTIME_SET (1 << 28)
-#endif
-
 #endif /* LUSTRE_PATCHLESS_COMPAT_H */
diff --git a/drivers/staging/lustre/lustre/include/obd.h b/drivers/staging/lustre/lustre/include/obd.h
index b1907bbffb19..d38b6bc6b246 100644
--- a/drivers/staging/lustre/lustre/include/obd.h
+++ b/drivers/staging/lustre/lustre/include/obd.h
@@ -723,6 +723,7 @@ struct md_op_data {
 
 	/* iattr fields and blocks. */
 	struct iattr	    op_attr;
+	unsigned int		op_xvalid; /* eXtra validity flags */
 	unsigned int	    op_attr_flags;
 	__u64		   op_valid;
 	loff_t		  op_attr_blocks;
@@ -751,6 +752,12 @@ struct md_op_data {
 	__u32			op_default_stripe_offset;
 };
 
+/* Flags for op_xvalid */
+#define OP_ATTR_CTIME_SET	(1 << 0)
+#define OP_ATTR_BLOCKS		(1 << 1)
+#define OP_ATTR_OWNEROVERRIDE	(1 << 2)
+#define OP_ATTR_FLAGS		(1 << 3)
+
 struct md_callback {
 	int (*md_blocking_ast)(struct ldlm_lock *lock,
 			       struct ldlm_lock_desc *desc,
diff --git a/drivers/staging/lustre/lustre/llite/file.c b/drivers/staging/lustre/lustre/llite/file.c
index e888ed6e74bc..e642996db53f 100644
--- a/drivers/staging/lustre/lustre/llite/file.c
+++ b/drivers/staging/lustre/lustre/llite/file.c
@@ -93,9 +93,10 @@ static void ll_prepare_close(struct inode *inode, struct md_op_data *op_data,
 	op_data->op_attr.ia_mtime = inode->i_mtime;
 	op_data->op_attr.ia_ctime = inode->i_ctime;
 	op_data->op_attr.ia_size = i_size_read(inode);
-	op_data->op_attr.ia_valid |= ATTR_MODE | ATTR_ATIME | ATTR_ATIME_SET |
-				     ATTR_MTIME | ATTR_MTIME_SET |
-				     ATTR_CTIME | ATTR_CTIME_SET;
+	op_data->op_attr.ia_valid |= (ATTR_MODE | ATTR_ATIME | ATTR_ATIME_SET |
+				      ATTR_MTIME | ATTR_MTIME_SET |
+				      ATTR_CTIME);
+	op_data->op_xvalid |= OP_ATTR_CTIME_SET;
 	op_data->op_attr_blocks = inode->i_blocks;
 	op_data->op_attr_flags = ll_inode_to_ext_flags(inode->i_flags);
 	op_data->op_handle = och->och_fh;
@@ -161,7 +162,8 @@ static int ll_close_inode_openhandle(struct inode *inode,
 		op_data->op_bias |= MDS_HSM_RELEASE;
 		op_data->op_data_version = *(__u64 *)data;
 		op_data->op_lease_handle = och->och_lease_handle;
-		op_data->op_attr.ia_valid |= ATTR_SIZE | ATTR_BLOCKS;
+		op_data->op_attr.ia_valid |= ATTR_SIZE;
+		op_data->op_xvalid |= OP_ATTR_BLOCKS;
 		break;
 
 	default:
@@ -1906,7 +1908,7 @@ static int ll_hsm_import(struct inode *inode, struct file *file,
 
 	inode_lock(inode);
 
-	rc = ll_setattr_raw(file->f_path.dentry, attr, true);
+	rc = ll_setattr_raw(file->f_path.dentry, attr, 0, true);
 	if (rc == -ENODATA)
 		rc = 0;
 
diff --git a/drivers/staging/lustre/lustre/llite/lcommon_cl.c b/drivers/staging/lustre/lustre/llite/lcommon_cl.c
index d7ea39ce0cb2..9d1f3881f860 100644
--- a/drivers/staging/lustre/lustre/llite/lcommon_cl.c
+++ b/drivers/staging/lustre/lustre/llite/lcommon_cl.c
@@ -80,7 +80,7 @@ u16 cl_inode_fini_refcheck;
 static DEFINE_MUTEX(cl_inode_fini_guard);
 
 int cl_setattr_ost(struct cl_object *obj, const struct iattr *attr,
-		   unsigned int attr_flags)
+		   unsigned int xvalid, unsigned int attr_flags)
 {
 	struct lu_env *env;
 	struct cl_io  *io;
@@ -100,7 +100,8 @@ int cl_setattr_ost(struct cl_object *obj, const struct iattr *attr,
 	io->u.ci_setattr.sa_attr.lvb_ctime = LTIME_S(attr->ia_ctime);
 	io->u.ci_setattr.sa_attr.lvb_size = attr->ia_size;
 	io->u.ci_setattr.sa_attr_flags = attr_flags;
-	io->u.ci_setattr.sa_valid = attr->ia_valid;
+	io->u.ci_setattr.sa_avalid = attr->ia_valid;
+	io->u.ci_setattr.sa_xvalid = xvalid;
 	io->u.ci_setattr.sa_parent_fid = lu_object_fid(&obj->co_lu);
 
 again:
diff --git a/drivers/staging/lustre/lustre/llite/llite_internal.h b/drivers/staging/lustre/lustre/llite/llite_internal.h
index 28cff58690d6..b750a48fd0e1 100644
--- a/drivers/staging/lustre/lustre/llite/llite_internal.h
+++ b/drivers/staging/lustre/lustre/llite/llite_internal.h
@@ -805,7 +805,8 @@ void ll_kill_super(struct super_block *sb);
 struct inode *ll_inode_from_resource_lock(struct ldlm_lock *lock);
 void ll_dir_clear_lsm_md(struct inode *inode);
 void ll_clear_inode(struct inode *inode);
-int ll_setattr_raw(struct dentry *dentry, struct iattr *attr, bool hsm_import);
+int ll_setattr_raw(struct dentry *dentry, struct iattr *attr,
+		   unsigned int xvalid, bool hsm_import);
 int ll_setattr(struct dentry *de, struct iattr *attr);
 int ll_statfs(struct dentry *de, struct kstatfs *sfs);
 int ll_statfs_internal(struct super_block *sb, struct obd_statfs *osfs,
@@ -1330,7 +1331,7 @@ int ll_getparent(struct file *file, struct getparent __user *arg);
 
 /* lcommon_cl.c */
 int cl_setattr_ost(struct cl_object *obj, const struct iattr *attr,
-		   unsigned int attr_flags);
+		   unsigned int xvalid, unsigned int attr_flags);
 
 extern struct lu_env *cl_inode_fini_env;
 extern u16 cl_inode_fini_refcheck;
diff --git a/drivers/staging/lustre/lustre/llite/llite_lib.c b/drivers/staging/lustre/lustre/llite/llite_lib.c
index 36066c839160..cd0f2517a3b3 100644
--- a/drivers/staging/lustre/lustre/llite/llite_lib.c
+++ b/drivers/staging/lustre/lustre/llite/llite_lib.c
@@ -1429,7 +1429,8 @@ static int ll_md_setattr(struct dentry *dentry, struct md_op_data *op_data)
  *
  * In case of HSMimport, we only set attr on MDS.
  */
-int ll_setattr_raw(struct dentry *dentry, struct iattr *attr, bool hsm_import)
+int ll_setattr_raw(struct dentry *dentry, struct iattr *attr,
+		   unsigned int xvalid, bool hsm_import)
 {
 	struct inode *inode = d_inode(dentry);
 	struct ll_inode_info *lli = ll_i2info(inode);
@@ -1470,7 +1471,7 @@ int ll_setattr_raw(struct dentry *dentry, struct iattr *attr, bool hsm_import)
 	/* We mark all of the fields "set" so MDS/OST does not re-set them */
 	if (attr->ia_valid & ATTR_CTIME) {
 		attr->ia_ctime = current_time(inode);
-		attr->ia_valid |= ATTR_CTIME_SET;
+		xvalid |= OP_ATTR_CTIME_SET;
 	}
 	if (!(attr->ia_valid & ATTR_ATIME_SET) &&
 	    (attr->ia_valid & ATTR_ATIME)) {
@@ -1506,12 +1507,13 @@ int ll_setattr_raw(struct dentry *dentry, struct iattr *attr, bool hsm_import)
 		 * If we are changing file size, file content is
 		 * modified, flag it.
 		 */
-		attr->ia_valid |= MDS_OPEN_OWNEROVERRIDE;
+		xvalid |= OP_ATTR_OWNEROVERRIDE;
 		op_data->op_bias |= MDS_DATA_MODIFIED;
 		clear_bit(LLIF_DATA_MODIFIED, &lli->lli_flags);
 	}
 
 	op_data->op_attr = *attr;
+	op_data->op_xvalid = xvalid;
 
 	rc = ll_md_setattr(dentry, op_data);
 	if (rc)
@@ -1532,7 +1534,8 @@ int ll_setattr_raw(struct dentry *dentry, struct iattr *attr, bool hsm_import)
 		 * setting times to past, but it is necessary due to possible
 		 * time de-synchronization between MDT inode and OST objects
 		 */
-		rc = cl_setattr_ost(ll_i2info(inode)->lli_clob, attr, 0);
+		rc = cl_setattr_ost(ll_i2info(inode)->lli_clob,
+				    attr, xvalid, 0);
 	}
 
 	/*
@@ -1589,10 +1592,11 @@ int ll_setattr_raw(struct dentry *dentry, struct iattr *attr, bool hsm_import)
 int ll_setattr(struct dentry *de, struct iattr *attr)
 {
 	int mode = d_inode(de)->i_mode;
+	unsigned int xvalid = 0;
 
 	if ((attr->ia_valid & (ATTR_CTIME | ATTR_SIZE | ATTR_MODE)) ==
 			      (ATTR_CTIME | ATTR_SIZE | ATTR_MODE))
-		attr->ia_valid |= MDS_OPEN_OWNEROVERRIDE;
+		xvalid |= OP_ATTR_OWNEROVERRIDE;
 
 	if (((attr->ia_valid & (ATTR_MODE | ATTR_FORCE | ATTR_SIZE)) ==
 			       (ATTR_SIZE | ATTR_MODE)) &&
@@ -1613,7 +1617,7 @@ int ll_setattr(struct dentry *de, struct iattr *attr)
 	    !(attr->ia_valid & ATTR_KILL_SGID))
 		attr->ia_valid |= ATTR_KILL_SGID;
 
-	return ll_setattr_raw(de, attr, false);
+	return ll_setattr_raw(de, attr, xvalid, false);
 }
 
 int ll_statfs_internal(struct super_block *sb, struct obd_statfs *osfs,
@@ -1946,7 +1950,7 @@ int ll_iocontrol(struct inode *inode, struct file *file,
 			return PTR_ERR(op_data);
 
 		op_data->op_attr_flags = flags;
-		op_data->op_attr.ia_valid |= ATTR_ATTR_FLAG;
+		op_data->op_xvalid |= OP_ATTR_FLAGS;
 		rc = md_setattr(sbi->ll_md_exp, op_data, NULL, 0, &req);
 		ll_finish_md_op_data(op_data);
 		ptlrpc_req_finished(req);
@@ -1963,8 +1967,7 @@ int ll_iocontrol(struct inode *inode, struct file *file,
 		if (!attr)
 			return -ENOMEM;
 
-		attr->ia_valid = ATTR_ATTR_FLAG;
-		rc = cl_setattr_ost(obj, attr, flags);
+		rc = cl_setattr_ost(obj, attr, OP_ATTR_FLAGS, flags);
 		kfree(attr);
 		return rc;
 	}
diff --git a/drivers/staging/lustre/lustre/llite/vvp_io.c b/drivers/staging/lustre/lustre/llite/vvp_io.c
index e7a4778e02e4..e65523f7628d 100644
--- a/drivers/staging/lustre/lustre/llite/vvp_io.c
+++ b/drivers/staging/lustre/lustre/llite/vvp_io.c
@@ -556,7 +556,7 @@ static int vvp_io_setattr_lock(const struct lu_env *env,
 		if (new_size == 0)
 			enqflags = CEF_DISCARD_DATA;
 	} else {
-		unsigned int valid = io->u.ci_setattr.sa_valid;
+		unsigned int valid = io->u.ci_setattr.sa_avalid;
 
 		if (!(valid & TIMES_SET_FLAGS))
 			return 0;
@@ -603,11 +603,11 @@ static int vvp_io_setattr_time(const struct lu_env *env,
 
 	cl_object_attr_lock(obj);
 	attr->cat_ctime = io->u.ci_setattr.sa_attr.lvb_ctime;
-	if (io->u.ci_setattr.sa_valid & ATTR_ATIME_SET) {
+	if (io->u.ci_setattr.sa_avalid & ATTR_ATIME_SET) {
 		attr->cat_atime = io->u.ci_setattr.sa_attr.lvb_atime;
 		valid |= CAT_ATIME;
 	}
-	if (io->u.ci_setattr.sa_valid & ATTR_MTIME_SET) {
+	if (io->u.ci_setattr.sa_avalid & ATTR_MTIME_SET) {
 		attr->cat_mtime = io->u.ci_setattr.sa_attr.lvb_mtime;
 		valid |= CAT_MTIME;
 	}
@@ -632,7 +632,7 @@ static int vvp_io_setattr_start(const struct lu_env *env,
 		inode_lock(inode);
 	}
 
-	if (io->u.ci_setattr.sa_valid & TIMES_SET_FLAGS)
+	if (io->u.ci_setattr.sa_avalid & TIMES_SET_FLAGS)
 		return vvp_io_setattr_time(env, ios);
 
 	return 0;
diff --git a/drivers/staging/lustre/lustre/lmv/lmv_obd.c b/drivers/staging/lustre/lustre/lmv/lmv_obd.c
index 65f94e6ecaad..9f427a7c7b03 100644
--- a/drivers/staging/lustre/lustre/lmv/lmv_obd.c
+++ b/drivers/staging/lustre/lustre/lmv/lmv_obd.c
@@ -1983,8 +1983,9 @@ static int lmv_setattr(struct obd_export *exp, struct md_op_data *op_data,
 	struct lmv_obd	  *lmv = &obd->u.lmv;
 	struct lmv_tgt_desc     *tgt;
 
-	CDEBUG(D_INODE, "SETATTR for " DFID ", valid 0x%x\n",
-	       PFID(&op_data->op_fid1), op_data->op_attr.ia_valid);
+	CDEBUG(D_INODE, "SETATTR for " DFID ", valid 0x%x/0x%x\n",
+	       PFID(&op_data->op_fid1), op_data->op_attr.ia_valid,
+		op_data->op_xvalid);
 
 	op_data->op_flags |= MF_MDC_CANCEL_FID1;
 	tgt = lmv_find_target(lmv, &op_data->op_fid1);
diff --git a/drivers/staging/lustre/lustre/lov/lov_io.c b/drivers/staging/lustre/lustre/lov/lov_io.c
index b823f8a21856..50982848bca4 100644
--- a/drivers/staging/lustre/lustre/lov/lov_io.c
+++ b/drivers/staging/lustre/lustre/lov/lov_io.c
@@ -77,7 +77,8 @@ static void lov_io_sub_inherit(struct cl_io *io, struct lov_io *lio,
 		io->u.ci_setattr.sa_attr = parent->u.ci_setattr.sa_attr;
 		io->u.ci_setattr.sa_attr_flags =
 					parent->u.ci_setattr.sa_attr_flags;
-		io->u.ci_setattr.sa_valid = parent->u.ci_setattr.sa_valid;
+		io->u.ci_setattr.sa_avalid = parent->u.ci_setattr.sa_avalid;
+		io->u.ci_setattr.sa_xvalid = parent->u.ci_setattr.sa_xvalid;
 		io->u.ci_setattr.sa_stripe_index = stripe;
 		io->u.ci_setattr.sa_parent_fid =
 					parent->u.ci_setattr.sa_parent_fid;
diff --git a/drivers/staging/lustre/lustre/mdc/mdc_lib.c b/drivers/staging/lustre/lustre/mdc/mdc_lib.c
index d582968987ff..ad7c13be5e3c 100644
--- a/drivers/staging/lustre/lustre/mdc/mdc_lib.c
+++ b/drivers/staging/lustre/lustre/mdc/mdc_lib.c
@@ -234,7 +234,7 @@ void mdc_open_pack(struct ptlrpc_request *req, struct md_op_data *op_data,
 	set_mrc_cr_flags(rec, cr_flags);
 }
 
-static inline __u64 attr_pack(unsigned int ia_valid)
+static inline __u64 attr_pack(unsigned int ia_valid, unsigned int ia_xvalid)
 {
 	__u64 sa_valid = 0;
 
@@ -258,19 +258,19 @@ static inline __u64 attr_pack(unsigned int ia_valid)
 		sa_valid |= MDS_ATTR_MTIME_SET;
 	if (ia_valid & ATTR_FORCE)
 		sa_valid |= MDS_ATTR_FORCE;
-	if (ia_valid & ATTR_ATTR_FLAG)
+	if (ia_xvalid & OP_ATTR_FLAGS)
 		sa_valid |= MDS_ATTR_ATTR_FLAG;
 	if (ia_valid & ATTR_KILL_SUID)
 		sa_valid |=  MDS_ATTR_KILL_SUID;
 	if (ia_valid & ATTR_KILL_SGID)
 		sa_valid |= MDS_ATTR_KILL_SGID;
-	if (ia_valid & ATTR_CTIME_SET)
+	if (ia_xvalid & OP_ATTR_CTIME_SET)
 		sa_valid |= MDS_ATTR_CTIME_SET;
 	if (ia_valid & ATTR_OPEN)
 		sa_valid |= MDS_ATTR_FROM_OPEN;
-	if (ia_valid & ATTR_BLOCKS)
+	if (ia_xvalid & OP_ATTR_BLOCKS)
 		sa_valid |= MDS_ATTR_BLOCKS;
-	if (ia_valid & MDS_OPEN_OWNEROVERRIDE)
+	if (ia_xvalid & OP_ATTR_OWNEROVERRIDE)
 		/* NFSD hack (see bug 5781) */
 		sa_valid |= MDS_OPEN_OWNEROVERRIDE;
 	return sa_valid;
@@ -286,7 +286,8 @@ static void mdc_setattr_pack_rec(struct mdt_rec_setattr *rec,
 	rec->sa_suppgid = -1;
 
 	rec->sa_fid    = op_data->op_fid1;
-	rec->sa_valid  = attr_pack(op_data->op_attr.ia_valid);
+	rec->sa_valid  = attr_pack(op_data->op_attr.ia_valid,
+				   op_data->op_xvalid);
 	rec->sa_mode   = op_data->op_attr.ia_mode;
 	rec->sa_uid    = from_kuid(&init_user_ns, op_data->op_attr.ia_uid);
 	rec->sa_gid    = from_kgid(&init_user_ns, op_data->op_attr.ia_gid);
diff --git a/drivers/staging/lustre/lustre/osc/osc_io.c b/drivers/staging/lustre/lustre/osc/osc_io.c
index 67734a8ed331..955525f69958 100644
--- a/drivers/staging/lustre/lustre/osc/osc_io.c
+++ b/drivers/staging/lustre/lustre/osc/osc_io.c
@@ -499,7 +499,8 @@ static int osc_io_setattr_start(const struct lu_env *env,
 	struct obdo *oa = &oio->oi_oa;
 	struct osc_async_cbargs *cbargs = &oio->oi_cbarg;
 	__u64 size = io->u.ci_setattr.sa_attr.lvb_size;
-	unsigned int ia_valid = io->u.ci_setattr.sa_valid;
+	unsigned int ia_avalid = io->u.ci_setattr.sa_avalid;
+	unsigned int ia_xvalid = io->u.ci_setattr.sa_xvalid;
 	int result = 0;
 
 	/* truncate cache dirty pages first */
@@ -514,20 +515,20 @@ static int osc_io_setattr_start(const struct lu_env *env,
 			struct ost_lvb *lvb = &io->u.ci_setattr.sa_attr;
 			unsigned int cl_valid = 0;
 
-			if (ia_valid & ATTR_SIZE) {
+			if (ia_avalid & ATTR_SIZE) {
 				attr->cat_size = size;
 				attr->cat_kms = size;
 				cl_valid = CAT_SIZE | CAT_KMS;
 			}
-			if (ia_valid & ATTR_MTIME_SET) {
+			if (ia_avalid & ATTR_MTIME_SET) {
 				attr->cat_mtime = lvb->lvb_mtime;
 				cl_valid |= CAT_MTIME;
 			}
-			if (ia_valid & ATTR_ATIME_SET) {
+			if (ia_avalid & ATTR_ATIME_SET) {
 				attr->cat_atime = lvb->lvb_atime;
 				cl_valid |= CAT_ATIME;
 			}
-			if (ia_valid & ATTR_CTIME_SET) {
+			if (ia_xvalid & OP_ATTR_CTIME_SET) {
 				attr->cat_ctime = lvb->lvb_ctime;
 				cl_valid |= CAT_CTIME;
 			}
@@ -542,19 +543,19 @@ static int osc_io_setattr_start(const struct lu_env *env,
 		obdo_set_parent_fid(oa, io->u.ci_setattr.sa_parent_fid);
 		oa->o_stripe_idx = io->u.ci_setattr.sa_stripe_index;
 		oa->o_valid |= OBD_MD_FLID | OBD_MD_FLGROUP;
-		if (ia_valid & ATTR_CTIME) {
+		if (ia_avalid & ATTR_CTIME) {
 			oa->o_valid |= OBD_MD_FLCTIME;
 			oa->o_ctime = attr->cat_ctime;
 		}
-		if (ia_valid & ATTR_ATIME) {
+		if (ia_avalid & ATTR_ATIME) {
 			oa->o_valid |= OBD_MD_FLATIME;
 			oa->o_atime = attr->cat_atime;
 		}
-		if (ia_valid & ATTR_MTIME) {
+		if (ia_avalid & ATTR_MTIME) {
 			oa->o_valid |= OBD_MD_FLMTIME;
 			oa->o_mtime = attr->cat_mtime;
 		}
-		if (ia_valid & ATTR_SIZE) {
+		if (ia_avalid & ATTR_SIZE) {
 			oa->o_size = size;
 			oa->o_blocks = OBD_OBJECT_EOF;
 			oa->o_valid |= OBD_MD_FLSIZE | OBD_MD_FLBLOCKS;
@@ -566,14 +567,14 @@ static int osc_io_setattr_start(const struct lu_env *env,
 		} else {
 			LASSERT(oio->oi_lockless == 0);
 		}
-		if (ia_valid & ATTR_ATTR_FLAG) {
+		if (ia_xvalid & OP_ATTR_FLAGS) {
 			oa->o_flags = io->u.ci_setattr.sa_attr_flags;
 			oa->o_valid |= OBD_MD_FLFLAGS;
 		}
 
 		init_completion(&cbargs->opc_sync);
 
-		if (ia_valid & ATTR_SIZE)
+		if (ia_avalid & ATTR_SIZE)
 			result = osc_punch_base(osc_export(cl2osc(obj)),
 						oa, osc_async_upcall,
 						cbargs, PTLRPCD_SET);



From neilb at suse.com  Fri Jun 15 07:11:30 2018
From: neilb at suse.com (NeilBrown)
Date: Fri, 15 Jun 2018 17:11:30 +1000
Subject: [lustre-devel] [PATCH 10/24] lustre: remove
 lustre_patchless_compat.h
In-Reply-To: <152904663333.10587.10934053155404014785.stgit@noble>
References: <152904663333.10587.10934053155404014785.stgit@noble>
Message-ID: <152904669041.10587.15188275655299631716.stgit@noble>

This is now mostly empty.  A few #includes need
to be added to other files to compensate.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 drivers/staging/lustre/lustre/fid/lproc_fid.c      |    1 
 drivers/staging/lustre/lustre/fld/lproc_fld.c      |    1 
 .../staging/lustre/lustre/include/lustre_compat.h  |    2 -
 drivers/staging/lustre/lustre/include/lustre_net.h |    1 
 .../lustre/include/lustre_patchless_compat.h       |   44 --------------------
 .../staging/lustre/lustre/llite/llite_internal.h   |    1 
 drivers/staging/lustre/lustre/lov/lov_obd.c        |    2 +
 drivers/staging/lustre/lustre/lov/lov_pack.c       |    1 
 drivers/staging/lustre/lustre/lov/lov_page.c       |    1 
 drivers/staging/lustre/lustre/obdclass/class_obd.c |    1 
 .../lustre/lustre/obdclass/lprocfs_status.c        |    1 
 .../staging/lustre/lustre/obdclass/obd_config.c    |    1 
 .../staging/lustre/lustre/obdecho/echo_client.c    |    1 
 drivers/staging/lustre/lustre/osc/lproc_osc.c      |    1 
 drivers/staging/lustre/lustre/osc/osc_request.c    |    1 
 .../staging/lustre/lustre/ptlrpc/lproc_ptlrpc.c    |    1 
 drivers/staging/lustre/lustre/ptlrpc/sec_plain.c   |    1 
 17 files changed, 15 insertions(+), 47 deletions(-)
 delete mode 100644 drivers/staging/lustre/lustre/include/lustre_patchless_compat.h

diff --git a/drivers/staging/lustre/lustre/fid/lproc_fid.c b/drivers/staging/lustre/lustre/fid/lproc_fid.c
index 0aabf473c9bd..aa2df68d5acd 100644
--- a/drivers/staging/lustre/lustre/fid/lproc_fid.c
+++ b/drivers/staging/lustre/lustre/fid/lproc_fid.c
@@ -40,6 +40,7 @@
 #define DEBUG_SUBSYSTEM S_FID
 
 #include <linux/module.h>
+#include <linux/uaccess.h>
 
 #include <obd.h>
 #include <obd_class.h>
diff --git a/drivers/staging/lustre/lustre/fld/lproc_fld.c b/drivers/staging/lustre/lustre/fld/lproc_fld.c
index 0bcfb26ef8aa..c85ef9ec639f 100644
--- a/drivers/staging/lustre/lustre/fld/lproc_fld.c
+++ b/drivers/staging/lustre/lustre/fld/lproc_fld.c
@@ -41,6 +41,7 @@
 #define DEBUG_SUBSYSTEM S_FLD
 
 #include <linux/module.h>
+#include <linux/uaccess.h>
 
 #include <obd.h>
 #include <obd_class.h>
diff --git a/drivers/staging/lustre/lustre/include/lustre_compat.h b/drivers/staging/lustre/lustre/include/lustre_compat.h
index 6d1a433e32c5..bf86b0612b50 100644
--- a/drivers/staging/lustre/lustre/include/lustre_compat.h
+++ b/drivers/staging/lustre/lustre/include/lustre_compat.h
@@ -39,8 +39,6 @@
 #include <linux/cred.h>
 #include <linux/module.h>
 
-#include <lustre_patchless_compat.h>
-
 #define current_ngroups current_cred()->group_info->ngroups
 #define current_groups current_cred()->group_info->small_block
 
diff --git a/drivers/staging/lustre/lustre/include/lustre_net.h b/drivers/staging/lustre/lustre/include/lustre_net.h
index 35b43a77eb18..dcad90bc97eb 100644
--- a/drivers/staging/lustre/lustre/include/lustre_net.h
+++ b/drivers/staging/lustre/lustre/include/lustre_net.h
@@ -51,6 +51,7 @@
  * @{
  */
 
+#include <linux/mm.h>
 #include <linux/uio.h>
 #include <linux/libcfs/libcfs.h>
 #include <uapi/linux/lnet/nidstr.h>
diff --git a/drivers/staging/lustre/lustre/include/lustre_patchless_compat.h b/drivers/staging/lustre/lustre/include/lustre_patchless_compat.h
deleted file mode 100644
index 1bbe1a779ae9..000000000000
--- a/drivers/staging/lustre/lustre/include/lustre_patchless_compat.h
+++ /dev/null
@@ -1,44 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-/*
- * GPL HEADER START
- *
- * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 only,
- * as published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful, but
- * WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
- * General Public License version 2 for more details (a copy is included
- * in the LICENSE file that accompanied this code).
- *
- * You should have received a copy of the GNU General Public License
- * version 2 along with this program; If not, see
- * http://www.gnu.org/licenses/gpl-2.0.html
- *
- * GPL HEADER END
- */
-/*
- * Copyright (c) 2007, 2010, Oracle and/or its affiliates. All rights reserved.
- * Use is subject to license terms.
- *
- * Copyright (c) 2011, 2012, Intel Corporation.
- */
-/*
- * This file is part of Lustre, http://www.lustre.org/
- * Lustre is a trademark of Sun Microsystems, Inc.
- */
-
-#ifndef LUSTRE_PATCHLESS_COMPAT_H
-#define LUSTRE_PATCHLESS_COMPAT_H
-
-#include <linux/fs.h>
-
-#include <linux/list.h>
-#include <linux/mm.h>
-#include <linux/hash.h>
-#include <linux/pagemap.h>
-
-#endif /* LUSTRE_PATCHLESS_COMPAT_H */
diff --git a/drivers/staging/lustre/lustre/llite/llite_internal.h b/drivers/staging/lustre/lustre/llite/llite_internal.h
index b750a48fd0e1..0ae5185a4998 100644
--- a/drivers/staging/lustre/lustre/llite/llite_internal.h
+++ b/drivers/staging/lustre/lustre/llite/llite_internal.h
@@ -39,7 +39,6 @@
 #include <lustre_linkea.h>
 
 /* for struct cl_lock_descr and struct cl_io */
-#include <lustre_patchless_compat.h>
 #include <lustre_compat.h>
 #include <cl_object.h>
 #include <lustre_lmv.h>
diff --git a/drivers/staging/lustre/lustre/lov/lov_obd.c b/drivers/staging/lustre/lustre/lov/lov_obd.c
index 344ff4b20168..ee0898cbd9c9 100644
--- a/drivers/staging/lustre/lustre/lov/lov_obd.c
+++ b/drivers/staging/lustre/lustre/lov/lov_obd.c
@@ -40,6 +40,8 @@
 
 #define DEBUG_SUBSYSTEM S_LOV
 
+#include <linux/uaccess.h>
+
 #include <uapi/linux/lustre/lustre_idl.h>
 #include <uapi/linux/lustre/lustre_ioctl.h>
 
diff --git a/drivers/staging/lustre/lustre/lov/lov_pack.c b/drivers/staging/lustre/lustre/lov/lov_pack.c
index b1060d02a164..98b114b8e93b 100644
--- a/drivers/staging/lustre/lustre/lov/lov_pack.c
+++ b/drivers/staging/lustre/lustre/lov/lov_pack.c
@@ -39,6 +39,7 @@
 
 #define DEBUG_SUBSYSTEM S_LOV
 
+#include <linux/uaccess.h>
 #include <lustre_net.h>
 #include <lustre_swab.h>
 #include <obd.h>
diff --git a/drivers/staging/lustre/lustre/lov/lov_page.c b/drivers/staging/lustre/lustre/lov/lov_page.c
index cfae1294d77a..f1c99a2a466e 100644
--- a/drivers/staging/lustre/lustre/lov/lov_page.c
+++ b/drivers/staging/lustre/lustre/lov/lov_page.c
@@ -38,6 +38,7 @@
 
 #define DEBUG_SUBSYSTEM S_LOV
 
+#include <linux/highmem.h>
 #include "lov_cl_internal.h"
 
 /** \addtogroup lov
diff --git a/drivers/staging/lustre/lustre/obdclass/class_obd.c b/drivers/staging/lustre/lustre/obdclass/class_obd.c
index d6c46858941b..81a4c666bb69 100644
--- a/drivers/staging/lustre/lustre/obdclass/class_obd.c
+++ b/drivers/staging/lustre/lustre/obdclass/class_obd.c
@@ -36,6 +36,7 @@
 #include <linux/atomic.h>
 #include <linux/miscdevice.h>
 #include <linux/libcfs/libcfs.h>
+#include <linux/uaccess.h>
 
 #include <obd_support.h>
 #include <obd_class.h>
diff --git a/drivers/staging/lustre/lustre/obdclass/lprocfs_status.c b/drivers/staging/lustre/lustre/obdclass/lprocfs_status.c
index bdbe6f52031a..a40ec42637e6 100644
--- a/drivers/staging/lustre/lustre/obdclass/lprocfs_status.c
+++ b/drivers/staging/lustre/lustre/obdclass/lprocfs_status.c
@@ -37,6 +37,7 @@
 
 #define DEBUG_SUBSYSTEM S_CLASS
 
+#include <linux/uaccess.h>
 #include <obd_class.h>
 #include <lprocfs_status.h>
 #include <uapi/linux/lustre/lustre_idl.h>
diff --git a/drivers/staging/lustre/lustre/obdclass/obd_config.c b/drivers/staging/lustre/lustre/obdclass/obd_config.c
index ffc1814398a5..cfcd17e679dd 100644
--- a/drivers/staging/lustre/lustre/obdclass/obd_config.c
+++ b/drivers/staging/lustre/lustre/obdclass/obd_config.c
@@ -37,6 +37,7 @@
 
 #define DEBUG_SUBSYSTEM S_CLASS
 
+#include <linux/uaccess.h>
 #include <linux/string.h>
 
 #include <uapi/linux/lustre/lustre_ioctl.h>
diff --git a/drivers/staging/lustre/lustre/obdecho/echo_client.c b/drivers/staging/lustre/lustre/obdecho/echo_client.c
index b692e76e7108..484b8d6db6ef 100644
--- a/drivers/staging/lustre/lustre/obdecho/echo_client.c
+++ b/drivers/staging/lustre/lustre/obdecho/echo_client.c
@@ -33,6 +33,7 @@
 
 #define DEBUG_SUBSYSTEM S_ECHO
 
+#include <linux/highmem.h>
 #include <obd.h>
 #include <obd_support.h>
 #include <obd_class.h>
diff --git a/drivers/staging/lustre/lustre/osc/lproc_osc.c b/drivers/staging/lustre/lustre/osc/lproc_osc.c
index 6a705bc5420c..64931b97c887 100644
--- a/drivers/staging/lustre/lustre/osc/lproc_osc.c
+++ b/drivers/staging/lustre/lustre/osc/lproc_osc.c
@@ -32,6 +32,7 @@
  */
 #define DEBUG_SUBSYSTEM S_CLASS
 
+#include <linux/uaccess.h>
 #include <linux/statfs.h>
 #include <obd_cksum.h>
 #include <obd_class.h>
diff --git a/drivers/staging/lustre/lustre/osc/osc_request.c b/drivers/staging/lustre/lustre/osc/osc_request.c
index 61ef6c8d7a12..bcb9b91aa5e0 100644
--- a/drivers/staging/lustre/lustre/osc/osc_request.c
+++ b/drivers/staging/lustre/lustre/osc/osc_request.c
@@ -33,6 +33,7 @@
 
 #define DEBUG_SUBSYSTEM S_OSC
 
+#include <linux/highmem.h>
 #include <linux/libcfs/libcfs_hash.h>
 #include <linux/sched/mm.h>
 
diff --git a/drivers/staging/lustre/lustre/ptlrpc/lproc_ptlrpc.c b/drivers/staging/lustre/lustre/ptlrpc/lproc_ptlrpc.c
index 0b638837f88b..a61e800cd2b5 100644
--- a/drivers/staging/lustre/lustre/ptlrpc/lproc_ptlrpc.c
+++ b/drivers/staging/lustre/lustre/ptlrpc/lproc_ptlrpc.c
@@ -32,6 +32,7 @@
  */
 #define DEBUG_SUBSYSTEM S_CLASS
 
+#include <linux/uaccess.h>
 #include <obd_support.h>
 #include <obd.h>
 #include <lprocfs_status.h>
diff --git a/drivers/staging/lustre/lustre/ptlrpc/sec_plain.c b/drivers/staging/lustre/lustre/ptlrpc/sec_plain.c
index ec3d9af76b17..ead1df71a909 100644
--- a/drivers/staging/lustre/lustre/ptlrpc/sec_plain.c
+++ b/drivers/staging/lustre/lustre/ptlrpc/sec_plain.c
@@ -37,6 +37,7 @@
 
 #define DEBUG_SUBSYSTEM S_SEC
 
+#include <linux/highmem.h>
 #include <obd_support.h>
 #include <obd_cksum.h>
 #include <obd_class.h>



From neilb at suse.com  Fri Jun 15 07:11:30 2018
From: neilb at suse.com (NeilBrown)
Date: Fri, 15 Jun 2018 17:11:30 +1000
Subject: [lustre-devel] [PATCH 11/24] lustre: discard current_n*groups
 macros.
In-Reply-To: <152904663333.10587.10934053155404014785.stgit@noble>
References: <152904663333.10587.10934053155404014785.stgit@noble>
Message-ID: <152904669046.10587.7089479780701336009.stgit@noble>

Just open-code the access required.  This makes
the code clearer.
Also replace the 'memcpy' which violates typing
with a more correct for-loop.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 .../staging/lustre/lustre/include/lustre_compat.h  |    3 ---
 drivers/staging/lustre/lustre/ptlrpc/sec.c         |   15 ++++++++++-----
 2 files changed, 10 insertions(+), 8 deletions(-)

diff --git a/drivers/staging/lustre/lustre/include/lustre_compat.h b/drivers/staging/lustre/lustre/include/lustre_compat.h
index bf86b0612b50..481eb4f31cfa 100644
--- a/drivers/staging/lustre/lustre/include/lustre_compat.h
+++ b/drivers/staging/lustre/lustre/include/lustre_compat.h
@@ -39,9 +39,6 @@
 #include <linux/cred.h>
 #include <linux/module.h>
 
-#define current_ngroups current_cred()->group_info->ngroups
-#define current_groups current_cred()->group_info->small_block
-
 /*
  * OBD need working random driver, thus all our
  * initialization routines must be called after device
diff --git a/drivers/staging/lustre/lustre/ptlrpc/sec.c b/drivers/staging/lustre/lustre/ptlrpc/sec.c
index e193f3346e6f..9b60292370a7 100644
--- a/drivers/staging/lustre/lustre/ptlrpc/sec.c
+++ b/drivers/staging/lustre/lustre/ptlrpc/sec.c
@@ -2215,7 +2215,7 @@ int sptlrpc_current_user_desc_size(void)
 {
 	int ngroups;
 
-	ngroups = current_ngroups;
+	ngroups = current_cred()->group_info->ngroups;
 
 	if (ngroups > LUSTRE_MAX_GROUPS)
 		ngroups = LUSTRE_MAX_GROUPS;
@@ -2226,6 +2226,9 @@ EXPORT_SYMBOL(sptlrpc_current_user_desc_size);
 int sptlrpc_pack_user_desc(struct lustre_msg *msg, int offset)
 {
 	struct ptlrpc_user_desc *pud;
+	int ngroups;
+	kgid_t *gid;
+	int g;
 
 	pud = lustre_msg_buf(msg, offset, 0);
 
@@ -2240,10 +2243,12 @@ int sptlrpc_pack_user_desc(struct lustre_msg *msg, int offset)
 	pud->pud_ngroups = (msg->lm_buflens[offset] - sizeof(*pud)) / 4;
 
 	task_lock(current);
-	if (pud->pud_ngroups > current_ngroups)
-		pud->pud_ngroups = current_ngroups;
-	memcpy(pud->pud_groups, current_cred()->group_info->gid,
-	       pud->pud_ngroups * sizeof(__u32));
+	ngroups = current_cred()->group_info->ngroups;
+	gid = current_cred()->group_info->gid;
+	if (pud->pud_ngroups > ngroups)
+		pud->pud_ngroups = ngroups;
+	for (g = 0; g < pud->pud_ngroups; g++)
+		pud->pud_groups[g] = from_kgid(&init_user_ns, gid[g]);
 	task_unlock(current);
 
 	return 0;



From neilb at suse.com  Fri Jun 15 07:11:30 2018
From: neilb at suse.com (NeilBrown)
Date: Fri, 15 Jun 2018 17:11:30 +1000
Subject: [lustre-devel] [PATCH 12/24] lustre: discard LTIME_S macro
In-Reply-To: <152904663333.10587.10934053155404014785.stgit@noble>
References: <152904663333.10587.10934053155404014785.stgit@noble>
Message-ID: <152904669049.10587.5561340259770955128.stgit@noble>

Rather than using a macro, just access the required
field directly.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 .../staging/lustre/lustre/include/lustre_compat.h  |    2 -
 drivers/staging/lustre/lustre/llite/file.c         |   24 +++++++++--------
 drivers/staging/lustre/lustre/llite/lcommon_cl.c   |    6 ++--
 drivers/staging/lustre/lustre/llite/llite_lib.c    |   28 ++++++++++----------
 drivers/staging/lustre/lustre/llite/namei.c        |   16 ++++++-----
 drivers/staging/lustre/lustre/llite/vvp_object.c   |    6 ++--
 drivers/staging/lustre/lustre/lmv/lmv_intent.c     |    6 ++--
 drivers/staging/lustre/lustre/lmv/lmv_obd.c        |   16 ++++++-----
 drivers/staging/lustre/lustre/mdc/mdc_lib.c        |    6 ++--
 drivers/staging/lustre/lustre/mdc/mdc_reint.c      |    4 +--
 drivers/staging/lustre/lustre/obdclass/obdo.c      |   10 ++++---
 11 files changed, 61 insertions(+), 63 deletions(-)

diff --git a/drivers/staging/lustre/lustre/include/lustre_compat.h b/drivers/staging/lustre/lustre/include/lustre_compat.h
index 481eb4f31cfa..5e412fa8efbd 100644
--- a/drivers/staging/lustre/lustre/include/lustre_compat.h
+++ b/drivers/staging/lustre/lustre/include/lustre_compat.h
@@ -49,8 +49,6 @@
 #define module_init(a)     late_initcall(a)
 #endif
 
-#define LTIME_S(time)		   (time.tv_sec)
-
 #ifndef QUOTA_OK
 # define QUOTA_OK 0
 #endif
diff --git a/drivers/staging/lustre/lustre/llite/file.c b/drivers/staging/lustre/lustre/llite/file.c
index e642996db53f..59b5fbc75a51 100644
--- a/drivers/staging/lustre/lustre/llite/file.c
+++ b/drivers/staging/lustre/lustre/llite/file.c
@@ -956,13 +956,13 @@ int ll_merge_attr(const struct lu_env *env, struct inode *inode)
 	/* merge timestamps the most recently obtained from mds with
 	 * timestamps obtained from osts
 	 */
-	LTIME_S(inode->i_atime) = lli->lli_atime;
-	LTIME_S(inode->i_mtime) = lli->lli_mtime;
-	LTIME_S(inode->i_ctime) = lli->lli_ctime;
+	inode->i_atime.tv_sec = lli->lli_atime;
+	inode->i_mtime.tv_sec = lli->lli_mtime;
+	inode->i_ctime.tv_sec = lli->lli_ctime;
 
-	mtime = LTIME_S(inode->i_mtime);
-	atime = LTIME_S(inode->i_atime);
-	ctime = LTIME_S(inode->i_ctime);
+	mtime = inode->i_mtime.tv_sec;
+	atime = inode->i_atime.tv_sec;
+	ctime = inode->i_ctime.tv_sec;
 
 	cl_object_attr_lock(obj);
 	rc = cl_object_attr_get(env, obj, attr);
@@ -987,9 +987,9 @@ int ll_merge_attr(const struct lu_env *env, struct inode *inode)
 
 	inode->i_blocks = attr->cat_blocks;
 
-	LTIME_S(inode->i_mtime) = mtime;
-	LTIME_S(inode->i_atime) = atime;
-	LTIME_S(inode->i_ctime) = ctime;
+	inode->i_mtime.tv_sec = mtime;
+	inode->i_atime.tv_sec = atime;
+	inode->i_ctime.tv_sec = ctime;
 
 out_size_unlock:
 	ll_inode_size_unlock(inode);
@@ -2933,9 +2933,9 @@ static int ll_inode_revalidate(struct dentry *dentry, __u64 ibits)
 				return rc;
 		}
 
-		LTIME_S(inode->i_atime) = ll_i2info(inode)->lli_atime;
-		LTIME_S(inode->i_mtime) = ll_i2info(inode)->lli_mtime;
-		LTIME_S(inode->i_ctime) = ll_i2info(inode)->lli_ctime;
+		inode->i_atime.tv_sec = ll_i2info(inode)->lli_atime;
+		inode->i_mtime.tv_sec = ll_i2info(inode)->lli_mtime;
+		inode->i_ctime.tv_sec = ll_i2info(inode)->lli_ctime;
 	} else {
 		struct ll_inode_info *lli = ll_i2info(inode);
 
diff --git a/drivers/staging/lustre/lustre/llite/lcommon_cl.c b/drivers/staging/lustre/lustre/llite/lcommon_cl.c
index 9d1f3881f860..6c9fe498dbc3 100644
--- a/drivers/staging/lustre/lustre/llite/lcommon_cl.c
+++ b/drivers/staging/lustre/lustre/llite/lcommon_cl.c
@@ -95,9 +95,9 @@ int cl_setattr_ost(struct cl_object *obj, const struct iattr *attr,
 	io->ci_obj = obj;
 	io->ci_verify_layout = 1;
 
-	io->u.ci_setattr.sa_attr.lvb_atime = LTIME_S(attr->ia_atime);
-	io->u.ci_setattr.sa_attr.lvb_mtime = LTIME_S(attr->ia_mtime);
-	io->u.ci_setattr.sa_attr.lvb_ctime = LTIME_S(attr->ia_ctime);
+	io->u.ci_setattr.sa_attr.lvb_atime = attr->ia_atime.tv_sec;
+	io->u.ci_setattr.sa_attr.lvb_mtime = attr->ia_mtime.tv_sec;
+	io->u.ci_setattr.sa_attr.lvb_ctime = attr->ia_ctime.tv_sec;
 	io->u.ci_setattr.sa_attr.lvb_size = attr->ia_size;
 	io->u.ci_setattr.sa_attr_flags = attr_flags;
 	io->u.ci_setattr.sa_avalid = attr->ia_valid;
diff --git a/drivers/staging/lustre/lustre/llite/llite_lib.c b/drivers/staging/lustre/lustre/llite/llite_lib.c
index cd0f2517a3b3..d72674873a70 100644
--- a/drivers/staging/lustre/lustre/llite/llite_lib.c
+++ b/drivers/staging/lustre/lustre/llite/llite_lib.c
@@ -1108,9 +1108,9 @@ static struct inode *ll_iget_anon_dir(struct super_block *sb,
 		LASSERTF(S_ISDIR(inode->i_mode), "Not slave inode " DFID "\n",
 			 PFID(fid));
 
-		LTIME_S(inode->i_mtime) = 0;
-		LTIME_S(inode->i_atime) = 0;
-		LTIME_S(inode->i_ctime) = 0;
+		inode->i_mtime.tv_sec = 0;
+		inode->i_atime.tv_sec = 0;
+		inode->i_ctime.tv_sec = 0;
 		inode->i_rdev = 0;
 
 		inode->i_op = &ll_dir_inode_operations;
@@ -1486,7 +1486,7 @@ int ll_setattr_raw(struct dentry *dentry, struct iattr *attr,
 
 	if (attr->ia_valid & (ATTR_MTIME | ATTR_CTIME))
 		CDEBUG(D_INODE, "setting mtime %lu, ctime %lu, now = %llu\n",
-		       LTIME_S(attr->ia_mtime), LTIME_S(attr->ia_ctime),
+		       attr->ia_mtime.tv_sec, attr->ia_ctime.tv_sec,
 		       (s64)ktime_get_real_seconds());
 
 	if (S_ISREG(inode->i_mode))
@@ -1759,23 +1759,23 @@ int ll_update_inode(struct inode *inode, struct lustre_md *md)
 	inode->i_generation = cl_fid_build_gen(&body->mbo_fid1);
 
 	if (body->mbo_valid & OBD_MD_FLATIME) {
-		if (body->mbo_atime > LTIME_S(inode->i_atime))
-			LTIME_S(inode->i_atime) = body->mbo_atime;
+		if (body->mbo_atime > inode->i_atime.tv_sec)
+			inode->i_atime.tv_sec = body->mbo_atime;
 		lli->lli_atime = body->mbo_atime;
 	}
 	if (body->mbo_valid & OBD_MD_FLMTIME) {
-		if (body->mbo_mtime > LTIME_S(inode->i_mtime)) {
+		if (body->mbo_mtime > inode->i_mtime.tv_sec) {
 			CDEBUG(D_INODE,
 			       "setting ino %lu mtime from %lu to %llu\n",
-			       inode->i_ino, LTIME_S(inode->i_mtime),
+			       inode->i_ino, inode->i_mtime.tv_sec,
 			       body->mbo_mtime);
-			LTIME_S(inode->i_mtime) = body->mbo_mtime;
+			inode->i_mtime.tv_sec = body->mbo_mtime;
 		}
 		lli->lli_mtime = body->mbo_mtime;
 	}
 	if (body->mbo_valid & OBD_MD_FLCTIME) {
-		if (body->mbo_ctime > LTIME_S(inode->i_ctime))
-			LTIME_S(inode->i_ctime) = body->mbo_ctime;
+		if (body->mbo_ctime > inode->i_ctime.tv_sec)
+			inode->i_ctime.tv_sec = body->mbo_ctime;
 		lli->lli_ctime = body->mbo_ctime;
 	}
 	if (body->mbo_valid & OBD_MD_FLMODE)
@@ -1848,9 +1848,9 @@ int ll_read_inode2(struct inode *inode, void *opaque)
 	 * it ourselves.  They will be overwritten by either MDS or OST
 	 * attributes - we just need to make sure they aren't newer.
 	 */
-	LTIME_S(inode->i_mtime) = 0;
-	LTIME_S(inode->i_atime) = 0;
-	LTIME_S(inode->i_ctime) = 0;
+	inode->i_mtime.tv_sec = 0;
+	inode->i_atime.tv_sec = 0;
+	inode->i_ctime.tv_sec = 0;
 	inode->i_rdev = 0;
 	rc = ll_update_inode(inode, md);
 	if (rc)
diff --git a/drivers/staging/lustre/lustre/llite/namei.c b/drivers/staging/lustre/lustre/llite/namei.c
index d5f6d20afe8c..eb9dc532b353 100644
--- a/drivers/staging/lustre/lustre/llite/namei.c
+++ b/drivers/staging/lustre/lustre/llite/namei.c
@@ -266,9 +266,9 @@ int ll_md_blocking_ast(struct ldlm_lock *lock, struct ldlm_lock_desc *desc,
 			struct ll_inode_info *lli = ll_i2info(inode);
 
 			spin_lock(&lli->lli_lock);
-			LTIME_S(inode->i_mtime) = 0;
-			LTIME_S(inode->i_atime) = 0;
-			LTIME_S(inode->i_ctime) = 0;
+			inode->i_mtime.tv_sec = 0;
+			inode->i_atime.tv_sec = 0;
+			inode->i_ctime.tv_sec = 0;
 			spin_unlock(&lli->lli_lock);
 		}
 
@@ -848,15 +848,15 @@ void ll_update_times(struct ptlrpc_request *request, struct inode *inode)
 
 	LASSERT(body);
 	if (body->mbo_valid & OBD_MD_FLMTIME &&
-	    body->mbo_mtime > LTIME_S(inode->i_mtime)) {
+	    body->mbo_mtime > inode->i_mtime.tv_sec) {
 		CDEBUG(D_INODE, "setting fid " DFID " mtime from %lu to %llu\n",
-		       PFID(ll_inode2fid(inode)), LTIME_S(inode->i_mtime),
+		       PFID(ll_inode2fid(inode)), inode->i_mtime.tv_sec,
 		       body->mbo_mtime);
-		LTIME_S(inode->i_mtime) = body->mbo_mtime;
+		inode->i_mtime.tv_sec = body->mbo_mtime;
 	}
 	if (body->mbo_valid & OBD_MD_FLCTIME &&
-	    body->mbo_ctime > LTIME_S(inode->i_ctime))
-		LTIME_S(inode->i_ctime) = body->mbo_ctime;
+	    body->mbo_ctime > inode->i_ctime.tv_sec)
+		inode->i_ctime.tv_sec = body->mbo_ctime;
 }
 
 static int ll_new_node(struct inode *dir, struct dentry *dentry,
diff --git a/drivers/staging/lustre/lustre/llite/vvp_object.c b/drivers/staging/lustre/lustre/llite/vvp_object.c
index b2cb51c8f7f4..301a90d6e5f5 100644
--- a/drivers/staging/lustre/lustre/llite/vvp_object.c
+++ b/drivers/staging/lustre/lustre/llite/vvp_object.c
@@ -170,9 +170,9 @@ static int vvp_object_glimpse(const struct lu_env *env,
 {
 	struct inode *inode = vvp_object_inode(obj);
 
-	lvb->lvb_mtime = LTIME_S(inode->i_mtime);
-	lvb->lvb_atime = LTIME_S(inode->i_atime);
-	lvb->lvb_ctime = LTIME_S(inode->i_ctime);
+	lvb->lvb_mtime = inode->i_mtime.tv_sec;
+	lvb->lvb_atime = inode->i_atime.tv_sec;
+	lvb->lvb_ctime = inode->i_ctime.tv_sec;
 	/*
 	 * LU-417: Add dirty pages block count lest i_blocks reports 0, some
 	 * "cp" or "tar" on remote node may think it's a completely sparse file
diff --git a/drivers/staging/lustre/lustre/lmv/lmv_intent.c b/drivers/staging/lustre/lustre/lmv/lmv_intent.c
index 1e850fdbc623..ba6410e800a7 100644
--- a/drivers/staging/lustre/lustre/lmv/lmv_intent.c
+++ b/drivers/staging/lustre/lustre/lmv/lmv_intent.c
@@ -233,9 +233,9 @@ int lmv_revalidate_slaves(struct obd_export *exp,
 			i_size_write(inode, body->mbo_size);
 			inode->i_blocks = body->mbo_blocks;
 			set_nlink(inode, body->mbo_nlink);
-			LTIME_S(inode->i_atime) = body->mbo_atime;
-			LTIME_S(inode->i_ctime) = body->mbo_ctime;
-			LTIME_S(inode->i_mtime) = body->mbo_mtime;
+			inode->i_atime.tv_sec = body->mbo_atime;
+			inode->i_ctime.tv_sec = body->mbo_ctime;
+			inode->i_mtime.tv_sec = body->mbo_mtime;
 		}
 
 		md_set_lock_data(tgt->ltd_exp, lockh, inode, NULL);
diff --git a/drivers/staging/lustre/lustre/lmv/lmv_obd.c b/drivers/staging/lustre/lustre/lmv/lmv_obd.c
index 9f427a7c7b03..58c9a8d503ec 100644
--- a/drivers/staging/lustre/lustre/lmv/lmv_obd.c
+++ b/drivers/staging/lustre/lustre/lmv/lmv_obd.c
@@ -3030,8 +3030,8 @@ static int lmv_merge_attr(struct obd_export *exp,
 		CDEBUG(D_INFO, "" DFID " size %llu, blocks %llu nlink %u, atime %lu ctime %lu, mtime %lu.\n",
 		       PFID(&lsm->lsm_md_oinfo[i].lmo_fid),
 		       i_size_read(inode), (unsigned long long)inode->i_blocks,
-		       inode->i_nlink, LTIME_S(inode->i_atime),
-		       LTIME_S(inode->i_ctime), LTIME_S(inode->i_mtime));
+		       inode->i_nlink, inode->i_atime.tv_sec,
+		       inode->i_ctime.tv_sec, inode->i_mtime.tv_sec);
 
 		/* for slave stripe, it needs to subtract nlink for . and .. */
 		if (i)
@@ -3042,14 +3042,14 @@ static int lmv_merge_attr(struct obd_export *exp,
 		attr->cat_size += i_size_read(inode);
 		attr->cat_blocks += inode->i_blocks;
 
-		if (attr->cat_atime < LTIME_S(inode->i_atime))
-			attr->cat_atime = LTIME_S(inode->i_atime);
+		if (attr->cat_atime < inode->i_atime.tv_sec)
+			attr->cat_atime = inode->i_atime.tv_sec;
 
-		if (attr->cat_ctime < LTIME_S(inode->i_ctime))
-			attr->cat_ctime = LTIME_S(inode->i_ctime);
+		if (attr->cat_ctime < inode->i_ctime.tv_sec)
+			attr->cat_ctime = inode->i_ctime.tv_sec;
 
-		if (attr->cat_mtime < LTIME_S(inode->i_mtime))
-			attr->cat_mtime = LTIME_S(inode->i_mtime);
+		if (attr->cat_mtime < inode->i_mtime.tv_sec)
+			attr->cat_mtime = inode->i_mtime.tv_sec;
 	}
 	return 0;
 }
diff --git a/drivers/staging/lustre/lustre/mdc/mdc_lib.c b/drivers/staging/lustre/lustre/mdc/mdc_lib.c
index ad7c13be5e3c..9cb4d242d7de 100644
--- a/drivers/staging/lustre/lustre/mdc/mdc_lib.c
+++ b/drivers/staging/lustre/lustre/mdc/mdc_lib.c
@@ -293,9 +293,9 @@ static void mdc_setattr_pack_rec(struct mdt_rec_setattr *rec,
 	rec->sa_gid    = from_kgid(&init_user_ns, op_data->op_attr.ia_gid);
 	rec->sa_size   = op_data->op_attr.ia_size;
 	rec->sa_blocks = op_data->op_attr_blocks;
-	rec->sa_atime  = LTIME_S(op_data->op_attr.ia_atime);
-	rec->sa_mtime  = LTIME_S(op_data->op_attr.ia_mtime);
-	rec->sa_ctime  = LTIME_S(op_data->op_attr.ia_ctime);
+	rec->sa_atime  = op_data->op_attr.ia_atime.tv_sec;
+	rec->sa_mtime  = op_data->op_attr.ia_mtime.tv_sec;
+	rec->sa_ctime  = op_data->op_attr.ia_ctime.tv_sec;
 	rec->sa_attr_flags = op_data->op_attr_flags;
 	if ((op_data->op_attr.ia_valid & ATTR_GID) &&
 	    in_group_p(op_data->op_attr.ia_gid))
diff --git a/drivers/staging/lustre/lustre/mdc/mdc_reint.c b/drivers/staging/lustre/lustre/mdc/mdc_reint.c
index e77c00df0693..b014eb235ce1 100644
--- a/drivers/staging/lustre/lustre/mdc/mdc_reint.c
+++ b/drivers/staging/lustre/lustre/mdc/mdc_reint.c
@@ -130,8 +130,8 @@ int mdc_setattr(struct obd_export *exp, struct md_op_data *op_data,
 
 	if (op_data->op_attr.ia_valid & (ATTR_MTIME | ATTR_CTIME))
 		CDEBUG(D_INODE, "setting mtime %ld, ctime %ld\n",
-		       LTIME_S(op_data->op_attr.ia_mtime),
-		       LTIME_S(op_data->op_attr.ia_ctime));
+		       op_data->op_attr.ia_mtime.tv_sec,
+		       op_data->op_attr.ia_ctime.tv_sec);
 	mdc_setattr_pack(req, op_data, ea, ealen);
 
 	req_capsule_set_size(&req->rq_pill, &RMF_ACL, RCL_SERVER,
diff --git a/drivers/staging/lustre/lustre/obdclass/obdo.c b/drivers/staging/lustre/lustre/obdclass/obdo.c
index c4503bc36591..3bc8ca8363de 100644
--- a/drivers/staging/lustre/lustre/obdclass/obdo.c
+++ b/drivers/staging/lustre/lustre/obdclass/obdo.c
@@ -61,19 +61,19 @@ void obdo_from_inode(struct obdo *dst, struct inode *src, u32 valid)
 
 	if (valid & (OBD_MD_FLCTIME | OBD_MD_FLMTIME))
 		CDEBUG(D_INODE, "valid %x, new time %lu/%lu\n",
-		       valid, LTIME_S(src->i_mtime),
-		       LTIME_S(src->i_ctime));
+		       valid, src->i_mtime.tv_sec,
+		       src->i_ctime.tv_sec);
 
 	if (valid & OBD_MD_FLATIME) {
-		dst->o_atime = LTIME_S(src->i_atime);
+		dst->o_atime = src->i_atime.tv_sec;
 		newvalid |= OBD_MD_FLATIME;
 	}
 	if (valid & OBD_MD_FLMTIME) {
-		dst->o_mtime = LTIME_S(src->i_mtime);
+		dst->o_mtime = src->i_mtime.tv_sec;
 		newvalid |= OBD_MD_FLMTIME;
 	}
 	if (valid & OBD_MD_FLCTIME) {
-		dst->o_ctime = LTIME_S(src->i_ctime);
+		dst->o_ctime = src->i_ctime.tv_sec;
 		newvalid |= OBD_MD_FLCTIME;
 	}
 	if (valid & OBD_MD_FLSIZE) {



From neilb at suse.com  Fri Jun 15 07:11:30 2018
From: neilb at suse.com (NeilBrown)
Date: Fri, 15 Jun 2018 17:11:30 +1000
Subject: [lustre-devel] [PATCH 13/24] lustre: discard NO_QUOTA and QUOTA_OK
In-Reply-To: <152904663333.10587.10934053155404014785.stgit@noble>
References: <152904663333.10587.10934053155404014785.stgit@noble>
Message-ID: <152904669053.10587.9299734790900311212.stgit@noble>

They don't help readability.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 .../staging/lustre/lustre/include/lustre_compat.h  |    7 -------
 drivers/staging/lustre/lustre/osc/osc_cache.c      |    4 ++--
 drivers/staging/lustre/lustre/osc/osc_quota.c      |    4 ++--
 3 files changed, 4 insertions(+), 11 deletions(-)

diff --git a/drivers/staging/lustre/lustre/include/lustre_compat.h b/drivers/staging/lustre/lustre/include/lustre_compat.h
index 5e412fa8efbd..cb6e2aa35960 100644
--- a/drivers/staging/lustre/lustre/include/lustre_compat.h
+++ b/drivers/staging/lustre/lustre/include/lustre_compat.h
@@ -49,13 +49,6 @@
 #define module_init(a)     late_initcall(a)
 #endif
 
-#ifndef QUOTA_OK
-# define QUOTA_OK 0
-#endif
-#ifndef NO_QUOTA
-# define NO_QUOTA (-EDQUOT)
-#endif
-
 #if !defined(_ASM_GENERIC_BITOPS_EXT2_NON_ATOMIC_H_) && !defined(ext2_set_bit)
 # define ext2_set_bit	     __test_and_set_bit_le
 # define ext2_clear_bit	   __test_and_clear_bit_le
diff --git a/drivers/staging/lustre/lustre/osc/osc_cache.c b/drivers/staging/lustre/lustre/osc/osc_cache.c
index f26983004843..99de672fd078 100644
--- a/drivers/staging/lustre/lustre/osc/osc_cache.c
+++ b/drivers/staging/lustre/lustre/osc/osc_cache.c
@@ -2404,8 +2404,8 @@ int osc_queue_async_io(const struct lu_env *env, struct cl_io *io,
 
 		qid[USRQUOTA] = attr->cat_uid;
 		qid[GRPQUOTA] = attr->cat_gid;
-		if (rc == 0 && osc_quota_chkdq(cli, qid) == NO_QUOTA)
-			rc = -EDQUOT;
+		if (rc == 0)
+			rc = osc_quota_chkdq(cli, qid);
 		if (rc)
 			return rc;
 	}
diff --git a/drivers/staging/lustre/lustre/osc/osc_quota.c b/drivers/staging/lustre/lustre/osc/osc_quota.c
index 723ec2fb18bf..25d06dec949e 100644
--- a/drivers/staging/lustre/lustre/osc/osc_quota.c
+++ b/drivers/staging/lustre/lustre/osc/osc_quota.c
@@ -64,11 +64,11 @@ int osc_quota_chkdq(struct client_obd *cli, const unsigned int qid[])
 			 */
 			CDEBUG(D_QUOTA, "chkdq found noquota for %s %d\n",
 			       type == USRQUOTA ? "user" : "grout", qid[type]);
-			return NO_QUOTA;
+			return -EDQUOT;
 		}
 	}
 
-	return QUOTA_OK;
+	return 0;
 }
 
 static void osc_quota_free(struct rcu_head *head)



From neilb at suse.com  Fri Jun 15 07:11:30 2018
From: neilb at suse.com (NeilBrown)
Date: Fri, 15 Jun 2018 17:11:30 +1000
Subject: [lustre-devel] [PATCH 14/24] lustre: discard ext2* bit operations
In-Reply-To: <152904663333.10587.10934053155404014785.stgit@noble>
References: <152904663333.10587.10934053155404014785.stgit@noble>
Message-ID: <152904669056.10587.1595912848911465438.stgit@noble>

Just use the relevant *_le bit operation directly.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 .../staging/lustre/lustre/include/lustre_compat.h  |    8 --------
 drivers/staging/lustre/lustre/obdclass/llog.c      |    6 +++---
 2 files changed, 3 insertions(+), 11 deletions(-)

diff --git a/drivers/staging/lustre/lustre/include/lustre_compat.h b/drivers/staging/lustre/lustre/include/lustre_compat.h
index cb6e2aa35960..534846f5561d 100644
--- a/drivers/staging/lustre/lustre/include/lustre_compat.h
+++ b/drivers/staging/lustre/lustre/include/lustre_compat.h
@@ -49,14 +49,6 @@
 #define module_init(a)     late_initcall(a)
 #endif
 
-#if !defined(_ASM_GENERIC_BITOPS_EXT2_NON_ATOMIC_H_) && !defined(ext2_set_bit)
-# define ext2_set_bit	     __test_and_set_bit_le
-# define ext2_clear_bit	   __test_and_clear_bit_le
-# define ext2_test_bit	    test_bit_le
-# define ext2_find_first_zero_bit find_first_zero_bit_le
-# define ext2_find_next_zero_bit  find_next_zero_bit_le
-#endif
-
 #define TIMES_SET_FLAGS (ATTR_MTIME_SET | ATTR_ATIME_SET | ATTR_TIMES_SET)
 
 #endif /* _LUSTRE_COMPAT_H */
diff --git a/drivers/staging/lustre/lustre/obdclass/llog.c b/drivers/staging/lustre/lustre/obdclass/llog.c
index bba84eae1e19..53e723d990e4 100644
--- a/drivers/staging/lustre/lustre/obdclass/llog.c
+++ b/drivers/staging/lustre/lustre/obdclass/llog.c
@@ -137,7 +137,7 @@ static int llog_read_header(const struct lu_env *env,
 		 */
 		len = llh->llh_hdr.lrh_len - llh->llh_bitmap_offset;
 		memset(LLOG_HDR_BITMAP(llh), 0, len - sizeof(llh->llh_tail));
-		ext2_set_bit(0, LLOG_HDR_BITMAP(llh));
+		set_bit_le(0, LLOG_HDR_BITMAP(llh));
 		LLOG_HDR_TAIL(llh)->lrt_len = llh->llh_hdr.lrh_len;
 		LLOG_HDR_TAIL(llh)->lrt_index = llh->llh_hdr.lrh_index;
 		rc = 0;
@@ -264,7 +264,7 @@ static int llog_process_thread(void *arg)
 
 		/* skip records not set in bitmap */
 		while (index <= last_index &&
-		       !ext2_test_bit(index, LLOG_HDR_BITMAP(llh)))
+		       !test_bit_le(index, LLOG_HDR_BITMAP(llh)))
 			++index;
 
 		if (index > last_index)
@@ -365,7 +365,7 @@ static int llog_process_thread(void *arg)
 						    chunk_offset;
 
 			/* if set, process the callback on this record */
-			if (ext2_test_bit(index, LLOG_HDR_BITMAP(llh))) {
+			if (test_bit_le(index, LLOG_HDR_BITMAP(llh))) {
 				rc = lpi->lpi_cb(lpi->lpi_env, loghandle, rec,
 						 lpi->lpi_cbdata);
 				last_called_index = index;



From neilb at suse.com  Fri Jun 15 07:11:30 2018
From: neilb at suse.com (NeilBrown)
Date: Fri, 15 Jun 2018 17:11:30 +1000
Subject: [lustre-devel] [PATCH 15/24] lustre: centralize TIMES_SET_FLAGS
In-Reply-To: <152904663333.10587.10934053155404014785.stgit@noble>
References: <152904663333.10587.10934053155404014785.stgit@noble>
Message-ID: <152904669060.10587.11467294807244836929.stgit@noble>

This macro is only used in llite, so move both definitions to
llite_internal.h.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 .../staging/lustre/lustre/include/lustre_compat.h  |    2 --
 .../staging/lustre/lustre/llite/llite_internal.h   |    2 ++
 drivers/staging/lustre/lustre/llite/llite_lib.c    |    2 --
 3 files changed, 2 insertions(+), 4 deletions(-)

diff --git a/drivers/staging/lustre/lustre/include/lustre_compat.h b/drivers/staging/lustre/lustre/include/lustre_compat.h
index 534846f5561d..6d659b696b5f 100644
--- a/drivers/staging/lustre/lustre/include/lustre_compat.h
+++ b/drivers/staging/lustre/lustre/include/lustre_compat.h
@@ -49,6 +49,4 @@
 #define module_init(a)     late_initcall(a)
 #endif
 
-#define TIMES_SET_FLAGS (ATTR_MTIME_SET | ATTR_ATIME_SET | ATTR_TIMES_SET)
-
 #endif /* _LUSTRE_COMPAT_H */
diff --git a/drivers/staging/lustre/lustre/llite/llite_internal.h b/drivers/staging/lustre/lustre/llite/llite_internal.h
index 0ae5185a4998..91212e608c0b 100644
--- a/drivers/staging/lustre/lustre/llite/llite_internal.h
+++ b/drivers/staging/lustre/lustre/llite/llite_internal.h
@@ -69,6 +69,8 @@
 #define LL_IT2STR(it) ((it) ? ldlm_it2str((it)->it_op) : "0")
 #define LUSTRE_FPRIVATE(file) ((file)->private_data)
 
+#define TIMES_SET_FLAGS (ATTR_MTIME_SET | ATTR_ATIME_SET | ATTR_TIMES_SET)
+
 struct ll_dentry_data {
 	struct lookup_intent		*lld_it;
 	unsigned int			lld_sa_generation;
diff --git a/drivers/staging/lustre/lustre/llite/llite_lib.c b/drivers/staging/lustre/lustre/llite/llite_lib.c
index d72674873a70..e6e02fe7703b 100644
--- a/drivers/staging/lustre/lustre/llite/llite_lib.c
+++ b/drivers/staging/lustre/lustre/llite/llite_lib.c
@@ -1353,8 +1353,6 @@ void ll_clear_inode(struct inode *inode)
 	cl_inode_fini(inode);
 }
 
-#define TIMES_SET_FLAGS (ATTR_MTIME_SET | ATTR_ATIME_SET | ATTR_TIMES_SET)
-
 static int ll_md_setattr(struct dentry *dentry, struct md_op_data *op_data)
 {
 	struct lustre_md md;



From neilb at suse.com  Fri Jun 15 07:11:30 2018
From: neilb at suse.com (NeilBrown)
Date: Fri, 15 Jun 2018 17:11:30 +1000
Subject: [lustre-devel] [PATCH 16/24] lustre: remove redefinition of
 module_init().
In-Reply-To: <152904663333.10587.10934053155404014785.stgit@noble>
References: <152904663333.10587.10934053155404014785.stgit@noble>
Message-ID: <152904669064.10587.7203081913142739014.stgit@noble>

As lustre has only recently support building
as non-modules, this code must be very old.
If there is a dependency ordering, this is not
the way to handle it - random numbers should
be generated when filesystem is activated, not
when the module is loaded.
So just remove this code.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 .../staging/lustre/lustre/include/lustre_compat.h  |   10 ----------
 1 file changed, 10 deletions(-)

diff --git a/drivers/staging/lustre/lustre/include/lustre_compat.h b/drivers/staging/lustre/lustre/include/lustre_compat.h
index 6d659b696b5f..f555391d5ae3 100644
--- a/drivers/staging/lustre/lustre/include/lustre_compat.h
+++ b/drivers/staging/lustre/lustre/include/lustre_compat.h
@@ -39,14 +39,4 @@
 #include <linux/cred.h>
 #include <linux/module.h>
 
-/*
- * OBD need working random driver, thus all our
- * initialization routines must be called after device
- * driver initialization
- */
-#ifndef MODULE
-#undef module_init
-#define module_init(a)     late_initcall(a)
-#endif
-
 #endif /* _LUSTRE_COMPAT_H */



From neilb at suse.com  Fri Jun 15 07:11:30 2018
From: neilb at suse.com (NeilBrown)
Date: Fri, 15 Jun 2018 17:11:30 +1000
Subject: [lustre-devel] [PATCH 17/24] lustre: remove lustre_compat.h
In-Reply-To: <152904663333.10587.10934053155404014785.stgit@noble>
References: <152904663333.10587.10934053155404014785.stgit@noble>
Message-ID: <152904669067.10587.14431331340197955105.stgit@noble>

Only thing it provided was one include file,
so include that directly where needed.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 drivers/staging/lustre/lustre/include/cl_object.h  |    1 
 .../staging/lustre/lustre/include/lustre_compat.h  |   42 --------------------
 .../staging/lustre/lustre/include/obd_support.h    |    1 
 .../staging/lustre/lustre/llite/llite_internal.h   |    1 
 drivers/staging/lustre/lustre/llite/llite_lib.c    |    1 
 drivers/staging/lustre/lustre/obdclass/llog.c      |    1 
 drivers/staging/lustre/lustre/obdclass/obd_mount.c |    1 
 drivers/staging/lustre/lustre/ptlrpc/import.c      |    1 
 drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c     |    1 
 drivers/staging/lustre/lustre/ptlrpc/service.c     |    1 
 10 files changed, 5 insertions(+), 46 deletions(-)
 delete mode 100644 drivers/staging/lustre/lustre/include/lustre_compat.h

diff --git a/drivers/staging/lustre/lustre/include/cl_object.h b/drivers/staging/lustre/lustre/include/cl_object.h
index caa66763c70d..1491beb2b9a9 100644
--- a/drivers/staging/lustre/lustre/include/cl_object.h
+++ b/drivers/staging/lustre/lustre/include/cl_object.h
@@ -90,7 +90,6 @@
  * super-class definitions.
  */
 #include <lu_object.h>
-#include <lustre_compat.h>
 #include <linux/atomic.h>
 #include <linux/mutex.h>
 #include <linux/radix-tree.h>
diff --git a/drivers/staging/lustre/lustre/include/lustre_compat.h b/drivers/staging/lustre/lustre/include/lustre_compat.h
deleted file mode 100644
index f555391d5ae3..000000000000
--- a/drivers/staging/lustre/lustre/include/lustre_compat.h
+++ /dev/null
@@ -1,42 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-/*
- * GPL HEADER START
- *
- * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 only,
- * as published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful, but
- * WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
- * General Public License version 2 for more details (a copy is included
- * in the LICENSE file that accompanied this code).
- *
- * You should have received a copy of the GNU General Public License
- * version 2 along with this program; If not, see
- * http://www.gnu.org/licenses/gpl-2.0.html
- *
- * GPL HEADER END
- */
-/*
- * Copyright (c) 2003, 2010, Oracle and/or its affiliates. All rights reserved.
- * Use is subject to license terms.
- *
- * Copyright (c) 2011, 2012, Intel Corporation.
- */
-/*
- * This file is part of Lustre, http://www.lustre.org/
- * Lustre is a trademark of Sun Microsystems, Inc.
- */
-
-#ifndef _LUSTRE_COMPAT_H
-#define _LUSTRE_COMPAT_H
-
-#include <linux/fs_struct.h>
-#include <linux/namei.h>
-#include <linux/cred.h>
-#include <linux/module.h>
-
-#endif /* _LUSTRE_COMPAT_H */
diff --git a/drivers/staging/lustre/lustre/include/obd_support.h b/drivers/staging/lustre/lustre/include/obd_support.h
index 9e41633823f7..070a281ebb7d 100644
--- a/drivers/staging/lustre/lustre/include/obd_support.h
+++ b/drivers/staging/lustre/lustre/include/obd_support.h
@@ -37,7 +37,6 @@
 #include <linux/slab.h>
 #include <linux/sched/signal.h>
 
-#include <lustre_compat.h>
 #include <lprocfs_status.h>
 
 /* global variables */
diff --git a/drivers/staging/lustre/lustre/llite/llite_internal.h b/drivers/staging/lustre/lustre/llite/llite_internal.h
index 91212e608c0b..539ed3b6b64a 100644
--- a/drivers/staging/lustre/lustre/llite/llite_internal.h
+++ b/drivers/staging/lustre/lustre/llite/llite_internal.h
@@ -39,7 +39,6 @@
 #include <lustre_linkea.h>
 
 /* for struct cl_lock_descr and struct cl_io */
-#include <lustre_compat.h>
 #include <cl_object.h>
 #include <lustre_lmv.h>
 #include <lustre_mdc.h>
diff --git a/drivers/staging/lustre/lustre/llite/llite_lib.c b/drivers/staging/lustre/lustre/llite/llite_lib.c
index e6e02fe7703b..a0a143bf4989 100644
--- a/drivers/staging/lustre/lustre/llite/llite_lib.c
+++ b/drivers/staging/lustre/lustre/llite/llite_lib.c
@@ -42,6 +42,7 @@
 #include <linux/types.h>
 #include <linux/mm.h>
 #include <linux/random.h>
+#include <linux/fs_struct.h>
 
 #include <uapi/linux/lustre/lustre_ioctl.h>
 #include <lustre_ha.h>
diff --git a/drivers/staging/lustre/lustre/obdclass/llog.c b/drivers/staging/lustre/lustre/obdclass/llog.c
index 53e723d990e4..f59f89a76c91 100644
--- a/drivers/staging/lustre/lustre/obdclass/llog.c
+++ b/drivers/staging/lustre/lustre/obdclass/llog.c
@@ -45,6 +45,7 @@
 #define DEBUG_SUBSYSTEM S_LOG
 
 #include <linux/kthread.h>
+#include <linux/fs_struct.h>
 #include <llog_swab.h>
 #include <lustre_log.h>
 #include <obd_class.h>
diff --git a/drivers/staging/lustre/lustre/obdclass/obd_mount.c b/drivers/staging/lustre/lustre/obdclass/obd_mount.c
index 06c38fdef7ba..232bbfa9242c 100644
--- a/drivers/staging/lustre/lustre/obdclass/obd_mount.c
+++ b/drivers/staging/lustre/lustre/obdclass/obd_mount.c
@@ -43,7 +43,6 @@
 
 #include <linux/random.h>
 #include <obd.h>
-#include <lustre_compat.h>
 #include <obd_class.h>
 #include <uapi/linux/lustre/lustre_idl.h>
 #include <lustre_log.h>
diff --git a/drivers/staging/lustre/lustre/ptlrpc/import.c b/drivers/staging/lustre/lustre/ptlrpc/import.c
index 1a0f35dfab97..54ceac56c524 100644
--- a/drivers/staging/lustre/lustre/ptlrpc/import.c
+++ b/drivers/staging/lustre/lustre/ptlrpc/import.c
@@ -38,6 +38,7 @@
 #define DEBUG_SUBSYSTEM S_RPC
 
 #include <linux/kthread.h>
+#include <linux/fs_struct.h>
 #include <obd_support.h>
 #include <lustre_ha.h>
 #include <lustre_net.h>
diff --git a/drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c b/drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c
index 531005411edf..66295b4fcdab 100644
--- a/drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c
+++ b/drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c
@@ -53,6 +53,7 @@
 #define DEBUG_SUBSYSTEM S_RPC
 
 #include <linux/kthread.h>
+#include <linux/fs_struct.h>
 #include <linux/libcfs/libcfs.h>
 #include <linux/libcfs/libcfs_cpu.h>
 #include <linux/libcfs/libcfs_string.h>
diff --git a/drivers/staging/lustre/lustre/ptlrpc/service.c b/drivers/staging/lustre/lustre/ptlrpc/service.c
index 3fd8c746f460..12442425702a 100644
--- a/drivers/staging/lustre/lustre/ptlrpc/service.c
+++ b/drivers/staging/lustre/lustre/ptlrpc/service.c
@@ -34,6 +34,7 @@
 #define DEBUG_SUBSYSTEM S_RPC
 
 #include <linux/kthread.h>
+#include <linux/fs_struct.h>
 #include <obd_support.h>
 #include <obd_class.h>
 #include <lustre_net.h>



From neilb at suse.com  Fri Jun 15 07:11:30 2018
From: neilb at suse.com (NeilBrown)
Date: Fri, 15 Jun 2018 17:11:30 +1000
Subject: [lustre-devel] [PATCH 18/24] lustre/lnet: move tracefile locking
 from linux-tracefile.c to tracefile.c
In-Reply-To: <152904663333.10587.10934053155404014785.stgit@noble>
References: <152904663333.10587.10934053155404014785.stgit@noble>
Message-ID: <152904669072.10587.15169492766245566638.stgit@noble>

There is no value in keeping it separate.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 .../staging/lustre/lnet/libcfs/linux-tracefile.c   |   22 ------------------
 drivers/staging/lustre/lnet/libcfs/tracefile.c     |   25 ++++++++++----------
 drivers/staging/lustre/lnet/libcfs/tracefile.h     |    5 ----
 3 files changed, 13 insertions(+), 39 deletions(-)

diff --git a/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c b/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
index 347138409eba..9e72220299af 100644
--- a/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
+++ b/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
@@ -47,8 +47,6 @@ static unsigned int pages_factor[CFS_TCD_TYPE_MAX] = {
 
 char *cfs_trace_console_buffers[NR_CPUS][CFS_TCD_TYPE_MAX];
 
-static DECLARE_RWSEM(cfs_tracefile_sem);
-
 int cfs_tracefile_init_arch(void)
 {
 	int i;
@@ -109,26 +107,6 @@ void cfs_tracefile_fini_arch(void)
 	}
 }
 
-void cfs_tracefile_read_lock(void)
-{
-	down_read(&cfs_tracefile_sem);
-}
-
-void cfs_tracefile_read_unlock(void)
-{
-	up_read(&cfs_tracefile_sem);
-}
-
-void cfs_tracefile_write_lock(void)
-{
-	down_write(&cfs_tracefile_sem);
-}
-
-void cfs_tracefile_write_unlock(void)
-{
-	up_write(&cfs_tracefile_sem);
-}
-
 enum cfs_trace_buf_type cfs_trace_buf_idx_get(void)
 {
 	if (in_irq())
diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.c b/drivers/staging/lustre/lnet/libcfs/tracefile.c
index 7ca562e156f0..5f319332f60b 100644
--- a/drivers/staging/lustre/lnet/libcfs/tracefile.c
+++ b/drivers/staging/lustre/lnet/libcfs/tracefile.c
@@ -59,6 +59,7 @@ static DEFINE_MUTEX(cfs_trace_thread_mutex);
 static int thread_running;
 
 static atomic_t cfs_tage_allocated = ATOMIC_INIT(0);
+static DECLARE_RWSEM(cfs_tracefile_sem);
 
 struct page_collection {
 	struct list_head	pc_pages;
@@ -711,7 +712,7 @@ int cfs_tracefile_dump_all_pages(char *filename)
 	mm_segment_t __oldfs;
 	int rc;
 
-	cfs_tracefile_write_lock();
+	down_write(&cfs_tracefile_sem);
 
 	filp = filp_open(filename, O_CREAT | O_EXCL | O_WRONLY | O_LARGEFILE,
 			 0600);
@@ -759,7 +760,7 @@ int cfs_tracefile_dump_all_pages(char *filename)
 close:
 	filp_close(filp, NULL);
 out:
-	cfs_tracefile_write_unlock();
+	up_write(&cfs_tracefile_sem);
 	return rc;
 }
 
@@ -873,12 +874,12 @@ int cfs_trace_daemon_command(char *str)
 {
 	int rc = 0;
 
-	cfs_tracefile_write_lock();
+	down_write(&cfs_tracefile_sem);
 
 	if (!strcmp(str, "stop")) {
-		cfs_tracefile_write_unlock();
+		up_write(&cfs_tracefile_sem);
 		cfs_trace_stop_thread();
-		cfs_tracefile_write_lock();
+		down_write(&cfs_tracefile_sem);
 		memset(cfs_tracefile, 0, sizeof(cfs_tracefile));
 
 	} else if (!strncmp(str, "size=", 5)) {
@@ -905,7 +906,7 @@ int cfs_trace_daemon_command(char *str)
 		cfs_trace_start_thread();
 	}
 
-	cfs_tracefile_write_unlock();
+	up_write(&cfs_tracefile_sem);
 	return rc;
 }
 
@@ -950,12 +951,12 @@ int cfs_trace_set_debug_mb(int mb)
 	mb /= num_possible_cpus();
 	pages = mb << (20 - PAGE_SHIFT);
 
-	cfs_tracefile_write_lock();
+	down_write(&cfs_tracefile_sem);
 
 	cfs_tcd_for_each(tcd, i, j)
 		tcd->tcd_max_pages = (pages * tcd->tcd_pages_factor) / 100;
 
-	cfs_tracefile_write_unlock();
+	up_write(&cfs_tracefile_sem);
 
 	return 0;
 }
@@ -967,12 +968,12 @@ int cfs_trace_get_debug_mb(void)
 	struct cfs_trace_cpu_data *tcd;
 	int total_pages = 0;
 
-	cfs_tracefile_read_lock();
+	down_read(&cfs_tracefile_sem);
 
 	cfs_tcd_for_each(tcd, i, j)
 		total_pages += tcd->tcd_max_pages;
 
-	cfs_tracefile_read_unlock();
+	up_read(&cfs_tracefile_sem);
 
 	return (total_pages >> (20 - PAGE_SHIFT)) + 1;
 }
@@ -1002,7 +1003,7 @@ static int tracefiled(void *arg)
 			goto end_loop;
 
 		filp = NULL;
-		cfs_tracefile_read_lock();
+		down_read(&cfs_tracefile_sem);
 		if (cfs_tracefile[0]) {
 			filp = filp_open(cfs_tracefile,
 					 O_CREAT | O_RDWR | O_LARGEFILE,
@@ -1014,7 +1015,7 @@ static int tracefiled(void *arg)
 					rc);
 			}
 		}
-		cfs_tracefile_read_unlock();
+		up_read(&cfs_tracefile_sem);
 		if (!filp) {
 			put_pages_on_daemon_list(&pc);
 			__LASSERT(list_empty(&pc.pc_pages));
diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.h b/drivers/staging/lustre/lnet/libcfs/tracefile.h
index 0608240d897f..9f6b73d3b6fd 100644
--- a/drivers/staging/lustre/lnet/libcfs/tracefile.h
+++ b/drivers/staging/lustre/lnet/libcfs/tracefile.h
@@ -65,11 +65,6 @@ void libcfs_run_debug_log_upcall(char *file);
 int  cfs_tracefile_init_arch(void);
 void cfs_tracefile_fini_arch(void);
 
-void cfs_tracefile_read_lock(void);
-void cfs_tracefile_read_unlock(void);
-void cfs_tracefile_write_lock(void);
-void cfs_tracefile_write_unlock(void);
-
 int cfs_tracefile_dump_all_pages(char *filename);
 void cfs_trace_debug_print(void);
 void cfs_trace_flush_pages(void);



From neilb at suse.com  Fri Jun 15 07:11:30 2018
From: neilb at suse.com (NeilBrown)
Date: Fri, 15 Jun 2018 17:11:30 +1000
Subject: [lustre-devel] [PATCH 19/24] lustre: open code
 cfs_trace_max_debug_mb() into cfs_trace_set_debug_mb()
In-Reply-To: <152904663333.10587.10934053155404014785.stgit@noble>
References: <152904663333.10587.10934053155404014785.stgit@noble>
Message-ID: <152904669076.10587.12692774514244094201.stgit@noble>

This function in used twice.
In libcfs_debug_init() the usage is pointless as
the only place that set libcfs_debug_mb ensures
that it does not exceed the maximum.  So checking
again can never help.

So open-code the small function into the only
other place that it is used - in cfs_trace_set_debug_mb(),
which is used to set libcfs_debug_mb.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 drivers/staging/lustre/lnet/libcfs/debug.c         |    6 +++---
 .../staging/lustre/lnet/libcfs/linux-tracefile.c   |    7 -------
 drivers/staging/lustre/lnet/libcfs/tracefile.c     |    3 ++-
 3 files changed, 5 insertions(+), 11 deletions(-)

diff --git a/drivers/staging/lustre/lnet/libcfs/debug.c b/drivers/staging/lustre/lnet/libcfs/debug.c
index 06f694f6a28f..71effcf3ebe9 100644
--- a/drivers/staging/lustre/lnet/libcfs/debug.c
+++ b/drivers/staging/lustre/lnet/libcfs/debug.c
@@ -411,10 +411,10 @@ int libcfs_debug_init(unsigned long bufsize)
 			sizeof(libcfs_debug_file_path_arr));
 	}
 
-	/* If libcfs_debug_mb is set to an invalid value or uninitialized
-	 * then just make the total buffers smp_num_cpus * TCD_MAX_PAGES
+	/* If libcfs_debug_mb is uninitialized then just make the
+	 * total buffers smp_num_cpus * TCD_MAX_PAGES
 	 */
-	if (max > cfs_trace_max_debug_mb() || max < num_possible_cpus()) {
+	if (max < num_possible_cpus()) {
 		max = TCD_MAX_PAGES;
 	} else {
 		max = max / num_possible_cpus();
diff --git a/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c b/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
index 9e72220299af..64a5bc1e2db8 100644
--- a/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
+++ b/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
@@ -227,10 +227,3 @@ void cfs_print_to_console(struct ptldebug_header *hdr, int mask,
 			hdr->ph_line_num, fn, len, buf);
 	}
 }
-
-int cfs_trace_max_debug_mb(void)
-{
-	int  total_mb = (totalram_pages >> (20 - PAGE_SHIFT));
-
-	return max(512, (total_mb * 80) / 100);
-}
diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.c b/drivers/staging/lustre/lnet/libcfs/tracefile.c
index 5f319332f60b..72321ce4c394 100644
--- a/drivers/staging/lustre/lnet/libcfs/tracefile.c
+++ b/drivers/staging/lustre/lnet/libcfs/tracefile.c
@@ -933,7 +933,8 @@ int cfs_trace_set_debug_mb(int mb)
 	int i;
 	int j;
 	int pages;
-	int limit = cfs_trace_max_debug_mb();
+	int total_mb = (totalram_pages >> (20 - PAGE_SHIFT));
+	int limit = max(512, (total_mb * 80) / 100);
 	struct cfs_trace_cpu_data *tcd;
 
 	if (mb < num_possible_cpus()) {



From neilb at suse.com  Fri Jun 15 07:11:30 2018
From: neilb at suse.com (NeilBrown)
Date: Fri, 15 Jun 2018 17:11:30 +1000
Subject: [lustre-devel] [PATCH 20/24] lustre: move tcd locking across to
 tracefile.c
In-Reply-To: <152904663333.10587.10934053155404014785.stgit@noble>
References: <152904663333.10587.10934053155404014785.stgit@noble>
Message-ID: <152904669080.10587.2737350683529078044.stgit@noble>

No need to have this in linux-tracefile.c

Signed-off-by: NeilBrown <neilb at suse.com>
---
 .../staging/lustre/lnet/libcfs/linux-tracefile.c   |   35 ------------
 drivers/staging/lustre/lnet/libcfs/tracefile.c     |   59 ++++++++++++++++++++
 drivers/staging/lustre/lnet/libcfs/tracefile.h     |   28 ---------
 3 files changed, 59 insertions(+), 63 deletions(-)

diff --git a/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c b/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
index 64a5bc1e2db8..3af77228cd7a 100644
--- a/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
+++ b/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
@@ -116,41 +116,6 @@ enum cfs_trace_buf_type cfs_trace_buf_idx_get(void)
 	return CFS_TCD_TYPE_PROC;
 }
 
-/*
- * The walking argument indicates the locking comes from all tcd types
- * iterator and we must lock it and dissable local irqs to avoid deadlocks
- * with other interrupt locks that might be happening. See LU-1311
- * for details.
- */
-int cfs_trace_lock_tcd(struct cfs_trace_cpu_data *tcd, int walking)
-	__acquires(&tcd->tc_lock)
-{
-	__LASSERT(tcd->tcd_type < CFS_TCD_TYPE_MAX);
-	if (tcd->tcd_type == CFS_TCD_TYPE_IRQ)
-		spin_lock_irqsave(&tcd->tcd_lock, tcd->tcd_lock_flags);
-	else if (tcd->tcd_type == CFS_TCD_TYPE_SOFTIRQ)
-		spin_lock_bh(&tcd->tcd_lock);
-	else if (unlikely(walking))
-		spin_lock_irq(&tcd->tcd_lock);
-	else
-		spin_lock(&tcd->tcd_lock);
-	return 1;
-}
-
-void cfs_trace_unlock_tcd(struct cfs_trace_cpu_data *tcd, int walking)
-	__releases(&tcd->tcd_lock)
-{
-	__LASSERT(tcd->tcd_type < CFS_TCD_TYPE_MAX);
-	if (tcd->tcd_type == CFS_TCD_TYPE_IRQ)
-		spin_unlock_irqrestore(&tcd->tcd_lock, tcd->tcd_lock_flags);
-	else if (tcd->tcd_type == CFS_TCD_TYPE_SOFTIRQ)
-		spin_unlock_bh(&tcd->tcd_lock);
-	else if (unlikely(walking))
-		spin_unlock_irq(&tcd->tcd_lock);
-	else
-		spin_unlock(&tcd->tcd_lock);
-}
-
 void
 cfs_set_ptldebug_header(struct ptldebug_header *header,
 			struct libcfs_debug_msg_data *msgdata,
diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.c b/drivers/staging/lustre/lnet/libcfs/tracefile.c
index 72321ce4c394..6d567a9ad8ea 100644
--- a/drivers/staging/lustre/lnet/libcfs/tracefile.c
+++ b/drivers/staging/lustre/lnet/libcfs/tracefile.c
@@ -109,6 +109,65 @@ struct cfs_trace_page {
 static void put_pages_on_tcd_daemon_list(struct page_collection *pc,
 					 struct cfs_trace_cpu_data *tcd);
 
+/* trace file lock routines */
+/*
+ * The walking argument indicates the locking comes from all tcd types
+ * iterator and we must lock it and dissable local irqs to avoid deadlocks
+ * with other interrupt locks that might be happening. See LU-1311
+ * for details.
+ */
+int cfs_trace_lock_tcd(struct cfs_trace_cpu_data *tcd, int walking)
+	__acquires(&tcd->tc_lock)
+{
+	__LASSERT(tcd->tcd_type < CFS_TCD_TYPE_MAX);
+	if (tcd->tcd_type == CFS_TCD_TYPE_IRQ)
+		spin_lock_irqsave(&tcd->tcd_lock, tcd->tcd_lock_flags);
+	else if (tcd->tcd_type == CFS_TCD_TYPE_SOFTIRQ)
+		spin_lock_bh(&tcd->tcd_lock);
+	else if (unlikely(walking))
+		spin_lock_irq(&tcd->tcd_lock);
+	else
+		spin_lock(&tcd->tcd_lock);
+	return 1;
+}
+
+void cfs_trace_unlock_tcd(struct cfs_trace_cpu_data *tcd, int walking)
+	__releases(&tcd->tcd_lock)
+{
+	__LASSERT(tcd->tcd_type < CFS_TCD_TYPE_MAX);
+	if (tcd->tcd_type == CFS_TCD_TYPE_IRQ)
+		spin_unlock_irqrestore(&tcd->tcd_lock, tcd->tcd_lock_flags);
+	else if (tcd->tcd_type == CFS_TCD_TYPE_SOFTIRQ)
+		spin_unlock_bh(&tcd->tcd_lock);
+	else if (unlikely(walking))
+		spin_unlock_irq(&tcd->tcd_lock);
+	else
+		spin_unlock(&tcd->tcd_lock);
+}
+
+#define cfs_tcd_for_each_type_lock(tcd, i, cpu)			   \
+	for (i = 0; cfs_trace_data[i] &&				\
+	     (tcd = &(*cfs_trace_data[i])[cpu].tcd) &&			\
+	     cfs_trace_lock_tcd(tcd, 1); cfs_trace_unlock_tcd(tcd, 1), i++)
+
+static inline struct cfs_trace_cpu_data *
+cfs_trace_get_tcd(void)
+{
+	struct cfs_trace_cpu_data *tcd =
+		&(*cfs_trace_data[cfs_trace_buf_idx_get()])[get_cpu()].tcd;
+
+	cfs_trace_lock_tcd(tcd, 0);
+
+	return tcd;
+}
+
+static inline void cfs_trace_put_tcd(struct cfs_trace_cpu_data *tcd)
+{
+	cfs_trace_unlock_tcd(tcd, 0);
+
+	put_cpu();
+}
+
 static inline struct cfs_trace_page *
 cfs_tage_from_list(struct list_head *list)
 {
diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.h b/drivers/staging/lustre/lnet/libcfs/tracefile.h
index 9f6b73d3b6fd..f49a9ba00864 100644
--- a/drivers/staging/lustre/lnet/libcfs/tracefile.h
+++ b/drivers/staging/lustre/lnet/libcfs/tracefile.h
@@ -49,8 +49,6 @@ enum cfs_trace_buf_type {
 	CFS_TCD_TYPE_MAX
 };
 
-/* trace file lock routines */
-
 #define TRACEFILE_NAME_SIZE 1024
 extern char cfs_tracefile[TRACEFILE_NAME_SIZE];
 extern long long cfs_tracefile_size;
@@ -195,11 +193,6 @@ extern union cfs_trace_data_union (*cfs_trace_data[TCD_MAX_TYPES])[NR_CPUS];
 		     j < num_possible_cpus();				 \
 		     j++, (tcd) = &(*cfs_trace_data[i])[j].tcd)
 
-#define cfs_tcd_for_each_type_lock(tcd, i, cpu)			   \
-	for (i = 0; cfs_trace_data[i] &&				\
-	     (tcd = &(*cfs_trace_data[i])[cpu].tcd) &&			\
-	     cfs_trace_lock_tcd(tcd, 1); cfs_trace_unlock_tcd(tcd, 1), i++)
-
 void cfs_set_ptldebug_header(struct ptldebug_header *header,
 			     struct libcfs_debug_msg_data *m,
 			     unsigned long stack);
@@ -207,9 +200,6 @@ void cfs_print_to_console(struct ptldebug_header *hdr, int mask,
 			  const char *buf, int len, const char *file,
 			  const char *fn);
 
-int cfs_trace_lock_tcd(struct cfs_trace_cpu_data *tcd, int walking);
-void cfs_trace_unlock_tcd(struct cfs_trace_cpu_data *tcd, int walking);
-
 extern char *cfs_trace_console_buffers[NR_CPUS][CFS_TCD_TYPE_MAX];
 enum cfs_trace_buf_type cfs_trace_buf_idx_get(void);
 
@@ -222,24 +212,6 @@ cfs_trace_get_console_buffer(void)
 	return cfs_trace_console_buffers[i][j];
 }
 
-static inline struct cfs_trace_cpu_data *
-cfs_trace_get_tcd(void)
-{
-	struct cfs_trace_cpu_data *tcd =
-		&(*cfs_trace_data[cfs_trace_buf_idx_get()])[get_cpu()].tcd;
-
-	cfs_trace_lock_tcd(tcd, 0);
-
-	return tcd;
-}
-
-static inline void cfs_trace_put_tcd(struct cfs_trace_cpu_data *tcd)
-{
-	cfs_trace_unlock_tcd(tcd, 0);
-
-	put_cpu();
-}
-
 int cfs_trace_refill_stock(struct cfs_trace_cpu_data *tcd, gfp_t gfp,
 			   struct list_head *stock);
 



From neilb at suse.com  Fri Jun 15 07:11:30 2018
From: neilb at suse.com (NeilBrown)
Date: Fri, 15 Jun 2018 17:11:30 +1000
Subject: [lustre-devel] [PATCH 21/24] lustre: move remainder of
 linux-tracefile.c to tracefile.c
In-Reply-To: <152904663333.10587.10934053155404014785.stgit@noble>
References: <152904663333.10587.10934053155404014785.stgit@noble>
Message-ID: <152904669084.10587.6920959206498193596.stgit@noble>

It's good to keep related code together.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 drivers/staging/lustre/lnet/libcfs/Makefile        |    2 
 .../staging/lustre/lnet/libcfs/linux-tracefile.c   |  194 --------------------
 drivers/staging/lustre/lnet/libcfs/tracefile.c     |  155 ++++++++++++++++
 3 files changed, 156 insertions(+), 195 deletions(-)
 delete mode 100644 drivers/staging/lustre/lnet/libcfs/linux-tracefile.c

diff --git a/drivers/staging/lustre/lnet/libcfs/Makefile b/drivers/staging/lustre/lnet/libcfs/Makefile
index 1b6937c3deee..22c0e0c2e9cf 100644
--- a/drivers/staging/lustre/lnet/libcfs/Makefile
+++ b/drivers/staging/lustre/lnet/libcfs/Makefile
@@ -3,7 +3,7 @@ ccflags-y += -I$(srctree)/drivers/staging/lustre/include
 ccflags-y += -I$(srctree)/drivers/staging/lustre/lustre/include
 ccflags-y += -DKBUILD_MODNAME='"lnet"'
 
-libcfs-obj-y += linux-tracefile.o linux-debug.o
+libcfs-obj-y += linux-debug.o
 libcfs-obj-y += linux-crypto.o
 libcfs-obj-y += linux-crypto-adler.o
 
diff --git a/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c b/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
deleted file mode 100644
index 3af77228cd7a..000000000000
--- a/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
+++ /dev/null
@@ -1,194 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-/*
- * GPL HEADER START
- *
- * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 only,
- * as published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful, but
- * WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
- * General Public License version 2 for more details (a copy is included
- * in the LICENSE file that accompanied this code).
- *
- * You should have received a copy of the GNU General Public License
- * version 2 along with this program; If not, see
- * http://www.gnu.org/licenses/gpl-2.0.html
- *
- * GPL HEADER END
- */
-/*
- * Copyright (c) 2008, 2010, Oracle and/or its affiliates. All rights reserved.
- * Use is subject to license terms.
- *
- * Copyright (c) 2012, Intel Corporation.
- */
-/*
- * This file is part of Lustre, http://www.lustre.org/
- * Lustre is a trademark of Sun Microsystems, Inc.
- */
-
-#define DEBUG_SUBSYSTEM S_LNET
-#define LUSTRE_TRACEFILE_PRIVATE
-
-#include <linux/slab.h>
-#include <linux/mm.h>
-#include "tracefile.h"
-
-/* percents to share the total debug memory for each type */
-static unsigned int pages_factor[CFS_TCD_TYPE_MAX] = {
-	80,  /* 80% pages for CFS_TCD_TYPE_PROC */
-	10,  /* 10% pages for CFS_TCD_TYPE_SOFTIRQ */
-	10   /* 10% pages for CFS_TCD_TYPE_IRQ */
-};
-
-char *cfs_trace_console_buffers[NR_CPUS][CFS_TCD_TYPE_MAX];
-
-int cfs_tracefile_init_arch(void)
-{
-	int i;
-	int j;
-	struct cfs_trace_cpu_data *tcd;
-
-	/* initialize trace_data */
-	memset(cfs_trace_data, 0, sizeof(cfs_trace_data));
-	for (i = 0; i < CFS_TCD_TYPE_MAX; i++) {
-		cfs_trace_data[i] =
-			kmalloc_array(num_possible_cpus(),
-				      sizeof(union cfs_trace_data_union),
-				      GFP_KERNEL);
-		if (!cfs_trace_data[i])
-			goto out;
-	}
-
-	/* arch related info initialized */
-	cfs_tcd_for_each(tcd, i, j) {
-		spin_lock_init(&tcd->tcd_lock);
-		tcd->tcd_pages_factor = pages_factor[i];
-		tcd->tcd_type = i;
-		tcd->tcd_cpu = j;
-	}
-
-	for (i = 0; i < num_possible_cpus(); i++)
-		for (j = 0; j < 3; j++) {
-			cfs_trace_console_buffers[i][j] =
-				kmalloc(CFS_TRACE_CONSOLE_BUFFER_SIZE,
-					GFP_KERNEL);
-
-			if (!cfs_trace_console_buffers[i][j])
-				goto out;
-		}
-
-	return 0;
-
-out:
-	cfs_tracefile_fini_arch();
-	pr_err("lnet: Not enough memory\n");
-	return -ENOMEM;
-}
-
-void cfs_tracefile_fini_arch(void)
-{
-	int i;
-	int j;
-
-	for (i = 0; i < num_possible_cpus(); i++)
-		for (j = 0; j < 3; j++) {
-			kfree(cfs_trace_console_buffers[i][j]);
-			cfs_trace_console_buffers[i][j] = NULL;
-		}
-
-	for (i = 0; cfs_trace_data[i]; i++) {
-		kfree(cfs_trace_data[i]);
-		cfs_trace_data[i] = NULL;
-	}
-}
-
-enum cfs_trace_buf_type cfs_trace_buf_idx_get(void)
-{
-	if (in_irq())
-		return CFS_TCD_TYPE_IRQ;
-	if (in_softirq())
-		return CFS_TCD_TYPE_SOFTIRQ;
-	return CFS_TCD_TYPE_PROC;
-}
-
-void
-cfs_set_ptldebug_header(struct ptldebug_header *header,
-			struct libcfs_debug_msg_data *msgdata,
-			unsigned long stack)
-{
-	struct timespec64 ts;
-
-	ktime_get_real_ts64(&ts);
-
-	header->ph_subsys = msgdata->msg_subsys;
-	header->ph_mask = msgdata->msg_mask;
-	header->ph_cpu_id = smp_processor_id();
-	header->ph_type = cfs_trace_buf_idx_get();
-	/* y2038 safe since all user space treats this as unsigned, but
-	 * will overflow in 2106
-	 */
-	header->ph_sec = (u32)ts.tv_sec;
-	header->ph_usec = ts.tv_nsec / NSEC_PER_USEC;
-	header->ph_stack = stack;
-	header->ph_pid = current->pid;
-	header->ph_line_num = msgdata->msg_line;
-	header->ph_extern_pid = 0;
-}
-
-static char *
-dbghdr_to_err_string(struct ptldebug_header *hdr)
-{
-	switch (hdr->ph_subsys) {
-	case S_LND:
-	case S_LNET:
-		return "LNetError";
-	default:
-		return "LustreError";
-	}
-}
-
-static char *
-dbghdr_to_info_string(struct ptldebug_header *hdr)
-{
-	switch (hdr->ph_subsys) {
-	case S_LND:
-	case S_LNET:
-		return "LNet";
-	default:
-		return "Lustre";
-	}
-}
-
-void cfs_print_to_console(struct ptldebug_header *hdr, int mask,
-			  const char *buf, int len, const char *file,
-			  const char *fn)
-{
-	char *prefix = "Lustre", *ptype = NULL;
-
-	if (mask & D_EMERG) {
-		prefix = dbghdr_to_err_string(hdr);
-		ptype = KERN_EMERG;
-	} else if (mask & D_ERROR) {
-		prefix = dbghdr_to_err_string(hdr);
-		ptype = KERN_ERR;
-	} else if (mask & D_WARNING) {
-		prefix = dbghdr_to_info_string(hdr);
-		ptype = KERN_WARNING;
-	} else if (mask & (D_CONSOLE | libcfs_printk)) {
-		prefix = dbghdr_to_info_string(hdr);
-		ptype = KERN_INFO;
-	}
-
-	if (mask & D_CONSOLE) {
-		pr_info("%s%s: %.*s", ptype, prefix, len, buf);
-	} else {
-		pr_info("%s%s: %d:%d:(%s:%d:%s()) %.*s", ptype, prefix,
-			hdr->ph_pid, hdr->ph_extern_pid, file,
-			hdr->ph_line_num, fn, len, buf);
-	}
-}
diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.c b/drivers/staging/lustre/lnet/libcfs/tracefile.c
index 6d567a9ad8ea..617edc291035 100644
--- a/drivers/staging/lustre/lnet/libcfs/tracefile.c
+++ b/drivers/staging/lustre/lnet/libcfs/tracefile.c
@@ -168,6 +168,161 @@ static inline void cfs_trace_put_tcd(struct cfs_trace_cpu_data *tcd)
 	put_cpu();
 }
 
+/* percents to share the total debug memory for each type */
+static unsigned int pages_factor[CFS_TCD_TYPE_MAX] = {
+	80,  /* 80% pages for CFS_TCD_TYPE_PROC */
+	10,  /* 10% pages for CFS_TCD_TYPE_SOFTIRQ */
+	10   /* 10% pages for CFS_TCD_TYPE_IRQ */
+};
+
+char *cfs_trace_console_buffers[NR_CPUS][CFS_TCD_TYPE_MAX];
+
+int cfs_tracefile_init_arch(void)
+{
+	int i;
+	int j;
+	struct cfs_trace_cpu_data *tcd;
+
+	/* initialize trace_data */
+	memset(cfs_trace_data, 0, sizeof(cfs_trace_data));
+	for (i = 0; i < CFS_TCD_TYPE_MAX; i++) {
+		cfs_trace_data[i] =
+			kmalloc_array(num_possible_cpus(),
+				      sizeof(union cfs_trace_data_union),
+				      GFP_KERNEL);
+		if (!cfs_trace_data[i])
+			goto out;
+	}
+
+	/* arch related info initialized */
+	cfs_tcd_for_each(tcd, i, j) {
+		spin_lock_init(&tcd->tcd_lock);
+		tcd->tcd_pages_factor = pages_factor[i];
+		tcd->tcd_type = i;
+		tcd->tcd_cpu = j;
+	}
+
+	for (i = 0; i < num_possible_cpus(); i++)
+		for (j = 0; j < 3; j++) {
+			cfs_trace_console_buffers[i][j] =
+				kmalloc(CFS_TRACE_CONSOLE_BUFFER_SIZE,
+					GFP_KERNEL);
+
+			if (!cfs_trace_console_buffers[i][j])
+				goto out;
+		}
+
+	return 0;
+
+out:
+	cfs_tracefile_fini_arch();
+	pr_err("lnet: Not enough memory\n");
+	return -ENOMEM;
+}
+
+void cfs_tracefile_fini_arch(void)
+{
+	int i;
+	int j;
+
+	for (i = 0; i < num_possible_cpus(); i++)
+		for (j = 0; j < 3; j++) {
+			kfree(cfs_trace_console_buffers[i][j]);
+			cfs_trace_console_buffers[i][j] = NULL;
+		}
+
+	for (i = 0; cfs_trace_data[i]; i++) {
+		kfree(cfs_trace_data[i]);
+		cfs_trace_data[i] = NULL;
+	}
+}
+
+enum cfs_trace_buf_type cfs_trace_buf_idx_get(void)
+{
+	if (in_irq())
+		return CFS_TCD_TYPE_IRQ;
+	if (in_softirq())
+		return CFS_TCD_TYPE_SOFTIRQ;
+	return CFS_TCD_TYPE_PROC;
+}
+
+void
+cfs_set_ptldebug_header(struct ptldebug_header *header,
+			struct libcfs_debug_msg_data *msgdata,
+			unsigned long stack)
+{
+	struct timespec64 ts;
+
+	ktime_get_real_ts64(&ts);
+
+	header->ph_subsys = msgdata->msg_subsys;
+	header->ph_mask = msgdata->msg_mask;
+	header->ph_cpu_id = smp_processor_id();
+	header->ph_type = cfs_trace_buf_idx_get();
+	/* y2038 safe since all user space treats this as unsigned, but
+	 * will overflow in 2106
+	 */
+	header->ph_sec = (u32)ts.tv_sec;
+	header->ph_usec = ts.tv_nsec / NSEC_PER_USEC;
+	header->ph_stack = stack;
+	header->ph_pid = current->pid;
+	header->ph_line_num = msgdata->msg_line;
+	header->ph_extern_pid = 0;
+}
+
+static char *
+dbghdr_to_err_string(struct ptldebug_header *hdr)
+{
+	switch (hdr->ph_subsys) {
+	case S_LND:
+	case S_LNET:
+		return "LNetError";
+	default:
+		return "LustreError";
+	}
+}
+
+static char *
+dbghdr_to_info_string(struct ptldebug_header *hdr)
+{
+	switch (hdr->ph_subsys) {
+	case S_LND:
+	case S_LNET:
+		return "LNet";
+	default:
+		return "Lustre";
+	}
+}
+
+void cfs_print_to_console(struct ptldebug_header *hdr, int mask,
+			  const char *buf, int len, const char *file,
+			  const char *fn)
+{
+	char *prefix = "Lustre", *ptype = NULL;
+
+	if (mask & D_EMERG) {
+		prefix = dbghdr_to_err_string(hdr);
+		ptype = KERN_EMERG;
+	} else if (mask & D_ERROR) {
+		prefix = dbghdr_to_err_string(hdr);
+		ptype = KERN_ERR;
+	} else if (mask & D_WARNING) {
+		prefix = dbghdr_to_info_string(hdr);
+		ptype = KERN_WARNING;
+	} else if (mask & (D_CONSOLE | libcfs_printk)) {
+		prefix = dbghdr_to_info_string(hdr);
+		ptype = KERN_INFO;
+	}
+
+	if (mask & D_CONSOLE) {
+		pr_info("%s%s: %.*s", ptype, prefix, len, buf);
+	} else {
+		pr_info("%s%s: %d:%d:(%s:%d:%s()) %.*s", ptype, prefix,
+			hdr->ph_pid, hdr->ph_extern_pid, file,
+			hdr->ph_line_num, fn, len, buf);
+	}
+}
+
 static inline struct cfs_trace_page *
 cfs_tage_from_list(struct list_head *list)
 {



From neilb at suse.com  Fri Jun 15 07:11:30 2018
From: neilb at suse.com (NeilBrown)
Date: Fri, 15 Jun 2018 17:11:30 +1000
Subject: [lustre-devel] [PATCH 22/24] lustre: fold cfs_tracefile_*_arch into
 their only callers.
In-Reply-To: <152904663333.10587.10934053155404014785.stgit@noble>
References: <152904663333.10587.10934053155404014785.stgit@noble>
Message-ID: <152904669087.10587.4562289436203420686.stgit@noble>

There is no need to separate "arch" init/fini from
the rest, so fold it all in.
This requires some slightly subtle changes to clean-up
to make sure we don't walk lists before they are
initialized.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 drivers/staging/lustre/lnet/libcfs/tracefile.c |  116 ++++++++++--------------
 1 file changed, 49 insertions(+), 67 deletions(-)

diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.c b/drivers/staging/lustre/lnet/libcfs/tracefile.c
index 617edc291035..fbd05905e1c4 100644
--- a/drivers/staging/lustre/lnet/libcfs/tracefile.c
+++ b/drivers/staging/lustre/lnet/libcfs/tracefile.c
@@ -177,65 +177,6 @@ static unsigned int pages_factor[CFS_TCD_TYPE_MAX] = {
 
 char *cfs_trace_console_buffers[NR_CPUS][CFS_TCD_TYPE_MAX];
 
-int cfs_tracefile_init_arch(void)
-{
-	int i;
-	int j;
-	struct cfs_trace_cpu_data *tcd;
-
-	/* initialize trace_data */
-	memset(cfs_trace_data, 0, sizeof(cfs_trace_data));
-	for (i = 0; i < CFS_TCD_TYPE_MAX; i++) {
-		cfs_trace_data[i] =
-			kmalloc_array(num_possible_cpus(),
-				      sizeof(union cfs_trace_data_union),
-				      GFP_KERNEL);
-		if (!cfs_trace_data[i])
-			goto out;
-	}
-
-	/* arch related info initialized */
-	cfs_tcd_for_each(tcd, i, j) {
-		spin_lock_init(&tcd->tcd_lock);
-		tcd->tcd_pages_factor = pages_factor[i];
-		tcd->tcd_type = i;
-		tcd->tcd_cpu = j;
-	}
-
-	for (i = 0; i < num_possible_cpus(); i++)
-		for (j = 0; j < 3; j++) {
-			cfs_trace_console_buffers[i][j] =
-				kmalloc(CFS_TRACE_CONSOLE_BUFFER_SIZE,
-					GFP_KERNEL);
-
-			if (!cfs_trace_console_buffers[i][j])
-				goto out;
-		}
-
-	return 0;
-
-out:
-	cfs_tracefile_fini_arch();
-	pr_err("lnet: Not enough memory\n");
-	return -ENOMEM;
-}
-
-void cfs_tracefile_fini_arch(void)
-{
-	int i;
-	int j;
-
-	for (i = 0; i < num_possible_cpus(); i++)
-		for (j = 0; j < 3; j++) {
-			kfree(cfs_trace_console_buffers[i][j]);
-			cfs_trace_console_buffers[i][j] = NULL;
-		}
-
-	for (i = 0; cfs_trace_data[i]; i++) {
-		kfree(cfs_trace_data[i]);
-		cfs_trace_data[i] = NULL;
-	}
-}
 
 enum cfs_trace_buf_type cfs_trace_buf_idx_get(void)
 {
@@ -1343,21 +1284,33 @@ void cfs_trace_stop_thread(void)
 	mutex_unlock(&cfs_trace_thread_mutex);
 }
 
+static void cfs_trace_cleanup(void);
+
 int cfs_tracefile_init(int max_pages)
 {
 	struct cfs_trace_cpu_data *tcd;
 	int i;
 	int j;
-	int rc;
-	int factor;
 
-	rc = cfs_tracefile_init_arch();
-	if (rc)
-		return rc;
+	/* initialize trace_data */
+	memset(cfs_trace_data, 0, sizeof(cfs_trace_data));
+	for (i = 0; i < CFS_TCD_TYPE_MAX; i++) {
+		cfs_trace_data[i] =
+			kmalloc_array(num_possible_cpus(),
+				      sizeof(union cfs_trace_data_union),
+				      GFP_KERNEL);
+		if (!cfs_trace_data[i])
+			goto out;
+	}
 
+	/* arch related info initialized */
 	cfs_tcd_for_each(tcd, i, j) {
-		/* tcd_pages_factor is initialized int tracefile_init_arch. */
-		factor = tcd->tcd_pages_factor;
+		int factor = pages_factor[i];
+
+		spin_lock_init(&tcd->tcd_lock);
+		tcd->tcd_pages_factor = factor;
+		tcd->tcd_type = i;
+		tcd->tcd_cpu = j;
 		INIT_LIST_HEAD(&tcd->tcd_pages);
 		INIT_LIST_HEAD(&tcd->tcd_stock_pages);
 		INIT_LIST_HEAD(&tcd->tcd_daemon_pages);
@@ -1369,7 +1322,21 @@ int cfs_tracefile_init(int max_pages)
 		tcd->tcd_shutting_down = 0;
 	}
 
+	for (i = 0; i < num_possible_cpus(); i++)
+		for (j = 0; j < 3; j++) {
+			cfs_trace_console_buffers[i][j] =
+				kmalloc(CFS_TRACE_CONSOLE_BUFFER_SIZE,
+					GFP_KERNEL);
+
+			if (!cfs_trace_console_buffers[i][j])
+				goto out;
+		}
+
 	return 0;
+out:
+	cfs_trace_cleanup();
+	pr_err("lnet: Not enough memory\n");
+	return -ENOMEM;
 }
 
 static void trace_cleanup_on_all_cpus(void)
@@ -1381,6 +1348,9 @@ static void trace_cleanup_on_all_cpus(void)
 
 	for_each_possible_cpu(cpu) {
 		cfs_tcd_for_each_type_lock(tcd, i, cpu) {
+			if (!tcd->tcd_pages_factor)
+				/* Not initialised */
+				continue;
 			tcd->tcd_shutting_down = 1;
 
 			list_for_each_entry_safe(tage, tmp, &tcd->tcd_pages,
@@ -1399,12 +1369,24 @@ static void trace_cleanup_on_all_cpus(void)
 static void cfs_trace_cleanup(void)
 {
 	struct page_collection pc;
+	int i;
+	int j;
 
 	INIT_LIST_HEAD(&pc.pc_pages);
 
 	trace_cleanup_on_all_cpus();
 
-	cfs_tracefile_fini_arch();
+	for (i = 0; i < num_possible_cpus(); i++)
+		for (j = 0; j < 3; j++) {
+			kfree(cfs_trace_console_buffers[i][j]);
+			cfs_trace_console_buffers[i][j] = NULL;
+		}
+
+	for (i = 0; cfs_trace_data[i]; i++) {
+		kfree(cfs_trace_data[i]);
+		cfs_trace_data[i] = NULL;
+	}
+
 }
 
 void cfs_tracefile_exit(void)



From neilb at suse.com  Fri Jun 15 07:11:30 2018
From: neilb at suse.com (NeilBrown)
Date: Fri, 15 Jun 2018 17:11:30 +1000
Subject: [lustre-devel] [PATCH 23/24] lustre: renamed CFS_TCD_TYPE_MAX to
 CFS_TCD_TYPE_CNT
In-Reply-To: <152904663333.10587.10934053155404014785.stgit@noble>
References: <152904663333.10587.10934053155404014785.stgit@noble>
Message-ID: <152904669090.10587.11055011083558241355.stgit@noble>

The possible TCD types are 0, 1, 2.
So the MAX is 2.
The count of the number of types is 3.

CFS_TCD_TYPE_MAX is 3 - obviously wrong.

So rename it to CFS_TCD_TYPE_CNT.

Also there are 2 places where "3" is used rather
than the macro - fix them.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 drivers/staging/lustre/lnet/libcfs/tracefile.c |   15 +++++++--------
 drivers/staging/lustre/lnet/libcfs/tracefile.h |    4 ++--
 2 files changed, 9 insertions(+), 10 deletions(-)

diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.c b/drivers/staging/lustre/lnet/libcfs/tracefile.c
index fbd05905e1c4..cdef67391a72 100644
--- a/drivers/staging/lustre/lnet/libcfs/tracefile.c
+++ b/drivers/staging/lustre/lnet/libcfs/tracefile.c
@@ -119,7 +119,7 @@ static void put_pages_on_tcd_daemon_list(struct page_collection *pc,
 int cfs_trace_lock_tcd(struct cfs_trace_cpu_data *tcd, int walking)
 	__acquires(&tcd->tc_lock)
 {
-	__LASSERT(tcd->tcd_type < CFS_TCD_TYPE_MAX);
+	__LASSERT(tcd->tcd_type < CFS_TCD_TYPE_CNT);
 	if (tcd->tcd_type == CFS_TCD_TYPE_IRQ)
 		spin_lock_irqsave(&tcd->tcd_lock, tcd->tcd_lock_flags);
 	else if (tcd->tcd_type == CFS_TCD_TYPE_SOFTIRQ)
@@ -134,7 +134,7 @@ int cfs_trace_lock_tcd(struct cfs_trace_cpu_data *tcd, int walking)
 void cfs_trace_unlock_tcd(struct cfs_trace_cpu_data *tcd, int walking)
 	__releases(&tcd->tcd_lock)
 {
-	__LASSERT(tcd->tcd_type < CFS_TCD_TYPE_MAX);
+	__LASSERT(tcd->tcd_type < CFS_TCD_TYPE_CNT);
 	if (tcd->tcd_type == CFS_TCD_TYPE_IRQ)
 		spin_unlock_irqrestore(&tcd->tcd_lock, tcd->tcd_lock_flags);
 	else if (tcd->tcd_type == CFS_TCD_TYPE_SOFTIRQ)
@@ -169,14 +169,13 @@ static inline void cfs_trace_put_tcd(struct cfs_trace_cpu_data *tcd)
 }
 
 /* percents to share the total debug memory for each type */
-static unsigned int pages_factor[CFS_TCD_TYPE_MAX] = {
+static unsigned int pages_factor[CFS_TCD_TYPE_CNT] = {
 	80,  /* 80% pages for CFS_TCD_TYPE_PROC */
 	10,  /* 10% pages for CFS_TCD_TYPE_SOFTIRQ */
 	10   /* 10% pages for CFS_TCD_TYPE_IRQ */
 };
 
-char *cfs_trace_console_buffers[NR_CPUS][CFS_TCD_TYPE_MAX];
-
+char *cfs_trace_console_buffers[NR_CPUS][CFS_TCD_TYPE_CNT];
 
 enum cfs_trace_buf_type cfs_trace_buf_idx_get(void)
 {
@@ -1294,7 +1293,7 @@ int cfs_tracefile_init(int max_pages)
 
 	/* initialize trace_data */
 	memset(cfs_trace_data, 0, sizeof(cfs_trace_data));
-	for (i = 0; i < CFS_TCD_TYPE_MAX; i++) {
+	for (i = 0; i < CFS_TCD_TYPE_CNT; i++) {
 		cfs_trace_data[i] =
 			kmalloc_array(num_possible_cpus(),
 				      sizeof(union cfs_trace_data_union),
@@ -1323,7 +1322,7 @@ int cfs_tracefile_init(int max_pages)
 	}
 
 	for (i = 0; i < num_possible_cpus(); i++)
-		for (j = 0; j < 3; j++) {
+		for (j = 0; j < CFS_TCD_TYPE_CNT; j++) {
 			cfs_trace_console_buffers[i][j] =
 				kmalloc(CFS_TRACE_CONSOLE_BUFFER_SIZE,
 					GFP_KERNEL);
@@ -1377,7 +1376,7 @@ static void cfs_trace_cleanup(void)
 	trace_cleanup_on_all_cpus();
 
 	for (i = 0; i < num_possible_cpus(); i++)
-		for (j = 0; j < 3; j++) {
+		for (j = 0; j < CFS_TCD_TYPE_CNT; j++) {
 			kfree(cfs_trace_console_buffers[i][j]);
 			cfs_trace_console_buffers[i][j] = NULL;
 		}
diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.h b/drivers/staging/lustre/lnet/libcfs/tracefile.h
index f49a9ba00864..23faecf886c1 100644
--- a/drivers/staging/lustre/lnet/libcfs/tracefile.h
+++ b/drivers/staging/lustre/lnet/libcfs/tracefile.h
@@ -46,7 +46,7 @@ enum cfs_trace_buf_type {
 	CFS_TCD_TYPE_PROC = 0,
 	CFS_TCD_TYPE_SOFTIRQ,
 	CFS_TCD_TYPE_IRQ,
-	CFS_TCD_TYPE_MAX
+	CFS_TCD_TYPE_CNT
 };
 
 #define TRACEFILE_NAME_SIZE 1024
@@ -200,7 +200,7 @@ void cfs_print_to_console(struct ptldebug_header *hdr, int mask,
 			  const char *buf, int len, const char *file,
 			  const char *fn);
 
-extern char *cfs_trace_console_buffers[NR_CPUS][CFS_TCD_TYPE_MAX];
+extern char *cfs_trace_console_buffers[NR_CPUS][CFS_TCD_TYPE_CNT];
 enum cfs_trace_buf_type cfs_trace_buf_idx_get(void);
 
 static inline char *



From neilb at suse.com  Fri Jun 15 07:11:30 2018
From: neilb at suse.com (NeilBrown)
Date: Fri, 15 Jun 2018 17:11:30 +1000
Subject: [lustre-devel] [PATCH 24/24] lustre: discard TCD_MAX_TYPES
In-Reply-To: <152904663333.10587.10934053155404014785.stgit@noble>
References: <152904663333.10587.10934053155404014785.stgit@noble>
Message-ID: <152904669094.10587.17019477478226952858.stgit@noble>

As well as CFS_TCD_TYPE_CNT we have TCD_MAX_TYPES which has a larger
value but a similar meaning.  Discard it and just use
CFS_TCD_TYPE_CNT.

Two places relied on the fact that TCD_MAX_TYPES was larger and so
there would be NULLs at the end of the array.  Change
them to check the array size properly.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 drivers/staging/lustre/lnet/libcfs/tracefile.c |    8 ++++----
 drivers/staging/lustre/lnet/libcfs/tracefile.h |    3 +--
 2 files changed, 5 insertions(+), 6 deletions(-)

diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.c b/drivers/staging/lustre/lnet/libcfs/tracefile.c
index cdef67391a72..555ade0ddafb 100644
--- a/drivers/staging/lustre/lnet/libcfs/tracefile.c
+++ b/drivers/staging/lustre/lnet/libcfs/tracefile.c
@@ -50,7 +50,7 @@
 #include "tracefile.h"
 
 /* XXX move things up to the top, comment */
-union cfs_trace_data_union (*cfs_trace_data[TCD_MAX_TYPES])[NR_CPUS] __cacheline_aligned;
+union cfs_trace_data_union (*cfs_trace_data[CFS_TCD_TYPE_CNT])[NR_CPUS] __cacheline_aligned;
 
 char cfs_tracefile[TRACEFILE_NAME_SIZE];
 long long cfs_tracefile_size = CFS_TRACEFILE_SIZE;
@@ -145,8 +145,8 @@ void cfs_trace_unlock_tcd(struct cfs_trace_cpu_data *tcd, int walking)
 		spin_unlock(&tcd->tcd_lock);
 }
 
-#define cfs_tcd_for_each_type_lock(tcd, i, cpu)			   \
-	for (i = 0; cfs_trace_data[i] &&				\
+#define cfs_tcd_for_each_type_lock(tcd, i, cpu)				\
+	for (i = 0; i < CFS_TCD_TYPE_CNT && cfs_trace_data[i] &&	\
 	     (tcd = &(*cfs_trace_data[i])[cpu].tcd) &&			\
 	     cfs_trace_lock_tcd(tcd, 1); cfs_trace_unlock_tcd(tcd, 1), i++)
 
@@ -1381,7 +1381,7 @@ static void cfs_trace_cleanup(void)
 			cfs_trace_console_buffers[i][j] = NULL;
 		}
 
-	for (i = 0; cfs_trace_data[i]; i++) {
+	for (i = 0; cfs_trace_data[i] && i < CFS_TCD_TYPE_CNT; i++) {
 		kfree(cfs_trace_data[i]);
 		cfs_trace_data[i] = NULL;
 	}
diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.h b/drivers/staging/lustre/lnet/libcfs/tracefile.h
index 23faecf886c1..a09d5b9349f0 100644
--- a/drivers/staging/lustre/lnet/libcfs/tracefile.h
+++ b/drivers/staging/lustre/lnet/libcfs/tracefile.h
@@ -184,8 +184,7 @@ union cfs_trace_data_union {
 	char __pad[L1_CACHE_ALIGN(sizeof(struct cfs_trace_cpu_data))];
 };
 
-#define TCD_MAX_TYPES      8
-extern union cfs_trace_data_union (*cfs_trace_data[TCD_MAX_TYPES])[NR_CPUS];
+extern union cfs_trace_data_union (*cfs_trace_data[CFS_TCD_TYPE_CNT])[NR_CPUS];
 
 #define cfs_tcd_for_each(tcd, i, j)				       \
 	for (i = 0; cfs_trace_data[i]; i++)				\



From jrdr.linux at gmail.com  Tue Jun 12 19:11:12 2018
From: jrdr.linux at gmail.com (Souptick Joarder)
Date: Wed, 13 Jun 2018 00:41:12 +0530
Subject: [lustre-devel] [PATCH v3] staging: lustre: Change return type
 to vm_fault_t
In-Reply-To: <20180611210714.GB4023@kroah.com>
References: <20180521180937.GA19605@jordon-HP-15-Notebook-PC>
 <CAFqt6zamp=+adkyM+_4z405hN-9qOn1CsY_BgVtCrAfscBHhWg@mail.gmail.com>
 <20180611205058.GA30123@kroah.com>
 <CAFqt6zZJcN4=FDQSqiSznoEogavV9GBpqxx-onWG5Q3wQEM9cQ@mail.gmail.com>
 <20180611210714.GB4023@kroah.com>
Message-ID: <CAFqt6zYyy2yygmU+_KSNydjkW8V7A5oefcpMyMrFDd=2q1Ppdw@mail.gmail.com>

On Tue, Jun 12, 2018 at 2:37 AM, Greg KH <gregkh at linuxfoundation.org> wrote:
> On Tue, Jun 12, 2018 at 02:30:27AM +0530, Souptick Joarder wrote:
>> > >
>> > > If no further comment, we would like to get this patch in 4.18-rc-X.
>> >
>> > Why?  Is it a regression fix?  That's all that is allowed after -rc1.
>>
>> No, this is not regression fix. We need to get this into 4.18-rc-1.  But
>> mostly it can't make into linus tree in rc-1 :)
>
> Why does it _have_ to get into 4.18-rc1?  My tree is long-closed and
> Linus already has all of my patches in his tree for the staging section
> of the kernel.
>
>> > And have you tried applying it to Linus's current tree?  :)
>>
>> Last tested on 4.17-rc-6 and it worked fine. Let me verify in current tree.
>
> Try it, you might be surprised :)

Yes, got the surprise :)
Sorry for making noise, I will drop this patch as it is no more valid
in current Linus's tree.

From David.Laight at ACULAB.COM  Wed Jun 13 10:53:07 2018
From: David.Laight at ACULAB.COM (David Laight)
Date: Wed, 13 Jun 2018 10:53:07 +0000
Subject: [lustre-devel] [PATCH] staging: lustre: add error handling for
 try_module_get
In-Reply-To: <1528778968-42225-1-git-send-email-jiazhouyang09@gmail.com>
References: <1528778968-42225-1-git-send-email-jiazhouyang09@gmail.com>
Message-ID: <ed5d91f54b5a4776a66df87b645b0ee9@AcuMS.aculab.com>

From: Zhouyang Jia
> Sent: 12 June 2018 05:49
> 
> When try_module_get fails, the lack of error-handling code may
> cause unexpected results.
> 
> This patch adds error-handling code after calling try_module_get.
...
> +++ b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c
> @@ -2422,7 +2422,10 @@ ksocknal_base_startup(void)
> 
>  	/* flag lists/ptrs/locks initialised */
>  	ksocknal_data.ksnd_init = SOCKNAL_INIT_DATA;
> -	try_module_get(THIS_MODULE);
> +	if (!try_module_get(THIS_MODULE)) {
> +		CERROR("%s: cannot get module\n", __func__);
> +		goto failed;
> +	}


Can try_module_get(THIS_MODULE) ever fail?
Since you are running code in 'THIS_MODULE' the caller must have a
reference that can't go away.
So try_module_get() just increments the count that is already greater
than zero.

Similarly module_put(THIS_MODULE) must never be able to release the
last reference.
Any such calls that aren't in error paths after try_module_get() are
probably buggy.

	David


From jrdr.linux at gmail.com  Thu Jun 14 11:13:16 2018
From: jrdr.linux at gmail.com (Souptick Joarder)
Date: Thu, 14 Jun 2018 16:43:16 +0530
Subject: [lustre-devel] [PATCH v3] staging: lustre: Change return type
 to vm_fault_t
In-Reply-To: <87lgbhhlme.fsf@notabene.neil.brown.name>
References: <20180521180937.GA19605@jordon-HP-15-Notebook-PC>
 <CAFqt6zamp=+adkyM+_4z405hN-9qOn1CsY_BgVtCrAfscBHhWg@mail.gmail.com>
 <20180611205058.GA30123@kroah.com>
 <CAFqt6zZJcN4=FDQSqiSznoEogavV9GBpqxx-onWG5Q3wQEM9cQ@mail.gmail.com>
 <87lgbhhlme.fsf@notabene.neil.brown.name>
Message-ID: <CAFqt6zYn2FjX2Sy5CFmMqvwPxys=SamMy+7=14dVNCR0Kt6R7w@mail.gmail.com>

On Thu, Jun 14, 2018 at 12:59 PM, NeilBrown <neilb at suse.com> wrote:
> On Tue, Jun 12 2018, Souptick Joarder wrote:
>
>> On 12-Jun-2018 2:21 AM, "Greg KH" <gregkh at linuxfoundation.org> wrote:
>>>
>>> On Tue, Jun 12, 2018 at 02:00:47AM +0530, Souptick Joarder wrote:
>>> > On Mon, May 21, 2018 at 11:39 PM, Souptick Joarder <jrdr.linux at gmail.com>
>>> >
>>> > If no further comment, we would like to get this patch in 4.18-rc-X.
>>>
>>> Why?  Is it a regression fix?  That's all that is allowed after -rc1.
>>
>> No, this is not regression fix. We need to get this into 4.18-rc-1.  But
>> mostly it can't make into linus tree in rc-1 :)
>>>
>>> And have you tried applying it to Linus's current tree?  :)
>>
>> Last tested on 4.17-rc-6 and it worked fine. Let me verify in current tree.
>>
>
> As you have undoubtedly noticed, lustre is no longer in Linus' tree.
> I'm experimenting with maintaining a branch which retains the code
> (lustre/* in github.com/neilbrown/linux) so we can get it ready for
> merging properly.
> I've added you patch to my tree.

You need to add this patch in your tree as well.
https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/commit/?h=next-20180614&id=1c8f422059ae5da07db7406ab916203f9417e396

This patch appears to be missing in your github branch.

From jsimmons at infradead.org  Sat Jun 16 00:17:37 2018
From: jsimmons at infradead.org (James Simmons)
Date: Sat, 16 Jun 2018 01:17:37 +0100 (BST)
Subject: [lustre-devel] [PATCH 08/24] lustre: use truncate_inode_page in
 place of truncate_complete_page
In-Reply-To: <152904669032.10587.11474850109829193046.stgit@noble>
References: <152904663333.10587.10934053155404014785.stgit@noble>
 <152904669032.10587.11474850109829193046.stgit@noble>
Message-ID: <alpine.LFD.2.21.1806160115430.15679@casper.infradead.org>


> Carrying a local truncate_complete_page() is not good
> for maintainability.  Linux now used truncate_inode_page,
> so use that instead.
> For correct use, we need to pass the mapping to mdc_release_page()
> rather than trusting page->mapping.

Nak: truncate_inode_page is not exported so it fails to build when lustre
is configured as a module.
 
> Signed-off-by: NeilBrown <neilb at suse.com>
> ---
>  .../lustre/include/lustre_patchless_compat.h       |   16 ----------------
>  drivers/staging/lustre/lustre/llite/dir.c          |    3 +--
>  .../staging/lustre/lustre/llite/llite_internal.h   |    4 ++--
>  drivers/staging/lustre/lustre/mdc/mdc_request.c    |   12 ++++++------
>  4 files changed, 9 insertions(+), 26 deletions(-)
> 
> diff --git a/drivers/staging/lustre/lustre/include/lustre_patchless_compat.h b/drivers/staging/lustre/lustre/include/lustre_patchless_compat.h
> index 298476ea7557..dca1e3dbd183 100644
> --- a/drivers/staging/lustre/lustre/include/lustre_patchless_compat.h
> +++ b/drivers/staging/lustre/lustre/include/lustre_patchless_compat.h
> @@ -41,22 +41,6 @@
>  #include <linux/hash.h>
>  #include <linux/pagemap.h>
>  
> -#define ll_delete_from_page_cache(page) delete_from_page_cache(page)
> -
> -static inline void
> -truncate_complete_page(struct address_space *mapping, struct page *page)
> -{
> -	if (page->mapping != mapping)
> -		return;
> -
> -	if (PagePrivate(page))
> -		page->mapping->a_ops->invalidatepage(page, 0, PAGE_SIZE);
> -
> -	cancel_dirty_page(page);
> -	ClearPageMappedToDisk(page);
> -	ll_delete_from_page_cache(page);
> -}
> -
>  #ifndef ATTR_CTIME_SET
>  /*
>   * set ATTR_CTIME_SET to a high value to avoid any risk of collision with other
> diff --git a/drivers/staging/lustre/lustre/llite/dir.c b/drivers/staging/lustre/lustre/llite/dir.c
> index 688dddf3ca47..cc5869cf9f93 100644
> --- a/drivers/staging/lustre/lustre/llite/dir.c
> +++ b/drivers/staging/lustre/lustre/llite/dir.c
> @@ -167,8 +167,7 @@ void ll_release_page(struct inode *inode, struct page *page, bool remove)
>  
>  	if (remove) {
>  		lock_page(page);
> -		if (likely(page->mapping))
> -			truncate_complete_page(page->mapping, page);
> +		truncate_inode_page(inode->i_mapping, page);
>  		unlock_page(page);
>  	}
>  	put_page(page);
> diff --git a/drivers/staging/lustre/lustre/llite/llite_internal.h b/drivers/staging/lustre/lustre/llite/llite_internal.h
> index c08a6e14b6d7..28cff58690d6 100644
> --- a/drivers/staging/lustre/lustre/llite/llite_internal.h
> +++ b/drivers/staging/lustre/lustre/llite/llite_internal.h
> @@ -938,11 +938,11 @@ static inline void ll_invalidate_page(struct page *vmpage)
>  		return;
>  
>  	/*
> -	 * truncate_complete_page() calls
> +	 * truncate_inode_page() calls
>  	 * a_ops->invalidatepage()->cl_page_delete()->vvp_page_delete().
>  	 */
>  	ll_teardown_mmaps(mapping, offset, offset + PAGE_SIZE);
> -	truncate_complete_page(mapping, vmpage);
> +	truncate_inode_page(mapping, vmpage);
>  }
>  
>  #define    ll_s2sbi(sb)	(s2lsi(sb)->lsi_llsbi)
> diff --git a/drivers/staging/lustre/lustre/mdc/mdc_request.c b/drivers/staging/lustre/lustre/mdc/mdc_request.c
> index cff31cb0a9ac..856e6a376852 100644
> --- a/drivers/staging/lustre/lustre/mdc/mdc_request.c
> +++ b/drivers/staging/lustre/lustre/mdc/mdc_request.c
> @@ -922,12 +922,12 @@ static int mdc_getpage(struct obd_export *exp, const struct lu_fid *fid,
>  	return 0;
>  }
>  
> -static void mdc_release_page(struct page *page, int remove)
> +static void mdc_release_page(struct address_space *mapping,
> +			     struct page *page, int remove)
>  {
>  	if (remove) {
>  		lock_page(page);
> -		if (likely(page->mapping))
> -			truncate_complete_page(page->mapping, page);
> +		truncate_inode_page(mapping, page);
>  		unlock_page(page);
>  	}
>  	put_page(page);
> @@ -981,7 +981,7 @@ static struct page *mdc_page_locate(struct address_space *mapping, __u64 *hash,
>  			       offset, *start, *end, *hash);
>  			if (*hash > *end) {
>  				kunmap(page);
> -				mdc_release_page(page, 0);
> +				mdc_release_page(mapping, page, 0);
>  				page = NULL;
>  			} else if (*end != *start && *hash == *end) {
>  				/*
> @@ -991,7 +991,7 @@ static struct page *mdc_page_locate(struct address_space *mapping, __u64 *hash,
>  				 * fetch the page we want.
>  				 */
>  				kunmap(page);
> -				mdc_release_page(page,
> +				mdc_release_page(mapping, page,
>  						 le32_to_cpu(dp->ldp_flags) & LDF_COLLIDE);
>  				page = NULL;
>  			}
> @@ -1365,7 +1365,7 @@ static int mdc_read_page(struct obd_export *exp, struct md_op_data *op_data,
>  	return rc;
>  fail:
>  	kunmap(page);
> -	mdc_release_page(page, 1);
> +	mdc_release_page(mapping, page, 1);
>  	rc = -EIO;
>  	goto out_unlock;
>  }
> 
> 
> 

From neilb at suse.com  Sat Jun 16 00:52:31 2018
From: neilb at suse.com (NeilBrown)
Date: Sat, 16 Jun 2018 10:52:31 +1000
Subject: [lustre-devel] [PATCH 08/24] lustre: use truncate_inode_page in
 place of truncate_complete_page
In-Reply-To: <alpine.LFD.2.21.1806160115430.15679@casper.infradead.org>
References: <152904663333.10587.10934053155404014785.stgit@noble>
 <152904669032.10587.11474850109829193046.stgit@noble>
 <alpine.LFD.2.21.1806160115430.15679@casper.infradead.org>
Message-ID: <87y3ffft80.fsf@notabene.neil.brown.name>

On Sat, Jun 16 2018, James Simmons wrote:

>> Carrying a local truncate_complete_page() is not good
>> for maintainability.  Linux now used truncate_inode_page,
>> so use that instead.
>> For correct use, we need to pass the mapping to mdc_release_page()
>> rather than trusting page->mapping.
>
> Nak: truncate_inode_page is not exported so it fails to build when lustre
> is configured as a module.

Bother, nor it is.
We could probably use truncate_inode_pages_range() but it isn't a
perfect fit, which seems to suggest that lustre is trying to do
something that no other filesystem finds the need to do.
I'll dig deeper and work out what is really going on.

Thanks,
NeilBrown
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180616/0991e7c2/attachment.sig>

From green at linuxhacker.ru  Sat Jun 16 00:59:43 2018
From: green at linuxhacker.ru (Oleg Drokin)
Date: Fri, 15 Jun 2018 20:59:43 -0400
Subject: [lustre-devel] [PATCH 08/24] lustre: use truncate_inode_page in
 place of truncate_complete_page
In-Reply-To: <87y3ffft80.fsf@notabene.neil.brown.name>
References: <152904663333.10587.10934053155404014785.stgit@noble>
 <152904669032.10587.11474850109829193046.stgit@noble>
 <alpine.LFD.2.21.1806160115430.15679@casper.infradead.org>
 <87y3ffft80.fsf@notabene.neil.brown.name>
Message-ID: <61C2F92E-8F88-49C6-83FB-F3013289265C@linuxhacker.ru>


> On Jun 15, 2018, at 8:52 PM, NeilBrown <neilb at suse.com> wrote:
> 
> On Sat, Jun 16 2018, James Simmons wrote:
> 
>>> Carrying a local truncate_complete_page() is not good
>>> for maintainability.  Linux now used truncate_inode_page,
>>> so use that instead.
>>> For correct use, we need to pass the mapping to mdc_release_page()
>>> rather than trusting page->mapping.
>> 
>> Nak: truncate_inode_page is not exported so it fails to build when lustre
>> is configured as a module.
> 
> Bother, nor it is.
> We could probably use truncate_inode_pages_range() but it isn't a
> perfect fit, which seems to suggest that lustre is trying to do
> something that no other filesystem finds the need to do.
> I'll dig deeper and work out what is really going on.

Lustre evicts pages from mapping when the lock that covers the pages
is canceled.
Normally this only applies to data pages, and I see this is in mdc which is for metadata.

For metadata the only pages we hold are readdir pages and the lock currently
covers entire dir so lock cancelation should not need this.

So looking at the code the other two users are readdir where we failed to populate page
somehow and now need to throw it out and then a collision event for hashes, it appears.
Both of these should apply to other filesystems I am sure.



From jsimmons at infradead.org  Sat Jun 16 03:00:30 2018
From: jsimmons at infradead.org (James Simmons)
Date: Sat, 16 Jun 2018 04:00:30 +0100 (BST)
Subject: [lustre-devel] [PATCH 01/11] staging: lustre: simplify use of
 interval-tree.
In-Reply-To: <152826511890.16761.16115276596203531205.stgit@noble>
References: <152826510267.16761.14361003167157833896.stgit@noble>
 <152826511890.16761.16115276596203531205.stgit@noble>
Message-ID: <alpine.LFD.2.21.1806160344210.6450@casper.infradead.org>


> Lustre has a private interval-tree implementation.  This
> implementation (inexplicably) refuses to insert an interval if an
> identical interval already exists.  It is OK with all sorts of
> overlapping intervals, but identical intervals are rejected.

I talked to Oleg about this since this changes the behavior. He is worried
about having identical items that would end up being merged.  If we can 
guarantee by some other means there are no identical nodes, we are 
probably fine with the interval tree code allowing this. Oleg can explain 
better than me in this case.
 
> Both users of interval-tree in lustre would be simpler if this was not
> the case.  They need to store all intervals, even if some are
> identical.
> 
> llite/range_lock.c add a rl_next_lock list_head to each lock.
> If it cannot insert a new lock because the range is in use, it
> attached the new lock to the existing lock using rl_next_lock.
> This requires extra code to iterate over the rl_next_lock lists when
> iterating over locks, and to update the list when deleting a lock from
> the tree.
> 
> ldlm_extend allocates a separate ldlm_interval which as a list of
> ldlm_locks which share the same interval.  This is linked together
> by over-loading the l_sl_policy which, for non-extent locks, is used
> for linking together locks with the same policy.
> This doesn't only require extra code, but also an extra memory
> allocation.
> 
> This patch removes all that complexity.
> - interval_insert() now never fails.

Its not really a failure. What it does is if it finds a already existing
node with the range requested it returns the already existing node 
pointer. If not it just creates a new node and returns NULL. Sometimes
identical request can happen. A good example of this is with HSM request
on the MDS server. In that case sometimes we get identical progress
reports which we want to filter out so not add the same data.

> - consequently rl_next_lock is always empty and
>   rl_lock_count is always zero. so they are removed
> - every ldlm_lock has linked directly into the
>   interval tree, so each has an embedded interval_node
>   rather than a pointer to a 'struct ldlm_interval'
> - ldlm_interval is now unused, so it is gone as it
>   the kmemcache from which they were allocated.
> - the various functions for allocating an ldlm_interval
>   and attaching to a lock or detaching from a lock
>   are also gone.
> 
> Signed-off-by: NeilBrown <neilb at suse.com>
> ---
>  .../staging/lustre/lustre/include/interval_tree.h  |    4 +
>  drivers/staging/lustre/lustre/include/lustre_dlm.h |   12 ---
>  drivers/staging/lustre/lustre/ldlm/interval_tree.c |   13 +--
>  drivers/staging/lustre/lustre/ldlm/ldlm_extent.c   |   76 ++------------------
>  drivers/staging/lustre/lustre/ldlm/ldlm_internal.h |   17 ----
>  drivers/staging/lustre/lustre/ldlm/ldlm_lock.c     |   25 +------
>  drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c    |    9 --
>  drivers/staging/lustre/lustre/llite/range_lock.c   |   59 +---------------
>  drivers/staging/lustre/lustre/llite/range_lock.h   |    8 --
>  9 files changed, 17 insertions(+), 206 deletions(-)
> 
> diff --git a/drivers/staging/lustre/lustre/include/interval_tree.h b/drivers/staging/lustre/lustre/include/interval_tree.h
> index 7d119c1a0469..bcda74fc7875 100644
> --- a/drivers/staging/lustre/lustre/include/interval_tree.h
> +++ b/drivers/staging/lustre/lustre/include/interval_tree.h
> @@ -100,8 +100,8 @@ static inline int interval_set(struct interval_node *node,
>  typedef enum interval_iter (*interval_callback_t)(struct interval_node *node,
>  						  void *args);
>  
> -struct interval_node *interval_insert(struct interval_node *node,
> -				      struct interval_node **root);
> +void interval_insert(struct interval_node *node,
> +		     struct interval_node **root);
>  void interval_erase(struct interval_node *node, struct interval_node **root);
>  
>  /*
> diff --git a/drivers/staging/lustre/lustre/include/lustre_dlm.h b/drivers/staging/lustre/lustre/include/lustre_dlm.h
> index 2c55241258cc..baeb8c63352b 100644
> --- a/drivers/staging/lustre/lustre/include/lustre_dlm.h
> +++ b/drivers/staging/lustre/lustre/include/lustre_dlm.h
> @@ -513,16 +513,6 @@ struct ldlm_glimpse_work {
>  /** The ldlm_glimpse_work is allocated on the stack and should not be freed. */
>  #define LDLM_GL_WORK_NOFREE 0x1
>  
> -/** Interval node data for each LDLM_EXTENT lock. */
> -struct ldlm_interval {
> -	struct interval_node	li_node;  /* node for tree management */
> -	struct list_head	li_group; /* the locks which have the same
> -					   * policy - group of the policy
> -					   */
> -};
> -
> -#define to_ldlm_interval(n) container_of(n, struct ldlm_interval, li_node)
> -
>  /**
>   * Interval tree for extent locks.
>   * The interval tree must be accessed under the resource lock.
> @@ -631,7 +621,7 @@ struct ldlm_lock {
>  	/**
>  	 * Tree node for ldlm_extent.
>  	 */
> -	struct ldlm_interval	*l_tree_node;
> +	struct interval_node	l_tree_node;
>  	/**
>  	 * Requested mode.
>  	 * Protected by lr_lock.
> diff --git a/drivers/staging/lustre/lustre/ldlm/interval_tree.c b/drivers/staging/lustre/lustre/ldlm/interval_tree.c
> index 8df7a4463c21..f5232059d1b1 100644
> --- a/drivers/staging/lustre/lustre/ldlm/interval_tree.c
> +++ b/drivers/staging/lustre/lustre/ldlm/interval_tree.c
> @@ -97,11 +97,6 @@ static inline int extent_overlapped(struct interval_node_extent *e1,
>  	return (e1->start <= e2->end) && (e2->start <= e1->end);
>  }
>  
> -static inline int node_equal(struct interval_node *n1, struct interval_node *n2)
> -{
> -	return extent_equal(&n1->in_extent, &n2->in_extent);
> -}
> -
>  static struct interval_node *interval_first(struct interval_node *node)
>  {
>  	if (!node)
> @@ -299,8 +294,8 @@ static void interval_insert_color(struct interval_node *node,
>  	(*root)->in_color = INTERVAL_BLACK;
>  }
>  
> -struct interval_node *interval_insert(struct interval_node *node,
> -				      struct interval_node **root)
> +void interval_insert(struct interval_node *node,
> +		     struct interval_node **root)
>  
>  {
>  	struct interval_node **p, *parent = NULL;
> @@ -309,8 +304,6 @@ struct interval_node *interval_insert(struct interval_node *node,
>  	p = root;
>  	while (*p) {
>  		parent = *p;
> -		if (node_equal(parent, node))
> -			return parent;
>  
>  		/* max_high field must be updated after each iteration */
>  		if (parent->in_max_high < interval_high(node))
> @@ -331,8 +324,6 @@ struct interval_node *interval_insert(struct interval_node *node,
>  
>  	interval_insert_color(node, root);
>  	node->in_intree = 1;
> -
> -	return NULL;
>  }
>  EXPORT_SYMBOL(interval_insert);
>  
> diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_extent.c b/drivers/staging/lustre/lustre/ldlm/ldlm_extent.c
> index 4da23ade2bb3..2f4c305bb340 100644
> --- a/drivers/staging/lustre/lustre/ldlm/ldlm_extent.c
> +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_extent.c
> @@ -92,55 +92,6 @@ __u64 ldlm_extent_shift_kms(struct ldlm_lock *lock, __u64 old_kms)
>  }
>  EXPORT_SYMBOL(ldlm_extent_shift_kms);
>  
> -struct kmem_cache *ldlm_interval_slab;
> -
> -/* interval tree, for LDLM_EXTENT. */
> -static void ldlm_interval_attach(struct ldlm_interval *n, struct ldlm_lock *l)
> -{
> -	LASSERT(!l->l_tree_node);
> -	LASSERT(l->l_resource->lr_type == LDLM_EXTENT);
> -
> -	list_add_tail(&l->l_sl_policy, &n->li_group);
> -	l->l_tree_node = n;
> -}
> -
> -struct ldlm_interval *ldlm_interval_alloc(struct ldlm_lock *lock)
> -{
> -	struct ldlm_interval *node;
> -
> -	LASSERT(lock->l_resource->lr_type == LDLM_EXTENT);
> -	node = kmem_cache_zalloc(ldlm_interval_slab, GFP_NOFS);
> -	if (!node)
> -		return NULL;
> -
> -	INIT_LIST_HEAD(&node->li_group);
> -	ldlm_interval_attach(node, lock);
> -	return node;
> -}
> -
> -void ldlm_interval_free(struct ldlm_interval *node)
> -{
> -	if (node) {
> -		LASSERT(list_empty(&node->li_group));
> -		LASSERT(!interval_is_intree(&node->li_node));
> -		kmem_cache_free(ldlm_interval_slab, node);
> -	}
> -}
> -
> -struct ldlm_interval *ldlm_interval_detach(struct ldlm_lock *l)
> -{
> -	struct ldlm_interval *n = l->l_tree_node;
> -
> -	if (!n)
> -		return NULL;
> -
> -	LASSERT(!list_empty(&n->li_group));
> -	l->l_tree_node = NULL;
> -	list_del_init(&l->l_sl_policy);
> -
> -	return list_empty(&n->li_group) ? n : NULL;
> -}
> -
>  static inline int lock_mode_to_index(enum ldlm_mode mode)
>  {
>  	int index;
> @@ -157,16 +108,13 @@ static inline int lock_mode_to_index(enum ldlm_mode mode)
>  void ldlm_extent_add_lock(struct ldlm_resource *res,
>  			  struct ldlm_lock *lock)
>  {
> -	struct interval_node *found, **root;
> -	struct ldlm_interval *node;
> +	struct interval_node **root;
>  	struct ldlm_extent *extent;
>  	int idx, rc;
>  
>  	LASSERT(lock->l_granted_mode == lock->l_req_mode);
>  
> -	node = lock->l_tree_node;
> -	LASSERT(node);
> -	LASSERT(!interval_is_intree(&node->li_node));
> +	LASSERT(!interval_is_intree(&lock->l_tree_node));
>  
>  	idx = lock_mode_to_index(lock->l_granted_mode);
>  	LASSERT(lock->l_granted_mode == 1 << idx);
> @@ -174,18 +122,11 @@ void ldlm_extent_add_lock(struct ldlm_resource *res,
>  
>  	/* node extent initialize */
>  	extent = &lock->l_policy_data.l_extent;
> -	rc = interval_set(&node->li_node, extent->start, extent->end);
> +	rc = interval_set(&lock->l_tree_node, extent->start, extent->end);
>  	LASSERT(!rc);
>  
>  	root = &res->lr_itree[idx].lit_root;
> -	found = interval_insert(&node->li_node, root);
> -	if (found) { /* The policy group found. */
> -		struct ldlm_interval *tmp;
> -
> -		tmp = ldlm_interval_detach(lock);
> -		ldlm_interval_free(tmp);
> -		ldlm_interval_attach(to_ldlm_interval(found), lock);
> -	}
> +	interval_insert(&lock->l_tree_node, root);
>  	res->lr_itree[idx].lit_size++;
>  
>  	/* even though we use interval tree to manage the extent lock, we also
> @@ -219,11 +160,10 @@ void ldlm_extent_add_lock(struct ldlm_resource *res,
>  void ldlm_extent_unlink_lock(struct ldlm_lock *lock)
>  {
>  	struct ldlm_resource *res = lock->l_resource;
> -	struct ldlm_interval *node = lock->l_tree_node;
>  	struct ldlm_interval_tree *tree;
>  	int idx;
>  
> -	if (!node || !interval_is_intree(&node->li_node)) /* duplicate unlink */
> +	if (!interval_is_intree(&lock->l_tree_node)) /* duplicate unlink */
>  		return;
>  
>  	idx = lock_mode_to_index(lock->l_granted_mode);
> @@ -233,11 +173,7 @@ void ldlm_extent_unlink_lock(struct ldlm_lock *lock)
>  	LASSERT(tree->lit_root); /* assure the tree is not null */
>  
>  	tree->lit_size--;
> -	node = ldlm_interval_detach(lock);
> -	if (node) {
> -		interval_erase(&node->li_node, &tree->lit_root);
> -		ldlm_interval_free(node);
> -	}
> +	interval_erase(&lock->l_tree_node, &tree->lit_root);
>  }
>  
>  void ldlm_extent_policy_wire_to_local(const union ldlm_wire_policy_data *wpolicy,
> diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_internal.h b/drivers/staging/lustre/lustre/ldlm/ldlm_internal.h
> index bc33ca100620..159de8a59cbb 100644
> --- a/drivers/staging/lustre/lustre/ldlm/ldlm_internal.h
> +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_internal.h
> @@ -189,23 +189,6 @@ __u64 ldlm_pool_get_slv(struct ldlm_pool *pl);
>  void ldlm_pool_set_clv(struct ldlm_pool *pl, __u64 clv);
>  __u32 ldlm_pool_get_lvf(struct ldlm_pool *pl);
>  
> -/* interval tree, for LDLM_EXTENT. */
> -extern struct kmem_cache *ldlm_interval_slab; /* slab cache for ldlm_interval */
> -struct ldlm_interval *ldlm_interval_detach(struct ldlm_lock *l);
> -struct ldlm_interval *ldlm_interval_alloc(struct ldlm_lock *lock);
> -void ldlm_interval_free(struct ldlm_interval *node);
> -/* this function must be called with res lock held */
> -static inline struct ldlm_extent *
> -ldlm_interval_extent(struct ldlm_interval *node)
> -{
> -	struct ldlm_lock *lock;
> -
> -	LASSERT(!list_empty(&node->li_group));
> -
> -	lock = list_entry(node->li_group.next, struct ldlm_lock, l_sl_policy);
> -	return &lock->l_policy_data.l_extent;
> -}
> -
>  int ldlm_init(void);
>  void ldlm_exit(void);
>  
> diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c b/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c
> index a644d133063b..13b1b5fdada9 100644
> --- a/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c
> +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c
> @@ -185,7 +185,6 @@ void ldlm_lock_put(struct ldlm_lock *lock)
>  
>  		kfree(lock->l_lvb_data);
>  
> -		ldlm_interval_free(ldlm_interval_detach(lock));
>  		lu_ref_fini(&lock->l_reference);
>  		OBD_FREE_RCU(lock, sizeof(*lock), &lock->l_handle);
>  	}
> @@ -1138,17 +1137,10 @@ static int lock_matches(struct ldlm_lock *lock, struct lock_match_data *data)
>  
>  static enum interval_iter itree_overlap_cb(struct interval_node *in, void *args)
>  {
> -	struct ldlm_interval *node = to_ldlm_interval(in);
>  	struct lock_match_data *data = args;
> -	struct ldlm_lock *lock;
> -	int rc;
> +	struct ldlm_lock *lock = container_of(in, struct ldlm_lock, l_tree_node);
>  
> -	list_for_each_entry(lock, &node->li_group, l_sl_policy) {
> -		rc = lock_matches(lock, data);
> -		if (rc == INTERVAL_ITER_STOP)
> -			return INTERVAL_ITER_STOP;
> -	}
> -	return INTERVAL_ITER_CONT;
> +	return lock_matches(lock, data);
>  }
>  
>  /**
> @@ -1564,15 +1556,6 @@ struct ldlm_lock *ldlm_lock_create(struct ldlm_namespace *ns,
>  		lock->l_glimpse_ast = cbs->lcs_glimpse;
>  	}
>  
> -	lock->l_tree_node = NULL;
> -	/* if this is the extent lock, allocate the interval tree node */
> -	if (type == LDLM_EXTENT) {
> -		if (!ldlm_interval_alloc(lock)) {
> -			rc = -ENOMEM;
> -			goto out;
> -		}
> -	}
> -
>  	if (lvb_len) {
>  		lock->l_lvb_len = lvb_len;
>  		lock->l_lvb_data = kzalloc(lvb_len, GFP_NOFS);
> @@ -1625,10 +1608,6 @@ enum ldlm_error ldlm_lock_enqueue(struct ldlm_namespace *ns,
>  
>  	ldlm_resource_unlink_lock(lock);
>  
> -	/* Cannot happen unless on the server */
> -	if (res->lr_type == LDLM_EXTENT && !lock->l_tree_node)
> -		LBUG();
> -
>  	/* Some flags from the enqueue want to make it into the AST, via the
>  	 * lock's l_flags.
>  	 */
> diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c b/drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c
> index 5963e90d0938..f410ef6c02ef 100644
> --- a/drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c
> +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c
> @@ -1134,14 +1134,6 @@ int ldlm_init(void)
>  		return -ENOMEM;
>  	}
>  
> -	ldlm_interval_slab = kmem_cache_create("interval_node",
> -					       sizeof(struct ldlm_interval),
> -					       0, SLAB_HWCACHE_ALIGN, NULL);
> -	if (!ldlm_interval_slab) {
> -		kmem_cache_destroy(ldlm_resource_slab);
> -		kmem_cache_destroy(ldlm_lock_slab);
> -		return -ENOMEM;
> -	}
>  #if LUSTRE_TRACKS_LOCK_EXP_REFS
>  	class_export_dump_hook = ldlm_dump_export_locks;
>  #endif
> @@ -1159,5 +1151,4 @@ void ldlm_exit(void)
>  	 */
>  	synchronize_rcu();
>  	kmem_cache_destroy(ldlm_lock_slab);
> -	kmem_cache_destroy(ldlm_interval_slab);
>  }
> diff --git a/drivers/staging/lustre/lustre/llite/range_lock.c b/drivers/staging/lustre/lustre/llite/range_lock.c
> index 008a8874118d..eaa23f4c414e 100644
> --- a/drivers/staging/lustre/lustre/llite/range_lock.c
> +++ b/drivers/staging/lustre/lustre/llite/range_lock.c
> @@ -74,19 +74,12 @@ int range_lock_init(struct range_lock *lock, __u64 start, __u64 end)
>  	if (rc)
>  		return rc;
>  
> -	INIT_LIST_HEAD(&lock->rl_next_lock);
>  	lock->rl_task = NULL;
> -	lock->rl_lock_count = 0;
>  	lock->rl_blocking_ranges = 0;
>  	lock->rl_sequence = 0;
>  	return rc;
>  }
>  
> -static inline struct range_lock *next_lock(struct range_lock *lock)
> -{
> -	return list_entry(lock->rl_next_lock.next, typeof(*lock), rl_next_lock);
> -}
> -
>  /**
>   * Helper function of range_unlock()
>   *
> @@ -102,14 +95,7 @@ static enum interval_iter range_unlock_cb(struct interval_node *node, void *arg)
>  {
>  	struct range_lock *lock = arg;
>  	struct range_lock *overlap = node2rangelock(node);
> -	struct range_lock *iter;
>  
> -	list_for_each_entry(iter, &overlap->rl_next_lock, rl_next_lock) {
> -		if (iter->rl_sequence > lock->rl_sequence) {
> -			--iter->rl_blocking_ranges;
> -			LASSERT(iter->rl_blocking_ranges > 0);
> -		}
> -	}
>  	if (overlap->rl_sequence > lock->rl_sequence) {
>  		--overlap->rl_blocking_ranges;
>  		if (overlap->rl_blocking_ranges == 0)
> @@ -131,32 +117,8 @@ static enum interval_iter range_unlock_cb(struct interval_node *node, void *arg)
>  void range_unlock(struct range_lock_tree *tree, struct range_lock *lock)
>  {
>  	spin_lock(&tree->rlt_lock);
> -	if (!list_empty(&lock->rl_next_lock)) {
> -		struct range_lock *next;
> -
> -		if (interval_is_intree(&lock->rl_node)) { /* first lock */
> -			/* Insert the next same range lock into the tree */
> -			next = next_lock(lock);
> -			next->rl_lock_count = lock->rl_lock_count - 1;
> -			interval_erase(&lock->rl_node, &tree->rlt_root);
> -			interval_insert(&next->rl_node, &tree->rlt_root);
> -		} else {
> -			/* find the first lock in tree */
> -			list_for_each_entry(next, &lock->rl_next_lock,
> -					    rl_next_lock) {
> -				if (!interval_is_intree(&next->rl_node))
> -					continue;
> -
> -				LASSERT(next->rl_lock_count > 0);
> -				next->rl_lock_count--;
> -				break;
> -			}
> -		}
> -		list_del_init(&lock->rl_next_lock);
> -	} else {
> -		LASSERT(interval_is_intree(&lock->rl_node));
> -		interval_erase(&lock->rl_node, &tree->rlt_root);
> -	}
> +	LASSERT(interval_is_intree(&lock->rl_node));
> +	interval_erase(&lock->rl_node, &tree->rlt_root);
>  
>  	interval_search(tree->rlt_root, &lock->rl_node.in_extent,
>  			range_unlock_cb, lock);
> @@ -177,9 +139,8 @@ void range_unlock(struct range_lock_tree *tree, struct range_lock *lock)
>  static enum interval_iter range_lock_cb(struct interval_node *node, void *arg)
>  {
>  	struct range_lock *lock = arg;
> -	struct range_lock *overlap = node2rangelock(node);
>  
> -	lock->rl_blocking_ranges += overlap->rl_lock_count + 1;
> +	lock->rl_blocking_ranges++;
>  	return INTERVAL_ITER_CONT;
>  }
>  
> @@ -198,7 +159,6 @@ static enum interval_iter range_lock_cb(struct interval_node *node, void *arg)
>   */
>  int range_lock(struct range_lock_tree *tree, struct range_lock *lock)
>  {
> -	struct interval_node *node;
>  	int rc = 0;
>  
>  	spin_lock(&tree->rlt_lock);
> @@ -208,18 +168,7 @@ int range_lock(struct range_lock_tree *tree, struct range_lock *lock)
>  	 */
>  	interval_search(tree->rlt_root, &lock->rl_node.in_extent,
>  			range_lock_cb, lock);
> -	/*
> -	 * Insert to the tree if I am unique, otherwise I've been linked to
> -	 * the rl_next_lock of another lock which has the same range as mine
> -	 * in range_lock_cb().
> -	 */
> -	node = interval_insert(&lock->rl_node, &tree->rlt_root);
> -	if (node) {
> -		struct range_lock *tmp = node2rangelock(node);
> -
> -		list_add_tail(&lock->rl_next_lock, &tmp->rl_next_lock);
> -		tmp->rl_lock_count++;
> -	}
> +	interval_insert(&lock->rl_node, &tree->rlt_root);
>  	lock->rl_sequence = ++tree->rlt_sequence;
>  
>  	while (lock->rl_blocking_ranges > 0) {
> diff --git a/drivers/staging/lustre/lustre/llite/range_lock.h b/drivers/staging/lustre/lustre/llite/range_lock.h
> index 9ebac09160f2..10ef1a995d26 100644
> --- a/drivers/staging/lustre/lustre/llite/range_lock.h
> +++ b/drivers/staging/lustre/lustre/llite/range_lock.h
> @@ -46,14 +46,6 @@ struct range_lock {
>  	 * Process to enqueue this lock.
>  	 */
>  	struct task_struct	*rl_task;
> -	/**
> -	 * List of locks with the same range.
> -	 */
> -	struct list_head	rl_next_lock;
> -	/**
> -	 * Number of locks in the list rl_next_lock
> -	 */
> -	unsigned int		rl_lock_count;
>  	/**
>  	 * Number of ranges which are blocking acquisition of the lock
>  	 */
> 
> 
> 

From jsimmons at infradead.org  Sat Jun 16 14:33:59 2018
From: jsimmons at infradead.org (James Simmons)
Date: Sat, 16 Jun 2018 15:33:59 +0100 (BST)
Subject: [lustre-devel] testing lustre-testing
Message-ID: <alpine.LFD.2.21.1806161533400.8039@casper.infradead.org>


I tried your latest tree and it crash at start up with:

2018-06-16 10:16:01 [  150.547229] RIP: 
0010:deactivate_slab.isra.69+0x170/0x650
2018-06-16 10:16:01 [  150.553650] RSP: 0018:ffffc90000053a88 EFLAGS: 
00010082
2018-06-16 10:16:01 [  150.559871] RAX: 845da87dd009dac1 RBX: 
ffff8808535c0700 RCX: 000000018040002d
2018-06-16 10:16:01 [  150.567988] RDX: 000000018040002e RSI: 
0000000000000000 RDI: 0000000000000000
2018-06-16 10:16:01 [  150.576071] RBP: ffffc90000053b88 R08: 
ffff88085fc24050 R09: ffff88085f8000c0
2018-06-16 10:16:01 [  150.584131] R10: 0000000000000001 R11: 
0000000000000007 R12: ffffea00214d7000
2018-06-16 10:16:01 [  150.592163] R13: ffff88085f803800 R14: 
845da87dd009dac1 R15: 00000000014080c0
2018-06-16 10:16:01 [  150.600176]  ? deactivate_slab.isra.69+0x595/0x650
2018-06-16 10:16:01 [  150.605831]  ? deactivate_slab.isra.69+0x595/0x650
2018-06-16 10:16:01 [  150.611464]  ? get_page_from_freelist+0x335/0x1410
2018-06-16 10:16:01 [  150.617091]  ? deactivate_slab.isra.69+0x595/0x650
2018-06-16 10:16:01 [  150.622711]  ___slab_alloc+0x70/0x580
2018-06-16 10:16:01 [  150.627187]  ? __get_vm_area_node+0x7a/0x160
2018-06-16 10:16:01 [  150.632275]  ? ___slab_alloc+0x70/0x580
2018-06-16 10:16:01 [  150.636928]  ? _cond_resched+0x15/0x30
2018-06-16 10:16:01 [  150.641486]  ? 
kmem_cache_alloc_node_trace+0x1ab/0x1f0
2018-06-16 10:16:01 [  150.647439]  ? alloc_vmap_area+0x81/0x370
2018-06-16 10:16:01 [  150.652248]  __slab_alloc+0xe/0x12
2018-06-16 10:16:01 [  150.656428]  kmem_cache_alloc_node_trace+0xca/0x1f0

Not the easiest to track down to why its crashing. Also based on other 
patches in flight it looks like their are questions about that changes.

For me should I not based my patches on lustre-testing and test it their 
so it is stable. Would you do testing of the lustre branch in that case
or do I need to make sure my patches apply to your tree? Note some patches
I have will collide with what you are doing.

From neilb at suse.com  Sat Jun 16 22:49:40 2018
From: neilb at suse.com (NeilBrown)
Date: Sun, 17 Jun 2018 08:49:40 +1000
Subject: [lustre-devel] [PATCH 01/11] staging: lustre: simplify use of
 interval-tree.
In-Reply-To: <alpine.LFD.2.21.1806160344210.6450@casper.infradead.org>
References: <152826510267.16761.14361003167157833896.stgit@noble>
 <152826511890.16761.16115276596203531205.stgit@noble>
 <alpine.LFD.2.21.1806160344210.6450@casper.infradead.org>
Message-ID: <87sh5mfit7.fsf@notabene.neil.brown.name>

On Sat, Jun 16 2018, James Simmons wrote:

>> Lustre has a private interval-tree implementation.  This
>> implementation (inexplicably) refuses to insert an interval if an
>> identical interval already exists.  It is OK with all sorts of
>> overlapping intervals, but identical intervals are rejected.
>
> I talked to Oleg about this since this changes the behavior. He is worried
> about having identical items that would end up being merged.  If we can 
> guarantee by some other means there are no identical nodes, we are 
> probably fine with the interval tree code allowing this. Oleg can explain 
> better than me in this case.

I don't think this is a change in behaviour.
In the driver/staging client code, interval tree is being used in two
places and both of them have clumsy work-arounds for the fact that they
cannot insert duplicates in the interval tree.
The patch just cleans his up.

However if I have missed something, please provide details.
What "identical items" might get merged?

>  
>> Both users of interval-tree in lustre would be simpler if this was not
>> the case.  They need to store all intervals, even if some are
>> identical.
>> 
>> llite/range_lock.c add a rl_next_lock list_head to each lock.
>> If it cannot insert a new lock because the range is in use, it
>> attached the new lock to the existing lock using rl_next_lock.
>> This requires extra code to iterate over the rl_next_lock lists when
>> iterating over locks, and to update the list when deleting a lock from
>> the tree.
>> 
>> ldlm_extend allocates a separate ldlm_interval which as a list of
>> ldlm_locks which share the same interval.  This is linked together
>> by over-loading the l_sl_policy which, for non-extent locks, is used
>> for linking together locks with the same policy.
>> This doesn't only require extra code, but also an extra memory
>> allocation.
>> 
>> This patch removes all that complexity.
>> - interval_insert() now never fails.
>
> Its not really a failure. What it does is if it finds a already existing
> node with the range requested it returns the already existing node 
> pointer. If not it just creates a new node and returns NULL. Sometimes
> identical request can happen. A good example of this is with HSM request
> on the MDS server. In that case sometimes we get identical progress
> reports which we want to filter out so not add the same data.

This example is server-side code which is not a focus at present.
Having a quick look, it looks like it would be easy enough to do a
lookup first and then only insert if the lookup failed.
I think this is a much nicer approach than never allowing duplicates in
the interval tree.

Thanks,
NeilBrown
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180617/87d764cd/attachment.sig>

From gregkh at linuxfoundation.org  Sun Jun 17 11:22:03 2018
From: gregkh at linuxfoundation.org (Greg KH)
Date: Sun, 17 Jun 2018 13:22:03 +0200
Subject: [lustre-devel] [PATCH] Staging: lustre: Use kmemdup() instead
 of kzalloc and memcpy
In-Reply-To: <1529234215-4010-1-git-send-email-shreeya.patel23498@gmail.com>
References: <1529234215-4010-1-git-send-email-shreeya.patel23498@gmail.com>
Message-ID: <20180617112203.GA9357@kroah.com>

On Sun, Jun 17, 2018 at 04:46:55PM +0530, Shreeya Patel wrote:
> Replace calls to kzalloc or kmalloc followed by a memcpy with
> a direct call to kmemdup to shorten the code.
> 
> The Coccinelle semantic patch used to make this change is as follows:
> @@
> expression from,to,size,flag;
> statement S;
> @@
> 
> -  to = \(kmalloc\|kzalloc\)(size,flag);
> +  to = kmemdup(from,size,flag);
>    if (to==NULL || ...) S
> -  memcpy(to, from, size);
> 
> Signed-off-by: Shreeya Patel <shreeya.patel23498 at gmail.com>
> ---
>  drivers/staging/lustre/lnet/lnet/api-ni.c | 5 ++---

Always work off of the latest kernel tree.  Ideally linux-next or the
staging.git staging-next tree.  Worse case, Linus's tree.  But never off
of an old kernel tree, that will only cause you to duplicate work that
others have done :(

Please resync against 4.18-rc1 and see why I say this when trying to
apply your patch...

thanks,

greg k-h

From neilb at suse.com  Mon Jun 18 02:03:43 2018
From: neilb at suse.com (NeilBrown)
Date: Mon, 18 Jun 2018 12:03:43 +1000
Subject: [lustre-devel] [PATCH 08/24] lustre: use truncate_inode_page in
 place of truncate_complete_page
In-Reply-To: <61C2F92E-8F88-49C6-83FB-F3013289265C@linuxhacker.ru>
References: <152904663333.10587.10934053155404014785.stgit@noble>
 <152904669032.10587.11474850109829193046.stgit@noble>
 <alpine.LFD.2.21.1806160115430.15679@casper.infradead.org>
 <87y3ffft80.fsf@notabene.neil.brown.name>
 <61C2F92E-8F88-49C6-83FB-F3013289265C@linuxhacker.ru>
Message-ID: <877emwg8ao.fsf@notabene.neil.brown.name>

On Fri, Jun 15 2018, Oleg Drokin wrote:

>> On Jun 15, 2018, at 8:52 PM, NeilBrown <neilb at suse.com> wrote:
>> 
>> On Sat, Jun 16 2018, James Simmons wrote:
>> 
>>>> Carrying a local truncate_complete_page() is not good
>>>> for maintainability.  Linux now used truncate_inode_page,
>>>> so use that instead.
>>>> For correct use, we need to pass the mapping to mdc_release_page()
>>>> rather than trusting page->mapping.
>>> 
>>> Nak: truncate_inode_page is not exported so it fails to build when lustre
>>> is configured as a module.
>> 
>> Bother, nor it is.
>> We could probably use truncate_inode_pages_range() but it isn't a
>> perfect fit, which seems to suggest that lustre is trying to do
>> something that no other filesystem finds the need to do.
>> I'll dig deeper and work out what is really going on.
>
> Lustre evicts pages from mapping when the lock that covers the pages
> is canceled.
> Normally this only applies to data pages, and I see this is in mdc which is for metadata.

The invalidation of data pages is presumably where vvp_page_discard()
calls  ll_invalidate_page() [which is not the same as ll_invalidatepage() :-( ]

ll_invalidate_page does almost exactly the same as
truncate_inode_page(), which isn't exported.
We cannot just call
   truncate_inode_pages_range(page->mapping,
		page->index<<PAGE_SHIFT, page->index<<PAGE_SHIFT + 1);
as that will try to lock the page, which is already locked.

We could use generic_error_remove_page() instead (that is exported).
This condition isn't exactly an error, but it is a situation where
memory soon will not contain correct data.
afs uses this in afs_kill_pages() though the purpose is different.

cepfs uses it in writepages_finish in a context described as:

        /*
         * We lost the cache cap, need to truncate the page before
         * it is unlocked, otherwise we'd truncate it later in the
         * page truncation thread, possibly losing some data that
         * raced its way in
         */
which sounds similar to what lustre is doing.

NFS can only lose a lock (delegation) on the whole file, not for
individual pages, so it just invalidates a range.

I think I'd like to change lustre to use generic_error_remove_page
instead of ll_invalidate_page() - for S_ISREG mappings.

>
> For metadata the only pages we hold are readdir pages and the lock currently
> covers entire dir so lock cancelation should not need this.
>
> So looking at the code the other two users are readdir where we failed to populate page
> somehow and now need to throw it out and then a collision event for hashes, it appears.
> Both of these should apply to other filesystems I am sure.

In the directory case there is no invalidatepage operation, so the
current truncate_complete_page() function just does:

       cancel_dirty_page(page);
       ClearPageMappedToDisk(page);
       ll_delete_from_page_cache(page);

I'd guess that lustre directory pages don't get PageMappedToDisk set,
and in this situation there aren't dirty.
So this just boils down to
   delete_from_page_cache(page)

Do you know if my guesses are correct?  If they are, lets just use
delete_from_page_cache() for unwanted directory pages.

Thanks,
NeilBrown
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180618/b06cae4e/attachment.sig>

From shreeya.patel23498 at gmail.com  Sun Jun 17 11:16:55 2018
From: shreeya.patel23498 at gmail.com (Shreeya Patel)
Date: Sun, 17 Jun 2018 16:46:55 +0530
Subject: [lustre-devel] [PATCH] Staging: lustre: Use kmemdup() instead of
 kzalloc and memcpy
Message-ID: <1529234215-4010-1-git-send-email-shreeya.patel23498@gmail.com>

Replace calls to kzalloc or kmalloc followed by a memcpy with
a direct call to kmemdup to shorten the code.

The Coccinelle semantic patch used to make this change is as follows:
@@
expression from,to,size,flag;
statement S;
@@

-  to = \(kmalloc\|kzalloc\)(size,flag);
+  to = kmemdup(from,size,flag);
   if (to==NULL || ...) S
-  memcpy(to, from, size);

Signed-off-by: Shreeya Patel <shreeya.patel23498 at gmail.com>
---
 drivers/staging/lustre/lnet/lnet/api-ni.c | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

diff --git a/drivers/staging/lustre/lnet/lnet/api-ni.c b/drivers/staging/lustre/lnet/lnet/api-ni.c
index f9ed697..36ea14e 100644
--- a/drivers/staging/lustre/lnet/lnet/api-ni.c
+++ b/drivers/staging/lustre/lnet/lnet/api-ni.c
@@ -1271,15 +1271,14 @@ lnet_startup_lndni(struct lnet_ni *ni, struct lnet_ioctl_config_data *conf)
 		lnd_tunables = (struct lnet_ioctl_config_lnd_tunables *)conf->cfg_bulk;
 
 	if (lnd_tunables) {
-		ni->ni_lnd_tunables = kzalloc(sizeof(*ni->ni_lnd_tunables),
+		ni->ni_lnd_tunables = kmemdup(lnd_tunables,
+					      sizeof(*ni->ni_lnd_tunables),
 					      GFP_NOFS);
 		if (!ni->ni_lnd_tunables) {
 			mutex_unlock(&the_lnet.ln_lnd_mutex);
 			rc = -ENOMEM;
 			goto failed0;
 		}
-		memcpy(ni->ni_lnd_tunables, lnd_tunables,
-		       sizeof(*ni->ni_lnd_tunables));
 	}
 
 	/*
-- 
2.7.4


From jsimmons at infradead.org  Thu Jun 21 01:35:26 2018
From: jsimmons at infradead.org (James Simmons)
Date: Thu, 21 Jun 2018 02:35:26 +0100 (BST)
Subject: [lustre-devel] [PATCH 09/24] lustre: don't use spare bits in
 iattr.ia_valid
In-Reply-To: <152904669036.10587.17899565981649604892.stgit@noble>
References: <152904663333.10587.10934053155404014785.stgit@noble>
 <152904669036.10587.17899565981649604892.stgit@noble>
Message-ID: <alpine.LFD.2.21.1806210233170.3886@casper.infradead.org>


> Using spare bits in iattr.ia_valid is not safe as the
> bits might get used for some other purpose without
> us noticing.
> lustre currently used 4 spare bit, but they are all
> defined in different places so this isn't immediately
> obvious.
> 
> This patch brings all those bit definitions together
> and defined a new op_data field (op_xvalid) to hold
> the extra validity bits.
> 
> It also replaces sa_valid in struct cl_setattr_io
> with sa_avalid and sa_xvalid.  Changing the name is
> helpful as sa_valid already has another use within
> lustre.
> 
> Signed-off-by: NeilBrown <neilb at suse.com>

Nak: I see regressions with this patch.

[12368.453655] Lustre: DEBUG MARKER: == sanity test 240: race between ldlm 
enqueue and the connection RPC (no ASSERT) ===================== 21:
16:30 (1529543790)
[12368.760832] BUG: Dentry 
000000002646a847{i=200004282000008,n=f237.sanity}  still in use (1) 
[unmount of lustre lustre]
[12368.773746] WARNING: CPU: 1 PID: 10861 at fs/dcache.c:1514 
umount_check+0x66/0x70
[12368.783364] Modules linked in: ext4(E) mbcache(E) jbd2(E) loop(E) 
lustre(CE) obdecho(CE) mgc(CE) lov(CE) osc(CE) mdc(CE) lmv(CE) fid(CE) fld
(CE) ptlrpc(CE) obdclass(CE) ksocklnd(CE) sha512_generic(E) lnet(CE) 
libcfs(CE) xt_comment(E) xt_multiport(E) nfsv3(E) nfs_acl(E) sg(E) ip6t_RE
JECT(E) nf_reject_ipv6(E) nf_conntrack_ipv6(E) nf_defrag_ipv6(E) 
ip6table_filter(E) ip6_tables(E) ipt_REJECT(E) nf_reject_ipv4(E) 
nf_conntrack_
ipv4(E) nf_defrag_ipv4(E) xt_conntrack(E) nf_conntrack(E) libcrc32c(E) 
iptable_filter(E) dm_mirror(E) dm_region_hash(E) dm_log(E) dm_mod(E) ib_
isert(E) iscsi_target_mod(E) mgag200(E) ttm(E) target_core_mod(E) 
drm_kms_helper(E) rpcrdma(E) x86_pkg_temp_thermal(E) drm(E) 
crc32_pclmul(E) i
b_ucm(E) rdma_ucm(E) ib_uverbs(E) fb_sys_fops(E) syscopyarea(E) ib_iser(E) 
mpt3sas(E) sysfillrect(E)
[12368.869470]  ipmi_si(E) sysimgblt(E) raid_class(E) ib_umad(E) 
rdma_cm(E) ipmi_devintf(E) scsi_transport_sas(E) wmi(E) ib_ipoib(E) 
ipmi_msgha
ndler(E) iw_cm(E) libiscsi(E) ib_cm(E) scsi_transport_iscsi(E) mlx4_ib(E) 
ib_core(E) binfmt_misc(E) ip_tables(E) rpcsec_gss_krb5(E) auth_rpcgss
(E) nfsv4(E) dns_resolver(E) nfs(E) lockd(E) grace(E) fscache(E) 
mlx4_en(E) mlx4_core(E) igb(E) i2c_algo_bit(E) devlink(E) crc32c_intel(E) 
i2c_
core(E) sunrpc(E)
[12368.917172] CPU: 1 PID: 10861 Comm: umount Tainted: G        WC  E     
4.17.0+ #1
[12368.926825] Hardware name: Supermicro X9DRT/X9DRT, BIOS 3.0a 02/19/2014
[12368.935595] RIP: 0010:umount_check+0x66/0x70
[12368.941993] RSP: 0018:ffffc90007313da8 EFLAGS: 00010286
[12368.949342] RAX: 000000000000006a RBX: ffff8810557dcb40 RCX: 
0000000000000000
[12368.958619] RDX: 0000000000000001 RSI: ffff88085fc954f8 RDI: 
ffff88085fc954f8
[12368.967868] RBP: ffff880fec9ef600 R08: 0000000000000000 R09: 
0000000000000896
[12368.977102] R10: 00000000000003ff R11: 0000000000aaaaaa R12: 
ffffffff8124a320
[12368.986315] R13: ffff8810557dcbe0 R14: ffff880fec9ef690 R15: 
ffff8810557dcbe0
[12368.995505] FS:  00007fc174af5880(0000) GS:ffff88085fc80000(0000) 
knlGS:0000000000000000
[12369.005631] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[12369.013395] CR2: 00007fc174b04000 CR3: 0000000822922002 CR4: 
00000000000606e0
[12369.022530] Call Trace:
[12369.026962]  d_walk+0xe1/0x270
[12369.031970]  do_one_tree+0x22/0x40
[12369.037280]  shrink_dcache_for_umount+0x28/0x80
[12369.043685]  generic_shutdown_super+0x1a/0x110
[12369.049977]  kill_anon_super+0xe/0x20
[12369.055463]  deactivate_locked_super+0x39/0x70
[12369.061708]  cleanup_mnt+0x3b/0x70
[12369.066871]  task_work_run+0x7c/0xa0
[12369.072174]  exit_to_usermode_loop+0x5a/0x7e
[12369.078140]  do_syscall_64+0x2b7/0x347
[12369.083547]  ? do_page_fault+0x2d/0x110
[12369.089008]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
[12369.095655] RIP: 0033:0x7fc173d69bd7
[12369.100789] RSP: 002b:00007ffc4c89fcc8 EFLAGS: 00000246 ORIG_RAX: 
00000000000000a6
[12369.109902] RAX: 0000000000000000 RBX: 000056521cd63040 RCX: 
00007fc173d69bd7
[12369.118564] RDX: 0000000000000001 RSI: 0000000000000000 RDI: 
000056521cd642b0
[12369.127188] RBP: 000056521cd642b0 R08: 0000000000000000 R09: 
000000000000000f
[12369.135778] R10: 00007ffc4c89f720 R11: 0000000000000246 R12: 
00007fc1748e4d58
[12369.144336] R13: 0000000000000000 R14: 000056521cd648a0 R15: 
000056521cd63040
[12369.152868] Code: 00 00 48 8b 40 28 4c 8b 08 48 8b 46 30 48 85 c0 74 1d 
48 8b 50 40 48 89 0c 24 48 c7 c7 58 5c e6 81 48 89 f1 31 c0 e8 38 cf ea ff 
<0f> 0b eb b2 31 d2 eb e3 66 90 66 66 66 66 90 55 48 89 fd 53 48 
[12369.174646] ---[ end trace 053c2a5a718d3440 ]---
[12369.385564] Lustre: Unmounted lustre-client
[12369.393247] VFS: Busy inodes after unmount of lustre. Self-destruct in 
5 seconds.  Have a nice day...

When I remove this patch things go back to normal. This will not show up
if you do a ONLY="240" sh ./sanity.sh. You have to run the sanity.sh in
total to make this show up.

> ---
>  drivers/staging/lustre/lustre/include/cl_object.h  |    5 +++-
>  .../staging/lustre/lustre/include/lustre_compat.h  |    6 -----
>  .../lustre/include/lustre_patchless_compat.h       |    8 -------
>  drivers/staging/lustre/lustre/include/obd.h        |    7 ++++++
>  drivers/staging/lustre/lustre/llite/file.c         |   12 ++++++----
>  drivers/staging/lustre/lustre/llite/lcommon_cl.c   |    5 +++-
>  .../staging/lustre/lustre/llite/llite_internal.h   |    5 +++-
>  drivers/staging/lustre/lustre/llite/llite_lib.c    |   21 ++++++++++--------
>  drivers/staging/lustre/lustre/llite/vvp_io.c       |    8 +++----
>  drivers/staging/lustre/lustre/lmv/lmv_obd.c        |    5 +++-
>  drivers/staging/lustre/lustre/lov/lov_io.c         |    3 ++-
>  drivers/staging/lustre/lustre/mdc/mdc_lib.c        |   13 ++++++-----
>  drivers/staging/lustre/lustre/osc/osc_io.c         |   23 ++++++++++----------
>  13 files changed, 63 insertions(+), 58 deletions(-)
> 
> diff --git a/drivers/staging/lustre/lustre/include/cl_object.h b/drivers/staging/lustre/lustre/include/cl_object.h
> index 6f7b991be809..caa66763c70d 100644
> --- a/drivers/staging/lustre/lustre/include/cl_object.h
> +++ b/drivers/staging/lustre/lustre/include/cl_object.h
> @@ -1773,7 +1773,8 @@ struct cl_io {
>  		struct cl_setattr_io {
>  			struct ost_lvb   sa_attr;
>  			unsigned int		 sa_attr_flags;
> -			unsigned int     sa_valid;
> +			unsigned int     sa_avalid;
> +			unsigned int     sa_xvalid;
>  			int		sa_stripe_index;
>  			const struct lu_fid	*sa_parent_fid;
>  		} ci_setattr;
> @@ -2304,7 +2305,7 @@ static inline int cl_io_is_mkwrite(const struct cl_io *io)
>  static inline int cl_io_is_trunc(const struct cl_io *io)
>  {
>  	return io->ci_type == CIT_SETATTR &&
> -		(io->u.ci_setattr.sa_valid & ATTR_SIZE);
> +		(io->u.ci_setattr.sa_avalid & ATTR_SIZE);
>  }
>  
>  struct cl_io *cl_io_top(struct cl_io *io);
> diff --git a/drivers/staging/lustre/lustre/include/lustre_compat.h b/drivers/staging/lustre/lustre/include/lustre_compat.h
> index 3c6db0d632dc..6d1a433e32c5 100644
> --- a/drivers/staging/lustre/lustre/include/lustre_compat.h
> +++ b/drivers/staging/lustre/lustre/include/lustre_compat.h
> @@ -41,12 +41,6 @@
>  
>  #include <lustre_patchless_compat.h>
>  
> -/*
> - * set ATTR_BLOCKS to a high value to avoid any risk of collision with other
> - * ATTR_* attributes (see bug 13828)
> - */
> -#define ATTR_BLOCKS    (1 << 27)
> -
>  #define current_ngroups current_cred()->group_info->ngroups
>  #define current_groups current_cred()->group_info->small_block
>  
> diff --git a/drivers/staging/lustre/lustre/include/lustre_patchless_compat.h b/drivers/staging/lustre/lustre/include/lustre_patchless_compat.h
> index dca1e3dbd183..1bbe1a779ae9 100644
> --- a/drivers/staging/lustre/lustre/include/lustre_patchless_compat.h
> +++ b/drivers/staging/lustre/lustre/include/lustre_patchless_compat.h
> @@ -41,12 +41,4 @@
>  #include <linux/hash.h>
>  #include <linux/pagemap.h>
>  
> -#ifndef ATTR_CTIME_SET
> -/*
> - * set ATTR_CTIME_SET to a high value to avoid any risk of collision with other
> - * ATTR_* attributes (see bug 13828)
> - */
> -#define ATTR_CTIME_SET (1 << 28)
> -#endif
> -
>  #endif /* LUSTRE_PATCHLESS_COMPAT_H */
> diff --git a/drivers/staging/lustre/lustre/include/obd.h b/drivers/staging/lustre/lustre/include/obd.h
> index b1907bbffb19..d38b6bc6b246 100644
> --- a/drivers/staging/lustre/lustre/include/obd.h
> +++ b/drivers/staging/lustre/lustre/include/obd.h
> @@ -723,6 +723,7 @@ struct md_op_data {
>  
>  	/* iattr fields and blocks. */
>  	struct iattr	    op_attr;
> +	unsigned int		op_xvalid; /* eXtra validity flags */
>  	unsigned int	    op_attr_flags;
>  	__u64		   op_valid;
>  	loff_t		  op_attr_blocks;
> @@ -751,6 +752,12 @@ struct md_op_data {
>  	__u32			op_default_stripe_offset;
>  };
>  
> +/* Flags for op_xvalid */
> +#define OP_ATTR_CTIME_SET	(1 << 0)
> +#define OP_ATTR_BLOCKS		(1 << 1)
> +#define OP_ATTR_OWNEROVERRIDE	(1 << 2)
> +#define OP_ATTR_FLAGS		(1 << 3)
> +
>  struct md_callback {
>  	int (*md_blocking_ast)(struct ldlm_lock *lock,
>  			       struct ldlm_lock_desc *desc,
> diff --git a/drivers/staging/lustre/lustre/llite/file.c b/drivers/staging/lustre/lustre/llite/file.c
> index e888ed6e74bc..e642996db53f 100644
> --- a/drivers/staging/lustre/lustre/llite/file.c
> +++ b/drivers/staging/lustre/lustre/llite/file.c
> @@ -93,9 +93,10 @@ static void ll_prepare_close(struct inode *inode, struct md_op_data *op_data,
>  	op_data->op_attr.ia_mtime = inode->i_mtime;
>  	op_data->op_attr.ia_ctime = inode->i_ctime;
>  	op_data->op_attr.ia_size = i_size_read(inode);
> -	op_data->op_attr.ia_valid |= ATTR_MODE | ATTR_ATIME | ATTR_ATIME_SET |
> -				     ATTR_MTIME | ATTR_MTIME_SET |
> -				     ATTR_CTIME | ATTR_CTIME_SET;
> +	op_data->op_attr.ia_valid |= (ATTR_MODE | ATTR_ATIME | ATTR_ATIME_SET |
> +				      ATTR_MTIME | ATTR_MTIME_SET |
> +				      ATTR_CTIME);
> +	op_data->op_xvalid |= OP_ATTR_CTIME_SET;
>  	op_data->op_attr_blocks = inode->i_blocks;
>  	op_data->op_attr_flags = ll_inode_to_ext_flags(inode->i_flags);
>  	op_data->op_handle = och->och_fh;
> @@ -161,7 +162,8 @@ static int ll_close_inode_openhandle(struct inode *inode,
>  		op_data->op_bias |= MDS_HSM_RELEASE;
>  		op_data->op_data_version = *(__u64 *)data;
>  		op_data->op_lease_handle = och->och_lease_handle;
> -		op_data->op_attr.ia_valid |= ATTR_SIZE | ATTR_BLOCKS;
> +		op_data->op_attr.ia_valid |= ATTR_SIZE;
> +		op_data->op_xvalid |= OP_ATTR_BLOCKS;
>  		break;
>  
>  	default:
> @@ -1906,7 +1908,7 @@ static int ll_hsm_import(struct inode *inode, struct file *file,
>  
>  	inode_lock(inode);
>  
> -	rc = ll_setattr_raw(file->f_path.dentry, attr, true);
> +	rc = ll_setattr_raw(file->f_path.dentry, attr, 0, true);
>  	if (rc == -ENODATA)
>  		rc = 0;
>  
> diff --git a/drivers/staging/lustre/lustre/llite/lcommon_cl.c b/drivers/staging/lustre/lustre/llite/lcommon_cl.c
> index d7ea39ce0cb2..9d1f3881f860 100644
> --- a/drivers/staging/lustre/lustre/llite/lcommon_cl.c
> +++ b/drivers/staging/lustre/lustre/llite/lcommon_cl.c
> @@ -80,7 +80,7 @@ u16 cl_inode_fini_refcheck;
>  static DEFINE_MUTEX(cl_inode_fini_guard);
>  
>  int cl_setattr_ost(struct cl_object *obj, const struct iattr *attr,
> -		   unsigned int attr_flags)
> +		   unsigned int xvalid, unsigned int attr_flags)
>  {
>  	struct lu_env *env;
>  	struct cl_io  *io;
> @@ -100,7 +100,8 @@ int cl_setattr_ost(struct cl_object *obj, const struct iattr *attr,
>  	io->u.ci_setattr.sa_attr.lvb_ctime = LTIME_S(attr->ia_ctime);
>  	io->u.ci_setattr.sa_attr.lvb_size = attr->ia_size;
>  	io->u.ci_setattr.sa_attr_flags = attr_flags;
> -	io->u.ci_setattr.sa_valid = attr->ia_valid;
> +	io->u.ci_setattr.sa_avalid = attr->ia_valid;
> +	io->u.ci_setattr.sa_xvalid = xvalid;
>  	io->u.ci_setattr.sa_parent_fid = lu_object_fid(&obj->co_lu);
>  
>  again:
> diff --git a/drivers/staging/lustre/lustre/llite/llite_internal.h b/drivers/staging/lustre/lustre/llite/llite_internal.h
> index 28cff58690d6..b750a48fd0e1 100644
> --- a/drivers/staging/lustre/lustre/llite/llite_internal.h
> +++ b/drivers/staging/lustre/lustre/llite/llite_internal.h
> @@ -805,7 +805,8 @@ void ll_kill_super(struct super_block *sb);
>  struct inode *ll_inode_from_resource_lock(struct ldlm_lock *lock);
>  void ll_dir_clear_lsm_md(struct inode *inode);
>  void ll_clear_inode(struct inode *inode);
> -int ll_setattr_raw(struct dentry *dentry, struct iattr *attr, bool hsm_import);
> +int ll_setattr_raw(struct dentry *dentry, struct iattr *attr,
> +		   unsigned int xvalid, bool hsm_import);
>  int ll_setattr(struct dentry *de, struct iattr *attr);
>  int ll_statfs(struct dentry *de, struct kstatfs *sfs);
>  int ll_statfs_internal(struct super_block *sb, struct obd_statfs *osfs,
> @@ -1330,7 +1331,7 @@ int ll_getparent(struct file *file, struct getparent __user *arg);
>  
>  /* lcommon_cl.c */
>  int cl_setattr_ost(struct cl_object *obj, const struct iattr *attr,
> -		   unsigned int attr_flags);
> +		   unsigned int xvalid, unsigned int attr_flags);
>  
>  extern struct lu_env *cl_inode_fini_env;
>  extern u16 cl_inode_fini_refcheck;
> diff --git a/drivers/staging/lustre/lustre/llite/llite_lib.c b/drivers/staging/lustre/lustre/llite/llite_lib.c
> index 36066c839160..cd0f2517a3b3 100644
> --- a/drivers/staging/lustre/lustre/llite/llite_lib.c
> +++ b/drivers/staging/lustre/lustre/llite/llite_lib.c
> @@ -1429,7 +1429,8 @@ static int ll_md_setattr(struct dentry *dentry, struct md_op_data *op_data)
>   *
>   * In case of HSMimport, we only set attr on MDS.
>   */
> -int ll_setattr_raw(struct dentry *dentry, struct iattr *attr, bool hsm_import)
> +int ll_setattr_raw(struct dentry *dentry, struct iattr *attr,
> +		   unsigned int xvalid, bool hsm_import)
>  {
>  	struct inode *inode = d_inode(dentry);
>  	struct ll_inode_info *lli = ll_i2info(inode);
> @@ -1470,7 +1471,7 @@ int ll_setattr_raw(struct dentry *dentry, struct iattr *attr, bool hsm_import)
>  	/* We mark all of the fields "set" so MDS/OST does not re-set them */
>  	if (attr->ia_valid & ATTR_CTIME) {
>  		attr->ia_ctime = current_time(inode);
> -		attr->ia_valid |= ATTR_CTIME_SET;
> +		xvalid |= OP_ATTR_CTIME_SET;
>  	}
>  	if (!(attr->ia_valid & ATTR_ATIME_SET) &&
>  	    (attr->ia_valid & ATTR_ATIME)) {
> @@ -1506,12 +1507,13 @@ int ll_setattr_raw(struct dentry *dentry, struct iattr *attr, bool hsm_import)
>  		 * If we are changing file size, file content is
>  		 * modified, flag it.
>  		 */
> -		attr->ia_valid |= MDS_OPEN_OWNEROVERRIDE;
> +		xvalid |= OP_ATTR_OWNEROVERRIDE;
>  		op_data->op_bias |= MDS_DATA_MODIFIED;
>  		clear_bit(LLIF_DATA_MODIFIED, &lli->lli_flags);
>  	}
>  
>  	op_data->op_attr = *attr;
> +	op_data->op_xvalid = xvalid;
>  
>  	rc = ll_md_setattr(dentry, op_data);
>  	if (rc)
> @@ -1532,7 +1534,8 @@ int ll_setattr_raw(struct dentry *dentry, struct iattr *attr, bool hsm_import)
>  		 * setting times to past, but it is necessary due to possible
>  		 * time de-synchronization between MDT inode and OST objects
>  		 */
> -		rc = cl_setattr_ost(ll_i2info(inode)->lli_clob, attr, 0);
> +		rc = cl_setattr_ost(ll_i2info(inode)->lli_clob,
> +				    attr, xvalid, 0);
>  	}
>  
>  	/*
> @@ -1589,10 +1592,11 @@ int ll_setattr_raw(struct dentry *dentry, struct iattr *attr, bool hsm_import)
>  int ll_setattr(struct dentry *de, struct iattr *attr)
>  {
>  	int mode = d_inode(de)->i_mode;
> +	unsigned int xvalid = 0;
>  
>  	if ((attr->ia_valid & (ATTR_CTIME | ATTR_SIZE | ATTR_MODE)) ==
>  			      (ATTR_CTIME | ATTR_SIZE | ATTR_MODE))
> -		attr->ia_valid |= MDS_OPEN_OWNEROVERRIDE;
> +		xvalid |= OP_ATTR_OWNEROVERRIDE;
>  
>  	if (((attr->ia_valid & (ATTR_MODE | ATTR_FORCE | ATTR_SIZE)) ==
>  			       (ATTR_SIZE | ATTR_MODE)) &&
> @@ -1613,7 +1617,7 @@ int ll_setattr(struct dentry *de, struct iattr *attr)
>  	    !(attr->ia_valid & ATTR_KILL_SGID))
>  		attr->ia_valid |= ATTR_KILL_SGID;
>  
> -	return ll_setattr_raw(de, attr, false);
> +	return ll_setattr_raw(de, attr, xvalid, false);
>  }
>  
>  int ll_statfs_internal(struct super_block *sb, struct obd_statfs *osfs,
> @@ -1946,7 +1950,7 @@ int ll_iocontrol(struct inode *inode, struct file *file,
>  			return PTR_ERR(op_data);
>  
>  		op_data->op_attr_flags = flags;
> -		op_data->op_attr.ia_valid |= ATTR_ATTR_FLAG;
> +		op_data->op_xvalid |= OP_ATTR_FLAGS;
>  		rc = md_setattr(sbi->ll_md_exp, op_data, NULL, 0, &req);
>  		ll_finish_md_op_data(op_data);
>  		ptlrpc_req_finished(req);
> @@ -1963,8 +1967,7 @@ int ll_iocontrol(struct inode *inode, struct file *file,
>  		if (!attr)
>  			return -ENOMEM;
>  
> -		attr->ia_valid = ATTR_ATTR_FLAG;
> -		rc = cl_setattr_ost(obj, attr, flags);
> +		rc = cl_setattr_ost(obj, attr, OP_ATTR_FLAGS, flags);
>  		kfree(attr);
>  		return rc;
>  	}
> diff --git a/drivers/staging/lustre/lustre/llite/vvp_io.c b/drivers/staging/lustre/lustre/llite/vvp_io.c
> index e7a4778e02e4..e65523f7628d 100644
> --- a/drivers/staging/lustre/lustre/llite/vvp_io.c
> +++ b/drivers/staging/lustre/lustre/llite/vvp_io.c
> @@ -556,7 +556,7 @@ static int vvp_io_setattr_lock(const struct lu_env *env,
>  		if (new_size == 0)
>  			enqflags = CEF_DISCARD_DATA;
>  	} else {
> -		unsigned int valid = io->u.ci_setattr.sa_valid;
> +		unsigned int valid = io->u.ci_setattr.sa_avalid;
>  
>  		if (!(valid & TIMES_SET_FLAGS))
>  			return 0;
> @@ -603,11 +603,11 @@ static int vvp_io_setattr_time(const struct lu_env *env,
>  
>  	cl_object_attr_lock(obj);
>  	attr->cat_ctime = io->u.ci_setattr.sa_attr.lvb_ctime;
> -	if (io->u.ci_setattr.sa_valid & ATTR_ATIME_SET) {
> +	if (io->u.ci_setattr.sa_avalid & ATTR_ATIME_SET) {
>  		attr->cat_atime = io->u.ci_setattr.sa_attr.lvb_atime;
>  		valid |= CAT_ATIME;
>  	}
> -	if (io->u.ci_setattr.sa_valid & ATTR_MTIME_SET) {
> +	if (io->u.ci_setattr.sa_avalid & ATTR_MTIME_SET) {
>  		attr->cat_mtime = io->u.ci_setattr.sa_attr.lvb_mtime;
>  		valid |= CAT_MTIME;
>  	}
> @@ -632,7 +632,7 @@ static int vvp_io_setattr_start(const struct lu_env *env,
>  		inode_lock(inode);
>  	}
>  
> -	if (io->u.ci_setattr.sa_valid & TIMES_SET_FLAGS)
> +	if (io->u.ci_setattr.sa_avalid & TIMES_SET_FLAGS)
>  		return vvp_io_setattr_time(env, ios);
>  
>  	return 0;
> diff --git a/drivers/staging/lustre/lustre/lmv/lmv_obd.c b/drivers/staging/lustre/lustre/lmv/lmv_obd.c
> index 65f94e6ecaad..9f427a7c7b03 100644
> --- a/drivers/staging/lustre/lustre/lmv/lmv_obd.c
> +++ b/drivers/staging/lustre/lustre/lmv/lmv_obd.c
> @@ -1983,8 +1983,9 @@ static int lmv_setattr(struct obd_export *exp, struct md_op_data *op_data,
>  	struct lmv_obd	  *lmv = &obd->u.lmv;
>  	struct lmv_tgt_desc     *tgt;
>  
> -	CDEBUG(D_INODE, "SETATTR for " DFID ", valid 0x%x\n",
> -	       PFID(&op_data->op_fid1), op_data->op_attr.ia_valid);
> +	CDEBUG(D_INODE, "SETATTR for " DFID ", valid 0x%x/0x%x\n",
> +	       PFID(&op_data->op_fid1), op_data->op_attr.ia_valid,
> +		op_data->op_xvalid);
>  
>  	op_data->op_flags |= MF_MDC_CANCEL_FID1;
>  	tgt = lmv_find_target(lmv, &op_data->op_fid1);
> diff --git a/drivers/staging/lustre/lustre/lov/lov_io.c b/drivers/staging/lustre/lustre/lov/lov_io.c
> index b823f8a21856..50982848bca4 100644
> --- a/drivers/staging/lustre/lustre/lov/lov_io.c
> +++ b/drivers/staging/lustre/lustre/lov/lov_io.c
> @@ -77,7 +77,8 @@ static void lov_io_sub_inherit(struct cl_io *io, struct lov_io *lio,
>  		io->u.ci_setattr.sa_attr = parent->u.ci_setattr.sa_attr;
>  		io->u.ci_setattr.sa_attr_flags =
>  					parent->u.ci_setattr.sa_attr_flags;
> -		io->u.ci_setattr.sa_valid = parent->u.ci_setattr.sa_valid;
> +		io->u.ci_setattr.sa_avalid = parent->u.ci_setattr.sa_avalid;
> +		io->u.ci_setattr.sa_xvalid = parent->u.ci_setattr.sa_xvalid;
>  		io->u.ci_setattr.sa_stripe_index = stripe;
>  		io->u.ci_setattr.sa_parent_fid =
>  					parent->u.ci_setattr.sa_parent_fid;
> diff --git a/drivers/staging/lustre/lustre/mdc/mdc_lib.c b/drivers/staging/lustre/lustre/mdc/mdc_lib.c
> index d582968987ff..ad7c13be5e3c 100644
> --- a/drivers/staging/lustre/lustre/mdc/mdc_lib.c
> +++ b/drivers/staging/lustre/lustre/mdc/mdc_lib.c
> @@ -234,7 +234,7 @@ void mdc_open_pack(struct ptlrpc_request *req, struct md_op_data *op_data,
>  	set_mrc_cr_flags(rec, cr_flags);
>  }
>  
> -static inline __u64 attr_pack(unsigned int ia_valid)
> +static inline __u64 attr_pack(unsigned int ia_valid, unsigned int ia_xvalid)
>  {
>  	__u64 sa_valid = 0;
>  
> @@ -258,19 +258,19 @@ static inline __u64 attr_pack(unsigned int ia_valid)
>  		sa_valid |= MDS_ATTR_MTIME_SET;
>  	if (ia_valid & ATTR_FORCE)
>  		sa_valid |= MDS_ATTR_FORCE;
> -	if (ia_valid & ATTR_ATTR_FLAG)
> +	if (ia_xvalid & OP_ATTR_FLAGS)
>  		sa_valid |= MDS_ATTR_ATTR_FLAG;
>  	if (ia_valid & ATTR_KILL_SUID)
>  		sa_valid |=  MDS_ATTR_KILL_SUID;
>  	if (ia_valid & ATTR_KILL_SGID)
>  		sa_valid |= MDS_ATTR_KILL_SGID;
> -	if (ia_valid & ATTR_CTIME_SET)
> +	if (ia_xvalid & OP_ATTR_CTIME_SET)
>  		sa_valid |= MDS_ATTR_CTIME_SET;
>  	if (ia_valid & ATTR_OPEN)
>  		sa_valid |= MDS_ATTR_FROM_OPEN;
> -	if (ia_valid & ATTR_BLOCKS)
> +	if (ia_xvalid & OP_ATTR_BLOCKS)
>  		sa_valid |= MDS_ATTR_BLOCKS;
> -	if (ia_valid & MDS_OPEN_OWNEROVERRIDE)
> +	if (ia_xvalid & OP_ATTR_OWNEROVERRIDE)
>  		/* NFSD hack (see bug 5781) */
>  		sa_valid |= MDS_OPEN_OWNEROVERRIDE;
>  	return sa_valid;
> @@ -286,7 +286,8 @@ static void mdc_setattr_pack_rec(struct mdt_rec_setattr *rec,
>  	rec->sa_suppgid = -1;
>  
>  	rec->sa_fid    = op_data->op_fid1;
> -	rec->sa_valid  = attr_pack(op_data->op_attr.ia_valid);
> +	rec->sa_valid  = attr_pack(op_data->op_attr.ia_valid,
> +				   op_data->op_xvalid);
>  	rec->sa_mode   = op_data->op_attr.ia_mode;
>  	rec->sa_uid    = from_kuid(&init_user_ns, op_data->op_attr.ia_uid);
>  	rec->sa_gid    = from_kgid(&init_user_ns, op_data->op_attr.ia_gid);
> diff --git a/drivers/staging/lustre/lustre/osc/osc_io.c b/drivers/staging/lustre/lustre/osc/osc_io.c
> index 67734a8ed331..955525f69958 100644
> --- a/drivers/staging/lustre/lustre/osc/osc_io.c
> +++ b/drivers/staging/lustre/lustre/osc/osc_io.c
> @@ -499,7 +499,8 @@ static int osc_io_setattr_start(const struct lu_env *env,
>  	struct obdo *oa = &oio->oi_oa;
>  	struct osc_async_cbargs *cbargs = &oio->oi_cbarg;
>  	__u64 size = io->u.ci_setattr.sa_attr.lvb_size;
> -	unsigned int ia_valid = io->u.ci_setattr.sa_valid;
> +	unsigned int ia_avalid = io->u.ci_setattr.sa_avalid;
> +	unsigned int ia_xvalid = io->u.ci_setattr.sa_xvalid;
>  	int result = 0;
>  
>  	/* truncate cache dirty pages first */
> @@ -514,20 +515,20 @@ static int osc_io_setattr_start(const struct lu_env *env,
>  			struct ost_lvb *lvb = &io->u.ci_setattr.sa_attr;
>  			unsigned int cl_valid = 0;
>  
> -			if (ia_valid & ATTR_SIZE) {
> +			if (ia_avalid & ATTR_SIZE) {
>  				attr->cat_size = size;
>  				attr->cat_kms = size;
>  				cl_valid = CAT_SIZE | CAT_KMS;
>  			}
> -			if (ia_valid & ATTR_MTIME_SET) {
> +			if (ia_avalid & ATTR_MTIME_SET) {
>  				attr->cat_mtime = lvb->lvb_mtime;
>  				cl_valid |= CAT_MTIME;
>  			}
> -			if (ia_valid & ATTR_ATIME_SET) {
> +			if (ia_avalid & ATTR_ATIME_SET) {
>  				attr->cat_atime = lvb->lvb_atime;
>  				cl_valid |= CAT_ATIME;
>  			}
> -			if (ia_valid & ATTR_CTIME_SET) {
> +			if (ia_xvalid & OP_ATTR_CTIME_SET) {
>  				attr->cat_ctime = lvb->lvb_ctime;
>  				cl_valid |= CAT_CTIME;
>  			}
> @@ -542,19 +543,19 @@ static int osc_io_setattr_start(const struct lu_env *env,
>  		obdo_set_parent_fid(oa, io->u.ci_setattr.sa_parent_fid);
>  		oa->o_stripe_idx = io->u.ci_setattr.sa_stripe_index;
>  		oa->o_valid |= OBD_MD_FLID | OBD_MD_FLGROUP;
> -		if (ia_valid & ATTR_CTIME) {
> +		if (ia_avalid & ATTR_CTIME) {
>  			oa->o_valid |= OBD_MD_FLCTIME;
>  			oa->o_ctime = attr->cat_ctime;
>  		}
> -		if (ia_valid & ATTR_ATIME) {
> +		if (ia_avalid & ATTR_ATIME) {
>  			oa->o_valid |= OBD_MD_FLATIME;
>  			oa->o_atime = attr->cat_atime;
>  		}
> -		if (ia_valid & ATTR_MTIME) {
> +		if (ia_avalid & ATTR_MTIME) {
>  			oa->o_valid |= OBD_MD_FLMTIME;
>  			oa->o_mtime = attr->cat_mtime;
>  		}
> -		if (ia_valid & ATTR_SIZE) {
> +		if (ia_avalid & ATTR_SIZE) {
>  			oa->o_size = size;
>  			oa->o_blocks = OBD_OBJECT_EOF;
>  			oa->o_valid |= OBD_MD_FLSIZE | OBD_MD_FLBLOCKS;
> @@ -566,14 +567,14 @@ static int osc_io_setattr_start(const struct lu_env *env,
>  		} else {
>  			LASSERT(oio->oi_lockless == 0);
>  		}
> -		if (ia_valid & ATTR_ATTR_FLAG) {
> +		if (ia_xvalid & OP_ATTR_FLAGS) {
>  			oa->o_flags = io->u.ci_setattr.sa_attr_flags;
>  			oa->o_valid |= OBD_MD_FLFLAGS;
>  		}
>  
>  		init_completion(&cbargs->opc_sync);
>  
> -		if (ia_valid & ATTR_SIZE)
> +		if (ia_avalid & ATTR_SIZE)
>  			result = osc_punch_base(osc_export(cl2osc(obj)),
>  						oa, osc_async_upcall,
>  						cbargs, PTLRPCD_SET);
> 
> 
> 

From jsimmons at infradead.org  Thu Jun 21 01:37:35 2018
From: jsimmons at infradead.org (James Simmons)
Date: Thu, 21 Jun 2018 02:37:35 +0100 (BST)
Subject: [lustre-devel] [PATCH 21/24] lustre: move remainder of
 linux-tracefile.c to tracefile.c
In-Reply-To: <152904669084.10587.6920959206498193596.stgit@noble>
References: <152904663333.10587.10934053155404014785.stgit@noble>
 <152904669084.10587.6920959206498193596.stgit@noble>
Message-ID: <alpine.LFD.2.21.1806210236200.3886@casper.infradead.org>


> It's good to keep related code together.
> 
> Signed-off-by: NeilBrown <neilb at suse.com>

Nak. For some reason this corrupts my dmesg output. Its really strange and
I haven't figured it out yet.

> ---
>  drivers/staging/lustre/lnet/libcfs/Makefile        |    2 
>  .../staging/lustre/lnet/libcfs/linux-tracefile.c   |  194 --------------------
>  drivers/staging/lustre/lnet/libcfs/tracefile.c     |  155 ++++++++++++++++
>  3 files changed, 156 insertions(+), 195 deletions(-)
>  delete mode 100644 drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
> 
> diff --git a/drivers/staging/lustre/lnet/libcfs/Makefile b/drivers/staging/lustre/lnet/libcfs/Makefile
> index 1b6937c3deee..22c0e0c2e9cf 100644
> --- a/drivers/staging/lustre/lnet/libcfs/Makefile
> +++ b/drivers/staging/lustre/lnet/libcfs/Makefile
> @@ -3,7 +3,7 @@ ccflags-y += -I$(srctree)/drivers/staging/lustre/include
>  ccflags-y += -I$(srctree)/drivers/staging/lustre/lustre/include
>  ccflags-y += -DKBUILD_MODNAME='"lnet"'
>  
> -libcfs-obj-y += linux-tracefile.o linux-debug.o
> +libcfs-obj-y += linux-debug.o
>  libcfs-obj-y += linux-crypto.o
>  libcfs-obj-y += linux-crypto-adler.o
>  
> diff --git a/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c b/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
> deleted file mode 100644
> index 3af77228cd7a..000000000000
> --- a/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
> +++ /dev/null
> @@ -1,194 +0,0 @@
> -// SPDX-License-Identifier: GPL-2.0
> -/*
> - * GPL HEADER START
> - *
> - * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
> - *
> - * This program is free software; you can redistribute it and/or modify
> - * it under the terms of the GNU General Public License version 2 only,
> - * as published by the Free Software Foundation.
> - *
> - * This program is distributed in the hope that it will be useful, but
> - * WITHOUT ANY WARRANTY; without even the implied warranty of
> - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> - * General Public License version 2 for more details (a copy is included
> - * in the LICENSE file that accompanied this code).
> - *
> - * You should have received a copy of the GNU General Public License
> - * version 2 along with this program; If not, see
> - * http://www.gnu.org/licenses/gpl-2.0.html
> - *
> - * GPL HEADER END
> - */
> -/*
> - * Copyright (c) 2008, 2010, Oracle and/or its affiliates. All rights reserved.
> - * Use is subject to license terms.
> - *
> - * Copyright (c) 2012, Intel Corporation.
> - */
> -/*
> - * This file is part of Lustre, http://www.lustre.org/
> - * Lustre is a trademark of Sun Microsystems, Inc.
> - */
> -
> -#define DEBUG_SUBSYSTEM S_LNET
> -#define LUSTRE_TRACEFILE_PRIVATE
> -
> -#include <linux/slab.h>
> -#include <linux/mm.h>
> -#include "tracefile.h"
> -
> -/* percents to share the total debug memory for each type */
> -static unsigned int pages_factor[CFS_TCD_TYPE_MAX] = {
> -	80,  /* 80% pages for CFS_TCD_TYPE_PROC */
> -	10,  /* 10% pages for CFS_TCD_TYPE_SOFTIRQ */
> -	10   /* 10% pages for CFS_TCD_TYPE_IRQ */
> -};
> -
> -char *cfs_trace_console_buffers[NR_CPUS][CFS_TCD_TYPE_MAX];
> -
> -int cfs_tracefile_init_arch(void)
> -{
> -	int i;
> -	int j;
> -	struct cfs_trace_cpu_data *tcd;
> -
> -	/* initialize trace_data */
> -	memset(cfs_trace_data, 0, sizeof(cfs_trace_data));
> -	for (i = 0; i < CFS_TCD_TYPE_MAX; i++) {
> -		cfs_trace_data[i] =
> -			kmalloc_array(num_possible_cpus(),
> -				      sizeof(union cfs_trace_data_union),
> -				      GFP_KERNEL);
> -		if (!cfs_trace_data[i])
> -			goto out;
> -	}
> -
> -	/* arch related info initialized */
> -	cfs_tcd_for_each(tcd, i, j) {
> -		spin_lock_init(&tcd->tcd_lock);
> -		tcd->tcd_pages_factor = pages_factor[i];
> -		tcd->tcd_type = i;
> -		tcd->tcd_cpu = j;
> -	}
> -
> -	for (i = 0; i < num_possible_cpus(); i++)
> -		for (j = 0; j < 3; j++) {
> -			cfs_trace_console_buffers[i][j] =
> -				kmalloc(CFS_TRACE_CONSOLE_BUFFER_SIZE,
> -					GFP_KERNEL);
> -
> -			if (!cfs_trace_console_buffers[i][j])
> -				goto out;
> -		}
> -
> -	return 0;
> -
> -out:
> -	cfs_tracefile_fini_arch();
> -	pr_err("lnet: Not enough memory\n");
> -	return -ENOMEM;
> -}
> -
> -void cfs_tracefile_fini_arch(void)
> -{
> -	int i;
> -	int j;
> -
> -	for (i = 0; i < num_possible_cpus(); i++)
> -		for (j = 0; j < 3; j++) {
> -			kfree(cfs_trace_console_buffers[i][j]);
> -			cfs_trace_console_buffers[i][j] = NULL;
> -		}
> -
> -	for (i = 0; cfs_trace_data[i]; i++) {
> -		kfree(cfs_trace_data[i]);
> -		cfs_trace_data[i] = NULL;
> -	}
> -}
> -
> -enum cfs_trace_buf_type cfs_trace_buf_idx_get(void)
> -{
> -	if (in_irq())
> -		return CFS_TCD_TYPE_IRQ;
> -	if (in_softirq())
> -		return CFS_TCD_TYPE_SOFTIRQ;
> -	return CFS_TCD_TYPE_PROC;
> -}
> -
> -void
> -cfs_set_ptldebug_header(struct ptldebug_header *header,
> -			struct libcfs_debug_msg_data *msgdata,
> -			unsigned long stack)
> -{
> -	struct timespec64 ts;
> -
> -	ktime_get_real_ts64(&ts);
> -
> -	header->ph_subsys = msgdata->msg_subsys;
> -	header->ph_mask = msgdata->msg_mask;
> -	header->ph_cpu_id = smp_processor_id();
> -	header->ph_type = cfs_trace_buf_idx_get();
> -	/* y2038 safe since all user space treats this as unsigned, but
> -	 * will overflow in 2106
> -	 */
> -	header->ph_sec = (u32)ts.tv_sec;
> -	header->ph_usec = ts.tv_nsec / NSEC_PER_USEC;
> -	header->ph_stack = stack;
> -	header->ph_pid = current->pid;
> -	header->ph_line_num = msgdata->msg_line;
> -	header->ph_extern_pid = 0;
> -}
> -
> -static char *
> -dbghdr_to_err_string(struct ptldebug_header *hdr)
> -{
> -	switch (hdr->ph_subsys) {
> -	case S_LND:
> -	case S_LNET:
> -		return "LNetError";
> -	default:
> -		return "LustreError";
> -	}
> -}
> -
> -static char *
> -dbghdr_to_info_string(struct ptldebug_header *hdr)
> -{
> -	switch (hdr->ph_subsys) {
> -	case S_LND:
> -	case S_LNET:
> -		return "LNet";
> -	default:
> -		return "Lustre";
> -	}
> -}
> -
> -void cfs_print_to_console(struct ptldebug_header *hdr, int mask,
> -			  const char *buf, int len, const char *file,
> -			  const char *fn)
> -{
> -	char *prefix = "Lustre", *ptype = NULL;
> -
> -	if (mask & D_EMERG) {
> -		prefix = dbghdr_to_err_string(hdr);
> -		ptype = KERN_EMERG;
> -	} else if (mask & D_ERROR) {
> -		prefix = dbghdr_to_err_string(hdr);
> -		ptype = KERN_ERR;
> -	} else if (mask & D_WARNING) {
> -		prefix = dbghdr_to_info_string(hdr);
> -		ptype = KERN_WARNING;
> -	} else if (mask & (D_CONSOLE | libcfs_printk)) {
> -		prefix = dbghdr_to_info_string(hdr);
> -		ptype = KERN_INFO;
> -	}
> -
> -	if (mask & D_CONSOLE) {
> -		pr_info("%s%s: %.*s", ptype, prefix, len, buf);
> -	} else {
> -		pr_info("%s%s: %d:%d:(%s:%d:%s()) %.*s", ptype, prefix,
> -			hdr->ph_pid, hdr->ph_extern_pid, file,
> -			hdr->ph_line_num, fn, len, buf);
> -	}
> -}
> diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.c b/drivers/staging/lustre/lnet/libcfs/tracefile.c
> index 6d567a9ad8ea..617edc291035 100644
> --- a/drivers/staging/lustre/lnet/libcfs/tracefile.c
> +++ b/drivers/staging/lustre/lnet/libcfs/tracefile.c
> @@ -168,6 +168,161 @@ static inline void cfs_trace_put_tcd(struct cfs_trace_cpu_data *tcd)
>  	put_cpu();
>  }
>  
> +/* percents to share the total debug memory for each type */
> +static unsigned int pages_factor[CFS_TCD_TYPE_MAX] = {
> +	80,  /* 80% pages for CFS_TCD_TYPE_PROC */
> +	10,  /* 10% pages for CFS_TCD_TYPE_SOFTIRQ */
> +	10   /* 10% pages for CFS_TCD_TYPE_IRQ */
> +};
> +
> +char *cfs_trace_console_buffers[NR_CPUS][CFS_TCD_TYPE_MAX];
> +
> +int cfs_tracefile_init_arch(void)
> +{
> +	int i;
> +	int j;
> +	struct cfs_trace_cpu_data *tcd;
> +
> +	/* initialize trace_data */
> +	memset(cfs_trace_data, 0, sizeof(cfs_trace_data));
> +	for (i = 0; i < CFS_TCD_TYPE_MAX; i++) {
> +		cfs_trace_data[i] =
> +			kmalloc_array(num_possible_cpus(),
> +				      sizeof(union cfs_trace_data_union),
> +				      GFP_KERNEL);
> +		if (!cfs_trace_data[i])
> +			goto out;
> +	}
> +
> +	/* arch related info initialized */
> +	cfs_tcd_for_each(tcd, i, j) {
> +		spin_lock_init(&tcd->tcd_lock);
> +		tcd->tcd_pages_factor = pages_factor[i];
> +		tcd->tcd_type = i;
> +		tcd->tcd_cpu = j;
> +	}
> +
> +	for (i = 0; i < num_possible_cpus(); i++)
> +		for (j = 0; j < 3; j++) {
> +			cfs_trace_console_buffers[i][j] =
> +				kmalloc(CFS_TRACE_CONSOLE_BUFFER_SIZE,
> +					GFP_KERNEL);
> +
> +			if (!cfs_trace_console_buffers[i][j])
> +				goto out;
> +		}
> +
> +	return 0;
> +
> +out:
> +	cfs_tracefile_fini_arch();
> +	pr_err("lnet: Not enough memory\n");
> +	return -ENOMEM;
> +}
> +
> +void cfs_tracefile_fini_arch(void)
> +{
> +	int i;
> +	int j;
> +
> +	for (i = 0; i < num_possible_cpus(); i++)
> +		for (j = 0; j < 3; j++) {
> +			kfree(cfs_trace_console_buffers[i][j]);
> +			cfs_trace_console_buffers[i][j] = NULL;
> +		}
> +
> +	for (i = 0; cfs_trace_data[i]; i++) {
> +		kfree(cfs_trace_data[i]);
> +		cfs_trace_data[i] = NULL;
> +	}
> +}
> +
> +enum cfs_trace_buf_type cfs_trace_buf_idx_get(void)
> +{
> +	if (in_irq())
> +		return CFS_TCD_TYPE_IRQ;
> +	if (in_softirq())
> +		return CFS_TCD_TYPE_SOFTIRQ;
> +	return CFS_TCD_TYPE_PROC;
> +}
> +
> +void
> +cfs_set_ptldebug_header(struct ptldebug_header *header,
> +			struct libcfs_debug_msg_data *msgdata,
> +			unsigned long stack)
> +{
> +	struct timespec64 ts;
> +
> +	ktime_get_real_ts64(&ts);
> +
> +	header->ph_subsys = msgdata->msg_subsys;
> +	header->ph_mask = msgdata->msg_mask;
> +	header->ph_cpu_id = smp_processor_id();
> +	header->ph_type = cfs_trace_buf_idx_get();
> +	/* y2038 safe since all user space treats this as unsigned, but
> +	 * will overflow in 2106
> +	 */
> +	header->ph_sec = (u32)ts.tv_sec;
> +	header->ph_usec = ts.tv_nsec / NSEC_PER_USEC;
> +	header->ph_stack = stack;
> +	header->ph_pid = current->pid;
> +	header->ph_line_num = msgdata->msg_line;
> +	header->ph_extern_pid = 0;
> +}
> +
> +static char *
> +dbghdr_to_err_string(struct ptldebug_header *hdr)
> +{
> +	switch (hdr->ph_subsys) {
> +	case S_LND:
> +	case S_LNET:
> +		return "LNetError";
> +	default:
> +		return "LustreError";
> +	}
> +}
> +
> +static char *
> +dbghdr_to_info_string(struct ptldebug_header *hdr)
> +{
> +	switch (hdr->ph_subsys) {
> +	case S_LND:
> +	case S_LNET:
> +		return "LNet";
> +	default:
> +		return "Lustre";
> +	}
> +}
> +
> +void cfs_print_to_console(struct ptldebug_header *hdr, int mask,
> +			  const char *buf, int len, const char *file,
> +			  const char *fn)
> +{
> +	char *prefix = "Lustre", *ptype = NULL;
> +
> +	if (mask & D_EMERG) {
> +		prefix = dbghdr_to_err_string(hdr);
> +		ptype = KERN_EMERG;
> +	} else if (mask & D_ERROR) {
> +		prefix = dbghdr_to_err_string(hdr);
> +		ptype = KERN_ERR;
> +	} else if (mask & D_WARNING) {
> +		prefix = dbghdr_to_info_string(hdr);
> +		ptype = KERN_WARNING;
> +	} else if (mask & (D_CONSOLE | libcfs_printk)) {
> +		prefix = dbghdr_to_info_string(hdr);
> +		ptype = KERN_INFO;
> +	}
> +
> +	if (mask & D_CONSOLE) {
> +		pr_info("%s%s: %.*s", ptype, prefix, len, buf);
> +	} else {
> +		pr_info("%s%s: %d:%d:(%s:%d:%s()) %.*s", ptype, prefix,
> +			hdr->ph_pid, hdr->ph_extern_pid, file,
> +			hdr->ph_line_num, fn, len, buf);
> +	}
> +}
> +
>  static inline struct cfs_trace_page *
>  cfs_tage_from_list(struct list_head *list)
>  {
> 
> 
> 

From jsimmons at infradead.org  Thu Jun 21 01:38:28 2018
From: jsimmons at infradead.org (James Simmons)
Date: Thu, 21 Jun 2018 02:38:28 +0100 (BST)
Subject: [lustre-devel] [PATCH 24/24] lustre: discard TCD_MAX_TYPES
In-Reply-To: <152904669094.10587.17019477478226952858.stgit@noble>
References: <152904663333.10587.10934053155404014785.stgit@noble>
 <152904669094.10587.17019477478226952858.stgit@noble>
Message-ID: <alpine.LFD.2.21.1806210237500.3886@casper.infradead.org>


> As well as CFS_TCD_TYPE_CNT we have TCD_MAX_TYPES which has a larger
> value but a similar meaning.  Discard it and just use
> CFS_TCD_TYPE_CNT.
> 
> Two places relied on the fact that TCD_MAX_TYPES was larger and so
> there would be NULLs at the end of the array.  Change
> them to check the array size properly.
> 
> Signed-off-by: NeilBrown <neilb at suse.com>

Nak: This one really makes my nodes go up in flames when I mount a
file system.

> ---
>  drivers/staging/lustre/lnet/libcfs/tracefile.c |    8 ++++----
>  drivers/staging/lustre/lnet/libcfs/tracefile.h |    3 +--
>  2 files changed, 5 insertions(+), 6 deletions(-)
> 
> diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.c b/drivers/staging/lustre/lnet/libcfs/tracefile.c
> index cdef67391a72..555ade0ddafb 100644
> --- a/drivers/staging/lustre/lnet/libcfs/tracefile.c
> +++ b/drivers/staging/lustre/lnet/libcfs/tracefile.c
> @@ -50,7 +50,7 @@
>  #include "tracefile.h"
>  
>  /* XXX move things up to the top, comment */
> -union cfs_trace_data_union (*cfs_trace_data[TCD_MAX_TYPES])[NR_CPUS] __cacheline_aligned;
> +union cfs_trace_data_union (*cfs_trace_data[CFS_TCD_TYPE_CNT])[NR_CPUS] __cacheline_aligned;
>  
>  char cfs_tracefile[TRACEFILE_NAME_SIZE];
>  long long cfs_tracefile_size = CFS_TRACEFILE_SIZE;
> @@ -145,8 +145,8 @@ void cfs_trace_unlock_tcd(struct cfs_trace_cpu_data *tcd, int walking)
>  		spin_unlock(&tcd->tcd_lock);
>  }
>  
> -#define cfs_tcd_for_each_type_lock(tcd, i, cpu)			   \
> -	for (i = 0; cfs_trace_data[i] &&				\
> +#define cfs_tcd_for_each_type_lock(tcd, i, cpu)				\
> +	for (i = 0; i < CFS_TCD_TYPE_CNT && cfs_trace_data[i] &&	\
>  	     (tcd = &(*cfs_trace_data[i])[cpu].tcd) &&			\
>  	     cfs_trace_lock_tcd(tcd, 1); cfs_trace_unlock_tcd(tcd, 1), i++)
>  
> @@ -1381,7 +1381,7 @@ static void cfs_trace_cleanup(void)
>  			cfs_trace_console_buffers[i][j] = NULL;
>  		}
>  
> -	for (i = 0; cfs_trace_data[i]; i++) {
> +	for (i = 0; cfs_trace_data[i] && i < CFS_TCD_TYPE_CNT; i++) {
>  		kfree(cfs_trace_data[i]);
>  		cfs_trace_data[i] = NULL;
>  	}
> diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.h b/drivers/staging/lustre/lnet/libcfs/tracefile.h
> index 23faecf886c1..a09d5b9349f0 100644
> --- a/drivers/staging/lustre/lnet/libcfs/tracefile.h
> +++ b/drivers/staging/lustre/lnet/libcfs/tracefile.h
> @@ -184,8 +184,7 @@ union cfs_trace_data_union {
>  	char __pad[L1_CACHE_ALIGN(sizeof(struct cfs_trace_cpu_data))];
>  };
>  
> -#define TCD_MAX_TYPES      8
> -extern union cfs_trace_data_union (*cfs_trace_data[TCD_MAX_TYPES])[NR_CPUS];
> +extern union cfs_trace_data_union (*cfs_trace_data[CFS_TCD_TYPE_CNT])[NR_CPUS];
>  
>  #define cfs_tcd_for_each(tcd, i, j)				       \
>  	for (i = 0; cfs_trace_data[i]; i++)				\
> 
> 
> 

From jsimmons at infradead.org  Thu Jun 21 01:48:07 2018
From: jsimmons at infradead.org (James Simmons)
Date: Thu, 21 Jun 2018 02:48:07 +0100 (BST)
Subject: [lustre-devel] [PATCH 00/24] lustre - more cleanups including
 module reduction.
In-Reply-To: <152904663333.10587.10934053155404014785.stgit@noble>
References: <152904663333.10587.10934053155404014785.stgit@noble>
Message-ID: <alpine.LFD.2.21.1806210239190.3886@casper.infradead.org>


> Here are a bunch more cleanups that will appear in my lustre-testing
> tree shortly.
> 
> The interesting stuff is some enhancements to kbuild to let us
> build modules from multiple directories.
> I've used that to clean up the build of ptlrpc+ldlm and to
> merge libcfs and lnet into the one module - I've left the lnds
> separate.

That looks pretty good. It doesn't apply to the stable lustre tree but I
did try it out on lustre-testing. The module building does seem to 
function well. Only thing not clear is if do a

make SUBDIRS=drivers/staging/lustre/lustre/lmv modules

do lmv.a and the other *.a files get relinked int lustre.ko ?
 
> I've haven't progressed further in lustre because I'm not sure which -
> if any - modules are also used by the server.
> I'd rather shared code was in a separate module to client-only.
>
> Which parts of lustre/lustre/* are also used by server?

Basically all the client code is reused on the server except for llite.
The reason is each server type MDS, MGS, or OSS is treated as a client
to each other. So in affect the server code is a middle layer level on
top of the client code.

> After that, it is all fairly simple cleanups.
> 
> I plan to submit the kbuild stuff upstream next week.
> 
> Thanks,
> NeilBrown
> 
> ---
> 
> NeilBrown (24):
>       kbuild: detect directories in components of a module.
>       kbuild: treat a directory listed in a composite object as foo/mod.a
>       kbuild: support building of per-directory mod.a
>       kbuild: disable KBUILD_MODNAME when building for mod.a
>       kbuild: Add documentation for modobj-m
>       lustre: build ldlm in the ldlm directory.
>       lustre: merge libcfs module into lnet.

This seems to work when I tried the lustre-testing tree. I did see 
regression to related to this work in that tree.

>       lustre: use truncate_inode_page in place of truncate_complete_page
>       lustre: don't use spare bits in iattr.ia_valid
>       lustre: remove lustre_patchless_compat.h
>       lustre: discard current_n*groups macros.
>       lustre: discard LTIME_S macro
>       lustre: discard NO_QUOTA and QUOTA_OK
>       lustre: discard ext2* bit operations
>       lustre: centralize TIMES_SET_FLAGS
>       lustre: remove redefinition of module_init().
>       lustre: remove lustre_compat.h

The first two patches of the above seem to break things. The first one was 
pointed out so I avoid issues in my testing by changing the patch to just
move truncate_inode_page() to lustre_mdc.h for now. The next patch does 
break things.

>       lustre/lnet: move tracefile locking from linux-tracefile.c to tracefile.c
>       lustre: open code cfs_trace_max_debug_mb() into cfs_trace_set_debug_mb()
>       lustre: move tcd locking across to tracefile.c
>       lustre: move remainder of linux-tracefile.c to tracefile.c
>       lustre: fold cfs_tracefile_*_arch into their only callers.
>       lustre: renamed CFS_TCD_TYPE_MAX to CFS_TCD_TYPE_CNT
>       lustre: discard TCD_MAX_TYPES

I see breakage in this code as well.

>  Documentation/kbuild/makefiles.txt                 |   65 +++++
>  drivers/staging/lustre/lnet/Makefile               |    7 -
>  drivers/staging/lustre/lnet/libcfs/Makefile        |    7 -
>  drivers/staging/lustre/lnet/libcfs/debug.c         |    6 
>  .../staging/lustre/lnet/libcfs/linux-tracefile.c   |  258 --------------------
>  drivers/staging/lustre/lnet/libcfs/module.c        |   26 +-
>  drivers/staging/lustre/lnet/libcfs/tracefile.c     |  241 +++++++++++++++++--
>  drivers/staging/lustre/lnet/libcfs/tracefile.h     |   40 ---
>  drivers/staging/lustre/lnet/lnet/Makefile          |   11 -
>  drivers/staging/lustre/lnet/lnet/module.c          |    9 +
>  drivers/staging/lustre/lustre/fid/lproc_fid.c      |    1 
>  drivers/staging/lustre/lustre/fld/lproc_fld.c      |    1 
>  drivers/staging/lustre/lustre/include/cl_object.h  |    6 
>  .../staging/lustre/lustre/include/lustre_compat.h  |   82 ------
>  drivers/staging/lustre/lustre/include/lustre_net.h |    1 
>  .../lustre/include/lustre_patchless_compat.h       |   68 -----
>  drivers/staging/lustre/lustre/include/obd.h        |    7 +
>  .../staging/lustre/lustre/include/obd_support.h    |    1 
>  drivers/staging/lustre/lustre/ldlm/Makefile        |   11 +
>  drivers/staging/lustre/lustre/llite/dir.c          |    3 
>  drivers/staging/lustre/lustre/llite/file.c         |   36 +--
>  drivers/staging/lustre/lustre/llite/lcommon_cl.c   |   11 -
>  .../staging/lustre/lustre/llite/llite_internal.h   |   13 +
>  drivers/staging/lustre/lustre/llite/llite_lib.c    |   52 ++--
>  drivers/staging/lustre/lustre/llite/namei.c        |   16 +
>  drivers/staging/lustre/lustre/llite/vvp_io.c       |    8 -
>  drivers/staging/lustre/lustre/llite/vvp_object.c   |    6 
>  drivers/staging/lustre/lustre/lmv/lmv_intent.c     |    6 
>  drivers/staging/lustre/lustre/lmv/lmv_obd.c        |   21 +-
>  drivers/staging/lustre/lustre/lov/lov_io.c         |    3 
>  drivers/staging/lustre/lustre/lov/lov_obd.c        |    2 
>  drivers/staging/lustre/lustre/lov/lov_pack.c       |    1 
>  drivers/staging/lustre/lustre/lov/lov_page.c       |    1 
>  drivers/staging/lustre/lustre/mdc/mdc_lib.c        |   19 +
>  drivers/staging/lustre/lustre/mdc/mdc_reint.c      |    4 
>  drivers/staging/lustre/lustre/mdc/mdc_request.c    |   12 -
>  drivers/staging/lustre/lustre/obdclass/class_obd.c |    1 
>  drivers/staging/lustre/lustre/obdclass/llog.c      |    7 -
>  .../lustre/lustre/obdclass/lprocfs_status.c        |    1 
>  .../staging/lustre/lustre/obdclass/obd_config.c    |    1 
>  drivers/staging/lustre/lustre/obdclass/obd_mount.c |    1 
>  drivers/staging/lustre/lustre/obdclass/obdo.c      |   10 -
>  .../staging/lustre/lustre/obdecho/echo_client.c    |    1 
>  drivers/staging/lustre/lustre/osc/lproc_osc.c      |    1 
>  drivers/staging/lustre/lustre/osc/osc_cache.c      |    4 
>  drivers/staging/lustre/lustre/osc/osc_io.c         |   23 +-
>  drivers/staging/lustre/lustre/osc/osc_quota.c      |    4 
>  drivers/staging/lustre/lustre/osc/osc_request.c    |    1 
>  drivers/staging/lustre/lustre/ptlrpc/Makefile      |   12 -
>  drivers/staging/lustre/lustre/ptlrpc/import.c      |    1 
>  .../staging/lustre/lustre/ptlrpc/lproc_ptlrpc.c    |    1 
>  drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c     |    1 
>  drivers/staging/lustre/lustre/ptlrpc/sec.c         |   15 +
>  drivers/staging/lustre/lustre/ptlrpc/sec_plain.c   |    1 
>  drivers/staging/lustre/lustre/ptlrpc/service.c     |    1 
>  scripts/Makefile.build                             |   57 +++-
>  scripts/Makefile.lib                               |   63 +++--
>  57 files changed, 584 insertions(+), 685 deletions(-)
>  delete mode 100644 drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
>  delete mode 100644 drivers/staging/lustre/lustre/include/lustre_compat.h
>  delete mode 100644 drivers/staging/lustre/lustre/include/lustre_patchless_compat.h
>  create mode 100644 drivers/staging/lustre/lustre/ldlm/Makefile
> 
> --
> Signature
> 
> 

From paf at cray.com  Thu Jun 21 02:29:01 2018
From: paf at cray.com (Patrick Farrell)
Date: Thu, 21 Jun 2018 02:29:01 +0000
Subject: [lustre-devel] [PATCH 00/24] lustre - more cleanups including
 module reduction.
In-Reply-To: <alpine.LFD.2.21.1806210239190.3886@casper.infradead.org>
References: <152904663333.10587.10934053155404014785.stgit@noble>,
 <alpine.LFD.2.21.1806210239190.3886@casper.infradead.org>
Message-ID: <DM3PR1101MB1213DDB7FC857EE6B1444153CB760@DM3PR1101MB1213.namprd11.prod.outlook.com>

Huh?  Very little of the client is used on the server.  The vast majority of the client is llite, vvp, then lov, osc, and mdc.  These are not used on the servers at all, the modules won't even be loaded.


Servers are definitely *not* clients of one another in the sense of "Lustre clients".  They interact with each other, but in a manner very different from client to server.

The ldlm and ptlrpc layers are *partly* shared, and of course, the networking from lnet down is.  But that's not "the client".

The statement that the server layer is a middle layer on top of the client code doesn't make any sense to me.  It's like this:


vfs
client code

ptlrpc

networking

------ PHYSICAL BOUNDARY --------
networking

ptlrpc

server code

ldiskfs



with LDLM also being explicitly shared, though client & server code is not all shared.  Very little to nothing outside of the networking & ldlm layers is shared.

________________________________
From: lustre-devel <lustre-devel-bounces at lists.lustre.org> on behalf of James Simmons <jsimmons at infradead.org>
Sent: Wednesday, June 20, 2018 8:48:07 PM
To: NeilBrown
Cc: Oleg Drokin; Andreas Dilger; Lustre Development List
Subject: Re: [lustre-devel] [PATCH 00/24] lustre - more cleanups including module reduction.


> Here are a bunch more cleanups that will appear in my lustre-testing
> tree shortly.
>
> The interesting stuff is some enhancements to kbuild to let us
> build modules from multiple directories.
> I've used that to clean up the build of ptlrpc+ldlm and to
> merge libcfs and lnet into the one module - I've left the lnds
> separate.

That looks pretty good. It doesn't apply to the stable lustre tree but I
did try it out on lustre-testing. The module building does seem to
function well. Only thing not clear is if do a

make SUBDIRS=drivers/staging/lustre/lustre/lmv modules

do lmv.a and the other *.a files get relinked int lustre.ko ?

> I've haven't progressed further in lustre because I'm not sure which -
> if any - modules are also used by the server.
> I'd rather shared code was in a separate module to client-only.
>
> Which parts of lustre/lustre/* are also used by server?

Basically all the client code is reused on the server except for llite.
The reason is each server type MDS, MGS, or OSS is treated as a client
to each other. So in affect the server code is a middle layer level on
top of the client code.

> After that, it is all fairly simple cleanups.
>
> I plan to submit the kbuild stuff upstream next week.
>
> Thanks,
> NeilBrown
>
> ---
>
> NeilBrown (24):
>       kbuild: detect directories in components of a module.
>       kbuild: treat a directory listed in a composite object as foo/mod.a
>       kbuild: support building of per-directory mod.a
>       kbuild: disable KBUILD_MODNAME when building for mod.a
>       kbuild: Add documentation for modobj-m
>       lustre: build ldlm in the ldlm directory.
>       lustre: merge libcfs module into lnet.

This seems to work when I tried the lustre-testing tree. I did see
regression to related to this work in that tree.

>       lustre: use truncate_inode_page in place of truncate_complete_page
>       lustre: don't use spare bits in iattr.ia_valid
>       lustre: remove lustre_patchless_compat.h
>       lustre: discard current_n*groups macros.
>       lustre: discard LTIME_S macro
>       lustre: discard NO_QUOTA and QUOTA_OK
>       lustre: discard ext2* bit operations
>       lustre: centralize TIMES_SET_FLAGS
>       lustre: remove redefinition of module_init().
>       lustre: remove lustre_compat.h

The first two patches of the above seem to break things. The first one was
pointed out so I avoid issues in my testing by changing the patch to just
move truncate_inode_page() to lustre_mdc.h for now. The next patch does
break things.

>       lustre/lnet: move tracefile locking from linux-tracefile.c to tracefile.c
>       lustre: open code cfs_trace_max_debug_mb() into cfs_trace_set_debug_mb()
>       lustre: move tcd locking across to tracefile.c
>       lustre: move remainder of linux-tracefile.c to tracefile.c
>       lustre: fold cfs_tracefile_*_arch into their only callers.
>       lustre: renamed CFS_TCD_TYPE_MAX to CFS_TCD_TYPE_CNT
>       lustre: discard TCD_MAX_TYPES

I see breakage in this code as well.

>  Documentation/kbuild/makefiles.txt                 |   65 +++++
>  drivers/staging/lustre/lnet/Makefile               |    7 -
>  drivers/staging/lustre/lnet/libcfs/Makefile        |    7 -
>  drivers/staging/lustre/lnet/libcfs/debug.c         |    6
>  .../staging/lustre/lnet/libcfs/linux-tracefile.c   |  258 --------------------
>  drivers/staging/lustre/lnet/libcfs/module.c        |   26 +-
>  drivers/staging/lustre/lnet/libcfs/tracefile.c     |  241 +++++++++++++++++--
>  drivers/staging/lustre/lnet/libcfs/tracefile.h     |   40 ---
>  drivers/staging/lustre/lnet/lnet/Makefile          |   11 -
>  drivers/staging/lustre/lnet/lnet/module.c          |    9 +
>  drivers/staging/lustre/lustre/fid/lproc_fid.c      |    1
>  drivers/staging/lustre/lustre/fld/lproc_fld.c      |    1
>  drivers/staging/lustre/lustre/include/cl_object.h  |    6
>  .../staging/lustre/lustre/include/lustre_compat.h  |   82 ------
>  drivers/staging/lustre/lustre/include/lustre_net.h |    1
>  .../lustre/include/lustre_patchless_compat.h       |   68 -----
>  drivers/staging/lustre/lustre/include/obd.h        |    7 +
>  .../staging/lustre/lustre/include/obd_support.h    |    1
>  drivers/staging/lustre/lustre/ldlm/Makefile        |   11 +
>  drivers/staging/lustre/lustre/llite/dir.c          |    3
>  drivers/staging/lustre/lustre/llite/file.c         |   36 +--
>  drivers/staging/lustre/lustre/llite/lcommon_cl.c   |   11 -
>  .../staging/lustre/lustre/llite/llite_internal.h   |   13 +
>  drivers/staging/lustre/lustre/llite/llite_lib.c    |   52 ++--
>  drivers/staging/lustre/lustre/llite/namei.c        |   16 +
>  drivers/staging/lustre/lustre/llite/vvp_io.c       |    8 -
>  drivers/staging/lustre/lustre/llite/vvp_object.c   |    6
>  drivers/staging/lustre/lustre/lmv/lmv_intent.c     |    6
>  drivers/staging/lustre/lustre/lmv/lmv_obd.c        |   21 +-
>  drivers/staging/lustre/lustre/lov/lov_io.c         |    3
>  drivers/staging/lustre/lustre/lov/lov_obd.c        |    2
>  drivers/staging/lustre/lustre/lov/lov_pack.c       |    1
>  drivers/staging/lustre/lustre/lov/lov_page.c       |    1
>  drivers/staging/lustre/lustre/mdc/mdc_lib.c        |   19 +
>  drivers/staging/lustre/lustre/mdc/mdc_reint.c      |    4
>  drivers/staging/lustre/lustre/mdc/mdc_request.c    |   12 -
>  drivers/staging/lustre/lustre/obdclass/class_obd.c |    1
>  drivers/staging/lustre/lustre/obdclass/llog.c      |    7 -
>  .../lustre/lustre/obdclass/lprocfs_status.c        |    1
>  .../staging/lustre/lustre/obdclass/obd_config.c    |    1
>  drivers/staging/lustre/lustre/obdclass/obd_mount.c |    1
>  drivers/staging/lustre/lustre/obdclass/obdo.c      |   10 -
>  .../staging/lustre/lustre/obdecho/echo_client.c    |    1
>  drivers/staging/lustre/lustre/osc/lproc_osc.c      |    1
>  drivers/staging/lustre/lustre/osc/osc_cache.c      |    4
>  drivers/staging/lustre/lustre/osc/osc_io.c         |   23 +-
>  drivers/staging/lustre/lustre/osc/osc_quota.c      |    4
>  drivers/staging/lustre/lustre/osc/osc_request.c    |    1
>  drivers/staging/lustre/lustre/ptlrpc/Makefile      |   12 -
>  drivers/staging/lustre/lustre/ptlrpc/import.c      |    1
>  .../staging/lustre/lustre/ptlrpc/lproc_ptlrpc.c    |    1
>  drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c     |    1
>  drivers/staging/lustre/lustre/ptlrpc/sec.c         |   15 +
>  drivers/staging/lustre/lustre/ptlrpc/sec_plain.c   |    1
>  drivers/staging/lustre/lustre/ptlrpc/service.c     |    1
>  scripts/Makefile.build                             |   57 +++-
>  scripts/Makefile.lib                               |   63 +++--
>  57 files changed, 584 insertions(+), 685 deletions(-)
>  delete mode 100644 drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
>  delete mode 100644 drivers/staging/lustre/lustre/include/lustre_compat.h
>  delete mode 100644 drivers/staging/lustre/lustre/include/lustre_patchless_compat.h
>  create mode 100644 drivers/staging/lustre/lustre/ldlm/Makefile
>
> --
> Signature
>
>
_______________________________________________
lustre-devel mailing list
lustre-devel at lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180621/c4685e67/attachment.html>

From neilb at suse.com  Thu Jun 21 02:54:58 2018
From: neilb at suse.com (NeilBrown)
Date: Thu, 21 Jun 2018 12:54:58 +1000
Subject: [lustre-devel] [PATCH 00/24] lustre - more cleanups including
 module reduction.
In-Reply-To: <alpine.LFD.2.21.1806210239190.3886@casper.infradead.org>
References: <152904663333.10587.10934053155404014785.stgit@noble>
 <alpine.LFD.2.21.1806210239190.3886@casper.infradead.org>
Message-ID: <87r2l0df25.fsf@notabene.neil.brown.name>

On Thu, Jun 21 2018, James Simmons wrote:

>> Here are a bunch more cleanups that will appear in my lustre-testing
>> tree shortly.
>> 
>> The interesting stuff is some enhancements to kbuild to let us
>> build modules from multiple directories.
>> I've used that to clean up the build of ptlrpc+ldlm and to
>> merge libcfs and lnet into the one module - I've left the lnds
>> separate.
>
> That looks pretty good. It doesn't apply to the stable lustre tree but I
> did try it out on lustre-testing. The module building does seem to 
> function well. Only thing not clear is if do a
>
> make SUBDIRS=drivers/staging/lustre/lustre/lmv modules

Why would you do that?  SUBDIRS= is for building external modules.
What are you trying to achieve?

>
> do lmv.a and the other *.a files get relinked int lustre.ko ?

It isn't "lmv.a", it is ".../lustre/lmv/mod.a".

>  
>> I've haven't progressed further in lustre because I'm not sure which -
>> if any - modules are also used by the server.
>> I'd rather shared code was in a separate module to client-only.
>>
>> Which parts of lustre/lustre/* are also used by server?
>
> Basically all the client code is reused on the server except for llite.
> The reason is each server type MDS, MGS, or OSS is treated as a client
> to each other. So in affect the server code is a middle layer level on
> top of the client code.
>
>> After that, it is all fairly simple cleanups.
>> 
>> I plan to submit the kbuild stuff upstream next week.
>> 
>> Thanks,
>> NeilBrown
>> 
>> ---
>> 
>> NeilBrown (24):
>>       kbuild: detect directories in components of a module.
>>       kbuild: treat a directory listed in a composite object as foo/mod.a
>>       kbuild: support building of per-directory mod.a
>>       kbuild: disable KBUILD_MODNAME when building for mod.a
>>       kbuild: Add documentation for modobj-m
>>       lustre: build ldlm in the ldlm directory.
>>       lustre: merge libcfs module into lnet.
>
> This seems to work when I tried the lustre-testing tree. I did see 
> regression to related to this work in that tree.
>
>>       lustre: use truncate_inode_page in place of truncate_complete_page
>>       lustre: don't use spare bits in iattr.ia_valid
>>       lustre: remove lustre_patchless_compat.h
>>       lustre: discard current_n*groups macros.
>>       lustre: discard LTIME_S macro
>>       lustre: discard NO_QUOTA and QUOTA_OK
>>       lustre: discard ext2* bit operations
>>       lustre: centralize TIMES_SET_FLAGS
>>       lustre: remove redefinition of module_init().
>>       lustre: remove lustre_compat.h
>
> The first two patches of the above seem to break things. The first one was 
> pointed out so I avoid issues in my testing by changing the patch to just
> move truncate_inode_page() to lustre_mdc.h for now. The next patch does 
> break things.

Can you be specific about what things (or just one thing) that the next
patch breaks?  I haven't seen anything in my testing.
The only problem I've seen is that kvmalloc(GFP_NOFS) has been changed to
silently become kmalloc(GFP_NOFS) so there are memory allocation
problems.

>
>>       lustre/lnet: move tracefile locking from linux-tracefile.c to tracefile.c
>>       lustre: open code cfs_trace_max_debug_mb() into cfs_trace_set_debug_mb()
>>       lustre: move tcd locking across to tracefile.c
>>       lustre: move remainder of linux-tracefile.c to tracefile.c
>>       lustre: fold cfs_tracefile_*_arch into their only callers.
>>       lustre: renamed CFS_TCD_TYPE_MAX to CFS_TCD_TYPE_CNT
>>       lustre: discard TCD_MAX_TYPES
>
> I see breakage in this code as well.

Specifically??...

Thanks,
NeilBrown
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180621/ed188aeb/attachment.sig>

From jsimmons at infradead.org  Thu Jun 21 02:57:49 2018
From: jsimmons at infradead.org (James Simmons)
Date: Thu, 21 Jun 2018 03:57:49 +0100 (BST)
Subject: [lustre-devel] [PATCH 00/24] lustre - more cleanups including
 module reduction.
In-Reply-To: <DM3PR1101MB1213DDB7FC857EE6B1444153CB760@DM3PR1101MB1213.namprd11.prod.outlook.com>
References: <152904663333.10587.10934053155404014785.stgit@noble>,
 <alpine.LFD.2.21.1806210239190.3886@casper.infradead.org>
 <DM3PR1101MB1213DDB7FC857EE6B1444153CB760@DM3PR1101MB1213.namprd11.prod.outlook.com>
Message-ID: <alpine.LFD.2.21.1806210354260.3886@casper.infradead.org>


> Huh?  Very little of the client is used on the server.  The vast majority of the client is llite, vvp, then lov, osc, and mdc.  These are
> not used on the servers at all, the modules won't even be loaded.
> 
> 
> Servers are definitely *not* clients of one another in the sense of "Lustre clients".  They interact with each other, but in a manner very
> different from client to server.
> 
> The ldlm and ptlrpc layers are *partly* shared, and of course, the networking from lnet down is.  But that's not "the client".
> 
> The statement that the server layer is a middle layer on top of the client code doesn't make any sense to me.  It's like this:
> 
> 
> vfs
> client code
> 
> ptlrpc
> 
> networking
> 
> ------ PHYSICAL BOUNDARY --------
> networking
> 
> ptlrpc
> 
> server code
> 
> ldiskfs
> 
> 
> 
> with LDLM also being explicitly shared, though client & server code is not all shared.  Very little to nothing outside of the networking &
> ldlm layers is shared.

When I first started this (pre 2.3 days) that is how it was explained to 
me. I just took it at face value but when it comes to the separation as I 
don't have a clue. Patrick since you have a better grasp of the 
architecture can you provide details to Neil. Perhaps to documentation on
this separation if it exist.

> > Here are a bunch more cleanups that will appear in my lustre-testing
> > tree shortly.
> >
> > The interesting stuff is some enhancements to kbuild to let us
> > build modules from multiple directories.
> > I've used that to clean up the build of ptlrpc+ldlm and to
> > merge libcfs and lnet into the one module - I've left the lnds
> > separate.
> 
> That looks pretty good. It doesn't apply to the stable lustre tree but I
> did try it out on lustre-testing. The module building does seem to
> function well. Only thing not clear is if do a
> 
> make SUBDIRS=drivers/staging/lustre/lustre/lmv modules
> 
> do lmv.a and the other *.a files get relinked int lustre.ko ?
>  
> > I've haven't progressed further in lustre because I'm not sure which -
> > if any - modules are also used by the server.
> > I'd rather shared code was in a separate module to client-only.
> >
> > Which parts of lustre/lustre/* are also used by server?
> 
> Basically all the client code is reused on the server except for llite.
> The reason is each server type MDS, MGS, or OSS is treated as a client
> to each other. So in affect the server code is a middle layer level on
> top of the client code.
> 
> > After that, it is all fairly simple cleanups.
> >
> > I plan to submit the kbuild stuff upstream next week.
> >
> > Thanks,
> > NeilBrown
> >
> > ---
> >
> > NeilBrown (24):
> >       kbuild: detect directories in components of a module.
> >       kbuild: treat a directory listed in a composite object as foo/mod.a
> >       kbuild: support building of per-directory mod.a
> >       kbuild: disable KBUILD_MODNAME when building for mod.a
> >       kbuild: Add documentation for modobj-m
> >       lustre: build ldlm in the ldlm directory.
> >       lustre: merge libcfs module into lnet.
> 
> This seems to work when I tried the lustre-testing tree. I did see
> regression to related to this work in that tree.
> 
> >       lustre: use truncate_inode_page in place of truncate_complete_page
> >       lustre: don't use spare bits in iattr.ia_valid
> >       lustre: remove lustre_patchless_compat.h
> >       lustre: discard current_n*groups macros.
> >       lustre: discard LTIME_S macro
> >       lustre: discard NO_QUOTA and QUOTA_OK
> >       lustre: discard ext2* bit operations
> >       lustre: centralize TIMES_SET_FLAGS
> >       lustre: remove redefinition of module_init().
> >       lustre: remove lustre_compat.h
> 
> The first two patches of the above seem to break things. The first one was
> pointed out so I avoid issues in my testing by changing the patch to just
> move truncate_inode_page() to lustre_mdc.h for now. The next patch does
> break things.
> 
> >       lustre/lnet: move tracefile locking from linux-tracefile.c to tracefile.c
> >       lustre: open code cfs_trace_max_debug_mb() into cfs_trace_set_debug_mb()
> >       lustre: move tcd locking across to tracefile.c
> >       lustre: move remainder of linux-tracefile.c to tracefile.c
> >       lustre: fold cfs_tracefile_*_arch into their only callers.
> >       lustre: renamed CFS_TCD_TYPE_MAX to CFS_TCD_TYPE_CNT
> >       lustre: discard TCD_MAX_TYPES
> 
> I see breakage in this code as well.
> 
> >  Documentation/kbuild/makefiles.txt                 |   65 +++++
> >  drivers/staging/lustre/lnet/Makefile               |    7 -
> >  drivers/staging/lustre/lnet/libcfs/Makefile        |    7 -
> >  drivers/staging/lustre/lnet/libcfs/debug.c         |    6
> >  .../staging/lustre/lnet/libcfs/linux-tracefile.c   |  258 --------------------
> >  drivers/staging/lustre/lnet/libcfs/module.c        |   26 +-
> >  drivers/staging/lustre/lnet/libcfs/tracefile.c     |  241 +++++++++++++++++--
> >  drivers/staging/lustre/lnet/libcfs/tracefile.h     |   40 ---
> >  drivers/staging/lustre/lnet/lnet/Makefile          |   11 -
> >  drivers/staging/lustre/lnet/lnet/module.c          |    9 +
> >  drivers/staging/lustre/lustre/fid/lproc_fid.c      |    1
> >  drivers/staging/lustre/lustre/fld/lproc_fld.c      |    1
> >  drivers/staging/lustre/lustre/include/cl_object.h  |    6
> >  .../staging/lustre/lustre/include/lustre_compat.h  |   82 ------
> >  drivers/staging/lustre/lustre/include/lustre_net.h |    1
> >  .../lustre/include/lustre_patchless_compat.h       |   68 -----
> >  drivers/staging/lustre/lustre/include/obd.h        |    7 +
> >  .../staging/lustre/lustre/include/obd_support.h    |    1
> >  drivers/staging/lustre/lustre/ldlm/Makefile        |   11 +
> >  drivers/staging/lustre/lustre/llite/dir.c          |    3
> >  drivers/staging/lustre/lustre/llite/file.c         |   36 +--
> >  drivers/staging/lustre/lustre/llite/lcommon_cl.c   |   11 -
> >  .../staging/lustre/lustre/llite/llite_internal.h   |   13 +
> >  drivers/staging/lustre/lustre/llite/llite_lib.c    |   52 ++--
> >  drivers/staging/lustre/lustre/llite/namei.c        |   16 +
> >  drivers/staging/lustre/lustre/llite/vvp_io.c       |    8 -
> >  drivers/staging/lustre/lustre/llite/vvp_object.c   |    6
> >  drivers/staging/lustre/lustre/lmv/lmv_intent.c     |    6
> >  drivers/staging/lustre/lustre/lmv/lmv_obd.c        |   21 +-
> >  drivers/staging/lustre/lustre/lov/lov_io.c         |    3
> >  drivers/staging/lustre/lustre/lov/lov_obd.c        |    2
> >  drivers/staging/lustre/lustre/lov/lov_pack.c       |    1
> >  drivers/staging/lustre/lustre/lov/lov_page.c       |    1
> >  drivers/staging/lustre/lustre/mdc/mdc_lib.c        |   19 +
> >  drivers/staging/lustre/lustre/mdc/mdc_reint.c      |    4
> >  drivers/staging/lustre/lustre/mdc/mdc_request.c    |   12 -
> >  drivers/staging/lustre/lustre/obdclass/class_obd.c |    1
> >  drivers/staging/lustre/lustre/obdclass/llog.c      |    7 -
> >  .../lustre/lustre/obdclass/lprocfs_status.c        |    1
> >  .../staging/lustre/lustre/obdclass/obd_config.c    |    1
> >  drivers/staging/lustre/lustre/obdclass/obd_mount.c |    1
> >  drivers/staging/lustre/lustre/obdclass/obdo.c      |   10 -
> >  .../staging/lustre/lustre/obdecho/echo_client.c    |    1
> >  drivers/staging/lustre/lustre/osc/lproc_osc.c      |    1
> >  drivers/staging/lustre/lustre/osc/osc_cache.c      |    4
> >  drivers/staging/lustre/lustre/osc/osc_io.c         |   23 +-
> >  drivers/staging/lustre/lustre/osc/osc_quota.c      |    4
> >  drivers/staging/lustre/lustre/osc/osc_request.c    |    1
> >  drivers/staging/lustre/lustre/ptlrpc/Makefile      |   12 -
> >  drivers/staging/lustre/lustre/ptlrpc/import.c      |    1
> >  .../staging/lustre/lustre/ptlrpc/lproc_ptlrpc.c    |    1
> >  drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c     |    1
> >  drivers/staging/lustre/lustre/ptlrpc/sec.c         |   15 +
> >  drivers/staging/lustre/lustre/ptlrpc/sec_plain.c   |    1
> >  drivers/staging/lustre/lustre/ptlrpc/service.c     |    1
> >  scripts/Makefile.build                             |   57 +++-
> >  scripts/Makefile.lib                               |   63 +++--
> >  57 files changed, 584 insertions(+), 685 deletions(-)
> >  delete mode 100644 drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
> >  delete mode 100644 drivers/staging/lustre/lustre/include/lustre_compat.h
> >  delete mode 100644 drivers/staging/lustre/lustre/include/lustre_patchless_compat.h
> >  create mode 100644 drivers/staging/lustre/lustre/ldlm/Makefile
> >
> > --
> > Signature
> >
> >
> _______________________________________________
> lustre-devel mailing list
> lustre-devel at lists.lustre.org
> http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org
> 
> 

From neilb at suse.com  Thu Jun 21 03:00:54 2018
From: neilb at suse.com (NeilBrown)
Date: Thu, 21 Jun 2018 13:00:54 +1000
Subject: [lustre-devel] [PATCH 00/24] lustre - more cleanups including
 module reduction.
In-Reply-To: <DM3PR1101MB1213DDB7FC857EE6B1444153CB760@DM3PR1101MB1213.namprd11.prod.outlook.com>
References: <152904663333.10587.10934053155404014785.stgit@noble>
 <alpine.LFD.2.21.1806210239190.3886@casper.infradead.org>
 <DM3PR1101MB1213DDB7FC857EE6B1444153CB760@DM3PR1101MB1213.namprd11.prod.outlook.com>
Message-ID: <87o9g4des9.fsf@notabene.neil.brown.name>

On Thu, Jun 21 2018, Patrick Farrell wrote:

> Huh?  Very little of the client is used on the server.  The vast majority of the client is llite, vvp, then lov, osc, and mdc.  These are not used on the servers at all, the modules won't even be loaded.
>
>
> Servers are definitely *not* clients of one another in the sense of "Lustre clients".  They interact with each other, but in a manner very different from client to server.
>
> The ldlm and ptlrpc layers are *partly* shared, and of course, the networking from lnet down is.  But that's not "the client".
>
> The statement that the server layer is a middle layer on top of the client code doesn't make any sense to me.  It's like this:

So llite fid fld lmv lov mdc mgc osc and obdclass
can all be in a "lustre" module.
ldlm and ptlrpc should be in a separate "ptlrpc" module.
And lnet and the klnds are also separate.

On the server side we would have
  lod mdd mdt mgs ofd osp ost

does that seem approximately right?

Thanks,
NeilBrown


>
>
> vfs
> client code
>
> ptlrpc
>
> networking
>
> ------ PHYSICAL BOUNDARY --------
> networking
>
> ptlrpc
>
> server code
>
> ldiskfs
>
>
>
> with LDLM also being explicitly shared, though client & server code is not all shared.  Very little to nothing outside of the networking & ldlm layers is shared.
>
> ________________________________
> From: lustre-devel <lustre-devel-bounces at lists.lustre.org> on behalf of James Simmons <jsimmons at infradead.org>
> Sent: Wednesday, June 20, 2018 8:48:07 PM
> To: NeilBrown
> Cc: Oleg Drokin; Andreas Dilger; Lustre Development List
> Subject: Re: [lustre-devel] [PATCH 00/24] lustre - more cleanups including module reduction.
>
>
>> Here are a bunch more cleanups that will appear in my lustre-testing
>> tree shortly.
>>
>> The interesting stuff is some enhancements to kbuild to let us
>> build modules from multiple directories.
>> I've used that to clean up the build of ptlrpc+ldlm and to
>> merge libcfs and lnet into the one module - I've left the lnds
>> separate.
>
> That looks pretty good. It doesn't apply to the stable lustre tree but I
> did try it out on lustre-testing. The module building does seem to
> function well. Only thing not clear is if do a
>
> make SUBDIRS=drivers/staging/lustre/lustre/lmv modules
>
> do lmv.a and the other *.a files get relinked int lustre.ko ?
>
>> I've haven't progressed further in lustre because I'm not sure which -
>> if any - modules are also used by the server.
>> I'd rather shared code was in a separate module to client-only.
>>
>> Which parts of lustre/lustre/* are also used by server?
>
> Basically all the client code is reused on the server except for llite.
> The reason is each server type MDS, MGS, or OSS is treated as a client
> to each other. So in affect the server code is a middle layer level on
> top of the client code.
>
>> After that, it is all fairly simple cleanups.
>>
>> I plan to submit the kbuild stuff upstream next week.
>>
>> Thanks,
>> NeilBrown
>>
>> ---
>>
>> NeilBrown (24):
>>       kbuild: detect directories in components of a module.
>>       kbuild: treat a directory listed in a composite object as foo/mod.a
>>       kbuild: support building of per-directory mod.a
>>       kbuild: disable KBUILD_MODNAME when building for mod.a
>>       kbuild: Add documentation for modobj-m
>>       lustre: build ldlm in the ldlm directory.
>>       lustre: merge libcfs module into lnet.
>
> This seems to work when I tried the lustre-testing tree. I did see
> regression to related to this work in that tree.
>
>>       lustre: use truncate_inode_page in place of truncate_complete_page
>>       lustre: don't use spare bits in iattr.ia_valid
>>       lustre: remove lustre_patchless_compat.h
>>       lustre: discard current_n*groups macros.
>>       lustre: discard LTIME_S macro
>>       lustre: discard NO_QUOTA and QUOTA_OK
>>       lustre: discard ext2* bit operations
>>       lustre: centralize TIMES_SET_FLAGS
>>       lustre: remove redefinition of module_init().
>>       lustre: remove lustre_compat.h
>
> The first two patches of the above seem to break things. The first one was
> pointed out so I avoid issues in my testing by changing the patch to just
> move truncate_inode_page() to lustre_mdc.h for now. The next patch does
> break things.
>
>>       lustre/lnet: move tracefile locking from linux-tracefile.c to tracefile.c
>>       lustre: open code cfs_trace_max_debug_mb() into cfs_trace_set_debug_mb()
>>       lustre: move tcd locking across to tracefile.c
>>       lustre: move remainder of linux-tracefile.c to tracefile.c
>>       lustre: fold cfs_tracefile_*_arch into their only callers.
>>       lustre: renamed CFS_TCD_TYPE_MAX to CFS_TCD_TYPE_CNT
>>       lustre: discard TCD_MAX_TYPES
>
> I see breakage in this code as well.
>
>>  Documentation/kbuild/makefiles.txt                 |   65 +++++
>>  drivers/staging/lustre/lnet/Makefile               |    7 -
>>  drivers/staging/lustre/lnet/libcfs/Makefile        |    7 -
>>  drivers/staging/lustre/lnet/libcfs/debug.c         |    6
>>  .../staging/lustre/lnet/libcfs/linux-tracefile.c   |  258 --------------------
>>  drivers/staging/lustre/lnet/libcfs/module.c        |   26 +-
>>  drivers/staging/lustre/lnet/libcfs/tracefile.c     |  241 +++++++++++++++++--
>>  drivers/staging/lustre/lnet/libcfs/tracefile.h     |   40 ---
>>  drivers/staging/lustre/lnet/lnet/Makefile          |   11 -
>>  drivers/staging/lustre/lnet/lnet/module.c          |    9 +
>>  drivers/staging/lustre/lustre/fid/lproc_fid.c      |    1
>>  drivers/staging/lustre/lustre/fld/lproc_fld.c      |    1
>>  drivers/staging/lustre/lustre/include/cl_object.h  |    6
>>  .../staging/lustre/lustre/include/lustre_compat.h  |   82 ------
>>  drivers/staging/lustre/lustre/include/lustre_net.h |    1
>>  .../lustre/include/lustre_patchless_compat.h       |   68 -----
>>  drivers/staging/lustre/lustre/include/obd.h        |    7 +
>>  .../staging/lustre/lustre/include/obd_support.h    |    1
>>  drivers/staging/lustre/lustre/ldlm/Makefile        |   11 +
>>  drivers/staging/lustre/lustre/llite/dir.c          |    3
>>  drivers/staging/lustre/lustre/llite/file.c         |   36 +--
>>  drivers/staging/lustre/lustre/llite/lcommon_cl.c   |   11 -
>>  .../staging/lustre/lustre/llite/llite_internal.h   |   13 +
>>  drivers/staging/lustre/lustre/llite/llite_lib.c    |   52 ++--
>>  drivers/staging/lustre/lustre/llite/namei.c        |   16 +
>>  drivers/staging/lustre/lustre/llite/vvp_io.c       |    8 -
>>  drivers/staging/lustre/lustre/llite/vvp_object.c   |    6
>>  drivers/staging/lustre/lustre/lmv/lmv_intent.c     |    6
>>  drivers/staging/lustre/lustre/lmv/lmv_obd.c        |   21 +-
>>  drivers/staging/lustre/lustre/lov/lov_io.c         |    3
>>  drivers/staging/lustre/lustre/lov/lov_obd.c        |    2
>>  drivers/staging/lustre/lustre/lov/lov_pack.c       |    1
>>  drivers/staging/lustre/lustre/lov/lov_page.c       |    1
>>  drivers/staging/lustre/lustre/mdc/mdc_lib.c        |   19 +
>>  drivers/staging/lustre/lustre/mdc/mdc_reint.c      |    4
>>  drivers/staging/lustre/lustre/mdc/mdc_request.c    |   12 -
>>  drivers/staging/lustre/lustre/obdclass/class_obd.c |    1
>>  drivers/staging/lustre/lustre/obdclass/llog.c      |    7 -
>>  .../lustre/lustre/obdclass/lprocfs_status.c        |    1
>>  .../staging/lustre/lustre/obdclass/obd_config.c    |    1
>>  drivers/staging/lustre/lustre/obdclass/obd_mount.c |    1
>>  drivers/staging/lustre/lustre/obdclass/obdo.c      |   10 -
>>  .../staging/lustre/lustre/obdecho/echo_client.c    |    1
>>  drivers/staging/lustre/lustre/osc/lproc_osc.c      |    1
>>  drivers/staging/lustre/lustre/osc/osc_cache.c      |    4
>>  drivers/staging/lustre/lustre/osc/osc_io.c         |   23 +-
>>  drivers/staging/lustre/lustre/osc/osc_quota.c      |    4
>>  drivers/staging/lustre/lustre/osc/osc_request.c    |    1
>>  drivers/staging/lustre/lustre/ptlrpc/Makefile      |   12 -
>>  drivers/staging/lustre/lustre/ptlrpc/import.c      |    1
>>  .../staging/lustre/lustre/ptlrpc/lproc_ptlrpc.c    |    1
>>  drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c     |    1
>>  drivers/staging/lustre/lustre/ptlrpc/sec.c         |   15 +
>>  drivers/staging/lustre/lustre/ptlrpc/sec_plain.c   |    1
>>  drivers/staging/lustre/lustre/ptlrpc/service.c     |    1
>>  scripts/Makefile.build                             |   57 +++-
>>  scripts/Makefile.lib                               |   63 +++--
>>  57 files changed, 584 insertions(+), 685 deletions(-)
>>  delete mode 100644 drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
>>  delete mode 100644 drivers/staging/lustre/lustre/include/lustre_compat.h
>>  delete mode 100644 drivers/staging/lustre/lustre/include/lustre_patchless_compat.h
>>  create mode 100644 drivers/staging/lustre/lustre/ldlm/Makefile
>>
>> --
>> Signature
>>
>>
> _______________________________________________
> lustre-devel mailing list
> lustre-devel at lists.lustre.org
> http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180621/c2bc7f0e/attachment.sig>

From paf at cray.com  Thu Jun 21 03:42:54 2018
From: paf at cray.com (Patrick Farrell)
Date: Thu, 21 Jun 2018 03:42:54 +0000
Subject: [lustre-devel] [PATCH 00/24] lustre - more cleanups including
 module reduction.
In-Reply-To: <87o9g4des9.fsf@notabene.neil.brown.name>
References: <152904663333.10587.10934053155404014785.stgit@noble>
 <alpine.LFD.2.21.1806210239190.3886@casper.infradead.org>
 <DM3PR1101MB1213DDB7FC857EE6B1444153CB760@DM3PR1101MB1213.namprd11.prod.outlook.com>,
 <87o9g4des9.fsf@notabene.neil.brown.name>
Message-ID: <DM3PR1101MB1213A65A088018D497D20FB1CB760@DM3PR1101MB1213.namprd11.prod.outlook.com>


I’m hoping someone with a better memory or who’s in a position to check this a bit better (no computer access this second) will speak up, but...

To a first approximation, yes, that sounds close.  Probably missing something but...

Also, not sure about fid and fld.  Might be needed on the server...?  But you will of course find you when you try.

________________________________
From: NeilBrown <neilb at suse.com>
Sent: Wednesday, June 20, 2018 10:00:54 PM
To: Patrick Farrell; James Simmons
Cc: Lustre Development List
Subject: Re: [lustre-devel] [PATCH 00/24] lustre - more cleanups including module reduction.

On Thu, Jun 21 2018, Patrick Farrell wrote:

> Huh?  Very little of the client is used on the server.  The vast majority of the client is llite, vvp, then lov, osc, and mdc.  These are not used on the servers at all, the modules won't even be loaded.
>
>
> Servers are definitely *not* clients of one another in the sense of "Lustre clients".  They interact with each other, but in a manner very different from client to server.
>
> The ldlm and ptlrpc layers are *partly* shared, and of course, the networking from lnet down is.  But that's not "the client".
>
> The statement that the server layer is a middle layer on top of the client code doesn't make any sense to me.  It's like this:

So llite fid fld lmv lov mdc mgc osc and obdclass
can all be in a "lustre" module.
ldlm and ptlrpc should be in a separate "ptlrpc" module.
And lnet and the klnds are also separate.

On the server side we would have
  lod mdd mdt mgs ofd osp ost

does that seem approximately right?

Thanks,
NeilBrown


>
>
> vfs
> client code
>
> ptlrpc
>
> networking
>
> ------ PHYSICAL BOUNDARY --------
> networking
>
> ptlrpc
>
> server code
>
> ldiskfs
>
>
>
> with LDLM also being explicitly shared, though client & server code is not all shared.  Very little to nothing outside of the networking & ldlm layers is shared.
>
> ________________________________
> From: lustre-devel <lustre-devel-bounces at lists.lustre.org> on behalf of James Simmons <jsimmons at infradead.org>
> Sent: Wednesday, June 20, 2018 8:48:07 PM
> To: NeilBrown
> Cc: Oleg Drokin; Andreas Dilger; Lustre Development List
> Subject: Re: [lustre-devel] [PATCH 00/24] lustre - more cleanups including module reduction.
>
>
>> Here are a bunch more cleanups that will appear in my lustre-testing
>> tree shortly.
>>
>> The interesting stuff is some enhancements to kbuild to let us
>> build modules from multiple directories.
>> I've used that to clean up the build of ptlrpc+ldlm and to
>> merge libcfs and lnet into the one module - I've left the lnds
>> separate.
>
> That looks pretty good. It doesn't apply to the stable lustre tree but I
> did try it out on lustre-testing. The module building does seem to
> function well. Only thing not clear is if do a
>
> make SUBDIRS=drivers/staging/lustre/lustre/lmv modules
>
> do lmv.a and the other *.a files get relinked int lustre.ko ?
>
>> I've haven't progressed further in lustre because I'm not sure which -
>> if any - modules are also used by the server.
>> I'd rather shared code was in a separate module to client-only.
>>
>> Which parts of lustre/lustre/* are also used by server?
>
> Basically all the client code is reused on the server except for llite.
> The reason is each server type MDS, MGS, or OSS is treated as a client
> to each other. So in affect the server code is a middle layer level on
> top of the client code.
>
>> After that, it is all fairly simple cleanups.
>>
>> I plan to submit the kbuild stuff upstream next week.
>>
>> Thanks,
>> NeilBrown
>>
>> ---
>>
>> NeilBrown (24):
>>       kbuild: detect directories in components of a module.
>>       kbuild: treat a directory listed in a composite object as foo/mod.a
>>       kbuild: support building of per-directory mod.a
>>       kbuild: disable KBUILD_MODNAME when building for mod.a
>>       kbuild: Add documentation for modobj-m
>>       lustre: build ldlm in the ldlm directory.
>>       lustre: merge libcfs module into lnet.
>
> This seems to work when I tried the lustre-testing tree. I did see
> regression to related to this work in that tree.
>
>>       lustre: use truncate_inode_page in place of truncate_complete_page
>>       lustre: don't use spare bits in iattr.ia_valid
>>       lustre: remove lustre_patchless_compat.h
>>       lustre: discard current_n*groups macros.
>>       lustre: discard LTIME_S macro
>>       lustre: discard NO_QUOTA and QUOTA_OK
>>       lustre: discard ext2* bit operations
>>       lustre: centralize TIMES_SET_FLAGS
>>       lustre: remove redefinition of module_init().
>>       lustre: remove lustre_compat.h
>
> The first two patches of the above seem to break things. The first one was
> pointed out so I avoid issues in my testing by changing the patch to just
> move truncate_inode_page() to lustre_mdc.h for now. The next patch does
> break things.
>
>>       lustre/lnet: move tracefile locking from linux-tracefile.c to tracefile.c
>>       lustre: open code cfs_trace_max_debug_mb() into cfs_trace_set_debug_mb()
>>       lustre: move tcd locking across to tracefile.c
>>       lustre: move remainder of linux-tracefile.c to tracefile.c
>>       lustre: fold cfs_tracefile_*_arch into their only callers.
>>       lustre: renamed CFS_TCD_TYPE_MAX to CFS_TCD_TYPE_CNT
>>       lustre: discard TCD_MAX_TYPES
>
> I see breakage in this code as well.
>
>>  Documentation/kbuild/makefiles.txt                 |   65 +++++
>>  drivers/staging/lustre/lnet/Makefile               |    7 -
>>  drivers/staging/lustre/lnet/libcfs/Makefile        |    7 -
>>  drivers/staging/lustre/lnet/libcfs/debug.c         |    6
>>  .../staging/lustre/lnet/libcfs/linux-tracefile.c   |  258 --------------------
>>  drivers/staging/lustre/lnet/libcfs/module.c        |   26 +-
>>  drivers/staging/lustre/lnet/libcfs/tracefile.c     |  241 +++++++++++++++++--
>>  drivers/staging/lustre/lnet/libcfs/tracefile.h     |   40 ---
>>  drivers/staging/lustre/lnet/lnet/Makefile          |   11 -
>>  drivers/staging/lustre/lnet/lnet/module.c          |    9 +
>>  drivers/staging/lustre/lustre/fid/lproc_fid.c      |    1
>>  drivers/staging/lustre/lustre/fld/lproc_fld.c      |    1
>>  drivers/staging/lustre/lustre/include/cl_object.h  |    6
>>  .../staging/lustre/lustre/include/lustre_compat.h  |   82 ------
>>  drivers/staging/lustre/lustre/include/lustre_net.h |    1
>>  .../lustre/include/lustre_patchless_compat.h       |   68 -----
>>  drivers/staging/lustre/lustre/include/obd.h        |    7 +
>>  .../staging/lustre/lustre/include/obd_support.h    |    1
>>  drivers/staging/lustre/lustre/ldlm/Makefile        |   11 +
>>  drivers/staging/lustre/lustre/llite/dir.c          |    3
>>  drivers/staging/lustre/lustre/llite/file.c         |   36 +--
>>  drivers/staging/lustre/lustre/llite/lcommon_cl.c   |   11 -
>>  .../staging/lustre/lustre/llite/llite_internal.h   |   13 +
>>  drivers/staging/lustre/lustre/llite/llite_lib.c    |   52 ++--
>>  drivers/staging/lustre/lustre/llite/namei.c        |   16 +
>>  drivers/staging/lustre/lustre/llite/vvp_io.c       |    8 -
>>  drivers/staging/lustre/lustre/llite/vvp_object.c   |    6
>>  drivers/staging/lustre/lustre/lmv/lmv_intent.c     |    6
>>  drivers/staging/lustre/lustre/lmv/lmv_obd.c        |   21 +-
>>  drivers/staging/lustre/lustre/lov/lov_io.c         |    3
>>  drivers/staging/lustre/lustre/lov/lov_obd.c        |    2
>>  drivers/staging/lustre/lustre/lov/lov_pack.c       |    1
>>  drivers/staging/lustre/lustre/lov/lov_page.c       |    1
>>  drivers/staging/lustre/lustre/mdc/mdc_lib.c        |   19 +
>>  drivers/staging/lustre/lustre/mdc/mdc_reint.c      |    4
>>  drivers/staging/lustre/lustre/mdc/mdc_request.c    |   12 -
>>  drivers/staging/lustre/lustre/obdclass/class_obd.c |    1
>>  drivers/staging/lustre/lustre/obdclass/llog.c      |    7 -
>>  .../lustre/lustre/obdclass/lprocfs_status.c        |    1
>>  .../staging/lustre/lustre/obdclass/obd_config.c    |    1
>>  drivers/staging/lustre/lustre/obdclass/obd_mount.c |    1
>>  drivers/staging/lustre/lustre/obdclass/obdo.c      |   10 -
>>  .../staging/lustre/lustre/obdecho/echo_client.c    |    1
>>  drivers/staging/lustre/lustre/osc/lproc_osc.c      |    1
>>  drivers/staging/lustre/lustre/osc/osc_cache.c      |    4
>>  drivers/staging/lustre/lustre/osc/osc_io.c         |   23 +-
>>  drivers/staging/lustre/lustre/osc/osc_quota.c      |    4
>>  drivers/staging/lustre/lustre/osc/osc_request.c    |    1
>>  drivers/staging/lustre/lustre/ptlrpc/Makefile      |   12 -
>>  drivers/staging/lustre/lustre/ptlrpc/import.c      |    1
>>  .../staging/lustre/lustre/ptlrpc/lproc_ptlrpc.c    |    1
>>  drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c     |    1
>>  drivers/staging/lustre/lustre/ptlrpc/sec.c         |   15 +
>>  drivers/staging/lustre/lustre/ptlrpc/sec_plain.c   |    1
>>  drivers/staging/lustre/lustre/ptlrpc/service.c     |    1
>>  scripts/Makefile.build                             |   57 +++-
>>  scripts/Makefile.lib                               |   63 +++--
>>  57 files changed, 584 insertions(+), 685 deletions(-)
>>  delete mode 100644 drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
>>  delete mode 100644 drivers/staging/lustre/lustre/include/lustre_compat.h
>>  delete mode 100644 drivers/staging/lustre/lustre/include/lustre_patchless_compat.h
>>  create mode 100644 drivers/staging/lustre/lustre/ldlm/Makefile
>>
>> --
>> Signature
>>
>>
> _______________________________________________
> lustre-devel mailing list
> lustre-devel at lists.lustre.org
> http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180621/75c72d16/attachment.html>

From neilb at suse.com  Thu Jun 21 04:19:43 2018
From: neilb at suse.com (NeilBrown)
Date: Thu, 21 Jun 2018 14:19:43 +1000
Subject: [lustre-devel] [PATCH 09/24] lustre: don't use spare bits in
 iattr.ia_valid
In-Reply-To: <alpine.LFD.2.21.1806210233170.3886@casper.infradead.org>
References: <152904663333.10587.10934053155404014785.stgit@noble>
 <152904669036.10587.17899565981649604892.stgit@noble>
 <alpine.LFD.2.21.1806210233170.3886@casper.infradead.org>
Message-ID: <87fu1gdb4w.fsf@notabene.neil.brown.name>

On Thu, Jun 21 2018, James Simmons wrote:

>> Using spare bits in iattr.ia_valid is not safe as the
>> bits might get used for some other purpose without
>> us noticing.
>> lustre currently used 4 spare bit, but they are all
>> defined in different places so this isn't immediately
>> obvious.
>> 
>> This patch brings all those bit definitions together
>> and defined a new op_data field (op_xvalid) to hold
>> the extra validity bits.
>> 
>> It also replaces sa_valid in struct cl_setattr_io
>> with sa_avalid and sa_xvalid.  Changing the name is
>> helpful as sa_valid already has another use within
>> lustre.
>> 
>> Signed-off-by: NeilBrown <neilb at suse.com>
>
> Nak: I see regressions with this patch.
>
> [12368.453655] Lustre: DEBUG MARKER: == sanity test 240: race between ldlm 
> enqueue and the connection RPC (no ASSERT) ===================== 21:
> 16:30 (1529543790)
> [12368.760832] BUG: Dentry 
> 000000002646a847{i=200004282000008,n=f237.sanity}  still in use (1) 
> [unmount of lustre lustre]
> [12368.773746] WARNING: CPU: 1 PID: 10861 at fs/dcache.c:1514 
...
> [12369.385564] Lustre: Unmounted lustre-client
> [12369.393247] VFS: Busy inodes after unmount of lustre. Self-destruct in 
> 5 seconds.  Have a nice day...
>
> When I remove this patch things go back to normal. This will not show up
> if you do a ONLY="240" sh ./sanity.sh. You have to run the sanity.sh in
> total to make this show up.
>

(clearly I read my email in the wrong order)
Very odd.  This suggests some sort of life-time-management problem
with inodes, but the changes shouldn't affect that at all.
I've gone back over the patch closely and cannot see anything wrong - it
is really very simple: it just moves flag bits around.

I haven't got as far as 240 yet as changes to kvmalloc have caused
earlier problem (my VMs don't have much RAM). I'll see if I can
coax it all the way to 240 and see what happens.

Thanks for the report.

NeilBrown
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180621/0a7c0f77/attachment.sig>

From neilb at suse.com  Thu Jun 21 04:28:39 2018
From: neilb at suse.com (NeilBrown)
Date: Thu, 21 Jun 2018 14:28:39 +1000
Subject: [lustre-devel] [PATCH 24/24] lustre: discard TCD_MAX_TYPES
In-Reply-To: <alpine.LFD.2.21.1806210237500.3886@casper.infradead.org>
References: <152904663333.10587.10934053155404014785.stgit@noble>
 <152904669094.10587.17019477478226952858.stgit@noble>
 <alpine.LFD.2.21.1806210237500.3886@casper.infradead.org>
Message-ID: <87d0wkdaq0.fsf@notabene.neil.brown.name>


As well as CFS_TCD_TYPE_CNT we have TCD_MAX_TYPES which has a larger
value but a similar meaning.  Discard it and just use
CFS_TCD_TYPE_CNT.

Two places relied on the fact that TCD_MAX_TYPES was larger and so
there would be NULLs at the end of the array.  Change
them to check the array size properly.

Signed-off-by: NeilBrown <neilb at suse.com>
---

Thanks for testing James!
I found two problems.
1/ I had
-       for (i = 0; cfs_trace_data[i] && i < CFS_TCD_TYPE_CNT; i++) {
instead of
+       for (i = 0; i < CFS_TCD_TYPE_CNT && cfs_trace_data[i]; i++) {

So it could dereference beyond the end of an array.  I don't this was
the problem.

2/ I hadn't changed
-       for (i = 0; cfs_trace_data[i]; i++)                             \
to
+       for (i = 0; i < CFS_TCD_TYPE_CNT && cfs_trace_data[i]; i++)                             \

so if cfs_trace_data[4] was non NULL, bad things could happen.
I suspect this is what happened to you.

Thanks,
NeilBrown


 drivers/staging/lustre/lnet/libcfs/tracefile.c | 8 ++++----
 drivers/staging/lustre/lnet/libcfs/tracefile.h | 5 ++---
 2 files changed, 6 insertions(+), 7 deletions(-)

diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.c b/drivers/staging/lustre/lnet/libcfs/tracefile.c
index cdef67391a72..cc399d580444 100644
--- a/drivers/staging/lustre/lnet/libcfs/tracefile.c
+++ b/drivers/staging/lustre/lnet/libcfs/tracefile.c
@@ -50,7 +50,7 @@
 #include "tracefile.h"
 
 /* XXX move things up to the top, comment */
-union cfs_trace_data_union (*cfs_trace_data[TCD_MAX_TYPES])[NR_CPUS] __cacheline_aligned;
+union cfs_trace_data_union (*cfs_trace_data[CFS_TCD_TYPE_CNT])[NR_CPUS] __cacheline_aligned;
 
 char cfs_tracefile[TRACEFILE_NAME_SIZE];
 long long cfs_tracefile_size = CFS_TRACEFILE_SIZE;
@@ -145,8 +145,8 @@ void cfs_trace_unlock_tcd(struct cfs_trace_cpu_data *tcd, int walking)
 		spin_unlock(&tcd->tcd_lock);
 }
 
-#define cfs_tcd_for_each_type_lock(tcd, i, cpu)			   \
-	for (i = 0; cfs_trace_data[i] &&				\
+#define cfs_tcd_for_each_type_lock(tcd, i, cpu)				\
+	for (i = 0; i < CFS_TCD_TYPE_CNT && cfs_trace_data[i] &&	\
 	     (tcd = &(*cfs_trace_data[i])[cpu].tcd) &&			\
 	     cfs_trace_lock_tcd(tcd, 1); cfs_trace_unlock_tcd(tcd, 1), i++)
 
@@ -1381,7 +1381,7 @@ static void cfs_trace_cleanup(void)
 			cfs_trace_console_buffers[i][j] = NULL;
 		}
 
-	for (i = 0; cfs_trace_data[i]; i++) {
+	for (i = 0; i < CFS_TCD_TYPE_CNT && cfs_trace_data[i]; i++) {
 		kfree(cfs_trace_data[i]);
 		cfs_trace_data[i] = NULL;
 	}
diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.h b/drivers/staging/lustre/lnet/libcfs/tracefile.h
index 23faecf886c1..87b00fc70b70 100644
--- a/drivers/staging/lustre/lnet/libcfs/tracefile.h
+++ b/drivers/staging/lustre/lnet/libcfs/tracefile.h
@@ -184,11 +184,10 @@ union cfs_trace_data_union {
 	char __pad[L1_CACHE_ALIGN(sizeof(struct cfs_trace_cpu_data))];
 };
 
-#define TCD_MAX_TYPES      8
-extern union cfs_trace_data_union (*cfs_trace_data[TCD_MAX_TYPES])[NR_CPUS];
+extern union cfs_trace_data_union (*cfs_trace_data[CFS_TCD_TYPE_CNT])[NR_CPUS];
 
 #define cfs_tcd_for_each(tcd, i, j)				       \
-	for (i = 0; cfs_trace_data[i]; i++)				\
+	for (i = 0; i < CFS_TCD_TYPE_CNT && cfs_trace_data[i]; i++)				\
 		for (j = 0, ((tcd) = &(*cfs_trace_data[i])[j].tcd);	\
 		     j < num_possible_cpus();				 \
 		     j++, (tcd) = &(*cfs_trace_data[i])[j].tcd)
-- 
2.14.0.rc0.dirty

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180621/7a937256/attachment.sig>

From adilger at dilger.ca  Thu Jun 21 07:22:45 2018
From: adilger at dilger.ca (Andreas Dilger)
Date: Thu, 21 Jun 2018 01:22:45 -0600
Subject: [lustre-devel] [PATCH 00/24] lustre - more cleanups including
 module reduction.
In-Reply-To: <alpine.LFD.2.21.1806210354260.3886@casper.infradead.org>
References: <152904663333.10587.10934053155404014785.stgit@noble>
 <alpine.LFD.2.21.1806210239190.3886@casper.infradead.org>
 <DM3PR1101MB1213DDB7FC857EE6B1444153CB760@DM3PR1101MB1213.namprd11.prod.outlook.com>
 <alpine.LFD.2.21.1806210354260.3886@casper.infradead.org>
Message-ID: <3FD4D051-9C95-449D-8A23-D42B271E55B8@dilger.ca>

On Jun 20, 2018, at 8:57 PM, James Simmons <jsimmons at infradead.org> wrote:
> 
>> 
>> Huh?  Very little of the client is used on the server.  The vast majority of the client is llite, vvp, then lov, osc, and mdc.  These are
>> not used on the servers at all, the modules won't even be loaded.
>> 
>> 
>> Servers are definitely *not* clients of one another in the sense of "Lustre clients".  They interact with each other, but in a manner very
>> different from client to server.
>> 
>> The ldlm and ptlrpc layers are *partly* shared, and of course, the networking from lnet down is.  But that's not "the client".
>> 
>> The statement that the server layer is a middle layer on top of the client code doesn't make any sense to me.  It's like this:
>> 
>> 
>> vfs
>> client code
>> ptlrpc
>> networking
>> ------ PHYSICAL BOUNDARY --------
>> networking
>> ptlrpc
>> server code
>> ldiskfs
>> 
>> 
>> 
>> with LDLM also being explicitly shared, though client & server code is not all shared.  Very little to nothing outside of the networking &
>> ldlm layers is shared.
> 
> When I first started this (pre 2.3 days) that is how it was explained to
> me. I just took it at face value but when it comes to the separation as I
> don't have a clue. Patrick since you have a better grasp of the
> architecture can you provide details to Neil. Perhaps to documentation on
> this separation if it exist.

James,
you are (or were) correct, but the code has evolved over the years.  The lov/osc and
modules were formerly used on the MDS in order to connect to the OSS, and lmv/mdc were
used with the old CMD code to connect to other MDS nodes.  However, that ended up with
a lot of code that was in those modules that was only used on the client or server.  In
the CMD code it also meant that a lot of complexity existed with making "fake" RPCs to
other MDS nodes that tried to keep the same RPC opcode but were interpreted differently
at the MDS when they were being sent from another MDS.

In the 2.4 release (which included both DNE and the OSD rewrite for ZFS) the lov/osc
server code was copied/split into lod/osp, and this is used for remote connections to
both the OSSs and other MDS nodes.  The MDS is still a client of the OSS for precreate
and recovery operations (and soon also statfs), but the code is no longer shared with
the regular clients.

Cheers, Andreas





-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 873 bytes
Desc: Message signed with OpenPGP
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180621/d24975d1/attachment.sig>

From paf at cray.com  Thu Jun 21 12:28:11 2018
From: paf at cray.com (Patrick Farrell)
Date: Thu, 21 Jun 2018 12:28:11 +0000
Subject: [lustre-devel] [PATCH 00/24] lustre - more cleanups including
 module reduction.
In-Reply-To: <3FD4D051-9C95-449D-8A23-D42B271E55B8@dilger.ca>
References: <152904663333.10587.10934053155404014785.stgit@noble>
 <alpine.LFD.2.21.1806210239190.3886@casper.infradead.org>
 <DM3PR1101MB1213DDB7FC857EE6B1444153CB760@DM3PR1101MB1213.namprd11.prod.outlook.com>
 <alpine.LFD.2.21.1806210354260.3886@casper.infradead.org>,
 <3FD4D051-9C95-449D-8A23-D42B271E55B8@dilger.ca>
Message-ID: <DM3PR1101MB121322E85FDE1E5310F6F4B7CB760@DM3PR1101MB1213.namprd11.prod.outlook.com>

Ah, thanks Andreas.  Perhaps not coincidentally, Lustre 2.4 is the first release I worked on.

Neil, I am really unenamored of the idea of the shared ptlrpc ldlm module being named ptlrpc...

Also, Lustre currently has a bunch of module parameters which are used for configuration.  Thoughts on that?


________________________________
From: Andreas Dilger <adilger at dilger.ca>
Sent: Thursday, June 21, 2018 2:22:45 AM
To: James Simmons
Cc: Patrick Farrell; Lustre Development List
Subject: Re: [lustre-devel] [PATCH 00/24] lustre - more cleanups including module reduction.

On Jun 20, 2018, at 8:57 PM, James Simmons <jsimmons at infradead.org> wrote:
>
>>
>> Huh?  Very little of the client is used on the server.  The vast majority of the client is llite, vvp, then lov, osc, and mdc.  These are
>> not used on the servers at all, the modules won't even be loaded.
>>
>>
>> Servers are definitely *not* clients of one another in the sense of "Lustre clients".  They interact with each other, but in a manner very
>> different from client to server.
>>
>> The ldlm and ptlrpc layers are *partly* shared, and of course, the networking from lnet down is.  But that's not "the client".
>>
>> The statement that the server layer is a middle layer on top of the client code doesn't make any sense to me.  It's like this:
>>
>>
>> vfs
>> client code
>> ptlrpc
>> networking
>> ------ PHYSICAL BOUNDARY --------
>> networking
>> ptlrpc
>> server code
>> ldiskfs
>>
>>
>>
>> with LDLM also being explicitly shared, though client & server code is not all shared.  Very little to nothing outside of the networking &
>> ldlm layers is shared.
>
> When I first started this (pre 2.3 days) that is how it was explained to
> me. I just took it at face value but when it comes to the separation as I
> don't have a clue. Patrick since you have a better grasp of the
> architecture can you provide details to Neil. Perhaps to documentation on
> this separation if it exist.

James,
you are (or were) correct, but the code has evolved over the years.  The lov/osc and
modules were formerly used on the MDS in order to connect to the OSS, and lmv/mdc were
used with the old CMD code to connect to other MDS nodes.  However, that ended up with
a lot of code that was in those modules that was only used on the client or server.  In
the CMD code it also meant that a lot of complexity existed with making "fake" RPCs to
other MDS nodes that tried to keep the same RPC opcode but were interpreted differently
at the MDS when they were being sent from another MDS.

In the 2.4 release (which included both DNE and the OSD rewrite for ZFS) the lov/osc
server code was copied/split into lod/osp, and this is used for remote connections to
both the OSSs and other MDS nodes.  The MDS is still a client of the OSS for precreate
and recovery operations (and soon also statfs), but the code is no longer shared with
the regular clients.

Cheers, Andreas





-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180621/79f84741/attachment.html>

From neilb at suse.com  Fri Jun 22 02:23:13 2018
From: neilb at suse.com (NeilBrown)
Date: Fri, 22 Jun 2018 12:23:13 +1000
Subject: [lustre-devel] [PATCH 09/24] lustre: don't use spare bits in
 iattr.ia_valid
In-Reply-To: <87fu1gdb4w.fsf@notabene.neil.brown.name>
References: <152904663333.10587.10934053155404014785.stgit@noble>
 <152904669036.10587.17899565981649604892.stgit@noble>
 <alpine.LFD.2.21.1806210233170.3886@casper.infradead.org>
 <87fu1gdb4w.fsf@notabene.neil.brown.name>
Message-ID: <871sczd0fi.fsf@notabene.neil.brown.name>

On Thu, Jun 21 2018, NeilBrown wrote:

> On Thu, Jun 21 2018, James Simmons wrote:
>
>>> Using spare bits in iattr.ia_valid is not safe as the
>>> bits might get used for some other purpose without
>>> us noticing.
>>> lustre currently used 4 spare bit, but they are all
>>> defined in different places so this isn't immediately
>>> obvious.
>>> 
>>> This patch brings all those bit definitions together
>>> and defined a new op_data field (op_xvalid) to hold
>>> the extra validity bits.
>>> 
>>> It also replaces sa_valid in struct cl_setattr_io
>>> with sa_avalid and sa_xvalid.  Changing the name is
>>> helpful as sa_valid already has another use within
>>> lustre.
>>> 
>>> Signed-off-by: NeilBrown <neilb at suse.com>
>>
>> Nak: I see regressions with this patch.
>>
>> [12368.453655] Lustre: DEBUG MARKER: == sanity test 240: race between ldlm 
>> enqueue and the connection RPC (no ASSERT) ===================== 21:
>> 16:30 (1529543790)
>> [12368.760832] BUG: Dentry 
>> 000000002646a847{i=200004282000008,n=f237.sanity}  still in use (1) 
>> [unmount of lustre lustre]
>> [12368.773746] WARNING: CPU: 1 PID: 10861 at fs/dcache.c:1514 
> ...
>> [12369.385564] Lustre: Unmounted lustre-client
>> [12369.393247] VFS: Busy inodes after unmount of lustre. Self-destruct in 
>> 5 seconds.  Have a nice day...
>>
>> When I remove this patch things go back to normal. This will not show up
>> if you do a ONLY="240" sh ./sanity.sh. You have to run the sanity.sh in
>> total to make this show up.
>>
>
> (clearly I read my email in the wrong order)
> Very odd.  This suggests some sort of life-time-management problem
> with inodes, but the changes shouldn't affect that at all.
> I've gone back over the patch closely and cannot see anything wrong - it
> is really very simple: it just moves flag bits around.
>
> I haven't got as far as 240 yet as changes to kvmalloc have caused
> earlier problem (my VMs don't have much RAM). I'll see if I can
> coax it all the way to 240 and see what happens.

I've now tested all the way to the end and see no new failures.
Specifically test 200 passes.

Have you had it fail more than once?  If it was only once, then maybe it
was some random occurrence not related to the patch.  We clearly still
want to fix it if we can...

Thanks,
NeilBrown
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180622/9e82110f/attachment.sig>

From neilb at suse.com  Fri Jun 22 03:19:07 2018
From: neilb at suse.com (NeilBrown)
Date: Fri, 22 Jun 2018 13:19:07 +1000
Subject: [lustre-devel] [PATCH 21/24] lustre: move remainder of
 linux-tracefile.c to tracefile.c
In-Reply-To: <alpine.LFD.2.21.1806210236200.3886@casper.infradead.org>
References: <152904663333.10587.10934053155404014785.stgit@noble>
 <152904669084.10587.6920959206498193596.stgit@noble>
 <alpine.LFD.2.21.1806210236200.3886@casper.infradead.org>
Message-ID: <87y3f7bj9w.fsf@notabene.neil.brown.name>

On Thu, Jun 21 2018, James Simmons wrote:

>> It's good to keep related code together.
>> 
>> Signed-off-by: NeilBrown <neilb at suse.com>
>
> Nak. For some reason this corrupts my dmesg output. Its really strange and
> I haven't figured it out yet.

This patch cannot possibly be the cause of any such behavioural change.
It purely moves code from one file to another, it doesn't change the
code at all.

Thanks,
NeilBrown
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180622/aaa02fe3/attachment.sig>

From jsimmons at infradead.org  Sun Jun 24 20:29:05 2018
From: jsimmons at infradead.org (James Simmons)
Date: Sun, 24 Jun 2018 21:29:05 +0100 (BST)
Subject: [lustre-devel] [PATCH 21/24] lustre: move remainder of
 linux-tracefile.c to tracefile.c
In-Reply-To: <87y3f7bj9w.fsf@notabene.neil.brown.name>
References: <152904663333.10587.10934053155404014785.stgit@noble>
 <152904669084.10587.6920959206498193596.stgit@noble>
 <alpine.LFD.2.21.1806210236200.3886@casper.infradead.org>
 <87y3f7bj9w.fsf@notabene.neil.brown.name>
Message-ID: <alpine.LFD.2.21.1806242125460.28698@casper.infradead.org>


> On Thu, Jun 21 2018, James Simmons wrote:
> 
> >> It's good to keep related code together.
> >> 
> >> Signed-off-by: NeilBrown <neilb at suse.com>
> >
> > Nak. For some reason this corrupts my dmesg output. Its really strange and
> > I haven't figured it out yet.
> 
> This patch cannot possibly be the cause of any such behavioural change.
> It purely moves code from one file to another, it doesn't change the
> code at all.

In a way you are correct. What this patch does is expose how badly messed
up cfs_print_to_console() is. Originally it was a printk() call but some
newbie changed it incorrectly into a pr_info() for all cases. This is 
totally incorrect but it landed anyways :-( For some bizarre reason it
works even with it being completely wrong. I did fix this patch up
and redid the later patches due the changes from the fix up. Is it okay
it I push the newer version of these patches with my changes?

From jsimmons at infradead.org  Sun Jun 24 20:33:52 2018
From: jsimmons at infradead.org (James Simmons)
Date: Sun, 24 Jun 2018 21:33:52 +0100 (BST)
Subject: [lustre-devel] [PATCH 09/24] lustre: don't use spare bits in
 iattr.ia_valid
In-Reply-To: <871sczd0fi.fsf@notabene.neil.brown.name>
References: <152904663333.10587.10934053155404014785.stgit@noble>
 <152904669036.10587.17899565981649604892.stgit@noble>
 <alpine.LFD.2.21.1806210233170.3886@casper.infradead.org>
 <87fu1gdb4w.fsf@notabene.neil.brown.name>
 <871sczd0fi.fsf@notabene.neil.brown.name>
Message-ID: <alpine.LFD.2.21.1806242129180.28698@casper.infradead.org>


> On Thu, Jun 21 2018, NeilBrown wrote:
> 
> > On Thu, Jun 21 2018, James Simmons wrote:
> >
> >>> Using spare bits in iattr.ia_valid is not safe as the
> >>> bits might get used for some other purpose without
> >>> us noticing.
> >>> lustre currently used 4 spare bit, but they are all
> >>> defined in different places so this isn't immediately
> >>> obvious.
> >>> 
> >>> This patch brings all those bit definitions together
> >>> and defined a new op_data field (op_xvalid) to hold
> >>> the extra validity bits.
> >>> 
> >>> It also replaces sa_valid in struct cl_setattr_io
> >>> with sa_avalid and sa_xvalid.  Changing the name is
> >>> helpful as sa_valid already has another use within
> >>> lustre.
> >>> 
> >>> Signed-off-by: NeilBrown <neilb at suse.com>
> >>
> >> Nak: I see regressions with this patch.
> >>
> >> [12368.453655] Lustre: DEBUG MARKER: == sanity test 240: race between ldlm 
> >> enqueue and the connection RPC (no ASSERT) ===================== 21:
> >> 16:30 (1529543790)
> >> [12368.760832] BUG: Dentry 
> >> 000000002646a847{i=200004282000008,n=f237.sanity}  still in use (1) 
> >> [unmount of lustre lustre]
> >> [12368.773746] WARNING: CPU: 1 PID: 10861 at fs/dcache.c:1514 
> > ...
> >> [12369.385564] Lustre: Unmounted lustre-client
> >> [12369.393247] VFS: Busy inodes after unmount of lustre. Self-destruct in 
> >> 5 seconds.  Have a nice day...
> >>
> >> When I remove this patch things go back to normal. This will not show up
> >> if you do a ONLY="240" sh ./sanity.sh. You have to run the sanity.sh in
> >> total to make this show up.
> >>
> >
> > (clearly I read my email in the wrong order)
> > Very odd.  This suggests some sort of life-time-management problem
> > with inodes, but the changes shouldn't affect that at all.
> > I've gone back over the patch closely and cannot see anything wrong - it
> > is really very simple: it just moves flag bits around.
> >
> > I haven't got as far as 240 yet as changes to kvmalloc have caused
> > earlier problem (my VMs don't have much RAM). I'll see if I can
> > coax it all the way to 240 and see what happens.
> 
> I've now tested all the way to the end and see no new failures.
> Specifically test 200 passes.
> 
> Have you had it fail more than once?  If it was only once, then maybe it
> was some random occurrence not related to the patch.  We clearly still
> want to fix it if we can...

Okay I did some more testing and found I get this error with or without
this patch. Its due to a regression from an earlier patch. Can't figure
out where its coming from. I looked at the d_lock usage and it seems
balanced. So all but the first patch for the lustre_compact* header
changes seem okay. BTW they need a rebasing. Also since I have separated
out your libcfs tracefile fixes which I will push your okay with it so
it might be best to break the patches into two series.

From jsimmons at infradead.org  Sun Jun 24 20:35:33 2018
From: jsimmons at infradead.org (James Simmons)
Date: Sun, 24 Jun 2018 21:35:33 +0100 (BST)
Subject: [lustre-devel] [PATCH 00/24] lustre - more cleanups including
 module reduction.
In-Reply-To: <DM3PR1101MB121322E85FDE1E5310F6F4B7CB760@DM3PR1101MB1213.namprd11.prod.outlook.com>
References: <152904663333.10587.10934053155404014785.stgit@noble>
 <alpine.LFD.2.21.1806210239190.3886@casper.infradead.org>
 <DM3PR1101MB1213DDB7FC857EE6B1444153CB760@DM3PR1101MB1213.namprd11.prod.outlook.com>
 <alpine.LFD.2.21.1806210354260.3886@casper.infradead.org>,
 <3FD4D051-9C95-449D-8A23-D42B271E55B8@dilger.ca>
 <DM3PR1101MB121322E85FDE1E5310F6F4B7CB760@DM3PR1101MB1213.namprd11.prod.outlook.com>
Message-ID: <alpine.LFD.2.21.1806242134170.28698@casper.infradead.org>


> Ah, thanks Andreas.  Perhaps not coincidentally, Lustre 2.4 is the first release I worked on.
> 
> Neil, I am really unenamored of the idea of the shared ptlrpc ldlm module being named ptlrpc...
> 
> Also, Lustre currently has a bunch of module parameters which are used for configuration.  Thoughts on that?

Currently ldlm is built into the ptlrpc module for the OpenSFS branch. 
Are you suggesting a different approach?

From jsimmons at infradead.org  Sun Jun 24 20:37:10 2018
From: jsimmons at infradead.org (James Simmons)
Date: Sun, 24 Jun 2018 21:37:10 +0100 (BST)
Subject: [lustre-devel] [PATCH 24/24] lustre: discard TCD_MAX_TYPES
In-Reply-To: <87d0wkdaq0.fsf@notabene.neil.brown.name>
References: <152904663333.10587.10934053155404014785.stgit@noble>
 <152904669094.10587.17019477478226952858.stgit@noble>
 <alpine.LFD.2.21.1806210237500.3886@casper.infradead.org>
 <87d0wkdaq0.fsf@notabene.neil.brown.name>
Message-ID: <alpine.LFD.2.21.1806242136380.28698@casper.infradead.org>


> As well as CFS_TCD_TYPE_CNT we have TCD_MAX_TYPES which has a larger
> value but a similar meaning.  Discard it and just use
> CFS_TCD_TYPE_CNT.
> 
> Two places relied on the fact that TCD_MAX_TYPES was larger and so
> there would be NULLs at the end of the array.  Change
> them to check the array size properly.
> 
> Signed-off-by: NeilBrown <neilb at suse.com>
> ---
> 
> Thanks for testing James!
> I found two problems.
> 1/ I had
> -       for (i = 0; cfs_trace_data[i] && i < CFS_TCD_TYPE_CNT; i++) {
> instead of
> +       for (i = 0; i < CFS_TCD_TYPE_CNT && cfs_trace_data[i]; i++) {
> 
> So it could dereference beyond the end of an array.  I don't this was
> the problem.
> 
> 2/ I hadn't changed
> -       for (i = 0; cfs_trace_data[i]; i++)                             \
> to
> +       for (i = 0; i < CFS_TCD_TYPE_CNT && cfs_trace_data[i]; i++)                             \
> 
> so if cfs_trace_data[4] was non NULL, bad things could happen.
> I suspect this is what happened to you.
> 
> Thanks,
> NeilBrown

Much better :-)

>  drivers/staging/lustre/lnet/libcfs/tracefile.c | 8 ++++----
>  drivers/staging/lustre/lnet/libcfs/tracefile.h | 5 ++---
>  2 files changed, 6 insertions(+), 7 deletions(-)
> 
> diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.c b/drivers/staging/lustre/lnet/libcfs/tracefile.c
> index cdef67391a72..cc399d580444 100644
> --- a/drivers/staging/lustre/lnet/libcfs/tracefile.c
> +++ b/drivers/staging/lustre/lnet/libcfs/tracefile.c
> @@ -50,7 +50,7 @@
>  #include "tracefile.h"
>  
>  /* XXX move things up to the top, comment */
> -union cfs_trace_data_union (*cfs_trace_data[TCD_MAX_TYPES])[NR_CPUS] __cacheline_aligned;
> +union cfs_trace_data_union (*cfs_trace_data[CFS_TCD_TYPE_CNT])[NR_CPUS] __cacheline_aligned;
>  
>  char cfs_tracefile[TRACEFILE_NAME_SIZE];
>  long long cfs_tracefile_size = CFS_TRACEFILE_SIZE;
> @@ -145,8 +145,8 @@ void cfs_trace_unlock_tcd(struct cfs_trace_cpu_data *tcd, int walking)
>  		spin_unlock(&tcd->tcd_lock);
>  }
>  
> -#define cfs_tcd_for_each_type_lock(tcd, i, cpu)			   \
> -	for (i = 0; cfs_trace_data[i] &&				\
> +#define cfs_tcd_for_each_type_lock(tcd, i, cpu)				\
> +	for (i = 0; i < CFS_TCD_TYPE_CNT && cfs_trace_data[i] &&	\
>  	     (tcd = &(*cfs_trace_data[i])[cpu].tcd) &&			\
>  	     cfs_trace_lock_tcd(tcd, 1); cfs_trace_unlock_tcd(tcd, 1), i++)
>  
> @@ -1381,7 +1381,7 @@ static void cfs_trace_cleanup(void)
>  			cfs_trace_console_buffers[i][j] = NULL;
>  		}
>  
> -	for (i = 0; cfs_trace_data[i]; i++) {
> +	for (i = 0; i < CFS_TCD_TYPE_CNT && cfs_trace_data[i]; i++) {
>  		kfree(cfs_trace_data[i]);
>  		cfs_trace_data[i] = NULL;
>  	}
> diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.h b/drivers/staging/lustre/lnet/libcfs/tracefile.h
> index 23faecf886c1..87b00fc70b70 100644
> --- a/drivers/staging/lustre/lnet/libcfs/tracefile.h
> +++ b/drivers/staging/lustre/lnet/libcfs/tracefile.h
> @@ -184,11 +184,10 @@ union cfs_trace_data_union {
>  	char __pad[L1_CACHE_ALIGN(sizeof(struct cfs_trace_cpu_data))];
>  };
>  
> -#define TCD_MAX_TYPES      8
> -extern union cfs_trace_data_union (*cfs_trace_data[TCD_MAX_TYPES])[NR_CPUS];
> +extern union cfs_trace_data_union (*cfs_trace_data[CFS_TCD_TYPE_CNT])[NR_CPUS];
>  
>  #define cfs_tcd_for_each(tcd, i, j)				       \
> -	for (i = 0; cfs_trace_data[i]; i++)				\
> +	for (i = 0; i < CFS_TCD_TYPE_CNT && cfs_trace_data[i]; i++)				\
>  		for (j = 0, ((tcd) = &(*cfs_trace_data[i])[j].tcd);	\
>  		     j < num_possible_cpus();				 \
>  		     j++, (tcd) = &(*cfs_trace_data[i])[j].tcd)
> -- 
> 2.14.0.rc0.dirty
> 
> 

From jsimmons at infradead.org  Sun Jun 24 20:39:49 2018
From: jsimmons at infradead.org (James Simmons)
Date: Sun, 24 Jun 2018 21:39:49 +0100 (BST)
Subject: [lustre-devel] [PATCH 00/24] lustre - more cleanups including
 module reduction.
In-Reply-To: <87o9g4des9.fsf@notabene.neil.brown.name>
References: <152904663333.10587.10934053155404014785.stgit@noble>
 <alpine.LFD.2.21.1806210239190.3886@casper.infradead.org>
 <DM3PR1101MB1213DDB7FC857EE6B1444153CB760@DM3PR1101MB1213.namprd11.prod.outlook.com>
 <87o9g4des9.fsf@notabene.neil.brown.name>
Message-ID: <alpine.LFD.2.21.1806242138170.28698@casper.infradead.org>


> So llite fid fld lmv lov mdc mgc osc and obdclass
> can all be in a "lustre" module.
>
> ldlm and ptlrpc should be in a separate "ptlrpc" module.
> And lnet and the klnds are also separate.
> 
> On the server side we would have
>   lod mdd mdt mgs ofd osp ost
> 
> does that seem approximately right?

The best way to see what modules contain both server and client code is to
search for HAVE_SERVER_SUPPROT in the OpenSFS source tree.

From jsimmons at infradead.org  Sun Jun 24 20:46:22 2018
From: jsimmons at infradead.org (James Simmons)
Date: Sun, 24 Jun 2018 21:46:22 +0100 (BST)
Subject: [lustre-devel] [PATCH 00/24] lustre - more cleanups including
 module reduction.
In-Reply-To: <87r2l0df25.fsf@notabene.neil.brown.name>
References: <152904663333.10587.10934053155404014785.stgit@noble>
 <alpine.LFD.2.21.1806210239190.3886@casper.infradead.org>
 <87r2l0df25.fsf@notabene.neil.brown.name>
Message-ID: <alpine.LFD.2.21.1806242140210.28698@casper.infradead.org>


> >> Here are a bunch more cleanups that will appear in my lustre-testing
> >> tree shortly.
> >> 
> >> The interesting stuff is some enhancements to kbuild to let us
> >> build modules from multiple directories.
> >> I've used that to clean up the build of ptlrpc+ldlm and to
> >> merge libcfs and lnet into the one module - I've left the lnds
> >> separate.
> >
> > That looks pretty good. It doesn't apply to the stable lustre tree but I
> > did try it out on lustre-testing. The module building does seem to 
> > function well. Only thing not clear is if do a
> >
> > make SUBDIRS=drivers/staging/lustre/lustre/lmv modules
> 
> Why would you do that?  SUBDIRS= is for building external modules.
> What are you trying to achieve?
> 
> >
> > do lmv.a and the other *.a files get relinked int lustre.ko ?
> 
> It isn't "lmv.a", it is ".../lustre/lmv/mod.a".

Sorry I forgot the mod.a naming. I was asking to get a feel of how the
work flow would change. It is common for lustre developers to build
in tree and test without installing anything. The many module approach
might of been a way for people to rebuild a module, unload module and
then reload module quickly to test a change. I don't work this way so
I can't say. I was looking to see if that work flow could still be
done.

From jsimmons at infradead.org  Sun Jun 24 21:20:24 2018
From: jsimmons at infradead.org (James Simmons)
Date: Sun, 24 Jun 2018 17:20:24 -0400
Subject: [lustre-devel] [PATCH v3 00/26] staging: lustre: libcfs: SMP rework
Message-ID: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>

Recently lustre support has been expanded to extreme machines with as
many as a 1000+ cores. On the other end lustre also has been ported
to platforms like ARM and KNL which have uniquie NUMA and core setup.
For example some devices exist that have NUMA nodes with no cores.
With these new platforms the limitations of the Lustre's SMP code
came to light so a lot of work was needed. This resulted in this
patch set which has been tested on these platforms.

This is the 3rd version of this patch set with the first two submitted
to the staging list. This latest patchset is identical to the 2nd one
expect that the UMP support has been moved to the last patches in this
collection. The approach to support UMP also has changed with using
static initialization to greatly simplify the code.

Amir Shehata (8):
  staging: lustre: libcfs: replace MAX_NUMNODES with nr_node_ids
  staging: lustre: libcfs: remove excess space
  staging: lustre: libcfs: replace num_possible_cpus() with nr_cpu_ids
  staging: lustre: libcfs: NUMA support
  staging: lustre: libcfs: add cpu distance handling
  staging: lustre: libcfs: use distance in cpu and node handling
  staging: lustre: libcfs: provide debugfs files for distance handling
  staging: lustre: libcfs: invert error handling for cfs_cpt_table_print

Dmitry Eremin (14):
  staging: lustre: libcfs: remove useless CPU partition code
  staging: lustre: libcfs: rename variable i to cpu
  staging: lustre: libcfs: fix libcfs_cpu coding style
  staging: lustre: libcfs: use int type for CPT identification.
  staging: lustre: libcfs: rename i to node for cfs_cpt_set_nodemask
  staging: lustre: libcfs: rename i to cpu for cfs_cpt_bind
  staging: lustre: libcfs: rename cpumask_var_t variables to *_mask
  staging: lustre: libcfs: update debug messages
  staging: lustre: libcfs: make tolerant to offline CPUs and empty NUMA nodes
  staging: lustre: libcfs: report NUMA node instead of just node
  staging: lustre: libcfs: update debug messages in CPT code
  staging: lustre: libcfs: rework CPU pattern parsing code
  staging: lustre: libcfs: change CPT estimate algorithm
  staging: lustre: ptlrpc: use current CPU instead of hardcoded 0

James Simmons (4):
  staging: lustre: libcfs: properly handle failure cases in SMP code
  staging: lustre: libcfs: restore debugfs table reporting for UMP
  staging: lustre: libcfs: make cfs_cpt_tab a static structure
  staging: lustre: libcfs: restore UMP support

 .../lustre/include/linux/libcfs/libcfs_cpu.h       |  203 ++--
 drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c    | 1020 +++++++++++---------
 drivers/staging/lustre/lnet/libcfs/module.c        |   52 +-
 drivers/staging/lustre/lnet/lnet/api-ni.c          |    4 +-
 drivers/staging/lustre/lnet/lnet/lib-msg.c         |    2 +
 drivers/staging/lustre/lnet/selftest/framework.c   |    2 +-
 drivers/staging/lustre/lustre/ptlrpc/client.c      |    4 +-
 drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c     |   10 +-
 drivers/staging/lustre/lustre/ptlrpc/service.c     |   15 +-
 9 files changed, 750 insertions(+), 562 deletions(-)

-- 
1.8.3.1


From jsimmons at infradead.org  Sun Jun 24 21:20:27 2018
From: jsimmons at infradead.org (James Simmons)
Date: Sun, 24 Jun 2018 17:20:27 -0400
Subject: [lustre-devel] [PATCH v3 03/26] staging: lustre: libcfs: properly
 handle failure cases in SMP code
In-Reply-To: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
References: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1529875250-11531-4-git-send-email-jsimmons@infradead.org>

While pushing the SMP work some bugs were pointed out by Dan
Carpenter in the code. Due to single err label in cfs_cpu_init()
and cfs_cpt_table_alloc() a few items were being cleaned up that
were never initialized. This can lead to crashed and other problems.
In those initialization function introduce individual labels to
jump to only the thing initialized get freed on failure.

Signed-off-by: James Simmons <uja.ornl at yahoo.com>
WC-bug-id: https://jira.whamcloud.com/browse/LU-10932
Reviewed-on: https://review.whamcloud.com/32085
Reviewed-by: Dmitry Eremin <dmitry.eremin at intel.com>
Reviewed-by: Andreas Dilger <adilger at whamcloud.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c | 72 ++++++++++++++++++-------
 1 file changed, 52 insertions(+), 20 deletions(-)

diff --git a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
index 46d3530..bdd71a3 100644
--- a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
+++ b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
@@ -85,17 +85,19 @@ struct cfs_cpt_table *
 
 	cptab->ctb_nparts = ncpt;
 
+	if (!zalloc_cpumask_var(&cptab->ctb_cpumask, GFP_NOFS))
+		goto failed_alloc_cpumask;
+
 	cptab->ctb_nodemask = kzalloc(sizeof(*cptab->ctb_nodemask),
 				      GFP_NOFS);
-	if (!zalloc_cpumask_var(&cptab->ctb_cpumask, GFP_NOFS) ||
-	    !cptab->ctb_nodemask)
-		goto failed;
+	if (!cptab->ctb_nodemask)
+		goto failed_alloc_nodemask;
 
 	cptab->ctb_cpu2cpt = kvmalloc_array(num_possible_cpus(),
 					    sizeof(cptab->ctb_cpu2cpt[0]),
 					    GFP_KERNEL);
 	if (!cptab->ctb_cpu2cpt)
-		goto failed;
+		goto failed_alloc_cpu2cpt;
 
 	memset(cptab->ctb_cpu2cpt, -1,
 	       num_possible_cpus() * sizeof(cptab->ctb_cpu2cpt[0]));
@@ -103,22 +105,41 @@ struct cfs_cpt_table *
 	cptab->ctb_parts = kvmalloc_array(ncpt, sizeof(cptab->ctb_parts[0]),
 					  GFP_KERNEL);
 	if (!cptab->ctb_parts)
-		goto failed;
+		goto failed_alloc_ctb_parts;
+
+	memset(cptab->ctb_parts, -1, ncpt * sizeof(cptab->ctb_parts[0]));
 
 	for (i = 0; i < ncpt; i++) {
 		struct cfs_cpu_partition *part = &cptab->ctb_parts[i];
 
+		if (!zalloc_cpumask_var(&part->cpt_cpumask, GFP_NOFS))
+			goto failed_setting_ctb_parts;
+
 		part->cpt_nodemask = kzalloc(sizeof(*part->cpt_nodemask),
 					     GFP_NOFS);
-		if (!zalloc_cpumask_var(&part->cpt_cpumask, GFP_NOFS) ||
-		    !part->cpt_nodemask)
-			goto failed;
+		if (!part->cpt_nodemask)
+			goto failed_setting_ctb_parts;
 	}
 
 	return cptab;
 
- failed:
-	cfs_cpt_table_free(cptab);
+failed_setting_ctb_parts:
+	while (i-- >= 0) {
+		struct cfs_cpu_partition *part = &cptab->ctb_parts[i];
+
+		kfree(part->cpt_nodemask);
+		free_cpumask_var(part->cpt_cpumask);
+	}
+
+	kvfree(cptab->ctb_parts);
+failed_alloc_ctb_parts:
+	kvfree(cptab->ctb_cpu2cpt);
+failed_alloc_cpu2cpt:
+	kfree(cptab->ctb_nodemask);
+failed_alloc_nodemask:
+	free_cpumask_var(cptab->ctb_cpumask);
+failed_alloc_cpumask:
+	kfree(cptab);
 	return NULL;
 }
 EXPORT_SYMBOL(cfs_cpt_table_alloc);
@@ -944,7 +965,7 @@ static int cfs_cpu_dead(unsigned int cpu)
 int
 cfs_cpu_init(void)
 {
-	int ret = 0;
+	int ret;
 
 	LASSERT(!cfs_cpt_tab);
 
@@ -953,23 +974,23 @@ static int cfs_cpu_dead(unsigned int cpu)
 					"staging/lustre/cfe:dead", NULL,
 					cfs_cpu_dead);
 	if (ret < 0)
-		goto failed;
+		goto failed_cpu_dead;
+
 	ret = cpuhp_setup_state_nocalls(CPUHP_AP_ONLINE_DYN,
 					"staging/lustre/cfe:online",
 					cfs_cpu_online, NULL);
 	if (ret < 0)
-		goto failed;
+		goto failed_cpu_online;
+
 	lustre_cpu_online = ret;
 #endif
-	ret = -EINVAL;
-
 	get_online_cpus();
 	if (*cpu_pattern) {
 		char *cpu_pattern_dup = kstrdup(cpu_pattern, GFP_KERNEL);
 
 		if (!cpu_pattern_dup) {
 			CERROR("Failed to duplicate cpu_pattern\n");
-			goto failed;
+			goto failed_alloc_table;
 		}
 
 		cfs_cpt_tab = cfs_cpt_table_create_pattern(cpu_pattern_dup);
@@ -977,7 +998,7 @@ static int cfs_cpu_dead(unsigned int cpu)
 		if (!cfs_cpt_tab) {
 			CERROR("Failed to create cptab from pattern %s\n",
 			       cpu_pattern);
-			goto failed;
+			goto failed_alloc_table;
 		}
 
 	} else {
@@ -985,7 +1006,7 @@ static int cfs_cpu_dead(unsigned int cpu)
 		if (!cfs_cpt_tab) {
 			CERROR("Failed to create ptable with npartitions %d\n",
 			       cpu_npartitions);
-			goto failed;
+			goto failed_alloc_table;
 		}
 	}
 
@@ -996,8 +1017,19 @@ static int cfs_cpu_dead(unsigned int cpu)
 		 cfs_cpt_number(cfs_cpt_tab));
 	return 0;
 
- failed:
+failed_alloc_table:
 	put_online_cpus();
-	cfs_cpu_fini();
+
+	if (cfs_cpt_tab)
+		cfs_cpt_table_free(cfs_cpt_tab);
+
+	ret = -EINVAL;
+#ifdef CONFIG_HOTPLUG_CPU
+	if (lustre_cpu_online > 0)
+		cpuhp_remove_state_nocalls(lustre_cpu_online);
+failed_cpu_online:
+	cpuhp_remove_state_nocalls(CPUHP_LUSTRE_CFS_DEAD);
+failed_cpu_dead:
+#endif
 	return ret;
 }
-- 
1.8.3.1


From jsimmons at infradead.org  Sun Jun 24 21:20:30 2018
From: jsimmons at infradead.org (James Simmons)
Date: Sun, 24 Jun 2018 17:20:30 -0400
Subject: [lustre-devel] [PATCH v3 06/26] staging: lustre: libcfs: replace
 num_possible_cpus() with nr_cpu_ids
In-Reply-To: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
References: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1529875250-11531-7-git-send-email-jsimmons@infradead.org>

From: Amir Shehata <amir.shehata at intel.com>

Move from num_possible_cpus() to nr_cpu_ids.

Signed-off-by: Amir Shehata <amir.shehata at intel.com>
WC-bug-id: https://jira.whamcloud.com/browse/LU-7734
Reviewed-on: http://review.whamcloud.com/18916
Reviewed-by: Olaf Weber <olaf at sgi.com>
Reviewed-by: Doug Oucharek <dougso at me.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
index 680a2b1..33294da 100644
--- a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
+++ b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
@@ -93,14 +93,14 @@ struct cfs_cpt_table *
 	if (!cptab->ctb_nodemask)
 		goto failed_alloc_nodemask;
 
-	cptab->ctb_cpu2cpt = kvmalloc_array(num_possible_cpus(),
+	cptab->ctb_cpu2cpt = kvmalloc_array(nr_cpu_ids,
 					    sizeof(cptab->ctb_cpu2cpt[0]),
 					    GFP_KERNEL);
 	if (!cptab->ctb_cpu2cpt)
 		goto failed_alloc_cpu2cpt;
 
 	memset(cptab->ctb_cpu2cpt, -1,
-	       num_possible_cpus() * sizeof(cptab->ctb_cpu2cpt[0]));
+	       nr_cpu_ids * sizeof(cptab->ctb_cpu2cpt[0]));
 
 	cptab->ctb_parts = kvmalloc_array(ncpt, sizeof(cptab->ctb_parts[0]),
 					  GFP_KERNEL);
-- 
1.8.3.1


From jsimmons at infradead.org  Sun Jun 24 21:20:29 2018
From: jsimmons at infradead.org (James Simmons)
Date: Sun, 24 Jun 2018 17:20:29 -0400
Subject: [lustre-devel] [PATCH v3 05/26] staging: lustre: libcfs: remove
 excess space
In-Reply-To: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
References: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1529875250-11531-6-git-send-email-jsimmons@infradead.org>

From: Amir Shehata <amir.shehata at intel.com>

The function cfs_cpt_table_print() was adding two spaces
to the string buffer. Just add it once.

Signed-off-by: Amir Shehata <amir.shehata at intel.com>
WC-bug-id: https://jira.whamcloud.com/browse/LU-7734
Reviewed-on: http://review.whamcloud.com/18916
Reviewed-by: Olaf Weber <olaf at sgi.com>
Reviewed-by: Doug Oucharek <dougso at me.com>
Reviewed-by: Oleg Drokin <green at whamcloud.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
index ea8d55c..680a2b1 100644
--- a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
+++ b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
@@ -177,7 +177,7 @@ struct cfs_cpt_table *
 
 	for (i = 0; i < cptab->ctb_nparts; i++) {
 		if (len > 0) {
-			rc = snprintf(tmp, len, "%d\t: ", i);
+			rc = snprintf(tmp, len, "%d\t:", i);
 			len -= rc;
 		}
 
-- 
1.8.3.1


From jsimmons at infradead.org  Sun Jun 24 21:20:33 2018
From: jsimmons at infradead.org (James Simmons)
Date: Sun, 24 Jun 2018 17:20:33 -0400
Subject: [lustre-devel] [PATCH v3 09/26] staging: lustre: libcfs: use
 distance in cpu and node handling
In-Reply-To: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
References: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1529875250-11531-10-git-send-email-jsimmons@infradead.org>

From: Amir Shehata <amir.shehata at intel.com>

Take into consideration the location of NUMA nodes and core
when calling cfs_cpt_[un]set_cpu() and cfs_cpt_[un]set_node().
This enables functioning on platforms with 100s of cores and
NUMA nodes.

Signed-off-by: Amir Shehata <amir.shehata at intel.com>
WC-bug-id: https://jira.whamcloud.com/browse/LU-7734
Reviewed-on: http://review.whamcloud.com/18916
Reviewed-by: Olaf Weber <olaf at sgi.com>
Reviewed-by: Doug Oucharek <dougso at me.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c | 192 ++++++++++++++++++------
 1 file changed, 143 insertions(+), 49 deletions(-)

diff --git a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
index b315fb2..3b4a9dc 100644
--- a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
+++ b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
@@ -334,11 +334,134 @@ unsigned int cfs_cpt_distance(struct cfs_cpt_table *cptab, int cpt1, int cpt2)
 }
 EXPORT_SYMBOL(cfs_cpt_distance);
 
+/*
+ * Calculate the maximum NUMA distance between all nodes in the
+ * from_mask and all nodes in the to_mask.
+ */
+static unsigned int cfs_cpt_distance_calculate(nodemask_t *from_mask,
+					       nodemask_t *to_mask)
+{
+	unsigned int maximum;
+	unsigned int distance;
+	int from;
+	int to;
+
+	maximum = 0;
+	for_each_node_mask(from, *from_mask) {
+		for_each_node_mask(to, *to_mask) {
+			distance = node_distance(from, to);
+			if (maximum < distance)
+				maximum = distance;
+		}
+	}
+	return maximum;
+}
+
+static void cfs_cpt_add_cpu(struct cfs_cpt_table *cptab, int cpt, int cpu)
+{
+	cptab->ctb_cpu2cpt[cpu] = cpt;
+
+	cpumask_set_cpu(cpu, cptab->ctb_cpumask);
+	cpumask_set_cpu(cpu, cptab->ctb_parts[cpt].cpt_cpumask);
+}
+
+static void cfs_cpt_del_cpu(struct cfs_cpt_table *cptab, int cpt, int cpu)
+{
+	cpumask_clear_cpu(cpu, cptab->ctb_parts[cpt].cpt_cpumask);
+	cpumask_clear_cpu(cpu, cptab->ctb_cpumask);
+
+	cptab->ctb_cpu2cpt[cpu] = -1;
+}
+
+static void cfs_cpt_add_node(struct cfs_cpt_table *cptab, int cpt, int node)
+{
+	struct cfs_cpu_partition *part;
+
+	if (!node_isset(node, *cptab->ctb_nodemask)) {
+		unsigned int dist;
+
+		/* first time node is added to the CPT table */
+		node_set(node, *cptab->ctb_nodemask);
+		cptab->ctb_node2cpt[node] = cpt;
+
+		dist = cfs_cpt_distance_calculate(cptab->ctb_nodemask,
+						  cptab->ctb_nodemask);
+		cptab->ctb_distance = dist;
+	}
+
+	part = &cptab->ctb_parts[cpt];
+	if (!node_isset(node, *part->cpt_nodemask)) {
+		int cpt2;
+
+		/* first time node is added to this CPT */
+		node_set(node, *part->cpt_nodemask);
+		for (cpt2 = 0; cpt2 < cptab->ctb_nparts; cpt2++) {
+			struct cfs_cpu_partition *part2;
+			unsigned int dist;
+
+			part2 = &cptab->ctb_parts[cpt2];
+			dist = cfs_cpt_distance_calculate(part->cpt_nodemask,
+							  part2->cpt_nodemask);
+			part->cpt_distance[cpt2] = dist;
+			dist = cfs_cpt_distance_calculate(part2->cpt_nodemask,
+							  part->cpt_nodemask);
+			part2->cpt_distance[cpt] = dist;
+		}
+	}
+}
+
+static void cfs_cpt_del_node(struct cfs_cpt_table *cptab, int cpt, int node)
+{
+	struct cfs_cpu_partition *part = &cptab->ctb_parts[cpt];
+	int cpu;
+
+	for_each_cpu(cpu, part->cpt_cpumask) {
+		/* this CPT has other CPU belonging to this node? */
+		if (cpu_to_node(cpu) == node)
+			break;
+	}
+
+	if (cpu >= nr_cpu_ids && node_isset(node,  *part->cpt_nodemask)) {
+		int cpt2;
+
+		/* No more CPUs in the node for this CPT. */
+		node_clear(node, *part->cpt_nodemask);
+		for (cpt2 = 0; cpt2 < cptab->ctb_nparts; cpt2++) {
+			struct cfs_cpu_partition *part2;
+			unsigned int dist;
+
+			part2 = &cptab->ctb_parts[cpt2];
+			if (node_isset(node, *part2->cpt_nodemask))
+				cptab->ctb_node2cpt[node] = cpt2;
+
+			dist = cfs_cpt_distance_calculate(part->cpt_nodemask,
+							  part2->cpt_nodemask);
+			part->cpt_distance[cpt2] = dist;
+			dist = cfs_cpt_distance_calculate(part2->cpt_nodemask,
+							  part->cpt_nodemask);
+			part2->cpt_distance[cpt] = dist;
+		}
+	}
+
+	for_each_cpu(cpu, cptab->ctb_cpumask) {
+		/* this CPT-table has other CPUs belonging to this node? */
+		if (cpu_to_node(cpu) == node)
+			break;
+	}
+
+	if (cpu >= nr_cpu_ids && node_isset(node, *cptab->ctb_nodemask)) {
+		/* No more CPUs in the table for this node. */
+		node_clear(node, *cptab->ctb_nodemask);
+		cptab->ctb_node2cpt[node] = -1;
+		cptab->ctb_distance =
+			cfs_cpt_distance_calculate(cptab->ctb_nodemask,
+						   cptab->ctb_nodemask);
+	}
+}
+
 int
 cfs_cpt_set_cpu(struct cfs_cpt_table *cptab, int cpt, int cpu)
 {
-	int node;
-
 	LASSERT(cpt >= 0 && cpt < cptab->ctb_nparts);
 
 	if (cpu < 0 || cpu >= nr_cpu_ids || !cpu_online(cpu)) {
@@ -352,23 +475,11 @@ unsigned int cfs_cpt_distance(struct cfs_cpt_table *cptab, int cpt1, int cpt2)
 		return 0;
 	}
 
-	cptab->ctb_cpu2cpt[cpu] = cpt;
-
 	LASSERT(!cpumask_test_cpu(cpu, cptab->ctb_cpumask));
 	LASSERT(!cpumask_test_cpu(cpu, cptab->ctb_parts[cpt].cpt_cpumask));
 
-	cpumask_set_cpu(cpu, cptab->ctb_cpumask);
-	cpumask_set_cpu(cpu, cptab->ctb_parts[cpt].cpt_cpumask);
-
-	node = cpu_to_node(cpu);
-
-	/* first CPU of @node in this CPT table */
-	if (!node_isset(node, *cptab->ctb_nodemask))
-		node_set(node, *cptab->ctb_nodemask);
-
-	/* first CPU of @node in this partition */
-	if (!node_isset(node, *cptab->ctb_parts[cpt].cpt_nodemask))
-		node_set(node, *cptab->ctb_parts[cpt].cpt_nodemask);
+	cfs_cpt_add_cpu(cptab, cpt, cpu);
+	cfs_cpt_add_node(cptab, cpt, cpu_to_node(cpu));
 
 	return 1;
 }
@@ -377,9 +488,6 @@ unsigned int cfs_cpt_distance(struct cfs_cpt_table *cptab, int cpt1, int cpt2)
 void
 cfs_cpt_unset_cpu(struct cfs_cpt_table *cptab, int cpt, int cpu)
 {
-	int node;
-	int i;
-
 	LASSERT(cpt == CFS_CPT_ANY || (cpt >= 0 && cpt < cptab->ctb_nparts));
 
 	if (cpu < 0 || cpu >= nr_cpu_ids) {
@@ -405,32 +513,8 @@ unsigned int cfs_cpt_distance(struct cfs_cpt_table *cptab, int cpt1, int cpt2)
 	LASSERT(cpumask_test_cpu(cpu, cptab->ctb_parts[cpt].cpt_cpumask));
 	LASSERT(cpumask_test_cpu(cpu, cptab->ctb_cpumask));
 
-	cpumask_clear_cpu(cpu, cptab->ctb_parts[cpt].cpt_cpumask);
-	cpumask_clear_cpu(cpu, cptab->ctb_cpumask);
-	cptab->ctb_cpu2cpt[cpu] = -1;
-
-	node = cpu_to_node(cpu);
-
-	LASSERT(node_isset(node, *cptab->ctb_parts[cpt].cpt_nodemask));
-	LASSERT(node_isset(node, *cptab->ctb_nodemask));
-
-	for_each_cpu(i, cptab->ctb_parts[cpt].cpt_cpumask) {
-		/* this CPT has other CPU belonging to this node? */
-		if (cpu_to_node(i) == node)
-			break;
-	}
-
-	if (i >= nr_cpu_ids)
-		node_clear(node, *cptab->ctb_parts[cpt].cpt_nodemask);
-
-	for_each_cpu(i, cptab->ctb_cpumask) {
-		/* this CPT-table has other CPU belonging to this node? */
-		if (cpu_to_node(i) == node)
-			break;
-	}
-
-	if (i >= nr_cpu_ids)
-		node_clear(node, *cptab->ctb_nodemask);
+	cfs_cpt_del_cpu(cptab, cpt, cpu);
+	cfs_cpt_del_node(cptab, cpt, cpu_to_node(cpu));
 }
 EXPORT_SYMBOL(cfs_cpt_unset_cpu);
 
@@ -448,8 +532,8 @@ unsigned int cfs_cpt_distance(struct cfs_cpt_table *cptab, int cpt1, int cpt2)
 	}
 
 	for_each_cpu(cpu, mask) {
-		if (!cfs_cpt_set_cpu(cptab, cpt, cpu))
-			return 0;
+		cfs_cpt_add_cpu(cptab, cpt, cpu);
+		cfs_cpt_add_node(cptab, cpt, cpu_to_node(cpu));
 	}
 
 	return 1;
@@ -471,6 +555,7 @@ unsigned int cfs_cpt_distance(struct cfs_cpt_table *cptab, int cpt1, int cpt2)
 cfs_cpt_set_node(struct cfs_cpt_table *cptab, int cpt, int node)
 {
 	const cpumask_t *mask;
+	int cpu;
 
 	if (node < 0 || node >= nr_node_ids) {
 		CDEBUG(D_INFO,
@@ -480,7 +565,12 @@ unsigned int cfs_cpt_distance(struct cfs_cpt_table *cptab, int cpt1, int cpt2)
 
 	mask = cpumask_of_node(node);
 
-	return cfs_cpt_set_cpumask(cptab, cpt, mask);
+	for_each_cpu(cpu, mask)
+		cfs_cpt_add_cpu(cptab, cpt, cpu);
+
+	cfs_cpt_add_node(cptab, cpt, node);
+
+	return 1;
 }
 EXPORT_SYMBOL(cfs_cpt_set_node);
 
@@ -488,6 +578,7 @@ unsigned int cfs_cpt_distance(struct cfs_cpt_table *cptab, int cpt1, int cpt2)
 cfs_cpt_unset_node(struct cfs_cpt_table *cptab, int cpt, int node)
 {
 	const cpumask_t *mask;
+	int cpu;
 
 	if (node < 0 || node >= nr_node_ids) {
 		CDEBUG(D_INFO,
@@ -497,7 +588,10 @@ unsigned int cfs_cpt_distance(struct cfs_cpt_table *cptab, int cpt1, int cpt2)
 
 	mask = cpumask_of_node(node);
 
-	cfs_cpt_unset_cpumask(cptab, cpt, mask);
+	for_each_cpu(cpu, mask)
+		cfs_cpt_del_cpu(cptab, cpt, cpu);
+
+	cfs_cpt_del_node(cptab, cpt, node);
 }
 EXPORT_SYMBOL(cfs_cpt_unset_node);
 
-- 
1.8.3.1


From jsimmons at infradead.org  Sun Jun 24 21:20:37 2018
From: jsimmons at infradead.org (James Simmons)
Date: Sun, 24 Jun 2018 17:20:37 -0400
Subject: [lustre-devel] [PATCH v3 13/26] staging: lustre: libcfs: use int
 type for CPT identification.
In-Reply-To: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
References: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1529875250-11531-14-git-send-email-jsimmons@infradead.org>

From: Dmitry Eremin <dmitry.eremin at intel.com>

Use int type for CPT identification to match the linux kernel
CPU identification.

Signed-off-by: Dmitry Eremin <dmitry.eremin at intel.com>
WC-bug-id: https://jira.whamcloud.com/browse/LU-8703
Reviewed-on: https://review.whamcloud.com/23304
Reviewed-by: James Simmons <uja.ornl at yahoo.com>
Reviewed-by: Doug Oucharek <dougso at me.com>
Reviewed-by: Oleg Drokin <green at whamcloud.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h |  8 ++++----
 drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c          | 14 +++++++-------
 2 files changed, 11 insertions(+), 11 deletions(-)

diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h
index 9dbb0b1..2bb2140 100644
--- a/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h
+++ b/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h
@@ -89,18 +89,18 @@ struct cfs_cpu_partition {
 	/* NUMA distance between CPTs */
 	unsigned int			*cpt_distance;
 	/* spread rotor for NUMA allocator */
-	unsigned int			cpt_spread_rotor;
+	int				cpt_spread_rotor;
 };
 
 
 /** descriptor for CPU partitions */
 struct cfs_cpt_table {
 	/* spread rotor for NUMA allocator */
-	unsigned int			ctb_spread_rotor;
+	int				ctb_spread_rotor;
 	/* maximum NUMA distance between all nodes in table */
 	unsigned int			ctb_distance;
 	/* # of CPU partitions */
-	unsigned int			ctb_nparts;
+	int				ctb_nparts;
 	/* partitions tables */
 	struct cfs_cpu_partition	*ctb_parts;
 	/* shadow HW CPU to CPU partition ID */
@@ -355,7 +355,7 @@ static inline void cfs_cpu_fini(void)
 /**
  * create a cfs_cpt_table with \a ncpt number of partitions
  */
-struct cfs_cpt_table *cfs_cpt_table_alloc(unsigned int ncpt);
+struct cfs_cpt_table *cfs_cpt_table_alloc(int ncpt);
 
 /*
  * allocate per-cpu-partition data, returned value is an array of pointers,
diff --git a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
index aaab7cb..8f7de59 100644
--- a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
+++ b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
@@ -73,7 +73,7 @@
 module_param(cpu_pattern, charp, 0444);
 MODULE_PARM_DESC(cpu_pattern, "CPU partitions pattern");
 
-struct cfs_cpt_table *cfs_cpt_table_alloc(unsigned int ncpt)
+struct cfs_cpt_table *cfs_cpt_table_alloc(int ncpt)
 {
 	struct cfs_cpt_table *cptab;
 	int i;
@@ -788,13 +788,13 @@ static int cfs_cpt_choose_ncpus(struct cfs_cpt_table *cptab, int cpt,
 	return rc;
 }
 
-#define CPT_WEIGHT_MIN  4u
+#define CPT_WEIGHT_MIN 4
 
-static unsigned int cfs_cpt_num_estimate(void)
+static int cfs_cpt_num_estimate(void)
 {
-	unsigned int nnode = num_online_nodes();
-	unsigned int ncpu = num_online_cpus();
-	unsigned int ncpt;
+	int nnode = num_online_nodes();
+	int ncpu = num_online_cpus();
+	int ncpt;
 
 	if (ncpu <= CPT_WEIGHT_MIN) {
 		ncpt = 1;
@@ -824,7 +824,7 @@ static unsigned int cfs_cpt_num_estimate(void)
 	/* config many CPU partitions on 32-bit system could consume
 	 * too much memory
 	 */
-	ncpt = min(2U, ncpt);
+	ncpt = min(2, ncpt);
 #endif
 	while (ncpu % ncpt)
 		ncpt--; /* worst case is 1 */
-- 
1.8.3.1


From jsimmons at infradead.org  Sun Jun 24 21:20:46 2018
From: jsimmons at infradead.org (James Simmons)
Date: Sun, 24 Jun 2018 17:20:46 -0400
Subject: [lustre-devel] [PATCH v3 22/26] staging: lustre: libcfs: change CPT
 estimate algorithm
In-Reply-To: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
References: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1529875250-11531-23-git-send-email-jsimmons@infradead.org>

From: Dmitry Eremin <dmitry.eremin at intel.com>

The main idea to have more CPU partitions is based on KNL experience.
When a thread submit IO for network communication one of threads from
current CPT is used for network stack. Whith high parallelization many
threads become involved in network submission but having less CPU
partitions they will wait until single thread process them from network
queue. So, the bottleneck just moves into network layer in case of
small amount of CPU partitions. My experiments showed that the best
performance was when for each IO thread we have one network thread.
This condition can be provided having 2 real HW cores (without hyper
threads) per CPT. This is exactly what implemented in this patch.

Change CPT estimate algorithm from 2 * (N - 1)^2 < NCPUS <= 2 * N^2
to 2 HW cores per CPT. This is critical for machines with number of
cores different from 2^N.

Current algorithm splits CPTs in KNL:
LNet: HW CPU cores: 272, npartitions: 16
cpu_partition_table=
    0       : 0-4,68-71,136-139,204-207
    1       : 5-9,73-76,141-144,209-212
    2       : 10-14,78-81,146-149,214-217
    3       : 15-17,72,77,83-85,140,145,151-153,208,219-221
    4       : 18-21,82,86-88,150,154-156,213,218,222-224
    5       : 22-26,90-93,158-161,226-229
    6       : 27-31,95-98,163-166,231-234
    7       : 32-35,89,100-103,168-171,236-239
    8       : 36-38,94,99,104-105,157,162,167,172-173,225,230,235,240-241
    9       : 39-43,107-110,175-178,243-246
    10      : 44-48,112-115,180-183,248-251
    11      : 49-51,106,111,117-119,174,179,185-187,242,253-255
    12      : 52-55,116,120-122,184,188-190,247,252,256-258
    13      : 56-60,124-127,192-195,260-263
    14      : 61-65,129-132,197-200,265-268
    15      : 66-67,123,128,133-135,191,196,201-203,259,264,269-271

New algorithm will split CPTs in KNL:
LNet: HW CPU cores: 272, npartitions: 34
cpu_partition_table=
    0       : 0-1,68-69,136-137,204-205
    1       : 2-3,70-71,138-139,206-207
    2       : 4-5,72-73,140-141,208-209
    3       : 6-7,74-75,142-143,210-211
    4       : 8-9,76-77,144-145,212-213
    5       : 10-11,78-79,146-147,214-215
    6       : 12-13,80-81,148-149,216-217
    7       : 14-15,82-83,150-151,218-219
    8       : 16-17,84-85,152-153,220-221
    9       : 18-19,86-87,154-155,222-223
    10      : 20-21,88-89,156-157,224-225
    11      : 22-23,90-91,158-159,226-227
    12      : 24-25,92-93,160-161,228-229
    13      : 26-27,94-95,162-163,230-231
    14      : 28-29,96-97,164-165,232-233
    15      : 30-31,98-99,166-167,234-235
    16      : 32-33,100-101,168-169,236-237
    17      : 34-35,102-103,170-171,238-239
    18      : 36-37,104-105,172-173,240-241
    19      : 38-39,106-107,174-175,242-243
    20      : 40-41,108-109,176-177,244-245
    21      : 42-43,110-111,178-179,246-247
    22      : 44-45,112-113,180-181,248-249
    23      : 46-47,114-115,182-183,250-251
    24      : 48-49,116-117,184-185,252-253
    25      : 50-51,118-119,186-187,254-255
    26      : 52-53,120-121,188-189,256-257
    27      : 54-55,122-123,190-191,258-259
    28      : 56-57,124-125,192-193,260-261
    29      : 58-59,126-127,194-195,262-263
    30      : 60-61,128-129,196-197,264-265
    31      : 62-63,130-131,198-199,266-267
    32      : 64-65,132-133,200-201,268-269
    33      : 66-67,134-135,202-203,270-271

'N' pattern in KNL works is not always good.
in flat mode it will be one CPT with all CPUs inside.

in SNC-4 mode:
cpu_partition_table=
    0       : 0-17,68-85,136-153,204-221
    1       : 18-35,86-103,154-171,222-239
    2       : 36-51,104-119,172-187,240-255
    3       : 52-67,120-135,188-203,256-271

Signed-off-by: Dmitry Eremin <dmitry.eremin at intel.com>
WC-bug-id: https://jira.whamcloud.com/browse/LU-8703
Reviewed-on: https://review.whamcloud.com/24304
Reviewed-by: James Simmons <uja.ornl at yahoo.com>
Reviewed-by: Andreas Dilger <adilger at whamcloud.com>
Reviewed-by: Oleg Drokin <green at whamcloud.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c | 30 +++++--------------------
 1 file changed, 5 insertions(+), 25 deletions(-)

diff --git a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
index 2fdea11..3f4a7c7 100644
--- a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
+++ b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
@@ -802,34 +802,14 @@ static int cfs_cpt_choose_ncpus(struct cfs_cpt_table *cptab, int cpt,
 
 static int cfs_cpt_num_estimate(void)
 {
-	int nnode = num_online_nodes();
+	int nthr = cpumask_weight(topology_sibling_cpumask(smp_processor_id()));
 	int ncpu = num_online_cpus();
-	int ncpt;
+	int ncpt = 1;
 
-	if (ncpu <= CPT_WEIGHT_MIN) {
-		ncpt = 1;
-		goto out;
-	}
-
-	/* generate reasonable number of CPU partitions based on total number
-	 * of CPUs, Preferred N should be power2 and match this condition:
-	 * 2 * (N - 1)^2 < NCPUS <= 2 * N^2
-	 */
-	for (ncpt = 2; ncpu > 2 * ncpt * ncpt; ncpt <<= 1)
-		;
-
-	if (ncpt <= nnode) { /* fat numa system */
-		while (nnode > ncpt)
-			nnode >>= 1;
+	if (ncpu > CPT_WEIGHT_MIN)
+		for (ncpt = 2; ncpu > 2 * nthr * ncpt; ncpt++)
+			; /* nothing */
 
-	} else { /* ncpt > nnode */
-		while ((nnode << 1) <= ncpt)
-			nnode <<= 1;
-	}
-
-	ncpt = nnode;
-
-out:
 #if (BITS_PER_LONG == 32)
 	/* config many CPU partitions on 32-bit system could consume
 	 * too much memory
-- 
1.8.3.1


From jsimmons at infradead.org  Sun Jun 24 21:20:31 2018
From: jsimmons at infradead.org (James Simmons)
Date: Sun, 24 Jun 2018 17:20:31 -0400
Subject: [lustre-devel] [PATCH v3 07/26] staging: lustre: libcfs: NUMA
 support
In-Reply-To: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
References: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1529875250-11531-8-git-send-email-jsimmons@infradead.org>

From: Amir Shehata <amir.shehata at intel.com>

This patch adds NUMA node support. NUMA node information is stored
in the CPT table. A NUMA node mask is maintained for the entire
table as well as for each CPT to track the NUMA nodes related to
each of the CPTs. Add new function cfs_cpt_of_node() which returns
the CPT of a particular NUMA node.

Signed-off-by: Amir Shehata <amir.shehata at intel.com>
WC-bug-id: https://jira.whamcloud.com/browse/LU-7734
Reviewed-on: http://review.whamcloud.com/18916
Reviewed-by: Olaf Weber <olaf at sgi.com>
Reviewed-by: Doug Oucharek <dougso at me.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 .../lustre/include/linux/libcfs/libcfs_cpu.h        | 11 +++++++++++
 drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c     | 21 +++++++++++++++++++++
 2 files changed, 32 insertions(+)

diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h
index 1b4333d..ff3ecf5 100644
--- a/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h
+++ b/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h
@@ -103,6 +103,8 @@ struct cfs_cpt_table {
 	int				*ctb_cpu2cpt;
 	/* all cpus in this partition table */
 	cpumask_var_t			ctb_cpumask;
+	/* shadow HW node to CPU partition ID */
+	int				*ctb_node2cpt;
 	/* all nodes in this partition table */
 	nodemask_t			*ctb_nodemask;
 };
@@ -143,6 +145,10 @@ struct cfs_cpt_table {
  */
 int cfs_cpt_of_cpu(struct cfs_cpt_table *cptab, int cpu);
 /**
+ * shadow HW node ID \a NODE to CPU-partition ID by \a cptab
+ */
+int cfs_cpt_of_node(struct cfs_cpt_table *cptab, int node);
+/**
  * bind current thread on a CPU-partition \a cpt of \a cptab
  */
 int cfs_cpt_bind(struct cfs_cpt_table *cptab, int cpt);
@@ -299,6 +305,11 @@ void cfs_cpt_unset_nodemask(struct cfs_cpt_table *cptab,
 	return 0;
 }
 
+static inline int cfs_cpt_of_node(struct cfs_cpt_table *cptab, int node)
+{
+	return 0;
+}
+
 static inline int
 cfs_cpt_bind(struct cfs_cpt_table *cptab, int cpt)
 {
diff --git a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
index 33294da..8c5cf7b 100644
--- a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
+++ b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
@@ -102,6 +102,15 @@ struct cfs_cpt_table *
 	memset(cptab->ctb_cpu2cpt, -1,
 	       nr_cpu_ids * sizeof(cptab->ctb_cpu2cpt[0]));
 
+	cptab->ctb_node2cpt = kvmalloc_array(nr_node_ids,
+					     sizeof(cptab->ctb_node2cpt[0]),
+					     GFP_KERNEL);
+	if (!cptab->ctb_node2cpt)
+		goto failed_alloc_node2cpt;
+
+	memset(cptab->ctb_node2cpt, -1,
+	       nr_node_ids * sizeof(cptab->ctb_node2cpt[0]));
+
 	cptab->ctb_parts = kvmalloc_array(ncpt, sizeof(cptab->ctb_parts[0]),
 					  GFP_KERNEL);
 	if (!cptab->ctb_parts)
@@ -133,6 +142,8 @@ struct cfs_cpt_table *
 
 	kvfree(cptab->ctb_parts);
 failed_alloc_ctb_parts:
+	kvfree(cptab->ctb_node2cpt);
+failed_alloc_node2cpt:
 	kvfree(cptab->ctb_cpu2cpt);
 failed_alloc_cpu2cpt:
 	kfree(cptab->ctb_nodemask);
@@ -150,6 +161,7 @@ struct cfs_cpt_table *
 	int i;
 
 	kvfree(cptab->ctb_cpu2cpt);
+	kvfree(cptab->ctb_node2cpt);
 
 	for (i = 0; cptab->ctb_parts && i < cptab->ctb_nparts; i++) {
 		struct cfs_cpu_partition *part = &cptab->ctb_parts[i];
@@ -515,6 +527,15 @@ struct cfs_cpt_table *
 }
 EXPORT_SYMBOL(cfs_cpt_of_cpu);
 
+int cfs_cpt_of_node(struct cfs_cpt_table *cptab, int node)
+{
+	if (node < 0 || node > nr_node_ids)
+		return CFS_CPT_ANY;
+
+	return cptab->ctb_node2cpt[node];
+}
+EXPORT_SYMBOL(cfs_cpt_of_node);
+
 int
 cfs_cpt_bind(struct cfs_cpt_table *cptab, int cpt)
 {
-- 
1.8.3.1


From jsimmons at infradead.org  Sun Jun 24 21:20:32 2018
From: jsimmons at infradead.org (James Simmons)
Date: Sun, 24 Jun 2018 17:20:32 -0400
Subject: [lustre-devel] [PATCH v3 08/26] staging: lustre: libcfs: add cpu
 distance handling
In-Reply-To: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
References: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1529875250-11531-9-git-send-email-jsimmons@infradead.org>

From: Amir Shehata <amir.shehata at intel.com>

Add functionality to calculate the distance between two CPTs.
Expose those distance in debugfs so people deploying a setup
can debug what is being created for CPTs.

Signed-off-by: Amir Shehata <amir.shehata at intel.com>
WC-bug-id: https://jira.whamcloud.com/browse/LU-7734
Reviewed-on: http://review.whamcloud.com/18916
Reviewed-by: Olaf Weber <olaf at sgi.com>
Reviewed-by: Doug Oucharek <dougso at me.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 .../lustre/include/linux/libcfs/libcfs_cpu.h       | 31 +++++++++++
 drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c    | 61 ++++++++++++++++++++++
 2 files changed, 92 insertions(+)

diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h
index ff3ecf5..a015ac1 100644
--- a/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h
+++ b/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h
@@ -86,6 +86,8 @@ struct cfs_cpu_partition {
 	cpumask_var_t			cpt_cpumask;
 	/* nodes mask for this partition */
 	nodemask_t			*cpt_nodemask;
+	/* NUMA distance between CPTs */
+	unsigned int			*cpt_distance;
 	/* spread rotor for NUMA allocator */
 	unsigned int			cpt_spread_rotor;
 };
@@ -95,6 +97,8 @@ struct cfs_cpu_partition {
 struct cfs_cpt_table {
 	/* spread rotor for NUMA allocator */
 	unsigned int			ctb_spread_rotor;
+	/* maximum NUMA distance between all nodes in table */
+	unsigned int			ctb_distance;
 	/* # of CPU partitions */
 	unsigned int			ctb_nparts;
 	/* partitions tables */
@@ -120,6 +124,10 @@ struct cfs_cpt_table {
  */
 int cfs_cpt_table_print(struct cfs_cpt_table *cptab, char *buf, int len);
 /**
+ * print distance information of cpt-table
+ */
+int cfs_cpt_distance_print(struct cfs_cpt_table *cptab, char *buf, int len);
+/**
  * return total number of CPU partitions in \a cptab
  */
 int
@@ -149,6 +157,10 @@ struct cfs_cpt_table {
  */
 int cfs_cpt_of_node(struct cfs_cpt_table *cptab, int node);
 /**
+ * NUMA distance between \a cpt1 and \a cpt2 in \a cptab
+ */
+unsigned int cfs_cpt_distance(struct cfs_cpt_table *cptab, int cpt1, int cpt2);
+/**
  * bind current thread on a CPU-partition \a cpt of \a cptab
  */
 int cfs_cpt_bind(struct cfs_cpt_table *cptab, int cpt);
@@ -206,6 +218,19 @@ void cfs_cpt_unset_nodemask(struct cfs_cpt_table *cptab,
 struct cfs_cpt_table;
 #define cfs_cpt_tab ((struct cfs_cpt_table *)NULL)
 
+static inline int cfs_cpt_distance_print(struct cfs_cpt_table *cptab,
+					 char *buf, int len)
+{
+	int rc;
+
+	rc = snprintf(buf, len, "0\t: 0:1\n");
+	len -= rc;
+	if (len <= 0)
+		return -EFBIG;
+
+	return rc;
+}
+
 static inline cpumask_var_t *
 cfs_cpt_cpumask(struct cfs_cpt_table *cptab, int cpt)
 {
@@ -241,6 +266,12 @@ void cfs_cpt_unset_nodemask(struct cfs_cpt_table *cptab,
 	return NULL;
 }
 
+static inline unsigned int cfs_cpt_distance(struct cfs_cpt_table *cptab,
+					    int cpt1, int cpt2)
+{
+	return 1;
+}
+
 static inline int
 cfs_cpt_set_cpu(struct cfs_cpt_table *cptab, int cpt, int cpu)
 {
diff --git a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
index 8c5cf7b..b315fb2 100644
--- a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
+++ b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
@@ -128,6 +128,15 @@ struct cfs_cpt_table *
 					     GFP_NOFS);
 		if (!part->cpt_nodemask)
 			goto failed_setting_ctb_parts;
+
+		part->cpt_distance = kvmalloc_array(cptab->ctb_nparts,
+						    sizeof(part->cpt_distance[0]),
+						    GFP_KERNEL);
+		if (!part->cpt_distance)
+			goto failed_setting_ctb_parts;
+
+		memset(part->cpt_distance, -1,
+		       cptab->ctb_nparts * sizeof(part->cpt_distance[0]));
 	}
 
 	return cptab;
@@ -138,6 +147,7 @@ struct cfs_cpt_table *
 
 		kfree(part->cpt_nodemask);
 		free_cpumask_var(part->cpt_cpumask);
+		kvfree(part->cpt_distance);
 	}
 
 	kvfree(cptab->ctb_parts);
@@ -168,6 +178,7 @@ struct cfs_cpt_table *
 
 		kfree(part->cpt_nodemask);
 		free_cpumask_var(part->cpt_cpumask);
+		kvfree(part->cpt_distance);
 	}
 
 	kvfree(cptab->ctb_parts);
@@ -222,6 +233,44 @@ struct cfs_cpt_table *
 }
 EXPORT_SYMBOL(cfs_cpt_table_print);
 
+int cfs_cpt_distance_print(struct cfs_cpt_table *cptab, char *buf, int len)
+{
+	char *tmp = buf;
+	int rc;
+	int i;
+	int j;
+
+	for (i = 0; i < cptab->ctb_nparts; i++) {
+		if (len <= 0)
+			goto err;
+
+		rc = snprintf(tmp, len, "%d\t:", i);
+		len -= rc;
+
+		if (len <= 0)
+			goto err;
+
+		tmp += rc;
+		for (j = 0; j < cptab->ctb_nparts; j++) {
+			rc = snprintf(tmp, len, " %d:%d", j,
+				      cptab->ctb_parts[i].cpt_distance[j]);
+			len -= rc;
+			if (len <= 0)
+				goto err;
+			tmp += rc;
+		}
+
+		*tmp = '\n';
+		tmp++;
+		len--;
+	}
+
+	return tmp - buf;
+err:
+	return -E2BIG;
+}
+EXPORT_SYMBOL(cfs_cpt_distance_print);
+
 int
 cfs_cpt_number(struct cfs_cpt_table *cptab)
 {
@@ -273,6 +322,18 @@ struct cfs_cpt_table *
 }
 EXPORT_SYMBOL(cfs_cpt_nodemask);
 
+unsigned int cfs_cpt_distance(struct cfs_cpt_table *cptab, int cpt1, int cpt2)
+{
+	LASSERT(cpt1 == CFS_CPT_ANY || (cpt1 >= 0 && cpt1 < cptab->ctb_nparts));
+	LASSERT(cpt2 == CFS_CPT_ANY || (cpt2 >= 0 && cpt2 < cptab->ctb_nparts));
+
+	if (cpt1 == CFS_CPT_ANY || cpt2 == CFS_CPT_ANY)
+		return cptab->ctb_distance;
+
+	return cptab->ctb_parts[cpt1].cpt_distance[cpt2];
+}
+EXPORT_SYMBOL(cfs_cpt_distance);
+
 int
 cfs_cpt_set_cpu(struct cfs_cpt_table *cptab, int cpt, int cpu)
 {
-- 
1.8.3.1


From jsimmons at infradead.org  Sun Jun 24 21:20:34 2018
From: jsimmons at infradead.org (James Simmons)
Date: Sun, 24 Jun 2018 17:20:34 -0400
Subject: [lustre-devel] [PATCH v3 10/26] staging: lustre: libcfs: provide
 debugfs files for distance handling
In-Reply-To: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
References: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1529875250-11531-11-git-send-email-jsimmons@infradead.org>

From: Amir Shehata <amir.shehata at intel.com>

On systems with large number of NUMA nodes and cores it is easy
to incorrectly configure their use with Lustre. Provide debugfs
files which can help track down any issues.

Signed-off-by: Amir Shehata <amir.shehata at intel.com>
WC-bug-id: https://jira.whamcloud.com/browse/LU-7734
Reviewed-on: http://review.whamcloud.com/18916
Reviewed-by: Olaf Weber <olaf at sgi.com>
Reviewed-by: Doug Oucharek <dougso at me.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/lnet/libcfs/module.c | 48 +++++++++++++++++++++++++++++
 1 file changed, 48 insertions(+)

diff --git a/drivers/staging/lustre/lnet/libcfs/module.c b/drivers/staging/lustre/lnet/libcfs/module.c
index 02c404c..2281f08 100644
--- a/drivers/staging/lustre/lnet/libcfs/module.c
+++ b/drivers/staging/lustre/lnet/libcfs/module.c
@@ -425,6 +425,48 @@ static int proc_cpt_table(struct ctl_table *table, int write,
 	return rc;
 }
 
+static int proc_cpt_distance(struct ctl_table *table, int write,
+			     void __user *buffer, size_t *lenp, loff_t *ppos)
+{
+	size_t nob = *lenp;
+	loff_t pos = *ppos;
+	char *buf = NULL;
+	int len = 4096;
+	int rc = 0;
+
+	if (write)
+		return -EPERM;
+
+	LASSERT(cfs_cpt_tab);
+
+	while (1) {
+		buf = kzalloc(len, GFP_KERNEL);
+		if (!buf)
+			return -ENOMEM;
+
+		rc = cfs_cpt_distance_print(cfs_cpt_tab, buf, len);
+		if (rc >= 0)
+			break;
+
+		if (rc == -EFBIG) {
+			kfree(buf);
+			len <<= 1;
+			continue;
+		}
+		goto out;
+	}
+
+	if (pos >= rc) {
+		rc = 0;
+		goto out;
+	}
+
+	rc = cfs_trace_copyout_string(buffer, nob, buf + pos, NULL);
+out:
+	kfree(buf);
+	return rc;
+}
+
 static struct ctl_table lnet_table[] = {
 	{
 		.procname = "debug",
@@ -454,6 +496,12 @@ static int proc_cpt_table(struct ctl_table *table, int write,
 		.proc_handler = &proc_cpt_table,
 	},
 	{
+		.procname = "cpu_partition_distance",
+		.maxlen	  = 128,
+		.mode	  = 0444,
+		.proc_handler = &proc_cpt_distance,
+	},
+	{
 		.procname = "debug_log_upcall",
 		.data     = lnet_debug_log_upcall,
 		.maxlen   = sizeof(lnet_debug_log_upcall),
-- 
1.8.3.1


From jsimmons at infradead.org  Sun Jun 24 21:20:35 2018
From: jsimmons at infradead.org (James Simmons)
Date: Sun, 24 Jun 2018 17:20:35 -0400
Subject: [lustre-devel] [PATCH v3 11/26] staging: lustre: libcfs: invert
 error handling for cfs_cpt_table_print
In-Reply-To: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
References: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1529875250-11531-12-git-send-email-jsimmons@infradead.org>

From: Amir Shehata <amir.shehata at intel.com>

Instead of setting rc to -EFBIG for several cases in the loop lets
just go to the out label on error which returns -E2BIG directly.

Signed-off-by: Amir Shehata <amir.shehata at intel.com>
WC-bug-id: https://jira.whamcloud.com/browse/LU-7734
Reviewed-on: http://review.whamcloud.com/18916
Reviewed-by: Olaf Weber <olaf at sgi.com>
Reviewed-by: Doug Oucharek <dougso at me.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c | 29 ++++++++++---------------
 1 file changed, 12 insertions(+), 17 deletions(-)

diff --git a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
index 3b4a9dc..cdfa77b 100644
--- a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
+++ b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
@@ -194,29 +194,26 @@ struct cfs_cpt_table *
 cfs_cpt_table_print(struct cfs_cpt_table *cptab, char *buf, int len)
 {
 	char *tmp = buf;
-	int rc = 0;
+	int rc;
 	int i;
 	int j;
 
 	for (i = 0; i < cptab->ctb_nparts; i++) {
-		if (len > 0) {
-			rc = snprintf(tmp, len, "%d\t:", i);
-			len -= rc;
-		}
+		if (len <= 0)
+			goto err;
 
-		if (len <= 0) {
-			rc = -EFBIG;
-			goto out;
-		}
+		rc = snprintf(tmp, len, "%d\t:", i);
+		len -= rc;
+
+		if (len <= 0)
+			goto err;
 
 		tmp += rc;
 		for_each_cpu(j, cptab->ctb_parts[i].cpt_cpumask) {
 			rc = snprintf(tmp, len, "%d ", j);
 			len -= rc;
-			if (len <= 0) {
-				rc = -EFBIG;
-				goto out;
-			}
+			if (len <= 0)
+				goto err;
 			tmp += rc;
 		}
 
@@ -225,11 +222,9 @@ struct cfs_cpt_table *
 		len--;
 	}
 
- out:
-	if (rc < 0)
-		return rc;
-
 	return tmp - buf;
+err:
+	return -E2BIG;
 }
 EXPORT_SYMBOL(cfs_cpt_table_print);
 
-- 
1.8.3.1


From jsimmons at infradead.org  Sun Jun 24 21:20:36 2018
From: jsimmons at infradead.org (James Simmons)
Date: Sun, 24 Jun 2018 17:20:36 -0400
Subject: [lustre-devel] [PATCH v3 12/26] staging: lustre: libcfs: fix
 libcfs_cpu coding style
In-Reply-To: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
References: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1529875250-11531-13-git-send-email-jsimmons@infradead.org>

From: Dmitry Eremin <dmitry.eremin at intel.com>

This patch bring the lustre CPT code into alignment with the
Linux kernel coding style.

Signed-off-by: Dmitry Eremin <dmitry.eremin at intel.com>
WC-bug-id: https://jira.whamcloud.com/browse/LU-8703
Reviewed-on: https://review.whamcloud.com/23304
Reviewed-by: James Simmons <uja.ornl at yahoo.com>
Reviewed-by: Doug Oucharek <dougso at me.com>
Reviewed-by: Oleg Drokin <green at whamcloud.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 .../lustre/include/linux/libcfs/libcfs_cpu.h       | 81 ++++++++-----------
 drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c    | 92 ++++++++--------------
 2 files changed, 69 insertions(+), 104 deletions(-)

diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h
index a015ac1..9dbb0b1 100644
--- a/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h
+++ b/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h
@@ -130,8 +130,7 @@ struct cfs_cpt_table {
 /**
  * return total number of CPU partitions in \a cptab
  */
-int
-cfs_cpt_number(struct cfs_cpt_table *cptab);
+int cfs_cpt_number(struct cfs_cpt_table *cptab);
 /**
  * return number of HW cores or hyper-threadings in a CPU partition \a cpt
  */
@@ -193,25 +192,24 @@ void cfs_cpt_unset_cpumask(struct cfs_cpt_table *cptab,
  * remove all cpus in NUMA node \a node from CPU partition \a cpt
  */
 void cfs_cpt_unset_node(struct cfs_cpt_table *cptab, int cpt, int node);
-
 /**
  * add all cpus in node mask \a mask to CPU partition \a cpt
  * return 1 if successfully set all CPUs, otherwise return 0
  */
 int cfs_cpt_set_nodemask(struct cfs_cpt_table *cptab,
-			 int cpt, nodemask_t *mask);
+			 int cpt, const nodemask_t *mask);
 /**
  * remove all cpus in node mask \a mask from CPU partition \a cpt
  */
 void cfs_cpt_unset_nodemask(struct cfs_cpt_table *cptab,
-			    int cpt, nodemask_t *mask);
+			    int cpt, const nodemask_t *mask);
 /**
  * convert partition id \a cpt to numa node id, if there are more than one
  * nodes in this partition, it might return a different node id each time.
  */
 int cfs_cpt_spread_node(struct cfs_cpt_table *cptab, int cpt);
 
-int  cfs_cpu_init(void);
+int cfs_cpu_init(void);
 void cfs_cpu_fini(void);
 
 #else /* !CONFIG_SMP */
@@ -231,37 +229,35 @@ static inline int cfs_cpt_distance_print(struct cfs_cpt_table *cptab,
 	return rc;
 }
 
-static inline cpumask_var_t *
-cfs_cpt_cpumask(struct cfs_cpt_table *cptab, int cpt)
+static inline cpumask_var_t *cfs_cpt_cpumask(struct cfs_cpt_table *cptab,
+					     int cpt)
 {
 	return NULL;
 }
 
-static inline int
-cfs_cpt_table_print(struct cfs_cpt_table *cptab, char *buf, int len)
+static inline int cfs_cpt_table_print(struct cfs_cpt_table *cptab, char *buf,
+				      int len)
 {
 	return 0;
 }
-static inline int
-cfs_cpt_number(struct cfs_cpt_table *cptab)
+
+static inline int cfs_cpt_number(struct cfs_cpt_table *cptab)
 {
 	return 1;
 }
 
-static inline int
-cfs_cpt_weight(struct cfs_cpt_table *cptab, int cpt)
+static inline int cfs_cpt_weight(struct cfs_cpt_table *cptab, int cpt)
 {
 	return 1;
 }
 
-static inline int
-cfs_cpt_online(struct cfs_cpt_table *cptab, int cpt)
+static inline int cfs_cpt_online(struct cfs_cpt_table *cptab, int cpt)
 {
 	return 1;
 }
 
-static inline nodemask_t *
-cfs_cpt_nodemask(struct cfs_cpt_table *cptab, int cpt)
+static inline nodemask_t *cfs_cpt_nodemask(struct cfs_cpt_table *cptab,
+					   int cpt)
 {
 	return NULL;
 }
@@ -272,66 +268,61 @@ static inline unsigned int cfs_cpt_distance(struct cfs_cpt_table *cptab,
 	return 1;
 }
 
-static inline int
-cfs_cpt_set_cpu(struct cfs_cpt_table *cptab, int cpt, int cpu)
+static inline int cfs_cpt_set_cpu(struct cfs_cpt_table *cptab, int cpt,
+				  int cpu)
 {
 	return 1;
 }
 
-static inline void
-cfs_cpt_unset_cpu(struct cfs_cpt_table *cptab, int cpt, int cpu)
+static inline void cfs_cpt_unset_cpu(struct cfs_cpt_table *cptab, int cpt,
+				     int cpu)
 {
 }
 
-static inline int
-cfs_cpt_set_cpumask(struct cfs_cpt_table *cptab, int cpt,
-		    const cpumask_t *mask)
+static inline int cfs_cpt_set_cpumask(struct cfs_cpt_table *cptab, int cpt,
+				      const cpumask_t *mask)
 {
 	return 1;
 }
 
-static inline void
-cfs_cpt_unset_cpumask(struct cfs_cpt_table *cptab, int cpt,
-		      const cpumask_t *mask)
+static inline void cfs_cpt_unset_cpumask(struct cfs_cpt_table *cptab, int cpt,
+					 const cpumask_t *mask)
 {
 }
 
-static inline int
-cfs_cpt_set_node(struct cfs_cpt_table *cptab, int cpt, int node)
+static inline int cfs_cpt_set_node(struct cfs_cpt_table *cptab, int cpt,
+				   int node)
 {
 	return 1;
 }
 
-static inline void
-cfs_cpt_unset_node(struct cfs_cpt_table *cptab, int cpt, int node)
+static inline void cfs_cpt_unset_node(struct cfs_cpt_table *cptab, int cpt,
+				      int node)
 {
 }
 
-static inline int
-cfs_cpt_set_nodemask(struct cfs_cpt_table *cptab, int cpt, nodemask_t *mask)
+static inline int cfs_cpt_set_nodemask(struct cfs_cpt_table *cptab, int cpt,
+				       const nodemask_t *mask)
 {
 	return 1;
 }
 
-static inline void
-cfs_cpt_unset_nodemask(struct cfs_cpt_table *cptab, int cpt, nodemask_t *mask)
+static inline void cfs_cpt_unset_nodemask(struct cfs_cpt_table *cptab,
+					  int cpt, const nodemask_t *mask)
 {
 }
 
-static inline int
-cfs_cpt_spread_node(struct cfs_cpt_table *cptab, int cpt)
+static inline int cfs_cpt_spread_node(struct cfs_cpt_table *cptab, int cpt)
 {
 	return 0;
 }
 
-static inline int
-cfs_cpt_current(struct cfs_cpt_table *cptab, int remap)
+static inline int cfs_cpt_current(struct cfs_cpt_table *cptab, int remap)
 {
 	return 0;
 }
 
-static inline int
-cfs_cpt_of_cpu(struct cfs_cpt_table *cptab, int cpu)
+static inline int cfs_cpt_of_cpu(struct cfs_cpt_table *cptab, int cpu)
 {
 	return 0;
 }
@@ -341,14 +332,12 @@ static inline int cfs_cpt_of_node(struct cfs_cpt_table *cptab, int node)
 	return 0;
 }
 
-static inline int
-cfs_cpt_bind(struct cfs_cpt_table *cptab, int cpt)
+static inline int cfs_cpt_bind(struct cfs_cpt_table *cptab, int cpt)
 {
 	return 0;
 }
 
-static inline int
-cfs_cpu_init(void)
+static inline int cfs_cpu_init(void)
 {
 	return 0;
 }
diff --git a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
index cdfa77b..aaab7cb 100644
--- a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
+++ b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
@@ -73,8 +73,7 @@
 module_param(cpu_pattern, charp, 0444);
 MODULE_PARM_DESC(cpu_pattern, "CPU partitions pattern");
 
-struct cfs_cpt_table *
-cfs_cpt_table_alloc(unsigned int ncpt)
+struct cfs_cpt_table *cfs_cpt_table_alloc(unsigned int ncpt)
 {
 	struct cfs_cpt_table *cptab;
 	int i;
@@ -165,8 +164,7 @@ struct cfs_cpt_table *
 }
 EXPORT_SYMBOL(cfs_cpt_table_alloc);
 
-void
-cfs_cpt_table_free(struct cfs_cpt_table *cptab)
+void cfs_cpt_table_free(struct cfs_cpt_table *cptab)
 {
 	int i;
 
@@ -190,8 +188,7 @@ struct cfs_cpt_table *
 }
 EXPORT_SYMBOL(cfs_cpt_table_free);
 
-int
-cfs_cpt_table_print(struct cfs_cpt_table *cptab, char *buf, int len)
+int cfs_cpt_table_print(struct cfs_cpt_table *cptab, char *buf, int len)
 {
 	char *tmp = buf;
 	int rc;
@@ -266,15 +263,13 @@ int cfs_cpt_distance_print(struct cfs_cpt_table *cptab, char *buf, int len)
 }
 EXPORT_SYMBOL(cfs_cpt_distance_print);
 
-int
-cfs_cpt_number(struct cfs_cpt_table *cptab)
+int cfs_cpt_number(struct cfs_cpt_table *cptab)
 {
 	return cptab->ctb_nparts;
 }
 EXPORT_SYMBOL(cfs_cpt_number);
 
-int
-cfs_cpt_weight(struct cfs_cpt_table *cptab, int cpt)
+int cfs_cpt_weight(struct cfs_cpt_table *cptab, int cpt)
 {
 	LASSERT(cpt == CFS_CPT_ANY || (cpt >= 0 && cpt < cptab->ctb_nparts));
 
@@ -284,8 +279,7 @@ int cfs_cpt_distance_print(struct cfs_cpt_table *cptab, char *buf, int len)
 }
 EXPORT_SYMBOL(cfs_cpt_weight);
 
-int
-cfs_cpt_online(struct cfs_cpt_table *cptab, int cpt)
+int cfs_cpt_online(struct cfs_cpt_table *cptab, int cpt)
 {
 	LASSERT(cpt == CFS_CPT_ANY || (cpt >= 0 && cpt < cptab->ctb_nparts));
 
@@ -297,8 +291,7 @@ int cfs_cpt_distance_print(struct cfs_cpt_table *cptab, char *buf, int len)
 }
 EXPORT_SYMBOL(cfs_cpt_online);
 
-cpumask_var_t *
-cfs_cpt_cpumask(struct cfs_cpt_table *cptab, int cpt)
+cpumask_var_t *cfs_cpt_cpumask(struct cfs_cpt_table *cptab, int cpt)
 {
 	LASSERT(cpt == CFS_CPT_ANY || (cpt >= 0 && cpt < cptab->ctb_nparts));
 
@@ -307,8 +300,7 @@ int cfs_cpt_distance_print(struct cfs_cpt_table *cptab, char *buf, int len)
 }
 EXPORT_SYMBOL(cfs_cpt_cpumask);
 
-nodemask_t *
-cfs_cpt_nodemask(struct cfs_cpt_table *cptab, int cpt)
+nodemask_t *cfs_cpt_nodemask(struct cfs_cpt_table *cptab, int cpt)
 {
 	LASSERT(cpt == CFS_CPT_ANY || (cpt >= 0 && cpt < cptab->ctb_nparts));
 
@@ -454,8 +446,7 @@ static void cfs_cpt_del_node(struct cfs_cpt_table *cptab, int cpt, int node)
 	}
 }
 
-int
-cfs_cpt_set_cpu(struct cfs_cpt_table *cptab, int cpt, int cpu)
+int cfs_cpt_set_cpu(struct cfs_cpt_table *cptab, int cpt, int cpu)
 {
 	LASSERT(cpt >= 0 && cpt < cptab->ctb_nparts);
 
@@ -480,8 +471,7 @@ static void cfs_cpt_del_node(struct cfs_cpt_table *cptab, int cpt, int node)
 }
 EXPORT_SYMBOL(cfs_cpt_set_cpu);
 
-void
-cfs_cpt_unset_cpu(struct cfs_cpt_table *cptab, int cpt, int cpu)
+void cfs_cpt_unset_cpu(struct cfs_cpt_table *cptab, int cpt, int cpu)
 {
 	LASSERT(cpt == CFS_CPT_ANY || (cpt >= 0 && cpt < cptab->ctb_nparts));
 
@@ -513,9 +503,8 @@ static void cfs_cpt_del_node(struct cfs_cpt_table *cptab, int cpt, int node)
 }
 EXPORT_SYMBOL(cfs_cpt_unset_cpu);
 
-int
-cfs_cpt_set_cpumask(struct cfs_cpt_table *cptab, int cpt,
-		    const cpumask_t *mask)
+int cfs_cpt_set_cpumask(struct cfs_cpt_table *cptab, int cpt,
+			const cpumask_t *mask)
 {
 	int cpu;
 
@@ -535,9 +524,8 @@ static void cfs_cpt_del_node(struct cfs_cpt_table *cptab, int cpt, int node)
 }
 EXPORT_SYMBOL(cfs_cpt_set_cpumask);
 
-void
-cfs_cpt_unset_cpumask(struct cfs_cpt_table *cptab, int cpt,
-		      const cpumask_t *mask)
+void cfs_cpt_unset_cpumask(struct cfs_cpt_table *cptab, int cpt,
+			   const cpumask_t *mask)
 {
 	int cpu;
 
@@ -546,8 +534,7 @@ static void cfs_cpt_del_node(struct cfs_cpt_table *cptab, int cpt, int node)
 }
 EXPORT_SYMBOL(cfs_cpt_unset_cpumask);
 
-int
-cfs_cpt_set_node(struct cfs_cpt_table *cptab, int cpt, int node)
+int cfs_cpt_set_node(struct cfs_cpt_table *cptab, int cpt, int node)
 {
 	const cpumask_t *mask;
 	int cpu;
@@ -569,8 +556,7 @@ static void cfs_cpt_del_node(struct cfs_cpt_table *cptab, int cpt, int node)
 }
 EXPORT_SYMBOL(cfs_cpt_set_node);
 
-void
-cfs_cpt_unset_node(struct cfs_cpt_table *cptab, int cpt, int node)
+void cfs_cpt_unset_node(struct cfs_cpt_table *cptab, int cpt, int node)
 {
 	const cpumask_t *mask;
 	int cpu;
@@ -590,8 +576,8 @@ static void cfs_cpt_del_node(struct cfs_cpt_table *cptab, int cpt, int node)
 }
 EXPORT_SYMBOL(cfs_cpt_unset_node);
 
-int
-cfs_cpt_set_nodemask(struct cfs_cpt_table *cptab, int cpt, nodemask_t *mask)
+int cfs_cpt_set_nodemask(struct cfs_cpt_table *cptab, int cpt,
+			 const nodemask_t *mask)
 {
 	int i;
 
@@ -604,8 +590,8 @@ static void cfs_cpt_del_node(struct cfs_cpt_table *cptab, int cpt, int node)
 }
 EXPORT_SYMBOL(cfs_cpt_set_nodemask);
 
-void
-cfs_cpt_unset_nodemask(struct cfs_cpt_table *cptab, int cpt, nodemask_t *mask)
+void cfs_cpt_unset_nodemask(struct cfs_cpt_table *cptab, int cpt,
+			    const nodemask_t *mask)
 {
 	int i;
 
@@ -614,8 +600,7 @@ static void cfs_cpt_del_node(struct cfs_cpt_table *cptab, int cpt, int node)
 }
 EXPORT_SYMBOL(cfs_cpt_unset_nodemask);
 
-int
-cfs_cpt_spread_node(struct cfs_cpt_table *cptab, int cpt)
+int cfs_cpt_spread_node(struct cfs_cpt_table *cptab, int cpt)
 {
 	nodemask_t *mask;
 	int weight;
@@ -647,8 +632,7 @@ static void cfs_cpt_del_node(struct cfs_cpt_table *cptab, int cpt, int node)
 }
 EXPORT_SYMBOL(cfs_cpt_spread_node);
 
-int
-cfs_cpt_current(struct cfs_cpt_table *cptab, int remap)
+int cfs_cpt_current(struct cfs_cpt_table *cptab, int remap)
 {
 	int cpu;
 	int cpt;
@@ -668,8 +652,7 @@ static void cfs_cpt_del_node(struct cfs_cpt_table *cptab, int cpt, int node)
 }
 EXPORT_SYMBOL(cfs_cpt_current);
 
-int
-cfs_cpt_of_cpu(struct cfs_cpt_table *cptab, int cpu)
+int cfs_cpt_of_cpu(struct cfs_cpt_table *cptab, int cpu)
 {
 	LASSERT(cpu >= 0 && cpu < nr_cpu_ids);
 
@@ -686,8 +669,7 @@ int cfs_cpt_of_node(struct cfs_cpt_table *cptab, int node)
 }
 EXPORT_SYMBOL(cfs_cpt_of_node);
 
-int
-cfs_cpt_bind(struct cfs_cpt_table *cptab, int cpt)
+int cfs_cpt_bind(struct cfs_cpt_table *cptab, int cpt)
 {
 	cpumask_var_t *cpumask;
 	nodemask_t *nodemask;
@@ -731,9 +713,8 @@ int cfs_cpt_of_node(struct cfs_cpt_table *cptab, int node)
  * Choose max to \a number CPUs from \a node and set them in \a cpt.
  * We always prefer to choose CPU in the same core/socket.
  */
-static int
-cfs_cpt_choose_ncpus(struct cfs_cpt_table *cptab, int cpt,
-		     cpumask_t *node, int number)
+static int cfs_cpt_choose_ncpus(struct cfs_cpt_table *cptab, int cpt,
+				cpumask_t *node, int number)
 {
 	cpumask_var_t socket;
 	cpumask_var_t core;
@@ -809,8 +790,7 @@ int cfs_cpt_of_node(struct cfs_cpt_table *cptab, int node)
 
 #define CPT_WEIGHT_MIN  4u
 
-static unsigned int
-cfs_cpt_num_estimate(void)
+static unsigned int cfs_cpt_num_estimate(void)
 {
 	unsigned int nnode = num_online_nodes();
 	unsigned int ncpu = num_online_cpus();
@@ -852,8 +832,7 @@ int cfs_cpt_of_node(struct cfs_cpt_table *cptab, int node)
 	return ncpt;
 }
 
-static struct cfs_cpt_table *
-cfs_cpt_table_create(int ncpt)
+static struct cfs_cpt_table *cfs_cpt_table_create(int ncpt)
 {
 	struct cfs_cpt_table *cptab = NULL;
 	cpumask_var_t mask;
@@ -936,9 +915,9 @@ int cfs_cpt_of_node(struct cfs_cpt_table *cptab, int node)
 
 	return cptab;
 
- failed_mask:
+failed_mask:
 	free_cpumask_var(mask);
- failed:
+failed:
 	CERROR("Failed to setup CPU-partition-table with %d CPU-partitions, online HW nodes: %d, HW cpus: %d.\n",
 	       ncpt, num_online_nodes(), num_online_cpus());
 
@@ -948,8 +927,7 @@ int cfs_cpt_of_node(struct cfs_cpt_table *cptab, int node)
 	return NULL;
 }
 
-static struct cfs_cpt_table *
-cfs_cpt_table_create_pattern(char *pattern)
+static struct cfs_cpt_table *cfs_cpt_table_create_pattern(char *pattern)
 {
 	struct cfs_cpt_table *cptab;
 	char *str;
@@ -1093,7 +1071,7 @@ int cfs_cpt_of_node(struct cfs_cpt_table *cptab, int node)
 
 	return cptab;
 
- failed:
+failed:
 	cfs_cpt_table_free(cptab);
 	return NULL;
 }
@@ -1120,8 +1098,7 @@ static int cfs_cpu_dead(unsigned int cpu)
 }
 #endif
 
-void
-cfs_cpu_fini(void)
+void cfs_cpu_fini(void)
 {
 	if (cfs_cpt_tab)
 		cfs_cpt_table_free(cfs_cpt_tab);
@@ -1133,8 +1110,7 @@ static int cfs_cpu_dead(unsigned int cpu)
 #endif
 }
 
-int
-cfs_cpu_init(void)
+int cfs_cpu_init(void)
 {
 	int ret;
 
-- 
1.8.3.1


From jsimmons at infradead.org  Sun Jun 24 21:20:40 2018
From: jsimmons at infradead.org (James Simmons)
Date: Sun, 24 Jun 2018 17:20:40 -0400
Subject: [lustre-devel] [PATCH v3 16/26] staging: lustre: libcfs: rename
 cpumask_var_t variables to *_mask
In-Reply-To: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
References: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1529875250-11531-17-git-send-email-jsimmons@infradead.org>

From: Dmitry Eremin <dmitry.eremin at intel.com>

Because we handle both cpu mask as well as core identifiers it can
easily be confused. To avoid this rename various cpumask_var_t to
have appended *_mask to their names.

Signed-off-by: Dmitry Eremin <dmitry.eremin at intel.com>
WC-bug-id: https://jira.whamcloud.com/browse/LU-8703
Reviewed-on: https://review.whamcloud.com/23222
Reviewed-by: Amir Shehata <amir.shehata at intel.com>
Reviewed-by: James Simmons <uja.ornl at yahoo.com>
Reviewed-by: Oleg Drokin <green at whamcloud.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c | 62 ++++++++++++-------------
 1 file changed, 31 insertions(+), 31 deletions(-)

diff --git a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
index 5654fbe..fcb068a 100644
--- a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
+++ b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
@@ -714,23 +714,23 @@ int cfs_cpt_bind(struct cfs_cpt_table *cptab, int cpt)
  * We always prefer to choose CPU in the same core/socket.
  */
 static int cfs_cpt_choose_ncpus(struct cfs_cpt_table *cptab, int cpt,
-				cpumask_t *node, int number)
+				cpumask_t *node_mask, int number)
 {
-	cpumask_var_t socket;
-	cpumask_var_t core;
+	cpumask_var_t socket_mask;
+	cpumask_var_t core_mask;
 	int rc = 0;
 	int cpu;
 
 	LASSERT(number > 0);
 
-	if (number >= cpumask_weight(node)) {
-		while (!cpumask_empty(node)) {
-			cpu = cpumask_first(node);
+	if (number >= cpumask_weight(node_mask)) {
+		while (!cpumask_empty(node_mask)) {
+			cpu = cpumask_first(node_mask);
 
 			rc = cfs_cpt_set_cpu(cptab, cpt, cpu);
 			if (!rc)
 				return -EINVAL;
-			cpumask_clear_cpu(cpu, node);
+			cpumask_clear_cpu(cpu, node_mask);
 		}
 		return 0;
 	}
@@ -740,34 +740,34 @@ static int cfs_cpt_choose_ncpus(struct cfs_cpt_table *cptab, int cpt,
 	 * As we cannot initialize a cpumask_var_t, we need
 	 * to alloc both before we can risk trying to free either
 	 */
-	if (!zalloc_cpumask_var(&socket, GFP_NOFS))
+	if (!zalloc_cpumask_var(&socket_mask, GFP_NOFS))
 		rc = -ENOMEM;
-	if (!zalloc_cpumask_var(&core, GFP_NOFS))
+	if (!zalloc_cpumask_var(&core_mask, GFP_NOFS))
 		rc = -ENOMEM;
 	if (rc)
 		goto out;
 
-	while (!cpumask_empty(node)) {
-		cpu = cpumask_first(node);
+	while (!cpumask_empty(node_mask)) {
+		cpu = cpumask_first(node_mask);
 
 		/* get cpumask for cores in the same socket */
-		cpumask_copy(socket, topology_core_cpumask(cpu));
-		cpumask_and(socket, socket, node);
+		cpumask_copy(socket_mask, topology_core_cpumask(cpu));
+		cpumask_and(socket_mask, socket_mask, node_mask);
 
-		LASSERT(!cpumask_empty(socket));
+		LASSERT(!cpumask_empty(socket_mask));
 
-		while (!cpumask_empty(socket)) {
+		while (!cpumask_empty(socket_mask)) {
 			int i;
 
 			/* get cpumask for hts in the same core */
-			cpumask_copy(core, topology_sibling_cpumask(cpu));
-			cpumask_and(core, core, node);
+			cpumask_copy(core_mask, topology_sibling_cpumask(cpu));
+			cpumask_and(core_mask, core_mask, node_mask);
 
-			LASSERT(!cpumask_empty(core));
+			LASSERT(!cpumask_empty(core_mask));
 
-			for_each_cpu(i, core) {
-				cpumask_clear_cpu(i, socket);
-				cpumask_clear_cpu(i, node);
+			for_each_cpu(i, core_mask) {
+				cpumask_clear_cpu(i, socket_mask);
+				cpumask_clear_cpu(i, node_mask);
 
 				rc = cfs_cpt_set_cpu(cptab, cpt, i);
 				if (!rc) {
@@ -778,13 +778,13 @@ static int cfs_cpt_choose_ncpus(struct cfs_cpt_table *cptab, int cpt,
 				if (!--number)
 					goto out;
 			}
-			cpu = cpumask_first(socket);
+			cpu = cpumask_first(socket_mask);
 		}
 	}
 
 out:
-	free_cpumask_var(socket);
-	free_cpumask_var(core);
+	free_cpumask_var(socket_mask);
+	free_cpumask_var(core_mask);
 	return rc;
 }
 
@@ -835,7 +835,7 @@ static int cfs_cpt_num_estimate(void)
 static struct cfs_cpt_table *cfs_cpt_table_create(int ncpt)
 {
 	struct cfs_cpt_table *cptab = NULL;
-	cpumask_var_t mask;
+	cpumask_var_t node_mask;
 	int cpt = 0;
 	int num;
 	int rc;
@@ -868,15 +868,15 @@ static struct cfs_cpt_table *cfs_cpt_table_create(int ncpt)
 		goto failed;
 	}
 
-	if (!zalloc_cpumask_var(&mask, GFP_NOFS)) {
+	if (!zalloc_cpumask_var(&node_mask, GFP_NOFS)) {
 		CERROR("Failed to allocate scratch cpumask\n");
 		goto failed;
 	}
 
 	for_each_online_node(i) {
-		cpumask_copy(mask, cpumask_of_node(i));
+		cpumask_copy(node_mask, cpumask_of_node(i));
 
-		while (!cpumask_empty(mask)) {
+		while (!cpumask_empty(node_mask)) {
 			struct cfs_cpu_partition *part;
 			int n;
 
@@ -893,7 +893,7 @@ static struct cfs_cpt_table *cfs_cpt_table_create(int ncpt)
 			n = num - cpumask_weight(part->cpt_cpumask);
 			LASSERT(n > 0);
 
-			rc = cfs_cpt_choose_ncpus(cptab, cpt, mask, n);
+			rc = cfs_cpt_choose_ncpus(cptab, cpt, node_mask, n);
 			if (rc < 0)
 				goto failed_mask;
 
@@ -911,12 +911,12 @@ static struct cfs_cpt_table *cfs_cpt_table_create(int ncpt)
 		goto failed_mask;
 	}
 
-	free_cpumask_var(mask);
+	free_cpumask_var(node_mask);
 
 	return cptab;
 
 failed_mask:
-	free_cpumask_var(mask);
+	free_cpumask_var(node_mask);
 failed:
 	CERROR("Failed to setup CPU-partition-table with %d CPU-partitions, online HW nodes: %d, HW cpus: %d.\n",
 	       ncpt, num_online_nodes(), num_online_cpus());
-- 
1.8.3.1


From jsimmons at infradead.org  Sun Jun 24 21:20:50 2018
From: jsimmons at infradead.org (James Simmons)
Date: Sun, 24 Jun 2018 17:20:50 -0400
Subject: [lustre-devel] [PATCH v3 26/26] staging: lustre: libcfs: restore
 UMP support
In-Reply-To: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
References: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1529875250-11531-27-git-send-email-jsimmons@infradead.org>

Pieces needed for UMP platforms was removed which can currently
crash a node. The first promblem was cfs_cpt_table_alloc() and
cfs_cpt_table_free() was completely missing for UMP platforms.
Since only one valid configuration exist just report the default
static cfs_cpt_tab. Don't return NULL for cfs_cpt_cpumask() and
cfs_cpt_nodemask() but real mask instead.

WC-bug-id: https://jira.whamcloud.com/browse/LU-9856
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 .../lustre/include/linux/libcfs/libcfs_cpu.h       | 41 ++++++++++++++++++++--
 1 file changed, 38 insertions(+), 3 deletions(-)

diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h
index df7e16b..ff1a24d 100644
--- a/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h
+++ b/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h
@@ -94,7 +94,6 @@ struct cfs_cpu_partition {
 	int				cpt_node;
 };
 
-
 /** descriptor for CPU partitions */
 struct cfs_cpt_table {
 	/* spread rotor for NUMA allocator */
@@ -114,9 +113,27 @@ struct cfs_cpt_table {
 	/* all nodes in this partition table */
 	nodemask_t			*ctb_nodemask;
 };
+#else /* !CONFIG_SMP */
+
+/** UMP descriptor for CPU partitions */
+struct cfs_cpt_table {
+	cpumask_var_t			ctb_cpumask;
+	nodemask_t			ctb_nodemask;
+};
+
+#endif /* CONFIG_SMP */
 
 extern struct cfs_cpt_table	cfs_cpt_tab;
 
+#ifdef CONFIG_SMP
+/**
+ * create a cfs_cpt_table with \a ncpt number of partitions
+ */
+struct cfs_cpt_table *cfs_cpt_table_alloc(int ncpt);
+/**
+ * destroy a CPU partition table
+ */
+void cfs_cpt_table_free(struct cfs_cpt_table *cptab);
 /**
  * return cpumask of CPU partition \a cpt
  */
@@ -216,6 +233,18 @@ void cfs_cpt_unset_nodemask(struct cfs_cpt_table *cptab,
 
 #else /* !CONFIG_SMP */
 
+static inline struct cfs_cpt_table *cfs_cpt_table_alloc(int ncpt)
+{
+	if (ncpt != 1)
+                return NULL;
+
+	return &cfs_cpt_tab;
+}
+
+static inline void cfs_cpt_table_free(struct cfs_cpt_table *cptab)
+{
+}
+
 static inline int cfs_cpt_table_print(struct cfs_cpt_table *cptab,
 				      char *buf, int len)
 {
@@ -245,7 +274,7 @@ static inline int cfs_cpt_distance_print(struct cfs_cpt_table *cptab,
 static inline cpumask_var_t *cfs_cpt_cpumask(struct cfs_cpt_table *cptab,
 					     int cpt)
 {
-	return NULL;
+	return &cptab->ctb_cpumask;
 }
 
 static inline int cfs_cpt_number(struct cfs_cpt_table *cptab)
@@ -266,7 +295,7 @@ static inline int cfs_cpt_online(struct cfs_cpt_table *cptab, int cpt)
 static inline nodemask_t *cfs_cpt_nodemask(struct cfs_cpt_table *cptab,
 					   int cpt)
 {
-	return NULL;
+	return &cptab->ctb_nodemask;
 }
 
 static inline unsigned int cfs_cpt_distance(struct cfs_cpt_table *cptab,
@@ -346,11 +375,17 @@ static inline int cfs_cpt_bind(struct cfs_cpt_table *cptab, int cpt)
 
 static inline int cfs_cpu_init(void)
 {
+	if (!zalloc_cpumask_var(&cfs_cpt_tab.ctb_cpumask, GFP_NOFS))
+		return -ENOMEM;
+
+	cpumask_set_cpu(0, cfs_cpt_tab.ctb_cpumask);
+	node_set(0, cfs_cpt_tab.ctb_nodemask);
 	return 0;
 }
 
 static inline void cfs_cpu_fini(void)
 {
+	free_cpumask_var(cfs_cpt_tab.ctb_cpumask);
 }
 
 #endif /* CONFIG_SMP */
-- 
1.8.3.1


From jsimmons at infradead.org  Sun Jun 24 21:20:28 2018
From: jsimmons at infradead.org (James Simmons)
Date: Sun, 24 Jun 2018 17:20:28 -0400
Subject: [lustre-devel] [PATCH v3 04/26] staging: lustre: libcfs: replace
 MAX_NUMNODES with nr_node_ids
In-Reply-To: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
References: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1529875250-11531-5-git-send-email-jsimmons@infradead.org>

From: Amir Shehata <amir.shehata at intel.com>

Replace MAX_NUMNODES which is considered deprocated with
nr_nodes_ids. Looking at page_malloc.c you will see that
nr_nodes_ids is equal to MAX_NUMNODES. MAX_NUMNODES is
actually setup with Kconfig.

Signed-off-by: Amir Shehata <amir.shehata at intel.com>
WC-bug-id: https://jira.whamcloud.com/browse/LU-7734
Reviewed-on: http://review.whamcloud.com/18916
Reviewed-by: Olaf Weber <olaf at sgi.com>
Reviewed-by: Doug Oucharek <dougso at me.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
index bdd71a3..ea8d55c 100644
--- a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
+++ b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
@@ -399,7 +399,7 @@ struct cfs_cpt_table *
 {
 	const cpumask_t *mask;
 
-	if (node < 0 || node >= MAX_NUMNODES) {
+	if (node < 0 || node >= nr_node_ids) {
 		CDEBUG(D_INFO,
 		       "Invalid NUMA id %d for CPU partition %d\n", node, cpt);
 		return 0;
@@ -416,7 +416,7 @@ struct cfs_cpt_table *
 {
 	const cpumask_t *mask;
 
-	if (node < 0 || node >= MAX_NUMNODES) {
+	if (node < 0 || node >= nr_node_ids) {
 		CDEBUG(D_INFO,
 		       "Invalid NUMA id %d for CPU partition %d\n", node, cpt);
 		return;
@@ -840,7 +840,7 @@ struct cfs_cpt_table *
 		return cptab;
 	}
 
-	high = node ? MAX_NUMNODES - 1 : nr_cpu_ids - 1;
+	high = node ? nr_node_ids - 1 : nr_cpu_ids - 1;
 
 	for (str = strim(pattern), c = 0;; c++) {
 		struct cfs_range_expr *range;
-- 
1.8.3.1


From jsimmons at infradead.org  Sun Jun 24 21:20:39 2018
From: jsimmons at infradead.org (James Simmons)
Date: Sun, 24 Jun 2018 17:20:39 -0400
Subject: [lustre-devel] [PATCH v3 15/26] staging: lustre: libcfs: rename i
 to cpu for cfs_cpt_bind
In-Reply-To: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
References: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1529875250-11531-16-git-send-email-jsimmons@infradead.org>

From: Dmitry Eremin <dmitry.eremin at intel.com>

Rename variable i to cpu to make code easier to understand.

Signed-off-by: Dmitry Eremin <dmitry.eremin at intel.com>
WC-bug-id: https://jira.whamcloud.com/browse/LU-8703
Reviewed-on: https://review.whamcloud.com/23222
Reviewed-by: Amir Shehata <amir.shehata at intel.com>
Reviewed-by: James Simmons <uja.ornl at yahoo.com>
Reviewed-by: Oleg Drokin <green at whamcloud.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
index a2c1068..5654fbe 100644
--- a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
+++ b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
@@ -673,8 +673,8 @@ int cfs_cpt_bind(struct cfs_cpt_table *cptab, int cpt)
 {
 	cpumask_var_t *cpumask;
 	nodemask_t *nodemask;
+	int cpu;
 	int rc;
-	int i;
 
 	LASSERT(cpt == CFS_CPT_ANY || (cpt >= 0 && cpt < cptab->ctb_nparts));
 
@@ -692,8 +692,8 @@ int cfs_cpt_bind(struct cfs_cpt_table *cptab, int cpt)
 		return -EINVAL;
 	}
 
-	for_each_online_cpu(i) {
-		if (cpumask_test_cpu(i, *cpumask))
+	for_each_online_cpu(cpu) {
+		if (cpumask_test_cpu(cpu, *cpumask))
 			continue;
 
 		rc = set_cpus_allowed_ptr(current, *cpumask);
-- 
1.8.3.1


From jsimmons at infradead.org  Sun Jun 24 21:20:48 2018
From: jsimmons at infradead.org (James Simmons)
Date: Sun, 24 Jun 2018 17:20:48 -0400
Subject: [lustre-devel] [PATCH v3 24/26] staging: lustre: libcfs: restore
 debugfs table reporting for UMP
In-Reply-To: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
References: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1529875250-11531-25-git-send-email-jsimmons@infradead.org>

With the cleanup of the libcfs SMP handling the function
cfs_cpt_table_print() was turned into an empty funciton.
This function is called by a debugfs reporting function for
debugging which now means for UMP machines it reports nothing
which breaks previous behavior exposed to users. Restore the
original behavior.

WC-bug-id: https://jira.whamcloud.com/browse/LU-9856
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 .../staging/lustre/include/linux/libcfs/libcfs_cpu.h  | 19 +++++++++++++------
 1 file changed, 13 insertions(+), 6 deletions(-)

diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h
index 29c5071..32776d2 100644
--- a/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h
+++ b/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h
@@ -218,6 +218,19 @@ void cfs_cpt_unset_nodemask(struct cfs_cpt_table *cptab,
 struct cfs_cpt_table;
 #define cfs_cpt_tab ((struct cfs_cpt_table *)NULL)
 
+static inline int cfs_cpt_table_print(struct cfs_cpt_table *cptab,
+				      char *buf, int len)
+{
+	int rc;
+
+	rc = snprintf(buf, len, "0\t: 0\n");
+	len -= rc;
+	if (len <= 0)
+		return -EFBIG;
+
+	return rc;
+}
+
 static inline int cfs_cpt_distance_print(struct cfs_cpt_table *cptab,
 					 char *buf, int len)
 {
@@ -237,12 +250,6 @@ static inline cpumask_var_t *cfs_cpt_cpumask(struct cfs_cpt_table *cptab,
 	return NULL;
 }
 
-static inline int cfs_cpt_table_print(struct cfs_cpt_table *cptab, char *buf,
-				      int len)
-{
-	return 0;
-}
-
 static inline int cfs_cpt_number(struct cfs_cpt_table *cptab)
 {
 	return 1;
-- 
1.8.3.1


From jsimmons at infradead.org  Sun Jun 24 21:20:38 2018
From: jsimmons at infradead.org (James Simmons)
Date: Sun, 24 Jun 2018 17:20:38 -0400
Subject: [lustre-devel] [PATCH v3 14/26] staging: lustre: libcfs: rename i
 to node for cfs_cpt_set_nodemask
In-Reply-To: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
References: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1529875250-11531-15-git-send-email-jsimmons@infradead.org>

From: Dmitry Eremin <dmitry.eremin at intel.com>

Rename variable i to node to make code easier to understand.

Signed-off-by: Dmitry Eremin <dmitry.eremin at intel.com>
WC-bug-id: https://jira.whamcloud.com/browse/LU-8703
Reviewed-on: https://review.whamcloud.com/23222
Reviewed-by: Amir Shehata <amir.shehata at intel.com>
Reviewed-by: James Simmons <uja.ornl at yahoo.com>
Reviewed-by: Oleg Drokin <green at whamcloud.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
index 8f7de59..a2c1068 100644
--- a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
+++ b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
@@ -579,10 +579,10 @@ void cfs_cpt_unset_node(struct cfs_cpt_table *cptab, int cpt, int node)
 int cfs_cpt_set_nodemask(struct cfs_cpt_table *cptab, int cpt,
 			 const nodemask_t *mask)
 {
-	int i;
+	int node;
 
-	for_each_node_mask(i, *mask) {
-		if (!cfs_cpt_set_node(cptab, cpt, i))
+	for_each_node_mask(node, *mask) {
+		if (!cfs_cpt_set_node(cptab, cpt, node))
 			return 0;
 	}
 
@@ -593,10 +593,10 @@ int cfs_cpt_set_nodemask(struct cfs_cpt_table *cptab, int cpt,
 void cfs_cpt_unset_nodemask(struct cfs_cpt_table *cptab, int cpt,
 			    const nodemask_t *mask)
 {
-	int i;
+	int node;
 
-	for_each_node_mask(i, *mask)
-		cfs_cpt_unset_node(cptab, cpt, i);
+	for_each_node_mask(node, *mask)
+		cfs_cpt_unset_node(cptab, cpt, node);
 }
 EXPORT_SYMBOL(cfs_cpt_unset_nodemask);
 
-- 
1.8.3.1


From jsimmons at infradead.org  Sun Jun 24 21:20:43 2018
From: jsimmons at infradead.org (James Simmons)
Date: Sun, 24 Jun 2018 17:20:43 -0400
Subject: [lustre-devel] [PATCH v3 19/26] staging: lustre: libcfs: report
 NUMA node instead of just node
In-Reply-To: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
References: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1529875250-11531-20-git-send-email-jsimmons@infradead.org>

From: Dmitry Eremin <dmitry.eremin at intel.com>

Reporting "HW nodes" is too generic. It really is reporting
"HW NUMA nodes". Update the debug message.

Signed-off-by: Dmitry Eremin <dmitry.eremin at intel.com>
WC-bug-id: https://jira.whamcloud.com/browse/LU-8703
Reviewed-on: https://review.whamcloud.com/23306
Reviewed-by: James Simmons <uja.ornl at yahoo.com>
Reviewed-by: Andreas Dilger <adilger at whamcloud.com>
Reviewed-by: Patrick Farrell <paf at cray.com>
Reviewed-by: Olaf Weber <olaf.weber at hpe.com>
Reviewed-by: Oleg Drokin <green at whamcloud.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h | 2 ++
 drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c          | 2 +-
 drivers/staging/lustre/lnet/lnet/lib-msg.c               | 2 ++
 3 files changed, 5 insertions(+), 1 deletion(-)

diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h
index 2bb2140..29c5071 100644
--- a/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h
+++ b/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h
@@ -90,6 +90,8 @@ struct cfs_cpu_partition {
 	unsigned int			*cpt_distance;
 	/* spread rotor for NUMA allocator */
 	int				cpt_spread_rotor;
+	/* NUMA node if cpt_nodemask is empty */
+	int				cpt_node;
 };
 
 
diff --git a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
index 18925c7..86afa31 100644
--- a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
+++ b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
@@ -1142,7 +1142,7 @@ int cfs_cpu_init(void)
 
 	put_online_cpus();
 
-	LCONSOLE(0, "HW nodes: %d, HW CPU cores: %d, npartitions: %d\n",
+	LCONSOLE(0, "HW NUMA nodes: %d, HW CPU cores: %d, npartitions: %d\n",
 		 num_online_nodes(), num_online_cpus(),
 		 cfs_cpt_number(cfs_cpt_tab));
 	return 0;
diff --git a/drivers/staging/lustre/lnet/lnet/lib-msg.c b/drivers/staging/lustre/lnet/lnet/lib-msg.c
index 0091273..27bdefa 100644
--- a/drivers/staging/lustre/lnet/lnet/lib-msg.c
+++ b/drivers/staging/lustre/lnet/lnet/lib-msg.c
@@ -568,6 +568,8 @@
 
 	/* number of CPUs */
 	container->msc_nfinalizers = cfs_cpt_weight(lnet_cpt_table(), cpt);
+	if (container->msc_nfinalizers == 0)
+		container->msc_nfinalizers = 1;
 
 	container->msc_finalizers = kvzalloc_cpt(container->msc_nfinalizers *
 						 sizeof(*container->msc_finalizers),
-- 
1.8.3.1


From jsimmons at infradead.org  Sun Jun 24 21:20:49 2018
From: jsimmons at infradead.org (James Simmons)
Date: Sun, 24 Jun 2018 17:20:49 -0400
Subject: [lustre-devel] [PATCH v3 25/26] staging: lustre: libcfs: make
 cfs_cpt_tab a static structure
In-Reply-To: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
References: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1529875250-11531-26-git-send-email-jsimmons@infradead.org>

Only one cfs_cpt_tab exist and its created only at libcfs modules
loading and removal. Instead of dynamically allocating it lets
statically allocate it. This will help to reenable UMP support.

WC-bug-id: https://jira.whamcloud.com/browse/LU-9856
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 .../lustre/include/linux/libcfs/libcfs_cpu.h       |   4 +-
 drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c    | 111 ++++++++++-----------
 drivers/staging/lustre/lnet/libcfs/module.c        |  10 +-
 drivers/staging/lustre/lnet/lnet/api-ni.c          |   4 +-
 drivers/staging/lustre/lnet/selftest/framework.c   |   2 +-
 drivers/staging/lustre/lustre/ptlrpc/client.c      |   4 +-
 drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c     |  10 +-
 drivers/staging/lustre/lustre/ptlrpc/service.c     |   4 +-
 8 files changed, 73 insertions(+), 76 deletions(-)

diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h
index 32776d2..df7e16b 100644
--- a/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h
+++ b/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h
@@ -115,7 +115,7 @@ struct cfs_cpt_table {
 	nodemask_t			*ctb_nodemask;
 };
 
-extern struct cfs_cpt_table	*cfs_cpt_tab;
+extern struct cfs_cpt_table	cfs_cpt_tab;
 
 /**
  * return cpumask of CPU partition \a cpt
@@ -215,8 +215,6 @@ void cfs_cpt_unset_nodemask(struct cfs_cpt_table *cptab,
 void cfs_cpu_fini(void);
 
 #else /* !CONFIG_SMP */
-struct cfs_cpt_table;
-#define cfs_cpt_tab ((struct cfs_cpt_table *)NULL)
 
 static inline int cfs_cpt_table_print(struct cfs_cpt_table *cptab,
 				      char *buf, int len)
diff --git a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
index 3f4a7c7..9fd324d 100644
--- a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
+++ b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
@@ -41,10 +41,6 @@
 #include <linux/libcfs/libcfs_string.h>
 #include <linux/libcfs/libcfs.h>
 
-/** Global CPU partition table */
-struct cfs_cpt_table   *cfs_cpt_tab __read_mostly;
-EXPORT_SYMBOL(cfs_cpt_tab);
-
 /**
  * modparam for setting number of partitions
  *
@@ -73,15 +69,10 @@
 module_param(cpu_pattern, charp, 0444);
 MODULE_PARM_DESC(cpu_pattern, "CPU partitions pattern");
 
-struct cfs_cpt_table *cfs_cpt_table_alloc(int ncpt)
+static int cfs_cpt_table_setup(struct cfs_cpt_table *cptab, int ncpt)
 {
-	struct cfs_cpt_table *cptab;
 	int i;
 
-	cptab = kzalloc(sizeof(*cptab), GFP_NOFS);
-	if (!cptab)
-		return NULL;
-
 	cptab->ctb_nparts = ncpt;
 
 	if (!zalloc_cpumask_var(&cptab->ctb_cpumask, GFP_NOFS))
@@ -138,7 +129,7 @@ struct cfs_cpt_table *cfs_cpt_table_alloc(int ncpt)
 		       cptab->ctb_nparts * sizeof(part->cpt_distance[0]));
 	}
 
-	return cptab;
+	return 0;
 
 failed_setting_ctb_parts:
 	while (i-- >= 0) {
@@ -159,8 +150,24 @@ struct cfs_cpt_table *cfs_cpt_table_alloc(int ncpt)
 failed_alloc_nodemask:
 	free_cpumask_var(cptab->ctb_cpumask);
 failed_alloc_cpumask:
-	kfree(cptab);
-	return NULL;
+	return -ENOMEM;
+}
+
+struct cfs_cpt_table *cfs_cpt_table_alloc(int ncpt)
+{
+	struct cfs_cpt_table *cptab;
+	int rc;
+
+	cptab = kzalloc(sizeof(*cptab), GFP_NOFS);
+	if (!cptab)
+		return NULL;
+
+	rc = cfs_cpt_table_setup(cptab, ncpt);
+	if (rc) {
+		kfree(cptab);
+		cptab = NULL;
+	}
+	return cptab;
 }
 EXPORT_SYMBOL(cfs_cpt_table_alloc);
 
@@ -183,8 +190,6 @@ void cfs_cpt_table_free(struct cfs_cpt_table *cptab)
 
 	kfree(cptab->ctb_nodemask);
 	free_cpumask_var(cptab->ctb_cpumask);
-
-	kfree(cptab);
 }
 EXPORT_SYMBOL(cfs_cpt_table_free);
 
@@ -822,9 +827,8 @@ static int cfs_cpt_num_estimate(void)
 	return ncpt;
 }
 
-static struct cfs_cpt_table *cfs_cpt_table_create(int ncpt)
+static int cfs_cpt_table_create(int ncpt)
 {
-	struct cfs_cpt_table *cptab = NULL;
 	cpumask_var_t node_mask;
 	int cpt = 0;
 	int node;
@@ -841,10 +845,9 @@ static struct cfs_cpt_table *cfs_cpt_table_create(int ncpt)
 		      ncpt, num);
 	}
 
-	cptab = cfs_cpt_table_alloc(ncpt);
-	if (!cptab) {
-		CERROR("Failed to allocate CPU map(%d)\n", ncpt);
-		rc = -ENOMEM;
+	rc = cfs_cpt_table_setup(&cfs_cpt_tab, ncpt);
+	if (rc) {
+		CERROR("Failed to setup CPU map(%d)\n", ncpt);
 		goto failed;
 	}
 
@@ -860,10 +863,13 @@ static struct cfs_cpt_table *cfs_cpt_table_create(int ncpt)
 		cpumask_copy(node_mask, cpumask_of_node(node));
 
 		while (cpt < ncpt && !cpumask_empty(node_mask)) {
-			struct cfs_cpu_partition *part = &cptab->ctb_parts[cpt];
-			int ncpu = cpumask_weight(part->cpt_cpumask);
+			struct cfs_cpu_partition *part;
+			int ncpu;
+
+			part = &cfs_cpt_tab.ctb_parts[cpt];
+			ncpu = cpumask_weight(part->cpt_cpumask);
 
-			rc = cfs_cpt_choose_ncpus(cptab, cpt, node_mask,
+			rc = cfs_cpt_choose_ncpus(&cfs_cpt_tab, cpt, node_mask,
 						  num - ncpu);
 			if (rc < 0) {
 				rc = -EINVAL;
@@ -880,7 +886,7 @@ static struct cfs_cpt_table *cfs_cpt_table_create(int ncpt)
 
 	free_cpumask_var(node_mask);
 
-	return cptab;
+	return 0;
 
 failed_mask:
 	free_cpumask_var(node_mask);
@@ -888,15 +894,13 @@ static struct cfs_cpt_table *cfs_cpt_table_create(int ncpt)
 	CERROR("Failed (rc = %d) to setup CPU partition table with %d partitions, online HW NUMA nodes: %d, HW CPU cores: %d.\n",
 	       rc, ncpt, num_online_nodes(), num_online_cpus());
 
-	if (cptab)
-		cfs_cpt_table_free(cptab);
+	cfs_cpt_table_free(&cfs_cpt_tab);
 
-	return ERR_PTR(rc);
+	return rc;
 }
 
-static struct cfs_cpt_table *cfs_cpt_table_create_pattern(const char *pattern)
+static int cfs_cpt_table_create_pattern(const char *pattern)
 {
-	struct cfs_cpt_table *cptab;
 	char *pattern_dup;
 	char *bracket;
 	char *str;
@@ -911,7 +915,7 @@ static struct cfs_cpt_table *cfs_cpt_table_create_pattern(const char *pattern)
 	pattern_dup = kstrdup(pattern, GFP_KERNEL);
 	if (!pattern_dup) {
 		CERROR("Failed to duplicate pattern '%s'\n", pattern);
-		return ERR_PTR(-ENOMEM);
+		return -ENOMEM;
 	}
 
 	str = strim(pattern_dup);
@@ -948,10 +952,9 @@ static struct cfs_cpt_table *cfs_cpt_table_create_pattern(const char *pattern)
 		goto err_free_str;
 	}
 
-	cptab = cfs_cpt_table_alloc(ncpt);
-	if (!cptab) {
-		CERROR("Failed to allocate CPU partition table\n");
-		rc = -ENOMEM;
+	rc = cfs_cpt_table_setup(&cfs_cpt_tab, ncpt);
+	if (rc) {
+		CERROR("Failed to setup CPU partition table\n");
 		goto err_free_str;
 	}
 
@@ -960,14 +963,14 @@ static struct cfs_cpt_table *cfs_cpt_table_create_pattern(const char *pattern)
 			if (cpumask_empty(cpumask_of_node(i)))
 				continue;
 
-			rc = cfs_cpt_set_node(cptab, cpt++, i);
+			rc = cfs_cpt_set_node(&cfs_cpt_tab, cpt++, i);
 			if (!rc) {
 				rc = -EINVAL;
 				goto err_free_table;
 			}
 		}
 		kfree(pattern_dup);
-		return cptab;
+		return 0;
 	}
 
 	high = node ? nr_node_ids - 1 : nr_cpu_ids - 1;
@@ -1006,7 +1009,7 @@ static struct cfs_cpt_table *cfs_cpt_table_create_pattern(const char *pattern)
 			goto err_free_table;
 		}
 
-		if (cfs_cpt_weight(cptab, cpt)) {
+		if (cfs_cpt_weight(&cfs_cpt_tab, cpt)) {
 			CERROR("Partition %d has already been set.\n", cpt);
 			rc = -EPERM;
 			goto err_free_table;
@@ -1040,8 +1043,8 @@ static struct cfs_cpt_table *cfs_cpt_table_create_pattern(const char *pattern)
 				if ((i - range->re_lo) % range->re_stride)
 					continue;
 
-				rc = node ? cfs_cpt_set_node(cptab, cpt, i) :
-					    cfs_cpt_set_cpu(cptab, cpt, i);
+				rc = node ? cfs_cpt_set_node(&cfs_cpt_tab, cpt, i) :
+					    cfs_cpt_set_cpu(&cfs_cpt_tab, cpt, i);
 				if (!rc) {
 					cfs_expr_list_free(el);
 					rc = -EINVAL;
@@ -1052,7 +1055,7 @@ static struct cfs_cpt_table *cfs_cpt_table_create_pattern(const char *pattern)
 
 		cfs_expr_list_free(el);
 
-		if (!cfs_cpt_online(cptab, cpt)) {
+		if (!cfs_cpt_online(&cfs_cpt_tab, cpt)) {
 			CERROR("No online CPU is found on partition %d\n", cpt);
 			rc = -ENODEV;
 			goto err_free_table;
@@ -1062,13 +1065,13 @@ static struct cfs_cpt_table *cfs_cpt_table_create_pattern(const char *pattern)
 	}
 
 	kfree(pattern_dup);
-	return cptab;
+	return 0;
 
 err_free_table:
-	cfs_cpt_table_free(cptab);
+	cfs_cpt_table_free(&cfs_cpt_tab);
 err_free_str:
 	kfree(pattern_dup);
-	return ERR_PTR(rc);
+	return rc;
 }
 
 #ifdef CONFIG_HOTPLUG_CPU
@@ -1095,8 +1098,7 @@ static int cfs_cpu_dead(unsigned int cpu)
 
 void cfs_cpu_fini(void)
 {
-	if (!IS_ERR_OR_NULL(cfs_cpt_tab))
-		cfs_cpt_table_free(cfs_cpt_tab);
+	cfs_cpt_table_free(&cfs_cpt_tab);
 
 #ifdef CONFIG_HOTPLUG_CPU
 	if (lustre_cpu_online > 0)
@@ -1109,8 +1111,6 @@ int cfs_cpu_init(void)
 {
 	int ret;
 
-	LASSERT(!cfs_cpt_tab);
-
 #ifdef CONFIG_HOTPLUG_CPU
 	ret = cpuhp_setup_state_nocalls(CPUHP_LUSTRE_CFS_DEAD,
 					"staging/lustre/cfe:dead", NULL,
@@ -1128,20 +1128,18 @@ int cfs_cpu_init(void)
 #endif
 	get_online_cpus();
 	if (*cpu_pattern) {
-		cfs_cpt_tab = cfs_cpt_table_create_pattern(cpu_pattern);
-		if (IS_ERR(cfs_cpt_tab)) {
+		ret = cfs_cpt_table_create_pattern(cpu_pattern);
+		if (ret) {
 			CERROR("Failed to create cptab from pattern '%s'\n",
 			       cpu_pattern);
-			ret = PTR_ERR(cfs_cpt_tab);
 			goto failed_alloc_table;
 		}
 
 	} else {
-		cfs_cpt_tab = cfs_cpt_table_create(cpu_npartitions);
-		if (IS_ERR(cfs_cpt_tab)) {
+		ret = cfs_cpt_table_create(cpu_npartitions);
+		if (ret) {
 			CERROR("Failed to create cptab with npartitions %d\n",
 			       cpu_npartitions);
-			ret = PTR_ERR(cfs_cpt_tab);
 			goto failed_alloc_table;
 		}
 	}
@@ -1150,14 +1148,13 @@ int cfs_cpu_init(void)
 
 	LCONSOLE(0, "HW NUMA nodes: %d, HW CPU cores: %d, npartitions: %d\n",
 		 num_online_nodes(), num_online_cpus(),
-		 cfs_cpt_number(cfs_cpt_tab));
+		 cfs_cpt_number(&cfs_cpt_tab));
 	return 0;
 
 failed_alloc_table:
 	put_online_cpus();
 
-	if (!IS_ERR_OR_NULL(cfs_cpt_tab))
-		cfs_cpt_table_free(cfs_cpt_tab);
+	cfs_cpt_table_free(&cfs_cpt_tab);
 
 	ret = -EINVAL;
 #ifdef CONFIG_HOTPLUG_CPU
diff --git a/drivers/staging/lustre/lnet/libcfs/module.c b/drivers/staging/lustre/lnet/libcfs/module.c
index 2281f08..35c3959 100644
--- a/drivers/staging/lustre/lnet/libcfs/module.c
+++ b/drivers/staging/lustre/lnet/libcfs/module.c
@@ -66,6 +66,10 @@ struct lnet_debugfs_symlink_def {
 
 static struct dentry *lnet_debugfs_root;
 
+/** Global CPU partition table */
+struct cfs_cpt_table cfs_cpt_tab __read_mostly;
+EXPORT_SYMBOL(cfs_cpt_tab);
+
 BLOCKING_NOTIFIER_HEAD(libcfs_ioctl_list);
 EXPORT_SYMBOL(libcfs_ioctl_list);
 
@@ -402,7 +406,7 @@ static int proc_cpt_table(struct ctl_table *table, int write,
 		if (!buf)
 			return -ENOMEM;
 
-		rc = cfs_cpt_table_print(cfs_cpt_tab, buf, len);
+		rc = cfs_cpt_table_print(&cfs_cpt_tab, buf, len);
 		if (rc >= 0)
 			break;
 
@@ -437,14 +441,12 @@ static int proc_cpt_distance(struct ctl_table *table, int write,
 	if (write)
 		return -EPERM;
 
-	LASSERT(cfs_cpt_tab);
-
 	while (1) {
 		buf = kzalloc(len, GFP_KERNEL);
 		if (!buf)
 			return -ENOMEM;
 
-		rc = cfs_cpt_distance_print(cfs_cpt_tab, buf, len);
+		rc = cfs_cpt_distance_print(&cfs_cpt_tab, buf, len);
 		if (rc >= 0)
 			break;
 
diff --git a/drivers/staging/lustre/lnet/lnet/api-ni.c b/drivers/staging/lustre/lnet/lnet/api-ni.c
index f9ed697..98a4942 100644
--- a/drivers/staging/lustre/lnet/lnet/api-ni.c
+++ b/drivers/staging/lustre/lnet/lnet/api-ni.c
@@ -1414,8 +1414,8 @@ int lnet_lib_init(void)
 	memset(&the_lnet, 0, sizeof(the_lnet));
 
 	/* refer to global cfs_cpt_tab for now */
-	the_lnet.ln_cpt_table	= cfs_cpt_tab;
-	the_lnet.ln_cpt_number	= cfs_cpt_number(cfs_cpt_tab);
+	the_lnet.ln_cpt_table = &cfs_cpt_tab;
+	the_lnet.ln_cpt_number = cfs_cpt_number(&cfs_cpt_tab);
 
 	LASSERT(the_lnet.ln_cpt_number > 0);
 	if (the_lnet.ln_cpt_number > LNET_CPT_MAX) {
diff --git a/drivers/staging/lustre/lnet/selftest/framework.c b/drivers/staging/lustre/lnet/selftest/framework.c
index 741af10..939b7ec 100644
--- a/drivers/staging/lustre/lnet/selftest/framework.c
+++ b/drivers/staging/lustre/lnet/selftest/framework.c
@@ -588,7 +588,7 @@
 
 	CDEBUG(D_NET, "Reserved %d buffers for test %s\n",
 	       nbuf * (srpc_serv_is_framework(svc) ?
-		       2 : cfs_cpt_number(cfs_cpt_tab)), svc->sv_name);
+		       2 : cfs_cpt_number(&cfs_cpt_tab)), svc->sv_name);
 	return 0;
 }
 
diff --git a/drivers/staging/lustre/lustre/ptlrpc/client.c b/drivers/staging/lustre/lustre/ptlrpc/client.c
index c1b82bf..c569a8b 100644
--- a/drivers/staging/lustre/lustre/ptlrpc/client.c
+++ b/drivers/staging/lustre/lustre/ptlrpc/client.c
@@ -940,9 +940,9 @@ struct ptlrpc_request_set *ptlrpc_prep_set(void)
 	struct ptlrpc_request_set *set;
 	int cpt;
 
-	cpt = cfs_cpt_current(cfs_cpt_tab, 0);
+	cpt = cfs_cpt_current(&cfs_cpt_tab, 0);
 	set = kzalloc_node(sizeof(*set), GFP_NOFS,
-			   cfs_cpt_spread_node(cfs_cpt_tab, cpt));
+			   cfs_cpt_spread_node(&cfs_cpt_tab, cpt));
 	if (!set)
 		return NULL;
 	atomic_set(&set->set_refcount, 1);
diff --git a/drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c b/drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c
index 5310054..d496521 100644
--- a/drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c
+++ b/drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c
@@ -177,7 +177,7 @@ void ptlrpcd_wake(struct ptlrpc_request *req)
 	if (req && req->rq_send_state != LUSTRE_IMP_FULL)
 		return &ptlrpcd_rcv;
 
-	cpt = cfs_cpt_current(cfs_cpt_tab, 1);
+	cpt = cfs_cpt_current(&cfs_cpt_tab, 1);
 	if (!ptlrpcds_cpt_idx)
 		idx = cpt;
 	else
@@ -389,7 +389,7 @@ static int ptlrpcd(void *arg)
 	int exit = 0;
 
 	unshare_fs_struct();
-	if (cfs_cpt_bind(cfs_cpt_tab, pc->pc_cpt) != 0)
+	if (cfs_cpt_bind(&cfs_cpt_tab, pc->pc_cpt) != 0)
 		CWARN("Failed to bind %s on CPT %d\n", pc->pc_name, pc->pc_cpt);
 
 	/*
@@ -531,7 +531,7 @@ static int ptlrpcd_partners(struct ptlrpcd *pd, int index)
 
 	size = sizeof(struct ptlrpcd_ctl *) * pc->pc_npartners;
 	pc->pc_partners = kzalloc_node(size, GFP_NOFS,
-				       cfs_cpt_spread_node(cfs_cpt_tab,
+				       cfs_cpt_spread_node(&cfs_cpt_tab,
 							   pc->pc_cpt));
 	if (!pc->pc_partners) {
 		pc->pc_npartners = 0;
@@ -677,7 +677,7 @@ static int ptlrpcd_init(void)
 	/*
 	 * Determine the CPTs that ptlrpcd threads will run on.
 	 */
-	cptable = cfs_cpt_tab;
+	cptable = &cfs_cpt_tab;
 	ncpts = cfs_cpt_number(cptable);
 	if (ptlrpcd_cpts) {
 		struct cfs_expr_list *el;
@@ -831,7 +831,7 @@ static int ptlrpcd_init(void)
 
 		size = offsetof(struct ptlrpcd, pd_threads[nthreads]);
 		pd = kzalloc_node(size, GFP_NOFS,
-				  cfs_cpt_spread_node(cfs_cpt_tab, cpt));
+				  cfs_cpt_spread_node(&cfs_cpt_tab, cpt));
 		if (!pd) {
 			rc = -ENOMEM;
 			goto out;
diff --git a/drivers/staging/lustre/lustre/ptlrpc/service.c b/drivers/staging/lustre/lustre/ptlrpc/service.c
index 8e74a45..853676f 100644
--- a/drivers/staging/lustre/lustre/ptlrpc/service.c
+++ b/drivers/staging/lustre/lustre/ptlrpc/service.c
@@ -565,7 +565,7 @@ struct ptlrpc_service *
 
 	cptable = cconf->cc_cptable;
 	if (!cptable)
-		cptable = cfs_cpt_tab;
+		cptable = &cfs_cpt_tab;
 
 	if (!conf->psc_thr.tc_cpu_affinity) {
 		ncpts = 1;
@@ -2520,7 +2520,7 @@ int ptlrpc_hr_init(void)
 	int weight;
 
 	memset(&ptlrpc_hr, 0, sizeof(ptlrpc_hr));
-	ptlrpc_hr.hr_cpt_table = cfs_cpt_tab;
+	ptlrpc_hr.hr_cpt_table = &cfs_cpt_tab;
 
 	ptlrpc_hr.hr_partitions = cfs_percpt_alloc(ptlrpc_hr.hr_cpt_table,
 						   sizeof(*hrp));
-- 
1.8.3.1


From jsimmons at infradead.org  Sun Jun 24 21:20:25 2018
From: jsimmons at infradead.org (James Simmons)
Date: Sun, 24 Jun 2018 17:20:25 -0400
Subject: [lustre-devel] [PATCH v3 01/26] staging: lustre: libcfs: remove
 useless CPU partition code
In-Reply-To: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
References: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1529875250-11531-2-git-send-email-jsimmons@infradead.org>

From: Dmitry Eremin <dmitry.eremin at intel.com>

* remove scratch buffer and mutex which guard it.
* remove global cpumask and spinlock which guard it.
* remove cpt_version for checking CPUs state change during setup
  because of just disable CPUs state change during setup.
* remove whole global struct cfs_cpt_data cpt_data.
* remove few unused APIs.

Signed-off-by: Dmitry Eremin <dmitry.eremin at intel.com>
WC-bug-id: https://jira.whamcloud.com/browse/LU-8703
Reviewed-on: https://review.whamcloud.com/23303
Reviewed-on: https://review.whamcloud.com/25048
Reviewed-by: James Simmons <uja.ornl at yahoo.com>
Reviewed-by: Doug Oucharek <dougso at me.com>
Reviewed-by: Andreas Dilger <adilger at whamcloud.com>
Reviewed-by: Olaf Weber <olaf at sgi.com>
Reviewed-by: Oleg Drokin <green at whamcloud.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 .../lustre/include/linux/libcfs/libcfs_cpu.h       |  32 ++----
 drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c    | 115 +++------------------
 2 files changed, 22 insertions(+), 125 deletions(-)

diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h
index 61641c4..1b4333d 100644
--- a/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h
+++ b/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h
@@ -93,8 +93,6 @@ struct cfs_cpu_partition {
 
 /** descriptor for CPU partitions */
 struct cfs_cpt_table {
-	/* version, reserved for hotplug */
-	unsigned int			ctb_version;
 	/* spread rotor for NUMA allocator */
 	unsigned int			ctb_spread_rotor;
 	/* # of CPU partitions */
@@ -162,12 +160,12 @@ struct cfs_cpt_table {
  * return 1 if successfully set all CPUs, otherwise return 0
  */
 int cfs_cpt_set_cpumask(struct cfs_cpt_table *cptab,
-			int cpt, cpumask_t *mask);
+			int cpt, const cpumask_t *mask);
 /**
  * remove all cpus in \a mask from CPU partition \a cpt
  */
 void cfs_cpt_unset_cpumask(struct cfs_cpt_table *cptab,
-			   int cpt, cpumask_t *mask);
+			   int cpt, const cpumask_t *mask);
 /**
  * add all cpus in NUMA node \a node to CPU partition \a cpt
  * return 1 if successfully set all CPUs, otherwise return 0
@@ -190,20 +188,11 @@ int cfs_cpt_set_nodemask(struct cfs_cpt_table *cptab,
 void cfs_cpt_unset_nodemask(struct cfs_cpt_table *cptab,
 			    int cpt, nodemask_t *mask);
 /**
- * unset all cpus for CPU partition \a cpt
- */
-void cfs_cpt_clear(struct cfs_cpt_table *cptab, int cpt);
-/**
  * convert partition id \a cpt to numa node id, if there are more than one
  * nodes in this partition, it might return a different node id each time.
  */
 int cfs_cpt_spread_node(struct cfs_cpt_table *cptab, int cpt);
 
-/**
- * return number of HTs in the same core of \a cpu
- */
-int cfs_cpu_ht_nsiblings(int cpu);
-
 int  cfs_cpu_init(void);
 void cfs_cpu_fini(void);
 
@@ -258,13 +247,15 @@ void cfs_cpt_unset_nodemask(struct cfs_cpt_table *cptab,
 }
 
 static inline int
-cfs_cpt_set_cpumask(struct cfs_cpt_table *cptab, int cpt, cpumask_t *mask)
+cfs_cpt_set_cpumask(struct cfs_cpt_table *cptab, int cpt,
+		    const cpumask_t *mask)
 {
 	return 1;
 }
 
 static inline void
-cfs_cpt_unset_cpumask(struct cfs_cpt_table *cptab, int cpt, cpumask_t *mask)
+cfs_cpt_unset_cpumask(struct cfs_cpt_table *cptab, int cpt,
+		      const cpumask_t *mask)
 {
 }
 
@@ -290,11 +281,6 @@ void cfs_cpt_unset_nodemask(struct cfs_cpt_table *cptab,
 {
 }
 
-static inline void
-cfs_cpt_clear(struct cfs_cpt_table *cptab, int cpt)
-{
-}
-
 static inline int
 cfs_cpt_spread_node(struct cfs_cpt_table *cptab, int cpt)
 {
@@ -302,12 +288,6 @@ void cfs_cpt_unset_nodemask(struct cfs_cpt_table *cptab,
 }
 
 static inline int
-cfs_cpu_ht_nsiblings(int cpu)
-{
-	return 1;
-}
-
-static inline int
 cfs_cpt_current(struct cfs_cpt_table *cptab, int remap)
 {
 	return 0;
diff --git a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
index 3d1cf45..b363a3d 100644
--- a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
+++ b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
@@ -73,19 +73,6 @@
 module_param(cpu_pattern, charp, 0444);
 MODULE_PARM_DESC(cpu_pattern, "CPU partitions pattern");
 
-static struct cfs_cpt_data {
-	/* serialize hotplug etc */
-	spinlock_t		cpt_lock;
-	/* reserved for hotplug */
-	unsigned long		cpt_version;
-	/* mutex to protect cpt_cpumask */
-	struct mutex		cpt_mutex;
-	/* scratch buffer for set/unset_node */
-	cpumask_var_t		cpt_cpumask;
-} cpt_data;
-
-#define CFS_CPU_VERSION_MAGIC	   0xbabecafe
-
 struct cfs_cpt_table *
 cfs_cpt_table_alloc(unsigned int ncpt)
 {
@@ -128,11 +115,6 @@ struct cfs_cpt_table *
 			goto failed;
 	}
 
-	spin_lock(&cpt_data.cpt_lock);
-	/* Reserved for hotplug */
-	cptab->ctb_version = cpt_data.cpt_version;
-	spin_unlock(&cpt_data.cpt_lock);
-
 	return cptab;
 
  failed:
@@ -207,17 +189,6 @@ struct cfs_cpt_table *
 }
 EXPORT_SYMBOL(cfs_cpt_table_print);
 
-static void
-cfs_node_to_cpumask(int node, cpumask_t *mask)
-{
-	const cpumask_t *tmp = cpumask_of_node(node);
-
-	if (tmp)
-		cpumask_copy(mask, tmp);
-	else
-		cpumask_clear(mask);
-}
-
 int
 cfs_cpt_number(struct cfs_cpt_table *cptab)
 {
@@ -370,7 +341,8 @@ struct cfs_cpt_table *
 EXPORT_SYMBOL(cfs_cpt_unset_cpu);
 
 int
-cfs_cpt_set_cpumask(struct cfs_cpt_table *cptab, int cpt, cpumask_t *mask)
+cfs_cpt_set_cpumask(struct cfs_cpt_table *cptab, int cpt,
+		    const cpumask_t *mask)
 {
 	int i;
 
@@ -391,7 +363,8 @@ struct cfs_cpt_table *
 EXPORT_SYMBOL(cfs_cpt_set_cpumask);
 
 void
-cfs_cpt_unset_cpumask(struct cfs_cpt_table *cptab, int cpt, cpumask_t *mask)
+cfs_cpt_unset_cpumask(struct cfs_cpt_table *cptab, int cpt,
+		      const cpumask_t *mask)
 {
 	int i;
 
@@ -403,7 +376,7 @@ struct cfs_cpt_table *
 int
 cfs_cpt_set_node(struct cfs_cpt_table *cptab, int cpt, int node)
 {
-	int rc;
+	const cpumask_t *mask;
 
 	if (node < 0 || node >= MAX_NUMNODES) {
 		CDEBUG(D_INFO,
@@ -411,34 +384,26 @@ struct cfs_cpt_table *
 		return 0;
 	}
 
-	mutex_lock(&cpt_data.cpt_mutex);
-
-	cfs_node_to_cpumask(node, cpt_data.cpt_cpumask);
-
-	rc = cfs_cpt_set_cpumask(cptab, cpt, cpt_data.cpt_cpumask);
+	mask = cpumask_of_node(node);
 
-	mutex_unlock(&cpt_data.cpt_mutex);
-
-	return rc;
+	return cfs_cpt_set_cpumask(cptab, cpt, mask);
 }
 EXPORT_SYMBOL(cfs_cpt_set_node);
 
 void
 cfs_cpt_unset_node(struct cfs_cpt_table *cptab, int cpt, int node)
 {
+	const cpumask_t *mask;
+
 	if (node < 0 || node >= MAX_NUMNODES) {
 		CDEBUG(D_INFO,
 		       "Invalid NUMA id %d for CPU partition %d\n", node, cpt);
 		return;
 	}
 
-	mutex_lock(&cpt_data.cpt_mutex);
-
-	cfs_node_to_cpumask(node, cpt_data.cpt_cpumask);
-
-	cfs_cpt_unset_cpumask(cptab, cpt, cpt_data.cpt_cpumask);
+	mask = cpumask_of_node(node);
 
-	mutex_unlock(&cpt_data.cpt_mutex);
+	cfs_cpt_unset_cpumask(cptab, cpt, mask);
 }
 EXPORT_SYMBOL(cfs_cpt_unset_node);
 
@@ -466,26 +431,6 @@ struct cfs_cpt_table *
 }
 EXPORT_SYMBOL(cfs_cpt_unset_nodemask);
 
-void
-cfs_cpt_clear(struct cfs_cpt_table *cptab, int cpt)
-{
-	int last;
-	int i;
-
-	if (cpt == CFS_CPT_ANY) {
-		last = cptab->ctb_nparts - 1;
-		cpt = 0;
-	} else {
-		last = cpt;
-	}
-
-	for (; cpt <= last; cpt++) {
-		for_each_cpu(i, cptab->ctb_parts[cpt].cpt_cpumask)
-			cfs_cpt_unset_cpu(cptab, cpt, i);
-	}
-}
-EXPORT_SYMBOL(cfs_cpt_clear);
-
 int
 cfs_cpt_spread_node(struct cfs_cpt_table *cptab, int cpt)
 {
@@ -758,7 +703,7 @@ struct cfs_cpt_table *
 	}
 
 	for_each_online_node(i) {
-		cfs_node_to_cpumask(i, mask);
+		cpumask_copy(mask, cpumask_of_node(i));
 
 		while (!cpumask_empty(mask)) {
 			struct cfs_cpu_partition *part;
@@ -964,16 +909,8 @@ struct cfs_cpt_table *
 #ifdef CONFIG_HOTPLUG_CPU
 static enum cpuhp_state lustre_cpu_online;
 
-static void cfs_cpu_incr_cpt_version(void)
-{
-	spin_lock(&cpt_data.cpt_lock);
-	cpt_data.cpt_version++;
-	spin_unlock(&cpt_data.cpt_lock);
-}
-
 static int cfs_cpu_online(unsigned int cpu)
 {
-	cfs_cpu_incr_cpt_version();
 	return 0;
 }
 
@@ -981,14 +918,9 @@ static int cfs_cpu_dead(unsigned int cpu)
 {
 	bool warn;
 
-	cfs_cpu_incr_cpt_version();
-
-	mutex_lock(&cpt_data.cpt_mutex);
 	/* if all HTs in a core are offline, it may break affinity */
-	cpumask_copy(cpt_data.cpt_cpumask, topology_sibling_cpumask(cpu));
-	warn = cpumask_any_and(cpt_data.cpt_cpumask,
+	warn = cpumask_any_and(topology_sibling_cpumask(cpu),
 			       cpu_online_mask) >= nr_cpu_ids;
-	mutex_unlock(&cpt_data.cpt_mutex);
 	CDEBUG(warn ? D_WARNING : D_INFO,
 	       "Lustre: can't support CPU plug-out well now, performance and stability could be impacted [CPU %u]\n",
 	       cpu);
@@ -1007,7 +939,6 @@ static int cfs_cpu_dead(unsigned int cpu)
 		cpuhp_remove_state_nocalls(lustre_cpu_online);
 	cpuhp_remove_state_nocalls(CPUHP_LUSTRE_CFS_DEAD);
 #endif
-	free_cpumask_var(cpt_data.cpt_cpumask);
 }
 
 int
@@ -1017,16 +948,6 @@ static int cfs_cpu_dead(unsigned int cpu)
 
 	LASSERT(!cfs_cpt_tab);
 
-	memset(&cpt_data, 0, sizeof(cpt_data));
-
-	if (!zalloc_cpumask_var(&cpt_data.cpt_cpumask, GFP_NOFS)) {
-		CERROR("Failed to allocate scratch buffer\n");
-		return -1;
-	}
-
-	spin_lock_init(&cpt_data.cpt_lock);
-	mutex_init(&cpt_data.cpt_mutex);
-
 #ifdef CONFIG_HOTPLUG_CPU
 	ret = cpuhp_setup_state_nocalls(CPUHP_LUSTRE_CFS_DEAD,
 					"staging/lustre/cfe:dead", NULL,
@@ -1042,6 +963,7 @@ static int cfs_cpu_dead(unsigned int cpu)
 #endif
 	ret = -EINVAL;
 
+	get_online_cpus();
 	if (*cpu_pattern) {
 		char *cpu_pattern_dup = kstrdup(cpu_pattern, GFP_KERNEL);
 
@@ -1067,13 +989,7 @@ static int cfs_cpu_dead(unsigned int cpu)
 		}
 	}
 
-	spin_lock(&cpt_data.cpt_lock);
-	if (cfs_cpt_tab->ctb_version != cpt_data.cpt_version) {
-		spin_unlock(&cpt_data.cpt_lock);
-		CERROR("CPU hotplug/unplug during setup\n");
-		goto failed;
-	}
-	spin_unlock(&cpt_data.cpt_lock);
+	put_online_cpus();
 
 	LCONSOLE(0, "HW nodes: %d, HW CPU cores: %d, npartitions: %d\n",
 		 num_online_nodes(), num_online_cpus(),
@@ -1081,6 +997,7 @@ static int cfs_cpu_dead(unsigned int cpu)
 	return 0;
 
  failed:
+	put_online_cpus();
 	cfs_cpu_fini();
 	return ret;
 }
-- 
1.8.3.1


From jsimmons at infradead.org  Sun Jun 24 21:20:47 2018
From: jsimmons at infradead.org (James Simmons)
Date: Sun, 24 Jun 2018 17:20:47 -0400
Subject: [lustre-devel] [PATCH v3 23/26] staging: lustre: ptlrpc: use
 current CPU instead of hardcoded 0
In-Reply-To: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
References: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1529875250-11531-24-git-send-email-jsimmons@infradead.org>

From: Dmitry Eremin <dmitry.eremin at intel.com>

fix crash if CPU 0 disabled.

Signed-off-by: Dmitry Eremin <dmitry.eremin at intel.com>
WC-bug-id: https://jira.whamcloud.com/browse/LU-8710
Reviewed-on: https://review.whamcloud.com/23305
Reviewed-by: Doug Oucharek <dougso at me.com>
Reviewed-by: Andreas Dilger <adilger at whamcloud.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/lustre/ptlrpc/service.c | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)

diff --git a/drivers/staging/lustre/lustre/ptlrpc/service.c b/drivers/staging/lustre/lustre/ptlrpc/service.c
index 3fd8c74..8e74a45 100644
--- a/drivers/staging/lustre/lustre/ptlrpc/service.c
+++ b/drivers/staging/lustre/lustre/ptlrpc/service.c
@@ -421,7 +421,7 @@ static void ptlrpc_at_timer(struct timer_list *t)
 		 * there are.
 		 */
 		/* weight is # of HTs */
-		if (cpumask_weight(topology_sibling_cpumask(0)) > 1) {
+		if (cpumask_weight(topology_sibling_cpumask(smp_processor_id())) > 1) {
 			/* depress thread factor for hyper-thread */
 			factor = factor - (factor >> 1) + (factor >> 3);
 		}
@@ -2221,15 +2221,16 @@ static int ptlrpc_hr_main(void *arg)
 	struct ptlrpc_hr_thread	*hrt = arg;
 	struct ptlrpc_hr_partition *hrp = hrt->hrt_partition;
 	LIST_HEAD(replies);
-	char threadname[20];
 	int rc;
 
-	snprintf(threadname, sizeof(threadname), "ptlrpc_hr%02d_%03d",
-		 hrp->hrp_cpt, hrt->hrt_id);
 	unshare_fs_struct();
 
 	rc = cfs_cpt_bind(ptlrpc_hr.hr_cpt_table, hrp->hrp_cpt);
 	if (rc != 0) {
+		char threadname[20];
+
+		snprintf(threadname, sizeof(threadname), "ptlrpc_hr%02d_%03d",
+			 hrp->hrp_cpt, hrt->hrt_id);
 		CWARN("Failed to bind %s on CPT %d of CPT table %p: rc = %d\n",
 		      threadname, hrp->hrp_cpt, ptlrpc_hr.hr_cpt_table, rc);
 	}
@@ -2528,7 +2529,7 @@ int ptlrpc_hr_init(void)
 
 	init_waitqueue_head(&ptlrpc_hr.hr_waitq);
 
-	weight = cpumask_weight(topology_sibling_cpumask(0));
+	weight = cpumask_weight(topology_sibling_cpumask(smp_processor_id()));
 
 	cfs_percpt_for_each(hrp, i, ptlrpc_hr.hr_partitions) {
 		hrp->hrp_cpt = i;
-- 
1.8.3.1


From jsimmons at infradead.org  Sun Jun 24 21:20:26 2018
From: jsimmons at infradead.org (James Simmons)
Date: Sun, 24 Jun 2018 17:20:26 -0400
Subject: [lustre-devel] [PATCH v3 02/26] staging: lustre: libcfs: rename
 variable i to cpu
In-Reply-To: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
References: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1529875250-11531-3-git-send-email-jsimmons@infradead.org>

From: Dmitry Eremin <dmitry.eremin at intel.com>

Change the name of the variable i used for for_each_cpu() to cpu
for code readability.

Signed-off-by: Dmitry Eremin <dmitry.eremin at intel.com>
WC-bug-id: https://jira.whamcloud.com/browse/LU-8703
Reviewed-on: https://review.whamcloud.com/23303
Reviewed-by: James Simmons <uja.ornl at yahoo.com>
Reviewed-by: Doug Oucharek <dougso at me.com>
Reviewed-by: Oleg Drokin <green at whamcloud.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
index b363a3d..46d3530 100644
--- a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
+++ b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
@@ -344,7 +344,7 @@ struct cfs_cpt_table *
 cfs_cpt_set_cpumask(struct cfs_cpt_table *cptab, int cpt,
 		    const cpumask_t *mask)
 {
-	int i;
+	int cpu;
 
 	if (!cpumask_weight(mask) ||
 	    cpumask_any_and(mask, cpu_online_mask) >= nr_cpu_ids) {
@@ -353,8 +353,8 @@ struct cfs_cpt_table *
 		return 0;
 	}
 
-	for_each_cpu(i, mask) {
-		if (!cfs_cpt_set_cpu(cptab, cpt, i))
+	for_each_cpu(cpu, mask) {
+		if (!cfs_cpt_set_cpu(cptab, cpt, cpu))
 			return 0;
 	}
 
@@ -366,10 +366,10 @@ struct cfs_cpt_table *
 cfs_cpt_unset_cpumask(struct cfs_cpt_table *cptab, int cpt,
 		      const cpumask_t *mask)
 {
-	int i;
+	int cpu;
 
-	for_each_cpu(i, mask)
-		cfs_cpt_unset_cpu(cptab, cpt, i);
+	for_each_cpu(cpu, mask)
+		cfs_cpt_unset_cpu(cptab, cpt, cpu);
 }
 EXPORT_SYMBOL(cfs_cpt_unset_cpumask);
 
-- 
1.8.3.1


From jsimmons at infradead.org  Sun Jun 24 21:20:45 2018
From: jsimmons at infradead.org (James Simmons)
Date: Sun, 24 Jun 2018 17:20:45 -0400
Subject: [lustre-devel] [PATCH v3 21/26] staging: lustre: libcfs: rework CPU
 pattern parsing code
In-Reply-To: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
References: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1529875250-11531-22-git-send-email-jsimmons@infradead.org>

From: Dmitry Eremin <dmitry.eremin at intel.com>

Currently the module param string for CPU pattern can be
modified which is wrong. Rewrite CPU pattern parsing code
to avoid the passed buffer from being changed. This change
also enables us to add real errors propogation to the caller
functions.

Signed-off-by: Dmitry Eremin <dmitry.eremin at intel.com>
Signed-off-by: Amir Shehata <amir.shehata at intel.com>
Signed-off-by: Andreas Dilger <adilger at whamcloud.com>
WC-bug-id: https://jira.whamcloud.com/browse/LU-8703
Reviewed-on: https://review.whamcloud.com/23306
WC-bug-id: https://jira.whamcloud.com/browse/LU-9715
Reviewed-on: https://review.whamcloud.com/27872
Reviewed-by: James Simmons <uja.ornl at yahoo.com>
Reviewed-by: Andreas Dilger <adilger at whamcloud.com>
Reviewed-by: Patrick Farrell <paf at cray.com>
Reviewed-by: Olaf Weber <olaf.weber at hpe.com>
Reviewed-by: Oleg Drokin <green at whamcloud.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c | 146 ++++++++++++++----------
 1 file changed, 86 insertions(+), 60 deletions(-)

diff --git a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
index c2bad0d..2fdea11 100644
--- a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
+++ b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
@@ -696,11 +696,11 @@ int cfs_cpt_bind(struct cfs_cpt_table *cptab, int cpt)
 		nodemask = cptab->ctb_parts[cpt].cpt_nodemask;
 	}
 
-	if (cpumask_any_and(*cpumask, cpu_online_mask) >= nr_cpu_ids) {
+	if (!cpumask_intersects(*cpumask, cpu_online_mask)) {
 		CDEBUG(D_INFO,
 		       "No online CPU found in CPU partition %d, did someone do CPU hotplug on system? You might need to reload Lustre modules to keep system working well.\n",
 		       cpt);
-		return -EINVAL;
+		return -ENODEV;
 	}
 
 	for_each_online_cpu(cpu) {
@@ -864,11 +864,13 @@ static struct cfs_cpt_table *cfs_cpt_table_create(int ncpt)
 	cptab = cfs_cpt_table_alloc(ncpt);
 	if (!cptab) {
 		CERROR("Failed to allocate CPU map(%d)\n", ncpt);
+		rc = -ENOMEM;
 		goto failed;
 	}
 
 	if (!zalloc_cpumask_var(&node_mask, GFP_NOFS)) {
 		CERROR("Failed to allocate scratch cpumask\n");
+		rc = -ENOMEM;
 		goto failed;
 	}
 
@@ -883,8 +885,10 @@ static struct cfs_cpt_table *cfs_cpt_table_create(int ncpt)
 
 			rc = cfs_cpt_choose_ncpus(cptab, cpt, node_mask,
 						  num - ncpu);
-			if (rc < 0)
+			if (rc < 0) {
+				rc = -EINVAL;
 				goto failed_mask;
+			}
 
 			ncpu = cpumask_weight(part->cpt_cpumask);
 			if (ncpu == num + !!(rem > 0)) {
@@ -907,37 +911,51 @@ static struct cfs_cpt_table *cfs_cpt_table_create(int ncpt)
 	if (cptab)
 		cfs_cpt_table_free(cptab);
 
-	return NULL;
+	return ERR_PTR(rc);
 }
 
-static struct cfs_cpt_table *cfs_cpt_table_create_pattern(char *pattern)
+static struct cfs_cpt_table *cfs_cpt_table_create_pattern(const char *pattern)
 {
 	struct cfs_cpt_table *cptab;
+	char *pattern_dup;
+	char *bracket;
 	char *str;
 	int node = 0;
-	int high;
 	int ncpt = 0;
-	int cpt;
+	int cpt = 0;
+	int high;
 	int rc;
 	int c;
 	int i;
 
-	str = strim(pattern);
+	pattern_dup = kstrdup(pattern, GFP_KERNEL);
+	if (!pattern_dup) {
+		CERROR("Failed to duplicate pattern '%s'\n", pattern);
+		return ERR_PTR(-ENOMEM);
+	}
+
+	str = strim(pattern_dup);
 	if (*str == 'n' || *str == 'N') {
-		pattern = str + 1;
-		if (*pattern != '\0') {
-			node = 1;
-		} else { /* shortcut to create CPT from NUMA & CPU topology */
+		str++; /* skip 'N' char */
+		node = 1; /* NUMA pattern */
+		if (*str == '\0') {
 			node = -1;
-			ncpt = num_online_nodes();
+			for_each_online_node(i) {
+				if (!cpumask_empty(cpumask_of_node(i)))
+					ncpt++;
+			}
+			if (ncpt == 1) { /* single NUMA node */
+				kfree(pattern_dup);
+				return cfs_cpt_table_create(cpu_npartitions);
+			}
 		}
 	}
 
 	if (!ncpt) { /* scanning bracket which is mark of partition */
-		for (str = pattern;; str++, ncpt++) {
-			str = strchr(str, '[');
-			if (!str)
-				break;
+		bracket = str;
+		while ((bracket = strchr(bracket, '['))) {
+			bracket++;
+			ncpt++;
 		}
 	}
 
@@ -945,87 +963,96 @@ static struct cfs_cpt_table *cfs_cpt_table_create_pattern(char *pattern)
 	    (node && ncpt > num_online_nodes()) ||
 	    (!node && ncpt > num_online_cpus())) {
 		CERROR("Invalid pattern '%s', or too many partitions %d\n",
-		       pattern, ncpt);
-		return NULL;
+		       pattern_dup, ncpt);
+		rc = -EINVAL;
+		goto err_free_str;
 	}
 
 	cptab = cfs_cpt_table_alloc(ncpt);
 	if (!cptab) {
 		CERROR("Failed to allocate CPU partition table\n");
-		return NULL;
+		rc = -ENOMEM;
+		goto err_free_str;
 	}
 
 	if (node < 0) { /* shortcut to create CPT from NUMA & CPU topology */
-		cpt = 0;
-
 		for_each_online_node(i) {
-			if (cpt >= ncpt) {
-				CERROR("CPU changed while setting CPU partition table, %d/%d\n",
-				       cpt, ncpt);
-				goto failed;
-			}
+			if (cpumask_empty(cpumask_of_node(i)))
+				continue;
 
 			rc = cfs_cpt_set_node(cptab, cpt++, i);
-			if (!rc)
-				goto failed;
+			if (!rc) {
+				rc = -EINVAL;
+				goto err_free_table;
+			}
 		}
+		kfree(pattern_dup);
 		return cptab;
 	}
 
 	high = node ? nr_node_ids - 1 : nr_cpu_ids - 1;
 
-	for (str = strim(pattern), c = 0;; c++) {
+	for (str = strim(str), c = 0; /* until break */; c++) {
 		struct cfs_range_expr *range;
 		struct cfs_expr_list *el;
-		char *bracket = strchr(str, '[');
 		int n;
 
+		bracket = strchr(str, '[');
 		if (!bracket) {
 			if (*str) {
 				CERROR("Invalid pattern '%s'\n", str);
-				goto failed;
+				rc = -EINVAL;
+				goto err_free_table;
 			}
 			if (c != ncpt) {
 				CERROR("Expect %d partitions but found %d\n",
 				       ncpt, c);
-				goto failed;
+				rc = -EINVAL;
+				goto err_free_table;
 			}
 			break;
 		}
 
 		if (sscanf(str, "%d%n", &cpt, &n) < 1) {
 			CERROR("Invalid CPU pattern '%s'\n", str);
-			goto failed;
+			rc = -EINVAL;
+			goto err_free_table;
 		}
 
 		if (cpt < 0 || cpt >= ncpt) {
 			CERROR("Invalid partition id %d, total partitions %d\n",
 			       cpt, ncpt);
-			goto failed;
+			rc = -EINVAL;
+			goto err_free_table;
 		}
 
 		if (cfs_cpt_weight(cptab, cpt)) {
 			CERROR("Partition %d has already been set.\n", cpt);
-			goto failed;
+			rc = -EPERM;
+			goto err_free_table;
 		}
 
 		str = strim(str + n);
 		if (str != bracket) {
 			CERROR("Invalid pattern '%s'\n", str);
-			goto failed;
+			rc = -EINVAL;
+			goto err_free_table;
 		}
 
 		bracket = strchr(str, ']');
 		if (!bracket) {
 			CERROR("Missing right bracket for partition %d in '%s'\n",
 			       cpt, str);
-			goto failed;
+			rc = -EINVAL;
+			goto err_free_table;
 		}
 
-		if (cfs_expr_list_parse(str, (bracket - str) + 1,
-					0, high, &el)) {
+		rc = cfs_expr_list_parse(str, (bracket - str) + 1, 0, high,
+					 &el);
+		if (rc) {
 			CERROR("Can't parse number range in '%s'\n", str);
-			goto failed;
+			rc = -ERANGE;
+			goto err_free_table;
 		}
 
 		list_for_each_entry(range, &el->el_exprs, re_link) {
@@ -1037,7 +1064,8 @@ static struct cfs_cpt_table *cfs_cpt_table_create_pattern(char *pattern)
 					    cfs_cpt_set_cpu(cptab, cpt, i);
 				if (!rc) {
 					cfs_expr_list_free(el);
-					goto failed;
+					rc = -EINVAL;
+					goto err_free_table;
 				}
 			}
 		}
@@ -1046,17 +1074,21 @@ static struct cfs_cpt_table *cfs_cpt_table_create_pattern(char *pattern)
 
 		if (!cfs_cpt_online(cptab, cpt)) {
 			CERROR("No online CPU is found on partition %d\n", cpt);
-			goto failed;
+			rc = -ENODEV;
+			goto err_free_table;
 		}
 
 		str = strim(bracket + 1);
 	}
 
+	kfree(pattern_dup);
 	return cptab;
 
-failed:
+err_free_table:
 	cfs_cpt_table_free(cptab);
-	return NULL;
+err_free_str:
+	kfree(pattern_dup);
+	return ERR_PTR(rc);
 }
 
 #ifdef CONFIG_HOTPLUG_CPU
@@ -1083,7 +1115,7 @@ static int cfs_cpu_dead(unsigned int cpu)
 
 void cfs_cpu_fini(void)
 {
-	if (cfs_cpt_tab)
+	if (!IS_ERR_OR_NULL(cfs_cpt_tab))
 		cfs_cpt_table_free(cfs_cpt_tab);
 
 #ifdef CONFIG_HOTPLUG_CPU
@@ -1116,26 +1148,20 @@ int cfs_cpu_init(void)
 #endif
 	get_online_cpus();
 	if (*cpu_pattern) {
-		char *cpu_pattern_dup = kstrdup(cpu_pattern, GFP_KERNEL);
-
-		if (!cpu_pattern_dup) {
-			CERROR("Failed to duplicate cpu_pattern\n");
-			goto failed_alloc_table;
-		}
-
-		cfs_cpt_tab = cfs_cpt_table_create_pattern(cpu_pattern_dup);
-		kfree(cpu_pattern_dup);
-		if (!cfs_cpt_tab) {
-			CERROR("Failed to create cptab from pattern %s\n",
+		cfs_cpt_tab = cfs_cpt_table_create_pattern(cpu_pattern);
+		if (IS_ERR(cfs_cpt_tab)) {
+			CERROR("Failed to create cptab from pattern '%s'\n",
 			       cpu_pattern);
+			ret = PTR_ERR(cfs_cpt_tab);
 			goto failed_alloc_table;
 		}
 
 	} else {
 		cfs_cpt_tab = cfs_cpt_table_create(cpu_npartitions);
-		if (!cfs_cpt_tab) {
-			CERROR("Failed to create ptable with npartitions %d\n",
+		if (IS_ERR(cfs_cpt_tab)) {
+			CERROR("Failed to create cptab with npartitions %d\n",
 			       cpu_npartitions);
+			ret = PTR_ERR(cfs_cpt_tab);
 			goto failed_alloc_table;
 		}
 	}
@@ -1150,7 +1176,7 @@ int cfs_cpu_init(void)
 failed_alloc_table:
 	put_online_cpus();
 
-	if (cfs_cpt_tab)
+	if (!IS_ERR_OR_NULL(cfs_cpt_tab))
 		cfs_cpt_table_free(cfs_cpt_tab);
 
 	ret = -EINVAL;
-- 
1.8.3.1


From jsimmons at infradead.org  Sun Jun 24 21:20:41 2018
From: jsimmons at infradead.org (James Simmons)
Date: Sun, 24 Jun 2018 17:20:41 -0400
Subject: [lustre-devel] [PATCH v3 17/26] staging: lustre: libcfs: update
 debug messages
In-Reply-To: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
References: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1529875250-11531-18-git-send-email-jsimmons@infradead.org>

From: Dmitry Eremin <dmitry.eremin at intel.com>

For cfs_cpt_bind() change the CERROR to CDEBUG. Make the debug
message in cfs_cpt_table_create_pattern() more understandable.
Report rc value for when cfs_cpt_create_table() fails.

Signed-off-by: Dmitry Eremin <dmitry.eremin at intel.com>
WC-bug-id: https://jira.whamcloud.com/browse/LU-8703
Reviewed-on: https://review.whamcloud.com/23222
Reviewed-by: Amir Shehata <amir.shehata at intel.com>
Reviewed-by: James Simmons <uja.ornl at yahoo.com>
Reviewed-by: Oleg Drokin <green at whamcloud.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c | 15 +++++++++------
 1 file changed, 9 insertions(+), 6 deletions(-)

diff --git a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
index fcb068a..ebfa4e3 100644
--- a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
+++ b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
@@ -484,7 +484,8 @@ void cfs_cpt_unset_cpu(struct cfs_cpt_table *cptab, int cpt, int cpu)
 		/* caller doesn't know the partition ID */
 		cpt = cptab->ctb_cpu2cpt[cpu];
 		if (cpt < 0) { /* not set in this CPT-table */
-			CDEBUG(D_INFO, "Try to unset cpu %d which is not in CPT-table %p\n",
+			CDEBUG(D_INFO,
+			       "Try to unset cpu %d which is not in CPT-table %p\n",
 			       cpt, cptab);
 			return;
 		}
@@ -510,7 +511,8 @@ int cfs_cpt_set_cpumask(struct cfs_cpt_table *cptab, int cpt,
 
 	if (!cpumask_weight(mask) ||
 	    cpumask_any_and(mask, cpu_online_mask) >= nr_cpu_ids) {
-		CDEBUG(D_INFO, "No online CPU is found in the CPU mask for CPU partition %d\n",
+		CDEBUG(D_INFO,
+		       "No online CPU is found in the CPU mask for CPU partition %d\n",
 		       cpt);
 		return 0;
 	}
@@ -687,7 +689,8 @@ int cfs_cpt_bind(struct cfs_cpt_table *cptab, int cpt)
 	}
 
 	if (cpumask_any_and(*cpumask, cpu_online_mask) >= nr_cpu_ids) {
-		CERROR("No online CPU found in CPU partition %d, did someone do CPU hotplug on system? You might need to reload Lustre modules to keep system working well.\n",
+		CDEBUG(D_INFO,
+		       "No online CPU found in CPU partition %d, did someone do CPU hotplug on system? You might need to reload Lustre modules to keep system working well.\n",
 		       cpt);
 		return -EINVAL;
 	}
@@ -918,8 +921,8 @@ static struct cfs_cpt_table *cfs_cpt_table_create(int ncpt)
 failed_mask:
 	free_cpumask_var(node_mask);
 failed:
-	CERROR("Failed to setup CPU-partition-table with %d CPU-partitions, online HW nodes: %d, HW cpus: %d.\n",
-	       ncpt, num_online_nodes(), num_online_cpus());
+	CERROR("Failed (rc = %d) to setup CPU partition table with %d partitions, online HW NUMA nodes: %d, HW CPU cores: %d.\n",
+	       rc, ncpt, num_online_nodes(), num_online_cpus());
 
 	if (cptab)
 		cfs_cpt_table_free(cptab);
@@ -1034,7 +1037,7 @@ static struct cfs_cpt_table *cfs_cpt_table_create_pattern(char *pattern)
 
 		bracket = strchr(str, ']');
 		if (!bracket) {
-			CERROR("missing right bracket for cpt %d, %s\n",
+			CERROR("Missing right bracket for partition %d, %s\n",
 			       cpt, str);
 			goto failed;
 		}
-- 
1.8.3.1


From jsimmons at infradead.org  Sun Jun 24 21:20:44 2018
From: jsimmons at infradead.org (James Simmons)
Date: Sun, 24 Jun 2018 17:20:44 -0400
Subject: [lustre-devel] [PATCH v3 20/26] staging: lustre: libcfs: update
 debug messages in CPT code
In-Reply-To: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
References: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1529875250-11531-21-git-send-email-jsimmons@infradead.org>

From: Dmitry Eremin <dmitry.eremin at intel.com>

Update the debug messages for the CPT table creation code. Place
the passed in string in quotes to make it clear what it is.
Captialize cpu in the debug strings.

Signed-off-by: Dmitry Eremin <dmitry.eremin at intel.com>
WC-bug-id: https://jira.whamcloud.com/browse/LU-8703
Reviewed-on: https://review.whamcloud.com/23306
Reviewed-by: James Simmons <uja.ornl at yahoo.com>
Reviewed-by: Andreas Dilger <adilger at whamcloud.com>
Reviewed-by: Patrick Farrell <paf at cray.com>
Reviewed-by: Olaf Weber <olaf.weber at hpe.com>
Reviewed-by: Oleg Drokin <green at whamcloud.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c | 18 +++++++++---------
 1 file changed, 9 insertions(+), 9 deletions(-)

diff --git a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
index 86afa31..c2bad0d 100644
--- a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
+++ b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
@@ -500,7 +500,7 @@ void cfs_cpt_unset_cpu(struct cfs_cpt_table *cptab, int cpt, int cpu)
 
 	} else if (cpt != cptab->ctb_cpu2cpt[cpu]) {
 		CDEBUG(D_INFO,
-		       "CPU %d is not in cpu-partition %d\n", cpu, cpt);
+		       "CPU %d is not in CPU partition %d\n", cpu, cpt);
 		return;
 	}
 
@@ -944,14 +944,14 @@ static struct cfs_cpt_table *cfs_cpt_table_create_pattern(char *pattern)
 	if (!ncpt ||
 	    (node && ncpt > num_online_nodes()) ||
 	    (!node && ncpt > num_online_cpus())) {
-		CERROR("Invalid pattern %s, or too many partitions %d\n",
+		CERROR("Invalid pattern '%s', or too many partitions %d\n",
 		       pattern, ncpt);
 		return NULL;
 	}
 
 	cptab = cfs_cpt_table_alloc(ncpt);
 	if (!cptab) {
-		CERROR("Failed to allocate cpu partition table\n");
+		CERROR("Failed to allocate CPU partition table\n");
 		return NULL;
 	}
 
@@ -982,11 +982,11 @@ static struct cfs_cpt_table *cfs_cpt_table_create_pattern(char *pattern)
 
 		if (!bracket) {
 			if (*str) {
-				CERROR("Invalid pattern %s\n", str);
+				CERROR("Invalid pattern '%s'\n", str);
 				goto failed;
 			}
 			if (c != ncpt) {
-				CERROR("expect %d partitions but found %d\n",
+				CERROR("Expect %d partitions but found %d\n",
 				       ncpt, c);
 				goto failed;
 			}
@@ -994,7 +994,7 @@ static struct cfs_cpt_table *cfs_cpt_table_create_pattern(char *pattern)
 		}
 
 		if (sscanf(str, "%d%n", &cpt, &n) < 1) {
-			CERROR("Invalid cpu pattern %s\n", str);
+			CERROR("Invalid CPU pattern '%s'\n", str);
 			goto failed;
 		}
 
@@ -1011,20 +1011,20 @@ static struct cfs_cpt_table *cfs_cpt_table_create_pattern(char *pattern)
 
 		str = strim(str + n);
 		if (str != bracket) {
-			CERROR("Invalid pattern %s\n", str);
+			CERROR("Invalid pattern '%s'\n", str);
 			goto failed;
 		}
 
 		bracket = strchr(str, ']');
 		if (!bracket) {
-			CERROR("Missing right bracket for partition %d, %s\n",
+			CERROR("Missing right bracket for partition %d in '%s'\n",
 			       cpt, str);
 			goto failed;
 		}
 
 		if (cfs_expr_list_parse(str, (bracket - str) + 1,
 					0, high, &el)) {
-			CERROR("Can't parse number range: %s\n", str);
+			CERROR("Can't parse number range in '%s'\n", str);
 			goto failed;
 		}
 
-- 
1.8.3.1


From jsimmons at infradead.org  Sun Jun 24 21:20:42 2018
From: jsimmons at infradead.org (James Simmons)
Date: Sun, 24 Jun 2018 17:20:42 -0400
Subject: [lustre-devel] [PATCH v3 18/26] staging: lustre: libcfs: make
 tolerant to offline CPUs and empty NUMA nodes
In-Reply-To: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
References: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1529875250-11531-19-git-send-email-jsimmons@infradead.org>

From: Dmitry Eremin <dmitry.eremin at intel.com>

Rework CPU partition code in the way of make it more tolerant to
offline CPUs and empty nodes.

Signed-off-by: Dmitry Eremin <dmitry.eremin at intel.com>
WC-bug-id: https://jira.whamcloud.com/browse/LU-8703
Reviewed-on: https://review.whamcloud.com/23222
Reviewed-by: Amir Shehata <amir.shehata at intel.com>
Reviewed-by: James Simmons <uja.ornl at yahoo.com>
Reviewed-by: Oleg Drokin <green at whamcloud.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c | 132 ++++++++++--------------
 1 file changed, 56 insertions(+), 76 deletions(-)

diff --git a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
index ebfa4e3..18925c7 100644
--- a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
+++ b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
@@ -461,8 +461,16 @@ int cfs_cpt_set_cpu(struct cfs_cpt_table *cptab, int cpt, int cpu)
 		return 0;
 	}
 
-	LASSERT(!cpumask_test_cpu(cpu, cptab->ctb_cpumask));
-	LASSERT(!cpumask_test_cpu(cpu, cptab->ctb_parts[cpt].cpt_cpumask));
+	if (cpumask_test_cpu(cpu, cptab->ctb_cpumask)) {
+		CDEBUG(D_INFO, "CPU %d is already in cpumask\n", cpu);
+		return 0;
+	}
+
+	if (cpumask_test_cpu(cpu, cptab->ctb_parts[cpt].cpt_cpumask)) {
+		CDEBUG(D_INFO, "CPU %d is already in partition %d cpumask\n",
+		       cpu, cptab->ctb_cpu2cpt[cpu]);
+		return 0;
+	}
 
 	cfs_cpt_add_cpu(cptab, cpt, cpu);
 	cfs_cpt_add_node(cptab, cpt, cpu_to_node(cpu));
@@ -531,8 +539,10 @@ void cfs_cpt_unset_cpumask(struct cfs_cpt_table *cptab, int cpt,
 {
 	int cpu;
 
-	for_each_cpu(cpu, mask)
-		cfs_cpt_unset_cpu(cptab, cpt, cpu);
+	for_each_cpu(cpu, mask) {
+		cfs_cpt_del_cpu(cptab, cpt, cpu);
+		cfs_cpt_del_node(cptab, cpt, cpu_to_node(cpu));
+	}
 }
 EXPORT_SYMBOL(cfs_cpt_unset_cpumask);
 
@@ -583,10 +593,8 @@ int cfs_cpt_set_nodemask(struct cfs_cpt_table *cptab, int cpt,
 {
 	int node;
 
-	for_each_node_mask(node, *mask) {
-		if (!cfs_cpt_set_node(cptab, cpt, node))
-			return 0;
-	}
+	for_each_node_mask(node, *mask)
+		cfs_cpt_set_node(cptab, cpt, node);
 
 	return 1;
 }
@@ -607,7 +615,7 @@ int cfs_cpt_spread_node(struct cfs_cpt_table *cptab, int cpt)
 	nodemask_t *mask;
 	int weight;
 	int rotor;
-	int node;
+	int node = 0;
 
 	/* convert CPU partition ID to HW node id */
 
@@ -617,20 +625,20 @@ int cfs_cpt_spread_node(struct cfs_cpt_table *cptab, int cpt)
 	} else {
 		mask = cptab->ctb_parts[cpt].cpt_nodemask;
 		rotor = cptab->ctb_parts[cpt].cpt_spread_rotor++;
+		node  = cptab->ctb_parts[cpt].cpt_node;
 	}
 
 	weight = nodes_weight(*mask);
-	LASSERT(weight > 0);
-
-	rotor %= weight;
+	if (weight > 0) {
+		rotor %= weight;
 
-	for_each_node_mask(node, *mask) {
-		if (!rotor--)
-			return node;
+		for_each_node_mask(node, *mask) {
+			if (!rotor--)
+				return node;
+		}
 	}
 
-	LBUG();
-	return 0;
+	return node;
 }
 EXPORT_SYMBOL(cfs_cpt_spread_node);
 
@@ -723,17 +731,21 @@ static int cfs_cpt_choose_ncpus(struct cfs_cpt_table *cptab, int cpt,
 	cpumask_var_t core_mask;
 	int rc = 0;
 	int cpu;
+	int i;
 
 	LASSERT(number > 0);
 
 	if (number >= cpumask_weight(node_mask)) {
 		while (!cpumask_empty(node_mask)) {
 			cpu = cpumask_first(node_mask);
+			cpumask_clear_cpu(cpu, node_mask);
+
+			if (!cpu_online(cpu))
+				continue;
 
 			rc = cfs_cpt_set_cpu(cptab, cpt, cpu);
 			if (!rc)
 				return -EINVAL;
-			cpumask_clear_cpu(cpu, node_mask);
 		}
 		return 0;
 	}
@@ -754,24 +766,19 @@ static int cfs_cpt_choose_ncpus(struct cfs_cpt_table *cptab, int cpt,
 		cpu = cpumask_first(node_mask);
 
 		/* get cpumask for cores in the same socket */
-		cpumask_copy(socket_mask, topology_core_cpumask(cpu));
-		cpumask_and(socket_mask, socket_mask, node_mask);
-
-		LASSERT(!cpumask_empty(socket_mask));
-
+		cpumask_and(socket_mask, topology_core_cpumask(cpu), node_mask);
 		while (!cpumask_empty(socket_mask)) {
-			int i;
-
 			/* get cpumask for hts in the same core */
-			cpumask_copy(core_mask, topology_sibling_cpumask(cpu));
-			cpumask_and(core_mask, core_mask, node_mask);
-
-			LASSERT(!cpumask_empty(core_mask));
+			cpumask_and(core_mask, topology_sibling_cpumask(cpu),
+				    node_mask);
 
 			for_each_cpu(i, core_mask) {
 				cpumask_clear_cpu(i, socket_mask);
 				cpumask_clear_cpu(i, node_mask);
 
+				if (!cpu_online(i))
+					continue;
+
 				rc = cfs_cpt_set_cpu(cptab, cpt, i);
 				if (!rc) {
 					rc = -EINVAL;
@@ -840,23 +847,18 @@ static struct cfs_cpt_table *cfs_cpt_table_create(int ncpt)
 	struct cfs_cpt_table *cptab = NULL;
 	cpumask_var_t node_mask;
 	int cpt = 0;
+	int node;
 	int num;
-	int rc;
-	int i;
+	int rem;
+	int rc = 0;
 
-	rc = cfs_cpt_num_estimate();
+	num = cfs_cpt_num_estimate();
 	if (ncpt <= 0)
-		ncpt = rc;
+		ncpt = num;
 
-	if (ncpt > num_online_cpus() || ncpt > 4 * rc) {
+	if (ncpt > num_online_cpus() || ncpt > 4 * num) {
 		CWARN("CPU partition number %d is larger than suggested value (%d), your system may have performance issue or run out of memory while under pressure\n",
-		      ncpt, rc);
-	}
-
-	if (num_online_cpus() % ncpt) {
-		CERROR("CPU number %d is not multiple of cpu_npartition %d, please try different cpu_npartitions value or set pattern string by cpu_pattern=STRING\n",
-		       (int)num_online_cpus(), ncpt);
-		goto failed;
+		      ncpt, num);
 	}
 
 	cptab = cfs_cpt_table_alloc(ncpt);
@@ -865,55 +867,33 @@ static struct cfs_cpt_table *cfs_cpt_table_create(int ncpt)
 		goto failed;
 	}
 
-	num = num_online_cpus() / ncpt;
-	if (!num) {
-		CERROR("CPU changed while setting CPU partition\n");
-		goto failed;
-	}
-
 	if (!zalloc_cpumask_var(&node_mask, GFP_NOFS)) {
 		CERROR("Failed to allocate scratch cpumask\n");
 		goto failed;
 	}
 
-	for_each_online_node(i) {
-		cpumask_copy(node_mask, cpumask_of_node(i));
-
-		while (!cpumask_empty(node_mask)) {
-			struct cfs_cpu_partition *part;
-			int n;
-
-			/*
-			 * Each emulated NUMA node has all allowed CPUs in
-			 * the mask.
-			 * End loop when all partitions have assigned CPUs.
-			 */
-			if (cpt == ncpt)
-				break;
-
-			part = &cptab->ctb_parts[cpt];
+	num = num_online_cpus() / ncpt;
+	rem = num_online_cpus() % ncpt;
+	for_each_online_node(node) {
+		cpumask_copy(node_mask, cpumask_of_node(node));
 
-			n = num - cpumask_weight(part->cpt_cpumask);
-			LASSERT(n > 0);
+		while (cpt < ncpt && !cpumask_empty(node_mask)) {
+			struct cfs_cpu_partition *part = &cptab->ctb_parts[cpt];
+			int ncpu = cpumask_weight(part->cpt_cpumask);
 
-			rc = cfs_cpt_choose_ncpus(cptab, cpt, node_mask, n);
+			rc = cfs_cpt_choose_ncpus(cptab, cpt, node_mask,
+						  num - ncpu);
 			if (rc < 0)
 				goto failed_mask;
 
-			LASSERT(num >= cpumask_weight(part->cpt_cpumask));
-			if (num == cpumask_weight(part->cpt_cpumask))
+			ncpu = cpumask_weight(part->cpt_cpumask);
+			if (ncpu == num + !!(rem > 0)) {
 				cpt++;
+				rem--;
+			}
 		}
 	}
 
-	if (cpt != ncpt ||
-	    num != cpumask_weight(cptab->ctb_parts[ncpt - 1].cpt_cpumask)) {
-		CERROR("Expect %d(%d) CPU partitions but got %d(%d), CPU hotplug/unplug while setting?\n",
-		       cptab->ctb_nparts, num, cpt,
-		       cpumask_weight(cptab->ctb_parts[ncpt - 1].cpt_cpumask));
-		goto failed_mask;
-	}
-
 	free_cpumask_var(node_mask);
 
 	return cptab;
-- 
1.8.3.1


From jsimmons at infradead.org  Sun Jun 24 21:53:44 2018
From: jsimmons at infradead.org (James Simmons)
Date: Sun, 24 Jun 2018 17:53:44 -0400
Subject: [lustre-devel] [PATCH 0/8] staging: lustre: lnet: resolve time
 handling issues
Message-ID: <1529877232-15582-1-git-send-email-jsimmons@infradead.org>

The migration of lustre to 64 bit time handling exposed various bugs
which have lead to problems like routers no longer functioning with
the upstream client. This is due to the complex handling of time by
lustre with seconds time units and jiffies. Add in the fact that
jiffies can vary from node to node and you can encounter corner
case issues in a large node deployment. The solution to resolve this
was to migrate as much as possible to time64_t when all you needed
was seconds percision and ktime_t if better percision was required.
With these changes it is reduced down to using jiffies only for
schedule_timeout calls.

James Simmons (8):
  staging: lustre: lnet: simplify lnet_eq_wait_locked
  staging: lustre: o2iblnd: use ktime_get_real_ns for ibn_incarnation
  staging: lustre: ksocklnd: use ktime_get_real_ns for ksnn_incarnation
  staging: lustre: ksocklnd: move remaining time handling to 64 bits
  staging: lustre: lnet: use ktime_get_real_ns for cookie creation
  staging: lustre: lnet: move ping and delay injection to time64_t
  staging: lustre: ko2iblnd: remove remaining jiffy use
  staging: lustre: selftest: fix time reporting

 drivers/staging/lustre/include/linux/lnet/api.h    |   2 +-
 .../staging/lustre/include/linux/lnet/lib-lnet.h   |   8 +-
 .../staging/lustre/include/linux/lnet/lib-types.h  |  13 ++-
 .../lustre/include/uapi/linux/lnet/lnet-types.h    |   2 -
 .../lustre/include/uapi/linux/lnet/lnetst.h        |   2 +-
 .../staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c    |  53 +++++-----
 .../staging/lustre/lnet/klnds/o2iblnd/o2iblnd.h    |  28 ++---
 .../staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c |  36 +++----
 .../staging/lustre/lnet/klnds/socklnd/socklnd.c    |  44 ++++----
 .../staging/lustre/lnet/klnds/socklnd/socklnd.h    |  22 ++--
 .../staging/lustre/lnet/klnds/socklnd/socklnd_cb.c | 117 ++++++++++-----------
 drivers/staging/lustre/lnet/lnet/api-ni.c          |  54 ++++++----
 drivers/staging/lustre/lnet/lnet/lib-eq.c          |  26 ++---
 drivers/staging/lustre/lnet/lnet/lib-move.c        |  24 ++---
 drivers/staging/lustre/lnet/lnet/net_fault.c       |  72 ++++++-------
 drivers/staging/lustre/lnet/lnet/peer.c            |   2 +-
 drivers/staging/lustre/lnet/lnet/router.c          |  35 +++---
 drivers/staging/lustre/lnet/lnet/router_proc.c     |  27 +++--
 drivers/staging/lustre/lnet/selftest/conrpc.c      |  39 +++----
 drivers/staging/lustre/lnet/selftest/conrpc.h      |   2 +-
 drivers/staging/lustre/lnet/selftest/console.c     |   4 +-
 drivers/staging/lustre/lnet/selftest/console.h     |   2 +-
 drivers/staging/lustre/lnet/selftest/framework.c   |   4 +-
 drivers/staging/lustre/lnet/selftest/selftest.h    |   2 +-
 drivers/staging/lustre/lnet/selftest/timer.c       |   8 +-
 25 files changed, 304 insertions(+), 324 deletions(-)

-- 
1.8.3.1


From jsimmons at infradead.org  Sun Jun 24 21:53:46 2018
From: jsimmons at infradead.org (James Simmons)
Date: Sun, 24 Jun 2018 17:53:46 -0400
Subject: [lustre-devel] [PATCH 2/8] staging: lustre: o2iblnd: use
 ktime_get_real_ns for ibn_incarnation
In-Reply-To: <1529877232-15582-1-git-send-email-jsimmons@infradead.org>
References: <1529877232-15582-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1529877232-15582-3-git-send-email-jsimmons@infradead.org>

Instead of getting the real time using ktime_get_real_ts64() and
turning it into usecs we can simplify the code and call
ktime_get_real_ns() instead.

Signed-off-by: James Simmons <uja.ornl at yahoo.com>
WC-bug-id: https://jira.whamcloud.com/browse/LU-9019
Reviewed-on: https://review.whamcloud.com/23267
Reviewed-by: Doug Oucharek <dougso at me.com>
Reviewed-by: Andreas Dilger <adilger at whamcloud.com>
Reviewed-by: Oleg Drokin <green at whamcloud.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c | 5 +----
 1 file changed, 1 insertion(+), 4 deletions(-)

diff --git a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c
index f0b4eb42..5761e3d 100644
--- a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c
+++ b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c
@@ -2822,7 +2822,6 @@ static int kiblnd_startup(struct lnet_ni *ni)
 	char *ifname;
 	struct kib_dev *ibdev = NULL;
 	struct kib_net *net;
-	struct timespec64 tv;
 	unsigned long flags;
 	int rc;
 	int newdev;
@@ -2840,9 +2839,7 @@ static int kiblnd_startup(struct lnet_ni *ni)
 	if (!net)
 		goto net_failed;
 
-	ktime_get_real_ts64(&tv);
-	net->ibn_incarnation = tv.tv_sec * USEC_PER_SEC +
-			       tv.tv_nsec / NSEC_PER_USEC;
+	net->ibn_incarnation = ktime_get_real_ns() / NSEC_PER_USEC;
 
 	rc = kiblnd_tunables_setup(ni);
 	if (rc)
-- 
1.8.3.1


From jsimmons at infradead.org  Sun Jun 24 21:53:50 2018
From: jsimmons at infradead.org (James Simmons)
Date: Sun, 24 Jun 2018 17:53:50 -0400
Subject: [lustre-devel] [PATCH 6/8] staging: lustre: lnet: move ping and
 delay injection to time64_t
In-Reply-To: <1529877232-15582-1-git-send-email-jsimmons@infradead.org>
References: <1529877232-15582-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1529877232-15582-7-git-send-email-jsimmons@infradead.org>

Migrate away from jiffies for the pinger to time_64_t to one make
it clear its for time keeping and secondly to ensure the behavior
is consistent across any nodes. Besides the lnet pinger code
move the lnet delay injection code to time64_t as well.

Signed-off-by: James Simmons <uja.ornl at yahoo.com>
WC-bug-id: https://jira.whamcloud.com/browse/LU-9019
Reviewed-on: https://review.whamcloud.com/30658
WC-bug-id: https://jira.whamcloud.com/browse/LU-10672
Reviewed-on: https://review.whamcloud.com/31339
Reviewed-by: Dmitry Eremin <dmitry.eremin at intel.com>
Reviewed-by: Doug Oucharek <dougso at me.com>
Reviewed-by: John L. Hammond <jhammond at whamcloud.com>
Reviewed-by: Oleg Drokin <green at whamcloud.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 .../staging/lustre/include/linux/lnet/lib-lnet.h   |  8 +--
 .../staging/lustre/include/linux/lnet/lib-types.h  | 13 ++--
 .../staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c    | 11 ++--
 .../staging/lustre/lnet/klnds/o2iblnd/o2iblnd.h    |  4 +-
 .../staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c | 10 +--
 .../staging/lustre/lnet/klnds/socklnd/socklnd.c    |  4 +-
 .../staging/lustre/lnet/klnds/socklnd/socklnd.h    |  2 +-
 drivers/staging/lustre/lnet/lnet/api-ni.c          | 14 +++--
 drivers/staging/lustre/lnet/lnet/lib-move.c        | 24 ++++----
 drivers/staging/lustre/lnet/lnet/net_fault.c       | 72 ++++++++++------------
 drivers/staging/lustre/lnet/lnet/peer.c            |  2 +-
 drivers/staging/lustre/lnet/lnet/router.c          | 35 +++++------
 drivers/staging/lustre/lnet/lnet/router_proc.c     | 27 ++++----
 13 files changed, 110 insertions(+), 116 deletions(-)

diff --git a/drivers/staging/lustre/include/linux/lnet/lib-lnet.h b/drivers/staging/lustre/include/linux/lnet/lib-lnet.h
index 973c17a..6b6289c 100644
--- a/drivers/staging/lustre/include/linux/lnet/lib-lnet.h
+++ b/drivers/staging/lustre/include/linux/lnet/lib-lnet.h
@@ -404,9 +404,9 @@ struct lnet_ni *
 void lnet_lib_exit(void);
 
 int lnet_notify(struct lnet_ni *ni, lnet_nid_t peer, int alive,
-		unsigned long when);
+		time64_t when);
 void lnet_notify_locked(struct lnet_peer *lp, int notifylnd, int alive,
-			unsigned long when);
+			time64_t when);
 int lnet_add_route(__u32 net, __u32 hops, lnet_nid_t gateway_nid,
 		   unsigned int priority);
 int lnet_check_routes(void);
@@ -643,8 +643,8 @@ int lnet_get_peer_info(__u32 peer_index, __u64 *nid,
 static inline void
 lnet_peer_set_alive(struct lnet_peer *lp)
 {
-	lp->lp_last_query = jiffies;
-	lp->lp_last_alive = jiffies;
+	lp->lp_last_query = ktime_get_seconds();
+	lp->lp_last_alive = lp->lp_last_query;
 	if (!lp->lp_alive)
 		lnet_notify_locked(lp, 0, 1, lp->lp_last_alive);
 }
diff --git a/drivers/staging/lustre/include/linux/lnet/lib-types.h b/drivers/staging/lustre/include/linux/lnet/lib-types.h
index cfe8ee4..6d4106f 100644
--- a/drivers/staging/lustre/include/linux/lnet/lib-types.h
+++ b/drivers/staging/lustre/include/linux/lnet/lib-types.h
@@ -239,8 +239,7 @@ struct lnet_lnd {
 	void (*lnd_notify)(struct lnet_ni *ni, lnet_nid_t peer, int alive);
 
 	/* query of peer aliveness */
-	void (*lnd_query)(struct lnet_ni *ni, lnet_nid_t peer,
-			  unsigned long *when);
+	void (*lnd_query)(struct lnet_ni *ni, lnet_nid_t peer, time64_t *when);
 
 	/* accept a new connection */
 	int (*lnd_accept)(struct lnet_ni *ni, struct socket *sock);
@@ -334,17 +333,17 @@ struct lnet_peer {
 						  * dead<->alive
 						  */
 	long			 lp_txqnob;	 /* ytes queued for sending */
-	unsigned long		 lp_timestamp;	 /* time of last aliveness
+	time64_t		 lp_timestamp;	 /* time of last aliveness
 						  * news
 						  */
-	unsigned long		 lp_ping_timestamp;/* time of last ping
+	time64_t		 lp_ping_timestamp;/* time of last ping
 						    * attempt
 						    */
-	unsigned long		 lp_ping_deadline; /* != 0 if ping reply
+	time64_t		 lp_ping_deadline; /* != 0 if ping reply
 						    * expected
 						    */
-	unsigned long		 lp_last_alive;	/* when I was last alive */
-	unsigned long		 lp_last_query;	/* when lp_ni was queried
+	time64_t		 lp_last_alive;	/* when I was last alive */
+	time64_t		 lp_last_query;	/* when lp_ni was queried
 						 * last time
 						 */
 	struct lnet_ni		*lp_ni;		/* interface peer is on */
diff --git a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c
index 5761e3d..edae4eb 100644
--- a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c
+++ b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c
@@ -1039,11 +1039,10 @@ static int kiblnd_ctl(struct lnet_ni *ni, unsigned int cmd, void *arg)
 	return rc;
 }
 
-static void kiblnd_query(struct lnet_ni *ni, lnet_nid_t nid,
-			 unsigned long *when)
+static void kiblnd_query(struct lnet_ni *ni, lnet_nid_t nid, time64_t *when)
 {
-	unsigned long last_alive = 0;
-	unsigned long now = jiffies;
+	time64_t last_alive = 0;
+	time64_t now = ktime_get_seconds();
 	rwlock_t *glock = &kiblnd_data.kib_global_lock;
 	struct kib_peer *peer;
 	unsigned long flags;
@@ -1066,9 +1065,9 @@ static void kiblnd_query(struct lnet_ni *ni, lnet_nid_t nid,
 	if (!peer)
 		kiblnd_launch_tx(ni, NULL, nid);
 
-	CDEBUG(D_NET, "Peer %s %p, alive %ld secs ago\n",
+	CDEBUG(D_NET, "Peer %s %p, alive %lld secs ago\n",
 	       libcfs_nid2str(nid), peer,
-	       last_alive ? (now - last_alive) / HZ : -1);
+	       last_alive ? now - last_alive : -1);
 }
 
 static void kiblnd_free_pages(struct kib_pages *p)
diff --git a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.h b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.h
index 217503f..1b141cc 100644
--- a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.h
+++ b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.h
@@ -571,8 +571,8 @@ struct kib_peer {
 					  * round robin */
 	struct list_head ibp_tx_queue;    /* msgs waiting for a conn */
 	__u64            ibp_incarnation; /* incarnation of peer */
-	/* when (in jiffies) I was last alive */
-	unsigned long		ibp_last_alive;
+	/* when (in seconds) I was last alive */
+	time64_t		ibp_last_alive;
 	/* # users */
 	atomic_t		ibp_refcount;
 	/* version of peer */
diff --git a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c
index 47eb8b4..b046f00 100644
--- a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c
+++ b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c
@@ -1847,8 +1847,8 @@ static int kiblnd_resolve_addr(struct rdma_cm_id *cmid,
 static void
 kiblnd_peer_alive(struct kib_peer *peer)
 {
-	/* This is racy, but everyone's only writing jiffies */
-	peer->ibp_last_alive = jiffies;
+	/* This is racy, but everyone's only writing ktime_get_seconds() */
+	peer->ibp_last_alive = ktime_get_seconds();
 	mb();
 }
 
@@ -1856,7 +1856,7 @@ static int kiblnd_resolve_addr(struct rdma_cm_id *cmid,
 kiblnd_peer_notify(struct kib_peer *peer)
 {
 	int error = 0;
-	unsigned long last_alive = 0;
+	time64_t last_alive = 0;
 	unsigned long flags;
 
 	read_lock_irqsave(&kiblnd_data.kib_global_lock, flags);
@@ -3216,9 +3216,9 @@ static int kiblnd_resolve_addr(struct rdma_cm_id *cmid,
 			}
 
 			if (timedout) {
-				CERROR("Timed out RDMA with %s (%lu): c: %u, oc: %u, rc: %u\n",
+				CERROR("Timed out RDMA with %s (%lld): c: %u, oc: %u, rc: %u\n",
 				       libcfs_nid2str(peer->ibp_nid),
-				       (jiffies - peer->ibp_last_alive) / HZ,
+				       ktime_get_seconds() - peer->ibp_last_alive,
 				       conn->ibc_credits,
 				       conn->ibc_outstanding_credits,
 				       conn->ibc_reserved_credits);
diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c
index 72c98ee..c6e74ee 100644
--- a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c
+++ b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c
@@ -1537,7 +1537,7 @@ struct ksock_peer *
 
 	if (notify)
 		lnet_notify(peer->ksnp_ni, peer->ksnp_id.nid, 0,
-			    last_alive * HZ);
+			    last_alive);
 }
 
 void
@@ -1828,7 +1828,7 @@ struct ksock_peer *
 }
 
 void
-ksocknal_query(struct lnet_ni *ni, lnet_nid_t nid, unsigned long *when)
+ksocknal_query(struct lnet_ni *ni, lnet_nid_t nid, time64_t *when)
 {
 	int connect = 1;
 	time64_t last_alive = 0;
diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.h b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.h
index 20f5de6..2a61929 100644
--- a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.h
+++ b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.h
@@ -658,7 +658,7 @@ int  ksocknal_launch_packet(struct lnet_ni *ni, struct ksock_tx *tx,
 void ksocknal_queue_tx_locked(struct ksock_tx *tx, struct ksock_conn *conn);
 void ksocknal_txlist_done(struct lnet_ni *ni, struct list_head *txlist, int error);
 void ksocknal_notify(struct lnet_ni *ni, lnet_nid_t gw_nid, int alive);
-void ksocknal_query(struct lnet_ni *ni, lnet_nid_t nid, unsigned long *when);
+void ksocknal_query(struct lnet_ni *ni, lnet_nid_t nid, time64_t *when);
 int ksocknal_thread_start(int (*fn)(void *arg), void *arg, char *name);
 void ksocknal_thread_fini(void);
 void ksocknal_launch_all_connections_locked(struct ksock_peer *peer);
diff --git a/drivers/staging/lustre/lnet/lnet/api-ni.c b/drivers/staging/lustre/lnet/lnet/api-ni.c
index 20dce34..d8d76ab 100644
--- a/drivers/staging/lustre/lnet/lnet/api-ni.c
+++ b/drivers/staging/lustre/lnet/lnet/api-ni.c
@@ -1888,7 +1888,6 @@ void lnet_lib_exit(void)
 	struct lnet_process_id id = {0};
 	struct lnet_ni *ni;
 	int rc;
-	unsigned long secs_passed;
 
 	BUILD_BUG_ON(LIBCFS_IOC_DATA_MAX <
 		     sizeof(struct lnet_ioctl_net_config) +
@@ -2028,12 +2027,17 @@ void lnet_lib_exit(void)
 			&peer_info->pr_lnd_u.pr_peer_credits.cr_peer_tx_qnob);
 	}
 
-	case IOC_LIBCFS_NOTIFY_ROUTER:
-		secs_passed = (ktime_get_real_seconds() - data->ioc_u64[0]);
-		secs_passed *= msecs_to_jiffies(MSEC_PER_SEC);
+	case IOC_LIBCFS_NOTIFY_ROUTER: {
+		time64_t deadline = ktime_get_real_seconds() - data->ioc_u64[0];
 
+		/* The deadline passed in by the user should be some time in
+		 * seconds in the future since the UNIX epoch. We have to map
+		 * that deadline to the wall clock.
+		 */
+		deadline += ktime_get_seconds();
 		return lnet_notify(NULL, data->ioc_nid, data->ioc_flags,
-				   jiffies - secs_passed);
+				   deadline);
+	}
 
 	case IOC_LIBCFS_LNET_DIST:
 		rc = LNetDist(data->ioc_nid, &data->ioc_nid, &data->ioc_u32[1]);
diff --git a/drivers/staging/lustre/lnet/lnet/lib-move.c b/drivers/staging/lustre/lnet/lnet/lib-move.c
index f8eaf8f..e8092b1 100644
--- a/drivers/staging/lustre/lnet/lnet/lib-move.c
+++ b/drivers/staging/lustre/lnet/lnet/lib-move.c
@@ -492,7 +492,7 @@
 static void
 lnet_ni_query_locked(struct lnet_ni *ni, struct lnet_peer *lp)
 {
-	unsigned long last_alive = 0;
+	time64_t last_alive = 0;
 
 	LASSERT(lnet_peer_aliveness_enabled(lp));
 	LASSERT(ni->ni_lnd->lnd_query);
@@ -501,7 +501,7 @@
 	ni->ni_lnd->lnd_query(ni, lp->lp_nid, &last_alive);
 	lnet_net_lock(lp->lp_cpt);
 
-	lp->lp_last_query = jiffies;
+	lp->lp_last_query = ktime_get_seconds();
 
 	if (last_alive) /* NI has updated timestamp */
 		lp->lp_last_alive = last_alive;
@@ -512,7 +512,7 @@
 lnet_peer_is_alive(struct lnet_peer *lp, unsigned long now)
 {
 	int alive;
-	unsigned long deadline;
+	time64_t deadline;
 
 	LASSERT(lnet_peer_aliveness_enabled(lp));
 
@@ -520,11 +520,11 @@
 	 * ignore the initial assumed death (see lnet_peers_start_down()).
 	 */
 	if (!lp->lp_alive && lp->lp_alive_count > 0 &&
-	    time_after_eq(lp->lp_timestamp, lp->lp_last_alive))
+	    lp->lp_timestamp >= lp->lp_last_alive)
 		return 0;
 
-	deadline = lp->lp_last_alive + lp->lp_ni->ni_peertimeout * HZ;
-	alive = time_after(deadline, now);
+	deadline = lp->lp_last_alive + lp->lp_ni->ni_peertimeout;
+	alive = deadline > now;
 
 	/* Update obsolete lp_alive except for routers assumed to be dead
 	 * initially, because router checker would update aliveness in this
@@ -544,7 +544,7 @@
 static int
 lnet_peer_alive_locked(struct lnet_peer *lp)
 {
-	unsigned long now = jiffies;
+	time64_t now = ktime_get_seconds();
 
 	if (!lnet_peer_aliveness_enabled(lp))
 		return -ENODEV;
@@ -558,15 +558,15 @@
 	 */
 	if (lp->lp_last_query) {
 		static const int lnet_queryinterval = 1;
+		time64_t next_query;
 
-		unsigned long next_query =
-			   lp->lp_last_query + lnet_queryinterval * HZ;
+		next_query = lp->lp_last_query + lnet_queryinterval;
 
-		if (time_before(now, next_query)) {
+		if (now < next_query) {
 			if (lp->lp_alive)
-				CWARN("Unexpected aliveness of peer %s: %d < %d (%d/%d)\n",
+				CWARN("Unexpected aliveness of peer %s: %lld < %lld (%d/%d)\n",
 				      libcfs_nid2str(lp->lp_nid),
-				      (int)now, (int)next_query,
+				      now, next_query,
 				      lnet_queryinterval,
 				      lp->lp_ni->ni_peertimeout);
 			return 0;
diff --git a/drivers/staging/lustre/lnet/lnet/net_fault.c b/drivers/staging/lustre/lnet/lnet/net_fault.c
index 0066394..cb1f905 100644
--- a/drivers/staging/lustre/lnet/lnet/net_fault.c
+++ b/drivers/staging/lustre/lnet/lnet/net_fault.c
@@ -57,9 +57,9 @@ struct lnet_drop_rule {
 	/**
 	 * seconds to drop the next message, it's exclusive with dr_drop_at
 	 */
-	unsigned long		dr_drop_time;
+	time64_t		dr_drop_time;
 	/** baseline to caculate dr_drop_time */
-	unsigned long		dr_time_base;
+	time64_t		dr_time_base;
 	/** statistic of dropped messages */
 	struct lnet_fault_stat	dr_stat;
 };
@@ -169,9 +169,10 @@ struct lnet_drop_rule {
 
 	rule->dr_attr = *attr;
 	if (attr->u.drop.da_interval) {
-		rule->dr_time_base = jiffies + attr->u.drop.da_interval * HZ;
-		rule->dr_drop_time = jiffies +
-			prandom_u32_max(attr->u.drop.da_interval) * HZ;
+		rule->dr_time_base = ktime_get_seconds() +
+				     attr->u.drop.da_interval;
+		rule->dr_drop_time = ktime_get_seconds() +
+				     prandom_u32_max(attr->u.drop.da_interval);
 	} else {
 		rule->dr_drop_at = prandom_u32_max(attr->u.drop.da_rate);
 	}
@@ -279,9 +280,10 @@ struct lnet_drop_rule {
 		if (attr->u.drop.da_rate) {
 			rule->dr_drop_at = prandom_u32_max(attr->u.drop.da_rate);
 		} else {
-			rule->dr_drop_time = jiffies +
-				prandom_u32_max(attr->u.drop.da_interval) * HZ;
-			rule->dr_time_base = jiffies + attr->u.drop.da_interval * HZ;
+			rule->dr_drop_time = ktime_get_seconds() +
+				prandom_u32_max(attr->u.drop.da_interval);
+			rule->dr_time_base = ktime_get_seconds() +
+					     attr->u.drop.da_interval;
 		}
 		spin_unlock(&rule->dr_lock);
 	}
@@ -306,19 +308,19 @@ struct lnet_drop_rule {
 	/* match this rule, check drop rate now */
 	spin_lock(&rule->dr_lock);
 	if (rule->dr_drop_time) { /* time based drop */
-		unsigned long now = jiffies;
+		time64_t now = ktime_get_seconds();
 
 		rule->dr_stat.fs_count++;
-		drop = time_after_eq(now, rule->dr_drop_time);
+		drop = now >= rule->dr_drop_time;
 		if (drop) {
-			if (time_after(now, rule->dr_time_base))
+			if (now > rule->dr_time_base)
 				rule->dr_time_base = now;
 
 			rule->dr_drop_time = rule->dr_time_base +
-				prandom_u32_max(attr->u.drop.da_interval) * HZ;
-			rule->dr_time_base += attr->u.drop.da_interval * HZ;
+				prandom_u32_max(attr->u.drop.da_interval);
+			rule->dr_time_base += attr->u.drop.da_interval;
 
-			CDEBUG(D_NET, "Drop Rule %s->%s: next drop : %lu\n",
+			CDEBUG(D_NET, "Drop Rule %s->%s: next drop : %lld\n",
 			       libcfs_nid2str(attr->fa_src),
 			       libcfs_nid2str(attr->fa_dst),
 			       rule->dr_drop_time);
@@ -404,9 +406,9 @@ struct lnet_delay_rule {
 	/**
 	 * seconds to delay the next message, it's exclusive with dl_delay_at
 	 */
-	unsigned long		dl_delay_time;
+	time64_t		dl_delay_time;
 	/** baseline to caculate dl_delay_time */
-	unsigned long		dl_time_base;
+	time64_t		dl_time_base;
 	/** jiffies to send the next delayed message */
 	unsigned long		dl_msg_send;
 	/** delayed message list */
@@ -436,12 +438,6 @@ struct delay_daemon_data {
 
 static struct delay_daemon_data	delay_dd;
 
-static unsigned long
-round_timeout(unsigned long timeout)
-{
-	return (unsigned int)rounddown(timeout, HZ) + HZ;
-}
-
 static void
 delay_rule_decref(struct lnet_delay_rule *rule)
 {
@@ -472,19 +468,19 @@ struct delay_daemon_data {
 	/* match this rule, check delay rate now */
 	spin_lock(&rule->dl_lock);
 	if (rule->dl_delay_time) { /* time based delay */
-		unsigned long now = jiffies;
+		time64_t now = ktime_get_seconds();
 
 		rule->dl_stat.fs_count++;
-		delay = time_after_eq(now, rule->dl_delay_time);
+		delay = now >= rule->dl_delay_time;
 		if (delay) {
-			if (time_after(now, rule->dl_time_base))
+			if (now > rule->dl_time_base)
 				rule->dl_time_base = now;
 
 			rule->dl_delay_time = rule->dl_time_base +
-				prandom_u32_max(attr->u.delay.la_interval) * HZ;
-			rule->dl_time_base += attr->u.delay.la_interval * HZ;
+				prandom_u32_max(attr->u.delay.la_interval);
+			rule->dl_time_base += attr->u.delay.la_interval;
 
-			CDEBUG(D_NET, "Delay Rule %s->%s: next delay : %lu\n",
+			CDEBUG(D_NET, "Delay Rule %s->%s: next delay : %lld\n",
 			       libcfs_nid2str(attr->fa_src),
 			       libcfs_nid2str(attr->fa_dst),
 			       rule->dl_delay_time);
@@ -512,8 +508,7 @@ struct delay_daemon_data {
 	rule->dl_stat.u.delay.ls_delayed++;
 
 	list_add_tail(&msg->msg_list, &rule->dl_msg_list);
-	msg->msg_delay_send = round_timeout(
-			jiffies + attr->u.delay.la_latency * HZ);
+	msg->msg_delay_send = ktime_get_seconds() + attr->u.delay.la_latency;
 	if (rule->dl_msg_send == -1) {
 		rule->dl_msg_send = msg->msg_delay_send;
 		mod_timer(&rule->dl_timer, rule->dl_msg_send);
@@ -562,7 +557,7 @@ struct delay_daemon_data {
 {
 	struct lnet_msg *msg;
 	struct lnet_msg *tmp;
-	unsigned long now = jiffies;
+	time64_t now = ktime_get_seconds();
 
 	if (!all && rule->dl_msg_send > now)
 		return;
@@ -767,9 +762,10 @@ struct delay_daemon_data {
 
 	rule->dl_attr = *attr;
 	if (attr->u.delay.la_interval) {
-		rule->dl_time_base = jiffies + attr->u.delay.la_interval * HZ;
-		rule->dl_delay_time = jiffies + 
-			prandom_u32_max(attr->u.delay.la_interval) * HZ;
+		rule->dl_time_base = ktime_get_seconds() +
+				     attr->u.delay.la_interval;
+		rule->dl_delay_time = ktime_get_seconds() +
+				      prandom_u32_max(attr->u.delay.la_interval);
 	} else {
 		rule->dl_delay_at = prandom_u32_max(attr->u.delay.la_rate);
 	}
@@ -919,10 +915,10 @@ struct delay_daemon_data {
 		if (attr->u.delay.la_rate) {
 			rule->dl_delay_at = prandom_u32_max(attr->u.delay.la_rate);
 		} else {
-			rule->dl_delay_time =
-				jiffies + prandom_u32_max(
-					attr->u.delay.la_interval) * HZ;
-			rule->dl_time_base = jiffies + attr->u.delay.la_interval * HZ;
+			rule->dl_delay_time = ktime_get_seconds() +
+				prandom_u32_max(attr->u.delay.la_interval);
+			rule->dl_time_base = ktime_get_seconds() +
+					     attr->u.delay.la_interval;
 		}
 		spin_unlock(&rule->dl_lock);
 	}
diff --git a/drivers/staging/lustre/lnet/lnet/peer.c b/drivers/staging/lustre/lnet/lnet/peer.c
index 5829414..6ce175d 100644
--- a/drivers/staging/lustre/lnet/lnet/peer.c
+++ b/drivers/staging/lustre/lnet/lnet/peer.c
@@ -315,7 +315,7 @@ struct lnet_peer *
 	lp->lp_alive_count = 0;
 	lp->lp_timestamp = 0;
 	lp->lp_alive = !lnet_peers_start_down(); /* 1 bit!! */
-	lp->lp_last_alive = jiffies; /* assumes alive */
+	lp->lp_last_alive = ktime_get_seconds(); /* assumes alive */
 	lp->lp_last_query = 0; /* haven't asked NI yet */
 	lp->lp_ping_timestamp = 0;
 	lp->lp_ping_feats = LNET_PING_FEAT_INVAL;
diff --git a/drivers/staging/lustre/lnet/lnet/router.c b/drivers/staging/lustre/lnet/lnet/router.c
index 6267d5e..3f84df6 100644
--- a/drivers/staging/lustre/lnet/lnet/router.c
+++ b/drivers/staging/lustre/lnet/lnet/router.c
@@ -101,9 +101,9 @@
 
 void
 lnet_notify_locked(struct lnet_peer *lp, int notifylnd, int alive,
-		   unsigned long when)
+		   time64_t when)
 {
-	if (time_before(when, lp->lp_timestamp)) { /* out of date information */
+	if (lp->lp_timestamp > when) { /* out of date information */
 		CDEBUG(D_NET, "Out of date\n");
 		return;
 	}
@@ -765,7 +765,7 @@ int lnet_get_rtr_pool_cfg(int idx, struct lnet_ioctl_pool_cfg *pool_cfg)
 	 * we ping alive routers to try to detect router death before
 	 * apps get burned).
 	 */
-	lnet_notify_locked(lp, 1, !event->status, jiffies);
+	lnet_notify_locked(lp, 1, !event->status, ktime_get_seconds());
 
 	/*
 	 * The router checker will wake up very shortly and do the
@@ -832,7 +832,7 @@ int lnet_get_rtr_pool_cfg(int idx, struct lnet_ioctl_pool_cfg *pool_cfg)
 {
 	struct lnet_ni *ni;
 	time64_t now;
-	int timeout;
+	time64_t timeout;
 
 	LASSERT(the_lnet.ln_routing);
 
@@ -857,7 +857,7 @@ int lnet_get_rtr_pool_cfg(int idx, struct lnet_ioctl_pool_cfg *pool_cfg)
 		LASSERT(ni->ni_status);
 
 		if (ni->ni_status->ns_status != LNET_NI_STATUS_DOWN) {
-			CDEBUG(D_NET, "NI(%s:%d) status changed to down\n",
+			CDEBUG(D_NET, "NI(%s:%lld) status changed to down\n",
 			       libcfs_nid2str(ni->ni_nid), timeout);
 			/*
 			 * NB: so far, this is the only place to set
@@ -976,13 +976,13 @@ int lnet_get_rtr_pool_cfg(int idx, struct lnet_ioctl_pool_cfg *pool_cfg)
 lnet_ping_router_locked(struct lnet_peer *rtr)
 {
 	struct lnet_rc_data *rcd = NULL;
-	unsigned long now = jiffies;
-	int secs;
+	time64_t now = ktime_get_seconds();
+	time64_t secs;
 
 	lnet_peer_addref_locked(rtr);
 
 	if (rtr->lp_ping_deadline && /* ping timed out? */
-	    time_after(now, rtr->lp_ping_deadline))
+	    now > rtr->lp_ping_deadline)
 		lnet_notify_locked(rtr, 1, 0, now);
 
 	/* Run any outstanding notifications */
@@ -1004,13 +1004,13 @@ int lnet_get_rtr_pool_cfg(int idx, struct lnet_ioctl_pool_cfg *pool_cfg)
 	secs = lnet_router_check_interval(rtr);
 
 	CDEBUG(D_NET,
-	       "rtr %s %d: deadline %lu ping_notsent %d alive %d alive_count %d lp_ping_timestamp %lu\n",
+	       "rtr %s %lldd: deadline %lld ping_notsent %d alive %d alive_count %d lp_ping_timestamp %lld\n",
 	       libcfs_nid2str(rtr->lp_nid), secs,
 	       rtr->lp_ping_deadline, rtr->lp_ping_notsent,
 	       rtr->lp_alive, rtr->lp_alive_count, rtr->lp_ping_timestamp);
 
 	if (secs && !rtr->lp_ping_notsent &&
-	    time_after(now, rtr->lp_ping_timestamp + secs * HZ)) {
+	    now > rtr->lp_ping_timestamp + secs) {
 		int rc;
 		struct lnet_process_id id;
 		struct lnet_handle_md mdh;
@@ -1025,8 +1025,8 @@ int lnet_get_rtr_pool_cfg(int idx, struct lnet_ioctl_pool_cfg *pool_cfg)
 		mdh = rcd->rcd_mdh;
 
 		if (!rtr->lp_ping_deadline) {
-			rtr->lp_ping_deadline =
-				jiffies + router_ping_timeout * HZ;
+			rtr->lp_ping_deadline = ktime_get_seconds() +
+						router_ping_timeout;
 		}
 
 		lnet_net_unlock(rtr->lp_cpt);
@@ -1726,10 +1726,10 @@ int lnet_get_rtr_pool_cfg(int idx, struct lnet_ioctl_pool_cfg *pool_cfg)
 }
 
 int
-lnet_notify(struct lnet_ni *ni, lnet_nid_t nid, int alive, unsigned long when)
+lnet_notify(struct lnet_ni *ni, lnet_nid_t nid, int alive, time64_t when)
 {
 	struct lnet_peer *lp = NULL;
-	unsigned long now = jiffies;
+	time64_t now = ktime_get_seconds();
 	int cpt = lnet_cpt_of_nid(nid);
 
 	LASSERT(!in_interrupt());
@@ -1748,11 +1748,10 @@ int lnet_get_rtr_pool_cfg(int idx, struct lnet_ioctl_pool_cfg *pool_cfg)
 	}
 
 	/* can't do predictions... */
-	if (time_after(when, now)) {
-		CWARN("Ignoring prediction from %s of %s %s %ld seconds in the future\n",
+	if (when > now) {
+		CWARN("Ignoring prediction from %s of %s %s %lld seconds in the future\n",
 		      !ni ? "userspace" : libcfs_nid2str(ni->ni_nid),
-		      libcfs_nid2str(nid), alive ? "up" : "down",
-		      (when - now) / HZ);
+		      libcfs_nid2str(nid), alive ? "up" : "down", when - now);
 		return -EINVAL;
 	}
 
diff --git a/drivers/staging/lustre/lnet/lnet/router_proc.c b/drivers/staging/lustre/lnet/lnet/router_proc.c
index ae4b7f5..c0d30970 100644
--- a/drivers/staging/lustre/lnet/lnet/router_proc.c
+++ b/drivers/staging/lustre/lnet/lnet/router_proc.c
@@ -323,14 +323,14 @@ static int proc_lnet_routers(struct ctl_table *table, int write,
 
 		if (peer) {
 			lnet_nid_t nid = peer->lp_nid;
-			unsigned long now = jiffies;
-			unsigned long deadline = peer->lp_ping_deadline;
+			time64_t now = ktime_get_seconds();
+			time64_t deadline = peer->lp_ping_deadline;
 			int nrefs = peer->lp_refcount;
 			int nrtrrefs = peer->lp_rtr_refcount;
 			int alive_cnt = peer->lp_alive_count;
 			int alive = peer->lp_alive;
 			int pingsent = !peer->lp_ping_notsent;
-			int last_ping = (now - peer->lp_ping_timestamp) / HZ;
+			time64_t last_ping = now - peer->lp_ping_timestamp;
 			int down_ni = 0;
 			struct lnet_route *rtr;
 
@@ -351,18 +351,17 @@ static int proc_lnet_routers(struct ctl_table *table, int write,
 
 			if (!deadline)
 				s += snprintf(s, tmpstr + tmpsiz - s,
-					      "%-4d %7d %9d %6s %12d %9d %8s %7d %s\n",
+					      "%-4d %7d %9d %6s %12llu %9d %8s %7d %s\n",
 					      nrefs, nrtrrefs, alive_cnt,
 					      alive ? "up" : "down", last_ping,
 					      pingsent, "NA", down_ni,
 					      libcfs_nid2str(nid));
 			else
 				s += snprintf(s, tmpstr + tmpsiz - s,
-					      "%-4d %7d %9d %6s %12d %9d %8lu %7d %s\n",
+					      "%-4d %7d %9d %6s %12llu %9d %8llu %7d %s\n",
 					      nrefs, nrtrrefs, alive_cnt,
 					      alive ? "up" : "down", last_ping,
-					      pingsent,
-					      (deadline - now) / HZ,
+					      pingsent, deadline - now,
 					      down_ni, libcfs_nid2str(nid));
 			LASSERT(tmpstr + tmpsiz - s > 0);
 		}
@@ -493,7 +492,7 @@ static int proc_lnet_peers(struct ctl_table *table, int write,
 		if (peer) {
 			lnet_nid_t nid = peer->lp_nid;
 			int nrefs = peer->lp_refcount;
-			int lastalive = -1;
+			time64_t lastalive = -1;
 			char *aliveness = "NA";
 			int maxcr = peer->lp_ni->ni_peertxcredits;
 			int txcr = peer->lp_txcredits;
@@ -507,11 +506,9 @@ static int proc_lnet_peers(struct ctl_table *table, int write,
 				aliveness = peer->lp_alive ? "up" : "down";
 
 			if (lnet_peer_aliveness_enabled(peer)) {
-				unsigned long now = jiffies;
-				long delta;
+				time64_t now = ktime_get_seconds();
 
-				delta = now - peer->lp_last_alive;
-				lastalive = (delta) / HZ;
+				lastalive = now - peer->lp_last_alive;
 
 				/* No need to mess up peers contents with
 				 * arbitrarily long integers - it suffices to
@@ -524,7 +521,7 @@ static int proc_lnet_peers(struct ctl_table *table, int write,
 			lnet_net_unlock(cpt);
 
 			s += snprintf(s, tmpstr + tmpsiz - s,
-				      "%-24s %4d %5s %5d %5d %5d %5d %5d %5d %d\n",
+				      "%-24s %4d %5s %5lld %5d %5d %5d %5d %5d %d\n",
 				      libcfs_nid2str(nid), nrefs, aliveness,
 				      lastalive, maxcr, rtrcr, minrtrcr, txcr,
 				      mintxcr, txqnob);
@@ -679,7 +676,7 @@ static int proc_lnet_nis(struct ctl_table *table, int write,
 			struct lnet_tx_queue *tq;
 			char *stat;
 			time64_t now = ktime_get_real_seconds();
-			int last_alive = -1;
+			time64_t last_alive = -1;
 			int i;
 			int j;
 
@@ -714,7 +711,7 @@ static int proc_lnet_nis(struct ctl_table *table, int write,
 					lnet_net_lock(i);
 
 				s += snprintf(s, tmpstr + tmpsiz - s,
-					      "%-24s %6s %5d %4d %4d %4d %5d %5d %5d\n",
+					      "%-24s %6s %5lld %4d %4d %4d %5d %5d %5d\n",
 					      libcfs_nid2str(ni->ni_nid), stat,
 					      last_alive, *ni->ni_refs[i],
 					      ni->ni_peertxcredits,
-- 
1.8.3.1


From jsimmons at infradead.org  Sun Jun 24 21:53:51 2018
From: jsimmons at infradead.org (James Simmons)
Date: Sun, 24 Jun 2018 17:53:51 -0400
Subject: [lustre-devel] [PATCH 7/8] staging: lustre: ko2iblnd: remove
 remaining jiffy use
In-Reply-To: <1529877232-15582-1-git-send-email-jsimmons@infradead.org>
References: <1529877232-15582-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1529877232-15582-8-git-send-email-jsimmons@infradead.org>

Convert the remaining jiffy use in ko2iblnd to time64_t and ktime_t.
The HZ value can be configured differently for a set of nodes and
this can lead to corner case problems with node to node interactions.
This allows lustre to have consistent behavior across nodes.

Signed-off-by: James Simmons <uja.ornl at yahoo.com>
WC-bug-id: https://jira.whamcloud.com/browse/LU-9019
Reviewed-on: https://review.whamcloud.com/31042
Reviewed-by: Andreas Dilger <adilger at whamcloud.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin at intel.com>
Reviewed-by: Oleg Drokin <green at whamcloud.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 .../staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c    | 37 +++++++++++-----------
 .../staging/lustre/lnet/klnds/o2iblnd/o2iblnd.h    | 24 +++++++-------
 .../staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c | 26 +++++++--------
 3 files changed, 45 insertions(+), 42 deletions(-)

diff --git a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c
index edae4eb..9f3fef5 100644
--- a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c
+++ b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c
@@ -1445,7 +1445,7 @@ static int kiblnd_create_fmr_pool(struct kib_fmr_poolset *fps,
 	if (rc)
 		goto out_fpo;
 
-	fpo->fpo_deadline = jiffies + IBLND_POOL_DEADLINE * HZ;
+	fpo->fpo_deadline = ktime_get_seconds() + IBLND_POOL_DEADLINE;
 	fpo->fpo_owner = fps;
 	*pp_fpo = fpo;
 
@@ -1515,13 +1515,13 @@ static void kiblnd_fini_fmr_poolset(struct kib_fmr_poolset *fps)
 	return rc;
 }
 
-static int kiblnd_fmr_pool_is_idle(struct kib_fmr_pool *fpo, unsigned long now)
+static int kiblnd_fmr_pool_is_idle(struct kib_fmr_pool *fpo, time64_t now)
 {
 	if (fpo->fpo_map_count) /* still in use */
 		return 0;
 	if (fpo->fpo_failed)
 		return 1;
-	return time_after_eq(now, fpo->fpo_deadline);
+	return now >= fpo->fpo_deadline;
 }
 
 static int
@@ -1551,7 +1551,7 @@ void kiblnd_fmr_pool_unmap(struct kib_fmr *fmr, int status)
 	LIST_HEAD(zombies);
 	struct kib_fmr_pool *fpo = fmr->fmr_pool;
 	struct kib_fmr_poolset *fps;
-	unsigned long now = jiffies;
+	time64_t now = ktime_get_seconds();
 	struct kib_fmr_pool *tmp;
 	int rc;
 
@@ -1618,7 +1618,7 @@ int kiblnd_fmr_pool_map(struct kib_fmr_poolset *fps, struct kib_tx *tx,
 	spin_lock(&fps->fps_lock);
 	version = fps->fps_version;
 	list_for_each_entry(fpo, &fps->fps_pool_list, fpo_list) {
-		fpo->fpo_deadline = jiffies + IBLND_POOL_DEADLINE * HZ;
+		fpo->fpo_deadline = ktime_get_seconds() + IBLND_POOL_DEADLINE;
 		fpo->fpo_map_count++;
 
 		if (fpo->fpo_is_fmr) {
@@ -1725,7 +1725,7 @@ int kiblnd_fmr_pool_map(struct kib_fmr_poolset *fps, struct kib_tx *tx,
 		goto again;
 	}
 
-	if (time_before(jiffies, fps->fps_next_retry)) {
+	if (ktime_get_seconds() < fps->fps_next_retry) {
 		/* someone failed recently */
 		spin_unlock(&fps->fps_lock);
 		return -EAGAIN;
@@ -1742,7 +1742,7 @@ int kiblnd_fmr_pool_map(struct kib_fmr_poolset *fps, struct kib_tx *tx,
 		fps->fps_version++;
 		list_add_tail(&fpo->fpo_list, &fps->fps_pool_list);
 	} else {
-		fps->fps_next_retry = jiffies + IBLND_POOL_RETRY * HZ;
+		fps->fps_next_retry = ktime_get_seconds() + IBLND_POOL_RETRY;
 	}
 	spin_unlock(&fps->fps_lock);
 
@@ -1763,7 +1763,7 @@ static void kiblnd_init_pool(struct kib_poolset *ps, struct kib_pool *pool, int
 
 	memset(pool, 0, sizeof(*pool));
 	INIT_LIST_HEAD(&pool->po_free_list);
-	pool->po_deadline = jiffies + IBLND_POOL_DEADLINE * HZ;
+	pool->po_deadline = ktime_get_seconds() + IBLND_POOL_DEADLINE;
 	pool->po_owner    = ps;
 	pool->po_size     = size;
 }
@@ -1843,13 +1843,13 @@ static int kiblnd_init_poolset(struct kib_poolset *ps, int cpt,
 	return rc;
 }
 
-static int kiblnd_pool_is_idle(struct kib_pool *pool, unsigned long now)
+static int kiblnd_pool_is_idle(struct kib_pool *pool, time64_t now)
 {
 	if (pool->po_allocated) /* still in use */
 		return 0;
 	if (pool->po_failed)
 		return 1;
-	return time_after_eq(now, pool->po_deadline);
+	return now >= pool->po_deadline;
 }
 
 void kiblnd_pool_free_node(struct kib_pool *pool, struct list_head *node)
@@ -1857,7 +1857,7 @@ void kiblnd_pool_free_node(struct kib_pool *pool, struct list_head *node)
 	LIST_HEAD(zombies);
 	struct kib_poolset *ps = pool->po_owner;
 	struct kib_pool *tmp;
-	unsigned long now = jiffies;
+	time64_t now = ktime_get_seconds();
 
 	spin_lock(&ps->ps_lock);
 
@@ -1887,7 +1887,7 @@ struct list_head *kiblnd_pool_alloc_node(struct kib_poolset *ps)
 	struct list_head *node;
 	struct kib_pool *pool;
 	unsigned int interval = 1;
-	unsigned long time_before;
+	ktime_t time_before;
 	unsigned int trips = 0;
 	int rc;
 
@@ -1898,7 +1898,8 @@ struct list_head *kiblnd_pool_alloc_node(struct kib_poolset *ps)
 			continue;
 
 		pool->po_allocated++;
-		pool->po_deadline = jiffies + IBLND_POOL_DEADLINE * HZ;
+		pool->po_deadline = ktime_get_seconds() +
+				    IBLND_POOL_DEADLINE;
 		node = pool->po_free_list.next;
 		list_del(node);
 
@@ -1926,7 +1927,7 @@ struct list_head *kiblnd_pool_alloc_node(struct kib_poolset *ps)
 		goto again;
 	}
 
-	if (time_before(jiffies, ps->ps_next_retry)) {
+	if (ktime_get_seconds() < ps->ps_next_retry) {
 		/* someone failed recently */
 		spin_unlock(&ps->ps_lock);
 		return NULL;
@@ -1936,17 +1937,17 @@ struct list_head *kiblnd_pool_alloc_node(struct kib_poolset *ps)
 	spin_unlock(&ps->ps_lock);
 
 	CDEBUG(D_NET, "%s pool exhausted, allocate new pool\n", ps->ps_name);
-	time_before = jiffies;
+	time_before = ktime_get();
 	rc = ps->ps_pool_create(ps, ps->ps_pool_size, &pool);
-	CDEBUG(D_NET, "ps_pool_create took %lu HZ to complete",
-	       jiffies - time_before);
+	CDEBUG(D_NET, "ps_pool_create took %lld ms to complete",
+	       ktime_ms_delta(ktime_get(), time_before));
 
 	spin_lock(&ps->ps_lock);
 	ps->ps_increasing = 0;
 	if (!rc) {
 		list_add_tail(&pool->po_list, &ps->ps_pool_list);
 	} else {
-		ps->ps_next_retry = jiffies + IBLND_POOL_RETRY * HZ;
+		ps->ps_next_retry = ktime_get_seconds() + IBLND_POOL_RETRY;
 		CERROR("Can't allocate new %s pool because out of memory\n",
 		       ps->ps_name);
 	}
diff --git a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.h b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.h
index 1b141cc..635d7c6 100644
--- a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.h
+++ b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.h
@@ -155,7 +155,7 @@ struct kib_dev {
 	char               ibd_ifname[KIB_IFNAME_SIZE];
 	int                ibd_nnets;           /* # nets extant */
 
-	unsigned long      ibd_next_failover;
+	time64_t	   ibd_next_failover;
 	int                ibd_failed_failover; /* # failover failures */
 	unsigned int       ibd_failover;        /* failover in progress */
 	unsigned int ibd_can_failover; /* IPoIB interface is a bonding master */
@@ -205,7 +205,7 @@ struct kib_poolset {
 	char                  ps_name[IBLND_POOL_NAME_LEN]; /* pool set name */
 	struct list_head      ps_pool_list;       /* list of pools */
 	struct list_head      ps_failed_pool_list;/* failed pool list */
-	unsigned long         ps_next_retry;      /* time stamp for retry if */
+	time64_t	      ps_next_retry;	  /* time stamp for retry if */
 						  /* failed to allocate */
 	int                   ps_increasing;      /* is allocating new pool */
 	int                   ps_pool_size;       /* new pool size */
@@ -221,7 +221,7 @@ struct kib_pool {
 	struct list_head      po_list;       /* chain on pool list */
 	struct list_head      po_free_list;  /* pre-allocated node */
 	struct kib_poolset	*po_owner;	/* pool_set of this pool */
-	unsigned long         po_deadline;   /* deadline of this pool */
+	time64_t		po_deadline;	/* deadline of this pool */
 	int                   po_allocated;  /* # of elements in use */
 	int                   po_failed;     /* pool is created on failed HCA */
 	int                   po_size;       /* # of pre-allocated elements */
@@ -250,8 +250,9 @@ struct kib_fmr_poolset {
 	int                   fps_flush_trigger;
 	int		      fps_cache;
 	int                   fps_increasing;      /* is allocating new pool */
-	unsigned long         fps_next_retry;      /* time stamp for retry if*/
-						   /* failed to allocate */
+	time64_t		fps_next_retry;		/* time stamp for retry
+							 * if failed to allocate
+							 */
 };
 
 struct kib_fast_reg_descriptor { /* For fast registration */
@@ -275,7 +276,7 @@ struct kib_fmr_pool {
 			int		    fpo_pool_size;
 		} fast_reg;
 	};
-	unsigned long         fpo_deadline;        /* deadline of this pool */
+	time64_t		fpo_deadline;	/* deadline of this pool */
 	int                   fpo_failed;          /* fmr pool is failed */
 	int                   fpo_map_count;       /* # of mapped FMR */
 	int		      fpo_is_fmr;
@@ -487,7 +488,7 @@ struct kib_tx {					/* transmit message */
 	short                 tx_queued;      /* queued for sending */
 	short                 tx_waiting;     /* waiting for peer */
 	int                   tx_status;      /* LNET completion status */
-	unsigned long         tx_deadline;    /* completion deadline */
+	ktime_t			tx_deadline;	/* completion deadline */
 	__u64                 tx_cookie;      /* completion cookie */
 	struct lnet_msg		*tx_lntmsg[2];	/* lnet msgs to finalize on completion */
 	struct kib_msg	      *tx_msg;        /* message buffer (host vaddr) */
@@ -533,7 +534,7 @@ struct kib_conn {
 	unsigned int          ibc_nrx:16;      /* receive buffers owned */
 	unsigned int          ibc_scheduled:1; /* scheduled for attention */
 	unsigned int          ibc_ready:1;     /* CQ callback fired */
-	unsigned long         ibc_last_send;   /* time of last send */
+	ktime_t			ibc_last_send;	/* time of last send */
 	struct list_head      ibc_connd_list;  /* link chain for */
 					       /* kiblnd_check_conns only */
 	struct list_head ibc_early_rxs; /* rxs completed before ESTABLISHED */
@@ -763,10 +764,11 @@ struct kib_peer {
 static inline int
 kiblnd_send_keepalive(struct kib_conn *conn)
 {
+	s64 keepalive_ns = *kiblnd_tunables.kib_keepalive * NSEC_PER_SEC;
+
 	return (*kiblnd_tunables.kib_keepalive > 0) &&
-		time_after(jiffies, conn->ibc_last_send +
-			   msecs_to_jiffies(*kiblnd_tunables.kib_keepalive *
-					    MSEC_PER_SEC));
+		ktime_after(ktime_get(),
+			    ktime_add_ns(conn->ibc_last_send, keepalive_ns));
 }
 
 static inline int
diff --git a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c
index b046f00..e4e8c3b 100644
--- a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c
+++ b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c
@@ -852,7 +852,7 @@ static int kiblnd_map_tx(struct lnet_ni *ni, struct kib_tx *tx,
 		rc = ib_post_send(conn->ibc_cmid->qp, wrq, &bad);
 	}
 
-	conn->ibc_last_send = jiffies;
+	conn->ibc_last_send = ktime_get();
 
 	if (!rc)
 		return 0;
@@ -1137,15 +1137,15 @@ static int kiblnd_map_tx(struct lnet_ni *ni, struct kib_tx *tx,
 kiblnd_queue_tx_locked(struct kib_tx *tx, struct kib_conn *conn)
 {
 	struct list_head *q;
+	s64 timeout_ns;
 
 	LASSERT(tx->tx_nwrq > 0);	      /* work items set up */
 	LASSERT(!tx->tx_queued);	       /* not queued for sending already */
 	LASSERT(conn->ibc_state >= IBLND_CONN_ESTABLISHED);
 
+	timeout_ns = *kiblnd_tunables.kib_timeout * NSEC_PER_SEC;
 	tx->tx_queued = 1;
-	tx->tx_deadline = jiffies +
-			  msecs_to_jiffies(*kiblnd_tunables.kib_timeout *
-					   MSEC_PER_SEC);
+	tx->tx_deadline = ktime_add_ns(ktime_get(), timeout_ns);
 
 	if (!tx->tx_conn) {
 		kiblnd_conn_addref(conn);
@@ -2135,7 +2135,7 @@ static int kiblnd_resolve_addr(struct rdma_cm_id *cmid,
 	/* connection established */
 	write_lock_irqsave(&kiblnd_data.kib_global_lock, flags);
 
-	conn->ibc_last_send = jiffies;
+	conn->ibc_last_send = ktime_get();
 	kiblnd_set_conn_state(conn, IBLND_CONN_ESTABLISHED);
 	kiblnd_peer_alive(peer);
 
@@ -3153,10 +3153,11 @@ static int kiblnd_resolve_addr(struct rdma_cm_id *cmid,
 			LASSERT(tx->tx_waiting || tx->tx_sending);
 		}
 
-		if (time_after_eq(jiffies, tx->tx_deadline)) {
-			CERROR("Timed out tx: %s, %lu seconds\n",
+		if (ktime_compare(ktime_get(), tx->tx_deadline) >= 0) {
+			CERROR("Timed out tx: %s, %lld seconds\n",
 			       kiblnd_queue2str(conn, txs),
-			       (jiffies - tx->tx_deadline) / HZ);
+			       ktime_ms_delta(ktime_get(),
+					      tx->tx_deadline) / MSEC_PER_SEC);
 			return 1;
 		}
 	}
@@ -3692,8 +3693,7 @@ static int kiblnd_resolve_addr(struct rdma_cm_id *cmid,
 
 		list_for_each_entry(dev, &kiblnd_data.kib_failed_devs,
 				    ibd_fail_list) {
-			if (time_before(jiffies,
-					dev->ibd_next_failover))
+			if (ktime_get_seconds() < dev->ibd_next_failover)
 				continue;
 			do_failover = 1;
 			break;
@@ -3711,13 +3711,13 @@ static int kiblnd_resolve_addr(struct rdma_cm_id *cmid,
 			LASSERT(dev->ibd_failover);
 			dev->ibd_failover = 0;
 			if (rc >= 0) { /* Device is OK or failover succeed */
-				dev->ibd_next_failover = jiffies + 3 * HZ;
+				dev->ibd_next_failover = ktime_get_seconds() + 3;
 				continue;
 			}
 
 			/* failed to failover, retry later */
-			dev->ibd_next_failover =
-				jiffies + min(dev->ibd_failed_failover, 10) * HZ;
+			dev->ibd_next_failover = ktime_get_seconds() +
+						 min(dev->ibd_failed_failover, 10);
 			if (kiblnd_dev_can_failover(dev)) {
 				list_add_tail(&dev->ibd_fail_list,
 					      &kiblnd_data.kib_failed_devs);
-- 
1.8.3.1


From jsimmons at infradead.org  Sun Jun 24 21:53:47 2018
From: jsimmons at infradead.org (James Simmons)
Date: Sun, 24 Jun 2018 17:53:47 -0400
Subject: [lustre-devel] [PATCH 3/8] staging: lustre: ksocklnd: use
 ktime_get_real_ns for ksnn_incarnation
In-Reply-To: <1529877232-15582-1-git-send-email-jsimmons@infradead.org>
References: <1529877232-15582-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1529877232-15582-4-git-send-email-jsimmons@infradead.org>

The value generated for ksnn_incarnation is used as a unique identifier.
Their exist a small chance that two nodes could end up with the same
value but if we use ktime_get_real_ns() instead the change is even less
so.

Signed-off-by: James Simmons <uja.ornl at yahoo.com>
WC-bug-id: https://jira.whamcloud.com/browse/LU-9019
Reviewed-on: https://review.whamcloud.com/25690
Reviewed-by: Doug Oucharek <dougso at me.com>
Reviewed-by: Olaf Weber <olaf at sgi.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin at intel.com>
Reviewed-by: Oleg Drokin <green at whamcloud.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c | 11 +----------
 1 file changed, 1 insertion(+), 10 deletions(-)

diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c
index f01b34a..694f1d0 100644
--- a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c
+++ b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c
@@ -2369,15 +2369,6 @@ static int ksocknal_push(struct lnet_ni *ni, struct lnet_process_id id)
 	module_put(THIS_MODULE);
 }
 
-static __u64
-ksocknal_new_incarnation(void)
-{
-	/* The incarnation number is the time this module loaded and it
-	 * identifies this particular instance of the socknal.
-	 */
-	return ktime_get_ns();
-}
-
 static int
 ksocknal_base_startup(void)
 {
@@ -2810,7 +2801,7 @@ static int ksocknal_push(struct lnet_ni *ni, struct lnet_process_id id)
 		goto fail_0;
 
 	spin_lock_init(&net->ksnn_lock);
-	net->ksnn_incarnation = ksocknal_new_incarnation();
+	net->ksnn_incarnation = ktime_get_real_ns();
 	ni->ni_data = net;
 	ni->ni_peertimeout    = *ksocknal_tunables.ksnd_peertimeout;
 	ni->ni_maxtxcredits   = *ksocknal_tunables.ksnd_credits;
-- 
1.8.3.1


From jsimmons at infradead.org  Sun Jun 24 21:53:49 2018
From: jsimmons at infradead.org (James Simmons)
Date: Sun, 24 Jun 2018 17:53:49 -0400
Subject: [lustre-devel] [PATCH 5/8] staging: lustre: lnet: use
 ktime_get_real_ns for cookie creation
In-Reply-To: <1529877232-15582-1-git-send-email-jsimmons@infradead.org>
References: <1529877232-15582-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1529877232-15582-6-git-send-email-jsimmons@infradead.org>

The ln_interface_cookie is used to ensure that a node can tell whether
the following sequence of events has happened:

node sends GET or PUT to peer
node is rebooted
peer sends REPLY or ACK to node

The ln_interface_cookie is set once, when LNet starts, and remains
unchanged afterwards. To avoid accidentally obtaining the same cookie
after a reboot, the code generated ths cookie using ktime_get_ns().
Once generated, the value of the cookie is not interpreted, only
compared for equality. Olaf Weber reported that due to the use of
ktime_get_ns() a small chance exist of generating a cookie of identical
value across reboots. Using ktime_get_real_ns() removes any chance of
this from happening.

Signed-off-by: James Simmons <uja.ornl at yahoo.com>
WC-bug-id: https://jira.whamcloud.com/browse/LU-4423
Reviewed-on: https://review.whamcloud.com/24682
Reviewed-by: Doug Oucharek <dougso at me.com>
Reviewed-by: Olaf Weber <olaf at sgi.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin at intel.com>
Reviewed-by: Oleg Drokin <green at whamcloud.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/lnet/lnet/api-ni.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/staging/lustre/lnet/lnet/api-ni.c b/drivers/staging/lustre/lnet/lnet/api-ni.c
index 878c92c..20dce34 100644
--- a/drivers/staging/lustre/lnet/lnet/api-ni.c
+++ b/drivers/staging/lustre/lnet/lnet/api-ni.c
@@ -554,7 +554,7 @@ struct lnet_libhandle *
 	 * NB the interface cookie in wire handles guards against delayed
 	 * replies and ACKs appearing valid after reboot.
 	 */
-	the_lnet.ln_interface_cookie = ktime_get_ns();
+	the_lnet.ln_interface_cookie = ktime_get_real_ns();
 
 	the_lnet.ln_counters = cfs_percpt_alloc(lnet_cpt_table(),
 						sizeof(struct lnet_counters));
-- 
1.8.3.1


From jsimmons at infradead.org  Sun Jun 24 21:53:48 2018
From: jsimmons at infradead.org (James Simmons)
Date: Sun, 24 Jun 2018 17:53:48 -0400
Subject: [lustre-devel] [PATCH 4/8] staging: lustre: ksocklnd: move
 remaining time handling to 64 bits
In-Reply-To: <1529877232-15582-1-git-send-email-jsimmons@infradead.org>
References: <1529877232-15582-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1529877232-15582-5-git-send-email-jsimmons@infradead.org>

Examination of the ksocklnd time handle revealed that the
code only requires second level precision. Since this is
the case we can move away from using jiffies to time64_t.
This allows us to be independent of the HZ settings in
addition to making it clear what is time handling, using
time64_t verses unsigned long.

Signed-off-by: James Simmons <uja.ornl at yahoo.com>
WC-bug-id: https://jira.whamcloud.com/browse/LU-9397
Reviewed-on: https://review.whamcloud.com/26813
Reviewed-by: Doug Oucharek <dougso at me.com>
Reviewed-by: Amir Shehata <ashehata at whamcloud.com>
Reviewed-by: Oleg Drokin <green at whamcloud.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 .../staging/lustre/lnet/klnds/socklnd/socklnd.c    |  33 +++---
 .../staging/lustre/lnet/klnds/socklnd/socklnd.h    |  20 ++--
 .../staging/lustre/lnet/klnds/socklnd/socklnd_cb.c | 117 ++++++++++-----------
 3 files changed, 83 insertions(+), 87 deletions(-)

diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c
index 694f1d0..72c98ee 100644
--- a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c
+++ b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c
@@ -1276,7 +1276,7 @@ struct ksock_peer *
 	}
 
 	conn->ksnc_peer = peer;		 /* conn takes my ref on peer */
-	peer->ksnp_last_alive = jiffies;
+	peer->ksnp_last_alive = ktime_get_seconds();
 	peer->ksnp_send_keepalive = 0;
 	peer->ksnp_error = 0;
 
@@ -1284,10 +1284,11 @@ struct ksock_peer *
 	sched->kss_nconns++;
 	conn->ksnc_scheduler = sched;
 
-	conn->ksnc_tx_last_post = jiffies;
+	conn->ksnc_tx_last_post = ktime_get_seconds();
 	/* Set the deadline for the outgoing HELLO to drain */
 	conn->ksnc_tx_bufnob = sock->sk->sk_wmem_queued;
-	conn->ksnc_tx_deadline = jiffies + *ksocknal_tunables.ksnd_timeout * HZ;
+	conn->ksnc_tx_deadline = ktime_get_seconds() +
+				 *ksocknal_tunables.ksnd_timeout;
 	mb();   /* order with adding to peer's conn list */
 
 	list_add(&conn->ksnc_list, &peer->ksnp_conns);
@@ -1515,7 +1516,7 @@ struct ksock_peer *
 ksocknal_peer_failed(struct ksock_peer *peer)
 {
 	int notify = 0;
-	unsigned long last_alive = 0;
+	time64_t last_alive = 0;
 
 	/*
 	 * There has been a connection failure or comms error; but I'll only
@@ -1536,7 +1537,7 @@ struct ksock_peer *
 
 	if (notify)
 		lnet_notify(peer->ksnp_ni, peer->ksnp_id.nid, 0,
-			    last_alive);
+			    last_alive * HZ);
 }
 
 void
@@ -1660,7 +1661,7 @@ struct ksock_peer *
 void
 ksocknal_destroy_conn(struct ksock_conn *conn)
 {
-	unsigned long last_rcv;
+	time64_t last_rcv;
 
 	/* Final coup-de-grace of the reaper */
 	CDEBUG(D_NET, "connection %p\n", conn);
@@ -1677,12 +1678,12 @@ struct ksock_peer *
 	switch (conn->ksnc_rx_state) {
 	case SOCKNAL_RX_LNET_PAYLOAD:
 		last_rcv = conn->ksnc_rx_deadline -
-			   *ksocknal_tunables.ksnd_timeout * HZ;
-		CERROR("Completing partial receive from %s[%d], ip %pI4h:%d, with error, wanted: %zd, left: %d, last alive is %ld secs ago\n",
+			   *ksocknal_tunables.ksnd_timeout;
+		CERROR("Completing partial receive from %s[%d], ip %pI4h:%d, with error, wanted: %zd, left: %d, last alive is %lld secs ago\n",
 		       libcfs_id2str(conn->ksnc_peer->ksnp_id), conn->ksnc_type,
 		       &conn->ksnc_ipaddr, conn->ksnc_port,
 		       iov_iter_count(&conn->ksnc_rx_to), conn->ksnc_rx_nob_left,
-		       (jiffies - last_rcv) / HZ);
+		       ktime_get_seconds() - last_rcv);
 		lnet_finalize(conn->ksnc_peer->ksnp_ni,
 			      conn->ksnc_cookie, -EIO);
 		break;
@@ -1830,8 +1831,8 @@ struct ksock_peer *
 ksocknal_query(struct lnet_ni *ni, lnet_nid_t nid, unsigned long *when)
 {
 	int connect = 1;
-	unsigned long last_alive = 0;
-	unsigned long now = jiffies;
+	time64_t last_alive = 0;
+	time64_t now = ktime_get_seconds();
 	struct ksock_peer *peer = NULL;
 	rwlock_t *glock = &ksocknal_data.ksnd_global_lock;
 	struct lnet_process_id id = {
@@ -1851,8 +1852,8 @@ struct ksock_peer *
 
 			if (bufnob < conn->ksnc_tx_bufnob) {
 				/* something got ACKed */
-				conn->ksnc_tx_deadline =
-					jiffies + *ksocknal_tunables.ksnd_timeout * HZ;
+				conn->ksnc_tx_deadline = ktime_get_seconds() +
+							 *ksocknal_tunables.ksnd_timeout;
 				peer->ksnp_last_alive = now;
 				conn->ksnc_tx_bufnob = bufnob;
 			}
@@ -1866,11 +1867,11 @@ struct ksock_peer *
 	read_unlock(glock);
 
 	if (last_alive)
-		*when = last_alive;
+		*when = last_alive * HZ;
 
-	CDEBUG(D_NET, "Peer %s %p, alive %ld secs ago, connect %d\n",
+	CDEBUG(D_NET, "Peer %s %p, alive %lld secs ago, connect %d\n",
 	       libcfs_nid2str(nid), peer,
-	       last_alive ? (now - last_alive) / HZ : -1,
+	       last_alive ? now - last_alive : -1,
 	       connect);
 
 	if (!connect)
diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.h b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.h
index 4e5c89a..20f5de6 100644
--- a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.h
+++ b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.h
@@ -57,7 +57,7 @@
 #define SOCKNAL_PEER_HASH_SIZE  101   /* # peer lists */
 #define SOCKNAL_RESCHED         100   /* # scheduler loops before reschedule */
 #define SOCKNAL_INSANITY_RECONN 5000  /* connd is trying on reconn infinitely */
-#define SOCKNAL_ENOMEM_RETRY    1     /* jiffies between retries */
+#define SOCKNAL_ENOMEM_RETRY    1     /* seconds between retries */
 
 #define SOCKNAL_SINGLE_FRAG_TX  0     /* disable multi-fragment sends */
 #define SOCKNAL_SINGLE_FRAG_RX  0     /* disable multi-fragment receives */
@@ -210,7 +210,7 @@ struct ksock_nal_data {
 							 * reaper_lock
 							 */
 	wait_queue_head_t       ksnd_reaper_waitq;      /* reaper sleeps here */
-	unsigned long	        ksnd_reaper_waketime;   /* when reaper will wake
+	time64_t		ksnd_reaper_waketime;   /* when reaper will wake
 							 */
 	spinlock_t              ksnd_reaper_lock;       /* serialise */
 
@@ -285,7 +285,7 @@ struct ksock_tx {			   /* transmit packet */
 	struct ksock_conn *tx_conn;        /* owning conn */
 	struct lnet_msg        *tx_lnetmsg;     /* lnet message for lnet_finalize()
 					    */
-	unsigned long     tx_deadline;     /* when (in jiffies) tx times out */
+	time64_t		tx_deadline;	/* when (in secs) tx times out */
 	struct ksock_msg       tx_msg;          /* socklnd message buffer */
 	int               tx_desc_size;    /* size of this descriptor */
 	union {
@@ -340,7 +340,7 @@ struct ksock_conn {
 	struct list_head   ksnc_rx_list;      /* where I enq waiting input or a
 					       * forwarding descriptor
 					       */
-	unsigned long      ksnc_rx_deadline;  /* when (in jiffies) receive times
+	time64_t	   ksnc_rx_deadline;  /* when (in secs) receive times
 					       * out
 					       */
 	__u8               ksnc_rx_started;   /* started receiving a message */
@@ -370,13 +370,13 @@ struct ksock_conn {
 	struct ksock_tx	  *ksnc_tx_carrier;   /* next TX that can carry a LNet
 					       * message or ZC-ACK
 					       */
-	unsigned long      ksnc_tx_deadline;  /* when (in jiffies) tx times out
+	time64_t	   ksnc_tx_deadline;  /* when (in secs) tx times out
 					       */
 	int                ksnc_tx_bufnob;    /* send buffer marker */
 	atomic_t           ksnc_tx_nob;       /* # bytes queued */
 	int		   ksnc_tx_ready;     /* write space */
 	int		   ksnc_tx_scheduled; /* being progressed */
-	unsigned long      ksnc_tx_last_post; /* time stamp of the last posted
+	time64_t	   ksnc_tx_last_post; /* time stamp of the last posted
 					       * TX
 					       */
 };
@@ -386,10 +386,10 @@ struct ksock_route {
 	struct list_head  ksnr_connd_list;     /* chain on ksnr_connd_routes */
 	struct ksock_peer *ksnr_peer;          /* owning peer */
 	atomic_t          ksnr_refcount;       /* # users */
-	unsigned long     ksnr_timeout;        /* when (in jiffies) reconnection
+	time64_t	  ksnr_timeout;        /* when (in secs) reconnection
 						* can happen next
 						*/
-	long              ksnr_retry_interval; /* how long between retries */
+	time64_t	  ksnr_retry_interval; /* how long between retries */
 	__u32             ksnr_myipaddr;       /* my IP */
 	__u32             ksnr_ipaddr;         /* IP address to connect to */
 	int               ksnr_port;           /* port to connect to */
@@ -411,7 +411,7 @@ struct ksock_route {
 
 struct ksock_peer {
 	struct list_head   ksnp_list;           /* stash on global peer list */
-	unsigned long      ksnp_last_alive;     /* when (in jiffies) I was last
+	time64_t	   ksnp_last_alive;     /* when (in seconds) I was last
 						 * alive
 						 */
 	struct lnet_process_id  ksnp_id;	/* who's on the other end(s) */
@@ -432,7 +432,7 @@ struct ksock_peer {
 	struct list_head   ksnp_zc_req_list;    /* zero copy requests wait for
 						 * ACK
 						 */
-	unsigned long      ksnp_send_keepalive; /* time to send keepalive */
+	time64_t	   ksnp_send_keepalive; /* time to send keepalive */
 	struct lnet_ni	   *ksnp_ni;		/* which network */
 	int                ksnp_n_passive_ips;  /* # of... */
 
diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c
index 01b31a6..3f69618 100644
--- a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c
+++ b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c
@@ -221,9 +221,9 @@ struct ksock_tx *
 			 * allocated send buffer bytes < computed; infer
 			 * something got ACKed
 			 */
-			conn->ksnc_tx_deadline =
-				jiffies + *ksocknal_tunables.ksnd_timeout * HZ;
-			conn->ksnc_peer->ksnp_last_alive = jiffies;
+			conn->ksnc_tx_deadline = ktime_get_seconds() +
+						 *ksocknal_tunables.ksnd_timeout;
+			conn->ksnc_peer->ksnp_last_alive = ktime_get_seconds();
 			conn->ksnc_tx_bufnob = bufnob;
 			mb();
 		}
@@ -268,9 +268,9 @@ struct ksock_tx *
 	/* received something... */
 	nob = rc;
 
-	conn->ksnc_peer->ksnp_last_alive = jiffies;
-	conn->ksnc_rx_deadline =
-		jiffies + *ksocknal_tunables.ksnd_timeout * HZ;
+	conn->ksnc_peer->ksnp_last_alive = ktime_get_seconds();
+	conn->ksnc_rx_deadline = ktime_get_seconds() +
+				 *ksocknal_tunables.ksnd_timeout;
 	mb();		       /* order with setting rx_started */
 	conn->ksnc_rx_started = 1;
 
@@ -405,8 +405,8 @@ struct ksock_tx *
 	spin_lock(&peer->ksnp_lock);
 
 	/* ZC_REQ is going to be pinned to the peer */
-	tx->tx_deadline =
-		jiffies + *ksocknal_tunables.ksnd_timeout * HZ;
+	tx->tx_deadline = ktime_get_seconds() +
+			  *ksocknal_tunables.ksnd_timeout;
 
 	LASSERT(!tx->tx_msg.ksm_zc_cookies[0]);
 
@@ -482,8 +482,8 @@ struct ksock_tx *
 		LASSERT(conn->ksnc_tx_scheduled);
 		list_add_tail(&conn->ksnc_tx_list,
 			      &ksocknal_data.ksnd_enomem_conns);
-		if (!time_after_eq(jiffies + SOCKNAL_ENOMEM_RETRY,
-				   ksocknal_data.ksnd_reaper_waketime))
+		if (ktime_get_seconds() + SOCKNAL_ENOMEM_RETRY <
+		    ksocknal_data.ksnd_reaper_waketime)
 			wake_up(&ksocknal_data.ksnd_reaper_waitq);
 
 		spin_unlock_bh(&ksocknal_data.ksnd_reaper_lock);
@@ -591,7 +591,7 @@ struct ksock_conn *
 		case SOCKNAL_MATCH_YES: /* typed connection */
 			if (!typed || tnob > nob ||
 			    (tnob == nob && *ksocknal_tunables.ksnd_round_robin &&
-			     time_after(typed->ksnc_tx_last_post, c->ksnc_tx_last_post))) {
+			     typed->ksnc_tx_last_post > c->ksnc_tx_last_post)) {
 				typed = c;
 				tnob  = nob;
 			}
@@ -600,7 +600,7 @@ struct ksock_conn *
 		case SOCKNAL_MATCH_MAY: /* fallback connection */
 			if (!fallback || fnob > nob ||
 			    (fnob == nob && *ksocknal_tunables.ksnd_round_robin &&
-			     time_after(fallback->ksnc_tx_last_post, c->ksnc_tx_last_post))) {
+			     fallback->ksnc_tx_last_post > c->ksnc_tx_last_post)) {
 				fallback = c;
 				fnob = nob;
 			}
@@ -612,7 +612,7 @@ struct ksock_conn *
 	conn = (typed) ? typed : fallback;
 
 	if (conn)
-		conn->ksnc_tx_last_post = jiffies;
+		conn->ksnc_tx_last_post = ktime_get_seconds();
 
 	return conn;
 }
@@ -677,10 +677,10 @@ struct ksock_conn *
 
 	if (list_empty(&conn->ksnc_tx_queue) && !bufnob) {
 		/* First packet starts the timeout */
-		conn->ksnc_tx_deadline =
-			jiffies + *ksocknal_tunables.ksnd_timeout * HZ;
+		conn->ksnc_tx_deadline = ktime_get_seconds() +
+					 *ksocknal_tunables.ksnd_timeout;
 		if (conn->ksnc_tx_bufnob > 0) /* something got ACKed */
-			conn->ksnc_peer->ksnp_last_alive = jiffies;
+			conn->ksnc_peer->ksnp_last_alive = ktime_get_seconds();
 		conn->ksnc_tx_bufnob = 0;
 		mb(); /* order with adding to tx_queue */
 	}
@@ -728,7 +728,7 @@ struct ksock_conn *
 struct ksock_route *
 ksocknal_find_connectable_route_locked(struct ksock_peer *peer)
 {
-	unsigned long now = jiffies;
+	time64_t now = ktime_get_seconds();
 	struct list_head *tmp;
 	struct ksock_route *route;
 
@@ -746,13 +746,13 @@ struct ksock_route *
 			continue;
 
 		if (!(!route->ksnr_retry_interval || /* first attempt */
-		      time_after_eq(now, route->ksnr_timeout))) {
+		      now >= route->ksnr_timeout)) {
 			CDEBUG(D_NET,
-			       "Too soon to retry route %pI4h (cnted %d, interval %ld, %ld secs later)\n",
+			       "Too soon to retry route %pI4h (cnted %d, interval %lld, %lld secs later)\n",
 			       &route->ksnr_ipaddr,
 			       route->ksnr_connected,
 			       route->ksnr_retry_interval,
-			       (route->ksnr_timeout - now) / HZ);
+			       route->ksnr_timeout - now);
 			continue;
 		}
 
@@ -858,8 +858,8 @@ struct ksock_route *
 	if (peer->ksnp_accepting > 0 ||
 	    ksocknal_find_connecting_route_locked(peer)) {
 		/* the message is going to be pinned to the peer */
-		tx->tx_deadline =
-			jiffies + *ksocknal_tunables.ksnd_timeout * HZ;
+		tx->tx_deadline = ktime_get_seconds() +
+				  *ksocknal_tunables.ksnd_timeout;
 
 		/* Queue the message until a connection is established */
 		list_add_tail(&tx->tx_list, &peer->ksnp_tx_queue);
@@ -1773,11 +1773,11 @@ void ksocknal_write_callback(struct ksock_conn *conn)
 	int type;
 	int wanted;
 	struct socket *sock;
-	unsigned long deadline;
+	time64_t deadline;
 	int retry_later = 0;
 	int rc = 0;
 
-	deadline = jiffies + *ksocknal_tunables.ksnd_timeout * HZ;
+	deadline = ktime_get_seconds() + *ksocknal_tunables.ksnd_timeout;
 
 	write_lock_bh(&ksocknal_data.ksnd_global_lock);
 
@@ -1824,7 +1824,7 @@ void ksocknal_write_callback(struct ksock_conn *conn)
 
 		write_unlock_bh(&ksocknal_data.ksnd_global_lock);
 
-		if (time_after_eq(jiffies, deadline)) {
+		if (ktime_get_seconds() >= deadline) {
 			rc = -ETIMEDOUT;
 			lnet_connect_console_error(rc, peer->ksnp_id.nid,
 						   route->ksnr_ipaddr,
@@ -1875,8 +1875,9 @@ void ksocknal_write_callback(struct ksock_conn *conn)
 			 * so min_reconnectms should be good heuristic
 			 */
 			route->ksnr_retry_interval =
-				*ksocknal_tunables.ksnd_min_reconnectms * HZ / 1000;
-			route->ksnr_timeout = jiffies + route->ksnr_retry_interval;
+				*ksocknal_tunables.ksnd_min_reconnectms / 1000;
+			route->ksnr_timeout = ktime_get_seconds() +
+					      route->ksnr_retry_interval;
 		}
 
 		ksocknal_launch_connection_locked(route);
@@ -1894,14 +1895,14 @@ void ksocknal_write_callback(struct ksock_conn *conn)
 	/* This is a retry rather than a new connection */
 	route->ksnr_retry_interval *= 2;
 	route->ksnr_retry_interval =
-		max(route->ksnr_retry_interval,
-		    (long)*ksocknal_tunables.ksnd_min_reconnectms * HZ / 1000);
+		max_t(time64_t, route->ksnr_retry_interval,
+		      *ksocknal_tunables.ksnd_min_reconnectms / 1000);
 	route->ksnr_retry_interval =
-		min(route->ksnr_retry_interval,
-		    (long)*ksocknal_tunables.ksnd_max_reconnectms * HZ / 1000);
+		min_t(time64_t, route->ksnr_retry_interval,
+		      *ksocknal_tunables.ksnd_max_reconnectms / 1000);
 
 	LASSERT(route->ksnr_retry_interval);
-	route->ksnr_timeout = jiffies + route->ksnr_retry_interval;
+	route->ksnr_timeout = ktime_get_seconds() + route->ksnr_retry_interval;
 
 	if (!list_empty(&peer->ksnp_tx_queue) &&
 	    !peer->ksnp_accepting &&
@@ -2044,21 +2045,19 @@ void ksocknal_write_callback(struct ksock_conn *conn)
 static struct ksock_route *
 ksocknal_connd_get_route_locked(signed long *timeout_p)
 {
+	time64_t now = ktime_get_seconds();
 	struct ksock_route *route;
-	unsigned long now;
-
-	now = jiffies;
 
 	/* connd_routes can contain both pending and ordinary routes */
 	list_for_each_entry(route, &ksocknal_data.ksnd_connd_routes,
 			    ksnr_connd_list) {
 		if (!route->ksnr_retry_interval ||
-		    time_after_eq(now, route->ksnr_timeout))
+		    now >= route->ksnr_timeout)
 			return route;
 
 		if (*timeout_p == MAX_SCHEDULE_TIMEOUT ||
-		    (int)*timeout_p > (int)(route->ksnr_timeout - now))
-			*timeout_p = (int)(route->ksnr_timeout - now);
+		    *timeout_p > (signed long)(route->ksnr_timeout - now) * HZ)
+			*timeout_p = (signed long)(route->ksnr_timeout - now) * HZ;
 	}
 
 	return NULL;
@@ -2225,8 +2224,7 @@ void ksocknal_write_callback(struct ksock_conn *conn)
 		}
 
 		if (conn->ksnc_rx_started &&
-		    time_after_eq(jiffies,
-				  conn->ksnc_rx_deadline)) {
+		    ktime_get_seconds() >= conn->ksnc_rx_deadline) {
 			/* Timed out incomplete incoming message */
 			ksocknal_conn_addref(conn);
 			CNETERR("Timeout receiving from %s (%pI4h:%d), state %d wanted %zd left %d\n",
@@ -2241,8 +2239,7 @@ void ksocknal_write_callback(struct ksock_conn *conn)
 
 		if ((!list_empty(&conn->ksnc_tx_queue) ||
 		     conn->ksnc_sock->sk->sk_wmem_queued) &&
-		    time_after_eq(jiffies,
-				  conn->ksnc_tx_deadline)) {
+		    ktime_get_seconds() >= conn->ksnc_tx_deadline) {
 			/*
 			 * Timed out messages queued for sending or
 			 * buffered in the socket's send buffer
@@ -2269,8 +2266,7 @@ void ksocknal_write_callback(struct ksock_conn *conn)
 	write_lock_bh(&ksocknal_data.ksnd_global_lock);
 
 	list_for_each_entry_safe(tx, tmp, &peer->ksnp_tx_queue, tx_list) {
-		if (!time_after_eq(jiffies,
-				   tx->tx_deadline))
+		if (ktime_get_seconds() < tx->tx_deadline)
 			break;
 
 		list_del(&tx->tx_list);
@@ -2298,18 +2294,18 @@ void ksocknal_write_callback(struct ksock_conn *conn)
 		return 0;
 
 	if (*ksocknal_tunables.ksnd_keepalive <= 0 ||
-	    time_before(jiffies,
-			peer->ksnp_last_alive + *ksocknal_tunables.ksnd_keepalive * HZ))
+	    ktime_get_seconds() < peer->ksnp_last_alive +
+				  *ksocknal_tunables.ksnd_keepalive)
 		return 0;
 
-	if (time_before(jiffies, peer->ksnp_send_keepalive))
+	if (ktime_get_seconds() < peer->ksnp_send_keepalive)
 		return 0;
 
 	/*
 	 * retry 10 secs later, so we wouldn't put pressure
 	 * on this peer if we failed to send keepalive this time
 	 */
-	peer->ksnp_send_keepalive = jiffies + 10 * HZ;
+	peer->ksnp_send_keepalive = ktime_get_seconds() + 10;
 
 	conn = ksocknal_find_conn_locked(peer, NULL, 1);
 	if (conn) {
@@ -2362,8 +2358,8 @@ void ksocknal_write_callback(struct ksock_conn *conn)
 	read_lock(&ksocknal_data.ksnd_global_lock);
 
 	list_for_each_entry(peer, peers, ksnp_list) {
-		unsigned long deadline = 0;
 		struct ksock_tx *tx_stale;
+		time64_t deadline = 0;
 		int resid = 0;
 		int n = 0;
 
@@ -2396,8 +2392,7 @@ void ksocknal_write_callback(struct ksock_conn *conn)
 			tx = list_entry(peer->ksnp_tx_queue.next,
 					struct ksock_tx, tx_list);
 
-			if (time_after_eq(jiffies,
-					  tx->tx_deadline)) {
+			if (ktime_get_seconds() >= tx->tx_deadline) {
 				ksocknal_peer_addref(peer);
 				read_unlock(&ksocknal_data.ksnd_global_lock);
 
@@ -2414,8 +2409,7 @@ void ksocknal_write_callback(struct ksock_conn *conn)
 		tx_stale = NULL;
 		spin_lock(&peer->ksnp_lock);
 		list_for_each_entry(tx, &peer->ksnp_zc_req_list, tx_zc_list) {
-			if (!time_after_eq(jiffies,
-					   tx->tx_deadline))
+			if (ktime_get_seconds() < tx->tx_deadline)
 				break;
 			/* ignore the TX if connection is being closed */
 			if (tx->tx_conn->ksnc_closing)
@@ -2438,9 +2432,9 @@ void ksocknal_write_callback(struct ksock_conn *conn)
 		spin_unlock(&peer->ksnp_lock);
 		read_unlock(&ksocknal_data.ksnd_global_lock);
 
-		CERROR("Total %d stale ZC_REQs for peer %s detected; the oldest(%p) timed out %ld secs ago, resid: %d, wmem: %d\n",
+		CERROR("Total %d stale ZC_REQs for peer %s detected; the oldest(%p) timed out %lld secs ago, resid: %d, wmem: %d\n",
 		       n, libcfs_nid2str(peer->ksnp_id.nid), tx_stale,
-		       (jiffies - deadline) / HZ,
+		       ktime_get_seconds() - deadline,
 		       resid, conn->ksnc_sock->sk->sk_wmem_queued);
 
 		ksocknal_close_conn_and_siblings(conn, -ETIMEDOUT);
@@ -2459,10 +2453,10 @@ void ksocknal_write_callback(struct ksock_conn *conn)
 	struct ksock_sched *sched;
 	struct list_head enomem_conns;
 	int nenomem_conns;
-	long timeout;
+	time64_t timeout;
 	int i;
 	int peer_index = 0;
-	unsigned long deadline = jiffies;
+	time64_t deadline = ktime_get_seconds();
 
 	INIT_LIST_HEAD(&enomem_conns);
 	init_waitqueue_entry(&wait, current);
@@ -2527,7 +2521,7 @@ void ksocknal_write_callback(struct ksock_conn *conn)
 		}
 
 		/* careful with the jiffy wrap... */
-		while ((timeout = deadline - jiffies) <= 0) {
+		while ((timeout = deadline - ktime_get_seconds()) <= 0) {
 			const int n = 4;
 			const int p = 1;
 			int chunk = ksocknal_data.ksnd_peer_hash_size;
@@ -2552,7 +2546,7 @@ void ksocknal_write_callback(struct ksock_conn *conn)
 					     ksocknal_data.ksnd_peer_hash_size;
 			}
 
-			deadline = deadline + p * HZ;
+			deadline += p;
 		}
 
 		if (nenomem_conns) {
@@ -2563,7 +2557,8 @@ void ksocknal_write_callback(struct ksock_conn *conn)
 			 */
 			timeout = SOCKNAL_ENOMEM_RETRY;
 		}
-		ksocknal_data.ksnd_reaper_waketime = jiffies + timeout;
+		ksocknal_data.ksnd_reaper_waketime = ktime_get_seconds() +
+						     timeout;
 
 		set_current_state(TASK_INTERRUPTIBLE);
 		add_wait_queue(&ksocknal_data.ksnd_reaper_waitq, &wait);
@@ -2571,7 +2566,7 @@ void ksocknal_write_callback(struct ksock_conn *conn)
 		if (!ksocknal_data.ksnd_shuttingdown &&
 		    list_empty(&ksocknal_data.ksnd_deathrow_conns) &&
 		    list_empty(&ksocknal_data.ksnd_zombie_conns))
-			schedule_timeout(timeout);
+			schedule_timeout(timeout * HZ);
 
 		set_current_state(TASK_RUNNING);
 		remove_wait_queue(&ksocknal_data.ksnd_reaper_waitq, &wait);
-- 
1.8.3.1


From jsimmons at infradead.org  Sun Jun 24 21:53:45 2018
From: jsimmons at infradead.org (James Simmons)
Date: Sun, 24 Jun 2018 17:53:45 -0400
Subject: [lustre-devel] [PATCH 1/8] staging: lustre: lnet: simplify
 lnet_eq_wait_locked
In-Reply-To: <1529877232-15582-1-git-send-email-jsimmons@infradead.org>
References: <1529877232-15582-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1529877232-15582-2-git-send-email-jsimmons@infradead.org>

We can simplify the code by taking advantage of the behavior
of schedule_timeout_interruptible(). Instead of testing if
tms is less than zero we can pass in a signed long that
schedule_timeout_interruptible is expecting and for the case
of no timeout we can pass in MAX_SCHEDULE_TIMEOUT.

Signed-off-by: James Simmons <uja.ornl at yahoo.com>
WC-bug-id: https://jira.whamcloud.com/browse/LU-9019
Reviewed-on: https://review.whamcloud.com/23147
Reviewed-by: Doug Oucharek <dougso at me.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin at intel.com>
Reviewed-by: Oleg Drokin <green at whamcloud.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/include/linux/lnet/api.h    |  2 +-
 .../lustre/include/uapi/linux/lnet/lnet-types.h    |  2 --
 drivers/staging/lustre/lnet/lnet/api-ni.c          | 38 ++++++++++++++--------
 drivers/staging/lustre/lnet/lnet/lib-eq.c          | 26 +++++----------
 4 files changed, 33 insertions(+), 35 deletions(-)

diff --git a/drivers/staging/lustre/include/linux/lnet/api.h b/drivers/staging/lustre/include/linux/lnet/api.h
index dae2e4f..2242921 100644
--- a/drivers/staging/lustre/include/linux/lnet/api.h
+++ b/drivers/staging/lustre/include/linux/lnet/api.h
@@ -168,7 +168,7 @@ int LNetEQAlloc(unsigned int       count_in,
 
 int LNetEQPoll(struct lnet_handle_eq *eventqs_in,
 	       int		 neq_in,
-	       int		 timeout_ms,
+	       signed long	 timeout,
 	       int		 interruptible,
 	       struct lnet_event *event_out,
 	       int		*which_eq_out);
diff --git a/drivers/staging/lustre/include/uapi/linux/lnet/lnet-types.h b/drivers/staging/lustre/include/uapi/linux/lnet/lnet-types.h
index 1be9b7a..14715a8 100644
--- a/drivers/staging/lustre/include/uapi/linux/lnet/lnet-types.h
+++ b/drivers/staging/lustre/include/uapi/linux/lnet/lnet-types.h
@@ -77,8 +77,6 @@
 #define LNET_PID_USERFLAG 0x80000000 /* set in userspace peers */
 #define LNET_PID_LUSTRE	  12345
 
-#define LNET_TIME_FOREVER (-1)
-
 /* how an LNET NID encodes net:address */
 /** extract the address part of an lnet_nid_t */
 
diff --git a/drivers/staging/lustre/lnet/lnet/api-ni.c b/drivers/staging/lustre/lnet/lnet/api-ni.c
index f9ed697..878c92c 100644
--- a/drivers/staging/lustre/lnet/lnet/api-ni.c
+++ b/drivers/staging/lustre/lnet/lnet/api-ni.c
@@ -59,7 +59,7 @@
 module_param(rnet_htable_size, int, 0444);
 MODULE_PARM_DESC(rnet_htable_size, "size of remote network hash table");
 
-static int lnet_ping(struct lnet_process_id id, int timeout_ms,
+static int lnet_ping(struct lnet_process_id id, signed long timeout,
 		     struct lnet_process_id __user *ids, int n_ids);
 
 static char *
@@ -2052,17 +2052,29 @@ void lnet_lib_exit(void)
 	case IOC_LIBCFS_LNET_FAULT:
 		return lnet_fault_ctl(data->ioc_flags, data);
 
-	case IOC_LIBCFS_PING:
+	case IOC_LIBCFS_PING: {
+		signed long timeout;
+
 		id.nid = data->ioc_nid;
 		id.pid = data->ioc_u32[0];
-		rc = lnet_ping(id, data->ioc_u32[1], /* timeout */
-			       data->ioc_pbuf1,
+
+		/* Don't block longer than 2 minutes */
+		if (data->ioc_u32[1] > 120 * MSEC_PER_SEC)
+			return -EINVAL;
+
+		/* If timestamp is negative then disable timeout */
+		if ((s32)data->ioc_u32[1] < 0)
+			timeout = MAX_SCHEDULE_TIMEOUT;
+		else
+			timeout = msecs_to_jiffies(data->ioc_u32[1]);
+
+		rc = lnet_ping(id, timeout, data->ioc_pbuf1,
 			       data->ioc_plen1 / sizeof(struct lnet_process_id));
 		if (rc < 0)
 			return rc;
 		data->ioc_count = rc;
 		return 0;
-
+	}
 	default:
 		ni = lnet_net2ni(data->ioc_net);
 		if (!ni)
@@ -2126,7 +2138,7 @@ void LNetDebugPeer(struct lnet_process_id id)
 }
 EXPORT_SYMBOL(LNetGetId);
 
-static int lnet_ping(struct lnet_process_id id, int timeout_ms,
+static int lnet_ping(struct lnet_process_id id, signed long timeout,
 		     struct lnet_process_id __user *ids, int n_ids)
 {
 	struct lnet_handle_eq eqh;
@@ -2136,7 +2148,7 @@ static int lnet_ping(struct lnet_process_id id, int timeout_ms,
 	int which;
 	int unlinked = 0;
 	int replied = 0;
-	const int a_long_time = 60000; /* mS */
+	const signed long a_long_time = msecs_to_jiffies(60 * MSEC_PER_SEC);
 	int infosz;
 	struct lnet_ping_info *info;
 	struct lnet_process_id tmpid;
@@ -2147,10 +2159,8 @@ static int lnet_ping(struct lnet_process_id id, int timeout_ms,
 
 	infosz = offsetof(struct lnet_ping_info, pi_ni[n_ids]);
 
-	if (n_ids <= 0 ||
-	    id.nid == LNET_NID_ANY ||
-	    timeout_ms > 500000 ||	      /* arbitrary limit! */
-	    n_ids > 20)			 /* arbitrary limit! */
+	/* n_ids limit is arbitrary */
+	if (n_ids <= 0 || n_ids > 20 || id.nid == LNET_NID_ANY)
 		return -EINVAL;
 
 	if (id.pid == LNET_PID_ANY)
@@ -2194,13 +2204,13 @@ static int lnet_ping(struct lnet_process_id id, int timeout_ms,
 
 		/* NB must wait for the UNLINK event below... */
 		unlinked = 1;
-		timeout_ms = a_long_time;
+		timeout = a_long_time;
 	}
 
 	do {
 		/* MUST block for unlink to complete */
 
-		rc2 = LNetEQPoll(&eqh, 1, timeout_ms, !unlinked,
+		rc2 = LNetEQPoll(&eqh, 1, timeout, !unlinked,
 				 &event, &which);
 
 		CDEBUG(D_NET, "poll %d(%d %d)%s\n", rc2,
@@ -2222,7 +2232,7 @@ static int lnet_ping(struct lnet_process_id id, int timeout_ms,
 				LNetMDUnlink(mdh);
 				/* No assertion (racing with network) */
 				unlinked = 1;
-				timeout_ms = a_long_time;
+				timeout = a_long_time;
 			} else if (!rc2) {
 				/* timed out waiting for unlink */
 				CWARN("ping %s: late network completion\n",
diff --git a/drivers/staging/lustre/lnet/lnet/lib-eq.c b/drivers/staging/lustre/lnet/lnet/lib-eq.c
index c78e703..366ad8a 100644
--- a/drivers/staging/lustre/lnet/lnet/lib-eq.c
+++ b/drivers/staging/lustre/lnet/lnet/lib-eq.c
@@ -308,13 +308,12 @@
  */
 
 static int
-lnet_eq_wait_locked(int *timeout_ms, long state)
+lnet_eq_wait_locked(signed long *timeout, long state)
 __must_hold(&the_lnet.ln_eq_wait_lock)
 {
-	int tms = *timeout_ms;
+	signed long tms = *timeout;
 	int wait;
 	wait_queue_entry_t wl;
-	unsigned long now;
 
 	if (!tms)
 		return -ENXIO; /* don't want to wait and no new event */
@@ -325,18 +324,9 @@
 
 	lnet_eq_wait_unlock();
 
-	if (tms < 0) {
-		schedule();
-	} else {
-		now = jiffies;
-		schedule_timeout(msecs_to_jiffies(tms));
-		tms -= jiffies_to_msecs(jiffies - now);
-		if (tms < 0) /* no more wait but may have new event */
-			tms = 0;
-	}
-
+	tms = schedule_timeout_interruptible(tms);
 	wait = tms; /* might need to call here again */
-	*timeout_ms = tms;
+	*timeout = tms;
 
 	lnet_eq_wait_lock();
 	remove_wait_queue(&the_lnet.ln_eq_waitq, &wl);
@@ -356,8 +346,8 @@
  * fixed period, or block indefinitely.
  *
  * \param eventqs,neq An array of EQ handles, and size of the array.
- * \param timeout_ms Time in milliseconds to wait for an event to occur on
- * one of the EQs. The constant LNET_TIME_FOREVER can be used to indicate an
+ * \param timeout Time in jiffies to wait for an event to occur on
+ * one of the EQs. The constant MAX_SCHEDULE_TIMEOUT can be used to indicate an
  * infinite timeout.
  * \param interruptible, if true, use TASK_INTERRUPTIBLE, else TASK_NOLOAD
  * \param event,which On successful return (1 or -EOVERFLOW), \a event will
@@ -372,7 +362,7 @@
  * \retval -ENOENT    If there's an invalid handle in \a eventqs.
  */
 int
-LNetEQPoll(struct lnet_handle_eq *eventqs, int neq, int timeout_ms,
+LNetEQPoll(struct lnet_handle_eq *eventqs, int neq, signed long timeout,
 	   int interruptible,
 	   struct lnet_event *event, int *which)
 {
@@ -414,7 +404,7 @@
 		 *  0 : don't want to wait anymore, but might have new event
 		 *      so need to call dequeue again
 		 */
-		wait = lnet_eq_wait_locked(&timeout_ms,
+		wait = lnet_eq_wait_locked(&timeout,
 					   interruptible ? TASK_INTERRUPTIBLE
 					   : TASK_NOLOAD);
 		if (wait < 0) /* no new event */
-- 
1.8.3.1


From jsimmons at infradead.org  Sun Jun 24 21:53:52 2018
From: jsimmons at infradead.org (James Simmons)
Date: Sun, 24 Jun 2018 17:53:52 -0400
Subject: [lustre-devel] [PATCH 8/8] staging: lustre: selftest: fix time
 reporting
In-Reply-To: <1529877232-15582-1-git-send-email-jsimmons@infradead.org>
References: <1529877232-15582-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1529877232-15582-9-git-send-email-jsimmons@infradead.org>

Migrate crp_stamp to nanoseconds and both timestamps nd_stamp,
sn_start to ktime. The move away from jiffies, which can vary on
platforms, to something that is the consistent on every node.
Currently jiffies data was being transmitted over the wire and
the other node could have a different HZ value configured which
lead to incorrect results. This will ensure that the reported
results to the user will always be correct.

Signed-off-by: James Simmons <uja.ornl at yahoo.com>
WC-bug-id: https://jira.whamcloud.com/browse/LU-9019
Reviewed-on: https://review.whamcloud.com/31041
Reviewed-by: Doug Oucharek <dougso at me.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin at intel.com>
Reviewed-by: Oleg Drokin <green at whamcloud.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 .../lustre/include/uapi/linux/lnet/lnetst.h        |  2 +-
 drivers/staging/lustre/lnet/selftest/conrpc.c      | 39 +++++++++++-----------
 drivers/staging/lustre/lnet/selftest/conrpc.h      |  2 +-
 drivers/staging/lustre/lnet/selftest/console.c     |  4 +--
 drivers/staging/lustre/lnet/selftest/console.h     |  2 +-
 drivers/staging/lustre/lnet/selftest/framework.c   |  4 +--
 drivers/staging/lustre/lnet/selftest/selftest.h    |  2 +-
 drivers/staging/lustre/lnet/selftest/timer.c       |  8 ++---
 8 files changed, 32 insertions(+), 31 deletions(-)

diff --git a/drivers/staging/lustre/include/uapi/linux/lnet/lnetst.h b/drivers/staging/lustre/include/uapi/linux/lnet/lnetst.h
index a4f9ff0..7cfc241 100644
--- a/drivers/staging/lustre/include/uapi/linux/lnet/lnetst.h
+++ b/drivers/staging/lustre/include/uapi/linux/lnet/lnetst.h
@@ -71,7 +71,7 @@
 
 struct lst_sid {
 	lnet_nid_t	ses_nid;	/* nid of console node */
-	__u64		ses_stamp;	/* time stamp */
+	__s64		ses_stamp;	/* time stamp */
 };					/*** session id */
 
 extern struct lst_sid LST_INVALID_SID;
diff --git a/drivers/staging/lustre/lnet/selftest/conrpc.c b/drivers/staging/lustre/lnet/selftest/conrpc.c
index 0dabade..95cbd1a 100644
--- a/drivers/staging/lustre/lnet/selftest/conrpc.c
+++ b/drivers/staging/lustre/lnet/selftest/conrpc.c
@@ -70,11 +70,11 @@ void lstcon_rpc_stat_reply(struct lstcon_rpc_trans *, struct srpc_msg *,
 	/* not an orphan RPC */
 	crpc->crp_finished = 1;
 
-	if (!crpc->crp_stamp) {
+	if (!crpc->crp_stamp_ns) {
 		/* not aborted */
 		LASSERT(!crpc->crp_status);
 
-		crpc->crp_stamp = jiffies;
+		crpc->crp_stamp_ns = ktime_get_ns();
 		crpc->crp_status = rpc->crpc_status;
 	}
 
@@ -102,7 +102,7 @@ void lstcon_rpc_stat_reply(struct lstcon_rpc_trans *, struct srpc_msg *,
 	crpc->crp_finished = 0;
 	crpc->crp_unpacked = 0;
 	crpc->crp_status = 0;
-	crpc->crp_stamp = 0;
+	crpc->crp_stamp_ns = 0;
 	crpc->crp_embedded = embedded;
 	INIT_LIST_HEAD(&crpc->crp_link);
 
@@ -294,16 +294,16 @@ void lstcon_rpc_stat_reply(struct lstcon_rpc_trans *, struct srpc_msg *,
 		spin_lock(&rpc->crpc_lock);
 
 		if (!crpc->crp_posted || /* not posted */
-		    crpc->crp_stamp) {	 /* rpc done or aborted already */
-			if (!crpc->crp_stamp) {
-				crpc->crp_stamp = jiffies;
+		    crpc->crp_stamp_ns) {	 /* rpc done or aborted already */
+			if (!crpc->crp_stamp_ns) {
+				crpc->crp_stamp_ns = ktime_get_ns();
 				crpc->crp_status = -EINTR;
 			}
 			spin_unlock(&rpc->crpc_lock);
 			continue;
 		}
 
-		crpc->crp_stamp = jiffies;
+		crpc->crp_stamp_ns = ktime_get_ns();
 		crpc->crp_status = error;
 
 		spin_unlock(&rpc->crpc_lock);
@@ -314,10 +314,10 @@ void lstcon_rpc_stat_reply(struct lstcon_rpc_trans *, struct srpc_msg *,
 			continue;
 
 		nd = crpc->crp_node;
-		if (time_after(nd->nd_stamp, crpc->crp_stamp))
+		if (ktime_to_ns(nd->nd_stamp) > crpc->crp_stamp_ns)
 			continue;
 
-		nd->nd_stamp = crpc->crp_stamp;
+		nd->nd_stamp = ktime_set(0, crpc->crp_stamp_ns);
 		nd->nd_state = LST_NODE_DOWN;
 	}
 }
@@ -390,7 +390,7 @@ void lstcon_rpc_stat_reply(struct lstcon_rpc_trans *, struct srpc_msg *,
 	struct srpc_generic_reply *rep;
 
 	LASSERT(nd && rpc);
-	LASSERT(crpc->crp_stamp);
+	LASSERT(crpc->crp_stamp_ns);
 
 	if (crpc->crp_status) {
 		*msgpp = NULL;
@@ -403,10 +403,10 @@ void lstcon_rpc_stat_reply(struct lstcon_rpc_trans *, struct srpc_msg *,
 		crpc->crp_unpacked = 1;
 	}
 
-	if (time_after(nd->nd_stamp, crpc->crp_stamp))
+	if (ktime_to_ns(nd->nd_stamp) > crpc->crp_stamp_ns)
 		return 0;
 
-	nd->nd_stamp = crpc->crp_stamp;
+	nd->nd_stamp = ktime_set(0, crpc->crp_stamp_ns);
 	rep = &(*msgpp)->msg_body.reply;
 
 	if (rep->sid.ses_nid == LNET_NID_ANY)
@@ -433,7 +433,7 @@ void lstcon_rpc_stat_reply(struct lstcon_rpc_trans *, struct srpc_msg *,
 	list_for_each_entry(crpc, &trans->tas_rpcs_list, crp_link) {
 		lstcon_rpc_stat_total(stat, 1);
 
-		LASSERT(crpc->crp_stamp);
+		LASSERT(crpc->crp_stamp_ns);
 
 		error = lstcon_rpc_get_reply(crpc, &rep);
 		if (error) {
@@ -474,9 +474,9 @@ void lstcon_rpc_stat_reply(struct lstcon_rpc_trans *, struct srpc_msg *,
 	struct lstcon_rpc *crpc;
 	struct srpc_msg *msg;
 	struct lstcon_node *nd;
-	long dur;
 	struct timeval tv;
 	int error;
+	s64 dur;
 
 	LASSERT(head_up);
 
@@ -493,15 +493,15 @@ void lstcon_rpc_stat_reply(struct lstcon_rpc_trans *, struct srpc_msg *,
 
 		ent = list_entry(next, struct lstcon_rpc_ent, rpe_link);
 
-		LASSERT(crpc->crp_stamp);
+		LASSERT(crpc->crp_stamp_ns);
 
 		error = lstcon_rpc_get_reply(crpc, &msg);
 
 		nd = crpc->crp_node;
 
-		dur = (long)(crpc->crp_stamp -
-			     (unsigned long)console_session.ses_id.ses_stamp);
-		jiffies_to_timeval(dur, &tv);
+		dur = crpc->crp_stamp_ns -
+		      console_session.ses_id.ses_stamp * NSEC_PER_MSEC;
+		tv = ns_to_timeval(dur);
 
 		if (copy_to_user(&ent->rpe_peer, &nd->nd_id,
 				 sizeof(struct lnet_process_id)) ||
@@ -1246,7 +1246,8 @@ void lstcon_rpc_stat_reply(struct lstcon_rpc_trans *, struct srpc_msg *,
 		if (nd->nd_state != LST_NODE_ACTIVE)
 			continue;
 
-		intv = (jiffies - nd->nd_stamp) / msecs_to_jiffies(MSEC_PER_SEC);
+		intv = div_u64(ktime_ms_delta(ktime_get(), nd->nd_stamp),
+			       MSEC_PER_SEC);
 		if (intv < nd->nd_timeout / 2)
 			continue;
 
diff --git a/drivers/staging/lustre/lnet/selftest/conrpc.h b/drivers/staging/lustre/lnet/selftest/conrpc.h
index ce2f92d..dfc31b8 100644
--- a/drivers/staging/lustre/lnet/selftest/conrpc.h
+++ b/drivers/staging/lustre/lnet/selftest/conrpc.h
@@ -70,7 +70,7 @@ struct lstcon_rpc {
 	/** RPC is embedded in other structure and can't free it */
 	unsigned int		 crp_embedded:1;
 	int			 crp_status;	/* console rpc errors */
-	unsigned long		 crp_stamp;	/* replied time stamp */
+	s64			 crp_stamp_ns;	/* replied time stamp */
 };
 
 struct lstcon_rpc_trans {
diff --git a/drivers/staging/lustre/lnet/selftest/console.c b/drivers/staging/lustre/lnet/selftest/console.c
index 3c1c1b5..8e6f875 100644
--- a/drivers/staging/lustre/lnet/selftest/console.c
+++ b/drivers/staging/lustre/lnet/selftest/console.c
@@ -97,7 +97,7 @@
 
 	ndl->ndl_node->nd_ref = 1;
 	ndl->ndl_node->nd_id = id;
-	ndl->ndl_node->nd_stamp = jiffies;
+	ndl->ndl_node->nd_stamp = ktime_get();
 	ndl->ndl_node->nd_state = LST_NODE_UNKNOWN;
 	ndl->ndl_node->nd_timeout = 0;
 	memset(&ndl->ndl_node->nd_ping, 0, sizeof(struct lstcon_rpc));
@@ -1700,7 +1700,7 @@ static void lstcon_group_ndlink_release(struct lstcon_group *,
 
 	LNetGetId(1, &id);
 	sid->ses_nid = id.nid;
-	sid->ses_stamp = jiffies;
+	sid->ses_stamp = div_u64(ktime_get_ns(), NSEC_PER_MSEC);
 }
 
 int
diff --git a/drivers/staging/lustre/lnet/selftest/console.h b/drivers/staging/lustre/lnet/selftest/console.h
index 2826205..eaad07c 100644
--- a/drivers/staging/lustre/lnet/selftest/console.h
+++ b/drivers/staging/lustre/lnet/selftest/console.h
@@ -51,7 +51,7 @@ struct lstcon_node {
 	int		  nd_ref;     /* reference count */
 	int		  nd_state;   /* state of the node */
 	int		  nd_timeout; /* session timeout */
-	unsigned long	  nd_stamp;   /* timestamp of last replied RPC */
+	ktime_t			nd_stamp;   /* timestamp of last replied RPC */
 	struct lstcon_rpc nd_ping;    /* ping rpc */
 };
 
diff --git a/drivers/staging/lustre/lnet/selftest/framework.c b/drivers/staging/lustre/lnet/selftest/framework.c
index 741af10..6768aef 100644
--- a/drivers/staging/lustre/lnet/selftest/framework.c
+++ b/drivers/staging/lustre/lnet/selftest/framework.c
@@ -272,7 +272,7 @@
 	sn->sn_id = sid;
 	sn->sn_features = features;
 	sn->sn_timeout = session_timeout;
-	sn->sn_started = jiffies;
+	sn->sn_started = ktime_get();
 
 	timer->stt_data = sn;
 	timer->stt_func = sfw_session_expired;
@@ -384,7 +384,7 @@
 	 * send over the msecs since the session was started
 	 * with 32 bits to send, this is ~49 days
 	 */
-	cnt->running_ms = jiffies_to_msecs(jiffies - sn->sn_started);
+	cnt->running_ms = ktime_ms_delta(ktime_get(), sn->sn_started);
 	cnt->brw_errors = atomic_read(&sn->sn_brw_errors);
 	cnt->ping_errors = atomic_read(&sn->sn_ping_errors);
 	cnt->zombie_sessions = atomic_read(&sfw_data.fw_nzombies);
diff --git a/drivers/staging/lustre/lnet/selftest/selftest.h b/drivers/staging/lustre/lnet/selftest/selftest.h
index 8737fa9..76e02cd 100644
--- a/drivers/staging/lustre/lnet/selftest/selftest.h
+++ b/drivers/staging/lustre/lnet/selftest/selftest.h
@@ -331,7 +331,7 @@ struct sfw_session {
 	atomic_t	 sn_refcount;
 	atomic_t	 sn_brw_errors;
 	atomic_t	 sn_ping_errors;
-	unsigned long	 sn_started;
+	ktime_t		 sn_started;
 };
 
 #define sfw_sid_equal(sid0, sid1)     ((sid0).ses_nid == (sid1).ses_nid && \
diff --git a/drivers/staging/lustre/lnet/selftest/timer.c b/drivers/staging/lustre/lnet/selftest/timer.c
index 582f252..c83473e 100644
--- a/drivers/staging/lustre/lnet/selftest/timer.c
+++ b/drivers/staging/lustre/lnet/selftest/timer.c
@@ -55,7 +55,7 @@
 
 static struct st_timer_data {
 	spinlock_t	  stt_lock;
-	unsigned long	  stt_prev_slot; /* start time of the slot processed
+	time64_t	  stt_prev_slot; /* start time of the slot processed
 					  * previously
 					  */
 	struct list_head  stt_hash[STTIMER_NSLOTS];
@@ -144,18 +144,18 @@
 }
 
 static int
-stt_check_timers(unsigned long *last)
+stt_check_timers(time64_t *last)
 {
 	int expired = 0;
 	time64_t now;
-	unsigned long this_slot;
+	time64_t this_slot;
 
 	now = ktime_get_real_seconds();
 	this_slot = now & STTIMER_SLOTTIMEMASK;
 
 	spin_lock(&stt_data.stt_lock);
 
-	while (time_after_eq(this_slot, *last)) {
+	while (this_slot >= *last) {
 		expired += stt_expire_list(STTIMER_SLOT(this_slot), now);
 		this_slot = this_slot - STTIMER_SLOTTIME;
 	}
-- 
1.8.3.1


From neilb at suse.com  Sun Jun 24 23:25:38 2018
From: neilb at suse.com (NeilBrown)
Date: Mon, 25 Jun 2018 09:25:38 +1000
Subject: [lustre-devel] [PATCH 21/24] lustre: move remainder of
 linux-tracefile.c to tracefile.c
In-Reply-To: <alpine.LFD.2.21.1806242125460.28698@casper.infradead.org>
References: <152904663333.10587.10934053155404014785.stgit@noble>
 <152904669084.10587.6920959206498193596.stgit@noble>
 <alpine.LFD.2.21.1806210236200.3886@casper.infradead.org>
 <87y3f7bj9w.fsf@notabene.neil.brown.name>
 <alpine.LFD.2.21.1806242125460.28698@casper.infradead.org>
Message-ID: <87zhzjahsd.fsf@notabene.neil.brown.name>

On Sun, Jun 24 2018, James Simmons wrote:

>> On Thu, Jun 21 2018, James Simmons wrote:
>> 
>> >> It's good to keep related code together.
>> >> 
>> >> Signed-off-by: NeilBrown <neilb at suse.com>
>> >
>> > Nak. For some reason this corrupts my dmesg output. Its really strange and
>> > I haven't figured it out yet.
>> 
>> This patch cannot possibly be the cause of any such behavioural change.
>> It purely moves code from one file to another, it doesn't change the
>> code at all.
>
> In a way you are correct. What this patch does is expose how badly messed
> up cfs_print_to_console() is. Originally it was a printk() call but some
> newbie changed it incorrectly into a pr_info() for all cases. This is 
> totally incorrect but it landed anyways :-( For some bizarre reason it
> works even with it being completely wrong.

I see the problem with cfs_print_to_console().  It chooses a message
type (KERN_EMERG, KERN_ERR, KERN_WARNING, KERN_INFO) then uses pr_info()
to print that, which adds another KERN_INFO.
However vprintk_emit() calls printk_get_level() repeatedly while it
succeeds, so the extra KERNE_INFO is ignored.

So while the code is strange and should use printk(), I don't see
how it could mess up your dmesg output.

>                                              I did fix this patch up
> and redid the later patches due the changes from the fix up.  Is it okay
> it I push the newer version of these patches with my changes?

I'm not really sure what you are asking here, but the answer is probably
"yes, it is okay".

Thanks,
NeilBrown
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180625/5ef026fb/attachment.sig>

From neilb at suse.com  Sun Jun 24 23:26:11 2018
From: neilb at suse.com (NeilBrown)
Date: Mon, 25 Jun 2018 09:26:11 +1000
Subject: [lustre-devel] [PATCH 24/24] lustre: discard TCD_MAX_TYPES
In-Reply-To: <alpine.LFD.2.21.1806242136380.28698@casper.infradead.org>
References: <152904663333.10587.10934053155404014785.stgit@noble>
 <152904669094.10587.17019477478226952858.stgit@noble>
 <alpine.LFD.2.21.1806210237500.3886@casper.infradead.org>
 <87d0wkdaq0.fsf@notabene.neil.brown.name>
 <alpine.LFD.2.21.1806242136380.28698@casper.infradead.org>
Message-ID: <87wounahrg.fsf@notabene.neil.brown.name>

On Sun, Jun 24 2018, James Simmons wrote:

>> As well as CFS_TCD_TYPE_CNT we have TCD_MAX_TYPES which has a larger
>> value but a similar meaning.  Discard it and just use
>> CFS_TCD_TYPE_CNT.
>> 
>> Two places relied on the fact that TCD_MAX_TYPES was larger and so
>> there would be NULLs at the end of the array.  Change
>> them to check the array size properly.
>> 
>> Signed-off-by: NeilBrown <neilb at suse.com>
>> ---
>> 
>> Thanks for testing James!
>> I found two problems.
>> 1/ I had
>> -       for (i = 0; cfs_trace_data[i] && i < CFS_TCD_TYPE_CNT; i++) {
>> instead of
>> +       for (i = 0; i < CFS_TCD_TYPE_CNT && cfs_trace_data[i]; i++) {
>> 
>> So it could dereference beyond the end of an array.  I don't this was
>> the problem.
>> 
>> 2/ I hadn't changed
>> -       for (i = 0; cfs_trace_data[i]; i++)                             \
>> to
>> +       for (i = 0; i < CFS_TCD_TYPE_CNT && cfs_trace_data[i]; i++)                             \
>> 
>> so if cfs_trace_data[4] was non NULL, bad things could happen.
>> I suspect this is what happened to you.
>> 
>> Thanks,
>> NeilBrown
>
> Much better :-)

Thanks for the confirmation!

NeilBrown
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180625/499fd3ae/attachment.sig>

From neilb at suse.com  Sun Jun 24 23:40:49 2018
From: neilb at suse.com (NeilBrown)
Date: Mon, 25 Jun 2018 09:40:49 +1000
Subject: [lustre-devel] [PATCH 00/24] lustre - more cleanups including
 module reduction.
In-Reply-To: <DM3PR1101MB121322E85FDE1E5310F6F4B7CB760@DM3PR1101MB1213.namprd11.prod.outlook.com>
References: <152904663333.10587.10934053155404014785.stgit@noble>
 <alpine.LFD.2.21.1806210239190.3886@casper.infradead.org>
 <DM3PR1101MB1213DDB7FC857EE6B1444153CB760@DM3PR1101MB1213.namprd11.prod.outlook.com>
 <alpine.LFD.2.21.1806210354260.3886@casper.infradead.org>
 <3FD4D051-9C95-449D-8A23-D42B271E55B8@dilger.ca>
 <DM3PR1101MB121322E85FDE1E5310F6F4B7CB760@DM3PR1101MB1213.namprd11.prod.outlook.com>
Message-ID: <87tvprah32.fsf@notabene.neil.brown.name>

On Thu, Jun 21 2018, Patrick Farrell wrote:

> Ah, thanks Andreas.  Perhaps not coincidentally, Lustre 2.4 is the first release I worked on.
>
> Neil, I am really unenamored of the idea of the shared ptlrpc ldlm module being named ptlrpc...

Can you say why?  Is it the choice of name that bothers you, or the
combining of two things into the one kernel module, or something else?
Currently in  git://git.hpdd.intel.com/fs/lustre-release.git
(or the git tree that was until recently at the above address),
the module named "ptlrpc" contains ptlrpc code, ldlm code, and target
code.  I wonder what "target" means in this context.

>
> Also, Lustre currently has a bunch of module parameters which are used for configuration.  Thoughts on that?

Yes, the module parameters are an interesting part of the story.
libcfs has a bunch of module parameters that are symlinked from debugfs.
It seems that user-space largely uses the debugfs links to access them,
so they can become part of the "lnet" module with minimal pain.
Other modules have parameters that I haven't yet looked in to.

There probably will need to be user-space changes to support reduction
in the number of modules.

Thanks,
NeilBrown
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180625/0350635b/attachment.sig>

From neilb at suse.com  Sun Jun 24 23:50:38 2018
From: neilb at suse.com (NeilBrown)
Date: Mon, 25 Jun 2018 09:50:38 +1000
Subject: [lustre-devel] [PATCH 09/24] lustre: don't use spare bits in
 iattr.ia_valid
In-Reply-To: <alpine.LFD.2.21.1806242129180.28698@casper.infradead.org>
References: <152904663333.10587.10934053155404014785.stgit@noble>
 <152904669036.10587.17899565981649604892.stgit@noble>
 <alpine.LFD.2.21.1806210233170.3886@casper.infradead.org>
 <87fu1gdb4w.fsf@notabene.neil.brown.name>
 <871sczd0fi.fsf@notabene.neil.brown.name>
 <alpine.LFD.2.21.1806242129180.28698@casper.infradead.org>
Message-ID: <87r2kvagmp.fsf@notabene.neil.brown.name>

On Sun, Jun 24 2018, James Simmons wrote:

>
> Okay I did some more testing and found I get this error with or without
> this patch. Its due to a regression from an earlier patch. Can't figure
> out where its coming from. I looked at the d_lock usage and it seems
> balanced. So all but the first patch for the lustre_compact* header
> changes seem okay. BTW they need a rebasing. Also since I have separated
> out your libcfs tracefile fixes which I will push your okay with it so
> it might be best to break the patches into two series.

I've changed the problematic lustre_compat patch after a closer look at
the code.  I'll post the result shortly.

I don't understand your comment about "break the patches into two
series".
The goal for these patches isn't to make series', it is to apply them to
the "lustre" branch, when they are ready.  Some are ready (thanks for
the confirmation), some aren't.  The ones that aren't will get revised
and reposted.

I'm okay with you posting any patches that are useful, whether or not
they conflict with anything in my lustre-testing branch.  My tendency
will be to apply your patches first, then rebase my outstanding
lustre-testing patches on top of them.

Thanks,
NeilBrown
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180625/f89db3e4/attachment.sig>

From neilb at suse.com  Sun Jun 24 23:58:04 2018
From: neilb at suse.com (NeilBrown)
Date: Mon, 25 Jun 2018 09:58:04 +1000
Subject: [lustre-devel] [PATCH 00/24] lustre - more cleanups including
 module reduction.
In-Reply-To: <alpine.LFD.2.21.1806242140210.28698@casper.infradead.org>
References: <152904663333.10587.10934053155404014785.stgit@noble>
 <alpine.LFD.2.21.1806210239190.3886@casper.infradead.org>
 <87r2l0df25.fsf@notabene.neil.brown.name>
 <alpine.LFD.2.21.1806242140210.28698@casper.infradead.org>
Message-ID: <87o9fzagab.fsf@notabene.neil.brown.name>

On Sun, Jun 24 2018, James Simmons wrote:

>> >> Here are a bunch more cleanups that will appear in my lustre-testing
>> >> tree shortly.
>> >> 
>> >> The interesting stuff is some enhancements to kbuild to let us
>> >> build modules from multiple directories.
>> >> I've used that to clean up the build of ptlrpc+ldlm and to
>> >> merge libcfs and lnet into the one module - I've left the lnds
>> >> separate.
>> >
>> > That looks pretty good. It doesn't apply to the stable lustre tree but I
>> > did try it out on lustre-testing. The module building does seem to 
>> > function well. Only thing not clear is if do a
>> >
>> > make SUBDIRS=drivers/staging/lustre/lustre/lmv modules
>> 
>> Why would you do that?  SUBDIRS= is for building external modules.
>> What are you trying to achieve?
>> 
>> >
>> > do lmv.a and the other *.a files get relinked int lustre.ko ?
>> 
>> It isn't "lmv.a", it is ".../lustre/lmv/mod.a".
>
> Sorry I forgot the mod.a naming. I was asking to get a feel of how the
> work flow would change. It is common for lustre developers to build
> in tree and test without installing anything. The many module approach
> might of been a way for people to rebuild a module, unload module and
> then reload module quickly to test a change. I don't work this way so
> I can't say. I was looking to see if that work flow could still be
> done.

This general approach should still work.
You wouldn't do
  make SUBDIRS=drivers/staging/lustre/lustre/lmv modules

unless there was a module built in that directory, but you might

  make SUBDIRS=drivers/staging/lustre modules

to just build whatever lustre modules there are.

NeilBrown

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180625/d09e4f75/attachment.sig>

From neilb at suse.com  Mon Jun 25 00:02:07 2018
From: neilb at suse.com (NeilBrown)
Date: Mon, 25 Jun 2018 10:02:07 +1000
Subject: [lustre-devel] [PATCH 1/3] lustre: use generic_error_remove_page()
In-Reply-To: <877emwg8ao.fsf@notabene.neil.brown.name>
References: <152904663333.10587.10934053155404014785.stgit@noble>
 <152904669032.10587.11474850109829193046.stgit@noble>
 <alpine.LFD.2.21.1806160115430.15679@casper.infradead.org>
 <87y3ffft80.fsf@notabene.neil.brown.name>
 <61C2F92E-8F88-49C6-83FB-F3013289265C@linuxhacker.ru>
 <877emwg8ao.fsf@notabene.neil.brown.name>
Message-ID: <87lgb3ag3k.fsf@notabene.neil.brown.name>


lustre's internal ll_invalidate_page() is behaviourally identical to
generic_error_remove_page().
In the case of lustre it isn't a memory hardware error that requires
the page being invalidated, it is the loss of a lock, which will like
result in the data changing on the server.
In either case, we don't want the page to be accessed any more, so the
same removal is appropriate.

Signed-off-by: NeilBrown <neilb at suse.com>
---

I've replaced
 [PATCH 08/24] lustre: use truncate_inode_page in place of truncate_complete_page
with 3 patches, this and the next two.

Thanks,
NeilBrown


 drivers/staging/lustre/lustre/llite/llite_internal.h | 17 -----------------
 drivers/staging/lustre/lustre/llite/vvp_io.c         |  2 +-
 drivers/staging/lustre/lustre/llite/vvp_page.c       |  2 +-
 3 files changed, 2 insertions(+), 19 deletions(-)

diff --git a/drivers/staging/lustre/lustre/llite/llite_internal.h b/drivers/staging/lustre/lustre/llite/llite_internal.h
index c08a6e14b6d7..22dcabf6de0f 100644
--- a/drivers/staging/lustre/lustre/llite/llite_internal.h
+++ b/drivers/staging/lustre/lustre/llite/llite_internal.h
@@ -928,23 +928,6 @@ void policy_from_vma(union ldlm_policy_data *policy, struct vm_area_struct *vma,
 struct vm_area_struct *our_vma(struct mm_struct *mm, unsigned long addr,
 			       size_t count);
 
-static inline void ll_invalidate_page(struct page *vmpage)
-{
-	struct address_space *mapping = vmpage->mapping;
-	loff_t offset = vmpage->index << PAGE_SHIFT;
-
-	LASSERT(PageLocked(vmpage));
-	if (!mapping)
-		return;
-
-	/*
-	 * truncate_complete_page() calls
-	 * a_ops->invalidatepage()->cl_page_delete()->vvp_page_delete().
-	 */
-	ll_teardown_mmaps(mapping, offset, offset + PAGE_SIZE);
-	truncate_complete_page(mapping, vmpage);
-}
-
 #define    ll_s2sbi(sb)	(s2lsi(sb)->lsi_llsbi)
 
 /* don't need an addref as the sb_info should be holding one */
diff --git a/drivers/staging/lustre/lustre/llite/vvp_io.c b/drivers/staging/lustre/lustre/llite/vvp_io.c
index e7a4778e02e4..5a67955974ad 100644
--- a/drivers/staging/lustre/lustre/llite/vvp_io.c
+++ b/drivers/staging/lustre/lustre/llite/vvp_io.c
@@ -1098,7 +1098,7 @@ static int vvp_io_fault_start(const struct lu_env *env,
 	LASSERT(PageLocked(vmpage));
 
 	if (OBD_FAIL_CHECK(OBD_FAIL_LLITE_FAULT_TRUNC_RACE))
-		ll_invalidate_page(vmpage);
+		generic_error_remove_page(vmpage->mapping, vmpage);
 
 	size = i_size_read(inode);
 	/* Though we have already held a cl_lock upon this page, but
diff --git a/drivers/staging/lustre/lustre/llite/vvp_page.c b/drivers/staging/lustre/lustre/llite/vvp_page.c
index 6eb0565ddc22..dcc4d8faa0cd 100644
--- a/drivers/staging/lustre/lustre/llite/vvp_page.c
+++ b/drivers/staging/lustre/lustre/llite/vvp_page.c
@@ -147,7 +147,7 @@ static void vvp_page_discard(const struct lu_env *env,
 	if (vpg->vpg_defer_uptodate && !vpg->vpg_ra_used)
 		ll_ra_stats_inc(vmpage->mapping->host, RA_STAT_DISCARDED);
 
-	ll_invalidate_page(vmpage);
+	generic_error_remove_page(vmpage->mapping, vmpage);
 }
 
 static void vvp_page_delete(const struct lu_env *env,
-- 
2.14.0.rc0.dirty

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180625/20e31790/attachment.sig>

From neilb at suse.com  Mon Jun 25 00:02:55 2018
From: neilb at suse.com (NeilBrown)
Date: Mon, 25 Jun 2018 10:02:55 +1000
Subject: [lustre-devel] [PATCH 2/3] lustre: use delete_from_page_cache() for
 directory pages.
In-Reply-To: <87lgb3ag3k.fsf@notabene.neil.brown.name>
References: <152904663333.10587.10934053155404014785.stgit@noble>
 <152904669032.10587.11474850109829193046.stgit@noble>
 <alpine.LFD.2.21.1806160115430.15679@casper.infradead.org>
 <87y3ffft80.fsf@notabene.neil.brown.name>
 <61C2F92E-8F88-49C6-83FB-F3013289265C@linuxhacker.ru>
 <877emwg8ao.fsf@notabene.neil.brown.name>
 <87lgb3ag3k.fsf@notabene.neil.brown.name>
Message-ID: <87in67ag28.fsf@notabene.neil.brown.name>


lustre sometimes uses the internal function truncate_complete_page()
to remove a page of a directory.
Must of what this function does, does not apply to directory pages
as there is no invalidatepage function, and at these times, the
page is not dirty.
The only useful part of the function is delete_from_page_cache(),
so just call that directly.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 drivers/staging/lustre/lustre/llite/dir.c       | 2 +-
 drivers/staging/lustre/lustre/mdc/mdc_request.c | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/staging/lustre/lustre/llite/dir.c b/drivers/staging/lustre/lustre/llite/dir.c
index 688dddf3ca47..52a8ecc2e2e0 100644
--- a/drivers/staging/lustre/lustre/llite/dir.c
+++ b/drivers/staging/lustre/lustre/llite/dir.c
@@ -168,7 +168,7 @@ void ll_release_page(struct inode *inode, struct page *page, bool remove)
 	if (remove) {
 		lock_page(page);
 		if (likely(page->mapping))
-			truncate_complete_page(page->mapping, page);
+			delete_from_page_cache(page);
 		unlock_page(page);
 	}
 	put_page(page);
diff --git a/drivers/staging/lustre/lustre/mdc/mdc_request.c b/drivers/staging/lustre/lustre/mdc/mdc_request.c
index cff31cb0a9ac..827ed0ca0987 100644
--- a/drivers/staging/lustre/lustre/mdc/mdc_request.c
+++ b/drivers/staging/lustre/lustre/mdc/mdc_request.c
@@ -927,7 +927,7 @@ static void mdc_release_page(struct page *page, int remove)
 	if (remove) {
 		lock_page(page);
 		if (likely(page->mapping))
-			truncate_complete_page(page->mapping, page);
+			delete_from_page_cache(page);
 		unlock_page(page);
 	}
 	put_page(page);
-- 
2.14.0.rc0.dirty

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180625/9f5eb849/attachment.sig>

From neilb at suse.com  Mon Jun 25 00:03:26 2018
From: neilb at suse.com (NeilBrown)
Date: Mon, 25 Jun 2018 10:03:26 +1000
Subject: [lustre-devel] [PATCH 3/3] lustre: discard truncate_complete_page()
In-Reply-To: <87in67ag28.fsf@notabene.neil.brown.name>
References: <152904663333.10587.10934053155404014785.stgit@noble>
 <152904669032.10587.11474850109829193046.stgit@noble>
 <alpine.LFD.2.21.1806160115430.15679@casper.infradead.org>
 <87y3ffft80.fsf@notabene.neil.brown.name>
 <61C2F92E-8F88-49C6-83FB-F3013289265C@linuxhacker.ru>
 <877emwg8ao.fsf@notabene.neil.brown.name>
 <87lgb3ag3k.fsf@notabene.neil.brown.name>
 <87in67ag28.fsf@notabene.neil.brown.name>
Message-ID: <87fu1bag1d.fsf@notabene.neil.brown.name>


This function is no longer used, so remove it.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 .../lustre/lustre/include/lustre_patchless_compat.h      | 16 ----------------
 1 file changed, 16 deletions(-)

diff --git a/drivers/staging/lustre/lustre/include/lustre_patchless_compat.h b/drivers/staging/lustre/lustre/include/lustre_patchless_compat.h
index 298476ea7557..dca1e3dbd183 100644
--- a/drivers/staging/lustre/lustre/include/lustre_patchless_compat.h
+++ b/drivers/staging/lustre/lustre/include/lustre_patchless_compat.h
@@ -41,22 +41,6 @@
 #include <linux/hash.h>
 #include <linux/pagemap.h>
 
-#define ll_delete_from_page_cache(page) delete_from_page_cache(page)
-
-static inline void
-truncate_complete_page(struct address_space *mapping, struct page *page)
-{
-	if (page->mapping != mapping)
-		return;
-
-	if (PagePrivate(page))
-		page->mapping->a_ops->invalidatepage(page, 0, PAGE_SIZE);
-
-	cancel_dirty_page(page);
-	ClearPageMappedToDisk(page);
-	ll_delete_from_page_cache(page);
-}
-
 #ifndef ATTR_CTIME_SET
 /*
  * set ATTR_CTIME_SET to a high value to avoid any risk of collision with other
-- 
2.14.0.rc0.dirty

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180625/bb5e4e4c/attachment.sig>

From neilb at suse.com  Mon Jun 25 00:20:27 2018
From: neilb at suse.com (NeilBrown)
Date: Mon, 25 Jun 2018 10:20:27 +1000
Subject: [lustre-devel] [PATCH v3 03/26] staging: lustre: libcfs:
 properly handle failure cases in SMP code
In-Reply-To: <1529875250-11531-4-git-send-email-jsimmons@infradead.org>
References: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
 <1529875250-11531-4-git-send-email-jsimmons@infradead.org>
Message-ID: <87d0wfaf90.fsf@notabene.neil.brown.name>

On Sun, Jun 24 2018, James Simmons wrote:

> While pushing the SMP work some bugs were pointed out by Dan
> Carpenter in the code. Due to single err label in cfs_cpu_init()
> and cfs_cpt_table_alloc() a few items were being cleaned up that
> were never initialized. This can lead to crashed and other problems.
> In those initialization function introduce individual labels to
> jump to only the thing initialized get freed on failure.
>
> Signed-off-by: James Simmons <uja.ornl at yahoo.com>
> WC-bug-id: https://jira.whamcloud.com/browse/LU-10932
> Reviewed-on: https://review.whamcloud.com/32085
> Reviewed-by: Dmitry Eremin <dmitry.eremin at intel.com>
> Reviewed-by: Andreas Dilger <adilger at whamcloud.com>
> Signed-off-by: James Simmons <jsimmons at infradead.org>
> ---
>  drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c | 72 ++++++++++++++++++-------
>  1 file changed, 52 insertions(+), 20 deletions(-)
>
> diff --git a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
> index 46d3530..bdd71a3 100644
> --- a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
> +++ b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
> @@ -85,17 +85,19 @@ struct cfs_cpt_table *
>  
>  	cptab->ctb_nparts = ncpt;
>  
> +	if (!zalloc_cpumask_var(&cptab->ctb_cpumask, GFP_NOFS))
> +		goto failed_alloc_cpumask;
> +
>  	cptab->ctb_nodemask = kzalloc(sizeof(*cptab->ctb_nodemask),
>  				      GFP_NOFS);
> -	if (!zalloc_cpumask_var(&cptab->ctb_cpumask, GFP_NOFS) ||
> -	    !cptab->ctb_nodemask)
> -		goto failed;
> +	if (!cptab->ctb_nodemask)
> +		goto failed_alloc_nodemask;
>  
>  	cptab->ctb_cpu2cpt = kvmalloc_array(num_possible_cpus(),
>  					    sizeof(cptab->ctb_cpu2cpt[0]),
>  					    GFP_KERNEL);
>  	if (!cptab->ctb_cpu2cpt)
> -		goto failed;
> +		goto failed_alloc_cpu2cpt;
>  
>  	memset(cptab->ctb_cpu2cpt, -1,
>  	       num_possible_cpus() * sizeof(cptab->ctb_cpu2cpt[0]));
> @@ -103,22 +105,41 @@ struct cfs_cpt_table *
>  	cptab->ctb_parts = kvmalloc_array(ncpt, sizeof(cptab->ctb_parts[0]),
>  					  GFP_KERNEL);
>  	if (!cptab->ctb_parts)
> -		goto failed;
> +		goto failed_alloc_ctb_parts;
> +
> +	memset(cptab->ctb_parts, -1, ncpt * sizeof(cptab->ctb_parts[0]));
>  
>  	for (i = 0; i < ncpt; i++) {
>  		struct cfs_cpu_partition *part = &cptab->ctb_parts[i];
>  
> +		if (!zalloc_cpumask_var(&part->cpt_cpumask, GFP_NOFS))
> +			goto failed_setting_ctb_parts;
> +
>  		part->cpt_nodemask = kzalloc(sizeof(*part->cpt_nodemask),
>  					     GFP_NOFS);
> -		if (!zalloc_cpumask_var(&part->cpt_cpumask, GFP_NOFS) ||
> -		    !part->cpt_nodemask)
> -			goto failed;
> +		if (!part->cpt_nodemask)
> +			goto failed_setting_ctb_parts;

If zalloc_cpumask_var() succeeds, but kzalloc() fails (which is almost
impossible, but still) we go to failed_setting_ctb_parts, with
 cptab->ctb_parts[i]->cpt_cpumask needing to be freed.

>  	}
>  
>  	return cptab;
>  
> - failed:
> -	cfs_cpt_table_free(cptab);
> +failed_setting_ctb_parts:
> +	while (i-- >= 0) {

but we don't free anything in cptab->ctb_parts[i].
I've fix this by calling free_cpumask_var() before the goto.

And will propagate the change through future patches in this series.

> +		struct cfs_cpu_partition *part = &cptab->ctb_parts[i];
> +
> +		kfree(part->cpt_nodemask);
> +		free_cpumask_var(part->cpt_cpumask);
> +	}
> +
> +	kvfree(cptab->ctb_parts);
> +failed_alloc_ctb_parts:
> +	kvfree(cptab->ctb_cpu2cpt);
> +failed_alloc_cpu2cpt:
> +	kfree(cptab->ctb_nodemask);
> +failed_alloc_nodemask:
> +	free_cpumask_var(cptab->ctb_cpumask);
> +failed_alloc_cpumask:
> +	kfree(cptab);
>  	return NULL;
>  }
>  EXPORT_SYMBOL(cfs_cpt_table_alloc);
> @@ -944,7 +965,7 @@ static int cfs_cpu_dead(unsigned int cpu)
>  int
>  cfs_cpu_init(void)
>  {
> -	int ret = 0;
> +	int ret;
>  
>  	LASSERT(!cfs_cpt_tab);
>  
> @@ -953,23 +974,23 @@ static int cfs_cpu_dead(unsigned int cpu)
>  					"staging/lustre/cfe:dead", NULL,
>  					cfs_cpu_dead);
>  	if (ret < 0)
> -		goto failed;
> +		goto failed_cpu_dead;
> +
>  	ret = cpuhp_setup_state_nocalls(CPUHP_AP_ONLINE_DYN,
>  					"staging/lustre/cfe:online",
>  					cfs_cpu_online, NULL);
>  	if (ret < 0)
> -		goto failed;
> +		goto failed_cpu_online;
> +
>  	lustre_cpu_online = ret;
>  #endif
> -	ret = -EINVAL;
> -
>  	get_online_cpus();
>  	if (*cpu_pattern) {
>  		char *cpu_pattern_dup = kstrdup(cpu_pattern, GFP_KERNEL);
>  
>  		if (!cpu_pattern_dup) {
>  			CERROR("Failed to duplicate cpu_pattern\n");
> -			goto failed;
> +			goto failed_alloc_table;
>  		}
>  
>  		cfs_cpt_tab = cfs_cpt_table_create_pattern(cpu_pattern_dup);
> @@ -977,7 +998,7 @@ static int cfs_cpu_dead(unsigned int cpu)
>  		if (!cfs_cpt_tab) {
>  			CERROR("Failed to create cptab from pattern %s\n",
>  			       cpu_pattern);
> -			goto failed;
> +			goto failed_alloc_table;
>  		}
>  
>  	} else {
> @@ -985,7 +1006,7 @@ static int cfs_cpu_dead(unsigned int cpu)
>  		if (!cfs_cpt_tab) {
>  			CERROR("Failed to create ptable with npartitions %d\n",
>  			       cpu_npartitions);
> -			goto failed;
> +			goto failed_alloc_table;
>  		}
>  	}
>  
> @@ -996,8 +1017,19 @@ static int cfs_cpu_dead(unsigned int cpu)
>  		 cfs_cpt_number(cfs_cpt_tab));
>  	return 0;
>  
> - failed:
> +failed_alloc_table:
>  	put_online_cpus();
> -	cfs_cpu_fini();
> +
> +	if (cfs_cpt_tab)
> +		cfs_cpt_table_free(cfs_cpt_tab);
> +
> +	ret = -EINVAL;
> +#ifdef CONFIG_HOTPLUG_CPU
> +	if (lustre_cpu_online > 0)
> +		cpuhp_remove_state_nocalls(lustre_cpu_online);
> +failed_cpu_online:
> +	cpuhp_remove_state_nocalls(CPUHP_LUSTRE_CFS_DEAD);
> +failed_cpu_dead:
> +#endif
>  	return ret;
>  }
> -- 
> 1.8.3.1
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180625/120e6fbe/attachment.sig>

From neilb at suse.com  Mon Jun 25 00:35:24 2018
From: neilb at suse.com (NeilBrown)
Date: Mon, 25 Jun 2018 10:35:24 +1000
Subject: [lustre-devel] [PATCH v3 05/26] staging: lustre: libcfs: remove
 excess space
In-Reply-To: <1529875250-11531-6-git-send-email-jsimmons@infradead.org>
References: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
 <1529875250-11531-6-git-send-email-jsimmons@infradead.org>
Message-ID: <87a7rjaek3.fsf@notabene.neil.brown.name>

On Sun, Jun 24 2018, James Simmons wrote:

> From: Amir Shehata <amir.shehata at intel.com>
>
> The function cfs_cpt_table_print() was adding two spaces
> to the string buffer. Just add it once.

No it doesn't.  Maybe it did in the out-of-tree code, but the linux code
is different.

The extra space is

                       rc = snprintf(tmp, len, " %d", j);

But in Linux that is

			rc = snprintf(tmp, len, "%d ", j);

Both are wrong, but for different reasons.
I've change this patch to be:

			rc = snprintf(tmp, len, "%d\t:", i);
and
			rc = snprintf(tmp, len, " %d", j);
and changed the comment to say that we don't need a stray space at the
end of the line.

NeilBrown



>
> Signed-off-by: Amir Shehata <amir.shehata at intel.com>
> WC-bug-id: https://jira.whamcloud.com/browse/LU-7734
> Reviewed-on: http://review.whamcloud.com/18916
> Reviewed-by: Olaf Weber <olaf at sgi.com>
> Reviewed-by: Doug Oucharek <dougso at me.com>
> Reviewed-by: Oleg Drokin <green at whamcloud.com>
> Signed-off-by: James Simmons <jsimmons at infradead.org>
> ---
>  drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
> index ea8d55c..680a2b1 100644
> --- a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
> +++ b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
> @@ -177,7 +177,7 @@ struct cfs_cpt_table *
>  
>  	for (i = 0; i < cptab->ctb_nparts; i++) {
>  		if (len > 0) {
> -			rc = snprintf(tmp, len, "%d\t: ", i);
> +			rc = snprintf(tmp, len, "%d\t:", i);
>  			len -= rc;
>  		}
>  
> -- 
> 1.8.3.1
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180625/a16a4b8f/attachment.sig>

From green at linuxhacker.ru  Mon Jun 25 00:38:02 2018
From: green at linuxhacker.ru (Oleg Drokin)
Date: Sun, 24 Jun 2018 20:38:02 -0400
Subject: [lustre-devel] [PATCH 2/3] lustre: use delete_from_page_cache()
 for directory pages.
In-Reply-To: <87in67ag28.fsf@notabene.neil.brown.name>
References: <152904663333.10587.10934053155404014785.stgit@noble>
 <152904669032.10587.11474850109829193046.stgit@noble>
 <alpine.LFD.2.21.1806160115430.15679@casper.infradead.org>
 <87y3ffft80.fsf@notabene.neil.brown.name>
 <61C2F92E-8F88-49C6-83FB-F3013289265C@linuxhacker.ru>
 <877emwg8ao.fsf@notabene.neil.brown.name>
 <87lgb3ag3k.fsf@notabene.neil.brown.name>
 <87in67ag28.fsf@notabene.neil.brown.name>
Message-ID: <1ADABFB0-16E3-447C-B6F3-03674F324E1F@linuxhacker.ru>


> On Jun 24, 2018, at 8:02 PM, NeilBrown <neilb at suse.com> wrote:
> 
> 
> lustre sometimes uses the internal function truncate_complete_page()
> to remove a page of a directory.
> Must of what this function does, does not apply to directory pages
> as there is no invalidatepage function, and at these times, the
> page is not dirty.

> The only useful part of the function is delete_from_page_cache(),
> so just call that directly.

I guess this is also fine since we indeed do not have dirty directory
pages in Lustre

Acked-by: Oleg Drokin <green at linuxhacker.ru>

> 
> Signed-off-by: NeilBrown <neilb at suse.com>
> ---
> drivers/staging/lustre/lustre/llite/dir.c       | 2 +-
> drivers/staging/lustre/lustre/mdc/mdc_request.c | 2 +-
> 2 files changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/staging/lustre/lustre/llite/dir.c b/drivers/staging/lustre/lustre/llite/dir.c
> index 688dddf3ca47..52a8ecc2e2e0 100644
> --- a/drivers/staging/lustre/lustre/llite/dir.c
> +++ b/drivers/staging/lustre/lustre/llite/dir.c
> @@ -168,7 +168,7 @@ void ll_release_page(struct inode *inode, struct page *page, bool remove)
> 	if (remove) {
> 		lock_page(page);
> 		if (likely(page->mapping))
> -			truncate_complete_page(page->mapping, page);
> +			delete_from_page_cache(page);
> 		unlock_page(page);
> 	}
> 	put_page(page);
> diff --git a/drivers/staging/lustre/lustre/mdc/mdc_request.c b/drivers/staging/lustre/lustre/mdc/mdc_request.c
> index cff31cb0a9ac..827ed0ca0987 100644
> --- a/drivers/staging/lustre/lustre/mdc/mdc_request.c
> +++ b/drivers/staging/lustre/lustre/mdc/mdc_request.c
> @@ -927,7 +927,7 @@ static void mdc_release_page(struct page *page, int remove)
> 	if (remove) {
> 		lock_page(page);
> 		if (likely(page->mapping))
> -			truncate_complete_page(page->mapping, page);
> +			delete_from_page_cache(page);
> 		unlock_page(page);
> 	}
> 	put_page(page);
> -- 
> 2.14.0.rc0.dirty
> 


From green at linuxhacker.ru  Mon Jun 25 00:38:00 2018
From: green at linuxhacker.ru (Oleg Drokin)
Date: Sun, 24 Jun 2018 20:38:00 -0400
Subject: [lustre-devel] [PATCH 1/3] lustre: use
 generic_error_remove_page()
In-Reply-To: <87lgb3ag3k.fsf@notabene.neil.brown.name>
References: <152904663333.10587.10934053155404014785.stgit@noble>
 <152904669032.10587.11474850109829193046.stgit@noble>
 <alpine.LFD.2.21.1806160115430.15679@casper.infradead.org>
 <87y3ffft80.fsf@notabene.neil.brown.name>
 <61C2F92E-8F88-49C6-83FB-F3013289265C@linuxhacker.ru>
 <877emwg8ao.fsf@notabene.neil.brown.name>
 <87lgb3ag3k.fsf@notabene.neil.brown.name>
Message-ID: <EAC0696A-9C7F-419A-A360-C352537B1DA2@linuxhacker.ru>


> On Jun 24, 2018, at 8:02 PM, NeilBrown <neilb at suse.com> wrote:
> 
> 
> lustre's internal ll_invalidate_page() is behaviourally identical to
> generic_error_remove_page().
> In the case of lustre it isn't a memory hardware error that requires
> the page being invalidated, it is the loss of a lock, which will like
> result in the data changing on the server.
> In either case, we don't want the page to be accessed any more, so the
> same removal is appropriate.
> 
> Signed-off-by: NeilBrown <neilb at suse.com>
> ---
> 
> I've replaced
> [PATCH 08/24] lustre: use truncate_inode_page in place of truncate_complete_page
> with 3 patches, this and the next two.

This looks reasonable. Are you running any tests on this?

Acked-by: Oleg Drokin <green at linuxhacker.ru>

> 
> Thanks,
> NeilBrown
> 
> 
> drivers/staging/lustre/lustre/llite/llite_internal.h | 17 -----------------
> drivers/staging/lustre/lustre/llite/vvp_io.c         |  2 +-
> drivers/staging/lustre/lustre/llite/vvp_page.c       |  2 +-
> 3 files changed, 2 insertions(+), 19 deletions(-)
> 
> diff --git a/drivers/staging/lustre/lustre/llite/llite_internal.h b/drivers/staging/lustre/lustre/llite/llite_internal.h
> index c08a6e14b6d7..22dcabf6de0f 100644
> --- a/drivers/staging/lustre/lustre/llite/llite_internal.h
> +++ b/drivers/staging/lustre/lustre/llite/llite_internal.h
> @@ -928,23 +928,6 @@ void policy_from_vma(union ldlm_policy_data *policy, struct vm_area_struct *vma,
> struct vm_area_struct *our_vma(struct mm_struct *mm, unsigned long addr,
> 			       size_t count);
> 
> -static inline void ll_invalidate_page(struct page *vmpage)
> -{
> -	struct address_space *mapping = vmpage->mapping;
> -	loff_t offset = vmpage->index << PAGE_SHIFT;
> -
> -	LASSERT(PageLocked(vmpage));
> -	if (!mapping)
> -		return;
> -
> -	/*
> -	 * truncate_complete_page() calls
> -	 * a_ops->invalidatepage()->cl_page_delete()->vvp_page_delete().
> -	 */
> -	ll_teardown_mmaps(mapping, offset, offset + PAGE_SIZE);
> -	truncate_complete_page(mapping, vmpage);
> -}
> -
> #define    ll_s2sbi(sb)	(s2lsi(sb)->lsi_llsbi)
> 
> /* don't need an addref as the sb_info should be holding one */
> diff --git a/drivers/staging/lustre/lustre/llite/vvp_io.c b/drivers/staging/lustre/lustre/llite/vvp_io.c
> index e7a4778e02e4..5a67955974ad 100644
> --- a/drivers/staging/lustre/lustre/llite/vvp_io.c
> +++ b/drivers/staging/lustre/lustre/llite/vvp_io.c
> @@ -1098,7 +1098,7 @@ static int vvp_io_fault_start(const struct lu_env *env,
> 	LASSERT(PageLocked(vmpage));
> 
> 	if (OBD_FAIL_CHECK(OBD_FAIL_LLITE_FAULT_TRUNC_RACE))
> -		ll_invalidate_page(vmpage);
> +		generic_error_remove_page(vmpage->mapping, vmpage);
> 
> 	size = i_size_read(inode);
> 	/* Though we have already held a cl_lock upon this page, but
> diff --git a/drivers/staging/lustre/lustre/llite/vvp_page.c b/drivers/staging/lustre/lustre/llite/vvp_page.c
> index 6eb0565ddc22..dcc4d8faa0cd 100644
> --- a/drivers/staging/lustre/lustre/llite/vvp_page.c
> +++ b/drivers/staging/lustre/lustre/llite/vvp_page.c
> @@ -147,7 +147,7 @@ static void vvp_page_discard(const struct lu_env *env,
> 	if (vpg->vpg_defer_uptodate && !vpg->vpg_ra_used)
> 		ll_ra_stats_inc(vmpage->mapping->host, RA_STAT_DISCARDED);
> 
> -	ll_invalidate_page(vmpage);
> +	generic_error_remove_page(vmpage->mapping, vmpage);
> }
> 
> static void vvp_page_delete(const struct lu_env *env,
> -- 
> 2.14.0.rc0.dirty
> 


From neilb at suse.com  Mon Jun 25 00:39:31 2018
From: neilb at suse.com (NeilBrown)
Date: Mon, 25 Jun 2018 10:39:31 +1000
Subject: [lustre-devel] [PATCH v3 07/26] staging: lustre: libcfs: NUMA
 support
In-Reply-To: <1529875250-11531-8-git-send-email-jsimmons@infradead.org>
References: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
 <1529875250-11531-8-git-send-email-jsimmons@infradead.org>
Message-ID: <877emnaed8.fsf@notabene.neil.brown.name>

On Sun, Jun 24 2018, James Simmons wrote:

> From: Amir Shehata <amir.shehata at intel.com>
>
> This patch adds NUMA node support. NUMA node information is stored
> in the CPT table. A NUMA node mask is maintained for the entire
> table as well as for each CPT to track the NUMA nodes related to
> each of the CPTs. Add new function cfs_cpt_of_node() which returns
> the CPT of a particular NUMA node.

I note that you didn't respond to Greg's questions about this patch.
I'll accept it anyway in the interests of moving forward, but I think
his comments were probably valid, and need to be considered at some
stage.

There is a bug though....
>
> Signed-off-by: Amir Shehata <amir.shehata at intel.com>
> WC-bug-id: https://jira.whamcloud.com/browse/LU-7734
> Reviewed-on: http://review.whamcloud.com/18916
> Reviewed-by: Olaf Weber <olaf at sgi.com>
> Reviewed-by: Doug Oucharek <dougso at me.com>
> Signed-off-by: James Simmons <jsimmons at infradead.org>
> ---
>  .../lustre/include/linux/libcfs/libcfs_cpu.h        | 11 +++++++++++
>  drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c     | 21 +++++++++++++++++++++
>  2 files changed, 32 insertions(+)
>
> diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h
> index 1b4333d..ff3ecf5 100644
> --- a/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h
> +++ b/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h
> @@ -103,6 +103,8 @@ struct cfs_cpt_table {
>  	int				*ctb_cpu2cpt;
>  	/* all cpus in this partition table */
>  	cpumask_var_t			ctb_cpumask;
> +	/* shadow HW node to CPU partition ID */
> +	int				*ctb_node2cpt;
>  	/* all nodes in this partition table */
>  	nodemask_t			*ctb_nodemask;
>  };
> @@ -143,6 +145,10 @@ struct cfs_cpt_table {
>   */
>  int cfs_cpt_of_cpu(struct cfs_cpt_table *cptab, int cpu);
>  /**
> + * shadow HW node ID \a NODE to CPU-partition ID by \a cptab
> + */
> +int cfs_cpt_of_node(struct cfs_cpt_table *cptab, int node);
> +/**
>   * bind current thread on a CPU-partition \a cpt of \a cptab
>   */
>  int cfs_cpt_bind(struct cfs_cpt_table *cptab, int cpt);
> @@ -299,6 +305,11 @@ void cfs_cpt_unset_nodemask(struct cfs_cpt_table *cptab,
>  	return 0;
>  }
>  
> +static inline int cfs_cpt_of_node(struct cfs_cpt_table *cptab, int node)
> +{
> +	return 0;
> +}
> +
>  static inline int
>  cfs_cpt_bind(struct cfs_cpt_table *cptab, int cpt)
>  {
> diff --git a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
> index 33294da..8c5cf7b 100644
> --- a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
> +++ b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
> @@ -102,6 +102,15 @@ struct cfs_cpt_table *
>  	memset(cptab->ctb_cpu2cpt, -1,
>  	       nr_cpu_ids * sizeof(cptab->ctb_cpu2cpt[0]));
>  
> +	cptab->ctb_node2cpt = kvmalloc_array(nr_node_ids,
> +					     sizeof(cptab->ctb_node2cpt[0]),
> +					     GFP_KERNEL);
> +	if (!cptab->ctb_node2cpt)
> +		goto failed_alloc_node2cpt;
> +
> +	memset(cptab->ctb_node2cpt, -1,
> +	       nr_node_ids * sizeof(cptab->ctb_node2cpt[0]));
> +
>  	cptab->ctb_parts = kvmalloc_array(ncpt, sizeof(cptab->ctb_parts[0]),
>  					  GFP_KERNEL);
>  	if (!cptab->ctb_parts)
> @@ -133,6 +142,8 @@ struct cfs_cpt_table *
>  
>  	kvfree(cptab->ctb_parts);
>  failed_alloc_ctb_parts:
> +	kvfree(cptab->ctb_node2cpt);
> +failed_alloc_node2cpt:
>  	kvfree(cptab->ctb_cpu2cpt);
>  failed_alloc_cpu2cpt:
>  	kfree(cptab->ctb_nodemask);
> @@ -150,6 +161,7 @@ struct cfs_cpt_table *
>  	int i;
>  
>  	kvfree(cptab->ctb_cpu2cpt);
> +	kvfree(cptab->ctb_node2cpt);
>  
>  	for (i = 0; cptab->ctb_parts && i < cptab->ctb_nparts; i++) {
>  		struct cfs_cpu_partition *part = &cptab->ctb_parts[i];
> @@ -515,6 +527,15 @@ struct cfs_cpt_table *
>  }
>  EXPORT_SYMBOL(cfs_cpt_of_cpu);
>  
> +int cfs_cpt_of_node(struct cfs_cpt_table *cptab, int node)
> +{
> +	if (node < 0 || node > nr_node_ids)
> +		return CFS_CPT_ANY;
> +
> +	return cptab->ctb_node2cpt[node];
> +}

So if node == nr_node_ids, we access beyond the end of the ctb_node2cpt array.
Oops.
I've fixed this before applying.

Thanks,
NeilBrown


> +EXPORT_SYMBOL(cfs_cpt_of_node);
> +
>  int
>  cfs_cpt_bind(struct cfs_cpt_table *cptab, int cpt)
>  {
> -- 
> 1.8.3.1
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180625/770edf72/attachment.sig>

From neilb at suse.com  Mon Jun 25 00:48:56 2018
From: neilb at suse.com (NeilBrown)
Date: Mon, 25 Jun 2018 10:48:56 +1000
Subject: [lustre-devel] [PATCH v3 08/26] staging: lustre: libcfs: add
 cpu distance handling
In-Reply-To: <1529875250-11531-9-git-send-email-jsimmons@infradead.org>
References: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
 <1529875250-11531-9-git-send-email-jsimmons@infradead.org>
Message-ID: <874lhradxj.fsf@notabene.neil.brown.name>

On Sun, Jun 24 2018, James Simmons wrote:

> From: Amir Shehata <amir.shehata at intel.com>
>
> Add functionality to calculate the distance between two CPTs.
> Expose those distance in debugfs so people deploying a setup
> can debug what is being created for CPTs.

This patch doesn't expose anything in debugfs - a later patch
does that.
So I've changed the comment to "Prepare to expose those ...."

NeilBrown


>
> Signed-off-by: Amir Shehata <amir.shehata at intel.com>
> WC-bug-id: https://jira.whamcloud.com/browse/LU-7734
> Reviewed-on: http://review.whamcloud.com/18916
> Reviewed-by: Olaf Weber <olaf at sgi.com>
> Reviewed-by: Doug Oucharek <dougso at me.com>
> Signed-off-by: James Simmons <jsimmons at infradead.org>
> ---
>  .../lustre/include/linux/libcfs/libcfs_cpu.h       | 31 +++++++++++
>  drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c    | 61 ++++++++++++++++++++++
>  2 files changed, 92 insertions(+)
>
> diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h
> index ff3ecf5..a015ac1 100644
> --- a/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h
> +++ b/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h
> @@ -86,6 +86,8 @@ struct cfs_cpu_partition {
>  	cpumask_var_t			cpt_cpumask;
>  	/* nodes mask for this partition */
>  	nodemask_t			*cpt_nodemask;
> +	/* NUMA distance between CPTs */
> +	unsigned int			*cpt_distance;
>  	/* spread rotor for NUMA allocator */
>  	unsigned int			cpt_spread_rotor;
>  };
> @@ -95,6 +97,8 @@ struct cfs_cpu_partition {
>  struct cfs_cpt_table {
>  	/* spread rotor for NUMA allocator */
>  	unsigned int			ctb_spread_rotor;
> +	/* maximum NUMA distance between all nodes in table */
> +	unsigned int			ctb_distance;
>  	/* # of CPU partitions */
>  	unsigned int			ctb_nparts;
>  	/* partitions tables */
> @@ -120,6 +124,10 @@ struct cfs_cpt_table {
>   */
>  int cfs_cpt_table_print(struct cfs_cpt_table *cptab, char *buf, int len);
>  /**
> + * print distance information of cpt-table
> + */
> +int cfs_cpt_distance_print(struct cfs_cpt_table *cptab, char *buf, int len);
> +/**
>   * return total number of CPU partitions in \a cptab
>   */
>  int
> @@ -149,6 +157,10 @@ struct cfs_cpt_table {
>   */
>  int cfs_cpt_of_node(struct cfs_cpt_table *cptab, int node);
>  /**
> + * NUMA distance between \a cpt1 and \a cpt2 in \a cptab
> + */
> +unsigned int cfs_cpt_distance(struct cfs_cpt_table *cptab, int cpt1, int cpt2);
> +/**
>   * bind current thread on a CPU-partition \a cpt of \a cptab
>   */
>  int cfs_cpt_bind(struct cfs_cpt_table *cptab, int cpt);
> @@ -206,6 +218,19 @@ void cfs_cpt_unset_nodemask(struct cfs_cpt_table *cptab,
>  struct cfs_cpt_table;
>  #define cfs_cpt_tab ((struct cfs_cpt_table *)NULL)
>  
> +static inline int cfs_cpt_distance_print(struct cfs_cpt_table *cptab,
> +					 char *buf, int len)
> +{
> +	int rc;
> +
> +	rc = snprintf(buf, len, "0\t: 0:1\n");
> +	len -= rc;
> +	if (len <= 0)
> +		return -EFBIG;
> +
> +	return rc;
> +}
> +
>  static inline cpumask_var_t *
>  cfs_cpt_cpumask(struct cfs_cpt_table *cptab, int cpt)
>  {
> @@ -241,6 +266,12 @@ void cfs_cpt_unset_nodemask(struct cfs_cpt_table *cptab,
>  	return NULL;
>  }
>  
> +static inline unsigned int cfs_cpt_distance(struct cfs_cpt_table *cptab,
> +					    int cpt1, int cpt2)
> +{
> +	return 1;
> +}
> +
>  static inline int
>  cfs_cpt_set_cpu(struct cfs_cpt_table *cptab, int cpt, int cpu)
>  {
> diff --git a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
> index 8c5cf7b..b315fb2 100644
> --- a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
> +++ b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
> @@ -128,6 +128,15 @@ struct cfs_cpt_table *
>  					     GFP_NOFS);
>  		if (!part->cpt_nodemask)
>  			goto failed_setting_ctb_parts;
> +
> +		part->cpt_distance = kvmalloc_array(cptab->ctb_nparts,
> +						    sizeof(part->cpt_distance[0]),
> +						    GFP_KERNEL);
> +		if (!part->cpt_distance)
> +			goto failed_setting_ctb_parts;
> +
> +		memset(part->cpt_distance, -1,
> +		       cptab->ctb_nparts * sizeof(part->cpt_distance[0]));
>  	}
>  
>  	return cptab;
> @@ -138,6 +147,7 @@ struct cfs_cpt_table *
>  
>  		kfree(part->cpt_nodemask);
>  		free_cpumask_var(part->cpt_cpumask);
> +		kvfree(part->cpt_distance);
>  	}
>  
>  	kvfree(cptab->ctb_parts);
> @@ -168,6 +178,7 @@ struct cfs_cpt_table *
>  
>  		kfree(part->cpt_nodemask);
>  		free_cpumask_var(part->cpt_cpumask);
> +		kvfree(part->cpt_distance);
>  	}
>  
>  	kvfree(cptab->ctb_parts);
> @@ -222,6 +233,44 @@ struct cfs_cpt_table *
>  }
>  EXPORT_SYMBOL(cfs_cpt_table_print);
>  
> +int cfs_cpt_distance_print(struct cfs_cpt_table *cptab, char *buf, int len)
> +{
> +	char *tmp = buf;
> +	int rc;
> +	int i;
> +	int j;
> +
> +	for (i = 0; i < cptab->ctb_nparts; i++) {
> +		if (len <= 0)
> +			goto err;
> +
> +		rc = snprintf(tmp, len, "%d\t:", i);
> +		len -= rc;
> +
> +		if (len <= 0)
> +			goto err;
> +
> +		tmp += rc;
> +		for (j = 0; j < cptab->ctb_nparts; j++) {
> +			rc = snprintf(tmp, len, " %d:%d", j,
> +				      cptab->ctb_parts[i].cpt_distance[j]);
> +			len -= rc;
> +			if (len <= 0)
> +				goto err;
> +			tmp += rc;
> +		}
> +
> +		*tmp = '\n';
> +		tmp++;
> +		len--;
> +	}
> +
> +	return tmp - buf;
> +err:
> +	return -E2BIG;
> +}
> +EXPORT_SYMBOL(cfs_cpt_distance_print);
> +
>  int
>  cfs_cpt_number(struct cfs_cpt_table *cptab)
>  {
> @@ -273,6 +322,18 @@ struct cfs_cpt_table *
>  }
>  EXPORT_SYMBOL(cfs_cpt_nodemask);
>  
> +unsigned int cfs_cpt_distance(struct cfs_cpt_table *cptab, int cpt1, int cpt2)
> +{
> +	LASSERT(cpt1 == CFS_CPT_ANY || (cpt1 >= 0 && cpt1 < cptab->ctb_nparts));
> +	LASSERT(cpt2 == CFS_CPT_ANY || (cpt2 >= 0 && cpt2 < cptab->ctb_nparts));
> +
> +	if (cpt1 == CFS_CPT_ANY || cpt2 == CFS_CPT_ANY)
> +		return cptab->ctb_distance;
> +
> +	return cptab->ctb_parts[cpt1].cpt_distance[cpt2];
> +}
> +EXPORT_SYMBOL(cfs_cpt_distance);
> +
>  int
>  cfs_cpt_set_cpu(struct cfs_cpt_table *cptab, int cpt, int cpu)
>  {
> -- 
> 1.8.3.1
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180625/574935ba/attachment.sig>

From neilb at suse.com  Mon Jun 25 00:57:33 2018
From: neilb at suse.com (NeilBrown)
Date: Mon, 25 Jun 2018 10:57:33 +1000
Subject: [lustre-devel] [PATCH v3 13/26] staging: lustre: libcfs: use
 int type for CPT identification.
In-Reply-To: <1529875250-11531-14-git-send-email-jsimmons@infradead.org>
References: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
 <1529875250-11531-14-git-send-email-jsimmons@infradead.org>
Message-ID: <871scvadj6.fsf@notabene.neil.brown.name>

On Sun, Jun 24 2018, James Simmons wrote:

> From: Dmitry Eremin <dmitry.eremin at intel.com>
>
> Use int type for CPT identification to match the linux kernel
> CPU identification.

Can someone site evidence for "int" being the dominant choice for CPU
identification in the kernel?
I looked in cpumask.h and found plenty of "unsigned int".
I also found

Commit: 9b130ad5bb82 ("treewide: make "nr_cpu_ids" unsigned")

which makes nr_cpu_ids unsigned.

So I'm dropping this patch for now as the justification is not
convincing.

If there is a real case to be made, please resubmit.

Thanks,
NeilBrown



>
> Signed-off-by: Dmitry Eremin <dmitry.eremin at intel.com>
> WC-bug-id: https://jira.whamcloud.com/browse/LU-8703
> Reviewed-on: https://review.whamcloud.com/23304
> Reviewed-by: James Simmons <uja.ornl at yahoo.com>
> Reviewed-by: Doug Oucharek <dougso at me.com>
> Reviewed-by: Oleg Drokin <green at whamcloud.com>
> Signed-off-by: James Simmons <jsimmons at infradead.org>
> ---
>  drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h |  8 ++++----
>  drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c          | 14 +++++++-------
>  2 files changed, 11 insertions(+), 11 deletions(-)
>
> diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h
> index 9dbb0b1..2bb2140 100644
> --- a/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h
> +++ b/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h
> @@ -89,18 +89,18 @@ struct cfs_cpu_partition {
>  	/* NUMA distance between CPTs */
>  	unsigned int			*cpt_distance;
>  	/* spread rotor for NUMA allocator */
> -	unsigned int			cpt_spread_rotor;
> +	int				cpt_spread_rotor;
>  };
>  
>  
>  /** descriptor for CPU partitions */
>  struct cfs_cpt_table {
>  	/* spread rotor for NUMA allocator */
> -	unsigned int			ctb_spread_rotor;
> +	int				ctb_spread_rotor;
>  	/* maximum NUMA distance between all nodes in table */
>  	unsigned int			ctb_distance;
>  	/* # of CPU partitions */
> -	unsigned int			ctb_nparts;
> +	int				ctb_nparts;
>  	/* partitions tables */
>  	struct cfs_cpu_partition	*ctb_parts;
>  	/* shadow HW CPU to CPU partition ID */
> @@ -355,7 +355,7 @@ static inline void cfs_cpu_fini(void)
>  /**
>   * create a cfs_cpt_table with \a ncpt number of partitions
>   */
> -struct cfs_cpt_table *cfs_cpt_table_alloc(unsigned int ncpt);
> +struct cfs_cpt_table *cfs_cpt_table_alloc(int ncpt);
>  
>  /*
>   * allocate per-cpu-partition data, returned value is an array of pointers,
> diff --git a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
> index aaab7cb..8f7de59 100644
> --- a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
> +++ b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
> @@ -73,7 +73,7 @@
>  module_param(cpu_pattern, charp, 0444);
>  MODULE_PARM_DESC(cpu_pattern, "CPU partitions pattern");
>  
> -struct cfs_cpt_table *cfs_cpt_table_alloc(unsigned int ncpt)
> +struct cfs_cpt_table *cfs_cpt_table_alloc(int ncpt)
>  {
>  	struct cfs_cpt_table *cptab;
>  	int i;
> @@ -788,13 +788,13 @@ static int cfs_cpt_choose_ncpus(struct cfs_cpt_table *cptab, int cpt,
>  	return rc;
>  }
>  
> -#define CPT_WEIGHT_MIN  4u
> +#define CPT_WEIGHT_MIN 4
>  
> -static unsigned int cfs_cpt_num_estimate(void)
> +static int cfs_cpt_num_estimate(void)
>  {
> -	unsigned int nnode = num_online_nodes();
> -	unsigned int ncpu = num_online_cpus();
> -	unsigned int ncpt;
> +	int nnode = num_online_nodes();
> +	int ncpu = num_online_cpus();
> +	int ncpt;
>  
>  	if (ncpu <= CPT_WEIGHT_MIN) {
>  		ncpt = 1;
> @@ -824,7 +824,7 @@ static unsigned int cfs_cpt_num_estimate(void)
>  	/* config many CPU partitions on 32-bit system could consume
>  	 * too much memory
>  	 */
> -	ncpt = min(2U, ncpt);
> +	ncpt = min(2, ncpt);
>  #endif
>  	while (ncpu % ncpt)
>  		ncpt--; /* worst case is 1 */
> -- 
> 1.8.3.1
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180625/de069a31/attachment.sig>

From neilb at suse.com  Mon Jun 25 01:09:53 2018
From: neilb at suse.com (NeilBrown)
Date: Mon, 25 Jun 2018 11:09:53 +1000
Subject: [lustre-devel] [PATCH v3 19/26] staging: lustre: libcfs: report
 NUMA node instead of just node
In-Reply-To: <1529875250-11531-20-git-send-email-jsimmons@infradead.org>
References: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
 <1529875250-11531-20-git-send-email-jsimmons@infradead.org>
Message-ID: <87y3f38ye6.fsf@notabene.neil.brown.name>

On Sun, Jun 24 2018, James Simmons wrote:

> From: Dmitry Eremin <dmitry.eremin at intel.com>
>
> Reporting "HW nodes" is too generic. It really is reporting
> "HW NUMA nodes". Update the debug message.

I'm not happy with this patch description.....

>
> Signed-off-by: Dmitry Eremin <dmitry.eremin at intel.com>
> WC-bug-id: https://jira.whamcloud.com/browse/LU-8703
> Reviewed-on: https://review.whamcloud.com/23306
> Reviewed-by: James Simmons <uja.ornl at yahoo.com>
> Reviewed-by: Andreas Dilger <adilger at whamcloud.com>
> Reviewed-by: Patrick Farrell <paf at cray.com>
> Reviewed-by: Olaf Weber <olaf.weber at hpe.com>
> Reviewed-by: Oleg Drokin <green at whamcloud.com>
> Signed-off-by: James Simmons <jsimmons at infradead.org>
> ---
>  drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h | 2 ++
>  drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c          | 2 +-
>  drivers/staging/lustre/lnet/lnet/lib-msg.c               | 2 ++
>  3 files changed, 5 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h
> index 2bb2140..29c5071 100644
> --- a/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h
> +++ b/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h
> @@ -90,6 +90,8 @@ struct cfs_cpu_partition {
>  	unsigned int			*cpt_distance;
>  	/* spread rotor for NUMA allocator */
>  	int				cpt_spread_rotor;
> +	/* NUMA node if cpt_nodemask is empty */
> +	int				cpt_node;
>  };

It doesn't give any reason why this (unused) field was added.
So I've removed it.


>  
>  
> diff --git a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
> index 18925c7..86afa31 100644
> --- a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
> +++ b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
> @@ -1142,7 +1142,7 @@ int cfs_cpu_init(void)
>  
>  	put_online_cpus();
>  
> -	LCONSOLE(0, "HW nodes: %d, HW CPU cores: %d, npartitions: %d\n",
> +	LCONSOLE(0, "HW NUMA nodes: %d, HW CPU cores: %d, npartitions: %d\n",
>  		 num_online_nodes(), num_online_cpus(),
>  		 cfs_cpt_number(cfs_cpt_tab));
>  	return 0;

It does explain this hunk, which is fine.


> diff --git a/drivers/staging/lustre/lnet/lnet/lib-msg.c b/drivers/staging/lustre/lnet/lnet/lib-msg.c
> index 0091273..27bdefa 100644
> --- a/drivers/staging/lustre/lnet/lnet/lib-msg.c
> +++ b/drivers/staging/lustre/lnet/lnet/lib-msg.c
> @@ -568,6 +568,8 @@
>  
>  	/* number of CPUs */
>  	container->msc_nfinalizers = cfs_cpt_weight(lnet_cpt_table(), cpt);
> +	if (container->msc_nfinalizers == 0)
> +		container->msc_nfinalizers = 1;

It doesn't justify this at all.

I guess this was meant to be in the previous patch, so I've moved it.

Thanks,
NeilBrown


>  
>  	container->msc_finalizers = kvzalloc_cpt(container->msc_nfinalizers *
>  						 sizeof(*container->msc_finalizers),
> -- 
> 1.8.3.1
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180625/f1f67ad2/attachment.sig>

From neilb at suse.com  Mon Jun 25 01:11:17 2018
From: neilb at suse.com (NeilBrown)
Date: Mon, 25 Jun 2018 11:11:17 +1000
Subject: [lustre-devel] [PATCH v3 19/26] staging: lustre: libcfs: report
 NUMA node instead of just node
In-Reply-To: <87y3f38ye6.fsf@notabene.neil.brown.name>
References: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
 <1529875250-11531-20-git-send-email-jsimmons@infradead.org>
 <87y3f38ye6.fsf@notabene.neil.brown.name>
Message-ID: <87vaa78ybu.fsf@notabene.neil.brown.name>

On Mon, Jun 25 2018, NeilBrown wrote:

> On Sun, Jun 24 2018, James Simmons wrote:
>
>> From: Dmitry Eremin <dmitry.eremin at intel.com>
>>
>> Reporting "HW nodes" is too generic. It really is reporting
>> "HW NUMA nodes". Update the debug message.
>
> I'm not happy with this patch description.....
>
>>
>> Signed-off-by: Dmitry Eremin <dmitry.eremin at intel.com>
>> WC-bug-id: https://jira.whamcloud.com/browse/LU-8703
>> Reviewed-on: https://review.whamcloud.com/23306
>> Reviewed-by: James Simmons <uja.ornl at yahoo.com>
>> Reviewed-by: Andreas Dilger <adilger at whamcloud.com>
>> Reviewed-by: Patrick Farrell <paf at cray.com>
>> Reviewed-by: Olaf Weber <olaf.weber at hpe.com>
>> Reviewed-by: Oleg Drokin <green at whamcloud.com>
>> Signed-off-by: James Simmons <jsimmons at infradead.org>
>> ---
>>  drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h | 2 ++
>>  drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c          | 2 +-
>>  drivers/staging/lustre/lnet/lnet/lib-msg.c               | 2 ++
>>  3 files changed, 5 insertions(+), 1 deletion(-)
>>
>> diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h
>> index 2bb2140..29c5071 100644
>> --- a/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h
>> +++ b/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h
>> @@ -90,6 +90,8 @@ struct cfs_cpu_partition {
>>  	unsigned int			*cpt_distance;
>>  	/* spread rotor for NUMA allocator */
>>  	int				cpt_spread_rotor;
>> +	/* NUMA node if cpt_nodemask is empty */
>> +	int				cpt_node;
>>  };
>
> It doesn't give any reason why this (unused) field was added.
> So I've removed it.

Ahhhh. this was meant to be in the previous patch too.
I've moved it.

Thanks,
NeilBrown


>
>
>>  
>>  
>> diff --git a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
>> index 18925c7..86afa31 100644
>> --- a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
>> +++ b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
>> @@ -1142,7 +1142,7 @@ int cfs_cpu_init(void)
>>  
>>  	put_online_cpus();
>>  
>> -	LCONSOLE(0, "HW nodes: %d, HW CPU cores: %d, npartitions: %d\n",
>> +	LCONSOLE(0, "HW NUMA nodes: %d, HW CPU cores: %d, npartitions: %d\n",
>>  		 num_online_nodes(), num_online_cpus(),
>>  		 cfs_cpt_number(cfs_cpt_tab));
>>  	return 0;
>
> It does explain this hunk, which is fine.
>
>
>> diff --git a/drivers/staging/lustre/lnet/lnet/lib-msg.c b/drivers/staging/lustre/lnet/lnet/lib-msg.c
>> index 0091273..27bdefa 100644
>> --- a/drivers/staging/lustre/lnet/lnet/lib-msg.c
>> +++ b/drivers/staging/lustre/lnet/lnet/lib-msg.c
>> @@ -568,6 +568,8 @@
>>  
>>  	/* number of CPUs */
>>  	container->msc_nfinalizers = cfs_cpt_weight(lnet_cpt_table(), cpt);
>> +	if (container->msc_nfinalizers == 0)
>> +		container->msc_nfinalizers = 1;
>
> It doesn't justify this at all.
>
> I guess this was meant to be in the previous patch, so I've moved it.
>
> Thanks,
> NeilBrown
>
>
>>  
>>  	container->msc_finalizers = kvzalloc_cpt(container->msc_nfinalizers *
>>  						 sizeof(*container->msc_finalizers),
>> -- 
>> 1.8.3.1
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180625/267e618b/attachment.sig>

From neilb at suse.com  Mon Jun 25 01:27:08 2018
From: neilb at suse.com (NeilBrown)
Date: Mon, 25 Jun 2018 11:27:08 +1000
Subject: [lustre-devel] [PATCH v3 24/26] staging: lustre: libcfs:
 restore debugfs table reporting for UMP
In-Reply-To: <1529875250-11531-25-git-send-email-jsimmons@infradead.org>
References: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
 <1529875250-11531-25-git-send-email-jsimmons@infradead.org>
Message-ID: <87r2kv8xlf.fsf@notabene.neil.brown.name>

On Sun, Jun 24 2018, James Simmons wrote:

> With the cleanup of the libcfs SMP handling the function
> cfs_cpt_table_print() was turned into an empty funciton.
> This function is called by a debugfs reporting function for
> debugging which now means for UMP machines it reports nothing
> which breaks previous behavior exposed to users. Restore the
> original behavior.

Which cleanup was this?
cfs_cpt_table_print seems to have been empty on UMP since at least

commit 3867ea5a4bc4d428f8d93557fb0fbc2cac2f2cdf
Author: Peng Tao <bergwolf at gmail.com>
Date:   Mon Jul 15 22:27:10 2013 +0800

    staging/lustre: fix build error when !CONFIG_SMP
    
    Three functions cfs_cpu_ht_nsiblings, cfs_cpt_cpumask and
    cfs_cpt_table_print are missing if !CONFIG_SMP.

5 years ago.

>
> WC-bug-id: https://jira.whamcloud.com/browse/LU-9856

This link seems completely irrelevant.

However, the patch looks sensible enough, so I've applied it.

Thanks,
NeilBrown


> Signed-off-by: James Simmons <jsimmons at infradead.org>
> ---
>  .../staging/lustre/include/linux/libcfs/libcfs_cpu.h  | 19 +++++++++++++------
>  1 file changed, 13 insertions(+), 6 deletions(-)
>
> diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h
> index 29c5071..32776d2 100644
> --- a/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h
> +++ b/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h
> @@ -218,6 +218,19 @@ void cfs_cpt_unset_nodemask(struct cfs_cpt_table *cptab,
>  struct cfs_cpt_table;
>  #define cfs_cpt_tab ((struct cfs_cpt_table *)NULL)
>  
> +static inline int cfs_cpt_table_print(struct cfs_cpt_table *cptab,
> +				      char *buf, int len)
> +{
> +	int rc;
> +
> +	rc = snprintf(buf, len, "0\t: 0\n");
> +	len -= rc;
> +	if (len <= 0)
> +		return -EFBIG;
> +
> +	return rc;
> +}
> +
>  static inline int cfs_cpt_distance_print(struct cfs_cpt_table *cptab,
>  					 char *buf, int len)
>  {
> @@ -237,12 +250,6 @@ static inline cpumask_var_t *cfs_cpt_cpumask(struct cfs_cpt_table *cptab,
>  	return NULL;
>  }
>  
> -static inline int cfs_cpt_table_print(struct cfs_cpt_table *cptab, char *buf,
> -				      int len)
> -{
> -	return 0;
> -}
> -
>  static inline int cfs_cpt_number(struct cfs_cpt_table *cptab)
>  {
>  	return 1;
> -- 
> 1.8.3.1
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180625/4454b313/attachment.sig>

From neilb at suse.com  Mon Jun 25 01:32:38 2018
From: neilb at suse.com (NeilBrown)
Date: Mon, 25 Jun 2018 11:32:38 +1000
Subject: [lustre-devel] [PATCH v3 25/26] staging: lustre: libcfs: make
 cfs_cpt_tab a static structure
In-Reply-To: <1529875250-11531-26-git-send-email-jsimmons@infradead.org>
References: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
 <1529875250-11531-26-git-send-email-jsimmons@infradead.org>
Message-ID: <87o9fz8xc9.fsf@notabene.neil.brown.name>

On Sun, Jun 24 2018, James Simmons wrote:

> Only one cfs_cpt_tab exist and its created only at libcfs modules
> loading and removal. Instead of dynamically allocating it lets
> statically allocate it. This will help to reenable UMP support.
>
> WC-bug-id: https://jira.whamcloud.com/browse/LU-9856
> Signed-off-by: James Simmons <jsimmons at infradead.org>

While this patch is quite possibly a good idea, I'm not applying it or
the following one until you explain what is currently broken with "UMP"
support (I assume you mean UP - uni-processor).

Again, the WC-bug-id you linked gives no useful hint.

Thanks,
NeilBrown


> ---
>  .../lustre/include/linux/libcfs/libcfs_cpu.h       |   4 +-
>  drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c    | 111 ++++++++++-----------
>  drivers/staging/lustre/lnet/libcfs/module.c        |  10 +-
>  drivers/staging/lustre/lnet/lnet/api-ni.c          |   4 +-
>  drivers/staging/lustre/lnet/selftest/framework.c   |   2 +-
>  drivers/staging/lustre/lustre/ptlrpc/client.c      |   4 +-
>  drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c     |  10 +-
>  drivers/staging/lustre/lustre/ptlrpc/service.c     |   4 +-
>  8 files changed, 73 insertions(+), 76 deletions(-)
>
> diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h
> index 32776d2..df7e16b 100644
> --- a/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h
> +++ b/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h
> @@ -115,7 +115,7 @@ struct cfs_cpt_table {
>  	nodemask_t			*ctb_nodemask;
>  };
>  
> -extern struct cfs_cpt_table	*cfs_cpt_tab;
> +extern struct cfs_cpt_table	cfs_cpt_tab;
>  
>  /**
>   * return cpumask of CPU partition \a cpt
> @@ -215,8 +215,6 @@ void cfs_cpt_unset_nodemask(struct cfs_cpt_table *cptab,
>  void cfs_cpu_fini(void);
>  
>  #else /* !CONFIG_SMP */
> -struct cfs_cpt_table;
> -#define cfs_cpt_tab ((struct cfs_cpt_table *)NULL)
>  
>  static inline int cfs_cpt_table_print(struct cfs_cpt_table *cptab,
>  				      char *buf, int len)
> diff --git a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
> index 3f4a7c7..9fd324d 100644
> --- a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
> +++ b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
> @@ -41,10 +41,6 @@
>  #include <linux/libcfs/libcfs_string.h>
>  #include <linux/libcfs/libcfs.h>
>  
> -/** Global CPU partition table */
> -struct cfs_cpt_table   *cfs_cpt_tab __read_mostly;
> -EXPORT_SYMBOL(cfs_cpt_tab);
> -
>  /**
>   * modparam for setting number of partitions
>   *
> @@ -73,15 +69,10 @@
>  module_param(cpu_pattern, charp, 0444);
>  MODULE_PARM_DESC(cpu_pattern, "CPU partitions pattern");
>  
> -struct cfs_cpt_table *cfs_cpt_table_alloc(int ncpt)
> +static int cfs_cpt_table_setup(struct cfs_cpt_table *cptab, int ncpt)
>  {
> -	struct cfs_cpt_table *cptab;
>  	int i;
>  
> -	cptab = kzalloc(sizeof(*cptab), GFP_NOFS);
> -	if (!cptab)
> -		return NULL;
> -
>  	cptab->ctb_nparts = ncpt;
>  
>  	if (!zalloc_cpumask_var(&cptab->ctb_cpumask, GFP_NOFS))
> @@ -138,7 +129,7 @@ struct cfs_cpt_table *cfs_cpt_table_alloc(int ncpt)
>  		       cptab->ctb_nparts * sizeof(part->cpt_distance[0]));
>  	}
>  
> -	return cptab;
> +	return 0;
>  
>  failed_setting_ctb_parts:
>  	while (i-- >= 0) {
> @@ -159,8 +150,24 @@ struct cfs_cpt_table *cfs_cpt_table_alloc(int ncpt)
>  failed_alloc_nodemask:
>  	free_cpumask_var(cptab->ctb_cpumask);
>  failed_alloc_cpumask:
> -	kfree(cptab);
> -	return NULL;
> +	return -ENOMEM;
> +}
> +
> +struct cfs_cpt_table *cfs_cpt_table_alloc(int ncpt)
> +{
> +	struct cfs_cpt_table *cptab;
> +	int rc;
> +
> +	cptab = kzalloc(sizeof(*cptab), GFP_NOFS);
> +	if (!cptab)
> +		return NULL;
> +
> +	rc = cfs_cpt_table_setup(cptab, ncpt);
> +	if (rc) {
> +		kfree(cptab);
> +		cptab = NULL;
> +	}
> +	return cptab;
>  }
>  EXPORT_SYMBOL(cfs_cpt_table_alloc);
>  
> @@ -183,8 +190,6 @@ void cfs_cpt_table_free(struct cfs_cpt_table *cptab)
>  
>  	kfree(cptab->ctb_nodemask);
>  	free_cpumask_var(cptab->ctb_cpumask);
> -
> -	kfree(cptab);
>  }
>  EXPORT_SYMBOL(cfs_cpt_table_free);
>  
> @@ -822,9 +827,8 @@ static int cfs_cpt_num_estimate(void)
>  	return ncpt;
>  }
>  
> -static struct cfs_cpt_table *cfs_cpt_table_create(int ncpt)
> +static int cfs_cpt_table_create(int ncpt)
>  {
> -	struct cfs_cpt_table *cptab = NULL;
>  	cpumask_var_t node_mask;
>  	int cpt = 0;
>  	int node;
> @@ -841,10 +845,9 @@ static struct cfs_cpt_table *cfs_cpt_table_create(int ncpt)
>  		      ncpt, num);
>  	}
>  
> -	cptab = cfs_cpt_table_alloc(ncpt);
> -	if (!cptab) {
> -		CERROR("Failed to allocate CPU map(%d)\n", ncpt);
> -		rc = -ENOMEM;
> +	rc = cfs_cpt_table_setup(&cfs_cpt_tab, ncpt);
> +	if (rc) {
> +		CERROR("Failed to setup CPU map(%d)\n", ncpt);
>  		goto failed;
>  	}
>  
> @@ -860,10 +863,13 @@ static struct cfs_cpt_table *cfs_cpt_table_create(int ncpt)
>  		cpumask_copy(node_mask, cpumask_of_node(node));
>  
>  		while (cpt < ncpt && !cpumask_empty(node_mask)) {
> -			struct cfs_cpu_partition *part = &cptab->ctb_parts[cpt];
> -			int ncpu = cpumask_weight(part->cpt_cpumask);
> +			struct cfs_cpu_partition *part;
> +			int ncpu;
> +
> +			part = &cfs_cpt_tab.ctb_parts[cpt];
> +			ncpu = cpumask_weight(part->cpt_cpumask);
>  
> -			rc = cfs_cpt_choose_ncpus(cptab, cpt, node_mask,
> +			rc = cfs_cpt_choose_ncpus(&cfs_cpt_tab, cpt, node_mask,
>  						  num - ncpu);
>  			if (rc < 0) {
>  				rc = -EINVAL;
> @@ -880,7 +886,7 @@ static struct cfs_cpt_table *cfs_cpt_table_create(int ncpt)
>  
>  	free_cpumask_var(node_mask);
>  
> -	return cptab;
> +	return 0;
>  
>  failed_mask:
>  	free_cpumask_var(node_mask);
> @@ -888,15 +894,13 @@ static struct cfs_cpt_table *cfs_cpt_table_create(int ncpt)
>  	CERROR("Failed (rc = %d) to setup CPU partition table with %d partitions, online HW NUMA nodes: %d, HW CPU cores: %d.\n",
>  	       rc, ncpt, num_online_nodes(), num_online_cpus());
>  
> -	if (cptab)
> -		cfs_cpt_table_free(cptab);
> +	cfs_cpt_table_free(&cfs_cpt_tab);
>  
> -	return ERR_PTR(rc);
> +	return rc;
>  }
>  
> -static struct cfs_cpt_table *cfs_cpt_table_create_pattern(const char *pattern)
> +static int cfs_cpt_table_create_pattern(const char *pattern)
>  {
> -	struct cfs_cpt_table *cptab;
>  	char *pattern_dup;
>  	char *bracket;
>  	char *str;
> @@ -911,7 +915,7 @@ static struct cfs_cpt_table *cfs_cpt_table_create_pattern(const char *pattern)
>  	pattern_dup = kstrdup(pattern, GFP_KERNEL);
>  	if (!pattern_dup) {
>  		CERROR("Failed to duplicate pattern '%s'\n", pattern);
> -		return ERR_PTR(-ENOMEM);
> +		return -ENOMEM;
>  	}
>  
>  	str = strim(pattern_dup);
> @@ -948,10 +952,9 @@ static struct cfs_cpt_table *cfs_cpt_table_create_pattern(const char *pattern)
>  		goto err_free_str;
>  	}
>  
> -	cptab = cfs_cpt_table_alloc(ncpt);
> -	if (!cptab) {
> -		CERROR("Failed to allocate CPU partition table\n");
> -		rc = -ENOMEM;
> +	rc = cfs_cpt_table_setup(&cfs_cpt_tab, ncpt);
> +	if (rc) {
> +		CERROR("Failed to setup CPU partition table\n");
>  		goto err_free_str;
>  	}
>  
> @@ -960,14 +963,14 @@ static struct cfs_cpt_table *cfs_cpt_table_create_pattern(const char *pattern)
>  			if (cpumask_empty(cpumask_of_node(i)))
>  				continue;
>  
> -			rc = cfs_cpt_set_node(cptab, cpt++, i);
> +			rc = cfs_cpt_set_node(&cfs_cpt_tab, cpt++, i);
>  			if (!rc) {
>  				rc = -EINVAL;
>  				goto err_free_table;
>  			}
>  		}
>  		kfree(pattern_dup);
> -		return cptab;
> +		return 0;
>  	}
>  
>  	high = node ? nr_node_ids - 1 : nr_cpu_ids - 1;
> @@ -1006,7 +1009,7 @@ static struct cfs_cpt_table *cfs_cpt_table_create_pattern(const char *pattern)
>  			goto err_free_table;
>  		}
>  
> -		if (cfs_cpt_weight(cptab, cpt)) {
> +		if (cfs_cpt_weight(&cfs_cpt_tab, cpt)) {
>  			CERROR("Partition %d has already been set.\n", cpt);
>  			rc = -EPERM;
>  			goto err_free_table;
> @@ -1040,8 +1043,8 @@ static struct cfs_cpt_table *cfs_cpt_table_create_pattern(const char *pattern)
>  				if ((i - range->re_lo) % range->re_stride)
>  					continue;
>  
> -				rc = node ? cfs_cpt_set_node(cptab, cpt, i) :
> -					    cfs_cpt_set_cpu(cptab, cpt, i);
> +				rc = node ? cfs_cpt_set_node(&cfs_cpt_tab, cpt, i) :
> +					    cfs_cpt_set_cpu(&cfs_cpt_tab, cpt, i);
>  				if (!rc) {
>  					cfs_expr_list_free(el);
>  					rc = -EINVAL;
> @@ -1052,7 +1055,7 @@ static struct cfs_cpt_table *cfs_cpt_table_create_pattern(const char *pattern)
>  
>  		cfs_expr_list_free(el);
>  
> -		if (!cfs_cpt_online(cptab, cpt)) {
> +		if (!cfs_cpt_online(&cfs_cpt_tab, cpt)) {
>  			CERROR("No online CPU is found on partition %d\n", cpt);
>  			rc = -ENODEV;
>  			goto err_free_table;
> @@ -1062,13 +1065,13 @@ static struct cfs_cpt_table *cfs_cpt_table_create_pattern(const char *pattern)
>  	}
>  
>  	kfree(pattern_dup);
> -	return cptab;
> +	return 0;
>  
>  err_free_table:
> -	cfs_cpt_table_free(cptab);
> +	cfs_cpt_table_free(&cfs_cpt_tab);
>  err_free_str:
>  	kfree(pattern_dup);
> -	return ERR_PTR(rc);
> +	return rc;
>  }
>  
>  #ifdef CONFIG_HOTPLUG_CPU
> @@ -1095,8 +1098,7 @@ static int cfs_cpu_dead(unsigned int cpu)
>  
>  void cfs_cpu_fini(void)
>  {
> -	if (!IS_ERR_OR_NULL(cfs_cpt_tab))
> -		cfs_cpt_table_free(cfs_cpt_tab);
> +	cfs_cpt_table_free(&cfs_cpt_tab);
>  
>  #ifdef CONFIG_HOTPLUG_CPU
>  	if (lustre_cpu_online > 0)
> @@ -1109,8 +1111,6 @@ int cfs_cpu_init(void)
>  {
>  	int ret;
>  
> -	LASSERT(!cfs_cpt_tab);
> -
>  #ifdef CONFIG_HOTPLUG_CPU
>  	ret = cpuhp_setup_state_nocalls(CPUHP_LUSTRE_CFS_DEAD,
>  					"staging/lustre/cfe:dead", NULL,
> @@ -1128,20 +1128,18 @@ int cfs_cpu_init(void)
>  #endif
>  	get_online_cpus();
>  	if (*cpu_pattern) {
> -		cfs_cpt_tab = cfs_cpt_table_create_pattern(cpu_pattern);
> -		if (IS_ERR(cfs_cpt_tab)) {
> +		ret = cfs_cpt_table_create_pattern(cpu_pattern);
> +		if (ret) {
>  			CERROR("Failed to create cptab from pattern '%s'\n",
>  			       cpu_pattern);
> -			ret = PTR_ERR(cfs_cpt_tab);
>  			goto failed_alloc_table;
>  		}
>  
>  	} else {
> -		cfs_cpt_tab = cfs_cpt_table_create(cpu_npartitions);
> -		if (IS_ERR(cfs_cpt_tab)) {
> +		ret = cfs_cpt_table_create(cpu_npartitions);
> +		if (ret) {
>  			CERROR("Failed to create cptab with npartitions %d\n",
>  			       cpu_npartitions);
> -			ret = PTR_ERR(cfs_cpt_tab);
>  			goto failed_alloc_table;
>  		}
>  	}
> @@ -1150,14 +1148,13 @@ int cfs_cpu_init(void)
>  
>  	LCONSOLE(0, "HW NUMA nodes: %d, HW CPU cores: %d, npartitions: %d\n",
>  		 num_online_nodes(), num_online_cpus(),
> -		 cfs_cpt_number(cfs_cpt_tab));
> +		 cfs_cpt_number(&cfs_cpt_tab));
>  	return 0;
>  
>  failed_alloc_table:
>  	put_online_cpus();
>  
> -	if (!IS_ERR_OR_NULL(cfs_cpt_tab))
> -		cfs_cpt_table_free(cfs_cpt_tab);
> +	cfs_cpt_table_free(&cfs_cpt_tab);
>  
>  	ret = -EINVAL;
>  #ifdef CONFIG_HOTPLUG_CPU
> diff --git a/drivers/staging/lustre/lnet/libcfs/module.c b/drivers/staging/lustre/lnet/libcfs/module.c
> index 2281f08..35c3959 100644
> --- a/drivers/staging/lustre/lnet/libcfs/module.c
> +++ b/drivers/staging/lustre/lnet/libcfs/module.c
> @@ -66,6 +66,10 @@ struct lnet_debugfs_symlink_def {
>  
>  static struct dentry *lnet_debugfs_root;
>  
> +/** Global CPU partition table */
> +struct cfs_cpt_table cfs_cpt_tab __read_mostly;
> +EXPORT_SYMBOL(cfs_cpt_tab);
> +
>  BLOCKING_NOTIFIER_HEAD(libcfs_ioctl_list);
>  EXPORT_SYMBOL(libcfs_ioctl_list);
>  
> @@ -402,7 +406,7 @@ static int proc_cpt_table(struct ctl_table *table, int write,
>  		if (!buf)
>  			return -ENOMEM;
>  
> -		rc = cfs_cpt_table_print(cfs_cpt_tab, buf, len);
> +		rc = cfs_cpt_table_print(&cfs_cpt_tab, buf, len);
>  		if (rc >= 0)
>  			break;
>  
> @@ -437,14 +441,12 @@ static int proc_cpt_distance(struct ctl_table *table, int write,
>  	if (write)
>  		return -EPERM;
>  
> -	LASSERT(cfs_cpt_tab);
> -
>  	while (1) {
>  		buf = kzalloc(len, GFP_KERNEL);
>  		if (!buf)
>  			return -ENOMEM;
>  
> -		rc = cfs_cpt_distance_print(cfs_cpt_tab, buf, len);
> +		rc = cfs_cpt_distance_print(&cfs_cpt_tab, buf, len);
>  		if (rc >= 0)
>  			break;
>  
> diff --git a/drivers/staging/lustre/lnet/lnet/api-ni.c b/drivers/staging/lustre/lnet/lnet/api-ni.c
> index f9ed697..98a4942 100644
> --- a/drivers/staging/lustre/lnet/lnet/api-ni.c
> +++ b/drivers/staging/lustre/lnet/lnet/api-ni.c
> @@ -1414,8 +1414,8 @@ int lnet_lib_init(void)
>  	memset(&the_lnet, 0, sizeof(the_lnet));
>  
>  	/* refer to global cfs_cpt_tab for now */
> -	the_lnet.ln_cpt_table	= cfs_cpt_tab;
> -	the_lnet.ln_cpt_number	= cfs_cpt_number(cfs_cpt_tab);
> +	the_lnet.ln_cpt_table = &cfs_cpt_tab;
> +	the_lnet.ln_cpt_number = cfs_cpt_number(&cfs_cpt_tab);
>  
>  	LASSERT(the_lnet.ln_cpt_number > 0);
>  	if (the_lnet.ln_cpt_number > LNET_CPT_MAX) {
> diff --git a/drivers/staging/lustre/lnet/selftest/framework.c b/drivers/staging/lustre/lnet/selftest/framework.c
> index 741af10..939b7ec 100644
> --- a/drivers/staging/lustre/lnet/selftest/framework.c
> +++ b/drivers/staging/lustre/lnet/selftest/framework.c
> @@ -588,7 +588,7 @@
>  
>  	CDEBUG(D_NET, "Reserved %d buffers for test %s\n",
>  	       nbuf * (srpc_serv_is_framework(svc) ?
> -		       2 : cfs_cpt_number(cfs_cpt_tab)), svc->sv_name);
> +		       2 : cfs_cpt_number(&cfs_cpt_tab)), svc->sv_name);
>  	return 0;
>  }
>  
> diff --git a/drivers/staging/lustre/lustre/ptlrpc/client.c b/drivers/staging/lustre/lustre/ptlrpc/client.c
> index c1b82bf..c569a8b 100644
> --- a/drivers/staging/lustre/lustre/ptlrpc/client.c
> +++ b/drivers/staging/lustre/lustre/ptlrpc/client.c
> @@ -940,9 +940,9 @@ struct ptlrpc_request_set *ptlrpc_prep_set(void)
>  	struct ptlrpc_request_set *set;
>  	int cpt;
>  
> -	cpt = cfs_cpt_current(cfs_cpt_tab, 0);
> +	cpt = cfs_cpt_current(&cfs_cpt_tab, 0);
>  	set = kzalloc_node(sizeof(*set), GFP_NOFS,
> -			   cfs_cpt_spread_node(cfs_cpt_tab, cpt));
> +			   cfs_cpt_spread_node(&cfs_cpt_tab, cpt));
>  	if (!set)
>  		return NULL;
>  	atomic_set(&set->set_refcount, 1);
> diff --git a/drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c b/drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c
> index 5310054..d496521 100644
> --- a/drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c
> +++ b/drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c
> @@ -177,7 +177,7 @@ void ptlrpcd_wake(struct ptlrpc_request *req)
>  	if (req && req->rq_send_state != LUSTRE_IMP_FULL)
>  		return &ptlrpcd_rcv;
>  
> -	cpt = cfs_cpt_current(cfs_cpt_tab, 1);
> +	cpt = cfs_cpt_current(&cfs_cpt_tab, 1);
>  	if (!ptlrpcds_cpt_idx)
>  		idx = cpt;
>  	else
> @@ -389,7 +389,7 @@ static int ptlrpcd(void *arg)
>  	int exit = 0;
>  
>  	unshare_fs_struct();
> -	if (cfs_cpt_bind(cfs_cpt_tab, pc->pc_cpt) != 0)
> +	if (cfs_cpt_bind(&cfs_cpt_tab, pc->pc_cpt) != 0)
>  		CWARN("Failed to bind %s on CPT %d\n", pc->pc_name, pc->pc_cpt);
>  
>  	/*
> @@ -531,7 +531,7 @@ static int ptlrpcd_partners(struct ptlrpcd *pd, int index)
>  
>  	size = sizeof(struct ptlrpcd_ctl *) * pc->pc_npartners;
>  	pc->pc_partners = kzalloc_node(size, GFP_NOFS,
> -				       cfs_cpt_spread_node(cfs_cpt_tab,
> +				       cfs_cpt_spread_node(&cfs_cpt_tab,
>  							   pc->pc_cpt));
>  	if (!pc->pc_partners) {
>  		pc->pc_npartners = 0;
> @@ -677,7 +677,7 @@ static int ptlrpcd_init(void)
>  	/*
>  	 * Determine the CPTs that ptlrpcd threads will run on.
>  	 */
> -	cptable = cfs_cpt_tab;
> +	cptable = &cfs_cpt_tab;
>  	ncpts = cfs_cpt_number(cptable);
>  	if (ptlrpcd_cpts) {
>  		struct cfs_expr_list *el;
> @@ -831,7 +831,7 @@ static int ptlrpcd_init(void)
>  
>  		size = offsetof(struct ptlrpcd, pd_threads[nthreads]);
>  		pd = kzalloc_node(size, GFP_NOFS,
> -				  cfs_cpt_spread_node(cfs_cpt_tab, cpt));
> +				  cfs_cpt_spread_node(&cfs_cpt_tab, cpt));
>  		if (!pd) {
>  			rc = -ENOMEM;
>  			goto out;
> diff --git a/drivers/staging/lustre/lustre/ptlrpc/service.c b/drivers/staging/lustre/lustre/ptlrpc/service.c
> index 8e74a45..853676f 100644
> --- a/drivers/staging/lustre/lustre/ptlrpc/service.c
> +++ b/drivers/staging/lustre/lustre/ptlrpc/service.c
> @@ -565,7 +565,7 @@ struct ptlrpc_service *
>  
>  	cptable = cconf->cc_cptable;
>  	if (!cptable)
> -		cptable = cfs_cpt_tab;
> +		cptable = &cfs_cpt_tab;
>  
>  	if (!conf->psc_thr.tc_cpu_affinity) {
>  		ncpts = 1;
> @@ -2520,7 +2520,7 @@ int ptlrpc_hr_init(void)
>  	int weight;
>  
>  	memset(&ptlrpc_hr, 0, sizeof(ptlrpc_hr));
> -	ptlrpc_hr.hr_cpt_table = cfs_cpt_tab;
> +	ptlrpc_hr.hr_cpt_table = &cfs_cpt_tab;
>  
>  	ptlrpc_hr.hr_partitions = cfs_percpt_alloc(ptlrpc_hr.hr_cpt_table,
>  						   sizeof(*hrp));
> -- 
> 1.8.3.1
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180625/02020e75/attachment.sig>

From neilb at suse.com  Mon Jun 25 01:33:53 2018
From: neilb at suse.com (NeilBrown)
Date: Mon, 25 Jun 2018 11:33:53 +1000
Subject: [lustre-devel] [PATCH v3 00/26] staging: lustre: libcfs: SMP
 rework
In-Reply-To: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
References: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
Message-ID: <87lgb38xa6.fsf@notabene.neil.brown.name>

On Sun, Jun 24 2018, James Simmons wrote:

> Recently lustre support has been expanded to extreme machines with as
> many as a 1000+ cores. On the other end lustre also has been ported
> to platforms like ARM and KNL which have uniquie NUMA and core setup.
> For example some devices exist that have NUMA nodes with no cores.
> With these new platforms the limitations of the Lustre's SMP code
> came to light so a lot of work was needed. This resulted in this
> patch set which has been tested on these platforms.
>
> This is the 3rd version of this patch set with the first two submitted
> to the staging list. This latest patchset is identical to the 2nd one
> expect that the UMP support has been moved to the last patches in this
> collection. The approach to support UMP also has changed with using
> static initialization to greatly simplify the code.

Thanks for these.
Apart from some patches that I've rejected and some that I've modified,
these should appear in my lustre-testing branch in the next 24 hours.
I expect them to migrate to my lustre branch next Monday if no problems
surface.

Thanks,
NeilBrown


>
> Amir Shehata (8):
>   staging: lustre: libcfs: replace MAX_NUMNODES with nr_node_ids
>   staging: lustre: libcfs: remove excess space
>   staging: lustre: libcfs: replace num_possible_cpus() with nr_cpu_ids
>   staging: lustre: libcfs: NUMA support
>   staging: lustre: libcfs: add cpu distance handling
>   staging: lustre: libcfs: use distance in cpu and node handling
>   staging: lustre: libcfs: provide debugfs files for distance handling
>   staging: lustre: libcfs: invert error handling for cfs_cpt_table_print
>
> Dmitry Eremin (14):
>   staging: lustre: libcfs: remove useless CPU partition code
>   staging: lustre: libcfs: rename variable i to cpu
>   staging: lustre: libcfs: fix libcfs_cpu coding style
>   staging: lustre: libcfs: use int type for CPT identification.
>   staging: lustre: libcfs: rename i to node for cfs_cpt_set_nodemask
>   staging: lustre: libcfs: rename i to cpu for cfs_cpt_bind
>   staging: lustre: libcfs: rename cpumask_var_t variables to *_mask
>   staging: lustre: libcfs: update debug messages
>   staging: lustre: libcfs: make tolerant to offline CPUs and empty NUMA nodes
>   staging: lustre: libcfs: report NUMA node instead of just node
>   staging: lustre: libcfs: update debug messages in CPT code
>   staging: lustre: libcfs: rework CPU pattern parsing code
>   staging: lustre: libcfs: change CPT estimate algorithm
>   staging: lustre: ptlrpc: use current CPU instead of hardcoded 0
>
> James Simmons (4):
>   staging: lustre: libcfs: properly handle failure cases in SMP code
>   staging: lustre: libcfs: restore debugfs table reporting for UMP
>   staging: lustre: libcfs: make cfs_cpt_tab a static structure
>   staging: lustre: libcfs: restore UMP support
>
>  .../lustre/include/linux/libcfs/libcfs_cpu.h       |  203 ++--
>  drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c    | 1020 +++++++++++---------
>  drivers/staging/lustre/lnet/libcfs/module.c        |   52 +-
>  drivers/staging/lustre/lnet/lnet/api-ni.c          |    4 +-
>  drivers/staging/lustre/lnet/lnet/lib-msg.c         |    2 +
>  drivers/staging/lustre/lnet/selftest/framework.c   |    2 +-
>  drivers/staging/lustre/lustre/ptlrpc/client.c      |    4 +-
>  drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c     |   10 +-
>  drivers/staging/lustre/lustre/ptlrpc/service.c     |   15 +-
>  9 files changed, 750 insertions(+), 562 deletions(-)
>
> -- 
> 1.8.3.1
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180625/50229983/attachment.sig>

From neilb at suse.com  Mon Jun 25 01:42:44 2018
From: neilb at suse.com (NeilBrown)
Date: Mon, 25 Jun 2018 11:42:44 +1000
Subject: [lustre-devel] [PATCH 1/8] staging: lustre: lnet: simplify
 lnet_eq_wait_locked
In-Reply-To: <1529877232-15582-2-git-send-email-jsimmons@infradead.org>
References: <1529877232-15582-1-git-send-email-jsimmons@infradead.org>
 <1529877232-15582-2-git-send-email-jsimmons@infradead.org>
Message-ID: <87in678wvf.fsf@notabene.neil.brown.name>

On Sun, Jun 24 2018, James Simmons wrote:

> We can simplify the code by taking advantage of the behavior
> of schedule_timeout_interruptible(). Instead of testing if
> tms is less than zero we can pass in a signed long that
> schedule_timeout_interruptible is expecting and for the case
> of no timeout we can pass in MAX_SCHEDULE_TIMEOUT.
>
> Signed-off-by: James Simmons <uja.ornl at yahoo.com>
> WC-bug-id: https://jira.whamcloud.com/browse/LU-9019
> Reviewed-on: https://review.whamcloud.com/23147
> Reviewed-by: Doug Oucharek <dougso at me.com>
> Reviewed-by: Dmitry Eremin <dmitry.eremin at intel.com>
> Reviewed-by: Oleg Drokin <green at whamcloud.com>
> Signed-off-by: James Simmons <jsimmons at infradead.org>
> ---
>  drivers/staging/lustre/include/linux/lnet/api.h    |  2 +-
>  .../lustre/include/uapi/linux/lnet/lnet-types.h    |  2 --
>  drivers/staging/lustre/lnet/lnet/api-ni.c          | 38 ++++++++++++++--------
>  drivers/staging/lustre/lnet/lnet/lib-eq.c          | 26 +++++----------
>  4 files changed, 33 insertions(+), 35 deletions(-)
>
> diff --git a/drivers/staging/lustre/include/linux/lnet/api.h b/drivers/staging/lustre/include/linux/lnet/api.h
> index dae2e4f..2242921 100644
> --- a/drivers/staging/lustre/include/linux/lnet/api.h
> +++ b/drivers/staging/lustre/include/linux/lnet/api.h
> @@ -168,7 +168,7 @@ int LNetEQAlloc(unsigned int       count_in,
>  
>  int LNetEQPoll(struct lnet_handle_eq *eventqs_in,
>  	       int		 neq_in,
> -	       int		 timeout_ms,
> +	       signed long	 timeout,
>  	       int		 interruptible,
>  	       struct lnet_event *event_out,
>  	       int		*which_eq_out);
> diff --git a/drivers/staging/lustre/include/uapi/linux/lnet/lnet-types.h b/drivers/staging/lustre/include/uapi/linux/lnet/lnet-types.h
> index 1be9b7a..14715a8 100644
> --- a/drivers/staging/lustre/include/uapi/linux/lnet/lnet-types.h
> +++ b/drivers/staging/lustre/include/uapi/linux/lnet/lnet-types.h
> @@ -77,8 +77,6 @@
>  #define LNET_PID_USERFLAG 0x80000000 /* set in userspace peers */
>  #define LNET_PID_LUSTRE	  12345
>  
> -#define LNET_TIME_FOREVER (-1)
> -
>  /* how an LNET NID encodes net:address */
>  /** extract the address part of an lnet_nid_t */
>  
> diff --git a/drivers/staging/lustre/lnet/lnet/api-ni.c b/drivers/staging/lustre/lnet/lnet/api-ni.c
> index f9ed697..878c92c 100644
> --- a/drivers/staging/lustre/lnet/lnet/api-ni.c
> +++ b/drivers/staging/lustre/lnet/lnet/api-ni.c
> @@ -59,7 +59,7 @@
>  module_param(rnet_htable_size, int, 0444);
>  MODULE_PARM_DESC(rnet_htable_size, "size of remote network hash table");
>  
> -static int lnet_ping(struct lnet_process_id id, int timeout_ms,
> +static int lnet_ping(struct lnet_process_id id, signed long timeout,
>  		     struct lnet_process_id __user *ids, int n_ids);
>  
>  static char *
> @@ -2052,17 +2052,29 @@ void lnet_lib_exit(void)
>  	case IOC_LIBCFS_LNET_FAULT:
>  		return lnet_fault_ctl(data->ioc_flags, data);
>  
> -	case IOC_LIBCFS_PING:
> +	case IOC_LIBCFS_PING: {
> +		signed long timeout;
> +
>  		id.nid = data->ioc_nid;
>  		id.pid = data->ioc_u32[0];
> -		rc = lnet_ping(id, data->ioc_u32[1], /* timeout */
> -			       data->ioc_pbuf1,
> +
> +		/* Don't block longer than 2 minutes */
> +		if (data->ioc_u32[1] > 120 * MSEC_PER_SEC)
> +			return -EINVAL;
> +
> +		/* If timestamp is negative then disable timeout */
> +		if ((s32)data->ioc_u32[1] < 0)
> +			timeout = MAX_SCHEDULE_TIMEOUT;
> +		else
> +			timeout = msecs_to_jiffies(data->ioc_u32[1]);
> +
> +		rc = lnet_ping(id, timeout, data->ioc_pbuf1,
>  			       data->ioc_plen1 / sizeof(struct lnet_process_id));
>  		if (rc < 0)
>  			return rc;
>  		data->ioc_count = rc;
>  		return 0;
> -
> +	}
>  	default:
>  		ni = lnet_net2ni(data->ioc_net);
>  		if (!ni)
> @@ -2126,7 +2138,7 @@ void LNetDebugPeer(struct lnet_process_id id)
>  }
>  EXPORT_SYMBOL(LNetGetId);
>  
> -static int lnet_ping(struct lnet_process_id id, int timeout_ms,
> +static int lnet_ping(struct lnet_process_id id, signed long timeout,
>  		     struct lnet_process_id __user *ids, int n_ids)
>  {
>  	struct lnet_handle_eq eqh;
> @@ -2136,7 +2148,7 @@ static int lnet_ping(struct lnet_process_id id, int timeout_ms,
>  	int which;
>  	int unlinked = 0;
>  	int replied = 0;
> -	const int a_long_time = 60000; /* mS */
> +	const signed long a_long_time = msecs_to_jiffies(60 * MSEC_PER_SEC);

I've changed this to
	a_long_toime = 60*HZ;

as that is the more common pattern in the kernel.

>  	int infosz;
>  	struct lnet_ping_info *info;
>  	struct lnet_process_id tmpid;
> @@ -2147,10 +2159,8 @@ static int lnet_ping(struct lnet_process_id id, int timeout_ms,
>  
>  	infosz = offsetof(struct lnet_ping_info, pi_ni[n_ids]);
>  
> -	if (n_ids <= 0 ||
> -	    id.nid == LNET_NID_ANY ||
> -	    timeout_ms > 500000 ||	      /* arbitrary limit! */
> -	    n_ids > 20)			 /* arbitrary limit! */
> +	/* n_ids limit is arbitrary */
> +	if (n_ids <= 0 || n_ids > 20 || id.nid == LNET_NID_ANY)
>  		return -EINVAL;
>  
>  	if (id.pid == LNET_PID_ANY)
> @@ -2194,13 +2204,13 @@ static int lnet_ping(struct lnet_process_id id, int timeout_ms,
>  
>  		/* NB must wait for the UNLINK event below... */
>  		unlinked = 1;
> -		timeout_ms = a_long_time;
> +		timeout = a_long_time;
>  	}
>  
>  	do {
>  		/* MUST block for unlink to complete */
>  
> -		rc2 = LNetEQPoll(&eqh, 1, timeout_ms, !unlinked,
> +		rc2 = LNetEQPoll(&eqh, 1, timeout, !unlinked,
>  				 &event, &which);
>  
>  		CDEBUG(D_NET, "poll %d(%d %d)%s\n", rc2,
> @@ -2222,7 +2232,7 @@ static int lnet_ping(struct lnet_process_id id, int timeout_ms,
>  				LNetMDUnlink(mdh);
>  				/* No assertion (racing with network) */
>  				unlinked = 1;
> -				timeout_ms = a_long_time;
> +				timeout = a_long_time;
>  			} else if (!rc2) {
>  				/* timed out waiting for unlink */
>  				CWARN("ping %s: late network completion\n",
> diff --git a/drivers/staging/lustre/lnet/lnet/lib-eq.c b/drivers/staging/lustre/lnet/lnet/lib-eq.c
> index c78e703..366ad8a 100644
> --- a/drivers/staging/lustre/lnet/lnet/lib-eq.c
> +++ b/drivers/staging/lustre/lnet/lnet/lib-eq.c
> @@ -308,13 +308,12 @@
>   */
>  
>  static int
> -lnet_eq_wait_locked(int *timeout_ms, long state)
> +lnet_eq_wait_locked(signed long *timeout, long state)
>  __must_hold(&the_lnet.ln_eq_wait_lock)
>  {
> -	int tms = *timeout_ms;
> +	signed long tms = *timeout;
>  	int wait;
>  	wait_queue_entry_t wl;
> -	unsigned long now;
>  
>  	if (!tms)
>  		return -ENXIO; /* don't want to wait and no new event */
> @@ -325,18 +324,9 @@
>  
>  	lnet_eq_wait_unlock();
>  
> -	if (tms < 0) {
> -		schedule();
> -	} else {
> -		now = jiffies;
> -		schedule_timeout(msecs_to_jiffies(tms));
> -		tms -= jiffies_to_msecs(jiffies - now);
> -		if (tms < 0) /* no more wait but may have new event */
> -			tms = 0;
> -	}
> -
> +	tms = schedule_timeout_interruptible(tms);

I've changed this to schedule_timeout().
The task state has already been set (to INTERRUPTIBLE or NOLOAD) and
we don't want to set it again.

Thanks,
NeilBrown


>  	wait = tms; /* might need to call here again */
> -	*timeout_ms = tms;
> +	*timeout = tms;
>  
>  	lnet_eq_wait_lock();
>  	remove_wait_queue(&the_lnet.ln_eq_waitq, &wl);
> @@ -356,8 +346,8 @@
>   * fixed period, or block indefinitely.
>   *
>   * \param eventqs,neq An array of EQ handles, and size of the array.
> - * \param timeout_ms Time in milliseconds to wait for an event to occur on
> - * one of the EQs. The constant LNET_TIME_FOREVER can be used to indicate an
> + * \param timeout Time in jiffies to wait for an event to occur on
> + * one of the EQs. The constant MAX_SCHEDULE_TIMEOUT can be used to indicate an
>   * infinite timeout.
>   * \param interruptible, if true, use TASK_INTERRUPTIBLE, else TASK_NOLOAD
>   * \param event,which On successful return (1 or -EOVERFLOW), \a event will
> @@ -372,7 +362,7 @@
>   * \retval -ENOENT    If there's an invalid handle in \a eventqs.
>   */
>  int
> -LNetEQPoll(struct lnet_handle_eq *eventqs, int neq, int timeout_ms,
> +LNetEQPoll(struct lnet_handle_eq *eventqs, int neq, signed long timeout,
>  	   int interruptible,
>  	   struct lnet_event *event, int *which)
>  {
> @@ -414,7 +404,7 @@
>  		 *  0 : don't want to wait anymore, but might have new event
>  		 *      so need to call dequeue again
>  		 */
> -		wait = lnet_eq_wait_locked(&timeout_ms,
> +		wait = lnet_eq_wait_locked(&timeout,
>  					   interruptible ? TASK_INTERRUPTIBLE
>  					   : TASK_NOLOAD);
>  		if (wait < 0) /* no new event */
> -- 
> 1.8.3.1
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180625/42a1c406/attachment.sig>

From neilb at suse.com  Mon Jun 25 01:54:38 2018
From: neilb at suse.com (NeilBrown)
Date: Mon, 25 Jun 2018 11:54:38 +1000
Subject: [lustre-devel] [PATCH 0/8] staging: lustre: lnet: resolve time
 handling issues
In-Reply-To: <1529877232-15582-1-git-send-email-jsimmons@infradead.org>
References: <1529877232-15582-1-git-send-email-jsimmons@infradead.org>
Message-ID: <87fu1b8wbl.fsf@notabene.neil.brown.name>

On Sun, Jun 24 2018, James Simmons wrote:

> The migration of lustre to 64 bit time handling exposed various bugs
> which have lead to problems like routers no longer functioning with
> the upstream client. This is due to the complex handling of time by
> lustre with seconds time units and jiffies. Add in the fact that
> jiffies can vary from node to node and you can encounter corner
> case issues in a large node deployment. The solution to resolve this
> was to migrate as much as possible to time64_t when all you needed
> was seconds percision and ktime_t if better percision was required.
> With these changes it is reduced down to using jiffies only for
> schedule_timeout calls.
>
> James Simmons (8):
>   staging: lustre: lnet: simplify lnet_eq_wait_locked
>   staging: lustre: o2iblnd: use ktime_get_real_ns for ibn_incarnation
>   staging: lustre: ksocklnd: use ktime_get_real_ns for ksnn_incarnation
>   staging: lustre: ksocklnd: move remaining time handling to 64 bits
>   staging: lustre: lnet: use ktime_get_real_ns for cookie creation
>   staging: lustre: lnet: move ping and delay injection to time64_t
>   staging: lustre: ko2iblnd: remove remaining jiffy use
>   staging: lustre: selftest: fix time reporting

Lots of nice improvements in there - thanks.
I've applied all patches, though I made small changed to the first
as described separately.

Thanks,
NeilBrown
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180625/8baaf29f/attachment.sig>

From neilb at suse.com  Mon Jun 25 01:57:06 2018
From: neilb at suse.com (NeilBrown)
Date: Mon, 25 Jun 2018 11:57:06 +1000
Subject: [lustre-devel] [PATCH 1/3] lustre: use
 generic_error_remove_page()
In-Reply-To: <EAC0696A-9C7F-419A-A360-C352537B1DA2@linuxhacker.ru>
References: <152904663333.10587.10934053155404014785.stgit@noble>
 <152904669032.10587.11474850109829193046.stgit@noble>
 <alpine.LFD.2.21.1806160115430.15679@casper.infradead.org>
 <87y3ffft80.fsf@notabene.neil.brown.name>
 <61C2F92E-8F88-49C6-83FB-F3013289265C@linuxhacker.ru>
 <877emwg8ao.fsf@notabene.neil.brown.name>
 <87lgb3ag3k.fsf@notabene.neil.brown.name>
 <EAC0696A-9C7F-419A-A360-C352537B1DA2@linuxhacker.ru>
Message-ID: <87d0wf8w7h.fsf@notabene.neil.brown.name>

On Sun, Jun 24 2018, Oleg Drokin wrote:

>> On Jun 24, 2018, at 8:02 PM, NeilBrown <neilb at suse.com> wrote:
>> 
>> 
>> lustre's internal ll_invalidate_page() is behaviourally identical to
>> generic_error_remove_page().
>> In the case of lustre it isn't a memory hardware error that requires
>> the page being invalidated, it is the loss of a lock, which will like
>> result in the data changing on the server.
>> In either case, we don't want the page to be accessed any more, so the
>> same removal is appropriate.
>> 
>> Signed-off-by: NeilBrown <neilb at suse.com>
>> ---
>> 
>> I've replaced
>> [PATCH 08/24] lustre: use truncate_inode_page in place of truncate_complete_page
>> with 3 patches, this and the next two.
>
> This looks reasonable. Are you running any tests on this?

Just the sanity tests on a 4-node vcluster.  They haven't caused any
noticeable regressions.

>
> Acked-by: Oleg Drokin <green at linuxhacker.ru>

Thanks,
NeilBrown

>
>> 
>> Thanks,
>> NeilBrown
>> 
>> 
>> drivers/staging/lustre/lustre/llite/llite_internal.h | 17 -----------------
>> drivers/staging/lustre/lustre/llite/vvp_io.c         |  2 +-
>> drivers/staging/lustre/lustre/llite/vvp_page.c       |  2 +-
>> 3 files changed, 2 insertions(+), 19 deletions(-)
>> 
>> diff --git a/drivers/staging/lustre/lustre/llite/llite_internal.h b/drivers/staging/lustre/lustre/llite/llite_internal.h
>> index c08a6e14b6d7..22dcabf6de0f 100644
>> --- a/drivers/staging/lustre/lustre/llite/llite_internal.h
>> +++ b/drivers/staging/lustre/lustre/llite/llite_internal.h
>> @@ -928,23 +928,6 @@ void policy_from_vma(union ldlm_policy_data *policy, struct vm_area_struct *vma,
>> struct vm_area_struct *our_vma(struct mm_struct *mm, unsigned long addr,
>> 			       size_t count);
>> 
>> -static inline void ll_invalidate_page(struct page *vmpage)
>> -{
>> -	struct address_space *mapping = vmpage->mapping;
>> -	loff_t offset = vmpage->index << PAGE_SHIFT;
>> -
>> -	LASSERT(PageLocked(vmpage));
>> -	if (!mapping)
>> -		return;
>> -
>> -	/*
>> -	 * truncate_complete_page() calls
>> -	 * a_ops->invalidatepage()->cl_page_delete()->vvp_page_delete().
>> -	 */
>> -	ll_teardown_mmaps(mapping, offset, offset + PAGE_SIZE);
>> -	truncate_complete_page(mapping, vmpage);
>> -}
>> -
>> #define    ll_s2sbi(sb)	(s2lsi(sb)->lsi_llsbi)
>> 
>> /* don't need an addref as the sb_info should be holding one */
>> diff --git a/drivers/staging/lustre/lustre/llite/vvp_io.c b/drivers/staging/lustre/lustre/llite/vvp_io.c
>> index e7a4778e02e4..5a67955974ad 100644
>> --- a/drivers/staging/lustre/lustre/llite/vvp_io.c
>> +++ b/drivers/staging/lustre/lustre/llite/vvp_io.c
>> @@ -1098,7 +1098,7 @@ static int vvp_io_fault_start(const struct lu_env *env,
>> 	LASSERT(PageLocked(vmpage));
>> 
>> 	if (OBD_FAIL_CHECK(OBD_FAIL_LLITE_FAULT_TRUNC_RACE))
>> -		ll_invalidate_page(vmpage);
>> +		generic_error_remove_page(vmpage->mapping, vmpage);
>> 
>> 	size = i_size_read(inode);
>> 	/* Though we have already held a cl_lock upon this page, but
>> diff --git a/drivers/staging/lustre/lustre/llite/vvp_page.c b/drivers/staging/lustre/lustre/llite/vvp_page.c
>> index 6eb0565ddc22..dcc4d8faa0cd 100644
>> --- a/drivers/staging/lustre/lustre/llite/vvp_page.c
>> +++ b/drivers/staging/lustre/lustre/llite/vvp_page.c
>> @@ -147,7 +147,7 @@ static void vvp_page_discard(const struct lu_env *env,
>> 	if (vpg->vpg_defer_uptodate && !vpg->vpg_ra_used)
>> 		ll_ra_stats_inc(vmpage->mapping->host, RA_STAT_DISCARDED);
>> 
>> -	ll_invalidate_page(vmpage);
>> +	generic_error_remove_page(vmpage->mapping, vmpage);
>> }
>> 
>> static void vvp_page_delete(const struct lu_env *env,
>> -- 
>> 2.14.0.rc0.dirty
>> 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180625/a2a1b8c7/attachment.sig>

From neilb at suse.com  Mon Jun 25 02:38:02 2018
From: neilb at suse.com (NeilBrown)
Date: Mon, 25 Jun 2018 12:38:02 +1000
Subject: [lustre-devel] [PATCH v3 23/26] staging: lustre: ptlrpc: use
 current CPU instead of hardcoded 0
In-Reply-To: <1529875250-11531-24-git-send-email-jsimmons@infradead.org>
References: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
 <1529875250-11531-24-git-send-email-jsimmons@infradead.org>
Message-ID: <87a7rj8ub9.fsf@notabene.neil.brown.name>

On Sun, Jun 24 2018, James Simmons wrote:

> From: Dmitry Eremin <dmitry.eremin at intel.com>
>
> fix crash if CPU 0 disabled.
>
> Signed-off-by: Dmitry Eremin <dmitry.eremin at intel.com>
> WC-bug-id: https://jira.whamcloud.com/browse/LU-8710
> Reviewed-on: https://review.whamcloud.com/23305
> Reviewed-by: Doug Oucharek <dougso at me.com>
> Reviewed-by: Andreas Dilger <adilger at whamcloud.com>
> Signed-off-by: James Simmons <jsimmons at infradead.org>
> ---
>  drivers/staging/lustre/lustre/ptlrpc/service.c | 11 ++++++-----
>  1 file changed, 6 insertions(+), 5 deletions(-)
>
> diff --git a/drivers/staging/lustre/lustre/ptlrpc/service.c b/drivers/staging/lustre/lustre/ptlrpc/service.c
> index 3fd8c74..8e74a45 100644
> --- a/drivers/staging/lustre/lustre/ptlrpc/service.c
> +++ b/drivers/staging/lustre/lustre/ptlrpc/service.c
> @@ -421,7 +421,7 @@ static void ptlrpc_at_timer(struct timer_list *t)
>  		 * there are.
>  		 */
>  		/* weight is # of HTs */
> -		if (cpumask_weight(topology_sibling_cpumask(0)) > 1) {
> +		if (cpumask_weight(topology_sibling_cpumask(smp_processor_id())) > 1) {

This pops a warning for me:
[ 1877.516799] BUG: using smp_processor_id() in preemptible [00000000] code: mount.lustre/14077

I'll change it to disable preemption, both here and below.

Thanks,
NeilBrown


>  			/* depress thread factor for hyper-thread */
>  			factor = factor - (factor >> 1) + (factor >> 3);
>  		}
> @@ -2221,15 +2221,16 @@ static int ptlrpc_hr_main(void *arg)
>  	struct ptlrpc_hr_thread	*hrt = arg;
>  	struct ptlrpc_hr_partition *hrp = hrt->hrt_partition;
>  	LIST_HEAD(replies);
> -	char threadname[20];
>  	int rc;
>  
> -	snprintf(threadname, sizeof(threadname), "ptlrpc_hr%02d_%03d",
> -		 hrp->hrp_cpt, hrt->hrt_id);
>  	unshare_fs_struct();
>  
>  	rc = cfs_cpt_bind(ptlrpc_hr.hr_cpt_table, hrp->hrp_cpt);
>  	if (rc != 0) {
> +		char threadname[20];
> +
> +		snprintf(threadname, sizeof(threadname), "ptlrpc_hr%02d_%03d",
> +			 hrp->hrp_cpt, hrt->hrt_id);
>  		CWARN("Failed to bind %s on CPT %d of CPT table %p: rc = %d\n",
>  		      threadname, hrp->hrp_cpt, ptlrpc_hr.hr_cpt_table, rc);
>  	}
> @@ -2528,7 +2529,7 @@ int ptlrpc_hr_init(void)
>  
>  	init_waitqueue_head(&ptlrpc_hr.hr_waitq);
>  
> -	weight = cpumask_weight(topology_sibling_cpumask(0));
> +	weight = cpumask_weight(topology_sibling_cpumask(smp_processor_id()));
>  
>  	cfs_percpt_for_each(hrp, i, ptlrpc_hr.hr_partitions) {
>  		hrp->hrp_cpt = i;
> -- 
> 1.8.3.1
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180625/36ca15f0/attachment.sig>

From doucharek at cray.com  Mon Jun 25 18:22:59 2018
From: doucharek at cray.com (Doug Oucharek)
Date: Mon, 25 Jun 2018 18:22:59 +0000
Subject: [lustre-devel] [PATCH v3 07/26] staging: lustre: libcfs: NUMA
 support
In-Reply-To: <877emnaed8.fsf@notabene.neil.brown.name>
References: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
 <1529875250-11531-8-git-send-email-jsimmons@infradead.org>
 <877emnaed8.fsf@notabene.neil.brown.name>
Message-ID: <A764DED1-A9A8-4FA5-A0F1-922C3E2D822A@cray.com>

Some background on this NUMA change:

First off, this is just a first step to a bigger set of changes which include changes to the Lustre utilities.  This was done as part of the Multi-Rail feature.  One of the systems that feature is meant to support is the SGI UV system (now HPE) which has a massive number of NUMA nodes connected by a NUMA Link.  There are multiple fabric cards spread throughout the system and Multi-Rail needs to know which fabric cards are nearest to the NUMA node we are running on.  To do that, the “distance” between NUMA nodes needs to be configured.

This patch is preparing the infrastructure for the Multi-Rail feature to support configuring NUMA node distances.  Technically, this patch should be landing with the Multi-Rail feature (still to be pushed) for it to make proper sense.

Doug

On Jun 24, 2018, at 5:39 PM, NeilBrown <neilb at suse.com<mailto:neilb at suse.com>> wrote:

On Sun, Jun 24 2018, James Simmons wrote:

From: Amir Shehata <amir.shehata at intel.com<mailto:amir.shehata at intel.com>>

This patch adds NUMA node support. NUMA node information is stored
in the CPT table. A NUMA node mask is maintained for the entire
table as well as for each CPT to track the NUMA nodes related to
each of the CPTs. Add new function cfs_cpt_of_node() which returns
the CPT of a particular NUMA node.

I note that you didn't respond to Greg's questions about this patch.
I'll accept it anyway in the interests of moving forward, but I think
his comments were probably valid, and need to be considered at some
stage.

There is a bug though....

Signed-off-by: Amir Shehata <amir.shehata at intel.com<mailto:amir.shehata at intel.com>>
WC-bug-id: https://jira.whamcloud.com/browse/LU-7734
Reviewed-on: http://review.whamcloud.com/18916
Reviewed-by: Olaf Weber <olaf at sgi.com<mailto:olaf at sgi.com>>
Reviewed-by: Doug Oucharek <dougso at me.com<mailto:dougso at me.com>>
Signed-off-by: James Simmons <jsimmons at infradead.org<mailto:jsimmons at infradead.org>>
---
.../lustre/include/linux/libcfs/libcfs_cpu.h        | 11 +++++++++++
drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c     | 21 +++++++++++++++++++++
2 files changed, 32 insertions(+)

diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h
index 1b4333d..ff3ecf5 100644
--- a/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h
+++ b/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h
@@ -103,6 +103,8 @@ struct cfs_cpt_table {
int *ctb_cpu2cpt;
/* all cpus in this partition table */
cpumask_var_t ctb_cpumask;
+ /* shadow HW node to CPU partition ID */
+ int *ctb_node2cpt;
/* all nodes in this partition table */
nodemask_t *ctb_nodemask;
};
@@ -143,6 +145,10 @@ struct cfs_cpt_table {
 */
int cfs_cpt_of_cpu(struct cfs_cpt_table *cptab, int cpu);
/**
+ * shadow HW node ID \a NODE to CPU-partition ID by \a cptab
+ */
+int cfs_cpt_of_node(struct cfs_cpt_table *cptab, int node);
+/**
 * bind current thread on a CPU-partition \a cpt of \a cptab
 */
int cfs_cpt_bind(struct cfs_cpt_table *cptab, int cpt);
@@ -299,6 +305,11 @@ void cfs_cpt_unset_nodemask(struct cfs_cpt_table *cptab,
return 0;
}

+static inline int cfs_cpt_of_node(struct cfs_cpt_table *cptab, int node)
+{
+ return 0;
+}
+
static inline int
cfs_cpt_bind(struct cfs_cpt_table *cptab, int cpt)
{
diff --git a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
index 33294da..8c5cf7b 100644
--- a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
+++ b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
@@ -102,6 +102,15 @@ struct cfs_cpt_table *
memset(cptab->ctb_cpu2cpt, -1,
       nr_cpu_ids * sizeof(cptab->ctb_cpu2cpt[0]));

+ cptab->ctb_node2cpt = kvmalloc_array(nr_node_ids,
+      sizeof(cptab->ctb_node2cpt[0]),
+      GFP_KERNEL);
+ if (!cptab->ctb_node2cpt)
+ goto failed_alloc_node2cpt;
+
+ memset(cptab->ctb_node2cpt, -1,
+        nr_node_ids * sizeof(cptab->ctb_node2cpt[0]));
+
cptab->ctb_parts = kvmalloc_array(ncpt, sizeof(cptab->ctb_parts[0]),
  GFP_KERNEL);
if (!cptab->ctb_parts)
@@ -133,6 +142,8 @@ struct cfs_cpt_table *

kvfree(cptab->ctb_parts);
failed_alloc_ctb_parts:
+ kvfree(cptab->ctb_node2cpt);
+failed_alloc_node2cpt:
kvfree(cptab->ctb_cpu2cpt);
failed_alloc_cpu2cpt:
kfree(cptab->ctb_nodemask);
@@ -150,6 +161,7 @@ struct cfs_cpt_table *
int i;

kvfree(cptab->ctb_cpu2cpt);
+ kvfree(cptab->ctb_node2cpt);

for (i = 0; cptab->ctb_parts && i < cptab->ctb_nparts; i++) {
struct cfs_cpu_partition *part = &cptab->ctb_parts[i];
@@ -515,6 +527,15 @@ struct cfs_cpt_table *
}
EXPORT_SYMBOL(cfs_cpt_of_cpu);

+int cfs_cpt_of_node(struct cfs_cpt_table *cptab, int node)
+{
+ if (node < 0 || node > nr_node_ids)
+ return CFS_CPT_ANY;
+
+ return cptab->ctb_node2cpt[node];
+}

So if node == nr_node_ids, we access beyond the end of the ctb_node2cpt array.
Oops.
I've fixed this before applying.

Thanks,
NeilBrown


+EXPORT_SYMBOL(cfs_cpt_of_node);
+
int
cfs_cpt_bind(struct cfs_cpt_table *cptab, int cpt)
{
--
1.8.3.1
_______________________________________________
lustre-devel mailing list
lustre-devel at lists.lustre.org<mailto:lustre-devel at lists.lustre.org>
http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180625/7c60ef32/attachment.html>

From jsimmons at infradead.org  Mon Jun 25 21:42:51 2018
From: jsimmons at infradead.org (James Simmons)
Date: Mon, 25 Jun 2018 17:42:51 -0400
Subject: [lustre-devel] [PATCH v2 2/7] lustre: libcfs: open code
 cfs_trace_max_debug_mb() into cfs_trace_set_debug_mb()
In-Reply-To: <1529962976-15048-1-git-send-email-jsimmons@infradead.org>
References: <1529962976-15048-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1529962976-15048-3-git-send-email-jsimmons@infradead.org>

From: NeilBrown <neilb at suse.com>

This function in used twice.
In libcfs_debug_init() the usage is pointless as
the only place that set libcfs_debug_mb ensures
that it does not exceed the maximum.  So checking
again can never help.

So open-code the small function into the only
other place that it is used - in cfs_trace_set_debug_mb(),
which is used to set libcfs_debug_mb.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 drivers/staging/lustre/lnet/libcfs/debug.c           | 6 +++---
 drivers/staging/lustre/lnet/libcfs/linux-tracefile.c | 7 -------
 drivers/staging/lustre/lnet/libcfs/tracefile.c       | 3 ++-
 3 files changed, 5 insertions(+), 11 deletions(-)

diff --git a/drivers/staging/lustre/lnet/libcfs/debug.c b/drivers/staging/lustre/lnet/libcfs/debug.c
index 06f694f..71effcf 100644
--- a/drivers/staging/lustre/lnet/libcfs/debug.c
+++ b/drivers/staging/lustre/lnet/libcfs/debug.c
@@ -411,10 +411,10 @@ int libcfs_debug_init(unsigned long bufsize)
 			sizeof(libcfs_debug_file_path_arr));
 	}
 
-	/* If libcfs_debug_mb is set to an invalid value or uninitialized
-	 * then just make the total buffers smp_num_cpus * TCD_MAX_PAGES
+	/* If libcfs_debug_mb is uninitialized then just make the
+	 * total buffers smp_num_cpus * TCD_MAX_PAGES
 	 */
-	if (max > cfs_trace_max_debug_mb() || max < num_possible_cpus()) {
+	if (max < num_possible_cpus()) {
 		max = TCD_MAX_PAGES;
 	} else {
 		max = max / num_possible_cpus();
diff --git a/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c b/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
index 9e72220..64a5bc1 100644
--- a/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
+++ b/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
@@ -227,10 +227,3 @@ void cfs_print_to_console(struct ptldebug_header *hdr, int mask,
 			hdr->ph_line_num, fn, len, buf);
 	}
 }
-
-int cfs_trace_max_debug_mb(void)
-{
-	int  total_mb = (totalram_pages >> (20 - PAGE_SHIFT));
-
-	return max(512, (total_mb * 80) / 100);
-}
diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.c b/drivers/staging/lustre/lnet/libcfs/tracefile.c
index 5f31933..72321ce 100644
--- a/drivers/staging/lustre/lnet/libcfs/tracefile.c
+++ b/drivers/staging/lustre/lnet/libcfs/tracefile.c
@@ -933,7 +933,8 @@ int cfs_trace_set_debug_mb(int mb)
 	int i;
 	int j;
 	int pages;
-	int limit = cfs_trace_max_debug_mb();
+	int total_mb = (totalram_pages >> (20 - PAGE_SHIFT));
+	int limit = max(512, (total_mb * 80) / 100);
 	struct cfs_trace_cpu_data *tcd;
 
 	if (mb < num_possible_cpus()) {
-- 
1.8.3.1


From jsimmons at infradead.org  Mon Jun 25 21:42:49 2018
From: jsimmons at infradead.org (James Simmons)
Date: Mon, 25 Jun 2018 17:42:49 -0400
Subject: [lustre-devel] [PATCH v2 0/7] lustre: libcfs: tracefile cleanups
Message-ID: <1529962976-15048-1-git-send-email-jsimmons@infradead.org>

The first three patches in this series are a repost of Neil's patches.
The 4th patch fixed the brokeness of cfs_print_to_console() which shows
up in my testing. Besides the fixing more cleanups of tracefile.h was
done. The rest of the patches are Neil's work rebased due the changes
of the 4th patch.

NeilBrown (7):
  lustre: libcfs: discard TCD_MAX_TYPES
  lustre: libcfs: move tracefile locking from linux-tracefile.c to tracefile.c
  lustre: libcfs: open code cfs_trace_max_debug_mb() into cfs_trace_set_debug_mb()
  lustre: libcfs: move tcd locking across to tracefile.c
  lustre: libcfs: merge linux-tracefile.c into tracefile.c
  lustre: libcfs: fold cfs_tracefile_*_arch into their only callers.
  lustre: libcfs: renamed CFS_TCD_TYPE_MAX to CFS_TCD_TYPE_CNT

 drivers/staging/lustre/lnet/libcfs/Makefile        |   2 +-
 drivers/staging/lustre/lnet/libcfs/debug.c         |   6 +-
 .../staging/lustre/lnet/libcfs/linux-tracefile.c   | 258 --------------------
 drivers/staging/lustre/lnet/libcfs/tracefile.c     | 266 +++++++++++++++++----
 drivers/staging/lustre/lnet/libcfs/tracefile.h     | 112 +--------
 5 files changed, 234 insertions(+), 410 deletions(-)
 delete mode 100644 drivers/staging/lustre/lnet/libcfs/linux-tracefile.c

-- 
1.8.3.1


From jsimmons at infradead.org  Mon Jun 25 21:42:54 2018
From: jsimmons at infradead.org (James Simmons)
Date: Mon, 25 Jun 2018 17:42:54 -0400
Subject: [lustre-devel] [PATCH v2 5/7] lustre: libcfs: fold
 cfs_tracefile_*_arch into their only callers.
In-Reply-To: <1529962976-15048-1-git-send-email-jsimmons@infradead.org>
References: <1529962976-15048-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1529962976-15048-6-git-send-email-jsimmons@infradead.org>

From: NeilBrown <neilb at suse.com>

There is no need to separate "arch" init/fini from
the rest, so fold it all in.
This requires some slightly subtle changes to clean-up
to make sure we don't walk lists before they are
initialized.

Signed-off-by: NeilBrown <neilb at suse.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/lnet/libcfs/tracefile.c | 138 +++++++++++--------------
 1 file changed, 59 insertions(+), 79 deletions(-)

diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.c b/drivers/staging/lustre/lnet/libcfs/tracefile.c
index 229bbb8..b370c56 100644
--- a/drivers/staging/lustre/lnet/libcfs/tracefile.c
+++ b/drivers/staging/lustre/lnet/libcfs/tracefile.c
@@ -200,77 +200,6 @@ static inline void cfs_trace_put_tcd(struct cfs_trace_cpu_data *tcd)
 	put_cpu();
 }
 
-/* percents to share the total debug memory for each type */
-static unsigned int pages_factor[CFS_TCD_TYPE_MAX] = {
-	80,  /* 80% pages for CFS_TCD_TYPE_PROC */
-	10,  /* 10% pages for CFS_TCD_TYPE_SOFTIRQ */
-	10   /* 10% pages for CFS_TCD_TYPE_IRQ */
-};
-
-int cfs_tracefile_init_arch(void)
-{
-	struct cfs_trace_cpu_data *tcd;
-	int i;
-	int j;
-
-	/* initialize trace_data */
-	memset(cfs_trace_data, 0, sizeof(cfs_trace_data));
-	for (i = 0; i < CFS_TCD_TYPE_MAX; i++) {
-		cfs_trace_data[i] =
-			kmalloc_array(num_possible_cpus(),
-				      sizeof(union cfs_trace_data_union),
-				      GFP_KERNEL);
-		if (!cfs_trace_data[i])
-			goto out_trace_data;
-	}
-
-	/* arch related info initialized */
-	cfs_tcd_for_each(tcd, i, j) {
-		spin_lock_init(&tcd->tcd_lock);
-		tcd->tcd_pages_factor = pages_factor[i];
-		tcd->tcd_type = i;
-		tcd->tcd_cpu = j;
-	}
-
-	for (i = 0; i < num_possible_cpus(); i++)
-		for (j = 0; j < 3; j++) {
-			cfs_trace_console_buffers[i][j] =
-				kmalloc(CFS_TRACE_CONSOLE_BUFFER_SIZE,
-					GFP_KERNEL);
-
-			if (!cfs_trace_console_buffers[i][j])
-				goto out_buffers;
-		}
-
-	return 0;
-
-out_buffers:
-	for (i = 0; cfs_trace_data[i]; i++) {
-		kfree(cfs_trace_data[i]);
-		cfs_trace_data[i] = NULL;
-	}
-out_trace_data:
-	pr_err("lnet: Not enough memory\n");
-	return -ENOMEM;
-}
-
-void cfs_tracefile_fini_arch(void)
-{
-	int i;
-	int j;
-
-	for (i = 0; i < num_possible_cpus(); i++)
-		for (j = 0; j < 3; j++) {
-			kfree(cfs_trace_console_buffers[i][j]);
-			cfs_trace_console_buffers[i][j] = NULL;
-		}
-
-	for (i = 0; cfs_trace_data[i]; i++) {
-		kfree(cfs_trace_data[i]);
-		cfs_trace_data[i] = NULL;
-	}
-}
-
 static inline struct cfs_trace_page *
 cfs_tage_from_list(struct list_head *list)
 {
@@ -1322,21 +1251,38 @@ void cfs_trace_stop_thread(void)
 	mutex_unlock(&cfs_trace_thread_mutex);
 }
 
+/* percents to share the total debug memory for each type */
+static unsigned int pages_factor[CFS_TCD_TYPE_MAX] = {
+	80,  /* 80% pages for CFS_TCD_TYPE_PROC */
+	10,  /* 10% pages for CFS_TCD_TYPE_SOFTIRQ */
+	10   /* 10% pages for CFS_TCD_TYPE_IRQ */
+};
+
 int cfs_tracefile_init(int max_pages)
 {
 	struct cfs_trace_cpu_data *tcd;
 	int i;
 	int j;
-	int rc;
-	int factor;
 
-	rc = cfs_tracefile_init_arch();
-	if (rc)
-		return rc;
+	/* initialize trace_data */
+	memset(cfs_trace_data, 0, sizeof(cfs_trace_data));
+	for (i = 0; i < CFS_TCD_TYPE_MAX; i++) {
+		cfs_trace_data[i] =
+			kmalloc_array(num_possible_cpus(),
+				      sizeof(union cfs_trace_data_union),
+				      GFP_KERNEL);
+		if (!cfs_trace_data[i])
+			goto out_trace_data;
+	}
 
+	/* arch related info initialized */
 	cfs_tcd_for_each(tcd, i, j) {
-		/* tcd_pages_factor is initialized int tracefile_init_arch. */
-		factor = tcd->tcd_pages_factor;
+		int factor = pages_factor[i];
+
+		spin_lock_init(&tcd->tcd_lock);
+		tcd->tcd_pages_factor = factor;
+		tcd->tcd_type = i;
+		tcd->tcd_cpu = j;
 		INIT_LIST_HEAD(&tcd->tcd_pages);
 		INIT_LIST_HEAD(&tcd->tcd_stock_pages);
 		INIT_LIST_HEAD(&tcd->tcd_daemon_pages);
@@ -1348,7 +1294,27 @@ int cfs_tracefile_init(int max_pages)
 		tcd->tcd_shutting_down = 0;
 	}
 
+	for (i = 0; i < num_possible_cpus(); i++)
+		for (j = 0; j < 3; j++) {
+			cfs_trace_console_buffers[i][j] =
+				kmalloc(CFS_TRACE_CONSOLE_BUFFER_SIZE,
+					GFP_KERNEL);
+
+			if (!cfs_trace_console_buffers[i][j])
+				goto out_console_buffers;
+		}
+
 	return 0;
+
+out_console_buffers:
+	for (i = 0; i < num_possible_cpus(); i++)
+		for (j = 0; j < 3; j++) {
+			kfree(cfs_trace_console_buffers[i][j]);
+			cfs_trace_console_buffers[i][j] = NULL;
+		}
+out_trace_data:
+	pr_err("lnet: Not enough memory\n");
+	return -ENOMEM;
 }
 
 static void trace_cleanup_on_all_cpus(void)
@@ -1360,6 +1326,9 @@ static void trace_cleanup_on_all_cpus(void)
 
 	for_each_possible_cpu(cpu) {
 		cfs_tcd_for_each_type_lock(tcd, i, cpu) {
+			if (!tcd->tcd_pages_factor)
+				/* Not initialised */
+				continue;
 			tcd->tcd_shutting_down = 1;
 
 			list_for_each_entry_safe(tage, tmp, &tcd->tcd_pages,
@@ -1378,12 +1347,23 @@ static void trace_cleanup_on_all_cpus(void)
 static void cfs_trace_cleanup(void)
 {
 	struct page_collection pc;
+	int i;
+	int j;
 
 	INIT_LIST_HEAD(&pc.pc_pages);
 
 	trace_cleanup_on_all_cpus();
 
-	cfs_tracefile_fini_arch();
+	for (i = 0; i < num_possible_cpus(); i++)
+		for (j = 0; j < 3; j++) {
+			kfree(cfs_trace_console_buffers[i][j]);
+			cfs_trace_console_buffers[i][j] = NULL;
+		}
+
+	for (i = 0; cfs_trace_data[i]; i++) {
+		kfree(cfs_trace_data[i]);
+		cfs_trace_data[i] = NULL;
+	}
 }
 
 void cfs_tracefile_exit(void)
-- 
1.8.3.1


From jsimmons at infradead.org  Mon Jun 25 21:42:53 2018
From: jsimmons at infradead.org (James Simmons)
Date: Mon, 25 Jun 2018 17:42:53 -0400
Subject: [lustre-devel] [PATCH v2 4/7] lustre: libcfs: merge
 linux-tracefile.c into tracefile.c
In-Reply-To: <1529962976-15048-1-git-send-email-jsimmons@infradead.org>
References: <1529962976-15048-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1529962976-15048-5-git-send-email-jsimmons@infradead.org>

From: NeilBrown <neilb at suse.com>

It's good to keep related code together. The merger exposed the
flaws of cfs_print_to_console() so rework it to behavior properly.

Signed-off-by: NeilBrown <neilb at suse.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/lnet/libcfs/Makefile        |   2 +-
 .../staging/lustre/lnet/libcfs/linux-tracefile.c   | 194 ---------------------
 drivers/staging/lustre/lnet/libcfs/tracefile.c     | 184 ++++++++++++++++---
 drivers/staging/lustre/lnet/libcfs/tracefile.h     |  79 ++-------
 4 files changed, 170 insertions(+), 289 deletions(-)
 delete mode 100644 drivers/staging/lustre/lnet/libcfs/linux-tracefile.c

diff --git a/drivers/staging/lustre/lnet/libcfs/Makefile b/drivers/staging/lustre/lnet/libcfs/Makefile
index 5b13edc..cd96434 100644
--- a/drivers/staging/lustre/lnet/libcfs/Makefile
+++ b/drivers/staging/lustre/lnet/libcfs/Makefile
@@ -4,7 +4,7 @@ ccflags-y += -I$(srctree)/drivers/staging/lustre/lustre/include
 
 obj-$(CONFIG_LNET) += libcfs.o
 
-libcfs-obj-y += linux-tracefile.o linux-debug.o
+libcfs-obj-y += linux-debug.o
 libcfs-obj-y += linux-crypto.o
 libcfs-obj-y += linux-crypto-adler.o
 
diff --git a/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c b/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
deleted file mode 100644
index 3af7722..0000000
--- a/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
+++ /dev/null
@@ -1,194 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-/*
- * GPL HEADER START
- *
- * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 only,
- * as published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful, but
- * WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
- * General Public License version 2 for more details (a copy is included
- * in the LICENSE file that accompanied this code).
- *
- * You should have received a copy of the GNU General Public License
- * version 2 along with this program; If not, see
- * http://www.gnu.org/licenses/gpl-2.0.html
- *
- * GPL HEADER END
- */
-/*
- * Copyright (c) 2008, 2010, Oracle and/or its affiliates. All rights reserved.
- * Use is subject to license terms.
- *
- * Copyright (c) 2012, Intel Corporation.
- */
-/*
- * This file is part of Lustre, http://www.lustre.org/
- * Lustre is a trademark of Sun Microsystems, Inc.
- */
-
-#define DEBUG_SUBSYSTEM S_LNET
-#define LUSTRE_TRACEFILE_PRIVATE
-
-#include <linux/slab.h>
-#include <linux/mm.h>
-#include "tracefile.h"
-
-/* percents to share the total debug memory for each type */
-static unsigned int pages_factor[CFS_TCD_TYPE_MAX] = {
-	80,  /* 80% pages for CFS_TCD_TYPE_PROC */
-	10,  /* 10% pages for CFS_TCD_TYPE_SOFTIRQ */
-	10   /* 10% pages for CFS_TCD_TYPE_IRQ */
-};
-
-char *cfs_trace_console_buffers[NR_CPUS][CFS_TCD_TYPE_MAX];
-
-int cfs_tracefile_init_arch(void)
-{
-	int i;
-	int j;
-	struct cfs_trace_cpu_data *tcd;
-
-	/* initialize trace_data */
-	memset(cfs_trace_data, 0, sizeof(cfs_trace_data));
-	for (i = 0; i < CFS_TCD_TYPE_MAX; i++) {
-		cfs_trace_data[i] =
-			kmalloc_array(num_possible_cpus(),
-				      sizeof(union cfs_trace_data_union),
-				      GFP_KERNEL);
-		if (!cfs_trace_data[i])
-			goto out;
-	}
-
-	/* arch related info initialized */
-	cfs_tcd_for_each(tcd, i, j) {
-		spin_lock_init(&tcd->tcd_lock);
-		tcd->tcd_pages_factor = pages_factor[i];
-		tcd->tcd_type = i;
-		tcd->tcd_cpu = j;
-	}
-
-	for (i = 0; i < num_possible_cpus(); i++)
-		for (j = 0; j < 3; j++) {
-			cfs_trace_console_buffers[i][j] =
-				kmalloc(CFS_TRACE_CONSOLE_BUFFER_SIZE,
-					GFP_KERNEL);
-
-			if (!cfs_trace_console_buffers[i][j])
-				goto out;
-		}
-
-	return 0;
-
-out:
-	cfs_tracefile_fini_arch();
-	pr_err("lnet: Not enough memory\n");
-	return -ENOMEM;
-}
-
-void cfs_tracefile_fini_arch(void)
-{
-	int i;
-	int j;
-
-	for (i = 0; i < num_possible_cpus(); i++)
-		for (j = 0; j < 3; j++) {
-			kfree(cfs_trace_console_buffers[i][j]);
-			cfs_trace_console_buffers[i][j] = NULL;
-		}
-
-	for (i = 0; cfs_trace_data[i]; i++) {
-		kfree(cfs_trace_data[i]);
-		cfs_trace_data[i] = NULL;
-	}
-}
-
-enum cfs_trace_buf_type cfs_trace_buf_idx_get(void)
-{
-	if (in_irq())
-		return CFS_TCD_TYPE_IRQ;
-	if (in_softirq())
-		return CFS_TCD_TYPE_SOFTIRQ;
-	return CFS_TCD_TYPE_PROC;
-}
-
-void
-cfs_set_ptldebug_header(struct ptldebug_header *header,
-			struct libcfs_debug_msg_data *msgdata,
-			unsigned long stack)
-{
-	struct timespec64 ts;
-
-	ktime_get_real_ts64(&ts);
-
-	header->ph_subsys = msgdata->msg_subsys;
-	header->ph_mask = msgdata->msg_mask;
-	header->ph_cpu_id = smp_processor_id();
-	header->ph_type = cfs_trace_buf_idx_get();
-	/* y2038 safe since all user space treats this as unsigned, but
-	 * will overflow in 2106
-	 */
-	header->ph_sec = (u32)ts.tv_sec;
-	header->ph_usec = ts.tv_nsec / NSEC_PER_USEC;
-	header->ph_stack = stack;
-	header->ph_pid = current->pid;
-	header->ph_line_num = msgdata->msg_line;
-	header->ph_extern_pid = 0;
-}
-
-static char *
-dbghdr_to_err_string(struct ptldebug_header *hdr)
-{
-	switch (hdr->ph_subsys) {
-	case S_LND:
-	case S_LNET:
-		return "LNetError";
-	default:
-		return "LustreError";
-	}
-}
-
-static char *
-dbghdr_to_info_string(struct ptldebug_header *hdr)
-{
-	switch (hdr->ph_subsys) {
-	case S_LND:
-	case S_LNET:
-		return "LNet";
-	default:
-		return "Lustre";
-	}
-}
-
-void cfs_print_to_console(struct ptldebug_header *hdr, int mask,
-			  const char *buf, int len, const char *file,
-			  const char *fn)
-{
-	char *prefix = "Lustre", *ptype = NULL;
-
-	if (mask & D_EMERG) {
-		prefix = dbghdr_to_err_string(hdr);
-		ptype = KERN_EMERG;
-	} else if (mask & D_ERROR) {
-		prefix = dbghdr_to_err_string(hdr);
-		ptype = KERN_ERR;
-	} else if (mask & D_WARNING) {
-		prefix = dbghdr_to_info_string(hdr);
-		ptype = KERN_WARNING;
-	} else if (mask & (D_CONSOLE | libcfs_printk)) {
-		prefix = dbghdr_to_info_string(hdr);
-		ptype = KERN_INFO;
-	}
-
-	if (mask & D_CONSOLE) {
-		pr_info("%s%s: %.*s", ptype, prefix, len, buf);
-	} else {
-		pr_info("%s%s: %d:%d:(%s:%d:%s()) %.*s", ptype, prefix,
-			hdr->ph_pid, hdr->ph_extern_pid, file,
-			hdr->ph_line_num, fn, len, buf);
-	}
-}
diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.c b/drivers/staging/lustre/lnet/libcfs/tracefile.c
index 6d567a9..229bbb8 100644
--- a/drivers/staging/lustre/lnet/libcfs/tracefile.c
+++ b/drivers/staging/lustre/lnet/libcfs/tracefile.c
@@ -37,8 +37,6 @@
  */
 
 #define DEBUG_SUBSYSTEM S_LNET
-#define LUSTRE_TRACEFILE_PRIVATE
-#define pr_fmt(fmt) "Lustre: " fmt
 
 #include <linux/ratelimit.h>
 #include <linux/highmem.h>
@@ -49,9 +47,19 @@
 #include <linux/uaccess.h>
 #include "tracefile.h"
 
-/* XXX move things up to the top, comment */
+#define CFS_TRACE_CONSOLE_BUFFER_SIZE	1024
+#define TCD_MAX_TYPES			8
+
+enum cfs_trace_buf_type {
+	CFS_TCD_TYPE_PROC = 0,
+	CFS_TCD_TYPE_SOFTIRQ,
+	CFS_TCD_TYPE_IRQ,
+	CFS_TCD_TYPE_MAX
+};
+
 union cfs_trace_data_union (*cfs_trace_data[TCD_MAX_TYPES])[NR_CPUS] __cacheline_aligned;
 
+char *cfs_trace_console_buffers[NR_CPUS][CFS_TCD_TYPE_MAX];
 char cfs_tracefile[TRACEFILE_NAME_SIZE];
 long long cfs_tracefile_size = CFS_TRACEFILE_SIZE;
 static struct tracefiled_ctl trace_tctl;
@@ -145,11 +153,35 @@ void cfs_trace_unlock_tcd(struct cfs_trace_cpu_data *tcd, int walking)
 		spin_unlock(&tcd->tcd_lock);
 }
 
-#define cfs_tcd_for_each_type_lock(tcd, i, cpu)			   \
+#define cfs_tcd_for_each(tcd, i, j)					\
+	for (i = 0; cfs_trace_data[i]; i++)				\
+		for (j = 0, ((tcd) = &(*cfs_trace_data[i])[j].tcd);	\
+		     j < num_possible_cpus();				\
+		     j++, (tcd) = &(*cfs_trace_data[i])[j].tcd)
+
+
+#define cfs_tcd_for_each_type_lock(tcd, i, cpu)				\
 	for (i = 0; cfs_trace_data[i] &&				\
 	     (tcd = &(*cfs_trace_data[i])[cpu].tcd) &&			\
 	     cfs_trace_lock_tcd(tcd, 1); cfs_trace_unlock_tcd(tcd, 1), i++)
 
+enum cfs_trace_buf_type cfs_trace_buf_idx_get(void)
+{
+	if (in_irq())
+		return CFS_TCD_TYPE_IRQ;
+	if (in_softirq())
+		return CFS_TCD_TYPE_SOFTIRQ;
+	return CFS_TCD_TYPE_PROC;
+}
+
+static inline char *cfs_trace_get_console_buffer(void)
+{
+	unsigned int i = get_cpu();
+	unsigned int j = cfs_trace_buf_idx_get();
+
+	return cfs_trace_console_buffers[i][j];
+}
+
 static inline struct cfs_trace_cpu_data *
 cfs_trace_get_tcd(void)
 {
@@ -168,6 +200,77 @@ static inline void cfs_trace_put_tcd(struct cfs_trace_cpu_data *tcd)
 	put_cpu();
 }
 
+/* percents to share the total debug memory for each type */
+static unsigned int pages_factor[CFS_TCD_TYPE_MAX] = {
+	80,  /* 80% pages for CFS_TCD_TYPE_PROC */
+	10,  /* 10% pages for CFS_TCD_TYPE_SOFTIRQ */
+	10   /* 10% pages for CFS_TCD_TYPE_IRQ */
+};
+
+int cfs_tracefile_init_arch(void)
+{
+	struct cfs_trace_cpu_data *tcd;
+	int i;
+	int j;
+
+	/* initialize trace_data */
+	memset(cfs_trace_data, 0, sizeof(cfs_trace_data));
+	for (i = 0; i < CFS_TCD_TYPE_MAX; i++) {
+		cfs_trace_data[i] =
+			kmalloc_array(num_possible_cpus(),
+				      sizeof(union cfs_trace_data_union),
+				      GFP_KERNEL);
+		if (!cfs_trace_data[i])
+			goto out_trace_data;
+	}
+
+	/* arch related info initialized */
+	cfs_tcd_for_each(tcd, i, j) {
+		spin_lock_init(&tcd->tcd_lock);
+		tcd->tcd_pages_factor = pages_factor[i];
+		tcd->tcd_type = i;
+		tcd->tcd_cpu = j;
+	}
+
+	for (i = 0; i < num_possible_cpus(); i++)
+		for (j = 0; j < 3; j++) {
+			cfs_trace_console_buffers[i][j] =
+				kmalloc(CFS_TRACE_CONSOLE_BUFFER_SIZE,
+					GFP_KERNEL);
+
+			if (!cfs_trace_console_buffers[i][j])
+				goto out_buffers;
+		}
+
+	return 0;
+
+out_buffers:
+	for (i = 0; cfs_trace_data[i]; i++) {
+		kfree(cfs_trace_data[i]);
+		cfs_trace_data[i] = NULL;
+	}
+out_trace_data:
+	pr_err("lnet: Not enough memory\n");
+	return -ENOMEM;
+}
+
+void cfs_tracefile_fini_arch(void)
+{
+	int i;
+	int j;
+
+	for (i = 0; i < num_possible_cpus(); i++)
+		for (j = 0; j < 3; j++) {
+			kfree(cfs_trace_console_buffers[i][j]);
+			cfs_trace_console_buffers[i][j] = NULL;
+		}
+
+	for (i = 0; cfs_trace_data[i]; i++) {
+		kfree(cfs_trace_data[i]);
+		cfs_trace_data[i] = NULL;
+	}
+}
+
 static inline struct cfs_trace_page *
 cfs_tage_from_list(struct list_head *list)
 {
@@ -216,27 +319,6 @@ static void cfs_tage_to_tail(struct cfs_trace_page *tage,
 	list_move_tail(&tage->linkage, queue);
 }
 
-int cfs_trace_refill_stock(struct cfs_trace_cpu_data *tcd, gfp_t gfp,
-			   struct list_head *stock)
-{
-	int i;
-
-	/*
-	 * XXX nikita: do NOT call portals_debug_msg() (CDEBUG/ENTRY/EXIT)
-	 * from here: this will lead to infinite recursion.
-	 */
-
-	for (i = 0; i + tcd->tcd_cur_stock_pages < TCD_STOCK_PAGES ; ++i) {
-		struct cfs_trace_page *tage;
-
-		tage = cfs_tage_alloc(gfp);
-		if (!tage)
-			break;
-		list_add_tail(&tage->linkage, stock);
-	}
-	return i;
-}
-
 /* return a page that has 'len' bytes left at the end */
 static struct cfs_trace_page *
 cfs_trace_get_tage_try(struct cfs_trace_cpu_data *tcd, unsigned long len)
@@ -340,6 +422,57 @@ static struct cfs_trace_page *cfs_trace_get_tage(struct cfs_trace_cpu_data *tcd,
 	return tage;
 }
 
+
+
+static void cfs_set_ptldebug_header(struct ptldebug_header *header,
+				    struct libcfs_debug_msg_data *msgdata,
+				    unsigned long stack)
+{
+	struct timespec64 ts;
+
+	ktime_get_real_ts64(&ts);
+
+	header->ph_subsys = msgdata->msg_subsys;
+	header->ph_mask = msgdata->msg_mask;
+	header->ph_cpu_id = smp_processor_id();
+	header->ph_type = cfs_trace_buf_idx_get();
+	/* y2038 safe since all user space treats this as unsigned, but
+	 * will overflow in 2106
+	 */
+	header->ph_sec = (u32)ts.tv_sec;
+	header->ph_usec = ts.tv_nsec / NSEC_PER_USEC;
+	header->ph_stack = stack;
+	header->ph_pid = current->pid;
+	header->ph_line_num = msgdata->msg_line;
+	header->ph_extern_pid = 0;
+}
+
+static void cfs_print_to_console(struct ptldebug_header *hdr, int mask,
+				 const char *buf, int len, const char *file,
+				 const char *fn)
+{
+	char *prefix = "Lustre";
+
+	if (hdr->ph_subsys == S_LND || hdr->ph_subsys == S_LNET)
+		prefix = "LNet";
+
+	if (mask & D_EMERG) {
+		pr_emerg("%sError: %d:%d:(%s:%d:%s()) %.*s", prefix,
+			 hdr->ph_pid, hdr->ph_extern_pid, file,
+			 hdr->ph_line_num, fn, len, buf);
+	} else if (mask & D_ERROR) {
+		pr_err("%sError: %d:%d:(%s:%d:%s()) %.*s", prefix,
+		       hdr->ph_pid, hdr->ph_extern_pid, file,
+		       hdr->ph_line_num, fn, len, buf);
+	} else if (mask & D_WARNING) {
+		pr_warning("%s: %d:%d:(%s:%d:%s()) %.*s", prefix,
+			   hdr->ph_pid, hdr->ph_extern_pid, file,
+			   hdr->ph_line_num, fn, len, buf);
+	} else if (mask & (D_CONSOLE | libcfs_printk)) {
+		pr_info("%s: %.*s", prefix, len, buf);
+	}
+}
+
 int libcfs_debug_msg(struct libcfs_debug_msg_data *msgdata,
 		     const char *format, ...)
 {
@@ -531,6 +664,7 @@ int libcfs_debug_vmsg2(struct libcfs_debug_msg_data *msgdata,
 	} else {
 		string_buf = cfs_trace_get_console_buffer();
 
+		memset(string_buf, 0, CFS_TRACE_CONSOLE_BUFFER_SIZE);
 		needed = 0;
 		if (format1) {
 			va_copy(ap, args);
diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.h b/drivers/staging/lustre/lnet/libcfs/tracefile.h
index f49a9ba..4c79009 100644
--- a/drivers/staging/lustre/lnet/libcfs/tracefile.h
+++ b/drivers/staging/lustre/lnet/libcfs/tracefile.h
@@ -42,13 +42,6 @@
 #include <linux/smp.h>
 #include <linux/libcfs/libcfs.h>
 
-enum cfs_trace_buf_type {
-	CFS_TCD_TYPE_PROC = 0,
-	CFS_TCD_TYPE_SOFTIRQ,
-	CFS_TCD_TYPE_IRQ,
-	CFS_TCD_TYPE_MAX
-};
-
 #define TRACEFILE_NAME_SIZE 1024
 extern char cfs_tracefile[TRACEFILE_NAME_SIZE];
 extern long long cfs_tracefile_size;
@@ -60,9 +53,6 @@ enum cfs_trace_buf_type {
 
 void libcfs_run_debug_log_upcall(char *file);
 
-int  cfs_tracefile_init_arch(void);
-void cfs_tracefile_fini_arch(void);
-
 int cfs_tracefile_dump_all_pages(char *filename);
 void cfs_trace_debug_print(void);
 void cfs_trace_flush_pages(void);
@@ -92,22 +82,6 @@ int cfs_trace_copyout_string(char __user *usr_buffer, int usr_buffer_nob,
 #define TCD_STOCK_PAGES (TCD_MAX_PAGES)
 #define CFS_TRACEFILE_SIZE (500 << 20)
 
-#ifdef LUSTRE_TRACEFILE_PRIVATE
-
-/*
- * Private declare for tracefile
- */
-#define TCD_MAX_PAGES (5 << (20 - PAGE_SHIFT))
-#define TCD_STOCK_PAGES (TCD_MAX_PAGES)
-
-#define CFS_TRACEFILE_SIZE (500 << 20)
-
-/*
- * Size of a buffer for sprinting console messages if we can't get a page
- * from system
- */
-#define CFS_TRACE_CONSOLE_BUFFER_SIZE   1024
-
 union cfs_trace_data_union {
 	struct cfs_trace_cpu_data {
 		/*
@@ -184,58 +158,25 @@ int cfs_trace_copyout_string(char __user *usr_buffer, int usr_buffer_nob,
 	char __pad[L1_CACHE_ALIGN(sizeof(struct cfs_trace_cpu_data))];
 };
 
-#define TCD_MAX_TYPES      8
-extern union cfs_trace_data_union (*cfs_trace_data[TCD_MAX_TYPES])[NR_CPUS];
-
-#define cfs_tcd_for_each(tcd, i, j)				       \
-	for (i = 0; cfs_trace_data[i]; i++)				\
-		for (j = 0, ((tcd) = &(*cfs_trace_data[i])[j].tcd);	\
-		     j < num_possible_cpus();				 \
-		     j++, (tcd) = &(*cfs_trace_data[i])[j].tcd)
-
-void cfs_set_ptldebug_header(struct ptldebug_header *header,
-			     struct libcfs_debug_msg_data *m,
-			     unsigned long stack);
-void cfs_print_to_console(struct ptldebug_header *hdr, int mask,
-			  const char *buf, int len, const char *file,
-			  const char *fn);
-
-extern char *cfs_trace_console_buffers[NR_CPUS][CFS_TCD_TYPE_MAX];
-enum cfs_trace_buf_type cfs_trace_buf_idx_get(void);
-
-static inline char *
-cfs_trace_get_console_buffer(void)
-{
-	unsigned int i = get_cpu();
-	unsigned int j = cfs_trace_buf_idx_get();
-
-	return cfs_trace_console_buffers[i][j];
-}
-
-int cfs_trace_refill_stock(struct cfs_trace_cpu_data *tcd, gfp_t gfp,
-			   struct list_head *stock);
-
 void cfs_trace_assertion_failed(const char *str,
 				struct libcfs_debug_msg_data *m);
 
 /* ASSERTION that is safe to use within the debug system */
-#define __LASSERT(cond)						 \
-do {								    \
+#define __LASSERT(cond)							\
+do {									\
 	if (unlikely(!(cond))) {					\
 		LIBCFS_DEBUG_MSG_DATA_DECL(msgdata, D_EMERG, NULL);     \
 		cfs_trace_assertion_failed("ASSERTION("#cond") failed", \
-					   &msgdata);		   \
-	}							       \
+					   &msgdata);			\
+	}								\
 } while (0)
 
-#define __LASSERT_TAGE_INVARIANT(tage)				  \
-do {								    \
-	__LASSERT(tage);					\
-	__LASSERT(tage->page);				  \
-	__LASSERT(tage->used <= PAGE_SIZE);			 \
-	__LASSERT(page_count(tage->page) > 0);		      \
+#define __LASSERT_TAGE_INVARIANT(tage)			\
+do {							\
+	__LASSERT(tage);				\
+	__LASSERT(tage->page);				\
+	__LASSERT(tage->used <= PAGE_SIZE);		\
+	__LASSERT(page_count(tage->page) > 0);		\
 } while (0)
 
-#endif	/* LUSTRE_TRACEFILE_PRIVATE */
-
 #endif /* __LIBCFS_TRACEFILE_H__ */
-- 
1.8.3.1


From jsimmons at infradead.org  Mon Jun 25 21:42:52 2018
From: jsimmons at infradead.org (James Simmons)
Date: Mon, 25 Jun 2018 17:42:52 -0400
Subject: [lustre-devel] [PATCH v2 3/7] lustre: libcfs: move tcd locking
 across to tracefile.c
In-Reply-To: <1529962976-15048-1-git-send-email-jsimmons@infradead.org>
References: <1529962976-15048-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1529962976-15048-4-git-send-email-jsimmons@infradead.org>

From: NeilBrown <neilb at suse.com>

No need to have this in linux-tracefile.c

Signed-off-by: NeilBrown <neilb at suse.com>
---
 .../staging/lustre/lnet/libcfs/linux-tracefile.c   | 35 -------------
 drivers/staging/lustre/lnet/libcfs/tracefile.c     | 59 ++++++++++++++++++++++
 drivers/staging/lustre/lnet/libcfs/tracefile.h     | 28 ----------
 3 files changed, 59 insertions(+), 63 deletions(-)

diff --git a/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c b/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
index 64a5bc1..3af7722 100644
--- a/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
+++ b/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
@@ -116,41 +116,6 @@ enum cfs_trace_buf_type cfs_trace_buf_idx_get(void)
 	return CFS_TCD_TYPE_PROC;
 }
 
-/*
- * The walking argument indicates the locking comes from all tcd types
- * iterator and we must lock it and dissable local irqs to avoid deadlocks
- * with other interrupt locks that might be happening. See LU-1311
- * for details.
- */
-int cfs_trace_lock_tcd(struct cfs_trace_cpu_data *tcd, int walking)
-	__acquires(&tcd->tc_lock)
-{
-	__LASSERT(tcd->tcd_type < CFS_TCD_TYPE_MAX);
-	if (tcd->tcd_type == CFS_TCD_TYPE_IRQ)
-		spin_lock_irqsave(&tcd->tcd_lock, tcd->tcd_lock_flags);
-	else if (tcd->tcd_type == CFS_TCD_TYPE_SOFTIRQ)
-		spin_lock_bh(&tcd->tcd_lock);
-	else if (unlikely(walking))
-		spin_lock_irq(&tcd->tcd_lock);
-	else
-		spin_lock(&tcd->tcd_lock);
-	return 1;
-}
-
-void cfs_trace_unlock_tcd(struct cfs_trace_cpu_data *tcd, int walking)
-	__releases(&tcd->tcd_lock)
-{
-	__LASSERT(tcd->tcd_type < CFS_TCD_TYPE_MAX);
-	if (tcd->tcd_type == CFS_TCD_TYPE_IRQ)
-		spin_unlock_irqrestore(&tcd->tcd_lock, tcd->tcd_lock_flags);
-	else if (tcd->tcd_type == CFS_TCD_TYPE_SOFTIRQ)
-		spin_unlock_bh(&tcd->tcd_lock);
-	else if (unlikely(walking))
-		spin_unlock_irq(&tcd->tcd_lock);
-	else
-		spin_unlock(&tcd->tcd_lock);
-}
-
 void
 cfs_set_ptldebug_header(struct ptldebug_header *header,
 			struct libcfs_debug_msg_data *msgdata,
diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.c b/drivers/staging/lustre/lnet/libcfs/tracefile.c
index 72321ce..6d567a9 100644
--- a/drivers/staging/lustre/lnet/libcfs/tracefile.c
+++ b/drivers/staging/lustre/lnet/libcfs/tracefile.c
@@ -109,6 +109,65 @@ struct cfs_trace_page {
 static void put_pages_on_tcd_daemon_list(struct page_collection *pc,
 					 struct cfs_trace_cpu_data *tcd);
 
+/* trace file lock routines */
+/*
+ * The walking argument indicates the locking comes from all tcd types
+ * iterator and we must lock it and dissable local irqs to avoid deadlocks
+ * with other interrupt locks that might be happening. See LU-1311
+ * for details.
+ */
+int cfs_trace_lock_tcd(struct cfs_trace_cpu_data *tcd, int walking)
+	__acquires(&tcd->tc_lock)
+{
+	__LASSERT(tcd->tcd_type < CFS_TCD_TYPE_MAX);
+	if (tcd->tcd_type == CFS_TCD_TYPE_IRQ)
+		spin_lock_irqsave(&tcd->tcd_lock, tcd->tcd_lock_flags);
+	else if (tcd->tcd_type == CFS_TCD_TYPE_SOFTIRQ)
+		spin_lock_bh(&tcd->tcd_lock);
+	else if (unlikely(walking))
+		spin_lock_irq(&tcd->tcd_lock);
+	else
+		spin_lock(&tcd->tcd_lock);
+	return 1;
+}
+
+void cfs_trace_unlock_tcd(struct cfs_trace_cpu_data *tcd, int walking)
+	__releases(&tcd->tcd_lock)
+{
+	__LASSERT(tcd->tcd_type < CFS_TCD_TYPE_MAX);
+	if (tcd->tcd_type == CFS_TCD_TYPE_IRQ)
+		spin_unlock_irqrestore(&tcd->tcd_lock, tcd->tcd_lock_flags);
+	else if (tcd->tcd_type == CFS_TCD_TYPE_SOFTIRQ)
+		spin_unlock_bh(&tcd->tcd_lock);
+	else if (unlikely(walking))
+		spin_unlock_irq(&tcd->tcd_lock);
+	else
+		spin_unlock(&tcd->tcd_lock);
+}
+
+#define cfs_tcd_for_each_type_lock(tcd, i, cpu)			   \
+	for (i = 0; cfs_trace_data[i] &&				\
+	     (tcd = &(*cfs_trace_data[i])[cpu].tcd) &&			\
+	     cfs_trace_lock_tcd(tcd, 1); cfs_trace_unlock_tcd(tcd, 1), i++)
+
+static inline struct cfs_trace_cpu_data *
+cfs_trace_get_tcd(void)
+{
+	struct cfs_trace_cpu_data *tcd =
+		&(*cfs_trace_data[cfs_trace_buf_idx_get()])[get_cpu()].tcd;
+
+	cfs_trace_lock_tcd(tcd, 0);
+
+	return tcd;
+}
+
+static inline void cfs_trace_put_tcd(struct cfs_trace_cpu_data *tcd)
+{
+	cfs_trace_unlock_tcd(tcd, 0);
+
+	put_cpu();
+}
+
 static inline struct cfs_trace_page *
 cfs_tage_from_list(struct list_head *list)
 {
diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.h b/drivers/staging/lustre/lnet/libcfs/tracefile.h
index 9f6b73d..f49a9ba 100644
--- a/drivers/staging/lustre/lnet/libcfs/tracefile.h
+++ b/drivers/staging/lustre/lnet/libcfs/tracefile.h
@@ -49,8 +49,6 @@ enum cfs_trace_buf_type {
 	CFS_TCD_TYPE_MAX
 };
 
-/* trace file lock routines */
-
 #define TRACEFILE_NAME_SIZE 1024
 extern char cfs_tracefile[TRACEFILE_NAME_SIZE];
 extern long long cfs_tracefile_size;
@@ -195,11 +193,6 @@ int cfs_trace_copyout_string(char __user *usr_buffer, int usr_buffer_nob,
 		     j < num_possible_cpus();				 \
 		     j++, (tcd) = &(*cfs_trace_data[i])[j].tcd)
 
-#define cfs_tcd_for_each_type_lock(tcd, i, cpu)			   \
-	for (i = 0; cfs_trace_data[i] &&				\
-	     (tcd = &(*cfs_trace_data[i])[cpu].tcd) &&			\
-	     cfs_trace_lock_tcd(tcd, 1); cfs_trace_unlock_tcd(tcd, 1), i++)
-
 void cfs_set_ptldebug_header(struct ptldebug_header *header,
 			     struct libcfs_debug_msg_data *m,
 			     unsigned long stack);
@@ -207,9 +200,6 @@ void cfs_print_to_console(struct ptldebug_header *hdr, int mask,
 			  const char *buf, int len, const char *file,
 			  const char *fn);
 
-int cfs_trace_lock_tcd(struct cfs_trace_cpu_data *tcd, int walking);
-void cfs_trace_unlock_tcd(struct cfs_trace_cpu_data *tcd, int walking);
-
 extern char *cfs_trace_console_buffers[NR_CPUS][CFS_TCD_TYPE_MAX];
 enum cfs_trace_buf_type cfs_trace_buf_idx_get(void);
 
@@ -222,24 +212,6 @@ void cfs_print_to_console(struct ptldebug_header *hdr, int mask,
 	return cfs_trace_console_buffers[i][j];
 }
 
-static inline struct cfs_trace_cpu_data *
-cfs_trace_get_tcd(void)
-{
-	struct cfs_trace_cpu_data *tcd =
-		&(*cfs_trace_data[cfs_trace_buf_idx_get()])[get_cpu()].tcd;
-
-	cfs_trace_lock_tcd(tcd, 0);
-
-	return tcd;
-}
-
-static inline void cfs_trace_put_tcd(struct cfs_trace_cpu_data *tcd)
-{
-	cfs_trace_unlock_tcd(tcd, 0);
-
-	put_cpu();
-}
-
 int cfs_trace_refill_stock(struct cfs_trace_cpu_data *tcd, gfp_t gfp,
 			   struct list_head *stock);
 
-- 
1.8.3.1


From jsimmons at infradead.org  Mon Jun 25 21:42:55 2018
From: jsimmons at infradead.org (James Simmons)
Date: Mon, 25 Jun 2018 17:42:55 -0400
Subject: [lustre-devel] [PATCH v2 6/7] lustre: libcfs: renamed
 CFS_TCD_TYPE_MAX to CFS_TCD_TYPE_CNT
In-Reply-To: <1529962976-15048-1-git-send-email-jsimmons@infradead.org>
References: <1529962976-15048-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1529962976-15048-7-git-send-email-jsimmons@infradead.org>

From: NeilBrown <neilb at suse.com>

The possible TCD types are 0, 1, 2.
So the MAX is 2.
The count of the number of types is 3.

CFS_TCD_TYPE_MAX is 3 - obviously wrong.

So rename it to CFS_TCD_TYPE_CNT.

Also there are 2 places where "3" is used rather
than the macro - fix them.

Signed-off-by: NeilBrown <neilb at suse.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/lnet/libcfs/tracefile.c | 16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)

diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.c b/drivers/staging/lustre/lnet/libcfs/tracefile.c
index b370c56..e9d8b22 100644
--- a/drivers/staging/lustre/lnet/libcfs/tracefile.c
+++ b/drivers/staging/lustre/lnet/libcfs/tracefile.c
@@ -54,12 +54,12 @@ enum cfs_trace_buf_type {
 	CFS_TCD_TYPE_PROC = 0,
 	CFS_TCD_TYPE_SOFTIRQ,
 	CFS_TCD_TYPE_IRQ,
-	CFS_TCD_TYPE_MAX
+	CFS_TCD_TYPE_CNT
 };
 
 union cfs_trace_data_union (*cfs_trace_data[TCD_MAX_TYPES])[NR_CPUS] __cacheline_aligned;
 
-char *cfs_trace_console_buffers[NR_CPUS][CFS_TCD_TYPE_MAX];
+char *cfs_trace_console_buffers[NR_CPUS][CFS_TCD_TYPE_CNT];
 char cfs_tracefile[TRACEFILE_NAME_SIZE];
 long long cfs_tracefile_size = CFS_TRACEFILE_SIZE;
 static struct tracefiled_ctl trace_tctl;
@@ -127,7 +127,7 @@ static void put_pages_on_tcd_daemon_list(struct page_collection *pc,
 int cfs_trace_lock_tcd(struct cfs_trace_cpu_data *tcd, int walking)
 	__acquires(&tcd->tc_lock)
 {
-	__LASSERT(tcd->tcd_type < CFS_TCD_TYPE_MAX);
+	__LASSERT(tcd->tcd_type < CFS_TCD_TYPE_CNT);
 	if (tcd->tcd_type == CFS_TCD_TYPE_IRQ)
 		spin_lock_irqsave(&tcd->tcd_lock, tcd->tcd_lock_flags);
 	else if (tcd->tcd_type == CFS_TCD_TYPE_SOFTIRQ)
@@ -142,7 +142,7 @@ int cfs_trace_lock_tcd(struct cfs_trace_cpu_data *tcd, int walking)
 void cfs_trace_unlock_tcd(struct cfs_trace_cpu_data *tcd, int walking)
 	__releases(&tcd->tcd_lock)
 {
-	__LASSERT(tcd->tcd_type < CFS_TCD_TYPE_MAX);
+	__LASSERT(tcd->tcd_type < CFS_TCD_TYPE_CNT);
 	if (tcd->tcd_type == CFS_TCD_TYPE_IRQ)
 		spin_unlock_irqrestore(&tcd->tcd_lock, tcd->tcd_lock_flags);
 	else if (tcd->tcd_type == CFS_TCD_TYPE_SOFTIRQ)
@@ -1252,7 +1252,7 @@ void cfs_trace_stop_thread(void)
 }
 
 /* percents to share the total debug memory for each type */
-static unsigned int pages_factor[CFS_TCD_TYPE_MAX] = {
+static unsigned int pages_factor[CFS_TCD_TYPE_CNT] = {
 	80,  /* 80% pages for CFS_TCD_TYPE_PROC */
 	10,  /* 10% pages for CFS_TCD_TYPE_SOFTIRQ */
 	10   /* 10% pages for CFS_TCD_TYPE_IRQ */
@@ -1266,7 +1266,7 @@ int cfs_tracefile_init(int max_pages)
 
 	/* initialize trace_data */
 	memset(cfs_trace_data, 0, sizeof(cfs_trace_data));
-	for (i = 0; i < CFS_TCD_TYPE_MAX; i++) {
+	for (i = 0; i < CFS_TCD_TYPE_CNT; i++) {
 		cfs_trace_data[i] =
 			kmalloc_array(num_possible_cpus(),
 				      sizeof(union cfs_trace_data_union),
@@ -1295,7 +1295,7 @@ int cfs_tracefile_init(int max_pages)
 	}
 
 	for (i = 0; i < num_possible_cpus(); i++)
-		for (j = 0; j < 3; j++) {
+		for (j = 0; j < CFS_TCD_TYPE_CNT; j++) {
 			cfs_trace_console_buffers[i][j] =
 				kmalloc(CFS_TRACE_CONSOLE_BUFFER_SIZE,
 					GFP_KERNEL);
@@ -1355,7 +1355,7 @@ static void cfs_trace_cleanup(void)
 	trace_cleanup_on_all_cpus();
 
 	for (i = 0; i < num_possible_cpus(); i++)
-		for (j = 0; j < 3; j++) {
+		for (j = 0; j < CFS_TCD_TYPE_CNT; j++) {
 			kfree(cfs_trace_console_buffers[i][j]);
 			cfs_trace_console_buffers[i][j] = NULL;
 		}
-- 
1.8.3.1


From jsimmons at infradead.org  Mon Jun 25 21:42:56 2018
From: jsimmons at infradead.org (James Simmons)
Date: Mon, 25 Jun 2018 17:42:56 -0400
Subject: [lustre-devel] [PATCH v2 7/7] lustre: libcfs: discard TCD_MAX_TYPES
In-Reply-To: <1529962976-15048-1-git-send-email-jsimmons@infradead.org>
References: <1529962976-15048-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1529962976-15048-8-git-send-email-jsimmons@infradead.org>

From: NeilBrown <neilb at suse.com>

As well as CFS_TCD_TYPE_CNT we have TCD_MAX_TYPES which has a larger
value but a similar meaning.  Discard it and just use
CFS_TCD_TYPE_CNT.

Two places relied on the fact that TCD_MAX_TYPES was larger and so
there would be NULLs at the end of the array.  Change
them to check the array size properly.

Signed-off-by: NeilBrown <neilb at suse.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/lnet/libcfs/tracefile.c | 9 ++++-----
 1 file changed, 4 insertions(+), 5 deletions(-)

diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.c b/drivers/staging/lustre/lnet/libcfs/tracefile.c
index e9d8b22..47998e8 100644
--- a/drivers/staging/lustre/lnet/libcfs/tracefile.c
+++ b/drivers/staging/lustre/lnet/libcfs/tracefile.c
@@ -48,7 +48,6 @@
 #include "tracefile.h"
 
 #define CFS_TRACE_CONSOLE_BUFFER_SIZE	1024
-#define TCD_MAX_TYPES			8
 
 enum cfs_trace_buf_type {
 	CFS_TCD_TYPE_PROC = 0,
@@ -57,7 +56,7 @@ enum cfs_trace_buf_type {
 	CFS_TCD_TYPE_CNT
 };
 
-union cfs_trace_data_union (*cfs_trace_data[TCD_MAX_TYPES])[NR_CPUS] __cacheline_aligned;
+union cfs_trace_data_union (*cfs_trace_data[CFS_TCD_TYPE_CNT])[NR_CPUS] __cacheline_aligned;
 
 char *cfs_trace_console_buffers[NR_CPUS][CFS_TCD_TYPE_CNT];
 char cfs_tracefile[TRACEFILE_NAME_SIZE];
@@ -154,14 +153,14 @@ void cfs_trace_unlock_tcd(struct cfs_trace_cpu_data *tcd, int walking)
 }
 
 #define cfs_tcd_for_each(tcd, i, j)					\
-	for (i = 0; cfs_trace_data[i]; i++)				\
+	for (i = 0; i < CFS_TCD_TYPE_CNT && cfs_trace_data[i]; i++)	\
 		for (j = 0, ((tcd) = &(*cfs_trace_data[i])[j].tcd);	\
 		     j < num_possible_cpus();				\
 		     j++, (tcd) = &(*cfs_trace_data[i])[j].tcd)
 
 
 #define cfs_tcd_for_each_type_lock(tcd, i, cpu)				\
-	for (i = 0; cfs_trace_data[i] &&				\
+	for (i = 0;  i < CFS_TCD_TYPE_CNT && cfs_trace_data[i] &&	\
 	     (tcd = &(*cfs_trace_data[i])[cpu].tcd) &&			\
 	     cfs_trace_lock_tcd(tcd, 1); cfs_trace_unlock_tcd(tcd, 1), i++)
 
@@ -1360,7 +1359,7 @@ static void cfs_trace_cleanup(void)
 			cfs_trace_console_buffers[i][j] = NULL;
 		}
 
-	for (i = 0; cfs_trace_data[i]; i++) {
+	for (i = 0; i < CFS_TCD_TYPE_CNT && cfs_trace_data[i]; i++) {
 		kfree(cfs_trace_data[i]);
 		cfs_trace_data[i] = NULL;
 	}
-- 
1.8.3.1


From jsimmons at infradead.org  Mon Jun 25 21:42:50 2018
From: jsimmons at infradead.org (James Simmons)
Date: Mon, 25 Jun 2018 17:42:50 -0400
Subject: [lustre-devel] [PATCH v2 1/7] lustre: libcfs: move tracefile
 locking from linux-tracefile.c to tracefile.c
In-Reply-To: <1529962976-15048-1-git-send-email-jsimmons@infradead.org>
References: <1529962976-15048-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1529962976-15048-2-git-send-email-jsimmons@infradead.org>

From: NeilBrown <neilb at suse.com>

There is no value in keeping it separate.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 .../staging/lustre/lnet/libcfs/linux-tracefile.c   | 22 -------------------
 drivers/staging/lustre/lnet/libcfs/tracefile.c     | 25 +++++++++++-----------
 drivers/staging/lustre/lnet/libcfs/tracefile.h     |  5 -----
 3 files changed, 13 insertions(+), 39 deletions(-)

diff --git a/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c b/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
index 3471384..9e72220 100644
--- a/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
+++ b/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
@@ -47,8 +47,6 @@
 
 char *cfs_trace_console_buffers[NR_CPUS][CFS_TCD_TYPE_MAX];
 
-static DECLARE_RWSEM(cfs_tracefile_sem);
-
 int cfs_tracefile_init_arch(void)
 {
 	int i;
@@ -109,26 +107,6 @@ void cfs_tracefile_fini_arch(void)
 	}
 }
 
-void cfs_tracefile_read_lock(void)
-{
-	down_read(&cfs_tracefile_sem);
-}
-
-void cfs_tracefile_read_unlock(void)
-{
-	up_read(&cfs_tracefile_sem);
-}
-
-void cfs_tracefile_write_lock(void)
-{
-	down_write(&cfs_tracefile_sem);
-}
-
-void cfs_tracefile_write_unlock(void)
-{
-	up_write(&cfs_tracefile_sem);
-}
-
 enum cfs_trace_buf_type cfs_trace_buf_idx_get(void)
 {
 	if (in_irq())
diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.c b/drivers/staging/lustre/lnet/libcfs/tracefile.c
index 7ca562e..5f31933 100644
--- a/drivers/staging/lustre/lnet/libcfs/tracefile.c
+++ b/drivers/staging/lustre/lnet/libcfs/tracefile.c
@@ -59,6 +59,7 @@
 static int thread_running;
 
 static atomic_t cfs_tage_allocated = ATOMIC_INIT(0);
+static DECLARE_RWSEM(cfs_tracefile_sem);
 
 struct page_collection {
 	struct list_head	pc_pages;
@@ -711,7 +712,7 @@ int cfs_tracefile_dump_all_pages(char *filename)
 	mm_segment_t __oldfs;
 	int rc;
 
-	cfs_tracefile_write_lock();
+	down_write(&cfs_tracefile_sem);
 
 	filp = filp_open(filename, O_CREAT | O_EXCL | O_WRONLY | O_LARGEFILE,
 			 0600);
@@ -759,7 +760,7 @@ int cfs_tracefile_dump_all_pages(char *filename)
 close:
 	filp_close(filp, NULL);
 out:
-	cfs_tracefile_write_unlock();
+	up_write(&cfs_tracefile_sem);
 	return rc;
 }
 
@@ -873,12 +874,12 @@ int cfs_trace_daemon_command(char *str)
 {
 	int rc = 0;
 
-	cfs_tracefile_write_lock();
+	down_write(&cfs_tracefile_sem);
 
 	if (!strcmp(str, "stop")) {
-		cfs_tracefile_write_unlock();
+		up_write(&cfs_tracefile_sem);
 		cfs_trace_stop_thread();
-		cfs_tracefile_write_lock();
+		down_write(&cfs_tracefile_sem);
 		memset(cfs_tracefile, 0, sizeof(cfs_tracefile));
 
 	} else if (!strncmp(str, "size=", 5)) {
@@ -905,7 +906,7 @@ int cfs_trace_daemon_command(char *str)
 		cfs_trace_start_thread();
 	}
 
-	cfs_tracefile_write_unlock();
+	up_write(&cfs_tracefile_sem);
 	return rc;
 }
 
@@ -950,12 +951,12 @@ int cfs_trace_set_debug_mb(int mb)
 	mb /= num_possible_cpus();
 	pages = mb << (20 - PAGE_SHIFT);
 
-	cfs_tracefile_write_lock();
+	down_write(&cfs_tracefile_sem);
 
 	cfs_tcd_for_each(tcd, i, j)
 		tcd->tcd_max_pages = (pages * tcd->tcd_pages_factor) / 100;
 
-	cfs_tracefile_write_unlock();
+	up_write(&cfs_tracefile_sem);
 
 	return 0;
 }
@@ -967,12 +968,12 @@ int cfs_trace_get_debug_mb(void)
 	struct cfs_trace_cpu_data *tcd;
 	int total_pages = 0;
 
-	cfs_tracefile_read_lock();
+	down_read(&cfs_tracefile_sem);
 
 	cfs_tcd_for_each(tcd, i, j)
 		total_pages += tcd->tcd_max_pages;
 
-	cfs_tracefile_read_unlock();
+	up_read(&cfs_tracefile_sem);
 
 	return (total_pages >> (20 - PAGE_SHIFT)) + 1;
 }
@@ -1002,7 +1003,7 @@ static int tracefiled(void *arg)
 			goto end_loop;
 
 		filp = NULL;
-		cfs_tracefile_read_lock();
+		down_read(&cfs_tracefile_sem);
 		if (cfs_tracefile[0]) {
 			filp = filp_open(cfs_tracefile,
 					 O_CREAT | O_RDWR | O_LARGEFILE,
@@ -1014,7 +1015,7 @@ static int tracefiled(void *arg)
 					rc);
 			}
 		}
-		cfs_tracefile_read_unlock();
+		up_read(&cfs_tracefile_sem);
 		if (!filp) {
 			put_pages_on_daemon_list(&pc);
 			__LASSERT(list_empty(&pc.pc_pages));
diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.h b/drivers/staging/lustre/lnet/libcfs/tracefile.h
index 0608240..9f6b73d 100644
--- a/drivers/staging/lustre/lnet/libcfs/tracefile.h
+++ b/drivers/staging/lustre/lnet/libcfs/tracefile.h
@@ -65,11 +65,6 @@ enum cfs_trace_buf_type {
 int  cfs_tracefile_init_arch(void);
 void cfs_tracefile_fini_arch(void);
 
-void cfs_tracefile_read_lock(void);
-void cfs_tracefile_read_unlock(void);
-void cfs_tracefile_write_lock(void);
-void cfs_tracefile_write_unlock(void);
-
 int cfs_tracefile_dump_all_pages(char *filename);
 void cfs_trace_debug_print(void);
 void cfs_trace_flush_pages(void);
-- 
1.8.3.1


From jsimmons at infradead.org  Mon Jun 25 21:52:55 2018
From: jsimmons at infradead.org (James Simmons)
Date: Mon, 25 Jun 2018 22:52:55 +0100 (BST)
Subject: [lustre-devel] [PATCH 21/24] lustre: move remainder of
 linux-tracefile.c to tracefile.c
In-Reply-To: <87zhzjahsd.fsf@notabene.neil.brown.name>
References: <152904663333.10587.10934053155404014785.stgit@noble>
 <152904669084.10587.6920959206498193596.stgit@noble>
 <alpine.LFD.2.21.1806210236200.3886@casper.infradead.org>
 <87y3f7bj9w.fsf@notabene.neil.brown.name>
 <alpine.LFD.2.21.1806242125460.28698@casper.infradead.org>
 <87zhzjahsd.fsf@notabene.neil.brown.name>
Message-ID: <alpine.LFD.2.21.1806252243540.31196@casper.infradead.org>


> >> On Thu, Jun 21 2018, James Simmons wrote:
> >> 
> >> >> It's good to keep related code together.
> >> >> 
> >> >> Signed-off-by: NeilBrown <neilb at suse.com>
> >> >
> >> > Nak. For some reason this corrupts my dmesg output. Its really strange and
> >> > I haven't figured it out yet.
> >> 
> >> This patch cannot possibly be the cause of any such behavioural change.
> >> It purely moves code from one file to another, it doesn't change the
> >> code at all.
> >
> > In a way you are correct. What this patch does is expose how badly messed
> > up cfs_print_to_console() is. Originally it was a printk() call but some
> > newbie changed it incorrectly into a pr_info() for all cases. This is 
> > totally incorrect but it landed anyways :-( For some bizarre reason it
> > works even with it being completely wrong.
> 
> I see the problem with cfs_print_to_console().  It chooses a message
> type (KERN_EMERG, KERN_ERR, KERN_WARNING, KERN_INFO) then uses pr_info()
> to print that, which adds another KERN_INFO.
> However vprintk_emit() calls printk_get_level() repeatedly while it
> succeeds, so the extra KERNE_INFO is ignored.
> 
> So while the code is strange and should use printk(), I don't see
> how it could mess up your dmesg output.
> 
> >                                              I did fix this patch up 
> > and redid the later patches due the changes from the fix up.  Is it 
> > okay it I push the newer version of these patches with my changes?
> 
> I'm not really sure what you are asking here, but the answer is probably 
> "yes, it is okay".

Pushed the updated patches. Sorry I wasn't clear on what I was asking.
I was weird stuff like Lustre: LNet6: .....
Well in any case the pushed patch set cleans all that mess up.

From jsimmons at infradead.org  Mon Jun 25 22:51:38 2018
From: jsimmons at infradead.org (James Simmons)
Date: Mon, 25 Jun 2018 23:51:38 +0100 (BST)
Subject: [lustre-devel] [PATCH v3 23/26] staging: lustre: ptlrpc: use
 current CPU instead of hardcoded 0
In-Reply-To: <87a7rj8ub9.fsf@notabene.neil.brown.name>
References: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
 <1529875250-11531-24-git-send-email-jsimmons@infradead.org>
 <87a7rj8ub9.fsf@notabene.neil.brown.name>
Message-ID: <alpine.LFD.2.21.1806252350310.31196@casper.infradead.org>


> > From: Dmitry Eremin <dmitry.eremin at intel.com>
> >
> > fix crash if CPU 0 disabled.
> >
> > Signed-off-by: Dmitry Eremin <dmitry.eremin at intel.com>
> > WC-bug-id: https://jira.whamcloud.com/browse/LU-8710
> > Reviewed-on: https://review.whamcloud.com/23305
> > Reviewed-by: Doug Oucharek <dougso at me.com>
> > Reviewed-by: Andreas Dilger <adilger at whamcloud.com>
> > Signed-off-by: James Simmons <jsimmons at infradead.org>
> > ---
> >  drivers/staging/lustre/lustre/ptlrpc/service.c | 11 ++++++-----
> >  1 file changed, 6 insertions(+), 5 deletions(-)
> >
> > diff --git a/drivers/staging/lustre/lustre/ptlrpc/service.c b/drivers/staging/lustre/lustre/ptlrpc/service.c
> > index 3fd8c74..8e74a45 100644
> > --- a/drivers/staging/lustre/lustre/ptlrpc/service.c
> > +++ b/drivers/staging/lustre/lustre/ptlrpc/service.c
> > @@ -421,7 +421,7 @@ static void ptlrpc_at_timer(struct timer_list *t)
> >  		 * there are.
> >  		 */
> >  		/* weight is # of HTs */
> > -		if (cpumask_weight(topology_sibling_cpumask(0)) > 1) {
> > +		if (cpumask_weight(topology_sibling_cpumask(smp_processor_id())) > 1) {
> 
> This pops a warning for me:
> [ 1877.516799] BUG: using smp_processor_id() in preemptible [00000000] code: mount.lustre/14077
> 
> I'll change it to disable preemption, both here and below.

For .config I have:

# CONFIG_PREEMPT_NONE is not set
CONFIG_PREEMPT_VOLUNTARY=y
# CONFIG_PREEMPT is not set

What does yours look like? Strange no one has ever reported an error 
before. Thanks for finding this!!!

> 
> >  			/* depress thread factor for hyper-thread */
> >  			factor = factor - (factor >> 1) + (factor >> 3);
> >  		}
> > @@ -2221,15 +2221,16 @@ static int ptlrpc_hr_main(void *arg)
> >  	struct ptlrpc_hr_thread	*hrt = arg;
> >  	struct ptlrpc_hr_partition *hrp = hrt->hrt_partition;
> >  	LIST_HEAD(replies);
> > -	char threadname[20];
> >  	int rc;
> >  
> > -	snprintf(threadname, sizeof(threadname), "ptlrpc_hr%02d_%03d",
> > -		 hrp->hrp_cpt, hrt->hrt_id);
> >  	unshare_fs_struct();
> >  
> >  	rc = cfs_cpt_bind(ptlrpc_hr.hr_cpt_table, hrp->hrp_cpt);
> >  	if (rc != 0) {
> > +		char threadname[20];
> > +
> > +		snprintf(threadname, sizeof(threadname), "ptlrpc_hr%02d_%03d",
> > +			 hrp->hrp_cpt, hrt->hrt_id);
> >  		CWARN("Failed to bind %s on CPT %d of CPT table %p: rc = %d\n",
> >  		      threadname, hrp->hrp_cpt, ptlrpc_hr.hr_cpt_table, rc);
> >  	}
> > @@ -2528,7 +2529,7 @@ int ptlrpc_hr_init(void)
> >  
> >  	init_waitqueue_head(&ptlrpc_hr.hr_waitq);
> >  
> > -	weight = cpumask_weight(topology_sibling_cpumask(0));
> > +	weight = cpumask_weight(topology_sibling_cpumask(smp_processor_id()));
> >  
> >  	cfs_percpt_for_each(hrp, i, ptlrpc_hr.hr_partitions) {
> >  		hrp->hrp_cpt = i;
> > -- 
> > 1.8.3.1
> 

From jsimmons at infradead.org  Mon Jun 25 22:57:25 2018
From: jsimmons at infradead.org (James Simmons)
Date: Mon, 25 Jun 2018 23:57:25 +0100 (BST)
Subject: [lustre-devel] [PATCH v3 19/26] staging: lustre: libcfs: report
 NUMA node instead of just node
In-Reply-To: <87vaa78ybu.fsf@notabene.neil.brown.name>
References: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
 <1529875250-11531-20-git-send-email-jsimmons@infradead.org>
 <87y3f38ye6.fsf@notabene.neil.brown.name>
 <87vaa78ybu.fsf@notabene.neil.brown.name>
Message-ID: <alpine.LFD.2.21.1806252354180.31196@casper.infradead.org>


> > On Sun, Jun 24 2018, James Simmons wrote:
> >
> >> From: Dmitry Eremin <dmitry.eremin at intel.com>
> >>
> >> Reporting "HW nodes" is too generic. It really is reporting
> >> "HW NUMA nodes". Update the debug message.
> >
> > I'm not happy with this patch description.....
> >
> >>
> >> Signed-off-by: Dmitry Eremin <dmitry.eremin at intel.com>
> >> WC-bug-id: https://jira.whamcloud.com/browse/LU-8703
> >> Reviewed-on: https://review.whamcloud.com/23306
> >> Reviewed-by: James Simmons <uja.ornl at yahoo.com>
> >> Reviewed-by: Andreas Dilger <adilger at whamcloud.com>
> >> Reviewed-by: Patrick Farrell <paf at cray.com>
> >> Reviewed-by: Olaf Weber <olaf.weber at hpe.com>
> >> Reviewed-by: Oleg Drokin <green at whamcloud.com>
> >> Signed-off-by: James Simmons <jsimmons at infradead.org>
> >> ---
> >>  drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h | 2 ++
> >>  drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c          | 2 +-
> >>  drivers/staging/lustre/lnet/lnet/lib-msg.c               | 2 ++
> >>  3 files changed, 5 insertions(+), 1 deletion(-)
> >>
> >> diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h
> >> index 2bb2140..29c5071 100644
> >> --- a/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h
> >> +++ b/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h
> >> @@ -90,6 +90,8 @@ struct cfs_cpu_partition {
> >>  	unsigned int			*cpt_distance;
> >>  	/* spread rotor for NUMA allocator */
> >>  	int				cpt_spread_rotor;
> >> +	/* NUMA node if cpt_nodemask is empty */
> >> +	int				cpt_node;
> >>  };
> >
> > It doesn't give any reason why this (unused) field was added.
> > So I've removed it.
> 
> Ahhhh. this was meant to be in the previous patch too.
> I've moved it.

This is stupid NFS. So my test nodes are NFS root which will
contain the source tree. The test nodes don't have access to
the outside world so from another system I do my pulling and
pushing. What happens is that patches applied will not show
up right away before I push. By the time the next patch is
applied it shows up. Sorry about that.


From jsimmons at infradead.org  Tue Jun 26 00:14:30 2018
From: jsimmons at infradead.org (James Simmons)
Date: Tue, 26 Jun 2018 01:14:30 +0100 (BST)
Subject: [lustre-devel] [PATCH 1/3] lustre: use
 generic_error_remove_page()
In-Reply-To: <87lgb3ag3k.fsf@notabene.neil.brown.name>
References: <152904663333.10587.10934053155404014785.stgit@noble>
 <152904669032.10587.11474850109829193046.stgit@noble>
 <alpine.LFD.2.21.1806160115430.15679@casper.infradead.org>
 <87y3ffft80.fsf@notabene.neil.brown.name>
 <61C2F92E-8F88-49C6-83FB-F3013289265C@linuxhacker.ru>
 <877emwg8ao.fsf@notabene.neil.brown.name>
 <87lgb3ag3k.fsf@notabene.neil.brown.name>
Message-ID: <alpine.LFD.2.21.1806260113420.31196@casper.infradead.org>


> lustre's internal ll_invalidate_page() is behaviourally identical to
> generic_error_remove_page().
> In the case of lustre it isn't a memory hardware error that requires
> the page being invalidated, it is the loss of a lock, which will like
> result in the data changing on the server.
> In either case, we don't want the page to be accessed any more, so the
> same removal is appropriate.

Reviewed-by: James Simmons <jsimmons at infradead.org>
 
> Signed-off-by: NeilBrown <neilb at suse.com>
> ---
> 
> I've replaced
>  [PATCH 08/24] lustre: use truncate_inode_page in place of truncate_complete_page
> with 3 patches, this and the next two.
> 
> Thanks,
> NeilBrown
> 
> 
>  drivers/staging/lustre/lustre/llite/llite_internal.h | 17 -----------------
>  drivers/staging/lustre/lustre/llite/vvp_io.c         |  2 +-
>  drivers/staging/lustre/lustre/llite/vvp_page.c       |  2 +-
>  3 files changed, 2 insertions(+), 19 deletions(-)
> 
> diff --git a/drivers/staging/lustre/lustre/llite/llite_internal.h b/drivers/staging/lustre/lustre/llite/llite_internal.h
> index c08a6e14b6d7..22dcabf6de0f 100644
> --- a/drivers/staging/lustre/lustre/llite/llite_internal.h
> +++ b/drivers/staging/lustre/lustre/llite/llite_internal.h
> @@ -928,23 +928,6 @@ void policy_from_vma(union ldlm_policy_data *policy, struct vm_area_struct *vma,
>  struct vm_area_struct *our_vma(struct mm_struct *mm, unsigned long addr,
>  			       size_t count);
>  
> -static inline void ll_invalidate_page(struct page *vmpage)
> -{
> -	struct address_space *mapping = vmpage->mapping;
> -	loff_t offset = vmpage->index << PAGE_SHIFT;
> -
> -	LASSERT(PageLocked(vmpage));
> -	if (!mapping)
> -		return;
> -
> -	/*
> -	 * truncate_complete_page() calls
> -	 * a_ops->invalidatepage()->cl_page_delete()->vvp_page_delete().
> -	 */
> -	ll_teardown_mmaps(mapping, offset, offset + PAGE_SIZE);
> -	truncate_complete_page(mapping, vmpage);
> -}
> -
>  #define    ll_s2sbi(sb)	(s2lsi(sb)->lsi_llsbi)
>  
>  /* don't need an addref as the sb_info should be holding one */
> diff --git a/drivers/staging/lustre/lustre/llite/vvp_io.c b/drivers/staging/lustre/lustre/llite/vvp_io.c
> index e7a4778e02e4..5a67955974ad 100644
> --- a/drivers/staging/lustre/lustre/llite/vvp_io.c
> +++ b/drivers/staging/lustre/lustre/llite/vvp_io.c
> @@ -1098,7 +1098,7 @@ static int vvp_io_fault_start(const struct lu_env *env,
>  	LASSERT(PageLocked(vmpage));
>  
>  	if (OBD_FAIL_CHECK(OBD_FAIL_LLITE_FAULT_TRUNC_RACE))
> -		ll_invalidate_page(vmpage);
> +		generic_error_remove_page(vmpage->mapping, vmpage);
>  
>  	size = i_size_read(inode);
>  	/* Though we have already held a cl_lock upon this page, but
> diff --git a/drivers/staging/lustre/lustre/llite/vvp_page.c b/drivers/staging/lustre/lustre/llite/vvp_page.c
> index 6eb0565ddc22..dcc4d8faa0cd 100644
> --- a/drivers/staging/lustre/lustre/llite/vvp_page.c
> +++ b/drivers/staging/lustre/lustre/llite/vvp_page.c
> @@ -147,7 +147,7 @@ static void vvp_page_discard(const struct lu_env *env,
>  	if (vpg->vpg_defer_uptodate && !vpg->vpg_ra_used)
>  		ll_ra_stats_inc(vmpage->mapping->host, RA_STAT_DISCARDED);
>  
> -	ll_invalidate_page(vmpage);
> +	generic_error_remove_page(vmpage->mapping, vmpage);
>  }
>  
>  static void vvp_page_delete(const struct lu_env *env,
> -- 
> 2.14.0.rc0.dirty
> 
> 

From jsimmons at infradead.org  Tue Jun 26 00:18:55 2018
From: jsimmons at infradead.org (James Simmons)
Date: Tue, 26 Jun 2018 01:18:55 +0100 (BST)
Subject: [lustre-devel] [PATCH 2/3] lustre: use delete_from_page_cache()
 for directory pages.
In-Reply-To: <87in67ag28.fsf@notabene.neil.brown.name>
References: <152904663333.10587.10934053155404014785.stgit@noble>
 <152904669032.10587.11474850109829193046.stgit@noble>
 <alpine.LFD.2.21.1806160115430.15679@casper.infradead.org>
 <87y3ffft80.fsf@notabene.neil.brown.name>
 <61C2F92E-8F88-49C6-83FB-F3013289265C@linuxhacker.ru>
 <877emwg8ao.fsf@notabene.neil.brown.name>
 <87lgb3ag3k.fsf@notabene.neil.brown.name>
 <87in67ag28.fsf@notabene.neil.brown.name>
Message-ID: <alpine.LFD.2.21.1806260118420.31196@casper.infradead.org>


> lustre sometimes uses the internal function truncate_complete_page()
> to remove a page of a directory.
> Must of what this function does, does not apply to directory pages
> as there is no invalidatepage function, and at these times, the
> page is not dirty.
> The only useful part of the function is delete_from_page_cache(),
> so just call that directly.
> 

Reviewed-by: James Simmons <jsimmons at infradead.org>

> Signed-off-by: NeilBrown <neilb at suse.com>
> ---
>  drivers/staging/lustre/lustre/llite/dir.c       | 2 +-
>  drivers/staging/lustre/lustre/mdc/mdc_request.c | 2 +-
>  2 files changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/staging/lustre/lustre/llite/dir.c b/drivers/staging/lustre/lustre/llite/dir.c
> index 688dddf3ca47..52a8ecc2e2e0 100644
> --- a/drivers/staging/lustre/lustre/llite/dir.c
> +++ b/drivers/staging/lustre/lustre/llite/dir.c
> @@ -168,7 +168,7 @@ void ll_release_page(struct inode *inode, struct page *page, bool remove)
>  	if (remove) {
>  		lock_page(page);
>  		if (likely(page->mapping))
> -			truncate_complete_page(page->mapping, page);
> +			delete_from_page_cache(page);
>  		unlock_page(page);
>  	}
>  	put_page(page);
> diff --git a/drivers/staging/lustre/lustre/mdc/mdc_request.c b/drivers/staging/lustre/lustre/mdc/mdc_request.c
> index cff31cb0a9ac..827ed0ca0987 100644
> --- a/drivers/staging/lustre/lustre/mdc/mdc_request.c
> +++ b/drivers/staging/lustre/lustre/mdc/mdc_request.c
> @@ -927,7 +927,7 @@ static void mdc_release_page(struct page *page, int remove)
>  	if (remove) {
>  		lock_page(page);
>  		if (likely(page->mapping))
> -			truncate_complete_page(page->mapping, page);
> +			delete_from_page_cache(page);
>  		unlock_page(page);
>  	}
>  	put_page(page);
> -- 
> 2.14.0.rc0.dirty
> 
> 

From jsimmons at infradead.org  Tue Jun 26 00:19:21 2018
From: jsimmons at infradead.org (James Simmons)
Date: Tue, 26 Jun 2018 01:19:21 +0100 (BST)
Subject: [lustre-devel] [PATCH 3/3] lustre: discard
 truncate_complete_page()
In-Reply-To: <87fu1bag1d.fsf@notabene.neil.brown.name>
References: <152904663333.10587.10934053155404014785.stgit@noble>
 <152904669032.10587.11474850109829193046.stgit@noble>
 <alpine.LFD.2.21.1806160115430.15679@casper.infradead.org>
 <87y3ffft80.fsf@notabene.neil.brown.name>
 <61C2F92E-8F88-49C6-83FB-F3013289265C@linuxhacker.ru>
 <877emwg8ao.fsf@notabene.neil.brown.name>
 <87lgb3ag3k.fsf@notabene.neil.brown.name>
 <87in67ag28.fsf@notabene.neil.brown.name>
 <87fu1bag1d.fsf@notabene.neil.brown.name>
Message-ID: <alpine.LFD.2.21.1806260119100.31196@casper.infradead.org>


> This function is no longer used, so remove it.

Reviewed-by: James Simmons <jsimmons at infradead.org>
 
> Signed-off-by: NeilBrown <neilb at suse.com>
> ---
>  .../lustre/lustre/include/lustre_patchless_compat.h      | 16 ----------------
>  1 file changed, 16 deletions(-)
> 
> diff --git a/drivers/staging/lustre/lustre/include/lustre_patchless_compat.h b/drivers/staging/lustre/lustre/include/lustre_patchless_compat.h
> index 298476ea7557..dca1e3dbd183 100644
> --- a/drivers/staging/lustre/lustre/include/lustre_patchless_compat.h
> +++ b/drivers/staging/lustre/lustre/include/lustre_patchless_compat.h
> @@ -41,22 +41,6 @@
>  #include <linux/hash.h>
>  #include <linux/pagemap.h>
>  
> -#define ll_delete_from_page_cache(page) delete_from_page_cache(page)
> -
> -static inline void
> -truncate_complete_page(struct address_space *mapping, struct page *page)
> -{
> -	if (page->mapping != mapping)
> -		return;
> -
> -	if (PagePrivate(page))
> -		page->mapping->a_ops->invalidatepage(page, 0, PAGE_SIZE);
> -
> -	cancel_dirty_page(page);
> -	ClearPageMappedToDisk(page);
> -	ll_delete_from_page_cache(page);
> -}
> -
>  #ifndef ATTR_CTIME_SET
>  /*
>   * set ATTR_CTIME_SET to a high value to avoid any risk of collision with other
> -- 
> 2.14.0.rc0.dirty
> 
> 

From jsimmons at infradead.org  Tue Jun 26 00:26:31 2018
From: jsimmons at infradead.org (James Simmons)
Date: Tue, 26 Jun 2018 01:26:31 +0100 (BST)
Subject: [lustre-devel] [PATCH 1/3] lustre: use
 generic_error_remove_page()
In-Reply-To: <87d0wf8w7h.fsf@notabene.neil.brown.name>
References: <152904663333.10587.10934053155404014785.stgit@noble>
 <152904669032.10587.11474850109829193046.stgit@noble>
 <alpine.LFD.2.21.1806160115430.15679@casper.infradead.org>
 <87y3ffft80.fsf@notabene.neil.brown.name>
 <61C2F92E-8F88-49C6-83FB-F3013289265C@linuxhacker.ru>
 <877emwg8ao.fsf@notabene.neil.brown.name>
 <87lgb3ag3k.fsf@notabene.neil.brown.name>
 <EAC0696A-9C7F-419A-A360-C352537B1DA2@linuxhacker.ru>
 <87d0wf8w7h.fsf@notabene.neil.brown.name>
Message-ID: <alpine.LFD.2.21.1806260120050.31196@casper.infradead.org>


> >> On Jun 24, 2018, at 8:02 PM, NeilBrown <neilb at suse.com> wrote:
> >> 
> >> 
> >> lustre's internal ll_invalidate_page() is behaviourally identical to
> >> generic_error_remove_page().
> >> In the case of lustre it isn't a memory hardware error that requires
> >> the page being invalidated, it is the loss of a lock, which will like
> >> result in the data changing on the server.
> >> In either case, we don't want the page to be accessed any more, so the
> >> same removal is appropriate.
> >> 
> >> Signed-off-by: NeilBrown <neilb at suse.com>
> >> ---
> >> 
> >> I've replaced
> >> [PATCH 08/24] lustre: use truncate_inode_page in place of truncate_complete_page
> >> with 3 patches, this and the next two.
> >
> > This looks reasonable. Are you running any tests on this?
> 
> Just the sanity tests on a 4-node vcluster.  They haven't caused any
> noticeable regressions.
> 

I also have tested this patch set using a single client node with
a stand alone MGS server, 2 MDS servers with 2 MDTs each, and a
single OSS server with 2 OST.

> >
> > Acked-by: Oleg Drokin <green at linuxhacker.ru>
> 
> Thanks,
> NeilBrown
> 
> >
> >> 
> >> Thanks,
> >> NeilBrown
> >> 
> >> 
> >> drivers/staging/lustre/lustre/llite/llite_internal.h | 17 -----------------
> >> drivers/staging/lustre/lustre/llite/vvp_io.c         |  2 +-
> >> drivers/staging/lustre/lustre/llite/vvp_page.c       |  2 +-
> >> 3 files changed, 2 insertions(+), 19 deletions(-)
> >> 
> >> diff --git a/drivers/staging/lustre/lustre/llite/llite_internal.h b/drivers/staging/lustre/lustre/llite/llite_internal.h
> >> index c08a6e14b6d7..22dcabf6de0f 100644
> >> --- a/drivers/staging/lustre/lustre/llite/llite_internal.h
> >> +++ b/drivers/staging/lustre/lustre/llite/llite_internal.h
> >> @@ -928,23 +928,6 @@ void policy_from_vma(union ldlm_policy_data *policy, struct vm_area_struct *vma,
> >> struct vm_area_struct *our_vma(struct mm_struct *mm, unsigned long addr,
> >> 			       size_t count);
> >> 
> >> -static inline void ll_invalidate_page(struct page *vmpage)
> >> -{
> >> -	struct address_space *mapping = vmpage->mapping;
> >> -	loff_t offset = vmpage->index << PAGE_SHIFT;
> >> -
> >> -	LASSERT(PageLocked(vmpage));
> >> -	if (!mapping)
> >> -		return;
> >> -
> >> -	/*
> >> -	 * truncate_complete_page() calls
> >> -	 * a_ops->invalidatepage()->cl_page_delete()->vvp_page_delete().
> >> -	 */
> >> -	ll_teardown_mmaps(mapping, offset, offset + PAGE_SIZE);
> >> -	truncate_complete_page(mapping, vmpage);
> >> -}
> >> -
> >> #define    ll_s2sbi(sb)	(s2lsi(sb)->lsi_llsbi)
> >> 
> >> /* don't need an addref as the sb_info should be holding one */
> >> diff --git a/drivers/staging/lustre/lustre/llite/vvp_io.c b/drivers/staging/lustre/lustre/llite/vvp_io.c
> >> index e7a4778e02e4..5a67955974ad 100644
> >> --- a/drivers/staging/lustre/lustre/llite/vvp_io.c
> >> +++ b/drivers/staging/lustre/lustre/llite/vvp_io.c
> >> @@ -1098,7 +1098,7 @@ static int vvp_io_fault_start(const struct lu_env *env,
> >> 	LASSERT(PageLocked(vmpage));
> >> 
> >> 	if (OBD_FAIL_CHECK(OBD_FAIL_LLITE_FAULT_TRUNC_RACE))
> >> -		ll_invalidate_page(vmpage);
> >> +		generic_error_remove_page(vmpage->mapping, vmpage);
> >> 
> >> 	size = i_size_read(inode);
> >> 	/* Though we have already held a cl_lock upon this page, but
> >> diff --git a/drivers/staging/lustre/lustre/llite/vvp_page.c b/drivers/staging/lustre/lustre/llite/vvp_page.c
> >> index 6eb0565ddc22..dcc4d8faa0cd 100644
> >> --- a/drivers/staging/lustre/lustre/llite/vvp_page.c
> >> +++ b/drivers/staging/lustre/lustre/llite/vvp_page.c
> >> @@ -147,7 +147,7 @@ static void vvp_page_discard(const struct lu_env *env,
> >> 	if (vpg->vpg_defer_uptodate && !vpg->vpg_ra_used)
> >> 		ll_ra_stats_inc(vmpage->mapping->host, RA_STAT_DISCARDED);
> >> 
> >> -	ll_invalidate_page(vmpage);
> >> +	generic_error_remove_page(vmpage->mapping, vmpage);
> >> }
> >> 
> >> static void vvp_page_delete(const struct lu_env *env,
> >> -- 
> >> 2.14.0.rc0.dirty
> >> 
> 

From neilb at suse.com  Tue Jun 26 00:29:26 2018
From: neilb at suse.com (NeilBrown)
Date: Tue, 26 Jun 2018 10:29:26 +1000
Subject: [lustre-devel] [PATCH v2 4/7] lustre: libcfs: merge
 linux-tracefile.c into tracefile.c
In-Reply-To: <1529962976-15048-5-git-send-email-jsimmons@infradead.org>
References: <1529962976-15048-1-git-send-email-jsimmons@infradead.org>
 <1529962976-15048-5-git-send-email-jsimmons@infradead.org>
Message-ID: <87y3f275ll.fsf@notabene.neil.brown.name>

On Mon, Jun 25 2018, James Simmons wrote:

> From: NeilBrown <neilb at suse.com>
>
> It's good to keep related code together. The merger exposed the
> flaws of cfs_print_to_console() so rework it to behavior properly.

I'm sorry but I cannot apply this as it is.
The patch contains various changes that are not mentioned at all in the
above description, and the above description doesn't really say anything
useful at all about why anything needed fixing.

You've moved content from tracefile.h into tracefile.c
You've reformatted some code - lining up '\' for example
You've removed some code (e.g. cfs_tracefile_refill_stock)
You've added a memset into libcfs_debug_vmsg2

I don't doubt that these are useful, but they don't all belong in the
same patch that claims to just move code from linux-tracefile.c into
tracefile.c

Sometimes it is necessary to combine lots of things in one patch, though
we avoid it whenever possible.  When it cannot be avoided, the multiple
changes *must* be described together with some indication of the reason
that they are all interdependent.

It is important that patches can be reviewed easily.  This one cannot.

Thanks,
NeilBrown


>
> Signed-off-by: NeilBrown <neilb at suse.com>
> Signed-off-by: James Simmons <jsimmons at infradead.org>
> ---
>  drivers/staging/lustre/lnet/libcfs/Makefile        |   2 +-
>  .../staging/lustre/lnet/libcfs/linux-tracefile.c   | 194 ---------------------
>  drivers/staging/lustre/lnet/libcfs/tracefile.c     | 184 ++++++++++++++++---
>  drivers/staging/lustre/lnet/libcfs/tracefile.h     |  79 ++-------
>  4 files changed, 170 insertions(+), 289 deletions(-)
>  delete mode 100644 drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
>
> diff --git a/drivers/staging/lustre/lnet/libcfs/Makefile b/drivers/staging/lustre/lnet/libcfs/Makefile
> index 5b13edc..cd96434 100644
> --- a/drivers/staging/lustre/lnet/libcfs/Makefile
> +++ b/drivers/staging/lustre/lnet/libcfs/Makefile
> @@ -4,7 +4,7 @@ ccflags-y += -I$(srctree)/drivers/staging/lustre/lustre/include
>  
>  obj-$(CONFIG_LNET) += libcfs.o
>  
> -libcfs-obj-y += linux-tracefile.o linux-debug.o
> +libcfs-obj-y += linux-debug.o
>  libcfs-obj-y += linux-crypto.o
>  libcfs-obj-y += linux-crypto-adler.o
>  
> diff --git a/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c b/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
> deleted file mode 100644
> index 3af7722..0000000
> --- a/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
> +++ /dev/null
> @@ -1,194 +0,0 @@
> -// SPDX-License-Identifier: GPL-2.0
> -/*
> - * GPL HEADER START
> - *
> - * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
> - *
> - * This program is free software; you can redistribute it and/or modify
> - * it under the terms of the GNU General Public License version 2 only,
> - * as published by the Free Software Foundation.
> - *
> - * This program is distributed in the hope that it will be useful, but
> - * WITHOUT ANY WARRANTY; without even the implied warranty of
> - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> - * General Public License version 2 for more details (a copy is included
> - * in the LICENSE file that accompanied this code).
> - *
> - * You should have received a copy of the GNU General Public License
> - * version 2 along with this program; If not, see
> - * http://www.gnu.org/licenses/gpl-2.0.html
> - *
> - * GPL HEADER END
> - */
> -/*
> - * Copyright (c) 2008, 2010, Oracle and/or its affiliates. All rights reserved.
> - * Use is subject to license terms.
> - *
> - * Copyright (c) 2012, Intel Corporation.
> - */
> -/*
> - * This file is part of Lustre, http://www.lustre.org/
> - * Lustre is a trademark of Sun Microsystems, Inc.
> - */
> -
> -#define DEBUG_SUBSYSTEM S_LNET
> -#define LUSTRE_TRACEFILE_PRIVATE
> -
> -#include <linux/slab.h>
> -#include <linux/mm.h>
> -#include "tracefile.h"
> -
> -/* percents to share the total debug memory for each type */
> -static unsigned int pages_factor[CFS_TCD_TYPE_MAX] = {
> -	80,  /* 80% pages for CFS_TCD_TYPE_PROC */
> -	10,  /* 10% pages for CFS_TCD_TYPE_SOFTIRQ */
> -	10   /* 10% pages for CFS_TCD_TYPE_IRQ */
> -};
> -
> -char *cfs_trace_console_buffers[NR_CPUS][CFS_TCD_TYPE_MAX];
> -
> -int cfs_tracefile_init_arch(void)
> -{
> -	int i;
> -	int j;
> -	struct cfs_trace_cpu_data *tcd;
> -
> -	/* initialize trace_data */
> -	memset(cfs_trace_data, 0, sizeof(cfs_trace_data));
> -	for (i = 0; i < CFS_TCD_TYPE_MAX; i++) {
> -		cfs_trace_data[i] =
> -			kmalloc_array(num_possible_cpus(),
> -				      sizeof(union cfs_trace_data_union),
> -				      GFP_KERNEL);
> -		if (!cfs_trace_data[i])
> -			goto out;
> -	}
> -
> -	/* arch related info initialized */
> -	cfs_tcd_for_each(tcd, i, j) {
> -		spin_lock_init(&tcd->tcd_lock);
> -		tcd->tcd_pages_factor = pages_factor[i];
> -		tcd->tcd_type = i;
> -		tcd->tcd_cpu = j;
> -	}
> -
> -	for (i = 0; i < num_possible_cpus(); i++)
> -		for (j = 0; j < 3; j++) {
> -			cfs_trace_console_buffers[i][j] =
> -				kmalloc(CFS_TRACE_CONSOLE_BUFFER_SIZE,
> -					GFP_KERNEL);
> -
> -			if (!cfs_trace_console_buffers[i][j])
> -				goto out;
> -		}
> -
> -	return 0;
> -
> -out:
> -	cfs_tracefile_fini_arch();
> -	pr_err("lnet: Not enough memory\n");
> -	return -ENOMEM;
> -}
> -
> -void cfs_tracefile_fini_arch(void)
> -{
> -	int i;
> -	int j;
> -
> -	for (i = 0; i < num_possible_cpus(); i++)
> -		for (j = 0; j < 3; j++) {
> -			kfree(cfs_trace_console_buffers[i][j]);
> -			cfs_trace_console_buffers[i][j] = NULL;
> -		}
> -
> -	for (i = 0; cfs_trace_data[i]; i++) {
> -		kfree(cfs_trace_data[i]);
> -		cfs_trace_data[i] = NULL;
> -	}
> -}
> -
> -enum cfs_trace_buf_type cfs_trace_buf_idx_get(void)
> -{
> -	if (in_irq())
> -		return CFS_TCD_TYPE_IRQ;
> -	if (in_softirq())
> -		return CFS_TCD_TYPE_SOFTIRQ;
> -	return CFS_TCD_TYPE_PROC;
> -}
> -
> -void
> -cfs_set_ptldebug_header(struct ptldebug_header *header,
> -			struct libcfs_debug_msg_data *msgdata,
> -			unsigned long stack)
> -{
> -	struct timespec64 ts;
> -
> -	ktime_get_real_ts64(&ts);
> -
> -	header->ph_subsys = msgdata->msg_subsys;
> -	header->ph_mask = msgdata->msg_mask;
> -	header->ph_cpu_id = smp_processor_id();
> -	header->ph_type = cfs_trace_buf_idx_get();
> -	/* y2038 safe since all user space treats this as unsigned, but
> -	 * will overflow in 2106
> -	 */
> -	header->ph_sec = (u32)ts.tv_sec;
> -	header->ph_usec = ts.tv_nsec / NSEC_PER_USEC;
> -	header->ph_stack = stack;
> -	header->ph_pid = current->pid;
> -	header->ph_line_num = msgdata->msg_line;
> -	header->ph_extern_pid = 0;
> -}
> -
> -static char *
> -dbghdr_to_err_string(struct ptldebug_header *hdr)
> -{
> -	switch (hdr->ph_subsys) {
> -	case S_LND:
> -	case S_LNET:
> -		return "LNetError";
> -	default:
> -		return "LustreError";
> -	}
> -}
> -
> -static char *
> -dbghdr_to_info_string(struct ptldebug_header *hdr)
> -{
> -	switch (hdr->ph_subsys) {
> -	case S_LND:
> -	case S_LNET:
> -		return "LNet";
> -	default:
> -		return "Lustre";
> -	}
> -}
> -
> -void cfs_print_to_console(struct ptldebug_header *hdr, int mask,
> -			  const char *buf, int len, const char *file,
> -			  const char *fn)
> -{
> -	char *prefix = "Lustre", *ptype = NULL;
> -
> -	if (mask & D_EMERG) {
> -		prefix = dbghdr_to_err_string(hdr);
> -		ptype = KERN_EMERG;
> -	} else if (mask & D_ERROR) {
> -		prefix = dbghdr_to_err_string(hdr);
> -		ptype = KERN_ERR;
> -	} else if (mask & D_WARNING) {
> -		prefix = dbghdr_to_info_string(hdr);
> -		ptype = KERN_WARNING;
> -	} else if (mask & (D_CONSOLE | libcfs_printk)) {
> -		prefix = dbghdr_to_info_string(hdr);
> -		ptype = KERN_INFO;
> -	}
> -
> -	if (mask & D_CONSOLE) {
> -		pr_info("%s%s: %.*s", ptype, prefix, len, buf);
> -	} else {
> -		pr_info("%s%s: %d:%d:(%s:%d:%s()) %.*s", ptype, prefix,
> -			hdr->ph_pid, hdr->ph_extern_pid, file,
> -			hdr->ph_line_num, fn, len, buf);
> -	}
> -}
> diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.c b/drivers/staging/lustre/lnet/libcfs/tracefile.c
> index 6d567a9..229bbb8 100644
> --- a/drivers/staging/lustre/lnet/libcfs/tracefile.c
> +++ b/drivers/staging/lustre/lnet/libcfs/tracefile.c
> @@ -37,8 +37,6 @@
>   */
>  
>  #define DEBUG_SUBSYSTEM S_LNET
> -#define LUSTRE_TRACEFILE_PRIVATE
> -#define pr_fmt(fmt) "Lustre: " fmt
>  
>  #include <linux/ratelimit.h>
>  #include <linux/highmem.h>
> @@ -49,9 +47,19 @@
>  #include <linux/uaccess.h>
>  #include "tracefile.h"
>  
> -/* XXX move things up to the top, comment */
> +#define CFS_TRACE_CONSOLE_BUFFER_SIZE	1024
> +#define TCD_MAX_TYPES			8
> +
> +enum cfs_trace_buf_type {
> +	CFS_TCD_TYPE_PROC = 0,
> +	CFS_TCD_TYPE_SOFTIRQ,
> +	CFS_TCD_TYPE_IRQ,
> +	CFS_TCD_TYPE_MAX
> +};
> +
>  union cfs_trace_data_union (*cfs_trace_data[TCD_MAX_TYPES])[NR_CPUS] __cacheline_aligned;
>  
> +char *cfs_trace_console_buffers[NR_CPUS][CFS_TCD_TYPE_MAX];
>  char cfs_tracefile[TRACEFILE_NAME_SIZE];
>  long long cfs_tracefile_size = CFS_TRACEFILE_SIZE;
>  static struct tracefiled_ctl trace_tctl;
> @@ -145,11 +153,35 @@ void cfs_trace_unlock_tcd(struct cfs_trace_cpu_data *tcd, int walking)
>  		spin_unlock(&tcd->tcd_lock);
>  }
>  
> -#define cfs_tcd_for_each_type_lock(tcd, i, cpu)			   \
> +#define cfs_tcd_for_each(tcd, i, j)					\
> +	for (i = 0; cfs_trace_data[i]; i++)				\
> +		for (j = 0, ((tcd) = &(*cfs_trace_data[i])[j].tcd);	\
> +		     j < num_possible_cpus();				\
> +		     j++, (tcd) = &(*cfs_trace_data[i])[j].tcd)
> +
> +
> +#define cfs_tcd_for_each_type_lock(tcd, i, cpu)				\
>  	for (i = 0; cfs_trace_data[i] &&				\
>  	     (tcd = &(*cfs_trace_data[i])[cpu].tcd) &&			\
>  	     cfs_trace_lock_tcd(tcd, 1); cfs_trace_unlock_tcd(tcd, 1), i++)
>  
> +enum cfs_trace_buf_type cfs_trace_buf_idx_get(void)
> +{
> +	if (in_irq())
> +		return CFS_TCD_TYPE_IRQ;
> +	if (in_softirq())
> +		return CFS_TCD_TYPE_SOFTIRQ;
> +	return CFS_TCD_TYPE_PROC;
> +}
> +
> +static inline char *cfs_trace_get_console_buffer(void)
> +{
> +	unsigned int i = get_cpu();
> +	unsigned int j = cfs_trace_buf_idx_get();
> +
> +	return cfs_trace_console_buffers[i][j];
> +}
> +
>  static inline struct cfs_trace_cpu_data *
>  cfs_trace_get_tcd(void)
>  {
> @@ -168,6 +200,77 @@ static inline void cfs_trace_put_tcd(struct cfs_trace_cpu_data *tcd)
>  	put_cpu();
>  }
>  
> +/* percents to share the total debug memory for each type */
> +static unsigned int pages_factor[CFS_TCD_TYPE_MAX] = {
> +	80,  /* 80% pages for CFS_TCD_TYPE_PROC */
> +	10,  /* 10% pages for CFS_TCD_TYPE_SOFTIRQ */
> +	10   /* 10% pages for CFS_TCD_TYPE_IRQ */
> +};
> +
> +int cfs_tracefile_init_arch(void)
> +{
> +	struct cfs_trace_cpu_data *tcd;
> +	int i;
> +	int j;
> +
> +	/* initialize trace_data */
> +	memset(cfs_trace_data, 0, sizeof(cfs_trace_data));
> +	for (i = 0; i < CFS_TCD_TYPE_MAX; i++) {
> +		cfs_trace_data[i] =
> +			kmalloc_array(num_possible_cpus(),
> +				      sizeof(union cfs_trace_data_union),
> +				      GFP_KERNEL);
> +		if (!cfs_trace_data[i])
> +			goto out_trace_data;
> +	}
> +
> +	/* arch related info initialized */
> +	cfs_tcd_for_each(tcd, i, j) {
> +		spin_lock_init(&tcd->tcd_lock);
> +		tcd->tcd_pages_factor = pages_factor[i];
> +		tcd->tcd_type = i;
> +		tcd->tcd_cpu = j;
> +	}
> +
> +	for (i = 0; i < num_possible_cpus(); i++)
> +		for (j = 0; j < 3; j++) {
> +			cfs_trace_console_buffers[i][j] =
> +				kmalloc(CFS_TRACE_CONSOLE_BUFFER_SIZE,
> +					GFP_KERNEL);
> +
> +			if (!cfs_trace_console_buffers[i][j])
> +				goto out_buffers;
> +		}
> +
> +	return 0;
> +
> +out_buffers:
> +	for (i = 0; cfs_trace_data[i]; i++) {
> +		kfree(cfs_trace_data[i]);
> +		cfs_trace_data[i] = NULL;
> +	}
> +out_trace_data:
> +	pr_err("lnet: Not enough memory\n");
> +	return -ENOMEM;
> +}
> +
> +void cfs_tracefile_fini_arch(void)
> +{
> +	int i;
> +	int j;
> +
> +	for (i = 0; i < num_possible_cpus(); i++)
> +		for (j = 0; j < 3; j++) {
> +			kfree(cfs_trace_console_buffers[i][j]);
> +			cfs_trace_console_buffers[i][j] = NULL;
> +		}
> +
> +	for (i = 0; cfs_trace_data[i]; i++) {
> +		kfree(cfs_trace_data[i]);
> +		cfs_trace_data[i] = NULL;
> +	}
> +}
> +
>  static inline struct cfs_trace_page *
>  cfs_tage_from_list(struct list_head *list)
>  {
> @@ -216,27 +319,6 @@ static void cfs_tage_to_tail(struct cfs_trace_page *tage,
>  	list_move_tail(&tage->linkage, queue);
>  }
>  
> -int cfs_trace_refill_stock(struct cfs_trace_cpu_data *tcd, gfp_t gfp,
> -			   struct list_head *stock)
> -{
> -	int i;
> -
> -	/*
> -	 * XXX nikita: do NOT call portals_debug_msg() (CDEBUG/ENTRY/EXIT)
> -	 * from here: this will lead to infinite recursion.
> -	 */
> -
> -	for (i = 0; i + tcd->tcd_cur_stock_pages < TCD_STOCK_PAGES ; ++i) {
> -		struct cfs_trace_page *tage;
> -
> -		tage = cfs_tage_alloc(gfp);
> -		if (!tage)
> -			break;
> -		list_add_tail(&tage->linkage, stock);
> -	}
> -	return i;
> -}
> -
>  /* return a page that has 'len' bytes left at the end */
>  static struct cfs_trace_page *
>  cfs_trace_get_tage_try(struct cfs_trace_cpu_data *tcd, unsigned long len)
> @@ -340,6 +422,57 @@ static struct cfs_trace_page *cfs_trace_get_tage(struct cfs_trace_cpu_data *tcd,
>  	return tage;
>  }
>  
> +
> +
> +static void cfs_set_ptldebug_header(struct ptldebug_header *header,
> +				    struct libcfs_debug_msg_data *msgdata,
> +				    unsigned long stack)
> +{
> +	struct timespec64 ts;
> +
> +	ktime_get_real_ts64(&ts);
> +
> +	header->ph_subsys = msgdata->msg_subsys;
> +	header->ph_mask = msgdata->msg_mask;
> +	header->ph_cpu_id = smp_processor_id();
> +	header->ph_type = cfs_trace_buf_idx_get();
> +	/* y2038 safe since all user space treats this as unsigned, but
> +	 * will overflow in 2106
> +	 */
> +	header->ph_sec = (u32)ts.tv_sec;
> +	header->ph_usec = ts.tv_nsec / NSEC_PER_USEC;
> +	header->ph_stack = stack;
> +	header->ph_pid = current->pid;
> +	header->ph_line_num = msgdata->msg_line;
> +	header->ph_extern_pid = 0;
> +}
> +
> +static void cfs_print_to_console(struct ptldebug_header *hdr, int mask,
> +				 const char *buf, int len, const char *file,
> +				 const char *fn)
> +{
> +	char *prefix = "Lustre";
> +
> +	if (hdr->ph_subsys == S_LND || hdr->ph_subsys == S_LNET)
> +		prefix = "LNet";
> +
> +	if (mask & D_EMERG) {
> +		pr_emerg("%sError: %d:%d:(%s:%d:%s()) %.*s", prefix,
> +			 hdr->ph_pid, hdr->ph_extern_pid, file,
> +			 hdr->ph_line_num, fn, len, buf);
> +	} else if (mask & D_ERROR) {
> +		pr_err("%sError: %d:%d:(%s:%d:%s()) %.*s", prefix,
> +		       hdr->ph_pid, hdr->ph_extern_pid, file,
> +		       hdr->ph_line_num, fn, len, buf);
> +	} else if (mask & D_WARNING) {
> +		pr_warning("%s: %d:%d:(%s:%d:%s()) %.*s", prefix,
> +			   hdr->ph_pid, hdr->ph_extern_pid, file,
> +			   hdr->ph_line_num, fn, len, buf);
> +	} else if (mask & (D_CONSOLE | libcfs_printk)) {
> +		pr_info("%s: %.*s", prefix, len, buf);
> +	}
> +}
> +
>  int libcfs_debug_msg(struct libcfs_debug_msg_data *msgdata,
>  		     const char *format, ...)
>  {
> @@ -531,6 +664,7 @@ int libcfs_debug_vmsg2(struct libcfs_debug_msg_data *msgdata,
>  	} else {
>  		string_buf = cfs_trace_get_console_buffer();
>  
> +		memset(string_buf, 0, CFS_TRACE_CONSOLE_BUFFER_SIZE);
>  		needed = 0;
>  		if (format1) {
>  			va_copy(ap, args);
> diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.h b/drivers/staging/lustre/lnet/libcfs/tracefile.h
> index f49a9ba..4c79009 100644
> --- a/drivers/staging/lustre/lnet/libcfs/tracefile.h
> +++ b/drivers/staging/lustre/lnet/libcfs/tracefile.h
> @@ -42,13 +42,6 @@
>  #include <linux/smp.h>
>  #include <linux/libcfs/libcfs.h>
>  
> -enum cfs_trace_buf_type {
> -	CFS_TCD_TYPE_PROC = 0,
> -	CFS_TCD_TYPE_SOFTIRQ,
> -	CFS_TCD_TYPE_IRQ,
> -	CFS_TCD_TYPE_MAX
> -};
> -
>  #define TRACEFILE_NAME_SIZE 1024
>  extern char cfs_tracefile[TRACEFILE_NAME_SIZE];
>  extern long long cfs_tracefile_size;
> @@ -60,9 +53,6 @@ enum cfs_trace_buf_type {
>  
>  void libcfs_run_debug_log_upcall(char *file);
>  
> -int  cfs_tracefile_init_arch(void);
> -void cfs_tracefile_fini_arch(void);
> -
>  int cfs_tracefile_dump_all_pages(char *filename);
>  void cfs_trace_debug_print(void);
>  void cfs_trace_flush_pages(void);
> @@ -92,22 +82,6 @@ int cfs_trace_copyout_string(char __user *usr_buffer, int usr_buffer_nob,
>  #define TCD_STOCK_PAGES (TCD_MAX_PAGES)
>  #define CFS_TRACEFILE_SIZE (500 << 20)
>  
> -#ifdef LUSTRE_TRACEFILE_PRIVATE
> -
> -/*
> - * Private declare for tracefile
> - */
> -#define TCD_MAX_PAGES (5 << (20 - PAGE_SHIFT))
> -#define TCD_STOCK_PAGES (TCD_MAX_PAGES)
> -
> -#define CFS_TRACEFILE_SIZE (500 << 20)
> -
> -/*
> - * Size of a buffer for sprinting console messages if we can't get a page
> - * from system
> - */
> -#define CFS_TRACE_CONSOLE_BUFFER_SIZE   1024
> -
>  union cfs_trace_data_union {
>  	struct cfs_trace_cpu_data {
>  		/*
> @@ -184,58 +158,25 @@ int cfs_trace_copyout_string(char __user *usr_buffer, int usr_buffer_nob,
>  	char __pad[L1_CACHE_ALIGN(sizeof(struct cfs_trace_cpu_data))];
>  };
>  
> -#define TCD_MAX_TYPES      8
> -extern union cfs_trace_data_union (*cfs_trace_data[TCD_MAX_TYPES])[NR_CPUS];
> -
> -#define cfs_tcd_for_each(tcd, i, j)				       \
> -	for (i = 0; cfs_trace_data[i]; i++)				\
> -		for (j = 0, ((tcd) = &(*cfs_trace_data[i])[j].tcd);	\
> -		     j < num_possible_cpus();				 \
> -		     j++, (tcd) = &(*cfs_trace_data[i])[j].tcd)
> -
> -void cfs_set_ptldebug_header(struct ptldebug_header *header,
> -			     struct libcfs_debug_msg_data *m,
> -			     unsigned long stack);
> -void cfs_print_to_console(struct ptldebug_header *hdr, int mask,
> -			  const char *buf, int len, const char *file,
> -			  const char *fn);
> -
> -extern char *cfs_trace_console_buffers[NR_CPUS][CFS_TCD_TYPE_MAX];
> -enum cfs_trace_buf_type cfs_trace_buf_idx_get(void);
> -
> -static inline char *
> -cfs_trace_get_console_buffer(void)
> -{
> -	unsigned int i = get_cpu();
> -	unsigned int j = cfs_trace_buf_idx_get();
> -
> -	return cfs_trace_console_buffers[i][j];
> -}
> -
> -int cfs_trace_refill_stock(struct cfs_trace_cpu_data *tcd, gfp_t gfp,
> -			   struct list_head *stock);
> -
>  void cfs_trace_assertion_failed(const char *str,
>  				struct libcfs_debug_msg_data *m);
>  
>  /* ASSERTION that is safe to use within the debug system */
> -#define __LASSERT(cond)						 \
> -do {								    \
> +#define __LASSERT(cond)							\
> +do {									\
>  	if (unlikely(!(cond))) {					\
>  		LIBCFS_DEBUG_MSG_DATA_DECL(msgdata, D_EMERG, NULL);     \
>  		cfs_trace_assertion_failed("ASSERTION("#cond") failed", \
> -					   &msgdata);		   \
> -	}							       \
> +					   &msgdata);			\
> +	}								\
>  } while (0)
>  
> -#define __LASSERT_TAGE_INVARIANT(tage)				  \
> -do {								    \
> -	__LASSERT(tage);					\
> -	__LASSERT(tage->page);				  \
> -	__LASSERT(tage->used <= PAGE_SIZE);			 \
> -	__LASSERT(page_count(tage->page) > 0);		      \
> +#define __LASSERT_TAGE_INVARIANT(tage)			\
> +do {							\
> +	__LASSERT(tage);				\
> +	__LASSERT(tage->page);				\
> +	__LASSERT(tage->used <= PAGE_SIZE);		\
> +	__LASSERT(page_count(tage->page) > 0);		\
>  } while (0)
>  
> -#endif	/* LUSTRE_TRACEFILE_PRIVATE */
> -
>  #endif /* __LIBCFS_TRACEFILE_H__ */
> -- 
> 1.8.3.1
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180626/5e05794e/attachment.sig>

From jsimmons at infradead.org  Tue Jun 26 00:33:55 2018
From: jsimmons at infradead.org (James Simmons)
Date: Tue, 26 Jun 2018 01:33:55 +0100 (BST)
Subject: [lustre-devel] [PATCH v3 03/26] staging: lustre: libcfs:
 properly handle failure cases in SMP code
In-Reply-To: <87d0wfaf90.fsf@notabene.neil.brown.name>
References: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
 <1529875250-11531-4-git-send-email-jsimmons@infradead.org>
 <87d0wfaf90.fsf@notabene.neil.brown.name>
Message-ID: <alpine.LFD.2.21.1806260132360.31196@casper.infradead.org>


> > While pushing the SMP work some bugs were pointed out by Dan
> > Carpenter in the code. Due to single err label in cfs_cpu_init()
> > and cfs_cpt_table_alloc() a few items were being cleaned up that
> > were never initialized. This can lead to crashed and other problems.
> > In those initialization function introduce individual labels to
> > jump to only the thing initialized get freed on failure.
> >
> > Signed-off-by: James Simmons <uja.ornl at yahoo.com>
> > WC-bug-id: https://jira.whamcloud.com/browse/LU-10932
> > Reviewed-on: https://review.whamcloud.com/32085
> > Reviewed-by: Dmitry Eremin <dmitry.eremin at intel.com>
> > Reviewed-by: Andreas Dilger <adilger at whamcloud.com>
> > Signed-off-by: James Simmons <jsimmons at infradead.org>
> > ---
> >  drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c | 72 ++++++++++++++++++-------
> >  1 file changed, 52 insertions(+), 20 deletions(-)
> >
> > diff --git a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
> > index 46d3530..bdd71a3 100644
> > --- a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
> > +++ b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
> > @@ -85,17 +85,19 @@ struct cfs_cpt_table *
> >  
> >  	cptab->ctb_nparts = ncpt;
> >  
> > +	if (!zalloc_cpumask_var(&cptab->ctb_cpumask, GFP_NOFS))
> > +		goto failed_alloc_cpumask;
> > +
> >  	cptab->ctb_nodemask = kzalloc(sizeof(*cptab->ctb_nodemask),
> >  				      GFP_NOFS);
> > -	if (!zalloc_cpumask_var(&cptab->ctb_cpumask, GFP_NOFS) ||
> > -	    !cptab->ctb_nodemask)
> > -		goto failed;
> > +	if (!cptab->ctb_nodemask)
> > +		goto failed_alloc_nodemask;
> >  
> >  	cptab->ctb_cpu2cpt = kvmalloc_array(num_possible_cpus(),
> >  					    sizeof(cptab->ctb_cpu2cpt[0]),
> >  					    GFP_KERNEL);
> >  	if (!cptab->ctb_cpu2cpt)
> > -		goto failed;
> > +		goto failed_alloc_cpu2cpt;
> >  
> >  	memset(cptab->ctb_cpu2cpt, -1,
> >  	       num_possible_cpus() * sizeof(cptab->ctb_cpu2cpt[0]));
> > @@ -103,22 +105,41 @@ struct cfs_cpt_table *
> >  	cptab->ctb_parts = kvmalloc_array(ncpt, sizeof(cptab->ctb_parts[0]),
> >  					  GFP_KERNEL);
> >  	if (!cptab->ctb_parts)
> > -		goto failed;
> > +		goto failed_alloc_ctb_parts;
> > +
> > +	memset(cptab->ctb_parts, -1, ncpt * sizeof(cptab->ctb_parts[0]));
> >  
> >  	for (i = 0; i < ncpt; i++) {
> >  		struct cfs_cpu_partition *part = &cptab->ctb_parts[i];
> >  
> > +		if (!zalloc_cpumask_var(&part->cpt_cpumask, GFP_NOFS))
> > +			goto failed_setting_ctb_parts;
> > +
> >  		part->cpt_nodemask = kzalloc(sizeof(*part->cpt_nodemask),
> >  					     GFP_NOFS);
> > -		if (!zalloc_cpumask_var(&part->cpt_cpumask, GFP_NOFS) ||
> > -		    !part->cpt_nodemask)
> > -			goto failed;
> > +		if (!part->cpt_nodemask)
> > +			goto failed_setting_ctb_parts;
> 
> If zalloc_cpumask_var() succeeds, but kzalloc() fails (which is almost
> impossible, but still) we go to failed_setting_ctb_parts, with
>  cptab->ctb_parts[i]->cpt_cpumask needing to be freed.
> 
> >  	}
> >  
> >  	return cptab;
> >  
> > - failed:
> > -	cfs_cpt_table_free(cptab);
> > +failed_setting_ctb_parts:
> > +	while (i-- >= 0) {
> 
> but we don't free anything in cptab->ctb_parts[i].
> I've fix this by calling free_cpumask_var() before the goto.
> 
> And will propagate the change through future patches in this series.

Thanks. I will grab the updated patches from your testing tree. 

 
> > +		struct cfs_cpu_partition *part = &cptab->ctb_parts[i];
> > +
> > +		kfree(part->cpt_nodemask);
> > +		free_cpumask_var(part->cpt_cpumask);
> > +	}
> > +
> > +	kvfree(cptab->ctb_parts);
> > +failed_alloc_ctb_parts:
> > +	kvfree(cptab->ctb_cpu2cpt);
> > +failed_alloc_cpu2cpt:
> > +	kfree(cptab->ctb_nodemask);
> > +failed_alloc_nodemask:
> > +	free_cpumask_var(cptab->ctb_cpumask);
> > +failed_alloc_cpumask:
> > +	kfree(cptab);
> >  	return NULL;
> >  }
> >  EXPORT_SYMBOL(cfs_cpt_table_alloc);
> > @@ -944,7 +965,7 @@ static int cfs_cpu_dead(unsigned int cpu)
> >  int
> >  cfs_cpu_init(void)
> >  {
> > -	int ret = 0;
> > +	int ret;
> >  
> >  	LASSERT(!cfs_cpt_tab);
> >  
> > @@ -953,23 +974,23 @@ static int cfs_cpu_dead(unsigned int cpu)
> >  					"staging/lustre/cfe:dead", NULL,
> >  					cfs_cpu_dead);
> >  	if (ret < 0)
> > -		goto failed;
> > +		goto failed_cpu_dead;
> > +
> >  	ret = cpuhp_setup_state_nocalls(CPUHP_AP_ONLINE_DYN,
> >  					"staging/lustre/cfe:online",
> >  					cfs_cpu_online, NULL);
> >  	if (ret < 0)
> > -		goto failed;
> > +		goto failed_cpu_online;
> > +
> >  	lustre_cpu_online = ret;
> >  #endif
> > -	ret = -EINVAL;
> > -
> >  	get_online_cpus();
> >  	if (*cpu_pattern) {
> >  		char *cpu_pattern_dup = kstrdup(cpu_pattern, GFP_KERNEL);
> >  
> >  		if (!cpu_pattern_dup) {
> >  			CERROR("Failed to duplicate cpu_pattern\n");
> > -			goto failed;
> > +			goto failed_alloc_table;
> >  		}
> >  
> >  		cfs_cpt_tab = cfs_cpt_table_create_pattern(cpu_pattern_dup);
> > @@ -977,7 +998,7 @@ static int cfs_cpu_dead(unsigned int cpu)
> >  		if (!cfs_cpt_tab) {
> >  			CERROR("Failed to create cptab from pattern %s\n",
> >  			       cpu_pattern);
> > -			goto failed;
> > +			goto failed_alloc_table;
> >  		}
> >  
> >  	} else {
> > @@ -985,7 +1006,7 @@ static int cfs_cpu_dead(unsigned int cpu)
> >  		if (!cfs_cpt_tab) {
> >  			CERROR("Failed to create ptable with npartitions %d\n",
> >  			       cpu_npartitions);
> > -			goto failed;
> > +			goto failed_alloc_table;
> >  		}
> >  	}
> >  
> > @@ -996,8 +1017,19 @@ static int cfs_cpu_dead(unsigned int cpu)
> >  		 cfs_cpt_number(cfs_cpt_tab));
> >  	return 0;
> >  
> > - failed:
> > +failed_alloc_table:
> >  	put_online_cpus();
> > -	cfs_cpu_fini();
> > +
> > +	if (cfs_cpt_tab)
> > +		cfs_cpt_table_free(cfs_cpt_tab);
> > +
> > +	ret = -EINVAL;
> > +#ifdef CONFIG_HOTPLUG_CPU
> > +	if (lustre_cpu_online > 0)
> > +		cpuhp_remove_state_nocalls(lustre_cpu_online);
> > +failed_cpu_online:
> > +	cpuhp_remove_state_nocalls(CPUHP_LUSTRE_CFS_DEAD);
> > +failed_cpu_dead:
> > +#endif
> >  	return ret;
> >  }
> > -- 
> > 1.8.3.1
> 

From neilb at suse.com  Tue Jun 26 00:34:57 2018
From: neilb at suse.com (NeilBrown)
Date: Tue, 26 Jun 2018 10:34:57 +1000
Subject: [lustre-devel] [PATCH v3 23/26] staging: lustre: ptlrpc: use
 current CPU instead of hardcoded 0
In-Reply-To: <alpine.LFD.2.21.1806252350310.31196@casper.infradead.org>
References: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
 <1529875250-11531-24-git-send-email-jsimmons@infradead.org>
 <87a7rj8ub9.fsf@notabene.neil.brown.name>
 <alpine.LFD.2.21.1806252350310.31196@casper.infradead.org>
Message-ID: <87vaa675ce.fsf@notabene.neil.brown.name>

On Mon, Jun 25 2018, James Simmons wrote:

>> > From: Dmitry Eremin <dmitry.eremin at intel.com>
>> >
>> > fix crash if CPU 0 disabled.
>> >
>> > Signed-off-by: Dmitry Eremin <dmitry.eremin at intel.com>
>> > WC-bug-id: https://jira.whamcloud.com/browse/LU-8710
>> > Reviewed-on: https://review.whamcloud.com/23305
>> > Reviewed-by: Doug Oucharek <dougso at me.com>
>> > Reviewed-by: Andreas Dilger <adilger at whamcloud.com>
>> > Signed-off-by: James Simmons <jsimmons at infradead.org>
>> > ---
>> >  drivers/staging/lustre/lustre/ptlrpc/service.c | 11 ++++++-----
>> >  1 file changed, 6 insertions(+), 5 deletions(-)
>> >
>> > diff --git a/drivers/staging/lustre/lustre/ptlrpc/service.c b/drivers/staging/lustre/lustre/ptlrpc/service.c
>> > index 3fd8c74..8e74a45 100644
>> > --- a/drivers/staging/lustre/lustre/ptlrpc/service.c
>> > +++ b/drivers/staging/lustre/lustre/ptlrpc/service.c
>> > @@ -421,7 +421,7 @@ static void ptlrpc_at_timer(struct timer_list *t)
>> >  		 * there are.
>> >  		 */
>> >  		/* weight is # of HTs */
>> > -		if (cpumask_weight(topology_sibling_cpumask(0)) > 1) {
>> > +		if (cpumask_weight(topology_sibling_cpumask(smp_processor_id())) > 1) {
>> 
>> This pops a warning for me:
>> [ 1877.516799] BUG: using smp_processor_id() in preemptible [00000000] code: mount.lustre/14077
>> 
>> I'll change it to disable preemption, both here and below.
>
> For .config I have:
>
> # CONFIG_PREEMPT_NONE is not set
> CONFIG_PREEMPT_VOLUNTARY=y
> # CONFIG_PREEMPT is not set
>
> What does yours look like? Strange no one has ever reported an error 
> before. Thanks for finding this!!!

I have
CONFIG_PREEMPT_RCU=y
CONFIG_PREEMPT_NOTIFIERS=y
# CONFIG_PREEMPT_NONE is not set
# CONFIG_PREEMPT_VOLUNTARY is not set
CONFIG_PREEMPT=y
CONFIG_PREEMPT_COUNT=y
CONFIG_DEBUG_PREEMPT=y
# CONFIG_PREEMPTIRQ_EVENTS is not set
# CONFIG_PREEMPT_TRACER is not set

CONFIG_PREEMPT=y helps find bugs - not just these but also some races.

Thanks,
NeilBrown
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180626/bb20ad16/attachment.sig>

From jsimmons at infradead.org  Tue Jun 26 00:39:33 2018
From: jsimmons at infradead.org (James Simmons)
Date: Tue, 26 Jun 2018 01:39:33 +0100 (BST)
Subject: [lustre-devel] [PATCH v3 07/26] staging: lustre: libcfs: NUMA
 support
In-Reply-To: <877emnaed8.fsf@notabene.neil.brown.name>
References: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
 <1529875250-11531-8-git-send-email-jsimmons@infradead.org>
 <877emnaed8.fsf@notabene.neil.brown.name>
Message-ID: <alpine.LFD.2.21.1806260137540.31196@casper.infradead.org>


> > From: Amir Shehata <amir.shehata at intel.com>
> >
> > This patch adds NUMA node support. NUMA node information is stored
> > in the CPT table. A NUMA node mask is maintained for the entire
> > table as well as for each CPT to track the NUMA nodes related to
> > each of the CPTs. Add new function cfs_cpt_of_node() which returns
> > the CPT of a particular NUMA node.
> 
> I note that you didn't respond to Greg's questions about this patch.
> I'll accept it anyway in the interests of moving forward, but I think
> his comments were probably valid, and need to be considered at some
> stage.

I hope Doug's response answers the questions. I can get Olaf from HPE 
involved if need be.
 
> There is a bug though....
> >
> > Signed-off-by: Amir Shehata <amir.shehata at intel.com>
> > WC-bug-id: https://jira.whamcloud.com/browse/LU-7734
> > Reviewed-on: http://review.whamcloud.com/18916
> > Reviewed-by: Olaf Weber <olaf at sgi.com>
> > Reviewed-by: Doug Oucharek <dougso at me.com>
> > Signed-off-by: James Simmons <jsimmons at infradead.org>
> > ---
> >  .../lustre/include/linux/libcfs/libcfs_cpu.h        | 11 +++++++++++
> >  drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c     | 21 +++++++++++++++++++++
> >  2 files changed, 32 insertions(+)
> >
> > diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h
> > index 1b4333d..ff3ecf5 100644
> > --- a/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h
> > +++ b/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h
> > @@ -103,6 +103,8 @@ struct cfs_cpt_table {
> >  	int				*ctb_cpu2cpt;
> >  	/* all cpus in this partition table */
> >  	cpumask_var_t			ctb_cpumask;
> > +	/* shadow HW node to CPU partition ID */
> > +	int				*ctb_node2cpt;
> >  	/* all nodes in this partition table */
> >  	nodemask_t			*ctb_nodemask;
> >  };
> > @@ -143,6 +145,10 @@ struct cfs_cpt_table {
> >   */
> >  int cfs_cpt_of_cpu(struct cfs_cpt_table *cptab, int cpu);
> >  /**
> > + * shadow HW node ID \a NODE to CPU-partition ID by \a cptab
> > + */
> > +int cfs_cpt_of_node(struct cfs_cpt_table *cptab, int node);
> > +/**
> >   * bind current thread on a CPU-partition \a cpt of \a cptab
> >   */
> >  int cfs_cpt_bind(struct cfs_cpt_table *cptab, int cpt);
> > @@ -299,6 +305,11 @@ void cfs_cpt_unset_nodemask(struct cfs_cpt_table *cptab,
> >  	return 0;
> >  }
> >  
> > +static inline int cfs_cpt_of_node(struct cfs_cpt_table *cptab, int node)
> > +{
> > +	return 0;
> > +}
> > +
> >  static inline int
> >  cfs_cpt_bind(struct cfs_cpt_table *cptab, int cpt)
> >  {
> > diff --git a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
> > index 33294da..8c5cf7b 100644
> > --- a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
> > +++ b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
> > @@ -102,6 +102,15 @@ struct cfs_cpt_table *
> >  	memset(cptab->ctb_cpu2cpt, -1,
> >  	       nr_cpu_ids * sizeof(cptab->ctb_cpu2cpt[0]));
> >  
> > +	cptab->ctb_node2cpt = kvmalloc_array(nr_node_ids,
> > +					     sizeof(cptab->ctb_node2cpt[0]),
> > +					     GFP_KERNEL);
> > +	if (!cptab->ctb_node2cpt)
> > +		goto failed_alloc_node2cpt;
> > +
> > +	memset(cptab->ctb_node2cpt, -1,
> > +	       nr_node_ids * sizeof(cptab->ctb_node2cpt[0]));
> > +
> >  	cptab->ctb_parts = kvmalloc_array(ncpt, sizeof(cptab->ctb_parts[0]),
> >  					  GFP_KERNEL);
> >  	if (!cptab->ctb_parts)
> > @@ -133,6 +142,8 @@ struct cfs_cpt_table *
> >  
> >  	kvfree(cptab->ctb_parts);
> >  failed_alloc_ctb_parts:
> > +	kvfree(cptab->ctb_node2cpt);
> > +failed_alloc_node2cpt:
> >  	kvfree(cptab->ctb_cpu2cpt);
> >  failed_alloc_cpu2cpt:
> >  	kfree(cptab->ctb_nodemask);
> > @@ -150,6 +161,7 @@ struct cfs_cpt_table *
> >  	int i;
> >  
> >  	kvfree(cptab->ctb_cpu2cpt);
> > +	kvfree(cptab->ctb_node2cpt);
> >  
> >  	for (i = 0; cptab->ctb_parts && i < cptab->ctb_nparts; i++) {
> >  		struct cfs_cpu_partition *part = &cptab->ctb_parts[i];
> > @@ -515,6 +527,15 @@ struct cfs_cpt_table *
> >  }
> >  EXPORT_SYMBOL(cfs_cpt_of_cpu);
> >  
> > +int cfs_cpt_of_node(struct cfs_cpt_table *cptab, int node)
> > +{
> > +	if (node < 0 || node > nr_node_ids)
> > +		return CFS_CPT_ANY;
> > +
> > +	return cptab->ctb_node2cpt[node];
> > +}
> 
> So if node == nr_node_ids, we access beyond the end of the ctb_node2cpt array.
> Oops.
> I've fixed this before applying.

Ouch. That bug has been around for a while :-(

> Thanks,
> NeilBrown
> 
> 
> > +EXPORT_SYMBOL(cfs_cpt_of_node);
> > +
> >  int
> >  cfs_cpt_bind(struct cfs_cpt_table *cptab, int cpt)
> >  {
> > -- 
> > 1.8.3.1
> 

From jsimmons at infradead.org  Tue Jun 26 00:42:20 2018
From: jsimmons at infradead.org (James Simmons)
Date: Tue, 26 Jun 2018 01:42:20 +0100 (BST)
Subject: [lustre-devel] [PATCH v3 13/26] staging: lustre: libcfs: use
 int type for CPT identification.
In-Reply-To: <871scvadj6.fsf@notabene.neil.brown.name>
References: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
 <1529875250-11531-14-git-send-email-jsimmons@infradead.org>
 <871scvadj6.fsf@notabene.neil.brown.name>
Message-ID: <alpine.LFD.2.21.1806260140480.31196@casper.infradead.org>


> > From: Dmitry Eremin <dmitry.eremin at intel.com>
> >
> > Use int type for CPT identification to match the linux kernel
> > CPU identification.
> 
> Can someone site evidence for "int" being the dominant choice for CPU
> identification in the kernel?
> I looked in cpumask.h and found plenty of "unsigned int".
> I also found
> 
> Commit: 9b130ad5bb82 ("treewide: make "nr_cpu_ids" unsigned")
> 
> which makes nr_cpu_ids unsigned.
> 
> So I'm dropping this patch for now as the justification is not
> convincing.
> 
> If there is a real case to be made, please resubmit.

Honestly using int doesn't make sense to me but Dmitry got the
impresssion that using int was more correct. Dmitry where did
you get that information about using int from?

> > Signed-off-by: Dmitry Eremin <dmitry.eremin at intel.com>
> > WC-bug-id: https://jira.whamcloud.com/browse/LU-8703
> > Reviewed-on: https://review.whamcloud.com/23304
> > Reviewed-by: James Simmons <uja.ornl at yahoo.com>
> > Reviewed-by: Doug Oucharek <dougso at me.com>
> > Reviewed-by: Oleg Drokin <green at whamcloud.com>
> > Signed-off-by: James Simmons <jsimmons at infradead.org>
> > ---
> >  drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h |  8 ++++----
> >  drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c          | 14 +++++++-------
> >  2 files changed, 11 insertions(+), 11 deletions(-)
> >
> > diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h
> > index 9dbb0b1..2bb2140 100644
> > --- a/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h
> > +++ b/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h
> > @@ -89,18 +89,18 @@ struct cfs_cpu_partition {
> >  	/* NUMA distance between CPTs */
> >  	unsigned int			*cpt_distance;
> >  	/* spread rotor for NUMA allocator */
> > -	unsigned int			cpt_spread_rotor;
> > +	int				cpt_spread_rotor;
> >  };
> >  
> >  
> >  /** descriptor for CPU partitions */
> >  struct cfs_cpt_table {
> >  	/* spread rotor for NUMA allocator */
> > -	unsigned int			ctb_spread_rotor;
> > +	int				ctb_spread_rotor;
> >  	/* maximum NUMA distance between all nodes in table */
> >  	unsigned int			ctb_distance;
> >  	/* # of CPU partitions */
> > -	unsigned int			ctb_nparts;
> > +	int				ctb_nparts;
> >  	/* partitions tables */
> >  	struct cfs_cpu_partition	*ctb_parts;
> >  	/* shadow HW CPU to CPU partition ID */
> > @@ -355,7 +355,7 @@ static inline void cfs_cpu_fini(void)
> >  /**
> >   * create a cfs_cpt_table with \a ncpt number of partitions
> >   */
> > -struct cfs_cpt_table *cfs_cpt_table_alloc(unsigned int ncpt);
> > +struct cfs_cpt_table *cfs_cpt_table_alloc(int ncpt);
> >  
> >  /*
> >   * allocate per-cpu-partition data, returned value is an array of pointers,
> > diff --git a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
> > index aaab7cb..8f7de59 100644
> > --- a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
> > +++ b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
> > @@ -73,7 +73,7 @@
> >  module_param(cpu_pattern, charp, 0444);
> >  MODULE_PARM_DESC(cpu_pattern, "CPU partitions pattern");
> >  
> > -struct cfs_cpt_table *cfs_cpt_table_alloc(unsigned int ncpt)
> > +struct cfs_cpt_table *cfs_cpt_table_alloc(int ncpt)
> >  {
> >  	struct cfs_cpt_table *cptab;
> >  	int i;
> > @@ -788,13 +788,13 @@ static int cfs_cpt_choose_ncpus(struct cfs_cpt_table *cptab, int cpt,
> >  	return rc;
> >  }
> >  
> > -#define CPT_WEIGHT_MIN  4u
> > +#define CPT_WEIGHT_MIN 4
> >  
> > -static unsigned int cfs_cpt_num_estimate(void)
> > +static int cfs_cpt_num_estimate(void)
> >  {
> > -	unsigned int nnode = num_online_nodes();
> > -	unsigned int ncpu = num_online_cpus();
> > -	unsigned int ncpt;
> > +	int nnode = num_online_nodes();
> > +	int ncpu = num_online_cpus();
> > +	int ncpt;
> >  
> >  	if (ncpu <= CPT_WEIGHT_MIN) {
> >  		ncpt = 1;
> > @@ -824,7 +824,7 @@ static unsigned int cfs_cpt_num_estimate(void)
> >  	/* config many CPU partitions on 32-bit system could consume
> >  	 * too much memory
> >  	 */
> > -	ncpt = min(2U, ncpt);
> > +	ncpt = min(2, ncpt);
> >  #endif
> >  	while (ncpu % ncpt)
> >  		ncpt--; /* worst case is 1 */
> > -- 
> > 1.8.3.1
> 

From jsimmons at infradead.org  Tue Jun 26 00:54:43 2018
From: jsimmons at infradead.org (James Simmons)
Date: Tue, 26 Jun 2018 01:54:43 +0100 (BST)
Subject: [lustre-devel] [PATCH v3 19/26] staging: lustre: libcfs: report
 NUMA node instead of just node
In-Reply-To: <87y3f38ye6.fsf@notabene.neil.brown.name>
References: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
 <1529875250-11531-20-git-send-email-jsimmons@infradead.org>
 <87y3f38ye6.fsf@notabene.neil.brown.name>
Message-ID: <alpine.LFD.2.21.1806260146520.31196@casper.infradead.org>

> On Sun, Jun 24 2018, James Simmons wrote:
> 
> > From: Dmitry Eremin <dmitry.eremin at intel.com>
> >
> > Reporting "HW nodes" is too generic. It really is reporting
> > "HW NUMA nodes". Update the debug message.
> 
> I'm not happy with this patch description.....

How about:

In the HPC world a node refers to the actual whole computer system
used in a cluster. Reporting just "HW nodes" is not clear so change
the debug report to "HW NUMA nodes" since this report the number
of NUMA nodes in use by Lustre.

 
> >
> > Signed-off-by: Dmitry Eremin <dmitry.eremin at intel.com>
> > WC-bug-id: https://jira.whamcloud.com/browse/LU-8703
> > Reviewed-on: https://review.whamcloud.com/23306
> > Reviewed-by: James Simmons <uja.ornl at yahoo.com>
> > Reviewed-by: Andreas Dilger <adilger at whamcloud.com>
> > Reviewed-by: Patrick Farrell <paf at cray.com>
> > Reviewed-by: Olaf Weber <olaf.weber at hpe.com>
> > Reviewed-by: Oleg Drokin <green at whamcloud.com>
> > Signed-off-by: James Simmons <jsimmons at infradead.org>
> > ---
> >  drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h | 2 ++
> >  drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c          | 2 +-
> >  drivers/staging/lustre/lnet/lnet/lib-msg.c               | 2 ++
> >  3 files changed, 5 insertions(+), 1 deletion(-)
> >
> > diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h
> > index 2bb2140..29c5071 100644
> > --- a/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h
> > +++ b/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h
> > @@ -90,6 +90,8 @@ struct cfs_cpu_partition {
> >  	unsigned int			*cpt_distance;
> >  	/* spread rotor for NUMA allocator */
> >  	int				cpt_spread_rotor;
> > +	/* NUMA node if cpt_nodemask is empty */
> > +	int				cpt_node;
> >  };
> 
> It doesn't give any reason why this (unused) field was added.
> So I've removed it.
> 
> 
> >  
> >  
> > diff --git a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
> > index 18925c7..86afa31 100644
> > --- a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
> > +++ b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
> > @@ -1142,7 +1142,7 @@ int cfs_cpu_init(void)
> >  
> >  	put_online_cpus();
> >  
> > -	LCONSOLE(0, "HW nodes: %d, HW CPU cores: %d, npartitions: %d\n",
> > +	LCONSOLE(0, "HW NUMA nodes: %d, HW CPU cores: %d, npartitions: %d\n",
> >  		 num_online_nodes(), num_online_cpus(),
> >  		 cfs_cpt_number(cfs_cpt_tab));
> >  	return 0;
> 
> It does explain this hunk, which is fine.
> 
> 
> > diff --git a/drivers/staging/lustre/lnet/lnet/lib-msg.c b/drivers/staging/lustre/lnet/lnet/lib-msg.c
> > index 0091273..27bdefa 100644
> > --- a/drivers/staging/lustre/lnet/lnet/lib-msg.c
> > +++ b/drivers/staging/lustre/lnet/lnet/lib-msg.c
> > @@ -568,6 +568,8 @@
> >  
> >  	/* number of CPUs */
> >  	container->msc_nfinalizers = cfs_cpt_weight(lnet_cpt_table(), cpt);
> > +	if (container->msc_nfinalizers == 0)
> > +		container->msc_nfinalizers = 1;
> 
> It doesn't justify this at all.
> 
> I guess this was meant to be in the previous patch, so I've moved it.
> 
> Thanks,
> NeilBrown
> 
> 
> >  
> >  	container->msc_finalizers = kvzalloc_cpt(container->msc_nfinalizers *
> >  						 sizeof(*container->msc_finalizers),
> > -- 
> > 1.8.3.1
> 

From jsimmons at infradead.org  Tue Jun 26 00:55:41 2018
From: jsimmons at infradead.org (James Simmons)
Date: Tue, 26 Jun 2018 01:55:41 +0100 (BST)
Subject: [lustre-devel] [PATCH v3 05/26] staging: lustre: libcfs: remove
 excess space
In-Reply-To: <87a7rjaek3.fsf@notabene.neil.brown.name>
References: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
 <1529875250-11531-6-git-send-email-jsimmons@infradead.org>
 <87a7rjaek3.fsf@notabene.neil.brown.name>
Message-ID: <alpine.LFD.2.21.1806260155290.31196@casper.infradead.org>


> > From: Amir Shehata <amir.shehata at intel.com>
> >
> > The function cfs_cpt_table_print() was adding two spaces
> > to the string buffer. Just add it once.
> 
> No it doesn't.  Maybe it did in the out-of-tree code, but the linux code
> is different.
> 
> The extra space is
> 
>                        rc = snprintf(tmp, len, " %d", j);
> 
> But in Linux that is
> 
> 			rc = snprintf(tmp, len, "%d ", j);
> 
> Both are wrong, but for different reasons.
> I've change this patch to be:
> 
> 			rc = snprintf(tmp, len, "%d\t:", i);
> and
> 			rc = snprintf(tmp, len, " %d", j);
> and changed the comment to say that we don't need a stray space at the
> end of the line.

Thank you.

> 
> NeilBrown
> 
> 
> 
> >
> > Signed-off-by: Amir Shehata <amir.shehata at intel.com>
> > WC-bug-id: https://jira.whamcloud.com/browse/LU-7734
> > Reviewed-on: http://review.whamcloud.com/18916
> > Reviewed-by: Olaf Weber <olaf at sgi.com>
> > Reviewed-by: Doug Oucharek <dougso at me.com>
> > Reviewed-by: Oleg Drokin <green at whamcloud.com>
> > Signed-off-by: James Simmons <jsimmons at infradead.org>
> > ---
> >  drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
> > index ea8d55c..680a2b1 100644
> > --- a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
> > +++ b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
> > @@ -177,7 +177,7 @@ struct cfs_cpt_table *
> >  
> >  	for (i = 0; i < cptab->ctb_nparts; i++) {
> >  		if (len > 0) {
> > -			rc = snprintf(tmp, len, "%d\t: ", i);
> > +			rc = snprintf(tmp, len, "%d\t:", i);
> >  			len -= rc;
> >  		}
> >  
> > -- 
> > 1.8.3.1
> 

From jsimmons at infradead.org  Tue Jun 26 01:13:22 2018
From: jsimmons at infradead.org (James Simmons)
Date: Tue, 26 Jun 2018 02:13:22 +0100 (BST)
Subject: [lustre-devel] [PATCH 00/24] lustre - more cleanups including
 module reduction.
In-Reply-To: <87tvprah32.fsf@notabene.neil.brown.name>
References: <152904663333.10587.10934053155404014785.stgit@noble>
 <alpine.LFD.2.21.1806210239190.3886@casper.infradead.org>
 <DM3PR1101MB1213DDB7FC857EE6B1444153CB760@DM3PR1101MB1213.namprd11.prod.outlook.com>
 <alpine.LFD.2.21.1806210354260.3886@casper.infradead.org>
 <3FD4D051-9C95-449D-8A23-D42B271E55B8@dilger.ca>
 <DM3PR1101MB121322E85FDE1E5310F6F4B7CB760@DM3PR1101MB1213.namprd11.prod.outlook.com>
 <87tvprah32.fsf@notabene.neil.brown.name>
Message-ID: <alpine.LFD.2.21.1806260206360.31196@casper.infradead.org>


> > Ah, thanks Andreas.  Perhaps not coincidentally, Lustre 2.4 is the first release I worked on.
> >
> > Neil, I am really unenamored of the idea of the shared ptlrpc ldlm module being named ptlrpc...
> 
> Can you say why?  Is it the choice of name that bothers you, or the
> combining of two things into the one kernel module, or something else?
> Currently in  git://git.hpdd.intel.com/fs/lustre-release.git
> (or the git tree that was until recently at the above address),
> the module named "ptlrpc" contains ptlrpc code, ldlm code, and target
> code.  I wonder what "target" means in this context.

target is the server code built into ptlrpc which is for the "Unified 
Target". Its the shared code that is used by both the MDS and OSS code.
Andreas or Oleg can add more details than this. BTW also the nodemap
code also get built into ptlrpc module as well for the server side.
 
> > Also, Lustre currently has a bunch of module parameters which are used for configuration.  Thoughts on that?
> 
> Yes, the module parameters are an interesting part of the story.
> libcfs has a bunch of module parameters that are symlinked from debugfs.
> It seems that user-space largely uses the debugfs links to access them,
> so they can become part of the "lnet" module with minimal pain.
> Other modules have parameters that I haven't yet looked in to.
> 
> There probably will need to be user-space changes to support reduction
> in the number of modules.

The test suite will need some changes since it loads the modules. We
need to handle the case if its built into the kernel as well.

From jsimmons at infradead.org  Tue Jun 26 01:15:03 2018
From: jsimmons at infradead.org (James Simmons)
Date: Tue, 26 Jun 2018 02:15:03 +0100 (BST)
Subject: [lustre-devel] [PATCH v3 08/26] staging: lustre: libcfs: add
 cpu distance handling
In-Reply-To: <874lhradxj.fsf@notabene.neil.brown.name>
References: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
 <1529875250-11531-9-git-send-email-jsimmons@infradead.org>
 <874lhradxj.fsf@notabene.neil.brown.name>
Message-ID: <alpine.LFD.2.21.1806260213340.31196@casper.infradead.org>


> On Sun, Jun 24 2018, James Simmons wrote:
> 
> > From: Amir Shehata <amir.shehata at intel.com>
> >
> > Add functionality to calculate the distance between two CPTs.
> > Expose those distance in debugfs so people deploying a setup
> > can debug what is being created for CPTs.
> 
> This patch doesn't expose anything in debugfs - a later patch
> does that.
> So I've changed the comment to "Prepare to expose those ...."

Doug Oucharek recommonds the following commit message body:

Add cpu distance routines which will be used by the Multi-Rail feature 
to determine what fabric interface is nearest to the core we are currently 
running on. Configuration of these distances will be provided from user 
space via configuration routines in Lustre.


> > Signed-off-by: Amir Shehata <amir.shehata at intel.com>
> > WC-bug-id: https://jira.whamcloud.com/browse/LU-7734
> > Reviewed-on: http://review.whamcloud.com/18916
> > Reviewed-by: Olaf Weber <olaf at sgi.com>
> > Reviewed-by: Doug Oucharek <dougso at me.com>
> > Signed-off-by: James Simmons <jsimmons at infradead.org>
> > ---
> >  .../lustre/include/linux/libcfs/libcfs_cpu.h       | 31 +++++++++++
> >  drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c    | 61 ++++++++++++++++++++++
> >  2 files changed, 92 insertions(+)
> >
> > diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h
> > index ff3ecf5..a015ac1 100644
> > --- a/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h
> > +++ b/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h
> > @@ -86,6 +86,8 @@ struct cfs_cpu_partition {
> >  	cpumask_var_t			cpt_cpumask;
> >  	/* nodes mask for this partition */
> >  	nodemask_t			*cpt_nodemask;
> > +	/* NUMA distance between CPTs */
> > +	unsigned int			*cpt_distance;
> >  	/* spread rotor for NUMA allocator */
> >  	unsigned int			cpt_spread_rotor;
> >  };
> > @@ -95,6 +97,8 @@ struct cfs_cpu_partition {
> >  struct cfs_cpt_table {
> >  	/* spread rotor for NUMA allocator */
> >  	unsigned int			ctb_spread_rotor;
> > +	/* maximum NUMA distance between all nodes in table */
> > +	unsigned int			ctb_distance;
> >  	/* # of CPU partitions */
> >  	unsigned int			ctb_nparts;
> >  	/* partitions tables */
> > @@ -120,6 +124,10 @@ struct cfs_cpt_table {
> >   */
> >  int cfs_cpt_table_print(struct cfs_cpt_table *cptab, char *buf, int len);
> >  /**
> > + * print distance information of cpt-table
> > + */
> > +int cfs_cpt_distance_print(struct cfs_cpt_table *cptab, char *buf, int len);
> > +/**
> >   * return total number of CPU partitions in \a cptab
> >   */
> >  int
> > @@ -149,6 +157,10 @@ struct cfs_cpt_table {
> >   */
> >  int cfs_cpt_of_node(struct cfs_cpt_table *cptab, int node);
> >  /**
> > + * NUMA distance between \a cpt1 and \a cpt2 in \a cptab
> > + */
> > +unsigned int cfs_cpt_distance(struct cfs_cpt_table *cptab, int cpt1, int cpt2);
> > +/**
> >   * bind current thread on a CPU-partition \a cpt of \a cptab
> >   */
> >  int cfs_cpt_bind(struct cfs_cpt_table *cptab, int cpt);
> > @@ -206,6 +218,19 @@ void cfs_cpt_unset_nodemask(struct cfs_cpt_table *cptab,
> >  struct cfs_cpt_table;
> >  #define cfs_cpt_tab ((struct cfs_cpt_table *)NULL)
> >  
> > +static inline int cfs_cpt_distance_print(struct cfs_cpt_table *cptab,
> > +					 char *buf, int len)
> > +{
> > +	int rc;
> > +
> > +	rc = snprintf(buf, len, "0\t: 0:1\n");
> > +	len -= rc;
> > +	if (len <= 0)
> > +		return -EFBIG;
> > +
> > +	return rc;
> > +}
> > +
> >  static inline cpumask_var_t *
> >  cfs_cpt_cpumask(struct cfs_cpt_table *cptab, int cpt)
> >  {
> > @@ -241,6 +266,12 @@ void cfs_cpt_unset_nodemask(struct cfs_cpt_table *cptab,
> >  	return NULL;
> >  }
> >  
> > +static inline unsigned int cfs_cpt_distance(struct cfs_cpt_table *cptab,
> > +					    int cpt1, int cpt2)
> > +{
> > +	return 1;
> > +}
> > +
> >  static inline int
> >  cfs_cpt_set_cpu(struct cfs_cpt_table *cptab, int cpt, int cpu)
> >  {
> > diff --git a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
> > index 8c5cf7b..b315fb2 100644
> > --- a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
> > +++ b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c
> > @@ -128,6 +128,15 @@ struct cfs_cpt_table *
> >  					     GFP_NOFS);
> >  		if (!part->cpt_nodemask)
> >  			goto failed_setting_ctb_parts;
> > +
> > +		part->cpt_distance = kvmalloc_array(cptab->ctb_nparts,
> > +						    sizeof(part->cpt_distance[0]),
> > +						    GFP_KERNEL);
> > +		if (!part->cpt_distance)
> > +			goto failed_setting_ctb_parts;
> > +
> > +		memset(part->cpt_distance, -1,
> > +		       cptab->ctb_nparts * sizeof(part->cpt_distance[0]));
> >  	}
> >  
> >  	return cptab;
> > @@ -138,6 +147,7 @@ struct cfs_cpt_table *
> >  
> >  		kfree(part->cpt_nodemask);
> >  		free_cpumask_var(part->cpt_cpumask);
> > +		kvfree(part->cpt_distance);
> >  	}
> >  
> >  	kvfree(cptab->ctb_parts);
> > @@ -168,6 +178,7 @@ struct cfs_cpt_table *
> >  
> >  		kfree(part->cpt_nodemask);
> >  		free_cpumask_var(part->cpt_cpumask);
> > +		kvfree(part->cpt_distance);
> >  	}
> >  
> >  	kvfree(cptab->ctb_parts);
> > @@ -222,6 +233,44 @@ struct cfs_cpt_table *
> >  }
> >  EXPORT_SYMBOL(cfs_cpt_table_print);
> >  
> > +int cfs_cpt_distance_print(struct cfs_cpt_table *cptab, char *buf, int len)
> > +{
> > +	char *tmp = buf;
> > +	int rc;
> > +	int i;
> > +	int j;
> > +
> > +	for (i = 0; i < cptab->ctb_nparts; i++) {
> > +		if (len <= 0)
> > +			goto err;
> > +
> > +		rc = snprintf(tmp, len, "%d\t:", i);
> > +		len -= rc;
> > +
> > +		if (len <= 0)
> > +			goto err;
> > +
> > +		tmp += rc;
> > +		for (j = 0; j < cptab->ctb_nparts; j++) {
> > +			rc = snprintf(tmp, len, " %d:%d", j,
> > +				      cptab->ctb_parts[i].cpt_distance[j]);
> > +			len -= rc;
> > +			if (len <= 0)
> > +				goto err;
> > +			tmp += rc;
> > +		}
> > +
> > +		*tmp = '\n';
> > +		tmp++;
> > +		len--;
> > +	}
> > +
> > +	return tmp - buf;
> > +err:
> > +	return -E2BIG;
> > +}
> > +EXPORT_SYMBOL(cfs_cpt_distance_print);
> > +
> >  int
> >  cfs_cpt_number(struct cfs_cpt_table *cptab)
> >  {
> > @@ -273,6 +322,18 @@ struct cfs_cpt_table *
> >  }
> >  EXPORT_SYMBOL(cfs_cpt_nodemask);
> >  
> > +unsigned int cfs_cpt_distance(struct cfs_cpt_table *cptab, int cpt1, int cpt2)
> > +{
> > +	LASSERT(cpt1 == CFS_CPT_ANY || (cpt1 >= 0 && cpt1 < cptab->ctb_nparts));
> > +	LASSERT(cpt2 == CFS_CPT_ANY || (cpt2 >= 0 && cpt2 < cptab->ctb_nparts));
> > +
> > +	if (cpt1 == CFS_CPT_ANY || cpt2 == CFS_CPT_ANY)
> > +		return cptab->ctb_distance;
> > +
> > +	return cptab->ctb_parts[cpt1].cpt_distance[cpt2];
> > +}
> > +EXPORT_SYMBOL(cfs_cpt_distance);
> > +
> >  int
> >  cfs_cpt_set_cpu(struct cfs_cpt_table *cptab, int cpt, int cpu)
> >  {
> > -- 
> > 1.8.3.1
> 

From jsimmons at infradead.org  Tue Jun 26 01:16:33 2018
From: jsimmons at infradead.org (James Simmons)
Date: Tue, 26 Jun 2018 02:16:33 +0100 (BST)
Subject: [lustre-devel] [PATCH v2 4/7] lustre: libcfs: merge
 linux-tracefile.c into tracefile.c
In-Reply-To: <87y3f275ll.fsf@notabene.neil.brown.name>
References: <1529962976-15048-1-git-send-email-jsimmons@infradead.org>
 <1529962976-15048-5-git-send-email-jsimmons@infradead.org>
 <87y3f275ll.fsf@notabene.neil.brown.name>
Message-ID: <alpine.LFD.2.21.1806260215310.31196@casper.infradead.org>


> > From: NeilBrown <neilb at suse.com>
> >
> > It's good to keep related code together. The merger exposed the
> > flaws of cfs_print_to_console() so rework it to behavior properly.
> 
> I'm sorry but I cannot apply this as it is.
> The patch contains various changes that are not mentioned at all in the
> above description, and the above description doesn't really say anything
> useful at all about why anything needed fixing.
> 
> You've moved content from tracefile.h into tracefile.c
> You've reformatted some code - lining up '\' for example
> You've removed some code (e.g. cfs_tracefile_refill_stock)
> You've added a memset into libcfs_debug_vmsg2
>
> I don't doubt that these are useful, but they don't all belong in the
> same patch that claims to just move code from linux-tracefile.c into
> tracefile.c
> 
> Sometimes it is necessary to combine lots of things in one patch, though
> we avoid it whenever possible.  When it cannot be avoided, the multiple
> changes *must* be described together with some indication of the reason
> that they are all interdependent.
> 
> It is important that patches can be reviewed easily.  This one cannot.

No problem. I can break it up. Yeah the remaining cruft bothered me as I
was tacking down why it was broken.

> > Signed-off-by: James Simmons <jsimmons at infradead.org>
> > ---
> >  drivers/staging/lustre/lnet/libcfs/Makefile        |   2 +-
> >  .../staging/lustre/lnet/libcfs/linux-tracefile.c   | 194 ---------------------
> >  drivers/staging/lustre/lnet/libcfs/tracefile.c     | 184 ++++++++++++++++---
> >  drivers/staging/lustre/lnet/libcfs/tracefile.h     |  79 ++-------
> >  4 files changed, 170 insertions(+), 289 deletions(-)
> >  delete mode 100644 drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
> >
> > diff --git a/drivers/staging/lustre/lnet/libcfs/Makefile b/drivers/staging/lustre/lnet/libcfs/Makefile
> > index 5b13edc..cd96434 100644
> > --- a/drivers/staging/lustre/lnet/libcfs/Makefile
> > +++ b/drivers/staging/lustre/lnet/libcfs/Makefile
> > @@ -4,7 +4,7 @@ ccflags-y += -I$(srctree)/drivers/staging/lustre/lustre/include
> >  
> >  obj-$(CONFIG_LNET) += libcfs.o
> >  
> > -libcfs-obj-y += linux-tracefile.o linux-debug.o
> > +libcfs-obj-y += linux-debug.o
> >  libcfs-obj-y += linux-crypto.o
> >  libcfs-obj-y += linux-crypto-adler.o
> >  
> > diff --git a/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c b/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
> > deleted file mode 100644
> > index 3af7722..0000000
> > --- a/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
> > +++ /dev/null
> > @@ -1,194 +0,0 @@
> > -// SPDX-License-Identifier: GPL-2.0
> > -/*
> > - * GPL HEADER START
> > - *
> > - * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
> > - *
> > - * This program is free software; you can redistribute it and/or modify
> > - * it under the terms of the GNU General Public License version 2 only,
> > - * as published by the Free Software Foundation.
> > - *
> > - * This program is distributed in the hope that it will be useful, but
> > - * WITHOUT ANY WARRANTY; without even the implied warranty of
> > - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> > - * General Public License version 2 for more details (a copy is included
> > - * in the LICENSE file that accompanied this code).
> > - *
> > - * You should have received a copy of the GNU General Public License
> > - * version 2 along with this program; If not, see
> > - * http://www.gnu.org/licenses/gpl-2.0.html
> > - *
> > - * GPL HEADER END
> > - */
> > -/*
> > - * Copyright (c) 2008, 2010, Oracle and/or its affiliates. All rights reserved.
> > - * Use is subject to license terms.
> > - *
> > - * Copyright (c) 2012, Intel Corporation.
> > - */
> > -/*
> > - * This file is part of Lustre, http://www.lustre.org/
> > - * Lustre is a trademark of Sun Microsystems, Inc.
> > - */
> > -
> > -#define DEBUG_SUBSYSTEM S_LNET
> > -#define LUSTRE_TRACEFILE_PRIVATE
> > -
> > -#include <linux/slab.h>
> > -#include <linux/mm.h>
> > -#include "tracefile.h"
> > -
> > -/* percents to share the total debug memory for each type */
> > -static unsigned int pages_factor[CFS_TCD_TYPE_MAX] = {
> > -	80,  /* 80% pages for CFS_TCD_TYPE_PROC */
> > -	10,  /* 10% pages for CFS_TCD_TYPE_SOFTIRQ */
> > -	10   /* 10% pages for CFS_TCD_TYPE_IRQ */
> > -};
> > -
> > -char *cfs_trace_console_buffers[NR_CPUS][CFS_TCD_TYPE_MAX];
> > -
> > -int cfs_tracefile_init_arch(void)
> > -{
> > -	int i;
> > -	int j;
> > -	struct cfs_trace_cpu_data *tcd;
> > -
> > -	/* initialize trace_data */
> > -	memset(cfs_trace_data, 0, sizeof(cfs_trace_data));
> > -	for (i = 0; i < CFS_TCD_TYPE_MAX; i++) {
> > -		cfs_trace_data[i] =
> > -			kmalloc_array(num_possible_cpus(),
> > -				      sizeof(union cfs_trace_data_union),
> > -				      GFP_KERNEL);
> > -		if (!cfs_trace_data[i])
> > -			goto out;
> > -	}
> > -
> > -	/* arch related info initialized */
> > -	cfs_tcd_for_each(tcd, i, j) {
> > -		spin_lock_init(&tcd->tcd_lock);
> > -		tcd->tcd_pages_factor = pages_factor[i];
> > -		tcd->tcd_type = i;
> > -		tcd->tcd_cpu = j;
> > -	}
> > -
> > -	for (i = 0; i < num_possible_cpus(); i++)
> > -		for (j = 0; j < 3; j++) {
> > -			cfs_trace_console_buffers[i][j] =
> > -				kmalloc(CFS_TRACE_CONSOLE_BUFFER_SIZE,
> > -					GFP_KERNEL);
> > -
> > -			if (!cfs_trace_console_buffers[i][j])
> > -				goto out;
> > -		}
> > -
> > -	return 0;
> > -
> > -out:
> > -	cfs_tracefile_fini_arch();
> > -	pr_err("lnet: Not enough memory\n");
> > -	return -ENOMEM;
> > -}
> > -
> > -void cfs_tracefile_fini_arch(void)
> > -{
> > -	int i;
> > -	int j;
> > -
> > -	for (i = 0; i < num_possible_cpus(); i++)
> > -		for (j = 0; j < 3; j++) {
> > -			kfree(cfs_trace_console_buffers[i][j]);
> > -			cfs_trace_console_buffers[i][j] = NULL;
> > -		}
> > -
> > -	for (i = 0; cfs_trace_data[i]; i++) {
> > -		kfree(cfs_trace_data[i]);
> > -		cfs_trace_data[i] = NULL;
> > -	}
> > -}
> > -
> > -enum cfs_trace_buf_type cfs_trace_buf_idx_get(void)
> > -{
> > -	if (in_irq())
> > -		return CFS_TCD_TYPE_IRQ;
> > -	if (in_softirq())
> > -		return CFS_TCD_TYPE_SOFTIRQ;
> > -	return CFS_TCD_TYPE_PROC;
> > -}
> > -
> > -void
> > -cfs_set_ptldebug_header(struct ptldebug_header *header,
> > -			struct libcfs_debug_msg_data *msgdata,
> > -			unsigned long stack)
> > -{
> > -	struct timespec64 ts;
> > -
> > -	ktime_get_real_ts64(&ts);
> > -
> > -	header->ph_subsys = msgdata->msg_subsys;
> > -	header->ph_mask = msgdata->msg_mask;
> > -	header->ph_cpu_id = smp_processor_id();
> > -	header->ph_type = cfs_trace_buf_idx_get();
> > -	/* y2038 safe since all user space treats this as unsigned, but
> > -	 * will overflow in 2106
> > -	 */
> > -	header->ph_sec = (u32)ts.tv_sec;
> > -	header->ph_usec = ts.tv_nsec / NSEC_PER_USEC;
> > -	header->ph_stack = stack;
> > -	header->ph_pid = current->pid;
> > -	header->ph_line_num = msgdata->msg_line;
> > -	header->ph_extern_pid = 0;
> > -}
> > -
> > -static char *
> > -dbghdr_to_err_string(struct ptldebug_header *hdr)
> > -{
> > -	switch (hdr->ph_subsys) {
> > -	case S_LND:
> > -	case S_LNET:
> > -		return "LNetError";
> > -	default:
> > -		return "LustreError";
> > -	}
> > -}
> > -
> > -static char *
> > -dbghdr_to_info_string(struct ptldebug_header *hdr)
> > -{
> > -	switch (hdr->ph_subsys) {
> > -	case S_LND:
> > -	case S_LNET:
> > -		return "LNet";
> > -	default:
> > -		return "Lustre";
> > -	}
> > -}
> > -
> > -void cfs_print_to_console(struct ptldebug_header *hdr, int mask,
> > -			  const char *buf, int len, const char *file,
> > -			  const char *fn)
> > -{
> > -	char *prefix = "Lustre", *ptype = NULL;
> > -
> > -	if (mask & D_EMERG) {
> > -		prefix = dbghdr_to_err_string(hdr);
> > -		ptype = KERN_EMERG;
> > -	} else if (mask & D_ERROR) {
> > -		prefix = dbghdr_to_err_string(hdr);
> > -		ptype = KERN_ERR;
> > -	} else if (mask & D_WARNING) {
> > -		prefix = dbghdr_to_info_string(hdr);
> > -		ptype = KERN_WARNING;
> > -	} else if (mask & (D_CONSOLE | libcfs_printk)) {
> > -		prefix = dbghdr_to_info_string(hdr);
> > -		ptype = KERN_INFO;
> > -	}
> > -
> > -	if (mask & D_CONSOLE) {
> > -		pr_info("%s%s: %.*s", ptype, prefix, len, buf);
> > -	} else {
> > -		pr_info("%s%s: %d:%d:(%s:%d:%s()) %.*s", ptype, prefix,
> > -			hdr->ph_pid, hdr->ph_extern_pid, file,
> > -			hdr->ph_line_num, fn, len, buf);
> > -	}
> > -}
> > diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.c b/drivers/staging/lustre/lnet/libcfs/tracefile.c
> > index 6d567a9..229bbb8 100644
> > --- a/drivers/staging/lustre/lnet/libcfs/tracefile.c
> > +++ b/drivers/staging/lustre/lnet/libcfs/tracefile.c
> > @@ -37,8 +37,6 @@
> >   */
> >  
> >  #define DEBUG_SUBSYSTEM S_LNET
> > -#define LUSTRE_TRACEFILE_PRIVATE
> > -#define pr_fmt(fmt) "Lustre: " fmt
> >  
> >  #include <linux/ratelimit.h>
> >  #include <linux/highmem.h>
> > @@ -49,9 +47,19 @@
> >  #include <linux/uaccess.h>
> >  #include "tracefile.h"
> >  
> > -/* XXX move things up to the top, comment */
> > +#define CFS_TRACE_CONSOLE_BUFFER_SIZE	1024
> > +#define TCD_MAX_TYPES			8
> > +
> > +enum cfs_trace_buf_type {
> > +	CFS_TCD_TYPE_PROC = 0,
> > +	CFS_TCD_TYPE_SOFTIRQ,
> > +	CFS_TCD_TYPE_IRQ,
> > +	CFS_TCD_TYPE_MAX
> > +};
> > +
> >  union cfs_trace_data_union (*cfs_trace_data[TCD_MAX_TYPES])[NR_CPUS] __cacheline_aligned;
> >  
> > +char *cfs_trace_console_buffers[NR_CPUS][CFS_TCD_TYPE_MAX];
> >  char cfs_tracefile[TRACEFILE_NAME_SIZE];
> >  long long cfs_tracefile_size = CFS_TRACEFILE_SIZE;
> >  static struct tracefiled_ctl trace_tctl;
> > @@ -145,11 +153,35 @@ void cfs_trace_unlock_tcd(struct cfs_trace_cpu_data *tcd, int walking)
> >  		spin_unlock(&tcd->tcd_lock);
> >  }
> >  
> > -#define cfs_tcd_for_each_type_lock(tcd, i, cpu)			   \
> > +#define cfs_tcd_for_each(tcd, i, j)					\
> > +	for (i = 0; cfs_trace_data[i]; i++)				\
> > +		for (j = 0, ((tcd) = &(*cfs_trace_data[i])[j].tcd);	\
> > +		     j < num_possible_cpus();				\
> > +		     j++, (tcd) = &(*cfs_trace_data[i])[j].tcd)
> > +
> > +
> > +#define cfs_tcd_for_each_type_lock(tcd, i, cpu)				\
> >  	for (i = 0; cfs_trace_data[i] &&				\
> >  	     (tcd = &(*cfs_trace_data[i])[cpu].tcd) &&			\
> >  	     cfs_trace_lock_tcd(tcd, 1); cfs_trace_unlock_tcd(tcd, 1), i++)
> >  
> > +enum cfs_trace_buf_type cfs_trace_buf_idx_get(void)
> > +{
> > +	if (in_irq())
> > +		return CFS_TCD_TYPE_IRQ;
> > +	if (in_softirq())
> > +		return CFS_TCD_TYPE_SOFTIRQ;
> > +	return CFS_TCD_TYPE_PROC;
> > +}
> > +
> > +static inline char *cfs_trace_get_console_buffer(void)
> > +{
> > +	unsigned int i = get_cpu();
> > +	unsigned int j = cfs_trace_buf_idx_get();
> > +
> > +	return cfs_trace_console_buffers[i][j];
> > +}
> > +
> >  static inline struct cfs_trace_cpu_data *
> >  cfs_trace_get_tcd(void)
> >  {
> > @@ -168,6 +200,77 @@ static inline void cfs_trace_put_tcd(struct cfs_trace_cpu_data *tcd)
> >  	put_cpu();
> >  }
> >  
> > +/* percents to share the total debug memory for each type */
> > +static unsigned int pages_factor[CFS_TCD_TYPE_MAX] = {
> > +	80,  /* 80% pages for CFS_TCD_TYPE_PROC */
> > +	10,  /* 10% pages for CFS_TCD_TYPE_SOFTIRQ */
> > +	10   /* 10% pages for CFS_TCD_TYPE_IRQ */
> > +};
> > +
> > +int cfs_tracefile_init_arch(void)
> > +{
> > +	struct cfs_trace_cpu_data *tcd;
> > +	int i;
> > +	int j;
> > +
> > +	/* initialize trace_data */
> > +	memset(cfs_trace_data, 0, sizeof(cfs_trace_data));
> > +	for (i = 0; i < CFS_TCD_TYPE_MAX; i++) {
> > +		cfs_trace_data[i] =
> > +			kmalloc_array(num_possible_cpus(),
> > +				      sizeof(union cfs_trace_data_union),
> > +				      GFP_KERNEL);
> > +		if (!cfs_trace_data[i])
> > +			goto out_trace_data;
> > +	}
> > +
> > +	/* arch related info initialized */
> > +	cfs_tcd_for_each(tcd, i, j) {
> > +		spin_lock_init(&tcd->tcd_lock);
> > +		tcd->tcd_pages_factor = pages_factor[i];
> > +		tcd->tcd_type = i;
> > +		tcd->tcd_cpu = j;
> > +	}
> > +
> > +	for (i = 0; i < num_possible_cpus(); i++)
> > +		for (j = 0; j < 3; j++) {
> > +			cfs_trace_console_buffers[i][j] =
> > +				kmalloc(CFS_TRACE_CONSOLE_BUFFER_SIZE,
> > +					GFP_KERNEL);
> > +
> > +			if (!cfs_trace_console_buffers[i][j])
> > +				goto out_buffers;
> > +		}
> > +
> > +	return 0;
> > +
> > +out_buffers:
> > +	for (i = 0; cfs_trace_data[i]; i++) {
> > +		kfree(cfs_trace_data[i]);
> > +		cfs_trace_data[i] = NULL;
> > +	}
> > +out_trace_data:
> > +	pr_err("lnet: Not enough memory\n");
> > +	return -ENOMEM;
> > +}
> > +
> > +void cfs_tracefile_fini_arch(void)
> > +{
> > +	int i;
> > +	int j;
> > +
> > +	for (i = 0; i < num_possible_cpus(); i++)
> > +		for (j = 0; j < 3; j++) {
> > +			kfree(cfs_trace_console_buffers[i][j]);
> > +			cfs_trace_console_buffers[i][j] = NULL;
> > +		}
> > +
> > +	for (i = 0; cfs_trace_data[i]; i++) {
> > +		kfree(cfs_trace_data[i]);
> > +		cfs_trace_data[i] = NULL;
> > +	}
> > +}
> > +
> >  static inline struct cfs_trace_page *
> >  cfs_tage_from_list(struct list_head *list)
> >  {
> > @@ -216,27 +319,6 @@ static void cfs_tage_to_tail(struct cfs_trace_page *tage,
> >  	list_move_tail(&tage->linkage, queue);
> >  }
> >  
> > -int cfs_trace_refill_stock(struct cfs_trace_cpu_data *tcd, gfp_t gfp,
> > -			   struct list_head *stock)
> > -{
> > -	int i;
> > -
> > -	/*
> > -	 * XXX nikita: do NOT call portals_debug_msg() (CDEBUG/ENTRY/EXIT)
> > -	 * from here: this will lead to infinite recursion.
> > -	 */
> > -
> > -	for (i = 0; i + tcd->tcd_cur_stock_pages < TCD_STOCK_PAGES ; ++i) {
> > -		struct cfs_trace_page *tage;
> > -
> > -		tage = cfs_tage_alloc(gfp);
> > -		if (!tage)
> > -			break;
> > -		list_add_tail(&tage->linkage, stock);
> > -	}
> > -	return i;
> > -}
> > -
> >  /* return a page that has 'len' bytes left at the end */
> >  static struct cfs_trace_page *
> >  cfs_trace_get_tage_try(struct cfs_trace_cpu_data *tcd, unsigned long len)
> > @@ -340,6 +422,57 @@ static struct cfs_trace_page *cfs_trace_get_tage(struct cfs_trace_cpu_data *tcd,
> >  	return tage;
> >  }
> >  
> > +
> > +
> > +static void cfs_set_ptldebug_header(struct ptldebug_header *header,
> > +				    struct libcfs_debug_msg_data *msgdata,
> > +				    unsigned long stack)
> > +{
> > +	struct timespec64 ts;
> > +
> > +	ktime_get_real_ts64(&ts);
> > +
> > +	header->ph_subsys = msgdata->msg_subsys;
> > +	header->ph_mask = msgdata->msg_mask;
> > +	header->ph_cpu_id = smp_processor_id();
> > +	header->ph_type = cfs_trace_buf_idx_get();
> > +	/* y2038 safe since all user space treats this as unsigned, but
> > +	 * will overflow in 2106
> > +	 */
> > +	header->ph_sec = (u32)ts.tv_sec;
> > +	header->ph_usec = ts.tv_nsec / NSEC_PER_USEC;
> > +	header->ph_stack = stack;
> > +	header->ph_pid = current->pid;
> > +	header->ph_line_num = msgdata->msg_line;
> > +	header->ph_extern_pid = 0;
> > +}
> > +
> > +static void cfs_print_to_console(struct ptldebug_header *hdr, int mask,
> > +				 const char *buf, int len, const char *file,
> > +				 const char *fn)
> > +{
> > +	char *prefix = "Lustre";
> > +
> > +	if (hdr->ph_subsys == S_LND || hdr->ph_subsys == S_LNET)
> > +		prefix = "LNet";
> > +
> > +	if (mask & D_EMERG) {
> > +		pr_emerg("%sError: %d:%d:(%s:%d:%s()) %.*s", prefix,
> > +			 hdr->ph_pid, hdr->ph_extern_pid, file,
> > +			 hdr->ph_line_num, fn, len, buf);
> > +	} else if (mask & D_ERROR) {
> > +		pr_err("%sError: %d:%d:(%s:%d:%s()) %.*s", prefix,
> > +		       hdr->ph_pid, hdr->ph_extern_pid, file,
> > +		       hdr->ph_line_num, fn, len, buf);
> > +	} else if (mask & D_WARNING) {
> > +		pr_warning("%s: %d:%d:(%s:%d:%s()) %.*s", prefix,
> > +			   hdr->ph_pid, hdr->ph_extern_pid, file,
> > +			   hdr->ph_line_num, fn, len, buf);
> > +	} else if (mask & (D_CONSOLE | libcfs_printk)) {
> > +		pr_info("%s: %.*s", prefix, len, buf);
> > +	}
> > +}
> > +
> >  int libcfs_debug_msg(struct libcfs_debug_msg_data *msgdata,
> >  		     const char *format, ...)
> >  {
> > @@ -531,6 +664,7 @@ int libcfs_debug_vmsg2(struct libcfs_debug_msg_data *msgdata,
> >  	} else {
> >  		string_buf = cfs_trace_get_console_buffer();
> >  
> > +		memset(string_buf, 0, CFS_TRACE_CONSOLE_BUFFER_SIZE);
> >  		needed = 0;
> >  		if (format1) {
> >  			va_copy(ap, args);
> > diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.h b/drivers/staging/lustre/lnet/libcfs/tracefile.h
> > index f49a9ba..4c79009 100644
> > --- a/drivers/staging/lustre/lnet/libcfs/tracefile.h
> > +++ b/drivers/staging/lustre/lnet/libcfs/tracefile.h
> > @@ -42,13 +42,6 @@
> >  #include <linux/smp.h>
> >  #include <linux/libcfs/libcfs.h>
> >  
> > -enum cfs_trace_buf_type {
> > -	CFS_TCD_TYPE_PROC = 0,
> > -	CFS_TCD_TYPE_SOFTIRQ,
> > -	CFS_TCD_TYPE_IRQ,
> > -	CFS_TCD_TYPE_MAX
> > -};
> > -
> >  #define TRACEFILE_NAME_SIZE 1024
> >  extern char cfs_tracefile[TRACEFILE_NAME_SIZE];
> >  extern long long cfs_tracefile_size;
> > @@ -60,9 +53,6 @@ enum cfs_trace_buf_type {
> >  
> >  void libcfs_run_debug_log_upcall(char *file);
> >  
> > -int  cfs_tracefile_init_arch(void);
> > -void cfs_tracefile_fini_arch(void);
> > -
> >  int cfs_tracefile_dump_all_pages(char *filename);
> >  void cfs_trace_debug_print(void);
> >  void cfs_trace_flush_pages(void);
> > @@ -92,22 +82,6 @@ int cfs_trace_copyout_string(char __user *usr_buffer, int usr_buffer_nob,
> >  #define TCD_STOCK_PAGES (TCD_MAX_PAGES)
> >  #define CFS_TRACEFILE_SIZE (500 << 20)
> >  
> > -#ifdef LUSTRE_TRACEFILE_PRIVATE
> > -
> > -/*
> > - * Private declare for tracefile
> > - */
> > -#define TCD_MAX_PAGES (5 << (20 - PAGE_SHIFT))
> > -#define TCD_STOCK_PAGES (TCD_MAX_PAGES)
> > -
> > -#define CFS_TRACEFILE_SIZE (500 << 20)
> > -
> > -/*
> > - * Size of a buffer for sprinting console messages if we can't get a page
> > - * from system
> > - */
> > -#define CFS_TRACE_CONSOLE_BUFFER_SIZE   1024
> > -
> >  union cfs_trace_data_union {
> >  	struct cfs_trace_cpu_data {
> >  		/*
> > @@ -184,58 +158,25 @@ int cfs_trace_copyout_string(char __user *usr_buffer, int usr_buffer_nob,
> >  	char __pad[L1_CACHE_ALIGN(sizeof(struct cfs_trace_cpu_data))];
> >  };
> >  
> > -#define TCD_MAX_TYPES      8
> > -extern union cfs_trace_data_union (*cfs_trace_data[TCD_MAX_TYPES])[NR_CPUS];
> > -
> > -#define cfs_tcd_for_each(tcd, i, j)				       \
> > -	for (i = 0; cfs_trace_data[i]; i++)				\
> > -		for (j = 0, ((tcd) = &(*cfs_trace_data[i])[j].tcd);	\
> > -		     j < num_possible_cpus();				 \
> > -		     j++, (tcd) = &(*cfs_trace_data[i])[j].tcd)
> > -
> > -void cfs_set_ptldebug_header(struct ptldebug_header *header,
> > -			     struct libcfs_debug_msg_data *m,
> > -			     unsigned long stack);
> > -void cfs_print_to_console(struct ptldebug_header *hdr, int mask,
> > -			  const char *buf, int len, const char *file,
> > -			  const char *fn);
> > -
> > -extern char *cfs_trace_console_buffers[NR_CPUS][CFS_TCD_TYPE_MAX];
> > -enum cfs_trace_buf_type cfs_trace_buf_idx_get(void);
> > -
> > -static inline char *
> > -cfs_trace_get_console_buffer(void)
> > -{
> > -	unsigned int i = get_cpu();
> > -	unsigned int j = cfs_trace_buf_idx_get();
> > -
> > -	return cfs_trace_console_buffers[i][j];
> > -}
> > -
> > -int cfs_trace_refill_stock(struct cfs_trace_cpu_data *tcd, gfp_t gfp,
> > -			   struct list_head *stock);
> > -
> >  void cfs_trace_assertion_failed(const char *str,
> >  				struct libcfs_debug_msg_data *m);
> >  
> >  /* ASSERTION that is safe to use within the debug system */
> > -#define __LASSERT(cond)						 \
> > -do {								    \
> > +#define __LASSERT(cond)							\
> > +do {									\
> >  	if (unlikely(!(cond))) {					\
> >  		LIBCFS_DEBUG_MSG_DATA_DECL(msgdata, D_EMERG, NULL);     \
> >  		cfs_trace_assertion_failed("ASSERTION("#cond") failed", \
> > -					   &msgdata);		   \
> > -	}							       \
> > +					   &msgdata);			\
> > +	}								\
> >  } while (0)
> >  
> > -#define __LASSERT_TAGE_INVARIANT(tage)				  \
> > -do {								    \
> > -	__LASSERT(tage);					\
> > -	__LASSERT(tage->page);				  \
> > -	__LASSERT(tage->used <= PAGE_SIZE);			 \
> > -	__LASSERT(page_count(tage->page) > 0);		      \
> > +#define __LASSERT_TAGE_INVARIANT(tage)			\
> > +do {							\
> > +	__LASSERT(tage);				\
> > +	__LASSERT(tage->page);				\
> > +	__LASSERT(tage->used <= PAGE_SIZE);		\
> > +	__LASSERT(page_count(tage->page) > 0);		\
> >  } while (0)
> >  
> > -#endif	/* LUSTRE_TRACEFILE_PRIVATE */
> > -
> >  #endif /* __LIBCFS_TRACEFILE_H__ */
> > -- 
> > 1.8.3.1
> 

From paf at cray.com  Tue Jun 26 13:51:13 2018
From: paf at cray.com (Patrick Farrell)
Date: Tue, 26 Jun 2018 13:51:13 +0000
Subject: [lustre-devel] [PATCH 00/24] lustre - more cleanups including
 module reduction.
In-Reply-To: <alpine.LFD.2.21.1806260206360.31196@casper.infradead.org>
References: <152904663333.10587.10934053155404014785.stgit@noble>
 <alpine.LFD.2.21.1806210239190.3886@casper.infradead.org>
 <DM3PR1101MB1213DDB7FC857EE6B1444153CB760@DM3PR1101MB1213.namprd11.prod.outlook.com>
 <alpine.LFD.2.21.1806210354260.3886@casper.infradead.org>
 <3FD4D051-9C95-449D-8A23-D42B271E55B8@dilger.ca>
 <DM3PR1101MB121322E85FDE1E5310F6F4B7CB760@DM3PR1101MB1213.namprd11.prod.outlook.com>
 <87tvprah32.fsf@notabene.neil.brown.name>,
 <alpine.LFD.2.21.1806260206360.31196@casper.infradead.org>
Message-ID: <DM3PR1101MB121358672276198B843DE5A7CB490@DM3PR1101MB1213.namprd11.prod.outlook.com>

Ah, sorry, lost your mail in the shuffle, Neil.

The name, mostly.  ptlrpc is one subdirectory and one subsystem, so I don’t want to use it for a module that explicitly includes all of several.  I’m not sure of a better name immediately - is this module intended to be shared between client and server?


________________________________
From: James Simmons <jsimmons at infradead.org>
Sent: Monday, June 25, 2018 8:13:22 PM
To: NeilBrown
Cc: Patrick Farrell; Andreas Dilger; Oleg Drokin; Lustre Development List
Subject: Re: [lustre-devel] [PATCH 00/24] lustre - more cleanups including module reduction.


> > Ah, thanks Andreas.  Perhaps not coincidentally, Lustre 2.4 is the first release I worked on.
> >
> > Neil, I am really unenamored of the idea of the shared ptlrpc ldlm module being named ptlrpc...
>
> Can you say why?  Is it the choice of name that bothers you, or the
> combining of two things into the one kernel module, or something else?
> Currently in  git://git.hpdd.intel.com/fs/lustre-release.git
> (or the git tree that was until recently at the above address),
> the module named "ptlrpc" contains ptlrpc code, ldlm code, and target
> code.  I wonder what "target" means in this context.

target is the server code built into ptlrpc which is for the "Unified
Target". Its the shared code that is used by both the MDS and OSS code.
Andreas or Oleg can add more details than this. BTW also the nodemap
code also get built into ptlrpc module as well for the server side.

> > Also, Lustre currently has a bunch of module parameters which are used for configuration.  Thoughts on that?
>
> Yes, the module parameters are an interesting part of the story.
> libcfs has a bunch of module parameters that are symlinked from debugfs.
> It seems that user-space largely uses the debugfs links to access them,
> so they can become part of the "lnet" module with minimal pain.
> Other modules have parameters that I haven't yet looked in to.
>
> There probably will need to be user-space changes to support reduction
> in the number of modules.

The test suite will need some changes since it loads the modules. We
need to handle the case if its built into the kernel as well.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180626/bc8c6124/attachment.html>

From neilb at suse.com  Wed Jun 27 02:44:37 2018
From: neilb at suse.com (NeilBrown)
Date: Wed, 27 Jun 2018 12:44:37 +1000
Subject: [lustre-devel] [PATCH v3 07/26] staging: lustre: libcfs: NUMA
 support
In-Reply-To: <A764DED1-A9A8-4FA5-A0F1-922C3E2D822A@cray.com>
References: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
 <1529875250-11531-8-git-send-email-jsimmons@infradead.org>
 <877emnaed8.fsf@notabene.neil.brown.name>
 <A764DED1-A9A8-4FA5-A0F1-922C3E2D822A@cray.com>
Message-ID: <87lgb16j8q.fsf@notabene.neil.brown.name>

On Mon, Jun 25 2018, Doug Oucharek wrote:

> Some background on this NUMA change:
>
> First off, this is just a first step to a bigger set of changes which include changes to the Lustre utilities.  This was done as part of the Multi-Rail feature.  One of the systems that feature is meant to support is the SGI UV system (now HPE) which has a massive number of NUMA nodes connected by a NUMA Link.  There are multiple fabric cards spread throughout the system and Multi-Rail needs to know which fabric cards are nearest to the NUMA node we are running on.  To do that, the “distance” between NUMA nodes needs to be configured.
>
> This patch is preparing the infrastructure for the Multi-Rail feature to support configuring NUMA node distances.  Technically, this patch should be landing with the Multi-Rail feature (still to be pushed) for it to make proper sense.
>

Thanks a lot for the background.

If these NUMA nodes have a 'distance' between them, and if lustre can
benefit from knowing the distance, then is seems likely that other code
might also benefit.  In that case it would be best if the distance were
encoded in some global state information so that lustre and any other
subsystem can extract it.

Do you know if there is any work underway by anyone to make this
information generally available?  If there is, we should make sure that
lustre works in a compatible way so that once that work lands, lustre
can use it directly and not need extra configuration.
If no such work is underway, then it would be really good if something
were done in that direction.  If no-one here is able to work on this, I
can ask around in SUSE and see if anyone here knows anything relevant.

Thanks,
NeilBrown
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180627/6b26a3de/attachment.sig>

From neilb at suse.com  Wed Jun 27 02:49:07 2018
From: neilb at suse.com (NeilBrown)
Date: Wed, 27 Jun 2018 12:49:07 +1000
Subject: [lustre-devel] [PATCH v3 19/26] staging: lustre: libcfs: report
 NUMA node instead of just node
In-Reply-To: <alpine.LFD.2.21.1806260146520.31196@casper.infradead.org>
References: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
 <1529875250-11531-20-git-send-email-jsimmons@infradead.org>
 <87y3f38ye6.fsf@notabene.neil.brown.name>
 <alpine.LFD.2.21.1806260146520.31196@casper.infradead.org>
Message-ID: <87in656j18.fsf@notabene.neil.brown.name>

On Tue, Jun 26 2018, James Simmons wrote:

>> On Sun, Jun 24 2018, James Simmons wrote:
>> 
>> > From: Dmitry Eremin <dmitry.eremin at intel.com>
>> >
>> > Reporting "HW nodes" is too generic. It really is reporting
>> > "HW NUMA nodes". Update the debug message.
>> 
>> I'm not happy with this patch description.....
>
> How about:
>
> In the HPC world a node refers to the actual whole computer system
> used in a cluster. Reporting just "HW nodes" is not clear so change
> the debug report to "HW NUMA nodes" since this report the number
> of NUMA nodes in use by Lustre.
>

That's not actually the part I was complaining about.
It was the fact that there were multiple parts of the patch that weren't
mentioned at all.

But I like your revision anyway, so I've updated the patch.
Thanks!

NeilBrown
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180627/05d1e08e/attachment.sig>

From neilb at suse.com  Wed Jun 27 02:50:54 2018
From: neilb at suse.com (NeilBrown)
Date: Wed, 27 Jun 2018 12:50:54 +1000
Subject: [lustre-devel] [PATCH v3 08/26] staging: lustre: libcfs: add
 cpu distance handling
In-Reply-To: <alpine.LFD.2.21.1806260213340.31196@casper.infradead.org>
References: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
 <1529875250-11531-9-git-send-email-jsimmons@infradead.org>
 <874lhradxj.fsf@notabene.neil.brown.name>
 <alpine.LFD.2.21.1806260213340.31196@casper.infradead.org>
Message-ID: <87fu196iy9.fsf@notabene.neil.brown.name>

On Tue, Jun 26 2018, James Simmons wrote:

>> On Sun, Jun 24 2018, James Simmons wrote:
>> 
>> > From: Amir Shehata <amir.shehata at intel.com>
>> >
>> > Add functionality to calculate the distance between two CPTs.
>> > Expose those distance in debugfs so people deploying a setup
>> > can debug what is being created for CPTs.
>> 
>> This patch doesn't expose anything in debugfs - a later patch
>> does that.
>> So I've changed the comment to "Prepare to expose those ...."
>
> Doug Oucharek recommonds the following commit message body:
>
> Add cpu distance routines which will be used by the Multi-Rail feature 
> to determine what fabric interface is nearest to the core we are currently 
> running on. Configuration of these distances will be provided from user 
> space via configuration routines in Lustre.

Thanks - I like this more.  Updated.

NeilBrown
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180627/cc1c67b7/attachment.sig>

From neilb at suse.com  Wed Jun 27 03:08:41 2018
From: neilb at suse.com (NeilBrown)
Date: Wed, 27 Jun 2018 13:08:41 +1000
Subject: [lustre-devel] [PATCH 00/24] lustre - more cleanups including
 module reduction.
In-Reply-To: <DM3PR1101MB121358672276198B843DE5A7CB490@DM3PR1101MB1213.namprd11.prod.outlook.com>
References: <152904663333.10587.10934053155404014785.stgit@noble>
 <alpine.LFD.2.21.1806210239190.3886@casper.infradead.org>
 <DM3PR1101MB1213DDB7FC857EE6B1444153CB760@DM3PR1101MB1213.namprd11.prod.outlook.com>
 <alpine.LFD.2.21.1806210354260.3886@casper.infradead.org>
 <3FD4D051-9C95-449D-8A23-D42B271E55B8@dilger.ca>
 <DM3PR1101MB121322E85FDE1E5310F6F4B7CB760@DM3PR1101MB1213.namprd11.prod.outlook.com>
 <87tvprah32.fsf@notabene.neil.brown.name>
 <alpine.LFD.2.21.1806260206360.31196@casper.infradead.org>
 <DM3PR1101MB121358672276198B843DE5A7CB490@DM3PR1101MB1213.namprd11.prod.outlook.com>
Message-ID: <87d0wd6i4m.fsf@notabene.neil.brown.name>

On Tue, Jun 26 2018, Patrick Farrell wrote:

> Ah, sorry, lost your mail in the shuffle, Neil.
>
> The name, mostly.  ptlrpc is one subdirectory and one subsystem, so I don’t want to use it for a module that explicitly includes all of several.  I’m not sure of a better name immediately - is this module intended to be shared between client and server?
>

Thanks for clarifying.
Note that I'm still feeling my way around here so "intended" can at most
be a soft intention, open to change.  But yes, I expect this module
would ultimately be shared with the server.

I think things should only be in different modules if it might sometimes
make sense to use one without the other.

So everything that is always used for a lustre client mount, and is only
used for a lustre client mount, should be in a module called "lustre".
And everything always and only used for a lustre server should be in a
module called (something like) "lustre-server".

It appears to me that ptlrpc, ldlm, and obdclass are always and only
used together, but can be used for client or server.  So I think they
should be together in a module.
I'm tempted to add lnet to that list.  There has been mention that cray
has some other functionality that uses lnet - if that doesn't use
ptlrpc, then maybe that is a case for keeping lnet separate.  However I
would rather see it as a potential case to separate lnet from the others
at a later date if that functionality from cray ever goes into mainline.

The klnds modules can presumably be used independently(?) so they can
sensibly remain as separate modules, though I'm beginning to wonder if
lnet can actually function without socklnd, so I wonder if that should
be permanently part of the lnet module (with NFS, the xprt_sock code is
a permanent part of "sunrpc", while the xprt_rdma (aka rpcrdma) code
can be a separate module).

In the interests of a concrete strawman, what objections would I get if
I suggested that ptlrpc, ldlm, obdclass, lnet, and socklnd were all included in
the one module named "lnet" ??

Thanks,
NeilBrown
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180627/02d56082/attachment.sig>

From spitzcor at cray.com  Wed Jun 27 04:00:28 2018
From: spitzcor at cray.com (Cory Spitz)
Date: Wed, 27 Jun 2018 04:00:28 +0000
Subject: [lustre-devel] [PATCH 00/24] lustre - more cleanups including
 module reduction.
In-Reply-To: <87d0wd6i4m.fsf@notabene.neil.brown.name>
References: <152904663333.10587.10934053155404014785.stgit@noble>
 <alpine.LFD.2.21.1806210239190.3886@casper.infradead.org>
 <DM3PR1101MB1213DDB7FC857EE6B1444153CB760@DM3PR1101MB1213.namprd11.prod.outlook.com>
 <alpine.LFD.2.21.1806210354260.3886@casper.infradead.org>
 <3FD4D051-9C95-449D-8A23-D42B271E55B8@dilger.ca>
 <DM3PR1101MB121322E85FDE1E5310F6F4B7CB760@DM3PR1101MB1213.namprd11.prod.outlook.com>
 <87tvprah32.fsf@notabene.neil.brown.name>
 <alpine.LFD.2.21.1806260206360.31196@casper.infradead.org>
 <DM3PR1101MB121358672276198B843DE5A7CB490@DM3PR1101MB1213.namprd11.prod.outlook.com>
 <87d0wd6i4m.fsf@notabene.neil.brown.name>
Message-ID: <913847E3-EB34-4B9D-9314-0BCF59837AFC@cray.com>

Hello, Neil.

Cray does indeed have another user of LNet.  While it is now GPL, I can't say if it will ever be in mainline.  However, even without it there is a case for LNet to live on its own as Andreas pointed out recently with http://lists.lustre.org/pipermail/lustre-devel-lustre.org/2018-June/007098.html.

But, if you want to group everything together now, we can hopefully tease it apart again in the future.  If so, I think that it would be misleading to call a module lnet if it contained distributed locking functionality (ldlm).  How about lustre-common, lustre-core, or some such?

Thanks,
-Cory

-- 

On 6/26/18, 10:09 PM, "lustre-devel on behalf of NeilBrown" <lustre-devel-bounces at lists.lustre.org on behalf of neilb at suse.com> wrote:

    On Tue, Jun 26 2018, Patrick Farrell wrote:
    
    > Ah, sorry, lost your mail in the shuffle, Neil.
    >
    > The name, mostly.  ptlrpc is one subdirectory and one subsystem, so I don’t want to use it for a module that explicitly includes all of several.  I’m not sure of a better name immediately - is this module intended to be shared between client and server?
    >
    
    Thanks for clarifying.
    Note that I'm still feeling my way around here so "intended" can at most
    be a soft intention, open to change.  But yes, I expect this module
    would ultimately be shared with the server.
    
    I think things should only be in different modules if it might sometimes
    make sense to use one without the other.
    
    So everything that is always used for a lustre client mount, and is only
    used for a lustre client mount, should be in a module called "lustre".
    And everything always and only used for a lustre server should be in a
    module called (something like) "lustre-server".
    
    It appears to me that ptlrpc, ldlm, and obdclass are always and only
    used together, but can be used for client or server.  So I think they
    should be together in a module.
    I'm tempted to add lnet to that list.  There has been mention that cray
    has some other functionality that uses lnet - if that doesn't use
    ptlrpc, then maybe that is a case for keeping lnet separate.  However I
    would rather see it as a potential case to separate lnet from the others
    at a later date if that functionality from cray ever goes into mainline.
    
    The klnds modules can presumably be used independently(?) so they can
    sensibly remain as separate modules, though I'm beginning to wonder if
    lnet can actually function without socklnd, so I wonder if that should
    be permanently part of the lnet module (with NFS, the xprt_sock code is
    a permanent part of "sunrpc", while the xprt_rdma (aka rpcrdma) code
    can be a separate module).
    
    In the interests of a concrete strawman, what objections would I get if
    I suggested that ptlrpc, ldlm, obdclass, lnet, and socklnd were all included in
    the one module named "lnet" ??
    
    Thanks,
    NeilBrown
    


From adilger at whamcloud.com  Wed Jun 27 11:01:27 2018
From: adilger at whamcloud.com (Andreas Dilger)
Date: Wed, 27 Jun 2018 11:01:27 +0000
Subject: [lustre-devel] [PATCH 00/24] lustre - more cleanups including
 module reduction.
In-Reply-To: <87d0wd6i4m.fsf@notabene.neil.brown.name>
References: <152904663333.10587.10934053155404014785.stgit@noble>
 <alpine.LFD.2.21.1806210239190.3886@casper.infradead.org>
 <DM3PR1101MB1213DDB7FC857EE6B1444153CB760@DM3PR1101MB1213.namprd11.prod.outlook.com>
 <alpine.LFD.2.21.1806210354260.3886@casper.infradead.org>
 <3FD4D051-9C95-449D-8A23-D42B271E55B8@dilger.ca>
 <DM3PR1101MB121322E85FDE1E5310F6F4B7CB760@DM3PR1101MB1213.namprd11.prod.outlook.com>
 <87tvprah32.fsf@notabene.neil.brown.name>
 <alpine.LFD.2.21.1806260206360.31196@casper.infradead.org>
 <DM3PR1101MB121358672276198B843DE5A7CB490@DM3PR1101MB1213.namprd11.prod.outlook.com>
 <87d0wd6i4m.fsf@notabene.neil.brown.name>
Message-ID: <44633E99-07CB-4C40-967C-0B4FE206F29A@whamcloud.com>

On Jun 27, 2018, at 05:08, NeilBrown <neilb at suse.com> wrote:
> 
> On Tue, Jun 26 2018, Patrick Farrell wrote:
> 
>> Ah, sorry, lost your mail in the shuffle, Neil.
>> 
>> The name, mostly.  ptlrpc is one subdirectory and one subsystem, so I don’t want to use it for a module that explicitly includes all of several.  I’m not sure of a better name immediately - is this module intended to be shared between client and server?
>> 
> 
> Thanks for clarifying.
> Note that I'm still feeling my way around here so "intended" can at most
> be a soft intention, open to change.  But yes, I expect this module
> would ultimately be shared with the server.
> 
> I think things should only be in different modules if it might sometimes
> make sense to use one without the other.
> 
> So everything that is always used for a lustre client mount, and is only
> used for a lustre client mount, should be in a module called "lustre".
> And everything always and only used for a lustre server should be in a
> module called (something like) "lustre-server".
> 
> It appears to me that ptlrpc, ldlm, and obdclass are always and only
> used together, but can be used for client or server.  So I think they
> should be together in a module.
> 
> I'm tempted to add lnet to that list.  There has been mention that Cray
> has some other functionality that uses lnet - if that doesn't use
> ptlrpc, then maybe that is a case for keeping lnet separate.  However I
> would rather see it as a potential case to separate lnet from the others
> at a later date if that functionality from cray ever goes into mainline.
> 
> The klnds modules can presumably be used independently(?) so they can
> sensibly remain as separate modules, though I'm beginning to wonder if
> lnet can actually function without socklnd, so I wonder if that should
> be permanently part of the lnet module (with NFS, the xprt_sock code is
> a permanent part of "sunrpc", while the xprt_rdma (aka rpcrdma) code
> can be a separate module).

Definitely LNet can run without ksocklnd, and it does so on clients that
only have IB connections.  Lots of HPC systems have client nodes that do
not have Ethernet interfaces, since it is a cabling/networking nightmare
to add a few thousand additional Ethernet cables/ports in addition to the
IB cables/ports, and it is always possible to run IPoIB for TCP traffic.

> In the interests of a concrete strawman, what objections would I get if
> I suggested that ptlrpc, ldlm, obdclass, lnet, and socklnd were all included in
> the one module named "lnet" ??

This would break all of the module parameters for ptlrpc.  There are only
a few obdclass module parameters that are rarely used, so I'm less worried
about those ones.

Given that lnet is a separate thing, I'd prefer to keep "ptlrpc" and "lnet"
as separate modules, then "lustre" for the remainder of the client code.

The "ptlrpc" module should also include the "fld", "fid", "quota", "mgc"
modules, since these are shared with the server as well.

Server modules:
[adilger at mookie ~]$ lsmod | grep ^obdclass
obdclass             1731964  83 mdd,lod,mdt,osp,ofd,lfsck,ost,mgs,mgc,osd_ldiskfs,fid,fld,lquota,ptlrpc

Client modules:
[adilger at twoshoes ~]$ lsmod | grep ^obdclass
obdclass             1568040  86 osc,mgc,lustre,lov,mdc,fid,lmv,fld,ptlrpc


James, if we are going to move in the direction of having a separate
"lustre-server" module, it would make sense to land a patch to master
now that adds both a module alias, as well as a filesystem type alias
for lustre-server, so that we can start moving systems over to using
"lustre-server" as the filesystem type in /etc/fstab or HA mounting
scripts, and it will auto-load the module at the same time.  This might
also allow the Lustre mount code to be simplified, since I think it has
some messy code to have the same code, but distinguish between client
and server mounts.

Cheers, Andreas
---
Andreas Dilger
Principal Lustre Architect
Whamcloud






-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 235 bytes
Desc: Message signed with OpenPGP
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180627/91d93896/attachment.sig>

From paf at cray.com  Wed Jun 27 12:06:26 2018
From: paf at cray.com (Patrick Farrell)
Date: Wed, 27 Jun 2018 12:06:26 +0000
Subject: [lustre-devel] [PATCH 00/24] lustre - more cleanups including
 module reduction.
In-Reply-To: <44633E99-07CB-4C40-967C-0B4FE206F29A@whamcloud.com>
References: <152904663333.10587.10934053155404014785.stgit@noble>
 <alpine.LFD.2.21.1806210239190.3886@casper.infradead.org>
 <DM3PR1101MB1213DDB7FC857EE6B1444153CB760@DM3PR1101MB1213.namprd11.prod.outlook.com>
 <alpine.LFD.2.21.1806210354260.3886@casper.infradead.org>
 <3FD4D051-9C95-449D-8A23-D42B271E55B8@dilger.ca>
 <DM3PR1101MB121322E85FDE1E5310F6F4B7CB760@DM3PR1101MB1213.namprd11.prod.outlook.com>
 <87tvprah32.fsf@notabene.neil.brown.name>
 <alpine.LFD.2.21.1806260206360.31196@casper.infradead.org>
 <DM3PR1101MB121358672276198B843DE5A7CB490@DM3PR1101MB1213.namprd11.prod.outlook.com>
 <87d0wd6i4m.fsf@notabene.neil.brown.name>,
 <44633E99-07CB-4C40-967C-0B4FE206F29A@whamcloud.com>
Message-ID: <DM3PR1101MB12132AC3E33E7D7C8FBE4536CB480@DM3PR1101MB1213.namprd11.prod.outlook.com>


Neil,

We do indeed have such functionality (it’s called DVS and it’s basically a high speed file system projection framework, ala NFS but faster), so the ability to build lnet separately is valuable to us.  While it is being open sourced under the GPL, I don’t think there’s any intention to try to upstream it.  The current code isn’t even usable off of Cray systems as it depends on info from user space (that is provided, in the end, from Cray proprietary hardware) to keep its connection/routing tables up to date.  That’s supposedly in the pipeline to get fixed, but it’s still pretty far from generally usable.

But we’d still really appreciate it if lnet stayed separate.  Don’t know if that’s enough for you - I know sometimes *small* stuff is done for out of tree users.  Hopefully this meets that standard.

- Patrick

________________________________
From: Andreas Dilger <adilger at whamcloud.com>
Sent: Wednesday, June 27, 2018 6:01:27 AM
To: NeilBrown; James Simmons
Cc: Patrick Farrell; Oleg Drokin; Lustre Development List
Subject: Re: [lustre-devel] [PATCH 00/24] lustre - more cleanups including module reduction.

On Jun 27, 2018, at 05:08, NeilBrown <neilb at suse.com> wrote:
>
> On Tue, Jun 26 2018, Patrick Farrell wrote:
>
>> Ah, sorry, lost your mail in the shuffle, Neil.
>>
>> The name, mostly.  ptlrpc is one subdirectory and one subsystem, so I don’t want to use it for a module that explicitly includes all of several.  I’m not sure of a better name immediately - is this module intended to be shared between client and server?
>>
>
> Thanks for clarifying.
> Note that I'm still feeling my way around here so "intended" can at most
> be a soft intention, open to change.  But yes, I expect this module
> would ultimately be shared with the server.
>
> I think things should only be in different modules if it might sometimes
> make sense to use one without the other.
>
> So everything that is always used for a lustre client mount, and is only
> used for a lustre client mount, should be in a module called "lustre".
> And everything always and only used for a lustre server should be in a
> module called (something like) "lustre-server".
>
> It appears to me that ptlrpc, ldlm, and obdclass are always and only
> used together, but can be used for client or server.  So I think they
> should be together in a module.
>
> I'm tempted to add lnet to that list.  There has been mention that Cray
> has some other functionality that uses lnet - if that doesn't use
> ptlrpc, then maybe that is a case for keeping lnet separate.  However I
> would rather see it as a potential case to separate lnet from the others
> at a later date if that functionality from cray ever goes into mainline.
>
> The klnds modules can presumably be used independently(?) so they can
> sensibly remain as separate modules, though I'm beginning to wonder if
> lnet can actually function without socklnd, so I wonder if that should
> be permanently part of the lnet module (with NFS, the xprt_sock code is
> a permanent part of "sunrpc", while the xprt_rdma (aka rpcrdma) code
> can be a separate module).

Definitely LNet can run without ksocklnd, and it does so on clients that
only have IB connections.  Lots of HPC systems have client nodes that do
not have Ethernet interfaces, since it is a cabling/networking nightmare
to add a few thousand additional Ethernet cables/ports in addition to the
IB cables/ports, and it is always possible to run IPoIB for TCP traffic.

> In the interests of a concrete strawman, what objections would I get if
> I suggested that ptlrpc, ldlm, obdclass, lnet, and socklnd were all included in
> the one module named "lnet" ??

This would break all of the module parameters for ptlrpc.  There are only
a few obdclass module parameters that are rarely used, so I'm less worried
about those ones.

Given that lnet is a separate thing, I'd prefer to keep "ptlrpc" and "lnet"
as separate modules, then "lustre" for the remainder of the client code.

The "ptlrpc" module should also include the "fld", "fid", "quota", "mgc"
modules, since these are shared with the server as well.

Server modules:
[adilger at mookie ~]$ lsmod | grep ^obdclass
obdclass             1731964  83 mdd,lod,mdt,osp,ofd,lfsck,ost,mgs,mgc,osd_ldiskfs,fid,fld,lquota,ptlrpc

Client modules:
[adilger at twoshoes ~]$ lsmod | grep ^obdclass
obdclass             1568040  86 osc,mgc,lustre,lov,mdc,fid,lmv,fld,ptlrpc


James, if we are going to move in the direction of having a separate
"lustre-server" module, it would make sense to land a patch to master
now that adds both a module alias, as well as a filesystem type alias
for lustre-server, so that we can start moving systems over to using
"lustre-server" as the filesystem type in /etc/fstab or HA mounting
scripts, and it will auto-load the module at the same time.  This might
also allow the Lustre mount code to be simplified, since I think it has
some messy code to have the same code, but distinguish between client
and server mounts.

Cheers, Andreas
---
Andreas Dilger
Principal Lustre Architect
Whamcloud






-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180627/b0c53d64/attachment.html>

From paf at cray.com  Wed Jun 27 12:42:37 2018
From: paf at cray.com (Patrick Farrell)
Date: Wed, 27 Jun 2018 12:42:37 +0000
Subject: [lustre-devel] [PATCH v3 07/26] staging: lustre: libcfs: NUMA
 support
In-Reply-To: <87lgb16j8q.fsf@notabene.neil.brown.name>
References: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
 <1529875250-11531-8-git-send-email-jsimmons@infradead.org>
 <877emnaed8.fsf@notabene.neil.brown.name>
 <A764DED1-A9A8-4FA5-A0F1-922C3E2D822A@cray.com>,
 <87lgb16j8q.fsf@notabene.neil.brown.name>
Message-ID: <DM3PR1101MB1213FD1001256E565FF011FFCB480@DM3PR1101MB1213.namprd11.prod.outlook.com>


Neil,

I am not the person at Cray for this, but if SUSE does take an interest in this, Cray would probably be interested in weighing in and contributing info if not actually code.  In fact, other HPC vendors like HPE(by which I mostly mean the old SGI) or IBM might as well.  NUMA optimization is a persistent fascination in our area of the industry...

- Patrick

________________________________
From: lustre-devel <lustre-devel-bounces at lists.lustre.org> on behalf of NeilBrown <neilb at suse.com>
Sent: Tuesday, June 26, 2018 9:44:37 PM
To: Doug Oucharek
Cc: Amir Shehata; Lustre Development List
Subject: Re: [lustre-devel] [PATCH v3 07/26] staging: lustre: libcfs: NUMA support

On Mon, Jun 25 2018, Doug Oucharek wrote:

> Some background on this NUMA change:
>
> First off, this is just a first step to a bigger set of changes which include changes to the Lustre utilities.  This was done as part of the Multi-Rail feature.  One of the systems that feature is meant to support is the SGI UV system (now HPE) which has a massive number of NUMA nodes connected by a NUMA Link.  There are multiple fabric cards spread throughout the system and Multi-Rail needs to know which fabric cards are nearest to the NUMA node we are running on.  To do that, the “distance” between NUMA nodes needs to be configured.
>
> This patch is preparing the infrastructure for the Multi-Rail feature to support configuring NUMA node distances.  Technically, this patch should be landing with the Multi-Rail feature (still to be pushed) for it to make proper sense.
>

Thanks a lot for the background.

If these NUMA nodes have a 'distance' between them, and if lustre can
benefit from knowing the distance, then is seems likely that other code
might also benefit.  In that case it would be best if the distance were
encoded in some global state information so that lustre and any other
subsystem can extract it.

Do you know if there is any work underway by anyone to make this
information generally available?  If there is, we should make sure that
lustre works in a compatible way so that once that work lands, lustre
can use it directly and not need extra configuration.
If no such work is underway, then it would be really good if something
were done in that direction.  If no-one here is able to work on this, I
can ask around in SUSE and see if anyone here knows anything relevant.

Thanks,
NeilBrown
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180627/5fb02a73/attachment.html>

From jsimmons at infradead.org  Wed Jun 27 19:38:30 2018
From: jsimmons at infradead.org (James Simmons)
Date: Wed, 27 Jun 2018 15:38:30 -0400
Subject: [lustre-devel] [PATCH v3 01/13] lustre: libcfs: move tracefile
 locking from linux-tracefile.c to tracefile.c
In-Reply-To: <1530128322-32535-1-git-send-email-jsimmons@infradead.org>
References: <1530128322-32535-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1530128322-32535-2-git-send-email-jsimmons@infradead.org>

From: NeilBrown <neilb at suse.com>

There is no value in keeping it separate.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 .../staging/lustre/lnet/libcfs/linux-tracefile.c   | 22 -------------------
 drivers/staging/lustre/lnet/libcfs/tracefile.c     | 25 +++++++++++-----------
 drivers/staging/lustre/lnet/libcfs/tracefile.h     |  5 -----
 3 files changed, 13 insertions(+), 39 deletions(-)

diff --git a/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c b/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
index 3471384..9e72220 100644
--- a/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
+++ b/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
@@ -47,8 +47,6 @@
 
 char *cfs_trace_console_buffers[NR_CPUS][CFS_TCD_TYPE_MAX];
 
-static DECLARE_RWSEM(cfs_tracefile_sem);
-
 int cfs_tracefile_init_arch(void)
 {
 	int i;
@@ -109,26 +107,6 @@ void cfs_tracefile_fini_arch(void)
 	}
 }
 
-void cfs_tracefile_read_lock(void)
-{
-	down_read(&cfs_tracefile_sem);
-}
-
-void cfs_tracefile_read_unlock(void)
-{
-	up_read(&cfs_tracefile_sem);
-}
-
-void cfs_tracefile_write_lock(void)
-{
-	down_write(&cfs_tracefile_sem);
-}
-
-void cfs_tracefile_write_unlock(void)
-{
-	up_write(&cfs_tracefile_sem);
-}
-
 enum cfs_trace_buf_type cfs_trace_buf_idx_get(void)
 {
 	if (in_irq())
diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.c b/drivers/staging/lustre/lnet/libcfs/tracefile.c
index 7ca562e..5f31933 100644
--- a/drivers/staging/lustre/lnet/libcfs/tracefile.c
+++ b/drivers/staging/lustre/lnet/libcfs/tracefile.c
@@ -59,6 +59,7 @@
 static int thread_running;
 
 static atomic_t cfs_tage_allocated = ATOMIC_INIT(0);
+static DECLARE_RWSEM(cfs_tracefile_sem);
 
 struct page_collection {
 	struct list_head	pc_pages;
@@ -711,7 +712,7 @@ int cfs_tracefile_dump_all_pages(char *filename)
 	mm_segment_t __oldfs;
 	int rc;
 
-	cfs_tracefile_write_lock();
+	down_write(&cfs_tracefile_sem);
 
 	filp = filp_open(filename, O_CREAT | O_EXCL | O_WRONLY | O_LARGEFILE,
 			 0600);
@@ -759,7 +760,7 @@ int cfs_tracefile_dump_all_pages(char *filename)
 close:
 	filp_close(filp, NULL);
 out:
-	cfs_tracefile_write_unlock();
+	up_write(&cfs_tracefile_sem);
 	return rc;
 }
 
@@ -873,12 +874,12 @@ int cfs_trace_daemon_command(char *str)
 {
 	int rc = 0;
 
-	cfs_tracefile_write_lock();
+	down_write(&cfs_tracefile_sem);
 
 	if (!strcmp(str, "stop")) {
-		cfs_tracefile_write_unlock();
+		up_write(&cfs_tracefile_sem);
 		cfs_trace_stop_thread();
-		cfs_tracefile_write_lock();
+		down_write(&cfs_tracefile_sem);
 		memset(cfs_tracefile, 0, sizeof(cfs_tracefile));
 
 	} else if (!strncmp(str, "size=", 5)) {
@@ -905,7 +906,7 @@ int cfs_trace_daemon_command(char *str)
 		cfs_trace_start_thread();
 	}
 
-	cfs_tracefile_write_unlock();
+	up_write(&cfs_tracefile_sem);
 	return rc;
 }
 
@@ -950,12 +951,12 @@ int cfs_trace_set_debug_mb(int mb)
 	mb /= num_possible_cpus();
 	pages = mb << (20 - PAGE_SHIFT);
 
-	cfs_tracefile_write_lock();
+	down_write(&cfs_tracefile_sem);
 
 	cfs_tcd_for_each(tcd, i, j)
 		tcd->tcd_max_pages = (pages * tcd->tcd_pages_factor) / 100;
 
-	cfs_tracefile_write_unlock();
+	up_write(&cfs_tracefile_sem);
 
 	return 0;
 }
@@ -967,12 +968,12 @@ int cfs_trace_get_debug_mb(void)
 	struct cfs_trace_cpu_data *tcd;
 	int total_pages = 0;
 
-	cfs_tracefile_read_lock();
+	down_read(&cfs_tracefile_sem);
 
 	cfs_tcd_for_each(tcd, i, j)
 		total_pages += tcd->tcd_max_pages;
 
-	cfs_tracefile_read_unlock();
+	up_read(&cfs_tracefile_sem);
 
 	return (total_pages >> (20 - PAGE_SHIFT)) + 1;
 }
@@ -1002,7 +1003,7 @@ static int tracefiled(void *arg)
 			goto end_loop;
 
 		filp = NULL;
-		cfs_tracefile_read_lock();
+		down_read(&cfs_tracefile_sem);
 		if (cfs_tracefile[0]) {
 			filp = filp_open(cfs_tracefile,
 					 O_CREAT | O_RDWR | O_LARGEFILE,
@@ -1014,7 +1015,7 @@ static int tracefiled(void *arg)
 					rc);
 			}
 		}
-		cfs_tracefile_read_unlock();
+		up_read(&cfs_tracefile_sem);
 		if (!filp) {
 			put_pages_on_daemon_list(&pc);
 			__LASSERT(list_empty(&pc.pc_pages));
diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.h b/drivers/staging/lustre/lnet/libcfs/tracefile.h
index 0608240..9f6b73d 100644
--- a/drivers/staging/lustre/lnet/libcfs/tracefile.h
+++ b/drivers/staging/lustre/lnet/libcfs/tracefile.h
@@ -65,11 +65,6 @@ enum cfs_trace_buf_type {
 int  cfs_tracefile_init_arch(void);
 void cfs_tracefile_fini_arch(void);
 
-void cfs_tracefile_read_lock(void);
-void cfs_tracefile_read_unlock(void);
-void cfs_tracefile_write_lock(void);
-void cfs_tracefile_write_unlock(void);
-
 int cfs_tracefile_dump_all_pages(char *filename);
 void cfs_trace_debug_print(void);
 void cfs_trace_flush_pages(void);
-- 
1.8.3.1


From jsimmons at infradead.org  Wed Jun 27 19:38:34 2018
From: jsimmons at infradead.org (James Simmons)
Date: Wed, 27 Jun 2018 15:38:34 -0400
Subject: [lustre-devel] [PATCH v3 05/13] lustre: libcfs: properly handle
 failure paths in cfs_tracefile_init_arch()
In-Reply-To: <1530128322-32535-1-git-send-email-jsimmons@infradead.org>
References: <1530128322-32535-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1530128322-32535-6-git-send-email-jsimmons@infradead.org>

It is considered poor coding style in the linux kernel to call a
catch all for cleanup in the initialization function. Also their
is no reason to cleanup cfs_trace_console_buffers if they never
been allocated. Lets unroll the error handling.

Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/lnet/libcfs/linux-tracefile.c | 19 ++++++++++++++-----
 1 file changed, 14 insertions(+), 5 deletions(-)

diff --git a/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c b/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
index f100cb9..a334b63 100644
--- a/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
+++ b/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
@@ -49,9 +49,9 @@
 
 int cfs_tracefile_init_arch(void)
 {
+	struct cfs_trace_cpu_data *tcd;
 	int i;
 	int j;
-	struct cfs_trace_cpu_data *tcd;
 
 	/* initialize trace_data */
 	memset(cfs_trace_data, 0, sizeof(cfs_trace_data));
@@ -61,7 +61,7 @@ int cfs_tracefile_init_arch(void)
 				      sizeof(union cfs_trace_data_union),
 				      GFP_KERNEL);
 		if (!cfs_trace_data[i])
-			goto out;
+			goto out_trace_data;
 	}
 
 	/* arch related info initialized */
@@ -79,13 +79,22 @@ int cfs_tracefile_init_arch(void)
 					GFP_KERNEL);
 
 			if (!cfs_trace_console_buffers[i][j])
-				goto out;
+				goto out_buffers;
 		}
 
 	return 0;
 
-out:
-	cfs_tracefile_fini_arch();
+out_buffers:
+	for (i = 0; i < num_possible_cpus(); i++)
+		for (j = 0; j < 3; j++) {
+			kfree(cfs_trace_console_buffers[i][j]);
+			cfs_trace_console_buffers[i][j] = NULL;
+		}
+out_trace_data:
+	for (i = 0; cfs_trace_data[i]; i++) {
+		kfree(cfs_trace_data[i]);
+		cfs_trace_data[i] = NULL;
+	}
 	pr_err("lnet: Not enough memory\n");
 	return -ENOMEM;
 }
-- 
1.8.3.1


From jsimmons at infradead.org  Wed Jun 27 19:38:31 2018
From: jsimmons at infradead.org (James Simmons)
Date: Wed, 27 Jun 2018 15:38:31 -0400
Subject: [lustre-devel] [PATCH v3 02/13] lustre: libcfs: open code
 cfs_trace_max_debug_mb() into cfs_trace_set_debug_mb()
In-Reply-To: <1530128322-32535-1-git-send-email-jsimmons@infradead.org>
References: <1530128322-32535-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1530128322-32535-3-git-send-email-jsimmons@infradead.org>

From: NeilBrown <neilb at suse.com>

This function in used twice.
In libcfs_debug_init() the usage is pointless as
the only place that set libcfs_debug_mb ensures
that it does not exceed the maximum.  So checking
again can never help.

So open-code the small function into the only
other place that it is used - in cfs_trace_set_debug_mb(),
which is used to set libcfs_debug_mb.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 drivers/staging/lustre/lnet/libcfs/debug.c           | 6 +++---
 drivers/staging/lustre/lnet/libcfs/linux-tracefile.c | 7 -------
 drivers/staging/lustre/lnet/libcfs/tracefile.c       | 3 ++-
 drivers/staging/lustre/lnet/libcfs/tracefile.h       | 1 -
 4 files changed, 5 insertions(+), 12 deletions(-)

diff --git a/drivers/staging/lustre/lnet/libcfs/debug.c b/drivers/staging/lustre/lnet/libcfs/debug.c
index 06f694f..71effcf 100644
--- a/drivers/staging/lustre/lnet/libcfs/debug.c
+++ b/drivers/staging/lustre/lnet/libcfs/debug.c
@@ -411,10 +411,10 @@ int libcfs_debug_init(unsigned long bufsize)
 			sizeof(libcfs_debug_file_path_arr));
 	}
 
-	/* If libcfs_debug_mb is set to an invalid value or uninitialized
-	 * then just make the total buffers smp_num_cpus * TCD_MAX_PAGES
+	/* If libcfs_debug_mb is uninitialized then just make the
+	 * total buffers smp_num_cpus * TCD_MAX_PAGES
 	 */
-	if (max > cfs_trace_max_debug_mb() || max < num_possible_cpus()) {
+	if (max < num_possible_cpus()) {
 		max = TCD_MAX_PAGES;
 	} else {
 		max = max / num_possible_cpus();
diff --git a/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c b/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
index 9e72220..64a5bc1 100644
--- a/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
+++ b/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
@@ -227,10 +227,3 @@ void cfs_print_to_console(struct ptldebug_header *hdr, int mask,
 			hdr->ph_line_num, fn, len, buf);
 	}
 }
-
-int cfs_trace_max_debug_mb(void)
-{
-	int  total_mb = (totalram_pages >> (20 - PAGE_SHIFT));
-
-	return max(512, (total_mb * 80) / 100);
-}
diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.c b/drivers/staging/lustre/lnet/libcfs/tracefile.c
index 5f31933..72321ce 100644
--- a/drivers/staging/lustre/lnet/libcfs/tracefile.c
+++ b/drivers/staging/lustre/lnet/libcfs/tracefile.c
@@ -933,7 +933,8 @@ int cfs_trace_set_debug_mb(int mb)
 	int i;
 	int j;
 	int pages;
-	int limit = cfs_trace_max_debug_mb();
+	int total_mb = (totalram_pages >> (20 - PAGE_SHIFT));
+	int limit = max(512, (total_mb * 80) / 100);
 	struct cfs_trace_cpu_data *tcd;
 
 	if (mb < num_possible_cpus()) {
diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.h b/drivers/staging/lustre/lnet/libcfs/tracefile.h
index 9f6b73d..bd1a1ef 100644
--- a/drivers/staging/lustre/lnet/libcfs/tracefile.h
+++ b/drivers/staging/lustre/lnet/libcfs/tracefile.h
@@ -88,7 +88,6 @@ int cfs_trace_copyout_string(char __user *usr_buffer, int usr_buffer_nob,
 void libcfs_register_panic_notifier(void);
 void libcfs_unregister_panic_notifier(void);
 extern int libcfs_panic_in_progress;
-int cfs_trace_max_debug_mb(void);
 
 #define TCD_MAX_PAGES (5 << (20 - PAGE_SHIFT))
 #define TCD_STOCK_PAGES (TCD_MAX_PAGES)
-- 
1.8.3.1


From jsimmons at infradead.org  Wed Jun 27 19:38:29 2018
From: jsimmons at infradead.org (James Simmons)
Date: Wed, 27 Jun 2018 15:38:29 -0400
Subject: [lustre-devel] [PATCH v3 00/13] lustre: libcfs: tracefile cleanups
Message-ID: <1530128322-32535-1-git-send-email-jsimmons@infradead.org>

The first three patches in this series are a repost of Neil's patches.
I updated the 4th patch but it contained to many changes so I broke
it into 6 smaller patches for easier review. It address all the
brokeness of cfs_print_to_console() which shows up in my testing.
Besides the fixing more cleanups of tracefile.h was done in the
new patches. The rest of the patches are Neil's work rebased due
the changes I introduced.

James Simmons (6):
  lustre: libcfs: fix cfs_print_to_console()
  lustre: libcfs: properly handle failure paths in cfs_tracefile_init_arch()
  lustre: libcfs: remove cfs_trace_refill_stack()
  lustre: libcfs: move cfs_trace_data data to tracefile.c
  lustre: libcfs: cleanup tracefile.h
  lustre: libcfs: format macros in tracefile.h

NeilBrown (7):
  lustre: libcfs: move tracefile locking from linux-tracefile.c to tracefile.c
  lustre: libcfs: open code cfs_trace_max_debug_mb() into cfs_trace_set_debug_mb()
  lustre: libcfs: move tcd locking across to tracefile.c
  lustre: libcfs: merge linux-tracefile.c into tracefile.c
  lustre: libcfs: fold cfs_tracefile_*_arch into their only callers.
  lustre: libcfs: renamed CFS_TCD_TYPE_MAX to CFS_TCD_TYPE_CNT
  lustre: libcfs: discard TCD_MAX_TYPES

-------------
Changelog

1) Initial patches from Neil Brown

2) Neil's patches pushed with the 4th patch heavly updated.

3) Broke up the 4th patch I pushed into smaller patches for
   easier review.

 drivers/staging/lustre/lnet/libcfs/Makefile        |   2 +-
 drivers/staging/lustre/lnet/libcfs/debug.c         |   6 +-
 .../staging/lustre/lnet/libcfs/linux-tracefile.c   | 258 --------------------
 drivers/staging/lustre/lnet/libcfs/tracefile.c     | 267 +++++++++++++++++----
 drivers/staging/lustre/lnet/libcfs/tracefile.h     | 117 +--------
 5 files changed, 237 insertions(+), 413 deletions(-)
 delete mode 100644 drivers/staging/lustre/lnet/libcfs/linux-tracefile.c

-- 
1.8.3.1


From jsimmons at infradead.org  Wed Jun 27 19:38:36 2018
From: jsimmons at infradead.org (James Simmons)
Date: Wed, 27 Jun 2018 15:38:36 -0400
Subject: [lustre-devel] [PATCH v3 07/13] lustre: libcfs: remove
 cfs_trace_refill_stack()
In-Reply-To: <1530128322-32535-1-git-send-email-jsimmons@infradead.org>
References: <1530128322-32535-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1530128322-32535-8-git-send-email-jsimmons@infradead.org>

The function cfs_trace_refill_stack() is not used anywhere so
remove it.

Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/lnet/libcfs/tracefile.c | 21 ---------------------
 drivers/staging/lustre/lnet/libcfs/tracefile.h |  3 ---
 2 files changed, 24 deletions(-)

diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.c b/drivers/staging/lustre/lnet/libcfs/tracefile.c
index a2d4ee9..6a42d7c 100644
--- a/drivers/staging/lustre/lnet/libcfs/tracefile.c
+++ b/drivers/staging/lustre/lnet/libcfs/tracefile.c
@@ -302,27 +302,6 @@ static void cfs_tage_to_tail(struct cfs_trace_page *tage,
 	list_move_tail(&tage->linkage, queue);
 }
 
-int cfs_trace_refill_stock(struct cfs_trace_cpu_data *tcd, gfp_t gfp,
-			   struct list_head *stock)
-{
-	int i;
-
-	/*
-	 * XXX nikita: do NOT call portals_debug_msg() (CDEBUG/ENTRY/EXIT)
-	 * from here: this will lead to infinite recursion.
-	 */
-
-	for (i = 0; i + tcd->tcd_cur_stock_pages < TCD_STOCK_PAGES ; ++i) {
-		struct cfs_trace_page *tage;
-
-		tage = cfs_tage_alloc(gfp);
-		if (!tage)
-			break;
-		list_add_tail(&tage->linkage, stock);
-	}
-	return i;
-}
-
 /* return a page that has 'len' bytes left at the end */
 static struct cfs_trace_page *
 cfs_trace_get_tage_try(struct cfs_trace_cpu_data *tcd, unsigned long len)
diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.h b/drivers/staging/lustre/lnet/libcfs/tracefile.h
index 399e0bf..b9e4a8e 100644
--- a/drivers/staging/lustre/lnet/libcfs/tracefile.h
+++ b/drivers/staging/lustre/lnet/libcfs/tracefile.h
@@ -204,9 +204,6 @@ int cfs_trace_copyout_string(char __user *usr_buffer, int usr_buffer_nob,
 	return cfs_trace_console_buffers[i][j];
 }
 
-int cfs_trace_refill_stock(struct cfs_trace_cpu_data *tcd, gfp_t gfp,
-			   struct list_head *stock);
-
 void cfs_trace_assertion_failed(const char *str,
 				struct libcfs_debug_msg_data *m);
 
-- 
1.8.3.1


From jsimmons at infradead.org  Wed Jun 27 19:38:37 2018
From: jsimmons at infradead.org (James Simmons)
Date: Wed, 27 Jun 2018 15:38:37 -0400
Subject: [lustre-devel] [PATCH v3 08/13] lustre: libcfs: move cfs_trace_data
 data to tracefile.c
In-Reply-To: <1530128322-32535-1-git-send-email-jsimmons@infradead.org>
References: <1530128322-32535-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1530128322-32535-9-git-send-email-jsimmons@infradead.org>

The macro cfs_tcd_for_each() is only used in tracefile.c so move
it from the header tracefile.h along with related material in
the header file.

Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/lnet/libcfs/tracefile.c | 12 ++++++++++--
 drivers/staging/lustre/lnet/libcfs/tracefile.h |  9 ---------
 2 files changed, 10 insertions(+), 11 deletions(-)

diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.c b/drivers/staging/lustre/lnet/libcfs/tracefile.c
index 6a42d7c..914cd94 100644
--- a/drivers/staging/lustre/lnet/libcfs/tracefile.c
+++ b/drivers/staging/lustre/lnet/libcfs/tracefile.c
@@ -49,7 +49,8 @@
 #include <linux/uaccess.h>
 #include "tracefile.h"
 
-/* XXX move things up to the top, comment */
+#define TCD_MAX_TYPES			8
+
 union cfs_trace_data_union (*cfs_trace_data[TCD_MAX_TYPES])[NR_CPUS] __cacheline_aligned;
 
 char *cfs_trace_console_buffers[NR_CPUS][CFS_TCD_TYPE_MAX];
@@ -146,7 +147,14 @@ void cfs_trace_unlock_tcd(struct cfs_trace_cpu_data *tcd, int walking)
 		spin_unlock(&tcd->tcd_lock);
 }
 
-#define cfs_tcd_for_each_type_lock(tcd, i, cpu)			   \
+#define cfs_tcd_for_each(tcd, i, j)					\
+	for (i = 0; cfs_trace_data[i]; i++)				\
+		for (j = 0, ((tcd) = &(*cfs_trace_data[i])[j].tcd);	\
+		     j < num_possible_cpus();				\
+		     j++, (tcd) = &(*cfs_trace_data[i])[j].tcd)
+
+
+#define cfs_tcd_for_each_type_lock(tcd, i, cpu)				\
 	for (i = 0; cfs_trace_data[i] &&				\
 	     (tcd = &(*cfs_trace_data[i])[cpu].tcd) &&			\
 	     cfs_trace_lock_tcd(tcd, 1); cfs_trace_unlock_tcd(tcd, 1), i++)
diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.h b/drivers/staging/lustre/lnet/libcfs/tracefile.h
index b9e4a8e..072c720 100644
--- a/drivers/staging/lustre/lnet/libcfs/tracefile.h
+++ b/drivers/staging/lustre/lnet/libcfs/tracefile.h
@@ -183,15 +183,6 @@ int cfs_trace_copyout_string(char __user *usr_buffer, int usr_buffer_nob,
 	char __pad[L1_CACHE_ALIGN(sizeof(struct cfs_trace_cpu_data))];
 };
 
-#define TCD_MAX_TYPES      8
-extern union cfs_trace_data_union (*cfs_trace_data[TCD_MAX_TYPES])[NR_CPUS];
-
-#define cfs_tcd_for_each(tcd, i, j)				       \
-	for (i = 0; cfs_trace_data[i]; i++)				\
-		for (j = 0, ((tcd) = &(*cfs_trace_data[i])[j].tcd);	\
-		     j < num_possible_cpus();				 \
-		     j++, (tcd) = &(*cfs_trace_data[i])[j].tcd)
-
 extern char *cfs_trace_console_buffers[NR_CPUS][CFS_TCD_TYPE_MAX];
 enum cfs_trace_buf_type cfs_trace_buf_idx_get(void);
 
-- 
1.8.3.1


From jsimmons at infradead.org  Wed Jun 27 19:38:32 2018
From: jsimmons at infradead.org (James Simmons)
Date: Wed, 27 Jun 2018 15:38:32 -0400
Subject: [lustre-devel] [PATCH v3 03/13] lustre: libcfs: move tcd locking
 across to tracefile.c
In-Reply-To: <1530128322-32535-1-git-send-email-jsimmons@infradead.org>
References: <1530128322-32535-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1530128322-32535-4-git-send-email-jsimmons@infradead.org>

From: NeilBrown <neilb at suse.com>

No need to have this in linux-tracefile.c

Signed-off-by: NeilBrown <neilb at suse.com>
---
 .../staging/lustre/lnet/libcfs/linux-tracefile.c   | 35 -------------
 drivers/staging/lustre/lnet/libcfs/tracefile.c     | 59 ++++++++++++++++++++++
 drivers/staging/lustre/lnet/libcfs/tracefile.h     | 28 ----------
 3 files changed, 59 insertions(+), 63 deletions(-)

diff --git a/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c b/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
index 64a5bc1..3af7722 100644
--- a/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
+++ b/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
@@ -116,41 +116,6 @@ enum cfs_trace_buf_type cfs_trace_buf_idx_get(void)
 	return CFS_TCD_TYPE_PROC;
 }
 
-/*
- * The walking argument indicates the locking comes from all tcd types
- * iterator and we must lock it and dissable local irqs to avoid deadlocks
- * with other interrupt locks that might be happening. See LU-1311
- * for details.
- */
-int cfs_trace_lock_tcd(struct cfs_trace_cpu_data *tcd, int walking)
-	__acquires(&tcd->tc_lock)
-{
-	__LASSERT(tcd->tcd_type < CFS_TCD_TYPE_MAX);
-	if (tcd->tcd_type == CFS_TCD_TYPE_IRQ)
-		spin_lock_irqsave(&tcd->tcd_lock, tcd->tcd_lock_flags);
-	else if (tcd->tcd_type == CFS_TCD_TYPE_SOFTIRQ)
-		spin_lock_bh(&tcd->tcd_lock);
-	else if (unlikely(walking))
-		spin_lock_irq(&tcd->tcd_lock);
-	else
-		spin_lock(&tcd->tcd_lock);
-	return 1;
-}
-
-void cfs_trace_unlock_tcd(struct cfs_trace_cpu_data *tcd, int walking)
-	__releases(&tcd->tcd_lock)
-{
-	__LASSERT(tcd->tcd_type < CFS_TCD_TYPE_MAX);
-	if (tcd->tcd_type == CFS_TCD_TYPE_IRQ)
-		spin_unlock_irqrestore(&tcd->tcd_lock, tcd->tcd_lock_flags);
-	else if (tcd->tcd_type == CFS_TCD_TYPE_SOFTIRQ)
-		spin_unlock_bh(&tcd->tcd_lock);
-	else if (unlikely(walking))
-		spin_unlock_irq(&tcd->tcd_lock);
-	else
-		spin_unlock(&tcd->tcd_lock);
-}
-
 void
 cfs_set_ptldebug_header(struct ptldebug_header *header,
 			struct libcfs_debug_msg_data *msgdata,
diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.c b/drivers/staging/lustre/lnet/libcfs/tracefile.c
index 72321ce..6d567a9 100644
--- a/drivers/staging/lustre/lnet/libcfs/tracefile.c
+++ b/drivers/staging/lustre/lnet/libcfs/tracefile.c
@@ -109,6 +109,65 @@ struct cfs_trace_page {
 static void put_pages_on_tcd_daemon_list(struct page_collection *pc,
 					 struct cfs_trace_cpu_data *tcd);
 
+/* trace file lock routines */
+/*
+ * The walking argument indicates the locking comes from all tcd types
+ * iterator and we must lock it and dissable local irqs to avoid deadlocks
+ * with other interrupt locks that might be happening. See LU-1311
+ * for details.
+ */
+int cfs_trace_lock_tcd(struct cfs_trace_cpu_data *tcd, int walking)
+	__acquires(&tcd->tc_lock)
+{
+	__LASSERT(tcd->tcd_type < CFS_TCD_TYPE_MAX);
+	if (tcd->tcd_type == CFS_TCD_TYPE_IRQ)
+		spin_lock_irqsave(&tcd->tcd_lock, tcd->tcd_lock_flags);
+	else if (tcd->tcd_type == CFS_TCD_TYPE_SOFTIRQ)
+		spin_lock_bh(&tcd->tcd_lock);
+	else if (unlikely(walking))
+		spin_lock_irq(&tcd->tcd_lock);
+	else
+		spin_lock(&tcd->tcd_lock);
+	return 1;
+}
+
+void cfs_trace_unlock_tcd(struct cfs_trace_cpu_data *tcd, int walking)
+	__releases(&tcd->tcd_lock)
+{
+	__LASSERT(tcd->tcd_type < CFS_TCD_TYPE_MAX);
+	if (tcd->tcd_type == CFS_TCD_TYPE_IRQ)
+		spin_unlock_irqrestore(&tcd->tcd_lock, tcd->tcd_lock_flags);
+	else if (tcd->tcd_type == CFS_TCD_TYPE_SOFTIRQ)
+		spin_unlock_bh(&tcd->tcd_lock);
+	else if (unlikely(walking))
+		spin_unlock_irq(&tcd->tcd_lock);
+	else
+		spin_unlock(&tcd->tcd_lock);
+}
+
+#define cfs_tcd_for_each_type_lock(tcd, i, cpu)			   \
+	for (i = 0; cfs_trace_data[i] &&				\
+	     (tcd = &(*cfs_trace_data[i])[cpu].tcd) &&			\
+	     cfs_trace_lock_tcd(tcd, 1); cfs_trace_unlock_tcd(tcd, 1), i++)
+
+static inline struct cfs_trace_cpu_data *
+cfs_trace_get_tcd(void)
+{
+	struct cfs_trace_cpu_data *tcd =
+		&(*cfs_trace_data[cfs_trace_buf_idx_get()])[get_cpu()].tcd;
+
+	cfs_trace_lock_tcd(tcd, 0);
+
+	return tcd;
+}
+
+static inline void cfs_trace_put_tcd(struct cfs_trace_cpu_data *tcd)
+{
+	cfs_trace_unlock_tcd(tcd, 0);
+
+	put_cpu();
+}
+
 static inline struct cfs_trace_page *
 cfs_tage_from_list(struct list_head *list)
 {
diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.h b/drivers/staging/lustre/lnet/libcfs/tracefile.h
index bd1a1ef..91968cf 100644
--- a/drivers/staging/lustre/lnet/libcfs/tracefile.h
+++ b/drivers/staging/lustre/lnet/libcfs/tracefile.h
@@ -49,8 +49,6 @@ enum cfs_trace_buf_type {
 	CFS_TCD_TYPE_MAX
 };
 
-/* trace file lock routines */
-
 #define TRACEFILE_NAME_SIZE 1024
 extern char cfs_tracefile[TRACEFILE_NAME_SIZE];
 extern long long cfs_tracefile_size;
@@ -194,11 +192,6 @@ int cfs_trace_copyout_string(char __user *usr_buffer, int usr_buffer_nob,
 		     j < num_possible_cpus();				 \
 		     j++, (tcd) = &(*cfs_trace_data[i])[j].tcd)
 
-#define cfs_tcd_for_each_type_lock(tcd, i, cpu)			   \
-	for (i = 0; cfs_trace_data[i] &&				\
-	     (tcd = &(*cfs_trace_data[i])[cpu].tcd) &&			\
-	     cfs_trace_lock_tcd(tcd, 1); cfs_trace_unlock_tcd(tcd, 1), i++)
-
 void cfs_set_ptldebug_header(struct ptldebug_header *header,
 			     struct libcfs_debug_msg_data *m,
 			     unsigned long stack);
@@ -206,9 +199,6 @@ void cfs_print_to_console(struct ptldebug_header *hdr, int mask,
 			  const char *buf, int len, const char *file,
 			  const char *fn);
 
-int cfs_trace_lock_tcd(struct cfs_trace_cpu_data *tcd, int walking);
-void cfs_trace_unlock_tcd(struct cfs_trace_cpu_data *tcd, int walking);
-
 extern char *cfs_trace_console_buffers[NR_CPUS][CFS_TCD_TYPE_MAX];
 enum cfs_trace_buf_type cfs_trace_buf_idx_get(void);
 
@@ -221,24 +211,6 @@ void cfs_print_to_console(struct ptldebug_header *hdr, int mask,
 	return cfs_trace_console_buffers[i][j];
 }
 
-static inline struct cfs_trace_cpu_data *
-cfs_trace_get_tcd(void)
-{
-	struct cfs_trace_cpu_data *tcd =
-		&(*cfs_trace_data[cfs_trace_buf_idx_get()])[get_cpu()].tcd;
-
-	cfs_trace_lock_tcd(tcd, 0);
-
-	return tcd;
-}
-
-static inline void cfs_trace_put_tcd(struct cfs_trace_cpu_data *tcd)
-{
-	cfs_trace_unlock_tcd(tcd, 0);
-
-	put_cpu();
-}
-
 int cfs_trace_refill_stock(struct cfs_trace_cpu_data *tcd, gfp_t gfp,
 			   struct list_head *stock);
 
-- 
1.8.3.1


From jsimmons at infradead.org  Wed Jun 27 19:38:33 2018
From: jsimmons at infradead.org (James Simmons)
Date: Wed, 27 Jun 2018 15:38:33 -0400
Subject: [lustre-devel] [PATCH v3 04/13] lustre: libcfs: fix
 cfs_print_to_console()
In-Reply-To: <1530128322-32535-1-git-send-email-jsimmons@infradead.org>
References: <1530128322-32535-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1530128322-32535-5-git-send-email-jsimmons@infradead.org>

The original code for cfs_print_to_console() used printk() and
used tricks to select which printk level to use. Later a cleanup
patch landed the just converted printk directly to pr_info which
is not exactly correct. Instead of converting back to printk lets
move everything to pr_* type functions which simplify the code.
This allows us to fold both dbghdr_to_err_string() and the
function dbghdr_to_info_string() into cfs_print_to_console().

Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 .../staging/lustre/lnet/libcfs/linux-tracefile.c   | 55 ++++++----------------
 1 file changed, 14 insertions(+), 41 deletions(-)

diff --git a/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c b/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
index 3af7722..c1747c4 100644
--- a/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
+++ b/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
@@ -140,54 +140,27 @@ enum cfs_trace_buf_type cfs_trace_buf_idx_get(void)
 	header->ph_extern_pid = 0;
 }
 
-static char *
-dbghdr_to_err_string(struct ptldebug_header *hdr)
-{
-	switch (hdr->ph_subsys) {
-	case S_LND:
-	case S_LNET:
-		return "LNetError";
-	default:
-		return "LustreError";
-	}
-}
-
-static char *
-dbghdr_to_info_string(struct ptldebug_header *hdr)
-{
-	switch (hdr->ph_subsys) {
-	case S_LND:
-	case S_LNET:
-		return "LNet";
-	default:
-		return "Lustre";
-	}
-}
-
 void cfs_print_to_console(struct ptldebug_header *hdr, int mask,
 			  const char *buf, int len, const char *file,
 			  const char *fn)
 {
-	char *prefix = "Lustre", *ptype = NULL;
+	char *prefix = "Lustre";
+
+	if (hdr->ph_subsys == S_LND || hdr->ph_subsys == S_LNET)
+		prefix = "LNet";
 
-	if (mask & D_EMERG) {
-		prefix = dbghdr_to_err_string(hdr);
-		ptype = KERN_EMERG;
+	if (mask & (D_CONSOLE | libcfs_printk)) {
+		pr_info("%s: %.*s", prefix, len, buf);
+	} else if (mask & D_EMERG) {
+		pr_emerg("%sError: %d:%d:(%s:%d:%s()) %.*s", prefix,
+			 hdr->ph_pid, hdr->ph_extern_pid, file,
+			 hdr->ph_line_num, fn, len, buf);
 	} else if (mask & D_ERROR) {
-		prefix = dbghdr_to_err_string(hdr);
-		ptype = KERN_ERR;
+		pr_err("%sError: %d:%d:(%s:%d:%s()) %.*s", prefix,
+		       hdr->ph_pid, hdr->ph_extern_pid, file,
+		       hdr->ph_line_num, fn, len, buf);
 	} else if (mask & D_WARNING) {
-		prefix = dbghdr_to_info_string(hdr);
-		ptype = KERN_WARNING;
-	} else if (mask & (D_CONSOLE | libcfs_printk)) {
-		prefix = dbghdr_to_info_string(hdr);
-		ptype = KERN_INFO;
-	}
-
-	if (mask & D_CONSOLE) {
-		pr_info("%s%s: %.*s", ptype, prefix, len, buf);
-	} else {
-		pr_info("%s%s: %d:%d:(%s:%d:%s()) %.*s", ptype, prefix,
+		pr_warn("%s: %d:%d:(%s:%d:%s()) %.*s", prefix,
 			hdr->ph_pid, hdr->ph_extern_pid, file,
 			hdr->ph_line_num, fn, len, buf);
 	}
-- 
1.8.3.1


From jsimmons at infradead.org  Wed Jun 27 19:38:40 2018
From: jsimmons at infradead.org (James Simmons)
Date: Wed, 27 Jun 2018 15:38:40 -0400
Subject: [lustre-devel] [PATCH v3 11/13] lustre: libcfs: renamed
 CFS_TCD_TYPE_MAX to CFS_TCD_TYPE_CNT
In-Reply-To: <1530128322-32535-1-git-send-email-jsimmons@infradead.org>
References: <1530128322-32535-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1530128322-32535-12-git-send-email-jsimmons@infradead.org>

From: NeilBrown <neilb at suse.com>

The possible TCD types are 0, 1, 2.
So the MAX is 2.
The count of the number of types is 3.

CFS_TCD_TYPE_MAX is 3 - obviously wrong.

So rename it to CFS_TCD_TYPE_CNT.

Also there are 2 places where "3" is used rather
than the macro - fix them.

Signed-off-by: NeilBrown <neilb at suse.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/lnet/libcfs/tracefile.c | 16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)

diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.c b/drivers/staging/lustre/lnet/libcfs/tracefile.c
index a2b5004..b102465 100644
--- a/drivers/staging/lustre/lnet/libcfs/tracefile.c
+++ b/drivers/staging/lustre/lnet/libcfs/tracefile.c
@@ -54,12 +54,12 @@ enum cfs_trace_buf_type {
 	CFS_TCD_TYPE_PROC = 0,
 	CFS_TCD_TYPE_SOFTIRQ,
 	CFS_TCD_TYPE_IRQ,
-	CFS_TCD_TYPE_MAX
+	CFS_TCD_TYPE_CNT
 };
 
 union cfs_trace_data_union (*cfs_trace_data[TCD_MAX_TYPES])[NR_CPUS] __cacheline_aligned;
 
-char *cfs_trace_console_buffers[NR_CPUS][CFS_TCD_TYPE_MAX];
+char *cfs_trace_console_buffers[NR_CPUS][CFS_TCD_TYPE_CNT];
 char cfs_tracefile[TRACEFILE_NAME_SIZE];
 long long cfs_tracefile_size = CFS_TRACEFILE_SIZE;
 static struct tracefiled_ctl trace_tctl;
@@ -127,7 +127,7 @@ static void put_pages_on_tcd_daemon_list(struct page_collection *pc,
 int cfs_trace_lock_tcd(struct cfs_trace_cpu_data *tcd, int walking)
 	__acquires(&tcd->tc_lock)
 {
-	__LASSERT(tcd->tcd_type < CFS_TCD_TYPE_MAX);
+	__LASSERT(tcd->tcd_type < CFS_TCD_TYPE_CNT);
 	if (tcd->tcd_type == CFS_TCD_TYPE_IRQ)
 		spin_lock_irqsave(&tcd->tcd_lock, tcd->tcd_lock_flags);
 	else if (tcd->tcd_type == CFS_TCD_TYPE_SOFTIRQ)
@@ -142,7 +142,7 @@ int cfs_trace_lock_tcd(struct cfs_trace_cpu_data *tcd, int walking)
 void cfs_trace_unlock_tcd(struct cfs_trace_cpu_data *tcd, int walking)
 	__releases(&tcd->tcd_lock)
 {
-	__LASSERT(tcd->tcd_type < CFS_TCD_TYPE_MAX);
+	__LASSERT(tcd->tcd_type < CFS_TCD_TYPE_CNT);
 	if (tcd->tcd_type == CFS_TCD_TYPE_IRQ)
 		spin_unlock_irqrestore(&tcd->tcd_lock, tcd->tcd_lock_flags);
 	else if (tcd->tcd_type == CFS_TCD_TYPE_SOFTIRQ)
@@ -1249,7 +1249,7 @@ void cfs_trace_stop_thread(void)
 }
 
 /* percents to share the total debug memory for each type */
-static unsigned int pages_factor[CFS_TCD_TYPE_MAX] = {
+static unsigned int pages_factor[CFS_TCD_TYPE_CNT] = {
 	80,  /* 80% pages for CFS_TCD_TYPE_PROC */
 	10,  /* 10% pages for CFS_TCD_TYPE_SOFTIRQ */
 	10   /* 10% pages for CFS_TCD_TYPE_IRQ */
@@ -1263,7 +1263,7 @@ int cfs_tracefile_init(int max_pages)
 
 	/* initialize trace_data */
 	memset(cfs_trace_data, 0, sizeof(cfs_trace_data));
-	for (i = 0; i < CFS_TCD_TYPE_MAX; i++) {
+	for (i = 0; i < CFS_TCD_TYPE_CNT; i++) {
 		cfs_trace_data[i] =
 			kmalloc_array(num_possible_cpus(),
 				      sizeof(union cfs_trace_data_union),
@@ -1292,7 +1292,7 @@ int cfs_tracefile_init(int max_pages)
 	}
 
 	for (i = 0; i < num_possible_cpus(); i++)
-		for (j = 0; j < 3; j++) {
+		for (j = 0; j < CFS_TCD_TYPE_CNT; j++) {
 			cfs_trace_console_buffers[i][j] =
 				kmalloc(CFS_TRACE_CONSOLE_BUFFER_SIZE,
 					GFP_KERNEL);
@@ -1356,7 +1356,7 @@ static void cfs_trace_cleanup(void)
 	trace_cleanup_on_all_cpus();
 
 	for (i = 0; i < num_possible_cpus(); i++)
-		for (j = 0; j < 3; j++) {
+		for (j = 0; j < CFS_TCD_TYPE_CNT; j++) {
 			kfree(cfs_trace_console_buffers[i][j]);
 			cfs_trace_console_buffers[i][j] = NULL;
 		}
-- 
1.8.3.1


From jsimmons at infradead.org  Wed Jun 27 19:38:42 2018
From: jsimmons at infradead.org (James Simmons)
Date: Wed, 27 Jun 2018 15:38:42 -0400
Subject: [lustre-devel] [PATCH v3 13/13] lustre: libcfs: format macros in
 tracefile.h
In-Reply-To: <1530128322-32535-1-git-send-email-jsimmons@infradead.org>
References: <1530128322-32535-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1530128322-32535-14-git-send-email-jsimmons@infradead.org>

The __LASSERT_* macros are ugly and hard to read. This patch
beautifies them.

Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/lnet/libcfs/tracefile.h | 24 ++++++++++++------------
 1 file changed, 12 insertions(+), 12 deletions(-)

diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.h b/drivers/staging/lustre/lnet/libcfs/tracefile.h
index dc782a6..67107b1 100644
--- a/drivers/staging/lustre/lnet/libcfs/tracefile.h
+++ b/drivers/staging/lustre/lnet/libcfs/tracefile.h
@@ -161,21 +161,21 @@ void cfs_trace_assertion_failed(const char *str,
 				struct libcfs_debug_msg_data *m);
 
 /* ASSERTION that is safe to use within the debug system */
-#define __LASSERT(cond)						 \
-do {								    \
+#define __LASSERT(cond)							\
+do {									\
 	if (unlikely(!(cond))) {					\
-		LIBCFS_DEBUG_MSG_DATA_DECL(msgdata, D_EMERG, NULL);     \
-		cfs_trace_assertion_failed("ASSERTION("#cond") failed", \
-					   &msgdata);		   \
-	}							       \
+		LIBCFS_DEBUG_MSG_DATA_DECL(msgdata, D_EMERG, NULL);	\
+		cfs_trace_assertion_failed("ASSERTION("#cond") failed",	\
+					   &msgdata);			\
+	}								\
 } while (0)
 
-#define __LASSERT_TAGE_INVARIANT(tage)				  \
-do {								    \
-	__LASSERT(tage);					\
-	__LASSERT(tage->page);				  \
-	__LASSERT(tage->used <= PAGE_SIZE);			 \
-	__LASSERT(page_count(tage->page) > 0);		      \
+#define __LASSERT_TAGE_INVARIANT(tage)			\
+do {							\
+	__LASSERT(tage);				\
+	__LASSERT(tage->page);				\
+	__LASSERT(tage->used <= PAGE_SIZE);		\
+	__LASSERT(page_count(tage->page) > 0);		\
 } while (0)
 
 #endif /* __LIBCFS_TRACEFILE_H__ */
-- 
1.8.3.1


From jsimmons at infradead.org  Wed Jun 27 19:38:35 2018
From: jsimmons at infradead.org (James Simmons)
Date: Wed, 27 Jun 2018 15:38:35 -0400
Subject: [lustre-devel] [PATCH v3 06/13] lustre: libcfs: merge
 linux-tracefile.c into tracefile.c
In-Reply-To: <1530128322-32535-1-git-send-email-jsimmons@infradead.org>
References: <1530128322-32535-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1530128322-32535-7-git-send-email-jsimmons@infradead.org>

From: NeilBrown <neilb at suse.com>

It's good to keep related code together. The merger exposed the
flaws of cfs_print_to_console() so rework it to behavior properly.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 drivers/staging/lustre/lnet/libcfs/Makefile        |   2 +-
 .../staging/lustre/lnet/libcfs/linux-tracefile.c   | 176 ---------------------
 drivers/staging/lustre/lnet/libcfs/tracefile.c     | 135 ++++++++++++++++
 drivers/staging/lustre/lnet/libcfs/tracefile.h     |   7 -
 4 files changed, 136 insertions(+), 184 deletions(-)
 delete mode 100644 drivers/staging/lustre/lnet/libcfs/linux-tracefile.c

diff --git a/drivers/staging/lustre/lnet/libcfs/Makefile b/drivers/staging/lustre/lnet/libcfs/Makefile
index 5b13edc..cd96434 100644
--- a/drivers/staging/lustre/lnet/libcfs/Makefile
+++ b/drivers/staging/lustre/lnet/libcfs/Makefile
@@ -4,7 +4,7 @@ ccflags-y += -I$(srctree)/drivers/staging/lustre/lustre/include
 
 obj-$(CONFIG_LNET) += libcfs.o
 
-libcfs-obj-y += linux-tracefile.o linux-debug.o
+libcfs-obj-y += linux-debug.o
 libcfs-obj-y += linux-crypto.o
 libcfs-obj-y += linux-crypto-adler.o
 
diff --git a/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c b/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
deleted file mode 100644
index becc773..0000000
--- a/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
+++ /dev/null
@@ -1,176 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-/*
- * GPL HEADER START
- *
- * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 only,
- * as published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful, but
- * WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
- * General Public License version 2 for more details (a copy is included
- * in the LICENSE file that accompanied this code).
- *
- * You should have received a copy of the GNU General Public License
- * version 2 along with this program; If not, see
- * http://www.gnu.org/licenses/gpl-2.0.html
- *
- * GPL HEADER END
- */
-/*
- * Copyright (c) 2008, 2010, Oracle and/or its affiliates. All rights reserved.
- * Use is subject to license terms.
- *
- * Copyright (c) 2012, Intel Corporation.
- */
-/*
- * This file is part of Lustre, http://www.lustre.org/
- * Lustre is a trademark of Sun Microsystems, Inc.
- */
-
-#define DEBUG_SUBSYSTEM S_LNET
-#define LUSTRE_TRACEFILE_PRIVATE
-
-#include <linux/slab.h>
-#include <linux/mm.h>
-#include "tracefile.h"
-
-/* percents to share the total debug memory for each type */
-static unsigned int pages_factor[CFS_TCD_TYPE_MAX] = {
-	80,  /* 80% pages for CFS_TCD_TYPE_PROC */
-	10,  /* 10% pages for CFS_TCD_TYPE_SOFTIRQ */
-	10   /* 10% pages for CFS_TCD_TYPE_IRQ */
-};
-
-char *cfs_trace_console_buffers[NR_CPUS][CFS_TCD_TYPE_MAX];
-
-int cfs_tracefile_init_arch(void)
-{
-	struct cfs_trace_cpu_data *tcd;
-	int i;
-	int j;
-
-	/* initialize trace_data */
-	memset(cfs_trace_data, 0, sizeof(cfs_trace_data));
-	for (i = 0; i < CFS_TCD_TYPE_MAX; i++) {
-		cfs_trace_data[i] =
-			kmalloc_array(num_possible_cpus(),
-				      sizeof(union cfs_trace_data_union),
-				      GFP_KERNEL);
-		if (!cfs_trace_data[i])
-			goto out_trace_data;
-	}
-
-	/* arch related info initialized */
-	cfs_tcd_for_each(tcd, i, j) {
-		spin_lock_init(&tcd->tcd_lock);
-		tcd->tcd_pages_factor = pages_factor[i];
-		tcd->tcd_type = i;
-		tcd->tcd_cpu = j;
-	}
-
-	for (i = 0; i < num_possible_cpus(); i++)
-		for (j = 0; j < 3; j++) {
-			cfs_trace_console_buffers[i][j] =
-				kmalloc(CFS_TRACE_CONSOLE_BUFFER_SIZE,
-					GFP_KERNEL);
-
-			if (!cfs_trace_console_buffers[i][j])
-				goto out_buffers;
-		}
-
-	return 0;
-
-out_buffers:
-	for (i = 0; i < num_possible_cpus(); i++)
-		for (j = 0; j < 3; j++) {
-			kfree(cfs_trace_console_buffers[i][j]);
-			cfs_trace_console_buffers[i][j] = NULL;
-		}
-out_trace_data:
-	for (i = 0; cfs_trace_data[i]; i++) {
-		kfree(cfs_trace_data[i]);
-		cfs_trace_data[i] = NULL;
-	}
-	pr_err("lnet: Not enough memory\n");
-	return -ENOMEM;
-}
-
-void cfs_tracefile_fini_arch(void)
-{
-	int i;
-	int j;
-
-	for (i = 0; i < num_possible_cpus(); i++)
-		for (j = 0; j < 3; j++) {
-			kfree(cfs_trace_console_buffers[i][j]);
-			cfs_trace_console_buffers[i][j] = NULL;
-		}
-
-	for (i = 0; cfs_trace_data[i]; i++) {
-		kfree(cfs_trace_data[i]);
-		cfs_trace_data[i] = NULL;
-	}
-}
-
-enum cfs_trace_buf_type cfs_trace_buf_idx_get(void)
-{
-	if (in_irq())
-		return CFS_TCD_TYPE_IRQ;
-	if (in_softirq())
-		return CFS_TCD_TYPE_SOFTIRQ;
-	return CFS_TCD_TYPE_PROC;
-}
-
-void
-cfs_set_ptldebug_header(struct ptldebug_header *header,
-			struct libcfs_debug_msg_data *msgdata,
-			unsigned long stack)
-{
-	struct timespec64 ts;
-
-	ktime_get_real_ts64(&ts);
-
-	header->ph_subsys = msgdata->msg_subsys;
-	header->ph_mask = msgdata->msg_mask;
-	header->ph_cpu_id = smp_processor_id();
-	header->ph_type = cfs_trace_buf_idx_get();
-	/* y2038 safe since all user space treats this as unsigned, but
-	 * will overflow in 2106
-	 */
-	header->ph_sec = (u32)ts.tv_sec;
-	header->ph_usec = ts.tv_nsec / NSEC_PER_USEC;
-	header->ph_stack = stack;
-	header->ph_pid = current->pid;
-	header->ph_line_num = msgdata->msg_line;
-	header->ph_extern_pid = 0;
-}
-
-void cfs_print_to_console(struct ptldebug_header *hdr, int mask,
-			  const char *buf, int len, const char *file,
-			  const char *fn)
-{
-	char *prefix = "Lustre";
-
-	if (hdr->ph_subsys == S_LND || hdr->ph_subsys == S_LNET)
-		prefix = "LNet";
-
-	if (mask & (D_CONSOLE | libcfs_printk)) {
-		pr_info("%s: %.*s", prefix, len, buf);
-	} else if (mask & D_EMERG) {
-		pr_emerg("%sError: %d:%d:(%s:%d:%s()) %.*s", prefix,
-			 hdr->ph_pid, hdr->ph_extern_pid, file,
-			 hdr->ph_line_num, fn, len, buf);
-	} else if (mask & D_ERROR) {
-		pr_err("%sError: %d:%d:(%s:%d:%s()) %.*s", prefix,
-		       hdr->ph_pid, hdr->ph_extern_pid, file,
-		       hdr->ph_line_num, fn, len, buf);
-	} else if (mask & D_WARNING) {
-		pr_warn("%s: %d:%d:(%s:%d:%s()) %.*s", prefix,
-			hdr->ph_pid, hdr->ph_extern_pid, file,
-			hdr->ph_line_num, fn, len, buf);
-	}
-}
diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.c b/drivers/staging/lustre/lnet/libcfs/tracefile.c
index 6d567a9..fbf54d4 100644
--- a/drivers/staging/lustre/lnet/libcfs/tracefile.c
+++ b/drivers/staging/lustre/lnet/libcfs/tracefile.c
@@ -52,6 +52,7 @@
 /* XXX move things up to the top, comment */
 union cfs_trace_data_union (*cfs_trace_data[TCD_MAX_TYPES])[NR_CPUS] __cacheline_aligned;
 
+char *cfs_trace_console_buffers[NR_CPUS][CFS_TCD_TYPE_MAX];
 char cfs_tracefile[TRACEFILE_NAME_SIZE];
 long long cfs_tracefile_size = CFS_TRACEFILE_SIZE;
 static struct tracefiled_ctl trace_tctl;
@@ -150,6 +151,15 @@ void cfs_trace_unlock_tcd(struct cfs_trace_cpu_data *tcd, int walking)
 	     (tcd = &(*cfs_trace_data[i])[cpu].tcd) &&			\
 	     cfs_trace_lock_tcd(tcd, 1); cfs_trace_unlock_tcd(tcd, 1), i++)
 
+enum cfs_trace_buf_type cfs_trace_buf_idx_get(void)
+{
+	if (in_irq())
+		return CFS_TCD_TYPE_IRQ;
+	if (in_softirq())
+		return CFS_TCD_TYPE_SOFTIRQ;
+	return CFS_TCD_TYPE_PROC;
+}
+
 static inline struct cfs_trace_cpu_data *
 cfs_trace_get_tcd(void)
 {
@@ -168,6 +178,82 @@ static inline void cfs_trace_put_tcd(struct cfs_trace_cpu_data *tcd)
 	put_cpu();
 }
 
+/* percents to share the total debug memory for each type */
+static unsigned int pages_factor[CFS_TCD_TYPE_MAX] = {
+	80,  /* 80% pages for CFS_TCD_TYPE_PROC */
+	10,  /* 10% pages for CFS_TCD_TYPE_SOFTIRQ */
+	10   /* 10% pages for CFS_TCD_TYPE_IRQ */
+};
+
+int cfs_tracefile_init_arch(void)
+{
+	struct cfs_trace_cpu_data *tcd;
+	int i;
+	int j;
+
+	/* initialize trace_data */
+	memset(cfs_trace_data, 0, sizeof(cfs_trace_data));
+	for (i = 0; i < CFS_TCD_TYPE_MAX; i++) {
+		cfs_trace_data[i] =
+			kmalloc_array(num_possible_cpus(),
+				      sizeof(union cfs_trace_data_union),
+				      GFP_KERNEL);
+		if (!cfs_trace_data[i])
+			goto out_trace_data;
+	}
+
+	/* arch related info initialized */
+	cfs_tcd_for_each(tcd, i, j) {
+		spin_lock_init(&tcd->tcd_lock);
+		tcd->tcd_pages_factor = pages_factor[i];
+		tcd->tcd_type = i;
+		tcd->tcd_cpu = j;
+	}
+
+	for (i = 0; i < num_possible_cpus(); i++)
+		for (j = 0; j < 3; j++) {
+			cfs_trace_console_buffers[i][j] =
+				kmalloc(CFS_TRACE_CONSOLE_BUFFER_SIZE,
+					GFP_KERNEL);
+
+			if (!cfs_trace_console_buffers[i][j])
+				goto out_buffers;
+		}
+
+	return 0;
+
+out_buffers:
+	for (i = 0; i < num_possible_cpus(); i++)
+		for (j = 0; j < 3; j++) {
+			kfree(cfs_trace_console_buffers[i][j]);
+			cfs_trace_console_buffers[i][j] = NULL;
+		}
+out_trace_data:
+	for (i = 0; cfs_trace_data[i]; i++) {
+		kfree(cfs_trace_data[i]);
+		cfs_trace_data[i] = NULL;
+	}
+	pr_err("lnet: Not enough memory\n");
+	return -ENOMEM;
+}
+
+void cfs_tracefile_fini_arch(void)
+{
+	int i;
+	int j;
+
+	for (i = 0; i < num_possible_cpus(); i++)
+		for (j = 0; j < 3; j++) {
+			kfree(cfs_trace_console_buffers[i][j]);
+			cfs_trace_console_buffers[i][j] = NULL;
+		}
+
+	for (i = 0; cfs_trace_data[i]; i++) {
+		kfree(cfs_trace_data[i]);
+		cfs_trace_data[i] = NULL;
+	}
+}
+
 static inline struct cfs_trace_page *
 cfs_tage_from_list(struct list_head *list)
 {
@@ -340,6 +426,55 @@ static struct cfs_trace_page *cfs_trace_get_tage(struct cfs_trace_cpu_data *tcd,
 	return tage;
 }
 
+static void cfs_set_ptldebug_header(struct ptldebug_header *header,
+				    struct libcfs_debug_msg_data *msgdata,
+				    unsigned long stack)
+{
+	struct timespec64 ts;
+
+	ktime_get_real_ts64(&ts);
+
+	header->ph_subsys = msgdata->msg_subsys;
+	header->ph_mask = msgdata->msg_mask;
+	header->ph_cpu_id = smp_processor_id();
+	header->ph_type = cfs_trace_buf_idx_get();
+	/* y2038 safe since all user space treats this as unsigned, but
+	 * will overflow in 2106
+	 */
+	header->ph_sec = (u32)ts.tv_sec;
+	header->ph_usec = ts.tv_nsec / NSEC_PER_USEC;
+	header->ph_stack = stack;
+	header->ph_pid = current->pid;
+	header->ph_line_num = msgdata->msg_line;
+	header->ph_extern_pid = 0;
+}
+
+static void cfs_print_to_console(struct ptldebug_header *hdr, int mask,
+				 const char *buf, int len, const char *file,
+				 const char *fn)
+{
+	char *prefix = "Lustre";
+
+	if (hdr->ph_subsys == S_LND || hdr->ph_subsys == S_LNET)
+		prefix = "LNet";
+
+	if (mask & (D_CONSOLE | libcfs_printk)) {
+		pr_info("%s: %.*s", prefix, len, buf);
+	} else if (mask & D_EMERG) {
+		pr_emerg("%sError: %d:%d:(%s:%d:%s()) %.*s", prefix,
+			 hdr->ph_pid, hdr->ph_extern_pid, file,
+			 hdr->ph_line_num, fn, len, buf);
+	} else if (mask & D_ERROR) {
+		pr_err("%sError: %d:%d:(%s:%d:%s()) %.*s", prefix,
+		       hdr->ph_pid, hdr->ph_extern_pid, file,
+		       hdr->ph_line_num, fn, len, buf);
+	} else if (mask & D_WARNING) {
+		pr_warn("%s: %d:%d:(%s:%d:%s()) %.*s", prefix,
+			hdr->ph_pid, hdr->ph_extern_pid, file,
+			hdr->ph_line_num, fn, len, buf);
+	}
+}
+
 int libcfs_debug_msg(struct libcfs_debug_msg_data *msgdata,
 		     const char *format, ...)
 {
diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.h b/drivers/staging/lustre/lnet/libcfs/tracefile.h
index 91968cf..399e0bf 100644
--- a/drivers/staging/lustre/lnet/libcfs/tracefile.h
+++ b/drivers/staging/lustre/lnet/libcfs/tracefile.h
@@ -192,13 +192,6 @@ int cfs_trace_copyout_string(char __user *usr_buffer, int usr_buffer_nob,
 		     j < num_possible_cpus();				 \
 		     j++, (tcd) = &(*cfs_trace_data[i])[j].tcd)
 
-void cfs_set_ptldebug_header(struct ptldebug_header *header,
-			     struct libcfs_debug_msg_data *m,
-			     unsigned long stack);
-void cfs_print_to_console(struct ptldebug_header *hdr, int mask,
-			  const char *buf, int len, const char *file,
-			  const char *fn);
-
 extern char *cfs_trace_console_buffers[NR_CPUS][CFS_TCD_TYPE_MAX];
 enum cfs_trace_buf_type cfs_trace_buf_idx_get(void);
 
-- 
1.8.3.1


From jsimmons at infradead.org  Wed Jun 27 19:38:38 2018
From: jsimmons at infradead.org (James Simmons)
Date: Wed, 27 Jun 2018 15:38:38 -0400
Subject: [lustre-devel] [PATCH v3 09/13] lustre: libcfs: cleanup tracefile.h
In-Reply-To: <1530128322-32535-1-git-send-email-jsimmons@infradead.org>
References: <1530128322-32535-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1530128322-32535-10-git-send-email-jsimmons@infradead.org>

With many things moved into tracefile.c we can cleanup a lot of
things in tracefile.h. Move some items that are only used in
tracefile.c from tracefile.h into tracefile.c. In tracefile.h
we have the ifdef LUSTRE_TRACEFILE_PRIVATE which has several
duplicate defines. We can remove those duplicates.

Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/lnet/libcfs/tracefile.c | 18 +++++++++++--
 drivers/staging/lustre/lnet/libcfs/tracefile.h | 37 --------------------------
 2 files changed, 16 insertions(+), 39 deletions(-)

diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.c b/drivers/staging/lustre/lnet/libcfs/tracefile.c
index 914cd94..5095e66 100644
--- a/drivers/staging/lustre/lnet/libcfs/tracefile.c
+++ b/drivers/staging/lustre/lnet/libcfs/tracefile.c
@@ -37,8 +37,6 @@
  */
 
 #define DEBUG_SUBSYSTEM S_LNET
-#define LUSTRE_TRACEFILE_PRIVATE
-#define pr_fmt(fmt) "Lustre: " fmt
 
 #include <linux/ratelimit.h>
 #include <linux/highmem.h>
@@ -49,8 +47,16 @@
 #include <linux/uaccess.h>
 #include "tracefile.h"
 
+#define CFS_TRACE_CONSOLE_BUFFER_SIZE	1024
 #define TCD_MAX_TYPES			8
 
+enum cfs_trace_buf_type {
+	CFS_TCD_TYPE_PROC = 0,
+	CFS_TCD_TYPE_SOFTIRQ,
+	CFS_TCD_TYPE_IRQ,
+	CFS_TCD_TYPE_MAX
+};
+
 union cfs_trace_data_union (*cfs_trace_data[TCD_MAX_TYPES])[NR_CPUS] __cacheline_aligned;
 
 char *cfs_trace_console_buffers[NR_CPUS][CFS_TCD_TYPE_MAX];
@@ -168,6 +174,14 @@ enum cfs_trace_buf_type cfs_trace_buf_idx_get(void)
 	return CFS_TCD_TYPE_PROC;
 }
 
+static inline char *cfs_trace_get_console_buffer(void)
+{
+	unsigned int i = get_cpu();
+	unsigned int j = cfs_trace_buf_idx_get();
+
+	return cfs_trace_console_buffers[i][j];
+}
+
 static inline struct cfs_trace_cpu_data *
 cfs_trace_get_tcd(void)
 {
diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.h b/drivers/staging/lustre/lnet/libcfs/tracefile.h
index 072c720..3e7b6fa 100644
--- a/drivers/staging/lustre/lnet/libcfs/tracefile.h
+++ b/drivers/staging/lustre/lnet/libcfs/tracefile.h
@@ -42,13 +42,6 @@
 #include <linux/smp.h>
 #include <linux/libcfs/libcfs.h>
 
-enum cfs_trace_buf_type {
-	CFS_TCD_TYPE_PROC = 0,
-	CFS_TCD_TYPE_SOFTIRQ,
-	CFS_TCD_TYPE_IRQ,
-	CFS_TCD_TYPE_MAX
-};
-
 #define TRACEFILE_NAME_SIZE 1024
 extern char cfs_tracefile[TRACEFILE_NAME_SIZE];
 extern long long cfs_tracefile_size;
@@ -91,22 +84,6 @@ int cfs_trace_copyout_string(char __user *usr_buffer, int usr_buffer_nob,
 #define TCD_STOCK_PAGES (TCD_MAX_PAGES)
 #define CFS_TRACEFILE_SIZE (500 << 20)
 
-#ifdef LUSTRE_TRACEFILE_PRIVATE
-
-/*
- * Private declare for tracefile
- */
-#define TCD_MAX_PAGES (5 << (20 - PAGE_SHIFT))
-#define TCD_STOCK_PAGES (TCD_MAX_PAGES)
-
-#define CFS_TRACEFILE_SIZE (500 << 20)
-
-/*
- * Size of a buffer for sprinting console messages if we can't get a page
- * from system
- */
-#define CFS_TRACE_CONSOLE_BUFFER_SIZE   1024
-
 union cfs_trace_data_union {
 	struct cfs_trace_cpu_data {
 		/*
@@ -183,18 +160,6 @@ int cfs_trace_copyout_string(char __user *usr_buffer, int usr_buffer_nob,
 	char __pad[L1_CACHE_ALIGN(sizeof(struct cfs_trace_cpu_data))];
 };
 
-extern char *cfs_trace_console_buffers[NR_CPUS][CFS_TCD_TYPE_MAX];
-enum cfs_trace_buf_type cfs_trace_buf_idx_get(void);
-
-static inline char *
-cfs_trace_get_console_buffer(void)
-{
-	unsigned int i = get_cpu();
-	unsigned int j = cfs_trace_buf_idx_get();
-
-	return cfs_trace_console_buffers[i][j];
-}
-
 void cfs_trace_assertion_failed(const char *str,
 				struct libcfs_debug_msg_data *m);
 
@@ -216,6 +181,4 @@ void cfs_trace_assertion_failed(const char *str,
 	__LASSERT(page_count(tage->page) > 0);		      \
 } while (0)
 
-#endif	/* LUSTRE_TRACEFILE_PRIVATE */
-
 #endif /* __LIBCFS_TRACEFILE_H__ */
-- 
1.8.3.1


From jsimmons at infradead.org  Wed Jun 27 19:38:39 2018
From: jsimmons at infradead.org (James Simmons)
Date: Wed, 27 Jun 2018 15:38:39 -0400
Subject: [lustre-devel] [PATCH v3 10/13] lustre: libcfs: fold
 cfs_tracefile_*_arch into their only callers.
In-Reply-To: <1530128322-32535-1-git-send-email-jsimmons@infradead.org>
References: <1530128322-32535-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1530128322-32535-11-git-send-email-jsimmons@infradead.org>

From: NeilBrown <neilb at suse.com>

There is no need to separate "arch" init/fini from
the rest, so fold it all in.
This requires some slightly subtle changes to clean-up
to make sure we don't walk lists before they are
initialized.

Signed-off-by: NeilBrown <neilb at suse.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/lnet/libcfs/tracefile.c | 147 +++++++++++--------------
 drivers/staging/lustre/lnet/libcfs/tracefile.h |   3 -
 2 files changed, 63 insertions(+), 87 deletions(-)

diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.c b/drivers/staging/lustre/lnet/libcfs/tracefile.c
index 5095e66..a2b5004 100644
--- a/drivers/staging/lustre/lnet/libcfs/tracefile.c
+++ b/drivers/staging/lustre/lnet/libcfs/tracefile.c
@@ -200,82 +200,6 @@ static inline void cfs_trace_put_tcd(struct cfs_trace_cpu_data *tcd)
 	put_cpu();
 }
 
-/* percents to share the total debug memory for each type */
-static unsigned int pages_factor[CFS_TCD_TYPE_MAX] = {
-	80,  /* 80% pages for CFS_TCD_TYPE_PROC */
-	10,  /* 10% pages for CFS_TCD_TYPE_SOFTIRQ */
-	10   /* 10% pages for CFS_TCD_TYPE_IRQ */
-};
-
-int cfs_tracefile_init_arch(void)
-{
-	struct cfs_trace_cpu_data *tcd;
-	int i;
-	int j;
-
-	/* initialize trace_data */
-	memset(cfs_trace_data, 0, sizeof(cfs_trace_data));
-	for (i = 0; i < CFS_TCD_TYPE_MAX; i++) {
-		cfs_trace_data[i] =
-			kmalloc_array(num_possible_cpus(),
-				      sizeof(union cfs_trace_data_union),
-				      GFP_KERNEL);
-		if (!cfs_trace_data[i])
-			goto out_trace_data;
-	}
-
-	/* arch related info initialized */
-	cfs_tcd_for_each(tcd, i, j) {
-		spin_lock_init(&tcd->tcd_lock);
-		tcd->tcd_pages_factor = pages_factor[i];
-		tcd->tcd_type = i;
-		tcd->tcd_cpu = j;
-	}
-
-	for (i = 0; i < num_possible_cpus(); i++)
-		for (j = 0; j < 3; j++) {
-			cfs_trace_console_buffers[i][j] =
-				kmalloc(CFS_TRACE_CONSOLE_BUFFER_SIZE,
-					GFP_KERNEL);
-
-			if (!cfs_trace_console_buffers[i][j])
-				goto out_buffers;
-		}
-
-	return 0;
-
-out_buffers:
-	for (i = 0; i < num_possible_cpus(); i++)
-		for (j = 0; j < 3; j++) {
-			kfree(cfs_trace_console_buffers[i][j]);
-			cfs_trace_console_buffers[i][j] = NULL;
-		}
-out_trace_data:
-	for (i = 0; cfs_trace_data[i]; i++) {
-		kfree(cfs_trace_data[i]);
-		cfs_trace_data[i] = NULL;
-	}
-	pr_err("lnet: Not enough memory\n");
-	return -ENOMEM;
-}
-
-void cfs_tracefile_fini_arch(void)
-{
-	int i;
-	int j;
-
-	for (i = 0; i < num_possible_cpus(); i++)
-		for (j = 0; j < 3; j++) {
-			kfree(cfs_trace_console_buffers[i][j]);
-			cfs_trace_console_buffers[i][j] = NULL;
-		}
-
-	for (i = 0; cfs_trace_data[i]; i++) {
-		kfree(cfs_trace_data[i]);
-		cfs_trace_data[i] = NULL;
-	}
-}
-
 static inline struct cfs_trace_page *
 cfs_tage_from_list(struct list_head *list)
 {
@@ -1324,21 +1248,38 @@ void cfs_trace_stop_thread(void)
 	mutex_unlock(&cfs_trace_thread_mutex);
 }
 
+/* percents to share the total debug memory for each type */
+static unsigned int pages_factor[CFS_TCD_TYPE_MAX] = {
+	80,  /* 80% pages for CFS_TCD_TYPE_PROC */
+	10,  /* 10% pages for CFS_TCD_TYPE_SOFTIRQ */
+	10   /* 10% pages for CFS_TCD_TYPE_IRQ */
+};
+
 int cfs_tracefile_init(int max_pages)
 {
 	struct cfs_trace_cpu_data *tcd;
 	int i;
 	int j;
-	int rc;
-	int factor;
 
-	rc = cfs_tracefile_init_arch();
-	if (rc)
-		return rc;
+	/* initialize trace_data */
+	memset(cfs_trace_data, 0, sizeof(cfs_trace_data));
+	for (i = 0; i < CFS_TCD_TYPE_MAX; i++) {
+		cfs_trace_data[i] =
+			kmalloc_array(num_possible_cpus(),
+				      sizeof(union cfs_trace_data_union),
+				      GFP_KERNEL);
+		if (!cfs_trace_data[i])
+			goto out_trace_data;
+	}
 
+	/* arch related info initialized */
 	cfs_tcd_for_each(tcd, i, j) {
-		/* tcd_pages_factor is initialized int tracefile_init_arch. */
-		factor = tcd->tcd_pages_factor;
+		int factor = pages_factor[i];
+
+		spin_lock_init(&tcd->tcd_lock);
+		tcd->tcd_pages_factor = factor;
+		tcd->tcd_type = i;
+		tcd->tcd_cpu = j;
 		INIT_LIST_HEAD(&tcd->tcd_pages);
 		INIT_LIST_HEAD(&tcd->tcd_stock_pages);
 		INIT_LIST_HEAD(&tcd->tcd_daemon_pages);
@@ -1350,7 +1291,31 @@ int cfs_tracefile_init(int max_pages)
 		tcd->tcd_shutting_down = 0;
 	}
 
+	for (i = 0; i < num_possible_cpus(); i++)
+		for (j = 0; j < 3; j++) {
+			cfs_trace_console_buffers[i][j] =
+				kmalloc(CFS_TRACE_CONSOLE_BUFFER_SIZE,
+					GFP_KERNEL);
+
+			if (!cfs_trace_console_buffers[i][j])
+				goto out_buffers;
+		}
+
 	return 0;
+
+out_buffers:
+	for (i = 0; i < num_possible_cpus(); i++)
+		for (j = 0; j < 3; j++) {
+			kfree(cfs_trace_console_buffers[i][j]);
+			cfs_trace_console_buffers[i][j] = NULL;
+		}
+out_trace_data:
+	for (i = 0; cfs_trace_data[i]; i++) {
+		kfree(cfs_trace_data[i]);
+		cfs_trace_data[i] = NULL;
+	}
+	pr_err("lnet: Not enough memory\n");
+	return -ENOMEM;
 }
 
 static void trace_cleanup_on_all_cpus(void)
@@ -1362,6 +1327,9 @@ static void trace_cleanup_on_all_cpus(void)
 
 	for_each_possible_cpu(cpu) {
 		cfs_tcd_for_each_type_lock(tcd, i, cpu) {
+			if (!tcd->tcd_pages_factor)
+				/* Not initialised */
+				continue;
 			tcd->tcd_shutting_down = 1;
 
 			list_for_each_entry_safe(tage, tmp, &tcd->tcd_pages,
@@ -1380,12 +1348,23 @@ static void trace_cleanup_on_all_cpus(void)
 static void cfs_trace_cleanup(void)
 {
 	struct page_collection pc;
+	int i;
+	int j;
 
 	INIT_LIST_HEAD(&pc.pc_pages);
 
 	trace_cleanup_on_all_cpus();
 
-	cfs_tracefile_fini_arch();
+	for (i = 0; i < num_possible_cpus(); i++)
+		for (j = 0; j < 3; j++) {
+			kfree(cfs_trace_console_buffers[i][j]);
+			cfs_trace_console_buffers[i][j] = NULL;
+		}
+
+	for (i = 0; cfs_trace_data[i]; i++) {
+		kfree(cfs_trace_data[i]);
+		cfs_trace_data[i] = NULL;
+	}
 }
 
 void cfs_tracefile_exit(void)
diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.h b/drivers/staging/lustre/lnet/libcfs/tracefile.h
index 3e7b6fa..dc782a6 100644
--- a/drivers/staging/lustre/lnet/libcfs/tracefile.h
+++ b/drivers/staging/lustre/lnet/libcfs/tracefile.h
@@ -53,9 +53,6 @@
 
 void libcfs_run_debug_log_upcall(char *file);
 
-int  cfs_tracefile_init_arch(void);
-void cfs_tracefile_fini_arch(void);
-
 int cfs_tracefile_dump_all_pages(char *filename);
 void cfs_trace_debug_print(void);
 void cfs_trace_flush_pages(void);
-- 
1.8.3.1


From jsimmons at infradead.org  Wed Jun 27 19:38:41 2018
From: jsimmons at infradead.org (James Simmons)
Date: Wed, 27 Jun 2018 15:38:41 -0400
Subject: [lustre-devel] [PATCH v3 12/13] lustre: libcfs: discard
 TCD_MAX_TYPES
In-Reply-To: <1530128322-32535-1-git-send-email-jsimmons@infradead.org>
References: <1530128322-32535-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1530128322-32535-13-git-send-email-jsimmons@infradead.org>

From: NeilBrown <neilb at suse.com>

As well as CFS_TCD_TYPE_CNT we have TCD_MAX_TYPES which has a larger
value but a similar meaning.  Discard it and just use
CFS_TCD_TYPE_CNT.

Two places relied on the fact that TCD_MAX_TYPES was larger and so
there would be NULLs at the end of the array.  Change
them to check the array size properly.

Signed-off-by: NeilBrown <neilb at suse.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/lnet/libcfs/tracefile.c | 9 ++++-----
 1 file changed, 4 insertions(+), 5 deletions(-)

diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.c b/drivers/staging/lustre/lnet/libcfs/tracefile.c
index b102465..0f64fa2 100644
--- a/drivers/staging/lustre/lnet/libcfs/tracefile.c
+++ b/drivers/staging/lustre/lnet/libcfs/tracefile.c
@@ -48,7 +48,6 @@
 #include "tracefile.h"
 
 #define CFS_TRACE_CONSOLE_BUFFER_SIZE	1024
-#define TCD_MAX_TYPES			8
 
 enum cfs_trace_buf_type {
 	CFS_TCD_TYPE_PROC = 0,
@@ -57,7 +56,7 @@ enum cfs_trace_buf_type {
 	CFS_TCD_TYPE_CNT
 };
 
-union cfs_trace_data_union (*cfs_trace_data[TCD_MAX_TYPES])[NR_CPUS] __cacheline_aligned;
+union cfs_trace_data_union (*cfs_trace_data[CFS_TCD_TYPE_CNT])[NR_CPUS] __cacheline_aligned;
 
 char *cfs_trace_console_buffers[NR_CPUS][CFS_TCD_TYPE_CNT];
 char cfs_tracefile[TRACEFILE_NAME_SIZE];
@@ -154,14 +153,14 @@ void cfs_trace_unlock_tcd(struct cfs_trace_cpu_data *tcd, int walking)
 }
 
 #define cfs_tcd_for_each(tcd, i, j)					\
-	for (i = 0; cfs_trace_data[i]; i++)				\
+	for (i = 0; i < CFS_TCD_TYPE_CNT && cfs_trace_data[i]; i++)	\
 		for (j = 0, ((tcd) = &(*cfs_trace_data[i])[j].tcd);	\
 		     j < num_possible_cpus();				\
 		     j++, (tcd) = &(*cfs_trace_data[i])[j].tcd)
 
 
 #define cfs_tcd_for_each_type_lock(tcd, i, cpu)				\
-	for (i = 0; cfs_trace_data[i] &&				\
+	for (i = 0;  i < CFS_TCD_TYPE_CNT && cfs_trace_data[i] &&	\
 	     (tcd = &(*cfs_trace_data[i])[cpu].tcd) &&			\
 	     cfs_trace_lock_tcd(tcd, 1); cfs_trace_unlock_tcd(tcd, 1), i++)
 
@@ -1361,7 +1360,7 @@ static void cfs_trace_cleanup(void)
 			cfs_trace_console_buffers[i][j] = NULL;
 		}
 
-	for (i = 0; cfs_trace_data[i]; i++) {
+	for (i = 0; i < CFS_TCD_TYPE_CNT && cfs_trace_data[i]; i++) {
 		kfree(cfs_trace_data[i]);
 		cfs_trace_data[i] = NULL;
 	}
-- 
1.8.3.1


From neilb at suse.com  Wed Jun 27 21:27:16 2018
From: neilb at suse.com (NeilBrown)
Date: Thu, 28 Jun 2018 07:27:16 +1000
Subject: [lustre-devel] [PATCH v3 04/13] lustre: libcfs: fix
 cfs_print_to_console()
In-Reply-To: <1530128322-32535-5-git-send-email-jsimmons@infradead.org>
References: <1530128322-32535-1-git-send-email-jsimmons@infradead.org>
 <1530128322-32535-5-git-send-email-jsimmons@infradead.org>
Message-ID: <8760237wej.fsf@notabene.neil.brown.name>

On Wed, Jun 27 2018, James Simmons wrote:

> The original code for cfs_print_to_console() used printk() and
> used tricks to select which printk level to use. Later a cleanup
> patch landed the just converted printk directly to pr_info which
> is not exactly correct. Instead of converting back to printk lets
> move everything to pr_* type functions which simplify the code.
> This allows us to fold both dbghdr_to_err_string() and the
> function dbghdr_to_info_string() into cfs_print_to_console().
>
> Signed-off-by: James Simmons <jsimmons at infradead.org>

Thanks!
This code is much nicer now, and the whole series is much easier to
review now that you've broken it up.

It'll all appear in my lustre-testing shortly.

Thanks,
NeilBrown


> ---
>  .../staging/lustre/lnet/libcfs/linux-tracefile.c   | 55 ++++++----------------
>  1 file changed, 14 insertions(+), 41 deletions(-)
>
> diff --git a/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c b/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
> index 3af7722..c1747c4 100644
> --- a/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
> +++ b/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
> @@ -140,54 +140,27 @@ enum cfs_trace_buf_type cfs_trace_buf_idx_get(void)
>  	header->ph_extern_pid = 0;
>  }
>  
> -static char *
> -dbghdr_to_err_string(struct ptldebug_header *hdr)
> -{
> -	switch (hdr->ph_subsys) {
> -	case S_LND:
> -	case S_LNET:
> -		return "LNetError";
> -	default:
> -		return "LustreError";
> -	}
> -}
> -
> -static char *
> -dbghdr_to_info_string(struct ptldebug_header *hdr)
> -{
> -	switch (hdr->ph_subsys) {
> -	case S_LND:
> -	case S_LNET:
> -		return "LNet";
> -	default:
> -		return "Lustre";
> -	}
> -}
> -
>  void cfs_print_to_console(struct ptldebug_header *hdr, int mask,
>  			  const char *buf, int len, const char *file,
>  			  const char *fn)
>  {
> -	char *prefix = "Lustre", *ptype = NULL;
> +	char *prefix = "Lustre";
> +
> +	if (hdr->ph_subsys == S_LND || hdr->ph_subsys == S_LNET)
> +		prefix = "LNet";
>  
> -	if (mask & D_EMERG) {
> -		prefix = dbghdr_to_err_string(hdr);
> -		ptype = KERN_EMERG;
> +	if (mask & (D_CONSOLE | libcfs_printk)) {
> +		pr_info("%s: %.*s", prefix, len, buf);
> +	} else if (mask & D_EMERG) {
> +		pr_emerg("%sError: %d:%d:(%s:%d:%s()) %.*s", prefix,
> +			 hdr->ph_pid, hdr->ph_extern_pid, file,
> +			 hdr->ph_line_num, fn, len, buf);
>  	} else if (mask & D_ERROR) {
> -		prefix = dbghdr_to_err_string(hdr);
> -		ptype = KERN_ERR;
> +		pr_err("%sError: %d:%d:(%s:%d:%s()) %.*s", prefix,
> +		       hdr->ph_pid, hdr->ph_extern_pid, file,
> +		       hdr->ph_line_num, fn, len, buf);
>  	} else if (mask & D_WARNING) {
> -		prefix = dbghdr_to_info_string(hdr);
> -		ptype = KERN_WARNING;
> -	} else if (mask & (D_CONSOLE | libcfs_printk)) {
> -		prefix = dbghdr_to_info_string(hdr);
> -		ptype = KERN_INFO;
> -	}
> -
> -	if (mask & D_CONSOLE) {
> -		pr_info("%s%s: %.*s", ptype, prefix, len, buf);
> -	} else {
> -		pr_info("%s%s: %d:%d:(%s:%d:%s()) %.*s", ptype, prefix,
> +		pr_warn("%s: %d:%d:(%s:%d:%s()) %.*s", prefix,
>  			hdr->ph_pid, hdr->ph_extern_pid, file,
>  			hdr->ph_line_num, fn, len, buf);
>  	}
> -- 
> 1.8.3.1
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180628/510b8380/attachment.sig>

From neilb at suse.com  Thu Jun 28 01:17:03 2018
From: neilb at suse.com (NeilBrown)
Date: Thu, 28 Jun 2018 11:17:03 +1000
Subject: [lustre-devel] [PATCH v3 07/26] staging: lustre: libcfs: NUMA
 support
In-Reply-To: <DM3PR1101MB1213FD1001256E565FF011FFCB480@DM3PR1101MB1213.namprd11.prod.outlook.com>
References: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
 <1529875250-11531-8-git-send-email-jsimmons@infradead.org>
 <877emnaed8.fsf@notabene.neil.brown.name>
 <A764DED1-A9A8-4FA5-A0F1-922C3E2D822A@cray.com>
 <87lgb16j8q.fsf@notabene.neil.brown.name>
 <DM3PR1101MB1213FD1001256E565FF011FFCB480@DM3PR1101MB1213.namprd11.prod.outlook.com>
Message-ID: <8736x77lrk.fsf@notabene.neil.brown.name>


I went digging and found that Linux already has a well defined concept
of distance between NUMA nodes.
On x86 (and amd64?), this is loaded from ACPI.  Other platforms can
describe it in devicetree.
You can view distance information in
  /sys/devices/system/node/node*/distance

or using "numactl --hardware".

Why doesn't lustre simple extract and use this information?  Why does
lustre need to allow it to be configured?

Thanks,
NeilBrown

On Wed, Jun 27 2018, Patrick Farrell wrote:

> Neil,
>
> I am not the person at Cray for this, but if SUSE does take an interest in this, Cray would probably be interested in weighing in and contributing info if not actually code.  In fact, other HPC vendors like HPE(by which I mostly mean the old SGI) or IBM might as well.  NUMA optimization is a persistent fascination in our area of the industry...
>
> - Patrick
>
> ________________________________
> From: lustre-devel <lustre-devel-bounces at lists.lustre.org> on behalf of NeilBrown <neilb at suse.com>
> Sent: Tuesday, June 26, 2018 9:44:37 PM
> To: Doug Oucharek
> Cc: Amir Shehata; Lustre Development List
> Subject: Re: [lustre-devel] [PATCH v3 07/26] staging: lustre: libcfs: NUMA support
>
> On Mon, Jun 25 2018, Doug Oucharek wrote:
>
>> Some background on this NUMA change:
>>
>> First off, this is just a first step to a bigger set of changes which include changes to the Lustre utilities.  This was done as part of the Multi-Rail feature.  One of the systems that feature is meant to support is the SGI UV system (now HPE) which has a massive number of NUMA nodes connected by a NUMA Link.  There are multiple fabric cards spread throughout the system and Multi-Rail needs to know which fabric cards are nearest to the NUMA node we are running on.  To do that, the “distance” between NUMA nodes needs to be configured.
>>
>> This patch is preparing the infrastructure for the Multi-Rail feature to support configuring NUMA node distances.  Technically, this patch should be landing with the Multi-Rail feature (still to be pushed) for it to make proper sense.
>>
>
> Thanks a lot for the background.
>
> If these NUMA nodes have a 'distance' between them, and if lustre can
> benefit from knowing the distance, then is seems likely that other code
> might also benefit.  In that case it would be best if the distance were
> encoded in some global state information so that lustre and any other
> subsystem can extract it.
>
> Do you know if there is any work underway by anyone to make this
> information generally available?  If there is, we should make sure that
> lustre works in a compatible way so that once that work lands, lustre
> can use it directly and not need extra configuration.
> If no such work is underway, then it would be really good if something
> were done in that direction.  If no-one here is able to work on this, I
> can ask around in SUSE and see if anyone here knows anything relevant.
>
> Thanks,
> NeilBrown
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180628/2e5f3d39/attachment.sig>

From neilb at suse.com  Thu Jun 28 01:26:50 2018
From: neilb at suse.com (NeilBrown)
Date: Thu, 28 Jun 2018 11:26:50 +1000
Subject: [lustre-devel] [PATCH 00/24] lustre - more cleanups including
 module reduction.
In-Reply-To: <913847E3-EB34-4B9D-9314-0BCF59837AFC@cray.com>
References: <152904663333.10587.10934053155404014785.stgit@noble>
 <alpine.LFD.2.21.1806210239190.3886@casper.infradead.org>
 <DM3PR1101MB1213DDB7FC857EE6B1444153CB760@DM3PR1101MB1213.namprd11.prod.outlook.com>
 <alpine.LFD.2.21.1806210354260.3886@casper.infradead.org>
 <3FD4D051-9C95-449D-8A23-D42B271E55B8@dilger.ca>
 <DM3PR1101MB121322E85FDE1E5310F6F4B7CB760@DM3PR1101MB1213.namprd11.prod.outlook.com>
 <87tvprah32.fsf@notabene.neil.brown.name>
 <alpine.LFD.2.21.1806260206360.31196@casper.infradead.org>
 <DM3PR1101MB121358672276198B843DE5A7CB490@DM3PR1101MB1213.namprd11.prod.outlook.com>
 <87d0wd6i4m.fsf@notabene.neil.brown.name>
 <913847E3-EB34-4B9D-9314-0BCF59837AFC@cray.com>
Message-ID: <87zhzf66qt.fsf@notabene.neil.brown.name>

On Wed, Jun 27 2018, Cory Spitz wrote:

> Hello, Neil.
>
> Cray does indeed have another user of LNet.  While it is now GPL, I can't say if it will ever be in mainline.  However, even without it there is a case for LNet to live on its own as Andreas pointed out recently with http://lists.lustre.org/pipermail/lustre-devel-lustre.org/2018-June/007098.html.
>
> But, if you want to group everything together now, we can hopefully tease it apart again in the future.  If so, I think that it would be misleading to call a module lnet if it contained distributed locking functionality (ldlm).  How about lustre-common, lustre-core, or some such?
>

Do you have a link to this open-source thing that uses LNet?  Does it
have a name?  Is it the Zest thing that Andreas' email linked to?  Or
maybe it's Cray-DVS??  A link to a git tree would be very helpful.

(I'm not committed to any name before I understand what the encapsulated
functionality would be)

Thanks,
NeilBrown
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180628/2a55161c/attachment.sig>

From neilb at suse.com  Thu Jun 28 01:39:38 2018
From: neilb at suse.com (NeilBrown)
Date: Thu, 28 Jun 2018 11:39:38 +1000
Subject: [lustre-devel] [PATCH 00/24] lustre - more cleanups including
 module reduction.
In-Reply-To: <44633E99-07CB-4C40-967C-0B4FE206F29A@whamcloud.com>
References: <152904663333.10587.10934053155404014785.stgit@noble>
 <alpine.LFD.2.21.1806210239190.3886@casper.infradead.org>
 <DM3PR1101MB1213DDB7FC857EE6B1444153CB760@DM3PR1101MB1213.namprd11.prod.outlook.com>
 <alpine.LFD.2.21.1806210354260.3886@casper.infradead.org>
 <3FD4D051-9C95-449D-8A23-D42B271E55B8@dilger.ca>
 <DM3PR1101MB121322E85FDE1E5310F6F4B7CB760@DM3PR1101MB1213.namprd11.prod.outlook.com>
 <87tvprah32.fsf@notabene.neil.brown.name>
 <alpine.LFD.2.21.1806260206360.31196@casper.infradead.org>
 <DM3PR1101MB121358672276198B843DE5A7CB490@DM3PR1101MB1213.namprd11.prod.outlook.com>
 <87d0wd6i4m.fsf@notabene.neil.brown.name>
 <44633E99-07CB-4C40-967C-0B4FE206F29A@whamcloud.com>
Message-ID: <87wouj665h.fsf@notabene.neil.brown.name>

On Wed, Jun 27 2018, Andreas Dilger wrote:

> On Jun 27, 2018, at 05:08, NeilBrown <neilb at suse.com> wrote:
>> 
>> On Tue, Jun 26 2018, Patrick Farrell wrote:
>> 
>>> Ah, sorry, lost your mail in the shuffle, Neil.
>>> 
>>> The name, mostly.  ptlrpc is one subdirectory and one subsystem, so I don’t want to use it for a module that explicitly includes all of several.  I’m not sure of a better name immediately - is this module intended to be shared between client and server?
>>> 
>> 
>> Thanks for clarifying.
>> Note that I'm still feeling my way around here so "intended" can at most
>> be a soft intention, open to change.  But yes, I expect this module
>> would ultimately be shared with the server.
>> 
>> I think things should only be in different modules if it might sometimes
>> make sense to use one without the other.
>> 
>> So everything that is always used for a lustre client mount, and is only
>> used for a lustre client mount, should be in a module called "lustre".
>> And everything always and only used for a lustre server should be in a
>> module called (something like) "lustre-server".
>> 
>> It appears to me that ptlrpc, ldlm, and obdclass are always and only
>> used together, but can be used for client or server.  So I think they
>> should be together in a module.
>> 
>> I'm tempted to add lnet to that list.  There has been mention that Cray
>> has some other functionality that uses lnet - if that doesn't use
>> ptlrpc, then maybe that is a case for keeping lnet separate.  However I
>> would rather see it as a potential case to separate lnet from the others
>> at a later date if that functionality from cray ever goes into mainline.
>> 
>> The klnds modules can presumably be used independently(?) so they can
>> sensibly remain as separate modules, though I'm beginning to wonder if
>> lnet can actually function without socklnd, so I wonder if that should
>> be permanently part of the lnet module (with NFS, the xprt_sock code is
>> a permanent part of "sunrpc", while the xprt_rdma (aka rpcrdma) code
>> can be a separate module).
>
> Definitely LNet can run without ksocklnd, and it does so on clients that
> only have IB connections.  Lots of HPC systems have client nodes that do
> not have Ethernet interfaces, since it is a cabling/networking nightmare
> to add a few thousand additional Ethernet cables/ports in addition to the
> IB cables/ports, and it is always possible to run IPoIB for TCP traffic.

Thanks for the clarification.
This came up because I'd been looking at lnet/lib-socket.c and wondering
why that code wasn't in socklnd.  If found that it was also used by
lnet/acceptor which seems to always run a thread that listens on a TCP
port (I think - I skim lots of code while trying to figure out the
big-picture structure).
If there is always a socket acceptor, I thought you might always need a
socket back-end (lnd).

Should the 'acceptor' thing really be part of socklnd?

>
>> In the interests of a concrete strawman, what objections would I get if
>> I suggested that ptlrpc, ldlm, obdclass, lnet, and socklnd were all included in
>> the one module named "lnet" ??
>
> This would break all of the module parameters for ptlrpc.  There are only
> a few obdclass module parameters that are rarely used, so I'm less worried
> about those ones.
>
> Given that lnet is a separate thing, I'd prefer to keep "ptlrpc" and "lnet"
> as separate modules, then "lustre" for the remainder of the client
> code.

This does seem to be the idea that will be most widely accepted.
The module parameters seem to be the main practical issue with merging
modules.  I wouldn't want that to force us to keep things separate that
should be together, but nor do I want to introduce unnecessary
breakage.

>
> The "ptlrpc" module should also include the "fld", "fid", "quota", "mgc"
> modules, since these are shared with the server as well.
>
> Server modules:
> [adilger at mookie ~]$ lsmod | grep ^obdclass
> obdclass             1731964  83 mdd,lod,mdt,osp,ofd,lfsck,ost,mgs,mgc,osd_ldiskfs,fid,fld,lquota,ptlrpc
>
> Client modules:
> [adilger at twoshoes ~]$ lsmod | grep ^obdclass
> obdclass             1568040  86 osc,mgc,lustre,lov,mdc,fid,lmv,fld,ptlrpc

Thanks for this - a useful perspective.
I don't see quota (or lquota) on the client side though ??

>
>
> James, if we are going to move in the direction of having a separate
> "lustre-server" module, it would make sense to land a patch to master
> now that adds both a module alias, as well as a filesystem type alias
> for lustre-server, so that we can start moving systems over to using
> "lustre-server" as the filesystem type in /etc/fstab or HA mounting
> scripts, and it will auto-load the module at the same time.  This might
> also allow the Lustre mount code to be simplified, since I think it has
> some messy code to have the same code, but distinguish between client
> and server mounts.

This sounds like an excellent idea!
I have a patch in my lustre-testing tree (probably will post next week)
which moves the client-side mounting from obdclass to llite.  It is a
nice simplification.

Thanks,
NeilBrown

>
> Cheers, Andreas
> ---
> Andreas Dilger
> Principal Lustre Architect
> Whamcloud
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180628/5d3ab435/attachment.sig>

From neilb at suse.com  Thu Jun 28 01:59:05 2018
From: neilb at suse.com (NeilBrown)
Date: Thu, 28 Jun 2018 11:59:05 +1000
Subject: [lustre-devel] [PATCH 00/24] lustre - more cleanups including
 module reduction.
In-Reply-To: <DM3PR1101MB12132AC3E33E7D7C8FBE4536CB480@DM3PR1101MB1213.namprd11.prod.outlook.com>
References: <152904663333.10587.10934053155404014785.stgit@noble>
 <alpine.LFD.2.21.1806210239190.3886@casper.infradead.org>
 <DM3PR1101MB1213DDB7FC857EE6B1444153CB760@DM3PR1101MB1213.namprd11.prod.outlook.com>
 <alpine.LFD.2.21.1806210354260.3886@casper.infradead.org>
 <3FD4D051-9C95-449D-8A23-D42B271E55B8@dilger.ca>
 <DM3PR1101MB121322E85FDE1E5310F6F4B7CB760@DM3PR1101MB1213.namprd11.prod.outlook.com>
 <87tvprah32.fsf@notabene.neil.brown.name>
 <alpine.LFD.2.21.1806260206360.31196@casper.infradead.org>
 <DM3PR1101MB121358672276198B843DE5A7CB490@DM3PR1101MB1213.namprd11.prod.outlook.com>
 <87d0wd6i4m.fsf@notabene.neil.brown.name>
 <44633E99-07CB-4C40-967C-0B4FE206F29A@whamcloud.com>
 <DM3PR1101MB12132AC3E33E7D7C8FBE4536CB480@DM3PR1101MB1213.namprd11.prod.outlook.com>
Message-ID: <87tvpn6592.fsf@notabene.neil.brown.name>

On Wed, Jun 27 2018, Patrick Farrell wrote:

> Neil,
>
> We do indeed have such functionality (it’s called DVS and it’s
> basically a high speed file system projection framework, ala NFS but
> faster), so the ability to build lnet separately is valuable to us.
> While it is being open sourced under the GPL, I don’t think there’s
> any intention to try to upstream it.  The current code isn’t even
> usable off of Cray systems as it depends on info from user space (that
> is provided, in the end, from Cray proprietary hardware) to keep its
> connection/routing tables up to date.  That’s supposedly in the
> pipeline to get fixed, but it’s still pretty far from generally
> usable.
>
> But we’d still really appreciate it if lnet stayed separate.  Don’t
> know if that’s enough for you - I know sometimes *small* stuff is done
> for out of tree users.  Hopefully this meets that standard.
>

Ahh - DVS.  That answers a question I just asked in another email.
My google-skills don't seem to be up to locating the source code though
:-(

While I wouldn't knowingly break an interface used by some out-of-tree
code without good reason, it is hard to avoid if you don't know what the
out-of-tree code does.  It can be very tempting to remove something that
isn't being used, but that can certainly hurt out-of-tree code
sometimes.

A particular example I'm exploring at present is the dual data paths in
LNet.  Or maybe it is dual types of Memory Descriptors.
There is 'kiov' which uses kernel-virtual addresses and 'iovec' which
uses page+offset.
The kiov option isn't used in the client code and it seems likely that
the server-side code could be converted to use iovec without problems.

I'd like to remove the kiov as I wouldn't be able to justify its
existence when submitting the client-only code upstream.  But I don't
want to remove the option of having an alternate MD type if it really is
significantly more efficient in some context.
If I know whether DVS used kiov or iovec - and in what way - that would
help me to know if I might break something, and to be able to assess the
cost.

In my mind, the "standard" that you mention is always about
practicality.   Code needs to be maintainable - easy to understand and
hard to break.  If the LNet interface is clean and well documented in
the kernel, then I don't see why we would not at least attempt to
preserve it.

Thanks,
NeilBrown
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180628/d261f326/attachment.sig>

From paf at cray.com  Thu Jun 28 02:35:46 2018
From: paf at cray.com (Patrick Farrell)
Date: Thu, 28 Jun 2018 02:35:46 +0000
Subject: [lustre-devel] [PATCH 00/24] lustre - more cleanups including
 module reduction.
In-Reply-To: <87tvpn6592.fsf@notabene.neil.brown.name>
References: <152904663333.10587.10934053155404014785.stgit@noble>
 <alpine.LFD.2.21.1806210239190.3886@casper.infradead.org>
 <DM3PR1101MB1213DDB7FC857EE6B1444153CB760@DM3PR1101MB1213.namprd11.prod.outlook.com>
 <alpine.LFD.2.21.1806210354260.3886@casper.infradead.org>
 <3FD4D051-9C95-449D-8A23-D42B271E55B8@dilger.ca>
 <DM3PR1101MB121322E85FDE1E5310F6F4B7CB760@DM3PR1101MB1213.namprd11.prod.outlook.com>
 <87tvprah32.fsf@notabene.neil.brown.name>
 <alpine.LFD.2.21.1806260206360.31196@casper.infradead.org>
 <DM3PR1101MB121358672276198B843DE5A7CB490@DM3PR1101MB1213.namprd11.prod.outlook.com>
 <87d0wd6i4m.fsf@notabene.neil.brown.name>
 <44633E99-07CB-4C40-967C-0B4FE206F29A@whamcloud.com>
 <DM3PR1101MB12132AC3E33E7D7C8FBE4536CB480@DM3PR1101MB1213.namprd11.prod.outlook.com>,
 <87tvpn6592.fsf@notabene.neil.brown.name>
Message-ID: <DM3PR1101MB12139B5B51BCE76A79DEC383CB4F0@DM3PR1101MB1213.namprd11.prod.outlook.com>


I will ask about source and plans to publish it, Neil - I’m almost certain it has not been publicly released yet and I’m not actually sure the relicensing is really complete.  So I’d be shocked and possibly mildly concerned if you found the source online today.

Understood about the rest, and thanks!  I have no idea which DVS uses, and I understand that as an out of tree user, we are in the end responsible for it.  DVS has in fact had this relationship with the master branch for a long time - Andreas and company are accommodating about interface changes, but in the end, we are responsible for keeping in sync.

I would say about the acceptor and sock lnd: I believe Lustre assumes some IP transport is available for configuration, but does NOT necessarily use it for primary communication.  Fabrics - Like infiniband or Cray Aries - are more or less always configured to provide IP transport, to enable the panoply of tools and apps that rely on it.  But they perform better if their native protocols are used, which is of course what the other LNDs do.

________________________________
From: NeilBrown <neilb at suse.com>
Sent: Wednesday, June 27, 2018 8:59:05 PM
To: Patrick Farrell; Andreas Dilger; James Simmons
Cc: Oleg Drokin; Lustre Development List
Subject: Re: [lustre-devel] [PATCH 00/24] lustre - more cleanups including module reduction.

On Wed, Jun 27 2018, Patrick Farrell wrote:

> Neil,
>
> We do indeed have such functionality (it’s called DVS and it’s
> basically a high speed file system projection framework, ala NFS but
> faster), so the ability to build lnet separately is valuable to us.
> While it is being open sourced under the GPL, I don’t think there’s
> any intention to try to upstream it.  The current code isn’t even
> usable off of Cray systems as it depends on info from user space (that
> is provided, in the end, from Cray proprietary hardware) to keep its
> connection/routing tables up to date.  That’s supposedly in the
> pipeline to get fixed, but it’s still pretty far from generally
> usable.
>
> But we’d still really appreciate it if lnet stayed separate.  Don’t
> know if that’s enough for you - I know sometimes *small* stuff is done
> for out of tree users.  Hopefully this meets that standard.
>

Ahh - DVS.  That answers a question I just asked in another email.
My google-skills don't seem to be up to locating the source code though
:-(

While I wouldn't knowingly break an interface used by some out-of-tree
code without good reason, it is hard to avoid if you don't know what the
out-of-tree code does.  It can be very tempting to remove something that
isn't being used, but that can certainly hurt out-of-tree code
sometimes.

A particular example I'm exploring at present is the dual data paths in
LNet.  Or maybe it is dual types of Memory Descriptors.
There is 'kiov' which uses kernel-virtual addresses and 'iovec' which
uses page+offset.
The kiov option isn't used in the client code and it seems likely that
the server-side code could be converted to use iovec without problems.

I'd like to remove the kiov as I wouldn't be able to justify its
existence when submitting the client-only code upstream.  But I don't
want to remove the option of having an alternate MD type if it really is
significantly more efficient in some context.
If I know whether DVS used kiov or iovec - and in what way - that would
help me to know if I might break something, and to be able to assess the
cost.

In my mind, the "standard" that you mention is always about
practicality.   Code needs to be maintainable - easy to understand and
hard to break.  If the LNet interface is clean and well documented in
the kernel, then I don't see why we would not at least attempt to
preserve it.

Thanks,
NeilBrown
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180628/6750beca/attachment.html>

From spitzcor at cray.com  Thu Jun 28 15:03:39 2018
From: spitzcor at cray.com (Cory Spitz)
Date: Thu, 28 Jun 2018 15:03:39 +0000
Subject: [lustre-devel] [PATCH 00/24] lustre - more cleanups including
 module reduction.
In-Reply-To: <87tvpn6592.fsf@notabene.neil.brown.name>
References: <152904663333.10587.10934053155404014785.stgit@noble>
 <alpine.LFD.2.21.1806210239190.3886@casper.infradead.org>
 <DM3PR1101MB1213DDB7FC857EE6B1444153CB760@DM3PR1101MB1213.namprd11.prod.outlook.com>
 <alpine.LFD.2.21.1806210354260.3886@casper.infradead.org>
 <3FD4D051-9C95-449D-8A23-D42B271E55B8@dilger.ca>
 <DM3PR1101MB121322E85FDE1E5310F6F4B7CB760@DM3PR1101MB1213.namprd11.prod.outlook.com>
 <87tvprah32.fsf@notabene.neil.brown.name>
 <alpine.LFD.2.21.1806260206360.31196@casper.infradead.org>
 <DM3PR1101MB121358672276198B843DE5A7CB490@DM3PR1101MB1213.namprd11.prod.outlook.com>
 <87d0wd6i4m.fsf@notabene.neil.brown.name>
 <44633E99-07CB-4C40-967C-0B4FE206F29A@whamcloud.com>
 <DM3PR1101MB12132AC3E33E7D7C8FBE4536CB480@DM3PR1101MB1213.namprd11.prod.outlook.com>
 <87tvpn6592.fsf@notabene.neil.brown.name>
Message-ID: <E70889E4-794F-495D-B40C-97BA4F05EBB0@cray.com>

I know that Doug follows this list closely, but I'm going to flag him here directly, because I know that he has an opinion about kiov / iovec (and a 3rd if I recall correctly).

Doug told me that (possibly) Al Viro had a proposal or submitted code to simply the kiov usage and that Doug was in favor of it.  Let's choose to make the right architectural call for Lustre/LNet and let Cray worry about keeping our out-of-tree code working.  As Patrick said, Cray pays the price of keeping DVS off to the side (we're wising up, which is why it is now GPL... but don't expect us to publish it on the Internet necessarily... baby-steps).

Doug, can you weigh-in on this case?

Thanks,
-Cory

-- 

On 6/27/18, 8:59 PM, "lustre-devel on behalf of NeilBrown" <lustre-devel-bounces at lists.lustre.org on behalf of neilb at suse.com> wrote:

    On Wed, Jun 27 2018, Patrick Farrell wrote:
    
    > Neil,
    >
    > We do indeed have such functionality (it’s called DVS and it’s
    > basically a high speed file system projection framework, ala NFS but
    > faster), so the ability to build lnet separately is valuable to us.
    > While it is being open sourced under the GPL, I don’t think there’s
    > any intention to try to upstream it.  The current code isn’t even
    > usable off of Cray systems as it depends on info from user space (that
    > is provided, in the end, from Cray proprietary hardware) to keep its
    > connection/routing tables up to date.  That’s supposedly in the
    > pipeline to get fixed, but it’s still pretty far from generally
    > usable.
    >
    > But we’d still really appreciate it if lnet stayed separate.  Don’t
    > know if that’s enough for you - I know sometimes *small* stuff is done
    > for out of tree users.  Hopefully this meets that standard.
    >
    
    Ahh - DVS.  That answers a question I just asked in another email.
    My google-skills don't seem to be up to locating the source code though
    :-(
    
    While I wouldn't knowingly break an interface used by some out-of-tree
    code without good reason, it is hard to avoid if you don't know what the
    out-of-tree code does.  It can be very tempting to remove something that
    isn't being used, but that can certainly hurt out-of-tree code
    sometimes.
    
    A particular example I'm exploring at present is the dual data paths in
    LNet.  Or maybe it is dual types of Memory Descriptors.
    There is 'kiov' which uses kernel-virtual addresses and 'iovec' which
    uses page+offset.
    The kiov option isn't used in the client code and it seems likely that
    the server-side code could be converted to use iovec without problems.
    
    I'd like to remove the kiov as I wouldn't be able to justify its
    existence when submitting the client-only code upstream.  But I don't
    want to remove the option of having an alternate MD type if it really is
    significantly more efficient in some context.
    If I know whether DVS used kiov or iovec - and in what way - that would
    help me to know if I might break something, and to be able to assess the
    cost.
    
    In my mind, the "standard" that you mention is always about
    practicality.   Code needs to be maintainable - easy to understand and
    hard to break.  If the LNet interface is clean and well documented in
    the kernel, then I don't see why we would not at least attempt to
    preserve it.
    
    Thanks,
    NeilBrown
    


From doucharek at cray.com  Thu Jun 28 17:03:55 2018
From: doucharek at cray.com (Doug Oucharek)
Date: Thu, 28 Jun 2018 17:03:55 +0000
Subject: [lustre-devel] [PATCH 00/24] lustre - more cleanups including
 module reduction.
In-Reply-To: <E70889E4-794F-495D-B40C-97BA4F05EBB0@cray.com>
References: <152904663333.10587.10934053155404014785.stgit@noble>
 <alpine.LFD.2.21.1806210239190.3886@casper.infradead.org>
 <DM3PR1101MB1213DDB7FC857EE6B1444153CB760@DM3PR1101MB1213.namprd11.prod.outlook.com>
 <alpine.LFD.2.21.1806210354260.3886@casper.infradead.org>
 <3FD4D051-9C95-449D-8A23-D42B271E55B8@dilger.ca>
 <DM3PR1101MB121322E85FDE1E5310F6F4B7CB760@DM3PR1101MB1213.namprd11.prod.outlook.com>
 <87tvprah32.fsf@notabene.neil.brown.name>
 <alpine.LFD.2.21.1806260206360.31196@casper.infradead.org>
 <DM3PR1101MB121358672276198B843DE5A7CB490@DM3PR1101MB1213.namprd11.prod.outlook.com>
 <87d0wd6i4m.fsf@notabene.neil.brown.name>
 <44633E99-07CB-4C40-967C-0B4FE206F29A@whamcloud.com>
 <DM3PR1101MB12132AC3E33E7D7C8FBE4536CB480@DM3PR1101MB1213.namprd11.prod.outlook.com>
 <87tvpn6592.fsf@notabene.neil.brown.name>
 <E70889E4-794F-495D-B40C-97BA4F05EBB0@cray.com>
Message-ID: <5993F8BE-E55C-4E75-BB1B-E7C25B79D73A@cray.com>

Yes, Al Viro had started the process of getting Lustre in the upstream staging area to use iov_iter and, I believe, biovec.  A very good change in my opinion as it gets rid of the duality.  Unfortunately, those changes have not made their way into the Whamcloud tree yet so there is a disconnect when working between the two trees.  James Simmons might know more, but I don’t believe Al’s changes are complete so there is still a need to support iovec and kvec.  Could we replace these two with biovecs?  James?

As Cory says, lets make the right architectural decisions and let companies (Whamcloud, Cray, etc) absorb the improvements.  We can adapt DVS as required.

As for LNet as a module: please keep it as a separate module.  Merging it into Lustre would create a dependency for DVS to have Lustre installed and possibly running.  This would also stand in the way of any potential future users of LNet.  Andreas pointed out one, Zest (search for LNET in this paper: https://www.psc.edu/images/zest/zest-pdsw08-paper.pdf), but there may be others developing.  LNet could become its own project at some point because of multiple users.

Doug
 
> On Jun 28, 2018, at 8:03 AM, Cory Spitz <spitzcor at cray.com> wrote:
> 
> I know that Doug follows this list closely, but I'm going to flag him here directly, because I know that he has an opinion about kiov / iovec (and a 3rd if I recall correctly).
> 
> Doug told me that (possibly) Al Viro had a proposal or submitted code to simply the kiov usage and that Doug was in favor of it.  Let's choose to make the right architectural call for Lustre/LNet and let Cray worry about keeping our out-of-tree code working.  As Patrick said, Cray pays the price of keeping DVS off to the side (we're wising up, which is why it is now GPL... but don't expect us to publish it on the Internet necessarily... baby-steps).
> 
> Doug, can you weigh-in on this case?
> 
> Thanks,
> -Cory
> 
> -- 
> 
> On 6/27/18, 8:59 PM, "lustre-devel on behalf of NeilBrown" <lustre-devel-bounces at lists.lustre.org on behalf of neilb at suse.com> wrote:
> 
>    On Wed, Jun 27 2018, Patrick Farrell wrote:
> 
>> Neil,
>> 
>> We do indeed have such functionality (it’s called DVS and it’s
>> basically a high speed file system projection framework, ala NFS but
>> faster), so the ability to build lnet separately is valuable to us.
>> While it is being open sourced under the GPL, I don’t think there’s
>> any intention to try to upstream it.  The current code isn’t even
>> usable off of Cray systems as it depends on info from user space (that
>> is provided, in the end, from Cray proprietary hardware) to keep its
>> connection/routing tables up to date.  That’s supposedly in the
>> pipeline to get fixed, but it’s still pretty far from generally
>> usable.
>> 
>> But we’d still really appreciate it if lnet stayed separate.  Don’t
>> know if that’s enough for you - I know sometimes *small* stuff is done
>> for out of tree users.  Hopefully this meets that standard.
>> 
> 
>    Ahh - DVS.  That answers a question I just asked in another email.
>    My google-skills don't seem to be up to locating the source code though
>    :-(
> 
>    While I wouldn't knowingly break an interface used by some out-of-tree
>    code without good reason, it is hard to avoid if you don't know what the
>    out-of-tree code does.  It can be very tempting to remove something that
>    isn't being used, but that can certainly hurt out-of-tree code
>    sometimes.
> 
>    A particular example I'm exploring at present is the dual data paths in
>    LNet.  Or maybe it is dual types of Memory Descriptors.
>    There is 'kiov' which uses kernel-virtual addresses and 'iovec' which
>    uses page+offset.
>    The kiov option isn't used in the client code and it seems likely that
>    the server-side code could be converted to use iovec without problems.
> 
>    I'd like to remove the kiov as I wouldn't be able to justify its
>    existence when submitting the client-only code upstream.  But I don't
>    want to remove the option of having an alternate MD type if it really is
>    significantly more efficient in some context.
>    If I know whether DVS used kiov or iovec - and in what way - that would
>    help me to know if I might break something, and to be able to assess the
>    cost.
> 
>    In my mind, the "standard" that you mention is always about
>    practicality.   Code needs to be maintainable - easy to understand and
>    hard to break.  If the LNet interface is clean and well documented in
>    the kernel, then I don't see why we would not at least attempt to
>    preserve it.
> 
>    Thanks,
>    NeilBrown
> 
> 


From adilger at whamcloud.com  Thu Jun 28 22:30:15 2018
From: adilger at whamcloud.com (Andreas Dilger)
Date: Thu, 28 Jun 2018 22:30:15 +0000
Subject: [lustre-devel] [PATCH v3 04/13] lustre: libcfs: fix
 cfs_print_to_console()
In-Reply-To: <1530128322-32535-5-git-send-email-jsimmons@infradead.org>
References: <1530128322-32535-1-git-send-email-jsimmons@infradead.org>
 <1530128322-32535-5-git-send-email-jsimmons@infradead.org>
Message-ID: <FD87D2B8-27A8-45DF-9C7C-BA33CF940C1D@whamcloud.com>

On Jun 27, 2018, at 13:38, James Simmons <jsimmons at infradead.org> wrote:
> 
> The original code for cfs_print_to_console() used printk() and
> used tricks to select which printk level to use. Later a cleanup
> patch landed the just converted printk directly to pr_info which
> is not exactly correct. Instead of converting back to printk lets
> move everything to pr_* type functions which simplify the code.
> This allows us to fold both dbghdr_to_err_string() and the
> function dbghdr_to_info_string() into cfs_print_to_console().
> 
> Signed-off-by: James Simmons <jsimmons at infradead.org>
> ---
> .../staging/lustre/lnet/libcfs/linux-tracefile.c   | 55 ++++++----------------
> 1 file changed, 14 insertions(+), 41 deletions(-)
> 
> diff --git a/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c b/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
> index 3af7722..c1747c4 100644
> --- a/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
> +++ b/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
> @@ -140,54 +140,27 @@ enum cfs_trace_buf_type cfs_trace_buf_idx_get(void)
> void cfs_print_to_console(struct ptldebug_header *hdr, int mask,
> 			  const char *buf, int len, const char *file,
> 			  const char *fn)
> {
> +	char *prefix = "Lustre";
> +
> +	if (hdr->ph_subsys == S_LND || hdr->ph_subsys == S_LNET)
> +		prefix = "LNet";
> 
> -	if (mask & D_EMERG) {
> -		prefix = dbghdr_to_err_string(hdr);
> -		ptype = KERN_EMERG;
> +	if (mask & (D_CONSOLE | libcfs_printk)) {

This check is broken.  The default value of libcfs_printk (the mask
that controls which messages should be printed to the console, and
which ones should only be logged into the internal buffer) is:

    #define D_CANTMASK (D_ERROR | D_EMERG | D_WARNING | D_CONSOLE)

so that means virtually every console message will be printed with
pr_info() because this is matched first, instead of pr_emerg() or
pr_err() below.

That is why the previous code was matching D_EMERG and D_ERROR first,
then D_WARNING, and (D_CONSOLE | libcfs_printk) at the end.

Cheers, Andreas

> +		pr_info("%s: %.*s", prefix, len, buf);
> +	} else if (mask & D_EMERG) {
> +		pr_emerg("%sError: %d:%d:(%s:%d:%s()) %.*s", prefix,
> +			 hdr->ph_pid, hdr->ph_extern_pid, file,
> +			 hdr->ph_line_num, fn, len, buf);
> 	} else if (mask & D_ERROR) {
> -		prefix = dbghdr_to_err_string(hdr);
> -		ptype = KERN_ERR;
> +		pr_err("%sError: %d:%d:(%s:%d:%s()) %.*s", prefix,
> +		       hdr->ph_pid, hdr->ph_extern_pid, file,
> +		       hdr->ph_line_num, fn, len, buf);
> 	} else if (mask & D_WARNING) {
> -		prefix = dbghdr_to_info_string(hdr);
> -		ptype = KERN_WARNING;
> -	} else if (mask & (D_CONSOLE | libcfs_printk)) {
> -		prefix = dbghdr_to_info_string(hdr);
> -		ptype = KERN_INFO;
> -	}
> -
> -	if (mask & D_CONSOLE) {
> -		pr_info("%s%s: %.*s", ptype, prefix, len, buf);
> -	} else {
> -		pr_info("%s%s: %d:%d:(%s:%d:%s()) %.*s", ptype, prefix,
> +		pr_warn("%s: %d:%d:(%s:%d:%s()) %.*s", prefix,
> 			hdr->ph_pid, hdr->ph_extern_pid, file,
> 			hdr->ph_line_num, fn, len, buf);
> 	}
> --
> 1.8.3.1
> 

Cheers, Andreas
---
Andreas Dilger
Principal Lustre Architect
Whamcloud







-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 235 bytes
Desc: Message signed with OpenPGP
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180628/b49c7062/attachment.sig>

From adilger at whamcloud.com  Thu Jun 28 22:39:00 2018
From: adilger at whamcloud.com (Andreas Dilger)
Date: Thu, 28 Jun 2018 22:39:00 +0000
Subject: [lustre-devel] [PATCH v3 02/13] lustre: libcfs: open code
 cfs_trace_max_debug_mb() into cfs_trace_set_debug_mb()
In-Reply-To: <1530128322-32535-3-git-send-email-jsimmons@infradead.org>
References: <1530128322-32535-1-git-send-email-jsimmons@infradead.org>
 <1530128322-32535-3-git-send-email-jsimmons@infradead.org>
Message-ID: <D3940542-5913-4E4D-8FEB-E3B27CB7AD04@whamcloud.com>

On Jun 27, 2018, at 13:38, James Simmons <jsimmons at infradead.org> wrote:
> 
> From: NeilBrown <neilb at suse.com>
> 
> This function in used twice.
> In libcfs_debug_init() the usage is pointless as
> the only place that set libcfs_debug_mb ensures
> that it does not exceed the maximum.  So checking
> again can never help.
> 
> So open-code the small function into the only
> other place that it is used - in cfs_trace_set_debug_mb(),
> which is used to set libcfs_debug_mb.
> 
> Signed-off-by: NeilBrown <neilb at suse.com>
> ---
> drivers/staging/lustre/lnet/libcfs/debug.c           | 6 +++---
> drivers/staging/lustre/lnet/libcfs/linux-tracefile.c | 7 -------
> drivers/staging/lustre/lnet/libcfs/tracefile.c       | 3 ++-
> drivers/staging/lustre/lnet/libcfs/tracefile.h       | 1 -
> 4 files changed, 5 insertions(+), 12 deletions(-)
> 
> diff --git a/drivers/staging/lustre/lnet/libcfs/debug.c b/drivers/staging/lustre/lnet/libcfs/debug.c
> index 06f694f..71effcf 100644
> --- a/drivers/staging/lustre/lnet/libcfs/debug.c
> +++ b/drivers/staging/lustre/lnet/libcfs/debug.c
> @@ -411,10 +411,10 @@ int libcfs_debug_init(unsigned long bufsize)
> 			sizeof(libcfs_debug_file_path_arr));
> 	}
> 
> -	/* If libcfs_debug_mb is set to an invalid value or uninitialized
> -	 * then just make the total buffers smp_num_cpus * TCD_MAX_PAGES
> +	/* If libcfs_debug_mb is uninitialized then just make the
> +	 * total buffers smp_num_cpus * TCD_MAX_PAGES
> 	 */
> -	if (max > cfs_trace_max_debug_mb() || max < num_possible_cpus()) {
> +	if (max < num_possible_cpus()) {

The libcfs_debug_mb value may be set as a module option, so that the debug
buffer can be sized before any debugging messages are logged (in case of
problems early on in module loading and such).  This code validates that
the value set via module parameter is sane.

It may be that there is a mechanism to limit the value set by module option
in newer kernels, but that wasn't the case in the past.

Cheers, Andreas

> 		max = TCD_MAX_PAGES;
> 	} else {
> 		max = max / num_possible_cpus();
> diff --git a/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c b/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
> index 9e72220..64a5bc1 100644
> --- a/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
> +++ b/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
> @@ -227,10 +227,3 @@ void cfs_print_to_console(struct ptldebug_header *hdr, int mask,
> 			hdr->ph_line_num, fn, len, buf);
> 	}
> }
> -
> -int cfs_trace_max_debug_mb(void)
> -{
> -	int  total_mb = (totalram_pages >> (20 - PAGE_SHIFT));
> -
> -	return max(512, (total_mb * 80) / 100);
> -}
> diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.c b/drivers/staging/lustre/lnet/libcfs/tracefile.c
> index 5f31933..72321ce 100644
> --- a/drivers/staging/lustre/lnet/libcfs/tracefile.c
> +++ b/drivers/staging/lustre/lnet/libcfs/tracefile.c
> @@ -933,7 +933,8 @@ int cfs_trace_set_debug_mb(int mb)
> 	int i;
> 	int j;
> 	int pages;
> -	int limit = cfs_trace_max_debug_mb();
> +	int total_mb = (totalram_pages >> (20 - PAGE_SHIFT));
> +	int limit = max(512, (total_mb * 80) / 100);
> 	struct cfs_trace_cpu_data *tcd;
> 
> 	if (mb < num_possible_cpus()) {
> diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.h b/drivers/staging/lustre/lnet/libcfs/tracefile.h
> index 9f6b73d..bd1a1ef 100644
> --- a/drivers/staging/lustre/lnet/libcfs/tracefile.h
> +++ b/drivers/staging/lustre/lnet/libcfs/tracefile.h
> @@ -88,7 +88,6 @@ int cfs_trace_copyout_string(char __user *usr_buffer, int usr_buffer_nob,
> void libcfs_register_panic_notifier(void);
> void libcfs_unregister_panic_notifier(void);
> extern int libcfs_panic_in_progress;
> -int cfs_trace_max_debug_mb(void);
> 
> #define TCD_MAX_PAGES (5 << (20 - PAGE_SHIFT))
> #define TCD_STOCK_PAGES (TCD_MAX_PAGES)
> --
> 1.8.3.1
> 

Cheers, Andreas
---
Andreas Dilger
Principal Lustre Architect
Whamcloud







-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 235 bytes
Desc: Message signed with OpenPGP
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180628/70844d26/attachment.sig>

From adilger at whamcloud.com  Thu Jun 28 23:12:42 2018
From: adilger at whamcloud.com (Andreas Dilger)
Date: Thu, 28 Jun 2018 23:12:42 +0000
Subject: [lustre-devel] [PATCH 00/24] lustre - more cleanups including
 module reduction.
In-Reply-To: <DM3PR1101MB12139B5B51BCE76A79DEC383CB4F0@DM3PR1101MB1213.namprd11.prod.outlook.com>
References: <152904663333.10587.10934053155404014785.stgit@noble>
 <alpine.LFD.2.21.1806210239190.3886@casper.infradead.org>
 <DM3PR1101MB1213DDB7FC857EE6B1444153CB760@DM3PR1101MB1213.namprd11.prod.outlook.com>
 <alpine.LFD.2.21.1806210354260.3886@casper.infradead.org>
 <3FD4D051-9C95-449D-8A23-D42B271E55B8@dilger.ca>
 <DM3PR1101MB121322E85FDE1E5310F6F4B7CB760@DM3PR1101MB1213.namprd11.prod.outlook.com>
 <87tvprah32.fsf@notabene.neil.brown.name>
 <alpine.LFD.2.21.1806260206360.31196@casper.infradead.org>
 <DM3PR1101MB121358672276198B843DE5A7CB490@DM3PR1101MB1213.namprd11.prod.outlook.com>
 <87d0wd6i4m.fsf@notabene.neil.brown.name>
 <44633E99-07CB-4C40-967C-0B4FE206F29A@whamcloud.com>
 <DM3PR1101MB12132AC3E33E7D7C8FBE4536CB480@DM3PR1101MB1213.namprd11.prod.outlook.com>
 <87tvpn6592.fsf@notabene.neil.brown.name>
 <DM3PR1101MB12139B5B51BCE76A79DEC383CB4F0@DM3PR1101MB1213.namprd11.prod.outlook.com>
Message-ID: <D6CA5D91-2344-446F-9621-935A52E47A24@whamcloud.com>

On Jun 27, 2018, at 20:35, Patrick Farrell <paf at cray.com> wrote:
> 
> I would say about the acceptor and sock lnd: I believe Lustre assumes some IP transport is available for configuration, but does NOT necessarily use it for primary communication.  Fabrics - Like infiniband or Cray Aries - are more or less always configured to provide IP transport, to enable the panoply of tools and apps that rely on it.  But they perform better if their native protocols are used, which is of course what the other LNDs do.

It is worthwhile to clarify this a bit - LNet uses IP for *addressing* of the nodes at connection time, but I don't think it even uses the TCP interface for any communication itself (though I could be mistaken, as LNet isn't my specialty).  After the initial connection, the only other thing the IP addresses are used for is printing in error messages.

At one time or another we've discussed how we might get rid of the need for IPoIB on client nodes, since some sites don't want any IP connectivity to the client nodes for security and performance reasons.  That said, I don't think we've come up with a good solution yet.  LNet itself allows alternate addressing schemes to be used.  The former qswlnd (for Quadrics Elan networks) just used an integer node number, like 1 at elan for the NID, but I don't think there is any such alternative for IB node addresses except using a MAC hardware address or similar.

Cheers, Andreas

> 
> Neil Brown <neilb at suse.com> wrote:
>> 
>> On Wed, Jun 27 2018, Patrick Farrell wrote:
>> 
>> > Neil,
>> >
>> > We do indeed have such functionality (it’s called DVS and it’s
>> > basically a high speed file system projection framework, ala NFS but
>> > faster), so the ability to build lnet separately is valuable to us.
>> > While it is being open sourced under the GPL, I don’t think there’s
>> > any intention to try to upstream it.  The current code isn’t even
>> > usable off of Cray systems as it depends on info from user space (that
>> > is provided, in the end, from Cray proprietary hardware) to keep its
>> > connection/routing tables up to date.  That’s supposedly in the
>> > pipeline to get fixed, but it’s still pretty far from generally
>> > usable.
>> >
>> > But we’d still really appreciate it if lnet stayed separate.  Don’t
>> > know if that’s enough for you - I know sometimes *small* stuff is done
>> > for out of tree users.  Hopefully this meets that standard.
>> >
>> 
>> Ahh - DVS.  That answers a question I just asked in another email.
>> My google-skills don't seem to be up to locating the source code though
>> :-(
>> 
>> While I wouldn't knowingly break an interface used by some out-of-tree
>> code without good reason, it is hard to avoid if you don't know what the
>> out-of-tree code does.  It can be very tempting to remove something that
>> isn't being used, but that can certainly hurt out-of-tree code
>> sometimes.
>> 
>> A particular example I'm exploring at present is the dual data paths in
>> LNet.  Or maybe it is dual types of Memory Descriptors.
>> There is 'kiov' which uses kernel-virtual addresses and 'iovec' which
>> uses page+offset.
>> The kiov option isn't used in the client code and it seems likely that
>> the server-side code could be converted to use iovec without problems.
>> 
>> I'd like to remove the kiov as I wouldn't be able to justify its
>> existence when submitting the client-only code upstream.  But I don't
>> want to remove the option of having an alternate MD type if it really is
>> significantly more efficient in some context.
>> If I know whether DVS used kiov or iovec - and in what way - that would
>> help me to know if I might break something, and to be able to assess the
>> cost.
>> 
>> In my mind, the "standard" that you mention is always about
>> practicality.   Code needs to be maintainable - easy to understand and
>> hard to break.  If the LNet interface is clean and well documented in
>> the kernel, then I don't see why we would not at least attempt to
>> preserve it.
>> 
>> Thanks,
>> NeilBrown

Cheers, Andreas
---
Andreas Dilger
Principal Lustre Architect
Whamcloud







-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 235 bytes
Desc: Message signed with OpenPGP
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180628/9e2591e1/attachment.sig>

From neilb at suse.com  Fri Jun 29 03:55:47 2018
From: neilb at suse.com (NeilBrown)
Date: Fri, 29 Jun 2018 13:55:47 +1000
Subject: [lustre-devel] [PATCH v3 02/13] lustre: libcfs: open code
 cfs_trace_max_debug_mb() into cfs_trace_set_debug_mb()
In-Reply-To: <D3940542-5913-4E4D-8FEB-E3B27CB7AD04@whamcloud.com>
References: <1530128322-32535-1-git-send-email-jsimmons@infradead.org>
 <1530128322-32535-3-git-send-email-jsimmons@infradead.org>
 <D3940542-5913-4E4D-8FEB-E3B27CB7AD04@whamcloud.com>
Message-ID: <8760225jr0.fsf@notabene.neil.brown.name>

On Thu, Jun 28 2018, Andreas Dilger wrote:

> On Jun 27, 2018, at 13:38, James Simmons <jsimmons at infradead.org> wrote:
>> 
>> From: NeilBrown <neilb at suse.com>
>> 
>> This function in used twice.
>> In libcfs_debug_init() the usage is pointless as
>> the only place that set libcfs_debug_mb ensures
>> that it does not exceed the maximum.  So checking
>> again can never help.
>> 
>> So open-code the small function into the only
>> other place that it is used - in cfs_trace_set_debug_mb(),
>> which is used to set libcfs_debug_mb.
>> 
>> Signed-off-by: NeilBrown <neilb at suse.com>
>> ---
>> drivers/staging/lustre/lnet/libcfs/debug.c           | 6 +++---
>> drivers/staging/lustre/lnet/libcfs/linux-tracefile.c | 7 -------
>> drivers/staging/lustre/lnet/libcfs/tracefile.c       | 3 ++-
>> drivers/staging/lustre/lnet/libcfs/tracefile.h       | 1 -
>> 4 files changed, 5 insertions(+), 12 deletions(-)
>> 
>> diff --git a/drivers/staging/lustre/lnet/libcfs/debug.c b/drivers/staging/lustre/lnet/libcfs/debug.c
>> index 06f694f..71effcf 100644
>> --- a/drivers/staging/lustre/lnet/libcfs/debug.c
>> +++ b/drivers/staging/lustre/lnet/libcfs/debug.c
>> @@ -411,10 +411,10 @@ int libcfs_debug_init(unsigned long bufsize)
>> 			sizeof(libcfs_debug_file_path_arr));
>> 	}
>> 
>> -	/* If libcfs_debug_mb is set to an invalid value or uninitialized
>> -	 * then just make the total buffers smp_num_cpus * TCD_MAX_PAGES
>> +	/* If libcfs_debug_mb is uninitialized then just make the
>> +	 * total buffers smp_num_cpus * TCD_MAX_PAGES
>> 	 */
>> -	if (max > cfs_trace_max_debug_mb() || max < num_possible_cpus()) {
>> +	if (max < num_possible_cpus()) {
>
> The libcfs_debug_mb value may be set as a module option, so that the debug
> buffer can be sized before any debugging messages are logged (in case of
> problems early on in module loading and such).  This code validates that
> the value set via module parameter is sane.
>
> It may be that there is a mechanism to limit the value set by module option
> in newer kernels, but that wasn't the case in the past.

There is such a mechanism now.
When the libcfs_debug_mb module parameter is detected,
libcfs_param_debug_mb_set() is called to parse it.
This function uses kstrtouint() to convert the string
to unsigned int, and then...... oh, that's odd.
If the current value of libcfs_debug_mb is zero, then
then the number is used without further checking.  I hadn't noticed
that.
If that current value is non-zero, then the parsed number
is passed to cfs_trace_set_debug_mb().  I had assumed that
always happens, and that function imposes the required limit.

I'll fix that up - make sure it always imposes the
appropriate limit.

Thanks for the review!

NeilBrown


>
> Cheers, Andreas
>
>> 		max = TCD_MAX_PAGES;
>> 	} else {
>> 		max = max / num_possible_cpus();
>> diff --git a/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c b/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
>> index 9e72220..64a5bc1 100644
>> --- a/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
>> +++ b/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
>> @@ -227,10 +227,3 @@ void cfs_print_to_console(struct ptldebug_header *hdr, int mask,
>> 			hdr->ph_line_num, fn, len, buf);
>> 	}
>> }
>> -
>> -int cfs_trace_max_debug_mb(void)
>> -{
>> -	int  total_mb = (totalram_pages >> (20 - PAGE_SHIFT));
>> -
>> -	return max(512, (total_mb * 80) / 100);
>> -}
>> diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.c b/drivers/staging/lustre/lnet/libcfs/tracefile.c
>> index 5f31933..72321ce 100644
>> --- a/drivers/staging/lustre/lnet/libcfs/tracefile.c
>> +++ b/drivers/staging/lustre/lnet/libcfs/tracefile.c
>> @@ -933,7 +933,8 @@ int cfs_trace_set_debug_mb(int mb)
>> 	int i;
>> 	int j;
>> 	int pages;
>> -	int limit = cfs_trace_max_debug_mb();
>> +	int total_mb = (totalram_pages >> (20 - PAGE_SHIFT));
>> +	int limit = max(512, (total_mb * 80) / 100);
>> 	struct cfs_trace_cpu_data *tcd;
>> 
>> 	if (mb < num_possible_cpus()) {
>> diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.h b/drivers/staging/lustre/lnet/libcfs/tracefile.h
>> index 9f6b73d..bd1a1ef 100644
>> --- a/drivers/staging/lustre/lnet/libcfs/tracefile.h
>> +++ b/drivers/staging/lustre/lnet/libcfs/tracefile.h
>> @@ -88,7 +88,6 @@ int cfs_trace_copyout_string(char __user *usr_buffer, int usr_buffer_nob,
>> void libcfs_register_panic_notifier(void);
>> void libcfs_unregister_panic_notifier(void);
>> extern int libcfs_panic_in_progress;
>> -int cfs_trace_max_debug_mb(void);
>> 
>> #define TCD_MAX_PAGES (5 << (20 - PAGE_SHIFT))
>> #define TCD_STOCK_PAGES (TCD_MAX_PAGES)
>> --
>> 1.8.3.1
>> 
>
> Cheers, Andreas
> ---
> Andreas Dilger
> Principal Lustre Architect
> Whamcloud
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180629/dad555dd/attachment.sig>

From jsimmons at infradead.org  Fri Jun 29 16:17:31 2018
From: jsimmons at infradead.org (James Simmons)
Date: Fri, 29 Jun 2018 17:17:31 +0100 (BST)
Subject: [lustre-devel] [PATCH v3 04/13] lustre: libcfs: fix
 cfs_print_to_console()
In-Reply-To: <FD87D2B8-27A8-45DF-9C7C-BA33CF940C1D@whamcloud.com>
References: <1530128322-32535-1-git-send-email-jsimmons@infradead.org>
 <1530128322-32535-5-git-send-email-jsimmons@infradead.org>
 <FD87D2B8-27A8-45DF-9C7C-BA33CF940C1D@whamcloud.com>
Message-ID: <alpine.LFD.2.21.1806291716050.26339@casper.infradead.org>


> > The original code for cfs_print_to_console() used printk() and
> > used tricks to select which printk level to use. Later a cleanup
> > patch landed the just converted printk directly to pr_info which
> > is not exactly correct. Instead of converting back to printk lets
> > move everything to pr_* type functions which simplify the code.
> > This allows us to fold both dbghdr_to_err_string() and the
> > function dbghdr_to_info_string() into cfs_print_to_console().
> > 
> > Signed-off-by: James Simmons <jsimmons at infradead.org>
> > ---
> > .../staging/lustre/lnet/libcfs/linux-tracefile.c   | 55 ++++++----------------
> > 1 file changed, 14 insertions(+), 41 deletions(-)
> > 
> > diff --git a/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c b/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
> > index 3af7722..c1747c4 100644
> > --- a/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
> > +++ b/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
> > @@ -140,54 +140,27 @@ enum cfs_trace_buf_type cfs_trace_buf_idx_get(void)
> > void cfs_print_to_console(struct ptldebug_header *hdr, int mask,
> > 			  const char *buf, int len, const char *file,
> > 			  const char *fn)
> > {
> > +	char *prefix = "Lustre";
> > +
> > +	if (hdr->ph_subsys == S_LND || hdr->ph_subsys == S_LNET)
> > +		prefix = "LNet";
> > 
> > -	if (mask & D_EMERG) {
> > -		prefix = dbghdr_to_err_string(hdr);
> > -		ptype = KERN_EMERG;
> > +	if (mask & (D_CONSOLE | libcfs_printk)) {
> 
> This check is broken.  The default value of libcfs_printk (the mask
> that controls which messages should be printed to the console, and
> which ones should only be logged into the internal buffer) is:
> 
>     #define D_CANTMASK (D_ERROR | D_EMERG | D_WARNING | D_CONSOLE)
> 
> so that means virtually every console message will be printed with
> pr_info() because this is matched first, instead of pr_emerg() or
> pr_err() below.
> 
> That is why the previous code was matching D_EMERG and D_ERROR first,
> then D_WARNING, and (D_CONSOLE | libcfs_printk) at the end.

So to do this right we need:

static void cfs_print_to_console(struct ptldebug_header *hdr, int mask,
                                 const char *buf, int len, const char 
*file,
                                 const char *fn)
{
        char *prefix = "Lustre";

        if (hdr->ph_subsys == S_LND || hdr->ph_subsys == S_LNET)
                prefix = "LNet";

        if (mask & D_CONSOLE) {
                if (mask & D_EMERG) {
                        pr_emerg("%sError: %.*s", prefix, len, buf);
                } else if (mask & D_ERROR) {
                        pr_err("%sError: %.*s", prefix, len, buf);
                } else if (mask & D_WARNING) {
                        pr_warn("%s: %.*s", prefix, len, buf);
                } else if (mask & libcfs_printk) {
                        pr_info("%s: %.*s", prefix, len, buf);
                }
        } else {
                if (mask & D_EMERG) {
                        pr_emerg("%sError: %d:%d:(%s:%d:%s()) %.*s", prefix,
                                 hdr->ph_pid, hdr->ph_extern_pid, file,
                                 hdr->ph_line_num, fn, len, buf);
                } else if (mask & D_ERROR) {
                        pr_err("%sError: %d:%d:(%s:%d:%s()) %.*s", prefix,
                               hdr->ph_pid, hdr->ph_extern_pid, file,
                               hdr->ph_line_num, fn, len, buf);
                } else if (mask & D_WARNING) {
                        pr_warn("%s: %d:%d:(%s:%d:%s()) %.*s", prefix,
                                hdr->ph_pid, hdr->ph_extern_pid, file,
                                hdr->ph_line_num, fn, len, buf);
                } else if (mask & libcfs_printk) {
                        pr_info("%s: %.*s", prefix, len, buf);
                }
        }
}

From doucharek at cray.com  Fri Jun 29 17:19:41 2018
From: doucharek at cray.com (Doug Oucharek)
Date: Fri, 29 Jun 2018 17:19:41 +0000
Subject: [lustre-devel] [PATCH v3 07/26] staging: lustre: libcfs: NUMA
 support
In-Reply-To: <8736x77lrk.fsf@notabene.neil.brown.name>
References: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
 <1529875250-11531-8-git-send-email-jsimmons@infradead.org>
 <877emnaed8.fsf@notabene.neil.brown.name>
 <A764DED1-A9A8-4FA5-A0F1-922C3E2D822A@cray.com>
 <87lgb16j8q.fsf@notabene.neil.brown.name>
 <DM3PR1101MB1213FD1001256E565FF011FFCB480@DM3PR1101MB1213.namprd11.prod.outlook.com>
 <8736x77lrk.fsf@notabene.neil.brown.name>
Message-ID: <9DE389AB-C7E1-4336-B7E8-604581EFD53E@cray.com>

I’ll leave Olaf of HPE answer questions about the distance code.  I was only an inspector as it relates to the Multi-Rail feature in the community tree.  

Doug

> On Jun 27, 2018, at 6:17 PM, NeilBrown <neilb at suse.com> wrote:
> 
> 
> I went digging and found that Linux already has a well defined concept
> of distance between NUMA nodes.
> On x86 (and amd64?), this is loaded from ACPI.  Other platforms can
> describe it in devicetree.
> You can view distance information in
>  /sys/devices/system/node/node*/distance
> 
> or using "numactl --hardware".
> 
> Why doesn't lustre simple extract and use this information?  Why does
> lustre need to allow it to be configured?
> 
> Thanks,
> NeilBrown
> 
> On Wed, Jun 27 2018, Patrick Farrell wrote:
> 
>> Neil,
>> 
>> I am not the person at Cray for this, but if SUSE does take an interest in this, Cray would probably be interested in weighing in and contributing info if not actually code.  In fact, other HPC vendors like HPE(by which I mostly mean the old SGI) or IBM might as well.  NUMA optimization is a persistent fascination in our area of the industry...
>> 
>> - Patrick
>> 
>> ________________________________
>> From: lustre-devel <lustre-devel-bounces at lists.lustre.org> on behalf of NeilBrown <neilb at suse.com>
>> Sent: Tuesday, June 26, 2018 9:44:37 PM
>> To: Doug Oucharek
>> Cc: Amir Shehata; Lustre Development List
>> Subject: Re: [lustre-devel] [PATCH v3 07/26] staging: lustre: libcfs: NUMA support
>> 
>> On Mon, Jun 25 2018, Doug Oucharek wrote:
>> 
>>> Some background on this NUMA change:
>>> 
>>> First off, this is just a first step to a bigger set of changes which include changes to the Lustre utilities.  This was done as part of the Multi-Rail feature.  One of the systems that feature is meant to support is the SGI UV system (now HPE) which has a massive number of NUMA nodes connected by a NUMA Link.  There are multiple fabric cards spread throughout the system and Multi-Rail needs to know which fabric cards are nearest to the NUMA node we are running on.  To do that, the “distance” between NUMA nodes needs to be configured.
>>> 
>>> This patch is preparing the infrastructure for the Multi-Rail feature to support configuring NUMA node distances.  Technically, this patch should be landing with the Multi-Rail feature (still to be pushed) for it to make proper sense.
>>> 
>> 
>> Thanks a lot for the background.
>> 
>> If these NUMA nodes have a 'distance' between them, and if lustre can
>> benefit from knowing the distance, then is seems likely that other code
>> might also benefit.  In that case it would be best if the distance were
>> encoded in some global state information so that lustre and any other
>> subsystem can extract it.
>> 
>> Do you know if there is any work underway by anyone to make this
>> information generally available?  If there is, we should make sure that
>> lustre works in a compatible way so that once that work lands, lustre
>> can use it directly and not need extra configuration.
>> If no such work is underway, then it would be really good if something
>> were done in that direction.  If no-one here is able to work on this, I
>> can ask around in SUSE and see if anyone here knows anything relevant.
>> 
>> Thanks,
>> NeilBrown


From amir.shehata.whamcloud at gmail.com  Fri Jun 29 17:27:46 2018
From: amir.shehata.whamcloud at gmail.com (Amir Shehata)
Date: Fri, 29 Jun 2018 10:27:46 -0700
Subject: [lustre-devel] [PATCH v3 07/26] staging: lustre: libcfs: NUMA
 support
In-Reply-To: <9DE389AB-C7E1-4336-B7E8-604581EFD53E@cray.com>
References: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
 <1529875250-11531-8-git-send-email-jsimmons@infradead.org>
 <877emnaed8.fsf@notabene.neil.brown.name>
 <A764DED1-A9A8-4FA5-A0F1-922C3E2D822A@cray.com>
 <87lgb16j8q.fsf@notabene.neil.brown.name>
 <DM3PR1101MB1213FD1001256E565FF011FFCB480@DM3PR1101MB1213.namprd11.prod.outlook.com>
 <8736x77lrk.fsf@notabene.neil.brown.name>
 <9DE389AB-C7E1-4336-B7E8-604581EFD53E@cray.com>
Message-ID: <CAAqp6i0uwmpdFYUJ7mCBz+MH+gVBa6S=P2t5Yn7wcFOtrTD-Fg@mail.gmail.com>

Olaf can add more details, but I believe we are using the linux distance
infrastructure. Take a look at cfs_cpt_distance_calculate(). What we're
doing is extracting the NUMA distances provided in the kernel and building
an internal representation of distances between CPU partitions (CPTs) since
that's what's used in the code.

On 29 June 2018 at 10:19, Doug Oucharek <doucharek at cray.com> wrote:

> I’ll leave Olaf of HPE answer questions about the distance code.  I was
> only an inspector as it relates to the Multi-Rail feature in the community
> tree.
>
> Doug
>
> > On Jun 27, 2018, at 6:17 PM, NeilBrown <neilb at suse.com> wrote:
> >
> >
> > I went digging and found that Linux already has a well defined concept
> > of distance between NUMA nodes.
> > On x86 (and amd64?), this is loaded from ACPI.  Other platforms can
> > describe it in devicetree.
> > You can view distance information in
> >  /sys/devices/system/node/node*/distance
> >
> > or using "numactl --hardware".
> >
> > Why doesn't lustre simple extract and use this information?  Why does
> > lustre need to allow it to be configured?
> >
> > Thanks,
> > NeilBrown
> >
> > On Wed, Jun 27 2018, Patrick Farrell wrote:
> >
> >> Neil,
> >>
> >> I am not the person at Cray for this, but if SUSE does take an interest
> in this, Cray would probably be interested in weighing in and contributing
> info if not actually code.  In fact, other HPC vendors like HPE(by which I
> mostly mean the old SGI) or IBM might as well.  NUMA optimization is a
> persistent fascination in our area of the industry...
> >>
> >> - Patrick
> >>
> >> ________________________________
> >> From: lustre-devel <lustre-devel-bounces at lists.lustre.org> on behalf
> of NeilBrown <neilb at suse.com>
> >> Sent: Tuesday, June 26, 2018 9:44:37 PM
> >> To: Doug Oucharek
> >> Cc: Amir Shehata; Lustre Development List
> >> Subject: Re: [lustre-devel] [PATCH v3 07/26] staging: lustre: libcfs:
> NUMA support
> >>
> >> On Mon, Jun 25 2018, Doug Oucharek wrote:
> >>
> >>> Some background on this NUMA change:
> >>>
> >>> First off, this is just a first step to a bigger set of changes which
> include changes to the Lustre utilities.  This was done as part of the
> Multi-Rail feature.  One of the systems that feature is meant to support is
> the SGI UV system (now HPE) which has a massive number of NUMA nodes
> connected by a NUMA Link.  There are multiple fabric cards spread
> throughout the system and Multi-Rail needs to know which fabric cards are
> nearest to the NUMA node we are running on.  To do that, the “distance”
> between NUMA nodes needs to be configured.
> >>>
> >>> This patch is preparing the infrastructure for the Multi-Rail feature
> to support configuring NUMA node distances.  Technically, this patch should
> be landing with the Multi-Rail feature (still to be pushed) for it to make
> proper sense.
> >>>
> >>
> >> Thanks a lot for the background.
> >>
> >> If these NUMA nodes have a 'distance' between them, and if lustre can
> >> benefit from knowing the distance, then is seems likely that other code
> >> might also benefit.  In that case it would be best if the distance were
> >> encoded in some global state information so that lustre and any other
> >> subsystem can extract it.
> >>
> >> Do you know if there is any work underway by anyone to make this
> >> information generally available?  If there is, we should make sure that
> >> lustre works in a compatible way so that once that work lands, lustre
> >> can use it directly and not need extra configuration.
> >> If no such work is underway, then it would be really good if something
> >> were done in that direction.  If no-one here is able to work on this, I
> >> can ask around in SUSE and see if anyone here knows anything relevant.
> >>
> >> Thanks,
> >> NeilBrown
>
> _______________________________________________
> lustre-devel mailing list
> lustre-devel at lists.lustre.org
> http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180629/b8f5825d/attachment.html>

From olaf.weber at hpe.com  Fri Jun 29 17:47:27 2018
From: olaf.weber at hpe.com (Weber, Olaf (HPC Data Management & Storage))
Date: Fri, 29 Jun 2018 17:47:27 +0000
Subject: [lustre-devel] [PATCH v3 07/26] staging: lustre: libcfs: NUMA
 support
In-Reply-To: <CAAqp6i0uwmpdFYUJ7mCBz+MH+gVBa6S=P2t5Yn7wcFOtrTD-Fg@mail.gmail.com>
References: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
 <1529875250-11531-8-git-send-email-jsimmons@infradead.org>
 <877emnaed8.fsf@notabene.neil.brown.name>
 <A764DED1-A9A8-4FA5-A0F1-922C3E2D822A@cray.com>
 <87lgb16j8q.fsf@notabene.neil.brown.name>
 <DM3PR1101MB1213FD1001256E565FF011FFCB480@DM3PR1101MB1213.namprd11.prod.outlook.com>
 <8736x77lrk.fsf@notabene.neil.brown.name>
 <9DE389AB-C7E1-4336-B7E8-604581EFD53E@cray.com>
 <CAAqp6i0uwmpdFYUJ7mCBz+MH+gVBa6S=P2t5Yn7wcFOtrTD-Fg@mail.gmail.com>
Message-ID: <AT5PR8401MB0900C67FADDF0EF7ED1CDBC4854E0@AT5PR8401MB0900.NAMPRD84.PROD.OUTLOOK.COM>

To add to Amir's point,  Lustre's CPTs are a way to partition a machine. The distance mechanism I added is one way to map the ACPI-reported distances on the Lustre CPT mapping. It tends to assume the worst case applies to the wholes. It is there because the rest of the Lustre code (at least in the tree I had to work on) "thinks" in CPTs.

Other CPT-related stuff that came in with the multi-rail code has the same rationale. If I'd been working against the kernel interfaces themselves it would have looked differently, but that was not an option at the time.

We've found it to be useful, so replacing it would be better than just ripping it out.

That's all there is to it.

Olaf

---
From: Amir Shehata [mailto:amir.shehata.whamcloud at gmail.com] 
Sent: Friday, June 29, 2018 19:28
To: Doug Oucharek <doucharek at cray.com>
Cc: NeilBrown <neilb at suse.com>; Weber, Olaf (HPC Data Management & Storage) <olaf.weber at hpe.com>; Amir Shehata <amir.shehata at intel.com>; Lustre Development List <lustre-devel at lists.lustre.org>
Subject: Re: [lustre-devel] [PATCH v3 07/26] staging: lustre: libcfs: NUMA support

Olaf can add more details, but I believe we are using the linux distance infrastructure. Take a look at cfs_cpt_distance_calculate(). What we're doing is extracting the NUMA distances provided in the kernel and building an internal representation of distances between CPU partitions (CPTs) since that's what's used in the code.

On 29 June 2018 at 10:19, Doug Oucharek <doucharek at cray.com> wrote:
I’ll leave Olaf of HPE answer questions about the distance code.  I was only an inspector as it relates to the Multi-Rail feature in the community tree.  

Doug

> On Jun 27, 2018, at 6:17 PM, NeilBrown <neilb at suse.com> wrote:
> 
> 
> I went digging and found that Linux already has a well defined concept
> of distance between NUMA nodes.
> On x86 (and amd64?), this is loaded from ACPI.  Other platforms can
> describe it in devicetree.
> You can view distance information in
>  /sys/devices/system/node/node*/distance
> 
> or using "numactl --hardware".
> 
> Why doesn't lustre simple extract and use this information?  Why does
> lustre need to allow it to be configured?
> 
> Thanks,
> NeilBrown
> 
> On Wed, Jun 27 2018, Patrick Farrell wrote:
> 
>> Neil,
>> 
>> I am not the person at Cray for this, but if SUSE does take an interest in this, Cray would probably be interested in weighing in and contributing info if not actually code.  In fact, other HPC vendors like HPE(by which I mostly mean the old SGI) or IBM might as well.  NUMA optimization is a persistent fascination in our area of the industry...
>> 
>> - Patrick
>> 
>> ________________________________
>> From: lustre-devel <lustre-devel-bounces at lists.lustre.org> on behalf of NeilBrown <neilb at suse.com>
>> Sent: Tuesday, June 26, 2018 9:44:37 PM
>> To: Doug Oucharek
>> Cc: Amir Shehata; Lustre Development List
>> Subject: Re: [lustre-devel] [PATCH v3 07/26] staging: lustre: libcfs: NUMA support
>> 
>> On Mon, Jun 25 2018, Doug Oucharek wrote:
>> 
>>> Some background on this NUMA change:
>>> 
>>> First off, this is just a first step to a bigger set of changes which include changes to the Lustre utilities.  This was done as part of the Multi-Rail feature.  One of the systems that feature is meant to support is the SGI UV system (now HPE) which has a massive number of NUMA nodes connected by a NUMA Link.  There are multiple fabric cards spread throughout the system and Multi-Rail needs to know which fabric cards are nearest to the NUMA node we are running on.  To do that, the “distance” between NUMA nodes needs to be configured.
>>> 
>>> This patch is preparing the infrastructure for the Multi-Rail feature to support configuring NUMA node distances.  Technically, this patch should be landing with the Multi-Rail feature (still to be pushed) for it to make proper sense.
>>> 
>> 
>> Thanks a lot for the background.
>> 
>> If these NUMA nodes have a 'distance' between them, and if lustre can
>> benefit from knowing the distance, then is seems likely that other code
>> might also benefit.  In that case it would be best if the distance were
>> encoded in some global state information so that lustre and any other
>> subsystem can extract it.
>> 
>> Do you know if there is any work underway by anyone to make this
>> information generally available?  If there is, we should make sure that
>> lustre works in a compatible way so that once that work lands, lustre
>> can use it directly and not need extra configuration.
>> If no such work is underway, then it would be really good if something
>> were done in that direction.  If no-one here is able to work on this, I
>> can ask around in SUSE and see if anyone here knows anything relevant.
>> 
>> Thanks,
>> NeilBrown
_______________________________________________
lustre-devel mailing list
lustre-devel at lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org