From adilger at whamcloud.com  Mon Jul  2 01:47:46 2018
From: adilger at whamcloud.com (Andreas Dilger)
Date: Mon, 2 Jul 2018 01:47:46 +0000
Subject: [lustre-devel] Autotest is now up and running
Message-ID: <35FBB859-38C2-4255-8CC5-4AB5C376FF16@whamcloud.com>

The autotest systems have been moved and re-enabled and are now
running tests on patches in Gerrit. 

Patches that were submitted while autotest was in transit should start
testing automatically, but if not they can be rebased or be resubmitted
to the builders. 

Thanks go to Minh, Charlie, and Lee for working hard to get the systems
up and running. 

Cheers, Andreas

From neilb at suse.com  Mon Jul  2 02:17:48 2018
From: neilb at suse.com (NeilBrown)
Date: Mon, 02 Jul 2018 12:17:48 +1000
Subject: [lustre-devel] [PATCH v3 02/13] lustre: libcfs: open code
 cfs_trace_max_debug_mb() into cfs_trace_set_debug_mb()
In-Reply-To: <8760225jr0.fsf@notabene.neil.brown.name>
References: <1530128322-32535-1-git-send-email-jsimmons@infradead.org>
 <1530128322-32535-3-git-send-email-jsimmons@infradead.org>
 <D3940542-5913-4E4D-8FEB-E3B27CB7AD04@whamcloud.com>
 <8760225jr0.fsf@notabene.neil.brown.name>
Message-ID: <87k1qe2xf7.fsf@notabene.neil.brown.name>

On Fri, Jun 29 2018, NeilBrown wrote:
>> It may be that there is a mechanism to limit the value set by module option
>> in newer kernels, but that wasn't the case in the past.
>
> There is such a mechanism now.
> When the libcfs_debug_mb module parameter is detected,
> libcfs_param_debug_mb_set() is called to parse it.
> This function uses kstrtouint() to convert the string
> to unsigned int, and then...... oh, that's odd.
> If the current value of libcfs_debug_mb is zero, then
> then the number is used without further checking.  I hadn't noticed
> that.
> If that current value is non-zero, then the parsed number
> is passed to cfs_trace_set_debug_mb().  I had assumed that
> always happens, and that function imposes the required limit.
>
> I'll fix that up - make sure it always imposes the
> appropriate limit.

I've inserted the following patch before this one.
That makes it correct :-)

Thanks,
NeilBrown

From: NeilBrown <neilb at suse.com>
Date: Mon, 2 Jul 2018 12:14:15 +1000
Subject: [PATCH] lustre: always range-check libcfs_debug_mb setting.

When the libcfs_debug_mb module parameter is set
at module-load time it isn't range-checked.  When
it is set via sysfs it is.  This inconsistency
makes the code harder to follow.

It is quite safe to call cfs_trace_set_debug_mb()
and cfs_trace_get_debug_mb() before the module
is initialized as cfs_tcd_for_each() does nothing
before initializtion.

So change cfs_trace_set_debug_mb() - which does
range checking - to returned the ranged checked number (it currently
always returns zero) and use that as the new value, unless
cfs_trace_get_debug_mb() now returns a non-zero value.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 drivers/staging/lustre/lnet/libcfs/debug.c     | 16 +++++++---------
 drivers/staging/lustre/lnet/libcfs/tracefile.c |  2 +-
 2 files changed, 8 insertions(+), 10 deletions(-)

diff --git a/drivers/staging/lustre/lnet/libcfs/debug.c b/drivers/staging/lustre/lnet/libcfs/debug.c
index 06f694f6a28f..50c2995c1c99 100644
--- a/drivers/staging/lustre/lnet/libcfs/debug.c
+++ b/drivers/staging/lustre/lnet/libcfs/debug.c
@@ -67,17 +67,15 @@ static int libcfs_param_debug_mb_set(const char *val,
 	if (rc < 0)
 		return rc;
 
-	if (!*((unsigned int *)kp->arg)) {
-		*((unsigned int *)kp->arg) = num;
-		return 0;
-	}
-
-	rc = cfs_trace_set_debug_mb(num);
+	num = cfs_trace_set_debug_mb(num);
 
-	if (!rc)
-		*((unsigned int *)kp->arg) = cfs_trace_get_debug_mb();
+	*((unsigned int *)kp->arg) = num;
+	num = cfs_trace_get_debug_mb();
+	if (num)
+		/* This value is more precise */
+		*((unsigned int *)kp->arg) = num;
 
-	return rc;
+	return 0;
 }
 
 /* While debug_mb setting look like unsigned int, in fact
diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.c b/drivers/staging/lustre/lnet/libcfs/tracefile.c
index 5f319332f60b..3b92fd7d3182 100644
--- a/drivers/staging/lustre/lnet/libcfs/tracefile.c
+++ b/drivers/staging/lustre/lnet/libcfs/tracefile.c
@@ -958,7 +958,7 @@ int cfs_trace_set_debug_mb(int mb)
 
 	up_write(&cfs_tracefile_sem);
 
-	return 0;
+	return mb;
 }
 
 int cfs_trace_get_debug_mb(void)
-- 
2.14.0.rc0.dirty

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180702/dc2265db/attachment.sig>

From neilb at suse.com  Mon Jul  2 02:27:24 2018
From: neilb at suse.com (NeilBrown)
Date: Mon, 02 Jul 2018 12:27:24 +1000
Subject: [lustre-devel] [PATCH v3 04/13] lustre: libcfs: fix
 cfs_print_to_console()
In-Reply-To: <alpine.LFD.2.21.1806291716050.26339@casper.infradead.org>
References: <1530128322-32535-1-git-send-email-jsimmons@infradead.org>
 <1530128322-32535-5-git-send-email-jsimmons@infradead.org>
 <FD87D2B8-27A8-45DF-9C7C-BA33CF940C1D@whamcloud.com>
 <alpine.LFD.2.21.1806291716050.26339@casper.infradead.org>
Message-ID: <87h8li2wz7.fsf@notabene.neil.brown.name>

On Fri, Jun 29 2018, James Simmons wrote:

>> > The original code for cfs_print_to_console() used printk() and
>> > used tricks to select which printk level to use. Later a cleanup
>> > patch landed the just converted printk directly to pr_info which
>> > is not exactly correct. Instead of converting back to printk lets
>> > move everything to pr_* type functions which simplify the code.
>> > This allows us to fold both dbghdr_to_err_string() and the
>> > function dbghdr_to_info_string() into cfs_print_to_console().
>> > 
>> > Signed-off-by: James Simmons <jsimmons at infradead.org>
>> > ---
>> > .../staging/lustre/lnet/libcfs/linux-tracefile.c   | 55 ++++++----------------
>> > 1 file changed, 14 insertions(+), 41 deletions(-)
>> > 
>> > diff --git a/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c b/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
>> > index 3af7722..c1747c4 100644
>> > --- a/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
>> > +++ b/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
>> > @@ -140,54 +140,27 @@ enum cfs_trace_buf_type cfs_trace_buf_idx_get(void)
>> > void cfs_print_to_console(struct ptldebug_header *hdr, int mask,
>> > 			  const char *buf, int len, const char *file,
>> > 			  const char *fn)
>> > {
>> > +	char *prefix = "Lustre";
>> > +
>> > +	if (hdr->ph_subsys == S_LND || hdr->ph_subsys == S_LNET)
>> > +		prefix = "LNet";
>> > 
>> > -	if (mask & D_EMERG) {
>> > -		prefix = dbghdr_to_err_string(hdr);
>> > -		ptype = KERN_EMERG;
>> > +	if (mask & (D_CONSOLE | libcfs_printk)) {
>> 
>> This check is broken.  The default value of libcfs_printk (the mask
>> that controls which messages should be printed to the console, and
>> which ones should only be logged into the internal buffer) is:
>> 
>>     #define D_CANTMASK (D_ERROR | D_EMERG | D_WARNING | D_CONSOLE)
>> 
>> so that means virtually every console message will be printed with
>> pr_info() because this is matched first, instead of pr_emerg() or
>> pr_err() below.
>> 
>> That is why the previous code was matching D_EMERG and D_ERROR first,
>> then D_WARNING, and (D_CONSOLE | libcfs_printk) at the end.
>
> So to do this right we need:
>
> static void cfs_print_to_console(struct ptldebug_header *hdr, int mask,
>                                  const char *buf, int len, const char 
> *file,
>                                  const char *fn)
> {
>         char *prefix = "Lustre";
>
>         if (hdr->ph_subsys == S_LND || hdr->ph_subsys == S_LNET)
>                 prefix = "LNet";
>
>         if (mask & D_CONSOLE) {
>                 if (mask & D_EMERG) {
>                         pr_emerg("%sError: %.*s", prefix, len, buf);
>                 } else if (mask & D_ERROR) {
>                         pr_err("%sError: %.*s", prefix, len, buf);
>                 } else if (mask & D_WARNING) {
>                         pr_warn("%s: %.*s", prefix, len, buf);
>                 } else if (mask & libcfs_printk) {
>                         pr_info("%s: %.*s", prefix, len, buf);
>                 }
>         } else {
>                 if (mask & D_EMERG) {
>                         pr_emerg("%sError: %d:%d:(%s:%d:%s()) %.*s", prefix,
>                                  hdr->ph_pid, hdr->ph_extern_pid, file,
>                                  hdr->ph_line_num, fn, len, buf);
>                 } else if (mask & D_ERROR) {
>                         pr_err("%sError: %d:%d:(%s:%d:%s()) %.*s", prefix,
>                                hdr->ph_pid, hdr->ph_extern_pid, file,
>                                hdr->ph_line_num, fn, len, buf);
>                 } else if (mask & D_WARNING) {
>                         pr_warn("%s: %d:%d:(%s:%d:%s()) %.*s", prefix,
>                                 hdr->ph_pid, hdr->ph_extern_pid, file,
>                                 hdr->ph_line_num, fn, len, buf);
>                 } else if (mask & libcfs_printk) {
>                         pr_info("%s: %.*s", prefix, len, buf);
>                 }
>         }
> }

That doesn't look right either.
The original code would *always* print something.
This code looks like it might not (e.g. for mask == 0).

What is "D_CONSOLE" suppose to mean?  It seems to me "make the messages
less verbose" and it isn't clear to me that what is called "D_CONSOLE".

Thanks,
NeilBrown
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180702/de30cd2c/attachment.sig>

From jsimmons at infradead.org  Mon Jul  2 23:24:17 2018
From: jsimmons at infradead.org (James Simmons)
Date: Mon,  2 Jul 2018 19:24:17 -0400
Subject: [lustre-devel] [PATCH 00/18] lustre: missing updates from lustre 2.9
Message-ID: <1530573875-20465-1-git-send-email-jsimmons@infradead.org>

This patch set fills in the known missing pieces from the Lustre
2.9 release. With these patches this brings us to the pre-2.10
development cycle. The grant allocation fixes were missing as
well as security hole closes. The rest are small changes in
addition to support for filesets and ladvise feature.

Alexander Boyko (1):
  lustre: obd: add callback for llog_cat_process_or_fork

Bobi Jam (1):
  lustre: osc: max_pages_per_rpc should be chunk size aligned

Fan Yong (1):
  lustre: ptlrpc: properly set "rq_xid" for 4MB IO

Henri Doreau (1):
  lustre: llite: restore fd_och when putting lease

James Simmons (2):
  lustre: libcfs: restore original behavior in cfs_str2num_check
  lustre: update version to 2.8.99

Jian Yu (1):
  lustre: mount: fix lmd_parse() to handle new delimiters

Jinshan Xiong (1):
  lustre: llite: fast read implementation

Johann Lombardi (1):
  lustre: grant: add support for OBD_CONNECT_GRANT_PARAM

John L. Hammond (3):
  lustre: obd: rename md_getstatus() to md_get_root()
  lustre: obd: reserve connection flag OBD_CONNECT2_FILE_SECCTX
  lustre: security: send file security context for creates

Lai Siyao (1):
  lustre: fileset: add fileset mount support

Li Xi (3):
  lustre: ladvise: Add feature of giving file access advices
  lustre: ladvise: Add willread advice support for ladvise
  lustre: ladvise: Add dontneed advice support for ladvise

Niu Yawei (1):
  lustre: ldlm: reduce mem footprint of ldlm_resource

Patrick Farrell (1):
  lustre: llite: ladvise protocol changes

 .../lustre/include/uapi/linux/lustre/lustre_idl.h  |  52 ++-
 .../lustre/include/uapi/linux/lustre/lustre_user.h |  51 +++
 .../lustre/include/uapi/linux/lustre/lustre_ver.h  |   4 +-
 drivers/staging/lustre/lnet/libcfs/libcfs_string.c |  17 +-
 drivers/staging/lustre/lustre/include/cl_object.h  |  13 +
 .../staging/lustre/lustre/include/lustre_disk.h    |   2 +
 drivers/staging/lustre/lustre/include/lustre_dlm.h |  24 +-
 .../staging/lustre/lustre/include/lustre_import.h  |   1 +
 .../lustre/lustre/include/lustre_req_layout.h      |  10 +-
 .../staging/lustre/lustre/include/lustre_swab.h    |   2 +
 drivers/staging/lustre/lustre/include/obd.h        |  17 +-
 drivers/staging/lustre/lustre/include/obd_class.h  |  16 +-
 .../staging/lustre/lustre/include/obd_support.h    |   5 +-
 drivers/staging/lustre/lustre/ldlm/ldlm_internal.h |   1 +
 drivers/staging/lustre/lustre/ldlm/ldlm_lib.c      |   1 +
 drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c    |  20 +-
 drivers/staging/lustre/lustre/ldlm/ldlm_resource.c |  32 +-
 drivers/staging/lustre/lustre/llite/dir.c          |  54 +++-
 drivers/staging/lustre/lustre/llite/file.c         | 360 ++++++++++++++++++---
 .../staging/lustre/lustre/llite/llite_internal.h   |  34 +-
 drivers/staging/lustre/lustre/llite/llite_lib.c    |  37 ++-
 drivers/staging/lustre/lustre/llite/llite_mmap.c   |  34 +-
 drivers/staging/lustre/lustre/llite/lproc_llite.c  |  38 +++
 drivers/staging/lustre/lustre/llite/namei.c        |  40 ++-
 drivers/staging/lustre/lustre/llite/rw.c           |  68 +++-
 drivers/staging/lustre/lustre/llite/vvp_internal.h |   1 +
 drivers/staging/lustre/lustre/llite/vvp_io.c       |   3 +
 .../staging/lustre/lustre/llite/xattr_security.c   |  38 ++-
 drivers/staging/lustre/lustre/lmv/lmv_obd.c        |  34 +-
 drivers/staging/lustre/lustre/lov/lov_io.c         |  28 ++
 drivers/staging/lustre/lustre/mdc/mdc_internal.h   |   4 +
 drivers/staging/lustre/lustre/mdc/mdc_lib.c        |  32 ++
 drivers/staging/lustre/lustre/mdc/mdc_locks.c      |   7 +
 drivers/staging/lustre/lustre/mdc/mdc_reint.c      |   7 +
 drivers/staging/lustre/lustre/mdc/mdc_request.c    |  50 ++-
 drivers/staging/lustre/lustre/obdclass/cl_io.c     |   2 +
 drivers/staging/lustre/lustre/obdclass/llog_cat.c  |  16 +-
 .../lustre/lustre/obdclass/lprocfs_status.c        |  72 ++++-
 drivers/staging/lustre/lustre/obdclass/obd_mount.c | 135 +++++++-
 drivers/staging/lustre/lustre/osc/lproc_osc.c      |  18 ++
 drivers/staging/lustre/lustre/osc/osc_cache.c      |  95 ++++--
 .../staging/lustre/lustre/osc/osc_cl_internal.h    |   1 +
 drivers/staging/lustre/lustre/osc/osc_dev.c        |   1 +
 drivers/staging/lustre/lustre/osc/osc_internal.h   |   4 +
 drivers/staging/lustre/lustre/osc/osc_io.c         |  79 +++++
 drivers/staging/lustre/lustre/osc/osc_request.c    | 184 +++++++++--
 drivers/staging/lustre/lustre/ptlrpc/client.c      |   9 +-
 drivers/staging/lustre/lustre/ptlrpc/import.c      |  10 +
 drivers/staging/lustre/lustre/ptlrpc/layout.c      |  66 +++-
 .../staging/lustre/lustre/ptlrpc/lproc_ptlrpc.c    |   3 +-
 .../staging/lustre/lustre/ptlrpc/pack_generic.c    |  27 +-
 drivers/staging/lustre/lustre/ptlrpc/wiretest.c    | 122 +++++--
 52 files changed, 1722 insertions(+), 259 deletions(-)

-- 
1.8.3.1


From jsimmons at infradead.org  Mon Jul  2 23:24:18 2018
From: jsimmons at infradead.org (James Simmons)
Date: Mon,  2 Jul 2018 19:24:18 -0400
Subject: [lustre-devel] [PATCH 01/18] lustre: grant: add support for
 OBD_CONNECT_GRANT_PARAM
In-Reply-To: <1530573875-20465-1-git-send-email-jsimmons@infradead.org>
References: <1530573875-20465-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1530573875-20465-2-git-send-email-jsimmons@infradead.org>

From: Johann Lombardi <jlombardi at whamcloud.com>

Add support for grant overhead calculation on the client side.
To do so, clients track usage on a per-extent basis. An extent is
composed of contiguous blocks.
The OST now returns to the OSC layer several parameters to consume
grant more accurately:

- the backend filesystem block size which is the minimal grant
  allocation unit;
- the maximum extent size;
- the extent insertion cost.
  Clients now pack in bulk write how much grant space was consumed for
  the RPC. Dirty data accounting also adopts the same scheme.

Moreover, each backend OSD now reports its own set of parameters:
- For ldiskfs, we usually have a 4KB block size with a maximum extent
  size of 32MB (theoretical limit of 128MB) and an extent insertion
  cost of 6 x 4KB = 24KB
- For ZFS, we report a block size of 128KB, an extent size of 128
  blocks (i.e. 16MB with 128KB block size) and a block insertion cost
  of 112KB.

Besides, there is now no more generic metadata overhead reservation
done inside each OSD. Instead grant space is inflated for clients
that do not support the new grant parameters. That said, a tiny
percentage (typically 0.76%) of the free space is still reserved
inside each OSD to avoid fragmentation which might hurt performance
and impact our grant calculation (e.g. extents are broken due to
fragmentation).

This patch also fixes several other issues:

- Bulk write resent by ptlrpc after reconnection could trigger
  spurious error messages related to broken dirty accounting.
  The issue was that oa_dirty is discarded for resent requests
  (grant flag cleared in ost_brw_write()), so we can legitimately
  have grant > fed_dirty in ofd_grant_check().
  This was fixed by resetting fed_dirty on reconnection and skipping
  the dirty accounting check in ofd_grant_check() in the case of
  ptlrpc resend.

- In obd_connect_data_seqprint(), the connection flags cannot fit
  in a 32-bit integer.

- When merging two OSC extents, an extent tax should be released
  in both the merged extent and in the grant accounting.

Signed-off-by: Johann Lombardi <jlombardi at whamcloud.com>
Signed-off-by: Jinshan Xiong <jinshan.xiong at gmail.com>
WC-bug-id: https://jira.whamcloud.com/browse/2049
Reviewed-on: http://review.whamcloud.com/7793
Reviewed-by: Andreas Dilger <adilger at whamcloud.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark at intel.com>
---
 .../lustre/include/uapi/linux/lustre/lustre_idl.h  |  8 +-
 drivers/staging/lustre/lustre/include/obd.h        |  9 ++-
 .../staging/lustre/lustre/include/obd_support.h    |  1 +
 drivers/staging/lustre/lustre/llite/llite_lib.c    |  9 +++
 .../lustre/lustre/obdclass/lprocfs_status.c        | 10 ++-
 drivers/staging/lustre/lustre/osc/lproc_osc.c      | 18 +++++
 drivers/staging/lustre/lustre/osc/osc_cache.c      | 84 +++++++++++++++------
 drivers/staging/lustre/lustre/osc/osc_request.c    | 85 ++++++++++++++++------
 .../staging/lustre/lustre/ptlrpc/pack_generic.c    |  4 +-
 drivers/staging/lustre/lustre/ptlrpc/wiretest.c    | 32 ++++----
 10 files changed, 191 insertions(+), 69 deletions(-)

diff --git a/drivers/staging/lustre/include/uapi/linux/lustre/lustre_idl.h b/drivers/staging/lustre/include/uapi/linux/lustre/lustre_idl.h
index 6c7e399..3d77ed6 100644
--- a/drivers/staging/lustre/include/uapi/linux/lustre/lustre_idl.h
+++ b/drivers/staging/lustre/include/uapi/linux/lustre/lustre_idl.h
@@ -732,10 +732,10 @@ struct obd_connect_data {
 	__u32 ocd_index;	 /* LOV index to connect to */
 	__u32 ocd_brw_size;	 /* Maximum BRW size in bytes */
 	__u64 ocd_ibits_known;   /* inode bits this client understands */
-	__u8  ocd_blocksize;     /* log2 of the backend filesystem blocksize */
-	__u8  ocd_inodespace;    /* log2 of the per-inode space consumption */
-	__u16 ocd_grant_extent;  /* per-extent grant overhead, in 1K blocks */
-	__u32 ocd_unused;	 /* also fix lustre_swab_connect */
+	__u8  ocd_grant_blkbits; /* log2 of the backend filesystem blocksize */
+	__u8  ocd_grant_inobits; /* log2 of the per-inode space consumption */
+	__u16 ocd_grant_tax_kb;  /* extent grant overhead, in 1K blocks */
+	__u32 ocd_grant_max_blks;/* maximum number of blocks per extent */
 	__u64 ocd_transno;       /* first transno from client to be replayed */
 	__u32 ocd_group;	 /* MDS group on OST */
 	__u32 ocd_cksum_types;   /* supported checksum algorithms */
diff --git a/drivers/staging/lustre/lustre/include/obd.h b/drivers/staging/lustre/lustre/include/obd.h
index d38b6bc..d6fd1ea 100644
--- a/drivers/staging/lustre/lustre/include/obd.h
+++ b/drivers/staging/lustre/lustre/include/obd.h
@@ -198,6 +198,8 @@ struct client_obd {
 	unsigned long		 cl_dirty_transit;	/* dirty synchronous */
 	unsigned long		 cl_avail_grant;	/* bytes of credit for ost */
 	unsigned long		 cl_lost_grant;		/* lost credits (trunc) */
+	/* grant consumed for dirty pages */
+	unsigned long		 cl_dirty_grant;
 
 	/* since we allocate grant by blocks, we don't know how many grant will
 	 * be used to add a page into cache. As a solution, we reserve maximum
@@ -214,7 +216,12 @@ struct client_obd {
 	 * the extent size. A chunk is max(PAGE_SIZE, OST block size)
 	 */
 	int		  cl_chunkbits;
-	unsigned int	  cl_extent_tax; /* extent overhead, by bytes */
+	/* extent insertion metadata overhead to be accounted in grant,
+	 * in bytes
+	 */
+	unsigned int	 cl_grant_extent_tax;
+	/* maximum extent size, in number of pages */
+	unsigned int	 cl_max_extent_pages;
 
 	/* keep track of objects that have lois that contain pages which
 	 * have been queued for async brw.  this lock also protects the
diff --git a/drivers/staging/lustre/lustre/include/obd_support.h b/drivers/staging/lustre/lustre/include/obd_support.h
index 070a281..ca28caf 100644
--- a/drivers/staging/lustre/lustre/include/obd_support.h
+++ b/drivers/staging/lustre/lustre/include/obd_support.h
@@ -320,6 +320,7 @@
 #define OBD_FAIL_OSC_CP_ENQ_RACE	 0x410
 #define OBD_FAIL_OSC_NO_GRANT	    0x411
 #define OBD_FAIL_OSC_DELAY_SETTIME	 0x412
+#define OBD_FAIL_OSC_CONNECT_GRANT_PARAM 0x413
 #define OBD_FAIL_OSC_DELAY_IO		 0x414
 
 #define OBD_FAIL_PTLRPC		  0x500
diff --git a/drivers/staging/lustre/lustre/llite/llite_lib.c b/drivers/staging/lustre/lustre/llite/llite_lib.c
index 9f6f061..df5bc0a 100644
--- a/drivers/staging/lustre/lustre/llite/llite_lib.c
+++ b/drivers/staging/lustre/lustre/llite/llite_lib.c
@@ -178,6 +178,12 @@ static int client_common_fill_super(struct super_block *sb, char *md, char *dt)
 		return -ENOMEM;
 	}
 
+	/*
+	 * pass client page size via ocd_grant_blkbits, the server should report
+	 * back its backend blocksize for grant calculation purpose
+	 */
+	data->ocd_grant_blkbits = PAGE_SHIFT;
+
 	/* indicate the features supported by this client */
 	data->ocd_connect_flags = OBD_CONNECT_IBITS    | OBD_CONNECT_NODEVOH  |
 				  OBD_CONNECT_ATTRFID  |
@@ -367,6 +373,9 @@ static int client_common_fill_super(struct super_block *sb, char *md, char *dt)
 				  OBD_CONNECT_PINGLESS | OBD_CONNECT_LFSCK |
 				  OBD_CONNECT_BULK_MBITS;
 
+	if (!OBD_FAIL_CHECK(OBD_FAIL_OSC_CONNECT_GRANT_PARAM))
+		data->ocd_connect_flags |= OBD_CONNECT_GRANT_PARAM;
+
 	if (!OBD_FAIL_CHECK(OBD_FAIL_OSC_CONNECT_CKSUM)) {
 		/* OBD_CONNECT_CKSUM should always be set, even if checksums are
 		 * disabled by default, because it can still be enabled on the
diff --git a/drivers/staging/lustre/lustre/obdclass/lprocfs_status.c b/drivers/staging/lustre/lustre/obdclass/lprocfs_status.c
index a40ec42..dd88179 100644
--- a/drivers/staging/lustre/lustre/obdclass/lprocfs_status.c
+++ b/drivers/staging/lustre/lustre/obdclass/lprocfs_status.c
@@ -163,10 +163,12 @@ static void obd_connect_data_seqprint(struct seq_file *m,
 	if (flags & OBD_CONNECT_GRANT_PARAM)
 		seq_printf(m, "       grant_block_size: %d\n"
 			   "       grant_inode_size: %d\n"
-			   "       grant_extent_overhead: %d\n",
-			   ocd->ocd_blocksize,
-			   ocd->ocd_inodespace,
-			   ocd->ocd_grant_extent);
+			   "       grant_max_extent_size: %d\n"
+			   "       grant_extent_tax: %d\n",
+			   1 << ocd->ocd_grant_blkbits,
+			   1 << ocd->ocd_grant_inobits,
+			   ocd->ocd_grant_max_blks << ocd->ocd_grant_blkbits,
+			   ocd->ocd_grant_tax_kb << 10);
 	if (flags & OBD_CONNECT_TRANSNO)
 		seq_printf(m, "       first_transno: %llx\n",
 			   ocd->ocd_transno);
diff --git a/drivers/staging/lustre/lustre/osc/lproc_osc.c b/drivers/staging/lustre/lustre/osc/lproc_osc.c
index 64931b9..81adf54 100644
--- a/drivers/staging/lustre/lustre/osc/lproc_osc.c
+++ b/drivers/staging/lustre/lustre/osc/lproc_osc.c
@@ -326,6 +326,23 @@ static ssize_t cur_lost_grant_bytes_show(struct kobject *kobj,
 }
 LUSTRE_RO_ATTR(cur_lost_grant_bytes);
 
+static ssize_t cur_dirty_grant_bytes_show(struct kobject *kobj,
+					  struct attribute *attr,
+					  char *buf)
+{
+	struct obd_device *dev = container_of(kobj, struct obd_device,
+					      obd_kobj);
+	struct client_obd *cli = &dev->u.cli;
+	int len;
+
+	spin_lock(&cli->cl_loi_list_lock);
+	len = sprintf(buf, "%lu\n", cli->cl_dirty_grant);
+	spin_unlock(&cli->cl_loi_list_lock);
+
+	return len;
+}
+LUSTRE_RO_ATTR(cur_dirty_grant_bytes);
+
 static ssize_t grant_shrink_interval_show(struct kobject *kobj,
 					  struct attribute *attr,
 					  char *buf)
@@ -817,6 +834,7 @@ void lproc_osc_attach_seqstat(struct obd_device *dev)
 	&lustre_attr_cur_dirty_bytes.attr,
 	&lustre_attr_cur_grant_bytes.attr,
 	&lustre_attr_cur_lost_grant_bytes.attr,
+	&lustre_attr_cur_dirty_grant_bytes.attr,
 	&lustre_attr_destroys_in_flight.attr,
 	&lustre_attr_grant_shrink_interval.attr,
 	&lustre_attr_lockless_truncate.attr,
diff --git a/drivers/staging/lustre/lustre/osc/osc_cache.c b/drivers/staging/lustre/lustre/osc/osc_cache.c
index 99de672..8d3f501 100644
--- a/drivers/staging/lustre/lustre/osc/osc_cache.c
+++ b/drivers/staging/lustre/lustre/osc/osc_cache.c
@@ -55,13 +55,16 @@ static int osc_refresh_count(const struct lu_env *env,
 static int osc_io_unplug_async(const struct lu_env *env,
 			       struct client_obd *cli, struct osc_object *osc);
 static void osc_free_grant(struct client_obd *cli, unsigned int nr_pages,
-			   unsigned int lost_grant);
+			   unsigned int lost_grant, unsigned int dirty_grant);
 
 static void osc_extent_tree_dump0(int level, struct osc_object *obj,
 				  const char *func, int line);
 #define osc_extent_tree_dump(lvl, obj) \
 	osc_extent_tree_dump0(lvl, obj, __func__, __LINE__)
 
+static void osc_unreserve_grant(struct client_obd *cli, unsigned int reserved,
+				unsigned int unused);
+
 /** \addtogroup osc
  *  @{
  */
@@ -532,12 +535,13 @@ static void osc_extent_remove(struct osc_extent *ext)
 
 /**
  * This function is used to merge extents to get better performance. It checks
- * if @cur and @victim are contiguous at chunk level.
+ * if @cur and @victim are contiguous at block level.
  */
 static int osc_extent_merge(const struct lu_env *env, struct osc_extent *cur,
 			    struct osc_extent *victim)
 {
 	struct osc_object *obj = cur->oe_obj;
+	struct client_obd *cli = osc_cli(obj);
 	pgoff_t chunk_start;
 	pgoff_t chunk_end;
 	int ppc_bits;
@@ -561,11 +565,20 @@ static int osc_extent_merge(const struct lu_env *env, struct osc_extent *cur,
 	    chunk_end + 1 != victim->oe_start >> ppc_bits)
 		return -ERANGE;
 
+	/*
+	 * overall extent size should not exceed the max supported limit
+	 * reported by the server
+	 */
+	if (cur->oe_end - cur->oe_start + 1 +
+	    victim->oe_end - victim->oe_start + 1 > cli->cl_max_extent_pages)
+		return -ERANGE;
+
 	OSC_EXTENT_DUMP(D_CACHE, victim, "will be merged by %p.\n", cur);
 
 	cur->oe_start = min(cur->oe_start, victim->oe_start);
 	cur->oe_end = max(cur->oe_end, victim->oe_end);
-	cur->oe_grants += victim->oe_grants;
+	/* per-extent tax should be accounted only once for the whole extent */
+	cur->oe_grants += victim->oe_grants - cli->cl_grant_extent_tax;
 	cur->oe_nr_pages += victim->oe_nr_pages;
 	/* only the following bits are needed to merge */
 	cur->oe_urgent |= victim->oe_urgent;
@@ -588,6 +601,7 @@ static int osc_extent_merge(const struct lu_env *env, struct osc_extent *cur,
 void osc_extent_release(const struct lu_env *env, struct osc_extent *ext)
 {
 	struct osc_object *obj = ext->oe_obj;
+	struct client_obd *cli = osc_cli(obj);
 
 	LASSERT(atomic_read(&ext->oe_users) > 0);
 	LASSERT(sanity_check(ext) == 0);
@@ -603,13 +617,19 @@ void osc_extent_release(const struct lu_env *env, struct osc_extent *ext)
 			osc_extent_state_set(ext, OES_TRUNC);
 			ext->oe_trunc_pending = 0;
 		} else {
+			int grant = 0;
+
 			osc_extent_state_set(ext, OES_CACHE);
 			osc_update_pending(obj, OBD_BRW_WRITE,
 					   ext->oe_nr_pages);
 
 			/* try to merge the previous and next extent. */
-			osc_extent_merge(env, ext, prev_extent(ext));
-			osc_extent_merge(env, ext, next_extent(ext));
+			if (!osc_extent_merge(env, ext, prev_extent(ext)))
+				grant += cli->cl_grant_extent_tax;
+			if (!osc_extent_merge(env, ext, next_extent(ext)))
+				grant += cli->cl_grant_extent_tax;
+			if (grant > 0)
+				osc_unreserve_grant(cli, 0, grant);
 
 			if (ext->oe_urgent)
 				list_move_tail(&ext->oe_link,
@@ -617,7 +637,7 @@ void osc_extent_release(const struct lu_env *env, struct osc_extent *ext)
 		}
 		osc_object_unlock(obj);
 
-		osc_io_unplug_async(env, osc_cli(obj), obj);
+		osc_io_unplug_async(env, cli, obj);
 	}
 	osc_extent_put(env, ext);
 }
@@ -690,8 +710,8 @@ static struct osc_extent *osc_extent_find(const struct lu_env *env,
 	}
 
 	/* grants has been allocated by caller */
-	LASSERTF(*grants >= chunksize + cli->cl_extent_tax,
-		 "%u/%u/%u.\n", *grants, chunksize, cli->cl_extent_tax);
+	LASSERTF(*grants >= chunksize + cli->cl_grant_extent_tax,
+		 "%u/%u/%u.\n", *grants, chunksize, cli->cl_grant_extent_tax);
 	LASSERTF((max_end - cur->oe_start) < max_pages, EXTSTR "\n",
 		 EXTPARA(cur));
 
@@ -770,6 +790,13 @@ static struct osc_extent *osc_extent_find(const struct lu_env *env,
 			continue;
 		}
 
+		/* check whether maximum extent size will be hit */
+		if ((ext_chk_end - ext_chk_start + 1) << ppc_bits >
+		    cli->cl_max_extent_pages) {
+			ext = next_extent(ext);
+			continue;
+		}
+
 		/* it's required that an extent must be contiguous at chunk
 		 * level so that we know the whole extent is covered by grant
 		 * (the pages in the extent are NOT required to be contiguous).
@@ -801,7 +828,7 @@ static struct osc_extent *osc_extent_find(const struct lu_env *env,
 			 */
 			if (osc_extent_merge(env, ext, next_extent(ext)) == 0)
 				/* we can save extent tax from next extent */
-				*grants += cli->cl_extent_tax;
+				*grants += cli->cl_grant_extent_tax;
 
 			found = osc_extent_hold(ext);
 		}
@@ -822,7 +849,7 @@ static struct osc_extent *osc_extent_find(const struct lu_env *env,
 	} else if (!conflict) {
 		/* create a new extent */
 		EASSERT(osc_extent_is_overlapped(obj, cur) == 0, cur);
-		cur->oe_grants = chunksize + cli->cl_extent_tax;
+		cur->oe_grants = chunksize + cli->cl_grant_extent_tax;
 		LASSERT(*grants >= cur->oe_grants);
 		*grants -= cur->oe_grants;
 
@@ -908,7 +935,7 @@ int osc_extent_finish(const struct lu_env *env, struct osc_extent *ext,
 		lost_grant = PAGE_SIZE - count;
 	}
 	if (ext->oe_grants > 0)
-		osc_free_grant(cli, nr_pages, lost_grant);
+		osc_free_grant(cli, nr_pages, lost_grant, ext->oe_grants);
 
 	osc_extent_remove(ext);
 	/* put the refcount for RPC */
@@ -1084,7 +1111,7 @@ static int osc_extent_truncate(struct osc_extent *ext, pgoff_t trunc_index,
 	osc_object_unlock(obj);
 
 	if (grants > 0 || nr_pages > 0)
-		osc_free_grant(cli, nr_pages, grants);
+		osc_free_grant(cli, nr_pages, grants, grants);
 
 out:
 	cl_io_fini(env, io);
@@ -1207,9 +1234,16 @@ static int osc_extent_expand(struct osc_extent *ext, pgoff_t index,
 	}
 
 	LASSERT(end_chunk + 1 == chunk);
+
 	/* try to expand this extent to cover @index */
 	end_index = min(ext->oe_max_end, ((chunk + 1) << ppc_bits) - 1);
 
+	/* don't go over the maximum extent size reported by server */
+	if (end_index - ext->oe_start + 1 > cli->cl_max_extent_pages) {
+		rc = -ERANGE;
+		goto out;
+	}
+
 	next = next_extent(ext);
 	if (next && next->oe_start <= end_index) {
 		/* complex mode - overlapped with the next extent,
@@ -1374,13 +1408,15 @@ static int osc_completion(const struct lu_env *env, struct osc_async_page *oap,
 
 #define OSC_DUMP_GRANT(lvl, cli, fmt, args...) do {			      \
 	struct client_obd *__tmp = (cli);				      \
-	CDEBUG(lvl, "%s: grant { dirty: %lu/%lu dirty_pages: %ld/%lu "	      \
-	       "dropped: %ld avail: %ld, reserved: %ld, flight: %d }"	      \
-	       "lru {in list: %ld, left: %ld, waiters: %d }" fmt "\n",	      \
+	CDEBUG(lvl, "%s: grant { dirty: %ld/%ld dirty_pages: %ld/%lu "	\
+	       "dropped: %ld avail: %ld, dirty_grant: %ld, "		\
+	       "reserved: %ld, flight: %d } lru {in list: %ld, "	\
+	       "left: %ld, waiters: %d }" fmt "\n",			\
 	       cli_name(__tmp),						      \
 	       __tmp->cl_dirty_pages, __tmp->cl_dirty_max_pages,	      \
 	       atomic_long_read(&obd_dirty_pages), obd_max_dirty_pages,	      \
 	       __tmp->cl_lost_grant, __tmp->cl_avail_grant,		      \
+	       __tmp->cl_dirty_grant,					\
 	       __tmp->cl_reserved_grant, __tmp->cl_w_in_flight,		      \
 	       atomic_long_read(&__tmp->cl_lru_in_list),		      \
 	       atomic_long_read(&__tmp->cl_lru_busy),			      \
@@ -1451,8 +1487,10 @@ static void __osc_unreserve_grant(struct client_obd *cli,
 	if (unused > reserved) {
 		cli->cl_avail_grant += reserved;
 		cli->cl_lost_grant  += unused - reserved;
+		cli->cl_dirty_grant -= unused - reserved;
 	} else {
 		cli->cl_avail_grant += unused;
+		cli->cl_dirty_grant += reserved - unused;
 	}
 }
 
@@ -1480,14 +1518,17 @@ static void osc_unreserve_grant(struct client_obd *cli,
  *    See filter_grant_check() for details.
  */
 static void osc_free_grant(struct client_obd *cli, unsigned int nr_pages,
-			   unsigned int lost_grant)
+			   unsigned int lost_grant, unsigned int dirty_grant)
 {
-	unsigned long grant = (1 << cli->cl_chunkbits) + cli->cl_extent_tax;
+	unsigned long grant;
+
+	grant = (1 << cli->cl_chunkbits) + cli->cl_grant_extent_tax;
 
 	spin_lock(&cli->cl_loi_list_lock);
 	atomic_long_sub(nr_pages, &obd_dirty_pages);
 	cli->cl_dirty_pages -= nr_pages;
 	cli->cl_lost_grant += lost_grant;
+	cli->cl_dirty_grant -= dirty_grant;
 	if (cli->cl_avail_grant < grant && cli->cl_lost_grant >= grant) {
 		/* borrow some grant from truncate to avoid the case that
 		 * truncate uses up all avail grant
@@ -1497,9 +1538,10 @@ static void osc_free_grant(struct client_obd *cli, unsigned int nr_pages,
 	}
 	osc_wake_cache_waiters(cli);
 	spin_unlock(&cli->cl_loi_list_lock);
-	CDEBUG(D_CACHE, "lost %u grant: %lu avail: %lu dirty: %lu\n",
+	CDEBUG(D_CACHE, "lost %u grant: %lu avail: %lu dirty: %lu/%lu\n",
 	       lost_grant, cli->cl_lost_grant,
-	       cli->cl_avail_grant, cli->cl_dirty_pages << PAGE_SHIFT);
+	       cli->cl_avail_grant, cli->cl_dirty_pages << PAGE_SHIFT,
+	       cli->cl_dirty_grant);
 }
 
 /**
@@ -2437,7 +2479,7 @@ int osc_queue_async_io(const struct lu_env *env, struct cl_io *io,
 		/* one chunk plus extent overhead must be enough to write this
 		 * page
 		 */
-		grants = (1 << cli->cl_chunkbits) + cli->cl_extent_tax;
+		grants = (1 << cli->cl_chunkbits) + cli->cl_grant_extent_tax;
 		if (ext->oe_end >= index)
 			grants = 0;
 
@@ -2474,7 +2516,7 @@ int osc_queue_async_io(const struct lu_env *env, struct cl_io *io,
 	}
 
 	if (!ext) {
-		tmp = (1 << cli->cl_chunkbits) + cli->cl_extent_tax;
+		tmp = (1 << cli->cl_chunkbits) + cli->cl_grant_extent_tax;
 
 		/* try to find new extent to cover this page */
 		LASSERT(!oio->oi_active);
diff --git a/drivers/staging/lustre/lustre/osc/osc_request.c b/drivers/staging/lustre/lustre/osc/osc_request.c
index bcb9b91..ce073b6 100644
--- a/drivers/staging/lustre/lustre/osc/osc_request.c
+++ b/drivers/staging/lustre/lustre/osc/osc_request.c
@@ -576,7 +576,10 @@ static void osc_announce_cached(struct client_obd *cli, struct obdo *oa,
 
 	oa->o_valid |= bits;
 	spin_lock(&cli->cl_loi_list_lock);
-	oa->o_dirty = cli->cl_dirty_pages << PAGE_SHIFT;
+	if (OCD_HAS_FLAG(&cli->cl_import->imp_connect_data, GRANT_PARAM))
+		oa->o_dirty = cli->cl_dirty_grant;
+	else
+		oa->o_dirty = cli->cl_dirty_pages << PAGE_SHIFT;
 	if (unlikely(cli->cl_dirty_pages - cli->cl_dirty_transit >
 		     cli->cl_dirty_max_pages)) {
 		CERROR("dirty %lu - %lu > dirty_max %lu\n",
@@ -601,12 +604,24 @@ static void osc_announce_cached(struct client_obd *cli, struct obdo *oa,
 		       cli->cl_dirty_pages, cli->cl_dirty_max_pages);
 		oa->o_undirty = 0;
 	} else {
-		unsigned long max_in_flight;
-
-		max_in_flight = (cli->cl_max_pages_per_rpc << PAGE_SHIFT) *
-				(cli->cl_max_rpcs_in_flight + 1);
-		oa->o_undirty = max(cli->cl_dirty_max_pages << PAGE_SHIFT,
-				    max_in_flight);
+		unsigned long nrpages;
+
+		nrpages = cli->cl_max_pages_per_rpc;
+		nrpages *= cli->cl_max_rpcs_in_flight + 1;
+		nrpages = max(nrpages, cli->cl_dirty_max_pages);
+		oa->o_undirty = nrpages << PAGE_SHIFT;
+		if (OCD_HAS_FLAG(&cli->cl_import->imp_connect_data,
+				 GRANT_PARAM)) {
+			int nrextents;
+
+			/*
+			 * take extent tax into account when asking for more
+			 * grant space
+			 */
+			nrextents = (nrpages + cli->cl_max_extent_pages - 1)  /
+				     cli->cl_max_extent_pages;
+			oa->o_undirty += nrextents * cli->cl_grant_extent_tax;
+		}
 	}
 	oa->o_grant = cli->cl_avail_grant + cli->cl_reserved_grant;
 	oa->o_dropped = cli->cl_lost_grant;
@@ -811,20 +826,40 @@ static void osc_init_grant(struct client_obd *cli, struct obd_connect_data *ocd)
 	 * race is tolerable here: if we're evicted, but imp_state already
 	 * left EVICTED state, then cl_dirty_pages must be 0 already.
 	 */
+	cli->cl_avail_grant = ocd->ocd_grant;
 	spin_lock(&cli->cl_loi_list_lock);
-	if (cli->cl_import->imp_state == LUSTRE_IMP_EVICTED)
-		cli->cl_avail_grant = ocd->ocd_grant;
-	else
-		cli->cl_avail_grant = ocd->ocd_grant -
-				      (cli->cl_dirty_pages << PAGE_SHIFT);
-
-	/* determine the appropriate chunk size used by osc_extent. */
-	cli->cl_chunkbits = max_t(int, PAGE_SHIFT, ocd->ocd_blocksize);
+	if (cli->cl_import->imp_state != LUSTRE_IMP_EVICTED) {
+		cli->cl_avail_grant -= cli->cl_reserved_grant;
+		if (OCD_HAS_FLAG(ocd, GRANT_PARAM))
+			cli->cl_avail_grant -= cli->cl_dirty_grant;
+		else
+			cli->cl_avail_grant -= cli->cl_dirty_pages << PAGE_SHIFT;
+	}
+
+	if (OCD_HAS_FLAG(ocd, GRANT_PARAM)) {
+		u64 size;
+
+		/* overhead for each extent insertion */
+		cli->cl_grant_extent_tax = ocd->ocd_grant_tax_kb << 10;
+		/* determine the appropriate chunk size used by osc_extent. */
+		cli->cl_chunkbits = max_t(int, PAGE_SHIFT,
+					  ocd->ocd_grant_blkbits);
+		/* determine maximum extent size, in #pages */
+		size = (u64)ocd->ocd_grant_max_blks << ocd->ocd_grant_blkbits;
+		cli->cl_max_extent_pages = size >> PAGE_SHIFT;
+		if (!cli->cl_max_extent_pages)
+			cli->cl_max_extent_pages = 1;
+	} else {
+		cli->cl_grant_extent_tax = 0;
+		cli->cl_chunkbits = PAGE_SHIFT;
+		cli->cl_max_extent_pages = DT_MAX_BRW_PAGES;
+	}
 	spin_unlock(&cli->cl_loi_list_lock);
 
-	CDEBUG(D_CACHE, "%s, setting cl_avail_grant: %ld cl_lost_grant: %ld chunk bits: %d\n",
+	CDEBUG(D_CACHE,
+	       "%s, setting cl_avail_grant: %ld cl_lost_grant: %ld chunk bits: %d cl_max_extent_pages: %d\n",
 	       cli_name(cli), cli->cl_avail_grant, cli->cl_lost_grant,
-	       cli->cl_chunkbits);
+	       cli->cl_chunkbits, cli->cl_max_extent_pages);
 
 	if (ocd->ocd_connect_flags & OBD_CONNECT_GRANT_SHRINK &&
 	    list_empty(&cli->cl_grant_shrink_list))
@@ -1661,6 +1696,7 @@ int osc_build_rpc(const struct lu_env *env, struct client_obd *cli,
 	int page_count = 0;
 	bool soft_sync = false;
 	bool interrupted = false;
+	int grant = 0;
 	int i;
 	int rc;
 	struct ost_body *body;
@@ -1672,6 +1708,7 @@ int osc_build_rpc(const struct lu_env *env, struct client_obd *cli,
 	list_for_each_entry(ext, ext_list, oe_link) {
 		LASSERT(ext->oe_state == OES_RPC);
 		mem_tight |= ext->oe_memalloc;
+		grant += ext->oe_grants;
 		page_count += ext->oe_nr_pages;
 		if (!obj)
 			obj = ext->oe_obj;
@@ -1732,6 +1769,9 @@ int osc_build_rpc(const struct lu_env *env, struct client_obd *cli,
 	crattr->cra_oa = oa;
 	cl_req_attr_set(env, osc2cl(obj), crattr);
 
+	if (cmd == OBD_BRW_WRITE)
+		oa->o_grant_used = grant;
+
 	sort_brw_pages(pga, page_count);
 	rc = osc_brw_prep_request(cmd, cli, oa, page_count, pga, &req, 1, 0);
 	if (rc != 0) {
@@ -2435,12 +2475,15 @@ static int osc_reconnect(const struct lu_env *env,
 	struct client_obd *cli = &obd->u.cli;
 
 	if (data && (data->ocd_connect_flags & OBD_CONNECT_GRANT)) {
-		long lost_grant;
+		long lost_grant, grant;
 
 		spin_lock(&cli->cl_loi_list_lock);
-		data->ocd_grant = (cli->cl_avail_grant +
-				   (cli->cl_dirty_pages << PAGE_SHIFT)) ?:
-				   2 * cli_brw_size(obd);
+		grant = cli->cl_avail_grant + cli->cl_reserved_grant;
+		if (data->ocd_connect_flags & OBD_CONNECT_GRANT_PARAM)
+			grant += cli->cl_dirty_grant;
+		else
+			grant += cli->cl_dirty_pages << PAGE_SHIFT;
+		data->ocd_grant = grant ? : 2 * cli_brw_size(obd);
 		lost_grant = cli->cl_lost_grant;
 		cli->cl_lost_grant = 0;
 		spin_unlock(&cli->cl_loi_list_lock);
diff --git a/drivers/staging/lustre/lustre/ptlrpc/pack_generic.c b/drivers/staging/lustre/lustre/ptlrpc/pack_generic.c
index 6ac9bb5..0337b33 100644
--- a/drivers/staging/lustre/lustre/ptlrpc/pack_generic.c
+++ b/drivers/staging/lustre/lustre/ptlrpc/pack_generic.c
@@ -1551,8 +1551,8 @@ void lustre_swab_connect(struct obd_connect_data *ocd)
 	/* ocd_blocksize and ocd_inodespace don't need to be swabbed because
 	 * they are 8-byte values
 	 */
-	__swab16s(&ocd->ocd_grant_extent);
-	__swab32s(&ocd->ocd_unused);
+	__swab16s(&ocd->ocd_grant_tax_kb);
+	__swab32s(&ocd->ocd_grant_max_blks);
 	__swab64s(&ocd->ocd_transno);
 	__swab32s(&ocd->ocd_group);
 	__swab32s(&ocd->ocd_cksum_types);
diff --git a/drivers/staging/lustre/lustre/ptlrpc/wiretest.c b/drivers/staging/lustre/lustre/ptlrpc/wiretest.c
index f9394c3..2b3608c 100644
--- a/drivers/staging/lustre/lustre/ptlrpc/wiretest.c
+++ b/drivers/staging/lustre/lustre/ptlrpc/wiretest.c
@@ -884,22 +884,22 @@ void lustre_assert_wire_constants(void)
 		 (long long)(int)offsetof(struct obd_connect_data, ocd_ibits_known));
 	LASSERTF((int)sizeof(((struct obd_connect_data *)0)->ocd_ibits_known) == 8, "found %lld\n",
 		 (long long)(int)sizeof(((struct obd_connect_data *)0)->ocd_ibits_known));
-	LASSERTF((int)offsetof(struct obd_connect_data, ocd_blocksize) == 32, "found %lld\n",
-		 (long long)(int)offsetof(struct obd_connect_data, ocd_blocksize));
-	LASSERTF((int)sizeof(((struct obd_connect_data *)0)->ocd_blocksize) == 1, "found %lld\n",
-		 (long long)(int)sizeof(((struct obd_connect_data *)0)->ocd_blocksize));
-	LASSERTF((int)offsetof(struct obd_connect_data, ocd_inodespace) == 33, "found %lld\n",
-		 (long long)(int)offsetof(struct obd_connect_data, ocd_inodespace));
-	LASSERTF((int)sizeof(((struct obd_connect_data *)0)->ocd_inodespace) == 1, "found %lld\n",
-		 (long long)(int)sizeof(((struct obd_connect_data *)0)->ocd_inodespace));
-	LASSERTF((int)offsetof(struct obd_connect_data, ocd_grant_extent) == 34, "found %lld\n",
-		 (long long)(int)offsetof(struct obd_connect_data, ocd_grant_extent));
-	LASSERTF((int)sizeof(((struct obd_connect_data *)0)->ocd_grant_extent) == 2, "found %lld\n",
-		 (long long)(int)sizeof(((struct obd_connect_data *)0)->ocd_grant_extent));
-	LASSERTF((int)offsetof(struct obd_connect_data, ocd_unused) == 36, "found %lld\n",
-		 (long long)(int)offsetof(struct obd_connect_data, ocd_unused));
-	LASSERTF((int)sizeof(((struct obd_connect_data *)0)->ocd_unused) == 4, "found %lld\n",
-		 (long long)(int)sizeof(((struct obd_connect_data *)0)->ocd_unused));
+	LASSERTF((int)offsetof(struct obd_connect_data, ocd_grant_blkbits) == 32, "found %lld\n",
+		 (long long)(int)offsetof(struct obd_connect_data, ocd_grant_blkbits));
+	LASSERTF((int)sizeof(((struct obd_connect_data *)0)->ocd_grant_blkbits) == 1, "found %lld\n",
+		 (long long)(int)sizeof(((struct obd_connect_data *)0)->ocd_grant_blkbits));
+	LASSERTF((int)offsetof(struct obd_connect_data, ocd_grant_inobits) == 33, "found %lld\n",
+		 (long long)(int)offsetof(struct obd_connect_data, ocd_grant_inobits));
+	LASSERTF((int)sizeof(((struct obd_connect_data *)0)->ocd_grant_inobits) == 1, "found %lld\n",
+		 (long long)(int)sizeof(((struct obd_connect_data *)0)->ocd_grant_inobits));
+	LASSERTF((int)offsetof(struct obd_connect_data, ocd_grant_tax_kb) == 34, "found %lld\n",
+		 (long long)(int)offsetof(struct obd_connect_data, ocd_grant_tax_kb));
+	LASSERTF((int)sizeof(((struct obd_connect_data *)0)->ocd_grant_tax_kb) == 2, "found %lld\n",
+		 (long long)(int)sizeof(((struct obd_connect_data *)0)->ocd_grant_tax_kb));
+	LASSERTF((int)offsetof(struct obd_connect_data, ocd_grant_max_blks) == 36, "found %lld\n",
+		 (long long)(int)offsetof(struct obd_connect_data, ocd_grant_max_blks));
+	LASSERTF((int)sizeof(((struct obd_connect_data *)0)->ocd_grant_max_blks) == 4, "found %lld\n",
+		 (long long)(int)sizeof(((struct obd_connect_data *)0)->ocd_grant_max_blks));
 	LASSERTF((int)offsetof(struct obd_connect_data, ocd_transno) == 40, "found %lld\n",
 		 (long long)(int)offsetof(struct obd_connect_data, ocd_transno));
 	LASSERTF((int)sizeof(((struct obd_connect_data *)0)->ocd_transno) == 8, "found %lld\n",
-- 
1.8.3.1


From jsimmons at infradead.org  Mon Jul  2 23:24:22 2018
From: jsimmons at infradead.org (James Simmons)
Date: Mon,  2 Jul 2018 19:24:22 -0400
Subject: [lustre-devel] [PATCH 05/18] lustre: llite: fast read implementation
In-Reply-To: <1530573875-20465-1-git-send-email-jsimmons@infradead.org>
References: <1530573875-20465-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1530573875-20465-6-git-send-email-jsimmons@infradead.org>

From: Jinshan Xiong <jinshan.xiong at gmail.com>

For read operation, if a page is already in cache, it must be covered
by a DLM lock. We can take advantage of this by reading cached page
without interacting with Lustre. Traditional read will go on if fast
read fails.

This patch can improve small read performance significantly.
These are the performance data I collected:

+------------+----------------+-----------------+
|            | read bs=4k     | read bs=1M      |
+------------+----------------+-----------------+
| w/o patch  | 257 MB/s       | 1.1 GB/s        |
+------------+----------------+-----------------+
| w/ patch   | 1.2 GB/s       | 1.4 GB/s        |
+------------+----------------+-----------------+

Signed-off-by: Jinshan Xiong <jinshan.xiong at gmail.com>
WC-bug-id: https://jira.whamcloud.com/browse/LU-4257
Reviewed-on: http://review.whamcloud.com/20255
Reviewed-by: Bobi Jam <bobijam at hotmail.com>
Reviewed-by: Andreas Dilger <adilger at whamcloud.com>
Reviewed-by: Oleg Drokin <green at whamcloud.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/lustre/llite/file.c         | 104 +++++++++++++++++++--
 .../staging/lustre/lustre/llite/llite_internal.h   |  23 ++++-
 drivers/staging/lustre/lustre/llite/llite_lib.c    |   1 +
 drivers/staging/lustre/lustre/llite/llite_mmap.c   |  34 ++++++-
 drivers/staging/lustre/lustre/llite/lproc_llite.c  |  38 ++++++++
 drivers/staging/lustre/lustre/llite/rw.c           |  68 ++++++++++++--
 drivers/staging/lustre/lustre/llite/vvp_internal.h |   1 +
 7 files changed, 252 insertions(+), 17 deletions(-)

diff --git a/drivers/staging/lustre/lustre/llite/file.c b/drivers/staging/lustre/lustre/llite/file.c
index 5f944ca..db18d1d 100644
--- a/drivers/staging/lustre/lustre/llite/file.c
+++ b/drivers/staging/lustre/lustre/llite/file.c
@@ -953,10 +953,21 @@ int ll_merge_attr(const struct lu_env *env, struct inode *inode)
 
 	ll_inode_size_lock(inode);
 
-	/* merge timestamps the most recently obtained from mds with
-	 * timestamps obtained from osts
+	/*
+	 * merge timestamps the most recently obtained from MDS with
+	 * timestamps obtained from OSTSs.
+	 *
+	 * Do not overwrite atime of inode because it may be refreshed
+	 * by file_accessed() function. If the read was served by cache
+	 * data, there is no RPC to be sent so that atime may not be
+	 * transferred to OSTs at all. MDT only updates atime at close time
+	 * if it's at least 'mdd.*.atime_diff' older.
+	 * All in all, the atime in Lustre does not strictly comply with
+	 * POSIX. Solving this problem needs to send an RPC to MDT for each
+	 * read, this will hurt performance.
 	 */
-	inode->i_atime.tv_sec = lli->lli_atime;
+	if (inode->i_atime.tv_sec < lli->lli_atime)
+		inode->i_atime.tv_sec = lli->lli_atime;
 	inode->i_mtime.tv_sec = lli->lli_mtime;
 	inode->i_ctime.tv_sec = lli->lli_ctime;
 
@@ -1096,7 +1107,7 @@ static void ll_io_init(struct cl_io *io, const struct file *file, int write)
 
 			range_locked = true;
 		}
-		ll_cl_add(file, env, io);
+		ll_cl_add(file, env, io, LCC_RW);
 		rc = cl_io_loop(env, io);
 		ll_cl_remove(file, env);
 		if (range_locked) {
@@ -1155,23 +1166,104 @@ static void ll_io_init(struct cl_io *io, const struct file *file, int write)
 	return result > 0 ? result : rc;
 }
 
+/**
+ * The purpose of fast read is to overcome per I/O overhead and improve IOPS
+ * especially for small I/O.
+ *
+ * To serve a read request, CLIO has to create and initialize a cl_io and
+ * then request DLM lock. This has turned out to have siginificant overhead
+ * and affects the performance of small I/O dramatically.
+ *
+ * It's not necessary to create a cl_io for each I/O. Under the help of read
+ * ahead, most of the pages being read are already in memory cache and we can
+ * read those pages directly because if the pages exist, the corresponding DLM
+ * lock must exist so that page content must be valid.
+ *
+ * In fast read implementation, the llite speculatively finds and reads pages
+ * in memory cache. There are three scenarios for fast read:
+ *   - If the page exists and is uptodate, kernel VM will provide the data and
+ *     CLIO won't be intervened;
+ *   - If the page was brought into memory by read ahead, it will be exported
+ *     and read ahead parameters will be updated;
+ *   - Otherwise the page is not in memory, we can't do fast read. Therefore,
+ *     it will go back and invoke normal read, i.e., a cl_io will be created
+ *     and DLM lock will be requested.
+ *
+ * POSIX compliance: posix standard states that read is intended to be atomic.
+ * Lustre read implementation is in line with Linux kernel read implementation
+ * and neither of them complies with POSIX standard in this matter. Fast read
+ * doesn't make the situation worse on single node but it may interleave write
+ * results from multiple nodes due to short read handling in ll_file_aio_read().
+ *
+ * @env   - lu_env
+ * @iocb  - kiocb from kernel
+ * @iter  - user space buffers where the data will be copied
+ *
+ * RETURN - number of bytes have been read, or error code if error occurred.
+ */
+static ssize_t
+ll_do_fast_read(const struct lu_env *env, struct kiocb *iocb,
+		struct iov_iter *iter)
+{
+	ssize_t result;
+
+	if (!ll_sbi_has_fast_read(ll_i2sbi(file_inode(iocb->ki_filp))))
+		return 0;
+
+	/*
+	 * NB: we can't do direct IO for fast read because it will need a lock
+	 * to make IO engine happy.
+	 */
+	if (iocb->ki_filp->f_flags & O_DIRECT)
+		return 0;
+
+	ll_cl_add(iocb->ki_filp, env, NULL, LCC_RW);
+	result = generic_file_read_iter(iocb, iter);
+	ll_cl_remove(iocb->ki_filp, env);
+
+	/*
+	 * If the first page is not in cache, generic_file_aio_read() will be
+	 * returned with -ENODATA.
+	 * See corresponding code in ll_readpage().
+	 */
+	if (result == -ENODATA)
+		result = 0;
+
+	if (result > 0)
+		ll_stats_ops_tally(ll_i2sbi(file_inode(iocb->ki_filp)),
+				   LPROC_LL_READ_BYTES, result);
+
+	return result;
+}
+
 static ssize_t ll_file_read_iter(struct kiocb *iocb, struct iov_iter *to)
 {
 	struct lu_env      *env;
 	struct vvp_io_args *args;
 	ssize_t	     result;
 	u16 refcheck;
+	ssize_t rc2;
 
 	env = cl_env_get(&refcheck);
 	if (IS_ERR(env))
 		return PTR_ERR(env);
 
+	result = ll_do_fast_read(env, iocb, to);
+	if (result < 0 || iov_iter_count(to) == 0)
+		goto out;
+
 	args = ll_env_args(env);
 	args->u.normal.via_iter = to;
 	args->u.normal.via_iocb = iocb;
 
-	result = ll_file_io_generic(env, args, iocb->ki_filp, CIT_READ,
-				    &iocb->ki_pos, iov_iter_count(to));
+	rc2 = ll_file_io_generic(env, args, iocb->ki_filp, CIT_READ,
+				 &iocb->ki_pos, iov_iter_count(to));
+	if (rc2 > 0)
+		result += rc2;
+	else if (result == 0)
+		result = rc2;
+
+out:
 	cl_env_put(env, &refcheck);
 	return result;
 }
diff --git a/drivers/staging/lustre/lustre/llite/llite_internal.h b/drivers/staging/lustre/lustre/llite/llite_internal.h
index 8770d10..86914c9 100644
--- a/drivers/staging/lustre/lustre/llite/llite_internal.h
+++ b/drivers/staging/lustre/lustre/llite/llite_internal.h
@@ -395,6 +395,7 @@ enum stats_track_type {
 #define LL_SBI_ALWAYS_PING	0x200000 /* always ping even if server
 					  * suppress_pings
 					  */
+#define LL_SBI_FAST_READ	0x400000 /* fast read support */
 
 #define LL_SBI_FLAGS {	\
 	"nolck",	\
@@ -419,6 +420,7 @@ enum stats_track_type {
 	"xattr_cache",	\
 	"norootsquash",	\
 	"always_ping",	\
+	"fast_read",    \
 }
 
 /*
@@ -646,6 +648,11 @@ static inline int ll_need_32bit_api(struct ll_sb_info *sbi)
 #endif
 }
 
+static inline bool ll_sbi_has_fast_read(struct ll_sb_info *sbi)
+{
+	return !!(sbi->ll_flags & LL_SBI_FAST_READ);
+}
+
 void ll_ras_enter(struct file *f);
 
 /* llite/lcommon_misc.c */
@@ -678,6 +685,8 @@ enum {
 	LPROC_LL_OPEN,
 	LPROC_LL_RELEASE,
 	LPROC_LL_MAP,
+	LPROC_LL_FAULT,
+	LPROC_LL_MKWRITE,
 	LPROC_LL_LLSEEK,
 	LPROC_LL_FSYNC,
 	LPROC_LL_READDIR,
@@ -732,9 +741,12 @@ int ll_md_blocking_ast(struct ldlm_lock *lock, struct ldlm_lock_desc *desc,
 int ll_readpage(struct file *file, struct page *page);
 void ll_readahead_init(struct inode *inode, struct ll_readahead_state *ras);
 int vvp_io_write_commit(const struct lu_env *env, struct cl_io *io);
-struct ll_cl_context *ll_cl_find(struct file *file);
-void ll_cl_add(struct file *file, const struct lu_env *env, struct cl_io *io);
+
+enum lcc_type;
+void ll_cl_add(struct file *file, const struct lu_env *env, struct cl_io *io,
+	       enum lcc_type type);
 void ll_cl_remove(struct file *file, const struct lu_env *env);
+struct ll_cl_context *ll_cl_find(struct file *file);
 
 extern const struct address_space_operations ll_aops;
 
@@ -891,15 +903,22 @@ struct vvp_io_args {
 	} u;
 };
 
+enum lcc_type {
+	LCC_RW = 1,
+	LCC_MMAP
+};
+
 struct ll_cl_context {
 	struct list_head	 lcc_list;
 	void	   *lcc_cookie;
 	const struct lu_env	*lcc_env;
 	struct cl_io   *lcc_io;
 	struct cl_page *lcc_page;
+	enum lcc_type		 lcc_type;
 };
 
 struct ll_thread_info {
+	struct iov_iter		lti_iter;
 	struct vvp_io_args   lti_args;
 	struct ra_io_arg     lti_ria;
 	struct ll_cl_context lti_io_ctx;
diff --git a/drivers/staging/lustre/lustre/llite/llite_lib.c b/drivers/staging/lustre/lustre/llite/llite_lib.c
index 90dff0a..6e47e5b 100644
--- a/drivers/staging/lustre/lustre/llite/llite_lib.c
+++ b/drivers/staging/lustre/lustre/llite/llite_lib.c
@@ -122,6 +122,7 @@ static struct ll_sb_info *ll_init_sbi(struct super_block *sb)
 	atomic_set(&sbi->ll_sa_running, 0);
 	atomic_set(&sbi->ll_agl_total, 0);
 	sbi->ll_flags |= LL_SBI_AGL_ENABLED;
+	sbi->ll_flags |= LL_SBI_FAST_READ;
 
 	/* root squash */
 	sbi->ll_squash.rsi_uid = 0;
diff --git a/drivers/staging/lustre/lustre/llite/llite_mmap.c b/drivers/staging/lustre/lustre/llite/llite_mmap.c
index 8cb8036..023d62e 100644
--- a/drivers/staging/lustre/lustre/llite/llite_mmap.c
+++ b/drivers/staging/lustre/lustre/llite/llite_mmap.c
@@ -277,6 +277,28 @@ static vm_fault_t ll_fault0(struct vm_area_struct *vma, struct vm_fault *vmf)
 	if (IS_ERR(env))
 		return VM_FAULT_ERROR;
 
+	if (ll_sbi_has_fast_read(ll_i2sbi(file_inode(vma->vm_file)))) {
+		/* do fast fault */
+		ll_cl_add(vma->vm_file, env, NULL, LCC_MMAP);
+		fault_ret = filemap_fault(vmf);
+		ll_cl_remove(vma->vm_file, env);
+
+		/*
+		 * - If there is no error, then the page was found in cache and
+		 *   uptodate;
+		 * - If VM_FAULT_RETRY is set, the page existed but failed to
+		 *   lock. It will return to kernel and retry;
+		 * - Otherwise, it should try normal fault under DLM lock.
+		 */
+		if ((fault_ret & VM_FAULT_RETRY) ||
+		    !(fault_ret & VM_FAULT_ERROR)) {
+			result = 0;
+			goto out;
+		}
+
+		fault_ret = 0;
+	}
+
 	io = ll_fault_io_init(env, vma, vmf->pgoff, &ra_flags);
 	if (IS_ERR(io)) {
 		fault_ret = to_fault_error(PTR_ERR(io));
@@ -293,7 +315,7 @@ static vm_fault_t ll_fault0(struct vm_area_struct *vma, struct vm_fault *vmf)
 		vio->u.fault.ft_flags_valid = false;
 
 		/* May call ll_readpage() */
-		ll_cl_add(vma->vm_file, env, io);
+		ll_cl_add(vma->vm_file, env, io, LCC_MMAP);
 
 		result = cl_io_loop(env, io);
 
@@ -326,6 +348,7 @@ static vm_fault_t ll_fault0(struct vm_area_struct *vma, struct vm_fault *vmf)
 
 static vm_fault_t ll_fault(struct vm_fault *vmf)
 {
+	struct vm_area_struct *vma = vmf->vma;
 	int count = 0;
 	bool printed = false;
 	vm_fault_t result;
@@ -338,10 +361,12 @@ static vm_fault_t ll_fault(struct vm_fault *vmf)
 	siginitsetinv(&new, sigmask(SIGKILL) | sigmask(SIGTERM));
 	sigprocmask(SIG_BLOCK, &new, &old);
 
+	ll_stats_ops_tally(ll_i2sbi(file_inode(vma->vm_file)),
+			   LPROC_LL_FAULT, 1);
+
 restart:
 	result = ll_fault0(vmf->vma, vmf);
-	LASSERT(!(result & VM_FAULT_LOCKED));
-	if (result == 0) {
+	if (!(result & (VM_FAULT_RETRY | VM_FAULT_ERROR | VM_FAULT_LOCKED))) {
 		struct page *vmpage = vmf->page;
 
 		/* check if this page has been truncated */
@@ -375,6 +400,9 @@ static vm_fault_t ll_page_mkwrite(struct vm_fault *vmf)
 	int err;
 	vm_fault_t ret;
 
+	ll_stats_ops_tally(ll_i2sbi(file_inode(vma->vm_file)),
+			   LPROC_LL_MKWRITE, 1);
+
 	file_update_time(vma->vm_file);
 	do {
 		retry = false;
diff --git a/drivers/staging/lustre/lustre/llite/lproc_llite.c b/drivers/staging/lustre/lustre/llite/lproc_llite.c
index 49bf1b7..9dcbe64 100644
--- a/drivers/staging/lustre/lustre/llite/lproc_llite.c
+++ b/drivers/staging/lustre/lustre/llite/lproc_llite.c
@@ -872,6 +872,41 @@ static ssize_t xattr_cache_store(struct kobject *kobj,
 }
 LUSTRE_RW_ATTR(xattr_cache);
 
+static ssize_t fast_read_show(struct kobject *kobj,
+			      struct attribute *attr,
+			      char *buf)
+{
+	struct ll_sb_info *sbi = container_of(kobj, struct ll_sb_info,
+					      ll_kobj);
+
+	return sprintf(buf, "%u\n", !!(sbi->ll_flags & LL_SBI_FAST_READ));
+}
+
+static ssize_t fast_read_store(struct kobject *kobj,
+			       struct attribute *attr,
+			       const char *buffer,
+			       size_t count)
+{
+	struct ll_sb_info *sbi = container_of(kobj, struct ll_sb_info,
+					      ll_kobj);
+	bool val;
+	int rc;
+
+	rc = kstrtobool(buffer, &val);
+	if (rc)
+		return rc;
+
+	spin_lock(&sbi->ll_lock);
+	if (val)
+		sbi->ll_flags |= LL_SBI_FAST_READ;
+	else
+		sbi->ll_flags &= ~LL_SBI_FAST_READ;
+	spin_unlock(&sbi->ll_lock);
+
+	return count;
+}
+LUSTRE_RW_ATTR(fast_read);
+
 static int ll_unstable_stats_seq_show(struct seq_file *m, void *v)
 {
 	struct super_block     *sb    = m->private;
@@ -1032,6 +1067,7 @@ static ssize_t ll_nosquash_nids_seq_write(struct file *file,
 	&lustre_attr_max_easize.attr,
 	&lustre_attr_default_easize.attr,
 	&lustre_attr_xattr_cache.attr,
+	&lustre_attr_fast_read.attr,
 	NULL,
 };
 
@@ -1068,6 +1104,8 @@ static void llite_sb_release(struct kobject *kobj)
 	{ LPROC_LL_OPEN,	   LPROCFS_TYPE_REGS, "open" },
 	{ LPROC_LL_RELEASE,	LPROCFS_TYPE_REGS, "close" },
 	{ LPROC_LL_MAP,	    LPROCFS_TYPE_REGS, "mmap" },
+	{ LPROC_LL_FAULT,		LPROCFS_TYPE_REGS, "page_fault" },
+	{ LPROC_LL_MKWRITE,		LPROCFS_TYPE_REGS, "page_mkwrite" },
 	{ LPROC_LL_LLSEEK,	 LPROCFS_TYPE_REGS, "seek" },
 	{ LPROC_LL_FSYNC,	  LPROCFS_TYPE_REGS, "fsync" },
 	{ LPROC_LL_READDIR,	LPROCFS_TYPE_REGS, "readdir" },
diff --git a/drivers/staging/lustre/lustre/llite/rw.c b/drivers/staging/lustre/lustre/llite/rw.c
index 3e008ce..59747da 100644
--- a/drivers/staging/lustre/lustre/llite/rw.c
+++ b/drivers/staging/lustre/lustre/llite/rw.c
@@ -1067,7 +1067,8 @@ struct ll_cl_context *ll_cl_find(struct file *file)
 	return found;
 }
 
-void ll_cl_add(struct file *file, const struct lu_env *env, struct cl_io *io)
+void ll_cl_add(struct file *file, const struct lu_env *env, struct cl_io *io,
+	       enum lcc_type type)
 {
 	struct ll_file_data *fd = LUSTRE_FPRIVATE(file);
 	struct ll_cl_context *lcc = &ll_env_info(env)->lti_io_ctx;
@@ -1077,6 +1078,7 @@ void ll_cl_add(struct file *file, const struct lu_env *env, struct cl_io *io)
 	lcc->lcc_cookie = current;
 	lcc->lcc_env = env;
 	lcc->lcc_io = io;
+	lcc->lcc_type = type;
 
 	write_lock(&fd->fd_lock);
 	list_add(&lcc->lcc_list, &fd->fd_lccs);
@@ -1094,10 +1096,10 @@ void ll_cl_remove(struct file *file, const struct lu_env *env)
 }
 
 static int ll_io_read_page(const struct lu_env *env, struct cl_io *io,
-			   struct cl_page *page)
+			   struct cl_page *page, struct file *file)
 {
 	struct inode *inode = vvp_object_inode(page->cp_obj);
-	struct ll_file_data *fd = vvp_env_io(env)->vui_fd;
+	struct ll_file_data *fd = LUSTRE_FPRIVATE(file);
 	struct ll_readahead_state *ras = &fd->fd_ras;
 	struct cl_2queue *queue  = &io->ci_queue;
 	struct ll_sb_info *sbi = ll_i2sbi(inode);
@@ -1109,7 +1111,8 @@ static int ll_io_read_page(const struct lu_env *env, struct cl_io *io,
 	uptodate = vpg->vpg_defer_uptodate;
 
 	if (sbi->ll_ra_info.ra_max_pages_per_file > 0 &&
-	    sbi->ll_ra_info.ra_max_pages > 0) {
+	    sbi->ll_ra_info.ra_max_pages > 0 &&
+	    !vpg->vpg_ra_updated) {
 		struct vvp_io *vio = vvp_env_io(env);
 		enum ras_update_flags flags = 0;
 
@@ -1168,13 +1171,66 @@ int ll_readpage(struct file *file, struct page *vmpage)
 
 	env = lcc->lcc_env;
 	io = lcc->lcc_io;
-	LASSERT(io->ci_state == CIS_IO_GOING);
+	if (!io) { /* fast read */
+		struct ll_file_data *fd = LUSTRE_FPRIVATE(file);
+		struct ll_readahead_state *ras = &fd->fd_ras;
+		struct inode *inode = file_inode(file);
+		struct vvp_page *vpg;
+
+		result = -ENODATA;
+
+		/*
+		 * TODO: need to verify the layout version to make sure
+		 * the page is not invalid due to layout change.
+		 */
+		page = cl_vmpage_page(vmpage, clob);
+		if (!page) {
+			unlock_page(vmpage);
+			return result;
+		}
+
+		vpg = cl2vvp_page(cl_object_page_slice(page->cp_obj, page));
+		if (vpg->vpg_defer_uptodate) {
+			enum ras_update_flags flags = LL_RAS_HIT;
+
+			if (lcc->lcc_type == LCC_MMAP)
+				flags |= LL_RAS_MMAP;
+
+			/*
+			 * For fast read, it updates read ahead state only
+			 * if the page is hit in cache because non cache page
+			 * case will be handled by slow read later.
+			 */
+			ras_update(ll_i2sbi(inode), inode, ras, vvp_index(vpg),
+				   flags);
+			/* avoid duplicate ras_update() call */
+			vpg->vpg_ra_updated = 1;
+
+			/*
+			 * Check if we can issue a readahead RPC, if that is
+			 * the case, we can't do fast IO because we will need
+			 * a cl_io to issue the RPC.
+			 */
+			if (ras->ras_window_start + ras->ras_window_len <
+			    ras->ras_next_readahead + PTLRPC_MAX_BRW_PAGES) {
+				/* export the page and skip io stack */
+				vpg->vpg_ra_used = 1;
+				cl_page_export(env, page, 1);
+				result = 0;
+			}
+		}
+
+		unlock_page(vmpage);
+		cl_page_put(env, page);
+		return result;
+	}
+
 	page = cl_page_find(env, clob, vmpage->index, vmpage, CPT_CACHEABLE);
 	if (!IS_ERR(page)) {
 		LASSERT(page->cp_type == CPT_CACHEABLE);
 		if (likely(!PageUptodate(vmpage))) {
 			cl_page_assume(env, io, page);
-			result = ll_io_read_page(env, io, page);
+			result = ll_io_read_page(env, io, page, file);
 		} else {
 			/* Page from a non-object file. */
 			unlock_page(vmpage);
diff --git a/drivers/staging/lustre/lustre/llite/vvp_internal.h b/drivers/staging/lustre/lustre/llite/vvp_internal.h
index 7d3abb4..70d62bf 100644
--- a/drivers/staging/lustre/lustre/llite/vvp_internal.h
+++ b/drivers/staging/lustre/lustre/llite/vvp_internal.h
@@ -225,6 +225,7 @@ struct vvp_object {
 struct vvp_page {
 	struct cl_page_slice vpg_cl;
 	unsigned int	vpg_defer_uptodate:1,
+			vpg_ra_updated:1,
 			vpg_ra_used:1;
 	/** VM page */
 	struct page	  *vpg_page;
-- 
1.8.3.1


From jsimmons at infradead.org  Mon Jul  2 23:24:19 2018
From: jsimmons at infradead.org (James Simmons)
Date: Mon,  2 Jul 2018 19:24:19 -0400
Subject: [lustre-devel] [PATCH 02/18] lustre: ladvise: Add feature of giving
 file access advices
In-Reply-To: <1530573875-20465-1-git-send-email-jsimmons@infradead.org>
References: <1530573875-20465-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1530573875-20465-3-git-send-email-jsimmons@infradead.org>

From: Li Xi <lixi at ddn.com>

The fadvise() system call provided by Linux kernel enables
applications to give advices or hints about how a file will be
accessed. However, It is only a client side mechanism which is
not enough for distributed file systems like Lustre, because in
order to tune system-wide cache or read-ahead policies, servers
need to understand the advices too.

This patch adds a new feature named ladvise which provides new
APIs and utils to give advices about the access pattern of Lustre
files with the purpose of performance improvement. It is similar
to Linux fadvise() system call, except it forwards the advices
directly from Lustre client to server. The server side codes will
apply appropriate read-ahead and caching techniques for the
corresponding files.

A typical workload for ladvise is e.g. a bunch of different
clients are doing small random reads of a file, so prefetching
pages into OSS cache with big linear reads before the random IO
is a net benefit. Fetching all that data into each client cache
with fadvise() may not be, due to much more data being sent to
the client.

Signed-off-by: Li Xi <lixi at ddn.com>
WC-bug-id: https://jira.whamcloud.com/browse/LU-4931
Reviewed-on: http://review.whamcloud.com/10029
Reviewed-by: Wang Shilong <wshilong at ddn.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong at gmail.com>
Reviewed-by: Andreas Dilger <adilger at whamcloud.com>
Reviewed-by: Oleg Drokin <green at whamcloud.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 .../lustre/include/uapi/linux/lustre/lustre_idl.h  |   3 +-
 .../lustre/include/uapi/linux/lustre/lustre_user.h |  32 +++++++
 drivers/staging/lustre/lustre/include/cl_object.h  |  13 +++
 .../lustre/lustre/include/lustre_req_layout.h      |   4 +
 .../staging/lustre/lustre/include/lustre_swab.h    |   2 +
 drivers/staging/lustre/lustre/llite/file.c         | 106 +++++++++++++++++++++
 drivers/staging/lustre/lustre/llite/vvp_io.c       |   3 +
 drivers/staging/lustre/lustre/lov/lov_io.c         |  28 ++++++
 drivers/staging/lustre/lustre/obdclass/cl_io.c     |   2 +
 .../staging/lustre/lustre/osc/osc_cl_internal.h    |   1 +
 drivers/staging/lustre/lustre/osc/osc_dev.c        |   1 +
 drivers/staging/lustre/lustre/osc/osc_internal.h   |   4 +
 drivers/staging/lustre/lustre/osc/osc_io.c         |  79 +++++++++++++++
 drivers/staging/lustre/lustre/osc/osc_request.c    |  94 ++++++++++++++++++
 drivers/staging/lustre/lustre/ptlrpc/layout.c      |  24 +++++
 .../staging/lustre/lustre/ptlrpc/lproc_ptlrpc.c    |   1 +
 .../staging/lustre/lustre/ptlrpc/pack_generic.c    |  19 ++++
 drivers/staging/lustre/lustre/ptlrpc/wiretest.c    |  54 ++++++++++-
 18 files changed, 468 insertions(+), 2 deletions(-)

diff --git a/drivers/staging/lustre/include/uapi/linux/lustre/lustre_idl.h b/drivers/staging/lustre/include/uapi/linux/lustre/lustre_idl.h
index 3d77ed6..5fab107 100644
--- a/drivers/staging/lustre/include/uapi/linux/lustre/lustre_idl.h
+++ b/drivers/staging/lustre/include/uapi/linux/lustre/lustre_idl.h
@@ -812,7 +812,8 @@ enum ost_cmd {
 	OST_QUOTACHECK = 18, /* not used since 2.4 */
 	OST_QUOTACTL   = 19,
 	OST_QUOTA_ADJUST_QUNIT = 20, /* not used since 2.4 */
-	OST_LAST_OPC
+	OST_LADVISE		= 21,
+	OST_LAST_OPC /* must be < 33 to avoid MDS_GETATTR */
 };
 #define OST_FIRST_OPC  OST_REPLY
 
diff --git a/drivers/staging/lustre/include/uapi/linux/lustre/lustre_user.h b/drivers/staging/lustre/include/uapi/linux/lustre/lustre_user.h
index 69387f3..fc33a43 100644
--- a/drivers/staging/lustre/include/uapi/linux/lustre/lustre_user.h
+++ b/drivers/staging/lustre/include/uapi/linux/lustre/lustre_user.h
@@ -276,6 +276,7 @@ struct ost_id {
 #define LL_IOC_MIGRATE			_IOR('f', 247, int)
 #define LL_IOC_FID2MDTIDX		_IOWR('f', 248, struct lu_fid)
 #define LL_IOC_GETPARENT		_IOWR('f', 249, struct getparent)
+#define LL_IOC_LADVISE			_IOR('f', 250, struct lu_ladvise)
 
 /* Lease types for use as arg and return of LL_IOC_{GET,SET}_LEASE ioctl. */
 enum ll_lease_type {
@@ -1322,6 +1323,37 @@ struct hsm_copy {
 	struct hsm_action_item	hc_hai;
 };
 
+enum lu_ladvise_type {
+	LU_LADVISE_INVALID	= 0,
+};
+
+#define LU_LADVISE_NAMES { }
+
+struct lu_ladvise {
+	__u64			lla_advice;
+	__u64			lla_start;
+	__u64			lla_end;
+	__u64			lla_padding;
+};
+
+enum ladvise_flag {
+	LF_ASYNC	= 0x00000001,
+};
+
+#define LADVISE_MAGIC 0x1ADF1CE0
+#define LF_MASK LF_ASYNC
+
+struct ladvise_hdr {
+	__u32			lah_magic;	/* LADVISE_MAGIC */
+	__u32			lah_count;	/* number of advices */
+	__u64			lah_flags;	/* from enum ladvise_flag */
+	__u64			lah_padding1;	/* unused */
+	__u64			lah_padding2;	/* unused */
+	struct lu_ladvise	lah_advise[0];
+};
+
+#define LAH_COUNT_MAX		1024
+
 /** @} lustreuser */
 
 #endif /* _LUSTRE_USER_H */
diff --git a/drivers/staging/lustre/lustre/include/cl_object.h b/drivers/staging/lustre/lustre/include/cl_object.h
index 1491beb..58af22e 100644
--- a/drivers/staging/lustre/lustre/include/cl_object.h
+++ b/drivers/staging/lustre/lustre/include/cl_object.h
@@ -1394,6 +1394,11 @@ enum cl_io_type {
 	 * cl_io_loop() is never called for it.
 	 */
 	CIT_MISC,
+	/**
+	 * ladvise handling
+	 * To give advice about access of a file
+	 */
+	CIT_LADVISE,
 	CIT_OP_NR
 };
 
@@ -1804,6 +1809,14 @@ struct cl_io {
 			/* how many pages were written/discarded */
 			unsigned int       fi_nr_written;
 		} ci_fsync;
+		struct cl_ladvise_io {
+			u64			li_start;
+			u64			li_end;
+			/** file system level fid */
+			struct lu_fid	       *li_fid;
+			enum lu_ladvise_type	li_advice;
+			u64			li_flags;
+		} ci_ladvise;
 	} u;
 	struct cl_2queue     ci_queue;
 	size_t	       ci_nob;
diff --git a/drivers/staging/lustre/lustre/include/lustre_req_layout.h b/drivers/staging/lustre/lustre/include/lustre_req_layout.h
index 213d0a0..db6d8ed 100644
--- a/drivers/staging/lustre/lustre/include/lustre_req_layout.h
+++ b/drivers/staging/lustre/lustre/include/lustre_req_layout.h
@@ -190,6 +190,7 @@ void req_capsule_shrink(struct req_capsule *pill,
 extern struct req_format RQF_OST_GET_INFO_LAST_FID;
 extern struct req_format RQF_OST_SET_INFO_LAST_FID;
 extern struct req_format RQF_OST_GET_INFO_FIEMAP;
+extern struct req_format RQF_OST_LADVISE;
 
 /* LDLM req_format */
 extern struct req_format RQF_LDLM_ENQUEUE;
@@ -299,6 +300,9 @@ void req_capsule_shrink(struct req_capsule *pill,
 extern struct req_msg_field RMF_MGS_CONFIG_BODY;
 extern struct req_msg_field RMF_MGS_CONFIG_RES;
 
+extern struct req_msg_field RMF_OST_LADVISE_HDR;
+extern struct req_msg_field RMF_OST_LADVISE;
+
 /* generic uint32 */
 extern struct req_msg_field RMF_U32;
 
diff --git a/drivers/staging/lustre/lustre/include/lustre_swab.h b/drivers/staging/lustre/lustre/include/lustre_swab.h
index 9d786bb..e09a3dc 100644
--- a/drivers/staging/lustre/lustre/include/lustre_swab.h
+++ b/drivers/staging/lustre/lustre/include/lustre_swab.h
@@ -99,6 +99,8 @@ void lustre_swab_lov_user_md_objects(struct lov_user_ost_data *lod,
 void lustre_swab_swap_layouts(struct mdc_swap_layouts *msl);
 void lustre_swab_close_data(struct close_data *data);
 void lustre_swab_lmv_user_md(struct lmv_user_md *lum);
+void lustre_swab_ladvise(struct lu_ladvise *ladvise);
+void lustre_swab_ladvise_hdr(struct ladvise_hdr *ladvise_hdr);
 
 /* Functions for dumping PTLRPC fields */
 void dump_rniobuf(struct niobuf_remote *rnb);
diff --git a/drivers/staging/lustre/lustre/llite/file.c b/drivers/staging/lustre/lustre/llite/file.c
index 59b5fbc..44bec1d 100644
--- a/drivers/staging/lustre/lustre/llite/file.c
+++ b/drivers/staging/lustre/lustre/llite/file.c
@@ -1926,6 +1926,54 @@ static inline long ll_lease_type_from_fmode(fmode_t fmode)
 	       ((fmode & FMODE_WRITE) ? LL_LEASE_WRLCK : 0);
 }
 
+/*
+ * Give file access advices
+ *
+ * The ladvise interface is similar to Linux fadvise() system call, except it
+ * forwards the advices directly from Lustre client to server. The server side
+ * codes will apply appropriate read-ahead and caching techniques for the
+ * corresponding files.
+ *
+ * A typical workload for ladvise is e.g. a bunch of different clients are
+ * doing small random reads of a file, so prefetching pages into OSS cache
+ * with big linear reads before the random IO is a net benefit. Fetching
+ * all that data into each client cache with fadvise() may not be, due to
+ * much more data being sent to the client.
+ */
+static int ll_ladvise(struct inode *inode, struct file *file, __u64 flags,
+		      struct lu_ladvise *ladvise)
+{
+	struct cl_ladvise_io *lio;
+	struct lu_env *env;
+	struct cl_io *io;
+	u16 refcheck;
+	int rc;
+
+	env = cl_env_get(&refcheck);
+	if (IS_ERR(env))
+		return PTR_ERR(env);
+
+	io = vvp_env_thread_io(env);
+	io->ci_obj = ll_i2info(inode)->lli_clob;
+
+	/* initialize parameters for ladvise */
+	lio = &io->u.ci_ladvise;
+	lio->li_start = ladvise->lla_start;
+	lio->li_end = ladvise->lla_end;
+	lio->li_fid = ll_inode2fid(inode);
+	lio->li_advice = ladvise->lla_advice;
+	lio->li_flags = flags;
+
+	if (!cl_io_init(env, io, CIT_LADVISE, io->ci_obj))
+		rc = cl_io_loop(env, io);
+	else
+		rc = io->ci_result;
+
+	cl_io_fini(env, io);
+	cl_env_put(env, &refcheck);
+	return rc;
+}
+
 static long
 ll_file_ioctl(struct file *file, unsigned int cmd, unsigned long arg)
 {
@@ -2248,6 +2296,64 @@ static inline long ll_lease_type_from_fmode(fmode_t fmode)
 		kfree(hui);
 		return rc;
 	}
+	case LL_IOC_LADVISE: {
+		struct ladvise_hdr *ladvise_hdr;
+		int alloc_size = sizeof(*ladvise_hdr);
+		int num_advise;
+		int i;
+
+		rc = 0;
+		ladvise_hdr = kzalloc(alloc_size, GFP_NOFS);
+		if (!ladvise_hdr)
+			return -ENOMEM;
+
+		if (copy_from_user(ladvise_hdr,
+				   (const struct ladvise_hdr __user *)arg,
+				   alloc_size)) {
+			rc = -EFAULT;
+			goto out_ladvise;
+		}
+
+		if (ladvise_hdr->lah_magic != LADVISE_MAGIC ||
+		    ladvise_hdr->lah_count < 1) {
+			rc = -EINVAL;
+			goto out_ladvise;
+		}
+
+		num_advise = ladvise_hdr->lah_count;
+		if (num_advise >= LAH_COUNT_MAX) {
+			rc = -EFBIG;
+			goto out_ladvise;
+		}
+
+		kfree(ladvise_hdr);
+		alloc_size = offsetof(typeof(*ladvise_hdr),
+				      lah_advise[num_advise]);
+		ladvise_hdr = kzalloc(alloc_size, GFP_NOFS);
+		if (!ladvise_hdr)
+			return -ENOMEM;
+
+		/*
+		 * TODO: submit multiple advices to one server in a single RPC
+		 */
+		if (copy_from_user(ladvise_hdr,
+				   (const struct ladvise_hdr __user *)arg,
+				   alloc_size)) {
+			rc = -EFAULT;
+			goto out_ladvise;
+		}
+
+		for (i = 0; i < num_advise; i++) {
+			rc = ll_ladvise(inode, file, ladvise_hdr->lah_flags,
+					&ladvise_hdr->lah_advise[i]);
+			if (rc)
+				break;
+		}
+
+out_ladvise:
+		kfree(ladvise_hdr);
+		return rc;
+	}
 	default: {
 		int err;
 
diff --git a/drivers/staging/lustre/lustre/llite/vvp_io.c b/drivers/staging/lustre/lustre/llite/vvp_io.c
index df47ed9..70d2387 100644
--- a/drivers/staging/lustre/lustre/llite/vvp_io.c
+++ b/drivers/staging/lustre/lustre/llite/vvp_io.c
@@ -1302,6 +1302,9 @@ static int vvp_io_read_ahead(const struct lu_env *env,
 		},
 		[CIT_MISC] = {
 			.cio_fini   = vvp_io_fini
+		},
+		[CIT_LADVISE] = {
+			.cio_fini	= vvp_io_fini
 		}
 	},
 	.cio_read_ahead	= vvp_io_read_ahead,
diff --git a/drivers/staging/lustre/lustre/lov/lov_io.c b/drivers/staging/lustre/lustre/lov/lov_io.c
index 5098284..6537ba3 100644
--- a/drivers/staging/lustre/lustre/lov/lov_io.c
+++ b/drivers/staging/lustre/lustre/lov/lov_io.c
@@ -123,6 +123,14 @@ static void lov_io_sub_inherit(struct cl_io *io, struct lov_io *lio,
 		}
 		break;
 	}
+	case CIT_LADVISE: {
+		io->u.ci_ladvise.li_start = start;
+		io->u.ci_ladvise.li_end = end;
+		io->u.ci_ladvise.li_fid = parent->u.ci_ladvise.li_fid;
+		io->u.ci_ladvise.li_advice = parent->u.ci_ladvise.li_advice;
+		io->u.ci_ladvise.li_flags = parent->u.ci_ladvise.li_flags;
+		break;
+	}
 	default:
 		break;
 	}
@@ -315,6 +323,12 @@ static int lov_io_slice_init(struct lov_io *lio, struct lov_object *obj,
 		break;
 	}
 
+	case CIT_LADVISE: {
+		lio->lis_pos = io->u.ci_ladvise.li_start;
+		lio->lis_endpos = io->u.ci_ladvise.li_end;
+		break;
+	}
+
 	case CIT_MISC:
 		lio->lis_pos = 0;
 		lio->lis_endpos = OBD_OBJECT_EOF;
@@ -837,6 +851,15 @@ static void lov_io_fsync_end(const struct lu_env *env,
 			.cio_start     = lov_io_start,
 			.cio_end       = lov_io_fsync_end
 		},
+		[CIT_LADVISE] = {
+			.cio_fini	= lov_io_fini,
+			.cio_iter_init	= lov_io_iter_init,
+			.cio_iter_fini	= lov_io_iter_fini,
+			.cio_lock	= lov_io_lock,
+			.cio_unlock	= lov_io_unlock,
+			.cio_start	= lov_io_start,
+			.cio_end	= lov_io_end
+		},
 		[CIT_MISC] = {
 			.cio_fini   = lov_io_fini
 		}
@@ -908,6 +931,9 @@ static void lov_empty_impossible(const struct lu_env *env,
 		[CIT_FSYNC] = {
 			.cio_fini   = lov_empty_io_fini
 		},
+		[CIT_LADVISE] = {
+			.cio_fini	= lov_empty_io_fini
+		},
 		[CIT_MISC] = {
 			.cio_fini   = lov_empty_io_fini
 		}
@@ -950,6 +976,7 @@ int lov_io_init_empty(const struct lu_env *env, struct cl_object *obj,
 		result = 0;
 		break;
 	case CIT_FSYNC:
+	case CIT_LADVISE:
 	case CIT_SETATTR:
 	case CIT_DATA_VERSION:
 		result = 1;
@@ -989,6 +1016,7 @@ int lov_io_init_released(const struct lu_env *env, struct cl_object *obj,
 		break;
 	case CIT_MISC:
 	case CIT_FSYNC:
+	case CIT_LADVISE:
 	case CIT_DATA_VERSION:
 		result = 1;
 		break;
diff --git a/drivers/staging/lustre/lustre/obdclass/cl_io.c b/drivers/staging/lustre/lustre/obdclass/cl_io.c
index fcdae60..2c77e72 100644
--- a/drivers/staging/lustre/lustre/obdclass/cl_io.c
+++ b/drivers/staging/lustre/lustre/obdclass/cl_io.c
@@ -140,6 +140,8 @@ void cl_io_fini(const struct lu_env *env, struct cl_io *io)
 		LASSERT(ergo(io->ci_ignore_layout || !io->ci_verify_layout,
 			     !io->ci_need_restart));
 		break;
+	case CIT_LADVISE:
+		break;
 	default:
 		LBUG();
 	}
diff --git a/drivers/staging/lustre/lustre/osc/osc_cl_internal.h b/drivers/staging/lustre/lustre/osc/osc_cl_internal.h
index 2d3cba1..d86d3f7 100644
--- a/drivers/staging/lustre/lustre/osc/osc_cl_internal.h
+++ b/drivers/staging/lustre/lustre/osc/osc_cl_internal.h
@@ -110,6 +110,7 @@ struct osc_thread_info {
 	pgoff_t			oti_fn_index; /* first non-overlapped index */
 	struct cl_sync_io	oti_anchor;
 	struct cl_req_attr	oti_req_attr;
+	struct lu_buf		oti_ladvise_buf;
 };
 
 struct osc_object {
diff --git a/drivers/staging/lustre/lustre/osc/osc_dev.c b/drivers/staging/lustre/lustre/osc/osc_dev.c
index 2b5f324..c767a3c 100644
--- a/drivers/staging/lustre/lustre/osc/osc_dev.c
+++ b/drivers/staging/lustre/lustre/osc/osc_dev.c
@@ -122,6 +122,7 @@ static void osc_key_fini(const struct lu_context *ctx,
 {
 	struct osc_thread_info *info = data;
 
+	kvfree(info->oti_ladvise_buf.lb_buf);
 	kmem_cache_free(osc_thread_kmem, info);
 }
 
diff --git a/drivers/staging/lustre/lustre/osc/osc_internal.h b/drivers/staging/lustre/lustre/osc/osc_internal.h
index 4ddba13..02e8318 100644
--- a/drivers/staging/lustre/lustre/osc/osc_internal.h
+++ b/drivers/staging/lustre/lustre/osc/osc_internal.h
@@ -129,6 +129,10 @@ int osc_sync_base(struct osc_object *exp, struct obdo *oa,
 		  obd_enqueue_update_f upcall, void *cookie,
 		  struct ptlrpc_request_set *rqset);
 
+int osc_ladvise_base(struct obd_export *exp, struct obdo *oa,
+		     struct ladvise_hdr *ladvise_hdr,
+		     obd_enqueue_update_f upcall, void *cookie,
+		     struct ptlrpc_request_set *rqset);
 int osc_process_config_base(struct obd_device *obd, struct lustre_cfg *cfg);
 int osc_build_rpc(const struct lu_env *env, struct client_obd *cli,
 		  struct list_head *ext_list, int cmd);
diff --git a/drivers/staging/lustre/lustre/osc/osc_io.c b/drivers/staging/lustre/lustre/osc/osc_io.c
index 955525f..628743b 100644
--- a/drivers/staging/lustre/lustre/osc/osc_io.c
+++ b/drivers/staging/lustre/lustre/osc/osc_io.c
@@ -843,6 +843,80 @@ static void osc_io_fsync_end(const struct lu_env *env,
 	slice->cis_io->ci_result = result;
 }
 
+static int osc_io_ladvise_start(const struct lu_env *env,
+				const struct cl_io_slice *slice)
+{
+	struct cl_io *io = slice->cis_io;
+	struct osc_io *oio = cl2osc_io(env, slice);
+	struct cl_object *obj = slice->cis_obj;
+	struct lov_oinfo *loi = cl2osc(obj)->oo_oinfo;
+	struct cl_ladvise_io *lio = &io->u.ci_ladvise;
+	struct obdo *oa = &oio->oi_oa;
+	struct osc_async_cbargs *cbargs = &oio->oi_cbarg;
+	struct ladvise_hdr *ladvise_hdr;
+	struct lu_ladvise *ladvise;
+	int num_advise = 1;
+	int result = 0;
+	int buf_size;
+
+	/* TODO: add multiple ladvise support in CLIO */
+	buf_size = offsetof(typeof(*ladvise_hdr), lah_advise[num_advise]);
+	if (osc_env_info(env)->oti_ladvise_buf.lb_len < buf_size) {
+		kvfree(osc_env_info(env)->oti_ladvise_buf.lb_buf);
+		osc_env_info(env)->oti_ladvise_buf.lb_buf = kvzalloc(buf_size,
+								     GFP_NOFS);
+		if (likely(osc_env_info(env)->oti_ladvise_buf.lb_buf))
+			osc_env_info(env)->oti_ladvise_buf.lb_len = buf_size;
+	}
+
+	ladvise_hdr = osc_env_info(env)->oti_ladvise_buf.lb_buf;
+	if (!ladvise_hdr)
+		return -ENOMEM;
+
+	memset(ladvise_hdr, 0, buf_size);
+	ladvise_hdr->lah_magic = LADVISE_MAGIC;
+	ladvise_hdr->lah_count = num_advise;
+	ladvise_hdr->lah_flags = lio->li_flags;
+
+	memset(oa, 0, sizeof(*oa));
+	oa->o_oi = loi->loi_oi;
+	oa->o_valid = OBD_MD_FLID;
+	obdo_set_parent_fid(oa, lio->li_fid);
+
+	ladvise = ladvise_hdr->lah_advise;
+	ladvise->lla_start = lio->li_start;
+	ladvise->lla_end = lio->li_end;
+	ladvise->lla_advice = lio->li_advice;
+
+	if (lio->li_flags & LF_ASYNC) {
+		result = osc_ladvise_base(osc_export(cl2osc(obj)), oa,
+					  ladvise_hdr, NULL, NULL, NULL);
+	} else {
+		init_completion(&cbargs->opc_sync);
+		result = osc_ladvise_base(osc_export(cl2osc(obj)), oa,
+					  ladvise_hdr, osc_async_upcall,
+					  cbargs, PTLRPCD_SET);
+		cbargs->opc_rpc_sent = !result;
+	}
+	return result;
+}
+
+static void osc_io_ladvise_end(const struct lu_env *env,
+			       const struct cl_io_slice *slice)
+{
+	struct cl_io *io = slice->cis_io;
+	struct osc_io *oio = cl2osc_io(env, slice);
+	struct osc_async_cbargs *cbargs = &oio->oi_cbarg;
+	struct cl_ladvise_io *lio = &io->u.ci_ladvise;
+	int result = 0;
+
+	if ((!(lio->li_flags & LF_ASYNC)) && cbargs->opc_rpc_sent) {
+		wait_for_completion(&cbargs->opc_sync);
+		result = cbargs->opc_rc;
+	}
+	slice->cis_io->ci_result = result;
+}
+
 static void osc_io_end(const struct lu_env *env,
 		       const struct cl_io_slice *slice)
 {
@@ -891,6 +965,11 @@ static void osc_io_end(const struct lu_env *env,
 			.cio_end    = osc_io_fsync_end,
 			.cio_fini   = osc_io_fini
 		},
+		[CIT_LADVISE] = {
+			.cio_start	= osc_io_ladvise_start,
+			.cio_end	= osc_io_ladvise_end,
+			.cio_fini	= osc_io_fini
+		},
 		[CIT_MISC] = {
 			.cio_fini   = osc_io_fini
 		}
diff --git a/drivers/staging/lustre/lustre/osc/osc_request.c b/drivers/staging/lustre/lustre/osc/osc_request.c
index ce073b6..0286f25 100644
--- a/drivers/staging/lustre/lustre/osc/osc_request.c
+++ b/drivers/staging/lustre/lustre/osc/osc_request.c
@@ -91,6 +91,12 @@ struct osc_fsync_args {
 	void		*fa_cookie;
 };
 
+struct osc_ladvise_args {
+	struct obdo		*la_oa;
+	obd_enqueue_update_f	 la_upcall;
+	void			*la_cookie;
+};
+
 struct osc_enqueue_args {
 	struct obd_export	*oa_exp;
 	enum ldlm_type		oa_type;
@@ -269,6 +275,94 @@ int osc_setattr_async(struct obd_export *exp, struct obdo *oa,
 	return 0;
 }
 
+static int osc_ladvise_interpret(const struct lu_env *env,
+				 struct ptlrpc_request *req,
+				 void *arg, int rc)
+{
+	struct osc_ladvise_args *la = arg;
+	struct ost_body *body;
+
+	if (rc)
+		goto out;
+
+	body = req_capsule_server_get(&req->rq_pill, &RMF_OST_BODY);
+	if (!body) {
+		rc = -EPROTO;
+		goto out;
+	}
+
+	*la->la_oa = body->oa;
+out:
+	rc = la->la_upcall(la->la_cookie, rc);
+	return rc;
+}
+
+/**
+ * If rqset is NULL, do not wait for response. Upcall and cookie could also
+ * be NULL in this case
+ */
+int osc_ladvise_base(struct obd_export *exp, struct obdo *oa,
+		     struct ladvise_hdr *ladvise_hdr,
+		     obd_enqueue_update_f upcall, void *cookie,
+		     struct ptlrpc_request_set *rqset)
+{
+	struct lu_ladvise *ladvise = ladvise_hdr->lah_advise;
+	int num_advise = ladvise_hdr->lah_count;
+	struct ladvise_hdr *req_ladvise_hdr;
+	struct lu_ladvise *req_ladvise;
+	struct osc_ladvise_args *la;
+	struct ptlrpc_request *req;
+	struct ost_body *body;
+	int rc;
+
+	req = ptlrpc_request_alloc(class_exp2cliimp(exp), &RQF_OST_LADVISE);
+	if (!req)
+		return -ENOMEM;
+
+	req_capsule_set_size(&req->rq_pill, &RMF_OST_LADVISE, RCL_CLIENT,
+			     num_advise * sizeof(*ladvise));
+	rc = ptlrpc_request_pack(req, LUSTRE_OST_VERSION, OST_LADVISE);
+	if (rc) {
+		ptlrpc_request_free(req);
+		return rc;
+	}
+	req->rq_request_portal = OST_IO_PORTAL;
+	ptlrpc_at_set_req_timeout(req);
+
+	body = req_capsule_client_get(&req->rq_pill, &RMF_OST_BODY);
+	LASSERT(body);
+	lustre_set_wire_obdo(&req->rq_import->imp_connect_data, &body->oa,
+			     oa);
+
+	req_ladvise_hdr = req_capsule_client_get(&req->rq_pill,
+						 &RMF_OST_LADVISE_HDR);
+	memcpy(req_ladvise_hdr, ladvise_hdr, sizeof(*ladvise_hdr));
+
+	req_ladvise = req_capsule_client_get(&req->rq_pill, &RMF_OST_LADVISE);
+	memcpy(req_ladvise, ladvise, sizeof(*ladvise) * num_advise);
+	ptlrpc_request_set_replen(req);
+
+	if (!rqset) {
+		/* Do not wait for response. */
+		ptlrpcd_add_req(req);
+		return 0;
+	}
+
+	req->rq_interpret_reply = osc_ladvise_interpret;
+	BUILD_BUG_ON(sizeof(*la) > sizeof(req->rq_async_args));
+	la = ptlrpc_req_async_args(req);
+	la->la_oa = oa;
+	la->la_upcall = upcall;
+	la->la_cookie = cookie;
+
+	if (rqset == PTLRPCD_SET)
+		ptlrpcd_add_req(req);
+	else
+		ptlrpc_set_add_req(rqset, req);
+
+	return 0;
+}
+
 static int osc_create(const struct lu_env *env, struct obd_export *exp,
 		      struct obdo *oa)
 {
diff --git a/drivers/staging/lustre/lustre/ptlrpc/layout.c b/drivers/staging/lustre/lustre/ptlrpc/layout.c
index 417d4a1..6ef8789 100644
--- a/drivers/staging/lustre/lustre/ptlrpc/layout.c
+++ b/drivers/staging/lustre/lustre/ptlrpc/layout.c
@@ -605,6 +605,13 @@
 	&RMF_FIEMAP_VAL
 };
 
+static const struct req_msg_field *ost_ladvise[] = {
+	&RMF_PTLRPC_BODY,
+	&RMF_OST_BODY,
+	&RMF_OST_LADVISE_HDR,
+	&RMF_OST_LADVISE,
+};
+
 static const struct req_msg_field *ost_get_fiemap_server[] = {
 	&RMF_PTLRPC_BODY,
 	&RMF_FIEMAP_VAL
@@ -716,6 +723,7 @@
 	&RQF_OST_GET_INFO_LAST_FID,
 	&RQF_OST_SET_INFO_LAST_FID,
 	&RQF_OST_GET_INFO_FIEMAP,
+	&RQF_OST_LADVISE,
 	&RQF_LDLM_ENQUEUE,
 	&RQF_LDLM_ENQUEUE_LVB,
 	&RQF_LDLM_CONVERT,
@@ -1110,6 +1118,18 @@ struct req_msg_field RMF_SWAP_LAYOUTS =
 	DEFINE_MSGF("swap_layouts", 0, sizeof(struct  mdc_swap_layouts),
 		    lustre_swab_swap_layouts, NULL);
 EXPORT_SYMBOL(RMF_SWAP_LAYOUTS);
+
+struct req_msg_field RMF_OST_LADVISE_HDR =
+	DEFINE_MSGF("ladvise_request", 0, sizeof(struct ladvise_hdr),
+		    lustre_swab_ladvise_hdr, NULL);
+EXPORT_SYMBOL(RMF_OST_LADVISE_HDR);
+
+struct req_msg_field RMF_OST_LADVISE =
+	DEFINE_MSGF("ladvise_request", RMF_F_STRUCT_ARRAY,
+		    sizeof(struct lu_ladvise),
+		    lustre_swab_ladvise, NULL);
+EXPORT_SYMBOL(RMF_OST_LADVISE);
+
 /*
  * Request formats.
  */
@@ -1552,6 +1572,10 @@ struct req_format RQF_OST_GET_INFO_FIEMAP =
 			ost_get_fiemap_server);
 EXPORT_SYMBOL(RQF_OST_GET_INFO_FIEMAP);
 
+struct req_format RQF_OST_LADVISE =
+	DEFINE_REQ_FMT0("OST_LADVISE", ost_ladvise, ost_body_only);
+EXPORT_SYMBOL(RQF_OST_LADVISE);
+
 /* Convenience macro */
 #define FMT_FIELD(fmt, i, j) ((fmt)->rf_fields[(i)].d[(j)])
 
diff --git a/drivers/staging/lustre/lustre/ptlrpc/lproc_ptlrpc.c b/drivers/staging/lustre/lustre/ptlrpc/lproc_ptlrpc.c
index a61e800..52b980c 100644
--- a/drivers/staging/lustre/lustre/ptlrpc/lproc_ptlrpc.c
+++ b/drivers/staging/lustre/lustre/ptlrpc/lproc_ptlrpc.c
@@ -66,6 +66,7 @@
 	{ OST_QUOTACHECK,   "ost_quotacheck" },
 	{ OST_QUOTACTL,     "ost_quotactl" },
 	{ OST_QUOTA_ADJUST_QUNIT, "ost_quota_adjust_qunit" },
+	{ OST_LADVISE,			"ost_ladvise" },
 	{ MDS_GETATTR,      "mds_getattr" },
 	{ MDS_GETATTR_NAME, "mds_getattr_lock" },
 	{ MDS_CLOSE,	"mds_close" },
diff --git a/drivers/staging/lustre/lustre/ptlrpc/pack_generic.c b/drivers/staging/lustre/lustre/ptlrpc/pack_generic.c
index 0337b33..468fa69 100644
--- a/drivers/staging/lustre/lustre/ptlrpc/pack_generic.c
+++ b/drivers/staging/lustre/lustre/ptlrpc/pack_generic.c
@@ -2309,3 +2309,22 @@ void lustre_swab_close_data(struct close_data *cd)
 	lustre_swab_lu_fid(&cd->cd_fid);
 	__swab64s(&cd->cd_data_version);
 }
+
+void lustre_swab_ladvise(struct lu_ladvise *ladvise)
+{
+	swab64s(&ladvise->lla_start);
+	swab64s(&ladvise->lla_end);
+	swab64s(&ladvise->lla_advice);
+	BUILD_BUG_ON(!offsetof(typeof(*ladvise), lla_padding));
+}
+EXPORT_SYMBOL(lustre_swab_ladvise);
+
+void lustre_swab_ladvise_hdr(struct ladvise_hdr *ladvise_hdr)
+{
+	swab32s(&ladvise_hdr->lah_magic);
+	swab32s(&ladvise_hdr->lah_count);
+	swab64s(&ladvise_hdr->lah_flags);
+	BUILD_BUG_ON(!offsetof(typeof(*ladvise_hdr), lah_padding1));
+	BUILD_BUG_ON(!offsetof(typeof(*ladvise_hdr), lah_padding2));
+}
+EXPORT_SYMBOL(lustre_swab_ladvise_hdr);
diff --git a/drivers/staging/lustre/lustre/ptlrpc/wiretest.c b/drivers/staging/lustre/lustre/ptlrpc/wiretest.c
index 2b3608c..5a68de5 100644
--- a/drivers/staging/lustre/lustre/ptlrpc/wiretest.c
+++ b/drivers/staging/lustre/lustre/ptlrpc/wiretest.c
@@ -101,7 +101,9 @@ void lustre_assert_wire_constants(void)
 		 (long long)OST_QUOTACTL);
 	LASSERTF(OST_QUOTA_ADJUST_QUNIT == 20, "found %lld\n",
 		 (long long)OST_QUOTA_ADJUST_QUNIT);
-	LASSERTF(OST_LAST_OPC == 21, "found %lld\n",
+	LASSERTF(OST_LADVISE == 21, "found %lld\n",
+		 (long long)OST_LADVISE);
+	LASSERTF(OST_LAST_OPC == 22, "found %lld\n",
 		 (long long)OST_LAST_OPC);
 	LASSERTF(OBD_OBJECT_EOF == 0xffffffffffffffffULL, "found 0x%.16llxULL\n",
 		 OBD_OBJECT_EOF);
@@ -4207,4 +4209,54 @@ void lustre_assert_wire_constants(void)
 	LASSERTF(sizeof(((struct hsm_user_import *)0)->hui_archive_id) == 4,
 		 "found %lld\n",
 	      (long long)sizeof(((struct hsm_user_import *)0)->hui_archive_id));
+
+	/* Checks for struct lu_ladvise */
+	LASSERTF((int)sizeof(struct lu_ladvise) == 32, "found %lld\n",
+		 (long long)(int)sizeof(struct lu_ladvise));
+	LASSERTF((int)offsetof(struct lu_ladvise, lla_advice) == 0, "found %lld\n",
+		 (long long)(int)offsetof(struct lu_ladvise, lla_advice));
+	LASSERTF((int)sizeof(((struct lu_ladvise *)0)->lla_advice) == 8, "found %lld\n",
+		 (long long)(int)sizeof(((struct lu_ladvise *)0)->lla_advice));
+	LASSERTF((int)offsetof(struct lu_ladvise, lla_start) == 8, "found %lld\n",
+		 (long long)(int)offsetof(struct lu_ladvise, lla_start));
+	LASSERTF((int)sizeof(((struct lu_ladvise *)0)->lla_start) == 8, "found %lld\n",
+		 (long long)(int)sizeof(((struct lu_ladvise *)0)->lla_start));
+	LASSERTF((int)offsetof(struct lu_ladvise, lla_end) == 16, "found %lld\n",
+		 (long long)(int)offsetof(struct lu_ladvise, lla_end));
+	LASSERTF((int)sizeof(((struct lu_ladvise *)0)->lla_end) == 8, "found %lld\n",
+		 (long long)(int)sizeof(((struct lu_ladvise *)0)->lla_end));
+	LASSERTF((int)offsetof(struct lu_ladvise, lla_padding) == 24, "found %lld\n",
+		 (long long)(int)offsetof(struct lu_ladvise, lla_padding));
+	LASSERTF((int)sizeof(((struct lu_ladvise *)0)->lla_padding) == 8, "found %lld\n",
+		 (long long)(int)sizeof(((struct lu_ladvise *)0)->lla_padding));
+
+	/* Checks for struct ladvise_hdr */
+	LASSERTF(LADVISE_MAGIC == 0x1ADF1CE0, "found 0x%.8x\n",
+		 LADVISE_MAGIC);
+	LASSERTF((int)sizeof(struct ladvise_hdr) == 32, "found %lld\n",
+		 (long long)(int)sizeof(struct ladvise_hdr));
+	LASSERTF((int)offsetof(struct ladvise_hdr, lah_magic) == 0, "found %lld\n",
+		 (long long)(int)offsetof(struct ladvise_hdr, lah_magic));
+	LASSERTF((int)sizeof(((struct ladvise_hdr *)0)->lah_magic) == 4, "found %lld\n",
+		 (long long)(int)sizeof(((struct ladvise_hdr *)0)->lah_magic));
+	LASSERTF((int)offsetof(struct ladvise_hdr, lah_count) == 4, "found %lld\n",
+		 (long long)(int)offsetof(struct ladvise_hdr, lah_count));
+	LASSERTF((int)sizeof(((struct ladvise_hdr *)0)->lah_count) == 4, "found %lld\n",
+		 (long long)(int)sizeof(((struct ladvise_hdr *)0)->lah_count));
+	LASSERTF((int)offsetof(struct ladvise_hdr, lah_flags) == 8, "found %lld\n",
+		 (long long)(int)offsetof(struct ladvise_hdr, lah_flags));
+	LASSERTF((int)sizeof(((struct ladvise_hdr *)0)->lah_flags) == 8, "found %lld\n",
+		 (long long)(int)sizeof(((struct ladvise_hdr *)0)->lah_flags));
+	LASSERTF((int)offsetof(struct ladvise_hdr, lah_padding1) == 16, "found %lld\n",
+		 (long long)(int)offsetof(struct ladvise_hdr, lah_padding1));
+	LASSERTF((int)sizeof(((struct ladvise_hdr *)0)->lah_padding1) == 8, "found %lld\n",
+		 (long long)(int)sizeof(((struct ladvise_hdr *)0)->lah_padding1));
+	LASSERTF((int)offsetof(struct ladvise_hdr, lah_padding2) == 24, "found %lld\n",
+		 (long long)(int)offsetof(struct ladvise_hdr, lah_padding2));
+	LASSERTF((int)sizeof(((struct ladvise_hdr *)0)->lah_padding2) == 8, "found %lld\n",
+		 (long long)(int)sizeof(((struct ladvise_hdr *)0)->lah_padding2));
+	LASSERTF((int)offsetof(struct ladvise_hdr, lah_advise) == 32, "found %lld\n",
+		 (long long)(int)offsetof(struct ladvise_hdr, lah_advise));
+	LASSERTF((int)sizeof(((struct ladvise_hdr *)0)->lah_advise) == 0, "found %lld\n",
+		 (long long)(int)sizeof(((struct ladvise_hdr *)0)->lah_advise));
 }
-- 
1.8.3.1


From jsimmons at infradead.org  Mon Jul  2 23:24:25 2018
From: jsimmons at infradead.org (James Simmons)
Date: Mon,  2 Jul 2018 19:24:25 -0400
Subject: [lustre-devel] [PATCH 08/18] lustre: security: send file security
 context for creates
In-Reply-To: <1530573875-20465-1-git-send-email-jsimmons@infradead.org>
References: <1530573875-20465-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1530573875-20465-9-git-send-email-jsimmons@infradead.org>

From: "John L. Hammond" <jhammond at whamcloud.com>

Send file security context to MDT along with create RPCs. This closes
the insecure window between creation and setting of the security
context that existed previously. It also avoids a potential LDLM hang
which arises from ll_create_it() when we send a MDS_SETXATTR RPC while
holding the lookup+layout lock returned from open.

Signed-off-by: John L. Hammond <jhammond at whamcloud.com>
Signed-off-by: Sebastien Buisson <sbuisson at ddn.com>
WC-bug-id: https://jira.whamcloud.com/browse/LU-5560
Reviewed-on: http://review.whamcloud.com/19971
Reviewed-by: Dmitry Eremin <dmitry.eremin at intel.com>
Reviewed-by: Oleg Drokin <green at whamcloud.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 .../lustre/lustre/include/lustre_req_layout.h      |  4 +-
 drivers/staging/lustre/lustre/include/obd.h        |  5 ++
 drivers/staging/lustre/lustre/llite/dir.c          | 54 ++++++++++++++++------
 .../staging/lustre/lustre/llite/llite_internal.h   | 11 ++++-
 drivers/staging/lustre/lustre/llite/llite_lib.c    | 14 ++++++
 drivers/staging/lustre/lustre/llite/namei.c        | 40 ++++++++++++++--
 .../staging/lustre/lustre/llite/xattr_security.c   | 38 ++++++++++++++-
 drivers/staging/lustre/lustre/mdc/mdc_internal.h   |  4 ++
 drivers/staging/lustre/lustre/mdc/mdc_lib.c        | 32 +++++++++++++
 drivers/staging/lustre/lustre/mdc/mdc_locks.c      |  7 +++
 drivers/staging/lustre/lustre/mdc/mdc_reint.c      |  7 +++
 drivers/staging/lustre/lustre/ptlrpc/layout.c      | 28 +++++++++--
 12 files changed, 218 insertions(+), 26 deletions(-)

diff --git a/drivers/staging/lustre/lustre/include/lustre_req_layout.h b/drivers/staging/lustre/lustre/include/lustre_req_layout.h
index 9d718b7..3387ab2 100644
--- a/drivers/staging/lustre/lustre/include/lustre_req_layout.h
+++ b/drivers/staging/lustre/lustre/include/lustre_req_layout.h
@@ -60,7 +60,7 @@ enum req_location {
 };
 
 /* Maximal number of fields (buffers) in a request message. */
-#define REQ_MAX_FIELD_NR  9
+#define REQ_MAX_FIELD_NR 10
 
 struct req_capsule {
 	struct ptlrpc_request   *rc_req;
@@ -236,6 +236,8 @@ void req_capsule_shrink(struct req_capsule *pill,
 extern struct req_msg_field RMF_GETINFO_VALLEN;
 extern struct req_msg_field RMF_GETINFO_KEY;
 extern struct req_msg_field RMF_CLOSE_DATA;
+extern struct req_msg_field RMF_FILE_SECCTX_NAME;
+extern struct req_msg_field RMF_FILE_SECCTX;
 
 /*
  * connection handle received in MDS_CONNECT request.
diff --git a/drivers/staging/lustre/lustre/include/obd.h b/drivers/staging/lustre/lustre/include/obd.h
index d4574ef..62f85a1 100644
--- a/drivers/staging/lustre/lustre/include/obd.h
+++ b/drivers/staging/lustre/lustre/include/obd.h
@@ -755,6 +755,11 @@ struct md_op_data {
 	__u64			op_data_version;
 	struct lustre_handle	op_lease_handle;
 
+	/* File security context, for creates. */
+	const char	       *op_file_secctx_name;
+	void		       *op_file_secctx;
+	u32			op_file_secctx_size;
+
 	/* default stripe offset */
 	__u32			op_default_stripe_offset;
 };
diff --git a/drivers/staging/lustre/lustre/llite/dir.c b/drivers/staging/lustre/lustre/llite/dir.c
index 52a8ecc..987f4b2 100644
--- a/drivers/staging/lustre/lustre/llite/dir.c
+++ b/drivers/staging/lustre/lustre/llite/dir.c
@@ -398,7 +398,7 @@ static int ll_send_mgc_param(struct obd_export *mgc, char *string)
 /**
  * Create striped directory with specified stripe(@lump)
  *
- * param[in] parent	the parent of the directory.
+ * param[in] dparent	the parent of the directory.
  * param[in] lump	the specified stripes.
  * param[in] dirname	the name of the directory.
  * param[in] mode	the specified mode of the directory.
@@ -406,14 +406,23 @@ static int ll_send_mgc_param(struct obd_export *mgc, char *string)
  * retval		=0 if striped directory is being created successfully.
  *			<0 if the creation is failed.
  */
-static int ll_dir_setdirstripe(struct inode *parent, struct lmv_user_md *lump,
+static int ll_dir_setdirstripe(struct dentry *dparent, struct lmv_user_md *lump,
 			       const char *dirname, umode_t mode)
 {
+	struct inode *parent = dparent->d_inode;
 	struct ptlrpc_request *request = NULL;
 	struct md_op_data *op_data;
 	struct ll_sb_info *sbi = ll_i2sbi(parent);
 	struct inode *inode = NULL;
-	struct dentry dentry;
+	struct dentry dentry = {
+		.d_parent = dparent,
+		.d_name = {
+			.name = dirname,
+			.len = strlen(dirname),
+			.hash = full_name_hash(dparent, dirname,
+					       strlen(dirname)),
+		},
+	};
 	int err;
 
 	if (unlikely(lump->lum_magic != LMV_USER_MAGIC))
@@ -436,9 +445,21 @@ static int ll_dir_setdirstripe(struct inode *parent, struct lmv_user_md *lump,
 	op_data = ll_prep_md_op_data(NULL, parent, NULL, dirname,
 				     strlen(dirname), mode, LUSTRE_OPC_MKDIR,
 				     lump);
-	if (IS_ERR(op_data)) {
-		err = PTR_ERR(op_data);
-		goto err_exit;
+	if (IS_ERR(op_data))
+		return PTR_ERR(op_data);
+
+	if (sbi->ll_flags & LL_SBI_FILE_SECCTX) {
+		/*
+		 * selinux_dentry_init_security() uses dentry->d_parent and name
+		 * to determine the security context for the file. So our fake
+		 * dentry should be real enough for this purpose.
+		 */
+		err = ll_dentry_init_security(&dentry, mode, &dentry.d_name,
+					      &op_data->op_file_secctx_name,
+					      &op_data->op_file_secctx,
+					      &op_data->op_file_secctx_size);
+		if (err < 0)
+			goto out_op_data;
 	}
 
 	op_data->op_cli_flags |= CLI_SET_MEA;
@@ -446,20 +467,26 @@ static int ll_dir_setdirstripe(struct inode *parent, struct lmv_user_md *lump,
 			from_kuid(&init_user_ns, current_fsuid()),
 			from_kgid(&init_user_ns, current_fsgid()),
 			current_cap(), 0, &request);
-	ll_finish_md_op_data(op_data);
+	if (err)
+		goto out_request;
 
 	err = ll_prep_inode(&inode, request, parent->i_sb, NULL);
 	if (err)
-		goto err_exit;
+		goto out_inode;
 
-	memset(&dentry, 0, sizeof(dentry));
 	dentry.d_inode = inode;
 
-	err = ll_init_security(&dentry, inode, parent);
-	iput(inode);
+	if (!(sbi->ll_flags & LL_SBI_FILE_SECCTX))
+		err = ll_inode_init_security(&dentry, inode, parent);
 
-err_exit:
+out_inode:
+	if (inode)
+		iput(inode);
+out_request:
 	ptlrpc_req_finished(request);
+out_op_data:
+	ll_finish_md_op_data(op_data);
+
 	return err;
 }
 
@@ -1033,6 +1060,7 @@ static char *ll_getname(const char __user *filename)
 
 static long ll_dir_ioctl(struct file *file, unsigned int cmd, unsigned long arg)
 {
+	struct dentry *dentry = file_dentry(file);
 	struct inode *inode = file_inode(file);
 	struct ll_sb_info *sbi = ll_i2sbi(inode);
 	struct obd_ioctl_data *data;
@@ -1146,7 +1174,7 @@ static long ll_dir_ioctl(struct file *file, unsigned int cmd, unsigned long arg)
 #else
 		mode = data->ioc_type;
 #endif
-		rc = ll_dir_setdirstripe(inode, lum, filename, mode);
+		rc = ll_dir_setdirstripe(dentry, lum, filename, mode);
 lmv_out_free:
 		kvfree(buf);
 		return rc;
diff --git a/drivers/staging/lustre/lustre/llite/llite_internal.h b/drivers/staging/lustre/lustre/llite/llite_internal.h
index 86914c9..8399501 100644
--- a/drivers/staging/lustre/lustre/llite/llite_internal.h
+++ b/drivers/staging/lustre/lustre/llite/llite_internal.h
@@ -261,8 +261,11 @@ static inline void ll_layout_version_set(struct ll_inode_info *lli, __u32 gen)
 int ll_xattr_cache_get(struct inode *inode, const char *name,
 		       char *buffer, size_t size, __u64 valid);
 
-int ll_init_security(struct dentry *dentry, struct inode *inode,
-		     struct inode *dir);
+int ll_dentry_init_security(struct dentry *dentry, int mode, struct qstr *name,
+			    const char **secctx_name, void **secctx,
+			    u32 *secctx_size);
+int ll_inode_init_security(struct dentry *dentry, struct inode *inode,
+			   struct inode *dir);
 
 /*
  * Locking to guarantee consistency of non-atomic updates to long long i_size,
@@ -396,6 +399,9 @@ enum stats_track_type {
 					  * suppress_pings
 					  */
 #define LL_SBI_FAST_READ	0x400000 /* fast read support */
+#define LL_SBI_FILE_SECCTX	0x800000 /* set file security context at
+					  * create
+					  */
 
 #define LL_SBI_FLAGS {	\
 	"nolck",	\
@@ -421,6 +427,7 @@ enum stats_track_type {
 	"norootsquash",	\
 	"always_ping",	\
 	"fast_read",    \
+	"file_secctx",	\
 }
 
 /*
diff --git a/drivers/staging/lustre/lustre/llite/llite_lib.c b/drivers/staging/lustre/lustre/llite/llite_lib.c
index 640205a..7a414e2 100644
--- a/drivers/staging/lustre/lustre/llite/llite_lib.c
+++ b/drivers/staging/lustre/lustre/llite/llite_lib.c
@@ -42,6 +42,7 @@
 #include <linux/types.h>
 #include <linux/mm.h>
 #include <linux/random.h>
+#include <linux/security.h>
 #include <linux/fs_struct.h>
 
 #include <uapi/linux/lustre/lustre_ioctl.h>
@@ -149,6 +150,12 @@ static void ll_free_sbi(struct super_block *sb)
 	kfree(sbi);
 }
 
+static inline int obd_connect_has_secctx(struct obd_connect_data *data)
+{
+	return data->ocd_connect_flags & OBD_CONNECT_FLAGS2 &&
+	       data->ocd_connect_flags2 & OBD_CONNECT2_FILE_SECCTX;
+}
+
 static int client_common_fill_super(struct super_block *sb, char *md, char *dt)
 {
 	struct inode *root = NULL;
@@ -240,6 +247,8 @@ static int client_common_fill_super(struct super_block *sb, char *md, char *dt)
 	if (sbi->ll_flags & LL_SBI_ALWAYS_PING)
 		data->ocd_connect_flags &= ~OBD_CONNECT_PINGLESS;
 
+	data->ocd_connect_flags2 |= OBD_CONNECT2_FILE_SECCTX;
+
 	data->ocd_brw_size = MD_MAX_BRW_SIZE;
 
 	err = obd_connect(NULL, &sbi->ll_md_exp, obd, &sbi->ll_sb_uuid,
@@ -347,6 +356,9 @@ static int client_common_fill_super(struct super_block *sb, char *md, char *dt)
 	if (data->ocd_connect_flags & OBD_CONNECT_LAYOUTLOCK)
 		sbi->ll_flags |= LL_SBI_LAYOUT_LOCK;
 
+	if (obd_connect_has_secctx(data))
+		sbi->ll_flags |= LL_SBI_FILE_SECCTX;
+
 	if (data->ocd_ibits_known & MDS_INODELOCK_XATTR) {
 		if (!(data->ocd_connect_flags & OBD_CONNECT_MAX_EASIZE)) {
 			LCONSOLE_INFO(
@@ -2370,6 +2382,8 @@ struct md_op_data *ll_prep_md_op_data(struct md_op_data *op_data,
 
 void ll_finish_md_op_data(struct md_op_data *op_data)
 {
+	security_release_secctx(op_data->op_file_secctx,
+				op_data->op_file_secctx_size);
 	kfree(op_data);
 }
 
diff --git a/drivers/staging/lustre/lustre/llite/namei.c b/drivers/staging/lustre/lustre/llite/namei.c
index abcb1c8..134cc31 100644
--- a/drivers/staging/lustre/lustre/llite/namei.c
+++ b/drivers/staging/lustre/lustre/llite/namei.c
@@ -577,13 +577,28 @@ static struct dentry *ll_lookup_it(struct inode *parent, struct dentry *dentry,
 
 	op_data = ll_prep_md_op_data(NULL, parent, NULL, dentry->d_name.name,
 				     dentry->d_name.len, 0, opc, NULL);
-	if (IS_ERR(op_data))
-		return (void *)op_data;
+	if (IS_ERR(op_data)) {
+		retval = ERR_CAST(op_data);
+		goto out;
+	}
 
 	/* enforce umask if acl disabled or MDS doesn't support umask */
 	if (!IS_POSIXACL(parent) || !exp_connect_umask(ll_i2mdexp(parent)))
 		it->it_create_mode &= ~current_umask();
 
+	if (it->it_op & IT_CREAT &&
+	    ll_i2sbi(parent)->ll_flags & LL_SBI_FILE_SECCTX) {
+		rc = ll_dentry_init_security(dentry, it->it_create_mode,
+					     &dentry->d_name,
+					     &op_data->op_file_secctx_name,
+					     &op_data->op_file_secctx,
+					     &op_data->op_file_secctx_size);
+		if (rc < 0) {
+			retval = ERR_PTR(rc);
+			goto out;
+		}
+	}
+
 	rc = md_intent_lock(ll_i2mdexp(parent), op_data, it, &req,
 			    &ll_md_blocking_ast, 0);
 	/*
@@ -838,7 +853,10 @@ static int ll_create_it(struct inode *dir, struct dentry *dentry,
 
 	d_instantiate(dentry, inode);
 
-	return ll_init_security(dentry, inode, dir);
+	if (!(ll_i2sbi(inode)->ll_flags & LL_SBI_FILE_SECCTX))
+		rc = ll_inode_init_security(dentry, inode, dir);
+
+	return rc;
 }
 
 void ll_update_times(struct ptlrpc_request *request, struct inode *inode)
@@ -882,11 +900,21 @@ static int ll_new_node(struct inode *dir, struct dentry *dentry,
 		goto err_exit;
 	}
 
+	if (sbi->ll_flags & LL_SBI_FILE_SECCTX) {
+		err = ll_dentry_init_security(dentry, mode, &dentry->d_name,
+					      &op_data->op_file_secctx_name,
+					      &op_data->op_file_secctx,
+					      &op_data->op_file_secctx_size);
+		if (err < 0)
+			goto err_exit;
+	}
+
 	err = md_create(sbi->ll_md_exp, op_data, tgt, tgt_len, mode,
 			from_kuid(&init_user_ns, current_fsuid()),
 			from_kgid(&init_user_ns, current_fsgid()),
 			current_cap(), rdev, &request);
 	ll_finish_md_op_data(op_data);
+	op_data = NULL;
 	if (err < 0 && err != -EREMOTE)
 		goto err_exit;
 
@@ -934,11 +962,15 @@ static int ll_new_node(struct inode *dir, struct dentry *dentry,
 
 	d_instantiate(dentry, inode);
 
-	err = ll_init_security(dentry, inode, dir);
+	if (!(sbi->ll_flags & LL_SBI_FILE_SECCTX))
+		err = ll_inode_init_security(dentry, inode, dir);
 err_exit:
 	if (request)
 		ptlrpc_req_finished(request);
 
+	if (!IS_ERR_OR_NULL(op_data))
+		ll_finish_md_op_data(op_data);
+
 	return err;
 }
 
diff --git a/drivers/staging/lustre/lustre/llite/xattr_security.c b/drivers/staging/lustre/lustre/llite/xattr_security.c
index 93ec075..b419d8f 100644
--- a/drivers/staging/lustre/lustre/llite/xattr_security.c
+++ b/drivers/staging/lustre/lustre/llite/xattr_security.c
@@ -36,6 +36,41 @@
 #include <linux/xattr.h>
 #include "llite_internal.h"
 
+/*
+ * Check for LL_SBI_FILE_SECCTX before calling.
+ */
+int ll_dentry_init_security(struct dentry *dentry, int mode, struct qstr *name,
+			    const char **secctx_name, void **secctx,
+			    u32 *secctx_size)
+{
+	int rc;
+
+	/*
+	 * security_dentry_init_security() is strange. Like
+	 * security_inode_init_security() it may return a context (provided a
+	 * Linux security module is enabled) but unlike
+	 * security_inode_init_security() it does not return to us the name of
+	 * the extended attribute to store the context under (for example
+	 * "security.selinux"). So we only call it when we think we know what
+	 * the name of the extended attribute will be. This is OK-ish since
+	 * SELinux is the only module that implements
+	 * security_dentry_init_security(). Note that the NFS client code just
+	 * calls it and assumes that if anything is returned then it must come
+	 * from SELinux.
+	 */
+	if (!selinux_is_enabled())
+		return 0;
+
+	rc = security_dentry_init_security(dentry, mode, name, secctx,
+					   secctx_size);
+	if (rc < 0)
+		return rc;
+
+	*secctx_name = XATTR_NAME_SELINUX;
+
+	return 0;
+}
+
 /**
  * A helper function for ll_security_inode_init_security()
  * that takes care of setting xattrs
@@ -86,7 +121,8 @@
  * \retval < 0      failure to get security context or set xattr
  */
 int
-ll_init_security(struct dentry *dentry, struct inode *inode, struct inode *dir)
+ll_inode_init_security(struct dentry *dentry, struct inode *inode,
+		       struct inode *dir)
 {
 	if (!selinux_is_enabled())
 		return 0;
diff --git a/drivers/staging/lustre/lustre/mdc/mdc_internal.h b/drivers/staging/lustre/lustre/mdc/mdc_internal.h
index 28924e9..f19b0ce 100644
--- a/drivers/staging/lustre/lustre/mdc/mdc_internal.h
+++ b/drivers/staging/lustre/lustre/mdc/mdc_internal.h
@@ -54,6 +54,10 @@ void mdc_create_pack(struct ptlrpc_request *req, struct md_op_data *op_data,
 void mdc_open_pack(struct ptlrpc_request *req, struct md_op_data *op_data,
 		   umode_t mode, __u64 rdev, __u64 flags, const void *data,
 		   size_t datalen);
+void mdc_file_secctx_pack(struct ptlrpc_request *req,
+			  const char *secctx_name,
+			  const void *secctx, size_t secctx_size);
+
 void mdc_unlink_pack(struct ptlrpc_request *req, struct md_op_data *op_data);
 void mdc_link_pack(struct ptlrpc_request *req, struct md_op_data *op_data);
 void mdc_rename_pack(struct ptlrpc_request *req, struct md_op_data *op_data,
diff --git a/drivers/staging/lustre/lustre/mdc/mdc_lib.c b/drivers/staging/lustre/lustre/mdc/mdc_lib.c
index 9cb4d24..fc5a51d 100644
--- a/drivers/staging/lustre/lustre/mdc/mdc_lib.c
+++ b/drivers/staging/lustre/lustre/mdc/mdc_lib.c
@@ -110,6 +110,30 @@ static void mdc_pack_name(struct ptlrpc_request *req,
 	LASSERT(cpy_len == name_len && lu_name_is_valid_2(buf, cpy_len));
 }
 
+void mdc_file_secctx_pack(struct ptlrpc_request *req, const char *secctx_name,
+			  const void *secctx, size_t secctx_size)
+{
+	size_t buf_size;
+	void *buf;
+
+	if (!secctx_name)
+		return;
+
+	buf = req_capsule_client_get(&req->rq_pill, &RMF_FILE_SECCTX_NAME);
+	buf_size = req_capsule_get_size(&req->rq_pill, &RMF_FILE_SECCTX_NAME,
+					RCL_CLIENT);
+
+	LASSERT(buf_size == strlen(secctx_name) + 1);
+	memcpy(buf, secctx_name, buf_size);
+
+	buf = req_capsule_client_get(&req->rq_pill, &RMF_FILE_SECCTX);
+	buf_size = req_capsule_get_size(&req->rq_pill, &RMF_FILE_SECCTX,
+					RCL_CLIENT);
+
+	LASSERT(buf_size == secctx_size);
+	memcpy(buf, secctx, buf_size);
+}
+
 void mdc_readdir_pack(struct ptlrpc_request *req, __u64 pgoff, size_t size,
 		      const struct lu_fid *fid)
 {
@@ -159,6 +183,10 @@ void mdc_create_pack(struct ptlrpc_request *req, struct md_op_data *op_data,
 		tmp = req_capsule_client_get(&req->rq_pill, &RMF_EADATA);
 		memcpy(tmp, data, datalen);
 	}
+
+	mdc_file_secctx_pack(req, op_data->op_file_secctx_name,
+			     op_data->op_file_secctx,
+			     op_data->op_file_secctx_size);
 }
 
 static inline __u64 mds_pack_open_flags(__u64 flags)
@@ -224,6 +252,10 @@ void mdc_open_pack(struct ptlrpc_request *req, struct md_op_data *op_data,
 
 		if (op_data->op_bias & MDS_CREATE_VOLATILE)
 			cr_flags |= MDS_OPEN_VOLATILE;
+
+		mdc_file_secctx_pack(req, op_data->op_file_secctx_name,
+				     op_data->op_file_secctx,
+				     op_data->op_file_secctx_size);
 	}
 
 	if (lmm) {
diff --git a/drivers/staging/lustre/lustre/mdc/mdc_locks.c b/drivers/staging/lustre/lustre/mdc/mdc_locks.c
index a8aa0fa..cfe917c 100644
--- a/drivers/staging/lustre/lustre/mdc/mdc_locks.c
+++ b/drivers/staging/lustre/lustre/mdc/mdc_locks.c
@@ -288,6 +288,13 @@ static void mdc_realloc_openmsg(struct ptlrpc_request *req,
 	req_capsule_set_size(&req->rq_pill, &RMF_EADATA, RCL_CLIENT,
 			     max(lmmsize, obddev->u.cli.cl_default_mds_easize));
 
+	req_capsule_set_size(&req->rq_pill, &RMF_FILE_SECCTX_NAME,
+			     RCL_CLIENT, op_data->op_file_secctx_name ?
+			     strlen(op_data->op_file_secctx_name) + 1 : 0);
+
+	req_capsule_set_size(&req->rq_pill, &RMF_FILE_SECCTX, RCL_CLIENT,
+			     op_data->op_file_secctx_size);
+
 	rc = ldlm_prep_enqueue_req(exp, req, &cancels, count);
 	if (rc < 0) {
 		ptlrpc_request_free(req);
diff --git a/drivers/staging/lustre/lustre/mdc/mdc_reint.c b/drivers/staging/lustre/lustre/mdc/mdc_reint.c
index da5f14c3..bdffe6d 100644
--- a/drivers/staging/lustre/lustre/mdc/mdc_reint.c
+++ b/drivers/staging/lustre/lustre/mdc/mdc_reint.c
@@ -190,6 +190,13 @@ int mdc_create(struct obd_export *exp, struct md_op_data *op_data,
 	req_capsule_set_size(&req->rq_pill, &RMF_EADATA, RCL_CLIENT,
 			     data && datalen ? datalen : 0);
 
+	req_capsule_set_size(&req->rq_pill, &RMF_FILE_SECCTX_NAME,
+			     RCL_CLIENT, op_data->op_file_secctx_name ?
+			     strlen(op_data->op_file_secctx_name) + 1 : 0);
+
+	req_capsule_set_size(&req->rq_pill, &RMF_FILE_SECCTX, RCL_CLIENT,
+			     op_data->op_file_secctx_size);
+
 	rc = mdc_prep_elc_req(exp, req, MDS_REINT, &cancels, count);
 	if (rc) {
 		ptlrpc_request_free(req);
diff --git a/drivers/staging/lustre/lustre/ptlrpc/layout.c b/drivers/staging/lustre/lustre/ptlrpc/layout.c
index 0b3ac14..d3c0dd6 100644
--- a/drivers/staging/lustre/lustre/ptlrpc/layout.c
+++ b/drivers/staging/lustre/lustre/ptlrpc/layout.c
@@ -196,7 +196,9 @@
 	&RMF_CAPA1,
 	&RMF_NAME,
 	&RMF_EADATA,
-	&RMF_DLM_REQ
+	&RMF_DLM_REQ,
+	&RMF_FILE_SECCTX_NAME,
+	&RMF_FILE_SECCTX
 };
 
 static const struct req_msg_field *mds_reint_create_sym_client[] = {
@@ -205,7 +207,9 @@
 	&RMF_CAPA1,
 	&RMF_NAME,
 	&RMF_SYMTGT,
-	&RMF_DLM_REQ
+	&RMF_DLM_REQ,
+	&RMF_FILE_SECCTX_NAME,
+	&RMF_FILE_SECCTX
 };
 
 static const struct req_msg_field *mds_reint_open_client[] = {
@@ -214,7 +218,9 @@
 	&RMF_CAPA1,
 	&RMF_CAPA2,
 	&RMF_NAME,
-	&RMF_EADATA
+	&RMF_EADATA,
+	&RMF_FILE_SECCTX_NAME,
+	&RMF_FILE_SECCTX
 };
 
 static const struct req_msg_field *mds_reint_open_server[] = {
@@ -435,7 +441,9 @@
 	&RMF_REC_REINT,    /* coincides with mds_reint_create_client[] */
 	&RMF_CAPA1,
 	&RMF_NAME,
-	&RMF_EADATA
+	&RMF_EADATA,
+	&RMF_FILE_SECCTX_NAME,
+	&RMF_FILE_SECCTX
 };
 
 static const struct req_msg_field *ldlm_intent_open_client[] = {
@@ -446,7 +454,9 @@
 	&RMF_CAPA1,
 	&RMF_CAPA2,
 	&RMF_NAME,
-	&RMF_EADATA
+	&RMF_EADATA,
+	&RMF_FILE_SECCTX_NAME,
+	&RMF_FILE_SECCTX
 };
 
 static const struct req_msg_field *ldlm_intent_unlink_client[] = {
@@ -931,6 +941,14 @@ struct req_msg_field RMF_STRING =
 	DEFINE_MSGF("string", RMF_F_STRING, -1, NULL, NULL);
 EXPORT_SYMBOL(RMF_STRING);
 
+struct req_msg_field RMF_FILE_SECCTX_NAME =
+	DEFINE_MSGF("file_secctx_name", RMF_F_STRING, -1, NULL, NULL);
+EXPORT_SYMBOL(RMF_FILE_SECCTX_NAME);
+
+struct req_msg_field RMF_FILE_SECCTX =
+	DEFINE_MSGF("file_secctx", 0, -1, NULL, NULL);
+EXPORT_SYMBOL(RMF_FILE_SECCTX);
+
 struct req_msg_field RMF_LLOGD_BODY =
 	DEFINE_MSGF("llogd_body", 0,
 		    sizeof(struct llogd_body), lustre_swab_llogd_body, NULL);
-- 
1.8.3.1


From jsimmons at infradead.org  Mon Jul  2 23:24:24 2018
From: jsimmons at infradead.org (James Simmons)
Date: Mon,  2 Jul 2018 19:24:24 -0400
Subject: [lustre-devel] [PATCH 07/18] lustre: llite: restore fd_och when
 putting lease
In-Reply-To: <1530573875-20465-1-git-send-email-jsimmons@infradead.org>
References: <1530573875-20465-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1530573875-20465-8-git-send-email-jsimmons@infradead.org>

From: Henri Doreau <henri.doreau at cea.fr>

fd_och was not restored when putting back a file lease, preventing
from getting a lease, putting it back and taking it again on a FD.

Signed-off-by: Henri Doreau <henri.doreau at cea.fr>
WC-bug-id: https://jira.whamcloud.com/browse/LU-8174
Reviewed-on: http://review.whamcloud.com/20331
Reviewed-by: Jinshan Xiong <jinshan.xiong at gmail.com>
Reviewed-by: Jean-Baptiste Riaux <riaux.jb at intel.com>
Reviewed-by: Oleg Drokin <green at whamcloud.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/lustre/llite/file.c | 132 +++++++++++++++++++++--------
 1 file changed, 97 insertions(+), 35 deletions(-)

diff --git a/drivers/staging/lustre/lustre/llite/file.c b/drivers/staging/lustre/lustre/llite/file.c
index db18d1d..d570232 100644
--- a/drivers/staging/lustre/lustre/llite/file.c
+++ b/drivers/staging/lustre/lustre/llite/file.c
@@ -705,6 +705,97 @@ static int ll_md_blocking_lease_ast(struct ldlm_lock *lock,
 }
 
 /**
+ * When setting a lease on a file, we take ownership of the lli_mds_*_och
+ * and save it as fd->fd_och so as to force client to reopen the file even
+ * if it has an open lock in cache already.
+ */
+static int ll_lease_och_acquire(struct inode *inode, struct file *file,
+				struct lustre_handle *old_handle)
+{
+	struct ll_file_data *fd = LUSTRE_FPRIVATE(file);
+	struct ll_inode_info *lli = ll_i2info(inode);
+	struct obd_client_handle **och_p;
+	u64 *och_usecount;
+	int rc = 0;
+
+	/* Get the openhandle of the file */
+	mutex_lock(&lli->lli_och_mutex);
+	if (fd->fd_lease_och) {
+		rc = -EBUSY;
+		goto out_unlock;
+	}
+
+	if (!fd->fd_och) {
+		if (file->f_mode & FMODE_WRITE) {
+			LASSERT(lli->lli_mds_write_och);
+			och_p = &lli->lli_mds_write_och;
+			och_usecount = &lli->lli_open_fd_write_count;
+		} else {
+			LASSERT(lli->lli_mds_read_och);
+			och_p = &lli->lli_mds_read_och;
+			och_usecount = &lli->lli_open_fd_read_count;
+		}
+
+		if (*och_usecount > 1) {
+			rc = -EBUSY;
+			goto out_unlock;
+		}
+
+		fd->fd_och = *och_p;
+		*och_usecount = 0;
+		*och_p = NULL;
+	}
+
+	*old_handle = fd->fd_och->och_fh;
+
+out_unlock:
+	mutex_unlock(&lli->lli_och_mutex);
+	return rc;
+}
+
+/**
+ * Release ownership on lli_mds_*_och when putting back a file lease.
+ */
+static int ll_lease_och_release(struct inode *inode, struct file *file)
+{
+	struct ll_file_data *fd = LUSTRE_FPRIVATE(file);
+	struct ll_inode_info *lli = ll_i2info(inode);
+	struct obd_client_handle *old_och = NULL;
+	struct obd_client_handle **och_p;
+	u64 *och_usecount;
+	int rc = 0;
+
+	mutex_lock(&lli->lli_och_mutex);
+	if (file->f_mode & FMODE_WRITE) {
+		och_p = &lli->lli_mds_write_och;
+		och_usecount = &lli->lli_open_fd_write_count;
+	} else {
+		och_p = &lli->lli_mds_read_och;
+		och_usecount = &lli->lli_open_fd_read_count;
+	}
+
+	/*
+	 * The file may have been open by another process (broken lease) so
+	 * *och_p is not NULL. In this case we should simply increase usecount
+	 * and close fd_och.
+	 */
+	if (*och_p) {
+		old_och = fd->fd_och;
+		(*och_usecount)++;
+	} else {
+		*och_p = fd->fd_och;
+		*och_usecount = 1;
+	}
+	fd->fd_och = NULL;
+	mutex_unlock(&lli->lli_och_mutex);
+
+	if (old_och)
+		rc = ll_close_inode_openhandle(inode, old_och, 0, NULL);
+
+	return rc;
+}
+
+/**
  * Acquire a lease and open the file.
  */
 static struct obd_client_handle *
@@ -724,45 +815,12 @@ static int ll_md_blocking_lease_ast(struct ldlm_lock *lock,
 		return ERR_PTR(-EINVAL);
 
 	if (file) {
-		struct ll_inode_info *lli = ll_i2info(inode);
-		struct ll_file_data *fd = LUSTRE_FPRIVATE(file);
-		struct obd_client_handle **och_p;
-		__u64 *och_usecount;
-
 		if (!(fmode & file->f_mode) || (file->f_mode & FMODE_EXEC))
 			return ERR_PTR(-EPERM);
 
-		/* Get the openhandle of the file */
-		rc = -EBUSY;
-		mutex_lock(&lli->lli_och_mutex);
-		if (fd->fd_lease_och) {
-			mutex_unlock(&lli->lli_och_mutex);
-			return ERR_PTR(rc);
-		}
-
-		if (!fd->fd_och) {
-			if (file->f_mode & FMODE_WRITE) {
-				LASSERT(lli->lli_mds_write_och);
-				och_p = &lli->lli_mds_write_och;
-				och_usecount = &lli->lli_open_fd_write_count;
-			} else {
-				LASSERT(lli->lli_mds_read_och);
-				och_p = &lli->lli_mds_read_och;
-				och_usecount = &lli->lli_open_fd_read_count;
-			}
-			if (*och_usecount == 1) {
-				fd->fd_och = *och_p;
-				*och_p = NULL;
-				*och_usecount = 0;
-				rc = 0;
-			}
-		}
-		mutex_unlock(&lli->lli_och_mutex);
-		if (rc < 0) /* more than 1 opener */
+		rc = ll_lease_och_acquire(inode, file, &old_handle);
+		if (rc)
 			return ERR_PTR(rc);
-
-		LASSERT(fd->fd_och);
-		old_handle = fd->fd_och->och_fh;
 	}
 
 	och = kzalloc(sizeof(*och), GFP_NOFS);
@@ -2333,6 +2391,10 @@ static int ll_ladvise(struct inode *inode, struct file *file, __u64 flags,
 			if (rc < 0)
 				return rc;
 
+			rc = ll_lease_och_release(inode, file);
+			if (rc < 0)
+				return rc;
+
 			if (lease_broken)
 				fmode = 0;
 
-- 
1.8.3.1


From jsimmons at infradead.org  Mon Jul  2 23:24:31 2018
From: jsimmons at infradead.org (James Simmons)
Date: Mon,  2 Jul 2018 19:24:31 -0400
Subject: [lustre-devel] [PATCH 14/18] lustre: obd: add callback for
 llog_cat_process_or_fork
In-Reply-To: <1530573875-20465-1-git-send-email-jsimmons@infradead.org>
References: <1530573875-20465-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1530573875-20465-15-git-send-email-jsimmons@infradead.org>

From: Alexander Boyko <c17825 at cray.com>

Currently llog_process_or_fork() is hard coded to
always pass the function pointer llog_cat_process_cb().
Change llog_cat_process_or_fork() to pass in any
function pointer which will allow us more options
for llog_cat callback routines in the future.

Signed-off-by: Alexander Boyko <c17825 at cray.com>
WC-bug-id: https://jira.whamcloud.com/browse/LU-7156
Seagate-bug-id: MRP-2383
Reviewed-on: http://review.whamcloud.com/16416
Reviewed-by: Andreas Dilger <adilger at whamcloud.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark at intel.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/lustre/obdclass/llog_cat.c | 16 +++++++---------
 1 file changed, 7 insertions(+), 9 deletions(-)

diff --git a/drivers/staging/lustre/lustre/obdclass/llog_cat.c b/drivers/staging/lustre/lustre/obdclass/llog_cat.c
index d9c63ad..58dbd50 100644
--- a/drivers/staging/lustre/lustre/obdclass/llog_cat.c
+++ b/drivers/staging/lustre/lustre/obdclass/llog_cat.c
@@ -189,7 +189,8 @@ static int llog_cat_process_cb(const struct lu_env *env,
 
 static int llog_cat_process_or_fork(const struct lu_env *env,
 				    struct llog_handle *cat_llh,
-				    llog_cb_t cb, void *data, int startcat,
+				    llog_cb_t cat_cb, llog_cb_t cb,
+				    void *data, int startcat,
 				    int startidx, bool fork)
 {
 	struct llog_process_data d;
@@ -210,18 +211,15 @@ static int llog_cat_process_or_fork(const struct lu_env *env,
 
 		cd.lpcd_first_idx = llh->llh_cat_idx;
 		cd.lpcd_last_idx = 0;
-		rc = llog_process_or_fork(env, cat_llh, llog_cat_process_cb,
-					  &d, &cd, fork);
+		rc = llog_process_or_fork(env, cat_llh, cat_cb, &d, &cd, fork);
 		if (rc != 0)
 			return rc;
 
 		cd.lpcd_first_idx = 0;
 		cd.lpcd_last_idx = cat_llh->lgh_last_idx;
-		rc = llog_process_or_fork(env, cat_llh, llog_cat_process_cb,
-					  &d, &cd, fork);
+		rc = llog_process_or_fork(env, cat_llh, cat_cb, &d, &cd, fork);
 	} else {
-		rc = llog_process_or_fork(env, cat_llh, llog_cat_process_cb,
-					  &d, NULL, fork);
+		rc = llog_process_or_fork(env, cat_llh, cat_cb, &d, NULL, fork);
 	}
 
 	return rc;
@@ -230,7 +228,7 @@ static int llog_cat_process_or_fork(const struct lu_env *env,
 int llog_cat_process(const struct lu_env *env, struct llog_handle *cat_llh,
 		     llog_cb_t cb, void *data, int startcat, int startidx)
 {
-	return llog_cat_process_or_fork(env, cat_llh, cb, data, startcat,
-					startidx, false);
+	return llog_cat_process_or_fork(env, cat_llh, llog_cat_process_cb, cb,
+					data, startcat, startidx, false);
 }
 EXPORT_SYMBOL(llog_cat_process);
-- 
1.8.3.1


From jsimmons at infradead.org  Mon Jul  2 23:24:26 2018
From: jsimmons at infradead.org (James Simmons)
Date: Mon,  2 Jul 2018 19:24:26 -0400
Subject: [lustre-devel] [PATCH 09/18] lustre: llite: ladvise protocol changes
In-Reply-To: <1530573875-20465-1-git-send-email-jsimmons@infradead.org>
References: <1530573875-20465-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1530573875-20465-10-git-send-email-jsimmons@infradead.org>

From: Patrick Farrell <paf at cray.com>

This patch makes some changes to the ladvise API and
protocol to support lock ahead and possible future users.

Primarily, it separates the userspace API arguments from
the structures which go out on the network, and adds a
number of 'value' fields without a predefined use.

The meaning of each value field can be different for
different advice types, allowing some extensibility.

Signed-off-by: Patrick Farrell <paf at cray.com>
WC-bug-id: https://jira.whamcloud.com/browse/LU-7225
Reviewed-on: http://review.whamcloud.com/20666
Reviewed-by: Andreas Dilger <adilger at whamcloud.com>
Reviewed-by: Oleg Drokin <green at whamcloud.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 .../lustre/include/uapi/linux/lustre/lustre_idl.h  | 30 ++++++++++++++++
 .../lustre/include/uapi/linux/lustre/lustre_user.h | 37 +++++++++++++------
 drivers/staging/lustre/lustre/llite/file.c         |  8 ++---
 .../staging/lustre/lustre/ptlrpc/pack_generic.c    | 12 ++++---
 drivers/staging/lustre/lustre/ptlrpc/wiretest.c    | 42 +++++++++++++++-------
 5 files changed, 97 insertions(+), 32 deletions(-)

diff --git a/drivers/staging/lustre/include/uapi/linux/lustre/lustre_idl.h b/drivers/staging/lustre/include/uapi/linux/lustre/lustre_idl.h
index 4e25521..029ac8e 100644
--- a/drivers/staging/lustre/include/uapi/linux/lustre/lustre_idl.h
+++ b/drivers/staging/lustre/include/uapi/linux/lustre/lustre_idl.h
@@ -2694,5 +2694,35 @@ struct close_data {
 	__u64			cd_reserved[8];
 };
 
+/*
+ * This is the lu_ladvise struct which goes out on the wire.
+ * Corresponds to the userspace arg llapi_lu_ladvise.
+ * value[1-4] are unspecified fields, used differently by different advices
+ */
+struct lu_ladvise {
+	__u16 lla_advice;	/* advice type */
+	__u16 lla_value1;	/* values for different advice types */
+	__u32 lla_value2;
+	__u64 lla_start;	/* first byte of extent for advice */
+	__u64 lla_end;		/* last byte of extent for advice */
+	__u32 lla_value3;
+	__u32 lla_value4;
+};
+
+/*
+ * This is the ladvise_hdr which goes on the wire, corresponds to the userspace
+ * arg llapi_ladvise_hdr.
+ * value[1-3] are unspecified fields, used differently by different advices
+ */
+struct ladvise_hdr {
+	__u32			lah_magic;	/* LADVISE_MAGIC */
+	__u32			lah_count;	/* number of advices */
+	__u64			lah_flags;	/* from enum ladvise_flag */
+	__u32			lah_value1;	/* unused */
+	__u32			lah_value2;	/* unused */
+	__u64			lah_value3;	/* unused */
+	struct lu_ladvise	lah_advise[0];	/* advices in this header */
+};
+
 #endif
 /** @} lustreidl */
diff --git a/drivers/staging/lustre/include/uapi/linux/lustre/lustre_user.h b/drivers/staging/lustre/include/uapi/linux/lustre/lustre_user.h
index fc33a43..063a7db 100644
--- a/drivers/staging/lustre/include/uapi/linux/lustre/lustre_user.h
+++ b/drivers/staging/lustre/include/uapi/linux/lustre/lustre_user.h
@@ -276,7 +276,7 @@ struct ost_id {
 #define LL_IOC_MIGRATE			_IOR('f', 247, int)
 #define LL_IOC_FID2MDTIDX		_IOWR('f', 248, struct lu_fid)
 #define LL_IOC_GETPARENT		_IOWR('f', 249, struct getparent)
-#define LL_IOC_LADVISE			_IOR('f', 250, struct lu_ladvise)
+#define LL_IOC_LADVISE			_IOR('f', 250, struct llapi_lu_ladvise)
 
 /* Lease types for use as arg and return of LL_IOC_{GET,SET}_LEASE ioctl. */
 enum ll_lease_type {
@@ -1327,13 +1327,22 @@ enum lu_ladvise_type {
 	LU_LADVISE_INVALID	= 0,
 };
 
-#define LU_LADVISE_NAMES { }
+#define LU_LADVISE_NAMES {				\
+}
 
-struct lu_ladvise {
-	__u64			lla_advice;
-	__u64			lla_start;
-	__u64			lla_end;
-	__u64			lla_padding;
+/*
+ * This is the userspace argument for ladvise. It is currently the same as
+ * what goes on the wire (struct lu_ladvise), but is defined separately as we
+ * may need info which is only used locally.
+ */
+struct llapi_lu_ladvise {
+	__u16 lla_advice;	/* advice type */
+	__u16 lla_value1;	/* values for different advice types */
+	__u32 lla_value2;
+	__u64 lla_start;	/* first byte of extent for advice */
+	__u64 lla_end;		/* last byte of extent for advice */
+	__u32 lla_value3;
+	__u32 lla_value4;
 };
 
 enum ladvise_flag {
@@ -1343,13 +1352,19 @@ enum ladvise_flag {
 #define LADVISE_MAGIC 0x1ADF1CE0
 #define LF_MASK LF_ASYNC
 
-struct ladvise_hdr {
+/*
+ * This is the userspace argument for ladvise, corresponds to ladvise_hdr which
+ * is used on the wire. It is defined separately as we may need info which is
+ * only used locally.
+ */
+struct llapi_ladvise_hdr {
 	__u32			lah_magic;	/* LADVISE_MAGIC */
 	__u32			lah_count;	/* number of advices */
 	__u64			lah_flags;	/* from enum ladvise_flag */
-	__u64			lah_padding1;	/* unused */
-	__u64			lah_padding2;	/* unused */
-	struct lu_ladvise	lah_advise[0];
+	__u32			lah_value1;	/* unused */
+	__u32			lah_value2;	/* unused */
+	__u64			lah_value3;	/* unused */
+	struct llapi_lu_ladvise	lah_advise[0];	/* advices in this header */
 };
 
 #define LAH_COUNT_MAX		1024
diff --git a/drivers/staging/lustre/lustre/llite/file.c b/drivers/staging/lustre/lustre/llite/file.c
index d570232..5d0f8c2 100644
--- a/drivers/staging/lustre/lustre/llite/file.c
+++ b/drivers/staging/lustre/lustre/llite/file.c
@@ -2098,7 +2098,7 @@ static inline long ll_lease_type_from_fmode(fmode_t fmode)
  * much more data being sent to the client.
  */
 static int ll_ladvise(struct inode *inode, struct file *file, __u64 flags,
-		      struct lu_ladvise *ladvise)
+		      struct llapi_lu_ladvise *ladvise)
 {
 	struct cl_ladvise_io *lio;
 	struct lu_env *env;
@@ -2458,7 +2458,7 @@ static int ll_ladvise(struct inode *inode, struct file *file, __u64 flags,
 		return rc;
 	}
 	case LL_IOC_LADVISE: {
-		struct ladvise_hdr *ladvise_hdr;
+		struct llapi_ladvise_hdr *ladvise_hdr;
 		int alloc_size = sizeof(*ladvise_hdr);
 		int num_advise;
 		int i;
@@ -2469,7 +2469,7 @@ static int ll_ladvise(struct inode *inode, struct file *file, __u64 flags,
 			return -ENOMEM;
 
 		if (copy_from_user(ladvise_hdr,
-				   (const struct ladvise_hdr __user *)arg,
+				   (const struct llapi_ladvise_hdr __user *)arg,
 				   alloc_size)) {
 			rc = -EFAULT;
 			goto out_ladvise;
@@ -2498,7 +2498,7 @@ static int ll_ladvise(struct inode *inode, struct file *file, __u64 flags,
 		 * TODO: submit multiple advices to one server in a single RPC
 		 */
 		if (copy_from_user(ladvise_hdr,
-				   (const struct ladvise_hdr __user *)arg,
+				   (const struct llapi_advise_hdr __user *)arg,
 				   alloc_size)) {
 			rc = -EFAULT;
 			goto out_ladvise;
diff --git a/drivers/staging/lustre/lustre/ptlrpc/pack_generic.c b/drivers/staging/lustre/lustre/ptlrpc/pack_generic.c
index 468fa69..86a64a6 100644
--- a/drivers/staging/lustre/lustre/ptlrpc/pack_generic.c
+++ b/drivers/staging/lustre/lustre/ptlrpc/pack_generic.c
@@ -2312,10 +2312,13 @@ void lustre_swab_close_data(struct close_data *cd)
 
 void lustre_swab_ladvise(struct lu_ladvise *ladvise)
 {
+	swab16s(&ladvise->lla_advice);
+	swab16s(&ladvise->lla_value1);
+	swab32s(&ladvise->lla_value2);
 	swab64s(&ladvise->lla_start);
 	swab64s(&ladvise->lla_end);
-	swab64s(&ladvise->lla_advice);
-	BUILD_BUG_ON(!offsetof(typeof(*ladvise), lla_padding));
+	swab32s(&ladvise->lla_value3);
+	swab32s(&ladvise->lla_value4);
 }
 EXPORT_SYMBOL(lustre_swab_ladvise);
 
@@ -2324,7 +2327,8 @@ void lustre_swab_ladvise_hdr(struct ladvise_hdr *ladvise_hdr)
 	swab32s(&ladvise_hdr->lah_magic);
 	swab32s(&ladvise_hdr->lah_count);
 	swab64s(&ladvise_hdr->lah_flags);
-	BUILD_BUG_ON(!offsetof(typeof(*ladvise_hdr), lah_padding1));
-	BUILD_BUG_ON(!offsetof(typeof(*ladvise_hdr), lah_padding2));
+	swab32s(&ladvise_hdr->lah_value1);
+	swab32s(&ladvise_hdr->lah_value2);
+	swab64s(&ladvise_hdr->lah_value3);
 }
 EXPORT_SYMBOL(lustre_swab_ladvise_hdr);
diff --git a/drivers/staging/lustre/lustre/ptlrpc/wiretest.c b/drivers/staging/lustre/lustre/ptlrpc/wiretest.c
index dae1b09..a1895cb 100644
--- a/drivers/staging/lustre/lustre/ptlrpc/wiretest.c
+++ b/drivers/staging/lustre/lustre/ptlrpc/wiretest.c
@@ -4217,8 +4217,16 @@ void lustre_assert_wire_constants(void)
 		 (long long)(int)sizeof(struct lu_ladvise));
 	LASSERTF((int)offsetof(struct lu_ladvise, lla_advice) == 0, "found %lld\n",
 		 (long long)(int)offsetof(struct lu_ladvise, lla_advice));
-	LASSERTF((int)sizeof(((struct lu_ladvise *)0)->lla_advice) == 8, "found %lld\n",
+	LASSERTF((int)sizeof(((struct lu_ladvise *)0)->lla_advice) == 2, "found %lld\n",
 		 (long long)(int)sizeof(((struct lu_ladvise *)0)->lla_advice));
+	LASSERTF((int)offsetof(struct lu_ladvise, lla_value1) == 2, "found %lld\n",
+		 (long long)(int)offsetof(struct lu_ladvise, lla_value1));
+	LASSERTF((int)sizeof(((struct lu_ladvise *)0)->lla_value1) == 2, "found %lld\n",
+		 (long long)(int)sizeof(((struct lu_ladvise *)0)->lla_value1));
+	LASSERTF((int)offsetof(struct lu_ladvise, lla_value2) == 4, "found %lld\n",
+		 (long long)(int)offsetof(struct lu_ladvise, lla_value2));
+	LASSERTF((int)sizeof(((struct lu_ladvise *)0)->lla_value2) == 4, "found %lld\n",
+		 (long long)(int)sizeof(((struct lu_ladvise *)0)->lla_value2));
 	LASSERTF((int)offsetof(struct lu_ladvise, lla_start) == 8, "found %lld\n",
 		 (long long)(int)offsetof(struct lu_ladvise, lla_start));
 	LASSERTF((int)sizeof(((struct lu_ladvise *)0)->lla_start) == 8, "found %lld\n",
@@ -4227,10 +4235,14 @@ void lustre_assert_wire_constants(void)
 		 (long long)(int)offsetof(struct lu_ladvise, lla_end));
 	LASSERTF((int)sizeof(((struct lu_ladvise *)0)->lla_end) == 8, "found %lld\n",
 		 (long long)(int)sizeof(((struct lu_ladvise *)0)->lla_end));
-	LASSERTF((int)offsetof(struct lu_ladvise, lla_padding) == 24, "found %lld\n",
-		 (long long)(int)offsetof(struct lu_ladvise, lla_padding));
-	LASSERTF((int)sizeof(((struct lu_ladvise *)0)->lla_padding) == 8, "found %lld\n",
-		 (long long)(int)sizeof(((struct lu_ladvise *)0)->lla_padding));
+	LASSERTF((int)offsetof(struct lu_ladvise, lla_value3) == 24, "found %lld\n",
+		 (long long)(int)offsetof(struct lu_ladvise, lla_value3));
+	LASSERTF((int)sizeof(((struct lu_ladvise *)0)->lla_value3) == 4, "found %lld\n",
+		 (long long)(int)sizeof(((struct lu_ladvise *)0)->lla_value3));
+	LASSERTF((int)offsetof(struct lu_ladvise, lla_value4) == 28, "found %lld\n",
+		 (long long)(int)offsetof(struct lu_ladvise, lla_value4));
+	LASSERTF((int)sizeof(((struct lu_ladvise *)0)->lla_value4) == 4, "found %lld\n",
+		 (long long)(int)sizeof(((struct lu_ladvise *)0)->lla_value4));
 
 	/* Checks for struct ladvise_hdr */
 	LASSERTF(LADVISE_MAGIC == 0x1ADF1CE0, "found 0x%.8x\n",
@@ -4249,14 +4261,18 @@ void lustre_assert_wire_constants(void)
 		 (long long)(int)offsetof(struct ladvise_hdr, lah_flags));
 	LASSERTF((int)sizeof(((struct ladvise_hdr *)0)->lah_flags) == 8, "found %lld\n",
 		 (long long)(int)sizeof(((struct ladvise_hdr *)0)->lah_flags));
-	LASSERTF((int)offsetof(struct ladvise_hdr, lah_padding1) == 16, "found %lld\n",
-		 (long long)(int)offsetof(struct ladvise_hdr, lah_padding1));
-	LASSERTF((int)sizeof(((struct ladvise_hdr *)0)->lah_padding1) == 8, "found %lld\n",
-		 (long long)(int)sizeof(((struct ladvise_hdr *)0)->lah_padding1));
-	LASSERTF((int)offsetof(struct ladvise_hdr, lah_padding2) == 24, "found %lld\n",
-		 (long long)(int)offsetof(struct ladvise_hdr, lah_padding2));
-	LASSERTF((int)sizeof(((struct ladvise_hdr *)0)->lah_padding2) == 8, "found %lld\n",
-		 (long long)(int)sizeof(((struct ladvise_hdr *)0)->lah_padding2));
+	LASSERTF((int)offsetof(struct ladvise_hdr, lah_value1) == 16, "found %lld\n",
+		 (long long)(int)offsetof(struct ladvise_hdr, lah_value1));
+	LASSERTF((int)sizeof(((struct ladvise_hdr *)0)->lah_value1) == 4, "found %lld\n",
+		 (long long)(int)sizeof(((struct ladvise_hdr *)0)->lah_value1));
+	LASSERTF((int)offsetof(struct ladvise_hdr, lah_value2) == 20, "found %lld\n",
+		 (long long)(int)offsetof(struct ladvise_hdr, lah_value2));
+	LASSERTF((int)sizeof(((struct ladvise_hdr *)0)->lah_value2) == 4, "found %lld\n",
+		 (long long)(int)sizeof(((struct ladvise_hdr *)0)->lah_value2));
+	LASSERTF((int)offsetof(struct ladvise_hdr, lah_value3) == 24, "found %lld\n",
+		 (long long)(int)offsetof(struct ladvise_hdr, lah_value3));
+	LASSERTF((int)sizeof(((struct ladvise_hdr *)0)->lah_value3) == 8, "found %lld\n",
+		 (long long)(int)sizeof(((struct ladvise_hdr *)0)->lah_value3));
 	LASSERTF((int)offsetof(struct ladvise_hdr, lah_advise) == 32, "found %lld\n",
 		 (long long)(int)offsetof(struct ladvise_hdr, lah_advise));
 	LASSERTF((int)sizeof(((struct ladvise_hdr *)0)->lah_advise) == 0, "found %lld\n",
-- 
1.8.3.1


From jsimmons at infradead.org  Mon Jul  2 23:24:27 2018
From: jsimmons at infradead.org (James Simmons)
Date: Mon,  2 Jul 2018 19:24:27 -0400
Subject: [lustre-devel] [PATCH 10/18] lustre: ladvise: Add willread advice
 support for ladvise
In-Reply-To: <1530573875-20465-1-git-send-email-jsimmons@infradead.org>
References: <1530573875-20465-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1530573875-20465-11-git-send-email-jsimmons@infradead.org>

From: Li Xi <lixi at ddn.com>

This patch adds WILLREAD advice to ladvise framework. OSS will
prefetch data into memory when this hint is provided. It is not
garanteed how long the cached pages will be kept in memory.

Signed-off-by: Li Xi <lixi at ddn.com>
WC-bug-id: https://jira.whamcloud.com/browse/LU-4931
Reviewed-on: http://review.whamcloud.com/12458
Reviewed-by: Gu Zheng <gzheng at ddn.com>
Reviewed-by: Andreas Dilger <adilger at whamcloud.com>
Reviewed-by: Oleg Drokin <green at whamcloud.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/include/uapi/linux/lustre/lustre_user.h | 2 ++
 drivers/staging/lustre/lustre/ptlrpc/wiretest.c                | 4 ++++
 2 files changed, 6 insertions(+)

diff --git a/drivers/staging/lustre/include/uapi/linux/lustre/lustre_user.h b/drivers/staging/lustre/include/uapi/linux/lustre/lustre_user.h
index 063a7db..02b51ca 100644
--- a/drivers/staging/lustre/include/uapi/linux/lustre/lustre_user.h
+++ b/drivers/staging/lustre/include/uapi/linux/lustre/lustre_user.h
@@ -1325,9 +1325,11 @@ struct hsm_copy {
 
 enum lu_ladvise_type {
 	LU_LADVISE_INVALID	= 0,
+	LU_LADVISE_WILLREAD	= 1,
 };
 
 #define LU_LADVISE_NAMES {				\
+	[LU_LADVISE_WILLREAD]	= "willread",		\
 }
 
 /*
diff --git a/drivers/staging/lustre/lustre/ptlrpc/wiretest.c b/drivers/staging/lustre/lustre/ptlrpc/wiretest.c
index a1895cb..aa17d01 100644
--- a/drivers/staging/lustre/lustre/ptlrpc/wiretest.c
+++ b/drivers/staging/lustre/lustre/ptlrpc/wiretest.c
@@ -4243,6 +4243,8 @@ void lustre_assert_wire_constants(void)
 		 (long long)(int)offsetof(struct lu_ladvise, lla_value4));
 	LASSERTF((int)sizeof(((struct lu_ladvise *)0)->lla_value4) == 4, "found %lld\n",
 		 (long long)(int)sizeof(((struct lu_ladvise *)0)->lla_value4));
+	LASSERTF(LU_LADVISE_WILLREAD == 1, "found %lld\n",
+		 (long long)LU_LADVISE_WILLREAD);
 
 	/* Checks for struct ladvise_hdr */
 	LASSERTF(LADVISE_MAGIC == 0x1ADF1CE0, "found 0x%.8x\n",
@@ -4277,4 +4279,6 @@ void lustre_assert_wire_constants(void)
 		 (long long)(int)offsetof(struct ladvise_hdr, lah_advise));
 	LASSERTF((int)sizeof(((struct ladvise_hdr *)0)->lah_advise) == 0, "found %lld\n",
 		 (long long)(int)sizeof(((struct ladvise_hdr *)0)->lah_advise));
+	LASSERTF(LF_ASYNC == 0x00000001UL, "found 0x%.8xUL\n",
+		 (unsigned int)LF_ASYNC);
 }
-- 
1.8.3.1


From jsimmons at infradead.org  Mon Jul  2 23:24:30 2018
From: jsimmons at infradead.org (James Simmons)
Date: Mon,  2 Jul 2018 19:24:30 -0400
Subject: [lustre-devel] [PATCH 13/18] lustre: ptlrpc: properly set "rq_xid"
 for 4MB IO
In-Reply-To: <1530573875-20465-1-git-send-email-jsimmons@infradead.org>
References: <1530573875-20465-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1530573875-20465-14-git-send-email-jsimmons@infradead.org>

From: Fan Yong <fyong at whamcloud.com>

The commit 8bcaef92664f ("staging: lustre: ptlrpc: mbits is sent
within ptlrpc_body") replaced the "rq_xid" with "rq_mbits"
as the matchbits of bulk data transferring. To be interoperable
with old servers, it introduced the new connection flag:
OBD_CONNECT_BULK_MBITS. If the server does not support such
feature, then the "rq_xid" would be set the same value as
"rq_mbits". Unfortunately, it forgot to handle multiple bulk
operations, for example 4MB IO. If the new client wants to
make 4MB IO with old server, it may send a small "rq_xid" to
the old server, as to the old server will regard it as an 1MB
IO or 2MB IO. So the data transfer will not complete because
of only part of data transferred. Then the client will timeout
failure and retry again and again.

Fixes: 8bcaef92664f ("staging: lustre: ptlrpc: mbits is sent within ptlrpc_body)
Signed-off-by: Fan Yong <fyong at whamcloud.com>
WC-bug-id: https://jira.whamcloud.com/browse/LU-6808
Reviewed-on: http://review.whamcloud.com/22373
Reviewed-by: Jinshan Xiong <jinshan.xiong at gmail.com>
Reviewed-by: Bobi Jam <bobijam at hotmail.com>
Reviewed-by: Liang Zhen <liang.zhen at intel.com>
Reviewed-by: Oleg Drokin <green at whamcloud.com>
Signed-off-by: Tyson Whitehead <twhitehead at gmail.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/lustre/ptlrpc/client.c | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/drivers/staging/lustre/lustre/ptlrpc/client.c b/drivers/staging/lustre/lustre/ptlrpc/client.c
index c1b82bf..fee4e49 100644
--- a/drivers/staging/lustre/lustre/ptlrpc/client.c
+++ b/drivers/staging/lustre/lustre/ptlrpc/client.c
@@ -3084,8 +3084,7 @@ void ptlrpc_set_bulk_mbits(struct ptlrpc_request *req)
 		 * 'resend for the -EINPROGRESS resend'. To make it simple,
 		 * we opt to generate mbits for all resend cases.
 		 */
-		if ((bd->bd_import->imp_connect_data.ocd_connect_flags &
-		     OBD_CONNECT_BULK_MBITS)) {
+		if (OCD_HAS_FLAG(&bd->bd_import->imp_connect_data, BULK_MBITS)) {
 			req->rq_mbits = ptlrpc_next_xid();
 		} else {
 			/* old version transfers rq_xid to peer as matchbits */
@@ -3115,6 +3114,12 @@ void ptlrpc_set_bulk_mbits(struct ptlrpc_request *req)
 	 * see LU-1431
 	 */
 	req->rq_mbits += DIV_ROUND_UP(bd->bd_iov_count, LNET_MAX_IOV) - 1;
+
+	/* Set rq_xid as rq_mbits to indicate the final bulk for the old
+	 * server which does not support OBD_CONNECT_BULK_MBITS. LU-6808
+	 */
+	if (!OCD_HAS_FLAG(&bd->bd_import->imp_connect_data, BULK_MBITS))
+		req->rq_xid = req->rq_mbits;
 }
 
 /**
-- 
1.8.3.1


From jsimmons at infradead.org  Mon Jul  2 23:24:34 2018
From: jsimmons at infradead.org (James Simmons)
Date: Mon,  2 Jul 2018 19:24:34 -0400
Subject: [lustre-devel] [PATCH 17/18] lustre: ldlm: reduce mem footprint of
 ldlm_resource
In-Reply-To: <1530573875-20465-1-git-send-email-jsimmons@infradead.org>
References: <1530573875-20465-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1530573875-20465-18-git-send-email-jsimmons@infradead.org>

From: Niu Yawei <yawei.niu at intel.com>

 - Allocating lr_itree only for LDLM_EXTENT resource, reduced
   120 bytes;
 - Moving fields around to eliminate holes, eliminated 3 holes,
   reduced 4 bytes;
 - Remove unused lr_contention_time, reduced 8 bytes;

   Reduced 132 bytes in total.

Signed-off-by: Niu Yawei <yawei.niu at intel.com>
WC-bug-id: https://jira.whamcloud.com/browse/LU-6775
Reviewed-on: http://review.whamcloud.com/15485
Reviewed-by: Yang Sheng <ys at whamcloud.com>
Reviewed-by: Andreas Dilger <adilger at whamcloud.com>
Reviewed-by: Oleg Drokin <green at whamcloud.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/lustre/include/lustre_dlm.h | 24 ++++++++--------
 drivers/staging/lustre/lustre/ldlm/ldlm_internal.h |  1 +
 drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c    | 20 +++++++++++---
 drivers/staging/lustre/lustre/ldlm/ldlm_resource.c | 32 ++++++++++++++++------
 4 files changed, 54 insertions(+), 23 deletions(-)

diff --git a/drivers/staging/lustre/lustre/include/lustre_dlm.h b/drivers/staging/lustre/lustre/include/lustre_dlm.h
index 4f196c2..2a05ab8 100644
--- a/drivers/staging/lustre/lustre/include/lustre_dlm.h
+++ b/drivers/staging/lustre/lustre/include/lustre_dlm.h
@@ -819,6 +819,9 @@ struct ldlm_resource {
 	 */
 	struct hlist_node	lr_hash;
 
+	/** Reference count for this resource */
+	atomic_t		lr_refcount;
+
 	/** Spinlock to protect locks under this resource. */
 	spinlock_t		lr_lock;
 
@@ -835,32 +838,31 @@ struct ldlm_resource {
 	struct list_head		lr_waiting;
 	/** @} */
 
-	/** Type of locks this resource can hold. Only one type per resource. */
-	enum ldlm_type		lr_type; /* LDLM_{PLAIN,EXTENT,FLOCK,IBITS} */
-
 	/** Resource name */
 	struct ldlm_res_id	lr_name;
-	/** Reference count for this resource */
-	atomic_t		lr_refcount;
 
 	/**
 	 * Interval trees (only for extent locks) for all modes of this resource
 	 */
-	struct ldlm_interval_tree lr_itree[LCK_MODE_NUM];
+	struct ldlm_interval_tree *lr_itree;
+
+	/** Type of locks this resource can hold. Only one type per resource. */
+	enum ldlm_type		lr_type; /* LDLM_{PLAIN,EXTENT,FLOCK,IBITS} */
 
 	/**
 	 * Server-side-only lock value block elements.
 	 * To serialize lvbo_init.
 	 */
-	struct mutex		lr_lvb_mutex;
 	int			lr_lvb_len;
+	struct mutex		lr_lvb_mutex;
+
+	/**
+	 * Associated inode, used only on client side.
+	 */
+	struct inode		*lr_lvb_inode;
 
-	/** When the resource was considered as contended. */
-	unsigned long		lr_contention_time;
 	/** List of references to this resource. For debugging. */
 	struct lu_ref		lr_reference;
-
-	struct inode		*lr_lvb_inode;
 };
 
 static inline bool ldlm_has_layout(struct ldlm_lock *lock)
diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_internal.h b/drivers/staging/lustre/lustre/ldlm/ldlm_internal.h
index 60a15b9..1d7c727 100644
--- a/drivers/staging/lustre/lustre/ldlm/ldlm_internal.h
+++ b/drivers/staging/lustre/lustre/ldlm/ldlm_internal.h
@@ -165,6 +165,7 @@ void ldlm_handle_bl_callback(struct ldlm_namespace *ns,
 
 /* ldlm_lockd.c & ldlm_lock.c */
 extern struct kmem_cache *ldlm_lock_slab;
+extern struct kmem_cache *ldlm_interval_tree_slab;
 
 /* ldlm_extent.c */
 void ldlm_extent_add_lock(struct ldlm_resource *res, struct ldlm_lock *lock);
diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c b/drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c
index f410ef6..5b125fdc 100644
--- a/drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c
+++ b/drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c
@@ -1129,15 +1129,26 @@ int ldlm_init(void)
 					   sizeof(struct ldlm_lock), 0,
 					   SLAB_HWCACHE_ALIGN |
 					   SLAB_TYPESAFE_BY_RCU, NULL);
-	if (!ldlm_lock_slab) {
-		kmem_cache_destroy(ldlm_resource_slab);
-		return -ENOMEM;
-	}
+	if (!ldlm_lock_slab)
+		goto out_resource;
+
+	ldlm_interval_tree_slab = kmem_cache_create("interval_tree",
+						    sizeof(struct ldlm_interval_tree) * LCK_MODE_NUM,
+						    0, SLAB_HWCACHE_ALIGN,
+						    NULL);
+	if (!ldlm_interval_tree_slab)
+		goto out_lock;
 
 #if LUSTRE_TRACKS_LOCK_EXP_REFS
 	class_export_dump_hook = ldlm_dump_export_locks;
 #endif
 	return 0;
+
+out_lock:
+	kmem_cache_destroy(ldlm_lock_slab);
+out_resource:
+	kmem_cache_destroy(ldlm_resource_slab);
+	return -ENOMEM;
 }
 
 void ldlm_exit(void)
@@ -1151,4 +1162,5 @@ void ldlm_exit(void)
 	 */
 	synchronize_rcu();
 	kmem_cache_destroy(ldlm_lock_slab);
+	kmem_cache_destroy(ldlm_interval_tree_slab);
 }
diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_resource.c b/drivers/staging/lustre/lustre/ldlm/ldlm_resource.c
index 3946d62..f06cbd8 100644
--- a/drivers/staging/lustre/lustre/ldlm/ldlm_resource.c
+++ b/drivers/staging/lustre/lustre/ldlm/ldlm_resource.c
@@ -44,6 +44,7 @@
 #include <linux/libcfs/libcfs_hash.h>
 
 struct kmem_cache *ldlm_resource_slab, *ldlm_lock_slab;
+struct kmem_cache *ldlm_interval_tree_slab;
 
 int ldlm_srv_namespace_nr;
 int ldlm_cli_namespace_nr;
@@ -1001,10 +1002,9 @@ struct ldlm_namespace *ldlm_namespace_first_locked(enum ldlm_side client)
 }
 
 /** Create and initialize new resource. */
-static struct ldlm_resource *ldlm_resource_new(void)
+static struct ldlm_resource *ldlm_resource_new(enum ldlm_type ldlm_type)
 {
 	struct ldlm_resource *res;
-	int idx;
 
 	res = kmem_cache_zalloc(ldlm_resource_slab, GFP_NOFS);
 	if (!res)
@@ -1013,11 +1013,22 @@ static struct ldlm_resource *ldlm_resource_new(void)
 	INIT_LIST_HEAD(&res->lr_granted);
 	INIT_LIST_HEAD(&res->lr_waiting);
 
-	/* Initialize interval trees for each lock mode. */
-	for (idx = 0; idx < LCK_MODE_NUM; idx++) {
-		res->lr_itree[idx].lit_size = 0;
-		res->lr_itree[idx].lit_mode = 1 << idx;
-		res->lr_itree[idx].lit_root = RB_ROOT_CACHED;
+	if (ldlm_type == LDLM_EXTENT) {
+		int idx;
+
+		res->lr_itree = kmem_cache_zalloc(ldlm_interval_tree_slab,
+						  GFP_NOFS);
+		if (!res->lr_itree) {
+			kmem_cache_free(ldlm_resource_slab, res);
+			return NULL;
+		}
+
+		/* Initialize interval trees for each lock mode. */
+		for (idx = 0; idx < LCK_MODE_NUM; idx++) {
+			res->lr_itree[idx].lit_size = 0;
+			res->lr_itree[idx].lit_mode = 1 << idx;
+			res->lr_itree[idx].lit_root = RB_ROOT_CACHED;
+		}
 	}
 
 	atomic_set(&res->lr_refcount, 1);
@@ -1070,7 +1081,7 @@ struct ldlm_resource *
 
 	LASSERTF(type >= LDLM_MIN_TYPE && type < LDLM_MAX_TYPE,
 		 "type: %d\n", type);
-	res = ldlm_resource_new();
+	res = ldlm_resource_new(type);
 	if (!res)
 		return ERR_PTR(-ENOMEM);
 
@@ -1089,6 +1100,9 @@ struct ldlm_resource *
 		lu_ref_fini(&res->lr_reference);
 		/* We have taken lr_lvb_mutex. Drop it. */
 		mutex_unlock(&res->lr_lvb_mutex);
+		if (res->lr_itree)
+			kmem_cache_free(ldlm_interval_tree_slab,
+					res->lr_itree);
 		kmem_cache_free(ldlm_resource_slab, res);
 lvbo_init:
 		res = hlist_entry(hnode, struct ldlm_resource, lr_hash);
@@ -1167,6 +1181,8 @@ static void __ldlm_resource_putref_final(struct cfs_hash_bd *bd,
 		ns->ns_lvbo->lvbo_free(res);
 	if (cfs_hash_bd_count_get(bd) == 0)
 		ldlm_namespace_put(ns);
+	if (res->lr_itree)
+		kmem_cache_free(ldlm_interval_tree_slab, res->lr_itree);
 	kmem_cache_free(ldlm_resource_slab, res);
 }
 
-- 
1.8.3.1


From jsimmons at infradead.org  Mon Jul  2 23:24:48 2018
From: jsimmons at infradead.org (James Simmons)
Date: Mon,  2 Jul 2018 19:24:48 -0400
Subject: [lustre-devel] [PATCH 04/10] lustre: libcfs: fix module loading and
 sysfs setting race
In-Reply-To: <1530573894-20592-1-git-send-email-jsimmons@infradead.org>
References: <1530573894-20592-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1530573894-20592-5-git-send-email-jsimmons@infradead.org>

The set method is called when a module parameter is parsed and
this happens before libcfs __init function has been called. Also
the sysfs files corresponding to libcfs module parameters are
also constructed before the __init function. Lastly the set
method can be called while the __init function does run. Add
kernel_param_[un]lock() to void stomping over the module
parameter setting.

Signed-off-by: James Simmons <uja.ornl at yahoo.com>
WC-bug-id: https://jira.whamcloud.com/browse/LU-8066
Reviewed-on: https://review.whamcloud.com/24688
Reviewed-by: Dmitry Eremin <dmitry.eremin at intel.com>
Reviewed-by: Olaf Weber <olaf.weber at hpe.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/lnet/libcfs/debug.c | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)

diff --git a/drivers/staging/lustre/lnet/libcfs/debug.c b/drivers/staging/lustre/lnet/libcfs/debug.c
index f2509f4..5d93341 100644
--- a/drivers/staging/lustre/lnet/libcfs/debug.c
+++ b/drivers/staging/lustre/lnet/libcfs/debug.c
@@ -422,18 +422,22 @@ int libcfs_debug_init(unsigned long bufsize)
 	}
 
 	rc = cfs_tracefile_init(max);
-	if (!rc) {
-		libcfs_register_panic_notifier();
-		libcfs_debug_mb = cfs_trace_get_debug_mb();
-	}
+	if (rc)
+		return rc;
 
+	libcfs_register_panic_notifier();
+	kernel_param_lock(THIS_MODULE);
+	libcfs_debug_mb = cfs_trace_get_debug_mb();
+	kernel_param_unlock(THIS_MODULE);
 	return rc;
 }
 
 int libcfs_debug_cleanup(void)
 {
 	libcfs_unregister_panic_notifier();
+	kernel_param_lock(THIS_MODULE);
 	cfs_tracefile_exit();
+	kernel_param_unlock(THIS_MODULE);
 	return 0;
 }
 
-- 
1.8.3.1


From jsimmons at infradead.org  Mon Jul  2 23:24:50 2018
From: jsimmons at infradead.org (James Simmons)
Date: Mon,  2 Jul 2018 19:24:50 -0400
Subject: [lustre-devel] [PATCH 06/10] lustre: libcfs: don't assume sysfs
 mount point
In-Reply-To: <1530573894-20592-1-git-send-email-jsimmons@infradead.org>
References: <1530573894-20592-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1530573894-20592-7-git-send-email-jsimmons@infradead.org>

The symlinks created for the libcfs module parameters assume the
mount point of syfs is always /sys. We shouldn't assume that is
the case so use relative paths that guaranteed to work.

Signed-off-by: James Simmons <uja.ornl at yahoo.com>
WC-bug-id: https://jira.whamcloud.com/browse/LU-8066
Reviewed-on: https://review.whamcloud.com/24688
Reviewed-by: Dmitry Eremin <dmitry.eremin at intel.com>
Reviewed-by: Olaf Weber <olaf.weber at hpe.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/lnet/libcfs/module.c | 14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)

diff --git a/drivers/staging/lustre/lnet/libcfs/module.c b/drivers/staging/lustre/lnet/libcfs/module.c
index 55bcb89..389e74c 100644
--- a/drivers/staging/lustre/lnet/libcfs/module.c
+++ b/drivers/staging/lustre/lnet/libcfs/module.c
@@ -563,19 +563,19 @@ static int proc_cpt_distance(struct ctl_table *table, int write,
 
 static const struct lnet_debugfs_symlink_def lnet_debugfs_symlinks[] = {
 	{ "console_ratelimit",
-	  "/sys/module/libcfs/parameters/libcfs_console_ratelimit"},
+	  "../../../module/libcfs/parameters/libcfs_console_ratelimit"},
 	{ "debug_path",
-	  "/sys/module/libcfs/parameters/libcfs_debug_file_path"},
+	  "../../../module/libcfs/parameters/libcfs_debug_file_path"},
 	{ "panic_on_lbug",
-	  "/sys/module/libcfs/parameters/libcfs_panic_on_lbug"},
+	  "../../../module/libcfs/parameters/libcfs_panic_on_lbug"},
 	{ "console_backoff",
-	  "/sys/module/libcfs/parameters/libcfs_console_backoff"},
+	  "../../../module/libcfs/parameters/libcfs_console_backoff"},
 	{ "debug_mb",
-	  "/sys/module/libcfs/parameters/libcfs_debug_mb"},
+	  "../../../module/libcfs/parameters/libcfs_debug_mb"},
 	{ "console_min_delay_centisecs",
-	  "/sys/module/libcfs/parameters/libcfs_console_min_delay"},
+	  "../../../module/libcfs/parameters/libcfs_console_min_delay"},
 	{ "console_max_delay_centisecs",
-	  "/sys/module/libcfs/parameters/libcfs_console_max_delay"},
+	  "../../../module/libcfs/parameters/libcfs_console_max_delay"},
 	{},
 };
 
-- 
1.8.3.1


From jsimmons at infradead.org  Mon Jul  2 23:24:51 2018
From: jsimmons at infradead.org (James Simmons)
Date: Mon,  2 Jul 2018 19:24:51 -0400
Subject: [lustre-devel] [PATCH 07/10] lustre: libcfs: properly initialize
 lnet_debugfs_symlink_def
In-Reply-To: <1530573894-20592-1-git-send-email-jsimmons@infradead.org>
References: <1530573894-20592-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1530573894-20592-8-git-send-email-jsimmons@infradead.org>

Mirgate struct lnet_debugfs_symlink_def to C99 struct initializers.

Signed-off-by: James Simmons <uja.ornl at yahoo.com>
WC-bug-id: https://jira.whamcloud.com/browse/LU-8066
Reviewed-on: https://review.whamcloud.com/24688
Reviewed-by: Dmitry Eremin <dmitry.eremin at intel.com>
Reviewed-by: Olaf Weber <olaf.weber at hpe.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/lnet/libcfs/module.c | 28 ++++++++++++++--------------
 1 file changed, 14 insertions(+), 14 deletions(-)

diff --git a/drivers/staging/lustre/lnet/libcfs/module.c b/drivers/staging/lustre/lnet/libcfs/module.c
index 389e74c..2c4268b 100644
--- a/drivers/staging/lustre/lnet/libcfs/module.c
+++ b/drivers/staging/lustre/lnet/libcfs/module.c
@@ -562,20 +562,20 @@ static int proc_cpt_distance(struct ctl_table *table, int write,
 };
 
 static const struct lnet_debugfs_symlink_def lnet_debugfs_symlinks[] = {
-	{ "console_ratelimit",
-	  "../../../module/libcfs/parameters/libcfs_console_ratelimit"},
-	{ "debug_path",
-	  "../../../module/libcfs/parameters/libcfs_debug_file_path"},
-	{ "panic_on_lbug",
-	  "../../../module/libcfs/parameters/libcfs_panic_on_lbug"},
-	{ "console_backoff",
-	  "../../../module/libcfs/parameters/libcfs_console_backoff"},
-	{ "debug_mb",
-	  "../../../module/libcfs/parameters/libcfs_debug_mb"},
-	{ "console_min_delay_centisecs",
-	  "../../../module/libcfs/parameters/libcfs_console_min_delay"},
-	{ "console_max_delay_centisecs",
-	  "../../../module/libcfs/parameters/libcfs_console_max_delay"},
+	{ .name		= "console_ratelimit",
+	  .target	= "../../../module/libcfs/parameters/libcfs_console_ratelimit"},
+	{ .name		= "debug_path",
+	  .target	= "../../../module/libcfs/parameters/libcfs_debug_file_path"},
+	{ .name		= "panic_on_lbug",
+	  .target	= "../../../module/libcfs/parameters/libcfs_panic_on_lbug"},
+	{ .name		= "console_backoff",
+	  .target	= "../../../module/libcfs/parameters/libcfs_console_backoff"},
+	{ .name		= "debug_mb",
+	  .target	= "../../../module/libcfs/parameters/libcfs_debug_mb"},
+	{ .name		= "console_min_delay_centisecs",
+	  .target	= "../../../module/libcfs/parameters/libcfs_console_min_delay"},
+	{ .name		= "console_max_delay_centisecs",
+	  .target	= "../../../module/libcfs/parameters/libcfs_console_max_delay"},
 	{},
 };
 
-- 
1.8.3.1


From jsimmons at infradead.org  Mon Jul  2 23:24:20 2018
From: jsimmons at infradead.org (James Simmons)
Date: Mon,  2 Jul 2018 19:24:20 -0400
Subject: [lustre-devel] [PATCH 03/18] lustre: fileset: add fileset mount
 support
In-Reply-To: <1530573875-20465-1-git-send-email-jsimmons@infradead.org>
References: <1530573875-20465-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1530573875-20465-4-git-send-email-jsimmons@infradead.org>

From: Lai Siyao <lsiyao at whamcloud.com>

This patch enables client to mount subdirectory as fileset.

usage: mount -t lustre mgsname:/fsname/subdir /mount/point

 * mdt lookup fileset fid and return to client during mount.
 * `fid2path` support for fileset.

Signed-off-by: Lai Siyao <lsiyao at whamcloud.com>
Signed-off-by: Kit Westneat <kwestneat at ddn.com>
Signed-off-by: Wang Shilong <wshilong at ddn.com>
WC-bug-id: https://jira.whamcloud.com/browse/LU-28
Reviewed-on: http://review.whamcloud.com/5007
Reviewed-by: Andreas Dilger <adilger at whamcloud.com>
Reviewed-by: Li Xi <lixi at ddn.com>
Reviewed-by: Oleg Drokin <green at whamcloud.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 .../lustre/include/uapi/linux/lustre/lustre_idl.h  |  7 ++-
 .../staging/lustre/lustre/include/lustre_disk.h    |  2 +
 .../lustre/lustre/include/lustre_req_layout.h      |  2 +-
 drivers/staging/lustre/lustre/include/obd.h        |  3 +-
 drivers/staging/lustre/lustre/include/obd_class.h  |  5 ++-
 .../staging/lustre/lustre/include/obd_support.h    |  4 +-
 drivers/staging/lustre/lustre/llite/file.c         | 18 ++++++++
 drivers/staging/lustre/lustre/llite/llite_lib.c    |  6 ++-
 drivers/staging/lustre/lustre/lmv/lmv_obd.c        | 30 +++++++------
 drivers/staging/lustre/lustre/mdc/mdc_request.c    | 50 ++++++++++++++++++----
 drivers/staging/lustre/lustre/obdclass/obd_mount.c | 23 +++++++++-
 drivers/staging/lustre/lustre/ptlrpc/layout.c      | 14 ++++--
 .../staging/lustre/lustre/ptlrpc/lproc_ptlrpc.c    |  2 +-
 drivers/staging/lustre/lustre/ptlrpc/wiretest.c    | 12 +++---
 14 files changed, 135 insertions(+), 43 deletions(-)

diff --git a/drivers/staging/lustre/include/uapi/linux/lustre/lustre_idl.h b/drivers/staging/lustre/include/uapi/linux/lustre/lustre_idl.h
index 5fab107..6defc6d 100644
--- a/drivers/staging/lustre/include/uapi/linux/lustre/lustre_idl.h
+++ b/drivers/staging/lustre/include/uapi/linux/lustre/lustre_idl.h
@@ -1244,7 +1244,7 @@ enum mds_cmd {
 	MDS_READPAGE		= 37,
 	MDS_CONNECT		= 38,
 	MDS_DISCONNECT		= 39,
-	MDS_GETSTATUS		= 40,
+	MDS_GET_ROOT		= 40,
 	MDS_STATFS		= 41,
 	MDS_PIN			= 42, /* obsolete, never used in a release */
 	MDS_UNPIN		= 43, /* obsolete, never used in a release */
@@ -2626,7 +2626,10 @@ struct getinfo_fid2path {
 	__u64	   gf_recno;
 	__u32	   gf_linkno;
 	__u32	   gf_pathlen;
-	char	    gf_path[0];
+	union {
+		char		gf_path[0];
+		struct lu_fid	gf_root_fid[0];
+	} gf_u;
 } __packed;
 
 /** path2parent request/reply structures */
diff --git a/drivers/staging/lustre/lustre/include/lustre_disk.h b/drivers/staging/lustre/lustre/include/lustre_disk.h
index 886e817..bd8fa71 100644
--- a/drivers/staging/lustre/lustre/include/lustre_disk.h
+++ b/drivers/staging/lustre/lustre/include/lustre_disk.h
@@ -78,6 +78,7 @@ struct lustre_mount_data {
 	int	lmd_recovery_time_hard;
 	char      *lmd_dev;	   /* device name */
 	char      *lmd_profile;    /* client only */
+	char	*lmd_fileset;	/* mount fileset */
 	char      *lmd_mgssec;	/* sptlrpc flavor to mgs */
 	char      *lmd_opts;	/* lustre mount options (as opposed to
 				 * _device_ mount options)
@@ -134,6 +135,7 @@ struct lustre_sb_info {
 #define     s2lsi_nocast(sb) ((sb)->s_fs_info)
 
 #define     get_profile_name(sb)   (s2lsi(sb)->lsi_lmd->lmd_profile)
+#define     get_mount_fileset(sb)  (s2lsi(sb)->lsi_lmd->lmd_fileset)
 
 /****************** prototypes *********************/
 
diff --git a/drivers/staging/lustre/lustre/include/lustre_req_layout.h b/drivers/staging/lustre/lustre/include/lustre_req_layout.h
index db6d8ed..9d718b7 100644
--- a/drivers/staging/lustre/lustre/include/lustre_req_layout.h
+++ b/drivers/staging/lustre/lustre/include/lustre_req_layout.h
@@ -133,7 +133,7 @@ void req_capsule_shrink(struct req_capsule *pill,
 extern struct req_format RQF_MDS_CONNECT;
 extern struct req_format RQF_MDS_DISCONNECT;
 extern struct req_format RQF_MDS_STATFS;
-extern struct req_format RQF_MDS_GETSTATUS;
+extern struct req_format RQF_MDS_GET_ROOT;
 extern struct req_format RQF_MDS_SYNC;
 extern struct req_format RQF_MDS_GETXATTR;
 extern struct req_format RQF_MDS_GETATTR;
diff --git a/drivers/staging/lustre/lustre/include/obd.h b/drivers/staging/lustre/lustre/include/obd.h
index d6fd1ea..cd2a2d0 100644
--- a/drivers/staging/lustre/lustre/include/obd.h
+++ b/drivers/staging/lustre/lustre/include/obd.h
@@ -920,7 +920,8 @@ struct obd_client_handle {
 struct cl_attr;
 
 struct md_ops {
-	int (*getstatus)(struct obd_export *, struct lu_fid *);
+	int (*getstatus)(struct obd_export *exp, const char *fileset,
+			 struct lu_fid *fid);
 	int (*null_inode)(struct obd_export *, const struct lu_fid *);
 	int (*close)(struct obd_export *, struct md_op_data *,
 		     struct md_open_data *, struct ptlrpc_request **);
diff --git a/drivers/staging/lustre/lustre/include/obd_class.h b/drivers/staging/lustre/lustre/include/obd_class.h
index fc9c772..797986b 100644
--- a/drivers/staging/lustre/lustre/include/obd_class.h
+++ b/drivers/staging/lustre/lustre/include/obd_class.h
@@ -1180,13 +1180,14 @@ static inline int obd_register_observer(struct obd_device *obd,
 }
 
 /* metadata helpers */
-static inline int md_getstatus(struct obd_export *exp, struct lu_fid *fid)
+static inline int md_get_root(struct obd_export *exp, const char *fileset,
+			      struct lu_fid *fid)
 {
 	int rc;
 
 	EXP_CHECK_MD_OP(exp, getstatus);
 	EXP_MD_COUNTER_INCREMENT(exp, getstatus);
-	rc = MDP(exp->exp_obd, getstatus)(exp, fid);
+	rc = MDP(exp->exp_obd, getstatus)(exp, fileset, fid);
 	return rc;
 }
 
diff --git a/drivers/staging/lustre/lustre/include/obd_support.h b/drivers/staging/lustre/lustre/include/obd_support.h
index ca28caf..87806e8 100644
--- a/drivers/staging/lustre/lustre/include/obd_support.h
+++ b/drivers/staging/lustre/lustre/include/obd_support.h
@@ -135,8 +135,8 @@
 #define OBD_FAIL_MDS_CONNECT_PACK	0x118
 #define OBD_FAIL_MDS_REINT_NET_REP       0x119
 #define OBD_FAIL_MDS_DISCONNECT_NET      0x11a
-#define OBD_FAIL_MDS_GETSTATUS_NET       0x11b
-#define OBD_FAIL_MDS_GETSTATUS_PACK      0x11c
+#define OBD_FAIL_MDS_GET_ROOT_NET	0x11b
+#define OBD_FAIL_MDS_GET_ROOT_PACK	0x11c
 #define OBD_FAIL_MDS_STATFS_PACK	 0x11d
 #define OBD_FAIL_MDS_STATFS_NET	  0x11e
 #define OBD_FAIL_MDS_GETATTR_NAME_NET    0x11f
diff --git a/drivers/staging/lustre/lustre/llite/file.c b/drivers/staging/lustre/lustre/llite/file.c
index 44bec1d..5f944ca 100644
--- a/drivers/staging/lustre/lustre/llite/file.c
+++ b/drivers/staging/lustre/lustre/llite/file.c
@@ -1594,6 +1594,13 @@ int ll_fid2path(struct inode *inode, void __user *arg)
 		goto gf_free;
 	}
 
+	/*
+	 * append root FID after gfout to let MDT know the root FID so that it
+	 * can lookup the correct path, this is mainly for fileset.
+	 * old server without fileset mount support will ignore this.
+	 */
+	*gfout->gf_u.gf_root_fid = *ll_inode2fid(inode);
+
 	/* Call mdc_iocontrol */
 	rc = obd_iocontrol(OBD_IOC_FID2PATH, exp, outsize, gfout, NULL);
 	if (rc != 0)
@@ -2725,6 +2732,16 @@ int ll_migrate(struct inode *parent, struct file *file, int mdtidx,
 		goto out_free;
 	}
 
+	/*
+	 * lfs migrate command needs to be blocked on the client
+	 * by checking the migrate FID against the FID of the
+	 * filesystem root.
+	 */
+	if (child_inode == parent->i_sb->s_root->d_inode) {
+		rc = -EINVAL;
+		goto out_iput;
+	}
+
 	inode_lock(child_inode);
 	op_data->op_fid3 = *ll_inode2fid(child_inode);
 	if (!fid_is_sane(&op_data->op_fid3)) {
@@ -2807,6 +2824,7 @@ int ll_migrate(struct inode *parent, struct file *file, int mdtidx,
 		clear_nlink(child_inode);
 out_unlock:
 	inode_unlock(child_inode);
+out_iput:
 	iput(child_inode);
 out_free:
 	ll_finish_md_op_data(op_data);
diff --git a/drivers/staging/lustre/lustre/llite/llite_lib.c b/drivers/staging/lustre/lustre/llite/llite_lib.c
index df5bc0a..90dff0a 100644
--- a/drivers/staging/lustre/lustre/llite/llite_lib.c
+++ b/drivers/staging/lustre/lustre/llite/llite_lib.c
@@ -201,7 +201,8 @@ static int client_common_fill_super(struct super_block *sb, char *md, char *dt)
 				  OBD_CONNECT_DISP_STRIPE | OBD_CONNECT_LFSCK |
 				  OBD_CONNECT_OPEN_BY_FID |
 				  OBD_CONNECT_DIR_STRIPE |
-				  OBD_CONNECT_BULK_MBITS;
+				  OBD_CONNECT_BULK_MBITS |
+				  OBD_CONNECT_SUBTREE;
 
 	if (sbi->ll_flags & LL_SBI_LRU_RESIZE)
 		data->ocd_connect_flags |= OBD_CONNECT_LRU_RESIZE;
@@ -436,7 +437,8 @@ static int client_common_fill_super(struct super_block *sb, char *md, char *dt)
 	mutex_unlock(&sbi->ll_lco.lco_lock);
 
 	fid_zero(&sbi->ll_root_fid);
-	err = md_getstatus(sbi->ll_md_exp, &sbi->ll_root_fid);
+	err = md_get_root(sbi->ll_md_exp, get_mount_fileset(sb),
+			  &sbi->ll_root_fid);
 	if (err) {
 		CERROR("cannot mds_connect: rc = %d\n", err);
 		goto out_lock_cn_cb;
diff --git a/drivers/staging/lustre/lustre/lmv/lmv_obd.c b/drivers/staging/lustre/lustre/lmv/lmv_obd.c
index d4e8ba8..44fbaa6 100644
--- a/drivers/staging/lustre/lustre/lmv/lmv_obd.c
+++ b/drivers/staging/lustre/lustre/lmv/lmv_obd.c
@@ -613,17 +613,20 @@ static int lmv_fid2path(struct obd_export *exp, int len, void *karg,
 {
 	struct obd_device	*obddev = class_exp2obd(exp);
 	struct lmv_obd		*lmv = &obddev->u.lmv;
-	struct getinfo_fid2path *gf;
+	struct getinfo_fid2path *gf = karg;
 	struct lmv_tgt_desc     *tgt;
 	struct getinfo_fid2path *remote_gf = NULL;
+	struct lu_fid root_fid;
 	int			remote_gf_size = 0;
 	int			rc;
 
-	gf = karg;
 	tgt = lmv_find_target(lmv, &gf->gf_fid);
 	if (IS_ERR(tgt))
 		return PTR_ERR(tgt);
 
+	root_fid = *gf->gf_u.gf_root_fid;
+	LASSERT(fid_is_sane(&root_fid));
+
 repeat_fid2path:
 	rc = obd_iocontrol(OBD_IOC_FID2PATH, tgt->ltd_exp, len, gf, uarg);
 	if (rc != 0 && rc != -EREMOTE)
@@ -637,25 +640,25 @@ static int lmv_fid2path(struct obd_export *exp, int len, void *karg,
 		char *ptr;
 
 		ori_gf = karg;
-		if (strlen(ori_gf->gf_path) + 1 +
-		    strlen(gf->gf_path) + 1 > ori_gf->gf_pathlen) {
+		if (strlen(ori_gf->gf_u.gf_path) + 1 +
+		    strlen(gf->gf_u.gf_path) + 1 > ori_gf->gf_pathlen) {
 			rc = -EOVERFLOW;
 			goto out_fid2path;
 		}
 
-		ptr = ori_gf->gf_path;
+		ptr = ori_gf->gf_u.gf_path;
 
-		memmove(ptr + strlen(gf->gf_path) + 1, ptr,
-			strlen(ori_gf->gf_path));
+		memmove(ptr + strlen(gf->gf_u.gf_path) + 1, ptr,
+			strlen(ori_gf->gf_u.gf_path));
 
-		strncpy(ptr, gf->gf_path, strlen(gf->gf_path));
-		ptr += strlen(gf->gf_path);
+		strncpy(ptr, gf->gf_u.gf_path, strlen(gf->gf_u.gf_path));
+		ptr += strlen(gf->gf_u.gf_path);
 		*ptr = '/';
 	}
 
 	CDEBUG(D_INFO, "%s: get path %s " DFID " rec: %llu ln: %u\n",
 	       tgt->ltd_exp->exp_obd->obd_name,
-	       gf->gf_path, PFID(&gf->gf_fid), gf->gf_recno,
+	       gf->gf_u.gf_path, PFID(&gf->gf_fid), gf->gf_recno,
 	       gf->gf_linkno);
 
 	if (rc == 0)
@@ -689,7 +692,8 @@ static int lmv_fid2path(struct obd_export *exp, int len, void *karg,
 	remote_gf->gf_fid = gf->gf_fid;
 	remote_gf->gf_recno = -1;
 	remote_gf->gf_linkno = -1;
-	memset(remote_gf->gf_path, 0, remote_gf->gf_pathlen);
+	memset(remote_gf->gf_u.gf_path, 0, remote_gf->gf_pathlen);
+	*remote_gf->gf_u.gf_root_fid = root_fid;
 	gf = remote_gf;
 	goto repeat_fid2path;
 
@@ -1387,13 +1391,13 @@ static int lmv_statfs(const struct lu_env *env, struct obd_export *exp,
 	return rc;
 }
 
-static int lmv_getstatus(struct obd_export *exp,
+static int lmv_getstatus(struct obd_export *exp, const char *fileset,
 			 struct lu_fid *fid)
 {
 	struct obd_device    *obd = exp->exp_obd;
 	struct lmv_obd       *lmv = &obd->u.lmv;
 
-	return md_getstatus(lmv->tgts[0]->ltd_exp, fid);
+	return md_get_root(lmv->tgts[0]->ltd_exp, fileset, fid);
 }
 
 static int lmv_getxattr(struct obd_export *exp, const struct lu_fid *fid,
diff --git a/drivers/staging/lustre/lustre/mdc/mdc_request.c b/drivers/staging/lustre/lustre/mdc/mdc_request.c
index 827ed0c..2e01f57 100644
--- a/drivers/staging/lustre/lustre/mdc/mdc_request.c
+++ b/drivers/staging/lustre/lustre/mdc/mdc_request.c
@@ -81,19 +81,50 @@ static inline int mdc_queue_wait(struct ptlrpc_request *req)
 	return rc;
 }
 
-static int mdc_getstatus(struct obd_export *exp, struct lu_fid *rootfid)
+/*
+ * Send MDS_GET_ROOT RPC to fetch root FID.
+ *
+ * If \a fileset is not NULL it should contain a subdirectory off
+ * the ROOT/ directory to be mounted on the client. Return the FID
+ * of the subdirectory to the client to mount onto its mountpoint.
+ *
+ * \param[in]  imp	MDC import
+ * \param[in]  fileset	fileset name, which could be NULL
+ * \param[out] rootfid	root FID of this mountpoint
+ * \param[out] pc	root capa will be unpacked and saved in this pointer
+ *
+ * \retval     0 on success, negative errno on failure
+ */
+static int mdc_get_root(struct obd_export *exp, const char *fileset,
+			struct lu_fid *rootfid)
 {
 	struct ptlrpc_request *req;
 	struct mdt_body       *body;
 	int		    rc;
 
-	req = ptlrpc_request_alloc_pack(class_exp2cliimp(exp),
-					&RQF_MDS_GETSTATUS,
-					LUSTRE_MDS_VERSION, MDS_GETSTATUS);
+	if (fileset && !(exp_connect_flags(exp) & OBD_CONNECT_SUBTREE))
+		return -ENOTSUPP;
+
+	req = ptlrpc_request_alloc(class_exp2cliimp(exp),
+				   &RQF_MDS_GET_ROOT);
 	if (!req)
 		return -ENOMEM;
 
+	if (fileset)
+		req_capsule_set_size(&req->rq_pill, &RMF_NAME, RCL_CLIENT,
+				     strlen(fileset) + 1);
+	rc = ptlrpc_request_pack(req, LUSTRE_MDS_VERSION, MDS_GET_ROOT);
+	if (rc) {
+		ptlrpc_request_free(req);
+		return rc;
+	}
 	mdc_pack_body(req, NULL, 0, 0, -1, 0);
+	if (fileset) {
+		char *name = req_capsule_client_get(&req->rq_pill, &RMF_NAME);
+
+		memcpy(name, fileset, strlen(fileset));
+	}
+	lustre_msg_add_flags(req->rq_reqmsg, LUSTRE_IMP_FULL);
 	req->rq_send_state = LUSTRE_IMP_FULL;
 
 	ptlrpc_request_set_replen(req);
@@ -1440,13 +1471,16 @@ static int mdc_ioc_fid2path(struct obd_export *exp, struct getinfo_fid2path *gf)
 		return -EOVERFLOW;
 
 	/* Key is KEY_FID2PATH + getinfo_fid2path description */
-	keylen = cfs_size_round(sizeof(KEY_FID2PATH)) + sizeof(*gf);
+	keylen = cfs_size_round(sizeof(KEY_FID2PATH)) + sizeof(*gf) +
+		 sizeof(struct lu_fid);
 	key = kzalloc(keylen, GFP_NOFS);
 	if (!key)
 		return -ENOMEM;
 	memcpy(key, KEY_FID2PATH, sizeof(KEY_FID2PATH));
 	memcpy(key + cfs_size_round(sizeof(KEY_FID2PATH)), gf, sizeof(*gf));
 
+	memcpy(key + cfs_size_round(sizeof(KEY_FID2PATH)) + sizeof(*gf),
+	       gf->gf_u.gf_root_fid, sizeof(struct lu_fid));
 	CDEBUG(D_IOCTL, "path get " DFID " from %llu #%d\n",
 	       PFID(&gf->gf_fid), gf->gf_recno, gf->gf_linkno);
 
@@ -1472,9 +1506,9 @@ static int mdc_ioc_fid2path(struct obd_export *exp, struct getinfo_fid2path *gf)
 
 	CDEBUG(D_IOCTL, "path got " DFID " from %llu #%d: %s\n",
 	       PFID(&gf->gf_fid), gf->gf_recno, gf->gf_linkno,
-	       gf->gf_pathlen < 512 ? gf->gf_path :
+	       gf->gf_pathlen < 512 ? gf->gf_u.gf_path :
 	       /* only log the last 512 characters of the path */
-	       gf->gf_path + gf->gf_pathlen - 512);
+	       gf->gf_u.gf_path + gf->gf_pathlen - 512);
 
 out:
 	kfree(key);
@@ -2713,7 +2747,7 @@ static int mdc_process_config(struct obd_device *obd, u32 len, void *buf)
 };
 
 static struct md_ops mdc_md_ops = {
-	.getstatus		= mdc_getstatus,
+	.getstatus		= mdc_get_root,
 	.null_inode		= mdc_null_inode,
 	.close			= mdc_close,
 	.create			= mdc_create,
diff --git a/drivers/staging/lustre/lustre/obdclass/obd_mount.c b/drivers/staging/lustre/lustre/obdclass/obd_mount.c
index 232bbfa..6e9803b 100644
--- a/drivers/staging/lustre/lustre/obdclass/obd_mount.c
+++ b/drivers/staging/lustre/lustre/obdclass/obd_mount.c
@@ -545,6 +545,7 @@ static int lustre_free_lsi(struct super_block *sb)
 	if (lsi->lsi_lmd) {
 		kfree(lsi->lsi_lmd->lmd_dev);
 		kfree(lsi->lsi_lmd->lmd_profile);
+		kfree(lsi->lsi_lmd->lmd_fileset);
 		kfree(lsi->lsi_lmd->lmd_mgssec);
 		kfree(lsi->lsi_lmd->lmd_opts);
 		if (lsi->lsi_lmd->lmd_exclude_count)
@@ -1073,10 +1074,30 @@ static int lmd_parse(char *options, struct lustre_mount_data *lmd)
 		/* Remove leading /s from fsname */
 		while (*++s1 == '/')
 			;
+		s2 = s1;
+		while (*s2 != '/' && *s2 != '\0')
+			s2++;
 		/* Freed in lustre_free_lsi */
-		lmd->lmd_profile = kasprintf(GFP_NOFS, "%s-client", s1);
+		lmd->lmd_profile = kzalloc(s2 - s1 + 8, GFP_NOFS);
 		if (!lmd->lmd_profile)
 			return -ENOMEM;
+
+		strncat(lmd->lmd_profile, s1, s2 - s1);
+		strncat(lmd->lmd_profile, "-client", 7);
+
+		s1 = s2;
+		s2 = s1 + strlen(s1) - 1;
+		/* Remove padding /s from fileset */
+		while (*s2 == '/')
+			s2--;
+		if (s2 > s1) {
+			lmd->lmd_fileset = kzalloc(s2 - s1 + 2, GFP_NOFS);
+			if (!lmd->lmd_fileset) {
+				kfree(lmd->lmd_profile);
+				return -ENOMEM;
+			}
+			strncat(lmd->lmd_fileset, s1, s2 - s1 + 1);
+		}
 	}
 
 	/* Freed in lustre_free_lsi */
diff --git a/drivers/staging/lustre/lustre/ptlrpc/layout.c b/drivers/staging/lustre/lustre/ptlrpc/layout.c
index 6ef8789..0b3ac14 100644
--- a/drivers/staging/lustre/lustre/ptlrpc/layout.c
+++ b/drivers/staging/lustre/lustre/ptlrpc/layout.c
@@ -477,6 +477,12 @@
 	&RMF_EAVALS_LENS
 };
 
+static const struct req_msg_field *mds_get_root_client[] = {
+	&RMF_PTLRPC_BODY,
+	&RMF_MDT_BODY,
+	&RMF_NAME
+};
+
 static const struct req_msg_field *mds_getxattr_client[] = {
 	&RMF_PTLRPC_BODY,
 	&RMF_MDT_BODY,
@@ -674,7 +680,7 @@
 	&RQF_MDS_CONNECT,
 	&RQF_MDS_DISCONNECT,
 	&RQF_MDS_GET_INFO,
-	&RQF_MDS_GETSTATUS,
+	&RQF_MDS_GET_ROOT,
 	&RQF_MDS_STATFS,
 	&RQF_MDS_GETATTR,
 	&RQF_MDS_GETATTR_NAME,
@@ -1228,9 +1234,9 @@ struct req_format RQF_OST_QUOTACTL =
 	DEFINE_REQ_FMT0("OST_QUOTACTL", quotactl_only, quotactl_only);
 EXPORT_SYMBOL(RQF_OST_QUOTACTL);
 
-struct req_format RQF_MDS_GETSTATUS =
-	DEFINE_REQ_FMT0("MDS_GETSTATUS", mdt_body_only, mdt_body_capa);
-EXPORT_SYMBOL(RQF_MDS_GETSTATUS);
+struct req_format RQF_MDS_GET_ROOT =
+	DEFINE_REQ_FMT0("MDS_GET_ROOT", mds_get_root_client, mdt_body_capa);
+EXPORT_SYMBOL(RQF_MDS_GET_ROOT);
 
 struct req_format RQF_MDS_STATFS =
 	DEFINE_REQ_FMT0("MDS_STATFS", empty, obd_statfs_server);
diff --git a/drivers/staging/lustre/lustre/ptlrpc/lproc_ptlrpc.c b/drivers/staging/lustre/lustre/ptlrpc/lproc_ptlrpc.c
index 52b980c..35120e7 100644
--- a/drivers/staging/lustre/lustre/ptlrpc/lproc_ptlrpc.c
+++ b/drivers/staging/lustre/lustre/ptlrpc/lproc_ptlrpc.c
@@ -74,7 +74,7 @@
 	{ MDS_READPAGE,     "mds_readpage" },
 	{ MDS_CONNECT,      "mds_connect" },
 	{ MDS_DISCONNECT,   "mds_disconnect" },
-	{ MDS_GETSTATUS,    "mds_getstatus" },
+	{ MDS_GET_ROOT,			"mds_get_root" },
 	{ MDS_STATFS,       "mds_statfs" },
 	{ MDS_PIN,	  "mds_pin" },
 	{ MDS_UNPIN,	"mds_unpin" },
diff --git a/drivers/staging/lustre/lustre/ptlrpc/wiretest.c b/drivers/staging/lustre/lustre/ptlrpc/wiretest.c
index 5a68de5..43931dd 100644
--- a/drivers/staging/lustre/lustre/ptlrpc/wiretest.c
+++ b/drivers/staging/lustre/lustre/ptlrpc/wiretest.c
@@ -131,8 +131,8 @@ void lustre_assert_wire_constants(void)
 		 (long long)MDS_CONNECT);
 	LASSERTF(MDS_DISCONNECT == 39, "found %lld\n",
 		 (long long)MDS_DISCONNECT);
-	LASSERTF(MDS_GETSTATUS == 40, "found %lld\n",
-		 (long long)MDS_GETSTATUS);
+	LASSERTF(MDS_GET_ROOT == 40, "found %lld\n",
+		 (long long)MDS_GET_ROOT);
 	LASSERTF(MDS_STATFS == 41, "found %lld\n",
 		 (long long)MDS_STATFS);
 	LASSERTF(MDS_PIN == 42, "found %lld\n",
@@ -3708,10 +3708,10 @@ void lustre_assert_wire_constants(void)
 		 (long long)(int)offsetof(struct getinfo_fid2path, gf_pathlen));
 	LASSERTF((int)sizeof(((struct getinfo_fid2path *)0)->gf_pathlen) == 4, "found %lld\n",
 		 (long long)(int)sizeof(((struct getinfo_fid2path *)0)->gf_pathlen));
-	LASSERTF((int)offsetof(struct getinfo_fid2path, gf_path[0]) == 32, "found %lld\n",
-		 (long long)(int)offsetof(struct getinfo_fid2path, gf_path[0]));
-	LASSERTF((int)sizeof(((struct getinfo_fid2path *)0)->gf_path[0]) == 1, "found %lld\n",
-		 (long long)(int)sizeof(((struct getinfo_fid2path *)0)->gf_path[0]));
+	LASSERTF((int)offsetof(struct getinfo_fid2path, gf_u.gf_path[0]) == 32, "found %lld\n",
+		 (long long)(int)offsetof(struct getinfo_fid2path, gf_u.gf_path[0]));
+	LASSERTF((int)sizeof(((struct getinfo_fid2path *)0)->gf_u.gf_path[0]) == 1, "found %lld\n",
+		 (long long)(int)sizeof(((struct getinfo_fid2path *)0)->gf_u.gf_path[0]));
 
 	/* Checks for struct fiemap */
 	LASSERTF((int)sizeof(struct fiemap) == 32, "found %lld\n",
-- 
1.8.3.1


From jsimmons at infradead.org  Mon Jul  2 23:24:21 2018
From: jsimmons at infradead.org (James Simmons)
Date: Mon,  2 Jul 2018 19:24:21 -0400
Subject: [lustre-devel] [PATCH 04/18] lustre: obd: rename md_getstatus() to
 md_get_root()
In-Reply-To: <1530573875-20465-1-git-send-email-jsimmons@infradead.org>
References: <1530573875-20465-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1530573875-20465-5-git-send-email-jsimmons@infradead.org>

From: "John L. Hammond" <jhammond at whamcloud.com>

Finish the partial renaming of the the OBD MD method
md_getstatus() to md_get_root().

Signed-off-by: John L. Hammond <jhammond at whamcloud.com>
WC-bug-id: https://jira.whamcloud.com/browse/LU-8035
Reviewed-on: http://review.whamcloud.com/19824
Reviewed-by: Andreas Dilger <adilger at whamcloud.com>
Reviewed-by: Wang Shilong <wshilong at ddn.com>
Reviewed-by: James Simmons <uja.ornl at yahoo.com>
Reviewed-by: Oleg Drokin <green at whamcloud.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/lustre/include/obd.h       |  2 +-
 drivers/staging/lustre/lustre/include/obd_class.h | 10 +++++-----
 drivers/staging/lustre/lustre/lmv/lmv_obd.c       |  6 +++---
 drivers/staging/lustre/lustre/mdc/mdc_request.c   |  2 +-
 4 files changed, 10 insertions(+), 10 deletions(-)

diff --git a/drivers/staging/lustre/lustre/include/obd.h b/drivers/staging/lustre/lustre/include/obd.h
index cd2a2d0..d4574ef 100644
--- a/drivers/staging/lustre/lustre/include/obd.h
+++ b/drivers/staging/lustre/lustre/include/obd.h
@@ -920,7 +920,7 @@ struct obd_client_handle {
 struct cl_attr;
 
 struct md_ops {
-	int (*getstatus)(struct obd_export *exp, const char *fileset,
+	int (*get_root)(struct obd_export *exp, const char *fileset,
 			 struct lu_fid *fid);
 	int (*null_inode)(struct obd_export *, const struct lu_fid *);
 	int (*close)(struct obd_export *, struct md_op_data *,
diff --git a/drivers/staging/lustre/lustre/include/obd_class.h b/drivers/staging/lustre/lustre/include/obd_class.h
index 797986b..20d07f8 100644
--- a/drivers/staging/lustre/lustre/include/obd_class.h
+++ b/drivers/staging/lustre/lustre/include/obd_class.h
@@ -356,8 +356,8 @@ static inline int obd_check_dev_active(struct obd_device *obd)
 
 #define MD_COUNTER_OFFSET(op)					\
 	((offsetof(struct md_ops, op) -				\
-		offsetof(struct md_ops, getstatus))		\
-		/ sizeof(((struct md_ops *)(0))->getstatus))
+		offsetof(struct md_ops, get_root))		\
+		/ sizeof(((struct md_ops *)(0))->get_root))
 
 #define MD_COUNTER_INCREMENT(obdx, op)				 \
 do {								 \
@@ -1185,9 +1185,9 @@ static inline int md_get_root(struct obd_export *exp, const char *fileset,
 {
 	int rc;
 
-	EXP_CHECK_MD_OP(exp, getstatus);
-	EXP_MD_COUNTER_INCREMENT(exp, getstatus);
-	rc = MDP(exp->exp_obd, getstatus)(exp, fileset, fid);
+	EXP_CHECK_MD_OP(exp, get_root);
+	EXP_MD_COUNTER_INCREMENT(exp, get_root);
+	rc = MDP(exp->exp_obd, get_root)(exp, fileset, fid);
 	return rc;
 }
 
diff --git a/drivers/staging/lustre/lustre/lmv/lmv_obd.c b/drivers/staging/lustre/lustre/lmv/lmv_obd.c
index 44fbaa6..71109ba 100644
--- a/drivers/staging/lustre/lustre/lmv/lmv_obd.c
+++ b/drivers/staging/lustre/lustre/lmv/lmv_obd.c
@@ -1391,8 +1391,8 @@ static int lmv_statfs(const struct lu_env *env, struct obd_export *exp,
 	return rc;
 }
 
-static int lmv_getstatus(struct obd_export *exp, const char *fileset,
-			 struct lu_fid *fid)
+static int lmv_get_root(struct obd_export *exp, const char *fileset,
+			struct lu_fid *fid)
 {
 	struct obd_device    *obd = exp->exp_obd;
 	struct lmv_obd       *lmv = &obd->u.lmv;
@@ -3076,7 +3076,7 @@ static int lmv_merge_attr(struct obd_export *exp,
 };
 
 static struct md_ops lmv_md_ops = {
-	.getstatus		= lmv_getstatus,
+	.get_root		= lmv_get_root,
 	.null_inode		= lmv_null_inode,
 	.close			= lmv_close,
 	.create			= lmv_create,
diff --git a/drivers/staging/lustre/lustre/mdc/mdc_request.c b/drivers/staging/lustre/lustre/mdc/mdc_request.c
index 2e01f57..7457039 100644
--- a/drivers/staging/lustre/lustre/mdc/mdc_request.c
+++ b/drivers/staging/lustre/lustre/mdc/mdc_request.c
@@ -2747,7 +2747,7 @@ static int mdc_process_config(struct obd_device *obd, u32 len, void *buf)
 };
 
 static struct md_ops mdc_md_ops = {
-	.getstatus		= mdc_get_root,
+	.get_root		= mdc_get_root,
 	.null_inode		= mdc_null_inode,
 	.close			= mdc_close,
 	.create			= mdc_create,
-- 
1.8.3.1


From jsimmons at infradead.org  Mon Jul  2 23:24:32 2018
From: jsimmons at infradead.org (James Simmons)
Date: Mon,  2 Jul 2018 19:24:32 -0400
Subject: [lustre-devel] [PATCH 15/18] lustre: mount: fix lmd_parse() to
 handle new delimiters
In-Reply-To: <1530573875-20465-1-git-send-email-jsimmons@infradead.org>
References: <1530573875-20465-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1530573875-20465-16-git-send-email-jsimmons@infradead.org>

From: Jian Yu <jian.yu at intel.com>

The lmd_parse() function parses mount options with comma as
delimiter without considering commas in expr_list as follows
is a valid LNET nid range syntax:

<expr_list>  :== '[' <range_expr> [ ',' <range_expr>] ']'

This patch fixes the above issue by using cfs_parse_nidlist()
to parse nid range list instead of using class_parse_nid_quiet()
to parse only one nid.

In multi-rail and failover configurations, colon is a valid
delimiter in LNET nid list to separate different hosts.
This patch fixes lmd_parse()->lmd_parse_nidlist() to handle
both comma and colon as delimiters.

Signed-off-by: Jian Yu <jian.yu at intel.com>
Signed-off-by: James Simmons <uja.ornl at yahoo.com>
WC-bug-id: https://jira.whamcloud.com/browse/LU-9325
Reviewed-on: https://review.whamcloud.com/26558
WC-bug-id: https://jira.whamcloud.com/browse/LU-8311
Reviewed-on: http://review.whamcloud.com/21329
WC-bug-id: https://jira.whamcloud.com/browse/LU-5690
Reviewed-on: http://review.whamcloud.com/17036
Reviewed-by: John L. Hammond <jhammond at whamcloud.com>
Reviewed-by: Bob Glossman <bob.glossman at intel.com>
Reviewed-by: Niu Yawei <yawei.niu at intel.com>
Reviewed-by: Oleg Drokin <green at whamcloud.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/lustre/obdclass/obd_mount.c | 112 +++++++++++++++++++--
 1 file changed, 104 insertions(+), 8 deletions(-)

diff --git a/drivers/staging/lustre/lustre/obdclass/obd_mount.c b/drivers/staging/lustre/lustre/obdclass/obd_mount.c
index 6e9803b..708c580 100644
--- a/drivers/staging/lustre/lustre/obdclass/obd_mount.c
+++ b/drivers/staging/lustre/lustre/obdclass/obd_mount.c
@@ -889,6 +889,104 @@ static int lmd_parse_mgs(struct lustre_mount_data *lmd, char **ptr)
 	return 0;
 }
 
+/**
+ * Find the first delimiter (comma or colon) from the specified \a buf and
+ * make \a *endh point to the string starting with the delimiter. The commas
+ * in expression list [...] will be skipped.
+ *
+ * @buf		a delimiter-separated string
+ * @endh	a pointer to a pointer that will point to the string
+ *		starting with the delimiter
+ *
+ * RETURNS	true if delimiter is found, false if delimiter is not found
+ */
+static bool lmd_find_delimiter(char *buf, char **endh)
+{
+	char *c = buf;
+	size_t pos;
+	bool found;
+
+	if (!buf)
+		return false;
+try_again:
+	if (*c == ',' || *c == ':')
+		return true;
+
+	pos = strcspn(c, "[:,]");
+	if (!pos)
+		return false;
+
+	/* Not a valid mount string */
+	if (*c == ']') {
+		CWARN("invalid mount string format\n");
+		return false;
+	}
+
+	c += pos;
+	if (*c == '[') {
+		c = strchr(c, ']');
+
+		/* invalid mount string */
+		if (!c) {
+			CWARN("invalid mount string format\n");
+			return false;
+		}
+		c++;
+		goto try_again;
+	}
+
+	found = *c != '\0';
+	if (found && endh)
+		*endh = c;
+
+	return found;
+}
+
+/**
+ * Find the first valid string delimited by comma or colon from the specified
+ * \a buf and parse it to see whether it's a valid nid list. If yes, \a *endh
+ * will point to the next string starting with the delimiter.
+ *
+ * \param[in] buf	a delimiter-separated string
+ * \param[in] endh	a pointer to a pointer that will point to the string
+ *			starting with the delimiter
+ *
+ * \retval 0		if the string is a valid nid list
+ * \retval 1		if the string is not a valid nid list
+ */
+static int lmd_parse_nidlist(char *buf, char **endh)
+{
+	struct list_head nidlist;
+	char *endp = buf;
+	int rc = 0;
+	char tmp;
+
+	if (!buf)
+		return 1;
+	while (*buf == ',' || *buf == ':')
+		buf++;
+	if (*buf == ' ' || *buf == '/' || *buf == '\0')
+		return 1;
+
+	if (!lmd_find_delimiter(buf, &endp))
+		endp = buf + strlen(buf);
+
+	tmp = *endp;
+	*endp = '\0';
+
+	INIT_LIST_HEAD(&nidlist);
+	if (cfs_parse_nidlist(buf, strlen(buf), &nidlist) <= 0)
+		rc = 1;
+	cfs_free_nidlist(&nidlist);
+
+	*endp = tmp;
+	if (rc)
+		return rc;
+	if (endh)
+		*endh = endp;
+	return 0;
+}
+
 /** Parse mount line options
  * e.g. mount -v -t lustre -o abort_recov uml1:uml2:/lustre-client /mnt/lustre
  * dev is passed as device=uml1:/lustre by mount.lustre
@@ -1006,20 +1104,18 @@ static int lmd_parse(char *options, struct lustre_mount_data *lmd)
 			clear++;
 		} else if (strncmp(s1, "param=", 6) == 0) {
 			size_t length, params_length;
-			char *tail = strchr(s1 + 6, ',');
+			char *tail = s1;
 
-			if (!tail) {
-				length = strlen(s1);
-			} else {
-				lnet_nid_t nid;
+			if (lmd_find_delimiter(s1 + 6, &tail)) {
 				char *param_str = tail + 1;
 				int supplementary = 1;
 
-				while (!class_parse_nid_quiet(param_str, &nid,
-							      &param_str)) {
+				while (!lmd_parse_nidlist(param_str,
+							  &param_str))
 					supplementary = 0;
-				}
 				length = param_str - s1 - supplementary;
+			} else {
+				length = strlen(s1);
 			}
 			length -= 6;
 			params_length = strlen(lmd->lmd_params);
-- 
1.8.3.1


From jsimmons at infradead.org  Mon Jul  2 23:24:33 2018
From: jsimmons at infradead.org (James Simmons)
Date: Mon,  2 Jul 2018 19:24:33 -0400
Subject: [lustre-devel] [PATCH 16/18] lustre: libcfs: restore original
 behavior in cfs_str2num_check
In-Reply-To: <1530573875-20465-1-git-send-email-jsimmons@infradead.org>
References: <1530573875-20465-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1530573875-20465-17-git-send-email-jsimmons@infradead.org>

When cfs_str2num_check() moved from simple_strtoul to kstrtoul
some of the functionality got lost. Restore handling hexidecimal
number as well as '+' and '-'. Also handle any trailing spaces.

Signed-off-by: James Simmons <uja.ornl at yahoo.com>
WC-bug-id: https://jira.whamcloud.com/browse/LU-9325
Reviewed-on: https://review.whamcloud.com/32217
Reviewed-by: John L. Hammond <jhammond at whamcloud.com>
Reviewed-by: Andreas Dilger <adilger at whamcloud.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin at intel.com>
Reviewed-by: Oleg Drokin <green at whamcloud.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/lnet/libcfs/libcfs_string.c | 17 ++++++++++++++---
 1 file changed, 14 insertions(+), 3 deletions(-)

diff --git a/drivers/staging/lustre/lnet/libcfs/libcfs_string.c b/drivers/staging/lustre/lnet/libcfs/libcfs_string.c
index e1fb126..e390b0b 100644
--- a/drivers/staging/lustre/lnet/libcfs/libcfs_string.c
+++ b/drivers/staging/lustre/lnet/libcfs/libcfs_string.c
@@ -214,8 +214,10 @@ char *cfs_firststr(char *str, size_t size)
 {
 	bool all_numbers = true;
 	char *endp, cache;
+	int len;
 	int rc;
 
+	endp = strim(str);
 	/**
 	 * kstrouint can only handle strings composed
 	 * of only numbers. We need to scan the string
@@ -228,16 +230,25 @@ char *cfs_firststr(char *str, size_t size)
 	 * After we are done the character at the
 	 * position we placed '\0' must be restored.
 	 */
-	for (endp = str; endp < str + nob; endp++) {
-		if (!isdigit(*endp)) {
+	len = min((int)strlen(endp), nob);
+	for (; endp < str + len; endp++) {
+		if (!isdigit(*endp) && *endp != '-' &&
+		    *endp != '+') {
 			all_numbers = false;
 			break;
 		}
 	}
+
+	/* Eat trailing space */
+	if (!all_numbers && isspace(*endp)) {
+		all_numbers = true;
+		endp--;
+	}
+
 	cache = *endp;
 	*endp = '\0';
 
-	rc = kstrtouint(str, 10, num);
+	rc = kstrtouint(str, 0, num);
 	*endp = cache;
 	if (rc || !all_numbers)
 		return 0;
-- 
1.8.3.1


From jsimmons at infradead.org  Mon Jul  2 23:24:52 2018
From: jsimmons at infradead.org (James Simmons)
Date: Mon,  2 Jul 2018 19:24:52 -0400
Subject: [lustre-devel] [PATCH 08/10] lustre: libcfs: test if table is NULL
 for lnet_insert_debugfs
In-Reply-To: <1530573894-20592-1-git-send-email-jsimmons@infradead.org>
References: <1530573894-20592-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1530573894-20592-9-git-send-email-jsimmons@infradead.org>

It is possible to call lnet_insert_debugfs() with table being NULL.
We can avoid a potential oops by adding a test to see if table is
not NULL.

Signed-off-by: James Simmons <uja.ornl at yahoo.com>
WC-bug-id: https://jira.whamcloud.com/browse/LU-8066
Reviewed-on: https://review.whamcloud.com/24688
Reviewed-by: Dmitry Eremin <dmitry.eremin at intel.com>
Reviewed-by: Olaf Weber <olaf.weber at hpe.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/lnet/libcfs/module.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/staging/lustre/lnet/libcfs/module.c b/drivers/staging/lustre/lnet/libcfs/module.c
index 2c4268b..0d36121 100644
--- a/drivers/staging/lustre/lnet/libcfs/module.c
+++ b/drivers/staging/lustre/lnet/libcfs/module.c
@@ -659,7 +659,7 @@ void lnet_insert_debugfs(struct ctl_table *table)
 	 * We don't save the dentry returned because we don't call
 	 * debugfs_remove() but rather remove_recursive()
 	 */
-	for (; table->procname; table++)
+	for (; table && table->procname; table++)
 		debugfs_create_file(table->procname, table->mode,
 				    lnet_debugfs_root, table,
 				    lnet_debugfs_fops_select(table->mode));
-- 
1.8.3.1


From jsimmons at infradead.org  Mon Jul  2 23:24:35 2018
From: jsimmons at infradead.org (James Simmons)
Date: Mon,  2 Jul 2018 19:24:35 -0400
Subject: [lustre-devel] [PATCH 18/18] lustre: update version to 2.8.99
In-Reply-To: <1530573875-20465-1-git-send-email-jsimmons@infradead.org>
References: <1530573875-20465-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1530573875-20465-19-git-send-email-jsimmons@infradead.org>

With the missing patches from the lustre 2.9 release merged
upstream its time to update the upstream client's version.

Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/include/uapi/linux/lustre/lustre_ver.h | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/staging/lustre/include/uapi/linux/lustre/lustre_ver.h b/drivers/staging/lustre/include/uapi/linux/lustre/lustre_ver.h
index 19c9135..1428fdd 100644
--- a/drivers/staging/lustre/include/uapi/linux/lustre/lustre_ver.h
+++ b/drivers/staging/lustre/include/uapi/linux/lustre/lustre_ver.h
@@ -2,10 +2,10 @@
 #define _LUSTRE_VER_H_
 
 #define LUSTRE_MAJOR 2
-#define LUSTRE_MINOR 6
+#define LUSTRE_MINOR 8
 #define LUSTRE_PATCH 99
 #define LUSTRE_FIX 0
-#define LUSTRE_VERSION_STRING "2.6.99"
+#define LUSTRE_VERSION_STRING "2.8.99"
 
 #define OBD_OCD_VERSION(major, minor, patch, fix)			\
 	(((major) << 24) + ((minor) << 16) + ((patch) << 8) + (fix))
-- 
1.8.3.1


From jsimmons at infradead.org  Mon Jul  2 23:24:28 2018
From: jsimmons at infradead.org (James Simmons)
Date: Mon,  2 Jul 2018 19:24:28 -0400
Subject: [lustre-devel] [PATCH 11/18] lustre: osc: max_pages_per_rpc should
 be chunk size aligned
In-Reply-To: <1530573875-20465-1-git-send-email-jsimmons@infradead.org>
References: <1530573875-20465-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1530573875-20465-12-git-send-email-jsimmons@infradead.org>

From: Bobi Jam <bobijam at hotmail.com>

max_pages_per_rpc should be chunk size aligned.

obd_brw_size need to be at least one block size.

Improve the LASSERT() to an LASSERTF() that prints the related
parameters to help debug problem.

Signed-off-by: Bobi Jam <bobijam at hotmail.com>
WC-bug-id: https://jira.whamcloud.com/browse/LU-8460
Reviewed-on: http://review.whamcloud.com/21825
Reviewed-by: Andreas Dilger <adilger at whamcloud.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong at gmail.com>
Reviewed-by: Oleg Drokin <green at whamcloud.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/lustre/osc/osc_cache.c   | 11 ++++++++---
 drivers/staging/lustre/lustre/osc/osc_request.c |  5 +++++
 2 files changed, 13 insertions(+), 3 deletions(-)

diff --git a/drivers/staging/lustre/lustre/osc/osc_cache.c b/drivers/staging/lustre/lustre/osc/osc_cache.c
index 8d3f501..15a4817 100644
--- a/drivers/staging/lustre/lustre/osc/osc_cache.c
+++ b/drivers/staging/lustre/lustre/osc/osc_cache.c
@@ -681,15 +681,20 @@ static struct osc_extent *osc_extent_find(const struct lu_env *env,
 	descr = &olck->ols_cl.cls_lock->cll_descr;
 	LASSERT(descr->cld_mode >= CLM_WRITE);
 
-	LASSERT(cli->cl_chunkbits >= PAGE_SHIFT);
+	LASSERTF(cli->cl_chunkbits >= PAGE_SHIFT,
+		 "chunkbits: %u\n", cli->cl_chunkbits);
 	ppc_bits = cli->cl_chunkbits - PAGE_SHIFT;
 	chunk_mask = ~((1 << ppc_bits) - 1);
 	chunksize = 1 << cli->cl_chunkbits;
 	chunk = index >> ppc_bits;
 
-	/* align end to rpc edge, rpc size may not be a power 2 integer. */
+	/* align end to RPC edge */
 	max_pages = cli->cl_max_pages_per_rpc;
-	LASSERT((max_pages & ~chunk_mask) == 0);
+	if ((max_pages & ~chunk_mask) != 0) {
+		CERROR("max_pages: %#x chunkbits: %u chunk_mask: %#lx\n",
+		       max_pages, cli->cl_chunkbits, chunk_mask);
+		return ERR_PTR(-EINVAL);
+	}
 	max_end = index - (index % max_pages) + max_pages - 1;
 	max_end = min_t(pgoff_t, max_end, descr->cld_end);
 
diff --git a/drivers/staging/lustre/lustre/osc/osc_request.c b/drivers/staging/lustre/lustre/osc/osc_request.c
index 0286f25..2d05387 100644
--- a/drivers/staging/lustre/lustre/osc/osc_request.c
+++ b/drivers/staging/lustre/lustre/osc/osc_request.c
@@ -931,6 +931,7 @@ static void osc_init_grant(struct client_obd *cli, struct obd_connect_data *ocd)
 	}
 
 	if (OCD_HAS_FLAG(ocd, GRANT_PARAM)) {
+		int chunk_mask;
 		u64 size;
 
 		/* overhead for each extent insertion */
@@ -938,6 +939,10 @@ static void osc_init_grant(struct client_obd *cli, struct obd_connect_data *ocd)
 		/* determine the appropriate chunk size used by osc_extent. */
 		cli->cl_chunkbits = max_t(int, PAGE_SHIFT,
 					  ocd->ocd_grant_blkbits);
+		/* max_pages_per_rpc must be chunk aligned */
+		chunk_mask = ~((1 << (cli->cl_chunkbits - PAGE_SHIFT)) - 1);
+		cli->cl_max_pages_per_rpc = (cli->cl_max_pages_per_rpc +
+					     ~chunk_mask) & chunk_mask;
 		/* determine maximum extent size, in #pages */
 		size = (u64)ocd->ocd_grant_max_blks << ocd->ocd_grant_blkbits;
 		cli->cl_max_extent_pages = size >> PAGE_SHIFT;
-- 
1.8.3.1


From jsimmons at infradead.org  Mon Jul  2 23:24:44 2018
From: jsimmons at infradead.org (James Simmons)
Date: Mon,  2 Jul 2018 19:24:44 -0400
Subject: [lustre-devel] [PATCH 00/10] lustre: libcfs: fix libcfs debugfs bugs
Message-ID: <1530573894-20592-1-git-send-email-jsimmons@infradead.org>

The original port to sysfs / debugfs was done in haste without any
testing so many bugs were introduced. This patch set resolves the
bugs found in the libcfs code when it was reviewed.

James Simmons (10):
  lustre: libcfs: rename lustre_*_debugfs to lnet_*_debugfs
  lustre: libcfs: rename *_debugmb to *_debug_mb
  lustre: libcfs: make lnet_debugfs_symlink_def fields const
  lustre: libcfs: fix module loading and sysfs setting race
  lustre: libcfs: fix wrong debugfs symlink name
  lustre: libcfs: don't assume sysfs mount point
  lustre: libcfs: properly initialize lnet_debugfs_symlink_def
  lustre: libcfs: test if table is NULL for lnet_insert_debugfs
  lustre: libcfs: return ssize_t for lnet_debugfs_*
  lustre: libcfs: small cleanups for debugfs code

 .../staging/lustre/include/linux/libcfs/libcfs.h   |  2 +-
 drivers/staging/lustre/lnet/libcfs/debug.c         | 28 ++++----
 drivers/staging/lustre/lnet/libcfs/module.c        | 78 +++++++++++-----------
 drivers/staging/lustre/lnet/lnet/router_proc.c     |  2 +-
 4 files changed, 57 insertions(+), 53 deletions(-)

-- 
1.8.3.1


From jsimmons at infradead.org  Mon Jul  2 23:24:45 2018
From: jsimmons at infradead.org (James Simmons)
Date: Mon,  2 Jul 2018 19:24:45 -0400
Subject: [lustre-devel] [PATCH 01/10] lustre: libcfs: rename
 lustre_*_debugfs to lnet_*_debugfs
In-Reply-To: <1530573894-20592-1-git-send-email-jsimmons@infradead.org>
References: <1530573894-20592-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1530573894-20592-2-git-send-email-jsimmons@infradead.org>

Both lustre_insert_debugfs() and lustre_remove_debugfs() are only
used by the LNet layer so replace lustre_* with lnet_* to make
the code more consistent.

Signed-off-by: James Simmons <uja.ornl at yahoo.com>
WC-bug-id: https://jira.whamcloud.com/browse/LU-8066
Reviewed-on: https://review.whamcloud.com/24688
Reviewed-by: Dmitry Eremin <dmitry.eremin at intel.com>
Reviewed-by: Olaf Weber <olaf.weber at hpe.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/include/linux/libcfs/libcfs.h |  2 +-
 drivers/staging/lustre/lnet/libcfs/module.c          | 14 +++++++-------
 drivers/staging/lustre/lnet/lnet/router_proc.c       |  2 +-
 3 files changed, 9 insertions(+), 9 deletions(-)

diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs.h b/drivers/staging/lustre/include/linux/libcfs/libcfs.h
index 7ac6093..19b2141 100644
--- a/drivers/staging/lustre/include/linux/libcfs/libcfs.h
+++ b/drivers/staging/lustre/include/linux/libcfs/libcfs.h
@@ -56,7 +56,7 @@ static inline int notifier_from_ioctl_errno(int err)
 
 extern struct workqueue_struct *cfs_rehash_wq;
 
-void lustre_insert_debugfs(struct ctl_table *table);
+void lnet_insert_debugfs(struct ctl_table *table);
 
 /*
  * Memory
diff --git a/drivers/staging/lustre/lnet/libcfs/module.c b/drivers/staging/lustre/lnet/libcfs/module.c
index 35c3959..522650b 100644
--- a/drivers/staging/lustre/lnet/libcfs/module.c
+++ b/drivers/staging/lustre/lnet/libcfs/module.c
@@ -646,7 +646,7 @@ static const struct file_operations *lnet_debugfs_fops_select(umode_t mode)
 	return &lnet_debugfs_file_operations_rw;
 }
 
-void lustre_insert_debugfs(struct ctl_table *table)
+void lnet_insert_debugfs(struct ctl_table *table)
 {
 	if (!lnet_debugfs_root)
 		lnet_debugfs_root = debugfs_create_dir("lnet", NULL);
@@ -664,9 +664,9 @@ void lustre_insert_debugfs(struct ctl_table *table)
 				    lnet_debugfs_root, table,
 				    lnet_debugfs_fops_select(table->mode));
 }
-EXPORT_SYMBOL_GPL(lustre_insert_debugfs);
+EXPORT_SYMBOL_GPL(lnet_insert_debugfs);
 
-static void lustre_insert_debugfs_links(
+static void lnet_insert_debugfs_links(
 	const struct lnet_debugfs_symlink_def *symlinks)
 {
 	for (; symlinks && symlinks->name; symlinks++)
@@ -674,7 +674,7 @@ static void lustre_insert_debugfs_links(
 				       symlinks->target);
 }
 
-static void lustre_remove_debugfs(void)
+static void lnet_remove_debugfs(void)
 {
 	debugfs_remove_recursive(lnet_debugfs_root);
 
@@ -718,9 +718,9 @@ int libcfs_setup(void)
 		goto err;
 	}
 
-	lustre_insert_debugfs(lnet_table);
+	lnet_insert_debugfs(lnet_table);
 	if (!IS_ERR_OR_NULL(lnet_debugfs_root))
-		lustre_insert_debugfs_links(lnet_debugfs_symlinks);
+		lnet_insert_debugfs_links(lnet_debugfs_symlinks);
 
 	CDEBUG(D_OTHER, "portals setup OK\n");
 out:
@@ -754,7 +754,7 @@ static void libcfs_exit(void)
 {
 	int rc;
 
-	lustre_remove_debugfs();
+	lnet_remove_debugfs();
 
 	if (cfs_rehash_wq)
 		destroy_workqueue(cfs_rehash_wq);
diff --git a/drivers/staging/lustre/lnet/lnet/router_proc.c b/drivers/staging/lustre/lnet/lnet/router_proc.c
index 53aa0e7..d779445 100644
--- a/drivers/staging/lustre/lnet/lnet/router_proc.c
+++ b/drivers/staging/lustre/lnet/lnet/router_proc.c
@@ -881,7 +881,7 @@ static int proc_lnet_portal_rotor(struct ctl_table *table, int write,
 
 void lnet_router_debugfs_init(void)
 {
-	lustre_insert_debugfs(lnet_table);
+	lnet_insert_debugfs(lnet_table);
 }
 
 void lnet_router_debugfs_fini(void)
-- 
1.8.3.1


From jsimmons at infradead.org  Mon Jul  2 23:24:46 2018
From: jsimmons at infradead.org (James Simmons)
Date: Mon,  2 Jul 2018 19:24:46 -0400
Subject: [lustre-devel] [PATCH 02/10] lustre: libcfs: rename *_debugmb to
 *_debug_mb
In-Reply-To: <1530573894-20592-1-git-send-email-jsimmons@infradead.org>
References: <1530573894-20592-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1530573894-20592-3-git-send-email-jsimmons@infradead.org>

The module parameter is called libcfs_debug_mb but the code
refers to it as *_debugmb. Change the naming to match the module
parameter.

Signed-off-by: James Simmons <uja.ornl at yahoo.com>
WC-bug-id: https://jira.whamcloud.com/browse/LU-8066
Reviewed-on: https://review.whamcloud.com/24688
Reviewed-by: Dmitry Eremin <dmitry.eremin at intel.com>
Reviewed-by: Olaf Weber <olaf.weber at hpe.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/lnet/libcfs/debug.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/drivers/staging/lustre/lnet/libcfs/debug.c b/drivers/staging/lustre/lnet/libcfs/debug.c
index 71effcf..f2509f4 100644
--- a/drivers/staging/lustre/lnet/libcfs/debug.c
+++ b/drivers/staging/lustre/lnet/libcfs/debug.c
@@ -82,18 +82,18 @@ static int libcfs_param_debug_mb_set(const char *val,
 
 /* While debug_mb setting look like unsigned int, in fact
  * it needs quite a bunch of extra processing, so we define special
- * debugmb parameter type with corresponding methods to handle this case
+ * debug_mb parameter type with corresponding methods to handle this case
  */
-static const struct kernel_param_ops param_ops_debugmb = {
+static const struct kernel_param_ops param_ops_debug_mb = {
 	.set = libcfs_param_debug_mb_set,
 	.get = param_get_uint,
 };
 
-#define param_check_debugmb(name, p) \
+#define param_check_debug_mb(name, p) \
 		__param_check(name, p, unsigned int)
 
 static unsigned int libcfs_debug_mb;
-module_param(libcfs_debug_mb, debugmb, 0644);
+module_param(libcfs_debug_mb, debug_mb, 0644);
 MODULE_PARM_DESC(libcfs_debug_mb, "Total debug buffer size.");
 
 unsigned int libcfs_printk = D_CANTMASK;
-- 
1.8.3.1


From jsimmons at infradead.org  Mon Jul  2 23:24:49 2018
From: jsimmons at infradead.org (James Simmons)
Date: Mon,  2 Jul 2018 19:24:49 -0400
Subject: [lustre-devel] [PATCH 05/10] lustre: libcfs: fix wrong debugfs
 symlink name
In-Reply-To: <1530573894-20592-1-git-send-email-jsimmons@infradead.org>
References: <1530573894-20592-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1530573894-20592-6-git-send-email-jsimmons@infradead.org>

The symlink libcfs_console_backoff is incorrect. Rename to the
correct name console_backoff.

Signed-off-by: James Simmons <uja.ornl at yahoo.com>
WC-bug-id: https://jira.whamcloud.com/browse/LU-8066
Reviewed-on: https://review.whamcloud.com/24688
Reviewed-by: Dmitry Eremin <dmitry.eremin at intel.com>
Reviewed-by: Olaf Weber <olaf.weber at hpe.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/lnet/libcfs/module.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/staging/lustre/lnet/libcfs/module.c b/drivers/staging/lustre/lnet/libcfs/module.c
index f25270a..55bcb89 100644
--- a/drivers/staging/lustre/lnet/libcfs/module.c
+++ b/drivers/staging/lustre/lnet/libcfs/module.c
@@ -568,7 +568,7 @@ static int proc_cpt_distance(struct ctl_table *table, int write,
 	  "/sys/module/libcfs/parameters/libcfs_debug_file_path"},
 	{ "panic_on_lbug",
 	  "/sys/module/libcfs/parameters/libcfs_panic_on_lbug"},
-	{ "libcfs_console_backoff",
+	{ "console_backoff",
 	  "/sys/module/libcfs/parameters/libcfs_console_backoff"},
 	{ "debug_mb",
 	  "/sys/module/libcfs/parameters/libcfs_debug_mb"},
-- 
1.8.3.1


From jsimmons at infradead.org  Mon Jul  2 23:24:29 2018
From: jsimmons at infradead.org (James Simmons)
Date: Mon,  2 Jul 2018 19:24:29 -0400
Subject: [lustre-devel] [PATCH 12/18] lustre: ladvise: Add dontneed advice
 support for ladvise
In-Reply-To: <1530573875-20465-1-git-send-email-jsimmons@infradead.org>
References: <1530573875-20465-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1530573875-20465-13-git-send-email-jsimmons@infradead.org>

From: Li Xi <lixi at ddn.com>

This patch addds DONTNEED advice to ladvise framework. OSS will
cleanup the page cache of the file when this hint is provided.

Signed-off-by: Li Xi <lixi at ddn.com>
Signed-off-by: Gu Zheng <gzheng at ddn.com>
WC-bug-id: https://jira.whamcloud.com/browse/LU-4931
Reviewed-on: http://review.whamcloud.com/20203
Reviewed-by: Wang Shilong <wshilong at ddn.com>
Reviewed-by: Andreas Dilger <adilger at whamcloud.com>
Reviewed-by: Oleg Drokin <green at whamcloud.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/include/uapi/linux/lustre/lustre_user.h | 2 ++
 drivers/staging/lustre/lustre/ptlrpc/wiretest.c                | 2 ++
 2 files changed, 4 insertions(+)

diff --git a/drivers/staging/lustre/include/uapi/linux/lustre/lustre_user.h b/drivers/staging/lustre/include/uapi/linux/lustre/lustre_user.h
index 02b51ca..4fa7796 100644
--- a/drivers/staging/lustre/include/uapi/linux/lustre/lustre_user.h
+++ b/drivers/staging/lustre/include/uapi/linux/lustre/lustre_user.h
@@ -1326,10 +1326,12 @@ struct hsm_copy {
 enum lu_ladvise_type {
 	LU_LADVISE_INVALID	= 0,
 	LU_LADVISE_WILLREAD	= 1,
+	LU_LADVISE_DONTNEED	= 2,
 };
 
 #define LU_LADVISE_NAMES {				\
 	[LU_LADVISE_WILLREAD]	= "willread",		\
+	[LU_LADVISE_DONTNEED]	= "dontneed",		\
 }
 
 /*
diff --git a/drivers/staging/lustre/lustre/ptlrpc/wiretest.c b/drivers/staging/lustre/lustre/ptlrpc/wiretest.c
index aa17d01..61995c3 100644
--- a/drivers/staging/lustre/lustre/ptlrpc/wiretest.c
+++ b/drivers/staging/lustre/lustre/ptlrpc/wiretest.c
@@ -4245,6 +4245,8 @@ void lustre_assert_wire_constants(void)
 		 (long long)(int)sizeof(((struct lu_ladvise *)0)->lla_value4));
 	LASSERTF(LU_LADVISE_WILLREAD == 1, "found %lld\n",
 		 (long long)LU_LADVISE_WILLREAD);
+	LASSERTF(LU_LADVISE_DONTNEED == 2, "found %lld\n",
+		 (long long)LU_LADVISE_DONTNEED);
 
 	/* Checks for struct ladvise_hdr */
 	LASSERTF(LADVISE_MAGIC == 0x1ADF1CE0, "found 0x%.8x\n",
-- 
1.8.3.1


From jsimmons at infradead.org  Mon Jul  2 23:24:47 2018
From: jsimmons at infradead.org (James Simmons)
Date: Mon,  2 Jul 2018 19:24:47 -0400
Subject: [lustre-devel] [PATCH 03/10] lustre: libcfs: make
 lnet_debugfs_symlink_def fields const
In-Reply-To: <1530573894-20592-1-git-send-email-jsimmons@infradead.org>
References: <1530573894-20592-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1530573894-20592-4-git-send-email-jsimmons@infradead.org>

Change the fields in struct lnet_debubfs_symlink_def to
const char *.

Signed-off-by: James Simmons <uja.ornl at yahoo.com>
WC-bug-id: https://jira.whamcloud.com/browse/LU-8066
Reviewed-on: https://review.whamcloud.com/24688
Reviewed-by: Dmitry Eremin <dmitry.eremin at intel.com>
Reviewed-by: Olaf Weber <olaf.weber at hpe.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/lnet/libcfs/module.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/staging/lustre/lnet/libcfs/module.c b/drivers/staging/lustre/lnet/libcfs/module.c
index 522650b..f25270a 100644
--- a/drivers/staging/lustre/lnet/libcfs/module.c
+++ b/drivers/staging/lustre/lnet/libcfs/module.c
@@ -60,8 +60,8 @@
 #include "tracefile.h"
 
 struct lnet_debugfs_symlink_def {
-	char *name;
-	char *target;
+	const char *name;
+	const char *target;
 };
 
 static struct dentry *lnet_debugfs_root;
-- 
1.8.3.1


From jsimmons at infradead.org  Mon Jul  2 23:24:54 2018
From: jsimmons at infradead.org (James Simmons)
Date: Mon,  2 Jul 2018 19:24:54 -0400
Subject: [lustre-devel] [PATCH 10/10] lustre: libcfs: small cleanups for
 debugfs code
In-Reply-To: <1530573894-20592-1-git-send-email-jsimmons@infradead.org>
References: <1530573894-20592-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1530573894-20592-11-git-send-email-jsimmons@infradead.org>

Small cleanups to make the code easier to read.

Signed-off-by: James Simmons <uja.ornl at yahoo.com>
WC-bug-id: https://jira.whamcloud.com/browse/LU-8066
Reviewed-on: https://review.whamcloud.com/24688
Reviewed-by: Dmitry Eremin <dmitry.eremin at intel.com>
Reviewed-by: Olaf Weber <olaf.weber at hpe.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/lnet/libcfs/debug.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/drivers/staging/lustre/lnet/libcfs/debug.c b/drivers/staging/lustre/lnet/libcfs/debug.c
index 5d93341..d925bf5 100644
--- a/drivers/staging/lustre/lnet/libcfs/debug.c
+++ b/drivers/staging/lustre/lnet/libcfs/debug.c
@@ -180,9 +180,11 @@ static int param_set_uint_minmax(const char *val,
 
 	if (!val)
 		return -EINVAL;
+
 	ret = kstrtouint(val, 0, &num);
 	if (ret < 0 || num < min || num > max)
 		return -EINVAL;
+
 	*((unsigned int *)kp->arg) = num;
 	return 0;
 }
@@ -229,8 +231,7 @@ static int param_set_uintpos(const char *val, const struct kernel_param *kp)
 int libcfs_panic_in_progress;
 
 /* libcfs_debug_token2mask() expects the returned string in lower-case */
-static const char *
-libcfs_debug_subsys2str(int subsys)
+static const char *libcfs_debug_subsys2str(int subsys)
 {
 	static const char * const libcfs_debug_subsystems[] =
 		LIBCFS_DEBUG_SUBSYS_NAMES;
@@ -242,8 +243,7 @@ static int param_set_uintpos(const char *val, const struct kernel_param *kp)
 }
 
 /* libcfs_debug_token2mask() expects the returned string in lower-case */
-static const char *
-libcfs_debug_dbg2str(int debug)
+static const char *libcfs_debug_dbg2str(int debug)
 {
 	static const char * const libcfs_debug_masks[] =
 		LIBCFS_DEBUG_MASKS_NAMES;
-- 
1.8.3.1


From jsimmons at infradead.org  Mon Jul  2 23:24:53 2018
From: jsimmons at infradead.org (James Simmons)
Date: Mon,  2 Jul 2018 19:24:53 -0400
Subject: [lustre-devel] [PATCH 09/10] lustre: libcfs: return ssize_t for
 lnet_debugfs_*
In-Reply-To: <1530573894-20592-1-git-send-email-jsimmons@infradead.org>
References: <1530573894-20592-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1530573894-20592-10-git-send-email-jsimmons@infradead.org>

The debugfs layer expects both lnet_debugfs_read() and
lnet_debugfs_write() to return ssize_t instead of int.
Change the return type.

Signed-off-by: James Simmons <uja.ornl at yahoo.com>
WC-bug-id: https://jira.whamcloud.com/browse/LU-8066
Reviewed-on: https://review.whamcloud.com/24688
Reviewed-by: Dmitry Eremin <dmitry.eremin at intel.com>
Reviewed-by: Olaf Weber <olaf.weber at hpe.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/lnet/libcfs/module.c | 30 ++++++++++++++---------------
 1 file changed, 15 insertions(+), 15 deletions(-)

diff --git a/drivers/staging/lustre/lnet/libcfs/module.c b/drivers/staging/lustre/lnet/libcfs/module.c
index 0d36121..793cc03 100644
--- a/drivers/staging/lustre/lnet/libcfs/module.c
+++ b/drivers/staging/lustre/lnet/libcfs/module.c
@@ -583,37 +583,37 @@ static ssize_t lnet_debugfs_read(struct file *filp, char __user *buf,
 				 size_t count, loff_t *ppos)
 {
 	struct ctl_table *table = filp->private_data;
-	int error;
 	loff_t old_pos = *ppos;
+	ssize_t rc;
 
-	error = table->proc_handler(table, 0, (void __user *)buf, &count, ppos);
+	rc = table->proc_handler(table, 0, (void __user *)buf, &count, ppos);
 	/*
 	 * On success, the length read is either in error or in count.
 	 * If ppos changed, then use count, else use error
 	 */
-	if (!error && *ppos != old_pos)
-		error = count;
-	else if (error > 0)
-		*ppos += error;
+	if (!rc && *ppos != old_pos)
+		rc = count;
+	else if (rc > 0)
+		*ppos += rc;
 
-	return error;
+	return rc;
 }
 
 static ssize_t lnet_debugfs_write(struct file *filp, const char __user *buf,
 				  size_t count, loff_t *ppos)
 {
 	struct ctl_table *table = filp->private_data;
-	int error;
 	loff_t old_pos = *ppos;
+	ssize_t rc;
 
-	error = table->proc_handler(table, 1, (void __user *)buf, &count, ppos);
-	if (!error) {
-		error = count;
-		if (*ppos == old_pos)
-			*ppos += count;
-	}
+	rc = table->proc_handler(table, 1, (void __user *)buf, &count, ppos);
+	if (rc)
+		return rc;
+
+	if (*ppos == old_pos)
+		*ppos += count;
 
-	return error;
+	return count;
 }
 
 static const struct file_operations lnet_debugfs_file_operations_rw = {
-- 
1.8.3.1


From jsimmons at infradead.org  Mon Jul  2 23:24:23 2018
From: jsimmons at infradead.org (James Simmons)
Date: Mon,  2 Jul 2018 19:24:23 -0400
Subject: [lustre-devel] [PATCH 06/18] lustre: obd: reserve connection flag
 OBD_CONNECT2_FILE_SECCTX
In-Reply-To: <1530573875-20465-1-git-send-email-jsimmons@infradead.org>
References: <1530573875-20465-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1530573875-20465-7-git-send-email-jsimmons@infradead.org>

From: "John L. Hammond" <jhammond at whamcloud.com>

The connection flag OBD_CONNECT2_FILE_SECCTX will be set (in
ocd_connect_flags2) if an MDT supports setting the file security
context at create time.

Signed-off-by: John L. Hammond <jhammond at whamcloud.com>
Signed-off-by: Sebastien Buisson <sbuisson at ddn.com>
WC-bug-id: https://jira.whamcloud.com/browse/LU-5560
Reviewed-on: http://review.whamcloud.com/19970
Reviewed-by: Fan Yong <fan.yong at intel.com>
Reviewed-by: Andreas Dilger <adilger at whamcloud.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 .../lustre/include/uapi/linux/lustre/lustre_idl.h  |  4 ++
 .../staging/lustre/lustre/include/lustre_import.h  |  1 +
 drivers/staging/lustre/lustre/include/obd_class.h  |  3 +-
 drivers/staging/lustre/lustre/ldlm/ldlm_lib.c      |  1 +
 drivers/staging/lustre/lustre/llite/llite_lib.c    |  9 +++-
 .../lustre/lustre/obdclass/lprocfs_status.c        | 62 ++++++++++++++++++----
 drivers/staging/lustre/lustre/ptlrpc/import.c      | 10 ++++
 drivers/staging/lustre/lustre/ptlrpc/wiretest.c    |  2 +
 8 files changed, 80 insertions(+), 12 deletions(-)

diff --git a/drivers/staging/lustre/include/uapi/linux/lustre/lustre_idl.h b/drivers/staging/lustre/include/uapi/linux/lustre/lustre_idl.h
index 6defc6d..4e25521 100644
--- a/drivers/staging/lustre/include/uapi/linux/lustre/lustre_idl.h
+++ b/drivers/staging/lustre/include/uapi/linux/lustre/lustre_idl.h
@@ -695,6 +695,10 @@ struct ptlrpc_body_v2 {
 #define OBD_CONNECT_BULK_MBITS	 0x2000000000000000ULL
 #define OBD_CONNECT_OBDOPACK	 0x4000000000000000ULL /* compact OUT obdo */
 #define OBD_CONNECT_FLAGS2	 0x8000000000000000ULL /* second flags word */
+/* ocd_connect_flags2 flags */
+#define OBD_CONNECT2_FILE_SECCTX	0x1ULL		/* set file security
+							 * context at create
+							 */
 
 /* XXX README XXX:
  * Please DO NOT add flag values here before first ensuring that this same
diff --git a/drivers/staging/lustre/lustre/include/lustre_import.h b/drivers/staging/lustre/lustre/include/lustre_import.h
index ac3805e..a629f6b 100644
--- a/drivers/staging/lustre/lustre/include/lustre_import.h
+++ b/drivers/staging/lustre/lustre/include/lustre_import.h
@@ -307,6 +307,7 @@ struct obd_import {
 	__u32		     imp_connect_op;
 	struct obd_connect_data   imp_connect_data;
 	__u64		     imp_connect_flags_orig;
+	u64			imp_connect_flags2_orig;
 	int		       imp_connect_error;
 
 	__u32		     imp_msg_magic;
diff --git a/drivers/staging/lustre/lustre/include/obd_class.h b/drivers/staging/lustre/lustre/include/obd_class.h
index 20d07f8..adfe2ab 100644
--- a/drivers/staging/lustre/lustre/include/obd_class.h
+++ b/drivers/staging/lustre/lustre/include/obd_class.h
@@ -80,7 +80,8 @@ struct obd_device *class_devices_in_group(struct obd_uuid *grp_uuid,
 
 int class_notify_sptlrpc_conf(const char *fsname, int namelen);
 
-int obd_connect_flags2str(char *page, int count, __u64 flags, char *sep);
+int obd_connect_flags2str(char *page, int count, u64 flags, u64 flags2,
+			  const char *sep);
 
 int obd_zombie_impexp_init(void);
 void obd_zombie_impexp_stop(void);
diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_lib.c b/drivers/staging/lustre/lustre/ldlm/ldlm_lib.c
index 0aa4f23..07baea7 100644
--- a/drivers/staging/lustre/lustre/ldlm/ldlm_lib.c
+++ b/drivers/staging/lustre/lustre/ldlm/ldlm_lib.c
@@ -518,6 +518,7 @@ int client_connect_import(const struct lu_env *env,
 		if (is_mdc)
 			data->ocd_connect_flags |= OBD_CONNECT_MULTIMODRPCS;
 		imp->imp_connect_flags_orig = data->ocd_connect_flags;
+		imp->imp_connect_flags2_orig = data->ocd_connect_flags2;
 	}
 
 	rc = ptlrpc_connect_import(imp);
diff --git a/drivers/staging/lustre/lustre/llite/llite_lib.c b/drivers/staging/lustre/lustre/llite/llite_lib.c
index 6e47e5b..640205a 100644
--- a/drivers/staging/lustre/lustre/llite/llite_lib.c
+++ b/drivers/staging/lustre/lustre/llite/llite_lib.c
@@ -203,7 +203,10 @@ static int client_common_fill_super(struct super_block *sb, char *md, char *dt)
 				  OBD_CONNECT_OPEN_BY_FID |
 				  OBD_CONNECT_DIR_STRIPE |
 				  OBD_CONNECT_BULK_MBITS |
-				  OBD_CONNECT_SUBTREE;
+				  OBD_CONNECT_SUBTREE |
+				  OBD_CONNECT_FLAGS2;
+
+	data->ocd_connect_flags2 = 0;
 
 	if (sbi->ll_flags & LL_SBI_LRU_RESIZE)
 		data->ocd_connect_flags |= OBD_CONNECT_LRU_RESIZE;
@@ -292,7 +295,7 @@ static int client_common_fill_super(struct super_block *sb, char *md, char *dt)
 			goto out_md_fid;
 		}
 		obd_connect_flags2str(buf, PAGE_SIZE,
-				      valid ^ CLIENT_CONNECT_MDT_REQD, ",");
+				      valid ^ CLIENT_CONNECT_MDT_REQD, 0, ",");
 		LCONSOLE_ERROR_MSG(0x170,
 				   "Server %s does not support feature(s) needed for correct operation of this client (%s). Please upgrade server or downgrade client.\n",
 				   sbi->ll_md_exp->exp_obd->obd_name, buf);
@@ -375,6 +378,8 @@ static int client_common_fill_super(struct super_block *sb, char *md, char *dt)
 				  OBD_CONNECT_PINGLESS | OBD_CONNECT_LFSCK |
 				  OBD_CONNECT_BULK_MBITS;
 
+	data->ocd_connect_flags2 = 0;
+
 	if (!OBD_FAIL_CHECK(OBD_FAIL_OSC_CONNECT_GRANT_PARAM))
 		data->ocd_connect_flags |= OBD_CONNECT_GRANT_PARAM;
 
diff --git a/drivers/staging/lustre/lustre/obdclass/lprocfs_status.c b/drivers/staging/lustre/lustre/obdclass/lprocfs_status.c
index dd88179..9f76d8a 100644
--- a/drivers/staging/lustre/lustre/obdclass/lprocfs_status.c
+++ b/drivers/staging/lustre/lustre/obdclass/lprocfs_status.c
@@ -45,6 +45,7 @@
 #include <linux/ctype.h>
 
 static const char * const obd_connect_names[] = {
+	/* flags names */
 	"read_only",
 	"lov_index",
 	"connect_from_mds",
@@ -109,23 +110,42 @@
 	"bulk_mbits",
 	"compact_obdo",
 	"second_flags",
+	/* flags2 names */
+	"file_secctx",
 	NULL
 };
 
-int obd_connect_flags2str(char *page, int count, __u64 flags, char *sep)
+int obd_connect_flags2str(char *page, int count, u64 flags, u64 flags2,
+			  const char *sep)
 {
-	__u64 mask = 1;
+	__u64 mask;
 	int i, ret = 0;
 
-	for (i = 0; obd_connect_names[i]; i++, mask <<= 1) {
+	for (i = 0, mask = 1; i < 64; i++, mask <<= 1) {
 		if (flags & mask)
 			ret += snprintf(page + ret, count - ret, "%s%s",
 					ret ? sep : "", obd_connect_names[i]);
 	}
+
 	if (flags & ~(mask - 1))
 		ret += snprintf(page + ret, count - ret,
 				"%sunknown flags %#llx",
 				ret ? sep : "", flags & ~(mask - 1));
+
+	if (!(flags & OBD_CONNECT_FLAGS2) || flags2 == 0)
+		return ret;
+
+	for (i = 64, mask = 1; obd_connect_names[i]; i++, mask <<= 1) {
+		if (flags2 & mask)
+			ret += snprintf(page + ret, count - ret, "%s%s",
+					ret ? sep : "", obd_connect_names[i]);
+	}
+
+	if (flags2 & ~(mask - 1))
+		ret += snprintf(page + ret, count - ret,
+				"%sunknown2_%#llx",
+				ret ? sep : "", flags2 & ~(mask - 1));
+
 	return ret;
 }
 EXPORT_SYMBOL(obd_connect_flags2str);
@@ -659,22 +679,43 @@ static int obd_import_flags2str(struct obd_import *imp, struct seq_file *m)
 
 #undef flags2str
 
-static void obd_connect_seq_flags2str(struct seq_file *m, __u64 flags, char *sep)
+static void obd_connect_seq_flags2str(struct seq_file *m, u64 flags,
+				      u64 flags2, const char *sep)
 {
-	__u64 mask = 1;
+	__u64 mask;
 	int i;
 	bool first = true;
 
-	for (i = 0; obd_connect_names[i]; i++, mask <<= 1) {
+	for (i = 0, mask = 1; i < 64; i++, mask <<= 1) {
 		if (flags & mask) {
 			seq_printf(m, "%s%s",
 				   first ? sep : "", obd_connect_names[i]);
 			first = false;
 		}
 	}
-	if (flags & ~(mask - 1))
+
+	if (flags & ~(mask - 1)) {
 		seq_printf(m, "%sunknown flags %#llx",
 			   first ? sep : "", flags & ~(mask - 1));
+		first = false;
+	}
+
+	if (!(flags & OBD_CONNECT_FLAGS2) || flags2 == 0)
+		return;
+
+	for (i = 64, mask = 1; obd_connect_names[i]; i++, mask <<= 1) {
+		if (flags2 & mask) {
+			seq_printf(m, "%s%s",
+				   first ? "" : sep, obd_connect_names[i]);
+			first = false;
+		}
+	}
+
+	if (flags2 & ~(mask - 1)) {
+		seq_printf(m, "%sunknown2_%#llx",
+			   first ? "" : sep, flags2 & ~(mask - 1));
+		first = false;
+	}
 }
 
 int lprocfs_rd_import(struct seq_file *m, void *data)
@@ -710,6 +751,7 @@ int lprocfs_rd_import(struct seq_file *m, void *data)
 		   ptlrpc_import_state_name(imp->imp_state),
 		   imp->imp_connect_data.ocd_instance);
 	obd_connect_seq_flags2str(m, imp->imp_connect_data.ocd_connect_flags,
+				  imp->imp_connect_data.ocd_connect_flags2,
 				  ", ");
 	seq_puts(m, " ]\n");
 	obd_connect_data_seqprint(m, ocd);
@@ -932,7 +974,7 @@ int lprocfs_rd_timeouts(struct seq_file *m, void *data)
 int lprocfs_rd_connect_flags(struct seq_file *m, void *data)
 {
 	struct obd_device *obd = data;
-	__u64 flags;
+	__u64 flags, flags2;
 	int rc;
 
 	rc = lprocfs_climp_check(obd);
@@ -940,8 +982,10 @@ int lprocfs_rd_connect_flags(struct seq_file *m, void *data)
 		return rc;
 
 	flags = obd->u.cli.cl_import->imp_connect_data.ocd_connect_flags;
+	flags2 = obd->u.cli.cl_import->imp_connect_data.ocd_connect_flags2;
 	seq_printf(m, "flags=%#llx\n", flags);
-	obd_connect_seq_flags2str(m, flags, "\n");
+	seq_printf(m, "flags2=%#llx\n", flags2);
+	obd_connect_seq_flags2str(m, flags, flags2, "\n");
 	seq_puts(m, "\n");
 	up_read(&obd->u.cli.cl_sem);
 	return 0;
diff --git a/drivers/staging/lustre/lustre/ptlrpc/import.c b/drivers/staging/lustre/lustre/ptlrpc/import.c
index 54ceac5..4db0d89 100644
--- a/drivers/staging/lustre/lustre/ptlrpc/import.c
+++ b/drivers/staging/lustre/lustre/ptlrpc/import.c
@@ -641,6 +641,7 @@ int ptlrpc_connect_import(struct obd_import *imp)
 	 * the server is updated on-the-fly we will get the new features.
 	 */
 	imp->imp_connect_data.ocd_connect_flags = imp->imp_connect_flags_orig;
+	imp->imp_connect_data.ocd_connect_flags2 = imp->imp_connect_flags2_orig;
 	/* Reset ocd_version each time so the server knows the exact versions */
 	imp->imp_connect_data.ocd_version = LUSTRE_VERSION_CODE;
 	imp->imp_msghdr_flags &= ~MSGHDR_AT_SUPPORT;
@@ -1019,6 +1020,15 @@ static int ptlrpc_connect_interpret(const struct lu_env *env,
 		goto out;
 	}
 
+	if ((ocd->ocd_connect_flags2 & imp->imp_connect_flags2_orig) !=
+	    ocd->ocd_connect_flags2) {
+		CERROR("%s: Server didn't grant requested subset of flags2: asked=%#llx granted=%#llx\n",
+		       imp->imp_obd->obd_name, imp->imp_connect_flags2_orig,
+		       ocd->ocd_connect_flags2);
+		rc = -EPROTO;
+		goto out;
+	}
+
 	old_connect_flags = exp_connect_flags(exp);
 	exp->exp_connect_data = *ocd;
 	imp->imp_obd->obd_self_export->exp_connect_data = *ocd;
diff --git a/drivers/staging/lustre/lustre/ptlrpc/wiretest.c b/drivers/staging/lustre/lustre/ptlrpc/wiretest.c
index 43931dd..dae1b09 100644
--- a/drivers/staging/lustre/lustre/ptlrpc/wiretest.c
+++ b/drivers/staging/lustre/lustre/ptlrpc/wiretest.c
@@ -1116,6 +1116,8 @@ void lustre_assert_wire_constants(void)
 		 OBD_CONNECT_OBDOPACK);
 	LASSERTF(OBD_CONNECT_FLAGS2 == 0x8000000000000000ULL, "found 0x%.16llxULL\n",
 		 OBD_CONNECT_FLAGS2);
+	LASSERTF(OBD_CONNECT2_FILE_SECCTX == 0x1ULL, "found 0x%.16llxULL\n",
+		 OBD_CONNECT2_FILE_SECCTX);
 	LASSERTF(OBD_CKSUM_CRC32 == 0x00000001UL, "found 0x%.8xUL\n",
 		 (unsigned int)OBD_CKSUM_CRC32);
 	LASSERTF(OBD_CKSUM_ADLER == 0x00000002UL, "found 0x%.8xUL\n",
-- 
1.8.3.1


From neilb at suse.com  Tue Jul  3 03:04:48 2018
From: neilb at suse.com (NeilBrown)
Date: Tue, 03 Jul 2018 13:04:48 +1000
Subject: [lustre-devel] [PATCH 01/18] lustre: grant: add support for
 OBD_CONNECT_GRANT_PARAM
In-Reply-To: <1530573875-20465-2-git-send-email-jsimmons@infradead.org>
References: <1530573875-20465-1-git-send-email-jsimmons@infradead.org>
 <1530573875-20465-2-git-send-email-jsimmons@infradead.org>
Message-ID: <87601x2f5b.fsf@notabene.neil.brown.name>

On Mon, Jul 02 2018, James Simmons wrote:

> From: Johann Lombardi <jlombardi at whamcloud.com>
>
> Add support for grant overhead calculation on the client side.
> To do so, clients track usage on a per-extent basis. An extent is
> composed of contiguous blocks.
> The OST now returns to the OSC layer several parameters to consume
> grant more accurately:

Just to be sure that I'm understanding correctly, should "consume"
above be "compute" ??

>
> - the backend filesystem block size which is the minimal grant
>   allocation unit;
> - the maximum extent size;
> - the extent insertion cost.
>   Clients now pack in bulk write how much grant space was consumed for
>   the RPC. Dirty data accounting also adopts the same scheme.
>
> Moreover, each backend OSD now reports its own set of parameters:
> - For ldiskfs, we usually have a 4KB block size with a maximum extent
>   size of 32MB (theoretical limit of 128MB) and an extent insertion
>   cost of 6 x 4KB = 24KB
> - For ZFS, we report a block size of 128KB, an extent size of 128
>   blocks (i.e. 16MB with 128KB block size) and a block insertion cost
>   of 112KB.
>
> Besides, there is now no more generic metadata overhead reservation
> done inside each OSD. Instead grant space is inflated for clients
> that do not support the new grant parameters. That said, a tiny
> percentage (typically 0.76%) of the free space is still reserved
> inside each OSD to avoid fragmentation which might hurt performance
> and impact our grant calculation (e.g. extents are broken due to
> fragmentation).
>
> This patch also fixes several other issues:
>
> - Bulk write resent by ptlrpc after reconnection could trigger
>   spurious error messages related to broken dirty accounting.
>   The issue was that oa_dirty is discarded for resent requests
>   (grant flag cleared in ost_brw_write()), so we can legitimately
>   have grant > fed_dirty in ofd_grant_check().
>   This was fixed by resetting fed_dirty on reconnection and skipping
>   the dirty accounting check in ofd_grant_check() in the case of
>   ptlrpc resend.

ofd_grant_check doesn't appear in this patch.  Presumably it was part of
the server code that was stripped from the patch.

>
> - In obd_connect_data_seqprint(), the connection flags cannot fit
>   in a 32-bit integer.

I cannot see what this might refer to either.

>
> - When merging two OSC extents, an extent tax should be released
>   in both the merged extent and in the grant accounting.

I think I might see what this relates to.

> +static ssize_t cur_dirty_grant_bytes_show(struct kobject *kobj,
> +					  struct attribute *attr,
> +					  char *buf)
> +{
> +	struct obd_device *dev = container_of(kobj, struct obd_device,
> +					      obd_kobj);
> +	struct client_obd *cli = &dev->u.cli;
> +	int len;
> +
> +	spin_lock(&cli->cl_loi_list_lock);
> +	len = sprintf(buf, "%lu\n", cli->cl_dirty_grant);
> +	spin_unlock(&cli->cl_loi_list_lock);

Why take the spinlock?? cl_dirty_grant is 'long', so reading it is
atomic.


> +		if (OCD_HAS_FLAG(&cli->cl_import->imp_connect_data,
> +				 GRANT_PARAM)) {
> +			int nrextents;
> +
> +			/*
> +			 * take extent tax into account when asking for more
> +			 * grant space
> +			 */
> +			nrextents = (nrpages + cli->cl_max_extent_pages - 1)  /
> +				     cli->cl_max_extent_pages;

I generally prefer DIV_ROUND_UP() for this.


I don't think any of the above will stop me from applying the patch,
though I might edit the description, and may add a fix-up patch afterwards.

Thanks.
NeilBrown
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180703/348eec1f/attachment.sig>

From neilb at suse.com  Tue Jul  3 03:32:45 2018
From: neilb at suse.com (NeilBrown)
Date: Tue, 03 Jul 2018 13:32:45 +1000
Subject: [lustre-devel] [PATCH 02/18] lustre: ladvise: Add feature of
 giving file access advices
In-Reply-To: <1530573875-20465-3-git-send-email-jsimmons@infradead.org>
References: <1530573875-20465-1-git-send-email-jsimmons@infradead.org>
 <1530573875-20465-3-git-send-email-jsimmons@infradead.org>
Message-ID: <8736x12duq.fsf@notabene.neil.brown.name>

On Mon, Jul 02 2018, James Simmons wrote:

> From: Li Xi <lixi at ddn.com>
>
> The fadvise() system call provided by Linux kernel enables
> applications to give advices or hints about how a file will be
> accessed. However, It is only a client side mechanism which is
> not enough for distributed file systems like Lustre, because in
> order to tune system-wide cache or read-ahead policies, servers
> need to understand the advices too.
>
> This patch adds a new feature named ladvise which provides new
> APIs and utils to give advices about the access pattern of Lustre
> files with the purpose of performance improvement. It is similar
> to Linux fadvise() system call, except it forwards the advices
> directly from Lustre client to server. The server side codes will
> apply appropriate read-ahead and caching techniques for the
> corresponding files.
>
> A typical workload for ladvise is e.g. a bunch of different
> clients are doing small random reads of a file, so prefetching
> pages into OSS cache with big linear reads before the random IO
> is a net benefit. Fetching all that data into each client cache
> with fadvise() may not be, due to much more data being sent to
> the client.
>
...
> +	case LL_IOC_LADVISE: {
> +		struct ladvise_hdr *ladvise_hdr;
> +		int alloc_size = sizeof(*ladvise_hdr);
> +		int num_advise;
> +		int i;
> +
> +		rc = 0;
> +		ladvise_hdr = kzalloc(alloc_size, GFP_NOFS);

Lustre really needs to get more sensible about when to use GFP_NOFS and
when to use GFP_KERNEL.
NOFS is only needed when the allocation cannot wait while reclaim enters
the filesystem, due to possible deadlock.  So if you are holding a lock
or some other resource that writeout might need, use NOFS.
In this case, there is no possible conflict with writeout, so GFP_KERNEL
should be used.

In this case, alloc_size is 32, so I wonder why it is allocating memory
at all rather than just using the stack.

Below the memory is freed and allocated again (why not realloc?) so an
allocation is justified there.  That should be GFP_KERNEL as well.

Later in osc_io_ladvise_start() there is a kvzalloc() with GFP_NOFS.
I suspect that should be GFP_KERNEL as well, especially as vmalloc
doesn't properly support GFP_NOFS and never has.


Again, I'll probably apply this anyway...

NeilBrown
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180703/1f610b00/attachment.sig>

From neilb at suse.com  Tue Jul  3 03:52:37 2018
From: neilb at suse.com (NeilBrown)
Date: Tue, 03 Jul 2018 13:52:37 +1000
Subject: [lustre-devel] [PATCH 03/18] lustre: fileset: add fileset mount
 support
In-Reply-To: <1530573875-20465-4-git-send-email-jsimmons@infradead.org>
References: <1530573875-20465-1-git-send-email-jsimmons@infradead.org>
 <1530573875-20465-4-git-send-email-jsimmons@infradead.org>
Message-ID: <87zhz90yd6.fsf@notabene.neil.brown.name>

On Mon, Jul 02 2018, James Simmons wrote:

> From: Lai Siyao <lsiyao at whamcloud.com>
>
> This patch enables client to mount subdirectory as fileset.
>
> usage: mount -t lustre mgsname:/fsname/subdir /mount/point
>
>  * mdt lookup fileset fid and return to client during mount.
>  * `fid2path` support for fileset.
>

> @@ -2626,7 +2626,10 @@ struct getinfo_fid2path {
>  	__u64	   gf_recno;
>  	__u32	   gf_linkno;
>  	__u32	   gf_pathlen;
> -	char	    gf_path[0];
> +	union {
> +		char		gf_path[0];
> +		struct lu_fid	gf_root_fid[0];
> +	} gf_u;

If you just left the "gf_u" off here, you wouldn't need changes like:

> -		ptr = ori_gf->gf_path;
> +		ptr = ori_gf->gf_u.gf_path;

throughout the code.

I'll probably make this change before applying the patch.
The rest can come afterwards.

>  } __packed;
>  
>  /** path2parent request/reply structures */
> diff --git a/drivers/staging/lustre/lustre/include/lustre_disk.h b/drivers/staging/lustre/lustre/include/lustre_disk.h
> index 886e817..bd8fa71 100644
> --- a/drivers/staging/lustre/lustre/include/lustre_disk.h
> +++ b/drivers/staging/lustre/lustre/include/lustre_disk.h
> @@ -78,6 +78,7 @@ struct lustre_mount_data {
>  	int	lmd_recovery_time_hard;
>  	char      *lmd_dev;	   /* device name */
>  	char      *lmd_profile;    /* client only */
> +	char	*lmd_fileset;	/* mount fileset */

Consistent indenting is so over-rated :-)
I really that there is *nothing* you could have done here to make
the indenting consistent without including irrelevant changes in the
patch, which is generally frowned upon.
This is why it is not unusually to have some clean-up patches first,
so that you functional changes can be applied to clean code.

>  
> +	if (fileset)
> +		req_capsule_set_size(&req->rq_pill, &RMF_NAME, RCL_CLIENT,
> +				     strlen(fileset) + 1);
> +	rc = ptlrpc_request_pack(req, LUSTRE_MDS_VERSION, MDS_GET_ROOT);
> +	if (rc) {
> +		ptlrpc_request_free(req);
> +		return rc;
> +	}
>  	mdc_pack_body(req, NULL, 0, 0, -1, 0);
> +	if (fileset) {
> +		char *name = req_capsule_client_get(&req->rq_pill, &RMF_NAME);
> +
> +		memcpy(name, fileset, strlen(fileset));

Why isn't this strcpy()?
I realize that strcpy will add a nul that memcpy doesn't add so if I
could see the length being stored, I would understand.
But above I see the size passed to req_capsule_set_size() includes
the trailing nul.

Am I confused, or is the code confusing?


> @@ -1073,10 +1074,30 @@ static int lmd_parse(char *options, struct lustre_mount_data *lmd)
>  		/* Remove leading /s from fsname */
>  		while (*++s1 == '/')
>  			;
> +		s2 = s1;
> +		while (*s2 != '/' && *s2 != '\0')
> +			s2++;

 s2 = strchrnul(s1, '/');

(I only discovered strchrnul() a few days ago).

>  		/* Freed in lustre_free_lsi */
> -		lmd->lmd_profile = kasprintf(GFP_NOFS, "%s-client", s1);
> +		lmd->lmd_profile = kzalloc(s2 - s1 + 8, GFP_NOFS);
>  		if (!lmd->lmd_profile)
>  			return -ENOMEM;
> +
> +		strncat(lmd->lmd_profile, s1, s2 - s1);
> +		strncat(lmd->lmd_profile, "-client", 7);

  kasprintf(GFP_KERN, "%.*s-client", s2-s1, s1);

??

> +
> +		s1 = s2;
> +		s2 = s1 + strlen(s1) - 1;
> +		/* Remove padding /s from fileset */
> +		while (*s2 == '/')
> +			s2--;
> +		if (s2 > s1) {
> +			lmd->lmd_fileset = kzalloc(s2 - s1 + 2, GFP_NOFS);
> +			if (!lmd->lmd_fileset) {
> +				kfree(lmd->lmd_profile);
> +				return -ENOMEM;
> +			}
> +			strncat(lmd->lmd_fileset, s1, s2 - s1 + 1);

This probably deserves to be kasprintf() too.

Thanks,
NeilBrown
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180703/bddfa938/attachment.sig>

From neilb at suse.com  Tue Jul  3 04:10:54 2018
From: neilb at suse.com (NeilBrown)
Date: Tue, 03 Jul 2018 14:10:54 +1000
Subject: [lustre-devel] [PATCH 05/18] lustre: llite: fast read
 implementation
In-Reply-To: <1530573875-20465-6-git-send-email-jsimmons@infradead.org>
References: <1530573875-20465-1-git-send-email-jsimmons@infradead.org>
 <1530573875-20465-6-git-send-email-jsimmons@infradead.org>
Message-ID: <87woud0xip.fsf@notabene.neil.brown.name>

On Mon, Jul 02 2018, James Simmons wrote:

> +static ssize_t fast_read_store(struct kobject *kobj,
> +			       struct attribute *attr,
> +			       const char *buffer,
> +			       size_t count)
> +{
> +	struct ll_sb_info *sbi = container_of(kobj, struct ll_sb_info,
> +					      ll_kobj);
> +	bool val;
> +	int rc;
> +
> +	rc = kstrtobool(buffer, &val);
> +	if (rc)
> +		return rc;
> +
> +	spin_lock(&sbi->ll_lock);
> +	if (val)
> +		sbi->ll_flags |= LL_SBI_FAST_READ;
> +	else
> +		sbi->ll_flags &= ~LL_SBI_FAST_READ;
> +	spin_unlock(&sbi->ll_lock);

This is a little odd... no other code uses ll_lock to protect updates to
ll_flags.  Various other places (e.g. checksum_pages_store()) updates it
unprotected.

Maybe there all need to use ll_lock for protection.


Thanks,
NeilBrown
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180703/0137e8ad/attachment.sig>

From neilb at suse.com  Tue Jul  3 04:20:06 2018
From: neilb at suse.com (NeilBrown)
Date: Tue, 03 Jul 2018 14:20:06 +1000
Subject: [lustre-devel] [PATCH 06/18] lustre: obd: reserve connection
 flag OBD_CONNECT2_FILE_SECCTX
In-Reply-To: <1530573875-20465-7-git-send-email-jsimmons@infradead.org>
References: <1530573875-20465-1-git-send-email-jsimmons@infradead.org>
 <1530573875-20465-7-git-send-email-jsimmons@infradead.org>
Message-ID: <87tvph0x3d.fsf@notabene.neil.brown.name>

On Mon, Jul 02 2018, James Simmons wrote:

> From: "John L. Hammond" <jhammond at whamcloud.com>
>
> The connection flag OBD_CONNECT2_FILE_SECCTX will be set (in
> ocd_connect_flags2) if an MDT supports setting the file security
> context at create time.
>

>  
> -static void obd_connect_seq_flags2str(struct seq_file *m, __u64 flags, char *sep)
> +static void obd_connect_seq_flags2str(struct seq_file *m, u64 flags,
> +				      u64 flags2, const char *sep)
>  {
> -	__u64 mask = 1;
> +	__u64 mask;
>  	int i;
>  	bool first = true;
>  
> -	for (i = 0; obd_connect_names[i]; i++, mask <<= 1) {
> +	for (i = 0, mask = 1; i < 64; i++, mask <<= 1) {

This changes where the loop stops.  Previously it stopped when
obd_connect_names[i] was NULL.  Now it goes on to 64.
obd_connect_names[] actually has 66 entries, so this isn't
an actual change.  However.... (continued below).

>  		if (flags & mask) {
>  			seq_printf(m, "%s%s",
>  				   first ? sep : "", obd_connect_names[i]);
>  			first = false;
>  		}
>  	}
> -	if (flags & ~(mask - 1))
> +
> +	if (flags & ~(mask - 1)) {
>  		seq_printf(m, "%sunknown flags %#llx",
>  			   first ? sep : "", flags & ~(mask - 1));
> +		first = false;
> +	}

Now that we know i goes up to 64, this code is pointless.
At this point, mask is 0, mask-1 is -1, ~(mask - 1) is zero.
So it cannot print anything.

Thanks,
NeilBrown
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180703/91b09b52/attachment.sig>

From neilb at suse.com  Tue Jul  3 04:26:08 2018
From: neilb at suse.com (NeilBrown)
Date: Tue, 03 Jul 2018 14:26:08 +1000
Subject: [lustre-devel] [PATCH 08/18] lustre: security: send file
 security context for creates
In-Reply-To: <1530573875-20465-9-git-send-email-jsimmons@infradead.org>
References: <1530573875-20465-1-git-send-email-jsimmons@infradead.org>
 <1530573875-20465-9-git-send-email-jsimmons@infradead.org>
Message-ID: <87r2kl0wtb.fsf@notabene.neil.brown.name>

On Mon, Jul 02 2018, James Simmons wrote:

> From: "John L. Hammond" <jhammond at whamcloud.com>
>
> Send file security context to MDT along with create RPCs. This closes
> the insecure window between creation and setting of the security
> context that existed previously. It also avoids a potential LDLM hang
> which arises from ll_create_it() when we send a MDS_SETXATTR RPC while
> holding the lookup+layout lock returned from open.
>


>  
> -err_exit:
> +out_inode:
> +	if (inode)
> +		iput(inode);

iput() allows NULL to be passes (as all 'release' function should)
so this can be just
	iput(inode);


Thanks,
NeilBrown
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180703/8744f815/attachment.sig>

From neilb at suse.com  Tue Jul  3 04:45:30 2018
From: neilb at suse.com (NeilBrown)
Date: Tue, 03 Jul 2018 14:45:30 +1000
Subject: [lustre-devel] [PATCH 16/18] lustre: libcfs: restore original
 behavior in cfs_str2num_check
In-Reply-To: <1530573875-20465-17-git-send-email-jsimmons@infradead.org>
References: <1530573875-20465-1-git-send-email-jsimmons@infradead.org>
 <1530573875-20465-17-git-send-email-jsimmons@infradead.org>
Message-ID: <87o9fp0vx1.fsf@notabene.neil.brown.name>

On Mon, Jul 02 2018, James Simmons wrote:

> When cfs_str2num_check() moved from simple_strtoul to kstrtoul
> some of the functionality got lost. Restore handling hexidecimal
> number as well as '+' and '-'. Also handle any trailing spaces.
>
> Signed-off-by: James Simmons <uja.ornl at yahoo.com>
> WC-bug-id: https://jira.whamcloud.com/browse/LU-9325
> Reviewed-on: https://review.whamcloud.com/32217
> Reviewed-by: John L. Hammond <jhammond at whamcloud.com>
> Reviewed-by: Andreas Dilger <adilger at whamcloud.com>
> Reviewed-by: Dmitry Eremin <dmitry.eremin at intel.com>
> Reviewed-by: Oleg Drokin <green at whamcloud.com>
> Signed-off-by: James Simmons <jsimmons at infradead.org>
> ---
>  drivers/staging/lustre/lnet/libcfs/libcfs_string.c | 17 ++++++++++++++---
>  1 file changed, 14 insertions(+), 3 deletions(-)
>
> diff --git a/drivers/staging/lustre/lnet/libcfs/libcfs_string.c b/drivers/staging/lustre/lnet/libcfs/libcfs_string.c
> index e1fb126..e390b0b 100644
> --- a/drivers/staging/lustre/lnet/libcfs/libcfs_string.c
> +++ b/drivers/staging/lustre/lnet/libcfs/libcfs_string.c
> @@ -214,8 +214,10 @@ char *cfs_firststr(char *str, size_t size)
>  {
>  	bool all_numbers = true;
>  	char *endp, cache;
> +	int len;
>  	int rc;
>  
> +	endp = strim(str);

This looks bad.  The function now changes the string buffer, where it
didn't before.

>  	/**
>  	 * kstrouint can only handle strings composed
>  	 * of only numbers. We need to scan the string
> @@ -228,16 +230,25 @@ char *cfs_firststr(char *str, size_t size)
>  	 * After we are done the character at the
>  	 * position we placed '\0' must be restored.
>  	 */
> -	for (endp = str; endp < str + nob; endp++) {
> -		if (!isdigit(*endp)) {
> +	len = min((int)strlen(endp), nob);

As endp might be different from 'str, 'nob' is not meaningful in the
context of endp.

I'm guessing that mainline commit

Commit: c948390f10cc ("staging: lustre: llite: fix inconsistencies of root squash feature")

is related to this.

I cannot apply this patch as-is.  It is broken.  Possibly changing
   endp = strim(str);
to
   endp = str;

will fix it.

Thanks,
NeilBrown

> +	for (; endp < str + len; endp++) {
> +		if (!isdigit(*endp) && *endp != '-' &&
> +		    *endp != '+') {
>  			all_numbers = false;
>  			break;
>  		}
>  	}
> +
> +	/* Eat trailing space */
> +	if (!all_numbers && isspace(*endp)) {
> +		all_numbers = true;
> +		endp--;
> +	}
> +
>  	cache = *endp;
>  	*endp = '\0';
>  
> -	rc = kstrtouint(str, 10, num);
> +	rc = kstrtouint(str, 0, num);
>  	*endp = cache;
>  	if (rc || !all_numbers)
>  		return 0;
> -- 
> 1.8.3.1
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180703/26ac325d/attachment.sig>

From neilb at suse.com  Tue Jul  3 04:57:33 2018
From: neilb at suse.com (NeilBrown)
Date: Tue, 03 Jul 2018 14:57:33 +1000
Subject: [lustre-devel] [PATCH 00/18] lustre: missing updates from
 lustre 2.9
In-Reply-To: <1530573875-20465-1-git-send-email-jsimmons@infradead.org>
References: <1530573875-20465-1-git-send-email-jsimmons@infradead.org>
Message-ID: <87lgas29xe.fsf@notabene.neil.brown.name>

On Mon, Jul 02 2018, James Simmons wrote:

> This patch set fills in the known missing pieces from the Lustre
> 2.9 release. With these patches this brings us to the pre-2.10
> development cycle. The grant allocation fixes were missing as
> well as security hole closes. The rest are small changes in
> addition to support for filesets and ladvise feature.
>
> Alexander Boyko (1):
>   lustre: obd: add callback for llog_cat_process_or_fork
>
> Bobi Jam (1):
>   lustre: osc: max_pages_per_rpc should be chunk size aligned
>
> Fan Yong (1):
>   lustre: ptlrpc: properly set "rq_xid" for 4MB IO
>
> Henri Doreau (1):
>   lustre: llite: restore fd_och when putting lease
>
> James Simmons (2):
>   lustre: libcfs: restore original behavior in cfs_str2num_check

Apart from this patch which I couldn't bring myself to apply, I've
applied all these patches - making changes to only one as mentioned
below.
I'll probably add some cleanups later as separately described.

>   lustre: update version to 2.8.99
>
> Jian Yu (1):
>   lustre: mount: fix lmd_parse() to handle new delimiters
>
> Jinshan Xiong (1):
>   lustre: llite: fast read implementation
>
> Johann Lombardi (1):
>   lustre: grant: add support for OBD_CONNECT_GRANT_PARAM
>
> John L. Hammond (3):
>   lustre: obd: rename md_getstatus() to md_get_root()
>   lustre: obd: reserve connection flag OBD_CONNECT2_FILE_SECCTX
>   lustre: security: send file security context for creates
>
> Lai Siyao (1):
>   lustre: fileset: add fileset mount support

When I removed the union name as promised, this went from
 14 files changed, 135 insertions(+), 43 deletions(-)
to
 14 files changed, 120 insertions(+), 28 deletions(-)

Not an enormous change, but worthwhile I think.

Thanks
NeilBrown

>
> Li Xi (3):
>   lustre: ladvise: Add feature of giving file access advices
>   lustre: ladvise: Add willread advice support for ladvise
>   lustre: ladvise: Add dontneed advice support for ladvise
>
> Niu Yawei (1):
>   lustre: ldlm: reduce mem footprint of ldlm_resource
>
> Patrick Farrell (1):
>   lustre: llite: ladvise protocol changes
>
>  .../lustre/include/uapi/linux/lustre/lustre_idl.h  |  52 ++-
>  .../lustre/include/uapi/linux/lustre/lustre_user.h |  51 +++
>  .../lustre/include/uapi/linux/lustre/lustre_ver.h  |   4 +-
>  drivers/staging/lustre/lnet/libcfs/libcfs_string.c |  17 +-
>  drivers/staging/lustre/lustre/include/cl_object.h  |  13 +
>  .../staging/lustre/lustre/include/lustre_disk.h    |   2 +
>  drivers/staging/lustre/lustre/include/lustre_dlm.h |  24 +-
>  .../staging/lustre/lustre/include/lustre_import.h  |   1 +
>  .../lustre/lustre/include/lustre_req_layout.h      |  10 +-
>  .../staging/lustre/lustre/include/lustre_swab.h    |   2 +
>  drivers/staging/lustre/lustre/include/obd.h        |  17 +-
>  drivers/staging/lustre/lustre/include/obd_class.h  |  16 +-
>  .../staging/lustre/lustre/include/obd_support.h    |   5 +-
>  drivers/staging/lustre/lustre/ldlm/ldlm_internal.h |   1 +
>  drivers/staging/lustre/lustre/ldlm/ldlm_lib.c      |   1 +
>  drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c    |  20 +-
>  drivers/staging/lustre/lustre/ldlm/ldlm_resource.c |  32 +-
>  drivers/staging/lustre/lustre/llite/dir.c          |  54 +++-
>  drivers/staging/lustre/lustre/llite/file.c         | 360 ++++++++++++++++++---
>  .../staging/lustre/lustre/llite/llite_internal.h   |  34 +-
>  drivers/staging/lustre/lustre/llite/llite_lib.c    |  37 ++-
>  drivers/staging/lustre/lustre/llite/llite_mmap.c   |  34 +-
>  drivers/staging/lustre/lustre/llite/lproc_llite.c  |  38 +++
>  drivers/staging/lustre/lustre/llite/namei.c        |  40 ++-
>  drivers/staging/lustre/lustre/llite/rw.c           |  68 +++-
>  drivers/staging/lustre/lustre/llite/vvp_internal.h |   1 +
>  drivers/staging/lustre/lustre/llite/vvp_io.c       |   3 +
>  .../staging/lustre/lustre/llite/xattr_security.c   |  38 ++-
>  drivers/staging/lustre/lustre/lmv/lmv_obd.c        |  34 +-
>  drivers/staging/lustre/lustre/lov/lov_io.c         |  28 ++
>  drivers/staging/lustre/lustre/mdc/mdc_internal.h   |   4 +
>  drivers/staging/lustre/lustre/mdc/mdc_lib.c        |  32 ++
>  drivers/staging/lustre/lustre/mdc/mdc_locks.c      |   7 +
>  drivers/staging/lustre/lustre/mdc/mdc_reint.c      |   7 +
>  drivers/staging/lustre/lustre/mdc/mdc_request.c    |  50 ++-
>  drivers/staging/lustre/lustre/obdclass/cl_io.c     |   2 +
>  drivers/staging/lustre/lustre/obdclass/llog_cat.c  |  16 +-
>  .../lustre/lustre/obdclass/lprocfs_status.c        |  72 ++++-
>  drivers/staging/lustre/lustre/obdclass/obd_mount.c | 135 +++++++-
>  drivers/staging/lustre/lustre/osc/lproc_osc.c      |  18 ++
>  drivers/staging/lustre/lustre/osc/osc_cache.c      |  95 ++++--
>  .../staging/lustre/lustre/osc/osc_cl_internal.h    |   1 +
>  drivers/staging/lustre/lustre/osc/osc_dev.c        |   1 +
>  drivers/staging/lustre/lustre/osc/osc_internal.h   |   4 +
>  drivers/staging/lustre/lustre/osc/osc_io.c         |  79 +++++
>  drivers/staging/lustre/lustre/osc/osc_request.c    | 184 +++++++++--
>  drivers/staging/lustre/lustre/ptlrpc/client.c      |   9 +-
>  drivers/staging/lustre/lustre/ptlrpc/import.c      |  10 +
>  drivers/staging/lustre/lustre/ptlrpc/layout.c      |  66 +++-
>  .../staging/lustre/lustre/ptlrpc/lproc_ptlrpc.c    |   3 +-
>  .../staging/lustre/lustre/ptlrpc/pack_generic.c    |  27 +-
>  drivers/staging/lustre/lustre/ptlrpc/wiretest.c    | 122 +++++--
>  52 files changed, 1722 insertions(+), 259 deletions(-)
>
> -- 
> 1.8.3.1
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180703/51b3eeb3/attachment.sig>

From paf at cray.com  Tue Jul  3 12:43:46 2018
From: paf at cray.com (Patrick Farrell)
Date: Tue, 3 Jul 2018 12:43:46 +0000
Subject: [lustre-devel] [PATCH 01/18] lustre: grant: add support for
 OBD_CONNECT_GRANT_PARAM
In-Reply-To: <87601x2f5b.fsf@notabene.neil.brown.name>
References: <1530573875-20465-1-git-send-email-jsimmons@infradead.org>
 <1530573875-20465-2-git-send-email-jsimmons@infradead.org>,
 <87601x2f5b.fsf@notabene.neil.brown.name>
Message-ID: <DM3PR1101MB1213CBD340C2EC3D7BECF557CB420@DM3PR1101MB1213.namprd11.prod.outlook.com>

Neil,

Compute vs consume: Honestly either would work.  The client consumes grant when it writes data, this fix to the calculation allows it to consume more accurately. *shrugs*

- Patrick


________________________________
From: lustre-devel <lustre-devel-bounces at lists.lustre.org> on behalf of NeilBrown <neilb at suse.com>
Sent: Monday, July 2, 2018 10:04:48 PM
To: James Simmons; Andreas Dilger; Oleg Drokin
Cc: Johann Lombardi; Lustre Development List
Subject: Re: [lustre-devel] [PATCH 01/18] lustre: grant: add support for OBD_CONNECT_GRANT_PARAM

On Mon, Jul 02 2018, James Simmons wrote:

> From: Johann Lombardi <jlombardi at whamcloud.com>
>
> Add support for grant overhead calculation on the client side.
> To do so, clients track usage on a per-extent basis. An extent is
> composed of contiguous blocks.
> The OST now returns to the OSC layer several parameters to consume
> grant more accurately:

Just to be sure that I'm understanding correctly, should "consume"
above be "compute" ??

>
> - the backend filesystem block size which is the minimal grant
>   allocation unit;
> - the maximum extent size;
> - the extent insertion cost.
>   Clients now pack in bulk write how much grant space was consumed for
>   the RPC. Dirty data accounting also adopts the same scheme.
>
> Moreover, each backend OSD now reports its own set of parameters:
> - For ldiskfs, we usually have a 4KB block size with a maximum extent
>   size of 32MB (theoretical limit of 128MB) and an extent insertion
>   cost of 6 x 4KB = 24KB
> - For ZFS, we report a block size of 128KB, an extent size of 128
>   blocks (i.e. 16MB with 128KB block size) and a block insertion cost
>   of 112KB.
>
> Besides, there is now no more generic metadata overhead reservation
> done inside each OSD. Instead grant space is inflated for clients
> that do not support the new grant parameters. That said, a tiny
> percentage (typically 0.76%) of the free space is still reserved
> inside each OSD to avoid fragmentation which might hurt performance
> and impact our grant calculation (e.g. extents are broken due to
> fragmentation).
>
> This patch also fixes several other issues:
>
> - Bulk write resent by ptlrpc after reconnection could trigger
>   spurious error messages related to broken dirty accounting.
>   The issue was that oa_dirty is discarded for resent requests
>   (grant flag cleared in ost_brw_write()), so we can legitimately
>   have grant > fed_dirty in ofd_grant_check().
>   This was fixed by resetting fed_dirty on reconnection and skipping
>   the dirty accounting check in ofd_grant_check() in the case of
>   ptlrpc resend.

ofd_grant_check doesn't appear in this patch.  Presumably it was part of
the server code that was stripped from the patch.

>
> - In obd_connect_data_seqprint(), the connection flags cannot fit
>   in a 32-bit integer.

I cannot see what this might refer to either.

>
> - When merging two OSC extents, an extent tax should be released
>   in both the merged extent and in the grant accounting.

I think I might see what this relates to.

> +static ssize_t cur_dirty_grant_bytes_show(struct kobject *kobj,
> +                                       struct attribute *attr,
> +                                       char *buf)
> +{
> +     struct obd_device *dev = container_of(kobj, struct obd_device,
> +                                           obd_kobj);
> +     struct client_obd *cli = &dev->u.cli;
> +     int len;
> +
> +     spin_lock(&cli->cl_loi_list_lock);
> +     len = sprintf(buf, "%lu\n", cli->cl_dirty_grant);
> +     spin_unlock(&cli->cl_loi_list_lock);

Why take the spinlock?? cl_dirty_grant is 'long', so reading it is
atomic.


> +             if (OCD_HAS_FLAG(&cli->cl_import->imp_connect_data,
> +                              GRANT_PARAM)) {
> +                     int nrextents;
> +
> +                     /*
> +                      * take extent tax into account when asking for more
> +                      * grant space
> +                      */
> +                     nrextents = (nrpages + cli->cl_max_extent_pages - 1)  /
> +                                  cli->cl_max_extent_pages;

I generally prefer DIV_ROUND_UP() for this.


I don't think any of the above will stop me from applying the patch,
though I might edit the description, and may add a fix-up patch afterwards.

Thanks.
NeilBrown
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180703/dc5329ff/attachment.html>

From paf at cray.com  Tue Jul  3 12:46:53 2018
From: paf at cray.com (Patrick Farrell)
Date: Tue, 3 Jul 2018 12:46:53 +0000
Subject: [lustre-devel] [PATCH 05/18] lustre: llite: fast read
 implementation
In-Reply-To: <87woud0xip.fsf@notabene.neil.brown.name>
References: <1530573875-20465-1-git-send-email-jsimmons@infradead.org>
 <1530573875-20465-6-git-send-email-jsimmons@infradead.org>,
 <87woud0xip.fsf@notabene.neil.brown.name>
Message-ID: <DM3PR1101MB121395BDC6EC6AA3130F42EFCB420@DM3PR1101MB1213.namprd11.prod.outlook.com>


They do.  It’s probably gone unnoticed because those updates are mostly directly triggered by users or one time only during initialization, so there’s little risk of colliding.

________________________________
From: lustre-devel <lustre-devel-bounces at lists.lustre.org> on behalf of NeilBrown <neilb at suse.com>
Sent: Monday, July 2, 2018 11:10:54 PM
To: James Simmons; Andreas Dilger; Oleg Drokin
Cc: Lustre Development List
Subject: Re: [lustre-devel] [PATCH 05/18] lustre: llite: fast read implementation

On Mon, Jul 02 2018, James Simmons wrote:

> +static ssize_t fast_read_store(struct kobject *kobj,
> +                            struct attribute *attr,
> +                            const char *buffer,
> +                            size_t count)
> +{
> +     struct ll_sb_info *sbi = container_of(kobj, struct ll_sb_info,
> +                                           ll_kobj);
> +     bool val;
> +     int rc;
> +
> +     rc = kstrtobool(buffer, &val);
> +     if (rc)
> +             return rc;
> +
> +     spin_lock(&sbi->ll_lock);
> +     if (val)
> +             sbi->ll_flags |= LL_SBI_FAST_READ;
> +     else
> +             sbi->ll_flags &= ~LL_SBI_FAST_READ;
> +     spin_unlock(&sbi->ll_lock);

This is a little odd... no other code uses ll_lock to protect updates to
ll_flags.  Various other places (e.g. checksum_pages_store()) updates it
unprotected.

Maybe there all need to use ll_lock for protection.


Thanks,
NeilBrown
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180703/4a1bdce9/attachment.html>

From adilger at whamcloud.com  Tue Jul  3 21:46:56 2018
From: adilger at whamcloud.com (Andreas Dilger)
Date: Tue, 3 Jul 2018 21:46:56 +0000
Subject: [lustre-devel] [PATCH 18/18] lustre: update version to 2.8.99
In-Reply-To: <1530573875-20465-19-git-send-email-jsimmons@infradead.org>
References: <1530573875-20465-1-git-send-email-jsimmons@infradead.org>,
 <1530573875-20465-19-git-send-email-jsimmons@infradead.org>
Message-ID: <29BE954A-B97C-4EED-8CE0-70FD0B14AE9E@whamcloud.com>

James, thanks for pushing up these patches.

Why not use 2.9.0? Are there still patches missing, or do you want this version to distinguish it from the actual 2.9.0 release?

Cheers, Andreas

> On Jul 2, 2018, at 17:36, James Simmons <jsimmons at infradead.org> wrote:
> 
> With the missing patches from the lustre 2.9 release merged
> upstream its time to update the upstream client's version.
> 
> Signed-off-by: James Simmons <jsimmons at infradead.org>
> ---
> drivers/staging/lustre/include/uapi/linux/lustre/lustre_ver.h | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/staging/lustre/include/uapi/linux/lustre/lustre_ver.h b/drivers/staging/lustre/include/uapi/linux/lustre/lustre_ver.h
> index 19c9135..1428fdd 100644
> --- a/drivers/staging/lustre/include/uapi/linux/lustre/lustre_ver.h
> +++ b/drivers/staging/lustre/include/uapi/linux/lustre/lustre_ver.h
> @@ -2,10 +2,10 @@
> #define _LUSTRE_VER_H_
> 
> #define LUSTRE_MAJOR 2
> -#define LUSTRE_MINOR 6
> +#define LUSTRE_MINOR 8
> #define LUSTRE_PATCH 99
> #define LUSTRE_FIX 0
> -#define LUSTRE_VERSION_STRING "2.6.99"
> +#define LUSTRE_VERSION_STRING "2.8.99"
> 
> #define OBD_OCD_VERSION(major, minor, patch, fix)            \
>    (((major) << 24) + ((minor) << 16) + ((patch) << 8) + (fix))
> -- 
> 1.8.3.1
> 

From neilb at suse.com  Tue Jul  3 21:59:34 2018
From: neilb at suse.com (NeilBrown)
Date: Wed, 04 Jul 2018 07:59:34 +1000
Subject: [lustre-devel] [PATCH 01/18] lustre: grant: add support for
 OBD_CONNECT_GRANT_PARAM
In-Reply-To: <DM3PR1101MB1213CBD340C2EC3D7BECF557CB420@DM3PR1101MB1213.namprd11.prod.outlook.com>
References: <1530573875-20465-1-git-send-email-jsimmons@infradead.org>
 <1530573875-20465-2-git-send-email-jsimmons@infradead.org>
 <87601x2f5b.fsf@notabene.neil.brown.name>
 <DM3PR1101MB1213CBD340C2EC3D7BECF557CB420@DM3PR1101MB1213.namprd11.prod.outlook.com>
Message-ID: <87in5w0ym1.fsf@notabene.neil.brown.name>

On Tue, Jul 03 2018, Patrick Farrell wrote:

> Neil,
>
> Compute vs consume: Honestly either would work.  The client consumes grant when it writes data, this fix to the calculation allows it to consume more accurately. *shrugs*

Good - thanks.
I would have said "this fix to the calculation allows it to track what it
has consumed more accurately", but it is a small point.  I just wanted to
be sure I understood, and now I'm sure I do.

Thanks,
NeilBrown
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180704/a4870fe1/attachment.sig>

From neilb at suse.com  Tue Jul  3 22:03:27 2018
From: neilb at suse.com (NeilBrown)
Date: Wed, 04 Jul 2018 08:03:27 +1000
Subject: [lustre-devel] [PATCH 05/18] lustre: llite: fast read
 implementation
In-Reply-To: <DM3PR1101MB121395BDC6EC6AA3130F42EFCB420@DM3PR1101MB1213.namprd11.prod.outlook.com>
References: <1530573875-20465-1-git-send-email-jsimmons@infradead.org>
 <1530573875-20465-6-git-send-email-jsimmons@infradead.org>
 <87woud0xip.fsf@notabene.neil.brown.name>
 <DM3PR1101MB121395BDC6EC6AA3130F42EFCB420@DM3PR1101MB1213.namprd11.prod.outlook.com>
Message-ID: <87fu100yfk.fsf@notabene.neil.brown.name>

On Tue, Jul 03 2018, Patrick Farrell wrote:

> They do.  It’s probably gone unnoticed because those updates are mostly directly triggered by users or one time only during initialization, so there’s little risk of colliding.

Agreed.  Had there been no locking in this new patch I possibly wouldn't
have noticed.  But I don't like inconsistencies.

Thanks,
NeilBrown
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180704/76065dfc/attachment.sig>

From neilb at suse.com  Wed Jul  4 02:17:06 2018
From: neilb at suse.com (NeilBrown)
Date: Wed, 04 Jul 2018 12:17:06 +1000
Subject: [lustre-devel] [PATCH 00/10] lustre: libcfs: fix libcfs debugfs
 bugs
In-Reply-To: <1530573894-20592-1-git-send-email-jsimmons@infradead.org>
References: <1530573894-20592-1-git-send-email-jsimmons@infradead.org>
Message-ID: <87y3erzqvx.fsf@notabene.neil.brown.name>

On Mon, Jul 02 2018, James Simmons wrote:

> The original port to sysfs / debugfs was done in haste without any
> testing so many bugs were introduced. This patch set resolves the
> bugs found in the libcfs code when it was reviewed.
>
> James Simmons (10):
>   lustre: libcfs: rename lustre_*_debugfs to lnet_*_debugfs
>   lustre: libcfs: rename *_debugmb to *_debug_mb
>   lustre: libcfs: make lnet_debugfs_symlink_def fields const
>   lustre: libcfs: fix module loading and sysfs setting race
>   lustre: libcfs: fix wrong debugfs symlink name
>   lustre: libcfs: don't assume sysfs mount point
>   lustre: libcfs: properly initialize lnet_debugfs_symlink_def
>   lustre: libcfs: test if table is NULL for lnet_insert_debugfs
>   lustre: libcfs: return ssize_t for lnet_debugfs_*
>   lustre: libcfs: small cleanups for debugfs code
>
>  .../staging/lustre/include/linux/libcfs/libcfs.h   |  2 +-
>  drivers/staging/lustre/lnet/libcfs/debug.c         | 28 ++++----
>  drivers/staging/lustre/lnet/libcfs/module.c        | 78 +++++++++++-----------
>  drivers/staging/lustre/lnet/lnet/router_proc.c     |  2 +-
>  4 files changed, 57 insertions(+), 53 deletions(-)
>
> -- 
> 1.8.3.1

These all look good to me, so I've (provisionally) applied them.
I didn't know about kernel_param_lock() before. I wonder if it
would make sense to use that to protect updates to sbi->ll_flags???

Thanks,
NeilBrown
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180704/9f21b8eb/attachment.sig>

From neilb at suse.com  Wed Jul  4 04:43:50 2018
From: neilb at suse.com (NeilBrown)
Date: Wed, 04 Jul 2018 14:43:50 +1000
Subject: [lustre-devel] [PATCH 00/11] lustre cleanups following "lustre:
 missing updates from lustre 2.9"
Message-ID: <153067941050.30111.13691201276059352926.stgit@noble>

This series covers all the issues that I found while reviewing
James' recent patch series names in $SUBJECT, except for
fixing the patch that I thought was broken.

In several cases the same issue that I saw in those patches was
already present elsewhere in the lustre code, and I cleaned
up all (or many) relevant occurrences.

Thanks,
NeilBrown

---

NeilBrown (11):
      lustre: use DIV_ROUND_UP() where appropriate.
      lustre: osc: don't take spinlock to read a "long"
      lustre: llite: replace several GFP_NOFS with GFP_KERNEL
      lustre: fix alignment in lustre_idl.h
      lustre: improve alignment in lustre_disk.h
      lustre: mdc: change memcpy to strcpy.
      lustre: simplify some parsing in lmd_parse().
      lustre: protect updates to ll_flags.
      lustre: remove dead code from obd_connect_flags2str()
      lustre: iput() accepts NULL.
      lustre: avoid alloc(sizeof(struct ...))


 .../lustre/include/uapi/linux/lustre/lustre_idl.h  | 1079 ++++++++++----------
 .../staging/lustre/lnet/klnds/o2iblnd/o2iblnd.h    |    3 
 drivers/staging/lustre/lnet/lnet/nidstrings.c      |    4 
 drivers/staging/lustre/lnet/selftest/console.c     |    8 
 drivers/staging/lustre/lnet/selftest/framework.c   |    8 
 drivers/staging/lustre/lnet/selftest/selftest.h    |    2 
 .../staging/lustre/lustre/include/lustre_disk.h    |   38 -
 drivers/staging/lustre/lustre/llite/dir.c          |    3 
 drivers/staging/lustre/lustre/llite/file.c         |   32 -
 drivers/staging/lustre/lustre/llite/llite_lib.c    |    4 
 drivers/staging/lustre/lustre/llite/lproc_llite.c  |    6 
 drivers/staging/lustre/lustre/llite/statahead.c    |    3 
 drivers/staging/lustre/lustre/llite/xattr.c        |    4 
 drivers/staging/lustre/lustre/lmv/lmv_obd.c        |    2 
 drivers/staging/lustre/lustre/mdc/mdc_request.c    |    2 
 .../lustre/lustre/obdclass/lprocfs_status.c        |    7 
 drivers/staging/lustre/lustre/obdclass/obd_mount.c |   24 
 drivers/staging/lustre/lustre/osc/lproc_osc.c      |   36 -
 drivers/staging/lustre/lustre/osc/osc_request.c    |    3 
 drivers/staging/lustre/lustre/ptlrpc/client.c      |    2 
 20 files changed, 622 insertions(+), 648 deletions(-)

--
Signature


From neilb at suse.com  Wed Jul  4 04:43:50 2018
From: neilb at suse.com (NeilBrown)
Date: Wed, 04 Jul 2018 14:43:50 +1000
Subject: [lustre-devel] [PATCH 01/11] lustre: use DIV_ROUND_UP() where
 appropriate.
In-Reply-To: <153067941050.30111.13691201276059352926.stgit@noble>
References: <153067941050.30111.13691201276059352926.stgit@noble>
Message-ID: <153067943057.30111.7389117779247056126.stgit@noble>

There places have code of the form
   ( x + y - 1 ) / y

replace all of these with DIV_ROUND_UP(x, y) to improve readability.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 .../staging/lustre/lnet/klnds/o2iblnd/o2iblnd.h    |    3 +--
 drivers/staging/lustre/lnet/selftest/selftest.h    |    2 +-
 drivers/staging/lustre/lustre/osc/osc_request.c    |    3 +--
 3 files changed, 3 insertions(+), 5 deletions(-)

diff --git a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.h b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.h
index 7d8429672616..41990020832a 100644
--- a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.h
+++ b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.h
@@ -127,8 +127,7 @@ extern struct kib_tunables  kiblnd_tunables;
 #define IBLND_RX_MSGS(c)	\
 	((c->ibc_queue_depth) * 2 + IBLND_OOB_MSGS(c->ibc_version))
 #define IBLND_RX_MSG_BYTES(c)	(IBLND_RX_MSGS(c) * IBLND_MSG_SIZE)
-#define IBLND_RX_MSG_PAGES(c)	\
-	((IBLND_RX_MSG_BYTES(c) + PAGE_SIZE - 1) / PAGE_SIZE)
+#define IBLND_RX_MSG_PAGES(c)	DIV_ROUND_UP(IBLND_RX_MSG_BYTES(c), PAGE_SIZE)
 
 /* WRs and CQEs (per connection) */
 #define IBLND_RECV_WRS(c)	IBLND_RX_MSGS(c)
diff --git a/drivers/staging/lustre/lnet/selftest/selftest.h b/drivers/staging/lustre/lnet/selftest/selftest.h
index 8737fa96b192..6690e8ec5155 100644
--- a/drivers/staging/lustre/lnet/selftest/selftest.h
+++ b/drivers/staging/lustre/lnet/selftest/selftest.h
@@ -397,7 +397,7 @@ struct sfw_test_instance {
 #define SFW_MAX_CONCUR	   LST_MAX_CONCUR
 #define SFW_ID_PER_PAGE    (PAGE_SIZE / sizeof(struct lnet_process_id_packed))
 #define SFW_MAX_NDESTS	   (LNET_MAX_IOV * SFW_ID_PER_PAGE)
-#define sfw_id_pages(n)    (((n) + SFW_ID_PER_PAGE - 1) / SFW_ID_PER_PAGE)
+#define sfw_id_pages(n)    DIV_ROUND_UP(n, SFW_ID_PER_PAGE)
 
 struct sfw_test_unit {
 	struct list_head    tsu_list;	   /* chain on lst_test_instance */
diff --git a/drivers/staging/lustre/lustre/osc/osc_request.c b/drivers/staging/lustre/lustre/osc/osc_request.c
index 2d05387d1b0d..21497eac5c81 100644
--- a/drivers/staging/lustre/lustre/osc/osc_request.c
+++ b/drivers/staging/lustre/lustre/osc/osc_request.c
@@ -712,8 +712,7 @@ static void osc_announce_cached(struct client_obd *cli, struct obdo *oa,
 			 * take extent tax into account when asking for more
 			 * grant space
 			 */
-			nrextents = (nrpages + cli->cl_max_extent_pages - 1)  /
-				     cli->cl_max_extent_pages;
+			nrextents = DIV_ROUND_UP(nrpages, cli->cl_max_extent_pages);
 			oa->o_undirty += nrextents * cli->cl_grant_extent_tax;
 		}
 	}



From neilb at suse.com  Wed Jul  4 04:43:50 2018
From: neilb at suse.com (NeilBrown)
Date: Wed, 04 Jul 2018 14:43:50 +1000
Subject: [lustre-devel] [PATCH 02/11] lustre: osc: don't take spinlock to
 read a "long"
In-Reply-To: <153067941050.30111.13691201276059352926.stgit@noble>
References: <153067941050.30111.13691201276059352926.stgit@noble>
Message-ID: <153067943061.30111.10303681433910918365.stgit@noble>

Reading a long is always atomic, so there is no
need to take a spinlock - the value read will always
be internally consistent and in none of these places
is consistency with another value relevant.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 drivers/staging/lustre/lustre/osc/lproc_osc.c |   36 +++----------------------
 1 file changed, 5 insertions(+), 31 deletions(-)

diff --git a/drivers/staging/lustre/lustre/osc/lproc_osc.c b/drivers/staging/lustre/lustre/osc/lproc_osc.c
index 81adf548f426..54a303e4e046 100644
--- a/drivers/staging/lustre/lustre/osc/lproc_osc.c
+++ b/drivers/staging/lustre/lustre/osc/lproc_osc.c
@@ -138,10 +138,7 @@ static ssize_t max_dirty_mb_show(struct kobject *kobj,
 	long val;
 	int mult;
 
-	spin_lock(&cli->cl_loi_list_lock);
 	val = cli->cl_dirty_max_pages;
-	spin_unlock(&cli->cl_loi_list_lock);
-
 	mult = 1 << (20 - PAGE_SHIFT);
 	return lprocfs_read_frac_helper(buf, PAGE_SIZE, val, mult);
 }
@@ -252,13 +249,9 @@ static ssize_t cur_dirty_bytes_show(struct kobject *kobj,
 	struct obd_device *dev = container_of(kobj, struct obd_device,
 					      obd_kobj);
 	struct client_obd *cli = &dev->u.cli;
-	int len;
 
-	spin_lock(&cli->cl_loi_list_lock);
-	len = sprintf(buf, "%lu\n", cli->cl_dirty_pages << PAGE_SHIFT);
-	spin_unlock(&cli->cl_loi_list_lock);
+	return sprintf(buf, "%lu\n", cli->cl_dirty_pages << PAGE_SHIFT);
 
-	return len;
 }
 LUSTRE_RO_ATTR(cur_dirty_bytes);
 
@@ -269,13 +262,8 @@ static ssize_t cur_grant_bytes_show(struct kobject *kobj,
 	struct obd_device *dev = container_of(kobj, struct obd_device,
 					      obd_kobj);
 	struct client_obd *cli = &dev->u.cli;
-	int len;
 
-	spin_lock(&cli->cl_loi_list_lock);
-	len = sprintf(buf, "%lu\n", cli->cl_avail_grant);
-	spin_unlock(&cli->cl_loi_list_lock);
-
-	return len;
+	return sprintf(buf, "%lu\n", cli->cl_avail_grant);
 }
 
 static ssize_t cur_grant_bytes_store(struct kobject *kobj,
@@ -294,12 +282,8 @@ static ssize_t cur_grant_bytes_store(struct kobject *kobj,
 		return rc;
 
 	/* this is only for shrinking grant */
-	spin_lock(&cli->cl_loi_list_lock);
-	if (val >= cli->cl_avail_grant) {
-		spin_unlock(&cli->cl_loi_list_lock);
+	if (val >= cli->cl_avail_grant)
 		return -EINVAL;
-	}
-	spin_unlock(&cli->cl_loi_list_lock);
 
 	if (cli->cl_import->imp_state == LUSTRE_IMP_FULL)
 		rc = osc_shrink_grant_to_target(cli, val);
@@ -316,13 +300,8 @@ static ssize_t cur_lost_grant_bytes_show(struct kobject *kobj,
 	struct obd_device *dev = container_of(kobj, struct obd_device,
 					      obd_kobj);
 	struct client_obd *cli = &dev->u.cli;
-	int len;
-
-	spin_lock(&cli->cl_loi_list_lock);
-	len = sprintf(buf, "%lu\n", cli->cl_lost_grant);
-	spin_unlock(&cli->cl_loi_list_lock);
 
-	return len;
+	return sprintf(buf, "%lu\n", cli->cl_lost_grant);
 }
 LUSTRE_RO_ATTR(cur_lost_grant_bytes);
 
@@ -333,13 +312,8 @@ static ssize_t cur_dirty_grant_bytes_show(struct kobject *kobj,
 	struct obd_device *dev = container_of(kobj, struct obd_device,
 					      obd_kobj);
 	struct client_obd *cli = &dev->u.cli;
-	int len;
 
-	spin_lock(&cli->cl_loi_list_lock);
-	len = sprintf(buf, "%lu\n", cli->cl_dirty_grant);
-	spin_unlock(&cli->cl_loi_list_lock);
-
-	return len;
+	return sprintf(buf, "%lu\n", cli->cl_dirty_grant);
 }
 LUSTRE_RO_ATTR(cur_dirty_grant_bytes);
 



From neilb at suse.com  Wed Jul  4 04:43:50 2018
From: neilb at suse.com (NeilBrown)
Date: Wed, 04 Jul 2018 14:43:50 +1000
Subject: [lustre-devel] [PATCH 03/11] lustre: llite: replace several
 GFP_NOFS with GFP_KERNEL
In-Reply-To: <153067941050.30111.13691201276059352926.stgit@noble>
References: <153067941050.30111.13691201276059352926.stgit@noble>
Message-ID: <153067943065.30111.11608217600471180041.stgit@noble>

GFP_KERNEL should be used preferentially unless there is
good reason to use something else.

GFP_NOFS is appropriate when the code holds some resource
that could blocks writeout triggered by memory reclaim.
So it is appropriate to us it in the writeout path, or anything
that could interact with the writeout paths.

The memory allocations affected here are either ioctl, or open/close
handling.  These can never block writeout, so GFP_NOFS is not needed.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 drivers/staging/lustre/lustre/llite/file.c |   32 ++++++++++++++--------------
 1 file changed, 16 insertions(+), 16 deletions(-)

diff --git a/drivers/staging/lustre/lustre/llite/file.c b/drivers/staging/lustre/lustre/llite/file.c
index 614588c365df..54da6f19da21 100644
--- a/drivers/staging/lustre/lustre/llite/file.c
+++ b/drivers/staging/lustre/lustre/llite/file.c
@@ -64,7 +64,7 @@ static struct ll_file_data *ll_file_data_get(void)
 {
 	struct ll_file_data *fd;
 
-	fd = kmem_cache_zalloc(ll_file_data_slab, GFP_NOFS);
+	fd = kmem_cache_zalloc(ll_file_data_slab, GFP_KERNEL);
 	if (!fd)
 		return NULL;
 	fd->fd_write_failed = false;
@@ -137,7 +137,7 @@ static int ll_close_inode_openhandle(struct inode *inode,
 		goto out;
 	}
 
-	op_data = kzalloc(sizeof(*op_data), GFP_NOFS);
+	op_data = kzalloc(sizeof(*op_data), GFP_KERNEL);
 	/*
 	 * We leak openhandle and request here on error, but not much to be
 	 * done in OOM case since app won't retry close on error either.
@@ -615,7 +615,7 @@ int ll_file_open(struct inode *inode, struct file *file)
 
 			goto restart;
 		}
-		*och_p = kzalloc(sizeof(struct obd_client_handle), GFP_NOFS);
+		*och_p = kzalloc(sizeof(struct obd_client_handle), GFP_KERNEL);
 		if (!*och_p) {
 			rc = -ENOMEM;
 			goto out_och_free;
@@ -823,7 +823,7 @@ ll_lease_open(struct inode *inode, struct file *file, fmode_t fmode,
 			return ERR_PTR(rc);
 	}
 
-	och = kzalloc(sizeof(*och), GFP_NOFS);
+	och = kzalloc(sizeof(*och), GFP_KERNEL);
 	if (!och)
 		return ERR_PTR(-ENOMEM);
 
@@ -1470,7 +1470,7 @@ static int ll_lov_setea(struct inode *inode, struct file *file,
 	if (!capable(CAP_SYS_ADMIN))
 		return -EPERM;
 
-	lump = kzalloc(lum_size, GFP_NOFS);
+	lump = kzalloc(lum_size, GFP_KERNEL);
 	if (!lump)
 		return -ENOMEM;
 
@@ -1637,7 +1637,7 @@ int ll_release_openhandle(struct inode *inode, struct lookup_intent *it)
 
 	LASSERT(it_open_error(DISP_OPEN_OPEN, it) == 0);
 
-	och = kzalloc(sizeof(*och), GFP_NOFS);
+	och = kzalloc(sizeof(*och), GFP_KERNEL);
 	if (!och) {
 		rc = -ENOMEM;
 		goto out;
@@ -1735,7 +1735,7 @@ int ll_fid2path(struct inode *inode, void __user *arg)
 
 	outsize = sizeof(*gfout) + pathlen;
 
-	gfout = kzalloc(outsize, GFP_NOFS);
+	gfout = kzalloc(outsize, GFP_KERNEL);
 	if (!gfout)
 		return -ENOMEM;
 
@@ -1885,7 +1885,7 @@ static int ll_swap_layouts(struct file *file1, struct file *file2,
 	struct ll_swap_stack	*llss = NULL;
 	int			 rc;
 
-	llss = kzalloc(sizeof(*llss), GFP_NOFS);
+	llss = kzalloc(sizeof(*llss), GFP_KERNEL);
 	if (!llss)
 		return -ENOMEM;
 
@@ -2031,7 +2031,7 @@ static int ll_hsm_import(struct inode *inode, struct file *file,
 		return -EINVAL;
 
 	/* set HSM flags */
-	hss = kzalloc(sizeof(*hss), GFP_NOFS);
+	hss = kzalloc(sizeof(*hss), GFP_KERNEL);
 	if (!hss)
 		return -ENOMEM;
 
@@ -2042,7 +2042,7 @@ static int ll_hsm_import(struct inode *inode, struct file *file,
 	if (rc != 0)
 		goto free_hss;
 
-	attr = kzalloc(sizeof(*attr), GFP_NOFS);
+	attr = kzalloc(sizeof(*attr), GFP_KERNEL);
 	if (!attr) {
 		rc = -ENOMEM;
 		goto free_hss;
@@ -2298,7 +2298,7 @@ ll_file_ioctl(struct file *file, unsigned int cmd, unsigned long arg)
 		struct hsm_user_state	*hus;
 		int			 rc;
 
-		hus = kzalloc(sizeof(*hus), GFP_NOFS);
+		hus = kzalloc(sizeof(*hus), GFP_KERNEL);
 		if (!hus)
 			return -ENOMEM;
 
@@ -2337,7 +2337,7 @@ ll_file_ioctl(struct file *file, unsigned int cmd, unsigned long arg)
 		struct hsm_current_action	*hca;
 		int				 rc;
 
-		hca = kzalloc(sizeof(*hca), GFP_NOFS);
+		hca = kzalloc(sizeof(*hca), GFP_KERNEL);
 		if (!hca)
 			return -ENOMEM;
 
@@ -2464,7 +2464,7 @@ ll_file_ioctl(struct file *file, unsigned int cmd, unsigned long arg)
 		int i;
 
 		rc = 0;
-		ladvise_hdr = kzalloc(alloc_size, GFP_NOFS);
+		ladvise_hdr = kzalloc(alloc_size, GFP_KERNEL);
 		if (!ladvise_hdr)
 			return -ENOMEM;
 
@@ -2490,7 +2490,7 @@ ll_file_ioctl(struct file *file, unsigned int cmd, unsigned long arg)
 		kfree(ladvise_hdr);
 		alloc_size = offsetof(typeof(*ladvise_hdr),
 				      lah_advise[num_advise]);
-		ladvise_hdr = kzalloc(alloc_size, GFP_NOFS);
+		ladvise_hdr = kzalloc(alloc_size, GFP_KERNEL);
 		if (!ladvise_hdr)
 			return -ENOMEM;
 
@@ -3458,7 +3458,7 @@ void *ll_iocontrol_register(llioc_callback_t cb, int count, unsigned int *cmd)
 		return NULL;
 
 	size = sizeof(*in_data) + count * sizeof(unsigned int);
-	in_data = kzalloc(size, GFP_NOFS);
+	in_data = kzalloc(size, GFP_KERNEL);
 	if (!in_data)
 		return NULL;
 
@@ -3625,7 +3625,7 @@ static int ll_layout_fetch(struct inode *inode, struct ldlm_lock *lock)
 		goto out;
 	}
 
-	lvbdata = kvzalloc(lmmsize, GFP_NOFS);
+	lvbdata = kvzalloc(lmmsize, GFP_KERNEL);
 	if (!lvbdata) {
 		rc = -ENOMEM;
 		goto out;



From neilb at suse.com  Wed Jul  4 04:43:50 2018
From: neilb at suse.com (NeilBrown)
Date: Wed, 04 Jul 2018 14:43:50 +1000
Subject: [lustre-devel] [PATCH 04/11] lustre: fix alignment in lustre_idl.h
In-Reply-To: <153067941050.30111.13691201276059352926.stgit@noble>
References: <153067941050.30111.13691201276059352926.stgit@noble>
Message-ID: <153067943069.30111.15939411313283828400.stgit@noble>

Lots of fields are aligned inconsistently
with various space and/or tabs.

This patch tries to make it all more consistent.
No code is changed (though some parentheses are added
for improved alignment).

Signed-off-by: NeilBrown <neilb at suse.com>
---
 .../lustre/include/uapi/linux/lustre/lustre_idl.h  | 1079 ++++++++++----------
 1 file changed, 540 insertions(+), 539 deletions(-)

diff --git a/drivers/staging/lustre/include/uapi/linux/lustre/lustre_idl.h b/drivers/staging/lustre/include/uapi/linux/lustre/lustre_idl.h
index 458a8090d9b9..c9b32ef21b16 100644
--- a/drivers/staging/lustre/include/uapi/linux/lustre/lustre_idl.h
+++ b/drivers/staging/lustre/include/uapi/linux/lustre/lustre_idl.h
@@ -88,39 +88,39 @@
  */
 #define LUSTRE_MDT_MAXNAMELEN	80
 
-#define CONNMGR_REQUEST_PORTAL	  1
-#define CONNMGR_REPLY_PORTAL	    2
-/*#define OSC_REQUEST_PORTAL	    3 */
-#define OSC_REPLY_PORTAL		4
-/*#define OSC_BULK_PORTAL	       5 */
-#define OST_IO_PORTAL		   6
-#define OST_CREATE_PORTAL	       7
-#define OST_BULK_PORTAL		 8
-/*#define MDC_REQUEST_PORTAL	    9 */
-#define MDC_REPLY_PORTAL	       10
-/*#define MDC_BULK_PORTAL	      11 */
-#define MDS_REQUEST_PORTAL	     12
-/*#define MDS_REPLY_PORTAL	     13 */
+#define CONNMGR_REQUEST_PORTAL	1
+#define CONNMGR_REPLY_PORTAL	2
+/*#define OSC_REQUEST_PORTAL	3 */
+#define OSC_REPLY_PORTAL	4
+/*#define OSC_BULK_PORTAL	5 */
+#define OST_IO_PORTAL		6
+#define OST_CREATE_PORTAL	7
+#define OST_BULK_PORTAL		8
+/*#define MDC_REQUEST_PORTAL	9 */
+#define MDC_REPLY_PORTAL	10
+/*#define MDC_BULK_PORTAL	11 */
+#define MDS_REQUEST_PORTAL	12
+/*#define MDS_REPLY_PORTAL	13 */
 #define MDS_BULK_PORTAL		14
-#define LDLM_CB_REQUEST_PORTAL	 15
-#define LDLM_CB_REPLY_PORTAL	   16
-#define LDLM_CANCEL_REQUEST_PORTAL     17
-#define LDLM_CANCEL_REPLY_PORTAL       18
-/*#define PTLBD_REQUEST_PORTAL	   19 */
-/*#define PTLBD_REPLY_PORTAL	     20 */
-/*#define PTLBD_BULK_PORTAL	      21 */
-#define MDS_SETATTR_PORTAL	     22
-#define MDS_READPAGE_PORTAL	    23
-#define OUT_PORTAL		    24
-
-#define MGC_REPLY_PORTAL	       25
-#define MGS_REQUEST_PORTAL	     26
-#define MGS_REPLY_PORTAL	       27
-#define OST_REQUEST_PORTAL	     28
-#define FLD_REQUEST_PORTAL	     29
-#define SEQ_METADATA_PORTAL	    30
+#define LDLM_CB_REQUEST_PORTAL	15
+#define LDLM_CB_REPLY_PORTAL	16
+#define LDLM_CANCEL_REQUEST_PORTAL 17
+#define LDLM_CANCEL_REPLY_PORTAL   18
+/*#define PTLBD_REQUEST_PORTAL	19 */
+/*#define PTLBD_REPLY_PORTAL	20 */
+/*#define PTLBD_BULK_PORTAL	21 */
+#define MDS_SETATTR_PORTAL	22
+#define MDS_READPAGE_PORTAL	23
+#define OUT_PORTAL		24
+
+#define MGC_REPLY_PORTAL	25
+#define MGS_REQUEST_PORTAL	26
+#define MGS_REPLY_PORTAL	27
+#define OST_REQUEST_PORTAL	28
+#define FLD_REQUEST_PORTAL	29
+#define SEQ_METADATA_PORTAL	30
 #define SEQ_DATA_PORTAL		31
-#define SEQ_CONTROLLER_PORTAL	  32
+#define SEQ_CONTROLLER_PORTAL	32
 #define MGS_BULK_PORTAL		33
 
 /* Portal 63 is reserved for the Cray Inc DVS - nic at cray.com, roe at cray.com,
@@ -128,13 +128,13 @@
  */
 
 /* packet types */
-#define PTL_RPC_MSG_REQUEST 4711
-#define PTL_RPC_MSG_ERR     4712
-#define PTL_RPC_MSG_REPLY   4713
+#define PTL_RPC_MSG_REQUEST	4711
+#define PTL_RPC_MSG_ERR		4712
+#define PTL_RPC_MSG_REPLY	4713
 
 /* DON'T use swabbed values of MAGIC as magic! */
-#define LUSTRE_MSG_MAGIC_V2 0x0BD00BD3
-#define LUSTRE_MSG_MAGIC_V2_SWABBED 0xD30BD00B
+#define LUSTRE_MSG_MAGIC_V2		0x0BD00BD3
+#define LUSTRE_MSG_MAGIC_V2_SWABBED	0xD30BD00B
 
 #define LUSTRE_MSG_MAGIC LUSTRE_MSG_MAGIC_V2
 
@@ -264,23 +264,23 @@ enum fid_seq {
 	FID_SEQ_LOV_DEFAULT	= 0xffffffffffffffffULL
 };
 
-#define OBIF_OID_MAX_BITS	   32
-#define OBIF_MAX_OID		(1ULL << OBIF_OID_MAX_BITS)
-#define OBIF_OID_MASK	       ((1ULL << OBIF_OID_MAX_BITS) - 1)
-#define IDIF_OID_MAX_BITS	   48
-#define IDIF_MAX_OID		(1ULL << IDIF_OID_MAX_BITS)
-#define IDIF_OID_MASK	       ((1ULL << IDIF_OID_MAX_BITS) - 1)
+#define OBIF_OID_MAX_BITS	32
+#define OBIF_MAX_OID		((1ULL << OBIF_OID_MAX_BITS))
+#define OBIF_OID_MASK		((1ULL << OBIF_OID_MAX_BITS) - 1)
+#define IDIF_OID_MAX_BITS	48
+#define IDIF_MAX_OID		((1ULL << IDIF_OID_MAX_BITS))
+#define IDIF_OID_MASK		((1ULL << IDIF_OID_MAX_BITS) - 1)
 
 /** OID for FID_SEQ_SPECIAL */
 enum special_oid {
 	/* Big Filesystem Lock to serialize rename operations */
-	FID_OID_SPECIAL_BFL     = 1UL,
+	FID_OID_SPECIAL_BFL	= 1UL,
 };
 
 /** OID for FID_SEQ_DOT_LUSTRE */
 enum dot_lustre_oid {
-	FID_OID_DOT_LUSTRE  = 1UL,
-	FID_OID_DOT_LUSTRE_OBF = 2UL,
+	FID_OID_DOT_LUSTRE	= 1UL,
+	FID_OID_DOT_LUSTRE_OBF	= 2UL,
 };
 
 /** OID for FID_SEQ_ROOT */
@@ -312,22 +312,22 @@ enum lu_dirent_attrs {
  */
 struct lu_dirent {
 	/** valid if LUDA_FID is set. */
-	struct lu_fid lde_fid;
+	struct lu_fid	lde_fid;
 	/** a unique entry identifier: a hash or an offset. */
-	__u64	 lde_hash;
+	__u64		lde_hash;
 	/** total record length, including all attributes. */
-	__u16	 lde_reclen;
+	__u16		lde_reclen;
 	/** name length */
-	__u16	 lde_namelen;
+	__u16		lde_namelen;
 	/** optional variable size attributes following this entry.
 	 *  taken from enum lu_dirent_attrs.
 	 */
-	__u32	 lde_attrs;
+	__u32		lde_attrs;
 	/** name is followed by the attributes indicated in ->ldp_attrs, in
 	 *  their natural order. After the last attribute, padding bytes are
 	 *  added to make ->lde_reclen a multiple of 8.
 	 */
-	char	  lde_name[0];
+	char		lde_name[0];
 };
 
 /*
@@ -370,10 +370,10 @@ struct luda_type {
 #endif
 
 struct lu_dirpage {
-	__le64	    ldp_hash_start;
-	__le64	    ldp_hash_end;
-	__le32	    ldp_flags;
-	__le32	    ldp_pad0;
+	__le64		ldp_hash_start;
+	__le64		ldp_hash_end;
+	__le32		ldp_flags;
+	__le32		ldp_pad0;
 	struct lu_dirent ldp_entries[0];
 };
 
@@ -551,59 +551,59 @@ struct ptlrpc_body_v2 {
 
 /* message body offset for lustre_msg_v2 */
 /* ptlrpc body offset in all request/reply messages */
-#define MSG_PTLRPC_BODY_OFF	     0
+#define MSG_PTLRPC_BODY_OFF		0
 
 /* normal request/reply message record offset */
-#define REQ_REC_OFF		     1
-#define REPLY_REC_OFF		   1
+#define REQ_REC_OFF			1
+#define REPLY_REC_OFF			1
 
 /* ldlm request message body offset */
-#define DLM_LOCKREQ_OFF		 1 /* lockreq offset */
-#define DLM_REQ_REC_OFF		 2 /* normal dlm request record offset */
+#define DLM_LOCKREQ_OFF			1 /* lockreq offset */
+#define DLM_REQ_REC_OFF			2 /* normal dlm request record offset */
 
 /* ldlm intent lock message body offset */
-#define DLM_INTENT_IT_OFF	       2 /* intent lock it offset */
-#define DLM_INTENT_REC_OFF	      3 /* intent lock record offset */
+#define DLM_INTENT_IT_OFF		2 /* intent lock it offset */
+#define DLM_INTENT_REC_OFF		3 /* intent lock record offset */
 
 /* ldlm reply message body offset */
-#define DLM_LOCKREPLY_OFF	       1 /* lockrep offset */
-#define DLM_REPLY_REC_OFF	       2 /* reply record offset */
+#define DLM_LOCKREPLY_OFF		1 /* lockrep offset */
+#define DLM_REPLY_REC_OFF		2 /* reply record offset */
 
 /** only use in req->rq_{req,rep}_swab_mask */
-#define MSG_PTLRPC_HEADER_OFF	   31
+#define MSG_PTLRPC_HEADER_OFF		31
 
 /* Flags that are operation-specific go in the top 16 bits. */
-#define MSG_OP_FLAG_MASK   0xffff0000
-#define MSG_OP_FLAG_SHIFT  16
+#define MSG_OP_FLAG_MASK		0xffff0000
+#define MSG_OP_FLAG_SHIFT		16
 
 /* Flags that apply to all requests are in the bottom 16 bits */
-#define MSG_GEN_FLAG_MASK     0x0000ffff
-#define MSG_LAST_REPLAY	   0x0001
+#define MSG_GEN_FLAG_MASK	0x0000ffff
+#define MSG_LAST_REPLAY		0x0001
 #define MSG_RESENT		0x0002
 #define MSG_REPLAY		0x0004
-/* #define MSG_AT_SUPPORT	 0x0008
+/* #define MSG_AT_SUPPORT	0x0008
  * This was used in early prototypes of adaptive timeouts, and while there
  * shouldn't be any users of that code there also isn't a need for using this
  * bits. Defer usage until at least 1.10 to avoid potential conflict.
  */
-#define MSG_DELAY_REPLAY	  0x0010
+#define MSG_DELAY_REPLAY	0x0010
 #define MSG_VERSION_REPLAY	0x0020
-#define MSG_REQ_REPLAY_DONE       0x0040
-#define MSG_LOCK_REPLAY_DONE      0x0080
+#define MSG_REQ_REPLAY_DONE	0x0040
+#define MSG_LOCK_REPLAY_DONE	0x0080
 
 /*
  * Flags for all connect opcodes (MDS_CONNECT, OST_CONNECT)
  */
 
-#define MSG_CONNECT_RECOVERING  0x00000001
-#define MSG_CONNECT_RECONNECT   0x00000002
-#define MSG_CONNECT_REPLAYABLE  0x00000004
+#define MSG_CONNECT_RECOVERING	0x00000001
+#define MSG_CONNECT_RECONNECT	0x00000002
+#define MSG_CONNECT_REPLAYABLE	0x00000004
 /*#define MSG_CONNECT_PEER	0x8 */
-#define MSG_CONNECT_LIBCLIENT   0x00000010
-#define MSG_CONNECT_INITIAL     0x00000020
-#define MSG_CONNECT_ASYNC       0x00000040
-#define MSG_CONNECT_NEXT_VER    0x00000080 /* use next version of lustre_msg */
-#define MSG_CONNECT_TRANSNO     0x00000100 /* report transno */
+#define MSG_CONNECT_LIBCLIENT	0x00000010
+#define MSG_CONNECT_INITIAL	0x00000020
+#define MSG_CONNECT_ASYNC	0x00000040
+#define MSG_CONNECT_NEXT_VER	0x00000080 /* use next version of lustre_msg */
+#define MSG_CONNECT_TRANSNO	0x00000100 /* report transno */
 
 /* Connect flags */
 #define OBD_CONNECT_RDONLY		  0x1ULL /*client has read-only access*/
@@ -715,7 +715,7 @@ struct ptlrpc_body_v2 {
  * between 2.2 and 2.3 x86/ppc nodes, and can be removed when interop for
  * 2.2 clients/servers is no longer needed.  LU-1252/LU-1644.
  */
-#define OBD_CONNECT_MNE_SWAB		 OBD_CONNECT_MDS_MDS
+#define OBD_CONNECT_MNE_SWAB		OBD_CONNECT_MDS_MDS
 
 #define OCD_HAS_FLAG(ocd, flg)  \
 	(!!((ocd)->ocd_connect_flags & OBD_CONNECT_##flg))
@@ -730,30 +730,30 @@ struct ptlrpc_body_v2 {
  * almost certainly will, then perhaps we stick a union in here.
  */
 struct obd_connect_data {
-	__u64 ocd_connect_flags; /* OBD_CONNECT_* per above */
-	__u32 ocd_version;	 /* lustre release version number */
-	__u32 ocd_grant;	 /* initial cache grant amount (bytes) */
-	__u32 ocd_index;	 /* LOV index to connect to */
-	__u32 ocd_brw_size;	 /* Maximum BRW size in bytes */
-	__u64 ocd_ibits_known;   /* inode bits this client understands */
-	__u8  ocd_grant_blkbits; /* log2 of the backend filesystem blocksize */
-	__u8  ocd_grant_inobits; /* log2 of the per-inode space consumption */
-	__u16 ocd_grant_tax_kb;  /* extent grant overhead, in 1K blocks */
-	__u32 ocd_grant_max_blks;/* maximum number of blocks per extent */
-	__u64 ocd_transno;       /* first transno from client to be replayed */
-	__u32 ocd_group;	 /* MDS group on OST */
-	__u32 ocd_cksum_types;   /* supported checksum algorithms */
-	__u32 ocd_max_easize;    /* How big LOV EA can be on MDS */
-	__u32 ocd_instance;      /* instance # of this target */
-	__u64 ocd_maxbytes;      /* Maximum stripe size in bytes */
+	__u64 ocd_connect_flags;	/* OBD_CONNECT_* per above */
+	__u32 ocd_version;		/* lustre release version number */
+	__u32 ocd_grant;		/* initial cache grant amount (bytes) */
+	__u32 ocd_index;		/* LOV index to connect to */
+	__u32 ocd_brw_size;		/* Maximum BRW size in bytes */
+	__u64 ocd_ibits_known;		/* inode bits this client understands */
+	__u8  ocd_grant_blkbits;	/* log2 of the backend filesystem blocksize */
+	__u8  ocd_grant_inobits;	/* log2 of the per-inode space consumption */
+	__u16 ocd_grant_tax_kb;		/* extent grant overhead, in 1K blocks */
+	__u32 ocd_grant_max_blks;	/* maximum number of blocks per extent */
+	__u64 ocd_transno;		/* first transno from client to be replayed */
+	__u32 ocd_group;		/* MDS group on OST */
+	__u32 ocd_cksum_types;		/* supported checksum algorithms */
+	__u32 ocd_max_easize;		/* How big LOV EA can be on MDS */
+	__u32 ocd_instance;		/* instance # of this target */
+	__u64 ocd_maxbytes;		/* Maximum stripe size in bytes */
 	/* Fields after ocd_maxbytes are only accessible by the receiver
 	 * if the corresponding flag in ocd_connect_flags is set. Accessing
 	 * any field after ocd_maxbytes on the receiver without a valid flag
 	 * may result in out-of-bound memory access and kernel oops.
 	 */
-	__u16 ocd_maxmodrpcs;	/* Maximum modify RPCs in parallel */
-	__u16 padding0;		/* added 2.1.0. also fix lustre_swab_connect */
-	__u32 padding1;		/* added 2.1.0. also fix lustre_swab_connect */
+	__u16 ocd_maxmodrpcs;		/* Maximum modify RPCs in parallel */
+	__u16 padding0;			/* added 2.1.0. also fix lustre_swab_connect */
+	__u32 padding1;			/* added 2.1.0. also fix lustre_swab_connect */
 	__u64 ocd_connect_flags2;
 	__u64 padding3;	  /* added 2.1.0. also fix lustre_swab_connect */
 	__u64 padding4;	  /* added 2.1.0. also fix lustre_swab_connect */
@@ -816,7 +816,7 @@ enum ost_cmd {
 	OST_QUOTACHECK = 18, /* not used since 2.4 */
 	OST_QUOTACTL   = 19,
 	OST_QUOTA_ADJUST_QUNIT = 20, /* not used since 2.4 */
-	OST_LADVISE		= 21,
+	OST_LADVISE    = 21,
 	OST_LAST_OPC /* must be < 33 to avoid MDS_GETATTR */
 };
 #define OST_FIRST_OPC  OST_REPLY
@@ -825,7 +825,7 @@ enum obdo_flags {
 	OBD_FL_INLINEDATA   = 0x00000001,
 	OBD_FL_OBDMDEXISTS  = 0x00000002,
 	OBD_FL_DELORPHAN    = 0x00000004, /* if set in o_flags delete orphans */
-	OBD_FL_NORPC	    = 0x00000008, /* set in o_flags do in OSC not OST */
+	OBD_FL_NORPC        = 0x00000008, /* set in o_flags do in OSC not OST */
 	OBD_FL_IDONLY       = 0x00000010, /* set in o_flags only adjust obj id*/
 	OBD_FL_RECREATE_OBJS = 0x00000020, /* recreate missing obj */
 	OBD_FL_DEBUG_CHECK  = 0x00000040, /* echo client/server debug check */
@@ -839,14 +839,14 @@ enum obdo_flags {
 	OBD_FL_CKSUM_RSVD2  = 0x00008000, /* for future cksum types */
 	OBD_FL_CKSUM_RSVD3  = 0x00010000, /* for future cksum types */
 	OBD_FL_SHRINK_GRANT = 0x00020000, /* object shrink the grant */
-	OBD_FL_MMAP	    = 0x00040000, /* object is mmapped on the client.
+	OBD_FL_MMAP         = 0x00040000, /* object is mmapped on the client.
 					   * XXX: obsoleted - reserved for old
 					   * clients prior than 2.2
 					   */
 	OBD_FL_RECOV_RESEND = 0x00080000, /* recoverable resent */
 	OBD_FL_NOSPC_BLK    = 0x00100000, /* no more block space on OST */
-	OBD_FL_FLUSH	    = 0x00200000, /* flush pages on the OST */
-	OBD_FL_SHORT_IO	    = 0x00400000, /* short io request */
+	OBD_FL_FLUSH        = 0x00200000, /* flush pages on the OST */
+	OBD_FL_SHORT_IO     = 0x00400000, /* short io request */
 
 	/* Note that while these checksum values are currently separate bits,
 	 * in 2.x we can actually allow all values from 1-31 if we wanted.
@@ -888,28 +888,28 @@ enum obdo_flags {
  * hints), so MDT replaces magic with appropriate one and now LOD can
  * easily understand what's inside -bzzz
  */
-#define LOV_MAGIC_V1_DEF  0x0CD10BD0
-#define LOV_MAGIC_V3_DEF  0x0CD30BD0
+#define LOV_MAGIC_V1_DEF	0x0CD10BD0
+#define LOV_MAGIC_V3_DEF	0x0CD30BD0
 
 #define lov_pattern(pattern)		(pattern & ~LOV_PATTERN_F_MASK)
 #define lov_pattern_flags(pattern)	(pattern & LOV_PATTERN_F_MASK)
 
 #define lov_ost_data lov_ost_data_v1
-struct lov_ost_data_v1 {	  /* per-stripe data structure (little-endian)*/
-	struct ost_id l_ost_oi;	  /* OST object ID */
-	__u32 l_ost_gen;	  /* generation of this l_ost_idx */
-	__u32 l_ost_idx;	  /* OST index in LOV (lov_tgt_desc->tgts) */
+struct lov_ost_data_v1 {	/* per-stripe data structure (little-endian)*/
+	struct ost_id l_ost_oi;	/* OST object ID */
+	__u32 l_ost_gen;	/* generation of this l_ost_idx */
+	__u32 l_ost_idx;	/* OST index in LOV (lov_tgt_desc->tgts) */
 };
 
 #define lov_mds_md lov_mds_md_v1
-struct lov_mds_md_v1 {	    /* LOV EA mds/wire data (little-endian) */
-	__u32 lmm_magic;	  /* magic number = LOV_MAGIC_V1 */
-	__u32 lmm_pattern;	/* LOV_PATTERN_RAID0, LOV_PATTERN_RAID1 */
-	struct ost_id	lmm_oi;	  /* LOV object ID */
-	__u32 lmm_stripe_size;    /* size of stripe in bytes */
+struct lov_mds_md_v1 {		/* LOV EA mds/wire data (little-endian) */
+	__u32		lmm_magic;	/* magic number = LOV_MAGIC_V1 */
+	__u32		lmm_pattern;	/* LOV_PATTERN_RAID0, LOV_PATTERN_RAID1 */
+	struct ost_id	lmm_oi;		/* LOV object ID */
+	__u32		lmm_stripe_size;/* size of stripe in bytes */
 	/* lmm_stripe_count used to be __u32 */
-	__u16 lmm_stripe_count;   /* num stripes in use for this object */
-	__u16 lmm_layout_gen;     /* layout generation number */
+	__u16		lmm_stripe_count;/* num stripes in use for this object */
+	__u16		lmm_layout_gen;	/* layout generation number */
 	struct lov_ost_data_v1 lmm_objects[0]; /* per-stripe data */
 };
 
@@ -918,32 +918,32 @@ struct lov_mds_md_v1 {	    /* LOV EA mds/wire data (little-endian) */
 #define MIN_MD_SIZE							\
 	(sizeof(struct lov_mds_md) + 1 * sizeof(struct lov_ost_data))
 
-#define XATTR_NAME_ACL_ACCESS   "system.posix_acl_access"
-#define XATTR_NAME_ACL_DEFAULT  "system.posix_acl_default"
-#define XATTR_USER_PREFIX       "user."
-#define XATTR_TRUSTED_PREFIX    "trusted."
-#define XATTR_SECURITY_PREFIX   "security."
-#define XATTR_LUSTRE_PREFIX     "lustre."
+#define XATTR_NAME_ACL_ACCESS	"system.posix_acl_access"
+#define XATTR_NAME_ACL_DEFAULT	"system.posix_acl_default"
+#define XATTR_USER_PREFIX	"user."
+#define XATTR_TRUSTED_PREFIX	"trusted."
+#define XATTR_SECURITY_PREFIX	"security."
+#define XATTR_LUSTRE_PREFIX	"lustre."
 
-#define XATTR_NAME_LOV	  "trusted.lov"
-#define XATTR_NAME_LMA	  "trusted.lma"
-#define XATTR_NAME_LMV	  "trusted.lmv"
+#define XATTR_NAME_LOV		"trusted.lov"
+#define XATTR_NAME_LMA		"trusted.lma"
+#define XATTR_NAME_LMV		"trusted.lmv"
 #define XATTR_NAME_DEFAULT_LMV	"trusted.dmv"
-#define XATTR_NAME_LINK	 "trusted.link"
-#define XATTR_NAME_FID	  "trusted.fid"
-#define XATTR_NAME_VERSION      "trusted.version"
+#define XATTR_NAME_LINK		"trusted.link"
+#define XATTR_NAME_FID		"trusted.fid"
+#define XATTR_NAME_VERSION	"trusted.version"
 #define XATTR_NAME_SOM		"trusted.som"
 #define XATTR_NAME_HSM		"trusted.hsm"
 #define XATTR_NAME_LFSCK_NAMESPACE "trusted.lfsck_namespace"
 
-struct lov_mds_md_v3 {	    /* LOV EA mds/wire data (little-endian) */
-	__u32 lmm_magic;	  /* magic number = LOV_MAGIC_V3 */
+struct lov_mds_md_v3 {		/* LOV EA mds/wire data (little-endian) */
+	__u32 lmm_magic;	/* magic number = LOV_MAGIC_V3 */
 	__u32 lmm_pattern;	/* LOV_PATTERN_RAID0, LOV_PATTERN_RAID1 */
-	struct ost_id	lmm_oi;	  /* LOV object ID */
-	__u32 lmm_stripe_size;    /* size of stripe in bytes */
+	struct ost_id	lmm_oi;	/* LOV object ID */
+	__u32 lmm_stripe_size;	/* size of stripe in bytes */
 	/* lmm_stripe_count used to be __u32 */
-	__u16 lmm_stripe_count;   /* num stripes in use for this object */
-	__u16 lmm_layout_gen;     /* layout generation number */
+	__u16 lmm_stripe_count;	/* num stripes in use for this object */
+	__u16 lmm_layout_gen;	/* layout generation number */
 	char  lmm_pool_name[LOV_MAXPOOLNAME + 1]; /* must be 32bit aligned */
 	struct lov_ost_data_v1 lmm_objects[0]; /* per-stripe data */
 };
@@ -983,72 +983,71 @@ lov_mds_md_max_stripe_count(size_t buf_size, __u32 lmm_magic)
 	}
 }
 
-#define OBD_MD_FLID	   (0x00000001ULL) /* object ID */
-#define OBD_MD_FLATIME     (0x00000002ULL) /* access time */
-#define OBD_MD_FLMTIME     (0x00000004ULL) /* data modification time */
-#define OBD_MD_FLCTIME     (0x00000008ULL) /* change time */
-#define OBD_MD_FLSIZE      (0x00000010ULL) /* size */
-#define OBD_MD_FLBLOCKS    (0x00000020ULL) /* allocated blocks count */
-#define OBD_MD_FLBLKSZ     (0x00000040ULL) /* block size */
-#define OBD_MD_FLMODE      (0x00000080ULL) /* access bits (mode & ~S_IFMT) */
-#define OBD_MD_FLTYPE      (0x00000100ULL) /* object type (mode & S_IFMT) */
-#define OBD_MD_FLUID       (0x00000200ULL) /* user ID */
-#define OBD_MD_FLGID       (0x00000400ULL) /* group ID */
-#define OBD_MD_FLFLAGS     (0x00000800ULL) /* flags word */
-#define OBD_MD_FLNLINK     (0x00002000ULL) /* link count */
-#define OBD_MD_FLGENER     (0x00004000ULL) /* generation number */
-/*#define OBD_MD_FLINLINE    (0x00008000ULL)  inline data. used until 1.6.5 */
-#define OBD_MD_FLRDEV      (0x00010000ULL) /* device number */
-#define OBD_MD_FLEASIZE    (0x00020000ULL) /* extended attribute data */
-#define OBD_MD_LINKNAME    (0x00040000ULL) /* symbolic link target */
-#define OBD_MD_FLHANDLE    (0x00080000ULL) /* file/lock handle */
-#define OBD_MD_FLCKSUM     (0x00100000ULL) /* bulk data checksum */
-#define OBD_MD_FLQOS       (0x00200000ULL) /* quality of service stats */
-/*#define OBD_MD_FLOSCOPQ    (0x00400000ULL) osc opaque data, never used */
-/*	OBD_MD_FLCOOKIE    (0x00800000ULL) obsolete in 2.8 */
-#define OBD_MD_FLGROUP     (0x01000000ULL) /* group */
-#define OBD_MD_FLFID       (0x02000000ULL) /* ->ost write inline fid */
-#define OBD_MD_FLEPOCH     (0x04000000ULL) /* ->ost write with ioepoch */
-					   /* ->mds if epoch opens or closes
-					    */
-#define OBD_MD_FLGRANT     (0x08000000ULL) /* ost preallocation space grant */
-#define OBD_MD_FLDIREA     (0x10000000ULL) /* dir's extended attribute data */
-#define OBD_MD_FLUSRQUOTA  (0x20000000ULL) /* over quota flags sent from ost */
-#define OBD_MD_FLGRPQUOTA  (0x40000000ULL) /* over quota flags sent from ost */
-#define OBD_MD_FLMODEASIZE (0x80000000ULL) /* EA size will be changed */
-
-#define OBD_MD_MDS	   (0x0000000100000000ULL) /* where an inode lives on */
-#define OBD_MD_REINT       (0x0000000200000000ULL) /* reintegrate oa */
-#define OBD_MD_MEA	   (0x0000000400000000ULL) /* CMD split EA  */
-#define OBD_MD_TSTATE      (0x0000000800000000ULL) /* transient state field */
-
-#define OBD_MD_FLXATTR       (0x0000001000000000ULL) /* xattr */
-#define OBD_MD_FLXATTRLS     (0x0000002000000000ULL) /* xattr list */
-#define OBD_MD_FLXATTRRM     (0x0000004000000000ULL) /* xattr remove */
-#define OBD_MD_FLACL	     (0x0000008000000000ULL) /* ACL */
-/*	OBD_MD_FLRMTPERM     (0x0000010000000000ULL) remote perm, obsolete */
-#define OBD_MD_FLMDSCAPA     (0x0000020000000000ULL) /* MDS capability */
-#define OBD_MD_FLOSSCAPA     (0x0000040000000000ULL) /* OSS capability */
-#define OBD_MD_FLCKSPLIT     (0x0000080000000000ULL) /* Check split on server */
-#define OBD_MD_FLCROSSREF    (0x0000100000000000ULL) /* Cross-ref case */
-#define OBD_MD_FLGETATTRLOCK (0x0000200000000000ULL) /* Get IOEpoch attributes
-						      * under lock; for xattr
-						      * requests means the
-						      * client holds the lock
-						      */
-#define OBD_MD_FLOBJCOUNT    (0x0000400000000000ULL) /* for multiple destroy */
-
-/*	OBD_MD_FLRMTLSETFACL (0x0001000000000000ULL) lfs lsetfacl, obsolete */
-/*	OBD_MD_FLRMTLGETFACL (0x0002000000000000ULL) lfs lgetfacl, obsolete */
-/*	OBD_MD_FLRMTRSETFACL (0x0004000000000000ULL) lfs rsetfacl, obsolete */
-/*	OBD_MD_FLRMTRGETFACL (0x0008000000000000ULL) lfs rgetfacl, obsolete */
-
-#define OBD_MD_FLDATAVERSION (0x0010000000000000ULL) /* iversion sum */
+#define OBD_MD_FLID		(0x00000001ULL) /* object ID */
+#define OBD_MD_FLATIME		(0x00000002ULL) /* access time */
+#define OBD_MD_FLMTIME		(0x00000004ULL) /* data modification time */
+#define OBD_MD_FLCTIME		(0x00000008ULL) /* change time */
+#define OBD_MD_FLSIZE		(0x00000010ULL) /* size */
+#define OBD_MD_FLBLOCKS		(0x00000020ULL) /* allocated blocks count */
+#define OBD_MD_FLBLKSZ		(0x00000040ULL) /* block size */
+#define OBD_MD_FLMODE		(0x00000080ULL) /* access bits (mode & ~S_IFMT) */
+#define OBD_MD_FLTYPE		(0x00000100ULL) /* object type (mode & S_IFMT) */
+#define OBD_MD_FLUID		(0x00000200ULL) /* user ID */
+#define OBD_MD_FLGID		(0x00000400ULL) /* group ID */
+#define OBD_MD_FLFLAGS		(0x00000800ULL) /* flags word */
+#define OBD_MD_FLNLINK		(0x00002000ULL) /* link count */
+#define OBD_MD_FLGENER		(0x00004000ULL) /* generation number */
+/*#define OBD_MD_FLINLINE	(0x00008000ULL)  inline data. used until 1.6.5 */
+#define OBD_MD_FLRDEV		(0x00010000ULL) /* device number */
+#define OBD_MD_FLEASIZE		(0x00020000ULL) /* extended attribute data */
+#define OBD_MD_LINKNAME		(0x00040000ULL) /* symbolic link target */
+#define OBD_MD_FLHANDLE		(0x00080000ULL) /* file/lock handle */
+#define OBD_MD_FLCKSUM		(0x00100000ULL) /* bulk data checksum */
+#define OBD_MD_FLQOS		(0x00200000ULL) /* quality of service stats */
+/*#define OBD_MD_FLOSCOPQ	(0x00400000ULL) osc opaque data, never used */
+/*	OBD_MD_FLCOOKIE		(0x00800000ULL) obsolete in 2.8 */
+#define OBD_MD_FLGROUP		(0x01000000ULL) /* group */
+#define OBD_MD_FLFID		(0x02000000ULL) /* ->ost write inline fid */
+#define OBD_MD_FLEPOCH		(0x04000000ULL) /* ->ost write with ioepoch */
+						/* ->mds if epoch opens or closes */
+#define OBD_MD_FLGRANT		(0x08000000ULL) /* ost preallocation space grant */
+#define OBD_MD_FLDIREA		(0x10000000ULL) /* dir's extended attribute data */
+#define OBD_MD_FLUSRQUOTA	(0x20000000ULL) /* over quota flags sent from ost */
+#define OBD_MD_FLGRPQUOTA	(0x40000000ULL) /* over quota flags sent from ost */
+#define OBD_MD_FLMODEASIZE	(0x80000000ULL) /* EA size will be changed */
+
+#define OBD_MD_MDS		(0x0000000100000000ULL) /* where an inode lives on */
+#define OBD_MD_REINT		(0x0000000200000000ULL) /* reintegrate oa */
+#define OBD_MD_MEA		(0x0000000400000000ULL) /* CMD split EA  */
+#define OBD_MD_TSTATE		(0x0000000800000000ULL) /* transient state field */
+
+#define OBD_MD_FLXATTR		(0x0000001000000000ULL) /* xattr */
+#define OBD_MD_FLXATTRLS	(0x0000002000000000ULL) /* xattr list */
+#define OBD_MD_FLXATTRRM	(0x0000004000000000ULL) /* xattr remove */
+#define OBD_MD_FLACL		(0x0000008000000000ULL) /* ACL */
+/*	OBD_MD_FLRMTPERM	(0x0000010000000000ULL) remote perm, obsolete */
+#define OBD_MD_FLMDSCAPA	(0x0000020000000000ULL) /* MDS capability */
+#define OBD_MD_FLOSSCAPA	(0x0000040000000000ULL) /* OSS capability */
+#define OBD_MD_FLCKSPLIT	(0x0000080000000000ULL) /* Check split on server */
+#define OBD_MD_FLCROSSREF	(0x0000100000000000ULL) /* Cross-ref case */
+#define OBD_MD_FLGETATTRLOCK	(0x0000200000000000ULL) /* Get IOEpoch attributes
+							 * under lock; for xattr
+							 * requests means the
+							 * client holds the lock
+							 */
+#define OBD_MD_FLOBJCOUNT	(0x0000400000000000ULL) /* for multiple destroy */
+
+/*	OBD_MD_FLRMTLSETFACL	(0x0001000000000000ULL) lfs lsetfacl, obsolete */
+/*	OBD_MD_FLRMTLGETFACL	(0x0002000000000000ULL) lfs lgetfacl, obsolete */
+/*	OBD_MD_FLRMTRSETFACL	(0x0004000000000000ULL) lfs rsetfacl, obsolete */
+/*	OBD_MD_FLRMTRGETFACL	(0x0008000000000000ULL) lfs rgetfacl, obsolete */
+
+#define OBD_MD_FLDATAVERSION	(0x0010000000000000ULL) /* iversion sum */
 #define OBD_MD_CLOSE_INTENT_EXECED (0x0020000000000000ULL) /* close intent
 							    * executed
 							    */
 
-#define OBD_MD_DEFAULT_MEA   (0x0040000000000000ULL) /* default MEA */
+#define OBD_MD_DEFAULT_MEA	(0x0040000000000000ULL) /* default MEA */
 
 #define OBD_MD_FLGETATTR (OBD_MD_FLID    | OBD_MD_FLATIME | OBD_MD_FLMTIME | \
 			  OBD_MD_FLCTIME | OBD_MD_FLSIZE  | OBD_MD_FLBLKSZ | \
@@ -1085,7 +1084,7 @@ struct hsm_state_set {
 				      * the grant.
 				      */
 #define OBD_BRW_CHECK		0x10
-#define OBD_BRW_FROM_GRANT      0x20 /* the osc manages this under llite */
+#define OBD_BRW_FROM_GRANT	0x20 /* the osc manages this under llite */
 #define OBD_BRW_GRANTED		0x40 /* the ost manages this */
 #define OBD_BRW_NOCACHE		0x80 /* this page is a part of non-cached IO */
 #define OBD_BRW_NOQUOTA	       0x100
@@ -1280,41 +1279,43 @@ enum mds_cmd {
  */
 
 enum mdt_reint_cmd {
-	REINT_SETATTR  = 1,
-	REINT_CREATE   = 2,
-	REINT_LINK     = 3,
-	REINT_UNLINK   = 4,
-	REINT_RENAME   = 5,
-	REINT_OPEN     = 6,
-	REINT_SETXATTR = 7,
-	REINT_RMENTRY  = 8,
-	REINT_MIGRATE  = 9,
+	REINT_SETATTR	= 1,
+	REINT_CREATE	= 2,
+	REINT_LINK	= 3,
+	REINT_UNLINK	= 4,
+	REINT_RENAME	= 5,
+	REINT_OPEN	= 6,
+	REINT_SETXATTR	= 7,
+	REINT_RMENTRY	= 8,
+	REINT_MIGRATE	= 9,
 	REINT_MAX
 };
 
 /* the disposition of the intent outlines what was executed */
-#define DISP_IT_EXECD	0x00000001
-#define DISP_LOOKUP_EXECD    0x00000002
-#define DISP_LOOKUP_NEG      0x00000004
-#define DISP_LOOKUP_POS      0x00000008
-#define DISP_OPEN_CREATE     0x00000010
-#define DISP_OPEN_OPEN       0x00000020
-#define DISP_ENQ_COMPLETE    0x00400000		/* obsolete and unused */
-#define DISP_ENQ_OPEN_REF    0x00800000
-#define DISP_ENQ_CREATE_REF  0x01000000
-#define DISP_OPEN_LOCK       0x02000000
-#define DISP_OPEN_LEASE      0x04000000
-#define DISP_OPEN_STRIPE     0x08000000
+#define DISP_IT_EXECD		0x00000001
+#define DISP_LOOKUP_EXECD	0x00000002
+#define DISP_LOOKUP_NEG		0x00000004
+#define DISP_LOOKUP_POS		0x00000008
+#define DISP_OPEN_CREATE	0x00000010
+#define DISP_OPEN_OPEN		0x00000020
+#define DISP_ENQ_COMPLETE	0x00400000		/* obsolete and unused */
+#define DISP_ENQ_OPEN_REF	0x00800000
+#define DISP_ENQ_CREATE_REF	0x01000000
+#define DISP_OPEN_LOCK		0x02000000
+#define DISP_OPEN_LEASE		0x04000000
+#define DISP_OPEN_STRIPE	0x08000000
 #define DISP_OPEN_DENY		0x10000000
 
 /* INODE LOCK PARTS */
-#define MDS_INODELOCK_LOOKUP 0x000001	/* For namespace, dentry etc, and also
-					 * was used to protect permission (mode,
-					 * owner, group etc) before 2.4.
-					 */
-#define MDS_INODELOCK_UPDATE 0x000002	/* size, links, timestamps */
-#define MDS_INODELOCK_OPEN   0x000004	/* For opened files */
-#define MDS_INODELOCK_LAYOUT 0x000008	/* for layout */
+#define MDS_INODELOCK_LOOKUP	0x000001	/*
+						 * For namespace, dentry etc, and
+						 * also was used to protect
+						 * permission (mode, owner, group
+						 * etc) before 2.4.
+						 */
+#define MDS_INODELOCK_UPDATE	0x000002	/* size, links, timestamps */
+#define MDS_INODELOCK_OPEN	0x000004	/* For opened files */
+#define MDS_INODELOCK_LAYOUT	0x000008	/* for layout */
 
 /* The PERM bit is added int 2.4, and it is used to protect permission(mode,
  * owner, group, acl etc), so to separate the permission from LOOKUP lock.
@@ -1326,8 +1327,8 @@ enum mdt_reint_cmd {
  * grant UPDATE_LOCK|PERM_LOCK, and the remote MDT, where the name entry is,
  * will grant LOOKUP_LOCK.
  */
-#define MDS_INODELOCK_PERM   0x000010
-#define MDS_INODELOCK_XATTR  0x000020	/* extended attributes */
+#define MDS_INODELOCK_PERM	0x000010
+#define MDS_INODELOCK_XATTR	0x000020	/* extended attributes */
 
 #define MDS_INODELOCK_MAXSHIFT 5
 /* This FULL lock is useful to take on unlink sort of operations */
@@ -1352,13 +1353,13 @@ enum {
 /* these should be identical to their EXT4_*_FL counterparts, they are
  * redefined here only to avoid dragging in fs/ext4/ext4.h
  */
-#define LUSTRE_SYNC_FL	 0x00000008 /* Synchronous updates */
-#define LUSTRE_IMMUTABLE_FL    0x00000010 /* Immutable file */
-#define LUSTRE_APPEND_FL       0x00000020 /* writes to file may only append */
+#define LUSTRE_SYNC_FL		0x00000008 /* Synchronous updates */
+#define LUSTRE_IMMUTABLE_FL	0x00000010 /* Immutable file */
+#define LUSTRE_APPEND_FL	0x00000020 /* writes to file may only append */
 #define LUSTRE_NODUMP_FL	0x00000040 /* do not dump file */
-#define LUSTRE_NOATIME_FL      0x00000080 /* do not update atime */
+#define LUSTRE_NOATIME_FL	0x00000080 /* do not update atime */
 #define LUSTRE_INDEX_FL		0x00001000 /* hash-indexed directory */
-#define LUSTRE_DIRSYNC_FL      0x00010000 /* dirsync behaviour (dir only) */
+#define LUSTRE_DIRSYNC_FL	0x00010000 /* dirsync behaviour (dir only) */
 #define LUSTRE_TOPDIR_FL	0x00020000 /* Top of directory hierarchies*/
 #define LUSTRE_DIRECTIO_FL	0x00100000 /* Use direct i/o */
 #define LUSTRE_INLINE_DATA_FL	0x10000000 /* Inode has inline data. */
@@ -1442,37 +1443,37 @@ struct mdt_ioepoch {
 
 /* permissions for md_perm.mp_perm */
 enum {
-	CFS_SETUID_PERM = 0x01,
-	CFS_SETGID_PERM = 0x02,
-	CFS_SETGRP_PERM = 0x04,
+	CFS_SETUID_PERM =	0x01,
+	CFS_SETGID_PERM =	0x02,
+	CFS_SETGRP_PERM =	0x04,
 };
 
 struct mdt_rec_setattr {
-	__u32	   sa_opcode;
-	__u32	   sa_cap;
-	__u32	   sa_fsuid;
-	__u32	   sa_fsuid_h;
-	__u32	   sa_fsgid;
-	__u32	   sa_fsgid_h;
-	__u32	   sa_suppgid;
-	__u32	   sa_suppgid_h;
-	__u32	   sa_padding_1;
-	__u32	   sa_padding_1_h;
-	struct lu_fid   sa_fid;
-	__u64	   sa_valid;
-	__u32	   sa_uid;
-	__u32	   sa_gid;
-	__u64	   sa_size;
-	__u64	   sa_blocks;
-	__s64	   sa_mtime;
-	__s64	   sa_atime;
-	__s64	   sa_ctime;
-	__u32	   sa_attr_flags;
-	__u32	   sa_mode;
-	__u32	   sa_bias;      /* some operation flags */
-	__u32	   sa_padding_3;
-	__u32	   sa_padding_4;
-	__u32	   sa_padding_5;
+	__u32		sa_opcode;
+	__u32		sa_cap;
+	__u32		sa_fsuid;
+	__u32		sa_fsuid_h;
+	__u32		sa_fsgid;
+	__u32		sa_fsgid_h;
+	__u32		sa_suppgid;
+	__u32		sa_suppgid_h;
+	__u32		sa_padding_1;
+	__u32		sa_padding_1_h;
+	struct lu_fid	sa_fid;
+	__u64		sa_valid;
+	__u32		sa_uid;
+	__u32		sa_gid;
+	__u64		sa_size;
+	__u64		sa_blocks;
+	__s64		sa_mtime;
+	__s64		sa_atime;
+	__s64		sa_ctime;
+	__u32		sa_attr_flags;
+	__u32		sa_mode;
+	__u32		sa_bias;      /* some operation flags */
+	__u32		sa_padding_3;
+	__u32		sa_padding_4;
+	__u32		sa_padding_5;
 };
 
 /*
@@ -1481,40 +1482,40 @@ struct mdt_rec_setattr {
  * since the client and MDS may run different kernels (see bug 13828)
  * Therefore, we should only use MDS_ATTR_* attributes for sa_valid.
  */
-#define MDS_ATTR_MODE	       0x1ULL /* = 1 */
-#define MDS_ATTR_UID	       0x2ULL /* = 2 */
-#define MDS_ATTR_GID	       0x4ULL /* = 4 */
-#define MDS_ATTR_SIZE	       0x8ULL /* = 8 */
-#define MDS_ATTR_ATIME	      0x10ULL /* = 16 */
-#define MDS_ATTR_MTIME	      0x20ULL /* = 32 */
-#define MDS_ATTR_CTIME	      0x40ULL /* = 64 */
-#define MDS_ATTR_ATIME_SET    0x80ULL /* = 128 */
-#define MDS_ATTR_MTIME_SET   0x100ULL /* = 256 */
-#define MDS_ATTR_FORCE       0x200ULL /* = 512, Not a change, but a change it */
-#define MDS_ATTR_ATTR_FLAG   0x400ULL /* = 1024 */
-#define MDS_ATTR_KILL_SUID   0x800ULL /* = 2048 */
-#define MDS_ATTR_KILL_SGID  0x1000ULL /* = 4096 */
-#define MDS_ATTR_CTIME_SET  0x2000ULL /* = 8192 */
-#define MDS_ATTR_FROM_OPEN  0x4000ULL /* = 16384, called from open path,
-				       * ie O_TRUNC
-				       */
-#define MDS_ATTR_BLOCKS     0x8000ULL /* = 32768 */
+#define MDS_ATTR_MODE		   0x1ULL /* = 1 */
+#define MDS_ATTR_UID		   0x2ULL /* = 2 */
+#define MDS_ATTR_GID		   0x4ULL /* = 4 */
+#define MDS_ATTR_SIZE		   0x8ULL /* = 8 */
+#define MDS_ATTR_ATIME		  0x10ULL /* = 16 */
+#define MDS_ATTR_MTIME		  0x20ULL /* = 32 */
+#define MDS_ATTR_CTIME		  0x40ULL /* = 64 */
+#define MDS_ATTR_ATIME_SET	  0x80ULL /* = 128 */
+#define MDS_ATTR_MTIME_SET	 0x100ULL /* = 256 */
+#define MDS_ATTR_FORCE		 0x200ULL /* = 512, Not a change, but a change it */
+#define MDS_ATTR_ATTR_FLAG	 0x400ULL /* = 1024 */
+#define MDS_ATTR_KILL_SUID	 0x800ULL /* = 2048 */
+#define MDS_ATTR_KILL_SGID	0x1000ULL /* = 4096 */
+#define MDS_ATTR_CTIME_SET	0x2000ULL /* = 8192 */
+#define MDS_ATTR_FROM_OPEN	0x4000ULL /* = 16384, called from open path,
+					   * ie O_TRUNC
+					   */
+#define MDS_ATTR_BLOCKS		0x8000ULL /* = 32768 */
 
-#define MDS_FMODE_CLOSED	 00000000
-#define MDS_FMODE_EXEC	   00000004
+#define MDS_FMODE_CLOSED	00000000
+#define MDS_FMODE_EXEC		00000004
 /*	MDS_FMODE_EPOCH		01000000 obsolete since 2.8.0 */
 /*	MDS_FMODE_TRUNC		02000000 obsolete since 2.8.0 */
 /*	MDS_FMODE_SOM		04000000 obsolete since 2.8.0 */
 
-#define MDS_OPEN_CREATED	 00000010
-#define MDS_OPEN_CROSS	   00000020
+#define MDS_OPEN_CREATED	00000010
+#define MDS_OPEN_CROSS		00000020
 
-#define MDS_OPEN_CREAT	   00000100
-#define MDS_OPEN_EXCL	    00000200
-#define MDS_OPEN_TRUNC	   00001000
-#define MDS_OPEN_APPEND	  00002000
-#define MDS_OPEN_SYNC	    00010000
-#define MDS_OPEN_DIRECTORY       00200000
+#define MDS_OPEN_CREAT		00000100
+#define MDS_OPEN_EXCL		00000200
+#define MDS_OPEN_TRUNC		00001000
+#define MDS_OPEN_APPEND		00002000
+#define MDS_OPEN_SYNC		00010000
+#define MDS_OPEN_DIRECTORY	00200000
 
 #define MDS_OPEN_BY_FID		040000000 /* open_by_fid for known object */
 #define MDS_OPEN_DELAY_CREATE  0100000000 /* delay initial object create */
@@ -1567,143 +1568,143 @@ enum mds_op_bias {
 
 /* instance of mdt_reint_rec */
 struct mdt_rec_create {
-	__u32	   cr_opcode;
-	__u32	   cr_cap;
-	__u32	   cr_fsuid;
-	__u32	   cr_fsuid_h;
-	__u32	   cr_fsgid;
-	__u32	   cr_fsgid_h;
-	__u32	   cr_suppgid1;
-	__u32	   cr_suppgid1_h;
-	__u32	   cr_suppgid2;
-	__u32	   cr_suppgid2_h;
+	__u32		cr_opcode;
+	__u32		cr_cap;
+	__u32		cr_fsuid;
+	__u32		cr_fsuid_h;
+	__u32		cr_fsgid;
+	__u32		cr_fsgid_h;
+	__u32		cr_suppgid1;
+	__u32		cr_suppgid1_h;
+	__u32		cr_suppgid2;
+	__u32		cr_suppgid2_h;
 	struct lu_fid   cr_fid1;
 	struct lu_fid   cr_fid2;
 	struct lustre_handle cr_old_handle; /* handle in case of open replay */
-	__s64	   cr_time;
-	__u64	   cr_rdev;
-	__u64	   cr_ioepoch;
-	__u64	   cr_padding_1;   /* rr_blocks */
-	__u32	   cr_mode;
-	__u32	   cr_bias;
+	__s64		cr_time;
+	__u64		cr_rdev;
+	__u64		cr_ioepoch;
+	__u64		cr_padding_1;   /* rr_blocks */
+	__u32		cr_mode;
+	__u32		cr_bias;
 	/* use of helpers set/get_mrc_cr_flags() is needed to access
 	 * 64 bits cr_flags [cr_flags_l, cr_flags_h], this is done to
 	 * extend cr_flags size without breaking 1.8 compat
 	 */
-	__u32	   cr_flags_l;     /* for use with open, low  32 bits  */
-	__u32	   cr_flags_h;     /* for use with open, high 32 bits */
-	__u32	   cr_umask;       /* umask for create */
-	__u32	   cr_padding_4;   /* rr_padding_4 */
+	__u32		cr_flags_l;     /* for use with open, low  32 bits  */
+	__u32		cr_flags_h;     /* for use with open, high 32 bits */
+	__u32		cr_umask;       /* umask for create */
+	__u32		cr_padding_4;   /* rr_padding_4 */
 };
 
 /* instance of mdt_reint_rec */
 struct mdt_rec_link {
-	__u32	   lk_opcode;
-	__u32	   lk_cap;
-	__u32	   lk_fsuid;
-	__u32	   lk_fsuid_h;
-	__u32	   lk_fsgid;
-	__u32	   lk_fsgid_h;
-	__u32	   lk_suppgid1;
-	__u32	   lk_suppgid1_h;
-	__u32	   lk_suppgid2;
-	__u32	   lk_suppgid2_h;
+	__u32		lk_opcode;
+	__u32		lk_cap;
+	__u32		lk_fsuid;
+	__u32		lk_fsuid_h;
+	__u32		lk_fsgid;
+	__u32		lk_fsgid_h;
+	__u32		lk_suppgid1;
+	__u32		lk_suppgid1_h;
+	__u32		lk_suppgid2;
+	__u32		lk_suppgid2_h;
 	struct lu_fid   lk_fid1;
 	struct lu_fid   lk_fid2;
-	__s64	   lk_time;
-	__u64	   lk_padding_1;   /* rr_atime */
-	__u64	   lk_padding_2;   /* rr_ctime */
-	__u64	   lk_padding_3;   /* rr_size */
-	__u64	   lk_padding_4;   /* rr_blocks */
-	__u32	   lk_bias;
-	__u32	   lk_padding_5;   /* rr_mode */
-	__u32	   lk_padding_6;   /* rr_flags */
-	__u32	   lk_padding_7;   /* rr_padding_2 */
-	__u32	   lk_padding_8;   /* rr_padding_3 */
-	__u32	   lk_padding_9;   /* rr_padding_4 */
+	__s64		lk_time;
+	__u64		lk_padding_1;   /* rr_atime */
+	__u64		lk_padding_2;   /* rr_ctime */
+	__u64		lk_padding_3;   /* rr_size */
+	__u64		lk_padding_4;   /* rr_blocks */
+	__u32		lk_bias;
+	__u32		lk_padding_5;   /* rr_mode */
+	__u32		lk_padding_6;   /* rr_flags */
+	__u32		lk_padding_7;   /* rr_padding_2 */
+	__u32		lk_padding_8;   /* rr_padding_3 */
+	__u32		lk_padding_9;   /* rr_padding_4 */
 };
 
 /* instance of mdt_reint_rec */
 struct mdt_rec_unlink {
-	__u32	   ul_opcode;
-	__u32	   ul_cap;
-	__u32	   ul_fsuid;
-	__u32	   ul_fsuid_h;
-	__u32	   ul_fsgid;
-	__u32	   ul_fsgid_h;
-	__u32	   ul_suppgid1;
-	__u32	   ul_suppgid1_h;
-	__u32	   ul_suppgid2;
-	__u32	   ul_suppgid2_h;
+	__u32		ul_opcode;
+	__u32		ul_cap;
+	__u32		ul_fsuid;
+	__u32		ul_fsuid_h;
+	__u32		ul_fsgid;
+	__u32		ul_fsgid_h;
+	__u32		ul_suppgid1;
+	__u32		ul_suppgid1_h;
+	__u32		ul_suppgid2;
+	__u32		ul_suppgid2_h;
 	struct lu_fid   ul_fid1;
 	struct lu_fid   ul_fid2;
-	__s64	   ul_time;
-	__u64	   ul_padding_2;   /* rr_atime */
-	__u64	   ul_padding_3;   /* rr_ctime */
-	__u64	   ul_padding_4;   /* rr_size */
-	__u64	   ul_padding_5;   /* rr_blocks */
-	__u32	   ul_bias;
-	__u32	   ul_mode;
-	__u32	   ul_padding_6;   /* rr_flags */
-	__u32	   ul_padding_7;   /* rr_padding_2 */
-	__u32	   ul_padding_8;   /* rr_padding_3 */
-	__u32	   ul_padding_9;   /* rr_padding_4 */
+	__s64		ul_time;
+	__u64		ul_padding_2;   /* rr_atime */
+	__u64		ul_padding_3;   /* rr_ctime */
+	__u64		ul_padding_4;   /* rr_size */
+	__u64		ul_padding_5;   /* rr_blocks */
+	__u32		ul_bias;
+	__u32		ul_mode;
+	__u32		ul_padding_6;   /* rr_flags */
+	__u32		ul_padding_7;   /* rr_padding_2 */
+	__u32		ul_padding_8;   /* rr_padding_3 */
+	__u32		ul_padding_9;   /* rr_padding_4 */
 };
 
 /* instance of mdt_reint_rec */
 struct mdt_rec_rename {
-	__u32	   rn_opcode;
-	__u32	   rn_cap;
-	__u32	   rn_fsuid;
-	__u32	   rn_fsuid_h;
-	__u32	   rn_fsgid;
-	__u32	   rn_fsgid_h;
-	__u32	   rn_suppgid1;
-	__u32	   rn_suppgid1_h;
-	__u32	   rn_suppgid2;
-	__u32	   rn_suppgid2_h;
+	__u32		rn_opcode;
+	__u32		rn_cap;
+	__u32		rn_fsuid;
+	__u32		rn_fsuid_h;
+	__u32		rn_fsgid;
+	__u32		rn_fsgid_h;
+	__u32		rn_suppgid1;
+	__u32		rn_suppgid1_h;
+	__u32		rn_suppgid2;
+	__u32		rn_suppgid2_h;
 	struct lu_fid   rn_fid1;
 	struct lu_fid   rn_fid2;
-	__s64	   rn_time;
-	__u64	   rn_padding_1;   /* rr_atime */
-	__u64	   rn_padding_2;   /* rr_ctime */
-	__u64	   rn_padding_3;   /* rr_size */
-	__u64	   rn_padding_4;   /* rr_blocks */
-	__u32	   rn_bias;	/* some operation flags */
-	__u32	   rn_mode;	/* cross-ref rename has mode */
-	__u32	   rn_padding_5;   /* rr_flags */
-	__u32	   rn_padding_6;   /* rr_padding_2 */
-	__u32	   rn_padding_7;   /* rr_padding_3 */
-	__u32	   rn_padding_8;   /* rr_padding_4 */
+	__s64		rn_time;
+	__u64		rn_padding_1;   /* rr_atime */
+	__u64		rn_padding_2;   /* rr_ctime */
+	__u64		rn_padding_3;   /* rr_size */
+	__u64		rn_padding_4;   /* rr_blocks */
+	__u32		rn_bias;	/* some operation flags */
+	__u32		rn_mode;	/* cross-ref rename has mode */
+	__u32		rn_padding_5;   /* rr_flags */
+	__u32		rn_padding_6;   /* rr_padding_2 */
+	__u32		rn_padding_7;   /* rr_padding_3 */
+	__u32		rn_padding_8;   /* rr_padding_4 */
 };
 
 /* instance of mdt_reint_rec */
 struct mdt_rec_setxattr {
-	__u32	   sx_opcode;
-	__u32	   sx_cap;
-	__u32	   sx_fsuid;
-	__u32	   sx_fsuid_h;
-	__u32	   sx_fsgid;
-	__u32	   sx_fsgid_h;
-	__u32	   sx_suppgid1;
-	__u32	   sx_suppgid1_h;
-	__u32	   sx_suppgid2;
-	__u32	   sx_suppgid2_h;
+	__u32		sx_opcode;
+	__u32		sx_cap;
+	__u32		sx_fsuid;
+	__u32		sx_fsuid_h;
+	__u32		sx_fsgid;
+	__u32		sx_fsgid_h;
+	__u32		sx_suppgid1;
+	__u32		sx_suppgid1_h;
+	__u32		sx_suppgid2;
+	__u32		sx_suppgid2_h;
 	struct lu_fid   sx_fid;
-	__u64	   sx_padding_1;   /* These three are rr_fid2 */
-	__u32	   sx_padding_2;
-	__u32	   sx_padding_3;
-	__u64	   sx_valid;
-	__s64	   sx_time;
-	__u64	   sx_padding_5;   /* rr_ctime */
-	__u64	   sx_padding_6;   /* rr_size */
-	__u64	   sx_padding_7;   /* rr_blocks */
-	__u32	   sx_size;
-	__u32	   sx_flags;
-	__u32	   sx_padding_8;   /* rr_flags */
-	__u32	   sx_padding_9;   /* rr_padding_2 */
-	__u32	   sx_padding_10;  /* rr_padding_3 */
-	__u32	   sx_padding_11;  /* rr_padding_4 */
+	__u64		sx_padding_1;   /* These three are rr_fid2 */
+	__u32		sx_padding_2;
+	__u32		sx_padding_3;
+	__u64		sx_valid;
+	__s64		sx_time;
+	__u64		sx_padding_5;   /* rr_ctime */
+	__u64		sx_padding_6;   /* rr_size */
+	__u64		sx_padding_7;   /* rr_blocks */
+	__u32		sx_size;
+	__u32		sx_flags;
+	__u32		sx_padding_8;   /* rr_flags */
+	__u32		sx_padding_9;   /* rr_padding_2 */
+	__u32		sx_padding_10;  /* rr_padding_3 */
+	__u32		sx_padding_11;  /* rr_padding_4 */
 };
 
 /*
@@ -1715,29 +1716,29 @@ struct mdt_rec_setxattr {
  * rr_padding_x fields, then update lustre_swab_mdt_rec_reint() also.
  */
 struct mdt_rec_reint {
-	__u32	   rr_opcode;
-	__u32	   rr_cap;
-	__u32	   rr_fsuid;
-	__u32	   rr_fsuid_h;
-	__u32	   rr_fsgid;
-	__u32	   rr_fsgid_h;
-	__u32	   rr_suppgid1;
-	__u32	   rr_suppgid1_h;
-	__u32	   rr_suppgid2;
-	__u32	   rr_suppgid2_h;
+	__u32		rr_opcode;
+	__u32		rr_cap;
+	__u32		rr_fsuid;
+	__u32		rr_fsuid_h;
+	__u32		rr_fsgid;
+	__u32		rr_fsgid_h;
+	__u32		rr_suppgid1;
+	__u32		rr_suppgid1_h;
+	__u32		rr_suppgid2;
+	__u32		rr_suppgid2_h;
 	struct lu_fid   rr_fid1;
 	struct lu_fid   rr_fid2;
-	__s64	   rr_mtime;
-	__s64	   rr_atime;
-	__s64	   rr_ctime;
-	__u64	   rr_size;
-	__u64	   rr_blocks;
-	__u32	   rr_bias;
-	__u32	   rr_mode;
-	__u32	   rr_flags;
-	__u32	   rr_flags_h;
-	__u32	   rr_umask;
-	__u32	   rr_padding_4; /* also fix lustre_swab_mdt_rec_reint */
+	__s64		rr_mtime;
+	__s64		rr_atime;
+	__s64		rr_ctime;
+	__u64		rr_size;
+	__u64		rr_blocks;
+	__u32		rr_bias;
+	__u32		rr_mode;
+	__u32		rr_flags;
+	__u32		rr_flags_h;
+	__u32		rr_umask;
+	__u32		rr_padding_4; /* also fix lustre_swab_mdt_rec_reint */
 };
 
 /* lmv structures */
@@ -1749,7 +1750,7 @@ struct lmv_desc {
 	__u64 ld_default_hash_size;
 	__u64 ld_padding_1;		/* also fix lustre_swab_lmv_desc */
 	__u32 ld_padding_2;		/* also fix lustre_swab_lmv_desc */
-	__u32 ld_qos_maxage;	       /* in second */
+	__u32 ld_qos_maxage;		    /* in second */
 	__u32 ld_padding_3;		/* also fix lustre_swab_lmv_desc */
 	__u32 ld_padding_4;		/* also fix lustre_swab_lmv_desc */
 	struct obd_uuid ld_uuid;
@@ -1866,9 +1867,9 @@ enum fld_rpc_opc {
 };
 
 enum seq_rpc_opc {
-	SEQ_QUERY		       = 700,
+	SEQ_QUERY			    = 700,
 	SEQ_LAST_OPC,
-	SEQ_FIRST_OPC		   = SEQ_QUERY
+	SEQ_FIRST_OPC			= SEQ_QUERY
 };
 
 enum seq_op {
@@ -2075,7 +2076,7 @@ enum mgs_cmd {
 #define KEY_SET_INFO "set_info"
 
 struct mgs_send_param {
-	char	     mgs_param[MGS_PARAM_MAXLEN];
+	char		  mgs_param[MGS_PARAM_MAXLEN];
 };
 
 /* We pass this info to the MGS so it can write config logs */
@@ -2083,64 +2084,64 @@ struct mgs_send_param {
 #define MTI_PARAM_MAXLEN 4096
 #define MTI_NIDS_MAX     32
 struct mgs_target_info {
-	__u32	    mti_lustre_ver;
-	__u32	    mti_stripe_index;
-	__u32	    mti_config_ver;
-	__u32	    mti_flags;
-	__u32	    mti_nid_count;
-	__u32	    mti_instance; /* Running instance of target */
-	char	     mti_fsname[MTI_NAME_MAXLEN];
-	char	     mti_svname[MTI_NAME_MAXLEN];
-	char	     mti_uuid[sizeof(struct obd_uuid)];
-	__u64	    mti_nids[MTI_NIDS_MAX];     /* host nids (lnet_nid_t)*/
-	char	     mti_params[MTI_PARAM_MAXLEN];
+	__u32		mti_lustre_ver;
+	__u32		mti_stripe_index;
+	__u32		mti_config_ver;
+	__u32		mti_flags;
+	__u32		mti_nid_count;
+	__u32		mti_instance; /* Running instance of target */
+	char		mti_fsname[MTI_NAME_MAXLEN];
+	char		mti_svname[MTI_NAME_MAXLEN];
+	char		mti_uuid[sizeof(struct obd_uuid)];
+	__u64		mti_nids[MTI_NIDS_MAX];     /* host nids (lnet_nid_t)*/
+	char		mti_params[MTI_PARAM_MAXLEN];
 };
 
 struct mgs_nidtbl_entry {
-	__u64	   mne_version;    /* table version of this entry */
-	__u32	   mne_instance;   /* target instance # */
-	__u32	   mne_index;      /* target index */
-	__u32	   mne_length;     /* length of this entry - by bytes */
-	__u8	    mne_type;       /* target type LDD_F_SV_TYPE_OST/MDT */
-	__u8	    mne_nid_type;   /* type of nid(mbz). for ipv6. */
-	__u8	    mne_nid_size;   /* size of each NID, by bytes */
-	__u8	    mne_nid_count;  /* # of NIDs in buffer */
+	__u64		mne_version;    /* table version of this entry */
+	__u32		mne_instance;   /* target instance # */
+	__u32		mne_index;      /* target index */
+	__u32		mne_length;     /* length of this entry - by bytes */
+	__u8		mne_type;       /* target type LDD_F_SV_TYPE_OST/MDT */
+	__u8		mne_nid_type;   /* type of nid(mbz). for ipv6. */
+	__u8		mne_nid_size;   /* size of each NID, by bytes */
+	__u8		mne_nid_count;  /* # of NIDs in buffer */
 	union {
 		lnet_nid_t nids[0];     /* variable size buffer for NIDs. */
 	} u;
 };
 
 struct mgs_config_body {
-	char     mcb_name[MTI_NAME_MAXLEN]; /* logname */
-	__u64    mcb_offset;    /* next index of config log to request */
-	__u16    mcb_type;      /* type of log: CONFIG_T_[CONFIG|RECOVER] */
-	__u8     mcb_reserved;
-	__u8     mcb_bits;      /* bits unit size of config log */
-	__u32    mcb_units;     /* # of units for bulk transfer */
+	char		mcb_name[MTI_NAME_MAXLEN]; /* logname */
+	__u64		mcb_offset;    /* next index of config log to request */
+	__u16		mcb_type;      /* type of log: CONFIG_T_[CONFIG|RECOVER] */
+	__u8		mcb_reserved;
+	__u8		mcb_bits;      /* bits unit size of config log */
+	__u32		mcb_units;     /* # of units for bulk transfer */
 };
 
 struct mgs_config_res {
-	__u64    mcr_offset;    /* index of last config log */
-	__u64    mcr_size;      /* size of the log */
+	__u64		mcr_offset;    /* index of last config log */
+	__u64		mcr_size;      /* size of the log */
 };
 
 /* Config marker flags (in config log) */
-#define CM_START       0x01
-#define CM_END	 0x02
-#define CM_SKIP	0x04
-#define CM_UPGRADE146  0x08
-#define CM_EXCLUDE     0x10
+#define CM_START	0x01
+#define CM_END		0x02
+#define CM_SKIP		0x04
+#define CM_UPGRADE146	0x08
+#define CM_EXCLUDE	0x10
 #define CM_START_SKIP (CM_START | CM_SKIP)
 
 struct cfg_marker {
-	__u32	     cm_step;       /* aka config version */
-	__u32	     cm_flags;
-	__u32	     cm_vers;       /* lustre release version number */
-	__u32	     cm_padding;    /* 64 bit align */
-	__s64	     cm_createtime; /*when this record was first created */
-	__s64	     cm_canceltime; /*when this record is no longer valid*/
-	char	      cm_tgtname[MTI_NAME_MAXLEN];
-	char	      cm_comment[MTI_NAME_MAXLEN];
+	__u32		cm_step;       /* aka config version */
+	__u32		cm_flags;
+	__u32		cm_vers;       /* lustre release version number */
+	__u32		cm_padding;    /* 64 bit align */
+	__s64		cm_createtime; /*when this record was first created */
+	__s64		cm_canceltime; /*when this record is no longer valid*/
+	char		cm_tgtname[MTI_NAME_MAXLEN];
+	char		cm_comment[MTI_NAME_MAXLEN];
 };
 
 /*
@@ -2183,16 +2184,16 @@ enum llog_ctxt_id {
 /** Identifier for a single log object */
 struct llog_logid {
 	struct ost_id		lgl_oi;
-	__u32		   lgl_ogen;
+	__u32			lgl_ogen;
 } __packed;
 
 /** Records written to the CATALOGS list */
 #define CATLIST "CATALOGS"
 struct llog_catid {
-	struct llog_logid       lci_logid;
-	__u32		   lci_padding1;
-	__u32		   lci_padding2;
-	__u32		   lci_padding3;
+	struct llog_logid	lci_logid;
+	__u32			lci_padding1;
+	__u32			lci_padding2;
+	__u32			lci_padding3;
 } __packed;
 
 /* Log data record types - there is no specific reason that these need to
@@ -2274,7 +2275,7 @@ struct llog_unlink64_rec {
 	__u32			lur_padding1;
 	__u64			lur_padding2;
 	__u64			lur_padding3;
-	struct llog_rec_tail    lur_tail;
+	struct llog_rec_tail	lur_tail;
 } __packed;
 
 struct llog_setattr64_rec {
@@ -2285,7 +2286,7 @@ struct llog_setattr64_rec {
 	__u32			lsr_gid;
 	__u32			lsr_gid_h;
 	__u64			lsr_valid;
-	struct llog_rec_tail    lsr_tail;
+	struct llog_rec_tail	lsr_tail;
 } __packed;
 
 struct llog_size_change_rec {
@@ -2314,11 +2315,11 @@ struct llog_changelog_rec {
 } __packed;
 
 struct llog_changelog_user_rec {
-	struct llog_rec_hdr   cur_hdr;
-	__u32		 cur_id;
-	__u32		 cur_padding;
-	__u64		 cur_endrec;
-	struct llog_rec_tail  cur_tail;
+	struct llog_rec_hdr	cur_hdr;
+	__u32			cur_id;
+	__u32			cur_padding;
+	__u64			cur_endrec;
+	struct llog_rec_tail	cur_tail;
 } __packed;
 
 enum agent_req_status {
@@ -2382,7 +2383,7 @@ enum llog_flag {
 	LLOG_F_ZAP_WHEN_EMPTY	= 0x1,
 	LLOG_F_IS_CAT		= 0x2,
 	LLOG_F_IS_PLAIN		= 0x4,
-	LLOG_F_EXT_JOBID        = 0x8,
+	LLOG_F_EXT_JOBID	= 0x8,
 	LLOG_F_IS_FIXSIZE	= 0x10,
 
 	/*
@@ -2404,16 +2405,16 @@ enum llog_flag {
 
 /* flags for the logs */
 struct llog_log_hdr {
-	struct llog_rec_hdr     llh_hdr;
-	__s64		   llh_timestamp;
-	__u32		   llh_count;
-	__u32		   llh_bitmap_offset;
-	__u32		   llh_size;
-	__u32		   llh_flags;
-	__u32		   llh_cat_idx;
+	struct llog_rec_hdr	llh_hdr;
+	__s64			llh_timestamp;
+	__u32			llh_count;
+	__u32			llh_bitmap_offset;
+	__u32			llh_size;
+	__u32			llh_flags;
+	__u32			llh_cat_idx;
 	/* for a catalog the first plain slot is next to it */
-	struct obd_uuid	 llh_tgtuuid;
-	__u32		   llh_reserved[LLOG_HEADER_SIZE / sizeof(__u32) - 23];
+	struct obd_uuid		llh_tgtuuid;
+	__u32			llh_reserved[LLOG_HEADER_SIZE / sizeof(__u32) - 23];
 	/* These fields must always be at the end of the llog_log_hdr.
 	 * Note: llh_bitmap size is variable because llog chunk size could be
 	 * bigger than LLOG_MIN_CHUNK_SIZE, i.e. sizeof(llog_log_hdr) > 8192
@@ -2423,8 +2424,8 @@ struct llog_log_hdr {
 	 * (see llog_client.c), it will be kept in its original way to avoid
 	 * compatibility issue.
 	 */
-	__u32		   llh_bitmap[LLOG_BITMAP_BYTES / sizeof(__u32)];
-	struct llog_rec_tail    llh_tail;
+	__u32			llh_bitmap[LLOG_BITMAP_BYTES / sizeof(__u32)];
+	struct llog_rec_tail	llh_tail;
 } __packed;
 
 #undef LLOG_HEADER_SIZE
@@ -2443,25 +2444,25 @@ struct llog_log_hdr {
  * therein
  */
 struct llog_cookie {
-	struct llog_logid       lgc_lgl;
-	__u32		   lgc_subsys;
-	__u32		   lgc_index;
-	__u32		   lgc_padding;
+	struct llog_logid	lgc_lgl;
+	__u32			lgc_subsys;
+	__u32			lgc_index;
+	__u32			lgc_padding;
 } __packed;
 
 /** llog protocol */
 enum llogd_rpc_ops {
-	LLOG_ORIGIN_HANDLE_CREATE       = 501,
-	LLOG_ORIGIN_HANDLE_NEXT_BLOCK   = 502,
-	LLOG_ORIGIN_HANDLE_READ_HEADER  = 503,
-	LLOG_ORIGIN_HANDLE_WRITE_REC    = 504,
+	LLOG_ORIGIN_HANDLE_CREATE	= 501,
+	LLOG_ORIGIN_HANDLE_NEXT_BLOCK	= 502,
+	LLOG_ORIGIN_HANDLE_READ_HEADER	= 503,
+	LLOG_ORIGIN_HANDLE_WRITE_REC	= 504,
 	LLOG_ORIGIN_HANDLE_CLOSE	= 505,
 	LLOG_ORIGIN_CONNECT		= 506,
 	LLOG_CATINFO			= 507,  /* deprecated */
-	LLOG_ORIGIN_HANDLE_PREV_BLOCK   = 508,
-	LLOG_ORIGIN_HANDLE_DESTROY      = 509,  /* for destroy llog object*/
+	LLOG_ORIGIN_HANDLE_PREV_BLOCK	= 508,
+	LLOG_ORIGIN_HANDLE_DESTROY	= 509,  /* for destroy llog object*/
 	LLOG_LAST_OPC,
-	LLOG_FIRST_OPC		  = LLOG_ORIGIN_HANDLE_CREATE
+	LLOG_FIRST_OPC			= LLOG_ORIGIN_HANDLE_CREATE
 };
 
 struct llogd_body {
@@ -2475,9 +2476,9 @@ struct llogd_body {
 } __packed;
 
 struct llogd_conn_body {
-	struct llog_gen	 lgdc_gen;
-	struct llog_logid       lgdc_logid;
-	__u32		   lgdc_ctxt_idx;
+	struct llog_gen		lgdc_gen;
+	struct llog_logid	lgdc_logid;
+	__u32			lgdc_ctxt_idx;
 } __packed;
 
 /* Note: 64-bit types are 64-bit aligned in structure */
@@ -2485,16 +2486,16 @@ struct obdo {
 	__u64		o_valid;	/* hot fields in this obdo */
 	struct ost_id	o_oi;
 	__u64		o_parent_seq;
-	__u64		o_size;	 /* o_size-o_blocks == ost_lvb */
+	__u64		o_size;		/* o_size-o_blocks == ost_lvb */
 	__s64		o_mtime;
 	__s64		o_atime;
 	__s64		o_ctime;
-	__u64		o_blocks;       /* brw: cli sent cached bytes */
+	__u64		o_blocks;	/* brw: cli sent cached bytes */
 	__u64		o_grant;
 
 	/* 32-bit fields start here: keep an even number of them via padding */
-	__u32		o_blksize;      /* optimal IO blocksize */
-	__u32		o_mode;	 /* brw: cli sent cache remain */
+	__u32		o_blksize;	/* optimal IO blocksize */
+	__u32		o_mode;		/* brw: cli sent cache remain */
 	__u32		o_uid;
 	__u32		o_gid;
 	__u32		o_flags;
@@ -2502,25 +2503,25 @@ struct obdo {
 	__u32		o_parent_oid;
 	__u32		o_misc;		/* brw: o_dropped */
 
-	__u64		   o_ioepoch;      /* epoch in ost writes */
-	__u32		   o_stripe_idx;   /* holds stripe idx */
-	__u32		   o_parent_ver;
+	__u64		o_ioepoch;      /* epoch in ost writes */
+	__u32		o_stripe_idx;   /* holds stripe idx */
+	__u32		o_parent_ver;
 	struct lustre_handle    o_handle;  /* brw: lock handle to prolong locks
 					    */
 	struct llog_cookie      o_lcookie; /* destroy: unlink cookie from MDS,
 					    * obsolete in 2.8, reused in OSP
 					    */
-	__u32			o_uid_h;
-	__u32			o_gid_h;
+	__u32		o_uid_h;
+	__u32		o_gid_h;
 
-	__u64			o_data_version; /* getattr: sum of iversion for
-						 * each stripe.
-						 * brw: grant space consumed on
-						 * the client for the write
-						 */
-	__u64			o_padding_4;
-	__u64			o_padding_5;
-	__u64			o_padding_6;
+	__u64		o_data_version; /* getattr: sum of iversion for
+					 * each stripe.
+					 * brw: grant space consumed on
+					 * the client for the write
+					 */
+	__u64		o_padding_4;
+	__u64		o_padding_5;
+	__u64		o_padding_6;
 };
 
 #define o_dirty   o_blocks
@@ -2543,11 +2544,11 @@ struct ll_fiemap_info_key {
 
 /* security opcodes */
 enum sec_cmd {
-	SEC_CTX_INIT	    = 801,
-	SEC_CTX_INIT_CONT       = 802,
-	SEC_CTX_FINI	    = 803,
+	SEC_CTX_INIT		= 801,
+	SEC_CTX_INIT_CONT	= 802,
+	SEC_CTX_FINI		= 803,
 	SEC_LAST_OPC,
-	SEC_FIRST_OPC	   = SEC_CTX_INIT
+	SEC_FIRST_OPC		= SEC_CTX_INIT
 };
 
 /*
@@ -2560,16 +2561,16 @@ enum sec_cmd {
  * because the offset info is used in find_capa()
  */
 struct lustre_capa {
-	struct lu_fid   lc_fid;	 /** fid */
-	__u64	   lc_opc;	 /** operations allowed */
-	__u64	   lc_uid;	 /** file owner */
-	__u64	   lc_gid;	 /** file group */
-	__u32	   lc_flags;       /** HMAC algorithm & flags */
-	__u32	   lc_keyid;       /** key# used for the capability */
-	__u32	   lc_timeout;     /** capa timeout value (sec) */
+	struct lu_fid	lc_fid;		/** fid */
+	__u64		lc_opc;		/** operations allowed */
+	__u64		lc_uid;		/** file owner */
+	__u64		lc_gid;		/** file group */
+	__u32		lc_flags;	/** HMAC algorithm & flags */
+	__u32		lc_keyid;	/** key# used for the capability */
+	__u32		lc_timeout;	/** capa timeout value (sec) */
 /* FIXME: y2038 time_t overflow: */
-	__u32	   lc_expiry;      /** expiry time (sec) */
-	__u8	    lc_hmac[CAPA_HMAC_MAX_LEN];   /** HMAC */
+	__u32		lc_expiry;	/** expiry time (sec) */
+	__u8		lc_hmac[CAPA_HMAC_MAX_LEN];   /** HMAC */
 } __packed;
 
 /** lustre_capa::lc_opc */
@@ -2619,17 +2620,17 @@ struct link_ea_header {
  */
 struct link_ea_entry {
 	/** __u16 stored big-endian, unaligned */
-	unsigned char      lee_reclen[2];
-	unsigned char      lee_parent_fid[sizeof(struct lu_fid)];
-	char	       lee_name[0];
+	unsigned char	lee_reclen[2];
+	unsigned char	lee_parent_fid[sizeof(struct lu_fid)];
+	char		lee_name[0];
 } __packed;
 
 /** fid2path request/reply structure */
 struct getinfo_fid2path {
-	struct lu_fid   gf_fid;
-	__u64	   gf_recno;
-	__u32	   gf_linkno;
-	__u32	   gf_pathlen;
+	struct lu_fid	gf_fid;
+	__u64		gf_recno;
+	__u32		gf_linkno;
+	__u32		gf_pathlen;
 	union {
 		char		gf_path[0];
 		struct lu_fid	gf_root_fid[0];
@@ -2684,7 +2685,7 @@ struct hsm_progress_kernel {
  * fid1 and fid2 are in mdt_body
  */
 struct mdc_swap_layouts {
-	__u64	   msl_flags;
+	__u64			msl_flags;
 } __packed;
 
 struct close_data {



From neilb at suse.com  Wed Jul  4 04:43:50 2018
From: neilb at suse.com (NeilBrown)
Date: Wed, 04 Jul 2018 14:43:50 +1000
Subject: [lustre-devel] [PATCH 05/11] lustre: improve alignment in
 lustre_disk.h
In-Reply-To: <153067941050.30111.13691201276059352926.stgit@noble>
References: <153067941050.30111.13691201276059352926.stgit@noble>
Message-ID: <153067943073.30111.15126443210273103850.stgit@noble>

Be more consistent with field and comment alignment.
Note that "git diff -b" shows no non-blank changes.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 .../staging/lustre/lustre/include/lustre_disk.h    |   38 ++++++++++----------
 1 file changed, 19 insertions(+), 19 deletions(-)

diff --git a/drivers/staging/lustre/lustre/include/lustre_disk.h b/drivers/staging/lustre/lustre/include/lustre_disk.h
index bd8fa717ff31..1a6d421ea64b 100644
--- a/drivers/staging/lustre/lustre/include/lustre_disk.h
+++ b/drivers/staging/lustre/lustre/include/lustre_disk.h
@@ -70,23 +70,23 @@
 
 /* gleaned from the mount command - no persistent info here */
 struct lustre_mount_data {
-	__u32      lmd_magic;
-	__u32      lmd_flags;	 /* lustre mount flags */
+	__u32	lmd_magic;
+	__u32	lmd_flags;	/* lustre mount flags */
 	int	lmd_mgs_failnodes; /* mgs failover node count */
 	int	lmd_exclude_count;
 	int	lmd_recovery_time_soft;
 	int	lmd_recovery_time_hard;
-	char      *lmd_dev;	   /* device name */
-	char      *lmd_profile;    /* client only */
+	char	*lmd_dev;	/* device name */
+	char	*lmd_profile;	/* client only */
 	char	*lmd_fileset;	/* mount fileset */
-	char      *lmd_mgssec;	/* sptlrpc flavor to mgs */
-	char      *lmd_opts;	/* lustre mount options (as opposed to
+	char	*lmd_mgssec;	/* sptlrpc flavor to mgs */
+	char	*lmd_opts;	/* lustre mount options (as opposed to
 				 * _device_ mount options)
 				 */
-	char      *lmd_params;	/* lustre params */
-	__u32     *lmd_exclude; /* array of OSTs to ignore */
+	char	*lmd_params;	/* lustre params */
+	__u32	*lmd_exclude;	/* array of OSTs to ignore */
 	char	*lmd_mgs;	/* MGS nid */
-	char	*lmd_osd_type;  /* OSD type */
+	char	*lmd_osd_type;	/* OSD type */
 };
 
 #define LMD_FLG_SERVER		0x0001	/* Mounting a server */
@@ -115,21 +115,21 @@ struct lustre_mount_data {
 struct ll_sb_info;
 
 struct lustre_sb_info {
-	int		       lsi_flags;
+	int			lsi_flags;
 	struct obd_device	*lsi_mgc;     /* mgc obd */
 	struct lustre_mount_data *lsi_lmd;     /* mount command info */
 	struct ll_sb_info	*lsi_llsbi;   /* add'l client sbi info */
-	struct dt_device	 *lsi_dt_dev;  /* dt device to access disk fs*/
-	atomic_t	      lsi_mounts;  /* references to the srv_mnt */
-	char			  lsi_svname[MTI_NAME_MAXLEN];
-	char			  lsi_osd_obdname[64];
-	char			  lsi_osd_uuid[64];
-	struct obd_export	 *lsi_osd_exp;
-	char			  lsi_osd_type[16];
-	char			  lsi_fstype[16];
+	struct dt_device	*lsi_dt_dev;  /* dt device to access disk fs*/
+	atomic_t		lsi_mounts;  /* references to the srv_mnt */
+	char			lsi_svname[MTI_NAME_MAXLEN];
+	char			lsi_osd_obdname[64];
+	char			lsi_osd_uuid[64];
+	struct obd_export	*lsi_osd_exp;
+	char			lsi_osd_type[16];
+	char			lsi_fstype[16];
 };
 
-#define LSI_UMOUNT_FAILOVER	      0x00200000
+#define LSI_UMOUNT_FAILOVER	0x00200000
 
 #define     s2lsi(sb)	((struct lustre_sb_info *)((sb)->s_fs_info))
 #define     s2lsi_nocast(sb) ((sb)->s_fs_info)



From neilb at suse.com  Wed Jul  4 04:43:50 2018
From: neilb at suse.com (NeilBrown)
Date: Wed, 04 Jul 2018 14:43:50 +1000
Subject: [lustre-devel] [PATCH 06/11] lustre: mdc: change memcpy to strcpy.
In-Reply-To: <153067941050.30111.13691201276059352926.stgit@noble>
References: <153067941050.30111.13691201276059352926.stgit@noble>
Message-ID: <153067943076.30111.10766325427125150068.stgit@noble>

We may not actually *need* to set the trailing nul here,
but space as allocate for it (req_capsule_set_size above)
so it is tidiest to just use strcpy.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 drivers/staging/lustre/lustre/mdc/mdc_request.c |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/staging/lustre/lustre/mdc/mdc_request.c b/drivers/staging/lustre/lustre/mdc/mdc_request.c
index 893a370dc763..bfa07d72d875 100644
--- a/drivers/staging/lustre/lustre/mdc/mdc_request.c
+++ b/drivers/staging/lustre/lustre/mdc/mdc_request.c
@@ -122,7 +122,7 @@ static int mdc_get_root(struct obd_export *exp, const char *fileset,
 	if (fileset) {
 		char *name = req_capsule_client_get(&req->rq_pill, &RMF_NAME);
 
-		memcpy(name, fileset, strlen(fileset));
+		strcpy(name, fileset);
 	}
 	lustre_msg_add_flags(req->rq_reqmsg, LUSTRE_IMP_FULL);
 	req->rq_send_state = LUSTRE_IMP_FULL;



From neilb at suse.com  Wed Jul  4 04:43:50 2018
From: neilb at suse.com (NeilBrown)
Date: Wed, 04 Jul 2018 14:43:50 +1000
Subject: [lustre-devel] [PATCH 07/11] lustre: simplify some parsing in
 lmd_parse().
In-Reply-To: <153067941050.30111.13691201276059352926.stgit@noble>
References: <153067941050.30111.13691201276059352926.stgit@noble>
Message-ID: <153067943079.30111.1400570516954800906.stgit@noble>

1/ use kstrdup() or kstrndup() where appropriate.
2/ use kasprintf() where appropriate.
3/ use GFP_KERNEL because GFP_NOFS is not needed here.
4/ use strchrnul() instead of open-coding it.
5/ don't kfree() on error, that is done in lustre_free_lsi().

Signed-off-by: NeilBrown <neilb at suse.com>
---
 drivers/staging/lustre/lustre/obdclass/obd_mount.c |   24 +++++++-------------
 1 file changed, 8 insertions(+), 16 deletions(-)

diff --git a/drivers/staging/lustre/lustre/obdclass/obd_mount.c b/drivers/staging/lustre/lustre/obdclass/obd_mount.c
index 708c5802ddc5..1a668874a165 100644
--- a/drivers/staging/lustre/lustre/obdclass/obd_mount.c
+++ b/drivers/staging/lustre/lustre/obdclass/obd_mount.c
@@ -1170,37 +1170,30 @@ static int lmd_parse(char *options, struct lustre_mount_data *lmd)
 		/* Remove leading /s from fsname */
 		while (*++s1 == '/')
 			;
-		s2 = s1;
-		while (*s2 != '/' && *s2 != '\0')
-			s2++;
+		s2 = strchrnul(s1, '/');
 		/* Freed in lustre_free_lsi */
-		lmd->lmd_profile = kzalloc(s2 - s1 + 8, GFP_NOFS);
+		lmd->lmd_profile = kasprintf(GFP_KERNEL, "%.*s-client",
+					     (int)(s2 - s1), s1);
 		if (!lmd->lmd_profile)
 			return -ENOMEM;
 
-		strncat(lmd->lmd_profile, s1, s2 - s1);
-		strncat(lmd->lmd_profile, "-client", 7);
-
 		s1 = s2;
 		s2 = s1 + strlen(s1) - 1;
 		/* Remove padding /s from fileset */
 		while (*s2 == '/')
 			s2--;
 		if (s2 > s1) {
-			lmd->lmd_fileset = kzalloc(s2 - s1 + 2, GFP_NOFS);
-			if (!lmd->lmd_fileset) {
-				kfree(lmd->lmd_profile);
+			lmd->lmd_fileset = kstrndup(s1, s2 - s1 + 1,
+						    GFP_KERNEL);
+			if (!lmd->lmd_fileset)
 				return -ENOMEM;
-			}
-			strncat(lmd->lmd_fileset, s1, s2 - s1 + 1);
 		}
 	}
 
 	/* Freed in lustre_free_lsi */
-	lmd->lmd_dev = kzalloc(strlen(devname) + 1, GFP_NOFS);
+	lmd->lmd_dev = kstrdup(devname, GFP_KERNEL);
 	if (!lmd->lmd_dev)
 		return -ENOMEM;
-	strcpy(lmd->lmd_dev, devname);
 
 	/* Save mount options */
 	s1 = options + strlen(options) - 1;
@@ -1208,10 +1201,9 @@ static int lmd_parse(char *options, struct lustre_mount_data *lmd)
 		*s1-- = 0;
 	if (*options != 0) {
 		/* Freed in lustre_free_lsi */
-		lmd->lmd_opts = kzalloc(strlen(options) + 1, GFP_NOFS);
+		lmd->lmd_opts = kstrdup(options, GFP_KERNEL);
 		if (!lmd->lmd_opts)
 			return -ENOMEM;
-		strcpy(lmd->lmd_opts, options);
 	}
 
 	lmd_print(lmd);



From neilb at suse.com  Wed Jul  4 04:43:50 2018
From: neilb at suse.com (NeilBrown)
Date: Wed, 04 Jul 2018 14:43:50 +1000
Subject: [lustre-devel] [PATCH 08/11] lustre: protect updates to ll_flags.
In-Reply-To: <153067941050.30111.13691201276059352926.stgit@noble>
References: <153067941050.30111.13691201276059352926.stgit@noble>
Message-ID: <153067943083.30111.5879041126521058346.stgit@noble>

While individual flags in sbi->ll_flags are rarely
updates after the superblocks in initialised, it
is possible for these updates to race.
This could cause an update to be lost.

So protect all updates with a spinlock.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 drivers/staging/lustre/lustre/llite/llite_lib.c   |    4 ++++
 drivers/staging/lustre/lustre/llite/lproc_llite.c |    6 ++++++
 drivers/staging/lustre/lustre/llite/xattr.c       |    4 ++++
 3 files changed, 14 insertions(+)

diff --git a/drivers/staging/lustre/lustre/llite/llite_lib.c b/drivers/staging/lustre/lustre/llite/llite_lib.c
index 7a414e2238a3..64cd69cb493d 100644
--- a/drivers/staging/lustre/lustre/llite/llite_lib.c
+++ b/drivers/staging/lustre/lustre/llite/llite_lib.c
@@ -2549,7 +2549,9 @@ void ll_compute_rootsquash_state(struct ll_sb_info *sbi)
 	/* Update norootsquash flag */
 	down_write(&squash->rsi_sem);
 	if (list_empty(&squash->rsi_nosquash_nids)) {
+		spin_lock(&sbi->ll_lock);
 		sbi->ll_flags &= ~LL_SBI_NOROOTSQUASH;
+		spin_unlock(&sbi->ll_lock);
 	} else {
 		/*
 		 * Do not apply root squash as soon as one of our NIDs is
@@ -2566,10 +2568,12 @@ void ll_compute_rootsquash_state(struct ll_sb_info *sbi)
 				break;
 			}
 		}
+		spin_lock(&sbi->ll_lock);
 		if (matched)
 			sbi->ll_flags |= LL_SBI_NOROOTSQUASH;
 		else
 			sbi->ll_flags &= ~LL_SBI_NOROOTSQUASH;
+		spin_unlock(&sbi->ll_lock);
 	}
 	up_write(&squash->rsi_sem);
 }
diff --git a/drivers/staging/lustre/lustre/llite/lproc_llite.c b/drivers/staging/lustre/lustre/llite/lproc_llite.c
index 9dcbe648777e..617aabb1b59a 100644
--- a/drivers/staging/lustre/lustre/llite/lproc_llite.c
+++ b/drivers/staging/lustre/lustre/llite/lproc_llite.c
@@ -514,10 +514,12 @@ static ssize_t checksum_pages_store(struct kobject *kobj,
 	rc = kstrtoul(buffer, 10, &val);
 	if (rc)
 		return rc;
+	spin_lock(&sbi->ll_lock);
 	if (val)
 		sbi->ll_flags |= LL_SBI_CHECKSUM;
 	else
 		sbi->ll_flags &= ~LL_SBI_CHECKSUM;
+	spin_unlock(&sbi->ll_lock);
 
 	rc = obd_set_info_async(NULL, sbi->ll_dt_exp, sizeof(KEY_CHECKSUM),
 				KEY_CHECKSUM, sizeof(val), &val, NULL);
@@ -669,10 +671,12 @@ static ssize_t statahead_agl_store(struct kobject *kobj,
 	if (rc)
 		return rc;
 
+	spin_lock(&sbi->ll_lock);
 	if (val)
 		sbi->ll_flags |= LL_SBI_AGL_ENABLED;
 	else
 		sbi->ll_flags &= ~LL_SBI_AGL_ENABLED;
+	spin_unlock(&sbi->ll_lock);
 
 	return count;
 }
@@ -719,10 +723,12 @@ static ssize_t lazystatfs_store(struct kobject *kobj,
 	if (rc)
 		return rc;
 
+	spin_lock(&sbi->ll_lock);
 	if (val)
 		sbi->ll_flags |= LL_SBI_LAZYSTATFS;
 	else
 		sbi->ll_flags &= ~LL_SBI_LAZYSTATFS;
+	spin_unlock(&sbi->ll_lock);
 
 	return count;
 }
diff --git a/drivers/staging/lustre/lustre/llite/xattr.c b/drivers/staging/lustre/lustre/llite/xattr.c
index 7fa0a419c094..5e27c85f104f 100644
--- a/drivers/staging/lustre/lustre/llite/xattr.c
+++ b/drivers/staging/lustre/lustre/llite/xattr.c
@@ -150,7 +150,9 @@ static int ll_xattr_set_common(const struct xattr_handler *handler,
 	if (rc) {
 		if (rc == -EOPNOTSUPP && handler->flags == XATTR_USER_T) {
 			LCONSOLE_INFO("Disabling user_xattr feature because it is not supported on the server\n");
+			spin_lock(&sbi->ll_lock);
 			sbi->ll_flags &= ~LL_SBI_USER_XATTR;
+			spin_unlock(&sbi->ll_lock);
 		}
 		return rc;
 	}
@@ -387,7 +389,9 @@ int ll_xattr_list(struct inode *inode, const char *name, int type, void *buffer,
 		LCONSOLE_INFO(
 			"%s: disabling user_xattr feature because it is not supported on the server: rc = %d\n",
 			ll_get_fsname(inode->i_sb, NULL, 0), rc);
+		spin_lock(&sbi->ll_lock);
 		sbi->ll_flags &= ~LL_SBI_USER_XATTR;
+		spin_unlock(&sbi->ll_lock);
 	}
 out:
 	ptlrpc_req_finished(req);



From neilb at suse.com  Wed Jul  4 04:43:50 2018
From: neilb at suse.com (NeilBrown)
Date: Wed, 04 Jul 2018 14:43:50 +1000
Subject: [lustre-devel] [PATCH 09/11] lustre: remove dead code from
 obd_connect_flags2str()
In-Reply-To: <153067941050.30111.13691201276059352926.stgit@noble>
References: <153067941050.30111.13691201276059352926.stgit@noble>
Message-ID: <153067943086.30111.2323331820343229570.stgit@noble>

In obd_connect_flags2str(), if obd_connect_names() has
fewer than 64 non-NULL entries, then the first for-loop
is dangerous.
If it has 64 or more, the first "unknown" error is pointless.

In fact, there are more than 64 entries, so provide a BUILD_BUG_ON
to that effect, and remove the pointless code.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 .../lustre/lustre/obdclass/lprocfs_status.c        |    7 ++-----
 1 file changed, 2 insertions(+), 5 deletions(-)

diff --git a/drivers/staging/lustre/lustre/obdclass/lprocfs_status.c b/drivers/staging/lustre/lustre/obdclass/lprocfs_status.c
index 9f76d8a0f5f1..377644737fb9 100644
--- a/drivers/staging/lustre/lustre/obdclass/lprocfs_status.c
+++ b/drivers/staging/lustre/lustre/obdclass/lprocfs_status.c
@@ -121,17 +121,14 @@ int obd_connect_flags2str(char *page, int count, u64 flags, u64 flags2,
 	__u64 mask;
 	int i, ret = 0;
 
+	BUILD_BUG_ON(ARRAY_SIZE(obd_connect_names) < 65);
+
 	for (i = 0, mask = 1; i < 64; i++, mask <<= 1) {
 		if (flags & mask)
 			ret += snprintf(page + ret, count - ret, "%s%s",
 					ret ? sep : "", obd_connect_names[i]);
 	}
 
-	if (flags & ~(mask - 1))
-		ret += snprintf(page + ret, count - ret,
-				"%sunknown flags %#llx",
-				ret ? sep : "", flags & ~(mask - 1));
-
 	if (!(flags & OBD_CONNECT_FLAGS2) || flags2 == 0)
 		return ret;
 



From neilb at suse.com  Wed Jul  4 04:43:50 2018
From: neilb at suse.com (NeilBrown)
Date: Wed, 04 Jul 2018 14:43:50 +1000
Subject: [lustre-devel] [PATCH 10/11] lustre: iput() accepts NULL.
In-Reply-To: <153067941050.30111.13691201276059352926.stgit@noble>
References: <153067941050.30111.13691201276059352926.stgit@noble>
Message-ID: <153067943090.30111.16321038675119336031.stgit@noble>

iput(NULL) does nothing, so any code like

  if (x)
      iput(x);

can become

  iput(x);

Signed-off-by: NeilBrown <neilb at suse.com>
---
 drivers/staging/lustre/lustre/llite/dir.c       |    3 +--
 drivers/staging/lustre/lustre/llite/statahead.c |    3 +--
 drivers/staging/lustre/lustre/lmv/lmv_obd.c     |    2 +-
 3 files changed, 3 insertions(+), 5 deletions(-)

diff --git a/drivers/staging/lustre/lustre/llite/dir.c b/drivers/staging/lustre/lustre/llite/dir.c
index 987f4b252c40..f352fab51726 100644
--- a/drivers/staging/lustre/lustre/llite/dir.c
+++ b/drivers/staging/lustre/lustre/llite/dir.c
@@ -480,8 +480,7 @@ static int ll_dir_setdirstripe(struct dentry *dparent, struct lmv_user_md *lump,
 		err = ll_inode_init_security(&dentry, inode, parent);
 
 out_inode:
-	if (inode)
-		iput(inode);
+	iput(inode);
 out_request:
 	ptlrpc_req_finished(request);
 out_op_data:
diff --git a/drivers/staging/lustre/lustre/llite/statahead.c b/drivers/staging/lustre/lustre/llite/statahead.c
index d864f5f36d85..ea2a00230eaf 100644
--- a/drivers/staging/lustre/lustre/llite/statahead.c
+++ b/drivers/staging/lustre/lustre/llite/statahead.c
@@ -259,8 +259,7 @@ sa_kill(struct ll_statahead_info *sai, struct sa_entry *entry)
 	list_del_init(&entry->se_list);
 	spin_unlock(&lli->lli_sa_lock);
 
-	if (entry->se_inode)
-		iput(entry->se_inode);
+	iput(entry->se_inode);
 
 	sa_free(sai, entry);
 }
diff --git a/drivers/staging/lustre/lustre/lmv/lmv_obd.c b/drivers/staging/lustre/lustre/lmv/lmv_obd.c
index 32aa664385fe..3da5a0a6db4d 100644
--- a/drivers/staging/lustre/lustre/lmv/lmv_obd.c
+++ b/drivers/staging/lustre/lustre/lmv/lmv_obd.c
@@ -2702,7 +2702,7 @@ static int lmv_unpackmd(struct obd_export *exp, struct lmv_stripe_md **lsmp,
 			 * ll_update_lsm_md
 			 */
 			if (!(lsm->lsm_md_hash_type & LMV_HASH_FLAG_MIGRATION &&
-			      !i) && lsm->lsm_md_oinfo[i].lmo_root)
+			      !i))
 				iput(lsm->lsm_md_oinfo[i].lmo_root);
 		}
 



From neilb at suse.com  Wed Jul  4 04:43:50 2018
From: neilb at suse.com (NeilBrown)
Date: Wed, 04 Jul 2018 14:43:50 +1000
Subject: [lustre-devel] [PATCH 11/11] lustre: avoid alloc(sizeof(struct ...))
In-Reply-To: <153067941050.30111.13691201276059352926.stgit@noble>
References: <153067941050.30111.13691201276059352926.stgit@noble>
Message-ID: <153067943093.30111.13601987501540697866.stgit@noble>

checkpatch suggests that

 struct foo *bar;

 bar = kmalloc(sizeof(struct foo), GFP_XX);

is better written as

 bar = kmalloc(sizeof(*bar), GFP_XX);

and this does seem more robust.  So make this
change throughput lustre.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 drivers/staging/lustre/lnet/lnet/nidstrings.c    |    4 ++--
 drivers/staging/lustre/lnet/selftest/console.c   |    8 ++++----
 drivers/staging/lustre/lnet/selftest/framework.c |    8 ++++----
 drivers/staging/lustre/lustre/llite/file.c       |    2 +-
 drivers/staging/lustre/lustre/ptlrpc/client.c    |    2 +-
 5 files changed, 12 insertions(+), 12 deletions(-)

diff --git a/drivers/staging/lustre/lnet/lnet/nidstrings.c b/drivers/staging/lustre/lnet/lnet/nidstrings.c
index 0f6c3fa16c65..43d957f9022a 100644
--- a/drivers/staging/lustre/lnet/lnet/nidstrings.c
+++ b/drivers/staging/lustre/lnet/lnet/nidstrings.c
@@ -169,7 +169,7 @@ parse_addrange(const struct cfs_lstr *src, struct nidrange *nidrange)
 		return 0;
 	}
 
-	addrrange = kzalloc(sizeof(struct addrrange), GFP_NOFS);
+	addrrange = kzalloc(sizeof(*addrrange), GFP_NOFS);
 	if (!addrrange)
 		return -ENOMEM;
 	list_add_tail(&addrrange->ar_link, &nidrange->nr_addrranges);
@@ -228,7 +228,7 @@ add_nidrange(const struct cfs_lstr *src,
 		return nr;
 	}
 
-	nr = kzalloc(sizeof(struct nidrange), GFP_NOFS);
+	nr = kzalloc(sizeof(*nr), GFP_NOFS);
 	if (!nr)
 		return NULL;
 	list_add_tail(&nr->nr_link, nidlist);
diff --git a/drivers/staging/lustre/lnet/selftest/console.c b/drivers/staging/lustre/lnet/selftest/console.c
index 3c1c1b5997e0..d5806f22ee94 100644
--- a/drivers/staging/lustre/lnet/selftest/console.c
+++ b/drivers/staging/lustre/lnet/selftest/console.c
@@ -165,7 +165,7 @@ lstcon_ndlink_find(struct list_head *hash, struct lnet_process_id id,
 	if (rc)
 		return rc;
 
-	ndl = kzalloc(sizeof(struct lstcon_ndlink), GFP_NOFS);
+	ndl = kzalloc(sizeof(*ndl), GFP_NOFS);
 	if (!ndl) {
 		lstcon_node_put(nd);
 		return -ENOMEM;
@@ -805,7 +805,7 @@ lstcon_group_info(char *name, struct lstcon_ndlist_ent __user *gents_p,
 	}
 
 	/* non-verbose query */
-	gentp = kzalloc(sizeof(struct lstcon_ndlist_ent), GFP_NOFS);
+	gentp = kzalloc(sizeof(*gentp), GFP_NOFS);
 	if (!gentp) {
 		CERROR("Can't allocate ndlist_ent\n");
 		lstcon_group_decref(grp);
@@ -854,7 +854,7 @@ lstcon_batch_add(char *name)
 		return rc;
 	}
 
-	bat = kzalloc(sizeof(struct lstcon_batch), GFP_NOFS);
+	bat = kzalloc(sizeof(*bat), GFP_NOFS);
 	if (!bat) {
 		CERROR("Can't allocate descriptor for batch %s\n", name);
 		return -ENOMEM;
@@ -969,7 +969,7 @@ lstcon_batch_info(char *name, struct lstcon_test_batch_ent __user *ent_up,
 	}
 
 	/* non-verbose query */
-	entp = kzalloc(sizeof(struct lstcon_test_batch_ent), GFP_NOFS);
+	entp = kzalloc(sizeof(*entp), GFP_NOFS);
 	if (!entp)
 		return -ENOMEM;
 
diff --git a/drivers/staging/lustre/lnet/selftest/framework.c b/drivers/staging/lustre/lnet/selftest/framework.c
index 741af10560ad..361571aa94ce 100644
--- a/drivers/staging/lustre/lnet/selftest/framework.c
+++ b/drivers/staging/lustre/lnet/selftest/framework.c
@@ -143,7 +143,7 @@ sfw_register_test(struct srpc_service *service,
 		return -EEXIST;
 	}
 
-	tsc = kzalloc(sizeof(struct sfw_test_case), GFP_NOFS);
+	tsc = kzalloc(sizeof(*tsc), GFP_NOFS);
 	if (!tsc)
 		return -ENOMEM;
 
@@ -344,7 +344,7 @@ sfw_bid2batch(struct lst_bid bid)
 	if (bat)
 		return bat;
 
-	bat = kzalloc(sizeof(struct sfw_batch), GFP_NOFS);
+	bat = kzalloc(sizeof(*bat), GFP_NOFS);
 	if (!bat)
 		return NULL;
 
@@ -447,7 +447,7 @@ sfw_make_session(struct srpc_mksn_reqst *request, struct srpc_mksn_reply *reply)
 	}
 
 	/* brand new or create by force */
-	sn = kzalloc(sizeof(struct sfw_session), GFP_NOFS);
+	sn = kzalloc(sizeof(*sn), GFP_NOFS);
 	if (!sn) {
 		CERROR("dropping RPC mksn under memory pressure\n");
 		return -ENOMEM;
@@ -795,7 +795,7 @@ sfw_add_test_instance(struct sfw_batch *tsb, struct srpc_server_rpc *rpc)
 			sfw_unpack_id(id);
 
 		for (j = 0; j < tsi->tsi_concur; j++) {
-			tsu = kzalloc(sizeof(struct sfw_test_unit), GFP_NOFS);
+			tsu = kzalloc(sizeof(*tsu), GFP_NOFS);
 			if (!tsu) {
 				rc = -ENOMEM;
 				CERROR("Can't allocate tsu for %d\n",
diff --git a/drivers/staging/lustre/lustre/llite/file.c b/drivers/staging/lustre/lustre/llite/file.c
index 54da6f19da21..3f0f379ce050 100644
--- a/drivers/staging/lustre/lustre/llite/file.c
+++ b/drivers/staging/lustre/lustre/llite/file.c
@@ -615,7 +615,7 @@ int ll_file_open(struct inode *inode, struct file *file)
 
 			goto restart;
 		}
-		*och_p = kzalloc(sizeof(struct obd_client_handle), GFP_KERNEL);
+		*och_p = kzalloc(sizeof(**och_p), GFP_KERNEL);
 		if (!*och_p) {
 			rc = -ENOMEM;
 			goto out_och_free;
diff --git a/drivers/staging/lustre/lustre/ptlrpc/client.c b/drivers/staging/lustre/lustre/ptlrpc/client.c
index fee4e498b143..7a3d83c0e50b 100644
--- a/drivers/staging/lustre/lustre/ptlrpc/client.c
+++ b/drivers/staging/lustre/lustre/ptlrpc/client.c
@@ -576,7 +576,7 @@ ptlrpc_init_rq_pool(int num_rq, int msgsize,
 {
 	struct ptlrpc_request_pool *pool;
 
-	pool = kzalloc(sizeof(struct ptlrpc_request_pool), GFP_NOFS);
+	pool = kzalloc(sizeof(*pool), GFP_NOFS);
 	if (!pool)
 		return NULL;
 



From neilb at suse.com  Wed Jul  4 05:22:29 2018
From: neilb at suse.com (NeilBrown)
Date: Wed, 04 Jul 2018 15:22:29 +1000
Subject: [lustre-devel] [PATCH v3 07/26] staging: lustre: libcfs: NUMA
 support
In-Reply-To: <AT5PR8401MB0900C67FADDF0EF7ED1CDBC4854E0@AT5PR8401MB0900.NAMPRD84.PROD.OUTLOOK.COM>
References: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
 <1529875250-11531-8-git-send-email-jsimmons@infradead.org>
 <877emnaed8.fsf@notabene.neil.brown.name>
 <A764DED1-A9A8-4FA5-A0F1-922C3E2D822A@cray.com>
 <87lgb16j8q.fsf@notabene.neil.brown.name>
 <DM3PR1101MB1213FD1001256E565FF011FFCB480@DM3PR1101MB1213.namprd11.prod.outlook.com>
 <8736x77lrk.fsf@notabene.neil.brown.name>
 <9DE389AB-C7E1-4336-B7E8-604581EFD53E@cray.com>
 <CAAqp6i0uwmpdFYUJ7mCBz+MH+gVBa6S=P2t5Yn7wcFOtrTD-Fg@mail.gmail.com>
 <AT5PR8401MB0900C67FADDF0EF7ED1CDBC4854E0@AT5PR8401MB0900.NAMPRD84.PROD.OUTLOOK.COM>
Message-ID: <87va9vziay.fsf@notabene.neil.brown.name>


Thanks everyone for your patience in explaining things to me.
I'm beginning to understand what to look for and where to find it.

So the answers to Greg's questions:

  Where are you reading the host memory NUMA information from?

  And why would a filesystem care about this type of thing?  Are you
  going to now mirror what the scheduler does with regards to NUMA
  topology issues?  How are you going to handle things when the topology
  changes?  What systems did you test this on?  What performance
  improvements were seen?  What downsides are there with all of this?


Are:
  - NUMA info comes from ACPI or device-tree just like for every one
      else.  Lustre just uses node_distance().

  - The filesystem cares about this because...  It has service
    thread that does part of the work of some filesystem operations
    (handling replies for example) and these are best handled "near"
    the CPU the initiated the request.  Lustre partitions
    all CPUs into "partitions" (cpt) each with a few cores.
    If the request thread and the reply thread are on different
    CPUs but in the same partition, then we get best throughput
    (is that close?)

  - Not really mirroring the scheduler, maybe mirroring parts of the
    network layer(?)

  - We don't handle topology changes yet except in very minimal ways
    (cpts *can* become empty, and that can cause problems).

  - This has been tested on .... great big things.
  - When multi-rails configurations are used (like ethernet-bonding,
    but for RDMA), we get ??? closer to theoretical bandwidth.
    Without these changes it scales poorly (??)

  - The down-sides primarily are that we don't auto-configure
    perfectly.  This particularly affects hot-plug, but without
    hotplug the grouping of cpus and interfaces are focussed
    on .... avoiding worst case rather than achieving best case.
    

I've made up a lot of stuff there.  I'm happy not to pursue this further
at the moment, but if anyone would like to enhance my understanding by
correcting the worst errors in the above, I wouldn't object :-)

Thanks,
NeilBrown




On Fri, Jun 29 2018, Weber, Olaf (HPC Data Management & Storage) wrote:

> To add to Amir's point,  Lustre's CPTs are a way to partition a machine. The distance mechanism I added is one way to map the ACPI-reported distances on the Lustre CPT mapping. It tends to assume the worst case applies to the wholes. It is there because the rest of the Lustre code (at least in the tree I had to work on) "thinks" in CPTs.
>
> Other CPT-related stuff that came in with the multi-rail code has the same rationale. If I'd been working against the kernel interfaces themselves it would have looked differently, but that was not an option at the time.
>
> We've found it to be useful, so replacing it would be better than just ripping it out.
>
> That's all there is to it.
>
> Olaf
>
> ---
> From: Amir Shehata [mailto:amir.shehata.whamcloud at gmail.com] 
> Sent: Friday, June 29, 2018 19:28
> To: Doug Oucharek <doucharek at cray.com>
> Cc: NeilBrown <neilb at suse.com>; Weber, Olaf (HPC Data Management & Storage) <olaf.weber at hpe.com>; Amir Shehata <amir.shehata at intel.com>; Lustre Development List <lustre-devel at lists.lustre.org>
> Subject: Re: [lustre-devel] [PATCH v3 07/26] staging: lustre: libcfs: NUMA support
>
> Olaf can add more details, but I believe we are using the linux distance infrastructure. Take a look at cfs_cpt_distance_calculate(). What we're doing is extracting the NUMA distances provided in the kernel and building an internal representation of distances between CPU partitions (CPTs) since that's what's used in the code.
>
> On 29 June 2018 at 10:19, Doug Oucharek <doucharek at cray.com> wrote:
> I’ll leave Olaf of HPE answer questions about the distance code.  I was only an inspector as it relates to the Multi-Rail feature in the community tree.  
>
> Doug
>
>> On Jun 27, 2018, at 6:17 PM, NeilBrown <neilb at suse.com> wrote:
>> 
>> 
>> I went digging and found that Linux already has a well defined concept
>> of distance between NUMA nodes.
>> On x86 (and amd64?), this is loaded from ACPI.  Other platforms can
>> describe it in devicetree.
>> You can view distance information in
>>  /sys/devices/system/node/node*/distance
>> 
>> or using "numactl --hardware".
>> 
>> Why doesn't lustre simple extract and use this information?  Why does
>> lustre need to allow it to be configured?
>> 
>> Thanks,
>> NeilBrown
>> 
>> On Wed, Jun 27 2018, Patrick Farrell wrote:
>> 
>>> Neil,
>>> 
>>> I am not the person at Cray for this, but if SUSE does take an interest in this, Cray would probably be interested in weighing in and contributing info if not actually code.  In fact, other HPC vendors like HPE(by which I mostly mean the old SGI) or IBM might as well.  NUMA optimization is a persistent fascination in our area of the industry...
>>> 
>>> - Patrick
>>> 
>>> ________________________________
>>> From: lustre-devel <lustre-devel-bounces at lists.lustre.org> on behalf of NeilBrown <neilb at suse.com>
>>> Sent: Tuesday, June 26, 2018 9:44:37 PM
>>> To: Doug Oucharek
>>> Cc: Amir Shehata; Lustre Development List
>>> Subject: Re: [lustre-devel] [PATCH v3 07/26] staging: lustre: libcfs: NUMA support
>>> 
>>> On Mon, Jun 25 2018, Doug Oucharek wrote:
>>> 
>>>> Some background on this NUMA change:
>>>> 
>>>> First off, this is just a first step to a bigger set of changes which include changes to the Lustre utilities.  This was done as part of the Multi-Rail feature.  One of the systems that feature is meant to support is the SGI UV system (now HPE) which has a massive number of NUMA nodes connected by a NUMA Link.  There are multiple fabric cards spread throughout the system and Multi-Rail needs to know which fabric cards are nearest to the NUMA node we are running on.  To do that, the “distance” between NUMA nodes needs to be configured.
>>>> 
>>>> This patch is preparing the infrastructure for the Multi-Rail feature to support configuring NUMA node distances.  Technically, this patch should be landing with the Multi-Rail feature (still to be pushed) for it to make proper sense.
>>>> 
>>> 
>>> Thanks a lot for the background.
>>> 
>>> If these NUMA nodes have a 'distance' between them, and if lustre can
>>> benefit from knowing the distance, then is seems likely that other code
>>> might also benefit.  In that case it would be best if the distance were
>>> encoded in some global state information so that lustre and any other
>>> subsystem can extract it.
>>> 
>>> Do you know if there is any work underway by anyone to make this
>>> information generally available?  If there is, we should make sure that
>>> lustre works in a compatible way so that once that work lands, lustre
>>> can use it directly and not need extra configuration.
>>> If no such work is underway, then it would be really good if something
>>> were done in that direction.  If no-one here is able to work on this, I
>>> can ask around in SUSE and see if anyone here knows anything relevant.
>>> 
>>> Thanks,
>>> NeilBrown
> _______________________________________________
> lustre-devel mailing list
> lustre-devel at lists.lustre.org
> http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180704/c2020123/attachment.sig>

From olaf.weber at hpe.com  Wed Jul  4 08:40:38 2018
From: olaf.weber at hpe.com (Weber, Olaf (HPC Data Management & Storage))
Date: Wed, 4 Jul 2018 08:40:38 +0000
Subject: [lustre-devel] [PATCH v3 07/26] staging: lustre: libcfs: NUMA
 support
In-Reply-To: <87va9vziay.fsf@notabene.neil.brown.name>
References: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
 <1529875250-11531-8-git-send-email-jsimmons@infradead.org>
 <877emnaed8.fsf@notabene.neil.brown.name>
 <A764DED1-A9A8-4FA5-A0F1-922C3E2D822A@cray.com>
 <87lgb16j8q.fsf@notabene.neil.brown.name>
 <DM3PR1101MB1213FD1001256E565FF011FFCB480@DM3PR1101MB1213.namprd11.prod.outlook.com>
 <8736x77lrk.fsf@notabene.neil.brown.name>
 <9DE389AB-C7E1-4336-B7E8-604581EFD53E@cray.com>
 <CAAqp6i0uwmpdFYUJ7mCBz+MH+gVBa6S=P2t5Yn7wcFOtrTD-Fg@mail.gmail.com>
 <AT5PR8401MB0900C67FADDF0EF7ED1CDBC4854E0@AT5PR8401MB0900.NAMPRD84.PROD.OUTLOOK.COM>
 <87va9vziay.fsf@notabene.neil.brown.name>
Message-ID: <AT5PR8401MB0900A5406BF0CB8C87B80EEC85410@AT5PR8401MB0900.NAMPRD84.PROD.OUTLOOK.COM>

NeilBrown [mailto:neilb at suse.com] wrote:

To help contextualize things: the Lustre code can be decomposed into three parts:

1) The filesystem proper: Lustre.
2) The communication protocol it uses: LNet.
3) Supporting code used by Lustre and LNet: CFS.

Part of the supporting code is the CPT mechanism, which provides a way to
partition the CPUs of a system. These partitions are used to distribute queues,
locks, and threads across the system. It was originally introduced years ago, as
far as I can tell mainly to deal with certain hot locks: these were converted into
read/write locks with one spinlock per CPT.

As a general rule, CPT boundaries should respect node and socket boundaries,
but at the higher end, where CPUs have 20+ cores, it may make sense to split
a CPUs cores across several CPTs.

> Thanks everyone for your patience in explaining things to me.
> I'm beginning to understand what to look for and where to find it.
> 
> So the answers to Greg's questions:
> 
>   Where are you reading the host memory NUMA information from?
> 
>   And why would a filesystem care about this type of thing?  Are you
>   going to now mirror what the scheduler does with regards to NUMA
>   topology issues?  How are you going to handle things when the topology
>   changes?  What systems did you test this on?  What performance
>   improvements were seen?  What downsides are there with all of this?
> 
> 
> Are:

>   - NUMA info comes from ACPI or device-tree just like for every one
>       else.  Lustre just uses node_distance().

Correct, the standard kernel interfaces for this information are used to
obtain it, so ultimately Lustre/LNet uses the same source of truth as
everyone else.

>   - The filesystem cares about this because...  It has service
>     thread that does part of the work of some filesystem operations
>     (handling replies for example) and these are best handled "near"
>     the CPU the initiated the request.  Lustre partitions
>     all CPUs into "partitions" (cpt) each with a few cores.
>     If the request thread and the reply thread are on different
>     CPUs but in the same partition, then we get best throughput
>     (is that close?)

At the filesystem level, it does indeed seem to help to have the service
threads that do work for requests run on a different core that is close to
the core that originated the request. So preferably on the same CPU, and
on certain multi-core CPUs there are also distance effects between cores.
That too is one of the things the CPT mechanism handles.

>   - Not really mirroring the scheduler, maybe mirroring parts of the
>     network layer(?)

The LNet code, which is derived from Portals 3.x, is mostly an easier-to-use
abstraction of RDMA interfaces provided by Infiniband and other similar
hardware. It can also use TCP/IP, but that's not the primary use case.

As a communication layer that builds on top of RDMA-capable hardware,
LNet cares about such things as whether the CPU driving communication
is close to the memory used, and also whether it is close to the interface
used. Even in a 2-socket machine, there are measurable performance
differences depending on whether the memory an interface connect
to the same socket or to different sockets. On bigger hardware, like a
32-socket machine, the penalties are much more pronounced. At the
time we found that the QPI links between sockets were a bottleneck
and that performance cratered if they had to handle too much traffic.

UPI, the successor to QPI is better -- has more bandwidth -- but with
the CPUs having more and more cores I expect the scaling issues to
remain similar.

>   - We don't handle topology changes yet except in very minimal ways
>     (cpts *can* become empty, and that can cause problems).

Yes, this is a known deficiency.

>   - This has been tested on .... great big things.

The basic CPT mechanism predates my involvement with Lustre. I did
work on making it more NUMA-aware. A 32-socket system was one of
the primary test beds.

>   - When multi-rails configurations are used (like ethernet-bonding,
>     but for RDMA), we get ??? closer to theoretical bandwidth.
>     Without these changes it scales poorly (??)

The basic idea behind muti-rail configurations is that we use several
Infiniband interfaces and LNet presents them as a single logical interface
to Lustre. For each message, LNet picks the IB interface it should go across
using several criteria, including NUMA distance of the interface and how
busy it is.

With these changes we could get pretty much linear scaling of LNet
throughput by adding more interfaces.

>   - The down-sides primarily are that we don't auto-configure
>     perfectly.  This particularly affects hot-plug, but without
>     hotplug the grouping of cpus and interfaces are focussed
>     on .... avoiding worst case rather than achieving best case.

Without hotplug the CPT grouping should be tuned to achieve a best
case in a static configuration.

Adding simple-minded hotplug tolerance (let's not call it support) would
focus on avoiding truly pathological behaviour.

> I've made up a lot of stuff there.  I'm happy not to pursue this further at the
> moment, but if anyone would like to enhance my understanding by
> correcting the worst errors in the above, I wouldn't object :-)
> 
> Thanks,
> NeilBrown

PS: the NUMA effects I've mentioned above have been making the news
lately under other names: they are part of the side channels used in various
timing based attacks.

Olaf


From neilb at suse.com  Thu Jul  5 01:57:32 2018
From: neilb at suse.com (NeilBrown)
Date: Thu, 05 Jul 2018 11:57:32 +1000
Subject: [lustre-devel] [PATCH v3 07/26] staging: lustre: libcfs: NUMA
 support
In-Reply-To: <AT5PR8401MB0900A5406BF0CB8C87B80EEC85410@AT5PR8401MB0900.NAMPRD84.PROD.OUTLOOK.COM>
References: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
 <1529875250-11531-8-git-send-email-jsimmons@infradead.org>
 <877emnaed8.fsf@notabene.neil.brown.name>
 <A764DED1-A9A8-4FA5-A0F1-922C3E2D822A@cray.com>
 <87lgb16j8q.fsf@notabene.neil.brown.name>
 <DM3PR1101MB1213FD1001256E565FF011FFCB480@DM3PR1101MB1213.namprd11.prod.outlook.com>
 <8736x77lrk.fsf@notabene.neil.brown.name>
 <9DE389AB-C7E1-4336-B7E8-604581EFD53E@cray.com>
 <CAAqp6i0uwmpdFYUJ7mCBz+MH+gVBa6S=P2t5Yn7wcFOtrTD-Fg@mail.gmail.com>
 <AT5PR8401MB0900C67FADDF0EF7ED1CDBC4854E0@AT5PR8401MB0900.NAMPRD84.PROD.OUTLOOK.COM>
 <87va9vziay.fsf@notabene.neil.brown.name>
 <AT5PR8401MB0900A5406BF0CB8C87B80EEC85410@AT5PR8401MB0900.NAMPRD84.PROD.OUTLOOK.COM>
Message-ID: <87k1qazboz.fsf@notabene.neil.brown.name>

On Wed, Jul 04 2018, Weber, Olaf (HPC Data Management & Storage) wrote:

> NeilBrown [mailto:neilb at suse.com] wrote:
>
> To help contextualize things: the Lustre code can be decomposed into three parts:
>
> 1) The filesystem proper: Lustre.
> 2) The communication protocol it uses: LNet.
> 3) Supporting code used by Lustre and LNet: CFS.
>
> Part of the supporting code is the CPT mechanism, which provides a way to
> partition the CPUs of a system. These partitions are used to distribute queues,
> locks, and threads across the system. It was originally introduced years ago, as
> far as I can tell mainly to deal with certain hot locks: these were converted into
> read/write locks with one spinlock per CPT.

Thanks for this context.
Looking in the client code there are 2 per-cpt locks: ln_res_lock and
ln_net_lock.

ln_res_lock protects:
 lnet_res_container -> rec_lh_hash hash chains.

 the_lnet.ln_eq_container.rec_active list of lnet_eq

 lists of memory descriptors (rec_active)

 lists of match entries - one table per cpt.
     Some match entries follow cpu affinity, some are global and hashed
     to choose a table (I think).

  lib-move seems to use the lock to protect the md itself,
  rather than just the list of md.... not sure.

  ptl_mt_maps (??) (rather inefficient ordered-insertion in
  		lnet_ptl_enable_mt())

  proc_lnet_portal_rotor() uses lnet_res_lock(0) to protect
     portal_rotos[]. I wonder why.


ln_net_lock protects:

   ni->ni_refs counter (why not atomic_t I wonder)
   the_lnet.ln_testprotocompat - I guess we don't want that changing
          while a per-cpt lock is held?

   the_lnet.ln_counters - keep them stable while reading.
   the_lnet.ln_nis ??
   the_lnet.ln_nis_cpt list of lnet_ni... list of network interfaces I guess.
     Locking a per-cpt lock stops updates as all locks are needed to
     change.  These days RCU is often used for this sort of thing.

   lnet_ping_info
   ... and maybe lots more.


So ln_net_lock seems to be  "read/write locks with one spinlock per
CPT." that you described.  I wonder how much of that could be converted
to use RCU - with just a single spinlock to protect updates, and
rcu_read_lock() to make reads safe.

ln_res_lock is quite different - it protects a selection of different
resources that are distributed across multiple cpts.
I wonder why we don't have one lock per resource...
I also wonder how import having these things per-cpt is.
Lots of other code in the kernel has per-CPU lists etc, and
some have per-numa-node, but no other code seems to be need an
intermediate granularity.

I might try to drill down into some of this code a bit more and see what
I can find.  There is probably something I'm still missing.

Thanks,
NeilBrown

   

>
> As a general rule, CPT boundaries should respect node and socket boundaries,
> but at the higher end, where CPUs have 20+ cores, it may make sense to split
> a CPUs cores across several CPTs.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180705/d978b412/attachment.sig>

From jsimmons at infradead.org  Thu Jul  5 22:01:52 2018
From: jsimmons at infradead.org (James Simmons)
Date: Thu, 5 Jul 2018 23:01:52 +0100 (BST)
Subject: [lustre-devel] [PATCH 02/18] lustre: ladvise: Add feature of
 giving file access advices
In-Reply-To: <8736x12duq.fsf@notabene.neil.brown.name>
References: <1530573875-20465-1-git-send-email-jsimmons@infradead.org>
 <1530573875-20465-3-git-send-email-jsimmons@infradead.org>
 <8736x12duq.fsf@notabene.neil.brown.name>
Message-ID: <alpine.LFD.2.21.1807052211530.12665@casper.infradead.org>


> > +	case LL_IOC_LADVISE: {
> > +		struct ladvise_hdr *ladvise_hdr;
> > +		int alloc_size = sizeof(*ladvise_hdr);
> > +		int num_advise;
> > +		int i;
> > +
> > +		rc = 0;
> > +		ladvise_hdr = kzalloc(alloc_size, GFP_NOFS);
> 
> Lustre really needs to get more sensible about when to use GFP_NOFS and
> when to use GFP_KERNEL.
> NOFS is only needed when the allocation cannot wait while reclaim enters
> the filesystem, due to possible deadlock.  So if you are holding a lock
> or some other resource that writeout might need, use NOFS.
> In this case, there is no possible conflict with writeout, so GFP_KERNEL
> should be used.
> 
> In this case, alloc_size is 32, so I wonder why it is allocating memory
> at all rather than just using the stack.
> 
> Below the memory is freed and allocated again (why not realloc?) so an
> allocation is justified there.  That should be GFP_KERNEL as well.
> 
> Later in osc_io_ladvise_start() there is a kvzalloc() with GFP_NOFS.
> I suspect that should be GFP_KERNEL as well, especially as vmalloc
> doesn't properly support GFP_NOFS and never has.

This is the result of the porting from the OpenSFS tree. So in the
OpenSFS branch it still does all memory allocations using its OBD_ALLOC*
wrappers. Unfolding those wrappers you will see its mostly kmallocs done
with the GFP_NOFS flag. This is the same reason lustre avoids using the
kasprintf() and related functions as well. Those wrappers were developed
to be used to locate memory leaks which was done long before KASAN and
trace events was created. Looking at __OBD_VMALLOC_VERBOSE() you will
see its using GFP_NOFS which appears to not to be even supported.
Oh fun!!!

Since we are on this topic it might be good idea to discuss getting onto
0day and work to get KASAN going with the linux lustre client branch. The
directions are not very clear on how to set it up. Have you ever worked
with zero day? As for KASAN that can be done locally but sadly you need
at least gcc 4.9.2 which in my setup I don't have. I did notice a lustre
instance exist on 0day but it doesn't look alive in any way.

As a side note I did apply for lustre-devel to be include on 
patchworks.linux.org. They are currently in the process to upgrade to
a newer software stack so it will another week or so for it to be visible.


From jsimmons at infradead.org  Thu Jul  5 22:11:51 2018
From: jsimmons at infradead.org (James Simmons)
Date: Thu, 5 Jul 2018 23:11:51 +0100 (BST)
Subject: [lustre-devel] [PATCH 00/10] lustre: libcfs: fix libcfs debugfs
 bugs
In-Reply-To: <87y3erzqvx.fsf@notabene.neil.brown.name>
References: <1530573894-20592-1-git-send-email-jsimmons@infradead.org>
 <87y3erzqvx.fsf@notabene.neil.brown.name>
Message-ID: <alpine.LFD.2.21.1807052305080.12665@casper.infradead.org>


> > The original port to sysfs / debugfs was done in haste without any
> > testing so many bugs were introduced. This patch set resolves the
> > bugs found in the libcfs code when it was reviewed.
> >
> > James Simmons (10):
> >   lustre: libcfs: rename lustre_*_debugfs to lnet_*_debugfs
> >   lustre: libcfs: rename *_debugmb to *_debug_mb
> >   lustre: libcfs: make lnet_debugfs_symlink_def fields const
> >   lustre: libcfs: fix module loading and sysfs setting race
> >   lustre: libcfs: fix wrong debugfs symlink name
> >   lustre: libcfs: don't assume sysfs mount point
> >   lustre: libcfs: properly initialize lnet_debugfs_symlink_def
> >   lustre: libcfs: test if table is NULL for lnet_insert_debugfs
> >   lustre: libcfs: return ssize_t for lnet_debugfs_*
> >   lustre: libcfs: small cleanups for debugfs code
> >
> >  .../staging/lustre/include/linux/libcfs/libcfs.h   |  2 +-
> >  drivers/staging/lustre/lnet/libcfs/debug.c         | 28 ++++----
> >  drivers/staging/lustre/lnet/libcfs/module.c        | 78 +++++++++++-----------
> >  drivers/staging/lustre/lnet/lnet/router_proc.c     |  2 +-
> >  4 files changed, 57 insertions(+), 53 deletions(-)
> >
> > -- 
> > 1.8.3.1
> 
> These all look good to me, so I've (provisionally) applied them.
> I didn't know about kernel_param_lock() before. I wonder if it
> would make sense to use that to protect updates to sbi->ll_flags???

Sadly no. The kernel_param_lock() takes a special mutex, param_lock to 
handle the racing between modules and sysfs. In the case of sbu->ll_flags
that is parsed from the mount string. You can see the flow in obd_mount.c
with lmd->lmd_opts being filled from the mount string and that ends up
being processed in llite_lib.c with the function ll_option(). Their it
translates to sbi->ll_flags.

From jsimmons at infradead.org  Thu Jul  5 22:50:39 2018
From: jsimmons at infradead.org (James Simmons)
Date: Thu, 5 Jul 2018 23:50:39 +0100 (BST)
Subject: [lustre-devel] [PATCH 16/18] lustre: libcfs: restore original
 behavior in cfs_str2num_check
In-Reply-To: <87o9fp0vx1.fsf@notabene.neil.brown.name>
References: <1530573875-20465-1-git-send-email-jsimmons@infradead.org>
 <1530573875-20465-17-git-send-email-jsimmons@infradead.org>
 <87o9fp0vx1.fsf@notabene.neil.brown.name>
Message-ID: <alpine.LFD.2.21.1807052316160.12665@casper.infradead.org>


> On Mon, Jul 02 2018, James Simmons wrote:
> 
> > When cfs_str2num_check() moved from simple_strtoul to kstrtoul
> > some of the functionality got lost. Restore handling hexidecimal
> > number as well as '+' and '-'. Also handle any trailing spaces.
> >
> > Signed-off-by: James Simmons <uja.ornl at yahoo.com>
> > WC-bug-id: https://jira.whamcloud.com/browse/LU-9325
> > Reviewed-on: https://review.whamcloud.com/32217
> > Reviewed-by: John L. Hammond <jhammond at whamcloud.com>
> > Reviewed-by: Andreas Dilger <adilger at whamcloud.com>
> > Reviewed-by: Dmitry Eremin <dmitry.eremin at intel.com>
> > Reviewed-by: Oleg Drokin <green at whamcloud.com>
> > Signed-off-by: James Simmons <jsimmons at infradead.org>
> > ---
> >  drivers/staging/lustre/lnet/libcfs/libcfs_string.c | 17 ++++++++++++++---
> >  1 file changed, 14 insertions(+), 3 deletions(-)
> >
> > diff --git a/drivers/staging/lustre/lnet/libcfs/libcfs_string.c b/drivers/staging/lustre/lnet/libcfs/libcfs_string.c
> > index e1fb126..e390b0b 100644
> > --- a/drivers/staging/lustre/lnet/libcfs/libcfs_string.c
> > +++ b/drivers/staging/lustre/lnet/libcfs/libcfs_string.c
> > @@ -214,8 +214,10 @@ char *cfs_firststr(char *str, size_t size)
> >  {
> >  	bool all_numbers = true;
> >  	char *endp, cache;
> > +	int len;
> >  	int rc;
> >  
> > +	endp = strim(str);
> 
> This looks bad.  The function now changes the string buffer, where it
> didn't before.

So original this code uses simple_strtoul() which always parsed the string 
for numbers and succeded even if non digits existed in the string. In fact
you could get pointer to the first non-digit. That was also used to trim
the string. This behavior is very different from kstroul. I guess I will
need to duplicate the string and free it at the end.
 
> >  	/**
> >  	 * kstrouint can only handle strings composed
> >  	 * of only numbers. We need to scan the string
> > @@ -228,16 +230,25 @@ char *cfs_firststr(char *str, size_t size)
> >  	 * After we are done the character at the
> >  	 * position we placed '\0' must be restored.
> >  	 */
> > -	for (endp = str; endp < str + nob; endp++) {
> > -		if (!isdigit(*endp)) {
> > +	len = min((int)strlen(endp), nob);
> 
> As endp might be different from 'str, 'nob' is not meaningful in the
> context of endp.

I believe nob is supposed to be the string length of "str" itself. We
could error on the side caution and make that

len = min(strlen(endp), strlen(str));
  
> I'm guessing that mainline commit
> 
> Commit: c948390f10cc ("staging: lustre: llite: fix inconsistencies of root squash feature")
> 
> is related to this.

Actually it is commit ae0246da1603be7e7374621741515c2b6f2d6332 which ended 
up breaking my setup. I later debugged it and pushed the patch:

3ad6152d766039cb8ffd8633d971fb79402e5464

Later I pushed that patch to the OpenSFS branch and Hammond pointed out 
that some supported behaviors were lost. You can see this at:

https://review.whamcloud.com/#/c/32217/
 
> I cannot apply this patch as-is.  It is broken.  Possibly changing
>    endp = strim(str);
> to
>    endp = str;
> 
> will fix it.

I do see that the trim thing was removed in the root squash handling.
We might of returned a bug by mistake. 

From jsimmons at infradead.org  Thu Jul  5 22:53:15 2018
From: jsimmons at infradead.org (James Simmons)
Date: Thu, 5 Jul 2018 23:53:15 +0100 (BST)
Subject: [lustre-devel] [PATCH 18/18] lustre: update version to 2.8.99
In-Reply-To: <29BE954A-B97C-4EED-8CE0-70FD0B14AE9E@whamcloud.com>
References: <1530573875-20465-1-git-send-email-jsimmons@infradead.org>,
 <1530573875-20465-19-git-send-email-jsimmons@infradead.org>
 <29BE954A-B97C-4EED-8CE0-70FD0B14AE9E@whamcloud.com>
Message-ID: <alpine.LFD.2.21.1807052351370.12665@casper.infradead.org>


> James, thanks for pushing up these patches.
> 
> Why not use 2.9.0? Are there still patches missing, or do you want this
> version to distinguish it from the actual 2.9.0 release?

Oleg asked that I use a version number never used by an offical release.
Should we can that policy? Anyways their might be stuff missing still.
Once we reach the 2.10 LTS release we can do a scan of the code to see 
if this is the case.
 
> Cheers, Andreas
> 
> > On Jul 2, 2018, at 17:36, James Simmons <jsimmons at infradead.org> wrote:
> > 
> > With the missing patches from the lustre 2.9 release merged
> > upstream its time to update the upstream client's version.
> > 
> > Signed-off-by: James Simmons <jsimmons at infradead.org>
> > ---
> > drivers/staging/lustre/include/uapi/linux/lustre/lustre_ver.h | 4 ++--
> > 1 file changed, 2 insertions(+), 2 deletions(-)
> > 
> > diff --git a/drivers/staging/lustre/include/uapi/linux/lustre/lustre_ver.h b/drivers/staging/lustre/include/uapi/linux/lustre/lustre_ver.h
> > index 19c9135..1428fdd 100644
> > --- a/drivers/staging/lustre/include/uapi/linux/lustre/lustre_ver.h
> > +++ b/drivers/staging/lustre/include/uapi/linux/lustre/lustre_ver.h
> > @@ -2,10 +2,10 @@
> > #define _LUSTRE_VER_H_
> > 
> > #define LUSTRE_MAJOR 2
> > -#define LUSTRE_MINOR 6
> > +#define LUSTRE_MINOR 8
> > #define LUSTRE_PATCH 99
> > #define LUSTRE_FIX 0
> > -#define LUSTRE_VERSION_STRING "2.6.99"
> > +#define LUSTRE_VERSION_STRING "2.8.99"
> > 
> > #define OBD_OCD_VERSION(major, minor, patch, fix)            \
> >    (((major) << 24) + ((minor) << 16) + ((patch) << 8) + (fix))
> > -- 
> > 1.8.3.1
> > 
> 

From jsimmons at infradead.org  Thu Jul  5 23:34:00 2018
From: jsimmons at infradead.org (James Simmons)
Date: Fri, 6 Jul 2018 00:34:00 +0100 (BST)
Subject: [lustre-devel] [PATCH v3 04/13] lustre: libcfs: fix
 cfs_print_to_console()
In-Reply-To: <87h8li2wz7.fsf@notabene.neil.brown.name>
References: <1530128322-32535-1-git-send-email-jsimmons@infradead.org>
 <1530128322-32535-5-git-send-email-jsimmons@infradead.org>
 <FD87D2B8-27A8-45DF-9C7C-BA33CF940C1D@whamcloud.com>
 <alpine.LFD.2.21.1806291716050.26339@casper.infradead.org>
 <87h8li2wz7.fsf@notabene.neil.brown.name>
Message-ID: <alpine.LFD.2.21.1807052354100.12665@casper.infradead.org>


> >> > The original code for cfs_print_to_console() used printk() and
> >> > used tricks to select which printk level to use. Later a cleanup
> >> > patch landed the just converted printk directly to pr_info which
> >> > is not exactly correct. Instead of converting back to printk lets
> >> > move everything to pr_* type functions which simplify the code.
> >> > This allows us to fold both dbghdr_to_err_string() and the
> >> > function dbghdr_to_info_string() into cfs_print_to_console().
> >> > 
> >> > Signed-off-by: James Simmons <jsimmons at infradead.org>
> >> > ---
> >> > .../staging/lustre/lnet/libcfs/linux-tracefile.c   | 55 ++++++----------------
> >> > 1 file changed, 14 insertions(+), 41 deletions(-)
> >> > 
> >> > diff --git a/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c b/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
> >> > index 3af7722..c1747c4 100644
> >> > --- a/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
> >> > +++ b/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
> >> > @@ -140,54 +140,27 @@ enum cfs_trace_buf_type cfs_trace_buf_idx_get(void)
> >> > void cfs_print_to_console(struct ptldebug_header *hdr, int mask,
> >> > 			  const char *buf, int len, const char *file,
> >> > 			  const char *fn)
> >> > {
> >> > +	char *prefix = "Lustre";
> >> > +
> >> > +	if (hdr->ph_subsys == S_LND || hdr->ph_subsys == S_LNET)
> >> > +		prefix = "LNet";
> >> > 
> >> > -	if (mask & D_EMERG) {
> >> > -		prefix = dbghdr_to_err_string(hdr);
> >> > -		ptype = KERN_EMERG;
> >> > +	if (mask & (D_CONSOLE | libcfs_printk)) {
> >> 
> >> This check is broken.  The default value of libcfs_printk (the mask
> >> that controls which messages should be printed to the console, and
> >> which ones should only be logged into the internal buffer) is:
> >> 
> >>     #define D_CANTMASK (D_ERROR | D_EMERG | D_WARNING | D_CONSOLE)
> >> 
> >> so that means virtually every console message will be printed with
> >> pr_info() because this is matched first, instead of pr_emerg() or
> >> pr_err() below.
> >> 
> >> That is why the previous code was matching D_EMERG and D_ERROR first,
> >> then D_WARNING, and (D_CONSOLE | libcfs_printk) at the end.
> >
> > So to do this right we need:
> >
> > static void cfs_print_to_console(struct ptldebug_header *hdr, int mask,
> >                                  const char *buf, int len, const char 
> > *file,
> >                                  const char *fn)
> > {
> >         char *prefix = "Lustre";
> >
> >         if (hdr->ph_subsys == S_LND || hdr->ph_subsys == S_LNET)
> >                 prefix = "LNet";
> >
> >         if (mask & D_CONSOLE) {
> >                 if (mask & D_EMERG) {
> >                         pr_emerg("%sError: %.*s", prefix, len, buf);
> >                 } else if (mask & D_ERROR) {
> >                         pr_err("%sError: %.*s", prefix, len, buf);
> >                 } else if (mask & D_WARNING) {
> >                         pr_warn("%s: %.*s", prefix, len, buf);
> >                 } else if (mask & libcfs_printk) {
> >                         pr_info("%s: %.*s", prefix, len, buf);
> >                 }
> >         } else {
> >                 if (mask & D_EMERG) {
> >                         pr_emerg("%sError: %d:%d:(%s:%d:%s()) %.*s", prefix,
> >                                  hdr->ph_pid, hdr->ph_extern_pid, file,
> >                                  hdr->ph_line_num, fn, len, buf);
> >                 } else if (mask & D_ERROR) {
> >                         pr_err("%sError: %d:%d:(%s:%d:%s()) %.*s", prefix,
> >                                hdr->ph_pid, hdr->ph_extern_pid, file,
> >                                hdr->ph_line_num, fn, len, buf);
> >                 } else if (mask & D_WARNING) {
> >                         pr_warn("%s: %d:%d:(%s:%d:%s()) %.*s", prefix,
> >                                 hdr->ph_pid, hdr->ph_extern_pid, file,
> >                                 hdr->ph_line_num, fn, len, buf);
> >                 } else if (mask & libcfs_printk) {
> >                         pr_info("%s: %.*s", prefix, len, buf);
> >                 }
> >         }
> > }
> 
> That doesn't look right either.
> The original code would *always* print something.
> This code looks like it might not (e.g. for mask == 0).

Is that correct behavior? A mask of zero means don't do anything.
Also if you look at the original in the OpenSFS branch you can see
if mask was to be 0 then ptype would be NULl :-( 

Note a D_CANT_MASK prevents some fields in the mask from being disabled.

> What is "D_CONSOLE" suppose to mean?  It seems to me "make the messages
> less verbose" and it isn't clear to me that what is called "D_CONSOLE".

It means two things. If by itself then it means use pr_info. If it is one
field in the mask then it means less detail. Confusing no :-( That is my
understanding of it. Maybe someone else can clarify if I'm wrong.

From jsimmons at infradead.org  Thu Jul  5 23:35:26 2018
From: jsimmons at infradead.org (James Simmons)
Date: Fri, 6 Jul 2018 00:35:26 +0100 (BST)
Subject: [lustre-devel] [PATCH v3 04/13] lustre: libcfs: fix
 cfs_print_to_console()
In-Reply-To: <87h8li2wz7.fsf@notabene.neil.brown.name>
References: <1530128322-32535-1-git-send-email-jsimmons@infradead.org>
 <1530128322-32535-5-git-send-email-jsimmons@infradead.org>
 <FD87D2B8-27A8-45DF-9C7C-BA33CF940C1D@whamcloud.com>
 <alpine.LFD.2.21.1806291716050.26339@casper.infradead.org>
 <87h8li2wz7.fsf@notabene.neil.brown.name>
Message-ID: <alpine.LFD.2.21.1807060034020.12665@casper.infradead.org>


> >> > The original code for cfs_print_to_console() used printk() and
> >> > used tricks to select which printk level to use. Later a cleanup
> >> > patch landed the just converted printk directly to pr_info which
> >> > is not exactly correct. Instead of converting back to printk lets
> >> > move everything to pr_* type functions which simplify the code.
> >> > This allows us to fold both dbghdr_to_err_string() and the
> >> > function dbghdr_to_info_string() into cfs_print_to_console().
> >> > 
> >> > Signed-off-by: James Simmons <jsimmons at infradead.org>
> >> > ---
> >> > .../staging/lustre/lnet/libcfs/linux-tracefile.c   | 55 ++++++----------------
> >> > 1 file changed, 14 insertions(+), 41 deletions(-)
> >> > 
> >> > diff --git a/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c b/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
> >> > index 3af7722..c1747c4 100644
> >> > --- a/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
> >> > +++ b/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
> >> > @@ -140,54 +140,27 @@ enum cfs_trace_buf_type cfs_trace_buf_idx_get(void)
> >> > void cfs_print_to_console(struct ptldebug_header *hdr, int mask,
> >> > 			  const char *buf, int len, const char *file,
> >> > 			  const char *fn)
> >> > {
> >> > +	char *prefix = "Lustre";
> >> > +
> >> > +	if (hdr->ph_subsys == S_LND || hdr->ph_subsys == S_LNET)
> >> > +		prefix = "LNet";
> >> > 
> >> > -	if (mask & D_EMERG) {
> >> > -		prefix = dbghdr_to_err_string(hdr);
> >> > -		ptype = KERN_EMERG;
> >> > +	if (mask & (D_CONSOLE | libcfs_printk)) {
> >> 
> >> This check is broken.  The default value of libcfs_printk (the mask
> >> that controls which messages should be printed to the console, and
> >> which ones should only be logged into the internal buffer) is:
> >> 
> >>     #define D_CANTMASK (D_ERROR | D_EMERG | D_WARNING | D_CONSOLE)
> >> 
> >> so that means virtually every console message will be printed with
> >> pr_info() because this is matched first, instead of pr_emerg() or
> >> pr_err() below.
> >> 
> >> That is why the previous code was matching D_EMERG and D_ERROR first,
> >> then D_WARNING, and (D_CONSOLE | libcfs_printk) at the end.
> >
> > So to do this right we need:
> >
> > static void cfs_print_to_console(struct ptldebug_header *hdr, int mask,
> >                                  const char *buf, int len, const char 
> > *file,
> >                                  const char *fn)
> > {
> >         char *prefix = "Lustre";
> >
> >         if (hdr->ph_subsys == S_LND || hdr->ph_subsys == S_LNET)
> >                 prefix = "LNet";
> >
> >         if (mask & D_CONSOLE) {
> >                 if (mask & D_EMERG) {
> >                         pr_emerg("%sError: %.*s", prefix, len, buf);
> >                 } else if (mask & D_ERROR) {
> >                         pr_err("%sError: %.*s", prefix, len, buf);
> >                 } else if (mask & D_WARNING) {
> >                         pr_warn("%s: %.*s", prefix, len, buf);
> >                 } else if (mask & libcfs_printk) {
> >                         pr_info("%s: %.*s", prefix, len, buf);
> >                 }
> >         } else {
> >                 if (mask & D_EMERG) {
> >                         pr_emerg("%sError: %d:%d:(%s:%d:%s()) %.*s", prefix,
> >                                  hdr->ph_pid, hdr->ph_extern_pid, file,
> >                                  hdr->ph_line_num, fn, len, buf);
> >                 } else if (mask & D_ERROR) {
> >                         pr_err("%sError: %d:%d:(%s:%d:%s()) %.*s", prefix,
> >                                hdr->ph_pid, hdr->ph_extern_pid, file,
> >                                hdr->ph_line_num, fn, len, buf);
> >                 } else if (mask & D_WARNING) {
> >                         pr_warn("%s: %d:%d:(%s:%d:%s()) %.*s", prefix,
> >                                 hdr->ph_pid, hdr->ph_extern_pid, file,
> >                                 hdr->ph_line_num, fn, len, buf);
> >                 } else if (mask & libcfs_printk) {
> >                         pr_info("%s: %.*s", prefix, len, buf);
> >                 }
> >         }
> > }
> 
> That doesn't look right either.
> The original code would *always* print something.
> This code looks like it might not (e.g. for mask == 0).

Is that correct behavior? A mask of zero means don't do anything.
Also if you look at the original in the OpenSFS branch you can see
if mask was to be 0 then ptype would be NULl :-( 

Note a D_CANT_MASK prevents some fields in the mask from being disabled.

> What is "D_CONSOLE" suppose to mean?  It seems to me "make the messages
> less verbose" and it isn't clear to me that what is called "D_CONSOLE".

It means two things. If by itself then it means use pr_info. If it is one
field in the mask then it means less detail. Confusing no :-( That is my
understanding of it. Maybe someone else can clarify if I'm wrong.

From jsimmons at infradead.org  Thu Jul  5 23:47:44 2018
From: jsimmons at infradead.org (James Simmons)
Date: Fri, 6 Jul 2018 00:47:44 +0100 (BST)
Subject: [lustre-devel] [PATCH 00/24] lustre - more cleanups including
 module reduction.
In-Reply-To: <D6CA5D91-2344-446F-9621-935A52E47A24@whamcloud.com>
References: <152904663333.10587.10934053155404014785.stgit@noble>
 <alpine.LFD.2.21.1806210239190.3886@casper.infradead.org>
 <DM3PR1101MB1213DDB7FC857EE6B1444153CB760@DM3PR1101MB1213.namprd11.prod.outlook.com>
 <alpine.LFD.2.21.1806210354260.3886@casper.infradead.org>
 <3FD4D051-9C95-449D-8A23-D42B271E55B8@dilger.ca>
 <DM3PR1101MB121322E85FDE1E5310F6F4B7CB760@DM3PR1101MB1213.namprd11.prod.outlook.com>
 <87tvprah32.fsf@notabene.neil.brown.name>
 <alpine.LFD.2.21.1806260206360.31196@casper.infradead.org>
 <DM3PR1101MB121358672276198B843DE5A7CB490@DM3PR1101MB1213.namprd11.prod.outlook.com>
 <87d0wd6i4m.fsf@notabene.neil.brown.name>
 <44633E99-07CB-4C40-967C-0B4FE206F29A@whamcloud.com>
 <DM3PR1101MB12132AC3E33E7D7C8FBE4536CB480@DM3PR1101MB1213.namprd11.prod.outlook.com>
 <87tvpn6592.fsf@notabene.neil.brown.name>
 <DM3PR1101MB12139B5B51BCE76A79DEC383CB4F0@DM3PR1101MB1213.namprd11.prod.outlook.com>
 <D6CA5D91-2344-446F-9621-935A52E47A24@whamcloud.com>
Message-ID: <alpine.LFD.2.21.1807060043430.12665@casper.infradead.org>


> > I would say about the acceptor and sock lnd: I believe Lustre assumes some IP transport is available for configuration, but does NOT necessarily use it for primary communication.  Fabrics - Like infiniband or Cray Aries - are more or less always configured to provide IP transport, to enable the panoply of tools and apps that rely on it.  But they perform better if their native protocols are used, which is of course what the other LNDs do.
> 
> It is worthwhile to clarify this a bit - LNet uses IP for *addressing* of the nodes at connection time, but I don't think it even uses the TCP interface for any communication itself (though I could be mistaken, as LNet isn't my specialty).  After the initial connection, the only other thing the IP addresses are used for is printing in error messages.
> 
> At one time or another we've discussed how we might get rid of the need for IPoIB on client nodes, since some sites don't want any IP connectivity to the client nodes for security and performance reasons.  That said, I don't think we've come up with a good solution yet.  LNet itself allows alternate addressing schemes to be used.  The former qswlnd (for Quadrics Elan networks) just used an integer node number, like 1 at elan for the NID, but I don't think there is any such alternative for IB node addresses except using a MAC hardware address or similar.

The challenge to moving away from IP address for addressing is the NID 
address itself. Currently it is 64 bits which is composed for two parts.

static inline lnet_nid_t LNET_MKNID(__u32 net, __u32 addr)
{
        return (((__u64)net) << 32) | addr;
}

To use IB harware addressing would require a full 64 bit space so
lnet_nid_t is to small. This is reason IPv6 is not supported. Ideas
have floated around on how to get around this but nothing yet.
It would be a massive API change currently.

> > Neil Brown <neilb at suse.com> wrote:
> >> 
> >> On Wed, Jun 27 2018, Patrick Farrell wrote:
> >> 
> >> > Neil,
> >> >
> >> > We do indeed have such functionality (it’s called DVS and it’s
> >> > basically a high speed file system projection framework, ala NFS but
> >> > faster), so the ability to build lnet separately is valuable to us.
> >> > While it is being open sourced under the GPL, I don’t think there’s
> >> > any intention to try to upstream it.  The current code isn’t even
> >> > usable off of Cray systems as it depends on info from user space (that
> >> > is provided, in the end, from Cray proprietary hardware) to keep its
> >> > connection/routing tables up to date.  That’s supposedly in the
> >> > pipeline to get fixed, but it’s still pretty far from generally
> >> > usable.
> >> >
> >> > But we’d still really appreciate it if lnet stayed separate.  Don’t
> >> > know if that’s enough for you - I know sometimes *small* stuff is done
> >> > for out of tree users.  Hopefully this meets that standard.
> >> >
> >> 
> >> Ahh - DVS.  That answers a question I just asked in another email.
> >> My google-skills don't seem to be up to locating the source code though
> >> :-(
> >> 
> >> While I wouldn't knowingly break an interface used by some out-of-tree
> >> code without good reason, it is hard to avoid if you don't know what the
> >> out-of-tree code does.  It can be very tempting to remove something that
> >> isn't being used, but that can certainly hurt out-of-tree code
> >> sometimes.
> >> 
> >> A particular example I'm exploring at present is the dual data paths in
> >> LNet.  Or maybe it is dual types of Memory Descriptors.
> >> There is 'kiov' which uses kernel-virtual addresses and 'iovec' which
> >> uses page+offset.
> >> The kiov option isn't used in the client code and it seems likely that
> >> the server-side code could be converted to use iovec without problems.
> >> 
> >> I'd like to remove the kiov as I wouldn't be able to justify its
> >> existence when submitting the client-only code upstream.  But I don't
> >> want to remove the option of having an alternate MD type if it really is
> >> significantly more efficient in some context.
> >> If I know whether DVS used kiov or iovec - and in what way - that would
> >> help me to know if I might break something, and to be able to assess the
> >> cost.
> >> 
> >> In my mind, the "standard" that you mention is always about
> >> practicality.   Code needs to be maintainable - easy to understand and
> >> hard to break.  If the LNet interface is clean and well documented in
> >> the kernel, then I don't see why we would not at least attempt to
> >> preserve it.
> >> 
> >> Thanks,
> >> NeilBrown
> 
> Cheers, Andreas
> ---
> Andreas Dilger
> Principal Lustre Architect
> Whamcloud
> 
> 
> 
> 
> 
> 
> 
> 

From doucharek at cray.com  Fri Jul  6 00:01:46 2018
From: doucharek at cray.com (Doug Oucharek)
Date: Fri, 6 Jul 2018 00:01:46 +0000
Subject: [lustre-devel] [PATCH 00/24] lustre - more cleanups including
 module reduction.
In-Reply-To: <alpine.LFD.2.21.1807060043430.12665@casper.infradead.org>
References: <152904663333.10587.10934053155404014785.stgit@noble>
 <alpine.LFD.2.21.1806210239190.3886@casper.infradead.org>
 <DM3PR1101MB1213DDB7FC857EE6B1444153CB760@DM3PR1101MB1213.namprd11.prod.outlook.com>
 <alpine.LFD.2.21.1806210354260.3886@casper.infradead.org>
 <3FD4D051-9C95-449D-8A23-D42B271E55B8@dilger.ca>
 <DM3PR1101MB121322E85FDE1E5310F6F4B7CB760@DM3PR1101MB1213.namprd11.prod.outlook.com>
 <87tvprah32.fsf@notabene.neil.brown.name>
 <alpine.LFD.2.21.1806260206360.31196@casper.infradead.org>
 <DM3PR1101MB121358672276198B843DE5A7CB490@DM3PR1101MB1213.namprd11.prod.outlook.com>
 <87d0wd6i4m.fsf@notabene.neil.brown.name>
 <44633E99-07CB-4C40-967C-0B4FE206F29A@whamcloud.com>
 <DM3PR1101MB12132AC3E33E7D7C8FBE4536CB480@DM3PR1101MB1213.namprd11.prod.outlook.com>
 <87tvpn6592.fsf@notabene.neil.brown.name>
 <DM3PR1101MB12139B5B51BCE76A79DEC383CB4F0@DM3PR1101MB1213.namprd11.prod.outlook.com>
 <D6CA5D91-2344-446F-9621-935A52E47A24@whamcloud.com>
 <alpine.LFD.2.21.1807060043430.12665@casper.infradead.org>
Message-ID: <D654AFAE-B860-4BE8-B988-57C481597AD5@cray.com>

Unfortunately, NIDs get stored outside of LNet in the Lustre configuration.  That’s why changing the nature of NIDs requires a lot of changes in many places in and out of LNet.  

In my opinion, the best gameplay is to get Lustre to stop even knowing what a NID is.  Have Lustre identify nodes via a name which LNet converts to a NID (sort of like how DNS works).  At that point, LNet can make the NID be anything you want: IPv6, IB address, etc.  This also means you don’t have to run writeconf (or something like that) when the network changes.  Only LNet needs to be updated.

Doug

> On Jul 5, 2018, at 4:47 PM, James Simmons <jsimmons at infradead.org> wrote:
> 
> 
>>> I would say about the acceptor and sock lnd: I believe Lustre assumes some IP transport is available for configuration, but does NOT necessarily use it for primary communication.  Fabrics - Like infiniband or Cray Aries - are more or less always configured to provide IP transport, to enable the panoply of tools and apps that rely on it.  But they perform better if their native protocols are used, which is of course what the other LNDs do.
>> 
>> It is worthwhile to clarify this a bit - LNet uses IP for *addressing* of the nodes at connection time, but I don't think it even uses the TCP interface for any communication itself (though I could be mistaken, as LNet isn't my specialty).  After the initial connection, the only other thing the IP addresses are used for is printing in error messages.
>> 
>> At one time or another we've discussed how we might get rid of the need for IPoIB on client nodes, since some sites don't want any IP connectivity to the client nodes for security and performance reasons.  That said, I don't think we've come up with a good solution yet.  LNet itself allows alternate addressing schemes to be used.  The former qswlnd (for Quadrics Elan networks) just used an integer node number, like 1 at elan for the NID, but I don't think there is any such alternative for IB node addresses except using a MAC hardware address or similar.
> 
> The challenge to moving away from IP address for addressing is the NID 
> address itself. Currently it is 64 bits which is composed for two parts.
> 
> static inline lnet_nid_t LNET_MKNID(__u32 net, __u32 addr)
> {
>        return (((__u64)net) << 32) | addr;
> }
> 
> To use IB harware addressing would require a full 64 bit space so
> lnet_nid_t is to small. This is reason IPv6 is not supported. Ideas
> have floated around on how to get around this but nothing yet.
> It would be a massive API change currently.
> 
>>> Neil Brown <neilb at suse.com> wrote:
>>>> 
>>>> On Wed, Jun 27 2018, Patrick Farrell wrote:
>>>> 
>>>>> Neil,
>>>>> 
>>>>> We do indeed have such functionality (it’s called DVS and it’s
>>>>> basically a high speed file system projection framework, ala NFS but
>>>>> faster), so the ability to build lnet separately is valuable to us.
>>>>> While it is being open sourced under the GPL, I don’t think there’s
>>>>> any intention to try to upstream it.  The current code isn’t even
>>>>> usable off of Cray systems as it depends on info from user space (that
>>>>> is provided, in the end, from Cray proprietary hardware) to keep its
>>>>> connection/routing tables up to date.  That’s supposedly in the
>>>>> pipeline to get fixed, but it’s still pretty far from generally
>>>>> usable.
>>>>> 
>>>>> But we’d still really appreciate it if lnet stayed separate.  Don’t
>>>>> know if that’s enough for you - I know sometimes *small* stuff is done
>>>>> for out of tree users.  Hopefully this meets that standard.
>>>>> 
>>>> 
>>>> Ahh - DVS.  That answers a question I just asked in another email.
>>>> My google-skills don't seem to be up to locating the source code though
>>>> :-(
>>>> 
>>>> While I wouldn't knowingly break an interface used by some out-of-tree
>>>> code without good reason, it is hard to avoid if you don't know what the
>>>> out-of-tree code does.  It can be very tempting to remove something that
>>>> isn't being used, but that can certainly hurt out-of-tree code
>>>> sometimes.
>>>> 
>>>> A particular example I'm exploring at present is the dual data paths in
>>>> LNet.  Or maybe it is dual types of Memory Descriptors.
>>>> There is 'kiov' which uses kernel-virtual addresses and 'iovec' which
>>>> uses page+offset.
>>>> The kiov option isn't used in the client code and it seems likely that
>>>> the server-side code could be converted to use iovec without problems.
>>>> 
>>>> I'd like to remove the kiov as I wouldn't be able to justify its
>>>> existence when submitting the client-only code upstream.  But I don't
>>>> want to remove the option of having an alternate MD type if it really is
>>>> significantly more efficient in some context.
>>>> If I know whether DVS used kiov or iovec - and in what way - that would
>>>> help me to know if I might break something, and to be able to assess the
>>>> cost.
>>>> 
>>>> In my mind, the "standard" that you mention is always about
>>>> practicality.   Code needs to be maintainable - easy to understand and
>>>> hard to break.  If the LNet interface is clean and well documented in
>>>> the kernel, then I don't see why we would not at least attempt to
>>>> preserve it.
>>>> 
>>>> Thanks,
>>>> NeilBrown
>> 
>> Cheers, Andreas
>> ---
>> Andreas Dilger
>> Principal Lustre Architect
>> Whamcloud
>> 
>> 
>> 
>> 
>> 
>> 
>> 
> _______________________________________________
> lustre-devel mailing list
> lustre-devel at lists.lustre.org
> http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org


From jsimmons at infradead.org  Fri Jul  6 00:20:37 2018
From: jsimmons at infradead.org (James Simmons)
Date: Fri, 6 Jul 2018 01:20:37 +0100 (BST)
Subject: [lustre-devel] [PATCH v3 07/26] staging: lustre: libcfs: NUMA
 support
In-Reply-To: <AT5PR8401MB0900A5406BF0CB8C87B80EEC85410@AT5PR8401MB0900.NAMPRD84.PROD.OUTLOOK.COM>
References: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
 <1529875250-11531-8-git-send-email-jsimmons@infradead.org>
 <877emnaed8.fsf@notabene.neil.brown.name>
 <A764DED1-A9A8-4FA5-A0F1-922C3E2D822A@cray.com>
 <87lgb16j8q.fsf@notabene.neil.brown.name>
 <DM3PR1101MB1213FD1001256E565FF011FFCB480@DM3PR1101MB1213.namprd11.prod.outlook.com>
 <8736x77lrk.fsf@notabene.neil.brown.name>
 <9DE389AB-C7E1-4336-B7E8-604581EFD53E@cray.com>
 <CAAqp6i0uwmpdFYUJ7mCBz+MH+gVBa6S=P2t5Yn7wcFOtrTD-Fg@mail.gmail.com>
 <AT5PR8401MB0900C67FADDF0EF7ED1CDBC4854E0@AT5PR8401MB0900.NAMPRD84.PROD.OUTLOOK.COM>
 <87va9vziay.fsf@notabene.neil.brown.name>
 <AT5PR8401MB0900A5406BF0CB8C87B80EEC85410@AT5PR8401MB0900.NAMPRD84.PROD.OUTLOOK.COM>
Message-ID: <alpine.LFD.2.21.1807060052470.12665@casper.infradead.org>


> NeilBrown [mailto:neilb at suse.com] wrote:
> 
> To help contextualize things: the Lustre code can be decomposed into three parts:
> 
> 1) The filesystem proper: Lustre.
> 2) The communication protocol it uses: LNet.
> 3) Supporting code used by Lustre and LNet: CFS.
> 
> Part of the supporting code is the CPT mechanism, which provides a way to
> partition the CPUs of a system. These partitions are used to distribute queues,
> locks, and threads across the system. It was originally introduced years ago, as
> far as I can tell mainly to deal with certain hot locks: these were converted into
> read/write locks with one spinlock per CPT.
> 
> As a general rule, CPT boundaries should respect node and socket boundaries,
> but at the higher end, where CPUs have 20+ cores, it may make sense to split
> a CPUs cores across several CPTs.
> 
> > Thanks everyone for your patience in explaining things to me.
> > I'm beginning to understand what to look for and where to find it.
> > 
> > So the answers to Greg's questions:
> > 
> >   Where are you reading the host memory NUMA information from?
> > 
> >   And why would a filesystem care about this type of thing?  Are you
> >   going to now mirror what the scheduler does with regards to NUMA
> >   topology issues?  How are you going to handle things when the topology
> >   changes?  What systems did you test this on?  What performance
> >   improvements were seen?  What downsides are there with all of this?
> > 
> > 
> > Are:
> 
> >   - NUMA info comes from ACPI or device-tree just like for every one
> >       else.  Lustre just uses node_distance().
> 
> Correct, the standard kernel interfaces for this information are used to
> obtain it, so ultimately Lustre/LNet uses the same source of truth as
> everyone else.
> 
> >   - The filesystem cares about this because...  It has service
> >     thread that does part of the work of some filesystem operations
> >     (handling replies for example) and these are best handled "near"
> >     the CPU the initiated the request.  Lustre partitions
> >     all CPUs into "partitions" (cpt) each with a few cores.
> >     If the request thread and the reply thread are on different
> >     CPUs but in the same partition, then we get best throughput
> >     (is that close?)
> 
> At the filesystem level, it does indeed seem to help to have the service
> threads that do work for requests run on a different core that is close to
> the core that originated the request. So preferably on the same CPU, and
> on certain multi-core CPUs there are also distance effects between cores.
> That too is one of the things the CPT mechanism handles.

Their is another very important aspect to why Lustre has a CPU partition 
layer. At least at the place I work at. While the Linux kernel manages all
the NUMA nodes and CPU cores Lustre adds the ability for us to specify a 
subset of everything on the system. The reason is to limit the impact of
noise on the compute nodes. Noise has a heavy impact on large scale HP
work loads that can run days or even weeks at a time. Lets take an 
example system:

	       |-------------|	   |-------------|
   |-------|   | NUMA  0     |	   | NUMA  1	 |   |-------|
   | eth0  | - |	     | --- |		 | - | eth1  |      
   |_______|   | CPU0  CPU1  |	   | CPU2  CPU3  |   |_______|
	       |_____________|	   |_____________|

In such a system it is possible with the right job schedular to start a 
large parallel application on NUMA 0/ (CPU0 and CPU1). Normally such
large parallel applications will communicate between nodes using MPI,
such as openmpi, which can be configured to use eth0 only. Using the
CPT layer in lustre we can isolate lustre to NUMA 1 and use only eth1.
This greatly reducess the noise impact on the application running.

BTW this is one of the reasons ko2iblnd for lustre doesn't use the
generic RDMA api. The core IB layer doesn't support such isolation.
At least to my knowledge.

> >   - Not really mirroring the scheduler, maybe mirroring parts of the
> >     network layer(?)
> 
> The LNet code, which is derived from Portals 3.x, is mostly an easier-to-use
> abstraction of RDMA interfaces provided by Infiniband and other similar
> hardware. It can also use TCP/IP, but that's not the primary use case.
> 
> As a communication layer that builds on top of RDMA-capable hardware,
> LNet cares about such things as whether the CPU driving communication
> is close to the memory used, and also whether it is close to the interface
> used. Even in a 2-socket machine, there are measurable performance
> differences depending on whether the memory an interface connect
> to the same socket or to different sockets. On bigger hardware, like a
> 32-socket machine, the penalties are much more pronounced. At the
> time we found that the QPI links between sockets were a bottleneck
> and that performance cratered if they had to handle too much traffic.
> 
> UPI, the successor to QPI is better -- has more bandwidth -- but with
> the CPUs having more and more cores I expect the scaling issues to
> remain similar.
> 
> >   - We don't handle topology changes yet except in very minimal ways
> >     (cpts *can* become empty, and that can cause problems).
> 
> Yes, this is a known deficiency.
> 
> >   - This has been tested on .... great big things.
> 
> The basic CPT mechanism predates my involvement with Lustre. I did
> work on making it more NUMA-aware. A 32-socket system was one of
> the primary test beds.
> 
> >   - When multi-rails configurations are used (like ethernet-bonding,
> >     but for RDMA), we get ??? closer to theoretical bandwidth.
> >     Without these changes it scales poorly (??)
> 
> The basic idea behind muti-rail configurations is that we use several
> Infiniband interfaces and LNet presents them as a single logical interface
> to Lustre. For each message, LNet picks the IB interface it should go across
> using several criteria, including NUMA distance of the interface and how
> busy it is.
> 
> With these changes we could get pretty much linear scaling of LNet
> throughput by adding more interfaces.
> 
> >   - The down-sides primarily are that we don't auto-configure
> >     perfectly.  This particularly affects hot-plug, but without
> >     hotplug the grouping of cpus and interfaces are focussed
> >     on .... avoiding worst case rather than achieving best case.
> 
> Without hotplug the CPT grouping should be tuned to achieve a best
> case in a static configuration.
> 
> Adding simple-minded hotplug tolerance (let's not call it support) would
> focus on avoiding truly pathological behaviour.
> 
> > I've made up a lot of stuff there.  I'm happy not to pursue this further at the
> > moment, but if anyone would like to enhance my understanding by
> > correcting the worst errors in the above, I wouldn't object :-)
> > 
> > Thanks,
> > NeilBrown
> 
> PS: the NUMA effects I've mentioned above have been making the news
> lately under other names: they are part of the side channels used in various
> timing based attacks.
> 
> Olaf
> 
> _______________________________________________
> lustre-devel mailing list
> lustre-devel at lists.lustre.org
> http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org
> 

From paf at cray.com  Fri Jul  6 00:40:29 2018
From: paf at cray.com (Patrick Farrell)
Date: Fri, 6 Jul 2018 00:40:29 +0000
Subject: [lustre-devel] [PATCH v3 07/26] staging: lustre: libcfs: NUMA
 support
In-Reply-To: <alpine.LFD.2.21.1807060052470.12665@casper.infradead.org>
References: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
 <1529875250-11531-8-git-send-email-jsimmons@infradead.org>
 <877emnaed8.fsf@notabene.neil.brown.name>
 <A764DED1-A9A8-4FA5-A0F1-922C3E2D822A@cray.com>
 <87lgb16j8q.fsf@notabene.neil.brown.name>
 <DM3PR1101MB1213FD1001256E565FF011FFCB480@DM3PR1101MB1213.namprd11.prod.outlook.com>
 <8736x77lrk.fsf@notabene.neil.brown.name>
 <9DE389AB-C7E1-4336-B7E8-604581EFD53E@cray.com>
 <CAAqp6i0uwmpdFYUJ7mCBz+MH+gVBa6S=P2t5Yn7wcFOtrTD-Fg@mail.gmail.com>
 <AT5PR8401MB0900C67FADDF0EF7ED1CDBC4854E0@AT5PR8401MB0900.NAMPRD84.PROD.OUTLOOK.COM>
 <87va9vziay.fsf@notabene.neil.brown.name>
 <AT5PR8401MB0900A5406BF0CB8C87B80EEC85410@AT5PR8401MB0900.NAMPRD84.PROD.OUTLOOK.COM>,
 <alpine.LFD.2.21.1807060052470.12665@casper.infradead.org>
Message-ID: <DM3PR1101MB1213C2A5A1CBC6631CB3FFCDCB470@DM3PR1101MB1213.namprd11.prod.outlook.com>


A tiny bit more about noise for Neil, since it’s a bit subtle and I had never heard of it before working in HPC.  Sorry if this is old news.

Noise here means differences in execution time. A typical HPC Job consists of thousands of processes running across a large system.  The basic model is they all run a compute step, then they all communicate part of their results (generally to some neighboring subset of processes, not all-to-all).  The results which are communicated are then used as part of the input to the next compute step.  As you can see, effectively, everyone must finish each step before anyone can continue (or at least, continue very far).

So if everyone finishes every step in the same amount of time, great.  But if there’s jitter in the completion time for a step for a particular process - as can be introduced by a scheduler with ideas that don’t quite line up with your job priorities - it delays the completion of the step overall.  This is compounded at each step of the job and so can be quite serious.  (Job steps can be quite short - double digit microseconds is not unusual - so relatively small jitter can really add up.)

So HPC users are really fussy about affinity and placement control.  Which isn’t to say Lustre gets it all right, but it’s why we care so much.
________________________________
From: lustre-devel <lustre-devel-bounces at lists.lustre.org> on behalf of James Simmons <jsimmons at infradead.org>
Sent: Thursday, July 5, 2018 7:20:37 PM
To: Weber, Olaf (HPC Data Management & Storage)
Cc: Lustre Development List
Subject: Re: [lustre-devel] [PATCH v3 07/26] staging: lustre: libcfs: NUMA support


> NeilBrown [mailto:neilb at suse.com] wrote:
>
> To help contextualize things: the Lustre code can be decomposed into three parts:
>
> 1) The filesystem proper: Lustre.
> 2) The communication protocol it uses: LNet.
> 3) Supporting code used by Lustre and LNet: CFS.
>
> Part of the supporting code is the CPT mechanism, which provides a way to
> partition the CPUs of a system. These partitions are used to distribute queues,
> locks, and threads across the system. It was originally introduced years ago, as
> far as I can tell mainly to deal with certain hot locks: these were converted into
> read/write locks with one spinlock per CPT.
>
> As a general rule, CPT boundaries should respect node and socket boundaries,
> but at the higher end, where CPUs have 20+ cores, it may make sense to split
> a CPUs cores across several CPTs.
>
> > Thanks everyone for your patience in explaining things to me.
> > I'm beginning to understand what to look for and where to find it.
> >
> > So the answers to Greg's questions:
> >
> >   Where are you reading the host memory NUMA information from?
> >
> >   And why would a filesystem care about this type of thing?  Are you
> >   going to now mirror what the scheduler does with regards to NUMA
> >   topology issues?  How are you going to handle things when the topology
> >   changes?  What systems did you test this on?  What performance
> >   improvements were seen?  What downsides are there with all of this?
> >
> >
> > Are:
>
> >   - NUMA info comes from ACPI or device-tree just like for every one
> >       else.  Lustre just uses node_distance().
>
> Correct, the standard kernel interfaces for this information are used to
> obtain it, so ultimately Lustre/LNet uses the same source of truth as
> everyone else.
>
> >   - The filesystem cares about this because...  It has service
> >     thread that does part of the work of some filesystem operations
> >     (handling replies for example) and these are best handled "near"
> >     the CPU the initiated the request.  Lustre partitions
> >     all CPUs into "partitions" (cpt) each with a few cores.
> >     If the request thread and the reply thread are on different
> >     CPUs but in the same partition, then we get best throughput
> >     (is that close?)
>
> At the filesystem level, it does indeed seem to help to have the service
> threads that do work for requests run on a different core that is close to
> the core that originated the request. So preferably on the same CPU, and
> on certain multi-core CPUs there are also distance effects between cores.
> That too is one of the things the CPT mechanism handles.

Their is another very important aspect to why Lustre has a CPU partition
layer. At least at the place I work at. While the Linux kernel manages all
the NUMA nodes and CPU cores Lustre adds the ability for us to specify a
subset of everything on the system. The reason is to limit the impact of
noise on the compute nodes. Noise has a heavy impact on large scale HP
work loads that can run days or even weeks at a time. Lets take an
example system:

               |-------------|      |-------------|
   |-------|   | NUMA  0     |     | NUMA  1      |   |-------|
   | eth0  | - |             | --- |              | - | eth1  |
   |_______|   | CPU0  CPU1  |     | CPU2  CPU3  |   |_______|
               |_____________|      |_____________|

In such a system it is possible with the right job schedular to start a
large parallel application on NUMA 0/ (CPU0 and CPU1). Normally such
large parallel applications will communicate between nodes using MPI,
such as openmpi, which can be configured to use eth0 only. Using the
CPT layer in lustre we can isolate lustre to NUMA 1 and use only eth1.
This greatly reducess the noise impact on the application running.

BTW this is one of the reasons ko2iblnd for lustre doesn't use the
generic RDMA api. The core IB layer doesn't support such isolation.
At least to my knowledge.

> >   - Not really mirroring the scheduler, maybe mirroring parts of the
> >     network layer(?)
>
> The LNet code, which is derived from Portals 3.x, is mostly an easier-to-use
> abstraction of RDMA interfaces provided by Infiniband and other similar
> hardware. It can also use TCP/IP, but that's not the primary use case.
>
> As a communication layer that builds on top of RDMA-capable hardware,
> LNet cares about such things as whether the CPU driving communication
> is close to the memory used, and also whether it is close to the interface
> used. Even in a 2-socket machine, there are measurable performance
> differences depending on whether the memory an interface connect
> to the same socket or to different sockets. On bigger hardware, like a
> 32-socket machine, the penalties are much more pronounced. At the
> time we found that the QPI links between sockets were a bottleneck
> and that performance cratered if they had to handle too much traffic.
>
> UPI, the successor to QPI is better -- has more bandwidth -- but with
> the CPUs having more and more cores I expect the scaling issues to
> remain similar.
>
> >   - We don't handle topology changes yet except in very minimal ways
> >     (cpts *can* become empty, and that can cause problems).
>
> Yes, this is a known deficiency.
>
> >   - This has been tested on .... great big things.
>
> The basic CPT mechanism predates my involvement with Lustre. I did
> work on making it more NUMA-aware. A 32-socket system was one of
> the primary test beds.
>
> >   - When multi-rails configurations are used (like ethernet-bonding,
> >     but for RDMA), we get ??? closer to theoretical bandwidth.
> >     Without these changes it scales poorly (??)
>
> The basic idea behind muti-rail configurations is that we use several
> Infiniband interfaces and LNet presents them as a single logical interface
> to Lustre. For each message, LNet picks the IB interface it should go across
> using several criteria, including NUMA distance of the interface and how
> busy it is.
>
> With these changes we could get pretty much linear scaling of LNet
> throughput by adding more interfaces.
>
> >   - The down-sides primarily are that we don't auto-configure
> >     perfectly.  This particularly affects hot-plug, but without
> >     hotplug the grouping of cpus and interfaces are focussed
> >     on .... avoiding worst case rather than achieving best case.
>
> Without hotplug the CPT grouping should be tuned to achieve a best
> case in a static configuration.
>
> Adding simple-minded hotplug tolerance (let's not call it support) would
> focus on avoiding truly pathological behaviour.
>
> > I've made up a lot of stuff there.  I'm happy not to pursue this further at the
> > moment, but if anyone would like to enhance my understanding by
> > correcting the worst errors in the above, I wouldn't object :-)
> >
> > Thanks,
> > NeilBrown
>
> PS: the NUMA effects I've mentioned above have been making the news
> lately under other names: they are part of the side channels used in various
> timing based attacks.
>
> Olaf
>
> _______________________________________________
> lustre-devel mailing list
> lustre-devel at lists.lustre.org
> http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org
>
_______________________________________________
lustre-devel mailing list
lustre-devel at lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180706/df9106cf/attachment.html>

From jsimmons at infradead.org  Fri Jul  6 01:36:06 2018
From: jsimmons at infradead.org (James Simmons)
Date: Fri, 6 Jul 2018 02:36:06 +0100 (BST)
Subject: [lustre-devel] [PATCH 01/11] staging: lustre: simplify use of
 interval-tree.
In-Reply-To: <87sh5mfit7.fsf@notabene.neil.brown.name>
References: <152826510267.16761.14361003167157833896.stgit@noble>
 <152826511890.16761.16115276596203531205.stgit@noble>
 <alpine.LFD.2.21.1806160344210.6450@casper.infradead.org>
 <87sh5mfit7.fsf@notabene.neil.brown.name>
Message-ID: <alpine.LFD.2.21.1807060235330.12665@casper.infradead.org>


> >> Lustre has a private interval-tree implementation.  This
> >> implementation (inexplicably) refuses to insert an interval if an
> >> identical interval already exists.  It is OK with all sorts of
> >> overlapping intervals, but identical intervals are rejected.
> >
> > I talked to Oleg about this since this changes the behavior. He is worried
> > about having identical items that would end up being merged.  If we can 
> > guarantee by some other means there are no identical nodes, we are 
> > probably fine with the interval tree code allowing this. Oleg can explain 
> > better than me in this case.
> 
> I don't think this is a change in behaviour.
> In the driver/staging client code, interval tree is being used in two
> places and both of them have clumsy work-arounds for the fact that they
> cannot insert duplicates in the interval tree.
> The patch just cleans his up.
> 
> However if I have missed something, please provide details.
> What "identical items" might get merged?

Oleg could you fill in detail what your concerns are?
 
> >  
> >> Both users of interval-tree in lustre would be simpler if this was not
> >> the case.  They need to store all intervals, even if some are
> >> identical.
> >> 
> >> llite/range_lock.c add a rl_next_lock list_head to each lock.
> >> If it cannot insert a new lock because the range is in use, it
> >> attached the new lock to the existing lock using rl_next_lock.
> >> This requires extra code to iterate over the rl_next_lock lists when
> >> iterating over locks, and to update the list when deleting a lock from
> >> the tree.
> >> 
> >> ldlm_extend allocates a separate ldlm_interval which as a list of
> >> ldlm_locks which share the same interval.  This is linked together
> >> by over-loading the l_sl_policy which, for non-extent locks, is used
> >> for linking together locks with the same policy.
> >> This doesn't only require extra code, but also an extra memory
> >> allocation.
> >> 
> >> This patch removes all that complexity.
> >> - interval_insert() now never fails.
> >
> > Its not really a failure. What it does is if it finds a already existing
> > node with the range requested it returns the already existing node 
> > pointer. If not it just creates a new node and returns NULL. Sometimes
> > identical request can happen. A good example of this is with HSM request
> > on the MDS server. In that case sometimes we get identical progress
> > reports which we want to filter out so not add the same data.
> 
> This example is server-side code which is not a focus at present.
> Having a quick look, it looks like it would be easy enough to do a
> lookup first and then only insert if the lookup failed.
> I think this is a much nicer approach than never allowing duplicates in
> the interval tree.
> 
> Thanks,
> NeilBrown
> 

From neilb at suse.com  Fri Jul  6 03:11:42 2018
From: neilb at suse.com (NeilBrown)
Date: Fri, 06 Jul 2018 13:11:42 +1000
Subject: [lustre-devel] [PATCH v3 07/26] staging: lustre: libcfs: NUMA
 support
In-Reply-To: <alpine.LFD.2.21.1807060052470.12665@casper.infradead.org>
References: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
 <1529875250-11531-8-git-send-email-jsimmons@infradead.org>
 <877emnaed8.fsf@notabene.neil.brown.name>
 <A764DED1-A9A8-4FA5-A0F1-922C3E2D822A@cray.com>
 <87lgb16j8q.fsf@notabene.neil.brown.name>
 <DM3PR1101MB1213FD1001256E565FF011FFCB480@DM3PR1101MB1213.namprd11.prod.outlook.com>
 <8736x77lrk.fsf@notabene.neil.brown.name>
 <9DE389AB-C7E1-4336-B7E8-604581EFD53E@cray.com>
 <CAAqp6i0uwmpdFYUJ7mCBz+MH+gVBa6S=P2t5Yn7wcFOtrTD-Fg@mail.gmail.com>
 <AT5PR8401MB0900C67FADDF0EF7ED1CDBC4854E0@AT5PR8401MB0900.NAMPRD84.PROD.OUTLOOK.COM>
 <87va9vziay.fsf@notabene.neil.brown.name>
 <AT5PR8401MB0900A5406BF0CB8C87B80EEC85410@AT5PR8401MB0900.NAMPRD84.PROD.OUTLOOK.COM>
 <alpine.LFD.2.21.1807060052470.12665@casper.infradead.org>
Message-ID: <87zhz5xdld.fsf@notabene.neil.brown.name>

On Fri, Jul 06 2018, James Simmons wrote:

>> NeilBrown [mailto:neilb at suse.com] wrote:
>> 
>> To help contextualize things: the Lustre code can be decomposed into three parts:
>> 
>> 1) The filesystem proper: Lustre.
>> 2) The communication protocol it uses: LNet.
>> 3) Supporting code used by Lustre and LNet: CFS.
>> 
>> Part of the supporting code is the CPT mechanism, which provides a way to
>> partition the CPUs of a system. These partitions are used to distribute queues,
>> locks, and threads across the system. It was originally introduced years ago, as
>> far as I can tell mainly to deal with certain hot locks: these were converted into
>> read/write locks with one spinlock per CPT.
>> 
>> As a general rule, CPT boundaries should respect node and socket boundaries,
>> but at the higher end, where CPUs have 20+ cores, it may make sense to split
>> a CPUs cores across several CPTs.
>> 
>> > Thanks everyone for your patience in explaining things to me.
>> > I'm beginning to understand what to look for and where to find it.
>> > 
>> > So the answers to Greg's questions:
>> > 
>> >   Where are you reading the host memory NUMA information from?
>> > 
>> >   And why would a filesystem care about this type of thing?  Are you
>> >   going to now mirror what the scheduler does with regards to NUMA
>> >   topology issues?  How are you going to handle things when the topology
>> >   changes?  What systems did you test this on?  What performance
>> >   improvements were seen?  What downsides are there with all of this?
>> > 
>> > 
>> > Are:
>> 
>> >   - NUMA info comes from ACPI or device-tree just like for every one
>> >       else.  Lustre just uses node_distance().
>> 
>> Correct, the standard kernel interfaces for this information are used to
>> obtain it, so ultimately Lustre/LNet uses the same source of truth as
>> everyone else.
>> 
>> >   - The filesystem cares about this because...  It has service
>> >     thread that does part of the work of some filesystem operations
>> >     (handling replies for example) and these are best handled "near"
>> >     the CPU the initiated the request.  Lustre partitions
>> >     all CPUs into "partitions" (cpt) each with a few cores.
>> >     If the request thread and the reply thread are on different
>> >     CPUs but in the same partition, then we get best throughput
>> >     (is that close?)
>> 
>> At the filesystem level, it does indeed seem to help to have the service
>> threads that do work for requests run on a different core that is close to
>> the core that originated the request. So preferably on the same CPU, and
>> on certain multi-core CPUs there are also distance effects between cores.
>> That too is one of the things the CPT mechanism handles.
>
> Their is another very important aspect to why Lustre has a CPU partition 
> layer. At least at the place I work at. While the Linux kernel manages all
> the NUMA nodes and CPU cores Lustre adds the ability for us to specify a 
> subset of everything on the system. The reason is to limit the impact of
> noise on the compute nodes. Noise has a heavy impact on large scale HP
> work loads that can run days or even weeks at a time. Lets take an 
> example system:
>
>                |-------------|     |-------------|
>    |-------|   | NUMA  0     |     | NUMA  1     |   |-------|
>    | eth0  | - |             | --- |             | - | eth1  |      
>    |_______|   | CPU0  CPU1  |     | CPU2  CPU3  |   |_______|
>                |_____________|     |_____________|
>
> In such a system it is possible with the right job schedular to start a 
> large parallel application on NUMA 0/ (CPU0 and CPU1). Normally such
> large parallel applications will communicate between nodes using MPI,
> such as openmpi, which can be configured to use eth0 only. Using the
> CPT layer in lustre we can isolate lustre to NUMA 1 and use only eth1.
> This greatly reducess the noise impact on the application running.
>
> BTW this is one of the reasons ko2iblnd for lustre doesn't use the
> generic RDMA api. The core IB layer doesn't support such isolation.
> At least to my knowledge.

Thanks for that background (and for the separate explanation of how
jitter multiplies when jobs needs to synchronize periodically).

I can see that setting CPU affinity for lustre/lnet worker threads could
be important, and that it can be valuable to tie services to a
particular interface.  I cannot yet see why we need partitions for this,
rather that doing it at the CPU (or NODE) level.

Thanks,
NeilBrown
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180706/b207f4ab/attachment.sig>

From spitzcor at cray.com  Fri Jul  6 04:02:29 2018
From: spitzcor at cray.com (Cory Spitz)
Date: Fri, 6 Jul 2018 04:02:29 +0000
Subject: [lustre-devel] [PATCH 01/18] lustre: grant: add support for
 OBD_CONNECT_GRANT_PARAM
In-Reply-To: <1530573875-20465-2-git-send-email-jsimmons@infradead.org>
References: <1530573875-20465-1-git-send-email-jsimmons@infradead.org>
 <1530573875-20465-2-git-send-email-jsimmons@infradead.org>
Message-ID: <0F41C25D-E055-489A-82B6-CF4728AFF905@cray.com>

James,

On 7/2/18, 6:25 PM, "lustre-devel on behalf of James Simmons" <lustre-devel-bounces at lists.lustre.org on behalf of jsimmons at infradead.org> wrote:

    From: Johann Lombardi <jlombardi at whamcloud.com>
    Signed-off-by: Johann Lombardi <jlombardi at whamcloud.com>
    Signed-off-by: Jinshan Xiong <jinshan.xiong at gmail.com>
    WC-bug-id: https://jira.whamcloud.com/browse/2049
    Reviewed-on: http://review.whamcloud.com/7793
    Reviewed-by: Andreas Dilger <adilger at whamcloud.com>
    Reviewed-by: Nathaniel Clark <nathaniel.l.clark at intel.com>
 
James, can you fix this up?  I'm not sure how historical patches should be attributed, but Johann's address seems wrong.

From should be attributed to Johann Lombardi <johann.lombardi at intel.com> .  Also, the Bug URL should be https://jira.whamcloud.com/browse/LU-2049 .

Thanks,
-Cory


From neilb at suse.com  Fri Jul  6 04:19:41 2018
From: neilb at suse.com (NeilBrown)
Date: Fri, 06 Jul 2018 14:19:41 +1000
Subject: [lustre-devel] [PATCH v3 04/13] lustre: libcfs: fix
 cfs_print_to_console()
In-Reply-To: <alpine.LFD.2.21.1807060034020.12665@casper.infradead.org>
References: <1530128322-32535-1-git-send-email-jsimmons@infradead.org>
 <1530128322-32535-5-git-send-email-jsimmons@infradead.org>
 <FD87D2B8-27A8-45DF-9C7C-BA33CF940C1D@whamcloud.com>
 <alpine.LFD.2.21.1806291716050.26339@casper.infradead.org>
 <87h8li2wz7.fsf@notabene.neil.brown.name>
 <alpine.LFD.2.21.1807060034020.12665@casper.infradead.org>
Message-ID: <87tvpdxag2.fsf@notabene.neil.brown.name>

On Fri, Jul 06 2018, James Simmons wrote:

>> >> > The original code for cfs_print_to_console() used printk() and
>> >> > used tricks to select which printk level to use. Later a cleanup
>> >> > patch landed the just converted printk directly to pr_info which
>> >> > is not exactly correct. Instead of converting back to printk lets
>> >> > move everything to pr_* type functions which simplify the code.
>> >> > This allows us to fold both dbghdr_to_err_string() and the
>> >> > function dbghdr_to_info_string() into cfs_print_to_console().
>> >> > 
>> >> > Signed-off-by: James Simmons <jsimmons at infradead.org>
>> >> > ---
>> >> > .../staging/lustre/lnet/libcfs/linux-tracefile.c   | 55 ++++++----------------
>> >> > 1 file changed, 14 insertions(+), 41 deletions(-)
>> >> > 
>> >> > diff --git a/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c b/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
>> >> > index 3af7722..c1747c4 100644
>> >> > --- a/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
>> >> > +++ b/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
>> >> > @@ -140,54 +140,27 @@ enum cfs_trace_buf_type cfs_trace_buf_idx_get(void)
>> >> > void cfs_print_to_console(struct ptldebug_header *hdr, int mask,
>> >> > 			  const char *buf, int len, const char *file,
>> >> > 			  const char *fn)
>> >> > {
>> >> > +	char *prefix = "Lustre";
>> >> > +
>> >> > +	if (hdr->ph_subsys == S_LND || hdr->ph_subsys == S_LNET)
>> >> > +		prefix = "LNet";
>> >> > 
>> >> > -	if (mask & D_EMERG) {
>> >> > -		prefix = dbghdr_to_err_string(hdr);
>> >> > -		ptype = KERN_EMERG;
>> >> > +	if (mask & (D_CONSOLE | libcfs_printk)) {
>> >> 
>> >> This check is broken.  The default value of libcfs_printk (the mask
>> >> that controls which messages should be printed to the console, and
>> >> which ones should only be logged into the internal buffer) is:
>> >> 
>> >>     #define D_CANTMASK (D_ERROR | D_EMERG | D_WARNING | D_CONSOLE)
>> >> 
>> >> so that means virtually every console message will be printed with
>> >> pr_info() because this is matched first, instead of pr_emerg() or
>> >> pr_err() below.
>> >> 
>> >> That is why the previous code was matching D_EMERG and D_ERROR first,
>> >> then D_WARNING, and (D_CONSOLE | libcfs_printk) at the end.
>> >
>> > So to do this right we need:
>> >
>> > static void cfs_print_to_console(struct ptldebug_header *hdr, int mask,
>> >                                  const char *buf, int len, const char 
>> > *file,
>> >                                  const char *fn)
>> > {
>> >         char *prefix = "Lustre";
>> >
>> >         if (hdr->ph_subsys == S_LND || hdr->ph_subsys == S_LNET)
>> >                 prefix = "LNet";
>> >
>> >         if (mask & D_CONSOLE) {
>> >                 if (mask & D_EMERG) {
>> >                         pr_emerg("%sError: %.*s", prefix, len, buf);
>> >                 } else if (mask & D_ERROR) {
>> >                         pr_err("%sError: %.*s", prefix, len, buf);
>> >                 } else if (mask & D_WARNING) {
>> >                         pr_warn("%s: %.*s", prefix, len, buf);
>> >                 } else if (mask & libcfs_printk) {
>> >                         pr_info("%s: %.*s", prefix, len, buf);
>> >                 }
>> >         } else {
>> >                 if (mask & D_EMERG) {
>> >                         pr_emerg("%sError: %d:%d:(%s:%d:%s()) %.*s", prefix,
>> >                                  hdr->ph_pid, hdr->ph_extern_pid, file,
>> >                                  hdr->ph_line_num, fn, len, buf);
>> >                 } else if (mask & D_ERROR) {
>> >                         pr_err("%sError: %d:%d:(%s:%d:%s()) %.*s", prefix,
>> >                                hdr->ph_pid, hdr->ph_extern_pid, file,
>> >                                hdr->ph_line_num, fn, len, buf);
>> >                 } else if (mask & D_WARNING) {
>> >                         pr_warn("%s: %d:%d:(%s:%d:%s()) %.*s", prefix,
>> >                                 hdr->ph_pid, hdr->ph_extern_pid, file,
>> >                                 hdr->ph_line_num, fn, len, buf);
>> >                 } else if (mask & libcfs_printk) {
>> >                         pr_info("%s: %.*s", prefix, len, buf);
>> >                 }
>> >         }
>> > }
>> 
>> That doesn't look right either.
>> The original code would *always* print something.
>> This code looks like it might not (e.g. for mask == 0).
>
> Is that correct behavior? A mask of zero means don't do anything.
> Also if you look at the original in the OpenSFS branch you can see
> if mask was to be 0 then ptype would be NULl :-( 

True... so it would print something, but the something would have (null)
in it.  So it probably never happens.

>
> Note a D_CANT_MASK prevents some fields in the mask from being disabled.
>
>> What is "D_CONSOLE" suppose to mean?  It seems to me "make the messages
>> less verbose" and it isn't clear to me that what is called "D_CONSOLE".
>
> It means two things. If by itself then it means use pr_info. If it is one
> field in the mask then it means less detail. Confusing no :-( That is my
> understanding of it. Maybe someone else can clarify if I'm wrong.

Yes, "confusing" seems an accurate description.  But if that is what it
is, then that is what it is.
If you send you new version as a patch, either against the original or
against the buggy version, I'll apply it.

Thanks,
NeilBrown
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180706/2cadd2aa/attachment.sig>

From doucharek at cray.com  Fri Jul  6 05:36:03 2018
From: doucharek at cray.com (Doug Oucharek)
Date: Fri, 6 Jul 2018 05:36:03 +0000
Subject: [lustre-devel] [PATCH v3 07/26] staging: lustre: libcfs: NUMA
 support
In-Reply-To: <87zhz5xdld.fsf@notabene.neil.brown.name>
References: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
 <1529875250-11531-8-git-send-email-jsimmons@infradead.org>
 <877emnaed8.fsf@notabene.neil.brown.name>
 <A764DED1-A9A8-4FA5-A0F1-922C3E2D822A@cray.com>
 <87lgb16j8q.fsf@notabene.neil.brown.name>
 <DM3PR1101MB1213FD1001256E565FF011FFCB480@DM3PR1101MB1213.namprd11.prod.outlook.com>
 <8736x77lrk.fsf@notabene.neil.brown.name>
 <9DE389AB-C7E1-4336-B7E8-604581EFD53E@cray.com>
 <CAAqp6i0uwmpdFYUJ7mCBz+MH+gVBa6S=P2t5Yn7wcFOtrTD-Fg@mail.gmail.com>
 <AT5PR8401MB0900C67FADDF0EF7ED1CDBC4854E0@AT5PR8401MB0900.NAMPRD84.PROD.OUTLOOK.COM>
 <87va9vziay.fsf@notabene.neil.brown.name>
 <AT5PR8401MB0900A5406BF0CB8C87B80EEC85410@AT5PR8401MB0900.NAMPRD84.PROD.OUTLOOK.COM>
 <alpine.LFD.2.21.1807060052470.12665@casper.infradead.org>
 <87zhz5xdld.fsf@notabene.neil.brown.name>
Message-ID: <02B7B718-E496-46CF-B55D-86EAB98381B6@cray.com>

When the CPT code was added to LNet back in 2012, it was to address one primary case: a need for finer grained locking on metadata servers.  LNet used to have global locks and metadata servers, which do many small messages (high IOPS), much time in the worker threads was spent in spinlocks.  So, CPT configuration was added so locks/resources could be allocated per CPT.  This way, users have control over how they want CPTs to be configured and how they want resources/locks to be divided.  For example, users may want finer grained locking on the metadata servers but not on clients.  Leaving this to be automatically configured by Linux API calls would take this flexibility away from the users who, for HPC, are very knowledgable about what they want (i.e. we do not want to protect them from themselves).

The CPT support in LNet and LNDs has morphed to encompass more traditional NUMA and core affinity performance improvements.  For example, you can restrict a network interface to a socket (NUMA node) which has better affinity to the PCIe lanes that interface is connected to.  Rather than try to do this sort of thing automatically, we have left it to the user to know what they are doing and configure the CPTs accordingly.

I think the many changes to the CPT code has realty clouded its purpose.  In summary, the original purpose was finer grained locking and that needs to be maintained as the IOPS requirements of metadata servers is paramount.

James: The Verbs RDMA interface has very poor support for NUMA/core affinity.  I was going to try to devise some patches to address that but have been too busy on other things.  Perhaps the RDMA maintainer could consider updating it?

Doug

On Jul 5, 2018, at 8:11 PM, NeilBrown <neilb at suse.com<mailto:neilb at suse.com>> wrote:

On Fri, Jul 06 2018, James Simmons wrote:

NeilBrown [mailto:neilb at suse.com] wrote:

To help contextualize things: the Lustre code can be decomposed into three parts:

1) The filesystem proper: Lustre.
2) The communication protocol it uses: LNet.
3) Supporting code used by Lustre and LNet: CFS.

Part of the supporting code is the CPT mechanism, which provides a way to
partition the CPUs of a system. These partitions are used to distribute queues,
locks, and threads across the system. It was originally introduced years ago, as
far as I can tell mainly to deal with certain hot locks: these were converted into
read/write locks with one spinlock per CPT.

As a general rule, CPT boundaries should respect node and socket boundaries,
but at the higher end, where CPUs have 20+ cores, it may make sense to split
a CPUs cores across several CPTs.

Thanks everyone for your patience in explaining things to me.
I'm beginning to understand what to look for and where to find it.

So the answers to Greg's questions:

 Where are you reading the host memory NUMA information from?

 And why would a filesystem care about this type of thing?  Are you
 going to now mirror what the scheduler does with regards to NUMA
 topology issues?  How are you going to handle things when the topology
 changes?  What systems did you test this on?  What performance
 improvements were seen?  What downsides are there with all of this?


Are:

 - NUMA info comes from ACPI or device-tree just like for every one
     else.  Lustre just uses node_distance().

Correct, the standard kernel interfaces for this information are used to
obtain it, so ultimately Lustre/LNet uses the same source of truth as
everyone else.

 - The filesystem cares about this because...  It has service
   thread that does part of the work of some filesystem operations
   (handling replies for example) and these are best handled "near"
   the CPU the initiated the request.  Lustre partitions
   all CPUs into "partitions" (cpt) each with a few cores.
   If the request thread and the reply thread are on different
   CPUs but in the same partition, then we get best throughput
   (is that close?)

At the filesystem level, it does indeed seem to help to have the service
threads that do work for requests run on a different core that is close to
the core that originated the request. So preferably on the same CPU, and
on certain multi-core CPUs there are also distance effects between cores.
That too is one of the things the CPT mechanism handles.

Their is another very important aspect to why Lustre has a CPU partition
layer. At least at the place I work at. While the Linux kernel manages all
the NUMA nodes and CPU cores Lustre adds the ability for us to specify a
subset of everything on the system. The reason is to limit the impact of
noise on the compute nodes. Noise has a heavy impact on large scale HP
work loads that can run days or even weeks at a time. Lets take an
example system:

              |-------------|     |-------------|
  |-------|   | NUMA  0     |     | NUMA  1     |   |-------|
  | eth0  | - |             | --- |             | - | eth1  |
  |_______|   | CPU0  CPU1  |     | CPU2  CPU3  |   |_______|
              |_____________|     |_____________|

In such a system it is possible with the right job schedular to start a
large parallel application on NUMA 0/ (CPU0 and CPU1). Normally such
large parallel applications will communicate between nodes using MPI,
such as openmpi, which can be configured to use eth0 only. Using the
CPT layer in lustre we can isolate lustre to NUMA 1 and use only eth1.
This greatly reducess the noise impact on the application running.

BTW this is one of the reasons ko2iblnd for lustre doesn't use the
generic RDMA api. The core IB layer doesn't support such isolation.
At least to my knowledge.

Thanks for that background (and for the separate explanation of how
jitter multiplies when jobs needs to synchronize periodically).

I can see that setting CPU affinity for lustre/lnet worker threads could
be important, and that it can be valuable to tie services to a
particular interface.  I cannot yet see why we need partitions for this,
rather that doing it at the CPU (or NODE) level.

Thanks,
NeilBrown

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180706/ba357d68/attachment.html>

From neilb at suse.com  Fri Jul  6 06:13:31 2018
From: neilb at suse.com (NeilBrown)
Date: Fri, 06 Jul 2018 16:13:31 +1000
Subject: [lustre-devel] [PATCH v3 07/26] staging: lustre: libcfs: NUMA
 support
In-Reply-To: <02B7B718-E496-46CF-B55D-86EAB98381B6@cray.com>
References: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
 <1529875250-11531-8-git-send-email-jsimmons@infradead.org>
 <877emnaed8.fsf@notabene.neil.brown.name>
 <A764DED1-A9A8-4FA5-A0F1-922C3E2D822A@cray.com>
 <87lgb16j8q.fsf@notabene.neil.brown.name>
 <DM3PR1101MB1213FD1001256E565FF011FFCB480@DM3PR1101MB1213.namprd11.prod.outlook.com>
 <8736x77lrk.fsf@notabene.neil.brown.name>
 <9DE389AB-C7E1-4336-B7E8-604581EFD53E@cray.com>
 <CAAqp6i0uwmpdFYUJ7mCBz+MH+gVBa6S=P2t5Yn7wcFOtrTD-Fg@mail.gmail.com>
 <AT5PR8401MB0900C67FADDF0EF7ED1CDBC4854E0@AT5PR8401MB0900.NAMPRD84.PROD.OUTLOOK.COM>
 <87va9vziay.fsf@notabene.neil.brown.name>
 <AT5PR8401MB0900A5406BF0CB8C87B80EEC85410@AT5PR8401MB0900.NAMPRD84.PROD.OUTLOOK.COM>
 <alpine.LFD.2.21.1807060052470.12665@casper.infradead.org>
 <87zhz5xdld.fsf@notabene.neil.brown.name>
 <02B7B718-E496-46CF-B55D-86EAB98381B6@cray.com>
Message-ID: <87o9fkyjqs.fsf@notabene.neil.brown.name>

On Fri, Jul 06 2018, Doug Oucharek wrote:

> When the CPT code was added to LNet back in 2012, it was to address
> one primary case: a need for finer grained locking on metadata
> servers.  LNet used to have global locks and metadata servers, which
> do many small messages (high IOPS), much time in the worker threads
> was spent in spinlocks.  So, CPT configuration was added so
> locks/resources could be allocated per CPT.  This way, users have
> control over how they want CPTs to be configured and how they want
> resources/locks to be divided.  For example, users may want finer
> grained locking on the metadata servers but not on clients.  Leaving
> this to be automatically configured by Linux API calls would take this
> flexibility away from the users who, for HPC, are very knowledgable
> about what they want (i.e. we do not want to protect them from
> themselves).
>
> The CPT support in LNet and LNDs has morphed to encompass more
> traditional NUMA and core affinity performance improvements.  For
> example, you can restrict a network interface to a socket (NUMA node)
> which has better affinity to the PCIe lanes that interface is
> connected to.  Rather than try to do this sort of thing automatically,
> we have left it to the user to know what they are doing and configure
> the CPTs accordingly.
>
> I think the many changes to the CPT code has realty clouded its
> purpose.  In summary, the original purpose was finer grained locking
> and that needs to be maintained as the IOPS requirements of metadata
> servers is paramount.

Thanks for the explanation.
I definitely get that fine-grained locking is a good thing.  Lustre is
not alone in this of course.
Even better than fine-grained locking is no locking.  That is not often
possible, but this
  https://github.com/neilbrown/linux/commit/ac3f8fd6e61b245fa9c14e3164203c1211c5ef6b

is an example of doing exactly that.

For the read/writer usage of CPT locks, RCU is a better approach if it
can be made to work (usually it can) - and it scales even better.

When I was digging through the usage of locks I saw some hash tables.
It seems that a lock protected a whole table.  It is usually sufficient
for the lock to just protect a single chain (bit spin-locks can easily
store one lock per chain) and then only for writes - RCU discipline can
allow reads to proceed with only rcu_read_lock().
Would we still need per-CPT tables once that was in place?  I don't know
yet, though per-node seems likely to be sufficient when locking is per-chain.

I certainly wouldn't discard CPTs without replacing them with something
better.  Near the top of my list for when I return from vacation
(leaving in a couple of days) will be to look closely at the current
fine-grained locking that you have helped me to see more clearly, and
see if I can make it even better.

Thanks,
NeilBrown

>
> James: The Verbs RDMA interface has very poor support for NUMA/core affinity.  I was going to try to devise some patches to address that but have been too busy on other things.  Perhaps the RDMA maintainer could consider updating it?
>
> Doug
>
> On Jul 5, 2018, at 8:11 PM, NeilBrown <neilb at suse.com<mailto:neilb at suse.com>> wrote:
>
> On Fri, Jul 06 2018, James Simmons wrote:
>
> NeilBrown [mailto:neilb at suse.com] wrote:
>
> To help contextualize things: the Lustre code can be decomposed into three parts:
>
> 1) The filesystem proper: Lustre.
> 2) The communication protocol it uses: LNet.
> 3) Supporting code used by Lustre and LNet: CFS.
>
> Part of the supporting code is the CPT mechanism, which provides a way to
> partition the CPUs of a system. These partitions are used to distribute queues,
> locks, and threads across the system. It was originally introduced years ago, as
> far as I can tell mainly to deal with certain hot locks: these were converted into
> read/write locks with one spinlock per CPT.
>
> As a general rule, CPT boundaries should respect node and socket boundaries,
> but at the higher end, where CPUs have 20+ cores, it may make sense to split
> a CPUs cores across several CPTs.
>
> Thanks everyone for your patience in explaining things to me.
> I'm beginning to understand what to look for and where to find it.
>
> So the answers to Greg's questions:
>
>  Where are you reading the host memory NUMA information from?
>
>  And why would a filesystem care about this type of thing?  Are you
>  going to now mirror what the scheduler does with regards to NUMA
>  topology issues?  How are you going to handle things when the topology
>  changes?  What systems did you test this on?  What performance
>  improvements were seen?  What downsides are there with all of this?
>
>
> Are:
>
>  - NUMA info comes from ACPI or device-tree just like for every one
>      else.  Lustre just uses node_distance().
>
> Correct, the standard kernel interfaces for this information are used to
> obtain it, so ultimately Lustre/LNet uses the same source of truth as
> everyone else.
>
>  - The filesystem cares about this because...  It has service
>    thread that does part of the work of some filesystem operations
>    (handling replies for example) and these are best handled "near"
>    the CPU the initiated the request.  Lustre partitions
>    all CPUs into "partitions" (cpt) each with a few cores.
>    If the request thread and the reply thread are on different
>    CPUs but in the same partition, then we get best throughput
>    (is that close?)
>
> At the filesystem level, it does indeed seem to help to have the service
> threads that do work for requests run on a different core that is close to
> the core that originated the request. So preferably on the same CPU, and
> on certain multi-core CPUs there are also distance effects between cores.
> That too is one of the things the CPT mechanism handles.
>
> Their is another very important aspect to why Lustre has a CPU partition
> layer. At least at the place I work at. While the Linux kernel manages all
> the NUMA nodes and CPU cores Lustre adds the ability for us to specify a
> subset of everything on the system. The reason is to limit the impact of
> noise on the compute nodes. Noise has a heavy impact on large scale HP
> work loads that can run days or even weeks at a time. Lets take an
> example system:
>
>               |-------------|     |-------------|
>   |-------|   | NUMA  0     |     | NUMA  1     |   |-------|
>   | eth0  | - |             | --- |             | - | eth1  |
>   |_______|   | CPU0  CPU1  |     | CPU2  CPU3  |   |_______|
>               |_____________|     |_____________|
>
> In such a system it is possible with the right job schedular to start a
> large parallel application on NUMA 0/ (CPU0 and CPU1). Normally such
> large parallel applications will communicate between nodes using MPI,
> such as openmpi, which can be configured to use eth0 only. Using the
> CPT layer in lustre we can isolate lustre to NUMA 1 and use only eth1.
> This greatly reducess the noise impact on the application running.
>
> BTW this is one of the reasons ko2iblnd for lustre doesn't use the
> generic RDMA api. The core IB layer doesn't support such isolation.
> At least to my knowledge.
>
> Thanks for that background (and for the separate explanation of how
> jitter multiplies when jobs needs to synchronize periodically).
>
> I can see that setting CPU affinity for lustre/lnet worker threads could
> be important, and that it can be valuable to tie services to a
> particular interface.  I cannot yet see why we need partitions for this,
> rather that doing it at the CPU (or NODE) level.
>
> Thanks,
> NeilBrown
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180706/e1e273c6/attachment.sig>

From c17817 at cray.com  Wed Jul  4 17:54:35 2018
From: c17817 at cray.com (Alexey Lyashkov)
Date: Wed, 4 Jul 2018 17:54:35 +0000
Subject: [lustre-devel] [PATCH 00/24] lustre - more cleanups including
 module reduction.
In-Reply-To: <5993F8BE-E55C-4E75-BB1B-E7C25B79D73A@cray.com>
References: <152904663333.10587.10934053155404014785.stgit@noble>
 <alpine.LFD.2.21.1806210239190.3886@casper.infradead.org>
 <DM3PR1101MB1213DDB7FC857EE6B1444153CB760@DM3PR1101MB1213.namprd11.prod.outlook.com>
 <alpine.LFD.2.21.1806210354260.3886@casper.infradead.org>
 <3FD4D051-9C95-449D-8A23-D42B271E55B8@dilger.ca>
 <DM3PR1101MB121322E85FDE1E5310F6F4B7CB760@DM3PR1101MB1213.namprd11.prod.outlook.com>
 <87tvprah32.fsf@notabene.neil.brown.name>
 <alpine.LFD.2.21.1806260206360.31196@casper.infradead.org>
 <DM3PR1101MB121358672276198B843DE5A7CB490@DM3PR1101MB1213.namprd11.prod.outlook.com>
 <87d0wd6i4m.fsf@notabene.neil.brown.name>
 <44633E99-07CB-4C40-967C-0B4FE206F29A@whamcloud.com>
 <DM3PR1101MB12132AC3E33E7D7C8FBE4536CB480@DM3PR1101MB1213.namprd11.prod.outlook.com>
 <87tvpn6592.fsf@notabene.neil.brown.name>
 <E70889E4-794F-495D-B40C-97BA4F05EBB0@cray.com>
 <5993F8BE-E55C-4E75-BB1B-E7C25B79D73A@cray.com>
Message-ID: <7FEEDB12-2000-4399-BB5F-573F64A14D0E@cray.com>

Doug,

What about LNet API change to use sge list? It's more natively  structure for variable page list, which is may a short sized for MDC links and full sized for OST links.

On 28/06/2018, 20:04, "Doug Oucharek" <doucharek at cray.com> wrote:

    Yes, Al Viro had started the process of getting Lustre in the upstream staging area to use iov_iter and, I believe, biovec.  A very good change in my opinion as it gets rid of the duality.  Unfortunately, those changes have not made their way into the Whamcloud tree yet so there is a disconnect when working between the two trees.  James Simmons might know more, but I don’t believe Al’s changes are complete so there is still a need to support iovec and kvec.  Could we replace these two with biovecs?  James?
    
    As Cory says, lets make the right architectural decisions and let companies (Whamcloud, Cray, etc) absorb the improvements.  We can adapt DVS as required.
    
    As for LNet as a module: please keep it as a separate module.  Merging it into Lustre would create a dependency for DVS to have Lustre installed and possibly running.  This would also stand in the way of any potential future users of LNet.  Andreas pointed out one, Zest (search for LNET in this paper: https://www.psc.edu/images/zest/zest-pdsw08-paper.pdf), but there may be others developing.  LNet could become its own project at some point because of multiple users.
    
    Doug
     
    > On Jun 28, 2018, at 8:03 AM, Cory Spitz <spitzcor at cray.com> wrote:
    > 
    > I know that Doug follows this list closely, but I'm going to flag him here directly, because I know that he has an opinion about kiov / iovec (and a 3rd if I recall correctly).
    > 
    > Doug told me that (possibly) Al Viro had a proposal or submitted code to simply the kiov usage and that Doug was in favor of it.  Let's choose to make the right architectural call for Lustre/LNet and let Cray worry about keeping our out-of-tree code working.  As Patrick said, Cray pays the price of keeping DVS off to the side (we're wising up, which is why it is now GPL... but don't expect us to publish it on the Internet necessarily... baby-steps).
    > 
    > Doug, can you weigh-in on this case?
    > 
    > Thanks,
    > -Cory
    > 
    > -- 
    > 
    > On 6/27/18, 8:59 PM, "lustre-devel on behalf of NeilBrown" <lustre-devel-bounces at lists.lustre.org on behalf of neilb at suse.com> wrote:
    > 
    >    On Wed, Jun 27 2018, Patrick Farrell wrote:
    > 
    >> Neil,
    >> 
    >> We do indeed have such functionality (it’s called DVS and it’s
    >> basically a high speed file system projection framework, ala NFS but
    >> faster), so the ability to build lnet separately is valuable to us.
    >> While it is being open sourced under the GPL, I don’t think there’s
    >> any intention to try to upstream it.  The current code isn’t even
    >> usable off of Cray systems as it depends on info from user space (that
    >> is provided, in the end, from Cray proprietary hardware) to keep its
    >> connection/routing tables up to date.  That’s supposedly in the
    >> pipeline to get fixed, but it’s still pretty far from generally
    >> usable.
    >> 
    >> But we’d still really appreciate it if lnet stayed separate.  Don’t
    >> know if that’s enough for you - I know sometimes *small* stuff is done
    >> for out of tree users.  Hopefully this meets that standard.
    >> 
    > 
    >    Ahh - DVS.  That answers a question I just asked in another email.
    >    My google-skills don't seem to be up to locating the source code though
    >    :-(
    > 
    >    While I wouldn't knowingly break an interface used by some out-of-tree
    >    code without good reason, it is hard to avoid if you don't know what the
    >    out-of-tree code does.  It can be very tempting to remove something that
    >    isn't being used, but that can certainly hurt out-of-tree code
    >    sometimes.
    > 
    >    A particular example I'm exploring at present is the dual data paths in
    >    LNet.  Or maybe it is dual types of Memory Descriptors.
    >    There is 'kiov' which uses kernel-virtual addresses and 'iovec' which
    >    uses page+offset.
    >    The kiov option isn't used in the client code and it seems likely that
    >    the server-side code could be converted to use iovec without problems.
    > 
    >    I'd like to remove the kiov as I wouldn't be able to justify its
    >    existence when submitting the client-only code upstream.  But I don't
    >    want to remove the option of having an alternate MD type if it really is
    >    significantly more efficient in some context.
    >    If I know whether DVS used kiov or iovec - and in what way - that would
    >    help me to know if I might break something, and to be able to assess the
    >    cost.
    > 
    >    In my mind, the "standard" that you mention is always about
    >    practicality.   Code needs to be maintainable - easy to understand and
    >    hard to break.  If the LNet interface is clean and well documented in
    >    the kernel, then I don't see why we would not at least attempt to
    >    preserve it.
    > 
    >    Thanks,
    >    NeilBrown
    > 
    > 
    
    


From jsimmons at infradead.org  Fri Jul  6 14:28:14 2018
From: jsimmons at infradead.org (James Simmons)
Date: Fri, 6 Jul 2018 15:28:14 +0100 (BST)
Subject: [lustre-devel] [PATCH 01/18] lustre: grant: add support for
 OBD_CONNECT_GRANT_PARAM
In-Reply-To: <0F41C25D-E055-489A-82B6-CF4728AFF905@cray.com>
References: <1530573875-20465-1-git-send-email-jsimmons@infradead.org>
 <1530573875-20465-2-git-send-email-jsimmons@infradead.org>
 <0F41C25D-E055-489A-82B6-CF4728AFF905@cray.com>
Message-ID: <alpine.LFD.2.21.1807061523210.15728@casper.infradead.org>


> On 7/2/18, 6:25 PM, "lustre-devel on behalf of James Simmons" <lustre-devel-bounces at lists.lustre.org on behalf of jsimmons at infradead.org> wrote:
> 
>     From: Johann Lombardi <jlombardi at whamcloud.com>
>     Signed-off-by: Johann Lombardi <jlombardi at whamcloud.com>
>     Signed-off-by: Jinshan Xiong <jinshan.xiong at gmail.com>
>     WC-bug-id: https://jira.whamcloud.com/browse/2049
>     Reviewed-on: http://review.whamcloud.com/7793
>     Reviewed-by: Andreas Dilger <adilger at whamcloud.com>
>     Reviewed-by: Nathaniel Clark <nathaniel.l.clark at intel.com>
>  
> James, can you fix this up?  I'm not sure how historical patches should be attributed, but Johann's address seems
> wrong.
> 
> From should be attributed to Johann Lombardi <johann.lombardi at intel.com> .  Also, the Bug URL should be
> https://jira.whamcloud.com/browse/LU-2049 .

This is tricky one. The reason I updated the email address to avoid 
bouncing emails. Also this allows the original author the change to
still reply. Does that seem reasonable? It is still in the air for
some people what their email address is to me. Ah you are right I
dropped the s. Its should be https://jira.whamcloud.com/....

From jsimmons at infradead.org  Fri Jul  6 14:29:24 2018
From: jsimmons at infradead.org (James Simmons)
Date: Fri, 6 Jul 2018 15:29:24 +0100 (BST)
Subject: [lustre-devel] [PATCH 01/18] lustre: grant: add support for
 OBD_CONNECT_GRANT_PARAM
In-Reply-To: <0F41C25D-E055-489A-82B6-CF4728AFF905@cray.com>
References: <1530573875-20465-1-git-send-email-jsimmons@infradead.org>
 <1530573875-20465-2-git-send-email-jsimmons@infradead.org>
 <0F41C25D-E055-489A-82B6-CF4728AFF905@cray.com>
Message-ID: <alpine.LFD.2.21.1807061528170.15728@casper.infradead.org>


> On 7/2/18, 6:25 PM, "lustre-devel on behalf of James Simmons" <lustre-devel-bounces at lists.lustre.org on behalf of jsimmons at infradead.org> wrote:
> 
>     From: Johann Lombardi <jlombardi at whamcloud.com>
>     Signed-off-by: Johann Lombardi <jlombardi at whamcloud.com>
>     Signed-off-by: Jinshan Xiong <jinshan.xiong at gmail.com>
>     WC-bug-id: https://jira.whamcloud.com/browse/2049
>     Reviewed-on: http://review.whamcloud.com/7793
>     Reviewed-by: Andreas Dilger <adilger at whamcloud.com>
>     Reviewed-by: Nathaniel Clark <nathaniel.l.clark at intel.com>
>  
> James, can you fix this up?  I'm not sure how historical patches should be attributed, but Johann's address seems
> wrong.
> 
> From should be attributed to Johann Lombardi <johann.lombardi at intel.com> .  Also, the Bug URL should be
> https://jira.whamcloud.com/browse/LU-2049 .

This is tricky one. The reason I updated the email address to avoid 
bouncing emails. Also this allows the original author the chance to
still reply. Does that seem reasonable? It is still in the air for
some people what their email address is to me. Ah you are right I
dropped the s. Its should be https://jira.whamcloud.com/....

From jsimmons at infradead.org  Fri Jul  6 14:33:47 2018
From: jsimmons at infradead.org (James Simmons)
Date: Fri, 6 Jul 2018 15:33:47 +0100 (BST)
Subject: [lustre-devel] New slack chat room for linux client work
Message-ID: <alpine.LFD.2.21.1807061531540.15728@casper.infradead.org>


Some people have asked for a chat room since it can be difficult
to filter between the mixture of patches and technical discusssions
going on here. If people are interested to join the site is below

https://lustreupstreamclient.slack.com/messages/CAV4QF0ER/

From adilger at whamcloud.com  Fri Jul  6 15:12:38 2018
From: adilger at whamcloud.com (Andreas Dilger)
Date: Fri, 6 Jul 2018 15:12:38 +0000
Subject: [lustre-devel] [PATCH 01/18] lustre: grant: add support for
 OBD_CONNECT_GRANT_PARAM
In-Reply-To: <alpine.LFD.2.21.1807061528170.15728@casper.infradead.org>
References: <1530573875-20465-1-git-send-email-jsimmons@infradead.org>
 <1530573875-20465-2-git-send-email-jsimmons@infradead.org>
 <0F41C25D-E055-489A-82B6-CF4728AFF905@cray.com>,
 <alpine.LFD.2.21.1807061528170.15728@casper.infradead.org>
Message-ID: <F517C66D-B066-47CC-80B4-A694E811D0E2@whamcloud.com>

And Johann is still at Intel...

Cheers, Andreas

> On Jul 6, 2018, at 08:30, James Simmons <jsimmons at infradead.org> wrote:
> 
> 
>> On 7/2/18, 6:25 PM, "lustre-devel on behalf of James Simmons" <lustre-devel-bounces at lists.lustre.org on behalf of jsimmons at infradead.org> wrote:
>> 
>>    From: Johann Lombardi <jlombardi at whamcloud.com>
>>    Signed-off-by: Johann Lombardi <jlombardi at whamcloud.com>
>>    Signed-off-by: Jinshan Xiong <jinshan.xiong at gmail.com>
>>    WC-bug-id: https://jira.whamcloud.com/browse/2049
>>    Reviewed-on: http://review.whamcloud.com/7793
>>    Reviewed-by: Andreas Dilger <adilger at whamcloud.com>
>>    Reviewed-by: Nathaniel Clark <nathaniel.l.clark at intel.com>
>> 
>> James, can you fix this up?  I'm not sure how historical patches should be attributed, but Johann's address seems
>> wrong.
>> 
>> From should be attributed to Johann Lombardi <johann.lombardi at intel.com> .  Also, the Bug URL should be
>> https://jira.whamcloud.com/browse/LU-2049 .
> 
> This is tricky one. The reason I updated the email address to avoid 
> bouncing emails. Also this allows the original author the chance to
> still reply. Does that seem reasonable? It is still in the air for
> some people what their email address is to me. Ah you are right I
> dropped the s. Its should be https://jira.whamcloud.com/....

From jsimmons at infradead.org  Fri Jul  6 15:50:06 2018
From: jsimmons at infradead.org (James Simmons)
Date: Fri, 6 Jul 2018 16:50:06 +0100 (BST)
Subject: [lustre-devel] [PATCH 01/18] lustre: grant: add support for
 OBD_CONNECT_GRANT_PARAM
In-Reply-To: <F517C66D-B066-47CC-80B4-A694E811D0E2@whamcloud.com>
References: <1530573875-20465-1-git-send-email-jsimmons@infradead.org>
 <1530573875-20465-2-git-send-email-jsimmons@infradead.org>
 <0F41C25D-E055-489A-82B6-CF4728AFF905@cray.com>,
 <alpine.LFD.2.21.1807061528170.15728@casper.infradead.org>
 <F517C66D-B066-47CC-80B4-A694E811D0E2@whamcloud.com>
Message-ID: <alpine.LFD.2.21.1807061649340.15728@casper.infradead.org>


> And Johann is still at Intel...

Its hard to tell. Is this true for WangDi and Fan Young as well?

> > On Jul 6, 2018, at 08:30, James Simmons <jsimmons at infradead.org> wrote:
> > 
> > 
> >> On 7/2/18, 6:25 PM, "lustre-devel on behalf of James Simmons" <lustre-devel-bounces at lists.lustre.org on behalf of jsimmons at infradead.org> wrote:
> >> 
> >>    From: Johann Lombardi <jlombardi at whamcloud.com>
> >>    Signed-off-by: Johann Lombardi <jlombardi at whamcloud.com>
> >>    Signed-off-by: Jinshan Xiong <jinshan.xiong at gmail.com>
> >>    WC-bug-id: https://jira.whamcloud.com/browse/2049
> >>    Reviewed-on: http://review.whamcloud.com/7793
> >>    Reviewed-by: Andreas Dilger <adilger at whamcloud.com>
> >>    Reviewed-by: Nathaniel Clark <nathaniel.l.clark at intel.com>
> >> 
> >> James, can you fix this up?  I'm not sure how historical patches should be attributed, but Johann's address seems
> >> wrong.
> >> 
> >> From should be attributed to Johann Lombardi <johann.lombardi at intel.com> .  Also, the Bug URL should be
> >> https://jira.whamcloud.com/browse/LU-2049 .
> > 
> > This is tricky one. The reason I updated the email address to avoid 
> > bouncing emails. Also this allows the original author the chance to
> > still reply. Does that seem reasonable? It is still in the air for
> > some people what their email address is to me. Ah you are right I
> > dropped the s. Its should be https://jira.whamcloud.com/....
> 

From jsimmons at infradead.org  Fri Jul  6 15:57:00 2018
From: jsimmons at infradead.org (James Simmons)
Date: Fri, 6 Jul 2018 16:57:00 +0100 (BST)
Subject: [lustre-devel] [PATCH v3 07/26] staging: lustre: libcfs: NUMA
 support
In-Reply-To: <87o9fkyjqs.fsf@notabene.neil.brown.name>
References: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
 <1529875250-11531-8-git-send-email-jsimmons@infradead.org>
 <877emnaed8.fsf@notabene.neil.brown.name>
 <A764DED1-A9A8-4FA5-A0F1-922C3E2D822A@cray.com>
 <87lgb16j8q.fsf@notabene.neil.brown.name>
 <DM3PR1101MB1213FD1001256E565FF011FFCB480@DM3PR1101MB1213.namprd11.prod.outlook.com>
 <8736x77lrk.fsf@notabene.neil.brown.name>
 <9DE389AB-C7E1-4336-B7E8-604581EFD53E@cray.com>
 <CAAqp6i0uwmpdFYUJ7mCBz+MH+gVBa6S=P2t5Yn7wcFOtrTD-Fg@mail.gmail.com>
 <AT5PR8401MB0900C67FADDF0EF7ED1CDBC4854E0@AT5PR8401MB0900.NAMPRD84.PROD.OUTLOOK.COM>
 <87va9vziay.fsf@notabene.neil.brown.name>
 <AT5PR8401MB0900A5406BF0CB8C87B80EEC85410@AT5PR8401MB0900.NAMPRD84.PROD.OUTLOOK.COM>
 <alpine.LFD.2.21.1807060052470.12665@casper.infradead.org>
 <87zhz5xdld.fsf@notabene.neil.brown.name>
 <02B7B718-E496-46CF-B55D-86EAB98381B6@cray.com>
 <87o9fkyjqs.fsf@notabene.neil.brown.name>
Message-ID: <alpine.LFD.2.21.1807061650320.15728@casper.infradead.org>


> > When the CPT code was added to LNet back in 2012, it was to address
> > one primary case: a need for finer grained locking on metadata
> > servers.  LNet used to have global locks and metadata servers, which
> > do many small messages (high IOPS), much time in the worker threads
> > was spent in spinlocks.  So, CPT configuration was added so
> > locks/resources could be allocated per CPT.  This way, users have
> > control over how they want CPTs to be configured and how they want
> > resources/locks to be divided.  For example, users may want finer
> > grained locking on the metadata servers but not on clients.  Leaving
> > this to be automatically configured by Linux API calls would take this
> > flexibility away from the users who, for HPC, are very knowledgable
> > about what they want (i.e. we do not want to protect them from
> > themselves).
> >
> > The CPT support in LNet and LNDs has morphed to encompass more
> > traditional NUMA and core affinity performance improvements.  For
> > example, you can restrict a network interface to a socket (NUMA node)
> > which has better affinity to the PCIe lanes that interface is
> > connected to.  Rather than try to do this sort of thing automatically,
> > we have left it to the user to know what they are doing and configure
> > the CPTs accordingly.
> >
> > I think the many changes to the CPT code has realty clouded its
> > purpose.  In summary, the original purpose was finer grained locking
> > and that needs to be maintained as the IOPS requirements of metadata
> > servers is paramount.
> 
> Thanks for the explanation.
> I definitely get that fine-grained locking is a good thing.  Lustre is
> not alone in this of course.
> Even better than fine-grained locking is no locking.  That is not often
> possible, but this
>   https://github.com/neilbrown/linux/commit/ac3f8fd6e61b245fa9c14e3164203c1211c5ef6b
> 
> is an example of doing exactly that.
> 
> For the read/writer usage of CPT locks, RCU is a better approach if it
> can be made to work (usually it can) - and it scales even better.
> 
> When I was digging through the usage of locks I saw some hash tables.
> It seems that a lock protected a whole table.  It is usually sufficient
> for the lock to just protect a single chain (bit spin-locks can easily
> store one lock per chain) and then only for writes - RCU discipline can
> allow reads to proceed with only rcu_read_lock().
> Would we still need per-CPT tables once that was in place?  I don't know
> yet, though per-node seems likely to be sufficient when locking is per-chain.
> 
> I certainly wouldn't discard CPTs without replacing them with something
> better.  Near the top of my list for when I return from vacation
> (leaving in a couple of days) will be to look closely at the current
> fine-grained locking that you have helped me to see more clearly, and
> see if I can make it even better.

If RCU can provide better scaling then its best to replace CPT handling in
those cases. Lets land the Mult-Rail stuff first since it makes the most
heavy use of the CPT code. From there we can get a good idea of how to
move forward. I don't think we can easily abandon the CPT infrastructure
in general since we need it for partitioning to reduce noise. What would
be ideal is integrate the partitoning work to the general linux kernel.
While lustre attempts to reduce noise on nodes the rest of the kernel 
doesn't. If the linux kernel supported this it would be a big win for
HPC systems. The monster HPC systems today will be general hardware 5+
years down the road.

From paf at cray.com  Fri Jul  6 16:04:42 2018
From: paf at cray.com (Patrick Farrell)
Date: Fri, 6 Jul 2018 16:04:42 +0000
Subject: [lustre-devel] [PATCH v3 07/26] staging: lustre: libcfs: NUMA
 support
In-Reply-To: <alpine.LFD.2.21.1807061650320.15728@casper.infradead.org>
References: <1529875250-11531-1-git-send-email-jsimmons@infradead.org>
 <1529875250-11531-8-git-send-email-jsimmons@infradead.org>
 <877emnaed8.fsf@notabene.neil.brown.name>
 <A764DED1-A9A8-4FA5-A0F1-922C3E2D822A@cray.com>
 <87lgb16j8q.fsf@notabene.neil.brown.name>
 <DM3PR1101MB1213FD1001256E565FF011FFCB480@DM3PR1101MB1213.namprd11.prod.outlook.com>
 <8736x77lrk.fsf@notabene.neil.brown.name>
 <9DE389AB-C7E1-4336-B7E8-604581EFD53E@cray.com>
 <CAAqp6i0uwmpdFYUJ7mCBz+MH+gVBa6S=P2t5Yn7wcFOtrTD-Fg@mail.gmail.com>
 <AT5PR8401MB0900C67FADDF0EF7ED1CDBC4854E0@AT5PR8401MB0900.NAMPRD84.PROD.OUTLOOK.COM>
 <87va9vziay.fsf@notabene.neil.brown.name>
 <AT5PR8401MB0900A5406BF0CB8C87B80EEC85410@AT5PR8401MB0900.NAMPRD84.PROD.OUTLOOK.COM>
 <alpine.LFD.2.21.1807060052470.12665@casper.infradead.org>
 <87zhz5xdld.fsf@notabene.neil.brown.name>
 <02B7B718-E496-46CF-B55D-86EAB98381B6@cray.com>
 <87o9fkyjqs.fsf@notabene.neil.brown.name>
 <alpine.LFD.2.21.1807061650320.15728@casper.infradead.org>
Message-ID: <994B573E-4C9E-4A2D-AE20-2C682097C74D@cray.com>

Yeah, but they still won't really care much about noise.  Noise is really only a big problem if you're compounding it like HPC jobs do, otherwise it's negligible.  You worry about average time and maybe worst case - Not how noisy the average is, unless it suffers from wide excursions.  Lots of small excursions in execution time ("noise/jitter") don't matter.  (Unless you're an HPC job.)

The real time people care more about noise, though I believe they're still more concerned about worst cases and bounds than jitter.  Maybe some real time people are intensely worried about jitter for some use cases.

So this concern is not going mainstream even if the systems do, and the scheduler behavior required to minimize noise is sometimes not the same behavior required to improve responsiveness, reduce power consumption, etc.

Just food for thought.

On 7/6/18, 10:57 AM, "lustre-devel on behalf of James Simmons" <lustre-devel-bounces at lists.lustre.org on behalf of jsimmons at infradead.org> wrote:

    
    > > When the CPT code was added to LNet back in 2012, it was to address
    > > one primary case: a need for finer grained locking on metadata
    > > servers.  LNet used to have global locks and metadata servers, which
    > > do many small messages (high IOPS), much time in the worker threads
    > > was spent in spinlocks.  So, CPT configuration was added so
    > > locks/resources could be allocated per CPT.  This way, users have
    > > control over how they want CPTs to be configured and how they want
    > > resources/locks to be divided.  For example, users may want finer
    > > grained locking on the metadata servers but not on clients.  Leaving
    > > this to be automatically configured by Linux API calls would take this
    > > flexibility away from the users who, for HPC, are very knowledgable
    > > about what they want (i.e. we do not want to protect them from
    > > themselves).
    > >
    > > The CPT support in LNet and LNDs has morphed to encompass more
    > > traditional NUMA and core affinity performance improvements.  For
    > > example, you can restrict a network interface to a socket (NUMA node)
    > > which has better affinity to the PCIe lanes that interface is
    > > connected to.  Rather than try to do this sort of thing automatically,
    > > we have left it to the user to know what they are doing and configure
    > > the CPTs accordingly.
    > >
    > > I think the many changes to the CPT code has realty clouded its
    > > purpose.  In summary, the original purpose was finer grained locking
    > > and that needs to be maintained as the IOPS requirements of metadata
    > > servers is paramount.
    > 
    > Thanks for the explanation.
    > I definitely get that fine-grained locking is a good thing.  Lustre is
    > not alone in this of course.
    > Even better than fine-grained locking is no locking.  That is not often
    > possible, but this
    >   https://github.com/neilbrown/linux/commit/ac3f8fd6e61b245fa9c14e3164203c1211c5ef6b
    > 
    > is an example of doing exactly that.
    > 
    > For the read/writer usage of CPT locks, RCU is a better approach if it
    > can be made to work (usually it can) - and it scales even better.
    > 
    > When I was digging through the usage of locks I saw some hash tables.
    > It seems that a lock protected a whole table.  It is usually sufficient
    > for the lock to just protect a single chain (bit spin-locks can easily
    > store one lock per chain) and then only for writes - RCU discipline can
    > allow reads to proceed with only rcu_read_lock().
    > Would we still need per-CPT tables once that was in place?  I don't know
    > yet, though per-node seems likely to be sufficient when locking is per-chain.
    > 
    > I certainly wouldn't discard CPTs without replacing them with something
    > better.  Near the top of my list for when I return from vacation
    > (leaving in a couple of days) will be to look closely at the current
    > fine-grained locking that you have helped me to see more clearly, and
    > see if I can make it even better.
    
    If RCU can provide better scaling then its best to replace CPT handling in
    those cases. Lets land the Mult-Rail stuff first since it makes the most
    heavy use of the CPT code. From there we can get a good idea of how to
    move forward. I don't think we can easily abandon the CPT infrastructure
    in general since we need it for partitioning to reduce noise. What would
    be ideal is integrate the partitoning work to the general linux kernel.
    While lustre attempts to reduce noise on nodes the rest of the kernel 
    doesn't. If the linux kernel supported this it would be a big win for
    HPC systems. The monster HPC systems today will be general hardware 5+
    years down the road.
    _______________________________________________
    lustre-devel mailing list
    lustre-devel at lists.lustre.org
    http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org
    


From spitzcor at cray.com  Fri Jul  6 16:26:00 2018
From: spitzcor at cray.com (Cory Spitz)
Date: Fri, 6 Jul 2018 16:26:00 +0000
Subject: [lustre-devel] New slack chat room for linux client work
In-Reply-To: <alpine.LFD.2.21.1807061531540.15728@casper.infradead.org>
References: <alpine.LFD.2.21.1807061531540.15728@casper.infradead.org>
Message-ID: <88A13941-47CB-4D08-8CA8-4FD041487322@cray.com>

James, Slack tells me:
"Don't have an @whamcloud.com email address? 
Contact your Workspace Administrator for an invitation."

-Cory

-- 

On 7/6/18, 9:34 AM, "lustre-devel on behalf of James Simmons" <lustre-devel-bounces at lists.lustre.org on behalf of jsimmons at infradead.org> wrote:

    
    Some people have asked for a chat room since it can be difficult
    to filter between the mixture of patches and technical discusssions
    going on here. If people are interested to join the site is below
    
    https://lustreupstreamclient.slack.com/messages/CAV4QF0ER/
    _______________________________________________
    lustre-devel mailing list
    lustre-devel at lists.lustre.org
    http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org
    


From adilger at whamcloud.com  Fri Jul  6 17:09:40 2018
From: adilger at whamcloud.com (Andreas Dilger)
Date: Fri, 6 Jul 2018 17:09:40 +0000
Subject: [lustre-devel] [PATCH v3 04/13] lustre: libcfs: fix
 cfs_print_to_console()
In-Reply-To: <alpine.LFD.2.21.1807060034020.12665@casper.infradead.org>
References: <1530128322-32535-1-git-send-email-jsimmons@infradead.org>
 <1530128322-32535-5-git-send-email-jsimmons@infradead.org>
 <FD87D2B8-27A8-45DF-9C7C-BA33CF940C1D@whamcloud.com>
 <alpine.LFD.2.21.1806291716050.26339@casper.infradead.org>
 <87h8li2wz7.fsf@notabene.neil.brown.name>
 <alpine.LFD.2.21.1807060034020.12665@casper.infradead.org>
Message-ID: <1C8DA303-7F46-4E26-9B13-42A3F491EB5B@whamcloud.com>

On Jul 5, 2018, at 17:35, James Simmons <jsimmons at infradead.org> wrote:
> 
>>>>> void cfs_print_to_console(struct ptldebug_header *hdr, int mask,
>>>>> 			  const char *buf, int len, const char *file,
>>>>> 			  const char *fn)
>>>>> {
>>>>> +	char *prefix = "Lustre";
>>>>> +
>>>>> +	if (hdr->ph_subsys == S_LND || hdr->ph_subsys == S_LNET)
>>>>> +		prefix = "LNet";
>>>>> 
>>>>> -	if (mask & D_EMERG) {
>>>>> -		prefix = dbghdr_to_err_string(hdr);
>>>>> -		ptype = KERN_EMERG;
>>>>> +	if (mask & (D_CONSOLE | libcfs_printk)) {
>>>> 
>>>> This check is broken.  The default value of libcfs_printk (the mask
>>>> that controls which messages should be printed to the console, and
>>>> which ones should only be logged into the internal buffer) is:
>>>> 
>>>>    #define D_CANTMASK (D_ERROR | D_EMERG | D_WARNING | D_CONSOLE)
>>>> 
>>>> so that means virtually every console message will be printed with
>>>> pr_info() because this is matched first, instead of pr_emerg() or
>>>> pr_err() below.
>>>> 
>>>> That is why the previous code was matching D_EMERG and D_ERROR first,
>>>> then D_WARNING, and (D_CONSOLE | libcfs_printk) at the end.
>>> 
>>> 
>> 
>> The original code would *always* print something.
>> This code looks like it might not (e.g. for mask == 0).
> 
> Is that correct behavior? A mask of zero means don't do anything.
> Also if you look at the original in the OpenSFS branch you can see
> if mask was to be 0 then ptype would be NULl :-(
> 
> Note a D_CANT_MASK prevents some fields in the mask from being disabled.

Typically, users will set "lctl set_param debug=0" to disable debugging
messages for performance, but we still want critical error messages to
be printed to the console, so D_CANT_MASK is always checked for messages
to be printed to the console.

>> What is "D_CONSOLE" suppose to mean?  It seems to me "make the messages
>> less verbose" and it isn't clear to me that what is called "D_CONSOLE".
> 
> It means two things. If by itself then it means use pr_info. If it is one
> field in the mask then it means less detail. Confusing no :-( That is my
> understanding of it. Maybe someone else can clarify if I'm wrong.

The D_CONSOLE mask is used for "pretty" messages on the console, that will
not print out the file/function/line/timestamp and other debugging fields.
The D_CONSOLE flag doesn't necessarily mean "more" or "less" information,
but is intended more for "ease of understanding".

Cheers, Andreas
---
Andreas Dilger
Principal Lustre Architect
Whamcloud







-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 235 bytes
Desc: Message signed with OpenPGP
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180706/7b8454f0/attachment.sig>

From jsimmons at infradead.org  Fri Jul  6 23:18:26 2018
From: jsimmons at infradead.org (James Simmons)
Date: Sat, 7 Jul 2018 00:18:26 +0100 (BST)
Subject: [lustre-devel] New slack chat room for linux client work
In-Reply-To: <88A13941-47CB-4D08-8CA8-4FD041487322@cray.com>
References: <alpine.LFD.2.21.1807061531540.15728@casper.infradead.org>
 <88A13941-47CB-4D08-8CA8-4FD041487322@cray.com>
Message-ID: <alpine.LFD.2.21.1807070013470.6125@casper.infradead.org>


> James, Slack tells me:
> "Don't have an @whamcloud.com email address? 
> Contact your Workspace Administrator for an invitation."

Yeah slack doesn't seem to be a open to anyone platform. You have to ask
the admin (me in this case) if you can join. I can let people in the same
domain like whamcloud.com or cray.com to join.

> 
> -Cory
> 
> -- 
> 
> On 7/6/18, 9:34 AM, "lustre-devel on behalf of James Simmons" <lustre-devel-bounces at lists.lustre.org on behalf of jsimmons at infradead.org> wrote:
> 
>     
>     Some people have asked for a chat room since it can be difficult
>     to filter between the mixture of patches and technical discusssions
>     going on here. If people are interested to join the site is below
>     
>     https://lustreupstreamclient.slack.com/messages/CAV4QF0ER/
>     _______________________________________________
>     lustre-devel mailing list
>     lustre-devel at lists.lustre.org
>     http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org
>     
> 
> 

From jsimmons at infradead.org  Sun Jul  8 00:14:14 2018
From: jsimmons at infradead.org (James Simmons)
Date: Sat,  7 Jul 2018 20:14:14 -0400
Subject: [lustre-devel] [PATCH v4 00/14] lustre: libcfs: tracefile cleanups
Message-ID: <1531008868-4194-1-git-send-email-jsimmons@infradead.org>

The 4th version of a patch series aimed to cleanup the lustre tracefile
handling. This set is mainly a repost of the 3rd set with the addition
of a bug fix, patch number 2, that added proper range checking for the
debugfs parameter debug_mb. The 6th patch has been updated based on
Andreas Dilger's feedback to resolve the brokeness of cfs_print_to_console().
This patch series has been reposted to allow anyone interested in testing
the the major changes are in the 6th and 7th patch which was rebased due
to changes in the code.

James Simmons (6):
  lustre: libcfs: properly handle failure paths in cfs_tracefile_init_arch()
  lustre: libcfs: fix cfs_print_to_console()
  lustre: libcfs: remove cfs_trace_refill_stack()
  lustre: libcfs: move cfs_trace_data data to tracefile.c
  lustre: libcfs: cleanup tracefile.h
  lustre: libcfs: format macros in tracefile.h

NeilBrown (8):
  lustre: libcfs: move tracefile locking from linux-tracefile.c to tracefile.c
  lustre: libcfs: always range-check libcfs_debug_mb setting.
  lustre: libcfs: open code cfs_trace_max_debug_mb() into cfs_trace_set_debug_mb()
  lustre: libcfs: move tcd locking across to tracefile.c
  lustre: libcfs: merge linux-tracefile.c into tracefile.c
  lustre: libcfs: fold cfs_tracefile_*_arch into their only callers.
  lustre: libcfs: renamed CFS_TCD_TYPE_MAX to CFS_TCD_TYPE_CNT
  lustre: libcfs: discard TCD_MAX_TYPES

 drivers/staging/lustre/lnet/libcfs/Makefile        |   2 +-
 drivers/staging/lustre/lnet/libcfs/debug.c         |  22 +-
 .../staging/lustre/lnet/libcfs/linux-tracefile.c   | 258 -------------------
 drivers/staging/lustre/lnet/libcfs/tracefile.c     | 279 +++++++++++++++++----
 drivers/staging/lustre/lnet/libcfs/tracefile.h     | 117 +--------
 5 files changed, 255 insertions(+), 423 deletions(-)
 delete mode 100644 drivers/staging/lustre/lnet/libcfs/linux-tracefile.c

-- 
1.8.3.1


From jsimmons at infradead.org  Sun Jul  8 00:14:16 2018
From: jsimmons at infradead.org (James Simmons)
Date: Sat,  7 Jul 2018 20:14:16 -0400
Subject: [lustre-devel] [PATCH v4 02/14] lustre: libcfs: always range-check
 libcfs_debug_mb setting.
In-Reply-To: <1531008868-4194-1-git-send-email-jsimmons@infradead.org>
References: <1531008868-4194-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1531008868-4194-3-git-send-email-jsimmons@infradead.org>

From: NeilBrown <neilb at suse.com>

When the libcfs_debug_mb module parameter is set
at module-load time it isn't range-checked.  When
it is set via sysfs it is.  This inconsistency
makes the code harder to follow.

It is quite safe to call cfs_trace_set_debug_mb()
and cfs_trace_get_debug_mb() before the module
is initialized as cfs_tcd_for_each() does nothing
before initializtion.

So change cfs_trace_set_debug_mb() - which does
range checking - to returned the ranged checked number (it currently
always returns zero) and use that as the new value, unless
cfs_trace_get_debug_mb() now returns a non-zero value.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 drivers/staging/lustre/lnet/libcfs/debug.c     | 16 +++++++---------
 drivers/staging/lustre/lnet/libcfs/tracefile.c |  2 +-
 2 files changed, 8 insertions(+), 10 deletions(-)

diff --git a/drivers/staging/lustre/lnet/libcfs/debug.c b/drivers/staging/lustre/lnet/libcfs/debug.c
index 06f694f..50c2995 100644
--- a/drivers/staging/lustre/lnet/libcfs/debug.c
+++ b/drivers/staging/lustre/lnet/libcfs/debug.c
@@ -67,17 +67,15 @@ static int libcfs_param_debug_mb_set(const char *val,
 	if (rc < 0)
 		return rc;
 
-	if (!*((unsigned int *)kp->arg)) {
-		*((unsigned int *)kp->arg) = num;
-		return 0;
-	}
-
-	rc = cfs_trace_set_debug_mb(num);
+	num = cfs_trace_set_debug_mb(num);
 
-	if (!rc)
-		*((unsigned int *)kp->arg) = cfs_trace_get_debug_mb();
+	*((unsigned int *)kp->arg) = num;
+	num = cfs_trace_get_debug_mb();
+	if (num)
+		/* This value is more precise */
+		*((unsigned int *)kp->arg) = num;
 
-	return rc;
+	return 0;
 }
 
 /* While debug_mb setting look like unsigned int, in fact
diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.c b/drivers/staging/lustre/lnet/libcfs/tracefile.c
index 5f31933..3b92fd7 100644
--- a/drivers/staging/lustre/lnet/libcfs/tracefile.c
+++ b/drivers/staging/lustre/lnet/libcfs/tracefile.c
@@ -958,7 +958,7 @@ int cfs_trace_set_debug_mb(int mb)
 
 	up_write(&cfs_tracefile_sem);
 
-	return 0;
+	return mb;
 }
 
 int cfs_trace_get_debug_mb(void)
-- 
1.8.3.1


From jsimmons at infradead.org  Sun Jul  8 00:14:17 2018
From: jsimmons at infradead.org (James Simmons)
Date: Sat,  7 Jul 2018 20:14:17 -0400
Subject: [lustre-devel] [PATCH v4 03/14] lustre: libcfs: open code
 cfs_trace_max_debug_mb() into cfs_trace_set_debug_mb()
In-Reply-To: <1531008868-4194-1-git-send-email-jsimmons@infradead.org>
References: <1531008868-4194-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1531008868-4194-4-git-send-email-jsimmons@infradead.org>

From: NeilBrown <neilb at suse.com>

This function in used twice.
In libcfs_debug_init() the usage is pointless as
the only place that set libcfs_debug_mb ensures
that it does not exceed the maximum.  So checking
again can never help.

So open-code the small function into the only
other place that it is used - in cfs_trace_set_debug_mb(),
which is used to set libcfs_debug_mb.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 drivers/staging/lustre/lnet/libcfs/debug.c           | 6 +++---
 drivers/staging/lustre/lnet/libcfs/linux-tracefile.c | 7 -------
 drivers/staging/lustre/lnet/libcfs/tracefile.c       | 3 ++-
 drivers/staging/lustre/lnet/libcfs/tracefile.h       | 1 -
 4 files changed, 5 insertions(+), 12 deletions(-)

diff --git a/drivers/staging/lustre/lnet/libcfs/debug.c b/drivers/staging/lustre/lnet/libcfs/debug.c
index 50c2995..3ec4adf 100644
--- a/drivers/staging/lustre/lnet/libcfs/debug.c
+++ b/drivers/staging/lustre/lnet/libcfs/debug.c
@@ -409,10 +409,10 @@ int libcfs_debug_init(unsigned long bufsize)
 			sizeof(libcfs_debug_file_path_arr));
 	}
 
-	/* If libcfs_debug_mb is set to an invalid value or uninitialized
-	 * then just make the total buffers smp_num_cpus * TCD_MAX_PAGES
+	/* If libcfs_debug_mb is uninitialized then just make the
+	 * total buffers smp_num_cpus * TCD_MAX_PAGES
 	 */
-	if (max > cfs_trace_max_debug_mb() || max < num_possible_cpus()) {
+	if (max < num_possible_cpus()) {
 		max = TCD_MAX_PAGES;
 	} else {
 		max = max / num_possible_cpus();
diff --git a/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c b/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
index 9e72220..64a5bc1 100644
--- a/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
+++ b/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
@@ -227,10 +227,3 @@ void cfs_print_to_console(struct ptldebug_header *hdr, int mask,
 			hdr->ph_line_num, fn, len, buf);
 	}
 }
-
-int cfs_trace_max_debug_mb(void)
-{
-	int  total_mb = (totalram_pages >> (20 - PAGE_SHIFT));
-
-	return max(512, (total_mb * 80) / 100);
-}
diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.c b/drivers/staging/lustre/lnet/libcfs/tracefile.c
index 3b92fd7..ae1a89c 100644
--- a/drivers/staging/lustre/lnet/libcfs/tracefile.c
+++ b/drivers/staging/lustre/lnet/libcfs/tracefile.c
@@ -933,7 +933,8 @@ int cfs_trace_set_debug_mb(int mb)
 	int i;
 	int j;
 	int pages;
-	int limit = cfs_trace_max_debug_mb();
+	int total_mb = (totalram_pages >> (20 - PAGE_SHIFT));
+	int limit = max(512, (total_mb * 80) / 100);
 	struct cfs_trace_cpu_data *tcd;
 
 	if (mb < num_possible_cpus()) {
diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.h b/drivers/staging/lustre/lnet/libcfs/tracefile.h
index 9f6b73d..bd1a1ef 100644
--- a/drivers/staging/lustre/lnet/libcfs/tracefile.h
+++ b/drivers/staging/lustre/lnet/libcfs/tracefile.h
@@ -88,7 +88,6 @@ int cfs_trace_copyout_string(char __user *usr_buffer, int usr_buffer_nob,
 void libcfs_register_panic_notifier(void);
 void libcfs_unregister_panic_notifier(void);
 extern int libcfs_panic_in_progress;
-int cfs_trace_max_debug_mb(void);
 
 #define TCD_MAX_PAGES (5 << (20 - PAGE_SHIFT))
 #define TCD_STOCK_PAGES (TCD_MAX_PAGES)
-- 
1.8.3.1


From jsimmons at infradead.org  Sun Jul  8 00:14:22 2018
From: jsimmons at infradead.org (James Simmons)
Date: Sat,  7 Jul 2018 20:14:22 -0400
Subject: [lustre-devel] [PATCH v4 08/14] lustre: libcfs: remove
 cfs_trace_refill_stack()
In-Reply-To: <1531008868-4194-1-git-send-email-jsimmons@infradead.org>
References: <1531008868-4194-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1531008868-4194-9-git-send-email-jsimmons@infradead.org>

The function cfs_trace_refill_stack() is not used anywhere so
remove it.

Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/lnet/libcfs/tracefile.c | 21 ---------------------
 drivers/staging/lustre/lnet/libcfs/tracefile.h |  3 ---
 2 files changed, 24 deletions(-)

diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.c b/drivers/staging/lustre/lnet/libcfs/tracefile.c
index 066476d..ec90b36 100644
--- a/drivers/staging/lustre/lnet/libcfs/tracefile.c
+++ b/drivers/staging/lustre/lnet/libcfs/tracefile.c
@@ -302,27 +302,6 @@ static void cfs_tage_to_tail(struct cfs_trace_page *tage,
 	list_move_tail(&tage->linkage, queue);
 }
 
-int cfs_trace_refill_stock(struct cfs_trace_cpu_data *tcd, gfp_t gfp,
-			   struct list_head *stock)
-{
-	int i;
-
-	/*
-	 * XXX nikita: do NOT call portals_debug_msg() (CDEBUG/ENTRY/EXIT)
-	 * from here: this will lead to infinite recursion.
-	 */
-
-	for (i = 0; i + tcd->tcd_cur_stock_pages < TCD_STOCK_PAGES ; ++i) {
-		struct cfs_trace_page *tage;
-
-		tage = cfs_tage_alloc(gfp);
-		if (!tage)
-			break;
-		list_add_tail(&tage->linkage, stock);
-	}
-	return i;
-}
-
 /* return a page that has 'len' bytes left at the end */
 static struct cfs_trace_page *
 cfs_trace_get_tage_try(struct cfs_trace_cpu_data *tcd, unsigned long len)
diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.h b/drivers/staging/lustre/lnet/libcfs/tracefile.h
index 399e0bf..b9e4a8e 100644
--- a/drivers/staging/lustre/lnet/libcfs/tracefile.h
+++ b/drivers/staging/lustre/lnet/libcfs/tracefile.h
@@ -204,9 +204,6 @@ int cfs_trace_copyout_string(char __user *usr_buffer, int usr_buffer_nob,
 	return cfs_trace_console_buffers[i][j];
 }
 
-int cfs_trace_refill_stock(struct cfs_trace_cpu_data *tcd, gfp_t gfp,
-			   struct list_head *stock);
-
 void cfs_trace_assertion_failed(const char *str,
 				struct libcfs_debug_msg_data *m);
 
-- 
1.8.3.1


From jsimmons at infradead.org  Sun Jul  8 00:14:15 2018
From: jsimmons at infradead.org (James Simmons)
Date: Sat,  7 Jul 2018 20:14:15 -0400
Subject: [lustre-devel] [PATCH v4 01/14] lustre: libcfs: move tracefile
 locking from linux-tracefile.c to tracefile.c
In-Reply-To: <1531008868-4194-1-git-send-email-jsimmons@infradead.org>
References: <1531008868-4194-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1531008868-4194-2-git-send-email-jsimmons@infradead.org>

From: NeilBrown <neilb at suse.com>

There is no value in keeping it separate.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 .../staging/lustre/lnet/libcfs/linux-tracefile.c   | 22 -------------------
 drivers/staging/lustre/lnet/libcfs/tracefile.c     | 25 +++++++++++-----------
 drivers/staging/lustre/lnet/libcfs/tracefile.h     |  5 -----
 3 files changed, 13 insertions(+), 39 deletions(-)

diff --git a/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c b/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
index 3471384..9e72220 100644
--- a/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
+++ b/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
@@ -47,8 +47,6 @@
 
 char *cfs_trace_console_buffers[NR_CPUS][CFS_TCD_TYPE_MAX];
 
-static DECLARE_RWSEM(cfs_tracefile_sem);
-
 int cfs_tracefile_init_arch(void)
 {
 	int i;
@@ -109,26 +107,6 @@ void cfs_tracefile_fini_arch(void)
 	}
 }
 
-void cfs_tracefile_read_lock(void)
-{
-	down_read(&cfs_tracefile_sem);
-}
-
-void cfs_tracefile_read_unlock(void)
-{
-	up_read(&cfs_tracefile_sem);
-}
-
-void cfs_tracefile_write_lock(void)
-{
-	down_write(&cfs_tracefile_sem);
-}
-
-void cfs_tracefile_write_unlock(void)
-{
-	up_write(&cfs_tracefile_sem);
-}
-
 enum cfs_trace_buf_type cfs_trace_buf_idx_get(void)
 {
 	if (in_irq())
diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.c b/drivers/staging/lustre/lnet/libcfs/tracefile.c
index 7ca562e..5f31933 100644
--- a/drivers/staging/lustre/lnet/libcfs/tracefile.c
+++ b/drivers/staging/lustre/lnet/libcfs/tracefile.c
@@ -59,6 +59,7 @@
 static int thread_running;
 
 static atomic_t cfs_tage_allocated = ATOMIC_INIT(0);
+static DECLARE_RWSEM(cfs_tracefile_sem);
 
 struct page_collection {
 	struct list_head	pc_pages;
@@ -711,7 +712,7 @@ int cfs_tracefile_dump_all_pages(char *filename)
 	mm_segment_t __oldfs;
 	int rc;
 
-	cfs_tracefile_write_lock();
+	down_write(&cfs_tracefile_sem);
 
 	filp = filp_open(filename, O_CREAT | O_EXCL | O_WRONLY | O_LARGEFILE,
 			 0600);
@@ -759,7 +760,7 @@ int cfs_tracefile_dump_all_pages(char *filename)
 close:
 	filp_close(filp, NULL);
 out:
-	cfs_tracefile_write_unlock();
+	up_write(&cfs_tracefile_sem);
 	return rc;
 }
 
@@ -873,12 +874,12 @@ int cfs_trace_daemon_command(char *str)
 {
 	int rc = 0;
 
-	cfs_tracefile_write_lock();
+	down_write(&cfs_tracefile_sem);
 
 	if (!strcmp(str, "stop")) {
-		cfs_tracefile_write_unlock();
+		up_write(&cfs_tracefile_sem);
 		cfs_trace_stop_thread();
-		cfs_tracefile_write_lock();
+		down_write(&cfs_tracefile_sem);
 		memset(cfs_tracefile, 0, sizeof(cfs_tracefile));
 
 	} else if (!strncmp(str, "size=", 5)) {
@@ -905,7 +906,7 @@ int cfs_trace_daemon_command(char *str)
 		cfs_trace_start_thread();
 	}
 
-	cfs_tracefile_write_unlock();
+	up_write(&cfs_tracefile_sem);
 	return rc;
 }
 
@@ -950,12 +951,12 @@ int cfs_trace_set_debug_mb(int mb)
 	mb /= num_possible_cpus();
 	pages = mb << (20 - PAGE_SHIFT);
 
-	cfs_tracefile_write_lock();
+	down_write(&cfs_tracefile_sem);
 
 	cfs_tcd_for_each(tcd, i, j)
 		tcd->tcd_max_pages = (pages * tcd->tcd_pages_factor) / 100;
 
-	cfs_tracefile_write_unlock();
+	up_write(&cfs_tracefile_sem);
 
 	return 0;
 }
@@ -967,12 +968,12 @@ int cfs_trace_get_debug_mb(void)
 	struct cfs_trace_cpu_data *tcd;
 	int total_pages = 0;
 
-	cfs_tracefile_read_lock();
+	down_read(&cfs_tracefile_sem);
 
 	cfs_tcd_for_each(tcd, i, j)
 		total_pages += tcd->tcd_max_pages;
 
-	cfs_tracefile_read_unlock();
+	up_read(&cfs_tracefile_sem);
 
 	return (total_pages >> (20 - PAGE_SHIFT)) + 1;
 }
@@ -1002,7 +1003,7 @@ static int tracefiled(void *arg)
 			goto end_loop;
 
 		filp = NULL;
-		cfs_tracefile_read_lock();
+		down_read(&cfs_tracefile_sem);
 		if (cfs_tracefile[0]) {
 			filp = filp_open(cfs_tracefile,
 					 O_CREAT | O_RDWR | O_LARGEFILE,
@@ -1014,7 +1015,7 @@ static int tracefiled(void *arg)
 					rc);
 			}
 		}
-		cfs_tracefile_read_unlock();
+		up_read(&cfs_tracefile_sem);
 		if (!filp) {
 			put_pages_on_daemon_list(&pc);
 			__LASSERT(list_empty(&pc.pc_pages));
diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.h b/drivers/staging/lustre/lnet/libcfs/tracefile.h
index 0608240..9f6b73d 100644
--- a/drivers/staging/lustre/lnet/libcfs/tracefile.h
+++ b/drivers/staging/lustre/lnet/libcfs/tracefile.h
@@ -65,11 +65,6 @@ enum cfs_trace_buf_type {
 int  cfs_tracefile_init_arch(void);
 void cfs_tracefile_fini_arch(void);
 
-void cfs_tracefile_read_lock(void);
-void cfs_tracefile_read_unlock(void);
-void cfs_tracefile_write_lock(void);
-void cfs_tracefile_write_unlock(void);
-
 int cfs_tracefile_dump_all_pages(char *filename);
 void cfs_trace_debug_print(void);
 void cfs_trace_flush_pages(void);
-- 
1.8.3.1


From jsimmons at infradead.org  Sun Jul  8 00:14:21 2018
From: jsimmons at infradead.org (James Simmons)
Date: Sat,  7 Jul 2018 20:14:21 -0400
Subject: [lustre-devel] [PATCH v4 07/14] lustre: libcfs: merge
 linux-tracefile.c into tracefile.c
In-Reply-To: <1531008868-4194-1-git-send-email-jsimmons@infradead.org>
References: <1531008868-4194-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1531008868-4194-8-git-send-email-jsimmons@infradead.org>

From: NeilBrown <neilb at suse.com>

It's good to keep related code together.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 drivers/staging/lustre/lnet/libcfs/Makefile        |   2 +-
 .../staging/lustre/lnet/libcfs/linux-tracefile.c   | 186 ---------------------
 drivers/staging/lustre/lnet/libcfs/tracefile.c     | 145 ++++++++++++++++
 drivers/staging/lustre/lnet/libcfs/tracefile.h     |   7 -
 4 files changed, 146 insertions(+), 194 deletions(-)
 delete mode 100644 drivers/staging/lustre/lnet/libcfs/linux-tracefile.c

diff --git a/drivers/staging/lustre/lnet/libcfs/Makefile b/drivers/staging/lustre/lnet/libcfs/Makefile
index 5b13edc..cd96434 100644
--- a/drivers/staging/lustre/lnet/libcfs/Makefile
+++ b/drivers/staging/lustre/lnet/libcfs/Makefile
@@ -4,7 +4,7 @@ ccflags-y += -I$(srctree)/drivers/staging/lustre/lustre/include
 
 obj-$(CONFIG_LNET) += libcfs.o
 
-libcfs-obj-y += linux-tracefile.o linux-debug.o
+libcfs-obj-y += linux-debug.o
 libcfs-obj-y += linux-crypto.o
 libcfs-obj-y += linux-crypto-adler.o
 
diff --git a/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c b/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
deleted file mode 100644
index ef26835..0000000
--- a/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
+++ /dev/null
@@ -1,186 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-/*
- * GPL HEADER START
- *
- * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 only,
- * as published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful, but
- * WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
- * General Public License version 2 for more details (a copy is included
- * in the LICENSE file that accompanied this code).
- *
- * You should have received a copy of the GNU General Public License
- * version 2 along with this program; If not, see
- * http://www.gnu.org/licenses/gpl-2.0.html
- *
- * GPL HEADER END
- */
-/*
- * Copyright (c) 2008, 2010, Oracle and/or its affiliates. All rights reserved.
- * Use is subject to license terms.
- *
- * Copyright (c) 2012, Intel Corporation.
- */
-/*
- * This file is part of Lustre, http://www.lustre.org/
- * Lustre is a trademark of Sun Microsystems, Inc.
- */
-
-#define DEBUG_SUBSYSTEM S_LNET
-#define LUSTRE_TRACEFILE_PRIVATE
-
-#include <linux/slab.h>
-#include <linux/mm.h>
-#include "tracefile.h"
-
-/* percents to share the total debug memory for each type */
-static unsigned int pages_factor[CFS_TCD_TYPE_MAX] = {
-	80,  /* 80% pages for CFS_TCD_TYPE_PROC */
-	10,  /* 10% pages for CFS_TCD_TYPE_SOFTIRQ */
-	10   /* 10% pages for CFS_TCD_TYPE_IRQ */
-};
-
-char *cfs_trace_console_buffers[NR_CPUS][CFS_TCD_TYPE_MAX];
-
-int cfs_tracefile_init_arch(void)
-{
-	struct cfs_trace_cpu_data *tcd;
-	int i;
-	int j;
-
-	/* initialize trace_data */
-	memset(cfs_trace_data, 0, sizeof(cfs_trace_data));
-	for (i = 0; i < CFS_TCD_TYPE_MAX; i++) {
-		cfs_trace_data[i] =
-			kmalloc_array(num_possible_cpus(),
-				      sizeof(union cfs_trace_data_union),
-				      GFP_KERNEL);
-		if (!cfs_trace_data[i])
-			goto out_trace_data;
-	}
-
-	/* arch related info initialized */
-	cfs_tcd_for_each(tcd, i, j) {
-		spin_lock_init(&tcd->tcd_lock);
-		tcd->tcd_pages_factor = pages_factor[i];
-		tcd->tcd_type = i;
-		tcd->tcd_cpu = j;
-	}
-
-	for (i = 0; i < num_possible_cpus(); i++)
-		for (j = 0; j < 3; j++) {
-			cfs_trace_console_buffers[i][j] =
-				kmalloc(CFS_TRACE_CONSOLE_BUFFER_SIZE,
-					GFP_KERNEL);
-
-			if (!cfs_trace_console_buffers[i][j])
-				goto out_buffers;
-		}
-
-	return 0;
-
-out_buffers:
-	for (i = 0; i < num_possible_cpus(); i++)
-		for (j = 0; j < 3; j++) {
-			kfree(cfs_trace_console_buffers[i][j]);
-			cfs_trace_console_buffers[i][j] = NULL;
-		}
-out_trace_data:
-	for (i = 0; cfs_trace_data[i]; i++) {
-		kfree(cfs_trace_data[i]);
-		cfs_trace_data[i] = NULL;
-	}
-	pr_err("lnet: Not enough memory\n");
-	return -ENOMEM;
-}
-
-void cfs_tracefile_fini_arch(void)
-{
-	int i;
-	int j;
-
-	for (i = 0; i < num_possible_cpus(); i++)
-		for (j = 0; j < 3; j++) {
-			kfree(cfs_trace_console_buffers[i][j]);
-			cfs_trace_console_buffers[i][j] = NULL;
-		}
-
-	for (i = 0; cfs_trace_data[i]; i++) {
-		kfree(cfs_trace_data[i]);
-		cfs_trace_data[i] = NULL;
-	}
-}
-
-enum cfs_trace_buf_type cfs_trace_buf_idx_get(void)
-{
-	if (in_irq())
-		return CFS_TCD_TYPE_IRQ;
-	if (in_softirq())
-		return CFS_TCD_TYPE_SOFTIRQ;
-	return CFS_TCD_TYPE_PROC;
-}
-
-void
-cfs_set_ptldebug_header(struct ptldebug_header *header,
-			struct libcfs_debug_msg_data *msgdata,
-			unsigned long stack)
-{
-	struct timespec64 ts;
-
-	ktime_get_real_ts64(&ts);
-
-	header->ph_subsys = msgdata->msg_subsys;
-	header->ph_mask = msgdata->msg_mask;
-	header->ph_cpu_id = smp_processor_id();
-	header->ph_type = cfs_trace_buf_idx_get();
-	/* y2038 safe since all user space treats this as unsigned, but
-	 * will overflow in 2106
-	 */
-	header->ph_sec = (u32)ts.tv_sec;
-	header->ph_usec = ts.tv_nsec / NSEC_PER_USEC;
-	header->ph_stack = stack;
-	header->ph_pid = current->pid;
-	header->ph_line_num = msgdata->msg_line;
-	header->ph_extern_pid = 0;
-}
-
-void cfs_print_to_console(struct ptldebug_header *hdr, int mask,
-			  const char *buf, int len, const char *file,
-			  const char *fn)
-{
-	char *prefix = "Lustre";
-
-	if (hdr->ph_subsys == S_LND || hdr->ph_subsys == S_LNET)
-		prefix = "LNet";
-
-	if (mask & D_CONSOLE) {
-		if (mask & D_EMERG)
-			pr_emerg("%sError: %.*s", prefix, len, buf);
-		else if (mask & D_ERROR)
-			pr_err("%sError: %.*s", prefix, len, buf);
-		else if (mask & D_WARNING)
-			pr_warn("%s: %.*s", prefix, len, buf);
-		else if (mask & libcfs_printk)
-			pr_info("%s: %.*s", prefix, len, buf);
-	} else {
-		if (mask & D_EMERG)
-			pr_emerg("%sError: %d:%d:(%s:%d:%s()) %.*s", prefix,
-				 hdr->ph_pid, hdr->ph_extern_pid, file,
-				 hdr->ph_line_num, fn, len, buf);
-		else if (mask & D_ERROR)
-			pr_err("%sError: %d:%d:(%s:%d:%s()) %.*s", prefix,
-			       hdr->ph_pid, hdr->ph_extern_pid, file,
-			       hdr->ph_line_num, fn, len, buf);
-		else if (mask & D_WARNING)
-			pr_warn("%s: %d:%d:(%s:%d:%s()) %.*s", prefix,
-				hdr->ph_pid, hdr->ph_extern_pid, file,
-				hdr->ph_line_num, fn, len, buf);
-		else if (mask & (D_CONSOLE | libcfs_printk))
-			pr_info("%s: %.*s", prefix, len, buf);
-	}
-}
diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.c b/drivers/staging/lustre/lnet/libcfs/tracefile.c
index c46dc56..066476d 100644
--- a/drivers/staging/lustre/lnet/libcfs/tracefile.c
+++ b/drivers/staging/lustre/lnet/libcfs/tracefile.c
@@ -52,6 +52,7 @@
 /* XXX move things up to the top, comment */
 union cfs_trace_data_union (*cfs_trace_data[TCD_MAX_TYPES])[NR_CPUS] __cacheline_aligned;
 
+char *cfs_trace_console_buffers[NR_CPUS][CFS_TCD_TYPE_MAX];
 char cfs_tracefile[TRACEFILE_NAME_SIZE];
 long long cfs_tracefile_size = CFS_TRACEFILE_SIZE;
 static struct tracefiled_ctl trace_tctl;
@@ -150,6 +151,15 @@ void cfs_trace_unlock_tcd(struct cfs_trace_cpu_data *tcd, int walking)
 	     (tcd = &(*cfs_trace_data[i])[cpu].tcd) &&			\
 	     cfs_trace_lock_tcd(tcd, 1); cfs_trace_unlock_tcd(tcd, 1), i++)
 
+enum cfs_trace_buf_type cfs_trace_buf_idx_get(void)
+{
+	if (in_irq())
+		return CFS_TCD_TYPE_IRQ;
+	if (in_softirq())
+		return CFS_TCD_TYPE_SOFTIRQ;
+	return CFS_TCD_TYPE_PROC;
+}
+
 static inline struct cfs_trace_cpu_data *
 cfs_trace_get_tcd(void)
 {
@@ -168,6 +178,82 @@ static inline void cfs_trace_put_tcd(struct cfs_trace_cpu_data *tcd)
 	put_cpu();
 }
 
+/* percents to share the total debug memory for each type */
+static unsigned int pages_factor[CFS_TCD_TYPE_MAX] = {
+	80,  /* 80% pages for CFS_TCD_TYPE_PROC */
+	10,  /* 10% pages for CFS_TCD_TYPE_SOFTIRQ */
+	10   /* 10% pages for CFS_TCD_TYPE_IRQ */
+};
+
+int cfs_tracefile_init_arch(void)
+{
+	struct cfs_trace_cpu_data *tcd;
+	int i;
+	int j;
+
+	/* initialize trace_data */
+	memset(cfs_trace_data, 0, sizeof(cfs_trace_data));
+	for (i = 0; i < CFS_TCD_TYPE_MAX; i++) {
+		cfs_trace_data[i] =
+			kmalloc_array(num_possible_cpus(),
+				      sizeof(union cfs_trace_data_union),
+				      GFP_KERNEL);
+		if (!cfs_trace_data[i])
+			goto out_trace_data;
+	}
+
+	/* arch related info initialized */
+	cfs_tcd_for_each(tcd, i, j) {
+		spin_lock_init(&tcd->tcd_lock);
+		tcd->tcd_pages_factor = pages_factor[i];
+		tcd->tcd_type = i;
+		tcd->tcd_cpu = j;
+	}
+
+	for (i = 0; i < num_possible_cpus(); i++)
+		for (j = 0; j < 3; j++) {
+			cfs_trace_console_buffers[i][j] =
+				kmalloc(CFS_TRACE_CONSOLE_BUFFER_SIZE,
+					GFP_KERNEL);
+
+			if (!cfs_trace_console_buffers[i][j])
+				goto out_buffers;
+		}
+
+	return 0;
+
+out_buffers:
+	for (i = 0; i < num_possible_cpus(); i++)
+		for (j = 0; j < 3; j++) {
+			kfree(cfs_trace_console_buffers[i][j]);
+			cfs_trace_console_buffers[i][j] = NULL;
+		}
+out_trace_data:
+	for (i = 0; cfs_trace_data[i]; i++) {
+		kfree(cfs_trace_data[i]);
+		cfs_trace_data[i] = NULL;
+	}
+	pr_err("lnet: Not enough memory\n");
+	return -ENOMEM;
+}
+
+void cfs_tracefile_fini_arch(void)
+{
+	int i;
+	int j;
+
+	for (i = 0; i < num_possible_cpus(); i++)
+		for (j = 0; j < 3; j++) {
+			kfree(cfs_trace_console_buffers[i][j]);
+			cfs_trace_console_buffers[i][j] = NULL;
+		}
+
+	for (i = 0; cfs_trace_data[i]; i++) {
+		kfree(cfs_trace_data[i]);
+		cfs_trace_data[i] = NULL;
+	}
+}
+
 static inline struct cfs_trace_page *
 cfs_tage_from_list(struct list_head *list)
 {
@@ -340,6 +426,65 @@ static struct cfs_trace_page *cfs_trace_get_tage(struct cfs_trace_cpu_data *tcd,
 	return tage;
 }
 
+static void cfs_set_ptldebug_header(struct ptldebug_header *header,
+				    struct libcfs_debug_msg_data *msgdata,
+				    unsigned long stack)
+{
+	struct timespec64 ts;
+
+	ktime_get_real_ts64(&ts);
+
+	header->ph_subsys = msgdata->msg_subsys;
+	header->ph_mask = msgdata->msg_mask;
+	header->ph_cpu_id = smp_processor_id();
+	header->ph_type = cfs_trace_buf_idx_get();
+	/* y2038 safe since all user space treats this as unsigned, but
+	 * will overflow in 2106
+	 */
+	header->ph_sec = (u32)ts.tv_sec;
+	header->ph_usec = ts.tv_nsec / NSEC_PER_USEC;
+	header->ph_stack = stack;
+	header->ph_pid = current->pid;
+	header->ph_line_num = msgdata->msg_line;
+	header->ph_extern_pid = 0;
+}
+
+static void cfs_print_to_console(struct ptldebug_header *hdr, int mask,
+				 const char *buf, int len, const char *file,
+				 const char *fn)
+{
+	char *prefix = "Lustre";
+
+	if (hdr->ph_subsys == S_LND || hdr->ph_subsys == S_LNET)
+		prefix = "LNet";
+
+	if (mask & D_CONSOLE) {
+		if (mask & D_EMERG)
+			pr_emerg("%sError: %.*s", prefix, len, buf);
+		else if (mask & D_ERROR)
+			pr_err("%sError: %.*s", prefix, len, buf);
+		else if (mask & D_WARNING)
+			pr_warn("%s: %.*s", prefix, len, buf);
+		else if (mask & libcfs_printk)
+			pr_info("%s: %.*s", prefix, len, buf);
+	} else {
+		if (mask & D_EMERG)
+			pr_emerg("%sError: %d:%d:(%s:%d:%s()) %.*s", prefix,
+				 hdr->ph_pid, hdr->ph_extern_pid, file,
+				 hdr->ph_line_num, fn, len, buf);
+		else if (mask & D_ERROR)
+			pr_err("%sError: %d:%d:(%s:%d:%s()) %.*s", prefix,
+			       hdr->ph_pid, hdr->ph_extern_pid, file,
+			       hdr->ph_line_num, fn, len, buf);
+		else if (mask & D_WARNING)
+			pr_warn("%s: %d:%d:(%s:%d:%s()) %.*s", prefix,
+				hdr->ph_pid, hdr->ph_extern_pid, file,
+				hdr->ph_line_num, fn, len, buf);
+		else if (mask & (D_CONSOLE | libcfs_printk))
+			pr_info("%s: %.*s", prefix, len, buf);
+	}
+}
+
 int libcfs_debug_msg(struct libcfs_debug_msg_data *msgdata,
 		     const char *format, ...)
 {
diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.h b/drivers/staging/lustre/lnet/libcfs/tracefile.h
index 91968cf..399e0bf 100644
--- a/drivers/staging/lustre/lnet/libcfs/tracefile.h
+++ b/drivers/staging/lustre/lnet/libcfs/tracefile.h
@@ -192,13 +192,6 @@ int cfs_trace_copyout_string(char __user *usr_buffer, int usr_buffer_nob,
 		     j < num_possible_cpus();				 \
 		     j++, (tcd) = &(*cfs_trace_data[i])[j].tcd)
 
-void cfs_set_ptldebug_header(struct ptldebug_header *header,
-			     struct libcfs_debug_msg_data *m,
-			     unsigned long stack);
-void cfs_print_to_console(struct ptldebug_header *hdr, int mask,
-			  const char *buf, int len, const char *file,
-			  const char *fn);
-
 extern char *cfs_trace_console_buffers[NR_CPUS][CFS_TCD_TYPE_MAX];
 enum cfs_trace_buf_type cfs_trace_buf_idx_get(void);
 
-- 
1.8.3.1


From jsimmons at infradead.org  Sun Jul  8 00:14:23 2018
From: jsimmons at infradead.org (James Simmons)
Date: Sat,  7 Jul 2018 20:14:23 -0400
Subject: [lustre-devel] [PATCH v4 09/14] lustre: libcfs: move cfs_trace_data
 data to tracefile.c
In-Reply-To: <1531008868-4194-1-git-send-email-jsimmons@infradead.org>
References: <1531008868-4194-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1531008868-4194-10-git-send-email-jsimmons@infradead.org>

The macro cfs_tcd_for_each() is only used in tracefile.c so move
it from the header tracefile.h along with related material in
the header file.

Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/lnet/libcfs/tracefile.c | 12 ++++++++++--
 drivers/staging/lustre/lnet/libcfs/tracefile.h |  9 ---------
 2 files changed, 10 insertions(+), 11 deletions(-)

diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.c b/drivers/staging/lustre/lnet/libcfs/tracefile.c
index ec90b36..d80685c 100644
--- a/drivers/staging/lustre/lnet/libcfs/tracefile.c
+++ b/drivers/staging/lustre/lnet/libcfs/tracefile.c
@@ -49,7 +49,8 @@
 #include <linux/uaccess.h>
 #include "tracefile.h"
 
-/* XXX move things up to the top, comment */
+#define TCD_MAX_TYPES			8
+
 union cfs_trace_data_union (*cfs_trace_data[TCD_MAX_TYPES])[NR_CPUS] __cacheline_aligned;
 
 char *cfs_trace_console_buffers[NR_CPUS][CFS_TCD_TYPE_MAX];
@@ -146,7 +147,14 @@ void cfs_trace_unlock_tcd(struct cfs_trace_cpu_data *tcd, int walking)
 		spin_unlock(&tcd->tcd_lock);
 }
 
-#define cfs_tcd_for_each_type_lock(tcd, i, cpu)			   \
+#define cfs_tcd_for_each(tcd, i, j)					\
+	for (i = 0; cfs_trace_data[i]; i++)				\
+		for (j = 0, ((tcd) = &(*cfs_trace_data[i])[j].tcd);	\
+		     j < num_possible_cpus();				\
+		     j++, (tcd) = &(*cfs_trace_data[i])[j].tcd)
+
+
+#define cfs_tcd_for_each_type_lock(tcd, i, cpu)				\
 	for (i = 0; cfs_trace_data[i] &&				\
 	     (tcd = &(*cfs_trace_data[i])[cpu].tcd) &&			\
 	     cfs_trace_lock_tcd(tcd, 1); cfs_trace_unlock_tcd(tcd, 1), i++)
diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.h b/drivers/staging/lustre/lnet/libcfs/tracefile.h
index b9e4a8e..072c720 100644
--- a/drivers/staging/lustre/lnet/libcfs/tracefile.h
+++ b/drivers/staging/lustre/lnet/libcfs/tracefile.h
@@ -183,15 +183,6 @@ int cfs_trace_copyout_string(char __user *usr_buffer, int usr_buffer_nob,
 	char __pad[L1_CACHE_ALIGN(sizeof(struct cfs_trace_cpu_data))];
 };
 
-#define TCD_MAX_TYPES      8
-extern union cfs_trace_data_union (*cfs_trace_data[TCD_MAX_TYPES])[NR_CPUS];
-
-#define cfs_tcd_for_each(tcd, i, j)				       \
-	for (i = 0; cfs_trace_data[i]; i++)				\
-		for (j = 0, ((tcd) = &(*cfs_trace_data[i])[j].tcd);	\
-		     j < num_possible_cpus();				 \
-		     j++, (tcd) = &(*cfs_trace_data[i])[j].tcd)
-
 extern char *cfs_trace_console_buffers[NR_CPUS][CFS_TCD_TYPE_MAX];
 enum cfs_trace_buf_type cfs_trace_buf_idx_get(void);
 
-- 
1.8.3.1


From jsimmons at infradead.org  Sun Jul  8 00:14:18 2018
From: jsimmons at infradead.org (James Simmons)
Date: Sat,  7 Jul 2018 20:14:18 -0400
Subject: [lustre-devel] [PATCH v4 04/14] lustre: libcfs: move tcd locking
 across to tracefile.c
In-Reply-To: <1531008868-4194-1-git-send-email-jsimmons@infradead.org>
References: <1531008868-4194-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1531008868-4194-5-git-send-email-jsimmons@infradead.org>

From: NeilBrown <neilb at suse.com>

No need to have this in linux-tracefile.c

Signed-off-by: NeilBrown <neilb at suse.com>
---
 .../staging/lustre/lnet/libcfs/linux-tracefile.c   | 35 -------------
 drivers/staging/lustre/lnet/libcfs/tracefile.c     | 59 ++++++++++++++++++++++
 drivers/staging/lustre/lnet/libcfs/tracefile.h     | 28 ----------
 3 files changed, 59 insertions(+), 63 deletions(-)

diff --git a/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c b/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
index 64a5bc1..3af7722 100644
--- a/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
+++ b/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
@@ -116,41 +116,6 @@ enum cfs_trace_buf_type cfs_trace_buf_idx_get(void)
 	return CFS_TCD_TYPE_PROC;
 }
 
-/*
- * The walking argument indicates the locking comes from all tcd types
- * iterator and we must lock it and dissable local irqs to avoid deadlocks
- * with other interrupt locks that might be happening. See LU-1311
- * for details.
- */
-int cfs_trace_lock_tcd(struct cfs_trace_cpu_data *tcd, int walking)
-	__acquires(&tcd->tc_lock)
-{
-	__LASSERT(tcd->tcd_type < CFS_TCD_TYPE_MAX);
-	if (tcd->tcd_type == CFS_TCD_TYPE_IRQ)
-		spin_lock_irqsave(&tcd->tcd_lock, tcd->tcd_lock_flags);
-	else if (tcd->tcd_type == CFS_TCD_TYPE_SOFTIRQ)
-		spin_lock_bh(&tcd->tcd_lock);
-	else if (unlikely(walking))
-		spin_lock_irq(&tcd->tcd_lock);
-	else
-		spin_lock(&tcd->tcd_lock);
-	return 1;
-}
-
-void cfs_trace_unlock_tcd(struct cfs_trace_cpu_data *tcd, int walking)
-	__releases(&tcd->tcd_lock)
-{
-	__LASSERT(tcd->tcd_type < CFS_TCD_TYPE_MAX);
-	if (tcd->tcd_type == CFS_TCD_TYPE_IRQ)
-		spin_unlock_irqrestore(&tcd->tcd_lock, tcd->tcd_lock_flags);
-	else if (tcd->tcd_type == CFS_TCD_TYPE_SOFTIRQ)
-		spin_unlock_bh(&tcd->tcd_lock);
-	else if (unlikely(walking))
-		spin_unlock_irq(&tcd->tcd_lock);
-	else
-		spin_unlock(&tcd->tcd_lock);
-}
-
 void
 cfs_set_ptldebug_header(struct ptldebug_header *header,
 			struct libcfs_debug_msg_data *msgdata,
diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.c b/drivers/staging/lustre/lnet/libcfs/tracefile.c
index ae1a89c..c46dc56 100644
--- a/drivers/staging/lustre/lnet/libcfs/tracefile.c
+++ b/drivers/staging/lustre/lnet/libcfs/tracefile.c
@@ -109,6 +109,65 @@ struct cfs_trace_page {
 static void put_pages_on_tcd_daemon_list(struct page_collection *pc,
 					 struct cfs_trace_cpu_data *tcd);
 
+/* trace file lock routines */
+/*
+ * The walking argument indicates the locking comes from all tcd types
+ * iterator and we must lock it and dissable local irqs to avoid deadlocks
+ * with other interrupt locks that might be happening. See LU-1311
+ * for details.
+ */
+int cfs_trace_lock_tcd(struct cfs_trace_cpu_data *tcd, int walking)
+	__acquires(&tcd->tc_lock)
+{
+	__LASSERT(tcd->tcd_type < CFS_TCD_TYPE_MAX);
+	if (tcd->tcd_type == CFS_TCD_TYPE_IRQ)
+		spin_lock_irqsave(&tcd->tcd_lock, tcd->tcd_lock_flags);
+	else if (tcd->tcd_type == CFS_TCD_TYPE_SOFTIRQ)
+		spin_lock_bh(&tcd->tcd_lock);
+	else if (unlikely(walking))
+		spin_lock_irq(&tcd->tcd_lock);
+	else
+		spin_lock(&tcd->tcd_lock);
+	return 1;
+}
+
+void cfs_trace_unlock_tcd(struct cfs_trace_cpu_data *tcd, int walking)
+	__releases(&tcd->tcd_lock)
+{
+	__LASSERT(tcd->tcd_type < CFS_TCD_TYPE_MAX);
+	if (tcd->tcd_type == CFS_TCD_TYPE_IRQ)
+		spin_unlock_irqrestore(&tcd->tcd_lock, tcd->tcd_lock_flags);
+	else if (tcd->tcd_type == CFS_TCD_TYPE_SOFTIRQ)
+		spin_unlock_bh(&tcd->tcd_lock);
+	else if (unlikely(walking))
+		spin_unlock_irq(&tcd->tcd_lock);
+	else
+		spin_unlock(&tcd->tcd_lock);
+}
+
+#define cfs_tcd_for_each_type_lock(tcd, i, cpu)			   \
+	for (i = 0; cfs_trace_data[i] &&				\
+	     (tcd = &(*cfs_trace_data[i])[cpu].tcd) &&			\
+	     cfs_trace_lock_tcd(tcd, 1); cfs_trace_unlock_tcd(tcd, 1), i++)
+
+static inline struct cfs_trace_cpu_data *
+cfs_trace_get_tcd(void)
+{
+	struct cfs_trace_cpu_data *tcd =
+		&(*cfs_trace_data[cfs_trace_buf_idx_get()])[get_cpu()].tcd;
+
+	cfs_trace_lock_tcd(tcd, 0);
+
+	return tcd;
+}
+
+static inline void cfs_trace_put_tcd(struct cfs_trace_cpu_data *tcd)
+{
+	cfs_trace_unlock_tcd(tcd, 0);
+
+	put_cpu();
+}
+
 static inline struct cfs_trace_page *
 cfs_tage_from_list(struct list_head *list)
 {
diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.h b/drivers/staging/lustre/lnet/libcfs/tracefile.h
index bd1a1ef..91968cf 100644
--- a/drivers/staging/lustre/lnet/libcfs/tracefile.h
+++ b/drivers/staging/lustre/lnet/libcfs/tracefile.h
@@ -49,8 +49,6 @@ enum cfs_trace_buf_type {
 	CFS_TCD_TYPE_MAX
 };
 
-/* trace file lock routines */
-
 #define TRACEFILE_NAME_SIZE 1024
 extern char cfs_tracefile[TRACEFILE_NAME_SIZE];
 extern long long cfs_tracefile_size;
@@ -194,11 +192,6 @@ int cfs_trace_copyout_string(char __user *usr_buffer, int usr_buffer_nob,
 		     j < num_possible_cpus();				 \
 		     j++, (tcd) = &(*cfs_trace_data[i])[j].tcd)
 
-#define cfs_tcd_for_each_type_lock(tcd, i, cpu)			   \
-	for (i = 0; cfs_trace_data[i] &&				\
-	     (tcd = &(*cfs_trace_data[i])[cpu].tcd) &&			\
-	     cfs_trace_lock_tcd(tcd, 1); cfs_trace_unlock_tcd(tcd, 1), i++)
-
 void cfs_set_ptldebug_header(struct ptldebug_header *header,
 			     struct libcfs_debug_msg_data *m,
 			     unsigned long stack);
@@ -206,9 +199,6 @@ void cfs_print_to_console(struct ptldebug_header *hdr, int mask,
 			  const char *buf, int len, const char *file,
 			  const char *fn);
 
-int cfs_trace_lock_tcd(struct cfs_trace_cpu_data *tcd, int walking);
-void cfs_trace_unlock_tcd(struct cfs_trace_cpu_data *tcd, int walking);
-
 extern char *cfs_trace_console_buffers[NR_CPUS][CFS_TCD_TYPE_MAX];
 enum cfs_trace_buf_type cfs_trace_buf_idx_get(void);
 
@@ -221,24 +211,6 @@ void cfs_print_to_console(struct ptldebug_header *hdr, int mask,
 	return cfs_trace_console_buffers[i][j];
 }
 
-static inline struct cfs_trace_cpu_data *
-cfs_trace_get_tcd(void)
-{
-	struct cfs_trace_cpu_data *tcd =
-		&(*cfs_trace_data[cfs_trace_buf_idx_get()])[get_cpu()].tcd;
-
-	cfs_trace_lock_tcd(tcd, 0);
-
-	return tcd;
-}
-
-static inline void cfs_trace_put_tcd(struct cfs_trace_cpu_data *tcd)
-{
-	cfs_trace_unlock_tcd(tcd, 0);
-
-	put_cpu();
-}
-
 int cfs_trace_refill_stock(struct cfs_trace_cpu_data *tcd, gfp_t gfp,
 			   struct list_head *stock);
 
-- 
1.8.3.1


From jsimmons at infradead.org  Sun Jul  8 00:14:19 2018
From: jsimmons at infradead.org (James Simmons)
Date: Sat,  7 Jul 2018 20:14:19 -0400
Subject: [lustre-devel] [PATCH v4 05/14] lustre: libcfs: properly handle
 failure paths in cfs_tracefile_init_arch()
In-Reply-To: <1531008868-4194-1-git-send-email-jsimmons@infradead.org>
References: <1531008868-4194-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1531008868-4194-6-git-send-email-jsimmons@infradead.org>

It is considered poor coding style in the linux kernel to call a
catch all for cleanup in the initialization function. Also their
is no reason to cleanup cfs_trace_console_buffers if they never
been allocated. Lets unroll the error handling.

Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/lnet/libcfs/linux-tracefile.c | 19 ++++++++++++++-----
 1 file changed, 14 insertions(+), 5 deletions(-)

diff --git a/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c b/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
index 3af7722..7c864d0 100644
--- a/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
+++ b/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
@@ -49,9 +49,9 @@
 
 int cfs_tracefile_init_arch(void)
 {
+	struct cfs_trace_cpu_data *tcd;
 	int i;
 	int j;
-	struct cfs_trace_cpu_data *tcd;
 
 	/* initialize trace_data */
 	memset(cfs_trace_data, 0, sizeof(cfs_trace_data));
@@ -61,7 +61,7 @@ int cfs_tracefile_init_arch(void)
 				      sizeof(union cfs_trace_data_union),
 				      GFP_KERNEL);
 		if (!cfs_trace_data[i])
-			goto out;
+			goto out_trace_data;
 	}
 
 	/* arch related info initialized */
@@ -79,13 +79,22 @@ int cfs_tracefile_init_arch(void)
 					GFP_KERNEL);
 
 			if (!cfs_trace_console_buffers[i][j])
-				goto out;
+				goto out_buffers;
 		}
 
 	return 0;
 
-out:
-	cfs_tracefile_fini_arch();
+out_buffers:
+	for (i = 0; i < num_possible_cpus(); i++)
+		for (j = 0; j < 3; j++) {
+			kfree(cfs_trace_console_buffers[i][j]);
+			cfs_trace_console_buffers[i][j] = NULL;
+		}
+out_trace_data:
+	for (i = 0; cfs_trace_data[i]; i++) {
+		kfree(cfs_trace_data[i]);
+		cfs_trace_data[i] = NULL;
+	}
 	pr_err("lnet: Not enough memory\n");
 	return -ENOMEM;
 }
-- 
1.8.3.1


From jsimmons at infradead.org  Sun Jul  8 00:14:20 2018
From: jsimmons at infradead.org (James Simmons)
Date: Sat,  7 Jul 2018 20:14:20 -0400
Subject: [lustre-devel] [PATCH v4 06/14] lustre: libcfs: fix
 cfs_print_to_console()
In-Reply-To: <1531008868-4194-1-git-send-email-jsimmons@infradead.org>
References: <1531008868-4194-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1531008868-4194-7-git-send-email-jsimmons@infradead.org>

The original code for cfs_print_to_console() used printk() and
used tricks to select which printk level to use. Later a cleanup
patch landed the just converted printk directly to pr_info which
is not exactly correct. Instead of converting back to printk lets
move everything to pr_* type functions which simplify the code.
This allows us to fold both dbghdr_to_err_string() and the
function dbghdr_to_info_string() into cfs_print_to_console().

Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 .../staging/lustre/lnet/libcfs/linux-tracefile.c   | 69 ++++++++--------------
 1 file changed, 26 insertions(+), 43 deletions(-)

diff --git a/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c b/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
index 7c864d0..ef26835 100644
--- a/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
+++ b/drivers/staging/lustre/lnet/libcfs/linux-tracefile.c
@@ -149,55 +149,38 @@ enum cfs_trace_buf_type cfs_trace_buf_idx_get(void)
 	header->ph_extern_pid = 0;
 }
 
-static char *
-dbghdr_to_err_string(struct ptldebug_header *hdr)
-{
-	switch (hdr->ph_subsys) {
-	case S_LND:
-	case S_LNET:
-		return "LNetError";
-	default:
-		return "LustreError";
-	}
-}
-
-static char *
-dbghdr_to_info_string(struct ptldebug_header *hdr)
-{
-	switch (hdr->ph_subsys) {
-	case S_LND:
-	case S_LNET:
-		return "LNet";
-	default:
-		return "Lustre";
-	}
-}
-
 void cfs_print_to_console(struct ptldebug_header *hdr, int mask,
 			  const char *buf, int len, const char *file,
 			  const char *fn)
 {
-	char *prefix = "Lustre", *ptype = NULL;
-
-	if (mask & D_EMERG) {
-		prefix = dbghdr_to_err_string(hdr);
-		ptype = KERN_EMERG;
-	} else if (mask & D_ERROR) {
-		prefix = dbghdr_to_err_string(hdr);
-		ptype = KERN_ERR;
-	} else if (mask & D_WARNING) {
-		prefix = dbghdr_to_info_string(hdr);
-		ptype = KERN_WARNING;
-	} else if (mask & (D_CONSOLE | libcfs_printk)) {
-		prefix = dbghdr_to_info_string(hdr);
-		ptype = KERN_INFO;
-	}
+	char *prefix = "Lustre";
+
+	if (hdr->ph_subsys == S_LND || hdr->ph_subsys == S_LNET)
+		prefix = "LNet";
 
 	if (mask & D_CONSOLE) {
-		pr_info("%s%s: %.*s", ptype, prefix, len, buf);
+		if (mask & D_EMERG)
+			pr_emerg("%sError: %.*s", prefix, len, buf);
+		else if (mask & D_ERROR)
+			pr_err("%sError: %.*s", prefix, len, buf);
+		else if (mask & D_WARNING)
+			pr_warn("%s: %.*s", prefix, len, buf);
+		else if (mask & libcfs_printk)
+			pr_info("%s: %.*s", prefix, len, buf);
 	} else {
-		pr_info("%s%s: %d:%d:(%s:%d:%s()) %.*s", ptype, prefix,
-			hdr->ph_pid, hdr->ph_extern_pid, file,
-			hdr->ph_line_num, fn, len, buf);
+		if (mask & D_EMERG)
+			pr_emerg("%sError: %d:%d:(%s:%d:%s()) %.*s", prefix,
+				 hdr->ph_pid, hdr->ph_extern_pid, file,
+				 hdr->ph_line_num, fn, len, buf);
+		else if (mask & D_ERROR)
+			pr_err("%sError: %d:%d:(%s:%d:%s()) %.*s", prefix,
+			       hdr->ph_pid, hdr->ph_extern_pid, file,
+			       hdr->ph_line_num, fn, len, buf);
+		else if (mask & D_WARNING)
+			pr_warn("%s: %d:%d:(%s:%d:%s()) %.*s", prefix,
+				hdr->ph_pid, hdr->ph_extern_pid, file,
+				hdr->ph_line_num, fn, len, buf);
+		else if (mask & (D_CONSOLE | libcfs_printk))
+			pr_info("%s: %.*s", prefix, len, buf);
 	}
 }
-- 
1.8.3.1


From jsimmons at infradead.org  Sun Jul  8 00:14:24 2018
From: jsimmons at infradead.org (James Simmons)
Date: Sat,  7 Jul 2018 20:14:24 -0400
Subject: [lustre-devel] [PATCH v4 10/14] lustre: libcfs: cleanup tracefile.h
In-Reply-To: <1531008868-4194-1-git-send-email-jsimmons@infradead.org>
References: <1531008868-4194-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1531008868-4194-11-git-send-email-jsimmons@infradead.org>

With many things moved into tracefile.c we can cleanup a lot of
things in tracefile.h. Move some items that are only used in
tracefile.c from tracefile.h into tracefile.c. In tracefile.h
we have the ifdef LUSTRE_TRACEFILE_PRIVATE which has several
duplicate defines. We can remove those duplicates.

Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/lnet/libcfs/tracefile.c | 18 +++++++++++--
 drivers/staging/lustre/lnet/libcfs/tracefile.h | 37 --------------------------
 2 files changed, 16 insertions(+), 39 deletions(-)

diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.c b/drivers/staging/lustre/lnet/libcfs/tracefile.c
index d80685c..a3fde61 100644
--- a/drivers/staging/lustre/lnet/libcfs/tracefile.c
+++ b/drivers/staging/lustre/lnet/libcfs/tracefile.c
@@ -37,8 +37,6 @@
  */
 
 #define DEBUG_SUBSYSTEM S_LNET
-#define LUSTRE_TRACEFILE_PRIVATE
-#define pr_fmt(fmt) "Lustre: " fmt
 
 #include <linux/ratelimit.h>
 #include <linux/highmem.h>
@@ -49,8 +47,16 @@
 #include <linux/uaccess.h>
 #include "tracefile.h"
 
+#define CFS_TRACE_CONSOLE_BUFFER_SIZE	1024
 #define TCD_MAX_TYPES			8
 
+enum cfs_trace_buf_type {
+	CFS_TCD_TYPE_PROC = 0,
+	CFS_TCD_TYPE_SOFTIRQ,
+	CFS_TCD_TYPE_IRQ,
+	CFS_TCD_TYPE_MAX
+};
+
 union cfs_trace_data_union (*cfs_trace_data[TCD_MAX_TYPES])[NR_CPUS] __cacheline_aligned;
 
 char *cfs_trace_console_buffers[NR_CPUS][CFS_TCD_TYPE_MAX];
@@ -168,6 +174,14 @@ enum cfs_trace_buf_type cfs_trace_buf_idx_get(void)
 	return CFS_TCD_TYPE_PROC;
 }
 
+static inline char *cfs_trace_get_console_buffer(void)
+{
+	unsigned int i = get_cpu();
+	unsigned int j = cfs_trace_buf_idx_get();
+
+	return cfs_trace_console_buffers[i][j];
+}
+
 static inline struct cfs_trace_cpu_data *
 cfs_trace_get_tcd(void)
 {
diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.h b/drivers/staging/lustre/lnet/libcfs/tracefile.h
index 072c720..3e7b6fa 100644
--- a/drivers/staging/lustre/lnet/libcfs/tracefile.h
+++ b/drivers/staging/lustre/lnet/libcfs/tracefile.h
@@ -42,13 +42,6 @@
 #include <linux/smp.h>
 #include <linux/libcfs/libcfs.h>
 
-enum cfs_trace_buf_type {
-	CFS_TCD_TYPE_PROC = 0,
-	CFS_TCD_TYPE_SOFTIRQ,
-	CFS_TCD_TYPE_IRQ,
-	CFS_TCD_TYPE_MAX
-};
-
 #define TRACEFILE_NAME_SIZE 1024
 extern char cfs_tracefile[TRACEFILE_NAME_SIZE];
 extern long long cfs_tracefile_size;
@@ -91,22 +84,6 @@ int cfs_trace_copyout_string(char __user *usr_buffer, int usr_buffer_nob,
 #define TCD_STOCK_PAGES (TCD_MAX_PAGES)
 #define CFS_TRACEFILE_SIZE (500 << 20)
 
-#ifdef LUSTRE_TRACEFILE_PRIVATE
-
-/*
- * Private declare for tracefile
- */
-#define TCD_MAX_PAGES (5 << (20 - PAGE_SHIFT))
-#define TCD_STOCK_PAGES (TCD_MAX_PAGES)
-
-#define CFS_TRACEFILE_SIZE (500 << 20)
-
-/*
- * Size of a buffer for sprinting console messages if we can't get a page
- * from system
- */
-#define CFS_TRACE_CONSOLE_BUFFER_SIZE   1024
-
 union cfs_trace_data_union {
 	struct cfs_trace_cpu_data {
 		/*
@@ -183,18 +160,6 @@ int cfs_trace_copyout_string(char __user *usr_buffer, int usr_buffer_nob,
 	char __pad[L1_CACHE_ALIGN(sizeof(struct cfs_trace_cpu_data))];
 };
 
-extern char *cfs_trace_console_buffers[NR_CPUS][CFS_TCD_TYPE_MAX];
-enum cfs_trace_buf_type cfs_trace_buf_idx_get(void);
-
-static inline char *
-cfs_trace_get_console_buffer(void)
-{
-	unsigned int i = get_cpu();
-	unsigned int j = cfs_trace_buf_idx_get();
-
-	return cfs_trace_console_buffers[i][j];
-}
-
 void cfs_trace_assertion_failed(const char *str,
 				struct libcfs_debug_msg_data *m);
 
@@ -216,6 +181,4 @@ void cfs_trace_assertion_failed(const char *str,
 	__LASSERT(page_count(tage->page) > 0);		      \
 } while (0)
 
-#endif	/* LUSTRE_TRACEFILE_PRIVATE */
-
 #endif /* __LIBCFS_TRACEFILE_H__ */
-- 
1.8.3.1


From jsimmons at infradead.org  Sun Jul  8 00:14:26 2018
From: jsimmons at infradead.org (James Simmons)
Date: Sat,  7 Jul 2018 20:14:26 -0400
Subject: [lustre-devel] [PATCH v4 12/14] lustre: libcfs: renamed
 CFS_TCD_TYPE_MAX to CFS_TCD_TYPE_CNT
In-Reply-To: <1531008868-4194-1-git-send-email-jsimmons@infradead.org>
References: <1531008868-4194-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1531008868-4194-13-git-send-email-jsimmons@infradead.org>

From: NeilBrown <neilb at suse.com>

The possible TCD types are 0, 1, 2.
So the MAX is 2.
The count of the number of types is 3.

CFS_TCD_TYPE_MAX is 3 - obviously wrong.

So rename it to CFS_TCD_TYPE_CNT.

Also there are 2 places where "3" is used rather
than the macro - fix them.

Signed-off-by: NeilBrown <neilb at suse.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/lnet/libcfs/tracefile.c | 16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)

diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.c b/drivers/staging/lustre/lnet/libcfs/tracefile.c
index 282b7f6..bf6bcd5 100644
--- a/drivers/staging/lustre/lnet/libcfs/tracefile.c
+++ b/drivers/staging/lustre/lnet/libcfs/tracefile.c
@@ -54,12 +54,12 @@ enum cfs_trace_buf_type {
 	CFS_TCD_TYPE_PROC = 0,
 	CFS_TCD_TYPE_SOFTIRQ,
 	CFS_TCD_TYPE_IRQ,
-	CFS_TCD_TYPE_MAX
+	CFS_TCD_TYPE_CNT
 };
 
 union cfs_trace_data_union (*cfs_trace_data[TCD_MAX_TYPES])[NR_CPUS] __cacheline_aligned;
 
-char *cfs_trace_console_buffers[NR_CPUS][CFS_TCD_TYPE_MAX];
+char *cfs_trace_console_buffers[NR_CPUS][CFS_TCD_TYPE_CNT];
 char cfs_tracefile[TRACEFILE_NAME_SIZE];
 long long cfs_tracefile_size = CFS_TRACEFILE_SIZE;
 static struct tracefiled_ctl trace_tctl;
@@ -127,7 +127,7 @@ static void put_pages_on_tcd_daemon_list(struct page_collection *pc,
 int cfs_trace_lock_tcd(struct cfs_trace_cpu_data *tcd, int walking)
 	__acquires(&tcd->tc_lock)
 {
-	__LASSERT(tcd->tcd_type < CFS_TCD_TYPE_MAX);
+	__LASSERT(tcd->tcd_type < CFS_TCD_TYPE_CNT);
 	if (tcd->tcd_type == CFS_TCD_TYPE_IRQ)
 		spin_lock_irqsave(&tcd->tcd_lock, tcd->tcd_lock_flags);
 	else if (tcd->tcd_type == CFS_TCD_TYPE_SOFTIRQ)
@@ -142,7 +142,7 @@ int cfs_trace_lock_tcd(struct cfs_trace_cpu_data *tcd, int walking)
 void cfs_trace_unlock_tcd(struct cfs_trace_cpu_data *tcd, int walking)
 	__releases(&tcd->tcd_lock)
 {
-	__LASSERT(tcd->tcd_type < CFS_TCD_TYPE_MAX);
+	__LASSERT(tcd->tcd_type < CFS_TCD_TYPE_CNT);
 	if (tcd->tcd_type == CFS_TCD_TYPE_IRQ)
 		spin_unlock_irqrestore(&tcd->tcd_lock, tcd->tcd_lock_flags);
 	else if (tcd->tcd_type == CFS_TCD_TYPE_SOFTIRQ)
@@ -1259,7 +1259,7 @@ void cfs_trace_stop_thread(void)
 }
 
 /* percents to share the total debug memory for each type */
-static unsigned int pages_factor[CFS_TCD_TYPE_MAX] = {
+static unsigned int pages_factor[CFS_TCD_TYPE_CNT] = {
 	80,  /* 80% pages for CFS_TCD_TYPE_PROC */
 	10,  /* 10% pages for CFS_TCD_TYPE_SOFTIRQ */
 	10   /* 10% pages for CFS_TCD_TYPE_IRQ */
@@ -1273,7 +1273,7 @@ int cfs_tracefile_init(int max_pages)
 
 	/* initialize trace_data */
 	memset(cfs_trace_data, 0, sizeof(cfs_trace_data));
-	for (i = 0; i < CFS_TCD_TYPE_MAX; i++) {
+	for (i = 0; i < CFS_TCD_TYPE_CNT; i++) {
 		cfs_trace_data[i] =
 			kmalloc_array(num_possible_cpus(),
 				      sizeof(union cfs_trace_data_union),
@@ -1302,7 +1302,7 @@ int cfs_tracefile_init(int max_pages)
 	}
 
 	for (i = 0; i < num_possible_cpus(); i++)
-		for (j = 0; j < 3; j++) {
+		for (j = 0; j < CFS_TCD_TYPE_CNT; j++) {
 			cfs_trace_console_buffers[i][j] =
 				kmalloc(CFS_TRACE_CONSOLE_BUFFER_SIZE,
 					GFP_KERNEL);
@@ -1366,7 +1366,7 @@ static void cfs_trace_cleanup(void)
 	trace_cleanup_on_all_cpus();
 
 	for (i = 0; i < num_possible_cpus(); i++)
-		for (j = 0; j < 3; j++) {
+		for (j = 0; j < CFS_TCD_TYPE_CNT; j++) {
 			kfree(cfs_trace_console_buffers[i][j]);
 			cfs_trace_console_buffers[i][j] = NULL;
 		}
-- 
1.8.3.1


From jsimmons at infradead.org  Sun Jul  8 00:14:25 2018
From: jsimmons at infradead.org (James Simmons)
Date: Sat,  7 Jul 2018 20:14:25 -0400
Subject: [lustre-devel] [PATCH v4 11/14] lustre: libcfs: fold
 cfs_tracefile_*_arch into their only callers.
In-Reply-To: <1531008868-4194-1-git-send-email-jsimmons@infradead.org>
References: <1531008868-4194-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1531008868-4194-12-git-send-email-jsimmons@infradead.org>

From: NeilBrown <neilb at suse.com>

There is no need to separate "arch" init/fini from
the rest, so fold it all in.
This requires some slightly subtle changes to clean-up
to make sure we don't walk lists before they are
initialized.

Signed-off-by: NeilBrown <neilb at suse.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/lnet/libcfs/tracefile.c | 147 +++++++++++--------------
 drivers/staging/lustre/lnet/libcfs/tracefile.h |   3 -
 2 files changed, 63 insertions(+), 87 deletions(-)

diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.c b/drivers/staging/lustre/lnet/libcfs/tracefile.c
index a3fde61..282b7f6 100644
--- a/drivers/staging/lustre/lnet/libcfs/tracefile.c
+++ b/drivers/staging/lustre/lnet/libcfs/tracefile.c
@@ -200,82 +200,6 @@ static inline void cfs_trace_put_tcd(struct cfs_trace_cpu_data *tcd)
 	put_cpu();
 }
 
-/* percents to share the total debug memory for each type */
-static unsigned int pages_factor[CFS_TCD_TYPE_MAX] = {
-	80,  /* 80% pages for CFS_TCD_TYPE_PROC */
-	10,  /* 10% pages for CFS_TCD_TYPE_SOFTIRQ */
-	10   /* 10% pages for CFS_TCD_TYPE_IRQ */
-};
-
-int cfs_tracefile_init_arch(void)
-{
-	struct cfs_trace_cpu_data *tcd;
-	int i;
-	int j;
-
-	/* initialize trace_data */
-	memset(cfs_trace_data, 0, sizeof(cfs_trace_data));
-	for (i = 0; i < CFS_TCD_TYPE_MAX; i++) {
-		cfs_trace_data[i] =
-			kmalloc_array(num_possible_cpus(),
-				      sizeof(union cfs_trace_data_union),
-				      GFP_KERNEL);
-		if (!cfs_trace_data[i])
-			goto out_trace_data;
-	}
-
-	/* arch related info initialized */
-	cfs_tcd_for_each(tcd, i, j) {
-		spin_lock_init(&tcd->tcd_lock);
-		tcd->tcd_pages_factor = pages_factor[i];
-		tcd->tcd_type = i;
-		tcd->tcd_cpu = j;
-	}
-
-	for (i = 0; i < num_possible_cpus(); i++)
-		for (j = 0; j < 3; j++) {
-			cfs_trace_console_buffers[i][j] =
-				kmalloc(CFS_TRACE_CONSOLE_BUFFER_SIZE,
-					GFP_KERNEL);
-
-			if (!cfs_trace_console_buffers[i][j])
-				goto out_buffers;
-		}
-
-	return 0;
-
-out_buffers:
-	for (i = 0; i < num_possible_cpus(); i++)
-		for (j = 0; j < 3; j++) {
-			kfree(cfs_trace_console_buffers[i][j]);
-			cfs_trace_console_buffers[i][j] = NULL;
-		}
-out_trace_data:
-	for (i = 0; cfs_trace_data[i]; i++) {
-		kfree(cfs_trace_data[i]);
-		cfs_trace_data[i] = NULL;
-	}
-	pr_err("lnet: Not enough memory\n");
-	return -ENOMEM;
-}
-
-void cfs_tracefile_fini_arch(void)
-{
-	int i;
-	int j;
-
-	for (i = 0; i < num_possible_cpus(); i++)
-		for (j = 0; j < 3; j++) {
-			kfree(cfs_trace_console_buffers[i][j]);
-			cfs_trace_console_buffers[i][j] = NULL;
-		}
-
-	for (i = 0; cfs_trace_data[i]; i++) {
-		kfree(cfs_trace_data[i]);
-		cfs_trace_data[i] = NULL;
-	}
-}
-
 static inline struct cfs_trace_page *
 cfs_tage_from_list(struct list_head *list)
 {
@@ -1334,21 +1258,38 @@ void cfs_trace_stop_thread(void)
 	mutex_unlock(&cfs_trace_thread_mutex);
 }
 
+/* percents to share the total debug memory for each type */
+static unsigned int pages_factor[CFS_TCD_TYPE_MAX] = {
+	80,  /* 80% pages for CFS_TCD_TYPE_PROC */
+	10,  /* 10% pages for CFS_TCD_TYPE_SOFTIRQ */
+	10   /* 10% pages for CFS_TCD_TYPE_IRQ */
+};
+
 int cfs_tracefile_init(int max_pages)
 {
 	struct cfs_trace_cpu_data *tcd;
 	int i;
 	int j;
-	int rc;
-	int factor;
 
-	rc = cfs_tracefile_init_arch();
-	if (rc)
-		return rc;
+	/* initialize trace_data */
+	memset(cfs_trace_data, 0, sizeof(cfs_trace_data));
+	for (i = 0; i < CFS_TCD_TYPE_MAX; i++) {
+		cfs_trace_data[i] =
+			kmalloc_array(num_possible_cpus(),
+				      sizeof(union cfs_trace_data_union),
+				      GFP_KERNEL);
+		if (!cfs_trace_data[i])
+			goto out_trace_data;
+	}
 
+	/* arch related info initialized */
 	cfs_tcd_for_each(tcd, i, j) {
-		/* tcd_pages_factor is initialized int tracefile_init_arch. */
-		factor = tcd->tcd_pages_factor;
+		int factor = pages_factor[i];
+
+		spin_lock_init(&tcd->tcd_lock);
+		tcd->tcd_pages_factor = factor;
+		tcd->tcd_type = i;
+		tcd->tcd_cpu = j;
 		INIT_LIST_HEAD(&tcd->tcd_pages);
 		INIT_LIST_HEAD(&tcd->tcd_stock_pages);
 		INIT_LIST_HEAD(&tcd->tcd_daemon_pages);
@@ -1360,7 +1301,31 @@ int cfs_tracefile_init(int max_pages)
 		tcd->tcd_shutting_down = 0;
 	}
 
+	for (i = 0; i < num_possible_cpus(); i++)
+		for (j = 0; j < 3; j++) {
+			cfs_trace_console_buffers[i][j] =
+				kmalloc(CFS_TRACE_CONSOLE_BUFFER_SIZE,
+					GFP_KERNEL);
+
+			if (!cfs_trace_console_buffers[i][j])
+				goto out_buffers;
+		}
+
 	return 0;
+
+out_buffers:
+	for (i = 0; i < num_possible_cpus(); i++)
+		for (j = 0; j < 3; j++) {
+			kfree(cfs_trace_console_buffers[i][j]);
+			cfs_trace_console_buffers[i][j] = NULL;
+		}
+out_trace_data:
+	for (i = 0; cfs_trace_data[i]; i++) {
+		kfree(cfs_trace_data[i]);
+		cfs_trace_data[i] = NULL;
+	}
+	pr_err("lnet: Not enough memory\n");
+	return -ENOMEM;
 }
 
 static void trace_cleanup_on_all_cpus(void)
@@ -1372,6 +1337,9 @@ static void trace_cleanup_on_all_cpus(void)
 
 	for_each_possible_cpu(cpu) {
 		cfs_tcd_for_each_type_lock(tcd, i, cpu) {
+			if (!tcd->tcd_pages_factor)
+				/* Not initialised */
+				continue;
 			tcd->tcd_shutting_down = 1;
 
 			list_for_each_entry_safe(tage, tmp, &tcd->tcd_pages,
@@ -1390,12 +1358,23 @@ static void trace_cleanup_on_all_cpus(void)
 static void cfs_trace_cleanup(void)
 {
 	struct page_collection pc;
+	int i;
+	int j;
 
 	INIT_LIST_HEAD(&pc.pc_pages);
 
 	trace_cleanup_on_all_cpus();
 
-	cfs_tracefile_fini_arch();
+	for (i = 0; i < num_possible_cpus(); i++)
+		for (j = 0; j < 3; j++) {
+			kfree(cfs_trace_console_buffers[i][j]);
+			cfs_trace_console_buffers[i][j] = NULL;
+		}
+
+	for (i = 0; cfs_trace_data[i]; i++) {
+		kfree(cfs_trace_data[i]);
+		cfs_trace_data[i] = NULL;
+	}
 }
 
 void cfs_tracefile_exit(void)
diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.h b/drivers/staging/lustre/lnet/libcfs/tracefile.h
index 3e7b6fa..dc782a6 100644
--- a/drivers/staging/lustre/lnet/libcfs/tracefile.h
+++ b/drivers/staging/lustre/lnet/libcfs/tracefile.h
@@ -53,9 +53,6 @@
 
 void libcfs_run_debug_log_upcall(char *file);
 
-int  cfs_tracefile_init_arch(void);
-void cfs_tracefile_fini_arch(void);
-
 int cfs_tracefile_dump_all_pages(char *filename);
 void cfs_trace_debug_print(void);
 void cfs_trace_flush_pages(void);
-- 
1.8.3.1


From jsimmons at infradead.org  Sun Jul  8 00:14:27 2018
From: jsimmons at infradead.org (James Simmons)
Date: Sat,  7 Jul 2018 20:14:27 -0400
Subject: [lustre-devel] [PATCH v4 13/14] lustre: libcfs: discard
 TCD_MAX_TYPES
In-Reply-To: <1531008868-4194-1-git-send-email-jsimmons@infradead.org>
References: <1531008868-4194-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1531008868-4194-14-git-send-email-jsimmons@infradead.org>

From: NeilBrown <neilb at suse.com>

As well as CFS_TCD_TYPE_CNT we have TCD_MAX_TYPES which has a larger
value but a similar meaning.  Discard it and just use
CFS_TCD_TYPE_CNT.

Two places relied on the fact that TCD_MAX_TYPES was larger and so
there would be NULLs at the end of the array.  Change
them to check the array size properly.

Signed-off-by: NeilBrown <neilb at suse.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/lnet/libcfs/tracefile.c | 9 ++++-----
 1 file changed, 4 insertions(+), 5 deletions(-)

diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.c b/drivers/staging/lustre/lnet/libcfs/tracefile.c
index bf6bcd5..07242d4e 100644
--- a/drivers/staging/lustre/lnet/libcfs/tracefile.c
+++ b/drivers/staging/lustre/lnet/libcfs/tracefile.c
@@ -48,7 +48,6 @@
 #include "tracefile.h"
 
 #define CFS_TRACE_CONSOLE_BUFFER_SIZE	1024
-#define TCD_MAX_TYPES			8
 
 enum cfs_trace_buf_type {
 	CFS_TCD_TYPE_PROC = 0,
@@ -57,7 +56,7 @@ enum cfs_trace_buf_type {
 	CFS_TCD_TYPE_CNT
 };
 
-union cfs_trace_data_union (*cfs_trace_data[TCD_MAX_TYPES])[NR_CPUS] __cacheline_aligned;
+union cfs_trace_data_union (*cfs_trace_data[CFS_TCD_TYPE_CNT])[NR_CPUS] __cacheline_aligned;
 
 char *cfs_trace_console_buffers[NR_CPUS][CFS_TCD_TYPE_CNT];
 char cfs_tracefile[TRACEFILE_NAME_SIZE];
@@ -154,14 +153,14 @@ void cfs_trace_unlock_tcd(struct cfs_trace_cpu_data *tcd, int walking)
 }
 
 #define cfs_tcd_for_each(tcd, i, j)					\
-	for (i = 0; cfs_trace_data[i]; i++)				\
+	for (i = 0; i < CFS_TCD_TYPE_CNT && cfs_trace_data[i]; i++)	\
 		for (j = 0, ((tcd) = &(*cfs_trace_data[i])[j].tcd);	\
 		     j < num_possible_cpus();				\
 		     j++, (tcd) = &(*cfs_trace_data[i])[j].tcd)
 
 
 #define cfs_tcd_for_each_type_lock(tcd, i, cpu)				\
-	for (i = 0; cfs_trace_data[i] &&				\
+	for (i = 0;  i < CFS_TCD_TYPE_CNT && cfs_trace_data[i] &&	\
 	     (tcd = &(*cfs_trace_data[i])[cpu].tcd) &&			\
 	     cfs_trace_lock_tcd(tcd, 1); cfs_trace_unlock_tcd(tcd, 1), i++)
 
@@ -1371,7 +1370,7 @@ static void cfs_trace_cleanup(void)
 			cfs_trace_console_buffers[i][j] = NULL;
 		}
 
-	for (i = 0; cfs_trace_data[i]; i++) {
+	for (i = 0; i < CFS_TCD_TYPE_CNT && cfs_trace_data[i]; i++) {
 		kfree(cfs_trace_data[i]);
 		cfs_trace_data[i] = NULL;
 	}
-- 
1.8.3.1


From jsimmons at infradead.org  Sun Jul  8 00:14:28 2018
From: jsimmons at infradead.org (James Simmons)
Date: Sat,  7 Jul 2018 20:14:28 -0400
Subject: [lustre-devel] [PATCH v4 14/14] lustre: libcfs: format macros in
 tracefile.h
In-Reply-To: <1531008868-4194-1-git-send-email-jsimmons@infradead.org>
References: <1531008868-4194-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1531008868-4194-15-git-send-email-jsimmons@infradead.org>

The __LASSERT_* macros are ugly and hard to read. This patch
beautifies them.

Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/lnet/libcfs/tracefile.h | 24 ++++++++++++------------
 1 file changed, 12 insertions(+), 12 deletions(-)

diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.h b/drivers/staging/lustre/lnet/libcfs/tracefile.h
index dc782a6..67107b1 100644
--- a/drivers/staging/lustre/lnet/libcfs/tracefile.h
+++ b/drivers/staging/lustre/lnet/libcfs/tracefile.h
@@ -161,21 +161,21 @@ void cfs_trace_assertion_failed(const char *str,
 				struct libcfs_debug_msg_data *m);
 
 /* ASSERTION that is safe to use within the debug system */
-#define __LASSERT(cond)						 \
-do {								    \
+#define __LASSERT(cond)							\
+do {									\
 	if (unlikely(!(cond))) {					\
-		LIBCFS_DEBUG_MSG_DATA_DECL(msgdata, D_EMERG, NULL);     \
-		cfs_trace_assertion_failed("ASSERTION("#cond") failed", \
-					   &msgdata);		   \
-	}							       \
+		LIBCFS_DEBUG_MSG_DATA_DECL(msgdata, D_EMERG, NULL);	\
+		cfs_trace_assertion_failed("ASSERTION("#cond") failed",	\
+					   &msgdata);			\
+	}								\
 } while (0)
 
-#define __LASSERT_TAGE_INVARIANT(tage)				  \
-do {								    \
-	__LASSERT(tage);					\
-	__LASSERT(tage->page);				  \
-	__LASSERT(tage->used <= PAGE_SIZE);			 \
-	__LASSERT(page_count(tage->page) > 0);		      \
+#define __LASSERT_TAGE_INVARIANT(tage)			\
+do {							\
+	__LASSERT(tage);				\
+	__LASSERT(tage->page);				\
+	__LASSERT(tage->used <= PAGE_SIZE);		\
+	__LASSERT(page_count(tage->page) > 0);		\
 } while (0)
 
 #endif /* __LIBCFS_TRACEFILE_H__ */
-- 
1.8.3.1


From AbeA at supermicro.com  Wed Jul 18 02:23:10 2018
From: AbeA at supermicro.com (Abe Asraoui)
Date: Wed, 18 Jul 2018 02:23:10 +0000
Subject: [lustre-devel] MDT test in rel2.11
Message-ID: <547D95FD-0CF3-4F04-B727-D22513D1C4B9@supermicro.com>

Hi All,


Has anyone done any MDT testing under the latest rel2.11 and have benchmark data to share?


Thanks,
Abe
 


From paf at cray.com  Wed Jul 18 03:33:08 2018
From: paf at cray.com (Patrick Farrell)
Date: Wed, 18 Jul 2018 03:33:08 +0000
Subject: [lustre-devel] MDT test in rel2.11
In-Reply-To: <547D95FD-0CF3-4F04-B727-D22513D1C4B9@supermicro.com>
References: <547D95FD-0CF3-4F04-B727-D22513D1C4B9@supermicro.com>
Message-ID: <BN3PR1101MB12022B362116346208D0B5F2CB530@BN3PR1101MB1202.namprd11.prod.outlook.com>


Abe,

Any benchmarking would be highly dependent on hardware, both client and server.  Is there a particular comparison (say, between versions) you’re interested in or something you’re concerned about?

- Patrick

________________________________
From: lustre-devel <lustre-devel-bounces at lists.lustre.org> on behalf of Abe Asraoui <AbeA at supermicro.com>
Sent: Tuesday, July 17, 2018 9:23:10 PM
To: lustre-devel at lists.lustre.org; lustre-discuss at lists.lustre.org; Abe Asraoui
Subject: [lustre-devel] MDT test in rel2.11

Hi All,


Has anyone done any MDT testing under the latest rel2.11 and have benchmark data to share?


Thanks,
Abe


_______________________________________________
lustre-devel mailing list
lustre-devel at lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180718/d25aae2b/attachment.html>

From johnbent at gmail.com  Wed Jul 18 03:55:24 2018
From: johnbent at gmail.com (John Bent)
Date: Tue, 17 Jul 2018 23:55:24 -0400
Subject: [lustre-devel] MDT test in rel2.11
In-Reply-To: <BN3PR1101MB12022B362116346208D0B5F2CB530@BN3PR1101MB1202.namprd11.prod.outlook.com>
References: <547D95FD-0CF3-4F04-B727-D22513D1C4B9@supermicro.com>
 <BN3PR1101MB12022B362116346208D0B5F2CB530@BN3PR1101MB1202.namprd11.prod.outlook.com>
Message-ID: <CAGfFL-yEhYXiRngSm2FBOBC+VLjrnTTWcYC7Frvh_6uE2RAHiw@mail.gmail.com>

I'm curious about how DOM improves IO500 scores.  :)
Also LSOM but I don't know actually whether that's in 2.11 or where.

On Tue, Jul 17, 2018 at 11:33 PM, Patrick Farrell <paf at cray.com> wrote:

>
> Abe,
>
> Any benchmarking would be highly dependent on hardware, both client and
> server.  Is there a particular comparison (say, between versions) you’re
> interested in or something you’re concerned about?
>
> - Patrick
>
> ------------------------------
> *From:* lustre-devel <lustre-devel-bounces at lists.lustre.org> on behalf of
> Abe Asraoui <AbeA at supermicro.com>
> *Sent:* Tuesday, July 17, 2018 9:23:10 PM
> *To:* lustre-devel at lists.lustre.org; lustre-discuss at lists.lustre.org; Abe
> Asraoui
> *Subject:* [lustre-devel] MDT test in rel2.11
>
> Hi All,
>
>
> Has anyone done any MDT testing under the latest rel2.11 and have
> benchmark data to share?
>
>
> Thanks,
> Abe
>
>
> _______________________________________________
> lustre-devel mailing list
> lustre-devel at lists.lustre.org
> http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org
>
> _______________________________________________
> lustre-devel mailing list
> lustre-devel at lists.lustre.org
> http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180717/29edbd32/attachment.html>

From AbeA at supermicro.com  Wed Jul 18 04:02:21 2018
From: AbeA at supermicro.com (Abe Asraoui)
Date: Wed, 18 Jul 2018 04:02:21 +0000
Subject: [lustre-devel] MDT test in rel2.11
In-Reply-To: <BN3PR1101MB12022B362116346208D0B5F2CB530@BN3PR1101MB1202.namprd11.prod.outlook.com>
References: <547D95FD-0CF3-4F04-B727-D22513D1C4B9@supermicro.com>
 <BN3PR1101MB12022B362116346208D0B5F2CB530@BN3PR1101MB1202.namprd11.prod.outlook.com>
Message-ID: <7ABC8054-7D06-47A8-95DB-645410C09837@supermicro.com>


Hi Patrick,

I’m more interested in what improvements have been in comparaison to rel2.10,
We have seen some performance degradation in rel2.10 with zfs and wondering if this is still the case in rel2.11 with the
Inclusion of DOM feature etc..


Thanks,
Abe


From: Patrick Farrell <paf at cray.com>
Date: Tuesday, July 17, 2018 at 8:32 PM
To: "Abe Asraoui (System)" <AbeA at supermicro.com>, "lustre-devel at lists.lustre.org" <lustre-devel at lists.lustre.org>, "lustre-discuss at lists.lustre.org" <lustre-discuss at lists.lustre.org>
Subject: Re: MDT test in rel2.11


Abe,

Any benchmarking would be highly dependent on hardware, both client and server.  Is there a particular comparison (say, between versions) you’re interested in or something you’re concerned about?

- Patrick
________________________________
From: lustre-devel <lustre-devel-bounces at lists.lustre.org> on behalf of Abe Asraoui <AbeA at supermicro.com>
Sent: Tuesday, July 17, 2018 9:23:10 PM
To: lustre-devel at lists.lustre.org; lustre-discuss at lists.lustre.org; Abe Asraoui
Subject: [lustre-devel] MDT test in rel2.11

Hi All,


Has anyone done any MDT testing under the latest rel2.11 and have benchmark data to share?


Thanks,
Abe


_______________________________________________
lustre-devel mailing list
lustre-devel at lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org<https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.lustre.org_listinfo.cgi_lustre-2Ddevel-2Dlustre.org&d=DwMF-g&c=4DxX-JX0i28X6V65hK0ft5M-1rZQeWgdMry9v8-eNr4&r=uOlOifyIDx4uh-m-EmK9TzkpR6RloKTYzjsbWwVKzmc&m=T3ZcoTwhQfWPNmVUj2vjMTfPAjwLEMX2xjMD-xadokU&s=dIbxsMKeXnEpMwOKA3OGr_0KQXbMONdnqxw9ZwTvUyY&e=>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180718/37224fa4/attachment.html>

From paf at cray.com  Wed Jul 18 04:49:48 2018
From: paf at cray.com (Patrick Farrell)
Date: Wed, 18 Jul 2018 04:49:48 +0000
Subject: [lustre-devel] MDT test in rel2.11
In-Reply-To: <CAGfFL-yEhYXiRngSm2FBOBC+VLjrnTTWcYC7Frvh_6uE2RAHiw@mail.gmail.com>
References: <547D95FD-0CF3-4F04-B727-D22513D1C4B9@supermicro.com>
 <BN3PR1101MB12022B362116346208D0B5F2CB530@BN3PR1101MB1202.namprd11.prod.outlook.com>,
 <CAGfFL-yEhYXiRngSm2FBOBC+VLjrnTTWcYC7Frvh_6uE2RAHiw@mail.gmail.com>
Message-ID: <BN3PR1101MB1202842CCA5F086263EB93BCCB530@BN3PR1101MB1202.namprd11.prod.outlook.com>

Lazy SoM is not landed yet, and it won’t be improving benchmark scores - it’s never “known 100% correct”, so it can’t be used for actual POSIX ops - if a file size read out is used for a write offset, then you’ve got data corruption.

So for now it’s strictly limited to tools that know about it (accessed via an ioctl) and can accept information that may be stale.  The intended use case is scanning the FS for policy application.

________________________________
From: John Bent <johnbent at gmail.com>
Sent: Tuesday, July 17, 2018 10:55:24 PM
To: Patrick Farrell
Cc: Abe Asraoui; lustre-devel at lists.lustre.org; lustre-discuss at lists.lustre.org
Subject: Re: [lustre-devel] MDT test in rel2.11

I'm curious about how DOM improves IO500 scores.  :)
Also LSOM but I don't know actually whether that's in 2.11 or where.

On Tue, Jul 17, 2018 at 11:33 PM, Patrick Farrell <paf at cray.com<mailto:paf at cray.com>> wrote:

Abe,

Any benchmarking would be highly dependent on hardware, both client and server.  Is there a particular comparison (say, between versions) you’re interested in or something you’re concerned about?

- Patrick

________________________________
From: lustre-devel <lustre-devel-bounces at lists.lustre.org<mailto:lustre-devel-bounces at lists.lustre.org>> on behalf of Abe Asraoui <AbeA at supermicro.com<mailto:AbeA at supermicro.com>>
Sent: Tuesday, July 17, 2018 9:23:10 PM
To: lustre-devel at lists.lustre.org<mailto:lustre-devel at lists.lustre.org>; lustre-discuss at lists.lustre.org<mailto:lustre-discuss at lists.lustre.org>; Abe Asraoui
Subject: [lustre-devel] MDT test in rel2.11

Hi All,


Has anyone done any MDT testing under the latest rel2.11 and have benchmark data to share?


Thanks,
Abe


_______________________________________________
lustre-devel mailing list
lustre-devel at lists.lustre.org<mailto:lustre-devel at lists.lustre.org>
http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org

_______________________________________________
lustre-devel mailing list
lustre-devel at lists.lustre.org<mailto:lustre-devel at lists.lustre.org>
http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180718/21c12c32/attachment.html>

From paf at cray.com  Wed Jul 18 04:51:22 2018
From: paf at cray.com (Patrick Farrell)
Date: Wed, 18 Jul 2018 04:51:22 +0000
Subject: [lustre-devel] MDT test in rel2.11
In-Reply-To: <BN3PR1101MB1202842CCA5F086263EB93BCCB530@BN3PR1101MB1202.namprd11.prod.outlook.com>
References: <547D95FD-0CF3-4F04-B727-D22513D1C4B9@supermicro.com>
 <BN3PR1101MB12022B362116346208D0B5F2CB530@BN3PR1101MB1202.namprd11.prod.outlook.com>,
 <CAGfFL-yEhYXiRngSm2FBOBC+VLjrnTTWcYC7Frvh_6uE2RAHiw@mail.gmail.com>,
 <BN3PR1101MB1202842CCA5F086263EB93BCCB530@BN3PR1101MB1202.namprd11.prod.outlook.com>
Message-ID: <BN3PR1101MB120213710483B30261428144CB530@BN3PR1101MB1202.namprd11.prod.outlook.com>

To be clear in case I sound too down on it - Lazy SoM is a very nice feature that will speed up important use cases.  It’s just not going to jazz up mdtest #s.


________________________________
From: Patrick Farrell
Sent: Tuesday, July 17, 2018 11:49:48 PM
To: John Bent
Cc: Abe Asraoui; lustre-devel at lists.lustre.org; lustre-discuss at lists.lustre.org
Subject: Re: [lustre-devel] MDT test in rel2.11

Lazy SoM is not landed yet, and it won’t be improving benchmark scores - it’s never “known 100% correct”, so it can’t be used for actual POSIX ops - if a file size read out is used for a write offset, then you’ve got data corruption.

So for now it’s strictly limited to tools that know about it (accessed via an ioctl) and can accept information that may be stale.  The intended use case is scanning the FS for policy application.

________________________________
From: John Bent <johnbent at gmail.com>
Sent: Tuesday, July 17, 2018 10:55:24 PM
To: Patrick Farrell
Cc: Abe Asraoui; lustre-devel at lists.lustre.org; lustre-discuss at lists.lustre.org
Subject: Re: [lustre-devel] MDT test in rel2.11

I'm curious about how DOM improves IO500 scores.  :)
Also LSOM but I don't know actually whether that's in 2.11 or where.

On Tue, Jul 17, 2018 at 11:33 PM, Patrick Farrell <paf at cray.com<mailto:paf at cray.com>> wrote:

Abe,

Any benchmarking would be highly dependent on hardware, both client and server.  Is there a particular comparison (say, between versions) you’re interested in or something you’re concerned about?

- Patrick

________________________________
From: lustre-devel <lustre-devel-bounces at lists.lustre.org<mailto:lustre-devel-bounces at lists.lustre.org>> on behalf of Abe Asraoui <AbeA at supermicro.com<mailto:AbeA at supermicro.com>>
Sent: Tuesday, July 17, 2018 9:23:10 PM
To: lustre-devel at lists.lustre.org<mailto:lustre-devel at lists.lustre.org>; lustre-discuss at lists.lustre.org<mailto:lustre-discuss at lists.lustre.org>; Abe Asraoui
Subject: [lustre-devel] MDT test in rel2.11

Hi All,


Has anyone done any MDT testing under the latest rel2.11 and have benchmark data to share?


Thanks,
Abe


_______________________________________________
lustre-devel mailing list
lustre-devel at lists.lustre.org<mailto:lustre-devel at lists.lustre.org>
http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org

_______________________________________________
lustre-devel mailing list
lustre-devel at lists.lustre.org<mailto:lustre-devel at lists.lustre.org>
http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180718/b5109142/attachment.html>

From johnbent at gmail.com  Wed Jul 18 04:54:32 2018
From: johnbent at gmail.com (John Bent)
Date: Wed, 18 Jul 2018 00:54:32 -0400
Subject: [lustre-devel] MDT test in rel2.11
In-Reply-To: <BN3PR1101MB1202842CCA5F086263EB93BCCB530@BN3PR1101MB1202.namprd11.prod.outlook.com>
References: <547D95FD-0CF3-4F04-B727-D22513D1C4B9@supermicro.com>
 <BN3PR1101MB12022B362116346208D0B5F2CB530@BN3PR1101MB1202.namprd11.prod.outlook.com>
 <CAGfFL-yEhYXiRngSm2FBOBC+VLjrnTTWcYC7Frvh_6uE2RAHiw@mail.gmail.com>
 <BN3PR1101MB1202842CCA5F086263EB93BCCB530@BN3PR1101MB1202.namprd11.prod.outlook.com>
Message-ID: <CAGfFL-ycwu=SOOoG_3Tmg_GuyCqgpJeyjN_H5win=eMriSqRMw@mail.gmail.com>

Thanks Patrick.  That's interesting.  However, the exact motivation why
IO500 has the 'find' command is this same intended use case; stale results
therefore actually present an interesting dilemma to IO500.  They are not
POSIX compliant but that loss of compliance shouldn't necessarily
disqualify this result...

On Wed, Jul 18, 2018 at 12:49 AM, Patrick Farrell <paf at cray.com> wrote:

> Lazy SoM is not landed yet, and it won’t be improving benchmark scores -
> it’s never “known 100% correct”, so it can’t be used for actual POSIX ops -
> if a file size read out is used for a write offset, then you’ve got data
> corruption.
>
> So for now it’s strictly limited to tools that know about it (accessed via
> an ioctl) and can accept information that may be stale.  The intended use
> case is scanning the FS for policy application.
>
> ------------------------------
> *From:* John Bent <johnbent at gmail.com>
> *Sent:* Tuesday, July 17, 2018 10:55:24 PM
> *To:* Patrick Farrell
> *Cc:* Abe Asraoui; lustre-devel at lists.lustre.org;
> lustre-discuss at lists.lustre.org
> *Subject:* Re: [lustre-devel] MDT test in rel2.11
>
> I'm curious about how DOM improves IO500 scores.  :)
> Also LSOM but I don't know actually whether that's in 2.11 or where.
>
> On Tue, Jul 17, 2018 at 11:33 PM, Patrick Farrell <paf at cray.com> wrote:
>
>
> Abe,
>
> Any benchmarking would be highly dependent on hardware, both client and
> server.  Is there a particular comparison (say, between versions) you’re
> interested in or something you’re concerned about?
>
> - Patrick
>
> ------------------------------
> *From:* lustre-devel <lustre-devel-bounces at lists.lustre.org> on behalf of
> Abe Asraoui <AbeA at supermicro.com>
> *Sent:* Tuesday, July 17, 2018 9:23:10 PM
> *To:* lustre-devel at lists.lustre.org; lustre-discuss at lists.lustre.org; Abe
> Asraoui
> *Subject:* [lustre-devel] MDT test in rel2.11
>
> Hi All,
>
>
> Has anyone done any MDT testing under the latest rel2.11 and have
> benchmark data to share?
>
>
> Thanks,
> Abe
>
>
> _______________________________________________
> lustre-devel mailing list
> lustre-devel at lists.lustre.org
> http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org
>
> _______________________________________________
> lustre-devel mailing list
> lustre-devel at lists.lustre.org
> http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180718/59074a8e/attachment.html>

From paf at cray.com  Wed Jul 18 11:49:39 2018
From: paf at cray.com (Patrick Farrell)
Date: Wed, 18 Jul 2018 11:49:39 +0000
Subject: [lustre-devel] MDT test in rel2.11
In-Reply-To: <CAGfFL-ycwu=SOOoG_3Tmg_GuyCqgpJeyjN_H5win=eMriSqRMw@mail.gmail.com>
References: <547D95FD-0CF3-4F04-B727-D22513D1C4B9@supermicro.com>
 <BN3PR1101MB12022B362116346208D0B5F2CB530@BN3PR1101MB1202.namprd11.prod.outlook.com>
 <CAGfFL-yEhYXiRngSm2FBOBC+VLjrnTTWcYC7Frvh_6uE2RAHiw@mail.gmail.com>
 <BN3PR1101MB1202842CCA5F086263EB93BCCB530@BN3PR1101MB1202.namprd11.prod.outlook.com>,
 <CAGfFL-ycwu=SOOoG_3Tmg_GuyCqgpJeyjN_H5win=eMriSqRMw@mail.gmail.com>
Message-ID: <DM3PR1101MB12137CA3B5F9ABB9A7474C2CCB530@DM3PR1101MB1213.namprd11.prod.outlook.com>


Yes, there is intention to add it to lfs find.  Whether or not it should disqualify results is up to you at I/O 500 - it seems like if most users would think it acceptable for find most of the time (and it should be), then it should probably be allowed.  But at the same time, its (theoretical - couldn’t today) use for mdtest would very much be “writing to the benchmark” and defeating the intent.

________________________________
From: John Bent <johnbent at gmail.com>
Sent: Tuesday, July 17, 2018 11:54:32 PM
To: Patrick Farrell
Cc: Abe Asraoui; lustre-devel at lists.lustre.org; lustre-discuss at lists.lustre.org
Subject: Re: [lustre-devel] MDT test in rel2.11

Thanks Patrick.  That's interesting.  However, the exact motivation why IO500 has the 'find' command is this same intended use case; stale results therefore actually present an interesting dilemma to IO500.  They are not POSIX compliant but that loss of compliance shouldn't necessarily disqualify this result...

On Wed, Jul 18, 2018 at 12:49 AM, Patrick Farrell <paf at cray.com<mailto:paf at cray.com>> wrote:
Lazy SoM is not landed yet, and it won’t be improving benchmark scores - it’s never “known 100% correct”, so it can’t be used for actual POSIX ops - if a file size read out is used for a write offset, then you’ve got data corruption.

So for now it’s strictly limited to tools that know about it (accessed via an ioctl) and can accept information that may be stale.  The intended use case is scanning the FS for policy application.

________________________________
From: John Bent <johnbent at gmail.com<mailto:johnbent at gmail.com>>
Sent: Tuesday, July 17, 2018 10:55:24 PM
To: Patrick Farrell
Cc: Abe Asraoui; lustre-devel at lists.lustre.org<mailto:lustre-devel at lists.lustre.org>; lustre-discuss at lists.lustre.org<mailto:lustre-discuss at lists.lustre.org>
Subject: Re: [lustre-devel] MDT test in rel2.11

I'm curious about how DOM improves IO500 scores.  :)
Also LSOM but I don't know actually whether that's in 2.11 or where.

On Tue, Jul 17, 2018 at 11:33 PM, Patrick Farrell <paf at cray.com<mailto:paf at cray.com>> wrote:

Abe,

Any benchmarking would be highly dependent on hardware, both client and server.  Is there a particular comparison (say, between versions) you’re interested in or something you’re concerned about?

- Patrick

________________________________
From: lustre-devel <lustre-devel-bounces at lists.lustre.org<mailto:lustre-devel-bounces at lists.lustre.org>> on behalf of Abe Asraoui <AbeA at supermicro.com<mailto:AbeA at supermicro.com>>
Sent: Tuesday, July 17, 2018 9:23:10 PM
To: lustre-devel at lists.lustre.org<mailto:lustre-devel at lists.lustre.org>; lustre-discuss at lists.lustre.org<mailto:lustre-discuss at lists.lustre.org>; Abe Asraoui
Subject: [lustre-devel] MDT test in rel2.11

Hi All,


Has anyone done any MDT testing under the latest rel2.11 and have benchmark data to share?


Thanks,
Abe


_______________________________________________
lustre-devel mailing list
lustre-devel at lists.lustre.org<mailto:lustre-devel at lists.lustre.org>
http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org

_______________________________________________
lustre-devel mailing list
lustre-devel at lists.lustre.org<mailto:lustre-devel at lists.lustre.org>
http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180718/b0c68d6a/attachment.html>

From michael.kuhn at informatik.uni-hamburg.de  Wed Jul 18 13:51:40 2018
From: michael.kuhn at informatik.uni-hamburg.de (Michael Kuhn)
Date: Wed, 18 Jul 2018 15:51:40 +0200
Subject: [lustre-devel] Debian packaging reviews (DKMS)
Message-ID: <c48157e1ce854fb2930a19eca1e7232451e9953a.camel@informatik.uni-hamburg.de>

Hi all,

I have recently rebased my change for supporting DKMS Debian/Ubuntu
packages: https://review.whamcloud.com/#/c/25328/

I know that not a lot of people use Lustre on Debian/Ubuntu but it
would be nice to get some reviews for this. We have been carrying this
change locally for more than a year now and it would be great to
finally get this upstream. :-)


Best regards,
Michael

From pjones at whamcloud.com  Wed Jul 18 14:02:40 2018
From: pjones at whamcloud.com (Peter Jones)
Date: Wed, 18 Jul 2018 14:02:40 +0000
Subject: [lustre-devel] Debian packaging reviews (DKMS)
In-Reply-To: <c48157e1ce854fb2930a19eca1e7232451e9953a.camel@informatik.uni-hamburg.de>
References: <c48157e1ce854fb2930a19eca1e7232451e9953a.camel@informatik.uni-hamburg.de>
Message-ID: <698AD41C-8716-437B-B7E1-4ECEECC1127D@ddn.com>

Michael

I think that there is a growing community of people using Ubuntu and there have been efforts focusing around that. See - https://review.whamcloud.com/#/c/32613/

Peter

On 2018-07-18, 6:51 AM, "lustre-devel on behalf of Michael Kuhn" <lustre-devel-bounces at lists.lustre.org on behalf of michael.kuhn at informatik.uni-hamburg.de> wrote:

    Hi all,
    
    I have recently rebased my change for supporting DKMS Debian/Ubuntu
    packages: https://review.whamcloud.com/#/c/25328/
    
    I know that not a lot of people use Lustre on Debian/Ubuntu but it
    would be nice to get some reviews for this. We have been carrying this
    change locally for more than a year now and it would be great to
    finally get this upstream. :-)
    
    
    Best regards,
    Michael
    _______________________________________________
    lustre-devel mailing list
    lustre-devel at lists.lustre.org
    http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org
    


From spitzcor at cray.com  Wed Jul 18 19:41:07 2018
From: spitzcor at cray.com (Cory Spitz)
Date: Wed, 18 Jul 2018 19:41:07 +0000
Subject: [lustre-devel] MDT test in rel2.11
In-Reply-To: <BN3PR1101MB1202842CCA5F086263EB93BCCB530@BN3PR1101MB1202.namprd11.prod.outlook.com>
References: <547D95FD-0CF3-4F04-B727-D22513D1C4B9@supermicro.com>
 <BN3PR1101MB12022B362116346208D0B5F2CB530@BN3PR1101MB1202.namprd11.prod.outlook.com>
 <CAGfFL-yEhYXiRngSm2FBOBC+VLjrnTTWcYC7Frvh_6uE2RAHiw@mail.gmail.com>
 <BN3PR1101MB1202842CCA5F086263EB93BCCB530@BN3PR1101MB1202.namprd11.prod.outlook.com>
Message-ID: <D05A283A-21D5-4E36-9142-70797E94692F@cray.com>

Yes, if the IO500 is representing a use case where the file size or block count must be correct, then LSoM can’t be used.  However, the IO500 can be changed by consensus and perhaps there is a reason to include a use case which fits LSoM?

If so, the IO500 could be changed to allow `lfs find` and it in-turn could be used to get LSoM info as Andreas pointed out in his comment of LU-9538: https://jira.whamcloud.com/browse/LU-9538?focusedCommentId=230392&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-230392 .

-Cory


--


From: lustre-devel <lustre-devel-bounces at lists.lustre.org> on behalf of Patrick Farrell <paf at cray.com>
Date: Tuesday, July 17, 2018 at 11:50 PM
To: John Bent <johnbent at gmail.com>
Cc: "lustre-discuss at lists.lustre.org" <lustre-discuss at lists.lustre.org>, "lustre-devel at lists.lustre.org" <lustre-devel at lists.lustre.org>
Subject: Re: [lustre-devel] MDT test in rel2.11

Lazy SoM is not landed yet, and it won’t be improving benchmark scores - it’s never “known 100% correct”, so it can’t be used for actual POSIX ops - if a file size read out is used for a write offset, then you’ve got data corruption.

So for now it’s strictly limited to tools that know about it (accessed via an ioctl) and can accept information that may be stale.  The intended use case is scanning the FS for policy application.
________________________________
From: John Bent <johnbent at gmail.com>
Sent: Tuesday, July 17, 2018 10:55:24 PM
To: Patrick Farrell
Cc: Abe Asraoui; lustre-devel at lists.lustre.org; lustre-discuss at lists.lustre.org
Subject: Re: [lustre-devel] MDT test in rel2.11

I'm curious about how DOM improves IO500 scores.  :)
Also LSOM but I don't know actually whether that's in 2.11 or where.

On Tue, Jul 17, 2018 at 11:33 PM, Patrick Farrell <paf at cray.com<mailto:paf at cray.com>> wrote:

Abe,

Any benchmarking would be highly dependent on hardware, both client and server.  Is there a particular comparison (say, between versions) you’re interested in or something you’re concerned about?

- Patrick
________________________________
From: lustre-devel <lustre-devel-bounces at lists.lustre.org<mailto:lustre-devel-bounces at lists.lustre.org>> on behalf of Abe Asraoui <AbeA at supermicro.com<mailto:AbeA at supermicro.com>>
Sent: Tuesday, July 17, 2018 9:23:10 PM
To: lustre-devel at lists.lustre.org<mailto:lustre-devel at lists.lustre.org>; lustre-discuss at lists.lustre.org<mailto:lustre-discuss at lists.lustre.org>; Abe Asraoui
Subject: [lustre-devel] MDT test in rel2.11

Hi All,


Has anyone done any MDT testing under the latest rel2.11 and have benchmark data to share?


Thanks,
Abe


_______________________________________________
lustre-devel mailing list
lustre-devel at lists.lustre.org<mailto:lustre-devel at lists.lustre.org>
http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org

_______________________________________________
lustre-devel mailing list
lustre-devel at lists.lustre.org<mailto:lustre-devel at lists.lustre.org>
http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180718/123aac39/attachment.html>

From johnbent at gmail.com  Wed Jul 18 21:12:19 2018
From: johnbent at gmail.com (John Bent)
Date: Wed, 18 Jul 2018 17:12:19 -0400
Subject: [lustre-devel] MDT test in rel2.11
In-Reply-To: <D05A283A-21D5-4E36-9142-70797E94692F@cray.com>
References: <547D95FD-0CF3-4F04-B727-D22513D1C4B9@supermicro.com>
 <BN3PR1101MB12022B362116346208D0B5F2CB530@BN3PR1101MB1202.namprd11.prod.outlook.com>
 <CAGfFL-yEhYXiRngSm2FBOBC+VLjrnTTWcYC7Frvh_6uE2RAHiw@mail.gmail.com>
 <BN3PR1101MB1202842CCA5F086263EB93BCCB530@BN3PR1101MB1202.namprd11.prod.outlook.com>
 <D05A283A-21D5-4E36-9142-70797E94692F@cray.com>
Message-ID: <CAGfFL-zAOpX1g1ejU+Y8Q16-91eMU4TVA2mZKupbCRaKj0ZbRw@mail.gmail.com>

Certainly we can allow non-POSIX compliant "find" results in the list with
a column identifying them as being non-POSIX compliant.  The larger
question is whether one of these results can be considered a "winner".

On Wed, Jul 18, 2018 at 3:41 PM, Cory Spitz <spitzcor at cray.com> wrote:

> Yes, if the IO500 is representing a use case where the file size or block
> count must be correct, then LSoM can’t be used.  However, the IO500 can be
> changed by consensus and perhaps there is a reason to include a use case
> which fits LSoM?
>
>
>
> If so, the IO500 could be changed to allow `lfs find` and it in-turn could
> be used to get LSoM info as Andreas pointed out in his comment of LU-9538:
> https://jira.whamcloud.com/browse/LU-9538?focusedCommentId=230392&page=
> com.atlassian.jira.plugin.system.issuetabpanels:comment-
> tabpanel#comment-230392 .
>
>
>
> -Cory
>
>
>
>
>
> --
>
>
>
>
>
> *From: *lustre-devel <lustre-devel-bounces at lists.lustre.org> on behalf of
> Patrick Farrell <paf at cray.com>
> *Date: *Tuesday, July 17, 2018 at 11:50 PM
> *To: *John Bent <johnbent at gmail.com>
> *Cc: *"lustre-discuss at lists.lustre.org" <lustre-discuss at lists.lustre.org>,
> "lustre-devel at lists.lustre.org" <lustre-devel at lists.lustre.org>
>
> *Subject: *Re: [lustre-devel] MDT test in rel2.11
>
>
>
> Lazy SoM is not landed yet, and it won’t be improving benchmark scores -
> it’s never “known 100% correct”, so it can’t be used for actual POSIX ops -
> if a file size read out is used for a write offset, then you’ve got data
> corruption.
>
> So for now it’s strictly limited to tools that know about it (accessed via
> an ioctl) and can accept information that may be stale.  The intended use
> case is scanning the FS for policy application.
> ------------------------------
>
> *From:* John Bent <johnbent at gmail.com>
> *Sent:* Tuesday, July 17, 2018 10:55:24 PM
> *To:* Patrick Farrell
> *Cc:* Abe Asraoui; lustre-devel at lists.lustre.org;
> lustre-discuss at lists.lustre.org
> *Subject:* Re: [lustre-devel] MDT test in rel2.11
>
>
>
> I'm curious about how DOM improves IO500 scores.  :)
>
> Also LSOM but I don't know actually whether that's in 2.11 or where.
>
>
>
> On Tue, Jul 17, 2018 at 11:33 PM, Patrick Farrell <paf at cray.com> wrote:
>
>
> Abe,
>
> Any benchmarking would be highly dependent on hardware, both client and
> server.  Is there a particular comparison (say, between versions) you’re
> interested in or something you’re concerned about?
>
> - Patrick
> ------------------------------
>
> *From:* lustre-devel <lustre-devel-bounces at lists.lustre.org> on behalf of
> Abe Asraoui <AbeA at supermicro.com>
> *Sent:* Tuesday, July 17, 2018 9:23:10 PM
> *To:* lustre-devel at lists.lustre.org; lustre-discuss at lists.lustre.org; Abe
> Asraoui
> *Subject:* [lustre-devel] MDT test in rel2.11
>
>
>
> Hi All,
>
>
> Has anyone done any MDT testing under the latest rel2.11 and have
> benchmark data to share?
>
>
> Thanks,
> Abe
>
>
> _______________________________________________
> lustre-devel mailing list
> lustre-devel at lists.lustre.org
> http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org
>
>
> _______________________________________________
> lustre-devel mailing list
> lustre-devel at lists.lustre.org
> http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180718/7d793fc1/attachment.html>

From neilb at suse.com  Mon Jul 23 00:36:12 2018
From: neilb at suse.com (NeilBrown)
Date: Mon, 23 Jul 2018 10:36:12 +1000
Subject: [lustre-devel] [PATCH 01/18] lustre: grant: add support for
 OBD_CONNECT_GRANT_PARAM
In-Reply-To: <alpine.LFD.2.21.1807061528170.15728@casper.infradead.org>
References: <1530573875-20465-1-git-send-email-jsimmons@infradead.org>
 <1530573875-20465-2-git-send-email-jsimmons@infradead.org>
 <0F41C25D-E055-489A-82B6-CF4728AFF905@cray.com>
 <alpine.LFD.2.21.1807061528170.15728@casper.infradead.org>
Message-ID: <87bmayrdoj.fsf@notabene.neil.brown.name>

On Fri, Jul 06 2018, James Simmons wrote:

>> On 7/2/18, 6:25 PM, "lustre-devel on behalf of James Simmons" <lustre-devel-bounces at lists.lustre.org on behalf of jsimmons at infradead.org> wrote:
>> 
>>     From: Johann Lombardi <jlombardi at whamcloud.com>
>>     Signed-off-by: Johann Lombardi <jlombardi at whamcloud.com>
>>     Signed-off-by: Jinshan Xiong <jinshan.xiong at gmail.com>
>>     WC-bug-id: https://jira.whamcloud.com/browse/2049
>>     Reviewed-on: http://review.whamcloud.com/7793
>>     Reviewed-by: Andreas Dilger <adilger at whamcloud.com>
>>     Reviewed-by: Nathaniel Clark <nathaniel.l.clark at intel.com>
>>  
>> James, can you fix this up?  I'm not sure how historical patches should be attributed, but Johann's address seems
>> wrong.
>> 
>> From should be attributed to Johann Lombardi <johann.lombardi at intel.com> .  Also, the Bug URL should be
>> https://jira.whamcloud.com/browse/LU-2049 .
>
> This is tricky one. The reason I updated the email address to avoid 
> bouncing emails.

I would recommend never trying to update email addresses.  The kernel
history is full of addresses that are no longer valid.
Discussion of patches should happen on lists, and interested parties
should subscribe to the lists.  If someone wants to particularly see
emails sent to some old address of their's it should be easy enough to
add some filtering.  If you particularly want to contact the author of
some old patch, it is unlikely to take to much searching to find a valid
address (just ask on the email list - someone is bound to know).

Bounces are a little bit annoying, but so is spam - we just learn ways
to deal with it.

For this particular patch I've restored the addresses from the original
patch, even though I know some of them are no longer valid.  They were
valid when the review was done, and the effectively give credit to the
company which paid for the review.  I won't bother the rest though.

Thanks,
NeilBrown


> Also this allows the original author the chance to
> still reply. Does that seem reasonable? It is still in the air for
> some people what their email address is to me. Ah you are right I
> dropped the s. Its should be https://jira.whamcloud.com/....
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180723/2d64cee1/attachment.sig>

From neilb at suse.com  Mon Jul 23 06:23:04 2018
From: neilb at suse.com (NeilBrown)
Date: Mon, 23 Jul 2018 16:23:04 +1000
Subject: [lustre-devel] [PATCH 0/9] lustre improvements
Message-ID: <153232696720.26222.9658151633697867322.stgit@noble>

This is an ad-hoc collection of improvments and bug fixes
(mostly my bugs :-( )
Most interesting is  that handling of 'mount' is moved
from obdclass to llite, which seems to be a more fitting
place for it.

Thanks,
NeilBrown

---

NeilBrown (9):
      lustre: merge linux-debug.c into debug.c
      lustre: llog: use GFP_KERNEL for all allocations.
      lustre: move server_name2index to obd_config.c
      lustre: move lustre_check_exclusion to obd_config.c
      lustre: move filesystem mounting from obdclass to llite
      lustre: hold ptlrpc active for lustre_start_mgc()
      lustre: ll_fill_super() must put sb on all failure paths.
      lustre: ensure libcfs is set up for ioctls.
      lustre: lnet: discard LNET_LOCK()


 .../staging/lustre/include/linux/lnet/lib-lnet.h   |    3 
 .../lustre/lnet/klnds/socklnd/socklnd_proto.c      |    8 
 drivers/staging/lustre/lnet/libcfs/Makefile        |    1 
 drivers/staging/lustre/lnet/libcfs/debug.c         |   86 +
 drivers/staging/lustre/lnet/libcfs/linux-debug.c   |  142 --
 drivers/staging/lustre/lnet/libcfs/module.c        |    3 
 drivers/staging/lustre/lnet/libcfs/tracefile.h     |    4 
 drivers/staging/lustre/lnet/selftest/rpc.c         |    2 
 drivers/staging/lustre/lustre/include/obd_class.h  |    7 
 drivers/staging/lustre/lustre/llite/Makefile       |    2 
 .../staging/lustre/lustre/llite/llite_internal.h   |    3 
 drivers/staging/lustre/lustre/llite/llite_lib.c    |   16 
 drivers/staging/lustre/lustre/llite/mount.c        | 1216 ++++++++++++++++++
 drivers/staging/lustre/lustre/llite/super25.c      |    8 
 drivers/staging/lustre/lustre/obdclass/Makefile    |    2 
 drivers/staging/lustre/lustre/obdclass/class_obd.c |    6 
 drivers/staging/lustre/lustre/obdclass/llog.c      |    6 
 .../staging/lustre/lustre/obdclass/obd_config.c    |  111 ++
 drivers/staging/lustre/lustre/obdclass/obd_mount.c | 1353 --------------------
 19 files changed, 1440 insertions(+), 1539 deletions(-)
 delete mode 100644 drivers/staging/lustre/lnet/libcfs/linux-debug.c
 create mode 100644 drivers/staging/lustre/lustre/llite/mount.c
 delete mode 100644 drivers/staging/lustre/lustre/obdclass/obd_mount.c

--
Signature


From neilb at suse.com  Mon Jul 23 06:23:04 2018
From: neilb at suse.com (NeilBrown)
Date: Mon, 23 Jul 2018 16:23:04 +1000
Subject: [lustre-devel] [PATCH 1/9] lustre: merge linux-debug.c into debug.c
In-Reply-To: <153232696720.26222.9658151633697867322.stgit@noble>
References: <153232696720.26222.9658151633697867322.stgit@noble>
Message-ID: <153232698479.26222.16079606642992274567.stgit@noble>

There is no important difference between the contents
of these files, so merge them into one.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 drivers/staging/lustre/lnet/libcfs/Makefile      |    1 
 drivers/staging/lustre/lnet/libcfs/debug.c       |   86 +++++++++++++
 drivers/staging/lustre/lnet/libcfs/linux-debug.c |  142 ----------------------
 drivers/staging/lustre/lnet/libcfs/tracefile.h   |    4 -
 4 files changed, 86 insertions(+), 147 deletions(-)
 delete mode 100644 drivers/staging/lustre/lnet/libcfs/linux-debug.c

diff --git a/drivers/staging/lustre/lnet/libcfs/Makefile b/drivers/staging/lustre/lnet/libcfs/Makefile
index cd96434e3b03..a45f87af3faf 100644
--- a/drivers/staging/lustre/lnet/libcfs/Makefile
+++ b/drivers/staging/lustre/lnet/libcfs/Makefile
@@ -4,7 +4,6 @@ ccflags-y += -I$(srctree)/drivers/staging/lustre/lustre/include
 
 obj-$(CONFIG_LNET) += libcfs.o
 
-libcfs-obj-y += linux-debug.o
 libcfs-obj-y += linux-crypto.o
 libcfs-obj-y += linux-crypto-adler.o
 
diff --git a/drivers/staging/lustre/lnet/libcfs/debug.c b/drivers/staging/lustre/lnet/libcfs/debug.c
index 844543b2317f..0289f24698fa 100644
--- a/drivers/staging/lustre/lnet/libcfs/debug.c
+++ b/drivers/staging/lustre/lnet/libcfs/debug.c
@@ -327,6 +327,40 @@ libcfs_debug_str2mask(int *mask, const char *str, int is_subsys)
 			    0xffffffff);
 }
 
+char lnet_debug_log_upcall[1024] = "/usr/lib/lustre/lnet_debug_log_upcall";
+
+/**
+ * Upcall function once a Lustre log has been dumped.
+ *
+ * \param file  path of the dumped log
+ */
+static void libcfs_run_debug_log_upcall(char *file)
+{
+	char *argv[3];
+	int rc;
+	static const char * const envp[] = {
+		"HOME=/",
+		"PATH=/sbin:/bin:/usr/sbin:/usr/bin",
+		NULL
+	};
+
+	argv[0] = lnet_debug_log_upcall;
+
+	LASSERTF(file, "called on a null filename\n");
+	argv[1] = file; /* only need to pass the path of the file */
+
+	argv[2] = NULL;
+
+	rc = call_usermodehelper(argv[0], argv, (char **)envp, 1);
+	if (rc < 0 && rc != -ENOENT) {
+		CERROR("Error %d invoking LNET debug log upcall %s %s; check /sys/kernel/debug/lnet/debug_log_upcall\n",
+		       rc, argv[0], argv[1]);
+	} else {
+		CDEBUG(D_HA, "Invoked LNET debug log upcall %s %s\n",
+		       argv[0], argv[1]);
+	}
+}
+
 /**
  * Dump Lustre log to ::debug_file_path by calling tracefile_dump_all_pages()
  */
@@ -389,6 +423,58 @@ void libcfs_debug_dumplog(void)
 }
 EXPORT_SYMBOL(libcfs_debug_dumplog);
 
+/* coverity[+kill] */
+void __noreturn lbug_with_loc(struct libcfs_debug_msg_data *msgdata)
+{
+	libcfs_catastrophe = 1;
+	libcfs_debug_msg(msgdata, "LBUG\n");
+
+	if (in_interrupt()) {
+		panic("LBUG in interrupt.\n");
+		/* not reached */
+	}
+
+	dump_stack();
+	if (!libcfs_panic_on_lbug)
+		libcfs_debug_dumplog();
+	if (libcfs_panic_on_lbug)
+		panic("LBUG");
+	set_current_state(TASK_UNINTERRUPTIBLE);
+	while (1)
+		schedule();
+}
+EXPORT_SYMBOL(lbug_with_loc);
+
+static int panic_notifier(struct notifier_block *self, unsigned long unused1,
+			  void *unused2)
+{
+	if (libcfs_panic_in_progress)
+		return 0;
+
+	libcfs_panic_in_progress = 1;
+	mb();
+
+	return 0;
+}
+
+static struct notifier_block libcfs_panic_notifier = {
+	.notifier_call	= panic_notifier,
+	.next		= NULL,
+	.priority	= 10000,
+};
+
+static void libcfs_register_panic_notifier(void)
+{
+	atomic_notifier_chain_register(&panic_notifier_list,
+				       &libcfs_panic_notifier);
+}
+
+static void libcfs_unregister_panic_notifier(void)
+{
+	atomic_notifier_chain_unregister(&panic_notifier_list,
+					 &libcfs_panic_notifier);
+}
+
 int libcfs_debug_init(unsigned long bufsize)
 {
 	unsigned int max = libcfs_debug_mb;
diff --git a/drivers/staging/lustre/lnet/libcfs/linux-debug.c b/drivers/staging/lustre/lnet/libcfs/linux-debug.c
deleted file mode 100644
index 15ab849374c2..000000000000
--- a/drivers/staging/lustre/lnet/libcfs/linux-debug.c
+++ /dev/null
@@ -1,142 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-/*
- * GPL HEADER START
- *
- * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 only,
- * as published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful, but
- * WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
- * General Public License version 2 for more details (a copy is included
- * in the LICENSE file that accompanied this code).
- *
- * You should have received a copy of the GNU General Public License
- * version 2 along with this program; If not, see
- * http://www.gnu.org/licenses/gpl-2.0.html
- *
- * GPL HEADER END
- */
-/*
- * Copyright (c) 2008, 2010, Oracle and/or its affiliates. All rights reserved.
- * Use is subject to license terms.
- *
- * Copyright (c) 2012, Intel Corporation.
- */
-/*
- * This file is part of Lustre, http://www.lustre.org/
- * Lustre is a trademark of Sun Microsystems, Inc.
- *
- * libcfs/libcfs/linux/linux-debug.c
- *
- * Author: Phil Schwan <phil at clusterfs.com>
- */
-
-#include <linux/module.h>
-#include <linux/kmod.h>
-#include <linux/notifier.h>
-#include <linux/kernel.h>
-#include <linux/mm.h>
-#include <linux/string.h>
-#include <linux/stat.h>
-#include <linux/errno.h>
-#include <linux/unistd.h>
-#include <linux/interrupt.h>
-#include <linux/completion.h>
-#include <linux/fs.h>
-#include <linux/uaccess.h>
-
-# define DEBUG_SUBSYSTEM S_LNET
-
-#include "tracefile.h"
-
-#include <linux/kallsyms.h>
-
-char lnet_debug_log_upcall[1024] = "/usr/lib/lustre/lnet_debug_log_upcall";
-
-/**
- * Upcall function once a Lustre log has been dumped.
- *
- * \param file  path of the dumped log
- */
-void libcfs_run_debug_log_upcall(char *file)
-{
-	char *argv[3];
-	int rc;
-	static const char * const envp[] = {
-		"HOME=/",
-		"PATH=/sbin:/bin:/usr/sbin:/usr/bin",
-		NULL
-	};
-
-	argv[0] = lnet_debug_log_upcall;
-
-	LASSERTF(file, "called on a null filename\n");
-	argv[1] = file; /* only need to pass the path of the file */
-
-	argv[2] = NULL;
-
-	rc = call_usermodehelper(argv[0], argv, (char **)envp, 1);
-	if (rc < 0 && rc != -ENOENT) {
-		CERROR("Error %d invoking LNET debug log upcall %s %s; check /sys/kernel/debug/lnet/debug_log_upcall\n",
-		       rc, argv[0], argv[1]);
-	} else {
-		CDEBUG(D_HA, "Invoked LNET debug log upcall %s %s\n",
-		       argv[0], argv[1]);
-	}
-}
-
-/* coverity[+kill] */
-void __noreturn lbug_with_loc(struct libcfs_debug_msg_data *msgdata)
-{
-	libcfs_catastrophe = 1;
-	libcfs_debug_msg(msgdata, "LBUG\n");
-
-	if (in_interrupt()) {
-		panic("LBUG in interrupt.\n");
-		/* not reached */
-	}
-
-	dump_stack();
-	if (!libcfs_panic_on_lbug)
-		libcfs_debug_dumplog();
-	if (libcfs_panic_on_lbug)
-		panic("LBUG");
-	set_current_state(TASK_UNINTERRUPTIBLE);
-	while (1)
-		schedule();
-}
-EXPORT_SYMBOL(lbug_with_loc);
-
-static int panic_notifier(struct notifier_block *self, unsigned long unused1,
-			  void *unused2)
-{
-	if (libcfs_panic_in_progress)
-		return 0;
-
-	libcfs_panic_in_progress = 1;
-	mb();
-
-	return 0;
-}
-
-static struct notifier_block libcfs_panic_notifier = {
-	.notifier_call	= panic_notifier,
-	.next		= NULL,
-	.priority	= 10000,
-};
-
-void libcfs_register_panic_notifier(void)
-{
-	atomic_notifier_chain_register(&panic_notifier_list,
-				       &libcfs_panic_notifier);
-}
-
-void libcfs_unregister_panic_notifier(void)
-{
-	atomic_notifier_chain_unregister(&panic_notifier_list,
-					 &libcfs_panic_notifier);
-}
diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.h b/drivers/staging/lustre/lnet/libcfs/tracefile.h
index 67107b1a705a..82f090fd8dfa 100644
--- a/drivers/staging/lustre/lnet/libcfs/tracefile.h
+++ b/drivers/staging/lustre/lnet/libcfs/tracefile.h
@@ -51,8 +51,6 @@ extern long long cfs_tracefile_size;
  */
 extern char lnet_debug_log_upcall[1024];
 
-void libcfs_run_debug_log_upcall(char *file);
-
 int cfs_tracefile_dump_all_pages(char *filename);
 void cfs_trace_debug_print(void);
 void cfs_trace_flush_pages(void);
@@ -73,8 +71,6 @@ int cfs_trace_set_debug_mb(int mb);
 int cfs_trace_get_debug_mb(void);
 
 void libcfs_debug_dumplog_internal(void *arg);
-void libcfs_register_panic_notifier(void);
-void libcfs_unregister_panic_notifier(void);
 extern int libcfs_panic_in_progress;
 
 #define TCD_MAX_PAGES (5 << (20 - PAGE_SHIFT))



From neilb at suse.com  Mon Jul 23 06:23:04 2018
From: neilb at suse.com (NeilBrown)
Date: Mon, 23 Jul 2018 16:23:04 +1000
Subject: [lustre-devel] [PATCH 2/9] lustre: llog: use GFP_KERNEL for all
 allocations.
In-Reply-To: <153232696720.26222.9658151633697867322.stgit@noble>
References: <153232696720.26222.9658151633697867322.stgit@noble>
Message-ID: <153232698483.26222.2842073304458010313.stgit@noble>

Processing the log happens at mount time so
memory allocations will not trigger write-out to
this filesystem which might deadlock.
So use of GFP_NOFS is not needed.
GFP_KERNEL is preferred, especially for kvmalloc()
which doesn't try the 'vmalloc' option when
GFP_KERNEL is not specified.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 drivers/staging/lustre/lustre/obdclass/llog.c |    6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/drivers/staging/lustre/lustre/obdclass/llog.c b/drivers/staging/lustre/lustre/obdclass/llog.c
index f59f89a76c91..8644d349535e 100644
--- a/drivers/staging/lustre/lustre/obdclass/llog.c
+++ b/drivers/staging/lustre/lustre/obdclass/llog.c
@@ -59,7 +59,7 @@ static struct llog_handle *llog_alloc_handle(void)
 {
 	struct llog_handle *loghandle;
 
-	loghandle = kzalloc(sizeof(*loghandle), GFP_NOFS);
+	loghandle = kzalloc(sizeof(*loghandle), GFP_KERNEL);
 	if (!loghandle)
 		return NULL;
 
@@ -242,7 +242,7 @@ static int llog_process_thread(void *arg)
 	/* expect chunk_size to be power of two */
 	LASSERT(is_power_of_2(chunk_size));
 
-	buf = kvzalloc(chunk_size, GFP_NOFS);
+	buf = kvzalloc(chunk_size, GFP_KERNEL);
 	if (!buf) {
 		lpi->lpi_rc = -ENOMEM;
 		return 0;
@@ -421,7 +421,7 @@ int llog_process_or_fork(const struct lu_env *env,
 	struct llog_process_info *lpi;
 	int		      rc;
 
-	lpi = kzalloc(sizeof(*lpi), GFP_NOFS);
+	lpi = kzalloc(sizeof(*lpi), GFP_KERNEL);
 	if (!lpi)
 		return -ENOMEM;
 	lpi->lpi_loghandle = loghandle;



From neilb at suse.com  Mon Jul 23 06:23:04 2018
From: neilb at suse.com (NeilBrown)
Date: Mon, 23 Jul 2018 16:23:04 +1000
Subject: [lustre-devel] [PATCH 3/9] lustre: move server_name2index to
 obd_config.c
In-Reply-To: <153232696720.26222.9658151633697867322.stgit@noble>
References: <153232696720.26222.9658151633697867322.stgit@noble>
Message-ID: <153232698486.26222.18197471449250973608.stgit@noble>

This is preparation for moving filesystem mounting
into llite/.
server_name2index belongs in obdclass, but the rest
of obd_mount.c fits better in llite.  So move
it and server_name2fsname() across to obd_config.c,
and expect it for later using by llite.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 drivers/staging/lustre/lustre/include/obd_class.h  |    2 
 .../staging/lustre/lustre/obdclass/obd_config.c    |   86 ++++++++++++++++++++
 drivers/staging/lustre/lustre/obdclass/obd_mount.c |   84 --------------------
 3 files changed, 88 insertions(+), 84 deletions(-)

diff --git a/drivers/staging/lustre/lustre/include/obd_class.h b/drivers/staging/lustre/lustre/include/obd_class.h
index adfe2abdf7bc..ef8c46cb9bee 100644
--- a/drivers/staging/lustre/lustre/include/obd_class.h
+++ b/drivers/staging/lustre/lustre/include/obd_class.h
@@ -134,6 +134,8 @@ int class_config_llog_handler(const struct lu_env *env,
 			      struct llog_handle *handle,
 			      struct llog_rec_hdr *rec, void *data);
 int class_add_uuid(const char *uuid, __u64 nid);
+int server_name2index(const char *svname, __u32 *idx,
+		      const char **endptr);
 
 /* obdecho */
 void lprocfs_echo_init_vars(struct lprocfs_static_vars *lvars);
diff --git a/drivers/staging/lustre/lustre/obdclass/obd_config.c b/drivers/staging/lustre/lustre/obdclass/obd_config.c
index cfcd17e679dd..382522fa1993 100644
--- a/drivers/staging/lustre/lustre/obdclass/obd_config.c
+++ b/drivers/staging/lustre/lustre/obdclass/obd_config.c
@@ -46,6 +46,7 @@
 #include <lustre_log.h>
 #include <uapi/linux/lustre/lustre_param.h>
 #include <obd_class.h>
+#include <lustre_disk.h>
 
 #include "llog_internal.h"
 
@@ -1172,6 +1173,91 @@ int class_process_proc_param(char *prefix, struct lprocfs_vars *lvars,
 }
 EXPORT_SYMBOL(class_process_proc_param);
 
+/*** SERVER NAME ***
+ * <FSNAME><SEPARATOR><TYPE><INDEX>
+ * FSNAME is between 1 and 8 characters (inclusive).
+ *	Excluded characters are '/' and ':'
+ * SEPARATOR is either ':' or '-'
+ * TYPE: "OST", "MDT", etc.
+ * INDEX: Hex representation of the index
+ */
+
+/** Get the fsname ("lustre") from the server name ("lustre-OST003F").
+ * @param [in] svname server name including type and index
+ * @param [out] fsname Buffer to copy filesystem name prefix into.
+ *  Must have at least 'strlen(fsname) + 1' chars.
+ * @param [out] endptr if endptr isn't NULL it is set to end of fsname
+ * rc < 0  on error
+ */
+static int server_name2fsname(const char *svname, char *fsname,
+			      const char **endptr)
+{
+	const char *dash;
+
+	dash = svname + strnlen(svname, 8); /* max fsname length is 8 */
+	for (; dash > svname && *dash != '-' && *dash != ':'; dash--)
+		;
+	if (dash == svname)
+		return -EINVAL;
+
+	if (fsname) {
+		strncpy(fsname, svname, dash - svname);
+		fsname[dash - svname] = '\0';
+	}
+
+	if (endptr)
+		*endptr = dash;
+
+	return 0;
+}
+
+/* Get the index from the obd name.
+ *  rc = server type, or
+ * rc < 0  on error
+ * if endptr isn't NULL it is set to end of name
+ */
+int server_name2index(const char *svname, __u32 *idx,
+		      const char **endptr)
+{
+	unsigned long index;
+	int rc;
+	const char *dash;
+
+	/* We use server_name2fsname() just for parsing */
+	rc = server_name2fsname(svname, NULL, &dash);
+	if (rc != 0)
+		return rc;
+
+	dash++;
+
+	if (strncmp(dash, "MDT", 3) == 0)
+		rc = LDD_F_SV_TYPE_MDT;
+	else if (strncmp(dash, "OST", 3) == 0)
+		rc = LDD_F_SV_TYPE_OST;
+	else
+		return -EINVAL;
+
+	dash += 3;
+
+	if (strncmp(dash, "all", 3) == 0) {
+		if (endptr)
+			*endptr = dash + 3;
+		return rc | LDD_F_SV_ALL;
+	}
+
+	index = simple_strtoul(dash, (char **)endptr, 16);
+	if (idx)
+		*idx = index;
+
+	/* Account for -mdc after index that is possible when specifying mdt */
+	if (endptr && strncmp(LUSTRE_MDC_NAME, *endptr + 1,
+			      sizeof(LUSTRE_MDC_NAME) - 1) == 0)
+		*endptr += sizeof(LUSTRE_MDC_NAME);
+
+	return rc;
+}
+EXPORT_SYMBOL(server_name2index);
+
 /** Parse a configuration llog, doing various manipulations on them
  * for various reasons, (modifications for compatibility, skip obsolete
  * records, change uuids, etc), then class_process_config() resulting
diff --git a/drivers/staging/lustre/lustre/obdclass/obd_mount.c b/drivers/staging/lustre/lustre/obdclass/obd_mount.c
index 1a668874a165..3420e87ad0cd 100644
--- a/drivers/staging/lustre/lustre/obdclass/obd_mount.c
+++ b/drivers/staging/lustre/lustre/obdclass/obd_mount.c
@@ -579,90 +579,6 @@ static int lustre_put_lsi(struct super_block *sb)
 	return 0;
 }
 
-/*** SERVER NAME ***
- * <FSNAME><SEPARATOR><TYPE><INDEX>
- * FSNAME is between 1 and 8 characters (inclusive).
- *	Excluded characters are '/' and ':'
- * SEPARATOR is either ':' or '-'
- * TYPE: "OST", "MDT", etc.
- * INDEX: Hex representation of the index
- */
-
-/** Get the fsname ("lustre") from the server name ("lustre-OST003F").
- * @param [in] svname server name including type and index
- * @param [out] fsname Buffer to copy filesystem name prefix into.
- *  Must have at least 'strlen(fsname) + 1' chars.
- * @param [out] endptr if endptr isn't NULL it is set to end of fsname
- * rc < 0  on error
- */
-static int server_name2fsname(const char *svname, char *fsname,
-			      const char **endptr)
-{
-	const char *dash;
-
-	dash = svname + strnlen(svname, 8); /* max fsname length is 8 */
-	for (; dash > svname && *dash != '-' && *dash != ':'; dash--)
-		;
-	if (dash == svname)
-		return -EINVAL;
-
-	if (fsname) {
-		strncpy(fsname, svname, dash - svname);
-		fsname[dash - svname] = '\0';
-	}
-
-	if (endptr)
-		*endptr = dash;
-
-	return 0;
-}
-
-/* Get the index from the obd name.
- *  rc = server type, or
- * rc < 0  on error
- * if endptr isn't NULL it is set to end of name
- */
-static int server_name2index(const char *svname, __u32 *idx,
-			     const char **endptr)
-{
-	unsigned long index;
-	int rc;
-	const char *dash;
-
-	/* We use server_name2fsname() just for parsing */
-	rc = server_name2fsname(svname, NULL, &dash);
-	if (rc != 0)
-		return rc;
-
-	dash++;
-
-	if (strncmp(dash, "MDT", 3) == 0)
-		rc = LDD_F_SV_TYPE_MDT;
-	else if (strncmp(dash, "OST", 3) == 0)
-		rc = LDD_F_SV_TYPE_OST;
-	else
-		return -EINVAL;
-
-	dash += 3;
-
-	if (strncmp(dash, "all", 3) == 0) {
-		if (endptr)
-			*endptr = dash + 3;
-		return rc | LDD_F_SV_ALL;
-	}
-
-	index = simple_strtoul(dash, (char **)endptr, 16);
-	if (idx)
-		*idx = index;
-
-	/* Account for -mdc after index that is possible when specifying mdt */
-	if (endptr && strncmp(LUSTRE_MDC_NAME, *endptr + 1,
-			      sizeof(LUSTRE_MDC_NAME) - 1) == 0)
-		*endptr += sizeof(LUSTRE_MDC_NAME);
-
-	return rc;
-}
-
 /*************** mount common between server and client ***************/
 
 /* Common umount */



From neilb at suse.com  Mon Jul 23 06:23:04 2018
From: neilb at suse.com (NeilBrown)
Date: Mon, 23 Jul 2018 16:23:04 +1000
Subject: [lustre-devel] [PATCH 4/9] lustre: move lustre_check_exclusion to
 obd_config.c
In-Reply-To: <153232696720.26222.9658151633697867322.stgit@noble>
References: <153232696720.26222.9658151633697867322.stgit@noble>
Message-ID: <153232698489.26222.11036165790183672424.stgit@noble>

lustre_check_exclusion is only used in obd_config.c, so
move the code there
Note that a CDEBUG is changed from D_MOUNT to D_INFO.
This prepares for obd_mount to move to llite/

Signed-off-by: NeilBrown <neilb at suse.com>
---
 drivers/staging/lustre/lustre/include/obd_class.h  |    1 -
 .../staging/lustre/lustre/obdclass/obd_config.c    |   25 ++++++++++++++++++++
 drivers/staging/lustre/lustre/obdclass/obd_mount.c |   25 --------------------
 3 files changed, 25 insertions(+), 26 deletions(-)

diff --git a/drivers/staging/lustre/lustre/include/obd_class.h b/drivers/staging/lustre/lustre/include/obd_class.h
index ef8c46cb9bee..90dbafd38d71 100644
--- a/drivers/staging/lustre/lustre/include/obd_class.h
+++ b/drivers/staging/lustre/lustre/include/obd_class.h
@@ -1559,7 +1559,6 @@ extern int (*ptlrpc_put_connection_superhack)(struct ptlrpc_connection *c);
 /* obd_mount.c */
 int lustre_unregister_fs(void);
 int lustre_register_fs(void);
-int lustre_check_exclusion(struct super_block *sb, char *svname);
 
 /* sysctl.c */
 int obd_sysctl_init(void);
diff --git a/drivers/staging/lustre/lustre/obdclass/obd_config.c b/drivers/staging/lustre/lustre/obdclass/obd_config.c
index 382522fa1993..f3dd020d04ef 100644
--- a/drivers/staging/lustre/lustre/obdclass/obd_config.c
+++ b/drivers/staging/lustre/lustre/obdclass/obd_config.c
@@ -1258,6 +1258,31 @@ int server_name2index(const char *svname, __u32 *idx,
 }
 EXPORT_SYMBOL(server_name2index);
 
+/* Is this server on the exclusion list */
+static int lustre_check_exclusion(struct super_block *sb, char *svname)
+{
+	struct lustre_sb_info *lsi = s2lsi(sb);
+	struct lustre_mount_data *lmd = lsi->lsi_lmd;
+	__u32 index;
+	int i, rc;
+
+	rc = server_name2index(svname, &index, NULL);
+	if (rc != LDD_F_SV_TYPE_OST)
+		/* Only exclude OSTs */
+		return 0;
+
+	CDEBUG(D_INFO, "Check exclusion %s (%d) in %d of %s\n", svname,
+	       index, lmd->lmd_exclude_count, lmd->lmd_dev);
+
+	for (i = 0; i < lmd->lmd_exclude_count; i++) {
+		if (index == lmd->lmd_exclude[i]) {
+			CWARN("Excluding %s (on exclusion list)\n", svname);
+			return 1;
+		}
+	}
+	return 0;
+}
+
 /** Parse a configuration llog, doing various manipulations on them
  * for various reasons, (modifications for compatibility, skip obsolete
  * records, change uuids, etc), then class_process_config() resulting
diff --git a/drivers/staging/lustre/lustre/obdclass/obd_mount.c b/drivers/staging/lustre/lustre/obdclass/obd_mount.c
index 3420e87ad0cd..4e4a8c35105b 100644
--- a/drivers/staging/lustre/lustre/obdclass/obd_mount.c
+++ b/drivers/staging/lustre/lustre/obdclass/obd_mount.c
@@ -633,31 +633,6 @@ static void lmd_print(struct lustre_mount_data *lmd)
 	}
 }
 
-/* Is this server on the exclusion list */
-int lustre_check_exclusion(struct super_block *sb, char *svname)
-{
-	struct lustre_sb_info *lsi = s2lsi(sb);
-	struct lustre_mount_data *lmd = lsi->lsi_lmd;
-	__u32 index;
-	int i, rc;
-
-	rc = server_name2index(svname, &index, NULL);
-	if (rc != LDD_F_SV_TYPE_OST)
-		/* Only exclude OSTs */
-		return 0;
-
-	CDEBUG(D_MOUNT, "Check exclusion %s (%d) in %d of %s\n", svname,
-	       index, lmd->lmd_exclude_count, lmd->lmd_dev);
-
-	for (i = 0; i < lmd->lmd_exclude_count; i++) {
-		if (index == lmd->lmd_exclude[i]) {
-			CWARN("Excluding %s (on exclusion list)\n", svname);
-			return 1;
-		}
-	}
-	return 0;
-}
-
 /* mount -v  -o exclude=lustre-OST0001:lustre-OST0002 -t lustre ... */
 static int lmd_make_exclusion(struct lustre_mount_data *lmd, const char *ptr)
 {



From neilb at suse.com  Mon Jul 23 06:23:04 2018
From: neilb at suse.com (NeilBrown)
Date: Mon, 23 Jul 2018 16:23:04 +1000
Subject: [lustre-devel] [PATCH 5/9] lustre: move filesystem mounting from
 obdclass to llite
In-Reply-To: <153232696720.26222.9658151633697867322.stgit@noble>
References: <153232696720.26222.9658151633697867322.stgit@noble>
Message-ID: <153232698492.26222.12045767311396974106.stgit@noble>

All filesystem-api aspects of the lustre filesystem are in
llite, except mounting which is in obdclass.
This is probably because the name filesystem type is/was
used for mounting an lustre filesystem on the client, and
for mounting a data-store on the server.
This is a confusion that is best discarded and forgotten.

So move the mount handling into llite.

This removes that rather awkward requirment that obdclass needed to
request_module() the lustre module, and makes lustre_fill_super()
significantly simpler.

Several symbols don't need to be exported any more as they are now
part of llite which is the only module that uses them.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 drivers/staging/lustre/lustre/include/obd_class.h  |    4 
 drivers/staging/lustre/lustre/llite/Makefile       |    2 
 .../staging/lustre/lustre/llite/llite_internal.h   |    3 
 drivers/staging/lustre/lustre/llite/mount.c        | 1208 +++++++++++++++++++
 drivers/staging/lustre/lustre/llite/super25.c      |    8 
 drivers/staging/lustre/lustre/obdclass/Makefile    |    2 
 drivers/staging/lustre/lustre/obdclass/class_obd.c |    6 
 drivers/staging/lustre/lustre/obdclass/obd_mount.c | 1244 --------------------
 8 files changed, 1218 insertions(+), 1259 deletions(-)
 create mode 100644 drivers/staging/lustre/lustre/llite/mount.c
 delete mode 100644 drivers/staging/lustre/lustre/obdclass/obd_mount.c

diff --git a/drivers/staging/lustre/lustre/include/obd_class.h b/drivers/staging/lustre/lustre/include/obd_class.h
index 90dbafd38d71..1a55f0215837 100644
--- a/drivers/staging/lustre/lustre/include/obd_class.h
+++ b/drivers/staging/lustre/lustre/include/obd_class.h
@@ -1556,10 +1556,6 @@ struct lwp_register_item {
  */
 extern int (*ptlrpc_put_connection_superhack)(struct ptlrpc_connection *c);
 
-/* obd_mount.c */
-int lustre_unregister_fs(void);
-int lustre_register_fs(void);
-
 /* sysctl.c */
 int obd_sysctl_init(void);
 
diff --git a/drivers/staging/lustre/lustre/llite/Makefile b/drivers/staging/lustre/lustre/llite/Makefile
index 7d0225476362..670f072ce370 100644
--- a/drivers/staging/lustre/lustre/llite/Makefile
+++ b/drivers/staging/lustre/lustre/llite/Makefile
@@ -3,7 +3,7 @@ ccflags-y += -I$(srctree)/drivers/staging/lustre/include
 ccflags-y += -I$(srctree)/drivers/staging/lustre/lustre/include
 
 obj-$(CONFIG_LUSTRE_FS) += lustre.o
-lustre-y := dcache.o dir.o file.o llite_lib.o llite_nfs.o \
+lustre-y := mount.o dcache.o dir.o file.o llite_lib.o llite_nfs.o \
 	    rw.o rw26.o namei.o symlink.o llite_mmap.o range_lock.o \
 	    xattr.o xattr_cache.o xattr_security.o \
 	    super25.o statahead.o glimpse.o lcommon_cl.o lcommon_misc.o \
diff --git a/drivers/staging/lustre/lustre/llite/llite_internal.h b/drivers/staging/lustre/lustre/llite/llite_internal.h
index 8399501c9122..1f832a5ba023 100644
--- a/drivers/staging/lustre/lustre/llite/llite_internal.h
+++ b/drivers/staging/lustre/lustre/llite/llite_internal.h
@@ -1351,4 +1351,7 @@ void cl_inode_fini(struct inode *inode);
 __u64 cl_fid_build_ino(const struct lu_fid *fid, int api32);
 __u32 cl_fid_build_gen(const struct lu_fid *fid);
 
+int lustre_unregister_fs(void);
+int lustre_register_fs(void);
+
 #endif /* LLITE_INTERNAL_H */
diff --git a/drivers/staging/lustre/lustre/llite/mount.c b/drivers/staging/lustre/lustre/llite/mount.c
new file mode 100644
index 000000000000..58e8b371f0c4
--- /dev/null
+++ b/drivers/staging/lustre/lustre/llite/mount.c
@@ -0,0 +1,1208 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * GPL HEADER START
+ *
+ * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 only,
+ * as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License version 2 for more details (a copy is included
+ * in the LICENSE file that accompanied this code).
+ *
+ * You should have received a copy of the GNU General Public License
+ * version 2 along with this program; If not, see
+ * http://www.gnu.org/licenses/gpl-2.0.html
+ *
+ * GPL HEADER END
+ */
+/*
+ * Copyright (c) 2007, 2010, Oracle and/or its affiliates. All rights reserved.
+ * Use is subject to license terms.
+ *
+ * Copyright (c) 2011, 2015, Intel Corporation.
+ */
+/*
+ * This file is part of Lustre, http://www.lustre.org/
+ * Lustre is a trademark of Sun Microsystems, Inc.
+ *
+ * lustre/obdclass/obd_mount.c
+ *
+ * Client mount routines
+ *
+ * Author: Nathan Rutman <nathan at clusterfs.com>
+ */
+
+#define DEBUG_SUBSYSTEM S_CLASS
+#define D_MOUNT (D_SUPER | D_CONFIG/*|D_WARNING */)
+#define PRINT_CMD CDEBUG
+
+#include <linux/random.h>
+#include <obd.h>
+#include <obd_class.h>
+#include <uapi/linux/lustre/lustre_idl.h>
+#include <lustre_log.h>
+#include <lustre_disk.h>
+#include <uapi/linux/lustre/lustre_param.h>
+#include <llite_internal.h>
+
+/**************** config llog ********************/
+
+/** Get a config log from the MGS and process it.
+ * This func is called for both clients and servers.
+ * Continue to process new statements appended to the logs
+ * (whenever the config lock is revoked) until lustre_end_log
+ * is called.
+ * @param sb The superblock is used by the MGC to write to the local copy of
+ *   the config log
+ * @param logname The name of the llog to replicate from the MGS
+ * @param cfg Since the same mgc may be used to follow multiple config logs
+ *   (e.g. ost1, ost2, client), the config_llog_instance keeps the state for
+ *   this log, and is added to the mgc's list of logs to follow.
+ */
+int lustre_process_log(struct super_block *sb, char *logname,
+		       struct config_llog_instance *cfg)
+{
+	struct lustre_cfg *lcfg;
+	struct lustre_cfg_bufs *bufs;
+	struct lustre_sb_info *lsi = s2lsi(sb);
+	struct obd_device *mgc = lsi->lsi_mgc;
+	int rc;
+
+	LASSERT(mgc);
+	LASSERT(cfg);
+
+	bufs = kzalloc(sizeof(*bufs), GFP_NOFS);
+	if (!bufs)
+		return -ENOMEM;
+
+	/* mgc_process_config */
+	lustre_cfg_bufs_reset(bufs, mgc->obd_name);
+	lustre_cfg_bufs_set_string(bufs, 1, logname);
+	lustre_cfg_bufs_set(bufs, 2, cfg, sizeof(*cfg));
+	lustre_cfg_bufs_set(bufs, 3, &sb, sizeof(sb));
+	lcfg = kzalloc(lustre_cfg_len(bufs->lcfg_bufcount, bufs->lcfg_buflen),
+		       GFP_NOFS);
+	if (!lcfg) {
+		rc = -ENOMEM;
+		goto out;
+	}
+	lustre_cfg_init(lcfg, LCFG_LOG_START, bufs);
+
+	rc = obd_process_config(mgc, sizeof(*lcfg), lcfg);
+	kfree(lcfg);
+out:
+	kfree(bufs);
+
+	if (rc == -EINVAL)
+		LCONSOLE_ERROR_MSG(0x15b, "%s: The configuration from log '%s' failed from the MGS (%d).  Make sure this client and the MGS are running compatible versions of Lustre.\n",
+				   mgc->obd_name, logname, rc);
+
+	else if (rc)
+		LCONSOLE_ERROR_MSG(0x15c, "%s: The configuration from log '%s' failed (%d). This may be the result of communication errors between this node and the MGS, a bad configuration, or other errors. See the syslog for more information.\n",
+				   mgc->obd_name, logname,
+				   rc);
+
+	/* class_obd_list(); */
+	return rc;
+}
+
+/* Stop watching this config log for updates */
+int lustre_end_log(struct super_block *sb, char *logname,
+		   struct config_llog_instance *cfg)
+{
+	struct lustre_cfg *lcfg;
+	struct lustre_cfg_bufs bufs;
+	struct lustre_sb_info *lsi = s2lsi(sb);
+	struct obd_device *mgc = lsi->lsi_mgc;
+	int rc;
+
+	if (!mgc)
+		return -ENOENT;
+
+	/* mgc_process_config */
+	lustre_cfg_bufs_reset(&bufs, mgc->obd_name);
+	lustre_cfg_bufs_set_string(&bufs, 1, logname);
+	if (cfg)
+		lustre_cfg_bufs_set(&bufs, 2, cfg, sizeof(*cfg));
+	lcfg = kzalloc(lustre_cfg_len(bufs.lcfg_bufcount, bufs.lcfg_buflen),
+		       GFP_NOFS);
+	if (!lcfg)
+		return -ENOMEM;
+	lustre_cfg_init(lcfg, LCFG_LOG_END, &bufs);
+
+	rc = obd_process_config(mgc, sizeof(*lcfg), lcfg);
+	kfree(lcfg);
+	return rc;
+}
+
+/**************** obd start *******************/
+
+/** lustre_cfg_bufs are a holdover from 1.4; we can still set these up from
+ * lctl (and do for echo cli/srv.
+ */
+static int do_lcfg(char *cfgname, lnet_nid_t nid, int cmd,
+		   char *s1, char *s2, char *s3, char *s4)
+{
+	struct lustre_cfg_bufs bufs;
+	struct lustre_cfg     *lcfg = NULL;
+	int rc;
+
+	CDEBUG(D_TRACE, "lcfg %s %#x %s %s %s %s\n", cfgname,
+	       cmd, s1, s2, s3, s4);
+
+	lustre_cfg_bufs_reset(&bufs, cfgname);
+	if (s1)
+		lustre_cfg_bufs_set_string(&bufs, 1, s1);
+	if (s2)
+		lustre_cfg_bufs_set_string(&bufs, 2, s2);
+	if (s3)
+		lustre_cfg_bufs_set_string(&bufs, 3, s3);
+	if (s4)
+		lustre_cfg_bufs_set_string(&bufs, 4, s4);
+
+	lcfg = kzalloc(lustre_cfg_len(bufs.lcfg_bufcount, bufs.lcfg_buflen),
+		       GFP_NOFS);
+	if (!lcfg)
+		return -ENOMEM;
+	lustre_cfg_init(lcfg, cmd, &bufs);
+	lcfg->lcfg_nid = nid;
+	rc = class_process_config(lcfg);
+	kfree(lcfg);
+	return rc;
+}
+
+/** Call class_attach and class_setup.  These methods in turn call
+ * obd type-specific methods.
+ */
+static int lustre_start_simple(char *obdname, char *type, char *uuid,
+			       char *s1, char *s2, char *s3, char *s4)
+{
+	int rc;
+
+	CDEBUG(D_MOUNT, "Starting obd %s (typ=%s)\n", obdname, type);
+
+	rc = do_lcfg(obdname, 0, LCFG_ATTACH, type, uuid, NULL, NULL);
+	if (rc) {
+		CERROR("%s attach error %d\n", obdname, rc);
+		return rc;
+	}
+	rc = do_lcfg(obdname, 0, LCFG_SETUP, s1, s2, s3, s4);
+	if (rc) {
+		CERROR("%s setup error %d\n", obdname, rc);
+		do_lcfg(obdname, 0, LCFG_DETACH, NULL, NULL, NULL, NULL);
+	}
+	return rc;
+}
+
+static DEFINE_MUTEX(mgc_start_lock);
+
+/** Set up a mgc obd to process startup logs
+ *
+ * \param sb [in] super block of the mgc obd
+ *
+ * \retval 0 success, otherwise error code
+ */
+int lustre_start_mgc(struct super_block *sb)
+{
+	struct obd_connect_data *data = NULL;
+	struct lustre_sb_info *lsi = s2lsi(sb);
+	struct obd_device *obd;
+	struct obd_export *exp;
+	struct obd_uuid *uuid;
+	class_uuid_t uuidc;
+	lnet_nid_t nid;
+	char nidstr[LNET_NIDSTR_SIZE];
+	char *mgcname = NULL, *niduuid = NULL, *mgssec = NULL;
+	char *ptr;
+	int rc = 0, i = 0, j;
+
+	LASSERT(lsi->lsi_lmd);
+
+	/* Use nids from mount line: uml1,1 at elan:uml2,2 at elan:/lustre */
+	ptr = lsi->lsi_lmd->lmd_dev;
+	if (class_parse_nid(ptr, &nid, &ptr) == 0)
+		i++;
+	if (i == 0) {
+		CERROR("No valid MGS nids found.\n");
+		return -EINVAL;
+	}
+
+	mutex_lock(&mgc_start_lock);
+
+	libcfs_nid2str_r(nid, nidstr, sizeof(nidstr));
+	mgcname = kasprintf(GFP_NOFS,
+			    "%s%s", LUSTRE_MGC_OBDNAME, nidstr);
+	niduuid = kasprintf(GFP_NOFS, "%s_%x", mgcname, 0);
+	if (!mgcname || !niduuid) {
+		rc = -ENOMEM;
+		goto out_free;
+	}
+
+	mgssec = lsi->lsi_lmd->lmd_mgssec ? lsi->lsi_lmd->lmd_mgssec : "";
+
+	data = kzalloc(sizeof(*data), GFP_NOFS);
+	if (!data) {
+		rc = -ENOMEM;
+		goto out_free;
+	}
+
+	obd = class_name2obd(mgcname);
+	if (obd && !obd->obd_stopping) {
+		int recov_bk;
+
+		rc = obd_set_info_async(NULL, obd->obd_self_export,
+					strlen(KEY_MGSSEC), KEY_MGSSEC,
+					strlen(mgssec), mgssec, NULL);
+		if (rc)
+			goto out_free;
+
+		/* Re-using an existing MGC */
+		atomic_inc(&obd->u.cli.cl_mgc_refcount);
+
+		/* IR compatibility check, only for clients */
+		if (lmd_is_client(lsi->lsi_lmd)) {
+			int has_ir;
+			int vallen = sizeof(*data);
+			__u32 *flags = &lsi->lsi_lmd->lmd_flags;
+
+			rc = obd_get_info(NULL, obd->obd_self_export,
+					  strlen(KEY_CONN_DATA), KEY_CONN_DATA,
+					  &vallen, data);
+			LASSERT(rc == 0);
+			has_ir = OCD_HAS_FLAG(data, IMP_RECOV);
+			if (has_ir ^ !(*flags & LMD_FLG_NOIR)) {
+				/* LMD_FLG_NOIR is for test purpose only */
+				LCONSOLE_WARN(
+					"Trying to mount a client with IR setting not compatible with current mgc. Force to use current mgc setting that is IR %s.\n",
+					has_ir ? "enabled" : "disabled");
+				if (has_ir)
+					*flags &= ~LMD_FLG_NOIR;
+				else
+					*flags |= LMD_FLG_NOIR;
+			}
+		}
+
+		recov_bk = 0;
+
+		/* Try all connections, but only once (again).
+		 * We don't want to block another target from starting
+		 * (using its local copy of the log), but we do want to connect
+		 * if at all possible.
+		 */
+		recov_bk++;
+		CDEBUG(D_MOUNT, "%s: Set MGC reconnect %d\n", mgcname,
+		       recov_bk);
+		rc = obd_set_info_async(NULL, obd->obd_self_export,
+					sizeof(KEY_INIT_RECOV_BACKUP),
+					KEY_INIT_RECOV_BACKUP,
+					sizeof(recov_bk), &recov_bk, NULL);
+		rc = 0;
+		goto out;
+	}
+
+	CDEBUG(D_MOUNT, "Start MGC '%s'\n", mgcname);
+
+	/* Add the primary nids for the MGS */
+	i = 0;
+	/* Use nids from mount line: uml1,1 at elan:uml2,2 at elan:/lustre */
+	ptr = lsi->lsi_lmd->lmd_dev;
+	while (class_parse_nid(ptr, &nid, &ptr) == 0) {
+		rc = do_lcfg(mgcname, nid,
+			     LCFG_ADD_UUID, niduuid, NULL, NULL, NULL);
+		if (!rc)
+			i++;
+		/* Stop at the first failover nid */
+		if (*ptr == ':')
+			break;
+	}
+	if (i == 0) {
+		CERROR("No valid MGS nids found.\n");
+		rc = -EINVAL;
+		goto out_free;
+	}
+	lsi->lsi_lmd->lmd_mgs_failnodes = 1;
+
+	/* Random uuid for MGC allows easier reconnects */
+	uuid = kzalloc(sizeof(*uuid), GFP_NOFS);
+	if (!uuid) {
+		rc = -ENOMEM;
+		goto out_free;
+	}
+
+	ll_generate_random_uuid(uuidc);
+	class_uuid_unparse(uuidc, uuid);
+
+	/* Start the MGC */
+	rc = lustre_start_simple(mgcname, LUSTRE_MGC_NAME,
+				 (char *)uuid->uuid, LUSTRE_MGS_OBDNAME,
+				 niduuid, NULL, NULL);
+	kfree(uuid);
+	if (rc)
+		goto out_free;
+
+	/* Add any failover MGS nids */
+	i = 1;
+	while (ptr && ((*ptr == ':' ||
+			class_find_param(ptr, PARAM_MGSNODE, &ptr) == 0))) {
+		/* New failover node */
+		sprintf(niduuid, "%s_%x", mgcname, i);
+		j = 0;
+		while (class_parse_nid_quiet(ptr, &nid, &ptr) == 0) {
+			rc = do_lcfg(mgcname, nid, LCFG_ADD_UUID, niduuid,
+				     NULL, NULL, NULL);
+			if (!rc)
+				++j;
+			if (*ptr == ':')
+				break;
+		}
+		if (j > 0) {
+			rc = do_lcfg(mgcname, 0, LCFG_ADD_CONN,
+				     niduuid, NULL, NULL, NULL);
+			if (!rc)
+				i++;
+		} else {
+			/* at ":/fsname" */
+			break;
+		}
+	}
+	lsi->lsi_lmd->lmd_mgs_failnodes = i;
+
+	obd = class_name2obd(mgcname);
+	if (!obd) {
+		CERROR("Can't find mgcobd %s\n", mgcname);
+		rc = -ENOTCONN;
+		goto out_free;
+	}
+
+	rc = obd_set_info_async(NULL, obd->obd_self_export,
+				strlen(KEY_MGSSEC), KEY_MGSSEC,
+				strlen(mgssec), mgssec, NULL);
+	if (rc)
+		goto out_free;
+
+	/* Keep a refcount of servers/clients who started with "mount",
+	 * so we know when we can get rid of the mgc.
+	 */
+	atomic_set(&obd->u.cli.cl_mgc_refcount, 1);
+
+	/* We connect to the MGS at setup, and don't disconnect until cleanup */
+	data->ocd_connect_flags = OBD_CONNECT_VERSION | OBD_CONNECT_AT |
+				  OBD_CONNECT_FULL20 | OBD_CONNECT_IMP_RECOV |
+				  OBD_CONNECT_LVB_TYPE | OBD_CONNECT_BULK_MBITS;
+
+#if OBD_OCD_VERSION(3, 0, 53, 0) > LUSTRE_VERSION_CODE
+	data->ocd_connect_flags |= OBD_CONNECT_MNE_SWAB;
+#endif
+
+	if (lmd_is_client(lsi->lsi_lmd) &&
+	    lsi->lsi_lmd->lmd_flags & LMD_FLG_NOIR)
+		data->ocd_connect_flags &= ~OBD_CONNECT_IMP_RECOV;
+	data->ocd_version = LUSTRE_VERSION_CODE;
+	rc = obd_connect(NULL, &exp, obd, &obd->obd_uuid, data, NULL);
+	if (rc) {
+		CERROR("connect failed %d\n", rc);
+		goto out;
+	}
+
+	obd->u.cli.cl_mgc_mgsexp = exp;
+
+out:
+	/* Keep the mgc info in the sb. Note that many lsi's can point
+	 * to the same mgc.
+	 */
+	lsi->lsi_mgc = obd;
+out_free:
+	mutex_unlock(&mgc_start_lock);
+
+	kfree(data);
+	kfree(mgcname);
+	kfree(niduuid);
+	return rc;
+}
+
+static int lustre_stop_mgc(struct super_block *sb)
+{
+	struct lustre_sb_info *lsi = s2lsi(sb);
+	struct obd_device *obd;
+	char *niduuid = NULL, *ptr = NULL;
+	int i, rc = 0, len = 0;
+
+	if (!lsi)
+		return -ENOENT;
+	obd = lsi->lsi_mgc;
+	if (!obd)
+		return -ENOENT;
+	lsi->lsi_mgc = NULL;
+
+	mutex_lock(&mgc_start_lock);
+	LASSERT(atomic_read(&obd->u.cli.cl_mgc_refcount) > 0);
+	if (!atomic_dec_and_test(&obd->u.cli.cl_mgc_refcount)) {
+		/* This is not fatal, every client that stops
+		 * will call in here.
+		 */
+		CDEBUG(D_MOUNT, "mgc still has %d references.\n",
+		       atomic_read(&obd->u.cli.cl_mgc_refcount));
+		rc = -EBUSY;
+		goto out;
+	}
+
+	/* The MGC has no recoverable data in any case.
+	 * force shutdown set in umount_begin
+	 */
+	obd->obd_no_recov = 1;
+
+	if (obd->u.cli.cl_mgc_mgsexp) {
+		/* An error is not fatal, if we are unable to send the
+		 * disconnect mgs ping evictor cleans up the export
+		 */
+		rc = obd_disconnect(obd->u.cli.cl_mgc_mgsexp);
+		if (rc)
+			CDEBUG(D_MOUNT, "disconnect failed %d\n", rc);
+	}
+
+	/* Save the obdname for cleaning the nid uuids, which are obdname_XX */
+	len = strlen(obd->obd_name) + 6;
+	niduuid = kzalloc(len, GFP_NOFS);
+	if (niduuid) {
+		strcpy(niduuid, obd->obd_name);
+		ptr = niduuid + strlen(niduuid);
+	}
+
+	rc = class_manual_cleanup(obd);
+	if (rc)
+		goto out;
+
+	/* Clean the nid uuids */
+	if (!niduuid) {
+		rc = -ENOMEM;
+		goto out;
+	}
+
+	for (i = 0; i < lsi->lsi_lmd->lmd_mgs_failnodes; i++) {
+		sprintf(ptr, "_%x", i);
+		rc = do_lcfg(LUSTRE_MGC_OBDNAME, 0, LCFG_DEL_UUID,
+			     niduuid, NULL, NULL, NULL);
+		if (rc)
+			CERROR("del MDC UUID %s failed: rc = %d\n",
+			       niduuid, rc);
+	}
+out:
+	kfree(niduuid);
+
+	/* class_import_put will get rid of the additional connections */
+	mutex_unlock(&mgc_start_lock);
+	return rc;
+}
+
+/***************** lustre superblock **************/
+
+static struct lustre_sb_info *lustre_init_lsi(struct super_block *sb)
+{
+	struct lustre_sb_info *lsi;
+
+	lsi = kzalloc(sizeof(*lsi), GFP_NOFS);
+	if (!lsi)
+		return NULL;
+	lsi->lsi_lmd = kzalloc(sizeof(*lsi->lsi_lmd), GFP_NOFS);
+	if (!lsi->lsi_lmd) {
+		kfree(lsi);
+		return NULL;
+	}
+
+	lsi->lsi_lmd->lmd_exclude_count = 0;
+	lsi->lsi_lmd->lmd_recovery_time_soft = 0;
+	lsi->lsi_lmd->lmd_recovery_time_hard = 0;
+	s2lsi_nocast(sb) = lsi;
+	/* we take 1 extra ref for our setup */
+	atomic_set(&lsi->lsi_mounts, 1);
+
+	/* Default umount style */
+	lsi->lsi_flags = LSI_UMOUNT_FAILOVER;
+
+	return lsi;
+}
+
+static int lustre_free_lsi(struct super_block *sb)
+{
+	struct lustre_sb_info *lsi = s2lsi(sb);
+
+	CDEBUG(D_MOUNT, "Freeing lsi %p\n", lsi);
+
+	/* someone didn't call server_put_mount. */
+	LASSERT(atomic_read(&lsi->lsi_mounts) == 0);
+
+	if (lsi->lsi_lmd) {
+		kfree(lsi->lsi_lmd->lmd_dev);
+		kfree(lsi->lsi_lmd->lmd_profile);
+		kfree(lsi->lsi_lmd->lmd_fileset);
+		kfree(lsi->lsi_lmd->lmd_mgssec);
+		kfree(lsi->lsi_lmd->lmd_opts);
+		if (lsi->lsi_lmd->lmd_exclude_count)
+			kfree(lsi->lsi_lmd->lmd_exclude);
+		kfree(lsi->lsi_lmd->lmd_mgs);
+		kfree(lsi->lsi_lmd->lmd_osd_type);
+		kfree(lsi->lsi_lmd->lmd_params);
+
+		kfree(lsi->lsi_lmd);
+	}
+
+	LASSERT(!lsi->lsi_llsbi);
+	kfree(lsi);
+	s2lsi_nocast(sb) = NULL;
+
+	return 0;
+}
+
+/* The lsi has one reference for every server that is using the disk -
+ * e.g. MDT, MGS, and potentially MGC
+ */
+static int lustre_put_lsi(struct super_block *sb)
+{
+	struct lustre_sb_info *lsi = s2lsi(sb);
+
+	CDEBUG(D_MOUNT, "put %p %d\n", sb, atomic_read(&lsi->lsi_mounts));
+	if (atomic_dec_and_test(&lsi->lsi_mounts)) {
+		lustre_free_lsi(sb);
+		return 1;
+	}
+	return 0;
+}
+
+/*************** mount common between server and client ***************/
+
+/* Common umount */
+int lustre_common_put_super(struct super_block *sb)
+{
+	int rc;
+
+	CDEBUG(D_MOUNT, "dropping sb %p\n", sb);
+
+	/* Drop a ref to the MGC */
+	rc = lustre_stop_mgc(sb);
+	if (rc && (rc != -ENOENT)) {
+		if (rc != -EBUSY) {
+			CERROR("Can't stop MGC: %d\n", rc);
+			return rc;
+		}
+		/* BUSY just means that there's some other obd that
+		 * needs the mgc.  Let him clean it up.
+		 */
+		CDEBUG(D_MOUNT, "MGC still in use\n");
+	}
+	/* Drop a ref to the mounted disk */
+	lustre_put_lsi(sb);
+	return rc;
+}
+
+static void lmd_print(struct lustre_mount_data *lmd)
+{
+	int i;
+
+	PRINT_CMD(D_MOUNT, "  mount data:\n");
+	if (lmd_is_client(lmd))
+		PRINT_CMD(D_MOUNT, "profile: %s\n", lmd->lmd_profile);
+	PRINT_CMD(D_MOUNT, "device:  %s\n", lmd->lmd_dev);
+	PRINT_CMD(D_MOUNT, "flags:   %x\n", lmd->lmd_flags);
+
+	if (lmd->lmd_opts)
+		PRINT_CMD(D_MOUNT, "options: %s\n", lmd->lmd_opts);
+
+	if (lmd->lmd_recovery_time_soft)
+		PRINT_CMD(D_MOUNT, "recovery time soft: %d\n",
+			  lmd->lmd_recovery_time_soft);
+
+	if (lmd->lmd_recovery_time_hard)
+		PRINT_CMD(D_MOUNT, "recovery time hard: %d\n",
+			  lmd->lmd_recovery_time_hard);
+
+	for (i = 0; i < lmd->lmd_exclude_count; i++) {
+		PRINT_CMD(D_MOUNT, "exclude %d:  OST%04x\n", i,
+			  lmd->lmd_exclude[i]);
+	}
+}
+
+/* mount -v  -o exclude=lustre-OST0001:lustre-OST0002 -t lustre ... */
+static int lmd_make_exclusion(struct lustre_mount_data *lmd, const char *ptr)
+{
+	const char *s1 = ptr, *s2;
+	__u32 index = 0, *exclude_list;
+	int rc = 0, devmax;
+
+	/* The shortest an ost name can be is 8 chars: -OST0000.
+	 * We don't actually know the fsname at this time, so in fact
+	 * a user could specify any fsname.
+	 */
+	devmax = strlen(ptr) / 8 + 1;
+
+	/* temp storage until we figure out how many we have */
+	exclude_list = kcalloc(devmax, sizeof(index), GFP_NOFS);
+	if (!exclude_list)
+		return -ENOMEM;
+
+	/* we enter this fn pointing at the '=' */
+	while (*s1 && *s1 != ' ' && *s1 != ',') {
+		s1++;
+		rc = server_name2index(s1, &index, &s2);
+		if (rc < 0) {
+			CERROR("Can't parse server name '%s': rc = %d\n",
+			       s1, rc);
+			break;
+		}
+		if (rc == LDD_F_SV_TYPE_OST)
+			exclude_list[lmd->lmd_exclude_count++] = index;
+		else
+			CDEBUG(D_MOUNT, "ignoring exclude %.*s: type = %#x\n",
+			       (uint)(s2 - s1), s1, rc);
+		s1 = s2;
+		/* now we are pointing at ':' (next exclude)
+		 * or ',' (end of excludes)
+		 */
+		if (lmd->lmd_exclude_count >= devmax)
+			break;
+	}
+	if (rc >= 0) /* non-err */
+		rc = 0;
+
+	if (lmd->lmd_exclude_count) {
+		/* permanent, freed in lustre_free_lsi */
+		lmd->lmd_exclude = kcalloc(lmd->lmd_exclude_count,
+					   sizeof(index), GFP_NOFS);
+		if (lmd->lmd_exclude) {
+			memcpy(lmd->lmd_exclude, exclude_list,
+			       sizeof(index) * lmd->lmd_exclude_count);
+		} else {
+			rc = -ENOMEM;
+			lmd->lmd_exclude_count = 0;
+		}
+	}
+	kfree(exclude_list);
+	return rc;
+}
+
+static int lmd_parse_mgssec(struct lustre_mount_data *lmd, char *ptr)
+{
+	char   *tail;
+	int     length;
+
+	kfree(lmd->lmd_mgssec);
+	lmd->lmd_mgssec = NULL;
+
+	tail = strchr(ptr, ',');
+	if (!tail)
+		length = strlen(ptr);
+	else
+		length = tail - ptr;
+
+	lmd->lmd_mgssec = kzalloc(length + 1, GFP_NOFS);
+	if (!lmd->lmd_mgssec)
+		return -ENOMEM;
+
+	memcpy(lmd->lmd_mgssec, ptr, length);
+	lmd->lmd_mgssec[length] = '\0';
+	return 0;
+}
+
+static int lmd_parse_string(char **handle, char *ptr)
+{
+	char   *tail;
+	int     length;
+
+	if (!handle || !ptr)
+		return -EINVAL;
+
+	kfree(*handle);
+	*handle = NULL;
+
+	tail = strchr(ptr, ',');
+	if (!tail)
+		length = strlen(ptr);
+	else
+		length = tail - ptr;
+
+	*handle = kzalloc(length + 1, GFP_NOFS);
+	if (!*handle)
+		return -ENOMEM;
+
+	memcpy(*handle, ptr, length);
+	(*handle)[length] = '\0';
+
+	return 0;
+}
+
+/* Collect multiple values for mgsnid specifiers */
+static int lmd_parse_mgs(struct lustre_mount_data *lmd, char **ptr)
+{
+	lnet_nid_t nid;
+	char *tail = *ptr;
+	char *mgsnid;
+	int   length;
+	int   oldlen = 0;
+
+	/* Find end of nidlist */
+	while (class_parse_nid_quiet(tail, &nid, &tail) == 0)
+		;
+	length = tail - *ptr;
+	if (length == 0) {
+		LCONSOLE_ERROR_MSG(0x159, "Can't parse NID '%s'\n", *ptr);
+		return -EINVAL;
+	}
+
+	if (lmd->lmd_mgs)
+		oldlen = strlen(lmd->lmd_mgs) + 1;
+
+	mgsnid = kzalloc(oldlen + length + 1, GFP_NOFS);
+	if (!mgsnid)
+		return -ENOMEM;
+
+	if (lmd->lmd_mgs) {
+		/* Multiple mgsnid= are taken to mean failover locations */
+		memcpy(mgsnid, lmd->lmd_mgs, oldlen);
+		mgsnid[oldlen - 1] = ':';
+		kfree(lmd->lmd_mgs);
+	}
+	memcpy(mgsnid + oldlen, *ptr, length);
+	mgsnid[oldlen + length] = '\0';
+	lmd->lmd_mgs = mgsnid;
+	*ptr = tail;
+
+	return 0;
+}
+
+/**
+ * Find the first delimiter (comma or colon) from the specified \a buf and
+ * make \a *endh point to the string starting with the delimiter. The commas
+ * in expression list [...] will be skipped.
+ *
+ * @buf		a delimiter-separated string
+ * @endh	a pointer to a pointer that will point to the string
+ *		starting with the delimiter
+ *
+ * RETURNS	true if delimiter is found, false if delimiter is not found
+ */
+static bool lmd_find_delimiter(char *buf, char **endh)
+{
+	char *c = buf;
+	size_t pos;
+	bool found;
+
+	if (!buf)
+		return false;
+try_again:
+	if (*c == ',' || *c == ':')
+		return true;
+
+	pos = strcspn(c, "[:,]");
+	if (!pos)
+		return false;
+
+	/* Not a valid mount string */
+	if (*c == ']') {
+		CWARN("invalid mount string format\n");
+		return false;
+	}
+
+	c += pos;
+	if (*c == '[') {
+		c = strchr(c, ']');
+
+		/* invalid mount string */
+		if (!c) {
+			CWARN("invalid mount string format\n");
+			return false;
+		}
+		c++;
+		goto try_again;
+	}
+
+	found = *c != '\0';
+	if (found && endh)
+		*endh = c;
+
+	return found;
+}
+
+/**
+ * Find the first valid string delimited by comma or colon from the specified
+ * \a buf and parse it to see whether it's a valid nid list. If yes, \a *endh
+ * will point to the next string starting with the delimiter.
+ *
+ * \param[in] buf	a delimiter-separated string
+ * \param[in] endh	a pointer to a pointer that will point to the string
+ *			starting with the delimiter
+ *
+ * \retval 0		if the string is a valid nid list
+ * \retval 1		if the string is not a valid nid list
+ */
+static int lmd_parse_nidlist(char *buf, char **endh)
+{
+	struct list_head nidlist;
+	char *endp = buf;
+	int rc = 0;
+	char tmp;
+
+	if (!buf)
+		return 1;
+	while (*buf == ',' || *buf == ':')
+		buf++;
+	if (*buf == ' ' || *buf == '/' || *buf == '\0')
+		return 1;
+
+	if (!lmd_find_delimiter(buf, &endp))
+		endp = buf + strlen(buf);
+
+	tmp = *endp;
+	*endp = '\0';
+
+	INIT_LIST_HEAD(&nidlist);
+	if (cfs_parse_nidlist(buf, strlen(buf), &nidlist) <= 0)
+		rc = 1;
+	cfs_free_nidlist(&nidlist);
+
+	*endp = tmp;
+	if (rc)
+		return rc;
+	if (endh)
+		*endh = endp;
+	return 0;
+}
+
+/** Parse mount line options
+ * e.g. mount -v -t lustre -o abort_recov uml1:uml2:/lustre-client /mnt/lustre
+ * dev is passed as device=uml1:/lustre by mount.lustre
+ */
+static int lmd_parse(char *options, struct lustre_mount_data *lmd)
+{
+	char *s1, *s2, *devname = NULL;
+	struct lustre_mount_data *raw = (struct lustre_mount_data *)options;
+	int rc = 0;
+
+	LASSERT(lmd);
+	if (!options) {
+		LCONSOLE_ERROR_MSG(0x162, "Missing mount data: check that /sbin/mount.lustre is installed.\n");
+		return -EINVAL;
+	}
+
+	/* Options should be a string - try to detect old lmd data */
+	if ((raw->lmd_magic & 0xffffff00) == (LMD_MAGIC & 0xffffff00)) {
+		LCONSOLE_ERROR_MSG(0x163, "You're using an old version of /sbin/mount.lustre.  Please install version %s\n",
+				   LUSTRE_VERSION_STRING);
+		return -EINVAL;
+	}
+	lmd->lmd_magic = LMD_MAGIC;
+
+	lmd->lmd_params = kzalloc(LMD_PARAMS_MAXLEN, GFP_NOFS);
+	if (!lmd->lmd_params)
+		return -ENOMEM;
+	lmd->lmd_params[0] = '\0';
+
+	/* Set default flags here */
+
+	s1 = options;
+	while (*s1) {
+		int clear = 0;
+		int time_min = OBD_RECOVERY_TIME_MIN;
+		char *s3;
+
+		/* Skip whitespace and extra commas */
+		while (*s1 == ' ' || *s1 == ',')
+			s1++;
+		s3 = s1;
+
+		/* Client options are parsed in ll_options: eg. flock,
+		 * user_xattr, acl
+		 */
+
+		/* Parse non-ldiskfs options here. Rather than modifying
+		 * ldiskfs, we just zero these out here
+		 */
+		if (strncmp(s1, "abort_recov", 11) == 0) {
+			lmd->lmd_flags |= LMD_FLG_ABORT_RECOV;
+			clear++;
+		} else if (strncmp(s1, "recovery_time_soft=", 19) == 0) {
+			lmd->lmd_recovery_time_soft = max_t(int,
+				simple_strtoul(s1 + 19, NULL, 10), time_min);
+			clear++;
+		} else if (strncmp(s1, "recovery_time_hard=", 19) == 0) {
+			lmd->lmd_recovery_time_hard = max_t(int,
+				simple_strtoul(s1 + 19, NULL, 10), time_min);
+			clear++;
+		} else if (strncmp(s1, "noir", 4) == 0) {
+			lmd->lmd_flags |= LMD_FLG_NOIR; /* test purpose only. */
+			clear++;
+		} else if (strncmp(s1, "nosvc", 5) == 0) {
+			lmd->lmd_flags |= LMD_FLG_NOSVC;
+			clear++;
+		} else if (strncmp(s1, "nomgs", 5) == 0) {
+			lmd->lmd_flags |= LMD_FLG_NOMGS;
+			clear++;
+		} else if (strncmp(s1, "noscrub", 7) == 0) {
+			lmd->lmd_flags |= LMD_FLG_NOSCRUB;
+			clear++;
+		} else if (strncmp(s1, PARAM_MGSNODE,
+				   sizeof(PARAM_MGSNODE) - 1) == 0) {
+			s2 = s1 + sizeof(PARAM_MGSNODE) - 1;
+			/* Assume the next mount opt is the first
+			 * invalid nid we get to.
+			 */
+			rc = lmd_parse_mgs(lmd, &s2);
+			if (rc)
+				goto invalid;
+			clear++;
+		} else if (strncmp(s1, "writeconf", 9) == 0) {
+			lmd->lmd_flags |= LMD_FLG_WRITECONF;
+			clear++;
+		} else if (strncmp(s1, "update", 6) == 0) {
+			lmd->lmd_flags |= LMD_FLG_UPDATE;
+			clear++;
+		} else if (strncmp(s1, "virgin", 6) == 0) {
+			lmd->lmd_flags |= LMD_FLG_VIRGIN;
+			clear++;
+		} else if (strncmp(s1, "noprimnode", 10) == 0) {
+			lmd->lmd_flags |= LMD_FLG_NO_PRIMNODE;
+			clear++;
+		} else if (strncmp(s1, "mgssec=", 7) == 0) {
+			rc = lmd_parse_mgssec(lmd, s1 + 7);
+			if (rc)
+				goto invalid;
+			s3 = s2;
+			clear++;
+		/* ost exclusion list */
+		} else if (strncmp(s1, "exclude=", 8) == 0) {
+			rc = lmd_make_exclusion(lmd, s1 + 7);
+			if (rc)
+				goto invalid;
+			clear++;
+		} else if (strncmp(s1, "mgs", 3) == 0) {
+			/* We are an MGS */
+			lmd->lmd_flags |= LMD_FLG_MGS;
+			clear++;
+		} else if (strncmp(s1, "svname=", 7) == 0) {
+			rc = lmd_parse_string(&lmd->lmd_profile, s1 + 7);
+			if (rc)
+				goto invalid;
+			clear++;
+		} else if (strncmp(s1, "param=", 6) == 0) {
+			size_t length, params_length;
+			char *tail = s1;
+
+			if (lmd_find_delimiter(s1 + 6, &tail)) {
+				char *param_str = tail + 1;
+				int supplementary = 1;
+
+				while (!lmd_parse_nidlist(param_str,
+							  &param_str))
+					supplementary = 0;
+				length = param_str - s1 - supplementary;
+			} else {
+				length = strlen(s1);
+			}
+			length -= 6;
+			params_length = strlen(lmd->lmd_params);
+			if (params_length + length + 1 >= LMD_PARAMS_MAXLEN)
+				return -E2BIG;
+			strncat(lmd->lmd_params, s1 + 6, length);
+			lmd->lmd_params[params_length + length] = '\0';
+			strlcat(lmd->lmd_params, " ", LMD_PARAMS_MAXLEN);
+			s3 = s1 + 6 + length;
+			clear++;
+		} else if (strncmp(s1, "osd=", 4) == 0) {
+			rc = lmd_parse_string(&lmd->lmd_osd_type, s1 + 4);
+			if (rc)
+				goto invalid;
+			clear++;
+		}
+		/* Linux 2.4 doesn't pass the device, so we stuck it at the
+		 * end of the options.
+		 */
+		else if (strncmp(s1, "device=", 7) == 0) {
+			devname = s1 + 7;
+			/* terminate options right before device.  device
+			 * must be the last one.
+			 */
+			*s1 = '\0';
+			break;
+		}
+
+		/* Find next opt */
+		s2 = strchr(s1, ',');
+		if (!s2) {
+			if (clear)
+				*s1 = '\0';
+			break;
+		}
+		s2++;
+		if (clear)
+			memmove(s1, s2, strlen(s2) + 1);
+		else
+			s1 = s2;
+	}
+
+	if (!devname) {
+		LCONSOLE_ERROR_MSG(0x164, "Can't find the device name (need mount option 'device=...')\n");
+		goto invalid;
+	}
+
+	s1 = strstr(devname, ":/");
+	if (s1) {
+		++s1;
+		lmd->lmd_flags |= LMD_FLG_CLIENT;
+		/* Remove leading /s from fsname */
+		while (*++s1 == '/')
+			;
+		s2 = strchrnul(s1, '/');
+		/* Freed in lustre_free_lsi */
+		lmd->lmd_profile = kasprintf(GFP_KERNEL, "%.*s-client",
+					     (int)(s2 - s1), s1);
+		if (!lmd->lmd_profile)
+			return -ENOMEM;
+
+		s1 = s2;
+		s2 = s1 + strlen(s1) - 1;
+		/* Remove padding /s from fileset */
+		while (*s2 == '/')
+			s2--;
+		if (s2 > s1) {
+			lmd->lmd_fileset = kstrndup(s1, s2 - s1 + 1,
+						    GFP_KERNEL);
+			if (!lmd->lmd_fileset)
+				return -ENOMEM;
+		}
+	}
+
+	/* Freed in lustre_free_lsi */
+	lmd->lmd_dev = kstrdup(devname, GFP_KERNEL);
+	if (!lmd->lmd_dev)
+		return -ENOMEM;
+
+	/* Save mount options */
+	s1 = options + strlen(options) - 1;
+	while (s1 >= options && (*s1 == ',' || *s1 == ' '))
+		*s1-- = 0;
+	if (*options != 0) {
+		/* Freed in lustre_free_lsi */
+		lmd->lmd_opts = kstrdup(options, GFP_KERNEL);
+		if (!lmd->lmd_opts)
+			return -ENOMEM;
+	}
+
+	lmd_print(lmd);
+	lmd->lmd_magic = LMD_MAGIC;
+
+	return rc;
+
+invalid:
+	CERROR("Bad mount options %s\n", options);
+	return -EINVAL;
+}
+
+/** This is the entry point for the mount call into Lustre.
+ * This is called when a server or client is mounted,
+ * and this is where we start setting things up.
+ * @param data Mount options (e.g. -o flock,abort_recov)
+ */
+static int lustre_fill_super(struct super_block *sb, void *lmd2_data, int silent)
+{
+	struct lustre_mount_data *lmd;
+	struct lustre_sb_info *lsi;
+	int rc;
+
+	CDEBUG(D_MOUNT | D_VFSTRACE, "VFS Op: sb %p\n", sb);
+
+	lsi = lustre_init_lsi(sb);
+	if (!lsi)
+		return -ENOMEM;
+	lmd = lsi->lsi_lmd;
+
+	/*
+	 * Disable lockdep during mount, because mount locking patterns are
+	 * `special'.
+	 */
+	lockdep_off();
+
+	/*
+	 * LU-639: the obd cleanup of last mount may not finish yet, wait here.
+	 */
+	obd_zombie_barrier();
+
+	/* Figure out the lmd from the mount options */
+	if (lmd_parse(lmd2_data, lmd)) {
+		lustre_put_lsi(sb);
+		rc = -EINVAL;
+		goto out;
+	}
+
+	if (lmd_is_client(lmd)) {
+		CDEBUG(D_MOUNT, "Mounting client %s\n", lmd->lmd_profile);
+
+		rc = lustre_start_mgc(sb);
+		if (rc) {
+			lustre_common_put_super(sb);
+			goto out;
+		}
+		/* Connect and start */
+		rc = ll_fill_super(sb);
+		/* l_f_s will call lustre_common_put_super on failure, otherwise
+		 * l_f_s will have taken another reference to the module */
+	} else {
+		CERROR("This is client-side-only module, cannot handle server mount.\n");
+		rc = -EINVAL;
+	}
+
+	/* If error happens in fill_super() call, @lsi will be killed there.
+	 * This is why we do not put it here.
+	 */
+	goto out;
+out:
+	if (rc) {
+		CERROR("Unable to mount %s (%d)\n",
+		       s2lsi(sb) ? lmd->lmd_dev : "", rc);
+	} else {
+		CDEBUG(D_SUPER, "Mount %s complete\n",
+		       lmd->lmd_dev);
+	}
+	lockdep_on();
+	return rc;
+}
+
+/***************** FS registration ******************/
+static struct dentry *lustre_mount(struct file_system_type *fs_type, int flags,
+				   const char *devname, void *data)
+{
+	return mount_nodev(fs_type, flags, data, lustre_fill_super);
+}
+
+static void lustre_kill_super(struct super_block *sb)
+{
+	struct lustre_sb_info *lsi = s2lsi(sb);
+
+	if (lsi)
+		ll_kill_super(sb);
+
+	kill_anon_super(sb);
+}
+
+/** Register the "lustre" fs type
+ */
+static struct file_system_type lustre_fs_type = {
+	.owner		= THIS_MODULE,
+	.name		= "lustre",
+	.mount		= lustre_mount,
+	.kill_sb	= lustre_kill_super,
+	.fs_flags	= FS_RENAME_DOES_D_MOVE,
+};
+MODULE_ALIAS_FS("lustre");
+
+int lustre_register_fs(void)
+{
+	return register_filesystem(&lustre_fs_type);
+}
+
+int lustre_unregister_fs(void)
+{
+	return unregister_filesystem(&lustre_fs_type);
+}
diff --git a/drivers/staging/lustre/lustre/llite/super25.c b/drivers/staging/lustre/lustre/llite/super25.c
index d335f29556c2..cb1e12e8097a 100644
--- a/drivers/staging/lustre/lustre/llite/super25.c
+++ b/drivers/staging/lustre/lustre/llite/super25.c
@@ -145,10 +145,11 @@ static int __init lustre_init(void)
 	if (rc != 0)
 		goto out_inode_fini_env;
 
-	lustre_register_super_ops(THIS_MODULE, ll_fill_super, ll_kill_super);
 	lustre_register_client_process_config(ll_process_config);
 
-	return 0;
+	rc = lustre_register_fs();
+
+	return rc;
 
 out_inode_fini_env:
 	cl_env_put(cl_inode_fini_env, &cl_inode_fini_refcheck);
@@ -166,7 +167,8 @@ static int __init lustre_init(void)
 
 static void __exit lustre_exit(void)
 {
-	lustre_register_super_ops(NULL, NULL, NULL);
+	lustre_unregister_fs();
+
 	lustre_register_client_process_config(NULL);
 
 	debugfs_remove(llite_root);
diff --git a/drivers/staging/lustre/lustre/obdclass/Makefile b/drivers/staging/lustre/lustre/obdclass/Makefile
index 686bd546f06d..2b99514a4966 100644
--- a/drivers/staging/lustre/lustre/obdclass/Makefile
+++ b/drivers/staging/lustre/lustre/obdclass/Makefile
@@ -8,5 +8,5 @@ obdclass-y := module.o sysctl.o \
 	      llog.o llog_cat.o llog_obd.o llog_swab.o class_obd.o debug.o \
 	      genops.o uuid.o lprocfs_status.o lprocfs_counters.o \
 	      lustre_handles.o lustre_peer.o statfs_pack.o linkea.o \
-	      obdo.o obd_config.o obd_mount.o lu_object.o lu_ref.o \
+	      obdo.o obd_config.o lu_object.o lu_ref.o \
 	      cl_object.o cl_page.o cl_lock.o cl_io.o kernelcomm.o
diff --git a/drivers/staging/lustre/lustre/obdclass/class_obd.c b/drivers/staging/lustre/lustre/obdclass/class_obd.c
index 81a4c666bb69..cdaf7293bc18 100644
--- a/drivers/staging/lustre/lustre/obdclass/class_obd.c
+++ b/drivers/staging/lustre/lustre/obdclass/class_obd.c
@@ -510,18 +510,12 @@ static int __init obdclass_init(void)
 		return err;
 
 	err = llog_info_init();
-	if (err)
-		return err;
-
-	err = lustre_register_fs();
 
 	return err;
 }
 
 static void obdclass_exit(void)
 {
-	lustre_unregister_fs();
-
 	misc_deregister(&obd_psdev);
 	llog_info_fini();
 	cl_global_fini();
diff --git a/drivers/staging/lustre/lustre/obdclass/obd_mount.c b/drivers/staging/lustre/lustre/obdclass/obd_mount.c
deleted file mode 100644
index 4e4a8c35105b..000000000000
--- a/drivers/staging/lustre/lustre/obdclass/obd_mount.c
+++ /dev/null
@@ -1,1244 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-/*
- * GPL HEADER START
- *
- * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 only,
- * as published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful, but
- * WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
- * General Public License version 2 for more details (a copy is included
- * in the LICENSE file that accompanied this code).
- *
- * You should have received a copy of the GNU General Public License
- * version 2 along with this program; If not, see
- * http://www.gnu.org/licenses/gpl-2.0.html
- *
- * GPL HEADER END
- */
-/*
- * Copyright (c) 2007, 2010, Oracle and/or its affiliates. All rights reserved.
- * Use is subject to license terms.
- *
- * Copyright (c) 2011, 2015, Intel Corporation.
- */
-/*
- * This file is part of Lustre, http://www.lustre.org/
- * Lustre is a trademark of Sun Microsystems, Inc.
- *
- * lustre/obdclass/obd_mount.c
- *
- * Client mount routines
- *
- * Author: Nathan Rutman <nathan at clusterfs.com>
- */
-
-#define DEBUG_SUBSYSTEM S_CLASS
-#define D_MOUNT (D_SUPER | D_CONFIG/*|D_WARNING */)
-#define PRINT_CMD CDEBUG
-
-#include <linux/random.h>
-#include <obd.h>
-#include <obd_class.h>
-#include <uapi/linux/lustre/lustre_idl.h>
-#include <lustre_log.h>
-#include <lustre_disk.h>
-#include <uapi/linux/lustre/lustre_param.h>
-
-static DEFINE_SPINLOCK(client_lock);
-static struct module *client_mod;
-static int (*client_fill_super)(struct super_block *sb);
-static void (*kill_super_cb)(struct super_block *sb);
-
-/**************** config llog ********************/
-
-/** Get a config log from the MGS and process it.
- * This func is called for both clients and servers.
- * Continue to process new statements appended to the logs
- * (whenever the config lock is revoked) until lustre_end_log
- * is called.
- * @param sb The superblock is used by the MGC to write to the local copy of
- *   the config log
- * @param logname The name of the llog to replicate from the MGS
- * @param cfg Since the same mgc may be used to follow multiple config logs
- *   (e.g. ost1, ost2, client), the config_llog_instance keeps the state for
- *   this log, and is added to the mgc's list of logs to follow.
- */
-int lustre_process_log(struct super_block *sb, char *logname,
-		       struct config_llog_instance *cfg)
-{
-	struct lustre_cfg *lcfg;
-	struct lustre_cfg_bufs *bufs;
-	struct lustre_sb_info *lsi = s2lsi(sb);
-	struct obd_device *mgc = lsi->lsi_mgc;
-	int rc;
-
-	LASSERT(mgc);
-	LASSERT(cfg);
-
-	bufs = kzalloc(sizeof(*bufs), GFP_NOFS);
-	if (!bufs)
-		return -ENOMEM;
-
-	/* mgc_process_config */
-	lustre_cfg_bufs_reset(bufs, mgc->obd_name);
-	lustre_cfg_bufs_set_string(bufs, 1, logname);
-	lustre_cfg_bufs_set(bufs, 2, cfg, sizeof(*cfg));
-	lustre_cfg_bufs_set(bufs, 3, &sb, sizeof(sb));
-	lcfg = kzalloc(lustre_cfg_len(bufs->lcfg_bufcount, bufs->lcfg_buflen),
-		       GFP_NOFS);
-	if (!lcfg) {
-		rc = -ENOMEM;
-		goto out;
-	}
-	lustre_cfg_init(lcfg, LCFG_LOG_START, bufs);
-
-	rc = obd_process_config(mgc, sizeof(*lcfg), lcfg);
-	kfree(lcfg);
-out:
-	kfree(bufs);
-
-	if (rc == -EINVAL)
-		LCONSOLE_ERROR_MSG(0x15b, "%s: The configuration from log '%s' failed from the MGS (%d).  Make sure this client and the MGS are running compatible versions of Lustre.\n",
-				   mgc->obd_name, logname, rc);
-
-	else if (rc)
-		LCONSOLE_ERROR_MSG(0x15c, "%s: The configuration from log '%s' failed (%d). This may be the result of communication errors between this node and the MGS, a bad configuration, or other errors. See the syslog for more information.\n",
-				   mgc->obd_name, logname,
-				   rc);
-
-	/* class_obd_list(); */
-	return rc;
-}
-EXPORT_SYMBOL(lustre_process_log);
-
-/* Stop watching this config log for updates */
-int lustre_end_log(struct super_block *sb, char *logname,
-		   struct config_llog_instance *cfg)
-{
-	struct lustre_cfg *lcfg;
-	struct lustre_cfg_bufs bufs;
-	struct lustre_sb_info *lsi = s2lsi(sb);
-	struct obd_device *mgc = lsi->lsi_mgc;
-	int rc;
-
-	if (!mgc)
-		return -ENOENT;
-
-	/* mgc_process_config */
-	lustre_cfg_bufs_reset(&bufs, mgc->obd_name);
-	lustre_cfg_bufs_set_string(&bufs, 1, logname);
-	if (cfg)
-		lustre_cfg_bufs_set(&bufs, 2, cfg, sizeof(*cfg));
-	lcfg = kzalloc(lustre_cfg_len(bufs.lcfg_bufcount, bufs.lcfg_buflen),
-		       GFP_NOFS);
-	if (!lcfg)
-		return -ENOMEM;
-	lustre_cfg_init(lcfg, LCFG_LOG_END, &bufs);
-
-	rc = obd_process_config(mgc, sizeof(*lcfg), lcfg);
-	kfree(lcfg);
-	return rc;
-}
-EXPORT_SYMBOL(lustre_end_log);
-
-/**************** obd start *******************/
-
-/** lustre_cfg_bufs are a holdover from 1.4; we can still set these up from
- * lctl (and do for echo cli/srv.
- */
-static int do_lcfg(char *cfgname, lnet_nid_t nid, int cmd,
-		   char *s1, char *s2, char *s3, char *s4)
-{
-	struct lustre_cfg_bufs bufs;
-	struct lustre_cfg     *lcfg = NULL;
-	int rc;
-
-	CDEBUG(D_TRACE, "lcfg %s %#x %s %s %s %s\n", cfgname,
-	       cmd, s1, s2, s3, s4);
-
-	lustre_cfg_bufs_reset(&bufs, cfgname);
-	if (s1)
-		lustre_cfg_bufs_set_string(&bufs, 1, s1);
-	if (s2)
-		lustre_cfg_bufs_set_string(&bufs, 2, s2);
-	if (s3)
-		lustre_cfg_bufs_set_string(&bufs, 3, s3);
-	if (s4)
-		lustre_cfg_bufs_set_string(&bufs, 4, s4);
-
-	lcfg = kzalloc(lustre_cfg_len(bufs.lcfg_bufcount, bufs.lcfg_buflen),
-		       GFP_NOFS);
-	if (!lcfg)
-		return -ENOMEM;
-	lustre_cfg_init(lcfg, cmd, &bufs);
-	lcfg->lcfg_nid = nid;
-	rc = class_process_config(lcfg);
-	kfree(lcfg);
-	return rc;
-}
-
-/** Call class_attach and class_setup.  These methods in turn call
- * obd type-specific methods.
- */
-static int lustre_start_simple(char *obdname, char *type, char *uuid,
-			       char *s1, char *s2, char *s3, char *s4)
-{
-	int rc;
-
-	CDEBUG(D_MOUNT, "Starting obd %s (typ=%s)\n", obdname, type);
-
-	rc = do_lcfg(obdname, 0, LCFG_ATTACH, type, uuid, NULL, NULL);
-	if (rc) {
-		CERROR("%s attach error %d\n", obdname, rc);
-		return rc;
-	}
-	rc = do_lcfg(obdname, 0, LCFG_SETUP, s1, s2, s3, s4);
-	if (rc) {
-		CERROR("%s setup error %d\n", obdname, rc);
-		do_lcfg(obdname, 0, LCFG_DETACH, NULL, NULL, NULL, NULL);
-	}
-	return rc;
-}
-
-static DEFINE_MUTEX(mgc_start_lock);
-
-/** Set up a mgc obd to process startup logs
- *
- * \param sb [in] super block of the mgc obd
- *
- * \retval 0 success, otherwise error code
- */
-int lustre_start_mgc(struct super_block *sb)
-{
-	struct obd_connect_data *data = NULL;
-	struct lustre_sb_info *lsi = s2lsi(sb);
-	struct obd_device *obd;
-	struct obd_export *exp;
-	struct obd_uuid *uuid;
-	class_uuid_t uuidc;
-	lnet_nid_t nid;
-	char nidstr[LNET_NIDSTR_SIZE];
-	char *mgcname = NULL, *niduuid = NULL, *mgssec = NULL;
-	char *ptr;
-	int rc = 0, i = 0, j;
-
-	LASSERT(lsi->lsi_lmd);
-
-	/* Use nids from mount line: uml1,1 at elan:uml2,2 at elan:/lustre */
-	ptr = lsi->lsi_lmd->lmd_dev;
-	if (class_parse_nid(ptr, &nid, &ptr) == 0)
-		i++;
-	if (i == 0) {
-		CERROR("No valid MGS nids found.\n");
-		return -EINVAL;
-	}
-
-	mutex_lock(&mgc_start_lock);
-
-	libcfs_nid2str_r(nid, nidstr, sizeof(nidstr));
-	mgcname = kasprintf(GFP_NOFS,
-			    "%s%s", LUSTRE_MGC_OBDNAME, nidstr);
-	niduuid = kasprintf(GFP_NOFS, "%s_%x", mgcname, 0);
-	if (!mgcname || !niduuid) {
-		rc = -ENOMEM;
-		goto out_free;
-	}
-
-	mgssec = lsi->lsi_lmd->lmd_mgssec ? lsi->lsi_lmd->lmd_mgssec : "";
-
-	data = kzalloc(sizeof(*data), GFP_NOFS);
-	if (!data) {
-		rc = -ENOMEM;
-		goto out_free;
-	}
-
-	obd = class_name2obd(mgcname);
-	if (obd && !obd->obd_stopping) {
-		int recov_bk;
-
-		rc = obd_set_info_async(NULL, obd->obd_self_export,
-					strlen(KEY_MGSSEC), KEY_MGSSEC,
-					strlen(mgssec), mgssec, NULL);
-		if (rc)
-			goto out_free;
-
-		/* Re-using an existing MGC */
-		atomic_inc(&obd->u.cli.cl_mgc_refcount);
-
-		/* IR compatibility check, only for clients */
-		if (lmd_is_client(lsi->lsi_lmd)) {
-			int has_ir;
-			int vallen = sizeof(*data);
-			__u32 *flags = &lsi->lsi_lmd->lmd_flags;
-
-			rc = obd_get_info(NULL, obd->obd_self_export,
-					  strlen(KEY_CONN_DATA), KEY_CONN_DATA,
-					  &vallen, data);
-			LASSERT(rc == 0);
-			has_ir = OCD_HAS_FLAG(data, IMP_RECOV);
-			if (has_ir ^ !(*flags & LMD_FLG_NOIR)) {
-				/* LMD_FLG_NOIR is for test purpose only */
-				LCONSOLE_WARN(
-					"Trying to mount a client with IR setting not compatible with current mgc. Force to use current mgc setting that is IR %s.\n",
-					has_ir ? "enabled" : "disabled");
-				if (has_ir)
-					*flags &= ~LMD_FLG_NOIR;
-				else
-					*flags |= LMD_FLG_NOIR;
-			}
-		}
-
-		recov_bk = 0;
-
-		/* Try all connections, but only once (again).
-		 * We don't want to block another target from starting
-		 * (using its local copy of the log), but we do want to connect
-		 * if at all possible.
-		 */
-		recov_bk++;
-		CDEBUG(D_MOUNT, "%s: Set MGC reconnect %d\n", mgcname,
-		       recov_bk);
-		rc = obd_set_info_async(NULL, obd->obd_self_export,
-					sizeof(KEY_INIT_RECOV_BACKUP),
-					KEY_INIT_RECOV_BACKUP,
-					sizeof(recov_bk), &recov_bk, NULL);
-		rc = 0;
-		goto out;
-	}
-
-	CDEBUG(D_MOUNT, "Start MGC '%s'\n", mgcname);
-
-	/* Add the primary nids for the MGS */
-	i = 0;
-	/* Use nids from mount line: uml1,1 at elan:uml2,2 at elan:/lustre */
-	ptr = lsi->lsi_lmd->lmd_dev;
-	while (class_parse_nid(ptr, &nid, &ptr) == 0) {
-		rc = do_lcfg(mgcname, nid,
-			     LCFG_ADD_UUID, niduuid, NULL, NULL, NULL);
-		if (!rc)
-			i++;
-		/* Stop at the first failover nid */
-		if (*ptr == ':')
-			break;
-	}
-	if (i == 0) {
-		CERROR("No valid MGS nids found.\n");
-		rc = -EINVAL;
-		goto out_free;
-	}
-	lsi->lsi_lmd->lmd_mgs_failnodes = 1;
-
-	/* Random uuid for MGC allows easier reconnects */
-	uuid = kzalloc(sizeof(*uuid), GFP_NOFS);
-	if (!uuid) {
-		rc = -ENOMEM;
-		goto out_free;
-	}
-
-	ll_generate_random_uuid(uuidc);
-	class_uuid_unparse(uuidc, uuid);
-
-	/* Start the MGC */
-	rc = lustre_start_simple(mgcname, LUSTRE_MGC_NAME,
-				 (char *)uuid->uuid, LUSTRE_MGS_OBDNAME,
-				 niduuid, NULL, NULL);
-	kfree(uuid);
-	if (rc)
-		goto out_free;
-
-	/* Add any failover MGS nids */
-	i = 1;
-	while (ptr && ((*ptr == ':' ||
-			class_find_param(ptr, PARAM_MGSNODE, &ptr) == 0))) {
-		/* New failover node */
-		sprintf(niduuid, "%s_%x", mgcname, i);
-		j = 0;
-		while (class_parse_nid_quiet(ptr, &nid, &ptr) == 0) {
-			rc = do_lcfg(mgcname, nid, LCFG_ADD_UUID, niduuid,
-				     NULL, NULL, NULL);
-			if (!rc)
-				++j;
-			if (*ptr == ':')
-				break;
-		}
-		if (j > 0) {
-			rc = do_lcfg(mgcname, 0, LCFG_ADD_CONN,
-				     niduuid, NULL, NULL, NULL);
-			if (!rc)
-				i++;
-		} else {
-			/* at ":/fsname" */
-			break;
-		}
-	}
-	lsi->lsi_lmd->lmd_mgs_failnodes = i;
-
-	obd = class_name2obd(mgcname);
-	if (!obd) {
-		CERROR("Can't find mgcobd %s\n", mgcname);
-		rc = -ENOTCONN;
-		goto out_free;
-	}
-
-	rc = obd_set_info_async(NULL, obd->obd_self_export,
-				strlen(KEY_MGSSEC), KEY_MGSSEC,
-				strlen(mgssec), mgssec, NULL);
-	if (rc)
-		goto out_free;
-
-	/* Keep a refcount of servers/clients who started with "mount",
-	 * so we know when we can get rid of the mgc.
-	 */
-	atomic_set(&obd->u.cli.cl_mgc_refcount, 1);
-
-	/* We connect to the MGS at setup, and don't disconnect until cleanup */
-	data->ocd_connect_flags = OBD_CONNECT_VERSION | OBD_CONNECT_AT |
-				  OBD_CONNECT_FULL20 | OBD_CONNECT_IMP_RECOV |
-				  OBD_CONNECT_LVB_TYPE | OBD_CONNECT_BULK_MBITS;
-
-#if OBD_OCD_VERSION(3, 0, 53, 0) > LUSTRE_VERSION_CODE
-	data->ocd_connect_flags |= OBD_CONNECT_MNE_SWAB;
-#endif
-
-	if (lmd_is_client(lsi->lsi_lmd) &&
-	    lsi->lsi_lmd->lmd_flags & LMD_FLG_NOIR)
-		data->ocd_connect_flags &= ~OBD_CONNECT_IMP_RECOV;
-	data->ocd_version = LUSTRE_VERSION_CODE;
-	rc = obd_connect(NULL, &exp, obd, &obd->obd_uuid, data, NULL);
-	if (rc) {
-		CERROR("connect failed %d\n", rc);
-		goto out;
-	}
-
-	obd->u.cli.cl_mgc_mgsexp = exp;
-
-out:
-	/* Keep the mgc info in the sb. Note that many lsi's can point
-	 * to the same mgc.
-	 */
-	lsi->lsi_mgc = obd;
-out_free:
-	mutex_unlock(&mgc_start_lock);
-
-	kfree(data);
-	kfree(mgcname);
-	kfree(niduuid);
-	return rc;
-}
-
-static int lustre_stop_mgc(struct super_block *sb)
-{
-	struct lustre_sb_info *lsi = s2lsi(sb);
-	struct obd_device *obd;
-	char *niduuid = NULL, *ptr = NULL;
-	int i, rc = 0, len = 0;
-
-	if (!lsi)
-		return -ENOENT;
-	obd = lsi->lsi_mgc;
-	if (!obd)
-		return -ENOENT;
-	lsi->lsi_mgc = NULL;
-
-	mutex_lock(&mgc_start_lock);
-	LASSERT(atomic_read(&obd->u.cli.cl_mgc_refcount) > 0);
-	if (!atomic_dec_and_test(&obd->u.cli.cl_mgc_refcount)) {
-		/* This is not fatal, every client that stops
-		 * will call in here.
-		 */
-		CDEBUG(D_MOUNT, "mgc still has %d references.\n",
-		       atomic_read(&obd->u.cli.cl_mgc_refcount));
-		rc = -EBUSY;
-		goto out;
-	}
-
-	/* The MGC has no recoverable data in any case.
-	 * force shutdown set in umount_begin
-	 */
-	obd->obd_no_recov = 1;
-
-	if (obd->u.cli.cl_mgc_mgsexp) {
-		/* An error is not fatal, if we are unable to send the
-		 * disconnect mgs ping evictor cleans up the export
-		 */
-		rc = obd_disconnect(obd->u.cli.cl_mgc_mgsexp);
-		if (rc)
-			CDEBUG(D_MOUNT, "disconnect failed %d\n", rc);
-	}
-
-	/* Save the obdname for cleaning the nid uuids, which are obdname_XX */
-	len = strlen(obd->obd_name) + 6;
-	niduuid = kzalloc(len, GFP_NOFS);
-	if (niduuid) {
-		strcpy(niduuid, obd->obd_name);
-		ptr = niduuid + strlen(niduuid);
-	}
-
-	rc = class_manual_cleanup(obd);
-	if (rc)
-		goto out;
-
-	/* Clean the nid uuids */
-	if (!niduuid) {
-		rc = -ENOMEM;
-		goto out;
-	}
-
-	for (i = 0; i < lsi->lsi_lmd->lmd_mgs_failnodes; i++) {
-		sprintf(ptr, "_%x", i);
-		rc = do_lcfg(LUSTRE_MGC_OBDNAME, 0, LCFG_DEL_UUID,
-			     niduuid, NULL, NULL, NULL);
-		if (rc)
-			CERROR("del MDC UUID %s failed: rc = %d\n",
-			       niduuid, rc);
-	}
-out:
-	kfree(niduuid);
-
-	/* class_import_put will get rid of the additional connections */
-	mutex_unlock(&mgc_start_lock);
-	return rc;
-}
-
-/***************** lustre superblock **************/
-
-static struct lustre_sb_info *lustre_init_lsi(struct super_block *sb)
-{
-	struct lustre_sb_info *lsi;
-
-	lsi = kzalloc(sizeof(*lsi), GFP_NOFS);
-	if (!lsi)
-		return NULL;
-	lsi->lsi_lmd = kzalloc(sizeof(*lsi->lsi_lmd), GFP_NOFS);
-	if (!lsi->lsi_lmd) {
-		kfree(lsi);
-		return NULL;
-	}
-
-	lsi->lsi_lmd->lmd_exclude_count = 0;
-	lsi->lsi_lmd->lmd_recovery_time_soft = 0;
-	lsi->lsi_lmd->lmd_recovery_time_hard = 0;
-	s2lsi_nocast(sb) = lsi;
-	/* we take 1 extra ref for our setup */
-	atomic_set(&lsi->lsi_mounts, 1);
-
-	/* Default umount style */
-	lsi->lsi_flags = LSI_UMOUNT_FAILOVER;
-
-	return lsi;
-}
-
-static int lustre_free_lsi(struct super_block *sb)
-{
-	struct lustre_sb_info *lsi = s2lsi(sb);
-
-	CDEBUG(D_MOUNT, "Freeing lsi %p\n", lsi);
-
-	/* someone didn't call server_put_mount. */
-	LASSERT(atomic_read(&lsi->lsi_mounts) == 0);
-
-	if (lsi->lsi_lmd) {
-		kfree(lsi->lsi_lmd->lmd_dev);
-		kfree(lsi->lsi_lmd->lmd_profile);
-		kfree(lsi->lsi_lmd->lmd_fileset);
-		kfree(lsi->lsi_lmd->lmd_mgssec);
-		kfree(lsi->lsi_lmd->lmd_opts);
-		if (lsi->lsi_lmd->lmd_exclude_count)
-			kfree(lsi->lsi_lmd->lmd_exclude);
-		kfree(lsi->lsi_lmd->lmd_mgs);
-		kfree(lsi->lsi_lmd->lmd_osd_type);
-		kfree(lsi->lsi_lmd->lmd_params);
-
-		kfree(lsi->lsi_lmd);
-	}
-
-	LASSERT(!lsi->lsi_llsbi);
-	kfree(lsi);
-	s2lsi_nocast(sb) = NULL;
-
-	return 0;
-}
-
-/* The lsi has one reference for every server that is using the disk -
- * e.g. MDT, MGS, and potentially MGC
- */
-static int lustre_put_lsi(struct super_block *sb)
-{
-	struct lustre_sb_info *lsi = s2lsi(sb);
-
-	CDEBUG(D_MOUNT, "put %p %d\n", sb, atomic_read(&lsi->lsi_mounts));
-	if (atomic_dec_and_test(&lsi->lsi_mounts)) {
-		lustre_free_lsi(sb);
-		return 1;
-	}
-	return 0;
-}
-
-/*************** mount common between server and client ***************/
-
-/* Common umount */
-int lustre_common_put_super(struct super_block *sb)
-{
-	int rc;
-
-	CDEBUG(D_MOUNT, "dropping sb %p\n", sb);
-
-	/* Drop a ref to the MGC */
-	rc = lustre_stop_mgc(sb);
-	if (rc && (rc != -ENOENT)) {
-		if (rc != -EBUSY) {
-			CERROR("Can't stop MGC: %d\n", rc);
-			return rc;
-		}
-		/* BUSY just means that there's some other obd that
-		 * needs the mgc.  Let him clean it up.
-		 */
-		CDEBUG(D_MOUNT, "MGC still in use\n");
-	}
-	/* Drop a ref to the mounted disk */
-	lustre_put_lsi(sb);
-	return rc;
-}
-EXPORT_SYMBOL(lustre_common_put_super);
-
-static void lmd_print(struct lustre_mount_data *lmd)
-{
-	int i;
-
-	PRINT_CMD(D_MOUNT, "  mount data:\n");
-	if (lmd_is_client(lmd))
-		PRINT_CMD(D_MOUNT, "profile: %s\n", lmd->lmd_profile);
-	PRINT_CMD(D_MOUNT, "device:  %s\n", lmd->lmd_dev);
-	PRINT_CMD(D_MOUNT, "flags:   %x\n", lmd->lmd_flags);
-
-	if (lmd->lmd_opts)
-		PRINT_CMD(D_MOUNT, "options: %s\n", lmd->lmd_opts);
-
-	if (lmd->lmd_recovery_time_soft)
-		PRINT_CMD(D_MOUNT, "recovery time soft: %d\n",
-			  lmd->lmd_recovery_time_soft);
-
-	if (lmd->lmd_recovery_time_hard)
-		PRINT_CMD(D_MOUNT, "recovery time hard: %d\n",
-			  lmd->lmd_recovery_time_hard);
-
-	for (i = 0; i < lmd->lmd_exclude_count; i++) {
-		PRINT_CMD(D_MOUNT, "exclude %d:  OST%04x\n", i,
-			  lmd->lmd_exclude[i]);
-	}
-}
-
-/* mount -v  -o exclude=lustre-OST0001:lustre-OST0002 -t lustre ... */
-static int lmd_make_exclusion(struct lustre_mount_data *lmd, const char *ptr)
-{
-	const char *s1 = ptr, *s2;
-	__u32 index = 0, *exclude_list;
-	int rc = 0, devmax;
-
-	/* The shortest an ost name can be is 8 chars: -OST0000.
-	 * We don't actually know the fsname at this time, so in fact
-	 * a user could specify any fsname.
-	 */
-	devmax = strlen(ptr) / 8 + 1;
-
-	/* temp storage until we figure out how many we have */
-	exclude_list = kcalloc(devmax, sizeof(index), GFP_NOFS);
-	if (!exclude_list)
-		return -ENOMEM;
-
-	/* we enter this fn pointing at the '=' */
-	while (*s1 && *s1 != ' ' && *s1 != ',') {
-		s1++;
-		rc = server_name2index(s1, &index, &s2);
-		if (rc < 0) {
-			CERROR("Can't parse server name '%s': rc = %d\n",
-			       s1, rc);
-			break;
-		}
-		if (rc == LDD_F_SV_TYPE_OST)
-			exclude_list[lmd->lmd_exclude_count++] = index;
-		else
-			CDEBUG(D_MOUNT, "ignoring exclude %.*s: type = %#x\n",
-			       (uint)(s2 - s1), s1, rc);
-		s1 = s2;
-		/* now we are pointing at ':' (next exclude)
-		 * or ',' (end of excludes)
-		 */
-		if (lmd->lmd_exclude_count >= devmax)
-			break;
-	}
-	if (rc >= 0) /* non-err */
-		rc = 0;
-
-	if (lmd->lmd_exclude_count) {
-		/* permanent, freed in lustre_free_lsi */
-		lmd->lmd_exclude = kcalloc(lmd->lmd_exclude_count,
-					   sizeof(index), GFP_NOFS);
-		if (lmd->lmd_exclude) {
-			memcpy(lmd->lmd_exclude, exclude_list,
-			       sizeof(index) * lmd->lmd_exclude_count);
-		} else {
-			rc = -ENOMEM;
-			lmd->lmd_exclude_count = 0;
-		}
-	}
-	kfree(exclude_list);
-	return rc;
-}
-
-static int lmd_parse_mgssec(struct lustre_mount_data *lmd, char *ptr)
-{
-	char   *tail;
-	int     length;
-
-	kfree(lmd->lmd_mgssec);
-	lmd->lmd_mgssec = NULL;
-
-	tail = strchr(ptr, ',');
-	if (!tail)
-		length = strlen(ptr);
-	else
-		length = tail - ptr;
-
-	lmd->lmd_mgssec = kzalloc(length + 1, GFP_NOFS);
-	if (!lmd->lmd_mgssec)
-		return -ENOMEM;
-
-	memcpy(lmd->lmd_mgssec, ptr, length);
-	lmd->lmd_mgssec[length] = '\0';
-	return 0;
-}
-
-static int lmd_parse_string(char **handle, char *ptr)
-{
-	char   *tail;
-	int     length;
-
-	if (!handle || !ptr)
-		return -EINVAL;
-
-	kfree(*handle);
-	*handle = NULL;
-
-	tail = strchr(ptr, ',');
-	if (!tail)
-		length = strlen(ptr);
-	else
-		length = tail - ptr;
-
-	*handle = kzalloc(length + 1, GFP_NOFS);
-	if (!*handle)
-		return -ENOMEM;
-
-	memcpy(*handle, ptr, length);
-	(*handle)[length] = '\0';
-
-	return 0;
-}
-
-/* Collect multiple values for mgsnid specifiers */
-static int lmd_parse_mgs(struct lustre_mount_data *lmd, char **ptr)
-{
-	lnet_nid_t nid;
-	char *tail = *ptr;
-	char *mgsnid;
-	int   length;
-	int   oldlen = 0;
-
-	/* Find end of nidlist */
-	while (class_parse_nid_quiet(tail, &nid, &tail) == 0)
-		;
-	length = tail - *ptr;
-	if (length == 0) {
-		LCONSOLE_ERROR_MSG(0x159, "Can't parse NID '%s'\n", *ptr);
-		return -EINVAL;
-	}
-
-	if (lmd->lmd_mgs)
-		oldlen = strlen(lmd->lmd_mgs) + 1;
-
-	mgsnid = kzalloc(oldlen + length + 1, GFP_NOFS);
-	if (!mgsnid)
-		return -ENOMEM;
-
-	if (lmd->lmd_mgs) {
-		/* Multiple mgsnid= are taken to mean failover locations */
-		memcpy(mgsnid, lmd->lmd_mgs, oldlen);
-		mgsnid[oldlen - 1] = ':';
-		kfree(lmd->lmd_mgs);
-	}
-	memcpy(mgsnid + oldlen, *ptr, length);
-	mgsnid[oldlen + length] = '\0';
-	lmd->lmd_mgs = mgsnid;
-	*ptr = tail;
-
-	return 0;
-}
-
-/**
- * Find the first delimiter (comma or colon) from the specified \a buf and
- * make \a *endh point to the string starting with the delimiter. The commas
- * in expression list [...] will be skipped.
- *
- * @buf		a delimiter-separated string
- * @endh	a pointer to a pointer that will point to the string
- *		starting with the delimiter
- *
- * RETURNS	true if delimiter is found, false if delimiter is not found
- */
-static bool lmd_find_delimiter(char *buf, char **endh)
-{
-	char *c = buf;
-	size_t pos;
-	bool found;
-
-	if (!buf)
-		return false;
-try_again:
-	if (*c == ',' || *c == ':')
-		return true;
-
-	pos = strcspn(c, "[:,]");
-	if (!pos)
-		return false;
-
-	/* Not a valid mount string */
-	if (*c == ']') {
-		CWARN("invalid mount string format\n");
-		return false;
-	}
-
-	c += pos;
-	if (*c == '[') {
-		c = strchr(c, ']');
-
-		/* invalid mount string */
-		if (!c) {
-			CWARN("invalid mount string format\n");
-			return false;
-		}
-		c++;
-		goto try_again;
-	}
-
-	found = *c != '\0';
-	if (found && endh)
-		*endh = c;
-
-	return found;
-}
-
-/**
- * Find the first valid string delimited by comma or colon from the specified
- * \a buf and parse it to see whether it's a valid nid list. If yes, \a *endh
- * will point to the next string starting with the delimiter.
- *
- * \param[in] buf	a delimiter-separated string
- * \param[in] endh	a pointer to a pointer that will point to the string
- *			starting with the delimiter
- *
- * \retval 0		if the string is a valid nid list
- * \retval 1		if the string is not a valid nid list
- */
-static int lmd_parse_nidlist(char *buf, char **endh)
-{
-	struct list_head nidlist;
-	char *endp = buf;
-	int rc = 0;
-	char tmp;
-
-	if (!buf)
-		return 1;
-	while (*buf == ',' || *buf == ':')
-		buf++;
-	if (*buf == ' ' || *buf == '/' || *buf == '\0')
-		return 1;
-
-	if (!lmd_find_delimiter(buf, &endp))
-		endp = buf + strlen(buf);
-
-	tmp = *endp;
-	*endp = '\0';
-
-	INIT_LIST_HEAD(&nidlist);
-	if (cfs_parse_nidlist(buf, strlen(buf), &nidlist) <= 0)
-		rc = 1;
-	cfs_free_nidlist(&nidlist);
-
-	*endp = tmp;
-	if (rc)
-		return rc;
-	if (endh)
-		*endh = endp;
-	return 0;
-}
-
-/** Parse mount line options
- * e.g. mount -v -t lustre -o abort_recov uml1:uml2:/lustre-client /mnt/lustre
- * dev is passed as device=uml1:/lustre by mount.lustre
- */
-static int lmd_parse(char *options, struct lustre_mount_data *lmd)
-{
-	char *s1, *s2, *devname = NULL;
-	struct lustre_mount_data *raw = (struct lustre_mount_data *)options;
-	int rc = 0;
-
-	LASSERT(lmd);
-	if (!options) {
-		LCONSOLE_ERROR_MSG(0x162, "Missing mount data: check that /sbin/mount.lustre is installed.\n");
-		return -EINVAL;
-	}
-
-	/* Options should be a string - try to detect old lmd data */
-	if ((raw->lmd_magic & 0xffffff00) == (LMD_MAGIC & 0xffffff00)) {
-		LCONSOLE_ERROR_MSG(0x163, "You're using an old version of /sbin/mount.lustre.  Please install version %s\n",
-				   LUSTRE_VERSION_STRING);
-		return -EINVAL;
-	}
-	lmd->lmd_magic = LMD_MAGIC;
-
-	lmd->lmd_params = kzalloc(LMD_PARAMS_MAXLEN, GFP_NOFS);
-	if (!lmd->lmd_params)
-		return -ENOMEM;
-	lmd->lmd_params[0] = '\0';
-
-	/* Set default flags here */
-
-	s1 = options;
-	while (*s1) {
-		int clear = 0;
-		int time_min = OBD_RECOVERY_TIME_MIN;
-		char *s3;
-
-		/* Skip whitespace and extra commas */
-		while (*s1 == ' ' || *s1 == ',')
-			s1++;
-		s3 = s1;
-
-		/* Client options are parsed in ll_options: eg. flock,
-		 * user_xattr, acl
-		 */
-
-		/* Parse non-ldiskfs options here. Rather than modifying
-		 * ldiskfs, we just zero these out here
-		 */
-		if (strncmp(s1, "abort_recov", 11) == 0) {
-			lmd->lmd_flags |= LMD_FLG_ABORT_RECOV;
-			clear++;
-		} else if (strncmp(s1, "recovery_time_soft=", 19) == 0) {
-			lmd->lmd_recovery_time_soft = max_t(int,
-				simple_strtoul(s1 + 19, NULL, 10), time_min);
-			clear++;
-		} else if (strncmp(s1, "recovery_time_hard=", 19) == 0) {
-			lmd->lmd_recovery_time_hard = max_t(int,
-				simple_strtoul(s1 + 19, NULL, 10), time_min);
-			clear++;
-		} else if (strncmp(s1, "noir", 4) == 0) {
-			lmd->lmd_flags |= LMD_FLG_NOIR; /* test purpose only. */
-			clear++;
-		} else if (strncmp(s1, "nosvc", 5) == 0) {
-			lmd->lmd_flags |= LMD_FLG_NOSVC;
-			clear++;
-		} else if (strncmp(s1, "nomgs", 5) == 0) {
-			lmd->lmd_flags |= LMD_FLG_NOMGS;
-			clear++;
-		} else if (strncmp(s1, "noscrub", 7) == 0) {
-			lmd->lmd_flags |= LMD_FLG_NOSCRUB;
-			clear++;
-		} else if (strncmp(s1, PARAM_MGSNODE,
-				   sizeof(PARAM_MGSNODE) - 1) == 0) {
-			s2 = s1 + sizeof(PARAM_MGSNODE) - 1;
-			/* Assume the next mount opt is the first
-			 * invalid nid we get to.
-			 */
-			rc = lmd_parse_mgs(lmd, &s2);
-			if (rc)
-				goto invalid;
-			clear++;
-		} else if (strncmp(s1, "writeconf", 9) == 0) {
-			lmd->lmd_flags |= LMD_FLG_WRITECONF;
-			clear++;
-		} else if (strncmp(s1, "update", 6) == 0) {
-			lmd->lmd_flags |= LMD_FLG_UPDATE;
-			clear++;
-		} else if (strncmp(s1, "virgin", 6) == 0) {
-			lmd->lmd_flags |= LMD_FLG_VIRGIN;
-			clear++;
-		} else if (strncmp(s1, "noprimnode", 10) == 0) {
-			lmd->lmd_flags |= LMD_FLG_NO_PRIMNODE;
-			clear++;
-		} else if (strncmp(s1, "mgssec=", 7) == 0) {
-			rc = lmd_parse_mgssec(lmd, s1 + 7);
-			if (rc)
-				goto invalid;
-			s3 = s2;
-			clear++;
-		/* ost exclusion list */
-		} else if (strncmp(s1, "exclude=", 8) == 0) {
-			rc = lmd_make_exclusion(lmd, s1 + 7);
-			if (rc)
-				goto invalid;
-			clear++;
-		} else if (strncmp(s1, "mgs", 3) == 0) {
-			/* We are an MGS */
-			lmd->lmd_flags |= LMD_FLG_MGS;
-			clear++;
-		} else if (strncmp(s1, "svname=", 7) == 0) {
-			rc = lmd_parse_string(&lmd->lmd_profile, s1 + 7);
-			if (rc)
-				goto invalid;
-			clear++;
-		} else if (strncmp(s1, "param=", 6) == 0) {
-			size_t length, params_length;
-			char *tail = s1;
-
-			if (lmd_find_delimiter(s1 + 6, &tail)) {
-				char *param_str = tail + 1;
-				int supplementary = 1;
-
-				while (!lmd_parse_nidlist(param_str,
-							  &param_str))
-					supplementary = 0;
-				length = param_str - s1 - supplementary;
-			} else {
-				length = strlen(s1);
-			}
-			length -= 6;
-			params_length = strlen(lmd->lmd_params);
-			if (params_length + length + 1 >= LMD_PARAMS_MAXLEN)
-				return -E2BIG;
-			strncat(lmd->lmd_params, s1 + 6, length);
-			lmd->lmd_params[params_length + length] = '\0';
-			strlcat(lmd->lmd_params, " ", LMD_PARAMS_MAXLEN);
-			s3 = s1 + 6 + length;
-			clear++;
-		} else if (strncmp(s1, "osd=", 4) == 0) {
-			rc = lmd_parse_string(&lmd->lmd_osd_type, s1 + 4);
-			if (rc)
-				goto invalid;
-			clear++;
-		}
-		/* Linux 2.4 doesn't pass the device, so we stuck it at the
-		 * end of the options.
-		 */
-		else if (strncmp(s1, "device=", 7) == 0) {
-			devname = s1 + 7;
-			/* terminate options right before device.  device
-			 * must be the last one.
-			 */
-			*s1 = '\0';
-			break;
-		}
-
-		/* Find next opt */
-		s2 = strchr(s1, ',');
-		if (!s2) {
-			if (clear)
-				*s1 = '\0';
-			break;
-		}
-		s2++;
-		if (clear)
-			memmove(s1, s2, strlen(s2) + 1);
-		else
-			s1 = s2;
-	}
-
-	if (!devname) {
-		LCONSOLE_ERROR_MSG(0x164, "Can't find the device name (need mount option 'device=...')\n");
-		goto invalid;
-	}
-
-	s1 = strstr(devname, ":/");
-	if (s1) {
-		++s1;
-		lmd->lmd_flags |= LMD_FLG_CLIENT;
-		/* Remove leading /s from fsname */
-		while (*++s1 == '/')
-			;
-		s2 = strchrnul(s1, '/');
-		/* Freed in lustre_free_lsi */
-		lmd->lmd_profile = kasprintf(GFP_KERNEL, "%.*s-client",
-					     (int)(s2 - s1), s1);
-		if (!lmd->lmd_profile)
-			return -ENOMEM;
-
-		s1 = s2;
-		s2 = s1 + strlen(s1) - 1;
-		/* Remove padding /s from fileset */
-		while (*s2 == '/')
-			s2--;
-		if (s2 > s1) {
-			lmd->lmd_fileset = kstrndup(s1, s2 - s1 + 1,
-						    GFP_KERNEL);
-			if (!lmd->lmd_fileset)
-				return -ENOMEM;
-		}
-	}
-
-	/* Freed in lustre_free_lsi */
-	lmd->lmd_dev = kstrdup(devname, GFP_KERNEL);
-	if (!lmd->lmd_dev)
-		return -ENOMEM;
-
-	/* Save mount options */
-	s1 = options + strlen(options) - 1;
-	while (s1 >= options && (*s1 == ',' || *s1 == ' '))
-		*s1-- = 0;
-	if (*options != 0) {
-		/* Freed in lustre_free_lsi */
-		lmd->lmd_opts = kstrdup(options, GFP_KERNEL);
-		if (!lmd->lmd_opts)
-			return -ENOMEM;
-	}
-
-	lmd_print(lmd);
-	lmd->lmd_magic = LMD_MAGIC;
-
-	return rc;
-
-invalid:
-	CERROR("Bad mount options %s\n", options);
-	return -EINVAL;
-}
-
-/** This is the entry point for the mount call into Lustre.
- * This is called when a server or client is mounted,
- * and this is where we start setting things up.
- * @param data Mount options (e.g. -o flock,abort_recov)
- */
-static int lustre_fill_super(struct super_block *sb, void *lmd2_data, int silent)
-{
-	struct lustre_mount_data *lmd;
-	struct lustre_sb_info *lsi;
-	int rc;
-
-	CDEBUG(D_MOUNT | D_VFSTRACE, "VFS Op: sb %p\n", sb);
-
-	lsi = lustre_init_lsi(sb);
-	if (!lsi)
-		return -ENOMEM;
-	lmd = lsi->lsi_lmd;
-
-	/*
-	 * Disable lockdep during mount, because mount locking patterns are
-	 * `special'.
-	 */
-	lockdep_off();
-
-	/*
-	 * LU-639: the obd cleanup of last mount may not finish yet, wait here.
-	 */
-	obd_zombie_barrier();
-
-	/* Figure out the lmd from the mount options */
-	if (lmd_parse(lmd2_data, lmd)) {
-		lustre_put_lsi(sb);
-		rc = -EINVAL;
-		goto out;
-	}
-
-	if (lmd_is_client(lmd)) {
-		bool have_client = false;
-		CDEBUG(D_MOUNT, "Mounting client %s\n", lmd->lmd_profile);
-		if (!client_fill_super)
-			request_module("lustre");
-		spin_lock(&client_lock);
-		if (client_fill_super && try_module_get(client_mod))
-			have_client = true;
-		spin_unlock(&client_lock);
-		if (!have_client) {
-			LCONSOLE_ERROR_MSG(0x165, "Nothing registered for client mount! Is the 'lustre' module loaded?\n");
-			lustre_put_lsi(sb);
-			rc = -ENODEV;
-		} else {
-			rc = lustre_start_mgc(sb);
-			if (rc) {
-				lustre_common_put_super(sb);
-				goto out;
-			}
-			/* Connect and start */
-			/* (should always be ll_fill_super) */
-			rc = (*client_fill_super)(sb);
-			/* c_f_s will call lustre_common_put_super on failure, otherwise
-			 * c_f_s will have taken another reference to the module */
-			module_put(client_mod);
-		}
-	} else {
-		CERROR("This is client-side-only module, cannot handle server mount.\n");
-		rc = -EINVAL;
-	}
-
-	/* If error happens in fill_super() call, @lsi will be killed there.
-	 * This is why we do not put it here.
-	 */
-	goto out;
-out:
-	if (rc) {
-		CERROR("Unable to mount %s (%d)\n",
-		       s2lsi(sb) ? lmd->lmd_dev : "", rc);
-	} else {
-		CDEBUG(D_SUPER, "Mount %s complete\n",
-		       lmd->lmd_dev);
-	}
-	lockdep_on();
-	return rc;
-}
-
-/* We can't call ll_fill_super by name because it lives in a module that
- * must be loaded after this one.
- */
-void lustre_register_super_ops(struct module *mod,
-			       int (*cfs)(struct super_block *sb),
-			       void (*ksc)(struct super_block *sb))
-{
-	spin_lock(&client_lock);
-	client_mod = mod;
-	client_fill_super = cfs;
-	kill_super_cb = ksc;
-	spin_unlock(&client_lock);
-}
-EXPORT_SYMBOL(lustre_register_super_ops);
-
-/***************** FS registration ******************/
-static struct dentry *lustre_mount(struct file_system_type *fs_type, int flags,
-				   const char *devname, void *data)
-{
-	return mount_nodev(fs_type, flags, data, lustre_fill_super);
-}
-
-static void lustre_kill_super(struct super_block *sb)
-{
-	struct lustre_sb_info *lsi = s2lsi(sb);
-
-	if (kill_super_cb && lsi)
-		(*kill_super_cb)(sb);
-
-	kill_anon_super(sb);
-}
-
-/** Register the "lustre" fs type
- */
-static struct file_system_type lustre_fs_type = {
-	.owner		= THIS_MODULE,
-	.name		= "lustre",
-	.mount		= lustre_mount,
-	.kill_sb	= lustre_kill_super,
-	.fs_flags	= FS_RENAME_DOES_D_MOVE,
-};
-MODULE_ALIAS_FS("lustre");
-
-int lustre_register_fs(void)
-{
-	return register_filesystem(&lustre_fs_type);
-}
-
-int lustre_unregister_fs(void)
-{
-	return unregister_filesystem(&lustre_fs_type);
-}



From neilb at suse.com  Mon Jul 23 06:23:04 2018
From: neilb at suse.com (NeilBrown)
Date: Mon, 23 Jul 2018 16:23:04 +1000
Subject: [lustre-devel] [PATCH 6/9] lustre: hold ptlrpc active for
 lustre_start_mgc()
In-Reply-To: <153232696720.26222.9658151633697867322.stgit@noble>
References: <153232696720.26222.9658151633697867322.stgit@noble>
Message-ID: <153232698496.26222.2533442172154163310.stgit@noble>

Commit 26f7a294e5ec ("staging: lustre: ptlrpc: move thread creation
out of module initialization") move ptlrpc initialization out of
module_init() and into explicit ptlrpc_inc_ref() calls.
This was added in ll_fill_super with a put_ref in ll_put_super().
Unfortunately this means lustre_fill_super calls lustre_start_mgc()
without first activating ptlrpc.
So move the inc/put ref calls to lustre_fill_super() and
lustre_common_put_super() respectively.

Fixes: 26f7a294e5ec ("staging: lustre: ptlrpc: move thread creation out of module initialization")
Signed-off-by: NeilBrown <neilb at suse.com>
---
 drivers/staging/lustre/lustre/llite/llite_lib.c |   17 +++--------------
 drivers/staging/lustre/lustre/llite/mount.c     |   10 ++++++++--
 2 files changed, 11 insertions(+), 16 deletions(-)

diff --git a/drivers/staging/lustre/lustre/llite/llite_lib.c b/drivers/staging/lustre/lustre/llite/llite_lib.c
index 64cd69cb493d..e6348345505c 100644
--- a/drivers/staging/lustre/lustre/llite/llite_lib.c
+++ b/drivers/staging/lustre/lustre/llite/llite_lib.c
@@ -911,15 +911,9 @@ int ll_fill_super(struct super_block *sb)
 
 	CDEBUG(D_VFSTRACE, "VFS Op: sb %p\n", sb);
 
-	err = ptlrpc_inc_ref();
-	if (err)
-		return err;
-
 	cfg = kzalloc(sizeof(*cfg), GFP_NOFS);
-	if (!cfg) {
-		err = -ENOMEM;
-		goto out_put;
-	}
+	if (!cfg)
+		return -ENOMEM;
 
 	try_module_get(THIS_MODULE);
 
@@ -929,8 +923,7 @@ int ll_fill_super(struct super_block *sb)
 	if (!sbi) {
 		module_put(THIS_MODULE);
 		kfree(cfg);
-		err = -ENOMEM;
-		goto out_put;
+		return -ENOMEM;
 	}
 
 	err = ll_options(lsi->lsi_lmd->lmd_opts, &sbi->ll_flags);
@@ -997,9 +990,6 @@ int ll_fill_super(struct super_block *sb)
 		LCONSOLE_WARN("Mounted %s\n", profilenm);
 
 	kfree(cfg);
-out_put:
-	if (err)
-		ptlrpc_dec_ref();
 	return err;
 } /* ll_fill_super */
 
@@ -1071,7 +1061,6 @@ void ll_put_super(struct super_block *sb)
 
 	module_put(THIS_MODULE);
 
-	ptlrpc_dec_ref();
 } /* client_put_super */
 
 struct inode *ll_inode_from_resource_lock(struct ldlm_lock *lock)
diff --git a/drivers/staging/lustre/lustre/llite/mount.c b/drivers/staging/lustre/lustre/llite/mount.c
index 58e8b371f0c4..85f0a7fdcd39 100644
--- a/drivers/staging/lustre/lustre/llite/mount.c
+++ b/drivers/staging/lustre/lustre/llite/mount.c
@@ -596,6 +596,7 @@ int lustre_common_put_super(struct super_block *sb)
 	}
 	/* Drop a ref to the mounted disk */
 	lustre_put_lsi(sb);
+	ptlrpc_dec_ref();
 	return rc;
 }
 
@@ -1131,16 +1132,19 @@ static int lustre_fill_super(struct super_block *sb, void *lmd2_data, int silent
 
 	/* Figure out the lmd from the mount options */
 	if (lmd_parse(lmd2_data, lmd)) {
-		lustre_put_lsi(sb);
 		rc = -EINVAL;
-		goto out;
+		goto out_put_lsi;
 	}
 
 	if (lmd_is_client(lmd)) {
 		CDEBUG(D_MOUNT, "Mounting client %s\n", lmd->lmd_profile);
 
+		rc = ptlrpc_inc_ref();
+		if (rc)
+			goto out_put_lsi;
 		rc = lustre_start_mgc(sb);
 		if (rc) {
+			/* This will put_lsi and ptlrpc_dec_ref() */
 			lustre_common_put_super(sb);
 			goto out;
 		}
@@ -1157,6 +1161,8 @@ static int lustre_fill_super(struct super_block *sb, void *lmd2_data, int silent
 	 * This is why we do not put it here.
 	 */
 	goto out;
+out_put_lsi:
+	lustre_put_lsi(sb);
 out:
 	if (rc) {
 		CERROR("Unable to mount %s (%d)\n",



From neilb at suse.com  Mon Jul 23 06:23:05 2018
From: neilb at suse.com (NeilBrown)
Date: Mon, 23 Jul 2018 16:23:05 +1000
Subject: [lustre-devel] [PATCH 7/9] lustre: ll_fill_super() must put sb on
 all failure paths.
In-Reply-To: <153232696720.26222.9658151633697867322.stgit@noble>
References: <153232696720.26222.9658151633697867322.stgit@noble>
Message-ID: <153232698499.26222.12345415044809848331.stgit@noble>

lustre_fill_super() has comment saying:
	/*
	 * l_f_s will call lustre_common_put_super on failure, otherwise
	 * l_f_s will have taken another reference to the module
	 */

But this is not the case - ENOMEM error do not result in a 'put'.
So add appropriate calls to lustre_common_put_super().

Signed-off-by: NeilBrown <neilb at suse.com>
---
 drivers/staging/lustre/lustre/llite/llite_lib.c |    5 ++++-
 drivers/staging/lustre/lustre/llite/mount.c     |    6 ++++--
 2 files changed, 8 insertions(+), 3 deletions(-)

diff --git a/drivers/staging/lustre/lustre/llite/llite_lib.c b/drivers/staging/lustre/lustre/llite/llite_lib.c
index e6348345505c..5c8d0fe7217e 100644
--- a/drivers/staging/lustre/lustre/llite/llite_lib.c
+++ b/drivers/staging/lustre/lustre/llite/llite_lib.c
@@ -912,8 +912,10 @@ int ll_fill_super(struct super_block *sb)
 	CDEBUG(D_VFSTRACE, "VFS Op: sb %p\n", sb);
 
 	cfg = kzalloc(sizeof(*cfg), GFP_NOFS);
-	if (!cfg)
+	if (!cfg) {
+		lustre_common_put_super(sb);
 		return -ENOMEM;
+	}
 
 	try_module_get(THIS_MODULE);
 
@@ -923,6 +925,7 @@ int ll_fill_super(struct super_block *sb)
 	if (!sbi) {
 		module_put(THIS_MODULE);
 		kfree(cfg);
+		lustre_common_put_super(sb);
 		return -ENOMEM;
 	}
 
diff --git a/drivers/staging/lustre/lustre/llite/mount.c b/drivers/staging/lustre/lustre/llite/mount.c
index 85f0a7fdcd39..a560098689ac 100644
--- a/drivers/staging/lustre/lustre/llite/mount.c
+++ b/drivers/staging/lustre/lustre/llite/mount.c
@@ -1150,8 +1150,10 @@ static int lustre_fill_super(struct super_block *sb, void *lmd2_data, int silent
 		}
 		/* Connect and start */
 		rc = ll_fill_super(sb);
-		/* l_f_s will call lustre_common_put_super on failure, otherwise
-		 * l_f_s will have taken another reference to the module */
+		/*
+		 * l_f_s will call lustre_common_put_super on failure, otherwise
+		 * l_f_s will have taken another reference to the module
+		 */
 	} else {
 		CERROR("This is client-side-only module, cannot handle server mount.\n");
 		rc = -EINVAL;



From neilb at suse.com  Mon Jul 23 06:23:05 2018
From: neilb at suse.com (NeilBrown)
Date: Mon, 23 Jul 2018 16:23:05 +1000
Subject: [lustre-devel] [PATCH 8/9] lustre: ensure libcfs is set up for
 ioctls.
In-Reply-To: <153232696720.26222.9658151633697867322.stgit@noble>
References: <153232696720.26222.9658151633697867322.stgit@noble>
Message-ID: <153232698502.26222.8106098993681745318.stgit@noble>

libcfs only allocated various buffers when libcfs_setup()
is called.
This should be called before any significant libcfs related
activity.  However it isn't called by libcfs_ioctl().
So if the first thing that happens is an ioctl, tracing can cause
NULL pointer dereferences.

Fixes: 64bf0b1a079d ("staging: lustre: refactor libcfs initialization.")
Signed-off-by: NeilBrown <neilb at suse.com>
---
 drivers/staging/lustre/lnet/libcfs/module.c |    3 +++
 1 file changed, 3 insertions(+)

diff --git a/drivers/staging/lustre/lnet/libcfs/module.c b/drivers/staging/lustre/lnet/libcfs/module.c
index ad654b56814d..bfadfcfa3c44 100644
--- a/drivers/staging/lustre/lnet/libcfs/module.c
+++ b/drivers/staging/lustre/lnet/libcfs/module.c
@@ -206,6 +206,9 @@ static int libcfs_ioctl(unsigned long cmd, void __user *uparam)
 	struct libcfs_ioctl_hdr *hdr;
 	int err;
 
+	err = libcfs_setup();
+	if (err)
+		return err;
 	/* 'cmd' and permissions get checked in our arch-specific caller */
 	err = libcfs_ioctl_getdata(&hdr, uparam);
 	if (err) {



From neilb at suse.com  Mon Jul 23 06:23:05 2018
From: neilb at suse.com (NeilBrown)
Date: Mon, 23 Jul 2018 16:23:05 +1000
Subject: [lustre-devel] [PATCH 9/9] lustre: lnet: discard LNET_LOCK()
In-Reply-To: <153232696720.26222.9658151633697867322.stgit@noble>
References: <153232696720.26222.9658151633697867322.stgit@noble>
Message-ID: <153232698505.26222.7268545921228126117.stgit@noble>

This macro, and LNET_UNLOCK() are rarely used, don't
add clarify, and make greping for lock usage harder.
So discard macro and just the lnet_net_{un,}lock()
like everyone else.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 .../staging/lustre/include/linux/lnet/lib-lnet.h   |    3 ---
 .../lustre/lnet/klnds/socklnd/socklnd_proto.c      |    8 ++++----
 drivers/staging/lustre/lnet/selftest/rpc.c         |    2 +-
 3 files changed, 5 insertions(+), 8 deletions(-)

diff --git a/drivers/staging/lustre/include/linux/lnet/lib-lnet.h b/drivers/staging/lustre/include/linux/lnet/lib-lnet.h
index 6b6289cfcd3d..8ff8139e04fe 100644
--- a/drivers/staging/lustre/include/linux/lnet/lib-lnet.h
+++ b/drivers/staging/lustre/include/linux/lnet/lib-lnet.h
@@ -177,9 +177,6 @@ lnet_net_lock_current(void)
 	return cpt;
 }
 
-#define LNET_LOCK()		lnet_net_lock(LNET_LOCK_EX)
-#define LNET_UNLOCK()		lnet_net_unlock(LNET_LOCK_EX)
-
 #define lnet_ptl_lock(ptl)	spin_lock(&(ptl)->ptl_lock)
 #define lnet_ptl_unlock(ptl)	spin_unlock(&(ptl)->ptl_lock)
 #define lnet_eq_wait_lock()	spin_lock(&the_lnet.ln_eq_wait_lock)
diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_proto.c b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_proto.c
index 05982dac781c..aaa04a5f0527 100644
--- a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_proto.c
+++ b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_proto.c
@@ -485,7 +485,7 @@ ksocknal_send_hello_v1(struct ksock_conn *conn, struct ksock_hello_msg *hello)
 
 	if (the_lnet.ln_testprotocompat) {
 		/* single-shot proto check */
-		LNET_LOCK();
+		lnet_net_lock(LNET_LOCK_EX);
 		if (the_lnet.ln_testprotocompat & 1) {
 			hmv->version_major++;   /* just different! */
 			the_lnet.ln_testprotocompat &= ~1;
@@ -494,7 +494,7 @@ ksocknal_send_hello_v1(struct ksock_conn *conn, struct ksock_hello_msg *hello)
 			hmv->magic = LNET_PROTO_MAGIC;
 			the_lnet.ln_testprotocompat &= ~2;
 		}
-		LNET_UNLOCK();
+		lnet_net_unlock(LNET_LOCK_EX);
 	}
 
 	hdr->src_nid = cpu_to_le64(hello->kshm_src_nid);
@@ -542,12 +542,12 @@ ksocknal_send_hello_v2(struct ksock_conn *conn, struct ksock_hello_msg *hello)
 
 	if (the_lnet.ln_testprotocompat) {
 		/* single-shot proto check */
-		LNET_LOCK();
+		lnet_net_lock(LNET_LOCK_EX);
 		if (the_lnet.ln_testprotocompat & 1) {
 			hello->kshm_version++;   /* just different! */
 			the_lnet.ln_testprotocompat &= ~1;
 		}
-		LNET_UNLOCK();
+		lnet_net_unlock(LNET_LOCK_EX);
 	}
 
 	rc = lnet_sock_write(sock, hello, offsetof(struct ksock_hello_msg, kshm_ips),
diff --git a/drivers/staging/lustre/lnet/selftest/rpc.c b/drivers/staging/lustre/lnet/selftest/rpc.c
index 9613b0a77007..e097ef8414a6 100644
--- a/drivers/staging/lustre/lnet/selftest/rpc.c
+++ b/drivers/staging/lustre/lnet/selftest/rpc.c
@@ -1399,7 +1399,7 @@ srpc_send_reply(struct srpc_server_rpc *rpc)
 	return rc;
 }
 
-/* when in kernel always called with LNET_LOCK() held, and in thread context */
+/* when in kernel always called with lnet_net_lock() held, and in thread context */
 static void
 srpc_lnet_ev_handler(struct lnet_event *ev)
 {



From green at whamcloud.com  Tue Jul 24 04:00:39 2018
From: green at whamcloud.com (Oleg Drokin)
Date: Tue, 24 Jul 2018 04:00:39 +0000
Subject: [lustre-devel] New tag 2.11.53
Message-ID: <20C0EAB6-A384-435E-96D4-22840D361C42@whamcloud.com>

I just tagged a new tag 2.11.53 in Lustre master tree.

Here’s the changelog:

Alex Zhuravlev (3):
      LU-7236 ptlrpc: idle connections can disconnect
      LU-10048 osd: async truncate
      LU-11097 utils: add libuuid for llverdev

Alexander Boyko (4):
      LU-6399 lnet: socket cleanup
      LU-10945 ldlm: fix l_last_activity usage
      LU-11117 ptlrpc: don't zero request handle
      LU-10893 tests: allow to disable dm-flakey layer

Amir Shehata (1):
      LU-11064 lnd: determine gaps correctly

Andreas Dilger (5):
      LU-10264 misc: fix possible array overflow
      LU-10921 utils: improve lfs setstripe error message
      LU-10855 ptlrpc: remove obsolete OBD RPC opcodes
      LU-10855 ptlrpc: assign specific values to MGS opcodes
      LU-10855 ptlrpc: remove obsolete LLOG_ORIGIN_* RPCs

Andrew Perepechko (1):
      LU-10964 build: armv7 client build fixes

Andriy Skulysh (2):
      LU-11004 ptlrpc: Serialize procfs access to scp_hist_reqs using mutex
      LU-11098 ptlrpc: ASSERTION(!list_empty(imp->imp_replay_cursor))

Arshad Hussain (2):
      LU-7943 mdd: Move assignment after LASSERT()
      LU-10370 ofd: truncate does not update blocks count on client

Bob Glossman (3):
      LU-10897 kernel: kernel upgrade RHEL7.5 [3.10.0-862.2.3.el7]
      LU-11043 kernel: kernel update RHEL7.5 [3.10.0-862.3.2.el7]
      LU-11065 kernel: kernel update [SLES12 SP3 4.4.132-94.33]

Bruno Faccini (3):
      LU-10680 mdd: create gc thread when no current transaction
      LU-10527 obdclass: don't recycle loghandle upon ENOSPC
      LU-10734 tests: ensure current GC interval is over

Chris Horn (1):
      LU-11044 osd-ldiskfs: ext4_dir_operations uses iterate_shared

Chuck Fossen (1):
      LU-10963 gnilnd: stats variables overflow assert

Dmitry Eremin (1):
      LU-4423 ptlrpc: use delayed_work in sec_gc

Emoly Liu (6):
      LU-10907 tests: improve nodemap_test_setup()
      LU-10979 test: correct the version code check in sanity-sec.sh
      LU-10980 test: correct the version code check in sanityn.sh
      LU-10972 test: get client uuid correctly
      LU-10906 checksum: enable/disable checksum correctly
      LU-11099 doc: include "-N" option to lfs_setstripe.1

Fan Yong (6):
      LU-11024 osd-zfs: properly detect ZFS dnode accounting
      LU-10419 lfsck: signal master engine when stop
      LU-9764 lfsck: reset LFSCK trace file if fail to load it
      LU-9751 snapshot: set PATH for remote zfs commands
      LU-10120 lsnapshot: handle dash in fsname
      LU-11079 llite: control concurrent statahead instances

Giuseppe Di Natale (1):
      LU-6160 osd-zfs: Fix refcount_add call

Gregoire Pichon (1):
      LU-739 utils: remove references to lustre 1.4 upgrade flag

Hongchao Zhang (1):
      LU-7816 quota: add default quota setting support

James Nunez (5):
      LU-11010 tests: remove calls to return after skip()
      LU-8972 tests: remove conf-sanity test from ALWAYS_EXCEPT
      LU-10971 tests: use changelog routines in lustre-rsync-test
      LU-11058 tests: stop running sanity test 77k
      LU-11161 tests: stop running sanity test 160g

James Simmons (9):
      LU-9325 obdclass: handle strings correctly in lmd_find_delimiter
      LU-9325 libcfs: handle complex strings in cfs_str2num_check
      LU-10886 build: fix WARNING: modpost: missing MODULE_LICENSE()
      LU-10997 build: add files to .gitignore
      LU-8066 llite: Preparation to move /proc/fs/lustre/llite to sysfs
      LU-8066 llite: replace ll_process_config with class_modify_config
      LU-9325 llog: replace simple_strtol with kstrtol
      LU-8066 osc: fix idle_timeout handling
      LU-8066 llite: replace ll_process_config with class_modify_config

Joe Grund (1):
      LU-11026 lustre-dkms should require patch or quilt

John L. Hammond (14):
      LU-10699 hsm: add local object storage to MDTs
      LU-10855 llog: remove llog_cancel()
      LU-10978 utils: preserve lustre_rsync state
      LU-10989 utils: correct lustre_rsync changelog clear logic
      LU-10855 llog: remove obsolete llog handlers
      LU-11014 mdt: intent handling simplification
      LU-11054 lnet: remove non-error error message
      LU-11069 llite: correct file position after appending writes
      LU-11051 obd: remove obd_{get,put}ref()
      LU-11014 mdt: remove enum mdt_it_code
      LU-11045 test: use provided directory in racer/racer.sh
      LU-11052 obd: remove OBD ops based stats
      LU-11074 mdc: set correct body eadatasize for getxattr()
      LU-11157 obd: keep dirty_max_pages a round number of MB

Lai Siyao (2):
      LU-4684 mdt: improve directory stripe lock
      LU-4684 xattr: add list support for remote object

Li Dongyang (1):
      LU-10560 osd: bio_integrity_enabled was removed

Li Xi (2):
      LU-10472 osd-ldiskfs: add T10PI support for BIO
      LU-10472 osc: add T10PI support for RPC checksum

Mikhail Pershin (5):
      LU-10808 lod: align wrong DoM stripe values with defaults
      LU-10808 lod: remove DoM component if DoM is disabled
      LU-10175 ptlrpc: add LOCK_CONVERT connection flag
      LU-10175 ldlm: handle lock converts in cancel handler
      LU-11003 ldlm: don't add canceling lock back to LRU

Minh Diep (1):
      LU-11034 build: update changelog for Ubuntu 18.04

Nathaniel Clark (4):
      LU-10843 mgs: allow snapshot after MGS remount
      LU-10898 tests: enable conf-sanity 32a/32d
      LU-11019 build: Update ZFS/SPL to 0.7.9
      LU-11066 systemd: Add IB dependencies to lnet.service

NeilBrown (5):
      LU-9859 libcfs: rearrange placement of CPU partition management code.
      LU-4423 libcfs: disable preempt while sampling processor id.
      LU-8130 ldlm: store name directly in namespace.
      LU-4423 obd: backport of lu_object changes upstream
      LU-4423 ldlm: use delayed_work for ldlm_pools_recalc

Oleg Drokin (5):
      LU-10938 ptlrpc: Add WBC connect flag
      LU-11015 lov: Move lov_tgts_kobj init to lov_setup
      Revert "LU-8066 llite: replace ll_process_config with class_modify_config"
      LU-10990 osc: increase default max_dirty_mb to 2G
      New tag 2.11.53

Parinay Kondekar (1):
      LU-6511 osd-ldiskfs: Fix all irregular indentation for osd_iam.c

Patrick Farrell (1):
      LU-10648 ldlm: Reduce debug to console during eviction

Quentin Bouget (2):
      LU-9474 tests: remove unneeded hsm_set_param for raolu tests
      LU-10796 tests: standardize changelog testing in sanity-hsm

Saurabh Tandan (1):
      LU-10977 test: add version check to sanity test_60ab

Sebastien Buisson (2):
      LU-9727 tests: exercise new changelog fields and records
      LU-11049 ssk: correctly handle null byte by lgss_sk

Vladimir Saveliev (2):
      LU-11131 target: keep reply data bit set on failover
      LU-11132 compile: fix LC_BI_BDEV for old kernels

Wang Shilong (3):
      LU-11017 quota: ignore quota for CAP_SYS_RESOURCE properly
      LU-11086 test: reset quota setting properly
      LU-10986 lfs: make lfs project tolerant errors

Yang Sheng (2):
      LU-9230 ldlm: speed up preparation for list of lock cancel
      LU-11003 ldlm: fix for l_lru usage



From neilb at suse.com  Tue Jul 24 23:04:51 2018
From: neilb at suse.com (NeilBrown)
Date: Wed, 25 Jul 2018 09:04:51 +1000
Subject: [lustre-devel] [PATCH v4 00/14] lustre: libcfs: tracefile
 cleanups
In-Reply-To: <1531008868-4194-1-git-send-email-jsimmons@infradead.org>
References: <1531008868-4194-1-git-send-email-jsimmons@infradead.org>
Message-ID: <871sbsp758.fsf@notabene.neil.brown.name>

On Sat, Jul 07 2018, James Simmons wrote:

> The 4th version of a patch series aimed to cleanup the lustre tracefile
> handling. This set is mainly a repost of the 3rd set with the addition
> of a bug fix, patch number 2, that added proper range checking for the
> debugfs parameter debug_mb. The 6th patch has been updated based on
> Andreas Dilger's feedback to resolve the brokeness of cfs_print_to_console().
> This patch series has been reposted to allow anyone interested in testing
> the the major changes are in the 6th and 7th patch which was rebased due
> to changes in the code.
>
> James Simmons (6):
>   lustre: libcfs: properly handle failure paths in cfs_tracefile_init_arch()
>   lustre: libcfs: fix cfs_print_to_console()
>   lustre: libcfs: remove cfs_trace_refill_stack()
>   lustre: libcfs: move cfs_trace_data data to tracefile.c
>   lustre: libcfs: cleanup tracefile.h
>   lustre: libcfs: format macros in tracefile.h
>
> NeilBrown (8):
>   lustre: libcfs: move tracefile locking from linux-tracefile.c to tracefile.c
>   lustre: libcfs: always range-check libcfs_debug_mb setting.
>   lustre: libcfs: open code cfs_trace_max_debug_mb() into cfs_trace_set_debug_mb()
>   lustre: libcfs: move tcd locking across to tracefile.c
>   lustre: libcfs: merge linux-tracefile.c into tracefile.c
>   lustre: libcfs: fold cfs_tracefile_*_arch into their only callers.
>   lustre: libcfs: renamed CFS_TCD_TYPE_MAX to CFS_TCD_TYPE_CNT
>   lustre: libcfs: discard TCD_MAX_TYPES

Thanks.
I've applied these now, included them in the 'lustre' branch, and pushed
it all out.

Thanks,
NeilBrown
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180725/d6bca0ff/attachment.sig>

From neilb at suse.com  Tue Jul 24 23:07:16 2018
From: neilb at suse.com (NeilBrown)
Date: Wed, 25 Jul 2018 09:07:16 +1000
Subject: [lustre-devel] [PATCH 0/5] lnet: reduce code in lib-socket
Message-ID: <153247352950.25051.3479450943810758169.stgit@noble>

lib-socket contains code to iterate over network interfaces and
determine the IPv4 address of each.  It does this but using
socket-ioctls.
There are more direct interfaces within the kernel for
accessing this information, so change to use those directly.

---

NeilBrown (5):
      lustre: socklnd: use for_each_netdev() instead of lnet_ipif_enumerate()
      lustre: socklnd: use ksocknal_enumerate_interfaces for individual interfaces.
      lustre: change lnet_ipaddr_enumerate() to use for_each_netdev()
      lustre: o2iblnd: get IP address more directly.
      lustre: lnet: remove lnet_ipif_enumerate()


 .../staging/lustre/include/linux/lnet/lib-lnet.h   |    3 
 .../staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c    |   48 +++--
 .../staging/lustre/lnet/klnds/socklnd/socklnd.c    |   96 ++++-----
 drivers/staging/lustre/lnet/lnet/config.c          |   78 +++----
 drivers/staging/lustre/lnet/lnet/lib-socket.c      |  211 --------------------
 5 files changed, 103 insertions(+), 333 deletions(-)

--
Signature


From neilb at suse.com  Tue Jul 24 23:07:16 2018
From: neilb at suse.com (NeilBrown)
Date: Wed, 25 Jul 2018 09:07:16 +1000
Subject: [lustre-devel] [PATCH 1/5] lustre: socklnd: use for_each_netdev()
 instead of lnet_ipif_enumerate()
In-Reply-To: <153247352950.25051.3479450943810758169.stgit@noble>
References: <153247352950.25051.3479450943810758169.stgit@noble>
Message-ID: <153247363628.25051.7385338424046780220.stgit@noble>

for_each_netdev() is a more direct interface and doesn't require
library support.

Also get the ip address directly from the net_device, rather than
using lnet_ipif_query().

Signed-off-by: NeilBrown <neilb at suse.com>
---
 .../staging/lustre/lnet/klnds/socklnd/socklnd.c    |   63 +++++++++-----------
 1 file changed, 28 insertions(+), 35 deletions(-)

diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c
index c6e74ee01d27..895f744bb959 100644
--- a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c
+++ b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c
@@ -39,6 +39,7 @@
  */
 
 #include "socklnd.h"
+#include <linux/inetdevice.h>
 
 static struct lnet_lnd the_ksocklnd;
 struct ksock_nal_data ksocknal_data;
@@ -2614,53 +2615,45 @@ ksocknal_shutdown(struct lnet_ni *ni)
 static int
 ksocknal_enumerate_interfaces(struct ksock_net *net)
 {
-	char **names;
-	int i;
-	int j;
-	int rc;
-	int n;
-
-	n = lnet_ipif_enumerate(&names);
-	if (n <= 0) {
-		CERROR("Can't enumerate interfaces: %d\n", n);
-		return n;
-	}
+	int j = 0;
+	struct net_device *dev;
 
-	for (i = j = 0; i < n; i++) {
-		int up;
-		__u32 ip;
-		__u32 mask;
+	rtnl_lock();
+	for_each_netdev(&init_net, dev) {
+		const char *name = dev->name;
+		struct in_device *in_dev;
+		struct ksock_interface *ksi = &net->ksnn_interfaces[j];
 
-		if (!strcmp(names[i], "lo")) /* skip the loopback IF */
+		if (strcmp(name, "lo") == 0) /* skip the loopback IF */
 			continue;
 
-		rc = lnet_ipif_query(names[i], &up, &ip, &mask);
-		if (rc) {
-			CWARN("Can't get interface %s info: %d\n",
-			      names[i], rc);
-			continue;
-		}
-
-		if (!up) {
-			CWARN("Ignoring interface %s (down)\n",
-			      names[i]);
+		if (!(dev_get_flags(dev) & IFF_UP)) {
+			CWARN("Ignoring interface %s (down)\n", name);
 			continue;
 		}
 
 		if (j == LNET_MAX_INTERFACES) {
 			CWARN("Ignoring interface %s (too many interfaces)\n",
-			      names[i]);
+			      name);
 			continue;
 		}
-
-		net->ksnn_interfaces[j].ksni_ipaddr = ip;
-		net->ksnn_interfaces[j].ksni_netmask = mask;
-		strlcpy(net->ksnn_interfaces[j].ksni_name,
-			names[i], sizeof(net->ksnn_interfaces[j].ksni_name));
-		j++;
+		in_dev = __in_dev_get_rtnl(dev);
+		if (!in_dev) {
+			CWARN("Interface %s has no IPv4 status.\n", name);
+			continue;
+		}
+		for_primary_ifa(in_dev)
+			if (strcmp(ifa->ifa_label, name) == 0) {
+				ksi->ksni_ipaddr = ifa->ifa_local;
+				ksi->ksni_netmask = ifa->ifa_mask;
+				strlcpy(ksi->ksni_name,
+					name, sizeof(ksi->ksni_name));
+				j++;
+				break;
+			}
+		endfor_ifa(in_dev);
 	}
-
-	lnet_ipif_free_enumeration(names, n);
+	rtnl_unlock();
 
 	if (!j)
 		CERROR("Can't find any usable interfaces\n");



From neilb at suse.com  Tue Jul 24 23:07:16 2018
From: neilb at suse.com (NeilBrown)
Date: Wed, 25 Jul 2018 09:07:16 +1000
Subject: [lustre-devel] [PATCH 2/5] lustre: socklnd: use
 ksocknal_enumerate_interfaces for individual interfaces.
In-Reply-To: <153247352950.25051.3479450943810758169.stgit@noble>
References: <153247352950.25051.3479450943810758169.stgit@noble>
Message-ID: <153247363637.25051.1816811991601800158.stgit@noble>

When individual interfaces are listed, such a via a module parameter,
it is easy to use ksocknal_enumerate_interfaces() to find them, and
save duplicating code.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 .../staging/lustre/lnet/klnds/socklnd/socklnd.c    |   37 ++++++--------------
 1 file changed, 12 insertions(+), 25 deletions(-)

diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c
index 895f744bb959..d37f7134b31d 100644
--- a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c
+++ b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c
@@ -2613,7 +2613,7 @@ ksocknal_shutdown(struct lnet_ni *ni)
 }
 
 static int
-ksocknal_enumerate_interfaces(struct ksock_net *net)
+ksocknal_enumerate_interfaces(struct ksock_net *net, char *iname)
 {
 	int j = 0;
 	struct net_device *dev;
@@ -2622,10 +2622,13 @@ ksocknal_enumerate_interfaces(struct ksock_net *net)
 	for_each_netdev(&init_net, dev) {
 		const char *name = dev->name;
 		struct in_device *in_dev;
-		struct ksock_interface *ksi = &net->ksnn_interfaces[j];
+		struct ksock_interface *ksi =
+			&net->ksnn_interfaces[net->ksnn_ninterfaces + j];
 
 		if (strcmp(name, "lo") == 0) /* skip the loopback IF */
 			continue;
+		if (iname && strcmp(name, iname) != 0)
+			continue;
 
 		if (!(dev_get_flags(dev) & IFF_UP)) {
 			CWARN("Ignoring interface %s (down)\n", name);
@@ -2655,7 +2658,7 @@ ksocknal_enumerate_interfaces(struct ksock_net *net)
 	}
 	rtnl_unlock();
 
-	if (!j)
+	if (!iname && !j)
 		CERROR("Can't find any usable interfaces\n");
 
 	return j;
@@ -2802,40 +2805,24 @@ ksocknal_startup(struct lnet_ni *ni)
 	ni->ni_peertxcredits  = *ksocknal_tunables.ksnd_peertxcredits;
 	ni->ni_peerrtrcredits = *ksocknal_tunables.ksnd_peerrtrcredits;
 
+	net->ksnn_ninterfaces = 0;
 	if (!ni->ni_interfaces[0]) {
-		rc = ksocknal_enumerate_interfaces(net);
+		rc = ksocknal_enumerate_interfaces(net, NULL);
 		if (rc <= 0)
 			goto fail_1;
 
+		/* FIXME why is this 1, not rc ?? */
 		net->ksnn_ninterfaces = 1;
 	} else {
 		for (i = 0; i < LNET_MAX_INTERFACES; i++) {
-			int up;
-
 			if (!ni->ni_interfaces[i])
 				break;
+			rc = ksocknal_enumerate_interfaces(net, ni->ni_interfaces[i]);
 
-			rc = lnet_ipif_query(ni->ni_interfaces[i], &up,
-					     &net->ksnn_interfaces[i].ksni_ipaddr,
-					     &net->ksnn_interfaces[i].ksni_netmask);
-
-			if (rc) {
-				CERROR("Can't get interface %s info: %d\n",
-				       ni->ni_interfaces[i], rc);
-				goto fail_1;
-			}
-
-			if (!up) {
-				CERROR("Interface %s is down\n",
-				       ni->ni_interfaces[i]);
+			if (rc <= 0)
 				goto fail_1;
-			}
-
-			strlcpy(net->ksnn_interfaces[i].ksni_name,
-				ni->ni_interfaces[i],
-				sizeof(net->ksnn_interfaces[i].ksni_name));
+			net->ksnn_ninterfaces += rc;
 		}
-		net->ksnn_ninterfaces = i;
 	}
 
 	/* call it before add it to ksocknal_data.ksnd_nets */



From neilb at suse.com  Tue Jul 24 23:07:16 2018
From: neilb at suse.com (NeilBrown)
Date: Wed, 25 Jul 2018 09:07:16 +1000
Subject: [lustre-devel] [PATCH 3/5] lustre: change lnet_ipaddr_enumerate()
 to use for_each_netdev()
In-Reply-To: <153247352950.25051.3479450943810758169.stgit@noble>
References: <153247352950.25051.3479450943810758169.stgit@noble>
Message-ID: <153247363640.25051.8583622796462340468.stgit@noble>

for_each_netdev() is a more direct interface than
lnet_ipif_enumerate(), so use it instead.  Also get
address and 'up' status directly from the device.

This means we need to possible re-allocate the storage
space if there are lots of IP addresses.

However there is no need to resize the allocation down if we
over-allocated.  This is only used once, and is freed soon
after it is allocated, so that is a false optimization.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 drivers/staging/lustre/lnet/lnet/config.c |   78 +++++++++++++----------------
 1 file changed, 35 insertions(+), 43 deletions(-)

diff --git a/drivers/staging/lustre/lnet/lnet/config.c b/drivers/staging/lustre/lnet/lnet/config.c
index 55ecc1998b7e..136905db2746 100644
--- a/drivers/staging/lustre/lnet/lnet/config.c
+++ b/drivers/staging/lustre/lnet/lnet/config.c
@@ -36,6 +36,7 @@
 #include <net/net_namespace.h>
 #include <linux/ctype.h>
 #include <linux/lnet/lib-lnet.h>
+#include <linux/inetdevice.h>
 
 struct lnet_text_buf {	    /* tmp struct for parsing routes */
 	struct list_head ltb_list;	/* stash on lists */
@@ -1134,66 +1135,57 @@ lnet_match_networks(char **networksp, char *ip2nets, __u32 *ipaddrs, int nip)
 static int
 lnet_ipaddr_enumerate(__u32 **ipaddrsp)
 {
-	int up;
-	__u32 netmask;
+	struct net_device *dev;
 	__u32 *ipaddrs;
-	__u32 *ipaddrs2;
+	int nalloc = 64;
 	int nip;
-	char **ifnames;
-	int nif = lnet_ipif_enumerate(&ifnames);
-	int i;
-	int rc;
 
-	if (nif <= 0)
-		return nif;
-
-	ipaddrs = kcalloc(nif, sizeof(*ipaddrs), GFP_KERNEL);
+	ipaddrs = kcalloc(nalloc, sizeof(*ipaddrs), GFP_KERNEL);
 	if (!ipaddrs) {
-		CERROR("Can't allocate ipaddrs[%d]\n", nif);
-		lnet_ipif_free_enumeration(ifnames, nif);
+		CERROR("Can't allocate ipaddrs[%d]\n", nalloc);
 		return -ENOMEM;
 	}
 
-	for (i = nip = 0; i < nif; i++) {
-		if (!strcmp(ifnames[i], "lo"))
+	rtnl_lock();
+	for_each_netdev(&init_net, dev) {
+		struct in_device *in_dev;
+
+		if (strcmp(dev->name, "lo") == 0)
 			continue;
 
-		rc = lnet_ipif_query(ifnames[i], &up, &ipaddrs[nip], &netmask);
-		if (rc) {
-			CWARN("Can't query interface %s: %d\n",
-			      ifnames[i], rc);
+		if (!(dev_get_flags(dev) & IFF_UP)) {
+			CWARN("Ignoring interface %s: it's down\n", dev->name);
 			continue;
 		}
-
-		if (!up) {
-			CWARN("Ignoring interface %s: it's down\n",
-			      ifnames[i]);
+		in_dev = __in_dev_get_rtnl(dev);
+		if (!in_dev) {
+			CWARN("Interface %s has no IPv4 status.\n", dev->name);
 			continue;
 		}
 
-		nip++;
-	}
-
-	lnet_ipif_free_enumeration(ifnames, nif);
-
-	if (nip == nif) {
-		*ipaddrsp = ipaddrs;
-	} else {
-		if (nip > 0) {
-			ipaddrs2 = kcalloc(nip, sizeof(*ipaddrs2),
-					   GFP_KERNEL);
-			if (!ipaddrs2) {
-				CERROR("Can't allocate ipaddrs[%d]\n", nip);
-				nip = -ENOMEM;
-			} else {
-				memcpy(ipaddrs2, ipaddrs,
-				       nip * sizeof(*ipaddrs));
-				*ipaddrsp = ipaddrs2;
-				rc = nip;
+		if (nip >= nalloc) {
+			__u32 *ipaddrs2;
+			nalloc += nalloc;
+			ipaddrs2 = krealloc(ipaddrs, nalloc * sizeof(*ipaddrs2),
+					    GFP_KERNEL);
+			if (ipaddrs2 == NULL) {
+				kfree(ipaddrs);
+				CERROR("Can't allocate ipaddrs[%d]\n", nalloc);
+				return -ENOMEM;
 			}
+			ipaddrs = ipaddrs2;
 		}
-		kfree(ipaddrs);
+
+		for_primary_ifa(in_dev)
+			if (strcmp(ifa->ifa_label, dev->name) == 0) {
+				ipaddrs[nip++] = ifa->ifa_local;
+				break;
+			}
+		endfor_ifa(in_dev);
 	}
+	rtnl_unlock();
+
+	*ipaddrsp = ipaddrs;
 	return nip;
 }
 



From neilb at suse.com  Tue Jul 24 23:07:16 2018
From: neilb at suse.com (NeilBrown)
Date: Wed, 25 Jul 2018 09:07:16 +1000
Subject: [lustre-devel] [PATCH 4/5] lustre: o2iblnd: get IP address more
 directly.
In-Reply-To: <153247352950.25051.3479450943810758169.stgit@noble>
References: <153247352950.25051.3479450943810758169.stgit@noble>
Message-ID: <153247363644.25051.16677553336215434742.stgit@noble>

Use dev_get_by_name() and for_primary_ifa() to
get IP address for a named device.  This is more
direct.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 .../staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c    |   48 +++++++++++++-------
 1 file changed, 30 insertions(+), 18 deletions(-)

diff --git a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c
index 9f3fef5da38c..05835cc0f0a5 100644
--- a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c
+++ b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c
@@ -38,6 +38,7 @@
 #include <asm/div64.h>
 #include <asm/page.h>
 #include "o2iblnd.h"
+#include <linux/inetdevice.h>
 
 static struct lnet_lnd the_o2iblnd;
 
@@ -2450,40 +2451,48 @@ void kiblnd_destroy_dev(struct kib_dev *dev)
 static struct kib_dev *kiblnd_create_dev(char *ifname)
 {
 	struct net_device *netdev;
+	struct in_device *in_dev;
 	struct kib_dev *dev;
-	__u32 netmask;
-	__u32 ip;
-	int up;
+	int flags;
 	int rc;
 
-	rc = lnet_ipif_query(ifname, &up, &ip, &netmask);
-	if (rc) {
-		CERROR("Can't query IPoIB interface %s: %d\n",
-		       ifname, rc);
-		return NULL;
+	rtnl_lock();
+	netdev = dev_get_by_name(&init_net, ifname);
+	if (!netdev) {
+		CERROR("Can't find IPoIB interface %s\n",
+		       ifname);
+		goto unlock;
 	}
 
-	if (!up) {
+	flags = dev_get_flags(netdev);
+	if (!(flags & IFF_UP)) {
 		CERROR("Can't query IPoIB interface %s: it's down\n", ifname);
+		rtnl_unlock();
 		return NULL;
 	}
 
 	dev = kzalloc(sizeof(*dev), GFP_NOFS);
 	if (!dev)
-		return NULL;
+		goto unlock;
 
-	netdev = dev_get_by_name(&init_net, ifname);
-	if (!netdev) {
-		dev->ibd_can_failover = 0;
-	} else {
-		dev->ibd_can_failover = !!(netdev->flags & IFF_MASTER);
-		dev_put(netdev);
-	}
+	dev->ibd_can_failover = !!(flags & IFF_MASTER);
 
 	INIT_LIST_HEAD(&dev->ibd_nets);
 	INIT_LIST_HEAD(&dev->ibd_list); /* not yet in kib_devs */
 	INIT_LIST_HEAD(&dev->ibd_fail_list);
-	dev->ibd_ifip = ip;
+	for_primary_ifa(in_dev)
+		if (strcmp(ifa->ifa_label, ifname) == 0) {
+			dev->ibd_ifip = ifa->ifa_local;
+			break;
+		}
+	endfor_ifa(in_dev);
+	rtnl_unlock();
+
+	if (dev->ibd_ifip == 0) {
+		CERROR("Can't initialize device: no IP address\n");
+		kfree(dev);
+		return NULL;
+	}
 	strcpy(&dev->ibd_ifname[0], ifname);
 
 	/* initialize the device */
@@ -2496,6 +2505,9 @@ static struct kib_dev *kiblnd_create_dev(char *ifname)
 
 	list_add_tail(&dev->ibd_list, &kiblnd_data.kib_devs);
 	return dev;
+unlock:
+	rtnl_unlock();
+	return NULL;
 }
 
 static void kiblnd_base_shutdown(void)



From neilb at suse.com  Tue Jul 24 23:07:16 2018
From: neilb at suse.com (NeilBrown)
Date: Wed, 25 Jul 2018 09:07:16 +1000
Subject: [lustre-devel] [PATCH 5/5] lustre: lnet: remove
 lnet_ipif_enumerate()
In-Reply-To: <153247352950.25051.3479450943810758169.stgit@noble>
References: <153247352950.25051.3479450943810758169.stgit@noble>
Message-ID: <153247363648.25051.9690425452792746820.stgit@noble>

Also remove lnet_ipif_query() and related functions.

There are no longer any users of these functions, so remove them.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 .../staging/lustre/include/linux/lnet/lib-lnet.h   |    3 
 drivers/staging/lustre/lnet/lnet/lib-socket.c      |  211 --------------------
 2 files changed, 214 deletions(-)

diff --git a/drivers/staging/lustre/include/linux/lnet/lib-lnet.h b/drivers/staging/lustre/include/linux/lnet/lib-lnet.h
index 8ff8139e04fe..0fecf0d32c58 100644
--- a/drivers/staging/lustre/include/linux/lnet/lib-lnet.h
+++ b/drivers/staging/lustre/include/linux/lnet/lib-lnet.h
@@ -594,9 +594,6 @@ int lnet_acceptor_port(void);
 int lnet_acceptor_start(void);
 void lnet_acceptor_stop(void);
 
-int lnet_ipif_query(char *name, int *up, __u32 *ip, __u32 *mask);
-int lnet_ipif_enumerate(char ***names);
-void lnet_ipif_free_enumeration(char **names, int n);
 int lnet_sock_setbuf(struct socket *socket, int txbufsize, int rxbufsize);
 int lnet_sock_getbuf(struct socket *socket, int *txbufsize, int *rxbufsize);
 int lnet_sock_getaddr(struct socket *socket, bool remote, __u32 *ip, int *port);
diff --git a/drivers/staging/lustre/lnet/lnet/lib-socket.c b/drivers/staging/lustre/lnet/lnet/lib-socket.c
index 9b61260155f2..6758090d4165 100644
--- a/drivers/staging/lustre/lnet/lnet/lib-socket.c
+++ b/drivers/staging/lustre/lnet/lnet/lib-socket.c
@@ -43,217 +43,6 @@
 
 #include <linux/lnet/lib-lnet.h>
 
-static int
-kernel_sock_unlocked_ioctl(struct file *filp, int cmd, unsigned long arg)
-{
-	mm_segment_t oldfs = get_fs();
-	int err;
-
-	set_fs(KERNEL_DS);
-	err = filp->f_op->unlocked_ioctl(filp, cmd, arg);
-	set_fs(oldfs);
-
-	return err;
-}
-
-static int
-lnet_sock_ioctl(int cmd, unsigned long arg)
-{
-	struct file *sock_filp;
-	struct socket *sock;
-	int rc;
-
-	rc = sock_create(PF_INET, SOCK_STREAM, 0, &sock);
-	if (rc) {
-		CERROR("Can't create socket: %d\n", rc);
-		return rc;
-	}
-
-	sock_filp = sock_alloc_file(sock, 0, NULL);
-	if (IS_ERR(sock_filp))
-		return PTR_ERR(sock_filp);
-
-	rc = kernel_sock_unlocked_ioctl(sock_filp, cmd, arg);
-
-	fput(sock_filp);
-	return rc;
-}
-
-int
-lnet_ipif_query(char *name, int *up, __u32 *ip, __u32 *mask)
-{
-	struct ifreq ifr;
-	int nob;
-	int rc;
-	__be32 val;
-
-	nob = strnlen(name, IFNAMSIZ);
-	if (nob == IFNAMSIZ) {
-		CERROR("Interface name %s too long\n", name);
-		return -EINVAL;
-	}
-
-	BUILD_BUG_ON(sizeof(ifr.ifr_name) < IFNAMSIZ);
-
-	if (strlen(name) > sizeof(ifr.ifr_name) - 1)
-		return -E2BIG;
-	strncpy(ifr.ifr_name, name, sizeof(ifr.ifr_name));
-
-	rc = lnet_sock_ioctl(SIOCGIFFLAGS, (unsigned long)&ifr);
-	if (rc) {
-		CERROR("Can't get flags for interface %s\n", name);
-		return rc;
-	}
-
-	if (!(ifr.ifr_flags & IFF_UP)) {
-		CDEBUG(D_NET, "Interface %s down\n", name);
-		*up = 0;
-		*ip = *mask = 0;
-		return 0;
-	}
-	*up = 1;
-
-	if (strlen(name) > sizeof(ifr.ifr_name) - 1)
-		return -E2BIG;
-	strncpy(ifr.ifr_name, name, sizeof(ifr.ifr_name));
-
-	ifr.ifr_addr.sa_family = AF_INET;
-	rc = lnet_sock_ioctl(SIOCGIFADDR, (unsigned long)&ifr);
-	if (rc) {
-		CERROR("Can't get IP address for interface %s\n", name);
-		return rc;
-	}
-
-	val = ((struct sockaddr_in *)&ifr.ifr_addr)->sin_addr.s_addr;
-	*ip = ntohl(val);
-
-	if (strlen(name) > sizeof(ifr.ifr_name) - 1)
-		return -E2BIG;
-	strncpy(ifr.ifr_name, name, sizeof(ifr.ifr_name));
-
-	ifr.ifr_addr.sa_family = AF_INET;
-	rc = lnet_sock_ioctl(SIOCGIFNETMASK, (unsigned long)&ifr);
-	if (rc) {
-		CERROR("Can't get netmask for interface %s\n", name);
-		return rc;
-	}
-
-	val = ((struct sockaddr_in *)&ifr.ifr_netmask)->sin_addr.s_addr;
-	*mask = ntohl(val);
-
-	return 0;
-}
-EXPORT_SYMBOL(lnet_ipif_query);
-
-int
-lnet_ipif_enumerate(char ***namesp)
-{
-	/* Allocate and fill in 'names', returning # interfaces/error */
-	char **names;
-	int toobig;
-	int nalloc;
-	int nfound;
-	struct ifreq *ifr;
-	struct ifconf ifc;
-	int rc;
-	int nob;
-	int i;
-
-	nalloc = 16;	/* first guess at max interfaces */
-	toobig = 0;
-	for (;;) {
-		if (nalloc * sizeof(*ifr) > PAGE_SIZE) {
-			toobig = 1;
-			nalloc = PAGE_SIZE / sizeof(*ifr);
-			CWARN("Too many interfaces: only enumerating first %d\n",
-			      nalloc);
-		}
-
-		ifr = kzalloc(nalloc * sizeof(*ifr), GFP_KERNEL);
-		if (!ifr) {
-			CERROR("ENOMEM enumerating up to %d interfaces\n",
-			       nalloc);
-			rc = -ENOMEM;
-			goto out0;
-		}
-
-		ifc.ifc_buf = (char *)ifr;
-		ifc.ifc_len = nalloc * sizeof(*ifr);
-
-		rc = lnet_sock_ioctl(SIOCGIFCONF, (unsigned long)&ifc);
-		if (rc < 0) {
-			CERROR("Error %d enumerating interfaces\n", rc);
-			goto out1;
-		}
-
-		LASSERT(!rc);
-
-		nfound = ifc.ifc_len / sizeof(*ifr);
-		LASSERT(nfound <= nalloc);
-
-		if (nfound < nalloc || toobig)
-			break;
-
-		kfree(ifr);
-		nalloc *= 2;
-	}
-
-	if (!nfound)
-		goto out1;
-
-	names = kzalloc(nfound * sizeof(*names), GFP_KERNEL);
-	if (!names) {
-		rc = -ENOMEM;
-		goto out1;
-	}
-
-	for (i = 0; i < nfound; i++) {
-		nob = strnlen(ifr[i].ifr_name, IFNAMSIZ);
-		if (nob == IFNAMSIZ) {
-			/* no space for terminating NULL */
-			CERROR("interface name %.*s too long (%d max)\n",
-			       nob, ifr[i].ifr_name, IFNAMSIZ);
-			rc = -ENAMETOOLONG;
-			goto out2;
-		}
-
-		names[i] = kmalloc(IFNAMSIZ, GFP_KERNEL);
-		if (!names[i]) {
-			rc = -ENOMEM;
-			goto out2;
-		}
-
-		memcpy(names[i], ifr[i].ifr_name, nob);
-		names[i][nob] = 0;
-	}
-
-	*namesp = names;
-	rc = nfound;
-
-out2:
-	if (rc < 0)
-		lnet_ipif_free_enumeration(names, nfound);
-out1:
-	kfree(ifr);
-out0:
-	return rc;
-}
-EXPORT_SYMBOL(lnet_ipif_enumerate);
-
-void
-lnet_ipif_free_enumeration(char **names, int n)
-{
-	int i;
-
-	LASSERT(n > 0);
-
-	for (i = 0; i < n && names[i]; i++)
-		kfree(names[i]);
-
-	kfree(names);
-}
-EXPORT_SYMBOL(lnet_ipif_free_enumeration);
-
 int
 lnet_sock_write(struct socket *sock, void *buffer, int nob, int timeout)
 {



From neilb at suse.com  Wed Jul 25 02:17:27 2018
From: neilb at suse.com (NeilBrown)
Date: Wed, 25 Jul 2018 12:17:27 +1000
Subject: [lustre-devel] [PATCH] lustre: fix inode refcount problem with
 fhandle access
Message-ID: <87sh48njns.fsf@notabene.neil.brown.name>


ll_iget_for_nfs() currently calls d_obtain_alias() twice.
This decrements the refcount on the inode twice, and
increments it on the dentry twice.
In each case, only once is correct.
The result is that the refcounts are wrong and warnings
result.

Also, if the first d_obtain_alias() returns an error,
iput() is called.  As the comment at the second d_obtain_alias()
explains, this is incorrect.

Reported-by: James Simmons <jsimmons at infradead.org>
Signed-off-by: NeilBrown <neilb at suse.com>
---
 drivers/staging/lustre/lustre/llite/llite_nfs.c | 20 +++++++-------------
 1 file changed, 7 insertions(+), 13 deletions(-)

diff --git a/drivers/staging/lustre/lustre/llite/llite_nfs.c b/drivers/staging/lustre/lustre/llite/llite_nfs.c
index 14172688d55f..9efb20e91476 100644
--- a/drivers/staging/lustre/lustre/llite/llite_nfs.c
+++ b/drivers/staging/lustre/lustre/llite/llite_nfs.c
@@ -150,10 +150,8 @@ ll_iget_for_nfs(struct super_block *sb,
 	}
 
 	result = d_obtain_alias(inode);
-	if (IS_ERR(result)) {
-		iput(inode);
+	if (IS_ERR(result))
 		return result;
-	}
 
 	/**
 	 * In case d_obtain_alias() found a disconnected dentry, always update
@@ -168,16 +166,12 @@ ll_iget_for_nfs(struct super_block *sb,
 		spin_unlock(&lli->lli_lock);
 	}
 
-	/* N.B. d_obtain_alias() drops inode ref on error */
-	result = d_obtain_alias(inode);
-	if (!IS_ERR(result)) {
-		/*
-		 * Need to signal to the ll_intent_file_open that
-		 * we came from NFS and so opencache needs to be
-		 * enabled for this one
-		 */
-		ll_d2d(result)->lld_nfs_dentry = 1;
-	}
+	/*
+	 * Need to signal to the ll_intent_file_open that
+	 * we came from NFS and so opencache needs to be
+	 * enabled for this one
+	 */
+	ll_d2d(result)->lld_nfs_dentry = 1;
 
 	return result;
 }
-- 
2.14.0.rc0.dirty

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180725/0c66be35/attachment.sig>

From doucharek at cray.com  Wed Jul 25 22:17:15 2018
From: doucharek at cray.com (Doug Oucharek)
Date: Wed, 25 Jul 2018 22:17:15 +0000
Subject: [lustre-devel] [PATCH 1/5] lustre: socklnd: use
 for_each_netdev() instead of lnet_ipif_enumerate()
In-Reply-To: <153247363628.25051.7385338424046780220.stgit@noble>
References: <153247352950.25051.3479450943810758169.stgit@noble>
 <153247363628.25051.7385338424046780220.stgit@noble>
Message-ID: <39FF8C8A-A60E-428A-ADC2-4AEC6443F67C@cray.com>

Reviewed-by: Doug Oucharek <
dougso at me.com<mailto:dougso at me.com>
>

On Jul 24, 2018, at 4:07 PM, NeilBrown <neilb at suse.com<mailto:neilb at suse.com>> wrote:

for_each_netdev() is a more direct interface and doesn't require
library support.

Also get the ip address directly from the net_device, rather than
using lnet_ipif_query().

Signed-off-by: NeilBrown <neilb at suse.com<mailto:neilb at suse.com>>
---
.../staging/lustre/lnet/klnds/socklnd/socklnd.c    |   63 +++++++++-----------
1 file changed, 28 insertions(+), 35 deletions(-)

diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c
index c6e74ee01d27..895f744bb959 100644
--- a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c
+++ b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c
@@ -39,6 +39,7 @@
 */

#include "socklnd.h"
+#include <linux/inetdevice.h>

static struct lnet_lnd the_ksocklnd;
struct ksock_nal_data ksocknal_data;
@@ -2614,53 +2615,45 @@ ksocknal_shutdown(struct lnet_ni *ni)
static int
ksocknal_enumerate_interfaces(struct ksock_net *net)
{
- char **names;
- int i;
- int j;
- int rc;
- int n;
-
- n = lnet_ipif_enumerate(&names);
- if (n <= 0) {
- CERROR("Can't enumerate interfaces: %d\n", n);
- return n;
- }
+ int j = 0;
+ struct net_device *dev;

- for (i = j = 0; i < n; i++) {
- int up;
- __u32 ip;
- __u32 mask;
+ rtnl_lock();
+ for_each_netdev(&init_net, dev) {
+ const char *name = dev->name;
+ struct in_device *in_dev;
+ struct ksock_interface *ksi = &net->ksnn_interfaces[j];

- if (!strcmp(names[i], "lo")) /* skip the loopback IF */
+ if (strcmp(name, "lo") == 0) /* skip the loopback IF */
continue;

- rc = lnet_ipif_query(names[i], &up, &ip, &mask);
- if (rc) {
- CWARN("Can't get interface %s info: %d\n",
-      names[i], rc);
- continue;
- }
-
- if (!up) {
- CWARN("Ignoring interface %s (down)\n",
-      names[i]);
+ if (!(dev_get_flags(dev) & IFF_UP)) {
+ CWARN("Ignoring interface %s (down)\n", name);
continue;
}

if (j == LNET_MAX_INTERFACES) {
CWARN("Ignoring interface %s (too many interfaces)\n",
-      names[i]);
+      name);
continue;
}
-
- net->ksnn_interfaces[j].ksni_ipaddr = ip;
- net->ksnn_interfaces[j].ksni_netmask = mask;
- strlcpy(net->ksnn_interfaces[j].ksni_name,
- names[i], sizeof(net->ksnn_interfaces[j].ksni_name));
- j++;
+ in_dev = __in_dev_get_rtnl(dev);
+ if (!in_dev) {
+ CWARN("Interface %s has no IPv4 status.\n", name);
+ continue;
+ }
+ for_primary_ifa(in_dev)
+ if (strcmp(ifa->ifa_label, name) == 0) {
+ ksi->ksni_ipaddr = ifa->ifa_local;
+ ksi->ksni_netmask = ifa->ifa_mask;
+ strlcpy(ksi->ksni_name,
+ name, sizeof(ksi->ksni_name));
+ j++;
+ break;
+ }
+ endfor_ifa(in_dev);
}
-
- lnet_ipif_free_enumeration(names, n);
+ rtnl_unlock();

if (!j)
CERROR("Can't find any usable interfaces\n");


_______________________________________________
lustre-devel mailing list
lustre-devel at lists.lustre.org<mailto:lustre-devel at lists.lustre.org>
http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180725/59bb2f7c/attachment.html>

From doucharek at cray.com  Wed Jul 25 22:28:52 2018
From: doucharek at cray.com (Doug Oucharek)
Date: Wed, 25 Jul 2018 22:28:52 +0000
Subject: [lustre-devel] [PATCH 2/5] lustre: socklnd: use
 ksocknal_enumerate_interfaces for individual interfaces.
In-Reply-To: <153247363637.25051.1816811991601800158.stgit@noble>
References: <153247352950.25051.3479450943810758169.stgit@noble>
 <153247363637.25051.1816811991601800158.stgit@noble>
Message-ID: <36D26881-EC61-4CFF-B655-EB8EA1FCD609@cray.com>

Reviewed-by: Doug Oucharek <dougso at me.com<mailto:dougso at me.com>>

With regards to the FIXME comment:
That does seem to be a bug.  There is an assumption that we will only get one address back from the enumerate call.  That has probably been true in the field thus far.  Should be safe to change it to rc as the comment indicates.

Doug

On Jul 24, 2018, at 4:07 PM, NeilBrown <neilb at suse.com<mailto:neilb at suse.com>> wrote:

When individual interfaces are listed, such a via a module parameter,
it is easy to use ksocknal_enumerate_interfaces() to find them, and
save duplicating code.

Signed-off-by: NeilBrown <neilb at suse.com<mailto:neilb at suse.com>>
---
.../staging/lustre/lnet/klnds/socklnd/socklnd.c    |   37 ++++++--------------
1 file changed, 12 insertions(+), 25 deletions(-)

diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c
index 895f744bb959..d37f7134b31d 100644
--- a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c
+++ b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c
@@ -2613,7 +2613,7 @@ ksocknal_shutdown(struct lnet_ni *ni)
}

static int
-ksocknal_enumerate_interfaces(struct ksock_net *net)
+ksocknal_enumerate_interfaces(struct ksock_net *net, char *iname)
{
int j = 0;
struct net_device *dev;
@@ -2622,10 +2622,13 @@ ksocknal_enumerate_interfaces(struct ksock_net *net)
for_each_netdev(&init_net, dev) {
const char *name = dev->name;
struct in_device *in_dev;
- struct ksock_interface *ksi = &net->ksnn_interfaces[j];
+ struct ksock_interface *ksi =
+ &net->ksnn_interfaces[net->ksnn_ninterfaces + j];

if (strcmp(name, "lo") == 0) /* skip the loopback IF */
continue;
+ if (iname && strcmp(name, iname) != 0)
+ continue;

if (!(dev_get_flags(dev) & IFF_UP)) {
CWARN("Ignoring interface %s (down)\n", name);
@@ -2655,7 +2658,7 @@ ksocknal_enumerate_interfaces(struct ksock_net *net)
}
rtnl_unlock();

- if (!j)
+ if (!iname && !j)
CERROR("Can't find any usable interfaces\n");

return j;
@@ -2802,40 +2805,24 @@ ksocknal_startup(struct lnet_ni *ni)
ni->ni_peertxcredits  = *ksocknal_tunables.ksnd_peertxcredits;
ni->ni_peerrtrcredits = *ksocknal_tunables.ksnd_peerrtrcredits;

+ net->ksnn_ninterfaces = 0;
if (!ni->ni_interfaces[0]) {
- rc = ksocknal_enumerate_interfaces(net);
+ rc = ksocknal_enumerate_interfaces(net, NULL);
if (rc <= 0)
goto fail_1;

+ /* FIXME why is this 1, not rc ?? */
net->ksnn_ninterfaces = 1;
} else {
for (i = 0; i < LNET_MAX_INTERFACES; i++) {
- int up;
-
if (!ni->ni_interfaces[i])
break;
+ rc = ksocknal_enumerate_interfaces(net, ni->ni_interfaces[i]);

- rc = lnet_ipif_query(ni->ni_interfaces[i], &up,
-     &net->ksnn_interfaces[i].ksni_ipaddr,
-     &net->ksnn_interfaces[i].ksni_netmask);
-
- if (rc) {
- CERROR("Can't get interface %s info: %d\n",
-       ni->ni_interfaces[i], rc);
- goto fail_1;
- }
-
- if (!up) {
- CERROR("Interface %s is down\n",
-       ni->ni_interfaces[i]);
+ if (rc <= 0)
goto fail_1;
- }
-
- strlcpy(net->ksnn_interfaces[i].ksni_name,
- ni->ni_interfaces[i],
- sizeof(net->ksnn_interfaces[i].ksni_name));
+ net->ksnn_ninterfaces += rc;
}
- net->ksnn_ninterfaces = i;
}

/* call it before add it to ksocknal_data.ksnd_nets */


_______________________________________________
lustre-devel mailing list
lustre-devel at lists.lustre.org<mailto:lustre-devel at lists.lustre.org>
http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180725/65b89749/attachment.html>

From doucharek at cray.com  Wed Jul 25 22:39:12 2018
From: doucharek at cray.com (Doug Oucharek)
Date: Wed, 25 Jul 2018 22:39:12 +0000
Subject: [lustre-devel] [PATCH 3/5] lustre: change
 lnet_ipaddr_enumerate() to use for_each_netdev()
In-Reply-To: <153247363640.25051.8583622796462340468.stgit@noble>
References: <153247352950.25051.3479450943810758169.stgit@noble>
 <153247363640.25051.8583622796462340468.stgit@noble>
Message-ID: <2D7F51A3-AE1F-4BBE-BF52-C3EFA99DF5B4@cray.com>

Reviewed-by: Doug Oucharek <dougso at me.com<mailto:dougso at me.com>>

Note:
I did not think your changes would collide with Multi-Rail.  Unfortunately, it looks like this one does.  This config code was changed quite a bit by the Multi-Rail feature.  However, it is ok as landing this change now will prompt the Multi-Rail changes to utilize the better way of getting interface information.  Just a heads up to Multi-Rail porters.

Doug

On Jul 24, 2018, at 4:07 PM, NeilBrown <neilb at suse.com<mailto:neilb at suse.com>> wrote:

for_each_netdev() is a more direct interface than
lnet_ipif_enumerate(), so use it instead.  Also get
address and 'up' status directly from the device.

This means we need to possible re-allocate the storage
space if there are lots of IP addresses.

However there is no need to resize the allocation down if we
over-allocated.  This is only used once, and is freed soon
after it is allocated, so that is a false optimization.

Signed-off-by: NeilBrown <neilb at suse.com<mailto:neilb at suse.com>>
---
drivers/staging/lustre/lnet/lnet/config.c |   78 +++++++++++++----------------
1 file changed, 35 insertions(+), 43 deletions(-)

diff --git a/drivers/staging/lustre/lnet/lnet/config.c b/drivers/staging/lustre/lnet/lnet/config.c
index 55ecc1998b7e..136905db2746 100644
--- a/drivers/staging/lustre/lnet/lnet/config.c
+++ b/drivers/staging/lustre/lnet/lnet/config.c
@@ -36,6 +36,7 @@
#include <net/net_namespace.h>
#include <linux/ctype.h>
#include <linux/lnet/lib-lnet.h>
+#include <linux/inetdevice.h>

struct lnet_text_buf {    /* tmp struct for parsing routes */
struct list_head ltb_list; /* stash on lists */
@@ -1134,66 +1135,57 @@ lnet_match_networks(char **networksp, char *ip2nets, __u32 *ipaddrs, int nip)
static int
lnet_ipaddr_enumerate(__u32 **ipaddrsp)
{
- int up;
- __u32 netmask;
+ struct net_device *dev;
__u32 *ipaddrs;
- __u32 *ipaddrs2;
+ int nalloc = 64;
int nip;
- char **ifnames;
- int nif = lnet_ipif_enumerate(&ifnames);
- int i;
- int rc;

- if (nif <= 0)
- return nif;
-
- ipaddrs = kcalloc(nif, sizeof(*ipaddrs), GFP_KERNEL);
+ ipaddrs = kcalloc(nalloc, sizeof(*ipaddrs), GFP_KERNEL);
if (!ipaddrs) {
- CERROR("Can't allocate ipaddrs[%d]\n", nif);
- lnet_ipif_free_enumeration(ifnames, nif);
+ CERROR("Can't allocate ipaddrs[%d]\n", nalloc);
return -ENOMEM;
}

- for (i = nip = 0; i < nif; i++) {
- if (!strcmp(ifnames[i], "lo"))
+ rtnl_lock();
+ for_each_netdev(&init_net, dev) {
+ struct in_device *in_dev;
+
+ if (strcmp(dev->name, "lo") == 0)
continue;

- rc = lnet_ipif_query(ifnames[i], &up, &ipaddrs[nip], &netmask);
- if (rc) {
- CWARN("Can't query interface %s: %d\n",
-      ifnames[i], rc);
+ if (!(dev_get_flags(dev) & IFF_UP)) {
+ CWARN("Ignoring interface %s: it's down\n", dev->name);
continue;
}
-
- if (!up) {
- CWARN("Ignoring interface %s: it's down\n",
-      ifnames[i]);
+ in_dev = __in_dev_get_rtnl(dev);
+ if (!in_dev) {
+ CWARN("Interface %s has no IPv4 status.\n", dev->name);
continue;
}

- nip++;
- }
-
- lnet_ipif_free_enumeration(ifnames, nif);
-
- if (nip == nif) {
- *ipaddrsp = ipaddrs;
- } else {
- if (nip > 0) {
- ipaddrs2 = kcalloc(nip, sizeof(*ipaddrs2),
-   GFP_KERNEL);
- if (!ipaddrs2) {
- CERROR("Can't allocate ipaddrs[%d]\n", nip);
- nip = -ENOMEM;
- } else {
- memcpy(ipaddrs2, ipaddrs,
-       nip * sizeof(*ipaddrs));
- *ipaddrsp = ipaddrs2;
- rc = nip;
+ if (nip >= nalloc) {
+ __u32 *ipaddrs2;
+ nalloc += nalloc;
+ ipaddrs2 = krealloc(ipaddrs, nalloc * sizeof(*ipaddrs2),
+    GFP_KERNEL);
+ if (ipaddrs2 == NULL) {
+ kfree(ipaddrs);
+ CERROR("Can't allocate ipaddrs[%d]\n", nalloc);
+ return -ENOMEM;
}
+ ipaddrs = ipaddrs2;
}
- kfree(ipaddrs);
+
+ for_primary_ifa(in_dev)
+ if (strcmp(ifa->ifa_label, dev->name) == 0) {
+ ipaddrs[nip++] = ifa->ifa_local;
+ break;
+ }
+ endfor_ifa(in_dev);
}
+ rtnl_unlock();
+
+ *ipaddrsp = ipaddrs;
return nip;
}



_______________________________________________
lustre-devel mailing list
lustre-devel at lists.lustre.org<mailto:lustre-devel at lists.lustre.org>
http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180725/e7ceb368/attachment.html>

From doucharek at cray.com  Wed Jul 25 22:43:00 2018
From: doucharek at cray.com (Doug Oucharek)
Date: Wed, 25 Jul 2018 22:43:00 +0000
Subject: [lustre-devel] [PATCH 4/5] lustre: o2iblnd: get IP address more
 directly.
In-Reply-To: <153247363644.25051.16677553336215434742.stgit@noble>
References: <153247352950.25051.3479450943810758169.stgit@noble>
 <153247363644.25051.16677553336215434742.stgit@noble>
Message-ID: <9016B622-1E72-4A5A-9CC1-8D460AE7BA1E@cray.com>

Reviewed-by: Doug Oucharek <dougso at me.com<mailto:dougso at me.com>>

Doug

On Jul 24, 2018, at 4:07 PM, NeilBrown <neilb at suse.com<mailto:neilb at suse.com>> wrote:

Use dev_get_by_name() and for_primary_ifa() to
get IP address for a named device.  This is more
direct.

Signed-off-by: NeilBrown <neilb at suse.com<mailto:neilb at suse.com>>
---
.../staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c    |   48 +++++++++++++-------
1 file changed, 30 insertions(+), 18 deletions(-)

diff --git a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c
index 9f3fef5da38c..05835cc0f0a5 100644
--- a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c
+++ b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c
@@ -38,6 +38,7 @@
#include <asm/div64.h>
#include <asm/page.h>
#include "o2iblnd.h"
+#include <linux/inetdevice.h>

static struct lnet_lnd the_o2iblnd;

@@ -2450,40 +2451,48 @@ void kiblnd_destroy_dev(struct kib_dev *dev)
static struct kib_dev *kiblnd_create_dev(char *ifname)
{
struct net_device *netdev;
+ struct in_device *in_dev;
struct kib_dev *dev;
- __u32 netmask;
- __u32 ip;
- int up;
+ int flags;
int rc;

- rc = lnet_ipif_query(ifname, &up, &ip, &netmask);
- if (rc) {
- CERROR("Can't query IPoIB interface %s: %d\n",
-       ifname, rc);
- return NULL;
+ rtnl_lock();
+ netdev = dev_get_by_name(&init_net, ifname);
+ if (!netdev) {
+ CERROR("Can't find IPoIB interface %s\n",
+       ifname);
+ goto unlock;
}

- if (!up) {
+ flags = dev_get_flags(netdev);
+ if (!(flags & IFF_UP)) {
CERROR("Can't query IPoIB interface %s: it's down\n", ifname);
+ rtnl_unlock();
return NULL;
}

dev = kzalloc(sizeof(*dev), GFP_NOFS);
if (!dev)
- return NULL;
+ goto unlock;

- netdev = dev_get_by_name(&init_net, ifname);
- if (!netdev) {
- dev->ibd_can_failover = 0;
- } else {
- dev->ibd_can_failover = !!(netdev->flags & IFF_MASTER);
- dev_put(netdev);
- }
+ dev->ibd_can_failover = !!(flags & IFF_MASTER);

INIT_LIST_HEAD(&dev->ibd_nets);
INIT_LIST_HEAD(&dev->ibd_list); /* not yet in kib_devs */
INIT_LIST_HEAD(&dev->ibd_fail_list);
- dev->ibd_ifip = ip;
+ for_primary_ifa(in_dev)
+ if (strcmp(ifa->ifa_label, ifname) == 0) {
+ dev->ibd_ifip = ifa->ifa_local;
+ break;
+ }
+ endfor_ifa(in_dev);
+ rtnl_unlock();
+
+ if (dev->ibd_ifip == 0) {
+ CERROR("Can't initialize device: no IP address\n");
+ kfree(dev);
+ return NULL;
+ }
strcpy(&dev->ibd_ifname[0], ifname);

/* initialize the device */
@@ -2496,6 +2505,9 @@ static struct kib_dev *kiblnd_create_dev(char *ifname)

list_add_tail(&dev->ibd_list, &kiblnd_data.kib_devs);
return dev;
+unlock:
+ rtnl_unlock();
+ return NULL;
}

static void kiblnd_base_shutdown(void)


_______________________________________________
lustre-devel mailing list
lustre-devel at lists.lustre.org<mailto:lustre-devel at lists.lustre.org>
http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180725/618c236d/attachment.html>

From doucharek at cray.com  Wed Jul 25 22:44:27 2018
From: doucharek at cray.com (Doug Oucharek)
Date: Wed, 25 Jul 2018 22:44:27 +0000
Subject: [lustre-devel] [PATCH 5/5] lustre: lnet: remove
 lnet_ipif_enumerate()
In-Reply-To: <153247363648.25051.9690425452792746820.stgit@noble>
References: <153247352950.25051.3479450943810758169.stgit@noble>
 <153247363648.25051.9690425452792746820.stgit@noble>
Message-ID: <6FA73135-C969-4577-B2D0-2F6DE9ABF7F7@cray.com>

Reviewed-by: Doug Oucharek <dougso at me.com<mailto:dougso at me.com>>

Doug

On Jul 24, 2018, at 4:07 PM, NeilBrown <neilb at suse.com<mailto:neilb at suse.com>> wrote:

Also remove lnet_ipif_query() and related functions.

There are no longer any users of these functions, so remove them.

Signed-off-by: NeilBrown <neilb at suse.com<mailto:neilb at suse.com>>
---
.../staging/lustre/include/linux/lnet/lib-lnet.h   |    3
drivers/staging/lustre/lnet/lnet/lib-socket.c      |  211 --------------------
2 files changed, 214 deletions(-)

diff --git a/drivers/staging/lustre/include/linux/lnet/lib-lnet.h b/drivers/staging/lustre/include/linux/lnet/lib-lnet.h
index 8ff8139e04fe..0fecf0d32c58 100644
--- a/drivers/staging/lustre/include/linux/lnet/lib-lnet.h
+++ b/drivers/staging/lustre/include/linux/lnet/lib-lnet.h
@@ -594,9 +594,6 @@ int lnet_acceptor_port(void);
int lnet_acceptor_start(void);
void lnet_acceptor_stop(void);

-int lnet_ipif_query(char *name, int *up, __u32 *ip, __u32 *mask);
-int lnet_ipif_enumerate(char ***names);
-void lnet_ipif_free_enumeration(char **names, int n);
int lnet_sock_setbuf(struct socket *socket, int txbufsize, int rxbufsize);
int lnet_sock_getbuf(struct socket *socket, int *txbufsize, int *rxbufsize);
int lnet_sock_getaddr(struct socket *socket, bool remote, __u32 *ip, int *port);
diff --git a/drivers/staging/lustre/lnet/lnet/lib-socket.c b/drivers/staging/lustre/lnet/lnet/lib-socket.c
index 9b61260155f2..6758090d4165 100644
--- a/drivers/staging/lustre/lnet/lnet/lib-socket.c
+++ b/drivers/staging/lustre/lnet/lnet/lib-socket.c
@@ -43,217 +43,6 @@

#include <linux/lnet/lib-lnet.h>

-static int
-kernel_sock_unlocked_ioctl(struct file *filp, int cmd, unsigned long arg)
-{
- mm_segment_t oldfs = get_fs();
- int err;
-
- set_fs(KERNEL_DS);
- err = filp->f_op->unlocked_ioctl(filp, cmd, arg);
- set_fs(oldfs);
-
- return err;
-}
-
-static int
-lnet_sock_ioctl(int cmd, unsigned long arg)
-{
- struct file *sock_filp;
- struct socket *sock;
- int rc;
-
- rc = sock_create(PF_INET, SOCK_STREAM, 0, &sock);
- if (rc) {
- CERROR("Can't create socket: %d\n", rc);
- return rc;
- }
-
- sock_filp = sock_alloc_file(sock, 0, NULL);
- if (IS_ERR(sock_filp))
- return PTR_ERR(sock_filp);
-
- rc = kernel_sock_unlocked_ioctl(sock_filp, cmd, arg);
-
- fput(sock_filp);
- return rc;
-}
-
-int
-lnet_ipif_query(char *name, int *up, __u32 *ip, __u32 *mask)
-{
- struct ifreq ifr;
- int nob;
- int rc;
- __be32 val;
-
- nob = strnlen(name, IFNAMSIZ);
- if (nob == IFNAMSIZ) {
- CERROR("Interface name %s too long\n", name);
- return -EINVAL;
- }
-
- BUILD_BUG_ON(sizeof(ifr.ifr_name) < IFNAMSIZ);
-
- if (strlen(name) > sizeof(ifr.ifr_name) - 1)
- return -E2BIG;
- strncpy(ifr.ifr_name, name, sizeof(ifr.ifr_name));
-
- rc = lnet_sock_ioctl(SIOCGIFFLAGS, (unsigned long)&ifr);
- if (rc) {
- CERROR("Can't get flags for interface %s\n", name);
- return rc;
- }
-
- if (!(ifr.ifr_flags & IFF_UP)) {
- CDEBUG(D_NET, "Interface %s down\n", name);
- *up = 0;
- *ip = *mask = 0;
- return 0;
- }
- *up = 1;
-
- if (strlen(name) > sizeof(ifr.ifr_name) - 1)
- return -E2BIG;
- strncpy(ifr.ifr_name, name, sizeof(ifr.ifr_name));
-
- ifr.ifr_addr.sa_family = AF_INET;
- rc = lnet_sock_ioctl(SIOCGIFADDR, (unsigned long)&ifr);
- if (rc) {
- CERROR("Can't get IP address for interface %s\n", name);
- return rc;
- }
-
- val = ((struct sockaddr_in *)&ifr.ifr_addr)->sin_addr.s_addr;
- *ip = ntohl(val);
-
- if (strlen(name) > sizeof(ifr.ifr_name) - 1)
- return -E2BIG;
- strncpy(ifr.ifr_name, name, sizeof(ifr.ifr_name));
-
- ifr.ifr_addr.sa_family = AF_INET;
- rc = lnet_sock_ioctl(SIOCGIFNETMASK, (unsigned long)&ifr);
- if (rc) {
- CERROR("Can't get netmask for interface %s\n", name);
- return rc;
- }
-
- val = ((struct sockaddr_in *)&ifr.ifr_netmask)->sin_addr.s_addr;
- *mask = ntohl(val);
-
- return 0;
-}
-EXPORT_SYMBOL(lnet_ipif_query);
-
-int
-lnet_ipif_enumerate(char ***namesp)
-{
- /* Allocate and fill in 'names', returning # interfaces/error */
- char **names;
- int toobig;
- int nalloc;
- int nfound;
- struct ifreq *ifr;
- struct ifconf ifc;
- int rc;
- int nob;
- int i;
-
- nalloc = 16; /* first guess at max interfaces */
- toobig = 0;
- for (;;) {
- if (nalloc * sizeof(*ifr) > PAGE_SIZE) {
- toobig = 1;
- nalloc = PAGE_SIZE / sizeof(*ifr);
- CWARN("Too many interfaces: only enumerating first %d\n",
-      nalloc);
- }
-
- ifr = kzalloc(nalloc * sizeof(*ifr), GFP_KERNEL);
- if (!ifr) {
- CERROR("ENOMEM enumerating up to %d interfaces\n",
-       nalloc);
- rc = -ENOMEM;
- goto out0;
- }
-
- ifc.ifc_buf = (char *)ifr;
- ifc.ifc_len = nalloc * sizeof(*ifr);
-
- rc = lnet_sock_ioctl(SIOCGIFCONF, (unsigned long)&ifc);
- if (rc < 0) {
- CERROR("Error %d enumerating interfaces\n", rc);
- goto out1;
- }
-
- LASSERT(!rc);
-
- nfound = ifc.ifc_len / sizeof(*ifr);
- LASSERT(nfound <= nalloc);
-
- if (nfound < nalloc || toobig)
- break;
-
- kfree(ifr);
- nalloc *= 2;
- }
-
- if (!nfound)
- goto out1;
-
- names = kzalloc(nfound * sizeof(*names), GFP_KERNEL);
- if (!names) {
- rc = -ENOMEM;
- goto out1;
- }
-
- for (i = 0; i < nfound; i++) {
- nob = strnlen(ifr[i].ifr_name, IFNAMSIZ);
- if (nob == IFNAMSIZ) {
- /* no space for terminating NULL */
- CERROR("interface name %.*s too long (%d max)\n",
-       nob, ifr[i].ifr_name, IFNAMSIZ);
- rc = -ENAMETOOLONG;
- goto out2;
- }
-
- names[i] = kmalloc(IFNAMSIZ, GFP_KERNEL);
- if (!names[i]) {
- rc = -ENOMEM;
- goto out2;
- }
-
- memcpy(names[i], ifr[i].ifr_name, nob);
- names[i][nob] = 0;
- }
-
- *namesp = names;
- rc = nfound;
-
-out2:
- if (rc < 0)
- lnet_ipif_free_enumeration(names, nfound);
-out1:
- kfree(ifr);
-out0:
- return rc;
-}
-EXPORT_SYMBOL(lnet_ipif_enumerate);
-
-void
-lnet_ipif_free_enumeration(char **names, int n)
-{
- int i;
-
- LASSERT(n > 0);
-
- for (i = 0; i < n && names[i]; i++)
- kfree(names[i]);
-
- kfree(names);
-}
-EXPORT_SYMBOL(lnet_ipif_free_enumeration);
-
int
lnet_sock_write(struct socket *sock, void *buffer, int nob, int timeout)
{


_______________________________________________
lustre-devel mailing list
lustre-devel at lists.lustre.org<mailto:lustre-devel at lists.lustre.org>
http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180725/d8be92f2/attachment.html>

From neilb at suse.com  Thu Jul 26 04:00:23 2018
From: neilb at suse.com (NeilBrown)
Date: Thu, 26 Jul 2018 14:00:23 +1000
Subject: [lustre-devel] [PATCH 3/5] lustre: change
 lnet_ipaddr_enumerate() to use for_each_netdev()
In-Reply-To: <2D7F51A3-AE1F-4BBE-BF52-C3EFA99DF5B4@cray.com>
References: <153247352950.25051.3479450943810758169.stgit@noble>
 <153247363640.25051.8583622796462340468.stgit@noble>
 <2D7F51A3-AE1F-4BBE-BF52-C3EFA99DF5B4@cray.com>
Message-ID: <87fu06odd4.fsf@notabene.neil.brown.name>


Hi Doug,
 thanks for your review to this patch and the others.
 Thanks particularly for noticing the FIXME :-)  I'll make that change.

Thanks,
NeilBrown

On Wed, Jul 25 2018, Doug Oucharek wrote:

> Reviewed-by: Doug Oucharek <dougso at me.com<mailto:dougso at me.com>>
>
> Note:
> I did not think your changes would collide with Multi-Rail.  Unfortunately, it looks like this one does.  This config code was changed quite a bit by the Multi-Rail feature.  However, it is ok as landing this change now will prompt the Multi-Rail changes to utilize the better way of getting interface information.  Just a heads up to Multi-Rail porters.
>
> Doug
>
> On Jul 24, 2018, at 4:07 PM, NeilBrown <neilb at suse.com<mailto:neilb at suse.com>> wrote:
>
> for_each_netdev() is a more direct interface than
> lnet_ipif_enumerate(), so use it instead.  Also get
> address and 'up' status directly from the device.
>
> This means we need to possible re-allocate the storage
> space if there are lots of IP addresses.
>
> However there is no need to resize the allocation down if we
> over-allocated.  This is only used once, and is freed soon
> after it is allocated, so that is a false optimization.
>
> Signed-off-by: NeilBrown <neilb at suse.com<mailto:neilb at suse.com>>
> ---
> drivers/staging/lustre/lnet/lnet/config.c |   78 +++++++++++++----------------
> 1 file changed, 35 insertions(+), 43 deletions(-)
>
> diff --git a/drivers/staging/lustre/lnet/lnet/config.c b/drivers/staging/lustre/lnet/lnet/config.c
> index 55ecc1998b7e..136905db2746 100644
> --- a/drivers/staging/lustre/lnet/lnet/config.c
> +++ b/drivers/staging/lustre/lnet/lnet/config.c
> @@ -36,6 +36,7 @@
> #include <net/net_namespace.h>
> #include <linux/ctype.h>
> #include <linux/lnet/lib-lnet.h>
> +#include <linux/inetdevice.h>
>
> struct lnet_text_buf {    /* tmp struct for parsing routes */
> struct list_head ltb_list; /* stash on lists */
> @@ -1134,66 +1135,57 @@ lnet_match_networks(char **networksp, char *ip2nets, __u32 *ipaddrs, int nip)
> static int
> lnet_ipaddr_enumerate(__u32 **ipaddrsp)
> {
> - int up;
> - __u32 netmask;
> + struct net_device *dev;
> __u32 *ipaddrs;
> - __u32 *ipaddrs2;
> + int nalloc = 64;
> int nip;
> - char **ifnames;
> - int nif = lnet_ipif_enumerate(&ifnames);
> - int i;
> - int rc;
>
> - if (nif <= 0)
> - return nif;
> -
> - ipaddrs = kcalloc(nif, sizeof(*ipaddrs), GFP_KERNEL);
> + ipaddrs = kcalloc(nalloc, sizeof(*ipaddrs), GFP_KERNEL);
> if (!ipaddrs) {
> - CERROR("Can't allocate ipaddrs[%d]\n", nif);
> - lnet_ipif_free_enumeration(ifnames, nif);
> + CERROR("Can't allocate ipaddrs[%d]\n", nalloc);
> return -ENOMEM;
> }
>
> - for (i = nip = 0; i < nif; i++) {
> - if (!strcmp(ifnames[i], "lo"))
> + rtnl_lock();
> + for_each_netdev(&init_net, dev) {
> + struct in_device *in_dev;
> +
> + if (strcmp(dev->name, "lo") == 0)
> continue;
>
> - rc = lnet_ipif_query(ifnames[i], &up, &ipaddrs[nip], &netmask);
> - if (rc) {
> - CWARN("Can't query interface %s: %d\n",
> -      ifnames[i], rc);
> + if (!(dev_get_flags(dev) & IFF_UP)) {
> + CWARN("Ignoring interface %s: it's down\n", dev->name);
> continue;
> }
> -
> - if (!up) {
> - CWARN("Ignoring interface %s: it's down\n",
> -      ifnames[i]);
> + in_dev = __in_dev_get_rtnl(dev);
> + if (!in_dev) {
> + CWARN("Interface %s has no IPv4 status.\n", dev->name);
> continue;
> }
>
> - nip++;
> - }
> -
> - lnet_ipif_free_enumeration(ifnames, nif);
> -
> - if (nip == nif) {
> - *ipaddrsp = ipaddrs;
> - } else {
> - if (nip > 0) {
> - ipaddrs2 = kcalloc(nip, sizeof(*ipaddrs2),
> -   GFP_KERNEL);
> - if (!ipaddrs2) {
> - CERROR("Can't allocate ipaddrs[%d]\n", nip);
> - nip = -ENOMEM;
> - } else {
> - memcpy(ipaddrs2, ipaddrs,
> -       nip * sizeof(*ipaddrs));
> - *ipaddrsp = ipaddrs2;
> - rc = nip;
> + if (nip >= nalloc) {
> + __u32 *ipaddrs2;
> + nalloc += nalloc;
> + ipaddrs2 = krealloc(ipaddrs, nalloc * sizeof(*ipaddrs2),
> +    GFP_KERNEL);
> + if (ipaddrs2 == NULL) {
> + kfree(ipaddrs);
> + CERROR("Can't allocate ipaddrs[%d]\n", nalloc);
> + return -ENOMEM;
> }
> + ipaddrs = ipaddrs2;
> }
> - kfree(ipaddrs);
> +
> + for_primary_ifa(in_dev)
> + if (strcmp(ifa->ifa_label, dev->name) == 0) {
> + ipaddrs[nip++] = ifa->ifa_local;
> + break;
> + }
> + endfor_ifa(in_dev);
> }
> + rtnl_unlock();
> +
> + *ipaddrsp = ipaddrs;
> return nip;
> }
>
>
>
> _______________________________________________
> lustre-devel mailing list
> lustre-devel at lists.lustre.org<mailto:lustre-devel at lists.lustre.org>
> http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180726/b4ffb366/attachment.sig>

From greg at kroah.com  Thu Jul 26 14:12:27 2018
From: greg at kroah.com (Greg KH)
Date: Thu, 26 Jul 2018 16:12:27 +0200
Subject: [lustre-devel] [BUG] staging: lustre: Possible null function
 pointer in ctx_refresh_timeout()
In-Reply-To: <6920e093-ad62-56dc-6f58-36cf8d7ed3b4@gmail.com>
References: <6920e093-ad62-56dc-6f58-36cf8d7ed3b4@gmail.com>
Message-ID: <20180726141227.GA12445@kroah.com>

On Thu, Jul 26, 2018 at 10:02:22PM +0800, Jia-Ju Bai wrote:
> In Linux-4.16, drivers/staging/lustre/lustre/ptlrp/sec.c,

Please look at the 4.18-rc6 release for this file.

In short, nothing to worry about anymore :)

thanks,

greg k-h

From baijiaju1990 at gmail.com  Thu Jul 26 14:02:22 2018
From: baijiaju1990 at gmail.com (Jia-Ju Bai)
Date: Thu, 26 Jul 2018 22:02:22 +0800
Subject: [lustre-devel] [BUG] staging: lustre: Possible null function
 pointer in ctx_refresh_timeout()
Message-ID: <6920e093-ad62-56dc-6f58-36cf8d7ed3b4@gmail.com>

In Linux-4.16, drivers/staging/lustre/lustre/ptlrp/sec.c,

557. int ctx_refresh_timeout(...) {
             .......
573. req->rq_cli_ctx->cc_ops->force_die(req->rq_cli_ctx, 0);
             ......
575. }

For x86 kernel configuration, there is no assignment of the function 
pointer ".force_die" in the kernel code.
So calling the function pointer in line 573 may cause a null pointer 
dereference.


Best wishes,
Jia-Ju Bai

From baijiaju1990 at gmail.com  Thu Jul 26 14:18:35 2018
From: baijiaju1990 at gmail.com (Jia-Ju Bai)
Date: Thu, 26 Jul 2018 22:18:35 +0800
Subject: [lustre-devel] [BUG] staging: lustre: Possible null function
 pointer in ctx_refresh_timeout()
In-Reply-To: <20180726141227.GA12445@kroah.com>
References: <6920e093-ad62-56dc-6f58-36cf8d7ed3b4@gmail.com>
 <20180726141227.GA12445@kroah.com>
Message-ID: <001325ce-9edd-74a7-1359-c47b3ee66df1@gmail.com>



On 2018/7/26 22:12, Greg KH wrote:
> On Thu, Jul 26, 2018 at 10:02:22PM +0800, Jia-Ju Bai wrote:
>> In Linux-4.16, drivers/staging/lustre/lustre/ptlrp/sec.c,
> Please look at the 4.18-rc6 release for this file.
>
> In short, nothing to worry about anymore :)

Looks good now :)


Best wishes,
Jia-Ju Bai

From jsimmons at infradead.org  Sun Jul 29 16:58:25 2018
From: jsimmons at infradead.org (James Simmons)
Date: Sun, 29 Jul 2018 17:58:25 +0100 (BST)
Subject: [lustre-devel] [PATCH 2/9] lustre: llog: use GFP_KERNEL for all
 allocations.
In-Reply-To: <153232698483.26222.2842073304458010313.stgit@noble>
References: <153232696720.26222.9658151633697867322.stgit@noble>
 <153232698483.26222.2842073304458010313.stgit@noble>
Message-ID: <alpine.LFD.2.21.1807291757210.2402@casper.infradead.org>


> Processing the log happens at mount time so
> memory allocations will not trigger write-out to
> this filesystem which might deadlock.
> So use of GFP_NOFS is not needed.
> GFP_KERNEL is preferred, especially for kvmalloc()
> which doesn't try the 'vmalloc' option when
> GFP_KERNEL is not specified.

Reviewed-by: James Simmons <jsimmons at infradead.org>
 
> Signed-off-by: NeilBrown <neilb at suse.com>
> ---
>  drivers/staging/lustre/lustre/obdclass/llog.c |    6 +++---
>  1 file changed, 3 insertions(+), 3 deletions(-)
> 
> diff --git a/drivers/staging/lustre/lustre/obdclass/llog.c b/drivers/staging/lustre/lustre/obdclass/llog.c
> index f59f89a76c91..8644d349535e 100644
> --- a/drivers/staging/lustre/lustre/obdclass/llog.c
> +++ b/drivers/staging/lustre/lustre/obdclass/llog.c
> @@ -59,7 +59,7 @@ static struct llog_handle *llog_alloc_handle(void)
>  {
>  	struct llog_handle *loghandle;
>  
> -	loghandle = kzalloc(sizeof(*loghandle), GFP_NOFS);
> +	loghandle = kzalloc(sizeof(*loghandle), GFP_KERNEL);
>  	if (!loghandle)
>  		return NULL;
>  
> @@ -242,7 +242,7 @@ static int llog_process_thread(void *arg)
>  	/* expect chunk_size to be power of two */
>  	LASSERT(is_power_of_2(chunk_size));
>  
> -	buf = kvzalloc(chunk_size, GFP_NOFS);
> +	buf = kvzalloc(chunk_size, GFP_KERNEL);
>  	if (!buf) {
>  		lpi->lpi_rc = -ENOMEM;
>  		return 0;
> @@ -421,7 +421,7 @@ int llog_process_or_fork(const struct lu_env *env,
>  	struct llog_process_info *lpi;
>  	int		      rc;
>  
> -	lpi = kzalloc(sizeof(*lpi), GFP_NOFS);
> +	lpi = kzalloc(sizeof(*lpi), GFP_KERNEL);
>  	if (!lpi)
>  		return -ENOMEM;
>  	lpi->lpi_loghandle = loghandle;
> 
> 
> 

From jsimmons at infradead.org  Sun Jul 29 16:58:51 2018
From: jsimmons at infradead.org (James Simmons)
Date: Sun, 29 Jul 2018 17:58:51 +0100 (BST)
Subject: [lustre-devel] [PATCH 1/9] lustre: merge linux-debug.c into
 debug.c
In-Reply-To: <153232698479.26222.16079606642992274567.stgit@noble>
References: <153232696720.26222.9658151633697867322.stgit@noble>
 <153232698479.26222.16079606642992274567.stgit@noble>
Message-ID: <alpine.LFD.2.21.1807291758420.2402@casper.infradead.org>


> There is no important difference between the contents
> of these files, so merge them into one.

Reviewed-by: James Simmons <jsimmons at infradead.org>
 
> Signed-off-by: NeilBrown <neilb at suse.com>
> ---
>  drivers/staging/lustre/lnet/libcfs/Makefile      |    1 
>  drivers/staging/lustre/lnet/libcfs/debug.c       |   86 +++++++++++++
>  drivers/staging/lustre/lnet/libcfs/linux-debug.c |  142 ----------------------
>  drivers/staging/lustre/lnet/libcfs/tracefile.h   |    4 -
>  4 files changed, 86 insertions(+), 147 deletions(-)
>  delete mode 100644 drivers/staging/lustre/lnet/libcfs/linux-debug.c
> 
> diff --git a/drivers/staging/lustre/lnet/libcfs/Makefile b/drivers/staging/lustre/lnet/libcfs/Makefile
> index cd96434e3b03..a45f87af3faf 100644
> --- a/drivers/staging/lustre/lnet/libcfs/Makefile
> +++ b/drivers/staging/lustre/lnet/libcfs/Makefile
> @@ -4,7 +4,6 @@ ccflags-y += -I$(srctree)/drivers/staging/lustre/lustre/include
>  
>  obj-$(CONFIG_LNET) += libcfs.o
>  
> -libcfs-obj-y += linux-debug.o
>  libcfs-obj-y += linux-crypto.o
>  libcfs-obj-y += linux-crypto-adler.o
>  
> diff --git a/drivers/staging/lustre/lnet/libcfs/debug.c b/drivers/staging/lustre/lnet/libcfs/debug.c
> index 844543b2317f..0289f24698fa 100644
> --- a/drivers/staging/lustre/lnet/libcfs/debug.c
> +++ b/drivers/staging/lustre/lnet/libcfs/debug.c
> @@ -327,6 +327,40 @@ libcfs_debug_str2mask(int *mask, const char *str, int is_subsys)
>  			    0xffffffff);
>  }
>  
> +char lnet_debug_log_upcall[1024] = "/usr/lib/lustre/lnet_debug_log_upcall";
> +
> +/**
> + * Upcall function once a Lustre log has been dumped.
> + *
> + * \param file  path of the dumped log
> + */
> +static void libcfs_run_debug_log_upcall(char *file)
> +{
> +	char *argv[3];
> +	int rc;
> +	static const char * const envp[] = {
> +		"HOME=/",
> +		"PATH=/sbin:/bin:/usr/sbin:/usr/bin",
> +		NULL
> +	};
> +
> +	argv[0] = lnet_debug_log_upcall;
> +
> +	LASSERTF(file, "called on a null filename\n");
> +	argv[1] = file; /* only need to pass the path of the file */
> +
> +	argv[2] = NULL;
> +
> +	rc = call_usermodehelper(argv[0], argv, (char **)envp, 1);
> +	if (rc < 0 && rc != -ENOENT) {
> +		CERROR("Error %d invoking LNET debug log upcall %s %s; check /sys/kernel/debug/lnet/debug_log_upcall\n",
> +		       rc, argv[0], argv[1]);
> +	} else {
> +		CDEBUG(D_HA, "Invoked LNET debug log upcall %s %s\n",
> +		       argv[0], argv[1]);
> +	}
> +}
> +
>  /**
>   * Dump Lustre log to ::debug_file_path by calling tracefile_dump_all_pages()
>   */
> @@ -389,6 +423,58 @@ void libcfs_debug_dumplog(void)
>  }
>  EXPORT_SYMBOL(libcfs_debug_dumplog);
>  
> +/* coverity[+kill] */
> +void __noreturn lbug_with_loc(struct libcfs_debug_msg_data *msgdata)
> +{
> +	libcfs_catastrophe = 1;
> +	libcfs_debug_msg(msgdata, "LBUG\n");
> +
> +	if (in_interrupt()) {
> +		panic("LBUG in interrupt.\n");
> +		/* not reached */
> +	}
> +
> +	dump_stack();
> +	if (!libcfs_panic_on_lbug)
> +		libcfs_debug_dumplog();
> +	if (libcfs_panic_on_lbug)
> +		panic("LBUG");
> +	set_current_state(TASK_UNINTERRUPTIBLE);
> +	while (1)
> +		schedule();
> +}
> +EXPORT_SYMBOL(lbug_with_loc);
> +
> +static int panic_notifier(struct notifier_block *self, unsigned long unused1,
> +			  void *unused2)
> +{
> +	if (libcfs_panic_in_progress)
> +		return 0;
> +
> +	libcfs_panic_in_progress = 1;
> +	mb();
> +
> +	return 0;
> +}
> +
> +static struct notifier_block libcfs_panic_notifier = {
> +	.notifier_call	= panic_notifier,
> +	.next		= NULL,
> +	.priority	= 10000,
> +};
> +
> +static void libcfs_register_panic_notifier(void)
> +{
> +	atomic_notifier_chain_register(&panic_notifier_list,
> +				       &libcfs_panic_notifier);
> +}
> +
> +static void libcfs_unregister_panic_notifier(void)
> +{
> +	atomic_notifier_chain_unregister(&panic_notifier_list,
> +					 &libcfs_panic_notifier);
> +}
> +
>  int libcfs_debug_init(unsigned long bufsize)
>  {
>  	unsigned int max = libcfs_debug_mb;
> diff --git a/drivers/staging/lustre/lnet/libcfs/linux-debug.c b/drivers/staging/lustre/lnet/libcfs/linux-debug.c
> deleted file mode 100644
> index 15ab849374c2..000000000000
> --- a/drivers/staging/lustre/lnet/libcfs/linux-debug.c
> +++ /dev/null
> @@ -1,142 +0,0 @@
> -// SPDX-License-Identifier: GPL-2.0
> -/*
> - * GPL HEADER START
> - *
> - * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
> - *
> - * This program is free software; you can redistribute it and/or modify
> - * it under the terms of the GNU General Public License version 2 only,
> - * as published by the Free Software Foundation.
> - *
> - * This program is distributed in the hope that it will be useful, but
> - * WITHOUT ANY WARRANTY; without even the implied warranty of
> - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> - * General Public License version 2 for more details (a copy is included
> - * in the LICENSE file that accompanied this code).
> - *
> - * You should have received a copy of the GNU General Public License
> - * version 2 along with this program; If not, see
> - * http://www.gnu.org/licenses/gpl-2.0.html
> - *
> - * GPL HEADER END
> - */
> -/*
> - * Copyright (c) 2008, 2010, Oracle and/or its affiliates. All rights reserved.
> - * Use is subject to license terms.
> - *
> - * Copyright (c) 2012, Intel Corporation.
> - */
> -/*
> - * This file is part of Lustre, http://www.lustre.org/
> - * Lustre is a trademark of Sun Microsystems, Inc.
> - *
> - * libcfs/libcfs/linux/linux-debug.c
> - *
> - * Author: Phil Schwan <phil at clusterfs.com>
> - */
> -
> -#include <linux/module.h>
> -#include <linux/kmod.h>
> -#include <linux/notifier.h>
> -#include <linux/kernel.h>
> -#include <linux/mm.h>
> -#include <linux/string.h>
> -#include <linux/stat.h>
> -#include <linux/errno.h>
> -#include <linux/unistd.h>
> -#include <linux/interrupt.h>
> -#include <linux/completion.h>
> -#include <linux/fs.h>
> -#include <linux/uaccess.h>
> -
> -# define DEBUG_SUBSYSTEM S_LNET
> -
> -#include "tracefile.h"
> -
> -#include <linux/kallsyms.h>
> -
> -char lnet_debug_log_upcall[1024] = "/usr/lib/lustre/lnet_debug_log_upcall";
> -
> -/**
> - * Upcall function once a Lustre log has been dumped.
> - *
> - * \param file  path of the dumped log
> - */
> -void libcfs_run_debug_log_upcall(char *file)
> -{
> -	char *argv[3];
> -	int rc;
> -	static const char * const envp[] = {
> -		"HOME=/",
> -		"PATH=/sbin:/bin:/usr/sbin:/usr/bin",
> -		NULL
> -	};
> -
> -	argv[0] = lnet_debug_log_upcall;
> -
> -	LASSERTF(file, "called on a null filename\n");
> -	argv[1] = file; /* only need to pass the path of the file */
> -
> -	argv[2] = NULL;
> -
> -	rc = call_usermodehelper(argv[0], argv, (char **)envp, 1);
> -	if (rc < 0 && rc != -ENOENT) {
> -		CERROR("Error %d invoking LNET debug log upcall %s %s; check /sys/kernel/debug/lnet/debug_log_upcall\n",
> -		       rc, argv[0], argv[1]);
> -	} else {
> -		CDEBUG(D_HA, "Invoked LNET debug log upcall %s %s\n",
> -		       argv[0], argv[1]);
> -	}
> -}
> -
> -/* coverity[+kill] */
> -void __noreturn lbug_with_loc(struct libcfs_debug_msg_data *msgdata)
> -{
> -	libcfs_catastrophe = 1;
> -	libcfs_debug_msg(msgdata, "LBUG\n");
> -
> -	if (in_interrupt()) {
> -		panic("LBUG in interrupt.\n");
> -		/* not reached */
> -	}
> -
> -	dump_stack();
> -	if (!libcfs_panic_on_lbug)
> -		libcfs_debug_dumplog();
> -	if (libcfs_panic_on_lbug)
> -		panic("LBUG");
> -	set_current_state(TASK_UNINTERRUPTIBLE);
> -	while (1)
> -		schedule();
> -}
> -EXPORT_SYMBOL(lbug_with_loc);
> -
> -static int panic_notifier(struct notifier_block *self, unsigned long unused1,
> -			  void *unused2)
> -{
> -	if (libcfs_panic_in_progress)
> -		return 0;
> -
> -	libcfs_panic_in_progress = 1;
> -	mb();
> -
> -	return 0;
> -}
> -
> -static struct notifier_block libcfs_panic_notifier = {
> -	.notifier_call	= panic_notifier,
> -	.next		= NULL,
> -	.priority	= 10000,
> -};
> -
> -void libcfs_register_panic_notifier(void)
> -{
> -	atomic_notifier_chain_register(&panic_notifier_list,
> -				       &libcfs_panic_notifier);
> -}
> -
> -void libcfs_unregister_panic_notifier(void)
> -{
> -	atomic_notifier_chain_unregister(&panic_notifier_list,
> -					 &libcfs_panic_notifier);
> -}
> diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.h b/drivers/staging/lustre/lnet/libcfs/tracefile.h
> index 67107b1a705a..82f090fd8dfa 100644
> --- a/drivers/staging/lustre/lnet/libcfs/tracefile.h
> +++ b/drivers/staging/lustre/lnet/libcfs/tracefile.h
> @@ -51,8 +51,6 @@ extern long long cfs_tracefile_size;
>   */
>  extern char lnet_debug_log_upcall[1024];
>  
> -void libcfs_run_debug_log_upcall(char *file);
> -
>  int cfs_tracefile_dump_all_pages(char *filename);
>  void cfs_trace_debug_print(void);
>  void cfs_trace_flush_pages(void);
> @@ -73,8 +71,6 @@ int cfs_trace_set_debug_mb(int mb);
>  int cfs_trace_get_debug_mb(void);
>  
>  void libcfs_debug_dumplog_internal(void *arg);
> -void libcfs_register_panic_notifier(void);
> -void libcfs_unregister_panic_notifier(void);
>  extern int libcfs_panic_in_progress;
>  
>  #define TCD_MAX_PAGES (5 << (20 - PAGE_SHIFT))
> 
> 
> 

From jsimmons at infradead.org  Sun Jul 29 16:59:57 2018
From: jsimmons at infradead.org (James Simmons)
Date: Sun, 29 Jul 2018 17:59:57 +0100 (BST)
Subject: [lustre-devel] [PATCH 4/9] lustre: move lustre_check_exclusion
 to obd_config.c
In-Reply-To: <153232698489.26222.11036165790183672424.stgit@noble>
References: <153232696720.26222.9658151633697867322.stgit@noble>
 <153232698489.26222.11036165790183672424.stgit@noble>
Message-ID: <alpine.LFD.2.21.1807291759230.2402@casper.infradead.org>


> lustre_check_exclusion is only used in obd_config.c, so
> move the code there
> Note that a CDEBUG is changed from D_MOUNT to D_INFO.
> This prepares for obd_mount to move to llite/

Reviewed-by: James Simmons <jsimmons at infradead.org>
 
> Signed-off-by: NeilBrown <neilb at suse.com>
> ---
>  drivers/staging/lustre/lustre/include/obd_class.h  |    1 -
>  .../staging/lustre/lustre/obdclass/obd_config.c    |   25 ++++++++++++++++++++
>  drivers/staging/lustre/lustre/obdclass/obd_mount.c |   25 --------------------
>  3 files changed, 25 insertions(+), 26 deletions(-)
> 
> diff --git a/drivers/staging/lustre/lustre/include/obd_class.h b/drivers/staging/lustre/lustre/include/obd_class.h
> index ef8c46cb9bee..90dbafd38d71 100644
> --- a/drivers/staging/lustre/lustre/include/obd_class.h
> +++ b/drivers/staging/lustre/lustre/include/obd_class.h
> @@ -1559,7 +1559,6 @@ extern int (*ptlrpc_put_connection_superhack)(struct ptlrpc_connection *c);
>  /* obd_mount.c */
>  int lustre_unregister_fs(void);
>  int lustre_register_fs(void);
> -int lustre_check_exclusion(struct super_block *sb, char *svname);
>  
>  /* sysctl.c */
>  int obd_sysctl_init(void);
> diff --git a/drivers/staging/lustre/lustre/obdclass/obd_config.c b/drivers/staging/lustre/lustre/obdclass/obd_config.c
> index 382522fa1993..f3dd020d04ef 100644
> --- a/drivers/staging/lustre/lustre/obdclass/obd_config.c
> +++ b/drivers/staging/lustre/lustre/obdclass/obd_config.c
> @@ -1258,6 +1258,31 @@ int server_name2index(const char *svname, __u32 *idx,
>  }
>  EXPORT_SYMBOL(server_name2index);
>  
> +/* Is this server on the exclusion list */
> +static int lustre_check_exclusion(struct super_block *sb, char *svname)
> +{
> +	struct lustre_sb_info *lsi = s2lsi(sb);
> +	struct lustre_mount_data *lmd = lsi->lsi_lmd;
> +	__u32 index;
> +	int i, rc;
> +
> +	rc = server_name2index(svname, &index, NULL);
> +	if (rc != LDD_F_SV_TYPE_OST)
> +		/* Only exclude OSTs */
> +		return 0;
> +
> +	CDEBUG(D_INFO, "Check exclusion %s (%d) in %d of %s\n", svname,
> +	       index, lmd->lmd_exclude_count, lmd->lmd_dev);
> +
> +	for (i = 0; i < lmd->lmd_exclude_count; i++) {
> +		if (index == lmd->lmd_exclude[i]) {
> +			CWARN("Excluding %s (on exclusion list)\n", svname);
> +			return 1;
> +		}
> +	}
> +	return 0;
> +}
> +
>  /** Parse a configuration llog, doing various manipulations on them
>   * for various reasons, (modifications for compatibility, skip obsolete
>   * records, change uuids, etc), then class_process_config() resulting
> diff --git a/drivers/staging/lustre/lustre/obdclass/obd_mount.c b/drivers/staging/lustre/lustre/obdclass/obd_mount.c
> index 3420e87ad0cd..4e4a8c35105b 100644
> --- a/drivers/staging/lustre/lustre/obdclass/obd_mount.c
> +++ b/drivers/staging/lustre/lustre/obdclass/obd_mount.c
> @@ -633,31 +633,6 @@ static void lmd_print(struct lustre_mount_data *lmd)
>  	}
>  }
>  
> -/* Is this server on the exclusion list */
> -int lustre_check_exclusion(struct super_block *sb, char *svname)
> -{
> -	struct lustre_sb_info *lsi = s2lsi(sb);
> -	struct lustre_mount_data *lmd = lsi->lsi_lmd;
> -	__u32 index;
> -	int i, rc;
> -
> -	rc = server_name2index(svname, &index, NULL);
> -	if (rc != LDD_F_SV_TYPE_OST)
> -		/* Only exclude OSTs */
> -		return 0;
> -
> -	CDEBUG(D_MOUNT, "Check exclusion %s (%d) in %d of %s\n", svname,
> -	       index, lmd->lmd_exclude_count, lmd->lmd_dev);
> -
> -	for (i = 0; i < lmd->lmd_exclude_count; i++) {
> -		if (index == lmd->lmd_exclude[i]) {
> -			CWARN("Excluding %s (on exclusion list)\n", svname);
> -			return 1;
> -		}
> -	}
> -	return 0;
> -}
> -
>  /* mount -v  -o exclude=lustre-OST0001:lustre-OST0002 -t lustre ... */
>  static int lmd_make_exclusion(struct lustre_mount_data *lmd, const char *ptr)
>  {
> 
> 
> 

From jsimmons at infradead.org  Sun Jul 29 17:28:01 2018
From: jsimmons at infradead.org (James Simmons)
Date: Sun, 29 Jul 2018 18:28:01 +0100 (BST)
Subject: [lustre-devel] [PATCH 5/9] lustre: move filesystem mounting
 from obdclass to llite
In-Reply-To: <153232698492.26222.12045767311396974106.stgit@noble>
References: <153232696720.26222.9658151633697867322.stgit@noble>
 <153232698492.26222.12045767311396974106.stgit@noble>
Message-ID: <alpine.LFD.2.21.1807291807490.2402@casper.infradead.org>


> All filesystem-api aspects of the lustre filesystem are in
> llite, except mounting which is in obdclass.
> This is probably because the name filesystem type is/was
> used for mounting an lustre filesystem on the client, and
> for mounting a data-store on the server.
> This is a confusion that is best discarded and forgotten.
> 
> So move the mount handling into llite.
> 
> This removes that rather awkward requirment that obdclass needed to
> request_module() the lustre module, and makes lustre_fill_super()
> significantly simpler.
> 
> Several symbols don't need to be exported any more as they are now
> part of llite which is the only module that uses them.

Nak. The majority of this code is shared between both server and client
code. Moving this directly to llite would make server code dependent on
llite which is what is not desired. Things like lustre_process_log() and
lustre_end_log() have no client or server specific code and all nodes
running lustre independent of the function of the node execute these code
paths. Please see obd_mount_server.c to see what functions are used.

Do you mind if I take a stab at this ?
 
> Signed-off-by: NeilBrown <neilb at suse.com>
> ---
>  drivers/staging/lustre/lustre/include/obd_class.h  |    4 
>  drivers/staging/lustre/lustre/llite/Makefile       |    2 
>  .../staging/lustre/lustre/llite/llite_internal.h   |    3 
>  drivers/staging/lustre/lustre/llite/mount.c        | 1208 +++++++++++++++++++
>  drivers/staging/lustre/lustre/llite/super25.c      |    8 
>  drivers/staging/lustre/lustre/obdclass/Makefile    |    2 
>  drivers/staging/lustre/lustre/obdclass/class_obd.c |    6 
>  drivers/staging/lustre/lustre/obdclass/obd_mount.c | 1244 --------------------
>  8 files changed, 1218 insertions(+), 1259 deletions(-)
>  create mode 100644 drivers/staging/lustre/lustre/llite/mount.c
>  delete mode 100644 drivers/staging/lustre/lustre/obdclass/obd_mount.c
> 
> diff --git a/drivers/staging/lustre/lustre/include/obd_class.h b/drivers/staging/lustre/lustre/include/obd_class.h
> index 90dbafd38d71..1a55f0215837 100644
> --- a/drivers/staging/lustre/lustre/include/obd_class.h
> +++ b/drivers/staging/lustre/lustre/include/obd_class.h
> @@ -1556,10 +1556,6 @@ struct lwp_register_item {
>   */
>  extern int (*ptlrpc_put_connection_superhack)(struct ptlrpc_connection *c);
>  
> -/* obd_mount.c */
> -int lustre_unregister_fs(void);
> -int lustre_register_fs(void);
> -
>  /* sysctl.c */
>  int obd_sysctl_init(void);
>  
> diff --git a/drivers/staging/lustre/lustre/llite/Makefile b/drivers/staging/lustre/lustre/llite/Makefile
> index 7d0225476362..670f072ce370 100644
> --- a/drivers/staging/lustre/lustre/llite/Makefile
> +++ b/drivers/staging/lustre/lustre/llite/Makefile
> @@ -3,7 +3,7 @@ ccflags-y += -I$(srctree)/drivers/staging/lustre/include
>  ccflags-y += -I$(srctree)/drivers/staging/lustre/lustre/include
>  
>  obj-$(CONFIG_LUSTRE_FS) += lustre.o
> -lustre-y := dcache.o dir.o file.o llite_lib.o llite_nfs.o \
> +lustre-y := mount.o dcache.o dir.o file.o llite_lib.o llite_nfs.o \
>  	    rw.o rw26.o namei.o symlink.o llite_mmap.o range_lock.o \
>  	    xattr.o xattr_cache.o xattr_security.o \
>  	    super25.o statahead.o glimpse.o lcommon_cl.o lcommon_misc.o \
> diff --git a/drivers/staging/lustre/lustre/llite/llite_internal.h b/drivers/staging/lustre/lustre/llite/llite_internal.h
> index 8399501c9122..1f832a5ba023 100644
> --- a/drivers/staging/lustre/lustre/llite/llite_internal.h
> +++ b/drivers/staging/lustre/lustre/llite/llite_internal.h
> @@ -1351,4 +1351,7 @@ void cl_inode_fini(struct inode *inode);
>  __u64 cl_fid_build_ino(const struct lu_fid *fid, int api32);
>  __u32 cl_fid_build_gen(const struct lu_fid *fid);
>  
> +int lustre_unregister_fs(void);
> +int lustre_register_fs(void);
> +
>  #endif /* LLITE_INTERNAL_H */
> diff --git a/drivers/staging/lustre/lustre/llite/mount.c b/drivers/staging/lustre/lustre/llite/mount.c
> new file mode 100644
> index 000000000000..58e8b371f0c4
> --- /dev/null
> +++ b/drivers/staging/lustre/lustre/llite/mount.c
> @@ -0,0 +1,1208 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * GPL HEADER START
> + *
> + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 only,
> + * as published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful, but
> + * WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> + * General Public License version 2 for more details (a copy is included
> + * in the LICENSE file that accompanied this code).
> + *
> + * You should have received a copy of the GNU General Public License
> + * version 2 along with this program; If not, see
> + * http://www.gnu.org/licenses/gpl-2.0.html
> + *
> + * GPL HEADER END
> + */
> +/*
> + * Copyright (c) 2007, 2010, Oracle and/or its affiliates. All rights reserved.
> + * Use is subject to license terms.
> + *
> + * Copyright (c) 2011, 2015, Intel Corporation.
> + */
> +/*
> + * This file is part of Lustre, http://www.lustre.org/
> + * Lustre is a trademark of Sun Microsystems, Inc.
> + *
> + * lustre/obdclass/obd_mount.c
> + *
> + * Client mount routines
> + *
> + * Author: Nathan Rutman <nathan at clusterfs.com>
> + */
> +
> +#define DEBUG_SUBSYSTEM S_CLASS
> +#define D_MOUNT (D_SUPER | D_CONFIG/*|D_WARNING */)
> +#define PRINT_CMD CDEBUG
> +
> +#include <linux/random.h>
> +#include <obd.h>
> +#include <obd_class.h>
> +#include <uapi/linux/lustre/lustre_idl.h>
> +#include <lustre_log.h>
> +#include <lustre_disk.h>
> +#include <uapi/linux/lustre/lustre_param.h>
> +#include <llite_internal.h>
> +
> +/**************** config llog ********************/
> +
> +/** Get a config log from the MGS and process it.
> + * This func is called for both clients and servers.
> + * Continue to process new statements appended to the logs
> + * (whenever the config lock is revoked) until lustre_end_log
> + * is called.
> + * @param sb The superblock is used by the MGC to write to the local copy of
> + *   the config log
> + * @param logname The name of the llog to replicate from the MGS
> + * @param cfg Since the same mgc may be used to follow multiple config logs
> + *   (e.g. ost1, ost2, client), the config_llog_instance keeps the state for
> + *   this log, and is added to the mgc's list of logs to follow.
> + */
> +int lustre_process_log(struct super_block *sb, char *logname,
> +		       struct config_llog_instance *cfg)
> +{
> +	struct lustre_cfg *lcfg;
> +	struct lustre_cfg_bufs *bufs;
> +	struct lustre_sb_info *lsi = s2lsi(sb);
> +	struct obd_device *mgc = lsi->lsi_mgc;
> +	int rc;
> +
> +	LASSERT(mgc);
> +	LASSERT(cfg);
> +
> +	bufs = kzalloc(sizeof(*bufs), GFP_NOFS);
> +	if (!bufs)
> +		return -ENOMEM;
> +
> +	/* mgc_process_config */
> +	lustre_cfg_bufs_reset(bufs, mgc->obd_name);
> +	lustre_cfg_bufs_set_string(bufs, 1, logname);
> +	lustre_cfg_bufs_set(bufs, 2, cfg, sizeof(*cfg));
> +	lustre_cfg_bufs_set(bufs, 3, &sb, sizeof(sb));
> +	lcfg = kzalloc(lustre_cfg_len(bufs->lcfg_bufcount, bufs->lcfg_buflen),
> +		       GFP_NOFS);
> +	if (!lcfg) {
> +		rc = -ENOMEM;
> +		goto out;
> +	}
> +	lustre_cfg_init(lcfg, LCFG_LOG_START, bufs);
> +
> +	rc = obd_process_config(mgc, sizeof(*lcfg), lcfg);
> +	kfree(lcfg);
> +out:
> +	kfree(bufs);
> +
> +	if (rc == -EINVAL)
> +		LCONSOLE_ERROR_MSG(0x15b, "%s: The configuration from log '%s' failed from the MGS (%d).  Make sure this client and the MGS are running compatible versions of Lustre.\n",
> +				   mgc->obd_name, logname, rc);
> +
> +	else if (rc)
> +		LCONSOLE_ERROR_MSG(0x15c, "%s: The configuration from log '%s' failed (%d). This may be the result of communication errors between this node and the MGS, a bad configuration, or other errors. See the syslog for more information.\n",
> +				   mgc->obd_name, logname,
> +				   rc);
> +
> +	/* class_obd_list(); */
> +	return rc;
> +}
> +
> +/* Stop watching this config log for updates */
> +int lustre_end_log(struct super_block *sb, char *logname,
> +		   struct config_llog_instance *cfg)
> +{
> +	struct lustre_cfg *lcfg;
> +	struct lustre_cfg_bufs bufs;
> +	struct lustre_sb_info *lsi = s2lsi(sb);
> +	struct obd_device *mgc = lsi->lsi_mgc;
> +	int rc;
> +
> +	if (!mgc)
> +		return -ENOENT;
> +
> +	/* mgc_process_config */
> +	lustre_cfg_bufs_reset(&bufs, mgc->obd_name);
> +	lustre_cfg_bufs_set_string(&bufs, 1, logname);
> +	if (cfg)
> +		lustre_cfg_bufs_set(&bufs, 2, cfg, sizeof(*cfg));
> +	lcfg = kzalloc(lustre_cfg_len(bufs.lcfg_bufcount, bufs.lcfg_buflen),
> +		       GFP_NOFS);
> +	if (!lcfg)
> +		return -ENOMEM;
> +	lustre_cfg_init(lcfg, LCFG_LOG_END, &bufs);
> +
> +	rc = obd_process_config(mgc, sizeof(*lcfg), lcfg);
> +	kfree(lcfg);
> +	return rc;
> +}
> +
> +/**************** obd start *******************/
> +
> +/** lustre_cfg_bufs are a holdover from 1.4; we can still set these up from
> + * lctl (and do for echo cli/srv.
> + */
> +static int do_lcfg(char *cfgname, lnet_nid_t nid, int cmd,
> +		   char *s1, char *s2, char *s3, char *s4)
> +{
> +	struct lustre_cfg_bufs bufs;
> +	struct lustre_cfg     *lcfg = NULL;
> +	int rc;
> +
> +	CDEBUG(D_TRACE, "lcfg %s %#x %s %s %s %s\n", cfgname,
> +	       cmd, s1, s2, s3, s4);
> +
> +	lustre_cfg_bufs_reset(&bufs, cfgname);
> +	if (s1)
> +		lustre_cfg_bufs_set_string(&bufs, 1, s1);
> +	if (s2)
> +		lustre_cfg_bufs_set_string(&bufs, 2, s2);
> +	if (s3)
> +		lustre_cfg_bufs_set_string(&bufs, 3, s3);
> +	if (s4)
> +		lustre_cfg_bufs_set_string(&bufs, 4, s4);
> +
> +	lcfg = kzalloc(lustre_cfg_len(bufs.lcfg_bufcount, bufs.lcfg_buflen),
> +		       GFP_NOFS);
> +	if (!lcfg)
> +		return -ENOMEM;
> +	lustre_cfg_init(lcfg, cmd, &bufs);
> +	lcfg->lcfg_nid = nid;
> +	rc = class_process_config(lcfg);
> +	kfree(lcfg);
> +	return rc;
> +}
> +
> +/** Call class_attach and class_setup.  These methods in turn call
> + * obd type-specific methods.
> + */
> +static int lustre_start_simple(char *obdname, char *type, char *uuid,
> +			       char *s1, char *s2, char *s3, char *s4)
> +{
> +	int rc;
> +
> +	CDEBUG(D_MOUNT, "Starting obd %s (typ=%s)\n", obdname, type);
> +
> +	rc = do_lcfg(obdname, 0, LCFG_ATTACH, type, uuid, NULL, NULL);
> +	if (rc) {
> +		CERROR("%s attach error %d\n", obdname, rc);
> +		return rc;
> +	}
> +	rc = do_lcfg(obdname, 0, LCFG_SETUP, s1, s2, s3, s4);
> +	if (rc) {
> +		CERROR("%s setup error %d\n", obdname, rc);
> +		do_lcfg(obdname, 0, LCFG_DETACH, NULL, NULL, NULL, NULL);
> +	}
> +	return rc;
> +}
> +
> +static DEFINE_MUTEX(mgc_start_lock);
> +
> +/** Set up a mgc obd to process startup logs
> + *
> + * \param sb [in] super block of the mgc obd
> + *
> + * \retval 0 success, otherwise error code
> + */
> +int lustre_start_mgc(struct super_block *sb)
> +{
> +	struct obd_connect_data *data = NULL;
> +	struct lustre_sb_info *lsi = s2lsi(sb);
> +	struct obd_device *obd;
> +	struct obd_export *exp;
> +	struct obd_uuid *uuid;
> +	class_uuid_t uuidc;
> +	lnet_nid_t nid;
> +	char nidstr[LNET_NIDSTR_SIZE];
> +	char *mgcname = NULL, *niduuid = NULL, *mgssec = NULL;
> +	char *ptr;
> +	int rc = 0, i = 0, j;
> +
> +	LASSERT(lsi->lsi_lmd);
> +
> +	/* Use nids from mount line: uml1,1 at elan:uml2,2 at elan:/lustre */
> +	ptr = lsi->lsi_lmd->lmd_dev;
> +	if (class_parse_nid(ptr, &nid, &ptr) == 0)
> +		i++;
> +	if (i == 0) {
> +		CERROR("No valid MGS nids found.\n");
> +		return -EINVAL;
> +	}
> +
> +	mutex_lock(&mgc_start_lock);
> +
> +	libcfs_nid2str_r(nid, nidstr, sizeof(nidstr));
> +	mgcname = kasprintf(GFP_NOFS,
> +			    "%s%s", LUSTRE_MGC_OBDNAME, nidstr);
> +	niduuid = kasprintf(GFP_NOFS, "%s_%x", mgcname, 0);
> +	if (!mgcname || !niduuid) {
> +		rc = -ENOMEM;
> +		goto out_free;
> +	}
> +
> +	mgssec = lsi->lsi_lmd->lmd_mgssec ? lsi->lsi_lmd->lmd_mgssec : "";
> +
> +	data = kzalloc(sizeof(*data), GFP_NOFS);
> +	if (!data) {
> +		rc = -ENOMEM;
> +		goto out_free;
> +	}
> +
> +	obd = class_name2obd(mgcname);
> +	if (obd && !obd->obd_stopping) {
> +		int recov_bk;
> +
> +		rc = obd_set_info_async(NULL, obd->obd_self_export,
> +					strlen(KEY_MGSSEC), KEY_MGSSEC,
> +					strlen(mgssec), mgssec, NULL);
> +		if (rc)
> +			goto out_free;
> +
> +		/* Re-using an existing MGC */
> +		atomic_inc(&obd->u.cli.cl_mgc_refcount);
> +
> +		/* IR compatibility check, only for clients */
> +		if (lmd_is_client(lsi->lsi_lmd)) {
> +			int has_ir;
> +			int vallen = sizeof(*data);
> +			__u32 *flags = &lsi->lsi_lmd->lmd_flags;
> +
> +			rc = obd_get_info(NULL, obd->obd_self_export,
> +					  strlen(KEY_CONN_DATA), KEY_CONN_DATA,
> +					  &vallen, data);
> +			LASSERT(rc == 0);
> +			has_ir = OCD_HAS_FLAG(data, IMP_RECOV);
> +			if (has_ir ^ !(*flags & LMD_FLG_NOIR)) {
> +				/* LMD_FLG_NOIR is for test purpose only */
> +				LCONSOLE_WARN(
> +					"Trying to mount a client with IR setting not compatible with current mgc. Force to use current mgc setting that is IR %s.\n",
> +					has_ir ? "enabled" : "disabled");
> +				if (has_ir)
> +					*flags &= ~LMD_FLG_NOIR;
> +				else
> +					*flags |= LMD_FLG_NOIR;
> +			}
> +		}
> +
> +		recov_bk = 0;
> +
> +		/* Try all connections, but only once (again).
> +		 * We don't want to block another target from starting
> +		 * (using its local copy of the log), but we do want to connect
> +		 * if at all possible.
> +		 */
> +		recov_bk++;
> +		CDEBUG(D_MOUNT, "%s: Set MGC reconnect %d\n", mgcname,
> +		       recov_bk);
> +		rc = obd_set_info_async(NULL, obd->obd_self_export,
> +					sizeof(KEY_INIT_RECOV_BACKUP),
> +					KEY_INIT_RECOV_BACKUP,
> +					sizeof(recov_bk), &recov_bk, NULL);
> +		rc = 0;
> +		goto out;
> +	}
> +
> +	CDEBUG(D_MOUNT, "Start MGC '%s'\n", mgcname);
> +
> +	/* Add the primary nids for the MGS */
> +	i = 0;
> +	/* Use nids from mount line: uml1,1 at elan:uml2,2 at elan:/lustre */
> +	ptr = lsi->lsi_lmd->lmd_dev;
> +	while (class_parse_nid(ptr, &nid, &ptr) == 0) {
> +		rc = do_lcfg(mgcname, nid,
> +			     LCFG_ADD_UUID, niduuid, NULL, NULL, NULL);
> +		if (!rc)
> +			i++;
> +		/* Stop at the first failover nid */
> +		if (*ptr == ':')
> +			break;
> +	}
> +	if (i == 0) {
> +		CERROR("No valid MGS nids found.\n");
> +		rc = -EINVAL;
> +		goto out_free;
> +	}
> +	lsi->lsi_lmd->lmd_mgs_failnodes = 1;
> +
> +	/* Random uuid for MGC allows easier reconnects */
> +	uuid = kzalloc(sizeof(*uuid), GFP_NOFS);
> +	if (!uuid) {
> +		rc = -ENOMEM;
> +		goto out_free;
> +	}
> +
> +	ll_generate_random_uuid(uuidc);
> +	class_uuid_unparse(uuidc, uuid);
> +
> +	/* Start the MGC */
> +	rc = lustre_start_simple(mgcname, LUSTRE_MGC_NAME,
> +				 (char *)uuid->uuid, LUSTRE_MGS_OBDNAME,
> +				 niduuid, NULL, NULL);
> +	kfree(uuid);
> +	if (rc)
> +		goto out_free;
> +
> +	/* Add any failover MGS nids */
> +	i = 1;
> +	while (ptr && ((*ptr == ':' ||
> +			class_find_param(ptr, PARAM_MGSNODE, &ptr) == 0))) {
> +		/* New failover node */
> +		sprintf(niduuid, "%s_%x", mgcname, i);
> +		j = 0;
> +		while (class_parse_nid_quiet(ptr, &nid, &ptr) == 0) {
> +			rc = do_lcfg(mgcname, nid, LCFG_ADD_UUID, niduuid,
> +				     NULL, NULL, NULL);
> +			if (!rc)
> +				++j;
> +			if (*ptr == ':')
> +				break;
> +		}
> +		if (j > 0) {
> +			rc = do_lcfg(mgcname, 0, LCFG_ADD_CONN,
> +				     niduuid, NULL, NULL, NULL);
> +			if (!rc)
> +				i++;
> +		} else {
> +			/* at ":/fsname" */
> +			break;
> +		}
> +	}
> +	lsi->lsi_lmd->lmd_mgs_failnodes = i;
> +
> +	obd = class_name2obd(mgcname);
> +	if (!obd) {
> +		CERROR("Can't find mgcobd %s\n", mgcname);
> +		rc = -ENOTCONN;
> +		goto out_free;
> +	}
> +
> +	rc = obd_set_info_async(NULL, obd->obd_self_export,
> +				strlen(KEY_MGSSEC), KEY_MGSSEC,
> +				strlen(mgssec), mgssec, NULL);
> +	if (rc)
> +		goto out_free;
> +
> +	/* Keep a refcount of servers/clients who started with "mount",
> +	 * so we know when we can get rid of the mgc.
> +	 */
> +	atomic_set(&obd->u.cli.cl_mgc_refcount, 1);
> +
> +	/* We connect to the MGS at setup, and don't disconnect until cleanup */
> +	data->ocd_connect_flags = OBD_CONNECT_VERSION | OBD_CONNECT_AT |
> +				  OBD_CONNECT_FULL20 | OBD_CONNECT_IMP_RECOV |
> +				  OBD_CONNECT_LVB_TYPE | OBD_CONNECT_BULK_MBITS;
> +
> +#if OBD_OCD_VERSION(3, 0, 53, 0) > LUSTRE_VERSION_CODE
> +	data->ocd_connect_flags |= OBD_CONNECT_MNE_SWAB;
> +#endif
> +
> +	if (lmd_is_client(lsi->lsi_lmd) &&
> +	    lsi->lsi_lmd->lmd_flags & LMD_FLG_NOIR)
> +		data->ocd_connect_flags &= ~OBD_CONNECT_IMP_RECOV;
> +	data->ocd_version = LUSTRE_VERSION_CODE;
> +	rc = obd_connect(NULL, &exp, obd, &obd->obd_uuid, data, NULL);
> +	if (rc) {
> +		CERROR("connect failed %d\n", rc);
> +		goto out;
> +	}
> +
> +	obd->u.cli.cl_mgc_mgsexp = exp;
> +
> +out:
> +	/* Keep the mgc info in the sb. Note that many lsi's can point
> +	 * to the same mgc.
> +	 */
> +	lsi->lsi_mgc = obd;
> +out_free:
> +	mutex_unlock(&mgc_start_lock);
> +
> +	kfree(data);
> +	kfree(mgcname);
> +	kfree(niduuid);
> +	return rc;
> +}
> +
> +static int lustre_stop_mgc(struct super_block *sb)
> +{
> +	struct lustre_sb_info *lsi = s2lsi(sb);
> +	struct obd_device *obd;
> +	char *niduuid = NULL, *ptr = NULL;
> +	int i, rc = 0, len = 0;
> +
> +	if (!lsi)
> +		return -ENOENT;
> +	obd = lsi->lsi_mgc;
> +	if (!obd)
> +		return -ENOENT;
> +	lsi->lsi_mgc = NULL;
> +
> +	mutex_lock(&mgc_start_lock);
> +	LASSERT(atomic_read(&obd->u.cli.cl_mgc_refcount) > 0);
> +	if (!atomic_dec_and_test(&obd->u.cli.cl_mgc_refcount)) {
> +		/* This is not fatal, every client that stops
> +		 * will call in here.
> +		 */
> +		CDEBUG(D_MOUNT, "mgc still has %d references.\n",
> +		       atomic_read(&obd->u.cli.cl_mgc_refcount));
> +		rc = -EBUSY;
> +		goto out;
> +	}
> +
> +	/* The MGC has no recoverable data in any case.
> +	 * force shutdown set in umount_begin
> +	 */
> +	obd->obd_no_recov = 1;
> +
> +	if (obd->u.cli.cl_mgc_mgsexp) {
> +		/* An error is not fatal, if we are unable to send the
> +		 * disconnect mgs ping evictor cleans up the export
> +		 */
> +		rc = obd_disconnect(obd->u.cli.cl_mgc_mgsexp);
> +		if (rc)
> +			CDEBUG(D_MOUNT, "disconnect failed %d\n", rc);
> +	}
> +
> +	/* Save the obdname for cleaning the nid uuids, which are obdname_XX */
> +	len = strlen(obd->obd_name) + 6;
> +	niduuid = kzalloc(len, GFP_NOFS);
> +	if (niduuid) {
> +		strcpy(niduuid, obd->obd_name);
> +		ptr = niduuid + strlen(niduuid);
> +	}
> +
> +	rc = class_manual_cleanup(obd);
> +	if (rc)
> +		goto out;
> +
> +	/* Clean the nid uuids */
> +	if (!niduuid) {
> +		rc = -ENOMEM;
> +		goto out;
> +	}
> +
> +	for (i = 0; i < lsi->lsi_lmd->lmd_mgs_failnodes; i++) {
> +		sprintf(ptr, "_%x", i);
> +		rc = do_lcfg(LUSTRE_MGC_OBDNAME, 0, LCFG_DEL_UUID,
> +			     niduuid, NULL, NULL, NULL);
> +		if (rc)
> +			CERROR("del MDC UUID %s failed: rc = %d\n",
> +			       niduuid, rc);
> +	}
> +out:
> +	kfree(niduuid);
> +
> +	/* class_import_put will get rid of the additional connections */
> +	mutex_unlock(&mgc_start_lock);
> +	return rc;
> +}
> +
> +/***************** lustre superblock **************/
> +
> +static struct lustre_sb_info *lustre_init_lsi(struct super_block *sb)
> +{
> +	struct lustre_sb_info *lsi;
> +
> +	lsi = kzalloc(sizeof(*lsi), GFP_NOFS);
> +	if (!lsi)
> +		return NULL;
> +	lsi->lsi_lmd = kzalloc(sizeof(*lsi->lsi_lmd), GFP_NOFS);
> +	if (!lsi->lsi_lmd) {
> +		kfree(lsi);
> +		return NULL;
> +	}
> +
> +	lsi->lsi_lmd->lmd_exclude_count = 0;
> +	lsi->lsi_lmd->lmd_recovery_time_soft = 0;
> +	lsi->lsi_lmd->lmd_recovery_time_hard = 0;
> +	s2lsi_nocast(sb) = lsi;
> +	/* we take 1 extra ref for our setup */
> +	atomic_set(&lsi->lsi_mounts, 1);
> +
> +	/* Default umount style */
> +	lsi->lsi_flags = LSI_UMOUNT_FAILOVER;
> +
> +	return lsi;
> +}
> +
> +static int lustre_free_lsi(struct super_block *sb)
> +{
> +	struct lustre_sb_info *lsi = s2lsi(sb);
> +
> +	CDEBUG(D_MOUNT, "Freeing lsi %p\n", lsi);
> +
> +	/* someone didn't call server_put_mount. */
> +	LASSERT(atomic_read(&lsi->lsi_mounts) == 0);
> +
> +	if (lsi->lsi_lmd) {
> +		kfree(lsi->lsi_lmd->lmd_dev);
> +		kfree(lsi->lsi_lmd->lmd_profile);
> +		kfree(lsi->lsi_lmd->lmd_fileset);
> +		kfree(lsi->lsi_lmd->lmd_mgssec);
> +		kfree(lsi->lsi_lmd->lmd_opts);
> +		if (lsi->lsi_lmd->lmd_exclude_count)
> +			kfree(lsi->lsi_lmd->lmd_exclude);
> +		kfree(lsi->lsi_lmd->lmd_mgs);
> +		kfree(lsi->lsi_lmd->lmd_osd_type);
> +		kfree(lsi->lsi_lmd->lmd_params);
> +
> +		kfree(lsi->lsi_lmd);
> +	}
> +
> +	LASSERT(!lsi->lsi_llsbi);
> +	kfree(lsi);
> +	s2lsi_nocast(sb) = NULL;
> +
> +	return 0;
> +}
> +
> +/* The lsi has one reference for every server that is using the disk -
> + * e.g. MDT, MGS, and potentially MGC
> + */
> +static int lustre_put_lsi(struct super_block *sb)
> +{
> +	struct lustre_sb_info *lsi = s2lsi(sb);
> +
> +	CDEBUG(D_MOUNT, "put %p %d\n", sb, atomic_read(&lsi->lsi_mounts));
> +	if (atomic_dec_and_test(&lsi->lsi_mounts)) {
> +		lustre_free_lsi(sb);
> +		return 1;
> +	}
> +	return 0;
> +}
> +
> +/*************** mount common between server and client ***************/
> +
> +/* Common umount */
> +int lustre_common_put_super(struct super_block *sb)
> +{
> +	int rc;
> +
> +	CDEBUG(D_MOUNT, "dropping sb %p\n", sb);
> +
> +	/* Drop a ref to the MGC */
> +	rc = lustre_stop_mgc(sb);
> +	if (rc && (rc != -ENOENT)) {
> +		if (rc != -EBUSY) {
> +			CERROR("Can't stop MGC: %d\n", rc);
> +			return rc;
> +		}
> +		/* BUSY just means that there's some other obd that
> +		 * needs the mgc.  Let him clean it up.
> +		 */
> +		CDEBUG(D_MOUNT, "MGC still in use\n");
> +	}
> +	/* Drop a ref to the mounted disk */
> +	lustre_put_lsi(sb);
> +	return rc;
> +}
> +
> +static void lmd_print(struct lustre_mount_data *lmd)
> +{
> +	int i;
> +
> +	PRINT_CMD(D_MOUNT, "  mount data:\n");
> +	if (lmd_is_client(lmd))
> +		PRINT_CMD(D_MOUNT, "profile: %s\n", lmd->lmd_profile);
> +	PRINT_CMD(D_MOUNT, "device:  %s\n", lmd->lmd_dev);
> +	PRINT_CMD(D_MOUNT, "flags:   %x\n", lmd->lmd_flags);
> +
> +	if (lmd->lmd_opts)
> +		PRINT_CMD(D_MOUNT, "options: %s\n", lmd->lmd_opts);
> +
> +	if (lmd->lmd_recovery_time_soft)
> +		PRINT_CMD(D_MOUNT, "recovery time soft: %d\n",
> +			  lmd->lmd_recovery_time_soft);
> +
> +	if (lmd->lmd_recovery_time_hard)
> +		PRINT_CMD(D_MOUNT, "recovery time hard: %d\n",
> +			  lmd->lmd_recovery_time_hard);
> +
> +	for (i = 0; i < lmd->lmd_exclude_count; i++) {
> +		PRINT_CMD(D_MOUNT, "exclude %d:  OST%04x\n", i,
> +			  lmd->lmd_exclude[i]);
> +	}
> +}
> +
> +/* mount -v  -o exclude=lustre-OST0001:lustre-OST0002 -t lustre ... */
> +static int lmd_make_exclusion(struct lustre_mount_data *lmd, const char *ptr)
> +{
> +	const char *s1 = ptr, *s2;
> +	__u32 index = 0, *exclude_list;
> +	int rc = 0, devmax;
> +
> +	/* The shortest an ost name can be is 8 chars: -OST0000.
> +	 * We don't actually know the fsname at this time, so in fact
> +	 * a user could specify any fsname.
> +	 */
> +	devmax = strlen(ptr) / 8 + 1;
> +
> +	/* temp storage until we figure out how many we have */
> +	exclude_list = kcalloc(devmax, sizeof(index), GFP_NOFS);
> +	if (!exclude_list)
> +		return -ENOMEM;
> +
> +	/* we enter this fn pointing at the '=' */
> +	while (*s1 && *s1 != ' ' && *s1 != ',') {
> +		s1++;
> +		rc = server_name2index(s1, &index, &s2);
> +		if (rc < 0) {
> +			CERROR("Can't parse server name '%s': rc = %d\n",
> +			       s1, rc);
> +			break;
> +		}
> +		if (rc == LDD_F_SV_TYPE_OST)
> +			exclude_list[lmd->lmd_exclude_count++] = index;
> +		else
> +			CDEBUG(D_MOUNT, "ignoring exclude %.*s: type = %#x\n",
> +			       (uint)(s2 - s1), s1, rc);
> +		s1 = s2;
> +		/* now we are pointing at ':' (next exclude)
> +		 * or ',' (end of excludes)
> +		 */
> +		if (lmd->lmd_exclude_count >= devmax)
> +			break;
> +	}
> +	if (rc >= 0) /* non-err */
> +		rc = 0;
> +
> +	if (lmd->lmd_exclude_count) {
> +		/* permanent, freed in lustre_free_lsi */
> +		lmd->lmd_exclude = kcalloc(lmd->lmd_exclude_count,
> +					   sizeof(index), GFP_NOFS);
> +		if (lmd->lmd_exclude) {
> +			memcpy(lmd->lmd_exclude, exclude_list,
> +			       sizeof(index) * lmd->lmd_exclude_count);
> +		} else {
> +			rc = -ENOMEM;
> +			lmd->lmd_exclude_count = 0;
> +		}
> +	}
> +	kfree(exclude_list);
> +	return rc;
> +}
> +
> +static int lmd_parse_mgssec(struct lustre_mount_data *lmd, char *ptr)
> +{
> +	char   *tail;
> +	int     length;
> +
> +	kfree(lmd->lmd_mgssec);
> +	lmd->lmd_mgssec = NULL;
> +
> +	tail = strchr(ptr, ',');
> +	if (!tail)
> +		length = strlen(ptr);
> +	else
> +		length = tail - ptr;
> +
> +	lmd->lmd_mgssec = kzalloc(length + 1, GFP_NOFS);
> +	if (!lmd->lmd_mgssec)
> +		return -ENOMEM;
> +
> +	memcpy(lmd->lmd_mgssec, ptr, length);
> +	lmd->lmd_mgssec[length] = '\0';
> +	return 0;
> +}
> +
> +static int lmd_parse_string(char **handle, char *ptr)
> +{
> +	char   *tail;
> +	int     length;
> +
> +	if (!handle || !ptr)
> +		return -EINVAL;
> +
> +	kfree(*handle);
> +	*handle = NULL;
> +
> +	tail = strchr(ptr, ',');
> +	if (!tail)
> +		length = strlen(ptr);
> +	else
> +		length = tail - ptr;
> +
> +	*handle = kzalloc(length + 1, GFP_NOFS);
> +	if (!*handle)
> +		return -ENOMEM;
> +
> +	memcpy(*handle, ptr, length);
> +	(*handle)[length] = '\0';
> +
> +	return 0;
> +}
> +
> +/* Collect multiple values for mgsnid specifiers */
> +static int lmd_parse_mgs(struct lustre_mount_data *lmd, char **ptr)
> +{
> +	lnet_nid_t nid;
> +	char *tail = *ptr;
> +	char *mgsnid;
> +	int   length;
> +	int   oldlen = 0;
> +
> +	/* Find end of nidlist */
> +	while (class_parse_nid_quiet(tail, &nid, &tail) == 0)
> +		;
> +	length = tail - *ptr;
> +	if (length == 0) {
> +		LCONSOLE_ERROR_MSG(0x159, "Can't parse NID '%s'\n", *ptr);
> +		return -EINVAL;
> +	}
> +
> +	if (lmd->lmd_mgs)
> +		oldlen = strlen(lmd->lmd_mgs) + 1;
> +
> +	mgsnid = kzalloc(oldlen + length + 1, GFP_NOFS);
> +	if (!mgsnid)
> +		return -ENOMEM;
> +
> +	if (lmd->lmd_mgs) {
> +		/* Multiple mgsnid= are taken to mean failover locations */
> +		memcpy(mgsnid, lmd->lmd_mgs, oldlen);
> +		mgsnid[oldlen - 1] = ':';
> +		kfree(lmd->lmd_mgs);
> +	}
> +	memcpy(mgsnid + oldlen, *ptr, length);
> +	mgsnid[oldlen + length] = '\0';
> +	lmd->lmd_mgs = mgsnid;
> +	*ptr = tail;
> +
> +	return 0;
> +}
> +
> +/**
> + * Find the first delimiter (comma or colon) from the specified \a buf and
> + * make \a *endh point to the string starting with the delimiter. The commas
> + * in expression list [...] will be skipped.
> + *
> + * @buf		a delimiter-separated string
> + * @endh	a pointer to a pointer that will point to the string
> + *		starting with the delimiter
> + *
> + * RETURNS	true if delimiter is found, false if delimiter is not found
> + */
> +static bool lmd_find_delimiter(char *buf, char **endh)
> +{
> +	char *c = buf;
> +	size_t pos;
> +	bool found;
> +
> +	if (!buf)
> +		return false;
> +try_again:
> +	if (*c == ',' || *c == ':')
> +		return true;
> +
> +	pos = strcspn(c, "[:,]");
> +	if (!pos)
> +		return false;
> +
> +	/* Not a valid mount string */
> +	if (*c == ']') {
> +		CWARN("invalid mount string format\n");
> +		return false;
> +	}
> +
> +	c += pos;
> +	if (*c == '[') {
> +		c = strchr(c, ']');
> +
> +		/* invalid mount string */
> +		if (!c) {
> +			CWARN("invalid mount string format\n");
> +			return false;
> +		}
> +		c++;
> +		goto try_again;
> +	}
> +
> +	found = *c != '\0';
> +	if (found && endh)
> +		*endh = c;
> +
> +	return found;
> +}
> +
> +/**
> + * Find the first valid string delimited by comma or colon from the specified
> + * \a buf and parse it to see whether it's a valid nid list. If yes, \a *endh
> + * will point to the next string starting with the delimiter.
> + *
> + * \param[in] buf	a delimiter-separated string
> + * \param[in] endh	a pointer to a pointer that will point to the string
> + *			starting with the delimiter
> + *
> + * \retval 0		if the string is a valid nid list
> + * \retval 1		if the string is not a valid nid list
> + */
> +static int lmd_parse_nidlist(char *buf, char **endh)
> +{
> +	struct list_head nidlist;
> +	char *endp = buf;
> +	int rc = 0;
> +	char tmp;
> +
> +	if (!buf)
> +		return 1;
> +	while (*buf == ',' || *buf == ':')
> +		buf++;
> +	if (*buf == ' ' || *buf == '/' || *buf == '\0')
> +		return 1;
> +
> +	if (!lmd_find_delimiter(buf, &endp))
> +		endp = buf + strlen(buf);
> +
> +	tmp = *endp;
> +	*endp = '\0';
> +
> +	INIT_LIST_HEAD(&nidlist);
> +	if (cfs_parse_nidlist(buf, strlen(buf), &nidlist) <= 0)
> +		rc = 1;
> +	cfs_free_nidlist(&nidlist);
> +
> +	*endp = tmp;
> +	if (rc)
> +		return rc;
> +	if (endh)
> +		*endh = endp;
> +	return 0;
> +}
> +
> +/** Parse mount line options
> + * e.g. mount -v -t lustre -o abort_recov uml1:uml2:/lustre-client /mnt/lustre
> + * dev is passed as device=uml1:/lustre by mount.lustre
> + */
> +static int lmd_parse(char *options, struct lustre_mount_data *lmd)
> +{
> +	char *s1, *s2, *devname = NULL;
> +	struct lustre_mount_data *raw = (struct lustre_mount_data *)options;
> +	int rc = 0;
> +
> +	LASSERT(lmd);
> +	if (!options) {
> +		LCONSOLE_ERROR_MSG(0x162, "Missing mount data: check that /sbin/mount.lustre is installed.\n");
> +		return -EINVAL;
> +	}
> +
> +	/* Options should be a string - try to detect old lmd data */
> +	if ((raw->lmd_magic & 0xffffff00) == (LMD_MAGIC & 0xffffff00)) {
> +		LCONSOLE_ERROR_MSG(0x163, "You're using an old version of /sbin/mount.lustre.  Please install version %s\n",
> +				   LUSTRE_VERSION_STRING);
> +		return -EINVAL;
> +	}
> +	lmd->lmd_magic = LMD_MAGIC;
> +
> +	lmd->lmd_params = kzalloc(LMD_PARAMS_MAXLEN, GFP_NOFS);
> +	if (!lmd->lmd_params)
> +		return -ENOMEM;
> +	lmd->lmd_params[0] = '\0';
> +
> +	/* Set default flags here */
> +
> +	s1 = options;
> +	while (*s1) {
> +		int clear = 0;
> +		int time_min = OBD_RECOVERY_TIME_MIN;
> +		char *s3;
> +
> +		/* Skip whitespace and extra commas */
> +		while (*s1 == ' ' || *s1 == ',')
> +			s1++;
> +		s3 = s1;
> +
> +		/* Client options are parsed in ll_options: eg. flock,
> +		 * user_xattr, acl
> +		 */
> +
> +		/* Parse non-ldiskfs options here. Rather than modifying
> +		 * ldiskfs, we just zero these out here
> +		 */
> +		if (strncmp(s1, "abort_recov", 11) == 0) {
> +			lmd->lmd_flags |= LMD_FLG_ABORT_RECOV;
> +			clear++;
> +		} else if (strncmp(s1, "recovery_time_soft=", 19) == 0) {
> +			lmd->lmd_recovery_time_soft = max_t(int,
> +				simple_strtoul(s1 + 19, NULL, 10), time_min);
> +			clear++;
> +		} else if (strncmp(s1, "recovery_time_hard=", 19) == 0) {
> +			lmd->lmd_recovery_time_hard = max_t(int,
> +				simple_strtoul(s1 + 19, NULL, 10), time_min);
> +			clear++;
> +		} else if (strncmp(s1, "noir", 4) == 0) {
> +			lmd->lmd_flags |= LMD_FLG_NOIR; /* test purpose only. */
> +			clear++;
> +		} else if (strncmp(s1, "nosvc", 5) == 0) {
> +			lmd->lmd_flags |= LMD_FLG_NOSVC;
> +			clear++;
> +		} else if (strncmp(s1, "nomgs", 5) == 0) {
> +			lmd->lmd_flags |= LMD_FLG_NOMGS;
> +			clear++;
> +		} else if (strncmp(s1, "noscrub", 7) == 0) {
> +			lmd->lmd_flags |= LMD_FLG_NOSCRUB;
> +			clear++;
> +		} else if (strncmp(s1, PARAM_MGSNODE,
> +				   sizeof(PARAM_MGSNODE) - 1) == 0) {
> +			s2 = s1 + sizeof(PARAM_MGSNODE) - 1;
> +			/* Assume the next mount opt is the first
> +			 * invalid nid we get to.
> +			 */
> +			rc = lmd_parse_mgs(lmd, &s2);
> +			if (rc)
> +				goto invalid;
> +			clear++;
> +		} else if (strncmp(s1, "writeconf", 9) == 0) {
> +			lmd->lmd_flags |= LMD_FLG_WRITECONF;
> +			clear++;
> +		} else if (strncmp(s1, "update", 6) == 0) {
> +			lmd->lmd_flags |= LMD_FLG_UPDATE;
> +			clear++;
> +		} else if (strncmp(s1, "virgin", 6) == 0) {
> +			lmd->lmd_flags |= LMD_FLG_VIRGIN;
> +			clear++;
> +		} else if (strncmp(s1, "noprimnode", 10) == 0) {
> +			lmd->lmd_flags |= LMD_FLG_NO_PRIMNODE;
> +			clear++;
> +		} else if (strncmp(s1, "mgssec=", 7) == 0) {
> +			rc = lmd_parse_mgssec(lmd, s1 + 7);
> +			if (rc)
> +				goto invalid;
> +			s3 = s2;
> +			clear++;
> +		/* ost exclusion list */
> +		} else if (strncmp(s1, "exclude=", 8) == 0) {
> +			rc = lmd_make_exclusion(lmd, s1 + 7);
> +			if (rc)
> +				goto invalid;
> +			clear++;
> +		} else if (strncmp(s1, "mgs", 3) == 0) {
> +			/* We are an MGS */
> +			lmd->lmd_flags |= LMD_FLG_MGS;
> +			clear++;
> +		} else if (strncmp(s1, "svname=", 7) == 0) {
> +			rc = lmd_parse_string(&lmd->lmd_profile, s1 + 7);
> +			if (rc)
> +				goto invalid;
> +			clear++;
> +		} else if (strncmp(s1, "param=", 6) == 0) {
> +			size_t length, params_length;
> +			char *tail = s1;
> +
> +			if (lmd_find_delimiter(s1 + 6, &tail)) {
> +				char *param_str = tail + 1;
> +				int supplementary = 1;
> +
> +				while (!lmd_parse_nidlist(param_str,
> +							  &param_str))
> +					supplementary = 0;
> +				length = param_str - s1 - supplementary;
> +			} else {
> +				length = strlen(s1);
> +			}
> +			length -= 6;
> +			params_length = strlen(lmd->lmd_params);
> +			if (params_length + length + 1 >= LMD_PARAMS_MAXLEN)
> +				return -E2BIG;
> +			strncat(lmd->lmd_params, s1 + 6, length);
> +			lmd->lmd_params[params_length + length] = '\0';
> +			strlcat(lmd->lmd_params, " ", LMD_PARAMS_MAXLEN);
> +			s3 = s1 + 6 + length;
> +			clear++;
> +		} else if (strncmp(s1, "osd=", 4) == 0) {
> +			rc = lmd_parse_string(&lmd->lmd_osd_type, s1 + 4);
> +			if (rc)
> +				goto invalid;
> +			clear++;
> +		}
> +		/* Linux 2.4 doesn't pass the device, so we stuck it at the
> +		 * end of the options.
> +		 */
> +		else if (strncmp(s1, "device=", 7) == 0) {
> +			devname = s1 + 7;
> +			/* terminate options right before device.  device
> +			 * must be the last one.
> +			 */
> +			*s1 = '\0';
> +			break;
> +		}
> +
> +		/* Find next opt */
> +		s2 = strchr(s1, ',');
> +		if (!s2) {
> +			if (clear)
> +				*s1 = '\0';
> +			break;
> +		}
> +		s2++;
> +		if (clear)
> +			memmove(s1, s2, strlen(s2) + 1);
> +		else
> +			s1 = s2;
> +	}
> +
> +	if (!devname) {
> +		LCONSOLE_ERROR_MSG(0x164, "Can't find the device name (need mount option 'device=...')\n");
> +		goto invalid;
> +	}
> +
> +	s1 = strstr(devname, ":/");
> +	if (s1) {
> +		++s1;
> +		lmd->lmd_flags |= LMD_FLG_CLIENT;
> +		/* Remove leading /s from fsname */
> +		while (*++s1 == '/')
> +			;
> +		s2 = strchrnul(s1, '/');
> +		/* Freed in lustre_free_lsi */
> +		lmd->lmd_profile = kasprintf(GFP_KERNEL, "%.*s-client",
> +					     (int)(s2 - s1), s1);
> +		if (!lmd->lmd_profile)
> +			return -ENOMEM;
> +
> +		s1 = s2;
> +		s2 = s1 + strlen(s1) - 1;
> +		/* Remove padding /s from fileset */
> +		while (*s2 == '/')
> +			s2--;
> +		if (s2 > s1) {
> +			lmd->lmd_fileset = kstrndup(s1, s2 - s1 + 1,
> +						    GFP_KERNEL);
> +			if (!lmd->lmd_fileset)
> +				return -ENOMEM;
> +		}
> +	}
> +
> +	/* Freed in lustre_free_lsi */
> +	lmd->lmd_dev = kstrdup(devname, GFP_KERNEL);
> +	if (!lmd->lmd_dev)
> +		return -ENOMEM;
> +
> +	/* Save mount options */
> +	s1 = options + strlen(options) - 1;
> +	while (s1 >= options && (*s1 == ',' || *s1 == ' '))
> +		*s1-- = 0;
> +	if (*options != 0) {
> +		/* Freed in lustre_free_lsi */
> +		lmd->lmd_opts = kstrdup(options, GFP_KERNEL);
> +		if (!lmd->lmd_opts)
> +			return -ENOMEM;
> +	}
> +
> +	lmd_print(lmd);
> +	lmd->lmd_magic = LMD_MAGIC;
> +
> +	return rc;
> +
> +invalid:
> +	CERROR("Bad mount options %s\n", options);
> +	return -EINVAL;
> +}
> +
> +/** This is the entry point for the mount call into Lustre.
> + * This is called when a server or client is mounted,
> + * and this is where we start setting things up.
> + * @param data Mount options (e.g. -o flock,abort_recov)
> + */
> +static int lustre_fill_super(struct super_block *sb, void *lmd2_data, int silent)
> +{
> +	struct lustre_mount_data *lmd;
> +	struct lustre_sb_info *lsi;
> +	int rc;
> +
> +	CDEBUG(D_MOUNT | D_VFSTRACE, "VFS Op: sb %p\n", sb);
> +
> +	lsi = lustre_init_lsi(sb);
> +	if (!lsi)
> +		return -ENOMEM;
> +	lmd = lsi->lsi_lmd;
> +
> +	/*
> +	 * Disable lockdep during mount, because mount locking patterns are
> +	 * `special'.
> +	 */
> +	lockdep_off();
> +
> +	/*
> +	 * LU-639: the obd cleanup of last mount may not finish yet, wait here.
> +	 */
> +	obd_zombie_barrier();
> +
> +	/* Figure out the lmd from the mount options */
> +	if (lmd_parse(lmd2_data, lmd)) {
> +		lustre_put_lsi(sb);
> +		rc = -EINVAL;
> +		goto out;
> +	}
> +
> +	if (lmd_is_client(lmd)) {
> +		CDEBUG(D_MOUNT, "Mounting client %s\n", lmd->lmd_profile);
> +
> +		rc = lustre_start_mgc(sb);
> +		if (rc) {
> +			lustre_common_put_super(sb);
> +			goto out;
> +		}
> +		/* Connect and start */
> +		rc = ll_fill_super(sb);
> +		/* l_f_s will call lustre_common_put_super on failure, otherwise
> +		 * l_f_s will have taken another reference to the module */
> +	} else {
> +		CERROR("This is client-side-only module, cannot handle server mount.\n");
> +		rc = -EINVAL;
> +	}
> +
> +	/* If error happens in fill_super() call, @lsi will be killed there.
> +	 * This is why we do not put it here.
> +	 */
> +	goto out;
> +out:
> +	if (rc) {
> +		CERROR("Unable to mount %s (%d)\n",
> +		       s2lsi(sb) ? lmd->lmd_dev : "", rc);
> +	} else {
> +		CDEBUG(D_SUPER, "Mount %s complete\n",
> +		       lmd->lmd_dev);
> +	}
> +	lockdep_on();
> +	return rc;
> +}
> +
> +/***************** FS registration ******************/
> +static struct dentry *lustre_mount(struct file_system_type *fs_type, int flags,
> +				   const char *devname, void *data)
> +{
> +	return mount_nodev(fs_type, flags, data, lustre_fill_super);
> +}
> +
> +static void lustre_kill_super(struct super_block *sb)
> +{
> +	struct lustre_sb_info *lsi = s2lsi(sb);
> +
> +	if (lsi)
> +		ll_kill_super(sb);
> +
> +	kill_anon_super(sb);
> +}
> +
> +/** Register the "lustre" fs type
> + */
> +static struct file_system_type lustre_fs_type = {
> +	.owner		= THIS_MODULE,
> +	.name		= "lustre",
> +	.mount		= lustre_mount,
> +	.kill_sb	= lustre_kill_super,
> +	.fs_flags	= FS_RENAME_DOES_D_MOVE,
> +};
> +MODULE_ALIAS_FS("lustre");
> +
> +int lustre_register_fs(void)
> +{
> +	return register_filesystem(&lustre_fs_type);
> +}
> +
> +int lustre_unregister_fs(void)
> +{
> +	return unregister_filesystem(&lustre_fs_type);
> +}
> diff --git a/drivers/staging/lustre/lustre/llite/super25.c b/drivers/staging/lustre/lustre/llite/super25.c
> index d335f29556c2..cb1e12e8097a 100644
> --- a/drivers/staging/lustre/lustre/llite/super25.c
> +++ b/drivers/staging/lustre/lustre/llite/super25.c
> @@ -145,10 +145,11 @@ static int __init lustre_init(void)
>  	if (rc != 0)
>  		goto out_inode_fini_env;
>  
> -	lustre_register_super_ops(THIS_MODULE, ll_fill_super, ll_kill_super);
>  	lustre_register_client_process_config(ll_process_config);
>  
> -	return 0;
> +	rc = lustre_register_fs();
> +
> +	return rc;
>  
>  out_inode_fini_env:
>  	cl_env_put(cl_inode_fini_env, &cl_inode_fini_refcheck);
> @@ -166,7 +167,8 @@ static int __init lustre_init(void)
>  
>  static void __exit lustre_exit(void)
>  {
> -	lustre_register_super_ops(NULL, NULL, NULL);
> +	lustre_unregister_fs();
> +
>  	lustre_register_client_process_config(NULL);
>  
>  	debugfs_remove(llite_root);
> diff --git a/drivers/staging/lustre/lustre/obdclass/Makefile b/drivers/staging/lustre/lustre/obdclass/Makefile
> index 686bd546f06d..2b99514a4966 100644
> --- a/drivers/staging/lustre/lustre/obdclass/Makefile
> +++ b/drivers/staging/lustre/lustre/obdclass/Makefile
> @@ -8,5 +8,5 @@ obdclass-y := module.o sysctl.o \
>  	      llog.o llog_cat.o llog_obd.o llog_swab.o class_obd.o debug.o \
>  	      genops.o uuid.o lprocfs_status.o lprocfs_counters.o \
>  	      lustre_handles.o lustre_peer.o statfs_pack.o linkea.o \
> -	      obdo.o obd_config.o obd_mount.o lu_object.o lu_ref.o \
> +	      obdo.o obd_config.o lu_object.o lu_ref.o \
>  	      cl_object.o cl_page.o cl_lock.o cl_io.o kernelcomm.o
> diff --git a/drivers/staging/lustre/lustre/obdclass/class_obd.c b/drivers/staging/lustre/lustre/obdclass/class_obd.c
> index 81a4c666bb69..cdaf7293bc18 100644
> --- a/drivers/staging/lustre/lustre/obdclass/class_obd.c
> +++ b/drivers/staging/lustre/lustre/obdclass/class_obd.c
> @@ -510,18 +510,12 @@ static int __init obdclass_init(void)
>  		return err;
>  
>  	err = llog_info_init();
> -	if (err)
> -		return err;
> -
> -	err = lustre_register_fs();
>  
>  	return err;
>  }
>  
>  static void obdclass_exit(void)
>  {
> -	lustre_unregister_fs();
> -
>  	misc_deregister(&obd_psdev);
>  	llog_info_fini();
>  	cl_global_fini();
> diff --git a/drivers/staging/lustre/lustre/obdclass/obd_mount.c b/drivers/staging/lustre/lustre/obdclass/obd_mount.c
> deleted file mode 100644
> index 4e4a8c35105b..000000000000
> --- a/drivers/staging/lustre/lustre/obdclass/obd_mount.c
> +++ /dev/null
> @@ -1,1244 +0,0 @@
> -// SPDX-License-Identifier: GPL-2.0
> -/*
> - * GPL HEADER START
> - *
> - * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
> - *
> - * This program is free software; you can redistribute it and/or modify
> - * it under the terms of the GNU General Public License version 2 only,
> - * as published by the Free Software Foundation.
> - *
> - * This program is distributed in the hope that it will be useful, but
> - * WITHOUT ANY WARRANTY; without even the implied warranty of
> - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> - * General Public License version 2 for more details (a copy is included
> - * in the LICENSE file that accompanied this code).
> - *
> - * You should have received a copy of the GNU General Public License
> - * version 2 along with this program; If not, see
> - * http://www.gnu.org/licenses/gpl-2.0.html
> - *
> - * GPL HEADER END
> - */
> -/*
> - * Copyright (c) 2007, 2010, Oracle and/or its affiliates. All rights reserved.
> - * Use is subject to license terms.
> - *
> - * Copyright (c) 2011, 2015, Intel Corporation.
> - */
> -/*
> - * This file is part of Lustre, http://www.lustre.org/
> - * Lustre is a trademark of Sun Microsystems, Inc.
> - *
> - * lustre/obdclass/obd_mount.c
> - *
> - * Client mount routines
> - *
> - * Author: Nathan Rutman <nathan at clusterfs.com>
> - */
> -
> -#define DEBUG_SUBSYSTEM S_CLASS
> -#define D_MOUNT (D_SUPER | D_CONFIG/*|D_WARNING */)
> -#define PRINT_CMD CDEBUG
> -
> -#include <linux/random.h>
> -#include <obd.h>
> -#include <obd_class.h>
> -#include <uapi/linux/lustre/lustre_idl.h>
> -#include <lustre_log.h>
> -#include <lustre_disk.h>
> -#include <uapi/linux/lustre/lustre_param.h>
> -
> -static DEFINE_SPINLOCK(client_lock);
> -static struct module *client_mod;
> -static int (*client_fill_super)(struct super_block *sb);
> -static void (*kill_super_cb)(struct super_block *sb);
> -
> -/**************** config llog ********************/
> -
> -/** Get a config log from the MGS and process it.
> - * This func is called for both clients and servers.
> - * Continue to process new statements appended to the logs
> - * (whenever the config lock is revoked) until lustre_end_log
> - * is called.
> - * @param sb The superblock is used by the MGC to write to the local copy of
> - *   the config log
> - * @param logname The name of the llog to replicate from the MGS
> - * @param cfg Since the same mgc may be used to follow multiple config logs
> - *   (e.g. ost1, ost2, client), the config_llog_instance keeps the state for
> - *   this log, and is added to the mgc's list of logs to follow.
> - */
> -int lustre_process_log(struct super_block *sb, char *logname,
> -		       struct config_llog_instance *cfg)
> -{
> -	struct lustre_cfg *lcfg;
> -	struct lustre_cfg_bufs *bufs;
> -	struct lustre_sb_info *lsi = s2lsi(sb);
> -	struct obd_device *mgc = lsi->lsi_mgc;
> -	int rc;
> -
> -	LASSERT(mgc);
> -	LASSERT(cfg);
> -
> -	bufs = kzalloc(sizeof(*bufs), GFP_NOFS);
> -	if (!bufs)
> -		return -ENOMEM;
> -
> -	/* mgc_process_config */
> -	lustre_cfg_bufs_reset(bufs, mgc->obd_name);
> -	lustre_cfg_bufs_set_string(bufs, 1, logname);
> -	lustre_cfg_bufs_set(bufs, 2, cfg, sizeof(*cfg));
> -	lustre_cfg_bufs_set(bufs, 3, &sb, sizeof(sb));
> -	lcfg = kzalloc(lustre_cfg_len(bufs->lcfg_bufcount, bufs->lcfg_buflen),
> -		       GFP_NOFS);
> -	if (!lcfg) {
> -		rc = -ENOMEM;
> -		goto out;
> -	}
> -	lustre_cfg_init(lcfg, LCFG_LOG_START, bufs);
> -
> -	rc = obd_process_config(mgc, sizeof(*lcfg), lcfg);
> -	kfree(lcfg);
> -out:
> -	kfree(bufs);
> -
> -	if (rc == -EINVAL)
> -		LCONSOLE_ERROR_MSG(0x15b, "%s: The configuration from log '%s' failed from the MGS (%d).  Make sure this client and the MGS are running compatible versions of Lustre.\n",
> -				   mgc->obd_name, logname, rc);
> -
> -	else if (rc)
> -		LCONSOLE_ERROR_MSG(0x15c, "%s: The configuration from log '%s' failed (%d). This may be the result of communication errors between this node and the MGS, a bad configuration, or other errors. See the syslog for more information.\n",
> -				   mgc->obd_name, logname,
> -				   rc);
> -
> -	/* class_obd_list(); */
> -	return rc;
> -}
> -EXPORT_SYMBOL(lustre_process_log);
> -
> -/* Stop watching this config log for updates */
> -int lustre_end_log(struct super_block *sb, char *logname,
> -		   struct config_llog_instance *cfg)
> -{
> -	struct lustre_cfg *lcfg;
> -	struct lustre_cfg_bufs bufs;
> -	struct lustre_sb_info *lsi = s2lsi(sb);
> -	struct obd_device *mgc = lsi->lsi_mgc;
> -	int rc;
> -
> -	if (!mgc)
> -		return -ENOENT;
> -
> -	/* mgc_process_config */
> -	lustre_cfg_bufs_reset(&bufs, mgc->obd_name);
> -	lustre_cfg_bufs_set_string(&bufs, 1, logname);
> -	if (cfg)
> -		lustre_cfg_bufs_set(&bufs, 2, cfg, sizeof(*cfg));
> -	lcfg = kzalloc(lustre_cfg_len(bufs.lcfg_bufcount, bufs.lcfg_buflen),
> -		       GFP_NOFS);
> -	if (!lcfg)
> -		return -ENOMEM;
> -	lustre_cfg_init(lcfg, LCFG_LOG_END, &bufs);
> -
> -	rc = obd_process_config(mgc, sizeof(*lcfg), lcfg);
> -	kfree(lcfg);
> -	return rc;
> -}
> -EXPORT_SYMBOL(lustre_end_log);
> -
> -/**************** obd start *******************/
> -
> -/** lustre_cfg_bufs are a holdover from 1.4; we can still set these up from
> - * lctl (and do for echo cli/srv.
> - */
> -static int do_lcfg(char *cfgname, lnet_nid_t nid, int cmd,
> -		   char *s1, char *s2, char *s3, char *s4)
> -{
> -	struct lustre_cfg_bufs bufs;
> -	struct lustre_cfg     *lcfg = NULL;
> -	int rc;
> -
> -	CDEBUG(D_TRACE, "lcfg %s %#x %s %s %s %s\n", cfgname,
> -	       cmd, s1, s2, s3, s4);
> -
> -	lustre_cfg_bufs_reset(&bufs, cfgname);
> -	if (s1)
> -		lustre_cfg_bufs_set_string(&bufs, 1, s1);
> -	if (s2)
> -		lustre_cfg_bufs_set_string(&bufs, 2, s2);
> -	if (s3)
> -		lustre_cfg_bufs_set_string(&bufs, 3, s3);
> -	if (s4)
> -		lustre_cfg_bufs_set_string(&bufs, 4, s4);
> -
> -	lcfg = kzalloc(lustre_cfg_len(bufs.lcfg_bufcount, bufs.lcfg_buflen),
> -		       GFP_NOFS);
> -	if (!lcfg)
> -		return -ENOMEM;
> -	lustre_cfg_init(lcfg, cmd, &bufs);
> -	lcfg->lcfg_nid = nid;
> -	rc = class_process_config(lcfg);
> -	kfree(lcfg);
> -	return rc;
> -}
> -
> -/** Call class_attach and class_setup.  These methods in turn call
> - * obd type-specific methods.
> - */
> -static int lustre_start_simple(char *obdname, char *type, char *uuid,
> -			       char *s1, char *s2, char *s3, char *s4)
> -{
> -	int rc;
> -
> -	CDEBUG(D_MOUNT, "Starting obd %s (typ=%s)\n", obdname, type);
> -
> -	rc = do_lcfg(obdname, 0, LCFG_ATTACH, type, uuid, NULL, NULL);
> -	if (rc) {
> -		CERROR("%s attach error %d\n", obdname, rc);
> -		return rc;
> -	}
> -	rc = do_lcfg(obdname, 0, LCFG_SETUP, s1, s2, s3, s4);
> -	if (rc) {
> -		CERROR("%s setup error %d\n", obdname, rc);
> -		do_lcfg(obdname, 0, LCFG_DETACH, NULL, NULL, NULL, NULL);
> -	}
> -	return rc;
> -}
> -
> -static DEFINE_MUTEX(mgc_start_lock);
> -
> -/** Set up a mgc obd to process startup logs
> - *
> - * \param sb [in] super block of the mgc obd
> - *
> - * \retval 0 success, otherwise error code
> - */
> -int lustre_start_mgc(struct super_block *sb)
> -{
> -	struct obd_connect_data *data = NULL;
> -	struct lustre_sb_info *lsi = s2lsi(sb);
> -	struct obd_device *obd;
> -	struct obd_export *exp;
> -	struct obd_uuid *uuid;
> -	class_uuid_t uuidc;
> -	lnet_nid_t nid;
> -	char nidstr[LNET_NIDSTR_SIZE];
> -	char *mgcname = NULL, *niduuid = NULL, *mgssec = NULL;
> -	char *ptr;
> -	int rc = 0, i = 0, j;
> -
> -	LASSERT(lsi->lsi_lmd);
> -
> -	/* Use nids from mount line: uml1,1 at elan:uml2,2 at elan:/lustre */
> -	ptr = lsi->lsi_lmd->lmd_dev;
> -	if (class_parse_nid(ptr, &nid, &ptr) == 0)
> -		i++;
> -	if (i == 0) {
> -		CERROR("No valid MGS nids found.\n");
> -		return -EINVAL;
> -	}
> -
> -	mutex_lock(&mgc_start_lock);
> -
> -	libcfs_nid2str_r(nid, nidstr, sizeof(nidstr));
> -	mgcname = kasprintf(GFP_NOFS,
> -			    "%s%s", LUSTRE_MGC_OBDNAME, nidstr);
> -	niduuid = kasprintf(GFP_NOFS, "%s_%x", mgcname, 0);
> -	if (!mgcname || !niduuid) {
> -		rc = -ENOMEM;
> -		goto out_free;
> -	}
> -
> -	mgssec = lsi->lsi_lmd->lmd_mgssec ? lsi->lsi_lmd->lmd_mgssec : "";
> -
> -	data = kzalloc(sizeof(*data), GFP_NOFS);
> -	if (!data) {
> -		rc = -ENOMEM;
> -		goto out_free;
> -	}
> -
> -	obd = class_name2obd(mgcname);
> -	if (obd && !obd->obd_stopping) {
> -		int recov_bk;
> -
> -		rc = obd_set_info_async(NULL, obd->obd_self_export,
> -					strlen(KEY_MGSSEC), KEY_MGSSEC,
> -					strlen(mgssec), mgssec, NULL);
> -		if (rc)
> -			goto out_free;
> -
> -		/* Re-using an existing MGC */
> -		atomic_inc(&obd->u.cli.cl_mgc_refcount);
> -
> -		/* IR compatibility check, only for clients */
> -		if (lmd_is_client(lsi->lsi_lmd)) {
> -			int has_ir;
> -			int vallen = sizeof(*data);
> -			__u32 *flags = &lsi->lsi_lmd->lmd_flags;
> -
> -			rc = obd_get_info(NULL, obd->obd_self_export,
> -					  strlen(KEY_CONN_DATA), KEY_CONN_DATA,
> -					  &vallen, data);
> -			LASSERT(rc == 0);
> -			has_ir = OCD_HAS_FLAG(data, IMP_RECOV);
> -			if (has_ir ^ !(*flags & LMD_FLG_NOIR)) {
> -				/* LMD_FLG_NOIR is for test purpose only */
> -				LCONSOLE_WARN(
> -					"Trying to mount a client with IR setting not compatible with current mgc. Force to use current mgc setting that is IR %s.\n",
> -					has_ir ? "enabled" : "disabled");
> -				if (has_ir)
> -					*flags &= ~LMD_FLG_NOIR;
> -				else
> -					*flags |= LMD_FLG_NOIR;
> -			}
> -		}
> -
> -		recov_bk = 0;
> -
> -		/* Try all connections, but only once (again).
> -		 * We don't want to block another target from starting
> -		 * (using its local copy of the log), but we do want to connect
> -		 * if at all possible.
> -		 */
> -		recov_bk++;
> -		CDEBUG(D_MOUNT, "%s: Set MGC reconnect %d\n", mgcname,
> -		       recov_bk);
> -		rc = obd_set_info_async(NULL, obd->obd_self_export,
> -					sizeof(KEY_INIT_RECOV_BACKUP),
> -					KEY_INIT_RECOV_BACKUP,
> -					sizeof(recov_bk), &recov_bk, NULL);
> -		rc = 0;
> -		goto out;
> -	}
> -
> -	CDEBUG(D_MOUNT, "Start MGC '%s'\n", mgcname);
> -
> -	/* Add the primary nids for the MGS */
> -	i = 0;
> -	/* Use nids from mount line: uml1,1 at elan:uml2,2 at elan:/lustre */
> -	ptr = lsi->lsi_lmd->lmd_dev;
> -	while (class_parse_nid(ptr, &nid, &ptr) == 0) {
> -		rc = do_lcfg(mgcname, nid,
> -			     LCFG_ADD_UUID, niduuid, NULL, NULL, NULL);
> -		if (!rc)
> -			i++;
> -		/* Stop at the first failover nid */
> -		if (*ptr == ':')
> -			break;
> -	}
> -	if (i == 0) {
> -		CERROR("No valid MGS nids found.\n");
> -		rc = -EINVAL;
> -		goto out_free;
> -	}
> -	lsi->lsi_lmd->lmd_mgs_failnodes = 1;
> -
> -	/* Random uuid for MGC allows easier reconnects */
> -	uuid = kzalloc(sizeof(*uuid), GFP_NOFS);
> -	if (!uuid) {
> -		rc = -ENOMEM;
> -		goto out_free;
> -	}
> -
> -	ll_generate_random_uuid(uuidc);
> -	class_uuid_unparse(uuidc, uuid);
> -
> -	/* Start the MGC */
> -	rc = lustre_start_simple(mgcname, LUSTRE_MGC_NAME,
> -				 (char *)uuid->uuid, LUSTRE_MGS_OBDNAME,
> -				 niduuid, NULL, NULL);
> -	kfree(uuid);
> -	if (rc)
> -		goto out_free;
> -
> -	/* Add any failover MGS nids */
> -	i = 1;
> -	while (ptr && ((*ptr == ':' ||
> -			class_find_param(ptr, PARAM_MGSNODE, &ptr) == 0))) {
> -		/* New failover node */
> -		sprintf(niduuid, "%s_%x", mgcname, i);
> -		j = 0;
> -		while (class_parse_nid_quiet(ptr, &nid, &ptr) == 0) {
> -			rc = do_lcfg(mgcname, nid, LCFG_ADD_UUID, niduuid,
> -				     NULL, NULL, NULL);
> -			if (!rc)
> -				++j;
> -			if (*ptr == ':')
> -				break;
> -		}
> -		if (j > 0) {
> -			rc = do_lcfg(mgcname, 0, LCFG_ADD_CONN,
> -				     niduuid, NULL, NULL, NULL);
> -			if (!rc)
> -				i++;
> -		} else {
> -			/* at ":/fsname" */
> -			break;
> -		}
> -	}
> -	lsi->lsi_lmd->lmd_mgs_failnodes = i;
> -
> -	obd = class_name2obd(mgcname);
> -	if (!obd) {
> -		CERROR("Can't find mgcobd %s\n", mgcname);
> -		rc = -ENOTCONN;
> -		goto out_free;
> -	}
> -
> -	rc = obd_set_info_async(NULL, obd->obd_self_export,
> -				strlen(KEY_MGSSEC), KEY_MGSSEC,
> -				strlen(mgssec), mgssec, NULL);
> -	if (rc)
> -		goto out_free;
> -
> -	/* Keep a refcount of servers/clients who started with "mount",
> -	 * so we know when we can get rid of the mgc.
> -	 */
> -	atomic_set(&obd->u.cli.cl_mgc_refcount, 1);
> -
> -	/* We connect to the MGS at setup, and don't disconnect until cleanup */
> -	data->ocd_connect_flags = OBD_CONNECT_VERSION | OBD_CONNECT_AT |
> -				  OBD_CONNECT_FULL20 | OBD_CONNECT_IMP_RECOV |
> -				  OBD_CONNECT_LVB_TYPE | OBD_CONNECT_BULK_MBITS;
> -
> -#if OBD_OCD_VERSION(3, 0, 53, 0) > LUSTRE_VERSION_CODE
> -	data->ocd_connect_flags |= OBD_CONNECT_MNE_SWAB;
> -#endif
> -
> -	if (lmd_is_client(lsi->lsi_lmd) &&
> -	    lsi->lsi_lmd->lmd_flags & LMD_FLG_NOIR)
> -		data->ocd_connect_flags &= ~OBD_CONNECT_IMP_RECOV;
> -	data->ocd_version = LUSTRE_VERSION_CODE;
> -	rc = obd_connect(NULL, &exp, obd, &obd->obd_uuid, data, NULL);
> -	if (rc) {
> -		CERROR("connect failed %d\n", rc);
> -		goto out;
> -	}
> -
> -	obd->u.cli.cl_mgc_mgsexp = exp;
> -
> -out:
> -	/* Keep the mgc info in the sb. Note that many lsi's can point
> -	 * to the same mgc.
> -	 */
> -	lsi->lsi_mgc = obd;
> -out_free:
> -	mutex_unlock(&mgc_start_lock);
> -
> -	kfree(data);
> -	kfree(mgcname);
> -	kfree(niduuid);
> -	return rc;
> -}
> -
> -static int lustre_stop_mgc(struct super_block *sb)
> -{
> -	struct lustre_sb_info *lsi = s2lsi(sb);
> -	struct obd_device *obd;
> -	char *niduuid = NULL, *ptr = NULL;
> -	int i, rc = 0, len = 0;
> -
> -	if (!lsi)
> -		return -ENOENT;
> -	obd = lsi->lsi_mgc;
> -	if (!obd)
> -		return -ENOENT;
> -	lsi->lsi_mgc = NULL;
> -
> -	mutex_lock(&mgc_start_lock);
> -	LASSERT(atomic_read(&obd->u.cli.cl_mgc_refcount) > 0);
> -	if (!atomic_dec_and_test(&obd->u.cli.cl_mgc_refcount)) {
> -		/* This is not fatal, every client that stops
> -		 * will call in here.
> -		 */
> -		CDEBUG(D_MOUNT, "mgc still has %d references.\n",
> -		       atomic_read(&obd->u.cli.cl_mgc_refcount));
> -		rc = -EBUSY;
> -		goto out;
> -	}
> -
> -	/* The MGC has no recoverable data in any case.
> -	 * force shutdown set in umount_begin
> -	 */
> -	obd->obd_no_recov = 1;
> -
> -	if (obd->u.cli.cl_mgc_mgsexp) {
> -		/* An error is not fatal, if we are unable to send the
> -		 * disconnect mgs ping evictor cleans up the export
> -		 */
> -		rc = obd_disconnect(obd->u.cli.cl_mgc_mgsexp);
> -		if (rc)
> -			CDEBUG(D_MOUNT, "disconnect failed %d\n", rc);
> -	}
> -
> -	/* Save the obdname for cleaning the nid uuids, which are obdname_XX */
> -	len = strlen(obd->obd_name) + 6;
> -	niduuid = kzalloc(len, GFP_NOFS);
> -	if (niduuid) {
> -		strcpy(niduuid, obd->obd_name);
> -		ptr = niduuid + strlen(niduuid);
> -	}
> -
> -	rc = class_manual_cleanup(obd);
> -	if (rc)
> -		goto out;
> -
> -	/* Clean the nid uuids */
> -	if (!niduuid) {
> -		rc = -ENOMEM;
> -		goto out;
> -	}
> -
> -	for (i = 0; i < lsi->lsi_lmd->lmd_mgs_failnodes; i++) {
> -		sprintf(ptr, "_%x", i);
> -		rc = do_lcfg(LUSTRE_MGC_OBDNAME, 0, LCFG_DEL_UUID,
> -			     niduuid, NULL, NULL, NULL);
> -		if (rc)
> -			CERROR("del MDC UUID %s failed: rc = %d\n",
> -			       niduuid, rc);
> -	}
> -out:
> -	kfree(niduuid);
> -
> -	/* class_import_put will get rid of the additional connections */
> -	mutex_unlock(&mgc_start_lock);
> -	return rc;
> -}
> -
> -/***************** lustre superblock **************/
> -
> -static struct lustre_sb_info *lustre_init_lsi(struct super_block *sb)
> -{
> -	struct lustre_sb_info *lsi;
> -
> -	lsi = kzalloc(sizeof(*lsi), GFP_NOFS);
> -	if (!lsi)
> -		return NULL;
> -	lsi->lsi_lmd = kzalloc(sizeof(*lsi->lsi_lmd), GFP_NOFS);
> -	if (!lsi->lsi_lmd) {
> -		kfree(lsi);
> -		return NULL;
> -	}
> -
> -	lsi->lsi_lmd->lmd_exclude_count = 0;
> -	lsi->lsi_lmd->lmd_recovery_time_soft = 0;
> -	lsi->lsi_lmd->lmd_recovery_time_hard = 0;
> -	s2lsi_nocast(sb) = lsi;
> -	/* we take 1 extra ref for our setup */
> -	atomic_set(&lsi->lsi_mounts, 1);
> -
> -	/* Default umount style */
> -	lsi->lsi_flags = LSI_UMOUNT_FAILOVER;
> -
> -	return lsi;
> -}
> -
> -static int lustre_free_lsi(struct super_block *sb)
> -{
> -	struct lustre_sb_info *lsi = s2lsi(sb);
> -
> -	CDEBUG(D_MOUNT, "Freeing lsi %p\n", lsi);
> -
> -	/* someone didn't call server_put_mount. */
> -	LASSERT(atomic_read(&lsi->lsi_mounts) == 0);
> -
> -	if (lsi->lsi_lmd) {
> -		kfree(lsi->lsi_lmd->lmd_dev);
> -		kfree(lsi->lsi_lmd->lmd_profile);
> -		kfree(lsi->lsi_lmd->lmd_fileset);
> -		kfree(lsi->lsi_lmd->lmd_mgssec);
> -		kfree(lsi->lsi_lmd->lmd_opts);
> -		if (lsi->lsi_lmd->lmd_exclude_count)
> -			kfree(lsi->lsi_lmd->lmd_exclude);
> -		kfree(lsi->lsi_lmd->lmd_mgs);
> -		kfree(lsi->lsi_lmd->lmd_osd_type);
> -		kfree(lsi->lsi_lmd->lmd_params);
> -
> -		kfree(lsi->lsi_lmd);
> -	}
> -
> -	LASSERT(!lsi->lsi_llsbi);
> -	kfree(lsi);
> -	s2lsi_nocast(sb) = NULL;
> -
> -	return 0;
> -}
> -
> -/* The lsi has one reference for every server that is using the disk -
> - * e.g. MDT, MGS, and potentially MGC
> - */
> -static int lustre_put_lsi(struct super_block *sb)
> -{
> -	struct lustre_sb_info *lsi = s2lsi(sb);
> -
> -	CDEBUG(D_MOUNT, "put %p %d\n", sb, atomic_read(&lsi->lsi_mounts));
> -	if (atomic_dec_and_test(&lsi->lsi_mounts)) {
> -		lustre_free_lsi(sb);
> -		return 1;
> -	}
> -	return 0;
> -}
> -
> -/*************** mount common between server and client ***************/
> -
> -/* Common umount */
> -int lustre_common_put_super(struct super_block *sb)
> -{
> -	int rc;
> -
> -	CDEBUG(D_MOUNT, "dropping sb %p\n", sb);
> -
> -	/* Drop a ref to the MGC */
> -	rc = lustre_stop_mgc(sb);
> -	if (rc && (rc != -ENOENT)) {
> -		if (rc != -EBUSY) {
> -			CERROR("Can't stop MGC: %d\n", rc);
> -			return rc;
> -		}
> -		/* BUSY just means that there's some other obd that
> -		 * needs the mgc.  Let him clean it up.
> -		 */
> -		CDEBUG(D_MOUNT, "MGC still in use\n");
> -	}
> -	/* Drop a ref to the mounted disk */
> -	lustre_put_lsi(sb);
> -	return rc;
> -}
> -EXPORT_SYMBOL(lustre_common_put_super);
> -
> -static void lmd_print(struct lustre_mount_data *lmd)
> -{
> -	int i;
> -
> -	PRINT_CMD(D_MOUNT, "  mount data:\n");
> -	if (lmd_is_client(lmd))
> -		PRINT_CMD(D_MOUNT, "profile: %s\n", lmd->lmd_profile);
> -	PRINT_CMD(D_MOUNT, "device:  %s\n", lmd->lmd_dev);
> -	PRINT_CMD(D_MOUNT, "flags:   %x\n", lmd->lmd_flags);
> -
> -	if (lmd->lmd_opts)
> -		PRINT_CMD(D_MOUNT, "options: %s\n", lmd->lmd_opts);
> -
> -	if (lmd->lmd_recovery_time_soft)
> -		PRINT_CMD(D_MOUNT, "recovery time soft: %d\n",
> -			  lmd->lmd_recovery_time_soft);
> -
> -	if (lmd->lmd_recovery_time_hard)
> -		PRINT_CMD(D_MOUNT, "recovery time hard: %d\n",
> -			  lmd->lmd_recovery_time_hard);
> -
> -	for (i = 0; i < lmd->lmd_exclude_count; i++) {
> -		PRINT_CMD(D_MOUNT, "exclude %d:  OST%04x\n", i,
> -			  lmd->lmd_exclude[i]);
> -	}
> -}
> -
> -/* mount -v  -o exclude=lustre-OST0001:lustre-OST0002 -t lustre ... */
> -static int lmd_make_exclusion(struct lustre_mount_data *lmd, const char *ptr)
> -{
> -	const char *s1 = ptr, *s2;
> -	__u32 index = 0, *exclude_list;
> -	int rc = 0, devmax;
> -
> -	/* The shortest an ost name can be is 8 chars: -OST0000.
> -	 * We don't actually know the fsname at this time, so in fact
> -	 * a user could specify any fsname.
> -	 */
> -	devmax = strlen(ptr) / 8 + 1;
> -
> -	/* temp storage until we figure out how many we have */
> -	exclude_list = kcalloc(devmax, sizeof(index), GFP_NOFS);
> -	if (!exclude_list)
> -		return -ENOMEM;
> -
> -	/* we enter this fn pointing at the '=' */
> -	while (*s1 && *s1 != ' ' && *s1 != ',') {
> -		s1++;
> -		rc = server_name2index(s1, &index, &s2);
> -		if (rc < 0) {
> -			CERROR("Can't parse server name '%s': rc = %d\n",
> -			       s1, rc);
> -			break;
> -		}
> -		if (rc == LDD_F_SV_TYPE_OST)
> -			exclude_list[lmd->lmd_exclude_count++] = index;
> -		else
> -			CDEBUG(D_MOUNT, "ignoring exclude %.*s: type = %#x\n",
> -			       (uint)(s2 - s1), s1, rc);
> -		s1 = s2;
> -		/* now we are pointing at ':' (next exclude)
> -		 * or ',' (end of excludes)
> -		 */
> -		if (lmd->lmd_exclude_count >= devmax)
> -			break;
> -	}
> -	if (rc >= 0) /* non-err */
> -		rc = 0;
> -
> -	if (lmd->lmd_exclude_count) {
> -		/* permanent, freed in lustre_free_lsi */
> -		lmd->lmd_exclude = kcalloc(lmd->lmd_exclude_count,
> -					   sizeof(index), GFP_NOFS);
> -		if (lmd->lmd_exclude) {
> -			memcpy(lmd->lmd_exclude, exclude_list,
> -			       sizeof(index) * lmd->lmd_exclude_count);
> -		} else {
> -			rc = -ENOMEM;
> -			lmd->lmd_exclude_count = 0;
> -		}
> -	}
> -	kfree(exclude_list);
> -	return rc;
> -}
> -
> -static int lmd_parse_mgssec(struct lustre_mount_data *lmd, char *ptr)
> -{
> -	char   *tail;
> -	int     length;
> -
> -	kfree(lmd->lmd_mgssec);
> -	lmd->lmd_mgssec = NULL;
> -
> -	tail = strchr(ptr, ',');
> -	if (!tail)
> -		length = strlen(ptr);
> -	else
> -		length = tail - ptr;
> -
> -	lmd->lmd_mgssec = kzalloc(length + 1, GFP_NOFS);
> -	if (!lmd->lmd_mgssec)
> -		return -ENOMEM;
> -
> -	memcpy(lmd->lmd_mgssec, ptr, length);
> -	lmd->lmd_mgssec[length] = '\0';
> -	return 0;
> -}
> -
> -static int lmd_parse_string(char **handle, char *ptr)
> -{
> -	char   *tail;
> -	int     length;
> -
> -	if (!handle || !ptr)
> -		return -EINVAL;
> -
> -	kfree(*handle);
> -	*handle = NULL;
> -
> -	tail = strchr(ptr, ',');
> -	if (!tail)
> -		length = strlen(ptr);
> -	else
> -		length = tail - ptr;
> -
> -	*handle = kzalloc(length + 1, GFP_NOFS);
> -	if (!*handle)
> -		return -ENOMEM;
> -
> -	memcpy(*handle, ptr, length);
> -	(*handle)[length] = '\0';
> -
> -	return 0;
> -}
> -
> -/* Collect multiple values for mgsnid specifiers */
> -static int lmd_parse_mgs(struct lustre_mount_data *lmd, char **ptr)
> -{
> -	lnet_nid_t nid;
> -	char *tail = *ptr;
> -	char *mgsnid;
> -	int   length;
> -	int   oldlen = 0;
> -
> -	/* Find end of nidlist */
> -	while (class_parse_nid_quiet(tail, &nid, &tail) == 0)
> -		;
> -	length = tail - *ptr;
> -	if (length == 0) {
> -		LCONSOLE_ERROR_MSG(0x159, "Can't parse NID '%s'\n", *ptr);
> -		return -EINVAL;
> -	}
> -
> -	if (lmd->lmd_mgs)
> -		oldlen = strlen(lmd->lmd_mgs) + 1;
> -
> -	mgsnid = kzalloc(oldlen + length + 1, GFP_NOFS);
> -	if (!mgsnid)
> -		return -ENOMEM;
> -
> -	if (lmd->lmd_mgs) {
> -		/* Multiple mgsnid= are taken to mean failover locations */
> -		memcpy(mgsnid, lmd->lmd_mgs, oldlen);
> -		mgsnid[oldlen - 1] = ':';
> -		kfree(lmd->lmd_mgs);
> -	}
> -	memcpy(mgsnid + oldlen, *ptr, length);
> -	mgsnid[oldlen + length] = '\0';
> -	lmd->lmd_mgs = mgsnid;
> -	*ptr = tail;
> -
> -	return 0;
> -}
> -
> -/**
> - * Find the first delimiter (comma or colon) from the specified \a buf and
> - * make \a *endh point to the string starting with the delimiter. The commas
> - * in expression list [...] will be skipped.
> - *
> - * @buf		a delimiter-separated string
> - * @endh	a pointer to a pointer that will point to the string
> - *		starting with the delimiter
> - *
> - * RETURNS	true if delimiter is found, false if delimiter is not found
> - */
> -static bool lmd_find_delimiter(char *buf, char **endh)
> -{
> -	char *c = buf;
> -	size_t pos;
> -	bool found;
> -
> -	if (!buf)
> -		return false;
> -try_again:
> -	if (*c == ',' || *c == ':')
> -		return true;
> -
> -	pos = strcspn(c, "[:,]");
> -	if (!pos)
> -		return false;
> -
> -	/* Not a valid mount string */
> -	if (*c == ']') {
> -		CWARN("invalid mount string format\n");
> -		return false;
> -	}
> -
> -	c += pos;
> -	if (*c == '[') {
> -		c = strchr(c, ']');
> -
> -		/* invalid mount string */
> -		if (!c) {
> -			CWARN("invalid mount string format\n");
> -			return false;
> -		}
> -		c++;
> -		goto try_again;
> -	}
> -
> -	found = *c != '\0';
> -	if (found && endh)
> -		*endh = c;
> -
> -	return found;
> -}
> -
> -/**
> - * Find the first valid string delimited by comma or colon from the specified
> - * \a buf and parse it to see whether it's a valid nid list. If yes, \a *endh
> - * will point to the next string starting with the delimiter.
> - *
> - * \param[in] buf	a delimiter-separated string
> - * \param[in] endh	a pointer to a pointer that will point to the string
> - *			starting with the delimiter
> - *
> - * \retval 0		if the string is a valid nid list
> - * \retval 1		if the string is not a valid nid list
> - */
> -static int lmd_parse_nidlist(char *buf, char **endh)
> -{
> -	struct list_head nidlist;
> -	char *endp = buf;
> -	int rc = 0;
> -	char tmp;
> -
> -	if (!buf)
> -		return 1;
> -	while (*buf == ',' || *buf == ':')
> -		buf++;
> -	if (*buf == ' ' || *buf == '/' || *buf == '\0')
> -		return 1;
> -
> -	if (!lmd_find_delimiter(buf, &endp))
> -		endp = buf + strlen(buf);
> -
> -	tmp = *endp;
> -	*endp = '\0';
> -
> -	INIT_LIST_HEAD(&nidlist);
> -	if (cfs_parse_nidlist(buf, strlen(buf), &nidlist) <= 0)
> -		rc = 1;
> -	cfs_free_nidlist(&nidlist);
> -
> -	*endp = tmp;
> -	if (rc)
> -		return rc;
> -	if (endh)
> -		*endh = endp;
> -	return 0;
> -}
> -
> -/** Parse mount line options
> - * e.g. mount -v -t lustre -o abort_recov uml1:uml2:/lustre-client /mnt/lustre
> - * dev is passed as device=uml1:/lustre by mount.lustre
> - */
> -static int lmd_parse(char *options, struct lustre_mount_data *lmd)
> -{
> -	char *s1, *s2, *devname = NULL;
> -	struct lustre_mount_data *raw = (struct lustre_mount_data *)options;
> -	int rc = 0;
> -
> -	LASSERT(lmd);
> -	if (!options) {
> -		LCONSOLE_ERROR_MSG(0x162, "Missing mount data: check that /sbin/mount.lustre is installed.\n");
> -		return -EINVAL;
> -	}
> -
> -	/* Options should be a string - try to detect old lmd data */
> -	if ((raw->lmd_magic & 0xffffff00) == (LMD_MAGIC & 0xffffff00)) {
> -		LCONSOLE_ERROR_MSG(0x163, "You're using an old version of /sbin/mount.lustre.  Please install version %s\n",
> -				   LUSTRE_VERSION_STRING);
> -		return -EINVAL;
> -	}
> -	lmd->lmd_magic = LMD_MAGIC;
> -
> -	lmd->lmd_params = kzalloc(LMD_PARAMS_MAXLEN, GFP_NOFS);
> -	if (!lmd->lmd_params)
> -		return -ENOMEM;
> -	lmd->lmd_params[0] = '\0';
> -
> -	/* Set default flags here */
> -
> -	s1 = options;
> -	while (*s1) {
> -		int clear = 0;
> -		int time_min = OBD_RECOVERY_TIME_MIN;
> -		char *s3;
> -
> -		/* Skip whitespace and extra commas */
> -		while (*s1 == ' ' || *s1 == ',')
> -			s1++;
> -		s3 = s1;
> -
> -		/* Client options are parsed in ll_options: eg. flock,
> -		 * user_xattr, acl
> -		 */
> -
> -		/* Parse non-ldiskfs options here. Rather than modifying
> -		 * ldiskfs, we just zero these out here
> -		 */
> -		if (strncmp(s1, "abort_recov", 11) == 0) {
> -			lmd->lmd_flags |= LMD_FLG_ABORT_RECOV;
> -			clear++;
> -		} else if (strncmp(s1, "recovery_time_soft=", 19) == 0) {
> -			lmd->lmd_recovery_time_soft = max_t(int,
> -				simple_strtoul(s1 + 19, NULL, 10), time_min);
> -			clear++;
> -		} else if (strncmp(s1, "recovery_time_hard=", 19) == 0) {
> -			lmd->lmd_recovery_time_hard = max_t(int,
> -				simple_strtoul(s1 + 19, NULL, 10), time_min);
> -			clear++;
> -		} else if (strncmp(s1, "noir", 4) == 0) {
> -			lmd->lmd_flags |= LMD_FLG_NOIR; /* test purpose only. */
> -			clear++;
> -		} else if (strncmp(s1, "nosvc", 5) == 0) {
> -			lmd->lmd_flags |= LMD_FLG_NOSVC;
> -			clear++;
> -		} else if (strncmp(s1, "nomgs", 5) == 0) {
> -			lmd->lmd_flags |= LMD_FLG_NOMGS;
> -			clear++;
> -		} else if (strncmp(s1, "noscrub", 7) == 0) {
> -			lmd->lmd_flags |= LMD_FLG_NOSCRUB;
> -			clear++;
> -		} else if (strncmp(s1, PARAM_MGSNODE,
> -				   sizeof(PARAM_MGSNODE) - 1) == 0) {
> -			s2 = s1 + sizeof(PARAM_MGSNODE) - 1;
> -			/* Assume the next mount opt is the first
> -			 * invalid nid we get to.
> -			 */
> -			rc = lmd_parse_mgs(lmd, &s2);
> -			if (rc)
> -				goto invalid;
> -			clear++;
> -		} else if (strncmp(s1, "writeconf", 9) == 0) {
> -			lmd->lmd_flags |= LMD_FLG_WRITECONF;
> -			clear++;
> -		} else if (strncmp(s1, "update", 6) == 0) {
> -			lmd->lmd_flags |= LMD_FLG_UPDATE;
> -			clear++;
> -		} else if (strncmp(s1, "virgin", 6) == 0) {
> -			lmd->lmd_flags |= LMD_FLG_VIRGIN;
> -			clear++;
> -		} else if (strncmp(s1, "noprimnode", 10) == 0) {
> -			lmd->lmd_flags |= LMD_FLG_NO_PRIMNODE;
> -			clear++;
> -		} else if (strncmp(s1, "mgssec=", 7) == 0) {
> -			rc = lmd_parse_mgssec(lmd, s1 + 7);
> -			if (rc)
> -				goto invalid;
> -			s3 = s2;
> -			clear++;
> -		/* ost exclusion list */
> -		} else if (strncmp(s1, "exclude=", 8) == 0) {
> -			rc = lmd_make_exclusion(lmd, s1 + 7);
> -			if (rc)
> -				goto invalid;
> -			clear++;
> -		} else if (strncmp(s1, "mgs", 3) == 0) {
> -			/* We are an MGS */
> -			lmd->lmd_flags |= LMD_FLG_MGS;
> -			clear++;
> -		} else if (strncmp(s1, "svname=", 7) == 0) {
> -			rc = lmd_parse_string(&lmd->lmd_profile, s1 + 7);
> -			if (rc)
> -				goto invalid;
> -			clear++;
> -		} else if (strncmp(s1, "param=", 6) == 0) {
> -			size_t length, params_length;
> -			char *tail = s1;
> -
> -			if (lmd_find_delimiter(s1 + 6, &tail)) {
> -				char *param_str = tail + 1;
> -				int supplementary = 1;
> -
> -				while (!lmd_parse_nidlist(param_str,
> -							  &param_str))
> -					supplementary = 0;
> -				length = param_str - s1 - supplementary;
> -			} else {
> -				length = strlen(s1);
> -			}
> -			length -= 6;
> -			params_length = strlen(lmd->lmd_params);
> -			if (params_length + length + 1 >= LMD_PARAMS_MAXLEN)
> -				return -E2BIG;
> -			strncat(lmd->lmd_params, s1 + 6, length);
> -			lmd->lmd_params[params_length + length] = '\0';
> -			strlcat(lmd->lmd_params, " ", LMD_PARAMS_MAXLEN);
> -			s3 = s1 + 6 + length;
> -			clear++;
> -		} else if (strncmp(s1, "osd=", 4) == 0) {
> -			rc = lmd_parse_string(&lmd->lmd_osd_type, s1 + 4);
> -			if (rc)
> -				goto invalid;
> -			clear++;
> -		}
> -		/* Linux 2.4 doesn't pass the device, so we stuck it at the
> -		 * end of the options.
> -		 */
> -		else if (strncmp(s1, "device=", 7) == 0) {
> -			devname = s1 + 7;
> -			/* terminate options right before device.  device
> -			 * must be the last one.
> -			 */
> -			*s1 = '\0';
> -			break;
> -		}
> -
> -		/* Find next opt */
> -		s2 = strchr(s1, ',');
> -		if (!s2) {
> -			if (clear)
> -				*s1 = '\0';
> -			break;
> -		}
> -		s2++;
> -		if (clear)
> -			memmove(s1, s2, strlen(s2) + 1);
> -		else
> -			s1 = s2;
> -	}
> -
> -	if (!devname) {
> -		LCONSOLE_ERROR_MSG(0x164, "Can't find the device name (need mount option 'device=...')\n");
> -		goto invalid;
> -	}
> -
> -	s1 = strstr(devname, ":/");
> -	if (s1) {
> -		++s1;
> -		lmd->lmd_flags |= LMD_FLG_CLIENT;
> -		/* Remove leading /s from fsname */
> -		while (*++s1 == '/')
> -			;
> -		s2 = strchrnul(s1, '/');
> -		/* Freed in lustre_free_lsi */
> -		lmd->lmd_profile = kasprintf(GFP_KERNEL, "%.*s-client",
> -					     (int)(s2 - s1), s1);
> -		if (!lmd->lmd_profile)
> -			return -ENOMEM;
> -
> -		s1 = s2;
> -		s2 = s1 + strlen(s1) - 1;
> -		/* Remove padding /s from fileset */
> -		while (*s2 == '/')
> -			s2--;
> -		if (s2 > s1) {
> -			lmd->lmd_fileset = kstrndup(s1, s2 - s1 + 1,
> -						    GFP_KERNEL);
> -			if (!lmd->lmd_fileset)
> -				return -ENOMEM;
> -		}
> -	}
> -
> -	/* Freed in lustre_free_lsi */
> -	lmd->lmd_dev = kstrdup(devname, GFP_KERNEL);
> -	if (!lmd->lmd_dev)
> -		return -ENOMEM;
> -
> -	/* Save mount options */
> -	s1 = options + strlen(options) - 1;
> -	while (s1 >= options && (*s1 == ',' || *s1 == ' '))
> -		*s1-- = 0;
> -	if (*options != 0) {
> -		/* Freed in lustre_free_lsi */
> -		lmd->lmd_opts = kstrdup(options, GFP_KERNEL);
> -		if (!lmd->lmd_opts)
> -			return -ENOMEM;
> -	}
> -
> -	lmd_print(lmd);
> -	lmd->lmd_magic = LMD_MAGIC;
> -
> -	return rc;
> -
> -invalid:
> -	CERROR("Bad mount options %s\n", options);
> -	return -EINVAL;
> -}
> -
> -/** This is the entry point for the mount call into Lustre.
> - * This is called when a server or client is mounted,
> - * and this is where we start setting things up.
> - * @param data Mount options (e.g. -o flock,abort_recov)
> - */
> -static int lustre_fill_super(struct super_block *sb, void *lmd2_data, int silent)
> -{
> -	struct lustre_mount_data *lmd;
> -	struct lustre_sb_info *lsi;
> -	int rc;
> -
> -	CDEBUG(D_MOUNT | D_VFSTRACE, "VFS Op: sb %p\n", sb);
> -
> -	lsi = lustre_init_lsi(sb);
> -	if (!lsi)
> -		return -ENOMEM;
> -	lmd = lsi->lsi_lmd;
> -
> -	/*
> -	 * Disable lockdep during mount, because mount locking patterns are
> -	 * `special'.
> -	 */
> -	lockdep_off();
> -
> -	/*
> -	 * LU-639: the obd cleanup of last mount may not finish yet, wait here.
> -	 */
> -	obd_zombie_barrier();
> -
> -	/* Figure out the lmd from the mount options */
> -	if (lmd_parse(lmd2_data, lmd)) {
> -		lustre_put_lsi(sb);
> -		rc = -EINVAL;
> -		goto out;
> -	}
> -
> -	if (lmd_is_client(lmd)) {
> -		bool have_client = false;
> -		CDEBUG(D_MOUNT, "Mounting client %s\n", lmd->lmd_profile);
> -		if (!client_fill_super)
> -			request_module("lustre");
> -		spin_lock(&client_lock);
> -		if (client_fill_super && try_module_get(client_mod))
> -			have_client = true;
> -		spin_unlock(&client_lock);
> -		if (!have_client) {
> -			LCONSOLE_ERROR_MSG(0x165, "Nothing registered for client mount! Is the 'lustre' module loaded?\n");
> -			lustre_put_lsi(sb);
> -			rc = -ENODEV;
> -		} else {
> -			rc = lustre_start_mgc(sb);
> -			if (rc) {
> -				lustre_common_put_super(sb);
> -				goto out;
> -			}
> -			/* Connect and start */
> -			/* (should always be ll_fill_super) */
> -			rc = (*client_fill_super)(sb);
> -			/* c_f_s will call lustre_common_put_super on failure, otherwise
> -			 * c_f_s will have taken another reference to the module */
> -			module_put(client_mod);
> -		}
> -	} else {
> -		CERROR("This is client-side-only module, cannot handle server mount.\n");
> -		rc = -EINVAL;
> -	}
> -
> -	/* If error happens in fill_super() call, @lsi will be killed there.
> -	 * This is why we do not put it here.
> -	 */
> -	goto out;
> -out:
> -	if (rc) {
> -		CERROR("Unable to mount %s (%d)\n",
> -		       s2lsi(sb) ? lmd->lmd_dev : "", rc);
> -	} else {
> -		CDEBUG(D_SUPER, "Mount %s complete\n",
> -		       lmd->lmd_dev);
> -	}
> -	lockdep_on();
> -	return rc;
> -}
> -
> -/* We can't call ll_fill_super by name because it lives in a module that
> - * must be loaded after this one.
> - */
> -void lustre_register_super_ops(struct module *mod,
> -			       int (*cfs)(struct super_block *sb),
> -			       void (*ksc)(struct super_block *sb))
> -{
> -	spin_lock(&client_lock);
> -	client_mod = mod;
> -	client_fill_super = cfs;
> -	kill_super_cb = ksc;
> -	spin_unlock(&client_lock);
> -}
> -EXPORT_SYMBOL(lustre_register_super_ops);
> -
> -/***************** FS registration ******************/
> -static struct dentry *lustre_mount(struct file_system_type *fs_type, int flags,
> -				   const char *devname, void *data)
> -{
> -	return mount_nodev(fs_type, flags, data, lustre_fill_super);
> -}
> -
> -static void lustre_kill_super(struct super_block *sb)
> -{
> -	struct lustre_sb_info *lsi = s2lsi(sb);
> -
> -	if (kill_super_cb && lsi)
> -		(*kill_super_cb)(sb);
> -
> -	kill_anon_super(sb);
> -}
> -
> -/** Register the "lustre" fs type
> - */
> -static struct file_system_type lustre_fs_type = {
> -	.owner		= THIS_MODULE,
> -	.name		= "lustre",
> -	.mount		= lustre_mount,
> -	.kill_sb	= lustre_kill_super,
> -	.fs_flags	= FS_RENAME_DOES_D_MOVE,
> -};
> -MODULE_ALIAS_FS("lustre");
> -
> -int lustre_register_fs(void)
> -{
> -	return register_filesystem(&lustre_fs_type);
> -}
> -
> -int lustre_unregister_fs(void)
> -{
> -	return unregister_filesystem(&lustre_fs_type);
> -}
> 
> 
> 

From jsimmons at infradead.org  Sun Jul 29 17:29:29 2018
From: jsimmons at infradead.org (James Simmons)
Date: Sun, 29 Jul 2018 18:29:29 +0100 (BST)
Subject: [lustre-devel] [PATCH 3/9] lustre: move server_name2index to
 obd_config.c
In-Reply-To: <153232698486.26222.18197471449250973608.stgit@noble>
References: <153232696720.26222.9658151633697867322.stgit@noble>
 <153232698486.26222.18197471449250973608.stgit@noble>
Message-ID: <alpine.LFD.2.21.1807291828210.2402@casper.infradead.org>


> This is preparation for moving filesystem mounting
> into llite/.
> server_name2index belongs in obdclass, but the rest
> of obd_mount.c fits better in llite.  So move
> it and server_name2fsname() across to obd_config.c,
> and expect it for later using by llite.

Hmm. These are very different from the OpenSFS branch. Appears some
rearranging of the code dropped some stuff. I will look into cleaning
separating the server and client.
 
> Signed-off-by: NeilBrown <neilb at suse.com>
> ---
>  drivers/staging/lustre/lustre/include/obd_class.h  |    2 
>  .../staging/lustre/lustre/obdclass/obd_config.c    |   86 ++++++++++++++++++++
>  drivers/staging/lustre/lustre/obdclass/obd_mount.c |   84 --------------------
>  3 files changed, 88 insertions(+), 84 deletions(-)
> 
> diff --git a/drivers/staging/lustre/lustre/include/obd_class.h b/drivers/staging/lustre/lustre/include/obd_class.h
> index adfe2abdf7bc..ef8c46cb9bee 100644
> --- a/drivers/staging/lustre/lustre/include/obd_class.h
> +++ b/drivers/staging/lustre/lustre/include/obd_class.h
> @@ -134,6 +134,8 @@ int class_config_llog_handler(const struct lu_env *env,
>  			      struct llog_handle *handle,
>  			      struct llog_rec_hdr *rec, void *data);
>  int class_add_uuid(const char *uuid, __u64 nid);
> +int server_name2index(const char *svname, __u32 *idx,
> +		      const char **endptr);
>  
>  /* obdecho */
>  void lprocfs_echo_init_vars(struct lprocfs_static_vars *lvars);
> diff --git a/drivers/staging/lustre/lustre/obdclass/obd_config.c b/drivers/staging/lustre/lustre/obdclass/obd_config.c
> index cfcd17e679dd..382522fa1993 100644
> --- a/drivers/staging/lustre/lustre/obdclass/obd_config.c
> +++ b/drivers/staging/lustre/lustre/obdclass/obd_config.c
> @@ -46,6 +46,7 @@
>  #include <lustre_log.h>
>  #include <uapi/linux/lustre/lustre_param.h>
>  #include <obd_class.h>
> +#include <lustre_disk.h>
>  
>  #include "llog_internal.h"
>  
> @@ -1172,6 +1173,91 @@ int class_process_proc_param(char *prefix, struct lprocfs_vars *lvars,
>  }
>  EXPORT_SYMBOL(class_process_proc_param);
>  
> +/*** SERVER NAME ***
> + * <FSNAME><SEPARATOR><TYPE><INDEX>
> + * FSNAME is between 1 and 8 characters (inclusive).
> + *	Excluded characters are '/' and ':'
> + * SEPARATOR is either ':' or '-'
> + * TYPE: "OST", "MDT", etc.
> + * INDEX: Hex representation of the index
> + */
> +
> +/** Get the fsname ("lustre") from the server name ("lustre-OST003F").
> + * @param [in] svname server name including type and index
> + * @param [out] fsname Buffer to copy filesystem name prefix into.
> + *  Must have at least 'strlen(fsname) + 1' chars.
> + * @param [out] endptr if endptr isn't NULL it is set to end of fsname
> + * rc < 0  on error
> + */
> +static int server_name2fsname(const char *svname, char *fsname,
> +			      const char **endptr)
> +{
> +	const char *dash;
> +
> +	dash = svname + strnlen(svname, 8); /* max fsname length is 8 */
> +	for (; dash > svname && *dash != '-' && *dash != ':'; dash--)
> +		;
> +	if (dash == svname)
> +		return -EINVAL;
> +
> +	if (fsname) {
> +		strncpy(fsname, svname, dash - svname);
> +		fsname[dash - svname] = '\0';
> +	}
> +
> +	if (endptr)
> +		*endptr = dash;
> +
> +	return 0;
> +}
> +
> +/* Get the index from the obd name.
> + *  rc = server type, or
> + * rc < 0  on error
> + * if endptr isn't NULL it is set to end of name
> + */
> +int server_name2index(const char *svname, __u32 *idx,
> +		      const char **endptr)
> +{
> +	unsigned long index;
> +	int rc;
> +	const char *dash;
> +
> +	/* We use server_name2fsname() just for parsing */
> +	rc = server_name2fsname(svname, NULL, &dash);
> +	if (rc != 0)
> +		return rc;
> +
> +	dash++;
> +
> +	if (strncmp(dash, "MDT", 3) == 0)
> +		rc = LDD_F_SV_TYPE_MDT;
> +	else if (strncmp(dash, "OST", 3) == 0)
> +		rc = LDD_F_SV_TYPE_OST;
> +	else
> +		return -EINVAL;
> +
> +	dash += 3;
> +
> +	if (strncmp(dash, "all", 3) == 0) {
> +		if (endptr)
> +			*endptr = dash + 3;
> +		return rc | LDD_F_SV_ALL;
> +	}
> +
> +	index = simple_strtoul(dash, (char **)endptr, 16);
> +	if (idx)
> +		*idx = index;
> +
> +	/* Account for -mdc after index that is possible when specifying mdt */
> +	if (endptr && strncmp(LUSTRE_MDC_NAME, *endptr + 1,
> +			      sizeof(LUSTRE_MDC_NAME) - 1) == 0)
> +		*endptr += sizeof(LUSTRE_MDC_NAME);
> +
> +	return rc;
> +}
> +EXPORT_SYMBOL(server_name2index);
> +
>  /** Parse a configuration llog, doing various manipulations on them
>   * for various reasons, (modifications for compatibility, skip obsolete
>   * records, change uuids, etc), then class_process_config() resulting
> diff --git a/drivers/staging/lustre/lustre/obdclass/obd_mount.c b/drivers/staging/lustre/lustre/obdclass/obd_mount.c
> index 1a668874a165..3420e87ad0cd 100644
> --- a/drivers/staging/lustre/lustre/obdclass/obd_mount.c
> +++ b/drivers/staging/lustre/lustre/obdclass/obd_mount.c
> @@ -579,90 +579,6 @@ static int lustre_put_lsi(struct super_block *sb)
>  	return 0;
>  }
>  
> -/*** SERVER NAME ***
> - * <FSNAME><SEPARATOR><TYPE><INDEX>
> - * FSNAME is between 1 and 8 characters (inclusive).
> - *	Excluded characters are '/' and ':'
> - * SEPARATOR is either ':' or '-'
> - * TYPE: "OST", "MDT", etc.
> - * INDEX: Hex representation of the index
> - */
> -
> -/** Get the fsname ("lustre") from the server name ("lustre-OST003F").
> - * @param [in] svname server name including type and index
> - * @param [out] fsname Buffer to copy filesystem name prefix into.
> - *  Must have at least 'strlen(fsname) + 1' chars.
> - * @param [out] endptr if endptr isn't NULL it is set to end of fsname
> - * rc < 0  on error
> - */
> -static int server_name2fsname(const char *svname, char *fsname,
> -			      const char **endptr)
> -{
> -	const char *dash;
> -
> -	dash = svname + strnlen(svname, 8); /* max fsname length is 8 */
> -	for (; dash > svname && *dash != '-' && *dash != ':'; dash--)
> -		;
> -	if (dash == svname)
> -		return -EINVAL;
> -
> -	if (fsname) {
> -		strncpy(fsname, svname, dash - svname);
> -		fsname[dash - svname] = '\0';
> -	}
> -
> -	if (endptr)
> -		*endptr = dash;
> -
> -	return 0;
> -}
> -
> -/* Get the index from the obd name.
> - *  rc = server type, or
> - * rc < 0  on error
> - * if endptr isn't NULL it is set to end of name
> - */
> -static int server_name2index(const char *svname, __u32 *idx,
> -			     const char **endptr)
> -{
> -	unsigned long index;
> -	int rc;
> -	const char *dash;
> -
> -	/* We use server_name2fsname() just for parsing */
> -	rc = server_name2fsname(svname, NULL, &dash);
> -	if (rc != 0)
> -		return rc;
> -
> -	dash++;
> -
> -	if (strncmp(dash, "MDT", 3) == 0)
> -		rc = LDD_F_SV_TYPE_MDT;
> -	else if (strncmp(dash, "OST", 3) == 0)
> -		rc = LDD_F_SV_TYPE_OST;
> -	else
> -		return -EINVAL;
> -
> -	dash += 3;
> -
> -	if (strncmp(dash, "all", 3) == 0) {
> -		if (endptr)
> -			*endptr = dash + 3;
> -		return rc | LDD_F_SV_ALL;
> -	}
> -
> -	index = simple_strtoul(dash, (char **)endptr, 16);
> -	if (idx)
> -		*idx = index;
> -
> -	/* Account for -mdc after index that is possible when specifying mdt */
> -	if (endptr && strncmp(LUSTRE_MDC_NAME, *endptr + 1,
> -			      sizeof(LUSTRE_MDC_NAME) - 1) == 0)
> -		*endptr += sizeof(LUSTRE_MDC_NAME);
> -
> -	return rc;
> -}
> -
>  /*************** mount common between server and client ***************/
>  
>  /* Common umount */
> 
> 
> 

From jsimmons at infradead.org  Sun Jul 29 17:30:38 2018
From: jsimmons at infradead.org (James Simmons)
Date: Sun, 29 Jul 2018 18:30:38 +0100 (BST)
Subject: [lustre-devel] [PATCH 8/9] lustre: ensure libcfs is set up for
 ioctls.
In-Reply-To: <153232698502.26222.8106098993681745318.stgit@noble>
References: <153232696720.26222.9658151633697867322.stgit@noble>
 <153232698502.26222.8106098993681745318.stgit@noble>
Message-ID: <alpine.LFD.2.21.1807291829560.2402@casper.infradead.org>


> libcfs only allocated various buffers when libcfs_setup()
> is called.
> This should be called before any significant libcfs related
> activity.  However it isn't called by libcfs_ioctl().
> So if the first thing that happens is an ioctl, tracing can cause
> NULL pointer dereferences.

Reviewed-by: James Simmons <jsimmons at infradead.org>
 
> Fixes: 64bf0b1a079d ("staging: lustre: refactor libcfs initialization.")
> Signed-off-by: NeilBrown <neilb at suse.com>
> ---
>  drivers/staging/lustre/lnet/libcfs/module.c |    3 +++
>  1 file changed, 3 insertions(+)
> 
> diff --git a/drivers/staging/lustre/lnet/libcfs/module.c b/drivers/staging/lustre/lnet/libcfs/module.c
> index ad654b56814d..bfadfcfa3c44 100644
> --- a/drivers/staging/lustre/lnet/libcfs/module.c
> +++ b/drivers/staging/lustre/lnet/libcfs/module.c
> @@ -206,6 +206,9 @@ static int libcfs_ioctl(unsigned long cmd, void __user *uparam)
>  	struct libcfs_ioctl_hdr *hdr;
>  	int err;
>  
> +	err = libcfs_setup();
> +	if (err)
> +		return err;
>  	/* 'cmd' and permissions get checked in our arch-specific caller */
>  	err = libcfs_ioctl_getdata(&hdr, uparam);
>  	if (err) {
> 
> 
> 

From jsimmons at infradead.org  Sun Jul 29 17:31:35 2018
From: jsimmons at infradead.org (James Simmons)
Date: Sun, 29 Jul 2018 18:31:35 +0100 (BST)
Subject: [lustre-devel] [PATCH 9/9] lustre: lnet: discard LNET_LOCK()
In-Reply-To: <153232698505.26222.7268545921228126117.stgit@noble>
References: <153232696720.26222.9658151633697867322.stgit@noble>
 <153232698505.26222.7268545921228126117.stgit@noble>
Message-ID: <alpine.LFD.2.21.1807291831260.2402@casper.infradead.org>


> This macro, and LNET_UNLOCK() are rarely used, don't
> add clarify, and make greping for lock usage harder.
> So discard macro and just the lnet_net_{un,}lock()
> like everyone else.

Reviewed-by: James Simmons <jsimmons at infradead.org>
 
> Signed-off-by: NeilBrown <neilb at suse.com>
> ---
>  .../staging/lustre/include/linux/lnet/lib-lnet.h   |    3 ---
>  .../lustre/lnet/klnds/socklnd/socklnd_proto.c      |    8 ++++----
>  drivers/staging/lustre/lnet/selftest/rpc.c         |    2 +-
>  3 files changed, 5 insertions(+), 8 deletions(-)
> 
> diff --git a/drivers/staging/lustre/include/linux/lnet/lib-lnet.h b/drivers/staging/lustre/include/linux/lnet/lib-lnet.h
> index 6b6289cfcd3d..8ff8139e04fe 100644
> --- a/drivers/staging/lustre/include/linux/lnet/lib-lnet.h
> +++ b/drivers/staging/lustre/include/linux/lnet/lib-lnet.h
> @@ -177,9 +177,6 @@ lnet_net_lock_current(void)
>  	return cpt;
>  }
>  
> -#define LNET_LOCK()		lnet_net_lock(LNET_LOCK_EX)
> -#define LNET_UNLOCK()		lnet_net_unlock(LNET_LOCK_EX)
> -
>  #define lnet_ptl_lock(ptl)	spin_lock(&(ptl)->ptl_lock)
>  #define lnet_ptl_unlock(ptl)	spin_unlock(&(ptl)->ptl_lock)
>  #define lnet_eq_wait_lock()	spin_lock(&the_lnet.ln_eq_wait_lock)
> diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_proto.c b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_proto.c
> index 05982dac781c..aaa04a5f0527 100644
> --- a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_proto.c
> +++ b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_proto.c
> @@ -485,7 +485,7 @@ ksocknal_send_hello_v1(struct ksock_conn *conn, struct ksock_hello_msg *hello)
>  
>  	if (the_lnet.ln_testprotocompat) {
>  		/* single-shot proto check */
> -		LNET_LOCK();
> +		lnet_net_lock(LNET_LOCK_EX);
>  		if (the_lnet.ln_testprotocompat & 1) {
>  			hmv->version_major++;   /* just different! */
>  			the_lnet.ln_testprotocompat &= ~1;
> @@ -494,7 +494,7 @@ ksocknal_send_hello_v1(struct ksock_conn *conn, struct ksock_hello_msg *hello)
>  			hmv->magic = LNET_PROTO_MAGIC;
>  			the_lnet.ln_testprotocompat &= ~2;
>  		}
> -		LNET_UNLOCK();
> +		lnet_net_unlock(LNET_LOCK_EX);
>  	}
>  
>  	hdr->src_nid = cpu_to_le64(hello->kshm_src_nid);
> @@ -542,12 +542,12 @@ ksocknal_send_hello_v2(struct ksock_conn *conn, struct ksock_hello_msg *hello)
>  
>  	if (the_lnet.ln_testprotocompat) {
>  		/* single-shot proto check */
> -		LNET_LOCK();
> +		lnet_net_lock(LNET_LOCK_EX);
>  		if (the_lnet.ln_testprotocompat & 1) {
>  			hello->kshm_version++;   /* just different! */
>  			the_lnet.ln_testprotocompat &= ~1;
>  		}
> -		LNET_UNLOCK();
> +		lnet_net_unlock(LNET_LOCK_EX);
>  	}
>  
>  	rc = lnet_sock_write(sock, hello, offsetof(struct ksock_hello_msg, kshm_ips),
> diff --git a/drivers/staging/lustre/lnet/selftest/rpc.c b/drivers/staging/lustre/lnet/selftest/rpc.c
> index 9613b0a77007..e097ef8414a6 100644
> --- a/drivers/staging/lustre/lnet/selftest/rpc.c
> +++ b/drivers/staging/lustre/lnet/selftest/rpc.c
> @@ -1399,7 +1399,7 @@ srpc_send_reply(struct srpc_server_rpc *rpc)
>  	return rc;
>  }
>  
> -/* when in kernel always called with LNET_LOCK() held, and in thread context */
> +/* when in kernel always called with lnet_net_lock() held, and in thread context */
>  static void
>  srpc_lnet_ev_handler(struct lnet_event *ev)
>  {
> 
> 
> 

From jsimmons at infradead.org  Sun Jul 29 17:33:46 2018
From: jsimmons at infradead.org (James Simmons)
Date: Sun, 29 Jul 2018 18:33:46 +0100 (BST)
Subject: [lustre-devel] [PATCH 0/5] lnet: reduce code in lib-socket
In-Reply-To: <153247352950.25051.3479450943810758169.stgit@noble>
References: <153247352950.25051.3479450943810758169.stgit@noble>
Message-ID: <alpine.LFD.2.21.1807291832170.2402@casper.infradead.org>


> lib-socket contains code to iterate over network interfaces and
> determine the IPv4 address of each.  It does this but using
> socket-ioctls.
> There are more direct interfaces within the kernel for
> accessing this information, so change to use those directly.

With an updated based on Doug's reviewer to check the return value
I give this a positive review for the patch series.

Reviewed-by: James Simmons <jsimmons at infradead.org>

> ---
> 
> NeilBrown (5):
>       lustre: socklnd: use for_each_netdev() instead of lnet_ipif_enumerate()
>       lustre: socklnd: use ksocknal_enumerate_interfaces for individual interfaces.
>       lustre: change lnet_ipaddr_enumerate() to use for_each_netdev()
>       lustre: o2iblnd: get IP address more directly.
>       lustre: lnet: remove lnet_ipif_enumerate()
> 
> 
>  .../staging/lustre/include/linux/lnet/lib-lnet.h   |    3 
>  .../staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c    |   48 +++--
>  .../staging/lustre/lnet/klnds/socklnd/socklnd.c    |   96 ++++-----
>  drivers/staging/lustre/lnet/lnet/config.c          |   78 +++----
>  drivers/staging/lustre/lnet/lnet/lib-socket.c      |  211 --------------------
>  5 files changed, 103 insertions(+), 333 deletions(-)
> 
> --
> Signature
> 
> 

From jsimmons at infradead.org  Sun Jul 29 17:34:20 2018
From: jsimmons at infradead.org (James Simmons)
Date: Sun, 29 Jul 2018 18:34:20 +0100 (BST)
Subject: [lustre-devel] [PATCH] lustre: fix inode refcount problem with
 fhandle access
In-Reply-To: <87sh48njns.fsf@notabene.neil.brown.name>
References: <87sh48njns.fsf@notabene.neil.brown.name>
Message-ID: <alpine.LFD.2.21.1807291833580.2402@casper.infradead.org>

> ll_iget_for_nfs() currently calls d_obtain_alias() twice.
> This decrements the refcount on the inode twice, and
> increments it on the dentry twice.
> In each case, only once is correct.
> The result is that the refcounts are wrong and warnings
> result.
> 
> Also, if the first d_obtain_alias() returns an error,
> iput() is called.  As the comment at the second d_obtain_alias()
> explains, this is incorrect.

Fixed my problems!!! Thank you.

Reviewed-by: James Simmons <jsimmons at infradead.org>
 
> Reported-by: James Simmons <jsimmons at infradead.org>
> Signed-off-by: NeilBrown <neilb at suse.com>
> ---
>  drivers/staging/lustre/lustre/llite/llite_nfs.c | 20 +++++++-------------
>  1 file changed, 7 insertions(+), 13 deletions(-)
> 
> diff --git a/drivers/staging/lustre/lustre/llite/llite_nfs.c b/drivers/staging/lustre/lustre/llite/llite_nfs.c
> index 14172688d55f..9efb20e91476 100644
> --- a/drivers/staging/lustre/lustre/llite/llite_nfs.c
> +++ b/drivers/staging/lustre/lustre/llite/llite_nfs.c
> @@ -150,10 +150,8 @@ ll_iget_for_nfs(struct super_block *sb,
>  	}
>  
>  	result = d_obtain_alias(inode);
> -	if (IS_ERR(result)) {
> -		iput(inode);
> +	if (IS_ERR(result))
>  		return result;
> -	}
>  
>  	/**
>  	 * In case d_obtain_alias() found a disconnected dentry, always update
> @@ -168,16 +166,12 @@ ll_iget_for_nfs(struct super_block *sb,
>  		spin_unlock(&lli->lli_lock);
>  	}
>  
> -	/* N.B. d_obtain_alias() drops inode ref on error */
> -	result = d_obtain_alias(inode);
> -	if (!IS_ERR(result)) {
> -		/*
> -		 * Need to signal to the ll_intent_file_open that
> -		 * we came from NFS and so opencache needs to be
> -		 * enabled for this one
> -		 */
> -		ll_d2d(result)->lld_nfs_dentry = 1;
> -	}
> +	/*
> +	 * Need to signal to the ll_intent_file_open that
> +	 * we came from NFS and so opencache needs to be
> +	 * enabled for this one
> +	 */
> +	ll_d2d(result)->lld_nfs_dentry = 1;
>  
>  	return result;
>  }
> -- 
> 2.14.0.rc0.dirty
> 
> 

From neilb at suse.com  Sun Jul 29 23:29:42 2018
From: neilb at suse.com (NeilBrown)
Date: Mon, 30 Jul 2018 09:29:42 +1000
Subject: [lustre-devel] [PATCH 5/9] lustre: move filesystem mounting
 from obdclass to llite
In-Reply-To: <alpine.LFD.2.21.1807291807490.2402@casper.infradead.org>
References: <153232696720.26222.9658151633697867322.stgit@noble>
 <153232698492.26222.12045767311396974106.stgit@noble>
 <alpine.LFD.2.21.1807291807490.2402@casper.infradead.org>
Message-ID: <87y3dtlixl.fsf@notabene.neil.brown.name>

On Sun, Jul 29 2018, James Simmons wrote:

>> All filesystem-api aspects of the lustre filesystem are in
>> llite, except mounting which is in obdclass.
>> This is probably because the name filesystem type is/was
>> used for mounting an lustre filesystem on the client, and
>> for mounting a data-store on the server.
>> This is a confusion that is best discarded and forgotten.
>> 
>> So move the mount handling into llite.
>> 
>> This removes that rather awkward requirment that obdclass needed to
>> request_module() the lustre module, and makes lustre_fill_super()
>> significantly simpler.
>> 
>> Several symbols don't need to be exported any more as they are now
>> part of llite which is the only module that uses them.
>
> Nak. The majority of this code is shared between both server and client
> code. Moving this directly to llite would make server code dependent on
> llite which is what is not desired. Things like lustre_process_log() and
> lustre_end_log() have no client or server specific code and all nodes
> running lustre independent of the function of the node execute these code
> paths. Please see obd_mount_server.c to see what functions are used.
>
> Do you mind if I take a stab at this ?

Sure, go for it.
I assume we agree that the actual mounting belongs in llite, but that
there is other code in obd_mount.c which is not directly related to
mounting, which is used by the server as well.  I agree that such code
needs to be left in obdclass.

Thanks,
NeilBrown


>  
>> Signed-off-by: NeilBrown <neilb at suse.com>
>> ---
>>  drivers/staging/lustre/lustre/include/obd_class.h  |    4 
>>  drivers/staging/lustre/lustre/llite/Makefile       |    2 
>>  .../staging/lustre/lustre/llite/llite_internal.h   |    3 
>>  drivers/staging/lustre/lustre/llite/mount.c        | 1208 +++++++++++++++++++
>>  drivers/staging/lustre/lustre/llite/super25.c      |    8 
>>  drivers/staging/lustre/lustre/obdclass/Makefile    |    2 
>>  drivers/staging/lustre/lustre/obdclass/class_obd.c |    6 
>>  drivers/staging/lustre/lustre/obdclass/obd_mount.c | 1244 --------------------
>>  8 files changed, 1218 insertions(+), 1259 deletions(-)
>>  create mode 100644 drivers/staging/lustre/lustre/llite/mount.c
>>  delete mode 100644 drivers/staging/lustre/lustre/obdclass/obd_mount.c
>> 
>> diff --git a/drivers/staging/lustre/lustre/include/obd_class.h b/drivers/staging/lustre/lustre/include/obd_class.h
>> index 90dbafd38d71..1a55f0215837 100644
>> --- a/drivers/staging/lustre/lustre/include/obd_class.h
>> +++ b/drivers/staging/lustre/lustre/include/obd_class.h
>> @@ -1556,10 +1556,6 @@ struct lwp_register_item {
>>   */
>>  extern int (*ptlrpc_put_connection_superhack)(struct ptlrpc_connection *c);
>>  
>> -/* obd_mount.c */
>> -int lustre_unregister_fs(void);
>> -int lustre_register_fs(void);
>> -
>>  /* sysctl.c */
>>  int obd_sysctl_init(void);
>>  
>> diff --git a/drivers/staging/lustre/lustre/llite/Makefile b/drivers/staging/lustre/lustre/llite/Makefile
>> index 7d0225476362..670f072ce370 100644
>> --- a/drivers/staging/lustre/lustre/llite/Makefile
>> +++ b/drivers/staging/lustre/lustre/llite/Makefile
>> @@ -3,7 +3,7 @@ ccflags-y += -I$(srctree)/drivers/staging/lustre/include
>>  ccflags-y += -I$(srctree)/drivers/staging/lustre/lustre/include
>>  
>>  obj-$(CONFIG_LUSTRE_FS) += lustre.o
>> -lustre-y := dcache.o dir.o file.o llite_lib.o llite_nfs.o \
>> +lustre-y := mount.o dcache.o dir.o file.o llite_lib.o llite_nfs.o \
>>  	    rw.o rw26.o namei.o symlink.o llite_mmap.o range_lock.o \
>>  	    xattr.o xattr_cache.o xattr_security.o \
>>  	    super25.o statahead.o glimpse.o lcommon_cl.o lcommon_misc.o \
>> diff --git a/drivers/staging/lustre/lustre/llite/llite_internal.h b/drivers/staging/lustre/lustre/llite/llite_internal.h
>> index 8399501c9122..1f832a5ba023 100644
>> --- a/drivers/staging/lustre/lustre/llite/llite_internal.h
>> +++ b/drivers/staging/lustre/lustre/llite/llite_internal.h
>> @@ -1351,4 +1351,7 @@ void cl_inode_fini(struct inode *inode);
>>  __u64 cl_fid_build_ino(const struct lu_fid *fid, int api32);
>>  __u32 cl_fid_build_gen(const struct lu_fid *fid);
>>  
>> +int lustre_unregister_fs(void);
>> +int lustre_register_fs(void);
>> +
>>  #endif /* LLITE_INTERNAL_H */
>> diff --git a/drivers/staging/lustre/lustre/llite/mount.c b/drivers/staging/lustre/lustre/llite/mount.c
>> new file mode 100644
>> index 000000000000..58e8b371f0c4
>> --- /dev/null
>> +++ b/drivers/staging/lustre/lustre/llite/mount.c
>> @@ -0,0 +1,1208 @@
>> +// SPDX-License-Identifier: GPL-2.0
>> +/*
>> + * GPL HEADER START
>> + *
>> + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
>> + *
>> + * This program is free software; you can redistribute it and/or modify
>> + * it under the terms of the GNU General Public License version 2 only,
>> + * as published by the Free Software Foundation.
>> + *
>> + * This program is distributed in the hope that it will be useful, but
>> + * WITHOUT ANY WARRANTY; without even the implied warranty of
>> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
>> + * General Public License version 2 for more details (a copy is included
>> + * in the LICENSE file that accompanied this code).
>> + *
>> + * You should have received a copy of the GNU General Public License
>> + * version 2 along with this program; If not, see
>> + * http://www.gnu.org/licenses/gpl-2.0.html
>> + *
>> + * GPL HEADER END
>> + */
>> +/*
>> + * Copyright (c) 2007, 2010, Oracle and/or its affiliates. All rights reserved.
>> + * Use is subject to license terms.
>> + *
>> + * Copyright (c) 2011, 2015, Intel Corporation.
>> + */
>> +/*
>> + * This file is part of Lustre, http://www.lustre.org/
>> + * Lustre is a trademark of Sun Microsystems, Inc.
>> + *
>> + * lustre/obdclass/obd_mount.c
>> + *
>> + * Client mount routines
>> + *
>> + * Author: Nathan Rutman <nathan at clusterfs.com>
>> + */
>> +
>> +#define DEBUG_SUBSYSTEM S_CLASS
>> +#define D_MOUNT (D_SUPER | D_CONFIG/*|D_WARNING */)
>> +#define PRINT_CMD CDEBUG
>> +
>> +#include <linux/random.h>
>> +#include <obd.h>
>> +#include <obd_class.h>
>> +#include <uapi/linux/lustre/lustre_idl.h>
>> +#include <lustre_log.h>
>> +#include <lustre_disk.h>
>> +#include <uapi/linux/lustre/lustre_param.h>
>> +#include <llite_internal.h>
>> +
>> +/**************** config llog ********************/
>> +
>> +/** Get a config log from the MGS and process it.
>> + * This func is called for both clients and servers.
>> + * Continue to process new statements appended to the logs
>> + * (whenever the config lock is revoked) until lustre_end_log
>> + * is called.
>> + * @param sb The superblock is used by the MGC to write to the local copy of
>> + *   the config log
>> + * @param logname The name of the llog to replicate from the MGS
>> + * @param cfg Since the same mgc may be used to follow multiple config logs
>> + *   (e.g. ost1, ost2, client), the config_llog_instance keeps the state for
>> + *   this log, and is added to the mgc's list of logs to follow.
>> + */
>> +int lustre_process_log(struct super_block *sb, char *logname,
>> +		       struct config_llog_instance *cfg)
>> +{
>> +	struct lustre_cfg *lcfg;
>> +	struct lustre_cfg_bufs *bufs;
>> +	struct lustre_sb_info *lsi = s2lsi(sb);
>> +	struct obd_device *mgc = lsi->lsi_mgc;
>> +	int rc;
>> +
>> +	LASSERT(mgc);
>> +	LASSERT(cfg);
>> +
>> +	bufs = kzalloc(sizeof(*bufs), GFP_NOFS);
>> +	if (!bufs)
>> +		return -ENOMEM;
>> +
>> +	/* mgc_process_config */
>> +	lustre_cfg_bufs_reset(bufs, mgc->obd_name);
>> +	lustre_cfg_bufs_set_string(bufs, 1, logname);
>> +	lustre_cfg_bufs_set(bufs, 2, cfg, sizeof(*cfg));
>> +	lustre_cfg_bufs_set(bufs, 3, &sb, sizeof(sb));
>> +	lcfg = kzalloc(lustre_cfg_len(bufs->lcfg_bufcount, bufs->lcfg_buflen),
>> +		       GFP_NOFS);
>> +	if (!lcfg) {
>> +		rc = -ENOMEM;
>> +		goto out;
>> +	}
>> +	lustre_cfg_init(lcfg, LCFG_LOG_START, bufs);
>> +
>> +	rc = obd_process_config(mgc, sizeof(*lcfg), lcfg);
>> +	kfree(lcfg);
>> +out:
>> +	kfree(bufs);
>> +
>> +	if (rc == -EINVAL)
>> +		LCONSOLE_ERROR_MSG(0x15b, "%s: The configuration from log '%s' failed from the MGS (%d).  Make sure this client and the MGS are running compatible versions of Lustre.\n",
>> +				   mgc->obd_name, logname, rc);
>> +
>> +	else if (rc)
>> +		LCONSOLE_ERROR_MSG(0x15c, "%s: The configuration from log '%s' failed (%d). This may be the result of communication errors between this node and the MGS, a bad configuration, or other errors. See the syslog for more information.\n",
>> +				   mgc->obd_name, logname,
>> +				   rc);
>> +
>> +	/* class_obd_list(); */
>> +	return rc;
>> +}
>> +
>> +/* Stop watching this config log for updates */
>> +int lustre_end_log(struct super_block *sb, char *logname,
>> +		   struct config_llog_instance *cfg)
>> +{
>> +	struct lustre_cfg *lcfg;
>> +	struct lustre_cfg_bufs bufs;
>> +	struct lustre_sb_info *lsi = s2lsi(sb);
>> +	struct obd_device *mgc = lsi->lsi_mgc;
>> +	int rc;
>> +
>> +	if (!mgc)
>> +		return -ENOENT;
>> +
>> +	/* mgc_process_config */
>> +	lustre_cfg_bufs_reset(&bufs, mgc->obd_name);
>> +	lustre_cfg_bufs_set_string(&bufs, 1, logname);
>> +	if (cfg)
>> +		lustre_cfg_bufs_set(&bufs, 2, cfg, sizeof(*cfg));
>> +	lcfg = kzalloc(lustre_cfg_len(bufs.lcfg_bufcount, bufs.lcfg_buflen),
>> +		       GFP_NOFS);
>> +	if (!lcfg)
>> +		return -ENOMEM;
>> +	lustre_cfg_init(lcfg, LCFG_LOG_END, &bufs);
>> +
>> +	rc = obd_process_config(mgc, sizeof(*lcfg), lcfg);
>> +	kfree(lcfg);
>> +	return rc;
>> +}
>> +
>> +/**************** obd start *******************/
>> +
>> +/** lustre_cfg_bufs are a holdover from 1.4; we can still set these up from
>> + * lctl (and do for echo cli/srv.
>> + */
>> +static int do_lcfg(char *cfgname, lnet_nid_t nid, int cmd,
>> +		   char *s1, char *s2, char *s3, char *s4)
>> +{
>> +	struct lustre_cfg_bufs bufs;
>> +	struct lustre_cfg     *lcfg = NULL;
>> +	int rc;
>> +
>> +	CDEBUG(D_TRACE, "lcfg %s %#x %s %s %s %s\n", cfgname,
>> +	       cmd, s1, s2, s3, s4);
>> +
>> +	lustre_cfg_bufs_reset(&bufs, cfgname);
>> +	if (s1)
>> +		lustre_cfg_bufs_set_string(&bufs, 1, s1);
>> +	if (s2)
>> +		lustre_cfg_bufs_set_string(&bufs, 2, s2);
>> +	if (s3)
>> +		lustre_cfg_bufs_set_string(&bufs, 3, s3);
>> +	if (s4)
>> +		lustre_cfg_bufs_set_string(&bufs, 4, s4);
>> +
>> +	lcfg = kzalloc(lustre_cfg_len(bufs.lcfg_bufcount, bufs.lcfg_buflen),
>> +		       GFP_NOFS);
>> +	if (!lcfg)
>> +		return -ENOMEM;
>> +	lustre_cfg_init(lcfg, cmd, &bufs);
>> +	lcfg->lcfg_nid = nid;
>> +	rc = class_process_config(lcfg);
>> +	kfree(lcfg);
>> +	return rc;
>> +}
>> +
>> +/** Call class_attach and class_setup.  These methods in turn call
>> + * obd type-specific methods.
>> + */
>> +static int lustre_start_simple(char *obdname, char *type, char *uuid,
>> +			       char *s1, char *s2, char *s3, char *s4)
>> +{
>> +	int rc;
>> +
>> +	CDEBUG(D_MOUNT, "Starting obd %s (typ=%s)\n", obdname, type);
>> +
>> +	rc = do_lcfg(obdname, 0, LCFG_ATTACH, type, uuid, NULL, NULL);
>> +	if (rc) {
>> +		CERROR("%s attach error %d\n", obdname, rc);
>> +		return rc;
>> +	}
>> +	rc = do_lcfg(obdname, 0, LCFG_SETUP, s1, s2, s3, s4);
>> +	if (rc) {
>> +		CERROR("%s setup error %d\n", obdname, rc);
>> +		do_lcfg(obdname, 0, LCFG_DETACH, NULL, NULL, NULL, NULL);
>> +	}
>> +	return rc;
>> +}
>> +
>> +static DEFINE_MUTEX(mgc_start_lock);
>> +
>> +/** Set up a mgc obd to process startup logs
>> + *
>> + * \param sb [in] super block of the mgc obd
>> + *
>> + * \retval 0 success, otherwise error code
>> + */
>> +int lustre_start_mgc(struct super_block *sb)
>> +{
>> +	struct obd_connect_data *data = NULL;
>> +	struct lustre_sb_info *lsi = s2lsi(sb);
>> +	struct obd_device *obd;
>> +	struct obd_export *exp;
>> +	struct obd_uuid *uuid;
>> +	class_uuid_t uuidc;
>> +	lnet_nid_t nid;
>> +	char nidstr[LNET_NIDSTR_SIZE];
>> +	char *mgcname = NULL, *niduuid = NULL, *mgssec = NULL;
>> +	char *ptr;
>> +	int rc = 0, i = 0, j;
>> +
>> +	LASSERT(lsi->lsi_lmd);
>> +
>> +	/* Use nids from mount line: uml1,1 at elan:uml2,2 at elan:/lustre */
>> +	ptr = lsi->lsi_lmd->lmd_dev;
>> +	if (class_parse_nid(ptr, &nid, &ptr) == 0)
>> +		i++;
>> +	if (i == 0) {
>> +		CERROR("No valid MGS nids found.\n");
>> +		return -EINVAL;
>> +	}
>> +
>> +	mutex_lock(&mgc_start_lock);
>> +
>> +	libcfs_nid2str_r(nid, nidstr, sizeof(nidstr));
>> +	mgcname = kasprintf(GFP_NOFS,
>> +			    "%s%s", LUSTRE_MGC_OBDNAME, nidstr);
>> +	niduuid = kasprintf(GFP_NOFS, "%s_%x", mgcname, 0);
>> +	if (!mgcname || !niduuid) {
>> +		rc = -ENOMEM;
>> +		goto out_free;
>> +	}
>> +
>> +	mgssec = lsi->lsi_lmd->lmd_mgssec ? lsi->lsi_lmd->lmd_mgssec : "";
>> +
>> +	data = kzalloc(sizeof(*data), GFP_NOFS);
>> +	if (!data) {
>> +		rc = -ENOMEM;
>> +		goto out_free;
>> +	}
>> +
>> +	obd = class_name2obd(mgcname);
>> +	if (obd && !obd->obd_stopping) {
>> +		int recov_bk;
>> +
>> +		rc = obd_set_info_async(NULL, obd->obd_self_export,
>> +					strlen(KEY_MGSSEC), KEY_MGSSEC,
>> +					strlen(mgssec), mgssec, NULL);
>> +		if (rc)
>> +			goto out_free;
>> +
>> +		/* Re-using an existing MGC */
>> +		atomic_inc(&obd->u.cli.cl_mgc_refcount);
>> +
>> +		/* IR compatibility check, only for clients */
>> +		if (lmd_is_client(lsi->lsi_lmd)) {
>> +			int has_ir;
>> +			int vallen = sizeof(*data);
>> +			__u32 *flags = &lsi->lsi_lmd->lmd_flags;
>> +
>> +			rc = obd_get_info(NULL, obd->obd_self_export,
>> +					  strlen(KEY_CONN_DATA), KEY_CONN_DATA,
>> +					  &vallen, data);
>> +			LASSERT(rc == 0);
>> +			has_ir = OCD_HAS_FLAG(data, IMP_RECOV);
>> +			if (has_ir ^ !(*flags & LMD_FLG_NOIR)) {
>> +				/* LMD_FLG_NOIR is for test purpose only */
>> +				LCONSOLE_WARN(
>> +					"Trying to mount a client with IR setting not compatible with current mgc. Force to use current mgc setting that is IR %s.\n",
>> +					has_ir ? "enabled" : "disabled");
>> +				if (has_ir)
>> +					*flags &= ~LMD_FLG_NOIR;
>> +				else
>> +					*flags |= LMD_FLG_NOIR;
>> +			}
>> +		}
>> +
>> +		recov_bk = 0;
>> +
>> +		/* Try all connections, but only once (again).
>> +		 * We don't want to block another target from starting
>> +		 * (using its local copy of the log), but we do want to connect
>> +		 * if at all possible.
>> +		 */
>> +		recov_bk++;
>> +		CDEBUG(D_MOUNT, "%s: Set MGC reconnect %d\n", mgcname,
>> +		       recov_bk);
>> +		rc = obd_set_info_async(NULL, obd->obd_self_export,
>> +					sizeof(KEY_INIT_RECOV_BACKUP),
>> +					KEY_INIT_RECOV_BACKUP,
>> +					sizeof(recov_bk), &recov_bk, NULL);
>> +		rc = 0;
>> +		goto out;
>> +	}
>> +
>> +	CDEBUG(D_MOUNT, "Start MGC '%s'\n", mgcname);
>> +
>> +	/* Add the primary nids for the MGS */
>> +	i = 0;
>> +	/* Use nids from mount line: uml1,1 at elan:uml2,2 at elan:/lustre */
>> +	ptr = lsi->lsi_lmd->lmd_dev;
>> +	while (class_parse_nid(ptr, &nid, &ptr) == 0) {
>> +		rc = do_lcfg(mgcname, nid,
>> +			     LCFG_ADD_UUID, niduuid, NULL, NULL, NULL);
>> +		if (!rc)
>> +			i++;
>> +		/* Stop at the first failover nid */
>> +		if (*ptr == ':')
>> +			break;
>> +	}
>> +	if (i == 0) {
>> +		CERROR("No valid MGS nids found.\n");
>> +		rc = -EINVAL;
>> +		goto out_free;
>> +	}
>> +	lsi->lsi_lmd->lmd_mgs_failnodes = 1;
>> +
>> +	/* Random uuid for MGC allows easier reconnects */
>> +	uuid = kzalloc(sizeof(*uuid), GFP_NOFS);
>> +	if (!uuid) {
>> +		rc = -ENOMEM;
>> +		goto out_free;
>> +	}
>> +
>> +	ll_generate_random_uuid(uuidc);
>> +	class_uuid_unparse(uuidc, uuid);
>> +
>> +	/* Start the MGC */
>> +	rc = lustre_start_simple(mgcname, LUSTRE_MGC_NAME,
>> +				 (char *)uuid->uuid, LUSTRE_MGS_OBDNAME,
>> +				 niduuid, NULL, NULL);
>> +	kfree(uuid);
>> +	if (rc)
>> +		goto out_free;
>> +
>> +	/* Add any failover MGS nids */
>> +	i = 1;
>> +	while (ptr && ((*ptr == ':' ||
>> +			class_find_param(ptr, PARAM_MGSNODE, &ptr) == 0))) {
>> +		/* New failover node */
>> +		sprintf(niduuid, "%s_%x", mgcname, i);
>> +		j = 0;
>> +		while (class_parse_nid_quiet(ptr, &nid, &ptr) == 0) {
>> +			rc = do_lcfg(mgcname, nid, LCFG_ADD_UUID, niduuid,
>> +				     NULL, NULL, NULL);
>> +			if (!rc)
>> +				++j;
>> +			if (*ptr == ':')
>> +				break;
>> +		}
>> +		if (j > 0) {
>> +			rc = do_lcfg(mgcname, 0, LCFG_ADD_CONN,
>> +				     niduuid, NULL, NULL, NULL);
>> +			if (!rc)
>> +				i++;
>> +		} else {
>> +			/* at ":/fsname" */
>> +			break;
>> +		}
>> +	}
>> +	lsi->lsi_lmd->lmd_mgs_failnodes = i;
>> +
>> +	obd = class_name2obd(mgcname);
>> +	if (!obd) {
>> +		CERROR("Can't find mgcobd %s\n", mgcname);
>> +		rc = -ENOTCONN;
>> +		goto out_free;
>> +	}
>> +
>> +	rc = obd_set_info_async(NULL, obd->obd_self_export,
>> +				strlen(KEY_MGSSEC), KEY_MGSSEC,
>> +				strlen(mgssec), mgssec, NULL);
>> +	if (rc)
>> +		goto out_free;
>> +
>> +	/* Keep a refcount of servers/clients who started with "mount",
>> +	 * so we know when we can get rid of the mgc.
>> +	 */
>> +	atomic_set(&obd->u.cli.cl_mgc_refcount, 1);
>> +
>> +	/* We connect to the MGS at setup, and don't disconnect until cleanup */
>> +	data->ocd_connect_flags = OBD_CONNECT_VERSION | OBD_CONNECT_AT |
>> +				  OBD_CONNECT_FULL20 | OBD_CONNECT_IMP_RECOV |
>> +				  OBD_CONNECT_LVB_TYPE | OBD_CONNECT_BULK_MBITS;
>> +
>> +#if OBD_OCD_VERSION(3, 0, 53, 0) > LUSTRE_VERSION_CODE
>> +	data->ocd_connect_flags |= OBD_CONNECT_MNE_SWAB;
>> +#endif
>> +
>> +	if (lmd_is_client(lsi->lsi_lmd) &&
>> +	    lsi->lsi_lmd->lmd_flags & LMD_FLG_NOIR)
>> +		data->ocd_connect_flags &= ~OBD_CONNECT_IMP_RECOV;
>> +	data->ocd_version = LUSTRE_VERSION_CODE;
>> +	rc = obd_connect(NULL, &exp, obd, &obd->obd_uuid, data, NULL);
>> +	if (rc) {
>> +		CERROR("connect failed %d\n", rc);
>> +		goto out;
>> +	}
>> +
>> +	obd->u.cli.cl_mgc_mgsexp = exp;
>> +
>> +out:
>> +	/* Keep the mgc info in the sb. Note that many lsi's can point
>> +	 * to the same mgc.
>> +	 */
>> +	lsi->lsi_mgc = obd;
>> +out_free:
>> +	mutex_unlock(&mgc_start_lock);
>> +
>> +	kfree(data);
>> +	kfree(mgcname);
>> +	kfree(niduuid);
>> +	return rc;
>> +}
>> +
>> +static int lustre_stop_mgc(struct super_block *sb)
>> +{
>> +	struct lustre_sb_info *lsi = s2lsi(sb);
>> +	struct obd_device *obd;
>> +	char *niduuid = NULL, *ptr = NULL;
>> +	int i, rc = 0, len = 0;
>> +
>> +	if (!lsi)
>> +		return -ENOENT;
>> +	obd = lsi->lsi_mgc;
>> +	if (!obd)
>> +		return -ENOENT;
>> +	lsi->lsi_mgc = NULL;
>> +
>> +	mutex_lock(&mgc_start_lock);
>> +	LASSERT(atomic_read(&obd->u.cli.cl_mgc_refcount) > 0);
>> +	if (!atomic_dec_and_test(&obd->u.cli.cl_mgc_refcount)) {
>> +		/* This is not fatal, every client that stops
>> +		 * will call in here.
>> +		 */
>> +		CDEBUG(D_MOUNT, "mgc still has %d references.\n",
>> +		       atomic_read(&obd->u.cli.cl_mgc_refcount));
>> +		rc = -EBUSY;
>> +		goto out;
>> +	}
>> +
>> +	/* The MGC has no recoverable data in any case.
>> +	 * force shutdown set in umount_begin
>> +	 */
>> +	obd->obd_no_recov = 1;
>> +
>> +	if (obd->u.cli.cl_mgc_mgsexp) {
>> +		/* An error is not fatal, if we are unable to send the
>> +		 * disconnect mgs ping evictor cleans up the export
>> +		 */
>> +		rc = obd_disconnect(obd->u.cli.cl_mgc_mgsexp);
>> +		if (rc)
>> +			CDEBUG(D_MOUNT, "disconnect failed %d\n", rc);
>> +	}
>> +
>> +	/* Save the obdname for cleaning the nid uuids, which are obdname_XX */
>> +	len = strlen(obd->obd_name) + 6;
>> +	niduuid = kzalloc(len, GFP_NOFS);
>> +	if (niduuid) {
>> +		strcpy(niduuid, obd->obd_name);
>> +		ptr = niduuid + strlen(niduuid);
>> +	}
>> +
>> +	rc = class_manual_cleanup(obd);
>> +	if (rc)
>> +		goto out;
>> +
>> +	/* Clean the nid uuids */
>> +	if (!niduuid) {
>> +		rc = -ENOMEM;
>> +		goto out;
>> +	}
>> +
>> +	for (i = 0; i < lsi->lsi_lmd->lmd_mgs_failnodes; i++) {
>> +		sprintf(ptr, "_%x", i);
>> +		rc = do_lcfg(LUSTRE_MGC_OBDNAME, 0, LCFG_DEL_UUID,
>> +			     niduuid, NULL, NULL, NULL);
>> +		if (rc)
>> +			CERROR("del MDC UUID %s failed: rc = %d\n",
>> +			       niduuid, rc);
>> +	}
>> +out:
>> +	kfree(niduuid);
>> +
>> +	/* class_import_put will get rid of the additional connections */
>> +	mutex_unlock(&mgc_start_lock);
>> +	return rc;
>> +}
>> +
>> +/***************** lustre superblock **************/
>> +
>> +static struct lustre_sb_info *lustre_init_lsi(struct super_block *sb)
>> +{
>> +	struct lustre_sb_info *lsi;
>> +
>> +	lsi = kzalloc(sizeof(*lsi), GFP_NOFS);
>> +	if (!lsi)
>> +		return NULL;
>> +	lsi->lsi_lmd = kzalloc(sizeof(*lsi->lsi_lmd), GFP_NOFS);
>> +	if (!lsi->lsi_lmd) {
>> +		kfree(lsi);
>> +		return NULL;
>> +	}
>> +
>> +	lsi->lsi_lmd->lmd_exclude_count = 0;
>> +	lsi->lsi_lmd->lmd_recovery_time_soft = 0;
>> +	lsi->lsi_lmd->lmd_recovery_time_hard = 0;
>> +	s2lsi_nocast(sb) = lsi;
>> +	/* we take 1 extra ref for our setup */
>> +	atomic_set(&lsi->lsi_mounts, 1);
>> +
>> +	/* Default umount style */
>> +	lsi->lsi_flags = LSI_UMOUNT_FAILOVER;
>> +
>> +	return lsi;
>> +}
>> +
>> +static int lustre_free_lsi(struct super_block *sb)
>> +{
>> +	struct lustre_sb_info *lsi = s2lsi(sb);
>> +
>> +	CDEBUG(D_MOUNT, "Freeing lsi %p\n", lsi);
>> +
>> +	/* someone didn't call server_put_mount. */
>> +	LASSERT(atomic_read(&lsi->lsi_mounts) == 0);
>> +
>> +	if (lsi->lsi_lmd) {
>> +		kfree(lsi->lsi_lmd->lmd_dev);
>> +		kfree(lsi->lsi_lmd->lmd_profile);
>> +		kfree(lsi->lsi_lmd->lmd_fileset);
>> +		kfree(lsi->lsi_lmd->lmd_mgssec);
>> +		kfree(lsi->lsi_lmd->lmd_opts);
>> +		if (lsi->lsi_lmd->lmd_exclude_count)
>> +			kfree(lsi->lsi_lmd->lmd_exclude);
>> +		kfree(lsi->lsi_lmd->lmd_mgs);
>> +		kfree(lsi->lsi_lmd->lmd_osd_type);
>> +		kfree(lsi->lsi_lmd->lmd_params);
>> +
>> +		kfree(lsi->lsi_lmd);
>> +	}
>> +
>> +	LASSERT(!lsi->lsi_llsbi);
>> +	kfree(lsi);
>> +	s2lsi_nocast(sb) = NULL;
>> +
>> +	return 0;
>> +}
>> +
>> +/* The lsi has one reference for every server that is using the disk -
>> + * e.g. MDT, MGS, and potentially MGC
>> + */
>> +static int lustre_put_lsi(struct super_block *sb)
>> +{
>> +	struct lustre_sb_info *lsi = s2lsi(sb);
>> +
>> +	CDEBUG(D_MOUNT, "put %p %d\n", sb, atomic_read(&lsi->lsi_mounts));
>> +	if (atomic_dec_and_test(&lsi->lsi_mounts)) {
>> +		lustre_free_lsi(sb);
>> +		return 1;
>> +	}
>> +	return 0;
>> +}
>> +
>> +/*************** mount common between server and client ***************/
>> +
>> +/* Common umount */
>> +int lustre_common_put_super(struct super_block *sb)
>> +{
>> +	int rc;
>> +
>> +	CDEBUG(D_MOUNT, "dropping sb %p\n", sb);
>> +
>> +	/* Drop a ref to the MGC */
>> +	rc = lustre_stop_mgc(sb);
>> +	if (rc && (rc != -ENOENT)) {
>> +		if (rc != -EBUSY) {
>> +			CERROR("Can't stop MGC: %d\n", rc);
>> +			return rc;
>> +		}
>> +		/* BUSY just means that there's some other obd that
>> +		 * needs the mgc.  Let him clean it up.
>> +		 */
>> +		CDEBUG(D_MOUNT, "MGC still in use\n");
>> +	}
>> +	/* Drop a ref to the mounted disk */
>> +	lustre_put_lsi(sb);
>> +	return rc;
>> +}
>> +
>> +static void lmd_print(struct lustre_mount_data *lmd)
>> +{
>> +	int i;
>> +
>> +	PRINT_CMD(D_MOUNT, "  mount data:\n");
>> +	if (lmd_is_client(lmd))
>> +		PRINT_CMD(D_MOUNT, "profile: %s\n", lmd->lmd_profile);
>> +	PRINT_CMD(D_MOUNT, "device:  %s\n", lmd->lmd_dev);
>> +	PRINT_CMD(D_MOUNT, "flags:   %x\n", lmd->lmd_flags);
>> +
>> +	if (lmd->lmd_opts)
>> +		PRINT_CMD(D_MOUNT, "options: %s\n", lmd->lmd_opts);
>> +
>> +	if (lmd->lmd_recovery_time_soft)
>> +		PRINT_CMD(D_MOUNT, "recovery time soft: %d\n",
>> +			  lmd->lmd_recovery_time_soft);
>> +
>> +	if (lmd->lmd_recovery_time_hard)
>> +		PRINT_CMD(D_MOUNT, "recovery time hard: %d\n",
>> +			  lmd->lmd_recovery_time_hard);
>> +
>> +	for (i = 0; i < lmd->lmd_exclude_count; i++) {
>> +		PRINT_CMD(D_MOUNT, "exclude %d:  OST%04x\n", i,
>> +			  lmd->lmd_exclude[i]);
>> +	}
>> +}
>> +
>> +/* mount -v  -o exclude=lustre-OST0001:lustre-OST0002 -t lustre ... */
>> +static int lmd_make_exclusion(struct lustre_mount_data *lmd, const char *ptr)
>> +{
>> +	const char *s1 = ptr, *s2;
>> +	__u32 index = 0, *exclude_list;
>> +	int rc = 0, devmax;
>> +
>> +	/* The shortest an ost name can be is 8 chars: -OST0000.
>> +	 * We don't actually know the fsname at this time, so in fact
>> +	 * a user could specify any fsname.
>> +	 */
>> +	devmax = strlen(ptr) / 8 + 1;
>> +
>> +	/* temp storage until we figure out how many we have */
>> +	exclude_list = kcalloc(devmax, sizeof(index), GFP_NOFS);
>> +	if (!exclude_list)
>> +		return -ENOMEM;
>> +
>> +	/* we enter this fn pointing at the '=' */
>> +	while (*s1 && *s1 != ' ' && *s1 != ',') {
>> +		s1++;
>> +		rc = server_name2index(s1, &index, &s2);
>> +		if (rc < 0) {
>> +			CERROR("Can't parse server name '%s': rc = %d\n",
>> +			       s1, rc);
>> +			break;
>> +		}
>> +		if (rc == LDD_F_SV_TYPE_OST)
>> +			exclude_list[lmd->lmd_exclude_count++] = index;
>> +		else
>> +			CDEBUG(D_MOUNT, "ignoring exclude %.*s: type = %#x\n",
>> +			       (uint)(s2 - s1), s1, rc);
>> +		s1 = s2;
>> +		/* now we are pointing at ':' (next exclude)
>> +		 * or ',' (end of excludes)
>> +		 */
>> +		if (lmd->lmd_exclude_count >= devmax)
>> +			break;
>> +	}
>> +	if (rc >= 0) /* non-err */
>> +		rc = 0;
>> +
>> +	if (lmd->lmd_exclude_count) {
>> +		/* permanent, freed in lustre_free_lsi */
>> +		lmd->lmd_exclude = kcalloc(lmd->lmd_exclude_count,
>> +					   sizeof(index), GFP_NOFS);
>> +		if (lmd->lmd_exclude) {
>> +			memcpy(lmd->lmd_exclude, exclude_list,
>> +			       sizeof(index) * lmd->lmd_exclude_count);
>> +		} else {
>> +			rc = -ENOMEM;
>> +			lmd->lmd_exclude_count = 0;
>> +		}
>> +	}
>> +	kfree(exclude_list);
>> +	return rc;
>> +}
>> +
>> +static int lmd_parse_mgssec(struct lustre_mount_data *lmd, char *ptr)
>> +{
>> +	char   *tail;
>> +	int     length;
>> +
>> +	kfree(lmd->lmd_mgssec);
>> +	lmd->lmd_mgssec = NULL;
>> +
>> +	tail = strchr(ptr, ',');
>> +	if (!tail)
>> +		length = strlen(ptr);
>> +	else
>> +		length = tail - ptr;
>> +
>> +	lmd->lmd_mgssec = kzalloc(length + 1, GFP_NOFS);
>> +	if (!lmd->lmd_mgssec)
>> +		return -ENOMEM;
>> +
>> +	memcpy(lmd->lmd_mgssec, ptr, length);
>> +	lmd->lmd_mgssec[length] = '\0';
>> +	return 0;
>> +}
>> +
>> +static int lmd_parse_string(char **handle, char *ptr)
>> +{
>> +	char   *tail;
>> +	int     length;
>> +
>> +	if (!handle || !ptr)
>> +		return -EINVAL;
>> +
>> +	kfree(*handle);
>> +	*handle = NULL;
>> +
>> +	tail = strchr(ptr, ',');
>> +	if (!tail)
>> +		length = strlen(ptr);
>> +	else
>> +		length = tail - ptr;
>> +
>> +	*handle = kzalloc(length + 1, GFP_NOFS);
>> +	if (!*handle)
>> +		return -ENOMEM;
>> +
>> +	memcpy(*handle, ptr, length);
>> +	(*handle)[length] = '\0';
>> +
>> +	return 0;
>> +}
>> +
>> +/* Collect multiple values for mgsnid specifiers */
>> +static int lmd_parse_mgs(struct lustre_mount_data *lmd, char **ptr)
>> +{
>> +	lnet_nid_t nid;
>> +	char *tail = *ptr;
>> +	char *mgsnid;
>> +	int   length;
>> +	int   oldlen = 0;
>> +
>> +	/* Find end of nidlist */
>> +	while (class_parse_nid_quiet(tail, &nid, &tail) == 0)
>> +		;
>> +	length = tail - *ptr;
>> +	if (length == 0) {
>> +		LCONSOLE_ERROR_MSG(0x159, "Can't parse NID '%s'\n", *ptr);
>> +		return -EINVAL;
>> +	}
>> +
>> +	if (lmd->lmd_mgs)
>> +		oldlen = strlen(lmd->lmd_mgs) + 1;
>> +
>> +	mgsnid = kzalloc(oldlen + length + 1, GFP_NOFS);
>> +	if (!mgsnid)
>> +		return -ENOMEM;
>> +
>> +	if (lmd->lmd_mgs) {
>> +		/* Multiple mgsnid= are taken to mean failover locations */
>> +		memcpy(mgsnid, lmd->lmd_mgs, oldlen);
>> +		mgsnid[oldlen - 1] = ':';
>> +		kfree(lmd->lmd_mgs);
>> +	}
>> +	memcpy(mgsnid + oldlen, *ptr, length);
>> +	mgsnid[oldlen + length] = '\0';
>> +	lmd->lmd_mgs = mgsnid;
>> +	*ptr = tail;
>> +
>> +	return 0;
>> +}
>> +
>> +/**
>> + * Find the first delimiter (comma or colon) from the specified \a buf and
>> + * make \a *endh point to the string starting with the delimiter. The commas
>> + * in expression list [...] will be skipped.
>> + *
>> + * @buf		a delimiter-separated string
>> + * @endh	a pointer to a pointer that will point to the string
>> + *		starting with the delimiter
>> + *
>> + * RETURNS	true if delimiter is found, false if delimiter is not found
>> + */
>> +static bool lmd_find_delimiter(char *buf, char **endh)
>> +{
>> +	char *c = buf;
>> +	size_t pos;
>> +	bool found;
>> +
>> +	if (!buf)
>> +		return false;
>> +try_again:
>> +	if (*c == ',' || *c == ':')
>> +		return true;
>> +
>> +	pos = strcspn(c, "[:,]");
>> +	if (!pos)
>> +		return false;
>> +
>> +	/* Not a valid mount string */
>> +	if (*c == ']') {
>> +		CWARN("invalid mount string format\n");
>> +		return false;
>> +	}
>> +
>> +	c += pos;
>> +	if (*c == '[') {
>> +		c = strchr(c, ']');
>> +
>> +		/* invalid mount string */
>> +		if (!c) {
>> +			CWARN("invalid mount string format\n");
>> +			return false;
>> +		}
>> +		c++;
>> +		goto try_again;
>> +	}
>> +
>> +	found = *c != '\0';
>> +	if (found && endh)
>> +		*endh = c;
>> +
>> +	return found;
>> +}
>> +
>> +/**
>> + * Find the first valid string delimited by comma or colon from the specified
>> + * \a buf and parse it to see whether it's a valid nid list. If yes, \a *endh
>> + * will point to the next string starting with the delimiter.
>> + *
>> + * \param[in] buf	a delimiter-separated string
>> + * \param[in] endh	a pointer to a pointer that will point to the string
>> + *			starting with the delimiter
>> + *
>> + * \retval 0		if the string is a valid nid list
>> + * \retval 1		if the string is not a valid nid list
>> + */
>> +static int lmd_parse_nidlist(char *buf, char **endh)
>> +{
>> +	struct list_head nidlist;
>> +	char *endp = buf;
>> +	int rc = 0;
>> +	char tmp;
>> +
>> +	if (!buf)
>> +		return 1;
>> +	while (*buf == ',' || *buf == ':')
>> +		buf++;
>> +	if (*buf == ' ' || *buf == '/' || *buf == '\0')
>> +		return 1;
>> +
>> +	if (!lmd_find_delimiter(buf, &endp))
>> +		endp = buf + strlen(buf);
>> +
>> +	tmp = *endp;
>> +	*endp = '\0';
>> +
>> +	INIT_LIST_HEAD(&nidlist);
>> +	if (cfs_parse_nidlist(buf, strlen(buf), &nidlist) <= 0)
>> +		rc = 1;
>> +	cfs_free_nidlist(&nidlist);
>> +
>> +	*endp = tmp;
>> +	if (rc)
>> +		return rc;
>> +	if (endh)
>> +		*endh = endp;
>> +	return 0;
>> +}
>> +
>> +/** Parse mount line options
>> + * e.g. mount -v -t lustre -o abort_recov uml1:uml2:/lustre-client /mnt/lustre
>> + * dev is passed as device=uml1:/lustre by mount.lustre
>> + */
>> +static int lmd_parse(char *options, struct lustre_mount_data *lmd)
>> +{
>> +	char *s1, *s2, *devname = NULL;
>> +	struct lustre_mount_data *raw = (struct lustre_mount_data *)options;
>> +	int rc = 0;
>> +
>> +	LASSERT(lmd);
>> +	if (!options) {
>> +		LCONSOLE_ERROR_MSG(0x162, "Missing mount data: check that /sbin/mount.lustre is installed.\n");
>> +		return -EINVAL;
>> +	}
>> +
>> +	/* Options should be a string - try to detect old lmd data */
>> +	if ((raw->lmd_magic & 0xffffff00) == (LMD_MAGIC & 0xffffff00)) {
>> +		LCONSOLE_ERROR_MSG(0x163, "You're using an old version of /sbin/mount.lustre.  Please install version %s\n",
>> +				   LUSTRE_VERSION_STRING);
>> +		return -EINVAL;
>> +	}
>> +	lmd->lmd_magic = LMD_MAGIC;
>> +
>> +	lmd->lmd_params = kzalloc(LMD_PARAMS_MAXLEN, GFP_NOFS);
>> +	if (!lmd->lmd_params)
>> +		return -ENOMEM;
>> +	lmd->lmd_params[0] = '\0';
>> +
>> +	/* Set default flags here */
>> +
>> +	s1 = options;
>> +	while (*s1) {
>> +		int clear = 0;
>> +		int time_min = OBD_RECOVERY_TIME_MIN;
>> +		char *s3;
>> +
>> +		/* Skip whitespace and extra commas */
>> +		while (*s1 == ' ' || *s1 == ',')
>> +			s1++;
>> +		s3 = s1;
>> +
>> +		/* Client options are parsed in ll_options: eg. flock,
>> +		 * user_xattr, acl
>> +		 */
>> +
>> +		/* Parse non-ldiskfs options here. Rather than modifying
>> +		 * ldiskfs, we just zero these out here
>> +		 */
>> +		if (strncmp(s1, "abort_recov", 11) == 0) {
>> +			lmd->lmd_flags |= LMD_FLG_ABORT_RECOV;
>> +			clear++;
>> +		} else if (strncmp(s1, "recovery_time_soft=", 19) == 0) {
>> +			lmd->lmd_recovery_time_soft = max_t(int,
>> +				simple_strtoul(s1 + 19, NULL, 10), time_min);
>> +			clear++;
>> +		} else if (strncmp(s1, "recovery_time_hard=", 19) == 0) {
>> +			lmd->lmd_recovery_time_hard = max_t(int,
>> +				simple_strtoul(s1 + 19, NULL, 10), time_min);
>> +			clear++;
>> +		} else if (strncmp(s1, "noir", 4) == 0) {
>> +			lmd->lmd_flags |= LMD_FLG_NOIR; /* test purpose only. */
>> +			clear++;
>> +		} else if (strncmp(s1, "nosvc", 5) == 0) {
>> +			lmd->lmd_flags |= LMD_FLG_NOSVC;
>> +			clear++;
>> +		} else if (strncmp(s1, "nomgs", 5) == 0) {
>> +			lmd->lmd_flags |= LMD_FLG_NOMGS;
>> +			clear++;
>> +		} else if (strncmp(s1, "noscrub", 7) == 0) {
>> +			lmd->lmd_flags |= LMD_FLG_NOSCRUB;
>> +			clear++;
>> +		} else if (strncmp(s1, PARAM_MGSNODE,
>> +				   sizeof(PARAM_MGSNODE) - 1) == 0) {
>> +			s2 = s1 + sizeof(PARAM_MGSNODE) - 1;
>> +			/* Assume the next mount opt is the first
>> +			 * invalid nid we get to.
>> +			 */
>> +			rc = lmd_parse_mgs(lmd, &s2);
>> +			if (rc)
>> +				goto invalid;
>> +			clear++;
>> +		} else if (strncmp(s1, "writeconf", 9) == 0) {
>> +			lmd->lmd_flags |= LMD_FLG_WRITECONF;
>> +			clear++;
>> +		} else if (strncmp(s1, "update", 6) == 0) {
>> +			lmd->lmd_flags |= LMD_FLG_UPDATE;
>> +			clear++;
>> +		} else if (strncmp(s1, "virgin", 6) == 0) {
>> +			lmd->lmd_flags |= LMD_FLG_VIRGIN;
>> +			clear++;
>> +		} else if (strncmp(s1, "noprimnode", 10) == 0) {
>> +			lmd->lmd_flags |= LMD_FLG_NO_PRIMNODE;
>> +			clear++;
>> +		} else if (strncmp(s1, "mgssec=", 7) == 0) {
>> +			rc = lmd_parse_mgssec(lmd, s1 + 7);
>> +			if (rc)
>> +				goto invalid;
>> +			s3 = s2;
>> +			clear++;
>> +		/* ost exclusion list */
>> +		} else if (strncmp(s1, "exclude=", 8) == 0) {
>> +			rc = lmd_make_exclusion(lmd, s1 + 7);
>> +			if (rc)
>> +				goto invalid;
>> +			clear++;
>> +		} else if (strncmp(s1, "mgs", 3) == 0) {
>> +			/* We are an MGS */
>> +			lmd->lmd_flags |= LMD_FLG_MGS;
>> +			clear++;
>> +		} else if (strncmp(s1, "svname=", 7) == 0) {
>> +			rc = lmd_parse_string(&lmd->lmd_profile, s1 + 7);
>> +			if (rc)
>> +				goto invalid;
>> +			clear++;
>> +		} else if (strncmp(s1, "param=", 6) == 0) {
>> +			size_t length, params_length;
>> +			char *tail = s1;
>> +
>> +			if (lmd_find_delimiter(s1 + 6, &tail)) {
>> +				char *param_str = tail + 1;
>> +				int supplementary = 1;
>> +
>> +				while (!lmd_parse_nidlist(param_str,
>> +							  &param_str))
>> +					supplementary = 0;
>> +				length = param_str - s1 - supplementary;
>> +			} else {
>> +				length = strlen(s1);
>> +			}
>> +			length -= 6;
>> +			params_length = strlen(lmd->lmd_params);
>> +			if (params_length + length + 1 >= LMD_PARAMS_MAXLEN)
>> +				return -E2BIG;
>> +			strncat(lmd->lmd_params, s1 + 6, length);
>> +			lmd->lmd_params[params_length + length] = '\0';
>> +			strlcat(lmd->lmd_params, " ", LMD_PARAMS_MAXLEN);
>> +			s3 = s1 + 6 + length;
>> +			clear++;
>> +		} else if (strncmp(s1, "osd=", 4) == 0) {
>> +			rc = lmd_parse_string(&lmd->lmd_osd_type, s1 + 4);
>> +			if (rc)
>> +				goto invalid;
>> +			clear++;
>> +		}
>> +		/* Linux 2.4 doesn't pass the device, so we stuck it at the
>> +		 * end of the options.
>> +		 */
>> +		else if (strncmp(s1, "device=", 7) == 0) {
>> +			devname = s1 + 7;
>> +			/* terminate options right before device.  device
>> +			 * must be the last one.
>> +			 */
>> +			*s1 = '\0';
>> +			break;
>> +		}
>> +
>> +		/* Find next opt */
>> +		s2 = strchr(s1, ',');
>> +		if (!s2) {
>> +			if (clear)
>> +				*s1 = '\0';
>> +			break;
>> +		}
>> +		s2++;
>> +		if (clear)
>> +			memmove(s1, s2, strlen(s2) + 1);
>> +		else
>> +			s1 = s2;
>> +	}
>> +
>> +	if (!devname) {
>> +		LCONSOLE_ERROR_MSG(0x164, "Can't find the device name (need mount option 'device=...')\n");
>> +		goto invalid;
>> +	}
>> +
>> +	s1 = strstr(devname, ":/");
>> +	if (s1) {
>> +		++s1;
>> +		lmd->lmd_flags |= LMD_FLG_CLIENT;
>> +		/* Remove leading /s from fsname */
>> +		while (*++s1 == '/')
>> +			;
>> +		s2 = strchrnul(s1, '/');
>> +		/* Freed in lustre_free_lsi */
>> +		lmd->lmd_profile = kasprintf(GFP_KERNEL, "%.*s-client",
>> +					     (int)(s2 - s1), s1);
>> +		if (!lmd->lmd_profile)
>> +			return -ENOMEM;
>> +
>> +		s1 = s2;
>> +		s2 = s1 + strlen(s1) - 1;
>> +		/* Remove padding /s from fileset */
>> +		while (*s2 == '/')
>> +			s2--;
>> +		if (s2 > s1) {
>> +			lmd->lmd_fileset = kstrndup(s1, s2 - s1 + 1,
>> +						    GFP_KERNEL);
>> +			if (!lmd->lmd_fileset)
>> +				return -ENOMEM;
>> +		}
>> +	}
>> +
>> +	/* Freed in lustre_free_lsi */
>> +	lmd->lmd_dev = kstrdup(devname, GFP_KERNEL);
>> +	if (!lmd->lmd_dev)
>> +		return -ENOMEM;
>> +
>> +	/* Save mount options */
>> +	s1 = options + strlen(options) - 1;
>> +	while (s1 >= options && (*s1 == ',' || *s1 == ' '))
>> +		*s1-- = 0;
>> +	if (*options != 0) {
>> +		/* Freed in lustre_free_lsi */
>> +		lmd->lmd_opts = kstrdup(options, GFP_KERNEL);
>> +		if (!lmd->lmd_opts)
>> +			return -ENOMEM;
>> +	}
>> +
>> +	lmd_print(lmd);
>> +	lmd->lmd_magic = LMD_MAGIC;
>> +
>> +	return rc;
>> +
>> +invalid:
>> +	CERROR("Bad mount options %s\n", options);
>> +	return -EINVAL;
>> +}
>> +
>> +/** This is the entry point for the mount call into Lustre.
>> + * This is called when a server or client is mounted,
>> + * and this is where we start setting things up.
>> + * @param data Mount options (e.g. -o flock,abort_recov)
>> + */
>> +static int lustre_fill_super(struct super_block *sb, void *lmd2_data, int silent)
>> +{
>> +	struct lustre_mount_data *lmd;
>> +	struct lustre_sb_info *lsi;
>> +	int rc;
>> +
>> +	CDEBUG(D_MOUNT | D_VFSTRACE, "VFS Op: sb %p\n", sb);
>> +
>> +	lsi = lustre_init_lsi(sb);
>> +	if (!lsi)
>> +		return -ENOMEM;
>> +	lmd = lsi->lsi_lmd;
>> +
>> +	/*
>> +	 * Disable lockdep during mount, because mount locking patterns are
>> +	 * `special'.
>> +	 */
>> +	lockdep_off();
>> +
>> +	/*
>> +	 * LU-639: the obd cleanup of last mount may not finish yet, wait here.
>> +	 */
>> +	obd_zombie_barrier();
>> +
>> +	/* Figure out the lmd from the mount options */
>> +	if (lmd_parse(lmd2_data, lmd)) {
>> +		lustre_put_lsi(sb);
>> +		rc = -EINVAL;
>> +		goto out;
>> +	}
>> +
>> +	if (lmd_is_client(lmd)) {
>> +		CDEBUG(D_MOUNT, "Mounting client %s\n", lmd->lmd_profile);
>> +
>> +		rc = lustre_start_mgc(sb);
>> +		if (rc) {
>> +			lustre_common_put_super(sb);
>> +			goto out;
>> +		}
>> +		/* Connect and start */
>> +		rc = ll_fill_super(sb);
>> +		/* l_f_s will call lustre_common_put_super on failure, otherwise
>> +		 * l_f_s will have taken another reference to the module */
>> +	} else {
>> +		CERROR("This is client-side-only module, cannot handle server mount.\n");
>> +		rc = -EINVAL;
>> +	}
>> +
>> +	/* If error happens in fill_super() call, @lsi will be killed there.
>> +	 * This is why we do not put it here.
>> +	 */
>> +	goto out;
>> +out:
>> +	if (rc) {
>> +		CERROR("Unable to mount %s (%d)\n",
>> +		       s2lsi(sb) ? lmd->lmd_dev : "", rc);
>> +	} else {
>> +		CDEBUG(D_SUPER, "Mount %s complete\n",
>> +		       lmd->lmd_dev);
>> +	}
>> +	lockdep_on();
>> +	return rc;
>> +}
>> +
>> +/***************** FS registration ******************/
>> +static struct dentry *lustre_mount(struct file_system_type *fs_type, int flags,
>> +				   const char *devname, void *data)
>> +{
>> +	return mount_nodev(fs_type, flags, data, lustre_fill_super);
>> +}
>> +
>> +static void lustre_kill_super(struct super_block *sb)
>> +{
>> +	struct lustre_sb_info *lsi = s2lsi(sb);
>> +
>> +	if (lsi)
>> +		ll_kill_super(sb);
>> +
>> +	kill_anon_super(sb);
>> +}
>> +
>> +/** Register the "lustre" fs type
>> + */
>> +static struct file_system_type lustre_fs_type = {
>> +	.owner		= THIS_MODULE,
>> +	.name		= "lustre",
>> +	.mount		= lustre_mount,
>> +	.kill_sb	= lustre_kill_super,
>> +	.fs_flags	= FS_RENAME_DOES_D_MOVE,
>> +};
>> +MODULE_ALIAS_FS("lustre");
>> +
>> +int lustre_register_fs(void)
>> +{
>> +	return register_filesystem(&lustre_fs_type);
>> +}
>> +
>> +int lustre_unregister_fs(void)
>> +{
>> +	return unregister_filesystem(&lustre_fs_type);
>> +}
>> diff --git a/drivers/staging/lustre/lustre/llite/super25.c b/drivers/staging/lustre/lustre/llite/super25.c
>> index d335f29556c2..cb1e12e8097a 100644
>> --- a/drivers/staging/lustre/lustre/llite/super25.c
>> +++ b/drivers/staging/lustre/lustre/llite/super25.c
>> @@ -145,10 +145,11 @@ static int __init lustre_init(void)
>>  	if (rc != 0)
>>  		goto out_inode_fini_env;
>>  
>> -	lustre_register_super_ops(THIS_MODULE, ll_fill_super, ll_kill_super);
>>  	lustre_register_client_process_config(ll_process_config);
>>  
>> -	return 0;
>> +	rc = lustre_register_fs();
>> +
>> +	return rc;
>>  
>>  out_inode_fini_env:
>>  	cl_env_put(cl_inode_fini_env, &cl_inode_fini_refcheck);
>> @@ -166,7 +167,8 @@ static int __init lustre_init(void)
>>  
>>  static void __exit lustre_exit(void)
>>  {
>> -	lustre_register_super_ops(NULL, NULL, NULL);
>> +	lustre_unregister_fs();
>> +
>>  	lustre_register_client_process_config(NULL);
>>  
>>  	debugfs_remove(llite_root);
>> diff --git a/drivers/staging/lustre/lustre/obdclass/Makefile b/drivers/staging/lustre/lustre/obdclass/Makefile
>> index 686bd546f06d..2b99514a4966 100644
>> --- a/drivers/staging/lustre/lustre/obdclass/Makefile
>> +++ b/drivers/staging/lustre/lustre/obdclass/Makefile
>> @@ -8,5 +8,5 @@ obdclass-y := module.o sysctl.o \
>>  	      llog.o llog_cat.o llog_obd.o llog_swab.o class_obd.o debug.o \
>>  	      genops.o uuid.o lprocfs_status.o lprocfs_counters.o \
>>  	      lustre_handles.o lustre_peer.o statfs_pack.o linkea.o \
>> -	      obdo.o obd_config.o obd_mount.o lu_object.o lu_ref.o \
>> +	      obdo.o obd_config.o lu_object.o lu_ref.o \
>>  	      cl_object.o cl_page.o cl_lock.o cl_io.o kernelcomm.o
>> diff --git a/drivers/staging/lustre/lustre/obdclass/class_obd.c b/drivers/staging/lustre/lustre/obdclass/class_obd.c
>> index 81a4c666bb69..cdaf7293bc18 100644
>> --- a/drivers/staging/lustre/lustre/obdclass/class_obd.c
>> +++ b/drivers/staging/lustre/lustre/obdclass/class_obd.c
>> @@ -510,18 +510,12 @@ static int __init obdclass_init(void)
>>  		return err;
>>  
>>  	err = llog_info_init();
>> -	if (err)
>> -		return err;
>> -
>> -	err = lustre_register_fs();
>>  
>>  	return err;
>>  }
>>  
>>  static void obdclass_exit(void)
>>  {
>> -	lustre_unregister_fs();
>> -
>>  	misc_deregister(&obd_psdev);
>>  	llog_info_fini();
>>  	cl_global_fini();
>> diff --git a/drivers/staging/lustre/lustre/obdclass/obd_mount.c b/drivers/staging/lustre/lustre/obdclass/obd_mount.c
>> deleted file mode 100644
>> index 4e4a8c35105b..000000000000
>> --- a/drivers/staging/lustre/lustre/obdclass/obd_mount.c
>> +++ /dev/null
>> @@ -1,1244 +0,0 @@
>> -// SPDX-License-Identifier: GPL-2.0
>> -/*
>> - * GPL HEADER START
>> - *
>> - * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
>> - *
>> - * This program is free software; you can redistribute it and/or modify
>> - * it under the terms of the GNU General Public License version 2 only,
>> - * as published by the Free Software Foundation.
>> - *
>> - * This program is distributed in the hope that it will be useful, but
>> - * WITHOUT ANY WARRANTY; without even the implied warranty of
>> - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
>> - * General Public License version 2 for more details (a copy is included
>> - * in the LICENSE file that accompanied this code).
>> - *
>> - * You should have received a copy of the GNU General Public License
>> - * version 2 along with this program; If not, see
>> - * http://www.gnu.org/licenses/gpl-2.0.html
>> - *
>> - * GPL HEADER END
>> - */
>> -/*
>> - * Copyright (c) 2007, 2010, Oracle and/or its affiliates. All rights reserved.
>> - * Use is subject to license terms.
>> - *
>> - * Copyright (c) 2011, 2015, Intel Corporation.
>> - */
>> -/*
>> - * This file is part of Lustre, http://www.lustre.org/
>> - * Lustre is a trademark of Sun Microsystems, Inc.
>> - *
>> - * lustre/obdclass/obd_mount.c
>> - *
>> - * Client mount routines
>> - *
>> - * Author: Nathan Rutman <nathan at clusterfs.com>
>> - */
>> -
>> -#define DEBUG_SUBSYSTEM S_CLASS
>> -#define D_MOUNT (D_SUPER | D_CONFIG/*|D_WARNING */)
>> -#define PRINT_CMD CDEBUG
>> -
>> -#include <linux/random.h>
>> -#include <obd.h>
>> -#include <obd_class.h>
>> -#include <uapi/linux/lustre/lustre_idl.h>
>> -#include <lustre_log.h>
>> -#include <lustre_disk.h>
>> -#include <uapi/linux/lustre/lustre_param.h>
>> -
>> -static DEFINE_SPINLOCK(client_lock);
>> -static struct module *client_mod;
>> -static int (*client_fill_super)(struct super_block *sb);
>> -static void (*kill_super_cb)(struct super_block *sb);
>> -
>> -/**************** config llog ********************/
>> -
>> -/** Get a config log from the MGS and process it.
>> - * This func is called for both clients and servers.
>> - * Continue to process new statements appended to the logs
>> - * (whenever the config lock is revoked) until lustre_end_log
>> - * is called.
>> - * @param sb The superblock is used by the MGC to write to the local copy of
>> - *   the config log
>> - * @param logname The name of the llog to replicate from the MGS
>> - * @param cfg Since the same mgc may be used to follow multiple config logs
>> - *   (e.g. ost1, ost2, client), the config_llog_instance keeps the state for
>> - *   this log, and is added to the mgc's list of logs to follow.
>> - */
>> -int lustre_process_log(struct super_block *sb, char *logname,
>> -		       struct config_llog_instance *cfg)
>> -{
>> -	struct lustre_cfg *lcfg;
>> -	struct lustre_cfg_bufs *bufs;
>> -	struct lustre_sb_info *lsi = s2lsi(sb);
>> -	struct obd_device *mgc = lsi->lsi_mgc;
>> -	int rc;
>> -
>> -	LASSERT(mgc);
>> -	LASSERT(cfg);
>> -
>> -	bufs = kzalloc(sizeof(*bufs), GFP_NOFS);
>> -	if (!bufs)
>> -		return -ENOMEM;
>> -
>> -	/* mgc_process_config */
>> -	lustre_cfg_bufs_reset(bufs, mgc->obd_name);
>> -	lustre_cfg_bufs_set_string(bufs, 1, logname);
>> -	lustre_cfg_bufs_set(bufs, 2, cfg, sizeof(*cfg));
>> -	lustre_cfg_bufs_set(bufs, 3, &sb, sizeof(sb));
>> -	lcfg = kzalloc(lustre_cfg_len(bufs->lcfg_bufcount, bufs->lcfg_buflen),
>> -		       GFP_NOFS);
>> -	if (!lcfg) {
>> -		rc = -ENOMEM;
>> -		goto out;
>> -	}
>> -	lustre_cfg_init(lcfg, LCFG_LOG_START, bufs);
>> -
>> -	rc = obd_process_config(mgc, sizeof(*lcfg), lcfg);
>> -	kfree(lcfg);
>> -out:
>> -	kfree(bufs);
>> -
>> -	if (rc == -EINVAL)
>> -		LCONSOLE_ERROR_MSG(0x15b, "%s: The configuration from log '%s' failed from the MGS (%d).  Make sure this client and the MGS are running compatible versions of Lustre.\n",
>> -				   mgc->obd_name, logname, rc);
>> -
>> -	else if (rc)
>> -		LCONSOLE_ERROR_MSG(0x15c, "%s: The configuration from log '%s' failed (%d). This may be the result of communication errors between this node and the MGS, a bad configuration, or other errors. See the syslog for more information.\n",
>> -				   mgc->obd_name, logname,
>> -				   rc);
>> -
>> -	/* class_obd_list(); */
>> -	return rc;
>> -}
>> -EXPORT_SYMBOL(lustre_process_log);
>> -
>> -/* Stop watching this config log for updates */
>> -int lustre_end_log(struct super_block *sb, char *logname,
>> -		   struct config_llog_instance *cfg)
>> -{
>> -	struct lustre_cfg *lcfg;
>> -	struct lustre_cfg_bufs bufs;
>> -	struct lustre_sb_info *lsi = s2lsi(sb);
>> -	struct obd_device *mgc = lsi->lsi_mgc;
>> -	int rc;
>> -
>> -	if (!mgc)
>> -		return -ENOENT;
>> -
>> -	/* mgc_process_config */
>> -	lustre_cfg_bufs_reset(&bufs, mgc->obd_name);
>> -	lustre_cfg_bufs_set_string(&bufs, 1, logname);
>> -	if (cfg)
>> -		lustre_cfg_bufs_set(&bufs, 2, cfg, sizeof(*cfg));
>> -	lcfg = kzalloc(lustre_cfg_len(bufs.lcfg_bufcount, bufs.lcfg_buflen),
>> -		       GFP_NOFS);
>> -	if (!lcfg)
>> -		return -ENOMEM;
>> -	lustre_cfg_init(lcfg, LCFG_LOG_END, &bufs);
>> -
>> -	rc = obd_process_config(mgc, sizeof(*lcfg), lcfg);
>> -	kfree(lcfg);
>> -	return rc;
>> -}
>> -EXPORT_SYMBOL(lustre_end_log);
>> -
>> -/**************** obd start *******************/
>> -
>> -/** lustre_cfg_bufs are a holdover from 1.4; we can still set these up from
>> - * lctl (and do for echo cli/srv.
>> - */
>> -static int do_lcfg(char *cfgname, lnet_nid_t nid, int cmd,
>> -		   char *s1, char *s2, char *s3, char *s4)
>> -{
>> -	struct lustre_cfg_bufs bufs;
>> -	struct lustre_cfg     *lcfg = NULL;
>> -	int rc;
>> -
>> -	CDEBUG(D_TRACE, "lcfg %s %#x %s %s %s %s\n", cfgname,
>> -	       cmd, s1, s2, s3, s4);
>> -
>> -	lustre_cfg_bufs_reset(&bufs, cfgname);
>> -	if (s1)
>> -		lustre_cfg_bufs_set_string(&bufs, 1, s1);
>> -	if (s2)
>> -		lustre_cfg_bufs_set_string(&bufs, 2, s2);
>> -	if (s3)
>> -		lustre_cfg_bufs_set_string(&bufs, 3, s3);
>> -	if (s4)
>> -		lustre_cfg_bufs_set_string(&bufs, 4, s4);
>> -
>> -	lcfg = kzalloc(lustre_cfg_len(bufs.lcfg_bufcount, bufs.lcfg_buflen),
>> -		       GFP_NOFS);
>> -	if (!lcfg)
>> -		return -ENOMEM;
>> -	lustre_cfg_init(lcfg, cmd, &bufs);
>> -	lcfg->lcfg_nid = nid;
>> -	rc = class_process_config(lcfg);
>> -	kfree(lcfg);
>> -	return rc;
>> -}
>> -
>> -/** Call class_attach and class_setup.  These methods in turn call
>> - * obd type-specific methods.
>> - */
>> -static int lustre_start_simple(char *obdname, char *type, char *uuid,
>> -			       char *s1, char *s2, char *s3, char *s4)
>> -{
>> -	int rc;
>> -
>> -	CDEBUG(D_MOUNT, "Starting obd %s (typ=%s)\n", obdname, type);
>> -
>> -	rc = do_lcfg(obdname, 0, LCFG_ATTACH, type, uuid, NULL, NULL);
>> -	if (rc) {
>> -		CERROR("%s attach error %d\n", obdname, rc);
>> -		return rc;
>> -	}
>> -	rc = do_lcfg(obdname, 0, LCFG_SETUP, s1, s2, s3, s4);
>> -	if (rc) {
>> -		CERROR("%s setup error %d\n", obdname, rc);
>> -		do_lcfg(obdname, 0, LCFG_DETACH, NULL, NULL, NULL, NULL);
>> -	}
>> -	return rc;
>> -}
>> -
>> -static DEFINE_MUTEX(mgc_start_lock);
>> -
>> -/** Set up a mgc obd to process startup logs
>> - *
>> - * \param sb [in] super block of the mgc obd
>> - *
>> - * \retval 0 success, otherwise error code
>> - */
>> -int lustre_start_mgc(struct super_block *sb)
>> -{
>> -	struct obd_connect_data *data = NULL;
>> -	struct lustre_sb_info *lsi = s2lsi(sb);
>> -	struct obd_device *obd;
>> -	struct obd_export *exp;
>> -	struct obd_uuid *uuid;
>> -	class_uuid_t uuidc;
>> -	lnet_nid_t nid;
>> -	char nidstr[LNET_NIDSTR_SIZE];
>> -	char *mgcname = NULL, *niduuid = NULL, *mgssec = NULL;
>> -	char *ptr;
>> -	int rc = 0, i = 0, j;
>> -
>> -	LASSERT(lsi->lsi_lmd);
>> -
>> -	/* Use nids from mount line: uml1,1 at elan:uml2,2 at elan:/lustre */
>> -	ptr = lsi->lsi_lmd->lmd_dev;
>> -	if (class_parse_nid(ptr, &nid, &ptr) == 0)
>> -		i++;
>> -	if (i == 0) {
>> -		CERROR("No valid MGS nids found.\n");
>> -		return -EINVAL;
>> -	}
>> -
>> -	mutex_lock(&mgc_start_lock);
>> -
>> -	libcfs_nid2str_r(nid, nidstr, sizeof(nidstr));
>> -	mgcname = kasprintf(GFP_NOFS,
>> -			    "%s%s", LUSTRE_MGC_OBDNAME, nidstr);
>> -	niduuid = kasprintf(GFP_NOFS, "%s_%x", mgcname, 0);
>> -	if (!mgcname || !niduuid) {
>> -		rc = -ENOMEM;
>> -		goto out_free;
>> -	}
>> -
>> -	mgssec = lsi->lsi_lmd->lmd_mgssec ? lsi->lsi_lmd->lmd_mgssec : "";
>> -
>> -	data = kzalloc(sizeof(*data), GFP_NOFS);
>> -	if (!data) {
>> -		rc = -ENOMEM;
>> -		goto out_free;
>> -	}
>> -
>> -	obd = class_name2obd(mgcname);
>> -	if (obd && !obd->obd_stopping) {
>> -		int recov_bk;
>> -
>> -		rc = obd_set_info_async(NULL, obd->obd_self_export,
>> -					strlen(KEY_MGSSEC), KEY_MGSSEC,
>> -					strlen(mgssec), mgssec, NULL);
>> -		if (rc)
>> -			goto out_free;
>> -
>> -		/* Re-using an existing MGC */
>> -		atomic_inc(&obd->u.cli.cl_mgc_refcount);
>> -
>> -		/* IR compatibility check, only for clients */
>> -		if (lmd_is_client(lsi->lsi_lmd)) {
>> -			int has_ir;
>> -			int vallen = sizeof(*data);
>> -			__u32 *flags = &lsi->lsi_lmd->lmd_flags;
>> -
>> -			rc = obd_get_info(NULL, obd->obd_self_export,
>> -					  strlen(KEY_CONN_DATA), KEY_CONN_DATA,
>> -					  &vallen, data);
>> -			LASSERT(rc == 0);
>> -			has_ir = OCD_HAS_FLAG(data, IMP_RECOV);
>> -			if (has_ir ^ !(*flags & LMD_FLG_NOIR)) {
>> -				/* LMD_FLG_NOIR is for test purpose only */
>> -				LCONSOLE_WARN(
>> -					"Trying to mount a client with IR setting not compatible with current mgc. Force to use current mgc setting that is IR %s.\n",
>> -					has_ir ? "enabled" : "disabled");
>> -				if (has_ir)
>> -					*flags &= ~LMD_FLG_NOIR;
>> -				else
>> -					*flags |= LMD_FLG_NOIR;
>> -			}
>> -		}
>> -
>> -		recov_bk = 0;
>> -
>> -		/* Try all connections, but only once (again).
>> -		 * We don't want to block another target from starting
>> -		 * (using its local copy of the log), but we do want to connect
>> -		 * if at all possible.
>> -		 */
>> -		recov_bk++;
>> -		CDEBUG(D_MOUNT, "%s: Set MGC reconnect %d\n", mgcname,
>> -		       recov_bk);
>> -		rc = obd_set_info_async(NULL, obd->obd_self_export,
>> -					sizeof(KEY_INIT_RECOV_BACKUP),
>> -					KEY_INIT_RECOV_BACKUP,
>> -					sizeof(recov_bk), &recov_bk, NULL);
>> -		rc = 0;
>> -		goto out;
>> -	}
>> -
>> -	CDEBUG(D_MOUNT, "Start MGC '%s'\n", mgcname);
>> -
>> -	/* Add the primary nids for the MGS */
>> -	i = 0;
>> -	/* Use nids from mount line: uml1,1 at elan:uml2,2 at elan:/lustre */
>> -	ptr = lsi->lsi_lmd->lmd_dev;
>> -	while (class_parse_nid(ptr, &nid, &ptr) == 0) {
>> -		rc = do_lcfg(mgcname, nid,
>> -			     LCFG_ADD_UUID, niduuid, NULL, NULL, NULL);
>> -		if (!rc)
>> -			i++;
>> -		/* Stop at the first failover nid */
>> -		if (*ptr == ':')
>> -			break;
>> -	}
>> -	if (i == 0) {
>> -		CERROR("No valid MGS nids found.\n");
>> -		rc = -EINVAL;
>> -		goto out_free;
>> -	}
>> -	lsi->lsi_lmd->lmd_mgs_failnodes = 1;
>> -
>> -	/* Random uuid for MGC allows easier reconnects */
>> -	uuid = kzalloc(sizeof(*uuid), GFP_NOFS);
>> -	if (!uuid) {
>> -		rc = -ENOMEM;
>> -		goto out_free;
>> -	}
>> -
>> -	ll_generate_random_uuid(uuidc);
>> -	class_uuid_unparse(uuidc, uuid);
>> -
>> -	/* Start the MGC */
>> -	rc = lustre_start_simple(mgcname, LUSTRE_MGC_NAME,
>> -				 (char *)uuid->uuid, LUSTRE_MGS_OBDNAME,
>> -				 niduuid, NULL, NULL);
>> -	kfree(uuid);
>> -	if (rc)
>> -		goto out_free;
>> -
>> -	/* Add any failover MGS nids */
>> -	i = 1;
>> -	while (ptr && ((*ptr == ':' ||
>> -			class_find_param(ptr, PARAM_MGSNODE, &ptr) == 0))) {
>> -		/* New failover node */
>> -		sprintf(niduuid, "%s_%x", mgcname, i);
>> -		j = 0;
>> -		while (class_parse_nid_quiet(ptr, &nid, &ptr) == 0) {
>> -			rc = do_lcfg(mgcname, nid, LCFG_ADD_UUID, niduuid,
>> -				     NULL, NULL, NULL);
>> -			if (!rc)
>> -				++j;
>> -			if (*ptr == ':')
>> -				break;
>> -		}
>> -		if (j > 0) {
>> -			rc = do_lcfg(mgcname, 0, LCFG_ADD_CONN,
>> -				     niduuid, NULL, NULL, NULL);
>> -			if (!rc)
>> -				i++;
>> -		} else {
>> -			/* at ":/fsname" */
>> -			break;
>> -		}
>> -	}
>> -	lsi->lsi_lmd->lmd_mgs_failnodes = i;
>> -
>> -	obd = class_name2obd(mgcname);
>> -	if (!obd) {
>> -		CERROR("Can't find mgcobd %s\n", mgcname);
>> -		rc = -ENOTCONN;
>> -		goto out_free;
>> -	}
>> -
>> -	rc = obd_set_info_async(NULL, obd->obd_self_export,
>> -				strlen(KEY_MGSSEC), KEY_MGSSEC,
>> -				strlen(mgssec), mgssec, NULL);
>> -	if (rc)
>> -		goto out_free;
>> -
>> -	/* Keep a refcount of servers/clients who started with "mount",
>> -	 * so we know when we can get rid of the mgc.
>> -	 */
>> -	atomic_set(&obd->u.cli.cl_mgc_refcount, 1);
>> -
>> -	/* We connect to the MGS at setup, and don't disconnect until cleanup */
>> -	data->ocd_connect_flags = OBD_CONNECT_VERSION | OBD_CONNECT_AT |
>> -				  OBD_CONNECT_FULL20 | OBD_CONNECT_IMP_RECOV |
>> -				  OBD_CONNECT_LVB_TYPE | OBD_CONNECT_BULK_MBITS;
>> -
>> -#if OBD_OCD_VERSION(3, 0, 53, 0) > LUSTRE_VERSION_CODE
>> -	data->ocd_connect_flags |= OBD_CONNECT_MNE_SWAB;
>> -#endif
>> -
>> -	if (lmd_is_client(lsi->lsi_lmd) &&
>> -	    lsi->lsi_lmd->lmd_flags & LMD_FLG_NOIR)
>> -		data->ocd_connect_flags &= ~OBD_CONNECT_IMP_RECOV;
>> -	data->ocd_version = LUSTRE_VERSION_CODE;
>> -	rc = obd_connect(NULL, &exp, obd, &obd->obd_uuid, data, NULL);
>> -	if (rc) {
>> -		CERROR("connect failed %d\n", rc);
>> -		goto out;
>> -	}
>> -
>> -	obd->u.cli.cl_mgc_mgsexp = exp;
>> -
>> -out:
>> -	/* Keep the mgc info in the sb. Note that many lsi's can point
>> -	 * to the same mgc.
>> -	 */
>> -	lsi->lsi_mgc = obd;
>> -out_free:
>> -	mutex_unlock(&mgc_start_lock);
>> -
>> -	kfree(data);
>> -	kfree(mgcname);
>> -	kfree(niduuid);
>> -	return rc;
>> -}
>> -
>> -static int lustre_stop_mgc(struct super_block *sb)
>> -{
>> -	struct lustre_sb_info *lsi = s2lsi(sb);
>> -	struct obd_device *obd;
>> -	char *niduuid = NULL, *ptr = NULL;
>> -	int i, rc = 0, len = 0;
>> -
>> -	if (!lsi)
>> -		return -ENOENT;
>> -	obd = lsi->lsi_mgc;
>> -	if (!obd)
>> -		return -ENOENT;
>> -	lsi->lsi_mgc = NULL;
>> -
>> -	mutex_lock(&mgc_start_lock);
>> -	LASSERT(atomic_read(&obd->u.cli.cl_mgc_refcount) > 0);
>> -	if (!atomic_dec_and_test(&obd->u.cli.cl_mgc_refcount)) {
>> -		/* This is not fatal, every client that stops
>> -		 * will call in here.
>> -		 */
>> -		CDEBUG(D_MOUNT, "mgc still has %d references.\n",
>> -		       atomic_read(&obd->u.cli.cl_mgc_refcount));
>> -		rc = -EBUSY;
>> -		goto out;
>> -	}
>> -
>> -	/* The MGC has no recoverable data in any case.
>> -	 * force shutdown set in umount_begin
>> -	 */
>> -	obd->obd_no_recov = 1;
>> -
>> -	if (obd->u.cli.cl_mgc_mgsexp) {
>> -		/* An error is not fatal, if we are unable to send the
>> -		 * disconnect mgs ping evictor cleans up the export
>> -		 */
>> -		rc = obd_disconnect(obd->u.cli.cl_mgc_mgsexp);
>> -		if (rc)
>> -			CDEBUG(D_MOUNT, "disconnect failed %d\n", rc);
>> -	}
>> -
>> -	/* Save the obdname for cleaning the nid uuids, which are obdname_XX */
>> -	len = strlen(obd->obd_name) + 6;
>> -	niduuid = kzalloc(len, GFP_NOFS);
>> -	if (niduuid) {
>> -		strcpy(niduuid, obd->obd_name);
>> -		ptr = niduuid + strlen(niduuid);
>> -	}
>> -
>> -	rc = class_manual_cleanup(obd);
>> -	if (rc)
>> -		goto out;
>> -
>> -	/* Clean the nid uuids */
>> -	if (!niduuid) {
>> -		rc = -ENOMEM;
>> -		goto out;
>> -	}
>> -
>> -	for (i = 0; i < lsi->lsi_lmd->lmd_mgs_failnodes; i++) {
>> -		sprintf(ptr, "_%x", i);
>> -		rc = do_lcfg(LUSTRE_MGC_OBDNAME, 0, LCFG_DEL_UUID,
>> -			     niduuid, NULL, NULL, NULL);
>> -		if (rc)
>> -			CERROR("del MDC UUID %s failed: rc = %d\n",
>> -			       niduuid, rc);
>> -	}
>> -out:
>> -	kfree(niduuid);
>> -
>> -	/* class_import_put will get rid of the additional connections */
>> -	mutex_unlock(&mgc_start_lock);
>> -	return rc;
>> -}
>> -
>> -/***************** lustre superblock **************/
>> -
>> -static struct lustre_sb_info *lustre_init_lsi(struct super_block *sb)
>> -{
>> -	struct lustre_sb_info *lsi;
>> -
>> -	lsi = kzalloc(sizeof(*lsi), GFP_NOFS);
>> -	if (!lsi)
>> -		return NULL;
>> -	lsi->lsi_lmd = kzalloc(sizeof(*lsi->lsi_lmd), GFP_NOFS);
>> -	if (!lsi->lsi_lmd) {
>> -		kfree(lsi);
>> -		return NULL;
>> -	}
>> -
>> -	lsi->lsi_lmd->lmd_exclude_count = 0;
>> -	lsi->lsi_lmd->lmd_recovery_time_soft = 0;
>> -	lsi->lsi_lmd->lmd_recovery_time_hard = 0;
>> -	s2lsi_nocast(sb) = lsi;
>> -	/* we take 1 extra ref for our setup */
>> -	atomic_set(&lsi->lsi_mounts, 1);
>> -
>> -	/* Default umount style */
>> -	lsi->lsi_flags = LSI_UMOUNT_FAILOVER;
>> -
>> -	return lsi;
>> -}
>> -
>> -static int lustre_free_lsi(struct super_block *sb)
>> -{
>> -	struct lustre_sb_info *lsi = s2lsi(sb);
>> -
>> -	CDEBUG(D_MOUNT, "Freeing lsi %p\n", lsi);
>> -
>> -	/* someone didn't call server_put_mount. */
>> -	LASSERT(atomic_read(&lsi->lsi_mounts) == 0);
>> -
>> -	if (lsi->lsi_lmd) {
>> -		kfree(lsi->lsi_lmd->lmd_dev);
>> -		kfree(lsi->lsi_lmd->lmd_profile);
>> -		kfree(lsi->lsi_lmd->lmd_fileset);
>> -		kfree(lsi->lsi_lmd->lmd_mgssec);
>> -		kfree(lsi->lsi_lmd->lmd_opts);
>> -		if (lsi->lsi_lmd->lmd_exclude_count)
>> -			kfree(lsi->lsi_lmd->lmd_exclude);
>> -		kfree(lsi->lsi_lmd->lmd_mgs);
>> -		kfree(lsi->lsi_lmd->lmd_osd_type);
>> -		kfree(lsi->lsi_lmd->lmd_params);
>> -
>> -		kfree(lsi->lsi_lmd);
>> -	}
>> -
>> -	LASSERT(!lsi->lsi_llsbi);
>> -	kfree(lsi);
>> -	s2lsi_nocast(sb) = NULL;
>> -
>> -	return 0;
>> -}
>> -
>> -/* The lsi has one reference for every server that is using the disk -
>> - * e.g. MDT, MGS, and potentially MGC
>> - */
>> -static int lustre_put_lsi(struct super_block *sb)
>> -{
>> -	struct lustre_sb_info *lsi = s2lsi(sb);
>> -
>> -	CDEBUG(D_MOUNT, "put %p %d\n", sb, atomic_read(&lsi->lsi_mounts));
>> -	if (atomic_dec_and_test(&lsi->lsi_mounts)) {
>> -		lustre_free_lsi(sb);
>> -		return 1;
>> -	}
>> -	return 0;
>> -}
>> -
>> -/*************** mount common between server and client ***************/
>> -
>> -/* Common umount */
>> -int lustre_common_put_super(struct super_block *sb)
>> -{
>> -	int rc;
>> -
>> -	CDEBUG(D_MOUNT, "dropping sb %p\n", sb);
>> -
>> -	/* Drop a ref to the MGC */
>> -	rc = lustre_stop_mgc(sb);
>> -	if (rc && (rc != -ENOENT)) {
>> -		if (rc != -EBUSY) {
>> -			CERROR("Can't stop MGC: %d\n", rc);
>> -			return rc;
>> -		}
>> -		/* BUSY just means that there's some other obd that
>> -		 * needs the mgc.  Let him clean it up.
>> -		 */
>> -		CDEBUG(D_MOUNT, "MGC still in use\n");
>> -	}
>> -	/* Drop a ref to the mounted disk */
>> -	lustre_put_lsi(sb);
>> -	return rc;
>> -}
>> -EXPORT_SYMBOL(lustre_common_put_super);
>> -
>> -static void lmd_print(struct lustre_mount_data *lmd)
>> -{
>> -	int i;
>> -
>> -	PRINT_CMD(D_MOUNT, "  mount data:\n");
>> -	if (lmd_is_client(lmd))
>> -		PRINT_CMD(D_MOUNT, "profile: %s\n", lmd->lmd_profile);
>> -	PRINT_CMD(D_MOUNT, "device:  %s\n", lmd->lmd_dev);
>> -	PRINT_CMD(D_MOUNT, "flags:   %x\n", lmd->lmd_flags);
>> -
>> -	if (lmd->lmd_opts)
>> -		PRINT_CMD(D_MOUNT, "options: %s\n", lmd->lmd_opts);
>> -
>> -	if (lmd->lmd_recovery_time_soft)
>> -		PRINT_CMD(D_MOUNT, "recovery time soft: %d\n",
>> -			  lmd->lmd_recovery_time_soft);
>> -
>> -	if (lmd->lmd_recovery_time_hard)
>> -		PRINT_CMD(D_MOUNT, "recovery time hard: %d\n",
>> -			  lmd->lmd_recovery_time_hard);
>> -
>> -	for (i = 0; i < lmd->lmd_exclude_count; i++) {
>> -		PRINT_CMD(D_MOUNT, "exclude %d:  OST%04x\n", i,
>> -			  lmd->lmd_exclude[i]);
>> -	}
>> -}
>> -
>> -/* mount -v  -o exclude=lustre-OST0001:lustre-OST0002 -t lustre ... */
>> -static int lmd_make_exclusion(struct lustre_mount_data *lmd, const char *ptr)
>> -{
>> -	const char *s1 = ptr, *s2;
>> -	__u32 index = 0, *exclude_list;
>> -	int rc = 0, devmax;
>> -
>> -	/* The shortest an ost name can be is 8 chars: -OST0000.
>> -	 * We don't actually know the fsname at this time, so in fact
>> -	 * a user could specify any fsname.
>> -	 */
>> -	devmax = strlen(ptr) / 8 + 1;
>> -
>> -	/* temp storage until we figure out how many we have */
>> -	exclude_list = kcalloc(devmax, sizeof(index), GFP_NOFS);
>> -	if (!exclude_list)
>> -		return -ENOMEM;
>> -
>> -	/* we enter this fn pointing at the '=' */
>> -	while (*s1 && *s1 != ' ' && *s1 != ',') {
>> -		s1++;
>> -		rc = server_name2index(s1, &index, &s2);
>> -		if (rc < 0) {
>> -			CERROR("Can't parse server name '%s': rc = %d\n",
>> -			       s1, rc);
>> -			break;
>> -		}
>> -		if (rc == LDD_F_SV_TYPE_OST)
>> -			exclude_list[lmd->lmd_exclude_count++] = index;
>> -		else
>> -			CDEBUG(D_MOUNT, "ignoring exclude %.*s: type = %#x\n",
>> -			       (uint)(s2 - s1), s1, rc);
>> -		s1 = s2;
>> -		/* now we are pointing at ':' (next exclude)
>> -		 * or ',' (end of excludes)
>> -		 */
>> -		if (lmd->lmd_exclude_count >= devmax)
>> -			break;
>> -	}
>> -	if (rc >= 0) /* non-err */
>> -		rc = 0;
>> -
>> -	if (lmd->lmd_exclude_count) {
>> -		/* permanent, freed in lustre_free_lsi */
>> -		lmd->lmd_exclude = kcalloc(lmd->lmd_exclude_count,
>> -					   sizeof(index), GFP_NOFS);
>> -		if (lmd->lmd_exclude) {
>> -			memcpy(lmd->lmd_exclude, exclude_list,
>> -			       sizeof(index) * lmd->lmd_exclude_count);
>> -		} else {
>> -			rc = -ENOMEM;
>> -			lmd->lmd_exclude_count = 0;
>> -		}
>> -	}
>> -	kfree(exclude_list);
>> -	return rc;
>> -}
>> -
>> -static int lmd_parse_mgssec(struct lustre_mount_data *lmd, char *ptr)
>> -{
>> -	char   *tail;
>> -	int     length;
>> -
>> -	kfree(lmd->lmd_mgssec);
>> -	lmd->lmd_mgssec = NULL;
>> -
>> -	tail = strchr(ptr, ',');
>> -	if (!tail)
>> -		length = strlen(ptr);
>> -	else
>> -		length = tail - ptr;
>> -
>> -	lmd->lmd_mgssec = kzalloc(length + 1, GFP_NOFS);
>> -	if (!lmd->lmd_mgssec)
>> -		return -ENOMEM;
>> -
>> -	memcpy(lmd->lmd_mgssec, ptr, length);
>> -	lmd->lmd_mgssec[length] = '\0';
>> -	return 0;
>> -}
>> -
>> -static int lmd_parse_string(char **handle, char *ptr)
>> -{
>> -	char   *tail;
>> -	int     length;
>> -
>> -	if (!handle || !ptr)
>> -		return -EINVAL;
>> -
>> -	kfree(*handle);
>> -	*handle = NULL;
>> -
>> -	tail = strchr(ptr, ',');
>> -	if (!tail)
>> -		length = strlen(ptr);
>> -	else
>> -		length = tail - ptr;
>> -
>> -	*handle = kzalloc(length + 1, GFP_NOFS);
>> -	if (!*handle)
>> -		return -ENOMEM;
>> -
>> -	memcpy(*handle, ptr, length);
>> -	(*handle)[length] = '\0';
>> -
>> -	return 0;
>> -}
>> -
>> -/* Collect multiple values for mgsnid specifiers */
>> -static int lmd_parse_mgs(struct lustre_mount_data *lmd, char **ptr)
>> -{
>> -	lnet_nid_t nid;
>> -	char *tail = *ptr;
>> -	char *mgsnid;
>> -	int   length;
>> -	int   oldlen = 0;
>> -
>> -	/* Find end of nidlist */
>> -	while (class_parse_nid_quiet(tail, &nid, &tail) == 0)
>> -		;
>> -	length = tail - *ptr;
>> -	if (length == 0) {
>> -		LCONSOLE_ERROR_MSG(0x159, "Can't parse NID '%s'\n", *ptr);
>> -		return -EINVAL;
>> -	}
>> -
>> -	if (lmd->lmd_mgs)
>> -		oldlen = strlen(lmd->lmd_mgs) + 1;
>> -
>> -	mgsnid = kzalloc(oldlen + length + 1, GFP_NOFS);
>> -	if (!mgsnid)
>> -		return -ENOMEM;
>> -
>> -	if (lmd->lmd_mgs) {
>> -		/* Multiple mgsnid= are taken to mean failover locations */
>> -		memcpy(mgsnid, lmd->lmd_mgs, oldlen);
>> -		mgsnid[oldlen - 1] = ':';
>> -		kfree(lmd->lmd_mgs);
>> -	}
>> -	memcpy(mgsnid + oldlen, *ptr, length);
>> -	mgsnid[oldlen + length] = '\0';
>> -	lmd->lmd_mgs = mgsnid;
>> -	*ptr = tail;
>> -
>> -	return 0;
>> -}
>> -
>> -/**
>> - * Find the first delimiter (comma or colon) from the specified \a buf and
>> - * make \a *endh point to the string starting with the delimiter. The commas
>> - * in expression list [...] will be skipped.
>> - *
>> - * @buf		a delimiter-separated string
>> - * @endh	a pointer to a pointer that will point to the string
>> - *		starting with the delimiter
>> - *
>> - * RETURNS	true if delimiter is found, false if delimiter is not found
>> - */
>> -static bool lmd_find_delimiter(char *buf, char **endh)
>> -{
>> -	char *c = buf;
>> -	size_t pos;
>> -	bool found;
>> -
>> -	if (!buf)
>> -		return false;
>> -try_again:
>> -	if (*c == ',' || *c == ':')
>> -		return true;
>> -
>> -	pos = strcspn(c, "[:,]");
>> -	if (!pos)
>> -		return false;
>> -
>> -	/* Not a valid mount string */
>> -	if (*c == ']') {
>> -		CWARN("invalid mount string format\n");
>> -		return false;
>> -	}
>> -
>> -	c += pos;
>> -	if (*c == '[') {
>> -		c = strchr(c, ']');
>> -
>> -		/* invalid mount string */
>> -		if (!c) {
>> -			CWARN("invalid mount string format\n");
>> -			return false;
>> -		}
>> -		c++;
>> -		goto try_again;
>> -	}
>> -
>> -	found = *c != '\0';
>> -	if (found && endh)
>> -		*endh = c;
>> -
>> -	return found;
>> -}
>> -
>> -/**
>> - * Find the first valid string delimited by comma or colon from the specified
>> - * \a buf and parse it to see whether it's a valid nid list. If yes, \a *endh
>> - * will point to the next string starting with the delimiter.
>> - *
>> - * \param[in] buf	a delimiter-separated string
>> - * \param[in] endh	a pointer to a pointer that will point to the string
>> - *			starting with the delimiter
>> - *
>> - * \retval 0		if the string is a valid nid list
>> - * \retval 1		if the string is not a valid nid list
>> - */
>> -static int lmd_parse_nidlist(char *buf, char **endh)
>> -{
>> -	struct list_head nidlist;
>> -	char *endp = buf;
>> -	int rc = 0;
>> -	char tmp;
>> -
>> -	if (!buf)
>> -		return 1;
>> -	while (*buf == ',' || *buf == ':')
>> -		buf++;
>> -	if (*buf == ' ' || *buf == '/' || *buf == '\0')
>> -		return 1;
>> -
>> -	if (!lmd_find_delimiter(buf, &endp))
>> -		endp = buf + strlen(buf);
>> -
>> -	tmp = *endp;
>> -	*endp = '\0';
>> -
>> -	INIT_LIST_HEAD(&nidlist);
>> -	if (cfs_parse_nidlist(buf, strlen(buf), &nidlist) <= 0)
>> -		rc = 1;
>> -	cfs_free_nidlist(&nidlist);
>> -
>> -	*endp = tmp;
>> -	if (rc)
>> -		return rc;
>> -	if (endh)
>> -		*endh = endp;
>> -	return 0;
>> -}
>> -
>> -/** Parse mount line options
>> - * e.g. mount -v -t lustre -o abort_recov uml1:uml2:/lustre-client /mnt/lustre
>> - * dev is passed as device=uml1:/lustre by mount.lustre
>> - */
>> -static int lmd_parse(char *options, struct lustre_mount_data *lmd)
>> -{
>> -	char *s1, *s2, *devname = NULL;
>> -	struct lustre_mount_data *raw = (struct lustre_mount_data *)options;
>> -	int rc = 0;
>> -
>> -	LASSERT(lmd);
>> -	if (!options) {
>> -		LCONSOLE_ERROR_MSG(0x162, "Missing mount data: check that /sbin/mount.lustre is installed.\n");
>> -		return -EINVAL;
>> -	}
>> -
>> -	/* Options should be a string - try to detect old lmd data */
>> -	if ((raw->lmd_magic & 0xffffff00) == (LMD_MAGIC & 0xffffff00)) {
>> -		LCONSOLE_ERROR_MSG(0x163, "You're using an old version of /sbin/mount.lustre.  Please install version %s\n",
>> -				   LUSTRE_VERSION_STRING);
>> -		return -EINVAL;
>> -	}
>> -	lmd->lmd_magic = LMD_MAGIC;
>> -
>> -	lmd->lmd_params = kzalloc(LMD_PARAMS_MAXLEN, GFP_NOFS);
>> -	if (!lmd->lmd_params)
>> -		return -ENOMEM;
>> -	lmd->lmd_params[0] = '\0';
>> -
>> -	/* Set default flags here */
>> -
>> -	s1 = options;
>> -	while (*s1) {
>> -		int clear = 0;
>> -		int time_min = OBD_RECOVERY_TIME_MIN;
>> -		char *s3;
>> -
>> -		/* Skip whitespace and extra commas */
>> -		while (*s1 == ' ' || *s1 == ',')
>> -			s1++;
>> -		s3 = s1;
>> -
>> -		/* Client options are parsed in ll_options: eg. flock,
>> -		 * user_xattr, acl
>> -		 */
>> -
>> -		/* Parse non-ldiskfs options here. Rather than modifying
>> -		 * ldiskfs, we just zero these out here
>> -		 */
>> -		if (strncmp(s1, "abort_recov", 11) == 0) {
>> -			lmd->lmd_flags |= LMD_FLG_ABORT_RECOV;
>> -			clear++;
>> -		} else if (strncmp(s1, "recovery_time_soft=", 19) == 0) {
>> -			lmd->lmd_recovery_time_soft = max_t(int,
>> -				simple_strtoul(s1 + 19, NULL, 10), time_min);
>> -			clear++;
>> -		} else if (strncmp(s1, "recovery_time_hard=", 19) == 0) {
>> -			lmd->lmd_recovery_time_hard = max_t(int,
>> -				simple_strtoul(s1 + 19, NULL, 10), time_min);
>> -			clear++;
>> -		} else if (strncmp(s1, "noir", 4) == 0) {
>> -			lmd->lmd_flags |= LMD_FLG_NOIR; /* test purpose only. */
>> -			clear++;
>> -		} else if (strncmp(s1, "nosvc", 5) == 0) {
>> -			lmd->lmd_flags |= LMD_FLG_NOSVC;
>> -			clear++;
>> -		} else if (strncmp(s1, "nomgs", 5) == 0) {
>> -			lmd->lmd_flags |= LMD_FLG_NOMGS;
>> -			clear++;
>> -		} else if (strncmp(s1, "noscrub", 7) == 0) {
>> -			lmd->lmd_flags |= LMD_FLG_NOSCRUB;
>> -			clear++;
>> -		} else if (strncmp(s1, PARAM_MGSNODE,
>> -				   sizeof(PARAM_MGSNODE) - 1) == 0) {
>> -			s2 = s1 + sizeof(PARAM_MGSNODE) - 1;
>> -			/* Assume the next mount opt is the first
>> -			 * invalid nid we get to.
>> -			 */
>> -			rc = lmd_parse_mgs(lmd, &s2);
>> -			if (rc)
>> -				goto invalid;
>> -			clear++;
>> -		} else if (strncmp(s1, "writeconf", 9) == 0) {
>> -			lmd->lmd_flags |= LMD_FLG_WRITECONF;
>> -			clear++;
>> -		} else if (strncmp(s1, "update", 6) == 0) {
>> -			lmd->lmd_flags |= LMD_FLG_UPDATE;
>> -			clear++;
>> -		} else if (strncmp(s1, "virgin", 6) == 0) {
>> -			lmd->lmd_flags |= LMD_FLG_VIRGIN;
>> -			clear++;
>> -		} else if (strncmp(s1, "noprimnode", 10) == 0) {
>> -			lmd->lmd_flags |= LMD_FLG_NO_PRIMNODE;
>> -			clear++;
>> -		} else if (strncmp(s1, "mgssec=", 7) == 0) {
>> -			rc = lmd_parse_mgssec(lmd, s1 + 7);
>> -			if (rc)
>> -				goto invalid;
>> -			s3 = s2;
>> -			clear++;
>> -		/* ost exclusion list */
>> -		} else if (strncmp(s1, "exclude=", 8) == 0) {
>> -			rc = lmd_make_exclusion(lmd, s1 + 7);
>> -			if (rc)
>> -				goto invalid;
>> -			clear++;
>> -		} else if (strncmp(s1, "mgs", 3) == 0) {
>> -			/* We are an MGS */
>> -			lmd->lmd_flags |= LMD_FLG_MGS;
>> -			clear++;
>> -		} else if (strncmp(s1, "svname=", 7) == 0) {
>> -			rc = lmd_parse_string(&lmd->lmd_profile, s1 + 7);
>> -			if (rc)
>> -				goto invalid;
>> -			clear++;
>> -		} else if (strncmp(s1, "param=", 6) == 0) {
>> -			size_t length, params_length;
>> -			char *tail = s1;
>> -
>> -			if (lmd_find_delimiter(s1 + 6, &tail)) {
>> -				char *param_str = tail + 1;
>> -				int supplementary = 1;
>> -
>> -				while (!lmd_parse_nidlist(param_str,
>> -							  &param_str))
>> -					supplementary = 0;
>> -				length = param_str - s1 - supplementary;
>> -			} else {
>> -				length = strlen(s1);
>> -			}
>> -			length -= 6;
>> -			params_length = strlen(lmd->lmd_params);
>> -			if (params_length + length + 1 >= LMD_PARAMS_MAXLEN)
>> -				return -E2BIG;
>> -			strncat(lmd->lmd_params, s1 + 6, length);
>> -			lmd->lmd_params[params_length + length] = '\0';
>> -			strlcat(lmd->lmd_params, " ", LMD_PARAMS_MAXLEN);
>> -			s3 = s1 + 6 + length;
>> -			clear++;
>> -		} else if (strncmp(s1, "osd=", 4) == 0) {
>> -			rc = lmd_parse_string(&lmd->lmd_osd_type, s1 + 4);
>> -			if (rc)
>> -				goto invalid;
>> -			clear++;
>> -		}
>> -		/* Linux 2.4 doesn't pass the device, so we stuck it at the
>> -		 * end of the options.
>> -		 */
>> -		else if (strncmp(s1, "device=", 7) == 0) {
>> -			devname = s1 + 7;
>> -			/* terminate options right before device.  device
>> -			 * must be the last one.
>> -			 */
>> -			*s1 = '\0';
>> -			break;
>> -		}
>> -
>> -		/* Find next opt */
>> -		s2 = strchr(s1, ',');
>> -		if (!s2) {
>> -			if (clear)
>> -				*s1 = '\0';
>> -			break;
>> -		}
>> -		s2++;
>> -		if (clear)
>> -			memmove(s1, s2, strlen(s2) + 1);
>> -		else
>> -			s1 = s2;
>> -	}
>> -
>> -	if (!devname) {
>> -		LCONSOLE_ERROR_MSG(0x164, "Can't find the device name (need mount option 'device=...')\n");
>> -		goto invalid;
>> -	}
>> -
>> -	s1 = strstr(devname, ":/");
>> -	if (s1) {
>> -		++s1;
>> -		lmd->lmd_flags |= LMD_FLG_CLIENT;
>> -		/* Remove leading /s from fsname */
>> -		while (*++s1 == '/')
>> -			;
>> -		s2 = strchrnul(s1, '/');
>> -		/* Freed in lustre_free_lsi */
>> -		lmd->lmd_profile = kasprintf(GFP_KERNEL, "%.*s-client",
>> -					     (int)(s2 - s1), s1);
>> -		if (!lmd->lmd_profile)
>> -			return -ENOMEM;
>> -
>> -		s1 = s2;
>> -		s2 = s1 + strlen(s1) - 1;
>> -		/* Remove padding /s from fileset */
>> -		while (*s2 == '/')
>> -			s2--;
>> -		if (s2 > s1) {
>> -			lmd->lmd_fileset = kstrndup(s1, s2 - s1 + 1,
>> -						    GFP_KERNEL);
>> -			if (!lmd->lmd_fileset)
>> -				return -ENOMEM;
>> -		}
>> -	}
>> -
>> -	/* Freed in lustre_free_lsi */
>> -	lmd->lmd_dev = kstrdup(devname, GFP_KERNEL);
>> -	if (!lmd->lmd_dev)
>> -		return -ENOMEM;
>> -
>> -	/* Save mount options */
>> -	s1 = options + strlen(options) - 1;
>> -	while (s1 >= options && (*s1 == ',' || *s1 == ' '))
>> -		*s1-- = 0;
>> -	if (*options != 0) {
>> -		/* Freed in lustre_free_lsi */
>> -		lmd->lmd_opts = kstrdup(options, GFP_KERNEL);
>> -		if (!lmd->lmd_opts)
>> -			return -ENOMEM;
>> -	}
>> -
>> -	lmd_print(lmd);
>> -	lmd->lmd_magic = LMD_MAGIC;
>> -
>> -	return rc;
>> -
>> -invalid:
>> -	CERROR("Bad mount options %s\n", options);
>> -	return -EINVAL;
>> -}
>> -
>> -/** This is the entry point for the mount call into Lustre.
>> - * This is called when a server or client is mounted,
>> - * and this is where we start setting things up.
>> - * @param data Mount options (e.g. -o flock,abort_recov)
>> - */
>> -static int lustre_fill_super(struct super_block *sb, void *lmd2_data, int silent)
>> -{
>> -	struct lustre_mount_data *lmd;
>> -	struct lustre_sb_info *lsi;
>> -	int rc;
>> -
>> -	CDEBUG(D_MOUNT | D_VFSTRACE, "VFS Op: sb %p\n", sb);
>> -
>> -	lsi = lustre_init_lsi(sb);
>> -	if (!lsi)
>> -		return -ENOMEM;
>> -	lmd = lsi->lsi_lmd;
>> -
>> -	/*
>> -	 * Disable lockdep during mount, because mount locking patterns are
>> -	 * `special'.
>> -	 */
>> -	lockdep_off();
>> -
>> -	/*
>> -	 * LU-639: the obd cleanup of last mount may not finish yet, wait here.
>> -	 */
>> -	obd_zombie_barrier();
>> -
>> -	/* Figure out the lmd from the mount options */
>> -	if (lmd_parse(lmd2_data, lmd)) {
>> -		lustre_put_lsi(sb);
>> -		rc = -EINVAL;
>> -		goto out;
>> -	}
>> -
>> -	if (lmd_is_client(lmd)) {
>> -		bool have_client = false;
>> -		CDEBUG(D_MOUNT, "Mounting client %s\n", lmd->lmd_profile);
>> -		if (!client_fill_super)
>> -			request_module("lustre");
>> -		spin_lock(&client_lock);
>> -		if (client_fill_super && try_module_get(client_mod))
>> -			have_client = true;
>> -		spin_unlock(&client_lock);
>> -		if (!have_client) {
>> -			LCONSOLE_ERROR_MSG(0x165, "Nothing registered for client mount! Is the 'lustre' module loaded?\n");
>> -			lustre_put_lsi(sb);
>> -			rc = -ENODEV;
>> -		} else {
>> -			rc = lustre_start_mgc(sb);
>> -			if (rc) {
>> -				lustre_common_put_super(sb);
>> -				goto out;
>> -			}
>> -			/* Connect and start */
>> -			/* (should always be ll_fill_super) */
>> -			rc = (*client_fill_super)(sb);
>> -			/* c_f_s will call lustre_common_put_super on failure, otherwise
>> -			 * c_f_s will have taken another reference to the module */
>> -			module_put(client_mod);
>> -		}
>> -	} else {
>> -		CERROR("This is client-side-only module, cannot handle server mount.\n");
>> -		rc = -EINVAL;
>> -	}
>> -
>> -	/* If error happens in fill_super() call, @lsi will be killed there.
>> -	 * This is why we do not put it here.
>> -	 */
>> -	goto out;
>> -out:
>> -	if (rc) {
>> -		CERROR("Unable to mount %s (%d)\n",
>> -		       s2lsi(sb) ? lmd->lmd_dev : "", rc);
>> -	} else {
>> -		CDEBUG(D_SUPER, "Mount %s complete\n",
>> -		       lmd->lmd_dev);
>> -	}
>> -	lockdep_on();
>> -	return rc;
>> -}
>> -
>> -/* We can't call ll_fill_super by name because it lives in a module that
>> - * must be loaded after this one.
>> - */
>> -void lustre_register_super_ops(struct module *mod,
>> -			       int (*cfs)(struct super_block *sb),
>> -			       void (*ksc)(struct super_block *sb))
>> -{
>> -	spin_lock(&client_lock);
>> -	client_mod = mod;
>> -	client_fill_super = cfs;
>> -	kill_super_cb = ksc;
>> -	spin_unlock(&client_lock);
>> -}
>> -EXPORT_SYMBOL(lustre_register_super_ops);
>> -
>> -/***************** FS registration ******************/
>> -static struct dentry *lustre_mount(struct file_system_type *fs_type, int flags,
>> -				   const char *devname, void *data)
>> -{
>> -	return mount_nodev(fs_type, flags, data, lustre_fill_super);
>> -}
>> -
>> -static void lustre_kill_super(struct super_block *sb)
>> -{
>> -	struct lustre_sb_info *lsi = s2lsi(sb);
>> -
>> -	if (kill_super_cb && lsi)
>> -		(*kill_super_cb)(sb);
>> -
>> -	kill_anon_super(sb);
>> -}
>> -
>> -/** Register the "lustre" fs type
>> - */
>> -static struct file_system_type lustre_fs_type = {
>> -	.owner		= THIS_MODULE,
>> -	.name		= "lustre",
>> -	.mount		= lustre_mount,
>> -	.kill_sb	= lustre_kill_super,
>> -	.fs_flags	= FS_RENAME_DOES_D_MOVE,
>> -};
>> -MODULE_ALIAS_FS("lustre");
>> -
>> -int lustre_register_fs(void)
>> -{
>> -	return register_filesystem(&lustre_fs_type);
>> -}
>> -
>> -int lustre_unregister_fs(void)
>> -{
>> -	return unregister_filesystem(&lustre_fs_type);
>> -}
>> 
>> 
>> 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180730/a947e71e/attachment.sig>

From neilb at suse.com  Mon Jul 30 03:37:40 2018
From: neilb at suse.com (NeilBrown)
Date: Mon, 30 Jul 2018 13:37:40 +1000
Subject: [lustre-devel] [PATCH 00/22] Lustre: use while loop when emptying a
 list
Message-ID: <153292153459.13840.17465048403476297915.stgit@noble>

If you have a list that needs to be emptied, it is best to have a loop
like
  while (!list_empty(...))

because then it is obvious what the purpose of the loop is.
Many places in lustre use
  list_for_each_entry_safe()
instead, which obscures the purpose.
Several of these were from patches which deliberately converted from
the while loop the list list_for_each_entry_safe() loop, at least some
of which introduced real bugs.

This series reverts all those patches, and then makes other
conversions.
There are still several places which could be converted, but I got
bored...
I've particularly converted all where the list_head is a local
variable. In these cases it is obviously wrong not to empty the
list completely.

For those conversions that I did manual (not reverts) I use
  list_first_entry()
to get the first entry, rather then list_entry(head->next).
Others could be converted as well.

NeilBrown


---

NeilBrown (22):
      Revert "staging: lustre: lnet: api-ni: Use list_for_each_entry_safe"
      Revert "staging: lustre: o2iblnd: Use list_for_each_entry_safe in kiblnd_destroy_fmr_pool_list"
      Revert "staging: lustre: lnet: o2iblnd: Use list_for_each_entry_safe"
      Revert "staging: lustre: lnet: socklnd: Use list_for_each_entry_safe"
      Revert "staging: lustre: lnet: socklnd_proto: Use list_for_each_entry_safe"
      Revert "staging: lustre: osc_cache: Use list_for_each_entry_safe"
      Revert "staging: lustre: lnet: peer: Use list_for_each_entry_safe"
      Revert "staging: lustre: lnet: config: Use list_for_each_entry_safe"
      Revert "staging: lustre: lnet: router: Use list_for_each_entry_safe"
      Revert "staging: lustre: lnet: conrpc: Use list_for_each_entry_safe"
      Revert "staging: lustre: lnet: lib-move: Use list_for_each_entry_safe"
      Revert "staging: lustre: obdclass: Use list_for_each_entry_safe"
      Revert "staging: lustre: lnet: Use list_for_each_entry_safe"
      Revert: Staging: lustre: Iterate list using list_for_each_entry
      lustre: o2iblnd: convert list_for_each_entry_safe() to while(!list_empty())
      lustre: tracefile: convert list_for_each_entry_safe() to while(!list_empty())
      lustre: lib-move:  convert list_for_each_entry_safe() to while(!list_empty())
      lustre: net_fault: convert list_for_each_entry_safe() to while(!list_empty())
      lustre: fld_request: convert list_for_each_entry_safe() to while(!list_empty())
      lustre: lov_obd: convert list_for_each_entry_safe() to while(!list_empty())
      lustre: ldlm_request: convert list_for_each_entry_safe() to while(!list_empty())
      lustre: sec_config: convert list_for_each_entry_safe() to while(!list_empty())


 .../staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c    |   21 ++++++++++++--------
 .../staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c |   12 ++++++-----
 .../staging/lustre/lnet/klnds/socklnd/socklnd.c    |    9 +++++----
 .../staging/lustre/lnet/klnds/socklnd/socklnd_cb.c |    5 +++--
 .../lustre/lnet/klnds/socklnd/socklnd_proto.c      |    4 ++--
 drivers/staging/lustre/lnet/libcfs/tracefile.c     |   12 +++++++----
 drivers/staging/lustre/lnet/lnet/api-ni.c          |   10 ++++++----
 drivers/staging/lustre/lnet/lnet/config.c          |    5 +++--
 drivers/staging/lustre/lnet/lnet/lib-move.c        |   13 +++++++-----
 drivers/staging/lustre/lnet/lnet/net_fault.c       |    7 +++++--
 drivers/staging/lustre/lnet/lnet/peer.c            |    4 ++--
 drivers/staging/lustre/lnet/lnet/router.c          |    4 ++--
 drivers/staging/lustre/lnet/selftest/conrpc.c      |    5 +++--
 drivers/staging/lustre/lustre/fld/fld_request.c    |    6 ++++--
 drivers/staging/lustre/lustre/ldlm/ldlm_request.c  |    6 ++++--
 drivers/staging/lustre/lustre/lov/lov_obd.c        |    5 +++--
 .../staging/lustre/lustre/obdclass/lustre_peer.c   |    5 +++--
 drivers/staging/lustre/lustre/osc/osc_cache.c      |    9 +++++----
 drivers/staging/lustre/lustre/ptlrpc/sec_config.c  |    7 ++++---
 19 files changed, 87 insertions(+), 62 deletions(-)

--
Signature


From neilb at suse.com  Mon Jul 30 03:37:40 2018
From: neilb at suse.com (NeilBrown)
Date: Mon, 30 Jul 2018 13:37:40 +1000
Subject: [lustre-devel] [PATCH 01/22] Revert "staging: lustre: lnet: api-ni:
 Use list_for_each_entry_safe"
In-Reply-To: <153292153459.13840.17465048403476297915.stgit@noble>
References: <153292153459.13840.17465048403476297915.stgit@noble>
Message-ID: <153292186042.13840.10901014935265304451.stgit@noble>

This reverts commit c997866cd27495ae28bc07596457e2bd83fb3275.

These loops really want to remove everything, and using a
  while(!list_empty())
loop makes this more obvious.  In this case, the first loop is
currently buggy.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 drivers/staging/lustre/lnet/lnet/api-ni.c |   10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/drivers/staging/lustre/lnet/lnet/api-ni.c b/drivers/staging/lustre/lnet/lnet/api-ni.c
index d21bceeaceda..51a81075f8d5 100644
--- a/drivers/staging/lustre/lnet/lnet/api-ni.c
+++ b/drivers/staging/lustre/lnet/lnet/api-ni.c
@@ -1077,17 +1077,18 @@ lnet_clear_zombies_nis_locked(void)
 	int i;
 	int islo;
 	struct lnet_ni *ni;
-	struct lnet_ni *temp;
 
 	/*
 	 * Now wait for the NI's I just nuked to show up on ln_zombie_nis
 	 * and shut them down in guaranteed thread context
 	 */
 	i = 2;
-	list_for_each_entry_safe(ni, temp, &the_lnet.ln_nis_zombie, ni_list) {
+	while (!list_empty(&the_lnet.ln_nis_zombie)) {
 		int *ref;
 		int j;
 
+		ni = list_entry(the_lnet.ln_nis_zombie.next,
+				struct lnet_ni, ni_list);
 		list_del_init(&ni->ni_list);
 		cfs_percpt_for_each(ref, j, ni->ni_refs) {
 			if (!*ref)
@@ -1137,7 +1138,6 @@ static void
 lnet_shutdown_lndnis(void)
 {
 	struct lnet_ni *ni;
-	struct lnet_ni *temp;
 	int i;
 
 	/* NB called holding the global mutex */
@@ -1151,7 +1151,9 @@ lnet_shutdown_lndnis(void)
 	the_lnet.ln_shutdown = 1;	/* flag shutdown */
 
 	/* Unlink NIs from the global table */
-	list_for_each_entry_safe(ni, temp, &the_lnet.ln_nis, ni_list) {
+	while (!list_empty(&the_lnet.ln_nis)) {
+		ni = list_entry(the_lnet.ln_nis.next,
+				struct lnet_ni, ni_list);
 		lnet_ni_unlink_locked(ni);
 	}
 



From neilb at suse.com  Mon Jul 30 03:37:40 2018
From: neilb at suse.com (NeilBrown)
Date: Mon, 30 Jul 2018 13:37:40 +1000
Subject: [lustre-devel] [PATCH 02/22] Revert "staging: lustre: o2iblnd: Use
 list_for_each_entry_safe in kiblnd_destroy_fmr_pool_list"
In-Reply-To: <153292153459.13840.17465048403476297915.stgit@noble>
References: <153292153459.13840.17465048403476297915.stgit@noble>
Message-ID: <153292186069.13840.6426045683936468910.stgit@noble>

This reverts commit 0d33ec5f95fe068d7e96b6e7ed9216de93f6b5b0.

These loops really want to remove everything, and using a
  while(!list_empty())
loop makes this more obvious.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 .../staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c    |    5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c
index 05835cc0f0a5..124870ada28b 100644
--- a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c
+++ b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c
@@ -1306,9 +1306,10 @@ static void kiblnd_destroy_fmr_pool(struct kib_fmr_pool *fpo)
 
 static void kiblnd_destroy_fmr_pool_list(struct list_head *head)
 {
-	struct kib_fmr_pool *fpo, *tmp;
+	struct kib_fmr_pool *fpo;
 
-	list_for_each_entry_safe(fpo, tmp, head, fpo_list) {
+	while (!list_empty(head)) {
+		fpo = list_entry(head->next, struct kib_fmr_pool, fpo_list);
 		list_del(&fpo->fpo_list);
 		kiblnd_destroy_fmr_pool(fpo);
 	}



From neilb at suse.com  Mon Jul 30 03:37:40 2018
From: neilb at suse.com (NeilBrown)
Date: Mon, 30 Jul 2018 13:37:40 +1000
Subject: [lustre-devel] [PATCH 03/22] Revert "staging: lustre: lnet:
 o2iblnd: Use list_for_each_entry_safe"
In-Reply-To: <153292153459.13840.17465048403476297915.stgit@noble>
References: <153292153459.13840.17465048403476297915.stgit@noble>
Message-ID: <153292186073.13840.10228341482615967034.stgit@noble>

This reverts commit 995ae68c30a5d4947f7685f29b1e69b436ddcdb3.

These loops really want to remove everything, and using a
  while(!list_empty())
loop makes this more obvious.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 .../staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c
index a5ef4cc6c04f..9cf1f64e3d76 100644
--- a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c
+++ b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c
@@ -3185,7 +3185,6 @@ kiblnd_check_conns(int idx)
 	struct list_head *ptmp;
 	struct kib_peer *peer;
 	struct kib_conn *conn;
-	struct kib_conn *temp;
 	struct kib_conn *tmp;
 	struct list_head *ctmp;
 	unsigned long flags;
@@ -3253,7 +3252,8 @@ kiblnd_check_conns(int idx)
 	 * NOOP, but there were no non-blocking tx descs
 	 * free to do it last time...
 	 */
-	list_for_each_entry_safe(conn, temp, &checksends, ibc_connd_list) {
+	while (!list_empty(&checksends)) {
+		conn = list_entry(checksends.next, struct kib_conn, ibc_connd_list);
 		list_del(&conn->ibc_connd_list);
 
 		spin_lock(&conn->ibc_lock);



From neilb at suse.com  Mon Jul 30 03:37:40 2018
From: neilb at suse.com (NeilBrown)
Date: Mon, 30 Jul 2018 13:37:40 +1000
Subject: [lustre-devel] [PATCH 04/22] Revert "staging: lustre: lnet:
 socklnd: Use list_for_each_entry_safe"
In-Reply-To: <153292153459.13840.17465048403476297915.stgit@noble>
References: <153292153459.13840.17465048403476297915.stgit@noble>
Message-ID: <153292186077.13840.2365613926140981649.stgit@noble>

This reverts commit 2aff15d43a832cd0af0263e4456e5b01e15f86c6.

These loops really want to remove everything, and using a
  while(!list_empty())
loop makes this more obvious.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 .../staging/lustre/lnet/klnds/socklnd/socklnd.c    |    9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)

diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c
index 6b1df0c98cbd..a48b1b9a850b 100644
--- a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c
+++ b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c
@@ -1546,7 +1546,6 @@ ksocknal_finalize_zcreq(struct ksock_conn *conn)
 {
 	struct ksock_peer *peer = conn->ksnc_peer;
 	struct ksock_tx *tx;
-	struct ksock_tx *temp;
 	struct ksock_tx *tmp;
 	LIST_HEAD(zlist);
 
@@ -1572,7 +1571,9 @@ ksocknal_finalize_zcreq(struct ksock_conn *conn)
 
 	spin_unlock(&peer->ksnp_lock);
 
-	list_for_each_entry_safe(tx, temp, &zlist, tx_zc_list) {
+	while (!list_empty(&zlist)) {
+		tx = list_entry(zlist.next, struct ksock_tx, tx_zc_list);
+
 		list_del(&tx->tx_zc_list);
 		ksocknal_tx_decref(tx);
 	}
@@ -2270,13 +2271,13 @@ ksocknal_free_buffers(void)
 	if (!list_empty(&ksocknal_data.ksnd_idle_noop_txs)) {
 		struct list_head zlist;
 		struct ksock_tx *tx;
-		struct ksock_tx *temp;
 
 		list_add(&zlist, &ksocknal_data.ksnd_idle_noop_txs);
 		list_del_init(&ksocknal_data.ksnd_idle_noop_txs);
 		spin_unlock(&ksocknal_data.ksnd_tx_lock);
 
-		list_for_each_entry_safe(tx, temp, &zlist, tx_list) {
+		while (!list_empty(&zlist)) {
+			tx = list_entry(zlist.next, struct ksock_tx, tx_list);
 			list_del(&tx->tx_list);
 			kfree(tx);
 		}



From neilb at suse.com  Mon Jul 30 03:37:40 2018
From: neilb at suse.com (NeilBrown)
Date: Mon, 30 Jul 2018 13:37:40 +1000
Subject: [lustre-devel] [PATCH 05/22] Revert "staging: lustre: lnet:
 socklnd_proto: Use list_for_each_entry_safe"
In-Reply-To: <153292153459.13840.17465048403476297915.stgit@noble>
References: <153292153459.13840.17465048403476297915.stgit@noble>
Message-ID: <153292186081.13840.4047441745421559567.stgit@noble>

This reverts commit 1edae04ff85fe65a333949de6101578c015a21fa.

These loops really want to remove everything, and using a
  while(!list_empty())
loop makes this more obvious.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 .../lustre/lnet/klnds/socklnd/socklnd_proto.c      |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_proto.c b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_proto.c
index aaa04a5f0527..abfaf5701758 100644
--- a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_proto.c
+++ b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_proto.c
@@ -413,7 +413,6 @@ ksocknal_handle_zcack(struct ksock_conn *conn, __u64 cookie1, __u64 cookie2)
 {
 	struct ksock_peer *peer = conn->ksnc_peer;
 	struct ksock_tx *tx;
-	struct ksock_tx *temp;
 	struct ksock_tx *tmp;
 	LIST_HEAD(zlist);
 	int count;
@@ -448,7 +447,8 @@ ksocknal_handle_zcack(struct ksock_conn *conn, __u64 cookie1, __u64 cookie2)
 
 	spin_unlock(&peer->ksnp_lock);
 
-	list_for_each_entry_safe(tx, temp, &zlist, tx_zc_list) {
+	while (!list_empty(&zlist)) {
+		tx = list_entry(zlist.next, struct ksock_tx, tx_zc_list);
 		list_del(&tx->tx_zc_list);
 		ksocknal_tx_decref(tx);
 	}



From neilb at suse.com  Mon Jul 30 03:37:40 2018
From: neilb at suse.com (NeilBrown)
Date: Mon, 30 Jul 2018 13:37:40 +1000
Subject: [lustre-devel] [PATCH 06/22] Revert "staging: lustre: osc_cache:
 Use list_for_each_entry_safe"
In-Reply-To: <153292153459.13840.17465048403476297915.stgit@noble>
References: <153292153459.13840.17465048403476297915.stgit@noble>
Message-ID: <153292186084.13840.7272276537169852360.stgit@noble>

This reverts commit 4a81ce53a61c72afb079c096599a5d34749b9dd7.

These loops really want to remove everything, and using a
  while(!list_empty())
loop makes this more obvious.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 drivers/staging/lustre/lustre/osc/osc_cache.c |    9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)

diff --git a/drivers/staging/lustre/lustre/osc/osc_cache.c b/drivers/staging/lustre/lustre/osc/osc_cache.c
index 15a48173e148..87d0d16d942b 100644
--- a/drivers/staging/lustre/lustre/osc/osc_cache.c
+++ b/drivers/staging/lustre/lustre/osc/osc_cache.c
@@ -2060,7 +2060,6 @@ static unsigned int get_write_extents(struct osc_object *obj,
 {
 	struct client_obd *cli = osc_cli(obj);
 	struct osc_extent *ext;
-	struct osc_extent *temp;
 	struct extent_rpc_data data = {
 		.erd_rpc_list = rpclist,
 		.erd_page_count = 0,
@@ -2070,7 +2069,9 @@ static unsigned int get_write_extents(struct osc_object *obj,
 	};
 
 	LASSERT(osc_object_is_locked(obj));
-	list_for_each_entry_safe(ext, temp, &obj->oo_hp_exts, oe_link) {
+	while (!list_empty(&obj->oo_hp_exts)) {
+		ext = list_entry(obj->oo_hp_exts.next, struct osc_extent,
+				 oe_link);
 		LASSERT(ext->oe_state == OES_CACHE);
 		if (!try_to_add_extent_for_io(cli, ext, &data))
 			return data.erd_page_count;
@@ -2829,7 +2830,6 @@ int osc_cache_truncate_start(const struct lu_env *env, struct osc_object *obj,
 {
 	struct client_obd *cli = osc_cli(obj);
 	struct osc_extent *ext;
-	struct osc_extent *temp;
 	struct osc_extent *waiting = NULL;
 	pgoff_t index;
 	LIST_HEAD(list);
@@ -2888,9 +2888,10 @@ int osc_cache_truncate_start(const struct lu_env *env, struct osc_object *obj,
 
 	osc_list_maint(cli, obj);
 
-	list_for_each_entry_safe(ext, temp, &list, oe_link) {
+	while (!list_empty(&list)) {
 		int rc;
 
+		ext = list_entry(list.next, struct osc_extent, oe_link);
 		list_del_init(&ext->oe_link);
 
 		/* extent may be in OES_ACTIVE state because inode mutex



From neilb at suse.com  Mon Jul 30 03:37:40 2018
From: neilb at suse.com (NeilBrown)
Date: Mon, 30 Jul 2018 13:37:40 +1000
Subject: [lustre-devel] [PATCH 07/22] Revert "staging: lustre: lnet: peer:
 Use list_for_each_entry_safe"
In-Reply-To: <153292153459.13840.17465048403476297915.stgit@noble>
References: <153292153459.13840.17465048403476297915.stgit@noble>
Message-ID: <153292186088.13840.12536228855978938054.stgit@noble>

This reverts commit 3e47a1cfba5a8af7dc3c10a4705d8047abdc26c3.

These loops really want to remove everything, and using a
  while(!list_empty())
loop makes this more obvious.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 drivers/staging/lustre/lnet/lnet/peer.c |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/staging/lustre/lnet/lnet/peer.c b/drivers/staging/lustre/lnet/lnet/peer.c
index 6ce175d77d0f..7c303ef6bb34 100644
--- a/drivers/staging/lustre/lnet/lnet/peer.c
+++ b/drivers/staging/lustre/lnet/lnet/peer.c
@@ -176,7 +176,6 @@ lnet_peer_tables_cleanup(struct lnet_ni *ni)
 	struct lnet_peer_table *ptable;
 	struct list_head deathrow;
 	struct lnet_peer *lp;
-	struct lnet_peer *temp;
 	int i;
 
 	INIT_LIST_HEAD(&deathrow);
@@ -210,7 +209,8 @@ lnet_peer_tables_cleanup(struct lnet_ni *ni)
 		lnet_net_unlock(i);
 	}
 
-	list_for_each_entry_safe(lp, temp, &deathrow, lp_hashlist) {
+	while (!list_empty(&deathrow)) {
+		lp = list_entry(deathrow.next, struct lnet_peer, lp_hashlist);
 		list_del(&lp->lp_hashlist);
 		kfree(lp);
 	}



From neilb at suse.com  Mon Jul 30 03:37:40 2018
From: neilb at suse.com (NeilBrown)
Date: Mon, 30 Jul 2018 13:37:40 +1000
Subject: [lustre-devel] [PATCH 08/22] Revert "staging: lustre: lnet: config:
 Use list_for_each_entry_safe"
In-Reply-To: <153292153459.13840.17465048403476297915.stgit@noble>
References: <153292153459.13840.17465048403476297915.stgit@noble>
Message-ID: <153292186091.13840.114221205536468808.stgit@noble>

This reverts commit cb734cf73eaed9b9bb7f190cceaafc15af0d8815.

These loops really want to remove everything, and using a
  while(!list_empty())
loop makes this more obvious.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 drivers/staging/lustre/lnet/lnet/config.c |    5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/drivers/staging/lustre/lnet/lnet/config.c b/drivers/staging/lustre/lnet/lnet/config.c
index 136905db2746..26b799e66e53 100644
--- a/drivers/staging/lustre/lnet/lnet/config.c
+++ b/drivers/staging/lustre/lnet/lnet/config.c
@@ -1026,7 +1026,6 @@ lnet_match_networks(char **networksp, char *ip2nets, __u32 *ipaddrs, int nip)
 	struct list_head *t;
 	struct list_head *t2;
 	struct lnet_text_buf *tb;
-	struct lnet_text_buf *temp;
 	struct lnet_text_buf *tb2;
 	__u32 net1;
 	__u32 net2;
@@ -1049,7 +1048,9 @@ lnet_match_networks(char **networksp, char *ip2nets, __u32 *ipaddrs, int nip)
 	len = 0;
 	rc = 0;
 
-	list_for_each_entry_safe(tb, temp, &raw_entries, ltb_list) {
+	while (!list_empty(&raw_entries)) {
+		tb = list_entry(raw_entries.next, struct lnet_text_buf,
+				ltb_list);
 		strncpy(source, tb->ltb_text, sizeof(source));
 		source[sizeof(source) - 1] = '\0';
 



From neilb at suse.com  Mon Jul 30 03:37:40 2018
From: neilb at suse.com (NeilBrown)
Date: Mon, 30 Jul 2018 13:37:40 +1000
Subject: [lustre-devel] [PATCH 09/22] Revert "staging: lustre: lnet: router:
 Use list_for_each_entry_safe"
In-Reply-To: <153292153459.13840.17465048403476297915.stgit@noble>
References: <153292153459.13840.17465048403476297915.stgit@noble>
Message-ID: <153292186096.13840.6296110306335811050.stgit@noble>

This reverts commit 37cce1bcb750ac12773f1c53afe88a8433f53eb3.

These loops really want to remove everything, and using a
  while(!list_empty())
loop makes this more obvious.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 drivers/staging/lustre/lnet/lnet/router.c |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/staging/lustre/lnet/lnet/router.c b/drivers/staging/lustre/lnet/lnet/router.c
index 3f84df6cb3b1..965087b7359c 100644
--- a/drivers/staging/lustre/lnet/lnet/router.c
+++ b/drivers/staging/lustre/lnet/lnet/router.c
@@ -1340,7 +1340,6 @@ lnet_rtrpool_free_bufs(struct lnet_rtrbufpool *rbp, int cpt)
 	int npages = rbp->rbp_npages;
 	struct list_head tmp;
 	struct lnet_rtrbuf *rb;
-	struct lnet_rtrbuf *temp;
 
 	if (!rbp->rbp_nbuffers) /* not initialized or already freed */
 		return;
@@ -1357,7 +1356,8 @@ lnet_rtrpool_free_bufs(struct lnet_rtrbufpool *rbp, int cpt)
 	lnet_net_unlock(cpt);
 
 	/* Free buffers on the free list. */
-	list_for_each_entry_safe(rb, temp, &tmp, rb_list) {
+	while (!list_empty(&tmp)) {
+		rb = list_entry(tmp.next, struct lnet_rtrbuf, rb_list);
 		list_del(&rb->rb_list);
 		lnet_destroy_rtrbuf(rb, npages);
 	}



From neilb at suse.com  Mon Jul 30 03:37:41 2018
From: neilb at suse.com (NeilBrown)
Date: Mon, 30 Jul 2018 13:37:41 +1000
Subject: [lustre-devel] [PATCH 10/22] Revert "staging: lustre: lnet: conrpc:
 Use list_for_each_entry_safe"
In-Reply-To: <153292153459.13840.17465048403476297915.stgit@noble>
References: <153292153459.13840.17465048403476297915.stgit@noble>
Message-ID: <153292186099.13840.6595803769201521471.stgit@noble>

This reverts commit a9a6cb4f4693253349358f8163d826eb0cfecfbc.

These loops really want to remove everything, and using a
  while(!list_empty())
loop makes this more obvious.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 drivers/staging/lustre/lnet/selftest/conrpc.c |    5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/drivers/staging/lustre/lnet/selftest/conrpc.c b/drivers/staging/lustre/lnet/selftest/conrpc.c
index 95cbd1a14e1b..3afde0141db5 100644
--- a/drivers/staging/lustre/lnet/selftest/conrpc.c
+++ b/drivers/staging/lustre/lnet/selftest/conrpc.c
@@ -1327,7 +1327,6 @@ lstcon_rpc_cleanup_wait(void)
 {
 	struct lstcon_rpc_trans *trans;
 	struct lstcon_rpc *crpc;
-	struct lstcon_rpc *temp;
 	struct list_head *pacer;
 	struct list_head zlist;
 
@@ -1367,7 +1366,9 @@ lstcon_rpc_cleanup_wait(void)
 
 	spin_unlock(&console_session.ses_rpc_lock);
 
-	list_for_each_entry_safe(crpc, temp, &zlist, crp_link) {
+	while (!list_empty(&zlist)) {
+		crpc = list_entry(zlist.next, lstcon_rpc_t, crp_link);
+
 		list_del(&crpc->crp_link);
 		kfree(crpc);
 	}



From neilb at suse.com  Mon Jul 30 03:37:41 2018
From: neilb at suse.com (NeilBrown)
Date: Mon, 30 Jul 2018 13:37:41 +1000
Subject: [lustre-devel] [PATCH 11/22] Revert "staging: lustre: lnet:
 lib-move: Use list_for_each_entry_safe"
In-Reply-To: <153292153459.13840.17465048403476297915.stgit@noble>
References: <153292153459.13840.17465048403476297915.stgit@noble>
Message-ID: <153292186104.13840.12696227873574036682.stgit@noble>

This reverts commit 24f695909440b79b04bb75205384c9772e4c7d0a.

These loops really want to remove everything, and using a
  while(!list_empty())
loop makes this more obvious.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 drivers/staging/lustre/lnet/lnet/lib-move.c |    9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)

diff --git a/drivers/staging/lustre/lnet/lnet/lib-move.c b/drivers/staging/lustre/lnet/lnet/lib-move.c
index e8092b1b6b27..2756e91b34bb 100644
--- a/drivers/staging/lustre/lnet/lnet/lib-move.c
+++ b/drivers/staging/lustre/lnet/lnet/lib-move.c
@@ -49,7 +49,6 @@ int
 lnet_fail_nid(lnet_nid_t nid, unsigned int threshold)
 {
 	struct lnet_test_peer *tp;
-	struct lnet_test_peer *temp;
 	struct list_head *el;
 	struct list_head *next;
 	struct list_head cull;
@@ -88,7 +87,9 @@ lnet_fail_nid(lnet_nid_t nid, unsigned int threshold)
 
 	lnet_net_unlock(0);
 
-	list_for_each_entry_safe(tp, temp, &cull, tp_list) {
+	while (!list_empty(&cull)) {
+		tp = list_entry(cull.next, struct lnet_test_peer, tp_list);
+
 		list_del(&tp->tp_list);
 		kfree(tp);
 	}
@@ -99,7 +100,6 @@ static int
 fail_peer(lnet_nid_t nid, int outgoing)
 {
 	struct lnet_test_peer *tp;
-	struct lnet_test_peer *temp;
 	struct list_head *el;
 	struct list_head *next;
 	struct list_head cull;
@@ -146,7 +146,8 @@ fail_peer(lnet_nid_t nid, int outgoing)
 
 	lnet_net_unlock(0);
 
-	list_for_each_entry_safe(tp, temp, &cull, tp_list) {
+	while (!list_empty(&cull)) {
+		tp = list_entry(cull.next, struct lnet_test_peer, tp_list);
 		list_del(&tp->tp_list);
 
 		kfree(tp);



From neilb at suse.com  Mon Jul 30 03:37:41 2018
From: neilb at suse.com (NeilBrown)
Date: Mon, 30 Jul 2018 13:37:41 +1000
Subject: [lustre-devel] [PATCH 12/22] Revert "staging: lustre: obdclass: Use
 list_for_each_entry_safe"
In-Reply-To: <153292153459.13840.17465048403476297915.stgit@noble>
References: <153292153459.13840.17465048403476297915.stgit@noble>
Message-ID: <153292186108.13840.4929208218603720715.stgit@noble>

This reverts commit 6069f756d42160e454f49286183712514db3ca6b.

These loops really want to remove everything, and using a
  while(!list_empty())
loop makes this more obvious.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 .../staging/lustre/lustre/obdclass/lustre_peer.c   |    5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/drivers/staging/lustre/lustre/obdclass/lustre_peer.c b/drivers/staging/lustre/lustre/obdclass/lustre_peer.c
index e286a2665423..7fc62b7e95b4 100644
--- a/drivers/staging/lustre/lustre/obdclass/lustre_peer.c
+++ b/drivers/staging/lustre/lustre/obdclass/lustre_peer.c
@@ -146,7 +146,6 @@ int class_del_uuid(const char *uuid)
 {
 	LIST_HEAD(deathrow);
 	struct uuid_nid_data *data;
-	struct uuid_nid_data *temp;
 
 	spin_lock(&g_uuid_lock);
 	if (uuid) {
@@ -169,7 +168,9 @@ int class_del_uuid(const char *uuid)
 		return -EINVAL;
 	}
 
-	list_for_each_entry_safe(data, temp, &deathrow, un_list) {
+	while (!list_empty(&deathrow)) {
+		data = list_entry(deathrow.next, struct uuid_nid_data,
+				  un_list);
 		list_del(&data->un_list);
 
 		CDEBUG(D_INFO, "del uuid %s %s/%d\n",



From neilb at suse.com  Mon Jul 30 03:37:41 2018
From: neilb at suse.com (NeilBrown)
Date: Mon, 30 Jul 2018 13:37:41 +1000
Subject: [lustre-devel] [PATCH 13/22] Revert "staging: lustre: lnet: Use
 list_for_each_entry_safe"
In-Reply-To: <153292153459.13840.17465048403476297915.stgit@noble>
References: <153292153459.13840.17465048403476297915.stgit@noble>
Message-ID: <153292186113.13840.10926881392361029642.stgit@noble>

This reverts commit 0daec763260e4f0d8038512b72971ddbcf1c63a1.

These loops really want to remove everything, and using a
  while(!list_empty())
loop makes this more obvious.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 .../staging/lustre/lnet/klnds/socklnd/socklnd_cb.c |    5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c
index 3f69618ad85b..d531847305e4 100644
--- a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c
+++ b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c
@@ -2260,12 +2260,13 @@ static inline void
 ksocknal_flush_stale_txs(struct ksock_peer *peer)
 {
 	struct ksock_tx *tx;
-	struct ksock_tx *tmp;
 	LIST_HEAD(stale_txs);
 
 	write_lock_bh(&ksocknal_data.ksnd_global_lock);
 
-	list_for_each_entry_safe(tx, tmp, &peer->ksnp_tx_queue, tx_list) {
+	while (!list_empty(&peer->ksnp_tx_queue)) {
+		tx = list_entry(peer->ksnp_tx_queue.next, struct ksock_tx, tx_list);
+
 		if (ktime_get_seconds() < tx->tx_deadline)
 			break;
 



From neilb at suse.com  Mon Jul 30 03:37:41 2018
From: neilb at suse.com (NeilBrown)
Date: Mon, 30 Jul 2018 13:37:41 +1000
Subject: [lustre-devel] [PATCH 14/22] Revert: Staging: lustre: Iterate list
 using list_for_each_entry
In-Reply-To: <153292153459.13840.17465048403476297915.stgit@noble>
References: <153292153459.13840.17465048403476297915.stgit@noble>
Message-ID: <153292186117.13840.9041811734708835901.stgit@noble>

This reverts (what is left of) 5a2ca43fa54f561c252c2ceb986daa49e258ab13

These loops really want to remove everything, and using a
  while(!list_empty())
loop makes this more obvious.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 .../staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c |    8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c
index 9cf1f64e3d76..bda67d49597d 100644
--- a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c
+++ b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c
@@ -2106,7 +2106,6 @@ kiblnd_connreq_done(struct kib_conn *conn, int status)
 {
 	struct kib_peer *peer = conn->ibc_peer;
 	struct kib_tx *tx;
-	struct kib_tx *tmp;
 	struct list_head txs;
 	unsigned long flags;
 	int active;
@@ -2197,7 +2196,8 @@ kiblnd_connreq_done(struct kib_conn *conn, int status)
 	 * scheduled.  We won't be using round robin on this first batch.
 	 */
 	spin_lock(&conn->ibc_lock);
-	list_for_each_entry_safe(tx, tmp, &txs, tx_list) {
+	while (!list_empty(&txs)) {
+		tx = list_first_entry(&txs, struct kib_tx, tx_list);
 		list_del(&tx->tx_list);
 
 		kiblnd_queue_tx_locked(tx, conn);
@@ -3185,7 +3185,6 @@ kiblnd_check_conns(int idx)
 	struct list_head *ptmp;
 	struct kib_peer *peer;
 	struct kib_conn *conn;
-	struct kib_conn *tmp;
 	struct list_head *ctmp;
 	unsigned long flags;
 
@@ -3241,7 +3240,8 @@ kiblnd_check_conns(int idx)
 	 * connection. We can only be sure RDMA activity
 	 * has ceased once the QP has been modified.
 	 */
-	list_for_each_entry_safe(conn, tmp, &closes, ibc_connd_list) {
+	while (!list_empty(&closes)) {
+		conn = list_first_entry(&closes, struct kib_conn, ibc_connd_list);
 		list_del(&conn->ibc_connd_list);
 		kiblnd_close_conn(conn, -ETIMEDOUT);
 		kiblnd_conn_decref(conn);



From neilb at suse.com  Mon Jul 30 03:37:41 2018
From: neilb at suse.com (NeilBrown)
Date: Mon, 30 Jul 2018 13:37:41 +1000
Subject: [lustre-devel] [PATCH 15/22] lustre: o2iblnd: convert
 list_for_each_entry_safe() to while(!list_empty())
In-Reply-To: <153292153459.13840.17465048403476297915.stgit@noble>
References: <153292153459.13840.17465048403476297915.stgit@noble>
Message-ID: <153292186122.13840.1195251525324756487.stgit@noble>

These loops are removing all element from a list.
So using while(!list_empty()) makes the intent clearer.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 .../staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c    |   16 ++++++++++------
 1 file changed, 10 insertions(+), 6 deletions(-)

diff --git a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c
index 124870ada28b..830a5bf34c16 100644
--- a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c
+++ b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c
@@ -1283,11 +1283,13 @@ static void kiblnd_destroy_fmr_pool(struct kib_fmr_pool *fpo)
 		if (fpo->fmr.fpo_fmr_pool)
 			ib_destroy_fmr_pool(fpo->fmr.fpo_fmr_pool);
 	} else {
-		struct kib_fast_reg_descriptor *frd, *tmp;
+		struct kib_fast_reg_descriptor *frd;
 		int i = 0;
 
-		list_for_each_entry_safe(frd, tmp, &fpo->fast_reg.fpo_pool_list,
-					 frd_list) {
+		while (!list_empty(&fpo->fast_reg.fpo_pool_list)) {
+			frd = list_first_entry(&fpo->fast_reg.fpo_pool_list,
+					       struct kib_fast_reg_descriptor,
+					       frd_list);
 			list_del(&frd->frd_list);
 			ib_dereg_mr(frd->frd_mr);
 			kfree(frd);
@@ -1362,7 +1364,7 @@ static int kiblnd_alloc_fmr_pool(struct kib_fmr_poolset *fps, struct kib_fmr_poo
 
 static int kiblnd_alloc_freg_pool(struct kib_fmr_poolset *fps, struct kib_fmr_pool *fpo)
 {
-	struct kib_fast_reg_descriptor *frd, *tmp;
+	struct kib_fast_reg_descriptor *frd;
 	int i, rc;
 
 	INIT_LIST_HEAD(&fpo->fast_reg.fpo_pool_list);
@@ -1399,8 +1401,10 @@ static int kiblnd_alloc_freg_pool(struct kib_fmr_poolset *fps, struct kib_fmr_po
 	kfree(frd);
 
 out:
-	list_for_each_entry_safe(frd, tmp, &fpo->fast_reg.fpo_pool_list,
-				 frd_list) {
+	while (!list_empty(&fpo->fast_reg.fpo_pool_list)) {
+		frd = list_first_entry(&fpo->fast_reg.fpo_pool_list,
+				       struct kib_fast_reg_descriptor,
+				       frd_list);
 		list_del(&frd->frd_list);
 		ib_dereg_mr(frd->frd_mr);
 		kfree(frd);



From neilb at suse.com  Mon Jul 30 03:37:41 2018
From: neilb at suse.com (NeilBrown)
Date: Mon, 30 Jul 2018 13:37:41 +1000
Subject: [lustre-devel] [PATCH 16/22] lustre: tracefile: convert
 list_for_each_entry_safe() to while(!list_empty())
In-Reply-To: <153292153459.13840.17465048403476297915.stgit@noble>
References: <153292153459.13840.17465048403476297915.stgit@noble>
Message-ID: <153292186126.13840.13964285941561581566.stgit@noble>

These loops are removing all elements from a list.
So using while(!list_empty()) makes the intent clearer.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 drivers/staging/lustre/lnet/libcfs/tracefile.c |   12 +++++++-----
 1 file changed, 7 insertions(+), 5 deletions(-)

diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.c b/drivers/staging/lustre/lnet/libcfs/tracefile.c
index 07242d4ed41b..a4768e930021 100644
--- a/drivers/staging/lustre/lnet/libcfs/tracefile.c
+++ b/drivers/staging/lustre/lnet/libcfs/tracefile.c
@@ -896,11 +896,12 @@ void cfs_trace_flush_pages(void)
 {
 	struct page_collection pc;
 	struct cfs_trace_page *tage;
-	struct cfs_trace_page *tmp;
 
 	pc.pc_want_daemon_pages = 1;
 	collect_pages(&pc);
-	list_for_each_entry_safe(tage, tmp, &pc.pc_pages, linkage) {
+	while (!list_empty(&pc.pc_pages)) {
+		tage = list_first_entry(&pc.pc_pages,
+					struct cfs_trace_page, linkage);
 		__LASSERT_TAGE_INVARIANT(tage);
 
 		list_del(&tage->linkage);
@@ -1331,7 +1332,6 @@ static void trace_cleanup_on_all_cpus(void)
 {
 	struct cfs_trace_cpu_data *tcd;
 	struct cfs_trace_page *tage;
-	struct cfs_trace_page *tmp;
 	int i, cpu;
 
 	for_each_possible_cpu(cpu) {
@@ -1341,8 +1341,10 @@ static void trace_cleanup_on_all_cpus(void)
 				continue;
 			tcd->tcd_shutting_down = 1;
 
-			list_for_each_entry_safe(tage, tmp, &tcd->tcd_pages,
-						 linkage) {
+			while (!list_empty(&tcd->tcd_pages)) {
+				tage = list_first_entry(&tcd->tcd_pages,
+							struct cfs_trace_page,
+							linkage);
 				__LASSERT_TAGE_INVARIANT(tage);
 
 				list_del(&tage->linkage);



From neilb at suse.com  Mon Jul 30 03:37:41 2018
From: neilb at suse.com (NeilBrown)
Date: Mon, 30 Jul 2018 13:37:41 +1000
Subject: [lustre-devel] [PATCH 17/22] lustre: lib-move: convert
 list_for_each_entry_safe() to while(!list_empty())
In-Reply-To: <153292153459.13840.17465048403476297915.stgit@noble>
References: <153292153459.13840.17465048403476297915.stgit@noble>
Message-ID: <153292186130.13840.15403228991288155064.stgit@noble>

These loops are removing all elements from a list.
So using while(!list_empty()) makes the intent clearer.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 drivers/staging/lustre/lnet/lnet/lib-move.c |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/staging/lustre/lnet/lnet/lib-move.c b/drivers/staging/lustre/lnet/lnet/lib-move.c
index 2756e91b34bb..19cab374b6bc 100644
--- a/drivers/staging/lustre/lnet/lnet/lib-move.c
+++ b/drivers/staging/lustre/lnet/lnet/lib-move.c
@@ -853,7 +853,6 @@ lnet_drop_routed_msgs_locked(struct list_head *list, int cpt)
 {
 	struct list_head drop;
 	struct lnet_msg *msg;
-	struct lnet_msg *tmp;
 
 	INIT_LIST_HEAD(&drop);
 
@@ -861,7 +860,8 @@ lnet_drop_routed_msgs_locked(struct list_head *list, int cpt)
 
 	lnet_net_unlock(cpt);
 
-	list_for_each_entry_safe(msg, tmp, &drop, msg_list) {
+	while(!list_empty(&drop)) {
+		msg = list_first_entry(&drop, struct lnet_msg, msg_list);
 		lnet_ni_recv(msg->msg_rxpeer->lp_ni, msg->msg_private, NULL,
 			     0, 0, 0, msg->msg_hdr.payload_length);
 		list_del_init(&msg->msg_list);



From neilb at suse.com  Mon Jul 30 03:37:41 2018
From: neilb at suse.com (NeilBrown)
Date: Mon, 30 Jul 2018 13:37:41 +1000
Subject: [lustre-devel] [PATCH 18/22] lustre: net_fault: convert
 list_for_each_entry_safe() to while(!list_empty())
In-Reply-To: <153292153459.13840.17465048403476297915.stgit@noble>
References: <153292153459.13840.17465048403476297915.stgit@noble>
Message-ID: <153292186135.13840.3626019723821443531.stgit@noble>

These loops are removing all elements from a list.
So using while(!list_empty()) makes the intent clearer.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 drivers/staging/lustre/lnet/lnet/net_fault.c |    7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/drivers/staging/lustre/lnet/lnet/net_fault.c b/drivers/staging/lustre/lnet/lnet/net_fault.c
index cb1f9053ef48..41d6131ee15a 100644
--- a/drivers/staging/lustre/lnet/lnet/net_fault.c
+++ b/drivers/staging/lustre/lnet/lnet/net_fault.c
@@ -216,7 +216,8 @@ lnet_drop_rule_del(lnet_nid_t src, lnet_nid_t dst)
 	}
 	lnet_net_unlock(LNET_LOCK_EX);
 
-	list_for_each_entry_safe(rule, tmp, &zombies, dr_link) {
+	while (!list_empty(&zombies)) {
+		rule = list_first_entry(&zombies, struct lnet_drop_rule, dr_link);
 		CDEBUG(D_NET, "Remove drop rule: src %s->dst: %s (1/%d, %d)\n",
 		       libcfs_nid2str(rule->dr_attr.fa_src),
 		       libcfs_nid2str(rule->dr_attr.fa_dst),
@@ -841,7 +842,9 @@ lnet_delay_rule_del(lnet_nid_t src, lnet_nid_t dst, bool shutdown)
 		  !list_empty(&rule_list);
 	lnet_net_unlock(LNET_LOCK_EX);
 
-	list_for_each_entry_safe(rule, tmp, &rule_list, dl_link) {
+	while (!list_empty(&rule_list)) {
+		rule = list_first_entry(&rule_list,
+					struct lnet_delay_rule, dl_link);
 		list_del_init(&rule->dl_link);
 
 		del_timer_sync(&rule->dl_timer);



From neilb at suse.com  Mon Jul 30 03:37:41 2018
From: neilb at suse.com (NeilBrown)
Date: Mon, 30 Jul 2018 13:37:41 +1000
Subject: [lustre-devel] [PATCH 19/22] lustre: fld_request: convert
 list_for_each_entry_safe() to while(!list_empty())
In-Reply-To: <153292153459.13840.17465048403476297915.stgit@noble>
References: <153292153459.13840.17465048403476297915.stgit@noble>
Message-ID: <153292186139.13840.14587588638167610162.stgit@noble>

These loops are removing all elements from a list.
So using while(!list_empty()) makes the intent clearer.


Signed-off-by: NeilBrown <neilb at suse.com>
---
 drivers/staging/lustre/lustre/fld/fld_request.c |    6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/drivers/staging/lustre/lustre/fld/fld_request.c b/drivers/staging/lustre/lustre/fld/fld_request.c
index 97f7ea632346..7b0365b3e413 100644
--- a/drivers/staging/lustre/lustre/fld/fld_request.c
+++ b/drivers/staging/lustre/lustre/fld/fld_request.c
@@ -280,10 +280,12 @@ EXPORT_SYMBOL(fld_client_init);
 
 void fld_client_fini(struct lu_client_fld *fld)
 {
-	struct lu_fld_target *target, *tmp;
+	struct lu_fld_target *target;
 
 	spin_lock(&fld->lcf_lock);
-	list_for_each_entry_safe(target, tmp, &fld->lcf_targets, ft_chain) {
+	while (!list_empty(&fld->lcf_targets)) {
+		target = list_first_entry(&fld->lcf_targets,
+					  struct lu_fld_target, ft_chain);
 		fld->lcf_count--;
 		list_del(&target->ft_chain);
 		if (target->ft_exp)



From neilb at suse.com  Mon Jul 30 03:37:41 2018
From: neilb at suse.com (NeilBrown)
Date: Mon, 30 Jul 2018 13:37:41 +1000
Subject: [lustre-devel] [PATCH 20/22] lustre: lov_obd: convert
 list_for_each_entry_safe() to while(!list_empty())
In-Reply-To: <153292153459.13840.17465048403476297915.stgit@noble>
References: <153292153459.13840.17465048403476297915.stgit@noble>
Message-ID: <153292186143.13840.16227510388221165130.stgit@noble>

These loops are removing all elements from a list.
So using while(!list_empty()) makes the intent clearer.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 drivers/staging/lustre/lustre/lov/lov_obd.c |    5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/drivers/staging/lustre/lustre/lov/lov_obd.c b/drivers/staging/lustre/lustre/lov/lov_obd.c
index ee0898cbd9c9..0dd471cfe9a1 100644
--- a/drivers/staging/lustre/lustre/lov/lov_obd.c
+++ b/drivers/staging/lustre/lustre/lov/lov_obd.c
@@ -83,7 +83,7 @@ static void lov_putref(struct obd_device *obd)
 	if (atomic_dec_and_test(&lov->lov_refcount) && lov->lov_death_row) {
 		LIST_HEAD(kill);
 		int i;
-		struct lov_tgt_desc *tgt, *n;
+		struct lov_tgt_desc *tgt;
 
 		CDEBUG(D_CONFIG, "destroying %d lov targets\n",
 		       lov->lov_death_row);
@@ -103,7 +103,8 @@ static void lov_putref(struct obd_device *obd)
 		}
 		mutex_unlock(&lov->lov_lock);
 
-		list_for_each_entry_safe(tgt, n, &kill, ltd_kill) {
+		while (!list_empty(&kill)) {
+			tgt = list_first_entry(&kill, struct lov_tgt_desc, ltd_kill);
 			list_del(&tgt->ltd_kill);
 			/* Disconnect */
 			__lov_del_obd(obd, tgt);



From neilb at suse.com  Mon Jul 30 03:37:41 2018
From: neilb at suse.com (NeilBrown)
Date: Mon, 30 Jul 2018 13:37:41 +1000
Subject: [lustre-devel] [PATCH 21/22] lustre: ldlm_request: convert
 list_for_each_entry_safe() to while(!list_empty())
In-Reply-To: <153292153459.13840.17465048403476297915.stgit@noble>
References: <153292153459.13840.17465048403476297915.stgit@noble>
Message-ID: <153292186147.13840.314584204805282590.stgit@noble>

These loops are removing all elements from a list.
So using while(!list_empty()) makes the intent clearer.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 drivers/staging/lustre/lustre/ldlm/ldlm_request.c |    6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_request.c b/drivers/staging/lustre/lustre/ldlm/ldlm_request.c
index cdc52eed6d85..80260b07f0f0 100644
--- a/drivers/staging/lustre/lustre/ldlm/ldlm_request.c
+++ b/drivers/staging/lustre/lustre/ldlm/ldlm_request.c
@@ -2000,7 +2000,7 @@ int ldlm_replay_locks(struct obd_import *imp)
 {
 	struct ldlm_namespace *ns = imp->imp_obd->obd_namespace;
 	LIST_HEAD(list);
-	struct ldlm_lock *lock, *next;
+	struct ldlm_lock *lock;
 	int rc = 0;
 
 	LASSERT(atomic_read(&imp->imp_replay_inflight) == 0);
@@ -2017,7 +2017,9 @@ int ldlm_replay_locks(struct obd_import *imp)
 
 	ldlm_namespace_foreach(ns, ldlm_chain_lock_for_replay, &list);
 
-	list_for_each_entry_safe(lock, next, &list, l_pending_chain) {
+	while (!list_empty(&list)) {
+		lock = list_first_entry(&list, struct ldlm_lock,
+					l_pending_chain);
 		list_del_init(&lock->l_pending_chain);
 		if (rc) {
 			LDLM_LOCK_RELEASE(lock);



From neilb at suse.com  Mon Jul 30 03:37:41 2018
From: neilb at suse.com (NeilBrown)
Date: Mon, 30 Jul 2018 13:37:41 +1000
Subject: [lustre-devel] [PATCH 22/22] lustre: sec_config: convert
 list_for_each_entry_safe() to while(!list_empty())
In-Reply-To: <153292153459.13840.17465048403476297915.stgit@noble>
References: <153292153459.13840.17465048403476297915.stgit@noble>
Message-ID: <153292186151.13840.8095964948258952076.stgit@noble>

These loops are removing all elements from a list.
So using while(!list_empty()) makes the intent clearer.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 drivers/staging/lustre/lustre/ptlrpc/sec_config.c |    7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/drivers/staging/lustre/lustre/ptlrpc/sec_config.c b/drivers/staging/lustre/lustre/ptlrpc/sec_config.c
index 2389f9a8f534..1844ada6780f 100644
--- a/drivers/staging/lustre/lustre/ptlrpc/sec_config.c
+++ b/drivers/staging/lustre/lustre/ptlrpc/sec_config.c
@@ -839,12 +839,13 @@ int sptlrpc_conf_init(void)
 
 void sptlrpc_conf_fini(void)
 {
-	struct sptlrpc_conf *conf, *conf_next;
+	struct sptlrpc_conf *conf;
 
 	mutex_lock(&sptlrpc_conf_lock);
-	list_for_each_entry_safe(conf, conf_next, &sptlrpc_confs, sc_list) {
+	while (!list_empty(&sptlrpc_confs)) {
+		conf = list_first_entry(&sptlrpc_confs,
+					struct sptlrpc_conf, sc_list);
 		sptlrpc_conf_free(conf);
 	}
-	LASSERT(list_empty(&sptlrpc_confs));
 	mutex_unlock(&sptlrpc_conf_lock);
 }



From neilb at suse.com  Mon Jul 30 03:45:39 2018
From: neilb at suse.com (NeilBrown)
Date: Mon, 30 Jul 2018 13:45:39 +1000
Subject: [lustre-devel] [PATCH 0/6] lustre: convert list_for_each() to
 list_for_each_entry()
Message-ID: <153292233170.26104.16164388413209501300.stgit@noble>

list_for_each_entry(foo) is generally preferred to
  list_for_each(l,...)
     foo = list_entry(l,...)

as there is less noise in the code.
The first patch contains seveveral trivial conversions.
The remaining patches are less obvious and are separate
so they are easier to review.

NeilBrown

The following series implements...

---

NeilBrown (6):
      lustre: convert list_for_each() to list_for_each_entry().
      lustre: lnet/config: convert list_for_each to list_for_each_entry
      lustre: convert list_for_each in ksocknal_create_routes
      lustre: convert list_for_each in ksocknal_close_conn_locked()
      lustre: convert list_for_each() in ksocknal_push_peer()
      lustre: convert list_for_each() in ksocknal_debug_peerhash()


 .../staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c    |   17 --
 .../staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c |   20 +-
 .../staging/lustre/lnet/klnds/socklnd/socklnd.c    |  166 +++++++-------------
 .../staging/lustre/lnet/klnds/socklnd/socklnd_cb.c |   20 +-
 drivers/staging/lustre/lnet/lnet/api-ni.c          |   29 +--
 drivers/staging/lustre/lnet/lnet/config.c          |   28 +--
 drivers/staging/lustre/lnet/lnet/lib-move.c        |    9 -
 drivers/staging/lustre/lnet/lnet/router.c          |   47 +-----
 drivers/staging/lustre/lnet/selftest/conrpc.c      |    7 -
 drivers/staging/lustre/lustre/obdclass/genops.c    |    4 
 .../staging/lustre/lustre/obdecho/echo_client.c    |    4 
 11 files changed, 99 insertions(+), 252 deletions(-)

--
Signature


From neilb at suse.com  Mon Jul 30 03:45:39 2018
From: neilb at suse.com (NeilBrown)
Date: Mon, 30 Jul 2018 13:45:39 +1000
Subject: [lustre-devel] [PATCH 1/6] lustre: convert list_for_each() to
 list_for_each_entry().
In-Reply-To: <153292233170.26104.16164388413209501300.stgit@noble>
References: <153292233170.26104.16164388413209501300.stgit@noble>
Message-ID: <153292233953.26104.11218548856426762220.stgit@noble>

Where we have
  struct list_head *tmp;
  list_for_each(tmp,....) {
     foo = list_entry(tmp, ....);
     ...
  }

convert to
  list_for_each_entry(foo, .....) {
  .....
  }

This patch only changes instances which are straight forward, where
neither the 'tmp' variable nor the 'foo' variable is not used outside
the loop in any way that isn't trivially correct.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 .../staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c    |   17 ++-----
 .../staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c |   20 ++------
 .../staging/lustre/lnet/klnds/socklnd/socklnd.c    |   51 +++++---------------
 .../staging/lustre/lnet/klnds/socklnd/socklnd_cb.c |   20 ++------
 drivers/staging/lustre/lnet/lnet/api-ni.c          |   29 ++---------
 drivers/staging/lustre/lnet/lnet/config.c          |   17 ++-----
 drivers/staging/lustre/lnet/lnet/lib-move.c        |    9 +---
 drivers/staging/lustre/lnet/lnet/router.c          |   47 ++++--------------
 drivers/staging/lustre/lnet/selftest/conrpc.c      |    7 +--
 drivers/staging/lustre/lustre/obdclass/genops.c    |    4 --
 .../staging/lustre/lustre/obdecho/echo_client.c    |    4 --
 11 files changed, 53 insertions(+), 172 deletions(-)

diff --git a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c
index 830a5bf34c16..e15ad94151bd 100644
--- a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c
+++ b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c
@@ -386,11 +386,9 @@ struct kib_peer *kiblnd_find_peer_locked(lnet_nid_t nid)
 	 * that this creates
 	 */
 	struct list_head *peer_list = kiblnd_nid2peerlist(nid);
-	struct list_head *tmp;
 	struct kib_peer *peer;
 
-	list_for_each(tmp, peer_list) {
-		peer = list_entry(tmp, struct kib_peer, ibp_list);
+	list_for_each_entry(peer, peer_list, ibp_list) {
 		LASSERT(!kiblnd_peer_idle(peer));
 
 		if (peer->ibp_nid != nid)
@@ -419,15 +417,13 @@ static int kiblnd_get_peer_info(struct lnet_ni *ni, int index,
 				lnet_nid_t *nidp, int *count)
 {
 	struct kib_peer *peer;
-	struct list_head *ptmp;
 	int i;
 	unsigned long flags;
 
 	read_lock_irqsave(&kiblnd_data.kib_global_lock, flags);
 
 	for (i = 0; i < kiblnd_data.kib_peer_hash_size; i++) {
-		list_for_each(ptmp, &kiblnd_data.kib_peers[i]) {
-			peer = list_entry(ptmp, struct kib_peer, ibp_list);
+		list_for_each_entry(peer, &kiblnd_data.kib_peers[i], ibp_list) {
 			LASSERT(!kiblnd_peer_idle(peer));
 
 			if (peer->ibp_ni != ni)
@@ -526,28 +522,23 @@ static int kiblnd_del_peer(struct lnet_ni *ni, lnet_nid_t nid)
 static struct kib_conn *kiblnd_get_conn_by_idx(struct lnet_ni *ni, int index)
 {
 	struct kib_peer *peer;
-	struct list_head *ptmp;
 	struct kib_conn *conn;
-	struct list_head *ctmp;
 	int i;
 	unsigned long flags;
 
 	read_lock_irqsave(&kiblnd_data.kib_global_lock, flags);
 
 	for (i = 0; i < kiblnd_data.kib_peer_hash_size; i++) {
-		list_for_each(ptmp, &kiblnd_data.kib_peers[i]) {
-			peer = list_entry(ptmp, struct kib_peer, ibp_list);
+		list_for_each_entry(peer, &kiblnd_data.kib_peers[i], ibp_list) {
 			LASSERT(!kiblnd_peer_idle(peer));
 
 			if (peer->ibp_ni != ni)
 				continue;
 
-			list_for_each(ctmp, &peer->ibp_conns) {
+			list_for_each_entry(conn, &peer->ibp_conns, ibc_list) {
 				if (index-- > 0)
 					continue;
 
-				conn = list_entry(ctmp, struct kib_conn,
-						  ibc_list);
 				kiblnd_conn_addref(conn);
 				read_unlock_irqrestore(
 					&kiblnd_data.kib_global_lock,
diff --git a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c
index bda67d49597d..2f7a64f2f13a 100644
--- a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c
+++ b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c
@@ -225,11 +225,9 @@ kiblnd_post_rx(struct kib_rx *rx, int credit)
 static struct kib_tx *
 kiblnd_find_waiting_tx_locked(struct kib_conn *conn, int txtype, __u64 cookie)
 {
-	struct list_head *tmp;
-
-	list_for_each(tmp, &conn->ibc_active_txs) {
-		struct kib_tx *tx = list_entry(tmp, struct kib_tx, tx_list);
+	struct kib_tx *tx;
 
+	list_for_each_entry(tx, &conn->ibc_active_txs, tx_list) {
 		LASSERT(!tx->tx_queued);
 		LASSERT(tx->tx_sending || tx->tx_waiting);
 
@@ -3142,11 +3140,8 @@ static int
 kiblnd_check_txs_locked(struct kib_conn *conn, struct list_head *txs)
 {
 	struct kib_tx *tx;
-	struct list_head *ttmp;
-
-	list_for_each(ttmp, txs) {
-		tx = list_entry(ttmp, struct kib_tx, tx_list);
 
+	list_for_each_entry(tx, txs, tx_list) {
 		if (txs != &conn->ibc_active_txs) {
 			LASSERT(tx->tx_queued);
 		} else {
@@ -3182,10 +3177,8 @@ kiblnd_check_conns(int idx)
 	LIST_HEAD(closes);
 	LIST_HEAD(checksends);
 	struct list_head *peers = &kiblnd_data.kib_peers[idx];
-	struct list_head *ptmp;
 	struct kib_peer *peer;
 	struct kib_conn *conn;
-	struct list_head *ctmp;
 	unsigned long flags;
 
 	/*
@@ -3195,15 +3188,12 @@ kiblnd_check_conns(int idx)
 	 */
 	read_lock_irqsave(&kiblnd_data.kib_global_lock, flags);
 
-	list_for_each(ptmp, peers) {
-		peer = list_entry(ptmp, struct kib_peer, ibp_list);
+	list_for_each_entry(peer, peers, ibp_list) {
 
-		list_for_each(ctmp, &peer->ibp_conns) {
+		list_for_each_entry(conn, &peer->ibp_conns, ibc_list) {
 			int timedout;
 			int sendnoop;
 
-			conn = list_entry(ctmp, struct kib_conn, ibc_list);
-
 			LASSERT(conn->ibc_state == IBLND_CONN_ESTABLISHED);
 
 			spin_lock(&conn->ibc_lock);
diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c
index a48b1b9a850b..491d2ed5b673 100644
--- a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c
+++ b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c
@@ -250,9 +250,7 @@ ksocknal_get_peer_info(struct lnet_ni *ni, int index,
 		       int *port, int *conn_count, int *share_count)
 {
 	struct ksock_peer *peer;
-	struct list_head *ptmp;
 	struct ksock_route *route;
-	struct list_head *rtmp;
 	int i;
 	int j;
 	int rc = -ENOENT;
@@ -260,8 +258,7 @@ ksocknal_get_peer_info(struct lnet_ni *ni, int index,
 	read_lock(&ksocknal_data.ksnd_global_lock);
 
 	for (i = 0; i < ksocknal_data.ksnd_peer_hash_size; i++) {
-		list_for_each(ptmp, &ksocknal_data.ksnd_peers[i]) {
-			peer = list_entry(ptmp, struct ksock_peer, ksnp_list);
+		list_for_each_entry(peer, &ksocknal_data.ksnd_peers[i], ksnp_list) {
 
 			if (peer->ksnp_ni != ni)
 				continue;
@@ -295,13 +292,11 @@ ksocknal_get_peer_info(struct lnet_ni *ni, int index,
 				goto out;
 			}
 
-			list_for_each(rtmp, &peer->ksnp_routes) {
+			list_for_each_entry(route, &peer->ksnp_routes,
+					    ksnr_list) {
 				if (index-- > 0)
 					continue;
 
-				route = list_entry(rtmp, struct ksock_route,
-						   ksnr_list);
-
 				*id = peer->ksnp_id;
 				*myip = route->ksnr_myipaddr;
 				*peer_ip = route->ksnr_ipaddr;
@@ -368,7 +363,6 @@ ksocknal_associate_route_conn_locked(struct ksock_route *route,
 static void
 ksocknal_add_route_locked(struct ksock_peer *peer, struct ksock_route *route)
 {
-	struct list_head *tmp;
 	struct ksock_conn *conn;
 	struct ksock_route *route2;
 
@@ -379,9 +373,7 @@ ksocknal_add_route_locked(struct ksock_peer *peer, struct ksock_route *route)
 	LASSERT(!route->ksnr_connected);
 
 	/* LASSERT(unique) */
-	list_for_each(tmp, &peer->ksnp_routes) {
-		route2 = list_entry(tmp, struct ksock_route, ksnr_list);
-
+	list_for_each_entry(route2, &peer->ksnp_routes, ksnr_list) {
 		if (route2->ksnr_ipaddr == route->ksnr_ipaddr) {
 			CERROR("Duplicate route %s %pI4h\n",
 			       libcfs_id2str(peer->ksnp_id),
@@ -395,9 +387,7 @@ ksocknal_add_route_locked(struct ksock_peer *peer, struct ksock_route *route)
 	/* peer's routelist takes over my ref on 'route' */
 	list_add_tail(&route->ksnr_list, &peer->ksnp_routes);
 
-	list_for_each(tmp, &peer->ksnp_conns) {
-		conn = list_entry(tmp, struct ksock_conn, ksnc_list);
-
+	list_for_each_entry(conn, &peer->ksnp_conns, ksnc_list) {
 		if (conn->ksnc_ipaddr != route->ksnr_ipaddr)
 			continue;
 
@@ -624,28 +614,22 @@ static struct ksock_conn *
 ksocknal_get_conn_by_idx(struct lnet_ni *ni, int index)
 {
 	struct ksock_peer *peer;
-	struct list_head *ptmp;
 	struct ksock_conn *conn;
-	struct list_head *ctmp;
 	int i;
 
 	read_lock(&ksocknal_data.ksnd_global_lock);
 
 	for (i = 0; i < ksocknal_data.ksnd_peer_hash_size; i++) {
-		list_for_each(ptmp, &ksocknal_data.ksnd_peers[i]) {
-			peer = list_entry(ptmp, struct ksock_peer, ksnp_list);
-
+		list_for_each_entry(peer, &ksocknal_data.ksnd_peers[i], ksnp_list) {
 			LASSERT(!peer->ksnp_closing);
 
 			if (peer->ksnp_ni != ni)
 				continue;
 
-			list_for_each(ctmp, &peer->ksnp_conns) {
+			list_for_each_entry(conn, &peer->ksnp_conns, ksnc_list) {
 				if (index-- > 0)
 					continue;
 
-				conn = list_entry(ctmp, struct ksock_conn,
-						  ksnc_list);
 				ksocknal_conn_addref(conn);
 				read_unlock(&ksocknal_data.ksnd_global_lock);
 				return conn;
@@ -1025,7 +1009,6 @@ ksocknal_create_conn(struct lnet_ni *ni, struct ksock_route *route,
 	rwlock_t *global_lock = &ksocknal_data.ksnd_global_lock;
 	LIST_HEAD(zombies);
 	struct lnet_process_id peerid;
-	struct list_head *tmp;
 	__u64 incarnation;
 	struct ksock_conn *conn;
 	struct ksock_conn *conn2;
@@ -1226,8 +1209,7 @@ ksocknal_create_conn(struct lnet_ni *ni, struct ksock_route *route,
 	 * loopback connection
 	 */
 	if (conn->ksnc_ipaddr != conn->ksnc_myipaddr) {
-		list_for_each(tmp, &peer->ksnp_conns) {
-			conn2 = list_entry(tmp, struct ksock_conn, ksnc_list);
+		list_for_each_entry(conn2, &peer->ksnp_conns, ksnc_list) {
 
 			if (conn2->ksnc_ipaddr != conn->ksnc_ipaddr ||
 			    conn2->ksnc_myipaddr != conn->ksnc_myipaddr ||
@@ -1266,9 +1248,7 @@ ksocknal_create_conn(struct lnet_ni *ni, struct ksock_route *route,
 	 * by routes in my peer to match my own route entries so I don't
 	 * continually create duplicate routes.
 	 */
-	list_for_each(tmp, &peer->ksnp_routes) {
-		route = list_entry(tmp, struct ksock_route, ksnr_list);
-
+	list_for_each_entry(route, &peer->ksnp_routes, ksnr_list) {
 		if (route->ksnr_ipaddr != conn->ksnc_ipaddr)
 			continue;
 
@@ -1980,9 +1960,7 @@ ksocknal_add_interface(struct lnet_ni *ni, __u32 ipaddress, __u32 netmask)
 	int rc;
 	int i;
 	int j;
-	struct list_head *ptmp;
 	struct ksock_peer *peer;
-	struct list_head *rtmp;
 	struct ksock_route *route;
 
 	if (!ipaddress || !netmask)
@@ -2005,18 +1983,15 @@ ksocknal_add_interface(struct lnet_ni *ni, __u32 ipaddress, __u32 netmask)
 		iface->ksni_npeers = 0;
 
 		for (i = 0; i < ksocknal_data.ksnd_peer_hash_size; i++) {
-			list_for_each(ptmp, &ksocknal_data.ksnd_peers[i]) {
-				peer = list_entry(ptmp, struct ksock_peer,
-						  ksnp_list);
+			list_for_each_entry(peer, &ksocknal_data.ksnd_peers[i],
+					    ksnp_list) {
 
 				for (j = 0; j < peer->ksnp_n_passive_ips; j++)
 					if (peer->ksnp_passive_ips[j] == ipaddress)
 						iface->ksni_npeers++;
 
-				list_for_each(rtmp, &peer->ksnp_routes) {
-					route = list_entry(rtmp, struct ksock_route,
-							   ksnr_list);
-
+				list_for_each_entry(route, &peer->ksnp_routes,
+						    ksnr_list) {
 					if (route->ksnr_myipaddr == ipaddress)
 						iface->ksni_nroutes++;
 				}
diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c
index d531847305e4..a5c0e8a9bc40 100644
--- a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c
+++ b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c
@@ -561,18 +561,16 @@ struct ksock_conn *
 ksocknal_find_conn_locked(struct ksock_peer *peer, struct ksock_tx *tx,
 			  int nonblk)
 {
-	struct list_head *tmp;
+	struct ksock_conn *c;
 	struct ksock_conn *conn;
 	struct ksock_conn *typed = NULL;
 	struct ksock_conn *fallback = NULL;
 	int tnob = 0;
 	int fnob = 0;
 
-	list_for_each(tmp, &peer->ksnp_conns) {
-		struct ksock_conn *c;
+	list_for_each_entry(c, &peer->ksnp_conns, ksnc_list) {
 		int nob, rc;
 
-		c = list_entry(tmp, struct ksock_conn, ksnc_list);
 		nob = atomic_read(&c->ksnc_tx_nob) +
 		      c->ksnc_sock->sk->sk_wmem_queued;
 
@@ -729,12 +727,9 @@ struct ksock_route *
 ksocknal_find_connectable_route_locked(struct ksock_peer *peer)
 {
 	time64_t now = ktime_get_seconds();
-	struct list_head *tmp;
 	struct ksock_route *route;
 
-	list_for_each(tmp, &peer->ksnp_routes) {
-		route = list_entry(tmp, struct ksock_route, ksnr_list);
-
+	list_for_each_entry(route, &peer->ksnp_routes, ksnr_list) {
 		LASSERT(!route->ksnr_connecting || route->ksnr_scheduled);
 
 		/* connections being established */
@@ -765,11 +760,9 @@ ksocknal_find_connectable_route_locked(struct ksock_peer *peer)
 struct ksock_route *
 ksocknal_find_connecting_route_locked(struct ksock_peer *peer)
 {
-	struct list_head *tmp;
 	struct ksock_route *route;
 
-	list_for_each(tmp, &peer->ksnp_routes) {
-		route = list_entry(tmp, struct ksock_route, ksnr_list);
+	list_for_each_entry(route, &peer->ksnp_routes, ksnr_list) {
 
 		LASSERT(!route->ksnr_connecting || route->ksnr_scheduled);
 
@@ -2180,13 +2173,10 @@ ksocknal_find_timed_out_conn(struct ksock_peer *peer)
 {
 	/* We're called with a shared lock on ksnd_global_lock */
 	struct ksock_conn *conn;
-	struct list_head *ctmp;
 
-	list_for_each(ctmp, &peer->ksnp_conns) {
+	list_for_each_entry(conn, &peer->ksnp_conns, ksnc_list) {
 		int error;
 
-		conn = list_entry(ctmp, struct ksock_conn, ksnc_list);
-
 		/* Don't need the {get,put}connsock dance to deref ksnc_sock */
 		LASSERT(!conn->ksnc_closing);
 
diff --git a/drivers/staging/lustre/lnet/lnet/api-ni.c b/drivers/staging/lustre/lnet/lnet/api-ni.c
index 51a81075f8d5..14b797802a85 100644
--- a/drivers/staging/lustre/lnet/lnet/api-ni.c
+++ b/drivers/staging/lustre/lnet/lnet/api-ni.c
@@ -269,12 +269,9 @@ static struct lnet_lnd *
 lnet_find_lnd_by_type(__u32 type)
 {
 	struct lnet_lnd *lnd;
-	struct list_head *tmp;
 
 	/* holding lnd mutex */
-	list_for_each(tmp, &the_lnet.ln_lnds) {
-		lnd = list_entry(tmp, struct lnet_lnd, lnd_list);
-
+	list_for_each_entry(lnd, &the_lnet.ln_lnds, lnd_list) {
 		if (lnd->lnd_type == type)
 			return lnd;
 	}
@@ -653,14 +650,11 @@ lnet_unprepare(void)
 struct lnet_ni  *
 lnet_net2ni_locked(__u32 net, int cpt)
 {
-	struct list_head *tmp;
 	struct lnet_ni *ni;
 
 	LASSERT(cpt != LNET_LOCK_EX);
 
-	list_for_each(tmp, &the_lnet.ln_nis) {
-		ni = list_entry(tmp, struct lnet_ni, ni_list);
-
+	list_for_each_entry(ni, &the_lnet.ln_nis, ni_list) {
 		if (LNET_NIDNET(ni->ni_nid) == net) {
 			lnet_ni_addref_locked(ni, cpt);
 			return ni;
@@ -767,13 +761,10 @@ struct lnet_ni  *
 lnet_nid2ni_locked(lnet_nid_t nid, int cpt)
 {
 	struct lnet_ni *ni;
-	struct list_head *tmp;
 
 	LASSERT(cpt != LNET_LOCK_EX);
 
-	list_for_each(tmp, &the_lnet.ln_nis) {
-		ni = list_entry(tmp, struct lnet_ni, ni_list);
-
+	list_for_each_entry(ni, &the_lnet.ln_nis, ni_list) {
 		if (ni->ni_nid == nid) {
 			lnet_ni_addref_locked(ni, cpt);
 			return ni;
@@ -803,14 +794,11 @@ lnet_count_acceptor_nis(void)
 {
 	/* Return the # of NIs that need the acceptor. */
 	int count = 0;
-	struct list_head *tmp;
 	struct lnet_ni *ni;
 	int cpt;
 
 	cpt = lnet_net_lock_current();
-	list_for_each(tmp, &the_lnet.ln_nis) {
-		ni = list_entry(tmp, struct lnet_ni, ni_list);
-
+	list_for_each_entry(ni, &the_lnet.ln_nis, ni_list) {
 		if (ni->ni_lnd->lnd_accept)
 			count++;
 	}
@@ -1731,18 +1719,16 @@ static int
 lnet_get_net_config(struct lnet_ioctl_config_data *config)
 {
 	struct lnet_ni *ni;
-	struct list_head *tmp;
 	int idx = config->cfg_count;
 	int cpt, i = 0;
 	int rc = -ENOENT;
 
 	cpt = lnet_net_lock_current();
 
-	list_for_each(tmp, &the_lnet.ln_nis) {
+	list_for_each_entry(ni, &the_lnet.ln_nis, ni_list) {
 		if (i++ != idx)
 			continue;
 
-		ni = list_entry(tmp, struct lnet_ni, ni_list);
 		lnet_ni_lock(ni);
 		lnet_fill_ni_info(ni, config);
 		lnet_ni_unlock(ni);
@@ -2119,7 +2105,6 @@ int
 LNetGetId(unsigned int index, struct lnet_process_id *id)
 {
 	struct lnet_ni *ni;
-	struct list_head *tmp;
 	int cpt;
 	int rc = -ENOENT;
 
@@ -2127,12 +2112,10 @@ LNetGetId(unsigned int index, struct lnet_process_id *id)
 
 	cpt = lnet_net_lock_current();
 
-	list_for_each(tmp, &the_lnet.ln_nis) {
+	list_for_each_entry(ni, &the_lnet.ln_nis, ni_list) {
 		if (index--)
 			continue;
 
-		ni = list_entry(tmp, struct lnet_ni, ni_list);
-
 		id->nid = ni->ni_nid;
 		id->pid = the_lnet.ln_pid;
 		rc = 0;
diff --git a/drivers/staging/lustre/lnet/lnet/config.c b/drivers/staging/lustre/lnet/lnet/config.c
index 26b799e66e53..96336ecdacaf 100644
--- a/drivers/staging/lustre/lnet/lnet/config.c
+++ b/drivers/staging/lustre/lnet/lnet/config.c
@@ -81,12 +81,9 @@ lnet_issep(char c)
 int
 lnet_net_unique(__u32 net, struct list_head *nilist)
 {
-	struct list_head *tmp;
 	struct lnet_ni *ni;
 
-	list_for_each(tmp, nilist) {
-		ni = list_entry(tmp, struct lnet_ni, ni_list);
-
+	list_for_each_entry(ni, nilist, ni_list) {
 		if (LNET_NIDNET(ni->ni_nid) == net)
 			return 0;
 	}
@@ -942,7 +939,6 @@ lnet_splitnets(char *source, struct list_head *nets)
 	int len;
 	struct lnet_text_buf *tb;
 	struct lnet_text_buf *tb2;
-	struct list_head *t;
 	char *sep;
 	char *bracket;
 	__u32 net;
@@ -983,9 +979,7 @@ lnet_splitnets(char *source, struct list_head *nets)
 			return -EINVAL;
 		}
 
-		list_for_each(t, nets) {
-			tb2 = list_entry(t, struct lnet_text_buf, ltb_list);
-
+		list_for_each_entry(tb2, nets, ltb_list) {
 			if (tb2 == tb)
 				continue;
 
@@ -1074,14 +1068,11 @@ lnet_match_networks(char **networksp, char *ip2nets, __u32 *ipaddrs, int nip)
 			break;
 
 		dup = 0;
-		list_for_each(t, &current_nets) {
-			tb = list_entry(t, struct lnet_text_buf, ltb_list);
+		list_for_each_entry(tb, &current_nets, ltb_list) {
 			net1 = lnet_netspec2net(tb->ltb_text);
 			LASSERT(net1 != LNET_NIDNET(LNET_NID_ANY));
 
-			list_for_each(t2, &matched_nets) {
-				tb2 = list_entry(t2, struct lnet_text_buf,
-						 ltb_list);
+			list_for_each_entry(tb2, &matched_nets, ltb_list) {
 				net2 = lnet_netspec2net(tb2->ltb_text);
 				LASSERT(net2 != LNET_NIDNET(LNET_NID_ANY));
 
diff --git a/drivers/staging/lustre/lnet/lnet/lib-move.c b/drivers/staging/lustre/lnet/lnet/lib-move.c
index 19cab374b6bc..edcafac055ed 100644
--- a/drivers/staging/lustre/lnet/lnet/lib-move.c
+++ b/drivers/staging/lustre/lnet/lnet/lib-move.c
@@ -2288,7 +2288,6 @@ EXPORT_SYMBOL(LNetGet);
 int
 LNetDist(lnet_nid_t dstnid, lnet_nid_t *srcnidp, __u32 *orderp)
 {
-	struct list_head *e;
 	struct lnet_ni *ni;
 	struct lnet_remotenet *rnet;
 	__u32 dstnet = LNET_NIDNET(dstnid);
@@ -2307,9 +2306,7 @@ LNetDist(lnet_nid_t dstnid, lnet_nid_t *srcnidp, __u32 *orderp)
 
 	cpt = lnet_net_lock_current();
 
-	list_for_each(e, &the_lnet.ln_nis) {
-		ni = list_entry(e, struct lnet_ni, ni_list);
-
+	list_for_each_entry(ni, &the_lnet.ln_nis, ni_list) {
 		if (ni->ni_nid == dstnid) {
 			if (srcnidp)
 				*srcnidp = dstnid;
@@ -2346,9 +2343,7 @@ LNetDist(lnet_nid_t dstnid, lnet_nid_t *srcnidp, __u32 *orderp)
 	}
 
 	rn_list = lnet_net2rnethash(dstnet);
-	list_for_each(e, rn_list) {
-		rnet = list_entry(e, struct lnet_remotenet, lrn_list);
-
+	list_for_each_entry(rnet, rn_list, lrn_list) {
 		if (rnet->lrn_net == dstnet) {
 			struct lnet_route *route;
 			struct lnet_route *shortest = NULL;
diff --git a/drivers/staging/lustre/lnet/lnet/router.c b/drivers/staging/lustre/lnet/lnet/router.c
index 965087b7359c..53373372b526 100644
--- a/drivers/staging/lustre/lnet/lnet/router.c
+++ b/drivers/staging/lustre/lnet/lnet/router.c
@@ -292,10 +292,10 @@ int
 lnet_add_route(__u32 net, __u32 hops, lnet_nid_t gateway,
 	       unsigned int priority)
 {
-	struct list_head *e;
 	struct lnet_remotenet *rnet;
 	struct lnet_remotenet *rnet2;
 	struct lnet_route *route;
+	struct lnet_route *route2;
 	struct lnet_ni *ni;
 	int add_route;
 	int rc;
@@ -359,10 +359,8 @@ lnet_add_route(__u32 net, __u32 hops, lnet_nid_t gateway,
 
 	/* Search for a duplicate route (it's a NOOP if it is) */
 	add_route = 1;
-	list_for_each(e, &rnet2->lrn_routes) {
-		struct lnet_route *route2;
+	list_for_each_entry(route2, &rnet2->lrn_routes, lr_list) {
 
-		route2 = list_entry(e, struct lnet_route, lr_list);
 		if (route2->lr_gateway == route->lr_gateway) {
 			add_route = 0;
 			break;
@@ -411,8 +409,6 @@ lnet_check_routes(void)
 	struct lnet_remotenet *rnet;
 	struct lnet_route *route;
 	struct lnet_route *route2;
-	struct list_head *e1;
-	struct list_head *e2;
 	int cpt;
 	struct list_head *rn_list;
 	int i;
@@ -421,17 +417,13 @@ lnet_check_routes(void)
 
 	for (i = 0; i < LNET_REMOTE_NETS_HASH_SIZE; i++) {
 		rn_list = &the_lnet.ln_remote_nets_hash[i];
-		list_for_each(e1, rn_list) {
-			rnet = list_entry(e1, struct lnet_remotenet, lrn_list);
-
+		list_for_each_entry(rnet, rn_list, lrn_list) {
 			route2 = NULL;
-			list_for_each(e2, &rnet->lrn_routes) {
+			list_for_each_entry(route, &rnet->lrn_routes, lr_list) {
 				lnet_nid_t nid1;
 				lnet_nid_t nid2;
 				int net;
 
-				route = list_entry(e2, struct lnet_route, lr_list);
-
 				if (!route2) {
 					route2 = route;
 					continue;
@@ -466,8 +458,6 @@ lnet_del_route(__u32 net, lnet_nid_t gw_nid)
 	struct lnet_peer *gateway;
 	struct lnet_remotenet *rnet;
 	struct lnet_route *route;
-	struct list_head *e1;
-	struct list_head *e2;
 	int rc = -ENOENT;
 	struct list_head *rn_list;
 	int idx = 0;
@@ -486,16 +476,12 @@ lnet_del_route(__u32 net, lnet_nid_t gw_nid)
 		rn_list = lnet_net2rnethash(net);
 
  again:
-	list_for_each(e1, rn_list) {
-		rnet = list_entry(e1, struct lnet_remotenet, lrn_list);
-
+	list_for_each_entry(rnet, rn_list, lrn_list) {
 		if (!(net == LNET_NIDNET(LNET_NID_ANY) ||
 		      net == rnet->lrn_net))
 			continue;
 
-		list_for_each(e2, &rnet->lrn_routes) {
-			route = list_entry(e2, struct lnet_route, lr_list);
-
+		list_for_each_entry(route, &rnet->lrn_routes, lr_list) {
 			gateway = route->lr_gateway;
 			if (!(gw_nid == LNET_NID_ANY ||
 			      gw_nid == gateway->lp_nid))
@@ -576,8 +562,6 @@ int
 lnet_get_route(int idx, __u32 *net, __u32 *hops,
 	       lnet_nid_t *gateway, __u32 *alive, __u32 *priority)
 {
-	struct list_head *e1;
-	struct list_head *e2;
 	struct lnet_remotenet *rnet;
 	struct lnet_route *route;
 	int cpt;
@@ -588,13 +572,8 @@ lnet_get_route(int idx, __u32 *net, __u32 *hops,
 
 	for (i = 0; i < LNET_REMOTE_NETS_HASH_SIZE; i++) {
 		rn_list = &the_lnet.ln_remote_nets_hash[i];
-		list_for_each(e1, rn_list) {
-			rnet = list_entry(e1, struct lnet_remotenet, lrn_list);
-
-			list_for_each(e2, &rnet->lrn_routes) {
-				route = list_entry(e2, struct lnet_route,
-						   lr_list);
-
+		list_for_each_entry(rnet, rn_list, lrn_list) {
+			list_for_each_entry(route, &rnet->lrn_routes, lr_list) {
 				if (!idx--) {
 					*net      = rnet->lrn_net;
 					*hops     = route->lr_hops;
@@ -784,7 +763,6 @@ static void
 lnet_wait_known_routerstate(void)
 {
 	struct lnet_peer *rtr;
-	struct list_head *entry;
 	int all_known;
 
 	LASSERT(the_lnet.ln_rc_state == LNET_RC_STATE_RUNNING);
@@ -793,9 +771,7 @@ lnet_wait_known_routerstate(void)
 		int cpt = lnet_net_lock_current();
 
 		all_known = 1;
-		list_for_each(entry, &the_lnet.ln_routers) {
-			rtr = list_entry(entry, struct lnet_peer, lp_rtr_list);
-
+		list_for_each_entry(rtr, &the_lnet.ln_routers, lp_rtr_list) {
 			if (!rtr->lp_alive_count) {
 				all_known = 0;
 				break;
@@ -1223,7 +1199,6 @@ static int
 lnet_router_checker(void *arg)
 {
 	struct lnet_peer *rtr;
-	struct list_head *entry;
 
 	while (the_lnet.ln_rc_state == LNET_RC_STATE_RUNNING) {
 		__u64 version;
@@ -1234,9 +1209,7 @@ lnet_router_checker(void *arg)
 rescan:
 		version = the_lnet.ln_routers_version;
 
-		list_for_each(entry, &the_lnet.ln_routers) {
-			rtr = list_entry(entry, struct lnet_peer, lp_rtr_list);
-
+		list_for_each_entry(rtr, &the_lnet.ln_routers, lp_rtr_list) {
 			cpt2 = lnet_cpt_of_nid_locked(rtr->lp_nid);
 			if (cpt != cpt2) {
 				lnet_net_unlock(cpt);
diff --git a/drivers/staging/lustre/lnet/selftest/conrpc.c b/drivers/staging/lustre/lnet/selftest/conrpc.c
index 3afde0141db5..e73b956d15e4 100644
--- a/drivers/staging/lustre/lnet/selftest/conrpc.c
+++ b/drivers/staging/lustre/lnet/selftest/conrpc.c
@@ -1327,7 +1327,6 @@ lstcon_rpc_cleanup_wait(void)
 {
 	struct lstcon_rpc_trans *trans;
 	struct lstcon_rpc *crpc;
-	struct list_head *pacer;
 	struct list_head zlist;
 
 	/* Called with hold of global mutex */
@@ -1335,10 +1334,8 @@ lstcon_rpc_cleanup_wait(void)
 	LASSERT(console_session.ses_shutdown);
 
 	while (!list_empty(&console_session.ses_trans_list)) {
-		list_for_each(pacer, &console_session.ses_trans_list) {
-			trans = list_entry(pacer, struct lstcon_rpc_trans,
-					   tas_link);
-
+		list_for_each_entry(trans, &console_session.ses_trans_list,
+				    tas_link) {
 			CDEBUG(D_NET, "Session closed, wakeup transaction %s\n",
 			       lstcon_rpc_trans_name(trans->tas_opc));
 
diff --git a/drivers/staging/lustre/lustre/obdclass/genops.c b/drivers/staging/lustre/lustre/obdclass/genops.c
index 234f383ce6d9..8454b4470b8c 100644
--- a/drivers/staging/lustre/lustre/obdclass/genops.c
+++ b/drivers/staging/lustre/lustre/obdclass/genops.c
@@ -84,12 +84,10 @@ static void obd_device_free(struct obd_device *obd)
 
 static struct obd_type *class_search_type(const char *name)
 {
-	struct list_head *tmp;
 	struct obd_type *type;
 
 	spin_lock(&obd_types_lock);
-	list_for_each(tmp, &obd_types) {
-		type = list_entry(tmp, struct obd_type, typ_chain);
+	list_for_each_entry(type, &obd_types, typ_chain) {
 		if (strcmp(type->typ_name, name) == 0) {
 			spin_unlock(&obd_types_lock);
 			return type;
diff --git a/drivers/staging/lustre/lustre/obdecho/echo_client.c b/drivers/staging/lustre/lustre/obdecho/echo_client.c
index 484b8d6db6ef..3022706c6985 100644
--- a/drivers/staging/lustre/lustre/obdecho/echo_client.c
+++ b/drivers/staging/lustre/lustre/obdecho/echo_client.c
@@ -950,12 +950,10 @@ static int cl_echo_cancel0(struct lu_env *env, struct echo_device *ed,
 {
 	struct echo_client_obd *ec = ed->ed_ec;
 	struct echo_lock       *ecl = NULL;
-	struct list_head	     *el;
 	int found = 0, still_used = 0;
 
 	spin_lock(&ec->ec_lock);
-	list_for_each(el, &ec->ec_locks) {
-		ecl = list_entry(el, struct echo_lock, el_chain);
+	list_for_each_entry(ecl, &ec->ec_locks, el_chain) {
 		CDEBUG(D_INFO, "ecl: %p, cookie: %#llx\n", ecl, ecl->el_cookie);
 		found = (ecl->el_cookie == cookie);
 		if (found) {



From neilb at suse.com  Mon Jul 30 03:45:39 2018
From: neilb at suse.com (NeilBrown)
Date: Mon, 30 Jul 2018 13:45:39 +1000
Subject: [lustre-devel] [PATCH 2/6] lustre: lnet/config: convert
 list_for_each to list_for_each_entry
In-Reply-To: <153292233170.26104.16164388413209501300.stgit@noble>
References: <153292233170.26104.16164388413209501300.stgit@noble>
Message-ID: <153292233958.26104.10972399803477314475.stgit@noble>

This conversion to list_for_each_entry isn't quite trivial
as the 'tmp' loop variables are used elsewhere in the function.
This means we cannot just delete them, and so must inspect the
code to ensure the values aren't used elsewhere - they aren't.

Also we need to introduce new loop variables: ltb1 and ltb2 as
the lnet_text_buf was the some for both the original (nested) loops.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 drivers/staging/lustre/lnet/lnet/config.c |   11 +++++------
 1 file changed, 5 insertions(+), 6 deletions(-)

diff --git a/drivers/staging/lustre/lnet/lnet/config.c b/drivers/staging/lustre/lnet/lnet/config.c
index 96336ecdacaf..091c4f714e84 100644
--- a/drivers/staging/lustre/lnet/lnet/config.c
+++ b/drivers/staging/lustre/lnet/lnet/config.c
@@ -662,6 +662,7 @@ lnet_parse_route(char *str, int *im_a_router)
 	__u32 net;
 	lnet_nid_t nid;
 	struct lnet_text_buf *ltb;
+	struct lnet_text_buf *ltb1, *ltb2;
 	int rc;
 	char *sep;
 	char *token = str;
@@ -760,14 +761,12 @@ lnet_parse_route(char *str, int *im_a_router)
 	LASSERT(!list_empty(&nets));
 	LASSERT(!list_empty(&gateways));
 
-	list_for_each(tmp1, &nets) {
-		ltb = list_entry(tmp1, struct lnet_text_buf, ltb_list);
-		net = libcfs_str2net(ltb->ltb_text);
+	list_for_each_entry(ltb1, &nets, ltb_list) {
+		net = libcfs_str2net(ltb1->ltb_text);
 		LASSERT(net != LNET_NIDNET(LNET_NID_ANY));
 
-		list_for_each(tmp2, &gateways) {
-			ltb = list_entry(tmp2, struct lnet_text_buf, ltb_list);
-			nid = libcfs_str2nid(ltb->ltb_text);
+		list_for_each_entry(ltb2, &gateways, ltb_list) {
+			nid = libcfs_str2nid(ltb2->ltb_text);
 			LASSERT(nid != LNET_NID_ANY);
 
 			if (lnet_islocalnid(nid)) {



From neilb at suse.com  Mon Jul 30 03:45:39 2018
From: neilb at suse.com (NeilBrown)
Date: Mon, 30 Jul 2018 13:45:39 +1000
Subject: [lustre-devel] [PATCH 3/6] lustre: convert list_for_each in
 ksocknal_create_routes
In-Reply-To: <153292233170.26104.16164388413209501300.stgit@noble>
References: <153292233170.26104.16164388413209501300.stgit@noble>
Message-ID: <153292233962.26104.2678883514539213796.stgit@noble>

The two list_for_each() loops in ksocknal_create_routes() cannot be
trivially converted to list_for_each_entry() as the new loop variable
is expected to be NULL after the loop, if the loop completed normally.

If the loop did not complete normally - if an interesting entry was
found - a 'continue' is required in the higher level loop.  Change
this 'continue' to a 'goto' so it can be called in the inner loop, and
we don't need to check for a variable being NULL.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 .../staging/lustre/lnet/klnds/socklnd/socklnd.c    |   29 +++++---------------
 1 file changed, 7 insertions(+), 22 deletions(-)

diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c
index 491d2ed5b673..f495e0faeac4 100644
--- a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c
+++ b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c
@@ -846,7 +846,6 @@ ksocknal_create_routes(struct ksock_peer *peer, int port,
 	rwlock_t *global_lock = &ksocknal_data.ksnd_global_lock;
 	struct lnet_ni *ni = peer->ksnp_ni;
 	struct ksock_net *net = ni->ni_data;
-	struct list_head *rtmp;
 	struct ksock_route *route;
 	struct ksock_interface *iface;
 	struct ksock_interface *best_iface;
@@ -894,17 +893,9 @@ ksocknal_create_routes(struct ksock_peer *peer, int port,
 		}
 
 		/* Already got a route? */
-		route = NULL;
-		list_for_each(rtmp, &peer->ksnp_routes) {
-			route = list_entry(rtmp, struct ksock_route, ksnr_list);
-
-			if (route->ksnr_ipaddr == newroute->ksnr_ipaddr)
-				break;
-
-			route = NULL;
-		}
-		if (route)
-			continue;
+		list_for_each_entry(route, &peer->ksnp_routes, ksnr_list)
+			if (route->ksnr_ipaddr != newroute->ksnr_ipaddr)
+				goto next_ipaddr;
 
 		best_iface = NULL;
 		best_nroutes = 0;
@@ -917,17 +908,9 @@ ksocknal_create_routes(struct ksock_peer *peer, int port,
 			iface = &net->ksnn_interfaces[j];
 
 			/* Using this interface already? */
-			list_for_each(rtmp, &peer->ksnp_routes) {
-				route = list_entry(rtmp, struct ksock_route,
-						   ksnr_list);
-
+			list_for_each_entry(route, &peer->ksnp_routes, ksnr_list)
 				if (route->ksnr_myipaddr == iface->ksni_ipaddr)
-					break;
-
-				route = NULL;
-			}
-			if (route)
-				continue;
+					goto next_iface;
 
 			this_netmatch = (!((iface->ksni_ipaddr ^
 					   newroute->ksnr_ipaddr) &
@@ -942,6 +925,7 @@ ksocknal_create_routes(struct ksock_peer *peer, int port,
 			best_iface = iface;
 			best_netmatch = this_netmatch;
 			best_nroutes = iface->ksni_nroutes;
+		next_iface:;
 		}
 
 		if (!best_iface)
@@ -952,6 +936,7 @@ ksocknal_create_routes(struct ksock_peer *peer, int port,
 
 		ksocknal_add_route_locked(peer, newroute);
 		newroute = NULL;
+	next_ipaddr:;
 	}
 
 	write_unlock_bh(global_lock);



From neilb at suse.com  Mon Jul 30 03:45:39 2018
From: neilb at suse.com (NeilBrown)
Date: Mon, 30 Jul 2018 13:45:39 +1000
Subject: [lustre-devel] [PATCH 4/6] lustre: convert list_for_each in
 ksocknal_close_conn_locked()
In-Reply-To: <153292233170.26104.16164388413209501300.stgit@noble>
References: <153292233170.26104.16164388413209501300.stgit@noble>
Message-ID: <153292233965.26104.17508687927758478856.stgit@noble>

The use of list_for_each in ksocknal_close_conn_locked() cannot be
trivially converted to list_for_each_entry() as the new
loop variable has NULL tests outside the loop.

Change these to use a goto inside the loop.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 .../staging/lustre/lnet/klnds/socklnd/socklnd.c    |   15 ++++-----------
 1 file changed, 4 insertions(+), 11 deletions(-)

diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c
index f495e0faeac4..ea95bff79996 100644
--- a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c
+++ b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c
@@ -1402,7 +1402,6 @@ ksocknal_close_conn_locked(struct ksock_conn *conn, int error)
 	struct ksock_peer *peer = conn->ksnc_peer;
 	struct ksock_route *route;
 	struct ksock_conn *conn2;
-	struct list_head *tmp;
 
 	LASSERT(!peer->ksnp_error);
 	LASSERT(!conn->ksnc_closing);
@@ -1417,19 +1416,13 @@ ksocknal_close_conn_locked(struct ksock_conn *conn, int error)
 		LASSERT(!route->ksnr_deleted);
 		LASSERT(route->ksnr_connected & (1 << conn->ksnc_type));
 
-		conn2 = NULL;
-		list_for_each(tmp, &peer->ksnp_conns) {
-			conn2 = list_entry(tmp, struct ksock_conn, ksnc_list);
-
+		list_for_each_entry(conn2, &peer->ksnp_conns, ksnc_list) {
 			if (conn2->ksnc_route == route &&
 			    conn2->ksnc_type == conn->ksnc_type)
-				break;
-
-			conn2 = NULL;
+				goto conn2_found;
 		}
-		if (!conn2)
-			route->ksnr_connected &= ~(1 << conn->ksnc_type);
-
+		route->ksnr_connected &= ~(1 << conn->ksnc_type);
+	conn2_found:
 		conn->ksnc_route = NULL;
 
 		ksocknal_route_decref(route);     /* drop conn's ref on route */



From neilb at suse.com  Mon Jul 30 03:45:39 2018
From: neilb at suse.com (NeilBrown)
Date: Mon, 30 Jul 2018 13:45:39 +1000
Subject: [lustre-devel] [PATCH 5/6] lustre: convert list_for_each() in
 ksocknal_push_peer()
In-Reply-To: <153292233170.26104.16164388413209501300.stgit@noble>
References: <153292233170.26104.16164388413209501300.stgit@noble>
Message-ID: <153292233968.26104.1980002621622141877.stgit@noble>

The use of list_for_each() in ksocknal_push_peer() cannot be trivially
converted to list_for_each_entry() as the new loop variable
is tested outside the loop.  We can change that test to compare 'i'
to 'index' instead.  Then a trivial conversion is possible.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 .../staging/lustre/lnet/klnds/socklnd/socklnd.c    |    7 ++-----
 1 file changed, 2 insertions(+), 5 deletions(-)

diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c
index ea95bff79996..33335713b371 100644
--- a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c
+++ b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c
@@ -1853,7 +1853,6 @@ ksocknal_push_peer(struct ksock_peer *peer)
 {
 	int index;
 	int i;
-	struct list_head *tmp;
 	struct ksock_conn *conn;
 
 	for (index = 0; ; index++) {
@@ -1862,10 +1861,8 @@ ksocknal_push_peer(struct ksock_peer *peer)
 		i = 0;
 		conn = NULL;
 
-		list_for_each(tmp, &peer->ksnp_conns) {
+		list_for_each_entry(conn, &peer->ksnp_conns, ksnc_list) {
 			if (i++ == index) {
-				conn = list_entry(tmp, struct ksock_conn,
-						  ksnc_list);
 				ksocknal_conn_addref(conn);
 				break;
 			}
@@ -1873,7 +1870,7 @@ ksocknal_push_peer(struct ksock_peer *peer)
 
 		read_unlock(&ksocknal_data.ksnd_global_lock);
 
-		if (!conn)
+		if (i <= index)
 			break;
 
 		ksocknal_lib_push_conn(conn);



From neilb at suse.com  Mon Jul 30 03:45:39 2018
From: neilb at suse.com (NeilBrown)
Date: Mon, 30 Jul 2018 13:45:39 +1000
Subject: [lustre-devel] [PATCH 6/6] lustre: convert list_for_each() in
 ksocknal_debug_peerhash()
In-Reply-To: <153292233170.26104.16164388413209501300.stgit@noble>
References: <153292233170.26104.16164388413209501300.stgit@noble>
Message-ID: <153292233972.26104.9739049020807431895.stgit@noble>

This list_for_each() loop searches for a particular entry,
then acts of in.  It currently acts after the loop by testing
if the variable is NULL.  When we convert to list_for_each_entry()
it won't be NULL.

Change the code so the acting happens inside the loop.
 list_for_each_entry() {
    if (this isn't it)
        continue;
    act on entry;
    goto done; // break out of 2 loops
}

Signed-off-by: NeilBrown <neilb at suse.com>
---
 .../staging/lustre/lnet/klnds/socklnd/socklnd.c    |   64 +++++++++-----------
 1 file changed, 28 insertions(+), 36 deletions(-)

diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c
index 33335713b371..f0b0480686dc 100644
--- a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c
+++ b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c
@@ -2462,52 +2462,44 @@ static void
 ksocknal_debug_peerhash(struct lnet_ni *ni)
 {
 	struct ksock_peer *peer = NULL;
-	struct list_head *tmp;
 	int i;
 
 	read_lock(&ksocknal_data.ksnd_global_lock);
 
 	for (i = 0; i < ksocknal_data.ksnd_peer_hash_size; i++) {
-		list_for_each(tmp, &ksocknal_data.ksnd_peers[i]) {
-			peer = list_entry(tmp, struct ksock_peer, ksnp_list);
-
-			if (peer->ksnp_ni == ni)
-				break;
+		list_for_each_entry(peer, &ksocknal_data.ksnd_peers[i], ksnp_list) {
+			struct ksock_route *route;
+			struct ksock_conn  *conn;
 
-			peer = NULL;
-		}
-	}
+			if (peer->ksnp_ni != ni)
+				continue;
 
-	if (peer) {
-		struct ksock_route *route;
-		struct ksock_conn  *conn;
-
-		CWARN("Active peer on shutdown: %s, ref %d, scnt %d, closing %d, accepting %d, err %d, zcookie %llu, txq %d, zc_req %d\n",
-		      libcfs_id2str(peer->ksnp_id),
-		      atomic_read(&peer->ksnp_refcount),
-		      peer->ksnp_sharecount, peer->ksnp_closing,
-		      peer->ksnp_accepting, peer->ksnp_error,
-		      peer->ksnp_zc_next_cookie,
-		      !list_empty(&peer->ksnp_tx_queue),
-		      !list_empty(&peer->ksnp_zc_req_list));
-
-		list_for_each(tmp, &peer->ksnp_routes) {
-			route = list_entry(tmp, struct ksock_route, ksnr_list);
-			CWARN("Route: ref %d, schd %d, conn %d, cnted %d, del %d\n",
-			      atomic_read(&route->ksnr_refcount),
-			      route->ksnr_scheduled, route->ksnr_connecting,
-			      route->ksnr_connected, route->ksnr_deleted);
-		}
+			CWARN("Active peer on shutdown: %s, ref %d, scnt %d, closing %d, accepting %d, err %d, zcookie %llu, txq %d, zc_req %d\n",
+			      libcfs_id2str(peer->ksnp_id),
+			      atomic_read(&peer->ksnp_refcount),
+			      peer->ksnp_sharecount, peer->ksnp_closing,
+			      peer->ksnp_accepting, peer->ksnp_error,
+			      peer->ksnp_zc_next_cookie,
+			      !list_empty(&peer->ksnp_tx_queue),
+			      !list_empty(&peer->ksnp_zc_req_list));
+
+			list_for_each_entry(route, &peer->ksnp_routes, ksnr_list) {
+				CWARN("Route: ref %d, schd %d, conn %d, cnted %d, del %d\n",
+				      atomic_read(&route->ksnr_refcount),
+				      route->ksnr_scheduled, route->ksnr_connecting,
+				      route->ksnr_connected, route->ksnr_deleted);
+			}
 
-		list_for_each(tmp, &peer->ksnp_conns) {
-			conn = list_entry(tmp, struct ksock_conn, ksnc_list);
-			CWARN("Conn: ref %d, sref %d, t %d, c %d\n",
-			      atomic_read(&conn->ksnc_conn_refcount),
-			      atomic_read(&conn->ksnc_sock_refcount),
-			      conn->ksnc_type, conn->ksnc_closing);
+			list_for_each_entry(conn, &peer->ksnp_conns, ksnc_list) {
+				CWARN("Conn: ref %d, sref %d, t %d, c %d\n",
+				      atomic_read(&conn->ksnc_conn_refcount),
+				      atomic_read(&conn->ksnc_sock_refcount),
+				      conn->ksnc_type, conn->ksnc_closing);
+			}
+			goto done;
 		}
 	}
-
+done:
 	read_unlock(&ksocknal_data.ksnd_global_lock);
 }
 



From neilb at suse.com  Mon Jul 30 03:49:32 2018
From: neilb at suse.com (NeilBrown)
Date: Mon, 30 Jul 2018 13:49:32 +1000
Subject: [lustre-devel] [PATCH 0/7] lustre: ad-hoc fixes
Message-ID: <153292249340.26496.13985174182824912332.stgit@noble>

There is no real pattern here, just minor tidy-up and minor bug-fix.

Thanks,
NeilBrown


---

NeilBrown (7):
      lustre: use schedule_timeout_$state().
      lustre/libcfs: fix freeing after kmalloc failure.
      lustre/libfs: move debugfs registration from libcfs_setup back to libcfs_init
      lustre: give different tcd_lock types different classes.
      lustre/libcfs: discard cfs_trace_allocate_string_buffer()
      lustre: lnet: convert ni_refs to percpu_refcount.
      lustre: change TASK_NOLOAD to TASK_IDLE.


 .../staging/lustre/include/linux/lnet/lib-lnet.h   |   11 +---
 .../staging/lustre/include/linux/lnet/lib-types.h  |    2 -
 .../staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c    |   12 ++---
 .../staging/lustre/lnet/klnds/socklnd/socklnd.c    |    6 +-
 .../staging/lustre/lnet/klnds/socklnd/socklnd_cb.c |   14 ++----
 drivers/staging/lustre/lnet/libcfs/fail.c          |    3 -
 drivers/staging/lustre/lnet/libcfs/hash.c          |    2 -
 drivers/staging/lustre/lnet/libcfs/module.c        |   14 +++---
 drivers/staging/lustre/lnet/libcfs/tracefile.c     |   49 +++++++++++---------
 drivers/staging/lustre/lnet/libcfs/tracefile.h     |    1 
 drivers/staging/lustre/lnet/lnet/acceptor.c        |    3 -
 drivers/staging/lustre/lnet/lnet/api-ni.c          |   40 +++++++---------
 drivers/staging/lustre/lnet/lnet/config.c          |   13 +++--
 drivers/staging/lustre/lnet/lnet/lib-eq.c          |    4 +-
 drivers/staging/lustre/lnet/lnet/peer.c            |    3 -
 drivers/staging/lustre/lnet/lnet/router.c          |   19 ++------
 drivers/staging/lustre/lnet/lnet/router_proc.c     |    2 -
 drivers/staging/lustre/lnet/selftest/conrpc.c      |    3 -
 drivers/staging/lustre/lnet/selftest/rpc.c         |    3 -
 drivers/staging/lustre/lnet/selftest/selftest.h    |    3 -
 drivers/staging/lustre/lustre/include/lustre_mdc.h |    2 -
 drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c    |    3 -
 drivers/staging/lustre/lustre/ldlm/ldlm_resource.c |    8 +--
 drivers/staging/lustre/lustre/llite/llite_lib.c    |    6 +-
 .../staging/lustre/lustre/obdecho/echo_client.c    |    3 -
 drivers/staging/lustre/lustre/ptlrpc/client.c      |    4 --
 drivers/staging/lustre/lustre/ptlrpc/sec.c         |    3 -
 27 files changed, 101 insertions(+), 135 deletions(-)

--
Signature


From neilb at suse.com  Mon Jul 30 03:49:32 2018
From: neilb at suse.com (NeilBrown)
Date: Mon, 30 Jul 2018 13:49:32 +1000
Subject: [lustre-devel] [PATCH 1/7] lustre: use schedule_timeout_$state().
In-Reply-To: <153292249340.26496.13985174182824912332.stgit@noble>
References: <153292249340.26496.13985174182824912332.stgit@noble>
Message-ID: <153292257279.26496.13641642926795469723.stgit@noble>

Lustre has many calls to
   set_current_state(STATE);
   schedule_timeout(time);


These can more easily be done as
    schedule_timeout_STATE(time);

Also clean up some oddities, such as setting the state
to TASK_RUNNING after the timeout, and simplify
some time calculations.

Some schedule_timeout() calls remain as the state was set earlier,
before an 'add_wait_queue()'.  It would be incorrect to convert these.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 .../staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c    |   12 ++++--------
 .../staging/lustre/lnet/klnds/socklnd/socklnd.c    |    6 ++----
 .../staging/lustre/lnet/klnds/socklnd/socklnd_cb.c |   14 ++++++--------
 drivers/staging/lustre/lnet/libcfs/fail.c          |    3 +--
 drivers/staging/lustre/lnet/libcfs/tracefile.c     |    3 +--
 drivers/staging/lustre/lnet/lnet/acceptor.c        |    3 +--
 drivers/staging/lustre/lnet/lnet/api-ni.c          |    6 ++----
 drivers/staging/lustre/lnet/lnet/peer.c            |    3 +--
 drivers/staging/lustre/lnet/lnet/router.c          |   19 +++++--------------
 drivers/staging/lustre/lnet/selftest/conrpc.c      |    3 +--
 drivers/staging/lustre/lnet/selftest/rpc.c         |    3 +--
 drivers/staging/lustre/lnet/selftest/selftest.h    |    3 +--
 drivers/staging/lustre/lustre/include/lustre_mdc.h |    2 +-
 drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c    |    3 +--
 drivers/staging/lustre/lustre/ldlm/ldlm_resource.c |    8 +++-----
 drivers/staging/lustre/lustre/llite/llite_lib.c    |    6 ++----
 .../staging/lustre/lustre/obdecho/echo_client.c    |    3 +--
 drivers/staging/lustre/lustre/ptlrpc/client.c      |    4 +---
 drivers/staging/lustre/lustre/ptlrpc/sec.c         |    3 +--
 19 files changed, 36 insertions(+), 71 deletions(-)

diff --git a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c
index e15ad94151bd..f496e6fcc416 100644
--- a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c
+++ b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c
@@ -1201,8 +1201,7 @@ static struct kib_hca_dev *kiblnd_current_hdev(struct kib_dev *dev)
 		if (!(i++ % 50))
 			CDEBUG(D_NET, "%s: Wait for failover\n",
 			       dev->ibd_ifname);
-		set_current_state(TASK_INTERRUPTIBLE);
-		schedule_timeout(HZ / 100);
+		schedule_timeout_interruptible(HZ / 100);
 
 		read_lock_irqsave(&kiblnd_data.kib_global_lock, flags);
 	}
@@ -1916,8 +1915,7 @@ struct list_head *kiblnd_pool_alloc_node(struct kib_poolset *ps)
 		CDEBUG(D_NET, "Another thread is allocating new %s pool, waiting %d HZs for her to complete. trips = %d\n",
 		       ps->ps_name, interval, trips);
 
-		set_current_state(TASK_INTERRUPTIBLE);
-		schedule_timeout(interval);
+		schedule_timeout_interruptible(interval);
 		if (interval < HZ)
 			interval *= 2;
 
@@ -2548,8 +2546,7 @@ static void kiblnd_base_shutdown(void)
 			CDEBUG(((i & (-i)) == i) ? D_WARNING : D_NET,
 			       "Waiting for %d threads to terminate\n",
 			       atomic_read(&kiblnd_data.kib_nthreads));
-			set_current_state(TASK_UNINTERRUPTIBLE);
-			schedule_timeout(HZ);
+			schedule_timeout_uninterruptible(HZ);
 		}
 
 		/* fall through */
@@ -2599,8 +2596,7 @@ static void kiblnd_shutdown(struct lnet_ni *ni)
 			       "%s: waiting for %d peers to disconnect\n",
 			       libcfs_nid2str(ni->ni_nid),
 			       atomic_read(&net->ibn_npeers));
-			set_current_state(TASK_UNINTERRUPTIBLE);
-			schedule_timeout(HZ);
+			schedule_timeout_uninterruptible(HZ);
 		}
 
 		kiblnd_net_fini_pools(net);
diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c
index f0b0480686dc..4dde158451ea 100644
--- a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c
+++ b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c
@@ -2307,8 +2307,7 @@ ksocknal_base_shutdown(void)
 			       "waiting for %d threads to terminate\n",
 				ksocknal_data.ksnd_nthreads);
 			read_unlock(&ksocknal_data.ksnd_global_lock);
-			set_current_state(TASK_UNINTERRUPTIBLE);
-			schedule_timeout(HZ);
+			schedule_timeout_uninterruptible(HZ);
 			read_lock(&ksocknal_data.ksnd_global_lock);
 		}
 		read_unlock(&ksocknal_data.ksnd_global_lock);
@@ -2533,8 +2532,7 @@ ksocknal_shutdown(struct lnet_ni *ni)
 		CDEBUG(((i & (-i)) == i) ? D_WARNING : D_NET, /* power of 2? */
 		       "waiting for %d peers to disconnect\n",
 		       net->ksnn_npeers);
-		set_current_state(TASK_UNINTERRUPTIBLE);
-		schedule_timeout(HZ);
+		schedule_timeout_uninterruptible(HZ);
 
 		ksocknal_debug_peerhash(ni);
 
diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c
index a5c0e8a9bc40..32b76727f400 100644
--- a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c
+++ b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c
@@ -188,10 +188,9 @@ ksocknal_transmit(struct ksock_conn *conn, struct ksock_tx *tx)
 	int rc;
 	int bufnob;
 
-	if (ksocknal_data.ksnd_stall_tx) {
-		set_current_state(TASK_UNINTERRUPTIBLE);
-		schedule_timeout(ksocknal_data.ksnd_stall_tx * HZ);
-	}
+	if (ksocknal_data.ksnd_stall_tx)
+		schedule_timeout_uninterruptible(
+			ksocknal_data.ksnd_stall_tx * HZ);
 
 	LASSERT(tx->tx_resid);
 
@@ -293,10 +292,9 @@ ksocknal_receive(struct ksock_conn *conn)
 	 */
 	int rc;
 
-	if (ksocknal_data.ksnd_stall_rx) {
-		set_current_state(TASK_UNINTERRUPTIBLE);
-		schedule_timeout(ksocknal_data.ksnd_stall_rx * HZ);
-	}
+	if (ksocknal_data.ksnd_stall_rx)
+		schedule_timeout_uninterruptible(
+			ksocknal_data.ksnd_stall_rx * HZ);
 
 	rc = ksocknal_connsock_addref(conn);
 	if (rc) {
diff --git a/drivers/staging/lustre/lnet/libcfs/fail.c b/drivers/staging/lustre/lnet/libcfs/fail.c
index bd86b3b5bc34..6ee4de2178ce 100644
--- a/drivers/staging/lustre/lnet/libcfs/fail.c
+++ b/drivers/staging/lustre/lnet/libcfs/fail.c
@@ -137,8 +137,7 @@ int __cfs_fail_timeout_set(u32 id, u32 value, int ms, int set)
 	if (ret && likely(ms > 0)) {
 		CERROR("cfs_fail_timeout id %x sleeping for %dms\n",
 		       id, ms);
-		set_current_state(TASK_UNINTERRUPTIBLE);
-		schedule_timeout(ms * HZ / 1000);
+		schedule_timeout_uninterruptible(ms * HZ / 1000);
 		CERROR("cfs_fail_timeout id %x awake\n", id);
 	}
 	return ret;
diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.c b/drivers/staging/lustre/lnet/libcfs/tracefile.c
index a4768e930021..d4c80cf254e4 100644
--- a/drivers/staging/lustre/lnet/libcfs/tracefile.c
+++ b/drivers/staging/lustre/lnet/libcfs/tracefile.c
@@ -1208,8 +1208,7 @@ static int tracefiled(void *arg)
 		}
 		init_waitqueue_entry(&__wait, current);
 		add_wait_queue(&tctl->tctl_waitq, &__wait);
-		set_current_state(TASK_INTERRUPTIBLE);
-		schedule_timeout(HZ);
+		schedule_timeout_interruptible(HZ);
 		remove_wait_queue(&tctl->tctl_waitq, &__wait);
 	}
 	complete(&tctl->tctl_stop);
diff --git a/drivers/staging/lustre/lnet/lnet/acceptor.c b/drivers/staging/lustre/lnet/lnet/acceptor.c
index 5648f17eddc0..3ae3ca1311a1 100644
--- a/drivers/staging/lustre/lnet/lnet/acceptor.c
+++ b/drivers/staging/lustre/lnet/lnet/acceptor.c
@@ -362,8 +362,7 @@ lnet_acceptor(void *arg)
 		if (rc) {
 			if (rc != -EAGAIN) {
 				CWARN("Accept error %d: pausing...\n", rc);
-				set_current_state(TASK_UNINTERRUPTIBLE);
-				schedule_timeout(HZ);
+				schedule_timeout_uninterruptible(HZ);
 			}
 			continue;
 		}
diff --git a/drivers/staging/lustre/lnet/lnet/api-ni.c b/drivers/staging/lustre/lnet/lnet/api-ni.c
index 14b797802a85..cdbbe9cc8d95 100644
--- a/drivers/staging/lustre/lnet/lnet/api-ni.c
+++ b/drivers/staging/lustre/lnet/lnet/api-ni.c
@@ -955,8 +955,7 @@ lnet_ping_md_unlink(struct lnet_ping_info *pinfo,
 	/* NB md could be busy; this just starts the unlink */
 	while (pinfo->pi_features != LNET_PING_FEAT_INVAL) {
 		CDEBUG(D_NET, "Still waiting for ping MD to unlink\n");
-		set_current_state(TASK_NOLOAD);
-		schedule_timeout(HZ);
+		schedule_timeout_idle(HZ);
 	}
 }
 
@@ -1093,8 +1092,7 @@ lnet_clear_zombies_nis_locked(void)
 				CDEBUG(D_WARNING, "Waiting for zombie LNI %s\n",
 				       libcfs_nid2str(ni->ni_nid));
 			}
-			set_current_state(TASK_UNINTERRUPTIBLE);
-			schedule_timeout(HZ);
+			schedule_timeout_uninterruptible(HZ);
 			lnet_net_lock(LNET_LOCK_EX);
 			continue;
 		}
diff --git a/drivers/staging/lustre/lnet/lnet/peer.c b/drivers/staging/lustre/lnet/lnet/peer.c
index 7c303ef6bb34..d9452c322e4d 100644
--- a/drivers/staging/lustre/lnet/lnet/peer.c
+++ b/drivers/staging/lustre/lnet/lnet/peer.c
@@ -136,8 +136,7 @@ lnet_peer_table_deathrow_wait_locked(struct lnet_peer_table *ptable,
 			       "Waiting for %d zombies on peer table\n",
 			       ptable->pt_zombies);
 		}
-		set_current_state(TASK_UNINTERRUPTIBLE);
-		schedule_timeout(HZ >> 1);
+		schedule_timeout_uninterruptible(HZ >> 1);
 		lnet_net_lock(cpt_locked);
 	}
 }
diff --git a/drivers/staging/lustre/lnet/lnet/router.c b/drivers/staging/lustre/lnet/lnet/router.c
index 53373372b526..02241fbc9eaa 100644
--- a/drivers/staging/lustre/lnet/lnet/router.c
+++ b/drivers/staging/lustre/lnet/lnet/router.c
@@ -783,8 +783,7 @@ lnet_wait_known_routerstate(void)
 		if (all_known)
 			return;
 
-		set_current_state(TASK_UNINTERRUPTIBLE);
-		schedule_timeout(HZ);
+		schedule_timeout_uninterruptible(HZ);
 	}
 }
 
@@ -1159,8 +1158,7 @@ lnet_prune_rc_data(int wait_unlink)
 		i++;
 		CDEBUG(((i & (-i)) == i) ? D_WARNING : D_NET,
 		       "Waiting for rc buffers to unlink\n");
-		set_current_state(TASK_UNINTERRUPTIBLE);
-		schedule_timeout(HZ / 4);
+		schedule_timeout_uninterruptible(HZ / 4);
 
 		lnet_net_lock(LNET_LOCK_EX);
 	}
@@ -1236,23 +1234,16 @@ lnet_router_checker(void *arg)
 
 		lnet_prune_rc_data(0); /* don't wait for UNLINK */
 
-		/*
-		 * Call schedule_timeout() here always adds 1 to load average
-		 * because kernel counts # active tasks as nr_running
-		 * + nr_uninterruptible.
-		 */
 		/*
 		 * if there are any routes then wakeup every second.  If
 		 * there are no routes then sleep indefinitely until woken
 		 * up by a user adding a route
 		 */
 		if (!lnet_router_checker_active())
-			wait_event_interruptible(the_lnet.ln_rc_waitq,
-						 lnet_router_checker_active());
+			wait_event_idle(the_lnet.ln_rc_waitq,
+					lnet_router_checker_active());
 		else
-			wait_event_interruptible_timeout(the_lnet.ln_rc_waitq,
-							 false,
-							 HZ);
+			schedule_timeout_idle(HZ);
 	}
 
 	lnet_prune_rc_data(1); /* wait for UNLINK */
diff --git a/drivers/staging/lustre/lnet/selftest/conrpc.c b/drivers/staging/lustre/lnet/selftest/conrpc.c
index e73b956d15e4..7809c1fc6f73 100644
--- a/drivers/staging/lustre/lnet/selftest/conrpc.c
+++ b/drivers/staging/lustre/lnet/selftest/conrpc.c
@@ -1345,8 +1345,7 @@ lstcon_rpc_cleanup_wait(void)
 		mutex_unlock(&console_session.ses_mutex);
 
 		CWARN("Session is shutting down, waiting for termination of transactions\n");
-		set_current_state(TASK_UNINTERRUPTIBLE);
-		schedule_timeout(HZ);
+		schedule_timeout_uninterruptible(HZ);
 
 		mutex_lock(&console_session.ses_mutex);
 	}
diff --git a/drivers/staging/lustre/lnet/selftest/rpc.c b/drivers/staging/lustre/lnet/selftest/rpc.c
index e097ef8414a6..298de41444b3 100644
--- a/drivers/staging/lustre/lnet/selftest/rpc.c
+++ b/drivers/staging/lustre/lnet/selftest/rpc.c
@@ -1603,8 +1603,7 @@ srpc_startup(void)
 	spin_lock_init(&srpc_data.rpc_glock);
 
 	/* 1 second pause to avoid timestamp reuse */
-	set_current_state(TASK_UNINTERRUPTIBLE);
-	schedule_timeout(HZ);
+	schedule_timeout_uninterruptible(HZ);
 	srpc_data.rpc_matchbits = ((__u64)ktime_get_real_seconds()) << 48;
 
 	srpc_data.rpc_state = SRPC_STATE_NONE;
diff --git a/drivers/staging/lustre/lnet/selftest/selftest.h b/drivers/staging/lustre/lnet/selftest/selftest.h
index ad9be095c4ea..9dbb0a51d430 100644
--- a/drivers/staging/lustre/lnet/selftest/selftest.h
+++ b/drivers/staging/lustre/lnet/selftest/selftest.h
@@ -573,8 +573,7 @@ swi_state2str(int state)
 
 #define selftest_wait_events()					\
 	do {							\
-		set_current_state(TASK_UNINTERRUPTIBLE);	\
-		schedule_timeout(HZ / 10);	\
+		schedule_timeout_uninterruptible(HZ / 10);	\
 	} while (0)
 
 #define lst_wait_until(cond, lock, fmt, ...)				\
diff --git a/drivers/staging/lustre/lustre/include/lustre_mdc.h b/drivers/staging/lustre/lustre/include/lustre_mdc.h
index a9c9992a2502..6ac7fc4fa8c6 100644
--- a/drivers/staging/lustre/lustre/include/lustre_mdc.h
+++ b/drivers/staging/lustre/lustre/include/lustre_mdc.h
@@ -124,7 +124,7 @@ static inline void mdc_get_rpc_lock(struct mdc_rpc_lock *lck,
 	 */
 	while (unlikely(lck->rpcl_it == MDC_FAKE_RPCL_IT)) {
 		mutex_unlock(&lck->rpcl_mutex);
-		schedule_timeout(HZ / 4);
+		schedule_timeout_uninterruptible(HZ / 4);
 		goto again;
 	}
 
diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c b/drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c
index 5b125fdc7321..0ee4798f1bb9 100644
--- a/drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c
+++ b/drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c
@@ -167,8 +167,7 @@ static void ldlm_handle_cp_callback(struct ptlrpc_request *req,
 		int to = HZ;
 
 		while (to > 0) {
-			set_current_state(TASK_INTERRUPTIBLE);
-			schedule_timeout(to);
+			schedule_timeout_interruptible(to);
 			if (lock->l_granted_mode == lock->l_req_mode ||
 			    ldlm_is_destroyed(lock))
 				break;
diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_resource.c b/drivers/staging/lustre/lustre/ldlm/ldlm_resource.c
index f06cbd8b6d13..33d73fa8e9d5 100644
--- a/drivers/staging/lustre/lustre/ldlm/ldlm_resource.c
+++ b/drivers/staging/lustre/lustre/ldlm/ldlm_resource.c
@@ -749,11 +749,9 @@ static void cleanup_resource(struct ldlm_resource *res, struct list_head *q,
 			 */
 			unlock_res(res);
 			LDLM_DEBUG(lock, "setting FL_LOCAL_ONLY");
-			if (lock->l_flags & LDLM_FL_FAIL_LOC) {
-				set_current_state(TASK_UNINTERRUPTIBLE);
-				schedule_timeout(4 * HZ);
-				set_current_state(TASK_RUNNING);
-			}
+			if (lock->l_flags & LDLM_FL_FAIL_LOC)
+				schedule_timeout_uninterruptible(4 * HZ);
+
 			if (lock->l_completion_ast)
 				lock->l_completion_ast(lock, LDLM_FL_FAILED,
 						       NULL);
diff --git a/drivers/staging/lustre/lustre/llite/llite_lib.c b/drivers/staging/lustre/lustre/llite/llite_lib.c
index 5c8d0fe7217e..3dedc61d2257 100644
--- a/drivers/staging/lustre/lustre/llite/llite_lib.c
+++ b/drivers/staging/lustre/lustre/llite/llite_lib.c
@@ -706,10 +706,8 @@ void ll_kill_super(struct super_block *sb)
 		sbi->ll_umounting = 1;
 
 		/* wait running statahead threads to quit */
-		while (atomic_read(&sbi->ll_sa_running) > 0) {
-			set_current_state(TASK_UNINTERRUPTIBLE);
-			schedule_timeout(msecs_to_jiffies(MSEC_PER_SEC >> 3));
-		}
+		while (atomic_read(&sbi->ll_sa_running) > 0)
+			schedule_timeout_uninterruptible(HZ >> 3);
 	}
 }
 
diff --git a/drivers/staging/lustre/lustre/obdecho/echo_client.c b/drivers/staging/lustre/lustre/obdecho/echo_client.c
index 3022706c6985..1ddb4a6dd8f3 100644
--- a/drivers/staging/lustre/lustre/obdecho/echo_client.c
+++ b/drivers/staging/lustre/lustre/obdecho/echo_client.c
@@ -751,8 +751,7 @@ static struct lu_device *echo_device_free(const struct lu_env *env,
 	while (!list_empty(&ec->ec_objects)) {
 		spin_unlock(&ec->ec_lock);
 		CERROR("echo_client still has objects at cleanup time, wait for 1 second\n");
-		set_current_state(TASK_UNINTERRUPTIBLE);
-		schedule_timeout(HZ);
+		schedule_timeout_uninterruptible(HZ);
 		lu_site_purge(env, ed->ed_site, -1);
 		spin_lock(&ec->ec_lock);
 	}
diff --git a/drivers/staging/lustre/lustre/ptlrpc/client.c b/drivers/staging/lustre/lustre/ptlrpc/client.c
index 7a3d83c0e50b..91dd09867260 100644
--- a/drivers/staging/lustre/lustre/ptlrpc/client.c
+++ b/drivers/staging/lustre/lustre/ptlrpc/client.c
@@ -761,9 +761,7 @@ int ptlrpc_request_bufs_pack(struct ptlrpc_request *request,
 			/* The RPC is infected, let the test change the
 			 * fail_loc
 			 */
-			set_current_state(TASK_UNINTERRUPTIBLE);
-			schedule_timeout(2 * HZ);
-			set_current_state(TASK_RUNNING);
+			schedule_timeout_uninterruptible(2 * HZ);
 		}
 	}
 
diff --git a/drivers/staging/lustre/lustre/ptlrpc/sec.c b/drivers/staging/lustre/lustre/ptlrpc/sec.c
index 9b60292370a7..9c598710b576 100644
--- a/drivers/staging/lustre/lustre/ptlrpc/sec.c
+++ b/drivers/staging/lustre/lustre/ptlrpc/sec.c
@@ -514,8 +514,7 @@ static int sptlrpc_req_replace_dead_ctx(struct ptlrpc_request *req)
 		       "ctx (%p, fl %lx) doesn't switch, relax a little bit\n",
 		       newctx, newctx->cc_flags);
 
-		set_current_state(TASK_INTERRUPTIBLE);
-		schedule_timeout(msecs_to_jiffies(MSEC_PER_SEC));
+		schedule_timeout_interruptible(HZ);
 	} else if (unlikely(!test_bit(PTLRPC_CTX_UPTODATE_BIT, &newctx->cc_flags))) {
 		/*
 		 * new ctx not up to date yet



From neilb at suse.com  Mon Jul 30 03:49:32 2018
From: neilb at suse.com (NeilBrown)
Date: Mon, 30 Jul 2018 13:49:32 +1000
Subject: [lustre-devel] [PATCH 2/7] lustre/libcfs: fix freeing after kmalloc
 failure.
In-Reply-To: <153292249340.26496.13985174182824912332.stgit@noble>
References: <153292249340.26496.13985174182824912332.stgit@noble>
Message-ID: <153292257285.26496.5271671024215297861.stgit@noble>

The new_bkts array is *not* zeroed (any more) so when
freeing recently allocated buckets on failure, we
must no free beyond the last bucket successfully
allocated.

Fixes: 12e46c461cb9 ("staging: lustre: change some LIBCFS_ALLOC calls to k?alloc(GFP_KERNEL)")
Signed-off-by: NeilBrown <neilb at suse.com>
---
 drivers/staging/lustre/lnet/libcfs/hash.c |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/staging/lustre/lnet/libcfs/hash.c b/drivers/staging/lustre/lnet/libcfs/hash.c
index 48be66f0d654..f452c4540ca1 100644
--- a/drivers/staging/lustre/lnet/libcfs/hash.c
+++ b/drivers/staging/lustre/lnet/libcfs/hash.c
@@ -904,7 +904,7 @@ cfs_hash_buckets_realloc(struct cfs_hash *hs, struct cfs_hash_bucket **old_bkts,
 		new_bkts[i] = kzalloc(cfs_hash_bkt_size(hs), GFP_KERNEL);
 		if (!new_bkts[i]) {
 			cfs_hash_buckets_free(new_bkts, cfs_hash_bkt_size(hs),
-					      old_size, new_size);
+					      old_size, i);
 			return NULL;
 		}
 



From neilb at suse.com  Mon Jul 30 03:49:32 2018
From: neilb at suse.com (NeilBrown)
Date: Mon, 30 Jul 2018 13:49:32 +1000
Subject: [lustre-devel] [PATCH 3/7] lustre/libfs: move debugfs registration
 from libcfs_setup back to libcfs_init
In-Reply-To: <153292249340.26496.13985174182824912332.stgit@noble>
References: <153292249340.26496.13985174182824912332.stgit@noble>
Message-ID: <153292257289.26496.3865601138425685904.stgit@noble>

large memory allocations should be avoided at module-init,
but registering services is appropriate.
So move the registration of debugfs files
back into libcfs_init().
Without this, /sys/kernel/debug/lnet etc are not visible
immediately that libcfs is loaded.
No debugfs file access needs anything allocated by libcfs_setup().

Fixes: 64bf0b1a079d ("staging: lustre: refactor libcfs initialization.")
Signed-off-by: NeilBrown <neilb at suse.com>
---
 drivers/staging/lustre/lnet/libcfs/module.c |    8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/drivers/staging/lustre/lnet/libcfs/module.c b/drivers/staging/lustre/lnet/libcfs/module.c
index bfadfcfa3c44..5d2be941777e 100644
--- a/drivers/staging/lustre/lnet/libcfs/module.c
+++ b/drivers/staging/lustre/lnet/libcfs/module.c
@@ -719,10 +719,6 @@ int libcfs_setup(void)
 		goto err;
 	}
 
-	lnet_insert_debugfs(lnet_table);
-	if (!IS_ERR_OR_NULL(lnet_debugfs_root))
-		lnet_insert_debugfs_links(lnet_debugfs_symlinks);
-
 	CDEBUG(D_OTHER, "portals setup OK\n");
 out:
 	libcfs_active = 1;
@@ -743,6 +739,10 @@ static int libcfs_init(void)
 {
 	int rc;
 
+	lnet_insert_debugfs(lnet_table);
+	if (!IS_ERR_OR_NULL(lnet_debugfs_root))
+		lnet_insert_debugfs_links(lnet_debugfs_symlinks);
+
 	rc = misc_register(&libcfs_dev);
 	if (rc)
 		CERROR("misc_register: error %d\n", rc);



From neilb at suse.com  Mon Jul 30 03:49:32 2018
From: neilb at suse.com (NeilBrown)
Date: Mon, 30 Jul 2018 13:49:32 +1000
Subject: [lustre-devel] [PATCH 4/7] lustre: give different tcd_lock types
 different classes.
In-Reply-To: <153292249340.26496.13985174182824912332.stgit@noble>
References: <153292249340.26496.13985174182824912332.stgit@noble>
Message-ID: <153292257292.26496.6025900918938045378.stgit@noble>

There are three different trace contexts:
 process, softirq, irq.
Each has its own lock (tcd_lock) which is locked
as appropriate for that context.
lockdep currently doesn't see that they are different
and so deduces that the different uses might lead to
deadlocks.
So use separate calls to spin_lock_init() so that they
each get a separate lock class, and lockdep sees no
problem.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 drivers/staging/lustre/lnet/libcfs/tracefile.c |   18 +++++++++++++++++-
 1 file changed, 17 insertions(+), 1 deletion(-)

diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.c b/drivers/staging/lustre/lnet/libcfs/tracefile.c
index d4c80cf254e4..40048165fc16 100644
--- a/drivers/staging/lustre/lnet/libcfs/tracefile.c
+++ b/drivers/staging/lustre/lnet/libcfs/tracefile.c
@@ -1285,7 +1285,23 @@ int cfs_tracefile_init(int max_pages)
 	cfs_tcd_for_each(tcd, i, j) {
 		int factor = pages_factor[i];
 
-		spin_lock_init(&tcd->tcd_lock);
+		/* Note that we have three separate calls so
+		 * they the locks get three separate classes
+		 * and lockdep never thinks they are related.
+		 * As they are used in different interrupt
+		 * contexts, lockdep think the usage would conflict.
+		 */
+		switch(i) {
+		case CFS_TCD_TYPE_PROC:
+			spin_lock_init(&tcd->tcd_lock);
+			break;
+		case CFS_TCD_TYPE_SOFTIRQ:
+			spin_lock_init(&tcd->tcd_lock);
+			break;
+		case CFS_TCD_TYPE_IRQ:
+			spin_lock_init(&tcd->tcd_lock);
+			break;
+		}
 		tcd->tcd_pages_factor = factor;
 		tcd->tcd_type = i;
 		tcd->tcd_cpu = j;



From neilb at suse.com  Mon Jul 30 03:49:32 2018
From: neilb at suse.com (NeilBrown)
Date: Mon, 30 Jul 2018 13:49:32 +1000
Subject: [lustre-devel] [PATCH 5/7] lustre/libcfs: discard
 cfs_trace_allocate_string_buffer()
In-Reply-To: <153292249340.26496.13985174182824912332.stgit@noble>
References: <153292249340.26496.13985174182824912332.stgit@noble>
Message-ID: <153292257295.26496.1353870060207416213.stgit@noble>

cfs_trace_allocate_string_buffer() is a simple wrapper
around kzalloc() that adds little value.  The code is
clearer if we perform the test and the allocation
directly where needed.

Also change the test from '>' to '>=' to ensure we
never try to allocate more than 2 pages, as that seems
to be the intent.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 drivers/staging/lustre/lnet/libcfs/module.c    |    6 +++--
 drivers/staging/lustre/lnet/libcfs/tracefile.c |   28 +++++++++---------------
 drivers/staging/lustre/lnet/libcfs/tracefile.h |    1 -
 3 files changed, 13 insertions(+), 22 deletions(-)

diff --git a/drivers/staging/lustre/lnet/libcfs/module.c b/drivers/staging/lustre/lnet/libcfs/module.c
index 5d2be941777e..1de83b1997c6 100644
--- a/drivers/staging/lustre/lnet/libcfs/module.c
+++ b/drivers/staging/lustre/lnet/libcfs/module.c
@@ -305,9 +305,9 @@ static int proc_dobitmasks(struct ctl_table *table, int write,
 	int is_subsys = (mask == &libcfs_subsystem_debug) ? 1 : 0;
 	int is_printk = (mask == &libcfs_printk) ? 1 : 0;
 
-	rc = cfs_trace_allocate_string_buffer(&tmpstr, tmpstrlen);
-	if (rc < 0)
-		return rc;
+	tmpstr = kzalloc(tmpstrlen, GFP_KERNEL);
+	if (!tmpstr)
+		return -ENOMEM;
 
 	if (!write) {
 		libcfs_debug_mask2str(tmpstr, tmpstrlen, *mask, is_subsys);
diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.c b/drivers/staging/lustre/lnet/libcfs/tracefile.c
index 40048165fc16..b273107b3815 100644
--- a/drivers/staging/lustre/lnet/libcfs/tracefile.c
+++ b/drivers/staging/lustre/lnet/libcfs/tracefile.c
@@ -963,26 +963,16 @@ int cfs_trace_copyout_string(char __user *usr_buffer, int usr_buffer_nob,
 }
 EXPORT_SYMBOL(cfs_trace_copyout_string);
 
-int cfs_trace_allocate_string_buffer(char **str, int nob)
-{
-	if (nob > 2 * PAGE_SIZE)	    /* string must be "sensible" */
-		return -EINVAL;
-
-	*str = kmalloc(nob, GFP_KERNEL | __GFP_ZERO);
-	if (!*str)
-		return -ENOMEM;
-
-	return 0;
-}
-
 int cfs_trace_dump_debug_buffer_usrstr(void __user *usr_str, int usr_str_nob)
 {
 	char *str;
 	int rc;
 
-	rc = cfs_trace_allocate_string_buffer(&str, usr_str_nob + 1);
-	if (rc)
-		return rc;
+	if (usr_str_nob >= 2 * PAGE_SIZE)
+		return -EINVAL;
+	str = kzalloc(usr_str_nob + 1, GFP_KERNEL);
+	if (!str)
+		return -ENOMEM;
 
 	rc = cfs_trace_copyin_string(str, usr_str_nob + 1,
 				     usr_str, usr_str_nob);
@@ -1044,9 +1034,11 @@ int cfs_trace_daemon_command_usrstr(void __user *usr_str, int usr_str_nob)
 	char *str;
 	int rc;
 
-	rc = cfs_trace_allocate_string_buffer(&str, usr_str_nob + 1);
-	if (rc)
-		return rc;
+	if (usr_str_nob >= 2 * PAGE_SIZE)
+		return -EINVAL;
+	str = kzalloc(usr_str_nob + 1, GFP_KERNEL);
+	if (!str)
+		return -ENOMEM;
 
 	rc = cfs_trace_copyin_string(str, usr_str_nob + 1,
 				     usr_str, usr_str_nob);
diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.h b/drivers/staging/lustre/lnet/libcfs/tracefile.h
index 82f090fd8dfa..2134549bb3d7 100644
--- a/drivers/staging/lustre/lnet/libcfs/tracefile.h
+++ b/drivers/staging/lustre/lnet/libcfs/tracefile.h
@@ -63,7 +63,6 @@ int cfs_trace_copyin_string(char *knl_buffer, int knl_buffer_nob,
 			    const char __user *usr_buffer, int usr_buffer_nob);
 int cfs_trace_copyout_string(char __user *usr_buffer, int usr_buffer_nob,
 			     const char *knl_str, char *append);
-int cfs_trace_allocate_string_buffer(char **str, int nob);
 int cfs_trace_dump_debug_buffer_usrstr(void __user *usr_str, int usr_str_nob);
 int cfs_trace_daemon_command(char *str);
 int cfs_trace_daemon_command_usrstr(void __user *usr_str, int usr_str_nob);



From neilb at suse.com  Mon Jul 30 03:49:33 2018
From: neilb at suse.com (NeilBrown)
Date: Mon, 30 Jul 2018 13:49:33 +1000
Subject: [lustre-devel] [PATCH 6/7] lustre: lnet: convert ni_refs to
 percpu_refcount.
In-Reply-To: <153292249340.26496.13985174182824912332.stgit@noble>
References: <153292249340.26496.13985174182824912332.stgit@noble>
Message-ID: <153292257300.26496.12952743323721291033.stgit@noble>

ni_refs is a per-cpt refcount.
Linux already has a per-cpu refcount implementation
which doesn't require anylocking.

So convert ni_refs to percpu_refcount.
As a bonus, we can get a wake-up when the refcount
reaches zero, rather than having to wait a full second.
The waiting in lnet_clear_zombies_nis_locked() is
modified so that instead of waiting one second each
time, and printing a warning on power-of-two seconds,
we wait an increasing power-of-two seconds and print
a warning if the wait ever timed out.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 .../staging/lustre/include/linux/lnet/lib-lnet.h   |   11 +-----
 .../staging/lustre/include/linux/lnet/lib-types.h  |    2 +
 drivers/staging/lustre/lnet/lnet/api-ni.c          |   34 +++++++++-----------
 drivers/staging/lustre/lnet/lnet/config.c          |   13 +++++---
 drivers/staging/lustre/lnet/lnet/router_proc.c     |    2 +
 5 files changed, 28 insertions(+), 34 deletions(-)

diff --git a/drivers/staging/lustre/include/linux/lnet/lib-lnet.h b/drivers/staging/lustre/include/linux/lnet/lib-lnet.h
index 0fecf0d32c58..371002825a7d 100644
--- a/drivers/staging/lustre/include/linux/lnet/lib-lnet.h
+++ b/drivers/staging/lustre/include/linux/lnet/lib-lnet.h
@@ -338,34 +338,27 @@ static inline void
 lnet_ni_addref_locked(struct lnet_ni *ni, int cpt)
 {
 	LASSERT(cpt >= 0 && cpt < LNET_CPT_NUMBER);
-	LASSERT(*ni->ni_refs[cpt] >= 0);
-
-	(*ni->ni_refs[cpt])++;
+	percpu_ref_get(&ni->ni_refs);
 }
 
 static inline void
 lnet_ni_addref(struct lnet_ni *ni)
 {
-	lnet_net_lock(0);
 	lnet_ni_addref_locked(ni, 0);
-	lnet_net_unlock(0);
 }
 
 static inline void
 lnet_ni_decref_locked(struct lnet_ni *ni, int cpt)
 {
 	LASSERT(cpt >= 0 && cpt < LNET_CPT_NUMBER);
-	LASSERT(*ni->ni_refs[cpt] > 0);
 
-	(*ni->ni_refs[cpt])--;
+	percpu_ref_put(&ni->ni_refs);
 }
 
 static inline void
 lnet_ni_decref(struct lnet_ni *ni)
 {
-	lnet_net_lock(0);
 	lnet_ni_decref_locked(ni, 0);
-	lnet_net_unlock(0);
 }
 
 void lnet_ni_free(struct lnet_ni *ni);
diff --git a/drivers/staging/lustre/include/linux/lnet/lib-types.h b/drivers/staging/lustre/include/linux/lnet/lib-types.h
index 6d4106fd9039..7527fef90cac 100644
--- a/drivers/staging/lustre/include/linux/lnet/lib-types.h
+++ b/drivers/staging/lustre/include/linux/lnet/lib-types.h
@@ -269,7 +269,7 @@ struct lnet_ni {
 	void			 *ni_data;	/* instance-specific data */
 	struct lnet_lnd		 *ni_lnd;	/* procedural interface */
 	struct lnet_tx_queue	**ni_tx_queues;	/* percpt TX queues */
-	int			**ni_refs;	/* percpt reference count */
+	struct percpu_ref	  ni_refs;
 	time64_t		  ni_last_alive;/* when I was last alive */
 	struct lnet_ni_status	 *ni_status;	/* my health status */
 	/* per NI LND tunables */
diff --git a/drivers/staging/lustre/lnet/lnet/api-ni.c b/drivers/staging/lustre/lnet/lnet/api-ni.c
index cdbbe9cc8d95..fea03737439a 100644
--- a/drivers/staging/lustre/lnet/lnet/api-ni.c
+++ b/drivers/staging/lustre/lnet/lnet/api-ni.c
@@ -1055,7 +1055,7 @@ lnet_ni_unlink_locked(struct lnet_ni *ni)
 	/* move it to zombie list and nobody can find it anymore */
 	LASSERT(!list_empty(&ni->ni_list));
 	list_move(&ni->ni_list, &the_lnet.ln_nis_zombie);
-	lnet_ni_decref_locked(ni, 0);	/* drop ln_nis' ref */
+	percpu_ref_kill_and_confirm(&ni->ni_refs, NULL);	/* drop ln_nis' ref */
 }
 
 static void
@@ -1069,34 +1069,32 @@ lnet_clear_zombies_nis_locked(void)
 	 * Now wait for the NI's I just nuked to show up on ln_zombie_nis
 	 * and shut them down in guaranteed thread context
 	 */
-	i = 2;
+	i = 1;
 	while (!list_empty(&the_lnet.ln_nis_zombie)) {
-		int *ref;
-		int j;
 
 		ni = list_entry(the_lnet.ln_nis_zombie.next,
 				struct lnet_ni, ni_list);
-		list_del_init(&ni->ni_list);
-		cfs_percpt_for_each(ref, j, ni->ni_refs) {
-			if (!*ref)
-				continue;
-			/* still busy, add it back to zombie list */
-			list_add(&ni->ni_list, &the_lnet.ln_nis_zombie);
-			break;
-		}
 
-		if (!list_empty(&ni->ni_list)) {
+		if (!percpu_ref_is_zero(&ni->ni_refs)) {
+			/* still busy, wait a while */
+
 			lnet_net_unlock(LNET_LOCK_EX);
 			++i;
-			if ((i & (-i)) == i) {
+
+			if (wait_var_event_timeout(
+				    &ni->ni_refs,
+				    percpu_ref_is_zero(&ni->ni_refs),
+				    HZ << i) == 0)
 				CDEBUG(D_WARNING, "Waiting for zombie LNI %s\n",
 				       libcfs_nid2str(ni->ni_nid));
-			}
+
 			schedule_timeout_uninterruptible(HZ);
 			lnet_net_lock(LNET_LOCK_EX);
 			continue;
 		}
 
+		list_del_init(&ni->ni_list);
+
 		ni->ni_lnd->lnd_refcount--;
 		lnet_net_unlock(LNET_LOCK_EX);
 
@@ -1114,7 +1112,7 @@ lnet_clear_zombies_nis_locked(void)
 			       libcfs_nid2str(ni->ni_nid));
 
 		lnet_ni_free(ni);
-		i = 2;
+		i = 1;
 
 		lnet_net_lock(LNET_LOCK_EX);
 	}
@@ -1305,8 +1303,8 @@ lnet_startup_lndni(struct lnet_ni *ni, struct lnet_ioctl_config_data *conf)
 	LASSERT(ni->ni_peertimeout <= 0 || lnd->lnd_query);
 
 	lnet_net_lock(LNET_LOCK_EX);
-	/* refcount for ln_nis */
-	lnet_ni_addref_locked(ni, 0);
+	/* Initialise refcount for ln_nis to 1 */
+	percpu_ref_reinit(&ni->ni_refs);
 	list_add_tail(&ni->ni_list, &the_lnet.ln_nis);
 	if (ni->ni_cpts) {
 		lnet_ni_addref_locked(ni, 0);
diff --git a/drivers/staging/lustre/lnet/lnet/config.c b/drivers/staging/lustre/lnet/lnet/config.c
index 091c4f714e84..4145c7431576 100644
--- a/drivers/staging/lustre/lnet/lnet/config.c
+++ b/drivers/staging/lustre/lnet/lnet/config.c
@@ -96,8 +96,7 @@ lnet_ni_free(struct lnet_ni *ni)
 {
 	int i;
 
-	if (ni->ni_refs)
-		cfs_percpt_free(ni->ni_refs);
+	percpu_ref_exit(&ni->ni_refs);
 
 	if (ni->ni_tx_queues)
 		cfs_percpt_free(ni->ni_tx_queues);
@@ -117,6 +116,11 @@ lnet_ni_free(struct lnet_ni *ni)
 	kfree(ni);
 }
 
+static void ref_release(struct percpu_ref *ref)
+{
+	wake_up_var(ref);
+}
+
 struct lnet_ni *
 lnet_ni_alloc(__u32 net, struct cfs_expr_list *el, struct list_head *nilist)
 {
@@ -140,9 +144,8 @@ lnet_ni_alloc(__u32 net, struct cfs_expr_list *el, struct list_head *nilist)
 
 	spin_lock_init(&ni->ni_lock);
 	INIT_LIST_HEAD(&ni->ni_cptlist);
-	ni->ni_refs = cfs_percpt_alloc(lnet_cpt_table(),
-				       sizeof(*ni->ni_refs[0]));
-	if (!ni->ni_refs)
+	if (percpu_ref_init(&ni->ni_refs, ref_release,
+			    PERCPU_REF_INIT_DEAD, GFP_KERNEL) < 0)
 		goto failed;
 
 	ni->ni_tx_queues = cfs_percpt_alloc(lnet_cpt_table(),
diff --git a/drivers/staging/lustre/lnet/lnet/router_proc.c b/drivers/staging/lustre/lnet/lnet/router_proc.c
index d779445fefb5..8856798d263f 100644
--- a/drivers/staging/lustre/lnet/lnet/router_proc.c
+++ b/drivers/staging/lustre/lnet/lnet/router_proc.c
@@ -703,7 +703,7 @@ static int proc_lnet_nis(struct ctl_table *table, int write,
 				s += snprintf(s, tmpstr + tmpsiz - s,
 					      "%-24s %6s %5lld %4d %4d %4d %5d %5d %5d\n",
 					      libcfs_nid2str(ni->ni_nid), stat,
-					      last_alive, *ni->ni_refs[i],
+					      last_alive, 0/* No per-cpt refcount */,
 					      ni->ni_peertxcredits,
 					      ni->ni_peerrtrcredits,
 					      tq->tq_credits_max,



From neilb at suse.com  Mon Jul 30 03:49:33 2018
From: neilb at suse.com (NeilBrown)
Date: Mon, 30 Jul 2018 13:49:33 +1000
Subject: [lustre-devel] [PATCH 7/7] lustre: change TASK_NOLOAD to TASK_IDLE.
In-Reply-To: <153292249340.26496.13985174182824912332.stgit@noble>
References: <153292249340.26496.13985174182824912332.stgit@noble>
Message-ID: <153292257303.26496.13996008210683917463.stgit@noble>

TASK_NOLOAD is not a task state to be use by
itself, it should only be used together with
TASK_UNINTERRUPTIBLE, which easily done
by using TASK_IDLE.

So convert to TASK_IDLE.

Signed-off-by: NeilBrown <neilb at suse.com>
---
 drivers/staging/lustre/lnet/lnet/lib-eq.c |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/staging/lustre/lnet/lnet/lib-eq.c b/drivers/staging/lustre/lnet/lnet/lib-eq.c
index 8347cc44e47d..f085388895ea 100644
--- a/drivers/staging/lustre/lnet/lnet/lib-eq.c
+++ b/drivers/staging/lustre/lnet/lnet/lib-eq.c
@@ -349,7 +349,7 @@ __must_hold(&the_lnet.ln_eq_wait_lock)
  * \param timeout Time in jiffies to wait for an event to occur on
  * one of the EQs. The constant MAX_SCHEDULE_TIMEOUT can be used to indicate an
  * infinite timeout.
- * \param interruptible, if true, use TASK_INTERRUPTIBLE, else TASK_NOLOAD
+ * \param interruptible, if true, use TASK_INTERRUPTIBLE, else TASK_IDLE
  * \param event,which On successful return (1 or -EOVERFLOW), \a event will
  * hold the next event in the EQs, and \a which will contain the index of the
  * EQ from which the event was taken.
@@ -406,7 +406,7 @@ LNetEQPoll(struct lnet_handle_eq *eventqs, int neq, signed long timeout,
 		 */
 		wait = lnet_eq_wait_locked(&timeout,
 					   interruptible ? TASK_INTERRUPTIBLE
-					   : TASK_NOLOAD);
+					   : TASK_IDLE);
 		if (wait < 0) /* no new event */
 			break;
 	}



From adilger at whamcloud.com  Mon Jul 30 20:02:08 2018
From: adilger at whamcloud.com (Andreas Dilger)
Date: Mon, 30 Jul 2018 20:02:08 +0000
Subject: [lustre-devel] [PATCH 00/22] Lustre: use while loop when
 emptying a list
In-Reply-To: <153292153459.13840.17465048403476297915.stgit@noble>
References: <153292153459.13840.17465048403476297915.stgit@noble>
Message-ID: <F1535323-C137-4BF7-B1FA-D0858FFA619B@whamcloud.com>

On Jul 29, 2018, at 21:37, NeilBrown <neilb at suse.com> wrote:
> 
> If you have a list that needs to be emptied, it is best to have a loop
> like
>  while (!list_empty(...))
> 
> because then it is obvious what the purpose of the loop is.
> Many places in lustre use
>  list_for_each_entry_safe()
> instead, which obscures the purpose.
> Several of these were from patches which deliberately converted from
> the while loop the list list_for_each_entry_safe() loop, at least some
> of which introduced real bugs.
> 
> This series reverts all those patches, and then makes other
> conversions.
> There are still several places which could be converted, but I got
> bored...
> I've particularly converted all where the list_head is a local
> variable. In these cases it is obviously wrong not to empty the
> list completely.
> 
> For those conversions that I did manual (not reverts) I use
>  list_first_entry()
> to get the first entry, rather then list_entry(head->next).
> Others could be converted as well.
> 
> NeilBrown

Reviewed-by: Andreas Dilger <adilger at whamcloud.com>

> ---
> 
> NeilBrown (22):
>      Revert "staging: lustre: lnet: api-ni: Use list_for_each_entry_safe"
>      Revert "staging: lustre: o2iblnd: Use list_for_each_entry_safe in kiblnd_destroy_fmr_pool_list"
>      Revert "staging: lustre: lnet: o2iblnd: Use list_for_each_entry_safe"
>      Revert "staging: lustre: lnet: socklnd: Use list_for_each_entry_safe"
>      Revert "staging: lustre: lnet: socklnd_proto: Use list_for_each_entry_safe"
>      Revert "staging: lustre: osc_cache: Use list_for_each_entry_safe"
>      Revert "staging: lustre: lnet: peer: Use list_for_each_entry_safe"
>      Revert "staging: lustre: lnet: config: Use list_for_each_entry_safe"
>      Revert "staging: lustre: lnet: router: Use list_for_each_entry_safe"
>      Revert "staging: lustre: lnet: conrpc: Use list_for_each_entry_safe"
>      Revert "staging: lustre: lnet: lib-move: Use list_for_each_entry_safe"
>      Revert "staging: lustre: obdclass: Use list_for_each_entry_safe"
>      Revert "staging: lustre: lnet: Use list_for_each_entry_safe"
>      Revert: Staging: lustre: Iterate list using list_for_each_entry
>      lustre: o2iblnd: convert list_for_each_entry_safe() to while(!list_empty())
>      lustre: tracefile: convert list_for_each_entry_safe() to while(!list_empty())
>      lustre: lib-move:  convert list_for_each_entry_safe() to while(!list_empty())
>      lustre: net_fault: convert list_for_each_entry_safe() to while(!list_empty())
>      lustre: fld_request: convert list_for_each_entry_safe() to while(!list_empty())
>      lustre: lov_obd: convert list_for_each_entry_safe() to while(!list_empty())
>      lustre: ldlm_request: convert list_for_each_entry_safe() to while(!list_empty())
>      lustre: sec_config: convert list_for_each_entry_safe() to while(!list_empty())
> 
> 
> .../staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c    |   21 ++++++++++++--------
> .../staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c |   12 ++++++-----
> .../staging/lustre/lnet/klnds/socklnd/socklnd.c    |    9 +++++----
> .../staging/lustre/lnet/klnds/socklnd/socklnd_cb.c |    5 +++--
> .../lustre/lnet/klnds/socklnd/socklnd_proto.c      |    4 ++--
> drivers/staging/lustre/lnet/libcfs/tracefile.c     |   12 +++++++----
> drivers/staging/lustre/lnet/lnet/api-ni.c          |   10 ++++++----
> drivers/staging/lustre/lnet/lnet/config.c          |    5 +++--
> drivers/staging/lustre/lnet/lnet/lib-move.c        |   13 +++++++-----
> drivers/staging/lustre/lnet/lnet/net_fault.c       |    7 +++++--
> drivers/staging/lustre/lnet/lnet/peer.c            |    4 ++--
> drivers/staging/lustre/lnet/lnet/router.c          |    4 ++--
> drivers/staging/lustre/lnet/selftest/conrpc.c      |    5 +++--
> drivers/staging/lustre/lustre/fld/fld_request.c    |    6 ++++--
> drivers/staging/lustre/lustre/ldlm/ldlm_request.c  |    6 ++++--
> drivers/staging/lustre/lustre/lov/lov_obd.c        |    5 +++--
> .../staging/lustre/lustre/obdclass/lustre_peer.c   |    5 +++--
> drivers/staging/lustre/lustre/osc/osc_cache.c      |    9 +++++----
> drivers/staging/lustre/lustre/ptlrpc/sec_config.c  |    7 ++++---
> 19 files changed, 87 insertions(+), 62 deletions(-)
> 
> --
> Signature
> 

Cheers, Andreas
---
Andreas Dilger
CTO Whamcloud




-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 235 bytes
Desc: Message signed with OpenPGP
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180730/d82a84f4/attachment.sig>

From adilger at whamcloud.com  Mon Jul 30 20:42:05 2018
From: adilger at whamcloud.com (Andreas Dilger)
Date: Mon, 30 Jul 2018 20:42:05 +0000
Subject: [lustre-devel] [PATCH 2/6] lustre: lnet/config: convert
 list_for_each to list_for_each_entry
In-Reply-To: <153292233958.26104.10972399803477314475.stgit@noble>
References: <153292233170.26104.16164388413209501300.stgit@noble>
 <153292233958.26104.10972399803477314475.stgit@noble>
Message-ID: <C85DBB0B-C6EC-4D59-B0EA-5A8A94C96051@whamcloud.com>



> On Jul 29, 2018, at 21:45, NeilBrown <neilb at suse.com> wrote:
> 
> This conversion to list_for_each_entry isn't quite trivial
> as the 'tmp' loop variables are used elsewhere in the function.
> This means we cannot just delete them, and so must inspect the
> code to ensure the values aren't used elsewhere - they aren't.
> 
> Also we need to introduce new loop variables: ltb1 and ltb2 as
> the lnet_text_buf was the some for both the original (nested) loops.
> 
> Signed-off-by: NeilBrown <neilb at suse.com>

Reviewed-by: Andreas Dilger <adilger at whamcloud.com>

> ---
> drivers/staging/lustre/lnet/lnet/config.c |   11 +++++------
> 1 file changed, 5 insertions(+), 6 deletions(-)
> 
> diff --git a/drivers/staging/lustre/lnet/lnet/config.c b/drivers/staging/lustre/lnet/lnet/config.c
> index 96336ecdacaf..091c4f714e84 100644
> --- a/drivers/staging/lustre/lnet/lnet/config.c
> +++ b/drivers/staging/lustre/lnet/lnet/config.c
> @@ -662,6 +662,7 @@ lnet_parse_route(char *str, int *im_a_router)
> 	__u32 net;
> 	lnet_nid_t nid;
> 	struct lnet_text_buf *ltb;
> +	struct lnet_text_buf *ltb1, *ltb2;
> 	int rc;
> 	char *sep;
> 	char *token = str;
> @@ -760,14 +761,12 @@ lnet_parse_route(char *str, int *im_a_router)
> 	LASSERT(!list_empty(&nets));
> 	LASSERT(!list_empty(&gateways));
> 
> -	list_for_each(tmp1, &nets) {
> -		ltb = list_entry(tmp1, struct lnet_text_buf, ltb_list);
> -		net = libcfs_str2net(ltb->ltb_text);
> +	list_for_each_entry(ltb1, &nets, ltb_list) {
> +		net = libcfs_str2net(ltb1->ltb_text);
> 		LASSERT(net != LNET_NIDNET(LNET_NID_ANY));
> 
> -		list_for_each(tmp2, &gateways) {
> -			ltb = list_entry(tmp2, struct lnet_text_buf, ltb_list);
> -			nid = libcfs_str2nid(ltb->ltb_text);
> +		list_for_each_entry(ltb2, &gateways, ltb_list) {
> +			nid = libcfs_str2nid(ltb2->ltb_text);
> 			LASSERT(nid != LNET_NID_ANY);
> 
> 			if (lnet_islocalnid(nid)) {
> 
> 

Cheers, Andreas
---
Andreas Dilger
Principal Lustre Architect
Whamcloud







-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 235 bytes
Desc: Message signed with OpenPGP
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180730/db409863/attachment.sig>

From adilger at whamcloud.com  Mon Jul 30 20:49:58 2018
From: adilger at whamcloud.com (Andreas Dilger)
Date: Mon, 30 Jul 2018 20:49:58 +0000
Subject: [lustre-devel] [PATCH 1/6] lustre: convert list_for_each() to
 list_for_each_entry().
In-Reply-To: <153292233953.26104.11218548856426762220.stgit@noble>
References: <153292233170.26104.16164388413209501300.stgit@noble>
 <153292233953.26104.11218548856426762220.stgit@noble>
Message-ID: <072E1B45-CD98-4486-A90C-865CE0EB824D@whamcloud.com>

On Jul 29, 2018, at 21:45, NeilBrown <neilb at suse.com> wrote:
> 
> Where we have
>  struct list_head *tmp;
>  list_for_each(tmp,....) {
>     foo = list_entry(tmp, ....);
>     ...
>  }
> 
> convert to
>  list_for_each_entry(foo, .....) {
>  .....
>  }
> 
> This patch only changes instances which are straight forward, where
> neither the 'tmp' variable nor the 'foo' variable is not used outside
> the loop in any way that isn't trivially correct.
> 
> Signed-off-by: NeilBrown <neilb at suse.com>

Reviewed-by: Andreas Dilger <adilger at whamcloud.com>

> ---
> .../staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c    |   17 ++-----
> .../staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c |   20 ++------
> .../staging/lustre/lnet/klnds/socklnd/socklnd.c    |   51 +++++---------------
> .../staging/lustre/lnet/klnds/socklnd/socklnd_cb.c |   20 ++------
> drivers/staging/lustre/lnet/lnet/api-ni.c          |   29 ++---------
> drivers/staging/lustre/lnet/lnet/config.c          |   17 ++-----
> drivers/staging/lustre/lnet/lnet/lib-move.c        |    9 +---
> drivers/staging/lustre/lnet/lnet/router.c          |   47 ++++--------------
> drivers/staging/lustre/lnet/selftest/conrpc.c      |    7 +--
> drivers/staging/lustre/lustre/obdclass/genops.c    |    4 --
> .../staging/lustre/lustre/obdecho/echo_client.c    |    4 --
> 11 files changed, 53 insertions(+), 172 deletions(-)
> 
> diff --git a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c
> index 830a5bf34c16..e15ad94151bd 100644
> --- a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c
> +++ b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c
> @@ -386,11 +386,9 @@ struct kib_peer *kiblnd_find_peer_locked(lnet_nid_t nid)
> 	 * that this creates
> 	 */
> 	struct list_head *peer_list = kiblnd_nid2peerlist(nid);
> -	struct list_head *tmp;
> 	struct kib_peer *peer;
> 
> -	list_for_each(tmp, peer_list) {
> -		peer = list_entry(tmp, struct kib_peer, ibp_list);
> +	list_for_each_entry(peer, peer_list, ibp_list) {
> 		LASSERT(!kiblnd_peer_idle(peer));
> 
> 		if (peer->ibp_nid != nid)
> @@ -419,15 +417,13 @@ static int kiblnd_get_peer_info(struct lnet_ni *ni, int index,
> 				lnet_nid_t *nidp, int *count)
> {
> 	struct kib_peer *peer;
> -	struct list_head *ptmp;
> 	int i;
> 	unsigned long flags;
> 
> 	read_lock_irqsave(&kiblnd_data.kib_global_lock, flags);
> 
> 	for (i = 0; i < kiblnd_data.kib_peer_hash_size; i++) {
> -		list_for_each(ptmp, &kiblnd_data.kib_peers[i]) {
> -			peer = list_entry(ptmp, struct kib_peer, ibp_list);
> +		list_for_each_entry(peer, &kiblnd_data.kib_peers[i], ibp_list) {
> 			LASSERT(!kiblnd_peer_idle(peer));
> 
> 			if (peer->ibp_ni != ni)
> @@ -526,28 +522,23 @@ static int kiblnd_del_peer(struct lnet_ni *ni, lnet_nid_t nid)
> static struct kib_conn *kiblnd_get_conn_by_idx(struct lnet_ni *ni, int index)
> {
> 	struct kib_peer *peer;
> -	struct list_head *ptmp;
> 	struct kib_conn *conn;
> -	struct list_head *ctmp;
> 	int i;
> 	unsigned long flags;
> 
> 	read_lock_irqsave(&kiblnd_data.kib_global_lock, flags);
> 
> 	for (i = 0; i < kiblnd_data.kib_peer_hash_size; i++) {
> -		list_for_each(ptmp, &kiblnd_data.kib_peers[i]) {
> -			peer = list_entry(ptmp, struct kib_peer, ibp_list);
> +		list_for_each_entry(peer, &kiblnd_data.kib_peers[i], ibp_list) {
> 			LASSERT(!kiblnd_peer_idle(peer));
> 
> 			if (peer->ibp_ni != ni)
> 				continue;
> 
> -			list_for_each(ctmp, &peer->ibp_conns) {
> +			list_for_each_entry(conn, &peer->ibp_conns, ibc_list) {
> 				if (index-- > 0)
> 					continue;
> 
> -				conn = list_entry(ctmp, struct kib_conn,
> -						  ibc_list);
> 				kiblnd_conn_addref(conn);
> 				read_unlock_irqrestore(
> 					&kiblnd_data.kib_global_lock,
> diff --git a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c
> index bda67d49597d..2f7a64f2f13a 100644
> --- a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c
> +++ b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c
> @@ -225,11 +225,9 @@ kiblnd_post_rx(struct kib_rx *rx, int credit)
> static struct kib_tx *
> kiblnd_find_waiting_tx_locked(struct kib_conn *conn, int txtype, __u64 cookie)
> {
> -	struct list_head *tmp;
> -
> -	list_for_each(tmp, &conn->ibc_active_txs) {
> -		struct kib_tx *tx = list_entry(tmp, struct kib_tx, tx_list);
> +	struct kib_tx *tx;
> 
> +	list_for_each_entry(tx, &conn->ibc_active_txs, tx_list) {
> 		LASSERT(!tx->tx_queued);
> 		LASSERT(tx->tx_sending || tx->tx_waiting);
> 
> @@ -3142,11 +3140,8 @@ static int
> kiblnd_check_txs_locked(struct kib_conn *conn, struct list_head *txs)
> {
> 	struct kib_tx *tx;
> -	struct list_head *ttmp;
> -
> -	list_for_each(ttmp, txs) {
> -		tx = list_entry(ttmp, struct kib_tx, tx_list);
> 
> +	list_for_each_entry(tx, txs, tx_list) {
> 		if (txs != &conn->ibc_active_txs) {
> 			LASSERT(tx->tx_queued);
> 		} else {
> @@ -3182,10 +3177,8 @@ kiblnd_check_conns(int idx)
> 	LIST_HEAD(closes);
> 	LIST_HEAD(checksends);
> 	struct list_head *peers = &kiblnd_data.kib_peers[idx];
> -	struct list_head *ptmp;
> 	struct kib_peer *peer;
> 	struct kib_conn *conn;
> -	struct list_head *ctmp;
> 	unsigned long flags;
> 
> 	/*
> @@ -3195,15 +3188,12 @@ kiblnd_check_conns(int idx)
> 	 */
> 	read_lock_irqsave(&kiblnd_data.kib_global_lock, flags);
> 
> -	list_for_each(ptmp, peers) {
> -		peer = list_entry(ptmp, struct kib_peer, ibp_list);
> +	list_for_each_entry(peer, peers, ibp_list) {
> 
> -		list_for_each(ctmp, &peer->ibp_conns) {
> +		list_for_each_entry(conn, &peer->ibp_conns, ibc_list) {
> 			int timedout;
> 			int sendnoop;
> 
> -			conn = list_entry(ctmp, struct kib_conn, ibc_list);
> -
> 			LASSERT(conn->ibc_state == IBLND_CONN_ESTABLISHED);
> 
> 			spin_lock(&conn->ibc_lock);
> diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c
> index a48b1b9a850b..491d2ed5b673 100644
> --- a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c
> +++ b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c
> @@ -250,9 +250,7 @@ ksocknal_get_peer_info(struct lnet_ni *ni, int index,
> 		       int *port, int *conn_count, int *share_count)
> {
> 	struct ksock_peer *peer;
> -	struct list_head *ptmp;
> 	struct ksock_route *route;
> -	struct list_head *rtmp;
> 	int i;
> 	int j;
> 	int rc = -ENOENT;
> @@ -260,8 +258,7 @@ ksocknal_get_peer_info(struct lnet_ni *ni, int index,
> 	read_lock(&ksocknal_data.ksnd_global_lock);
> 
> 	for (i = 0; i < ksocknal_data.ksnd_peer_hash_size; i++) {
> -		list_for_each(ptmp, &ksocknal_data.ksnd_peers[i]) {
> -			peer = list_entry(ptmp, struct ksock_peer, ksnp_list);
> +		list_for_each_entry(peer, &ksocknal_data.ksnd_peers[i], ksnp_list) {
> 
> 			if (peer->ksnp_ni != ni)
> 				continue;
> @@ -295,13 +292,11 @@ ksocknal_get_peer_info(struct lnet_ni *ni, int index,
> 				goto out;
> 			}
> 
> -			list_for_each(rtmp, &peer->ksnp_routes) {
> +			list_for_each_entry(route, &peer->ksnp_routes,
> +					    ksnr_list) {
> 				if (index-- > 0)
> 					continue;
> 
> -				route = list_entry(rtmp, struct ksock_route,
> -						   ksnr_list);
> -
> 				*id = peer->ksnp_id;
> 				*myip = route->ksnr_myipaddr;
> 				*peer_ip = route->ksnr_ipaddr;
> @@ -368,7 +363,6 @@ ksocknal_associate_route_conn_locked(struct ksock_route *route,
> static void
> ksocknal_add_route_locked(struct ksock_peer *peer, struct ksock_route *route)
> {
> -	struct list_head *tmp;
> 	struct ksock_conn *conn;
> 	struct ksock_route *route2;
> 
> @@ -379,9 +373,7 @@ ksocknal_add_route_locked(struct ksock_peer *peer, struct ksock_route *route)
> 	LASSERT(!route->ksnr_connected);
> 
> 	/* LASSERT(unique) */
> -	list_for_each(tmp, &peer->ksnp_routes) {
> -		route2 = list_entry(tmp, struct ksock_route, ksnr_list);
> -
> +	list_for_each_entry(route2, &peer->ksnp_routes, ksnr_list) {
> 		if (route2->ksnr_ipaddr == route->ksnr_ipaddr) {
> 			CERROR("Duplicate route %s %pI4h\n",
> 			       libcfs_id2str(peer->ksnp_id),
> @@ -395,9 +387,7 @@ ksocknal_add_route_locked(struct ksock_peer *peer, struct ksock_route *route)
> 	/* peer's routelist takes over my ref on 'route' */
> 	list_add_tail(&route->ksnr_list, &peer->ksnp_routes);
> 
> -	list_for_each(tmp, &peer->ksnp_conns) {
> -		conn = list_entry(tmp, struct ksock_conn, ksnc_list);
> -
> +	list_for_each_entry(conn, &peer->ksnp_conns, ksnc_list) {
> 		if (conn->ksnc_ipaddr != route->ksnr_ipaddr)
> 			continue;
> 
> @@ -624,28 +614,22 @@ static struct ksock_conn *
> ksocknal_get_conn_by_idx(struct lnet_ni *ni, int index)
> {
> 	struct ksock_peer *peer;
> -	struct list_head *ptmp;
> 	struct ksock_conn *conn;
> -	struct list_head *ctmp;
> 	int i;
> 
> 	read_lock(&ksocknal_data.ksnd_global_lock);
> 
> 	for (i = 0; i < ksocknal_data.ksnd_peer_hash_size; i++) {
> -		list_for_each(ptmp, &ksocknal_data.ksnd_peers[i]) {
> -			peer = list_entry(ptmp, struct ksock_peer, ksnp_list);
> -
> +		list_for_each_entry(peer, &ksocknal_data.ksnd_peers[i], ksnp_list) {
> 			LASSERT(!peer->ksnp_closing);
> 
> 			if (peer->ksnp_ni != ni)
> 				continue;
> 
> -			list_for_each(ctmp, &peer->ksnp_conns) {
> +			list_for_each_entry(conn, &peer->ksnp_conns, ksnc_list) {
> 				if (index-- > 0)
> 					continue;
> 
> -				conn = list_entry(ctmp, struct ksock_conn,
> -						  ksnc_list);
> 				ksocknal_conn_addref(conn);
> 				read_unlock(&ksocknal_data.ksnd_global_lock);
> 				return conn;
> @@ -1025,7 +1009,6 @@ ksocknal_create_conn(struct lnet_ni *ni, struct ksock_route *route,
> 	rwlock_t *global_lock = &ksocknal_data.ksnd_global_lock;
> 	LIST_HEAD(zombies);
> 	struct lnet_process_id peerid;
> -	struct list_head *tmp;
> 	__u64 incarnation;
> 	struct ksock_conn *conn;
> 	struct ksock_conn *conn2;
> @@ -1226,8 +1209,7 @@ ksocknal_create_conn(struct lnet_ni *ni, struct ksock_route *route,
> 	 * loopback connection
> 	 */
> 	if (conn->ksnc_ipaddr != conn->ksnc_myipaddr) {
> -		list_for_each(tmp, &peer->ksnp_conns) {
> -			conn2 = list_entry(tmp, struct ksock_conn, ksnc_list);
> +		list_for_each_entry(conn2, &peer->ksnp_conns, ksnc_list) {
> 
> 			if (conn2->ksnc_ipaddr != conn->ksnc_ipaddr ||
> 			    conn2->ksnc_myipaddr != conn->ksnc_myipaddr ||
> @@ -1266,9 +1248,7 @@ ksocknal_create_conn(struct lnet_ni *ni, struct ksock_route *route,
> 	 * by routes in my peer to match my own route entries so I don't
> 	 * continually create duplicate routes.
> 	 */
> -	list_for_each(tmp, &peer->ksnp_routes) {
> -		route = list_entry(tmp, struct ksock_route, ksnr_list);
> -
> +	list_for_each_entry(route, &peer->ksnp_routes, ksnr_list) {
> 		if (route->ksnr_ipaddr != conn->ksnc_ipaddr)
> 			continue;
> 
> @@ -1980,9 +1960,7 @@ ksocknal_add_interface(struct lnet_ni *ni, __u32 ipaddress, __u32 netmask)
> 	int rc;
> 	int i;
> 	int j;
> -	struct list_head *ptmp;
> 	struct ksock_peer *peer;
> -	struct list_head *rtmp;
> 	struct ksock_route *route;
> 
> 	if (!ipaddress || !netmask)
> @@ -2005,18 +1983,15 @@ ksocknal_add_interface(struct lnet_ni *ni, __u32 ipaddress, __u32 netmask)
> 		iface->ksni_npeers = 0;
> 
> 		for (i = 0; i < ksocknal_data.ksnd_peer_hash_size; i++) {
> -			list_for_each(ptmp, &ksocknal_data.ksnd_peers[i]) {
> -				peer = list_entry(ptmp, struct ksock_peer,
> -						  ksnp_list);
> +			list_for_each_entry(peer, &ksocknal_data.ksnd_peers[i],
> +					    ksnp_list) {
> 
> 				for (j = 0; j < peer->ksnp_n_passive_ips; j++)
> 					if (peer->ksnp_passive_ips[j] == ipaddress)
> 						iface->ksni_npeers++;
> 
> -				list_for_each(rtmp, &peer->ksnp_routes) {
> -					route = list_entry(rtmp, struct ksock_route,
> -							   ksnr_list);
> -
> +				list_for_each_entry(route, &peer->ksnp_routes,
> +						    ksnr_list) {
> 					if (route->ksnr_myipaddr == ipaddress)
> 						iface->ksni_nroutes++;
> 				}
> diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c
> index d531847305e4..a5c0e8a9bc40 100644
> --- a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c
> +++ b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c
> @@ -561,18 +561,16 @@ struct ksock_conn *
> ksocknal_find_conn_locked(struct ksock_peer *peer, struct ksock_tx *tx,
> 			  int nonblk)
> {
> -	struct list_head *tmp;
> +	struct ksock_conn *c;
> 	struct ksock_conn *conn;
> 	struct ksock_conn *typed = NULL;
> 	struct ksock_conn *fallback = NULL;
> 	int tnob = 0;
> 	int fnob = 0;
> 
> -	list_for_each(tmp, &peer->ksnp_conns) {
> -		struct ksock_conn *c;
> +	list_for_each_entry(c, &peer->ksnp_conns, ksnc_list) {
> 		int nob, rc;
> 
> -		c = list_entry(tmp, struct ksock_conn, ksnc_list);
> 		nob = atomic_read(&c->ksnc_tx_nob) +
> 		      c->ksnc_sock->sk->sk_wmem_queued;
> 
> @@ -729,12 +727,9 @@ struct ksock_route *
> ksocknal_find_connectable_route_locked(struct ksock_peer *peer)
> {
> 	time64_t now = ktime_get_seconds();
> -	struct list_head *tmp;
> 	struct ksock_route *route;
> 
> -	list_for_each(tmp, &peer->ksnp_routes) {
> -		route = list_entry(tmp, struct ksock_route, ksnr_list);
> -
> +	list_for_each_entry(route, &peer->ksnp_routes, ksnr_list) {
> 		LASSERT(!route->ksnr_connecting || route->ksnr_scheduled);
> 
> 		/* connections being established */
> @@ -765,11 +760,9 @@ ksocknal_find_connectable_route_locked(struct ksock_peer *peer)
> struct ksock_route *
> ksocknal_find_connecting_route_locked(struct ksock_peer *peer)
> {
> -	struct list_head *tmp;
> 	struct ksock_route *route;
> 
> -	list_for_each(tmp, &peer->ksnp_routes) {
> -		route = list_entry(tmp, struct ksock_route, ksnr_list);
> +	list_for_each_entry(route, &peer->ksnp_routes, ksnr_list) {
> 
> 		LASSERT(!route->ksnr_connecting || route->ksnr_scheduled);
> 
> @@ -2180,13 +2173,10 @@ ksocknal_find_timed_out_conn(struct ksock_peer *peer)
> {
> 	/* We're called with a shared lock on ksnd_global_lock */
> 	struct ksock_conn *conn;
> -	struct list_head *ctmp;
> 
> -	list_for_each(ctmp, &peer->ksnp_conns) {
> +	list_for_each_entry(conn, &peer->ksnp_conns, ksnc_list) {
> 		int error;
> 
> -		conn = list_entry(ctmp, struct ksock_conn, ksnc_list);
> -
> 		/* Don't need the {get,put}connsock dance to deref ksnc_sock */
> 		LASSERT(!conn->ksnc_closing);
> 
> diff --git a/drivers/staging/lustre/lnet/lnet/api-ni.c b/drivers/staging/lustre/lnet/lnet/api-ni.c
> index 51a81075f8d5..14b797802a85 100644
> --- a/drivers/staging/lustre/lnet/lnet/api-ni.c
> +++ b/drivers/staging/lustre/lnet/lnet/api-ni.c
> @@ -269,12 +269,9 @@ static struct lnet_lnd *
> lnet_find_lnd_by_type(__u32 type)
> {
> 	struct lnet_lnd *lnd;
> -	struct list_head *tmp;
> 
> 	/* holding lnd mutex */
> -	list_for_each(tmp, &the_lnet.ln_lnds) {
> -		lnd = list_entry(tmp, struct lnet_lnd, lnd_list);
> -
> +	list_for_each_entry(lnd, &the_lnet.ln_lnds, lnd_list) {
> 		if (lnd->lnd_type == type)
> 			return lnd;
> 	}
> @@ -653,14 +650,11 @@ lnet_unprepare(void)
> struct lnet_ni  *
> lnet_net2ni_locked(__u32 net, int cpt)
> {
> -	struct list_head *tmp;
> 	struct lnet_ni *ni;
> 
> 	LASSERT(cpt != LNET_LOCK_EX);
> 
> -	list_for_each(tmp, &the_lnet.ln_nis) {
> -		ni = list_entry(tmp, struct lnet_ni, ni_list);
> -
> +	list_for_each_entry(ni, &the_lnet.ln_nis, ni_list) {
> 		if (LNET_NIDNET(ni->ni_nid) == net) {
> 			lnet_ni_addref_locked(ni, cpt);
> 			return ni;
> @@ -767,13 +761,10 @@ struct lnet_ni  *
> lnet_nid2ni_locked(lnet_nid_t nid, int cpt)
> {
> 	struct lnet_ni *ni;
> -	struct list_head *tmp;
> 
> 	LASSERT(cpt != LNET_LOCK_EX);
> 
> -	list_for_each(tmp, &the_lnet.ln_nis) {
> -		ni = list_entry(tmp, struct lnet_ni, ni_list);
> -
> +	list_for_each_entry(ni, &the_lnet.ln_nis, ni_list) {
> 		if (ni->ni_nid == nid) {
> 			lnet_ni_addref_locked(ni, cpt);
> 			return ni;
> @@ -803,14 +794,11 @@ lnet_count_acceptor_nis(void)
> {
> 	/* Return the # of NIs that need the acceptor. */
> 	int count = 0;
> -	struct list_head *tmp;
> 	struct lnet_ni *ni;
> 	int cpt;
> 
> 	cpt = lnet_net_lock_current();
> -	list_for_each(tmp, &the_lnet.ln_nis) {
> -		ni = list_entry(tmp, struct lnet_ni, ni_list);
> -
> +	list_for_each_entry(ni, &the_lnet.ln_nis, ni_list) {
> 		if (ni->ni_lnd->lnd_accept)
> 			count++;
> 	}
> @@ -1731,18 +1719,16 @@ static int
> lnet_get_net_config(struct lnet_ioctl_config_data *config)
> {
> 	struct lnet_ni *ni;
> -	struct list_head *tmp;
> 	int idx = config->cfg_count;
> 	int cpt, i = 0;
> 	int rc = -ENOENT;
> 
> 	cpt = lnet_net_lock_current();
> 
> -	list_for_each(tmp, &the_lnet.ln_nis) {
> +	list_for_each_entry(ni, &the_lnet.ln_nis, ni_list) {
> 		if (i++ != idx)
> 			continue;
> 
> -		ni = list_entry(tmp, struct lnet_ni, ni_list);
> 		lnet_ni_lock(ni);
> 		lnet_fill_ni_info(ni, config);
> 		lnet_ni_unlock(ni);
> @@ -2119,7 +2105,6 @@ int
> LNetGetId(unsigned int index, struct lnet_process_id *id)
> {
> 	struct lnet_ni *ni;
> -	struct list_head *tmp;
> 	int cpt;
> 	int rc = -ENOENT;
> 
> @@ -2127,12 +2112,10 @@ LNetGetId(unsigned int index, struct lnet_process_id *id)
> 
> 	cpt = lnet_net_lock_current();
> 
> -	list_for_each(tmp, &the_lnet.ln_nis) {
> +	list_for_each_entry(ni, &the_lnet.ln_nis, ni_list) {
> 		if (index--)
> 			continue;
> 
> -		ni = list_entry(tmp, struct lnet_ni, ni_list);
> -
> 		id->nid = ni->ni_nid;
> 		id->pid = the_lnet.ln_pid;
> 		rc = 0;
> diff --git a/drivers/staging/lustre/lnet/lnet/config.c b/drivers/staging/lustre/lnet/lnet/config.c
> index 26b799e66e53..96336ecdacaf 100644
> --- a/drivers/staging/lustre/lnet/lnet/config.c
> +++ b/drivers/staging/lustre/lnet/lnet/config.c
> @@ -81,12 +81,9 @@ lnet_issep(char c)
> int
> lnet_net_unique(__u32 net, struct list_head *nilist)
> {
> -	struct list_head *tmp;
> 	struct lnet_ni *ni;
> 
> -	list_for_each(tmp, nilist) {
> -		ni = list_entry(tmp, struct lnet_ni, ni_list);
> -
> +	list_for_each_entry(ni, nilist, ni_list) {
> 		if (LNET_NIDNET(ni->ni_nid) == net)
> 			return 0;
> 	}
> @@ -942,7 +939,6 @@ lnet_splitnets(char *source, struct list_head *nets)
> 	int len;
> 	struct lnet_text_buf *tb;
> 	struct lnet_text_buf *tb2;
> -	struct list_head *t;
> 	char *sep;
> 	char *bracket;
> 	__u32 net;
> @@ -983,9 +979,7 @@ lnet_splitnets(char *source, struct list_head *nets)
> 			return -EINVAL;
> 		}
> 
> -		list_for_each(t, nets) {
> -			tb2 = list_entry(t, struct lnet_text_buf, ltb_list);
> -
> +		list_for_each_entry(tb2, nets, ltb_list) {
> 			if (tb2 == tb)
> 				continue;
> 
> @@ -1074,14 +1068,11 @@ lnet_match_networks(char **networksp, char *ip2nets, __u32 *ipaddrs, int nip)
> 			break;
> 
> 		dup = 0;
> -		list_for_each(t, &current_nets) {
> -			tb = list_entry(t, struct lnet_text_buf, ltb_list);
> +		list_for_each_entry(tb, &current_nets, ltb_list) {
> 			net1 = lnet_netspec2net(tb->ltb_text);
> 			LASSERT(net1 != LNET_NIDNET(LNET_NID_ANY));
> 
> -			list_for_each(t2, &matched_nets) {
> -				tb2 = list_entry(t2, struct lnet_text_buf,
> -						 ltb_list);
> +			list_for_each_entry(tb2, &matched_nets, ltb_list) {
> 				net2 = lnet_netspec2net(tb2->ltb_text);
> 				LASSERT(net2 != LNET_NIDNET(LNET_NID_ANY));
> 
> diff --git a/drivers/staging/lustre/lnet/lnet/lib-move.c b/drivers/staging/lustre/lnet/lnet/lib-move.c
> index 19cab374b6bc..edcafac055ed 100644
> --- a/drivers/staging/lustre/lnet/lnet/lib-move.c
> +++ b/drivers/staging/lustre/lnet/lnet/lib-move.c
> @@ -2288,7 +2288,6 @@ EXPORT_SYMBOL(LNetGet);
> int
> LNetDist(lnet_nid_t dstnid, lnet_nid_t *srcnidp, __u32 *orderp)
> {
> -	struct list_head *e;
> 	struct lnet_ni *ni;
> 	struct lnet_remotenet *rnet;
> 	__u32 dstnet = LNET_NIDNET(dstnid);
> @@ -2307,9 +2306,7 @@ LNetDist(lnet_nid_t dstnid, lnet_nid_t *srcnidp, __u32 *orderp)
> 
> 	cpt = lnet_net_lock_current();
> 
> -	list_for_each(e, &the_lnet.ln_nis) {
> -		ni = list_entry(e, struct lnet_ni, ni_list);
> -
> +	list_for_each_entry(ni, &the_lnet.ln_nis, ni_list) {
> 		if (ni->ni_nid == dstnid) {
> 			if (srcnidp)
> 				*srcnidp = dstnid;
> @@ -2346,9 +2343,7 @@ LNetDist(lnet_nid_t dstnid, lnet_nid_t *srcnidp, __u32 *orderp)
> 	}
> 
> 	rn_list = lnet_net2rnethash(dstnet);
> -	list_for_each(e, rn_list) {
> -		rnet = list_entry(e, struct lnet_remotenet, lrn_list);
> -
> +	list_for_each_entry(rnet, rn_list, lrn_list) {
> 		if (rnet->lrn_net == dstnet) {
> 			struct lnet_route *route;
> 			struct lnet_route *shortest = NULL;
> diff --git a/drivers/staging/lustre/lnet/lnet/router.c b/drivers/staging/lustre/lnet/lnet/router.c
> index 965087b7359c..53373372b526 100644
> --- a/drivers/staging/lustre/lnet/lnet/router.c
> +++ b/drivers/staging/lustre/lnet/lnet/router.c
> @@ -292,10 +292,10 @@ int
> lnet_add_route(__u32 net, __u32 hops, lnet_nid_t gateway,
> 	       unsigned int priority)
> {
> -	struct list_head *e;
> 	struct lnet_remotenet *rnet;
> 	struct lnet_remotenet *rnet2;
> 	struct lnet_route *route;
> +	struct lnet_route *route2;
> 	struct lnet_ni *ni;
> 	int add_route;
> 	int rc;
> @@ -359,10 +359,8 @@ lnet_add_route(__u32 net, __u32 hops, lnet_nid_t gateway,
> 
> 	/* Search for a duplicate route (it's a NOOP if it is) */
> 	add_route = 1;
> -	list_for_each(e, &rnet2->lrn_routes) {
> -		struct lnet_route *route2;
> +	list_for_each_entry(route2, &rnet2->lrn_routes, lr_list) {
> 
> -		route2 = list_entry(e, struct lnet_route, lr_list);
> 		if (route2->lr_gateway == route->lr_gateway) {
> 			add_route = 0;
> 			break;
> @@ -411,8 +409,6 @@ lnet_check_routes(void)
> 	struct lnet_remotenet *rnet;
> 	struct lnet_route *route;
> 	struct lnet_route *route2;
> -	struct list_head *e1;
> -	struct list_head *e2;
> 	int cpt;
> 	struct list_head *rn_list;
> 	int i;
> @@ -421,17 +417,13 @@ lnet_check_routes(void)
> 
> 	for (i = 0; i < LNET_REMOTE_NETS_HASH_SIZE; i++) {
> 		rn_list = &the_lnet.ln_remote_nets_hash[i];
> -		list_for_each(e1, rn_list) {
> -			rnet = list_entry(e1, struct lnet_remotenet, lrn_list);
> -
> +		list_for_each_entry(rnet, rn_list, lrn_list) {
> 			route2 = NULL;
> -			list_for_each(e2, &rnet->lrn_routes) {
> +			list_for_each_entry(route, &rnet->lrn_routes, lr_list) {
> 				lnet_nid_t nid1;
> 				lnet_nid_t nid2;
> 				int net;
> 
> -				route = list_entry(e2, struct lnet_route, lr_list);
> -
> 				if (!route2) {
> 					route2 = route;
> 					continue;
> @@ -466,8 +458,6 @@ lnet_del_route(__u32 net, lnet_nid_t gw_nid)
> 	struct lnet_peer *gateway;
> 	struct lnet_remotenet *rnet;
> 	struct lnet_route *route;
> -	struct list_head *e1;
> -	struct list_head *e2;
> 	int rc = -ENOENT;
> 	struct list_head *rn_list;
> 	int idx = 0;
> @@ -486,16 +476,12 @@ lnet_del_route(__u32 net, lnet_nid_t gw_nid)
> 		rn_list = lnet_net2rnethash(net);
> 
>  again:
> -	list_for_each(e1, rn_list) {
> -		rnet = list_entry(e1, struct lnet_remotenet, lrn_list);
> -
> +	list_for_each_entry(rnet, rn_list, lrn_list) {
> 		if (!(net == LNET_NIDNET(LNET_NID_ANY) ||
> 		      net == rnet->lrn_net))
> 			continue;
> 
> -		list_for_each(e2, &rnet->lrn_routes) {
> -			route = list_entry(e2, struct lnet_route, lr_list);
> -
> +		list_for_each_entry(route, &rnet->lrn_routes, lr_list) {
> 			gateway = route->lr_gateway;
> 			if (!(gw_nid == LNET_NID_ANY ||
> 			      gw_nid == gateway->lp_nid))
> @@ -576,8 +562,6 @@ int
> lnet_get_route(int idx, __u32 *net, __u32 *hops,
> 	       lnet_nid_t *gateway, __u32 *alive, __u32 *priority)
> {
> -	struct list_head *e1;
> -	struct list_head *e2;
> 	struct lnet_remotenet *rnet;
> 	struct lnet_route *route;
> 	int cpt;
> @@ -588,13 +572,8 @@ lnet_get_route(int idx, __u32 *net, __u32 *hops,
> 
> 	for (i = 0; i < LNET_REMOTE_NETS_HASH_SIZE; i++) {
> 		rn_list = &the_lnet.ln_remote_nets_hash[i];
> -		list_for_each(e1, rn_list) {
> -			rnet = list_entry(e1, struct lnet_remotenet, lrn_list);
> -
> -			list_for_each(e2, &rnet->lrn_routes) {
> -				route = list_entry(e2, struct lnet_route,
> -						   lr_list);
> -
> +		list_for_each_entry(rnet, rn_list, lrn_list) {
> +			list_for_each_entry(route, &rnet->lrn_routes, lr_list) {
> 				if (!idx--) {
> 					*net      = rnet->lrn_net;
> 					*hops     = route->lr_hops;
> @@ -784,7 +763,6 @@ static void
> lnet_wait_known_routerstate(void)
> {
> 	struct lnet_peer *rtr;
> -	struct list_head *entry;
> 	int all_known;
> 
> 	LASSERT(the_lnet.ln_rc_state == LNET_RC_STATE_RUNNING);
> @@ -793,9 +771,7 @@ lnet_wait_known_routerstate(void)
> 		int cpt = lnet_net_lock_current();
> 
> 		all_known = 1;
> -		list_for_each(entry, &the_lnet.ln_routers) {
> -			rtr = list_entry(entry, struct lnet_peer, lp_rtr_list);
> -
> +		list_for_each_entry(rtr, &the_lnet.ln_routers, lp_rtr_list) {
> 			if (!rtr->lp_alive_count) {
> 				all_known = 0;
> 				break;
> @@ -1223,7 +1199,6 @@ static int
> lnet_router_checker(void *arg)
> {
> 	struct lnet_peer *rtr;
> -	struct list_head *entry;
> 
> 	while (the_lnet.ln_rc_state == LNET_RC_STATE_RUNNING) {
> 		__u64 version;
> @@ -1234,9 +1209,7 @@ lnet_router_checker(void *arg)
> rescan:
> 		version = the_lnet.ln_routers_version;
> 
> -		list_for_each(entry, &the_lnet.ln_routers) {
> -			rtr = list_entry(entry, struct lnet_peer, lp_rtr_list);
> -
> +		list_for_each_entry(rtr, &the_lnet.ln_routers, lp_rtr_list) {
> 			cpt2 = lnet_cpt_of_nid_locked(rtr->lp_nid);
> 			if (cpt != cpt2) {
> 				lnet_net_unlock(cpt);
> diff --git a/drivers/staging/lustre/lnet/selftest/conrpc.c b/drivers/staging/lustre/lnet/selftest/conrpc.c
> index 3afde0141db5..e73b956d15e4 100644
> --- a/drivers/staging/lustre/lnet/selftest/conrpc.c
> +++ b/drivers/staging/lustre/lnet/selftest/conrpc.c
> @@ -1327,7 +1327,6 @@ lstcon_rpc_cleanup_wait(void)
> {
> 	struct lstcon_rpc_trans *trans;
> 	struct lstcon_rpc *crpc;
> -	struct list_head *pacer;
> 	struct list_head zlist;
> 
> 	/* Called with hold of global mutex */
> @@ -1335,10 +1334,8 @@ lstcon_rpc_cleanup_wait(void)
> 	LASSERT(console_session.ses_shutdown);
> 
> 	while (!list_empty(&console_session.ses_trans_list)) {
> -		list_for_each(pacer, &console_session.ses_trans_list) {
> -			trans = list_entry(pacer, struct lstcon_rpc_trans,
> -					   tas_link);
> -
> +		list_for_each_entry(trans, &console_session.ses_trans_list,
> +				    tas_link) {
> 			CDEBUG(D_NET, "Session closed, wakeup transaction %s\n",
> 			       lstcon_rpc_trans_name(trans->tas_opc));
> 
> diff --git a/drivers/staging/lustre/lustre/obdclass/genops.c b/drivers/staging/lustre/lustre/obdclass/genops.c
> index 234f383ce6d9..8454b4470b8c 100644
> --- a/drivers/staging/lustre/lustre/obdclass/genops.c
> +++ b/drivers/staging/lustre/lustre/obdclass/genops.c
> @@ -84,12 +84,10 @@ static void obd_device_free(struct obd_device *obd)
> 
> static struct obd_type *class_search_type(const char *name)
> {
> -	struct list_head *tmp;
> 	struct obd_type *type;
> 
> 	spin_lock(&obd_types_lock);
> -	list_for_each(tmp, &obd_types) {
> -		type = list_entry(tmp, struct obd_type, typ_chain);
> +	list_for_each_entry(type, &obd_types, typ_chain) {
> 		if (strcmp(type->typ_name, name) == 0) {
> 			spin_unlock(&obd_types_lock);
> 			return type;
> diff --git a/drivers/staging/lustre/lustre/obdecho/echo_client.c b/drivers/staging/lustre/lustre/obdecho/echo_client.c
> index 484b8d6db6ef..3022706c6985 100644
> --- a/drivers/staging/lustre/lustre/obdecho/echo_client.c
> +++ b/drivers/staging/lustre/lustre/obdecho/echo_client.c
> @@ -950,12 +950,10 @@ static int cl_echo_cancel0(struct lu_env *env, struct echo_device *ed,
> {
> 	struct echo_client_obd *ec = ed->ed_ec;
> 	struct echo_lock       *ecl = NULL;
> -	struct list_head	     *el;
> 	int found = 0, still_used = 0;
> 
> 	spin_lock(&ec->ec_lock);
> -	list_for_each(el, &ec->ec_locks) {
> -		ecl = list_entry(el, struct echo_lock, el_chain);
> +	list_for_each_entry(ecl, &ec->ec_locks, el_chain) {
> 		CDEBUG(D_INFO, "ecl: %p, cookie: %#llx\n", ecl, ecl->el_cookie);
> 		found = (ecl->el_cookie == cookie);
> 		if (found) {
> 
> 

Cheers, Andreas
---
Andreas Dilger
Principal Lustre Architect
Whamcloud







-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 235 bytes
Desc: Message signed with OpenPGP
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180730/e4f15c10/attachment.sig>

From adilger at whamcloud.com  Mon Jul 30 21:09:56 2018
From: adilger at whamcloud.com (Andreas Dilger)
Date: Mon, 30 Jul 2018 21:09:56 +0000
Subject: [lustre-devel] [PATCH 6/6] lustre: convert list_for_each() in
 ksocknal_debug_peerhash()
In-Reply-To: <153292233972.26104.9739049020807431895.stgit@noble>
References: <153292233170.26104.16164388413209501300.stgit@noble>
 <153292233972.26104.9739049020807431895.stgit@noble>
Message-ID: <84837363-BB4D-40D4-9FA2-2F8A02915B04@whamcloud.com>


On Jul 29, 2018, at 21:45, NeilBrown <neilb at suse.com> wrote:
> 
> This list_for_each() loop searches for a particular entry,
> then acts of in.  It currently acts after the loop by testing
> if the variable is NULL.  When we convert to list_for_each_entry()
> it won't be NULL.
> 
> Change the code so the acting happens inside the loop.
> list_for_each_entry() {
>    if (this isn't it)
>        continue;
>    act on entry;
>    goto done; // break out of 2 loops
> }
> 
> Signed-off-by: NeilBrown <neilb at suse.com>

Reviewed-by: Andreas Dilger <adilger at whamcloud.com>

> ---
> .../staging/lustre/lnet/klnds/socklnd/socklnd.c    |   64 +++++++++-----------
> 1 file changed, 28 insertions(+), 36 deletions(-)
> 
> diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c
> index 33335713b371..f0b0480686dc 100644
> --- a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c
> +++ b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c
> @@ -2462,52 +2462,44 @@ static void
> ksocknal_debug_peerhash(struct lnet_ni *ni)
> {
> 	struct ksock_peer *peer = NULL;
> -	struct list_head *tmp;
> 	int i;
> 
> 	read_lock(&ksocknal_data.ksnd_global_lock);
> 
> 	for (i = 0; i < ksocknal_data.ksnd_peer_hash_size; i++) {
> -		list_for_each(tmp, &ksocknal_data.ksnd_peers[i]) {
> -			peer = list_entry(tmp, struct ksock_peer, ksnp_list);
> -
> -			if (peer->ksnp_ni == ni)
> -				break;
> +		list_for_each_entry(peer, &ksocknal_data.ksnd_peers[i], ksnp_list) {
> +			struct ksock_route *route;
> +			struct ksock_conn  *conn;
> 
> -			peer = NULL;
> -		}
> -	}
> +			if (peer->ksnp_ni != ni)
> +				continue;
> 
> -	if (peer) {
> -		struct ksock_route *route;
> -		struct ksock_conn  *conn;
> -
> -		CWARN("Active peer on shutdown: %s, ref %d, scnt %d, closing %d, accepting %d, err %d, zcookie %llu, txq %d, zc_req %d\n",
> -		      libcfs_id2str(peer->ksnp_id),
> -		      atomic_read(&peer->ksnp_refcount),
> -		      peer->ksnp_sharecount, peer->ksnp_closing,
> -		      peer->ksnp_accepting, peer->ksnp_error,
> -		      peer->ksnp_zc_next_cookie,
> -		      !list_empty(&peer->ksnp_tx_queue),
> -		      !list_empty(&peer->ksnp_zc_req_list));
> -
> -		list_for_each(tmp, &peer->ksnp_routes) {
> -			route = list_entry(tmp, struct ksock_route, ksnr_list);
> -			CWARN("Route: ref %d, schd %d, conn %d, cnted %d, del %d\n",
> -			      atomic_read(&route->ksnr_refcount),
> -			      route->ksnr_scheduled, route->ksnr_connecting,
> -			      route->ksnr_connected, route->ksnr_deleted);
> -		}
> +			CWARN("Active peer on shutdown: %s, ref %d, scnt %d, closing %d, accepting %d, err %d, zcookie %llu, txq %d, zc_req %d\n",
> +			      libcfs_id2str(peer->ksnp_id),
> +			      atomic_read(&peer->ksnp_refcount),
> +			      peer->ksnp_sharecount, peer->ksnp_closing,
> +			      peer->ksnp_accepting, peer->ksnp_error,
> +			      peer->ksnp_zc_next_cookie,
> +			      !list_empty(&peer->ksnp_tx_queue),
> +			      !list_empty(&peer->ksnp_zc_req_list));
> +
> +			list_for_each_entry(route, &peer->ksnp_routes, ksnr_list) {
> +				CWARN("Route: ref %d, schd %d, conn %d, cnted %d, del %d\n",
> +				      atomic_read(&route->ksnr_refcount),
> +				      route->ksnr_scheduled, route->ksnr_connecting,
> +				      route->ksnr_connected, route->ksnr_deleted);
> +			}
> 
> -		list_for_each(tmp, &peer->ksnp_conns) {
> -			conn = list_entry(tmp, struct ksock_conn, ksnc_list);
> -			CWARN("Conn: ref %d, sref %d, t %d, c %d\n",
> -			      atomic_read(&conn->ksnc_conn_refcount),
> -			      atomic_read(&conn->ksnc_sock_refcount),
> -			      conn->ksnc_type, conn->ksnc_closing);
> +			list_for_each_entry(conn, &peer->ksnp_conns, ksnc_list) {
> +				CWARN("Conn: ref %d, sref %d, t %d, c %d\n",
> +				      atomic_read(&conn->ksnc_conn_refcount),
> +				      atomic_read(&conn->ksnc_sock_refcount),
> +				      conn->ksnc_type, conn->ksnc_closing);
> +			}
> +			goto done;
> 		}
> 	}
> -
> +done:
> 	read_unlock(&ksocknal_data.ksnd_global_lock);
> }
> 
> 
> 

Cheers, Andreas
---
Andreas Dilger
CTO Whamcloud




-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 235 bytes
Desc: Message signed with OpenPGP
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180730/ec2a82cb/attachment.sig>

From adilger at whamcloud.com  Mon Jul 30 21:30:06 2018
From: adilger at whamcloud.com (Andreas Dilger)
Date: Mon, 30 Jul 2018 21:30:06 +0000
Subject: [lustre-devel] [PATCH 1/7] lustre: use
 schedule_timeout_$state().
In-Reply-To: <153292257279.26496.13641642926795469723.stgit@noble>
References: <153292249340.26496.13985174182824912332.stgit@noble>
 <153292257279.26496.13641642926795469723.stgit@noble>
Message-ID: <EBD164E9-9B9B-4F70-9F1E-9F4D5FBA42AE@whamcloud.com>

On Jul 29, 2018, at 21:49, NeilBrown <neilb at suse.com> wrote:
> 
> Lustre has many calls to
>   set_current_state(STATE);
>   schedule_timeout(time);
> 
> 
> These can more easily be done as
>    schedule_timeout_STATE(time);
> 
> Also clean up some oddities, such as setting the state
> to TASK_RUNNING after the timeout, and simplify
> some time calculations.
> 
> Some schedule_timeout() calls remain as the state was set earlier,
> before an 'add_wait_queue()'.  It would be incorrect to convert these.
> 
> Signed-off-by: NeilBrown <neilb at suse.com>

Reviewed-by: Andreas Dilger <adilger at whamcloud.com>

> ---
> .../staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c    |   12 ++++--------
> .../staging/lustre/lnet/klnds/socklnd/socklnd.c    |    6 ++----
> .../staging/lustre/lnet/klnds/socklnd/socklnd_cb.c |   14 ++++++--------
> drivers/staging/lustre/lnet/libcfs/fail.c          |    3 +--
> drivers/staging/lustre/lnet/libcfs/tracefile.c     |    3 +--
> drivers/staging/lustre/lnet/lnet/acceptor.c        |    3 +--
> drivers/staging/lustre/lnet/lnet/api-ni.c          |    6 ++----
> drivers/staging/lustre/lnet/lnet/peer.c            |    3 +--
> drivers/staging/lustre/lnet/lnet/router.c          |   19 +++++--------------
> drivers/staging/lustre/lnet/selftest/conrpc.c      |    3 +--
> drivers/staging/lustre/lnet/selftest/rpc.c         |    3 +--
> drivers/staging/lustre/lnet/selftest/selftest.h    |    3 +--
> drivers/staging/lustre/lustre/include/lustre_mdc.h |    2 +-
> drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c    |    3 +--
> drivers/staging/lustre/lustre/ldlm/ldlm_resource.c |    8 +++-----
> drivers/staging/lustre/lustre/llite/llite_lib.c    |    6 ++----
> .../staging/lustre/lustre/obdecho/echo_client.c    |    3 +--
> drivers/staging/lustre/lustre/ptlrpc/client.c      |    4 +---
> drivers/staging/lustre/lustre/ptlrpc/sec.c         |    3 +--
> 19 files changed, 36 insertions(+), 71 deletions(-)
> 
> diff --git a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c
> index e15ad94151bd..f496e6fcc416 100644
> --- a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c
> +++ b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c
> @@ -1201,8 +1201,7 @@ static struct kib_hca_dev *kiblnd_current_hdev(struct kib_dev *dev)
> 		if (!(i++ % 50))
> 			CDEBUG(D_NET, "%s: Wait for failover\n",
> 			       dev->ibd_ifname);
> -		set_current_state(TASK_INTERRUPTIBLE);
> -		schedule_timeout(HZ / 100);
> +		schedule_timeout_interruptible(HZ / 100);
> 
> 		read_lock_irqsave(&kiblnd_data.kib_global_lock, flags);
> 	}
> @@ -1916,8 +1915,7 @@ struct list_head *kiblnd_pool_alloc_node(struct kib_poolset *ps)
> 		CDEBUG(D_NET, "Another thread is allocating new %s pool, waiting %d HZs for her to complete. trips = %d\n",
> 		       ps->ps_name, interval, trips);
> 
> -		set_current_state(TASK_INTERRUPTIBLE);
> -		schedule_timeout(interval);
> +		schedule_timeout_interruptible(interval);
> 		if (interval < HZ)
> 			interval *= 2;
> 
> @@ -2548,8 +2546,7 @@ static void kiblnd_base_shutdown(void)
> 			CDEBUG(((i & (-i)) == i) ? D_WARNING : D_NET,
> 			       "Waiting for %d threads to terminate\n",
> 			       atomic_read(&kiblnd_data.kib_nthreads));
> -			set_current_state(TASK_UNINTERRUPTIBLE);
> -			schedule_timeout(HZ);
> +			schedule_timeout_uninterruptible(HZ);
> 		}
> 
> 		/* fall through */
> @@ -2599,8 +2596,7 @@ static void kiblnd_shutdown(struct lnet_ni *ni)
> 			       "%s: waiting for %d peers to disconnect\n",
> 			       libcfs_nid2str(ni->ni_nid),
> 			       atomic_read(&net->ibn_npeers));
> -			set_current_state(TASK_UNINTERRUPTIBLE);
> -			schedule_timeout(HZ);
> +			schedule_timeout_uninterruptible(HZ);
> 		}
> 
> 		kiblnd_net_fini_pools(net);
> diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c
> index f0b0480686dc..4dde158451ea 100644
> --- a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c
> +++ b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c
> @@ -2307,8 +2307,7 @@ ksocknal_base_shutdown(void)
> 			       "waiting for %d threads to terminate\n",
> 				ksocknal_data.ksnd_nthreads);
> 			read_unlock(&ksocknal_data.ksnd_global_lock);
> -			set_current_state(TASK_UNINTERRUPTIBLE);
> -			schedule_timeout(HZ);
> +			schedule_timeout_uninterruptible(HZ);
> 			read_lock(&ksocknal_data.ksnd_global_lock);
> 		}
> 		read_unlock(&ksocknal_data.ksnd_global_lock);
> @@ -2533,8 +2532,7 @@ ksocknal_shutdown(struct lnet_ni *ni)
> 		CDEBUG(((i & (-i)) == i) ? D_WARNING : D_NET, /* power of 2? */
> 		       "waiting for %d peers to disconnect\n",
> 		       net->ksnn_npeers);
> -		set_current_state(TASK_UNINTERRUPTIBLE);
> -		schedule_timeout(HZ);
> +		schedule_timeout_uninterruptible(HZ);
> 
> 		ksocknal_debug_peerhash(ni);
> 
> diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c
> index a5c0e8a9bc40..32b76727f400 100644
> --- a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c
> +++ b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c
> @@ -188,10 +188,9 @@ ksocknal_transmit(struct ksock_conn *conn, struct ksock_tx *tx)
> 	int rc;
> 	int bufnob;
> 
> -	if (ksocknal_data.ksnd_stall_tx) {
> -		set_current_state(TASK_UNINTERRUPTIBLE);
> -		schedule_timeout(ksocknal_data.ksnd_stall_tx * HZ);
> -	}
> +	if (ksocknal_data.ksnd_stall_tx)
> +		schedule_timeout_uninterruptible(
> +			ksocknal_data.ksnd_stall_tx * HZ);
> 
> 	LASSERT(tx->tx_resid);
> 
> @@ -293,10 +292,9 @@ ksocknal_receive(struct ksock_conn *conn)
> 	 */
> 	int rc;
> 
> -	if (ksocknal_data.ksnd_stall_rx) {
> -		set_current_state(TASK_UNINTERRUPTIBLE);
> -		schedule_timeout(ksocknal_data.ksnd_stall_rx * HZ);
> -	}
> +	if (ksocknal_data.ksnd_stall_rx)
> +		schedule_timeout_uninterruptible(
> +			ksocknal_data.ksnd_stall_rx * HZ);
> 
> 	rc = ksocknal_connsock_addref(conn);
> 	if (rc) {
> diff --git a/drivers/staging/lustre/lnet/libcfs/fail.c b/drivers/staging/lustre/lnet/libcfs/fail.c
> index bd86b3b5bc34..6ee4de2178ce 100644
> --- a/drivers/staging/lustre/lnet/libcfs/fail.c
> +++ b/drivers/staging/lustre/lnet/libcfs/fail.c
> @@ -137,8 +137,7 @@ int __cfs_fail_timeout_set(u32 id, u32 value, int ms, int set)
> 	if (ret && likely(ms > 0)) {
> 		CERROR("cfs_fail_timeout id %x sleeping for %dms\n",
> 		       id, ms);
> -		set_current_state(TASK_UNINTERRUPTIBLE);
> -		schedule_timeout(ms * HZ / 1000);
> +		schedule_timeout_uninterruptible(ms * HZ / 1000);
> 		CERROR("cfs_fail_timeout id %x awake\n", id);
> 	}
> 	return ret;
> diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.c b/drivers/staging/lustre/lnet/libcfs/tracefile.c
> index a4768e930021..d4c80cf254e4 100644
> --- a/drivers/staging/lustre/lnet/libcfs/tracefile.c
> +++ b/drivers/staging/lustre/lnet/libcfs/tracefile.c
> @@ -1208,8 +1208,7 @@ static int tracefiled(void *arg)
> 		}
> 		init_waitqueue_entry(&__wait, current);
> 		add_wait_queue(&tctl->tctl_waitq, &__wait);
> -		set_current_state(TASK_INTERRUPTIBLE);
> -		schedule_timeout(HZ);
> +		schedule_timeout_interruptible(HZ);
> 		remove_wait_queue(&tctl->tctl_waitq, &__wait);
> 	}
> 	complete(&tctl->tctl_stop);
> diff --git a/drivers/staging/lustre/lnet/lnet/acceptor.c b/drivers/staging/lustre/lnet/lnet/acceptor.c
> index 5648f17eddc0..3ae3ca1311a1 100644
> --- a/drivers/staging/lustre/lnet/lnet/acceptor.c
> +++ b/drivers/staging/lustre/lnet/lnet/acceptor.c
> @@ -362,8 +362,7 @@ lnet_acceptor(void *arg)
> 		if (rc) {
> 			if (rc != -EAGAIN) {
> 				CWARN("Accept error %d: pausing...\n", rc);
> -				set_current_state(TASK_UNINTERRUPTIBLE);
> -				schedule_timeout(HZ);
> +				schedule_timeout_uninterruptible(HZ);
> 			}
> 			continue;
> 		}
> diff --git a/drivers/staging/lustre/lnet/lnet/api-ni.c b/drivers/staging/lustre/lnet/lnet/api-ni.c
> index 14b797802a85..cdbbe9cc8d95 100644
> --- a/drivers/staging/lustre/lnet/lnet/api-ni.c
> +++ b/drivers/staging/lustre/lnet/lnet/api-ni.c
> @@ -955,8 +955,7 @@ lnet_ping_md_unlink(struct lnet_ping_info *pinfo,
> 	/* NB md could be busy; this just starts the unlink */
> 	while (pinfo->pi_features != LNET_PING_FEAT_INVAL) {
> 		CDEBUG(D_NET, "Still waiting for ping MD to unlink\n");
> -		set_current_state(TASK_NOLOAD);
> -		schedule_timeout(HZ);
> +		schedule_timeout_idle(HZ);
> 	}
> }
> 
> @@ -1093,8 +1092,7 @@ lnet_clear_zombies_nis_locked(void)
> 				CDEBUG(D_WARNING, "Waiting for zombie LNI %s\n",
> 				       libcfs_nid2str(ni->ni_nid));
> 			}
> -			set_current_state(TASK_UNINTERRUPTIBLE);
> -			schedule_timeout(HZ);
> +			schedule_timeout_uninterruptible(HZ);
> 			lnet_net_lock(LNET_LOCK_EX);
> 			continue;
> 		}
> diff --git a/drivers/staging/lustre/lnet/lnet/peer.c b/drivers/staging/lustre/lnet/lnet/peer.c
> index 7c303ef6bb34..d9452c322e4d 100644
> --- a/drivers/staging/lustre/lnet/lnet/peer.c
> +++ b/drivers/staging/lustre/lnet/lnet/peer.c
> @@ -136,8 +136,7 @@ lnet_peer_table_deathrow_wait_locked(struct lnet_peer_table *ptable,
> 			       "Waiting for %d zombies on peer table\n",
> 			       ptable->pt_zombies);
> 		}
> -		set_current_state(TASK_UNINTERRUPTIBLE);
> -		schedule_timeout(HZ >> 1);
> +		schedule_timeout_uninterruptible(HZ >> 1);
> 		lnet_net_lock(cpt_locked);
> 	}
> }
> diff --git a/drivers/staging/lustre/lnet/lnet/router.c b/drivers/staging/lustre/lnet/lnet/router.c
> index 53373372b526..02241fbc9eaa 100644
> --- a/drivers/staging/lustre/lnet/lnet/router.c
> +++ b/drivers/staging/lustre/lnet/lnet/router.c
> @@ -783,8 +783,7 @@ lnet_wait_known_routerstate(void)
> 		if (all_known)
> 			return;
> 
> -		set_current_state(TASK_UNINTERRUPTIBLE);
> -		schedule_timeout(HZ);
> +		schedule_timeout_uninterruptible(HZ);
> 	}
> }
> 
> @@ -1159,8 +1158,7 @@ lnet_prune_rc_data(int wait_unlink)
> 		i++;
> 		CDEBUG(((i & (-i)) == i) ? D_WARNING : D_NET,
> 		       "Waiting for rc buffers to unlink\n");
> -		set_current_state(TASK_UNINTERRUPTIBLE);
> -		schedule_timeout(HZ / 4);
> +		schedule_timeout_uninterruptible(HZ / 4);
> 
> 		lnet_net_lock(LNET_LOCK_EX);
> 	}
> @@ -1236,23 +1234,16 @@ lnet_router_checker(void *arg)
> 
> 		lnet_prune_rc_data(0); /* don't wait for UNLINK */
> 
> -		/*
> -		 * Call schedule_timeout() here always adds 1 to load average
> -		 * because kernel counts # active tasks as nr_running
> -		 * + nr_uninterruptible.
> -		 */
> 		/*
> 		 * if there are any routes then wakeup every second.  If
> 		 * there are no routes then sleep indefinitely until woken
> 		 * up by a user adding a route
> 		 */
> 		if (!lnet_router_checker_active())
> -			wait_event_interruptible(the_lnet.ln_rc_waitq,
> -						 lnet_router_checker_active());
> +			wait_event_idle(the_lnet.ln_rc_waitq,
> +					lnet_router_checker_active());
> 		else
> -			wait_event_interruptible_timeout(the_lnet.ln_rc_waitq,
> -							 false,
> -							 HZ);
> +			schedule_timeout_idle(HZ);
> 	}
> 
> 	lnet_prune_rc_data(1); /* wait for UNLINK */
> diff --git a/drivers/staging/lustre/lnet/selftest/conrpc.c b/drivers/staging/lustre/lnet/selftest/conrpc.c
> index e73b956d15e4..7809c1fc6f73 100644
> --- a/drivers/staging/lustre/lnet/selftest/conrpc.c
> +++ b/drivers/staging/lustre/lnet/selftest/conrpc.c
> @@ -1345,8 +1345,7 @@ lstcon_rpc_cleanup_wait(void)
> 		mutex_unlock(&console_session.ses_mutex);
> 
> 		CWARN("Session is shutting down, waiting for termination of transactions\n");
> -		set_current_state(TASK_UNINTERRUPTIBLE);
> -		schedule_timeout(HZ);
> +		schedule_timeout_uninterruptible(HZ);
> 
> 		mutex_lock(&console_session.ses_mutex);
> 	}
> diff --git a/drivers/staging/lustre/lnet/selftest/rpc.c b/drivers/staging/lustre/lnet/selftest/rpc.c
> index e097ef8414a6..298de41444b3 100644
> --- a/drivers/staging/lustre/lnet/selftest/rpc.c
> +++ b/drivers/staging/lustre/lnet/selftest/rpc.c
> @@ -1603,8 +1603,7 @@ srpc_startup(void)
> 	spin_lock_init(&srpc_data.rpc_glock);
> 
> 	/* 1 second pause to avoid timestamp reuse */
> -	set_current_state(TASK_UNINTERRUPTIBLE);
> -	schedule_timeout(HZ);
> +	schedule_timeout_uninterruptible(HZ);
> 	srpc_data.rpc_matchbits = ((__u64)ktime_get_real_seconds()) << 48;
> 
> 	srpc_data.rpc_state = SRPC_STATE_NONE;
> diff --git a/drivers/staging/lustre/lnet/selftest/selftest.h b/drivers/staging/lustre/lnet/selftest/selftest.h
> index ad9be095c4ea..9dbb0a51d430 100644
> --- a/drivers/staging/lustre/lnet/selftest/selftest.h
> +++ b/drivers/staging/lustre/lnet/selftest/selftest.h
> @@ -573,8 +573,7 @@ swi_state2str(int state)
> 
> #define selftest_wait_events()					\
> 	do {							\
> -		set_current_state(TASK_UNINTERRUPTIBLE);	\
> -		schedule_timeout(HZ / 10);	\
> +		schedule_timeout_uninterruptible(HZ / 10);	\
> 	} while (0)
> 
> #define lst_wait_until(cond, lock, fmt, ...)				\
> diff --git a/drivers/staging/lustre/lustre/include/lustre_mdc.h b/drivers/staging/lustre/lustre/include/lustre_mdc.h
> index a9c9992a2502..6ac7fc4fa8c6 100644
> --- a/drivers/staging/lustre/lustre/include/lustre_mdc.h
> +++ b/drivers/staging/lustre/lustre/include/lustre_mdc.h
> @@ -124,7 +124,7 @@ static inline void mdc_get_rpc_lock(struct mdc_rpc_lock *lck,
> 	 */
> 	while (unlikely(lck->rpcl_it == MDC_FAKE_RPCL_IT)) {
> 		mutex_unlock(&lck->rpcl_mutex);
> -		schedule_timeout(HZ / 4);
> +		schedule_timeout_uninterruptible(HZ / 4);
> 		goto again;
> 	}
> 
> diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c b/drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c
> index 5b125fdc7321..0ee4798f1bb9 100644
> --- a/drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c
> +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c
> @@ -167,8 +167,7 @@ static void ldlm_handle_cp_callback(struct ptlrpc_request *req,
> 		int to = HZ;
> 
> 		while (to > 0) {
> -			set_current_state(TASK_INTERRUPTIBLE);
> -			schedule_timeout(to);
> +			schedule_timeout_interruptible(to);
> 			if (lock->l_granted_mode == lock->l_req_mode ||
> 			    ldlm_is_destroyed(lock))
> 				break;
> diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_resource.c b/drivers/staging/lustre/lustre/ldlm/ldlm_resource.c
> index f06cbd8b6d13..33d73fa8e9d5 100644
> --- a/drivers/staging/lustre/lustre/ldlm/ldlm_resource.c
> +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_resource.c
> @@ -749,11 +749,9 @@ static void cleanup_resource(struct ldlm_resource *res, struct list_head *q,
> 			 */
> 			unlock_res(res);
> 			LDLM_DEBUG(lock, "setting FL_LOCAL_ONLY");
> -			if (lock->l_flags & LDLM_FL_FAIL_LOC) {
> -				set_current_state(TASK_UNINTERRUPTIBLE);
> -				schedule_timeout(4 * HZ);
> -				set_current_state(TASK_RUNNING);
> -			}
> +			if (lock->l_flags & LDLM_FL_FAIL_LOC)
> +				schedule_timeout_uninterruptible(4 * HZ);
> +
> 			if (lock->l_completion_ast)
> 				lock->l_completion_ast(lock, LDLM_FL_FAILED,
> 						       NULL);
> diff --git a/drivers/staging/lustre/lustre/llite/llite_lib.c b/drivers/staging/lustre/lustre/llite/llite_lib.c
> index 5c8d0fe7217e..3dedc61d2257 100644
> --- a/drivers/staging/lustre/lustre/llite/llite_lib.c
> +++ b/drivers/staging/lustre/lustre/llite/llite_lib.c
> @@ -706,10 +706,8 @@ void ll_kill_super(struct super_block *sb)
> 		sbi->ll_umounting = 1;
> 
> 		/* wait running statahead threads to quit */
> -		while (atomic_read(&sbi->ll_sa_running) > 0) {
> -			set_current_state(TASK_UNINTERRUPTIBLE);
> -			schedule_timeout(msecs_to_jiffies(MSEC_PER_SEC >> 3));
> -		}
> +		while (atomic_read(&sbi->ll_sa_running) > 0)
> +			schedule_timeout_uninterruptible(HZ >> 3);
> 	}
> }
> 
> diff --git a/drivers/staging/lustre/lustre/obdecho/echo_client.c b/drivers/staging/lustre/lustre/obdecho/echo_client.c
> index 3022706c6985..1ddb4a6dd8f3 100644
> --- a/drivers/staging/lustre/lustre/obdecho/echo_client.c
> +++ b/drivers/staging/lustre/lustre/obdecho/echo_client.c
> @@ -751,8 +751,7 @@ static struct lu_device *echo_device_free(const struct lu_env *env,
> 	while (!list_empty(&ec->ec_objects)) {
> 		spin_unlock(&ec->ec_lock);
> 		CERROR("echo_client still has objects at cleanup time, wait for 1 second\n");
> -		set_current_state(TASK_UNINTERRUPTIBLE);
> -		schedule_timeout(HZ);
> +		schedule_timeout_uninterruptible(HZ);
> 		lu_site_purge(env, ed->ed_site, -1);
> 		spin_lock(&ec->ec_lock);
> 	}
> diff --git a/drivers/staging/lustre/lustre/ptlrpc/client.c b/drivers/staging/lustre/lustre/ptlrpc/client.c
> index 7a3d83c0e50b..91dd09867260 100644
> --- a/drivers/staging/lustre/lustre/ptlrpc/client.c
> +++ b/drivers/staging/lustre/lustre/ptlrpc/client.c
> @@ -761,9 +761,7 @@ int ptlrpc_request_bufs_pack(struct ptlrpc_request *request,
> 			/* The RPC is infected, let the test change the
> 			 * fail_loc
> 			 */
> -			set_current_state(TASK_UNINTERRUPTIBLE);
> -			schedule_timeout(2 * HZ);
> -			set_current_state(TASK_RUNNING);
> +			schedule_timeout_uninterruptible(2 * HZ);
> 		}
> 	}
> 
> diff --git a/drivers/staging/lustre/lustre/ptlrpc/sec.c b/drivers/staging/lustre/lustre/ptlrpc/sec.c
> index 9b60292370a7..9c598710b576 100644
> --- a/drivers/staging/lustre/lustre/ptlrpc/sec.c
> +++ b/drivers/staging/lustre/lustre/ptlrpc/sec.c
> @@ -514,8 +514,7 @@ static int sptlrpc_req_replace_dead_ctx(struct ptlrpc_request *req)
> 		       "ctx (%p, fl %lx) doesn't switch, relax a little bit\n",
> 		       newctx, newctx->cc_flags);
> 
> -		set_current_state(TASK_INTERRUPTIBLE);
> -		schedule_timeout(msecs_to_jiffies(MSEC_PER_SEC));
> +		schedule_timeout_interruptible(HZ);
> 	} else if (unlikely(!test_bit(PTLRPC_CTX_UPTODATE_BIT, &newctx->cc_flags))) {
> 		/*
> 		 * new ctx not up to date yet
> 
> 

Cheers, Andreas
---
Andreas Dilger
CTO Whamcloud




-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 235 bytes
Desc: Message signed with OpenPGP
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180730/af0d9a23/attachment.sig>

From adilger at whamcloud.com  Mon Jul 30 21:31:05 2018
From: adilger at whamcloud.com (Andreas Dilger)
Date: Mon, 30 Jul 2018 21:31:05 +0000
Subject: [lustre-devel] [PATCH 2/7] lustre/libcfs: fix freeing after
 kmalloc failure.
In-Reply-To: <153292257285.26496.5271671024215297861.stgit@noble>
References: <153292249340.26496.13985174182824912332.stgit@noble>
 <153292257285.26496.5271671024215297861.stgit@noble>
Message-ID: <B252C8B5-92E6-4644-9958-D321462893A4@whamcloud.com>

On Jul 29, 2018, at 21:49, NeilBrown <neilb at suse.com> wrote:
> 
> The new_bkts array is *not* zeroed (any more) so when
> freeing recently allocated buckets on failure, we
> must no free beyond the last bucket successfully
> allocated.
> 
> Fixes: 12e46c461cb9 ("staging: lustre: change some LIBCFS_ALLOC calls to k?alloc(GFP_KERNEL)")
> Signed-off-by: NeilBrown <neilb at suse.com>

Reviewed-by: Andreas Dilger <adilger at whamcloud.com>

> ---
> drivers/staging/lustre/lnet/libcfs/hash.c |    2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/drivers/staging/lustre/lnet/libcfs/hash.c b/drivers/staging/lustre/lnet/libcfs/hash.c
> index 48be66f0d654..f452c4540ca1 100644
> --- a/drivers/staging/lustre/lnet/libcfs/hash.c
> +++ b/drivers/staging/lustre/lnet/libcfs/hash.c
> @@ -904,7 +904,7 @@ cfs_hash_buckets_realloc(struct cfs_hash *hs, struct cfs_hash_bucket **old_bkts,
> 		new_bkts[i] = kzalloc(cfs_hash_bkt_size(hs), GFP_KERNEL);
> 		if (!new_bkts[i]) {
> 			cfs_hash_buckets_free(new_bkts, cfs_hash_bkt_size(hs),
> -					      old_size, new_size);
> +					      old_size, i);
> 			return NULL;
> 		}
> 
> 
> 

Cheers, Andreas
---
Andreas Dilger
CTO Whamcloud




-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 235 bytes
Desc: Message signed with OpenPGP
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180730/3f7477a3/attachment.sig>

From adilger at whamcloud.com  Mon Jul 30 21:32:08 2018
From: adilger at whamcloud.com (Andreas Dilger)
Date: Mon, 30 Jul 2018 21:32:08 +0000
Subject: [lustre-devel] [PATCH 3/7] lustre/libfs: move debugfs
 registration from libcfs_setup back to libcfs_init
In-Reply-To: <153292257289.26496.3865601138425685904.stgit@noble>
References: <153292249340.26496.13985174182824912332.stgit@noble>
 <153292257289.26496.3865601138425685904.stgit@noble>
Message-ID: <A290D531-1A4B-4ABE-BA22-91F823F8C5BB@whamcloud.com>

On Jul 29, 2018, at 21:49, NeilBrown <neilb at suse.com> wrote:
> 
> large memory allocations should be avoided at module-init,
> but registering services is appropriate.
> So move the registration of debugfs files
> back into libcfs_init().
> Without this, /sys/kernel/debug/lnet etc are not visible
> immediately that libcfs is loaded.
> No debugfs file access needs anything allocated by libcfs_setup().
> 
> Fixes: 64bf0b1a079d ("staging: lustre: refactor libcfs initialization.")
> Signed-off-by: NeilBrown <neilb at suse.com>

Reviewed-by: Andreas Dilger <adilger at whamcloud.com>

> ---
> drivers/staging/lustre/lnet/libcfs/module.c |    8 ++++----
> 1 file changed, 4 insertions(+), 4 deletions(-)
> 
> diff --git a/drivers/staging/lustre/lnet/libcfs/module.c b/drivers/staging/lustre/lnet/libcfs/module.c
> index bfadfcfa3c44..5d2be941777e 100644
> --- a/drivers/staging/lustre/lnet/libcfs/module.c
> +++ b/drivers/staging/lustre/lnet/libcfs/module.c
> @@ -719,10 +719,6 @@ int libcfs_setup(void)
> 		goto err;
> 	}
> 
> -	lnet_insert_debugfs(lnet_table);
> -	if (!IS_ERR_OR_NULL(lnet_debugfs_root))
> -		lnet_insert_debugfs_links(lnet_debugfs_symlinks);
> -
> 	CDEBUG(D_OTHER, "portals setup OK\n");
> out:
> 	libcfs_active = 1;
> @@ -743,6 +739,10 @@ static int libcfs_init(void)
> {
> 	int rc;
> 
> +	lnet_insert_debugfs(lnet_table);
> +	if (!IS_ERR_OR_NULL(lnet_debugfs_root))
> +		lnet_insert_debugfs_links(lnet_debugfs_symlinks);
> +
> 	rc = misc_register(&libcfs_dev);
> 	if (rc)
> 		CERROR("misc_register: error %d\n", rc);
> 
> 

Cheers, Andreas
---
Andreas Dilger
CTO Whamcloud




-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 235 bytes
Desc: Message signed with OpenPGP
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180730/55c1b423/attachment.sig>

From adilger at whamcloud.com  Mon Jul 30 21:32:54 2018
From: adilger at whamcloud.com (Andreas Dilger)
Date: Mon, 30 Jul 2018 21:32:54 +0000
Subject: [lustre-devel] [PATCH 4/7] lustre: give different tcd_lock
 types different classes.
In-Reply-To: <153292257292.26496.6025900918938045378.stgit@noble>
References: <153292249340.26496.13985174182824912332.stgit@noble>
 <153292257292.26496.6025900918938045378.stgit@noble>
Message-ID: <737EAA0C-F4F2-4C5A-ACDC-85BB557ACA5F@whamcloud.com>

On Jul 29, 2018, at 21:49, NeilBrown <neilb at suse.com> wrote:
> 
> There are three different trace contexts:
> process, softirq, irq.
> Each has its own lock (tcd_lock) which is locked
> as appropriate for that context.
> lockdep currently doesn't see that they are different
> and so deduces that the different uses might lead to
> deadlocks.
> So use separate calls to spin_lock_init() so that they
> each get a separate lock class, and lockdep sees no
> problem.
> 
> Signed-off-by: NeilBrown <neilb at suse.com>

Reviewed-by: Andreas Dilger <adilger at whamcloud.com>

> ---
> drivers/staging/lustre/lnet/libcfs/tracefile.c |   18 +++++++++++++++++-
> 1 file changed, 17 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.c b/drivers/staging/lustre/lnet/libcfs/tracefile.c
> index d4c80cf254e4..40048165fc16 100644
> --- a/drivers/staging/lustre/lnet/libcfs/tracefile.c
> +++ b/drivers/staging/lustre/lnet/libcfs/tracefile.c
> @@ -1285,7 +1285,23 @@ int cfs_tracefile_init(int max_pages)
> 	cfs_tcd_for_each(tcd, i, j) {
> 		int factor = pages_factor[i];
> 
> -		spin_lock_init(&tcd->tcd_lock);
> +		/* Note that we have three separate calls so
> +		 * they the locks get three separate classes
> +		 * and lockdep never thinks they are related.
> +		 * As they are used in different interrupt
> +		 * contexts, lockdep think the usage would conflict.
> +		 */
> +		switch(i) {
> +		case CFS_TCD_TYPE_PROC:
> +			spin_lock_init(&tcd->tcd_lock);
> +			break;
> +		case CFS_TCD_TYPE_SOFTIRQ:
> +			spin_lock_init(&tcd->tcd_lock);
> +			break;
> +		case CFS_TCD_TYPE_IRQ:
> +			spin_lock_init(&tcd->tcd_lock);
> +			break;
> +		}
> 		tcd->tcd_pages_factor = factor;
> 		tcd->tcd_type = i;
> 		tcd->tcd_cpu = j;
> 
> 

Cheers, Andreas
---
Andreas Dilger
CTO Whamcloud




-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 235 bytes
Desc: Message signed with OpenPGP
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180730/6120dad5/attachment.sig>

From adilger at whamcloud.com  Mon Jul 30 21:37:33 2018
From: adilger at whamcloud.com (Andreas Dilger)
Date: Mon, 30 Jul 2018 21:37:33 +0000
Subject: [lustre-devel] [PATCH 5/7] lustre/libcfs: discard
 cfs_trace_allocate_string_buffer()
In-Reply-To: <153292257295.26496.1353870060207416213.stgit@noble>
References: <153292249340.26496.13985174182824912332.stgit@noble>
 <153292257295.26496.1353870060207416213.stgit@noble>
Message-ID: <0DB1DFBB-B989-43A0-BD9A-ABF8FC4DB084@whamcloud.com>



> On Jul 29, 2018, at 21:49, NeilBrown <neilb at suse.com> wrote:
> 
> cfs_trace_allocate_string_buffer() is a simple wrapper
> around kzalloc() that adds little value.  The code is
> clearer if we perform the test and the allocation
> directly where needed.
> 
> Also change the test from '>' to '>=' to ensure we
> never try to allocate more than 2 pages, as that seems
> to be the intent.
> 
> Signed-off-by: NeilBrown <neilb at suse.com>

I was going to say that we probably don't have even a single debug message that is larger than PAGE_SIZE, but I suspect in some cases, printing a message with a PATH_MAX-sized filename may result in a buffer that is larger than PAGE_SIZE, though smaller than 2x PAGE_SIZE.

It isn't clear that returning an error for = 2 * PAGE_SIZE makes sense, but I also don't think this corner case is critical (and it leaves a byte for a NUL
terminator in the buffer).

Reviewed-by: Andreas Dilger <adilger at whamcloud.com>

> ---
> drivers/staging/lustre/lnet/libcfs/module.c    |    6 +++--
> drivers/staging/lustre/lnet/libcfs/tracefile.c |   28 +++++++++---------------
> drivers/staging/lustre/lnet/libcfs/tracefile.h |    1 -
> 3 files changed, 13 insertions(+), 22 deletions(-)
> 
> diff --git a/drivers/staging/lustre/lnet/libcfs/module.c b/drivers/staging/lustre/lnet/libcfs/module.c
> index 5d2be941777e..1de83b1997c6 100644
> --- a/drivers/staging/lustre/lnet/libcfs/module.c
> +++ b/drivers/staging/lustre/lnet/libcfs/module.c
> @@ -305,9 +305,9 @@ static int proc_dobitmasks(struct ctl_table *table, int write,
> 	int is_subsys = (mask == &libcfs_subsystem_debug) ? 1 : 0;
> 	int is_printk = (mask == &libcfs_printk) ? 1 : 0;
> 
> -	rc = cfs_trace_allocate_string_buffer(&tmpstr, tmpstrlen);
> -	if (rc < 0)
> -		return rc;
> +	tmpstr = kzalloc(tmpstrlen, GFP_KERNEL);
> +	if (!tmpstr)
> +		return -ENOMEM;
> 
> 	if (!write) {
> 		libcfs_debug_mask2str(tmpstr, tmpstrlen, *mask, is_subsys);
> diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.c b/drivers/staging/lustre/lnet/libcfs/tracefile.c
> index 40048165fc16..b273107b3815 100644
> --- a/drivers/staging/lustre/lnet/libcfs/tracefile.c
> +++ b/drivers/staging/lustre/lnet/libcfs/tracefile.c
> @@ -963,26 +963,16 @@ int cfs_trace_copyout_string(char __user *usr_buffer, int usr_buffer_nob,
> }
> EXPORT_SYMBOL(cfs_trace_copyout_string);
> 
> -int cfs_trace_allocate_string_buffer(char **str, int nob)
> -{
> -	if (nob > 2 * PAGE_SIZE)	    /* string must be "sensible" */
> -		return -EINVAL;
> -
> -	*str = kmalloc(nob, GFP_KERNEL | __GFP_ZERO);
> -	if (!*str)
> -		return -ENOMEM;
> -
> -	return 0;
> -}
> -
> int cfs_trace_dump_debug_buffer_usrstr(void __user *usr_str, int usr_str_nob)
> {
> 	char *str;
> 	int rc;
> 
> -	rc = cfs_trace_allocate_string_buffer(&str, usr_str_nob + 1);
> -	if (rc)
> -		return rc;
> +	if (usr_str_nob >= 2 * PAGE_SIZE)
> +		return -EINVAL;
> +	str = kzalloc(usr_str_nob + 1, GFP_KERNEL);
> +	if (!str)
> +		return -ENOMEM;
> 
> 	rc = cfs_trace_copyin_string(str, usr_str_nob + 1,
> 				     usr_str, usr_str_nob);
> @@ -1044,9 +1034,11 @@ int cfs_trace_daemon_command_usrstr(void __user *usr_str, int usr_str_nob)
> 	char *str;
> 	int rc;
> 
> -	rc = cfs_trace_allocate_string_buffer(&str, usr_str_nob + 1);
> -	if (rc)
> -		return rc;
> +	if (usr_str_nob >= 2 * PAGE_SIZE)
> +		return -EINVAL;
> +	str = kzalloc(usr_str_nob + 1, GFP_KERNEL);
> +	if (!str)
> +		return -ENOMEM;
> 
> 	rc = cfs_trace_copyin_string(str, usr_str_nob + 1,
> 				     usr_str, usr_str_nob);
> diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.h b/drivers/staging/lustre/lnet/libcfs/tracefile.h
> index 82f090fd8dfa..2134549bb3d7 100644
> --- a/drivers/staging/lustre/lnet/libcfs/tracefile.h
> +++ b/drivers/staging/lustre/lnet/libcfs/tracefile.h
> @@ -63,7 +63,6 @@ int cfs_trace_copyin_string(char *knl_buffer, int knl_buffer_nob,
> 			    const char __user *usr_buffer, int usr_buffer_nob);
> int cfs_trace_copyout_string(char __user *usr_buffer, int usr_buffer_nob,
> 			     const char *knl_str, char *append);
> -int cfs_trace_allocate_string_buffer(char **str, int nob);
> int cfs_trace_dump_debug_buffer_usrstr(void __user *usr_str, int usr_str_nob);
> int cfs_trace_daemon_command(char *str);
> int cfs_trace_daemon_command_usrstr(void __user *usr_str, int usr_str_nob);
> 
> 

Cheers, Andreas
---
Andreas Dilger
CTO Whamcloud




-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 235 bytes
Desc: Message signed with OpenPGP
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180730/7ec4de1e/attachment.sig>

From adilger at whamcloud.com  Mon Jul 30 21:43:51 2018
From: adilger at whamcloud.com (Andreas Dilger)
Date: Mon, 30 Jul 2018 21:43:51 +0000
Subject: [lustre-devel] [PATCH 7/7] lustre: change TASK_NOLOAD to
 TASK_IDLE.
In-Reply-To: <153292257303.26496.13996008210683917463.stgit@noble>
References: <153292249340.26496.13985174182824912332.stgit@noble>
 <153292257303.26496.13996008210683917463.stgit@noble>
Message-ID: <2657E3CC-BC53-45C9-A6F4-65FEFE55409A@whamcloud.com>

On Jul 29, 2018, at 21:49, NeilBrown <neilb at suse.com> wrote:
> 
> TASK_NOLOAD is not a task state to be use by
> itself, it should only be used together with
> TASK_UNINTERRUPTIBLE, which easily done
> by using TASK_IDLE.
> 
> So convert to TASK_IDLE.
> 
> Signed-off-by: NeilBrown <neilb at suse.com>

Nice to see this infrastructure is available.  I'm no fan of l_wait_event()
complexity, but without TASK_IDLE (only added in 4.13) we had to find a way
for servers to have lots of service threads without a load average of 100
or whatever...

Reviewed-by: Andreas Dilger <adilger at dilger.ca>

> ---
> drivers/staging/lustre/lnet/lnet/lib-eq.c |    4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/staging/lustre/lnet/lnet/lib-eq.c b/drivers/staging/lustre/lnet/lnet/lib-eq.c
> index 8347cc44e47d..f085388895ea 100644
> --- a/drivers/staging/lustre/lnet/lnet/lib-eq.c
> +++ b/drivers/staging/lustre/lnet/lnet/lib-eq.c
> @@ -349,7 +349,7 @@ __must_hold(&the_lnet.ln_eq_wait_lock)
>  * \param timeout Time in jiffies to wait for an event to occur on
>  * one of the EQs. The constant MAX_SCHEDULE_TIMEOUT can be used to indicate an
>  * infinite timeout.
> - * \param interruptible, if true, use TASK_INTERRUPTIBLE, else TASK_NOLOAD
> + * \param interruptible, if true, use TASK_INTERRUPTIBLE, else TASK_IDLE
>  * \param event,which On successful return (1 or -EOVERFLOW), \a event will
>  * hold the next event in the EQs, and \a which will contain the index of the
>  * EQ from which the event was taken.
> @@ -406,7 +406,7 @@ LNetEQPoll(struct lnet_handle_eq *eventqs, int neq, signed long timeout,
> 		 */
> 		wait = lnet_eq_wait_locked(&timeout,
> 					   interruptible ? TASK_INTERRUPTIBLE
> -					   : TASK_NOLOAD);
> +					   : TASK_IDLE);
> 		if (wait < 0) /* no new event */
> 			break;
> 	}
> 
> 

Cheers, Andreas
---
Andreas Dilger
Principal Lustre Architect
Whamcloud







-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 235 bytes
Desc: Message signed with OpenPGP
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180730/b79ac136/attachment.sig>

From adilger at whamcloud.com  Mon Jul 30 21:45:30 2018
From: adilger at whamcloud.com (Andreas Dilger)
Date: Mon, 30 Jul 2018 21:45:30 +0000
Subject: [lustre-devel] [PATCH 0/7] lustre: ad-hoc fixes
In-Reply-To: <153292249340.26496.13985174182824912332.stgit@noble>
References: <153292249340.26496.13985174182824912332.stgit@noble>
Message-ID: <20CED990-BE60-4D19-B842-337CF9605835@whamcloud.com>

On Jul 29, 2018, at 21:49, NeilBrown <neilb at suse.com> wrote:
> 
> There is no real pattern here, just minor tidy-up and minor bug-fix.

I didn't review the 6/7 patch, nor a couple of the list_head changes
since they are more involved in LNet and better reviewed by someone
with more experience in that code.

Cheers, Andreas
---
Andreas Dilger
CTO Whamcloud




-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 235 bytes
Desc: Message signed with OpenPGP
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180730/36f78979/attachment.sig>

From jsimmons at infradead.org  Mon Jul 30 23:16:31 2018
From: jsimmons at infradead.org (James Simmons)
Date: Mon, 30 Jul 2018 19:16:31 -0400
Subject: [lustre-devel] [PATCH 0/3] lustre: llite: rework client mount code
Message-ID: <1532992594-2576-1-git-send-email-jsimmons@infradead.org>

Most of the lustre mount code is stuffed in obd_mount.c. That code
was designed with support for both servers and clients. Move the
code that is specific to just the client to the llite layer which
makes the code simpler.

James Simmons (3):
  lustre: obd: migrate lustre_fill_super() to llite
  lustre: llite: handle registering file system
  lustre: llite: simplify lustre_fill_super()

 .../staging/lustre/lustre/include/lustre_disk.h    |   6 +-
 drivers/staging/lustre/lustre/llite/llite_lib.c    |   3 +
 drivers/staging/lustre/lustre/llite/super25.c      |  98 +++++++++++-
 drivers/staging/lustre/lustre/obdclass/class_obd.c |   6 -
 drivers/staging/lustre/lustre/obdclass/obd_mount.c | 165 ++-------------------
 5 files changed, 115 insertions(+), 163 deletions(-)

-- 
1.8.3.1


From jsimmons at infradead.org  Mon Jul 30 23:16:34 2018
From: jsimmons at infradead.org (James Simmons)
Date: Mon, 30 Jul 2018 19:16:34 -0400
Subject: [lustre-devel] [PATCH 3/3] lustre: llite: simplify
 lustre_fill_super()
In-Reply-To: <1532992594-2576-1-git-send-email-jsimmons@infradead.org>
References: <1532992594-2576-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1532992594-2576-4-git-send-email-jsimmons@infradead.org>

With lustre_fill_super() being client specific we can easily
simplify the code.

Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/lustre/llite/super25.c | 51 ++++++++++-----------------
 1 file changed, 19 insertions(+), 32 deletions(-)

diff --git a/drivers/staging/lustre/lustre/llite/super25.c b/drivers/staging/lustre/lustre/llite/super25.c
index ac8f6f1..1ada5a7 100644
--- a/drivers/staging/lustre/lustre/llite/super25.c
+++ b/drivers/staging/lustre/lustre/llite/super25.c
@@ -113,42 +113,29 @@ int lustre_fill_super(struct super_block *sb, void *lmd2_data, int silent)
 
 	/* Figure out the lmd from the mount options */
 	if (lmd_parse(lmd2_data, lmd)) {
+		lustre_put_lsi(sb);
 		rc = -EINVAL;
-		goto out_put_lsi;
+		goto out;
 	}
 
-	if (lmd_is_client(lmd)) {
-		CDEBUG(D_SUPER | D_CONFIG, "Mounting client %s\n",
-		       lmd->lmd_profile);
-		rc = ptlrpc_inc_ref();
-		if (rc)
-			goto out_put_lsi;
-
-		rc = lustre_start_mgc(sb);
-		if (rc) {
-			/* This will put_lsi and ptlrpc_dec_ref() */
-			lustre_common_put_super(sb);
-			ptlrpc_dec_ref();
-			goto out;
-		}
-
-		/* Connect and start */
-		rc = ll_fill_super(sb);
-
-		/*
-		 * c_f_s will call lustre_common_put_super on failure, otherwise
-		 * c_f_s will have taken another reference to the module
-		 */
-	} else {
-		CERROR("This is client-side-only module, cannot handle server mount.\n");
-		rc = -EINVAL;
+	CDEBUG(D_SUPER | D_CONFIG, "Mounting client %s\n",
+	       lmd->lmd_profile);
+	rc = ptlrpc_inc_ref();
+	if (rc) {
+		lustre_put_lsi(sb);
+		goto out;
 	}
-	/* If error happens in fill_super() call, @lsi will be killed there.
-	 * This is why we do not put it here.
-	 */
-	goto out;
-out_put_lsi:
-	lustre_put_lsi(sb);
+
+	rc = lustre_start_mgc(sb);
+	if (rc) {
+		/* This will put_lsi and ptlrpc_dec_ref() */
+		lustre_common_put_super(sb);
+		ptlrpc_dec_ref();
+		goto out;
+	}
+
+	/* Connect and start. If error happens, @lsi will be killed there */
+	rc = ll_fill_super(sb);
 out:
 	if (rc) {
 		CERROR("Unable to mount %s (%d)\n",
-- 
1.8.3.1


From jsimmons at infradead.org  Mon Jul 30 23:16:32 2018
From: jsimmons at infradead.org (James Simmons)
Date: Mon, 30 Jul 2018 19:16:32 -0400
Subject: [lustre-devel] [PATCH 1/3] lustre: obd: migrate lustre_fill_super()
 to llite
In-Reply-To: <1532992594-2576-1-git-send-email-jsimmons@infradead.org>
References: <1532992594-2576-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1532992594-2576-2-git-send-email-jsimmons@infradead.org>

Currently both obdclass and ptlrpc are built as separate modules
with ptlrpc being a child of obdclass. A recent change placed
ptlrpc_[inc|dec]_ref() into the obdclass code i.e obd_mount.c
which now causes a curricular module dependency. Examining the
code the bulk use of these functions are in lustre_fill_super().
So move lustre_fill_super() to the llite layer where it really
belongs. This change also allows lustre_fill_super() to be
simplified and we can push lustre_fill_super() instead of
ll_fill_super(). Also ptlrpc_dec_ref() needed to be pulled out
of lustre_common_put_super() as well. Once obdclass and ptlrpc
are combined into one module we can restore ptlrpc_dec_rec()
in lustre_common_put_super().

Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 .../staging/lustre/lustre/include/lustre_disk.h    |   9 +-
 drivers/staging/lustre/lustre/llite/llite_lib.c    |   3 +
 drivers/staging/lustre/lustre/llite/super25.c      |  83 +++++++++++++-
 drivers/staging/lustre/lustre/obdclass/obd_mount.c | 125 +++++----------------
 4 files changed, 115 insertions(+), 105 deletions(-)

diff --git a/drivers/staging/lustre/lustre/include/lustre_disk.h b/drivers/staging/lustre/lustre/include/lustre_disk.h
index 1a6d421..772ecc9 100644
--- a/drivers/staging/lustre/lustre/include/lustre_disk.h
+++ b/drivers/staging/lustre/lustre/include/lustre_disk.h
@@ -141,10 +141,13 @@ struct lustre_sb_info {
 
 /* obd_mount.c */
 
+struct lustre_sb_info *lustre_init_lsi(struct super_block *sb);
+int lustre_put_lsi(struct super_block *sb);
+int lmd_parse(char *options, struct lustre_mount_data *lmd);
 int lustre_start_mgc(struct super_block *sb);
-void lustre_register_super_ops(struct module *mod,
-			       int (*cfs)(struct super_block *sb),
-			       void (*ksc)(struct super_block *sb));
+void lustre_register_super_ops(int (*cfs)(struct super_block *sb, void *data,
+					  int silent),
+                               void (*ksc)(struct super_block *sb));
 int lustre_common_put_super(struct super_block *sb);
 
 int mgc_fsname2resid(char *fsname, struct ldlm_res_id *res_id, int type);
diff --git a/drivers/staging/lustre/lustre/llite/llite_lib.c b/drivers/staging/lustre/lustre/llite/llite_lib.c
index 5c8d0fe..87a929c 100644
--- a/drivers/staging/lustre/lustre/llite/llite_lib.c
+++ b/drivers/staging/lustre/lustre/llite/llite_lib.c
@@ -914,6 +914,7 @@ int ll_fill_super(struct super_block *sb)
 	cfg = kzalloc(sizeof(*cfg), GFP_NOFS);
 	if (!cfg) {
 		lustre_common_put_super(sb);
+		ptlrpc_dec_ref();
 		return -ENOMEM;
 	}
 
@@ -926,6 +927,7 @@ int ll_fill_super(struct super_block *sb)
 		module_put(THIS_MODULE);
 		kfree(cfg);
 		lustre_common_put_super(sb);
+		ptlrpc_dec_ref();
 		return -ENOMEM;
 	}
 
@@ -1059,6 +1061,7 @@ void ll_put_super(struct super_block *sb)
 	lsi->lsi_llsbi = NULL;
 
 	lustre_common_put_super(sb);
+	ptlrpc_dec_ref();
 
 	cl_env_cache_purge(~0);
 
diff --git a/drivers/staging/lustre/lustre/llite/super25.c b/drivers/staging/lustre/lustre/llite/super25.c
index d335f29..de43d58 100644
--- a/drivers/staging/lustre/lustre/llite/super25.c
+++ b/drivers/staging/lustre/lustre/llite/super25.c
@@ -83,6 +83,85 @@ struct super_operations lustre_super_operations = {
 };
 MODULE_ALIAS_FS("lustre");
 
+/** This is the entry point for the mount call into Lustre.
+ * This is called when a server or client is mounted,
+ * and this is where we start setting things up.
+ * @param data Mount options (e.g. -o flock,abort_recov)
+ */
+int lustre_fill_super(struct super_block *sb, void *lmd2_data, int silent)
+{
+	struct lustre_mount_data *lmd;
+	struct lustre_sb_info *lsi;
+	int rc;
+
+	CDEBUG(D_SUPER | D_CONFIG | D_VFSTRACE, "VFS Op: sb %p\n", sb);
+
+	lsi = lustre_init_lsi(sb);
+	if (!lsi)
+		return -ENOMEM;
+	lmd = lsi->lsi_lmd;
+
+	/*
+	 * Disable lockdep during mount, because mount locking patterns are
+	 * `special'.
+	 */
+	lockdep_off();
+
+	/*
+	 * LU-639: the obd cleanup of last mount may not finish yet, wait here.
+	 */
+	obd_zombie_barrier();
+
+	/* Figure out the lmd from the mount options */
+	if (lmd_parse(lmd2_data, lmd)) {
+		rc = -EINVAL;
+		goto out_put_lsi;
+	}
+
+	if (lmd_is_client(lmd)) {
+		CDEBUG(D_SUPER | D_CONFIG, "Mounting client %s\n",
+		       lmd->lmd_profile);
+		rc = ptlrpc_inc_ref();
+		if (rc)
+			goto out_put_lsi;
+
+		rc = lustre_start_mgc(sb);
+		if (rc) {
+			/* This will put_lsi and ptlrpc_dec_ref() */
+			lustre_common_put_super(sb);
+			ptlrpc_dec_ref();
+			goto out;
+		}
+
+		/* Connect and start */
+		rc = ll_fill_super(sb);
+
+		/*
+		 * c_f_s will call lustre_common_put_super on failure, otherwise
+		 * c_f_s will have taken another reference to the module
+		 */
+	} else {
+		CERROR("This is client-side-only module, cannot handle server mount.\n");
+		rc = -EINVAL;
+	}
+	/* If error happens in fill_super() call, @lsi will be killed there.
+	 * This is why we do not put it here.
+	 */
+	goto out;
+out_put_lsi:
+	lustre_put_lsi(sb);
+out:
+	if (rc) {
+		CERROR("Unable to mount %s (%d)\n",
+		       s2lsi(sb) ? lmd->lmd_dev : "", rc);
+	} else {
+		CDEBUG(D_SUPER, "Mount %s complete\n",
+		       lmd->lmd_dev);
+	}
+	lockdep_on();
+	return rc;
+}
+
 static int __init lustre_init(void)
 {
 	int rc;
@@ -145,7 +224,7 @@ static int __init lustre_init(void)
 	if (rc != 0)
 		goto out_inode_fini_env;
 
-	lustre_register_super_ops(THIS_MODULE, ll_fill_super, ll_kill_super);
+	lustre_register_super_ops(lustre_fill_super, ll_kill_super);
 	lustre_register_client_process_config(ll_process_config);
 
 	return 0;
@@ -166,7 +245,7 @@ static int __init lustre_init(void)
 
 static void __exit lustre_exit(void)
 {
-	lustre_register_super_ops(NULL, NULL, NULL);
+	lustre_register_super_ops(NULL, NULL);
 	lustre_register_client_process_config(NULL);
 
 	debugfs_remove(llite_root);
diff --git a/drivers/staging/lustre/lustre/obdclass/obd_mount.c b/drivers/staging/lustre/lustre/obdclass/obd_mount.c
index 1bd5613..ac841f4 100644
--- a/drivers/staging/lustre/lustre/obdclass/obd_mount.c
+++ b/drivers/staging/lustre/lustre/obdclass/obd_mount.c
@@ -50,8 +50,8 @@
 #include <uapi/linux/lustre/lustre_param.h>
 
 static DEFINE_SPINLOCK(client_lock);
-static struct module *client_mod;
-static int (*client_fill_super)(struct super_block *sb);
+static int (*client_fill_super)(struct super_block *sb, void *data,
+                                int silent);
 static void (*kill_super_cb)(struct super_block *sb);
 
 /**************** config llog ********************/
@@ -430,6 +430,7 @@ int lustre_start_mgc(struct super_block *sb)
 	kfree(niduuid);
 	return rc;
 }
+EXPORT_SYMBOL(lustre_start_mgc);
 
 static int lustre_stop_mgc(struct super_block *sb)
 {
@@ -507,7 +508,7 @@ static int lustre_stop_mgc(struct super_block *sb)
 
 /***************** lustre superblock **************/
 
-static struct lustre_sb_info *lustre_init_lsi(struct super_block *sb)
+struct lustre_sb_info *lustre_init_lsi(struct super_block *sb)
 {
 	struct lustre_sb_info *lsi;
 
@@ -532,6 +533,7 @@ static struct lustre_sb_info *lustre_init_lsi(struct super_block *sb)
 
 	return lsi;
 }
+EXPORT_SYMBOL(lustre_init_lsi);
 
 static int lustre_free_lsi(struct super_block *sb)
 {
@@ -567,7 +569,7 @@ static int lustre_free_lsi(struct super_block *sb)
 /* The lsi has one reference for every server that is using the disk -
  * e.g. MDT, MGS, and potentially MGC
  */
-static int lustre_put_lsi(struct super_block *sb)
+int lustre_put_lsi(struct super_block *sb)
 {
 	struct lustre_sb_info *lsi = s2lsi(sb);
 
@@ -578,6 +580,7 @@ static int lustre_put_lsi(struct super_block *sb)
 	}
 	return 0;
 }
+EXPORT_SYMBOL(lustre_put_lsi);
 
 /*** SERVER NAME ***
  * <FSNAME><SEPARATOR><TYPE><INDEX>
@@ -686,7 +689,12 @@ int lustre_common_put_super(struct super_block *sb)
 	}
 	/* Drop a ref to the mounted disk */
 	lustre_put_lsi(sb);
-	ptlrpc_dec_ref();
+	/* !!! FIXME !!!!! */
+	/* ptlrpc_dec_ref() should go here but will
+	 * cause curricular module dependencies. Once
+	 * obdclass and ptlrpc are merged into one
+	 * module ptlrpc_dec_ref() can come back here
+	 */
 	return rc;
 }
 EXPORT_SYMBOL(lustre_common_put_super);
@@ -992,7 +1000,7 @@ static int lmd_parse_nidlist(char *buf, char **endh)
  * e.g. mount -v -t lustre -o abort_recov uml1:uml2:/lustre-client /mnt/lustre
  * dev is passed as device=uml1:/lustre by mount.lustre
  */
-static int lmd_parse(char *options, struct lustre_mount_data *lmd)
+int lmd_parse(char *options, struct lustre_mount_data *lmd)
 {
 	char *s1, *s2, *devname = NULL;
 	struct lustre_mount_data *raw = (struct lustre_mount_data *)options;
@@ -1216,106 +1224,16 @@ static int lmd_parse(char *options, struct lustre_mount_data *lmd)
 	CERROR("Bad mount options %s\n", options);
 	return -EINVAL;
 }
-
-/** This is the entry point for the mount call into Lustre.
- * This is called when a server or client is mounted,
- * and this is where we start setting things up.
- * @param data Mount options (e.g. -o flock,abort_recov)
- */
-static int lustre_fill_super(struct super_block *sb, void *lmd2_data, int silent)
-{
-	struct lustre_mount_data *lmd;
-	struct lustre_sb_info *lsi;
-	int rc;
-
-	CDEBUG(D_MOUNT | D_VFSTRACE, "VFS Op: sb %p\n", sb);
-
-	lsi = lustre_init_lsi(sb);
-	if (!lsi)
-		return -ENOMEM;
-	lmd = lsi->lsi_lmd;
-
-	/*
-	 * Disable lockdep during mount, because mount locking patterns are
-	 * `special'.
-	 */
-	lockdep_off();
-
-	/*
-	 * LU-639: the obd cleanup of last mount may not finish yet, wait here.
-	 */
-	obd_zombie_barrier();
-
-	/* Figure out the lmd from the mount options */
-	if (lmd_parse(lmd2_data, lmd)) {
-		rc = -EINVAL;
-		goto out_put_lsi;
-	}
-
-	if (lmd_is_client(lmd)) {
-		bool have_client = false;
-		CDEBUG(D_MOUNT, "Mounting client %s\n", lmd->lmd_profile);
-		if (!client_fill_super)
-			request_module("lustre");
-		rc = ptlrpc_inc_ref();
-		if (rc)
-			goto out_put_lsi;
-		spin_lock(&client_lock);
-		if (client_fill_super && try_module_get(client_mod))
-			have_client = true;
-		spin_unlock(&client_lock);
-		if (!have_client) {
-			LCONSOLE_ERROR_MSG(0x165, "Nothing registered for client mount! Is the 'lustre' module loaded?\n");
-			lustre_put_lsi(sb);
-			rc = -ENODEV;
-		} else {
-			rc = lustre_start_mgc(sb);
-			if (rc) {
-				/* This will put_lsi and ptlrpc_dec_ref() */
-				lustre_common_put_super(sb);
-				goto out;
-			}
-			/* Connect and start */
-			/* (should always be ll_fill_super) */
-			rc = (*client_fill_super)(sb);
-			/*
-			 * c_f_s will call lustre_common_put_super on failure, otherwise
-			 * c_f_s will have taken another reference to the module
-			 */
-			module_put(client_mod);
-		}
-	} else {
-		CERROR("This is client-side-only module, cannot handle server mount.\n");
-		rc = -EINVAL;
-	}
-
-	/* If error happens in fill_super() call, @lsi will be killed there.
-	 * This is why we do not put it here.
-	 */
-	goto out;
-out_put_lsi:
-	lustre_put_lsi(sb);
-out:
-	if (rc) {
-		CERROR("Unable to mount %s (%d)\n",
-		       s2lsi(sb) ? lmd->lmd_dev : "", rc);
-	} else {
-		CDEBUG(D_SUPER, "Mount %s complete\n",
-		       lmd->lmd_dev);
-	}
-	lockdep_on();
-	return rc;
-}
+EXPORT_SYMBOL(lmd_parse);
 
 /* We can't call ll_fill_super by name because it lives in a module that
  * must be loaded after this one.
  */
-void lustre_register_super_ops(struct module *mod,
-			       int (*cfs)(struct super_block *sb),
+void lustre_register_super_ops(int (*cfs)(struct super_block *sb, void *data,
+					  int silent),
 			       void (*ksc)(struct super_block *sb))
 {
 	spin_lock(&client_lock);
-	client_mod = mod;
 	client_fill_super = cfs;
 	kill_super_cb = ksc;
 	spin_unlock(&client_lock);
@@ -1326,7 +1244,14 @@ void lustre_register_super_ops(struct module *mod,
 static struct dentry *lustre_mount(struct file_system_type *fs_type, int flags,
 				   const char *devname, void *data)
 {
-	return mount_nodev(fs_type, flags, data, lustre_fill_super);
+	struct dentry *root;
+
+	spin_lock(&client_lock);
+	if (!client_fill_super)
+		request_module("lustre");
+	root = mount_nodev(fs_type, flags, data, client_fill_super);
+	spin_unlock(&client_lock);
+	return root;
 }
 
 static void lustre_kill_super(struct super_block *sb)
-- 
1.8.3.1


From jsimmons at infradead.org  Mon Jul 30 23:16:33 2018
From: jsimmons at infradead.org (James Simmons)
Date: Mon, 30 Jul 2018 19:16:33 -0400
Subject: [lustre-devel] [PATCH 2/3] lustre: llite: handle registering file
 system
In-Reply-To: <1532992594-2576-1-git-send-email-jsimmons@infradead.org>
References: <1532992594-2576-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1532992594-2576-3-git-send-email-jsimmons@infradead.org>

Move the code that registers struct file_system_type for lustre
with the VFS layer to the llite layer where it belongs. This
removes the ugly function pointer passing between obdclass and
llite.

Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 .../staging/lustre/lustre/include/lustre_disk.h    |  3 -
 drivers/staging/lustre/lustre/llite/super25.c      | 34 +++++++++++-
 drivers/staging/lustre/lustre/obdclass/class_obd.c |  6 --
 drivers/staging/lustre/lustre/obdclass/obd_mount.c | 64 ----------------------
 4 files changed, 31 insertions(+), 76 deletions(-)

diff --git a/drivers/staging/lustre/lustre/include/lustre_disk.h b/drivers/staging/lustre/lustre/include/lustre_disk.h
index 772ecc9..d5fadde 100644
--- a/drivers/staging/lustre/lustre/include/lustre_disk.h
+++ b/drivers/staging/lustre/lustre/include/lustre_disk.h
@@ -145,9 +145,6 @@ struct lustre_sb_info {
 int lustre_put_lsi(struct super_block *sb);
 int lmd_parse(char *options, struct lustre_mount_data *lmd);
 int lustre_start_mgc(struct super_block *sb);
-void lustre_register_super_ops(int (*cfs)(struct super_block *sb, void *data,
-					  int silent),
-                               void (*ksc)(struct super_block *sb));
 int lustre_common_put_super(struct super_block *sb);
 
 int mgc_fsname2resid(char *fsname, struct ldlm_res_id *res_id, int type);
diff --git a/drivers/staging/lustre/lustre/llite/super25.c b/drivers/staging/lustre/lustre/llite/super25.c
index de43d58..ac8f6f1 100644
--- a/drivers/staging/lustre/lustre/llite/super25.c
+++ b/drivers/staging/lustre/lustre/llite/super25.c
@@ -81,7 +81,6 @@ struct super_operations lustre_super_operations = {
 	.remount_fs    = ll_remount_fs,
 	.show_options  = ll_show_options,
 };
-MODULE_ALIAS_FS("lustre");
 
 /** This is the entry point for the mount call into Lustre.
  * This is called when a server or client is mounted,
@@ -162,6 +161,30 @@ int lustre_fill_super(struct super_block *sb, void *lmd2_data, int silent)
 	return rc;
 }
 
+/***************** FS registration ******************/
+static struct dentry *lustre_mount(struct file_system_type *fs_type, int flags,
+				   const char *devname, void *data)
+{
+	return mount_nodev(fs_type, flags, data, lustre_fill_super);
+}
+
+static void lustre_kill_super(struct super_block *sb)
+{
+	ll_kill_super(sb);
+	kill_anon_super(sb);
+}
+
+/** Register the "lustre" fs type
+ */
+static struct file_system_type lustre_fs_type = {
+	.owner		= THIS_MODULE,
+	.name		= "lustre",
+	.mount		= lustre_mount,
+	.kill_sb	= lustre_kill_super,
+	.fs_flags	= FS_RENAME_DOES_D_MOVE,
+};
+MODULE_ALIAS_FS("lustre");
+
 static int __init lustre_init(void)
 {
 	int rc;
@@ -224,11 +247,16 @@ static int __init lustre_init(void)
 	if (rc != 0)
 		goto out_inode_fini_env;
 
-	lustre_register_super_ops(lustre_fill_super, ll_kill_super);
+	rc = register_filesystem(&lustre_fs_type);
+	if (rc)
+		goto out_xattr_cache;
+
 	lustre_register_client_process_config(ll_process_config);
 
 	return 0;
 
+out_xattr_cache:
+	ll_xattr_fini();
 out_inode_fini_env:
 	cl_env_put(cl_inode_fini_env, &cl_inode_fini_refcheck);
 out_vvp:
@@ -245,8 +273,8 @@ static int __init lustre_init(void)
 
 static void __exit lustre_exit(void)
 {
-	lustre_register_super_ops(NULL, NULL);
 	lustre_register_client_process_config(NULL);
+	unregister_filesystem(&lustre_fs_type);
 
 	debugfs_remove(llite_root);
 	kset_unregister(llite_kset);
diff --git a/drivers/staging/lustre/lustre/obdclass/class_obd.c b/drivers/staging/lustre/lustre/obdclass/class_obd.c
index 81a4c66..cdaf729 100644
--- a/drivers/staging/lustre/lustre/obdclass/class_obd.c
+++ b/drivers/staging/lustre/lustre/obdclass/class_obd.c
@@ -510,18 +510,12 @@ static int __init obdclass_init(void)
 		return err;
 
 	err = llog_info_init();
-	if (err)
-		return err;
-
-	err = lustre_register_fs();
 
 	return err;
 }
 
 static void obdclass_exit(void)
 {
-	lustre_unregister_fs();
-
 	misc_deregister(&obd_psdev);
 	llog_info_fini();
 	cl_global_fini();
diff --git a/drivers/staging/lustre/lustre/obdclass/obd_mount.c b/drivers/staging/lustre/lustre/obdclass/obd_mount.c
index ac841f4..b84bca4 100644
--- a/drivers/staging/lustre/lustre/obdclass/obd_mount.c
+++ b/drivers/staging/lustre/lustre/obdclass/obd_mount.c
@@ -49,11 +49,6 @@
 #include <lustre_disk.h>
 #include <uapi/linux/lustre/lustre_param.h>
 
-static DEFINE_SPINLOCK(client_lock);
-static int (*client_fill_super)(struct super_block *sb, void *data,
-                                int silent);
-static void (*kill_super_cb)(struct super_block *sb);
-
 /**************** config llog ********************/
 
 /** Get a config log from the MGS and process it.
@@ -1225,62 +1220,3 @@ int lmd_parse(char *options, struct lustre_mount_data *lmd)
 	return -EINVAL;
 }
 EXPORT_SYMBOL(lmd_parse);
-
-/* We can't call ll_fill_super by name because it lives in a module that
- * must be loaded after this one.
- */
-void lustre_register_super_ops(int (*cfs)(struct super_block *sb, void *data,
-					  int silent),
-			       void (*ksc)(struct super_block *sb))
-{
-	spin_lock(&client_lock);
-	client_fill_super = cfs;
-	kill_super_cb = ksc;
-	spin_unlock(&client_lock);
-}
-EXPORT_SYMBOL(lustre_register_super_ops);
-
-/***************** FS registration ******************/
-static struct dentry *lustre_mount(struct file_system_type *fs_type, int flags,
-				   const char *devname, void *data)
-{
-	struct dentry *root;
-
-	spin_lock(&client_lock);
-	if (!client_fill_super)
-		request_module("lustre");
-	root = mount_nodev(fs_type, flags, data, client_fill_super);
-	spin_unlock(&client_lock);
-	return root;
-}
-
-static void lustre_kill_super(struct super_block *sb)
-{
-	struct lustre_sb_info *lsi = s2lsi(sb);
-
-	if (kill_super_cb && lsi)
-		(*kill_super_cb)(sb);
-
-	kill_anon_super(sb);
-}
-
-/** Register the "lustre" fs type
- */
-static struct file_system_type lustre_fs_type = {
-	.owner		= THIS_MODULE,
-	.name		= "lustre",
-	.mount		= lustre_mount,
-	.kill_sb	= lustre_kill_super,
-	.fs_flags	= FS_RENAME_DOES_D_MOVE,
-};
-MODULE_ALIAS_FS("lustre");
-
-int lustre_register_fs(void)
-{
-	return register_filesystem(&lustre_fs_type);
-}
-
-int lustre_unregister_fs(void)
-{
-	return unregister_filesystem(&lustre_fs_type);
-}
-- 
1.8.3.1


From jsimmons at infradead.org  Tue Jul 31 02:16:38 2018
From: jsimmons at infradead.org (James Simmons)
Date: Tue, 31 Jul 2018 03:16:38 +0100 (BST)
Subject: [lustre-devel] Debian packaging reviews (DKMS)
In-Reply-To: <698AD41C-8716-437B-B7E1-4ECEECC1127D@ddn.com>
References: <c48157e1ce854fb2930a19eca1e7232451e9953a.camel@informatik.uni-hamburg.de>
 <698AD41C-8716-437B-B7E1-4ECEECC1127D@ddn.com>
Message-ID: <alpine.LFD.2.21.1807310314150.20049@casper.infradead.org>


> Michael
> 
> I think that there is a growing community of people using Ubuntu and there have been efforts focusing around that. 
> See - https://review.whamcloud.com/#/c/32613/

That patch has landed. I rebased your patch Michael.

> Peter
> 
> On 2018-07-18, 6:51 AM, "lustre-devel on behalf of Michael Kuhn" <lustre-devel-bounces at lists.lustre.org on behalf of michael.kuhn at informatik.uni-hamburg.de> wrote:
> 
>     Hi all,
>     
>     I have recently rebased my change for supporting DKMS Debian/Ubuntu
>     packages: https://review.whamcloud.com/#/c/25328/
>     
>     I know that not a lot of people use Lustre on Debian/Ubuntu but it
>     would be nice to get some reviews for this. We have been carrying this
>     change locally for more than a year now and it would be great to
>     finally get this upstream. :-)
>     
>     
>     Best regards,
>     Michael
>     _______________________________________________
>     lustre-devel mailing list
>     lustre-devel at lists.lustre.org
>     http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org
>     
> 
> _______________________________________________
> lustre-devel mailing list
> lustre-devel at lists.lustre.org
> http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org
> 

From jsimmons at infradead.org  Tue Jul 31 02:25:52 2018
From: jsimmons at infradead.org (James Simmons)
Date: Mon, 30 Jul 2018 22:25:52 -0400
Subject: [lustre-devel] [PATCH 00/31] lustre: missing fixes and cleanups
 from lustre 2.10
Message-ID: <1533003983-29311-1-git-send-email-jsimmons@infradead.org>

This covers all the missing patches landed from the start of the
lustre 2.10 development cycle until right before the PFL feature
landed. Several bug fixes as well as cleanups. This is based on
top of the recent list patches as well as my mount code for llite
patch series.

Abrarahmed Momin (1):
  lustre: llite: Remove OBD_FAIL_OSC_CONNECT_CKSUM

Amir Shehata (1):
  lustre: lnet: Fix route hops print

Andreas Dilger (2):
  lustre: libcfs: reduce libcfs checksum speed test time
  lustre: obdclass: improve missing operation message

Andriy Skulysh (1):
  lustre: osc: hung in osc_destroy()

Bruno Faccini (3):
  lustre: obdclass: obdclass module cleanup upon load error
  lustre: obdclass: handle early requests vs CT registering
  lustre: llite: handle client racy case during create

Chris Horn (1):
  lustre: llite: Return -ERESTARTSYS in range_lock()

Di Wang (1):
  lustre: lmv: honour the specified stripe index

Dmitry Eremin (1):
  lustre: clio: remove unused members from struct cl_thread_info

Fan Yong (1):
  lustre: fid: race between client_fid_fini and seq_client_flush

Gu Zheng (1):
  lustre: libcfs: avoid overflow of crypto bandwidth calculation

Hongchao Zhang (1):
  lustre: mgc: relate sptlrpc & param to MGC

James Simmons (1):
  lustre: docs: update TODO file

Jinshan Xiong (1):
  lustre: llite: ignore layout for ll_writepages()

John L. Hammond (5):
  lustre: llite: return small device numbers for compat stat()
  lustre: obd: remove OBD_NOTIFY_SYNC{,_NONBLOCK}
  lustre: obd: remove OBD_NOTIFY_CONFIG
  lustre: obdclass: use static initializer macros where possible
  lustre: obd: remove unused data parameter from obd_notify()

Niu Yawei (4):
  lustre: llite: don't zero timestamps internally
  lustre: config: don't attach sub logs for LWP
  lustre: llite: buggy special handling on MULTIMODRPCS
  lustre: config: move config types into lustre_idl.h

Patrick Farrell (2):
  lustre: osc: Send RPCs when extents are full
  lustre: llite: reduce jobstats race window

Sebastien Buisson (1):
  lustre: obd: add 'network' client mount option

Sonia Sharma (1):
  lustre: lnet: removal of obsolete LNDs

Steve Guminski (2):
  lustre: obd: change positional struct initializers to C99
  lustre: lnet: change positional struct initializers to C99

 drivers/staging/lustre/TODO                        | 51 +-----------
 .../lustre/include/linux/libcfs/libcfs_crypto.h    |  4 +-
 .../lustre/include/uapi/linux/lnet/nidstr.h        | 18 ++---
 .../lustre/include/uapi/linux/lustre/lustre_idl.h  | 10 ++-
 drivers/staging/lustre/lnet/libcfs/linux-crypto.c  | 33 ++++++--
 drivers/staging/lustre/lnet/lnet/api-ni.c          |  9 +--
 drivers/staging/lustre/lnet/lnet/lo.c              | 20 +++--
 drivers/staging/lustre/lnet/lnet/router_proc.c     |  2 +-
 drivers/staging/lustre/lnet/selftest/framework.c   |  2 +-
 drivers/staging/lustre/lustre/fid/fid_request.c    | 21 +++--
 drivers/staging/lustre/lustre/include/cl_object.h  |  1 -
 drivers/staging/lustre/lustre/include/lu_object.h  |  6 --
 .../staging/lustre/lustre/include/lustre_disk.h    |  1 +
 drivers/staging/lustre/lustre/include/lustre_net.h |  3 +-
 drivers/staging/lustre/lustre/include/obd.h        | 10 +--
 drivers/staging/lustre/lustre/include/obd_class.h  | 66 +++++++---------
 .../staging/lustre/lustre/include/obd_support.h    |  4 +-
 drivers/staging/lustre/lustre/ldlm/ldlm_lib.c      | 42 ++++++----
 drivers/staging/lustre/lustre/llite/file.c         | 16 +++-
 drivers/staging/lustre/lustre/llite/lcommon_misc.c | 12 ++-
 .../staging/lustre/lustre/llite/llite_internal.h   | 10 ++-
 drivers/staging/lustre/lustre/llite/llite_lib.c    | 26 +++----
 drivers/staging/lustre/lustre/llite/namei.c        |  9 +--
 drivers/staging/lustre/lustre/llite/range_lock.c   |  2 +-
 drivers/staging/lustre/lustre/llite/rw.c           | 15 ++--
 drivers/staging/lustre/lustre/lmv/lmv_obd.c        | 51 +++++-------
 drivers/staging/lustre/lustre/lov/lov_obd.c        | 63 ++++-----------
 drivers/staging/lustre/lustre/mdc/mdc_request.c    | 17 ++--
 drivers/staging/lustre/lustre/mgc/mgc_request.c    | 47 ++++++-----
 .../staging/lustre/lustre/obdclass/cl_internal.h   | 45 +----------
 drivers/staging/lustre/lustre/obdclass/cl_io.c     | 21 +----
 drivers/staging/lustre/lustre/obdclass/cl_object.c | 16 +---
 drivers/staging/lustre/lustre/obdclass/class_obd.c | 91 +++++++++++++++-------
 drivers/staging/lustre/lustre/obdclass/genops.c    |  5 +-
 .../staging/lustre/lustre/obdclass/kernelcomm.c    |  8 ++
 drivers/staging/lustre/lustre/obdclass/lu_object.c | 15 ----
 .../staging/lustre/lustre/obdclass/lustre_peer.c   | 16 +---
 .../staging/lustre/lustre/obdclass/obd_config.c    | 46 ++++++++++-
 drivers/staging/lustre/lustre/obdclass/obd_mount.c | 38 +++++++++
 drivers/staging/lustre/lustre/osc/osc_cache.c      | 45 +++++++----
 .../staging/lustre/lustre/osc/osc_cl_internal.h    |  2 +-
 drivers/staging/lustre/lustre/osc/osc_object.c     |  4 +-
 drivers/staging/lustre/lustre/osc/osc_request.c    | 18 +++--
 drivers/staging/lustre/lustre/ptlrpc/client.c      |  4 +-
 drivers/staging/lustre/lustre/ptlrpc/events.c      |  4 +
 drivers/staging/lustre/lustre/ptlrpc/import.c      |  5 +-
 drivers/staging/lustre/lustre/ptlrpc/wiretest.c    | 85 ++++++++++++++++++++
 47 files changed, 561 insertions(+), 478 deletions(-)

-- 
1.8.3.1


From jsimmons at infradead.org  Tue Jul 31 02:25:56 2018
From: jsimmons at infradead.org (James Simmons)
Date: Mon, 30 Jul 2018 22:25:56 -0400
Subject: [lustre-devel] [PATCH 04/31] lustre: lmv: honour the specified
 stripe index
In-Reply-To: <1533003983-29311-1-git-send-email-jsimmons@infradead.org>
References: <1533003983-29311-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1533003983-29311-5-git-send-email-jsimmons@infradead.org>

From: Di Wang <di.wang at intel.com>

when creating the striped directory, specified
stripe index should always be used even the parent
has default stripe index.

Signed-off-by: Di Wang <di.wang at intel.com>
WC-id: https://jira.whamcloud.com/browse/LU-8994
Reviewed-on: https://review.whamcloud.com/24777
Reviewed-by: Andreas Dilger <adilger at whamcloud.com>
Reviewed-by: Fan Yong <fan.yong at intel.com>
Reviewed-by: Lai Siyao <lai.siyao at whamcloud.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/lustre/lmv/lmv_obd.c | 44 +++++++++++------------------
 1 file changed, 17 insertions(+), 27 deletions(-)

diff --git a/drivers/staging/lustre/lustre/lmv/lmv_obd.c b/drivers/staging/lustre/lustre/lmv/lmv_obd.c
index 3da5a0a..bbb1ddf 100644
--- a/drivers/staging/lustre/lustre/lmv/lmv_obd.c
+++ b/drivers/staging/lustre/lustre/lmv/lmv_obd.c
@@ -1128,7 +1128,8 @@ static int lmv_iocontrol(unsigned int cmd, struct obd_export *exp,
 static int lmv_placement_policy(struct obd_device *obd,
 				struct md_op_data *op_data, u32 *mds)
 {
-	struct lmv_obd	  *lmv = &obd->u.lmv;
+	struct lmv_obd *lmv = &obd->u.lmv;
+	struct lmv_user_md *lum;
 
 	LASSERT(mds);
 
@@ -1137,34 +1138,23 @@ static int lmv_placement_policy(struct obd_device *obd,
 		return 0;
 	}
 
-	if (op_data->op_default_stripe_offset != -1) {
-		*mds = op_data->op_default_stripe_offset;
-		return 0;
-	}
-
-	/**
-	 * If stripe_offset is provided during setdirstripe
-	 * (setdirstripe -i xx), xx MDS will be chosen.
+	lum = op_data->op_data;
+	/* Choose MDS by
+	 * 1. See if the stripe offset is specified by lum.
+	 * 2. Then check if there is default stripe offset.
+	 * 3. Finally choose MDS by name hash if the parent
+	 *    is striped directory. (see lmv_locate_mds()).
 	 */
-	if (op_data->op_cli_flags & CLI_SET_MEA && op_data->op_data) {
-		struct lmv_user_md *lum;
-
-		lum = op_data->op_data;
-		if (le32_to_cpu(lum->lum_stripe_offset) != (__u32)-1) {
-			*mds = le32_to_cpu(lum->lum_stripe_offset);
-		} else {
-			/*
-			 * -1 means default, which will be in the same MDT with
-			 * the stripe
-			 */
-			*mds = op_data->op_mds;
-			lum->lum_stripe_offset = cpu_to_le32(op_data->op_mds);
-		}
+	if (op_data->op_cli_flags & CLI_SET_MEA && lum &&
+	    le32_to_cpu(lum->lum_stripe_offset) != (u32)-1) {
+		*mds = le32_to_cpu(lum->lum_stripe_offset);
+	} else if (op_data->op_default_stripe_offset != (u32)-1) {
+		*mds = op_data->op_default_stripe_offset;
+		op_data->op_mds = *mds;
+		/* Correct the stripe offset in lum */
+		if (lum)
+			lum->lum_stripe_offset = cpu_to_le32(*mds);
 	} else {
-		/*
-		 * Allocate new fid on target according to operation type and
-		 * parent home mds.
-		 */
 		*mds = op_data->op_mds;
 	}
 
-- 
1.8.3.1


From jsimmons at infradead.org  Tue Jul 31 02:25:54 2018
From: jsimmons at infradead.org (James Simmons)
Date: Mon, 30 Jul 2018 22:25:54 -0400
Subject: [lustre-devel] [PATCH 02/31] lustre: obd: add 'network' client
 mount option
In-Reply-To: <1533003983-29311-1-git-send-email-jsimmons@infradead.org>
References: <1533003983-29311-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1533003983-29311-3-git-send-email-jsimmons@infradead.org>

From: Sebastien Buisson <sbuisson at ddn.com>

Add a 'network' mount option on client side. All connections made by
the client must be on the LNet network specified in the 'network'
option.

This option can be useful in case of several Lustre client mount
points on the same node, with each mount point using a different
network. It is also interesting when running Lustre clients from
containers, by restricting each container to a specific network.

This new option is added by tampering with two config commands:
- setup: add a fourth parameter, which is the net to restrict
  connections to. This parameter will be passed down to
  ptlrpc_uuid_to_peer() so that client only connects to peers on the
  restricted network.
- add_conn: skip this command if uuid to connect to is not on
  restricted network.

Signed-off-by: Sebastien Buisson <sbuisson at ddn.com>
WC-bug-id: https://jira.whamcloud.com/browse/LU-7845
Reviewed-on: https://review.whamcloud.com/19792
Reviewed-by: Doug Oucharek <dougso at me.com>
Reviewed-by: Li Xi <lixi at ddn.com>
Reviewed-by: Oleg Drokin <green at whamcloud.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 .../staging/lustre/lustre/include/lustre_disk.h    |  1 +
 drivers/staging/lustre/lustre/include/lustre_net.h |  3 +-
 drivers/staging/lustre/lustre/ldlm/ldlm_lib.c      | 28 ++++++++++++++-
 .../staging/lustre/lustre/obdclass/obd_config.c    | 42 ++++++++++++++++++++++
 drivers/staging/lustre/lustre/obdclass/obd_mount.c | 38 ++++++++++++++++++++
 drivers/staging/lustre/lustre/ptlrpc/client.c      |  4 ++-
 drivers/staging/lustre/lustre/ptlrpc/events.c      |  4 +++
 7 files changed, 117 insertions(+), 3 deletions(-)

diff --git a/drivers/staging/lustre/lustre/include/lustre_disk.h b/drivers/staging/lustre/lustre/include/lustre_disk.h
index d5fadde..103eb6a 100644
--- a/drivers/staging/lustre/lustre/include/lustre_disk.h
+++ b/drivers/staging/lustre/lustre/include/lustre_disk.h
@@ -87,6 +87,7 @@ struct lustre_mount_data {
 	__u32	*lmd_exclude;	/* array of OSTs to ignore */
 	char	*lmd_mgs;	/* MGS nid */
 	char	*lmd_osd_type;	/* OSD type */
+	char    *lmd_nidnet;	/* network to restrict this client to */
 };
 
 #define LMD_FLG_SERVER		0x0001	/* Mounting a server */
diff --git a/drivers/staging/lustre/lustre/include/lustre_net.h b/drivers/staging/lustre/lustre/include/lustre_net.h
index dcad90b..361b897 100644
--- a/drivers/staging/lustre/lustre/include/lustre_net.h
+++ b/drivers/staging/lustre/lustre/include/lustre_net.h
@@ -1812,7 +1812,8 @@ static inline int ptlrpc_client_bulk_active(struct ptlrpc_request *req)
 
 void ptlrpc_init_client(int req_portal, int rep_portal, char *name,
 			struct ptlrpc_client *);
-struct ptlrpc_connection *ptlrpc_uuid_to_connection(struct obd_uuid *uuid);
+struct ptlrpc_connection *ptlrpc_uuid_to_connection(struct obd_uuid *uuid,
+						    lnet_nid_t nid4refnet);
 
 int ptlrpc_queue_wait(struct ptlrpc_request *req);
 int ptlrpc_replay_req(struct ptlrpc_request *req);
diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_lib.c b/drivers/staging/lustre/lustre/ldlm/ldlm_lib.c
index 07baea7..5da8c88 100644
--- a/drivers/staging/lustre/lustre/ldlm/ldlm_lib.c
+++ b/drivers/staging/lustre/lustre/ldlm/ldlm_lib.c
@@ -55,6 +55,7 @@ static int import_set_conn(struct obd_import *imp, struct obd_uuid *uuid,
 {
 	struct ptlrpc_connection *ptlrpc_conn;
 	struct obd_import_conn *imp_conn = NULL, *item;
+	lnet_nid_t nid4refnet = LNET_NID_ANY;
 	int rc = 0;
 
 	if (!create && !priority) {
@@ -62,7 +63,12 @@ static int import_set_conn(struct obd_import *imp, struct obd_uuid *uuid,
 		return -EINVAL;
 	}
 
-	ptlrpc_conn = ptlrpc_uuid_to_connection(uuid);
+	if (imp->imp_connection &&
+	    imp->imp_connection->c_remote_uuid.uuid[0] == 0)
+		/* nid4refnet is used to restrict network connections */
+		nid4refnet = imp->imp_connection->c_self;
+
+	ptlrpc_conn = ptlrpc_uuid_to_connection(uuid, nid4refnet);
 	if (!ptlrpc_conn) {
 		CDEBUG(D_HA, "can't find connection %s\n", uuid->uuid);
 		return -ENOENT;
@@ -233,6 +239,7 @@ void client_destroy_import(struct obd_import *imp)
  * 1 - client UUID
  * 2 - server UUID
  * 3 - inactive-on-startup
+ * 4 - restrictive net
  */
 int client_obd_setup(struct obd_device *obddev, struct lustre_cfg *lcfg)
 {
@@ -242,6 +249,10 @@ int client_obd_setup(struct obd_device *obddev, struct lustre_cfg *lcfg)
 	int rq_portal, rp_portal, connect_op;
 	char *name = obddev->obd_type->typ_name;
 	enum ldlm_ns_type ns_type = LDLM_NS_TYPE_UNKNOWN;
+	struct ptlrpc_connection fake_conn = {
+		.c_self = 0,
+		.c_remote_uuid.uuid[0] = 0
+	};
 	int rc;
 
 	/* In a more perfect world, we would hang a ptlrpc_client off of
@@ -412,11 +423,26 @@ int client_obd_setup(struct obd_device *obddev, struct lustre_cfg *lcfg)
 	       LUSTRE_CFG_BUFLEN(lcfg, 1));
 	class_import_put(imp);
 
+	if (lustre_cfg_buf(lcfg, 4)) {
+		u32 refnet = libcfs_str2net(lustre_cfg_string(lcfg, 4));
+
+		if (refnet == LNET_NIDNET(LNET_NID_ANY)) {
+			rc = -EINVAL;
+			CERROR("%s: bad mount option 'network=%s': rc = %d\n",
+			       obddev->obd_name, lustre_cfg_string(lcfg, 4),
+			       rc);
+			goto err_import;
+		}
+		fake_conn.c_self = LNET_MKNID(refnet, 0);
+		imp->imp_connection = &fake_conn;
+	}
+
 	rc = client_import_add_conn(imp, &server_uuid, 1);
 	if (rc) {
 		CERROR("can't add initial connection\n");
 		goto err_import;
 	}
+	imp->imp_connection = NULL;
 
 	cli->cl_import = imp;
 	/* cli->cl_max_mds_easize updated by mdc_init_ea_size() */
diff --git a/drivers/staging/lustre/lustre/obdclass/obd_config.c b/drivers/staging/lustre/lustre/obdclass/obd_config.c
index cfcd17e..6d47435 100644
--- a/drivers/staging/lustre/lustre/obdclass/obd_config.c
+++ b/drivers/staging/lustre/lustre/obdclass/obd_config.c
@@ -40,6 +40,8 @@
 #include <linux/uaccess.h>
 #include <linux/string.h>
 
+#include <uapi/linux/lustre/lustre_idl.h>
+#include <lustre_disk.h>
 #include <uapi/linux/lustre/lustre_ioctl.h>
 #include <llog_swab.h>
 #include <lprocfs_status.h>
@@ -1280,6 +1282,7 @@ int class_config_llog_handler(const struct lu_env *env,
 				lcfg->lcfg_command = LCFG_LOV_ADD_INA;
 		}
 
+		lustre_cfg_bufs_reset(&bufs, NULL);
 		lustre_cfg_bufs_init(&bufs, lcfg);
 
 		if (clli && clli->cfg_instance &&
@@ -1323,6 +1326,45 @@ int class_config_llog_handler(const struct lu_env *env,
 						   clli->cfg_obdname);
 		}
 
+		/* Add net info to setup command
+		 * if given on command line.
+		 * So config log will be:
+		 * [0]: client name
+		 * [1]: client UUID
+		 * [2]: server UUID
+		 * [3]: inactive-on-startup
+		 * [4]: restrictive net
+		 */
+		if (clli && clli->cfg_sb && s2lsi(clli->cfg_sb)) {
+			struct lustre_sb_info *lsi = s2lsi(clli->cfg_sb);
+			char *nidnet = lsi->lsi_lmd->lmd_nidnet;
+
+			if (lcfg->lcfg_command == LCFG_SETUP &&
+			    lcfg->lcfg_bufcount != 2 && nidnet) {
+				CDEBUG(D_CONFIG,
+				       "Adding net %s info to setup command for client %s\n",
+				       nidnet, lustre_cfg_string(lcfg, 0));
+				lustre_cfg_bufs_set_string(&bufs, 4, nidnet);
+			}
+		}
+
+		/* Skip add_conn command if uuid is not on restricted net */
+		if (clli && clli->cfg_sb && s2lsi(clli->cfg_sb)) {
+			struct lustre_sb_info *lsi = s2lsi(clli->cfg_sb);
+			char *uuid_str = lustre_cfg_string(lcfg, 1);
+
+			if (lcfg->lcfg_command == LCFG_ADD_CONN &&
+			    lsi->lsi_lmd->lmd_nidnet &&
+			    LNET_NIDNET(libcfs_str2nid(uuid_str)) !=
+			    libcfs_str2net(lsi->lsi_lmd->lmd_nidnet)) {
+				CDEBUG(D_CONFIG, "skipping add_conn for %s\n",
+				       uuid_str);
+				rc = 0;
+				/* No processing! */
+				break;
+			}
+		}
+
 		lcfg_len = lustre_cfg_len(bufs.lcfg_bufcount, bufs.lcfg_buflen);
 		lcfg_new = kzalloc(lcfg_len, GFP_NOFS);
 		if (!lcfg_new) {
diff --git a/drivers/staging/lustre/lustre/obdclass/obd_mount.c b/drivers/staging/lustre/lustre/obdclass/obd_mount.c
index b84bca4..1d88e8c 100644
--- a/drivers/staging/lustre/lustre/obdclass/obd_mount.c
+++ b/drivers/staging/lustre/lustre/obdclass/obd_mount.c
@@ -550,6 +550,7 @@ static int lustre_free_lsi(struct super_block *sb)
 		kfree(lsi->lsi_lmd->lmd_mgs);
 		kfree(lsi->lsi_lmd->lmd_osd_type);
 		kfree(lsi->lsi_lmd->lmd_params);
+		kfree(lsi->lsi_lmd->lmd_nidnet);
 
 		kfree(lsi->lsi_lmd);
 	}
@@ -827,6 +828,27 @@ static int lmd_parse_mgssec(struct lustre_mount_data *lmd, char *ptr)
 	return 0;
 }
 
+static int lmd_parse_network(struct lustre_mount_data *lmd, char *ptr)
+{
+	char *tail;
+	int length;
+
+	kfree(lmd->lmd_nidnet);
+	lmd->lmd_nidnet = NULL;
+
+	tail = strchr(ptr, ',');
+	if (!tail)
+		length = strlen(ptr);
+	else
+		length = tail - ptr;
+
+	lmd->lmd_nidnet = kstrndup(ptr, length, GFP_KERNEL);
+	if (!lmd->lmd_nidnet)
+		return -ENOMEM;
+
+	return 0;
+}
+
 static int lmd_parse_string(char **handle, char *ptr)
 {
 	char   *tail;
@@ -1146,6 +1168,11 @@ int lmd_parse(char *options, struct lustre_mount_data *lmd)
 			 */
 			*s1 = '\0';
 			break;
+		} else if (strncmp(s1, "network=", 8) == 0) {
+			rc = lmd_parse_network(lmd, s1 + 8);
+			if (rc)
+				goto invalid;
+			clear++;
 		}
 
 		/* Find next opt */
@@ -1192,6 +1219,17 @@ int lmd_parse(char *options, struct lustre_mount_data *lmd)
 			if (!lmd->lmd_fileset)
 				return -ENOMEM;
 		}
+	} else {
+		/* server mount */
+		if (lmd->lmd_nidnet) {
+			/* 'network=' mount option forbidden for server */
+			kfree(lmd->lmd_nidnet);
+			lmd->lmd_nidnet = NULL;
+			rc = -EINVAL;
+			CERROR("%s: option 'network=' not allowed for Lustre servers: rc = %d\n",
+			       devname, rc);
+			return rc;
+		}
 	}
 
 	/* Freed in lustre_free_lsi */
diff --git a/drivers/staging/lustre/lustre/ptlrpc/client.c b/drivers/staging/lustre/lustre/ptlrpc/client.c
index 91dd098..4bf26a4 100644
--- a/drivers/staging/lustre/lustre/ptlrpc/client.c
+++ b/drivers/staging/lustre/lustre/ptlrpc/client.c
@@ -77,7 +77,8 @@ void ptlrpc_init_client(int req_portal, int rep_portal, char *name,
 /**
  * Return PortalRPC connection for remote uud \a uuid
  */
-struct ptlrpc_connection *ptlrpc_uuid_to_connection(struct obd_uuid *uuid)
+struct ptlrpc_connection *ptlrpc_uuid_to_connection(struct obd_uuid *uuid,
+						    lnet_nid_t nid4refnet)
 {
 	struct ptlrpc_connection *c;
 	lnet_nid_t self;
@@ -89,6 +90,7 @@ struct ptlrpc_connection *ptlrpc_uuid_to_connection(struct obd_uuid *uuid)
 	 * before accessing its values.
 	 * coverity[uninit_use_in_call]
 	 */
+	peer.nid = nid4refnet;
 	err = ptlrpc_uuid_to_peer(uuid, &peer, &self);
 	if (err != 0) {
 		CNETERR("cannot find peer %s!\n", uuid->uuid);
diff --git a/drivers/staging/lustre/lustre/ptlrpc/events.c b/drivers/staging/lustre/lustre/ptlrpc/events.c
index 130bacc..ebf985e 100644
--- a/drivers/staging/lustre/lustre/ptlrpc/events.c
+++ b/drivers/staging/lustre/lustre/ptlrpc/events.c
@@ -462,6 +462,10 @@ int ptlrpc_uuid_to_peer(struct obd_uuid *uuid,
 
 	/* Choose the matching UUID that's closest */
 	while (lustre_uuid_to_peer(uuid->uuid, &dst_nid, count++) == 0) {
+		if (peer->nid != LNET_NID_ANY && LNET_NIDADDR(peer->nid) == 0 &&
+		    LNET_NIDNET(dst_nid) != LNET_NIDNET(peer->nid))
+			continue;
+
 		dist = LNetDist(dst_nid, &src_nid, &order);
 		if (dist < 0)
 			continue;
-- 
1.8.3.1


From jsimmons at infradead.org  Tue Jul 31 02:25:59 2018
From: jsimmons at infradead.org (James Simmons)
Date: Mon, 30 Jul 2018 22:25:59 -0400
Subject: [lustre-devel] [PATCH 07/31] lustre: lnet: change positional struct
 initializers to C99
In-Reply-To: <1533003983-29311-1-git-send-email-jsimmons@infradead.org>
References: <1533003983-29311-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1533003983-29311-8-git-send-email-jsimmons@infradead.org>

From: Steve Guminski <stephenx.guminski at intel.com>

This patch makes no functional changes. Struct initializers in the
lnet directory that use C89 or GCC-only syntax are updated to C99
syntax. Whitespace is corrected to match coding style guidelines.

C89 positional initializers require values to be placed in the
correct order. This will cause errors if the fields of the struct
definition are reordered or fields are added or removed. C99 named
initializers avoid this problem, and also automatically clear any
values that are not explicitly set.

The following struct initializers have been updated:

lnet/lnet/api-ni.c:
        lnet_process_id_t id
lnet/lnet/lo.c:
        lnd_t the_lolnd
lnet/selftest/framework.c:
        struct lst_sid LST_INVALID_SID

Signed-off-by: Steve Guminski <stephenx.guminski at intel.com>
WC-id: https://jira.whamcloud.com/browse/LU-6210
Reviewed-on: https://review.whamcloud.com/23493
Reviewed-by: Doug Oucharek <dougso at me.com>
Reviewed-by: Olaf Weber <olaf at sgi.com>
Reviewed-by: Frank Zago <fzago at cray.com>
Reviewed-by: James Simmons <uja.ornl at yahoo.com>
Reviewed-by: Oleg Drokin <green at whamcloud.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/lnet/lnet/api-ni.c        |  3 ++-
 drivers/staging/lustre/lnet/lnet/lo.c            | 20 +++++++++-----------
 drivers/staging/lustre/lnet/selftest/framework.c |  2 +-
 3 files changed, 12 insertions(+), 13 deletions(-)

diff --git a/drivers/staging/lustre/lnet/lnet/api-ni.c b/drivers/staging/lustre/lnet/lnet/api-ni.c
index fea0373..e517893 100644
--- a/drivers/staging/lustre/lnet/lnet/api-ni.c
+++ b/drivers/staging/lustre/lnet/lnet/api-ni.c
@@ -884,7 +884,8 @@ struct lnet_ni  *
 		     struct lnet_handle_md *md_handle,
 		     int ni_count, bool set_eq)
 {
-	struct lnet_process_id id = {LNET_NID_ANY, LNET_PID_ANY};
+	struct lnet_process_id id = { .nid = LNET_NID_ANY,
+				      .pid = LNET_PID_ANY };
 	struct lnet_handle_me me_handle;
 	struct lnet_md md = { NULL };
 	int rc, rc2;
diff --git a/drivers/staging/lustre/lnet/lnet/lo.c b/drivers/staging/lustre/lnet/lnet/lo.c
index 7456b98..dd16cdf 100644
--- a/drivers/staging/lustre/lnet/lnet/lo.c
+++ b/drivers/staging/lustre/lnet/lnet/lo.c
@@ -91,15 +91,13 @@
 }
 
 struct lnet_lnd the_lolnd = {
-	/* .lnd_list       = */ {&the_lolnd.lnd_list, &the_lolnd.lnd_list},
-	/* .lnd_refcount   = */ 0,
-	/* .lnd_type       = */ LOLND,
-	/* .lnd_startup    = */ lolnd_startup,
-	/* .lnd_shutdown   = */ lolnd_shutdown,
-	/* .lnt_ctl        = */ NULL,
-	/* .lnd_send       = */ lolnd_send,
-	/* .lnd_recv       = */ lolnd_recv,
-	/* .lnd_eager_recv = */ NULL,
-	/* .lnd_notify     = */ NULL,
-	/* .lnd_accept     = */ NULL
+	.lnd_list	= {
+				.next	= &the_lolnd.lnd_list,
+				.prev	= &the_lolnd.lnd_list
+			},
+	.lnd_type	= LOLND,
+	.lnd_startup	= lolnd_startup,
+	.lnd_shutdown	= lolnd_shutdown,
+	.lnd_send	= lolnd_send,
+	.lnd_recv	= lolnd_recv,
 };
diff --git a/drivers/staging/lustre/lnet/selftest/framework.c b/drivers/staging/lustre/lnet/selftest/framework.c
index 03a64e3..944a2a6 100644
--- a/drivers/staging/lustre/lnet/selftest/framework.c
+++ b/drivers/staging/lustre/lnet/selftest/framework.c
@@ -40,7 +40,7 @@
 
 #include "selftest.h"
 
-struct lst_sid LST_INVALID_SID = {LNET_NID_ANY, -1};
+struct lst_sid LST_INVALID_SID = { .ses_nid = LNET_NID_ANY, .ses_stamp = -1 };
 
 static int session_timeout = 100;
 module_param(session_timeout, int, 0444);
-- 
1.8.3.1


From jsimmons at infradead.org  Tue Jul 31 02:25:53 2018
From: jsimmons at infradead.org (James Simmons)
Date: Mon, 30 Jul 2018 22:25:53 -0400
Subject: [lustre-devel] [PATCH 01/31] lustre: osc: Send RPCs when extents
 are full
In-Reply-To: <1533003983-29311-1-git-send-email-jsimmons@infradead.org>
References: <1533003983-29311-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1533003983-29311-2-git-send-email-jsimmons@infradead.org>

From: Patrick Farrell <paf at cray.com>

Currently, Lustre decides to send an RPC under a number of
conditions (such as memory pressure or lock cancellcation);
one of the conditions it looks for is "enough dirty pages
to fill an RPC". This worked fine when only one process
could be dirtying pages at a time, but in newer Lustre
versions, more than one process can write to the same
file (and the same osc object) at once.

In this case, the "count dirty pages method" will see there
are enough dirty pages to fill an RPC, but since the dirty
pages are being created by multiple writers, they are not
contiguous and will not fit in to one RPC. This resulted in
many RPCs of less than full size being sent, despite a
good I/O pattern. (Earlier versions of Lustre usually
send only full RPCs when presented with this pattern.)

Instead, we remove this check and add extents to a special
full extent list when they reach max pages per RPC, then
send from that list. (This is similar to high priority
and urgent extents.)

With a good I/O pattern, like usually used in benchmarking,
it should be possible to send only full size RPCs. This
patch achieves that without degrading performance in other
cases.

In IOR tests with multiple writers to a single file,
this patch improves performance by several times, and
returns performance to equal levels (single striped files)
or much greater levels (very high speed OSTs, files
with many stripes) vs earlier versions.

Supporting data is provided in LU-8515.

Signed-off-by: Patrick Farrell <paf at cray.com>
Intel-bug-id: https://jira.whamcloud.com/browse/LU-8515
Reviewed-on: https://review.whamcloud.com/22012
Reviewed-by: Jinshan Xiong <jinshan.xiong at gmail.com>
Reviewed-by: Oleg Drokin <green at whamcloud.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/lustre/osc/osc_cache.c      | 45 ++++++++++++++--------
 .../staging/lustre/lustre/osc/osc_cl_internal.h    |  2 +-
 drivers/staging/lustre/lustre/osc/osc_object.c     |  4 +-
 3 files changed, 32 insertions(+), 19 deletions(-)

diff --git a/drivers/staging/lustre/lustre/osc/osc_cache.c b/drivers/staging/lustre/lustre/osc/osc_cache.c
index 87d0d16..e44822a 100644
--- a/drivers/staging/lustre/lustre/osc/osc_cache.c
+++ b/drivers/staging/lustre/lustre/osc/osc_cache.c
@@ -634,6 +634,10 @@ void osc_extent_release(const struct lu_env *env, struct osc_extent *ext)
 			if (ext->oe_urgent)
 				list_move_tail(&ext->oe_link,
 					       &obj->oo_urgent_exts);
+			else if (ext->oe_nr_pages == ext->oe_mppr) {
+				list_move_tail(&ext->oe_link,
+					       &obj->oo_full_exts);
+			}
 		}
 		osc_object_unlock(obj);
 
@@ -1790,9 +1794,10 @@ static int osc_makes_rpc(struct client_obd *cli, struct osc_object *osc,
 			CDEBUG(D_CACHE, "cache waiters forcing RPC\n");
 			return 1;
 		}
-		if (atomic_read(&osc->oo_nr_writes) >=
-		    cli->cl_max_pages_per_rpc)
+		if (!list_empty(&osc->oo_full_exts)) {
+			CDEBUG(D_CACHE, "full extent ready, make an RPC\n");
 			return 1;
+		}
 	} else {
 		if (atomic_read(&osc->oo_nr_reads) == 0)
 			return 0;
@@ -1963,6 +1968,7 @@ static int try_to_add_extent_for_io(struct client_obd *cli,
 
 	EASSERT((ext->oe_state == OES_CACHE || ext->oe_state == OES_LOCK_DONE),
 		ext);
+	OSC_EXTENT_DUMP(D_CACHE, ext, "trying to add this extent\n");
 
 	if (!data->erd_max_extents)
 		return 0;
@@ -2085,19 +2091,22 @@ static unsigned int get_write_extents(struct osc_object *obj,
 				 struct osc_extent, oe_link);
 		if (!try_to_add_extent_for_io(cli, ext, &data))
 			return data.erd_page_count;
+	}
+	if (data.erd_page_count == data.erd_max_pages)
+		return data.erd_page_count;
 
-		if (!ext->oe_intree)
-			continue;
-
-		while ((ext = next_extent(ext)) != NULL) {
-			if ((ext->oe_state != OES_CACHE) ||
-			    (!list_empty(&ext->oe_link) &&
-			     ext->oe_owner))
-				continue;
-
-			if (!try_to_add_extent_for_io(cli, ext, &data))
-				return data.erd_page_count;
-		}
+	/*
+	 * One key difference between full extents and other extents: full
+	 * extents can usually only be added if the rpclist was empty, so if we
+	 * can't add one, we continue on to trying to add normal extents.  This
+	 * is so we don't miss adding extra extents to an RPC containing high
+	 * priority or urgent extents.
+	 */
+	while (!list_empty(&obj->oo_full_exts)) {
+		ext = list_entry(obj->oo_full_exts.next,
+				 struct osc_extent, oe_link);
+		if (!try_to_add_extent_for_io(cli, ext, &data))
+			break;
 	}
 	if (data.erd_page_count == data.erd_max_pages)
 		return data.erd_page_count;
@@ -2879,8 +2888,12 @@ int osc_cache_truncate_start(const struct lu_env *env, struct osc_object *obj,
 			osc_update_pending(obj, OBD_BRW_WRITE,
 					   -ext->oe_nr_pages);
 		}
-		EASSERT(list_empty(&ext->oe_link), ext);
-		list_add_tail(&ext->oe_link, &list);
+		/* This extent could be on the full extents list, that's OK */
+		EASSERT(!ext->oe_hp && !ext->oe_urgent, ext);
+		if (!list_empty(&ext->oe_link))
+			list_move_tail(&ext->oe_link, &list);
+		else
+			list_add_tail(&ext->oe_link, &list);
 
 		ext = next_extent(ext);
 	}
diff --git a/drivers/staging/lustre/lustre/osc/osc_cl_internal.h b/drivers/staging/lustre/lustre/osc/osc_cl_internal.h
index d86d3f7..da04c2c 100644
--- a/drivers/staging/lustre/lustre/osc/osc_cl_internal.h
+++ b/drivers/staging/lustre/lustre/osc/osc_cl_internal.h
@@ -139,7 +139,7 @@ struct osc_object {
 	 */
 	struct list_head	   oo_hp_exts; /* list of hp extents */
 	struct list_head	   oo_urgent_exts; /* list of writeback extents */
-	struct list_head	   oo_rpc_exts;
+	struct list_head	   oo_full_exts;
 
 	struct list_head	   oo_reading_exts;
 
diff --git a/drivers/staging/lustre/lustre/osc/osc_object.c b/drivers/staging/lustre/lustre/osc/osc_object.c
index 8424018..b9bf2b8 100644
--- a/drivers/staging/lustre/lustre/osc/osc_object.c
+++ b/drivers/staging/lustre/lustre/osc/osc_object.c
@@ -85,7 +85,7 @@ static int osc_object_init(const struct lu_env *env, struct lu_object *obj,
 	osc->oo_root.rb_node = NULL;
 	INIT_LIST_HEAD(&osc->oo_hp_exts);
 	INIT_LIST_HEAD(&osc->oo_urgent_exts);
-	INIT_LIST_HEAD(&osc->oo_rpc_exts);
+	INIT_LIST_HEAD(&osc->oo_full_exts);
 	INIT_LIST_HEAD(&osc->oo_reading_exts);
 	atomic_set(&osc->oo_nr_reads, 0);
 	atomic_set(&osc->oo_nr_writes, 0);
@@ -111,7 +111,7 @@ static void osc_object_free(const struct lu_env *env, struct lu_object *obj)
 	LASSERT(!osc->oo_root.rb_node);
 	LASSERT(list_empty(&osc->oo_hp_exts));
 	LASSERT(list_empty(&osc->oo_urgent_exts));
-	LASSERT(list_empty(&osc->oo_rpc_exts));
+	LASSERT(list_empty(&osc->oo_full_exts));
 	LASSERT(list_empty(&osc->oo_reading_exts));
 	LASSERT(atomic_read(&osc->oo_nr_reads) == 0);
 	LASSERT(atomic_read(&osc->oo_nr_writes) == 0);
-- 
1.8.3.1


From jsimmons at infradead.org  Tue Jul 31 02:26:00 2018
From: jsimmons at infradead.org (James Simmons)
Date: Mon, 30 Jul 2018 22:26:00 -0400
Subject: [lustre-devel] [PATCH 08/31] lustre: llite: don't zero timestamps
 internally
In-Reply-To: <1533003983-29311-1-git-send-email-jsimmons@infradead.org>
References: <1533003983-29311-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1533003983-29311-9-git-send-email-jsimmons@infradead.org>

From: Niu Yawei <yawei.niu at intel.com>

In ll_md_blocking_ast(), we zero all timestamps to avoid these
'leftovers' interfering the new timestamps from MDS, especially
when the timestamps are set back by other clients. It's not
quite right to change timestamps in this way, because:

1. The pending lock can be matched by getattr, so these zero
   timestamps can be fetched by application in a small race window.

2. It doesn't make sense to zero the mtime and ctime, because we
   always use the newest ctime and mtime from MDS when do attributes
   merge, they won't interfere new timestamps set by other clients.

Signed-off-by: Niu Yawei <yawei.niu at intel.com>
WC-id: https://jira.whamcloud.com/browse/LU-9033
Reviewed-on: https://review.whamcloud.com/24984
Reviewed-by: Jinshan Xiong <jinshan.xiong at gmail.com>
Reviewed-by: Andreas Dilger <adilger at whamcloud.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/lustre/llite/file.c           | 4 +++-
 drivers/staging/lustre/lustre/llite/llite_internal.h | 5 +++++
 drivers/staging/lustre/lustre/llite/namei.c          | 6 +-----
 3 files changed, 9 insertions(+), 6 deletions(-)

diff --git a/drivers/staging/lustre/lustre/llite/file.c b/drivers/staging/lustre/lustre/llite/file.c
index 684877c..e4aefb5 100644
--- a/drivers/staging/lustre/lustre/llite/file.c
+++ b/drivers/staging/lustre/lustre/llite/file.c
@@ -1024,8 +1024,10 @@ int ll_merge_attr(const struct lu_env *env, struct inode *inode)
 	 * POSIX. Solving this problem needs to send an RPC to MDT for each
 	 * read, this will hurt performance.
 	 */
-	if (inode->i_atime.tv_sec < lli->lli_atime)
+	if (inode->i_atime.tv_sec < lli->lli_atime || lli->lli_update_atime) {
 		inode->i_atime.tv_sec = lli->lli_atime;
+		lli->lli_update_atime = 0;
+	}
 	inode->i_mtime.tv_sec = lli->lli_mtime;
 	inode->i_ctime.tv_sec = lli->lli_ctime;
 
diff --git a/drivers/staging/lustre/lustre/llite/llite_internal.h b/drivers/staging/lustre/lustre/llite/llite_internal.h
index 8399501..f6c8daf 100644
--- a/drivers/staging/lustre/lustre/llite/llite_internal.h
+++ b/drivers/staging/lustre/lustre/llite/llite_internal.h
@@ -138,6 +138,11 @@ struct ll_inode_info {
 	s64				lli_ctime;
 	spinlock_t			lli_agl_lock;
 
+	/* update atime from MDS no matter if it's older than
+	 * local inode atime.
+	 */
+	unsigned int			lli_update_atime:1;
+
 	/* Try to make the d::member and f::member are aligned. Before using
 	 * these members, make clear whether it is directory or not.
 	 */
diff --git a/drivers/staging/lustre/lustre/llite/namei.c b/drivers/staging/lustre/lustre/llite/namei.c
index 134cc31..e541f78 100644
--- a/drivers/staging/lustre/lustre/llite/namei.c
+++ b/drivers/staging/lustre/lustre/llite/namei.c
@@ -265,11 +265,7 @@ int ll_md_blocking_ast(struct ldlm_lock *lock, struct ldlm_lock_desc *desc,
 		if (bits & MDS_INODELOCK_UPDATE) {
 			struct ll_inode_info *lli = ll_i2info(inode);
 
-			spin_lock(&lli->lli_lock);
-			inode->i_mtime.tv_sec = 0;
-			inode->i_atime.tv_sec = 0;
-			inode->i_ctime.tv_sec = 0;
-			spin_unlock(&lli->lli_lock);
+			lli->lli_update_atime = 1;
 		}
 
 		if ((bits & MDS_INODELOCK_UPDATE) && S_ISDIR(inode->i_mode)) {
-- 
1.8.3.1


From jsimmons at infradead.org  Tue Jul 31 02:26:03 2018
From: jsimmons at infradead.org (James Simmons)
Date: Mon, 30 Jul 2018 22:26:03 -0400
Subject: [lustre-devel] [PATCH 11/31] lustre: lnet: Fix route hops print
In-Reply-To: <1533003983-29311-1-git-send-email-jsimmons@infradead.org>
References: <1533003983-29311-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1533003983-29311-12-git-send-email-jsimmons@infradead.org>

From: Amir Shehata <ashehata at whamcloud.com>

The default number of hops for  a route is -1. This is
currently being printed as %u. Change that to %d to
make it print out properly.

Signed-off-by: Amir Shehata <ashehata at whamcloud.com>
WC-id: https://jira.whamcloud.com/browse/LU-9078
Reviewed-on: https://review.whamcloud.com/25250
Reviewed-by: Olaf Weber <olaf at sgi.com>
Reviewed-by: Doug Oucharek <dougso at me.com>
Reviewed-by: James Simmons <uja.ornl at yahoo.com>
Reviewed-by: Oleg Drokin <green at whamcloud.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/lnet/lnet/router_proc.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/staging/lustre/lnet/lnet/router_proc.c b/drivers/staging/lustre/lnet/lnet/router_proc.c
index 8856798..aa98ce5 100644
--- a/drivers/staging/lustre/lnet/lnet/router_proc.c
+++ b/drivers/staging/lustre/lnet/lnet/router_proc.c
@@ -218,7 +218,7 @@ static int proc_lnet_routes(struct ctl_table *table, int write,
 			int alive = lnet_is_route_alive(route);
 
 			s += snprintf(s, tmpstr + tmpsiz - s,
-				      "%-8s %4u %8u %7s %s\n",
+				      "%-8s %4d %8u %7s %s\n",
 				      libcfs_net2str(net), hops,
 				      priority,
 				      alive ? "up" : "down",
-- 
1.8.3.1


From jsimmons at infradead.org  Tue Jul 31 02:26:06 2018
From: jsimmons at infradead.org (James Simmons)
Date: Mon, 30 Jul 2018 22:26:06 -0400
Subject: [lustre-devel] [PATCH 14/31] lustre: llite: buggy special handling
 on MULTIMODRPCS
In-Reply-To: <1533003983-29311-1-git-send-email-jsimmons@infradead.org>
References: <1533003983-29311-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1533003983-29311-15-git-send-email-jsimmons@infradead.org>

From: Niu Yawei <yawei.niu at intel.com>

There is some special handling over MULTIMODPRCS flag in
client_connect_import(), it looks unnecessary and buggy,
the MULTIMODPRCS flag would be cleared unexpectedly from
imp_connect_data on reconnect.

This patch removed the special handling code and treat
MULTIMODRPCS normally just like other flags.

Signed-off-by: Niu Yawei <yawei.niu at intel.com>
WC-id: https://jira.whamcloud.com/browse/LU-9115
Reviewed-on: https://review.whamcloud.com/25435
Reviewed-by: Andreas Dilger <adilger at whamcloud.com>
Reviewed-by: John L. Hammond <jhammond at whamcloud.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/lustre/ldlm/ldlm_lib.c   | 12 ------------
 drivers/staging/lustre/lustre/llite/llite_lib.c |  2 +-
 2 files changed, 1 insertion(+), 13 deletions(-)

diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_lib.c b/drivers/staging/lustre/lustre/ldlm/ldlm_lib.c
index 5da8c88..c36d1e4 100644
--- a/drivers/staging/lustre/lustre/ldlm/ldlm_lib.c
+++ b/drivers/staging/lustre/lustre/ldlm/ldlm_lib.c
@@ -512,7 +512,6 @@ int client_connect_import(const struct lu_env *env,
 	struct obd_import       *imp    = cli->cl_import;
 	struct obd_connect_data *ocd;
 	struct lustre_handle    conn    = { 0 };
-	bool is_mdc = false;
 	int		     rc;
 
 	*exp = NULL;
@@ -539,18 +538,12 @@ int client_connect_import(const struct lu_env *env,
 	ocd = &imp->imp_connect_data;
 	if (data) {
 		*ocd = *data;
-		is_mdc = !strncmp(imp->imp_obd->obd_type->typ_name,
-				  LUSTRE_MDC_NAME, 3);
-		if (is_mdc)
-			data->ocd_connect_flags |= OBD_CONNECT_MULTIMODRPCS;
 		imp->imp_connect_flags_orig = data->ocd_connect_flags;
 		imp->imp_connect_flags2_orig = data->ocd_connect_flags2;
 	}
 
 	rc = ptlrpc_connect_import(imp);
 	if (rc != 0) {
-		if (data && is_mdc)
-			data->ocd_connect_flags &= ~OBD_CONNECT_MULTIMODRPCS;
 		LASSERT(imp->imp_state == LUSTRE_IMP_DISCON);
 		goto out_ldlm;
 	}
@@ -561,11 +554,6 @@ int client_connect_import(const struct lu_env *env,
 			 ocd->ocd_connect_flags, "old %#llx, new %#llx\n",
 			 data->ocd_connect_flags, ocd->ocd_connect_flags);
 		data->ocd_connect_flags = ocd->ocd_connect_flags;
-		/* clear the flag as it was not set and is not known
-		 * by upper layers
-		 */
-		if (is_mdc)
-			data->ocd_connect_flags &= ~OBD_CONNECT_MULTIMODRPCS;
 	}
 
 	ptlrpc_pinger_add_import(imp);
diff --git a/drivers/staging/lustre/lustre/llite/llite_lib.c b/drivers/staging/lustre/lustre/llite/llite_lib.c
index 71eb42d..ccb5bda 100644
--- a/drivers/staging/lustre/lustre/llite/llite_lib.c
+++ b/drivers/staging/lustre/lustre/llite/llite_lib.c
@@ -211,7 +211,7 @@ static int client_common_fill_super(struct super_block *sb, char *md, char *dt)
 				  OBD_CONNECT_DIR_STRIPE |
 				  OBD_CONNECT_BULK_MBITS |
 				  OBD_CONNECT_SUBTREE |
-				  OBD_CONNECT_FLAGS2;
+				  OBD_CONNECT_FLAGS2 | OBD_CONNECT_MULTIMODRPCS;
 
 	data->ocd_connect_flags2 = 0;
 
-- 
1.8.3.1


From jsimmons at infradead.org  Tue Jul 31 02:26:12 2018
From: jsimmons at infradead.org (James Simmons)
Date: Mon, 30 Jul 2018 22:26:12 -0400
Subject: [lustre-devel] [PATCH 20/31] lustre: llite: Remove
 OBD_FAIL_OSC_CONNECT_CKSUM
In-Reply-To: <1533003983-29311-1-git-send-email-jsimmons@infradead.org>
References: <1533003983-29311-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1533003983-29311-21-git-send-email-jsimmons@infradead.org>

From: Abrarahmed Momin <abrar.habib at seagate.com>

Remove OBD_FAIL_OSC_CONNECT_CKSUM as all clients and servers
since 1.8 support OBD_CONNECT_CKSUM. No reason to check
interoperability with older servers anymore.

Signed-off-by: Abrarahmed Momin <abrar.habib at seagate.com>
Seagate-bug-id: MRP-1421
WC-id: https://jira.whamcloud.com/browse/LU-5361
Reviewed-on: https://review.whamcloud.com/23644
Reviewed-by: Andreas Dilger <adilger at whamcloud.com>
Reviewed-by: Jian Yu <yujian at whamcloud.com>
Reviewed-by: Oleg Drokin <green at whamcloud.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 .../staging/lustre/lustre/include/obd_support.h    |  2 +-
 drivers/staging/lustre/lustre/llite/llite_lib.c    | 22 ++++++++++------------
 drivers/staging/lustre/lustre/ptlrpc/import.c      |  5 ++---
 3 files changed, 13 insertions(+), 16 deletions(-)

diff --git a/drivers/staging/lustre/lustre/include/obd_support.h b/drivers/staging/lustre/lustre/include/obd_support.h
index 726cc4d..80b9935 100644
--- a/drivers/staging/lustre/lustre/include/obd_support.h
+++ b/drivers/staging/lustre/lustre/include/obd_support.h
@@ -312,7 +312,7 @@
 #define OBD_FAIL_OSC_CHECKSUM_RECEIVE    0x408
 #define OBD_FAIL_OSC_CHECKSUM_SEND       0x409
 #define OBD_FAIL_OSC_BRW_PREP_REQ2       0x40a
-#define OBD_FAIL_OSC_CONNECT_CKSUM       0x40b
+/* #define OBD_FAIL_OSC_CONNECT_CKSUM	 0x40b Obsolete since 2.9 */
 #define OBD_FAIL_OSC_CKSUM_ADLER_ONLY    0x40c
 #define OBD_FAIL_OSC_DIO_PAUSE	   0x40d
 #define OBD_FAIL_OSC_OBJECT_CONTENTION   0x40e
diff --git a/drivers/staging/lustre/lustre/llite/llite_lib.c b/drivers/staging/lustre/lustre/llite/llite_lib.c
index ccb5bda..cd5b064 100644
--- a/drivers/staging/lustre/lustre/llite/llite_lib.c
+++ b/drivers/staging/lustre/lustre/llite/llite_lib.c
@@ -395,19 +395,17 @@ static int client_common_fill_super(struct super_block *sb, char *md, char *dt)
 	if (!OBD_FAIL_CHECK(OBD_FAIL_OSC_CONNECT_GRANT_PARAM))
 		data->ocd_connect_flags |= OBD_CONNECT_GRANT_PARAM;
 
-	if (!OBD_FAIL_CHECK(OBD_FAIL_OSC_CONNECT_CKSUM)) {
-		/* OBD_CONNECT_CKSUM should always be set, even if checksums are
-		 * disabled by default, because it can still be enabled on the
-		 * fly via /sys. As a consequence, we still need to come to an
-		 * agreement on the supported algorithms at connect time
-		 */
-		data->ocd_connect_flags |= OBD_CONNECT_CKSUM;
+	/* OBD_CONNECT_CKSUM should always be set, even if checksums are
+	 * disabled by default, because it can still be enabled on the
+	 * fly via /sys. As a consequence, we still need to come to an
+	 * agreement on the supported algorithms at connect time
+	 */
+	data->ocd_connect_flags |= OBD_CONNECT_CKSUM;
 
-		if (OBD_FAIL_CHECK(OBD_FAIL_OSC_CKSUM_ADLER_ONLY))
-			data->ocd_cksum_types = OBD_CKSUM_ADLER;
-		else
-			data->ocd_cksum_types = cksum_types_supported_client();
-	}
+	if (OBD_FAIL_CHECK(OBD_FAIL_OSC_CKSUM_ADLER_ONLY))
+		data->ocd_cksum_types = OBD_CKSUM_ADLER;
+	else
+		data->ocd_cksum_types = cksum_types_supported_client();
 
 	data->ocd_connect_flags |= OBD_CONNECT_LRU_RESIZE;
 
diff --git a/drivers/staging/lustre/lustre/ptlrpc/import.c b/drivers/staging/lustre/lustre/ptlrpc/import.c
index 4db0d89..07dc87d 100644
--- a/drivers/staging/lustre/lustre/ptlrpc/import.c
+++ b/drivers/staging/lustre/lustre/ptlrpc/import.c
@@ -814,12 +814,11 @@ static int ptlrpc_connect_set_flags(struct obd_import *imp,
 		 * the checksum types it doesn't support
 		 */
 		if (!(ocd->ocd_cksum_types & cksum_types_supported_client())) {
-			LCONSOLE_WARN("The negotiation of the checksum algorithm to use with server %s failed (%x/%x), disabling checksums\n",
+			LCONSOLE_ERROR("The negotiation of the checksum algorithm to use with server %s failed (%x/%x), disabling checksums\n",
 				      obd2cli_tgt(imp->imp_obd),
 				      ocd->ocd_cksum_types,
 				      cksum_types_supported_client());
-			cli->cl_checksum = 0;
-			cli->cl_supp_cksum_types = OBD_CKSUM_ADLER;
+			return -EPROTO;
 		} else {
 			cli->cl_supp_cksum_types = ocd->ocd_cksum_types;
 		}
-- 
1.8.3.1


From jsimmons at infradead.org  Tue Jul 31 02:26:01 2018
From: jsimmons at infradead.org (James Simmons)
Date: Mon, 30 Jul 2018 22:26:01 -0400
Subject: [lustre-devel] [PATCH 09/31] lustre: mgc: relate sptlrpc & param to
 MGC
In-Reply-To: <1533003983-29311-1-git-send-email-jsimmons@infradead.org>
References: <1533003983-29311-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1533003983-29311-10-git-send-email-jsimmons@infradead.org>

From: Hongchao Zhang <hongchao at whamcloud.com>

If sptlrpc or params config logs come from different MGC,
it should be regarded as different logs, this patch binds
these config logs with MGC obd device to separate them.

The fix for a bug discovered later is also included for
this patch. Since sb is NULL for config_log_find_or_add
the cfs_instance field was being set to the obd device.
This confused the sptlrpc layer so for now cfg_instance
is set to NULl in the sptlrpc case. This will be resolved
with the move to kobjects.

Signed-off-by: Hongchao Zhang <hongchao at whamcloud.com>
Signed-off-by: James Simmons <uja.ornl at yahoo.com>
Signed-off-by: John L. Hammond <jhammond at whamcloud.com>
Intel-bug-id: https://jira.whamcloud.com/browse/LU-9034
Reviewed-on: https://review.whamcloud.com/24988
Intel-bug-id: https://jira.whamcloud.com/browse/LU-9567
Reviewed-on: https://review.whamcloud.com/27320
Reviewed-by: Mike Pershin <mpershin at whamcloud.com>
Reviewed-by: Andreas Dilger <adilger at whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson at ddn.com>
Reviewed-by: Oleg Drokin <green at whamcloud.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/lustre/mgc/mgc_request.c | 36 ++++++++++++++-----------
 1 file changed, 20 insertions(+), 16 deletions(-)

diff --git a/drivers/staging/lustre/lustre/mgc/mgc_request.c b/drivers/staging/lustre/lustre/mgc/mgc_request.c
index 32df804..82acac0 100644
--- a/drivers/staging/lustre/lustre/mgc/mgc_request.c
+++ b/drivers/staging/lustre/lustre/mgc/mgc_request.c
@@ -263,18 +263,23 @@ struct config_llog_data *do_config_log_add(struct obd_device *obd,
 }
 
 static struct config_llog_data *
-config_params_log_add(struct obd_device *obd,
-		      struct config_llog_instance *cfg, struct super_block *sb)
+config_log_find_or_add(struct obd_device *obd, char *logname,
+		       struct super_block *sb, int type,
+		       struct config_llog_instance *cfg)
 {
 	struct config_llog_instance	lcfg = *cfg;
 	struct config_llog_data		*cld;
 
-	lcfg.cfg_instance = sb;
+	lcfg.cfg_instance = sb ? (void *)sb : (void *)obd;
 
-	cld = do_config_log_add(obd, PARAMS_FILENAME, CONFIG_T_PARAMS,
-				&lcfg, sb);
+	if (type == CONFIG_T_SPTLRPC)
+		lcfg.cfg_instance = NULL;
 
-	return cld;
+	cld = config_log_find(logname, &lcfg);
+	if (unlikely(cld))
+		return cld;
+
+	return do_config_log_add(obd, logname, type, &lcfg, sb);
 }
 
 /** Add this log to the list of active logs watched by an MGC.
@@ -310,17 +315,16 @@ struct config_llog_data *do_config_log_add(struct obd_device *obd,
 	memcpy(seclogname, logname, ptr - logname);
 	strcpy(seclogname + (ptr - logname), "-sptlrpc");
 
-	sptlrpc_cld = config_log_find(seclogname, NULL);
-	if (!sptlrpc_cld) {
-		sptlrpc_cld = do_config_log_add(obd, seclogname,
-						CONFIG_T_SPTLRPC, NULL, NULL);
-		if (IS_ERR(sptlrpc_cld)) {
-			CERROR("can't create sptlrpc log: %s\n", seclogname);
-			rc = PTR_ERR(sptlrpc_cld);
-			goto out_err;
-		}
+	sptlrpc_cld = config_log_find_or_add(obd, seclogname, NULL,
+					     CONFIG_T_SPTLRPC, cfg);
+	if (IS_ERR(sptlrpc_cld)) {
+		CERROR("can't create sptlrpc log: %s\n", seclogname);
+		rc = PTR_ERR(sptlrpc_cld);
+		goto out_err;
 	}
-	params_cld = config_params_log_add(obd, cfg, sb);
+
+	params_cld = config_log_find_or_add(obd, PARAMS_FILENAME, sb,
+					    CONFIG_T_PARAMS, cfg);
 	if (IS_ERR(params_cld)) {
 		rc = PTR_ERR(params_cld);
 		CERROR("%s: can't create params log: rc = %d\n",
-- 
1.8.3.1


From jsimmons at infradead.org  Tue Jul 31 02:26:15 2018
From: jsimmons at infradead.org (James Simmons)
Date: Mon, 30 Jul 2018 22:26:15 -0400
Subject: [lustre-devel] [PATCH 23/31] lustre: llite: Return -ERESTARTSYS in
 range_lock()
In-Reply-To: <1533003983-29311-1-git-send-email-jsimmons@infradead.org>
References: <1533003983-29311-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1533003983-29311-24-git-send-email-jsimmons@infradead.org>

From: Chris Horn <hornc at cray.com>

If we return -ERESTARTSYS rather than -EINTR then the syscall can be
retried rather than failing with -EINTR.

Signed-off-by: Chris Horn <hornc at cray.com>
WC-id: https://jira.whamcloud.com/browse/LU-8735
Reviewed-on: https://review.whamcloud.com/23259
Reviewed-by: James Simmons <uja.ornl at yahoo.com>
Reviewed-by: Andreas Dilger <adilger at whamcloud.com>
Reviewed-by: Oleg Drokin <green at whamcloud.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/lustre/llite/range_lock.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/staging/lustre/lustre/llite/range_lock.c b/drivers/staging/lustre/lustre/llite/range_lock.c
index acdb0dc..d37da8e 100644
--- a/drivers/staging/lustre/lustre/llite/range_lock.c
+++ b/drivers/staging/lustre/lustre/llite/range_lock.c
@@ -157,7 +157,7 @@ int range_lock(struct range_lock_tree *tree, struct range_lock *lock)
 
 		if (signal_pending(current)) {
 			range_unlock(tree, lock);
-			rc = -EINTR;
+			rc = -ERESTARTSYS;
 			goto out;
 		}
 		spin_lock(&tree->rlt_lock);
-- 
1.8.3.1


From jsimmons at infradead.org  Tue Jul 31 02:26:16 2018
From: jsimmons at infradead.org (James Simmons)
Date: Mon, 30 Jul 2018 22:26:16 -0400
Subject: [lustre-devel] [PATCH 24/31] lustre: obdclass: use static
 initializer macros where possible
In-Reply-To: <1533003983-29311-1-git-send-email-jsimmons@infradead.org>
References: <1533003983-29311-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1533003983-29311-25-git-send-email-jsimmons@infradead.org>

From: "John L. Hammond" <jhammond at whamcloud.com>

In obdclass replace module load time initialization of several
atomics, lists, locks, mutexes, wait queues, etc with static
initialization using the kernel provided macros.

Signed-off-by: John L. Hammond <jhammond at whamcloud.com>
WC-id: https://jira.whamcloud.com/browse/LU-9010
Reviewed-on: https://review.whamcloud.com/24827
Reviewed-by: Steve Guminski <stephenx.guminski at intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin at intel.com>
Reviewed-by: Oleg Drokin <green at whamcloud.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/lustre/include/lu_object.h  |  6 -----
 drivers/staging/lustre/lustre/include/obd_class.h  |  6 -----
 drivers/staging/lustre/lustre/obdclass/class_obd.c | 26 +++++-----------------
 drivers/staging/lustre/lustre/obdclass/genops.c    |  5 ++++-
 drivers/staging/lustre/lustre/obdclass/lu_object.c | 15 -------------
 .../staging/lustre/lustre/obdclass/lustre_peer.c   | 16 ++-----------
 6 files changed, 12 insertions(+), 62 deletions(-)

diff --git a/drivers/staging/lustre/lustre/include/lu_object.h b/drivers/staging/lustre/lustre/include/lu_object.h
index 4153db7..47f8021 100644
--- a/drivers/staging/lustre/lustre/include/lu_object.h
+++ b/drivers/staging/lustre/lustre/include/lu_object.h
@@ -330,12 +330,6 @@ struct lu_device_type {
 	 * Number of existing device type instances.
 	 */
 	atomic_t				ldt_device_nr;
-	/**
-	 * Linkage into a global list of all device types.
-	 *
-	 * \see lu_device_types.
-	 */
-	struct list_head			      ldt_linkage;
 };
 
 /**
diff --git a/drivers/staging/lustre/lustre/include/obd_class.h b/drivers/staging/lustre/lustre/include/obd_class.h
index e772e3d..184da99 100644
--- a/drivers/staging/lustre/lustre/include/obd_class.h
+++ b/drivers/staging/lustre/lustre/include/obd_class.h
@@ -48,7 +48,6 @@
 #define OBD_STATFS_FOR_MDT0	0x0004
 
 /* OBD Device Declarations */
-extern struct obd_device *obd_devs[MAX_OBD_DEVICES];
 extern rwlock_t obd_dev_lock;
 
 /* OBD Operations Declarations */
@@ -59,7 +58,6 @@
 struct lu_device_type;
 
 /* genops.c */
-extern struct list_head obd_types;
 struct obd_export *class_conn2export(struct lustre_handle *conn);
 int class_register_type(struct obd_ops *dt_ops, struct md_ops *md_ops,
 			const char *name, struct lu_device_type *ldt);
@@ -133,7 +131,6 @@ void class_decref(struct obd_device *obd,
 int class_config_llog_handler(const struct lu_env *env,
 			      struct llog_handle *handle,
 			      struct llog_rec_hdr *rec, void *data);
-int class_add_uuid(const char *uuid, __u64 nid);
 
 /* obdecho */
 void lprocfs_echo_init_vars(struct lprocfs_static_vars *lvars);
@@ -1576,13 +1573,10 @@ struct lwp_register_item {
 int class_add_uuid(const char *uuid, __u64 nid);
 int class_del_uuid(const char *uuid);
 int class_check_uuid(struct obd_uuid *uuid, __u64 nid);
-void class_init_uuidlist(void);
-void class_exit_uuidlist(void);
 
 /* class_obd.c */
 extern char obd_jobid_node[];
 extern struct miscdevice obd_psdev;
-extern spinlock_t obd_types_lock;
 int class_procfs_init(void);
 int class_procfs_clean(void);
 
diff --git a/drivers/staging/lustre/lustre/obdclass/class_obd.c b/drivers/staging/lustre/lustre/obdclass/class_obd.c
index 04e55fc..05ae6e1 100644
--- a/drivers/staging/lustre/lustre/obdclass/class_obd.c
+++ b/drivers/staging/lustre/lustre/obdclass/class_obd.c
@@ -49,10 +49,6 @@
 #include <uapi/linux/lnet/libcfs_ioctl.h>
 #include "llog_internal.h"
 
-struct obd_device *obd_devs[MAX_OBD_DEVICES];
-struct list_head obd_types;
-DEFINE_RWLOCK(obd_dev_lock);
-
 /* The following are visible and mutable through /sys/fs/lustre. */
 unsigned int obd_debug_peer_on_timeout;
 EXPORT_SYMBOL(obd_debug_peer_on_timeout);
@@ -455,28 +451,25 @@ static int obd_init_checks(void)
 
 static int __init obdclass_init(void)
 {
-	int i, err;
+	int err;
 
 	LCONSOLE_INFO("Lustre: Build Version: " LUSTRE_VERSION_STRING "\n");
 
-	spin_lock_init(&obd_types_lock);
-
 	err = libcfs_setup();
 	if (err)
 		return err;
 
-	obd_zombie_impexp_init();
+	err = obd_zombie_impexp_init();
+	if (err)
+		return err;
 
 	err = obd_init_checks();
 	if (err)
 		goto cleanup_zombie_impexp;
 
-	class_init_uuidlist();
 	err = class_handle_init();
 	if (err)
-		goto cleanup_uuidlist;
-
-	INIT_LIST_HEAD(&obd_types);
+		goto cleanup_zombie_impexp;
 
 	err = misc_register(&obd_psdev);
 	if (err) {
@@ -484,10 +477,6 @@ static int __init obdclass_init(void)
 		goto cleanup_class_handle;
 	}
 
-	/* This struct is already zeroed for us (static global) */
-	for (i = 0; i < class_devno_max(); i++)
-		obd_devs[i] = NULL;
-
 	/* Default the dirty page cache cap to 1/2 of system memory.
 	 * For clients with less memory, a larger fraction is needed
 	 * for other purposes (mostly for BGL).
@@ -550,9 +539,6 @@ static int __init obdclass_init(void)
 cleanup_class_handle:
 	class_handle_cleanup();
 
-cleanup_uuidlist:
-	class_exit_uuidlist();
-
 cleanup_zombie_impexp:
 	obd_zombie_impexp_stop();
 
@@ -571,7 +557,7 @@ static void obdclass_exit(void)
 	class_procfs_clean();
 
 	class_handle_cleanup();
-	class_exit_uuidlist();
+	class_del_uuid(NULL); /* Delete all UUIDs. */
 	obd_zombie_impexp_stop();
 }
 
diff --git a/drivers/staging/lustre/lustre/obdclass/genops.c b/drivers/staging/lustre/lustre/obdclass/genops.c
index 8454b44..532418e 100644
--- a/drivers/staging/lustre/lustre/obdclass/genops.c
+++ b/drivers/staging/lustre/lustre/obdclass/genops.c
@@ -41,7 +41,10 @@
 #include <lprocfs_status.h>
 #include <lustre_kernelcomm.h>
 
-spinlock_t obd_types_lock;
+static DEFINE_SPINLOCK(obd_types_lock);
+static LIST_HEAD(obd_types);
+DEFINE_RWLOCK(obd_dev_lock);
+static struct obd_device *obd_devs[MAX_OBD_DEVICES];
 
 static struct kmem_cache *obd_device_cachep;
 struct kmem_cache *obdo_cachep;
diff --git a/drivers/staging/lustre/lustre/obdclass/lu_object.c b/drivers/staging/lustre/lustre/obdclass/lu_object.c
index 2d24eb6..cb57abf 100644
--- a/drivers/staging/lustre/lustre/obdclass/lu_object.c
+++ b/drivers/staging/lustre/lustre/obdclass/lu_object.c
@@ -764,35 +764,20 @@ struct lu_object *lu_object_find_slice(const struct lu_env *env,
 }
 EXPORT_SYMBOL(lu_object_find_slice);
 
-/**
- * Global list of all device types.
- */
-static LIST_HEAD(lu_device_types);
-
 int lu_device_type_init(struct lu_device_type *ldt)
 {
 	int result = 0;
 
 	atomic_set(&ldt->ldt_device_nr, 0);
-	INIT_LIST_HEAD(&ldt->ldt_linkage);
 	if (ldt->ldt_ops->ldto_init)
 		result = ldt->ldt_ops->ldto_init(ldt);
 
-	if (!result) {
-		spin_lock(&obd_types_lock);
-		list_add(&ldt->ldt_linkage, &lu_device_types);
-		spin_unlock(&obd_types_lock);
-	}
-
 	return result;
 }
 EXPORT_SYMBOL(lu_device_type_init);
 
 void lu_device_type_fini(struct lu_device_type *ldt)
 {
-	spin_lock(&obd_types_lock);
-	list_del_init(&ldt->ldt_linkage);
-	spin_unlock(&obd_types_lock);
 	if (ldt->ldt_ops->ldto_fini)
 		ldt->ldt_ops->ldto_fini(ldt);
 }
diff --git a/drivers/staging/lustre/lustre/obdclass/lustre_peer.c b/drivers/staging/lustre/lustre/obdclass/lustre_peer.c
index 7fc62b7..5705b0a 100644
--- a/drivers/staging/lustre/lustre/obdclass/lustre_peer.c
+++ b/drivers/staging/lustre/lustre/obdclass/lustre_peer.c
@@ -51,20 +51,8 @@ struct uuid_nid_data {
 };
 
 /* FIXME: This should probably become more elegant than a global linked list */
-static struct list_head	g_uuid_list;
-static spinlock_t	g_uuid_lock;
-
-void class_init_uuidlist(void)
-{
-	INIT_LIST_HEAD(&g_uuid_list);
-	spin_lock_init(&g_uuid_lock);
-}
-
-void class_exit_uuidlist(void)
-{
-	/* delete all */
-	class_del_uuid(NULL);
-}
+static LIST_HEAD(g_uuid_list);
+static DEFINE_SPINLOCK(g_uuid_lock);
 
 int lustre_uuid_to_peer(const char *uuid, lnet_nid_t *peer_nid, int index)
 {
-- 
1.8.3.1


From jsimmons at infradead.org  Tue Jul 31 02:26:17 2018
From: jsimmons at infradead.org (James Simmons)
Date: Mon, 30 Jul 2018 22:26:17 -0400
Subject: [lustre-devel] [PATCH 25/31] lustre: config: move config types into
 lustre_idl.h
In-Reply-To: <1533003983-29311-1-git-send-email-jsimmons@infradead.org>
References: <1533003983-29311-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1533003983-29311-26-git-send-email-jsimmons@infradead.org>

From: Niu Yawei <yawei.niu at intel.com>

Move config type values CONFIG_T_XXX into lustre_idl.h since they
will be put on wire when reading config logs.

Add missing wire checks for mgs_nidtbl_entry, mgs_config_body and
mgs_config_res.

Redefine CONFIG_SUB_XXX for the sub clds attached on config log.

Signed-off-by: Niu Yawei <yawei.niu at intel.com>
WC-id: https://jira.whamcloud.com/browse/LU-9216
Reviewed-on: https://review.whamcloud.com/26022
Reviewed-by: Fan Yong <fan.yong at intel.com>
Reviewed-by: John L. Hammond <jhammond at whamcloud.com>
Reviewed-by: Oleg Drokin <green at whamcloud.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 .../lustre/include/uapi/linux/lustre/lustre_idl.h  | 10 ++-
 drivers/staging/lustre/lustre/include/obd_class.h  | 12 +--
 drivers/staging/lustre/lustre/mgc/mgc_request.c    |  6 +-
 drivers/staging/lustre/lustre/ptlrpc/wiretest.c    | 85 ++++++++++++++++++++++
 4 files changed, 103 insertions(+), 10 deletions(-)

diff --git a/drivers/staging/lustre/include/uapi/linux/lustre/lustre_idl.h b/drivers/staging/lustre/include/uapi/linux/lustre/lustre_idl.h
index c9b32ef..bd3b45a 100644
--- a/drivers/staging/lustre/include/uapi/linux/lustre/lustre_idl.h
+++ b/drivers/staging/lustre/include/uapi/linux/lustre/lustre_idl.h
@@ -2111,11 +2111,19 @@ struct mgs_nidtbl_entry {
 	} u;
 };
 
+enum {
+	CONFIG_T_CONFIG  = 0,
+	CONFIG_T_SPTLRPC = 1,
+	CONFIG_T_RECOVER = 2,
+	CONFIG_T_PARAMS  = 3,
+	CONFIG_T_MAX
+};
+
 struct mgs_config_body {
 	char		mcb_name[MTI_NAME_MAXLEN]; /* logname */
 	__u64		mcb_offset;    /* next index of config log to request */
 	__u16		mcb_type;      /* type of log: CONFIG_T_[CONFIG|RECOVER] */
-	__u8		mcb_reserved;
+	__u8		mcb_nm_cur_pass;
 	__u8		mcb_bits;      /* bits unit size of config log */
 	__u32		mcb_units;     /* # of units for bulk transfer */
 };
diff --git a/drivers/staging/lustre/lustre/include/obd_class.h b/drivers/staging/lustre/lustre/include/obd_class.h
index 184da99..647cc22 100644
--- a/drivers/staging/lustre/lustre/include/obd_class.h
+++ b/drivers/staging/lustre/lustre/include/obd_class.h
@@ -156,16 +156,16 @@ struct config_llog_instance {
 int class_config_parse_llog(const struct lu_env *env, struct llog_ctxt *ctxt,
 			    char *name, struct config_llog_instance *cfg);
 
-#define CONFIG_T_CONFIG		BIT(0)
-#define CONFIG_T_SPTLRPC	BIT(1)
-#define CONFIG_T_RECOVER	BIT(2)
-#define CONFIG_T_PARAMS		BIT(3)
+#define CONFIG_SUB_CONFIG	BIT(0)
+#define CONFIG_SUB_SPTLRPC	BIT(1)
+#define CONFIG_SUB_RECOVER	BIT(2)
+#define CONFIG_SUB_PARAMS	BIT(3)
 
 /* Sub clds should be attached to the config_llog_data when processing
  * config log for client or server target.
  */
-#define CONFIG_SUB_CLIENT	(CONFIG_T_SPTLRPC | CONFIG_T_RECOVER | \
-				 CONFIG_T_PARAMS)
+#define CONFIG_SUB_CLIENT	(CONFIG_SUB_SPTLRPC | CONFIG_SUB_RECOVER | \
+				 CONFIG_SUB_PARAMS)
 
 #define PARAMS_FILENAME	"params"
 #define LCTL_UPCALL	"lctl"
diff --git a/drivers/staging/lustre/lustre/mgc/mgc_request.c b/drivers/staging/lustre/lustre/mgc/mgc_request.c
index 06fcc7e..833e6a0 100644
--- a/drivers/staging/lustre/lustre/mgc/mgc_request.c
+++ b/drivers/staging/lustre/lustre/mgc/mgc_request.c
@@ -315,7 +315,7 @@ struct config_llog_data *do_config_log_add(struct obd_device *obd,
 	memcpy(seclogname, logname, ptr - logname);
 	strcpy(seclogname + (ptr - logname), "-sptlrpc");
 
-	if (cfg->cfg_sub_clds & CONFIG_T_SPTLRPC) {
+	if (cfg->cfg_sub_clds & CONFIG_SUB_SPTLRPC) {
 		sptlrpc_cld = config_log_find_or_add(obd, seclogname, NULL,
 						     CONFIG_T_SPTLRPC, cfg);
 		if (IS_ERR(sptlrpc_cld)) {
@@ -325,7 +325,7 @@ struct config_llog_data *do_config_log_add(struct obd_device *obd,
 		}
 	}
 
-	if (cfg->cfg_sub_clds & CONFIG_T_PARAMS) {
+	if (cfg->cfg_sub_clds & CONFIG_SUB_PARAMS) {
 		params_cld = config_log_find_or_add(obd, PARAMS_FILENAME, sb,
 						    CONFIG_T_PARAMS, cfg);
 		if (IS_ERR(params_cld)) {
@@ -345,7 +345,7 @@ struct config_llog_data *do_config_log_add(struct obd_device *obd,
 
 	LASSERT(lsi->lsi_lmd);
 	if (!(lsi->lsi_lmd->lmd_flags & LMD_FLG_NOIR) &&
-	    cfg->cfg_sub_clds & CONFIG_T_RECOVER) {
+	    cfg->cfg_sub_clds & CONFIG_SUB_RECOVER) {
 		ptr = strrchr(seclogname, '-');
 		if (ptr) {
 			*ptr = 0;
diff --git a/drivers/staging/lustre/lustre/ptlrpc/wiretest.c b/drivers/staging/lustre/lustre/ptlrpc/wiretest.c
index 2f081ed..09b1298 100644
--- a/drivers/staging/lustre/lustre/ptlrpc/wiretest.c
+++ b/drivers/staging/lustre/lustre/ptlrpc/wiretest.c
@@ -3629,6 +3629,91 @@ void lustre_assert_wire_constants(void)
 	LASSERTF((int)sizeof(((struct mgs_target_info *)0)->mti_params) == 4096, "found %lld\n",
 		 (long long)(int)sizeof(((struct mgs_target_info *)0)->mti_params));
 
+	/* Checks for struct mgs_nidtbl_entry */
+	LASSERTF((int)sizeof(struct mgs_nidtbl_entry) == 24, "found %lld\n",
+		 (long long)(int)sizeof(struct mgs_nidtbl_entry));
+	LASSERTF((int)offsetof(struct mgs_nidtbl_entry, mne_version) == 0, "found %lld\n",
+		 (long long)(int)offsetof(struct mgs_nidtbl_entry, mne_version));
+	LASSERTF((int)sizeof(((struct mgs_nidtbl_entry *)0)->mne_version) == 8, "found %lld\n",
+		 (long long)(int)sizeof(((struct mgs_nidtbl_entry *)0)->mne_version));
+	LASSERTF((int)offsetof(struct mgs_nidtbl_entry, mne_instance) == 8, "found %lld\n",
+		 (long long)(int)offsetof(struct mgs_nidtbl_entry, mne_instance));
+	LASSERTF((int)sizeof(((struct mgs_nidtbl_entry *)0)->mne_instance) == 4, "found %lld\n",
+		 (long long)(int)sizeof(((struct mgs_nidtbl_entry *)0)->mne_instance));
+	LASSERTF((int)offsetof(struct mgs_nidtbl_entry, mne_index) == 12, "found %lld\n",
+		 (long long)(int)offsetof(struct mgs_nidtbl_entry, mne_index));
+	LASSERTF((int)sizeof(((struct mgs_nidtbl_entry *)0)->mne_index) == 4, "found %lld\n",
+		 (long long)(int)sizeof(((struct mgs_nidtbl_entry *)0)->mne_index));
+	LASSERTF((int)offsetof(struct mgs_nidtbl_entry, mne_length) == 16, "found %lld\n",
+		 (long long)(int)offsetof(struct mgs_nidtbl_entry, mne_length));
+	LASSERTF((int)sizeof(((struct mgs_nidtbl_entry *)0)->mne_length) == 4, "found %lld\n",
+		 (long long)(int)sizeof(((struct mgs_nidtbl_entry *)0)->mne_length));
+	LASSERTF((int)offsetof(struct mgs_nidtbl_entry, mne_type) == 20, "found %lld\n",
+		 (long long)(int)offsetof(struct mgs_nidtbl_entry, mne_type));
+	LASSERTF((int)sizeof(((struct mgs_nidtbl_entry *)0)->mne_type) == 1, "found %lld\n",
+		 (long long)(int)sizeof(((struct mgs_nidtbl_entry *)0)->mne_type));
+	LASSERTF((int)offsetof(struct mgs_nidtbl_entry, mne_nid_type) == 21, "found %lld\n",
+		 (long long)(int)offsetof(struct mgs_nidtbl_entry, mne_nid_type));
+	LASSERTF((int)sizeof(((struct mgs_nidtbl_entry *)0)->mne_nid_type) == 1, "found %lld\n",
+		 (long long)(int)sizeof(((struct mgs_nidtbl_entry *)0)->mne_nid_type));
+	LASSERTF((int)offsetof(struct mgs_nidtbl_entry, mne_nid_size) == 22, "found %lld\n",
+		 (long long)(int)offsetof(struct mgs_nidtbl_entry, mne_nid_size));
+	LASSERTF((int)sizeof(((struct mgs_nidtbl_entry *)0)->mne_nid_size) == 1, "found %lld\n",
+		 (long long)(int)sizeof(((struct mgs_nidtbl_entry *)0)->mne_nid_size));
+	LASSERTF((int)offsetof(struct mgs_nidtbl_entry, mne_nid_count) == 23, "found %lld\n",
+		 (long long)(int)offsetof(struct mgs_nidtbl_entry, mne_nid_count));
+	LASSERTF((int)sizeof(((struct mgs_nidtbl_entry *)0)->mne_nid_count) == 1, "found %lld\n",
+		 (long long)(int)sizeof(((struct mgs_nidtbl_entry *)0)->mne_nid_count));
+	LASSERTF((int)offsetof(struct mgs_nidtbl_entry, u.nids[0]) == 24, "found %lld\n",
+		 (long long)(int)offsetof(struct mgs_nidtbl_entry, u.nids[0]));
+	LASSERTF((int)sizeof(((struct mgs_nidtbl_entry *)0)->u.nids[0]) == 8, "found %lld\n",
+		 (long long)(int)sizeof(((struct mgs_nidtbl_entry *)0)->u.nids[0]));
+
+	/* Checks for struct mgs_config_body */
+	LASSERTF((int)sizeof(struct mgs_config_body) == 80, "found %lld\n",
+		 (long long)(int)sizeof(struct mgs_config_body));
+	LASSERTF((int)offsetof(struct mgs_config_body, mcb_name) == 0, "found %lld\n",
+		 (long long)(int)offsetof(struct mgs_config_body, mcb_name));
+	LASSERTF((int)sizeof(((struct mgs_config_body *)0)->mcb_name) == 64, "found %lld\n",
+		 (long long)(int)sizeof(((struct mgs_config_body *)0)->mcb_name));
+	LASSERTF((int)offsetof(struct mgs_config_body, mcb_offset) == 64, "found %lld\n",
+		 (long long)(int)offsetof(struct mgs_config_body, mcb_offset));
+	LASSERTF((int)sizeof(((struct mgs_config_body *)0)->mcb_offset) == 8, "found %lld\n",
+		 (long long)(int)sizeof(((struct mgs_config_body *)0)->mcb_offset));
+	LASSERTF((int)offsetof(struct mgs_config_body, mcb_type) == 72, "found %lld\n",
+		 (long long)(int)offsetof(struct mgs_config_body, mcb_type));
+	LASSERTF((int)sizeof(((struct mgs_config_body *)0)->mcb_type) == 2, "found %lld\n",
+		 (long long)(int)sizeof(((struct mgs_config_body *)0)->mcb_type));
+	LASSERTF((int)offsetof(struct mgs_config_body, mcb_nm_cur_pass) == 74, "found %lld\n",
+		 (long long)(int)offsetof(struct mgs_config_body, mcb_nm_cur_pass));
+	LASSERTF((int)sizeof(((struct mgs_config_body *)0)->mcb_nm_cur_pass) == 1, "found %lld\n",
+		 (long long)(int)sizeof(((struct mgs_config_body *)0)->mcb_nm_cur_pass));
+	LASSERTF((int)offsetof(struct mgs_config_body, mcb_bits) == 75, "found %lld\n",
+		 (long long)(int)offsetof(struct mgs_config_body, mcb_bits));
+	LASSERTF((int)sizeof(((struct mgs_config_body *)0)->mcb_bits) == 1, "found %lld\n",
+		 (long long)(int)sizeof(((struct mgs_config_body *)0)->mcb_bits));
+	LASSERTF((int)offsetof(struct mgs_config_body, mcb_units) == 76, "found %lld\n",
+		 (long long)(int)offsetof(struct mgs_config_body, mcb_units));
+	LASSERTF((int)sizeof(((struct mgs_config_body *)0)->mcb_units) == 4, "found %lld\n",
+		 (long long)(int)sizeof(((struct mgs_config_body *)0)->mcb_units));
+
+	BUILD_BUG_ON(CONFIG_T_CONFIG != 0);
+	BUILD_BUG_ON(CONFIG_T_SPTLRPC != 1);
+	BUILD_BUG_ON(CONFIG_T_RECOVER != 2);
+	BUILD_BUG_ON(CONFIG_T_PARAMS != 3);
+
+	/* Checks for struct mgs_config_res */
+	LASSERTF((int)sizeof(struct mgs_config_res) == 16, "found %lld\n",
+		 (long long)(int)sizeof(struct mgs_config_res));
+	LASSERTF((int)offsetof(struct mgs_config_res, mcr_offset) == 0, "found %lld\n",
+		 (long long)(int)offsetof(struct mgs_config_res, mcr_offset));
+	LASSERTF((int)sizeof(((struct mgs_config_res *)0)->mcr_offset) == 8, "found %lld\n",
+		 (long long)(int)sizeof(((struct mgs_config_res *)0)->mcr_offset));
+	LASSERTF((int)offsetof(struct mgs_config_res, mcr_size) == 8, "found %lld\n",
+		 (long long)(int)offsetof(struct mgs_config_res, mcr_size));
+	LASSERTF((int)sizeof(((struct mgs_config_res *)0)->mcr_size) == 8, "found %lld\n",
+		 (long long)(int)sizeof(((struct mgs_config_res *)0)->mcr_size));
+
 	/* Checks for struct lustre_capa */
 	LASSERTF((int)sizeof(struct lustre_capa) == 120, "found %lld\n",
 		 (long long)(int)sizeof(struct lustre_capa));
-- 
1.8.3.1


From jsimmons at infradead.org  Tue Jul 31 02:26:08 2018
From: jsimmons at infradead.org (James Simmons)
Date: Mon, 30 Jul 2018 22:26:08 -0400
Subject: [lustre-devel] [PATCH 16/31] lustre: obd: remove OBD_NOTIFY_SYNC{,
 _NONBLOCK}
In-Reply-To: <1533003983-29311-1-git-send-email-jsimmons@infradead.org>
References: <1533003983-29311-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1533003983-29311-17-git-send-email-jsimmons@infradead.org>

From: "John L. Hammond" <jhammond at whamcloud.com>

None of the OBD notify handlers listen for OBD_NOTIFY_SYNC{,_NONBLOCK}
events so remove them and related code in lov_notify().

Signed-off-by: John L. Hammond <jhammond at whamcloud.com>
WC-id: https://jira.whamcloud.com/browse/LU-8403
Reviewed-on: https://review.whamcloud.com/21421
Reviewed-by: Andreas Dilger <adilger at whamcloud.com>
Reviewed-by: James Simmons <uja.ornl at yahoo.com>
Reviewed-by: Oleg Drokin <green at whamcloud.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/lustre/include/obd.h |  3 --
 drivers/staging/lustre/lustre/lov/lov_obd.c | 52 +++++------------------------
 2 files changed, 9 insertions(+), 46 deletions(-)

diff --git a/drivers/staging/lustre/lustre/include/obd.h b/drivers/staging/lustre/lustre/include/obd.h
index 62f85a1..5bf2be8 100644
--- a/drivers/staging/lustre/lustre/include/obd.h
+++ b/drivers/staging/lustre/lustre/include/obd.h
@@ -491,9 +491,6 @@ enum obd_notify_event {
 	OBD_NOTIFY_INACTIVE,
 	/* Connect data for import were changed */
 	OBD_NOTIFY_OCD,
-	/* Sync request */
-	OBD_NOTIFY_SYNC_NONBLOCK,
-	OBD_NOTIFY_SYNC,
 	/* Configuration event */
 	OBD_NOTIFY_CONFIG,
 	/* Administratively deactivate/activate event */
diff --git a/drivers/staging/lustre/lustre/lov/lov_obd.c b/drivers/staging/lustre/lustre/lov/lov_obd.c
index 0dd471c..85d3b29 100644
--- a/drivers/staging/lustre/lustre/lov/lov_obd.c
+++ b/drivers/staging/lustre/lustre/lov/lov_obd.c
@@ -429,10 +429,8 @@ static int lov_notify(struct obd_device *obd, struct obd_device *watched,
 	struct lov_obd *lov = &obd->u.lov;
 
 	down_read(&lov->lov_notify_lock);
-	if (!lov->lov_connects) {
-		up_read(&lov->lov_notify_lock);
-		return rc;
-	}
+	if (!lov->lov_connects)
+		goto out_notify_lock;
 
 	if (ev == OBD_NOTIFY_ACTIVE || ev == OBD_NOTIFY_INACTIVE ||
 	    ev == OBD_NOTIFY_ACTIVATE || ev == OBD_NOTIFY_DEACTIVATE) {
@@ -441,12 +439,13 @@ static int lov_notify(struct obd_device *obd, struct obd_device *watched,
 		LASSERT(watched);
 
 		if (strcmp(watched->obd_type->typ_name, LUSTRE_OSC_NAME)) {
-			up_read(&lov->lov_notify_lock);
 			CERROR("unexpected notification of %s %s!\n",
 			       watched->obd_type->typ_name,
 			       watched->obd_name);
-			return -EINVAL;
+			rc = -EINVAL;
+			goto out_notify_lock;
 		}
+
 		uuid = &watched->u.cli.cl_target_uuid;
 
 		/* Set OSC as active before notifying the observer, so the
@@ -454,53 +453,20 @@ static int lov_notify(struct obd_device *obd, struct obd_device *watched,
 		 */
 		rc = lov_set_osc_active(obd, uuid, ev);
 		if (rc < 0) {
-			up_read(&lov->lov_notify_lock);
 			CERROR("event(%d) of %s failed: %d\n", ev,
 			       obd_uuid2str(uuid), rc);
-			return rc;
+			goto out_notify_lock;
 		}
 		/* active event should be pass lov target index as data */
 		data = &rc;
 	}
 
 	/* Pass the notification up the chain. */
-	if (watched) {
-		rc = obd_notify_observer(obd, watched, ev, data);
-	} else {
-		/* NULL watched means all osc's in the lov (only for syncs) */
-		/* sync event should be send lov idx as data */
-		struct lov_obd *lov = &obd->u.lov;
-		int i, is_sync;
-
-		data = &i;
-		is_sync = (ev == OBD_NOTIFY_SYNC) ||
-			  (ev == OBD_NOTIFY_SYNC_NONBLOCK);
-
-		obd_getref(obd);
-		for (i = 0; i < lov->desc.ld_tgt_count; i++) {
-			if (!lov->lov_tgts[i])
-				continue;
-
-			/* don't send sync event if target not
-			 * connected/activated
-			 */
-			if (is_sync &&  !lov->lov_tgts[i]->ltd_active)
-				continue;
-
-			rc = obd_notify_observer(obd, lov->lov_tgts[i]->ltd_obd,
-						 ev, data);
-			if (rc) {
-				CERROR("%s: notify %s of %s failed %d\n",
-				       obd->obd_name,
-				       obd->obd_observer->obd_name,
-				       lov->lov_tgts[i]->ltd_obd->obd_name,
-				       rc);
-			}
-		}
-		obd_putref(obd);
-	}
+	rc = obd_notify_observer(obd, watched, ev, data);
 
+out_notify_lock:
 	up_read(&lov->lov_notify_lock);
+
 	return rc;
 }
 
-- 
1.8.3.1


From jsimmons at infradead.org  Tue Jul 31 02:26:09 2018
From: jsimmons at infradead.org (James Simmons)
Date: Mon, 30 Jul 2018 22:26:09 -0400
Subject: [lustre-devel] [PATCH 17/31] lustre: obdclass: handle early
 requests vs CT registering
In-Reply-To: <1533003983-29311-1-git-send-email-jsimmons@infradead.org>
References: <1533003983-29311-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1533003983-29311-18-git-send-email-jsimmons@infradead.org>

From: Bruno Faccini <bruno.faccini at intel.com>

This patch addresses cases where CDT may start to send requests
before CT has fully registered with all MDTs and thus when the KUC
pipe kernel side has still not been initialized in
lmv_hsm_ct_register().

This will avoid Oops'es due to kkuc_groups[KUC_GRP_HSM] being
uninitialized/zero'ed and we rely on CDT to later retry.

Signed-off-by: Bruno Faccini <bruno.faccini at intel.com>
WC-id: https://jira.whamcloud.com/browse/LU-9038
Reviewed-on: https://review.whamcloud.com/25050
Reviewed-by: Quentin Bouget <quentin.bouget at cea.fr>
Reviewed-by: Henri Doreau <henri.doreau at cea.fr>
Reviewed-by: Oleg Drokin <green at whamcloud.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/lustre/obdclass/kernelcomm.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/drivers/staging/lustre/lustre/obdclass/kernelcomm.c b/drivers/staging/lustre/lustre/obdclass/kernelcomm.c
index 63067a7..304288d 100644
--- a/drivers/staging/lustre/lustre/obdclass/kernelcomm.c
+++ b/drivers/staging/lustre/lustre/obdclass/kernelcomm.c
@@ -183,6 +183,14 @@ int libcfs_kkuc_group_put(unsigned int group, void *payload)
 	int one_success = 0;
 
 	down_write(&kg_sem);
+
+	if (unlikely(!kkuc_groups[group].next) ||
+	    unlikely(OBD_FAIL_CHECK(OBD_FAIL_MDS_HSM_CT_REGISTER_NET))) {
+		/* no agent have fully registered, CDT will retry */
+		up_write(&kg_sem);
+		return -EAGAIN;
+	}
+
 	list_for_each_entry(reg, &kkuc_groups[group], kr_chain) {
 		if (reg->kr_fp) {
 			rc = libcfs_kkuc_msg_put(reg->kr_fp, payload);
-- 
1.8.3.1


From jsimmons at infradead.org  Tue Jul 31 02:26:14 2018
From: jsimmons at infradead.org (James Simmons)
Date: Mon, 30 Jul 2018 22:26:14 -0400
Subject: [lustre-devel] [PATCH 22/31] lustre: libcfs: reduce libcfs checksum
 speed test time
In-Reply-To: <1533003983-29311-1-git-send-email-jsimmons@infradead.org>
References: <1533003983-29311-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1533003983-29311-23-git-send-email-jsimmons@infradead.org>

From: Andreas Dilger <adilger at whamcloud.com>

Loading the libcfs module is getting increasingly slow due to
multiple checksum types being speed tested at startup (8 different
checksums * 1s per checksum).

Reduce the number of checksum algorithms checked at module load
time to the ones that are actually need the speed (i.e. the bulk
data checksums), and reduce the amount of time taken to compute the
checksum. The other checksum types typically do not need the speed,
but rather are selected by the configuration.

Precompute the checksum speeds and supported types for the OST so
they are not recomputed for each new client that connects.

This reduces the module load time from 8.0s to 0.76s in my testing.

Signed-off-by: Andreas Dilger <adilger at whamcloud.com>
WC-id: https://jira.whamcloud.com/browse/LU-9201
Reviewed-on: https://review.whamcloud.com/25923
Reviewed-by: Alex Zhuravlev <bzzz at whamcloud.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong at gmail.com>
Reviewed-by: Oleg Drokin <green at whamcloud.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 .../lustre/include/linux/libcfs/libcfs_crypto.h    |  4 ++-
 drivers/staging/lustre/lnet/libcfs/linux-crypto.c  | 30 +++++++++++++++++-----
 2 files changed, 26 insertions(+), 8 deletions(-)

diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_crypto.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_crypto.h
index 176fae7..ca8620b 100644
--- a/drivers/staging/lustre/include/linux/libcfs/libcfs_crypto.h
+++ b/drivers/staging/lustre/include/linux/libcfs/libcfs_crypto.h
@@ -46,13 +46,15 @@ enum cfs_crypto_hash_alg {
 	CFS_HASH_ALG_NULL       = 0,
 	CFS_HASH_ALG_ADLER32,
 	CFS_HASH_ALG_CRC32,
+	CFS_HASH_ALG_CRC32C,
+	/* hashes before here will be speed-tested at module load */
 	CFS_HASH_ALG_MD5,
 	CFS_HASH_ALG_SHA1,
 	CFS_HASH_ALG_SHA256,
 	CFS_HASH_ALG_SHA384,
 	CFS_HASH_ALG_SHA512,
-	CFS_HASH_ALG_CRC32C,
 	CFS_HASH_ALG_MAX,
+	CFS_HASH_ALG_SPEED_MAX = CFS_HASH_ALG_MD5,
 	CFS_HASH_ALG_UNKNOWN	= 0xff
 };
 
diff --git a/drivers/staging/lustre/lnet/libcfs/linux-crypto.c b/drivers/staging/lustre/lnet/libcfs/linux-crypto.c
index cfff54d..b206e3c 100644
--- a/drivers/staging/lustre/lnet/libcfs/linux-crypto.c
+++ b/drivers/staging/lustre/lnet/libcfs/linux-crypto.c
@@ -300,7 +300,10 @@ int cfs_crypto_hash_final(struct ahash_request *req,
 /**
  * Compute the speed of specified hash function
  *
- * Run a speed test on the given hash algorithm on buffer of the given size.
+ * Run a speed test on the given hash algorithm on buffer using a 1MB buffer
+ * size.  This is a reasonable buffer size for Lustre RPCs, even if the actual
+ * RPC size is larger or smaller.
+ *
  * The speed is stored internally in the cfs_crypto_hash_speeds[] array, and
  * is available through the cfs_crypto_hash_speed() function.
  *
@@ -329,8 +332,8 @@ static void cfs_crypto_performance_test(enum cfs_crypto_hash_alg hash_alg)
 	memset(buf, 0xAD, PAGE_SIZE);
 	kunmap(page);
 
-	for (start = jiffies, end = start + msecs_to_jiffies(MSEC_PER_SEC),
-	     bcount = 0; time_before(jiffies, end); bcount++) {
+	for (start = jiffies, end = start + msecs_to_jiffies(MSEC_PER_SEC / 4),
+	     bcount = 0; time_before(jiffies, end) && err == 0; bcount++) {
 		struct ahash_request *hdesc;
 		int i;
 
@@ -373,8 +376,12 @@ static void cfs_crypto_performance_test(enum cfs_crypto_hash_alg hash_alg)
 /**
  * hash speed in Mbytes per second for valid hash algorithm
  *
- * Return the performance of the specified \a hash_alg that was previously
- * computed using cfs_crypto_performance_test().
+ * Return the performance of the specified \a hash_alg that was
+ * computed using cfs_crypto_performance_test().  If the performance
+ * has not yet been computed, do that when it is first requested.
+ * That avoids computing the speed when it is not actually needed.
+ * To avoid competing threads computing the checksum speed at the
+ * same time, only compute a single checksum speed at one time.
  *
  * \param[in] hash_alg	hash algorithm id (CFS_HASH_ALG_*)
  *
@@ -384,8 +391,17 @@ static void cfs_crypto_performance_test(enum cfs_crypto_hash_alg hash_alg)
  */
 int cfs_crypto_hash_speed(enum cfs_crypto_hash_alg hash_alg)
 {
-	if (hash_alg < CFS_HASH_ALG_MAX)
+	if (hash_alg < CFS_HASH_ALG_MAX) {
+		if (unlikely(cfs_crypto_hash_speeds[hash_alg] == 0)) {
+			static DEFINE_MUTEX(crypto_hash_speed_mutex);
+
+			mutex_lock(&crypto_hash_speed_mutex);
+			if (cfs_crypto_hash_speeds[hash_alg] == 0)
+				cfs_crypto_performance_test(hash_alg);
+			mutex_unlock(&crypto_hash_speed_mutex);
+		}
 		return cfs_crypto_hash_speeds[hash_alg];
+	}
 	return -ENOENT;
 }
 EXPORT_SYMBOL(cfs_crypto_hash_speed);
@@ -412,7 +428,7 @@ static int cfs_crypto_test_hashes(void)
 {
 	enum cfs_crypto_hash_alg hash_alg;
 
-	for (hash_alg = 0; hash_alg < CFS_HASH_ALG_MAX; hash_alg++)
+	for (hash_alg = 0; hash_alg < CFS_HASH_ALG_SPEED_MAX; hash_alg++)
 		cfs_crypto_performance_test(hash_alg);
 
 	return 0;
-- 
1.8.3.1


From jsimmons at infradead.org  Tue Jul 31 02:26:05 2018
From: jsimmons at infradead.org (James Simmons)
Date: Mon, 30 Jul 2018 22:26:05 -0400
Subject: [lustre-devel] [PATCH 13/31] lustre: config: don't attach sub logs
 for LWP
In-Reply-To: <1533003983-29311-1-git-send-email-jsimmons@infradead.org>
References: <1533003983-29311-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1533003983-29311-14-git-send-email-jsimmons@infradead.org>

From: Niu Yawei <yawei.niu at intel.com>

Lustre target processes client log to retrieve MDT NIDs and start
LWPs, it goes the same code path of mgc_process_config() just like
processing the target config log, so that sub clds for security,
nodemap, param & recovery will be attached unnecessarily.

The mgc subsystem is used by both server and client. This change
allows us to cleanly handle the future case when the mgc layer
would be built with server code. This way server specific config
logs will only be processed when a server mount occurs.

Signed-off-by: Niu Yawei <yawei.niu at intel.com>
WC-id: https://jira.whamcloud.com/browse/LU-9081
Reviewed-on: https://review.whamcloud.com/25293
Reviewed-by: Fan Yong <fan.yong at intel.com>
Reviewed-by: Hongchao Zhang <hongchao at whamcloud.com>
Reviewed-by: Oleg Drokin <green at whamcloud.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/lustre/include/obd_class.h | 19 +++++++-----
 drivers/staging/lustre/lustre/llite/llite_lib.c   |  1 +
 drivers/staging/lustre/lustre/mgc/mgc_request.c   | 37 +++++++++++++----------
 3 files changed, 34 insertions(+), 23 deletions(-)

diff --git a/drivers/staging/lustre/lustre/include/obd_class.h b/drivers/staging/lustre/lustre/include/obd_class.h
index adfe2ab..e772e3d 100644
--- a/drivers/staging/lustre/lustre/include/obd_class.h
+++ b/drivers/staging/lustre/lustre/include/obd_class.h
@@ -153,17 +153,22 @@ struct config_llog_instance {
 	llog_cb_t	    cfg_callback;
 	int		    cfg_last_idx; /* for partial llog processing */
 	int		    cfg_flags;
+	u32		    cfg_sub_clds;
 };
 
 int class_config_parse_llog(const struct lu_env *env, struct llog_ctxt *ctxt,
 			    char *name, struct config_llog_instance *cfg);
-enum {
-	CONFIG_T_CONFIG  = 0,
-	CONFIG_T_SPTLRPC = 1,
-	CONFIG_T_RECOVER = 2,
-	CONFIG_T_PARAMS  = 3,
-	CONFIG_T_MAX     = 4
-};
+
+#define CONFIG_T_CONFIG		BIT(0)
+#define CONFIG_T_SPTLRPC	BIT(1)
+#define CONFIG_T_RECOVER	BIT(2)
+#define CONFIG_T_PARAMS		BIT(3)
+
+/* Sub clds should be attached to the config_llog_data when processing
+ * config log for client or server target.
+ */
+#define CONFIG_SUB_CLIENT	(CONFIG_T_SPTLRPC | CONFIG_T_RECOVER | \
+				 CONFIG_T_PARAMS)
 
 #define PARAMS_FILENAME	"params"
 #define LCTL_UPCALL	"lctl"
diff --git a/drivers/staging/lustre/lustre/llite/llite_lib.c b/drivers/staging/lustre/lustre/llite/llite_lib.c
index 72b118a..71eb42d 100644
--- a/drivers/staging/lustre/lustre/llite/llite_lib.c
+++ b/drivers/staging/lustre/lustre/llite/llite_lib.c
@@ -949,6 +949,7 @@ int ll_fill_super(struct super_block *sb)
 	cfg->cfg_instance = sb;
 	cfg->cfg_uuid = lsi->lsi_llsbi->ll_sb_uuid;
 	cfg->cfg_callback = class_config_llog_handler;
+	cfg->cfg_sub_clds = CONFIG_SUB_CLIENT;
 	/* set up client obds */
 	err = lustre_process_log(sb, profilenm, cfg);
 	if (err < 0)
diff --git a/drivers/staging/lustre/lustre/mgc/mgc_request.c b/drivers/staging/lustre/lustre/mgc/mgc_request.c
index 82acac0..06fcc7e 100644
--- a/drivers/staging/lustre/lustre/mgc/mgc_request.c
+++ b/drivers/staging/lustre/lustre/mgc/mgc_request.c
@@ -293,8 +293,8 @@ struct config_llog_data *do_config_log_add(struct obd_device *obd,
 {
 	struct lustre_sb_info *lsi = s2lsi(sb);
 	struct config_llog_data *cld;
-	struct config_llog_data *sptlrpc_cld;
-	struct config_llog_data *params_cld;
+	struct config_llog_data *sptlrpc_cld = NULL;
+	struct config_llog_data *params_cld = NULL;
 	struct config_llog_data *recover_cld = NULL;
 	char			seclogname[32];
 	char			*ptr;
@@ -315,21 +315,25 @@ struct config_llog_data *do_config_log_add(struct obd_device *obd,
 	memcpy(seclogname, logname, ptr - logname);
 	strcpy(seclogname + (ptr - logname), "-sptlrpc");
 
-	sptlrpc_cld = config_log_find_or_add(obd, seclogname, NULL,
-					     CONFIG_T_SPTLRPC, cfg);
-	if (IS_ERR(sptlrpc_cld)) {
-		CERROR("can't create sptlrpc log: %s\n", seclogname);
-		rc = PTR_ERR(sptlrpc_cld);
-		goto out_err;
+	if (cfg->cfg_sub_clds & CONFIG_T_SPTLRPC) {
+		sptlrpc_cld = config_log_find_or_add(obd, seclogname, NULL,
+						     CONFIG_T_SPTLRPC, cfg);
+		if (IS_ERR(sptlrpc_cld)) {
+			CERROR("can't create sptlrpc log: %s\n", seclogname);
+			rc = PTR_ERR(sptlrpc_cld);
+			goto out_err;
+		}
 	}
 
-	params_cld = config_log_find_or_add(obd, PARAMS_FILENAME, sb,
-					    CONFIG_T_PARAMS, cfg);
-	if (IS_ERR(params_cld)) {
-		rc = PTR_ERR(params_cld);
-		CERROR("%s: can't create params log: rc = %d\n",
-		       obd->obd_name, rc);
-		goto out_sptlrpc;
+	if (cfg->cfg_sub_clds & CONFIG_T_PARAMS) {
+		params_cld = config_log_find_or_add(obd, PARAMS_FILENAME, sb,
+						    CONFIG_T_PARAMS, cfg);
+		if (IS_ERR(params_cld)) {
+			rc = PTR_ERR(params_cld);
+			CERROR("%s: can't create params log: rc = %d\n",
+			       obd->obd_name, rc);
+			goto out_sptlrpc;
+		}
 	}
 
 	cld = do_config_log_add(obd, logname, CONFIG_T_CONFIG, cfg, sb);
@@ -340,7 +344,8 @@ struct config_llog_data *do_config_log_add(struct obd_device *obd,
 	}
 
 	LASSERT(lsi->lsi_lmd);
-	if (!(lsi->lsi_lmd->lmd_flags & LMD_FLG_NOIR)) {
+	if (!(lsi->lsi_lmd->lmd_flags & LMD_FLG_NOIR) &&
+	    cfg->cfg_sub_clds & CONFIG_T_RECOVER) {
 		ptr = strrchr(seclogname, '-');
 		if (ptr) {
 			*ptr = 0;
-- 
1.8.3.1


From jsimmons at infradead.org  Tue Jul 31 02:25:57 2018
From: jsimmons at infradead.org (James Simmons)
Date: Mon, 30 Jul 2018 22:25:57 -0400
Subject: [lustre-devel] [PATCH 05/31] lustre: llite: return small device
 numbers for compat stat()
In-Reply-To: <1533003983-29311-1-git-send-email-jsimmons@infradead.org>
References: <1533003983-29311-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1533003983-29311-6-git-send-email-jsimmons@infradead.org>

From: "John L. Hammond" <jhammond at whamcloud.com>

The compat_sys_*stat*() syscalls will fail unless the devices majors
and minors are both less than 256. So in ll_getattr_it(), if we are in
32 bit compat mode then coerce the device numbers in to the expected
format.

Signed-off-by: John L. Hammond <jhammond at whamcloud.com>
WC-id: https://jira.whamcloud.com/browse/LU-8855
Reviewed-on: https://review.whamcloud.com/23877
Reviewed-by: Oleg Drokin <green at whamcloud.com>
Reviewed-by: Andreas Dilger <adilger at whamcloud.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/lustre/llite/file.c | 12 +++++++++---
 1 file changed, 9 insertions(+), 3 deletions(-)

diff --git a/drivers/staging/lustre/lustre/llite/file.c b/drivers/staging/lustre/lustre/llite/file.c
index 3f0f379..684877c 100644
--- a/drivers/staging/lustre/lustre/llite/file.c
+++ b/drivers/staging/lustre/lustre/llite/file.c
@@ -3248,14 +3248,20 @@ int ll_getattr(const struct path *path, struct kstat *stat,
 	OBD_FAIL_TIMEOUT(OBD_FAIL_GETATTR_DELAY, 30);
 
 	stat->dev = inode->i_sb->s_dev;
-	if (ll_need_32bit_api(sbi))
+	if (ll_need_32bit_api(sbi)) {
 		stat->ino = cl_fid_build_ino(&lli->lli_fid, 1);
-	else
+		stat->dev = MKDEV(MAJOR(inode->i_sb->s_dev) & 0xff,
+				  MINOR(inode->i_sb->s_dev) & 0xff);
+		stat->rdev = MKDEV(MAJOR(inode->i_rdev) & 0xff,
+				   MINOR(inode->i_rdev) & 0xff);
+	} else {
+		stat->dev = inode->i_sb->s_dev;
+		stat->rdev = inode->i_rdev;
 		stat->ino = inode->i_ino;
+	}
 	stat->mode = inode->i_mode;
 	stat->uid = inode->i_uid;
 	stat->gid = inode->i_gid;
-	stat->rdev = inode->i_rdev;
 	stat->atime = inode->i_atime;
 	stat->mtime = inode->i_mtime;
 	stat->ctime = inode->i_ctime;
-- 
1.8.3.1


From jsimmons at infradead.org  Tue Jul 31 02:26:02 2018
From: jsimmons at infradead.org (James Simmons)
Date: Mon, 30 Jul 2018 22:26:02 -0400
Subject: [lustre-devel] [PATCH 10/31] lustre: lnet: removal of obsolete LNDs
In-Reply-To: <1533003983-29311-1-git-send-email-jsimmons@infradead.org>
References: <1533003983-29311-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1533003983-29311-11-git-send-email-jsimmons@infradead.org>

From: Sonia Sharma <sharmaso at whamcloud.com>

Obsolete LNDs were already removed. Commented out the name<->network
number mapping for the obsolete LNDs.

Signed-off-by: Sonia Sharma <sharmaso at whamcloud.com>
WC-id: https://jira.whamcloud.com/browse/LU-8769
Reviewed-on: https://review.whamcloud.com/23621
Reviewed-by: Doug Oucharek <dougso at me.com>
Reviewed-by: Olaf Weber <olaf at sgi.com>
Reviewed-by: James Simmons <uja.ornl at yahoo.com>
Reviewed-by: Oleg Drokin <green at whamcloud.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 .../staging/lustre/include/uapi/linux/lnet/nidstr.h    | 18 +++++++++---------
 drivers/staging/lustre/lnet/lnet/api-ni.c              |  6 ------
 2 files changed, 9 insertions(+), 15 deletions(-)

diff --git a/drivers/staging/lustre/include/uapi/linux/lnet/nidstr.h b/drivers/staging/lustre/include/uapi/linux/lnet/nidstr.h
index 882074e..3354e5a 100644
--- a/drivers/staging/lustre/include/uapi/linux/lnet/nidstr.h
+++ b/drivers/staging/lustre/include/uapi/linux/lnet/nidstr.h
@@ -38,18 +38,18 @@ enum {
 	 * Only add to these values (i.e. don't ever change or redefine them):
 	 * network addresses depend on them...
 	 */
-	QSWLND		= 1,
+	/*QSWLND	= 1, removed v2_7_50			*/
 	SOCKLND		= 2,
-	GMLND		= 3,
-	PTLLND		= 4,
+	/*GMLND		= 3, removed v2_0_0-rc1a-16-gc660aac	*/
+	/*PTLLND	= 4, removed v2_7_50			*/
 	O2IBLND		= 5,
-	CIBLND		= 6,
-	OPENIBLND	= 7,
-	IIBLND		= 8,
+	/*CIBLND        = 6, removed v2_0_0-rc1a-175-gd2b8a0e	*/
+	/*OPENIBLND	= 7, removed v2_0_0-rc1a-175-gd2b8a0e	*/
+	/*IIBLND	= 8, removed v2_0_0-rc1a-175-gd2b8a0e	*/
 	LOLND		= 9,
-	RALND		= 10,
-	VIBLND		= 11,
-	MXLND		= 12,
+	/*RALND		= 10, removed v2_7_50_0-34-g8be9e41	*/
+	/*VIBLND	= 11, removed v2_0_0-rc1a-175-gd2b8a0e	*/
+	/*MXLND		= 12, removed v2_7_50_0-34-g8be9e41	*/
 	GNILND		= 13,
 	GNIIPLND	= 14,
 };
diff --git a/drivers/staging/lustre/lnet/lnet/api-ni.c b/drivers/staging/lustre/lnet/lnet/api-ni.c
index e517893..a949ac2 100644
--- a/drivers/staging/lustre/lnet/lnet/api-ni.c
+++ b/drivers/staging/lustre/lnet/lnet/api-ni.c
@@ -1207,12 +1207,6 @@ struct lnet_ni  *
 
 	LASSERT(libcfs_isknown_lnd(lnd_type));
 
-	if (lnd_type == CIBLND || lnd_type == OPENIBLND ||
-	    lnd_type == IIBLND || lnd_type == VIBLND) {
-		CERROR("LND %s obsoleted\n", libcfs_lnd2str(lnd_type));
-		goto failed0;
-	}
-
 	/* Make sure this new NI is unique. */
 	lnet_net_lock(LNET_LOCK_EX);
 	rc = lnet_net_unique(LNET_NIDNET(ni->ni_nid), &the_lnet.ln_nis);
-- 
1.8.3.1


From jsimmons at infradead.org  Tue Jul 31 02:26:04 2018
From: jsimmons at infradead.org (James Simmons)
Date: Mon, 30 Jul 2018 22:26:04 -0400
Subject: [lustre-devel] [PATCH 12/31] lustre: obdclass: obdclass module
 cleanup upon load error
In-Reply-To: <1533003983-29311-1-git-send-email-jsimmons@infradead.org>
References: <1533003983-29311-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1533003983-29311-13-git-send-email-jsimmons@infradead.org>

From: Bruno Faccini <bruno.faccini at intel.com>

Fix obdclass_init() error paths to proceed with cleanup.
This will particularly allow to no longer crash upon next
load attempt and this due to previous miscdevice not been
deregistered and thus still referenced in misc_list when
unmapped.

Signed-off-by: Bruno Faccini <bruno.faccini at intel.com>
WC-id: https://jira.whamcloud.com/browse/LU-6499
Reviewed-on: https://review.whamcloud.com/22544
Reviewed-by: Patrick Farrell <paf at cray.com>
Reviewed-by: James Simmons <uja.ornl at yahoo.com>
Reviewed-by: Oleg Drokin <green at whamcloud.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 .../staging/lustre/lustre/include/obd_support.h    |  1 +
 drivers/staging/lustre/lustre/obdclass/class_obd.c | 53 ++++++++++++++++++----
 2 files changed, 46 insertions(+), 8 deletions(-)

diff --git a/drivers/staging/lustre/lustre/include/obd_support.h b/drivers/staging/lustre/lustre/include/obd_support.h
index 87806e8..726cc4d 100644
--- a/drivers/staging/lustre/lustre/include/obd_support.h
+++ b/drivers/staging/lustre/lustre/include/obd_support.h
@@ -359,6 +359,7 @@
 #define OBD_FAIL_OBD_IDX_READ_NET	0x607
 #define OBD_FAIL_OBD_IDX_READ_BREAK	 0x608
 #define OBD_FAIL_OBD_NO_LRU		 0x609
+#define OBD_FAIL_OBDCLASS_MODULE_LOAD	 0x60a
 
 #define OBD_FAIL_TGT_REPLY_NET	   0x700
 #define OBD_FAIL_TGT_CONN_RACE	   0x701
diff --git a/drivers/staging/lustre/lustre/obdclass/class_obd.c b/drivers/staging/lustre/lustre/obdclass/class_obd.c
index 87327ef..04e55fc 100644
--- a/drivers/staging/lustre/lustre/obdclass/class_obd.c
+++ b/drivers/staging/lustre/lustre/obdclass/class_obd.c
@@ -469,19 +469,19 @@ static int __init obdclass_init(void)
 
 	err = obd_init_checks();
 	if (err)
-		return err;
+		goto cleanup_zombie_impexp;
 
 	class_init_uuidlist();
 	err = class_handle_init();
 	if (err)
-		return err;
+		goto cleanup_uuidlist;
 
 	INIT_LIST_HEAD(&obd_types);
 
 	err = misc_register(&obd_psdev);
 	if (err) {
 		CERROR("cannot register OBD miscdevices: err %d\n", err);
-		return err;
+		goto cleanup_class_handle;
 	}
 
 	/* This struct is already zeroed for us (static global) */
@@ -499,25 +499,62 @@ static int __init obdclass_init(void)
 
 	err = obd_init_caches();
 	if (err)
-		return err;
+		goto cleanup_deregister;
 
 	err = class_procfs_init();
 	if (err)
-		return err;
+		goto cleanup_caches;
 
 	err = obd_sysctl_init();
 	if (err)
-		return err;
+		goto cleanup_class_procfs;
 
 	err = lu_global_init();
 	if (err)
-		return err;
+		goto cleanup_class_procfs;
 
 	err = cl_global_init();
 	if (err != 0)
-		return err;
+		goto cleanup_lu_global;
 
 	err = llog_info_init();
+	if (err)
+		goto cleanup_cl_global;
+
+	/* simulate a late OOM situation now to require all
+	 * alloc'ed/initialized resources to be freed
+	 */
+	if (!OBD_FAIL_CHECK(OBD_FAIL_OBDCLASS_MODULE_LOAD))
+		return 0;
+
+	/* force error to ensure module will be unloaded/cleaned */
+	err = -ENOMEM;
+
+	llog_info_fini();
+
+cleanup_cl_global:
+	cl_global_fini();
+
+cleanup_lu_global:
+	lu_global_fini();
+
+cleanup_class_procfs:
+	class_procfs_clean();
+
+cleanup_caches:
+	obd_cleanup_caches();
+
+cleanup_deregister:
+	misc_deregister(&obd_psdev);
+
+cleanup_class_handle:
+	class_handle_cleanup();
+
+cleanup_uuidlist:
+	class_exit_uuidlist();
+
+cleanup_zombie_impexp:
+	obd_zombie_impexp_stop();
 
 	return err;
 }
-- 
1.8.3.1


From jsimmons at infradead.org  Tue Jul 31 02:26:07 2018
From: jsimmons at infradead.org (James Simmons)
Date: Mon, 30 Jul 2018 22:26:07 -0400
Subject: [lustre-devel] [PATCH 15/31] lustre: clio: remove unused members
 from struct cl_thread_info
In-Reply-To: <1533003983-29311-1-git-send-email-jsimmons@infradead.org>
References: <1533003983-29311-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1533003983-29311-16-git-send-email-jsimmons@infradead.org>

From: Dmitry Eremin <dmitry.eremin at intel.com>

The pointer to the topmost ongoing IO in the thread and
other members are not used any more.

Signed-off-by: Dmitry Eremin <dmitry.eremin at intel.com>
WC-id: https://jira.whamcloud.com/browse/LU-8888
Reviewed-on: https://review.whamcloud.com/24062
Reviewed-by: John L. Hammond <jhammond at whamcloud.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong at gmail.com>
Reviewed-by: James Simmons <uja.ornl at yahoo.com>
Reviewed-by: Oleg Drokin <green at whamcloud.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/lustre/include/cl_object.h  |  1 -
 .../staging/lustre/lustre/obdclass/cl_internal.h   | 45 +---------------------
 drivers/staging/lustre/lustre/obdclass/cl_io.c     | 21 +---------
 drivers/staging/lustre/lustre/obdclass/cl_object.c | 16 +-------
 4 files changed, 4 insertions(+), 79 deletions(-)

diff --git a/drivers/staging/lustre/lustre/include/cl_object.h b/drivers/staging/lustre/lustre/include/cl_object.h
index 58af22e..382bfe8 100644
--- a/drivers/staging/lustre/lustre/include/cl_object.h
+++ b/drivers/staging/lustre/lustre/include/cl_object.h
@@ -2291,7 +2291,6 @@ int cl_io_commit_async(const struct lu_env *env, struct cl_io *io,
 		       cl_commit_cbt cb);
 int cl_io_read_ahead(const struct lu_env *env, struct cl_io *io,
 		     pgoff_t start, struct cl_read_ahead *ra);
-int cl_io_is_going(const struct lu_env *env);
 
 /**
  * True, iff \a io is an O_APPEND write(2).
diff --git a/drivers/staging/lustre/lustre/obdclass/cl_internal.h b/drivers/staging/lustre/lustre/obdclass/cl_internal.h
index a0db830..8770e32 100644
--- a/drivers/staging/lustre/lustre/obdclass/cl_internal.h
+++ b/drivers/staging/lustre/lustre/obdclass/cl_internal.h
@@ -37,57 +37,14 @@
 #ifndef _CL_INTERNAL_H
 #define _CL_INTERNAL_H
 
-#define CLT_PVEC_SIZE (14)
-
-/**
- * Possible levels of the nesting. Currently this is 2: there are "top"
- * entities (files, extent locks), and "sub" entities (stripes and stripe
- * locks). This is used only for debugging counters right now.
- */
-enum clt_nesting_level {
-	CNL_TOP,
-	CNL_SUB,
-	CNL_NR
-};
-
 /**
  * Thread local state internal for generic cl-code.
  */
 struct cl_thread_info {
-	/*
-	 * Common fields.
-	 */
-	struct cl_io	 clt_io;
-	struct cl_2queue     clt_queue;
-
-	/*
-	 * Fields used by cl_lock.c
-	 */
-	struct cl_lock_descr clt_descr;
-	struct cl_page_list  clt_list;
-	/** @} debugging */
-
-	/*
-	 * Fields used by cl_page.c
-	 */
-	struct cl_page      *clt_pvec[CLT_PVEC_SIZE];
-
-	/*
-	 * Fields used by cl_io.c
-	 */
 	/**
-	 * Pointer to the topmost ongoing IO in this thread.
-	 */
-	struct cl_io	*clt_current_io;
-	/**
-	 * Used for submitting a sync io.
+	 * Used for submitting a sync I/O.
 	 */
 	struct cl_sync_io    clt_anchor;
-	/**
-	 * Fields used by cl_lock_discard_pages().
-	 */
-	pgoff_t	      clt_next_index;
-	pgoff_t	      clt_fn_index; /* first non-overlapped index */
 };
 
 struct cl_thread_info *cl_env_info(const struct lu_env *env);
diff --git a/drivers/staging/lustre/lustre/obdclass/cl_io.c b/drivers/staging/lustre/lustre/obdclass/cl_io.c
index 2c77e72..3a96d4a 100644
--- a/drivers/staging/lustre/lustre/obdclass/cl_io.c
+++ b/drivers/staging/lustre/lustre/obdclass/cl_io.c
@@ -68,14 +68,6 @@ static inline int cl_io_is_loopable(const struct cl_io *io)
 }
 
 /**
- * Returns true iff there is an IO ongoing in the given environment.
- */
-int cl_io_is_going(const struct lu_env *env)
-{
-	return cl_env_info(env)->clt_current_io != NULL;
-}
-
-/**
  * cl_io invariant that holds at all times when exported cl_io_*() functions
  * are entered and left.
  */
@@ -100,7 +92,6 @@ static int cl_io_invariant(const struct cl_io *io)
 void cl_io_fini(const struct lu_env *env, struct cl_io *io)
 {
 	struct cl_io_slice    *slice;
-	struct cl_thread_info *info;
 
 	LINVRNT(cl_io_type_is_valid(io->ci_type));
 	LINVRNT(cl_io_invariant(io));
@@ -119,9 +110,6 @@ void cl_io_fini(const struct lu_env *env, struct cl_io *io)
 		slice->cis_io = NULL;
 	}
 	io->ci_state = CIS_FINI;
-	info = cl_env_info(env);
-	if (info->clt_current_io == io)
-		info->clt_current_io = NULL;
 
 	/* sanity check for layout change */
 	switch (io->ci_type) {
@@ -184,11 +172,8 @@ static int cl_io_init0(const struct lu_env *env, struct cl_io *io,
 int cl_io_sub_init(const struct lu_env *env, struct cl_io *io,
 		   enum cl_io_type iot, struct cl_object *obj)
 {
-	struct cl_thread_info *info = cl_env_info(env);
-
 	LASSERT(obj != cl_object_top(obj));
-	if (!info->clt_current_io)
-		info->clt_current_io = io;
+
 	return cl_io_init0(env, io, iot, obj);
 }
 EXPORT_SYMBOL(cl_io_sub_init);
@@ -206,12 +191,8 @@ int cl_io_sub_init(const struct lu_env *env, struct cl_io *io,
 int cl_io_init(const struct lu_env *env, struct cl_io *io,
 	       enum cl_io_type iot, struct cl_object *obj)
 {
-	struct cl_thread_info *info = cl_env_info(env);
-
 	LASSERT(obj == cl_object_top(obj));
-	LASSERT(!info->clt_current_io);
 
-	info->clt_current_io = io;
 	return cl_io_init0(env, io, iot, obj);
 }
 EXPORT_SYMBOL(cl_io_init);
diff --git a/drivers/staging/lustre/lustre/obdclass/cl_object.c b/drivers/staging/lustre/lustre/obdclass/cl_object.c
index 42cce2d..d1d7bec 100644
--- a/drivers/staging/lustre/lustre/obdclass/cl_object.c
+++ b/drivers/staging/lustre/lustre/obdclass/cl_object.c
@@ -973,20 +973,8 @@ struct cl_thread_info *cl_env_info(const struct lu_env *env)
 	return lu_context_key_get(&env->le_ctx, &cl_key);
 }
 
-/* defines cl0_key_{init,fini}() */
-LU_KEY_INIT_FINI(cl0, struct cl_thread_info);
-
-static void *cl_key_init(const struct lu_context *ctx,
-			 struct lu_context_key *key)
-{
-	return cl0_key_init(ctx, key);
-}
-
-static void cl_key_fini(const struct lu_context *ctx,
-			struct lu_context_key *key, void *data)
-{
-	cl0_key_fini(ctx, key, data);
-}
+/* defines cl_key_{init,fini}() */
+LU_KEY_INIT_FINI(cl, struct cl_thread_info);
 
 static struct lu_context_key cl_key = {
 	.lct_tags = LCT_CL_THREAD,
-- 
1.8.3.1


From jsimmons at infradead.org  Tue Jul 31 02:26:10 2018
From: jsimmons at infradead.org (James Simmons)
Date: Mon, 30 Jul 2018 22:26:10 -0400
Subject: [lustre-devel] [PATCH 18/31] lustre: libcfs: avoid overflow of
 crypto bandwidth calculation
In-Reply-To: <1533003983-29311-1-git-send-email-jsimmons@infradead.org>
References: <1533003983-29311-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1533003983-29311-19-git-send-email-jsimmons@infradead.org>

From: Gu Zheng <gzheng at ddn.com>

bcount and buf_len are both int, and no force convert in the
calculation code:

tmp = ((bcount * buf_len / jiffies_to_msecs(end - start)) *
       1000) / (1024 * 1024);
That may cause overflow in modern fast machine.

Signed-off-by: Gu Zheng <gzheng at ddn.com>
WC-id: https://jira.whamcloud.com/browse/LU-9116
Reviewed-on: https://review.whamcloud.com/25436
Reviewed-by: Andreas Dilger <adilger at whamclud.com>
Reviewed-by: Emoly Liu <emoly at whamcloud.com>
Reviewed-by: James Simmons <uja.ornl at yahoo.com>
Reviewed-by: Li Xi <lixi at ddn.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/lnet/libcfs/linux-crypto.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/drivers/staging/lustre/lnet/libcfs/linux-crypto.c b/drivers/staging/lustre/lnet/libcfs/linux-crypto.c
index 21ff9bf..cfff54d 100644
--- a/drivers/staging/lustre/lnet/libcfs/linux-crypto.c
+++ b/drivers/staging/lustre/lnet/libcfs/linux-crypto.c
@@ -313,7 +313,8 @@ static void cfs_crypto_performance_test(enum cfs_crypto_hash_alg hash_alg)
 	int buf_len = max(PAGE_SIZE, 1048576UL);
 	void *buf;
 	unsigned long start, end;
-	int bcount, err = 0;
+	unsigned long bcount;
+	int err = 0;
 	struct page *page;
 	unsigned char hash[CFS_CRYPTO_HASH_DIGESTSIZE_MAX];
 	unsigned int hash_len = sizeof(hash);
-- 
1.8.3.1


From jsimmons at infradead.org  Tue Jul 31 02:26:13 2018
From: jsimmons at infradead.org (James Simmons)
Date: Mon, 30 Jul 2018 22:26:13 -0400
Subject: [lustre-devel] [PATCH 21/31] lustre: osc: hung in osc_destroy()
In-Reply-To: <1533003983-29311-1-git-send-email-jsimmons@infradead.org>
References: <1533003983-29311-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1533003983-29311-22-git-send-email-jsimmons@infradead.org>

From: Andriy Skulysh <c17819 at cray.com>

cl_destroy_in_flight becomes < 0 because the
osc_can_send_destroy() won't increment
cl_destroy_in_flight if l_wait_event() gets
interrupted by a signal, but the request will
still be sent and the request's interpret
function will decrease the counter.

Don't send OST_DESTROY request on signal
and return -EINTR.

Signed-off-by: Andriy Skulysh <c17819 at cray.com>
Seagate-bug-id: MRP-3834
WC-id: https://jira.whamcloud.com/browse/LU-8624
Reviewed-on: https://review.whamcloud.com/22588
Reviewed-by: Andreas Dilger <adilger at whamcloud.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong at gmail.com>
Reviewed-by: Oleg Drokin <green at whamcloud.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/lustre/osc/osc_request.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/drivers/staging/lustre/lustre/osc/osc_request.c b/drivers/staging/lustre/lustre/osc/osc_request.c
index 21497ea..b7f8e07 100644
--- a/drivers/staging/lustre/lustre/osc/osc_request.c
+++ b/drivers/staging/lustre/lustre/osc/osc_request.c
@@ -652,8 +652,12 @@ static int osc_destroy(const struct lu_env *env, struct obd_export *exp,
 		 * Wait until the number of on-going destroy RPCs drops
 		 * under max_rpc_in_flight
 		 */
-		l_wait_event_abortable_exclusive(cli->cl_destroy_waitq,
-					       osc_can_send_destroy(cli));
+		rc = l_wait_event_abortable_exclusive(cli->cl_destroy_waitq,
+						      osc_can_send_destroy(cli));
+		if (rc) {
+			ptlrpc_request_free(req);
+			return rc;
+		}
 	}
 
 	/* Do not wait for response */
-- 
1.8.3.1


From jsimmons at infradead.org  Tue Jul 31 02:26:18 2018
From: jsimmons at infradead.org (James Simmons)
Date: Mon, 30 Jul 2018 22:26:18 -0400
Subject: [lustre-devel] [PATCH 26/31] lustre: obd: remove unused data
 parameter from obd_notify()
In-Reply-To: <1533003983-29311-1-git-send-email-jsimmons@infradead.org>
References: <1533003983-29311-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1533003983-29311-27-git-send-email-jsimmons@infradead.org>

From: "John L. Hammond" <jhammond at whamcloud.com>

Remove the unused data parameter from obd_notify() and related
functions.

Signed-off-by: John L. Hammond <jhammond at whamcloud.com>
WC-id: https://jira.whamcloud.com/browse/LU-8403
Reviewed-on: https://review.whamcloud.com/24428
Reviewed-by: Steve Guminski <stephenx.guminski at intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin at intel.com>
Reviewed-by: James Simmons <uja.ornl at yahoo.com>
Reviewed-by: Oleg Drokin <green at whamcloud.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/lustre/include/obd.h        |  4 ++--
 drivers/staging/lustre/lustre/include/obd_class.h  | 25 ++++++++--------------
 drivers/staging/lustre/lustre/llite/lcommon_misc.c | 12 +++++------
 .../staging/lustre/lustre/llite/llite_internal.h   |  5 ++---
 drivers/staging/lustre/lustre/lmv/lmv_obd.c        |  7 +++---
 drivers/staging/lustre/lustre/lov/lov_obd.c        | 13 +++++------
 drivers/staging/lustre/lustre/mdc/mdc_request.c    |  6 +++---
 drivers/staging/lustre/lustre/osc/osc_request.c    | 10 ++++-----
 8 files changed, 34 insertions(+), 48 deletions(-)

diff --git a/drivers/staging/lustre/lustre/include/obd.h b/drivers/staging/lustre/lustre/include/obd.h
index 10e3bb8..333c703 100644
--- a/drivers/staging/lustre/lustre/include/obd.h
+++ b/drivers/staging/lustre/lustre/include/obd.h
@@ -502,7 +502,7 @@ enum obd_notify_event {
  */
 struct obd_notify_upcall {
 	int (*onu_upcall)(struct obd_device *host, struct obd_device *watched,
-			  enum obd_notify_event ev, void *owner, void *data);
+			  enum obd_notify_event ev, void *owner);
 	/* Opaque datum supplied by upper layer listener */
 	void *onu_owner;
 };
@@ -861,7 +861,7 @@ struct obd_ops {
 			    enum obd_import_event);
 
 	int (*notify)(struct obd_device *obd, struct obd_device *watched,
-		      enum obd_notify_event ev, void *data);
+		      enum obd_notify_event ev);
 
 	int (*health_check)(const struct lu_env *env, struct obd_device *);
 	struct obd_uuid *(*get_uuid)(struct obd_export *exp);
diff --git a/drivers/staging/lustre/lustre/include/obd_class.h b/drivers/staging/lustre/lustre/include/obd_class.h
index 647cc22..50d5ddb 100644
--- a/drivers/staging/lustre/lustre/include/obd_class.h
+++ b/drivers/staging/lustre/lustre/include/obd_class.h
@@ -1074,8 +1074,7 @@ static inline void obd_import_event(struct obd_device *obd,
 
 static inline int obd_notify(struct obd_device *obd,
 			     struct obd_device *watched,
-			     enum obd_notify_event ev,
-			     void *data)
+			     enum obd_notify_event ev)
 {
 	int rc;
 
@@ -1094,35 +1093,29 @@ static inline int obd_notify(struct obd_device *obd,
 	}
 
 	OBD_COUNTER_INCREMENT(obd, notify);
-	rc = OBP(obd, notify)(obd, watched, ev, data);
+	rc = OBP(obd, notify)(obd, watched, ev);
 	return rc;
 }
 
 static inline int obd_notify_observer(struct obd_device *observer,
 				      struct obd_device *observed,
-				      enum obd_notify_event ev,
-				      void *data)
+				      enum obd_notify_event ev)
 {
-	int rc1;
-	int rc2;
-
 	struct obd_notify_upcall *onu;
+	int rc = 0;
+	int rc2 = 0;
 
 	if (observer->obd_observer)
-		rc1 = obd_notify(observer->obd_observer, observed, ev, data);
-	else
-		rc1 = 0;
+		rc = obd_notify(observer->obd_observer, observed, ev);
+
 	/*
 	 * Also, call non-obd listener, if any
 	 */
 	onu = &observer->obd_upcall;
 	if (onu->onu_upcall)
-		rc2 = onu->onu_upcall(observer, observed, ev,
-				      onu->onu_owner, NULL);
-	else
-		rc2 = 0;
+		rc2 = onu->onu_upcall(observer, observed, ev, onu->onu_owner);
 
-	return rc1 ? rc1 : rc2;
+	return rc ? rc : rc2;
 }
 
 static inline int obd_quotactl(struct obd_export *exp,
diff --git a/drivers/staging/lustre/lustre/llite/lcommon_misc.c b/drivers/staging/lustre/lustre/llite/lcommon_misc.c
index a246b95..80563a2 100644
--- a/drivers/staging/lustre/lustre/llite/lcommon_misc.c
+++ b/drivers/staging/lustre/lustre/llite/lcommon_misc.c
@@ -76,14 +76,12 @@ int cl_init_ea_size(struct obd_export *md_exp, struct obd_export *dt_exp)
 }
 
 /**
- * This function is used as an upcall-callback hooked by liblustre and llite
- * clients into obd_notify() listeners chain to handle notifications about
- * change of import connect_flags. See llu_fsswop_mount() and
- * lustre_common_fill_super().
+ * This function is used as an upcall-callback hooked llite clients
+ * into obd_notify() listeners chain to handle notifications about
+ * change of import connect_flags. See lustre_common_fill_super().
  */
-int cl_ocd_update(struct obd_device *host,
-		  struct obd_device *watched,
-		  enum obd_notify_event ev, void *owner, void *data)
+int cl_ocd_update(struct obd_device *host, struct obd_device *watched,
+		  enum obd_notify_event ev, void *owner)
 {
 	struct lustre_client_ocd *lco;
 	struct client_obd	*cli;
diff --git a/drivers/staging/lustre/lustre/llite/llite_internal.h b/drivers/staging/lustre/lustre/llite/llite_internal.h
index f6c8daf..00fe706 100644
--- a/drivers/staging/lustre/lustre/llite/llite_internal.h
+++ b/drivers/staging/lustre/lustre/llite/llite_internal.h
@@ -669,9 +669,8 @@ static inline bool ll_sbi_has_fast_read(struct ll_sb_info *sbi)
 
 /* llite/lcommon_misc.c */
 int cl_init_ea_size(struct obd_export *md_exp, struct obd_export *dt_exp);
-int cl_ocd_update(struct obd_device *host,
-		  struct obd_device *watched,
-		  enum obd_notify_event ev, void *owner, void *data);
+int cl_ocd_update(struct obd_device *host, struct obd_device *watched,
+		  enum obd_notify_event ev, void *owner);
 int cl_get_grouplock(struct cl_object *obj, unsigned long gid, int nonblock,
 		     struct ll_grouplock *cg);
 void cl_put_grouplock(struct ll_grouplock *cg);
diff --git a/drivers/staging/lustre/lustre/lmv/lmv_obd.c b/drivers/staging/lustre/lustre/lmv/lmv_obd.c
index bbb1ddf..55db904 100644
--- a/drivers/staging/lustre/lustre/lmv/lmv_obd.c
+++ b/drivers/staging/lustre/lustre/lmv/lmv_obd.c
@@ -140,7 +140,7 @@ static struct obd_uuid *lmv_get_uuid(struct obd_export *exp)
 }
 
 static int lmv_notify(struct obd_device *obd, struct obd_device *watched,
-		      enum obd_notify_event ev, void *data)
+		      enum obd_notify_event ev)
 {
 	struct obd_connect_data *conn_data;
 	struct lmv_obd	  *lmv = &obd->u.lmv;
@@ -182,7 +182,7 @@ static int lmv_notify(struct obd_device *obd, struct obd_device *watched,
 	 * Pass the notification up the chain.
 	 */
 	if (obd->obd_observer)
-		rc = obd_notify(obd->obd_observer, watched, ev, data);
+		rc = obd_notify(obd->obd_observer, watched, ev);
 
 	return rc;
 }
@@ -330,8 +330,7 @@ static int lmv_connect_mdc(struct obd_device *obd, struct lmv_tgt_desc *tgt)
 		 * Tell the observer about the new target.
 		 */
 		rc = obd_notify(obd->obd_observer, mdc_exp->exp_obd,
-				OBD_NOTIFY_ACTIVE,
-				(void *)(tgt - lmv->tgts[0]));
+				OBD_NOTIFY_ACTIVE);
 		if (rc) {
 			obd_disconnect(mdc_exp);
 			return rc;
diff --git a/drivers/staging/lustre/lustre/lov/lov_obd.c b/drivers/staging/lustre/lustre/lov/lov_obd.c
index 85d3b29..07f6b1b 100644
--- a/drivers/staging/lustre/lustre/lov/lov_obd.c
+++ b/drivers/staging/lustre/lustre/lov/lov_obd.c
@@ -121,7 +121,7 @@ static void lov_putref(struct obd_device *obd)
 static int lov_set_osc_active(struct obd_device *obd, struct obd_uuid *uuid,
 			      enum obd_notify_event ev);
 static int lov_notify(struct obd_device *obd, struct obd_device *watched,
-		      enum obd_notify_event ev, void *data);
+		      enum obd_notify_event ev);
 
 int lov_connect_obd(struct obd_device *obd, __u32 index, int activate,
 		    struct obd_connect_data *data)
@@ -244,7 +244,7 @@ static int lov_connect(const struct lu_env *env,
 			continue;
 
 		rc = lov_notify(obd, lov->lov_tgts[i]->ltd_exp->exp_obd,
-				OBD_NOTIFY_CONNECT, (void *)&i);
+				OBD_NOTIFY_CONNECT);
 		if (rc) {
 			CERROR("%s error sending notify %d\n",
 			       obd->obd_name, rc);
@@ -423,7 +423,7 @@ static int lov_set_osc_active(struct obd_device *obd, struct obd_uuid *uuid,
 }
 
 static int lov_notify(struct obd_device *obd, struct obd_device *watched,
-		      enum obd_notify_event ev, void *data)
+		      enum obd_notify_event ev)
 {
 	int rc = 0;
 	struct lov_obd *lov = &obd->u.lov;
@@ -457,12 +457,10 @@ static int lov_notify(struct obd_device *obd, struct obd_device *watched,
 			       obd_uuid2str(uuid), rc);
 			goto out_notify_lock;
 		}
-		/* active event should be pass lov target index as data */
-		data = &rc;
 	}
 
 	/* Pass the notification up the chain. */
-	rc = obd_notify_observer(obd, watched, ev, data);
+	rc = obd_notify_observer(obd, watched, ev);
 
 out_notify_lock:
 	up_read(&lov->lov_notify_lock);
@@ -590,8 +588,7 @@ static int lov_add_target(struct obd_device *obd, struct obd_uuid *uuidp,
 	}
 
 	rc = lov_notify(obd, tgt->ltd_exp->exp_obd,
-			active ? OBD_NOTIFY_CONNECT : OBD_NOTIFY_INACTIVE,
-			(void *)&index);
+			active ? OBD_NOTIFY_CONNECT : OBD_NOTIFY_INACTIVE);
 
 out:
 	if (rc) {
diff --git a/drivers/staging/lustre/lustre/mdc/mdc_request.c b/drivers/staging/lustre/lustre/mdc/mdc_request.c
index bfa07d7..c2f0a54 100644
--- a/drivers/staging/lustre/lustre/mdc/mdc_request.c
+++ b/drivers/staging/lustre/lustre/mdc/mdc_request.c
@@ -2520,7 +2520,7 @@ static int mdc_import_event(struct obd_device *obd, struct obd_import *imp,
 		if (cli->cl_seq)
 			seq_client_flush(cli->cl_seq);
 
-		rc = obd_notify_observer(obd, obd, OBD_NOTIFY_INACTIVE, NULL);
+		rc = obd_notify_observer(obd, obd, OBD_NOTIFY_INACTIVE);
 		break;
 	}
 	case IMP_EVENT_INVALIDATE: {
@@ -2531,7 +2531,7 @@ static int mdc_import_event(struct obd_device *obd, struct obd_import *imp,
 		break;
 	}
 	case IMP_EVENT_ACTIVE:
-		rc = obd_notify_observer(obd, obd, OBD_NOTIFY_ACTIVE, NULL);
+		rc = obd_notify_observer(obd, obd, OBD_NOTIFY_ACTIVE);
 		/* redo the kuc registration after reconnecting */
 		if (rc == 0)
 			/* re-register HSM agents */
@@ -2540,7 +2540,7 @@ static int mdc_import_event(struct obd_device *obd, struct obd_import *imp,
 						       (void *)imp);
 		break;
 	case IMP_EVENT_OCD:
-		rc = obd_notify_observer(obd, obd, OBD_NOTIFY_OCD, NULL);
+		rc = obd_notify_observer(obd, obd, OBD_NOTIFY_OCD);
 		break;
 	case IMP_EVENT_DISCON:
 	case IMP_EVENT_DEACTIVATE:
diff --git a/drivers/staging/lustre/lustre/osc/osc_request.c b/drivers/staging/lustre/lustre/osc/osc_request.c
index b7f8e07..b2b55a7 100644
--- a/drivers/staging/lustre/lustre/osc/osc_request.c
+++ b/drivers/staging/lustre/lustre/osc/osc_request.c
@@ -2678,7 +2678,7 @@ static int osc_import_event(struct obd_device *obd,
 		break;
 	}
 	case IMP_EVENT_INACTIVE: {
-		rc = obd_notify_observer(obd, obd, OBD_NOTIFY_INACTIVE, NULL);
+		rc = obd_notify_observer(obd, obd, OBD_NOTIFY_INACTIVE);
 		break;
 	}
 	case IMP_EVENT_INVALIDATE: {
@@ -2704,7 +2704,7 @@ static int osc_import_event(struct obd_device *obd,
 		break;
 	}
 	case IMP_EVENT_ACTIVE: {
-		rc = obd_notify_observer(obd, obd, OBD_NOTIFY_ACTIVE, NULL);
+		rc = obd_notify_observer(obd, obd, OBD_NOTIFY_ACTIVE);
 		break;
 	}
 	case IMP_EVENT_OCD: {
@@ -2717,15 +2717,15 @@ static int osc_import_event(struct obd_device *obd,
 		if (ocd->ocd_connect_flags & OBD_CONNECT_REQPORTAL)
 			imp->imp_client->cli_request_portal = OST_REQUEST_PORTAL;
 
-		rc = obd_notify_observer(obd, obd, OBD_NOTIFY_OCD, NULL);
+		rc = obd_notify_observer(obd, obd, OBD_NOTIFY_OCD);
 		break;
 	}
 	case IMP_EVENT_DEACTIVATE: {
-		rc = obd_notify_observer(obd, obd, OBD_NOTIFY_DEACTIVATE, NULL);
+		rc = obd_notify_observer(obd, obd, OBD_NOTIFY_DEACTIVATE);
 		break;
 	}
 	case IMP_EVENT_ACTIVATE: {
-		rc = obd_notify_observer(obd, obd, OBD_NOTIFY_ACTIVATE, NULL);
+		rc = obd_notify_observer(obd, obd, OBD_NOTIFY_ACTIVATE);
 		break;
 	}
 	default:
-- 
1.8.3.1


From jsimmons at infradead.org  Tue Jul 31 02:25:58 2018
From: jsimmons at infradead.org (James Simmons)
Date: Mon, 30 Jul 2018 22:25:58 -0400
Subject: [lustre-devel] [PATCH 06/31] lustre: llite: reduce jobstats race
 window
In-Reply-To: <1533003983-29311-1-git-send-email-jsimmons@infradead.org>
References: <1533003983-29311-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1533003983-29311-7-git-send-email-jsimmons@infradead.org>

From: Patrick Farrell <paf at cray.com>

In the current code, lli_jobid is set to zero on every call
to lustre_get_jobid.  This causes problems, because it's
used asynchronously to set the job id in RPCs, and some
RPCs will falsely get no jobid set.  (For small IO sizes,
this can be up to 60% of RPCs.)

It would be very expensive to put hard synchronization
between this and every outbound RPC, and it's OK to very
rarely get an RPC without correct job stats info.

This patch only updates the lli_jobid when the job id has
changed, which leaves only a very small window for reading
an inconsistent job id.

Signed-off-by: Patrick Farrell <paf at cray.com>
WC-id: https://jira.whamcloud.com/browse/LU-8926
Reviewed-on: https://review.whamcloud.com/24253
Reviewed-by: Andreas Dilger <adilger at whamcloud.com>
Reviewed-by: Chris Horn <hornc at cray.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/lustre/llite/llite_lib.c    |  1 +
 drivers/staging/lustre/lustre/obdclass/class_obd.c | 20 ++++++++++++++------
 2 files changed, 15 insertions(+), 6 deletions(-)

diff --git a/drivers/staging/lustre/lustre/llite/llite_lib.c b/drivers/staging/lustre/lustre/llite/llite_lib.c
index c0861b9..72b118a 100644
--- a/drivers/staging/lustre/lustre/llite/llite_lib.c
+++ b/drivers/staging/lustre/lustre/llite/llite_lib.c
@@ -894,6 +894,7 @@ void ll_lli_init(struct ll_inode_info *lli)
 		lli->lli_async_rc = 0;
 	}
 	mutex_init(&lli->lli_layout_mutex);
+	memset(lli->lli_jobid, 0, LUSTRE_JOBID_SIZE);
 }
 
 int ll_fill_super(struct super_block *sb)
diff --git a/drivers/staging/lustre/lustre/obdclass/class_obd.c b/drivers/staging/lustre/lustre/obdclass/class_obd.c
index cdaf729..87327ef 100644
--- a/drivers/staging/lustre/lustre/obdclass/class_obd.c
+++ b/drivers/staging/lustre/lustre/obdclass/class_obd.c
@@ -95,26 +95,34 @@
  */
 int lustre_get_jobid(char *jobid)
 {
-	memset(jobid, 0, LUSTRE_JOBID_SIZE);
+	char tmp_jobid[LUSTRE_JOBID_SIZE] = { 0 };
+
 	/* Jobstats isn't enabled */
 	if (strcmp(obd_jobid_var, JOBSTATS_DISABLE) == 0)
-		return 0;
+		goto out_cache_jobid;
 
 	/* Use process name + fsuid as jobid */
 	if (strcmp(obd_jobid_var, JOBSTATS_PROCNAME_UID) == 0) {
-		snprintf(jobid, LUSTRE_JOBID_SIZE, "%s.%u",
+		snprintf(tmp_jobid, LUSTRE_JOBID_SIZE, "%s.%u",
 			 current->comm,
 			 from_kuid(&init_user_ns, current_fsuid()));
-		return 0;
+		goto out_cache_jobid;
 	}
 
 	/* Whole node dedicated to single job */
 	if (strcmp(obd_jobid_var, JOBSTATS_NODELOCAL) == 0) {
-		strcpy(jobid, obd_jobid_node);
-		return 0;
+		strcpy(tmp_jobid, obd_jobid_node);
+		goto out_cache_jobid;
 	}
 
 	return -ENOENT;
+
+out_cache_jobid:
+	/* Only replace the job ID if it changed. */
+	if (strcmp(jobid, tmp_jobid) != 0)
+		strcpy(jobid, tmp_jobid);
+
+	return 0;
 }
 EXPORT_SYMBOL(lustre_get_jobid);
 
-- 
1.8.3.1


From jsimmons at infradead.org  Tue Jul 31 02:26:21 2018
From: jsimmons at infradead.org (James Simmons)
Date: Mon, 30 Jul 2018 22:26:21 -0400
Subject: [lustre-devel] [PATCH 29/31] lustre: llite: ignore layout for
 ll_writepages()
In-Reply-To: <1533003983-29311-1-git-send-email-jsimmons@infradead.org>
References: <1533003983-29311-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1533003983-29311-30-git-send-email-jsimmons@infradead.org>

From: Jinshan Xiong <jinshan.xiong at gmail.com>

ll_writepages() would be called inside the direct IO context and
if the layout has been changed during this time, the layout_conf()
has to wait for active IO to complete before applying the layout
change, this is a case of deadlock.

It should ignore layout to avoid this problem. This is safe as long
as pages exist, the layout won't be changed on this client.

Signed-off-by: Jinshan Xiong <jinshan.xiong at gmail.com>
WC-id: https://jira.whamcloud.com/browse/LU-9129
Reviewed-on: https://review.whamcloud.com/25474
Reviewed-by: Bobi Jam <bobijam at hotmail.com>
Reviewed-by: John L. Hammond <jhammond at whamcloud.com>
Reviewed-by: Oleg Drokin <green at whamcloud.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/lustre/llite/rw.c | 15 +++++----------
 1 file changed, 5 insertions(+), 10 deletions(-)

diff --git a/drivers/staging/lustre/lustre/llite/rw.c b/drivers/staging/lustre/lustre/llite/rw.c
index 59747da..49ac723 100644
--- a/drivers/staging/lustre/lustre/llite/rw.c
+++ b/drivers/staging/lustre/lustre/llite/rw.c
@@ -1000,13 +1000,11 @@ int ll_writepage(struct page *vmpage, struct writeback_control *wbc)
 int ll_writepages(struct address_space *mapping, struct writeback_control *wbc)
 {
 	struct inode *inode = mapping->host;
-	struct ll_sb_info *sbi = ll_i2sbi(inode);
 	loff_t start;
 	loff_t end;
 	enum cl_fsync_mode mode;
 	int range_whole = 0;
 	int result;
-	int ignore_layout = 0;
 
 	if (wbc->range_cyclic) {
 		start = mapping->writeback_index << PAGE_SHIFT;
@@ -1024,17 +1022,14 @@ int ll_writepages(struct address_space *mapping, struct writeback_control *wbc)
 	if (wbc->sync_mode == WB_SYNC_ALL)
 		mode = CL_FSYNC_LOCAL;
 
-	if (sbi->ll_umounting)
-		/* if the mountpoint is being umounted, all pages have to be
-		 * evicted to avoid hitting LBUG when truncate_inode_pages()
-		 * is called later on.
-		 */
-		ignore_layout = 1;
-
 	if (!ll_i2info(inode)->lli_clob)
 		return 0;
 
-	result = cl_sync_file_range(inode, start, end, mode, ignore_layout);
+	/* for directio, it would call writepages() to evict cached pages
+	 * inside the IO context of write, which will cause deadlock at
+	 * layout_conf since it waits for active IOs to complete.
+	 */
+	result = cl_sync_file_range(inode, start, end, mode, 1);
 	if (result > 0) {
 		wbc->nr_to_write -= result;
 		result = 0;
-- 
1.8.3.1


From jsimmons at infradead.org  Tue Jul 31 02:26:22 2018
From: jsimmons at infradead.org (James Simmons)
Date: Mon, 30 Jul 2018 22:26:22 -0400
Subject: [lustre-devel] [PATCH 30/31] lustre: fid: race between
 client_fid_fini and seq_client_flush
In-Reply-To: <1533003983-29311-1-git-send-email-jsimmons@infradead.org>
References: <1533003983-29311-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1533003983-29311-31-git-send-email-jsimmons@infradead.org>

From: Fan Yong <fan.yong at intel.com>

When the client mount failed or umount, the client_fid_fini() will
be called. At that time, the async connection failure will trigger
seq_client_flush() which parameter may have been released by the
client_fid_fini() by race.

Introduce client_obd::cl_seq_rwsem to protect client_obd::cl_seq.

Signed-off-by: Fan Yong <fan.yong at intel.com>
WC-id: https://jira.whamcloud.com/browse/LU-9224
Reviewed-on: https://review.whamcloud.com/26079
Reviewed-by: John L. Hammond <jhammond at whamcloud.com>
Reviewed-by: Andreas Dilger <adilger at whamcloud.com>
Reviewed-by: Oleg Drokin <green at whamcloud.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/lustre/fid/fid_request.c | 21 +++++++++++++++------
 drivers/staging/lustre/lustre/include/obd.h     |  1 +
 drivers/staging/lustre/lustre/ldlm/ldlm_lib.c   |  2 ++
 drivers/staging/lustre/lustre/mdc/mdc_request.c | 11 +++++++++--
 4 files changed, 27 insertions(+), 8 deletions(-)

diff --git a/drivers/staging/lustre/lustre/fid/fid_request.c b/drivers/staging/lustre/lustre/fid/fid_request.c
index a34fd90..f91242c 100644
--- a/drivers/staging/lustre/lustre/fid/fid_request.c
+++ b/drivers/staging/lustre/lustre/fid/fid_request.c
@@ -343,11 +343,14 @@ int client_fid_init(struct obd_device *obd,
 {
 	struct client_obd *cli = &obd->u.cli;
 	char *prefix;
-	int rc;
+	int rc = 0;
 
+	down_write(&cli->cl_seq_rwsem);
 	cli->cl_seq = kzalloc(sizeof(*cli->cl_seq), GFP_NOFS);
-	if (!cli->cl_seq)
-		return -ENOMEM;
+	if (!cli->cl_seq) {
+		rc = -ENOMEM;
+		goto out_free_lock;
+	}
 
 	prefix = kzalloc(MAX_OBD_NAME + 5, GFP_NOFS);
 	if (!prefix) {
@@ -361,10 +364,14 @@ int client_fid_init(struct obd_device *obd,
 	seq_client_init(cli->cl_seq, exp, type, prefix);
 	kfree(prefix);
 
-	return 0;
 out_free_seq:
-	kfree(cli->cl_seq);
-	cli->cl_seq = NULL;
+	if (rc) {
+		kfree(cli->cl_seq);
+		cli->cl_seq = NULL;
+	}
+out_free_lock:
+	up_write(&cli->cl_seq_rwsem);
+
 	return rc;
 }
 EXPORT_SYMBOL(client_fid_init);
@@ -373,11 +380,13 @@ int client_fid_fini(struct obd_device *obd)
 {
 	struct client_obd *cli = &obd->u.cli;
 
+	down_write(&cli->cl_seq_rwsem);
 	if (cli->cl_seq) {
 		seq_client_fini(cli->cl_seq);
 		kfree(cli->cl_seq);
 		cli->cl_seq = NULL;
 	}
+	up_write(&cli->cl_seq_rwsem);
 
 	return 0;
 }
diff --git a/drivers/staging/lustre/lustre/include/obd.h b/drivers/staging/lustre/lustre/include/obd.h
index 333c703..3c0dbb6 100644
--- a/drivers/staging/lustre/lustre/include/obd.h
+++ b/drivers/staging/lustre/lustre/include/obd.h
@@ -333,6 +333,7 @@ struct client_obd {
 
 	/* sequence manager */
 	struct lu_client_seq    *cl_seq;
+	struct rw_semaphore	 cl_seq_rwsem;
 
 	atomic_t	     cl_resends; /* resend count */
 
diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_lib.c b/drivers/staging/lustre/lustre/ldlm/ldlm_lib.c
index c36d1e4..32eda4f 100644
--- a/drivers/staging/lustre/lustre/ldlm/ldlm_lib.c
+++ b/drivers/staging/lustre/lustre/ldlm/ldlm_lib.c
@@ -308,6 +308,8 @@ int client_obd_setup(struct obd_device *obddev, struct lustre_cfg *lcfg)
 	}
 
 	init_rwsem(&cli->cl_sem);
+	cli->cl_seq = NULL;
+	init_rwsem(&cli->cl_seq_rwsem);
 	cli->cl_conn_count = 0;
 	memcpy(server_uuid.uuid, lustre_cfg_buf(lcfg, 2),
 	       min_t(unsigned int, LUSTRE_CFG_BUFLEN(lcfg, 2),
diff --git a/drivers/staging/lustre/lustre/mdc/mdc_request.c b/drivers/staging/lustre/lustre/mdc/mdc_request.c
index c2f0a54..a759da2 100644
--- a/drivers/staging/lustre/lustre/mdc/mdc_request.c
+++ b/drivers/staging/lustre/lustre/mdc/mdc_request.c
@@ -2517,8 +2517,10 @@ static int mdc_import_event(struct obd_device *obd, struct obd_import *imp,
 		 * Flush current sequence to make client obtain new one
 		 * from server in case of disconnect/reconnect.
 		 */
+		down_read(&cli->cl_seq_rwsem);
 		if (cli->cl_seq)
 			seq_client_flush(cli->cl_seq);
+		up_read(&cli->cl_seq_rwsem);
 
 		rc = obd_notify_observer(obd, obd, OBD_NOTIFY_INACTIVE);
 		break;
@@ -2557,9 +2559,14 @@ int mdc_fid_alloc(const struct lu_env *env, struct obd_export *exp,
 		  struct lu_fid *fid, struct md_op_data *op_data)
 {
 	struct client_obd *cli = &exp->exp_obd->u.cli;
-	struct lu_client_seq *seq = cli->cl_seq;
+	int rc = -EIO;
 
-	return seq_client_alloc_fid(env, seq, fid);
+	down_read(&cli->cl_seq_rwsem);
+	if (cli->cl_seq)
+		rc = seq_client_alloc_fid(env, cli->cl_seq, fid);
+	up_read(&cli->cl_seq_rwsem);
+
+	return rc;
 }
 
 static struct obd_uuid *mdc_get_uuid(struct obd_export *exp)
-- 
1.8.3.1


From jsimmons at infradead.org  Tue Jul 31 02:26:11 2018
From: jsimmons at infradead.org (James Simmons)
Date: Mon, 30 Jul 2018 22:26:11 -0400
Subject: [lustre-devel] [PATCH 19/31] lustre: obd: remove OBD_NOTIFY_CONFIG
In-Reply-To: <1533003983-29311-1-git-send-email-jsimmons@infradead.org>
References: <1533003983-29311-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1533003983-29311-20-git-send-email-jsimmons@infradead.org>

From: "John L. Hammond" <jhammond at whamcloud.com>

None of the OBD notify handlers listen for the OBD_NOTIFY_CONFIG
event so remove it and its sole use in server_start_targets()
which is on the server side.

Signed-off-by: John L. Hammond <jhammond at whamcloud.com>
WC-id: https://jira.whamcloud.com/browse/LU-8403
Reviewed-on: https://review.whamcloud.com/21422
Reviewed-by: Andreas Dilger <adilger at whamcloud.com>
Reviewed-by: James Simmons <uja.ornl at yahoo.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/lustre/include/obd.h | 2 --
 1 file changed, 2 deletions(-)

diff --git a/drivers/staging/lustre/lustre/include/obd.h b/drivers/staging/lustre/lustre/include/obd.h
index 5bf2be8..10e3bb8 100644
--- a/drivers/staging/lustre/lustre/include/obd.h
+++ b/drivers/staging/lustre/lustre/include/obd.h
@@ -491,8 +491,6 @@ enum obd_notify_event {
 	OBD_NOTIFY_INACTIVE,
 	/* Connect data for import were changed */
 	OBD_NOTIFY_OCD,
-	/* Configuration event */
-	OBD_NOTIFY_CONFIG,
 	/* Administratively deactivate/activate event */
 	OBD_NOTIFY_DEACTIVATE,
 	OBD_NOTIFY_ACTIVATE
-- 
1.8.3.1


From jsimmons at infradead.org  Tue Jul 31 02:26:20 2018
From: jsimmons at infradead.org (James Simmons)
Date: Mon, 30 Jul 2018 22:26:20 -0400
Subject: [lustre-devel] [PATCH 28/31] lustre: obdclass: improve missing
 operation message
In-Reply-To: <1533003983-29311-1-git-send-email-jsimmons@infradead.org>
References: <1533003983-29311-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1533003983-29311-29-git-send-email-jsimmons@infradead.org>

From: Andreas Dilger <adilger at whamcloud.com>

Some tests in the past reported missing OBD operations, so improve
the error message to include the device name to make it easier to
debug where this is happening.

Signed-off-by: Andreas Dilger <adilger at whamcloud.com>
WC-id: https://jira.whamcloud.com/browse/LU-1095
Reviewed-on: https://review.whamcloud.com/25586
Reviewed-by: Alex Zhuravlev <bzzz at whamcloud.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin at intel.com>
Reviewed-by: Oleg Drokin <green at whamcloud.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/lustre/include/obd_class.h | 16 ++++++----------
 1 file changed, 6 insertions(+), 10 deletions(-)

diff --git a/drivers/staging/lustre/lustre/include/obd_class.h b/drivers/staging/lustre/lustre/include/obd_class.h
index 50d5ddb..fd9d99b 100644
--- a/drivers/staging/lustre/lustre/include/obd_class.h
+++ b/drivers/staging/lustre/lustre/include/obd_class.h
@@ -398,7 +398,7 @@ static inline int obd_check_dev_active(struct obd_device *obd)
 		return -EOPNOTSUPP;				\
 	}							\
 	if (!OBT((exp)->exp_obd) || !MDP((exp)->exp_obd, op)) {	\
-		CERROR("obd_" #op ": dev %s/%d no operation\n", \
+		CERROR("%s: obd_" #op ": dev %d no operation\n",\
 			(exp)->exp_obd->obd_name,		\
 			(exp)->exp_obd->obd_minor);		\
 		return -EOPNOTSUPP;				\
@@ -409,8 +409,8 @@ static inline int obd_check_dev_active(struct obd_device *obd)
 do {									\
 	if (!OBT(obd) || !OBP((obd), op)) {				\
 		if (err)						\
-			CERROR("obd_" #op ": dev %d no operation\n",	\
-				obd->obd_minor);			\
+			CERROR("%s: no obd_" #op " operation\n",	\
+				obd->obd_name);				\
 		return err;						\
 	}								\
 } while (0)
@@ -425,19 +425,15 @@ static inline int obd_check_dev_active(struct obd_device *obd)
 		CERROR("obd_" #op ": cleaned up obd\n");	\
 		return -EOPNOTSUPP;				\
 	}							\
-	if (!OBT((exp)->exp_obd) || !OBP((exp)->exp_obd, op)) {	\
-		CERROR("obd_" #op ": dev %d no operation\n",	\
-			(exp)->exp_obd->obd_minor);		\
-		return -EOPNOTSUPP;				\
-	}							\
+	OBD_CHECK_DT_OP((exp)->exp_obd, op, -EOPNOTSUPP);	\
 } while (0)
 
 #define CTXT_CHECK_OP(ctxt, op, err)					\
 do {									\
 	if (!OBT(ctxt->loc_obd) || !CTXTP((ctxt), op)) {		\
 		if (err)						\
-			CERROR("lop_" #op ": dev %d no operation\n",	\
-				ctxt->loc_obd->obd_minor);		\
+			CERROR("%s: no lop_" #op " operation\n",	\
+				ctxt->loc_obd->obd_name);		\
 		return err;						\
 	}								\
 } while (0)
-- 
1.8.3.1


From jsimmons at infradead.org  Tue Jul 31 02:26:19 2018
From: jsimmons at infradead.org (James Simmons)
Date: Mon, 30 Jul 2018 22:26:19 -0400
Subject: [lustre-devel] [PATCH 27/31] lustre: llite: handle client racy case
 during create
In-Reply-To: <1533003983-29311-1-git-send-email-jsimmons@infradead.org>
References: <1533003983-29311-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1533003983-29311-28-git-send-email-jsimmons@infradead.org>

From: Bruno Faccini <bruno.faccini at intel.com>

Some very infrequent situations exists on client side
able to cause a race during create when concurrent access
by fid occurs. The result of the race can allow a d_alias
to be already present when it was not expected when original
code/LBUG has been written.

One of the identified scenario is when a concurrent access of
inode thru the .lustre/fid/<[FID]> method occurs.

Final fix is to remove inaccurate
LASSERT(hlist_empty(&inode->i_dentry)); in ll_create_node().

Signed-off-by: Bruno Faccini <bruno.faccini at intel.com>
WC-id: https://jira.whamcloud.com/browse/LU-8907
Reviewed-on: https://review.whamcloud.com/25296
Reviewed-by: Lai Siyao <lai.siyao at intel.com>
Reviewed-by: Niu Yawei <yawei.niu at intel.com>
Reviewed-by: Oleg Drokin <green at whamcloud.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/lustre/include/obd_support.h | 1 +
 drivers/staging/lustre/lustre/llite/namei.c         | 3 ++-
 2 files changed, 3 insertions(+), 1 deletion(-)

diff --git a/drivers/staging/lustre/lustre/include/obd_support.h b/drivers/staging/lustre/lustre/include/obd_support.h
index 80b9935..1832193 100644
--- a/drivers/staging/lustre/lustre/include/obd_support.h
+++ b/drivers/staging/lustre/lustre/include/obd_support.h
@@ -441,6 +441,7 @@
 #define OBD_FAIL_MAKE_LOVEA_HOLE		    0x1406
 #define OBD_FAIL_LLITE_LOST_LAYOUT		    0x1407
 #define OBD_FAIL_GETATTR_DELAY			    0x1409
+#define OBD_FAIL_LLITE_CREATE_NODE_PAUSE	    0x140c
 
 #define OBD_FAIL_FID_INDIR	0x1501
 #define OBD_FAIL_FID_INLMA	0x1502
diff --git a/drivers/staging/lustre/lustre/llite/namei.c b/drivers/staging/lustre/lustre/llite/namei.c
index e541f78..da5854e 100644
--- a/drivers/staging/lustre/lustre/llite/namei.c
+++ b/drivers/staging/lustre/lustre/llite/namei.c
@@ -802,7 +802,8 @@ static struct inode *ll_create_node(struct inode *dir, struct lookup_intent *it)
 		goto out;
 	}
 
-	LASSERT(hlist_empty(&inode->i_dentry));
+	/* Pause to allow for a race with concurrent access by fid */
+	OBD_FAIL_TIMEOUT(OBD_FAIL_LLITE_CREATE_NODE_PAUSE, cfs_fail_val);
 
 	/* We asked for a lock on the directory, but were granted a
 	 * lock on the inode.  Since we finally have an inode pointer,
-- 
1.8.3.1


From jsimmons at infradead.org  Tue Jul 31 02:25:55 2018
From: jsimmons at infradead.org (James Simmons)
Date: Mon, 30 Jul 2018 22:25:55 -0400
Subject: [lustre-devel] [PATCH 03/31] lustre: obd: change positional struct
 initializers to C99
In-Reply-To: <1533003983-29311-1-git-send-email-jsimmons@infradead.org>
References: <1533003983-29311-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1533003983-29311-4-git-send-email-jsimmons@infradead.org>

From: Steve Guminski <stephenx.guminski at intel.com>

This patch makes no functional changes.  Struct initializers in the
obd directory that use C89 or GCC-only syntax are updated to C99
syntax.  Whitespace is changed to match the code style guidelines.

The C99 syntax prevents incorrect initialization if values are
accidently placed in the wrong position, allows changes in the struct
definition, and clears any members that are not given an explicit
value.

The following struct initializers have been updated:

lustre/obdclass/obd_config.c:
        struct llog_process_cat_data

Signed-off-by: Steve Guminski <stephenx.guminski at intel.com>
WC-id: https://jira.whamcloud.com/browse/LU-6210
Reviewed-on: https://review.whamcloud.com/23697
Reviewed-by: James Simmons <uja.ornl at yahoo.com>
Reviewed-by: John L. Hammond <jhammond at whamcloud.com>
Reviewed-by: Oleg Drokin <green at whamcloud.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/lustre/obdclass/obd_config.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/drivers/staging/lustre/lustre/obdclass/obd_config.c b/drivers/staging/lustre/lustre/obdclass/obd_config.c
index 6d47435..d962f0c 100644
--- a/drivers/staging/lustre/lustre/obdclass/obd_config.c
+++ b/drivers/staging/lustre/lustre/obdclass/obd_config.c
@@ -1417,7 +1417,9 @@ int class_config_llog_handler(const struct lu_env *env,
 int class_config_parse_llog(const struct lu_env *env, struct llog_ctxt *ctxt,
 			    char *name, struct config_llog_instance *cfg)
 {
-	struct llog_process_cat_data	 cd = {0, 0};
+	struct llog_process_cat_data cd = {
+		.lpcd_first_idx = 0,
+	};
 	struct llog_handle		*llh;
 	llog_cb_t			 callback;
 	int				 rc;
-- 
1.8.3.1


From jsimmons at infradead.org  Tue Jul 31 02:26:23 2018
From: jsimmons at infradead.org (James Simmons)
Date: Mon, 30 Jul 2018 22:26:23 -0400
Subject: [lustre-devel] [PATCH 31/31] lustre: docs: update TODO file
In-Reply-To: <1533003983-29311-1-git-send-email-jsimmons@infradead.org>
References: <1533003983-29311-1-git-send-email-jsimmons@infradead.org>
Message-ID: <1533003983-29311-32-git-send-email-jsimmons@infradead.org>

With several bugs fixed we can remove them from the TODO file.
Also update several email addresses.

Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/TODO | 51 ++++-----------------------------------------
 1 file changed, 4 insertions(+), 47 deletions(-)

diff --git a/drivers/staging/lustre/TODO b/drivers/staging/lustre/TODO
index 5332cdb..942280b 100644
--- a/drivers/staging/lustre/TODO
+++ b/drivers/staging/lustre/TODO
@@ -17,10 +17,6 @@ addressed:
 *
 ******************************************************************************
 
-https://jira.hpdd.intel.com/browse/LU-100086
-
-LNET_MINOR conflicts with USERIO_MINOR
-
 ------------------------------------------------------------------------------
 
 https://jira.hpdd.intel.com/browse/LU-8130
@@ -29,14 +25,6 @@ Fix and simplify libcfs hash handling
 
 ------------------------------------------------------------------------------
 
-https://jira.hpdd.intel.com/browse/LU-8703
-
-The current way we handle SMP is wrong. Platforms like ARM and KNL can have
-core and NUMA setups with things like NUMA nodes with no cores. We need to
-handle such cases. This work also greatly simplified the lustre SMP code.
-
-------------------------------------------------------------------------------
-
 https://jira.hpdd.intel.com/browse/LU-9019
 
 Replace libcfs time API with standard kernel APIs. Also migrate away from
@@ -53,14 +41,9 @@ Poor performance for the ko2iblnd driver. This is related to many of the
 patches below that are missing from the linux client.
 ------------------------------------------------------------------------------
 
-https://jira.hpdd.intel.com/browse/LU-9886
-
-Crash in upstream kiblnd_handle_early_rxs()
-------------------------------------------------------------------------------
-
 https://jira.hpdd.intel.com/browse/LU-10394 / LU-10526 / LU-10089
 
-Default to default to using MEM_REG
+Default to using MEM_REG
 ------------------------------------------------------------------------------
 
 https://jira.hpdd.intel.com/browse/LU-10459
@@ -98,11 +81,6 @@ https://jira.hpdd.intel.com/browse/LU-10129
 query device capabilities
 ------------------------------------------------------------------------------
 
-https://jira.hpdd.intel.com/browse/LU-10015
-
-fix race at kiblnd_connect_peer
-------------------------------------------------------------------------------
-
 https://jira.hpdd.intel.com/browse/LU-9983
 
 allow for discontiguous fragments
@@ -123,21 +101,11 @@ https://jira.hpdd.intel.com/browse/LU-9507
 Don't Assert On Reconnect with MultiQP
 ------------------------------------------------------------------------------
 
-https://jira.hpdd.intel.com/browse/LU-9472
-
-Fix FastReg map/unmap for MLX5
-------------------------------------------------------------------------------
-
 https://jira.hpdd.intel.com/browse/LU-9425
 
 Turn on 2 sges by default
 ------------------------------------------------------------------------------
 
-https://jira.hpdd.intel.com/browse/LU-8943
-
-Enable Multiple OPA Endpoints between Nodes
-------------------------------------------------------------------------------
-
 https://jira.hpdd.intel.com/browse/LU-5718
 
 multiple sges for work request
@@ -286,17 +254,6 @@ https://jira.hpdd.intel.com/browse/LU-9862
 Patch that landed for LU-7890 leads to static checker errors
 ------------------------------------------------------------------------------
 
-https://jira.hpdd.intel.com/browse/LU-9868
-
-dcache/namei fixes for lustre
-------------------------------------------------------------------------------
-
-https://jira.hpdd.intel.com/browse/LU-10467
-
-use standard linux wait_events macros work by Neil Brown
-
-------------------------------------------------------------------------------
-
-Please send any patches to Greg Kroah-Hartman <greg at kroah.com>, Andreas Dilger
-<andreas.dilger at intel.com>, James Simmons <jsimmons at infradead.org> and
-Oleg Drokin <oleg.drokin at intel.com>.
+Please send any patches to NeilBrown <neilb at suse.com>, Andreas Dilger
+<adilger at whamcloud.com>, James Simmons <jsimmons at infradead.org> and
+Oleg Drokin <green at whamcloud.com>.
-- 
1.8.3.1


From neilb at suse.com  Tue Jul 31 03:13:20 2018
From: neilb at suse.com (NeilBrown)
Date: Tue, 31 Jul 2018 13:13:20 +1000
Subject: [lustre-devel] [PATCH 1/3] lustre: obd: migrate
 lustre_fill_super() to llite
In-Reply-To: <1532992594-2576-2-git-send-email-jsimmons@infradead.org>
References: <1532992594-2576-1-git-send-email-jsimmons@infradead.org>
 <1532992594-2576-2-git-send-email-jsimmons@infradead.org>
Message-ID: <87tvogrtbj.fsf@notabene.neil.brown.name>

On Mon, Jul 30 2018, James Simmons wrote:

> Currently both obdclass and ptlrpc are built as separate modules
> with ptlrpc being a child of obdclass. A recent change placed
> ptlrpc_[inc|dec]_ref() into the obdclass code i.e obd_mount.c
> which now causes a curricular module dependency. Examining the

Oops, that was careless - sorry about that.
Maybe I should just revert that patch .... but that probably
doesn't actually make things easier.  Let's go with your fix.


> code the bulk use of these functions are in lustre_fill_super().
> So move lustre_fill_super() to the llite layer where it really
> belongs. This change also allows lustre_fill_super() to be
> simplified and we can push lustre_fill_super() instead of
> ll_fill_super(). Also ptlrpc_dec_ref() needed to be pulled out
> of lustre_common_put_super() as well. Once obdclass and ptlrpc
> are combined into one module we can restore ptlrpc_dec_rec()
> in lustre_common_put_super().

The path forward for unify the modules is not as smooth as I hoped.
Masahiro Yamada (kbuild maintainer) doesn't like the idea of building
one modules from multiple directories at all.  I might try again,
but it isn't looking good.

Two further comments below...

>
> Signed-off-by: James Simmons <jsimmons at infradead.org>
> ---
>  .../staging/lustre/lustre/include/lustre_disk.h    |   9 +-
>  drivers/staging/lustre/lustre/llite/llite_lib.c    |   3 +
>  drivers/staging/lustre/lustre/llite/super25.c      |  83 +++++++++++++-
>  drivers/staging/lustre/lustre/obdclass/obd_mount.c | 125 +++++----------------
>  4 files changed, 115 insertions(+), 105 deletions(-)
>
> diff --git a/drivers/staging/lustre/lustre/include/lustre_disk.h b/drivers/staging/lustre/lustre/include/lustre_disk.h
> index 1a6d421..772ecc9 100644
> --- a/drivers/staging/lustre/lustre/include/lustre_disk.h
> +++ b/drivers/staging/lustre/lustre/include/lustre_disk.h
> @@ -141,10 +141,13 @@ struct lustre_sb_info {
>  
>  /* obd_mount.c */
>  
> +struct lustre_sb_info *lustre_init_lsi(struct super_block *sb);
> +int lustre_put_lsi(struct super_block *sb);
> +int lmd_parse(char *options, struct lustre_mount_data *lmd);
>  int lustre_start_mgc(struct super_block *sb);
> -void lustre_register_super_ops(struct module *mod,
> -			       int (*cfs)(struct super_block *sb),
> -			       void (*ksc)(struct super_block *sb));
> +void lustre_register_super_ops(int (*cfs)(struct super_block *sb, void *data,
> +					  int silent),
> +                               void (*ksc)(struct super_block *sb));
>  int lustre_common_put_super(struct super_block *sb);
>  
>  int mgc_fsname2resid(char *fsname, struct ldlm_res_id *res_id, int type);
> diff --git a/drivers/staging/lustre/lustre/llite/llite_lib.c b/drivers/staging/lustre/lustre/llite/llite_lib.c
> index 5c8d0fe..87a929c 100644
> --- a/drivers/staging/lustre/lustre/llite/llite_lib.c
> +++ b/drivers/staging/lustre/lustre/llite/llite_lib.c
> @@ -914,6 +914,7 @@ int ll_fill_super(struct super_block *sb)
>  	cfg = kzalloc(sizeof(*cfg), GFP_NOFS);
>  	if (!cfg) {
>  		lustre_common_put_super(sb);
> +		ptlrpc_dec_ref();
>  		return -ENOMEM;
>  	}
>  
> @@ -926,6 +927,7 @@ int ll_fill_super(struct super_block *sb)
>  		module_put(THIS_MODULE);
>  		kfree(cfg);
>  		lustre_common_put_super(sb);
> +		ptlrpc_dec_ref();
>  		return -ENOMEM;
>  	}
>  
> @@ -1059,6 +1061,7 @@ void ll_put_super(struct super_block *sb)
>  	lsi->lsi_llsbi = NULL;
>  
>  	lustre_common_put_super(sb);
> +	ptlrpc_dec_ref();
>  
>  	cl_env_cache_purge(~0);
>  
> diff --git a/drivers/staging/lustre/lustre/llite/super25.c b/drivers/staging/lustre/lustre/llite/super25.c
> index d335f29..de43d58 100644
> --- a/drivers/staging/lustre/lustre/llite/super25.c
> +++ b/drivers/staging/lustre/lustre/llite/super25.c
> @@ -83,6 +83,85 @@ struct super_operations lustre_super_operations = {
>  };
>  MODULE_ALIAS_FS("lustre");
>  
> +/** This is the entry point for the mount call into Lustre.
> + * This is called when a server or client is mounted,
> + * and this is where we start setting things up.
> + * @param data Mount options (e.g. -o flock,abort_recov)
> + */
> +int lustre_fill_super(struct super_block *sb, void *lmd2_data, int silent)
> +{
> +	struct lustre_mount_data *lmd;
> +	struct lustre_sb_info *lsi;
> +	int rc;
> +
> +	CDEBUG(D_SUPER | D_CONFIG | D_VFSTRACE, "VFS Op: sb %p\n", sb);
> +
> +	lsi = lustre_init_lsi(sb);
> +	if (!lsi)
> +		return -ENOMEM;
> +	lmd = lsi->lsi_lmd;
> +
> +	/*
> +	 * Disable lockdep during mount, because mount locking patterns are
> +	 * `special'.
> +	 */
> +	lockdep_off();
> +
> +	/*
> +	 * LU-639: the obd cleanup of last mount may not finish yet, wait here.
> +	 */
> +	obd_zombie_barrier();
> +
> +	/* Figure out the lmd from the mount options */
> +	if (lmd_parse(lmd2_data, lmd)) {
> +		rc = -EINVAL;
> +		goto out_put_lsi;
> +	}
> +
> +	if (lmd_is_client(lmd)) {
> +		CDEBUG(D_SUPER | D_CONFIG, "Mounting client %s\n",
> +		       lmd->lmd_profile);
> +		rc = ptlrpc_inc_ref();
> +		if (rc)
> +			goto out_put_lsi;
> +
> +		rc = lustre_start_mgc(sb);
> +		if (rc) {
> +			/* This will put_lsi and ptlrpc_dec_ref() */

This comment is now wrong, or at least odd.

> +			lustre_common_put_super(sb);
> +			ptlrpc_dec_ref();
> +			goto out;
> +		}
> +
> +		/* Connect and start */
> +		rc = ll_fill_super(sb);
> +
> +		/*
> +		 * c_f_s will call lustre_common_put_super on failure, otherwise
> +		 * c_f_s will have taken another reference to the module
> +		 */
> +	} else {
> +		CERROR("This is client-side-only module, cannot handle server mount.\n");
> +		rc = -EINVAL;
> +	}
> +	/* If error happens in fill_super() call, @lsi will be killed there.
> +	 * This is why we do not put it here.
> +	 */
> +	goto out;
> +out_put_lsi:
> +	lustre_put_lsi(sb);
> +out:
> +	if (rc) {
> +		CERROR("Unable to mount %s (%d)\n",
> +		       s2lsi(sb) ? lmd->lmd_dev : "", rc);
> +	} else {
> +		CDEBUG(D_SUPER, "Mount %s complete\n",
> +		       lmd->lmd_dev);
> +	}
> +	lockdep_on();
> +	return rc;
> +}
> +
>  static int __init lustre_init(void)
>  {
>  	int rc;
> @@ -145,7 +224,7 @@ static int __init lustre_init(void)
>  	if (rc != 0)
>  		goto out_inode_fini_env;
>  
> -	lustre_register_super_ops(THIS_MODULE, ll_fill_super, ll_kill_super);
> +	lustre_register_super_ops(lustre_fill_super, ll_kill_super);
>  	lustre_register_client_process_config(ll_process_config);
>  
>  	return 0;
> @@ -166,7 +245,7 @@ static int __init lustre_init(void)
>  
>  static void __exit lustre_exit(void)
>  {
> -	lustre_register_super_ops(NULL, NULL, NULL);
> +	lustre_register_super_ops(NULL, NULL);
>  	lustre_register_client_process_config(NULL);
>  
>  	debugfs_remove(llite_root);
> diff --git a/drivers/staging/lustre/lustre/obdclass/obd_mount.c b/drivers/staging/lustre/lustre/obdclass/obd_mount.c
> index 1bd5613..ac841f4 100644
> --- a/drivers/staging/lustre/lustre/obdclass/obd_mount.c
> +++ b/drivers/staging/lustre/lustre/obdclass/obd_mount.c
> @@ -50,8 +50,8 @@
>  #include <uapi/linux/lustre/lustre_param.h>
>  
>  static DEFINE_SPINLOCK(client_lock);
> -static struct module *client_mod;
> -static int (*client_fill_super)(struct super_block *sb);
> +static int (*client_fill_super)(struct super_block *sb, void *data,
> +                                int silent);
>  static void (*kill_super_cb)(struct super_block *sb);
>  
>  /**************** config llog ********************/
> @@ -430,6 +430,7 @@ int lustre_start_mgc(struct super_block *sb)
>  	kfree(niduuid);
>  	return rc;
>  }
> +EXPORT_SYMBOL(lustre_start_mgc);
>  
>  static int lustre_stop_mgc(struct super_block *sb)
>  {
> @@ -507,7 +508,7 @@ static int lustre_stop_mgc(struct super_block *sb)
>  
>  /***************** lustre superblock **************/
>  
> -static struct lustre_sb_info *lustre_init_lsi(struct super_block *sb)
> +struct lustre_sb_info *lustre_init_lsi(struct super_block *sb)
>  {
>  	struct lustre_sb_info *lsi;
>  
> @@ -532,6 +533,7 @@ static struct lustre_sb_info *lustre_init_lsi(struct super_block *sb)
>  
>  	return lsi;
>  }
> +EXPORT_SYMBOL(lustre_init_lsi);
>  
>  static int lustre_free_lsi(struct super_block *sb)
>  {
> @@ -567,7 +569,7 @@ static int lustre_free_lsi(struct super_block *sb)
>  /* The lsi has one reference for every server that is using the disk -
>   * e.g. MDT, MGS, and potentially MGC
>   */
> -static int lustre_put_lsi(struct super_block *sb)
> +int lustre_put_lsi(struct super_block *sb)
>  {
>  	struct lustre_sb_info *lsi = s2lsi(sb);
>  
> @@ -578,6 +580,7 @@ static int lustre_put_lsi(struct super_block *sb)
>  	}
>  	return 0;
>  }
> +EXPORT_SYMBOL(lustre_put_lsi);
>  
>  /*** SERVER NAME ***
>   * <FSNAME><SEPARATOR><TYPE><INDEX>
> @@ -686,7 +689,12 @@ int lustre_common_put_super(struct super_block *sb)
>  	}
>  	/* Drop a ref to the mounted disk */
>  	lustre_put_lsi(sb);
> -	ptlrpc_dec_ref();
> +	/* !!! FIXME !!!!! */
> +	/* ptlrpc_dec_ref() should go here but will
> +	 * cause curricular module dependencies. Once
> +	 * obdclass and ptlrpc are merged into one
> +	 * module ptlrpc_dec_ref() can come back here
> +	 */
>  	return rc;
>  }
>  EXPORT_SYMBOL(lustre_common_put_super);
> @@ -992,7 +1000,7 @@ static int lmd_parse_nidlist(char *buf, char **endh)
>   * e.g. mount -v -t lustre -o abort_recov uml1:uml2:/lustre-client /mnt/lustre
>   * dev is passed as device=uml1:/lustre by mount.lustre
>   */
> -static int lmd_parse(char *options, struct lustre_mount_data *lmd)
> +int lmd_parse(char *options, struct lustre_mount_data *lmd)
>  {
>  	char *s1, *s2, *devname = NULL;
>  	struct lustre_mount_data *raw = (struct lustre_mount_data *)options;
> @@ -1216,106 +1224,16 @@ static int lmd_parse(char *options, struct lustre_mount_data *lmd)
>  	CERROR("Bad mount options %s\n", options);
>  	return -EINVAL;
>  }
> -
> -/** This is the entry point for the mount call into Lustre.
> - * This is called when a server or client is mounted,
> - * and this is where we start setting things up.
> - * @param data Mount options (e.g. -o flock,abort_recov)
> - */
> -static int lustre_fill_super(struct super_block *sb, void *lmd2_data, int silent)
> -{
> -	struct lustre_mount_data *lmd;
> -	struct lustre_sb_info *lsi;
> -	int rc;
> -
> -	CDEBUG(D_MOUNT | D_VFSTRACE, "VFS Op: sb %p\n", sb);
> -
> -	lsi = lustre_init_lsi(sb);
> -	if (!lsi)
> -		return -ENOMEM;
> -	lmd = lsi->lsi_lmd;
> -
> -	/*
> -	 * Disable lockdep during mount, because mount locking patterns are
> -	 * `special'.
> -	 */
> -	lockdep_off();
> -
> -	/*
> -	 * LU-639: the obd cleanup of last mount may not finish yet, wait here.
> -	 */
> -	obd_zombie_barrier();
> -
> -	/* Figure out the lmd from the mount options */
> -	if (lmd_parse(lmd2_data, lmd)) {
> -		rc = -EINVAL;
> -		goto out_put_lsi;
> -	}
> -
> -	if (lmd_is_client(lmd)) {
> -		bool have_client = false;
> -		CDEBUG(D_MOUNT, "Mounting client %s\n", lmd->lmd_profile);
> -		if (!client_fill_super)
> -			request_module("lustre");
> -		rc = ptlrpc_inc_ref();
> -		if (rc)
> -			goto out_put_lsi;
> -		spin_lock(&client_lock);
> -		if (client_fill_super && try_module_get(client_mod))
> -			have_client = true;
> -		spin_unlock(&client_lock);
> -		if (!have_client) {
> -			LCONSOLE_ERROR_MSG(0x165, "Nothing registered for client mount! Is the 'lustre' module loaded?\n");
> -			lustre_put_lsi(sb);
> -			rc = -ENODEV;
> -		} else {
> -			rc = lustre_start_mgc(sb);
> -			if (rc) {
> -				/* This will put_lsi and ptlrpc_dec_ref() */
> -				lustre_common_put_super(sb);
> -				goto out;
> -			}
> -			/* Connect and start */
> -			/* (should always be ll_fill_super) */
> -			rc = (*client_fill_super)(sb);
> -			/*
> -			 * c_f_s will call lustre_common_put_super on failure, otherwise
> -			 * c_f_s will have taken another reference to the module
> -			 */
> -			module_put(client_mod);
> -		}
> -	} else {
> -		CERROR("This is client-side-only module, cannot handle server mount.\n");
> -		rc = -EINVAL;
> -	}
> -
> -	/* If error happens in fill_super() call, @lsi will be killed there.
> -	 * This is why we do not put it here.
> -	 */
> -	goto out;
> -out_put_lsi:
> -	lustre_put_lsi(sb);
> -out:
> -	if (rc) {
> -		CERROR("Unable to mount %s (%d)\n",
> -		       s2lsi(sb) ? lmd->lmd_dev : "", rc);
> -	} else {
> -		CDEBUG(D_SUPER, "Mount %s complete\n",
> -		       lmd->lmd_dev);
> -	}
> -	lockdep_on();
> -	return rc;
> -}
> +EXPORT_SYMBOL(lmd_parse);
>  
>  /* We can't call ll_fill_super by name because it lives in a module that
>   * must be loaded after this one.
>   */
> -void lustre_register_super_ops(struct module *mod,
> -			       int (*cfs)(struct super_block *sb),
> +void lustre_register_super_ops(int (*cfs)(struct super_block *sb, void *data,
> +					  int silent),
>  			       void (*ksc)(struct super_block *sb))
>  {
>  	spin_lock(&client_lock);
> -	client_mod = mod;
>  	client_fill_super = cfs;
>  	kill_super_cb = ksc;
>  	spin_unlock(&client_lock);
> @@ -1326,7 +1244,14 @@ void lustre_register_super_ops(struct module *mod,
>  static struct dentry *lustre_mount(struct file_system_type *fs_type, int flags,
>  				   const char *devname, void *data)
>  {
> -	return mount_nodev(fs_type, flags, data, lustre_fill_super);
> +	struct dentry *root;
> +
> +	spin_lock(&client_lock);
> +	if (!client_fill_super)
> +		request_module("lustre");
> +	root = mount_nodev(fs_type, flags, data, client_fill_super);
> +	spin_unlock(&client_lock);

You cannot call request_module() inside spinlock - it waits for
'modprobe' to run.

I think it is best to move all the bits that need to move between
modules all at once.

Thanks,
NeilBrown


> +	return root;
>  }
>  
>  static void lustre_kill_super(struct super_block *sb)
> -- 
> 1.8.3.1
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180731/399c22aa/attachment.sig>

From paf at cray.com  Tue Jul 31 04:05:52 2018
From: paf at cray.com (Patrick Farrell)
Date: Tue, 31 Jul 2018 04:05:52 +0000
Subject: [lustre-devel] [PATCH 06/31] lustre: llite: reduce jobstats
 race window
In-Reply-To: <1533003983-29311-7-git-send-email-jsimmons@infradead.org>
References: <1533003983-29311-1-git-send-email-jsimmons@infradead.org>,
 <1533003983-29311-7-git-send-email-jsimmons@infradead.org>
Message-ID: <DM3PR1101MB12138EBC47D99DAE3752FCC0CB2E0@DM3PR1101MB1213.namprd11.prod.outlook.com>

I'm puzzled, James - Why is "cache_jobid" in there?  Isn't that from Ben Evans' work?  This patch landed before all of that...

________________________________
From: James Simmons <jsimmons at infradead.org>
Sent: Monday, July 30, 2018 9:25:58 PM
To: Andreas Dilger; Oleg Drokin; NeilBrown
Cc: Lustre Development List; Patrick Farrell; James Simmons
Subject: [PATCH 06/31] lustre: llite: reduce jobstats race window

From: Patrick Farrell <paf at cray.com>

In the current code, lli_jobid is set to zero on every call
to lustre_get_jobid.  This causes problems, because it's
used asynchronously to set the job id in RPCs, and some
RPCs will falsely get no jobid set.  (For small IO sizes,
this can be up to 60% of RPCs.)

It would be very expensive to put hard synchronization
between this and every outbound RPC, and it's OK to very
rarely get an RPC without correct job stats info.

This patch only updates the lli_jobid when the job id has
changed, which leaves only a very small window for reading
an inconsistent job id.

Signed-off-by: Patrick Farrell <paf at cray.com>
WC-id: https://jira.whamcloud.com/browse/LU-8926
Reviewed-on: https://review.whamcloud.com/24253
Reviewed-by: Andreas Dilger <adilger at whamcloud.com>
Reviewed-by: Chris Horn <hornc at cray.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
 drivers/staging/lustre/lustre/llite/llite_lib.c    |  1 +
 drivers/staging/lustre/lustre/obdclass/class_obd.c | 20 ++++++++++++++------
 2 files changed, 15 insertions(+), 6 deletions(-)

diff --git a/drivers/staging/lustre/lustre/llite/llite_lib.c b/drivers/staging/lustre/lustre/llite/llite_lib.c
index c0861b9..72b118a 100644
--- a/drivers/staging/lustre/lustre/llite/llite_lib.c
+++ b/drivers/staging/lustre/lustre/llite/llite_lib.c
@@ -894,6 +894,7 @@ void ll_lli_init(struct ll_inode_info *lli)
                 lli->lli_async_rc = 0;
         }
         mutex_init(&lli->lli_layout_mutex);
+       memset(lli->lli_jobid, 0, LUSTRE_JOBID_SIZE);
 }

 int ll_fill_super(struct super_block *sb)
diff --git a/drivers/staging/lustre/lustre/obdclass/class_obd.c b/drivers/staging/lustre/lustre/obdclass/class_obd.c
index cdaf729..87327ef 100644
--- a/drivers/staging/lustre/lustre/obdclass/class_obd.c
+++ b/drivers/staging/lustre/lustre/obdclass/class_obd.c
@@ -95,26 +95,34 @@
  */
 int lustre_get_jobid(char *jobid)
 {
-       memset(jobid, 0, LUSTRE_JOBID_SIZE);
+       char tmp_jobid[LUSTRE_JOBID_SIZE] = { 0 };
+
         /* Jobstats isn't enabled */
         if (strcmp(obd_jobid_var, JOBSTATS_DISABLE) == 0)
-               return 0;
+               goto out_cache_jobid;

         /* Use process name + fsuid as jobid */
         if (strcmp(obd_jobid_var, JOBSTATS_PROCNAME_UID) == 0) {
-               snprintf(jobid, LUSTRE_JOBID_SIZE, "%s.%u",
+               snprintf(tmp_jobid, LUSTRE_JOBID_SIZE, "%s.%u",
                          current->comm,
                          from_kuid(&init_user_ns, current_fsuid()));
-               return 0;
+               goto out_cache_jobid;
         }

         /* Whole node dedicated to single job */
         if (strcmp(obd_jobid_var, JOBSTATS_NODELOCAL) == 0) {
-               strcpy(jobid, obd_jobid_node);
-               return 0;
+               strcpy(tmp_jobid, obd_jobid_node);
+               goto out_cache_jobid;
         }

         return -ENOENT;
+
+out_cache_jobid:
+       /* Only replace the job ID if it changed. */
+       if (strcmp(jobid, tmp_jobid) != 0)
+               strcpy(jobid, tmp_jobid);
+
+       return 0;
 }
 EXPORT_SYMBOL(lustre_get_jobid);

--
1.8.3.1

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180731/9cd3af6a/attachment.html>

From neilb at suse.com  Tue Jul 31 22:31:21 2018
From: neilb at suse.com (NeilBrown)
Date: Wed, 01 Aug 2018 08:31:21 +1000
Subject: [lustre-devel] [PATCH 08/31] lustre: llite: don't zero
 timestamps internally
In-Reply-To: <1533003983-29311-9-git-send-email-jsimmons@infradead.org>
References: <1533003983-29311-1-git-send-email-jsimmons@infradead.org>
 <1533003983-29311-9-git-send-email-jsimmons@infradead.org>
Message-ID: <87d0v3rq9y.fsf@notabene.neil.brown.name>

On Mon, Jul 30 2018, James Simmons wrote:
> --- a/drivers/staging/lustre/lustre/llite/llite_internal.h
> +++ b/drivers/staging/lustre/lustre/llite/llite_internal.h
> @@ -138,6 +138,11 @@ struct ll_inode_info {
>  	s64				lli_ctime;
>  	spinlock_t			lli_agl_lock;
>  
> +	/* update atime from MDS no matter if it's older than
> +	 * local inode atime.
> +	 */
> +	unsigned int			lli_update_atime:1;
> +

Why not make this another flag bit in lli_flags ??

I might add a patch to do that.

NeilBrown
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180801/6c1efa30/attachment.sig>

From neilb at suse.com  Tue Jul 31 22:32:49 2018
From: neilb at suse.com (NeilBrown)
Date: Wed, 01 Aug 2018 08:32:49 +1000
Subject: [lustre-devel] [PATCH 07/31] lustre: lnet: change positional
 struct initializers to C99
In-Reply-To: <1533003983-29311-8-git-send-email-jsimmons@infradead.org>
References: <1533003983-29311-1-git-send-email-jsimmons@infradead.org>
 <1533003983-29311-8-git-send-email-jsimmons@infradead.org>
Message-ID: <87a7q7rq7i.fsf@notabene.neil.brown.name>

On Mon, Jul 30 2018, James Simmons wrote:
> --- a/drivers/staging/lustre/lnet/lnet/lo.c
> +++ b/drivers/staging/lustre/lnet/lnet/lo.c
> @@ -91,15 +91,13 @@
>  }
>  
>  struct lnet_lnd the_lolnd = {
> -	/* .lnd_list       = */ {&the_lolnd.lnd_list, &the_lolnd.lnd_list},
> -	/* .lnd_refcount   = */ 0,
> -	/* .lnd_type       = */ LOLND,
> -	/* .lnd_startup    = */ lolnd_startup,
> -	/* .lnd_shutdown   = */ lolnd_shutdown,
> -	/* .lnt_ctl        = */ NULL,
> -	/* .lnd_send       = */ lolnd_send,
> -	/* .lnd_recv       = */ lolnd_recv,
> -	/* .lnd_eager_recv = */ NULL,
> -	/* .lnd_notify     = */ NULL,
> -	/* .lnd_accept     = */ NULL
> +	.lnd_list	= {
> +				.next	= &the_lolnd.lnd_list,
> +				.prev	= &the_lolnd.lnd_list
> +			},

That would be better as
        .lnd_list       = LIST_HEAD_INIT(the_lolnd.lnd_list),

I'll queue a patch to make that change.

NeilBrown
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180801/c1dc9566/attachment.sig>

From neilb at suse.com  Tue Jul 31 22:38:50 2018
From: neilb at suse.com (NeilBrown)
Date: Wed, 01 Aug 2018 08:38:50 +1000
Subject: [lustre-devel] [PATCH 11/31] lustre: lnet: Fix route hops print
In-Reply-To: <1533003983-29311-12-git-send-email-jsimmons@infradead.org>
References: <1533003983-29311-1-git-send-email-jsimmons@infradead.org>
 <1533003983-29311-12-git-send-email-jsimmons@infradead.org>
Message-ID: <877elbrpxh.fsf@notabene.neil.brown.name>

On Mon, Jul 30 2018, James Simmons wrote:

> From: Amir Shehata <ashehata at whamcloud.com>
>
> The default number of hops for  a route is -1. This is
> currently being printed as %u. Change that to %d to
> make it print out properly.

-1 hops???  I wish I could hop -1 times - it would be a good party
trick!!

What does -1 mean?  Unlimited (just a guess).  If so, could we print
"unlimited"??

I'm fine with having magic numbers in the code, but I don't like them to
leak out.

NeilBrown

>
> Signed-off-by: Amir Shehata <ashehata at whamcloud.com>
> WC-id: https://jira.whamcloud.com/browse/LU-9078
> Reviewed-on: https://review.whamcloud.com/25250
> Reviewed-by: Olaf Weber <olaf at sgi.com>
> Reviewed-by: Doug Oucharek <dougso at me.com>
> Reviewed-by: James Simmons <uja.ornl at yahoo.com>
> Reviewed-by: Oleg Drokin <green at whamcloud.com>
> Signed-off-by: James Simmons <jsimmons at infradead.org>
> ---
>  drivers/staging/lustre/lnet/lnet/router_proc.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/drivers/staging/lustre/lnet/lnet/router_proc.c b/drivers/staging/lustre/lnet/lnet/router_proc.c
> index 8856798..aa98ce5 100644
> --- a/drivers/staging/lustre/lnet/lnet/router_proc.c
> +++ b/drivers/staging/lustre/lnet/lnet/router_proc.c
> @@ -218,7 +218,7 @@ static int proc_lnet_routes(struct ctl_table *table, int write,
>  			int alive = lnet_is_route_alive(route);
>  
>  			s += snprintf(s, tmpstr + tmpsiz - s,
> -				      "%-8s %4u %8u %7s %s\n",
> +				      "%-8s %4d %8u %7s %s\n",
>  				      libcfs_net2str(net), hops,
>  				      priority,
>  				      alive ? "up" : "down",
> -- 
> 1.8.3.1
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180801/ce9b5960/attachment.sig>

From neilb at suse.com  Tue Jul 31 22:41:34 2018
From: neilb at suse.com (NeilBrown)
Date: Wed, 01 Aug 2018 08:41:34 +1000
Subject: [lustre-devel] [PATCH 13/31] lustre: config: don't attach sub
 logs for LWP
In-Reply-To: <1533003983-29311-14-git-send-email-jsimmons@infradead.org>
References: <1533003983-29311-1-git-send-email-jsimmons@infradead.org>
 <1533003983-29311-14-git-send-email-jsimmons@infradead.org>
Message-ID: <874lgfrpsx.fsf@notabene.neil.brown.name>

On Mon, Jul 30 2018, James Simmons wrote:

> From: Niu Yawei <yawei.niu at intel.com>
>
> Lustre target processes client log to retrieve MDT NIDs and start
> LWPs, it goes the same code path of mgc_process_config() just like
> processing the target config log, so that sub clds for security,
> nodemap, param & recovery will be attached unnecessarily.
>
> The mgc subsystem is used by both server and client. This change
> allows us to cleanly handle the future case when the mgc layer
> would be built with server code. This way server specific config
> logs will only be processed when a server mount occurs.
>
> Signed-off-by: Niu Yawei <yawei.niu at intel.com>
> WC-id: https://jira.whamcloud.com/browse/LU-9081
> Reviewed-on: https://review.whamcloud.com/25293
> Reviewed-by: Fan Yong <fan.yong at intel.com>
> Reviewed-by: Hongchao Zhang <hongchao at whamcloud.com>
> Reviewed-by: Oleg Drokin <green at whamcloud.com>
> Signed-off-by: James Simmons <jsimmons at infradead.org>
> ---
>  drivers/staging/lustre/lustre/include/obd_class.h | 19 +++++++-----
>  drivers/staging/lustre/lustre/llite/llite_lib.c   |  1 +
>  drivers/staging/lustre/lustre/mgc/mgc_request.c   | 37 +++++++++++++----------
>  3 files changed, 34 insertions(+), 23 deletions(-)
>
> diff --git a/drivers/staging/lustre/lustre/include/obd_class.h b/drivers/staging/lustre/lustre/include/obd_class.h
> index adfe2ab..e772e3d 100644
> --- a/drivers/staging/lustre/lustre/include/obd_class.h
> +++ b/drivers/staging/lustre/lustre/include/obd_class.h
> @@ -153,17 +153,22 @@ struct config_llog_instance {
>  	llog_cb_t	    cfg_callback;
>  	int		    cfg_last_idx; /* for partial llog processing */
>  	int		    cfg_flags;
> +	u32		    cfg_sub_clds;
>  };
>  
>  int class_config_parse_llog(const struct lu_env *env, struct llog_ctxt *ctxt,
>  			    char *name, struct config_llog_instance *cfg);
> -enum {
> -	CONFIG_T_CONFIG  = 0,
> -	CONFIG_T_SPTLRPC = 1,
> -	CONFIG_T_RECOVER = 2,
> -	CONFIG_T_PARAMS  = 3,
> -	CONFIG_T_MAX     = 4
> -};
> +
> +#define CONFIG_T_CONFIG		BIT(0)
> +#define CONFIG_T_SPTLRPC	BIT(1)
> +#define CONFIG_T_RECOVER	BIT(2)
> +#define CONFIG_T_PARAMS		BIT(3)

This could still be an enum:

enum {
	CONFIG_T_CONFIG  = BIT(0),
	CONFIG_T_SPTLRPC = BIT(1),
	CONFIG_T_RECOVER = BIT(2),
	CONFIG_T_PARAMS  = BIT(3),
};

and I'm glad "CONFIG_T_MAX" is gone - as the MAX was clearly '3', not '4'.

NeilBrown
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180801/167df322/attachment.sig>

From neilb at suse.com  Tue Jul 31 22:47:28 2018
From: neilb at suse.com (NeilBrown)
Date: Wed, 01 Aug 2018 08:47:28 +1000
Subject: [lustre-devel] [PATCH 25/31] lustre: config: move config types
 into lustre_idl.h
In-Reply-To: <1533003983-29311-26-git-send-email-jsimmons@infradead.org>
References: <1533003983-29311-1-git-send-email-jsimmons@infradead.org>
 <1533003983-29311-26-git-send-email-jsimmons@infradead.org>
Message-ID: <871sbjrpj3.fsf@notabene.neil.brown.name>

On Mon, Jul 30 2018, James Simmons wrote:

> From: Niu Yawei <yawei.niu at intel.com>
>
> Move config type values CONFIG_T_XXX into lustre_idl.h since they
> will be put on wire when reading config logs.
>
> Add missing wire checks for mgs_nidtbl_entry, mgs_config_body and
> mgs_config_res.
>
> Redefine CONFIG_SUB_XXX for the sub clds attached on config log.
>
> Signed-off-by: Niu Yawei <yawei.niu at intel.com>
> WC-id: https://jira.whamcloud.com/browse/LU-9216
> Reviewed-on: https://review.whamcloud.com/26022
> Reviewed-by: Fan Yong <fan.yong at intel.com>
> Reviewed-by: John L. Hammond <jhammond at whamcloud.com>
> Reviewed-by: Oleg Drokin <green at whamcloud.com>
> Signed-off-by: James Simmons <jsimmons at infradead.org>
> ---
>  .../lustre/include/uapi/linux/lustre/lustre_idl.h  | 10 ++-
>  drivers/staging/lustre/lustre/include/obd_class.h  | 12 +--
>  drivers/staging/lustre/lustre/mgc/mgc_request.c    |  6 +-
>  drivers/staging/lustre/lustre/ptlrpc/wiretest.c    | 85 ++++++++++++++++++++++
>  4 files changed, 103 insertions(+), 10 deletions(-)
>
> diff --git a/drivers/staging/lustre/include/uapi/linux/lustre/lustre_idl.h b/drivers/staging/lustre/include/uapi/linux/lustre/lustre_idl.h
> index c9b32ef..bd3b45a 100644
> --- a/drivers/staging/lustre/include/uapi/linux/lustre/lustre_idl.h
> +++ b/drivers/staging/lustre/include/uapi/linux/lustre/lustre_idl.h
> @@ -2111,11 +2111,19 @@ struct mgs_nidtbl_entry {
>  	} u;
>  };
>  
> +enum {
> +	CONFIG_T_CONFIG  = 0,
> +	CONFIG_T_SPTLRPC = 1,
> +	CONFIG_T_RECOVER = 2,
> +	CONFIG_T_PARAMS  = 3,
> +	CONFIG_T_MAX

Arrrgggh.  It's back.  I thought we had killed CONFIG_T_MAX (which isn't
a MAX).
It's never used, so it'll have to go.

NeilBrown


> +};
> +
>  struct mgs_config_body {
>  	char		mcb_name[MTI_NAME_MAXLEN]; /* logname */
>  	__u64		mcb_offset;    /* next index of config log to request */
>  	__u16		mcb_type;      /* type of log: CONFIG_T_[CONFIG|RECOVER] */
> -	__u8		mcb_reserved;
> +	__u8		mcb_nm_cur_pass;
>  	__u8		mcb_bits;      /* bits unit size of config log */
>  	__u32		mcb_units;     /* # of units for bulk transfer */
>  };
> diff --git a/drivers/staging/lustre/lustre/include/obd_class.h b/drivers/staging/lustre/lustre/include/obd_class.h
> index 184da99..647cc22 100644
> --- a/drivers/staging/lustre/lustre/include/obd_class.h
> +++ b/drivers/staging/lustre/lustre/include/obd_class.h
> @@ -156,16 +156,16 @@ struct config_llog_instance {
>  int class_config_parse_llog(const struct lu_env *env, struct llog_ctxt *ctxt,
>  			    char *name, struct config_llog_instance *cfg);
>  
> -#define CONFIG_T_CONFIG		BIT(0)
> -#define CONFIG_T_SPTLRPC	BIT(1)
> -#define CONFIG_T_RECOVER	BIT(2)
> -#define CONFIG_T_PARAMS		BIT(3)
> +#define CONFIG_SUB_CONFIG	BIT(0)
> +#define CONFIG_SUB_SPTLRPC	BIT(1)
> +#define CONFIG_SUB_RECOVER	BIT(2)
> +#define CONFIG_SUB_PARAMS	BIT(3)
>  
>  /* Sub clds should be attached to the config_llog_data when processing
>   * config log for client or server target.
>   */
> -#define CONFIG_SUB_CLIENT	(CONFIG_T_SPTLRPC | CONFIG_T_RECOVER | \
> -				 CONFIG_T_PARAMS)
> +#define CONFIG_SUB_CLIENT	(CONFIG_SUB_SPTLRPC | CONFIG_SUB_RECOVER | \
> +				 CONFIG_SUB_PARAMS)
>  
>  #define PARAMS_FILENAME	"params"
>  #define LCTL_UPCALL	"lctl"
> diff --git a/drivers/staging/lustre/lustre/mgc/mgc_request.c b/drivers/staging/lustre/lustre/mgc/mgc_request.c
> index 06fcc7e..833e6a0 100644
> --- a/drivers/staging/lustre/lustre/mgc/mgc_request.c
> +++ b/drivers/staging/lustre/lustre/mgc/mgc_request.c
> @@ -315,7 +315,7 @@ struct config_llog_data *do_config_log_add(struct obd_device *obd,
>  	memcpy(seclogname, logname, ptr - logname);
>  	strcpy(seclogname + (ptr - logname), "-sptlrpc");
>  
> -	if (cfg->cfg_sub_clds & CONFIG_T_SPTLRPC) {
> +	if (cfg->cfg_sub_clds & CONFIG_SUB_SPTLRPC) {
>  		sptlrpc_cld = config_log_find_or_add(obd, seclogname, NULL,
>  						     CONFIG_T_SPTLRPC, cfg);
>  		if (IS_ERR(sptlrpc_cld)) {
> @@ -325,7 +325,7 @@ struct config_llog_data *do_config_log_add(struct obd_device *obd,
>  		}
>  	}
>  
> -	if (cfg->cfg_sub_clds & CONFIG_T_PARAMS) {
> +	if (cfg->cfg_sub_clds & CONFIG_SUB_PARAMS) {
>  		params_cld = config_log_find_or_add(obd, PARAMS_FILENAME, sb,
>  						    CONFIG_T_PARAMS, cfg);
>  		if (IS_ERR(params_cld)) {
> @@ -345,7 +345,7 @@ struct config_llog_data *do_config_log_add(struct obd_device *obd,
>  
>  	LASSERT(lsi->lsi_lmd);
>  	if (!(lsi->lsi_lmd->lmd_flags & LMD_FLG_NOIR) &&
> -	    cfg->cfg_sub_clds & CONFIG_T_RECOVER) {
> +	    cfg->cfg_sub_clds & CONFIG_SUB_RECOVER) {
>  		ptr = strrchr(seclogname, '-');
>  		if (ptr) {
>  			*ptr = 0;
> diff --git a/drivers/staging/lustre/lustre/ptlrpc/wiretest.c b/drivers/staging/lustre/lustre/ptlrpc/wiretest.c
> index 2f081ed..09b1298 100644
> --- a/drivers/staging/lustre/lustre/ptlrpc/wiretest.c
> +++ b/drivers/staging/lustre/lustre/ptlrpc/wiretest.c
> @@ -3629,6 +3629,91 @@ void lustre_assert_wire_constants(void)
>  	LASSERTF((int)sizeof(((struct mgs_target_info *)0)->mti_params) == 4096, "found %lld\n",
>  		 (long long)(int)sizeof(((struct mgs_target_info *)0)->mti_params));
>  
> +	/* Checks for struct mgs_nidtbl_entry */
> +	LASSERTF((int)sizeof(struct mgs_nidtbl_entry) == 24, "found %lld\n",
> +		 (long long)(int)sizeof(struct mgs_nidtbl_entry));
> +	LASSERTF((int)offsetof(struct mgs_nidtbl_entry, mne_version) == 0, "found %lld\n",
> +		 (long long)(int)offsetof(struct mgs_nidtbl_entry, mne_version));
> +	LASSERTF((int)sizeof(((struct mgs_nidtbl_entry *)0)->mne_version) == 8, "found %lld\n",
> +		 (long long)(int)sizeof(((struct mgs_nidtbl_entry *)0)->mne_version));
> +	LASSERTF((int)offsetof(struct mgs_nidtbl_entry, mne_instance) == 8, "found %lld\n",
> +		 (long long)(int)offsetof(struct mgs_nidtbl_entry, mne_instance));
> +	LASSERTF((int)sizeof(((struct mgs_nidtbl_entry *)0)->mne_instance) == 4, "found %lld\n",
> +		 (long long)(int)sizeof(((struct mgs_nidtbl_entry *)0)->mne_instance));
> +	LASSERTF((int)offsetof(struct mgs_nidtbl_entry, mne_index) == 12, "found %lld\n",
> +		 (long long)(int)offsetof(struct mgs_nidtbl_entry, mne_index));
> +	LASSERTF((int)sizeof(((struct mgs_nidtbl_entry *)0)->mne_index) == 4, "found %lld\n",
> +		 (long long)(int)sizeof(((struct mgs_nidtbl_entry *)0)->mne_index));
> +	LASSERTF((int)offsetof(struct mgs_nidtbl_entry, mne_length) == 16, "found %lld\n",
> +		 (long long)(int)offsetof(struct mgs_nidtbl_entry, mne_length));
> +	LASSERTF((int)sizeof(((struct mgs_nidtbl_entry *)0)->mne_length) == 4, "found %lld\n",
> +		 (long long)(int)sizeof(((struct mgs_nidtbl_entry *)0)->mne_length));
> +	LASSERTF((int)offsetof(struct mgs_nidtbl_entry, mne_type) == 20, "found %lld\n",
> +		 (long long)(int)offsetof(struct mgs_nidtbl_entry, mne_type));
> +	LASSERTF((int)sizeof(((struct mgs_nidtbl_entry *)0)->mne_type) == 1, "found %lld\n",
> +		 (long long)(int)sizeof(((struct mgs_nidtbl_entry *)0)->mne_type));
> +	LASSERTF((int)offsetof(struct mgs_nidtbl_entry, mne_nid_type) == 21, "found %lld\n",
> +		 (long long)(int)offsetof(struct mgs_nidtbl_entry, mne_nid_type));
> +	LASSERTF((int)sizeof(((struct mgs_nidtbl_entry *)0)->mne_nid_type) == 1, "found %lld\n",
> +		 (long long)(int)sizeof(((struct mgs_nidtbl_entry *)0)->mne_nid_type));
> +	LASSERTF((int)offsetof(struct mgs_nidtbl_entry, mne_nid_size) == 22, "found %lld\n",
> +		 (long long)(int)offsetof(struct mgs_nidtbl_entry, mne_nid_size));
> +	LASSERTF((int)sizeof(((struct mgs_nidtbl_entry *)0)->mne_nid_size) == 1, "found %lld\n",
> +		 (long long)(int)sizeof(((struct mgs_nidtbl_entry *)0)->mne_nid_size));
> +	LASSERTF((int)offsetof(struct mgs_nidtbl_entry, mne_nid_count) == 23, "found %lld\n",
> +		 (long long)(int)offsetof(struct mgs_nidtbl_entry, mne_nid_count));
> +	LASSERTF((int)sizeof(((struct mgs_nidtbl_entry *)0)->mne_nid_count) == 1, "found %lld\n",
> +		 (long long)(int)sizeof(((struct mgs_nidtbl_entry *)0)->mne_nid_count));
> +	LASSERTF((int)offsetof(struct mgs_nidtbl_entry, u.nids[0]) == 24, "found %lld\n",
> +		 (long long)(int)offsetof(struct mgs_nidtbl_entry, u.nids[0]));
> +	LASSERTF((int)sizeof(((struct mgs_nidtbl_entry *)0)->u.nids[0]) == 8, "found %lld\n",
> +		 (long long)(int)sizeof(((struct mgs_nidtbl_entry *)0)->u.nids[0]));
> +
> +	/* Checks for struct mgs_config_body */
> +	LASSERTF((int)sizeof(struct mgs_config_body) == 80, "found %lld\n",
> +		 (long long)(int)sizeof(struct mgs_config_body));
> +	LASSERTF((int)offsetof(struct mgs_config_body, mcb_name) == 0, "found %lld\n",
> +		 (long long)(int)offsetof(struct mgs_config_body, mcb_name));
> +	LASSERTF((int)sizeof(((struct mgs_config_body *)0)->mcb_name) == 64, "found %lld\n",
> +		 (long long)(int)sizeof(((struct mgs_config_body *)0)->mcb_name));
> +	LASSERTF((int)offsetof(struct mgs_config_body, mcb_offset) == 64, "found %lld\n",
> +		 (long long)(int)offsetof(struct mgs_config_body, mcb_offset));
> +	LASSERTF((int)sizeof(((struct mgs_config_body *)0)->mcb_offset) == 8, "found %lld\n",
> +		 (long long)(int)sizeof(((struct mgs_config_body *)0)->mcb_offset));
> +	LASSERTF((int)offsetof(struct mgs_config_body, mcb_type) == 72, "found %lld\n",
> +		 (long long)(int)offsetof(struct mgs_config_body, mcb_type));
> +	LASSERTF((int)sizeof(((struct mgs_config_body *)0)->mcb_type) == 2, "found %lld\n",
> +		 (long long)(int)sizeof(((struct mgs_config_body *)0)->mcb_type));
> +	LASSERTF((int)offsetof(struct mgs_config_body, mcb_nm_cur_pass) == 74, "found %lld\n",
> +		 (long long)(int)offsetof(struct mgs_config_body, mcb_nm_cur_pass));
> +	LASSERTF((int)sizeof(((struct mgs_config_body *)0)->mcb_nm_cur_pass) == 1, "found %lld\n",
> +		 (long long)(int)sizeof(((struct mgs_config_body *)0)->mcb_nm_cur_pass));
> +	LASSERTF((int)offsetof(struct mgs_config_body, mcb_bits) == 75, "found %lld\n",
> +		 (long long)(int)offsetof(struct mgs_config_body, mcb_bits));
> +	LASSERTF((int)sizeof(((struct mgs_config_body *)0)->mcb_bits) == 1, "found %lld\n",
> +		 (long long)(int)sizeof(((struct mgs_config_body *)0)->mcb_bits));
> +	LASSERTF((int)offsetof(struct mgs_config_body, mcb_units) == 76, "found %lld\n",
> +		 (long long)(int)offsetof(struct mgs_config_body, mcb_units));
> +	LASSERTF((int)sizeof(((struct mgs_config_body *)0)->mcb_units) == 4, "found %lld\n",
> +		 (long long)(int)sizeof(((struct mgs_config_body *)0)->mcb_units));
> +
> +	BUILD_BUG_ON(CONFIG_T_CONFIG != 0);
> +	BUILD_BUG_ON(CONFIG_T_SPTLRPC != 1);
> +	BUILD_BUG_ON(CONFIG_T_RECOVER != 2);
> +	BUILD_BUG_ON(CONFIG_T_PARAMS != 3);
> +
> +	/* Checks for struct mgs_config_res */
> +	LASSERTF((int)sizeof(struct mgs_config_res) == 16, "found %lld\n",
> +		 (long long)(int)sizeof(struct mgs_config_res));
> +	LASSERTF((int)offsetof(struct mgs_config_res, mcr_offset) == 0, "found %lld\n",
> +		 (long long)(int)offsetof(struct mgs_config_res, mcr_offset));
> +	LASSERTF((int)sizeof(((struct mgs_config_res *)0)->mcr_offset) == 8, "found %lld\n",
> +		 (long long)(int)sizeof(((struct mgs_config_res *)0)->mcr_offset));
> +	LASSERTF((int)offsetof(struct mgs_config_res, mcr_size) == 8, "found %lld\n",
> +		 (long long)(int)offsetof(struct mgs_config_res, mcr_size));
> +	LASSERTF((int)sizeof(((struct mgs_config_res *)0)->mcr_size) == 8, "found %lld\n",
> +		 (long long)(int)sizeof(((struct mgs_config_res *)0)->mcr_size));
> +
>  	/* Checks for struct lustre_capa */
>  	LASSERTF((int)sizeof(struct lustre_capa) == 120, "found %lld\n",
>  		 (long long)(int)sizeof(struct lustre_capa));
> -- 
> 1.8.3.1
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180801/7dfd1f53/attachment.sig>

From neilb at suse.com  Tue Jul 31 22:55:10 2018
From: neilb at suse.com (NeilBrown)
Date: Wed, 01 Aug 2018 08:55:10 +1000
Subject: [lustre-devel] [PATCH 30/31] lustre: fid: race between
 client_fid_fini and seq_client_flush
In-Reply-To: <1533003983-29311-31-git-send-email-jsimmons@infradead.org>
References: <1533003983-29311-1-git-send-email-jsimmons@infradead.org>
 <1533003983-29311-31-git-send-email-jsimmons@infradead.org>
Message-ID: <87y3drqalt.fsf@notabene.neil.brown.name>

On Mon, Jul 30 2018, James Simmons wrote:

> From: Fan Yong <fan.yong at intel.com>
>
> When the client mount failed or umount, the client_fid_fini() will
> be called. At that time, the async connection failure will trigger
> seq_client_flush() which parameter may have been released by the
> client_fid_fini() by race.
>
> Introduce client_obd::cl_seq_rwsem to protect client_obd::cl_seq.

This looks odd..

I think the cl_seq_rwsem is being used like a refcount on cl_seq,
to prevent it from being freed while it is still in use.
If I'm correct, then I would much prefer that a refcount was used.

Is this more than just a disguised refcount?

NeilBrown


>
> Signed-off-by: Fan Yong <fan.yong at intel.com>
> WC-id: https://jira.whamcloud.com/browse/LU-9224
> Reviewed-on: https://review.whamcloud.com/26079
> Reviewed-by: John L. Hammond <jhammond at whamcloud.com>
> Reviewed-by: Andreas Dilger <adilger at whamcloud.com>
> Reviewed-by: Oleg Drokin <green at whamcloud.com>
> Signed-off-by: James Simmons <jsimmons at infradead.org>
> ---
>  drivers/staging/lustre/lustre/fid/fid_request.c | 21 +++++++++++++++------
>  drivers/staging/lustre/lustre/include/obd.h     |  1 +
>  drivers/staging/lustre/lustre/ldlm/ldlm_lib.c   |  2 ++
>  drivers/staging/lustre/lustre/mdc/mdc_request.c | 11 +++++++++--
>  4 files changed, 27 insertions(+), 8 deletions(-)
>
> diff --git a/drivers/staging/lustre/lustre/fid/fid_request.c b/drivers/staging/lustre/lustre/fid/fid_request.c
> index a34fd90..f91242c 100644
> --- a/drivers/staging/lustre/lustre/fid/fid_request.c
> +++ b/drivers/staging/lustre/lustre/fid/fid_request.c
> @@ -343,11 +343,14 @@ int client_fid_init(struct obd_device *obd,
>  {
>  	struct client_obd *cli = &obd->u.cli;
>  	char *prefix;
> -	int rc;
> +	int rc = 0;
>  
> +	down_write(&cli->cl_seq_rwsem);
>  	cli->cl_seq = kzalloc(sizeof(*cli->cl_seq), GFP_NOFS);
> -	if (!cli->cl_seq)
> -		return -ENOMEM;
> +	if (!cli->cl_seq) {
> +		rc = -ENOMEM;
> +		goto out_free_lock;
> +	}
>  
>  	prefix = kzalloc(MAX_OBD_NAME + 5, GFP_NOFS);
>  	if (!prefix) {
> @@ -361,10 +364,14 @@ int client_fid_init(struct obd_device *obd,
>  	seq_client_init(cli->cl_seq, exp, type, prefix);
>  	kfree(prefix);
>  
> -	return 0;
>  out_free_seq:
> -	kfree(cli->cl_seq);
> -	cli->cl_seq = NULL;
> +	if (rc) {
> +		kfree(cli->cl_seq);
> +		cli->cl_seq = NULL;
> +	}
> +out_free_lock:
> +	up_write(&cli->cl_seq_rwsem);
> +
>  	return rc;
>  }
>  EXPORT_SYMBOL(client_fid_init);
> @@ -373,11 +380,13 @@ int client_fid_fini(struct obd_device *obd)
>  {
>  	struct client_obd *cli = &obd->u.cli;
>  
> +	down_write(&cli->cl_seq_rwsem);
>  	if (cli->cl_seq) {
>  		seq_client_fini(cli->cl_seq);
>  		kfree(cli->cl_seq);
>  		cli->cl_seq = NULL;
>  	}
> +	up_write(&cli->cl_seq_rwsem);
>  
>  	return 0;
>  }
> diff --git a/drivers/staging/lustre/lustre/include/obd.h b/drivers/staging/lustre/lustre/include/obd.h
> index 333c703..3c0dbb6 100644
> --- a/drivers/staging/lustre/lustre/include/obd.h
> +++ b/drivers/staging/lustre/lustre/include/obd.h
> @@ -333,6 +333,7 @@ struct client_obd {
>  
>  	/* sequence manager */
>  	struct lu_client_seq    *cl_seq;
> +	struct rw_semaphore	 cl_seq_rwsem;
>  
>  	atomic_t	     cl_resends; /* resend count */
>  
> diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_lib.c b/drivers/staging/lustre/lustre/ldlm/ldlm_lib.c
> index c36d1e4..32eda4f 100644
> --- a/drivers/staging/lustre/lustre/ldlm/ldlm_lib.c
> +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_lib.c
> @@ -308,6 +308,8 @@ int client_obd_setup(struct obd_device *obddev, struct lustre_cfg *lcfg)
>  	}
>  
>  	init_rwsem(&cli->cl_sem);
> +	cli->cl_seq = NULL;
> +	init_rwsem(&cli->cl_seq_rwsem);
>  	cli->cl_conn_count = 0;
>  	memcpy(server_uuid.uuid, lustre_cfg_buf(lcfg, 2),
>  	       min_t(unsigned int, LUSTRE_CFG_BUFLEN(lcfg, 2),
> diff --git a/drivers/staging/lustre/lustre/mdc/mdc_request.c b/drivers/staging/lustre/lustre/mdc/mdc_request.c
> index c2f0a54..a759da2 100644
> --- a/drivers/staging/lustre/lustre/mdc/mdc_request.c
> +++ b/drivers/staging/lustre/lustre/mdc/mdc_request.c
> @@ -2517,8 +2517,10 @@ static int mdc_import_event(struct obd_device *obd, struct obd_import *imp,
>  		 * Flush current sequence to make client obtain new one
>  		 * from server in case of disconnect/reconnect.
>  		 */
> +		down_read(&cli->cl_seq_rwsem);
>  		if (cli->cl_seq)
>  			seq_client_flush(cli->cl_seq);
> +		up_read(&cli->cl_seq_rwsem);
>  
>  		rc = obd_notify_observer(obd, obd, OBD_NOTIFY_INACTIVE);
>  		break;
> @@ -2557,9 +2559,14 @@ int mdc_fid_alloc(const struct lu_env *env, struct obd_export *exp,
>  		  struct lu_fid *fid, struct md_op_data *op_data)
>  {
>  	struct client_obd *cli = &exp->exp_obd->u.cli;
> -	struct lu_client_seq *seq = cli->cl_seq;
> +	int rc = -EIO;
>  
> -	return seq_client_alloc_fid(env, seq, fid);
> +	down_read(&cli->cl_seq_rwsem);
> +	if (cli->cl_seq)
> +		rc = seq_client_alloc_fid(env, cli->cl_seq, fid);
> +	up_read(&cli->cl_seq_rwsem);
> +
> +	return rc;
>  }
>  
>  static struct obd_uuid *mdc_get_uuid(struct obd_export *exp)
> -- 
> 1.8.3.1
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180801/cce690fa/attachment.sig>

From paf at cray.com  Tue Jul 31 23:04:05 2018
From: paf at cray.com (Patrick Farrell)
Date: Tue, 31 Jul 2018 23:04:05 +0000
Subject: [lustre-devel] [PATCH 25/31] lustre: config: move config types
 into lustre_idl.h
In-Reply-To: <871sbjrpj3.fsf@notabene.neil.brown.name>
References: <1533003983-29311-1-git-send-email-jsimmons@infradead.org>
 <1533003983-29311-26-git-send-email-jsimmons@infradead.org>,
 <871sbjrpj3.fsf@notabene.neil.brown.name>
Message-ID: <BN3PR1101MB1202DBDAC4159FA431E0D676CB2E0@BN3PR1101MB1202.namprd11.prod.outlook.com>

Neil,

Do you have an objection to the concept, or just because this one's not used?
Having a MAX makes it easy to write things like < MYENUM_MAX as sanity checking code, and then if the enum is added to, it still works.  Seems useful to me.

- Patrick
________________________________
From: lustre-devel <lustre-devel-bounces at lists.lustre.org> on behalf of NeilBrown <neilb at suse.com>
Sent: Tuesday, July 31, 2018 5:47:28 PM
To: James Simmons; Andreas Dilger; Oleg Drokin
Cc: Lustre Development List
Subject: Re: [lustre-devel] [PATCH 25/31] lustre: config: move config types into lustre_idl.h

On Mon, Jul 30 2018, James Simmons wrote:

> From: Niu Yawei <yawei.niu at intel.com>
>
> Move config type values CONFIG_T_XXX into lustre_idl.h since they
> will be put on wire when reading config logs.
>
> Add missing wire checks for mgs_nidtbl_entry, mgs_config_body and
> mgs_config_res.
>
> Redefine CONFIG_SUB_XXX for the sub clds attached on config log.
>
> Signed-off-by: Niu Yawei <yawei.niu at intel.com>
> WC-id: https://jira.whamcloud.com/browse/LU-9216
> Reviewed-on: https://review.whamcloud.com/26022
> Reviewed-by: Fan Yong <fan.yong at intel.com>
> Reviewed-by: John L. Hammond <jhammond at whamcloud.com>
> Reviewed-by: Oleg Drokin <green at whamcloud.com>
> Signed-off-by: James Simmons <jsimmons at infradead.org>
> ---
>  .../lustre/include/uapi/linux/lustre/lustre_idl.h  | 10 ++-
>  drivers/staging/lustre/lustre/include/obd_class.h  | 12 +--
>  drivers/staging/lustre/lustre/mgc/mgc_request.c    |  6 +-
>  drivers/staging/lustre/lustre/ptlrpc/wiretest.c    | 85 ++++++++++++++++++++++
>  4 files changed, 103 insertions(+), 10 deletions(-)
>
> diff --git a/drivers/staging/lustre/include/uapi/linux/lustre/lustre_idl.h b/drivers/staging/lustre/include/uapi/linux/lustre/lustre_idl.h
> index c9b32ef..bd3b45a 100644
> --- a/drivers/staging/lustre/include/uapi/linux/lustre/lustre_idl.h
> +++ b/drivers/staging/lustre/include/uapi/linux/lustre/lustre_idl.h
> @@ -2111,11 +2111,19 @@ struct mgs_nidtbl_entry {
>        } u;
>  };
>
> +enum {
> +     CONFIG_T_CONFIG  = 0,
> +     CONFIG_T_SPTLRPC = 1,
> +     CONFIG_T_RECOVER = 2,
> +     CONFIG_T_PARAMS  = 3,
> +     CONFIG_T_MAX

Arrrgggh.  It's back.  I thought we had killed CONFIG_T_MAX (which isn't
a MAX).
It's never used, so it'll have to go.

NeilBrown


> +};
> +
>  struct mgs_config_body {
>        char            mcb_name[MTI_NAME_MAXLEN]; /* logname */
>        __u64           mcb_offset;    /* next index of config log to request */
>        __u16           mcb_type;      /* type of log: CONFIG_T_[CONFIG|RECOVER] */
> -     __u8            mcb_reserved;
> +     __u8            mcb_nm_cur_pass;
>        __u8            mcb_bits;      /* bits unit size of config log */
>        __u32           mcb_units;     /* # of units for bulk transfer */
>  };
> diff --git a/drivers/staging/lustre/lustre/include/obd_class.h b/drivers/staging/lustre/lustre/include/obd_class.h
> index 184da99..647cc22 100644
> --- a/drivers/staging/lustre/lustre/include/obd_class.h
> +++ b/drivers/staging/lustre/lustre/include/obd_class.h
> @@ -156,16 +156,16 @@ struct config_llog_instance {
>  int class_config_parse_llog(const struct lu_env *env, struct llog_ctxt *ctxt,
>                            char *name, struct config_llog_instance *cfg);
>
> -#define CONFIG_T_CONFIG              BIT(0)
> -#define CONFIG_T_SPTLRPC     BIT(1)
> -#define CONFIG_T_RECOVER     BIT(2)
> -#define CONFIG_T_PARAMS              BIT(3)
> +#define CONFIG_SUB_CONFIG    BIT(0)
> +#define CONFIG_SUB_SPTLRPC   BIT(1)
> +#define CONFIG_SUB_RECOVER   BIT(2)
> +#define CONFIG_SUB_PARAMS    BIT(3)
>
>  /* Sub clds should be attached to the config_llog_data when processing
>   * config log for client or server target.
>   */
> -#define CONFIG_SUB_CLIENT    (CONFIG_T_SPTLRPC | CONFIG_T_RECOVER | \
> -                              CONFIG_T_PARAMS)
> +#define CONFIG_SUB_CLIENT    (CONFIG_SUB_SPTLRPC | CONFIG_SUB_RECOVER | \
> +                              CONFIG_SUB_PARAMS)
>
>  #define PARAMS_FILENAME      "params"
>  #define LCTL_UPCALL  "lctl"
> diff --git a/drivers/staging/lustre/lustre/mgc/mgc_request.c b/drivers/staging/lustre/lustre/mgc/mgc_request.c
> index 06fcc7e..833e6a0 100644
> --- a/drivers/staging/lustre/lustre/mgc/mgc_request.c
> +++ b/drivers/staging/lustre/lustre/mgc/mgc_request.c
> @@ -315,7 +315,7 @@ struct config_llog_data *do_config_log_add(struct obd_device *obd,
>        memcpy(seclogname, logname, ptr - logname);
>        strcpy(seclogname + (ptr - logname), "-sptlrpc");
>
> -     if (cfg->cfg_sub_clds & CONFIG_T_SPTLRPC) {
> +     if (cfg->cfg_sub_clds & CONFIG_SUB_SPTLRPC) {
>                sptlrpc_cld = config_log_find_or_add(obd, seclogname, NULL,
>                                                     CONFIG_T_SPTLRPC, cfg);
>                if (IS_ERR(sptlrpc_cld)) {
> @@ -325,7 +325,7 @@ struct config_llog_data *do_config_log_add(struct obd_device *obd,
>                }
>        }
>
> -     if (cfg->cfg_sub_clds & CONFIG_T_PARAMS) {
> +     if (cfg->cfg_sub_clds & CONFIG_SUB_PARAMS) {
>                params_cld = config_log_find_or_add(obd, PARAMS_FILENAME, sb,
>                                                    CONFIG_T_PARAMS, cfg);
>                if (IS_ERR(params_cld)) {
> @@ -345,7 +345,7 @@ struct config_llog_data *do_config_log_add(struct obd_device *obd,
>
>        LASSERT(lsi->lsi_lmd);
>        if (!(lsi->lsi_lmd->lmd_flags & LMD_FLG_NOIR) &&
> -         cfg->cfg_sub_clds & CONFIG_T_RECOVER) {
> +         cfg->cfg_sub_clds & CONFIG_SUB_RECOVER) {
>                ptr = strrchr(seclogname, '-');
>                if (ptr) {
>                        *ptr = 0;
> diff --git a/drivers/staging/lustre/lustre/ptlrpc/wiretest.c b/drivers/staging/lustre/lustre/ptlrpc/wiretest.c
> index 2f081ed..09b1298 100644
> --- a/drivers/staging/lustre/lustre/ptlrpc/wiretest.c
> +++ b/drivers/staging/lustre/lustre/ptlrpc/wiretest.c
> @@ -3629,6 +3629,91 @@ void lustre_assert_wire_constants(void)
>        LASSERTF((int)sizeof(((struct mgs_target_info *)0)->mti_params) == 4096, "found %lld\n",
>                 (long long)(int)sizeof(((struct mgs_target_info *)0)->mti_params));
>
> +     /* Checks for struct mgs_nidtbl_entry */
> +     LASSERTF((int)sizeof(struct mgs_nidtbl_entry) == 24, "found %lld\n",
> +              (long long)(int)sizeof(struct mgs_nidtbl_entry));
> +     LASSERTF((int)offsetof(struct mgs_nidtbl_entry, mne_version) == 0, "found %lld\n",
> +              (long long)(int)offsetof(struct mgs_nidtbl_entry, mne_version));
> +     LASSERTF((int)sizeof(((struct mgs_nidtbl_entry *)0)->mne_version) == 8, "found %lld\n",
> +              (long long)(int)sizeof(((struct mgs_nidtbl_entry *)0)->mne_version));
> +     LASSERTF((int)offsetof(struct mgs_nidtbl_entry, mne_instance) == 8, "found %lld\n",
> +              (long long)(int)offsetof(struct mgs_nidtbl_entry, mne_instance));
> +     LASSERTF((int)sizeof(((struct mgs_nidtbl_entry *)0)->mne_instance) == 4, "found %lld\n",
> +              (long long)(int)sizeof(((struct mgs_nidtbl_entry *)0)->mne_instance));
> +     LASSERTF((int)offsetof(struct mgs_nidtbl_entry, mne_index) == 12, "found %lld\n",
> +              (long long)(int)offsetof(struct mgs_nidtbl_entry, mne_index));
> +     LASSERTF((int)sizeof(((struct mgs_nidtbl_entry *)0)->mne_index) == 4, "found %lld\n",
> +              (long long)(int)sizeof(((struct mgs_nidtbl_entry *)0)->mne_index));
> +     LASSERTF((int)offsetof(struct mgs_nidtbl_entry, mne_length) == 16, "found %lld\n",
> +              (long long)(int)offsetof(struct mgs_nidtbl_entry, mne_length));
> +     LASSERTF((int)sizeof(((struct mgs_nidtbl_entry *)0)->mne_length) == 4, "found %lld\n",
> +              (long long)(int)sizeof(((struct mgs_nidtbl_entry *)0)->mne_length));
> +     LASSERTF((int)offsetof(struct mgs_nidtbl_entry, mne_type) == 20, "found %lld\n",
> +              (long long)(int)offsetof(struct mgs_nidtbl_entry, mne_type));
> +     LASSERTF((int)sizeof(((struct mgs_nidtbl_entry *)0)->mne_type) == 1, "found %lld\n",
> +              (long long)(int)sizeof(((struct mgs_nidtbl_entry *)0)->mne_type));
> +     LASSERTF((int)offsetof(struct mgs_nidtbl_entry, mne_nid_type) == 21, "found %lld\n",
> +              (long long)(int)offsetof(struct mgs_nidtbl_entry, mne_nid_type));
> +     LASSERTF((int)sizeof(((struct mgs_nidtbl_entry *)0)->mne_nid_type) == 1, "found %lld\n",
> +              (long long)(int)sizeof(((struct mgs_nidtbl_entry *)0)->mne_nid_type));
> +     LASSERTF((int)offsetof(struct mgs_nidtbl_entry, mne_nid_size) == 22, "found %lld\n",
> +              (long long)(int)offsetof(struct mgs_nidtbl_entry, mne_nid_size));
> +     LASSERTF((int)sizeof(((struct mgs_nidtbl_entry *)0)->mne_nid_size) == 1, "found %lld\n",
> +              (long long)(int)sizeof(((struct mgs_nidtbl_entry *)0)->mne_nid_size));
> +     LASSERTF((int)offsetof(struct mgs_nidtbl_entry, mne_nid_count) == 23, "found %lld\n",
> +              (long long)(int)offsetof(struct mgs_nidtbl_entry, mne_nid_count));
> +     LASSERTF((int)sizeof(((struct mgs_nidtbl_entry *)0)->mne_nid_count) == 1, "found %lld\n",
> +              (long long)(int)sizeof(((struct mgs_nidtbl_entry *)0)->mne_nid_count));
> +     LASSERTF((int)offsetof(struct mgs_nidtbl_entry, u.nids[0]) == 24, "found %lld\n",
> +              (long long)(int)offsetof(struct mgs_nidtbl_entry, u.nids[0]));
> +     LASSERTF((int)sizeof(((struct mgs_nidtbl_entry *)0)->u.nids[0]) == 8, "found %lld\n",
> +              (long long)(int)sizeof(((struct mgs_nidtbl_entry *)0)->u.nids[0]));
> +
> +     /* Checks for struct mgs_config_body */
> +     LASSERTF((int)sizeof(struct mgs_config_body) == 80, "found %lld\n",
> +              (long long)(int)sizeof(struct mgs_config_body));
> +     LASSERTF((int)offsetof(struct mgs_config_body, mcb_name) == 0, "found %lld\n",
> +              (long long)(int)offsetof(struct mgs_config_body, mcb_name));
> +     LASSERTF((int)sizeof(((struct mgs_config_body *)0)->mcb_name) == 64, "found %lld\n",
> +              (long long)(int)sizeof(((struct mgs_config_body *)0)->mcb_name));
> +     LASSERTF((int)offsetof(struct mgs_config_body, mcb_offset) == 64, "found %lld\n",
> +              (long long)(int)offsetof(struct mgs_config_body, mcb_offset));
> +     LASSERTF((int)sizeof(((struct mgs_config_body *)0)->mcb_offset) == 8, "found %lld\n",
> +              (long long)(int)sizeof(((struct mgs_config_body *)0)->mcb_offset));
> +     LASSERTF((int)offsetof(struct mgs_config_body, mcb_type) == 72, "found %lld\n",
> +              (long long)(int)offsetof(struct mgs_config_body, mcb_type));
> +     LASSERTF((int)sizeof(((struct mgs_config_body *)0)->mcb_type) == 2, "found %lld\n",
> +              (long long)(int)sizeof(((struct mgs_config_body *)0)->mcb_type));
> +     LASSERTF((int)offsetof(struct mgs_config_body, mcb_nm_cur_pass) == 74, "found %lld\n",
> +              (long long)(int)offsetof(struct mgs_config_body, mcb_nm_cur_pass));
> +     LASSERTF((int)sizeof(((struct mgs_config_body *)0)->mcb_nm_cur_pass) == 1, "found %lld\n",
> +              (long long)(int)sizeof(((struct mgs_config_body *)0)->mcb_nm_cur_pass));
> +     LASSERTF((int)offsetof(struct mgs_config_body, mcb_bits) == 75, "found %lld\n",
> +              (long long)(int)offsetof(struct mgs_config_body, mcb_bits));
> +     LASSERTF((int)sizeof(((struct mgs_config_body *)0)->mcb_bits) == 1, "found %lld\n",
> +              (long long)(int)sizeof(((struct mgs_config_body *)0)->mcb_bits));
> +     LASSERTF((int)offsetof(struct mgs_config_body, mcb_units) == 76, "found %lld\n",
> +              (long long)(int)offsetof(struct mgs_config_body, mcb_units));
> +     LASSERTF((int)sizeof(((struct mgs_config_body *)0)->mcb_units) == 4, "found %lld\n",
> +              (long long)(int)sizeof(((struct mgs_config_body *)0)->mcb_units));
> +
> +     BUILD_BUG_ON(CONFIG_T_CONFIG != 0);
> +     BUILD_BUG_ON(CONFIG_T_SPTLRPC != 1);
> +     BUILD_BUG_ON(CONFIG_T_RECOVER != 2);
> +     BUILD_BUG_ON(CONFIG_T_PARAMS != 3);
> +
> +     /* Checks for struct mgs_config_res */
> +     LASSERTF((int)sizeof(struct mgs_config_res) == 16, "found %lld\n",
> +              (long long)(int)sizeof(struct mgs_config_res));
> +     LASSERTF((int)offsetof(struct mgs_config_res, mcr_offset) == 0, "found %lld\n",
> +              (long long)(int)offsetof(struct mgs_config_res, mcr_offset));
> +     LASSERTF((int)sizeof(((struct mgs_config_res *)0)->mcr_offset) == 8, "found %lld\n",
> +              (long long)(int)sizeof(((struct mgs_config_res *)0)->mcr_offset));
> +     LASSERTF((int)offsetof(struct mgs_config_res, mcr_size) == 8, "found %lld\n",
> +              (long long)(int)offsetof(struct mgs_config_res, mcr_size));
> +     LASSERTF((int)sizeof(((struct mgs_config_res *)0)->mcr_size) == 8, "found %lld\n",
> +              (long long)(int)sizeof(((struct mgs_config_res *)0)->mcr_size));
> +
>        /* Checks for struct lustre_capa */
>        LASSERTF((int)sizeof(struct lustre_capa) == 120, "found %lld\n",
>                 (long long)(int)sizeof(struct lustre_capa));
> --
> 1.8.3.1
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20180731/cda9aafb/attachment.html>

From ashehata at whamcloud.com  Tue Jul 31 23:54:30 2018
From: ashehata at whamcloud.com (Amir Shehata)
Date: Tue, 31 Jul 2018 23:54:30 +0000
Subject: [lustre-devel] [PATCH 11/31] lustre: lnet: Fix route hops print
In-Reply-To: <877elbrpxh.fsf@notabene.neil.brown.name>
References: <1533003983-29311-1-git-send-email-jsimmons@infradead.org>
 <1533003983-29311-12-git-send-email-jsimmons@infradead.org>,
 <877elbrpxh.fsf@notabene.neil.brown.name>
Message-ID: <3C0BF267DA2B2343B6A8D6013E6C43381AAE2A@LAX-EX-MB3.datadirect.datadirectnet.com>

The way hop and priority work in the code is they serve to select the preferred route. If you have multiple gateways leading to the same destination, you select the one with the highest priority (0 being the highest), followed by selecting the one with the least number of hops. If you don't specify hops, then it's actually treated as the least favoured if there are other routes with hops specified. If hops and priority are equivalent between routes, then you select the one with the most credits available, if that's equivalent you select in round robin.
 
In that sense hops and priority really serve the same purpose, select the preferred route. If it was up to me I would keep only one of them, but for historical reasons, both are kept.

Therefore, I'm not sure if "unlimited" actually relays the correct interpretation of that value. Note there could be user scripts out there that are already parsing the output. So by changing the -1 you could break the scripts. Also changing that will create an inconsistency between the server and client.

thanks
amir
________________________________________
From: NeilBrown [neilb at suse.com]
Sent: Tuesday, July 31, 2018 3:38 PM
To: James Simmons; Andreas Dilger; Oleg Drokin
Cc: Lustre Development List; Amir Shehata; James Simmons
Subject: Re: [PATCH 11/31] lustre: lnet: Fix route hops print

On Mon, Jul 30 2018, James Simmons wrote:

> From: Amir Shehata <ashehata at whamcloud.com>
>
> The default number of hops for  a route is -1. This is
> currently being printed as %u. Change that to %d to
> make it print out properly.

-1 hops???  I wish I could hop -1 times - it would be a good party
trick!!

What does -1 mean?  Unlimited (just a guess).  If so, could we print
"unlimited"??

I'm fine with having magic numbers in the code, but I don't like them to
leak out.

NeilBrown

>
> Signed-off-by: Amir Shehata <ashehata at whamcloud.com>
> WC-id: https://jira.whamcloud.com/browse/LU-9078
> Reviewed-on: https://review.whamcloud.com/25250
> Reviewed-by: Olaf Weber <olaf at sgi.com>
> Reviewed-by: Doug Oucharek <dougso at me.com>
> Reviewed-by: James Simmons <uja.ornl at yahoo.com>
> Reviewed-by: Oleg Drokin <green at whamcloud.com>
> Signed-off-by: James Simmons <jsimmons at infradead.org>
> ---
>  drivers/staging/lustre/lnet/lnet/router_proc.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/drivers/staging/lustre/lnet/lnet/router_proc.c b/drivers/staging/lustre/lnet/lnet/router_proc.c
> index 8856798..aa98ce5 100644
> --- a/drivers/staging/lustre/lnet/lnet/router_proc.c
> +++ b/drivers/staging/lustre/lnet/lnet/router_proc.c
> @@ -218,7 +218,7 @@ static int proc_lnet_routes(struct ctl_table *table, int write,
>                       int alive = lnet_is_route_alive(route);
>
>                       s += snprintf(s, tmpstr + tmpsiz - s,
> -                                   "%-8s %4u %8u %7s %s\n",
> +                                   "%-8s %4d %8u %7s %s\n",
>                                     libcfs_net2str(net), hops,
>                                     priority,
>                                     alive ? "up" : "down",
> --
> 1.8.3.1