From andreas.dilger at intel.com  Fri Sep  1 20:13:18 2017
From: andreas.dilger at intel.com (Dilger, Andreas)
Date: Fri, 1 Sep 2017 20:13:18 +0000
Subject: [lustre-devel] [PATCH] lustre: ptlrpc: kfree used instead of
	kvfree
In-Reply-To: <20170831225516.41922-1-namit@vmware.com>
References: <20170831225516.41922-1-namit@vmware.com>
Message-ID: <23708A03-C3C1-4270-AB59-8E1A17FB6EBA@intel.com>

On Aug 31, 2017, at 16:55, Nadav Amit <namit at vmware.com> wrote:
> 
> rq_reqbuf is allocated using kvmalloc() but released in one occasion
> using kfree() instead of kvfree().
> 
> The issue was found using grep based on a similar bug.
> 
> Fixes: d7e09d0397e8 ("add Lustre file system client support")

It would appear that this bug was introduced by ee0ec1946ec2
"lustre: ptlrpc: Replace uses of OBD_{ALLOC,FREE}_LARGE", but
it looks like the problem predates this patch, since it was
originally using OBD_FREE() instead of OBD_FREE_LARGE() here
so the Coccinelle script wouldn't have caught the issue.

Reviewed-by: Andreas Dilger <andreas.dilger at intel.com>

> 
> Cc: Peng Tao <bergwolf at gmail.com>
> Cc: Oleg Drokin <oleg.drokin at intel.com>
> Cc: Andreas Dilger <andreas.dilger at intel.com>
> Cc: James Simmons <jsimmons at infradead.org>
> 
> Signed-off-by: Nadav Amit <namit at vmware.com>
> ---
> drivers/staging/lustre/lustre/ptlrpc/sec.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/drivers/staging/lustre/lustre/ptlrpc/sec.c b/drivers/staging/lustre/lustre/ptlrpc/sec.c
> index 366f2ce20f5e..2f1c9e15f47c 100644
> --- a/drivers/staging/lustre/lustre/ptlrpc/sec.c
> +++ b/drivers/staging/lustre/lustre/ptlrpc/sec.c
> @@ -847,7 +847,7 @@ void sptlrpc_request_out_callback(struct ptlrpc_request *req)
> 	if (req->rq_pool || !req->rq_reqbuf)
> 		return;
> 
> -	kfree(req->rq_reqbuf);
> +	kvfree(req->rq_reqbuf);
> 	req->rq_reqbuf = NULL;
> 	req->rq_reqbuf_len = 0;
> }
> -- 
> 2.11.0
> 

Cheers, Andreas
--
Andreas Dilger
Lustre Principal Architect
Intel Corporation







From gregkh at linuxfoundation.org  Sat Sep  2 05:40:21 2017
From: gregkh at linuxfoundation.org (Greg Kroah-Hartman)
Date: Sat, 2 Sep 2017 07:40:21 +0200
Subject: [lustre-devel] [PATCH] lustre: ptlrpc: kfree used instead of
	kvfree
In-Reply-To: <23708A03-C3C1-4270-AB59-8E1A17FB6EBA@intel.com>
References: <20170831225516.41922-1-namit@vmware.com>
	<23708A03-C3C1-4270-AB59-8E1A17FB6EBA@intel.com>
Message-ID: <20170902054021.GA22372@kroah.com>

On Fri, Sep 01, 2017 at 08:13:18PM +0000, Dilger, Andreas wrote:
> On Aug 31, 2017, at 16:55, Nadav Amit <namit at vmware.com> wrote:
> > 
> > rq_reqbuf is allocated using kvmalloc() but released in one occasion
> > using kfree() instead of kvfree().
> > 
> > The issue was found using grep based on a similar bug.
> > 
> > Fixes: d7e09d0397e8 ("add Lustre file system client support")
> 
> It would appear that this bug was introduced by ee0ec1946ec2
> "lustre: ptlrpc: Replace uses of OBD_{ALLOC,FREE}_LARGE", but
> it looks like the problem predates this patch, since it was
> originally using OBD_FREE() instead of OBD_FREE_LARGE() here
> so the Coccinelle script wouldn't have caught the issue.
> 
> Reviewed-by: Andreas Dilger <andreas.dilger at intel.com>
> 
> > 
> > Cc: Peng Tao <bergwolf at gmail.com>
> > Cc: Oleg Drokin <oleg.drokin at intel.com>
> > Cc: Andreas Dilger <andreas.dilger at intel.com>
> > Cc: James Simmons <jsimmons at infradead.org>
> > 
> > Signed-off-by: Nadav Amit <namit at vmware.com>
> > ---
> > drivers/staging/lustre/lustre/ptlrpc/sec.c | 2 +-
> > 1 file changed, 1 insertion(+), 1 deletion(-)

Can a lustre maintainer resend this in a format that I can apply it in?

thanks,

greg k-h

From quentin.bouget at cea.fr  Mon Sep  4 07:29:03 2017
From: quentin.bouget at cea.fr (quentin.bouget at cea.fr)
Date: Mon, 4 Sep 2017 09:29:03 +0200
Subject: [lustre-devel]  [gerrit] merge accounts
Message-ID: <8059fd2d-8e11-4cfe-70b6-16699757664f@cea.fr>

Hi all,

I mistakenly logged on lustre's gerrit from two different OpenID 
providers (actually, I used the same openID but from two different 
domains: login.ubuntu.com and login.launchpad.net/+openid) and now I 
would like to merge them.

I can see there is a feature for this in "settings > identities > Link 
Another Identity", but I cannot get it to work: it fails with "Not 
Found", and the url suggests I contact someone 
(https://review.whamcloud.com/#SignInFailure,LINK_IDENTIY,Contact+site+administrator).

Regards,
Quentin Bouget

From peter.a.jones at intel.com  Mon Sep  4 14:08:22 2017
From: peter.a.jones at intel.com (Jones, Peter A)
Date: Mon, 4 Sep 2017 14:08:22 +0000
Subject: [lustre-devel] [gerrit] merge accounts
In-Reply-To: <8059fd2d-8e11-4cfe-70b6-16699757664f@cea.fr>
References: <8059fd2d-8e11-4cfe-70b6-16699757664f@cea.fr>
Message-ID: <2E24D6B6-657B-4B17-AC26-034A6900E751@intel.com>

Send me the details and I can get this sorted out for you




On 9/4/17, 12:29 AM, "lustre-devel on behalf of quentin.bouget at cea.fr" <lustre-devel-bounces at lists.lustre.org on behalf of quentin.bouget at cea.fr> wrote:

>Hi all,
>
>I mistakenly logged on lustre's gerrit from two different OpenID 
>providers (actually, I used the same openID but from two different 
>domains: login.ubuntu.com and login.launchpad.net/+openid) and now I 
>would like to merge them.
>
>I can see there is a feature for this in "settings > identities > Link 
>Another Identity", but I cannot get it to work: it fails with "Not 
>Found", and the url suggests I contact someone 
>(https://review.whamcloud.com/#SignInFailure,LINK_IDENTIY,Contact+site+administrator).
>
>Regards,
>Quentin Bouget
>
>_______________________________________________
>lustre-devel mailing list
>lustre-devel at lists.lustre.org
>http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org

From andreas.dilger at intel.com  Tue Sep  5 20:25:25 2017
From: andreas.dilger at intel.com (Dilger, Andreas)
Date: Tue, 5 Sep 2017 20:25:25 +0000
Subject: [lustre-devel] [PATCH] lustre: ptlrpc: kfree used instead of kvfree
Message-ID: <4F34B6C1-3A14-4F29-B241-A80F5A66E90B@intel.com>

From: Nadav Amit <namit at vmware.com>

rq_reqbuf is allocated using kvmalloc() but released in one occasion
using kfree() instead of kvfree().

The issue was found using grep based on a similar bug.

Fixes: d7e09d0397e8 ("add Lustre file system client support")
Fixes: ee0ec1946ec2 ("lustre: ptlrpc: Replace uses of OBD_{ALLOC,FREE}_LARGE")

Cc: Peng Tao <bergwolf at gmail.com>
Cc: Oleg Drokin <oleg.drokin at intel.com>
Cc: James Simmons <jsimmons at infradead.org>

Signed-off-by: Nadav Amit <namit at vmware.com>
Signed-off-by: Andreas Dilger <andreas.dilger at intel.com>
---
drivers/staging/lustre/lustre/ptlrpc/sec.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/staging/lustre/lustre/ptlrpc/sec.c b/drivers/staging/lustre/lustre/ptlrpc/sec.c
index 366f2ce20f5e..2f1c9e15f47c 100644
--- a/drivers/staging/lustre/lustre/ptlrpc/sec.c
+++ b/drivers/staging/lustre/lustre/ptlrpc/sec.c
@@ -847,7 +847,7 @@ void sptlrpc_request_out_callback(struct ptlrpc_request *req)
 	if (req->rq_pool || !req->rq_reqbuf)
 		return;
 
-	kfree(req->rq_reqbuf);
+	kvfree(req->rq_reqbuf);
 	req->rq_reqbuf = NULL;
 	req->rq_reqbuf_len = 0;
 }
-- 
2.11.0





From gregkh at linuxfoundation.org  Wed Sep  6 17:28:52 2017
From: gregkh at linuxfoundation.org (Greg Kroah-Hartman)
Date: Wed, 6 Sep 2017 19:28:52 +0200
Subject: [lustre-devel] [PATCH 0/2] LU-6808: ported to upstream staging
	driver
In-Reply-To: <9525089.uibW5qMffG@whitehead.beowulf.uwo.ca>
References: <9525089.uibW5qMffG@whitehead.beowulf.uwo.ca>
Message-ID: <20170906172852.GA29067@kroah.com>

On Wed, Sep 06, 2017 at 12:04:04PM -0400, Tyson Whitehead wrote:
> Please find enclosed in the next two emails the two patches for LU-6808 ported to the in-tree staging lustre client driver in the 4.12 release.

What does that mean?  What can I do with these?  4.12 is about to be
end-of-life in a few days...

confused,

greg k-h

From cakturk at gmail.com  Thu Sep  7 10:57:42 2017
From: cakturk at gmail.com (Cihangir Akturk)
Date: Thu,  7 Sep 2017 13:57:42 +0300
Subject: [lustre-devel] [PATCH] staging: lustre: avoid going through
	unlock/lock overhead
Message-ID: <1504781862-15577-1-git-send-email-cakturk@gmail.com>

Unlocking a spin lock and then immediately locking without doing
anything useful in between buys us nothing, except wasting CPU cycles.

Also code size gets smaller.

Before:

 text  data   bss    dec    hex filename
70415  2356  4108  76879  12c4f drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.o

After:

 text  data   bss    dec    hex filename
70095  2356  4108  76559  12b0f drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.o

Signed-off-by: Cihangir Akturk <cakturk at gmail.com>
---
 drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c | 18 ++++++++----------
 1 file changed, 8 insertions(+), 10 deletions(-)

diff --git a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c
index 64763aa..5d9cd33 100644
--- a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c
+++ b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c
@@ -1624,8 +1624,9 @@ int kiblnd_fmr_pool_map(struct kib_fmr_poolset *fps, struct kib_tx *tx,
 	__u64 version;
 	int rc;
 
- again:
+again:
 	spin_lock(&fps->fps_lock);
+again_locked:
 	version = fps->fps_version;
 	list_for_each_entry(fpo, &fps->fps_pool_list, fpo_list) {
 		fpo->fpo_deadline = cfs_time_shift(IBLND_POOL_DEADLINE);
@@ -1722,10 +1723,8 @@ int kiblnd_fmr_pool_map(struct kib_fmr_poolset *fps, struct kib_tx *tx,
 		}
 
 		/* EAGAIN and ... */
-		if (version != fps->fps_version) {
-			spin_unlock(&fps->fps_lock);
-			goto again;
-		}
+		if (version != fps->fps_version)
+			goto again_locked;
 	}
 
 	if (fps->fps_increasing) {
@@ -1754,9 +1753,8 @@ int kiblnd_fmr_pool_map(struct kib_fmr_poolset *fps, struct kib_tx *tx,
 	} else {
 		fps->fps_next_retry = cfs_time_shift(IBLND_POOL_RETRY);
 	}
-	spin_unlock(&fps->fps_lock);
 
-	goto again;
+	goto again_locked;
 }
 
 static void kiblnd_fini_pool(struct kib_pool *pool)
@@ -1901,8 +1899,9 @@ struct list_head *kiblnd_pool_alloc_node(struct kib_poolset *ps)
 	unsigned int trips = 0;
 	int rc;
 
- again:
+again:
 	spin_lock(&ps->ps_lock);
+again_locked:
 	list_for_each_entry(pool, &ps->ps_pool_list, po_list) {
 		if (list_empty(&pool->po_free_list))
 			continue;
@@ -1960,9 +1959,8 @@ struct list_head *kiblnd_pool_alloc_node(struct kib_poolset *ps)
 		CERROR("Can't allocate new %s pool because out of memory\n",
 		       ps->ps_name);
 	}
-	spin_unlock(&ps->ps_lock);
 
-	goto again;
+	goto again_locked;
 }
 
 static void kiblnd_destroy_tx_pool(struct kib_pool *pool)
-- 
2.7.4

From gregkh at linuxfoundation.org  Thu Sep  7 12:33:49 2017
From: gregkh at linuxfoundation.org (Greg KH)
Date: Thu, 7 Sep 2017 14:33:49 +0200
Subject: [lustre-devel] [PATCH] staging: lustre: avoid going through
	unlock/lock overhead
In-Reply-To: <1504781862-15577-1-git-send-email-cakturk@gmail.com>
References: <1504781862-15577-1-git-send-email-cakturk@gmail.com>
Message-ID: <20170907123349.GB20922@kroah.com>

On Thu, Sep 07, 2017 at 01:57:42PM +0300, Cihangir Akturk wrote:
> Unlocking a spin lock and then immediately locking without doing
> anything useful in between buys us nothing, except wasting CPU cycles.

Not always, it can be a "gate" for other users of the lock.

Are you sure that is not what is going on here?  Did you test this out
on a lustre system?  The locks here are anything but trivial...

> 
> Also code size gets smaller.
> 
> Before:
> 
>  text  data   bss    dec    hex filename
> 70415  2356  4108  76879  12c4f drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.o
> 
> After:
> 
>  text  data   bss    dec    hex filename
> 70095  2356  4108  76559  12b0f drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.o
> 
> Signed-off-by: Cihangir Akturk <cakturk at gmail.com>
> ---
>  drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c | 18 ++++++++----------
>  1 file changed, 8 insertions(+), 10 deletions(-)
> 
> diff --git a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c
> index 64763aa..5d9cd33 100644
> --- a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c
> +++ b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c
> @@ -1624,8 +1624,9 @@ int kiblnd_fmr_pool_map(struct kib_fmr_poolset *fps, struct kib_tx *tx,
>  	__u64 version;
>  	int rc;
>  
> - again:
> +again:
>  	spin_lock(&fps->fps_lock);
> +again_locked:
>  	version = fps->fps_version;
>  	list_for_each_entry(fpo, &fps->fps_pool_list, fpo_list) {
>  		fpo->fpo_deadline = cfs_time_shift(IBLND_POOL_DEADLINE);
> @@ -1722,10 +1723,8 @@ int kiblnd_fmr_pool_map(struct kib_fmr_poolset *fps, struct kib_tx *tx,
>  		}
>  
>  		/* EAGAIN and ... */
> -		if (version != fps->fps_version) {
> -			spin_unlock(&fps->fps_lock);
> -			goto again;
> -		}
> +		if (version != fps->fps_version)
> +			goto again_locked;
>  	}
>  
>  	if (fps->fps_increasing) {
> @@ -1754,9 +1753,8 @@ int kiblnd_fmr_pool_map(struct kib_fmr_poolset *fps, struct kib_tx *tx,
>  	} else {
>  		fps->fps_next_retry = cfs_time_shift(IBLND_POOL_RETRY);
>  	}
> -	spin_unlock(&fps->fps_lock);
>  
> -	goto again;
> +	goto again_locked;

Really, gotos backwards?  Ick, that's horrid as well, so maybe this is
better?  I hate this whole codebase...

I'll let the Lustre maintainers decide about this one...

greg k-h

From cakturk at gmail.com  Thu Sep  7 13:29:37 2017
From: cakturk at gmail.com (Cihangir Akturk)
Date: Thu, 7 Sep 2017 16:29:37 +0300
Subject: [lustre-devel] [PATCH] staging: lustre: avoid going through
	unlock/lock overhead
In-Reply-To: <20170907123349.GB20922@kroah.com>
References: <1504781862-15577-1-git-send-email-cakturk@gmail.com>
	<20170907123349.GB20922@kroah.com>
Message-ID: <20170907132937.GA4147@yoga.localdomain>

On Thu, Sep 07, 2017 at 02:33:49PM +0200, Greg KH wrote:
> On Thu, Sep 07, 2017 at 01:57:42PM +0300, Cihangir Akturk wrote:
> > Unlocking a spin lock and then immediately locking without doing
> > anything useful in between buys us nothing, except wasting CPU cycles.
> 
> Not always, it can be a "gate" for other users of the lock.

OK, I get it.


> Are you sure that is not what is going on here?

No, I'm not sure. But yes, that's possible it might be used to let in
other users of the lock.

> Did you test this out on a lustre system?  The locks here are
> anything but trivial...

Unfortunately I haven't tested this change on a lustre system. Just
compile-tested.

> > 
> > Also code size gets smaller.
> > 
> > Before:
> > 
> >  text  data   bss    dec    hex filename
> > 70415  2356  4108  76879  12c4f drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.o
> > 
> > After:
> > 
> >  text  data   bss    dec    hex filename
> > 70095  2356  4108  76559  12b0f drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.o
> > 
> > Signed-off-by: Cihangir Akturk <cakturk at gmail.com>
> > ---
> >  drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c | 18 ++++++++----------
> >  1 file changed, 8 insertions(+), 10 deletions(-)
> > 
> > diff --git a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c
> > index 64763aa..5d9cd33 100644
> > --- a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c
> > +++ b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c
> > @@ -1624,8 +1624,9 @@ int kiblnd_fmr_pool_map(struct kib_fmr_poolset *fps, struct kib_tx *tx,
> >  	__u64 version;
> >  	int rc;
> >  
> > - again:
> > +again:
> >  	spin_lock(&fps->fps_lock);
> > +again_locked:
> >  	version = fps->fps_version;
> >  	list_for_each_entry(fpo, &fps->fps_pool_list, fpo_list) {
> >  		fpo->fpo_deadline = cfs_time_shift(IBLND_POOL_DEADLINE);
> > @@ -1722,10 +1723,8 @@ int kiblnd_fmr_pool_map(struct kib_fmr_poolset *fps, struct kib_tx *tx,
> >  		}
> >  
> >  		/* EAGAIN and ... */
> > -		if (version != fps->fps_version) {
> > -			spin_unlock(&fps->fps_lock);
> > -			goto again;
> > -		}
> > +		if (version != fps->fps_version)
> > +			goto again_locked;
> >  	}
> >  
> >  	if (fps->fps_increasing) {
> > @@ -1754,9 +1753,8 @@ int kiblnd_fmr_pool_map(struct kib_fmr_poolset *fps, struct kib_tx *tx,
> >  	} else {
> >  		fps->fps_next_retry = cfs_time_shift(IBLND_POOL_RETRY);
> >  	}
> > -	spin_unlock(&fps->fps_lock);
> >  
> > -	goto again;
> > +	goto again_locked;
> 
> Really, gotos backwards?  Ick, that's horrid as well, so maybe this is
> better?  I hate this whole codebase...
> 
> I'll let the Lustre maintainers decide about this one...
> 
> greg k-h

From twhitehead at gmail.com  Wed Sep  6 16:04:04 2017
From: twhitehead at gmail.com (Tyson Whitehead)
Date: Wed, 06 Sep 2017 12:04:04 -0400
Subject: [lustre-devel] [PATCH 0/2] LU-6808: ported to upstream staging
	driver
Message-ID: <9525089.uibW5qMffG@whitehead.beowulf.uwo.ca>

Please find enclosed in the next two emails the two patches for LU-6808 ported to the in-tree staging lustre client driver in the 4.12 release.

https://jira.hpdd.intel.com/browse/LU-6808

Fan Yong (1):
  LU-6808 ptlrpc: properly set "rq_xid" for 4MB IO

Niu Yawei (1):
  LU-6808 ptlrpc: no need to reassign mbits for replay

 drivers/staging/lustre/lustre/ptlrpc/client.c | 40 +++++++++++++++++++--------
 1 file changed, 29 insertions(+), 11 deletions(-)

-- 
2.14.0


From twhitehead at gmail.com  Wed Sep  6 16:05:01 2017
From: twhitehead at gmail.com (Tyson Whitehead)
Date: Wed, 06 Sep 2017 12:05:01 -0400
Subject: [lustre-devel] [PATCH 2/2] LU-6808 ptlrpc: no need to reassign
	mbits for replay
Message-ID: <2356378.V5F8jgOxsl@whitehead.beowulf.uwo.ca>

From: Niu Yawei <yawei.niu at intel.com>

It's not necessary reassgin & re-adjust rq_mbits for replay
request in ptlrpc_set_bulk_mbits(), they all must have already
been correctly assigned before.

Such unecessary reassign could make the first matchbit not
PTLRPC_BULK_OPS_MASK aligned, that'll trigger LASSERT in
ptlrpc_register_bulk():

- ptlrpc_set_bulk_mbits() is called when first time sending
  request, rq_mbits is set as xid, which is BULK_OPS aligned;

- ptlrpc_set_bulk_mbits() continue to adjust the mbits for
  multi-bulk RPC, rq_mbits is not aligned anymore, then rq_xid
  is changed accordingly if client is connecting to an old
  server, so rq_xid became unaligned too;

- The request is replayed, ptlrpc_set_bulk_mbits() reassign
  the rq_mbits as rq_xid, which isn't aligned already, but
  ptlrpc_register_bulk() still assumes this value as the
  first matchbits and LASSERT it's BULK_OPS aligned.

Signed-off-by: Niu Yawei <yawei.niu at intel.com>
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-6808
Reviewed-on: http://review.whamcloud.com/23048
Tested-by: Jenkins
Reviewed-by: Fan Yong <fan.yong at intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev at intel.com>
Tested-by: Maloo <hpdd-maloo at intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin at intel.com>
Signed-off-by: Tyson Whitehead <twhitehead at gmail.com>
---
 drivers/staging/lustre/lustre/ptlrpc/client.c | 34 +++++++++++++++++++--------
 1 file changed, 24 insertions(+), 10 deletions(-)

diff --git a/drivers/staging/lustre/lustre/ptlrpc/client.c b/drivers/staging/lustre/lustre/ptlrpc/client.c
index 0f9db5526ba5..6ddbd1289a2f 100644
--- a/drivers/staging/lustre/lustre/ptlrpc/client.c
+++ b/drivers/staging/lustre/lustre/ptlrpc/client.c
@@ -3116,26 +3116,37 @@ void ptlrpc_set_bulk_mbits(struct ptlrpc_request *req)
 
 	LASSERT(bd);
 
-	if (!req->rq_resend) {
-		/* this request has a new xid, just use it as bulk matchbits */
-		req->rq_mbits = req->rq_xid;
-
-	} else { /* needs to generate a new matchbits for resend */
-		u64 old_mbits = req->rq_mbits;
-
+	/* Generate new matchbits for all resend requests, including
+	 * resend replay. */
+	if (req->rq_resend) {
+		__u64 old_mbits = req->rq_mbits;
+
+		/* First time resend on -EINPROGRESS will generate new xid,
+		 * so we can actually use the rq_xid as rq_mbits in such case,
+		 * however, it's bit hard to distinguish such resend with a
+		 * 'resend for the -EINPROGRESS resend'. To make it simple,
+		 * we opt to generate mbits for all resend cases. */
 		if (OCD_HAS_FLAG(&bd->bd_import->imp_connect_data, BULK_MBITS)){
 			req->rq_mbits = ptlrpc_next_xid();
 		} else {
-			/* old version transfers rq_xid to peer as matchbits */
+			/* Old version transfers rq_xid to peer as
+			 * matchbits. */
 			spin_lock(&req->rq_import->imp_lock);
 			list_del_init(&req->rq_unreplied_list);
 			ptlrpc_assign_next_xid_nolock(req);
-			req->rq_mbits = req->rq_xid;
 			spin_unlock(&req->rq_import->imp_lock);
+			req->rq_mbits = req->rq_xid;
 		}
 
 		CDEBUG(D_HA, "resend bulk old x%llu new x%llu\n",
 		       old_mbits, req->rq_mbits);
+	} else if (!(lustre_msg_get_flags(req->rq_reqmsg) & MSG_REPLAY)) {
+		/* Request being sent first time, use xid as matchbits. */
+		req->rq_mbits = req->rq_xid;
+	} else {
+		/* Replay request, xid and matchbits have already been
+		 * correctly assigned. */
+		return;
 	}
 
 	/*
@@ -3146,7 +3157,10 @@ void ptlrpc_set_bulk_mbits(struct ptlrpc_request *req)
 	req->rq_mbits += DIV_ROUND_UP(bd->bd_iov_count, LNET_MAX_IOV) - 1;
 
 	/* Set rq_xid as rq_mbits to indicate the final bulk for the old
-	 * server which does not support OBD_CONNECT_BULK_MBITS. LU-6808 */
+	 * server which does not support OBD_CONNECT_BULK_MBITS. LU-6808.
+	 *
+	 * It's ok to directly set the rq_xid here, since this xid bump
+	 * won't affect the request position in unreplied list. */
 	if (!OCD_HAS_FLAG(&bd->bd_import->imp_connect_data, BULK_MBITS))
 		req->rq_xid = req->rq_mbits;
 }
-- 
2.14.0


From twhitehead at gmail.com  Wed Sep  6 16:04:39 2017
From: twhitehead at gmail.com (Tyson Whitehead)
Date: Wed, 06 Sep 2017 12:04:39 -0400
Subject: [lustre-devel] [PATCH 1/2] LU-6808 ptlrpc: properly set "rq_xid"
	for 4MB IO
Message-ID: <7562960.y17JJjcatH@whitehead.beowulf.uwo.ca>

From: Fan Yong <fan.yong at intel.com>

The commit d099fdd6 replaced the "rq_xid" with "rq_mbits" as
the matchbits of bulk data transferring. To be interoperable
with old servers, it introduced the new connection flag:
OBD_CONNECT_BULK_MBITS. If the server does not support such
feature, then the "rq_xid" would be set the same value as
"rq_mbits". Unfortunately, it forgot to handle multiple bulk
operations, for example 4MB IO. If the new client wants to
make 4MB IO with old server, it may send a small "rq_xid" to
the old server, as to the old server will regard it as an 1MB
IO or 2MB IO. So the data transfer will not complete because
of only part of data transferred. Then the client will timeout
failure and retry again and again.

Test-Parameters: alwaysuploadlogs testlist=sanity envdefinitions=ONLY=224c ossjob=lustre-b2_7_fe mdsjob=lustre-b2_7_fe ossbuildno=95 mdsbuildno=95 mdsdistro=el6.7 ossdistro=el6.7
Signed-off-by: Fan Yong <fan.yong at intel.com>
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-6808
Reviewed-on: http://review.whamcloud.com/22373
Tested-by: Jenkins
Reviewed-by: Jinshan Xiong <jinshan.xiong at intel.com>
Tested-by: Maloo <hpdd-maloo at intel.com>
Reviewed-by: Bobi Jam <bobijam at hotmail.com>
Reviewed-by: Liang Zhen <liang.zhen at intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin at intel.com>
Signed-off-by: Tyson Whitehead <twhitehead at gmail.com>
---
 drivers/staging/lustre/lustre/ptlrpc/client.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/drivers/staging/lustre/lustre/ptlrpc/client.c b/drivers/staging/lustre/lustre/ptlrpc/client.c
index 6466974a43e7..0f9db5526ba5 100644
--- a/drivers/staging/lustre/lustre/ptlrpc/client.c
+++ b/drivers/staging/lustre/lustre/ptlrpc/client.c
@@ -3123,8 +3123,7 @@ void ptlrpc_set_bulk_mbits(struct ptlrpc_request *req)
 	} else { /* needs to generate a new matchbits for resend */
 		u64 old_mbits = req->rq_mbits;
 
-		if ((bd->bd_import->imp_connect_data.ocd_connect_flags &
-		     OBD_CONNECT_BULK_MBITS)) {
+		if (OCD_HAS_FLAG(&bd->bd_import->imp_connect_data, BULK_MBITS)){
 			req->rq_mbits = ptlrpc_next_xid();
 		} else {
 			/* old version transfers rq_xid to peer as matchbits */
@@ -3145,6 +3144,11 @@ void ptlrpc_set_bulk_mbits(struct ptlrpc_request *req)
 	 * see LU-1431
 	 */
 	req->rq_mbits += DIV_ROUND_UP(bd->bd_iov_count, LNET_MAX_IOV) - 1;
+
+	/* Set rq_xid as rq_mbits to indicate the final bulk for the old
+	 * server which does not support OBD_CONNECT_BULK_MBITS. LU-6808 */
+	if (!OCD_HAS_FLAG(&bd->bd_import->imp_connect_data, BULK_MBITS))
+		req->rq_xid = req->rq_mbits;
 }
 
 /**
-- 
2.14.0


From twhitehead at gmail.com  Wed Sep  6 19:25:30 2017
From: twhitehead at gmail.com (Tyson Whitehead)
Date: Wed, 06 Sep 2017 15:25:30 -0400
Subject: [lustre-devel] [PATCH 0/2] LU-6808: ported to upstream staging
	driver
In-Reply-To: <20170906172852.GA29067@kroah.com>
References: <9525089.uibW5qMffG@whitehead.beowulf.uwo.ca>
	<20170906172852.GA29067@kroah.com>
Message-ID: <2026203.e3uc1VY80x@whitehead.beowulf.uwo.ca>

On Wednesday, September 6, 2017 7:28:52 PM EDT Greg Kroah-Hartman wrote:
> On Wed, Sep 06, 2017 at 12:04:04PM -0400, Tyson Whitehead wrote:
> > Please find enclosed in the next two emails the two patches for LU-6808 ported to the in-tree staging lustre client driver in the 4.12 release.
> 
> What does that mean?  What can I do with these?  4.12 is about to be end-of-life in a few days...
> 
> confused,

Those were some Lustre client driver patches I ported forward to the in-tree 4.12 kernel driver (as that is what our Fedora based system is running).

I was asked to pass them along on the Lustre bug tracking system for inclusion.

I see though that they should be against your staging-testing branch and not the previous release branch.  I'll move them over to this and post again.

Thanks!  -Tyson

From oleg.drokin at intel.com  Fri Sep 15 04:58:29 2017
From: oleg.drokin at intel.com (Oleg Drokin)
Date: Fri, 15 Sep 2017 00:58:29 -0400
Subject: [lustre-devel] new tag 2.10.53
Message-ID: <6AEDE68A-9D25-499C-9876-3B73ED80902B@intel.com>

Hello!

   I just tagged a new tag 2.10.53 in master development branch of the Lustre community tree.

   Here's the changelog:
Alexander Boyko (1):
      LU-7001 osp: fix llog processing

Alexey Lyashkov (2):
      LU-9810 lnet: fix build with M-OFED 4.1
      LU-9810 lnet: prefer Fast Reg

Amir Shehata (6):
      LU-9480 lnet: add enhanced statistics
      LU-9480 lnet: show peer state
      LU-9909 lnet: fix memory leak and lnet_interfaces_max
      LU-9918 lnet: decref on peer after use
      LU-9917 lnet: rediscover peer if it changed
      LU-9921 lnet: resolve unsafe list access

Andreas Dilger (4):
      LU-8653 lod: use stripe_count instead of stripe_nr
      LU-5965 tests: fix parsing for older Lustre versions
      LU-9679 ldlm: remove flock accessor macros
      LU-7746 tests: skip tests for older (upstream) client

Aurelien Degremont (1):
      LU-8435 tests: slab alloc error does not LBUG

Bob Glossman (2):
      LU-9866 kernel: kernel update [SLES12 SP2 4.4.74-92.35]
      LU-9882 kernel: kernel update RHEL7.4 [3.10.0-693.1.1.el7]

Bobi Jam (1):
      LU-9941 lov: lsm_is_composite isn't right

Bruno Faccini (1):
      LU-7988 hsm: update many cookie status at once

Chris Horn (1):
      LU-9803 tests: cast st_blksize for printf

Colin Ian King (1):
      LU-9869 lnet: fix incorrect arguments order calling lstcon_session_new

Dan Carpenter (1):
      LU-9863 lmv: Off by two in lmv_fid2path()

Doug Oucharek (1):
      LU-9828 ptlrpc: Do not assert when bd_nob_transferred != 0

Elena Gryaznova (1):
      LU-8691 tests: add mdtest to ha.sh

Emoly Liu (2):
      LU-9657 pfl: llapi_layout_comp_usei should handle non-pfl file
      LU-9347 ioctl: Add BLKSSZGET ioctl support

Fan Yong (2):
      LU-9842 osd: return ENODATA for XATTR_NAME_FID on MDT
      LU-9410 ldiskfs: no check mb bitmap if flex_bg enabled

Frank Zago (1):
      LU-7988 hsm: added coordinator housekeeping flag

Giuseppe Di Natale (3):
      LU-9781 llog: Improve catalog full warning
      LU-9890 osd-zfs: dmu_objset_own/disown changes
      LU-8342 utils: Set dnodesize/recordsize at zfs dataset create

Gu Zheng (1):
      LU-9950 build: add support for Ubuntu(debian) arm64

Henri Doreau (1):
      LU-9519 utils: liblustreapi header cleanup

Hongchao Zhang (1):
      LU-8066 libcfs: call kernel_param_unlock on error

James Nunez (3):
      LU-9860 tests: Add conf-sanity tests to ALWAYS_EXCEPT list
      LU-9899 tests: mount client on MGS for ost-pools
      LU-9891 tests: Increase space not released for ZFS

James Simmons (6):
      LU-9767 utils: validate filesystem name for mkfs.lustre
      LU-9019 mdd: migrate from jiffies64 to ktime
      LU-8066 libcfs: test for both __kernel_param_[un]lock and kernel_param_[un]lock
      LU-9915 build: remove LC_CONFIG_OBD_BUFFER_SIZE
      LU-9791 obd: always call lprocfs_obd_setup
      LU-9870 build: handle SNMP missing on build box

Jinshan Xiong (1):
      LU-9841 lov: do not split IO for single striped file

John L. Hammond (2):
      LU-9913 lnet: balance references in lnet_discover_peer_locked()
      LU-9856 mdd: handle NULL buffer in mdd_xattr_list()

Lai Siyao (2):
      LU-9848 llog: check padding size for update reclen
      LU-9857 lmv: stripe dir page may be released mistakenly

Minh Diep (1):
      LU-9907 build: add patchless server for lbuild

Olaf Weber (21):
      LU-9480 lnet: add lnet_interfaces_max tunable
      LU-9480 lnet: configure lnet_interfaces_max tunable from dlc
      LU-9480 lnet: add struct lnet_ping_buffer
      LU-9480 lnet: automatic sizing of router pinger buffers
      LU-9480 lnet: add Multi-Rail and Discovery ping feature bits
      LU-9480 lnet: add sanity checks on ping-related constants
      LU-9480 lnet: cleanup of lnet_peer_ni_addref/decref_locked()
      LU-9480 lnet: rename lnet_add/del_peer_ni_to/from_peer()
      LU-9480 lnet: refactor lnet_del_peer_ni()
      LU-9480 lnet: refactor lnet_add_peer_ni()
      LU-9480 lnet: introduce LNET_PEER_MULTI_RAIL flag bit
      LU-9480 lnet: preferred NIs for non-Multi-Rail peers
      LU-9480 lnet: add LNET_PEER_CONFIGURED flag
      LU-9480 lnet: reference counts on lnet_peer/lnet_peer_net
      LU-9480 lnet: add msg_type to lnet_event
      LU-9480 lnet: add discovery thread
      LU-9480 lnet: tune lnet_peer_discovery_disabled with lnetctl
      LU-9480 lnet: add the Push target
      LU-9480 lnet: implement Peer Discovery
      LU-9480 lnet: add "lnetctl peer list"
      LU-9480 lnet: add "lnetctl ping" command

Oleg Drokin (4):
      LU-9480 lnet: Multi-Rail Dynamic Discovery feature
      Revert "LU-5541 lustreapi: only export the API symbols"
      Revert "LU-5541 build: build static and dynamic liblustreapi"
      New tag 2.10.53

Patrick Farrell (1):
      LU-8276 ldlm: Make lru clear always discard read lock pages

Sonia (1):
      LU-9480 lnet: cleanup lnetctl and cyaml

Sonia Sharma (1):
      LU-9480 lnet: add "lnetctl discover"

Steve Guminski (8):
      LU-8993 utils: Use absolute pathname for debug_daemon log file
      LU-6210 utils: Use C99 initializers in lfs_changelog()
      LU-6210 utils: Use C99 struct initializers in lfs_getdirstripe
      LU-6210 ptlrpc: Use C99 initializer in ptlrpc_register_rqbd()
      LU-5170 lfs: Standardize error messages in lfs_setdirstripe()
      LU-6210 lnet: Use C99 struct initializer in framework.c
      LU-6210 utils: Use C99 struct initializers in lnetctl
      LU-6051 utils: Remove incorrect request for getstripe help

Wang Shilong (1):
      LU-9874 osd-ldiskfs: simplify project transfer codes

Wei Liu (2):
      LU-832 test: Add error check when running run-llog.sh
      LU-9260 test: Use the correct mount device when test against lustre

Yang Sheng (1):
      LU-2776 tests: waiting multiop finished in sanityn:51a

dilip krishnagiri (7):
      LU-9591 tests: remove replay-vbr tests 12a from ALWAYS_EXCEPT
      LU-9593 tests: remove sanity-sec tests from ALWAYS_EXCEPT
      LU-9580 tests: remove performance-sanity tests from ALWAYS_EXCEPT
      LU-9588 tests: remove replay-ost-single test from ALWAYS_EXCEPT
      LU-9042 test: Remove conf-sanity tests from ALWAYS_EXCEPT
      LU-9044 test: remove conf-sanity tests from ALWAYS_EXCEPT
      LU-9595 tests: remove sanityn test 18c from ALWAYS_EXCEPT

frank zago (2):
      LU-5541 build: build static and dynamic liblustreapi
      LU-5541 lustreapi: only export the API symbols

sharmaso (1):
      LU-9077 lnet: fix for static analysis issues


From julia.lawall at lip6.fr  Sat Sep 16 11:23:28 2017
From: julia.lawall at lip6.fr (Julia Lawall)
Date: Sat, 16 Sep 2017 13:23:28 +0200 (CEST)
Subject: [lustre-devel] [Outreachy kernel] [PATCH] staging: lustre:
 lnet: Replace list_for_each with list_for_each_entry
In-Reply-To: <20170916004148.GA25693@Haneen>
References: <20170916004148.GA25693@Haneen>
Message-ID: <alpine.DEB.2.20.1709161320250.2943@hadrien>



On Fri, 15 Sep 2017, Haneen Mohammed wrote:

> Replace use of the combination of list_for_each and list_entry
> with list_for_each_entry to simplify the code and remove variables
> that are used only in list_for_each.
> Issue found and corrected using Coccinelle script:
>
> @r@
> expression head, member, e;
> type T1, T2, T3;
> iterator name list_for_each, list_for_each_entry;
> identifier pos, var;
> @@
>
> -T1 *pos;
> ...when!=pos=e;
>
> -list_for_each(pos, head)
> +list_for_each_entry(var, head, member)
> {
> ...when!=pos=e;
>    when!=T3 *var;
> -var = list_entry(pos, T2, member);
> ...when!=pos=e;
> }
> ...when!=pos=e;

Actually, one could consider that there should be when != pos, not when !=
pos=e, because you need that there are no references at all to pos to be
able to delete it.  But it's true that if pos is not initialized then
there should be no other kinds of references either.  The other
possibility for initialization would be eg

f(...,&pos,...)

You could be suspicious of &pos in general.  But anyway there is no &pos
in this code.

julia

> Signed-off-by: Haneen Mohammed <hamohammed.sa at gmail.com>

Acked-by: Julia Lawall <julia.lawall at lip6.fr>

> ---
>  drivers/staging/lustre/lnet/lnet/router.c | 9 ++-------
>  1 file changed, 2 insertions(+), 7 deletions(-)
>
> diff --git a/drivers/staging/lustre/lnet/lnet/router.c b/drivers/staging/lustre/lnet/lnet/router.c
> index 3df101b..b8eba33 100644
> --- a/drivers/staging/lustre/lnet/lnet/router.c
> +++ b/drivers/staging/lustre/lnet/lnet/router.c
> @@ -222,15 +222,12 @@ struct lnet_remotenet *
>  lnet_find_net_locked(__u32 net)
>  {
>  	struct lnet_remotenet *rnet;
> -	struct list_head *tmp;
>  	struct list_head *rn_list;
>
>  	LASSERT(!the_lnet.ln_shutdown);
>
>  	rn_list = lnet_net2rnethash(net);
> -	list_for_each(tmp, rn_list) {
> -		rnet = list_entry(tmp, struct lnet_remotenet, lrn_list);
> -
> +	list_for_each_entry(rnet, rn_list, lrn_list) {
>  		if (rnet->lrn_net == net)
>  			return rnet;
>  	}
> @@ -243,7 +240,6 @@ static void lnet_shuffle_seed(void)
>  	__u32 lnd_type, seed[2];
>  	struct timespec64 ts;
>  	struct lnet_ni *ni;
> -	struct list_head *tmp;
>
>  	if (seeded)
>  		return;
> @@ -254,8 +250,7 @@ static void lnet_shuffle_seed(void)
>  	 * Nodes with small feet have little entropy
>  	 * the NID for this node gives the most entropy in the low bits
>  	 */
> -	list_for_each(tmp, &the_lnet.ln_nis) {
> -		ni = list_entry(tmp, struct lnet_ni, ni_list);
> +	list_for_each_entry(ni, &the_lnet.ln_nis, ni_list) {
>  		lnd_type = LNET_NETTYP(LNET_NIDNET(ni->ni_nid));
>
>  		if (lnd_type != LOLND)
> --
> 2.7.4
>
> --
> You received this message because you are subscribed to the Google Groups "outreachy-kernel" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to outreachy-kernel+unsubscribe at googlegroups.com.
> To post to this group, send email to outreachy-kernel at googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/outreachy-kernel/20170916004148.GA25693%40Haneen.
> For more options, visit https://groups.google.com/d/optout.
>