[lustre-discuss] max_pages_per_rpc=4096 fails on the client nodes

Pinkesh Valdria pinkesh.valdria at oracle.com
Wed Aug 14 12:02:14 PDT 2019


For others, incase they face this issue.  


Solution: I had to unmount and remount for the command to work. 

 

From: Pinkesh Valdria <pinkesh.valdria at oracle.com>
Date: Wednesday, August 14, 2019 at 9:25 AM
To: "lustre-discuss at lists.lustre.org" <lustre-discuss at lists.lustre.org>
Subject: max_pages_per_rpc=4096 fails on the client nodes

 

I want to enable large RPC size.   I followed the steps as per the Lustre manual section: 33.9.2 Usage (http://doc.lustre.org/lustre_manual.xhtml),  but I get the below we error when I try to update the client.    

 

Updated the OSS server: 

[root at lustre-oss-server-nic0-1 test]# lctl set_param obdfilter.lfsbv-*.brw_size=16

obdfilter.lfsbv-OST0000.brw_size=16

obdfilter.lfsbv-OST0001.brw_size=16

obdfilter.lfsbv-OST0002.brw_size=16

obdfilter.lfsbv-OST0003.brw_size=16

obdfilter.lfsbv-OST0004.brw_size=16

obdfilter.lfsbv-OST0005.brw_size=16

obdfilter.lfsbv-OST0006.brw_size=16

obdfilter.lfsbv-OST0007.brw_size=16

obdfilter.lfsbv-OST0008.brw_size=16

obdfilter.lfsbv-OST0009.brw_size=16

[root at lustre-oss-server-nic0-1 test]#

 

Add the above change permanently using MGS node: 

[root at lustre-mds-server-nic0-1 ~]# lctl set_param -P obdfilter.lfsbv-*.brw_size=16

[root at lustre-mds-server-nic0-1 ~]#

 

 

Client side update – failed 

[root at lustre-client-1 ~]# lctl set_param osc.lfsbv-OST*.max_pages_per_rpc=4096

error: set_param: setting /proc/fs/lustre/osc/lfsbv-OST0000-osc-ffff8e66b4b08000/max_pages_per_rpc=4096: Numerical result out of range

error: set_param: setting /proc/fs/lustre/osc/lfsbv-OST0001-osc-ffff8e66b4b08000/max_pages_per_rpc=4096: Numerical result out of range

error: set_param: setting /proc/fs/lustre/osc/lfsbv-OST0002-osc-ffff8e66b4b08000/max_pages_per_rpc=4096: Numerical result out of range

error: set_param: setting /proc/fs/lustre/osc/lfsbv-OST0003-osc-ffff8e66b4b08000/max_pages_per_rpc=4096: Numerical result out of range

…..

…..

 

 

33.9.2. Usage

In order to enable a larger RPC size, brw_size must be changed to an IO size value up to 16MB. To temporarily change brw_size, the following command should be run on the OSS:

oss# lctl set_param obdfilter.fsname-OST*.brw_size=16

To persistently change brw_size, the following command should be run:

oss# lctl set_param -P obdfilter.fsname-OST*.brw_size=16

When a client connects to an OST target, it will fetch brw_size from the target and pick the maximum value of brw_size and its local setting for max_pages_per_rpc as the actual RPC size. Therefore, the max_pages_per_rpc on the client side would have to be set to 16M, or 4096 if the PAGESIZE is 4KB, to enable a 16MB RPC. To temporarily make the change, the following command should be run on the client to setmax_pages_per_rpc:

client$ lctl set_param osc.fsname-OST*.max_pages_per_rpc=16M

To persistently make this change, the following command should be run:

client$ lctl set_param -P obdfilter.fsname-OST*.osc.max_pages_per_rpc=16M

Caution

The brw_size of an OST can be changed on the fly. However, clients have to be remounted to renegotiate the new maximum RPC size.

 

 

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20190814/9f1b7642/attachment-0001.html>


More information about the lustre-discuss mailing list