<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><meta http-equiv=Content-Type content="text/html; charset=utf-8"><meta name=Generator content="Microsoft Word 15 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Times;
panose-1:0 0 5 0 0 0 0 2 0 0;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Calibri",sans-serif;}
h3
{mso-style-priority:9;
mso-style-link:"Heading 3 Char";
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:13.5pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
code
{mso-style-priority:99;
font-family:"Courier New";}
pre
{mso-style-priority:99;
mso-style-link:"HTML Preformatted Char";
margin:0in;
margin-bottom:.0001pt;
font-size:10.0pt;
font-family:"Courier New";}
p.msonormal0, li.msonormal0, div.msonormal0
{mso-style-name:msonormal;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.Heading3Char
{mso-style-name:"Heading 3 Char";
mso-style-priority:9;
mso-style-link:"Heading 3";
font-family:"Calibri",sans-serif;
font-weight:bold;}
span.HTMLPreformattedChar
{mso-style-name:"HTML Preformatted Char";
mso-style-priority:99;
mso-style-link:"HTML Preformatted";
font-family:"Courier New";}
span.EmailStyle22
{mso-style-type:personal;
font-family:"Calibri",sans-serif;
color:windowtext;}
span.EmailStyle23
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style></head><body lang=EN-US link="#0563C1" vlink="#954F72"><div class=WordSection1><p class=MsoNormal><span style='font-size:11.0pt'>For others, incase they face this issue. <o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt'><br>Solution: I had to unmount and remount for the command to work. <o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt'><o:p> </o:p></span></p><div style='border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in'><p class=MsoNormal><b><span style='color:black'>From: </span></b><span style='color:black'>Pinkesh Valdria <pinkesh.valdria@oracle.com><br><b>Date: </b>Wednesday, August 14, 2019 at 9:25 AM<br><b>To: </b>"lustre-discuss@lists.lustre.org" <lustre-discuss@lists.lustre.org><br><b>Subject: </b>max_pages_per_rpc=4096 fails on the client nodes<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-size:11.0pt'><o:p> </o:p></span></p></div><p class=MsoNormal><span style='font-size:11.0pt'>I want to enable large RPC size. I followed the steps as per the Lustre manual section: 33.9.2 Usage (</span><a href="http://doc.lustre.org/lustre_manual.xhtml">http://doc.lustre.org/lustre_manual.xhtml</a><span style='font-size:11.0pt'>), but I get the below we error when I try to update the client. </span><o:p></o:p></p><p class=MsoNormal><span style='font-size:11.0pt'> </span><o:p></o:p></p><p class=MsoNormal><span style='font-size:11.0pt'>Updated the OSS server: </span><o:p></o:p></p><p class=MsoNormal><span style='font-size:11.0pt'>[root@lustre-oss-server-nic0-1 test]# lctl set_param obdfilter.lfsbv-*.brw_size=16</span><o:p></o:p></p><p class=MsoNormal><span style='font-size:11.0pt'>obdfilter.lfsbv-OST0000.brw_size=16</span><o:p></o:p></p><p class=MsoNormal><span style='font-size:11.0pt'>obdfilter.lfsbv-OST0001.brw_size=16</span><o:p></o:p></p><p class=MsoNormal><span style='font-size:11.0pt'>obdfilter.lfsbv-OST0002.brw_size=16</span><o:p></o:p></p><p class=MsoNormal><span style='font-size:11.0pt'>obdfilter.lfsbv-OST0003.brw_size=16</span><o:p></o:p></p><p class=MsoNormal><span style='font-size:11.0pt'>obdfilter.lfsbv-OST0004.brw_size=16</span><o:p></o:p></p><p class=MsoNormal><span style='font-size:11.0pt'>obdfilter.lfsbv-OST0005.brw_size=16</span><o:p></o:p></p><p class=MsoNormal><span style='font-size:11.0pt'>obdfilter.lfsbv-OST0006.brw_size=16</span><o:p></o:p></p><p class=MsoNormal><span style='font-size:11.0pt'>obdfilter.lfsbv-OST0007.brw_size=16</span><o:p></o:p></p><p class=MsoNormal><span style='font-size:11.0pt'>obdfilter.lfsbv-OST0008.brw_size=16</span><o:p></o:p></p><p class=MsoNormal><span style='font-size:11.0pt'>obdfilter.lfsbv-OST0009.brw_size=16</span><o:p></o:p></p><p class=MsoNormal><span style='font-size:11.0pt'>[root@lustre-oss-server-nic0-1 test]#</span><o:p></o:p></p><p class=MsoNormal><span style='font-size:11.0pt'> </span><o:p></o:p></p><p class=MsoNormal><span style='font-size:11.0pt'>Add the above change permanently using MGS node: </span><o:p></o:p></p><p class=MsoNormal><span style='font-size:11.0pt'>[root@lustre-mds-server-nic0-1 ~]# lctl set_param -P obdfilter.lfsbv-*.brw_size=16</span><o:p></o:p></p><p class=MsoNormal><span style='font-size:11.0pt'>[root@lustre-mds-server-nic0-1 ~]#</span><o:p></o:p></p><p class=MsoNormal><span style='font-size:11.0pt'> </span><o:p></o:p></p><p class=MsoNormal><span style='font-size:11.0pt'> </span><o:p></o:p></p><p class=MsoNormal><span style='font-size:11.0pt'>Client side update – failed </span><o:p></o:p></p><p class=MsoNormal><span style='font-size:11.0pt'>[root@lustre-client-1 ~]# lctl set_param osc.lfsbv-OST*.max_pages_per_rpc=4096</span><o:p></o:p></p><p class=MsoNormal><span style='font-size:11.0pt'>error: set_param: setting /proc/fs/lustre/osc/lfsbv-OST0000-osc-ffff8e66b4b08000/max_pages_per_rpc=4096: Numerical result out of range</span><o:p></o:p></p><p class=MsoNormal><span style='font-size:11.0pt'>error: set_param: setting /proc/fs/lustre/osc/lfsbv-OST0001-osc-ffff8e66b4b08000/max_pages_per_rpc=4096: Numerical result out of range</span><o:p></o:p></p><p class=MsoNormal><span style='font-size:11.0pt'>error: set_param: setting /proc/fs/lustre/osc/lfsbv-OST0002-osc-ffff8e66b4b08000/max_pages_per_rpc=4096: Numerical result out of range</span><o:p></o:p></p><p class=MsoNormal><span style='font-size:11.0pt'>error: set_param: setting /proc/fs/lustre/osc/lfsbv-OST0003-osc-ffff8e66b4b08000/max_pages_per_rpc=4096: Numerical result out of range</span><o:p></o:p></p><p class=MsoNormal><span style='font-size:11.0pt'>…..</span><o:p></o:p></p><p class=MsoNormal><span style='font-size:11.0pt'>…..</span><o:p></o:p></p><p class=MsoNormal><span style='font-size:11.0pt'> </span><o:p></o:p></p><p class=MsoNormal><span style='font-size:11.0pt'> </span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><b><span style='font-size:13.5pt;font-family:Times;color:black'>33.9.2. Usage</span></b><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:13.5pt;font-family:Times;color:black'>In order to enable a larger RPC size, </span><span style='font-size:10.0pt;font-family:"Courier New";color:black'>brw_size</span><span style='font-size:13.5pt;font-family:Times;color:black'> must be changed to an IO size value up to 16MB. To temporarily change </span><span style='font-size:10.0pt;font-family:"Courier New";color:black'>brw_size</span><span style='font-size:13.5pt;font-family:Times;color:black'>, the following command should be run on the OSS:</span><o:p></o:p></p><p class=MsoNormal style='background:#E0E0E0'><span style='font-size:10.0pt;font-family:"Courier New";color:black'>oss# lctl set_param obdfilter.<i>fsname</i>-OST*.brw_size=16</span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:13.5pt;font-family:Times;color:black'>To persistently change </span><span style='font-size:10.0pt;font-family:"Courier New";color:black'>brw_size</span><span style='font-size:13.5pt;font-family:Times;color:black'>, the following command should be run:</span><o:p></o:p></p><p class=MsoNormal style='background:#E0E0E0'><span style='font-size:10.0pt;font-family:"Courier New";color:black'>oss# lctl set_param -P obdfilter.<i>fsname</i>-OST*.brw_size=16</span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:13.5pt;font-family:Times;color:black'>When a client connects to an OST target, it will fetch </span><span style='font-size:10.0pt;font-family:"Courier New";color:black'>brw_size</span><span style='font-size:13.5pt;font-family:Times;color:black'> from the target and pick the maximum value of </span><span style='font-size:10.0pt;font-family:"Courier New";color:black'>brw_size</span><span style='font-size:13.5pt;font-family:Times;color:black'> and its local setting for </span><span style='font-size:10.0pt;font-family:"Courier New";color:black'>max_pages_per_rpc</span><span style='font-size:13.5pt;font-family:Times;color:black'> as the actual RPC size. Therefore, the </span><span style='font-size:10.0pt;font-family:"Courier New";color:black'>max_pages_per_rpc</span><span style='font-size:13.5pt;font-family:Times;color:black'> on the client side would have to be set to 16M, or 4096 if the PAGESIZE is 4KB, to enable a 16MB RPC. To temporarily make the change, the following command should be run on the client to set</span><span style='font-size:10.0pt;font-family:"Courier New";color:black'>max_pages_per_rpc</span><span style='font-size:13.5pt;font-family:Times;color:black'>:</span><o:p></o:p></p><p class=MsoNormal style='background:#E0E0E0'><span style='font-size:10.0pt;font-family:"Courier New";color:black'>client$ lctl set_param osc.<i>fsname</i>-OST*.max_pages_per_rpc=16M</span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:13.5pt;font-family:Times;color:black'>To persistently make this change, the following command should be run:</span><o:p></o:p></p><p class=MsoNormal style='background:#E0E0E0'><span style='font-size:10.0pt;font-family:"Courier New";color:black'>client$ lctl set_param -P obdfilter.<i>fsname</i>-OST*.osc.max_pages_per_rpc=16M</span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><b><span style='font-size:13.5pt;font-family:Times;color:black'>Caution</span></b><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:13.5pt;font-family:Times;color:black'>The </span><span style='font-size:10.0pt;font-family:"Courier New";color:black'>brw_size</span><span style='font-size:13.5pt;font-family:Times;color:black'> of an OST can be changed on the fly. However, clients have to be remounted to renegotiate the new maximum RPC size.</span><o:p></o:p></p><p class=MsoNormal><span style='font-size:11.0pt'> </span><o:p></o:p></p><p class=MsoNormal><span style='font-size:11.0pt'> </span><o:p></o:p></p><p class=MsoNormal><span style='font-size:11.0pt'> </span><o:p></o:p></p></div></body></html>