<html><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">
<br><div><div>On 7 Nov 2007, at 22:31, Nathan Rutman wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Cliff White wrote:</div> <blockquote type="cite"><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Wojciech Turek wrote:</div><p style="margin: 0.0px 0.0px 0.0px 0.0px; min-height: 14.0px"><span class="Apple-converted-space"> </span><br class="khtml-block-placeholder"></p> <blockquote type="cite"><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Hi Cliff,</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><br></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">On 7 Nov 2007, at 17:58, Cliff White wrote:</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><br></div><p style="margin: 0.0px 0.0px 0.0px 0.0px; min-height: 14.0px"><span class="Apple-converted-space"> </span><br class="khtml-block-placeholder"></p> <blockquote type="cite"><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Wojciech Turek wrote:</div><p style="margin: 0.0px 0.0px 0.0px 0.0px; min-height: 14.0px"><span class="Apple-converted-space"> </span><br class="khtml-block-placeholder"></p> <blockquote type="cite"><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Hi,</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Our lustre environment is:</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">2.6.9-55.0.9.EL_lustre.1.6.3smp</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">I would like to change recovery timeout from default value 250s to something longer</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">I tried example from manual:</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">set_timeout <secs> Sets the timeout (obd_timeout) for a server</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">to wait before failing recovery.</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">We performed that experiment on our test lustre installation with one OST.</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">storage02 is our OSS</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">[root@storage02 ~]# lctl dl</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><span class="Apple-converted-space"> </span>0 UP mgc MGC10.143.245.3@tcp 31259d9b-e655-cdc4-c760-45d3df426d86 5</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><span class="Apple-converted-space"> </span>1 UP ost OSS OSS_uuid 3</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><span class="Apple-converted-space"> </span>2 UP obdfilter home-md-OST0001 home-md-OST0001_UUID 7</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">[root@storage02 ~]# lctl --device 2 set_timeout 600</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">set_timeout has been deprecated. Use conf_param instead.</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">e.g. conf_param lustre-MDT0000 obd_timeout=50</div><p style="margin: 0.0px 0.0px 0.0px 0.0px; min-height: 14.0px"><span class="Apple-converted-space"> </span><br class="khtml-block-placeholder"></p> </blockquote></blockquote></blockquote></blockquote><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">sorry about this bad help message.<span class="Apple-converted-space"> </span>It's wrong.</div> <blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">usage: conf_param obd_timeout=<secs></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">run <command> after connecting to device <devno></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">--device <devno> <command [args ...]></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">[root@storage02 ~]# lctl --device 1 conf_param obd_timeout=600</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">No device found for name MGS: Invalid argument</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">error: conf_param: No such device</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">It looks like I need to run this command from MGS node so I<span class="Apple-converted-space"> </span>moved then to MGS server called storage03</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">[root@storage03 ~]# lctl dl</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><span class="Apple-converted-space"> </span>0 UP mgs MGS MGS 9</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><span class="Apple-converted-space"> </span>1 UP mgc MGC10.143.245.3@tcp f51a910b-a08e-4be6-5ada-b602a5ca9ab3 5</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><span class="Apple-converted-space"> </span>2 UP mdt MDS MDS_uuid 3</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><span class="Apple-converted-space"> </span>3 UP lov home-md-mdtlov home-md-mdtlov_UUID 4</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><span class="Apple-converted-space"> </span>4 UP mds home-md-MDT0000 home-md-MDT0000_UUID 5</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><span class="Apple-converted-space"> </span>5 UP osc home-md-OST0001-osc home-md-mdtlov_UUID 5</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">[root@storage03 ~]# lctl device 5</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">[root@storage03 ~]# lctl conf_param obd_timeout=600</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">error: conf_param: Function not implemented</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">[root@storage03 ~]# lctl --device 5 conf_param obd_timeout=600</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">error: conf_param: Function not implemented</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">[root@storage03 ~]# lctl help conf_param</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">conf_param: set a permanent config param. This command must be run on the MGS node</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">usage: conf_param <target.keyword=val> ...</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">[root@storage03 ~]# lctl conf_param home-md-MDT0000.obd_timeout=600</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">error: conf_param: Invalid argument</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">[root@storage03 ~]#</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">I searched whole /proc/*/lustre for file that can store this timeout value but nothing were found.</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Could someone advise how to change value for recovery timeout?</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Cheers,</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Wojciech Turek</div><p style="margin: 0.0px 0.0px 0.0px 0.0px; min-height: 14.0px"><span class="Apple-converted-space"> </span><br class="khtml-block-placeholder"></p> </blockquote><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">It looks like your file system is named 'home' - you can confirm with</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">tunefs.lustre --print <MDS device> | grep "Lustre FS"</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><br></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">The correct command (Run on the MGS) would be</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "># lctl conf_param home.sys.timeout=<val></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><br></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Example:</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">[root@ft4 ~]# tunefs.lustre --print /dev/sdb |grep "Lustre FS"</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Lustre FS:<span class="Apple-converted-space"> </span>lustre</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">[root@ft4 ~]# cat /proc/sys/lustre/timeout</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">130</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">[root@ft4 ~]# lctl conf_param lustre.sys.timeout=150</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">[root@ft4 ~]# cat /proc/sys/lustre/timeout</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">150</div><p style="margin: 0.0px 0.0px 0.0px 0.0px; min-height: 14.0px"><span class="Apple-converted-space"> </span><br class="khtml-block-placeholder"></p> </blockquote><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Thanks for your email. I am afraid your tips aren't very helpful in this case. As stated in the subject I am asking about recovery timeout.</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">You can find it for example in /proc/fs/lustre/obdfilter/<OST>/recovery_status whilst one of your OST's is in recovery state. By default this timeout is 250s.</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Whereas you are talking about system obd timeout (according to CFS documentation chapter 4.1.2 ) which is not a subject of my concern.</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><br></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Any way I tried your example just to see if it works and again I am afraid it doesn't work for me, see below:</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">I have combined mgs and mds configuration.</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><br></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">[[root@storage03 ~]# df</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Filesystem <span class="Apple-converted-space"> </span>1K-blocks<span class="Apple-converted-space"> </span>Used Available Use% Mounted on</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">/dev/sda1 <span class="Apple-converted-space"> </span>10317828 <span class="Apple-converted-space"> </span>3452824 <span class="Apple-converted-space"> </span>6340888<span class="Apple-converted-space"> </span>36% /</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">/dev/sda6<span class="Apple-converted-space"> </span>7605856 <span class="Apple-converted-space"> </span>49788 <span class="Apple-converted-space"> </span>7169708 <span class="Apple-converted-space"> </span>1% /local</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">/dev/sda3<span class="Apple-converted-space"> </span>4127108 <span class="Apple-converted-space"> </span>41000 <span class="Apple-converted-space"> </span>3876460 <span class="Apple-converted-space"> </span>2% /tmp</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">/dev/sda2<span class="Apple-converted-space"> </span>4127108<span class="Apple-converted-space"> </span>753668 <span class="Apple-converted-space"> </span>3163792<span class="Apple-converted-space"> </span>20% /var</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">/dev/dm-2<span class="Apple-converted-space"> </span>1845747840 447502120 1398245720<span class="Apple-converted-space"> </span>25% /mnt/sdb</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">/dev/dm-1<span class="Apple-converted-space"> </span>6140723200 4632947344 1507775856<span class="Apple-converted-space"> </span>76% /mnt/sdc</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">/dev/dm-3<span class="Apple-converted-space"> </span>286696376 <span class="Apple-converted-space"> </span>1461588 268850900 <span class="Apple-converted-space"> </span>1% /mnt/home-md/mdt</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">[root@storage03 ~]# tunefs.lustre --print /dev/dm-3 |grep "Lustre FS"</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Lustre FS:<span class="Apple-converted-space"> </span>home-md</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Lustre FS:<span class="Apple-converted-space"> </span>home-md</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">[root@storage03 ~]# cat /proc/sys/lustre/timeout</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">100</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">[root@storage03 ~]# lctl conf_param home-md.sys.timeout=150</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">error: conf_param: Invalid argument</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">[root@storage03 ~]#</div><p style="margin: 0.0px 0.0px 0.0px 0.0px; min-height: 14.0px"><span class="Apple-converted-space"> </span><br class="khtml-block-placeholder"></p> </blockquote></blockquote><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">You need to do this on the MGS node, with the MGS running.</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><br></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">mgs> lctl conf_param testfs.sys.timeout=150</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">anynode> cat /proc/sys/lustre/timeout</div></blockquote>This isn't working for me. In my production configuration I have MGS combined with MDT on the same server. My lustre configuration consists of two file systems.</div><div><div>[root@mds01 ~]# tunefs.lustre --print /dev/dm-0</div><div>checking for existing Lustre data: found CONFIGS/mountdata</div><div>Reading CONFIGS/mountdata</div><div><br class="webkit-block-placeholder"></div><div> Read previous values:</div><div>Target: ddn-home-MDT0000</div><div>Index: 0</div><div>Lustre FS: ddn-home</div><div>Mount type: ldiskfs</div><div>Flags: 0x5</div><div> (MDT MGS )</div><div>Persistent mount opts: errors=remount-ro,iopen_nopriv,user_xattr</div><div>Parameters: failover.node=10.143.245.202@tcp mgsnode=10.143.245.202@tcp</div><div><br class="webkit-block-placeholder"></div><div><br class="webkit-block-placeholder"></div><div> Permanent disk data:</div><div>Target: ddn-home-MDT0000</div><div>Index: 0</div><div>Lustre FS: ddn-home</div><div>Mount type: ldiskfs</div><div>Flags: 0x5</div><div> (MDT MGS )</div><div>Persistent mount opts: errors=remount-ro,iopen_nopriv,user_xattr</div><div>Parameters: failover.node=10.143.245.202@tcp mgsnode=10.143.245.202@tcp</div><div><br class="webkit-block-placeholder"></div><div>exiting before disk write.</div><div>[root@mds01 ~]# tunefs.lustre --print /dev/dm-1</div><div>checking for existing Lustre data: found CONFIGS/mountdata</div><div>Reading CONFIGS/mountdata</div><div><br class="webkit-block-placeholder"></div><div> Read previous values:</div><div>Target: ddn-data-MDT0000</div><div>Index: 0</div><div>Lustre FS: ddn-data</div><div>Mount type: ldiskfs</div><div>Flags: 0x1</div><div> (MDT )</div><div>Persistent mount opts: errors=remount-ro,iopen_nopriv,user_xattr</div><div>Parameters: mgsnode=10.143.245.201@tcp failover.node=10.143.245.202@tcp</div><div><br class="webkit-block-placeholder"></div><div><br class="webkit-block-placeholder"></div><div> Permanent disk data:</div><div>Target: ddn-data-MDT0000</div><div>Index: 0</div><div>Lustre FS: ddn-data</div><div>Mount type: ldiskfs</div><div>Flags: 0x1</div><div> (MDT )</div><div>Persistent mount opts: errors=remount-ro,iopen_nopriv,user_xattr</div><div>Parameters: mgsnode=10.143.245.201@tcp failover.node=10.143.245.202@tcp</div><div><br class="webkit-block-placeholder"></div><div>exiting before disk write.</div><div>[root@mds01 ~]# </div><div><br class="webkit-block-placeholder"></div><div>As you can see above MGS is on /dev/dm-0 combined with MDT for ddn-home file system.</div><div>If I try command line from your example I get this:</div><div><div>[root@mds01 ~]# lctl conf_param ddn-home.sys.timeout=200</div><div>error: conf_param: Invalid argument</div></div><div><br class="webkit-block-placeholder"></div><div>Server mds01 is 100% MGS node. What is wrong here then? The only two reasons for that problem I can think of is that file system name contain "-" character. However I didn't find anything in documentation that would say that this character is not allowed to be used. Another reason is that MGS is combined with MDS?</div><div><br class="webkit-block-placeholder"></div><div>syslog contains following messages:</div><div><br class="webkit-block-placeholder"></div><div><div>Nov 7 18:38:35 mds01 kernel: LustreError: 3273:0:(mgs_llog.c:1957:mgs_setparam()) No filesystem targets for ddn. cfg_device from lctl is 'ddn-home'</div><div>Nov 7 18:38:35 mds01 kernel: LustreError: 3273:0:(mgs_handler.c:605:mgs_iocontrol()) setparam err -22</div><div>Nov 7 18:39:46 mds01 kernel: LustreError: 3274:0:(mgs_llog.c:1957:mgs_setparam()) No filesystem targets for ddn. cfg_device from lctl is 'ddn-data'</div><div>Nov 7 18:39:46 mds01 kernel: LustreError: 3274:0:(mgs_handler.c:605:mgs_iocontrol()) setparam err -22</div><div>Nov 7 18:39:54 mds01 kernel: LustreError: 3275:0:(mgs_llog.c:1957:mgs_setparam()) No filesystem targets for ddn. cfg_device from lctl is 'ddn-data'</div><div>Nov 7 18:39:54 mds01 kernel: LustreError: 3275:0:(mgs_handler.c:605:mgs_iocontrol()) setparam err -22</div><div>Nov 7 18:40:01 mds01 kernel: LustreError: 3282:0:(mgs_llog.c:1957:mgs_setparam()) No filesystem targets for ddn. cfg_device from lctl is 'ddn-data'</div><div>Nov 7 18:40:01 mds01 kernel: LustreError: 3282:0:(mgs_handler.c:605:mgs_iocontrol()) setparam err -22</div><div>Nov 7 18:41:06 mds01 kernel: LustreError: 3305:0:(mgs_llog.c:1957:mgs_setparam()) No filesystem targets for ddn. cfg_device from lctl is 'ddn-data'</div><div>Nov 7 18:41:06 mds01 kernel: LustreError: 3305:0:(mgs_handler.c:605:mgs_iocontrol()) setparam err -22</div><div>Nov 7 18:41:15 mds01 kernel: LustreError: 3306:0:(mgs_llog.c:1957:mgs_setparam()) No filesystem targets for ddn. cfg_device from lctl is 'ddn-home'</div><div>Nov 7 18:41:15 mds01 kernel: LustreError: 3306:0:(mgs_handler.c:605:mgs_iocontrol()) setparam err -22</div><div><br class="webkit-block-placeholder"></div><div>From above it looks like only first part of file system name is recognized "ddn" and -home or -data is omitted.</div></div><div><br class="webkit-block-placeholder"></div><div>Please advise.</div><div><br class="webkit-block-placeholder"></div><div>Wojciech Turek</div><br class="webkit-block-placeholder"></div><div><blockquote type="cite"><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><br></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><br></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><br></div> <blockquote type="cite"><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Hmm, not sure why that isn't working for you, I tested the example I gave. Sorry about the mis-read. The obd recovery timeout is defined in relation to obd_timeout, and afaik not changeable at runtime:</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><br></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">lustre/include/lustre_lib.h</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">#define OBD_RECOVERY_TIMEOUT (obd_timeout * 5 / 2)</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">...which gives the default 250 seconds for the default obd_timeout (100 seconds)</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><br></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">cliffw</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><br></div><p style="margin: 0.0px 0.0px 0.0px 0.0px; min-height: 14.0px"><span class="Apple-converted-space"> </span><br class="khtml-block-placeholder"></p> </blockquote><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">That's correct.<span class="Apple-converted-space"> </span>These are tied together before lustre 1.6.4.</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><br></div> <blockquote type="cite"><blockquote type="cite"><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Cheers,</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><br></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Wojciech Turek</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><br></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><br></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><br></div><p style="margin: 0.0px 0.0px 0.0px 0.0px; min-height: 14.0px"><span class="Apple-converted-space"> </span><br class="khtml-block-placeholder"></p> <blockquote type="cite"><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">cliffw</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><br></div><p style="margin: 0.0px 0.0px 0.0px 0.0px; min-height: 14.0px"><span class="Apple-converted-space"> </span><br class="khtml-block-placeholder"></p> <blockquote type="cite"><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">------------------------------------------------------------------------</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">_______________________________________________</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Lustre-discuss mailing list</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><a href="mailto:Lustre-discuss@clusterfs.com">Lustre-discuss@clusterfs.com</a></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><a href="https://mail.clusterfs.com/mailman/listinfo/lustre-discuss">https://mail.clusterfs.com/mailman/listinfo/lustre-discuss</a></div><p style="margin: 0.0px 0.0px 0.0px 0.0px; min-height: 14.0px"><span class="Apple-converted-space"> </span><br class="khtml-block-placeholder"></p> </blockquote></blockquote><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Mr Wojciech Turek</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Assistant System Manager</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">University of Cambridge</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">High Performance Computing service</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">email: <a href="mailto:wjt27@cam.ac.uk">wjt27@cam.ac.uk</a></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">tel. +441223763517</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><br></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><br></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><br></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><br></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><br></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">------------------------------------------------------------------------</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><br></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">_______________________________________________</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Lustre-discuss mailing list</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><a href="mailto:Lustre-discuss@clusterfs.com">Lustre-discuss@clusterfs.com</a></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><a href="https://mail.clusterfs.com/mailman/listinfo/lustre-discuss">https://mail.clusterfs.com/mailman/listinfo/lustre-discuss</a></div><p style="margin: 0.0px 0.0px 0.0px 0.0px; min-height: 14.0px"><span class="Apple-converted-space"> </span><br class="khtml-block-placeholder"></p> </blockquote><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><br></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">_______________________________________________</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Lustre-discuss mailing list</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><a href="mailto:Lustre-discuss@clusterfs.com">Lustre-discuss@clusterfs.com</a></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><a href="https://mail.clusterfs.com/mailman/listinfo/lustre-discuss">https://mail.clusterfs.com/mailman/listinfo/lustre-discuss</a></div><p style="margin: 0.0px 0.0px 0.0px 0.0px; min-height: 14.0px"><span class="Apple-converted-space"> </span><br class="khtml-block-placeholder"></p> </blockquote><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><br></div> </blockquote></div><br><div> <span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0; "><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0; "><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0; "><div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Lucida Grande" size="3" style="font: 11.0px Lucida Grande">Mr Wojciech Turek</font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 11px/normal Lucida Grande; min-height: 13px; ">Assistant System Manager</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Lucida Grande" size="3" style="font: 11.0px Lucida Grande">University of Cambridge</font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Lucida Grande" size="3" style="font: 11.0px Lucida Grande">High Performance Computing service </font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font class="Apple-style-span" face="'Lucida Grande'" size="3"><span class="Apple-style-span" style="font-size: 11px;">email: <a href="mailto:wjt27@cam.ac.uk">wjt27@cam.ac.uk</a></span></font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Lucida Grande" size="3" style="font: 11.0px Lucida Grande">tel. +441223763517</font></div></div><div><br class="webkit-block-placeholder"></div></span><br class="Apple-interchange-newline"></span></span> </div><br></body></html>