<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body dir="auto">
Brian,
<div>Please file a ticket in LUDOC with details of how the manual should be updated. Ideally, including a patch. :-)<br>
<br>
<div dir="ltr">Cheers, Andreas</div>
<div dir="ltr"><br>
<blockquote type="cite">On Jul 11, 2023, at 15:39, Brad Merchant <bmerchant@cambridgecomputer.com> wrote:<br>
<br>
</blockquote>
</div>
<blockquote type="cite">
<div dir="ltr">
<div dir="ltr">We recreated the issue in a test cluster and it was definitely the llog_cancel steps that caused the issue. Clients couldn't process the llog properly on new mounts and would fail. We had to completely clear the llog and --writeconf every target
to regenerate it from scratch.
<div><br>
</div>
<div>The cluster is up and running now but I would certainly recommend at least revising that section of the manual.</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Mon, Jul 10, 2023 at 5:22 PM Brad Merchant <<a href="mailto:bmerchant@cambridgecomputer.com">bmerchant@cambridgecomputer.com</a>> wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div dir="ltr">We deactivated half of 32 OSTs after draining them. We followed the steps in section 14.9.3 of the lustre manual
<div><br>
</div>
<div><a href="https://doc.lustre.org/lustre_manual.xhtml#lustremaint.remove_ost" target="_blank">https://doc.lustre.org/lustre_manual.xhtml#lustremaint.remove_ost</a><br>
</div>
<div><br>
</div>
<div>After running the steps in subhead "3. Deactivate the OST." on OST0010-OST001f, new client mounts fail with the below log messages. Existing client mounts seem to function correctly but are on a bit of a ticking timebomb because they are configured with
autofs.</div>
<div><br>
</div>
<div>The llog_cancel steps are new to me and the issues seemed to appear after those commands were issued (can't say that 100% definitively however). Servers are running 2.12.5 and clients are on 2.14.x<br>
</div>
<div><br>
</div>
<div><br>
</div>
<div>Jul 10 15:22:40 adm-sup1 kernel: LustreError: 26814:0:(obd_config.c:1514:class_process_config()) no device for: hydra-OST0010-osc-ffff8be5340c2000<br>
Jul 10 15:22:40 adm-sup1 kernel: LustreError: 26814:0:(obd_config.c:2038:class_config_llog_handler()) MGC172.16.100.101@o2ib: cfg command failed: rc = -22<br>
Jul 10 15:22:40 adm-sup1 kernel: Lustre: cmd=cf00f 0:hydra-OST0010-osc 1:osc.active=0 <br>
Jul 10 15:22:40 adm-sup1 kernel: LustreError: 15b-f: MGC172.16.100.101@o2ib: Configuration from log hydra-client failed from MGS -22. Check client and MGS are on compatible version.<br>
Jul 10 15:22:40 adm-sup1 kernel: Lustre: hydra: root_squash is set to 99:99<br>
Jul 10 15:22:40 adm-sup1 systemd-udevd[26823]: Process '/usr/sbin/lctl set_param 'llite.hydra-ffff8be5340c2000.nosquash_nids=192.168.80.84@tcp 192.168.80.122@tcp 192.168.80.21@tcp 172.16.90.11@o2ib 172.16.100.211@o2ib 172.16.100.212@o2ib 172.16.100.213@o2ib
172.16.100.214@o2ib 172.16.100.215@o2ib 172.16.90.51@o2ib'' failed with exit code 2.<br>
Jul 10 15:22:40 adm-sup1 kernel: Lustre: Unmounted hydra-client<br>
Jul 10 15:22:40 adm-sup1 kernel: LustreError: 26803:0:(obd_mount.c:1680:lustre_fill_super()) Unable to mount (-22)<br>
</div>
<div><br>
</div>
<div><br>
<br>
</div>
</div>
</blockquote>
</div>
<span>_______________________________________________</span><br>
<span>lustre-discuss mailing list</span><br>
<span>lustre-discuss@lists.lustre.org</span><br>
<span>http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org</span><br>
</div>
</blockquote>
</div>
</body>
</html>