<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<style type="text/css" style="display:none;"><!-- P {margin-top:0;margin-bottom:0;} --></style>
</head>
<body dir="ltr">
<div id="divtagdefaultwrapper" style="font-size:12pt;color:#000000;font-family:Calibri,Helvetica,sans-serif;" dir="ltr">
<p style="margin-top:0;margin-bottom:0">Hi folks,</p>
<p style="margin-top:0;margin-bottom:0"><br>
</p>
<p style="margin-top:0;margin-bottom:0">I've just upgraded a 2.7.0 cluster to 2.10.3 and thought I'd take advantage of the new HA resource agents. Sadly, I find that the resource agent successfully mounts the OSDs, then the resource stops (leaving the OSDs
mounted). Here's an example case, the management OSD</p>
<p style="margin-top:0;margin-bottom:0"><br>
</p>
<p style="margin-top:0;margin-bottom:0"><span>Created with the following:</span></p>
<p style="margin-top:0;margin-bottom:0"><span># pcs resource create MGT ocf:lustre:Lustre target=/dev/disk/by-label/MGS mountpoint=/mnt/MGT; pcs constraint location MGT prefers hpctestmds1=100</span><br>
</p>
<p style="margin-top:0;margin-bottom:0"><br>
</p>
<p style="margin-top:0;margin-bottom:0">Results in the following, leaving the resource stopped but the MGT mounted:</p>
<p style="margin-top:0;margin-bottom:0"></p>
<div>
<div>Mar 07 13:28:22 hpctestmds1.our.domain Lustre(MGT)[32115]: ERROR: /dev/disk/by-label/MGS is not mounted</div>
<div>Mar 07 13:28:22 hpctestmds1.our.domain crmd[11459]: notice: Result of probe operation for MGT on hpctestmds1: 7 (not running)</div>
<div>Mar 07 13:28:22 hpctestmds1.our.domain Lustre(MGT)[32128]: INFO: Starting to mount /dev/disk/by-label/MGS</div>
<div>Mar 07 13:28:22 hpctestmds1.our.domain kernel: LDISKFS-fs (sde): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc</div>
<div>Mar 07 13:28:22 hpctestmds1.our.domain kernel: Lustre: MGS: Connection restored to 9eb39832-a281-1088-d816-410b918b5813 (at 0@lo)</div>
<div>Mar 07 13:28:22 hpctestmds1.our.domain kernel: Lustre: Skipped 6 previous similar messages</div>
<div>Mar 07 13:28:22 hpctestmds1.our.domain Lustre(MGT)[32173]: INFO: /dev/disk/by-label/MGS mounted successfully</div>
<div>Mar 07 13:28:22 hpctestmds1.our.domain crmd[11459]: notice: Result of start operation for MGT on hpctestmds1: 0 (ok)</div>
<div>Mar 07 13:28:22 hpctestmds1.our.domain Lustre(MGT)[32189]: ERROR: /dev/disk/by-label/MGS is not mounted</div>
<div>Mar 07 13:28:22 hpctestmds1.our.domain crmd[11459]: notice: Result of stop operation for MGT on hpctestmds1: 0 (ok)</div>
<div>Mar 07 13:28:23 hpctestmds1.our.domain Lustre(MGT)[32207]: INFO: Starting to mount /dev/disk/by-label/MGS</div>
<div>Mar 07 13:28:23 hpctestmds1.our.domain Lustre(MGT)[32215]: ERROR: mount failed</div>
<div>Mar 07 13:28:23 hpctestmds1.our.domain Lustre(MGT)[32221]: ERROR: /dev/disk/by-label/MGS can not be mounted with this error: 1</div>
<div>Mar 07 13:28:23 hpctestmds1.our.domain lrmd[11456]: notice: MGT_start_0:32200:stderr [ mount.lustre: according to /etc/mtab /dev/sde is already mounted on /mnt/MGT ]</div>
<div>Mar 07 13:28:23 hpctestmds1.our.domain crmd[11459]: notice: Result of start operation for MGT on hpctestmds1: 1 (unknown error)</div>
<div>Mar 07 13:28:23 hpctestmds1.our.domain crmd[11459]: notice: hpctestmds1-MGT_start_0:558 [ mount.lustre: according to /etc/mtab /dev/sde is already mounted on /mnt/MGT\n ]</div>
<div>Mar 07 13:28:23 hpctestmds1.our.domain crmd[11459]: notice: Result of stop operation for MGT on hpctestmds1: 0 (ok)</div>
<div><br>
</div>
</div>
<div>I then delete the resource, unmount the MGT, and make a new resource with the old <span style="font-size: 12pt;">ocf:heartbeat:Filesystem agent, setting the options to match the defaults from the <span>ocf:lustre:Lustre agent,</span> as follows:</span></div>
<p></p>
<p style="margin-top:0;margin-bottom:0"><span></span></p>
<p style="margin-top:0;margin-bottom:0"><span># pcs resource create MGT Filesystem device=/dev/disk/by-label/MGS directory=/mnt/MGT fstype="lustre" meta op monitor interval="20" timeout="300" op start interval="0" timeout="300" op stop interval="0" timeout="300";
pcs constraint location MGT prefers hpctestmds1=100</span><br>
</p>
<p style="margin-top:0;margin-bottom:0"><span><br>
</span></p>
<p style="margin-top:0;margin-bottom:0"><span>This results in a happier resource start. Pacemaker resource stays "Started" and mount persists. From journalctl:</span></p>
<p style="margin-top:0;margin-bottom:0"><span></p>
<div>
<div>Mar 07 13:35:07 hpctestmds1.our.domain crmd[11459]: notice: Result of probe operation for MGT on hpctestmds1: 7 (not running)</div>
<div>Mar 07 13:35:07 hpctestmds1.our.domain Filesystem(MGT)[744]: INFO: Running start for /dev/disk/by-label/MGS on /mnt/MGT</div>
<div>Mar 07 13:35:07 hpctestmds1.our.domain kernel: LDISKFS-fs (sde): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc</div>
<div>Mar 07 13:35:07 hpctestmds1.our.domain kernel: Lustre: MGS: Connection restored to 9eb39832-a281-1088-d816-410b918b5813 (at 0@lo)</div>
<div>Mar 07 13:35:07 hpctestmds1.our.domain crmd[11459]: notice: Result of start operation for MGT on hpctestmds1: 0 (ok)</div>
<div><br>
</div>
Has anyone experience similar results? Any tips?</div>
<div><br>
</div>
<div>Cheers</div>
<div>CanWood</div>
<div></div>
<div><br>
</div>
<br>
</span>
<p></p>
</div>
</body>
</html>