<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
  </head>
  <body text="#000000" bgcolor="#FFFFFF">
    <p>Colin, <br>
    </p>
    <p>This worked before, so I don't think that 2.8 itself is the
      problem. I can't just take down my clusters and upgrade a critical
      piece of my infrastructure without solid justification that it's
      necessary to fix this problem. <br>
    </p>
    <pre class="moz-signature" cols="72">Prentice </pre>
    <div class="moz-cite-prefix">On 02/15/2018 05:00 PM, Colin Faber
      wrote:<br>
    </div>
    <blockquote type="cite"
cite="mid:CAJcXmBnegWUm6fMv-3hmyUK_pCobyP-hbf_MXBonpD5qBNHPQg@mail.gmail.com">
      <div dir="auto">If the mount on the users clients had the various
        options enabled, and those aren't present in fstab, you'd end up
        with such behavior. Also 2.8? Can you upgrade to 2.10 LTS??
        <div dir="auto"><br>
        </div>
        <div dir="auto"><br>
        </div>
      </div>
      <div class="gmail_extra"><br>
        <div class="gmail_quote">On Feb 15, 2018 1:06 PM, "Prentice
          Bisbal" <<a href="mailto:pbisbal@pppl.gov"
            moz-do-not-send="true">pbisbal@pppl.gov</a>> wrote:<br
            type="attribution">
          <blockquote class="gmail_quote" style="margin:0 0 0
            .8ex;border-left:1px #ccc solid;padding-left:1ex">
            <div text="#000000" bgcolor="#FFFFFF">
              <p>No. Several others have asked me the same thing, so
                that seems like it might be the issue. The only problem
                with that solution is that the user claimed his program
                worked just fine up until a couple of weeks ago, so if
                that is the issue, I'll still be scratching my head
                trying to figure out how/what changed</p>
              <p><br>
              </p>
              <pre class="m_-1995677135630938282moz-signature" cols="72">Prentice </pre>
              <div class="m_-1995677135630938282moz-cite-prefix">On
                02/15/2018 12:31 PM, Alexander I Kulyavtsev wrote:<br>
              </div>
              <blockquote type="cite">
                <div>Do you have <b>flock</b> option in fstab for
                  lustre mount or in command you use to mount lustre on
                  client?</div>
                <div><br>
                </div>
                <div>Search for flock on lustre wiki</div>
                <div><span class="m_-1995677135630938282Apple-tab-span" style="white-space:pre-wrap"></span><a
href="http://wiki.lustre.org/Mounting_a_Lustre_File_System_on_Client_Nodes"
                    target="_blank" moz-do-not-send="true">http://wiki.lustre.org/<wbr>Mounting_a_Lustre_File_System_<wbr>on_Client_Nodes</a></div>
                <div>or lustre manual</div>
                <div><span class="m_-1995677135630938282Apple-tab-span" style="white-space:pre-wrap"></span><a
                    href="http://doc.lustre.org/lustre_manual.pdf"
                    target="_blank" moz-do-not-send="true">http://doc.lustre.org/lustre_<wbr>manual.pdf</a></div>
                <div><br>
                </div>
                <div>Here are links where to start learning about
                  lustre:</div>
                <div>* <a
                    href="http://lustre.org/getting-started-with-lustre/"
                    target="_blank" moz-do-not-send="true">
                    http://lustre.org/getting-<wbr>started-with-lustre/</a></div>
                <div>* <a href="http://wiki.lustre.org" target="_blank"
                    moz-do-not-send="true">http://wiki.lustre.org</a></div>
                <div>* <a href="https://wiki.hpdd.intel.com"
                    target="_blank" moz-do-not-send="true">https://wiki.hpdd.intel.com</a></div>
                <div>* <a href="http://jira.hpdd.intel.com"
                    target="_blank" moz-do-not-send="true">jira.hpdd.intel.com</a></div>
                <div>* <a href="http://opensfs.org/lustre/"
                    target="_blank" moz-do-not-send="true">http://opensfs.org/lustre/</a></div>
                <div><br>
                </div>
                <div>Alex.</div>
                <br>
                <div>
                  <blockquote type="cite">
                    <div>On Feb 15, 2018, at 11:02 AM, Prentice Bisbal
                      <<a href="mailto:pbisbal@pppl.gov"
                        target="_blank" moz-do-not-send="true">pbisbal@pppl.gov</a>>
                      wrote:</div>
                    <br
                      class="m_-1995677135630938282Apple-interchange-newline">
                    <div>
                      <div>Hi.<br>
                        <br>
                        I'm an experience HPC system admin, but I know
                        almost nothing about Lustre administration. The
                        system admin who administered our small Lustre
                        filesystem recently retired, and no one has
                        filled that gap yet. A user recently reported
                        they are now getting file-locking errors from a
                        program they've run repeatedly on Lustre in the
                        past. When the run the same program on an NFS
                        filesystem, the error goes away. I've
                        cut-and-pasted the error messages below.<br>
                        <br>
                        Since I have real experience as a Lustre admin,
                        I turned to google, and it looks like it might
                        be that the file-locking daemon died (if Lustre
                        has a separate file-lock daemon), or somehow
                        file-locking was recently disabled. If that is
                        possible, how do I check this, and restart or
                        re-enable if necessary?  I skimmed the user
                        manual, and could not find anything on either of
                        these issues.<br>
                        <br>
                        Any and all help will be greatly appreciated.<br>
                        <br>
                        Some of the error messages:<br>
                        <br>
                        HDF5-DIAG: Error detected in HDF5
                        (1.10.0-patch1) MPI-process 9:<br>
                          #000: H5F.c line 579 in H5Fopen(): unable to
                        open file<br>
                            major: File accessibilty<br>
                            minor: Unable to open file<br>
                          #001: H5Fint.c line 1168 in H5F_open(): unable
                        to lock the file or initialize file structure<br>
                            major: File accessibilty<br>
                            minor: Unable to open file<br>
                          #002: H5FD.c line 1821 in H5FD_lock(): driver
                        lock request failed<br>
                            major: Virtual File Layer<br>
                            minor: Can't update object<br>
                          #003: H5FDsec2.c line 939 in H5FD_sec2_lock():
                        unable to flock file, errno = 38, error message
                        = 'Function not implemented'<br>
                            major: File accessibilty<br>
                            minor: Bad file ID accessed<br>
                        Error: couldn't open file HDF5-DIAG: Error
                        detected in HDF5 (1.10.0-patch1) MPI-process 13:<br>
                          #000: H5F.c line 579 in H5Fopen(): unable to
                        open file<br>
                            major: File accessibilty<br>
                            minor: Unable to open file<br>
                          #001: H5Fint.c line 1168 in H5F_open(): unable
                        to lock the file or initialize file structure<br>
                            major: File accessibilty<br>
                            minor: Unable to open file<br>
                          #002: H5FD.c line 1821 in H5FD_lock(): driver
                        lock request failed<br>
                            major: Virtual File Layer<br>
                            minor: Can't update object<br>
                          #003: H5FDsec2.c line 939 in H5FD_sec2_lock():
                        unable to flock file, errno = 38, error message
                        = 'Function not implemented'<br>
                            major: File accessibilty<br>
                            minor: Bad file ID accessed<br>
                        <br>
                        -- <br>
                        Prentice<br>
                        <br>
                        ______________________________<wbr>_________________<br>
                        lustre-discuss mailing list<br>
                        <a href="mailto:lustre-discuss@lists.lustre.org"
                          target="_blank" moz-do-not-send="true">lustre-discuss@lists.lustre.<wbr>org</a><br>
                        <a
                          class="m_-1995677135630938282moz-txt-link-freetext"
href="http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org"
                          target="_blank" moz-do-not-send="true">http://lists.lustre.org/<wbr>listinfo.cgi/lustre-discuss-<wbr>lustre.org</a><br>
                      </div>
                    </div>
                  </blockquote>
                </div>
                <br>
              </blockquote>
              <br>
            </div>
            <br>
            ______________________________<wbr>_________________<br>
            lustre-discuss mailing list<br>
            <a href="mailto:lustre-discuss@lists.lustre.org"
              moz-do-not-send="true">lustre-discuss@lists.lustre.<wbr>org</a><br>
            <a
              href="http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org"
              rel="noreferrer" target="_blank" moz-do-not-send="true">http://lists.lustre.org/<wbr>listinfo.cgi/lustre-discuss-<wbr>lustre.org</a><br>
            <br>
          </blockquote>
        </div>
      </div>
    </blockquote>
    <br>
  </body>
</html>