<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; color: rgb(0, 0, 0); font-size: 14px; font-family: Calibri, sans-serif;">
<div>I also seem to recall that make install and RPM use different install directories for things, and you could be getting old libraries from the RPMs but the new lfs.</div>
<div><br>
</div>
<div>I know tests go into either /usr/lib/lustre or /usr/lib64/lustre depending, and the .ko's go into different places as well.</div>
<div><br>
</div>
<div>-Ben</div>
<div><br>
</div>
<span id="OLK_SRC_BODY_SECTION">
<div style="font-family:Calibri; font-size:11pt; text-align:left; color:black; BORDER-BOTTOM: medium none; BORDER-LEFT: medium none; PADDING-BOTTOM: 0in; PADDING-LEFT: 0in; PADDING-RIGHT: 0in; BORDER-TOP: #b5c4df 1pt solid; BORDER-RIGHT: medium none; PADDING-TOP: 3pt">
<span style="font-weight:bold">From: </span>lustre-discuss <<a href="mailto:lustre-discuss-bounces@lists.lustre.org">lustre-discuss-bounces@lists.lustre.org</a>> on behalf of "Dilger, Andreas" <<a href="mailto:andreas.dilger@intel.com">andreas.dilger@intel.com</a>><br>
<span style="font-weight:bold">Date: </span>Saturday, March 24, 2018 at 6:02 PM<br>
<span style="font-weight:bold">To: </span>Patrick Farrell <<a href="mailto:paf@cray.com">paf@cray.com</a>><br>
<span style="font-weight:bold">Cc: </span>"<a href="mailto:lustre-discuss@lists.lustre.org">lustre-discuss@lists.lustre.org</a>" <<a href="mailto:lustre-discuss@lists.lustre.org">lustre-discuss@lists.lustre.org</a>><br>
<span style="font-weight:bold">Subject: </span>Re: [lustre-discuss] Static lfs?<br>
</div>
<div><br>
</div>
<div>
<div dir="auto">The "lfs" found in the build tree is a script wrapper generated by libtool to set up LD_LIBRARY_PATH and call the actual binary, which is in lustre/utils/.libs/lt-lfs or something similar. I'm not much for libtool, but I figured out this much
a few weeks ago when I wanted to run strace on lfs.
<div><br>
</div>
<div>This is needed because the lfs binary is dynamically linked to libraries in the build tree, which are needed for it to run directly from the build tree, but would not be found by "ld" in their current locations otherwise.</div>
<div><br>
</div>
<div>If you run "ldd" on the real binary, it will tell you which libraries are needed. You can copy the requisite paths to the new system library directories to be able to run the actual binary there, and just ignore the lfs libtool wrapper script. <br>
<br>
<div id="AppleMailSignature">Cheers, Andreas</div>
<div><br>
On Mar 23, 2018, at 15:59, Patrick Farrell <<a href="mailto:paf@cray.com">paf@cray.com</a>> wrote:<br>
<br>
</div>
<blockquote type="cite">
<div>
<div id="divtagdefaultwrapper" style="font-size:12pt;color:#000000;font-family:Calibri,Helvetica,sans-serif;" dir="ltr">
<p style="margin-top:0;margin-bottom:0">Another off list note pointing out that lfs is likely a script now. So here's the bitter end:</p>
<p style="margin-top:0;margin-bottom:0"><br>
</p>
<p style="margin-top:0;margin-bottom:0"></p>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 16px; margin-top: 0px; margin-bottom: 0px;">
Ah, it looks like you're correct. There's still an lfs.c but it no longer generates the "lfs" executable as it previously - Instead there's a lengthy and complex script named "lfs" which is not invoked by "make", but only during the install process. That
generates the lfs binary that is actually installed...</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 16px; margin-top: 0px; margin-bottom: 0px;">
<br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 16px; margin-top: 0px; margin-bottom: 0px;">
Uck. Well, I found where it squirrels away the real binary when executed.</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 16px; margin-top: 0px; margin-bottom: 0px;">
<br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 16px; margin-top: 0px; margin-bottom: 0px;">
Run the script lustre/utils/lfs in your build dir, and it will start lfs. Quit it, and you will find the actual lfs binary in lustre/utils/.libs/lt-lfs</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 16px; margin-top: 0px; margin-bottom: 0px;">
<br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 16px; margin-top: 0px; margin-bottom: 0px;">
Maybe this particular bit of build tooling would be clearer if it didn't try to pretend it didn't exist by apeing the binary without actually being it?</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 16px; margin-top: 0px; margin-bottom: 0px;">
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 16px; margin-top: 0px; margin-bottom: 0px;">
<br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 16px; margin-top: 0px; margin-bottom: 0px;">
Thanks to John Bauer for help with this.</div>
<br>
<p></p>
</div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri,sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> lustre-discuss <<a href="mailto:lustre-discuss-bounces@lists.lustre.org">lustre-discuss-bounces@lists.lustre.org</a>> on behalf of Patrick
Farrell <<a href="mailto:paf@cray.com">paf@cray.com</a>><br>
<b>Sent:</b> Friday, March 23, 2018 3:17:14 PM<br>
<b>To:</b> <a href="mailto:lustre-discuss@lists.lustre.org">lustre-discuss@lists.lustre.org</a><br>
<b>Subject:</b> Re: [lustre-discuss] Static lfs?</font>
<div> </div>
</div>
<style>
<!--
@font-face
{font-family:"Cambria Math"}
@font-face
{font-family:Calibri}
p.x_MsoNormal, li.x_MsoNormal, div.x_MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Calibri",sans-serif}
a:x_link, span.x_MsoHyperlink
{color:#0563C1;
text-decoration:underline}
a:x_visited, span.x_MsoHyperlinkFollowed
{color:#954F72;
text-decoration:underline}
p.x_msonormal0, li.x_msonormal0, div.x_msonormal0
{margin-right:0in;
margin-left:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif}
span.x_EmailStyle18
{font-family:"Calibri",sans-serif;
color:windowtext}
span.x_EmailStyle19
{font-family:"Calibri",sans-serif;
color:windowtext}
.x_MsoChpDefault
{font-size:10.0pt}
@page WordSection1
{margin:1.0in 1.0in 1.0in 1.0in}
div.x_WordSection1
{}
-->
</style>
<div lang="EN-US" link="#0563C1" vlink="#954F72">
<div class="x_WordSection1">
<p class="x_MsoNormal"><span style="font-size:11.0pt">Ah, interesting – I got a question off list about this, but I thought I’d reply here.</span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt"> </span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt">‘ldd’ on the lfs binary says “not a dynamic executable”.</span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt"> </span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt">So it seems I’m confused (never was much for compilers and linkers). Here are the errors I get trying to run it on another node:<br>
./lfs: line 202: cd: /home/build/paf/[……..]/lustre/utils: No such file or directory</span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt">gcc: error: lfs.o: No such file or directory</span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt">gcc: error: lfs_project.o: No such file or directory</span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt">gcc: error: ./.libs/liblustreapi.so: No such file or directory</span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt">gcc: error: ../../lnet/utils/lnetconfig/.libs/liblnetconfig.so</span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt"> </span></p>
<div style="border:none; border-top:solid #B5C4DF 1.0pt; padding:3.0pt 0in 0in 0in">
<p class="x_MsoNormal"><b><span style="color:black">From: </span></b><span style="color:black">lustre-discuss <<a href="mailto:lustre-discuss-bounces@lists.lustre.org">lustre-discuss-bounces@lists.lustre.org</a>> on behalf of Patrick Farrell <<a href="mailto:paf@cray.com">paf@cray.com</a>><br>
<b>Date: </b>Friday, March 23, 2018 at 3:03 PM<br>
<b>To: </b>"<a href="mailto:lustre-discuss@lists.lustre.org">lustre-discuss@lists.lustre.org</a>" <<a href="mailto:lustre-discuss@lists.lustre.org">lustre-discuss@lists.lustre.org</a>><br>
<b>Subject: </b>[lustre-discuss] Static lfs?</span></p>
</div>
<div>
<p class="x_MsoNormal"><span style="font-size:11.0pt"> </span></p>
</div>
<p class="x_MsoNormal"><a name="x__MailOriginalBody"><span style="font-size:11.0pt; color:black">Good afternoon,</span></a></p>
<p class="x_MsoNormal"><span style=""><span style="font-size:11.0pt; color:black"> </span></span></p>
<p class="x_MsoNormal"><span style=""><span style="font-size:11.0pt; color:black">I’ve got a developer question that perhaps someone has some insight on. After some recent (a few months ago now) changes to make the Lustre libraries and utilities build dynamically
linked rather than statically linked, I’ve got a problem. If I build an lfs binary just by doing “make”, the resultant binary looks for various libraries in the build directories and cannot be run on any system other than the one it was built on (well, I
guess without replicating the build directory structure). When doing make rpms and installing the RPMs, it works fine. The problem is “make rpms” takes ~5 minutes, as opposed to ~1 second for “make” in /utils. (I assume “make install” works too, but I explicitly
need to test on nodes other than the one where I’m doing the build, so that’s not an option.)</span></span></p>
<p class="x_MsoNormal"><span style=""><span style="font-size:11.0pt; color:black"> </span></span></p>
<p class="x_MsoNormal"><span style=""><span style="font-size:11.0pt; color:black">Does anyone have any insight on a way around this for a developer? Either some tweak I can make locally to get static builds again, or some fix to make that would let the dynamically
linked binary from “make” have correct library paths? (To be completely clear: The dynamically linked binary from “make” looks for libraries in the locations where they are built, regardless of whether or not they’re already installed in the normal system
library locations.)</span></span></p>
<p class="x_MsoNormal"><span style=""><span style="font-size:11.0pt; color:black"> </span></span></p>
<p class="x_MsoNormal"><span style=""><span style="font-size:11.0pt; color:black">Regards,</span></span></p>
<p class="x_MsoNormal"><span style=""><span style="font-size:11.0pt; color:black">Patrick Farrell</span></span></p>
<p class="x_MsoNormal"><span style=""><span style="font-size:11.0pt"> </span></span></p>
</div>
</div>
</div>
</blockquote>
<blockquote type="cite">
<div><span>_______________________________________________</span><br>
<span>lustre-discuss mailing list</span><br>
<span><a href="mailto:lustre-discuss@lists.lustre.org">lustre-discuss@lists.lustre.org</a></span><br>
<span><a href="http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org">http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org</a></span><br>
</div>
</blockquote>
</div>
</div>
</div>
</span>
</body>
</html>