<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=ISO-8859-1" http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
Brian (and company),<br>
<br>
Thanks for taking an interest in this. I appreciate that you guys have
a lot on your plate, so I'm grateful for any feedback you can provide.
As requested, I have attached a transcript of the RPM installation
process as used against a completely clean RHEL 4.5 AS installation on
an X4200m2 server with PCIe Infiniband IB HCA.<br>
<br>
Taking your advice on board regarding the OFED kernel modules, I am
going to try creating the file system using only that material supplied
as part of the Lustre download plus the RDAC kernel modules for the
STK6140.<br>
<br>
Regards,<br>
<br>
Malcolm.<br>
<br>
<br>
Brian J. Murrell wrote:
<blockquote cite="mid:1223305173.1463.38.camel@pc.ilinx" type="cite">
<pre wrap="">On Mon, 2008-10-06 at 15:47 +0100, Malcolm Cowe wrote:
</pre>
<blockquote type="cite">
<pre wrap="">Hey Brian,
</pre>
</blockquote>
<pre wrap=""><!---->
Hey Malcolm,
</pre>
<blockquote type="cite">
<pre wrap="">I'll have to re-install the system from scratch in order to be able to
answer some of your questions, which I'll get started on this evening.
</pre>
</blockquote>
<pre wrap=""><!---->
OK.
</pre>
<blockquote type="cite">
<pre wrap="">What I was hoping for in the first instance was a sanity check of our
installation methods.
</pre>
</blockquote>
<pre wrap=""><!---->
I think I commented on those. If you are going to build your OFED stack
you don't need to install the one we provide.
</pre>
<blockquote type="cite">
<pre wrap="">With respect to the OFED stack used, we are using the latest official
software stack supplied by Voltaire. The reason for this is that there
is more to OFED than just the kernel modules, including many libraries
and tools,
</pre>
</blockquote>
<pre wrap=""><!---->
None of these should be necessary for Lustre to use I/B.
</pre>
<blockquote type="cite">
<pre wrap="">plus the latest firmware for the cards.
</pre>
</blockquote>
<pre wrap=""><!---->
Hrm. Can you not upgrade firmware independent of upgrading the whole
OFED stack? That seems very limiting.
</pre>
<blockquote type="cite">
<pre wrap="">It's what the customer has asked for, and it is what the card vendor
expects us to do.
</pre>
</blockquote>
<pre wrap=""><!---->
Fair enough. I was just pointing out that you don't need our OFED stack
if you are going to install your own.
</pre>
<blockquote type="cite">
<pre wrap="">We may be able to get away with OFED 1.3, but I would still like some
guidance on how to install the rest of the OFED stack
</pre>
</blockquote>
<pre wrap=""><!---->
We don't supply the userspace tools because they are not really
necessary for Lustre.
</pre>
<blockquote type="cite">
<pre wrap="">do we use the OFED source to rebuild everything, or can we pick the
Lustre supplied kernel modules and just layer on the other stuff
separately?
</pre>
</blockquote>
<pre wrap=""><!---->
Yes, you should be able to do that. I say that quite generally as I'm
not entirely clear on your operating environment.
</pre>
<blockquote type="cite">
<pre wrap="">Finally, when I said that one file system fails versus another passes,
I mean that the server locks solid, crashes, usually with no debug to
speak of (nothing in the system logs).
</pre>
</blockquote>
<pre wrap=""><!---->
Nothing on the console either?
</pre>
<blockquote type="cite">
<pre wrap="">Even while the system is up and running the lustre kernel, if we
attempt a clean shutdown, the kernel panics.
</pre>
</blockquote>
<pre wrap=""><!---->
Hrm. A panic is quite different than locking solid with no messages at
all. A solid lock with no messages is indicative of hardware problems.
</pre>
<blockquote type="cite">
<pre wrap="">Since I need to rebuild the systems anyway, I will also try to install
the packages in the order mentioned by Megan Larko, to see how that
affects the installation.
</pre>
</blockquote>
<pre wrap=""><!---->
I'm not entirely convinced of her process. You should not need to use
--force and reinstall packages already installed. I'd be more
interested in knowing exactly your installation steps and the errors you
get from it. Please try to avoid the use of --force so we can see why
it's necessary. You will have to use "rpm -U" with e2fsprogs though as
she mentions. Do all of your work with the "script(1)" tool so you can
easily log it.
b.
</pre>
<pre wrap="">
<hr size="4" width="90%">
_______________________________________________
Lustre-discuss mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Lustre-discuss@lists.lustre.org">Lustre-discuss@lists.lustre.org</a>
<a class="moz-txt-link-freetext" href="http://lists.lustre.org/mailman/listinfo/lustre-discuss">http://lists.lustre.org/mailman/listinfo/lustre-discuss</a>
</pre>
</blockquote>
<br>
<div class="moz-signature">-- <br>
<table border="0" cellpadding="0" cellspacing="0" width="450">
<tbody>
<tr valign="top">
<td height="121" width="121"><a href="http://www.sun.com"><img
moz-do-not-send="false" src="cid:part1.07080407.07080806@sun.com"
border="0" height="116" width="121"></a></td>
<td style="font-family: Arial; font-size: 11px;" height="121"
width="329"><br>
<b>Malcolm Cowe</b><br>
<em>Solutions Integration Engineer</em><br>
<br>
<b>Sun Microsystems, Inc.</b><br>
Blackness Road<br>
Linlithgow, West Lothian EH49 7LR UK<br>
Phone: x73602 / +44 1506 673 602<br>
Email: <a class="moz-txt-link-abbreviated" href="mailto:Malcolm.Cowe@Sun.COM">Malcolm.Cowe@Sun.COM</a><br>
</td>
</tr>
</tbody>
</table>
</div>
</body>
</html>