The problem disappeared after I disable SELinux with the following steps, it works very well. Maybe we need to mention disabling SELinux and probably creating extra firewall policy to enable Lustre inter-node communication in the setup guide. <div>
<br><div>Edit <span class="Apple-style-span" style="color: rgb(34, 34, 34); font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 12px; line-height: 16px; "><font class="inlinecode" style="font-family: monospace; font-size: 1em; background-color: rgb(238, 238, 238); color: rgb(0, 0, 0); overflow-x: auto; overflow-y: auto; ">/etc/selinux/config</font> and make sure the following line is </span></div>
<div><span class="Apple-style-span" style="font-family: monospace; font-size: 12px; line-height: 16px; white-space: pre; "><b>SELINUX=disabled</b></span></div><div><span class="Apple-style-span" style="font-family: monospace; font-size: 12px; line-height: 16px; white-space: pre; "><b>Reboot host. (this is a must, the test environment must have SELINUX disabled, </b></span><span class="Apple-style-span" style="font-family: monospace; font-size: 12px; line-height: 16px; white-space: pre; ">echo 0 >/selinux/enforce won't work)</span></div>
<div><span class="Apple-style-span" style="font-family: monospace; font-size: 12px; line-height: 16px; white-space: pre; "><b><br></b></span></div><div>Hope this helps, and thanks for all of your help. </div><div><br></div>
<div>Regards,</div><div>-Jon.<br><br>
<br><br><div class="gmail_quote">On Thu, Jun 23, 2011 at 8:26 AM, Jon Zhu <span dir="ltr"><<a href="mailto:jon.zhu@gmail.com">jon.zhu@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
Thanks, Peter. <div>The procedure has a step to create initrd:</div><div><span style="color:rgb(51, 51, 51);font-family:Helvetica, Arial, sans-serif;font-size:13px;line-height:17px"><pre style="padding-top:0px;padding-right:0px;padding-bottom:0px;padding-left:0px;margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;text-align:left;overflow-x:auto;overflow-y:auto;font-family:'Courier New', Courier, monospace;line-height:1.3">
# /sbin/<span style="color:rgb(0, 0, 145);background-color:inherit">new</span>-kernel-pkg --<span style="color:rgb(0, 0, 145);background-color:inherit">package</span> kernel --mkinitrd --dracut --depmod --install 2.6.32lustre21</pre>
</span></div><div><br></div><div>It doesn't work so I used this command instead:</div><div>mkinitrd /boot/initramfs-2.6.32.lustre21.img 2.6.32.lustre21<br><br>Looking forward to get the new procedure soon. </div><div>
<br></div><div>Regards,</div><div>-Jon.</div><div><div></div><div class="h5"><div><br><div class="gmail_quote">On Thu, Jun 23, 2011 at 1:02 AM, Peter Jones <span dir="ltr"><<a href="mailto:pjones@whamcloud.com" target="_blank">pjones@whamcloud.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><u></u>
<div text="#000000" bgcolor="#ffffff">
Jon<br>
<br>
It seems that the instructions (written at the end of May) have
grown stale since the recent realignment on RHEL6.1. We'll get the
instructions updated.<br>
<br>
Peter<div><div></div><div><br>
<br>
On 11-06-22 6:03 PM, Jon Zhu wrote:
</div></div><blockquote type="cite"><div><div></div><div>Hi, all
<div><br>
</div>
<div>As a fan of Lustre file system, recently I got a chance to
try to build Lustre whamcloud code v2.1 branch from scratch, the
following build guide works pretty well:</div>
<div><a href="http://wiki.whamcloud.com/display/PUB/Walk-thru-+Build+Lustre+2.1+on+RHEL6+from+Whamcloud+git" target="_blank">http://wiki.whamcloud.com/display/PUB/Walk-thru-+Build+Lustre+2.1+on+RHEL6+from+Whamcloud+git</a></div>
<div><br>
</div>
<div>Everything works well until the last step, when I
run /usr/lib64/lustre/tests/llmount.sh, it failed with the
following error: </div>
<div>
<div>#############################################################################################</div>
<div>[root@ip-10-204-59-145 tests]#
/usr/lib64/lustre/tests/llmount.sh</div>
<div>Stopping clients: ip-10-204-59-145 /mnt/lustre (opts:)</div>
<div>Stopping clients: ip-10-204-59-145 /mnt/lustre2 (opts:)</div>
<div>Loading modules from /usr/lib64/lustre/tests/..</div>
<div>debug=0x33f0404</div>
<div>subsystem_debug=0xffb7e3ff</div>
<div>gss/krb5 is not supported</div>
<div>WARNING: Deprecated config file /etc/modprobe.conf, all
config files belong into /etc/modprobe.d/.</div>
<div>WARNING: Deprecated config file /etc/modprobe.conf, all
config files belong into /etc/modprobe.d/.</div>
<div>WARNING: Deprecated config file /etc/modprobe.conf, all
config files belong into /etc/modprobe.d/.</div>
<div>WARNING: Deprecated config file /etc/modprobe.conf, all
config files belong into /etc/modprobe.d/.</div>
<div>
Formatting mgs, mds, osts</div>
<div>Format mds1: /tmp/lustre-mdt1</div>
<div>Format ost1: /tmp/lustre-ost1</div>
<div>Format ost2: /tmp/lustre-ost2</div>
<div>Checking servers environments</div>
<div>Checking clients ip-10-204-59-145 environments</div>
<div>Loading modules from /usr/lib64/lustre/tests/..</div>
<div>debug=0x33f0404</div>
<div>subsystem_debug=0xffb7e3ff</div>
<div>gss/krb5 is not supported</div>
<div>Setup mgs, mdt, osts</div>
<div>Starting mds1: -o loop,user_xattr,acl /tmp/lustre-mdt1
/mnt/mds1</div>
<div>mount.lustre: mount /dev/loop0 at /mnt/mds1 failed:
Operation not supported</div>
<div>#############################################################################################</div>
</div>
<div><br>
</div>
<div>Can anyone shed some light on this issue? Does this mean
current Whamcloud Lustre code base (<span style="font-family:'Courier New',Courier,monospace;font-size:13px;line-height:16px;white-space:pre-wrap"><font color="#333333">git:</font><span style="color:rgb(128, 128, 128);background-color:inherit">//<a href="http://git.whamcloud.com/fs/lustre-release.git" target="_blank">git.whamcloud.com/fs/lustre-release.git</a>)
</span><span style="background-color:inherit">is not ready for test drive? </span></span></div>
<div><br>
</div>
<div>Detailed system log messages is attached for your
information, the test machine's info is also dumped as
following: </div>
<div><br>
</div>
<div>
<div>[root@ip-10-204-59-145 tests]# uname -a</div>
<div>
Linux ip-10-204-59-145 2.6.32.lustre21 #6 SMP Wed Jun 22
15:27:40 EDT 2011 x86_64 x86_64 x86_64 GNU/Linux</div>
</div>
<div><br>
</div>
<div>
<div>[root@ip-10-204-59-145 tests]# cat /etc/redhat-release</div>
<div>Red Hat Enterprise Linux Server release 6.1 (Santiago)</div>
</div>
<div><br>
</div>
<div>BTW, thanks to the Whamcloud team for contributing so much to
Lustre, without your effort Lustre is almost dead.</div>
<div><br>
</div>
<div>Regards,</div>
<div>-Jon.</div>
<div><a href="mailto:jon.zhu@gmail.com" target="_blank">jon.zhu@gmail.com</a></div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
</div></div><pre><fieldset></fieldset>
_______________________________________________
Lustre-discuss mailing list
<a href="mailto:Lustre-discuss@lists.lustre.org" target="_blank">Lustre-discuss@lists.lustre.org</a>
<a href="http://lists.lustre.org/mailman/listinfo/lustre-discuss" target="_blank">http://lists.lustre.org/mailman/listinfo/lustre-discuss</a>
</pre>
</blockquote>
<br>
<pre cols="72">--
Peter Jones
Whamcloud, Inc.
<a href="http://www.whamcloud.com" target="_blank">www.whamcloud.com</a>
</pre>
</div>
<br>_______________________________________________<br>
Lustre-discuss mailing list<br>
<a href="mailto:Lustre-discuss@lists.lustre.org" target="_blank">Lustre-discuss@lists.lustre.org</a><br>
<a href="http://lists.lustre.org/mailman/listinfo/lustre-discuss" target="_blank">http://lists.lustre.org/mailman/listinfo/lustre-discuss</a><br>
<br></blockquote></div><br></div>
</div></div></blockquote></div><br></div></div>