<html><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><div><div>Aha! I think I've found the problem. It looks like the filesystem name on the OSTs is different than the MDT you are trying to use. On the MDT you have datafs and on the OSTs it looks like you have spfs. I'm guessing you wan the name "datafs" so you need to use tunefs.lustre to change the fs name on the OSTs. Here's how--</div><div><br class="webkit-block-placeholder"></div><div>1. Unmount the OSTs</div><div>2. Run "tunefs.lustre --fsname="datafs" /device/of/ost" for each of your OSTs</div><div>3. Mount everything back up and post the output of "lctl dl" on your OSSs and MDSs.</div><div><br class="webkit-block-placeholder"></div></div><div>-Aaron</div><div><br class="webkit-block-placeholder"></div><div>ps "lctl dl" is a quicker way of running cat /proc/fs/lustre/devices</div><br><div><div>On Dec 30, 2007, at 4:00 AM, Albert Ozilov wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"> <div style="WORD-WRAP: break-word; webkit-nbsp-mode: space; webkit-line-break: after-white-space"> <div><font face="Arial" size="2"><span class="734002608-30122007">Hi Aaron,</span></font></div> <div><font face="Arial" size="2"><span class="734002608-30122007"></span></font> </div> <div><font face="Arial" size="2"><span class="734002608-30122007">Thanks in advance for your help.</span></font></div> <div><font face="Arial" size="2"><span class="734002608-30122007"></span></font> </div> <div><font><span class="734002608-30122007"><font face="Arial" size="2">Here is some Info from my setup:</font></span></font></div><font> <div><font face="Arial" color="#0000ff" size="2"></font><br><font face="Arial" size="2"></font></div></font> <div><span class="734002608-30122007"><font face="Arial" size="2">MDT/MGS server IP is 11.4.3.241</font></span></div> <div><span class="734002608-30122007"><font face="Arial" size="2">Client IP is 11.4.3.242.</font></span></div> <div><span class="734002608-30122007"><font face="Arial" size="2">OST's IP's are 11.4.3.243 and 11.4.3.244.</font></span></div> <div><span class="734002608-30122007"><font face="Arial" size="2">===========================================</font></span></div> <div><font size="2"><font face="Arial"><span class="734002608-30122007"></span><span class="734002608-30122007"></span><span class="734002608-30122007"></span></font></font> </div> <div><span class="734002608-30122007"><font face="Arial" size="2">[root@sw241 ~]# cat /proc/fs/lustre/mds/datafs-MDT0000/recovery_status</font></span></div> <div><span class="734002608-30122007"><font face="Arial" size="2">status: INACTIVE</font></span></div> <div><span class="734002608-30122007"><font face="Arial" size="2">===========================================</font></span></div> <div><span class="734002608-30122007"><font face="Arial" size="2"></font></span> </div> <div><font face="Arial" size="2"></font><span class="734002608-30122007"><font face="Arial" size="2"><strong><u>cat /proc/fs/lustre/devices:</u></strong></font></span></div> <div><span class="734002608-30122007"><font face="Arial" size="2"></font> </span></div> <div><font face="Arial" size="2">[root@sw241 ~]# cat /proc/fs/lustre/devices</font></div> <div><font face="Arial" size="2">0 UP mgs MGS MGS 9</font></div> <div><font face="Arial" size="2">1 UP mgc MGC11.4.3.241@o2ib a4bcf80c-5a47-b79d-326a-6520a237ce4a 5</font></div> <div><font face="Arial" size="2">2 UP mdt MDS MDS_uuid 3</font></div> <div><font face="Arial" size="2">3 UP lov datafs-mdtlov datafs-mdtlov_UUID 4</font></div> <div><font face="Arial" size="2">4 UP mds datafs-MDT0000 datafs-MDT0000_UUID 3</font></div> <div><font face="Arial" size="2">[root@sw241 ~]# </font></div> <div> <div><span class="734002608-30122007"><font face="Arial" size="2">===========================================</font></span></div></div> <div><font face="Arial" size="2">[root@sw243 ~]# cat /proc/fs/lustre/devices</font></div> <div><font face="Arial" size="2">0 UP mgc MGC11.4.3.241@o2ib c44415f9-2bd0-420f-99a3-8792c190d45e 5</font></div> <div><font face="Arial" size="2">1 UP ost OSS OSS_uuid 3</font></div> <div><font face="Arial" size="2">2 UP obdfilter spfs-OST0000 spfs-OST0000_UUID 3</font></div> <div><font face="Arial" size="2">[root@sw243 ~]# </font></div> <div> <div><span class="734002608-30122007"><font face="Arial" size="2">===========================================</font></span></div> <div><font face="Arial" size="2">[root@sw244 ~]# cat /proc/fs/lustre/devices</font></div></div> <div><font face="Arial" size="2">0 UP mgc MGC11.4.3.241@o2ib d4639175-68e7-9482-2edd-e5ee29e3af2a 5</font></div> <div><font face="Arial" size="2">1 UP ost OSS OSS_uuid 3</font></div> <div><font face="Arial" size="2">2 UP obdfilter spfs-OST0001 spfs-OST0001_UUID 3</font></div> <div><font face="Arial" size="2">[root@sw244 ~]# </font></div> <div> <div><span class="734002608-30122007"><font face="Arial" size="2">===========================================</font></span></div></div> <div><font face="Arial" color="#0000ff" size="2"></font> </div> <div><strong><u><font face="Arial" size="2">lctl ping:</font></u></strong></div> <div><strong><u><font face="Arial" size="2"></font></u></strong> </div> <div><font face="Arial" size="2">[root@sw242 ~]# lctl ping </font><a href="mailto:11.4.3.241@o2ib"><font face="Arial" color="#000000" size="2">11.4.3.241@o2ib</font></a></div> <div><a href="mailto:12345-0@lo"><font face="Arial" color="#000000" size="2">12345-0@lo</font></a></div> <div><a href="mailto:12345-11.4.3.241@o2ib"><font face="Arial" color="#000000" size="2">12345-11.4.3.241@o2ib</font></a></div> <div><font face="Arial" size="2"></font> </div> <div><font face="Arial" size="2">[root@sw242 ~]# lctl ping </font><a href="mailto:11.4.3.243@o2ib"><font face="Arial" color="#000000" size="2">11.4.3.243@o2ib</font></a></div> <div><a href="mailto:12345-0@lo"><font face="Arial" color="#000000" size="2">12345-0@lo</font></a></div> <div><a href="mailto:12345-11.4.3.243@o2ib"><font face="Arial" color="#000000" size="2">12345-11.4.3.243@o2ib</font></a></div> <div><font face="Arial" color="#0000ff" size="2"></font> </div> <div><font face="Arial" size="2">[root@sw242 ~]# lctl ping </font><a href="mailto:11.4.3.244@o2ib"><font face="Arial" color="#000000" size="2">11.4.3.244@o2ib</font></a></div> <div><a href="mailto:12345-0@lo"><font face="Arial" color="#000000" size="2">12345-0@lo</font></a></div> <div><a href="mailto:12345-11.4.3.244@o2ib"><font face="Arial" color="#000000" size="2">12345-11.4.3.244@o2ib</font></a></div> <div> <div><span class="734002608-30122007"><font face="Arial" size="2">===========================================</font></span></div> <div><font face="Arial" size="2">[root@sw241 ~]# lctl ping </font><a href="mailto:11.4.3.242@o2ib"><font face="Arial" color="#000000" size="2">11.4.3.242@o2ib</font></a></div></div> <div><a href="mailto:12345-0@lo"><font face="Arial" color="#000000" size="2">12345-0@lo</font></a></div> <div><a href="mailto:12345-11.4.3.242@o2ib"><font face="Arial" color="#000000" size="2">12345-11.4.3.242@o2ib</font></a></div> <div><font face="Arial" size="2"></font> </div> <div><font face="Arial" size="2">[root@sw241 ~]# lctl ping </font><a href="mailto:11.4.3.243@o2ib"><font face="Arial" color="#000000" size="2">11.4.3.243@o2ib</font></a></div> <div><a href="mailto:12345-0@lo"><font face="Arial" color="#000000" size="2">12345-0@lo</font></a></div> <div><a href="mailto:12345-11.4.3.243@o2ib"><font face="Arial" color="#000000" size="2">12345-11.4.3.243@o2ib</font></a></div> <div><font face="Arial" size="2"></font> </div> <div><font face="Arial" size="2">[root@sw241 ~]# lctl ping </font><a href="mailto:11.4.3.244@o2ib"><font face="Arial" color="#000000" size="2">11.4.3.244@o2ib</font></a></div> <div><a href="mailto:12345-0@lo"><font face="Arial" color="#000000" size="2">12345-0@lo</font></a></div> <div><a href="mailto:12345-11.4.3.244@o2ib"><font face="Arial" color="#000000" size="2">12345-11.4.3.244@o2ib</font></a></div> <div><span class="734002608-30122007"><font face="Arial" size="2">===========================================</font></span></div> <div><span class="734002608-30122007"><font face="Arial" size="2"></font></span> </div> <div><font size="2"><font face="Arial"><span class="734002608-30122007">Y</span>our help<span class="734002608-30122007"> is appreciated.</span></font></font></div> <div><font face="Arial" size="2"></font> </div> <div dir="ltr" align="left"> <div dir="ltr" align="left"><font face="Arial" size="2"><span class="921194414-18122006">Best Regards, </span></font></div> <div><font face="Arial" size="2"><span class="921194414-18122006">Alberto. </span></font></div></div> <div><font face="Arial" size="2"></font> </div><br> <div class="OutlookMessageHeader" lang="en-us" dir="ltr" align="left"> <hr tabindex="-1"> <font face="Tahoma" size="2"><b>From:</b> Aaron Knister [<a href="mailto:aaron@iges.org">mailto:aaron@iges.org</a>] <br><b>Sent:</b> Sunday, December 30, 2007 4:05 AM<br><b>To:</b> Albert Ozilov<br><b>Cc:</b> <a href="mailto:lustre-discuss@clusterfs.com">lustre-discuss@clusterfs.com</a><br><b>Subject:</b> Re: [Lustre-discuss] Can someone help?<br></font><br></div> <div></div> <div>Hi!</div> <div><br class="webkit-block-placeholder"></div> <div>Could you post the IPs over your MDT/MGS, OSSs and Client?</div> <div><br class="webkit-block-placeholder"></div> <div>Also on the MDT/MGS I'm wondering if the MDT is in recovery. To check recovery status run--</div> <div><br class="webkit-block-placeholder"></div> <div>cat /proc/fs/lustre/mds/datafs-MDT0000/recovery_status </div> <div><br class="webkit-block-placeholder"></div> <div>and post what you get back. </div> <div><br class="webkit-block-placeholder"></div> <div>Also you can do an lnet ping to the various points in your system. The syntax is "lctl ping nid" so to ping 11.4.3.241@o2ib the syntax would be "lctl ping 11.4.3.241@o2ib". Also post what you get back from that.</div> <div><br class="webkit-block-placeholder"></div> <div>-Aaron</div><br> <div> <div>On Dec 29, 2007, at 7:29 PM, Albert Ozilov wrote:</div><br class="Apple-interchange-newline"> <blockquote type="cite"> <div> <div><font face="Arial" size="2"><span class="234134723-29122007">Hi,</span></font></div> <div><font face="Arial" size="2"><span class="234134723-29122007"></span></font> </div> <div><font face="Arial" size="2"><span class="234134723-29122007">I am new user with Lustre file system, I am trying to run over Infiniband fabric (ofed 1.2.xxx), I am using Lustre version 1.6.4.1</span></font></div> <div><font face="Arial" size="2"><span class="234134723-29122007">I have created mgs/mdt on the same node and two OST's on different nodes, all looks fine till now.</span></font></div> <div><font face="Arial" size="2"><span class="234134723-29122007">Now I am trying to mount the client node but getting connection refused from the MGS server, </span></font></div> <div><font face="Arial" size="2"><span class="234134723-29122007"><strong></strong></span></font> </div> <div><font face="Arial" size="2"><span class="234134723-29122007"><strong>This is my command on the client node:</strong></span></font></div> <div><font face="Arial" size="2"><span class="234134723-29122007">mount -t lustre <a href="mailto:11.4.3.241@o2ib:/datafs">11.4.3.241@o2ib:/datafs</a> /mnt/testfs</span></font></div> <div><font face="Arial" size="2"><span class="234134723-29122007"></span></font> </div> <div><font face="Arial" size="2"><span class="234134723-29122007"><strong>This is the dmesg on the MGS node:</strong></span></font></div> <div><font face="Arial" size="2"><span class="234134723-29122007"></span></font> </div> <div><font face="Arial" size="2">Lustre: datafs-MDT0000: temporarily refusing client connection from <a href="mailto:11.4.3.242@o2ib">11.4.3.242@o2ib</a><br>Lustre: Skipped 19 previous similar messages<br>LustreError: 6047:0:(ldlm_lib.c:1442:target_send_reply_msg()) @@@ processing error (-11) <a href="mailto:req@ffff810219855850">req@ffff810219855850</a> x6/t0 o38-><?>@<?>:-1 lens 240/0 ref 0 fl Interpret:/0/0 rc -11/0<br>LustreError: 6047:0:(ldlm_lib.c:1442:target_send_reply_msg()) Skipped 19 previous similar messages<br></font></div><font face="Arial" size="2"></font> <div><font face="Arial" size="2"><span class="234134723-29122007"><br>Is someone familiar with this problem, can someone help?</span></font></div> <div><font face="Arial" size="2"></font> </div> <div dir="ltr" align="left"><font face="Monotype Corsiva" size="6"> <div dir="ltr" align="left"><font face="Arial" size="2"><span class="921194414-18122006">Best Regards, </span></font></div> <div><font face="Arial" size="2"><span class="921194414-18122006">Alberto. </span></font></div></font></div> <div><font face="Arial" size="2"></font> </div></div>_______________________________________________<br>Lustre-discuss mailing list<br><a href="mailto:Lustre-discuss@clusterfs.com">Lustre-discuss@clusterfs.com</a><br><a href="https://mail.clusterfs.com/mailman/listinfo/lustre-discuss">https://mail.clusterfs.com/mailman/listinfo/lustre-discuss</a><br></blockquote></div><br> <div apple-content-edited="true"><span class="Apple-style-span" style="WORD-SPACING: 0px; FONT: 12px Helvetica; TEXT-TRANSFORM: none; COLOR: rgb(0,0,0); TEXT-INDENT: 0px; WHITE-SPACE: normal; LETTER-SPACING: normal; BORDER-COLLAPSE: separate; orphans: 2; widows: 2; webkit-border-horizontal-spacing: 0px; webkit-border-vertical-spacing: 0px; webkit-text-decorations-in-effect: none; webkit-text-size-adjust: auto; webkit-text-stroke-width: 0"> <div style="WORD-WRAP: break-word; webkit-nbsp-mode: space; webkit-line-break: after-white-space"><span class="Apple-style-span" style="WORD-SPACING: 0px; FONT: 12px Helvetica; TEXT-TRANSFORM: none; COLOR: rgb(0,0,0); TEXT-INDENT: 0px; WHITE-SPACE: normal; LETTER-SPACING: normal; BORDER-COLLAPSE: separate; orphans: 2; widows: 2; webkit-border-horizontal-spacing: 0px; webkit-border-vertical-spacing: 0px; webkit-text-decorations-in-effect: none; webkit-text-size-adjust: auto; webkit-text-stroke-width: 0px"> <div style="WORD-WRAP: break-word; webkit-nbsp-mode: space; webkit-line-break: after-white-space"><span class="Apple-style-span" style="WORD-SPACING: 0px; FONT: 12px Helvetica; TEXT-TRANSFORM: none; COLOR: rgb(0,0,0); TEXT-INDENT: 0px; WHITE-SPACE: normal; LETTER-SPACING: normal; BORDER-COLLAPSE: separate; orphans: 2; widows: 2; webkit-border-horizontal-spacing: 0px; webkit-border-vertical-spacing: 0px; webkit-text-decorations-in-effect: none; webkit-text-size-adjust: auto; webkit-text-stroke-width: 0px"> <div>Aaron Knister</div> <div>Associate Systems Analyst</div> <div> <div> <div>Center for Ocean-Land-Atmosphere Studies</div></div> <div></div></div> <div><br class="khtml-block-placeholder"></div> <div>(301) 595-7000</div> <div><a href="mailto:aaron@iges.org">aaron@iges.org</a></div> <div><br class="khtml-block-placeholder"></div><br class="Apple-interchange-newline"></span></div></span></div></span><br class="Apple-interchange-newline"></div><br></div></blockquote></div><br><div apple-content-edited="true"> <span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0; "><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; "><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; "><div>Aaron Knister</div><div>Associate Systems Analyst</div><div><div><div>Center for Ocean-Land-Atmosphere Studies</div></div><div></div></div><div><br class="khtml-block-placeholder"></div><div>(301) 595-7000</div><div><a href="mailto:aaron@iges.org">aaron@iges.org</a></div><div><br class="khtml-block-placeholder"></div><br class="Apple-interchange-newline"></span></div></span></div></span><br class="Apple-interchange-newline"> </div><br></body></html>