<div><font color="#ff0000">hi thanks for the quick response</font></div>
<div><font color="#ff0000"></font> </div>
<div><font color="#ff0000">****************************************************************************</font></div>
<div> ssh <a href="mailto:root@132.66.176.212">root@132.66.176.212</a><br>Scientific Linux CERN SLC release 4.6 (Beryllium)<br><a href="mailto:root@132.66.176.212's">root@132.66.176.212's</a> password:<br>Last login: Sun Dec 23 18:02:33 2007 from
<a href="http://x-mathr11.tau.ac.il">x-mathr11.tau.ac.il</a><br>[root@localhost ~]# iptables -L<br>Chain INPUT (policy ACCEPT)<br>target prot opt source destination</div>
<p>Chain FORWARD (policy ACCEPT)<br>target prot opt source destination</p>
<div>Chain OUTPUT (policy ACCEPT)<br>target prot opt source destination<br>[root@localhost ~]#<br>***************************************************************************</div>
<div> ssh <a href="mailto:root@132.66.176.215">root@132.66.176.215</a></div>
<div><a href="mailto:root@x-mathr11">root@x-mathr11</a> ~]# iptables -L<br>Chain INPUT (policy ACCEPT)<br>target prot opt source destination<br>RH-Firewall-1-INPUT all -- anywhere anywhere
</div>
<div>Chain FORWARD (policy ACCEPT)<br>target prot opt source destination<br>RH-Firewall-1-INPUT all -- anywhere anywhere</div>
<div>Chain OUTPUT (policy ACCEPT)<br>target prot opt source destination</div>
<div>Chain RH-Firewall-1-INPUT (2 references)<br>target prot opt source destination<br>ACCEPT all -- anywhere anywhere<br>ACCEPT icmp -- anywhere anywhere icmp any
<br>ACCEPT ipv6-crypt-- anywhere anywhere<br>ACCEPT ipv6-auth-- anywhere anywhere<br>ACCEPT udp -- anywhere <a href="http://224.0.0.251">224.0.0.251</a> udp dpt:5353
<br>ACCEPT udp -- anywhere anywhere udp dpt:ipp<br>ACCEPT all -- anywhere anywhere state RELATED,ESTABLISHED<br>ACCEPT tcp -- anywhere anywhere state NEW tcp dpts:30000:30101
<br>ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:ssh<br>ACCEPT udp -- anywhere anywhere state NEW udp dpt:afs3-callback<br>REJECT all -- anywhere anywhere reject-with icmp-host-prohibited
<br>[root@x-mathr11 ~]#<br></div>
<div><br>**********************************************************************************************</div>
<div> </div>
<div><font color="#ff0000">what do you mean by "is this network dedicated to Lustre?"</font></div>
<div><font color="#ff0000">I run only lustre on the machines since I would like to have a working configuration.</font></div>
<div><font color="#ff0000">can you recommend me an interface?</font></div>
<div><br> thanks !, Avi</div><br><br>
<div class="gmail_quote">On Dec 23, 2007 9:27 PM, Aaron Knister <<a href="mailto:aaron@iges.org">aaron@iges.org</a>> wrote:<br>
<blockquote class="gmail_quote" style="PADDING-LEFT: 1ex; MARGIN: 0px 0px 0px 0.8ex; BORDER-LEFT: #ccc 1px solid">
<div style="WORD-WRAP: break-word">
<div>Can you check the firewall on each of those machines ( iptables -L ) and paste that here. Also, is this network dedicated to Lustre? Lustre can easily saturate a network interface under load to the point it becomes difficult to login to a node if it only has one interface. I'd recommend using a different interface if you can.
</div>
<div>
<div></div>
<div class="Wj3C7c"><br>
<div>
<div>On Dec 23, 2007, at 11:03 AM, Avi Gershon wrote:</div><br>
<blockquote type="cite">
<div><font color="#cc0000">node 1 <u><a href="http://132.66.176.212/" target="_blank">132.66.176.212</a></u></font></div>
<div><u><font color="#cc0000">node 2 <a href="http://132.66.176.215/" target="_blank">132.66.176.215</a></font></u></div>
<div> </div>
<div>[root@x-math20 ~]# ssh <a href="http://132.66.176.215/" target="_blank">132.66.176.215</a><br><a href="mailto:root@132.66.176.215's" target="_blank">root@132.66.176.215's</a> password:<br>ssh(21957) Permission denied, please try again.
<br><a href="mailto:root@132.66.176.215's" target="_blank">root@132.66.176.215's</a> password:<br>Last login: Sun Dec 23 14:32:51 2007 from <a href="http://x-math20.tau.ac.il/" target="_blank">x-math20.tau.ac.il</a>
<br>[root@x-mathr11 ~]# lctl ping <a href="mailto:132.66.176.211@tcp0" target="_blank">132.66.176.211@tcp0 </a><br>failed to ping <a href="mailto:132.66.176.211@tcp" target="_blank">132.66.176.211@tcp</a>: Input/output error
<br>[root@x-mathr11 ~]# lctl list_nids<br><a href="mailto:132.66.176.215@tcp" target="_blank">132.66.176.215@tcp</a><br>[root@x-mathr11 ~]# ssh <a href="http://132.66.176.212/" target="_blank">132.66.176.212</a><br>The authenticity of host '
<a href="http://132.66.176.212/" target="_blank">132.66.176.212</a> (<a href="http://132.66.176.212/" target="_blank">132.66.176.212</a>)' can't be established. <br>RSA1 key fingerprint is 85:2a:c1:47:84:b7:b5:a6:cd:c4:57:86:af:ce:7e:74.
<br>Are you sure you want to continue connecting (yes/no)? yes<br>ssh(11526) Warning: Permanently added '<a href="http://132.66.176.212/" target="_blank">132.66.176.212 </a>' (RSA1) to the list of kno<br>wn hosts.
<br><a href="mailto:root@132.66.176.212's" target="_blank">root@132.66.176.212's</a> password:<br>Last login: Sun Dec 23 15:24:41 2007 from <a href="http://x-math20.tau.ac.il/" target="_blank">x-math20.tau.ac.il </a>
<br>[root@localhost ~]# lctl ping <a href="mailto:132.66.176.211@tcp0" target="_blank">132.66.176.211@tcp0</a><br>failed to ping <a href="mailto:132.66.176.211@tcp" target="_blank">132.66.176.211@tcp</a>: Input/output error
<br>[root@localhost ~]# lctl list_nids <br><a href="mailto:132.66.176.212@tcp" target="_blank">132.66.176.212@tcp</a><br>[root@localhost ~]#<br></div>
<div> </div>
<div>thanks for helping!!</div>
<div>Avi<br><br></div>
<div class="gmail_quote">On Dec 23, 2007 5:32 PM, Aaron Knister <<a href="mailto:aaron@iges.org" target="_blank">aaron@iges.org</a>> wrote:<br>
<blockquote class="gmail_quote" style="PADDING-LEFT: 1ex; MARGIN: 0px 0px 0px 0.8ex; BORDER-LEFT: #ccc 1px solid">
<div style="WORD-WRAP: break-word">
<div>On the oss can you ping the mds/mgs using this command--</div>
<div><br></div>
<div>lctl ping 132.66.176.211@tcp0</div>
<div><br></div>
<div>If it doesn't ping, list the nids on each node by running</div>
<div><br></div>
<div>lctl list_nids</div>
<div><br></div>
<div>and tell me what comes back.</div>
<div><br></div>
<div>-Aaron</div>
<div><br></div><br>
<div>
<div>
<div></div>
<div>
<div>On Dec 23, 2007, at 9:22 AM, Avi Gershon wrote:</div><br></div></div>
<blockquote type="cite">
<div>
<div></div>
<div><span style="COLOR: rgb(255,0,0)">HI I could use some help.</span><br style="COLOR: rgb(255,0,0)"><span style="COLOR: rgb(255,0,0)">I installed lustre on 3 computers </span><br style="COLOR: rgb(255,0,0)"><span style="COLOR: rgb(255,0,0)">
mdt/mgs :</span><br><br>************************************************************************************8<br>[root@x-math20 ~]#mkfs.lustre --reformat --fsname spfs --mdt --mgs /dev/hdb <br><br> Permanent disk data:
<br>Target: spfs-MDTffff<br>Index: unassigned<br>Lustre FS: spfs<br>Mount type: ldiskfs<br>Flags: 0x75<br> (MDT MGS needs_index first_time update )<br>Persistent mount opts: errors=remount-ro,iopen_nopriv,user_xattr
<br>Parameters:<br><br>device size = 19092MB<br>formatting backing filesystem ldiskfs on /dev/hdb<br> target name spfs-MDTffff<br> 4k blocks 0<br> options -J size=400 -i 4096 -I 512 -q -O dir_index -F
<br>mkfs_cmd = mkfs.ext2 -j -b 4096 -L spfs-MDTffff -J size=400 -i 4096 -I 512 -q -O dir_index -F /dev/hdb<br>Writing CONFIGS/mountdata<br>[root@x-math20 ~]# df<br>Filesystem 1K-blocks Used Available Use% Mounted on
<br>/dev/hda1 19228276 4855244 13396284 27% /<br>none 127432 0 127432 0% /dev/shm<br>/dev/hdb 17105436 455152 15672728 3% /mnt/test/mdt<br>[root@x-math20 ~]# cat /proc/fs/lustre/devices
<br> 0 <span style="COLOR: rgb(51,204,0)">UP mgs MGS MGS 5</span><br> 1 UP mgc MGC132.66.176.211@tcp 5f5ba729-6412-3843-2229-1310a0b48f71 5<br> 2 UP mdt MDS MDS_uuid 3<br> 3 UP lov spfs-mdtlov spfs-mdtlov_UUID 4<br> 4 UP mds spfs-MDT0000 spfs-MDT0000_UUID 3
<br>[root@x-math20 ~]#<br>*************************************************************end mdt******************************8<br><span style="COLOR: rgb(255,0,0)">so you can see that the MGS is up </span><br style="COLOR: rgb(255,0,0)">
<span style="COLOR: rgb(255,0,0)">ond on the ost's I get an error!! plz help...</span><br style="COLOR: rgb(255,0,0)"><br style="COLOR: rgb(255,0,0)"><span style="COLOR: rgb(255,0,0)">ost:</span><br>**********************************************************************
<br>[root@x-mathr11 ~]# mkfs.lustre --reformat --fsname spfs --ost --mgsnode=132.66. 176.211@tcp0 /dev/hdb1<br><br> Permanent disk data: <br>Target: spfs-OSTffff<br>Index: unassigned<br>Lustre FS: spfs<br>Mount type: ldiskfs
<br>Flags: 0x72<br> (OST needs_index first_time update )<br>Persistent mount opts: errors=remount-ro,extents,mballoc <br>Parameters: mgsnode=132.66.176.211@tcp<br><br>device size = 19594MB<br>formatting backing filesystem ldiskfs on /dev/hdb1
<br> target name spfs-OSTffff<br> 4k blocks 0<br> options -J size=400 -i 16384 -I 256 -q -O dir_index -F <br>mkfs_cmd = mkfs.ext2 -j -b 4096 -L spfs-OSTffff -J size=400 -i 16384 -I 256 -q -O dir_index -F /dev/hdb1
<br>Writing CONFIGS/mountdata<br>[root@x-mathr11 ~]# /CONFIGS/mountdata<br>-bash: /CONFIGS/mountdata: No such file or directory <br>[root@x-mathr11 ~]# mount -t lustre /dev/hdb1 /mnt/test/ost1<br><span style="COLOR: rgb(255,0,0)">
mount.lustre: mount /dev/hdb1 at /mnt/test/ost1 failed: Input/output error</span><br style="COLOR: rgb(255,0,0)"><span style="COLOR: rgb(255,0,0)">Is the MGS running?</span><br>***********************************************end ost********************************
<br><br><span style="COLOR: rgb(255,0,0)">can any one point out the problem? </span><br>thanks Avi.<br><br><br></div></div>_______________________________________________<br>Lustre-discuss mailing list<br><a href="mailto:Lustre-discuss@clusterfs.com" target="_blank">
Lustre-discuss@clusterfs.com</a><br><a href="https://mail.clusterfs.com/mailman/listinfo/lustre-discuss" target="_blank">https://mail.clusterfs.com/mailman/listinfo/lustre-discuss</a><br></blockquote></div><br>
<div><span style="WORD-SPACING: 0px; FONT: 12px Helvetica; TEXT-TRANSFORM: none; COLOR: rgb(0,0,0); TEXT-INDENT: 0px; WHITE-SPACE: normal; LETTER-SPACING: normal; BORDER-COLLAPSE: separate">
<div style="WORD-WRAP: break-word"><span style="WORD-SPACING: 0px; FONT: 12px Helvetica; TEXT-TRANSFORM: none; COLOR: rgb(0,0,0); TEXT-INDENT: 0px; WHITE-SPACE: normal; LETTER-SPACING: normal; BORDER-COLLAPSE: separate">
<div>Aaron Knister</div>
<div>Associate Systems Administrator/Web Designer</div>
<div>Center for Research on Environment and Water</div>
<div><br></div>
<div>(301) 595-7001</div>
<div><a href="mailto:aaron@iges.org" target="_blank">aaron@iges.org</a></div>
<div><br></div><br></span></div></span></div><br></div></blockquote></div><br></blockquote></div><br>
<div><span style="WORD-SPACING: 0px; FONT: 12px Helvetica; TEXT-TRANSFORM: none; COLOR: rgb(0,0,0); TEXT-INDENT: 0px; WHITE-SPACE: normal; LETTER-SPACING: normal; BORDER-COLLAPSE: separate">
<div style="WORD-WRAP: break-word"><span style="WORD-SPACING: 0px; FONT: 12px Helvetica; TEXT-TRANSFORM: none; COLOR: rgb(0,0,0); TEXT-INDENT: 0px; WHITE-SPACE: normal; LETTER-SPACING: normal; BORDER-COLLAPSE: separate">
<div>Aaron Knister</div>
<div>Associate Systems Administrator/Web Designer</div>
<div>Center for Research on Environment and Water</div>
<div><br></div>
<div>(301) 595-7001</div>
<div><a href="mailto:aaron@iges.org" target="_blank">aaron@iges.org</a></div>
<div><br></div><br></span></div></span></div><br></div></div></div></blockquote></div><br>