<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Exchange Server">
<!-- converted from rtf -->
<style>.EmailQuote { margin-left: 1pt; padding-left: 4pt; border-left: #800000 2px solid; }</style>
</head>
<body>
<font face="Bookman Old Style, serif" size="3">
<div><font color="#800000"> </font></div>
<div><font size="2" color="#800000">In my lustre 1.6 config, I have two MSA2000 and two DL380G5 servers. The servers sfs1 and sfs2 are the internal network Ethernet names for my two servers. The system interconnect names, ic-sfs1 and ic-sfs2, correspond to
the servers.</font></div>
<div><font size="2" color="#800000"> </font></div>
<div><font size="2" color="#800000">I've successfully (I think) run both "lctl ping ic-sfs1@o2ib" and "lctl ping ic-sfs1@o2ib" from server sfs1 and sfs2. Does this look correct. How do you read the output from this command?</font></div>
<div style="padding-left: 19pt; "><font face="Courier New, monospace" size="2">[root@hpcsfse2 ~]# <b>lctl ping 172.31.97.2@o2ib0</b></font></div>
<div style="padding-left: 19pt; "><font face="Courier New, monospace" size="2">12345-0@lo</font></div>
<div style="padding-left: 19pt; "><font face="Courier New, monospace" size="2">12345-172.31.97.2@o2ib</font></div>
<div style="padding-left: 19pt; "><font face="Courier New, monospace" size="2">[root@hpcsfse2 ~]# lctl ping 172.31.97.1@o2ib0</font></div>
<div style="padding-left: 19pt; "><font face="Courier New, monospace" size="2">12345-0@lo</font></div>
<div style="padding-left: 19pt; "><font face="Courier New, monospace" size="2">12345-172.31.97.1@o2ib</font></div>
<div style="padding-left: 19pt; "><font face="Courier New, monospace" size="2" color="#800000"> </font></div>
<div><font size="2" color="#800000">This is the .csv file I'm using as input to the lustre_config command. Note that the mdt and mgs components are mounted together.</font></div>
<div style="padding-left: 19pt; "><font face="Courier New, monospace" size="2">hpcsfse1:root> <b>cat src/scripts/hpcsfse_lustre_config.csv</b></font></div>
<div style="padding-left: 19pt; "><font face="Courier New, monospace" size="2">sfs1,options lnet networks=o2ib0,/dev/mapper/mpath0,/mnt/mdt_mgs,mdt|mgs,testfs,,,,,_netdev,ic-sfs2@o2ib0</font></div>
<div style="padding-left: 19pt; "><font face="Courier New, monospace" size="2">sfs2,options lnet networks=o2ib0,/dev/mapper/mpath1,/mnt/ost0,ost,testfs,ic-sfs1@o2ib0,,,,_netdev,ic-sfs1@o2ib0</font></div>
<div><font face="Courier New, monospace" size="2" color="#800000"> </font></div>
<div><font size="2" color="#800000">Configuration of sfs1 server seems to be OK.</font></div>
<div style="padding-left: 19pt; "><font face="Courier New, monospace" size="2">hpcsfse1:root> <b>lustre_config -vfw sfs1 src/scripts/hpcsfse_lustre_config.csv</b></font></div>
<div style="padding-left: 19pt; "><font face="Courier New, monospace" size="2">lustre_config: Operating on the following nodes: sfs1</font></div>
<div style="padding-left: 19pt; "><font face="Courier New, monospace" size="2">lustre_config: Checking the cluster network connectivity and hostnames...</font></div>
<div style="padding-left: 19pt; "><font face="Courier New, monospace" size="2">lc_net: Verifying network connectivity between "hpcsfse1.hpclab.usa.hp.com" and "sfs1"...</font></div>
<div style="padding-left: 19pt; "><font face="Courier New, monospace" size="2">lc_net: OK</font></div>
<div style="padding-left: 19pt; "><font face="Courier New, monospace" size="2">lustre_config: Check the cluster network connectivity and hostnames OK!</font></div>
<div style="padding-left: 19pt; "><font face="Courier New, monospace" size="2"> </font></div>
<div style="padding-left: 19pt; "><font face="Courier New, monospace" size="2">lustre_config: ******** Lustre cluster configuration START ********</font></div>
<div style="padding-left: 19pt; "><font face="Courier New, monospace" size="2">lustre_config: Explicit MGS target /dev/mapper/mpath0 in host sfs1.</font></div>
<div style="padding-left: 19pt; "><font face="Courier New, monospace" size="2">lustre_config: Adding lnet module options to sfs1</font></div>
<div style="padding-left: 19pt; "><font face="Courier New, monospace" size="2">lustre_config: Starting lnet network in sfs1</font></div>
<div style="padding-left: 19pt; "><font face="Courier New, monospace" size="2">lustre_config: Creating the mount point /mnt/mdt_mgs on sfs1</font></div>
<div style="padding-left: 19pt; "><font face="Courier New, monospace" size="2">lustre_config: Formatting Lustre target /dev/mapper/mpath0 on sfs1...</font></div>
<div style="padding-left: 19pt; "><font face="Courier New, monospace" size="2">lustre_config: Formatting command line is: ssh -x -q sfs1 "PATH=$PATH:/sbin:/usr/sbin; /usr/sbin/mkfs.lustre --reformat --mgs --mdt --fsname=testfs --failnode=ic-sfs2@o2ib0 /dev/mapper/mpath0"</font></div>
<div style="padding-left: 19pt; "><font face="Courier New, monospace" size="2">lustre_config: Waiting for the return of the remote command...</font></div>
<div style="padding-left: 19pt; "><font face="Courier New, monospace" size="2"> </font></div>
<div style="padding-left: 19pt; "><font face="Courier New, monospace" size="2"> Permanent disk data:</font></div>
<div style="padding-left: 19pt; "><font face="Courier New, monospace" size="2">Target: testfs-MDTffff</font></div>
<div style="padding-left: 19pt; "><font face="Courier New, monospace" size="2">Index: unassigned</font></div>
<div style="padding-left: 19pt; "><font face="Courier New, monospace" size="2">Lustre FS: testfs</font></div>
<div style="padding-left: 19pt; "><font face="Courier New, monospace" size="2">Mount type: ldiskfs</font></div>
<div style="padding-left: 19pt; "><font face="Courier New, monospace" size="2">Flags: 0x75</font></div>
<div style="padding-left: 19pt; "><font face="Courier New, monospace" size="2"> (MDT MGS needs_index first_time update )</font></div>
<div style="padding-left: 19pt; "><font face="Courier New, monospace" size="2">Persistent mount opts: errors=remount-ro,iopen_nopriv,user_xattr</font></div>
<div style="padding-left: 19pt; "><font face="Courier New, monospace" size="2">Parameters: failover.node=172.31.97.2@o2ib mgsnode=172.31.97.2@o2ib mdt.group_upcall=/usr/sbin/l_getgroups</font></div>
<div style="padding-left: 19pt; "><font face="Courier New, monospace" size="2"> </font></div>
<div style="padding-left: 19pt; "><font face="Courier New, monospace" size="2">device size = 1259804MB</font></div>
<div style="padding-left: 19pt; "><font face="Courier New, monospace" size="2">2 6 18</font></div>
<div style="padding-left: 19pt; "><font face="Courier New, monospace" size="2">formatting backing filesystem ldiskfs on /dev/mapper/mpath0</font></div>
<div style="padding-left: 19pt; "><font face="Courier New, monospace" size="2"> target name testfs-MDTffff</font></div>
<div style="padding-left: 19pt; "><font face="Courier New, monospace" size="2"> 4k blocks 0</font></div>
<div style="padding-left: 19pt; "><font face="Courier New, monospace" size="2"> options -J size=400 -i 4096 -I 512 -q -O dir_index,uninit_groups,mmp -F</font></div>
<div style="padding-left: 19pt; "><font face="Courier New, monospace" size="2">mkfs_cmd = mkfs.ext2 -j -b 4096 -L testfs-MDTffff -J size=400 -i 4096 -I 512 -q -O dir_index,uninit_groups,mmp -F /dev/mapper/mpath0</font></div>
<div style="padding-left: 19pt; "><font face="Courier New, monospace" size="2">Writing CONFIGS/mountdata</font></div>
<div style="padding-left: 19pt; "><font face="Courier New, monospace" size="2">lustre_config: Success on all Lustre targets!</font></div>
<div style="padding-left: 19pt; "><font face="Courier New, monospace" size="2">lustre_config: Modify /etc/fstab of host sfs1 to add Lustre target /dev/mapper/mpath0</font></div>
<div style="padding-left: 19pt; "><font face="Courier New, monospace" size="2">lustre_config: /dev/mapper/mpath0 /mnt/mdt_mgs lustre _netdev 0 0</font></div>
<div style="padding-left: 19pt; "><font face="Courier New, monospace" size="2">lustre_config: ******** Lustre cluster configuration END **********</font></div>
<div style="padding-left: 19pt; "><font face="Courier New, monospace" size="2"> </font></div>
<div style="padding-left: 19pt; "><font face="Courier New, monospace" size="2">hpcsfse1:root> <b>mount /mnt/mdt_mgs</b></font></div>
<div style="padding-left: 19pt; "><font face="Courier New, monospace" size="2"> </font></div>
<div><font size="2" color="#800000">Configuration of the sfs2 server fails. How do I debug and or correct this?</font></div>
<div style="padding-left: 19pt; "><font face="Courier New, monospace" size="2">hpcsfse1:root> <b>lustre_config -vfw sfs2 src/scripts/hpcsfse_lustre_config.csv</b></font></div>
<div style="padding-left: 19pt; "><font face="Courier New, monospace" size="2">lustre_config: Operating on the following nodes: sfs2</font></div>
<div style="padding-left: 19pt; "><font face="Courier New, monospace" size="2">lustre_config: Checking the cluster network connectivity and hostnames...</font></div>
<div style="padding-left: 19pt; "><font face="Courier New, monospace" size="2">lc_net: Verifying network connectivity between "hpcsfse1.hpclab.usa.hp.com" and "sfs2"...</font></div>
<div style="padding-left: 19pt; "><font face="Courier New, monospace" size="2">lc_net: OK</font></div>
<div style="padding-left: 19pt; "><font face="Courier New, monospace" size="2">lustre_config: Check the cluster network connectivity and hostnames OK!</font></div>
<div style="padding-left: 19pt; "><font face="Courier New, monospace" size="2"> </font></div>
<div style="padding-left: 19pt; "><font face="Courier New, monospace" size="2">lustre_config: ******** Lustre cluster configuration START ********</font></div>
<div style="padding-left: 19pt; "><font face="Courier New, monospace" size="2">lustre_config: There is no MGS target in the node list "sfs2".</font></div>
<div style="padding-left: 19pt; "><font face="Courier New, monospace" size="2">lustre_config: Creating the mount point /mnt/ost0 on sfs2</font></div>
<div style="padding-left: 19pt; "><font face="Courier New, monospace" size="2">lustre_config: Adding lnet module options to sfs2</font></div>
<div style="padding-left: 19pt; "><font face="Courier New, monospace" size="2">lustre_config: Starting lnet network in sfs2</font></div>
<div style="padding-left: 19pt; "><font face="Courier New, monospace" size="2">lustre_config: Checking lnet connectivity between sfs2 and the MGS node</font></div>
<div style="padding-left: 19pt; "><font face="Courier New, monospace" size="2" color="#FF0000"><b>lustre_config: check_lnet_connect() error: sfs2 cannot contact the MGS node with nids - "ic-sfs1@o2ib0"! Check /usr/sbin/lctl command!</b></font></div>
<div style="padding-left: 19pt; "><font face="Courier New, monospace" size="2"> </font></div>
<div style="padding-left: 19pt; "><font face="Courier New, monospace" size="2"><b>hpcsfse1:root> lctl dl</b></font></div>
<div style="padding-left: 19pt; "><font face="Courier New, monospace" size="2"> 0 UP mgs MGS MGS 5</font></div>
<div style="padding-left: 19pt; "><font face="Courier New, monospace" size="2"> 1 UP mgc MGC172.31.97.2@o2ib c2910fbc-b150-b759-0c41-a5851616e41e 5</font></div>
<div style="padding-left: 19pt; "><font face="Courier New, monospace" size="2"> 2 UP mdt MDS MDS_uuid 3</font></div>
<div style="padding-left: 19pt; "><font face="Courier New, monospace" size="2"> 3 UP lov testfs-mdtlov testfs-mdtlov_UUID 4</font></div>
<div style="padding-left: 19pt; "><font face="Courier New, monospace" size="2"> 4 UP mds testfs-MDT0000 testfs-MDT0000_UUID 3</font></div>
<div><font face="Courier New, monospace" size="2" color="#800000"> </font></div>
<div><font face="Courier New, monospace" size="2" color="#800000"> </font></div>
<div><font face="Courier New, monospace" size="2" color="#800000"> </font></div>
</font>
</body>
</html>