<html><head></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><br><div><div>On Aug 3, 2010, at 11:25 AM, Roger Spellman wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><o:smarttagtype namespaceuri="urn:schemas-microsoft-com:office:smarttags" name="PersonName">
<!--[if !mso]>
<style>
st1\:*{behavior:url(#default#ieooui) }
</style>
<![endif]-->
<style>
<!--
/* Font Definitions */
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman";}
a:link, span.MsoHyperlink
{color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{color:blue;
text-decoration:underline;}
p
{mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:12.0pt;
font-family:"Times New Roman";}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:Arial;
color:navy;}
@page Section1
{size:8.5in 11.0in;
margin:1.0in 1.25in 1.0in 1.25in;}
div.Section1
{page:Section1;}
-->
</style>
<div lang="EN-US" link="blue" vlink="blue" style="word-wrap: break-word;-webkit-nbsp-mode: space;
-webkit-line-break: after-white-space">
<div class="Section1"><p class="MsoNormal"><font size="2" color="navy" face="Arial"><span style="font-size:
10.0pt;font-family:Arial;color:navy">Nathan,<o:p></o:p></span></font></p><p class="MsoNormal"><font size="2" color="navy" face="Arial"><span style="font-size:
10.0pt;font-family:Arial;color:navy"><o:p> </o:p></span></font></p><p class="MsoNormal"><font size="2" color="navy" face="Arial"><span style="font-size:
10.0pt;font-family:Arial;color:navy">Thank you. That works!<o:p></o:p></span></font></p><p class="MsoNormal"><font size="2" color="navy" face="Arial"><span style="font-size:
10.0pt;font-family:Arial;color:navy"><o:p> </o:p></span></font></p><p class="MsoNormal"><font size="2" color="navy" face="Arial"><span style="font-size:
10.0pt;font-family:Arial;color:navy">I found that if I change IP address, I
also need to remove the file /mnt/mdt/CONFIGS/*-client.</span></font></p></div></div></o:smarttagtype></blockquote><div><br></div>This is what tunefs.lustre --writeconf on the MDT does, when you first mount it after the writeconf.</div><div>--writeconf on the MDT and all OSTs is the preferred way of changing a server nid.</div><div> <br><blockquote type="cite"><o:smarttagtype namespaceuri="urn:schemas-microsoft-com:office:smarttags" name="PersonName"><div lang="EN-US" link="blue" vlink="blue" style="word-wrap: break-word;-webkit-nbsp-mode: space;
-webkit-line-break: after-white-space"><div class="Section1"><p class="MsoNormal"><font size="2" color="navy" face="Arial"><span style="font-size:
10.0pt;font-family:Arial;color:navy"><o:p></o:p></span></font></p><p class="MsoNormal"><font size="2" color="navy" face="Arial"><span style="font-size:
10.0pt;font-family:Arial;color:navy"><o:p> </o:p></span></font></p><p class="MsoNormal"><font size="2" color="navy" face="Arial"><span style="font-size:
10.0pt;font-family:Arial;color:navy">The reason is that the OST mounts failed –
the OST was still looking for the old IP Address. I grepped for files
with the old IP Address, and I found those client files.<o:p></o:p></span></font></p><p class="MsoNormal"><font size="2" color="navy" face="Arial"><span style="font-size:
10.0pt;font-family:Arial;color:navy"><br>
Is that a safe thing to do? Please note that my mdt and mgs are on the
same LUN.<o:p></o:p></span></font></p><p class="MsoNormal"><font size="2" color="navy" face="Arial"><span style="font-size:
10.0pt;font-family:Arial;color:navy"><o:p> </o:p></span></font></p><p class="MsoNormal"><font size="2" color="navy" face="Arial"><span style="font-size:
10.0pt;font-family:Arial;color:navy">Thanks.<o:p></o:p></span></font></p><p class="MsoNormal"><font size="2" color="navy" face="Arial"><span style="font-size:
10.0pt;font-family:Arial;color:navy"><o:p> </o:p></span></font></p><p class="MsoNormal"><font size="2" color="navy" face="Arial"><span style="font-size:
10.0pt;font-family:Arial;color:navy">-Roger<o:p></o:p></span></font></p><p class="MsoNormal"><font size="2" color="navy" face="Arial"><span style="font-size:
10.0pt;font-family:Arial;color:navy"> <o:p></o:p></span></font></p><p class="MsoNormal"><font size="2" color="navy" face="Arial"><span style="font-size:
10.0pt;font-family:Arial;color:navy"><o:p> </o:p></span></font></p>
<div style="border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt">
<div>
<div class="MsoNormal" align="center" style="text-align:center"><font size="3" face="Times New Roman"><span style="font-size:12.0pt">
<hr size="2" width="100%" align="center" tabindex="-1">
</span></font></div><p class="MsoNormal"><b><font size="2" face="Tahoma"><span style="font-size:10.0pt;
font-family:Tahoma;font-weight:bold">From:</span></font></b><font size="2" face="Tahoma"><span style="font-size:10.0pt;font-family:Tahoma"> Nathan Rutman
[mailto:nathan.rutman@oracle.com] <br>
<b><span style="font-weight:bold">Sent:</span></b> Tuesday, August 03, 2010
2:03 PM<br>
<b><span style="font-weight:bold">To:</span></b> Roger Spellman<br>
<b><span style="font-weight:bold">Cc:</span></b>
<a href="mailto:lustre-discuss@lists.lustre.org">lustre-discuss@lists.lustre.org</a><br>
<b><span style="font-weight:bold">Subject:</span></b> Re: [Lustre-discuss]
Problem with write_conf</span></font><o:p></o:p></p>
</div><p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:
12.0pt"><o:p> </o:p></span></font></p><p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:
12.0pt">There's a 'failsafe' feature that prevents filesystem name
changes:<o:p></o:p></span></font></p>
<div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt" type="cite">
<div><p><font size="2" face="Times New Roman"><span style="font-size:10.0pt">LustreError:
157-3: Trying to start OBD AFTER-MDT0000_UUID using the wrong disk
BEFORE-MDT0000_UUID. Were the /dev/ assignments rearranged?</span></font><o:p></o:p></p>
</div>
</blockquote>
<div>
<div><p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:
12.0pt">You'll have to go and delete the last_rcvd file off the disk for all
the servers in the filesystem as well as tunefs --writeconf them all to the
name AFTER name. <o:p></o:p></span></font></p>
</div>
<div><p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:
12.0pt"><o:p> </o:p></span></font></p>
</div>
<div><p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:
12.0pt">On Aug 2, 2010, at 6:08 PM, Roger Spellman wrote:<o:p></o:p></span></font></p>
</div><p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:
12.0pt"><br>
<br>
<o:p></o:p></span></font></p>
<div><!-- Converted from text/plain format --><p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:
12.0pt"><o:p> </o:p></span></font></p><p><font size="2" face="Times New Roman"><span style="font-size:10.0pt">Hi,<br>
I would like to be able to change a file system name. Towards that end, I
have run the following commands as an experiment:<br>
<br>
mkfs.lustre --reformat --fsname BEFORE --device-size=10000 --mgs
--mdt --mgsnode=10.2.9.1@o2ib0 /dev/mapper/map0<br>
dmesg -c<br>
mount -t lustre /dev/mapper/map0 /mnt/mdt<br>
dmesg -c<br>
umount /mnt/mdt<br>
dmesg -c<br>
tunefs.lustre --writeconf --fsname=AFTER --mgs --mdt /dev/mapper/map0<br>
dmesg -c<br>
mount -t lustre /dev/mapper/map0 /mnt/mdt<br>
dmesg -c<br>
<br>
Unfortunately, this does not work. Can someone please explain the correct
sequence of commands to ues? The output of each command is as follows.<br>
<br>
Thanks.<br>
<br>
[root@ts-hss2-01 ~]# mkfs.lustre --reformat --fsname BEFORE
--device-size=10000 --mgs --mdt --mgsnode=10.2.9.1@o2ib0 /dev/mapper/map0<br>
<br>
Permanent disk data:<br>
Target: BEFORE-MDTffff<br>
Index: unassigned<br>
Lustre FS: BEFORE<br>
Mount type: ldiskfs<br>
Flags: 0x75<br>
(MDT MGS needs_index first_time update )<br>
Persistent mount opts: iopen_nopriv,user_xattr,errors=remount-ro<br>
Parameters: mgsnode=10.2.9.1@o2ib mdt.group_upcall=/usr/sbin/l_getgroups<br>
<br>
device size = 1632256MB<br>
2 6 18<br>
formatting backing filesystem ldiskfs on /dev/mapper/map0<br>
target name BEFORE-MDTffff<br>
4k blocks
2500<br>
options -i 4096 -I 512 -q -O dir_index,extents,uninit_groups
-F<br>
mkfs_cmd = mke2fs -j -b 4096 -L BEFORE-MDTffff -i 4096 -I 512 -q -O
dir_index,extents,uninit_groups -F /dev/mapper/map0 2500<br>
Writing CONFIGS/mountdata<br>
[root@ts-hss2-01 ~]# dmesg -c<br>
LDISKFS-fs: barriers enabled<br>
kjournald2 starting: pid 1388, dev dm-4:8, com<st1:personname w:st="on">mit</st1:personname>
interval 5 seconds<br>
LDISKFS FS on dm-4, internal journal on dm-4:8<br>
LDISKFS-fs: delayed allocation enabled<br>
LDISKFS-fs: file extents enabled<br>
LDISKFS-fs: mballoc enabled<br>
LDISKFS-fs: mounted filesystem dm-4 with ordered data mode<br>
LDISKFS-fs: mballoc: 1 blocks 1 reqs (0 success)<br>
LDISKFS-fs: mballoc: 1 extents scanned, 0 goal hits, 1 2^N hits, 0 breaks, 0
lost<br>
LDISKFS-fs: mballoc: 1 generated and it took 2142<br>
LDISKFS-fs: mballoc: 512 preallocated, 0 discarded<br>
<br>
<br>
[root@ts-hss2-01 ~]# mount -t lustre /dev/mapper/map0 /mnt/mdt<br>
[root@ts-hss2-01 ~]# dmesg -c<br>
LDISKFS-fs: barriers enabled<br>
kjournald2 starting: pid 1406, dev dm-4:8, com<st1:personname w:st="on">mit</st1:personname>
interval 5 seconds<br>
LDISKFS FS on dm-4, internal journal on dm-4:8<br>
LDISKFS-fs: delayed allocation enabled<br>
LDISKFS-fs: file extents enabled<br>
LDISKFS-fs: mballoc enabled<br>
LDISKFS-fs: mounted filesystem dm-4 with ordered data mode<br>
LDISKFS-fs: mballoc: 0 blocks 0 reqs (0 success)<br>
LDISKFS-fs: mballoc: 0 extents scanned, 0 goal hits, 0 2^N hits, 0 breaks, 0
lost<br>
LDISKFS-fs: mballoc: 0 generated and it took 0<br>
LDISKFS-fs: mballoc: 0 preallocated, 0 discarded<br>
LDISKFS-fs: barriers enabled<br>
kjournald2 starting: pid 1410, dev dm-4:8, com<st1:personname w:st="on">mit</st1:personname>
interval 5 seconds<br>
LDISKFS FS on dm-4, internal journal on dm-4:8<br>
LDISKFS-fs: delayed allocation enabled<br>
LDISKFS-fs: file extents enabled<br>
LDISKFS-fs: mballoc enabled<br>
LDISKFS-fs: mounted filesystem dm-4 with ordered data mode<br>
Lustre: MGS MGS started<br>
Lustre: MGC10.2.9.1@o2ib: Reactivating import<br>
Lustre: Setting parameter BEFORE-MDT0000.mdt.group_upcall in log BEFORE-MDT0000<br>
Lustre: Enabling user_xattr<br>
Lustre: BEFORE-MDT0000: new disk, initializing<br>
Lustre: BEFORE-MDT0000: Now serving BEFORE-MDT0000 on /dev/mapper/map0 with
recovery enabled<br>
Lustre: 1503:0:(lproc_mds.c:271:lprocfs_wr_group_upcall()) BEFORE-MDT0000:
group upcall set to /usr/sbin/l_getgroups<br>
Lustre: BEFORE-MDT0000.mdt: set parameter group_upcall=/usr/sbin/l_getgroups<br>
<br>
<br>
[root@ts-hss2-01 ~]# umount /mnt/mdt<br>
[root@ts-hss2-01 ~]# dmesg -c<br>
Lustre: Failing over BEFORE-MDT0000<br>
Lustre: Skipped 1 previous similar message<br>
Lustre: *** setting obd BEFORE-MDT0000 device 'dm-4' read-only ***<br>
Turning device dm-4 (0xfd00004) read-only<br>
Lustre: BEFORE-MDT0000: shutting down for failover; client state will be
preserved.<br>
Lustre: MDT BEFORE-MDT0000 has stopped.<br>
LustreError: 1517:0:(ldlm_request.c:1025:ldlm_cli_cancel_req()) Got rc -108
from cancel RPC: canceling anyway<br>
LustreError: 1517:0:(ldlm_request.c:1587:ldlm_cli_cancel_list())
ldlm_cli_cancel_list: -108<br>
Lustre: MGS has stopped.<br>
LDISKFS-fs: mballoc: 3 blocks 3 reqs (0 success)<br>
LDISKFS-fs: mballoc: 8 extents scanned, 0 goal hits, 0 2^N hits, 0 breaks, 0
lost<br>
LDISKFS-fs: mballoc: 1 generated and it took 2598<br>
LDISKFS-fs: mballoc: 1145 preallocated, 0 discarded<br>
Removing read-only on unknown block (0xfd00004)<br>
Lustre: server umount BEFORE-MDT0000 complete<br>
<br>
<br>
[root@ts-hss2-01 ~]# tunefs.lustre --writeconf --fsname=AFTER --mgs --mdt
/dev/mapper/map0<br>
checking for existing Lustre data: found CONFIGS/mountdata<br>
Reading CONFIGS/mountdata<br>
<br>
Read previous values:<br>
Target: BEFORE-MDT0000<br>
Index: 0<br>
Lustre FS: BEFORE<br>
Mount type: ldiskfs<br>
Flags: 0x5<br>
(MDT MGS )<br>
Persistent mount opts: iopen_nopriv,user_xattr,errors=remount-ro<br>
Parameters: mgsnode=10.2.9.1@o2ib mdt.group_upcall=/usr/sbin/l_getgroups<br>
<br>
<br>
Permanent disk data:<br>
Target: AFTER-MDT0000<br>
Index: 0<br>
Lustre FS: AFTER<br>
Mount type: ldiskfs<br>
Flags: 0x105<br>
(MDT MGS writeconf )<br>
Persistent mount opts: iopen_nopriv,user_xattr,errors=remount-ro<br>
Parameters: mgsnode=10.2.9.1@o2ib mdt.group_upcall=/usr/sbin/l_getgroups<br>
<br>
Writing CONFIGS/mountdata<br>
[root@ts-hss2-01 ~]# dmesg -c<br>
LDISKFS-fs: barriers enabled<br>
kjournald2 starting: pid 1539, dev dm-4:8, com<st1:personname w:st="on">mit</st1:personname>
interval 5 seconds<br>
LDISKFS FS on dm-4, internal journal on dm-4:8<br>
LDISKFS-fs: delayed allocation enabled<br>
LDISKFS-fs: file extents enabled<br>
LDISKFS-fs: mballoc enabled<br>
LDISKFS-fs: recovery complete.<br>
LDISKFS-fs: mounted filesystem dm-4 with ordered data mode<br>
LDISKFS-fs: mballoc: 1 blocks 1 reqs (0 success)<br>
LDISKFS-fs: mballoc: 6 extents scanned, 0 goal hits, 0 2^N hits, 0 breaks, 0
lost<br>
LDISKFS-fs: mballoc: 1 generated and it took 2905<br>
LDISKFS-fs: mballoc: 506 preallocated, 0 discarded<br>
<br>
<br>
[root@ts-hss2-01 ~]# mount -t lustre /dev/mapper/map0 /mnt/mdt<br>
mount.lustre: mount /dev/mapper/map0 at /mnt/mdt failed: Invalid argument<br>
This may have multiple causes.<br>
Are the mount options correct?<br>
Check the syslog for more info.<br>
[root@ts-hss2-01 ~]# dmesg -c<br>
LDISKFS-fs: barriers enabled<br>
kjournald2 starting: pid 1567, dev dm-4:8, com<st1:personname w:st="on">mit</st1:personname>
interval 5 seconds<br>
LDISKFS FS on dm-4, internal journal on dm-4:8<br>
LDISKFS-fs: delayed allocation enabled<br>
LDISKFS-fs: file extents enabled<br>
LDISKFS-fs: mballoc enabled<br>
LDISKFS-fs: mounted filesystem dm-4 with ordered data mode<br>
LDISKFS-fs: mballoc: 0 blocks 0 reqs (0 success)<br>
LDISKFS-fs: mballoc: 0 extents scanned, 0 goal hits, 0 2^N hits, 0 breaks, 0
lost<br>
LDISKFS-fs: mballoc: 0 generated and it took 0<br>
LDISKFS-fs: mballoc: 0 preallocated, 0 discarded<br>
LDISKFS-fs: barriers enabled<br>
kjournald2 starting: pid 1575, dev dm-4:8, com<st1:personname w:st="on">mit</st1:personname>
interval 5 seconds<br>
LDISKFS FS on dm-4, internal journal on dm-4:8<br>
LDISKFS-fs: delayed allocation enabled<br>
LDISKFS-fs: file extents enabled<br>
LDISKFS-fs: mballoc enabled<br>
LDISKFS-fs: mounted filesystem dm-4 with ordered data mode<br>
Lustre: MGS MGS started<br>
Lustre: MGC10.2.9.1@o2ib: Reactivating import<br>
Lustre: MGS: Logs for fs AFTER were removed by user request. All servers
must be restarted in order to regenerate the logs.<br>
Lustre: Setting parameter AFTER-MDT0000.mdt.group_upcall in log AFTER-MDT0000<br>
Lustre: Enabling user_xattr<br>
LustreError: 157-3: Trying to start OBD AFTER-MDT0000_UUID using the wrong disk
BEFORE-MDT0000_UUID. Were the /dev/ assignments rearranged?<br>
LustreError: 1665:0:(mds_fs.c:828:mds_fs_setup()) cannot read last_rcvd: rc =
-22<br>
LustreError: 1665:0:(handler.c:2007:mds_setup()) AFTER-MDT0000: MDS filesystem
method init failed: rc = -22<br>
LustreError: 1665:0:(obd_config.c:372:class_setup()) setup AFTER-MDT0000 failed
(-22)<br>
LustreError: 1665:0:(obd_config.c:1199:class_config_llog_handler()) Err -22 on
cfg command:<br>
Lustre: cmd=cf003 0:AFTER-MDT0000
1:AFTER-MDT0000_UUID 2:0 3:AFTER-MDT0000 <br>
LustreError: 15b-f: MGC10.2.9.1@o2ib: The configuration from log
'AFTER-MDT0000' failed (-22). Make sure this client and the MGS are running
compatible versions of Lustre.<br>
LustreError: 15c-8: MGC10.2.9.1@o2ib: The configuration from log
'AFTER-MDT0000' failed (-22). This may be the result of communication errors
between this node and the MGS, a bad configuration, or other errors. See the
syslog for more information.<br>
LustreError: 1566:0:(obd_mount.c:1124:server_start_targets()) failed to start
server AFTER-MDT0000: -22<br>
LustreError: 1566:0:(obd_mount.c:1653:server_fill_super()) Unable to start
targets: -22<br>
LustreError: 1566:0:(obd_config.c:443:class_cleanup()) Device 4 not setup<br>
LustreError: 1566:0:(ldlm_request.c:1025:ldlm_cli_cancel_req()) Got rc -108
from cancel RPC: canceling anyway<br>
LustreError: 1566:0:(ldlm_request.c:1587:ldlm_cli_cancel_list())
ldlm_cli_cancel_list: -108<br>
Lustre: MGS has stopped.<br>
LDISKFS-fs: mballoc: 1 blocks 1 reqs (0 success)<br>
LDISKFS-fs: mballoc: 6 extents scanned, 0 goal hits, 0 2^N hits, 0 breaks, 0
lost<br>
LDISKFS-fs: mballoc: 1 generated and it took 2883<br>
LDISKFS-fs: mballoc: 503 preallocated, 0 discarded<br>
Lustre: 1566:0:(obd_mount.c:1473:server_put_super()) Cleaning orphaned obd
AFTER-mdtlov<br>
Lustre: server umount AFTER-MDT0000 complete<br>
LustreError: 1566:0:(obd_mount.c:2045:lustre_fill_super()) Unable to
mount (-22)<br>
<br>
Roger Spellman<br>
Staff Engineer<br>
Terascala, Inc.<br>
508-588-1501<br>
<a href="http://www.terascala.com/">www.terascala.com</a> <<a href="http://www.terascala.com/">http://www.terascala.com/</a>></span></font><o:p></o:p></p>
</div><p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:
12.0pt">_______________________________________________<br>
Lustre-discuss mailing list<br>
<a href="mailto:Lustre-discuss@lists.lustre.org">Lustre-discuss@lists.lustre.org</a><br>
<a href="http://lists.lustre.org/mailman/listinfo/lustre-discuss">http://lists.lustre.org/mailman/listinfo/lustre-discuss</a><o:p></o:p></span></font></p>
</div><p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:
12.0pt"><o:p> </o:p></span></font></p>
</div>
</div>
</div>
</div>
</o:smarttagtype></blockquote></div><br></body></html>