<html><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">
Hi,<div><br class="webkit-block-placeholder"></div><div>Sorry for big delay in responses but I was away for christmas lunch.</div><div>Changing index may help but I am not certain of that. Definitly it is weird that you don't have OST0000 but you have OST0030 this may cause problems later with quotas when you try to turn them off. I think lustre will expect OST0000 to exist and if it don't find it, lustre will complain and quota will not work.</div><div>However if you do change indexes I think you need to do that in certain way I suggest do it as follows</div><div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" style="font: 12.0px Helvetica"># Umount all OST's and MDT's and run for each target:</font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" style="font: 12.0px Helvetica">tunefs.lustre --reformat --index=<index> --writeconf /dev/<block_device_name></font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "># This need to be done on all OSS's and on MDS</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" style="font: 12.0px Helvetica"># For each target mount it as ldiskfs file system. This need to be done on all OSS's and on MDS</font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">#for example:</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" style="font: 12.0px Helvetica">mount -t ldiskfs /dev/dm-0 /mnt/mdt</font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" style="font: 12.0px Helvetica"># delete file /mnt/mdt/last_rcvd</font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" style="font: 12.0px Helvetica"># mount filesystem</font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><br class="webkit-block-placeholder"></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">after that you can do writeconf for each target, then start MGS/MDT target, and then start one by one OST targets starting with mpath0 as first one</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><br class="webkit-block-placeholder"></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" style="font: 12.0px Helvetica">Also have a look at our /etc/multipath.conf</font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">As you can see it is very static but we can be sure that each dm-<number> device is always pointing to the same LUN</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><br class="webkit-block-placeholder"></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><div>defaults {</div><div> udev_dir /dev</div><div> polling_interval 10</div><div> selector "round-robin 0"</div><div> path_grouping_policy failover</div><div> getuid_callout "/sbin/scsi_id -g -u -s /block/%n"</div><div> prio_callout /bin/true</div><div> path_checker tur</div><div> rr_min_io 100</div><div> rr_weight priorities</div><div> failback immediate</div><div> no_path_retry fail</div><div> user_friendly_name yes</div><div> prio_callout "/sbin/mpath_prio_my %n"</div><div>}</div><div><div>devnode_blacklist {</div><div> devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st|sda)[0-9]*"</div><div> devnode "^hd[a-z]"</div><div> devnode "^cciss!c[0-9]d[0-9]*"</div><div>}</div><div><br class="webkit-block-placeholder"></div></div></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><div>multipaths {</div><div> multipath {</div><div> wwid 360001ff007e6173300000800001d1c17</div><div> alias dm-0</div><div> path_grouping_policy failover</div><div> path_checker tur</div><div> path_selector "round-robin 0"</div><div> failback immediate</div><div> rr_weight priorities</div><div> no_path_retry 5</div><div> }</div><div> multipath {</div><div> wwid 360001ff007e6173401000800001d1c17</div><div> alias dm-1</div><div> path_grouping_policy failover</div><div> path_checker tur</div><div> path_selector "round-robin 0"</div><div> failback immediate</div><div> rr_weight priorities</div><div> no_path_retry 5</div><div> }</div><div> multipath {</div><div> wwid 360001ff007e6173502000800001d1c17</div><div> alias dm-2</div><div> path_grouping_policy failover</div><div> path_checker tur</div><div> path_selector "round-robin 0"</div><div> failback immediate</div><div> rr_weight priorities</div><div> no_path_retry 5</div><div> }</div><div> multipath {</div><div> wwid 360001ff007e6173906000800001d1c17</div><div> alias dm-3</div><div> path_grouping_policy failover</div><div> path_checker tur</div><div> path_selector "round-robin 0"</div><div> failback immediate</div><div> rr_weight priorities</div><div> no_path_retry 5</div><div> }</div><div> multipath {</div><div> wwid 360001ff007e6173a07000800001d1c17</div><div> alias dm-4</div><div> path_grouping_policy failover</div><div> path_checker tur</div><div> path_selector "round-robin 0"</div><div> failback immediate</div><div> rr_weight priorities</div><div> no_path_retry 5</div><div> }</div><div> multipath {</div><div> wwid 360001ff007e6173b08000800001d1c17</div><div> alias dm-5</div><div> path_grouping_policy failover</div><div> path_checker tur</div><div> path_selector "round-robin 0"</div><div> failback immediate</div><div> rr_weight priorities</div><div> no_path_retry 5</div><div> }</div><div><div> multipath {</div><div> wwid 360001ff007e6173603000800001d1c17</div><div> alias dm-6</div><div> path_grouping_policy failover</div><div> path_checker tur</div><div> path_selector "round-robin 0"</div><div> failback immediate</div><div> rr_weight priorities</div><div> no_path_retry 5</div><div> }</div><div> multipath {</div><div> wwid 360001ff007e6173704000800001d1c17</div><div> alias dm-7</div><div> path_grouping_policy failover</div><div> path_checker tur</div><div> path_selector "round-robin 0"</div><div> failback immediate</div><div> rr_weight priorities</div><div> no_path_retry 5</div><div> }</div><div> multipath {</div><div> wwid 360001ff007e6173805000800001d1c17</div><div> alias dm-8</div><div> path_grouping_policy failover</div><div> path_checker tur</div><div> path_selector "round-robin 0"</div><div> failback immediate</div><div> rr_weight priorities</div><div> no_path_retry 5</div><div> }</div><div> multipath {</div><div> wwid 360001ff007e6173c09000800001d1c17</div><div> alias dm-9</div><div> path_grouping_policy failover</div><div> path_checker tur</div><div> path_selector "round-robin 0"</div><div> failback immediate</div><div> rr_weight priorities</div><div> no_path_retry 5</div><div> }</div><div> multipath {</div><div> wwid 360001ff007e6173d0a000800001d1c17</div><div> alias dm-10</div><div> path_grouping_policy failover</div><div> path_checker tur</div><div> path_selector "round-robin 0"</div><div> failback immediate</div><div> rr_weight priorities</div><div> no_path_retry 5</div><div> }</div><div> multipath {</div><div> wwid 360001ff007e6173e0b000800001d1c17</div><div> alias dm-11</div><div> path_grouping_policy failover</div><div> path_checker tur</div><div> path_selector "round-robin 0"</div><div> failback immediate</div><div> rr_weight priorities</div><div> no_path_retry 5</div><div> }</div><div>}</div><div><br class="webkit-block-placeholder"></div><div><br></div><div>I hope this helps</div><div><br class="webkit-block-placeholder"></div></div></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Wojciech</div><div><br class="webkit-block-placeholder"></div></div><div><div><div>On 13 Dec 2007, at 15:18, Ludovic Francois wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">On Dec 13, 3:12 pm, Ludovic Francois <lfranc...@gmail.com> wrote:</div> <blockquote type="cite"><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">On Dec 13, 2:59 pm, "Ludovic Francois" <lfranc...@gmail.com> wrote:</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><br></div> <blockquote type="cite"><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Do you think it's possible<span class="Apple-converted-space"> </span>someone overwrote the "label" with a tunefs command?</div> </blockquote><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><br></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">or the system</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><br></div> <blockquote type="cite"><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">I already saw it with some other file system.</div> </blockquote></blockquote><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><br></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">We recreated Target and Index with the tunefs.lustre command:</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><br></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">--8<---------------cut here---------------start------------->8---</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">[root@oss01 ~]# tunefs.lustre --writeconf --index 0 /dev/mpath/mpath0</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">checking for existing Lustre data: found CONFIGS/mountdata</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Reading CONFIGS/mountdata</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><br></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Read previous values:</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Target: lustre-OST0030</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Index: 48</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Lustre FS: lustre</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Mount type: ldiskfs</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Flags: 0x2</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">(OST )</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Persistent mount opts: errors=remount-ro,extents,mballoc</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Parameters: mgsnode=10.143.0.5@tcp mgsnode=10.143.0.6@tcp</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">failover.node=10.143.0.2@tcp sys.timeout=80 mgsnode=10.143.0.5@tcp</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">mgsnode=10.143.0.6@tcp failover.node=10.143.0.2@tcp sys.timeout=80</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><br></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><br></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Permanent disk data:</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Target: lustre-OST0000</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Index: 0</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Lustre FS: lustre</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Mount type: ldiskfs</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Flags: 0x102</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">(OST writeconf )</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Persistent mount opts: errors=remount-ro,extents,mballoc</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Parameters: mgsnode=10.143.0.5@tcp mgsnode=10.143.0.6@tcp</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">failover.node=10.143.0.2@tcp sys.timeout=80 mgsnode=10.143.0.5@tcp</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">mgsnode=10.143.0.6@tcp failover.node=10.143.0.2@tcp sys.timeout=80</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><br></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Writing CONFIGS/mountdata</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">[root@oss01 ~]#</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">--8<---------------cut here---------------end--------------->8---</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><br></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">But now<span class="Apple-converted-space"> </span>we have some problems<span class="Apple-converted-space"> </span>to remount the file<span class="Apple-converted-space"> </span>system, could you</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">confirm us this command just rewrite the index?</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><br></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Best Regards, Ludo</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><br></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">--</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Ludovic Francois <span class="Apple-converted-space"> </span>+33 (0)6 14 77 26 93</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">System Engineer<span class="Apple-converted-space"> </span>DataDirect Networks</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><br></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">_______________________________________________</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Lustre-discuss mailing list</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><a href="mailto:Lustre-discuss@clusterfs.com">Lustre-discuss@clusterfs.com</a></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><a href="https://mail.clusterfs.com/mailman/listinfo/lustre-discuss">https://mail.clusterfs.com/mailman/listinfo/lustre-discuss</a></div> </blockquote></div><br><div> <span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0; "><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0; "><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0; "><div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Lucida Grande" size="3" style="font: 11.0px Lucida Grande">Mr Wojciech Turek</font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 11px/normal Lucida Grande; min-height: 13px; ">Assistant System Manager</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Lucida Grande" size="3" style="font: 11.0px Lucida Grande">University of Cambridge</font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Lucida Grande" size="3" style="font: 11.0px Lucida Grande">High Performance Computing service </font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font class="Apple-style-span" face="'Lucida Grande'" size="3"><span class="Apple-style-span" style="font-size: 11px;">email: <a href="mailto:wjt27@cam.ac.uk">wjt27@cam.ac.uk</a></span></font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Lucida Grande" size="3" style="font: 11.0px Lucida Grande">tel. +441223763517</font></div></div><div><br class="webkit-block-placeholder"></div></span><br class="Apple-interchange-newline"></span></span> </div><br></div></body></html>