[lustre-discuss] How to remove an OST completely

[Tung-Han Hsieh]

Dear Hans,

Thank you very much. Replacing the OST is new to me and very very
useful. We will try it next time.

So, according to the description of the manual, to replace the OST
we probably need to:

1. Lock the old OST (e.g., chome-OST0008) such that it will not
   create new files (run in the MDT server):

   lctl set_param osc.chome-OST0008-osc-MDT0000.max_create_count=0

2. Locate the list of files in the old OST: (e.g., chome-OST0008):
   (run in the client):

   lfs find --obd chome-OST0008_UUID /home > /tmp/OST0008.txt

3. Migrate the listed files in /tmp/OST0008.txt out of the old OST.
   (run in the client).

4. Remove the old OST temporarily (run in the MDT server):

   lctl set_param osc.chome-OST0008-osc-MDT0000.active=0

   (Note: should use "set_param" instead of "conf_param")

5. Unmount the old OST partition (run in the old OST server)

6. Prepare the new OST for replacement by mkfs.lustre with --replace
   option, and set the index to the old OST index (e.g., 0x8):
   (run in the new OST server)

   mkfs.lustre --ost --mgsnode=XXXXXX --index=0x8 --replace <device_name>

7. Mount the new OST (run in the new OST server).


Best Regards,

T.H.Hsieh


On Thu, Mar 04, 2021 at 04:59:54PM +0100, Hans Henrik Happe via lustre-discuss wrote:
> Hi,
> 
> The manual describe this:
> 
> https://doc.lustre.org/lustre_manual.xhtml#lustremaint.remove_ost
> 
> There is a note telling you that it will still be there, but can be
> replaced.
> 
> Hope you migrated your data away from the OST also. Otherwise you would
> have lost it.
> 
> Cheers,
> Hans Henrik
> 
> On 03.03.2021 11.22, Tung-Han Hsieh via lustre-discuss wrote:
> > Dear All,
> >
> > Here is a question about how to remove an OST completely without
> > restarting the Lustre file system. Our Lustre version is 2.12.6.
> >
> > We did the following steps to remove the OST:
> >
> > 1. Lock the OST (e.g., chome-OST0008) such that it will not create
> >    new files (run in the MDT server):
> >
> >    lctl set_param osc.chome-OST0008-osc-MDT0000.max_create_count=0
> >
> > 2. Locate the list of files in the target OST: (e.g., chome-OST0008):
> >    (run in the client):
> >
> >    lfs find --obd chome-OST0008_UUID /home
> >
> > 3. Remove OST (run in the MDT server):
> >    lctl conf_param osc.chome-OST0008-osc-MDT0000.active=0
> >
> > 4. Unmount the OST partition (run in the OST server)
> >
> > After that, the total size of the Lustre file system decreased, and
> > everything looks fine. However, without restarting (i.e., rebooting
> > Lustre MDT / OST servers), we still feel that the removed OST is
> > still exists. For example, in MDT:
> >
> > # lctl get_param osc.*.active
> > osc.chome-OST0000-osc-MDT0000.active=1
> > osc.chome-OST0001-osc-MDT0000.active=1
> > osc.chome-OST0002-osc-MDT0000.active=1
> > osc.chome-OST0003-osc-MDT0000.active=1
> > osc.chome-OST0008-osc-MDT0000.active=0
> > osc.chome-OST0010-osc-MDT0000.active=1
> > osc.chome-OST0011-osc-MDT0000.active=1
> > osc.chome-OST0012-osc-MDT0000.active=1
> > osc.chome-OST0013-osc-MDT0000.active=1
> > osc.chome-OST0014-osc-MDT0000.active=1
> >
> > We still see chome-OST0008. And in dmesg of MDT, we see a lot of:
> >
> > LustreError: 4313:0:(osp_object.c:594:osp_attr_get()) chome-OST0008-osc-MDT0000:osp_attr_get update error [0x100080000:0x10a54c:0x0]: rc = -108
> >
> > In addition, when running LFSCK in the MDT server:
> >
> > 	lctl lfsck_start -A
> >
> > even after all the works of MDT and OST are completed, we still see that 
> > (run in MDT server):
> >
> > 	lctl get_param mdd.*.lfsck_layout
> >
> > the status is not completed:
> >
> > mdd.chome-MDT0000.lfsck_layout=
> > name: lfsck_layout
> > magic: 0xb1732fed
> > version: 2
> > status: partial
> > flags: incomplete
> > param: all_targets
> > last_completed_time: 1614762495
> > time_since_last_completed: 4325 seconds
> > ....
> >
> > We suspect that the "incomplete" part might due to the already removed
> > chome-OST0008.
> >
> > Is there any way to completely remove the chome-OST0008 from the Lustre
> > file system ? since that OST device has already been reformatted for
> > other usage.
> >
> > Thanks very much.
> >
> >
> > T.H.Hsieh
> > _______________________________________________
> > lustre-discuss mailing list
> > lustre-discuss at lists.lustre.org
> > http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
> 

> _______________________________________________
> lustre-discuss mailing list
> lustre-discuss at lists.lustre.org
> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org