[lustre-discuss] Mistake while adding OST - OST0003 : Invalid argument

[BALVERS Martin]

Hi,

I really hope someone can help me.

I setup a new lustre filesystem (version 2.15.1) and copied all my data from my old filesystem over to the new one.
I then wanted to add two old server to the new filesystem. I create a new zpool and created the lustre fs with:

mkfs.lustre --reformat --ost --backfstype=zfs --fsname=lustre --index=3 --mgsnode=192.168.6.1 at tcp lustre/ost

Then I noticed the pool was way smaller than expected. I did this on the mgs"
lctl --device MGS conf_param lustre-OST0003.target.active=0

Destroyed the zpool on the oss, wiped the disks and recreated the pool. Then used:

mkfs.lustre --reformat --ost --backfstype=zfs --fsname=lustre --index=3 --replace --mgsnode=192.168.6.1 at tcp lustre/ost

lctl --device 13 activate

Now I see this from a client, OST003: invalid argument:
# lfs df -h
UUID                       bytes        Used   Available Use% Mounted on
lustre-MDT0000_UUID        11.4T        2.8T        8.6T  25% /lustre[MDT:0]
lustre-OST0000_UUID        97.2T       74.4T       22.8T  77% /lustre[OST:0]
lustre-OST0001_UUID        97.2T       74.0T       23.2T  77% /lustre[OST:1]
lustre-OST0002_UUID        97.2T       74.1T       23.1T  77% /lustre[OST:2]
OST0003             : Invalid argument
lustre-OST0004_UUID        97.2T        8.0M       97.2T   1% /lustre[OST:4]

filesystem_summary:       388.9T      222.5T      166.4T  58% /lustre

How do I recover from this?

extra info:
[root at mds ~]# lctl dl
  0 UP osd-zfs MGS-osd MGS-osd_UUID 4
  1 UP mgs MGS MGS 18
  2 UP mgc MGC192.168.6.1 at tcp 74a2f74d-7309-414a-9d07-b5cf97a990a4 4
  3 UP osd-zfs lustre-MDT0000-osd lustre-MDT0000-osd_UUID 11
  4 UP mds MDS MDS_uuid 2
  5 UP lod lustre-MDT0000-mdtlov lustre-MDT0000-mdtlov_UUID 3
  6 UP mdt lustre-MDT0000 lustre-MDT0000_UUID 20
  7 UP mdd lustre-MDD0000 lustre-MDD0000_UUID 3
  8 UP qmt lustre-QMT0000 lustre-QMT0000_UUID 3
  9 UP lwp lustre-MDT0000-lwp-MDT0000 lustre-MDT0000-lwp-MDT0000_UUID 4
 10 UP osp lustre-OST0000-osc-MDT0000 lustre-MDT0000-mdtlov_UUID 4
 11 UP osp lustre-OST0002-osc-MDT0000 lustre-MDT0000-mdtlov_UUID 4
 12 UP osp lustre-OST0001-osc-MDT0000 lustre-MDT0000-mdtlov_UUID 4
 13 UP osp lustre-OST0003-osc-MDT0000 lustre-MDT0000-mdtlov_UUID 4
 14 UP osp lustre-OST0004-osc-MDT0000 lustre-MDT0000-mdtlov_UUID 4

[root at mds ~]# lctl get_param lov.*-mdtlov.target_obd
lov.lustre-MDT0000-mdtlov.target_obd=
0: lustre-OST0000_UUID ACTIVE
1: lustre-OST0001_UUID ACTIVE
2: lustre-OST0002_UUID ACTIVE
3: lustre-OST0003_UUID ACTIVE
4: lustre-OST0004_UUID ACTIVE

[root at mds ~]# lctl get_param mgs.MGS.live.*
mgs.MGS.live.lustre=
fsname: lustre
flags: 0x20     gen: 25
lustre-MDT0000
lustre-OST0000
lustre-OST0001
lustre-OST0002
lustre-OST0003
lustre-OST0004

Secure RPC Config Rules:

imperative_recovery_state:
    state: full
    nonir_clients: 0
    nidtbl_version: 9
    notify_duration_total: 0.002020998
    notify_duation_max: 0.000636370
    notify_count: 7
mgs.MGS.live.params=
fsname: params
flags: 0x20     gen: 1

Secure RPC Config Rules:

imperative_recovery_state:
    state: full
    nonir_clients: 0
    nidtbl_version: 2
    notify_duration_total: 0.000000000
    notify_duation_max: 0.000000000
    notify_count: 0

Ce message électronique et tous les fichiers attachés qu'il contient sont confidentiels et destinés exclusivement à l'usage de la personne à laquelle ils sont adressés. Si vous avez reçu ce message par erreur, merci de le retourner à son émetteur. Les idées et opinions présentées dans ce message sont celles de son auteur, et ne représentent pas nécessairement celles de DANONE ou d'une quelconque de ses filiales. La publication, l'usage, la distribution, l'impression ou la copie non autorisée de ce message et des attachements qu'il contient sont strictement interdits. 

This e-mail and any files transmitted with it are confidential and intended solely for the use of the individual to whom it is addressed. If you have received this email in error please send it back to the person that sent it to you. Any views or opinions presented are solely those of its author and do not necessarily represent those of DANONE or any of its subsidiary companies. Unauthorized publication, use, dissemination, forwarding, printing or copying of this email and its associated attachments is strictly prohibited.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20250813/64fc6ae9/attachment-0001.htm>