[lustre-discuss] [EXTERNAL] Re: Mistake while adding OST - OST0003 : Invalid argument

[BALVERS Martin]

Hi Rick,

Thank you, that pointed me i the right direction!

The MGD  still had OST0003 as inactive:

[root at mds ~]# lctl --device MGS llog_print lustre-MDT0000
<snip>
- { index: 32, event: add_uuid, nid: 192.168.6.5 at tcp(0x20000c0a80605), node: 192.168.6.5 at tcp }
- { index: 33, event: attach, device: lustre-OST0003-osc-MDT0000, type: osc, UUID: lustre-MDT0000-mdtlov_UUID }
- { index: 34, event: setup, device: lustre-OST0003-osc-MDT0000, UUID: lustre-OST0003_UUID, node: 192.168.6.5 at tcp }
- { index: 35, event: add_osc, device: lustre-MDT0000-mdtlov, ost: lustre-OST0003_UUID, index: 3, gen: 1 }
- { index: 38, event: conf_param, device: lustre-OST0003-osc-MDT0000, parameter: osc.active=0 }

After activting it with 'lctl conf_param lustre-OST0003.osc.active=1', the client can see the OST again.

# lfs df -h
UUID                       bytes        Used   Available Use% Mounted on
lustre-MDT0000_UUID        11.4T        2.8T        8.6T  25% /lustre[MDT:0]
lustre-OST0000_UUID        97.2T       74.4T       22.8T  77% /lustre[OST:0]
lustre-OST0001_UUID        97.2T       74.0T       23.2T  77% /lustre[OST:1]
lustre-OST0002_UUID        97.2T       74.1T       23.1T  77% /lustre[OST:2]
lustre-OST0003_UUID        97.2T        7.0M       97.2T   1% /lustre[OST:3]
lustre-OST0004_UUID        97.2T        7.0M       97.2T   1% /lustre[OST:4]

filesystem_summary:       486.2T      222.5T      263.7T  46% /lustre

Thank you all for the help! Lustre can get quite complicated when things do not work as expected ;)

gr,
Martin Balvers

________________________________
From: Mohr, Rick <mohrrf at ornl.gov>
Sent: Thursday, August 14, 2025 15:58
To: BALVERS Martin <Martin.BALVERS at danone.com>; lustre-discuss at lists.lustre.org <lustre-discuss at lists.lustre.org>
Subject: Re: [EXTERNAL] Re: [lustre-discuss] Mistake while adding OST - OST0003 : Invalid argument

WARNING - EXTERNAL SENDER - BE CYBERSAFE


Martin,

Have you tried doing a writeconf on everything to regenerate the config logs?  If the fs is not in use atm, this might be a good time to try it.

--Rick


On 8/14/25, 3:11 AM, "lustre-discuss on behalf of BALVERS Martin via lustre-discuss" <lustre-discuss-bounces at lists.lustre.org <mailto:lustre-discuss-bounces at lists.lustre.org> on behalf of lustre-discuss at lists.lustre.org <mailto:lustre-discuss at lists.lustre.org>> wrote:

Hi Thomas,

Thanks for the reply.

Here is the output:
[root at mds ~]# lctl get_param osp.lustre-OST0003-osc-MDT0000.active
osp.lustre-OST0003-osc-MDT0000.active=1


The problem is caused by the fact that I replaced a previously added OST using the --replace option in the mkfs.lustre command.

OST0004 was added correctly in one go, and that one is visible to the client, so I don't think it is a client problem. The client can reach the OSS that has OST0003.

It seems that the according to the MDS everything is fine. It is only the 'lfs df' command that shows a problem. Regular 'df' shows all the space of the lustre fs, but 'lfs df' does not.
# df -h
Filesystem Size Used Avail Use% Mounted on
192.168.6.1 at tcp:/lustre 487T 223T 264T 46% /lustre

# lfs df -h
UUID bytes Used Available Use% Mounted on
lustre-MDT0000_UUID 11.4T 2.8T 8.6T 25% /lustre[MDT:0]
lustre-OST0000_UUID 97.2T 74.4T 22.8T 77% /lustre[OST:0]
lustre-OST0001_UUID 97.2T 74.0T 23.2T 77% /lustre[OST:1]
lustre-OST0002_UUID 97.2T 74.1T 23.1T 77% /lustre[OST:2]
OST0003 : Invalid argument
lustre-OST0004_UUID 97.2T 7.0M 97.2T 1% /lustre[OST:4]

filesystem_summary: 388.9T 222.5T 166.4T 58% /lustre

The fs is not in use at the moment, no data has been written to it since adding OST0003 an 4. If there is a way to fix this removing and adding OST0003 again then I can try that. As long as the data that is on the other OST's is not lost.

Regards,
Martin Balvers




Ce message électronique et tous les fichiers attachés qu'il contient sont confidentiels et destinés exclusivement à l'usage de la personne à laquelle ils sont adressés. Si vous avez reçu ce message par erreur, merci de le retourner à son émetteur. Les idées et opinions présentées dans ce message sont celles de son auteur, et ne représentent pas nécessairement celles de DANONE ou d'une quelconque de ses filiales. La publication, l'usage, la distribution, l'impression ou la copie non autorisée de ce message et des attachements qu'il contient sont strictement interdits. 

This e-mail and any files transmitted with it are confidential and intended solely for the use of the individual to whom it is addressed. If you have received this email in error please send it back to the person that sent it to you. Any views or opinions presented are solely those of its author and do not necessarily represent those of DANONE or any of its subsidiary companies. Unauthorized publication, use, dissemination, forwarding, printing or copying of this email and its associated attachments is strictly prohibited.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20250815/926d7722/attachment-0001.htm>