<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);" class="elementToProof">
Hi Rick,</div>
<div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);" class="elementToProof">
<br>
</div>
<div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);" class="elementToProof">
Thank you, that pointed me i the right direction!</div>
<div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);" class="elementToProof">
<br>
</div>
<div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);" class="elementToProof">
The MGD still had OST0003 as inactive:</div>
<div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);" class="elementToProof">
<br>
</div>
<div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);" class="elementToProof">
[root@mds ~]# lctl --device MGS llog_print lustre-MDT0000</div>
<div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);" class="elementToProof">
<snip></div>
<div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);" class="elementToProof">
- { index: 32, event: add_uuid, nid: 192.168.6.5@tcp(0x20000c0a80605), node: 192.168.6.5@tcp }</div>
<div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);" class="elementToProof">
- { index: 33, event: attach, device: lustre-OST0003-osc-MDT0000, type: osc, UUID: lustre-MDT0000-mdtlov_UUID }</div>
<div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);" class="elementToProof">
- { index: 34, event: setup, device: lustre-OST0003-osc-MDT0000, UUID: lustre-OST0003_UUID, node: 192.168.6.5@tcp }</div>
<div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);" class="elementToProof">
- { index: 35, event: add_osc, device: lustre-MDT0000-mdtlov, ost: lustre-OST0003_UUID, index: 3, gen: 1 }</div>
<div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);" class="elementToProof">
- { index: 38, event: conf_param, device: lustre-OST0003-osc-MDT0000, parameter: osc.active=0 }</div>
<div id="appendonsend"></div>
<div class="elementToProof"><br>
</div>
<div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);" class="elementToProof">
After activting it with 'lctl conf_param lustre-OST0003.osc.active=1', the client can see the OST again.</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
# lfs df -h</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
UUID bytes Used Available Use% Mounted on</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
lustre-MDT0000_UUID 11.4T 2.8T 8.6T 25% /lustre[MDT:0]</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
lustre-OST0000_UUID 97.2T 74.4T 22.8T 77% /lustre[OST:0]</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
lustre-OST0001_UUID 97.2T 74.0T 23.2T 77% /lustre[OST:1]</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
lustre-OST0002_UUID 97.2T 74.1T 23.1T 77% /lustre[OST:2]</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
lustre-OST0003_UUID 97.2T 7.0M 97.2T 1% /lustre[OST:3]</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
lustre-OST0004_UUID 97.2T 7.0M 97.2T 1% /lustre[OST:4]</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
filesystem_summary: 486.2T 222.5T 263.7T 46% /lustre</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
Thank you all for the help! Lustre can get quite complicated when things do not work as expected ;)</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
gr,</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
Martin Balvers</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
<br>
</div>
<hr style="display: inline-block; width: 98%;">
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<b>From:</b> Mohr, Rick <mohrrf@ornl.gov><br>
<b>Sent:</b> Thursday, August 14, 2025 15:58<br>
<b>To:</b> BALVERS Martin <Martin.BALVERS@danone.com>; lustre-discuss@lists.lustre.org <lustre-discuss@lists.lustre.org><br>
<b>Subject:</b> Re: [EXTERNAL] Re: [lustre-discuss] Mistake while adding OST - OST0003 : Invalid argument
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-size: 11pt;">WARNING - EXTERNAL SENDER - BE CYBERSAFE<br>
<br>
<br>
Martin,<br>
<br>
Have you tried doing a writeconf on everything to regenerate the config logs? If the fs is not in use atm, this might be a good time to try it.<br>
<br>
--Rick<br>
<br>
<br>
On 8/14/25, 3:11 AM, "lustre-discuss on behalf of BALVERS Martin via lustre-discuss" <lustre-discuss-bounces@lists.lustre.org <<a class="OWAAutoLink" id="OWAb8dcbf35-3530-2d26-2990-a9d9b3d79612" href="mailto:lustre-discuss-bounces@lists.lustre.org">mailto:lustre-discuss-bounces@lists.lustre.org</a>>
on behalf of lustre-discuss@lists.lustre.org <<a class="OWAAutoLink" id="OWA6d82b5ac-802e-9d5c-8424-3d5298786364" href="mailto:lustre-discuss@lists.lustre.org">mailto:lustre-discuss@lists.lustre.org</a>>> wrote:<br>
<br>
Hi Thomas,<br>
<br>
Thanks for the reply.<br>
<br>
Here is the output:<br>
[root@mds ~]# lctl get_param osp.lustre-OST0003-osc-MDT0000.active<br>
osp.lustre-OST0003-osc-MDT0000.active=1<br>
<br>
<br>
The problem is caused by the fact that I replaced a previously added OST using the --replace option in the mkfs.lustre command.<br>
<br>
OST0004 was added correctly in one go, and that one is visible to the client, so I don't think it is a client problem. The client can reach the OSS that has OST0003.<br>
<br>
It seems that the according to the MDS everything is fine. It is only the 'lfs df' command that shows a problem. Regular 'df' shows all the space of the lustre fs, but 'lfs df' does not.<br>
# df -h<br>
Filesystem Size Used Avail Use% Mounted on<br>
192.168.6.1@tcp:/lustre 487T 223T 264T 46% /lustre<br>
<br>
# lfs df -h<br>
UUID bytes Used Available Use% Mounted on<br>
lustre-MDT0000_UUID 11.4T 2.8T 8.6T 25% /lustre[MDT:0]<br>
lustre-OST0000_UUID 97.2T 74.4T 22.8T 77% /lustre[OST:0]<br>
lustre-OST0001_UUID 97.2T 74.0T 23.2T 77% /lustre[OST:1]<br>
lustre-OST0002_UUID 97.2T 74.1T 23.1T 77% /lustre[OST:2]<br>
OST0003 : Invalid argument<br>
lustre-OST0004_UUID 97.2T 7.0M 97.2T 1% /lustre[OST:4]<br>
<br>
filesystem_summary: 388.9T 222.5T 166.4T 58% /lustre<br>
<br>
The fs is not in use at the moment, no data has been written to it since adding OST0003 an 4. If there is a way to fix this removing and adding OST0003 again then I can try that. As long as the data that is on the other OST's is not lost.<br>
<br>
Regards,<br>
Martin Balvers<br>
<br>
<br>
<br>
</div>
<DIV>
Ce message électronique et tous les fichiers attachés qu'il contient sont confidentiels et destinés exclusivement à l'usage de la personne à laquelle ils sont adressés. Si vous avez reçu ce message par erreur, merci de le retourner à son émetteur. Les idées et opinions présentées dans ce message sont celles de son auteur, et ne représentent pas nécessairement celles de DANONE ou d'une quelconque de ses filiales. La publication, l'usage, la distribution, l'impression ou la copie non autorisée de ce message et des attachements qu'il contient sont strictement interdits. <BR>
<BR>
This e-mail and any files transmitted with it are confidential and intended solely for the use of the individual to whom it is addressed. If you have received this email in error please send it back to the person that sent it to you. Any views or opinions presented are solely those of its author and do not necessarily represent those of DANONE or any of its subsidiary companies. Unauthorized publication, use, dissemination, forwarding, printing or copying of this email and its associated attachments is strictly prohibited.<BR>
</DIV></body>
</html>