<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body dir="auto">
<div dir="ltr">It really is that simple.
<div><br>
</div>
<div>You didn't mention what version you are using, but based on the hardware and sizes I would assume it is not the latest. </div>
<div><br>
</div>
<div>As such, there is a race that if clients are actively creating and writing new files at the instant the OSTs are added, those files may be inaccessible on some clients for a few seconds until the new OSTs are visible on all clients. </div>
<div><br>
</div>
<div> If the clients accessing the filesystem are quiesced during the initial mount then there is no race. Very recent servers and clients have fixed this race. </div>
<div><br id="lineBreakAtBeginningOfSignature">
<div dir="ltr">Cheers, Andreas</div>
<div dir="ltr"><br>
<blockquote type="cite">On Apr 3, 2025, at 15:12, Audet, Martin via lustre-discuss <lustre-discuss@lists.lustre.org> wrote:<br>
<br>
</blockquote>
</div>
<blockquote type="cite">
<div dir="ltr">
<div id="divtagdefaultwrapper" style="font-size:12pt;color:#000000;font-family:Calibri,Helvetica,sans-serif;" dir="ltr">
<div id="divtagdefaultwrapper" dir="ltr" style="font-size: 12pt; color: rgb(0, 0, 0); font-family: Calibri, Helvetica, sans-serif, EmojiFont, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols;">
<p></p>
<div>Hello Lustre community,</div>
<div><br>
</div>
<div>We are operating a small HPC cluster (576 compute cores) using a small Lustre parallel filesystem (64 TB) connected by Infiniband EDR network. The Lustre filesystem is implemented by a single HPE DL380 Gen10 server acting as MGS, MDS and OSS. It has two
32 TB OSTs (HPE MSA 2050). As new space is required, we will soon install 160 TB of additional storage implemented two 80 TB OSTs (HPE MSA 2060).</div>
<div><br>
</div>
<div>
<div>We looked in the Lustre documentation (10.2.1. Scaling the Lustre File System: https://doc.lustre.org/lustre_manual.xhtml#idm140220261007664) and made tests with small VMs. It appear that in our case adding this new storage would be very simple. From
what we understand we should do something like this:</div>
<div><br>
</div>
</div>
<p></p>
</div>
<blockquote style="margin: 0 0 0 40px; border: none; padding: 0px;">
<div dir="ltr" style="font-size: 12pt; color: rgb(0, 0, 0); font-family: Calibri, Helvetica, sans-serif, EmojiFont, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols;">
<p></p>
<div>
<div><span style="font-family: Consolas, Courier, monospace;">
<div># Create mount points for the new OSTs</div>
<div>mkdir /mnt/ost{2,3}</div>
<div><br>
</div>
<div># The MGS is running on the same node as the OSTs </div>
<div>mgs_node="$(sed -n -e 's/^ *- *nid: *//; T; p' < /etc/lnet.conf)"</div>
<div><br>
</div>
<div># Set the devices corresponding to the new OSTs using invariant names</div>
<div>ost2_device=/dev/disk/by-path/...</div>
<div>ost3_device=/dev/disk/by-path/...</div>
<div><br>
</div>
<div># Create the file systems on the new OSTs</div>
<div>mkfs.lustre --fsname=lustrevm --mgsnode=$mgs_node --ost --index=2 $ost2_device </div>
<div>mkfs.lustre --fsname=lustrevm --mgsnode=$mgs_node --ost --index=3 $ost3_device </div>
<div><br>
</div>
<div># Update fstab</div>
<div>cat >> /etc/fstab << _EOF_</div>
<div>$ost2_device /mnt/ost2 lustre defaults,_netdev 0 0</div>
<div>$ost3_device /mnt/ost3 lustre defaults,_netdev 0 0</div>
<div>_EOF_</div>
<div><br>
</div>
<div># Mount the new OSTs</div>
<div>mount /mnt/ost2</div>
<div>mount /mnt/ost3</div>
<div><br>
</div>
</span><br>
</div>
</div>
<p></p>
</div>
</blockquote>
<div dir="ltr" style="font-size: 12pt; color: rgb(0, 0, 0); font-family: Calibri, Helvetica, sans-serif, EmojiFont, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols;">
<p></p>
<div>
<div><span style="font-size: 12pt;">This appears too simple. Are we missing something ? Will the new files created by the clients use the four OSTs with no additional effort ?</span></div>
</div>
<div>
<div><br>
</div>
<div>Thanks in advance !</div>
<div><br>
</div>
<div>Martin Audet</div>
</div>
<p></p>
<p></p>
</div>
</div>
<span>_______________________________________________</span><br>
<span>lustre-discuss mailing list</span><br>
<span>lustre-discuss@lists.lustre.org</span><br>
<span>http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org</span><br>
</div>
</blockquote>
</div>
</div>
</body>
</html>