<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
You can also check "mdt.*.exports.*.open_files" on the MDTs for a list of FIDs open on each client, and use "lfs fid2path" to resolve them to a pathname.<br class="">
<div><br class="">
<blockquote type="cite" class="">
<div class="">On Sep 3, 2021, at 02:09, Degremont, Aurelien via lustre-discuss <<a href="mailto:lustre-discuss@lists.lustre.org" class="">lustre-discuss@lists.lustre.org</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div class="">Hi<br class="">
<br class="">
It could be a bug, but most of the time, this is due to an open-unlinked file, typically a log file which is still in use and some processes keep writing to it until it fills the OSTs it is using.<br class="">
<br class="">
Look for such files on your clients (use lsof). <br class="">
<br class="">
Aurélien<br class="">
<br class="">
<br class="">
Le 03/09/2021 09:50, « lustre-discuss au nom de Alastair Basden » <<a href="mailto:lustre-discuss-bounces@lists.lustre.org" class="">lustre-discuss-bounces@lists.lustre.org</a> au nom de
<a href="mailto:a.g.basden@durham.ac.uk" class="">a.g.basden@durham.ac.uk</a>> a écrit :<br class="">
<br class="">
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.<br class="">
<br class="">
<br class="">
<br class="">
Hi,<br class="">
<br class="">
We have a file system where each OST is a single SSD.<br class="">
<br class="">
One of those is reporting as 100% full (lfs df -h /snap8):<br class="">
snap8-OST004d_UUID 5.8T 2.0T 3.5T 37% /snap8[OST:77]<br class="">
snap8-OST004e_UUID 5.8T 5.5T 7.5G 100% /snap8[OST:78]<br class="">
snap8-OST004f_UUID 5.8T 2.0T 3.4T 38% /snap8[OST:79]<br class="">
<br class="">
However, I can't find any files on it:<br class="">
lfs find --ost snap8-OST004e /snap8/<br class="">
returns nothing.<br class="">
<br class="">
I guess that it has filled up, and that there is some bug or other that is<br class="">
now preventing proper behaviour - but I could be wrong.<br class="">
<br class="">
Does anyone have any suggestions?<br class="">
<br class="">
Essentially, I'd like to find some of the files and delete or migrate<br class="">
some, and thus return it to useful production.<br class="">
<br class="">
Cheers,<br class="">
Alastair.<br class="">
_______________________________________________<br class="">
lustre-discuss mailing list<br class="">
<a href="mailto:lustre-discuss@lists.lustre.org" class="">lustre-discuss@lists.lustre.org</a><br class="">
<a href="http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org" class="">http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org</a><br class="">
<br class="">
_______________________________________________<br class="">
lustre-discuss mailing list<br class="">
<a href="mailto:lustre-discuss@lists.lustre.org" class="">lustre-discuss@lists.lustre.org</a><br class="">
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org<br class="">
</div>
</div>
</blockquote>
</div>
<br class="">
<div class="">
<div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div>Cheers, Andreas</div>
<div>--</div>
<div>Andreas Dilger</div>
<div>Lustre Principal Architect</div>
<div>Whamcloud</div>
<div><br class="">
</div>
<div><br class="">
</div>
<div><br class="">
</div>
</div>
</div>
</div>
</div>
</div>
<br class="Apple-interchange-newline">
</div>
<br class="Apple-interchange-newline">
<br class="Apple-interchange-newline">
</div>
<br class="">
</body>
</html>