[lustre-discuss] Lustre doesn't use new OST

Massimo Sgaravatto massimo.sgaravatto at pd.infn.it
Tue Nov 10 03:52:19 PST 2015


Let me resurrect this old thread concerning the problem of OST not used 
if there are deactivated OSTs (see below for details)

I created a small testbed with a Lustre 2.5.3 server (MGS + MDT + OSS) 
and a Lustre 2.5.3 client.
I reproduced the problem creating a Lustre filesystem with:

- OST-0 OST-1 and OST-5 active
- OST-2, OST-3, OST-4 deactived

and I see that files are not written in OST-5.

I then tried to update the Lustre server machine to Lustre v. 2.7.0 
(while the client keeps running Lustre v. 2.5.3): the problem is still 

I then tried to also update the client to Lustre v. 2.7.0 and it looks 
like this fixes the problem.

Is this expected ? I thought the problem was in the server side ...

Thanks, Massimo

On 29/07/2015 17:47, Mohr Jr, Richard Frank (Rick Mohr) wrote:
> Have you taken a look at LU-5778 (https://jira.hpdd.intel.com/browse/LU-5778)?  It sounds like the same thing.
> --
> Rick Mohr
> Senior HPC System Administrator
> National Institute for Computational Sciences
> http://www.nics.tennessee.edu
>> On Jul 29, 2015, at 11:31 AM, Massimo Sgaravatto <massimo.sgaravatto at pd.infn.it> wrote:
>> Hi
>> We had a Lustre filesystem composed of 5 OSTs.
>> Because of a problem with 3 OSTs (the problem is described in the thread "Problems moving an OSS from an old Lustre installation to a new one"), we disabled them.
>> Now we want to reformat (mkfs.lustre --reformat ...) these 3 OSS and make them on-line.
>> For the time being we performed this operation just for one OSS (using a new index number).
>> The current scenario is the following (OST0005 is the reformatted OST):
>> lfs df -h /lustre/cmswork/
>> UUID                       bytes        Used   Available Use% Mounted on
>> cmswork-MDT0000_UUID      374.9G        3.5G      346.4G   1% /lustre/cmswork[MDT:0]
>> cmswork-OST0000_UUID       18.1T       14.5T        2.7T  84% /lustre/cmswork[OST:0]
>> cmswork-OST0001_UUID       18.1T       14.2T        3.0T  83% /lustre/cmswork[OST:1]
>> OST0002             : inactive device
>> OST0003             : inactive device
>> OST0004             : inactive device
>> cmswork-OST0005_UUID       13.6T      415.1M       12.9T   0% /lustre/cmswork[OST:5]
>> filesystem summary:        49.7T       28.7T       18.5T  61% /lustre/cmswork
>> The problem is that  the "Lustre scheduler" is not selecting OST0005 at all for new files.
>> Only if I use "lfs setstripe --index 5 " I see that the relevant files are written to this OST. Otherwise only OST0000 and OST0001 are used
>> We didn't change the values for qos_threshold_rr and qos_prio_free, which are therefore using the default values (17%, 91 %).
>> I can't find anything useful in the log files.
>> Any idea ?
>> Thanks, Massimo
>> _______________________________________________
>> lustre-discuss mailing list
>> lustre-discuss at lists.lustre.org
>> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 1897 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20151110/512f28cc/attachment.bin>

More information about the lustre-discuss mailing list