[lustre-discuss] Problems with quota after adding a new OST

[Massimo Sgaravatto]

Hi

Thanks for spending your time with my problem ...


Let's consider this [*] output. So I set a quota on 1100000 for that 
user and this got exceeded.


If I use the -v flag [**] I see that on OST-0 it is using a quota limit 
of 747152, but no quota for OST-2 !


So when a user has to write a new file, if Lustre chooses OST-0, it 
checks if the quota is ok, while if it chooses OST-2, no quota limits 
are enforced.

Please see this example:


$ dd if=/dev/zero of=f1  bs=1024  count=1000000
dd: writing `f1': Disk quota exceeded

So disk quota exceeded.
OST-0 was chosen for this file:

$ lfs getstripe f1
f1
lmm_stripe_count:   1
lmm_stripe_size:    1048576
lmm_pattern:        1
lmm_layout_gen:     0
lmm_stripe_offset:  0
	obdidx		 objid		 objid		 group
	     0	       1257369	     0x132f99	             0

$ dd if=/dev/zero of=f2  bs=1024  count=1000000
1000000+0 records in
1000000+0 records out
1024000000 bytes (1.0 GB) copied, 12.5362 s, 81.7 MB/s

Here no problems with disk quota. This is becauue OST-2 was chosen:

$ lfs getstripe f2
f2
lmm_stripe_count:   1
lmm_stripe_size:    1048576
lmm_pattern:        1
lmm_layout_gen:     0
lmm_stripe_offset:  2
	obdidx		 objid		 objid		 group
	     2	         23042	       0x5a02	             0


Thanks, Massimo

-------------------------

[*]

# lfs quota -u fanzago /homeui
Disk quotas for user fanzago (uid 716):
      Filesystem  kbytes   quota   limit   grace   files   quota   limit 
   grace
         /homeui 16777916* 1000000 1100000       -     187       0 
  0       -

-----------------------------


[**]

# lfs quota -v -u fanzago /homeui
Disk quotas for user fanzago (uid 716):
      Filesystem  kbytes   quota   limit   grace   files   quota   limit 
   grace
         /homeui 16777916* 1000000 1100000       -     187       0 
  0       -
cmshome-MDT0000_UUID
                     156       -       0       -     187       -       0 
       -
cmshome-OST0000_UUID
                  550648       -  747152       -       -       -       - 
       -
quotactl ost1 failed.
cmshome-OST0002_UUID
                 16227112       -       0       -       -       -
-       -
Total allocated inode limit: 0, total allocated block limit: 747152
Some errors happened when getting quota info. Some devices may be not 
working or deactivated. The data in "[]" is inaccurate.


On 12/04/2016 17:21, Mohr Jr, Richard Frank (Rick Mohr) wrote:
> Based on the output of your “lfs quota” command, I am not sure that I see a problem with it.  Both active osts (OST0000 and OST0002) and the MDT seem to be returning quota information.   Can you explain what you were expecting to see and how the output differs from your expectations?
>
> --
> Rick Mohr
> Senior HPC System Administrator
> National Institute for Computational Sciences
> http://www.nics.tennessee.edu
>
>> On Apr 8, 2016, at 12:40 PM, Massimo Sgaravatto <massimo.sgaravatto at pd.infn.it> wrote:
>>
>> Hi
>>
>> We are running Lustre 2.7.0. We have problems with quota on a filesystem.
>>
>> This filesystem was initially composed of a single OST, and at that time quota was enable on the MDT simply doing:
>>
>> lctl conf_param cmshome.quota.ost=u
>> lctl conf_param cmshome.quota.mdt=u
>>
>> Nothing was needed on the OST (besided specifying "--param ost.quota_type=ug" when formatting).
>>
>> Problems started when we added a new OST (OST0002: there is also a deactived OST0001). This was formatted using "--param ost.quota_type=ug" but quota is disabled on this OST. [*]
>>
>> Do I need to somehow explicitly enable quota on this OST ? How and why ?
>>
>>
>> Thanks a lot, Massimo
>>
>>
>> [*]
>>
>> # lfs quota -v -u sgaravat /homeui
>> Disk quotas for user sgaravat (uid 741):
>>      Filesystem  kbytes   quota   limit   grace   files   quota   limit   grace
>>         /homeui  357356  400000  410000       -       2       0       0       -
>> cmshome-MDT0000_UUID
>>                       0       -       0       -       2       -       0       -
>> cmshome-OST0000_UUID
>>                  355784       -  372168       -       -       -       -       -
>> quotactl ost1 failed.
>> cmshome-OST0002_UUID
>>                    1572       -       0       -       -       -       -       -
>> Total allocated inode limit: 0, total allocated block limit: 372168
>> Some errors happened when getting quota info. Some devices may be not working or deactivated. The data in "[]" is inaccurate.
>>
>> _______________________________________________
>> lustre-discuss mailing list
>> lustre-discuss at lists.lustre.org
>> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>
>
>

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 1901 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20160412/da8560d2/attachment.bin>