[Lustre-discuss] Huge Sparse files in ROOT partition of MDT

[Landen]

Andreas Dilger  wrote:
> On Mar 06, 2009  07:26 -0600, Nirmal Seenu wrote:
>   
>> While trying to figure out the reason for LVM2 snapshots failing on our 
>> MDT server I found that there are a lot of sparse files on the MDT 
>> volume. The file size as seen from a ls command output on the MDT is 
>> same as the real file size. The tar runs for a few hours at this point 
>> even if try to use the --spare option in the tar command.
>>     
>
> All of the files on the MDT are sparse.  The data lives on the OSTs.
>
>   
>> The total MDT partition usage itself is about 500MB (as reported by df) 
>> and it used to take me less than 10 minutes to create a LVM2 snapshot 
>> and tar it up when I was running the servers using Lustre 1.6.5 with no 
>> quota enabled.
>>
>> I recently upgraded my Lustre servers to 1.6.7 and tried to enable quota 
>> on the MDT and OST by doing the following commands:
>>     
>
> Also, the quota file is a huge sparse file, size proportional to the
> highest UID in use.
>   
Quota files are sparse files, but they aren't *only* size proportional 
to the highest UID in use. It has some
relationship with distribution of uid/gid.  For example,
1. if the system only has one user whose uid=1 or uid=33554432, sizes of 
user quota file will be same.
2. if the system only has two users whose uid=(1, 2) or uid=(1, 
33554432), the latter will take three more blocks.
This thing is quite like ext3's indirect blocks.

BTW, there are two kinds of quota files in lustre:
1. admin quota files(only exists on mdt)
2. operation quota files(exist on mdt and osts)
For 1, its size is relative to the number of users who have quota 
limitation; for 2, its size is relative to the number of
users who are in use in filesystem.

Landen