[Lustre-discuss] exceedingly slow lstats

Fan Yong yong.fan at whamcloud.com
Fri Jan 20 19:05:42 PST 2012


On 1/21/12 6:35 AM, John White wrote:
> Well, I was reading the strace wrong anyway:
> lstat("../403/a323", {st_mode=S_IFREG|0644, st_size=0, ...}) = 0<0.134326>
> getxattr("../403/a323", "system.posix_acl_access", 0x0, 0) = -1 EOPNOTSUPP (Operation not supported)<0.000018>
> lstat("../403/a330", {st_mode=S_IFREG|0644, st_size=0, ...}) = 0<0.158898>
> getxattr("../403/a330", "system.posix_acl_access", 0x0, 0) = -1 EOPNOTSUPP (Operation not supported)<0.000019>
> lstat("../403/a331", {st_mode=S_IFREG|0644, st_size=0, ...}) = 0<0.239466>
> getxattr("../403/a331", "system.posix_acl_access", 0x0, 0) = -1 EOPNOTSUPP (Operation not supported)<0.000012>
> lstat("../403/a332", {st_mode=S_IFREG|0644, st_size=0, ...}) = 0<0.130146>
> getxattr("../403/a332", "system.posix_acl_access", 0x0, 0) = -1 EOPNOTSUPP (Operation not supported)<0.000012>
>
> The getxattr takes an incredibly short amount of time, it's the lstat itself that's taking 0.1+s.

Which version Lustre they are using? for further investigation, you can 
collect detailed Lustre log on some client with the same test you did 
before on it to check which operation(s) slowdown the process.

Cheers,
Nasf
>
> ----------------
> John White
> HPC Systems Engineer
> (510) 486-7307
> One Cyclotron Rd, MS: 50C-3396
> Lawrence Berkeley National Lab
> Berkeley, CA 94720
>
> On Jan 20, 2012, at 2:28 PM, Mark Hahn wrote:
>
>>> I'm sorry, I'm not quite understanding what you're asking.  I don't have
>>> ACLs specifically enabled anywhere (and would expect the default is
>>> disabled).
>> I guess what I was suggesting is that you could try a simple experiment:
>> mount a client requesting the acl mount option.  (I don't know whether the "mount -oremount,acl ..." trick will work with Lustre.)
>>
>> if the problem goes away, you're done.
>>
>> you also mentioned OSS load - I don't see how that could be related,
>> since OSSs are not involved in metadata operations like lstat or getxattr.
>> (though depending on Lustre version, they could be involved in fetching
>> actual size of files, which is especially salient on striped files...)
>>
>>
>>
>>> ----------------
>>> John White
>>> HPC Systems Engineer
>>> (510) 486-7307
>>> One Cyclotron Rd, MS: 50C-3396
>>> Lawrence Berkeley National Lab
>>> Berkeley, CA 94720
>>>
>>> On Jan 20, 2012, at 12:49 PM, Mark Hahn wrote:
>>>
>>>>>    0.916908 getxattr("/global/scratch/jwhite/backuptest/highcount/3/a5", "system.posix_acl_access", 0x0, 0) = -1 EOPNOTSUPP (Operation not supported)
>>>> are your clients mounting with the acl option
>>>> and acl isn't missing on the mds mount?
>> -- 
>> operator may differ from spokesperson.	            hahn at mcmaster.ca
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-discuss




More information about the lustre-discuss mailing list