[Lustre-discuss] lustre 2.2.0 statfs performance
Andreas Dilger
adilger at whamcloud.com
Fri Apr 6 10:43:43 PDT 2012
On 2012-04-06, at 10:33 AM, Yevheniy Demchenko wrote:
> On 04/06/2012 06:40 AM, Andreas Dilger wrote:
>> On 2012-04-05, at 2:40 PM, Yevheniy Demchenko wrote:
>>> We are experimenting with a new lustre-2.2.0 and surprisingly not
>>> getting any statfs performance improvements. It's generally as poor as in previous versions. Also, statahead does not seem to make any
>>> difference to statfs performance.
>>>
>>> #strace -r ls -l
>>> ......
>>> 0.000041 lstat("77193", {st_mode=S_IFREG|0644, st_size=4096, ...}) = 0
>>> 0.000238 lgetxattr("77193", "security.selinux", 0x129d800, 255) =
>>> -1 ENODATA (No data available)
>>
>> This is selinux trying to fetch xattrs. A plague on selinux, IMHO, but we can't get rid of it. The prefetching and caching of existing and negative (i.e. non-existent) xattrs on the client is being discussed in http://bugs.whamcloud.com/browse/LU-549.
>>
> Thanks for reply, things are starting to be clearer. It seems, that
> problem is not in selinux itself, but in not-caching xattrs in lustre in
> general. if i understand correctly, statahead tries to read ahead file
> attributes and put them in cache, so that client would not contact mds
> for getting them. But if we don't cache xattrs, every tool like ls or
> rsync will contact mds for every file as it tries to get acls and
> selinux attributes, which can not be cached. Wouldn't it be better to
> catch the case when FS is mounted with -o noacl and selinux is disabled
> on the client side and immediately return EOPNOTSUPP without contacting
> mds? We would be happy to trade acls and xattrs for a better statfs
> performance.
Yes, that makes sense. It would probably be a simple patch in ll_getxattr_common() if you wanted to give it a try.
Cheers, Andreas
--
Andreas Dilger Whamcloud, Inc.
Principal Lustre Engineer http://www.whamcloud.com/
More information about the lustre-discuss
mailing list