[Lustre-discuss] strange performance with POSIX file capabilities

[Andreas Dilger]

On Oct 06, 2009  15:13 +0200, Ralf Utermann wrote:
> with newer vanilla kernels we  saw strange performance
> data with iozone on patchless clients: some OSTs had a lower write
> bandwith in the iozone benchmark, getting worse with record
> sizes below 1024.
> After lots of kernel builds, it looks like the kernel config entry
> CONFIG_SECURITY_FILE_CAPABILITIES is the one, wich
> introduces this problem. If CONFIG_SECURITY_FILE_CAPABILITIES
> is not set, iozone data look good, if it's compiled into the
> kernel, we see the problem:
> http://www.physik.uni-augsburg.de/~ralfu/LustreTest/Lustre_with_file_caps.html

Just to clarify, you are reporting the above config option affects
write performance when changed on the client, correct?  It appears
that this option is off by default in the upstream kernels, so I
suspect it doesn't get tested much.

> Any idea, why file capabilities should affect the write
> performance on Lustre, and why it should only affect some OSTs?

I can imagine that if this is adding some significant overhead on a
per-system-call basis that it would hurt performance.

It is definitely odd that it would affect the performance of only some
of the OSTs.  I assume they are otherwise identical?  The only thing
I can imagine is that this option is related to SELinux and has some
overhead in getting extended attributes, but even then the xattrs are
only stored on the MDS so this would hurt all OSTs uniformly.

Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.