<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
Ellis, the getfattr code has a poor "optimization", in that it first calls getxattr(buf=NULL, size=0) to have the kernel report the xattr size to allocate a buffer of exactly the right size. However, IMHO while this avoids a "too large" allocation (maybe a
few bytes vs. 64KB), it is inefficient by doubling the number of syscalls into the filesystem.
<div class=""><br class="">
</div>
<div class="">The other getxattr cache hit during lstat() is likely because of the kernel accessing an selinux/security xattr.<br class="">
<div><br class="">
<blockquote type="cite" class="">
<div class="">On Oct 8, 2021, at 07:04, Ellis Wilson via lustre-devel <<a href="mailto:lustre-devel@lists.lustre.org" class="">lustre-devel@lists.lustre.org</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div class="">Hi all,<br class="">
<br class="">
I'm trying to get to the bottom of a failure I'm seeing while running the lustre unit tests, specifically here for this test in the sanityn suite:<br class="">
= sanityn test 73: getxattr should not cause xattr lock cancellation =<br class="">
<br class="">
I'm running stock lustre 2.14.0 on Ubuntu 18.04 with Linux Kernel 5.4.0. I have six nodes total for the test: two clients, two mds, two oss. The test is running from one of the clients.<br class="">
<br class="">
The error I get is:<br class="">
============<br class="">
getfattr: Removing leading '/' from absolute path names<br class="">
# file: mnt/lustre/f73.sanityn<br class="">
user.attr1="value1"<br class="">
<br class="">
ELLIS: expected 2, but got 5<br class="">
sanityn test_73: @@@@@@ FAIL: not cached in /mnt/lustre <br class="">
Trace dump:<br class="">
= /usr/lib/lustre/tests/test-framework.sh:6273:error()<br class="">
= /usr/lib/lustre/tests/sanityn.sh:3557:test_73()<br class="">
= /usr/lib/lustre/tests/test-framework.sh:6581:run_one()<br class="">
= /usr/lib/lustre/tests/test-framework.sh:6628:run_one_logged()<br class="">
= /usr/lib/lustre/tests/test-framework.sh:6455:run_test()<br class="">
= /usr/lib/lustre/tests/sanityn.sh:3567:main()<br class="">
============<br class="">
<br class="">
I've instrumented the code to spit out the expected vs. discovered stat values. The failure indicates the file in question wasn't cached, but in fact the inverse is occurring -- it's both cached and hit more often than expected.<br class="">
<br class="">
The unadulterated test code follows:<br class="">
================<br class="">
3549 touch $DIR1/$tfile<br class="">
3550 setfattr -n user.attr1 -v value1 $DIR1/$tfile ||<br class="">
3551 error "setfattr1 failed"<br class="">
3552 getfattr -n user.attr1 $DIR2/$tfile || error "getfattr1 failed"<br class="">
3553 getfattr -n user.attr1 $DIR1/$tfile || error "getfattr2 failed"<br class="">
3554 clear_stats llite.*.stats<br class="">
3555 # PR lock should be cached by now on both clients<br class="">
3556 getfattr -n user.attr1 $DIR1/$tfile || error "getfattr3 failed"<br class="">
3557 # 2 hits for getfattr(0)+getfattr(size)<br class="">
3558 [ $(calc_stats llite.*.stats getxattr_hits) -eq 2 ] ||<br class="">
3559 error "not cached in $DIR1"<br class="">
================<br class="">
<br class="">
The failure occurs on line 3558.<br class="">
<br class="">
Manually performing these actions validates that indeed the jump is by 5, not 2:<br class="">
~# lctl get_param llite.*.stats | grep hits<br class="">
getxattr_hits 85 samples [reqs]<br class="">
getxattr_hits 4 samples [reqs]<br class="">
~# getfattr -n user.attr1 /mnt/lustre/f73.sanityn<br class="">
getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1"<br class="">
~# lctl get_param llite.*.stats | grep hits<br class="">
getxattr_hits 90 samples [reqs]<br class="">
getxattr_hits 4 samples [reqs]<br class="">
<br class="">
I straced getfattr as run in the test and found it issues the following:<br class="">
23262 lstat("/mnt/lustre/f73.sanityn", {st_mode=S_IFREG|0644, st_size=0, ...}) = 0<br class="">
23262 getxattr("/mnt/lustre/f73.sanityn", "user.attr1", NULL, 0) = 6<br class="">
23262 getxattr("/mnt/lustre/f73.sanityn", "user.attr1", "value1", 256) = 6<br class="">
<br class="">
I built a small C program to replicate just the above without all of the other fluff in getfattr, and I see 1 xattr cache hit occurring for the lstat, and two xattr cache hits occurring for each call of getxattr. So it replicates the 5 xattr cache hits. It
is notable that if one does NOT specify "user.attr1" and instead just uses an empty string you only get a single hit on each getxattr.<br class="">
<br class="">
I have a patch that revises the expected stat values from 2 to 5 and from 4 to 10, and while that works in my system I wanted to know:<br class="">
1. Are these changes expected? I don't know much about the xattr cache or when it's expected to be hit, but hitting twice for a single getxattr seemed high.<br class="">
2. Is there any location online where I can look at release testing results for these unit tests? I wanted to see if I was alone in hitting this many times, but couldn't locate such a repository of historical test results.<br class="">
<br class="">
Thanks for any and all help!<br class="">
</div>
</div>
</blockquote>
</div>
<br class="">
<div class="">
<div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div>Cheers, Andreas</div>
<div>--</div>
<div>Andreas Dilger</div>
<div>Lustre Principal Architect</div>
<div>Whamcloud</div>
<div><br class="">
</div>
<div><br class="">
</div>
<div><br class="">
</div>
</div>
</div>
</div>
</div>
</div>
<br class="Apple-interchange-newline">
</div>
<br class="Apple-interchange-newline">
<br class="Apple-interchange-newline">
</div>
<br class="">
</div>
</body>
</html>