[lustre-discuss] question on behavior of supplementary group permissions

[Bertschinger, Thomas Andrew Hjorth]

Hello, 

We have a curious issue with supplemental group permissions. There is a set of files where a user has group read permission to the files via a supplemental group. If the user tries to open() one of these files, they get EACCES. Then, if the user stat()s the file (or seemingly does any operation that caches the inode on the client), the next open() attempt succeeds. Interactively, this looks like: 

$ cat /lustre/problem_file
cat: /lustre/problem_file: Permission denied
$ stat /lustre/problem_file
(succeeds)
$ cat /lustre/problem_file
(succeeds)

We've only observed this on particular client/server pair: 

client kernel: 3.10.0-1160.95.1
client lustre: 2.15.3
server kernel: 4.18.0-477.21.1
server lustre: 2.15.3

We have mdt.*.identity_upcall=NONE set on every server.  Also, we cannot reproduce the issue with newly created files; it only appears to affect a set of existing files. 

I have 2 questions about this. The big one is, section 41.1.2.1 of the Lustre manual claims: 

> If there is no upcall or if there is an upcall and it fails, one supplementary group at most will be added as supplied by the client.

To my reading, this suggests that the "bug" we observe above is actually the correct behavior. Well, the manual is not precise about which single supplementary group will be supplied by the client, but the relevant group in this case is not the first supplemental group in the user's group list, it's in the middle of the list. So my question is, is the Lustre manual accurate (and then my followup question is, if so, why do supplemental group permissions appear to work for us in most cases...)? 

Or is the manual wrong/out of date here?

My second question is, assuming the behavior described above is a bug, are there any known issues here that we could be running into? 

Let me know if I can provide any more information.

Thanks, 
Thomas Bertschinger