[lustre-discuss] setting quotas from within a container

[Lisa Gerhardt]

Hi Andreas,
Unfortunately, the management of our cluster is very favored towards 
running these kinds of things in containers, so I don't have a lot of 
choice there.

I am able to create files from inside the container that show as owned 
by root outside the container, so I think it's not a uid mapping issue.

The version of lustre I'm running is a modified version of lustre 2.15 
(2.15.0.7_rc2_cray_26_g389e50f) and I've got 2.15.0 inside the container.

If I run an strace, I get this message for the failing run:

openat(AT_FDCWD, "/proc/mounts", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0444, st_size=0, ...}) = 0
read(3, "fuse-overlayfs / fuse.fuse-overl"..., 1024) = 1024
close(3)                                = 0
openat(AT_FDCWD, "/pscratch/sd/l/lgerhard", O_RDONLY|O_DIRECTORY) = 3
ioctl(3, _IOC(_IOC_READ|_IOC_WRITE, 0x66, 0xa2, 0xb0), 0x55c787a9c2c0) = 
-1 EPERM (Operation not permitted)
close(3)                                = 0
write(2, "lfs setquota: quotactl failed: O"..., 55) = 55
write(2, "setquota failed: Operation not p"..., 41) = 41
exit_group(1)                           = ?
+++ exited with 1 +++

Which is why I'm wondering if "setquota" tries to read extended 
attributes or something else that aren't getting passed through properly 
with the container mount.

Thanks,
Lisa

On 10/21/23 1:14 PM, Andreas Dilger wrote:
> Hi Lisa,
> The first question to ask is which Lustre version you are using?
>
> Second, are you using subdirectory mounts or other UID/GID mapping for the container? That could happen at both the Lustre level or by the kernel itself.  If you aren't sure, you could try creating a new file as root inside the container, then "ls -l" the file from outside the container to see if it is owned by root.
>
> You could try running "strace lfs setquota" to see what operation the -EPERM = -1 error is coming from.
>
> The other important question is whether you really want to allow root inside the container to be able to set the quota, or whether this should be reserved for root outside the container?
>
> Cheers, Andreas
>
>> On Oct 21, 2023, at 09:18, Lisa Gerhardt via lustre-discuss <lustre-discuss at lists.lustre.org> wrote:
>>
>> 
>> Hello,
>> I'm trying to set user quotas from within a container run as root. I can successfully do things like "lfs setstripe", but "lfs setquota" fails with
>>
>> lfs setquota: quotactl failed: Operation not permitted
>> setquota failed: Operation not permitted
>>
>> I suspect it might have something to do with how the file system is mounted in the container. I'm wondering if anyone has any experience with this or if someone could point me to some documentation to help me understand what "setquota" is doing differently from "setstripe" to see where things are going off the rails.
>>
>> Thanks,
>> Lisa
>> _______________________________________________
>> lustre-discuss mailing list
>> lustre-discuss at lists.lustre.org
>> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org