[lustre-discuss] setting quotas from within a container

[Lisa Gerhardt]

Okay, turns out that even if you can write files as root, you need to 
add "--privileged" to the podman invocation to get "lfs setquota" to 
work. When I do that, everything works. Thanks all for the help!

Lisa

On 10/23/23 9:48 AM, Lisa Gerhardt wrote:
> Hi Andreas,
> Unfortunately, the management of our cluster is very favored towards 
> running these kinds of things in containers, so I don't have a lot of 
> choice there.
>
> I am able to create files from inside the container that show as owned 
> by root outside the container, so I think it's not a uid mapping issue.
>
> The version of lustre I'm running is a modified version of lustre 2.15 
> (2.15.0.7_rc2_cray_26_g389e50f) and I've got 2.15.0 inside the container.
>
> If I run an strace, I get this message for the failing run:
>
> openat(AT_FDCWD, "/proc/mounts", O_RDONLY|O_CLOEXEC) = 3
> fstat(3, {st_mode=S_IFREG|0444, st_size=0, ...}) = 0
> read(3, "fuse-overlayfs / fuse.fuse-overl"..., 1024) = 1024
> close(3)                                = 0
> openat(AT_FDCWD, "/pscratch/sd/l/lgerhard", O_RDONLY|O_DIRECTORY) = 3
> ioctl(3, _IOC(_IOC_READ|_IOC_WRITE, 0x66, 0xa2, 0xb0), 0x55c787a9c2c0) 
> = -1 EPERM (Operation not permitted)
> close(3)                                = 0
> write(2, "lfs setquota: quotactl failed: O"..., 55) = 55
> write(2, "setquota failed: Operation not p"..., 41) = 41
> exit_group(1)                           = ?
> +++ exited with 1 +++
>
> Which is why I'm wondering if "setquota" tries to read extended 
> attributes or something else that aren't getting passed through 
> properly with the container mount.
>
> Thanks,
> Lisa
>
> On 10/21/23 1:14 PM, Andreas Dilger wrote:
>> Hi Lisa,
>> The first question to ask is which Lustre version you are using?
>>
>> Second, are you using subdirectory mounts or other UID/GID mapping 
>> for the container? That could happen at both the Lustre level or by 
>> the kernel itself.  If you aren't sure, you could try creating a new 
>> file as root inside the container, then "ls -l" the file from outside 
>> the container to see if it is owned by root.
>>
>> You could try running "strace lfs setquota" to see what operation the 
>> -EPERM = -1 error is coming from.
>>
>> The other important question is whether you really want to allow root 
>> inside the container to be able to set the quota, or whether this 
>> should be reserved for root outside the container?
>>
>> Cheers, Andreas
>>
>>> On Oct 21, 2023, at 09:18, Lisa Gerhardt via lustre-discuss 
>>> <lustre-discuss at lists.lustre.org> wrote:
>>>
>>> 
>>> Hello,
>>> I'm trying to set user quotas from within a container run as root. I 
>>> can successfully do things like "lfs setstripe", but "lfs setquota" 
>>> fails with
>>>
>>> lfs setquota: quotactl failed: Operation not permitted
>>> setquota failed: Operation not permitted
>>>
>>> I suspect it might have something to do with how the file system is 
>>> mounted in the container. I'm wondering if anyone has any experience 
>>> with this or if someone could point me to some documentation to help 
>>> me understand what "setquota" is doing differently from "setstripe" 
>>> to see where things are going off the rails.
>>>
>>> Thanks,
>>> Lisa
>>> _______________________________________________
>>> lustre-discuss mailing list
>>> lustre-discuss at lists.lustre.org
>>> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>