[lustre-discuss] [Samba] Odd "File exists" behavior when copy-pasting many files to an SMB exported Lustre FS

[Michael Weiser]

Hello Daniel,

> In the lustre.lov xattr, Lustre exposes layout information (ie. content
> distribution across servers) to regular users. In some cases, it's also
> possible to set the desired layout through this interface. Layout

Alright, that makes sense and indeed I can as unprivileged user write back at least the same value to lustre.lov without error:

$ getfattr -n lustre.lov blarf
# file: blarf
lustre.lov=0s0AvRCwEAAABFJAEAAAAAAIQjBAACAAAAAAAQAAEAAADrXSYBAAAAAAAAAAAAAAAAAAAAAAgAAAA=

$ setfattr -n lustre.lov -v 0s0AvRCwEAAABFJAEAAAAAAIQjBAACAAAAAAAQAAEAAADrXSYBAAAAAAAAAAAAAAAAAAAAAAgAAAA= blarf
<no error here>

> information depends on the innards of the specific fs setup, and should
> not be retained when moving files between different filesystems, hence
> the hint in the docs.

Then I guess it's time to give in and polish up the xattr-filtering VFS module I started work on so samba can replicate the xattr.conf workaround already documented.

> So for a working setup, it boils down to either turning off 'ea support'
> in Samba (on by default since 4.9), or turning on 'user_xattr' in the
> filesystem.

The first option would cement the notion in people's minds that somehow extended attributes cause problems and do not work reliably for yet another decade - I don't want that. The latter would still cause massive confusion, because lustre.lov would travel to Windows clients as LUSTRE.LOV and then back onto Lustre as user.lustre.lov and then back to Windows as USER.LUSTRE.LOV, polluting files everywhere with nonsensical information.

So best to tackle the problem at the root and filter the attribute out on the way to the client, I guess. I'd try to give the VFS module some config option where one could specify a list of attributes to filter, possibly with wildcards so xattr.conf could be replicated exactly.

Or is there a way to leverage libattr for exactly this functionality?
It seems to be a hard dependency already even though only used if the system's fgetxattr is unavailable or not acceptable (macos). Is that determination correct?

>From a quick glance at the code, attr_copy_action() could be leveraged to interrogate xattr.conf for what the actual libattr xattr copy functions attr_copy_fd()/attr_copy_file() would to with an attribute based on configuration in xattr.conf.
Would this be desireable or an smb.conf option be preferred?

> >> No. Lustre is returning "fictional" EA's that
> >> cannot be set. Linux filesystems that don't have
> >> EA's don't do that.

> On a Lustre system, the lustre.lov xattr can be set alright without
> receiving an error. But that's not what Samba does here. Instead, it
> tries to copy any user-readable xattr, and prepends a 'user.' prefix to
> any name unless it's already there. This only works if the filesystem
> has been mounted with the 'user_xattr' option.

So just to make a final push for a fix of that error code path: If I had a file with one of the XDG user attributes[1] such as user.xdg.origin.url on my local ext4 filesystem and copied that to Lustre with user_xattr enabled, all would be well. If user_xattr weren't enabled on the Lustre, I'd again run into this "file already exists" scenario and be confused. The alternative (with the error code path removing the file from the failed creation attempt) would  be for the extended attribute to vanish silently during copy which at least in my mind would be preferable.

[1] https://www.freedesktop.org/wiki/CommonExtendedAttributes/
-- 
Thanks,
Michael
________________________________________
From: Daniel Kobras <kobras at puzzle-itc.de>
Sent: 22 September 2022 15:53:23
To: Michael Weiser; Jeremy Allison
Cc: lustre-discuss at lists.lustre.org; samba at lists.samba.org
Subject: Re: [lustre-discuss] [Samba] Odd "File exists" behavior when copy-pasting many files to an SMB exported Lustre FS

Caution: External email. Do not open attachments or click links, unless this email comes from a known sender and you know the content is safe.


Hi Micha, hi Jeremy!

Am 22.09.22 um 11:21 schrieb Michael Weiser:
> [from the docs]:
>> If a client(s) will be mounted on several file systems, add the following line to /etc/xattr.conf  > file to avoid problems when files are moved between the file systems: lustre.* skip"
>
> What exactly were those problems hinted at in the documentation?
> Is the visibility of the lustre.lov attribute for unprivileged users actually needed for anything?
> Can exposing it to unprivileged users be switched off Lustre-side?

In the lustre.lov xattr, Lustre exposes layout information (ie. content
distribution across servers) to regular users. In some cases, it's also
possible to set the desired layout through this interface. Layout
information depends on the innards of the specific fs setup, and should
not be retained when moving files between different filesystems, hence
the hint in the docs.

> Jeremy wrote:
>
>> Lutre really should not be exposing EA's to callers if
>> it doesn't actually support EA's.

Just to clarify as Micha's original post had it hidden in parenthesis:
Lustre does support xattrs in general, and it does support the 'user.*'
namespace. It's just that the latter needs to be enabled explicitly with
the 'user_xattr' mount option. By default, access to 'user.*' xattrs is
rejected with EOPNOTSUPP.

>> No. Lustre is returning "fictional" EA's that
>> cannot be set. Linux filesystems that don't have
>> EA's don't do that.

On a Lustre system, the lustre.lov xattr can be set alright without
receiving an error. But that's not what Samba does here. Instead, it
tries to copy any user-readable xattr, and prepends a 'user.' prefix to
any name unless it's already there. This only works if the filesystem
has been mounted with the 'user_xattr' option.

So for a working setup, it boils down to either turning off 'ea support'
in Samba (on by default since 4.9), or turning on 'user_xattr' in the
filesystem.

Kind regards,

Daniel
--
Daniel Kobras
Principal Architect
Puzzle ITC Deutschland
+49 7071 14316 0
https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.puzzle-itc.de%2F&data=05%7C01%7Cmichael.weiser%40atos.net%7C2499367eca8543abdf5d08da9ca1d25a%7C33440fc6b7c7412cbb730e70b0198d5a%7C0%7C0%7C637994516076263555%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=jlZDne%2FdyPXTLjFd1O2HVgTFYk7e7RCFhWm31e78B6Y%3D&reserved=0

--
Puzzle ITC Deutschland GmbH
Sitz der Gesellschaft: Eisenbahnstraße 1, 72072
Tübingen

Eingetragen am Amtsgericht Stuttgart HRB 765802
Geschäftsführer:
Lukas Kallies, Daniel Kobras, Mark Pröhl