[lustre-discuss] llapi_layout_file_comp_del

[Andreas Dilger]

John, you are probably right that allowing a passed fd would also be useful, but nobody has done it this way before because of the need to use O_LOV_DELAY_CREATE within the application code...  and to be honest very few applications tune their IO to this extent, especially with PFL layouts being available to handle 99% of the usage.

Please feel free to submit a patch that splits llapi_layout_file_open() into two functions:
- llapi_layout_open_fd() (or ...fd_open()? not sure) that only opens the file with O_LOV_DELAY_CREATE and returns the fd
- llapi_layout_set_by_fd() that sets the layout on the provided fd (whatever the source)

Then llapi_layout_file_open() would just call those two functions internally, and you could use only the llapi_layout_set_by_fd() function, if available (you could add:

#defeine HAVE_LLAPI_LAYOUT_SET_BY_FD

in the lustreapi.h header to avoid the need for a separate configure check.   Otherwise, your code would call your own internal wrapper of the same name that calls llapi_layout_file_open() and immediately closes the returned fd.  That would be slightly less efficient than the new API (two opens and closes), but would allow you to migrate to the new (more efficient) implementation easily in the future.

Cheers, Andreas

On Jul 28, 2022, at 14:03, John Bauer <bauerj at iodoctors.com<mailto:bauerj at iodoctors.com>> wrote:


Andreas,

Thanks for the info.  A related question:  I am using the O_LOV_DELAY_CREATE open flag mechanism to open a file and then set the composite layout with llapi_layout_file_open().  I was kind of surprised this worked.  This ends up opening the file twice and I simply close the fd returned from llapi_layout_file_open().  It would seem there should be an llapi function such as llapi_layout_set_by_fd() to match the llapi_layout_get_by_fd().  I need to use this mechanism to set striping for files where the pathname is not necessarily known before the open, such as the mkstemps() family of opens.  It also makes it easier to handle setting striping for files opened with openat().  It seems it would be more straight forward for llapi to work with an fd than a pathname if a valid fd already exists. Am I missing an easier way to do this?

Thanks,

John

On 7/27/22 16:25, Andreas Dilger wrote:
The HLD document was written before the feature was implemented, and is outdated.  The lustreapi.h and llapi_layout_file_comp_del.3 man page are correct.  Feel free to update the wiki to use the correct argument list.

I believe that it is possible to delete multiple components that match the <flags> argument (e.g. LCME_FL_NEG|LCME_FL_INIT), but I haven't tested that.

On Jul 26, 2022, at 14:35, John Bauer <bauerj at iodoctors.com<mailto:bauerj at iodoctors.com>> wrote:

Hi all,

I would like to use the llapi_layout_file_comp_del() function.  I have found 2 prototypes in different places.  One has the 3rd argument, uint32_t flags, and the other doesn't.  I suspect the High Level Design document is incorrect.  The one line of documentation in lustreapi.h indicates I could delete multiple components with one call.  How does one do that? What are the applicable flags?

>From version 2.12.8 lustreapi.h

/**
* Delete component(s) by the specified component id or flags.
*/
int  llapi_layout_file_comp_del(const  char  *path, uint32_t  id, uint32_t  flags);


>From https://wiki.lustre.org/PFL2_High_Level_Design

A new interface llapi_layout_file_comp_del(3) to delete component(s) by the specified component id (accepting LCME_ID_* wildcards also) from an existing file:

int llapi_layout_file_comp_del(const char *path, uint32_t id);

John

_______________________________________________
lustre-discuss mailing list
lustre-discuss at lists.lustre.org<mailto:lustre-discuss at lists.lustre.org>
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Cheers, Andreas
--
Andreas Dilger
Lustre Principal Architect
Whamcloud








Cheers, Andreas
--
Andreas Dilger
Lustre Principal Architect
Whamcloud







-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20220729/b0c9542e/attachment.htm>