[lustre-discuss] Changelogs and jobstats with multitenancy and multiple cluster mounts

[Sebastien Buisson]

Hi,

Lustre Changelogs are properly generated when a client is accessing Lustre via a subdirectory mount. And the changelogs can also be retrieved as expected. What is not possible today is to configure the changelogs mask on a per-directory basis. This setting is global to each MDT. So what you can do is, for each directory that you use as a subdirectory mount, assign it to a different MDT, and define the desired changelogs mask on each MDT.

As for jobstats, one possibility is to make your batch schedulers on your different clusters insert something specific in their SLURM_JOB_ID environment variable, in addition to the job ID itself. Alternately, you can make use of Lustre's jobid_name parameter to build a JobID that fits your needs. See the Lustre manual for a comprehensive list of accepted format codes:
https://doc.lustre.org/lustre_manual.xhtml#jobstats

Cheers,
Sebastien.

> Le 24 nov. 2021 à 13:44, Passerini Marco <marco.passerini at cscs.ch> a écrit :
> 
> Hi,
> 
> Considering the following scenarios:
> 1) a single Lustre filesystem mounted on several clusters
> 2) a Lustre configured for multi-tenancy, exporting different filesystems on different VLANs which are mounted on several clusters
> 
> How does this interact with Lustre change logs, and with Lustre jobstats?
> 
> From what I saw, Lustre changelogs, in case of multitenancy, work only if the mounting client has visibility to the filesystem "root", or not to a "sub"-filesystem. Is it possible to have it enabled for each exported filesystem, in a more fine-grained way?
> 
> About jobstats: if there are multiple (e.g.) Slurm clusters which share the same filesystem, there might be conflicts on Job IDs. Is there a way to identify which cluster a job id belongs to?
> 
> Regards,
> Marco Passerini
> _______________________________________________
> lustre-discuss mailing list
> lustre-discuss at lists.lustre.org
> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org