[lustre-discuss] File Level Redundancy - Data Movement

[Spitz, Cory James]

Good questions, Indivar.

1) The Lustre Operations Manual doesn’t make this clear at https://doc.lustre.org/lustre_manual.xhtml#flr.operations.resyncmirror , but a mirror sync will be completed in the context of the client that it was executed on.

2) Mirror resync is entirely manual today.  Again, it is done in the context of the client from which the lfs command was executed.  Automatic or “Immediate” mirror is on the https://www.lustre.org/roadmap/ for 2.16.0.  However, as a side note, there are also plans to “externalize” and leverage the HSM coordinator to be able to hand-off the data movement to multiple data movers.  See LU-6081, LU-12890, and LU-13384 for an outline of that work.  I only mention it here because you’ve hinted at the need for this improvement, but it is not committed to the roadmap yet.

3) One way could use lfs find to find files needing a mirror update.  There are multiple options to zero in on the mirror count and/or state.  Consult the lfs-find man page for details.

-Cory

On 2/17/21, 6:43 AM, "lustre-discuss" <lustre-discuss-bounces at lists.lustre.org> wrote:

Hi All,

I was going through the FLR feature in the Lustre Manual and had some questions.

1. If we run the "lfs mirror" command on an existing file or directory, how is the file ' directory copied from one OST to another?
    Is the copying done by the Lustre client on which this command was run?

2. And, in case of delayed mirroring of newly created files, who does the copying?

3. How can we automate this process to automatically mirror certain file types?

Regards,


Indivar Nair

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20210218/b23751ad/attachment-0001.html>