[Lustre-discuss] Lustre client and OST on same machine

[Jack David]

On Wed, Feb 15, 2012 at 5:00 PM, Peter Grandi <pg_lus at lus.for.sabi.co.uk> wrote:
>> I have a small lustre setup having 1 MDS, 2 OSS and all of the
>> three machine also have lustre client mounted.
>
> Note that running the Lustre client on a Lustre server is not
> recommnended because there can be a resource deadlock between
> the client and server modules (involving the cache IIRC).
>
> However I suspect that this problem occurs only on OSSes, and I
> suspect that it is negligible on the MDS.
>
> Which makes me suspect that running some types of workloads on
> the MDS as a client may give some advantages.
>
>> Once the lustre client knows about the stripe information of a
>> file, will it directly communicate to OSS?
>
> After fetching the file metadata from (one of) the MDS(es)
> Lustre clients always communicate directly with the OSS(es)
> involved.  That's the whole point of having distinct metadata
> and data servers.
>
Right, I agree.


> A coarse way of understanding Lustre and similar filesystem
> types is to imagine that Lustre clients can "mount" invidual
> files (or strips of a file) from an OSS, and the MDS is the
> server with the automount maps.
>
>> If yes, is there any optimization possible if lustre client
>> learns that the data is in the same machine (when acting as
>> OST) ?
>
> Not directly. RPCs may be rather faster locally than over a
> network interface.

Hmm, but it will still involve multiple memory copy operations to
transfer data from _local_ client to _local_ OST, correct? Can that be
avoided? or The client is smart enough to share the memory when it
knows the client and OST are the same machine.

Thanks
J