[Lustre-discuss] InfiniBand QoS with Lustre ko2iblnd.
Sébastien Buisson
sebastien.buisson at bull.net
Mon Jun 22 07:49:03 PDT 2009
Hi all,
We have been thinking about this IB QoS thing in Lustre for a while, and
we would like to express a need that may not be satisfied by the current
solution exposed by Isaac (which consists in using the ko2iblnd
'service' option).
Let's consider we have two sets of OSSes, each set serving a different
Lustre file system (i.e. all the OSTs of an OSS are part of the same
Lustre file system). The same Lustre clients have access to both
filesystems.
In these conditions, how can we enforce different IB QoS in Lustre for
the 2 file systems?
- by using the ko2iblnd 'service' option, the o2iblnd SL would be the
same for all connections initiated by a given Lustre client, regardless
the destination file system. So we would not achieve our goal.
Unless what really matters is the SL of the connections created by the
servers (I think I have seen in the Lustre debug logs that the 'real'
data transfers are always done via the servers connections).
What do you think?
- if the 'service id' information was stored on the MGS on a file system
basis, one could imagine to retrieve it at mount time on the clients.
The 'service id' information stored on the MGS could consist in a port
space and a port id. Thus it would be possible to affect different
service ports to the various connections initiated by the client,
depending on the target file system.
What do you think? Would you say this is feasible, or can you see major
issues with this proposal?
Thanks in advance.
Sebastien.
Isaac Huang a écrit :
> On Mon, May 18, 2009 at 12:04:37PM +0200, Daniel Kobras wrote:
>> Hi!
>>
>> Does anyone know how to use QoS with Lustre's o2ib LND? The Voltaire IB
>> LND allowed to #define a service level, but I couldn't find a similar
>> facility in o2ib. Is there a different way to apply QoS rules?
>
> The o2iblnd SL is set by the OFED RDMA CM, indirectly based on the
> o2iblnd service port (set via ko2iblnd option 'service', 987 by
> default) and its port space (RDMA_PS_TCP). For a complete, and more
> complicated story, please see:
> https://bugzilla.lustre.org/show_bug.cgi?id=18360#c2
>
> Isaac
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-discuss
>
>
More information about the lustre-discuss
mailing list