[Lustre-discuss] InfiniBand QoS with Lustre ko2iblnd.

[Isaac Huang]

On Mon, Jun 22, 2009 at 04:49:03PM +0200, S?bastien Buisson wrote:
> ......
> Let's consider we have two sets of OSSes, each set serving a different 
> Lustre file system (i.e. all the OSTs of an OSS are part of the same 
> Lustre file system). The same Lustre clients have access to both 
> filesystems.
> In these conditions, how can we enforce different IB QoS in Lustre for 
> the 2 file systems?

By assigning different SLs to the two sets of servers based on server
GUIDs, i.e. target-port-guid in QoS policy file.

> - by using the ko2iblnd 'service' option, the o2iblnd SL would be the 
> same for all connections initiated by a given Lustre client, regardless 
> the destination file system. So we would not achieve our goal.

Not necessarily. The service-id would be the same, but SLs could be
different if the SM has been configured in a way that doesn't
determine SLs solely based on service-id (e.g. also based on target
GUIDs).

> ......
> - if the 'service id' information was stored on the MGS on a file system 
> basis, one could imagine to retrieve it at mount time on the clients. 
> The 'service id' information stored on the MGS could consist in a port 
> space and a port id. Thus it would be possible to affect different 
> service ports to the various connections initiated by the client, 
> depending on the target file system.
> What do you think? Would you say this is feasible, or can you see major 
> issues with this proposal?

The LNet configurations could not reside on the MGS because LNet must
have been properly configured so that configurations on MGS could be
fetched over the network.

Isaac