[Lustre-discuss] Multiple interfaces in LNet

Danny Sternkopf dsternkopf at hpce.nec.com
Mon Oct 20 04:20:46 PDT 2008


thank your for clarification!

>> Beside of TCP it is only possible to use multiple interfaces on the same
>> node with o2ib, right? With ko2iblnd one can setup several Lustre
>> networks for each IB interface. In fact you must setup several Lustre
>> networks otherwise only the 1st IB interface is used, correct?
> Yes, ksocklnd and ko2iblnd are the only LNDs that support multiple
> instances, i.e. LNet interfaces. Unlike ksocklnd, each ko2iblnd
> interface only sits on one IB interface, and multiple ko2iblnd
> interfaces can sit on a same IB interface.

That's interesting as well. Does it mean I can configure two LNet
networks over the same IB interface like this?:
options lnet networks=o2ib0(ib0),o2ib1(ib0)

>> It is not clear for me how MGS, MDS, OSS and Client choose a NID for
>> communication. I mean I know that LNet choose the best one, but who
>> provides the list with all available NIDs for a server? Or does it work
>> somehow different?
> To be precise, for outgoing messages, LNet only chooses a local NID to
> use, which is determined by the remote NID specified by PTLRPC (and
> ultimately by Lustre). Though it's possible to specify a local NID
> (e.g. the 'self' parameter of LNetPut), it is in fact used only for
> sanity checks.

Does MGS provide a list of NIDs to servers and clients? And then the
node chooses the best one. Who knows more about this?

Best regards,

Danny Sternkopf http://www.nec.de/hpc       dsternkopf at hpce.nec.com
HPCE Division  Germany phone: +49-711-68770-35 fax: +49-711-6877145
NEC Deutschland GmbH, Hansaallee 101, 40549 Düsseldorf
Geschäftsführer Yuya Momose
Handelsregister Düsseldorf HRB 57941; VAT ID DE129424743

More information about the lustre-discuss mailing list