[Lustre-discuss] Infiniband hot spot avoidance with LMC>0
Nathan Dauchy
Nathan.Dauchy at noaa.gov
Tue Apr 28 11:51:43 PDT 2009
Isaac Huang wrote:
> On Mon, Apr 27, 2009 at 12:21:41PM -0600, Nathan Dauchy wrote:
>> Greetings,
>>
>> Does Lustre's o2ib LND take advantage of Infiniband's LID Mask Count
>> (LMC) capability? Might it be included in the future? I'm looking for
>> something similar to the "MV2_USE_HSAM=1" option for Hot-Spot Avoidance
>> with MVAPICH2.
>
> Nothing like this exists so far in the o2iblnd. Currently between any
> two ports there's only one QP which uses the LIDs as returned by the
> SM.
OK, thanks for confirming, Isaac.
> The MVAPICH2 seems to be striping outgoing data over multiple paths
> and adjusting path weights dynamically based on perceived speed of the
> paths. We'd be interested to take a look if there's a high-level
> description of the mechanism.
>
The best high-level document I can find is a presentation: "Hot-Spot
Avoidance with Multi-Pathing over InfiniBand: An MPI Perspective", by
Abhinav Vishnu, Matthew Koop, et. al.
http://nowlab.cse.ohio-state.edu/publications/conf-presentations/2007/vishnu-ccgrid07.pdf
I imagine the Ohio State guys would be happy to work with you.
And, if Sun's code licensing plans allow for it, you can always look at
the MVAPICH code. :)
-Nathan
More information about the lustre-discuss
mailing list