[Lustre-discuss] Limits for o2ib lnet network numbers

Cory Spitz spitzcor at cray.com
Wed Aug 8 07:48:22 PDT 2012


Liang,

What main stream perf. issue do you refer to?  Is there a JIRA ticket
tracking it?

Thanks,
-Cory

On 08/08/2012 09:38 AM, Liang Zhen wrote:
> Hi, LNet reserved 32 bits for network number, so you can choose a very large network number if only have a few networks, but really create many networks will have some issues:
> - o2iblnd will pre-allocate memory resources for each network, so it will consume a lot of memory
> - Main stream LNet will have performance issue if there're many networks, for example, hundreds, although it's not difficult to fix this.
> 
> Liang
> 
> On Aug 8, 2012, at 10:23 PM, Rick Mohr wrote:
> 
>>
>> I was curious what limitations exist for o2ib network numbers.  Most of
>> the time I am dealing with o2ib0, o2ib1, etc.  As as experiment, I tried
>> configuring a machine with o2ib1000, and that seemed to be OK.  I
>> figured there must be some limit on how large the network number can
>> get, but after doing some searching, I have been unable to find any docs
>> that specify a limit.  Does any know what the max network number is?
>>
>> -- 
>> Rick Mohr
>> HPC Systems Administrator
>> National Institute for Computational Sciences
>> http://www.nics.tennessee.edu/
>>
>> _______________________________________________
>> Lustre-discuss mailing list
>> Lustre-discuss at lists.lustre.org
>> http://lists.lustre.org/mailman/listinfo/lustre-discuss
> 
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-discuss



More information about the lustre-discuss mailing list