[Lustre-discuss] Limits for o2ib lnet network numbers
liang at whamcloud.com
Thu Aug 9 00:14:10 PDT 2012
No Jira ticket yet, reason of the potential performance issue is straightforward, it's because all LNet NIs are linked on a plain list, and we need scan the whole list for each sending/receiving, it's not an issue for a few networks, but it could be problematic for hundreds or tens.
On Aug 8, 2012, at 10:48 PM, Cory Spitz wrote:
> What main stream perf. issue do you refer to? Is there a JIRA ticket
> tracking it?
> On 08/08/2012 09:38 AM, Liang Zhen wrote:
>> Hi, LNet reserved 32 bits for network number, so you can choose a very large network number if only have a few networks, but really create many networks will have some issues:
>> - o2iblnd will pre-allocate memory resources for each network, so it will consume a lot of memory
>> - Main stream LNet will have performance issue if there're many networks, for example, hundreds, although it's not difficult to fix this.
>> On Aug 8, 2012, at 10:23 PM, Rick Mohr wrote:
>>> I was curious what limitations exist for o2ib network numbers. Most of
>>> the time I am dealing with o2ib0, o2ib1, etc. As as experiment, I tried
>>> configuring a machine with o2ib1000, and that seemed to be OK. I
>>> figured there must be some limit on how large the network number can
>>> get, but after doing some searching, I have been unable to find any docs
>>> that specify a limit. Does any know what the max network number is?
>>> Rick Mohr
>>> HPC Systems Administrator
>>> National Institute for Computational Sciences
>>> Lustre-discuss mailing list
>>> Lustre-discuss at lists.lustre.org
>> Lustre-discuss mailing list
>> Lustre-discuss at lists.lustre.org
More information about the lustre-discuss