[lustre-discuss] lnet peer credits

Thomas Roth t.roth at gsi.de
Tue Aug 2 09:55:11 PDT 2016


Thanks, Chris, something less to worry about ;-)
Thomas

On 08/01/2016 08:16 PM, Christopher J. Morrone wrote:
> On 08/01/2016 06:33 AM, Thomas Roth wrote:
>> Hi all,
>>
>> is there a kind of a rule of thumb for the "min" number in
>> /proc/sys/lnet/peers?
>
> No, there is no rule of thumb.  It depends on too many factors in the
> system.  In my experience, numbers like you are showing here are
> completely normal.  The "min" field can be useful in context of problems
> that are occurring, but even then you need to have some way to know when
> the min occurred to corrolate it with whatever other issue is happening.
>  There is work under way on master to allow zeroing out those fields for
> just that purpose.  You are watching the min actively and see the
> numbers suddenly spike and corrolate that with some higher level issue,
> that can be useful.
>
> If you are not seeing any issues in the system, there is no need to be
> concerned about the numbers you posted.
>
>> Our ko2iblnd peer_credits-Parameter is at the default value, obviously 8.
>>
>> When I look up /proc/sys/lnet/peers (on an OSS), I typically get
>> something like
>>
>>
>> 10.20.1.76 at o2ib1            1    NA    -1     8     8     8     8   -18 0
>> 10.20.0.188 at o2ib1           1    NA    -1     8     8     8     8   -29 0
>> 10.20.0.44 at o2ib1            2    NA    -1     8     8     8     7   -15 72
>> 10.20.1.165 at o2ib1           1    NA    -1     8     8     8     8   -18 0
>> 10.20.1.21 at o2ib1            1    NA    -1     8     8     8     8 -2113 0
>> 10.20.0.133 at o2ib1           1    NA    -1     8     8     8     8   -28 0
>> 10.20.1.110 at o2ib1           1    NA    -1     8     8     8     8   -10 0
>> 10.20.0.222 at o2ib1           1    NA    -1     8     8     8     8   -20 0
>> 10.20.0.78 at o2ib1            1    NA    -1     8     8     8     8   -17 0
>> 10.20.1.55 at o2ib1            1    NA    -1     8     8     8     8    -7 0
>> 10.20.0.167 at o2ib1           1    NA    -1     8     8     8     8   -12 0
>> 10.20.1.144 at o2ib1           1    NA    -1     8     8     8     8    -8 0
>> 10.20.1.89 at o2ib1            1    NA    -1     8     8     8     8   -21 0
>> 10.20.1.34 at o2ib1            1    NA    -1     8     8     8     8   -11 0
>> 10.20.0.146 at o2ib1           1    NA    -1     8     8     8     8   -21 0
>> 10.20.0.2 at o2ib1             1    NA    -1     8     8     8     8  -584 0
>> 10.20.1.123 at o2ib1           1    NA    -1     8     8     8     8   -16 0
>> 10.20.0.91 at o2ib1            1    NA    -1     8     8     8     8   -25 0
>> 10.20.1.68 at o2ib1            1    NA    -1     8     8     8     8     1 0
>> 10.20.0.180 at o2ib1           1    NA    -1     8     8     8     8   -22 0
>> 10.20.0.185 at o2ib1           1    NA    -1     8     8     8     8   -17 0
>> 10.20.0.41 at o2ib1            1    NA    -1     8     8     8     8   -25 0
>> 10.20.1.162 at o2ib1           1    NA    -1     8     8     8     8   -14 0
>> 10.20.1.18 at o2ib1            1    NA    -1     8     8     8     8  -919 0
>> 10.20.0.130 at o2ib1           1    NA    -1     8     8     8     8   -13 0
>> 10.20.1.107 at o2ib1           1    NA    -1     8     8     8     8    -7 0
>> 10.20.0.219 at o2ib1           1    NA    -1     8     8     8     8   -12 0
>> 10.20.0.75 at o2ib1            1    NA    -1     8     8     8     8   -21 0
>> 10.20.1.196 at o2ib1           4    up    -1     8     8     8     8  -419 0
>>
>>
>> (The last line, the only peer that is "up", is an LNET-router)
>>
>>
>> Something to worry about?
>
> That is normal.  up/down state information is only given for peers that
> are routers.  "NA" means "Not Applicable".  That was an improvement over
> the past when, if I remember correctly, all non-router peers were listed
> as "down" regardless of their actual state.
>
> Chris
>
> _______________________________________________
> lustre-discuss mailing list
> lustre-discuss at lists.lustre.org
> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>

-- 
--------------------------------------------------------------------
Thomas Roth
Department: Informationstechnologie
Location: SB3 1.250
Phone: +49-6159-71 1453  Fax: +49-6159-71 2986

GSI Helmholtzzentrum für Schwerionenforschung GmbH
Planckstraße 1
64291 Darmstadt
www.gsi.de

Gesellschaft mit beschränkter Haftung
Sitz der Gesellschaft: Darmstadt
Handelsregister: Amtsgericht Darmstadt, HRB 1528

Geschäftsführung: Ursula Weyrich
Professor Dr. Karlheinz Langanke
Jörg Blaurock

Vorsitzende des Aufsichtsrates: St Dr. Georg Schütte
Stellvertreter: Ministerialdirigent Dr. Rolf Bernhardt



More information about the lustre-discuss mailing list