[lustre-discuss] IPoIB best practises
esr+lustre at mail.hebrew.edu
Sun Jan 16 08:38:53 PST 2022
Thanks for the answer!
According to the lecture Lauz just sent multirail is on by default no?
So I don't need to set IPs on each interface (using
interfaces/netplan/network scripts) just a single NID at the lnet level?
On Sun, Jan 16, 2022 at 5:20 PM Angelos Ching <angelosching at clustertech.com>
> Hi Eli,
> I think you can leave out the IP so long as you specific the interface,
> but IIRC you can't setup multirail via modprobe, you should setup Lnet
> using lnetctl
> Best regards,
> (Sent from mobile, please pardon me for typos and cursoriness.)
> 16/1/2022 21:38、E.S. Rosenberg <esr+lustre at mail.hebrew.edu>のメール:
> Hey Lauz,
> Thanks for the reply!
> Indeed it sounds like multirail is what I am referring to (my wrong memory
> of multirail was a failover technique allowing connection of both TCP and
> IPoIB or multiple IPoIB between nodes at the same time), but I am still
> left with a question.
> In the modprobe arguments for lnet I'm binding all the interfaces into a
> single network but I don't define IPs for the interfaces that is done at
> the OS level.
> Lustre as far as I recall does use IPoIB so IPs are needed or could I even
> leave out the IP definitions and let lustre figure out what it wants there?
> It just seems kind of excessive and adding points of failure that a node
> with 4 dual port IB cards would end up needing 8 IP addresses...
> Thanks again,
> On Sun, Jan 16, 2022 at 2:32 PM Laurence Horrocks-Barlow <
> laurence at qsplace.co.uk> wrote:
>> The limit of IPoIB is active/backup when using traditional bonding,
>> however I believe you are wanting to multirail your IB. This is achieved
>> buy using multiple LNet's (assuming it uses the same fabric), you should be
>> able to configure for active/active.
>> This should help with most of the concepts.
>> -- Lauz
>> On 16 January 2022 11:56:16 GMT, "E.S. Rosenberg" <
>> esr+lustre at mail.hebrew.edu> wrote:
>>> Hey everyone,
>>> This is probably off-topic but I can't find any documents on the subject
>>> and since Lustre uses IPoIB I suspect others here have dealt with this
>>> If I have a node connected with multiple IB links should each connected
>>> IB port have it's own IP address or is there a way similar to LACP on the
>>> Ethernet side to bond all the links and use only a single IP address to
>>> refer to the node? And what is the better method?
>>> In the past I never had this luxury, but now I'm starting a small new
>>> cluster currently made up of a few GPU nodes and a Lustre filesystem so
>>> plenty of IB ports to go around.
> lustre-discuss mailing list
> lustre-discuss at lists.lustre.org
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the lustre-discuss