[lustre-discuss] IPoIB best practises

Åke Sandgren ake.sandgren at hpc2n.umu.se
Sun Jan 16 23:10:17 PST 2022

On 1/17/22 2:36 AM, Angelos Ching via lustre-discuss wrote:
> Hi Eli,
> Yes & no; part of my info is a bit rusty because I carried them from
> version around 2.10. MR is now turned on by default.
> But you'll need to have an IP setup on each IPoIB interface, and for all
> ib0 & all ib1 interface, they should be in different subnet. Eg: all ib0
> on and all ib1 on

The multirail setup we have is that both ib0 and ib1 are on the same
subnet, that's how DDN configured it for us.

ip a s ib0 | grep inet
    inet brd scope global ib0
ip a s ib1 | grep inet
    inet brd scope global ib1

and the modprobe config is

options lnet networks="o2ib1(ib0,ib1)"

Ake Sandgren, HPC2N, Umea University, S-90187 Umea, Sweden
Internet: ake at hpc2n.umu.se   Phone: +46 90 7866134 Fax: +46 90-580 14
Mobile: +46 70 7716134 WWW: http://www.hpc2n.umu.se

More information about the lustre-discuss mailing list