[lustre-discuss] Multiple MGS interfaces config

[Exec Unerd]

> I'm not sure if I have understood your setup correctly.

In this case, the clients are a combination of all three: some are o2ib
only, some tcp only, and some o2ib+tcp with tcp as failover.

It sounds like I need a combination of configurations, one for the OSSes
and one for each client type.

So if I used this parameter in the OST,
--mgsnode="172.16.10.1 at o2ib0,192.168.10.1 at tcp0"

Then configured the modprobe.d/lustre.conf appropriately on the clients
tcp: options lnet networks="tcp0(ixgbe1)"
o2ib: options lnet networks="o2ib0(ib1)"
both: options lnet networks="o2ib0(ib1),tcp0(ixgbe1)"

And use these mount parameters:
tcp: mount -v -t lustre 192.168.10.1 at tcp0:/testfs /mnt/testfs
o2ib: mount -v -t lustre 172.16.10.1 at o2ib0:/testfs /mnt/testfs
both: mount -v -t lustre 172.16.10.1 at o2ib0,192.168.10.1 at tcp0:/testfs
/mnt/testfs

Everything should be happy?

On Thu, Sep 24, 2015 at 9:12 AM, Martin Hecht <hecht at hlrs.de> wrote:

> On 09/24/2015 05:33 PM, Chris Hunter wrote:
> > [...]
> >>    2. What's the best way to trace the TCP client interactions to see
> >> where
> >>    it's breaking down?
> > If lnet is running on the client, you can try "lctl ping"
> > eg) lctl ping 172.16.10.1 at o2ib
> >
> > I believe a lustre mount uses ipoib for initial handshake with a mds
> > o2ib interfaces. You should make sure regular ping over ipoib is
> > working before mounting lustre.
> if the client and the server is on the same network, yes, it's a good
> starting point. But it's not a prerequisite. In general you can have an
> lnet router in-between or have different ip subnets for ipoib, so you
> can't ping on the ipoib layer, but you can still lctl ping the whole
> path (although you could verify that you can ip ping to the next hop at
> least).
>
> We also have a case in which we tried to block ipoib completely with
> iptables, but we still could lctl ping, even after rebooting the host
> and ensuring that the firewall was up before loading the lnet module.
> So, I doubt that ipoib is needed at all for establishing the o2ib
> connection.
>
>
>
> _______________________________________________
> lustre-discuss mailing list
> lustre-discuss at lists.lustre.org
> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20150927/63cf1f85/attachment.htm>