[lustre-discuss] Multiple MGS interfaces config

[Exec Unerd]

My environment has both TCP and IB clients, so my Lustre config has to
accommodate both, but I'm having a hard time figuring out the proper syntax
for it. Theoretically, I should be able to use comma-separated interfaces
in the mgsnode parameter like this:

--mgsnode=192.168.10.1 at tcp0,172.16.10.1 at o2ib
--mgsnode=192.168.10.2 at tcp0,172.16.10.2 at o2ib

The problem is, this doesn't work for all clients all the time ...
randomly. It would work, then it wouldn't. Googling, I found some known
defects saying that the comma delimiter didn't work as per the manual and
recommending alternate syntaxes like using the colon instead of a comma. I
know what the manuals *say*about the syntax, I'm just having trouble
getting it to work.

This seems to affect only the TCP clients; at least I haven't seen it
affect any of the IB clients. It may be a comma parsing problem or
something else.

I have two questions for the group:

   1. Is there a known-working method for using both TCP and IB interface
   NIDs for the MGS in this manner?
   2. What's the best way to trace the TCP client interactions to see where
   it's breaking down?

Versions in use:
kernel: 2.6.32-504.23.4.el6.x86_64
lustre: lustre-2.7.58-2.6.32_504.23.4.el6.x86_64_g051c25b.x86_64
zfs: zfs-0.6.4-76_g87abfcb.el6.x86_64

My lustre.conf contents:
options lnet networks="o2ib0(ib1),tcp0(ixgbe1)"
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20150922/a45d0623/attachment.htm>