[Lustre-discuss] Added Dual-homed OSS; ethernet clients confused

[Chris Worley]

On Wed, Apr 23, 2008 at 8:10 AM, Chris Worley <worleys at gmail.com> wrote:
>
> On Wed, Apr 23, 2008 at 12:42 AM, Andreas Dilger <adilger at sun.com> wrote:
>  >  The problem is that the NID for the new OST is the IPoIB address, and this
>  >  is what the TCP client is trying to connect to.  If you specify the TCP
>  >  NID first this may help.  Also note that the client does not get the
>  >  config from the OSTs, but rather the MGS, so you need to do a --write-conf
>  >  on there.
>
>  This is confusing as the man page for "tunefs.lustre" wants a device
>  name at the end of the command... and the device is on another OSS...
>  "/dev/sda" on the MGS is a totally different drive.  Can I use the
>  label?

Another point of confusion with tunefs.lustre.  When I add, as you
suggest (although I'm still doing this on the OSS, as I don't know how
to specify /dev/sd? on another node from the MGS), the Ethernet IP for
both tcp and o2ib, (and have tcp first), as in:

   --mgsnode="36.101.29.1 at tcp0,36.101.29.1 at o2ib0"

tunefs.lustre adds this, rather than replacing it, so now I get:

 ...
Parameters: mgsnode=36.102.29.1 at o2ib,36.101.29.1 at tcp sys.timeout=40
lov.stripesize=2M mgsnode=36.102.29.1 at o2ib,36.101.29.1 at tcp
sys.timeout=40 lov.stripesize=2M
mgsnode=36.101.29.1 at tcp,36.101.29.1 at o2ib
...

The initial setting, yesterday's change, and today's change are in
there... I have no clue which takes precedence.

Chris