[Lustre-discuss] Added Dual-homed OSS; ethernet clients confused

[Andreas Dilger]

On Apr 22, 2008  18:08 -0600, Chris Worley wrote:
> The error specifically complains about the first OST/disk on the new
> OSS, OST0026.  It's tunefs.lustre output was:
> 
> >  On the OSS in question, for each OST, I did:
> >
> >  # tunefs.lustre --writeconf --ost
> >
> > --mgsnode="36.102.29.1 at o2ib0,36.101.29.1 at tcp0" --fsname=lfs --param
> >  sys.timeout=40 --param lov.stripesize=2M /dev/sdl
> >  Lustre:    cmd=cf003 0:lfs-OST0026-osc  1:lfs-OST0026_UUID  2:36.102.29.4 at o2ib
> >  LustreError: 15c-8: MGC36.101.29.1 at tcp: The configuration from log
> >  'lfs-client' failed (-2). This may be the result of communication
> >  errors between this node and the MGS, a bad configuration, or other
> >  errors. See the syslog for more information.

The problem is that the NID for the new OST is the IPoIB address, and this
is what the TCP client is trying to connect to.  If you specify the TCP
NID first this may help.  Also note that the client does not get the
config from the OSTs, but rather the MGS, so you need to do a --write-conf
on there.

Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.