[lustre-discuss] "Not on preferred path" error

[Lewis Hyatt]

Hello-

I am having an issue with a lustre 1.8 array that I have little hope
of figuring out on my own, so I thought I would try here to see if
anyone might know what this warning/error means. Our array was built
by Terascala, which no longer exists, so we have no support for it and
little documentation (and not much in-house knowledge). I see this
complaint "Not on preferred path" on the GUI that we have, which I
assume was something custom made by Terascala, and I am not sure even
what path it is referring to; we use infiniband for all connections
and it could relate to this, but not sure. We see this error on 3 of
the 12 OSTs. More specifically, we have 2 OSSs, each handling 6 OSTs,
and all 3 of the "not on optimal path" OSTs are on the same OSS.

We do not know if it's related, but this same OSS is in a very bad
state, with very high load average (200), very high I/O wait time, and
taking many seconds to respond to each read request, making the array
more or less unusable. That's the problem we are trying to fix.

I realize there's not much hope for anyone to help us with that given
how little information I am able to provide. But I was hoping someone
out there might know what this "not on optimal path" error means, and
if it matters for anything or not, so we have somewhere to start.
Thanks very much!

I could provide screen shots of the management GUI we have, if it
would be informative.

-Lewis