[lustre-discuss] Lustre and OFED

[E.S. Rosenberg]

On Fri, Jul 28, 2017 at 6:22 PM, Ben Evans <bevans at cray.com> wrote:

>
>
> On 7/28/17, 11:12 AM, "Harald van Pee" <pee at hiskp.uni-bonn.de> wrote:
>
> >Hello
> >
> >On Friday 28 July 2017 15:48:12 Ben Evans wrote:
> >> Eli, just to clarify are you talking about using the in-kernel OFED vs.
> >>a
> >> vendor (Mellanox) OFED, or
> >
> >In our case we are using the OFED of the debian distribution used.
>
I am using the IB support that ships with Debian/CentOS/mainline kernel and
did not install any OFED/Mellanox OFED package.
As far as I can tell RDMA does work. (using the various test tools
suggested here https://community.mellanox.com/docs/DOC-2086)

> >
> >> are you talking about using the ConnectX-3
> >> hardware in IPoIB mode and just using it as a faster Ethernet?
> >
> >is  possible? How one have to do this?
>
> You'd configure the lustre LNET to use it like any other ethernet device.
> The downside of this is that it's slower due to a lack of RDMA and other
> features that IB has.  I'm not sure if there's a real upside to it.
>

Now you've made me unsure of what whether or not my Lustre install is using
RDMA, how should I be able to tell (we are definitely using IPoIB/o2ib)?
Thanks,
Eli

>
> >
> >Harald
> >
> >
> >>
> >> -Ben Evans
> >>
> >> From: lustre-discuss
> >>
> >><lustre-discuss-bounces at lists.lustre.org<mailto:lustre
> -discuss-bounces at li
> >>s
> >> ts.lustre.org>> on behalf of "E.S. Rosenberg"
> >> <esr+lustre at mail.hebrew.edu<mailto:esr+lustre at mail.hebrew.edu>> Date:
> >> Thursday, July 27, 2017 at 4:55 PM
> >> To:
> >>
> >>"lustre-discuss at lists.lustre.org<mailto:lustre-discuss at lists.lustre.org
> >"
> >>
> >><lustre-discuss at lists.lustre.org<mailto:lustre-discuss at lists.lustre.org
> >>
> >> Subject: [lustre-discuss] Lustre and OFED
> >>
> >> Hi all,
> >>
> >> How 'needed' is OFED for Lustre? In the LUG talks it is mentioned every
> >> once in a while and that got me thinking a bit.
> >>
> >> What things are gained by installing OFED? Performance? Accurate traffic
> >> reports?
> >>
> >> Currently I am using a lustre system without OFED but our IB hardware is
> >> from the FDR generation so not bleeding edge and probably doesn't need
> >> OFED because of that....
> >>
> >> Thanks,
> >> Eli
> >>
> >> Tech specs:
> >> Servers: CentOS 6.8 + Lustre 2.8 (kernel from Lustre RPMs)
> >> Clients: Debian + kernel 4.2 + Lustre 2.8
> >> IB: ConnectX-3 FDR
> >
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20170730/073ad642/attachment.htm>