[lustre-discuss] Lustre and OFED

[E.S. Rosenberg]

Jeff (and Grigory - offlist),

Thanks for your fast replies!
On Fri, Jul 28, 2017 at 12:09 AM, Jeff Johnson <
jeff.johnson at aeoncomputing.com> wrote:

> Eli,
>
> The biggest driver is usually the drivers. Newer Mellanox hardware not yet
> supported, or supported well, by kernel IB. Way back in the days of old
> there were some interoperability issues where everything (clients and
> servers) needed to be the same drivers and libraries but much of that was
> cleaned up. There could be situations where OFED is needed on the server
> side to support something under the Lustre layer like OST or MDT block
> devices via iSER, SRP, NVMeF, etc.
>
> There may be other reasons but those are off the top of my head.
>
So currently everything seems to be working just fine without OFED, my only
complaint is that the normal Linux interface counters don't report traffic
properly which means I have to write my own perfquery wrappers for tools
like zabbix etc.

I may try adding OFED if I have time at some point but I hope by then to at
least have moved our servers to CentOS 7.3 + Lustre 2.9/10.

Has anyone ever run benchmarks of vanilla vs. OFED?
Thanks again,
Eli

>
> --Jeff
>
> On Thu, Jul 27, 2017 at 4:55 PM, E.S. Rosenberg <
> esr+lustre at mail.hebrew.edu> wrote:
>
>> Hi all,
>>
>> How 'needed' is OFED for Lustre? In the LUG talks it is mentioned every
>> once in a while and that got me thinking a bit.
>>
>> What things are gained by installing OFED? Performance? Accurate traffic
>> reports?
>>
>> Currently I am using a lustre system without OFED but our IB hardware is
>> from the FDR generation so not bleeding edge and probably doesn't need OFED
>> because of that....
>>
>> Thanks,
>> Eli
>>
>> Tech specs:
>> Servers: CentOS 6.8 + Lustre 2.8 (kernel from Lustre RPMs)
>> Clients: Debian + kernel 4.2 + Lustre 2.8
>> IB: ConnectX-3 FDR
>>
>> _______________________________________________
>> lustre-discuss mailing list
>> lustre-discuss at lists.lustre.org
>> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>>
>>
>
>
> --
> ------------------------------
> Jeff Johnson
> Co-Founder
> Aeon Computing
>
> jeff.johnson at aeoncomputing.com
> www.aeoncomputing.com
> t: 858-412-3810 x1001 <(858)%20412-3810>   f: 858-412-3845
> <(858)%20412-3845>
> m: 619-204-9061 <(619)%20204-9061>
>
> 4170 Morena Boulevard, Suite D - San Diego, CA 92117
>
> High-Performance Computing / Lustre Filesystems / Scale-out Storage
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20170728/5cfe2f15/attachment.htm>