[lustre-discuss] ko2iblnd optimizations for EDR

[Riccardo Veraldi]

thanks for sharing your experience.
I am running peer credits value 63 as it was for mlx4.
I did not notice problems with these settings anyway to saturate the EDR 
bandwith writing and reading using Lustre I have to do it from 2 
different clients,
so that I can reach up to 9GB/s per OSS. Otherwise from a single client 
I cannot reach more than 5GB/s. it is not clear to me why.

On 11/8/18 4:58 AM, Martin Hecht wrote:
> On 11/7/18 9:44 PM, Riccardo Veraldi wrote:
>> Anyway I Was wondering if something different is needed for mlx5 and
>> what are the suggested values in that case ?
>>
>> Anyone has experience with mlx5 LNET performance tunings ?
> Hi Riccardo,
>
> We have recently integrated mlx5 nodes into our fabric, and we had to
> reduce the values to
>
> peer_credits = 16
> concurrent_sends = 16
>
> because mlx5 doesn't support larger values for some reason. The peer_credits must have the same value in all connected lnets, even across routers (at least it used to be like this. I believe we are currently running some Lustre 2.5.x derivates on the server side, and newer versions on the various clients).
>
> kind regards,
> Martin
>
>
> _______________________________________________
> lustre-discuss mailing list
> lustre-discuss at lists.lustre.org
> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org