[lustre-discuss] MPI-IO performance doesn't scale with Lustre

Dong-In Kang dikang at gmail.com
Wed Apr 8 08:35:11 PDT 2020


Hi,

I'm running IOR benchmark on a big shared machine having no less than 10
sockets.
It is connected to Lustre FS (v. 2.10).
IOR benchmark measure file system write performance.
I'm seeing performance issues in two cases.
I know it can be either MPI problem or Luster problem
I want to get help if there is anything I should look into Luster side.

1. Independent file write test.
Theoretically this must scale very well.
It does scale well when MPI processes are put into a single socket (not
stritly linear, but quite well up to 10 MPI processes)
However, the same number of MPI processes are spread across multiple
sockets, its scalability is a lot worse. (OK up to 4 MPI processes, but
doesn't increase beyond that).
I expected both should show similar performance. But they are quite
different.
Any idea of why?
As an extra information, when I tested the same IOR test with ramdisk
target, both scales very well.

2. Single shared file write test.
Similar problem shown in the first issue, but a lot worse performance.
When MPI processes are spread across multiple sockets, its performance gets
worse as the number of MPI processes increase.

Any comments/help would be appreciated.

David



-- 
=========
Jesus is My Lord!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20200408/df5384e7/attachment.html>


More information about the lustre-discuss mailing list