[lustre-discuss] MPI-IO performance doesn't scale with Lustre

Dong-In Kang dikang at gmail.com
Wed Apr 8 08:35:11 PDT 2020


I'm running IOR benchmark on a big shared machine having no less than 10
It is connected to Lustre FS (v. 2.10).
IOR benchmark measure file system write performance.
I'm seeing performance issues in two cases.
I know it can be either MPI problem or Luster problem
I want to get help if there is anything I should look into Luster side.

1. Independent file write test.
Theoretically this must scale very well.
It does scale well when MPI processes are put into a single socket (not
stritly linear, but quite well up to 10 MPI processes)
However, the same number of MPI processes are spread across multiple
sockets, its scalability is a lot worse. (OK up to 4 MPI processes, but
doesn't increase beyond that).
I expected both should show similar performance. But they are quite
Any idea of why?
As an extra information, when I tested the same IOR test with ramdisk
target, both scales very well.

2. Single shared file write test.
Similar problem shown in the first issue, but a lot worse performance.
When MPI processes are spread across multiple sockets, its performance gets
worse as the number of MPI processes increase.

Any comments/help would be appreciated.


Jesus is My Lord!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20200408/df5384e7/attachment.html>

More information about the lustre-discuss mailing list