[Lustre-devel] extremely slow reads at 1024 procs

Dave Hysom hysom1 at llnl.gov
Tue Jun 14 16:45:59 PDT 2011


I've just joined to list and will be searching the archives in case
this has been addressed before -- so please point me to a past
thread as appropriate.


We have ~100K files.  Each is 8Mb.  Each is read once, by a single
processor, using fread.  Once we reach a certain number of processors
(512 or 1024) some of the reads take enormous amounts of time, up to
15 minutes.  Our files have stripe=2, which I'm told should be adequate.
Our application is I/O intensive.

Has anyone had similar experience, and/or have a clue what might be
going on, and/or let me know what additional details I should include?

thanks, David

More information about the lustre-devel mailing list