[Lustre-discuss] MPI-IO / ROMIO support for Lustre

[Mark Dixon]

On Mon, 1 Nov 2010, Martin Pokorny wrote:
...
> FWIW, we've been using MPICH2's MPI-IO/ROMIO/ADIO with Lustre (v 1.8)
> for several months now, and it's been working reliably. We do mount the
> Lustre filesystem with "flock"; at one time I thought it necessary, but
> I don't recall if I verified that after the initial problems with MPI-IO
> were resolved. Only a recent MPICH2 will have a working
> MPI-IO/ROMIO/ADIO for Lustre; perhaps the code would work with OpenMPI
> and MVAPICH2 as well.
...

Is MPICH2 where ROMIO is developed these days? I found it pretty difficult 
to work out where the public face of its development was...

On Tue, 2 Nov 2010, Larry wrote:

> we use  "localflock"  in order to work with MPI-IO. "flock" may
> consume more addtional resource than "localflock".
...

Thanks for posting both of you, much appreciated :)

Cripes, a tie (sort of). I think the following supports the "flock" 
option, but I would appreciate anyone putting-in a counter-opinion:

1) Is it a naive view that, if ROMIO asks for an flock, it needs it? And 
that if it doesn't on Lustre, then eventually ROMIO will be developed to 
stop asking for them?

2) A message in the list archive says that Cray recommend "flock" for 
their clusters, and it sounds like they use an enhanced version of ROMIO in 
their MPT product.


The ADIO driver for Lustre certainly looks like it's still being actively 
worked-on to get it to maturity, and that various MPI implementations 
still need time to incorporate those changes.

I note that the main MPI releases we use on our cluster (OpenMPI 1.4 and 
MVAPICH2 1.4 - we're a year behind) have V04 of ad_lustre, but MVAPICH2 is 
now closest to MPICH2 as it has since moved to V05. Looks like I need to 
do a software refresh... and recommend our MPI-IO users to use MVAPICH2 
for the time being.

Thanks,

Mark
-- 
-----------------------------------------------------------------
Mark Dixon                       Email    : m.c.dixon at leeds.ac.uk
HPC/Grid Systems Support         Tel (int): 35429
Information Systems Services     Tel (ext): +44(0)113 343 5429
University of Leeds, LS2 9JT, UK
-----------------------------------------------------------------