[Lustre-discuss] mpi-io support

[Robert Latham]

On Thu, May 08, 2008 at 12:35:11PM -0600, Marty Barnaby wrote:
> To return to this discussion, in recent testing, I have found that writing 
> to a Lustre FS via a higher level library, like PNetCDF, fails because the 
> default for value for romio_ds_write is not disable. This is set in the 
> mpich code in the file /src/mpi/romio/adio/common/ad_hints.c
>
> I believe it has something to do with locking issues. I'm not sure how best 
> to handle this, I'd prefer the data sieving default be disable, though I 
> don't know all the implications there. Maybe an ad_lustre_open should be a 
> place where the  _ds_  hints are set to disable.

I'm not sure why you want to turn off the data sieving optimization
for libraries such as pnetcdf and HDF5 which are almost certainly
going to exercise said optimization.  

However, if you really want to, just set up whatever you want in your
own MPI_Info structure and pass it to ncmpi_open, ncmpi_create, or
H5Pset_fapl_mpio.    

It sounds like you have a lot more information about what does and
does not work for your site.  I'm not sure it's appropriate to
unilaterally turn off this hint for all or even most codes running
against all lustre deployments.  

Don't get me wrong... there is undoubtedly a lot of lustre-specific
hint-tuning that can be done in ad_lustre. 

==rob

-- 
Rob Latham
Mathematics and Computer Science Division    A215 0178 EA2D B059 8CDF
Argonne National Lab, IL USA                 B29D F333 664A 4280 315B