[Lustre-discuss] mpi-io support

[Phil Dickens]

Many thanks to Marty, Wei-Keng, Tom and Andreas for their
very helpful support (I hope I didn't leave anyone out!).

Thanks very much, and I think I can now address the problem.

Phil


On Fri, 9 May 2008, Tom.Wang wrote:

> Hi,
>
> Andreas Dilger wrote:
>> On May 09, 2008  09:41 -0400, Phil Dickens wrote:
>> 
>>>   I am having similar struggles with locking on MPI-IO.
>>> I am doing a simple strided write, and it fails because
>>> of the locking. I'm a bit behind in the discussion, but
>>> is there a way to fix (workaround) this problem?? Is this
>>> something in my code, or the default driver (this is on
>>> lonestar at TACC)? I have even downloaded the most up to date
>>> version of MPICH, which I believe has a new Lustre ADIO
>>> driver, but I am running into the same issues.
>>> 
>>>   Any thoughts would be greatly appreciated!!
>>> 
>> One possibility is to mount the clients with "-o localflock", leaving all
>> of the locking internal to Lustre.  This in essence provides single-node
>> flock (i.e. coherent on that node, but not across all clients).  The other
>> alternative is "-o flock", which is coherent locking across all clients,
>> but has a noticable performance impact and may affect stability, depending
>> on the version of lustre being used (newer is better of course).
>> 
>> I'm not positive of the internals of the MPI-IO code, whether it depends
>> on flock providing a barrier across nodes, or if it does this only for
>> e.g. NFS not keeping writes coherent so they don't clobber the same page
>> when writing.
>> 
>> Tom is the expert here...
>> 
> It uses flock to keep the coherence of read-modify-write for data_sieving.
>
> You can use
>
> ierr = MPI_Info_create(&FILE_INFO_TEMPLATE);
>
> .............
> ierr = MPI_Info_set(FILE_INFO_TEMPLATE, "romio_ds_write", disable);
>
> to disable data_sieving in your application to avoid flock for stride_write.
>
> Thanks
> WangDi
>> 
>> 
>
>
> -- 
> Regards,
> Tom Wangdi    --
> Sun Lustre Group
> System Software Engineer http://www.sun.com
>