[Lustre-discuss] mpi-io support

Tom.Wang Tom.Wang at Sun.COM
Thu May 8 12:59:29 PDT 2008


Hi

Marty Barnaby wrote:
> To return to this discussion, in recent testing, I have found that 
> writing to a Lustre FS via a higher level library, like PNetCDF, fails 
> because the default for value for romio_ds_write is not disable. This 
> is set in the mpich code in the file /src/mpi/romio/adio/common/ad_hints.c
You can use MPI_Info_set to disable romio_ds_write.  What is the fail?  
flock? since data-sieving need flock.
>
> I believe it has something to do with locking issues. I'm not sure how 
> best to handle this, I'd prefer the data sieving default be disable, 
> though I don't know all the implications there. 
I agree data sieving should be disable. And also it check the contiguous 
buftype or filetype only by fileview, which is not enough sometimes, and 
trigger unnecessary read-modify-write even for contiguous 
write(especially for those higher level library, if you choose 
collective write). Since lustre has client cache and also the overhead 
of flock and read-modify-write, so I doubt the performance improvements 
we could  get from data-sieving on lustre, although I do not have 
performance data to prove that.
> Maybe an ad_lustre_open should be a place where the  _ds_  hints are 
> set to disable.
Yes, we should disable this for stride write in lustre. ad_lustre_open 
seems a right place to do this.

Thanks
WangDi
>
> Marty Barnaby
>
>
> Weikuan Yu wrote:
>> Andreas Dilger wrote:
>>   
>>> On Mar 11, 2008  16:10 -0600, Marty Barnaby wrote:
>>>     
>>>> I'm not actually sure what ROMIO abstract device the multiple CFS
>>>> deployments I utilize were defined with. Probably just UFS, or maybe NFS.
>>>> Did you have a recommended option yourself.
>>>>       
>>> The UFS driver is the one used for Lustre if no other one exists.
>>>
>>>     
>>>> Besides the fact that most of the adio that were created over the years are
>>>> completely obsolete and could be cleaned from ROMIO, what will the new one
>>>> for Lustre offer? Particularly with respect to controls via the lfs utility
>>>> that I can  already get?
>>>>       
>>> There is improved collective IO that aligns the IO on Lustre stripe
>>> boundaries.  Also the hints given to the MPIIO layer (before open,
>>> not after) result in lustre picking a better stripe count/size.
>>>
>>>     
>>
>> In addition, the one integrated into MPICH2-1.0.7 contains direct I/O
>> support. Lockless I/O support was purged out due into my lack of
>> confidence in low-level file system support. But it can be revived when
>> possible.
>>
>> --
>> Weikuan Yu <+> 1-865-574-7990
>> http://ft.ornl.gov/~wyu/
>>
>>   
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-discuss
>   


-- 
Regards,
Tom Wangdi    
--
Sun Lustre Group
System Software Engineer 
http://www.sun.com




More information about the lustre-discuss mailing list