[Lustre-discuss] Shared pointers, file locking and MPI-IO
adilger at sun.com
Mon Oct 27 12:20:14 PDT 2008
On Oct 23, 2008 16:06 +0200, S�bastien Buisson wrote:
> We are using MPI-IO on Lustre, so that we can achieve good performance
> when writing or reading the same file from multiple processes and clients.
> In the shared pointers read/write routines, MPI-IO makes an intensive
> use of file locks (for instance, see the file
> src/mpi/romio/adio/common/ad_get_sh_fp.c from mpich2-1.0.7). The problem
> is that file locking support in Lustre is very limited today:
> - there is the localflock option for the local mode. In this mode, locks
> are coherent on a node (a single-node flock), but not across all clients;
> - there is the flock option for the consistent mode. In this mode, locks
> are coherent across all clients. But the Lustre Operations Manual
> includes a big warning about this mode, saying that it can affect
> performance and stability.
> That being said, does anybody in the Lustre community know about current
> alternatives to the MPI-IO shared pointers read/write routines, which do
> not use the file locking mechanism but provide the same functionality?
Tom Wang is our MPI-IO expert, and has done a lot of work to improve the
performance of MPI-IO with Lustre, so he can probably answer this question
better than I can.
> By the way, do we have an idea of the Lustre version that will provide
> the reliable version of the flock consistent mode?
There are in fact a number of users that enable the flock support in
conjunction with MPI-IO already. The reason we haven't made it enabled
for general usage is that there may still be some bad interactions with
flock under less well-behaved applications.
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.
More information about the lustre-discuss