<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=ISO-8859-1" http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
Hi Rob,<br>
<br>
Here is the new patch for lustre adio driver, based on MPICH2-1.0.7.<br>
<br>
Per our discussion, I did the following changes:<br>
<br>
1) rename the hints<br>
- romio_lustre_CO -> romio_lustre_co_ratio<br>
- romio_lustre_bigsize -> romio_lustre_coll_threshold<br>
<br>
2) remove the two confusing hints<br>
I removed "contig_data" and "samesize", then use
ADIOI_Calc_others_req() instead of ADIOI_LUSTRE_Calc_others_req().<br>
<br>
I have tested the patch in a small scale environment.<br>
<br>
Please check and let me know if you have any questions.<br>
<br>
Thanks,<br>
LiuYing<br>
<br>
Robert Latham wrote:
<blockquote cite="mid:20090318193629.GD1408@mcs.anl.gov" type="cite">
<pre wrap="">On Mon, Mar 16, 2009 at 03:41:47PM +0800, emoly.liu wrote:
</pre>
<pre wrap=""><!---->Thanks for the documentation. These explanations are good, but now
we've found a few problems. The naming issues are rather minor, but
some of your hints aren't compliant with the MPI-IO spec,
unfortunately.
</pre>
<blockquote type="cite">
<blockquote type="cite">
<pre wrap="">romio_lustre_CO
</pre>
</blockquote>
<pre wrap="">In stripe-contiguous IO pattern, each OST will be accessed by a group of
IO clients. CO means *C*lient/*O*ST ratio, the max. number of IO clients
for each OST.
CO=1 by default.
</pre>
</blockquote>
<pre wrap=""><!---->
To make it more clear, how about calling it "romio_lustre_co_ratio" ?
</pre>
<blockquote type="cite">
<blockquote type="cite">
<pre wrap="">romio_lustre_bigsize
</pre>
</blockquote>
<pre wrap="">We won't do collective I/O if this hint is set and the IO request size
is bigger than this value. That's because when the request size is big,
the collective communication overhead increases and the benefits from
collective I/O becomes limited.
</pre>
</blockquote>
<pre wrap=""><!---->
Instead of 'bigzise' how about "romio_lustre_coll_highwater" or
"romio_lustre_coll_threshold"?
</pre>
<blockquote type="cite">
<blockquote type="cite">
<pre wrap="">romio_lustre_contig_data
romio_lustre_samesize
</pre>
</blockquote>
<pre wrap="">They are two hints to tell the driver whether the request data are
contiguous and whether each request IO has the same size. If they
are both "yes", we can optimize ADIOI_LUSTRE_Calc_others_req() by
removing MPI_Alltoall(). Because each process can easily calculate
the pairs of offset and length for each request without collective
communication. BTW, currently only when they are both positive, the
optimization can work. In the future, probably some efforts will be
made to other conditions.
</pre>
</blockquote>
<pre wrap=""><!---->
OK, here's the one with the major problem. RobR reminds me that
MPI-IO requires hints to be optional and cannot cause incorrect
behavior. A user supplying these hints and then giving you data that
is noncontiguous or not of the same size would cause incorrect
behavior, so these aren't appropriate.
Is there a way you can check what the caller is doing? caller can lie
to you via hints, but ROMIO still has to give the right answer. RobR
thought maybe MPI_Allreduce or something along those lines before the
MPI_Alltoall would let you check.
Your other hints make a lot of intuitive sense to me. Is this one a
big win, though? If MPI_Alltoall is giving you a big headache, then
maybe there is a more fundamental problem with the MPI implementation?
Thanks
==rob
</pre>
</blockquote>
<br>
<br>
<pre class="moz-signature" cols="72">--
Best regards,
LiuYing
System Software Engineer, Lustre Group
Sun Microsystems ( China ) Co. Limited
</pre>
</body>
</html>