<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=ISO-8859-1" http-equiv="Content-Type">
<title></title>
</head>
<body bgcolor="#ffffff" text="#000000">
Hi rob,<br>
<br>
Robert Latham wrote:
<blockquote cite="mid:20090318193629.GD1408@mcs.anl.gov" type="cite">
<pre wrap="">On Mon, Mar 16, 2009 at 03:41:47PM +0800, emoly.liu wrote:
</pre>
<pre wrap=""><!---->
Thanks for the documentation. These explanations are good, but now
we've found a few problems. The naming issues are rather minor, but
some of your hints aren't compliant with the MPI-IO spec,
unfortunately.
</pre>
<blockquote type="cite">
<blockquote type="cite">
<pre wrap="">romio_lustre_CO
</pre>
</blockquote>
<pre wrap="">In stripe-contiguous IO pattern, each OST will be accessed by a group of
IO clients. CO means *C*lient/*O*ST ratio, the max. number of IO clients
for each OST.
CO=1 by default.
</pre>
</blockquote>
<pre wrap=""><!---->
To make it more clear, how about calling it "romio_lustre_co_ratio" ?
</pre>
</blockquote>
OK.<br>
<blockquote cite="mid:20090318193629.GD1408@mcs.anl.gov" type="cite">
<pre wrap="">
</pre>
<blockquote type="cite">
<blockquote type="cite">
<pre wrap="">romio_lustre_bigsize
</pre>
</blockquote>
<pre wrap="">We won't do collective I/O if this hint is set and the IO request size
is bigger than this value. That's because when the request size is big,
the collective communication overhead increases and the benefits from
collective I/O becomes limited.
</pre>
</blockquote>
<pre wrap=""><!---->
Instead of 'bigzise' how about "romio_lustre_coll_highwater" or
"romio_lustre_coll_threshold"?
</pre>
</blockquote>
Both of them make sense to me.<br>
<blockquote cite="mid:20090318193629.GD1408@mcs.anl.gov" type="cite">
<pre wrap="">
</pre>
<blockquote type="cite">
<blockquote type="cite">
<pre wrap="">romio_lustre_contig_data
romio_lustre_samesize
</pre>
</blockquote>
<pre wrap="">They are two hints to tell the driver whether the request data are
contiguous and whether each request IO has the same size. If they
are both "yes", we can optimize ADIOI_LUSTRE_Calc_others_req() by
removing MPI_Alltoall(). Because each process can easily calculate
the pairs of offset and length for each request without collective
communication. BTW, currently only when they are both positive, the
optimization can work. In the future, probably some efforts will be
made to other conditions.
</pre>
</blockquote>
<pre wrap=""><!---->
OK, here's the one with the major problem. RobR reminds me that
MPI-IO requires hints to be optional and cannot cause incorrect
behavior. A user supplying these hints and then giving you data that
is noncontiguous or not of the same size would cause incorrect
behavior, so these aren't appropriate.
Is there a way you can check what the caller is doing? caller can lie
to you via hints, but ROMIO still has to give the right answer. RobR
thought maybe MPI_Allreduce or something along those lines before the
MPI_Alltoall would let you check.
</pre>
</blockquote>
Hmm, it is indeed a problem, although we did get benefits from them in
our previous tests.<br>
<br>
I will check it. But currently, is it possible to make mention of the
risk with some words, just like <font size="3">"Don't set these two
hints, until you know exactly what you are doing"</font> ?<br>
If it is still inappropriate, I will remove them in this version, then
submit another patch once I figure out how to check it with low
overhead.<br>
<br>
How about you ?<br>
<blockquote cite="mid:20090318193629.GD1408@mcs.anl.gov" type="cite">
<pre wrap="">
Your other hints make a lot of intuitive sense to me. Is this one a
big win, though? If MPI_Alltoall is giving you a big headache, then
maybe there is a more fundamental problem with the MPI implementation?
</pre>
</blockquote>
Thanks for your and RobR's careful review. Your comments are very
helpful. Some problems still need more investigation.<br>
<br>
B.R.<br>
-LiuYing<br>
<blockquote cite="mid:20090318193629.GD1408@mcs.anl.gov" type="cite">
<pre wrap="">
Thanks
==rob
</pre>
</blockquote>
<br>
<br>
<pre class="moz-signature" cols="72">--
Best regards,
LiuYing
System Software Engineer, Lustre Group
Sun Microsystems ( China ) Co. Limited
</pre>
</body>
</html>