<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=ISO-8859-1" http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
To refer to your original discussion, how were you writing your 1TB
file? I'm mainly wondering about the blocksize of the individual writes.<br>
<br>
Marty Barnaby<br>
<br>
<br>
Lundgren, Andrew wrote:
<blockquote
cite="mid:A3B72EB76E8D4647BCA6048E1C69011B3D3045800F@idc1embx0003.corp.global.level3.com"
type="cite">
<pre wrap="">I get it.
Thanks!
</pre>
<blockquote type="cite">
<pre wrap="">-----Original Message-----
From: Kumaran Rajaram [<a class="moz-txt-link-freetext" href="mailto:krajaram@sgi.com">mailto:krajaram@sgi.com</a>]
Sent: Friday, May 30, 2008 3:07 PM
To: Lundgren, Andrew
Cc: Roger Spellman; <a class="moz-txt-link-abbreviated" href="mailto:lustre-discuss@lists.lustre.org">lustre-discuss@lists.lustre.org</a>
Subject: RE: [Lustre-discuss] Performance Drop creating big files
I meant use file-size that are 4x the RAM size :-). So if
your RAM size
is 4GB, use atleast 16GB file-sizes for the benchmarks.
On Fri, 2008-05-30 at 15:03 -0600, Lundgren, Andrew wrote:
</pre>
<blockquote type="cite">
<pre wrap="">Are you suggesting he use machines with 40G of RAM to work
</pre>
</blockquote>
<pre wrap="">with 10G files?
</pre>
<blockquote type="cite">
<pre wrap="">We have many 800-900G files... I am not sure that is a
</pre>
</blockquote>
<pre wrap="">realistic number.
</pre>
<blockquote type="cite">
<pre wrap="">--
Andrew
</pre>
<blockquote type="cite">
<pre wrap="">-----Original Message-----
From: <a class="moz-txt-link-abbreviated" href="mailto:lustre-discuss-bounces@lists.lustre.org">lustre-discuss-bounces@lists.lustre.org</a>
[<a class="moz-txt-link-freetext" href="mailto:lustre-discuss-bounces@lists.lustre.org">mailto:lustre-discuss-bounces@lists.lustre.org</a>] On Behalf Of
Kumaran Rajaram
Sent: Friday, May 30, 2008 3:01 PM
To: Roger Spellman
Cc: <a class="moz-txt-link-abbreviated" href="mailto:lustre-discuss@lists.lustre.org">lustre-discuss@lists.lustre.org</a>
Subject: Re: [Lustre-discuss] Performance Drop creating big files
Roger,
I would suspect the Lustre client side caching
</pre>
</blockquote>
</blockquote>
<pre wrap="">influencing your write
</pre>
<blockquote type="cite">
<blockquote type="cite">
<pre wrap="">performance. 10GB is not adequate, try atleast 4x of the
</pre>
</blockquote>
</blockquote>
<pre wrap="">RAM size for
</pre>
<blockquote type="cite">
<blockquote type="cite">
<pre wrap="">file-sizes. Try doing the same tests with O_DIRECT flag
</pre>
</blockquote>
</blockquote>
<pre wrap="">as it'd truly
</pre>
<blockquote type="cite">
<blockquote type="cite">
<pre wrap="">measure your disk I/O performance bypassing FS + buffer cache.
HTH,
-Kums
On Fri, 2008-05-30 at 16:25 -0400, Roger Spellman wrote:
</pre>
<blockquote type="cite">
<pre wrap="">I am seeing the following odd behavior. I have several
</pre>
</blockquote>
</blockquote>
</blockquote>
<pre wrap="">OSSes, each
</pre>
<blockquote type="cite">
<blockquote type="cite">
<blockquote type="cite">
<pre wrap="">with a 7T RAID 5.
If I use a single client to create a single 1T file
</pre>
</blockquote>
</blockquote>
</blockquote>
<pre wrap="">which is striped
</pre>
<blockquote type="cite">
<blockquote type="cite">
<blockquote type="cite">
<pre wrap="">to a single OST, the performance starts off at about
</pre>
</blockquote>
</blockquote>
</blockquote>
<pre wrap="">400 MB/s (which
</pre>
<blockquote type="cite">
<blockquote type="cite">
<blockquote type="cite">
<pre wrap="">is typical for my HW), then gradually decreases, until it
</pre>
</blockquote>
<pre wrap="">reaches 250
</pre>
<blockquote type="cite">
<pre wrap="">MB/s. I've seen this with both IOZone and dd.
As an experiment, I wrote a script that creates two hundred
</pre>
</blockquote>
<pre wrap="">10G files,
</pre>
<blockquote type="cite">
<pre wrap="">without removing them. Again, the performance starts off
</pre>
</blockquote>
<pre wrap="">at 400 MB/s.
</pre>
<blockquote type="cite">
<pre wrap="">But, the performance stays nearly the same throughout the test.
The only difference between these tests is that in the
</pre>
</blockquote>
</blockquote>
</blockquote>
<pre wrap="">second case,
</pre>
<blockquote type="cite">
<blockquote type="cite">
<blockquote type="cite">
<pre wrap="">there are lots of opens & closes, and in the first case,
</pre>
</blockquote>
<pre wrap="">just a single
</pre>
<blockquote type="cite">
<pre wrap="">open and close.
Can anyone explain what is happening here, and how to
</pre>
</blockquote>
<pre wrap="">possible fix it?
</pre>
<blockquote type="cite">
<pre wrap="">
Thanks.
Roger Spellman
Sr. Staff Engineer
Terascala, Inc.
_______________________________________________
Lustre-discuss mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Lustre-discuss@lists.lustre.org">Lustre-discuss@lists.lustre.org</a>
<a class="moz-txt-link-freetext" href="http://lists.lustre.org/mailman/listinfo/lustre-discuss">http://lists.lustre.org/mailman/listinfo/lustre-discuss</a>
</pre>
</blockquote>
<pre wrap="">_______________________________________________
Lustre-discuss mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Lustre-discuss@lists.lustre.org">Lustre-discuss@lists.lustre.org</a>
<a class="moz-txt-link-freetext" href="http://lists.lustre.org/mailman/listinfo/lustre-discuss">http://lists.lustre.org/mailman/listinfo/lustre-discuss</a>
</pre>
</blockquote>
</blockquote>
</blockquote>
<pre wrap=""><!---->_______________________________________________
Lustre-discuss mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Lustre-discuss@lists.lustre.org">Lustre-discuss@lists.lustre.org</a>
<a class="moz-txt-link-freetext" href="http://lists.lustre.org/mailman/listinfo/lustre-discuss">http://lists.lustre.org/mailman/listinfo/lustre-discuss</a>
</pre>
</blockquote>
<br>
</body>
</html>