<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:SimSun;
panose-1:2 1 6 0 3 1 1 1 1 1;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:"\@SimSun";
panose-1:2 1 6 0 3 1 1 1 1 1;}
/* Style Definitions */
.MsoChpDefault
{mso-style-type:export-only;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 90.0pt 72.0pt 90.0pt;}
div.WordSection1
{page:WordSection1;}
--></style>
</head>
<body lang="ZH-CN" style="word-wrap:break-word">
<div class="WordSection1">
<div>
<p style="margin:0cm"><span lang="EN-US" style="font-family:"Arial",sans-serif;color:#0E101A">Hi all,<o:p></o:p></span></p>
<p style="margin:0cm"><span lang="EN-US" style="font-family:"Arial",sans-serif;color:#0E101A"><o:p> </o:p></span></p>
<p style="margin:0cm"><span lang="EN-US" style="font-family:"Arial",sans-serif;color:#0E101A">I have a question regarding the performance bottleneck of a single OST. We have high-performance SAN storage, which can provide high raw write bandwidth with a single
LUN (e.g., about 10GiB/s). <o:p></o:p></span></p>
<p style="margin:0cm"><span lang="EN-US" style="font-family:"Arial",sans-serif;color:#0E101A"><o:p> </o:p></span></p>
<p style="margin:0cm"><span lang="EN-US" style="font-family:"Arial",sans-serif;color:#0E101A">We evaluate the write performance of a single OST (based on a single LUN) via obdfilter-survey in OSS:<o:p></o:p></span></p>
<p style="margin:0cm"><em><span lang="EN-US" style="font-family:"Arial",sans-serif;color:#0E101A">$> nobjlo=32 nobjhi=32 thrlo=32 thrhi=32 rszlo=2048 rszhi=2048 size=102400 targets="l_lfs-OST0000" case=disk sh obdfilter-survey</span></em><span lang="EN-US" style="font-family:"Arial",sans-serif;color:#0E101A"><o:p></o:p></span></p>
<p style="margin:0cm"><span lang="EN-US" style="font-family:"Arial",sans-serif;color:#0E101A"><o:p> </o:p></span></p>
<p style="margin:0cm"><span lang="EN-US" style="font-family:"Arial",sans-serif;color:#0E101A">The output write bandwidth of this single OST is around 1.2GiB/s. However, if we increase the write pressure of obdfilter-survey in OSS:<o:p></o:p></span></p>
<p style="margin:0cm"><em><span lang="EN-US" style="font-family:"Arial",sans-serif;color:#0E101A">$> nobjlo=64 nobjhi=64 thrlo=64 thrhi=64 rszlo=2048 rszhi=2048 size=102400 targets="l_lfs-OST0000" case=disk sh obdfilter-survey</span></em><span lang="EN-US" style="font-family:"Arial",sans-serif;color:#0E101A"><o:p></o:p></span></p>
<p style="margin:0cm"><span lang="EN-US" style="font-family:"Arial",sans-serif;color:#0E101A"><o:p> </o:p></span></p>
<p style="margin:0cm"><span lang="EN-US" style="font-family:"Arial",sans-serif;color:#0E101A">The output write bandwidth of this single OST cannot increase anymore (i.e., still around 1.2GiB/s), which cannot further exploit the write bandwidth of a single LUN.
Also, if we use iostat to check this LUN, its write pressure does not increase (e.g., aqu-sz remains at around 20).<o:p></o:p></span></p>
<p style="margin:0cm"><span lang="EN-US" style="font-family:"Arial",sans-serif;color:#0E101A"><o:p> </o:p></span></p>
<p style="margin:0cm"><span lang="EN-US" style="font-family:"Arial",sans-serif;color:#0E101A">We consider that obdfilter-survey evaluation does not incur the impact on the client and LNet, it would only include oss layer, ofd layer, osd-ldiskfs layer, and ldiskfs
layer. For the ldiskfs layer, we think it would not be the performance bottleneck, since we have measured its performance by mounting the single LUN via ldiskfs and issuing "dd" via multi-threading (the write performance can reach 7-8GiB/s with 64 writer threads).<o:p></o:p></span></p>
<p style="margin:0cm"><span lang="EN-US" style="font-family:"Arial",sans-serif;color:#0E101A"><o:p> </o:p></span></p>
<p style="margin:0cm"><span lang="EN-US" style="font-family:"Arial",sans-serif;color:#0E101A">Can anyone provide some insights of performance bottleneck for a single OST? Is there any mechanism to control the write concurrency of a single OST? I also disable
writecache and readcache during the evaluation, so it should exclude the impact of pagecache. Thanks.<o:p></o:p></span></p>
<p style="margin:0cm"><span lang="EN-US" style="font-family:"Arial",sans-serif;color:#0E101A"><o:p> </o:p></span></p>
<p style="margin:0cm"><span lang="EN-US" style="font-family:"Arial",sans-serif;color:#0E101A">Regards,<o:p></o:p></span></p>
<p style="margin:0cm"><span lang="EN-US" style="font-family:"Arial",sans-serif;color:#0E101A">Zuoru<o:p></o:p></span></p>
</div>
</div>
</body>
</html>