<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD>
<META http-equiv=Content-Type content="text/html; charset=gb2312">
<STYLE>
BLOCKQUOTE {
MARGIN-TOP: 0px; MARGIN-BOTTOM: 0px; MARGIN-LEFT: 2em
}
OL {
MARGIN-TOP: 0px; MARGIN-BOTTOM: 0px
}
UL {
MARGIN-TOP: 0px; MARGIN-BOTTOM: 0px
}
P {
MARGIN-TOP: 0px; MARGIN-BOTTOM: 0px
}
DIV.FoxDiv20120315120836320583 {
FONT-SIZE: 10.5pt; MARGIN: 10px; COLOR: #000000; LINE-HEIGHT: 1.5; FONT-FAMILY:
}
BODY {
FONT-SIZE: 10.5pt; COLOR: #000080; LINE-HEIGHT: 1.5; FONT-FAMILY:
}
</STYLE>
<META content="MSHTML 6.00.2900.6182" name=GENERATOR>
<STYLE>BLOCKQUOTE {
MARGIN-TOP: 0px
}
OL {
MARGIN-TOP: 0px
}
UL {
MARGIN-TOP: 0px
}
</STYLE>
</HEAD>
<BODY style="MARGIN: 10px">
<DIV> </DIV>
<DIV>
<DIV class=FoxDiv20120315120836320583>
<STYLE>BLOCKQUOTE {
MARGIN-TOP: 0px; MARGIN-BOTTOM: 0px; MARGIN-LEFT: 2em
}
OL {
MARGIN-TOP: 0px; MARGIN-BOTTOM: 0px
}
UL {
MARGIN-TOP: 0px; MARGIN-BOTTOM: 0px
}
P {
MARGIN-TOP: 0px; MARGIN-BOTTOM: 0px
}
</STYLE>
<META content="MSHTML 6.00.2900.6182" name=GENERATOR>
<DIV>Dear all,</DIV>
<DIV style="TEXT-INDENT: 2em">We met one problem about Lustre read
performance decay when OSSes are assigned in two different subnet.</DIV>
<DIV style="TEXT-INDENT: 2em">Describing that in the following diagram:</DIV>
<DIV style="TEXT-INDENT: 2em">diagram 1, OSS in different subnets:</DIV>
<DIV style="TEXT-INDENT: 10em">Client (subnet 10.0.1.2)</DIV>
<DIV style="TEXT-INDENT: 10em"> |</DIV>
<DIV style="TEXT-INDENT: 10em"> |</DIV>
<DIV style="TEXT-INDENT: 10em"> |</DIV>
<DIV
style="TEXT-INDENT: 2em"> Switch
</DIV>
<DIV
style="TEXT-INDENT: 8em">|
|
<DIV style="TEXT-INDENT: 14em">
<DIV
style="TEXT-INDENT: 8em">|
|</DIV></DIV></DIV>
<DIV
style="TEXT-INDENT: 8em">|
|</DIV>
<DIV style="TEXT-INDENT: 6em">
OSS1 OSS2</DIV>
<DIV style="TEXT-INDENT: 6em">(10.0.2.2) (10.0.3.2)</DIV>
<DIV style="TEXT-INDENT: 2em">For diagram 1, we made the CLient OSS1 and OSS2 in
3 different subnets. the switch used is able forward all
packages. </DIV>
<DIV style="TEXT-INDENT: 2em">Use dd cmd to test r/w performance write/rad data
to/from to OSS1 and OSS2 at the same time:</DIV>
<DIV>test result:</DIV>
<DIV style="TEXT-INDENT: 2em">
<DIV
style="FONT-STYLE: italic">[root@client client]# time dd if=test2 of=/dev/null bs=1M count=2000</DIV>
<DIV style="FONT-STYLE: italic">2000+0 records in</DIV>
<DIV style="FONT-STYLE: italic">2000+0 records out</DIV>
<DIV
style="FONT-STYLE: italic">2097152000 bytes (2.1 GB) copied, 53.5922 seconds, 39.1 MB/s</DIV>
<DIV style="FONT-STYLE: italic"> </DIV>
<DIV style="FONT-STYLE: italic">real 0m53.796s</DIV>
<DIV style="FONT-STYLE: italic">user 0m0.005s</DIV>
<DIV style="FONT-STYLE: italic">sys 0m2.914s</DIV></DIV>
<DIV style="TEXT-INDENT: 2em"> </DIV>
<DIV style="TEXT-INDENT: 2em"> </DIV>
<DIV>
<DIV style="TEXT-INDENT: 2em">diagram 2, OSS in same subnet:</DIV>
<DIV style="TEXT-INDENT: 10em">Client (subnet 10.0.1.2)</DIV>
<DIV style="TEXT-INDENT: 10em"> |</DIV>
<DIV style="TEXT-INDENT: 10em"> |</DIV>
<DIV style="TEXT-INDENT: 10em"> |</DIV>
<DIV
style="TEXT-INDENT: 2em"> Switch
</DIV>
<DIV
style="TEXT-INDENT: 8em">|
|
<DIV style="TEXT-INDENT: 14em">
<DIV
style="TEXT-INDENT: 8em">|
|</DIV></DIV></DIV>
<DIV
style="TEXT-INDENT: 8em">|
|</DIV>
<DIV style="TEXT-INDENT: 6em">
OSS1 OSS2</DIV>
<DIV style="TEXT-INDENT: 6em">(10.0.2.2, 10.0.2.3, at same subnet)</DIV>
<DIV> </DIV>
<DIV>for diagram 2, we assigned OSS1 and OSS2 at the same subnet, then
test:</DIV>
<DIV>test result:</DIV>
<DIV>
<DIV
style="TEXT-INDENT: 2em">[root@client219 client]# time dd of=/dev/null if=test1 bs=1M </DIV></DIV>
<DIV>
<DIV style="FONT-STYLE: italic">10000+0 records in</DIV>
<DIV style="FONT-STYLE: italic">10000+0 records out</DIV>
<DIV
style="FONT-STYLE: italic">10485760000 bytes (10 GB) copied, 193.07 seconds, 54.3 MB/s</DIV>
<DIV style="FONT-STYLE: italic"> </DIV>
<DIV style="FONT-STYLE: italic"> </DIV></DIV></DIV>
<DIV>conclusion:</DIV>
<DIV style="TEXT-INDENT: 2em">In different subnets, the OSS read performance is
39.1 MB/s, while OSS in</DIV>
<DIV>same subnet, the read performance is 54.3 MB/s. the performance decays
so much.</DIV>
<DIV> </DIV>
<DIV>Question:</DIV>
<DIV style="TEXT-INDENT: 2em">Why using different subnets in lustre, the
performance decayed?</DIV>
<DIV style="TEXT-INDENT: 2em">Anyone had met such problems? Many thanks for your
answers and advice.</DIV>
<DIV> </DIV>
<DIV>
<HR style="WIDTH: 210px; HEIGHT: 1px" align=left color=#b5c4df SIZE=1>
<SPAN></DIV>
<DIV>
<DIV>
<DIV style="FONT-WEIGHT: bold; FONT-STYLE: italic">B.R.</DIV>
<DIV style="FONT-WEIGHT: bold; FONT-STYLE: italic">Feng</DIV>
<DIV> </DIV></DIV></SPAN></DIV></DIV></DIV></BODY></HTML>