[Lustre-discuss] lustre performance degrade on SuSE 11
Tharindu Rukshan Bamunuarachchi
tharindub at millenniumit.com
Wed Aug 12 05:20:54 PDT 2009
there is only one NIC in each machine. Does lustre need more than one nic in each machine ?
suse11-202:~ # ifconfig
eth0 Link encap:Ethernet HWaddr 00:14:4F:EB:47:8C
inet addr:172.25.88.202 Bcast:172.25.88.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:2672766 errors:0 dropped:0 overruns:0 frame:0
TX packets:2479712 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:3217001573 (3067.9 Mb) TX bytes:1817419819 (1733.2 Mb)
lustre-4200:~ # ifconfig
eth0 Link encap:Ethernet HWaddr 00:14:4F:3F:B8:38
inet addr:172.25.88.220 Bcast:172.25.91.255 Mask:255.255.252.0
inet6 addr: fe80::214:4fff:fe3f:b838/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:73757596 errors:0 dropped:0 overruns:0 frame:0
TX packets:20095469 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:110072295131 (104973.1 Mb) TX bytes:1525481556 (1454.8 Mb)
Base address:0x8c00 Memory:fbde0000-fbe00000
lustre-2100:~ # ifconfig
eth0 Link encap:Ethernet HWaddr 00:E0:81:5D:F7:BE
inet addr:172.25.88.120 Bcast:172.25.88.255 Mask:255.255.255.0
inet6 addr: fe80::2e0:81ff:fe5d:f7be/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:20975604 errors:0 dropped:0 overruns:0 frame:0
TX packets:15997617 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:2317735053 (2210.3 Mb) TX bytes:105271680726 (100394.8 Mb)
Interrupt:58 Base address:0xe000
cheers,
__
tharindu
-----Original Message-----
From: Mag Gam [mailto:magawake at gmail.com]
Sent: Wednesday, August 12, 2009 5:21 PM
To: Tharindu Rukshan Bamunuarachchi
Cc: lustre-discuss at lists.lustre.org
Subject: Re: [Lustre-discuss] lustre performance degrade on SuSE 11
How many NICs do you have on your client, MDS, OSS?
I am suspecting its a routing issue. Do you have more than 1 NIC on
all of these boxes?
options lnet 'networks="tcp0(eth0,eth1),elan0"'
I advise you to take a look at 5.1.1 in the Lustre manual,
manual.lustre.org/images/4/48/LustreManual_1.6_man_v19.pdf
On Wed, Aug 12, 2009 at 7:31 AM, Tharindu Rukshan
Bamunuarachchi<tharindub at millenniumit.com> wrote:
>
> I can SCP from/to client machine at 50 MB/s rate.
> Interface is up at 1000Full.
> No packet loss.
>
>
> Just like to emphasize ... Client is SuSE 11 (1.8.1) and OSS/MDS are SuSE 10 (1.8.0)
>
>
>
> suse11-202:~ # scp SLES-11-DVD-x86_64-GM-DVD1.iso 172.25.88.220:
> Password:
> SLES-11-DVD-x86_64-GM-DVD1.iso 100% 539MB 49.0MB/s 00:11
>
>
> Kernel Interface table
> Iface MTU Met RX-OK RX-ERR RX-DRP RX-OVR TX-OK TX-ERR TX-DRP TX-OVR Flg
> eth0 1500 0 1741587 0 0 0 2002121 0 0 0 BMRU
>
>
> cheers,
> __
> tharindu
>
> -----Original Message-----
> From: Mag Gam [mailto:magawake at gmail.com]
> Sent: Wednesday, August 12, 2009 4:40 PM
> To: Tharindu Rukshan Bamunuarachchi
> Cc: lustre-discuss at lists.lustre.org
> Subject: Re: [Lustre-discuss] lustre performance degrade on SuSE 11
>
> Is your network setup properly? Can you scp/ftp file to your OSS as a
> test? Check if your network interfaces are properly connected
> (autoneg, 1000Full). make sure you aren't getting any packet loss
> (netstat -i). Also First then that and see how it goes.
>
>
> On Wed, Aug 12, 2009 at 3:37 AM, Tharindu Rukshan
> Bamunuarachchi<tharindub at millenniumit.com> wrote:
>> hi All,
>>
>>
>>
>> I have following lustre installation.
>>
>>
>>
>> Node A : OSS (SuSE 10) – 1.8.0
>>
>> Node B : MDS (SuSE 10) – 1.8.0
>>
>>
>>
>> Node C: Client (SuSE 11) – 1.8.1
>>
>>
>>
>> When try to run “dd” command on lustre mount, it is only writing at 2.1 MB/s
>>
>> But If mount same file system on Node B, I can achive 45 MB/s performance.
>>
>>
>>
>> All nodes are connected to same 3560G cisco switch.
>>
>>
>>
>> I saw following error in MDS server message file. Is this related to
>> performance of lustre file system ?
>>
>>
>>
>> LustreError: 3039:0:(socklnd_cb.c:2228:ksocknal_recv_hello()) Unknown
>> protocol version (2.x expected) from 172.25.88.202
>>
>>
>>
>>
>>
>> cheers,
>>
>> __
>>
>> tharindu
>>
>>
>>
>> *******************************************************************************************************************************************************************
>>
>> "The information contained in this email including in any attachment is
>> confidential and is meant to be read only by the person to whom it is
>> addressed. If you are not the intended recipient(s), you are prohibited from
>> printing, forwarding, saving or copying this email. If you have received
>> this e-mail in error, please immediately notify the sender and delete this
>> e-mail and its attachments from your computer."
>>
>> *******************************************************************************************************************************************************************
>>
>> _______________________________________________
>> Lustre-discuss mailing list
>> Lustre-discuss at lists.lustre.org
>> http://lists.lustre.org/mailman/listinfo/lustre-discuss
>>
>>
>
>
> *******************************************************************************************************************************************************************
>
> "The information contained in this email including in any attachment is confidential and is meant to be read only by the person to whom it is addressed. If you are not the intended recipient(s), you are prohibited from printing, forwarding, saving or copying this email. If you have received this e-mail in error, please immediately notify the sender and delete this e-mail and its attachments from your computer."
>
> *******************************************************************************************************************************************************************
>
*******************************************************************************************************************************************************************
"The information contained in this email including in any attachment is confidential and is meant to be read only by the person to whom it is addressed. If you are not the intended recipient(s), you are prohibited from printing, forwarding, saving or copying this email. If you have received this e-mail in error, please immediately notify the sender and delete this e-mail and its attachments from your computer."
*******************************************************************************************************************************************************************
More information about the lustre-discuss
mailing list