[Lustre-discuss] performance issues in simple lustre setup

[Isaac Huang]

Since both the configuration and the IB link bandwidth looked fine,
I'd suggest to measure lnet throughput by lnet selftest:

1. On both client and server: modprobe lnet_selftest
2. On the client:
export LST_SESSION=$$
lst new_session --timeo 100000 test
lst add_group s 192.168.3.50 at o2ib
lst add_group c 192.168.3.30 at o2ib
lst add_batch bw
lst add_test --batch bw --loop 500 --concurrency 8 --distribute 1:1 --from c --to s brw write size=1M
lst run bw
lst stat c

Isaac

On Tue, Jun 03, 2008 at 07:39:44AM -0400, Murray Smigel wrote:
> ......
>    **
>    On server:  (I have tried both msi_x=1 and 0 with no difference)
>    modprobe.conf -->
>    options lnet networks="tcp,o2ib"
>    options mlx4_core       msi_x=0
>    alias ib0 ib_ipoib
>    alias ib1 ib_ipoib
>    [root at lusty lustre-iokit-1.2]# lctl list_nids
>    192.168.1.94 at tcp
>    192.168.3.50 at o2ib
>    On client:
>    /etc/modprobe.d/lustre -->
>    options lnet networks="tcp,o2ib"
>    options mlx4_core       msi_x=1
>    alias ib0 ib_ipoib
>    alias ib1 ib_ipoib
>    nasnu3:/home/murray/lustre-iokit-1.2# lctl list_nids
>    192.168.1.156 at tcp
>    192.168.3.30 at o2ib
>    Thanks,
>    murray