[Lustre-discuss] performance issues in simple lustre setup

[murray]

nasnu3:/slut# lst run bw;lst stat c
bw is running now
[LNet Rates of c]
[W] Avg: 800      RPC/s Min: 800      RPC/s Max: 800      RPC/s
[R] Avg: 1596     RPC/s Min: 1596     RPC/s Max: 1596     RPC/s
[LNet Bandwidth of c]
[W] Avg: 0.13     MB/s  Min: 0.13     MB/s  Max: 0.13     MB/s
[R] Avg: 798.09   MB/s  Min: 798.09   MB/s  Max: 798.09   MB/s
[LNet Rates of c]
[W] Avg: 0        RPC/s Min: 0        RPC/s Max: 0        RPC/s
[R] Avg: 0        RPC/s Min: 0        RPC/s Max: 0        RPC/s

Thanks,
murray

Isaac Huang wrote:
> Since both the configuration and the IB link bandwidth looked fine,
> I'd suggest to measure lnet throughput by lnet selftest:
>
> 1. On both client and server: modprobe lnet_selftest
> 2. On the client:
> export LST_SESSION=$$
> lst new_session --timeo 100000 test
> lst add_group s 192.168.3.50 at o2ib
> lst add_group c 192.168.3.30 at o2ib
> lst add_batch bw
> lst add_test --batch bw --loop 500 --concurrency 8 --distribute 1:1 --from c --to s brw write size=1M
> lst run bw
> lst stat c
>
> Isaac
>
> On Tue, Jun 03, 2008 at 07:39:44AM -0400, Murray Smigel wrote:
> > ......
> >    **
> >    On server:  (I have tried both msi_x=1 and 0 with no difference)
> >    modprobe.conf -->
> >    options lnet networks="tcp,o2ib"
> >    options mlx4_core       msi_x=0
> >    alias ib0 ib_ipoib
> >    alias ib1 ib_ipoib
> >    [root at lusty lustre-iokit-1.2]# lctl list_nids
> >    192.168.1.94 at tcp
> >    192.168.3.50 at o2ib
> >    On client:
> >    /etc/modprobe.d/lustre -->
> >    options lnet networks="tcp,o2ib"
> >    options mlx4_core       msi_x=1
> >    alias ib0 ib_ipoib
> >    alias ib1 ib_ipoib
> >    nasnu3:/home/murray/lustre-iokit-1.2# lctl list_nids
> >    192.168.1.156 at tcp
> >    192.168.3.30 at o2ib
> >    Thanks,
> >    murray
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-discuss