[lustre-discuss] LNET Self-test
Oucharek, Doug S
doug.s.oucharek at intel.com
Mon Feb 6 08:45:31 PST 2017
Try running just a read test and then just a write test rather than having both at the same time and see if the performance goes up.
Doug
> On Feb 6, 2017, at 4:40 AM, Jon Tegner <tegner at foi.se> wrote:
>
> Hi,
>
> I used the following script:
>
> #!/bin/bash
> export LST_SESSION=$$
> lst new_session read/write
> lst add_group servers 10.0.12.12 at o2ib
> lst add_group readers 10.0.12.11 at o2ib
> lst add_group writers 10.0.12.11 at o2ib
> lst add_batch bulk_rw
> lst add_test --batch bulk_rw --concurrency 12 --from readers --to servers \
> brw read check=simple size=1M
> lst add_test --batch bulk_rw --concurrency 12 --from writers --to servers \
> brw write check=simple size=1M
> # start running
> lst run bulk_rw
> # display server stats for 30 seconds
> lst stat servers & sleep 30; kill $!
> # tear down
> lst end_session
>
> and tried with concurrency from 0,2,4,8,12,16, results in
>
> http://renget.se/lnetBandwidth.png
> and
> http://renget.se/lnetRates.png
>
> From Bandwidth a max of just below 2800 MB/s can be noted. Since in this case "readers" and "writers" are the same, I did a few tests with the line
>
> lst add_test --batch bulk_rw --concurrency 12 --from writers --to servers \
> brw write check=simple size=1M
>
> removed from the script - which resulted in a bandwidth of around 3600 MB/s.
>
> I also did tests using mpitests-osu_bw from openmpi, and in that case I monitored a bandwidth of about 3900 MB/s.
>
> Considering the "openmpi-bandwidth" should I be happy with the numbers obtained by LNet selftest? Is there a way to modify the test so that the result gets closer to what openmpi is giving? And what can be said of the "Rates of servers (RPC/s)" - are they "good" or "bad"? What to compare them with?
>
> Thanks!
>
> /jon
>
> On 02/05/2017 08:55 PM, Jeff Johnson wrote:
>> Without seeing your entire command it is hard to say for sure but I would make sure your concurrency option is set to 8 for starters.
>>
>> --Jeff
>>
>> Sent from my iPhone
>>
>>> On Feb 5, 2017, at 11:30, Jon Tegner <tegner at foi.se> wrote:
>>>
>>> Hi,
>>>
>>> I'm trying to use lnet selftest to evaluate network performance on a test setup (only two machines). Using e.g., iperf or Netpipe I've managed to demonstrate the bandwidth of the underlying 10 Gbits/s network (and typically you reach the expected bandwidth as the packet size increases).
>>>
>>> How can I do the same using lnet selftest (i.e., verifying the bandwidth of the underlying hardware)? My initial thought was to increase the I/O size, but it seems the maximum size one can use is "--size=1M".
>>>
>>> Thanks,
>>>
>>> /jon
>>> _______________________________________________
>>> lustre-discuss mailing list
>>> lustre-discuss at lists.lustre.org
>>> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>
> _______________________________________________
> lustre-discuss mailing list
> lustre-discuss at lists.lustre.org
> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
More information about the lustre-discuss
mailing list