[Lustre-discuss] Modifying Lustre network (good practices)

James Robnett jrobnett at aoc.nrao.edu
Thu May 20 08:48:12 PDT 2010


     Can't really help with your larger question but I had a similar
experience with network appropriate write rates and slower reads.

    You might check that you have enabled TCP selective acknowledgments, 
echo 1 > /proc/sys/net/ipv4/tcp_sack
    or
net.ipv4.tcp_sack = 1

     This can help in cases where your OSS's have larger pipes than
your clients and your files are striped across multiple OSS's.
When multiple OSS's are transmitting to a single client they can
over run the switch buffers and drop packets.   This is particularly
noticeable when doing IOzone type benchmarking from a single client
with a wide lfs stripe setting.

     With selective ACKs enabled the client will request a more minimal
set of packets be retransmitted ... or at least that's what I finally
deduced when I ran into it.

James Robnett
NRAO/NM



More information about the lustre-discuss mailing list