[Lustre-discuss] Modifying Lustre network (good practices)
James Robnett
jrobnett at aoc.nrao.edu
Thu May 20 08:48:12 PDT 2010
Can't really help with your larger question but I had a similar
experience with network appropriate write rates and slower reads.
You might check that you have enabled TCP selective acknowledgments,
echo 1 > /proc/sys/net/ipv4/tcp_sack
or
net.ipv4.tcp_sack = 1
This can help in cases where your OSS's have larger pipes than
your clients and your files are striped across multiple OSS's.
When multiple OSS's are transmitting to a single client they can
over run the switch buffers and drop packets. This is particularly
noticeable when doing IOzone type benchmarking from a single client
with a wide lfs stripe setting.
With selective ACKs enabled the client will request a more minimal
set of packets be retransmitted ... or at least that's what I finally
deduced when I ran into it.
James Robnett
NRAO/NM
More information about the lustre-discuss
mailing list