<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 TRANSITIONAL//EN">
<HTML>
<HEAD>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; CHARSET=UTF-8">
<META NAME="GENERATOR" CONTENT="GtkHTML/3.10.3">
</HEAD>
<BODY>
Hi,<BR>
<BR>
With help from Oleg we got the right patches applied and NFS working well. Maximum performance was about 60 MB/sec. Last week that dropped to about 12.5 MB/sec and I cannot find a reason. Lustre clients all obtain 100+ MB/sec on GigE. Each OST is good for 270 MB/sec. When mounting the client on one of the OSSs I get 230 MB/sec. Seems the speed is there. How can NFS and Lustre be tuned better?<BR>
<BR>
Current config for 1.6.4.3 is below:<BR>
<BR>
1. MGS/OSS w/ 4 OSTs - mgs_max_num_threads=32, ost_max_num_threads=64<BR>
2. OSS w/ 6 OSTs - ost_max_num_threads=64<BR>
3. 20 Lustre clients - all perform well (GREAT Lustre developers!!!! this system is amazing!)<BR>
4. NFS server runs from a Lustre client machine for 12 to 15 MB/sec max.<BR>
5. NFS server from the MGS (client on MGS/OSS = bad, I know!) can get 20 to 30 MB/sec<BR>
- this got 60+ MB/sec in the past.<BR>
<BR>
bugs and patches applied:<BR>
<BR>
14360 - 14006 is the only patch<BR>
14379 - patch 14007 only since 14008 is reversed by 14591<BR>
13371 - bug for the above mentioned 14591 patch<BR>
<BR>
With these patches the system is stable unless I bump the OST or MGS threads too high. Performance doesn't seem to change much with any tuning. I've adjusted the client via /proc and the OSTs and MGS via /etc/modprobe.conf.<BR>
<BR>
Suggestions?<BR>
<BR>
Thank you,<BR>
<BR>
Dan
</BODY>
</HTML>