<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=UTF-8" http-equiv="Content-Type">
<title></title>
</head>
<body bgcolor="#ffffff" text="#000000">
<br>
There isn't much details on the tests below for someone to make any
recommendations. Are you NFS exporting a Lustre filesystem to a bunch
of NFS clients and then testing on the clients? What sort of workload
are you using to perform these tests? Is it reads, writes or a mix of
them? More details on the hardware/software and the testing methods
would help.<br>
<br>
Sridhar<br>
<br>
Dan wrote:
<blockquote cite="mid:1208286994.16573.15.camel@nad" type="cite">
<meta http-equiv="Content-Type" content="text/html; ">
<meta name="GENERATOR" content="GtkHTML/3.10.3">
Hi,<br>
<br>
With help from Oleg we got the right patches applied and NFS working
well. Maximum performance was about 60 MB/sec. Last week that dropped
to about 12.5 MB/sec and I cannot find a reason. Lustre clients all
obtain 100+ MB/sec on GigE. Each OST is good for 270 MB/sec. When
mounting the client on one of the OSSs I get 230 MB/sec. Seems the
speed is there. How can NFS and Lustre be tuned better?<br>
<br>
Current config for 1.6.4.3 is below:<br>
<br>
1. MGS/OSS w/ 4 OSTs - mgs_max_num_threads=32, ost_max_num_threads=64<br>
2. OSS w/ 6 OSTs - ost_max_num_threads=64<br>
3. 20 Lustre clients - all perform well (GREAT Lustre developers!!!!
this system is amazing!)<br>
4. NFS server runs from a Lustre client machine for 12 to 15 MB/sec
max.<br>
5. NFS server from the MGS (client on MGS/OSS = bad, I know!) can get
20 to 30 MB/sec<br>
- this got 60+ MB/sec in the past.<br>
<br>
bugs and patches applied:<br>
<br>
14360 - 14006 is the only patch<br>
14379 - patch 14007 only since 14008 is reversed by 14591<br>
13371 - bug for the above mentioned 14591 patch<br>
<br>
With these patches the system is stable unless I bump the OST or MGS
threads too high. Performance doesn't seem to change much with any
tuning. I've adjusted the client via /proc and the OSTs and MGS via
/etc/modprobe.conf.<br>
<br>
Suggestions?<br>
<br>
Thank you,<br>
<br>
Dan
<pre wrap="">
<hr size="4" width="90%">
_______________________________________________
Lustre-discuss mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Lustre-discuss@lists.lustre.org">Lustre-discuss@lists.lustre.org</a>
<a class="moz-txt-link-freetext" href="http://lists.lustre.org/mailman/listinfo/lustre-discuss">http://lists.lustre.org/mailman/listinfo/lustre-discuss</a>
</pre>
</blockquote>
<br>
</body>
</html>