[Lustre-discuss] High CPU load, only on 1 OSS

[Ronald K Long]

All of our OSS's are configured the same.  However on the first OSS which 
is experiencing the high CPU load the number of io threads being started 
is much higher than the other OSS servers.

The server that is experiencing high CPU

ost.OSS.ost_io.threads_started=175

Other OSS servers

ost.OSS.ost_io.threads_started=128

All off the OSS servers are configured like this.

ost.OSS.ost_io.threads_min=128
ost.OSS.ost_io.threads_max=512

Any direction/information to resolve this issue is greatly appreciated. If 
any other info is needed please let me know.




Rocky



From:
Wang Yibin <wang.yibin at oracle.com>
To:
Ronald K Long <rklong at usgs.gov>
Cc:
lustre-discuss at lists.lustre.org
Date:
11/16/2010 09:54 AM
Subject:
Re: [Lustre-discuss] High CPU load, only on 1 OSS



Hello,

Normally when stripe_offset is set to -1, MDS will do load/space balancing 
automatically.
What is your use pattern of the filesystem?
It sounds like that your applications are doing extensive I/O on that 
particular OSS.

To find out why the load on the OSS is so high, please 
- find what processes are hogging the CPUs using top(1).
- get the stripe info of your in-use files to see whether most of them 
reside on the same OSS.

If the files in use are not distributed among the OSS servers, or your 
file usage pattern is one-OSS bound, you may want to consider tuning the 
stripe_count/stripe_size.

在 2010-11-16，下午10:38， Ronald K Long 写道：


We recently setup a lustre config.  1 MDS 4 OSS's.  Everything is running 
fine except on the first OSS we are experiencing very high cpu load.  The 
first OSS is running a CPU load in the high 50's.  The other 3 OSS's are 
steady at around 8.  Everything is the same between all of the OSS's. 

The stripe is setup 

stripe_count:   1 stripe_offset:  -1 

Red Hat 5 64bit 

kernel-2.6.18-194.3.1.el5_lustre.1.8.4 
kernel-devel-2.6.18-194.3.1.el5_lustre.1.8.4 
lustre-ldiskfs-3.1.3-2.6.18_194.3.1.el5_lustre.1.8.4 
lustre-1.8.4-2.6.18_194.3.1.el5_lustre.1.8.4 
lustre-modules-1.8.4-2.6.18_194.3.1.el5_lustre.1.8.4 


Any thing I can check on the problem OSS to rectify this issue. 

Thank you in advance

Rocky 
_______________________________________________
Lustre-discuss mailing list
Lustre-discuss at lists.lustre.org
http://lists.lustre.org/mailman/listinfo/lustre-discuss



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20110126/79a80d24/attachment.htm>