[lustre-discuss] Acceptable thresholds

Dilger, Andreas andreas.dilger at intel.com
Fri Oct 20 00:25:57 PDT 2017


The number of threads needed on the OSS nodes is largely a function of how much network and storage is attached.  If you have a large number of disks, and want to keep them all busy, then more threads are needed to keep the disks well fed compared to a system with a single slow LUN.

On very large OSSes this might be in the hundreds of IO threads.  With smaller OSSes it might only be 32-64 threads.

Run a benchmark like obdfilter-survey to see how many threads are needed to keep all the disks busy, then use the oss.ko module parameters to limit the OSS thread count.

Cheers, Andreas

On Oct 19, 2017, at 13:26, Patrick Farrell <paf at cray.com> wrote:
> 
> Several processes per CPU core, probably?  It’s a lot.
> 
> But there’s a lot of environmental and configuration dependence here too.
> 
> Why not look at how many you have running currently when Lustre is set up and set the limit to double that?  Watching process count isn’t a good way to measure load anyway - it’s probably only good for watching for a fork-bomb type thing, where process count goes runaway.  So why not configure to catch that and otherwise don’t worry about it?
> 
> - Patrick
> 
>> From: lustre-discuss <lustre-discuss-bounces at lists.lustre.org> on behalf of "E.S. Rosenberg" <esr+lustre at mail.hebrew.edu>
>> Date: Thursday, October 19, 2017 at 2:20 PM
>> To: "lustre-discuss at lists.lustre.org" <lustre-discuss at lists.lustre.org>
>> Subject: [lustre-discuss] Acceptable thresholds
>> 
>> Hi,
>> This question is I guess not truly answerable because it is probably very specific for each environment etc. but I am still going to ask it to get a general idea.
>> 
>> We started testing monitoring using Zabbix, its' default 'too many processes' threshold is not very high, so I already raised it to 1024 but the Lustre servers are still well over even that count.
>> 
>> So what is a 'normal' process count for Lustre servers?
>> Should I assume X processes per client? What is X?
>> 
>> Thanks,
>> Eli

Cheers, Andreas
--
Andreas Dilger
Lustre Principal Architect
Intel Corporation









More information about the lustre-discuss mailing list