[lustre-discuss] 2.10.0 CentOS6.9 ksoftirqd CPU load

[Hans Henrik Happe]

On 27-09-2017 18:50, Dilger, Andreas wrote:
> On Sep 26, 2017, at 01:10, Hans Henrik Happe <happe at nbi.dk> wrote:
>> Hi,
>>
>> Did anyone else experience CPU load from ksoftirqd after 'modprobe
>> lustre'? On an otherwise idle node I see:
>>
>>  PID USER      PR   NI VIRT  RES  SHR S %CPU  %MEM TIME+   COMMAND
>>    9 root      20   0     0    0    0 S 28.5  0.0  2:05.58 ksoftirqd/1
>>
>>
>>   57 root      20   0     0    0    0 R 23.9  0.0  2:22.91 ksoftirqd/13
>>
>> The sum of those two is about 50% CPU.
>>
>> I have narrowed it down to the ptlrpc module. When I remove that, it stops.
>>
>> I also tested the 2.10.1-RC1, which is the same.
> If you can run "echo l > /proc/sysrq-trigger" it will report the processes
> that are currently running on the CPUs of your system to the console (and
> also /var/log/messages, if it can write everything in time).
>
> You might need to do this several times to get a representative sample of
> the ksoftirqd process stacks to see what they are doing that is consuming
> so much CPU.
>
> Alternately, "echo t > /proc/sysrq-trigger" will report the stacks of all
> processes to the console (and /v/l/m), but there will be a lot of them,
> and no better chance that it catches what ksoftirqd is doing 25% of the time.
I've attached the stacks. Some wakeup which I guess are initiated by
something in the ptlrpc code.

Cheers,
Hans Henrik


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20170930/f3cca144/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: callstack.log
Type: text/x-log
Size: 6039 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20170930/f3cca144/attachment-0001.bin>