[Lustre-discuss] >100% CPU on client

Andrus, Brian Contractor bdandrus at nps.edu
Thu Jan 29 12:56:47 PST 2015


All,

I have a lustre filesystem with an unregistered OST:

[root at submit-0 ~]# lfs df
UUID                   1K-blocks        Used   Available Use% Mounted on
WORK-MDT0000_UUID      393147136    15389780   350856224   4% /work[MDT:0]
WORK-OST0000_UUID    17170063404  8058677632  8236276288  49% /work[OST:0]
WORK-OST0001_UUID    15619171724 10586931900  5022400004  68% /work[OST:1]
WORK-OST0002_UUID    15619171724 10665764164  4163865152  72% /work[OST:2]
WORK-OST0003_UUID    15619171724 10918155860  3911965996  74% /work[OST:3]
WORK-OST0004_UUID     7809493364  6952521872   465368976  94% /work[OST:4]
OST0005             : Resource temporarily unavailable
WORK-OST0006_UUID     7809493364  6998187284   731734676  91% /work[OST:6]
WORK-OST0007_UUID     7809493364  7069544608   660517748  91% /work[OST:7]
WORK-OST0008_UUID     7809493364  6925302272   804469140  90% /work[OST:8]
WORK-OST0009_UUID     7809493364  6969095244   760770748  90% /work[OST:9]
WORK-OST000a_UUID     7809493364  6861984936   867665884  89% /work[OST:10]
WORK-OST000b_UUID     7809493364  6829602016   899989432  88% /work[OST:11]
WORK-OST000c_UUID     7809493364  6887468640   842232976  89% /work[OST:12]
WORK-OST000d_UUID     7809493364  6950260396   779567540  90% /work[OST:13]
WORK-OST000e_UUID    17170063404  3780405388 13192036608  22% /work[OST:14]
WORK-OST000f_UUID    17170063404  1129149016 15838115900   7% /work[OST:15]
WORK-OST0010_UUID    17170063404  1114116516 15853122088   7% /work[OST:16]
WORK-OST0011_UUID    17170063404   876476892 16090299340   5% /work[OST:17]

filesystem summary:  202993272468 109573644636 89120398496  55% /work


Now when someone connects and does an 'ls' that involves a file/directory that has parts on the missing OST, the CPU starts climbing until it is WAY high (eg: 171.51).

I see many process: ldlm_bl_xx where xx is a number. Each of those are eating 100% CPU.

What is happening and how to I prevent this? The OST cannot be registered for other problems we are looking into. If I do attempt register it, I get the same (and worse) behavior.
Since it is not registered, it does not exist in /proc/fs/lustre/* so I cannot make/change any settings there.


Brian Andrus
ITACS/Research Computing
Naval Postgraduate School
Monterey, California
voice: 831-656-6238

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20150129/47946a48/attachment.htm>


More information about the lustre-discuss mailing list