[Lustre-discuss] >100% CPU on client
Andrus, Brian Contractor
bdandrus at nps.edu
Thu Jan 29 12:56:47 PST 2015
All,
I have a lustre filesystem with an unregistered OST:
[root at submit-0 ~]# lfs df
UUID 1K-blocks Used Available Use% Mounted on
WORK-MDT0000_UUID 393147136 15389780 350856224 4% /work[MDT:0]
WORK-OST0000_UUID 17170063404 8058677632 8236276288 49% /work[OST:0]
WORK-OST0001_UUID 15619171724 10586931900 5022400004 68% /work[OST:1]
WORK-OST0002_UUID 15619171724 10665764164 4163865152 72% /work[OST:2]
WORK-OST0003_UUID 15619171724 10918155860 3911965996 74% /work[OST:3]
WORK-OST0004_UUID 7809493364 6952521872 465368976 94% /work[OST:4]
OST0005 : Resource temporarily unavailable
WORK-OST0006_UUID 7809493364 6998187284 731734676 91% /work[OST:6]
WORK-OST0007_UUID 7809493364 7069544608 660517748 91% /work[OST:7]
WORK-OST0008_UUID 7809493364 6925302272 804469140 90% /work[OST:8]
WORK-OST0009_UUID 7809493364 6969095244 760770748 90% /work[OST:9]
WORK-OST000a_UUID 7809493364 6861984936 867665884 89% /work[OST:10]
WORK-OST000b_UUID 7809493364 6829602016 899989432 88% /work[OST:11]
WORK-OST000c_UUID 7809493364 6887468640 842232976 89% /work[OST:12]
WORK-OST000d_UUID 7809493364 6950260396 779567540 90% /work[OST:13]
WORK-OST000e_UUID 17170063404 3780405388 13192036608 22% /work[OST:14]
WORK-OST000f_UUID 17170063404 1129149016 15838115900 7% /work[OST:15]
WORK-OST0010_UUID 17170063404 1114116516 15853122088 7% /work[OST:16]
WORK-OST0011_UUID 17170063404 876476892 16090299340 5% /work[OST:17]
filesystem summary: 202993272468 109573644636 89120398496 55% /work
Now when someone connects and does an 'ls' that involves a file/directory that has parts on the missing OST, the CPU starts climbing until it is WAY high (eg: 171.51).
I see many process: ldlm_bl_xx where xx is a number. Each of those are eating 100% CPU.
What is happening and how to I prevent this? The OST cannot be registered for other problems we are looking into. If I do attempt register it, I get the same (and worse) behavior.
Since it is not registered, it does not exist in /proc/fs/lustre/* so I cannot make/change any settings there.
Brian Andrus
ITACS/Research Computing
Naval Postgraduate School
Monterey, California
voice: 831-656-6238
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20150129/47946a48/attachment.htm>
More information about the lustre-discuss
mailing list