[Lustre-discuss] MDT crash: ll_mdt at 100%

[Thomas Roth]

Hi,

I didn't take notice of a discussion of such problems with 1.6.7.1.  Do
you know something more specific about it? We won't want to downgrade
since our users are happier after the last upgrade (1.6.5 -> 1.6.7). And
we don't have the 1.6.7.2 (Debian-) packages yet. But I could try to
speed that up and force an upgrade if you told me that 1.6.7.1 wasn't
really reliable.

For the moment the problem seems to have been fixed by shutdown,
fs-check and writeconf of all servers.
However, I don't want to do that every other week ...

Thanks a lot for your help,
Thomas

Mag Gam wrote:
> Hi Tom:
> 
> There was a known issue with 1.6.7.1. What I did was downgrade to
> 1.6.6 and everything worked well. Or you can try upgrading, but there
> is something def wrong with that version...
> 
> If you like, I can help you offline. I should be free this weekend (I
> have a long weekend)
> 
> 
> 
> On Thu, Jul 2, 2009 at 8:22 AM, Thomas Roth<t.roth at gsi.de> wrote:
>> Hi all,
>>
>> our MDT gets stuck and unresponsive with very high loads (Lustre
>> 1.6.7.1, Kernel 2.6.22, 8 Core, 32GB RAM). The only thing calling
>> attention is one ll_mt_?? process running with 100% cpu. Nothing unusual
>> happening on the cluster before that.
>> After reboot as well as after moving the service to another server, this
>> behavior reappears. The initial stages - mounting MGS, mouting MDT,
>> recovery - work fine, but then the load goes up and the system is
>> rendered unusable.
>>
>> Atm, I don't know what to do, except shutting down all servers and
>> possible do a writeconf everywhere.
>>
>> I see that a similar problem was reported by Mag in March this year, but
>> no clues or solutions appeared.
>> Any ideas?
>>
>> Yours,
>> Thomas
>>

-- 
--------------------------------------------------------------------
Thomas Roth
Department: Informationstechnologie
Location: SB3 1.262
Phone: +49-6159-71 1453  Fax: +49-6159-71 2986

GSI Helmholtzzentrum für Schwerionenforschung GmbH
Planckstraße 1
D-64291 Darmstadt
www.gsi.de

Gesellschaft mit beschränkter Haftung
Sitz der Gesellschaft: Darmstadt
Handelsregister: Amtsgericht Darmstadt, HRB 1528

Geschäftsführer: Professor Dr. Horst Stöcker

Vorsitzende des Aufsichtsrates: Dr. Beatrix Vierkorn-Rudolph,
Stellvertreter: Ministerialdirigent Dr. Rolf Bernhardt