[Lustre-discuss] Lots of "No ctxt" after OST crush

[Thomas Roth]

Hi all,

one of our OSTs crushed - actually we ran into Bug 17052
(https://bugzilla.lustre.org/show_bug.cgi?id=17052). The OST fscks
without errors, mouting and aborting recovery also works.

However, the MDT doesn't accept it anymore (I've attached the entire log
line of the event below):
LustreError:... (llog_lvfs.c: ...:llog_lvfs_create()) error looking up
logfile ...: rc -2

It would seem the Logfile was lost somewhere in the process (rc -2).
The MDT then deactivates this OST.

The clients can see the files on this OST, files can be read and deleted
- as expected.

So my idea was to leave the OST as it is now and try and move the file
off it to the other OSTs, eventually reformatting it sometime.

However, the MDT log now has a lot of
Apr 26 14:45:34 lxmds3 kernel: LustreError:
3804:0:(llog_obd.c:226:llog_add()) No ctxt
Apr 26 14:45:34 lxmds3 kernel: LustreError:
3804:0:(llog_obd.c:226:llog_add()) Skipped 4058 previous similar messages

These "No ctxt" appear immedeately after the MDT refuses the said OST,
so I assume a connection.
My question: Do these messages mean any further trouble? Could something
be building up and finally blwo up the MDT/the file system? Should I try
to do something with this Log-less OST instead, and would would that be?

This is Lustre 1.6.7.2 running under Debian Etch 64bit, Kernel 2.6.22.

Regards,
Thomas

--------------------------------------------------------------------------------------------------------------------------------

MDT-Log of OST-Remount-Attempt:

Apr 26 13:58:00 lxmds3 kernel: Lustre: gsilust-OST00b3-osc: Connection
restored to service gsilust-OST00b3 using nid 10.12.119.138 at tcp.
Apr 26 13:58:00 lxmds3 kernel: LustreError:
13532:0:(llog_lvfs.c:612:llog_lvfs_create()) error looking up logfile
0x2dd861c:0xe95a1032: rc -2
Apr 26 13:58:00 lxmds3 kernel: LustreError:
13532:0:(llog_cat.c:172:llog_cat_id2handle()) error opening log id
0x2dd861c:e95a1032: rc -2
Apr 26 13:58:00 lxmds3 kernel: LustreError:
13532:0:(llog_obd.c:279:cat_cancel_cb()) Cannot find handle for log
0x2dd861c
Apr 26 13:58:00 lxmds3 kernel: LustreError:
13531:0:(llog_obd.c:350:llog_obd_origin_setup()) llog_process with
cat_cancel_cb failed: -2
Apr 26 13:58:00 lxmds3 kernel: LustreError:
13531:0:(llog_obd.c:194:llog_setup()) obd gsilust-OST00b3-osc ctxt 2
lop_setup=ffffffff88354370 failed -2
Apr 26 13:58:00 lxmds3 kernel: LustreError:
13531:0:(osc_request.c:3724:osc_llog_init()) failed LLOG_MDS_OST_ORIG_CTXT
Apr 26 13:58:00 lxmds3 kernel: LustreError:
13531:0:(osc_request.c:3740:osc_llog_init()) osc 'gsilust-OST00b3-osc'
tgt 'gsilust-MDT0000' cnt 1 catid ffff8103aa285ce0 rc=-2
Apr 26 13:58:00 lxmds3 kernel: LustreError:
13531:0:(osc_request.c:3742:osc_llog_init()) logid 0x50fb811:0xf129dc6
Apr 26 13:58:00 lxmds3 kernel: LustreError:
13531:0:(lov_log.c:243:lov_llog_init()) error osc_llog_init idx 179 osc
'gsilust-OST00b3-osc' tgt 'gsilust-MDT0000' (rc=-2)
Apr 26 13:58:00 lxmds3 kernel: LustreError:
13531:0:(mds_log.c:219:mds_llog_init()) lov_llog_init err -2
Apr 26 13:58:00 lxmds3 kernel: LustreError:
13531:0:(llog_obd.c:439:llog_cat_initialize()) rc: -2
Apr 26 13:58:00 lxmds3 kernel: LustreError:
13531:0:(mds_lov.c:918:__mds_lov_synchronize()) gsilust-OST00b3_UUID
failed at update_mds: -2
Apr 26 13:58:00 lxmds3 kernel: LustreError:
13531:0:(mds_lov.c:960:__mds_lov_synchronize()) gsilust-OST00b3_UUID
sync failed -2, deactivating


-- 
--------------------------------------------------------------------
Thomas Roth
Department: Informationstechnologie
Location: SB3 1.262
Phone: +49-6159-71 1453  Fax: +49-6159-71 2986

GSI Helmholtzzentrum für Schwerionenforschung GmbH
Planckstraße 1
D-64291 Darmstadt
www.gsi.de

Gesellschaft mit beschränkter Haftung
Sitz der Gesellschaft: Darmstadt
Handelsregister: Amtsgericht Darmstadt, HRB 1528

Geschäftsführer: Professor Dr. Horst Stöcker (wissenschaftlich)
Geschäftsführer: Christiane Neumann (kaufmännisch)

Vorsitzende des Aufsichtsrates: Dr. Beatrix Vierkorn-Rudolph,
Stellvertreter: Ministerialdirigent Dr. Rolf Bernhardt