[Lustre-discuss] lustre-1.6.5.1 kernel panic
Andreas Dilger
adilger at sun.com
Fri Sep 26 13:25:29 PDT 2008
On Sep 26, 2008 15:14 +0100, Wojciech Turek wrote:
> We had another kernel panic this time on MDS server. Since we use lustre
> patched kernel downloaded from the SUN website we would like to ask if
> anyone else have seen such a problem while moving from 1.6.4.3 to
> 1.6.5.1 on RHEL4 x86_64
>
>
> slab: cache size-1620 error: slabs_full accounting error
> slab: cache size-1620 error: slabs_full accounting error
> slab: cache size-1620 error: slabs_full accounting error
I've never seen these errors before - I didn't even know a "size-1620"
slab existed.
> Unable to handle kernel paging request at 0000303a383a303a RIP:
> <ffffffff801623c4>{s_show+62}
> PML4 0
> Oops: 0000 [1] SMP
> CPU 3
> Modules linked in: mds(U) fsfilt_ldiskfs(U) mgs(U) mgc(U) ldiskfs(U)
> lustre(U) lov(U) mdc(U) lquota(U) osc(U) ko2iblnd(U) ptlrpc(U)
> obdclass(U) lnet(U) lvfs(U) libcfs(U) sg(U) dell_rbu(U) autofs4(U)
> i2c_nforce2(U) i2c_amd756(U) i2c_isa(U) i2c_amd8111(U) i2c_i801(U)
> i2c_core(U) qlgc_vnic(U) iw_cxgb3(U) cxgb3(U) mlx4_ib(U) mlx4_core(U)
> ib_mthca(U) ipmi_devintf(U) ipmi_si(U) ipmi_msghandler(U) rdma_ucm(U)
> ib_sdp(U) rdma_cm(U) iw_cm(U) ib_addr(U) ib_ipoib(U) md5(U) ipv6(U)
> cpufreq_powersave(U) mptctl(U) dm_mirror(U) dm_round_robin(U)
> dm_multipath(U) dm_mod(U) sr_mod(U) usb_storage(U) joydev(U) button(U)
> battery(U) ac(U) uhci_hcd(U) ehci_hcd(U) hw_random(U) ib_ipath(U)
> ib_umad(U) ib_ucm(U) ib_uverbs(U) ib_cm(U) ib_sa(U) ib_mad(U) ib_core(U)
> ata_piix(U) libata(U) ext3(U) jbd(U) tg3(U) s2io(U) qla2400(U)
> qla2xxx(U) scsi_transport_fc(U) nfs(U) nfs_acl(U) lockd(U) sunrpc(U)
> mptsas(U) mptscsi(U) mptbase(U) megaraid_sas(U) e1000(U) bnx2(U)
> sd_mod(U) scsi_mod(U)
> Pid: 15733, comm: collectl Not tainted 2.6.9-67.0.7.EL_lustre.1.6.5.1smp
> RIP: 0010:[<ffffffff801623c4>] <ffffffff801623c4>{s_show+62}
> RSP: 0018:0000010117989e68 EFLAGS: 00010006
> RAX: ffffffff80329f7a RBX: 00000100cffa5580 RCX: 00000100cffa5501
> RDX: 0000000000000004 RSI: 0000303a383a303a RDI: 00000100cffa56e8
> RBP: ffffffff80329f7a R08: 00000000fffffffd R09: 0000000000000000
> R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
> R13: 0000000000001000 R14: 000001004c636500 R15: 0000000000000024
> FS: 0000002a9630ee80(0000) GS:ffffffff8048e880(0000) knlGS:0000000000000000
> CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> CR2: 0000303a383a303a CR3: 00000000cfb24000 CR4: 00000000000006e0
> Process collectl (pid: 15733, threadinfo 0000010117988000, task
> 0000010127176030)
> Stack: 0000000000000000 0000000000000000 0000000000000009 00000100cffa5580
> 000001004c636500 0000000000000000 0000000000001000 0000000000000f0d
> 0000000000000000 ffffffff80196c1a
> Call Trace:<ffffffff80196c1a>{seq_read+445} <ffffffff80178c28>{vfs_read+207}
> <ffffffff80178e84>{sys_read+69} <ffffffff8011022a>{system_call+126}
>
>
> Code: 48 8b 06 0f 18 08 48 8d 83 18 01 00 00 48 39 c6 74 2e 8b 93
> RIP <ffffffff801623c4>{s_show+62} RSP <0000010117989e68>
> CR2: 0000303a383a303a
> <0>Kernel panic - not syncing: Oops
>
> Thanks,
>
> Wojciech
>
> Wojciech Turek wrote:
> > Hi,
> >
> > I upgraded our test lustre file system to the latest 1.6.5.1 version
> > available from the SUN website.
> > I have one OSS with one OST and one MDS with combined MGS and MDT
> > Both servers are running RHEL4 x86_64 and
> > 2.6.9-67.0.7.EL_lustre.1.6.5.1smp kernel, the interconnect is infiniband
> > and I am using ib modules provided with lustre.
> > When I mount filesystem and then start writing to it OSS crashes with
> > kernel panic, see log below:
> >
> >
> > Lustre: 0:0:(watchdog.c:130:lcw_cb()) Watchdog triggered for pid 17398:
> > it was inactive for 200s
> > Lustre: 0:0:(linux-debug.c:167:libcfs_debug_dumpstack()) showing stack
> > for process 17397
> > ll_ost_io_92 D 0000000000000002 0 17397 1 17398 17396
> > (L-TLB)
> > 00000101156bf538 0000000000000046 00000101956bf616 ffffffff801ece0f
> > 0000000000000000 ffffff0010776340 000001010e14c6c0 0000000100000001
> > 0000010113f90030 000000000012b585
> > Call Trace:ll_ost_io_82 D 000001012ab79400 0 17387 1
> > 17388 17386 (L-TLB)
> > 000001011c252d88 0000000000000046 ffffffffa000288c 0000010115b213c0
> > 0000000000000246 00000100cf851c00 000001012bafa940 0000000200000000
> > 000001010f71f030 0000000000000814
> > Call Trace:<ffffffffa000288c>{:scsi_mod:scsi_done+0}
> > <ffffffff801ece0f>{vsnprintf+1406} <ffffffff8024f658>{elv_next_request+238}
> > <ffffffffa0007df8>{:scsi_mod:scsi_request_fn+1100}
> > <ffffffff8030cc1f>{__down+147}
> > <ffffffff80133804>{default_wake_function+0}
> > <ffffffffa067b484>{:ko2iblnd:kiblnd_init_tx_msg+308}
> > <ffffffff8030e2f6>{io_schedule+38}
> > <ffffffff80179e24>{__wait_on_buffer+125}
> > <ffffffff80179caa>{bh_wake_function+0}
> > <ffffffff80179caa>{bh_wake_function+0}
> > <ffffffffa07cad2b>{:ldiskfs:ldiskfs_mb_init_cache+635}
> > <ffffffff8030e73d>{__down_failed+53}
> > <ffffffffa06c6670>{:lquota:filter_quota_check+0}
> > <ffffffffa0843acf>{:obdfilter:.text.lock.filter_io_26+35}
> > <ffffffffa083b01e>{:obdfilter:filter_commitrw+126}
> > <ffffffff80158b8e>{add_to_page_cache+167}
> > <ffffffffa07cb450>{:ldiskfs:ldiskfs_mb_load_buddy+304}
> > <ffffffffa07cc7b4>{:ldiskfs:ldiskfs_mb_regular_allocator+1028}
> > <4>Lustre: 0:0:(watchdog.c:130:lcw_cb()) Watchdog triggered for
> > pid 17388: it was inactive for 200s
> > Lustre: 0:0:(watchdog.c:130:lcw_cb()) Skipped 2 previous similar messages
> > Lustre: 0:0:(linux-debug.c:167:libcfs_debug_dumpstack()) showing stack
> > for process 17388
> > Lustre: 0:0:(linux-debug.c:167:libcfs_debug_dumpstack()) Skipped 1
> > previous similar message
> > ll_ost_io_83 D 0000000000000002 0 17388 1 17389 17387
> > (L-TLB)
> > 000001011c35f538 0000000000000046 000001019c35f616 ffffffff801ece0f
> > 0000000000000000 ffffff0010776508 000001010e14c6c0 0000000000000001
> > 000001011c336800 00000000000de7ad
> > Call Trace:<ffffffff801351dc>{autoremove_wake_function+0}
> > <ffffffffa07d10fd>{:ldiskfs:ldiskfs_mb_new_blocks+333}
> > <ffffffff801351dc>{autoremove_wake_function+0}
> > <ffffffffa0814eb4>{:fsfilt_ldiskfs:ldiskfs_ext_new_extent_cb+884}
> > <ffffffffa028e06a>{:ib_ipath:ipath_verbs_send+1883}
> > <ffffffffa07c6d7f>{:ldiskfs:ldiskfs_ext_find_extent+255}
> > <ffffffffa07c8972>{:ldiskfs:ldiskfs_ext_walk_space+482}
> > <ffffffffa0814b40>{:fsfilt_ldiskfs:ldiskfs_ext_new_extent_cb+0}
> > <ffffffffa0815343>{:fsfilt_ldiskfs:fsfilt_map_nblocks+307}
> > <ffffffffa028560f>{:ib_ipath:ipath_do_send+1852}
> > <ffffffff8013f734>{__mod_timer+293}
> >
> > <ffffffffa08155bb>{:fsfilt_ldiskfs:fsfilt_ldiskfs_map_ext_inode_pages+539}
> > <ffffffff801ece0f>{vsnprintf+1406}
> > <ffffffff801ece0f>{vsnprintf+1406}
> > <ffffffffa067cb38>{:ko2iblnd:kiblnd_check_sends+2040}
> > <ffffffffa08410b4>{:obdfilter:filter_direct_io+1108}
> > <ffffffffa01ba524>{:jbd:journal_start+223}
> > <ffffffffa081384b>{:fsfilt_ldiskfs:fsfilt_ldiskfs_brw_start+763}
> > <ffffffffa0842c1d>{:obdfilter:filter_commitrw_write+4957}
> > <ffffffffa04f5539>{:lvfs:pop_ctxt+505}
> > <ffffffff8030e4ce>{schedule_timeout+411}
> > <ffffffffa07f8688>{:ost:ost_checksum_bulk+200}
> > <ffffffffa083b01e>{:obdfilter:filter_commitrw+126}
> > <ffffffffa07fddd1>{:ost:ost_brw_write+9505}
> > <ffffffffa07f87ee>{:ost:ost_checksum_bulk+558}
> > <ffffffffa07f8688>{:ost:ost_checksum_bulk+200}
> > <ffffffffa07fddd1>{:ost:ost_brw_write+9505}
> > <ffffffff80133804>{default_wake_function+0}
> > <ffffffff80133804>{default_wake_function+0}
> > <ffffffffa06015ef>{:ptlrpc:lustre_msg_get_version+95}
> > <ffffffffa06015ef>{:ptlrpc:lustre_msg_get_version+95}
> > <ffffffffa07f8060>{:ost:ost_bulk_timeout+0}
> > <ffffffffa07f8060>{:ost:ost_bulk_timeout+0}
> > <ffffffffa06016e5>{:ptlrpc:lustre_msg_check_version+69}
> > <ffffffffa06016e5>{:ptlrpc:lustre_msg_check_version+69}
> > <ffffffffa07f8060>{:ost:ost_bulk_timeout+0}
> > <ffffffffa07f8060>{:ost:ost_bulk_timeout+0}
> > <ffffffffa080364d>{:ost:ost_handle+11661}
> > <ffffffffa080364d>{:ost:ost_handle+11661}
> > <ffffffff8015c830>{__rmqueue+218}
> > <ffffffff8015c830>{__rmqueue+218}
> > <ffffffffa060a451>{:ptlrpc:ptlrpc_check_req+17}
> > <ffffffffa060a451>{:ptlrpc:ptlrpc_check_req+17}
> > <ffffffffa060c629>{:ptlrpc:ptlrpc_server_handle_request+2457}
> > <ffffffffa060c629>{:ptlrpc:ptlrpc_server_handle_request+2457}
> > <ffffffffa04df45e>{:libcfs:lcw_update_time+30}
> > <ffffffffa04df45e>{:libcfs:lcw_update_time+30}
> > <ffffffff80133855>{__wake_up_common+67}
> > <ffffffffa060ed05>{:ptlrpc:ptlrpc_main+3989}
> > <ffffffffa060d270>{:ptlrpc:ptlrpc_retry_rqbds+0}
> > <ffffffffa060d270>{:ptlrpc:ptlrpc_retry_rqbds+0}
> > <ffffffff8013f734>{__mod_timer+293}
> > <ffffffffa060d270>{:ptlrpc:ptlrpc_retry_rqbds+0}
> > <ffffffffa060ed05>{:ptlrpc:ptlrpc_main+3989}
> > <ffffffff80110de3>{child_rip+8} <ffffffffa060dd70>{:ptlrpc:ptlrpc_main+0}
> > <ffffffff80110ddb>{child_rip+0}
> > <ffffffff8030cc1f>{__down+147}
> > <ffffffff80133804>{default_wake_function+0}
> > <ffffffffa060d270>{:ptlrpc:ptlrpc_retry_rqbds+0}
> > <ffffffffa060d270>{:ptlrpc:ptlrpc_retry_rqbds+0}
> > <ffffffff80110de3>{child_rip+8}
> > <ffffffffa060dd70>{:ptlrpc:ptlrpc_main+0}
> > <ffffffff80110ddb>{child_rip+0}
> > ll_ost_io_84 D 0000000000000002 0 17389 1 17390 17388
> > (L-TLB)
> > 000001011ac6d538 0000000000000046 000001019ac6d616 ffffffff801ece0f
> > 0000000000000000 ffffff0010776340 000001010e14c6c0 0000000100000001
> > 000001011c336030 000000000012c37b
> > Call Trace:<ffffffff80133804>{default_wake_function+0}
> > <ffffffffa067b484>{:ko2iblnd:kiblnd_init_tx_msg+308}
> > <ffffffff801ece0f>{vsnprintf+1406}
> > <ffffffff8030e73d>{__down_failed+53}
> > <ffffffffa06c6670>{:lquota:filter_quota_check+0}
> > <ffffffffa0843acf>{:obdfilter:.text.lock.filter_io_26+35}
> > <ffffffff80131bc7>{recalc_task_prio+337}
> > <ffffffffa05160a0>{:lnet:lnet_send+2544}
> > <ffffffffa083b01e>{:obdfilter:filter_commitrw+126}
> > <ffffffff8030cc1f>{__down+147}
> > <ffffffff80133804>{default_wake_function+0}
> > <ffffffffa067b484>{:ko2iblnd:kiblnd_init_tx_msg+308}
> > <ffffffff8030e4ce>{schedule_timeout+411}
> > <ffffffffa07f8688>{:ost:ost_checksum_bulk+200}
> > <ffffffffa07fddd1>{:ost:ost_brw_write+9505}
> > <ffffffff80133804>{default_wake_function+0}
> > <ffffffffa06015ef>{:ptlrpc:lustre_msg_get_version+95}
> > <ffffffffa07f8060>{:ost:ost_bulk_timeout+0}
> > <ffffffffa06016e5>{:ptlrpc:lustre_msg_check_version+69}
> > <ffffffffa07f8060>{:ost:ost_bulk_timeout+0}
> > <ffffffffa080364d>{:ost:ost_handle+11661}
> > <ffffffffa0516b48>{:lnet:lnet_match_blocked_msg+920}
> > <ffffffff8015c830>{__rmqueue+218}
> > <ffffffffa060a451>{:ptlrpc:ptlrpc_check_req+17}
> > <ffffffffa060c629>{:ptlrpc:ptlrpc_server_handle_request+2457}
> > <ffffffffa04df45e>{:libcfs:lcw_update_time+30}
> > <ffffffff8030e73d>{__down_failed+53}
> > <ffffffffa06c6670>{:lquota:filter_quota_check+0}
> > <ffffffffa0843acf>{:obdfilter:.text.lock.filter_io_26+35}
> > <ffffffff80133855>{__wake_up_common+67}
> > <ffffffffa060ed05>{:ptlrpc:ptlrpc_main+3989}
> > <ffffffffa060d270>{:ptlrpc:ptlrpc_retry_rqbds+0}
> > <ffffffffa060d270>{:ptlrpc:ptlrpc_retry_rqbds+0}
> > <ffffffffa060d270>{:ptlrpc:ptlrpc_retry_rqbds+0}
> > <ffffffff80110de3>{child_rip+8}
> > <ffffffffa060dd70>{:ptlrpc:ptlrpc_main+0}
> > <ffffffff80131bc7>{recalc_task_prio+337}
> > <ffffffff80110ddb>{child_rip+0}
> > ll_ost_io_91 D<ffffffffa05160a0>{:lnet:lnet_send+2544}
> > 0000000000000002 0 17396 1 17397 17395 (L-TLB)
> > 0000010113fb3538 0000000000000046 0000010193fb3616 ffffffff801ece0f
> > 0000000000000000 ffffff0010776638 000001010e14c6c0 0000000000000001
> > 0000010113f90800 00000000000d4d86
> > Call Trace:<ffffffffa083b01e>{:obdfilter:filter_commitrw+126}
> > <ffffffff801ece0f>{vsnprintf+1406}
> > <ffffffff8030e4ce>{schedule_timeout+411}
> > <ffffffffa07f8688>{:ost:ost_checksum_bulk+200}
> > <ffffffffa07fddd1>{:ost:ost_brw_write+9505}
> > <ffffffff80133804>{default_wake_function+0}
> > <ffffffffa06015ef>{:ptlrpc:lustre_msg_get_version+95}
> > <ffffffffa07f8060>{:ost:ost_bulk_timeout+0}
> > <ffffffffa06016e5>{:ptlrpc:lustre_msg_check_version+69}
> > <ffffffffa07f8060>{:ost:ost_bulk_timeout+0}
> > <ffffffffa080364d>{:ost:ost_handle+11661}
> > <ffffffffa0516ad7>{:lnet:lnet_match_blocked_msg+807}
> > <ffffffffa0516b48>{:lnet:lnet_match_blocked_msg+920}
> > <ffffffffa060a451>{:ptlrpc:ptlrpc_check_req+17}
> > <ffffffffa060c629>{:ptlrpc:ptlrpc_server_handle_request+2457}
> > <ffffffffa04df45e>{:libcfs:lcw_update_time+30}
> > <ffffffff80133855>{__wake_up_common+67}
> > <ffffffffa060ed05>{:ptlrpc:ptlrpc_main+3989}
> > <ffffffffa060d270>{:ptlrpc:ptlrpc_retry_rqbds+0}
> > <ffffffffa060d270>{:ptlrpc:ptlrpc_retry_rqbds+0}
> > <ffffffffa060d270>{:ptlrpc:ptlrpc_retry_rqbds+0}
> > <ffffffff80110de3>{child_rip+8}
> > <ffffffffa060dd70>{:ptlrpc:ptlrpc_main+0}
> > <ffffffff80110ddb>{child_rip+0}
> > ll_ost_io_107 D 0000000000000002 0 17412 1 17413 17411
> > (L-TLB)
> > 000001010ea6f538 0000000000000046 000001018ea6f616 ffffffff801ece0f
> > 0000000000000000 ffffff0010776768 000001010e14c6c0 0000000100000001
> > 000001010ea48800 000000000012d2f5
> > Call Trace:<ffffffff8030cc1f>{__down+147}
> > <ffffffff80133804>{default_wake_function+0}
> > <ffffffffa067b484>{:ko2iblnd:kiblnd_init_tx_msg+308}
> > <ffffffff801ece0f>{vsnprintf+1406}
> > <ffffffff8030e73d>{__down_failed+53}
> > <ffffffffa06c6670>{:lquota:filter_quota_check+0}
> > <ffffffffa0843acf>{:obdfilter:.text.lock.filter_io_26+35}
> > <ffffffffa083b01e>{:obdfilter:filter_commitrw+126}
> > <ffffffff8030cc1f>{__down+147}
> > <ffffffff80133804>{default_wake_function+0}
> > <ffffffffa067b484>{:ko2iblnd:kiblnd_init_tx_msg+308}
> > <ffffffff8030e4ce>{schedule_timeout+411}
> > <ffffffffa07f8688>{:ost:ost_checksum_bulk+200}
> > <ffffffffa07fddd1>{:ost:ost_brw_write+9505}
> > <ffffffff80133804>{default_wake_function+0}
> > <ffffffffa06015ef>{:ptlrpc:lustre_msg_get_version+95}
> > <ffffffffa07f8060>{:ost:ost_bulk_timeout+0}
> > <ffffffffa06016e5>{:ptlrpc:lustre_msg_check_version+69}
> > <ffffffffa07f8060>{:ost:ost_bulk_timeout+0}
> > <ffffffffa080364d>{:ost:ost_handle+11661}
> > <ffffffff8015c830>{__rmqueue+218}
> > <ffffffff8030e73d>{__down_failed+53}
> > <ffffffffa06c6670>{:lquota:filter_quota_check+0}
> > <ffffffffa0843acf>{:obdfilter:.text.lock.filter_io_26+35}
> > <ffffffffa060a451>{:ptlrpc:ptlrpc_check_req+17}
> > <ffffffffa060c629>{:ptlrpc:ptlrpc_server_handle_request+2457}
> > <ffffffffa04df45e>{:libcfs:lcw_update_time+30}
> > <ffffffff80131bc7>{recalc_task_prio+337}
> > <ffffffffa05160a0>{:lnet:lnet_send+2544}
> > <ffffffffa083b01e>{:obdfilter:filter_commitrw+126}
> > <ffffffff8013f734>{__mod_timer+293}
> > <ffffffffa060ed05>{:ptlrpc:ptlrpc_main+3989}
> > <ffffffff80133804>{default_wake_function+0}
> > <ffffffffa060d270>{:ptlrpc:ptlrpc_retry_rqbds+0}
> > <ffffffffa060d270>{:ptlrpc:ptlrpc_retry_rqbds+0}
> > <ffffffff80110de3>{child_rip+8}
> > <ffffffffa060dd70>{:ptlrpc:ptlrpc_main+0}
> > <ffffffff80110ddb>{child_rip+0}
> > <ffffffff8030e4ce>{schedule_timeout+411}
> > <ffffffffa07f8688>{:ost:ost_checksum_bulk+200}
> > <ffffffffa07fddd1>{:ost:ost_brw_write+9505}
> > <ffffffff80133804>{default_wake_function+0}
> > <ffffffffa06015ef>{:ptlrpc:lustre_msg_get_version+95}
> > <ffffffffa07f8060>{:ost:ost_bulk_timeout+0}
> > <ffffffffa06016e5>{:ptlrpc:lustre_msg_check_version+69}
> > <ffffffffa07f8060>{:ost:ost_bulk_timeout+0}
> > <ffffffffa080364d>{:ost:ost_handle+11661}
> > <ffffffffa0516b48>{:lnet:lnet_match_blocked_msg+920}
> > <ffffffffa060a451>{:ptlrpc:ptlrpc_check_req+17}
> > <ffffffffa060c629>{:ptlrpc:ptlrpc_server_handle_request+2457}
> > <ffffffffa04df45e>{:libcfs:lcw_update_time+30}
> > <ffffffff8013f734>{__mod_timer+293}
> > <ffffffffa060ed05>{:ptlrpc:ptlrpc_main+3989}
> > <ffffffff80133804>{default_wake_function+0}
> > <ffffffffa060d270>{:ptlrpc:ptlrpc_retry_rqbds+0}
> > <ffffffffa060d270>{:ptlrpc:ptlrpc_retry_rqbds+0}
> > <ffffffff80110de3>{child_rip+8}
> > <ffffffffa060dd70>{:ptlrpc:ptlrpc_main+0}
> > <ffffffff80110ddb>{child_rip+0}
> > ll_ost_io_121 D 0000000000000002 0 17426 1 17427 17425
> > (L-TLB)
> > 0000010110113538 0000000000000046 0000010190113616 ffffffff801ece0f
> > 0000000000000000 ffffff00107763d8 000001010e14c6c0 0000000100000001
> > 00000101100e6800 0000000000113501
> > Call Trace:<ffffffff801ece0f>{vsnprintf+1406} <ffffffff8030cc1f>{__down+147}
> > <ffffffff80133804>{default_wake_function+0}
> > <ffffffffa067b484>{:ko2iblnd:kiblnd_init_tx_msg+308}
> > <ffffffff8030e73d>{__down_failed+53}
> > <ffffffffa06c6670>{:lquota:filter_quota_check+0}
> > <ffffffffa0843acf>{:obdfilter:.text.lock.filter_io_26+35}
> > <ffffffffa083b01e>{:obdfilter:filter_commitrw+126}
> > <ffffffff8030e4ce>{schedule_timeout+411}
> > <ffffffffa07f8688>{:ost:ost_checksum_bulk+200}
> > <ffffffffa07fddd1>{:ost:ost_brw_write+9505}
> > <ffffffff80133804>{default_wake_function+0}
> > <ffffffffa06015ef>{:ptlrpc:lustre_msg_get_version+95}
> > <ffffffffa07f8060>{:ost:ost_bulk_timeout+0}
> > <ffffffffa06016e5>{:ptlrpc:lustre_msg_check_version+69}
> > <ffffffffa07f8060>{:ost:ost_bulk_timeout+0}
> > <ffffffffa080364d>{:ost:ost_handle+11661}
> > <ffffffff8015c830>{__rmqueue+218}
> > <ffffffffa060a451>{:ptlrpc:ptlrpc_check_req+17}
> > <ffffffffa060c629>{:ptlrpc:ptlrpc_server_handle_request+2457}
> > <ffffffffa04df45e>{:libcfs:lcw_update_time+30}
> > <ffffffff8013f734>{__mod_timer+293}
> > <ffffffffa060ed05>{:ptlrpc:ptlrpc_main+3989}
> > <ffffffff80133804>{default_wake_function+0}
> > <ffffffffa060d270>{:ptlrpc:ptlrpc_retry_rqbds+0}
> > <ffffffffa060d270>{:ptlrpc:ptlrpc_retry_rqbds+0}
> > <ffffffff80110de3>{child_rip+8}
> > <ffffffffa060dd70>{:ptlrpc:ptlrpc_main+0}
> > <ffffffff80110ddb>{child_rip+0}
> > Lustre: 0:0:(watchdog.c:130:lcw_cb()) Skipped 4 previous similar messages
> > Lustre: 0:0:(linux-debug.c:167:libcfs_debug_dumpstack()) showing stack
> > for process 17398
> > Lustre: 0:0:(linux-debug.c:167:libcfs_debug_dumpstack()) Skipped 4
> > previous similar messages
> > ll_ost_io_93 D 0000000000000002 0 17398 1 17399 17397
> > (L-TLB)
> > 00000101157cd538 0000000000000046 00000101957cd616 ffffffff00000073
> > 0000000000000000 0000000010776508 0000010001053a20 0000000200000001
> > 00000101157a7800 0000000000161205
> > Call Trace:<ffffffff8030ec6c>{.text.lock.spinlock+2}
> > <ffffffff8030cc1f>{__down+147}
> > <ffffffff80133804>{default_wake_function+0}
> > <ffffffffa067b484>{:ko2iblnd:kiblnd_init_tx_msg+308}
> > <ffffffff8030e73d>{__down_failed+53}
> > <ffffffffa06c6670>{:lquota:filter_quota_check+0}
> > <ffffffffa0843acf>{:obdfilter:.text.lock.filter_io_26+35}
> > <ffffffffa04f5539>{:lvfs:pop_ctxt+505}
> > <ffffffff80131bc7>{recalc_task_prio+337}
> > <ffffffffa05160a0>{:lnet:lnet_send+2544}
> > <ffffffffa083b01e>{:obdfilter:filter_commitrw+126}
> > <ffffffffa07f8801>{:ost:ost_checksum_bulk+577}
> > <ffffffffa07f8688>{:ost:ost_checksum_bulk+200}
> > <ffffffffa07fddd1>{:ost:ost_brw_write+9505}
> > <ffffffff8017a62e>{end_buffer_async_read+0}
> > <ffffffff80133804>{default_wake_function+0}
> > <ffffffffa06015ef>{:ptlrpc:lustre_msg_get_version+95}
> > <ffffffffa07f8060>{:ost:ost_bulk_timeout+0}
> > <ffffffffa06016e5>{:ptlrpc:lustre_msg_check_version+69}
> > <ffffffffa07f8060>{:ost:ost_bulk_timeout+0}
> > <ffffffffa080364d>{:ost:ost_handle+11661}
> > <ffffffff8017a62e>{end_buffer_async_read+0}
> > <ffffffff8017a62e>{end_buffer_async_read+0}
> > <ffffffff8017a62e>{end_buffer_async_read+0}
> > <ffffffff8017a62e>{end_buffer_async_read+0}
> > <ffffffff8015c830>{__rmqueue+218} <ffffffff8015c830>{__rmqueue+218}
> > <ffffffffa060a451>{:ptlrpc:ptlrpc_check_req+17}
> > <ffffffffa060c629>{:ptlrpc:ptlrpc_server_handle_request+2457}
> > <ffffffffa04df45e>{:libcfs:lcw_update_time+30}
> > <ffffffff8013f734>{__mod_timer+293}
> > <ffffffffa060ed05>{:ptlrpc:ptlrpc_main+3989}
> > <ffffffff8017a62e>{end_buffer_async_read+0}
> > <ffffffff80133804>{default_wake_function+0}
> > <ffffffffa060d270>{:ptlrpc:ptlrpc_retry_rqbds+0}
> > <ffffffffa060d270>{:ptlrpc:ptlrpc_retry_rqbds+0}
> > <ffffffff80110de3>{child_rip+8}
> > <ffffffffa060dd70>{:ptlrpc:ptlrpc_main+0}
> > <ffffffff80110ddb>{child_rip+0}
> >
> > LustreError: 17397:0:(filter_io_26.c:700:filter_commitrw_write())
> > testfs-OST0001: slow i_mutex 223s
> > LustreError: 17398:0:(filter_io_26.c:700:filter_commitrw_write())
> > testfs-OST0001: slow i_mutex 223s
> > LustreError: 17387:0:(filter_io_26.c:765:filter_commitrw_write())
> > testfs-OST0001: slow direct_io 223s
> > Lustre: 17397:0:(watchdog.c:312:lcw_update_time()) Expired watchdog for
> > pid 17397 disabled after 223.2918s
> > Lustre: 17387:0:(watchdog.c:312:lcw_update_time()) Expired watchdog for
> > pid 17387 disabled after 223.3108s
> > Lustre: 17387:0:(watchdog.c:312:lcw_update_time()) Skipped 5 previous
> > similar messages
> > slab: cache size-1620 error: slabs_full accounting error
> > slab: cache size-1620 error: slabs_full accounting error
> > slab: cache size-1620 error: slabs_full accounting error
> > Unable to handle kernel NULL pointer dereference at 0000000000000000 RIP:
> > <ffffffff801623c4>{s_show+62}
> > PML4 112a23067 PGD 114d4d067 PMD 0
> > Oops: 0000 [1] SMP
> > CPU 1
> > Modules linked in: obdfilter(U) fsfilt_ldiskfs(U) ost(U) mgc(U)
> > ldiskfs(U) lustre(U) lov(U) mdc(U) lquota(U) osc(U) ko2iblnd(U)
> > ptlrpc(U) obdclass(U) lnet(U) lvfs(U) libcfs(U) sg(U) dell_rbu(U)
> > autofs4(U) i2c_nforce2(U) i2c_amd756(U) i2c_isa(U) i2c_amd8111(U)
> > i2c_i801(U) i2c_core(U) qlgc_vnic(U) iw_cxgb3(U) cxgb3(U) mlx4_ib(U)
> > mlx4_core(U) ib_mthca(U) ipmi_devintf(U) ipmi_si(U) ipmi_msghandler(U)
> > rdma_ucm(U) ib_sdp(U) rdma_cm(U) iw_cm(U) ib_addr(U) ib_ipoib(U) md5(U)
> > ipv6(U) cpufreq_powersave(U) mptctl(U) dm_mirror(U) dm_round_robin(U)
> > dm_multipath(U) dm_mod(U) sr_mod(U) usb_storage(U) joydev(U) button(U)
> > battery(U) ac(U) uhci_hcd(U) ehci_hcd(U) hw_random(U) ib_ipath(U)
> > ib_umad(U) ib_ucm(U) ib_uverbs(U) ib_cm(U) ib_sa(U) ib_mad(U) ib_core(U)
> > ata_piix(U) libata(U) ext3(U) jbd(U) tg3(U) s2io(U) qla2400(U)
> > qla2xxx(U) scsi_transport_fc(U) nfs(U) nfs_acl(U) lockd(U) sunrpc(U)
> > mptsas(U) mptscsi(U) mptbase(U) megaraid_sas(U) e1000(U) bnx2(U)
> > sd_mod(U) scsi_mod(U)
> > Pid: 15569, comm: collectl Not tainted 2.6.9-67.0.7.EL_lustre.1.6.5.1smp
> > RIP: 0010:[<ffffffff801623c4>] <ffffffff801623c4>{s_show+62}
> > RSP: 0018:0000010115823e68 EFLAGS: 00010012
> > RAX: ffffffff80329f7a RBX: 00000100cffa5580 RCX: 00000100cffa5501
> > RDX: 0000000000000004 RSI: 0000000000000000 RDI: 00000100cffa56e8
> > RBP: ffffffff80329f7a R08: 00000000fffffffd R09: 0000000000000000
> > R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
> > R13: 0000000000001000 R14: 000001012b617b80 R15: 0000000000000020
> > FS: 0000002a9630ee80(0000) GS:ffffffff8048e780(0000) knlGS:0000000000000000
> > CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> > CR2: 0000000000000000 CR3: 000000012bd38000 CR4: 00000000000006e0
> > Process collectl (pid: 15569, threadinfo 0000010115822000, task
> > 00000101280a4800)
> > Stack: 0000000000000000 0000000000000000 0000000000000008 00000100cffa5580
> > 000001012b617b80 0000000000000000 0000000000001000 0000000000000ea2
> > 0000000000000000 ffffffff80196c1a
> > Call Trace:<ffffffff80196c1a>{seq_read+445} <ffffffff80178c28>{vfs_read+207}
> > <ffffffff80178e84>{sys_read+69} <ffffffff8011022a>{system_call+126}
> >
> >
> > Code: 48 8b 06 0f 18 08 48 8d 83 18 01 00 00 48 39 c6 74 2e 8b 93
> > RIP <ffffffff801623c4>{s_show+62} RSP <0000010115823e68>
> > CR2: 0000000000000000
> > <0>Kernel panic - not syncing: Oops
> > NMI Watchdog detected LOCKUP, CPU=2, registers:
> > CPU 2
> > Modules linked in: obdfilter(U) fsfilt_ldiskfs(U) ost(U) mgc(U)
> > ldiskfs(U) lustre(U) lov(U) mdc(U) lquota(U) osc(U) ko2iblnd(U)
> > ptlrpc(U) obdclass(U) lnet(U) lvfs(U) libcfs(U) sg(U) dell_rbu(U)
> > autofs4(U) i2c_nforce2(U) i2c_amd756(U) i2c_isa(U) i2c_amd8111(U)
> > i2c_i801(U) i2c_core(U) qlgc_vnic(U) iw_cxgb3(U) cxgb3(U) mlx4_ib(U)
> > mlx4_core(U) ib_mthca(U) ipmi_devintf(U) ipmi_si(U) ipmi_msghandler(U)
> > rdma_ucm(U) ib_sdp(U) rdma_cm(U) iw_cm(U) ib_addr(U) ib_ipoib(U) md5(U)
> > ipv6(U) cpufreq_powersave(U) mptctl(U) dm_mirror(U) dm_round_robin(U)
> > dm_multipath(U) dm_mod(U) sr_mod(U) usb_storage(U) joydev(U) button(U)
> > battery(U) ac(U) uhci_hcd(U) ehci_hcd(U) hw_random(U) ib_ipath(U)
> > ib_umad(U) ib_ucm(U) ib_uverbs(U) ib_cm(U) ib_sa(U) ib_mad(U) ib_core(U)
> > ata_piix(U) libata(U) ext3(U) jbd(U) tg3(U) s2io(U) qla2400(U)
> > qla2xxx(U) scsi_transport_fc(U) nfs(U) nfs_acl(U) lockd(U) sunrpc(U)
> > mptsas(U) mptscsi(U) mptbase(U) megaraid_sas(U) e1000(U) bnx2(U)
> > sd_mod(U) scsi_mod(U)
> > Pid: 12646, comm: klogd Not tainted 2.6.9-67.0.7.EL_lustre.1.6.5.1smp
> > RIP: 0010:[<ffffffff8030ec6c>] <ffffffff8030ec6c>{.text.lock.spinlock+2}
> > RSP: 0018:000001012a0a7b88 EFLAGS: 00000086
> > RAX: 0000000000000010 RBX: 00000100cffa56e8 RCX: 0000000000000000
> > RDX: 0000000000000000 RSI: 00000000000004d0 RDI: 00000100cffa56e8
> > RBP: 000001012bc1e0c0 R08: 000001012a0a7cf0 R09: 6d5f697363732029
> > R10: 0000000000000053 R11: 0000000000000246 R12: 00000100cffa5688
> > R13: 00000100cffa5580 R14: 00000000000004d0 R15: 00000000000003e6
> > FS: 0000002a958a5b00(0000) GS:ffffffff8048e800(0000) knlGS:0000000000000000
> > CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> > CR2: 0000007fbfffa868 CR3: 000000012bd6e000 CR4: 00000000000006e0
> > Process klogd (pid: 12646, threadinfo 000001012a0a6000, task
> > 00000101299e2030)
> > Stack: 000000000000000c ffffffff80160901 00000100cffa5580 00000100cffa5580
> > 00000000000004d0 0000000000000400 0000000000000400 0000000000000000
> > 00000000000003e6 ffffffff801607d0
> > Call Trace:<ffffffff80160901>{cache_alloc_refill+96}
> > <ffffffff801607d0>{__kmalloc+123}
> > <ffffffff802adc58>{alloc_skb+65}
> > <ffffffff802ac770>{sock_alloc_send_pskb+135}
> > <ffffffff80133855>{__wake_up_common+67}
> > <ffffffff801338ab>{__wake_up+54}
> > <ffffffff8030899a>{unix_dgram_sendmsg+364}
> > <ffffffff802aa430>{sock_aio_write+306}
> > <ffffffff80178d0f>{do_sync_write+178}
> > <ffffffff80137822>{do_syslog+482}
> > <ffffffff801351dc>{autoremove_wake_function+0}
> > <ffffffff801351dc>{autoremove_wake_function+0}
> > <ffffffff801351dc>{autoremove_wake_function+0}
> > <ffffffff80193ed0>{dnotify_parent+34}
> > <ffffffff80178e1d>{vfs_write+226} <ffffffff80178ef2>{sys_write+69}
> > <ffffffff8011022a>{system_call+126}
> >
> > Code: 83 3b 00 7e f9 e9 60 fc ff ff f3 90 83 3b 00 7e f9 e9 ce fc
> > Kernel panic - not syncing: nmi watchdog
> > <1>Unable to handle kernel NULL pointer dereference at 00000000000000ff
> > RIP:
> > [<00000000000000ff>]
> > PML4 11b4a6067 PGD 0
> > Oops: 0010 [2] SMP
> > CPU 2
> > Modules linked in: obdfilter(U) fsfilt_ldiskfs(U) ost(U) mgc(U)
> > ldiskfs(U) lustre(U) lov(U) mdc(U) lquota(U) osc(U) ko2iblnd(U)
> > ptlrpc(U) obdclass(U) lnet(U) lvfs(U) libcfs(U) sg(U) dell_rbu(U)
> > autofs4(U) i2c_nforce2(U) i2c_amd756(U) i2c_isa(U) i2c_amd8111(U)
> > i2c_i801(U) i2c_core(U) qlgc_vnic(U) iw_cxgb3(U) cxgb3(U) mlx4_ib(U)
> > mlx4_core(U) ib_mthca(U) ipmi_devintf(U) ipmi_si(U) ipmi_msghandler(U)
> > rdma_ucm(U) ib_sdp(U) rdma_cm(U) iw_cm(U) ib_addr(U) ib_ipoib(U) md5(U)
> > ipv6(U) cpufreq_powersave(U) mptctl(U) dm_mirror(U) dm_round_robin(U)
> > dm_multipath(U) dm_mod(U) sr_mod(U) usb_storage(U) joydev(U) button(U)
> > battery(U) ac(U) uhci_hcd(U) ehci_hcd(U) hw_random(U) ib_ipath(U)
> > ib_umad(U) ib_ucm(U) ib_uverbs(U) ib_cm(U) ib_sa(U) ib_mad(U) ib_core(U)
> > ata_piix(U) libata(U) ext3(U) jbd(U) tg3(U) s2io(U) qla2400(U)
> > qla2xxx(U) scsi_transport_fc(U) nfs(U) nfs_acl(U) lockd(U) sunrpc(U)
> > mptsas(U) mptscsi(U) mptbase(U) megaraid_sas(U) e1000(U) bnx2(U)
> > sd_mod(U) scsi_mod(U)
> > Pid: 12646, comm: klogd Not tainted 2.6.9-67.0.7.EL_lustre.1.6.5.1smp
> > RIP: 0010:[<00000000000000ff>] [<00000000000000ff>]
> > RSP: 0018:00000100cfb03fa0 EFLAGS: 00010006
> > RAX: 000001012a0a7fd8 RBX: 0000000000000000 RCX: 0000000000000002
> > RDX: 00000000000000ff RSI: 0000000000000000 RDI: 0000000000000002
> > RBP: 000001012bd71f58 R08: 0000000000000020 R09: 0000000000000000
> > R10: 0000000000000000 R11: 0000000000000000 R12: 00000100cffa5688
> > R13: 00000100cffa5580 R14: 00000000000004d0 R15: 00000000000003e6
> > FS: 0000002a958a5b00(0000) GS:ffffffff8048e800(0000) knlGS:0000000000000000
> > CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> > CR2: 00000000000000ff CR3: 000000012bd6e000 CR4: 00000000000006e0
> > Process klogd (pid: 12646, threadinfo 000001012a0a6000, task
> > 00000101299e2030)
> > Stack: ffffffff8011c56b ffffffff80321aac ffffffff80110a73
> > 000001012bd71cf8 <EOI>
> > 000000000000000d ffffffff80321aac 000000000000000d ffffffff80324122
> > 0000000000000001 0000000000000000
> > Call Trace:<IRQ> <ffffffff8011c56b>{smp_call_function_interrupt+64}
> > <ffffffff80110a73>{call_function_interrupt+99} <EOI>
> > <ffffffff8011c51e>{smp_send_stop+76}
> > <ffffffff8013744a>{panic+235} <ffffffff801116fc>{show_stack+241}
> > <ffffffff80111826>{show_registers+277}
> > <ffffffff80111b2d>{die_nmi+130}
> > <ffffffff8011d042>{nmi_watchdog_tick+210}
> > <ffffffff801123fa>{default_do_nmi+112}
> > <ffffffff8011d0f8>{do_nmi+115} <ffffffff8011100f>{paranoid_exit+0}
> > <ffffffff8030ec6c>{.text.lock.spinlock+2}
> >
> > Code: Bad RIP value.
> > RIP [<00000000000000ff>] RSP <00000100cfb03fa0>
> > CR2: 00000000000000ff
> > <0>Kernel panic - not syncing: Oops
> > Badness in panic at kernel/panic.c:118
> >
> > Call Trace:<IRQ> <ffffffff8013756e>{panic+527}
> > <ffffffff801130a5>{do_IRQ+266}
> > <ffffffff801107d1>{ret_from_intr+0} <ffffffff80111988>{oops_end+38}
> > <ffffffff801119a3>{oops_end+65}
> > <ffffffff80123e29>{do_page_fault+1125}
> > <ffffffff801ea99e>{kobject_release+0}
> > <ffffffff80131c55>{activate_task+124}
> > <ffffffff80132180>{try_to_wake_up+876}
> > <ffffffff80110c2d>{error_exit+0}
> > <ffffffff8011c56b>{smp_call_function_interrupt+64}
> > <ffffffff80110a73>{call_function_interrupt+99} <EOI>
> > <ffffffff8011c51e>{smp_send_stop+76}
> > <ffffffff8013744a>{panic+235} <ffffffff801116fc>{show_stack+241}
> > <ffffffff80111826>{show_registers+277}
> > <ffffffff80111b2d>{die_nmi+130}
> > <ffffffff8011d042>{nmi_watchdog_tick+210}
> > <ffffffff801440f3>{notifier_call_chain+31}
> > <ffffffff801123fa>{default_do_nmi+112} <ffffffff8011d0f8>{do_nmi+115}
> > <ffffffff8011100f>{paranoid_exit+0} <ffffffff80111988>{oops_end+38}
> >
> > Badness in i8042_panic_blink at drivers/input/serio/i8042.c:987
> >
> > Call Trace:<IRQ> <ffffffff8024478f>{i8042_panic_blink+238}
> > <ffffffff8013751c>{panic+445}
> > <ffffffff801130a5>{do_IRQ+266} <ffffffff801107d1>{ret_from_intr+0}
> > <ffffffff80111988>{oops_end+38} <ffffffff801119a3>{oops_end+65}
> > <ffffffff80123e29>{do_page_fault+1125}
> > <ffffffff801ea99e>{kobject_release+0}
> > <ffffffff80131c55>{activate_task+124}
> > <ffffffff80132180>{try_to_wake_up+876}
> > <ffffffff80110c2d>{error_exit+0}
> > <ffffffff8011c56b>{smp_call_function_interrupt+64}
> > <ffffffff80110a73>{call_function_interrupt+99} <EOI>
> > <ffffffff8011c51e>{smp_send_stop+76}
> > <ffffffff8013744a>{panic+235} <ffffffff801116fc>{show_stack+241}
> > <ffffffff80111826>{show_registers+277}
> > <ffffffff80111b2d>{die_nmi+130}
> > <ffffffff8011d042>{nmi_watchdog_tick+210}
> > <ffffffff801440f3>{notifier_call_chain+31}
> > <ffffffff801123fa>{default_do_nmi+112} <ffffffff8011d0f8>{do_nmi+115}
> > <ffffffff8011100f>{paranoid_exit+0} <ffffffff80111988>{oops_end+38}
> >
> > Badness in i8042_panic_blink at drivers/input/serio/i8042.c:990
> >
> > Call Trace:<IRQ> <ffffffff80244821>{i8042_panic_blink+384}
> > <ffffffff8013751c>{panic+445}
> > <ffffffff801130a5>{do_IRQ+266} <ffffffff801107d1>{ret_from_intr+0}
> > <ffffffff80111988>{oops_end+38} <ffffffff801119a3>{oops_end+65}
> > <ffffffff80123e29>{do_page_fault+1125}
> > <ffffffff801ea99e>{kobject_release+0}
> > <ffffffff80131c55>{activate_task+124}
> > <ffffffff80132180>{try_to_wake_up+876}
> > <ffffffff80110c2d>{error_exit+0}
> > <ffffffff8011c56b>{smp_call_function_interrupt+64}
> > <ffffffff80110a73>{call_function_interrupt+99} <EOI>
> > <ffffffff8011c51e>{smp_send_stop+76}
> > <ffffffff8013744a>{panic+235} <ffffffff801116fc>{show_stack+241}
> > <ffffffff80111826>{show_registers+277}
> > <ffffffff80111b2d>{die_nmi+130}
> > <ffffffff8011d042>{nmi_watchdog_tick+210}
> > <ffffffff801440f3>{notifier_call_chain+31}
> > <ffffffff801123fa>{default_do_nmi+112} <ffffffff8011d0f8>{do_nmi+115}
> > <ffffffff8011100f>{paranoid_exit+0} <ffffffff80111988>{oops_end+38}
> >
> > Badness in i8042_panic_blink at drivers/input/serio/i8042.c:992
> >
> > Call Trace:<IRQ> <ffffffff80244886>{i8042_panic_blink+485}
> > <ffffffff8013751c>{panic+445}
> > <ffffffff801130a5>{do_IRQ+266} <ffffffff801107d1>{ret_from_intr+0}
> > <ffffffff80111988>{oops_end+38} <ffffffff801119a3>{oops_end+65}
> > <ffffffff80123e29>{do_page_fault+1125}
> > <ffffffff801ea99e>{kobject_release+0}
> > <ffffffff80131c55>{activate_task+124}
> > <ffffffff80132180>{try_to_wake_up+876}
> > <ffffffff80110c2d>{error_exit+0}
> > <ffffffff8011c56b>{smp_call_function_interrupt+64}
> > <ffffffff80110a73>{call_function_interrupt+99} <EOI>
> > <ffffffff8011c51e>{smp_send_stop+76}
> > <ffffffff8013744a>{panic+235} <ffffffff801116fc>{show_stack+241}
> > <ffffffff80111826>{show_registers+277}
> > <ffffffff80111b2d>{die_nmi+130}
> > <ffffffff8011d042>{nmi_watchdog_tick+210}
> > <ffffffff801440f3>{notifier_call_chain+31}
> > <ffffffff801123fa>{default_do_nmi+112} <ffffffff8011d0f8>{do_nmi+115}
> > <ffffffff8011100f>{paranoid_exit+0} <ffffffff80111988>{oops_end+38}
> >
> >
> > Thank you in advance for helping me with this.
> >
> > Regards,
> >
> > Wojciech
> >
> >
>
> --
> Wojciech Turek
>
> Assistant System Manager
> High Performance Computing Service
> University of Cambridge
> Email: wjt27 at cam.ac.uk
> Tel: (+)44 1223 763517
>
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-discuss
Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.
More information about the lustre-discuss
mailing list