[lustre-discuss] lustre stability problem (2.12.0)
Bernd Melchers
melchers at zedat.fu-berlin.de
Tue Apr 9 04:41:40 PDT 2019
Hi All,
we have stability problems with our lustre on zfs installation:
CentOS 7.6, kernel 3.10.0-957.5.1.el7.x86_64, lustre 2.12.0, zfs 0.7.12
The ods servers have hanging kernel threads ll_ost_io. The threads locks
up the cpu and were killed after 22 sec, please have a look at the
attachement.
Is this a known problem and is this a zfs problem?
Mit freundlichen Grüßen
Bernd Melchers
--
Archiv- und Backup-Service | fab-service at zedat.fu-berlin.de
Freie Universität Berlin | Tel. +49-30-838-55905
-------------- next part --------------
[Fri Apr 5 14:49:05 2019] NMI watchdog: BUG: soft lockup - CPU#8 stuck for 22s! [ll_ost_io00_022:123082]
[Fri Apr 5 14:49:05 2019] Modules linked in: osp(OE) ofd(OE) lfsck(OE) ost(OE) mgc(OE) osd_zfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache libcfs(OE) iTCO_wdt iTCO_vendor_support skx_edac intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd opa_vnic pcspkr zfs(POE) zunicode(POE) zavl(POE) icp(POE) ses enclosure zcommon(POE) znvpair(POE) ast ttm spl(OE) drm_kms_helper syscopyarea sysfillrect rpcrdma sysimgblt fb_sys_fops sg joydev drm ib_isert iscsi_target_mod mei_me drm_panel_orientation_quirks mei lpc_ich i2c_i801 wmi ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp acpi_pad acpi_power_meter scsi_transport_srp
[Fri Apr 5 14:49:05 2019] scsi_tgt ib_ipoib(OE) rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm ip_tables dm_service_time sd_mod crc_t10dif crct10dif_generic crct10dif_pclmul crct10dif_common hfi1(OE) crc32c_intel rdmavt(OE) i40e(OE) ahci i2c_algo_bit mpt3sas ptp libahci ib_core pps_core raid_class libata scsi_transport_sas ipmi_si ipmi_devintf ipmi_msghandler nfit libnvdimm dm_multipath sunrpc dm_mirror dm_region_hash dm_log dm_mod
[Fri Apr 5 14:49:05 2019] CPU: 8 PID: 123082 Comm: ll_ost_io00_022 Tainted: P OEL ------------ 3.10.0-957.5.1.el7.x86_64 #1
[Fri Apr 5 14:49:05 2019] Hardware name: Intel Corporation S2600WFT/S2600WFT, BIOS SE5C620.86B.00.01.0013.030920180427 03/09/2018
[Fri Apr 5 14:49:05 2019] task: ffffa07371fcb0c0 ti: ffffa071e5c54000 task.ti: ffffa071e5c54000
[Fri Apr 5 14:49:05 2019] RIP: 0010:[<ffffffffc177484b>] [<ffffffffc177484b>] dbuf_prefetch+0xb/0x590 [zfs]
[Fri Apr 5 14:49:05 2019] RSP: 0018:ffffa071e5c57838 EFLAGS: 00000206
[Fri Apr 5 14:49:05 2019] RAX: 0000000000000000 RBX: 000000000074a315 RCX: 0000000000000002
[Fri Apr 5 14:49:05 2019] RDX: 000000162de556bc RSI: 0000000000000001 RDI: ffffa071ed958348
[Fri Apr 5 14:49:05 2019] RBP: ffffa071e5c57840 R08: 0000000000000020 R09: 0000000000000000
[Fri Apr 5 14:49:05 2019] R10: 000000000074a315 R11: ffffe1978c898e00 R12: 000000000074a315
[Fri Apr 5 14:49:05 2019] R13: ffffa071e5c57840 R14: ffffffffa9b65e92 R15: ffffa071e5c57840
[Fri Apr 5 14:49:05 2019] FS: 0000000000000000(0000) GS:ffffa073cdc00000(0000) knlGS:0000000000000000
[Fri Apr 5 14:49:05 2019] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[Fri Apr 5 14:49:05 2019] CR2: 00002aaaab9788a2 CR3: 0000000e50810000 CR4: 00000000007607e0
[Fri Apr 5 14:49:05 2019] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[Fri Apr 5 14:49:05 2019] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[Fri Apr 5 14:49:05 2019] PKRU: 00000000
[Fri Apr 5 14:49:05 2019] Call Trace:
[Fri Apr 5 14:49:05 2019] [<ffffffffc17909f0>] dmu_zfetch+0x320/0x520 [zfs]
[Fri Apr 5 14:49:05 2019] [<ffffffffc177d4f0>] dmu_buf_hold_array_by_dnode+0x420/0x4a0 [zfs]
[Fri Apr 5 14:49:05 2019] [<ffffffffc177d5d9>] dmu_buf_hold_array_by_bonus+0x69/0x90 [zfs]
[Fri Apr 5 14:49:05 2019] [<ffffffffc162644b>] osd_bufs_get+0x45b/0xd70 [osd_zfs]
[Fri Apr 5 14:49:05 2019] [<ffffffffc0ce832a>] ? cfs_percpt_unlock+0x1a/0xb0 [libcfs]
[Fri Apr 5 14:49:05 2019] [<ffffffffc0d78e67>] ofd_preprw+0x6b7/0x1160 [ofd]
[Fri Apr 5 14:49:05 2019] [<ffffffffc11b8d9f>] ? __req_capsule_get+0x15f/0x740 [ptlrpc]
[Fri Apr 5 14:49:05 2019] [<ffffffffc11fc7cb>] tgt_brw_read+0x9db/0x1e50 [ptlrpc]
[Fri Apr 5 14:49:05 2019] [<ffffffffc11cb1bd>] ? null_alloc_rs+0x16d/0x340 [ptlrpc]
[Fri Apr 5 14:49:05 2019] [<ffffffffc13a8929>] ? lprocfs_counter_add+0xf9/0x160 [obdclass]
[Fri Apr 5 14:49:05 2019] [<ffffffffc11cb1d6>] ? null_alloc_rs+0x186/0x340 [ptlrpc]
[Fri Apr 5 14:49:05 2019] [<ffffffffc1193a5f>] ? lustre_pack_reply_v2+0x14f/0x280 [ptlrpc]
[Fri Apr 5 14:49:05 2019] [<ffffffffc1193bff>] ? lustre_pack_reply_flags+0x6f/0x1e0 [ptlrpc]
[Fri Apr 5 14:49:05 2019] [<ffffffffc1193d81>] ? lustre_pack_reply+0x11/0x20 [ptlrpc]
[Fri Apr 5 14:49:05 2019] [<ffffffffc11fb35a>] tgt_request_handle+0xaea/0x1580 [ptlrpc]
[Fri Apr 5 14:49:05 2019] [<ffffffffc11d4a51>] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc]
[Fri Apr 5 14:49:05 2019] [<ffffffffc0cd0bde>] ? ktime_get_real_seconds+0xe/0x10 [libcfs]
[Fri Apr 5 14:49:05 2019] [<ffffffffc119f92b>] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc]
[Fri Apr 5 14:49:05 2019] [<ffffffffc119c7b5>] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc]
[Fri Apr 5 14:49:05 2019] [<ffffffffa94d6802>] ? default_wake_function+0x12/0x20
[Fri Apr 5 14:49:05 2019] [<ffffffffa94cbadb>] ? __wake_up_common+0x5b/0x90
[Fri Apr 5 14:49:05 2019] [<ffffffffc11a325c>] ptlrpc_main+0xafc/0x1fc0 [ptlrpc]
[Fri Apr 5 14:49:05 2019] [<ffffffffc11a2760>] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc]
[Fri Apr 5 14:49:05 2019] [<ffffffffa94c1c71>] kthread+0xd1/0xe0
[Fri Apr 5 14:49:05 2019] [<ffffffffa94c1ba0>] ? insert_kthread_work+0x40/0x40
[Fri Apr 5 14:49:05 2019] [<ffffffffa9b74c1d>] ret_from_fork_nospec_begin+0x7/0x21
[Fri Apr 5 14:49:05 2019] [<ffffffffa94c1ba0>] ? insert_kthread_work+0x40/0x40
[Fri Apr 5 14:49:05 2019] Code: 00 00 00 48 8b 53 40 e9 33 fe ff ff e8 6f 24 d5 e7 eb d4 0f 1f 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 41 57 <41> 56 41 55 41 54 49 89 d4 53 48 89 fb 48 81 ec f8 00 00 00 65
[Fri Apr 5 14:49:33 2019] NMI watchdog: BUG: soft lockup - CPU#8 stuck for 22s! [ll_ost_io00_022:123082]
[Fri Apr 5 14:49:33 2019] Modules linked in: osp(OE) ofd(OE) lfsck(OE) ost(OE) mgc(OE) osd_zfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache libcfs(OE) iTCO_wdt iTCO_vendor_support skx_edac intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd opa_vnic pcspkr zfs(POE) zunicode(POE) zavl(POE) icp(POE) ses enclosure zcommon(POE) znvpair(POE) ast ttm spl(OE) drm_kms_helper syscopyarea sysfillrect rpcrdma sysimgblt fb_sys_fops sg joydev drm ib_isert iscsi_target_mod mei_me drm_panel_orientation_quirks mei lpc_ich i2c_i801 wmi ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp acpi_pad acpi_power_meter scsi_transport_srp
[Fri Apr 5 14:49:33 2019] scsi_tgt ib_ipoib(OE) rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm ip_tables dm_service_time sd_mod crc_t10dif crct10dif_generic crct10dif_pclmul crct10dif_common hfi1(OE) crc32c_intel rdmavt(OE) i40e(OE) ahci i2c_algo_bit mpt3sas ptp libahci ib_core pps_core raid_class libata scsi_transport_sas ipmi_si ipmi_devintf ipmi_msghandler nfit libnvdimm dm_multipath sunrpc dm_mirror dm_region_hash dm_log dm_mod
[Fri Apr 5 14:49:33 2019] CPU: 8 PID: 123082 Comm: ll_ost_io00_022 Tainted: P OEL ------------ 3.10.0-957.5.1.el7.x86_64 #1
[Fri Apr 5 14:49:33 2019] Hardware name: Intel Corporation S2600WFT/S2600WFT, BIOS SE5C620.86B.00.01.0013.030920180427 03/09/2018
[Fri Apr 5 14:49:33 2019] task: ffffa07371fcb0c0 ti: ffffa071e5c54000 task.ti: ffffa071e5c54000
[Fri Apr 5 14:49:33 2019] RIP: 0010:[<ffffffffc1774874>] [<ffffffffc1774874>] dbuf_prefetch+0x34/0x590 [zfs]
[Fri Apr 5 14:49:33 2019] RSP: 0018:ffffa071e5c57720 EFLAGS: 00000287
[Fri Apr 5 14:49:33 2019] RAX: 0000000000000000 RBX: ffffa07fc5568000 RCX: 0000000000000002
[Fri Apr 5 14:49:33 2019] RDX: 00000016f20cd276 RSI: 0000000000000001 RDI: ffffa071ed958348
[Fri Apr 5 14:49:33 2019] RBP: ffffa071e5c57840 R08: 0000000000000020 R09: 0000000000000000
[Fri Apr 5 14:49:33 2019] R10: 000000000074a315 R11: ffffe1978c898e00 R12: ffffa06e3f6cf900
[Fri Apr 5 14:49:33 2019] R13: ffffa06e3f6cf978 R14: 0000008000000080 R15: 01ffad7ec748fe60
[Fri Apr 5 14:49:33 2019] FS: 0000000000000000(0000) GS:ffffa073cdc00000(0000) knlGS:0000000000000000
[Fri Apr 5 14:49:33 2019] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[Fri Apr 5 14:49:33 2019] CR2: 00002aaaab9788a2 CR3: 0000000e50810000 CR4: 00000000007607e0
[Fri Apr 5 14:49:33 2019] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[Fri Apr 5 14:49:33 2019] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[Fri Apr 5 14:49:33 2019] PKRU: 00000000
[Fri Apr 5 14:49:33 2019] Call Trace:
[Fri Apr 5 14:49:33 2019] [<ffffffffa961c0e5>] ? kmem_cache_alloc+0x35/0x1f0
[Fri Apr 5 14:49:33 2019] [<ffffffffc09753c9>] ? spl_kmem_cache_alloc+0x99/0x150 [spl]
[Fri Apr 5 14:49:33 2019] [<ffffffffa9b65e92>] ? mutex_lock+0x12/0x2f
[Fri Apr 5 14:49:33 2019] [<ffffffffc17909f0>] dmu_zfetch+0x320/0x520 [zfs]
[Fri Apr 5 14:49:33 2019] [<ffffffffc177d4f0>] dmu_buf_hold_array_by_dnode+0x420/0x4a0 [zfs]
[Fri Apr 5 14:49:33 2019] [<ffffffffc177d5d9>] dmu_buf_hold_array_by_bonus+0x69/0x90 [zfs]
[Fri Apr 5 14:49:33 2019] [<ffffffffc162644b>] osd_bufs_get+0x45b/0xd70 [osd_zfs]
[Fri Apr 5 14:49:33 2019] [<ffffffffc0ce832a>] ? cfs_percpt_unlock+0x1a/0xb0 [libcfs]
[Fri Apr 5 14:49:33 2019] [<ffffffffc0d78e67>] ofd_preprw+0x6b7/0x1160 [ofd]
[Fri Apr 5 14:49:33 2019] [<ffffffffc11b8d9f>] ? __req_capsule_get+0x15f/0x740 [ptlrpc]
[Fri Apr 5 14:49:33 2019] [<ffffffffc11fc7cb>] tgt_brw_read+0x9db/0x1e50 [ptlrpc]
[Fri Apr 5 14:49:33 2019] [<ffffffffc11cb1bd>] ? null_alloc_rs+0x16d/0x340 [ptlrpc]
[Fri Apr 5 14:49:33 2019] [<ffffffffc13a8929>] ? lprocfs_counter_add+0xf9/0x160 [obdclass]
[Fri Apr 5 14:49:33 2019] [<ffffffffc11cb1d6>] ? null_alloc_rs+0x186/0x340 [ptlrpc]
[Fri Apr 5 14:49:33 2019] [<ffffffffc1193a5f>] ? lustre_pack_reply_v2+0x14f/0x280 [ptlrpc]
[Fri Apr 5 14:49:33 2019] [<ffffffffc1193bff>] ? lustre_pack_reply_flags+0x6f/0x1e0 [ptlrpc]
[Fri Apr 5 14:49:33 2019] [<ffffffffc1193d81>] ? lustre_pack_reply+0x11/0x20 [ptlrpc]
[Fri Apr 5 14:49:33 2019] [<ffffffffc11fb35a>] tgt_request_handle+0xaea/0x1580 [ptlrpc]
[Fri Apr 5 14:49:33 2019] [<ffffffffc11d4a51>] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc]
[Fri Apr 5 14:49:33 2019] [<ffffffffc0cd0bde>] ? ktime_get_real_seconds+0xe/0x10 [libcfs]
[Fri Apr 5 14:49:33 2019] [<ffffffffc119f92b>] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc]
[Fri Apr 5 14:49:33 2019] [<ffffffffc119c7b5>] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc]
[Fri Apr 5 14:49:33 2019] [<ffffffffa94d6802>] ? default_wake_function+0x12/0x20
[Fri Apr 5 14:49:33 2019] [<ffffffffa94cbadb>] ? __wake_up_common+0x5b/0x90
[Fri Apr 5 14:49:33 2019] [<ffffffffc11a325c>] ptlrpc_main+0xafc/0x1fc0 [ptlrpc]
[Fri Apr 5 14:49:33 2019] [<ffffffffc11a2760>] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc]
[Fri Apr 5 14:49:33 2019] [<ffffffffa94c1c71>] kthread+0xd1/0xe0
[Fri Apr 5 14:49:33 2019] [<ffffffffa94c1ba0>] ? insert_kthread_work+0x40/0x40
[Fri Apr 5 14:49:33 2019] [<ffffffffa9b74c1d>] ret_from_fork_nospec_begin+0x7/0x21
[Fri Apr 5 14:49:33 2019] [<ffffffffa94c1ba0>] ? insert_kthread_work+0x40/0x40
[Fri Apr 5 14:49:33 2019] Code: 41 57 41 56 41 55 41 54 49 89 d4 53 48 89 fb 48 81 ec f8 00 00 00 65 48 8b 04 25 28 00 00 00 48 89 45 d0 31 c0 48 39 57 70 73 2c <48> 8b 45 d0 65 48 33 04 25 28 00 00 00 0f 85 f9 04 00 00 48 81
[Fri Apr 5 14:50:00 2019] NMI watchdog: BUG: soft lockup - CPU#8 stuck for 22s! [ll_ost_io00_022:123082]
[Fri Apr 5 14:50:00 2019] Modules linked in: osp(OE) ofd(OE) lfsck(OE) ost(OE) mgc(OE) osd_zfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache libcfs(OE) iTCO_wdt iTCO_vendor_support skx_edac intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd opa_vnic pcspkr zfs(POE) zunicode(POE) zavl(POE) icp(POE) ses enclosure zcommon(POE) znvpair(POE) ast ttm spl(OE) drm_kms_helper syscopyarea sysfillrect rpcrdma sysimgblt fb_sys_fops sg joydev drm ib_isert iscsi_target_mod mei_me drm_panel_orientation_quirks mei lpc_ich i2c_i801 wmi ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp acpi_pad acpi_power_meter scsi_transport_srp
[Fri Apr 5 14:50:01 2019] scsi_tgt ib_ipoib(OE) rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm ip_tables dm_service_time sd_mod crc_t10dif crct10dif_generic crct10dif_pclmul crct10dif_common hfi1(OE) crc32c_intel rdmavt(OE) i40e(OE) ahci i2c_algo_bit mpt3sas ptp libahci ib_core pps_core raid_class libata scsi_transport_sas ipmi_si ipmi_devintf ipmi_msghandler nfit libnvdimm dm_multipath sunrpc dm_mirror dm_region_hash dm_log dm_mod
[Fri Apr 5 14:50:01 2019] CPU: 8 PID: 123082 Comm: ll_ost_io00_022 Tainted: P OEL ------------ 3.10.0-957.5.1.el7.x86_64 #1
[Fri Apr 5 14:50:01 2019] Hardware name: Intel Corporation S2600WFT/S2600WFT, BIOS SE5C620.86B.00.01.0013.030920180427 03/09/2018
[Fri Apr 5 14:50:01 2019] task: ffffa07371fcb0c0 ti: ffffa071e5c54000 task.ti: ffffa071e5c54000
[Fri Apr 5 14:50:01 2019] RIP: 0010:[<ffffffffc1774840>] [<ffffffffc1774840>] dbuf_prefetch+0x0/0x590 [zfs]
[Fri Apr 5 14:50:01 2019] RSP: 0018:ffffa071e5c57848 EFLAGS: 00000202
[Fri Apr 5 14:50:01 2019] RAX: 0000000000000000 RBX: 000000000074a315 RCX: 0000000000000002
[Fri Apr 5 14:50:01 2019] RDX: 00000017b44f1bcc RSI: 0000000000000001 RDI: ffffa071ed958348
[Fri Apr 5 14:50:01 2019] RBP: ffffa071e5c578c0 R08: 0000000000000020 R09: 0000000000000000
[Fri Apr 5 14:50:01 2019] R10: 000000000074a315 R11: ffffe1978c898e00 R12: 0000000000000020
[Fri Apr 5 14:50:01 2019] R13: ffffa071e5c57840 R14: 000000000074a315 R15: ffffe1978c898e00
[Fri Apr 5 14:50:01 2019] FS: 0000000000000000(0000) GS:ffffa073cdc00000(0000) knlGS:0000000000000000
[Fri Apr 5 14:50:01 2019] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[Fri Apr 5 14:50:01 2019] CR2: 00002aaaab9788a2 CR3: 0000000e50810000 CR4: 00000000007607e0
[Fri Apr 5 14:50:01 2019] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[Fri Apr 5 14:50:01 2019] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[Fri Apr 5 14:50:01 2019] PKRU: 00000000
[Fri Apr 5 14:50:01 2019] Call Trace:
[Fri Apr 5 14:50:01 2019] [<ffffffffc17909f0>] ? dmu_zfetch+0x320/0x520 [zfs]
[Fri Apr 5 14:50:01 2019] [<ffffffffc177d4f0>] dmu_buf_hold_array_by_dnode+0x420/0x4a0 [zfs]
[Fri Apr 5 14:50:01 2019] [<ffffffffc177d5d9>] dmu_buf_hold_array_by_bonus+0x69/0x90 [zfs]
[Fri Apr 5 14:50:01 2019] [<ffffffffc162644b>] osd_bufs_get+0x45b/0xd70 [osd_zfs]
[Fri Apr 5 14:50:01 2019] [<ffffffffc0ce832a>] ? cfs_percpt_unlock+0x1a/0xb0 [libcfs]
[Fri Apr 5 14:50:01 2019] [<ffffffffc0d78e67>] ofd_preprw+0x6b7/0x1160 [ofd]
[Fri Apr 5 14:50:01 2019] [<ffffffffc11b8d9f>] ? __req_capsule_get+0x15f/0x740 [ptlrpc]
[Fri Apr 5 14:50:01 2019] [<ffffffffc11fc7cb>] tgt_brw_read+0x9db/0x1e50 [ptlrpc]
[Fri Apr 5 14:50:01 2019] [<ffffffffc11cb1bd>] ? null_alloc_rs+0x16d/0x340 [ptlrpc]
[Fri Apr 5 14:50:01 2019] [<ffffffffc13a8929>] ? lprocfs_counter_add+0xf9/0x160 [obdclass]
[Fri Apr 5 14:50:01 2019] [<ffffffffc11cb1d6>] ? null_alloc_rs+0x186/0x340 [ptlrpc]
[Fri Apr 5 14:50:01 2019] [<ffffffffc1193a5f>] ? lustre_pack_reply_v2+0x14f/0x280 [ptlrpc]
[Fri Apr 5 14:50:01 2019] [<ffffffffc1193bff>] ? lustre_pack_reply_flags+0x6f/0x1e0 [ptlrpc]
[Fri Apr 5 14:50:01 2019] [<ffffffffc1193d81>] ? lustre_pack_reply+0x11/0x20 [ptlrpc]
[Fri Apr 5 14:50:01 2019] [<ffffffffc11fb35a>] tgt_request_handle+0xaea/0x1580 [ptlrpc]
[Fri Apr 5 14:50:01 2019] [<ffffffffc11d4a51>] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc]
[Fri Apr 5 14:50:01 2019] [<ffffffffc0cd0bde>] ? ktime_get_real_seconds+0xe/0x10 [libcfs]
[Fri Apr 5 14:50:01 2019] [<ffffffffc119f92b>] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc]
[Fri Apr 5 14:50:01 2019] [<ffffffffc119c7b5>] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc]
[Fri Apr 5 14:50:01 2019] [<ffffffffa94d6802>] ? default_wake_function+0x12/0x20
[Fri Apr 5 14:50:01 2019] [<ffffffffa94cbadb>] ? __wake_up_common+0x5b/0x90
[Fri Apr 5 14:50:01 2019] [<ffffffffc11a325c>] ptlrpc_main+0xafc/0x1fc0 [ptlrpc]
[Fri Apr 5 14:50:01 2019] [<ffffffffc11a2760>] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc]
[Fri Apr 5 14:50:01 2019] [<ffffffffa94c1c71>] kthread+0xd1/0xe0
[Fri Apr 5 14:50:01 2019] [<ffffffffa94c1ba0>] ? insert_kthread_work+0x40/0x40
[Fri Apr 5 14:50:01 2019] [<ffffffffa9b74c1d>] ret_from_fork_nospec_begin+0x7/0x21
[Fri Apr 5 14:50:01 2019] [<ffffffffa94c1ba0>] ? insert_kthread_work+0x40/0x40
[Fri Apr 5 14:50:01 2019] Code: 0e 01 00 4c 89 6b 30 48 89 83 88 00 00 00 48 8b 53 40 e9 33 fe ff ff e8 6f 24 d5 e7 eb d4 0f 1f 00 66 2e 0f 1f 84 00 00 00 00 00 <0f> 1f 44 00 00 55 48 89 e5 41 57 41 56 41 55 41 54 49 89 d4 53
...
More information about the lustre-discuss
mailing list