[lustre-discuss] Kernel oops with lustre 2.17.0 on rocky 9.7 kernel 5.14.0-611.13.1

Nguyen Hoang Minh minhnh at cmsinc.co.jp
Fri Sep 4 13:19:02 UTC 2026


Hi,

Due to a known bug LU-19121, we got some inaccessible files on our 
Lustre 2.16.1.

Since this bug cannot be fixed in 2.16.1, we have recently upgrade our 
cluster to Lustre 2.17.0 and ran e2fsck on all MDTs. e2fsck was be able 
to fix inode corruptions on mdt1. However, if we mount mdt1, the host 
server will immediately crashes with the following log:

=====================

[  145.375972] Lustre: Lustre: Build Version: 2.17.0
[  148.190895] Lustre: MGS: mounting server target with '-t lustre' 
deprecated, use '-t lustre_tgt'
[  149.386969] LDISKFS-fs (sdb): mounted filesystem 
22a869df-1f5d-47b1-9d17-03380f3ab0b6 r/w with ordered data mode. Quota 
mode: journalled.
[  150.629175] SELinux: (dev lustre, type lustre) has no xattr support
[  150.629235] SELinux: (dev lustre, type lustre) falling back to genfs
[ 3307.791077] LDISKFS-fs warning (device sdc): 
ldiskfs_multi_mount_protect:334: MMP interval 42 higher than expected, 
please wait.

[ 3350.684500] LDISKFS-fs (sdc): 9 truncates cleaned up
[ 3350.684557] LDISKFS-fs (sdc): recovery complete
[ 3350.687957] LDISKFS-fs (sdc): mounted filesystem 
57aec60d-ce35-43e3-bb8c-ba329938b8f3 r/w with ordered data mode. Quota 
mode: journalled.
[ 3352.248028] Lustre: lustreFS-MDT0000: Imperative Recovery enabled, 
recovery window shrunk from 300-900 down to 150-900
[ 3352.253154] Lustre: lustreFS-MDT0000: in recovery but waiting for the 
first client to connect
[ 3352.253230] SELinux: (dev lustre, type lustre) has no xattr support
[ 3352.253281] SELinux: (dev lustre, type lustre) falling back to genfs
[ 3353.617185] LNet: 1 local NIs in recovery (showing 1): 
192.168.200.47 at o2ib10
[ 3374.119376] Lustre: lustreFS-MDT0000: Will be in recovery for at 
least 2:30, or until 3 clients reconnect
[ 3387.880357] Lustre: lustreFS-MDT0001-osp-MDT0000: Connection restored 
to 192.168.200.48 at o2ib10 (at 192.168.200.48 at o2ib10)
[ 3388.450341] Lustre: lustreFS-MDT0000: Recovery over after 0:15, of 3 
clients 3 recovered and 0 were evicted.
[ 3388.482527] BUG: unable to handle page fault for address: 
ffd317785f002008
[ 3388.482533] #PF: supervisor read access in kernel mode
[ 3388.482535] #PF: error_code(0x0000) - not-present page
[ 3388.482536] PGD 0
[ 3388.482541] Oops: 0000 [#1] PREEMPT SMP NOPTI
[ 3388.482546] CPU: 9 PID: 23517 Comm: qsd_reint_1.lus Kdump: loaded 
Tainted: G           OE      ------  --- 
5.14.0-611.13.1_lustre.el9.x86_64 #1
[ 3388.482551] Hardware name: Red Hat KVM/RHEL-AV, BIOS 
1.16.0-4.module+el8.9.0+90052+d3bf71d8 04/01/2014
[ 3388.482552] RIP: 0010:kfree+0x6e/0x330
[ 3388.482585] Code: 80 48 01 d8 0f 82 d5 02 00 00 48 c7 c2 00 00 00 80 
48 2b 15 fc eb 39 01 48 01 d0 48 c1 e8 0c 48 c1 e0 06 48 03 05 da eb 39 
01 <48> 8b 50 08 49 89 c4 f6 c2 01 0f 85 d8 01 00 00 0f 1f 44 00 00 49
[ 3388.482586] RSP: 0018:ff40544f8d71fdd0 EFLAGS: 00010286
[ 3388.482591] RAX: ffd317785f002000 RBX: 0000000000080003 RCX: 
0057088740207bf8
[ 3388.482592] RDX: 00e4b45740000000 RSI: ffffffffc19dacae RDI: 
0000000000080003
[ 3388.482593] RBP: ff40544f8d71fe18 R08: 0000000000000000 R09: 
0000000000000008
[ 3388.482594] R10: 0000000000000002 R11: 0000000000000001 R12: 
ffffffffc1a3d660
[ 3388.482595] R13: ff1b4ba99ea17200 R14: ff1b4ba94be68c00 R15: 
ff1b4ba99ea17230
[ 3388.482596] FS:  0000000000000000(0000) GS:ff1b4bc83fa40000(0000) 
knlGS:0000000000000000
[ 3388.482598] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 3388.482599] CR2: ffd317785f002008 CR3: 00000001269e0003 CR4: 
0000000000773ef0
[ 3388.482603] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 
0000000000000000
[ 3388.482604] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 
0000000000000400
[ 3388.482605] PKRU: 55555554
[ 3388.482606] Call Trace:
[ 3388.482607]  <TASK>
[ 3388.482609]  ? show_trace_log_lvl+0x1c4/0x2df
[ 3388.482635]  ? show_trace_log_lvl+0x1c4/0x2df
[ 3388.482644]  ? key_fini+0x4e/0x160 [obdclass]
[ 3388.482983]  ? __die_body.cold+0x8/0xd
[ 3388.482987]  ? page_fault_oops+0x132/0x170
[ 3388.483003]  ? kernelmode_fixup_or_oops+0x83/0x110
[ 3388.483005]  ? exc_page_fault+0xa7/0x150
[ 3388.483015]  ? asm_exc_page_fault+0x22/0x30
[ 3388.483045]  ? key_fini+0x4e/0x160 [obdclass]
[ 3388.483109]  ? kfree+0x6e/0x330
[ 3388.483112]  key_fini+0x4e/0x160 [obdclass]
[ 3388.483196]  lu_context_fini+0x42/0x160 [obdclass]
[ 3388.483265]  lu_env_fini+0x16/0x30 [obdclass]
[ 3388.483340]  qsd_reint_main+0x2ec/0xb70 [lquota]
[ 3388.483427]  ? __schedule+0x239/0x540
[ 3388.483433]  ? _raw_spin_unlock_irqrestore+0xa/0x30
[ 3388.483439]  ? __pfx_qsd_reint_main+0x10/0x10 [lquota]
[ 3388.483458]  ? kthread+0x101/0x110
[ 3388.483468]  kthread+0x101/0x110
[ 3388.483470]  ? __pfx_kthread+0x10/0x10
[ 3388.483473]  ret_from_fork+0x28/0x50
[ 3388.483486]  </TASK>
[ 3388.483490] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) 
lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) mbcache 
jbd2 lustre(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) 
ptlrpc(OE) obdclass(OE) ko2iblnd(OE) lnet(OE) libcfs(OE) rfkill 
nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet 
nf_reject_ipv4 nf_reject_ipv6 nft_reject nft_ct nft_chain_nat nf_nat 
nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 nf_tables nfnetlink 
intel_rapl_msr intel_rapl_common intel_uncore_frequency_common nfit 
libnvdimm snd_hda_codec_generic snd_hda_intel snd_intel_dspcfg 
snd_intel_sdw_acpi kvm_intel snd_hda_codec snd_hda_core snd_hwdep kvm 
snd_seq snd_seq_device qxl drm_ttm_helper iTCO_wdt snd_pcm ttm 
iTCO_vendor_support rapl snd_timer drm_exec drm_client_lib i2c_i801 
drm_kms_helper snd mlx5_fwctl i2c_smbus lpc_ich virtio_balloon pcspkr 
soundcore fwctl joydev drm fuse xfs libcrc32c mlx5_ib macsec mlx5_core 
sd_mod ahci sg libahci libata crct10dif_pclmul crc32_pclmul crc32c_intel 
mlxfw psample virtio_net tls
[ 3388.483681]  virtio_console ghash_clmulni_intel virtio_scsi(E) 
net_failover pci_hyperv_intf failover serio_raw dm_mirror dm_region_hash 
dm_log dm_mod rpcrdma sunrpc rdma_ucm ib_uverbs ib_srpt ib_isert 
iscsi_target_mod target_core_mod ib_iser libiscsi scsi_transport_iscsi 
ib_umad rdma_cm ib_ipoib iw_cm ib_cm ib_core
[ 3388.483758] CR2: ffd317785f002008

===================

We have tried to mount the other mds servers (which e2fsck found nothing 
to fix) and those servers were not crashed. The mdt1 and mgt share the 
same server in our setting.

Thank for any help that you can provide

Best regards,

NGUYEN Hoang Minh

HPC Researcher



More information about the lustre-discuss mailing list