[Lustre-discuss] Corrupted group descriptors / missing extent flag in backup superblocks

[rf at q-leap.de]

Hi,

due to a hardware failure, we have 3 corrupted OSTs that need to be
repaired (Lustre 1.6.7.2, kernel 2.6.22.x). Using the latest e2fsck from
oracle (1.41.12.2.ora3), we run into trouble, since we were mounting the
OSTs with the extent flag, but this information is not found in the
backup superblocks. Hence, we get:

----------------------------------
$ /apps/prod/e2fsprogs/1.41.12.2.ora3/sbin/e2fsck -n -f  /dev/md3
/apps/prod/e2fsprogs/1.41.12.2.ora3/sbin/e2fsck: Group descriptors look bad...
 trying backup blocks...
Superblock needs_recovery flag is clear, but journal has data.
Recovery flag not set in backup superblock, so running journal anyway.
Clear journal? no

Pass 1: Checking inodes, blocks, and sizes
Inode 24577 is in extent format, but superblock is missing EXTENTS feature
Fix? no

Inode 24577 has EXTENTS_FL flag set on filesystem without extents support.
Clear? no

 .....

----------------------------------

i.e. e2fsck wants to clear the EXTENTS_FL flag in every single inode
found. So the question is: Is there a way to manipulate one of the
backup superblocks to add the extent flag, or is it possible to copy the
group descriptors from a backup superblock to superblock 0?
I've appended dumps of the two superblocks 0 and 32768.

Thanks,

Roland

------------------------------------------------------------------------

pfs1n3:~# /apps/prod/e2fsprogs/1.41.12.2.ora3/sbin/dumpe2fs -h -o superblock=0 /dev/md3
dumpe2fs 1.41.12.2.ora3 (23-Feb-2011)
Filesystem volume name:   pfs1data-OST0000
Last mounted on:          <not available>
Filesystem UUID:          a18ac990-678e-4142-9332-3d200807a7cf
Filesystem magic number:  0xEF53
Filesystem revision #:    1 (dynamic)
Filesystem features:      has_journal ext_attr resize_inode dir_index filetype extent sparse_super large_file
Filesystem flags:         signed_directory_hash 
Default mount options:    (none)
Filesystem state:         clean with errors
Errors behavior:          Continue
Filesystem OS type:       Linux
Inode count:              427180032
Block count:              1708717312
Reserved block count:     85435865
Free blocks:              497243599
Free inodes:              430994114
First block:              0
Block size:               4096
Fragment size:            4096
Reserved GDT blocks:      616
Blocks per group:         32768
Fragments per group:      32768
Inodes per group:         8192
Inode blocks per group:   512
RAID stride:              64
Filesystem created:       Thu Mar 27 17:40:52 2008
Last mount time:          Mon Sep 19 12:55:34 2011
Last write time:          Tue Sep 20 14:54:25 2011
Mount count:              46
Maximum mount count:      39
Last checked:             Thu Mar 27 17:40:52 2008
Check interval:           15552000 (6 months)
Next check after:         Tue Sep 23 18:40:52 2008
Reserved blocks uid:      0 (user root)
Reserved blocks gid:      0 (group root)
First inode:              11
Inode size:               256
Journal inode:            8
Default directory hash:   tea
Directory Hash Seed:      d39dbbc8-1d4e-4fdd-af5a-86fcbbf333e0
Journal backup:           inode blocks
Journal features:         journal_incompat_revoke
Journal size:             400M
Journal length:           102400
Journal sequence:         0x196cd453
Journal start:            0

------------------------------------------------------------------------

pfs1n3:~# /apps/prod/e2fsprogs/1.41.12.2.ora3/sbin/dumpe2fs -h -o superblock=32768 /dev/md3
dumpe2fs 1.41.12.2.ora3 (23-Feb-2011)
Filesystem volume name:   pfs1data-OSTffff
Last mounted on:          <not available>
Filesystem UUID:          a18ac990-678e-4142-9332-3d200807a7cf
Filesystem magic number:  0xEF53
Filesystem revision #:    1 (dynamic)
Filesystem features:      has_journal resize_inode dir_index filetype sparse_super large_file
Filesystem flags:         signed_directory_hash
Default mount options:    (none)
Filesystem state:         not clean
Errors behavior:          Continue
Filesystem OS type:       Linux
Inode count:              427180032
Block count:              1708717312
Reserved block count:     85435865
Free blocks:              1681789210
Free inodes:              427180021
First block:              0
Block size:               4096
Fragment size:            4096
Reserved GDT blocks:      616
Blocks per group:         32768
Fragments per group:      32768
Inodes per group:         8192
Inode blocks per group:   512
RAID stride:              64
Filesystem created:       Thu Mar 27 17:40:52 2008
Last mount time:          n/a
Last write time:          Thu Mar 27 18:54:23 2008
Mount count:              0
Maximum mount count:      39
Last checked:             Thu Mar 27 17:40:52 2008
Check interval:           15552000 (6 months)
Next check after:         Tue Sep 23 18:40:52 2008
Reserved blocks uid:      0 (user root)
Reserved blocks gid:      0 (group root)
First inode:              11
Inode size:               256
Journal inode:            8
Default directory hash:   tea
Directory Hash Seed:      d39dbbc8-1d4e-4fdd-af5a-86fcbbf333e0
Journal backup:           inode blocks
Journal features:         journal_incompat_revoke
Journal size:             400M
Journal length:           102400
Journal sequence:         0x196cd453
Journal start:            0