[Lustre-discuss] ldiskfs_valid_block_bitmap

[John White]

Hello Folks,
	We've been hitting a recurring issue that flips seemingly random OSTs into read-only mode.  This has happened numerous times over the last month or two and we can't correlate it to any back-end storage events.  We umount the OST, e2fsck and remount.  e2fsck *does* find bad blocks and an e2fsck does fix the issue on the specific LUN (unlike some bugs, ie: 23959).  Any ideas where we might start looking?

Sep 16 06:45:13 n0004.lustre LDISKFS-fs error (device dm-2): ldiskfs_valid_block_bitmap: 
Sep 16 06:45:13 n0004.lustre Invalid block bitmap - block_group = 51581, block = 1690206210
Sep 16 06:45:13 n0004.lustre kernel: LDISKFS-fs error (device dm-2): ldiskfs_valid_block_bitmap: Invalid block bitmap - block_group = 51581, block = 1690206210 
Sep 16 06:45:13 n0004.lustre  
Sep 16 06:45:13 n0004.lustre kernel: Remounting filesystem read-only 
Sep 16 06:45:13 n0004.lustre Remounting filesystem read-only 
Sep 16 06:45:13 n0004.lustre LustreError: 6564:0:(fsfilt-ldiskfs.c:1319:fsfilt_ldiskfs_write_record()) can't start transaction for 37 blocks (128 bytes) 
Sep 16 06:45:13 n0004.lustre LustreError: 6555:0:(fsfilt-ldiskfs.c:495:fsfilt_ldiskfs_brw_start()) can't get handle for 555 credits: rc = -30 
Sep 16 06:45:13 n0004.lustre LustreError: 6555:0:(filter_io_26.c:690:filter_commitrw_write()) error starting transaction: rc = -30


----------------
John White
HPC Systems Engineer
(510) 486-7307
One Cyclotron Rd, MS: 50C-3396
University of California Berkeley
Berkeley, CA 94720