[Lustre-discuss] Disappearing OSTs

[Andreas Dilger]

On May 05, 2008  11:57 -0400, jrs wrote:
> mds01:/net/lmd01/space/lustre # mount -t lustre /dev/mapper/mdt_mds01_lustre0102 /mnt/lustre/mdt
> mount.lustre: mount /dev/mapper/mdt_mds01_lustre0102 at /mnt/lustre/mdt failed: Invalid argument
> This may have multiple causes.
> Are the mount options correct?
> Check the syslog for more info.
>
> Which produces this in /var/log/message
>
> May  5 09:35:41 mds01 kernel: VFS: Can't find ldiskfs filesystem on dev dm-1.
> May  5 09:35:41 mds01 multipathd: dm-1: umount map (uevent)
> May  5 09:35:41 mds01 kernel: LustreError: 
> 16215:0:(obd_mount.c:1229:server_kernel_mount()) premount 
> /dev/mapper/mdt_mds01_lustre0102:0x0 ldiskfs failed: -22, ldiskfs2 failed: 
> -19.  Is the ldiskfs module available?
> May  5 09:35:41 mds01 kernel: LustreError: 16215:0:(obd_mount.c:1533:server_fill_super()) Unable to mount device /dev/mapper/mdt_mds01_lustre0102: -22
> May  5 09:35:41 mds01 kernel: LustreError: 16215:0:(obd_mount.c:1924:lustre_fill_super()) Unable to mount  (-22)
>
> If I try to look at the partition table with parted I see:
>
> mds01:/net/oss02/space/parted-1.8.8 # /usr/local/sbin/parted /dev/mapper/mdt_mds01_lustre0102
> GNU Parted 1.8.8
> Using /dev/mapper/mdt_mds01_lustre0102
> Welcome to GNU Parted! Type 'help' to view a list of commands.
> (parted) p
> Error: /dev/mapper/mdt_mds01_lustre0102: unrecognised disk label
> (parted)
>
> A good filesystem looks like:
> mds01:/net/oss02/space/parted-1.8.8 # /usr/local/sbin/parted /dev/mapper/ost_oss01_lustre0304_01
> GNU Parted 1.8.8
> Using /dev/mapper/ost_oss01_lustre0304_01
> Welcome to GNU Parted! Type 'help' to view a list of commands.
> (parted) p
> Model: Unknown (unknown)
> Disk /dev/mapper/ost_oss01_lustre0304_01: 6001GB
> Sector size (logical/physical): 512B/512B
> Partition Table: loop
>
> Number  Start  End     Size    File system  Flags
>  1      0.00B  6001GB  6001GB  ext3
>
>
> NOTE: in another post someone commented on the loop partition type.
> I don't know what it is but all my lustre partitions are of that
> type.  The fact that a lustre person (I believe this individual was
> employed by Sun) was unfamiliar with it certainly is surprising.

I don't think that being employed by Sun makes everyone suddenly know
and understand everything :-).  That other person was me, and while
I've even contributed a significant amount of code to parted in the
past, I just haven't used it in several years and am not familiar with
the "loop" partition type.

> Perhaps my version of parted has an issue (the one shipped with SLES
> returns:
> mds01:/net/oss02/space/parted-1.8.8 # parted /dev/mapper/mdt_mds01_lustre0102
> Floating point exception

Two things of note:
- there have been ongoing issues with parted and ldiskfs with large disk
  devices, and I tend to avoid parted and fdisk entirely for these reasons.
  I've been using LVM (DM) to manage my storage for some time now, if it is
  needed.
- we generally do NOT recommend using partitions of any kind for production
  Lustre filesystems, because of problems like this, and the fact that in
  RAID setups this can hurt performance due to misaligned IO to the disk.

Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.