[Lustre-discuss] Disappearing OSTs

[Bernd Schubert]

On Monday 05 May 2008 19:21:43 Andreas Dilger wrote:
> >
> > (parted) p
> > Model: Unknown (unknown)
> > Disk /dev/mapper/ost_oss01_lustre0304_01: 6001GB
> > Sector size (logical/physical): 512B/512B
> > Partition Table: loop
> >
> > Number  Start  End     Size    File system  Flags
> >  1      0.00B  6001GB  6001GB  ext3
> >
> >
> > NOTE: in another post someone commented on the loop partition type.
> > I don't know what it is but all my lustre partitions are of that
> > type.  The fact that a lustre person (I believe this individual was
> > employed by Sun) was unfamiliar with it certainly is surprising.
>
> I don't think that being employed by Sun makes everyone suddenly know
> and understand everything :-).  That other person was me, and while
> I've even contributed a significant amount of code to parted in the
> past, I just haven't used it in several years and am not familiar with
> the "loop" partition type.

You definitely know more about filesystems and partitions than I do, but I'm 
sure this is a bug.

>
> > Perhaps my version of parted has an issue (the one shipped with SLES
> > returns:
> > mds01:/net/oss02/space/parted-1.8.8 # parted
> > /dev/mapper/mdt_mds01_lustre0102 Floating point exception

Probably this:

http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=259248

>
> Two things of note:
> - there have been ongoing issues with parted and ldiskfs with large disk
>   devices, and I tend to avoid parted and fdisk entirely for these reasons.
>   I've been using LVM (DM) to manage my storage for some time now, if it is
>   needed.
> - we generally do NOT recommend using partitions of any kind for production
>   Lustre filesystems, because of problems like this, and the fact that in
>   RAID setups this can hurt performance due to misaligned IO to the disk.
>

Well, I wish we wouldn't need to use partitions, but for some projects we need 
to do so:

- ldiskfs is still limited to 8TiB

- linux-md raid6 is not parallized and a single CPU becomes a limit, while 7 
other CPUs are idling. Creating several raid sets is then some kind of 
parallization


Cheers,
Bernd

-- 
Bernd Schubert
Q-Leap Networks GmbH