[Lustre-discuss] mkfs.lustre fails, ldiskfs: ext4 or ext3 ?

Andreas Dilger andreas.dilger at oracle.com
Wed Nov 3 13:15:54 PDT 2010


On 2010-11-03, at 13:25, Thomas Roth wrote:
> my attempt to format a new OST failed (mkfs.lustre: Unable to mount
> /dev/sdd: Invalid argument), obviously because sdd has "device size =
> 15253504MB", and the log tells me 'LDISKFS-fs does not support
> filesystems greater than 8TB and can cause data corruption.'
> 
> However, this is a Lustre 1.8.4 installation, I have compiled the kernel
> and modules myself and I have checked the 'modules/lustre/ldiskfs -
> config.log' to start with the following:
> 
> It was created by Lustre ldiskfs configure 3.1.3, which was
> generated by GNU Autoconf 2.61.  Invocation command line was
> 
>  $ ./configure --prefix=/usr --with-linux=/usr/src/linux-2.6.27.39-0.3
> --disable-quilt --disable-dependency-tracking --disable-doc
> --disable-utils --disable-liblustre --enable-snmp --enable-ldiskfs
> --disable-zerocopy --disable-tests --enable-quota CC=gcc-4.3
> --enable-ext4 --with-lustre-hack --with-sockets --cache-file=/dev/null
> --srcdir=.
> 
> All subsequent references to ext4 in this config.log seem to indicate
> that my ldiskfs should be based on ext4 and thus be able to handle a
> large device.

The ext4 that is included in 2.6.27 is (AFAIR) missing patches that would allow it to run with 16TB filesystems, or at least that was the case when we were testing 16TB filesystems.  As a result, we only allow 16TB filesystems with RHEL5.4 ext4-based ldiskfs, which was tested extensively.

> Is there a way to check ask the module itself about its capabilities?
> Or is there perhaps a magic switch for mkfs.lustre?

The code that limits the filesystem size to 8TB on untested platforms is in the ext4-force_over_8tb-sles11.patch patch.  Either you can test with the mount option "-O force_over_8tb", or rebuild the kernel without that patch.

There are tools included with Lustre called llverdev and llverfs that can be used to do partial or full data integrity testing of large filesystems (and the underlying block/SCSI/ATA drivers and h/w or s/w RAID).

Cheers, Andreas
--
Andreas Dilger
Lustre Technical Lead
Oracle Corporation Canada Inc.




More information about the lustre-discuss mailing list