[Lustre-discuss] large ost and fsck

[Andreas Dilger]

On Oct 20, 2008  13:16 -0700, Steden Klaus wrote:
> fsck'ing a Lustre volume doesn't take any more or less time than
> fsck'ing a traditional ext2/ext3 volume.

That isn't quite true.  Lustre uses extents in ext3 and also the
"uninit_groups" feature (both merged into the upstream kernel in ext4)
which can significantly reduce e2fsck times because much less metadata
is read from the disk (which can be slow and seeky).

> I've had to run fsck a few times over the years on 2 TB volumes on a
> DDN SAN, and depending on how much needs correcting, it usually takes
> about 15-20 minutes from start to finish.

In the past it used to take about 1h to run e2fsck for 1TB, but this
can be down as low as 5 minutes with Lustre filesystems today, especially
if the uninit_groups feature is enabled.

> The only real constraint with OST size is the Linux max file system size
> (2 TB if memory serves). I don't know if there's a performance benefit
> or penalty if you have multiple, smaller OSTs ... likely Andreas will
> be able to shed some light.

The current maximum OST size is 8TB.  We are testing with 16TB with RHEL5
kernels, though testing isn't finished yet, and also working to back-port
fixes to SLES10 to also allow 16TB OSTs.

> -----Original Message-----
> From: lustre-discuss-bounces at lists.lustre.org on behalf of Mag Gam
> Sent: Sun 10/19/2008 6:29 AM
> To: Lustre discuss
> Subject: [Lustre-discuss] large ost and fsck
>  
> We typically debate weather to create a large OST or smaller OSTs
> (100GB). We prefer large OSTs (1 TB) because the ease of management
> but how would fsck work? Would it take a long time? Also, if a large
> OST is preferred is it possible to consolidate a smaller OSTs into a
> larger one?

Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.