[lustre-discuss] Quick ZFS pool question?

[Dilger, Andreas]

On Oct 13, 2016, at 10:32, E.S. Rosenberg <esr+lustre at mail.hebrew.edu> wrote:
> 
> On Fri, Oct 7, 2016 at 9:16 AM, Xiong, Jinshan <jinshan.xiong at intel.com> wrote:
> 
>> > On Oct 6, 2016, at 2:04 AM, Phill Harvey-Smith <p.harvey-smith at warwick.ac.uk> wrote:
>> >
>> > Having tested a simple setup for lustre / zfs, I'd like to try and
>> > replicate on the test system what we currently have on the production
>> > system, which uses a much older version of lustre (2.0 IIRC).
>> >
>> > Currently we have a combined mgs / mds node and a single oss node.
>> > we have 3 filesystems : home, storage and scratch.
>> >
>> > The MGS/MDS node currently has the mgs on a seperate block device and
>> > the 3 mds on a combined lvm volume.
>> >
>> > The OSS has an ost each (on a separate disks) for scratch and home
>> > and two ost for storage.
>> >
>> > If we migrate this setup to a ZFS based one, will I need to create a
>> > separate zpool for each mdt / mgt / oss  or will I be able to create
>> > a single zpool and split it up between the individual mdt / oss blocks,
>> > if so how do I tell each filesystem how big it should be?
>> 
>> We strongly recommend to create separate ZFS pools for OSTs, otherwise grant, which is a Lustre internal space reserve algorithm, won’t work properly.
>> 
>> It’s possible to create a single zpool for MDTs and MGS, and you can use ‘zfs set reservation=<space> <target>’ to reserve spaces for different targets.
> 
> I thought ZFS was only recommended for OSTs and not for MDTs/MGS?

The MGT/MDT can definitely be on ZFS.  The performance of ZFS has been
trailing behind that of ldiskfs, but we've made significant performance
improvements with Lustre 2.9 and ZFS 0.7.0.  Many people use ZFS for the
MDT backend because of the checksums and integrated JBOD management, as
well as the ability to create snapshots, data compression, etc.

Cheers, Andreas