[Lustre-discuss] ldiskfs for MDT and zfs for OSTs?

[Anjana Kar]

On 10/07/2013 01:48 PM, Ned Bass wrote:
> On Mon, Oct 07, 2013 at 06:09:21AM -0400, Anjana Kar wrote:
>> Is it possible to configure the MDT as ldiskfs and the OSTs with zfs
>> in lustre 2.4? The server is running a lustre kernel on a Centos 6.4
>> system, has both lustre-osd-ldiskfs and lustre-osd-zfs rpms installed.
>> The MDT is up as ldiskfs, but get an error trying to configure the ost:
>>
>> mkfs.lustre --fsname=lustrefs --reformat --ost --backfstype=zfs .....
>>
>> mkfs.lustre FATAL: unable to prepare backend (22)
>> mkfs.lustre: exiting with 22 (Invalid argument)
> I suspect that the mkfs.lustre error is unrelated to the presence of an
> ldiskfs MDT.  We need to see the complete command line parameters and
> output to help you diagnose that problem.  It's often helpful to try
> creating the pool and dataset "by hand" with zpool create and zfs
> create, using the same names and devices.  It may reveal the problem
> with an error message that is masked by mkfs.lustre.  Also check dmesg
> to see if there was an error loading the ZFS module stack.
>
> Having said all that, as Andreas said, you may run into other problems
> running the ZFS and ldiskfs OSDs concurrently on the same node.  Using
> separate MDS and OSS nodes is more likely to work, but this
> configuration has not seen much testing.  So please report on your
> experience if you go this route.
>
> Thanks,
> Ned

Here is the exact command used to create a raidz2 pool with 8+2 drives,
followed by the error messages:

mkfs.lustre --fsname=cajalfs --reformat --ost --backfstype=zfs --index=0 
--mgsnode=10.10.101.171 at o2ib lustre-ost0/ost0 raidz2 /dev/sda /dev/sdc 
/dev/sde /dev/sdg /dev/sdi /dev/sdk /dev/sdm /dev/sdo /dev/sdq /dev/sds

mkfs.lustre FATAL: Invalid filesystem name /dev/sds

mkfs.lustre FATAL: unable to prepare backend (22)
mkfs.lustre: exiting with 22 (Invalid argument)

dmesg shows
ZFS: Loaded module v0.6.2-1, ZFS pool version 5000, ZFS filesystem version 5

Any suggestions on creating the pool separately?

Thanks,
-Anjana