[lustre-discuss] Experience with PCIe INTEL SSD

[Dilger, Andreas]

On Jun 6, 2017, at 04:34, Arman Khalatyan <arm2arm at gmail.com> wrote:
> 
> Hello everybody,
> Can some one share his experience of using NVMe Intel P3600 as a MDS with ldiskfs?
> We have two of them, is it better to use them with lvm mirror or mdraid?
> We tested with zfs mirror but somehow it is not performing as fast as we expected.

We've done a bunch of testing with the P3600 and P3700 and have seen good performance (mostly as OSTs, but also as MDTs, see https://www.eofs.eu/_media/events/lad16/17_dne_analysis_roe_2_.pdf).  It is likely that the ZFS metadata performance is limited by ZFS itself as is shown in the above tests, though that will be improved significantly with ZFS 0.7.0 and Lustre 2.9/2.10.

If you are just in the process of testing the hardware before deployment, you should try out ZFS 0.7.0-rc4 and Lustre master branch (pre-2.10), even if you don't necessarily plan on going into production with those versions initially, just to see where the performance will be in the near future.

It should be noted that since the P3600 is an internal PCI card that you cannot do MDT failover to a backup MDS in this case, only "reboot in place".

Cheers, Andreas
--
Andreas Dilger
Lustre Principal Architect
Intel Corporation