[lustre-discuss] zfs -- mds/mdt -- ssd model / type recommendation

[Andrew Wagner]

I can offer some guidance on our experiences with ZFS Lustre MDTs. 
Patrick and Charlie are right - you will get less performance per $ out 
of ZFS MDTs vs. LDISKFS MDTs. That said, our RAID10 with 4x Dell Mixed 
Use Enterprise SSDs achieves similar performance to most of our LDISKFS 
MDTs. Our MDS was a Dell server and we wanted complete support coverage.

One of the most important things for good performance with our ZFS MDS 
was RAM. We doubled the amount of RAM in the system after experiencing 
performance issues that were clearly memory pressure related. If you 
expect to have tens of millions of files, I wouldn't run the MDS without 
at least 128GB of RAM. I would be prepared to increase that number if 
you run into RAM bottlenecks - we ended up going to 256GB in the end.

For a single OSS, you may not need 4x SSDs to deal with the load. We use 
the 4 disk RAID10 setup with a 1PB filesystem and 1.8PB filesystem. Our 
use case was more for archive purposes, so we wanted to go with a 
complete ZFS solution.



On 5/4/2015 1:18 PM, Kevin Abbey wrote:
> Hi,
>
>  For a single node OSS I'm planning to use a combined MGS/MDS. Can 
> anyone recommend an enterprise ssd designed for this workload?  I'd 
> like to create a raid10  with 4x ssd using zfs as the backing fs.
>
> Are there any published/documented systems using zfs in raid 10 using 
> ssd?
>
> Thanks,
> Kevin
>
>