[lustre-discuss] [EXTERNAL] Re: Tuning for metadata performance

Mon Jan 11 10:45:44 PST 2021

Sure.  Its a custom configuration on commodity hardware, which is quite a bit newer than the luster servers.  The overall setup is a bit complicated to support HA - two servers with an external JBOD with ZFS to manage the drives and the file system.  PCS to do the failover.  But none of that is too relevant in terms of performance so here are the hardware specs.  

Servers:
192 GB DDR4 2666 MHz ECC Memory
16 total physical cores (2x Intel Xeon Gold 6144 CPU @ 3.50GHz) 
LSI SAS Card (can't find exact model but very similar to the cards in the lustre servers)

JBOD:
Supermicro 3.5"
24x 10TB 7200 RPM Seagate HDD's

ZFS is used to configure the drives in a RAID10 with a zfs file system built on the zpool.  This is exported via NFS.  The only NFS tuning we are doing is to increase RPCNFSDCOUNT to 128 and export with async.  

So the HW configuration is overall fairly similar.   This is another reason I'm hopeful that we'd be able to get our lustre MD performance as good or better than the NFS server given that the lustre MDS has SSD's and the NFS server has HDD's.  

-----Original Message-----
From: lustre-discuss <lustre-discuss-bounces at lists.lustre.org> on behalf of Michael Di Domenico <mdidomenico4 at gmail.com>
Date: Monday, January 11, 2021 at 8:07 AM
Cc: "lustre-discuss at lists.lustre.org" <lustre-discuss at lists.lustre.org>
Subject: [EXTERNAL] Re: [lustre-discuss] Tuning for metadata performance

perhaps i missed it somewhere, but in order to do a fair comparison
can you detail the hardware/software behind the nfs server?