[lustre-discuss] Lustre Server Sizing

Dilger, Andreas andreas.dilger at intel.com
Tue Jul 21 16:59:29 PDT 2015

Having only 3 OSS will limit the performance you can get, and having so many OSTs on each OSS will give sub-optimal performance. 4-6 OSTs/OSS is more reasonable.

It also isn't clear why you want RAID-60 instead of just RAID-10?

Finally, for Linux clients it is much better to use direct Lustre access instead of NFS as mentioned in another email.

Cheers, Andreas

On Jul 21, 2015, at 08:58, Indivar Nair <indivar.nair at techterra.in<mailto:indivar.nair at techterra.in>> wrote:

Hi ...,

One of our customers has a 3 x 240 Disk SAN Storage Array and would like to convert it to Lustre.

They have around 150 Workstations and around 200 Compute (Render) nodes.
The File Sizes they generally work with are -
1 to 1.5 million files (images) of 10-20MB in size.
And a few thousand files of 500-1000MB in size.

Almost 50% of the infra is on MS Windows or Apple MACs

I was thinking of the following configuration -
1 Failover MDS
3 OSS (failover to each other)
3 NFS+CIFS Gateway Servers
FDR Infiniband backend network (to connect the Gateways to Lustre)
Each Gateway Server will have 8 x 10GbE Frontend Network (connecting the clients)

Option A
    10+10 Disk RAID60 Array with 64KB Chunk Size i.e. 1MB Stripe Width
    720 Disks / (10+10) = 36 Arrays.
    12 OSTs per OSS
    18 OSTs per OSS in case of Failover

Option B
    10+10+10+10 Disk RAID60 Array with 128KB Chunk Size i.e. 4MB Stripe Width
    720 Disks / (10+10+10+10) = 18 Arrays
    6 OSTs per OSS
    9 OSTs per OSS in case of Failover
    4MB RPC and I/O

1. Would it be better to let Lustre do most of the striping / file distribution (as in Option A) OR would it be better to let the RAID Controllers do it (as in Option B)

2. Will Option B allow us to have lesser CPU/RAM than Option A?


Indivar Nair

lustre-discuss mailing list
lustre-discuss at lists.lustre.org<mailto:lustre-discuss at lists.lustre.org>

More information about the lustre-discuss mailing list