[lustre-discuss] Lustre Server Sizing

[Patrick Farrell]

Note the other email also seemed to suggest that multiple NFS exports of Lustre wouldn't work.  I don't think that's the case, as we have this sort of setup at a number of our customers without particular trouble.  In the abstract, I could see the possibility of some caching errors between different clients, but that would be only namespace stuff, not data.  And I think in practice that's ok.

But regardless, as Andreas said, for the Linux clients, Lustre directly will give much better results.
________________________________________
From: lustre-discuss [lustre-discuss-bounces at lists.lustre.org] on behalf of Dilger, Andreas [andreas.dilger at intel.com]
Sent: Tuesday, July 21, 2015 6:59 PM
To: Indivar Nair
Cc: hpdd-discuss; to: lustre-discuss
Subject: Re: [lustre-discuss] Lustre Server Sizing

Having only 3 OSS will limit the performance you can get, and having so many OSTs on each OSS will give sub-optimal performance. 4-6 OSTs/OSS is more reasonable.

It also isn't clear why you want RAID-60 instead of just RAID-10?

Finally, for Linux clients it is much better to use direct Lustre access instead of NFS as mentioned in another email.

Cheers, Andreas

On Jul 21, 2015, at 08:58, Indivar Nair <indivar.nair at techterra.in<mailto:indivar.nair at techterra.in>> wrote:

Hi ...,

One of our customers has a 3 x 240 Disk SAN Storage Array and would like to convert it to Lustre.

They have around 150 Workstations and around 200 Compute (Render) nodes.
The File Sizes they generally work with are -
1 to 1.5 million files (images) of 10-20MB in size.
And a few thousand files of 500-1000MB in size.

Almost 50% of the infra is on MS Windows or Apple MACs

I was thinking of the following configuration -
1 MDS
1 Failover MDS
3 OSS (failover to each other)
3 NFS+CIFS Gateway Servers
FDR Infiniband backend network (to connect the Gateways to Lustre)
Each Gateway Server will have 8 x 10GbE Frontend Network (connecting the clients)

Option A
    10+10 Disk RAID60 Array with 64KB Chunk Size i.e. 1MB Stripe Width
    720 Disks / (10+10) = 36 Arrays.
    12 OSTs per OSS
    18 OSTs per OSS in case of Failover

Option B
    10+10+10+10 Disk RAID60 Array with 128KB Chunk Size i.e. 4MB Stripe Width
    720 Disks / (10+10+10+10) = 18 Arrays
    6 OSTs per OSS
    9 OSTs per OSS in case of Failover
    4MB RPC and I/O

Questions
1. Would it be better to let Lustre do most of the striping / file distribution (as in Option A) OR would it be better to let the RAID Controllers do it (as in Option B)

2. Will Option B allow us to have lesser CPU/RAM than Option A?

Regards,


Indivar Nair

_______________________________________________
lustre-discuss mailing list
lustre-discuss at lists.lustre.org<mailto:lustre-discuss at lists.lustre.org>
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
_______________________________________________
lustre-discuss mailing list
lustre-discuss at lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org