[Lustre-discuss] RAID cards - what works well with Lustre?

[Daire Byrne]

Lior,

I am curious how you have the 16+2 drives connected to the LSI-9265. It has
eight internal ports so I'm guessing you are using some sort of SAS fan out
expander? We are thinking of using 3TB drives in a 36 bay enclosure/server
so 3 x RAID cards (8+2 RAID6 + 1 spare) seemed like a good fit. It also
means the resulting volumes are pretty much bang on the 24TB that Lustre
v1.8.6 now supports.

Daire

On Tue, Jul 19, 2011 at 7:19 AM, lior amar <liororama at gmail.com> wrote:

> Hi Daire,
>
> I am currently building a Lustre system using LSI-2965 (the successor of
> the LSI-2960 ) it has 1 GB of RAM and dual core processors.
> My experience with the card is quite good (Raid 6 performance of this card
> is very good).
>
> I am using it on my OSS's with the following configuration
>
> 18 1TB (7200) sas disks in a raid 6 (16 data + 2) with 256KB.
> 2 15K sas disks in raid 1 for the journal
>
> I know the Lustre manual recommends using a stripe unit that will give a
> total of 1MB.
> But the performance I got when using 64KB stripe units (16*64=1M) were
> lower then the case of 256KB.
>
> The max_sectors_kb is limited to 280 so I set it to 256
> And I tested varios I/O schedulers without getting any difference.
>
> Raw performance of the raid6 : ~960MB/sec write 1.5GB/sec Read
> Lustre performance using the obdfilter-survey (local mode) are also good. I
> will email you the results later (not in office now).
>
>
> Please let me know if there are any more details I can provide that might
> help you.
>
> Regards
>
> --lior
>
> ----------------------oo--o(:-:)o--oo----------------
> Lior Amar, Ph.D.
> Cluster Logic Ltd --> The Art of HPC
> www.clusterlogic.net
> ----------------------------------------------------------
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20110719/7b066d31/attachment.htm>