[lustre-discuss] OST partition sizes

[Scott Nolin]

Just to chime in on our 8+2 vs 10+2 tests, since I ran those.

I tested ldiskfs with hardware raid6 and raidz2. Hardware vs software 
raid I think made no noticeable difference.

I used IOR, singlefile, 100MB files. That's the most important workload 
for us. I tried several different file sizes, but 100MB seemed a 
reasonable compromise for what I see the most. We rarely or never do 
file striping.

I remember I did see a difference between 10+2 and 8+2. Especially at 
smaller numbers of clients and threads, the 8+2 performance numbers were 
more consistent, made a smoother curve. 10+2 with not a lot of threads 
the performance was more variable.

Still, it wasn't a big enough difference for *our* needs to make it a 
"never do 10 data + 2 parity". We get in situations where we have 
12-disk arrays sometimes and don't want to waste the capacity.

Definitely shows 8+2 is more ideal though, and a good general 
recommendation.

At higher numbers of clients and threads, the differences pretty much 
disappeared for total performance. That kind of makes sense to me.

Scott

On 4/29/2015 8:14 AM, Andrew Wagner wrote:
> We've continued using the 128KB stripes to the disks. The performance
> dropoff going from 8+2 128KB to 10+2 128KB was measurable but not a
> game-changer for us, particularly as the filesystems that are using 10+2
> are mixed-use rather than HPC scratch.
>
> I'm not sure if we tested using larger stripe sizes on the 10+2 arrays
> to see if performance got any better. Do you have any test results on
> 10+2 128KB vs 10+2 256KB?
>
> Andrew Wagner
> Research Systems Administrator
> Space Science and Engineering
> University of Wisconsin
> andrew.wagner at ssec.wisc.edu  | 608-261-1360
>
> On 04/28/2015 08:59 PM, Nguyen Viet Cuong wrote:
>> We normally use 8+2 in RAID6 with 128KB chunk.
>>
>> Andrew, what is stripe size in case of your 10+2 config? Sometime I
>> use 256KB to make it 2.5MB stripe because I think 1.25MB is unreasonable.
>>
>> On Wed, Apr 29, 2015 at 5:35 AM, Andrew Wagner
>> <andrew.wagner at ssec.wisc.edu <mailto:andrew.wagner at ssec.wisc.edu>> wrote:
>>
>>     We try to do 8+2 where it's convenient, but when we end up with 12
>>     disk raid arrays in some places we do 10+2.
>>
>>     Andrew Wagner
>>     Research Systems Administrator
>>     Space Science and Engineering
>>     University of Wisconsin
>>     andrew.wagner at ssec.wisc.edu  <mailto:andrew.wagner at ssec.wisc.edu>  | 608-261-1360
>>
>>     On 04/28/2015 03:01 PM, Andrus, Brian Contractor wrote:
>>>
>>>     Quick question/survey:
>>>
>>>     What is the partition size folks use for their OSTs and why?
>>>
>>>     Brian Andrus
>>>
>>>     ITACS/Research Computing
>>>
>>>     Naval Postgraduate School
>>>
>>>     Monterey, California
>>>
>>>     voice: 831-656-6238
>>>
>>>
>>>
>>>     _______________________________________________
>>>     lustre-discuss mailing list
>>>     lustre-discuss at lists.lustre.org  <mailto:lustre-discuss at lists.lustre.org>
>>>     http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>>
>>
>>     _______________________________________________
>>     lustre-discuss mailing list
>>     lustre-discuss at lists.lustre.org
>>     <mailto:lustre-discuss at lists.lustre.org>
>>     http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>>
>>
>>
>>
>> --
>> Nguyen Viet Cuong
>
>
>
> _______________________________________________
> lustre-discuss mailing list
> lustre-discuss at lists.lustre.org
> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>


-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 6248 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20150429/235398bc/attachment.bin>