[lustre-discuss] OST partition sizes
scott.nolin at ssec.wisc.edu
Wed Apr 29 07:38:12 PDT 2015
Just to chime in on our 8+2 vs 10+2 tests, since I ran those.
I tested ldiskfs with hardware raid6 and raidz2. Hardware vs software
raid I think made no noticeable difference.
I used IOR, singlefile, 100MB files. That's the most important workload
for us. I tried several different file sizes, but 100MB seemed a
reasonable compromise for what I see the most. We rarely or never do
I remember I did see a difference between 10+2 and 8+2. Especially at
smaller numbers of clients and threads, the 8+2 performance numbers were
more consistent, made a smoother curve. 10+2 with not a lot of threads
the performance was more variable.
Still, it wasn't a big enough difference for *our* needs to make it a
"never do 10 data + 2 parity". We get in situations where we have
12-disk arrays sometimes and don't want to waste the capacity.
Definitely shows 8+2 is more ideal though, and a good general
At higher numbers of clients and threads, the differences pretty much
disappeared for total performance. That kind of makes sense to me.
On 4/29/2015 8:14 AM, Andrew Wagner wrote:
> We've continued using the 128KB stripes to the disks. The performance
> dropoff going from 8+2 128KB to 10+2 128KB was measurable but not a
> game-changer for us, particularly as the filesystems that are using 10+2
> are mixed-use rather than HPC scratch.
> I'm not sure if we tested using larger stripe sizes on the 10+2 arrays
> to see if performance got any better. Do you have any test results on
> 10+2 128KB vs 10+2 256KB?
> Andrew Wagner
> Research Systems Administrator
> Space Science and Engineering
> University of Wisconsin
> andrew.wagner at ssec.wisc.edu | 608-261-1360
> On 04/28/2015 08:59 PM, Nguyen Viet Cuong wrote:
>> We normally use 8+2 in RAID6 with 128KB chunk.
>> Andrew, what is stripe size in case of your 10+2 config? Sometime I
>> use 256KB to make it 2.5MB stripe because I think 1.25MB is unreasonable.
>> On Wed, Apr 29, 2015 at 5:35 AM, Andrew Wagner
>> <andrew.wagner at ssec.wisc.edu <mailto:andrew.wagner at ssec.wisc.edu>> wrote:
>> We try to do 8+2 where it's convenient, but when we end up with 12
>> disk raid arrays in some places we do 10+2.
>> Andrew Wagner
>> Research Systems Administrator
>> Space Science and Engineering
>> University of Wisconsin
>> andrew.wagner at ssec.wisc.edu <mailto:andrew.wagner at ssec.wisc.edu> | 608-261-1360
>> On 04/28/2015 03:01 PM, Andrus, Brian Contractor wrote:
>>> Quick question/survey:
>>> What is the partition size folks use for their OSTs and why?
>>> Brian Andrus
>>> ITACS/Research Computing
>>> Naval Postgraduate School
>>> Monterey, California
>>> voice: 831-656-6238
>>> lustre-discuss mailing list
>>> lustre-discuss at lists.lustre.org <mailto:lustre-discuss at lists.lustre.org>
>> lustre-discuss mailing list
>> lustre-discuss at lists.lustre.org
>> <mailto:lustre-discuss at lists.lustre.org>
>> Nguyen Viet Cuong
> lustre-discuss mailing list
> lustre-discuss at lists.lustre.org
-------------- next part --------------
A non-text attachment was scrubbed...
Size: 6248 bytes
Desc: S/MIME Cryptographic Signature
More information about the lustre-discuss