[Lustre-discuss] One or two OSS, no difference?

[Jeffrey Bennett]

Andreas, if we are using 4kb blocks I understand we only transfer 1 page per RPC call, so are we limited to 10-15K RPC per second or what's the same, 10-15.000 IOPS?

jab


-----Original Message-----
From: Andreas.Dilger at sun.com [mailto:Andreas.Dilger at sun.com] On Behalf Of Andreas Dilger
Sent: Friday, March 05, 2010 2:05 AM
To: Jeffrey Bennett
Cc: Oleg.Drokin at Sun.COM; lustre-discuss at lists.lustre.org
Subject: Re: [Lustre-discuss] One or two OSS, no difference?

On 2010-03-04, at 14:18, Jeffrey Bennett wrote:
> I just noticed the sequential performance is ok, but the random IO  
> (which is what I am measuring) is not. Is there any way to increase  
> random IO performance on Lustre? We have LUNs that can provide  
> around 250.000 random read 4kb IOPS but we are only seeing 3.000 to  
> 10.000 on Lustre.

There is work currently underway to improve the SMP scaling  
performance for the RPC handling layer in Lustre.  Currently that  
limits the delivered RPC rate to 10-15k/sec or so.

> -----Original Message-----
> From: Oleg.Drokin at Sun.COM [mailto:Oleg.Drokin at Sun.COM]
> Sent: Thursday, March 04, 2010 12:49 PM
> To: Jeffrey Bennett
> Cc: lustre-discuss at lists.lustre.org
> Subject: Re: [Lustre-discuss] One or two OSS, no difference?
>
> Hello!
>
>   This is pretty strange. Are there any differences in network  
> topology that can explain this?
>   If you remove the first client, does the second one shows  
> performance
>   at the level of of the first, but as soon as you start the load on  
> the first again, the second
>   client performance drops?
>
> Bye,
>    Oleg
> On Mar 4, 2010, at 1:45 PM, Jeffrey Bennett wrote:
>
>> Hi Oleg, thanks for your reply
>>
>> I was actually testing with only one client. When adding a second  
>> client using a different file, one client gets all the performance  
>> and the other one gets very low performance, any recommendation?
>>
>> Thanks in advance
>>
>> jab
>>
>>
>> -----Original Message-----
>> From: Oleg.Drokin at Sun.COM [mailto:Oleg.Drokin at Sun.COM]
>> Sent: Wednesday, March 03, 2010 5:20 PM
>> To: Jeffrey Bennett
>> Cc: lustre-discuss at lists.lustre.org
>> Subject: Re: [Lustre-discuss] One or two OSS, no difference?
>>
>> Hello!
>>
>> On Mar 3, 2010, at 6:35 PM, Jeffrey Bennett wrote:
>>> We are building a very small Lustre cluster with 32 clients  
>>> (patchless) and two OSS servers. Each OSS server has 1 OST with 1  
>>> TB of Solid State Drives. All is connected using dual-port DDR IB.
>>>
>>> For testing purposes, I am enabling/disabling one of the OSS/OST  
>>> by using the "lfs setstripe" command. I am running XDD and vdbench  
>>> benchmarks.
>>>
>>> Does anybody have an idea why there is no difference in MB/sec or  
>>> random IOPS when using one OSS or two OSS? A quick test with "dd"  
>>> also shows the same MB/sec when using one or two OSTs.
>>
>> I wonder if you just don't saturate even one OST (both backend SSD  
>> and IB interconnect) with this number of clients? Does the total  
>> throughput decreases as you decrease
>> number of active clients and increases as you increase it even  
>> further?
>> Increasing maximum number of in-flight rpcs might help in that case.
>> Also are all of your clients writing to the same file or each  
>> client does io to a separate file (I hope)?
>>
>> Bye,
>>   Oleg
>
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-discuss


Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.