[Lustre-discuss] RPC limitation

[Andreas Dilger]

On 2010-03-09, at 17:40, Jeffrey Bennett wrote:
> Is there any way to "force" an RPC page size of 256kb and pack  
> several 4kb random operations into a 256k page so the final  
> throughput can increase?

As Oleg mentioned, Lustre will in fact pack up to 256 random 4kB  
writes into a single RPC/RDMA over the network, IFF they are for the  
same object.

There is the long-standing bug 988 that relates to packing multiple  
small writes for different objects on the same OST into a single RPC/ 
RDMA request, but there was never enough demand to finish it.

> -----Original Message-----
> From: Andreas.Dilger at sun.com [mailto:Andreas.Dilger at sun.com] On  
> Behalf Of Andreas Dilger
> Sent: Friday, March 05, 2010 2:05 AM
> To: Jeffrey Bennett
> Cc: Oleg.Drokin at Sun.COM; lustre-discuss at lists.lustre.org
> Subject: Re: [Lustre-discuss] One or two OSS, no difference?
>
> On 2010-03-04, at 14:18, Jeffrey Bennett wrote:
>> I just noticed the sequential performance is ok, but the random IO
>> (which is what I am measuring) is not. Is there any way to increase
>> random IO performance on Lustre? We have LUNs that can provide
>> around 250.000 random read 4kb IOPS but we are only seeing 3.000 to
>> 10.000 on Lustre.
>
> There is work currently underway to improve the SMP scaling
> performance for the RPC handling layer in Lustre.  Currently that
> limits the delivered RPC rate to 10-15k/sec or so.
>


Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.