[Lustre-devel] SMP Scalability, MDS, reducing cpu pingpong

Oleg Drokin Oleg.Drokin at Sun.COM
Wed Jul 29 12:05:41 PDT 2009


On Jul 29, 2009, at 2:55 PM, Ricardo M. Correia wrote:

> On Qua, 2009-07-29 at 12:01 -0400, Oleg Drokin wrote:
>> Yes. If clients are not contending, we have same benefits, but
>> this never happens in the real world.
>> Creates in a same dir is a contention point on the dir and there is
>> no point in scheduling all clients on different cpus and let them
>> serialize, where we can free those cpus for some other set of clients
>> doing something else.
> Will this still be true with the DMU?

To some degree.
Even without DMU HEAD has pdirops enabled which reduces the locking  
unit to
certain subset of hash values only.
Keep in mind that aside from the dir content locking there is another  
point of
serialization which is to lock entire directory inode to update times,  
and possibly link count (of course normally this is quite small section
that is lost in the noise of search in the directory, but when you  
have a lot of
CPUs, bouncing these pages and locks around cpu caches might raise level
of overhead significantly).

> I don't know which locks are involved at the Lustre level, but the DMU
> itself does leaf-level locking of ZAP structures (used for  
> directories).

Possibly more than one, since in order to ensure you are not inserting a
duplicate entry (with big enough dir) you need to lock all blocks that  
contain entries in a given hash range/whatever is the way ZIL(?)  
orders entries with


More information about the lustre-devel mailing list