[Lustre-discuss] OSSes on dual IOH motherboards

Andreas Dilger adilger at whamcloud.com
Thu Jun 2 18:34:19 PDT 2011

Look for the Bull NUMIOA presentation from the recent LUG. The short story is that OST thread pinning is critical to getting good performance.  The numbers are something like 3.6GB/s without, and 6.0 GB/s with thread affinity. 

Cheers, Andreas

On 2011-06-02, at 7:23 PM, Mark Nelson <mark at msi.umn.edu> wrote:

> Hi List,
> I was wondering if anyone here has looked at the performance 
> characteristics of lustre OSSes on dual tylersburg motherboards with 
> raid controllers split up onto separate IO hubs.  I imagine that without 
> proper pinning of service threads to the right CPUs/IOH and memory pools 
> this could cause some nasty QPI contention.  Is this actually a problem 
> in practice?  Is it possible to pin service threads in a reasonable way 
> based on which OST is involved?  Anyone doing this on purpose to try and 
> gain more overall PCIE bandwidth?
> I imagine that in general it's probably best to stick with a single 
> socket single IOH OSS.  No pinning to worry about, very direct QPI 
> setup, consistent performance characteristics, etc.
> Thanks,
> Mark
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-discuss

More information about the lustre-discuss mailing list