[Lustre-discuss] NUMA IO and Lustre

[Sébastien Buisson]

Hello,

At Bull we would like to use a specific machine as Lustre OSS server. 
This is a NUMA IO machine made of 2 Infiniband interfaces for the 
connections to the clients, and 2 FiberChannel interfaces giving access 
to 8 LUNs.

Given this architecture, the stake is to avoid as much as possible 
suffering from the NUMA factor. Ideally, this would require the ability 
from Lustre to 'bind' a given OST to a given IB interface (let's 
consider we know which IB interface best suits a given FC interface). 
The goal is to ensure that no 'NUMA IO tax' is paid when data is 
transferred between an FC interface and an IB interface, ie when a 
Lustre client reads or writes from/to an OST.

Concretely, we would like to know if it is possible in Lustre to bind an 
OST to a specific network interface, so that this OST is only reached 
through this interface (thus avoiding the NUMA IO factor in our case) ? 
For instance, we would like to have 4 OSTs attached to ib0 and the 4 
other OSTs attached to ib1.
We gave a look at the Lustre Manual, but we did not find any way to 
set-up this feature with the usual "mkfs.lustre" and "mount -t lustre" 
commands.

Cheers,
Sebastien.