[Lustre-discuss] NUMA IO and Lustre

[Daniel Kobras]

Hi!

On Tue, May 12, 2009 at 05:49:26PM +0200, Sébastien Buisson wrote:
> Your workaround sounds very interesting, because it shows that the 
> network settings of an OST cannot be fixed by any configuration 
> directive: this is done in an opaque way at mkfs.lustre time.

To clarify: It's the initial mount rather than mkfs. The process as far
as I understood works as follows (and hopefully someone will correct me
if I got it wrong):

* OST is formatted.
* OST is mounted for the first time.
  + OST queries the current list of nids on its OSS.
  + OST sends its list of nids off to the MGS.
* MGS registers new OST (and its nids).
* MGS advertises new OSTS (and its nids) to all (present and future) clients.

It's important to note that MGS registration is a one-off process that
cannot be changed or redone later on, unless you wipe the complete
Lustre configuration from all servers using the infamous --writeconf
method and restart all Lustre clients to remove any stale cached info.
(Which of course implies a downtime of the complete filesystem.) Once
the registration is done, you can change a server's LNET configuration,
but the MGS won't care, and the clients will never get notified of it.

Regards,

Daniel.