[Lustre-discuss] Lustre routers capabilities

[Isaac Huang]

On Thu, Apr 10, 2008 at 10:06:40AM +0200, S?bastien Buisson wrote:
> Let's consider that the internal bus of the machine is bigger enough so 
> that it will not be saturated. In that case, what will be the limiting 
> factor? memory? CPU?
> I know that it depends on how many I/B cards are plugged in the machine, 
> but generally speaking, is the routing activity CPU or memory hungry?
> 

LNet routing is always memory hungry. Be sure to use 64-bit routers,
because router buffers can only come from low system memory (i.e.
ZONE_DMA and ZONE_NORMAL) which is usually 896M on 32-bit systems.

Router CPU usage depends on network types. When routing between IB
networks, router's CPU usage shall be minimal. But when a router is 
homed in a TCP network, it can be CPU hungry since host CPU must be
involved in copying data received from the TCP network.

Isaac