[Lustre-discuss] Lnet configuration: 1 ost per gige interface.

[Brian J. Murrell]

On Tue, 2008-10-21 at 11:41 -0500, Hendelman, Rob wrote:
> Hello everyone,

Hi,

> I'd like to dedicate 1 gigabit Ethernet interface per OST.

So to be clear, you have multiple OSTs and multiple GigE interfaces in
each OSS?

I wonder why you want to try to (effectively partition the aggregate
network bandwidth of all your GigE NICs and) dedicate bandwidth on a per
OST basis.  Why not create a large pool of bandwidth on each OSS usable
by all clients and all OSTs and just let demand sort it out.  As long as
your network is not the bottleneck things should just work out.

> My OST's are fairly slow (raid 10, 4 sata drives ea) by
> enterprise standards but still can sustain 80-90MByte/second on large
> sequential reads.  I imagine if I striped a file over 3 OST's I would
> hit speed limitations using GigE.

A single GigE?  Yes.  A single GigE has a theoretical maximum of 125MB/s
so two of your OSTs would saturate that.

> What I'd like to do is have multiple gige nics connected to a (dumb)
> switch to start with.  I'd put each gige nic in a separate /28.  My
> first OSS box has 3 OST's (12 drives) and 3 gig-e nics.  My MDS/MGS has
> a single gige nic.

But why not just keep it all on the same network and just bond the GigE
NICs to increase your total per/OSS network bandwidth (i,e, if a single
interface) and ensure it's always more than the total disk bandwidth of
your OSTs?

Sounds like you are making it a _lot_ more complicated than it needs to
be.

b.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: This is a digitally signed message part
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20081021/db22e145/attachment.pgp>