[Lustre-discuss] LNET routing question

[Cliff White]

On Mon, Apr 4, 2011 at 1:32 PM, David Noriega <tsk133 at my.utsa.edu> wrote:

> Reading up on LNET routing and have a question. Currently have nothing
> special going on, simply specified tcp0(bond0) on the OSSs and MDS.
> Same for all the clients as well, we have an internal network for our
> cluster, 192.168.x.x.  How would I go about doing the following?
>
> Data1,Data2 = OSS, Meta1,Meta2 = MDS.
>
> Internally its 192.168.1.x for cluster nodes, 192.168.5.x for lustre nodes.
>
> But I would like a 1) a 'forwarding' sever, which would be our file
> server which exports lustre via samba/nfs to also be the outside
> world's access point to lustre(outside world being the rest of the
> campus). 2) a second internal network simply connecting the OSSs and
> MDS to the backup client to do backups outside of the cluster network.
>

Slightly confused am I.
 1) is just a samba/nfs exporter, while you might
have two networks in the one box, you wouldn't be doing any routing,
the Lustre client is re-exporting the FS.
The Lustre client has to find the Lustre servers, the samba/NFS clients only
have to find the Lustre client.

2) if the second internal net connects backup clients directly to OSS/MDS
you  again need no routing.

Lustre Routing is really to connect disparte network hardware for
Lustre traffic, for example Infiniband routed to TCP/IP, or Quadratics to
IB.

Also, file servers are never routers, since they have direct connections to
all clients. Routers are dedicated nodes that have both hardware interfaces
and
sit between a client and server.
Typical setup are things like a cluster with server and clients on IB, you
wish to add a second client pool on TCP/IP, you have to build nodes that
have both TCP/IP and IB interfaces, and those are Lustre Routers.

Since all your traffic is TCP/IP, sounds like normal TCP/IP network
manipulation
is all you are needing. You would need the 'lnet networks' stuff to
align nets with interfaces, and that part looks correct.
cliffw


>
> So would I do the following?
>
> OSS/MDS
> options lnet networks=tcp0(bond0),tcp1(eth3) routes="tcp2 192.168.2.1"
>
> Backup client
> options lnet networks=tcp1(eth1)
>
> Cluster clients
> options lnet networks=tcp0(eth0)
>
> File Server
> options lnet networks=tcp0(eth1),tcp2(eth2) forwarding="enabled"
>
> And for any outside clients I would do the following?
> options lnet networks=tcp2(eth0)
>
> And when mounting from the outside I would use in /etc/fstab the external
> ip?
> x.x.x.x at tcp2:/lustre /lustre lustre defaults,_netdev 0 0
>
> Is this how it would work? Also can I do this piece-meal or does it
> have to be done all at once?
>
> Thanks
> David
>
> --
> Personally, I liked the university. They gave us money and facilities,
> we didn't have to produce anything! You've never been out of college!
> You don't know what it's like out there! I've worked in the private
> sector. They expect results. -Ray Ghostbusters
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-discuss
>



-- 
cliffw
Support Guy
WhamCloud, Inc.
www.whamcloud.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20110404/24ba171c/attachment.htm>