[Lustre-discuss] LNET routing question

[David Noriega]

The file server does sit on both networks, internal and external. I
would just like to have a thrid option beyond nfs/samba, such as
making the linux workstations up in our lab, lustre clients. But you
are saying either 1) I do some sort of regular tcp routing? or 2) an
existing client cannot also work as a router?

On Mon, Apr 4, 2011 at 3:43 PM, Cliff White <cliffw at whamcloud.com> wrote:
>
>
> On Mon, Apr 4, 2011 at 1:32 PM, David Noriega <tsk133 at my.utsa.edu> wrote:
>>
>> Reading up on LNET routing and have a question. Currently have nothing
>> special going on, simply specified tcp0(bond0) on the OSSs and MDS.
>> Same for all the clients as well, we have an internal network for our
>> cluster, 192.168.x.x.  How would I go about doing the following?
>>
>> Data1,Data2 = OSS, Meta1,Meta2 = MDS.
>>
>> Internally its 192.168.1.x for cluster nodes, 192.168.5.x for lustre
>> nodes.
>>
>> But I would like a 1) a 'forwarding' sever, which would be our file
>> server which exports lustre via samba/nfs to also be the outside
>> world's access point to lustre(outside world being the rest of the
>> campus). 2) a second internal network simply connecting the OSSs and
>> MDS to the backup client to do backups outside of the cluster network.
>
> Slightly confused am I.
>  1) is just a samba/nfs exporter, while you might
> have two networks in the one box, you wouldn't be doing any routing,
> the Lustre client is re-exporting the FS.
> The Lustre client has to find the Lustre servers, the samba/NFS clients only
> have to find the Lustre client.
> 2) if the second internal net connects backup clients directly to OSS/MDS
> you  again need no routing.
> Lustre Routing is really to connect disparte network hardware for
> Lustre traffic, for example Infiniband routed to TCP/IP, or Quadratics to
> IB.
> Also, file servers are never routers, since they have direct connections to
> all clients. Routers are dedicated nodes that have both hardware interfaces
> and
> sit between a client and server.
> Typical setup are things like a cluster with server and clients on IB, you
> wish to add a second client pool on TCP/IP, you have to build nodes that
> have both TCP/IP and IB interfaces, and those are Lustre Routers.
> Since all your traffic is TCP/IP, sounds like normal TCP/IP network
> manipulation
> is all you are needing. You would need the 'lnet networks' stuff to
> align nets with interfaces, and that part looks correct.
> cliffw
>
>>
>> So would I do the following?
>>
>> OSS/MDS
>> options lnet networks=tcp0(bond0),tcp1(eth3) routes="tcp2 192.168.2.1"
>>
>> Backup client
>> options lnet networks=tcp1(eth1)
>>
>> Cluster clients
>> options lnet networks=tcp0(eth0)
>>
>> File Server
>> options lnet networks=tcp0(eth1),tcp2(eth2) forwarding="enabled"
>>
>> And for any outside clients I would do the following?
>> options lnet networks=tcp2(eth0)
>>
>> And when mounting from the outside I would use in /etc/fstab the external
>> ip?
>> x.x.x.x at tcp2:/lustre /lustre lustre defaults,_netdev 0 0
>>
>> Is this how it would work? Also can I do this piece-meal or does it
>> have to be done all at once?
>>
>> Thanks
>> David
>>
>> --
>> Personally, I liked the university. They gave us money and facilities,
>> we didn't have to produce anything! You've never been out of college!
>> You don't know what it's like out there! I've worked in the private
>> sector. They expect results. -Ray Ghostbusters
>> _______________________________________________
>> Lustre-discuss mailing list
>> Lustre-discuss at lists.lustre.org
>> http://lists.lustre.org/mailman/listinfo/lustre-discuss
>
>
>
> --
> cliffw
> Support Guy
> WhamCloud, Inc.
> www.whamcloud.com
>
>



-- 
Personally, I liked the university. They gave us money and facilities,
we didn't have to produce anything! You've never been out of college!
You don't know what it's like out there! I've worked in the private
sector. They expect results. -Ray Ghostbusters