[Lustre-discuss] Installing Lustre client on 2.6.18-194 kernel

[Albert Everett]

Here's what's in our /etc/modprobe.conf related to IB and lustre:

options ib_mthca msi_x=1
options lnet networks=o2ib0(ib0)
options ko2iblnd ipif_name=ib0

We have Mellanox Infinihost (III?) DDR cards and IPs defined for them.

$ /sbin/ifconfig ib0
ib0       Link encap:InfiniBand  HWaddr 80:00:04:04:FE: 
80:00:00:00:00:00:00:00:00:00:00:00:00:00:00
           inet addr:192.168.2.1  Bcast:192.168.2.255  Mask: 
255.255.255.0
           inet6 addr: fe80::202:c902:29:b341/64 Scope:Link
           UP BROADCAST RUNNING MULTICAST  MTU:65520  Metric:1
           RX packets:1719 errors:0 dropped:0 overruns:0 frame:0
           TX packets:30 errors:0 dropped:0 overruns:0 carrier:0
           collisions:0 txqueuelen:256
           RX bytes:96564 (94.3 KiB)  TX bytes:2420 (2.3 KiB)

Albert

On Feb 20, 2011, at 2:20 PM, Arya Mazaheri wrote:

> I have done what you said. I will test my client to the server  
> tomorrow. but would you tell me the tweaks you have done on /etc/ 
> modprobe.conf ?
>
> On Sun, Feb 20, 2011 at 1:56 AM, Albert Everett <aeeverett at ualr.edu>  
> wrote:
> For lustre client, we did not need to alter our kernel at all. We  
> just made and installed lustre-1.8.5 and lustre-modules-1.8.5 rpms. / 
> etc/modprobe.conf needs a tweak.
>
> For lustre server, I believe you will need to deal with a patched  
> kernel. We have not been down this road yet since our vendor  
> includes lustre server software with their hardware.
>
> Albert
>
>
> On Feb 19, 2011, at 12:18 PM, Arya Mazaheri wrote:
>
> Hi Albert,
> It seems that you have made a new kernel in order to run lustre on  
> clients. Am I right?
> I don't want to change kernel on clients at all...
>
> On Sat, Feb 19, 2011 at 8:57 PM, Albert Everett <aeeverett at ualr.edu>  
> wrote:
> Our kernel is also 2.6.18_194.17.4.el5.
>
> We installed OFED 1.5.2 from source, following this guide:
>
>  https://wiki.rocksclusters.org/wiki/index.php/Install_OFED_1.5.x_on_a_Rocks_5.3_cluster
>
> ... which left us, among other things, a folder /usr/src/ofa_kernel.
>
> Lustre on the server side is handled by our vendor, so all we needed  
> to worry about is the client.
>
> To build a lustre client, we then installed lustre-1.8.5.tar.gz from  
> source, not from rpms. Our first compile produced the error you show  
> below.
> # ./configure --with-linux=/lib/modules/`uname -r`/build
> # make rpms
>
> To get the lustre installation to use our new OFED, we tried this  
> and it worked.
>
> # ./configure --with-o2ib=/usr/src/ofa_kernel --with-linux=/lib/ 
> modules/`uname -r`/build
> # make rpms
>
> RPMs showed up in /usr/src/redhat/RPMS/x86_64, and we are using  
> lustre-1.8.5*.rpm and lustre-modules-*.rpm on our client machines.
>
> Albert
>
>
> On Feb 19, 2011, at 8:34 AM, Arya Mazaheri wrote:
>
> Hi,
> I have installed lustre client packages on a client node. But it  
> doesn't mount the lustre file system from lustre server. It gets the  
> following famous error:
>
> $ mount -t lustre 192.168.0.1:/lustre /mnt/lustre
> mount.lustre: mount 172.16.113.232:/lustre at /mnt/lustre failed: No  
> such device
> Are the lustre modules loaded?
> Check /etc/modprobe.conf and /proc/filesystems
> Note 'alias lustre llite' should be removed from modprobe.conf
>
>
> As I was searching through the mailing list, I have noticed that  
> lustre.ko should be present in this directory:
> /lib/modules/2.6.18-194.17.4.el5/kernel/fs/lustre/lustre.ko
>
> My current kernel is 2.6.18-194.17.4.el5. but lustre.ko is in  
> 2.6.18-164.11.1.el5 instead. So I guessed that this may be the  
> source of problem.
>
> Any ideas?
>
> Thanks
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-discuss
>
>
>
>