[Lustre-discuss] Future of lustre 1.8.3+(Debian and SLES/RH kernels)

Fri May 21 01:00:04 PDT 2010

Hi Andreas,

On Thu, 2010-05-20 at 14:45 -0600, Andreas Dilger wrote:

> > On Thu, 2010-05-20 at 10:16 -0600, Andreas Dilger wrote:
> >> The SLES11 kernel is at 2.6.27 so it could be usable for this.  
> >> Also, I
> >
> > Ok, I am getting
> > http://downloads.lustre.org/public/kernels/sles11/linux-2.6.27.39-0.3.1.tar.bz2
> >
> > but, please. Where can I get a suitable config file to apply both for
> > servers and clients?
> 
> The config files are in the Lustre source tree in lustre/ 
> kernel_patches/kernel_configs/

Of course. I did not remember. Anyway, I downloaded:
kernel-default-base-2.6.27.39-0.3_lustre.1.8.3.x86_64.rpm 
and extracted:

config-2.6.27.39-0.3_lustre.1.8.3-default

which I modified at:
CONFIG_LOCALVERSION="-0.3_lustre.1.8.3-default"
to
CONFIG_LOCALVERSION="-0.3.1-lustre.1.8.3"
because debian 'make-kpkg' command refused to create the package.

By the way. I've just compared that config file with the one in lustre
source tree:
kernel-2.6.27-2.6-sles11-x86_64.config 
and I found a difference:
CONFIG_THINKPAD_ACPI_BAY=y
NOT the one in rpm package. I do not think this is meaningful, do I?

Anyway, in future, which is the best config file to get from?

> 
> You can just use the .config file from your client kernel (ie  
> patchless) since you don't need the patched server kernel on the  
> clients.
> 
> >> thought that there were Debian packages for Lustre, why not use  
> >> those?
> >
> > Yes. I used them at the beginning, but after lustre 1.8.0.1 vanilla
> > servers are not any more supported to be used as servers and I started
> > to use RH5 kernels with lustre-1.8.1.1, both for clients and servers.
> 
> That's too bad that the Debiana maintainers have stopped making  
> updates. Doubly so because we worked to add the fixes needed to build  
> on Debian into the Lustre sources, so it is possible to do "make debs"  
> to get Debian packages using all of the standard Debian packaging tools.

No. They did not. Debian Squeeze (testing) is offering lustre-1.8.2, but
ONLY for clients and vanilla kernels 2.6.26 and 2.6.27. I my case I
intend to build the whole cluster (both servers and clients) with Ubuntu
10.04, so, though it is not a requirement, it is more practical, I
think, to use an only kernel for every node.
I started, when using lustre 1.6 series using vanilla kernel 2.6.22-14,
but then, at lustre-1.8.1 vanilla kernels are not supported any more to
be used in servers. So I started using RH5 kernels with Ubuntu 8.04
using udev-117 and lustre 1.8.1.1. I make some test and everything
seemed to be fine.
Now I intend to upgrade to Ubuntu 10.04 using udev-151, and I found the
problem with udev and kernel versions under 2.6.27, but using SLES11
kernel boots and seems to work (not tested lustre yet)

Sometimes I can not avoid to think in using RH5 or SLES11 on our cluster
to simplify problems, but I am too used to Debian/Ubuntu and I loved
them. So I'd like to know if:

- Anyone have been had a similar experience? 
- Any comments about the approach I am using to start lustre on Ubuntu?

> 
> > In my case using Ubuntu 10.04 with kernel 2.6.32 NOT suported as a  
> > lustre  patchless client, I prefer using an only kernel for the  
> > whole cluster,
> > do I?
> >
> That is by no means a requirement.
> 
> > Thanks for your answer.
> >
> > Regards
> >
> >>
> >> Cheers, Andreas
> >>
> >> On 2010-05-20, at 9:48, Ramiro Alba Queipo <raq at cttc.upc.edu> wrote:
> >>
> >>> Hi all,
> >>>
> >>> On Wed, 2010-05-19 at 14:43 +0200, Bernd Schubert wrote:
> >>>
> >>>> That is what I would recommend and what several groups do (usually
> >>>> with
> >>>> Debian, though).
> >>>>
> >>>>> not running the sles/rhel distro ? I know a lot of things can
> >>>>> happen but
> >>>>> are these rhel/sles patches do brake some key features of the
> >>>>> kernel which
> >>>>> would  only work under that specific distro ? I've positivley
> >>>>> tested a
> >>>>> lustre client with a sles patched kernel on a gentoo distro. But
> >>>>> i'am a
> >>>>> bit nervous about testing it on our live lustre server system.
> >>>>
> >>>> The only thing that really might cause trouble is udev, since sysfs
> >>>> maintainers like to break old udev versions. I think upcoming
> >>>> Debian Squeeze
> >>>> requires 2.6.27 at a minimum.
> >>>
> >>> Argg!!!, I think I am stacked with this situation. I compiled
> >>> linux-2.6.18-164.11.1.el5.tar.bz2 under Ubuntu 10.04 (from Debian
> >>> Squeeze) using make-kpkg and after dealing with postinst.d hooks to
> >>> create initrd image (not created by default!!!), I tried to boot and
> >>> then I got the following before dropping into a initramfs shell:
> >>>
> >>> ---
> >>> ---
> >>> --- 
> >>> ------------------------------------------------------------------
> >>> Begin: Mounting root file system... ...
> >>> Begin: Running /scripts/local-top ...
> >>> Done.
> >>> libudev: udev_monitor_new_from_netlink: error getting socket:  
> >>> Invalid
> >>> argument wait-for-root[431] : segfault
> >>> Gave up waiting form root device. Common problems:
> >>> - Boot args (cat /proc/cmdline)
> >>>   - Check rootdelay= (did the system wait long enough?)
> >>>   - Check root= (did the system wait for the right device?)
> >>> -Missing modules (cat /proc/modules; ls /dev)
> >>> ALERT! /dev/disk/by-uuid/0cff75ed-ba7f-4799-b596-cf2b214b9768 does  
> >>> not
> >>> exist. Dropping to a shell!
> >>>
> >>>
> >>> BusyBox v1.13.3 (Ubuntu 1:1.13.3-1ubuntu11) buit-in shell (ask)
> >>> Enter 'Help' for a list of built-in commands.
> >>>
> >>> (initramfs)
> >>> ---
> >>> ---
> >>> ---
> >>> --- 
> >>> ------------------------------------------------------------------
> >>>
> >>> Is this the problem you are talking about? Could you give me some
> >>> ideas
> >>> of how can be solved?
> >>>
> >>> Regards
> >>>
> >>>
> >>> -- 
> >>> Ramiro Alba
> >>>
> >>> Centre Tecnològic de Tranferència de Calor
> >>> http://www.cttc.upc.edu
> >>>
> >>>
> >>> Escola Tècnica Superior d'Enginyeries
> >>> Industrial i Aeronàutica de Terrassa
> >>> Colom 11, E-08222, Terrassa, Barcelona, Spain
> >>> Tel: (+34) 93 739 86 46
> >>>
> >>>
> >>> -- 
> >>> Aquest missatge ha estat analitzat per MailScanner
> >>> a la cerca de virus i d'altres continguts perillosos,
> >>> i es considera que està net.
> >>>
> >>> _______________________________________________
> >>> Lustre-discuss mailing list
> >>> Lustre-discuss at lists.lustre.org
> >>> http://lists.lustre.org/mailman/listinfo/lustre-discuss
> >>
> > -- 
> > Ramiro Alba
> >
> > Centre Tecnològic de Tranferència de Calor
> > http://www.cttc.upc.edu
> >
> >
> > Escola Tècnica Superior d'Enginyeries
> > Industrial i Aeronàutica de Terrassa
> > Colom 11, E-08222, Terrassa, Barcelona, Spain
> > Tel: (+34) 93 739 86 46
> >
> >
> > -- 
> > Aquest missatge ha estat analitzat per MailScanner
> > a la cerca de virus i d'altres continguts perillosos,
> > i es considera que està net.
> >
> 
-- 
Ramiro Alba

Centre Tecnològic de Tranferència de Calor
http://www.cttc.upc.edu

Escola Tècnica Superior d'Enginyeries
Industrial i Aeronàutica de Terrassa
Colom 11, E-08222, Terrassa, Barcelona, Spain
Tel: (+34) 93 739 86 46

-- 
Aquest missatge ha estat analitzat per MailScanner
a la cerca de virus i d'altres continguts perillosos,
i es considera que està net.