Creating Lustre Packages From Source for RHEL 4.5 AS, 64-bit ============================================================ Pre-requisites -------------- RHEL 4.5 AS Full Installation SUN-supplied Linux RDAC kernel modules installed. System running on Stock RHEL 4.5 AS kernel with RDAC support. Quilt Source Lustre Source RHEL 4.5 AS Kernel SRPM 1. Download and install Quilt: http://download.savannah.gnu.org/releases/quilt/ 2. Install the RHEL 4.5 AS Kernel SRPM, found on RHEL 4.5 source CD 4. 3. Change to the Red Hat RPM specs directory and extract the full kernel source tree. This will also apply Red Hat's patches to the source: cd /usr/src/redhat/SPECS rpmbuild -bp kernel-2.6.spec 4. Install the Lustre sources. rpm -ivh lustre-source-1.6.5.1-2.6.9_67.0.7.EL_lustre.1.6.5.1smp.x86_64.rpm 5. Prepare the kernel source for the Lustre patches: cd /usr/src/redhat/BUILD/kernel-2.6.9/linux-2.6.9 rm -f patches series ln -s /usr/src/lustre-1.6.5.1/lustre/kernel_patches/series/2.6-rhel4.series series ln -s /usr/src/lustre-1.6.5.1/lustre/kernel_patches/patches . 6. Apply the Lustre patches to the kernel sources: cd /usr/src/redhat/BUILD/kernel-2.6.9/linux-2.6.9 quilt push -av Points to note in output: Hunk #1 FAILED at 770. 1 out of 1 hunk FAILED -- rejects in file fs/nfs/nfs4proc.c Patch patches/vfs_intent-2.6-rhel4.patch does not apply (enforce with -f) Full output: [root@mds-2 linux-2.6.9]# quilt push -av Applying patch patches/vfs_intent-2.6-rhel4.patch patching file fs/cifs/dir.c patching file fs/exec.c patching file fs/inode.c patching file fs/namei.c patching file fs/namespace.c patching file fs/nfs/dir.c patching file fs/nfs/nfs4proc.c Hunk #1 FAILED at 770. 1 out of 1 hunk FAILED -- rejects in file fs/nfs/nfs4proc.c patching file fs/open.c patching file fs/stat.c patching file include/linux/dcache.h patching file include/linux/fs.h patching file include/linux/mount.h patching file include/linux/namei.h Restoring include/linux/dcache.h Restoring include/linux/namei.h Restoring include/linux/mount.h Restoring include/linux/fs.h Restoring fs/stat.c Restoring fs/open.c Restoring fs/namespace.c Restoring fs/exec.c Restoring fs/nfs/nfs4proc.c Restoring fs/nfs/dir.c Restoring fs/namei.c Restoring fs/cifs/dir.c Restoring fs/inode.c Patch patches/vfs_intent-2.6-rhel4.patch does not apply (enforce with -f) Restoring include/linux/dcache.h Restoring include/linux/namei.h Restoring include/linux/mount.h Restoring include/linux/fs.h Restoring fs/stat.c Restoring fs/open.c Restoring fs/namespace.c Restoring fs/exec.c Restoring fs/nfs/nfs4proc.c Restoring fs/nfs/dir.c Restoring fs/namei.c Restoring fs/cifs/dir.c Restoring fs/inode.c 7.0. Edit the Makefile in the Kernel Source and ensure that the EXTRAVERSION variable matches the suffix of the original kernel. e.g.: EXTRAVERSION = -55.ELsmp If this is not done, the OFED build will fail later on, as it will incorrectly identify the kernel and apply the wrong patches. This in turn leads to a build failure when compiling the "ofa_kernel" rpm. 7.2. Prepare the kernel source. cd /usr/src/redhat/BUILD/kernel-2.6.9/linux-2.6.9 cp /usr/src/lustre-1.6.5.1/lustre/kernel_patches/kernel_configs/kernel-2.6.9-2.6-rhel4-x86_64-smp.config .config make oldconfig || make menuconfig make include/asm make include/linux/version.h make SUBDIRS=scripts 7.5. Backup the old kernel modules. mv /lib/modules/2.6.9-55.ELsmp /lib/modules/2.6.9-55.ELsmp.orig 8. Build the kernel and the kernel modules. make bzImage make modules && make modules_install 9. Copy the kernel image into the /boot partition and create initial RAM-disk for the new kernel. cp arch/x86_64/boot/bzImage /boot/vmlinuz-2.6.9-55.EL_lustre.1.6.5.1smp cd /boot mkinitrd initrd-2.6.9-55.EL_lustre.1.6.5.1smp 2.6.9-55.ELsmp 10. Create a new menu entry in GRUB. 11. Reboot the system, select the new kernel entry to load the OS. 12. Build the RDAC modules. Install the mpp initrc image into the boot partition and update GRUB. Reboot. 13. Build and install OFED. Reboot when complete. ./install.pl -c ofed.conf.Volt -s /usr/src/redhat/BUILD/kernel-2.6.9/linux-2.6.9 cd /usr/src/ofa_kernel-1.3.1 /usr/share/doc/ofed-docs-1.3.1/create_Module.symvers.sh cat /usr/src/ofa_kernel-1.3.1/Module.symvers >> /usr/src/redhat/BUILD/kernel-2.6.9/linux-2.6.9/Module.symvers 14. Build Lustre cd /usr/src/lustre-1.6.5.1/ # Use the following command line: ./configure --with-linux=/usr/src/redhat/BUILD/kernel-2.6.9/linux-2.6.9 --with-o2ib=/usr/src/ofa_kernel-1.3.1 # This is the command line in the Lustre spec file: ./configure '--with-linux=/usr/src/redhat/BUILD/kernel-2.6.9/linux-2.6.9' '--with-o2ib=/usr/src/ofa_kernel-1.3.1' '--enable-liblustre' '--enable-liblustre-tests' '--with-lustre-hack' '--with-sockets' '--sysconfdir=/etc' '--mandir=/usr/share/man' '--libdir=/usr/lib64' --with-lustre-hack --with-sockets --sysconfdir=/etc --mandir=/usr/share/man --libdir=/usr/lib64 make make rpms