[Lustre-discuss] Problems & partial success on PPC64 (XServe G5) Debian cluster

[Robert Olson]

Since I've got my shiny new PPC64-based Debian Etch installation  
going, I decided to give Lustre another shot on my mac cluster (no  
cross-compilers required).

The kernel patch and build went fine, using vanilla 2.6.18.8.

I had some troubles with the lustre build itself; the main ones being  
that asm/segment.h doesn't exist in powerpc 64-bit, and that the  
generic_find_next_le_bit patch did not apply. Apparently bitops.c is  
now in lib/find_next_bit.c instead of under the arch directory. I  
added generic_find_next_le_bit to find_next_bit.c and things seemed  
to build okay.

I was able to fire everything up, creating merged MDT/MSG and an OST  
on one machine:

mkfs.lustre --reformat --fsname datafs --mdt --mgs /dev/md0
mount -t lustre /dev/md0 /mnt/data/mdt
mkfs.lustre --reformat --fsname datafs --ost -- 
mgsnode=192.5.200.12 at tcp /dev/sdc5
mount -t lustre /dev/sdc5 /mnt/data/ost0

and mounting on a client:

mount -t lustre 192.5.200.12 at tcp:/datafs /tmp/lus

However, when I tried to run bonnie++ I soon got errors & hangage.  
The kernel messages from the server machine are included below.

The client is a NFS-root netbooted machine, served from the same  
machine hosting the Lustre servers, if that makes any difference.  
Running the same kernel & linux distribution.

Thanks for any help / advice.

--bob


Lustre: Added LNI 192.5.200.12 at tcp [8/256]
Lustre: Accept secure, port 988
Lustre: OBD class driver, info at clusterfs.com
         Lustre Version: 1.6.3
         Build Version: 1.6.3-19700101000000- 
PRISTINE-.scratch.lustre.linux-2.6.18.8-2.6.18.8
Lustre: Lustre Client File System; info at clusterfs.com
Lustre: Binding irq 54 to CPU 0 with cmd: echo 1 > /proc/irq/54/ 
smp_affinity
kjournald starting.  Commit interval 5 seconds
LDISKFS FS on md0, internal journal
LDISKFS-fs: mounted filesystem with ordered data mode.
kjournald starting.  Commit interval 5 seconds
LDISKFS FS on md0, internal journal
LDISKFS-fs: mounted filesystem with ordered data mode.
kjournald starting.  Commit interval 5 seconds
LDISKFS FS on md0, internal journal
LDISKFS-fs: mounted filesystem with ordered data mode.
Lustre: MGS MGS started
Lustre: Enabling user_xattr
Lustre: datafs-MDT0000: new disk, initializing
Lustre: MDT datafs-MDT0000 now serving dev (datafs-MDT0000/7a7a4075- 
a2be-b14e-4c37-5d38acc1dbf0) with recovery enabled
Lustre: Server datafs-MDT0000 on device /dev/md0 has started
kjournald starting.  Commit interval 5 seconds
LDISKFS FS on sdc5, internal journal
LDISKFS-fs: mounted filesystem with ordered data mode.
kjournald starting.  Commit interval 5 seconds
LDISKFS FS on sdc5, internal journal
LDISKFS-fs: mounted filesystem with ordered data mode.
kjournald starting.  Commit interval 5 seconds
LDISKFS FS on sdc5, internal journal
LDISKFS-fs: mounted filesystem with ordered data mode.
LDISKFS-fs: file extents enabled
LDISKFS-fs: mballoc enabled
Lustre: Filtering OBD driver; info at clusterfs.com
Lustre: datafs-OST0000: new disk, initializing
Lustre: OST datafs-OST0000 now serving dev (datafs-OST0000/89964d15- 
f57b-8247-433d-ba88b70ed98d) with recovery enabled
Lustre: Server datafs-OST0000 on device /dev/sdc5 has started
Lustre: datafs-OST0000: received MDS connection from 0 at lo
Lustre: MDS datafs-MDT0000: datafs-OST0000_UUID now active, resetting  
orphans
LDISKFS-fs error (device sdc5): ldiskfs_ext_find_extent: bad header  
in inode #19431465: invalid eh_entries - magic f30a, entries 341, max  
340(340), depth 0(0)