[Lustre-discuss] Problems & partial success on PPC64 (XServe G5) Debian cluster
Peter Avakian
Peter.Avakian at Sun.COM
Fri Nov 16 22:09:02 PST 2007
>On Nov 12, 2007 14:55 -0600, Robert Olson wrote:
>> Since I've got my shiny new PPC64-based Debian Etch installation
>> going, I decided to give Lustre another shot on my mac cluster (no
>> cross-compilers required).
>On 14 November 2007 01:18 Andreas Dilger wrote:
>As a starting point - we basically never test Lustre with a big-endian
>server, so while it works in theory I would instead suggest starting
>with a big-endian client and little-endian servers first, get that working,
>and then tackle the big endian server separately (likely using something
>like 2.6.22 ext4 as the starting point for ldiskfs, since the extent code
>has proper endian swabbing already). You could also try without mballoc
>and extents on the OSTs.
I started reading this thread quite recently; I am not sure how exactly the
little/big endian would you like to be tested? But I thought you might
wanted to look at a simple Fortran based program (below) reflects the
read/write I/O pattern.
Using little-endian reads and writes a lot faster than big-endian (Linux of
course).
I get 500-550 MB/sec read and 200 to 375 MB/sec write with little-endian
files
(this was done on IA32/IA65 systems).
The code was complied with the following options using intel compiler:
1) ifort -O3 -assume byterecl writer.f
2) ifort -O3 -convert big_endian -assume byterecl writer.f
#cat writer.f
implicit none
integer, parameter :: number_x = 2000, number_y = 2000, number_z =
250
integer i,j,k
integer i_instant1, i_instant2, irate
real*8 plane(number_x,number_y),cube(number_x,number_y,number_z)
real*4 time,write_speed,read_speed
character*80 fname,gname
!test Fibre Channel disks using 8GB binary IEEE files:
fname = '/home/peter/test.ieee'
gname = '/home/peter/test2.ieee'
fname = '/home/peter/cmt2/test.ieee'
gname = '/home/peter/cmt2/test2.ieee'
fname = '/home/peter/cmt/test.ieee'
gname = '/home/peter/cmt/test2.ieee'
!print *,'Reading file...'
call system_clock(i_instant1,irate)
open(1,file=fname,form='unformatted',
& access='direct',recl=kind(plane)*number_x*number_y)
do k = 1,number_z
read(1,rec=k)cube(:,:,k)
enddo
close(unit=1)
call system_clock(i_instant2)
time = (i_instant2-i_instant1)/float(irate)
read_speed = 8.*number_x*number_y*number_z/time/1.e6
print *,'read:',read_speed
call system_clock(i_instant1,irate)
!print *,'Writing file...'
open(1,file=gname,form='unformatted',
!!!! & buffered='yes',blocksize=16384,
& access='direct',recl=kind(plane)*number_x*number_y)
do k = 1,number_z
write(1,rec=k)cube(:,:,k)
enddo
close(unit=1)
call system_clock(i_instant2)
time = (i_instant2-i_instant1)/float(irate)
write_speed = 8.*number_x*number_y*number_z/time/1.e6
print *,'write:',write_speed
end
setenv F_UFMTENDIAN big
read: 338.5341
write: 83.41084
read: 369.0582
write: 86.51231
read: 369.6755
..
.
write: 88.34927
read: 368.6313
setenv F_UFMTENDIAN big:10,20
read: 807.5425
..
.
write: 753.9417
read: 776.3146
write: 776.1639
Regards,
-Peter
More information about the lustre-discuss
mailing list