[Lustre-discuss] Problems & partial success on PPC64 (XServe G5) Debian cluster

Peter Avakian Peter.Avakian at Sun.COM
Fri Nov 16 22:09:02 PST 2007


>On Nov 12, 2007  14:55 -0600, Robert Olson wrote:
>> Since I've got my shiny new PPC64-based Debian Etch installation  
>> going, I decided to give Lustre another shot on my mac cluster (no  
>> cross-compilers required).


>On 14 November 2007 01:18 Andreas Dilger wrote:
>As a starting point - we basically never test Lustre with a big-endian
>server, so while it works in theory I would instead suggest starting
>with a big-endian client and little-endian servers first, get that working,
>and then tackle the big endian server separately (likely using something
>like 2.6.22 ext4 as the starting point for ldiskfs, since the extent code
>has proper endian swabbing already).  You could also try without mballoc
>and extents on the OSTs.


I started reading this thread quite recently; I am not sure how exactly the
little/big endian would you like to be tested? But I thought you might
wanted to look at a simple Fortran based program (below) reflects the
read/write I/O pattern.

Using little-endian reads and writes a lot faster than big-endian (Linux of
course). 
I get 500-550 MB/sec read and 200 to 375 MB/sec write with little-endian
files 

(this was done on IA32/IA65 systems). 



The code was complied with the following options using intel compiler:

1) ifort -O3 -assume byterecl writer.f 
2) ifort -O3 -convert big_endian  -assume byterecl writer.f 

#cat writer.f

       implicit none
       integer, parameter :: number_x = 2000, number_y = 2000, number_z =
250
       integer i,j,k
       integer i_instant1, i_instant2, irate
       real*8 plane(number_x,number_y),cube(number_x,number_y,number_z) 
       real*4 time,write_speed,read_speed
       character*80 fname,gname

!test Fibre Channel disks using 8GB binary IEEE files:
       fname = '/home/peter/test.ieee'
       gname = '/home/peter/test2.ieee'
       fname = '/home/peter/cmt2/test.ieee'
       gname = '/home/peter/cmt2/test2.ieee'
       fname = '/home/peter/cmt/test.ieee'
       gname = '/home/peter/cmt/test2.ieee'

       !print *,'Reading file...'
       call system_clock(i_instant1,irate)
       open(1,file=fname,form='unformatted',
     &           access='direct',recl=kind(plane)*number_x*number_y)
       do k = 1,number_z
          read(1,rec=k)cube(:,:,k)
       enddo
       close(unit=1)

       call system_clock(i_instant2)
       time = (i_instant2-i_instant1)/float(irate)
       read_speed = 8.*number_x*number_y*number_z/time/1.e6
       print *,'read:',read_speed
       call system_clock(i_instant1,irate)
       !print *,'Writing file...'
       open(1,file=gname,form='unformatted',
!!!!     &           buffered='yes',blocksize=16384,
     &           access='direct',recl=kind(plane)*number_x*number_y)
       do k = 1,number_z
          write(1,rec=k)cube(:,:,k)
       enddo
       close(unit=1)
       call system_clock(i_instant2)
       time = (i_instant2-i_instant1)/float(irate)
       write_speed = 8.*number_x*number_y*number_z/time/1.e6
       print *,'write:',write_speed
       end 

 


setenv F_UFMTENDIAN big 

read:   338.5341    
write:   83.41084
read:   369.0582    
write:   86.51231    
read:   369.6755    
..
.
write:   88.34927    
read:   368.6313

setenv F_UFMTENDIAN big:10,20
read:   807.5425    
..
.
write:   753.9417    
read:   776.3146    
write:   776.1639



Regards,
-Peter






More information about the lustre-discuss mailing list