[Lustre-discuss] IOR performance - Need help

[satish patil]

Hello,

Recently we installed 6 OSS pairs with 8 OST per pair. Total 48 OST's. Each OST is with 3.7 TB. In all it is 177 TB file system. Lustre version installed is 1.8.1.1 and currently using client based on RHEL 5U2 which is 1.6.x. When running the individual OST test from performance perspecitve we are able to get around 17.5 GB performance. Out target is to cross 10 GBPS write performance using single file w/o -F option avoiding client side cache.  I have reached max to 7.5GB for write performance , but not going beyond. I tried using stripe count as 48 for a single file along with default stripe size which is 1MB. But not able to cross 10 GBPS. 

Command line used for running the IOR as follow
/opt/intel/mpi/bin64/mpirun --totalnum=96 --file=$PBS_NODEFILE --rsh=/usr/bin/ssh -1 --ordered --verbose -l -machinefile $PBS_NODEFILE -np 96 /newScratch/IOR/src/C/IOR.mpiio -a MPIIO -b 22G -C -i 3 -k -t 1m -w -r -R -W -x -N 96 -o /newScratch/hp.stripeC48/IOR.dat

We have used lustre_config to create the file system. 

Appriciate your help.

Regards
SP