[Lustre-discuss] Performance for Lustre 1.6.5

[Aaron Knister]

Lustre doesn't perform well with these kinds of tiny file i/o and  
metadata intensive operations. If you search back through the lists  
this issue has been covered a bunch of times. Small i/o is not  
efficient for a large filesystem and lustre wasn't designed with this  
kind of workload in mind. It was designed for a proper HPC workload.

On Jul 17, 2008, at 4:45 AM, Johnlya wrote:

> ext3 Filesystem test:
> #!/bin/bash
> for ((i=0; $i<1000000; i++))
> do
>                fileid="/home/test/testm/$i";
>                touch $fileid
> done
>
> time ./fileid.sh
> real    16m44.438s
> user    3m0.860s
> sys     12m35.920s
>
> time ls
> real    1m54.691s
> user    0m13.770s
> sys     1m0.220s
>
> touch:1000000/(16*60+44.438)=995.58
> ls:1000000/(1*60+54.691)=8719.08
>
> Lustre 1.6.5 slower than normal file system?
>
> Add:
>    The network of Lustre Filesystem is Giga network.
>
> On Thu, 2008-07-17, at 03:50 PM, Johnlya <john... at gmail.com> wrote:
>> I test touch and ls command of Lustre on Linux  
>> 2.6.9-67.0.7.EL_lustre.
>> 1.6.5smp (x86, 64bit, mem:4G,two CPU).
>> Lustre Filesystem:
>> one MDS
>> one OSS(one OST)
>> I found it some problems:
>> With the increase in the number of files, Lustre system performance
>> decrease too much.
>> 1. fileid.sh
>> #!/bin/bash
>> for ((i=0; $i<10000; i++))
>> do
>>                 fileid="/mnt/webfile/test/$i";
>>                 touch $fileid
>> done
>> time ./fileid.sh
>> eal    0m20.493s
>> user    0m1.930s
>> sys     0m9.340s
>>
>> time ls
>> real    0m2.748s
>> user    0m0.050s
>> sys     0m0.610s
>> touch:10000/20.493=487.97
>> ls:10000/2.748=3639.01
>> 2.
>> #!/bin/bash
>> for ((i=0; $i<1000000; i++))
>> do
>>                 fileid="/mnt/webfile/test/testm/$i";
>>                 touch $fileid
>> done
>> time ./fileid.sh
>> real    60m56.666s
>> user    3m11.690s
>> sys     20m37.190s
>>
>> real    15m48.615s
>> user    0m14.530s
>> sys     3m52.960s
>>
>> touch:1000000/(60*60+56.666)=273.47
>> ls:1000000/(15*60+48.615)=1054.16
>>
>> _______________________________________________
>> Lustre-discuss mailing list
>> Lustre-disc... at lists.lustre.orghttp://lists.lustre.org/mailman/listinfo/lustre-discuss
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-discuss