[Lustre-discuss] bad 1.6.3 striped write performance

[Andrei Maslennikov]

Andreas, here are some numbers obtained against a file striped over 3 OSTs
(there were 8 cores; only 3 of them had any visible load, so I quote the
CPU usage only for these cores):

1) 1.6.3 client checksums enabled: 35 MB/sec
            ldlm_poold: all the time 85-100%,
            ptlrpcd: 15-23%
            dd: from 85% down gradually to 4-6%

2) Idem, after zeroed /proc/fs/lustre/llite/*/checksum_pages: 65 MB/sec
           loads are very much the same as in the first case

3) 1.6.2 client checksums enabled: 790 MB/sec
           dd: 85-95%
           kswapd: 35%
           ptlrpcd: 15-20%

Andrei.

On Nov 26, 2007 7:59 PM, Andreas Dilger <adilger at sun.com> wrote:

> Can you verify that you disabled data checksumming:
>
>         echo 0 > /proc/fs/lustre/llite/*/checksum_pages
>
> Note that there are 2 kinds of checksumming that Lustre does.  The first one
> is checksumming of data in client memory, and the second one is checksumming
> of data over the network.  Setting $LPROC/llite/*/checksum_pages turns on/off
> both in-memory and wire checksums.  Setting $LPROC/osc/*/checksums turns on/off
> the network checksums only.
>
> If checksums are disabled, can you please report if the CPU usage on the
> client is consuming all of the CPU, or possibly all of a single CPU on 1.6.3
> and on 1.6.2?
>