[Lustre-discuss] seek heavy workloads and client page cache

[Guy Coates]

I have a seek heavy workload on a couple of files which are small enough to fit
into the client page cache. If I run the workload on local disk, the first run
is comparatively slow but subsequent runs are much faster as the files are all
cached.

If I run the same workload over lustre the initial and subsequent run-times are
always the same (~10-15% slower than the local disk with no caching).

It looks as if there is some attempt at caching; during the initial lustre run
the client page cache grows and in subsequent runs there is no disk IO on the
OSTs.  However, the caching does not improve performance.


I am running lustre 1.6.4.3 / kernel 2.6.18 x86 on debian etch.


strace snippet of the workload:


read(5, "Q1CJ86.1/2-397 2 397 Q1CJ86.1\n", 2097152) = 30
_llseek(4, 92274688, [92274688], SEEK_SET) = 0
read(4, "z\0\0\0\0\0\0\1\217Q1AWM4.1\0\0\0\0\0\0\0S6\254\20\0\0"..., 2097152) =
2097152
_llseek(4, 46137344, [46137344], SEEK_SET) = 0
read(4, "\0\1IA5L198.1\0\0\0\0\0\0\0B\333\245\210\0\0\0\0B\333\245"..., 2097152)
= 2097152
_llseek(4, 69206016, [69206016], SEEK_SET) = 0
read(4, "\0\0\0\0\3\6A8D589.1\0\0\0\0\0\0\0p\3253\206\0\0\0\0p\325"..., 2097152)
= 2097152
_llseek(4, 79691776, [79691776], SEEK_SET) = 0
read(4, "dQ06L05.1\0\0\0\0\0\0\0q\37A\376\0\0\0\0q\37B&\0\0\0\0"..., 2097152) =
2097152
_llseek(4, 85983232, [85983232], SEEK_SET) = 0
read(4, "\0\0\0\1\254Q0RI72.1\0\0\0\0\0\0\0Q\22C\251\0\0\0\0Q\22"..., 2097152) =
2097152
_llseek(4, 90177536, [90177536], SEEK_SET) = 0
read(4, "Y2.1\0\0\0\0\0\0\0q\222+\10\0\0\0\0q\222+>\0\0\0\0\0\0"..., 2097152) =
2097152
_llseek(4, 92274688, [92274688], SEEK_SET) = 0
_llseek(4, 92274688, [92274688], SEEK_SET) = 0
read(4, "z\0\0\0\0\0\0\1\217Q1AWM4.1\0\0\0\0\0\0\0S6\254\20\0\0"..., 2097152) =
2097152
_llseek(4, 94371840, [94371840], SEEK_SET) = 0
_llseek(4, 94371840, [94371840], SEEK_SET) = 0
_llseek(4, 94371840, [94371840], SEEK_SET) = 0
_llseek(4, 94371840, [94371840], SEEK_SET) = 0
_llseek(4, 94371840, [94371840], SEEK_SET) = 0
_llseek(4, 94371840, [94371840], SEEK_SET) = 0
_llseek(4, 94371840, [94371840], SEEK_SET) = 0
_llseek(4, 94371840, [94371840], SEEK_SET) = 0
_llseek(4, 94371840, [94371840], SEEK_SET) = 0
_llseek(4, 94371840, [94371840], SEEK_SET) = 0
_llseek(4, 94371840, [94371840], SEEK_SET) = 0
_llseek(4, 94371840, [94371840], SEEK_SET) = 0
_llseek(4, 94371840, [94371840], SEEK_SET) = 0
_llseek(3, 1402994688, [1402994688], SEEK_SET) = 0
read(3, "NGLNDFIVINEKYNTTIASRKN\nSVGMTVIDD"..., 2097152) = 2097152
_llseek(3, 1405091840, [1405091840], SEEK_SET) = 0



Cheers,

Guy

-- 
Dr. Guy Coates,  Informatics System Group
The Wellcome Trust Sanger Institute, Hinxton, Cambridge, CB10 1HH, UK
Tel: +44 (0)1223 834244 x 6925
Fax: +44 (0)1223 496802



-- 
 The Wellcome Trust Sanger Institute is operated by Genome Research 
 Limited, a charity registered in England with number 1021457 and a 
 company registered in England with number 2742969, whose registered 
 office is 215 Euston Road, London, NW1 2BE.