[Lustre-discuss] [Discuss] coverage measurement for some lustre test suites
Roman_Grigoryev at xyratex.com
Sun Jul 22 01:24:10 PDT 2012
Andreas Dilger <adilger at whamcloud.com> wrote at Sat, 21 Jul 2012 21:35:50
> On 2012-07-18, at 6:49 AM, Roman Grigoryev wrote:
>> we did some work for collecting code coverage which generate some Lustre
>> Results are available on opensfs site
>> http://www.opensfs.org/foswiki/bin/view/Lustre/CodeCoverage ,
>> please look and comment it.
> This looks quite interesting. I guess the next step is to figure out
> which functions are receiving no coverage during testing, and write
> tests to exercise them.
This is the most important one of possible ways for using this info.
It is possible to use this info for finding probably death code. It will
be good to check it with static analyzer like Coverity or Polyspace too.
Also, I think, it could be helpful to know which test cover which code
(for example for developers, for quick testing new code after fix before
uploading for review/wide testing and maybe put changes to these tests). I
have coverage by test and have script for searching test by code line.
In future, we could create script for searching tests with maximum
codecoverage(and probably we could update SLOW subset) and searching tests
with minimum coverage or duplicate coverage and possible refactor them
Maybe this can be used for proving orientation of benchmark/stress tests
when we will have coverage for them.
> I suspect one type of function which gets very little testing is the
> lprocfs functions. Writing a simple test to read all of the functions
> would get coverage, but it would be better to have actual tests that
> verify the content of the files is actually correct.
Absolutely agree that just coverage is not enough for prove quality. In
coverage theory percent of covered set of all combination of execution
paths should be used for proving quality but this 'spherical cow' for the
most of application.
I think, in real life we should use rule like this 'dummy coverage is
better than zero coverage but coverage which provide real use case is
better dummy coverage'.
Ideally, I prefer to have real use cases in tests, dummy coverage in unit
tests and also use advanced static code analyzers for all code.
> Cheers, Andreas
> Andreas Dilger Whamcloud, Inc.
> Principal Lustre Engineer http://www.whamcloud.com/
More information about the lustre-discuss