[Lustre-discuss] [Discuss] coverage measurement for some lustre test suites
Henry Newman
hsn at instrumental.com
Sun Jul 22 09:15:19 PDT 2012
Roman,
> Maybe this can be used for proving orientation of benchmark/stress tests when we will have coverage for them.
I think the other area of interest would be failover and failback.
hsn
________________________________________
Henry Newman | CEO/CTO
Instrumental, Inc | High Performance Innovation
1450 Energy Park Drive Suite 375
St. Paul, MN 55108-5274
Direct: 651-280-4801
Fax: 651-280-4839
Toll-free: 1.800.886.6188
E-Mail: hsn at instrumental.com
Web: http://www.instrumental.com
STRICTLY PERSONAL AND CONFIDENTIAL. This email may contain confidential and proprietary material for the sole use of the intended recipient. Any review or distribution by others is strictly prohibited. If you are not the intended recipient please contact the sender and delete all copies.
-----Original Message-----
From: discuss-bounces at lists.opensfs.org [mailto:discuss-bounces at lists.opensfs.org] On Behalf Of Roman Grigoryev
Sent: Sunday, July 22, 2012 3:24 AM
To: Roman Grigoryev; Andreas Dilger
Cc: discuss at lists.opensfs.org; lustre-discuss at lists.lustre.org
Subject: Re: [Discuss] coverage measurement for some lustre test suites
Andreas Dilger <adilger at whamcloud.com> wrote at Sat, 21 Jul 2012 21:35:50
+0400:
> On 2012-07-18, at 6:49 AM, Roman Grigoryev wrote:
>> we did some work for collecting code coverage which generate some
>> Lustre tests.
>>
>> Results are available on opensfs site
>> http://www.opensfs.org/foswiki/bin/view/Lustre/CodeCoverage , please
>> look and comment it.
>
> This looks quite interesting. I guess the next step is to figure out
> which functions are receiving no coverage during testing, and write
> tests to exercise them.
This is the most important one of possible ways for using this info.
It is possible to use this info for finding probably death code. It will be good to check it with static analyzer like Coverity or Polyspace too.
Also, I think, it could be helpful to know which test cover which code (for example for developers, for quick testing new code after fix before uploading for review/wide testing and maybe put changes to these tests). I have coverage by test and have script for searching test by code line.
In future, we could create script for searching tests with maximum codecoverage(and probably we could update SLOW subset) and searching tests with minimum coverage or duplicate coverage and possible refactor them
Maybe this can be used for proving orientation of benchmark/stress tests when we will have coverage for them.
>
> I suspect one type of function which gets very little testing is the
> lprocfs functions. Writing a simple test to read all of the functions
> would get coverage, but it would be better to have actual tests that
> verify the content of the files is actually correct.
Absolutely agree that just coverage is not enough for prove quality. In
coverage theory percent of covered set of all combination of execution
paths should be used for proving quality but this 'spherical cow' for the
most of application.
I think, in real life we should use rule like this 'dummy coverage is
better than zero coverage but coverage which provide real use case is
better dummy coverage'.
Ideally, I prefer to have real use cases in tests, dummy coverage in unit
tests and also use advanced static code analyzers for all code.
--
Thanks,
Roman
>
> Cheers, Andreas
> --
> Andreas Dilger Whamcloud, Inc.
> Principal Lustre Engineer http://www.whamcloud.com/
>
>
>
_______________________________________________
discuss mailing list
discuss at lists.opensfs.org
http://lists.opensfs.org/listinfo.cgi/discuss-opensfs.org
STRICTLY PERSONAL AND CONFIDENTIAL. This email may contain confidential and proprietary material for the sole use of the intended recipient. Any review or distribution by others is strictly prohibited. If you are not the intended recipient please contact the sender and delete all copies.
More information about the lustre-discuss
mailing list