[Lustre-discuss] [Discuss] coverage measurement for some lustre test suites

Henry Newman hsn at instrumental.com
Sun Jul 22 09:15:19 PDT 2012


Roman,

> Maybe this can be used for proving orientation of benchmark/stress tests when we will have coverage for them.

I think the other area of interest would be failover and failback. 

hsn

________________________________________
Henry Newman | CEO/CTO
Instrumental, Inc | High Performance Innovation
1450 Energy Park Drive Suite 375
St. Paul, MN 55108-5274
Direct:      651-280-4801
Fax:         651-280-4839
Toll-free:   1.800.886.6188
E-Mail:      hsn at instrumental.com
Web:         http://www.instrumental.com


STRICTLY PERSONAL AND CONFIDENTIAL. This email may contain confidential and proprietary material for the sole use of the intended recipient. Any review or distribution by others is strictly prohibited. If you are not the intended recipient please contact the sender and delete all copies.

-----Original Message-----
From: discuss-bounces at lists.opensfs.org [mailto:discuss-bounces at lists.opensfs.org] On Behalf Of Roman Grigoryev
Sent: Sunday, July 22, 2012 3:24 AM
To: Roman Grigoryev; Andreas Dilger
Cc: discuss at lists.opensfs.org; lustre-discuss at lists.lustre.org
Subject: Re: [Discuss] coverage measurement for some lustre test suites

Andreas Dilger <adilger at whamcloud.com> wrote at  Sat, 21 Jul 2012 21:35:50  
+0400:

> On 2012-07-18, at 6:49 AM, Roman Grigoryev wrote:
>> we did some work for collecting code coverage which generate some 
>> Lustre tests.
>>
>> Results are available on opensfs site 
>> http://www.opensfs.org/foswiki/bin/view/Lustre/CodeCoverage , please 
>> look and comment it.
>
> This looks quite interesting.  I guess the next step is to figure out 
> which functions are receiving no coverage during testing, and write 
> tests to exercise them.

This is the most important one of possible ways for using this info.

It is possible to use this info for finding probably death code. It will be  good to check it with static analyzer like Coverity or Polyspace too.

Also, I think, it could be helpful to know which test cover which code (for example for developers, for quick testing new code after fix before uploading for review/wide testing and maybe put changes to these tests). I have coverage by test and have script for searching test by code line.
In future, we could create script for searching tests with maximum codecoverage(and probably we could update SLOW subset) and searching tests with  minimum coverage or duplicate coverage  and possible refactor them

Maybe this can be used for proving orientation of benchmark/stress tests when we will have coverage for them.

>
> I suspect one type of function which gets very little testing is the  
> lprocfs functions.  Writing a simple test to read all of the functions  
> would get coverage, but it would be better to have actual tests that  
> verify the content of the files is actually correct.

Absolutely agree that  just coverage is not enough for prove quality. In  
coverage theory percent of covered set of all combination of execution  
paths should be used for proving quality but this 'spherical cow' for the  
most of application.

I think,  in real life we should use rule like this 'dummy coverage is  
better than zero coverage but coverage which provide real use case is  
better dummy coverage'.

Ideally, I prefer to have real use cases in tests, dummy coverage in unit  
tests and also use advanced static code analyzers for all code.

-- 
Thanks,
	Roman

>
> Cheers, Andreas
> --
> Andreas Dilger                       Whamcloud, Inc.
> Principal Lustre Engineer            http://www.whamcloud.com/
>
>
>
_______________________________________________
discuss mailing list
discuss at lists.opensfs.org
http://lists.opensfs.org/listinfo.cgi/discuss-opensfs.org
STRICTLY PERSONAL AND CONFIDENTIAL. This email may contain confidential and proprietary material for the sole use of the intended recipient. Any review or distribution by others is strictly prohibited. If you are not the intended recipient please contact the sender and delete all copies.




More information about the lustre-discuss mailing list