[Lustre-discuss] lustre-discuss at lists.lustre.org

[DEGREMONT Aurelien]

Hi Chris,

Here is a rough example of junit report we produce. This was a quick and 
dirty implementation of Junit that could be improved. Some part of junit 
report content was limited due to lack of information acceptance small 
upcalls have.

<?xml version="1.0" encoding="UTF-8" ?>
<testsuite name="Lustre.acceptance-small">
<testcase classname="Lustre.sanity" name="Test #0: touch .../ ; rm .../" 
time="1" />
<testcase classname="Lustre.sanity" name="Test #0b: chmod 0755 
/mnt/lustre" time="1" />
<testcase classname="Lustre.sanity" name="Test #0c: check import proc" 
time="1" />
<testcase classname="Lustre.sanity" name="Test #1a: mkdir .../d1; mkdir 
.../d1/d2" time="1" />
<testcase classname="Lustre.sanity" name="Test #1b: rmdir .../d1/d2; 
rmdir .../d1" time="1" />
<testcase classname="Lustre.sanity" name="Test #2a: mkdir .../d2; touch 
.../d2/f" time="1" />
<testcase classname="Lustre.sanity" name="Test #2b: rm -r .../d2; 
checkstat .../d2/f" time="1" />
<testcase classname="Lustre.sanity" name="Test #3a: mkdir .../d3" 
time="1" />
<testcase classname="Lustre.sanity" name="Test #3b: touch .../d3/f" 
time="1" />
<testcase classname="Lustre.sanity" name="Test #3c: rm -r .../d3" 
time="1" />
<testcase classname="Lustre.sanity" name="Test #4a: mkdir .../d4" 
time="1" />
<testcase classname="Lustre.sanity" name="Test #4b: mkdir .../d4/d2" 
time="1" />
...
<testcase classname="Lustre.sanity" name="Test #180a: test obdecho on 
osc" time="80">
  <failure type="FAIL"><![CDATA[rc=1
test_180a failed with 1]]></failure>
</testcase>
...
<testcase classname="Lustre.sanity-benchmark" name="Test #dbench: 
test_dbench" time="0">
  <skipped/>
</testcase>
<testcase classname="Lustre.sanity-benchmark" name="Test #dbench: 
dbench" time="2" />
...
</testsuite>

This is a imple junit report. This could be improved. Partly in 
improving acceptance-small.sh and test-framework.sh.
I'm open to switch to any other standard format, supported by Hudson, if 
this can helps.

What's your needs/plan?

Aurélien

Chris Gearing a écrit :
> Hi Aurélien,
>
> Do you have a specification for the junit test results you produce, or 
> an example of one of your test results sets. We would be more than
> willing to pick up and go with something that can be used with a wider 
> set of tools, with the obvious caveat that it provides everything needed 
> to completely capture the test results for Lustre today and in the future.
>
> If you have some example results set's that you can forward please mail 
> them to chris whamcloud.com
>
> Thanks
>
> Chris
>
> I see that PerfPublisher uses xml, although this seems to be the only 
> specification.
>
> On 17/12/2010 20:11, Aurélien wrote:
>   
>>> Robert Read a écrit :
>>>       
>>>> We don't plan to use Hudson to manage our testing results as I don't 
>>>> think it would scale very well for all the testing we might do for 
>>>> each build. We're currently building a more custom results server 
>>>> that's similar (in spirit at least) to the kinds of tools we had at 
>>>> Oracle.  We'll make it available once it's in presentable form.
>>>> Actually, our first step was to replace the acceptance-small.sh 
>>>> driver script with one that has a more sensible user interface for 
>>>> running the standard tests.  Since the test-framework.sh on master 
>>>> has already been changed to produce test results in yaml format, 
>>>>  the new script collects these with the logs, and is capable of 
>>>> submitting them to the test results server.   Currently this is 
>>>> being run manually, though.  Automating the test execution and 
>>>> connecting all the pieces will be next step.
>>>>         
>>> Ok. I will be very interested in seeing the final result.
>>> But I think it is a good idea to stick to standard format and tools 
>>> as much as possible. This could be a pity if all your new work will 
>>> be only usable by  your tool.
>>>
>>> Junit is quite standard.
>>> PerfPublisher has its own format due to junit limitations. There is 
>>> other ones. It could be really good if you do not create a new one.
>>>
>>> And indeed, acc-sm is a bit limited and improve it could be really 
>>> interesting.
>>>
>>>
>>> Aurélien
>>>       
>>     
>
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-discuss
>