[Lustre-devel] [Twg] your opinion about testing improvements

Oleg Drokin green at whamcloud.com
Sun Apr 1 20:08:42 PDT 2012


On Mar 30, 2012, at 3:40 AM, Roman Grigoryev wrote:
> 2) it is not simple execute(especially in automation) testing for test.
> F.e. a bug is fixed, the test on it added. Executing the test on an old
> revision(probably on a previous release) should show failed test result.
> But with big difference between versions where fixed and where execute
> test-framework can fail to start.

I am not quite sure why would you want to constantly fail a test that is known not to work with a particular release due to a missing bugfix.
I think it's enough if a developer (or somebody else) runs the test manually once on an unfixed codebase to make sure the test does without the fix.

The issue of running older release against a newer one is a real one, but the truth is, when you run e.g. 1.8 vs 2.x, it's not just the tests that are different, the init code is different too, so it's not just a matter of separating tests subdir in its own repository.
On our side we just note known broken tests for such configurations and ignore the failures for the lack of better solution.

> Different test cases, ended with letter(f.e. 130c),  have an different
> idea of dependencies. Some test cases have dependences to previous test
> cases, some don't have.

Ideally dependencies should be eliminated (in my opinion, anyway).

Oleg Drokin
Senior Software Engineer
Whamcloud, Inc.

More information about the lustre-devel mailing list