<html><head><meta http-equiv="Content-Type" content="text/html charset=windows-1252"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><br><div><div>On Oct 25, 2012, at 2:13 PM, Christopher J. Morrone <<a href="mailto:morrone2@llnl.gov">morrone2@llnl.gov</a>> wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite">It really sounds like you are putting the cart before the horse. Picking a favorite language is probably a bad way to go. First we should figure out why we're changing things, and what the scope and goals of the new design are.<br><br>Brian Behlendorf pointed out the kernel Autotest system as an option, which sounds worthy of investigation to me. If we were to decide on Perl today, that would make working with that Autotest more difficult, because that is all done in Python. But if choose Python and then try to use an entirely Visual Basic based framework (Ha! I kid!) we also probably made the wrong choice.<br><br>In other words, it is important to know something about where we are going before we pick a language.<br></blockquote><div><br></div>As I said before:</div><div><div><div></div><blockquote type="cite"><div>For more context, including the motivation for changing things, see my LAD'12 presentation (more narrowly focused) on the <a href="http://www.opensfs.org/foswiki/bin/view/Lustre/TestingImprovements">OpenSFS Wiki</a>.</div></blockquote><div><br></div><div>Also, see Chris' preso on a language:</div><div><blockquote type="cite">As promised: <a href="http://wiki.whamcloud.com/download/attachments/7634975/new+test+framework.pdf">http://wiki.whamcloud.com/download/attachments/7634975/new+test+framework.pdf</a><br></blockquote></div><div><br></div><div>Picking a language is just one part of it, and definitely not the most important part. I'll be posting more on some other proposed changes in the future.</div><div><br></div><div><br></div><div><div><div>On Oct 25, 2012, at 10:23 AM, Brian Behlendorf <<a href="mailto:behlendorf1@llnl.gov">behlendorf1@llnl.gov</a>> wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite">On Wed, 2012-10-24 at 15:05 -0700, Nathan Rutman wrote:<br><blockquote type="cite">On Oct 24, 2012, at 1:02 PM, "Gearing, Chris" <<a href="mailto:chris.gearing@intel.com">chris.gearing@intel.com</a>> wrote:<br><br><blockquote type="cite">Nathan,<br><br>I'm not 100% sure what you are proposing here, your LAD presentation suggested a 'tune-up' of the current test framework rather than a complete re-write. Which of the two are we discussing?<br></blockquote>Both...<br><blockquote type="cite"><br></blockquote>The requirements on the framework language are more relaxed, but for ease of development and developer sanity, I assume that the framework language should match the test language. So I'm using the test language as the requirements driver, and to gage community preference for that test language.<br></blockquote><br>Before embarking on building yet another new and custom framework for<br>Lustre we should evaluate some existing frameworks. For example, the<br>Autotest project was specifically designed to test the Linux kernel.<br>It's open source, looks active, is flexible, and there is detailed<br>documentation on how to write tests. Plus it was designed specifically<br>for testing the kernel so there are likely existing file system tests.<br><br> <a href="http://autotest.github.com/">http://autotest.github.com/</a><br><br> "Autotest is a framework for fully automated testing. It is<br> designed primarily to test the Linux kernel, though it is useful<br> for many other functions such as qualifying new hardware. It's an<br> open-source project under the GPL and is used and developed by a<br> number of organizations, including Google, IBM, Red Hat, and many<br> others."<br><br><blockquote type="cite">Based on the responses so far, it seems that there is a fairly clear preference for Python as a test language, and so I'll propose that Python should be used shorter-term to start replacing test-framework.<br></blockquote><br>If we decide the Autotest framework is a good fit then we'll want to<br>write the tests in python to be consistent with the framework language.<br>However, for a first cut it looks like you could use the existing bash<br>tests largely unmodified.<br><br>-- <br>Thanks,<br>Brian<br><br>_______________________________________________<br>Lustre-devel mailing list<br><a href="mailto:Lustre-devel@lists.lustre.org">Lustre-devel@lists.lustre.org</a><br><a href="http://lists.lustre.org/mailman/listinfo/lustre-devel">http://lists.lustre.org/mailman/listinfo/lustre-devel</a><br></blockquote></div><br></div></div></div><div><br></div><div><br><blockquote type="cite"><br>On 10/25/2012 12:24 PM, Nathan Rutman wrote:<br><blockquote type="cite"><br>On Oct 25, 2012, at 11:20 AM, "Gearing, Chris" <<a href="mailto:chris.gearing@intel.com">chris.gearing@intel.com</a><br><<a href="mailto:chris.gearing@intel.com">mailto:chris.gearing@intel.com</a>>> wrote:<br><br><blockquote type="cite">Hi,<br><br>>Based on the responses so far, it seems that there is a fairly clear<br>preference for Python as a test language, and so I'll propose that<br>Python should be used shorter-term to start replacing test-framework.<br><br>I dont think that is the case Nathan, because unless I am mistaken<br>Jacques-Charles, Kilian Cavalotti and if it is a binary choice today<br>my preference is Python (but please see below)<br></blockquote>Aurelien - python 2<br><br><blockquote type="cite">Bruce was neutral<br>Roman and Colin (after your email) expressed Perl as a choice.<br>So the preference would seem to be for Python, counting by people and<br>organization although obviously its a statistically suspect poll.<br></blockquote>Indeed. As I stated before, a poll is not the best way to choose the<br>language; it's just one data point. But as a) we must choose something<br>to start making any headway toward improvements, and b) bash doesn't<br>meet the requirements, I'm trying to come up with some kind of baseline.<br><br><blockquote type="cite">Chris<br><br>NOTES OF MY VOTE: Given the importance of the topic I believe a much<br>broader piece of analysis needs to be done to select the right language,<br></blockquote>I'm just trying to keep things moving from theoretical optimum to<br>practical conclusion. We feel a need today for much better Lustre<br>testing, and there's a huge pile of work waiting, and so I feel we must<br>move forward.<br><br><blockquote type="cite">also if this is for an existing framework update the tests must all be<br>in bash or all converted to the new language of choice.<br></blockquote>Short term the idea would be to translate one of the test suites, for<br>example conf-sanity, to the new language/framework. The two systems<br>(old and new) would live side-by-side for some period. New tests should<br>be implemented under the new system (e.g. sanity2). Other old tests<br>could be converted piecemeal subsequently.<br><br><br><blockquote type="cite">---------------------------------------------------------------------<br>Intel Corporation (UK) Limited<br>Registered No. 1134945 (England)<br>Registered Office: Pipers Way, Swindon SN3 1RJ<br>VAT No: 860 2173 47<br><br>This e-mail and any attachments may contain confidential material for<br>the sole use of the intended recipient(s). Any review or distribution<br>by others is strictly prohibited. If you are not the intended<br>recipient, please contact the sender and delete all copies.<br><br></blockquote><br><br><br>_______________________________________________<br>Lustre-devel mailing list<br><a href="mailto:Lustre-devel@lists.lustre.org">Lustre-devel@lists.lustre.org</a><br>http://lists.lustre.org/mailman/listinfo/lustre-devel<br><br></blockquote><br></blockquote></div><br></body></html>