Test Runner - History

When sproutware first began unit testing (being on windows at the time) there was some anomocity towards running tests in a cmd.exe session.  Mostly due to the lack of experience, and the percentage complete on a large project much code was written and little of it was tested.  So we created what you would call 'legacy code' because as soon as it left the editor it was no longer maintainable in an automatic fashion.

Well, we'd like to think we learn from our mistakes so we decided to take another stab at keeping up with unit testing with another project.  Due to our issues with terminals and manual test running there was an effort to create an in browser runner for simpletest.  This worked out pretty well, there were a couple problems which kept us from getting 'excited' about running tests.  The benefits were obvious, but simpletest seemed more like a game as the 'success' rate was calculated at the assert level instead of the test case level.  We like to think of this is as incentive to get high points.  On the other hand this incarnation of the test runner was refresh based and, to much of our pain, was in ways even less efficient than using the terminal to run tests.

Eventually we found PHPUnit and with all of its nicities, reporting, profiling, code coverage, etc..  things became exciting.  It was determined that tests were run in a more rigid form and actually made more sense than simpletest.  After using phpunit with the refresh based test runner (with all of its inefficiencies) we noticed an old blog post [ http://sebastian-bergmann.de/archives/639-AJAX-Based-Test-Runner-for-PHPUnit.html ] which struck some curiousity.  We began some experimenting and a couple weeks later had a prototype which could effectively run tests inside of a 'sandbox' as mentioned in the blog post comments.  After that work was done to improve on the runner both in efficiency of code and coding.  Eventually we arrived at a 0.5 version which is considered 'generally stable and functional'.