--- On Tue, 8/11/09, Alexandre Emsenhuber alex.emsenhuber@bluewin.ch wrote:
My idea is the move the "backend" of ParserTest (parserTests.txt file processing, result reporting, ...) and the TestRecorder stuff to something like a MediaWikiTests class that extends Maintenance and move the rest in a file in /maintenance/tests/ (to be created) and re- use the backend to have files that have the same format, but test's input could be raw PHP code (a bit like PHP core's tests) with a new config variable that's like $wgParserTestFiles but for these kind of test. This mostly concerns the actual tests in /tests/ and /t/inc/). We can also port cdb-test.php, fuzz-tester.php, preprocessorFuzzTest.php and syntaxChecker.php to this new system and then create a script in /maintenance/ that runs all the tests in / maintenance/tests/. This allows to also upload all the tests to CodeReview, not only the parser tests. A benefit is that we can get ride of /tests/ and /t/.
One of the beauties of open source code development is he who does the work wins the prize. Of course, I am sure senior developers have discretionary power on what goes into a release and what does not. But, if you want to do the work, go for it (says the guy [me] who just joined the group).
However, I think you should consider the following:
* parserTests is designed to test parsing, which is predominantly a text manipulation task. Other parts of MW do not necessarily provide text processing markers that can be used to decide whether they are working correctly or not.
* Sometimes testing the action of a module requires determining whether a series of actions provide the correct behavior. As far as I am aware, parserTests has no facility to tie together a set of actions into a single test.
For example, consider two MW files in phase3/includes: 1) AutoLoader.php and 2) Hooks.php. In AutoLoader, the method loadAllExtensions() loads all extensions specified in $wgAutoloadClasses. It takes no parameters and has no return value. It simply walks through the entries specified in $wgAutoloadClasses and if the class specified as the key exists, executes a require of the file specified in the value. I don't see how you would specify a test of this method using the syntax of parserTests.txt.
In Hooks.php, there is a single function wfRunHooks(). It looks up hooks previously set and calls user code for them. It throws exceptions in certain error conditions and testing it requires setting a hook and seeing if it is called appropriately. I don't see how you could describe this behavior with parserTests.txt syntax.
Of course, you could create new syntax and behavior for the parserTests software components, but that is a lot of work that other infrastructure has already done. For example, see the set of assertions for phpunit (http://www.phpunit.de/manual/2.3/en/api.html#api.assert.tables.assertions).
Dan