On Tue, Aug 11, 2009 at 6:39 PM, dan nessettdnessett@yahoo.com wrote:
For example, consider two MW files in phase3/includes: 1) AutoLoader.php and 2) Hooks.php. In AutoLoader, the method loadAllExtensions() loads all extensions specified in $wgAutoloadClasses. It takes no parameters and has no return value. It simply walks through the entries specified in $wgAutoloadClasses and if the class specified as the key exists, executes a require of the file specified in the value. I don't see how you would specify a test of this method using the syntax of parserTests.txt.
In Hooks.php, there is a single function wfRunHooks(). It looks up hooks previously set and calls user code for them. It throws exceptions in certain error conditions and testing it requires setting a hook and seeing if it is called appropriately. I don't see how you could describe this behavior with parserTests.txt syntax.
Neither of these need to be tested directly. If AutoLoader breaks, then some other class won't load, and the tests for that class will fail. If wfRunHooks() fails, then some hook won't work, and any test of that hook will fail.
I think what's needed for decent test usage for MediaWiki is:
1) Some test suite is picked. PHPUnit is probably fine, if it runs out of the box and doesn't need some extra module to be installed.
2) The test suite is integrated into CodeReview with nag e-mails for broken tests.
3) A moderate number of tests are written for the test suite. Existing parser tests could be autoconverted, possibly. Maybe someone paid could be assigned to spend a day or two on this.
4) A new policy requires that everyone write tests for all their bug fixes and enhancements. Commits that don't add enough tests will be flagged as fixme, and reverted if not fixed.
(4) is critical here.
While we're at it, it would be nice if we instituted some other iron-clad policies. Here's a proposal:
* All new functions (including private helper functions, functions in JavaScript includes, whatever) must have function-level documentation that explains the purpose of the function and describes its parameters. The documentation must be enough that no MediaWiki developer should ever have to read the function's source code to be able to use it correctly. Exception: if a method is overridden which is already documented in the base class, it doesn't need to be documented again in the derived class, since the semantics should be the same. * All new classes must have high-level documentation that explains their purpose and structure, and how you should use them. The documentation must be sufficient that any MediaWiki developer could understand why they might want to use the class in another file, and how they could do so, without reading any of the source code. Of course, developers would still have to read the function documentation to learn how to use specific functions. There are no exceptions, but a derived class might only need very brief documentation. * All new config variables must have documentation explaining what they do in terms understandable to end-users. They must describe what values are accepted, and if the values are complicated (like associative arrays), must provide at least one example that can be copy-pasted. There are no exceptions. * If any code is changed so as to make a comment incorrect, the comment must be updated to match. There are no exceptions.
Or whatever. We have *way* too few high-level comments in our code. We have entire files -- added quite recently, mind you, by established developers -- that have no or almost no documentation on either the class or function level. We can really do better than this! If we had a clear list of requirements for comments in new code, we could start fixme'ing commits that don't have adequate comments. I think that would be enough to get people to add sufficient comments, for the most part. Without clear rules, though, backed up by the threat of reverting, I don't think the situation will improve here.
(Wow, this kind of turned into a thread hijack. :D)