Max Seminik raised the issue of codurr spamming the channel with a useless list of revisions that broke the tests.
Example:
[19:39:08] <codurr> Something broke. See http://ci.tesla.usability.wikimedia.org/cruisecontrol/buildresults/mw. Possible culprits: aashrh/r89027 /r89028 /r89029 nbiabriket/r89035 krenkli/r89036 eryed/r89037 /r89038 eixal/r89039 /r89040 /r89041 /r89043 /r89044 /r89047 /r89049 /r89051 /r89061 /r89062 ... and so on.
I repaired cruise control a few days after the hackaton. At that points tests were broken and I fixed most of them. Then I added back the Database and Parser groups which added some more interesting tests breakage (including the order in which files are loaded by PHPUnit).
Since cruisecontrol *does not remember the states of tests* from previous builds, it just assume the current build broke everything and hence report all the old breakages on each build :-(
To fix them you have to get to CruiseControl: http://ci.tesla.usability.wikimedia.org/cruisecontrol/ Clicking on 'mw' in the orange box will show you tests results for the latest build, below you will find the latest revision tested.
At the moment we can see : #288 - testParserTest with data set #287
Clicking on it will show the long list of tests. Look for the above message in the list, next to it is a "Failure >>" link which show the test output. In this case:
ParserTests::testParserTest with data set #287 ('pre-save transform: Signature expansion in nowiki tags (bug 93)', 'Shall not expand: <snip parsertest output with diff and backtrace>
To reproduce it locally:
$ cd tests $ php parserTests.php --quiet --filter 'bug 93' This is MediaWiki version 1.19alpha (r89273). <snip> Passed 0 of 1 tests (0%)... 1 tests failed! $ echo $? # show PHPUnit exit code: 1 $
$ svn blame tests/parser/parserTests.txt | grep 'bug 93' 89191 platonides pre-save transform: Signature expansion in nowiki tags (bug 93) $
So probably introduced in r89191 .. Check it out and run tests :
$ svn co -r 89191 <snip> $ php parserTests.php --quiet --filter 'bug 93' This is MediaWiki version 1.19alpha (r89191). <snip> Passed 0 of 1 tests (0%)... 1 tests failed! $
That one was broken on commit anyway. http://www.mediawiki.org/wiki/Special:Code/MediaWiki/89191
The BlockTest are broken too, most probably due to its rewrite. You can easily reproduce the test suite and most project manager we love the easy report functionality (--testdox):
$ cd tests/phpunit $ ./phpunit.php --filter BlockTest --testdox PHPUnit 3.5.13 by Sebastian Bergmann.
ApiBlock [ ] Make normal block
Block [ ] Initializer functions return correct block [x] Bug 26425 block timestamp defaults to time [ ] Bug 29116 load with empty ip [ ] Bug 29116 new from target with empty ip $
Have fun :-)
Ashar,
Just a quick note thanking you for your hard, continuing work on this.
-Sumana
Ashar Voultoiz hashar+wmf@free.fr writes:
Max Seminik raised the issue of codurr spamming the channel with a useless list of revisions that broke the tests.
Ashar,
Thank you SO MUCH for tracking this down. Although I set up CruiseControl, I really haven't had a lot of time recently to track down the problems since Bugzilla has been eating up my time.
You rock,
Mark.
On 01/06/11 19:28, Ashar Voultoiz wrote:
Max Seminik raised the issue of codurr spamming the channel with a useless list of revisions that broke the tests.
<snip>
The long post is now available on mw.org :
http://www.mediawiki.org/wiki/Fixing_broken_tests
wikitech-l@lists.wikimedia.org