On Sat, Feb 6, 2010 at 16:27, Chad innocentkiller@gmail.com wrote:
On Sat, Feb 6, 2010 at 11:24 AM, Ævar Arnfjörð Bjarmason avarab@gmail.com wrote:
On Sat, Feb 6, 2010 at 01:04, Ævar Arnfjörð Bjarmason avarab@gmail.com wrote:
Since the tests were ported from t/ to phpunit's phase3/maintenance/tests/ in r61938 and other commits running the tests on my machine takes up to 1GB of memory and grows as it runs more tests. It seems that phpunit uses the same instance of the php interpreter for running all the tests.
Is there some way around this? Perhaps phpunit.xml could be tweaked so that it runs a new php for each test?
Furthermore when I run `make test' I get this:
Time: 03:35, Memory: 1849.25Mb
There were 2 failures:
1) LanguageConverterTest::testGetPreferredVariantUserOption Failed asserting that two strings are equal. --- Expected +++ Actual @@ @@ -tg-latn +tg
/home/avar/src/mw/trunk/phase3/maintenance/tests/LanguageConverterTest.php:82
2) Warning No tests found in class "ParserUnitTest".
FAILURES! Tests: 686, Assertions: 3431, Failures: 2, Incomplete: 34
But when I run phpunit manually on the test then all tests pass:
$ phpunit LanguageConverterTest.php PHPUnit 3.4.5 by Sebastian Bergmann.
.........
Time: 23 seconds, Memory: 23.75Mb
OK (9 tests, 34 assertions)
Also after I get "Tests: 686, Assertions: 3431, Failures: 2, Incomplete: 34" in the first output phpunit doesn't exit and continues hugging my memory. Why is it still running? It has already run all the tests.
I've worked around this by adding a 'make tap' target which runs the phpunit tests individually with Test::Harness. I made it the default target due to the problems with running all the tests at once with phpunit:
http://www.mediawiki.org/wiki/Special:Code/MediaWiki/62071 http://www.mediawiki.org/wiki/Special:Code/MediaWiki/62072
Does something run these tests or the parsertests automatically? It would be really neat to test all svn revisions of MediaWiki and report the results on Special:Code. I think I read somewhere that something runs the parsertests automatically.
It's supposed to be running the parser tests and uploading them on commit, but that's been broken for a little while now.
What system is this that's running automatic tests on commits? I was investigating setting up a buildbot (http://buildbot.net/) which could have multiple test clients and report tests to IRC/XML which Special:Code could then use.
What does the now-broken Special:Code test system use?
If we're got a nice standard output from the tests (I think the XML is pretty suited for this), we should be able to upload that result to Code Review.
$ prove -j 10 -e 'phpunit --tap' -Q *Test*.php All tests successful. Files=20, Tests=692, 31 wallclock secs ( 0.34 usr 0.21 sys + 18443939634.30 cusr 2803481.20 csys = 18446743116.05 CPU) Result: PASS
You can get pretty HTML like this:
$ prove --formatter TAP::Formatter::HTML -j 10 -e 'phpunit --tap' -Q *Test*.php > ~/www/mw-tap-out.html
Which gives you something like this:
http://v.nix.is/~avar/mw-tap-out.html
That can be parsed with any XML parser that just has to look for <td class="results"> and <div id="summary" class="passed"> or <div id="summary" class="failed">