An automated run of parserTests.php showed the following failures:
This is MediaWiki version 1.10alpha (r19199). Reading tests from "maintenance/parserTests.txt"...
Reading tests from "extensions/Cite/citeParserTests.txt"...
Reading tests from "extensions/Poem/poemParserTests.txt"...
19 still FAILING test(s) :( * TODO: Table security: embedded pipes (http://mail.wikipedia.org/pipermail/wikitech-l/2006-April/034637.html) * TODO: Link containing double-single-quotes '' (bug 4598) * TODO: message transform: <noinclude> in transcluded template (bug 4926) * TODO: message transform: <onlyinclude> in transcluded template (bug 4926) * BUG 1887, part 2: A <math> with a thumbnail- math enabled * TODO: HTML bullet list, unclosed tags (bug 5497) * TODO: HTML ordered list, unclosed tags (bug 5497) * TODO: HTML nested bullet list, open tags (bug 5497) * TODO: HTML nested ordered list, open tags (bug 5497) * TODO: Parsing optional HTML elements (Bug 6171) * TODO: Inline HTML vs wiki block nesting * TODO: Mixing markup for italics and bold * TODO: 5 quotes, code coverage +1 line * TODO: dt/dd/dl test * TODO: Images with the "|" character in the comment * TODO: Parents of subpages, two levels up, without trailing slash or name. * TODO: Parents of subpages, two levels up, with lots of extra trailing slashes. * TODO: Don't fall for the self-closing div * TODO: Always escape literal '>' in output, not just after '<' Passed 484 of 503 tests (96.22%)... FAILED!
On Sat, Jan 13, 2007 at 12:15:59AM -0800, brion@pobox.com wrote:
Passed 484 of 503 tests (96.22%)... FAILED!
Wasn't a change made to the regression tester a while back to make that final status message less confusing?
Did it... regress? :-)
Cheer,s -- jra
On 13/01/07, Jay R. Ashworth jra@baylink.com wrote:
Wasn't a change made to the regression tester a while back to make that final status message less confusing?
Did it... regress? :-)
No, it's still there - IIRC, it was the addition of "..." to separate the statistics from the big "FAILED" part.
Rob Church
On 13/01/07, Jay R. Ashworth jra@baylink.com wrote:
Wasn't a change made to the regression tester a while back to make that final status message less confusing?
Did it... regress? :-)
No, it's still there - IIRC, it was the addition of "..." to separate the statistics from the big "FAILED" part.
Jay, how about: Passed 484 of 503 tests (96.22%)... Summary: Some tests failed. ... Versus the nirvana state: Passed 503 of 503 tests (100.00%)... Summary: ALL TESTS PASSED!
Less confusing?
All the best, Nick.
On 14/01/07, Gurch matthew.britton@btinternet.com wrote:
If it ever says that, is MediaWiki finished and everyone can go home? :)
No. It means that the parser has, in fact, escaped our control, and we should all be very very worried about the future...
Rob Church
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Rob Church wrote:
No. It means that the parser has, in fact, escaped our control, and we should all be very very worried about the future...
Or it could mean that the Parser has had it's architecture rewritten to be clean, extensible, backwards-compatible, intuitive, standards-compliant and fast.
On 14/01/07, Edward Z. Yang edwardzyang@thewritingpot.com wrote:
Or it could mean that the Parser has had it's architecture rewritten to be clean, extensible, backwards-compatible, intuitive, standards-compliant and fast.
bwaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaahaaaaaaaaaaaaaaaaaaaaaa
Rob Church
Edward Z. Yang wrote:
Or it could mean that the Parser has had it's architecture rewritten to be clean, extensible, backwards-compatible, intuitive, standards-compliant and fast.
I count two and a half more months 'til April 1st, but maybe you're on the same calendar as Brion's release notice :)
On Sun, Jan 14, 2007 at 01:11:26PM +1100, Nick Jenkins wrote:
On 13/01/07, Jay R. Ashworth jra@baylink.com wrote:
Wasn't a change made to the regression tester a while back to make that final status message less confusing?
Did it... regress? :-)
No, it's still there - IIRC, it was the addition of "..." to separate the statistics from the big "FAILED" part.
Jay, how about: Passed 484 of 503 tests (96.22%)... Summary: Some tests failed. ... Versus the nirvana state: Passed 503 of 503 tests (100.00%)... Summary: ALL TESTS PASSED!
Less confusing?
I should think that approach would be less confusing to those new potential developers everyone was so concerned about over there in the other thread.
Cheers, -- jra
Jay R. Ashworth wrote:
On Sun, Jan 14, 2007 at 01:11:26PM +1100, Nick Jenkins wrote:
Jay, how about: Passed 484 of 503 tests (96.22%)... Summary: Some tests failed. ... Versus the nirvana state: Passed 503 of 503 tests (100.00%)... Summary: ALL TESTS PASSED!
I should think that approach would be less confusing to those new potential developers everyone was so concerned about over there in the other thread.
Reminds me of the local unit test script we have at work that prints:
** ** COLOSSAL ERRORS WERE DETECTED!!! BLOCKER CRITICAL MAJOR PROBLEM VIOLATION!!! **
at the end of the run if any tests have failed. I bet it'd be red and blinking too, if it could only be made so. :-)
On 15/01/07, Ilmari Karonen nospam@vyznev.net wrote:
Reminds me of the local unit test script we have at work that prints:
** ** COLOSSAL ERRORS WERE DETECTED!!! BLOCKER CRITICAL MAJOR PROBLEM VIOLATION!!! **
at the end of the run if any tests have failed. I bet it'd be red and blinking too, if it could only be made so. :-)
Couple of klaxons in there, too...
Rob Church
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Rob Church wrote:
at the end of the run if any tests have failed. I bet it'd be red and blinking too, if it could only be made so. :-)
Couple of klaxons in there, too...
WEll, it totally depends on the culture you've made regarding unit tests. Here, it's "Oh, it's unreasonable to think that all the parsertests will pass" but in other places its "Unit tests are super important and no development goes forward until all tests pass". I myself fall into the latter camp.
On 1/15/07, Edward Z. Yang edwardzyang@thewritingpot.com wrote:
WEll, it totally depends on the culture you've made regarding unit tests. Here, it's "Oh, it's unreasonable to think that all the parsertests will pass" but in other places its "Unit tests are super important and no development goes forward until all tests pass". I myself fall into the latter camp.
In case you didn't notice, though, 18 of the 19 failing tests are actually unimplemented features, not regressions. :) Maybe it would be best if we removed the TODOs altogether. Then we would get, appropriately, passes when we have no regressions, and failures on regressions. People might actually even notice when regressions crop up! (How long has that 1887 part 2 thing been regressed?)
On Mon, Jan 15, 2007 at 11:45:13AM -0500, Simetrical wrote:
On 1/15/07, Edward Z. Yang edwardzyang@thewritingpot.com wrote:
WEll, it totally depends on the culture you've made regarding unit tests. Here, it's "Oh, it's unreasonable to think that all the parsertests will pass" but in other places its "Unit tests are super important and no development goes forward until all tests pass". I myself fall into the latter camp.
In case you didn't notice, though, 18 of the 19 failing tests are actually unimplemented features, not regressions. :) Maybe it would be best if we removed the TODOs altogether. Then we would get, appropriately, passes when we have no regressions, and failures on regressions. People might actually even notice when regressions crop up! (How long has that 1887 part 2 thing been regressed?)
I concur that this seems an excellent idea. If the terror level, er, um, error level is always greater than 0 then no one will pay any attention to it.
Cheers, -- jra
Jay R. Ashworth wrote:
On Mon, Jan 15, 2007 at 11:45:13AM -0500, Simetrical wrote:
On 1/15/07, Edward Z. Yang edwardzyang@thewritingpot.com wrote:
WEll, it totally depends on the culture you've made regarding unit tests. Here, it's "Oh, it's unreasonable to think that all the parsertests will pass" but in other places its "Unit tests are super important and no development goes forward until all tests pass". I myself fall into the latter camp.
In case you didn't notice, though, 18 of the 19 failing tests are actually unimplemented features, not regressions. :) Maybe it would be best if we removed the TODOs altogether. Then we would get, appropriately, passes when we have no regressions, and failures on regressions. People might actually even notice when regressions crop up! (How long has that 1887 part 2 thing been regressed?)
I concur that this seems an excellent idea. If the terror level, er, um, error level is always greater than 0 then no one will pay any attention to it.
There could still be a separate list of parser tests we want to pass.
Matthew Flaschen
On 1/16/07, Simetrical Simetrical+wikitech@gmail.com wrote:
In case you didn't notice, though, 18 of the 19 failing tests are actually unimplemented features, not regressions. :) Maybe it would be best if we removed the TODOs altogether. Then we would get, appropriately, passes when we have no regressions, and failures on regressions. People might actually even notice when regressions crop up! (How long has that 1887 part 2 thing been regressed?)
Wouldn't it be fairly simple to take the TODO tests out of the regular parserTests.txt and have them in another file, to be run alongside the regular tests, as with the Cite parser tests?
wikitech-l@lists.wikimedia.org