MediaWiki automated test run failure 2007-01-13

List overview All Threads
Download

newer

older

MediaWiki automated test run...

Announcement:ProcessCite Extension...

brion＠pobox.com

13 Jan 2007 13 Jan '07

8:15 a.m.

An automated run of parserTests.php showed the following failures: This is MediaWiki version 1.10alpha (r19199). Reading tests from "maintenance/parserTests.txt"... Reading tests from "extensions/Cite/citeParserTests.txt"... Reading tests from "extensions/Poem/poemParserTests.txt"... 19 still FAILING test(s) :( * TODO: Table security: embedded pipes (http://mail.wikipedia.org/pipermail/wikitech-l/2006-April/034637.html) * TODO: Link containing double-single-quotes '' (bug 4598) * TODO: message transform: <noinclude> in transcluded template (bug 4926) * TODO: message transform: <onlyinclude> in transcluded template (bug 4926) * BUG 1887, part 2: A <math> with a thumbnail- math enabled * TODO: HTML bullet list, unclosed tags (bug 5497) * TODO: HTML ordered list, unclosed tags (bug 5497) * TODO: HTML nested bullet list, open tags (bug 5497) * TODO: HTML nested ordered list, open tags (bug 5497) * TODO: Parsing optional HTML elements (Bug 6171) * TODO: Inline HTML vs wiki block nesting * TODO: Mixing markup for italics and bold * TODO: 5 quotes, code coverage +1 line * TODO: dt/dd/dl test * TODO: Images with the "|" character in the comment * TODO: Parents of subpages, two levels up, without trailing slash or name. * TODO: Parents of subpages, two levels up, with lots of extra trailing slashes. * TODO: Don't fall for the self-closing div * TODO: Always escape literal '>' in output, not just after '<' Passed 484 of 503 tests (96.22%)... FAILED!

Show replies by thread

Jay R. Ashworth

13 Jan 13 Jan

5:55 p.m.

On Sat, Jan 13, 2007 at 12:15:59AM -0800, brion(a)pobox.com wrote:

...

Passed 484 of 503 tests (96.22%)... FAILED!

Wasn't a change made to the regression tester a while back to make that final status message less confusing? Did it... regress? :-) Cheer,s -- jra -- Jay R. Ashworth jra(a)baylink.com Designer Baylink RFC 2100 Ashworth & Associates The Things I Think '87 e24 St Petersburg FL USA http://baylink.pitas.com +1 727 647 1274

Rob Church

8:08 p.m.

On 13/01/07, Jay R. Ashworth <jra(a)baylink.com> wrote:

...

Wasn't a change made to the regression tester a while back to make that final status message less confusing? Did it... regress? :-)

No, it's still there - IIRC, it was the addition of "..." to separate the statistics from the big "FAILED" part. Rob Church

Nick Jenkins

14 Jan 14 Jan

2:11 a.m.

...

On 13/01/07, Jay R. Ashworth <jra(a)baylink.com> wrote:

Wasn't a change made to the regression tester a while back to make that final status message less confusing? Did it... regress? :-)

No, it's still there - IIRC, it was the addition of "..." to separate the statistics from the big "FAILED" part.

Jay, how about: Passed 484 of 503 tests (96.22%)... Summary: Some tests failed. ... Versus the nirvana state: Passed 503 of 503 tests (100.00%)... Summary: ALL TESTS PASSED! Less confusing? All the best, Nick.

Gurch

2:33 a.m.

Nick Jenkins wrote:

...

Passed 503 of 503 tests (100.00%)... Summary: ALL TESTS PASSED!

If it ever says that, is MediaWiki finished and everyone can go home? :) -Gurch

Rob Church

2:44 a.m.

On 14/01/07, Gurch <matthew.britton(a)btinternet.com> wrote:

...

If it ever says that, is MediaWiki finished and everyone can go home? :)

No. It means that the parser has, in fact, escaped our control, and we should all be very very worried about the future... Rob Church

Edward Z. Yang

3:25 a.m.

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Rob Church wrote:

...

No. It means that the parser has, in fact, escaped our control, and we should all be very very worried about the future...

Or it could mean that the Parser has had it's architecture rewritten to be clean, extensible, backwards-compatible, intuitive, standards-compliant and fast.

...PGP SIGNATURE...

-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFFqaK7qTO+fYacSNoRAtTsAJsEFHHY2MtjaAahJhFZkJSNrKNEEQCdEQPi 8Wokki3hpD7IWqyhRYQrdzc= =vMLq -----END PGP SIGNATURE-----

Rob Church

3:28 a.m.

On 14/01/07, Edward Z. Yang <edwardzyang(a)thewritingpot.com> wrote:

...

Or it could mean that the Parser has had it's architecture rewritten to be clean, extensible, backwards-compatible, intuitive, standards-compliant and fast.

bwaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaahaaaaaaaaaaaaaaaaaaaaaa Rob Church

Ivan Krstić

3:31 a.m.

Edward Z. Yang wrote:

...

Or it could mean that the Parser has had it's architecture rewritten to be clean, extensible, backwards-compatible, intuitive, standards-compliant and fast.

I count two and a half more months 'til April 1st, but maybe you're on the same calendar as Brion's release notice :) -- Ivan Krstić <krstic(a)solarsail.hcs.harvard.edu> | GPG: 0x147C722D

Jay R. Ashworth

5:58 p.m.

On Sun, Jan 14, 2007 at 01:11:26PM +1100, Nick Jenkins wrote:

...

On 13/01/07, Jay R. Ashworth <jra(a)baylink.com> wrote:

Wasn't a change made to the regression tester a while back to make that final status message less confusing? Did it... regress? :-)

No, it's still there - IIRC, it was the addition of "..." to separate the statistics from the big "FAILED" part.

Jay, how about: Passed 484 of 503 tests (96.22%)... Summary: Some tests failed. ... Versus the nirvana state: Passed 503 of 503 tests (100.00%)... Summary: ALL TESTS PASSED! Less confusing?

I should think that approach would be less confusing to those new potential developers everyone was so concerned about over there in the other thread. Cheers, -- jra -- Jay R. Ashworth jra(a)baylink.com Designer Baylink RFC 2100 Ashworth & Associates The Things I Think '87 e24 St Petersburg FL USA http://baylink.pitas.com +1 727 647 1274

Ilmari Karonen

15 Jan 15 Jan

4:53 a.m.

Jay R. Ashworth wrote:

...

On Sun, Jan 14, 2007 at 01:11:26PM +1100, Nick Jenkins wrote:

Jay, how about: Passed 484 of 503 tests (96.22%)... Summary: Some tests failed. ... Versus the nirvana state: Passed 503 of 503 tests (100.00%)... Summary: ALL TESTS PASSED!

I should think that approach would be less confusing to those new potential developers everyone was so concerned about over there in the other thread.

Reminds me of the local unit test script we have at work that prints: ** ** COLOSSAL ERRORS WERE DETECTED!!! BLOCKER CRITICAL MAJOR PROBLEM VIOLATION!!! ** at the end of the run if any tests have failed. I bet it'd be red and blinking too, if it could only be made so. :-) -- Ilmari Karonen

Rob Church

5:02 a.m.

On 15/01/07, Ilmari Karonen <nospam(a)vyznev.net> wrote:

...

Couple of klaxons in there, too... Rob Church

Edward Z. Yang

2:46 p.m.

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Rob Church wrote:

...

at the end of the run if any tests have failed. I bet it'd be red and blinking too, if it could only be made so. :-)

Couple of klaxons in there, too...

WEll, it totally depends on the culture you've made regarding unit tests. Here, it's "Oh, it's unreasonable to think that all the parsertests will pass" but in other places its "Unit tests are super important and no development goes forward until all tests pass". I myself fall into the latter camp.

...PGP SIGNATURE...

-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFFq5O0qTO+fYacSNoRAjxvAKCCnWhWfsEzdCCsdgoYIEarZmZh8wCfdOLj oEBA3gJOjjRNGjnyDq2+mEU= =c1dq -----END PGP SIGNATURE-----

Simetrical

4:45 p.m.

On 1/15/07, Edward Z. Yang <edwardzyang(a)thewritingpot.com> wrote:

...

In case you didn't notice, though, 18 of the 19 failing tests are actually unimplemented features, not regressions. :) Maybe it would be best if we removed the TODOs altogether. Then we would get, appropriately, passes when we have no regressions, and failures on regressions. People might actually even notice when regressions crop up! (How long has that 1887 part 2 thing been regressed?)

Jay R. Ashworth

17 Jan 17 Jan

5:34 p.m.

On Mon, Jan 15, 2007 at 11:45:13AM -0500, Simetrical wrote:

...

On 1/15/07, Edward Z. Yang <edwardzyang(a)thewritingpot.com> wrote:

I concur that this seems an excellent idea. If the terror level, er, um, error level is always greater than 0 then no one will pay any attention to it. Cheers, -- jra -- Jay R. Ashworth jra(a)baylink.com Designer Baylink RFC 2100 Ashworth & Associates The Things I Think '87 e24 St Petersburg FL USA http://baylink.pitas.com +1 727 647 1274

Matthew Flaschen

25 Feb 25 Feb

1:43 a.m.

Jay R. Ashworth wrote:

...

On Mon, Jan 15, 2007 at 11:45:13AM -0500, Simetrical wrote:

On 1/15/07, Edward Z. Yang <edwardzyang(a)thewritingpot.com> wrote:

I concur that this seems an excellent idea. If the terror level, er, um, error level is always greater than 0 then no one will pay any attention to it.

There could still be a separate list of parser tests we want to pass. Matthew Flaschen

Stephen Bain

2:16 a.m.

On 1/16/07, Simetrical <Simetrical+wikitech(a)gmail.com> wrote:

...

Wouldn't it be fairly simple to take the TODO tests out of the regular parserTests.txt and have them in another file, to be run alongside the regular tests, as with the Cite parser tests? -- Stephen Bain stephen.bain(a)gmail.com

6263

days inactive

6306

days old

wikitech-l@lists.wikimedia.org

Manage subscription

16 comments

11 participants

tags (0)

participants (11)

brion＠pobox.com
Edward Z. Yang
Gurch
Ilmari Karonen
Ivan Krstić
Jay R. Ashworth
Matthew Flaschen
Nick Jenkins
Rob Church
Simetrical
Stephen Bain