Hello,
Ori and I attempted to run phpunit tests with all extensions deployed on the wmf cluster. With the help of Sam Reed we fixed must of the issues but there is one nasty one remaining.
The parser tests expectations can not be meet when one mix extensions together. For example the Cite tests have some images/thumbnails, when one installs the MultimediaViewer extension, the test file because the generated output adds data-file-width="1941" data-file-height="220" to the <img> elements.
That is due to MultimediaViewer registering the parser hook ThumbnailBeforeProduceHTML . And that is legit.
I am looking for ideas to properly fix extensions altering output and thus breaking other extensions parser tests.
A lame idea would be to have the Cite parser tests to unregister any hook altering HTML which are not registered by Cite. We could maybe come up with another repositories that has a different set of parsertests suitable for multiple extensions.
There is a few more extensions having a similar issue such as ProofReadpage inserting class="prp-pagequality-1 in title links.
Thoughts?
== References ==
A bug for MultimediaViewer / Cite has some more details: https://bugzilla.wikimedia.org/show_bug.cgi?id=67302
We have a tracking bug to get all extensions pass unit tests together: https://bugzilla.wikimedia.org/show_bug.cgi?id=67216
An experimental run is:
console: https://integration.wikimedia.org/ci/job/mediawiki-core-extensions-integrati...
test report which highlight other issues: https://integration.wikimedia.org/ci/job/mediawiki-core-extensions-integrati...
On Wed, Jul 9, 2014 at 7:54 AM, Antoine Musso hashar+wmf@free.fr wrote:
The parser tests expectations can not be meet when one mix extensions together. For example the Cite tests have some images/thumbnails, when one installs the MultimediaViewer extension, the test file because the generated output adds data-file-width="1941" data-file-height="220" to the <img> elements.
That is due to MultimediaViewer registering the parser hook ThumbnailBeforeProduceHTML . And that is legit.
I am looking for ideas to properly fix extensions altering output and thus breaking other extensions parser tests.
A lame idea would be to have the Cite parser tests to unregister any hook altering HTML which are not registered by Cite. We could maybe come up with another repositories that has a different set of parsertests suitable for multiple extensions
What is the goal of running a test with a bunch of unrelated hooks installed? Speed/ops convenience? Reusing unit tests as integration tests to check for inter-extension conflicts?
If it's speed, just don't do it, IMO. Repeating the PHP bootstrap process once per extension can't be that much of an overhead.
If the goal is to test whether extensions interfere with each other, then having the test artificially unregister other extensions makes utterly no sense to me. Instead, the tests should be changed so that they don't make assumptions about parts of the output which is not under their control.
Ideally, the tests would also compare expected output intelligently -- IIRC the parser tests still just use a fairly simple string comparison and even things like whitespace or attribute order that make no effect on the behavior of the HTML output get marked as differences. (This is also why we have to override things like $wgArticlePath and $wgUploadPath during the tests, as the output we compare against is hardcoded with expected values.)
Doing the checks on the DOM level, and being able to register additional or modified checks, might make sense here.
For instance Cite's test for images is just to make sure they go through intact; it doesn't actually care about which HTML attributes are there. If instead of comparing a raw chunk of output HTML the comparison used a programmatic check on the HTML DOM that "image is present and refers to the right link", then there wouldn't be an issue... and potentially the "image is present" check could be overridden by MultimediaViewer to also check for the data-file-width and data-file-height attributes to make sure they don't fall off in some circumstances.
But that's not going to be an easy transition, as it requires rewriting the check portion of a bunch of test cases. :)
-- brion
On Wed, Jul 9, 2014 at 10:57 AM, Gergo Tisza gtisza@wikimedia.org wrote:
On Wed, Jul 9, 2014 at 7:54 AM, Antoine Musso hashar+wmf@free.fr wrote:
The parser tests expectations can not be meet when one mix extensions together. For example the Cite tests have some images/thumbnails, when one installs the MultimediaViewer extension, the test file because the generated output adds data-file-width="1941" data-file-height="220" to the <img> elements.
That is due to MultimediaViewer registering the parser hook ThumbnailBeforeProduceHTML . And that is legit.
I am looking for ideas to properly fix extensions altering output and thus breaking other extensions parser tests.
A lame idea would be to have the Cite parser tests to unregister any hook altering HTML which are not registered by Cite. We could maybe come up with another repositories that has a different set of parsertests suitable for multiple extensions
What is the goal of running a test with a bunch of unrelated hooks installed? Speed/ops convenience? Reusing unit tests as integration tests to check for inter-extension conflicts?
If it's speed, just don't do it, IMO. Repeating the PHP bootstrap process once per extension can't be that much of an overhead.
If the goal is to test whether extensions interfere with each other, then having the test artificially unregister other extensions makes utterly no sense to me. Instead, the tests should be changed so that they don't make assumptions about parts of the output which is not under their control. _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
wikitech-l@lists.wikimedia.org