On Fri, Oct 18, 2013 at 8:23 AM, Brad Jorsch (Anomie) <bjorsch(a)wikimedia.org > On Fri, Oct 18, 2013 at 5:52 AM, Antoine Musso <hashar+wmf(a)free.fr> > > In June I enforced a PHPUnit feature which force us to mention which > > MediaWiki function is covered by a test method [FORCE COVER]. > > Why is this desirable? > > In my experience this is desirable in order to only mark the particular code you are testing as covered. So for example if you are testing database functions, you don't want it to mark wfRunHooks and friends as "covered" when it just happened to be called but wasn't specifically tested.

different definitions of test ;-) code touched seems like a much less useful metric than code specifically tested, but i could be convinced otherwise. On Fri, Oct 18, 2013 at 9:31 AM, Brad Jorsch (Anomie) &lt;bjorsch(a)wikimedia.org > On Fri, Oct 18, 2013 at 12:13 PM, Erik Bernhardson > &lt;ebernhardson(a)wikimedia.org>> On Fri, Oct 18, 2013 at 8:23 AM, Brad Jorsch (Anomie) < > bjorsch(a)wikimedia.org >>>> >>> On Fri, Oct 18, 2013 at 5:52 AM, Antoine Musso &lt;hashar+wmf(a)free.fr&gt; >>> >>> >>> In June I enforced a PHPUnit feature which force us to mention which >>>> MediaWiki function is covered by a test method [FORCE COVER]. >>> >>> Why is this desirable? >>> >>> >> In my experience this is desirable in order to only mark the particular >> code you are testing as covered. So for example if you are testing >> database functions, you don't want it to mark wfRunHooks and friends as >> "covered" when it just happened to be called but wasn't specifically > tested. > > On the other hand, you *did* test part of wfRunHooks, even if indirectly. > > > -- > Brad Jorsch (Anomie) > Software Engineer > Wikimedia Foundation > > _______________________________________________ > Wikitech-l mailing list > Wikitech-l(a)lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/wikitech-l > _______________________________________________ Wikitech-l mailing list Wikitech-l(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l

The whole line. This is a limitation of the PHP extension Xdebug. See http://phpunit.de/manual/current/en/code-coverage-analysis.html#code-covera… Based on my experience, there will also be some false negatives.

This is very cool, apart from the numbers of course. Is it hard to setup for extensions as well? Lots of development happens in extensions nowdays. I read from the linked bug that it is currently not possible due to bug in PHP.

Le 18/10/13 18:42, Niklas Laxström a écrit : I haven't looked at how to do it for extensions. All the core hooks and methods should ideally be covered by the core test suite itself, so we could discard them in the report. phpunit configuration has a <filter> which would let us get rid of core (aka includes, languages..): http://phpunit.de/manual/3.7/en/appendixes.configuration.html#appendixes.co… (yeah long url sorry) -- Antoine "hashar" Musso _______________________________________________ Wikitech-l mailing list Wikitech-l(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Hey, Thanks for setting this up hashar! It's a very useful metric to have. I'd like to bring some attention to the "dashboard" link at the top of coverage pages, which leads to for instance https://integration.wikimedia.org/cover/mediawiki-core/master/php/index.das… This page is very useful in finding the things most in need of attention. It also lists classes by complexity, in this particular case making it very clear that classes such as Parser are doing way to much stuff.

<s>watches the watchers</s> tests the tests?