I suspect that much of this has to do with the way Parser.php is eleven thousand lines of programmatic sewage, and the way the ParserTests infrastructure requires a lot of the rest of MediaWiki to be initialised in order to run the tests. As long as the rest of the system 'works' well enough to allow the parser to parse, ParserTests is happy: I wouldn't be confident to say that it "tests" anything outside of Parser.php; as you say, it only marks lines 'visited'.
--HM
"dan nessett" dnessett@yahoo.com wrote in message news:922893.25678.qm@web32507.mail.mud.yahoo.com...
I failed to mention that xdebug ignores non-executable lines of code. So, the statistics are for executable lines of code and do not include lines like comments (in either the covered or uncovered counts).
--- On Wed, 7/29/09, dan nessett dnessett@yahoo.com wrote:
From: dan nessett dnessett@yahoo.com Subject: [Wikitech-l] parserTests code coverage statistics To: wikitech-l@lists.wikimedia.org Date: Wednesday, July 29, 2009, 4:36 PM
I decided to investigate how well parserTests exercises the MW code. So, I threw together a couple of MacGyver tools that use xdebug's code coverage capability and analyzed the results. The results are very, very preliminary, but I thought I would get them out so others can look them over. In the next couple of days I hope to post more detailed results and the tools themselves on the Mediawiki wiki. (If someone could tell me the appropriate page to use that would be useful. Otherwise, I will just create a page in my own namespace).
The statistics (again very preliminary) are:
Number of files exercised: 141 Number of lines in those files: 85606 Lines covered: 59489 Lines not covered: 26117 Percentage covered: 0.694916244188
So, parserTests is getting (at best) about 70% code coverage. This is better than I expected, but still it means parserTests does not test 26117 lines of code. What I mean by "at best" is xdebug just notes whether a line of code is visited. It doesn't do any logic analysis on which branches are taken. Furthermore, parserTests may not visit some files that are critical to the operation of the MW software. Obviously, xdebug can only gather statistics on visited files.
I want to emphasize that there may be errors in these results due to bad assumptions on my part or bad coding. However, it is a place to start.
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
=