Right. I have looked at both t/ and tests/ and agree that they could use some work. But
when starting on a trip its best to walk in one direction to start. Otherwise you end up
going around in circles.
--- On Thu, 7/30/09, Chad <innocentkiller(a)gmail.com> wrote:
From: Chad <innocentkiller(a)gmail.com>
Subject: Re: [Wikitech-l] parserTests code coverage statistics
To: "Wikimedia developers" <wikitech-l(a)lists.wikimedia.org>
Date: Thursday, July 30, 2009, 8:18 AM
On Thu, Jul 30, 2009 at 10:53 AM, dan
nessett<dnessett(a)yahoo.com>
wrote:
True. However, knowing the coverage of parserTests and
knowing which code
isn't even being visited by it is the
first step in understanding where the holes are in testing.
Code coverage is a primitive metric. But, it's a place to
start.
--- On Thu, 7/30/09, Victor Vasiliev <vasilvv(a)gmail.com>
wrote:
> From: Victor Vasiliev <vasilvv(a)gmail.com>
> Subject: Re: [Wikitech-l] parserTests code
coverage statistics
> To: "Wikimedia developers"
<wikitech-l(a)lists.wikimedia.org>
> Date: Thursday, July 30, 2009, 1:28 AM
> dan nessett wrote:
> > I decided to investigate how well
parserTests
> exercises the MW code. So, I threw together a
couple of
> MacGyver tools that use xdebug's code
coverage
capability
> and analyzed the results. The results are
very,
very
> preliminary, but I thought I would get them
out so
others
> can look them over. In the next couple of
days I
hope to
> post more detailed results and the tools
themselves on the
> Mediawiki wiki. (If someone could tell me the
appropriate
> page to use that would be useful. Otherwise,
I
will just
> create a page in my own namespace).
> >
> > The statistics (again very preliminary) are:
> >
> > Number of files exercised: 141 Number of
lines
> in those files: 85606
> > Lines covered: 59489 Lines not covered:
> 26117 Percentage covered: 0.694916244188
> >
> > So, parserTests is getting (at best) about
70% code
> coverage. This is better than I expected, but
still it means
> parserTests does not test 26117 lines of
code.
What I mean
> by "at best" is xdebug just notes
whether a line
of code is
> visited. It doesn't do any logic analysis
on which
branches
> are taken. Furthermore, parserTests may not
visit
some files
> that are critical to the operation of the MW
software.
> Obviously, xdebug can only gather statistics
on
visited
> files.
> >
> > I want to emphasize that there may be errors
in these
> results due to bad assumptions on my part or
bad
coding.
> However, it is a place to start.
> >
>
> Well, they are *parser* tests, they are not
intended to
cover
Special:Version or something else.
--vvv
_______________________________________________
Wikitech-l mailing list
Wikitech-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
_______________________________________________
Wikitech-l mailing list
Wikitech-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
For more generic unit tests, check out the stuff in /t/ and
/tests/
Those could probably use improvement.
-Chad
_______________________________________________
Wikitech-l mailing list
Wikitech-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l