--- On Tue, 8/11/09, Chad <innocentkiller(a)gmail.com> wrote:
To be perfectly honest, I'm of the opinion that
tests/ and
t/
should be scrapped and it should all be done over,
properly.
What we need is an easy and straightforward way to write
test cases, so people are encouraged to write them. Right
now, nobody understands wtf is going on in tests/ and t/,
so
they get ignored and the /vast/ majority of the code isn't
tested.
What we need is something similar to parser tests, where
it's
absurdly easy to pop new tests in with little to no coding
required at all. Also, extensions having the ability to
inject
their own tests into the framework is a must IMHO.
There is a way to easily add tests, but it requires some community discipline. phpunit has
a --skeleton command (actually two variations of it) that automatically generates unit
tests. (see
http://www.phpunit.de/manual/current/en/skeleton-generator.html). All
developers have to do is add assertions (which have the appearance of comments) to their
code and call phpunit with the --skeleton flag. If you want even more hand holding,
Netbeans will do it for you.
This is all wonderful, but there are problems:
* Who is going to go back and create all of the assertions for existing code? Not me (at
least not alone). This is too big a job for one person. In order for this to work, you
need buy in from at least a reasonable number of developers. So far, the number of
developers expressing an interest in code quality and testing is pretty small.
* What motivation is there for those creating new code to do the work to add assertions
with good code coverage? So far I haven't seen anything in the MW code development
process that would encourage a developer to do this. Without some carrots (and maybe a few
sticks) this approach has failure written all over it.
* Even if we get a bunch of Unit tests, how are they integrated into a useful whole? That
requires some decisions on test infrastructure. This thread begins the discussion on that,
but it really needs a lot more.
* MW has a culture problem. Up to this point people just sling code into trunk and think
they are done. As far as I can tell, very few feel they have any responsibility for
ensuring their code won't break the product. [Perhaps I am being unkind on this.
Without any testing tools available, it is quite possible that developers want to ensure
the quality of their code, but don't have the means of doing so.]
I realize these observations may make me unpopular. However, someone has to make them. If
everyone just gets mad, it doesn't solve the problem. It just pushes it out to a time
when it is even more serious.
Dan