On Thu, Jul 9, 2009 at 11:39 AM, dan nessett<dnessett(a)yahoo.com> wrote:
When I worked on Solaris at Sun in the mid-90s,
developers were required to regression test their changes before submitting them (through
a gatekeeper) for inclusion in the nightly build. Those who failed to do so and broke the
build had their hands slapped. Perhaps something similar might be established for the
mediawiki development process. Extension authors might be required to: 1) provide some
extension tests that could included in the regression test set (if their extensions ever
become important enough to do that), and 2) run their extension tests and the standard
tests against a standard regression test installation and provide evidence that there are
no problems before their extensions are included in the mediawiki extensions matrix.
Dan
Hell, we barely have unit tests for Mediawiki itself, much less the many many
extensions in SVN. I can't think of a single one, offhand.
FWIW, handling updates between versions is a mess. There are two accepted
and documented ways to apply an extension's schema updates. There needs
to be one, period. There also needs to be a cleaner Update interface so things
like this can be handled more cleanly.
It's nice and great to talk about automated regression testing of the software,
but in reality there is no clean way to do it right now. I really admire Gerard
and Kim's work on this, but it's really a hack on top of a system that should
support this stuff natively.
Regression testing should be automatic, the test cases should be standardized,
and extensions should have an easy way to add their own tests to the core
set of them. None of these are currently the case. There's a bug open about
running parserTests and/or test cases in CodeReview so we can easily and
verifiably track regressions in the software. Can't do that until the system
makes some sense to begin with :)
-Chad