Hi Gerard,
I am very interested in the tool you mention. Let's keep the discussion on list, since
I suspect there are others who might want to set up a regression test environment either
now or later.
Can you provide some pointers how to use this tool? Is it described in the standard SVN
documentation or is that located somewhere else? What is its name? Would its use allow
testing against the most recent version in trunk?
My initial thoughts for such a regression test installation are:
I think some extensions involve database schema changes. My initial idea is to create a
new installation, make all of the schema changes necessary for any extensions in the
regression test set and then dump the database. This dump could then be used to set up new
regression test installations (or reinitialize existing test installations). Perhaps the
dump could be placed in Subversion in a "test" area.
Just loading a set of extensions in an installation doesn't really provide much in the
way of verifying they work together. The test needs to use them together. One way to do
this would be to create a set of pages that concurrently access the extensions when the
pages are rendered. Knowing which extensions to use together is a key to this approach.
Perhaps there is information in Bugzilla that would help determine that. Also, extension
authors might provide some tests that could be incorporated into the test set. If you or
others have some ideas on how to test extensions that would be very helpful.
When I worked on Solaris at Sun in the mid-90s, developers were required to regression
test their changes before submitting them (through a gatekeeper) for inclusion in the
nightly build. Those who failed to do so and broke the build had their hands slapped.
Perhaps something similar might be established for the mediawiki development process.
Extension authors might be required to: 1) provide some extension tests that could
included in the regression test set (if their extensions ever become important enough to
do that), and 2) run their extension tests and the standard tests against a standard
regression test installation and provide evidence that there are no problems before their
extensions are included in the mediawiki extensions matrix.
Dan
--- On Wed, 7/8/09, Gerard Meijssen <gerard.meijssen(a)gmail.com> wrote:
From: Gerard Meijssen
<gerard.meijssen(a)gmail.com>
Subject: Re: [Wikitech-l] Defining a configuration for regression testing
To: "Wikimedia developers" <wikitech-l(a)lists.wikimedia.org>
Cc: "Kim Bruning" <kim(a)bruning.xs4all.nl>
Date: Wednesday, July 8, 2009, 10:28 PM
Hoi.
In Subversion there is a tool created for the setup of
environments. What it
does well is setup an environment with a specific
configuration. This
configuration for a specific environment can be found in a
file. In this way
it is possible to define a specific revision or tag for
either MediaWiki
itself or for an extension. The software is such that you
can specify
multiple languages for an environment.. As there are
important differences
because of the language, the script you want to be able to
test for a key
subset of wikis. Duplication of an initial created wiki
works well.
When you look at ALL the extensions used whereever on WMF
projects, you will
find that they are not an homongous bunch; they are not
used together. This
means that you may want to have multiple environments
configured. At this
time there is a Wikipeida configuration and a Usability
Initiative
configuration. Given that the configuration is in a file,
there is room to
indicate a specific revision..
As you can imagine, there are scripts to install particular
extensions that
can not be installed in a default way.
When you have an interest, contact me or ask on this list.
Thanks,
GerardM
2009/7/9 dan nessett <dnessett(a)yahoo.com>
I am setting up a testing environment for mediawiki
and the first thing
that came to mind is testing new extensions
against a
"regression test
configuration". That raises the question of
what
should constitute such a
configuration. One issue is which extensions
should be
loaded.
There are over 2000 extensions in the mediawiki
extensions matrix and 512
stable extensions. It would be impractical to run
a
configuration with all
of either class. So, I asked around and received
a
suggestion that at the
very least the extensions on the wikimedia
servers
should be loaded. I went
CommonSettings.php. From it
I extracted 75 extensions that are used on
wikimedia's
servers. I list these
below.
A question for readers of this list is: should a
regression test
configuration load only these extensions or
should it
load others? Another
question is: what other settings should define a
regression test
configuration.
Wikimedia installed extensions:
Timeline, wikihiero, SiteMatrix, CharInsert,
CheckUser,
SpecialMakesysop, Makebot, ParserFunctions, Cite,
InputBox,
ExpandTemplates, ImageMap, SyntaxHighlight_GeSHi,
DoubleWiki, Poem,
PovWatch, AjaxTest, UnicodeConverter,
CategoryTree,
ProofreadPage, lst,
SpamBlacklist, UploadBlacklist, TitleBlacklist,
Quiz,
Gadgets,
OggHandler, AssertEdit, FormPreloadPostCache,
SkinPerPage, Schulenburg,
Tomas, ContributionReporting,
ContributionTracking,
ContactPage,
ExtensionDistributor, GlobalBlocking, TrustedXFF,
ContactPage,
SecurePoll, OAIRepo, DynamicPageList, Nogomatch,
SpecialCrossNamespaceLinks, SpecialRenameuser,
SpecialNuke, AntiBot,
TorBlock, CookieBlock, ScanSet, SpecialCite,
FixedImage, UserThrottle,
ConfirmEdit, FancyCaptcha, HideRevision,
AntiSpoof,
CentralAuth,
DismissableSiteNotice, UsernameBlacklist,
MiniDonation, CentralNotice,
TitleKey, WikimediaMessages, SimpleAntiSpam,
Collection, NewUserMessage,
CodeReview, Drafts, Configure, AbuseFilter,
ClientSide, CommunityVoice,
PdfHandler, UsabilityInitiative
Regards,
Dan Nessett
_______________________________________________
Wikitech-l mailing list
Wikitech-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
_______________________________________________
Wikitech-l mailing list
Wikitech-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l