Parser tests just set a bunch of configuration variables to particular values when they start. From maintenance/parserTests.inc, line 507 ff.:
$settings = array( 'wgServer' => 'http://localhost', 'wgScript' => '/index.php', 'wgScriptPath' => '/',
etc. This works fine, although when a new config variable is introduced, it might have to have a value added to the array if it causes test failures with some settings.
It would be good if all testing frameworks were fairly consistent. I believe a project is going to be set up to coordinate this.
I'm a little surprised by this altogether, actually. I had seen some mention of Selenium adoption, but nothing concrete. Was this discussed publicly anywhere, or has this kind of project started to migrate behind closed doors now that there are so many paid people? I'm subscribed to all major MediaWiki development discussion fora, as far as I know, but I've felt increasingly out of the loop lately. Having more paid developers is great (and so is a good testing framework!), but that shouldn't mean that decisions are made where volunteers can't weigh in.
As Roan mentioned, we are primarily setting up a selenium grid architecture up for doing testing for usability initiative. The long term goal for this is to integrate with code review, and to have a general selenium test suite for MediaWiki. We are in the initial stages for planning how selenium will fit in with the MediaWiki development process, and we welcome all input. If you, or anyone, want to be included in the planning process, let us know.
I was asking if we had a planning page so that we can keep this public and get input from all of the developers.
Here's the small of amount of documentation we currently have:
http://usability.wikimedia.org/wiki/Resources#Interaction_testing_automation
http://usability.wikimedia.org/wiki/File:Selenium_architecture_diagram.svg
Respectfully,
Ryan Lane