On Thu, 05 Aug 2010 18:47:58 -0400, Mark A. Hershberger wrote:
Markus Glaser <glaser(a)hallowelt.biz> writes:
1) Where are the tests located? I suggest for
core to put them into
maintenance/tests/selenium. That is where they are now. For extensions
I propse a similar structure, that is <extensiondir>/tests/selenium.
Sounds fine.
In the same way, since maintenance/tests contains tests that should be
run using PHPUnit, we can say that <extensiondir>/tests will contain
tests that should be run using PHPUnit.
Alternatively, we could use the word
"Selenium" somewhere in there in
order to be able to divide between unit and selenium tests.
I think putting them in the selenium directory (or the “Se” directory)
is sufficient.
3) How does the framework know there are tests?
Can I suggest that the framework can see that an extension has tests
simply by the presence of the <extensiondir>/tests directory containing
a <Extension>*TestSuite.php file?
The <extensiondir>/tests/<Extension>TestSuite.php file should define a
class using the name <Extension>TestSuite which has a static method
suite(). See the PHPUnit documentation at
http://bit.ly/b9L50r for how
this is set up.
This static suite() method should take care of letting the autoloader
know about any test classes so the test classes are only available
during testing.
So, for your example using PagedTiffHandler, there would be the files:
PagedTiffHandler/tests/PagedTiffHandlerTestSuite.php
PagedTiffHandler/tests/PagedTiffHandlerUploadsTestSuite.php
4) Which tests should be executed?
By default all the test suites in <extensiondir>/tests should be run.
It is should be possible to specify which particular test to run by
using whatever command line arguments to the CLI.
This seems better to me than defining a new global. If some tests
should only be run rarely, that information can be put in the TestSuite
class for te extension.
In this way, I think it is possible to remove all the $wgSelenium*
variables from the DefaultSettings.php file.
(I plan to do something similar with the $wgParserTest* variables as
well — these sorts of things don't seem like they belong in Core.)
Mark.
I don't think walking through all the extensions looking for test
subdirectories and then running all tests therein is a good idea. First,
in a large installation with many extensions, this takes time and delays
the test execution. Second, if a developer is working on a particular
extension or part of the core, it will be common to run the tests
associated with that for regression purposes. Making the developer
specify the extension or core tests to run on the RunSeleniumTests
command line is irritating (at least, it would irritate me). Specifying
the test suite(s) to be run in LocalSettings.php is a set and forget
approach that allows the developer to get on with the work.
However, I do agree that the number of global variables associated with
the selenium framework is getting large and has the potential of growing
over time. One solution is to use a multi-dimensional associative array
(much like $wgGroupPermissions). We could use a global variable
$wgSelenium and move all selenium framework values into it. For example:
$wgSelenium['wiki']['host'] = 'localhost';
$wgSelenium['wiki']['wikiurl'] = false;
$wgSelenium['wiki']['loginname'] = 'Wikiuser;
$wgSelenium['wiki']['password'] = '';
$wgSelenium['server']['port'] = 4444;
etc.
The only global we may wish to keep separate is $wgEnableSelenium, since
it specifies whether $wgSelenium is used.
--
-- Dan Nessett