Hello everybody,
in order to further develop the selenium framework [1], I need to make a few design
decisions, especially on coding conventions, which I'd like to discuss on this list,
since they affect the way of how extension- and core developers write their tests.
1) Where are the tests located? I suggest for core to put them into
maintenance/tests/selenium. That is where they are now. For extensions I propse a similar
structure, that is <extensiondir>/tests/selenium.
2) How are the tests organized? Tests are organized in testing suites. Each suite
represents a conhesive set of tests. So it is possible to have more than one test suite
per extension / core area. Test suites are technically classes. The files should follow
this naming convention: <NameOfExtension><[Subset]>TestSuite.php. The subset
is optional. For example, in the PagedTiffHandler extension, it would be
PagedTiffHandlerTestSuite.php and PagedTiffHandlerUploadsTestSuite.php. This should also
be the name of the class. Alternatively, we could use the word "Selenium"
somewhere in there in order to be able to divide between unit and selenium tests. In that
case I suggest to use PagedTiffHandlerSeleniumTestSuite.php and
PagedTiffHandlerUploadsSeleniumTestSuite.php. Hmmm... this gives pretty long names. Any
suggestions?
3) How does the framework know there are tests? The tests should be registered with the
autoloader in the extension entry file. In core, they should be registered directly with
the autoloader.
4) Which tests should be executed? Since Selenium tests are slow, not every test should be
executed in each test run. So At the moment, there is a variable $wgSeleniumTestSuites
which can be set in LocalSettings.php and which contains the tests that should be run. If
things become more dynamically (e.g. when tests should be run on svn commit), there could
be a function to add to this variable.
5) Aesthetics... There is an awful lot of "Selenium" in the class names, method
names, file names and variable names. It might be a good idea to use "Sn"
everywhere except for path names.
Two things need to be kept in mind:
* The idea is to use a similar structure for unit- and selenium tests (selenium tests are
based on unit tests anyway). I assume at some point, the tests should also be compatible
with a continuous integration server.
* The wiki that executes the selenium tests is not neccesarily the one that is being
tested if the tests run against a selenium grid.
If anybody would like to share their opinion on my suggestions, I'd be very glad!
Regards,
Markus
[1]
http://www.mediawiki.org/wiki/SeleniumFramework (documentation will be updated
soon..)