On 8/5/10 9:37 AM, Markus Glaser wrote:
Hello everybody,
in order to further develop the selenium framework [1], I need to make a few design
decisions, especially on coding conventions, which I'd like to discuss on this list,
since they affect the way of how extension- and core developers write their tests.
1) Where are the tests located? I suggest for core to put them into
maintenance/tests/selenium. That is where they are now. For extensions I propse a similar
structure, that is<extensiondir>/tests/selenium.
2) How are the tests organized? Tests are organized in testing suites. Each suite
represents a conhesive set of tests. So it is possible to have more than one test suite
per extension / core area. Test suites are technically classes. The files should follow
this naming convention:<NameOfExtension><[Subset]>TestSuite.php. The subset is
optional. For example, in the PagedTiffHandler extension, it would be
PagedTiffHandlerTestSuite.php and PagedTiffHandlerUploadsTestSuite.php. This should also
be the name of the class. Alternatively, we could use the word "Selenium"
somewhere in there in order to be able to divide between unit and selenium tests. In that
case I suggest to use PagedTiffHandlerSeleniumTestSuite.php and
PagedTiffHandlerUploadsSeleniumTestSuite.php. Hmmm... this gives pretty long names. Any
suggestions?
In PHPUnit a TestSuite is a class is a class that orchestrates the
execution of many Test classes, so PagedTiffHandlerTest would exercise
the PagedTiffHandler class, but ImagePagingHandlersTestSuite might
conduct tests for PagedTiffHandler and also PdfHandler or whatever.
3) How does the framework know there are tests? The
tests should be registered with the autoloader in the extension entry file. In core, they
should be registered directly with the autoloader.
PHPUnit uses either a TestSuite
class to determine what to run, or an
XML file. Explicitly listing each test is not neesecary to run all
tests, but TestSuite classes or XML defined suites help run a portion of
tests. I don't know if Selenium supports this kind of functionality, but
it would be nice if we could find a similar approach between the two
frameworks.
4) Which tests should be executed? Since Selenium
tests are slow, not every test should be executed in each test run. So At the moment,
there is a variable $wgSeleniumTestSuites which can be set in LocalSettings.php and which
contains the tests that should be run. If things become more dynamically (e.g. when tests
should be run on svn commit), there could be a function to add to this variable.
5) Aesthetics... There is an awful lot of "Selenium" in the class names, method
names, file names and variable names. It might be a good idea to use "Sn"
everywhere except for path names.
PHP 5.3 namespaces? (with backslashes! -- evil music --)
Two things need to be kept in mind:
* The idea is to use a similar structure for unit- and selenium tests (selenium tests are
based on unit tests anyway). I assume at some point, the tests should also be compatible
with a continuous integration server.
* The wiki that executes the selenium tests is not neccesarily the one that is being
tested if the tests run against a selenium grid.
If anybody would like to share their opinion on my suggestions, I'd be very glad!
Regards,
Markus
[1]
http://www.mediawiki.org/wiki/SeleniumFramework (documentation will be updated
soon..)
_______________________________________________
Wikitech-l mailing list
Wikitech-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l