On 05/02/2010 01:20 AM, Markus Glaser wrote: framework for MediaWiki. Since it has now been promoted to maintenance/tests, I have provided some initial information it on http://www.mediawiki.org/wiki/SeleniumFramework . I would be very happy about comments and ideas for further improvement. Also, if you intend to use the framework for your tests, please let me know. I will be happy to assist. It's a good leap in the right direction, a few comments: I spent about half an hour trying to get PagedDiffHandler tests to run, (though some uploads still seem to "fail" by uploading successfully): * Please distribute all the images needed with the extension, I followed the link in SOURCES.txt, and found that the link on that page was broken (though it wasn't hard to find a fixed link). * Don't hardcode "\\" as the directory separator - windows is not here! * Use parser tests instead of selenium where appropriate (this will save you "lots" of typing and a bit of test execution time) * Make the tests pass by default :p. Comments about the framework itself: * Don't hardcode the firefox path - selenium's guess is better than yours. * Should always shutdown selenium properly (I was often left with myriads of firefoxes) * !Always run against a clean database! (it is unrealistic to expect people to write perfect tests, and particularly while developing the tests having an automatically clean database is essential). * Get configuration from the main LocalSettings.php and use a mechanism like parser tests for registering selenium test files. (making it as easy as possible to run tests is a good idea). <waffle> How to deal with configuration (and by extension databases) for selenium tests... It is a tricky question, and to solve it properly requires a bit of effort. I think you need to send a selenium token with each request (like you suggested, though it'd be nicer to use a cookie rather than a url parameter in my opinion). Either you could do a log in and get a session id, or you could simply update the cookie on every request with: (selenium_test_name, timestamp, start_of_test, HASH(selenium_test_name, timestamp, shared_secret)) Providing mediawiki is in selenium mode, and the timestamp is recent, the selenium_test_name matches a registered selenium test and the hash is correct, then we can be reasonably sure this is a valid selenium request (I assume no-one steals the secret, much as you assume no-one reads wikisysop's password out of the file - besides you can always update it, and it's only a problem if selenium mode is enabled). If timestamp == start_of_test then you can clean the test databases/ upload directories (or do that on "log in"). You can then also load any test-specific configuration from the selenium_test_name test on every request (and change cache-prefixes and upload directories to avoid collisions). If you allow for one database (or pseudo database with table prefixes, like parser test) per selenium_test_name then you can run different tests in parallel; otherwise you could generate a new prefix based on the start_of_test and the selenium_test_name - but you'd quickly need a way to delete the databases completely once you'd finished.</waffle> Thanks for your great work so far! Conrad _______________________________________________ Wikitech-l mailing list Wikitech-l(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l

* Don't hardcode "\\" as the directory separator - windows is not here!

During the meeting last Friday, someone (I sorry, I don't remember who) mentioned he had created a test that runs with the currently checked in selenium code. Is that test code available somewhere (it doesn't appear to be in the current revision)?