If we are considering automated testing platforms, pushTotesthttp://www.pushtotest.com/index.phpis a good platform to consider. They come highly recommended by Tibco and is in use in many large scale corporations (including the one I work at). They have both community and enterprise editions around, which are fully functional.
In a nutshell, it can integrate many, like selenium, test4gen, soapui etc etc. The tool can take a unit test script in selenium and turn it into functional, load and service monitoring tests in very easy steps. I was a part of the initial evaluation team at my work place and found the tool to be quite useful. We had an opportunity to work with their core team and they were quite helpful as well. This works well with RIA, Cloud etc too. The way we plan to use it is to have the developers create the selenium unit test at the development stage so that we can re-use the same script for the following stages (functional test and regression testing) of development.
Just my two cents.
Regards, Jyothis.
http://ml.wikipedia.org/wiki/User:Jyothis http://meta.wikimedia.org/wiki/User:Jyothis I am the first customer of http://www.netdotnet.com
woods are lovely dark and deep, but i have promises to keep and miles to go before i sleep and lines to go before I press sleep
completion date = (start date + ((estimated effort x 3.1415926) / resources) + ((total coffee breaks x 0.25) / 24)) + Effort in meetings
On Sat, May 1, 2010 at 10:34 PM, Conrad Irwin conrad.irwin@googlemail.comwrote:
On 05/02/2010 01:20 AM, Markus Glaser wrote:
Hi everybody,
at the Wkimedia Developers' Workshop, I introduced a Selenium testing
framework for MediaWiki. Since it has now been promoted to maintenance/tests, I have provided some initial information it on http://www.mediawiki.org/wiki/SeleniumFramework . I would be very happy about comments and ideas for further improvement. Also, if you intend to use the framework for your tests, please let me know. I will be happy to assist.
Regards, Markus Glaser
It's a good leap in the right direction, a few comments:
I spent about half an hour trying to get PagedDiffHandler tests to run, (though some uploads still seem to "fail" by uploading successfully):
- Please distribute all the images needed with the extension, I followed
the link in SOURCES.txt, and found that the link on that page was broken (though it wasn't hard to find a fixed link).
- Don't hardcode "\" as the directory separator - windows is not here!
- Use parser tests instead of selenium where appropriate (this will save
you "lots" of typing and a bit of test execution time)
- Make the tests pass by default :p.
Comments about the framework itself:
- Don't hardcode the firefox path - selenium's guess is better than yours.
- Should always shutdown selenium properly (I was often left with
myriads of firefoxes)
- !Always run against a clean database! (it is unrealistic to expect
people to write perfect tests, and particularly while developing the tests having an automatically clean database is essential).
- Get configuration from the main LocalSettings.php and use a mechanism
like parser tests for registering selenium test files. (making it as easy as possible to run tests is a good idea).
<waffle> How to deal with configuration (and by extension databases) for selenium tests... It is a tricky question, and to solve it properly requires a bit of effort. I think you need to send a selenium token with each request (like you suggested, though it'd be nicer to use a cookie rather than a url parameter in my opinion).
Either you could do a log in and get a session id, or you could simply update the cookie on every request with:
(selenium_test_name, timestamp, start_of_test, HASH(selenium_test_name, timestamp, shared_secret))
Providing mediawiki is in selenium mode, and the timestamp is recent, the selenium_test_name matches a registered selenium test and the hash is correct, then we can be reasonably sure this is a valid selenium request (I assume no-one steals the secret, much as you assume no-one reads wikisysop's password out of the file - besides you can always update it, and it's only a problem if selenium mode is enabled).
If timestamp == start_of_test then you can clean the test databases/ upload directories (or do that on "log in"). You can then also load any test-specific configuration from the selenium_test_name test on every request (and change cache-prefixes and upload directories to avoid collisions).
If you allow for one database (or pseudo database with table prefixes, like parser test) per selenium_test_name then you can run different tests in parallel; otherwise you could generate a new prefix based on the start_of_test and the selenium_test_name - but you'd quickly need a way to delete the databases completely once you'd finished.</waffle>
Thanks for your great work so far!
Conrad
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l