Hi,
Automatic testing with a clean database sounds good, the scheme looks
reasonable. Will it be possible to have cleaning of separate databases, used
by extensions, added, also?
In the context of wiki identification, I was wondering: At the moment I am
mainly testing SMW instances having the selenium-server, the application
under test, and the tests each on the same machine. Wiki identification is
no issue, here. Of course, it would be great to eventually have the
WM/extension testing on external (possibly Wikimania) infrastructure,
resulting in higher security requirements; Is this planned or much
considered in the discussions?
Keep up the good work!
Regards,
Benedikt
--
Karlsruhe Institute of Technology (KIT)
Institute of Applied Informatics and Formal Description Methods (AIFB)
Benedikt Kämpgen
Research Associate
Kaiserstraße 12
Building 11.40
76131 Karlsruhe, Germany
Phone: +49 721 608-7946
Fax: +49 721 608-6580
Email: benedikt.kaempgen(a)kit.edu
Web:
http://www.kit.edu/
KIT University of the State of Baden-Wuerttemberg and
National Research Center of the Helmholtz Association
-----Original Message-----
From: wikitech-l-bounces(a)lists.wikimedia.org
[mailto:wikitech-l-bounces@lists.wikimedia.org] On Behalf Of Markus Glaser
Sent: Friday, October 15, 2010 3:54 PM
To: Wikimedia developers
Subject: Re: [Wikitech-l] using parserTests code for selenium test framework
Hi,
I recently suggested some scheme for dynamically creating clean wikis for
Selenium tests, which can be found here:
http://www.mediawiki.org/wiki/SeleniumFramework#Testing_with_a_clean_databas
e_and_file_state
After some discussion in the Testing group, I would like to elaborate a bit
further on some of the steps that need to be done:
2.2 Create temporal resources
2.2.1 create a new database with name "se"+testID
2.2.2 create a new images folder with name "se"+testID
2.2.3 populate database and images with template data. there is a standard
(vanilla) template, but also, test suites can have their own templates,
which then would be used. this test data should be placed in the same folder
as the tests, with the same name.
2.3 Create test tracker with timestamp
I suggest we use a textfile called "se"+testID.txt in a folder
wiki/seRunningTests. The timestamp would be the creation date of the file.
The next important question is, how should the wiki be identified? Brion
suggested using a subdomain, e.g. "sn"+testID.yourwiki.org. If I understand
webserver correctly, however, this would need some specific setup. In the
selenium testing group we were discussion identifying the wiki via cookie.
So the wiki under test would read the cookie and reconfigure accordingly.
The reconfiguration would need to take place right after LocalSettings.php,
since the following call to Setup.php already assumes some configurations as
set. As far as I know, Priyanka already has written some code to do this.
3.1 start testsuites via selenium
3.1.1 First, the SeleniumTestRunner needs to store the testID in order to
identify the ressources for teardown.
3.1.2 Start the test suite
5.1 send teardown request
fetch the testID stored in 3.1.1 and request the wiki under test to teardown
the ressources for that id
I would be very happy about comments and thoughts. Are we heading in the
right direction?
Cheers,
Markus
-----Ursprüngliche Nachricht-----
Von: wikitech-l-bounces(a)lists.wikimedia.org
[mailto:wikitech-l-bounces@lists.wikimedia.org] Im Auftrag von Markus Glaser
Gesendet: Mittwoch, 29. September 2010 20:02
An: Wikimedia developers
Betreff: Re: [Wikitech-l] using parserTests code for selenium test framework
Hi,
since the wiki under test is not neccessarily the wiki running the test, it
might be useful to visualize that (I have numbered the individual steps to
make reference to them easier in the discussion):
testrunner wiki under test
---------- ---------------
1.1 start selenium which in
turn starts a browser to
talk to the wiki under
test
1.2 send request for new test
with unique test id and tests
that will be fired
2.1 create cookie with test id
2.2 create temporal resources
according to tests list
2.3 create test tracker with timestamp
2.4 return success code
3.1 start testsuites via selenium
3.2 send a lot of individual
requests according to the tests
4.1 testrunner is identified by test id
4.2 reconfigure database and resources
according to test id
4.3 ? Do something with memcached ?
4.4 execute request
4.5 update timestamp in test tracker
5.1 send a teardown request
6.1 execute teardown, i.e. delete
all resources associated with
test id
6.2 delete test tracker
6.3 return success code
7.1 stop selenium
Now, if something breaks during the test, the test tracker will not be
deleted and can serve as as basis for a cleanup procedure that is triggered
by a cronjob.
Is this something we can all agree on? I assume, steps 2.2 (setting up
temporary test data) and 4.2 (find a mechanism to actually use the test
data) will be the ones we have to work on now.
Regards,
Markus
-----Ursprüngliche Nachricht-----
Von: wikitech-l-bounces(a)lists.wikimedia.org
[mailto:wikitech-l-bounces@lists.wikimedia.org] Im Auftrag von Ryan Lane
Gesendet: Freitag, 24. September 2010 20:22
An: Wikimedia developers
Betreff: Re: [Wikitech-l] using parserTests code for selenium test framework
Here is all that is required:
* a single wildcard entry in Apache configuration
* one or two lines in LocalSettings.php to pull a DB name from the
hostname/path/CLI parameters.
As for cleaning up resources to keep the machine from getting clogged,
it's very unlikely that your test wikis will fill up a
multi-hundred-gigabyte drive in the middle of a run. If you find that
they do, there's still no need to tie cleanup of any particular run to
any particular other run.
All you need to know is which runs have completed and can now be cleaned
up.
I'd like to add some ideas to this thread that were discussed in the
Selenium meeting this morning. The basic plan we discussed (and I'm sure
I'll be corrected some on this) is as follows:
When a run begins, it registers itself with the wiki and gets a session
back. The wiki software, on creating the session, makes a new run specific
wiki using the wiki family method. The test will pass both the session
cookie, and a test type cookie, which will dynamically configure the wiki as
the tests run. When the run is complete, it should notify the wiki that the
test run is complete. The wiki software will then destroy the session and
the dynamically created resources. If a run doesn't complete for some
reason, a cron can clean up resources that haven't been used in some
appropriate amount of time.
Respectfully,
Ryan Lane
_______________________________________________
Wikitech-l mailing list
Wikitech-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
_______________________________________________
Wikitech-l mailing list
Wikitech-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
_______________________________________________
Wikitech-l mailing list
Wikitech-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l