Hi All,
More Test Cases could be add to the existing Selenium Test framework. The idea should be to test the main Wiki Editor functionalities and Page view/preview functionalities in a smoke test suite.
The detailed administrative functionalities as User Preferences (more advanced features as Managing Watch-list, Advanced options in Editing, E-mail options) should be covered in a separate detailed test suite if necessary.
Wiki Editor functionalities can be tested by automating "Content Addition" in a new wiki page. All the basic text editing can be performed and should be verified in page preview and saved page view.
The same page can be tested for "Content Editing" by performing same editing functionalities in the previously saved page. The content changes can be verified in page preview and saved page view.
Set Basic preferences can be tested separately and basic settings as Basic Information, Signature, Date Format, Skin can be tested. The applied preferences can be verified against the saved page.
The other utility functionalities as Search for Pages, Move Page, My Watch-list, My Contributions, Delete Pages can be tested at the end of the test suite. Delete Page functionality can be used to delete all the newly created pages.
So the smoke test suite can be of 4 main test suites;
Add content to new Page (Wiki Editor for new page)
Edit content (Edit content in existing page)
Basic user preferences (Basic Information, Signature, Date Format, Skin)
Extra features (search, move, delete, watch-list, contribution)
All the above can be supported by some common functionalities as;
User Login
Search for Pages
Search and copy Text
Select applied skin
Get Page Name and other attributes (Page Headings/Titles etc.)
Get text atttributes (bold, italic, font size etc.)
Regards,
Jinesh De Silva
Calcey Technologies - http://www.calcey.com/ www.calcey.com
Voice: +94 11 2827560
Fax : +94 11 2827561
Hi everyone,
Some context here: Jinesh and the folks at Calcey are going to be pitching in on building out our Selenium test framework. I asked them to take a look here: http://svn.wikimedia.org/viewvc/mediawiki/trunk/phase3/maintenance/tests/sel...
...and then to suggest a plan of attack for building that out into a proper smoke test suite, as well as creating a larger battery of tests that we can run for comprehensive testing.
If people have thoughts on the types of things that developers inadvertently break that something like Selenium would be appropriate for trying to detect, your thoughts would be greatly appreciated here.
Jinesh, this looks like a great start. Comments inline:
On Thu, Nov 4, 2010 at 4:34 AM, Jinesh De Silva jinesh@calcey.com wrote:
More Test Cases could be add to the existing Selenium Test framework. The idea should be to test the main Wiki Editor functionalities and Page view/preview functionalities in a smoke test suite.
Yup, I agree, though one point we should probably clarify. The core smoke test should just test whether the plain editor works (without the WikiEditor extension). The WikiEditor extension should probably be tested as part of a suite of tests associated with all of the extensions we run in production.
By the way, here's a list of the extensions we run on English Wikipedia: http://en.wikipedia.org/wiki/Special:Version
...as well as the full list of extensions we currently have available in production: http://svn.wikimedia.org/viewvc/mediawiki/branches/wmf/1.16wmf4/extensions/
I don't have a strong opinion about which of these we should try tackling first, but generally testing the extensions that are most heavily used seems like a good thing to me.
The detailed administrative functionalities as User Preferences (more advanced features as Managing Watch-list, Advanced options in Editing, E-mail options) should be covered in a separate detailed test suite if necessary.
Also agreed. Perhaps others on this list can let us know: which aspects of these features are the most fragile + impact a lot of people when they break? My instinct is not to worry too much about these for now.
Wiki Editor functionalities can be tested by automating "Content Addition" in a new wiki page. All the basic text editing can be performed and should be verified in page preview and saved page view.
The same page can be tested for "Content Editing" by performing same editing functionalities in the previously saved page. The content changes can be verified in page preview and saved page view.
Set Basic preferences can be tested separately and basic settings as Basic
Information, Signature, Date Format, Skin can be tested. The applied preferences can be verified against the saved page.
The other utility functionalities as Search for Pages, Move Page, My Watch-list, My Contributions, Delete Pages can be tested at the end of the test suite. Delete Page functionality can be used to delete all the newly created pages.
So the smoke test suite can be of 4 main test suites;
Add content to new Page (Wiki Editor for new page) Edit content (Edit content in existing page) Basic user preferences (Basic Information, Signature, Date
Format, Skin)
Extra features (search, move, delete, watch-list,
contribution)
All the above can be supported by some common functionalities as;
User Login Search for Pages Search and copy Text Select applied skin Get Page Name and other attributes (Page Headings/Titles
etc.)
Get text atttributes (bold, italic, font size etc.)
This looks like a really good plan of attack. Anyone else care to weigh in?
Rob
On Fri, Nov 5, 2010 at 10:33 AM, Rob Lanphier robla@wikimedia.org wrote:
If people have thoughts on the types of things that developers inadvertently break that something like Selenium would be appropriate for trying to detect, your thoughts would be greatly appreciated here.
DB support for the other formats (aka, apart from just MySQL), if it installs, and some core functions work. -Peachey
Hi,
when I created tests for monobook and then switched to vector skin some tests did not work anymore, e.g., due to different naming schemes of input fields.
Although that would require only slight changes to the tests, it would increase redundancy and risk of inconsistency. How do you deal with this issue?
Cheers!
Benedikt
-- Karlsruhe Institute of Technology (KIT) Institute of Applied Informatics and Formal Description Methods (AIFB)
Benedikt Kämpgen Research Associate
Kaiserstraße 12 Building 11.40 76131 Karlsruhe, Germany
Phone: +49 721 608-7946 Fax: +49 721 608-6580 Email: benedikt.kaempgen@kit.edu Web: http://www.kit.edu/
KIT University of the State of Baden-Wuerttemberg and National Research Center of the Helmholtz Association
-----Original Message----- From: wikitech-l-bounces@lists.wikimedia.org [mailto:wikitech-l-bounces@lists.wikimedia.org] On Behalf Of K. Peachey Sent: Friday, November 05, 2010 1:47 AM To: Wikimedia developers Subject: Re: [Wikitech-l] Selenium Smoke Test Framework
On Fri, Nov 5, 2010 at 10:33 AM, Rob Lanphier robla@wikimedia.org wrote:
If people have thoughts on the types of things that developers
inadvertently
break that something like Selenium would be appropriate for trying to detect, your thoughts would be greatly appreciated here.
DB support for the other formats (aka, apart from just MySQL), if it installs, and some core functions work. -Peachey
_______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
These differences in field names aren't just to do with Vector, they are to do with the configuration of vector. If you have $wgVectorUseSimpleSearch = false; then the field names for search will be the same, the point is that it's a very different control, and a script that's hooking into it would fail because it's different. So what you are experiencing is not a bug, you just need to make your script work with one or the other.
- Trevor
On 11/10/10 2:36 AM, Benedikt Kaempgen wrote:
Hi,
when I created tests for monobook and then switched to vector skin some tests did not work anymore, e.g., due to different naming schemes of input fields.
Although that would require only slight changes to the tests, it would increase redundancy and risk of inconsistency. How do you deal with this issue?
Cheers!
Benedikt
-- Karlsruhe Institute of Technology (KIT) Institute of Applied Informatics and Formal Description Methods (AIFB)
Benedikt Kämpgen Research Associate
Kaiserstraße 12 Building 11.40 76131 Karlsruhe, Germany
Phone: +49 721 608-7946 Fax: +49 721 608-6580 Email: benedikt.kaempgen@kit.edu Web: http://www.kit.edu/
KIT -- University of the State of Baden-Wuerttemberg and National Research Center of the Helmholtz Association
-----Original Message----- From: wikitech-l-bounces@lists.wikimedia.org [mailto:wikitech-l-bounces@lists.wikimedia.org] On Behalf Of K. Peachey Sent: Friday, November 05, 2010 1:47 AM To: Wikimedia developers Subject: Re: [Wikitech-l] Selenium Smoke Test Framework
On Fri, Nov 5, 2010 at 10:33 AM, Rob Lanphierrobla@wikimedia.org wrote:
If people have thoughts on the types of things that developers
inadvertently
break that something like Selenium would be appropriate for trying to detect, your thoughts would be greatly appreciated here.
DB support for the other formats (aka, apart from just MySQL), if it installs, and some core functions work. -Peachey
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
wikitech-l@lists.wikimedia.org