[QA] Tester's Guide to Software Development at WMF

[Chris McMahon]

Tester's Guide to Software Development at WMF

circa June 2013

At any given time there are a number of software development projects
underway both by and for the Wikimedia Foundation.  All of these projects
produce copious amounts of documentation, information, and conversation
that newcomers will likely find daunting if not overwhelming.  Here is a
summary of those software development projects with notes on testing and
testability, as well as links to further information for those who would
like to have more context for testing.

Editor Engagement aka E2

The biggest challenge facing Wikipedia today is to bring in new editors and
to retain existing editors for Wikipedia.  The E2 team builds the
large-scale Encyclopedia-wide features intended to help editors of
Wikipedia both experienced and new.  Right now the three biggest projects
in E2 are

* Wrapping up development for the feature called "Echo", the new
notifications for Wikipedia
* Starting work on "Flow", the new user messaging system for Wikipedia
* Maintaining the Article Feedback feature on multiple language wikis.  New
feature development on AFT is finished, but certain ongoing maintenance
tasks are still required.

We have some automated browser tests for AFT.  Since Echo is an
asynchronous message delivery service it does not lend itself to automated
browser testing.  However, right now would be a great time to see if
exploratory testing can identify any undiscovered edge case issues in Echo
while the deployed code is still fresh.  This would also be a good time to
start considering test charters for Flow, before full-scale development
ramps up.

http://en.wikipedia.org/wiki/Wikipedia:Editor_Engagement
http://www.mediawiki.org/wiki/Echo
http://www.mediawiki.org/wiki/Flow_Portal
http://www.mediawiki.org/wiki/Article_feedback/Version_5
IRC: #wikimedia-ee

https://github.com/wikimedia/qa-browsertests/blob/master/features/aftv5.feature


Editor Engagement Experiments aka E3

Where E2 is building large-scale features for Wikipedia editors, E3 is
doing short, simple experiments and a lot of A/B testing to discover what
workflow and UX aspects may encourage or discourage users from editing
Wikipedia.  E3 moves very quickly, but when an E3 experiment succeeds, it
becomes part of the normal code base.  One example of an E3 experiment come
to maturity is a new-user experience called "Getting Started" which is
built on a foundation called a "Guided Tour".  We have some automated
browser tests in place for Getting Started that have found regression
problems and been proven valuable.

Some ongoing E3 projects are "Account Creation User Experience" or ACUX,
new login screens, etc.

http://www.mediawiki.org/wiki/E3
https://trello.com/board/editor-engagement-experiments/5032806a85f7309c527929de
IRC: #wikimedia-e3

https://github.com/wikimedia/qa-browsertests/blob/master/features/guided_tour.feature


VisualEditor aka VE

There has never been a WYSIWYG editor for Wikipedia, but the Foundation is
building one now.  A blog post on the WMF tech blog from some time ago
explains why such a project is both technically challenging and expensive
to implement.   The release schedule for VE is ambitious, and the
possibility of significant issues in production code is high, so testing VE
is important.  Rachel Thomas will be working to help us test VE over the
summer with both exploratory tests and browser test automation as part of
the Outreach Program for Women project, but any other scrutiny we can bring
to VE will be valuable.

http://www.mediawiki.org/wiki/VisualEditor
http://blog.wikimedia.org/2013/03/04/parsoid-how-wikipedia-catches-up-with-the-web/
http://www.mediawiki.org/wiki/User:Rachel99/OPW_proposal

https://github.com/wikimedia/qa-browsertests/blob/master/features/visual_editor.feature(spike,
likely to be refactored)

Language

The Language team at WMF is one of two projects with a dedicated QA/tester
person, Runa Batterji.  Language is close to releasing the Universal
Language Selector, an ambitious project to allow users to control the
language(s) their wikis display.  We have begun some automated browser
tests for ULS.

http://www.mediawiki.org/wiki/Wikimedia_Language_engineering
http://www.mediawiki.org/wiki/Extension:UniversalLanguageSelector

https://github.com/wikimedia/qa-browsertests/blob/master/features/uls_cog_sidebar_logged_user.feature
https://github.com/wikimedia/qa-browsertests/blob/master/features/uls.feature
https://github.com/wikimedia/qa-browsertests/blob/master/features/uls_accept_language.feature
https://github.com/wikimedia/qa-browsertests/blob/master/features/uls_ime.feature


Mobile

The Mobile team at WMF is the other team at WMF with a dedicated QA/tester
person, Michelle Grover.  The automated tests for our Mobile web are
somewhat out of date, and we are looking to explore new automation tools,
in particular Appium for Mobile testing.

http://www.mediawiki.org/wiki/Wikimedia_Mobile_engineering/Contribute
IRC: #wikimedia-mobile

Platform and/or Features

There are any number of software projects and features on Wikipedia for
which no team is directly responsible.  Some of these features are
developed and supported by volunteers, some by the Foundation, some by a
combination.  Some of these projects amenable to testing are:

* UploadWizard for uploading files to Commons
* The Math extension
* TMH, TimedMediaHandler for displaying video and audio
* Print/export (PediaPress)
* PageTriage/NewPagesFeed
* Editing on Wikisource (has implications for VE)

http://www.mediawiki.org/wiki/Extension:UploadWizard
http://www.mediawiki.org/wiki/Extension:TimedMediaHandler
http://www.mediawiki.org/wiki/Extension:Math
http://www.mediawiki.org/wiki/Extension:PDF_Writer
http://en.wikipedia.beta.wmflabs.org/wiki/Special:NewPagesFeed

IRC: #wikimedia-dev, #mediawiki
https://github.com/wikimedia/qa-browsertests/blob/master/features/math_readonly.feature
https://github.com/wikimedia/qa-browsertests/blob/master/features/page_triage_readonly.feature
https://github.com/wikimedia/qa-browsertests/blob/master/features/pdf_readonly.feature
https://github.com/wikimedia/qa-browsertests/blob/master/features/upload_wizard.feature


* etc., etc., etc.

Labs

Wikimedia Labs is the umbrella term for a number of different projects,
among them:

* to provide free hosted VMS for Mediawiki development
* to provide a free host for Wikipedia bots and other 3rd-party tools

and probably of most interest to testers,

* to provide a shared public test environment containing a reference
implementation of Wikipedia software http://en.wikipedia.beta.wmflabs.org/(etc.)

https://wikitech.wikimedia.org/wiki/Main_Page
http://en.wikipedia.beta.wmflabs.org/
IRC: #wikimedia-labs

Misc

This is mostly out of date but worth looking at:
https://mingle.corp.wikimedia.org/projects


>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.wikimedia.org/pipermail/qa/attachments/20130603/d4b88848/attachment.html>