[Wikimedia-l] [Wikimedia Announcements] Wikimedia engineering June 2013 report

Guillaume Paumier gpaumier at wikimedia.org
Fri Jul 12 11:29:04 UTC 2013 

Hi,

The report covering Wikimedia engineering activities in June 2013 is now
available.

Wiki version:
https://www.mediawiki.org/wiki/Wikimedia_engineering_report/2013/June
Blog version:
https://blog.wikimedia.org/2013/07/12/engineering-june-2013-report/

We're also proposing a shorter, simpler and translatable version of this
report that does not assume specialized technical knowledge:
https://www.mediawiki.org/wiki/Wikimedia_engineering_report/2013/June/summary

Below is the full HTML text of the report.

As always, feedback is appreciated on the usefulness of the report and its
summary, and on how to improve them.

------------------------------------------------------------------

Major news in June include:

   - The preparation for the activation of
VisualEditor<https://blog.wikimedia.org/2013/06/06/preparing-for-visualeditor-on-all-wikipedias/>to
most Wikipedia sites, and its
   debut on the English
Wikipedia<https://blog.wikimedia.org/2013/07/01/visualeditor-beta-rollout/>
   ;
   - News around Language
engineering<https://blog.wikimedia.org/2013/06/03/language-engineering-development-updates-and-events/>,
   including the
preparation<https://blog.wikimedia.org/2013/06/06/universal-language-selector-coming-to-all-wikis/>and
   activation<https://blog.wikimedia.org/2013/07/01/universal-language-selector-uls-deployed-on-more-than-150-wikis/>of
the Universal Language Selectors on many wikis;
   - An explanation of how bugs are discovered and
fixed<https://blog.wikimedia.org/2013/06/04/volunteers-and-staffers-teach-learn-make-at-amsterdam-hackathon/>
   .
   - A retrospective on the Amsterdam
hackathon<https://blog.wikimedia.org/2013/06/04/volunteers-and-staffers-teach-learn-make-at-amsterdam-hackathon/>
   .

*Note: We're also providing a shorter, simpler and translatable version of
this report<https://www.mediawiki.org/wiki/Wikimedia_engineering_report/2013/June/summary>that
does not assume specialized technical knowledge.
*
Personnel Work with us <https://wikimediafoundation.org/wiki/Work_with_us>

Are you looking to work for Wikimedia? We have a lot of hiring coming up,
and we really love talking to active community members about these roles.

   - Software Engineer -
Fundraising<http://hire.jobvite.com/Jobvite/Job.aspx?j=oawpXfwM>
   - Software Engineer - Language
Engineering<http://hire.jobvite.com/Jobvite/Job.aspx?j=oH3gXfwH>
   - Director of Program -
Mobile<http://hire.jobvite.com/Jobvite/Job.aspx?j=oGqAXfwn>
   - Software Engineer - Multimedia
Systems<http://hire.jobvite.com/Jobvite/Job.aspx?j=oj40Wfw3>
   - Senior Software Engineer -
Platform<http://hire.jobvite.com/Jobvite/Job.aspx?j=ouLnWfwi>
   - UX Designer <http://hire.jobvite.com/Jobvite/Job.aspx?j=onImXfw8>
   - Product Manager -
Platform<http://hire.jobvite.com/Jobvite/Job.aspx?j=o3vtXfwI>
   - Dev-Ops Engineer -
SRE<http://hire.jobvite.com/Jobvite/Job.aspx?j=ocLCWfwf>

Announcements

   - Sean Pringle joined the Technical Operations team as our Storage and
   Database Engineer
(announcement<http://www.gossamer-threads.com/lists/wiki/wikitech/368588>
   ).
   - Brian Wolff <https://www.mediawiki.org/wiki/User:Bawolff> joined
the Wikimedia
   Platform Engineering<https://www.mediawiki.org/wiki/Wikimedia_Platform_Engineering>group
as Software developer for the Summer, working on multimedia
   contribution and review
(announcement<http://lists.wikimedia.org/pipermail/wikitech-l/2013-June/069724.html>
   ).
   - Ken Snider joined the Technical Operations team as an international
   contractor, poised to fill the Director of Technical Operations position (
   announcement<http://lists.wikimedia.org/pipermail/wikitech-l/2013-June/069844.html>
   ).
   - Toby Negrin joined the Engineering department as Director of Analytics
   (announcement<http://lists.wikimedia.org/pipermail/wikimediaannounce-l/2013-June/000675.html>
   ).

Technical Operations

*Site infrastructure*
As part of our capacity planning work, Mark Bergsma upgraded most of our
Varnish infrastructure (in EQIAD & ESAMS) with newer and faster servers. He
will be adding new mobile Varnish servers in ESAMS next, this coming month.
Rob Halsell and Daniel Zahn are pushing ahead with the migration of the
other applications <https://wikitech.wikimedia.org/wiki/Tampa_cluster> from
Tampa to EQIAD. New Parsoid application and Varnish servers were also
deployed in anticipation of the coming VisualEditor deployment. Meantime,
Alexandros Kosiaris is starting the backup project work; read more about the
project <https://wikitech.wikimedia.org/wiki/Projects#Backup_infrastructure>and
the
technology <https://wikitech.wikimedia.org/wiki/Bacula>. Mark also put in
the finishing touches to deploy all the new network infrastructure at
ESAMS. With help from Mark and Leslie Carr, we finally got approval from
ARIN for some new IPv4 addresses, needed for our new ULSFO buildup.Many
people are refactoring Puppet code with the ultimate goal of having
everything organized into Puppet modules. Andrew Bogott, Antoine Musso and
Alexandros are setting up an automated testing infrastructure to support
these efforts.

*Data Dumps <https://www.mediawiki.org/wiki/WMF_Projects/Data_Dumps>*
Our GSoC student, Petr Onderka, is set up in gerrit and committed his first
contributions to the Incremental Dumps project; you can follow his
code<https://git.wikimedia.org/summary/operations%2Fdumps%2Fincremental>,
read his progress
reports<https://www.mediawiki.org/wiki/User:Svick/Incremental_dumps#Updates>and
check the current
discussion<http://lists.wikimedia.org/pipermail/xmldatadumps-l/2013-July/000834.html>on
the mailing list. Additionally, we hold IRC meetings on weekdays at
about 4:15 pm (UTC) in #wikimedia-tech; lurkers and contributors are
welcome.

*Wikimedia Labs <https://www.mediawiki.org/wiki/Wikimedia_Labs>*
Wikimedia Labs saw a lot of improvements in June, including the deployment
of AJAX improvements for OpenStackManager to wikitech (added actions:
console output; improvements: reboot), and a new interface for displaying
quotas for projects in OpenStackManager. We ensured that all instances were
properly running Puppet and Salt; Many instances were running
puppetmaster::self and needed to have local puppet repo merges or rebases.
We upgraded Salt everywhere and re-issued keys to fix a vulnerability in
Salt. The team also worked on stabilizing the NFS server. We've encountered
a kernel bug with NFS; we have changed the scheduler from cfq to deadline
and have decreased the read and write sizes of clients to 8k. Progress has
been made towards making the Labs database replicas available to the Labs
at large (as opposed to only the Tool Labs project). Last, much work has
been done towards user request fulfillment in Tool Labs, including work
towards WSGI support. Features
Engineering<https://www.mediawiki.org/wiki/Wikimedia_Features_engineering>
Editor
retention: Editing tools

*VisualEditor <https://www.mediawiki.org/wiki/VisualEditor>*
In June, the VisualEditor team completed the major new features that we
prioritised over the past few months, in preparation for making
VisualEditor available to most Wikipedia users in July. We have built an
editor that is capable of letting users edit the majority of content
without needing to use wikitext — text support, as well as adding and
editing inclusions of references, templates, categories and media items.
The deployed alpha of VisualEditor was updated four times as part of the
transition to weekly deployments
(1.22-wmf6<https://www.mediawiki.org/wiki/MediaWiki_1.22/wmf6#VisualEditor>,
1.22-wmf7 <https://www.mediawiki.org/wiki/MediaWiki_1.22/wmf7#VisualEditor>,
1.22-wmf8 <https://www.mediawiki.org/wiki/MediaWiki_1.22/wmf8#VisualEditor>and
1.22-wmf9 <https://www.mediawiki.org/wiki/MediaWiki_1.22/wmf9#VisualEditor>),
with several mid-deployment releases as the code was developed to patch
urgent issues. Part of this involved running an A/B test for new user
accounts on the English Wikipedia, with half of the users getting opt-in to
VisualEditor ahead of the wider release. Generally, there were a number of
user interface improvements, and fixing a number of bugs uncovered by the
community.

*Parsoid <https://www.mediawiki.org/wiki/Parsoid>*
Early this month, we deployed Parsoid to the new cluster and started to
track all edits and template / image updates from all Wikipedia sites,
which is close to the full load we'll see when VE is deployed to all of
them. Our earlier optimization work paid off as the Parsoid cluster and the
associated Varnish caches are handling the load very well. The extra load
we put on the API cluster is low enough to not cause a problem. As
expected, the VisualEditor deployment to the English Wikipedia hardly
showed up in the load graphs.

Despite being very short-staffed this month (only two full-time
developers), the absence of performance issues left us enough time for a
lot more polishing before the VisualEditor release on July 1. As a result,
the release went very well with clean diffs on almost all pages.
While more work is left to do, it is now clear that we have fundamentally
achieved our goal of a clean translation between WikiText and HTML + RDFa.
This does not only enable visual HTML editing, but also makes Wikipedia's
content easily accessible in a standardized format. It also opens up new
opportunities for MediaWiki's core architecture, which we'll pursue this
fiscal year.
Editor engagement features

*Notifications <https://www.mediawiki.org/wiki/Echo_%28Notifications%29>*
In June, we released more features and bug fixes for Notifications on the
English Wikipedia and mediawiki.org. Ryan Kaldari added a confirmation
button for the 'Thanks
feature<https://en.wikipedia.org/wiki/Wikipedia:Notifications/Thanks>',
and updated notification fly-outs to show diff
links<https://bugzilla.wikimedia.org/show_bug.cgi?id=48183>for talk
page and interactive notifications, based on a design by Vibha
Bamba. Benny Situ continued development of HTML Email
notifications<https://www.mediawiki.org/wiki/Echo/Feature_requirements#HTML_single_email_notifications>and
deployed a variety of feature updates. Erik Bernhardson developed a
special 'Suppressed' content feature, while Matthias Mullie developed a
range of new metrics
dashboards<http://ee-dashboard.wmflabs.org/dashboards/enwiki-features>.
Dario Taraborelli and Aaron Halfaker ran a week-long A/B test of new user
activity<https://meta.wikimedia.org/wiki/Research:Notifications/Experiment_1/Results>;
results show that new users who received Echo notifications made more edits
than those who did not, but their edits were reverted slightly more often.
Fabrice Florin led the planning process for Notifications, as outlined in
the 2013 roadmap<https://www.mediawiki.org/wiki/Editor_Engagement/2013_strategy_planning_%28Features%29>,
and hosted a day-long roundtable
discussion<https://meta.wikimedia.org/wiki/Roundtables/Roundtable_1>to
improve editor engagement features in collaboration with Wikipedia
users
(see Echo demo and Q&A video on
YouTube<http://www.youtube.com/watch?v=tyJANeVnbsQ>).
Later this summer, we plan to start deploying Notifications on more wiki
projects, starting with Meta and the French Wikipedia. To learn more, visit
the project portal <https://en.wikipedia.org/wiki/Wikipedia:Notifications>,
read the FAQ page<https://en.wikipedia.org/wiki/Wikipedia:Notifications/FAQ>and
join the discussion on the talk
page <https://en.wikipedia.org/wiki/Wikipedia_talk:Notifications>.

*Article feedback <https://www.mediawiki.org/wiki/Article_feedback>*
In June, we deployed final features and bug fixes for the Article Feedback
Tool <https://www.mediawiki.org/wiki/Article_feedback/Version_5> (AFT5) on
the English <https://en.wikipedia.org/wiki/Special:ArticleFeedbackv5>,
French <https://en.wikipedia.org/wiki/fr:Sp%C3%A9cial:ArticleFeedbackv5>and
German<https://en.wikipedia.org/wiki/de:Spezial:Artikelr%C3%BCckmeldungen_v5>Wikipedias.
Matthias Mullie released an
opt-in feature<https://www.mediawiki.org/wiki/Article_feedback/Version_5/Feature_Requirements#Enable.2Fdisable_feedback_on_a_page>to
enable or disable feedback on a page, based on designs by Pau Giner
and
specifications by Fabrice Florin. In collaboration with Dario Taraborelli,
Matthias also developed an updated set of metrics
dashboards<http://ee-dashboard.wmflabs.org/dashboards/enwiki-features#article_feedback_moderation-graphs-tab>showing
how the new moderation tools are being used: for example, about
half of moderated feedback is marked as 'no action needed', while about a
tenth is marked as 'useful' (these results are generally consistent across
different languages<https://docs.google.com/a/wikimedia.org/spreadsheet/ccc?key=0Aq_75_5y5sKWdDl0blpSbGRiS2ppRzlaaHZiV1dRMXc#gid=3>).
The team also supported a wider deployment of AFT5 on over 40,000 articles
on the French Wikipedia, as well as a poll by the German community, which
elected not to adopt the tool. Now that feature development has ended for
this project, we plan to make AFT5 available to other wiki projects in
coming weeks, as outlined in the release
plan<https://www.mediawiki.org/wiki/Article_feedback/Version_5/Release_Plan_2013>.
For tips on how to use Article feedback, visit the testing
page<https://www.mediawiki.org/wiki/Article_feedback/Version_5/Testing>,
and let us know what you think on this talk
page<https://en.wikipedia.org/wiki/Wikipedia_talk:Article_Feedback_Tool/Version_5>
.
Editor engagement experiments

*Editor engagement
experiment<https://www.mediawiki.org/wiki/Editor_engagement_experiments>
*
In June, the Editor Engagement Experiments (E3) team continued work on its
experiments related to onboarding new
Wikipedians<https://www.mediawiki.org/wiki/Onboarding_new_Wikipedians>,
and launched several new extensions to Wikimedia projects.

First, the new Campaigns
extension<https://www.mediawiki.org/wiki/Extension:Campaigns>was added
to all wikis. This analytics tool helps identify internal or
external sources of new registrations, by adding a "campaign" name to the
signup page URL. This month, E3 began running campaigns to learn about how
many anonymous editors sign up on the top 10 Wikipedias, as well as how
many sign up via the invitation to "Join Wikipedia" on the login page (see
the list of active
campaigns<https://meta.wikimedia.org/wiki/Research:Account_creation_campaigns>and
analysis). Another piece of analytics infrastructure by the team is
the
new CoreEvents extension<https://www.mediawiki.org/wiki/Extension:CoreEvents>,
which houses logging of MediaWiki core activity, like preference updates
and page saves across all projects.

For the Getting
Started<https://www.mediawiki.org/wiki/Extension:GettingStarted>project,
the team conducted usability testing (see results
and documentation<https://www.mediawiki.org/wiki/Onboarding_new_Wikipedians/user_testing#Testing_scenario_three>)
of new designs. E3 also refactored and refined the guided
tours<https://www.mediawiki.org/wiki/Guided_tours>extension in June,
including adding usability enhancements like new
interface animations, support for community tours, and bug fixing. The team
also planned and began work on an experiment to deliver guided
tours<https://meta.wikimedia.org/wiki/Research:Guided_tours>to all
first-time editors.
The team also assisted with A/B testing and research for
VisualEditor<https://www.mediawiki.org/wiki/VisualEditor>before its
July 1 launch date, assisting with experimental design,
EventLogging instrumentation, and other work. After the VisualEditor
launch, E3 started a week-long micro-survey of newly-registered
users<https://meta.wikimedia.org/wiki/Research:Gender_micro-survey>on
English Wikipedia, to give us a first systematic look at the gender
diversity of those creating accounts.
Support

*2012 Wikimedia
fundraiser<https://www.mediawiki.org/wiki/2012_Wikimedia_fundraiser>
*
The initial work on the Adyen payments gateway was finally completed and
deployed to production, though we have not yet used the gateway in a
campaign. Plans for a mobile fundraising campaign and workflow continued to
move forward: We expect to do the first mobile-targeted campaign in mid to
late July. Some last-minute tweaking was done to the payments cluster in
preparation for the resumption of continuous fundraising on July 1,
coinciding with the start of the fiscal year. Payments listener (thulium)
deploy was completed, db1013 was moved into the firewalled fundraising
cluster and rebuilt as a fundraising QA server, and work continued on the
new CiviCRM server (barium). Fundraising backups were overhauled.
Mobile <https://www.mediawiki.org/wiki/Wikimedia_Mobile_engineering>

*Wikipedia Zero <https://www.mediawiki.org/wiki/Wikipedia_Zero>*
This month, the team launched Wikipedia Zero with Dialog in Sri Lanka,
patched logic and user interface bugs, enhanced the configuration editor,
expanded logging and debugging for identification of anomalous access,
further decoupled ZeroRatedMobileAccess from MobileFrontend, and proposed
ESI- and JavaScript-based software re-architecture.

*Mobile Web Photo Upload<https://www.mediawiki.org/wiki/Mobile_design/Uploads>
*
This month, we focused on improving education around uploads, including an
interactive Commons tutorial and first-time user copyright and scope check.
We also released our "Nearby" feature to production, allowing users to find
articles near them that are in need of images, take photos and upload them
via mobile.

*Mobile Nav<https://www.mediawiki.org/wiki/Mobile_design/Wikipedia_navigation>
*
In beta, we started working on an update to our site and article
navigation, including design tweaks to the left navigation menu and a new
in-article contributory navigation that combines article actions (edit,
upload, and watch) with a talk page link. We also experimented with Echo
integration and successfully got Notifications up and running on the
English Wikipedia mobile site. We hope to push all of this work to
production next month.
Platform Engineering<https://www.mediawiki.org/wiki/Wikimedia_Platform_Engineering>
MediaWiki
Core

*MediaWiki 1.22 <https://www.mediawiki.org/wiki/MediaWiki_1.22/Roadmap>*
In June, the Platform Engineering group switched to a weekly deployment
cycle <https://wikitech.wikimedia.org/wiki/Deployments/One_week> for
MediaWiki to the Wikimedia Foundation servers. This means that we have
almost halved our previous cycle of 2 weeks. As such, we are progressing
through wmfXX versions of MediaWiki at a faster rate now. In June,
MediaWiki versions 1.22-wmf6 through wmf9 were branched and deployed.

*Git conversion <https://www.mediawiki.org/wiki/Git/Conversion>*
Chad Horohoe and Christian Aistleitner upgraded our Gerrit instance from a
pre-release version of 2.6 to a pre-release version of 2.7 on the last week
of June. They've additionally published a new version of the
Bugzilla/Gerrit integration plugin. Details about new functionality can be
found in the Gerrit 2.7 draft release
notes<http://gerrit-documentation.googlecode.com/svn/ReleaseNotes/ReleaseNotes-2.7.html>
.

*Multimedia <https://www.mediawiki.org/wiki/Multimedia>*
In June, we started expanding our multimedia team: Fabrice Florin joined as
product manager, and Brian Wolff began a summer contract as software
engineer. We started work on improving the display of images in galleries
and are now planning our next development steps in consultation with
community members. Some of the first features under consideration include
file curation and feedback tools, as well as media viewers, new video
formats and other platform improvements, to be prioritized based on user
feedback and technical feasibility. We are also recruiting for two more
positions: a multimedia systems
engineer<http://hire.jobvite.com/Jobvite/Job.aspx?j=oj40Wfw3&c=qSa9VfwQ>and
a senior
software engineer<http://hire.jobvite.com/Jobvite/Job.aspx?j=ouLnWfwi&c=qSa9VfwQ>.
Please spread the word about this unique opportunity to create a richer
multimedia experience for Wikipedia and MediaWiki sites!

*Admin tools development<https://www.mediawiki.org/wiki/Admin_tools_development>
*
In June, the team worked on making the last changes to enable global
AbuseFilter rules, and on the global account renaming tool. Some additional
work was done on Single User Login
finalisation<https://www.mediawiki.org/wiki/Admin_tools_development/SUL_Audit>,
which will mean that all user accounts will be global across all of
Wikimedia's public wikis, and so allowing for cross-wiki notifications and
better tools for editors.

*Search <https://www.mediawiki.org/wiki/Search>*
Work has pretty much shifted from supporting MWSearch/lsearchd to
investigating and implementing Solr. Nik Everett and Chad Horohoe have
begun writing an
extension<https://www.mediawiki.org/wiki/Extension:CirrusSearch>to
implement Solr searching for MediaWiki, and we've gotten a lot of the
initial basic functionality completed. Peter Youngmeister and Andrew Bogott
will be handling the operations tasks for the new setup. Initial operations
tasks will involve packaging Solr 4 and working with Chad to puppetize the
whole design. Additionally, we're going to do some investigation into
ElasticSearch, as it's been suggested as an alternative to Solr.

*Auth systems <https://www.mediawiki.org/wiki/Auth_systems>*
In June, the team worked with the Wikimedia Foundation's user experience
team to improve SUL2. The improvements were pushed to test wikis on July 1,
and will be rolled out to other wikis in July. Implementation of OAuth is
well underway, and planned for roll-out in July as well.

*HipHop deployment <https://www.mediawiki.org/wiki/HipHop_deployment>*
A Labs instance of MediaWiki running on HipHop is now available at
http://hhvm.wmflabs.org.

*Security auditing and
response<https://www.mediawiki.org/wiki/Security_auditing_and_response>
*
The team continued to respond to reported security issues, and gave
security-oriented tech talks on emerging DoS techniques and using OWASP's
ZAP tool for vulnerability scanning.
Quality assurance

*Quality Assurance <https://www.mediawiki.org/wiki/Quality_Assurance>*
This month saw a QA focus on automated browser tests. Besides creating new
tests and new builds, and reporting issues identified by tests, we
conducted a training session in San Francisco to create automated tests for
the Wikilove feature. We continue to support all WMF software development
projects, with the VisualEditor being a particular focus in June.

*Beta cluster <https://www.mediawiki.org/wiki/Beta_cluster>*
Max Semenik wrote a script to synchronize CSS from production on beta.
Steinsplitter and Antoine Musso fixed the AbuseFilter configuration to have
a global list of filters on the
labswiki<http://deployment.wikimedia.beta.wmflabs.org/wiki/Special:AbuseFilter>.
Filters should be configured there and will be used by all the wikis. The
PHP fatal errors catched by the wmerrors extension are now sent to the beta
udp2log instance. That will largely improve our troubleshooting process.

*Continuous integration<https://www.mediawiki.org/wiki/Continuous_integration>
*
Timo Tijhof and Antoine Musso triaged continuous integration bugs. Antoine
has setup a Jenkins slave and migrated most jobs on it. It will be very
easy to add new servers.

*Browser testing<https://www.mediawiki.org/wiki/Quality_Assurance/Browser_testing>
*
This month, the QA team added new browser tests for
UniveralLanguageSelector and for Mobile (contributed by the Language
engineering and Mobile engineering teams, respectively), as well as browser
test contributions from volunteers. We created new builds in Jenkins to run
browser tests against IE10. We created tests for VisualEditor, including
some with our intern with the Outreach Program for Women.
Analytics

*Analytics infrastructure<https://www.mediawiki.org/wiki/Analytics/Infrastructure>
*
We made significant progress with our preparations for replacing udp2log
with Kafka in our logging infrastructure. The C library
librdkafka<https://github.com/edenhill/librdkafka/tree/0.8-wip>has now
support for the 0.8 protocol, there is a first version of
varnishkafka <https://gerrit.wikimedia.org/r/#/c/70928/> ready that will
replace varnishncsa, the Apache Kafka project released their first beta of
Kafka 0.8, and we have a
Debianized<https://gerrit.wikimedia.org/r/#/c/68026/>and
Pupppetized <https://gerrit.wikimedia.org/r/#/c/50385/> version. We keep on
adding new metrics <https://gerrit.wikimedia.org/r/#/c/71089/> and alerts
to monitor all the different parts of the webrequest dataflows into Kraken.
We expect to keep making improvements in the coming months, until we have a
fully reliable data pipeline into Kraken. We also continued our efforts of
moving Kraken out of beta: we puppetized
Zookeeper<https://gerrit.wikimedia.org/r/#/c/66882/>,
JMXtrans <https://gerrit.wikimedia.org/r/#/c/70915/>, and the Hadoop client
nodes for Hive, Pig and Sqoop <https://gerrit.wikimedia.org/r/#/c/69521/>.
We started reinstalling the Hadoop Datanode workers with a fully puppetized
Hadoop installation; so far, we have replaced 3 nodes, and we'll replace
the other seven in the coming weeks. Last, we enabled Jenkins continuous
integration for the Grantmaking & Evaluation dashboards.

*Analytics Visualization, Reporting &
Applications<https://www.mediawiki.org/wiki/Analytics/Visualization,_Reporting_%26_Applications>
*
This month, we completed the end-user
documentation<https://www.mediawiki.org/wiki/UserMetrics/Guide>of
UserMetrics (v1). We rebranded UserMetrics as Wikimetrics, and we will
slowly start to use that as the new name when referring to UserMetrics v2
or UserMetrics replatforming. We focused on laying out the foundation of
Wikimetrics: a new database design, a new job queue design and lots of unit
tests. In addition, we started working on porting over some of the features
of UserMetrics v1 (like the 'namespace edits' metric and UI components), we
added user roles (so users can only see their own metrics) and
authentication using OAuth. Last, we fixed some minor issues in UserMetrics
v1, among which handling of user names with comma, single and double quotes.

*Data Releases <https://www.mediawiki.org/wiki/Analytics/Data_Releases>*
We delivered many following analyses in June, including one of Arabic
cohort using UMAPI v1. Erik Zachte provided an analysis of Commons
uploaders, and we provided the Wikipedia Zero team with a number of
datasets to help them in tracking adoption of the Wikipedia Zero project
across the globe. We supported the VisualEditor and Editor Engagement teams
with experimental design, data modeling and data analysis for two
controlled experiments: a test of the impact of impact of
notifications<https://meta.wikimedia.org/wiki/wiki/Research:Notifications/Experiment_1>and
a first
test <https://meta.wikimedia.org/wiki/Research:Visual_Editor> of the impact
of Visual Editor on new contributors. The tests were carried out in June
and the reports are being updated with the results of the analysis. We
started using the EE-dashboard instance on Labs to host dashboards related
to editor engagement projects, that were previously hosted on the
Toolserver (see the
metrics<http://ee-dashboard.wmflabs.org/dashboards/enwiki-metrics>and
features <http://ee-dashboard.wmflabs.org/dashboards/enwiki-features>dashboards
for the English Wikipedia). Last, we worked with the Features
engineering team to expand MediaWiki's instrumentation and collect
data on cluster-wide
user preference
changes<https://meta.wikimedia.org/wiki/Research:User_preferences>and
edit-related
events <https://meta.wikimedia.org/wiki/Schema:Edit> to support
VisualEditor analysis.
Engineering community team

*Bug management <https://www.mediawiki.org/wiki/Bug_management>*
Andre Klapper <https://www.mediawiki.org/wiki/User:AKlapper_%28WMF%29>published
the Bugzilla
administrator policy<https://www.mediawiki.org/wiki/User:AKlapper_%28WMF%29/BugzillaAdminPolicy>and
documented for
which specific tasks<http://blogs.gnome.org/aklapper/2013/05/28/understanding-bugzilla-groups-and-admin-rights/>Bugzilla
admin rights are actually needed (which might be also helpful for
other projects using Bugzilla). He started publishing weekly "Bugzilla tips
and best practices"<http://blogs.gnome.org/aklapper/category/computer/bugzilla/>blog
posts and reproposed introducing
a "PATCH AVAILABLE"
status<http://lists.wikimedia.org/pipermail/wikitech-l/2013-June/069805.html>in
Bugzilla (as requested by several developers at the Amsterdam
Hackathon)
whilst work is ongoing to fulfill prerequisites. On the code side of
Bugzilla, a new Bugzilla frontpage <https://bugzilla.wikimedia.org> went
live, providing useful links. Furthermore, the misleading term "login"
was replaced
by "email address" <https://bugzilla.wikimedia.org/show_bug.cgi?id=24992>,
it is now possible to set the "Assigned" status
directly<https://bugzilla.wikimedia.org/show_bug.cgi?id=49461>when
filing a new bug report, and smaller issues with the "Weekly Bugzilla
Report" email sent to the wikitech-l mailing list were fixed. In Bugzilla's
taxonomy, open tickets in the dormant "Wiktionary tools" product were
retriaged and the product closed for new bug entry, and Security-related
components in Bugzilla were reorganized after a meeting with the Wikimedia
Foundation's security engineer.

*Mentorship programs <https://www.mediawiki.org/wiki/Mentorship_programs>*
The 20 Google Summer of
Code<https://www.mediawiki.org/wiki/Summer_of_Code_2013>and the 1
Outreach
Program for Women<https://www.mediawiki.org/wiki/Outreach_Program_for_Women/Round_6>interns
have completed the bonding
period<https://www.mediawiki.org/wiki/Summer_of_Code_2013#Community_bonding_period>(with
3 exceptions, 2 of them justified) and they are now working on their
projects. One OPW accepted candidate declined her participation due to a
job offer. Monthly status updates are available for these projects:

   - Improving support for book
structures<https://meta.wikimedia.org/wiki/Book_management/Progress#June>
   - Mobilize Wikidata<https://www.mediawiki.org/wiki/User:Pragunbhutani/GSoC_2013_Updates#Monthly_Report:_June_2013>
   - jQuery.IME extensions for Firefox and
Chrome<https://www.mediawiki.org/wiki/User:Prageck/GSoC_2013_Application#Monthly_Report:_June_2013>
   - Browser Test Automation for Visual
Editor<https://www.mediawiki.org/wiki/User:Rachel99/proposal1#June>
   - Incremental
dumps<https://www.mediawiki.org/wiki/User:Svick/Incremental_dumps#June_report>
   - Refactoring of Extension
ProofreadPage<https://www.mediawiki.org/wiki/Extension:Proofread_Page/GSoC#June>
   - Language Coverage Matrix
Dashboard<https://www.mediawiki.org/wiki/User:Harsh4101991/GSoC_2013#June>
   - Section Handling in Semantic
Forms<https://www.mediawiki.org/wiki/User:Himeshi/GSoC_2013/Project>
   - VisualEditor Mathematical Editor
Plugin<https://www.mediawiki.org/wiki/User:Jiabao_wu/GSoC_2013_Project_Work#June>
   - ZIM Incremental Updates for
Kiwix<https://www.mediawiki.org/wiki/User:Kiran_mathew_1993/ZIM_incremental_updates_for_Kiwix#June_Report>
   - VisualEditor RTL
support<https://www.mediawiki.org/wiki/User:Mooeypoo/GSOC_2013_Proposal:_RTL_Support_in_VisualEditor#Monthly_Report:_June_2013>
   - UploadWizard: book upload
customization<https://www.mediawiki.org/wiki/User:Rasel160/GSoC2013/Status>
   - Android app for MediaWii
translation<https://www.mediawiki.org/wiki/User:Orsagi/GSoC_2013_proposal#Monthly_Report:_June_2013>
   - Pronunciation recording
tool<https://www.mediawiki.org/wiki/User:Rahul21/Gsoc2013/Project_Updates>
   - MediaWiki Moddle
extension<https://www.mediawiki.org/wiki/User:Clancer/GSoC_2013_Project>
   - VisualEditor plugin for source
code<https://www.mediawiki.org/wiki/User:Beanixster#June>

We also met with SocialCoding4Good <http://www.socialcoding4good.org/>, who
are relaunching their activities, and we refreshed the Wikimedia
page<http://www.socialcoding4good.org/organization/wikimedia>.
We expect this to become a regular channel for new technical contributors
working in corporations with social/training programs.

*Technical communications<https://www.mediawiki.org/wiki/Technical_communications>
*
In June, work on this topic mostly focused on perennial activities like Tech
news <https://meta.wikimedia.org/wiki/Tech/News> and ongoing communications
support to engineering staff, as Guillaume
Paumier<https://www.mediawiki.org/wiki/User:Guillom>was lent to the
VisualEditor deployment effort, working on communications,
documentation and liaising with the French Wikipedia.

*Volunteer coordination and
outreach<https://www.mediawiki.org/wiki/Volunteer_coordination_and_outreach>
*
The decision of focusing on fewer activities better executed and based on
demand seems to be working out, although it's too soon to confirm the
trend. Browser test automation is the number one priority to recruit new
contributors, and any help to succeed here is welcome. We created the QA
mailing list <https://lists.wikimedia.org/mailman/listinfo/qa> as an
umbrella to host people and discussions focusing on software quality
assurance in all its aspects. We have more than 40 subscribers and an
initial flow of activity. We had a successful first Browser Test Automation
Workshop <http://www.meetup.com/wikimedia-tech/events/124334432/>, with 40
participants in San Francisco and a few more online; we will iterate on
this model. We have also helped organizing a Tech Talk on Attack vectors &
MediaWiki and OWASP
ZAP<https://www.mediawiki.org/wiki/Meetings/2013-06-27-midday>,
and the upcoming Solr-based
Search<https://www.mediawiki.org/wiki/Meetings/2013-07-03>.
The project to get automated community
metrics<https://www.mediawiki.org/wiki/Community_metrics>based on
vizGrimoire <http://vizgrimoire.bitergia.org/> and provided by
Bitergia<http://bitergia.com/>has been approved, and a first prototype
can be seen at
http://korma.wmflabs.org. The project starts effectively on July 1 and
includes a one-year period of maintenance. We agreed with the
Analytics<https://www.mediawiki.org/wiki/Analytics>team that they will
assume the responsibility of this area during this
period.
Kiwix <http://www.kiwix.org>

*The Kiwix project is funded and executed by Wikimedia
CH<https://meta.wikimedia.org/wiki/Wikimedia_CH>
.*
Development of a new MediaWiki HTML dumper in
nodeJS<http://sourceforge.net/p/kiwix/other/ci/master/tree/mwhtmldumper/>has
started. This tool exports Wikipedia articles in static files based on
the Parsoid <http://www.mediawiki.org/wiki/Parsoid> output. This solution
looks really promising, and new JavaScript developers are welcome.
Wikidata<https://meta.wikimedia.org/wiki/Wikidata>

*The Wikidata project is funded and executed by Wikimedia
Deutschland<https://meta.wikimedia.org/wiki/Wikimedia_Deutschland/en>
.*
June in Wikidata was all about the sister projects. The development team
published proposals for how Wikidata can support
Commons<https://commons.wikimedia.org/wiki/Commons:Wikidata_for_media_info>and
Wiktionary<http://lists.wikimedia.org/pipermail/wikidata-l/2013-June/002380.html>.
Additionally, they worked on the ability of Wikidata to store language
links to Wikivoyage in addition to Wikipedia; as a result, Wikivoyage will
soon also be able to manage their language links via Wikidata. Another
important step was the deployment of the geocoordinate datatype. This makes
it possible, for example, to indicate the location of a city.
Geocoordinates that are already in Wikidata can be seen on this
map<https://dl.dropboxusercontent.com/u/172199972/map.png>(huge
version <https://dl.dropboxusercontent.com/u/172199972/map_huge.png>,
updated daily). In a blog
entry<http://blog.wikimedia.de/2013/06/04/on-truths-and-lies/>,
Denny Vrandečić explained his understanding of the relation of Wikidata and
the truth. In other news, further development of Wikidata has been
supported through a large donation by the search engine company
Yandex<http://blog.wikimedia.de/2013/06/05/grosspende-fur-wikidata/>
. Future The engineering management team continues to update the *
Deployments <https://wikitech.wikimedia.org/wiki/Deployments>* page weekly,
providing up-to-date information on the upcoming deployments to Wikimedia
sites, as well as the *engineering
roadmap<https://www.mediawiki.org/wiki/Roadmap>
*, listing ongoing and future Wikimedia engineering efforts. *Annual
goals*<https://www.mediawiki.org/wiki/Wikimedia_Engineering/2013-14_Goals>for
the 2013–2014 fiscal year are currently being drafted.
--
This report was reviewed and proofread using VisualEditor.

--
Guillaume Paumier
Technical Communications Manager — Wikimedia Foundation
https://donate.wikimedia.org
-------------- next part --------------
_______________________________________________
Please note: all replies sent to this mailing list will be immediately directed to Wikimedia-l, the public mailing list of the Wikimedia community. For more information about Wikimedia-l:
https://lists.wikimedia.org/mailman/listinfo/wikimedia-l
_______________________________________________
WikimediaAnnounce-l mailing list
WikimediaAnnounce-l at lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikimediaannounce-l

More information about the Wikimedia-l mailing list