Hi,
The report covering Wikimedia engineering activities in May 2014 is now
available.
Wiki version:
https://www.mediawiki.org/wiki/Wikimedia_engineering_report/2014/May
Blog version:
https://blog.wikimedia.org/2014/06/15/engineering-report-may-2014/
We're also proposing a shorter, simpler and translatable version of this
report that does not assume specialized technical knowledge:
https://www.mediawiki.org/wiki/Wikimedia_engineering_report/2014/May/summary
Below is the HTML text of the report.
As always, feedback is appreciated on the usefulness of the report and its
summary, and on how to improve them.
------------------------------------------------------------------
Major news in May include:
- changes to the mobile site to better show the editors behind the
curtain
<https://blog.wikimedia.org/2014/05/02/the-wikipedia-editors-behind-the-curtain/>
;
- the announcement of CyrusOne in Dallas
<https://blog.wikimedia.org/2014/05/05/wikimedia-foundation-selects-cyrusone-in-dallas-as-new-data-center/>
as
the location of the new Wikimedia data center;
- the Zürich hackathon
<https://blog.wikimedia.org/2014/05/10/tech-wizards-behind-wikipedia-meet-in-zurich-for-hackathon/>
and Lila Tretikov’s perspective
<https://blog.wikimedia.org/2014/05/15/hacker-osmosis-ideas-european-hackathon-zurich/>
on
it;
- experiments by the Growth team to encourage more contributors to
register
<https://blog.wikimedia.org/2014/05/16/anonymous-editor-acquisition/>;
- the one-year anniversary
<https://blog.wikimedia.org/2014/05/20/celebrating-one-year-of-tech-news/>
of
the launch of Tech News;
- the launch of Wikipedia Zero in Nepal
<https://blog.wikimedia.org/2014/05/22/wikipedia-zero-shall-accelerate-wikipedia-in-nepal/>
in
partnership with NCELL;
- the launch of a second request for proposals
<https://blog.wikimedia.org/2014/05/27/request-for-proposals-mediawiki-release-management-round-2/>
for
the release management of MediaWiki for third-party users.
*Note: We’re also providing a shorter, simpler and translatable version of
this report
<https://www.mediawiki.org/wiki/Wikimedia_engineering_report/2014/May/summary>
that
does not assume specialized technical knowledge.*
Engineering metrics in May:
- 154 unique committers contributed patchsets of code to MediaWiki.
- The total number of unresolved commits
<https://gerrit.wikimedia.org/r/#q,status:open+project:%255Emediawiki.*,n,z>went
from around 1305 to about 1440.
- About 15 shell requests
<https://www.mediawiki.org/wiki/Shell_requests> were
processed.
Contents
- Personnel
<https://blog.wikimedia.org/2014/06/15/engineering-report-may-2014/#Personnel>
- Work with us
<https://blog.wikimedia.org/2014/06/15/engineering-report-may-2014/#Work_with_us>
- Announcements
<https://blog.wikimedia.org/2014/06/15/engineering-report-may-2014/#Announcements>
- Technical Operations
<https://blog.wikimedia.org/2014/06/15/engineering-report-may-2014/#Technical_Operations>
- Features Engineering
<https://blog.wikimedia.org/2014/06/15/engineering-report-may-2014/#Features_Engineering>
- Editor retention: Editing tools
<https://blog.wikimedia.org/2014/06/15/engineering-report-may-2014/#Editor_retention:_Editing_tools>
- Core Features
<https://blog.wikimedia.org/2014/06/15/engineering-report-may-2014/#Core_Features>
- Growth
<https://blog.wikimedia.org/2014/06/15/engineering-report-may-2014/#Growth>
- Support
<https://blog.wikimedia.org/2014/06/15/engineering-report-may-2014/#Support>
- Mobile
<https://blog.wikimedia.org/2014/06/15/engineering-report-may-2014/#Mobile>
- Language Engineering
<https://blog.wikimedia.org/2014/06/15/engineering-report-may-2014/#Language_Engineering>
- Platform Engineering
<https://blog.wikimedia.org/2014/06/15/engineering-report-may-2014/#Platform_Engineering>
- MediaWiki Core
<https://blog.wikimedia.org/2014/06/15/engineering-report-may-2014/#MediaWiki_Core>
- Quality assurance
<https://blog.wikimedia.org/2014/06/15/engineering-report-may-2014/#Quality_assurance>
- Multimedia
<https://blog.wikimedia.org/2014/06/15/engineering-report-may-2014/#Multimedia>
- Engineering Community Team
<https://blog.wikimedia.org/2014/06/15/engineering-report-may-2014/#Engineering_Community_Team>
- Analytics
<https://blog.wikimedia.org/2014/06/15/engineering-report-may-2014/#Analytics>
- Kiwix
<https://blog.wikimedia.org/2014/06/15/engineering-report-may-2014/#Kiwix>
- Wikidata
<https://blog.wikimedia.org/2014/06/15/engineering-report-may-2014/#Wikidata>
- Future
<https://blog.wikimedia.org/2014/06/15/engineering-report-may-2014/#Future>
Personnel Work with us <https://wikimediafoundation.org/wiki/Work_with_us>
Are you looking to work for Wikimedia? We have a lot of hiring coming up,
and we really love talking to active community members about these roles.
- VP of Engineering
<http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&page=Job%20Description&j=ods8Xfwu>
- ScrumMaster
<http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&page=Job%20Description&j=oSrSYfwT>
- Software Engineer -Frontend Developer (VisualEditor)
<http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&page=Job%20Description&j=o8jyYfwH>
- Software Engineer – Services
<http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&page=Job%20Description&j=oAhYYfwx>
- Software Engineer – Internationalization
<http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&page=Job%20Description&j=oH3gXfwH>
- Software Engineer – Frontend
<http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&page=Job%20Description&j=oxgWYfwr>
- QA Tester
<http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&page=Job%20Description&j=oKIUYfw4>
- Full Stack Developer
<http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&page=Job%20Description&j=orQ2Yfw1>
- Product Manager – Language Engineering
<http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&page=Job%20Description&j=osiMYfwe>
- Operations Security Engineer
<http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&page=Job%20Description&j=oT6cYfwT>
Announcements
- Filippo Giunchedi joined the Operations team as Operations Engineer (
announcement
<http://lists.wikimedia.org/pipermail/wikitech-l/2014-May/076308.html>).
- Rob Moen moved from the VisualEditor team to the Growth team (
announcement
<http://lists.wikimedia.org/pipermail/wikitech-l/2014-May/076344.html>).
- Bernd Sitzmann joined the Mobile App Team as Software Developer (
announcement
<http://lists.wikimedia.org/pipermail/wikitech-l/2014-May/076379.html>).
- Mukunda Modell joined the new Platform Engineering team as Release
Engineer (announcement
<http://lists.wikimedia.org/pipermail/wikitech-l/2014-May/076380.html>).
- Alex Monk joined the Features team as a contractor (announcement
<http://lists.wikimedia.org/pipermail/wikitech-l/2014-May/076429.html>).
- Abigail Ripstra joined the User Experience team as User Researcher
Lead (announcement
<http://lists.wikimedia.org/pipermail/wikitech-l/2014-May/076497.html>)
- Rachel diCerbo joined the Wikimedia Foundation as Director of
Community Engagement (Product) (announcement
<http://lists.wikimedia.org/pipermail/wikimediaannounce-l/2014-May/000921.html>
).
- Dan Duvall joined the Release and QA team as Automation Engineer (
announcement
<http://lists.wikimedia.org/pipermail/wikitech-l/2014-May/076697.html>).
Technical Operations
*New Dallas data center*
Planning for the new Dallas data center has continued in May, and basic
data infrastructure components such as racks, PDUs, network equipment and
various supplies have been ordered. About 140 servers have been sent from
our Tampa data center, to be installed in Dallas. Racking and configuration
work in our Dallas data center will commence in June.
Labs metrics in May:
- Number of projects: 162
- Number of instances: 392
- Amount of RAM in use (in MBs): 1,618,432
- Amount of allocated storage (in GBs): 17,860
- Number of virtual CPUs in use: 795
- Number of users: 3,259
*Wikimedia Labs*
Labs has been upgraded to use Puppet version 3; Ubuntu Trusty (14.04) is
now available for instances, and Tool Labs now features 787 tools from 508
maintainers.
Features Engineering
<https://www.mediawiki.org/wiki/Wikimedia_Features_engineering> Editor
retention: Editing tools
*VisualEditor <https://www.mediawiki.org/wiki/VisualEditor>*
In May, the VisualEditor team worked on the performance stability of the
editor, rolled out a major new feature to help users better edit articles,
and made some improvements to other features to increase their ease of use
and understandability, fixing 75 bugs and tickets
<https://bugzilla.wikimedia.org/buglist.cgi?list_id=318847&order=priority%2Cbug_severity&product=VisualEditor&query_format=advanced&resolution=FIXED&target_milestone=VE-deploy-2014-05-01&target_milestone=VE-deploy-2014-05-08&target_milestone=VE-deploy-2014-05-15&target_milestone=VE-deploy-2014-05-22&target_milestone=VE-deploy-2014-05-29>.
The new citation editor is now available to all VisualEditor users on the
English, Polish, and Czech Wikipedias, withinstructions
<https://www.mediawiki.org/wiki/VisualEditor/Citation_tool> on how to
enable it on other wikis. The citation and template dialogs were simplified
to avoid technical language and some outcomes that were unexpected for
users. As part of this, the citation icons were replaced with a new,
clearer set, and the template hinting system now lets wikis mark template
parameters as “suggested”, as a step below the existing “required” state.
The formula editor
<https://www.mediawiki.org/wiki/VisualEditor/Beta_Features/Formulae> is now
available to all VisualEditor users, and a new Beta Feature giving a tool
that lets you set the language of content
<https://www.mediawiki.org/wiki/VisualEditor/Beta_Features/Language> was
made available for testing and feedback. Following a new set of user
testing, the toolbar was tweaked, moving the list and indent buttons to a
drop-down to make them less prominent, and removing the gallery button
which is rarely used and confused users. The mobile version of
VisualEditor, currently available for alpha testers, was expanded to also
have the new citation editor available, and had some significant
performance improvements made, especially for long or complex pages. Work
continued on making VisualEditor more performant and reliable, and key
tasks like keyboard accessibility have progressed. The deployed version of
the code was updated five times in the regular release cycle (1.24-wmf3
<https://www.mediawiki.org/wiki/MediaWiki_1.24/wmf3#VisualEditor>,1.24-wmf4
<https://www.mediawiki.org/wiki/MediaWiki_1.24/wmf4#VisualEditor>, 1.24-wmf5
<https://www.mediawiki.org/wiki/MediaWiki_1.24/wmf5#VisualEditor>, 1.24-wmf6
<https://www.mediawiki.org/wiki/MediaWiki_1.24/wmf6#VisualEditor> and
1.24-wmf7 <https://www.mediawiki.org/wiki/MediaWiki_1.24/wmf7#VisualEditor>
).
*Parsoid <https://www.mediawiki.org/wiki/Parsoid>*
In May, the Parsoid team continued with ongoing bug fixes and bi-weekly
deployments. Besides the user-facing bug fixes, we also improved our
tracing support (to aid debugging), and did some performance improvements.
We also finished implementing support for HTML/visual editing of
transclusion parameters. This is not yet enabled in production while we
finish up any additional performance tweaks on it.
GSoC 2014 also kicked off in May; we have one student working on a wikilint
project
<https://www.mediawiki.org/wiki/Parsoid/Linting/GSoC_2014_Application> to
detect broken/bad wikitext in wiki pages.
We also started planning and charting goals for 2014/2015.
Core Features
*Flow <https://www.mediawiki.org/wiki/Flow/Project_information>*
In May, the Flow team prepared the new front-end redesign for expected
release in mid-June. We completed work on sorting topics on a board by most
recent activity, also for mid-June release. We changed hidden post handling
so that everyone can see hidden posts, including anonymous users.
Back-end improvements include optimizations on UUID handling and
standardized URL generation. We also merged Special:Flow for release; it’s
a community-created improvement that makes it easier to create redirects to
Flow boards. We also made no-JS fixes for topic submission and replies.
Bug fixes include: Firefox errors, WhatLinksHere fixes, special characters
in topic titles, topic creation on empty boards, curr and prev links in
board history for topic summaries, and cross-wiki issues with user name
lookup.
Growth
*Growth <https://www.mediawiki.org/wiki/Growth>*
<https://www.mediawiki.org/w/index.php?title=File:Growth_team_update_(June_2014).pdf&page=4>
<https://commons.wikimedia.org/wiki/File:Growth_team_update_(June_2014).pdf?page=4>
Growth team presentation slides from the monthly Metrics meeting
This month the Growth team launched its A/B test of two methods forasking
anonymous editors to sign up
<https://www.mediawiki.org/wiki/Anonymous_editor_acquisition/Signup_invites> on
English, German, French, and Italian Wikipedias. Full analysis of the test
results
<https://meta.wikimedia.org/wiki/Research:Anonymous_editor_acquisition/Signup_CTA_experiment>
is
expected in June, though preliminary data strongly suggests a positive
impact on new registrations. We finished the mw.cookie
<https://doc.wikimedia.org/mediawiki-core/master/js/#!/api/mw.cookie> module,
assisted by Timo Tijhof. Matt and Aaron participated in the Zürich
hackathon. Last but not least, Growth released two smaller enhancements to
our data collection regarding article creation, including adding page
identifiers to MediaWiki core deletion logs and tracking page restorations
<https://meta.wikimedia.org/wiki/Schema:PageRestoration> across all wikis.
Support
*Wikipedia Education Program
<https://www.mediawiki.org/wiki/Wikipedia_Education_Program>*
This month we fixed bugs and made some improvements to the Education
Program extension. The biggest change was Sage Ross’s addition of an API
for listing students enrolled in courses. Also, students from Facebook Open
Academy worked on a new notification and a new activity feed.
Mobile <https://www.mediawiki.org/wiki/Wikimedia_mobile_engineering>
*Wikimedia Apps <https://www.mediawiki.org/wiki/Wikimedia_Apps>*
This month, the Apps team worked on a series of navigation improvements to
the iOS and Android alpha apps, focusing on the UI for searching, saving
and sharing pages, and navigating to the table of contents. We also worked
on restyling the global navigation menu and article content—typography,
color, and spacing—to create a standardized experience across the mobile
web and apps. In preparation for the launch of the Android app in June, we
tackled a number of user-reported crashing bugs to ensure a more stable and
reliable experience for our users.
*Mobile web projects <https://www.mediawiki.org/wiki/Mobile_web_projects>*
This month the Mobile Web team continued to build out the basic features of
VisualEditor for tablet users, providing the ability to add references via
VisualEditor. We hope to finish refining the add and modify references
workflow in preparation for graduating VE for tablets to the stable mobile
site sometime in July. On the reader features side, we’ve pushed a number
of tablet-related styling improvements (typography, spacing, and Table of
Contents) to the stable mobile site. This should greatly improve the
reading experience for tablet users who are already accessing the mobile
version of our projects, and it is one of the last pieces of work we
planned to get done before we begin redirecting all tablet users to the
mobile site mid-June.
*Wikipedia Zero <https://www.mediawiki.org/wiki/Wikipedia_Zero>*
During the last month, the team restored IP address dynamic updates for
Wikipedia Zero partner configurations, advanced refactoring of
ZeroRatedMobileAccess into multiple extensions, added support for graceful
image quality reduction (roll-out for Wikipedia Zero will be carefully
approached), fixed an HTTPS-to-HTTP redirection bug, and worked on an RFC
for GIFification of banners instead of ESI. We also added MCC/MNC sampling
and necessary library support to the reboots of the Wikipedia apps, cut an
alpha Android build, performed limited app code review, added support for
Nokia (now MS Mobile) proxies (next step is to add zero-rating with
operators who have a Nokia proxy arrangement), diagnosed configuration
retrieval issues, simplified automation tests down to just banner presence
checks, ran SMS usage analysis and a one-off operator pageview analysis,
and started work with the Design team on the final polish for the Wikipedia
Zero experience in the forthcoming apps. Additionally, the team started its
annual engineering goal setting for the upcoming fiscal year. Routine pre-
and post-launch configuration changes were made to support operator
zero-rating, with routine technical assistance provided to operators and
the partner management team to help add zero-rating and address anomalies.
*Wikipedia Zero (partnerships)*
In May we launched Wikipedia Zero with Ncell in Nepal, Sky Mobile (Beeline)
in Kyrgyzstan and Airtel in Nigeria. We also added Opera Mini zero-rating
in Umniah in Jordan. We served roughly 67 million free page views in May
across 30 partners in 28 countries. Adele Vrana attended the Wikipedia
Education Hackathon in Jordan, where she collaborated with community
members from Egypt, Saudi Arabia, Yemen and Jordan. While there, she
visited Umniah, our local operator partner. Adele also went to Brazil,
where she met with prospective partners. We kicked off the carrier portal
UX design with Noble studios.
Language Engineering
<https://www.mediawiki.org/wiki/Wikimedia_Language_engineering>
*Milkshake <https://www.mediawiki.org/wiki/Milkshake>*
The Sanskrit keyboard was updated according to user requests.
CLDRPluralParser was relicensed under the MIT license for possible reuse in
upstream jQuery libraries.
*Language Engineering Communications and Outreach
<https://www.mediawiki.org/wiki/Language_engineering_communications_and_outreach>*
Niklas Laxström participated in the Zürich hackathon
<http://laxstrom.name/blag/2014/05/22/summary-of-translate-workshop-at-zurich-hackathon/>
.
*Content translation <https://www.mediawiki.org/wiki/Content_translation>*
Most of the team met in Valencia to complete the ContentTranslation
architecture and roadmap. The dictionary feature is now up for limited
testing <http://language-stage.wmflabs.org/wiki/Main_Page>.
Platform Engineering
<https://www.mediawiki.org/wiki/Wikimedia_Platform_Engineering> MediaWiki
Core
*Site performance and architecture
<https://www.mediawiki.org/wiki/Site_performance_and_architecture>*
Aaron Schulz has been reviewing the Petition extension for deployment to
the cluster, working with Peter Coombe to improve its performance. In
addition, the reliability and speed of media uploads was increased by
removing many failure cases on Commons. There were many other minor fixes
over the course of the month.
*HHVM <https://www.mediawiki.org/wiki/HHVM>*
HHVM is running on a test machine (“osmium”) in our production cluster.
Most of Tim Starling’s work on the Zend compatibility layer have landed in
HHVM 3.1 <http://hhvm.com/blog/5195/hhvm-3-1-0>. Most jobs are working, but
bugfixing continues on osmium.
*Release & QA
<https://www.mediawiki.org/wiki/Wikimedia_Release_and_QA_Team>*
The Release and QA team expanded this past month with the addition of
Mukunda Modell as a new Release Engineer. His first work is addressing the
remaining technical issues blocking our adoption of Phabricator as a
WMF-wide tool. Antoine Musso has begun drafting an RFC to outline how the
WMF would support isolated test environments for automatic builds. The WMF
kicked off the second RFP for the release management of MediaWiki
<https://www.mediawiki.org/wiki/Release_Management_RFP>. Chris McMahon
fleshed out tooling for creating test data at test time more widely along
with building off of Antoine’s work for using the appropriate version of
browser tests for testing specific mediawiki test installs (browser tests
are versioned along with the other code).
*Admin tools development
<https://www.mediawiki.org/wiki/Admin_tools_development>*
Minor improvements were made to admin tools projects including having
global blocks shown on Special:Contributions (bug 52673
<https://bugzilla.wikimedia.org/show_bug.cgi?id=52673>). Code review also
continued on the global rename user tool.
*Search <https://www.mediawiki.org/wiki/Search>*
In May, we deployed changes to improve snippets generated by Cirrus to a
handful of wikis, spent some time improving its analysis for Hebrew, and
adding more backwards compatibility with lsearchd’s syntax to Cirrus.
*Auth systems <https://www.mediawiki.org/wiki/Auth_systems>*
We worked on the SOA Authentication RFC
<https://www.mediawiki.org/wiki/Requests_for_comment/SOA_Authentication> to
support the Services team. We also created a MediaWiki-vagrant role for
CentralAuth, including significant work to support multiple wikis on a
single Vagrant instance. We continued work on the Phabricator
<https://www.mediawiki.org/wiki/Phabricator>-MediaWiki OAuth integration,
and the patch was upstreamed. Last, we held an OAuth training session at
the Zürich Hackathon
<https://www.mediawiki.org/wiki/Z%C3%BCrich_Hackathon_2014>, resulting in
several new apps using OAuth.
*Wikimania Scholarships app
<https://www.mediawiki.org/wiki/Wikimania_Scholarships_app>*
The Wikimania Scholarships project for the 2014 cycle wrapped up with the
final awarding of scholarships for Wikimania 2014. The current hope is that
the 2015 application and review cycle will be managed mainly by the
Wikimania 2015 organizing group with limited technical support from
Platform Engineering.
*Deployment tooling <https://www.mediawiki.org/wiki/Deployment_tooling>*
Bryan Davis continued the Pythonification of our deployment tooling with
the conversion of the remaining bits, namely all sync-* scripts
(sync-common, sync-file, sync-dir, and sync-db). Work continues on
modifying our deployment tooling to provide easier and more robust
automatic access to our version information (available atSpecial:Version
<https://www.mediawiki.org/wiki/Special:Version>).
*Security auditing and response
<https://www.mediawiki.org/wiki/Security_auditing_and_response>*
MediaWiki (1.22.7) was released to fix an XSS vulnerability. A separate DOM
XSS issue was fixed in MobileFrontend. We also finished a review of
Hadoop’s Camus.
Quality assurance
*Quality Assurance <https://www.mediawiki.org/wiki/Quality_Assurance>*
The expertise of our new Automation Engineer, Dan Duvall, will allow us to
update, maintain and modernize our development environments using Vagrant
virtual machines, keep our Puppet site configuration running properly, and
contribute to the whole delivery pipeline as we continue to improve our
ability to deploy software features to Wikimedia sites quickly and safely.
*Browser testing
<https://www.mediawiki.org/wiki/Quality_Assurance/Browser_testing>*
Now that we have the ability to address the MediaWiki API from the browser
testing framework, we have changed the existing test suites to use this
powerful tool. This not only gives us a test of the API itself, but also
makes the browser tests faster and more reliable. Furthermore, it allows us
to easily create a set of acceptance tests that will pass on any MediaWiki
installation regardless of what extensions exist or what language the wiki
is. We plan for these acceptance tests to eventually become part of
MediaWiki core, and our existing tests continue to expose important issues
in Wikimedia software development projects.
Multimedia
*Multimedia <https://www.mediawiki.org/wiki/Multimedia>*
<https://www.mediawiki.org/w/index.php?title=File:Upload_Wizard_Update_-_June_5_2014.pdf&page=8>
<https://commons.wikimedia.org/wiki/File:Upload_Wizard_Update_-_June_5_2014.pdf?page=8>
Metrics meeting presentation slides about UploadWizard
In May, the multimedia team released Media Viewer v0.2
<https://www.mediawiki.org/wiki/Multimedia/About_Media_Viewer> on more large
wikis
<https://www.mediawiki.org/wiki/Multimedia/Media_Viewer/Release_Plan#Large_Wikis>
(Dutch,
French, Italian, Japanese, Portuguese, Spanish and Russian Wikipedias),
with over 10 million image views daily
<http://multimedia-metrics.wmflabs.org/graphs/mmv_image_views_global>. This
multimedia browser has been well received: 70% of survey respondents
<https://www.mediawiki.org/wiki/Multimedia/Media_Viewer/Survey>find this
tool useful; based on this favorable feedback, we plan to deploy Media
Viewer on all wikis in June. Gilles Dubuc, Mark Holmquist, and Gergő Tisza
fixed more bugs and features during this development cycle
<https://wikimedia.mingle.thoughtworks.com/projects/multimedia/cards?favorite_id=11008&view=Completed+this+cycle>,
with design help from Pau Giner.
The team has switched its focus to new projects, starting with the
UploadWizard <https://www.mediawiki.org/wiki/UploadWizard>, our main
user-facing feature this year: this month, we collected metrics, reviewed
user feedback, created new designs, fixed bugs and refactored code as part
of a major upgrade of this important contribution tool. We also allocated a
third of our time to technical debt
<https://wikimedia.mingle.thoughtworks.com/projects/multimedia/cards?favorite_id=11321&view=Technical+Debt+Wall>
and
bug fixes for other multimedia tools, with an initial focus on improving
image scalers, GWToolset and TimedMediaHandler.
Fabrice Florin managed product development, hosting an annual planning
meeting <https://www.mediawiki.org/wiki/Multimedia/2013-14_Goals> to define
our goals for 2014−15: this year, we aim to engage more users to contribute
media to our sites through tools like UploadWizard, while implementing
structured data on Commons and continuing to address our technical debt and
fix critical bugs. Keegan Peterzell and Fabrice also continued to engage
our community partners throughout the release of Media Viewer
<https://www.mediawiki.org/wiki/Multimedia/Media_Viewer/Release_Plan>. We
are planning new discussions in coming weeks to improve our plans together.
For join these conversations and keep up with our work, we invite you to
subscribe to the multimedia mailing list
<https://lists.wikimedia.org/mailman/listinfo/multimedia>.
Engineering Community Team
<https://www.mediawiki.org/wiki/Engineering_Community_Team>
*Bug management <https://www.mediawiki.org/wiki/Bug_management>*
Mark Holmquist and Chad Horohoe changed Bugzilla to automatically handle
out “editbugs” permissions
<http://lists.wikimedia.org/pipermail/wikitech-l/2014-May/076723.html> to
new accounts and to be able to hand out “editbugs” recursively (bug 40497
<https://bugzilla.wikimedia.org/show_bug.cgi?id=40497>). Apart from usual
bugtriage and major focus on Phabricator migration work
<https://www.mediawiki.org/wiki/Phabricator/Migration#Status>, Andre
Klapper retriaged some open major/high Multimedia tickets, created some
requested components
<https://bugzilla.wikimedia.org/buglist.cgi?bug_id=64785,64784,64769,65150> and
investigated Gerrit bot notification breakage
<https://bugzilla.wikimedia.org/show_bug.cgi?id=65370> with Daniel Zahn and
Christian Aistleitner. He also made small changes
<https://www.mediawiki.org/w/index.php?title=Annoying_little_bugs&diff=1012532&oldid=945839>
to
the Annoying little bugs page based on feedback, and reorganized theWikipedia
App <https://bugzilla.wikimedia.org/show_bug.cgi?id=64621> and the Analytics
<https://bugzilla.wikimedia.org/show_bug.cgi?id=65903> products in
Bugzilla. Andre and Mark Hershberger updated some open Bugzilla tickets
with Target Milestone 1.23.0 at the Zürich Hackathon 2014
<https://www.mediawiki.org/wiki/Z%C3%BCrich_Hackathon_2014>.
*Phabricator migration
<https://www.mediawiki.org/wiki/Phabricator/Migration>*
Mukunda Modell is currently addressing authentication
<http://fab.wmflabs.org/T314> and access restrictions for security tickets
<http://fab.wmflabs.org/T95> with upstream. Chase Pettet and Daniel Zahn of
Wikimedia Operations are spearheading the Phabricator installation. Andre
Klapper has upstreamed some issues, commented on numerous tickets, and
identified further tasks related to migration. An overview board of tasks
to solve for the first day <http://fab.wmflabs.org/project/view/31/> of
Phabricator in production is available. Furthermore, once Wikimedia SUL
authentication is sorted out, it is investigated to launch the Phabricator
production instance first with very limited functionality to provide a Trusted
User Tool <http://fab.wmflabs.org/T364>.
*Mentorship programs <https://www.mediawiki.org/wiki/Mentorship_programs>*
We hosted a Q&A session on IRC
<https://www.mediawiki.org/wiki/Engineering_Community_Team/Meetings/2014-05-20>
with
a high participation of Google Summer of Code
<https://www.mediawiki.org/wiki/Google_Summer_of_Code_2014> and FOSS
Outreach Program for Women
<https://www.mediawiki.org/wiki/FOSS_Outreach_Program_for_Women/Round_8>
interns
and mentors right at the beginning of the development phase. Below are the
first project reports, including the lessons learned during the community
bonding period:
- Tools for mass migration of legacy translated wiki content
<https://www.mediawiki.org/wiki/Extension:Translate/Mass_migration_tools/Project_updates>
- Wikidata annotation tool
<https://www.mediawiki.org/wiki/Wikidata_annotation_tool/updates>
- Email bounce handling to MediaWiki with VERP
<https://www.mediawiki.org/wiki/VERP/GSOC_Progress_Rerport>
- Google Books, Internet Archive, Commons upload cycle
<https://www.mediawiki.org/wiki/Google_Books,_Internet_Archive,_Commons_upload_cycle/Progress>
- UniversalLanguageSelector fonts for Chinese wikis
<https://www.mediawiki.org/wiki/Extension:UniversalLanguageSelector/Fonts_for_Chinese_wikis#Weekly_Report>
- MassMessage page input list improvements
<https://www.mediawiki.org/wiki/Extension:MassMessage/Page_input_list_improvements/Progress_reports>
- Book management in Wikibooks/Wikisource
<https://meta.wikimedia.org/wiki/Book_management_2014/Progress>
- Parsoid-based online-detection of broken wikitext
<https://www.mediawiki.org/wiki/User:Hardik95/GSoC_2014_Progress_Report>
- Usability improvements for the Translate extension
<https://www.mediawiki.org/wiki/User:Kunalgrover05/Progress_Report>
- A modern, scalable and attractive skin for MediaWiki
<https://www.mediawiki.org/wiki/User:Jack_Phoenix/GSoC_2014>
- Automatic cross-language screenshots for user documentation
<https://www.mediawiki.org/wiki/Automatic_cross-language_screenshots/progress>
- Separating skins from core MediaWiki
<https://www.mediawiki.org/wiki/Separating_skins_from_core_MediaWiki/Progress>
- Chemical Markup support for Wikimedia Commons
<https://www.mediawiki.org/wiki/Chemical_Markup_support_for_Wikimedia_Commons/Internship_Report>
- Improving URL citations on Wikimedia
<https://www.mediawiki.org/wiki/User:Mvolz/Weekly_Reports>
- Historical OpenStreetMap
<https://www.mediawiki.org/wiki/User:JaimeLyn/Weekly_Reports>
- Welcoming new contributors to Wikimedia Labs and Tool Labs
<https://www.mediawiki.org/wiki/Welcome_to_labs/Progress_Reports>
- Evaluating, documenting, and improving MediaWiki web API client
libraries
<https://www.mediawiki.org/wiki/Evaluating_and_Improving_MediaWiki_web_API_client_libraries/Progress_Reports>
- Feed the Gnomes – Wikidata Outreach
<https://www.mediawiki.org/wiki/User:Thepwnco/OPW_Reporting>
- Template Matching for RDFIO
<https://www.mediawiki.org/wiki/Extension:RDFIO/Template_matching_for_RDFIO/Reports>
- Switching Semantic Forms Autocompletion to Select2
<https://www.mediawiki.org/wiki/Extension:Semantic_Forms/Select2_for_autocompletion/Progress_Report>
- Catalogue for Mediawiki Extensions
<https://www.mediawiki.org/wiki/User:Adi.iiita/Gsoc2014/Report#Weekly_Report>
- Generic, efficient localisation update service
<https://www.mediawiki.org/wiki/Extension:LocalisationUpdate/LUv2/Updates>
.
*Technical communications
<https://www.mediawiki.org/wiki/Technical_communications>*
In addition to ongoing communications support
<https://www.mediawiki.org/wiki/Technical_communications/Tech_blog_activity>
for
the engineering staff, Guillaume Paumier
<https://www.mediawiki.org/wiki/User:Guillom> mostly focused on
improvements to the system of scripts and templates
<https://www.mediawiki.org/wiki/Category:Wikimedia_engineering_templates> used
to document Wikimedia engineering activities on
mediawiki.org. Active and
inactive projects can now be queried separately, which means that the list
of projects that appears in the drop-down of the status helper gadget
<https://www.mediawiki.org/wiki/MediaWiki:Gadget-WmfProjectStatusHelper.js> is
much shorter, now only listing active projects. Guillaume also wrote a
dedicated Lua module
<https://www.mediawiki.org/wiki/Module:Wikimedia_engineering> to manipulate
engineering activities automatically, in particular for theWikimedia
Engineering <https://www.mediawiki.org/wiki/Wikimedia_Engineering> portal.
It is now possible to query activities from a given team, and display them
on any page in various formats. Using the module, the ever-outdated list of
“current” activities on the portal was replaced by an
automatically-generated list based on projects listed on team hubs. The
module also allows to feature a random engineering activity on the portal.
Other additions to the portal include the latest issue of Tech News
<https://meta.wikimedia.org/wiki/Tech/News>(transcluded and automatically
updated every week), as well as the first paragraph of the latest monthly
engineering report (manually updated for now). Future improvements of the
portal are expected to be mostly aesthetic.
*Volunteer coordination and outreach
<https://www.mediawiki.org/wiki/Volunteer_coordination_and_outreach>*
The Wikimedia Hackathon in Zürich
<https://www.mediawiki.org/wiki/Z%C3%BCrich_Hackathon_2014> was a success
according to ad hoc feedback from the participants. A deeper review is
expected to be published in July, after compiling the results of the
survey. The main merit goes to Wikimedia CH for an efficient, warm, and
flexible organization. We also announced a process to request the
organization of Hackathons <https://www.mediawiki.org/wiki/Hackathons>. We
had an intense calendar of events in May
<https://www.mediawiki.org/wiki/Project:Calendar/2014/05>, including a Tech
Talk about Elasticsearch
<https://plus.google.com/b/103470172168784626509/events/cokipb2senmmvkvdjif7aq55kac>
and
a meetup in San Francisco on *Making Wikipedia Fast*
<https://plus.google.com/events/ce6kihklfld2p10ep28p5ia7klg>, organized
successfully together with the Web Performance SF meetup.
*Architecture and Requests for comment process
<https://www.mediawiki.org/wiki/Architecture_process>*
Engineers discussed the Performance guidelines
<https://www.mediawiki.org/wiki/Performance_guidelines> at the Zürich
Hackathon 2014 <https://www.mediawiki.org/wiki/Z%C3%BCrich_Hackathon_2014>.
They also had IRC discussions of several requests for comment and documents:
- 2014-05-14 — Discussion
<https://www.mediawiki.org/wiki/Architecture_meetings/Performance_guidelines_discussion_2014-05-14>
of Performance guidelines
<https://www.mediawiki.org/wiki/Performance_guidelines> draft
- 2014-05-21 — Discussion
<https://www.mediawiki.org/wiki/Architecture_meetings/RFC_review_2014-05-21>
of Typesafe enums
<https://www.mediawiki.org/wiki/Requests_for_comment/Typesafe_enums>
and Square
bounding boxes
<https://www.mediawiki.org/wiki/Requests_for_comment/Square_bounding_boxes>
- 2014-05-30 — Discussion
<https://www.mediawiki.org/wiki/Architecture_meetings/RFC_review_2014-05-30>
of Extension registration
<https://www.mediawiki.org/wiki/Requests_for_comment/Extension_registration>
- 2014-06-02 — Discussion
<https://www.mediawiki.org/wiki/Architecture_meetings/RFC_review_2014-06-02>
of Grid system
<https://www.mediawiki.org/wiki/Requests_for_comment/Grid_system>.
Analytics <https://www.mediawiki.org/wiki/Analytics>
*Wikimetrics <https://www.mediawiki.org/wiki/Analytics/Wikimetrics>*
The focus for this month was on extending Wikimetrics to support the Editor
Engagement Vital Signs
<https://www.mediawiki.org/wiki/Analytics/Editor_Engagement_Vital_Signs> (EEVS)
project. The team also fixed several bugs around Unicode support
(particularly non-Latin character sets) and implemented delete cohort
functionality.
*Data Processing <https://www.mediawiki.org/wiki/Analytics/Data_Processing>*
Capacity, deployment and CDH 5 (new Hadoop) version was worked on this
month. These initiatives should be resolved in June. A permissions issue
caused the page view dumps to stall for a weekend. The system was fixed
promptly and no data was lost.
*Editor Engagement Vital Signs
<https://www.mediawiki.org/wiki/Analytics/Editor_Engagement_Vital_Signs>*
We created a headless user to run recurrent reports and groundwork to
support creating new metrics. The team also discussed potential changes to
the first round of metrics implemented in the system to support a broader
view of participation, and reviewed mockups provided by the design team
with stakeholders.
*EventLogging <https://www.mediawiki.org/wiki/Analytics/EventLogging>*
The team has adopted event logging
<http://lists.wikimedia.org/pipermail/analytics/2014-June/002202.html> and
has identified some monitoring updates that need to be made.
Kiwix <http://www.kiwix.org/>
*The Kiwix project is funded and executed by Wikimedia CH
<https://meta.wikimedia.org/wiki/Wikimedia_CH>.*
The Hackathon in Zürich was our highlight for May. There, most of the team
met, and a few new people joined the development effort during the 3 days
in common with the Wikimedia hackathon. Most of the work done focused on
preparing the final Kiwix 0.9 release, to be released in June. It was also
necessary to release a new minor release (1.2)
<https://fr.wikipedia.org/wiki/Projet:Afrip%C3%A9dia> of libzim. Work on Kiwix
for Android
<https://play.google.com/store/apps/details?id=org.kiwix.kiwixmobile> continues
with the integration of a content/download manager. On the offline content
side, 6 ZIM files with all TED talks
<http://lists.wikimedia.org/pipermail/offline-l/2014-June/001254.html> were
also released; it’s the first time we provide files with so much multimedia
content. 50 USB flash drives with Kiwix and the *Wikipedia for schools* were
prepared and sent toWikiIndaba <http://wikiindaba.net/>; 15 Kiwix-plug
<http://www.kiwix.org/wiki/Kiwix-plug> are also going to be prepared for
Afripedia <https://fr.wikipedia.org/wiki/Projet:Afrip%C3%A9dia>. Finally,
in the scope of the Malebooks <http://malebooks.ml/>project, work to
prepare an offline version of the Gutenberg project
<http://www.gutenberg.org/>, with its 45,000 public domain books, has
started.
Wikidata <https://meta.wikimedia.org/wiki/Wikidata>
*The Wikidata project is funded and executed by Wikimedia Deutschland
<https://meta.wikimedia.org/wiki/Wikimedia_Deutschland/en>.*
It has been a busy month for Wikidata. The development team made
significant progress on a number of important features. These include
simple queries, the mono-lingual text datatype and redirects. We’ve also
done a lot of research for the upcoming user interface change and started
making mockups for it. Lydia Pintscher held an office hour on IRC
<https://meta.wikimedia.org/wiki/IRC_office_hours/Office_hours_2014-05-19> and
answered a lot of questions. Two interns (Anjali Sharma and Helen
Halbert) started
working with the team
<http://lists.wikimedia.org/pipermail/wikidata-l/2014-April/003753.html>.
They will improve the user documentation and social media outreach around
Wikidata. Wikidata was also well represented at the MediaWiki hackathon in
Zürich
<https://blog.wikimedia.de/2014/05/13/thank-you-zurich-merci-vielmal-zueri-wikimedia-hackathon-zuerich-2014/>.
Last but not least, Magnus Manske developed a number ofgames around Wikidata
<https://tools.wmflabs.org/wikidata-game/> that help improve the data in it
and add more. It’s a resounding success so far.
FutureThe engineering management team continues to update the *Deployments
<https://wikitech.wikimedia.org/wiki/Deployments>* page weekly, providing
up-to-date information on the upcoming deployments to Wikimedia sites, as
well as the *annual goals
<https://www.mediawiki.org/wiki/Wikimedia_Engineering/2014-15_Goals>*,
listing ongoing and future Wikimedia engineering efforts.
------------------------------
*This article was written collaboratively by Wikimedia engineers and
managers. See revision history
<https://www.mediawiki.org/w/index.php?title=Wikimedia_engineering_report/2014/May&action=history>
and
associated status pages. A wiki version
<https://www.mediawiki.org/wiki/Wikimedia_engineering_report/2014/May> is
also available.*
--
Guillaume Paumier
Technical Communications Manager — Wikimedia Foundation