Hi,
The report covering Wikimedia engineering activities in May 2014 is now available.
Wiki version: https://www.mediawiki.org/wiki/Wikimedia_engineering_report/2014/May Blog version: https://blog.wikimedia.org/2014/06/15/engineering-report-may-2014/
We're also proposing a shorter, simpler and translatable version of this report that does not assume specialized technical knowledge: https://www.mediawiki.org/wiki/Wikimedia_engineering_report/2014/May/summary
Below is the HTML text of the report.
As always, feedback is appreciated on the usefulness of the report and its summary, and on how to improve them.
------------------------------------------------------------------
Major news in May include:
- changes to the mobile site to better show the editors behind the curtain https://blog.wikimedia.org/2014/05/02/the-wikipedia-editors-behind-the-curtain/ ; - the announcement of CyrusOne in Dallas https://blog.wikimedia.org/2014/05/05/wikimedia-foundation-selects-cyrusone-in-dallas-as-new-data-center/ as the location of the new Wikimedia data center; - the Zürich hackathon https://blog.wikimedia.org/2014/05/10/tech-wizards-behind-wikipedia-meet-in-zurich-for-hackathon/ and Lila Tretikov’s perspective https://blog.wikimedia.org/2014/05/15/hacker-osmosis-ideas-european-hackathon-zurich/ on it; - experiments by the Growth team to encourage more contributors to register https://blog.wikimedia.org/2014/05/16/anonymous-editor-acquisition/; - the one-year anniversary https://blog.wikimedia.org/2014/05/20/celebrating-one-year-of-tech-news/ of the launch of Tech News; - the launch of Wikipedia Zero in Nepal https://blog.wikimedia.org/2014/05/22/wikipedia-zero-shall-accelerate-wikipedia-in-nepal/ in partnership with NCELL; - the launch of a second request for proposals https://blog.wikimedia.org/2014/05/27/request-for-proposals-mediawiki-release-management-round-2/ for the release management of MediaWiki for third-party users.
*Note: We’re also providing a shorter, simpler and translatable version of this report https://www.mediawiki.org/wiki/Wikimedia_engineering_report/2014/May/summary that does not assume specialized technical knowledge.*
Engineering metrics in May:
- 154 unique committers contributed patchsets of code to MediaWiki. - The total number of unresolved commits https://gerrit.wikimedia.org/r/#q,status:open+project:%255Emediawiki.*,n,zwent from around 1305 to about 1440. - About 15 shell requests https://www.mediawiki.org/wiki/Shell_requests were processed.
Contents
- Personnel https://blog.wikimedia.org/2014/06/15/engineering-report-may-2014/#Personnel - Work with us https://blog.wikimedia.org/2014/06/15/engineering-report-may-2014/#Work_with_us - Announcements https://blog.wikimedia.org/2014/06/15/engineering-report-may-2014/#Announcements - Technical Operations https://blog.wikimedia.org/2014/06/15/engineering-report-may-2014/#Technical_Operations - Features Engineering https://blog.wikimedia.org/2014/06/15/engineering-report-may-2014/#Features_Engineering - Editor retention: Editing tools https://blog.wikimedia.org/2014/06/15/engineering-report-may-2014/#Editor_retention:_Editing_tools - Core Features https://blog.wikimedia.org/2014/06/15/engineering-report-may-2014/#Core_Features - Growth https://blog.wikimedia.org/2014/06/15/engineering-report-may-2014/#Growth - Support https://blog.wikimedia.org/2014/06/15/engineering-report-may-2014/#Support - Mobile https://blog.wikimedia.org/2014/06/15/engineering-report-may-2014/#Mobile - Language Engineering https://blog.wikimedia.org/2014/06/15/engineering-report-may-2014/#Language_Engineering - Platform Engineering https://blog.wikimedia.org/2014/06/15/engineering-report-may-2014/#Platform_Engineering - MediaWiki Core https://blog.wikimedia.org/2014/06/15/engineering-report-may-2014/#MediaWiki_Core - Quality assurance https://blog.wikimedia.org/2014/06/15/engineering-report-may-2014/#Quality_assurance - Multimedia https://blog.wikimedia.org/2014/06/15/engineering-report-may-2014/#Multimedia - Engineering Community Team https://blog.wikimedia.org/2014/06/15/engineering-report-may-2014/#Engineering_Community_Team - Analytics https://blog.wikimedia.org/2014/06/15/engineering-report-may-2014/#Analytics - Kiwix https://blog.wikimedia.org/2014/06/15/engineering-report-may-2014/#Kiwix - Wikidata https://blog.wikimedia.org/2014/06/15/engineering-report-may-2014/#Wikidata - Future https://blog.wikimedia.org/2014/06/15/engineering-report-may-2014/#Future
Personnel Work with us https://wikimediafoundation.org/wiki/Work_with_us
Are you looking to work for Wikimedia? We have a lot of hiring coming up, and we really love talking to active community members about these roles.
- VP of Engineering http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&page=Job%20Description&j=ods8Xfwu - ScrumMaster http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&page=Job%20Description&j=oSrSYfwT - Software Engineer -Frontend Developer (VisualEditor) http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&page=Job%20Description&j=o8jyYfwH - Software Engineer – Services http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&page=Job%20Description&j=oAhYYfwx - Software Engineer – Internationalization http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&page=Job%20Description&j=oH3gXfwH - Software Engineer – Frontend http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&page=Job%20Description&j=oxgWYfwr - QA Tester http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&page=Job%20Description&j=oKIUYfw4 - Full Stack Developer http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&page=Job%20Description&j=orQ2Yfw1 - Product Manager – Language Engineering http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&page=Job%20Description&j=osiMYfwe - Operations Security Engineer http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&page=Job%20Description&j=oT6cYfwT
Announcements
- Filippo Giunchedi joined the Operations team as Operations Engineer ( announcement http://lists.wikimedia.org/pipermail/wikitech-l/2014-May/076308.html). - Rob Moen moved from the VisualEditor team to the Growth team ( announcement http://lists.wikimedia.org/pipermail/wikitech-l/2014-May/076344.html). - Bernd Sitzmann joined the Mobile App Team as Software Developer ( announcement http://lists.wikimedia.org/pipermail/wikitech-l/2014-May/076379.html). - Mukunda Modell joined the new Platform Engineering team as Release Engineer (announcement http://lists.wikimedia.org/pipermail/wikitech-l/2014-May/076380.html). - Alex Monk joined the Features team as a contractor (announcement http://lists.wikimedia.org/pipermail/wikitech-l/2014-May/076429.html). - Abigail Ripstra joined the User Experience team as User Researcher Lead (announcement http://lists.wikimedia.org/pipermail/wikitech-l/2014-May/076497.html) - Rachel diCerbo joined the Wikimedia Foundation as Director of Community Engagement (Product) (announcement http://lists.wikimedia.org/pipermail/wikimediaannounce-l/2014-May/000921.html ). - Dan Duvall joined the Release and QA team as Automation Engineer ( announcement http://lists.wikimedia.org/pipermail/wikitech-l/2014-May/076697.html).
Technical Operations
*New Dallas data center* Planning for the new Dallas data center has continued in May, and basic data infrastructure components such as racks, PDUs, network equipment and various supplies have been ordered. About 140 servers have been sent from our Tampa data center, to be installed in Dallas. Racking and configuration work in our Dallas data center will commence in June.
Labs metrics in May:
- Number of projects: 162 - Number of instances: 392 - Amount of RAM in use (in MBs): 1,618,432 - Amount of allocated storage (in GBs): 17,860 - Number of virtual CPUs in use: 795 - Number of users: 3,259
*Wikimedia Labs* Labs has been upgraded to use Puppet version 3; Ubuntu Trusty (14.04) is now available for instances, and Tool Labs now features 787 tools from 508 maintainers. Features Engineering https://www.mediawiki.org/wiki/Wikimedia_Features_engineering Editor retention: Editing tools
*VisualEditor https://www.mediawiki.org/wiki/VisualEditor* In May, the VisualEditor team worked on the performance stability of the editor, rolled out a major new feature to help users better edit articles, and made some improvements to other features to increase their ease of use and understandability, fixing 75 bugs and tickets https://bugzilla.wikimedia.org/buglist.cgi?list_id=318847&order=priority%2Cbug_severity&product=VisualEditor&query_format=advanced&resolution=FIXED&target_milestone=VE-deploy-2014-05-01&target_milestone=VE-deploy-2014-05-08&target_milestone=VE-deploy-2014-05-15&target_milestone=VE-deploy-2014-05-22&target_milestone=VE-deploy-2014-05-29. The new citation editor is now available to all VisualEditor users on the English, Polish, and Czech Wikipedias, withinstructions https://www.mediawiki.org/wiki/VisualEditor/Citation_tool on how to enable it on other wikis. The citation and template dialogs were simplified to avoid technical language and some outcomes that were unexpected for users. As part of this, the citation icons were replaced with a new, clearer set, and the template hinting system now lets wikis mark template parameters as “suggested”, as a step below the existing “required” state. The formula editor https://www.mediawiki.org/wiki/VisualEditor/Beta_Features/Formulae is now available to all VisualEditor users, and a new Beta Feature giving a tool that lets you set the language of content https://www.mediawiki.org/wiki/VisualEditor/Beta_Features/Language was made available for testing and feedback. Following a new set of user testing, the toolbar was tweaked, moving the list and indent buttons to a drop-down to make them less prominent, and removing the gallery button which is rarely used and confused users. The mobile version of VisualEditor, currently available for alpha testers, was expanded to also have the new citation editor available, and had some significant performance improvements made, especially for long or complex pages. Work continued on making VisualEditor more performant and reliable, and key tasks like keyboard accessibility have progressed. The deployed version of the code was updated five times in the regular release cycle (1.24-wmf3 https://www.mediawiki.org/wiki/MediaWiki_1.24/wmf3#VisualEditor,1.24-wmf4 https://www.mediawiki.org/wiki/MediaWiki_1.24/wmf4#VisualEditor, 1.24-wmf5 https://www.mediawiki.org/wiki/MediaWiki_1.24/wmf5#VisualEditor, 1.24-wmf6 https://www.mediawiki.org/wiki/MediaWiki_1.24/wmf6#VisualEditor and 1.24-wmf7 https://www.mediawiki.org/wiki/MediaWiki_1.24/wmf7#VisualEditor ).
*Parsoid https://www.mediawiki.org/wiki/Parsoid*
In May, the Parsoid team continued with ongoing bug fixes and bi-weekly deployments. Besides the user-facing bug fixes, we also improved our tracing support (to aid debugging), and did some performance improvements. We also finished implementing support for HTML/visual editing of transclusion parameters. This is not yet enabled in production while we finish up any additional performance tweaks on it.
GSoC 2014 also kicked off in May; we have one student working on a wikilint project https://www.mediawiki.org/wiki/Parsoid/Linting/GSoC_2014_Application to detect broken/bad wikitext in wiki pages.
We also started planning and charting goals for 2014/2015. Core Features
*Flow https://www.mediawiki.org/wiki/Flow/Project_information*
In May, the Flow team prepared the new front-end redesign for expected release in mid-June. We completed work on sorting topics on a board by most recent activity, also for mid-June release. We changed hidden post handling so that everyone can see hidden posts, including anonymous users.
Back-end improvements include optimizations on UUID handling and standardized URL generation. We also merged Special:Flow for release; it’s a community-created improvement that makes it easier to create redirects to Flow boards. We also made no-JS fixes for topic submission and replies.
Bug fixes include: Firefox errors, WhatLinksHere fixes, special characters in topic titles, topic creation on empty boards, curr and prev links in board history for topic summaries, and cross-wiki issues with user name lookup. Growth
*Growth https://www.mediawiki.org/wiki/Growth*
https://www.mediawiki.org/w/index.php?title=File:Growth_team_update_(June_2014).pdf&page=4
https://commons.wikimedia.org/wiki/File:Growth_team_update_(June_2014).pdf?page=4
Growth team presentation slides from the monthly Metrics meeting
This month the Growth team launched its A/B test of two methods forasking anonymous editors to sign up https://www.mediawiki.org/wiki/Anonymous_editor_acquisition/Signup_invites on English, German, French, and Italian Wikipedias. Full analysis of the test results https://meta.wikimedia.org/wiki/Research:Anonymous_editor_acquisition/Signup_CTA_experiment is expected in June, though preliminary data strongly suggests a positive impact on new registrations. We finished the mw.cookie https://doc.wikimedia.org/mediawiki-core/master/js/#!/api/mw.cookie module, assisted by Timo Tijhof. Matt and Aaron participated in the Zürich hackathon. Last but not least, Growth released two smaller enhancements to our data collection regarding article creation, including adding page identifiers to MediaWiki core deletion logs and tracking page restorations https://meta.wikimedia.org/wiki/Schema:PageRestoration across all wikis. Support
*Wikipedia Education Program https://www.mediawiki.org/wiki/Wikipedia_Education_Program* This month we fixed bugs and made some improvements to the Education Program extension. The biggest change was Sage Ross’s addition of an API for listing students enrolled in courses. Also, students from Facebook Open Academy worked on a new notification and a new activity feed. Mobile https://www.mediawiki.org/wiki/Wikimedia_mobile_engineering
*Wikimedia Apps https://www.mediawiki.org/wiki/Wikimedia_Apps* This month, the Apps team worked on a series of navigation improvements to the iOS and Android alpha apps, focusing on the UI for searching, saving and sharing pages, and navigating to the table of contents. We also worked on restyling the global navigation menu and article content—typography, color, and spacing—to create a standardized experience across the mobile web and apps. In preparation for the launch of the Android app in June, we tackled a number of user-reported crashing bugs to ensure a more stable and reliable experience for our users.
*Mobile web projects https://www.mediawiki.org/wiki/Mobile_web_projects* This month the Mobile Web team continued to build out the basic features of VisualEditor for tablet users, providing the ability to add references via VisualEditor. We hope to finish refining the add and modify references workflow in preparation for graduating VE for tablets to the stable mobile site sometime in July. On the reader features side, we’ve pushed a number of tablet-related styling improvements (typography, spacing, and Table of Contents) to the stable mobile site. This should greatly improve the reading experience for tablet users who are already accessing the mobile version of our projects, and it is one of the last pieces of work we planned to get done before we begin redirecting all tablet users to the mobile site mid-June.
*Wikipedia Zero https://www.mediawiki.org/wiki/Wikipedia_Zero* During the last month, the team restored IP address dynamic updates for Wikipedia Zero partner configurations, advanced refactoring of ZeroRatedMobileAccess into multiple extensions, added support for graceful image quality reduction (roll-out for Wikipedia Zero will be carefully approached), fixed an HTTPS-to-HTTP redirection bug, and worked on an RFC for GIFification of banners instead of ESI. We also added MCC/MNC sampling and necessary library support to the reboots of the Wikipedia apps, cut an alpha Android build, performed limited app code review, added support for Nokia (now MS Mobile) proxies (next step is to add zero-rating with operators who have a Nokia proxy arrangement), diagnosed configuration retrieval issues, simplified automation tests down to just banner presence checks, ran SMS usage analysis and a one-off operator pageview analysis, and started work with the Design team on the final polish for the Wikipedia Zero experience in the forthcoming apps. Additionally, the team started its annual engineering goal setting for the upcoming fiscal year. Routine pre- and post-launch configuration changes were made to support operator zero-rating, with routine technical assistance provided to operators and the partner management team to help add zero-rating and address anomalies.
*Wikipedia Zero (partnerships)* In May we launched Wikipedia Zero with Ncell in Nepal, Sky Mobile (Beeline) in Kyrgyzstan and Airtel in Nigeria. We also added Opera Mini zero-rating in Umniah in Jordan. We served roughly 67 million free page views in May across 30 partners in 28 countries. Adele Vrana attended the Wikipedia Education Hackathon in Jordan, where she collaborated with community members from Egypt, Saudi Arabia, Yemen and Jordan. While there, she visited Umniah, our local operator partner. Adele also went to Brazil, where she met with prospective partners. We kicked off the carrier portal UX design with Noble studios. Language Engineering https://www.mediawiki.org/wiki/Wikimedia_Language_engineering
*Milkshake https://www.mediawiki.org/wiki/Milkshake* The Sanskrit keyboard was updated according to user requests. CLDRPluralParser was relicensed under the MIT license for possible reuse in upstream jQuery libraries.
*Language Engineering Communications and Outreach https://www.mediawiki.org/wiki/Language_engineering_communications_and_outreach* Niklas Laxström participated in the Zürich hackathon http://laxstrom.name/blag/2014/05/22/summary-of-translate-workshop-at-zurich-hackathon/ .
*Content translation https://www.mediawiki.org/wiki/Content_translation* Most of the team met in Valencia to complete the ContentTranslation architecture and roadmap. The dictionary feature is now up for limited testing http://language-stage.wmflabs.org/wiki/Main_Page. Platform Engineering https://www.mediawiki.org/wiki/Wikimedia_Platform_Engineering MediaWiki Core
*Site performance and architecture https://www.mediawiki.org/wiki/Site_performance_and_architecture* Aaron Schulz has been reviewing the Petition extension for deployment to the cluster, working with Peter Coombe to improve its performance. In addition, the reliability and speed of media uploads was increased by removing many failure cases on Commons. There were many other minor fixes over the course of the month.
*HHVM https://www.mediawiki.org/wiki/HHVM* HHVM is running on a test machine (“osmium”) in our production cluster. Most of Tim Starling’s work on the Zend compatibility layer have landed in HHVM 3.1 http://hhvm.com/blog/5195/hhvm-3-1-0. Most jobs are working, but bugfixing continues on osmium.
*Release & QA https://www.mediawiki.org/wiki/Wikimedia_Release_and_QA_Team* The Release and QA team expanded this past month with the addition of Mukunda Modell as a new Release Engineer. His first work is addressing the remaining technical issues blocking our adoption of Phabricator as a WMF-wide tool. Antoine Musso has begun drafting an RFC to outline how the WMF would support isolated test environments for automatic builds. The WMF kicked off the second RFP for the release management of MediaWiki https://www.mediawiki.org/wiki/Release_Management_RFP. Chris McMahon fleshed out tooling for creating test data at test time more widely along with building off of Antoine’s work for using the appropriate version of browser tests for testing specific mediawiki test installs (browser tests are versioned along with the other code).
*Admin tools development https://www.mediawiki.org/wiki/Admin_tools_development* Minor improvements were made to admin tools projects including having global blocks shown on Special:Contributions (bug 52673 https://bugzilla.wikimedia.org/show_bug.cgi?id=52673). Code review also continued on the global rename user tool.
*Search https://www.mediawiki.org/wiki/Search* In May, we deployed changes to improve snippets generated by Cirrus to a handful of wikis, spent some time improving its analysis for Hebrew, and adding more backwards compatibility with lsearchd’s syntax to Cirrus.
*Auth systems https://www.mediawiki.org/wiki/Auth_systems* We worked on the SOA Authentication RFC https://www.mediawiki.org/wiki/Requests_for_comment/SOA_Authentication to support the Services team. We also created a MediaWiki-vagrant role for CentralAuth, including significant work to support multiple wikis on a single Vagrant instance. We continued work on the Phabricator https://www.mediawiki.org/wiki/Phabricator-MediaWiki OAuth integration, and the patch was upstreamed. Last, we held an OAuth training session at the Zürich Hackathon https://www.mediawiki.org/wiki/Z%C3%BCrich_Hackathon_2014, resulting in several new apps using OAuth.
*Wikimania Scholarships app https://www.mediawiki.org/wiki/Wikimania_Scholarships_app* The Wikimania Scholarships project for the 2014 cycle wrapped up with the final awarding of scholarships for Wikimania 2014. The current hope is that the 2015 application and review cycle will be managed mainly by the Wikimania 2015 organizing group with limited technical support from Platform Engineering.
*Deployment tooling https://www.mediawiki.org/wiki/Deployment_tooling* Bryan Davis continued the Pythonification of our deployment tooling with the conversion of the remaining bits, namely all sync-* scripts (sync-common, sync-file, sync-dir, and sync-db). Work continues on modifying our deployment tooling to provide easier and more robust automatic access to our version information (available atSpecial:Version https://www.mediawiki.org/wiki/Special:Version).
*Security auditing and response https://www.mediawiki.org/wiki/Security_auditing_and_response* MediaWiki (1.22.7) was released to fix an XSS vulnerability. A separate DOM XSS issue was fixed in MobileFrontend. We also finished a review of Hadoop’s Camus. Quality assurance
*Quality Assurance https://www.mediawiki.org/wiki/Quality_Assurance* The expertise of our new Automation Engineer, Dan Duvall, will allow us to update, maintain and modernize our development environments using Vagrant virtual machines, keep our Puppet site configuration running properly, and contribute to the whole delivery pipeline as we continue to improve our ability to deploy software features to Wikimedia sites quickly and safely.
*Browser testing https://www.mediawiki.org/wiki/Quality_Assurance/Browser_testing* Now that we have the ability to address the MediaWiki API from the browser testing framework, we have changed the existing test suites to use this powerful tool. This not only gives us a test of the API itself, but also makes the browser tests faster and more reliable. Furthermore, it allows us to easily create a set of acceptance tests that will pass on any MediaWiki installation regardless of what extensions exist or what language the wiki is. We plan for these acceptance tests to eventually become part of MediaWiki core, and our existing tests continue to expose important issues in Wikimedia software development projects. Multimedia
*Multimedia https://www.mediawiki.org/wiki/Multimedia*
https://www.mediawiki.org/w/index.php?title=File:Upload_Wizard_Update_-_June_5_2014.pdf&page=8
https://commons.wikimedia.org/wiki/File:Upload_Wizard_Update_-_June_5_2014.pdf?page=8
Metrics meeting presentation slides about UploadWizard
In May, the multimedia team released Media Viewer v0.2 https://www.mediawiki.org/wiki/Multimedia/About_Media_Viewer on more large wikis https://www.mediawiki.org/wiki/Multimedia/Media_Viewer/Release_Plan#Large_Wikis (Dutch, French, Italian, Japanese, Portuguese, Spanish and Russian Wikipedias), with over 10 million image views daily http://multimedia-metrics.wmflabs.org/graphs/mmv_image_views_global. This multimedia browser has been well received: 70% of survey respondents https://www.mediawiki.org/wiki/Multimedia/Media_Viewer/Surveyfind this tool useful; based on this favorable feedback, we plan to deploy Media Viewer on all wikis in June. Gilles Dubuc, Mark Holmquist, and Gergő Tisza fixed more bugs and features during this development cycle https://wikimedia.mingle.thoughtworks.com/projects/multimedia/cards?favorite_id=11008&view=Completed+this+cycle, with design help from Pau Giner.
The team has switched its focus to new projects, starting with the UploadWizard https://www.mediawiki.org/wiki/UploadWizard, our main user-facing feature this year: this month, we collected metrics, reviewed user feedback, created new designs, fixed bugs and refactored code as part of a major upgrade of this important contribution tool. We also allocated a third of our time to technical debt https://wikimedia.mingle.thoughtworks.com/projects/multimedia/cards?favorite_id=11321&view=Technical+Debt+Wall and bug fixes for other multimedia tools, with an initial focus on improving image scalers, GWToolset and TimedMediaHandler.
Fabrice Florin managed product development, hosting an annual planning meeting https://www.mediawiki.org/wiki/Multimedia/2013-14_Goals to define our goals for 2014−15: this year, we aim to engage more users to contribute media to our sites through tools like UploadWizard, while implementing structured data on Commons and continuing to address our technical debt and fix critical bugs. Keegan Peterzell and Fabrice also continued to engage our community partners throughout the release of Media Viewer https://www.mediawiki.org/wiki/Multimedia/Media_Viewer/Release_Plan. We are planning new discussions in coming weeks to improve our plans together. For join these conversations and keep up with our work, we invite you to subscribe to the multimedia mailing list https://lists.wikimedia.org/mailman/listinfo/multimedia. Engineering Community Team https://www.mediawiki.org/wiki/Engineering_Community_Team
*Bug management https://www.mediawiki.org/wiki/Bug_management* Mark Holmquist and Chad Horohoe changed Bugzilla to automatically handle out “editbugs” permissions http://lists.wikimedia.org/pipermail/wikitech-l/2014-May/076723.html to new accounts and to be able to hand out “editbugs” recursively (bug 40497 https://bugzilla.wikimedia.org/show_bug.cgi?id=40497). Apart from usual bugtriage and major focus on Phabricator migration work https://www.mediawiki.org/wiki/Phabricator/Migration#Status, Andre Klapper retriaged some open major/high Multimedia tickets, created some requested components https://bugzilla.wikimedia.org/buglist.cgi?bug_id=64785,64784,64769,65150 and investigated Gerrit bot notification breakage https://bugzilla.wikimedia.org/show_bug.cgi?id=65370 with Daniel Zahn and Christian Aistleitner. He also made small changes https://www.mediawiki.org/w/index.php?title=Annoying_little_bugs&diff=1012532&oldid=945839 to the Annoying little bugs page based on feedback, and reorganized theWikipedia App https://bugzilla.wikimedia.org/show_bug.cgi?id=64621 and the Analytics https://bugzilla.wikimedia.org/show_bug.cgi?id=65903 products in Bugzilla. Andre and Mark Hershberger updated some open Bugzilla tickets with Target Milestone 1.23.0 at the Zürich Hackathon 2014 https://www.mediawiki.org/wiki/Z%C3%BCrich_Hackathon_2014.
*Phabricator migration https://www.mediawiki.org/wiki/Phabricator/Migration* Mukunda Modell is currently addressing authentication http://fab.wmflabs.org/T314 and access restrictions for security tickets http://fab.wmflabs.org/T95 with upstream. Chase Pettet and Daniel Zahn of Wikimedia Operations are spearheading the Phabricator installation. Andre Klapper has upstreamed some issues, commented on numerous tickets, and identified further tasks related to migration. An overview board of tasks to solve for the first day http://fab.wmflabs.org/project/view/31/ of Phabricator in production is available. Furthermore, once Wikimedia SUL authentication is sorted out, it is investigated to launch the Phabricator production instance first with very limited functionality to provide a Trusted User Tool http://fab.wmflabs.org/T364.
*Mentorship programs https://www.mediawiki.org/wiki/Mentorship_programs*
We hosted a Q&A session on IRC https://www.mediawiki.org/wiki/Engineering_Community_Team/Meetings/2014-05-20 with a high participation of Google Summer of Code https://www.mediawiki.org/wiki/Google_Summer_of_Code_2014 and FOSS Outreach Program for Women https://www.mediawiki.org/wiki/FOSS_Outreach_Program_for_Women/Round_8 interns and mentors right at the beginning of the development phase. Below are the first project reports, including the lessons learned during the community bonding period:
- Tools for mass migration of legacy translated wiki content https://www.mediawiki.org/wiki/Extension:Translate/Mass_migration_tools/Project_updates - Wikidata annotation tool https://www.mediawiki.org/wiki/Wikidata_annotation_tool/updates - Email bounce handling to MediaWiki with VERP https://www.mediawiki.org/wiki/VERP/GSOC_Progress_Rerport - Google Books, Internet Archive, Commons upload cycle https://www.mediawiki.org/wiki/Google_Books,_Internet_Archive,_Commons_upload_cycle/Progress - UniversalLanguageSelector fonts for Chinese wikis https://www.mediawiki.org/wiki/Extension:UniversalLanguageSelector/Fonts_for_Chinese_wikis#Weekly_Report - MassMessage page input list improvements https://www.mediawiki.org/wiki/Extension:MassMessage/Page_input_list_improvements/Progress_reports - Book management in Wikibooks/Wikisource https://meta.wikimedia.org/wiki/Book_management_2014/Progress - Parsoid-based online-detection of broken wikitext https://www.mediawiki.org/wiki/User:Hardik95/GSoC_2014_Progress_Report - Usability improvements for the Translate extension https://www.mediawiki.org/wiki/User:Kunalgrover05/Progress_Report - A modern, scalable and attractive skin for MediaWiki https://www.mediawiki.org/wiki/User:Jack_Phoenix/GSoC_2014 - Automatic cross-language screenshots for user documentation https://www.mediawiki.org/wiki/Automatic_cross-language_screenshots/progress - Separating skins from core MediaWiki https://www.mediawiki.org/wiki/Separating_skins_from_core_MediaWiki/Progress - Chemical Markup support for Wikimedia Commons https://www.mediawiki.org/wiki/Chemical_Markup_support_for_Wikimedia_Commons/Internship_Report - Improving URL citations on Wikimedia https://www.mediawiki.org/wiki/User:Mvolz/Weekly_Reports - Historical OpenStreetMap https://www.mediawiki.org/wiki/User:JaimeLyn/Weekly_Reports - Welcoming new contributors to Wikimedia Labs and Tool Labs https://www.mediawiki.org/wiki/Welcome_to_labs/Progress_Reports - Evaluating, documenting, and improving MediaWiki web API client libraries https://www.mediawiki.org/wiki/Evaluating_and_Improving_MediaWiki_web_API_client_libraries/Progress_Reports - Feed the Gnomes – Wikidata Outreach https://www.mediawiki.org/wiki/User:Thepwnco/OPW_Reporting - Template Matching for RDFIO https://www.mediawiki.org/wiki/Extension:RDFIO/Template_matching_for_RDFIO/Reports - Switching Semantic Forms Autocompletion to Select2 https://www.mediawiki.org/wiki/Extension:Semantic_Forms/Select2_for_autocompletion/Progress_Report - Catalogue for Mediawiki Extensions https://www.mediawiki.org/wiki/User:Adi.iiita/Gsoc2014/Report#Weekly_Report - Generic, efficient localisation update service https://www.mediawiki.org/wiki/Extension:LocalisationUpdate/LUv2/Updates .
*Technical communications https://www.mediawiki.org/wiki/Technical_communications* In addition to ongoing communications support https://www.mediawiki.org/wiki/Technical_communications/Tech_blog_activity for the engineering staff, Guillaume Paumier https://www.mediawiki.org/wiki/User:Guillom mostly focused on improvements to the system of scripts and templates https://www.mediawiki.org/wiki/Category:Wikimedia_engineering_templates used to document Wikimedia engineering activities on mediawiki.org. Active and inactive projects can now be queried separately, which means that the list of projects that appears in the drop-down of the status helper gadget https://www.mediawiki.org/wiki/MediaWiki:Gadget-WmfProjectStatusHelper.js is much shorter, now only listing active projects. Guillaume also wrote a dedicated Lua module https://www.mediawiki.org/wiki/Module:Wikimedia_engineering to manipulate engineering activities automatically, in particular for theWikimedia Engineering https://www.mediawiki.org/wiki/Wikimedia_Engineering portal. It is now possible to query activities from a given team, and display them on any page in various formats. Using the module, the ever-outdated list of “current” activities on the portal was replaced by an automatically-generated list based on projects listed on team hubs. The module also allows to feature a random engineering activity on the portal. Other additions to the portal include the latest issue of Tech News https://meta.wikimedia.org/wiki/Tech/News(transcluded and automatically updated every week), as well as the first paragraph of the latest monthly engineering report (manually updated for now). Future improvements of the portal are expected to be mostly aesthetic.
*Volunteer coordination and outreach https://www.mediawiki.org/wiki/Volunteer_coordination_and_outreach* The Wikimedia Hackathon in Zürich https://www.mediawiki.org/wiki/Z%C3%BCrich_Hackathon_2014 was a success according to ad hoc feedback from the participants. A deeper review is expected to be published in July, after compiling the results of the survey. The main merit goes to Wikimedia CH for an efficient, warm, and flexible organization. We also announced a process to request the organization of Hackathons https://www.mediawiki.org/wiki/Hackathons. We had an intense calendar of events in May https://www.mediawiki.org/wiki/Project:Calendar/2014/05, including a Tech Talk about Elasticsearch https://plus.google.com/b/103470172168784626509/events/cokipb2senmmvkvdjif7aq55kac and a meetup in San Francisco on *Making Wikipedia Fast* https://plus.google.com/events/ce6kihklfld2p10ep28p5ia7klg, organized successfully together with the Web Performance SF meetup.
*Architecture and Requests for comment process https://www.mediawiki.org/wiki/Architecture_process*
Engineers discussed the Performance guidelines https://www.mediawiki.org/wiki/Performance_guidelines at the Zürich Hackathon 2014 https://www.mediawiki.org/wiki/Z%C3%BCrich_Hackathon_2014. They also had IRC discussions of several requests for comment and documents:
- 2014-05-14 — Discussion https://www.mediawiki.org/wiki/Architecture_meetings/Performance_guidelines_discussion_2014-05-14 of Performance guidelines https://www.mediawiki.org/wiki/Performance_guidelines draft - 2014-05-21 — Discussion https://www.mediawiki.org/wiki/Architecture_meetings/RFC_review_2014-05-21 of Typesafe enums https://www.mediawiki.org/wiki/Requests_for_comment/Typesafe_enums and Square bounding boxes https://www.mediawiki.org/wiki/Requests_for_comment/Square_bounding_boxes - 2014-05-30 — Discussion https://www.mediawiki.org/wiki/Architecture_meetings/RFC_review_2014-05-30 of Extension registration https://www.mediawiki.org/wiki/Requests_for_comment/Extension_registration - 2014-06-02 — Discussion https://www.mediawiki.org/wiki/Architecture_meetings/RFC_review_2014-06-02 of Grid system https://www.mediawiki.org/wiki/Requests_for_comment/Grid_system.
Analytics https://www.mediawiki.org/wiki/Analytics
*Wikimetrics https://www.mediawiki.org/wiki/Analytics/Wikimetrics* The focus for this month was on extending Wikimetrics to support the Editor Engagement Vital Signs https://www.mediawiki.org/wiki/Analytics/Editor_Engagement_Vital_Signs (EEVS) project. The team also fixed several bugs around Unicode support (particularly non-Latin character sets) and implemented delete cohort functionality.
*Data Processing https://www.mediawiki.org/wiki/Analytics/Data_Processing* Capacity, deployment and CDH 5 (new Hadoop) version was worked on this month. These initiatives should be resolved in June. A permissions issue caused the page view dumps to stall for a weekend. The system was fixed promptly and no data was lost.
*Editor Engagement Vital Signs https://www.mediawiki.org/wiki/Analytics/Editor_Engagement_Vital_Signs* We created a headless user to run recurrent reports and groundwork to support creating new metrics. The team also discussed potential changes to the first round of metrics implemented in the system to support a broader view of participation, and reviewed mockups provided by the design team with stakeholders.
*EventLogging https://www.mediawiki.org/wiki/Analytics/EventLogging* The team has adopted event logging http://lists.wikimedia.org/pipermail/analytics/2014-June/002202.html and has identified some monitoring updates that need to be made. Kiwix http://www.kiwix.org/
*The Kiwix project is funded and executed by Wikimedia CH https://meta.wikimedia.org/wiki/Wikimedia_CH.* The Hackathon in Zürich was our highlight for May. There, most of the team met, and a few new people joined the development effort during the 3 days in common with the Wikimedia hackathon. Most of the work done focused on preparing the final Kiwix 0.9 release, to be released in June. It was also necessary to release a new minor release (1.2) https://fr.wikipedia.org/wiki/Projet:Afrip%C3%A9dia of libzim. Work on Kiwix for Android https://play.google.com/store/apps/details?id=org.kiwix.kiwixmobile continues with the integration of a content/download manager. On the offline content side, 6 ZIM files with all TED talks http://lists.wikimedia.org/pipermail/offline-l/2014-June/001254.html were also released; it’s the first time we provide files with so much multimedia content. 50 USB flash drives with Kiwix and the *Wikipedia for schools* were prepared and sent toWikiIndaba http://wikiindaba.net/; 15 Kiwix-plug http://www.kiwix.org/wiki/Kiwix-plug are also going to be prepared for Afripedia https://fr.wikipedia.org/wiki/Projet:Afrip%C3%A9dia. Finally, in the scope of the Malebooks http://malebooks.ml/project, work to prepare an offline version of the Gutenberg project http://www.gutenberg.org/, with its 45,000 public domain books, has started. Wikidata https://meta.wikimedia.org/wiki/Wikidata
*The Wikidata project is funded and executed by Wikimedia Deutschland https://meta.wikimedia.org/wiki/Wikimedia_Deutschland/en.* It has been a busy month for Wikidata. The development team made significant progress on a number of important features. These include simple queries, the mono-lingual text datatype and redirects. We’ve also done a lot of research for the upcoming user interface change and started making mockups for it. Lydia Pintscher held an office hour on IRC https://meta.wikimedia.org/wiki/IRC_office_hours/Office_hours_2014-05-19 and answered a lot of questions. Two interns (Anjali Sharma and Helen Halbert) started working with the team http://lists.wikimedia.org/pipermail/wikidata-l/2014-April/003753.html. They will improve the user documentation and social media outreach around Wikidata. Wikidata was also well represented at the MediaWiki hackathon in Zürich https://blog.wikimedia.de/2014/05/13/thank-you-zurich-merci-vielmal-zueri-wikimedia-hackathon-zuerich-2014/. Last but not least, Magnus Manske developed a number ofgames around Wikidata https://tools.wmflabs.org/wikidata-game/ that help improve the data in it and add more. It’s a resounding success so far. FutureThe engineering management team continues to update the *Deployments https://wikitech.wikimedia.org/wiki/Deployments* page weekly, providing up-to-date information on the upcoming deployments to Wikimedia sites, as well as the *annual goals https://www.mediawiki.org/wiki/Wikimedia_Engineering/2014-15_Goals*, listing ongoing and future Wikimedia engineering efforts. ------------------------------ *This article was written collaboratively by Wikimedia engineers and managers. See revision history https://www.mediawiki.org/w/index.php?title=Wikimedia_engineering_report/2014/May&action=history and associated status pages. A wiki version https://www.mediawiki.org/wiki/Wikimedia_engineering_report/2014/May is also available.*