Hi,
The report covering Wikimedia engineering activities in March 2014 is now available.
Wiki version: https://www.mediawiki.org/wiki/Wikimedia_engineering_report/2014/March Blog version: http://blog.wikimedia.org/2014/04/18/engineering-report-march-2014/
We're also proposing a shorter, simpler and translatable version of this report that does not assume specialized technical knowledge: https://www.mediawiki.org/wiki/Wikimedia_engineering_report/2014/March/summa...
Below is the HTML text of the report.
As always, feedback is appreciated on the usefulness of the report and its summary, and on how to improve them.
------------------------------------------------------------------
Major news in March include:
- an overview of webfontshttps://blog.wikimedia.org/2014/03/07/webfonts-making-wikimedia-projects-readable-for-everyone/, and the advantages and challenges of using them on Wikimedia sites; - a series of essayshttps://blog.wikimedia.org/2014/03/25/seeing-through-the-eyes-of-new-technical-contributors/ written by Google Code-in students who shared their impressions, frustrations and surprises as they discovered the Wikimedia and MediaWiki technical community; - Hovercardshttps://blog.wikimedia.org/2014/03/26/hovercards-now-available-as-a-beta-feature-on-all-wikimedia-wikis/ now available as a Beta feature on all Wikimedia wikis, allowing readers to see a short summary of an article just by hovering a link; - a subtle typography changehttps://blog.wikimedia.org/2014/03/27/typography-refresh/ across Wikimedia sites for better readability, consistency and accessibility; - a recap of the upgrade and migrationhttps://blog.wikimedia.org/2014/03/28/wikimedias-road-to-bugzilla-4-4/ of our bug tracking software.
*Note: We’re also providing a shorter, simpler and translatable version of this report https://www.mediawiki.org/wiki/Wikimedia_engineering_report/2014/March/summary that does not assume specialized technical knowledge.*
Engineering metrics in March:
- 160 unique committers contributed patchsets of code to MediaWiki. - The total number of unresolved commitshttps://gerrit.wikimedia.org/r/#q,status:open+project:%255Emediawiki.*,n,zwent from around 1450 to about 1315. - About 25 shell requests https://www.mediawiki.org/wiki/Shell_requests were processed.
Contents
- 1 Personnelhttp://blog.wikimedia.org/2014/04/18/engineering-report-march-2014/#Personnel - 1.1 Work with ushttp://blog.wikimedia.org/2014/04/18/engineering-report-march-2014/#Work_with_us - 1.2 Announcementshttp://blog.wikimedia.org/2014/04/18/engineering-report-march-2014/#Announcements - 2 Technical Operationshttp://blog.wikimedia.org/2014/04/18/engineering-report-march-2014/#Technical_Operations - 3 Features Engineeringhttp://blog.wikimedia.org/2014/04/18/engineering-report-march-2014/#Features_Engineering - 3.1 Editor retention: Editing toolshttp://blog.wikimedia.org/2014/04/18/engineering-report-march-2014/#Editor_retention:_Editing_tools - 3.2 Core Featureshttp://blog.wikimedia.org/2014/04/18/engineering-report-march-2014/#Core_Features - 3.3 Growthhttp://blog.wikimedia.org/2014/04/18/engineering-report-march-2014/#Growth - 3.4 Supporthttp://blog.wikimedia.org/2014/04/18/engineering-report-march-2014/#Support - 4 Mobilehttp://blog.wikimedia.org/2014/04/18/engineering-report-march-2014/#Mobile - 5 Language Engineeringhttp://blog.wikimedia.org/2014/04/18/engineering-report-march-2014/#Language_Engineering - 6 Platform Engineeringhttp://blog.wikimedia.org/2014/04/18/engineering-report-march-2014/#Platform_Engineering - 6.1 MediaWiki Corehttp://blog.wikimedia.org/2014/04/18/engineering-report-march-2014/#MediaWiki_Core - 6.2 Quality assurancehttp://blog.wikimedia.org/2014/04/18/engineering-report-march-2014/#Quality_assurance - 6.3 Multimediahttp://blog.wikimedia.org/2014/04/18/engineering-report-march-2014/#Multimedia - 6.4 Engineering Community Teamhttp://blog.wikimedia.org/2014/04/18/engineering-report-march-2014/#Engineering_Community_Team - 7 Analyticshttp://blog.wikimedia.org/2014/04/18/engineering-report-march-2014/#Analytics - 8 Kiwixhttp://blog.wikimedia.org/2014/04/18/engineering-report-march-2014/#Kiwix - 9 Wikidatahttp://blog.wikimedia.org/2014/04/18/engineering-report-march-2014/#Wikidata - 10 Futurehttp://blog.wikimedia.org/2014/04/18/engineering-report-march-2014/#Future
Personnel Work with us https://wikimediafoundation.org/wiki/Work_with_us
Are you looking to work for Wikimedia? We have a lot of hiring coming up, and we really love talking to active community members about these roles.
- VP of Engineeringhttp://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&page=Job%20Description&j=ods8Xfwu - Software Engineer – Growthhttp://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&page=Job%20Description&j=o8NJXfwl - Software Engineer – VisualEditor (Features)http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&page=Job%20Description&j=oqo6XfwB - Software Engineer – Fundraising Teamhttp://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&page=Job%20Description&j=oWH6Vfwo - Software Engineer – Internationalizationhttp://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&page=Job%20Description&j=oH3gXfwH - Software Engineer- Mobile (Frontend)http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&page=Job%20Description&j=o09WXfwM - Automation Engineer (Ruby)http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&page=Job%20Description&j=oe09Yfw5 - Release Engineerhttp://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&page=Job%20Description&j=oFtlYfwb - Research Analyst – Fundraisinghttp://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&page=Job%20Description&j=o2AJYfw3 - Director of Community Engagement (Product)http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&page=Job%20Description&j=oX0kYfwZ - Product Manager – Language Engineeringhttp://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&page=Job%20Description&j=osiMYfwe - Operations Security Engineerhttp://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&page=Job%20Description&j=oT6cYfwT
Announcements
- Chase Pettet joined the Wikimedia Operations Team as as Operations Engineer (announcementhttp://lists.wikimedia.org/pipermail/wikitech-l/2014-March/075122.html ). - Following changes in the Engineering Community Teamhttps://www.mediawiki.org/wiki/Engineering_Community_Team, Quim Gil took over as Engineering Community Manager, and Sumana Harihareswara transitioned to the role of Senior Technical Writer ( announcementhttp://lists.wikimedia.org/pipermail/wikitech-l/2014-March/075308.html ). - Kevin LeDuc joined the Wikimedia Foundation as Analytics Product Manager (announcementhttp://lists.wikimedia.org/pipermail/analytics/2014-March/001737.html ).
Technical Operations
*Datacenter RFP https://wikimediafoundation.org/wiki/RFP/2013_Datacenter* Final negotiations and coordination are still ongoing for the data center RFP, but we expect to be able to make an announcement soon.
*Wikimedia Labs*
Labs metrics in March:
- Number of projects: 149 - Number of instances: 310 - Amount of RAM in use (in MBs): 1,288,704 - Amount of allocated storage (in GBs): 14,925 - Number of virtual CPUs in use: 635 - Number of users: 2,907
The Labs Ops team has spent the month shepherding projects from the Tampa cloud to the Ashburn cloud. Dozens of volunteers contributed to the move, and all tools and projects have now been copied to or rebuilt in Ashburn. Some projects and tools are in a non-running state pending action on the part of their owners or admins. Ashburn Labs is running OpenStack Havana, with NFS for shared storage. The usage stats this month are quite a bit different from last month. Quite a number of obsolete instances have been purged, and last month’s stats may have included some data center duplication.
*Tampa data center* During March, the Ops team has been decommissioning and shutting down a lot of hosts in the old Tampa data center, including all former appservers. The amount of energy consumed in the old data center has been greatly reduced. A few hosts are going to be migrated to another floor in the existing data center and physical data center work is coming up. Features Engineeringhttps://www.mediawiki.org/wiki/Wikimedia_Features_engineering Editor retention: Editing tools
*VisualEditor https://www.mediawiki.org/wiki/VisualEditor*
https://commons.wikimedia.org/wiki/File:VisualEditor_-_2013-14_Q2%E2%80%93Q3_quarterly_review_deck.pdf https://commons.wikimedia.org/wiki/File:VisualEditor_-_2013-14_Q2%E2%80%93Q3_quarterly_review_deck.pdf
Presentation slides from the VisualEditor team’s quarterly review meetinghttps://meta.wikimedia.org/wiki/Metrics_and_activities_meetings/Quarterly_eviews/VisualEditor/March_2014 on 26 March.
In March, the VisualEditor team continued their work on improving the stability and performance of the system, and added some new features and simplifications, helping users edit and create pages more swiftly and easily. Editing templates is now much simpler, moving most of the advanced controls that users don’t often need into a special version of that dialog. The media dialog was improved and stream-lined a little, adding some hinting to the controls to explain how they work a bit more. The cursor entry points inserted by VisualEditor next to items like images or templates to give users somewhere to put the cursor now animate on hover and cursor entry to show that they’re special. The overall design of dialogs and controls was improved a little to make it flow better, like double-clicking a block to open its dialog. A new system for quickly and simply inserting and editing “citations” (references based on templates) neared completion and will be deployed in the coming month. The deployed version of the code was updated four times in the regular releases ( 1.23-wmf17https://www.mediawiki.org/wiki/MediaWiki_1.23/wmf17#VisualEditor , 1.23-wmf18https://www.mediawiki.org/wiki/MediaWiki_1.23/wmf18#VisualEditor , 1.23-wmf19https://www.mediawiki.org/wiki/MediaWiki_1.23/wmf19#VisualEditor and 1.23-wmf20https://www.mediawiki.org/wiki/MediaWiki_1.23/wmf20#VisualEditor ).
*Parsoid https://www.mediawiki.org/wiki/Parsoid*
https://commons.wikimedia.org/wiki/File:Parsoid_review_Q3,_March_2014.pdf https://commons.wikimedia.org/wiki/File:Parsoid_review_Q3,_March_2014.pdf
Presentation slides from the Parsoid team’s quarterly review meetinghttps://meta.wikimedia.org/wiki/Metrics_and_activities_meetings/Quarterly_reviews/Parsoid/March_2014 on March 28
March saw the Parsoid team continuing with a lot of unglamorous bug fixing and tweaking https://www.mediawiki.org/wiki/Parsoid/Deployments. Media / image handling in particular received a good amount of love, and is now in a much better state than it used to be. In the process, we discovered a lot of edge cases and inconsistent behavior in the PHP parser, and fixed some of those issues there as well.
We wrapped up our mentorship for Be Birchall and Maria Pecana in the Outreach Program for Womenhttps://www.mediawiki.org/wiki/FOSS_Outreach_Program_for_Women/Round_7. We revamped our round-trip test server interfacehttp://parsoid-tests.wikimedia.org/ and fixed some diffing issues in the round-trip test system. Maria wrote a generic logging backend that lets us dynamically map an event stream to any number of logging sinks. A huge step up from our console.error based basic error logging so far.
We also designed and implemented a HTML templating library which combines the correctness and security support of a DOM-based solution with the performance of string-based templating. This is implemented as a compilerhttps://github.com/gwicke/knockoff from KnockoutJS-compatible HTML syntax to a JSON intermediate representationhttps://github.com/gwicke/tassembly, and a small and very fast runtime for the JSON representation. The runtime is now also being ported to PHP in order to gauge the performance there as well. It will also be a test bed for further forays into HTML templating for translation messages and eventually wiki content. Core Features
*Flow https://www.mediawiki.org/wiki/Flow/Project_information* This month the Core Features team focused on improvements to how Flow works with key MediaWiki tools and processes. We made changes to the history, watchlist, and recent changes views, adding more context and bringing them more in line with what experienced users expect from these features. We also worked on improvements to the API and links tables integration. On the core discussion side, we released a Flow thank feature, allowing users to thank each other for posts, and began work on a feature to close and summarize discussions. Lastly, we continued work on rewriting the Flow front-end to make it cleaner, faster, and more responsive across a wide number of browsers/devices, which will be ongoing over the next month. Growth
*Growth https://www.mediawiki.org/wiki/Growth* In March, the Growth team primarily focused on bug fixing, design enhancements, and refactoring of theGettingStartedhttps://www.mediawiki.org/wiki/Extension:GettingStarted and GuidedTour https://www.mediawiki.org/wiki/Extension:GuidedTour extensions, which were recently launched on 30 Wikipedias. We updated icons and button styles, rewrote the interface copy, and refactored the interface to be more usable in non-English languages. We also began work on a significant refactor of the GuidedTour API, in order to support interactive tours that are non-linear. Non-linear tours will not depend on a page load to run, which will enable better support for tours in VisualEditorhttps://www.mediawiki.org/wiki/VisualEditor, among other things. Last but not least, we made progress on measuring the impact of GettingStartedhttps://meta.wikimedia.org/wiki/Research:Onboarding_new_Wikipedians/Rollout across all wikis where it is deployed, with results for the first 30 days of editor activity expected in early April. Support
*Wikipedia Education Program https://www.mediawiki.org/wiki/Wikipedia_Education_Program* This month, thanks to the work of Facebook Open Academy student JJ Liu, we added a new type of notification for course pages: users are now notified whenever they get added to a course. We also fixed inconsistencies with interface messages, user rights, and the deletion of institutions from the system. Mobile https://www.mediawiki.org/wiki/Wikimedia_Mobile_engineering
*Wikimedia Apps https://www.mediawiki.org/wiki/Wikimedia_Apps* The team worked on logged-out editing to logged-in editing, and table of contents refinements.
*Mobile web projects https://www.mediawiki.org/wiki/Mobile_web_projects* The team worked on the link inspector for VisualEditor on tablets, and a switch between VisualEditor and wikitext on tablets. Both are in alpha.
*Wikipedia Zero https://www.mediawiki.org/wiki/Wikipedia_Zero*
During the last month, with the assistance of the Ops and Platform teams, the Wikipedia Zero team added hosting for the forthcoming Partners Portal and continued work on image size reduction for the mobile web. Additionally, the team added Wikipedia Zero detection to the Wikipedia for Firefox OS app and added contributory features support for users on partner networks supporting zero-rated HTTPS connections. The team also removed search thumbnails for zero.wikipedia.org connections in order to avoid spurious charges on devices supporting high-end JavaScript yet using zero.wikipedia.org. Analytics fields were added for the purpose of counting proxy-based and HTTPS-based connections. Routine pre- and post-launch configuration changes were made to support operator zero-rating, and technical assistance was provided to operators and the partner management team to help add zero-rating. The Wikipedia Zero automation testing server was also migrated. The forthcoming Android and iOS apps were also updated to make Wikipedia Zero detection a standard fixture.
Yuri continued analytics work on SMS/USSD pilot data. Post hoc analysis was performed on WML usage after its deprecation; it is still low, although obtaining more low-end phones to check for how well HTML renders and how to enhance the HTML could be useful. Post hoc analysis was also performed on anomalous declines and growth spurts in log lines (not strictly related to pageviews); in the former it much had to do with API changes and in the latter it had much to do with an external polling mechanisms.
With the assistance of the Apps team, User-Agent, Send App Feedback, and Random features were added to the forthcoming reboots of the Android and iOS apps, while making the Share feature for Android allow for a different target app each time and providing code review assistance on the Android and iOS apps code; proof of concept for fulltext search was started on iOS. Wikipedia for Firefox OS bugfixes were also pushed to production. Screencap workflows and preload information was put together for the Android reboot with respect to Wikipedia Zero as well.
The team worked with Ops on forward planning in light of the extremely infrastructure-oriented nature of the program. Quarterly review as held with the ED, VP of Engineering, and the W0 cross-functional team, and the W0 cross-functional team reviewed presentation material for publication. The team also continued work on additional proxy and gateway support. To help partner tech contacts, the team worked on reformatting the tech partner introductory documentation.
Finally, the team explored proactive MCC/MNC-to-IP address drift correction, and will be emailing the community for input soon.
*Wikipedia Zero (partnerships)* Smart, the largest mobile operator in the Philippines, is giving access to Wikipedia free of data charges through the end of April. They announced the promotion in a press release. Ingrid Flores, Wikipedia Zero Partner Manager, visited the Philippines and arranged a meeting with local community members and Smart. They are now exploring ways to collaborate in support of education. The partnerships team kicked off account reviews with the 27 existing Wikipedia Zero partners, to update the implementation, identify opportunities for collaboration in corporate social responsibility (CSR) initiatives and get feedback on the program. The account reviews will continue for the next few months. Last, we continued recruiting for Wikipedia Partner Managerhttp://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&page=Job%20Description&j=ok6oYfww for the Asia region. Language Engineeringhttps://www.mediawiki.org/wiki/Wikimedia_Language_engineering
*Language tools https://www.mediawiki.org/wiki/Language_tools* MediaWiki’s LocalisationUpdate extension was rewrittenhttps://blog.wikimedia.org/2014/04/03/modernising-mediawikis-localisation-update/ by Niklas Laxström to modernize its internal architecture to be able to support JSON message file formatshttps://www.mediawiki.org/wiki/Requests_for_comment/Localisation_format. Kartik Mistry released the team’s monthly MediaWiki Language Extension Bundle (MLEB 2014.3) with the latest version of LocalisationUpdate (see release noteshttp://lists.wikimedia.org/pipermail/mediawiki-i18n/2014-March/000838.html). Niklas Laxström also started migrating the Translate extension’s translation memory and translation search back-end from Solr to ElasticSearch in line with Wikimedia’s searchhttps://www.mediawiki.org/wiki/Search migration. David Chan continued his work on input method support for the VisualEditor project.
*Milkshake https://www.mediawiki.org/wiki/Milkshake* Santhosh Thottingal, Kartik Mistry and Niklas Laxström made numerous bugs and performance improvements in jquery.webfonts, jquery.ime and jquery.uls. Amir Aharoni started collecting metrics on usage of Universal Language Selector.
*Language Engineering Communications and Outreach https://www.mediawiki.org/wiki/Language_engineering_communications_and_outreach* Runa Bhattacharjee and Kartik Mistry set up a manual testing infrastructure using the Test Case Management Systemhttps://www.mediawiki.org/wiki/Language_Testing_Plan/TCMS_Guide (TCMS) to help get greater participation from the volunteer community of software tools and features developed by the team. Volunteer testing is expected to be kickstarted for language software this coming month. The team’s monthly office hourhttps://meta.wikimedia.org/wiki/IRC_office_hours/Office_hours_2014-03-12 was hosted by Runa Bhattacharjee on March 12. An overview of webfontshttps://blog.wikimedia.org/2014/03/07/webfonts-making-wikimedia-projects-readable-for-everyone/ with advantages and challenges of using them on Wikimedia sites was also published by the team.
*Content translation https://www.mediawiki.org/wiki/Content_translation* [image: File:CX Section Alignment Preview and Basic Editing.webm] https://upload.wikimedia.org/wikipedia/commons/0/0f/CX_Section_Alignment_Preview_and_Basic_Editing.webm https://commons.wikimedia.org/wiki/File:CX_Section_Alignment_Preview_and_Basic_Editing.webm
Preview of section alignment and basic editing in Content translation
Santhosh Thottingal and David Chan continued development and technology research on the Content Translation project. Development was focused specifically on updates to the side-by-side translation editor and section alignment of translated text. Kartik Mistry and Santhosh Thottingal worked on infrastructure for testing the Content Translation server. David Chan continued his technology research on sentence segmentation.
Pau Giner updated the Content Translation UI design specificationhttps://commons.wikimedia.org/wiki/File:Content-translation-designs.pdfincorporating review comments from UX and product reviews. The team also participated in a review of the Content Translation project with the product team leadership. Platform Engineeringhttps://www.mediawiki.org/wiki/Wikimedia_Platform_Engineering MediaWiki Core
*HHVM https://www.mediawiki.org/wiki/HHVM* The team continued to work on porting C extensions to HHVM. Tim Starling did major work on a compatibility layer allowing Zend extensions to be used by HHVM, and started further work on making the layer compatible with newer HHVM interfaces. The team has made a preliminary deployment of HHVM to the Beta cluster, but this still needs further debugging before it is useful to a wider audience.
*Release & QA https://www.mediawiki.org/wiki/Wikimedia_Release_and_QA_Team* The Beta Cluster http://en.wikipedia.beta.wmflabs.org/wiki/Main_Page has been migrated from the Tampa data center to the Ashburn data center. In the move, a ton of cleanup and Puppetization work was done. This will make future Beta Cluster work easier. In addition, the Beta Cluster is getting closer to a place where we can test our current main deployment tool known as “scap” along with future/other deployment tools. The team continued on the rewrite of scap into python (from Bash scripts + PHP), improving both performance and maintainability in addition to being in a better position to move to a new tool in the future. We have also started doing SWAT deploy windows https://wikitech.wikimedia.org/wiki/SWAT_deploys twice a day (Monday to Thursday) which has greatly increased momentum for many developers who would otherwise have to wait until the weekly deployment cycle.
*Search https://www.mediawiki.org/wiki/Search* In March we upgraded to the newest version of Elasticsearch and expanded onto more wikis. We also started a performance assessment which has started showing us the work required to use Cirrus as the primary search back-end for the larger wikis. We then started in on that work.
*Auth systems https://www.mediawiki.org/wiki/Auth_systems* The team prepared the migration of the central OAuth database from mediawiki.org to Meta-Wiki, and got input from the Wikimedia Foundation’s legal team regarding the OAuth process.
*Wikimania Scholarships app https://www.mediawiki.org/wiki/Wikimania_Scholarships_app* Support of production application during applicant review period continued in March. A dataset of applicants passing the phase 1 review criteria who had opted-in to sharing application details with chapters and thematic organizations was prepared and delivered to Foundation staff. The beta testing server was migrated from the Tampa data center to the Ashburn data center as a component of the Labs environment migration. The new beta server in Labs is now managed via the MediaWiki-Vagrant role::wikimania_scholarships puppet role and labs-vagranthttps://wikitech.wikimedia.org/wiki/Labs-vagrant. This should make keeping development changes and the testing application in sync easier in the future.
*Security auditing and response https://www.mediawiki.org/wiki/Security_auditing_and_response* MediaWiki 1.19.13, 1.22.5, 1.21.8 and 1.19.14 were released for security issues. An internal security training session was held for Wikimedia Foundation staff. Quality assurance
*Quality Assurance https://www.mediawiki.org/wiki/Quality_Assurance* The QA team continues to identify and report issues in a timely way. Of particular interest in March was that an automated test uncovered an issue in the interaction of the MobileFrontend and VisualEditor extensions. This is exactly the kind of cross-cutting concern that our QA systems are designed to uncover. It is likely that we will be in a position to discuss these systems at the Wikimania conference in London.
*Beta cluster https://www.mediawiki.org/wiki/Beta_cluster* There was a substantial effort to migrate the Beta cluster over from the Tampa data center to the Ashburn, VA (“eqiad”) data center. This was led by Antoine Musso with assistance from Bryan Davis and many others.
*Continuous integration https://www.mediawiki.org/wiki/Continuous_integration* Erik Bernhardson and Chad Horohoe managed to create jobs using Jenkins Job Builder (JJB) based on the tutorials on installing JJBhttps://www.mediawiki.org/wiki/Continuous_integration/Jenkins_job_builder and Adding a MediaWiki extensionhttps://www.mediawiki.org/wiki/Continuous_integration/Tutorials/Adding_a_MediaWiki_extension .
*Browser testing https://www.mediawiki.org/wiki/Quality_Assurance/Browser_testing* Besides a particular focus on MobileFrontend browser tests in March, we have also made available some new features, in particular shared code to upload files properly in all browsers, the ability to check for ResourceLoader problems in any test in any repository, and a basic wrapper in order to use the Mediawiki API from within browser tests to set up and tear down test data. Multimedia
*Multimedia https://www.mediawiki.org/wiki/Multimedia*
https://commons.wikimedia.org/wiki/File:Multimedia_Quarterly_Review_-_Q3_2013-14.pdf https://commons.wikimedia.org/wiki/File:Multimedia_Quarterly_Review_-_Q3_2013-14.pdf
Slides for the Multimedia Quarterly Review Meetinghttps://www.mediawiki.org/wiki/Multimedia/Meetings/Quarterly_Review_Q3_2013-14 for Q3 2013-14.
In March, the multimedia team’s main project was Media Viewer v0.2https://www.mediawiki.org/wiki/Multimedia/About_Media_Viewer, as we completed final features for the tool’s upcoming release next quarterhttps://www.mediawiki.org/wiki/Multimedia/Media_Viewer/Release_Plan. Gilles Dubuc, Mark Holmquist, Gergő Tisza and Aaron Arcos developed a number of new features, including: sharehttps://wikimedia.mingle.thoughtworks.com/projects/multimedia/cards/147 , embedhttps://wikimedia.mingle.thoughtworks.com/projects/multimedia/cards/148 ,downloadhttps://wikimedia.mingle.thoughtworks.com/projects/multimedia/cards/79 , opt-out preferencehttps://wikimedia.mingle.thoughtworks.com/projects/multimedia/cards/264 ,file page linkhttps://wikimedia.mingle.thoughtworks.com/projects/multimedia/cards/226 and feedback linkhttps://wikimedia.mingle.thoughtworks.com/projects/multimedia/cards/261, based on designs by Pau Giner. We invite you to test the latest versionhttps://www.mediawiki.org/wiki/Lightbox_demo (see thetesting tipshttps://www.mediawiki.org/wiki/Multimedia/About_Media_Viewer#How_can_I_help.3F) and share your feedbackhttps://www.mediawiki.org/wiki/Talk:Multimedia/About_Media_Viewer .
Fabrice Florin coached the multimedia team as product manager and hosted several planning and review meetings, including a cycle planning meetinghttp://etherpad.wikimedia.org/p/multimedia-planning-meeting-2014-03-12 (leading to the next cycle planhttps://wikimedia.mingle.thoughtworks.com/projects/multimedia/cards/grid?aggregate_property%5Bcolumn%5D=story+points&aggregate_property%5Brow%5D=story+points&aggregate_type%5Bcolumn%5D=sum&aggregate_type%5Brow%5D=sum&color_by=type&filters%5B%5D=%5BType%5D%5Bis%5D%5BStory%5D&filters%5B%5D=%5BType%5D%5Bis%5D%5BBug%5D&filters%5B%5D=%5BType%5D%5Bis%5D%5BTech+debt%5D&filters%5B%5D=%5BType%5D%5Bis%5D%5BScope+Increase+%28UNPLANNED%29%5D&filters%5B%5D=%5BType%5D%5Bis%5D%5BTeam+Meetings%5D&filters%5B%5D=%5BSprint%5D%5Bis%5D%5B%28Current+sprint%29%5D&group_by%5Blane%5D=status&group_by%5Brow%5D=blocked&lanes=Ready+for+Development%2CIn+Development%2CReviewed%5C%2C+Awaiting+Improvements%2CAwaiting+Code+Review%2CReviewed%5C%2C+Awaiting+Other+Input%2CReady+for+Testing%2CIn+Testing%2CReady+for+Signoff%2CAccepted%2CIn+Analysis&style=grid&tab=Current+cycle) and the Multimedia Quarterly Review Meetinghttps://www.mediawiki.org/wiki/Multimedia/Meetings/Quarterly_Review_Q3_2013-14 for the first quarter of 2014, which summarizes our progress and next steps for coming work (see slideshttps://commons.wikimedia.org/wiki/File:Multimedia_Quarterly_Review_-_Q3_2013-14.pdf). He also worked with Keegan Peterzell to engage community members for the gradual release of Media Viewerhttps://www.mediawiki.org/wiki/Multimedia/Media_Viewer/Release_Plan, to be enabled by default on a number of pilot sites next month, then deployed widely to all wikis a few weeks later. For more updates about our multimedia work, we invite you to join the multimedia mailing listhttps://lists.wikimedia.org/mailman/listinfo/multimedia . Engineering Community Teamhttps://www.mediawiki.org/wiki/Engineering_Community_Team
*Bug management https://www.mediawiki.org/wiki/Bug_management* Beside working on the Project Management Tools Reviewhttps://www.mediawiki.org/wiki/Project_management_tools/Review#Status, Andre Klapper retriaged many older tickets with high priority set for
2years, older PATCH_TO_REVIEW tickets and older critical tickets and
investigated moving the Bugzilla instance on Wikimedia Labs to the Ashburn data center (easier to set up from scratchhttps://bugzilla.wikimedia.org/show_bug.cgi?id=62658#c6). Andre addedhttps://www.mediawiki.org/w/index.php?title=Annoying_little_bugs&diff=922892&oldid=915158project-specific sections and Bugzilla queries to Annoying little bugshttps://www.mediawiki.org/wiki/Annoying_little_bugs to help newcomers finding an area of interest for contributing, and blogged about the 4.4 upgradehttps://blog.wikimedia.org/2014/03/28/wikimedias-road-to-bugzilla-4-4/ (which took place in February) and moving Bugzilla to a new server. In Bugzilla’s tickets, all remaining Cortado tickets were closedhttps://bugzilla.wikimedia.org/show_bug.cgi?id=54063#c4 and new Versions for “Wikipedia App” producthttps://bugzilla.wikimedia.org/show_bug.cgi?id=62660 set up.
*Project management tools review https://www.mediawiki.org/wiki/Project_management_tools/Review* Guillaume Paumier and Andre Klapper reached to the teampracticeshttp://lists.wikimedia.org/pipermail/teampractices/2014-March/000270.html and wikitech-lhttp://lists.wikimedia.org/pipermail/wikitech-l/2014-March/074896.html mailing lists in order to shorten the list of optionshttps://www.mediawiki.org/wiki/Talk:Project_management_tools/Review/Options that can come out of this review process. They also hosted a lively IRC office hourhttps://tools.wmflabs.org/meetbot/wikimedia-office/2014/wikimedia-office.2014-03-28-17.04.html to give an overview of the current situation, answer questions and discuss the first version of the related RFChttps://www.mediawiki.org/wiki/Requests_for_comment/Project_management_tools_review .
*Mentorship programs https://www.mediawiki.org/wiki/Mentorship_programs*
The six ongoing FOSS Outreach Program for Womenhttps://www.mediawiki.org/wiki/Outreach_Program_for_Women/Round_7 were completed successfully, setting a new benchmarkhttp://lists.wikimedia.org/pipermail/wikitech-l/2014-March/075146.html for success in our outreach programs. Check the results:
- Niharika’s compact language links is now a Beta featurehttps://www.mediawiki.org/wiki/Universal_Language_Selector/Design/Interlanguage_links - Anu’s upload wizard with OSM supporthttp://uploadwizard-osm.wmflabs.org/wiki/Special:UploadWizard - Diwanshi’s Wikipedia API courses in Codecademy 1http://www.codecademy.com/courses/web-beginner-en-vj9nh/0/1 and 2 http://www.codecademy.com/courses/web-beginner-en-yd3lp/0/1 - Brena’s prototype of mediawiki.org’s redesigned homepagehttps://www.mediawiki.org/wiki/MediaWiki/Homepage_redesign/Preview - Be’s clean up of Parsoid’s round-trip testing UI is mergedhttps://gerrit.wikimedia.org/r/#/q/owner:bebirchall%2540gmail.com,n,z - Maria’s clean up of Parsoid’s tracing/debugging/logging is mergedhttps://gerrit.wikimedia.org/r/#/q/owner:maria.pacana%2540gmail.com,n,z
We received 43 Google Summer of Codehttps://www.mediawiki.org/wiki/Google_Summer_of_Code_2014 proposals from 42 candidates, and 18 FOSS Outreach Program for Womenhttps://www.mediawiki.org/wiki/FOSS_Outreach_Program_for_Women/Round_8 proposals from 18 candidates. Dozens of mentors are pushing the selection processhttps://www.mediawiki.org/wiki/Mentorship_programs/Selection_process that will conclude on April 21 with the announcement of selected participants.
*Technical communications https://www.mediawiki.org/wiki/Technical_communications* In addition to ongoing communications support for the engineering staff, and contributing to the technical newsletterhttps://meta.wikimedia.org/wiki/Tech/News , Guillaume Paumier https://www.mediawiki.org/wiki/User:Guillom edited and published a series of essayshttps://blog.wikimedia.org/2014/03/25/seeing-through-the-eyes-of-new-technical-contributors/ on the Wikimedia Tech blog written by Google Code-in students, who shared their impressions, frustrations and surprises as they discovered the Wikimedia and MediaWiki technical community.
*Volunteer coordination and outreach https://www.mediawiki.org/wiki/Volunteer_coordination_and_outreach*
The bulk of work to create community metricshttps://www.mediawiki.org/wiki/Community_metrics around five Key Progress Indicators is completed, and now we are polishing help strings and usability details. The next step is to share the news with the community and start looking at bottlenecks and actions. Check:
- Who contributes codehttp://korma.wmflabs.org/browser/who_contributes_code.html - Gerrit review queuehttp://korma.wmflabs.org/browser/gerrit_review_queue.html - Code contributors new and gonehttp://korma.wmflabs.org/browser/code_contrib_new_gone.html - Bugzilla response timehttp://korma.wmflabs.org/browser/bugzilla_response_time.html - Top contributorshttp://korma.wmflabs.org/browser/top-contributors.html
A page about Upstream projectshttps://www.mediawiki.org/wiki/Upstream_projects was drafted collaboratively in order to start mapping the key communities where we Wikimedia should be active, either as contributor / stakeholder, or promoting our own tools. We helped selecting participants sponsored to travel to the Zürich Hackathon 2014https://www.mediawiki.org/wiki/Z%C3%BCrich_Hackathon_2014 in May.
*Architecture and Requests for comment process https://www.mediawiki.org/wiki/Architecture_process*
We held four RfC review meetings on IRC:
- on passwords, TitleValue, and inline diffshttps://www.mediawiki.org/wiki/Architecture_meetings/RFC_review_2014-03-05 - on configuration, URL shortener, Assert, and the Linker refactorhttps://www.mediawiki.org/wiki/Architecture_meetings/RFC_review_2014-03-12 - on MVC framework and structured logginghttps://www.mediawiki.org/wiki/Architecture_meetings/RFC_review_2014-03-19 - on allowing styling in templates and a Minifierhttps://www.mediawiki.org/wiki/Architecture_meetings/RFC_review_2014-03-26 .
Analytics https://www.mediawiki.org/wiki/Analytics
*Kraken https://www.mediawiki.org/wiki/Analytics/Kraken*
We reached a milestone in our ability to deploy Java applications at the Foundation this month when we stood up an Archiva build artifact repository. This enables us to consistently deploy Java libraries and applications and will be used in Hadoop and Search initially.
The first Analytics use case for this system will be Camus, Linked-In’s open source application for loading Kafka data into Hadoop. Once this is productized, we’ll have the ability to regularly load log data from our servers into Hadoop for processing and analysis.
*Wikimetrics https://www.mediawiki.org/wiki/Analytics/Wikimetrics* We did some significant architectural work on WikiMetrics this month to prepare it for its role as our recurrent report scheduling and generation system. The first use case for this system will be the Editor Engagement Vital Signs project, which will provide daily updates on key metrics around participation.
*Kafka https://www.mediawiki.org/wiki/Analytics/Logging_infrastructure* We continue to investigate network issues between our data centers that are causing occasionally delivery issues. As noted above, we are currently deploying Camus, our software for transferring data between Kafka and Hadoop.
*Data Quality https://www.mediawiki.org/wiki/Analytics/Data_Quality* We fixed a number of issues around data quality in Wikistats, Wikipedia Zero and Wikimetrics.
*Research and Data https://www.mediawiki.org/wiki/Analytics/Research_and_Data* [image: File:Wikimedia Research Showcase - March 2014.webmhd.webm] https://upload.wikimedia.org/wikipedia/commons/2/20/Wikimedia_Research_Showcase_-_March_2014.webmhd.webm https://commons.wikimedia.org/wiki/File:Wikimedia_Research_Showcase_-_March_2014.webmhd.webm
Video of the March session of theResearch and Data monthly showcasehttps://www.mediawiki.org/wiki/Analytics/Research_and_Data/Showcase#March_2014 .
This month we concluded the first stage of work on metrics standardization. We created an overviewhttps://meta.wikimedia.org/wiki/Research:Metrics_standardization of the project with a timeline and a list of milestones and deliverables. We also gave an updatehttps://commons.wikimedia.org/wiki/File:Metrics_Standardization_-_Wikimedia_Research_%26_Data_showcase_-_March_2014.pdf on metrics standardization during the March session of the Research and Data monthly showcasehttps://www.mediawiki.org/wiki/Analytics/Research_and_Data/Showcase#March_2014. The showcase also hosted a presentationhttps://commons.wikimedia.org/wiki/File:Maintaining_production_efficiency_(March,_2014).pdf by Aaron Halfaker on his research on the impact of quality control mechanisms on the growth of Wikipedia.
We published an extensive reporthttp://lists.wikimedia.org/pipermail/wiki-research-l/2014-March/003306.html from a session we hosted at CSCW ’14https://meta.wikimedia.org/wiki/Research:Newsletter/2014/February#CSCW_.2714_retrospective on Wikipedia research, discussing with academic researchers and students how to work with researchers at the Foundation.
We submitted 8 session proposalshttps://www.mediawiki.org/wiki/Analytics/Wikimania_2014 for Wikimania ’14, authored or co-authored by members of the research team.
We attended the Analytics team’s Q3 quarterly reviewhttps://meta.wikimedia.org/wiki/Metrics_and_activities_meetings/Quarterly_reviews/Analytics/March_2014 during which we presentedhttps://commons.wikimedia.org/wiki/File:Analytics_Quarterly_Review_Q3_2013_(Research_and_Data).pdf the work performed by the team in the past quarter and our goals for the upcoming quarter (April-June 2014).
We completed the handover of Fundraising analytics tools and knowledge transfer in preparation for a new full-time research position that we will be opening shortly to support the Fundraising team.
We continued to provide support to teams in focus area (Growth and Mobile) with an analysis of the impact of therollouthttps://meta.wikimedia.org/wiki/Research:Onboarding_new_Wikipedians/Rollout of the new onboarding workflows across multiple wikishttps://commons.wikimedia.org/wiki/File:Cross-wiki_deployments.slides.2014-03-27.pdf; an analysis of mobile browsing sessionshttps://meta.wikimedia.org/wiki/Research:Mobile_sessions and ongoing analysis of mobile user acquisition tests. We also supported the Ops team in measuringhttps://www.mediawiki.org/wiki/Analytics/Reports/ULSFOImpact the impact of the deployment of the ULSFO clusterhttps://wikitech.wikimedia.org/wiki/Ulsfo_cluster, which provides caching for West USA and East Asia. Kiwix http://www.kiwix.org/
*The Kiwix project is funded and executed by Wikimedia CH https://meta.wikimedia.org/wiki/Wikimedia_CH.* This month, we released a new version of Kiwix for Androidhttps://play.google.com/store/apps/details?id=org.kiwix.kiwixmobile that adds support for older versions of Android like Gingerbread; about 50% more devices than before are now supported. Wikidata https://meta.wikimedia.org/wiki/Wikidata
*The Wikidata project is funded and executed by Wikimedia Deutschland https://meta.wikimedia.org/wiki/Wikimedia_Deutschland/en.* The team worked on making ranks more useful. From now on, by default the property parser function and Lua always return the values with the “preferred” rank or, when none is available, the one with the “normal” rank. This allows for example to exclude past mayors when asking Wikidata for the mayor of a city. Additionally, considerable speed improvements have been made; browsing Wikidata is now a lot faster. Diffs between versions of pages on Wikidata have also been improved to make it easier to see what changes were made to an item. Last but not least, the user interface redesign research went on. FutureThe engineering management team continues to update the *Deployments https://wikitech.wikimedia.org/wiki/Deployments* page weekly, providing up-to-date information on the upcoming deployments to Wikimedia sites, as well as the *annual goals https://www.mediawiki.org/wiki/Wikimedia_Engineering/2013-14_Goals*, listing ongoing and future Wikimedia engineering efforts. ------------------------------
*This article was written collaboratively by Wikimedia engineers and managers. See revision history http://www.mediawiki.org/w/index.php?title=Wikimedia_engineering_report/2014/March&action=history and associated status pages. A wiki version http://www.mediawiki.org/wiki/Wikimedia_engineering_report/2014/March is also available.*