Hi,
The report covering Wikimedia engineering activities in January 2014 is now available.
Wiki version: https://www.mediawiki.org/wiki/Wikimedia_engineering_report/2014/January Blog version: https://blog.wikimedia.org/2014/02/13/engineering-report-january-2014/
We're also proposing a shorter, simpler and translatable version of this report that does not assume specialized technical knowledge: https://www.mediawiki.org/wiki/Wikimedia_engineering_report/2014/January/sum...
Below is the HTML text of the report.
As always, feedback is appreciated on the usefulness of the report and its summary, and on how to improve them.
------------------------------------------------------------------
Major news in January include:
- the transition of our search engineshttps://blog.wikimedia.org/2014/01/06/wikimedia-moving-to-elasticsearch/on Wikimedia sites to ElasticSearch; - a presentation of how the Tech newsletter workshttps://blog.wikimedia.org/2014/01/07/tech-news-fighting-technical-information-overload-for-wikimedians/, including a historical perspective; - an invitation to comment on the 2-year visionhttps://blog.wikimedia.org/2014/01/09/multimedia-vision-2016/of the multimedia engineering team; - a request for commentshttps://blog.wikimedia.org/2014/01/23/rfc-should-we-support-mp4-video-on-our-sites/on whether the MP4 video format should be supported on Wikimedia sites.
*Note: We’re also providing a shorter, simpler and translatable version of this report https://www.mediawiki.org/wiki/Wikimedia_engineering_report/2014/January/summary that does not assume specialized technical knowledge.*
Engineering metrics in January:
- 159 unique committers contributed patchsets of code to MediaWiki. - The total number of unresolved commitshttps://gerrit.wikimedia.org/r/#q,status:open+project:%255Emediawiki.*,n,zwent from around 1386 to about 1320. - About 22 shell requests https://www.mediawiki.org/wiki/Shell_requestswere processed.
Contents
- Personnelhttps://blog.wikimedia.org/2014/02/13/engineering-report-january-2014/#Personnel - Work with ushttps://blog.wikimedia.org/2014/02/13/engineering-report-january-2014/#Work_with_us - Announcementshttps://blog.wikimedia.org/2014/02/13/engineering-report-january-2014/#Announcements - Technical Operationshttps://blog.wikimedia.org/2014/02/13/engineering-report-january-2014/#Technical_Operations - Features Engineeringhttps://blog.wikimedia.org/2014/02/13/engineering-report-january-2014/#Features_Engineering - Editor retention: Editing toolshttps://blog.wikimedia.org/2014/02/13/engineering-report-january-2014/#Editor_retention:_Editing_tools - Core Featureshttps://blog.wikimedia.org/2014/02/13/engineering-report-january-2014/#Core_Features - Growthhttps://blog.wikimedia.org/2014/02/13/engineering-report-january-2014/#Growth - Supporthttps://blog.wikimedia.org/2014/02/13/engineering-report-january-2014/#Support - Mobilehttps://blog.wikimedia.org/2014/02/13/engineering-report-january-2014/#Mobile - Language Engineeringhttps://blog.wikimedia.org/2014/02/13/engineering-report-january-2014/#Language_Engineering - Platform Engineeringhttps://blog.wikimedia.org/2014/02/13/engineering-report-january-2014/#Platform_Engineering - MediaWiki Corehttps://blog.wikimedia.org/2014/02/13/engineering-report-january-2014/#MediaWiki_Core - Quality assurancehttps://blog.wikimedia.org/2014/02/13/engineering-report-january-2014/#Quality_assurance - Multimediahttps://blog.wikimedia.org/2014/02/13/engineering-report-january-2014/#Multimedia - Engineering Community Teamhttps://blog.wikimedia.org/2014/02/13/engineering-report-january-2014/#Engineering_Community_Team - Analyticshttps://blog.wikimedia.org/2014/02/13/engineering-report-january-2014/#Analytics - Offlinehttps://blog.wikimedia.org/2014/02/13/engineering-report-january-2014/#Offline - Wikidatahttps://blog.wikimedia.org/2014/02/13/engineering-report-january-2014/#Wikidata - Futurehttps://blog.wikimedia.org/2014/02/13/engineering-report-january-2014/#Future
Personnel Work with us https://wikimediafoundation.org/wiki/Work_with_us
Are you looking to work for Wikimedia? We have a lot of hiring coming up, and we really love talking to active community members about these roles.
- VP of Engineering http://hire.jobvite.com/Jobvite/Job.aspx?j=ods8Xfwu - Software Engineer – Growthhttp://hire.jobvite.com/Jobvite/Job.aspx?j=o8NJXfwl - Software Engineer – VisualEditor (Features)http://hire.jobvite.com/Jobvite/Job.aspx?j=oqo6XfwB - Software Engineer – Language Engineeringhttp://hire.jobvite.com/Jobvite/Job.aspx?j=oH3gXfwH&s - Software Engineer http://hire.jobvite.com/Jobvite/Job.aspx?j=o09WXfwM - Software Engineer – Mobile (Android Apps)http://hire.jobvite.com/Jobvite/Job.aspx?j=oi6lYfwr - QA Automation Engineerhttp://hire.jobvite.com/Jobvite/Job.aspx?j=oe09Yfw5 - Test Infrastructure Engineerhttp://hire.jobvite.com/Jobvite/Job.aspx?j=oFtlYfwb - Analytics – Product Managerhttp://hire.jobvite.com/Jobvite/Job.aspx?j=opkhYfwI - Director of Community Engagement (Product)http://hire.jobvite.com/Jobvite/Job.aspx?j=oX0kYfwZ - Sr. Operations Engineerhttp://hire.jobvite.com/Jobvite/Job.aspx?j=o66gYfwa - Operations Security Engineerhttp://hire.jobvite.com/Jobvite/Job.aspx?j=oT6cYfwT - User Experience Research Lead http://grnh.se/3vmjby
Announcements
- Shahyar Ghobadpour joined the Wikimedia Core features team as Software Engineer (announcementhttp://lists.wikimedia.org/pipermail/wikitech-l/2014-January/073870.html ). - David Chan joined the Language Engineering team as Software Engineer ( announcementhttp://lists.wikimedia.org/pipermail/wikitech-l/2014-January/073873.html ). - Gilles Dubuc joined the Multimedia team as Senior Software Engineer and Tech Lead (announcementhttp://lists.wikimedia.org/pipermail/wikitech-l/2014-January/073884.html ). - Charles Salvia joined the Analytics team (announcementhttp://lists.wikimedia.org/pipermail/analytics/2014-January/001526.html ). - Sam Smith joined the Features Engineering team as Software Engineer, working with the Growth team (announcementhttp://lists.wikimedia.org/pipermail/wikitech-l/2014-January/074118.html ). - Preteek Saxena joined the UX Design team as UX Prototyping contractor ( announcementhttp://lists.wikimedia.org/pipermail/design/2014-January/001441.html ).
Technical Operations
*Datacenter RFP https://wikimediafoundation.org/wiki/RFP/2013_Datacenter* The Wikimedia Operations team is in the final stages of the selection process. A short list of 4 bids has been created and final negotiations are underway. The winner of the bid will be selected in February based on the technical criteria listed in the RFP, and pricing.
Labs metrics in January:
- Number of projects: 131 - Number of instances: 441 - Amount of RAM in use (in MBs): 1,734,144 - Amount of allocated storage (in GBs): 23,505 - Number of virtual CPUs in use: 867 - Number of users: 2,595
*Wikimedia Labs https://www.mediawiki.org/wiki/Wikimedia_Labs* The Labs Migration team, consisting of Andrew Bogott and Marc-Andre Pelletier, have made good progress with testing the newest version of Openstack (called Havana https://www.openstack.org/software/havana/) and with Neutron https://wiki.openstack.org/wiki/Neutron, an OpenStack project to provide “networking as a service”. The plan is to upgrade the Openstack software when we migrate the Labs infrastructure out of the Tampa data center. Features Engineeringhttps://www.mediawiki.org/wiki/Wikimedia_Features_engineering Editor retention: Editing tools
*VisualEditor https://www.mediawiki.org/wiki/VisualEditor* In January, the VisualEditor team continued their work on improving the stability and performance of the system, and added some new features. Most of the team’s focus was on major new features and fixing bugs. You can now edit some page settings like whether to display a table of contents or whether to show section edit labels, set the size of a media file manually, see a keyboard shortcuts help screen, and create and edit media galleries using a very basic stand-in editor whilst the final form is being designed. Work also continued on a dialog for quickly adding “citation” references based on templates, more media and page settings, setting content language and right-to-left flags, and equation editing. The deployed version of the code was updated four times (1.23-wmf9https://www.mediawiki.org/wiki/MediaWiki_1.23/wmf9#VisualEditor, 1.23-wmf10https://www.mediawiki.org/wiki/MediaWiki_1.23/wmf10#VisualEditor, 1.23-wmf11https://www.mediawiki.org/wiki/MediaWiki_1.23/wmf11#VisualEditorand 1.23-wmf12https://www.mediawiki.org/wiki/MediaWiki_1.23/wmf12#VisualEditor ).
*Parsoid https://www.mediawiki.org/wiki/Parsoid*
In January, the Parsoid team did a lot of bug fixing around images, links, references and various other areas. See the deployment pagehttps://www.mediawiki.org/wiki/Parsoid/Deployments#Monday.2C_Feb_3.2C_2014_.40_11:30_am_PST_Y_Deployed_2d663ebfor a summary.
Part of the team has been mentoring two Outreach Program for Womenhttps://www.mediawiki.org/wiki/FOSS_Outreach_Program_for_Women/Round_7(OPW) interns. Others are mentoring a group of students in a Facebook Open Academy https://www.mediawiki.org/wiki/Facebook_Open_Academy project to build a Cassandra storage back-end for the Parsoid round-trip test server.
We also participated in the architecture summit, where our RFCs about embracing a service architecturehttps://www.mediawiki.org/wiki/Requests_for_comment/Services_and_narrow_interfaces, PHP bindings for serviceshttps://www.mediawiki.org/wiki/Requests_for_comment/PHP_Virtual_REST_Service, a general-purpose storage servicehttps://www.mediawiki.org/wiki/Requests_for_comment/Storage_servicebased on our Rashomon revision store, and a public content API based on thishttps://www.mediawiki.org/wiki/Requests_for_comment/Content_APIwere well received.
Following up on this, we started Debian packaging for Parsoid, which will soon make the installation of Parsoid as easy as apt-get install parsoid. Core Features
*Flow https://www.mediawiki.org/wiki/Flow/Project_information* This month, the Core Features team worked on integrating MediaWiki tools for dealing with spam and vandalism (AbuseFilter and Spam Blacklist) into Flow. We also launched an updated visual design and UI, based on the first round of experienced user feedback last month, as well as ongoing user testing with new users. Lastly, we created a script to disable Flow and return Flow discussions back into unstructured wikitext, so that we can begin trialing Flow in production in an extremely safe-to-fail manner. We are set to deploy our first trial on February 3, 2014 to two WikiProjects that volunteered on the English Wikipedia. Growth
*Growth https://www.mediawiki.org/wiki/Growth* In the first month of the year, the Growth team focused on two projects. First, we enhanced and refactored the GettingStartedhttps://www.mediawiki.org/wiki/Extension:GettingStartedextension, in part to support local configuration for different Wikipedias. The latest version of GettingStarted and GuidedTour will be released in English and 23 other languages in early February. Second, the team wrapped up several iterations of design and data analysis in support of upcoming work on Wikipedia article creationhttps://www.mediawiki.org/wiki/Wikipedia_article_creation. We presented new designs for the Draft namespace, and completed a series of remote usability testshttps://www.mediawiki.org/wiki/Draft_namespace/Usability_testing(see the resultshttps://www.mediawiki.org/wiki/Draft_namespace/Usability_testing/Results). We also finalized and published extensive quantitative analysishttps://meta.wikimedia.org/wiki/Research:Wikipedia_article_creationof trends in article creation across the largest Wikipedias. Last but not least, the Growth team welcomed its newest member in January, Software Engineer Sam Smith. Support
*Wikipedia Education Program https://www.mediawiki.org/wiki/Wikipedia_Education_Program* This month, once again we divided our time between the existing Education Program extension and work towards a new version of the software. We thoroughly analyzed database transactions in the current extension and fixed a slew of long-standing database-related bugs. Also on the current extension, we finished adding a notification type and notifications infrastructure, and worked on an improved course editing UX. For the new version, we studied workflow systems and considered how software for the Education Program and other outreach activities might use such a system. Adam Wight started on prototype workflow code. He also went through our code review backlog, bringing a multitude of new features and improvements to production. Mobile https://www.mediawiki.org/wiki/Wikimedia_Mobile_engineering
*Wikipedia Zero https://www.mediawiki.org/wiki/Wikipedia_Zero* During the last month, the team added forward compatibility to Varnish scripting for Wikipedia Zero, and resubmitted a Varnish script patch to support HTTPS for select Wikipedia Zero partners under the new IP address-based zero-rating scheme, after analysis with the Operations team. We also continued proof of concept work on an HTML5 web app for Firefox OS, fixed bugs in the legacy Firefox OS Wikipedia app, and prepared alpha functionality for the integration of Wikipedia Zero with the rebooted Android Wikipedia app. The team also continued work toward a generic JSON configuration extension for use by extensions like ZeroRatedMobileAcces, submitted code for the core MediaWiki API, submitted a ResourceLoader (RL) enhancement and cooperated on alternatives for performance enhancement of RL on non-WMF Redis-backed ResourceLoaders, and submitted a small UX enhancement for the Android rebooted Wikipedia app. January 2014 was also a month of planning: the partners engineering team met for two days with the business development team to plan for partners and Wikipedia Zero-related work at large. The partners engineering team also applied itself to two days of product planning for the Partner Portal. Finally, the team conducted normal tech facilitation to enable partner launches and align approaches with current and future partners.
*Mobile web projects https://www.mediawiki.org/wiki/Mobile_web_projects* We have been directing much of our attention over the last month at delivering a tablet-friendly MobileFrontend experience. We’ve added support for tables of contents in MobileFrontend for tablets, made some design improvements for tablets, and have worked towards making VisualEditor work with MobileFrontend for tablets (in alpha for now). We’ve hit some roadblocks and are hoping to collaborate more with the VE team in the near future to keep moving forward on the project. Following up from last month, we have also released our overlay UI improvements as well as an improved inline diff view for MobileFrontend into stable. Finally, we have also been working to expand our coverage of browser tests to facilitate quality assurance and help prevent the introduction of bugs and regressions. Language Engineeringhttps://www.mediawiki.org/wiki/Wikimedia_Language_engineering
*Language tools https://www.mediawiki.org/wiki/Language_tools* UniversalLanguageSelectorhttps://www.mediawiki.org/wiki/UniversalLanguageSelectorwas disabled on January 21 2014 in production for all language Wikimedia sites (other than wikidata.org) due to font delivery performance issues. Users can still enable ULS for their language needs by going to their user profile preferences and enabling ULS from the internationalization settings. Development is in progress for a solution to enable ULS when a user logs in and selects their language preferences explicitly to enable webfonts. David Chan continued his work on language support integration for VisualEditor https://www.mediawiki.org/wiki/VisualEditor for phase 5 languages https://www.mediawiki.org/wiki/VisualEditor#Timeline. Niklas Laxstrom and Santhosh Thottingal participated in the architecture summit in San Francisco in January in RFC discussions and JSONification of i18n supporthttps://wikimedia.mingle.thoughtworks.com/projects/scrum_of_scrums/cards/58for VisualEditor.
*Language Engineering Communications and Outreach https://www.mediawiki.org/wiki/Language_engineering_communications_and_outreach* The team continued their collaborative projects with Google, Twitter, Microsoft internationalization and MT teams on webfonts, input tools and machine translation.
*Content translation https://www.mediawiki.org/wiki/Content_translation* The language engineering team kicked off development of a prototype version of context translation workflow. This functionality aims to create a workspace for helping editors bootstrap new articles in non-Latin language Wikipedias. In the prototype, Russian and Welsh are being used for initial concept verification. Platform Engineeringhttps://www.mediawiki.org/wiki/Wikimedia_Platform_Engineering MediaWiki Core
*Site performance and architecture https://www.mediawiki.org/wiki/Site_performance_and_architecture* The team worked on performance dashboards for VisualEditorhttp://gdash.wikimedia.org/dashboards/ve/and page load time http://gdash.wikimedia.org/dashboards/frontend/, the ProfilerMwprof profiler class for MediaWiki, and draft performance guidelines.
*Admin tools development https://www.mediawiki.org/wiki/Admin_tools_development* This project is on hold, so there have been no significant developments, although some patches have been contributed by volunteer developers.
*Search https://www.mediawiki.org/wiki/Search* As of February 3, CirrusSearch is available as a Beta Feature on wikis representing about three quarters of all pages, and serves about 7.5% of our search traffic. Next month, we hope to get the hardware that we need to be a Beta Feature on the remaining wikis. We also hope to be the primary search back-end for more wikis. To that end, we’re working through performance and recall issues as well as trying to save space in the indexes.
*Auth systems https://www.mediawiki.org/wiki/Auth_systems* The team focused on minor updates to close some of the high priority OAuth bugs.
*Deployment Tooling https://www.mediawiki.org/wiki/Deployment_Tooling* The Logstash https://wikitech.wikimedia.org/wiki/Logstash service was deployed to production at https://logstash.wikimedia.org/. Work has started on an analysis of the current scap process which will be used to draft requirements for further deployment scripting work in the current quarter.
*Security auditing and response https://www.mediawiki.org/wiki/Security_auditing_and_response* We announced the MediaWiki 1.22.1 and 1.22.2 security releases, and continued to respond to reported vulnerabilities. Quality assurance
*Quality Assurance https://www.mediawiki.org/wiki/Quality_Assurance* January saw the QA team working closely with the Mobile team in particular to enhance the existing suite of test for MobileFrontend. We also participated in the discussion of the Release Engineering deployment process at the architecture summit. Hiring is underway for two open positions, QA Automation Engineer and Test Infrastructure Engineer.
*Beta cluster https://www.mediawiki.org/wiki/Beta_cluster* Beta is being used to test the Math extension rewrite. The Parsoid extension is now deploying continuously via a Jenkins job, status can be found on the CI dashboard job “Parsoid update”https://integration.wikimedia.org/dashboard/ bug 57233 https://bugzilla.wikimedia.org/show_bug.cgi?id=57233. The wikis now send updates to the irc.wikimedia.org server bug 60013https://bugzilla.wikimedia.org/show_bug.cgi?id=60013 .
*Continuous integration https://www.mediawiki.org/wiki/Continuous_integration* Zuul has been upgraded and uses a Gearman bus to communicate with Jenkins, the l10n-bot is no more triggering change and we enabled a proper gating system to test changes in parallel. The workflow is smoother and faster to provide feedback in Gerrit. Jenkins slave lanthanum does not offer direct access to internet, we configured the jobs to use a web proxy in MediaWiki (web proxy.eqiad.wmnet or webproxy.pmtpa.wmnet. Finally, the Zuul status page https://integration.wikimedia.org/zuul/ now shows the progress of jobs being run.
*Browser testing https://www.mediawiki.org/wiki/Quality_Assurance/Browser_testing* In January, we had a number of contributions from the students of Google Code-in https://www.mediawiki.org/wiki/Google_Code-in, from tests to Jenkins configuration to documentation. We released two entirely new features: one test that monitors the file upload API interface on both production Commons and beta labs Commons, and another test that monitors fatal errors in Beta Labs. We are very close to announcing general availability for two other new features: the ability to run tests headless using Firefox under Xvfb, and the ability to create test data like wiki pages in the target wiki at run time. Multimedia
*Multimedia https://www.mediawiki.org/wiki/Multimedia* In January, the multimedia team focused on developing the Media Viewer and planning our next projects for the year. Gilles Dubuc, Mark Holmquist, Gergo Tisza and volunteer Aaron Arcos implemented a number of improvements to the beta version of the Media Viewerhttps://www.mediawiki.org/wiki/Multimedia/About_Media_Viewer. Some of the features we created or improved include: faster image load, a full-screen mode, better navigation between files, an expanded meta-data panel with location, categories, permissions and assessments. We invite you to test the new UI features on this beta sitehttp://en.wikipedia.beta.wmflabs.org/wiki/Lightbox_demo; faster image load can be tested on this MediaWiki.org pagehttps://www.mediawiki.org/wiki/Lightbox_demo(In both cases, you need to create an account, then click on ‘Beta’ in your personal menu and enable Media Viewer.) Pau Giner also designed a new user interface for displaying slides, video and audio fileshttps://commons.wikimedia.org/w/index.php?title=File:Media_viewer_designs_-_media_support.pdfin the upcoming v0.3 version of Media Viewer, based on team recommendations. Fabrice Florin started a community discussion of our team’s Multimedia Vision for 2016https://commons.wikimedia.org/wiki/Commons:Multimedia_Features/Vision_2016, which proposes a range of improvements to help engage users and support productive collaborations in coming years (more comments welcomehttps://commons.wikimedia.org/wiki/Commons_talk:Multimedia_Features/Vision_2016). We also planned our work for this quarter’s releasehttps://wikimedia.mingle.thoughtworks.com/projects/multimedia/cards/grid?aggregate_property%5Bcolumn%5D=story+points&aggregate_property%5Brow%5D=story+points&aggregate_type%5Bcolumn%5D=sum&aggregate_type%5Brow%5D=sum&color_by=type&favorite_id=10762&filters%5B%5D=%5BType%5D%5Bis%5D%5BStory%5D&filters%5B%5D=%5BType%5D%5Bis%5D%5BBug%5D&filters%5B%5D=%5BType%5D%5Bis%5D%5BTech+debt%5D&filters%5B%5D=%5BType%5D%5Bis%5D%5BScope+Increase+%28UNPLANNED%29%5D&filters%5B%5D=%5BRelease%5D%5Bis%5D%5B%28Current+Release%29%5D&filters%5B%5D=%5BStatus%5D%5Bis%5D%5BIn+Analysis%5D&group_by%5Blane%5D=priority&group_by%5Brow%5D=status&lanes=Must+have%2CShould+have%2CCould+have&tab=Current+release, which focuses on Media Viewer through the end of March, and started planning our next big priorities for the rest of the year: UploadWizard and Structured Data on Commons. Lastly, we started a Request for Commentshttps://commons.wikimedia.org/wiki/Commons:Requests_for_comment/MP4_Videoabout possible support for the MP4 video standard: we invite you to participate in this discussion, which is due to end in mid-February; we will plan our next steps for video based on community feedback for this RfC. To discuss these projects and keep up with our work, we invite you to join the multimedia mailing listhttps://lists.wikimedia.org/mailman/listinfo/multimedia . Engineering Community Teamhttps://www.mediawiki.org/wiki/Engineering_Community_Team
*Bug management https://www.mediawiki.org/wiki/Bug_management* Valhallasw wrote a script to import tickets from JIRAhttps://github.com/wikimedia/pywikibot-sf-export/blob/master/jira.py(used by Toolserver) to Wikimedia Bugzilla, added a “Browse projects” link https://gerrit.wikimedia.org/r/#/c/104342/ to Bugzilla’s sidebar, and added an “Upload to Gerrit” buttonhttps://gerrit.wikimedia.org/r/#/c/110702/for Bugzilla attachments to the Greasemonkey triagescriptshttps://git.wikimedia.org/tree/wikimedia%2Fbugzilla%2Ftriagescripts. Inline displaying of image attachmentshttps://bugzilla.wikimedia.org/show_bug.cgi?id=54181#c44in Bugzilla was re-enabled and the default bug assignee was renamed from “Nobody” to “Nobody – You can work on this!”https://bugzilla.wikimedia.org/show_bug.cgi?id=57648to be more descriptive. The link to the guided bug entry form at the top of the standard bug entry form now sets the already chosen product directlyhttps://bugzilla.wikimedia.org/show_bug.cgi?id=59886, saving you two clicks when you switch to the guided form. Work continued on preparing the Bugzilla 4.4 upgrade: Andre Klapper’s patches for porting custom changes from 4.2 to 4.4 were deployed on the Bugzilla test instance on Zirconium and tested, and Daniel Zahn fixed a problem with Bugzilla’s collectstats.pl https://bugzilla.wikimedia.org/show_bug.cgi?id=29203, so the 4.4 upgrade and server move will take place in February. Bug management documentation related, Andre added a “Situation specific information” section to the Triage guidehttps://www.mediawiki.org/wiki/Bug_management/How_to_triagedocumentation about purging and profiling.
*Project management tools review https://www.mediawiki.org/wiki/Project_management_tools/Review* Andre Klapper reached out to the teampracticeshttps://lists.wikimedia.org/mailman/listinfo/teampracticesmailing list as well as individual stakeholders, asking users to share their workflow and needshttps://www.mediawiki.org/wiki/Talk:Project_management_tools/Reviewregarding project management and tracking tools. Guillaume Paumier summarized all that content into consolidated requirementshttps://www.mediawiki.org/wiki/Project_management_tools/Review/Requirements; those are now in the process of being compared to features offered by available tools, in order to assemble a shortlist of candidates for community discussion.
*Mentorship programs https://www.mediawiki.org/wiki/Mentorship_programs*
[image: Google Code-in 2013, the Wikimedia debut.pdf]https://commons.wikimedia.org/wiki/File:Google_Code-in_2013,_the_Wikimedia_debut.pdf
https://commons.wikimedia.org/wiki/File:Google_Code-in_2013,_the_Wikimedia_debut.pdf
Wikimedia’s first participation in the Google Code-Inhttps://www.mediawiki.org/wiki/Google_Code-Inprogram ended up with great success: 273 tasks completed by 46 students with the help of about 30 mentors. Theo Patt and Mateusz Maćkowski were selected winners for Wikimediahttp://google-opensource.blogspot.com/2014/01/google-code-in-2013-drumroll-please.html, and we sent a special mention to Mayank Madan.
Round 7 of the FOSS Outreach Program for Womenhttps://www.mediawiki.org/wiki/Outreach_Program_for_Women/Round_7started and all projects are on track so far:
- Compacting interlanguage linkshttps://www.mediawiki.org/wiki/User:Niharika/Project_Progress_Report#January_2014 - MediaWiki Homepage Redesignhttps://www.mediawiki.org/wiki/User:Monteirobrena/MediaWiki_Homepage_Redesign/Monthly_Reports#January - Complete the MediaWiki API development course on Codecademyhttps://www.mediawiki.org/wiki/User:Diwanshipandey/OPW_January_Report - Clean up Parsoid round-trip testing UIhttps://www.mediawiki.org/wiki/User:5xbe/OPW_Monthly_Progress_Reports#January - Clean up tracing/debugging/logging inside Parsoidhttps://www.mediawiki.org/wiki/User:Mariapacana/OPW_Progress_Report#January_2014 - UploadWizard: OSM Embeddinghttps://www.mediawiki.org/wiki/User:Inchikutty/OPW_Internship_Report#January
Facebook Open Academy https://www.mediawiki.org/wiki/Facebook_Open_Academy‘s warm-up period saw a slow progress in the beginning of the projects. At the end it seemed that everybody was waiting for the official start at the kick-off in Facebook headquarters on February 7−9.
*Technical communications https://www.mediawiki.org/wiki/Technical_communications* In January, Guillaume Paumier https://www.mediawiki.org/wiki/User:Guillomwrapped up work on mentoring Google Code-in https://www.mediawiki.org/wiki/Google_Code-in students and continued to provide ongoing communications supporthttps://www.mediawiki.org/wiki/Technical_communications/Tech_blog_activityfor the engineering staff. He contributed to writing, simplifying, publishing and distributing the weekly technical newsletterhttps://meta.wikimedia.org/wiki/Tech/News, and published an in-depth articlehttps://blog.wikimedia.org/2014/01/07/tech-news-fighting-technical-information-overload-for-wikimedians/explaining the process by which the newsletter is put together every week.
*Volunteer coordination and outreach https://www.mediawiki.org/wiki/Volunteer_coordination_and_outreach* We helped organizing the Architecture Summit 2014https://www.mediawiki.org/wiki/Architecture_Summit_2014in San Francisco (January 23−24) and we got everything ready for FOSDEM https://www.mediawiki.org/wiki/Events/FOSDEM in Brussels (February 1−2). We continued working with the tech community metricshttps://www.mediawiki.org/wiki/Community_metricsaround two key performance indicators: who contributes codehttp://korma.wmflabs.org/browser/who_contributes_code.html, and the Gerrit review queuehttp://korma.wmflabs.org/browser/gerrit_review_queue.html . Analytics https://www.mediawiki.org/wiki/Analytics
*Kraken https://www.mediawiki.org/wiki/Analytics/Kraken* The team has been monitoring the mobile stream and adding additional load to Kafka which has exposed some scaling issues. These have been resolved. In addition, work has been done with the Operations team on designing and implementing a Java deployment system for use with Hadoop and other systems. Finally, work has been initiated to use the data in the warehouse on mobile browser distribution and session length.
*Limn https://www.mediawiki.org/wiki/Analytics/Limn* Usability issues continue to be addressed while the team explores options around other visualization frameworks. This month we implemented a feature that simplifies the creation of dashboard by automatically inferring metadata from the data source.
*Wikimetrics https://www.mediawiki.org/wiki/Analytics/Wikimetrics* We are adding features to Wikimetrics to support scheduled jobs and data access via evergreen URLs. This will support dashboarding and other services that are will be built on top of the service. In addition, we are preparing a Wikimetrics-Vagrant image to help getting started with Wikimetrics development.
*Kafka https://www.mediawiki.org/wiki/Analytics/Logging_infrastructure* We’ve increased the throughput on Kafka from 6K/RPS to 50K/RPS to test stability under higher loads.
*Data Quality https://www.mediawiki.org/wiki/Analytics/Data_Quality*
https://commons.wikimedia.org/wiki/File:2013_Wikimedia_traffic_trends.pdf https://commons.wikimedia.org/wiki/File:2013_Wikimedia_traffic_trends.pdf
Review of 2013 traffic trends by the Wikimedia Analytics Team.
The team has spent an intense month analyzing data to explain the page view issues identified in December. The team’s reporthttps://commons.wikimedia.org/wiki/File:2013_Wikimedia_traffic_trends.pdfwas shared at the February metrics meeting.
*Research and Data https://www.mediawiki.org/wiki/Analytics/Research_and_Data*
We conducted a thorough review of traffic data and trendshttps://commons.wikimedia.org/wiki/File:2013_Wikimedia_traffic_trends.pdfand confirmed a downward trend in desktop pageviews in 2013. This trend is not reflected in desktop unique visitors or mobile traffic. We are working on complementing pageviews with other traffic metrics that will help us better monitor readership trends. We engaged with external parties (Google and comScore) to obtain data about referral and mobile traffic respectively.
We completed research on article creation trendshttps://meta.wikimedia.org/wiki/Research:Wikipedia_article_creationon the largest Wikipedias and found substantial differences between different language Wikipedias; specifically, where anonymous editors are allowed to create articles, their success rate (% of articles kept) is substantially higher than that of newly registered editors. We also found that articles that started as Articles for Creation (AfC) and userspace drafts have a near 100% success rate, but the transition that English Wikipedia made toward directing newcomers to start AfC drafts appears to have substantially reduced the amount of successful articles created by newcomers, presumably due to the large review backlog.
We published an updatehttps://meta.wikimedia.org/wiki/Research:Visual_editor_usage_%28January,_2014%29on Visual Editor usage on Wikipedia projects where the editor is enabled by default.
We continued work on metrics standardization for the editor engagement vital signshttps://www.mediawiki.org/wiki/Analytics/Epics/Editor_Engagement_Vital_Signsproject and published supportive analysis on definitions and parameter exploration for two proposed standardized user classes: new editorhttps://meta.wikimedia.org/wiki/Research:New_editorand productive new editor https://meta.wikimedia.org/wiki/Research:Productive_new_editor.
We worked with the Analytics Development and Legal teams to articulate use cases and the retention and anonymization strategyhttps://meta.wikimedia.org/wiki/Talk:Data_retention_guidelinesfor data subject to the retention guidelines, in particular with respect to user agents https://www.mediawiki.org/wiki/EventLogging/UserAgentSanitization.
We welcomed Sahar Massachi as a research contractor supporting the team with data analysis for fundraising tests and iterated on new modeling strategies for estimating test success (such as the number of dollars per banner impression). Before he joined us, Sahar worked with the fundraising team, where most recently he focused on writing tools to help the team easily and quickly understand the results of each test. Offline
*Kiwix http://www.kiwix.org* *The Kiwix project is funded and executed by Wikimedia CH https://meta.wikimedia.org/wiki/Wikimedia_CH.* Much time this month was spent planning for 2014. We mainly worked on mwofflinerhttps://sourceforge.net/p/kiwix/other/ci/master/tree/mwoffliner/and almost managed to create a full English Wikipedia ZIM file with thumbnails. The upgrade of our main storage platform allowed us to start our automatic ZIM file generation system. Wikidata https://meta.wikimedia.org/wiki/Wikidata
*The Wikidata project is funded and executed by Wikimedia Deutschland https://meta.wikimedia.org/wiki/Wikimedia_Deutschland/en.* In January, the team worked mainly on performance improvements around Wikidata. The Quantities datatype was deployed so it is now possible to enter data like the number of inhabitants of a country. Wikisource can now manage its language links via Wikidata as well just like Wikipedia, Wikivoyage and Commons could already. Two new front-end developers, Adrian and Thiemo joined the team to help improve Wikidata’s user interface. Last but not least, the team released their plan for the development of Wikidata in 2014 and beyond https://www.wikidata.org/wiki/Wikidata:Development_plan . Future The engineering management team continues to update the *Deployments https://wikitech.wikimedia.org/wiki/Deployments* page weekly, providing up-to-date information on the upcoming deployments to Wikimedia sites, as well as the *annual goals https://www.mediawiki.org/wiki/Wikimedia_Engineering/2013-14_Goals*, listing ongoing and future Wikimedia engineering efforts. ------------------------------
*This article was written collaboratively by Wikimedia engineers and managers. See revision history https://www.mediawiki.org/w/index.php?title=Wikimedia_engineering_report/2014/January&action=history and associated status pages. A wiki version https://www.mediawiki.org/wiki/Wikimedia_engineering_report/2014/January is also available.*