Hi,
The report covering Wikimedia engineering activities in July 2013 is now available.
Wiki version: https://www.mediawiki.org/wiki/Wikimedia_engineering_report/2013/July Blog version: https://blog.wikimedia.org/2013/08/05/engineering-july-2013-report/
We're also proposing a shorter, simpler and translatable version of this report that does not assume specialized technical knowledge: https://www.mediawiki.org/wiki/Wikimedia_engineering_report/2013/July/summar...
Below is the full HTML text of the report.
As always, feedback is appreciated on the usefulness of the report and its summary, and on how to improve them.
------------------------------------------------------------------
Major news in July include:
- Givinghttps://blog.wikimedia.org/2013/07/01/visualeditor-beta-rollout/more editors an easy-to-use editing interface (the VisualEditor) on several Wikipedias - Improving language support on our sites via summer interns' projectshttps://blog.wikimedia.org/2013/07/23/updates-from-the-language-engineering-google-summer-of-code-projects/and easier configuration optionshttps://blog.wikimedia.org/2013/07/01/universal-language-selector-uls-deployed-on-more-than-150-wikis/, and askinghttps://blog.wikimedia.org/2013/07/31/translate-ui-wikipedias-visual-editor/for help translating the VisualEditor interface - Enablinghttps://blog.wikimedia.org/2013/07/25/edit-wikipedia-on-the-go/users to edit our sites from mobile devices, like phones and tablets, and announcinghttps://blog.wikimedia.org/2013/07/31/cooper-ux-bootcamp-wikipedia-mobile-editing/a future user experience bootcamp focusing on mobile editing - Finishinghttps://blog.wikimedia.org/2013/07/23/pywikipediabot-moving-to-git-on-july-26/our transition from keeping source code in Subversion to storing it in Git - Launchinghttps://blog.wikimedia.org/2013/07/25/aircel-partnership-brings-wikipedia-zero-to-india/a Wikipedia Zero partnership with Aircel, giving mobile subscribers in India the potential to access Wikipedia at no data cost - Updatinghttps://blog.wikimedia.org/2013/08/01/future-https-wikimedia-projects/the Wikimedia movement on how we intend to protect our users' privacy with HTTPS - Signinghttps://blog.wikimedia.org/2013/07/26/future-third-party-releases-mediawiki/a contract with longtime MediaWiki contributors to manage MediaWiki releases for the open source community - Explaininghttps://blog.wikimedia.org/2013/07/03/wikimedia-foundation-fixes-software-bugs/how we find and gather software problems and deliver the fixes to users
*Note: We're also providing a shorter, simpler and translatable version of this reporthttps://www.mediawiki.org/wiki/Wikimedia_engineering_report/2013/July/summarythat does not assume specialized technical knowledge. *
Personnel Work with us https://wikimediafoundation.org/wiki/Work_with_us
Are you looking to work for Wikimedia? We have a lot of hiring coming up, and we really love talking to active community members about these roles.
- Software Engineer - Fundraisinghttp://hire.jobvite.com/Jobvite/Job.aspx?j=oawpXfwM - Software Engineer - Language Engineeringhttp://hire.jobvite.com/Jobvite/Job.aspx?j=oH3gXfwH - Software Engineer - Multimedia Systemshttp://hire.jobvite.com/Jobvite/Job.aspx?j=oj40Wfw3 - Senior Software Engineer - Platformhttp://hire.jobvite.com/Jobvite/Job.aspx?j=ouLnWfwi - Product Manager - Platformhttp://hire.jobvite.com/Jobvite/Job.aspx?j=o3vtXfwI - Dev-Ops Engineer - SREhttp://hire.jobvite.com/Jobvite/Job.aspx?j=ocLCWfwf - Software Engineer - Editor Engagement Experimentationhttp://hire.jobvite.com/Jobvite/Job.aspx?j=o8NJXfwl - Software Engineer - Editor Engagementhttp://hire.jobvite.com/Jobvite/Job.aspx?j=o6NJXfwj - Director of Program - Mobilehttp://hire.jobvite.com/Jobvite/Job.aspx?j=oGqAXfwn - Front-end Developer - Analyticshttp://hire.jobvite.com/Jobvite/Job.aspx?j=olyGXfwg
Announcements
- Bryan Davis joined the Platform Engineering team as a Senior Software Engineer, working generally on backend software issues and starting off supporting multimedia (announcementhttp://lists.wikimedia.org/pipermail/wikitech-l/2013-July/070923.html ). - C. Scott Ananian joined the Parsoid team as a Senior Features Engineer (announcementhttp://lists.wikimedia.org/pipermail/wikitech-l/2013-July/070316.html ). - Kenan Wang joined the Product team as Product Manager for Mobile ( announcementhttps://twitter.com/wikimediaatwork/status/359473872527044609 ).
Technical Operations
*Site infrastructure* Lots of Puppethttps://blog.wikimedia.org/2013/02/01/from-duct-tape-to-puppets/refactoring work got done this month, including considerable reorganization of the puppet masters. Several manifests have been moved into modules, but completing this project will take many months.
*Data Dumps https://www.mediawiki.org/wiki/WMF_Projects/Data_Dumps* The English Wikipedia dumps ran out of our Ashburn data center this month, and so did a number of other big wikis' dumps. There's an issue with the abstract dumps that needs to be sorted out for those, but other than that everything ran smoothly.Petr Onderka has been getting a lot of work done on the incremental dumps. A first preview of the code was announcedhttp://lists.wikimedia.org/pipermail/wikitech-l/2013-July/070971.htmlas well as a proposed binary file format which the program currently uses. For a preview of what's coming up, you can check the timelinehttps://www.mediawiki.org/wiki/User:Svick/Incremental_dumps#Timeline. Your comments and suggestions are welcome!
*Wikimedia Labs https://www.mediawiki.org/wiki/Wikimedia_Labs* Though there were some features introduced this month, the majority of our time was spent on documentation, tracking down bugs and improving usability. We had a documentation sprint this month, targeted at improving documentation for the Tool Labs http://tools.wmflabs.org/ project. Work continued on stabilizing the NFS server -- we believe we've tracked down the stability issues to RAID controller problems. The compute nodes are becoming increasingly low on disk space, but we've tracked this down in a change in behavior of nova and have deployed a fix. nova-network was starting to experience timeouts due to excessive load, leading to instance creation failures. We've extended dhcp renewal times to reduce load. We upgraded wikitech.wikimedia.org and wiktech-static to the 1.22wmf11 version of MediaWiki. We also: deployed the AJAX-enabled delete instance feature; deployed a change to display more informative instance statuses; fixed issues in LdapAuthentication that broke blocking and renaming users; and deployed a change to allow service groups to be added to service groups, to make sharing code and data between tools easier. Features Engineeringhttps://www.mediawiki.org/wiki/Wikimedia_Features_engineering Editor retention: Editing tools
*VisualEditor https://www.mediawiki.org/wiki/VisualEditor* In July, the VisualEditor team began switching the deployment from opt-in alpha to opt-out beta, so becoming the default editor for users of the various Wikipedias. The deployed version of the code was updated three times (1.22-wmf10https://www.mediawiki.org/wiki/MediaWiki_1.22/wmf10#VisualEditor, 1.22-wmf11https://www.mediawiki.org/wiki/MediaWiki_1.22/wmf11#VisualEditorand 1.22-wmf12https://www.mediawiki.org/wiki/MediaWiki_1.22/wmf12#VisualEditor), with several mid-deployment releases as the code was developed to patch urgent issues. There were a number of user interface improvements, most notably to the references insertion dialog, alongside fixes to a number of bugs uncovered by the community.
*Parsoid https://www.mediawiki.org/wiki/Parsoid* In July, the Parsoid team supported the deployment of VisualEditor as default editor on eight Wikipedias, continuing to monitor bug reports, feedback pages, and village pump and fixed a number of bugs to eliminate instances of dirty diffs and other corruption that were reported. An absence of performance issues let us focus our attention on functionality and dirty-diff related bugs. This continued to be the primary focus of our work this month. On the staffing side, C. Scott Ananian joined the Parsoid team as a full-time employee -- he has been working with us since earlier this year, first as a volunteer and then as a contractor. Marc Ordinas i Llopis from Spain and Arlo Breault from Canada joined the Parsoid team as contractors this month. Editor engagement features
*Flow https://www.mediawiki.org/wiki/Flow_Portal/Project_information* This month, we released two new prototypeshttps://www.mediawiki.org/wiki/Flow_Portal/Interactive_Prototypeto showcase some ideas around Flow-enabled user-to-user discussion. We continued to collect user feedback and prioritize use-cases for a potential minimum viable product.
*Notifications https://www.mediawiki.org/wiki/Echo_%28Notifications%29* In July, we released our final features for Notifications on the English Wikipedia and mediawiki.org and meta.wikimedia.org. Benny Situ completed development of HTML Email notificationshttps://www.mediawiki.org/wiki/Echo/Feature_requirements#HTML_single_email_notifications, as well as improved notificationshttps://www.mediawiki.org/wiki/Echo/Feature_requirements#Improved_Notification_Structure, based on designs by Vibha Bamba. Fabrice Florin managed the release of these final features, and prepared this release planhttps://www.mediawiki.org/wiki/Echo/Release_Plan_2013to deploy Notifications on more wiki projects, starting with French and Polish Wikipedias in August. Dario Taraborelli and Matthias Mullie updated our new metrics dashboardshttp://ee-dashboard.wmflabs.org/dashboards/enwiki-features, while Aaron Halfaker completed his report on our A/B test of new user activityhttps://meta.wikimedia.org/wiki/Research:Notifications/Experiment_1/Results. To learn more, visit the project portalhttps://en.wikipedia.org/wiki/Wikipedia:Notifications, read the FAQ pagehttps://en.wikipedia.org/wiki/Wikipedia:Notifications/FAQand join the discussion on the talk page https://en.wikipedia.org/wiki/Wikipedia_talk:Notifications.
*Article feedback https://www.mediawiki.org/wiki/Article_feedback* In July, we deployed a few last features and bug fixes for the Article Feedback Tool https://www.mediawiki.org/wiki/Article_feedback/Version_5(AFT5) on the English https://en.wikipedia.org/wiki/Special:ArticleFeedbackv5 and Frenchhttps://en.wikipedia.org/wiki/fr:Sp%C3%A9cial:ArticleFeedbackv5Wikipedias. Matthias Mullie released the auto-archivehttps://www.mediawiki.org/wiki/Article_feedback/Version_5/Feature_Requirements#Auto-archive_commentsfeature, as well as this list of articles with feedback enabled on enwikihttp://ee-prototype.wmflabs.org/metrics/aftv5_enabled_pages.php?wiki=enwikiand on frwikihttp://ee-prototype.wmflabs.org/metrics/aftv5_enabled_pages.php?wiki=frwiki. At the request of the French Wikipedia community, he also developed new feedback notificationshttps://www.mediawiki.org/wiki/Echo/Feature_requirements#Watched_Page_-_Feedbackto let users know when feedback is marked as useful for a page they watch (or for a comment they posted). The team plans to make the AFT5 tool available to other wiki projects interested in testing this tool, provided that no new development is required to support their needs, as outlined in the release planhttps://www.mediawiki.org/wiki/Article_feedback/Version_5/Release_Plan_2013 . Editor engagement experiments *We're hiring!* Are you a front-end developer? Do you know someone who is? Apply today http://hire.jobvite.com/Jobvite/Job.aspx?j=o8NJXfwl&c=qSa9VfwQ.
Editor engagement experimentshttps://www.mediawiki.org/wiki/Editor_engagement_experiments ** In July, the Editor Engagement Experiments (E3) team made progress on a number of continuing projects. In terms of features, the team also completed work to integrate the onboarding new Wikipedianshttps://www.mediawiki.org/wiki/Onboarding_new_Wikipediansproject with new infrastructural changes and feature releases.
For the GettingStartedhttps://www.mediawiki.org/wiki/Extension:GettingStarted, E3 collaborated with Platform engineering to ensure compatibility with the new "SUL2 https://www.mediawiki.org/wiki/SUL2" cross-wiki authentication architecture. For the GuidedTour extensionhttps://www.mediawiki.org/wiki/Extension:GuidedTour, the team completed a first release of support for guided tours of the VisualEditor https://www.mediawiki.org/wiki/VisualEditor interface, alongside tours of the legacy wikitext editor, and developed a plan to refactorhttps://www.mediawiki.org/wiki/Extension:GuidedTour/Refactoring_brainstormingthe GuidedTour extension as well as its API. E3 also planned for its sixth A/B test of the GettingStarted workflow (see proposed specification and mockups https://www.mediawiki.org/wiki/Onboarding_new_Wikipedians#Proposed). As an addition to the team's redesign of account creation and login (launched in May-June), we enhanced the design of the form for users who fulfill account creation requests for others.
E3 team member Matthew Flaschen also worked with two Google Summer of Codehttps://www.mediawiki.org/wiki/GSOC_2013students on their projects. Richa Jain https://www.mediawiki.org/wiki/User:Rjain is working on the Annotator extension https://www.mediawiki.org/wiki/Extension:Annotator, which allows adding inline comments to a wiki page. Rahul Maliakkalhttps://www.mediawiki.org/wiki/User:Rahul21is working on the Pronunciation Recording extensionhttps://www.mediawiki.org/wiki/Extension:PronunciationRecording, for adding audio of pronunciations to Wiktionary. On the experimental tools and data analysis front, E3 completed a significant rewrite of the Puppethttps://wikitech.wikimedia.org/wiki/Puppetconfiguration for EventLogging https://www.mediawiki.org/wiki/EventLogging, our data collection pipeline, among other changes. For the MediaWiki-Vagranthttps://www.mediawiki.org/wiki/MediaWiki-Vagrantportable desktop development environment, E3 added support for flexibly provisioning and unit testing extensions such as GettingStarted, GuidedTour, ParserFunctions https://www.mediawiki.org/wiki/ParserFunctions, EventLogging, and others. Last but not least, the micro-surveyhttps://meta.wikimedia.org/wiki/Research:Gender_micro-surveyof gender of new account registrations was enabled on German, French, Italian, and Polish Wikipedias, while data analysis on the English Wikipedia results began. Support
*2013 Wikimedia fundraiserhttps://www.mediawiki.org/wiki/2013_Wikimedia_fundraiser * In July, the fundraising team did its first successful tests of our new payments gateway: Adyen. The (as yet) US-only Credit Card backup gateway performed similarly to our primary credit card processor in A/B testing, and can be successfully used as a failover. We also ran, for the first time, several short campaign tests targeted at mobile devices in the US. In these tests, users were able to choose between Paypal or Amazon Payments. Additional tests to determine peak times, appropriate localities, and optimum messaging for mobile campaigns will continue throughout August, as the campaigns are prepared. Mobile https://www.mediawiki.org/wiki/Wikimedia_Mobile_engineering
*Wikipedia Zero https://www.mediawiki.org/wiki/Wikipedia_Zero* This month, the team launched Wikipedia Zero with Aircel in Indiahttps://blog.wikimedia.org/2013/07/25/aircel-partnership-brings-wikipedia-zero-to-india/, a carrier with about 60 million cellphone subscribers. We also completed our first cut of automation testing, started the implementation of the Wikipedia Zero software re-architecture, and patched bugs. During July we planned for the upcoming year in Wikipedia Zero. On the engineering front we are focusing first on test automation and re-architecture concurrent with SMS/USSD and J2ME releases, and afterward will be focusing efforts on end user UX and carrier-oriented enhancements that will support the continued growth of the program.
*Mobile web projects https://www.mediawiki.org/wiki/Mobile_web_projects* This month, the mobile web team released a new contributory navhttps://blog.wikimedia.org/2013/07/25/edit-wikipedia-on-the-go/to all Wikimedia mobile sites, including the existing upload and watchlist star features, as well as an edit button. This means that editing (in the form of section-level markup editing) is now enabled on all mobile Wikimedia sites for logged in users. In beta, we began work on mobile notifications restyling, as well as guiders for first-time editors and uploaders. Platform Engineeringhttps://www.mediawiki.org/wiki/Wikimedia_Platform_Engineering MediaWiki Core
*MediaWiki 1.22 https://www.mediawiki.org/wiki/MediaWiki_1.22/Roadmap* In July 2013, MediaWiki 1.22wmf10 through 1.22wmf13 were successfully deployed to Wikimedia project sites. We skipped the week of July 4th as there was reduced capacity in both engineering and operations due to the US holiday. We also named Markus Glaser and Mark Hershbergerhttps://blog.wikimedia.org/2013/07/26/future-third-party-releases-mediawiki/as the new contractors maintaining the MediaWiki "tarball" for release to other system administrators and organizations.
*Git conversion https://www.mediawiki.org/wiki/Git/Conversion* With the migration of pywikipediabot from Subversion to Githttps://blog.wikimedia.org/2013/07/23/pywikipediabot-moving-to-git-on-july-26/, we were able to switch svn.wikimedia.org to read-only mode, thus completing this migration. We plan to keep the Subversion service around indefinitely for archival purposes, and can still migrate any dormant project that hasn't already been migrated on request.
*Multimedia https://www.mediawiki.org/wiki/Multimedia* In July, we continued to expand our multimedia teamhttps://www.mediawiki.org/wiki/Multimedia: Mark Holmquist joined as front-end software developer, working with product manager Fabrice Florin and engineering director Rob Lanphier, as well as contractors Brian Wolff and Jan Gerber. We prepared a first multimedia plan for the coming yearhttps://www.mediawiki.org/wiki/Multimedia/2013-14_Goalsand discussed our goals with community members in two separate events: a multimedia roundtablehttps://meta.wikimedia.org/wiki/Roundtables/Roundtable_2and an IRC chathttps://meta.wikimedia.org/wiki/IRC_office_hours/Office_hours_2013-07-18b. Based on community feedback, we identified five main areas of activity 2013-2014: improving the viewing experience and upload pipeline in the first half of the year, then focusing on file curation, discovery and placement in articles for the second half of the year. Our overall goals for this year are to increase both the number of contributions and files used in Wikipedia articles. For now, we have started work on a new media viewer https://mingle.corp.wikimedia.org/projects/multimedia/cards/8 to display images in larger size when you click on a thumbnail, as well as display file information and a full-screen viewing option, right on the same page. We plan to have a first version of that feature next month, and will be testing it as part of a beta experimenthttps://mingle.corp.wikimedia.org/projects/multimedia/cards/2on a few pilot sites. We will also be hosting more community planning discussions, such as this multimedia roundtable at Wikimania 2013http://wikimania2013.wikimedia.org/wiki/Submissions/Multimedia_Roundtable. To participate in these discussions and keep up with our work, we invite you to join this new multimedia mailing listhttps://lists.wikimedia.org/mailman/listinfo/multimedia. Last but not least, we are also recruiting for two more positions for our team: a multimedia systems engineerhttp://hire.jobvite.com/Jobvite/Job.aspx?j=oj40Wfw3&c=qSa9VfwQand a senior software engineerhttp://hire.jobvite.com/Jobvite/Job.aspx?j=ouLnWfwi&c=qSa9VfwQ. Please spread the word about this unique opportunity to create a richer multimedia experience for Wikipedia and MediaWiki sites!
*Admin tools developmenthttps://www.mediawiki.org/wiki/Admin_tools_development * This activity was on hiatus in August.
*Search https://www.mediawiki.org/wiki/Search* Nik Everett and Chad Horohoe have continued writing an extension to implement ElasticSearch searching for MediaWiki, and we've finished most of the required features. Next comes getting it deployed, scaled, and fixing the inevitable bugs. We're aiming to deploy to the test site beta.wmflabs.org before the end of the month. Peter Youngmeister and Asher Feldman will be handling the operations tasks for the new setup.
*Auth systems https://www.mediawiki.org/wiki/Auth_systems* Engineers worked towards for an OAuthhttps://www.mediawiki.org/wiki/Auth_systems/OAuthdeployment to the beta cluster https://www.mediawiki.org/wiki/Beta_cluster in early August, and aim to roll OAuth out to the test wikis (e.g., test2.wikipedia.org) after Wikimania.
*HipHop deployment https://www.mediawiki.org/wiki/HipHop_deployment* HipHop work was mainly on hold in Julyhttp://lists.wikimedia.org/pipermail/hiphop/2013-July/000043.html, with the exception of some minor work on virtual machines.
*Security auditing and responsehttps://www.mediawiki.org/wiki/Security_auditing_and_response * The team continued to respond to reported security issues, and addressing outstanding bugs. Quality assurance
*Quality Assurance https://www.mediawiki.org/wiki/Quality_Assurance* This month QA made contributions to the VisualEditor, UniversalLanguageSelector and Mobile web projects, among others, finding and reporting issues in a timely manner. Our intern with the Outreach Program for Women https://www.mediawiki.org/wiki/User:Rachel99 is working on more automated browser tests. We continue to engage our community on the QA mail list https://lists.wikimedia.org/mailman/listinfo/qa and in live sessions, where we have several contributors (see Volunteer coordination and outreachhttps://www.mediawiki.org/wiki/Volunteer_coordination_and_outreach ).
*Beta cluster https://www.mediawiki.org/wiki/Beta_cluster* The Beta cluster continues to be a target for automated and manual testing. It also finally has a syslog receiver on deployment-bastion, thus solving bug 36748 https://bugzilla.wikimedia.org/show_bug.cgi?id=36748 (no syslog::server in beta). The logs can be accessed via either /home/wikipedia/syslog or /data/project/logs/syslog/ . This is thanks to Leslie Carr.
*Browser testinghttps://www.mediawiki.org/wiki/Quality_Assurance/Browser_testing * In July we added coverage for a number of features, including VisualEditor, UniversalLanguageSelector, and Mobile Search. We are making extensive use of beta labs as well as the test2wiki test environment. Our automated browser tests continue to identify important issues during feature development. Analytics
We reviewed our planning document with the Sue and Erik and the Engineering Directors. Reception was positive and we will be communicating next steps more widely in August. The Analytics team focused on short term deliverables, reliability and hiring in July. We identified two potential candidates for front-end/Python work. We have been performing multiple phone screens together with Recruiting, and the hiring pipelines are good.
*Analytics infrastructurehttps://www.mediawiki.org/wiki/Analytics/Infrastructure *
*Kraken*:
- We kicked off a reliability project with Ops with the end goal of stabilizing Hadoop and the logging infrastructure. Teams have been in discussions on architecture and planning, and should have a path forward in the next 2 weeks. We identified a consultant who will perform a system audit to aid the project.
- We continue adding new metricshttps://gerrit.wikimedia.org/r/#/c/75766/and alerts to monitor all the different parts of the webrequest dataflows into Kraken. We expect to keep making improvements in the coming months until we have a fully reliable data pipeline into Kraken.
- We puppetized Hue https://gerrit.wikimedia.org/r/74388, Hivehttps://gerrit.wikimedia.org/r/71569, and Oozie https://gerrit.wikimedia.org/r/69804. We also have a working setup of the Hadoop cluster in Labs for testing purposeshttps://kraken.wmflabs.org:8888/. All Puppet work is open sourced.
*Logging Infrastructure*:
- We started this month with designing a canary event monitoring systemhttps://mingle.corp.wikimedia.org/projects/analytics/cards/789. A canary event is an artificial event that is injected at the start of the data workflow and which we will monitor to see it reaches its final destination; that way we can ensure that the dataflows are functioning.
- We are investigating what data format to use for sending the webrequest messages from Varnish to the Hadoop cluster. Formats that we are scrutinizing are JSON, Protobuf and AVRO, but we are also looking at compressions algorithms such as Snappy.
*Analytics Visualization, Reporting & Applicationshttps://www.mediawiki.org/wiki/Analytics/Visualization,_Reporting_%26_Applications *
*Wikimetrics*: We successfully launched the initial version of Wikimetrics: see metrics.wmflabs.org. This version has support for cohort upload and two metrics: 1) bytes added and 2) namespace edits. We are working on adding support for time-series and aggregators. In the coming sprints we will focus on adding new metricshttps://mingle.corp.wikimedia.org/projects/analytics/cards/779 .
*Wikipedia Zero*: Dashboards have been moved off of Hadoop for the time being and are now being populated again. We have identified some issues with logrotation that are causing gaps in the graphs, and will look into these problems. Also, we have been working on technical handoff as Evan Rosen leaves the Foundation.
*Limn*: No development news. *Wikistats*: No development news.
*Data Releases https://www.mediawiki.org/wiki/Analytics/Data_Releases*
- Erik Zachte published data and longitudinal analyses of edit and revert trends for Wikimedia projects (read the announcementhttp://infodisiac.com/blog/2013/07/new-edit-and-revert-stats/). We provided data and ad-hoc analysis for the presentation *A State of Decline? The State of Wikimedia Communities as of July 2013* at the July 2013 Monthly Metrics Meetinghttps://meta.wikimedia.org/wiki/Metrics_and_activities_meetings/2013-07-11 . - We published the analysis of a controlled experiment that we ran in June to test the Impact of notificationshttps://meta.wikimedia.org/wiki/Research:Notifications/Experiment_1on new contributors and a pre-release A/B test of Visual Editor https://meta.wikimedia.org/wiki/Research:VE on the English Wikipedia. We performed an extensive audit of the quality of the data collected during and after the VE test, taking into account browser limitations and known bugs, and posted an updatehttps://meta.wikimedia.org/wiki/Research_talk:VisualEditor#Updates_on_VE_data_analysison the state of the analysis. We released via our open data repository the complete dataset http://datahub.io/dataset/wikimedia-visualeditor of the sample of new registered users who participated in the split test to ensure the replicability of the analysis. - We released real-time dashboards on edit activity, new account registrations and reverts for the 10 Wikipedias on which VE has been rolled out. (en http://ee-dashboard.wmflabs.org/dashboards/enwiki-metrics • dehttp://ee-dashboard.wmflabs.org/dashboards/dewiki-metrics• es http://ee-dashboard.wmflabs.org/dashboards/eswiki-metrics • frhttp://ee-dashboard.wmflabs.org/dashboards/frwiki-metrics• he http://ee-dashboard.wmflabs.org/dashboards/hewiki-metrics • ithttp://ee-dashboard.wmflabs.org/dashboards/itwiki-metrics• nl http://ee-dashboard.wmflabs.org/dashboards/nlwiki-metrics • plhttp://ee-dashboard.wmflabs.org/dashboards/plwiki-metrics• ru http://ee-dashboard.wmflabs.org/dashboards/ruwiki-metrics • svhttp://ee-dashboard.wmflabs.org/dashboards/svwiki-metrics )
Engineering community team
*Bug management https://www.mediawiki.org/wiki/Bug_management* A PATCH_TO_REVIEW status was introduced in Bugzillahttp://lists.wikimedia.org/pipermail/wikitech-l/2013-July/070759.htmlwhich is automatically set (by the Gerrit Notification Bot) on bug reports when a commit message in Gerrit mentions a corresponding bug number. Andre prepared a patch for using the InlineHistory extension in Bugzillahttps://bugzilla.wikimedia.org/show_bug.cgi?id=47256and a patch https://bugzilla.wikimedia.org/show_bug.cgi?id=36762 to make Bugzilla's guided bug entry formhttps://bugzilla.wikimedia.org/enter_bug.cgi?product=Wikimedia&component=Bugzilla&format=guidedfor new users usable for Wikimedia Bugzilla. Andre also continued his weekly blogposts of Bugzilla tipshttps://www.mediawiki.org/wiki/Bug_management#Tricks_and_best_practices_in_Bugzilla. Thanks to Daniel Zahn, Bugzilla administrators now regularly receive an email with a database dump of Bugzilla's "audit log"https://gerrit.wikimedia.org/r/#/c/56562/which lists the most recent taxonomy changes in Bugzilla (component or keyword additions, etc.). In Bugzilla's taxonomy, the components in the "Parsoid" product https://bugzilla.wikimedia.org/show_bug.cgi?id=50685were reorganized as requested by its main developer, and the remaining open "OggHandler" tickets were closed as it has been superseded by TimedMediaHandler.
*Mentorship programs https://www.mediawiki.org/wiki/Mentorship_programs* Quim Gil https://www.mediawiki.org/wiki/User:Qgil organized meetings with each Google Summer of Codehttps://www.mediawiki.org/wiki/Summer_of_Code_2013and Outreach Program for Womenhttps://www.mediawiki.org/wiki/Outreach_Program_for_Womenteam, one by one. Most projects were already at full speed, and for them, the meeting was primarily social and nice to have. A few really benefited from going through a checklist to highlight early problems easy to solve now. All GSoC and OPW projects, 21 in total, are now on track.
*Technical communicationshttps://www.mediawiki.org/wiki/Technical_communications * Like in June, Guillaume Paumierhttps://www.mediawiki.org/wiki/User:Guillomwas seconded to the VisualEditor deployment effort, working on communications, documentation and liaising with the French Wikipedia. Work on technical communications mostly focused on perennial activities like ongoing communications support to the engineering staff.
*Volunteer coordination and outreachhttps://www.mediawiki.org/wiki/Volunteer_coordination_and_outreach * On Community metrics https://www.mediawiki.org/wiki/Community_metrics, Quim Gil https://www.mediawiki.org/wiki/User:Qgil focused on the consolidation of korma.wmflabs.org, the new dashboard for automated community metrics. We have made good progress on this alpha, including basic metrics from Git, Bugzilla and mailing lists being retrieved on a daily basis, and have filed bugs and enhancement requests on GitHub (mediawiki-dashboardhttps://github.com/Bitergia/mediawiki-dashboard/issues, VizGrimoireJS https://github.com/VizGrimoire/VizGrimoireJS/issues). We are deciding on the key metrics we need in order to make decisions, e.g. average time to resolve on Gerrit changesets or bug reports. We also planned and promoted a Browser Testing Automation workshop with Cucumberhttps://www.mediawiki.org/wiki/Meetings/2013-07-18together with the QA team, with 13 people participating online. You can watch the session here https://www.mediawiki.org/wiki/Meetings/2013-07-18(1h40). The experience was useful, as we agreed on MediaWiki-Vagrant https://www.mediawiki.org/wiki/Vagrant as the default environment for automated testing and highlighted the list of easy bugshttps://bugzilla.wikimedia.org/buglist.cgi?keywords=easy%2C%20&keywords_type=allwords&resolution=---&resolution=LATER&resolution=DUPLICATE&query_format=advanced&component=Quality%20Assurance&product=Wikimedia&list_id=219895. Also, the Engineering Community team held its quarterly reviewhttp://lists.wikimedia.org/pipermail/wikitech-l/2013-July/070632.html . Language engineeringhttps://www.mediawiki.org/wiki/Wikimedia_Language_engineering The language team deployed Universal Language Selector (ULS)https://www.mediawiki.org/wiki/Universal_Language_Selectorto most Wikimedia wikis to provide easier configuration optionshttps://blog.wikimedia.org/2013/07/01/universal-language-selector-uls-deployed-on-more-than-150-wikis/to readers and contributors. ULS provides a flexible way to configure and deliver language settings like interface language, fonts, and input methods (keyboard mappings). Also, ULS allows users to type text in different languages not directly supported by their keyboard, read content in a script for which fonts are not available locally, or customise the language in which menus are displayed. For more information, please see the FAQhttps://www.mediawiki.org/wiki/Universal_Language_Selector/FAQ . The Language engineering team also mentored summer interns' projectshttps://blog.wikimedia.org/2013/07/23/updates-from-the-language-engineering-google-summer-of-code-projects/to improve language support on our sites, and askedhttps://blog.wikimedia.org/2013/07/31/translate-ui-wikipedias-visual-editor/for volunteer help translating the VisualEditor interface. Kiwix http://www.kiwix.org
*The Kiwix project is funded and executed by Wikimedia CHhttps://meta.wikimedia.org/wiki/Wikimedia_CH .* We are preparing the first release of a new Wikipedia ZIM creation solutionhttp://sourceforge.net/p/kiwix/other/ci/master/tree/mwhtmldumper/for August. We also have achieved a new release of Kiwix for Androidhttps://play.google.com/store/apps/details?id=org.kiwix.kiwixmobile; this new version includes a few bug fixes and new features. Beside the release of traditional Wikipedia ZIM files, we have also published two interesting ZIM files: one which includes 2,500 ebooks (EPUB & PDF) of French literaturehttp://download.kiwix.org/zim/other/bouquineux.zim.torrentand one with the new Wikipedia for Schools selectionhttp://download.kiwix.org/zim/wikipedia_en_for_schools.zim.torrent. The ZIM incremental update GSoC projecthttps://www.mediawiki.org/wiki/User:Kiran_mathew_1993/ZIM_incremental_updates_for_Kiwixprogresses well too: first working versions of zimpatch https://github.com/kiranmathewkoshy/zimpatch & zimdiffhttps://github.com/kiranmathewkoshy/zimdiffconsole tools are available, and integration with Kiwix has started. Kiwix developers will be available at Wikimaniahttp://wikimania2013.wikimedia.org, during the hacking days and at the WikimediaCH both during Wikimania itself. Wikidata https://meta.wikimedia.org/wiki/Wikidata
*The Wikidata project is funded and executed by Wikimedia Deutschlandhttps://meta.wikimedia.org/wiki/Wikimedia_Deutschland/en .* In July, we deployed Wikidata to all Wikivoyage sites in all languages, to manage their language links. We updated the continued Roadmap for Wikidata Development https://www.mediawiki.org/wiki/Roadmap#Wikidata_deployment. Coveralls.io support https://coveralls.io/r/wikimedia has been added to most of our components. Since the first deployment of Phase1 to Wikipedia, about 240 million interwikilinks (5GB text) have been removed from articles (2012http://simia.net/languagelinks/index.htmlvs 2013 http://simia.net/languagelinks/2013.html analysis). In other news, the AAAI Feigenbaum Prize for Watson was donated to the Wikimedia Foundation by IBM research to support work, especially on Wikidatahttps://blog.wikimedia.org/2013/07/16/ibm-research-watson-aaai-prize-wikimedia-foundation/ . Denny Vrandečić explainshttp://lists.wikimedia.org/pipermail/wikidata-l/2013-July/002518.htmlwhy Wikidata items are identified with a Q. Future The engineering management team continues to update the *Deploymentshttps://wikitech.wikimedia.org/wiki/Deployments * page weekly, providing up-to-date information on the upcoming deployments to Wikimedia sites, as well as the *engineering roadmaphttps://www.mediawiki.org/wiki/Roadmap *, listing ongoing and future Wikimedia engineering efforts. *Annual goals*https://www.mediawiki.org/wiki/Wikimedia_Engineering/2013-14_Goalsfor the 2013–2014 fiscal year are being drafted by some teams and have been finalized by others. * This article was written collaboratively by Wikimedia engineers and managers, and assembled by Sumana Harihareswara. See revision historyhttps://www.mediawiki.org/w/index.php?title=Wikimedia_engineering_report/2013/July&action=historyand associated status pages. A wiki versionhttps://www.mediawiki.org/wiki/Wikimedia_engineering_report/2013/Julyis also available. *