Hi,
The report covering Wikimedia engineering activities in June 2013 is now available.
Wiki version: https://www.mediawiki.org/wiki/Wikimedia_engineering_report/2013/June Blog version: https://blog.wikimedia.org/2013/07/12/engineering-june-2013-report/
We're also proposing a shorter, simpler and translatable version of this report that does not assume specialized technical knowledge: https://www.mediawiki.org/wiki/Wikimedia_engineering_report/2013/June/summar...
Below is the full HTML text of the report.
As always, feedback is appreciated on the usefulness of the report and its summary, and on how to improve them.
------------------------------------------------------------------
Major news in June include:
- The preparation for the activation of VisualEditorhttps://blog.wikimedia.org/2013/06/06/preparing-for-visualeditor-on-all-wikipedias/to most Wikipedia sites, and its debut on the English Wikipediahttps://blog.wikimedia.org/2013/07/01/visualeditor-beta-rollout/ ; - News around Language engineeringhttps://blog.wikimedia.org/2013/06/03/language-engineering-development-updates-and-events/, including the preparationhttps://blog.wikimedia.org/2013/06/06/universal-language-selector-coming-to-all-wikis/and activationhttps://blog.wikimedia.org/2013/07/01/universal-language-selector-uls-deployed-on-more-than-150-wikis/of the Universal Language Selectors on many wikis; - An explanation of how bugs are discovered and fixedhttps://blog.wikimedia.org/2013/06/04/volunteers-and-staffers-teach-learn-make-at-amsterdam-hackathon/ . - A retrospective on the Amsterdam hackathonhttps://blog.wikimedia.org/2013/06/04/volunteers-and-staffers-teach-learn-make-at-amsterdam-hackathon/ .
*Note: We're also providing a shorter, simpler and translatable version of this reporthttps://www.mediawiki.org/wiki/Wikimedia_engineering_report/2013/June/summarythat does not assume specialized technical knowledge. * Personnel Work with us https://wikimediafoundation.org/wiki/Work_with_us
Are you looking to work for Wikimedia? We have a lot of hiring coming up, and we really love talking to active community members about these roles.
- Software Engineer - Fundraisinghttp://hire.jobvite.com/Jobvite/Job.aspx?j=oawpXfwM - Software Engineer - Language Engineeringhttp://hire.jobvite.com/Jobvite/Job.aspx?j=oH3gXfwH - Director of Program - Mobilehttp://hire.jobvite.com/Jobvite/Job.aspx?j=oGqAXfwn - Software Engineer - Multimedia Systemshttp://hire.jobvite.com/Jobvite/Job.aspx?j=oj40Wfw3 - Senior Software Engineer - Platformhttp://hire.jobvite.com/Jobvite/Job.aspx?j=ouLnWfwi - UX Designer http://hire.jobvite.com/Jobvite/Job.aspx?j=onImXfw8 - Product Manager - Platformhttp://hire.jobvite.com/Jobvite/Job.aspx?j=o3vtXfwI - Dev-Ops Engineer - SREhttp://hire.jobvite.com/Jobvite/Job.aspx?j=ocLCWfwf
Announcements
- Sean Pringle joined the Technical Operations team as our Storage and Database Engineer (announcementhttp://www.gossamer-threads.com/lists/wiki/wikitech/368588 ). - Brian Wolff https://www.mediawiki.org/wiki/User:Bawolff joined the Wikimedia Platform Engineeringhttps://www.mediawiki.org/wiki/Wikimedia_Platform_Engineeringgroup as Software developer for the Summer, working on multimedia contribution and review (announcementhttp://lists.wikimedia.org/pipermail/wikitech-l/2013-June/069724.html ). - Ken Snider joined the Technical Operations team as an international contractor, poised to fill the Director of Technical Operations position ( announcementhttp://lists.wikimedia.org/pipermail/wikitech-l/2013-June/069844.html ). - Toby Negrin joined the Engineering department as Director of Analytics (announcementhttp://lists.wikimedia.org/pipermail/wikimediaannounce-l/2013-June/000675.html ).
Technical Operations
*Site infrastructure* As part of our capacity planning work, Mark Bergsma upgraded most of our Varnish infrastructure (in EQIAD & ESAMS) with newer and faster servers. He will be adding new mobile Varnish servers in ESAMS next, this coming month. Rob Halsell and Daniel Zahn are pushing ahead with the migration of the other applications https://wikitech.wikimedia.org/wiki/Tampa_cluster from Tampa to EQIAD. New Parsoid application and Varnish servers were also deployed in anticipation of the coming VisualEditor deployment. Meantime, Alexandros Kosiaris is starting the backup project work; read more about the project https://wikitech.wikimedia.org/wiki/Projects#Backup_infrastructureand the technology https://wikitech.wikimedia.org/wiki/Bacula. Mark also put in the finishing touches to deploy all the new network infrastructure at ESAMS. With help from Mark and Leslie Carr, we finally got approval from ARIN for some new IPv4 addresses, needed for our new ULSFO buildup.Many people are refactoring Puppet code with the ultimate goal of having everything organized into Puppet modules. Andrew Bogott, Antoine Musso and Alexandros are setting up an automated testing infrastructure to support these efforts.
*Data Dumps https://www.mediawiki.org/wiki/WMF_Projects/Data_Dumps* Our GSoC student, Petr Onderka, is set up in gerrit and committed his first contributions to the Incremental Dumps project; you can follow his codehttps://git.wikimedia.org/summary/operations%2Fdumps%2Fincremental, read his progress reportshttps://www.mediawiki.org/wiki/User:Svick/Incremental_dumps#Updatesand check the current discussionhttp://lists.wikimedia.org/pipermail/xmldatadumps-l/2013-July/000834.htmlon the mailing list. Additionally, we hold IRC meetings on weekdays at about 4:15 pm (UTC) in #wikimedia-tech; lurkers and contributors are welcome.
*Wikimedia Labs https://www.mediawiki.org/wiki/Wikimedia_Labs* Wikimedia Labs saw a lot of improvements in June, including the deployment of AJAX improvements for OpenStackManager to wikitech (added actions: console output; improvements: reboot), and a new interface for displaying quotas for projects in OpenStackManager. We ensured that all instances were properly running Puppet and Salt; Many instances were running puppetmaster::self and needed to have local puppet repo merges or rebases. We upgraded Salt everywhere and re-issued keys to fix a vulnerability in Salt. The team also worked on stabilizing the NFS server. We've encountered a kernel bug with NFS; we have changed the scheduler from cfq to deadline and have decreased the read and write sizes of clients to 8k. Progress has been made towards making the Labs database replicas available to the Labs at large (as opposed to only the Tool Labs project). Last, much work has been done towards user request fulfillment in Tool Labs, including work towards WSGI support. Features Engineeringhttps://www.mediawiki.org/wiki/Wikimedia_Features_engineering Editor retention: Editing tools
*VisualEditor https://www.mediawiki.org/wiki/VisualEditor* In June, the VisualEditor team completed the major new features that we prioritised over the past few months, in preparation for making VisualEditor available to most Wikipedia users in July. We have built an editor that is capable of letting users edit the majority of content without needing to use wikitext — text support, as well as adding and editing inclusions of references, templates, categories and media items. The deployed alpha of VisualEditor was updated four times as part of the transition to weekly deployments (1.22-wmf6https://www.mediawiki.org/wiki/MediaWiki_1.22/wmf6#VisualEditor, 1.22-wmf7 https://www.mediawiki.org/wiki/MediaWiki_1.22/wmf7#VisualEditor, 1.22-wmf8 https://www.mediawiki.org/wiki/MediaWiki_1.22/wmf8#VisualEditorand 1.22-wmf9 https://www.mediawiki.org/wiki/MediaWiki_1.22/wmf9#VisualEditor), with several mid-deployment releases as the code was developed to patch urgent issues. Part of this involved running an A/B test for new user accounts on the English Wikipedia, with half of the users getting opt-in to VisualEditor ahead of the wider release. Generally, there were a number of user interface improvements, and fixing a number of bugs uncovered by the community.
*Parsoid https://www.mediawiki.org/wiki/Parsoid* Early this month, we deployed Parsoid to the new cluster and started to track all edits and template / image updates from all Wikipedia sites, which is close to the full load we'll see when VE is deployed to all of them. Our earlier optimization work paid off as the Parsoid cluster and the associated Varnish caches are handling the load very well. The extra load we put on the API cluster is low enough to not cause a problem. As expected, the VisualEditor deployment to the English Wikipedia hardly showed up in the load graphs.
Despite being very short-staffed this month (only two full-time developers), the absence of performance issues left us enough time for a lot more polishing before the VisualEditor release on July 1. As a result, the release went very well with clean diffs on almost all pages. While more work is left to do, it is now clear that we have fundamentally achieved our goal of a clean translation between WikiText and HTML + RDFa. This does not only enable visual HTML editing, but also makes Wikipedia's content easily accessible in a standardized format. It also opens up new opportunities for MediaWiki's core architecture, which we'll pursue this fiscal year. Editor engagement features
*Notifications https://www.mediawiki.org/wiki/Echo_%28Notifications%29* In June, we released more features and bug fixes for Notifications on the English Wikipedia and mediawiki.org. Ryan Kaldari added a confirmation button for the 'Thanks featurehttps://en.wikipedia.org/wiki/Wikipedia:Notifications/Thanks', and updated notification fly-outs to show diff linkshttps://bugzilla.wikimedia.org/show_bug.cgi?id=48183for talk page and interactive notifications, based on a design by Vibha Bamba. Benny Situ continued development of HTML Email notificationshttps://www.mediawiki.org/wiki/Echo/Feature_requirements#HTML_single_email_notificationsand deployed a variety of feature updates. Erik Bernhardson developed a special 'Suppressed' content feature, while Matthias Mullie developed a range of new metrics dashboardshttp://ee-dashboard.wmflabs.org/dashboards/enwiki-features. Dario Taraborelli and Aaron Halfaker ran a week-long A/B test of new user activityhttps://meta.wikimedia.org/wiki/Research:Notifications/Experiment_1/Results; results show that new users who received Echo notifications made more edits than those who did not, but their edits were reverted slightly more often. Fabrice Florin led the planning process for Notifications, as outlined in the 2013 roadmaphttps://www.mediawiki.org/wiki/Editor_Engagement/2013_strategy_planning_%28Features%29, and hosted a day-long roundtable discussionhttps://meta.wikimedia.org/wiki/Roundtables/Roundtable_1to improve editor engagement features in collaboration with Wikipedia users (see Echo demo and Q&A video on YouTubehttp://www.youtube.com/watch?v=tyJANeVnbsQ). Later this summer, we plan to start deploying Notifications on more wiki projects, starting with Meta and the French Wikipedia. To learn more, visit the project portal https://en.wikipedia.org/wiki/Wikipedia:Notifications, read the FAQ pagehttps://en.wikipedia.org/wiki/Wikipedia:Notifications/FAQand join the discussion on the talk page https://en.wikipedia.org/wiki/Wikipedia_talk:Notifications.
*Article feedback https://www.mediawiki.org/wiki/Article_feedback* In June, we deployed final features and bug fixes for the Article Feedback Tool https://www.mediawiki.org/wiki/Article_feedback/Version_5 (AFT5) on the English https://en.wikipedia.org/wiki/Special:ArticleFeedbackv5, French https://en.wikipedia.org/wiki/fr:Sp%C3%A9cial:ArticleFeedbackv5and Germanhttps://en.wikipedia.org/wiki/de:Spezial:Artikelr%C3%BCckmeldungen_v5Wikipedias. Matthias Mullie released an opt-in featurehttps://www.mediawiki.org/wiki/Article_feedback/Version_5/Feature_Requirements#Enable.2Fdisable_feedback_on_a_pageto enable or disable feedback on a page, based on designs by Pau Giner and specifications by Fabrice Florin. In collaboration with Dario Taraborelli, Matthias also developed an updated set of metrics dashboardshttp://ee-dashboard.wmflabs.org/dashboards/enwiki-features#article_feedback_moderation-graphs-tabshowing how the new moderation tools are being used: for example, about half of moderated feedback is marked as 'no action needed', while about a tenth is marked as 'useful' (these results are generally consistent across different languageshttps://docs.google.com/a/wikimedia.org/spreadsheet/ccc?key=0Aq_75_5y5sKWdDl0blpSbGRiS2ppRzlaaHZiV1dRMXc#gid=3). The team also supported a wider deployment of AFT5 on over 40,000 articles on the French Wikipedia, as well as a poll by the German community, which elected not to adopt the tool. Now that feature development has ended for this project, we plan to make AFT5 available to other wiki projects in coming weeks, as outlined in the release planhttps://www.mediawiki.org/wiki/Article_feedback/Version_5/Release_Plan_2013. For tips on how to use Article feedback, visit the testing pagehttps://www.mediawiki.org/wiki/Article_feedback/Version_5/Testing, and let us know what you think on this talk pagehttps://en.wikipedia.org/wiki/Wikipedia_talk:Article_Feedback_Tool/Version_5 . Editor engagement experiments
*Editor engagement experimenthttps://www.mediawiki.org/wiki/Editor_engagement_experiments * In June, the Editor Engagement Experiments (E3) team continued work on its experiments related to onboarding new Wikipedianshttps://www.mediawiki.org/wiki/Onboarding_new_Wikipedians, and launched several new extensions to Wikimedia projects.
First, the new Campaigns extensionhttps://www.mediawiki.org/wiki/Extension:Campaignswas added to all wikis. This analytics tool helps identify internal or external sources of new registrations, by adding a "campaign" name to the signup page URL. This month, E3 began running campaigns to learn about how many anonymous editors sign up on the top 10 Wikipedias, as well as how many sign up via the invitation to "Join Wikipedia" on the login page (see the list of active campaignshttps://meta.wikimedia.org/wiki/Research:Account_creation_campaignsand analysis). Another piece of analytics infrastructure by the team is the new CoreEvents extensionhttps://www.mediawiki.org/wiki/Extension:CoreEvents, which houses logging of MediaWiki core activity, like preference updates and page saves across all projects.
For the Getting Startedhttps://www.mediawiki.org/wiki/Extension:GettingStartedproject, the team conducted usability testing (see results and documentationhttps://www.mediawiki.org/wiki/Onboarding_new_Wikipedians/user_testing#Testing_scenario_three) of new designs. E3 also refactored and refined the guided tourshttps://www.mediawiki.org/wiki/Guided_toursextension in June, including adding usability enhancements like new interface animations, support for community tours, and bug fixing. The team also planned and began work on an experiment to deliver guided tourshttps://meta.wikimedia.org/wiki/Research:Guided_toursto all first-time editors. The team also assisted with A/B testing and research for VisualEditorhttps://www.mediawiki.org/wiki/VisualEditorbefore its July 1 launch date, assisting with experimental design, EventLogging instrumentation, and other work. After the VisualEditor launch, E3 started a week-long micro-survey of newly-registered usershttps://meta.wikimedia.org/wiki/Research:Gender_micro-surveyon English Wikipedia, to give us a first systematic look at the gender diversity of those creating accounts. Support
*2012 Wikimedia fundraiserhttps://www.mediawiki.org/wiki/2012_Wikimedia_fundraiser * The initial work on the Adyen payments gateway was finally completed and deployed to production, though we have not yet used the gateway in a campaign. Plans for a mobile fundraising campaign and workflow continued to move forward: We expect to do the first mobile-targeted campaign in mid to late July. Some last-minute tweaking was done to the payments cluster in preparation for the resumption of continuous fundraising on July 1, coinciding with the start of the fiscal year. Payments listener (thulium) deploy was completed, db1013 was moved into the firewalled fundraising cluster and rebuilt as a fundraising QA server, and work continued on the new CiviCRM server (barium). Fundraising backups were overhauled. Mobile https://www.mediawiki.org/wiki/Wikimedia_Mobile_engineering
*Wikipedia Zero https://www.mediawiki.org/wiki/Wikipedia_Zero* This month, the team launched Wikipedia Zero with Dialog in Sri Lanka, patched logic and user interface bugs, enhanced the configuration editor, expanded logging and debugging for identification of anomalous access, further decoupled ZeroRatedMobileAccess from MobileFrontend, and proposed ESI- and JavaScript-based software re-architecture.
*Mobile Web Photo Uploadhttps://www.mediawiki.org/wiki/Mobile_design/Uploads * This month, we focused on improving education around uploads, including an interactive Commons tutorial and first-time user copyright and scope check. We also released our "Nearby" feature to production, allowing users to find articles near them that are in need of images, take photos and upload them via mobile.
*Mobile Navhttps://www.mediawiki.org/wiki/Mobile_design/Wikipedia_navigation * In beta, we started working on an update to our site and article navigation, including design tweaks to the left navigation menu and a new in-article contributory navigation that combines article actions (edit, upload, and watch) with a talk page link. We also experimented with Echo integration and successfully got Notifications up and running on the English Wikipedia mobile site. We hope to push all of this work to production next month. Platform Engineeringhttps://www.mediawiki.org/wiki/Wikimedia_Platform_Engineering MediaWiki Core
*MediaWiki 1.22 https://www.mediawiki.org/wiki/MediaWiki_1.22/Roadmap* In June, the Platform Engineering group switched to a weekly deployment cycle https://wikitech.wikimedia.org/wiki/Deployments/One_week for MediaWiki to the Wikimedia Foundation servers. This means that we have almost halved our previous cycle of 2 weeks. As such, we are progressing through wmfXX versions of MediaWiki at a faster rate now. In June, MediaWiki versions 1.22-wmf6 through wmf9 were branched and deployed.
*Git conversion https://www.mediawiki.org/wiki/Git/Conversion* Chad Horohoe and Christian Aistleitner upgraded our Gerrit instance from a pre-release version of 2.6 to a pre-release version of 2.7 on the last week of June. They've additionally published a new version of the Bugzilla/Gerrit integration plugin. Details about new functionality can be found in the Gerrit 2.7 draft release noteshttp://gerrit-documentation.googlecode.com/svn/ReleaseNotes/ReleaseNotes-2.7.html .
*Multimedia https://www.mediawiki.org/wiki/Multimedia* In June, we started expanding our multimedia team: Fabrice Florin joined as product manager, and Brian Wolff began a summer contract as software engineer. We started work on improving the display of images in galleries and are now planning our next development steps in consultation with community members. Some of the first features under consideration include file curation and feedback tools, as well as media viewers, new video formats and other platform improvements, to be prioritized based on user feedback and technical feasibility. We are also recruiting for two more positions: a multimedia systems engineerhttp://hire.jobvite.com/Jobvite/Job.aspx?j=oj40Wfw3&c=qSa9VfwQand a senior software engineerhttp://hire.jobvite.com/Jobvite/Job.aspx?j=ouLnWfwi&c=qSa9VfwQ. Please spread the word about this unique opportunity to create a richer multimedia experience for Wikipedia and MediaWiki sites!
*Admin tools developmenthttps://www.mediawiki.org/wiki/Admin_tools_development * In June, the team worked on making the last changes to enable global AbuseFilter rules, and on the global account renaming tool. Some additional work was done on Single User Login finalisationhttps://www.mediawiki.org/wiki/Admin_tools_development/SUL_Audit, which will mean that all user accounts will be global across all of Wikimedia's public wikis, and so allowing for cross-wiki notifications and better tools for editors.
*Search https://www.mediawiki.org/wiki/Search* Work has pretty much shifted from supporting MWSearch/lsearchd to investigating and implementing Solr. Nik Everett and Chad Horohoe have begun writing an extensionhttps://www.mediawiki.org/wiki/Extension:CirrusSearchto implement Solr searching for MediaWiki, and we've gotten a lot of the initial basic functionality completed. Peter Youngmeister and Andrew Bogott will be handling the operations tasks for the new setup. Initial operations tasks will involve packaging Solr 4 and working with Chad to puppetize the whole design. Additionally, we're going to do some investigation into ElasticSearch, as it's been suggested as an alternative to Solr.
*Auth systems https://www.mediawiki.org/wiki/Auth_systems* In June, the team worked with the Wikimedia Foundation's user experience team to improve SUL2. The improvements were pushed to test wikis on July 1, and will be rolled out to other wikis in July. Implementation of OAuth is well underway, and planned for roll-out in July as well.
*HipHop deployment https://www.mediawiki.org/wiki/HipHop_deployment* A Labs instance of MediaWiki running on HipHop is now available at http://hhvm.wmflabs.org.
*Security auditing and responsehttps://www.mediawiki.org/wiki/Security_auditing_and_response * The team continued to respond to reported security issues, and gave security-oriented tech talks on emerging DoS techniques and using OWASP's ZAP tool for vulnerability scanning. Quality assurance
*Quality Assurance https://www.mediawiki.org/wiki/Quality_Assurance* This month saw a QA focus on automated browser tests. Besides creating new tests and new builds, and reporting issues identified by tests, we conducted a training session in San Francisco to create automated tests for the Wikilove feature. We continue to support all WMF software development projects, with the VisualEditor being a particular focus in June.
*Beta cluster https://www.mediawiki.org/wiki/Beta_cluster* Max Semenik wrote a script to synchronize CSS from production on beta. Steinsplitter and Antoine Musso fixed the AbuseFilter configuration to have a global list of filters on the labswikihttp://deployment.wikimedia.beta.wmflabs.org/wiki/Special:AbuseFilter. Filters should be configured there and will be used by all the wikis. The PHP fatal errors catched by the wmerrors extension are now sent to the beta udp2log instance. That will largely improve our troubleshooting process.
*Continuous integrationhttps://www.mediawiki.org/wiki/Continuous_integration * Timo Tijhof and Antoine Musso triaged continuous integration bugs. Antoine has setup a Jenkins slave and migrated most jobs on it. It will be very easy to add new servers.
*Browser testinghttps://www.mediawiki.org/wiki/Quality_Assurance/Browser_testing * This month, the QA team added new browser tests for UniveralLanguageSelector and for Mobile (contributed by the Language engineering and Mobile engineering teams, respectively), as well as browser test contributions from volunteers. We created new builds in Jenkins to run browser tests against IE10. We created tests for VisualEditor, including some with our intern with the Outreach Program for Women. Analytics
*Analytics infrastructurehttps://www.mediawiki.org/wiki/Analytics/Infrastructure * We made significant progress with our preparations for replacing udp2log with Kafka in our logging infrastructure. The C library librdkafkahttps://github.com/edenhill/librdkafka/tree/0.8-wiphas now support for the 0.8 protocol, there is a first version of varnishkafka https://gerrit.wikimedia.org/r/#/c/70928/ ready that will replace varnishncsa, the Apache Kafka project released their first beta of Kafka 0.8, and we have a Debianizedhttps://gerrit.wikimedia.org/r/#/c/68026/and Pupppetized https://gerrit.wikimedia.org/r/#/c/50385/ version. We keep on adding new metrics https://gerrit.wikimedia.org/r/#/c/71089/ and alerts to monitor all the different parts of the webrequest dataflows into Kraken. We expect to keep making improvements in the coming months, until we have a fully reliable data pipeline into Kraken. We also continued our efforts of moving Kraken out of beta: we puppetized Zookeeperhttps://gerrit.wikimedia.org/r/#/c/66882/, JMXtrans https://gerrit.wikimedia.org/r/#/c/70915/, and the Hadoop client nodes for Hive, Pig and Sqoop https://gerrit.wikimedia.org/r/#/c/69521/. We started reinstalling the Hadoop Datanode workers with a fully puppetized Hadoop installation; so far, we have replaced 3 nodes, and we'll replace the other seven in the coming weeks. Last, we enabled Jenkins continuous integration for the Grantmaking & Evaluation dashboards.
*Analytics Visualization, Reporting & Applicationshttps://www.mediawiki.org/wiki/Analytics/Visualization,_Reporting_%26_Applications * This month, we completed the end-user documentationhttps://www.mediawiki.org/wiki/UserMetrics/Guideof UserMetrics (v1). We rebranded UserMetrics as Wikimetrics, and we will slowly start to use that as the new name when referring to UserMetrics v2 or UserMetrics replatforming. We focused on laying out the foundation of Wikimetrics: a new database design, a new job queue design and lots of unit tests. In addition, we started working on porting over some of the features of UserMetrics v1 (like the 'namespace edits' metric and UI components), we added user roles (so users can only see their own metrics) and authentication using OAuth. Last, we fixed some minor issues in UserMetrics v1, among which handling of user names with comma, single and double quotes.
*Data Releases https://www.mediawiki.org/wiki/Analytics/Data_Releases* We delivered many following analyses in June, including one of Arabic cohort using UMAPI v1. Erik Zachte provided an analysis of Commons uploaders, and we provided the Wikipedia Zero team with a number of datasets to help them in tracking adoption of the Wikipedia Zero project across the globe. We supported the VisualEditor and Editor Engagement teams with experimental design, data modeling and data analysis for two controlled experiments: a test of the impact of impact of notificationshttps://meta.wikimedia.org/wiki/wiki/Research:Notifications/Experiment_1and a first test https://meta.wikimedia.org/wiki/Research:Visual_Editor of the impact of Visual Editor on new contributors. The tests were carried out in June and the reports are being updated with the results of the analysis. We started using the EE-dashboard instance on Labs to host dashboards related to editor engagement projects, that were previously hosted on the Toolserver (see the metricshttp://ee-dashboard.wmflabs.org/dashboards/enwiki-metricsand features http://ee-dashboard.wmflabs.org/dashboards/enwiki-featuresdashboards for the English Wikipedia). Last, we worked with the Features engineering team to expand MediaWiki's instrumentation and collect data on cluster-wide user preference changeshttps://meta.wikimedia.org/wiki/Research:User_preferencesand edit-related events https://meta.wikimedia.org/wiki/Schema:Edit to support VisualEditor analysis. Engineering community team
*Bug management https://www.mediawiki.org/wiki/Bug_management* Andre Klapper https://www.mediawiki.org/wiki/User:AKlapper_%28WMF%29published the Bugzilla administrator policyhttps://www.mediawiki.org/wiki/User:AKlapper_%28WMF%29/BugzillaAdminPolicyand documented for which specific taskshttp://blogs.gnome.org/aklapper/2013/05/28/understanding-bugzilla-groups-and-admin-rights/Bugzilla admin rights are actually needed (which might be also helpful for other projects using Bugzilla). He started publishing weekly "Bugzilla tips and best practices"http://blogs.gnome.org/aklapper/category/computer/bugzilla/blog posts and reproposed introducing a "PATCH AVAILABLE" statushttp://lists.wikimedia.org/pipermail/wikitech-l/2013-June/069805.htmlin Bugzilla (as requested by several developers at the Amsterdam Hackathon) whilst work is ongoing to fulfill prerequisites. On the code side of Bugzilla, a new Bugzilla frontpage https://bugzilla.wikimedia.org went live, providing useful links. Furthermore, the misleading term "login" was replaced by "email address" https://bugzilla.wikimedia.org/show_bug.cgi?id=24992, it is now possible to set the "Assigned" status directlyhttps://bugzilla.wikimedia.org/show_bug.cgi?id=49461when filing a new bug report, and smaller issues with the "Weekly Bugzilla Report" email sent to the wikitech-l mailing list were fixed. In Bugzilla's taxonomy, open tickets in the dormant "Wiktionary tools" product were retriaged and the product closed for new bug entry, and Security-related components in Bugzilla were reorganized after a meeting with the Wikimedia Foundation's security engineer.
*Mentorship programs https://www.mediawiki.org/wiki/Mentorship_programs* The 20 Google Summer of Codehttps://www.mediawiki.org/wiki/Summer_of_Code_2013and the 1 Outreach Program for Womenhttps://www.mediawiki.org/wiki/Outreach_Program_for_Women/Round_6interns have completed the bonding periodhttps://www.mediawiki.org/wiki/Summer_of_Code_2013#Community_bonding_period(with 3 exceptions, 2 of them justified) and they are now working on their projects. One OPW accepted candidate declined her participation due to a job offer. Monthly status updates are available for these projects:
- Improving support for book structureshttps://meta.wikimedia.org/wiki/Book_management/Progress#June - Mobilize Wikidatahttps://www.mediawiki.org/wiki/User:Pragunbhutani/GSoC_2013_Updates#Monthly_Report:_June_2013 - jQuery.IME extensions for Firefox and Chromehttps://www.mediawiki.org/wiki/User:Prageck/GSoC_2013_Application#Monthly_Report:_June_2013 - Browser Test Automation for Visual Editorhttps://www.mediawiki.org/wiki/User:Rachel99/proposal1#June - Incremental dumpshttps://www.mediawiki.org/wiki/User:Svick/Incremental_dumps#June_report - Refactoring of Extension ProofreadPagehttps://www.mediawiki.org/wiki/Extension:Proofread_Page/GSoC#June - Language Coverage Matrix Dashboardhttps://www.mediawiki.org/wiki/User:Harsh4101991/GSoC_2013#June - Section Handling in Semantic Formshttps://www.mediawiki.org/wiki/User:Himeshi/GSoC_2013/Project - VisualEditor Mathematical Editor Pluginhttps://www.mediawiki.org/wiki/User:Jiabao_wu/GSoC_2013_Project_Work#June - ZIM Incremental Updates for Kiwixhttps://www.mediawiki.org/wiki/User:Kiran_mathew_1993/ZIM_incremental_updates_for_Kiwix#June_Report - VisualEditor RTL supporthttps://www.mediawiki.org/wiki/User:Mooeypoo/GSOC_2013_Proposal:_RTL_Support_in_VisualEditor#Monthly_Report:_June_2013 - UploadWizard: book upload customizationhttps://www.mediawiki.org/wiki/User:Rasel160/GSoC2013/Status - Android app for MediaWii translationhttps://www.mediawiki.org/wiki/User:Orsagi/GSoC_2013_proposal#Monthly_Report:_June_2013 - Pronunciation recording toolhttps://www.mediawiki.org/wiki/User:Rahul21/Gsoc2013/Project_Updates - MediaWiki Moddle extensionhttps://www.mediawiki.org/wiki/User:Clancer/GSoC_2013_Project - VisualEditor plugin for source codehttps://www.mediawiki.org/wiki/User:Beanixster#June
We also met with SocialCoding4Good http://www.socialcoding4good.org/, who are relaunching their activities, and we refreshed the Wikimedia pagehttp://www.socialcoding4good.org/organization/wikimedia. We expect this to become a regular channel for new technical contributors working in corporations with social/training programs.
*Technical communicationshttps://www.mediawiki.org/wiki/Technical_communications * In June, work on this topic mostly focused on perennial activities like Tech news https://meta.wikimedia.org/wiki/Tech/News and ongoing communications support to engineering staff, as Guillaume Paumierhttps://www.mediawiki.org/wiki/User:Guillomwas lent to the VisualEditor deployment effort, working on communications, documentation and liaising with the French Wikipedia.
*Volunteer coordination and outreachhttps://www.mediawiki.org/wiki/Volunteer_coordination_and_outreach * The decision of focusing on fewer activities better executed and based on demand seems to be working out, although it's too soon to confirm the trend. Browser test automation is the number one priority to recruit new contributors, and any help to succeed here is welcome. We created the QA mailing list https://lists.wikimedia.org/mailman/listinfo/qa as an umbrella to host people and discussions focusing on software quality assurance in all its aspects. We have more than 40 subscribers and an initial flow of activity. We had a successful first Browser Test Automation Workshop http://www.meetup.com/wikimedia-tech/events/124334432/, with 40 participants in San Francisco and a few more online; we will iterate on this model. We have also helped organizing a Tech Talk on Attack vectors & MediaWiki and OWASP ZAPhttps://www.mediawiki.org/wiki/Meetings/2013-06-27-midday, and the upcoming Solr-based Searchhttps://www.mediawiki.org/wiki/Meetings/2013-07-03. The project to get automated community metricshttps://www.mediawiki.org/wiki/Community_metricsbased on vizGrimoire http://vizgrimoire.bitergia.org/ and provided by Bitergiahttp://bitergia.com/has been approved, and a first prototype can be seen at http://korma.wmflabs.org. The project starts effectively on July 1 and includes a one-year period of maintenance. We agreed with the Analyticshttps://www.mediawiki.org/wiki/Analyticsteam that they will assume the responsibility of this area during this period. Kiwix http://www.kiwix.org
*The Kiwix project is funded and executed by Wikimedia CHhttps://meta.wikimedia.org/wiki/Wikimedia_CH .* Development of a new MediaWiki HTML dumper in nodeJShttp://sourceforge.net/p/kiwix/other/ci/master/tree/mwhtmldumper/has started. This tool exports Wikipedia articles in static files based on the Parsoid http://www.mediawiki.org/wiki/Parsoid output. This solution looks really promising, and new JavaScript developers are welcome. Wikidatahttps://meta.wikimedia.org/wiki/Wikidata
*The Wikidata project is funded and executed by Wikimedia Deutschlandhttps://meta.wikimedia.org/wiki/Wikimedia_Deutschland/en .* June in Wikidata was all about the sister projects. The development team published proposals for how Wikidata can support Commonshttps://commons.wikimedia.org/wiki/Commons:Wikidata_for_media_infoand Wiktionaryhttp://lists.wikimedia.org/pipermail/wikidata-l/2013-June/002380.html. Additionally, they worked on the ability of Wikidata to store language links to Wikivoyage in addition to Wikipedia; as a result, Wikivoyage will soon also be able to manage their language links via Wikidata. Another important step was the deployment of the geocoordinate datatype. This makes it possible, for example, to indicate the location of a city. Geocoordinates that are already in Wikidata can be seen on this maphttps://dl.dropboxusercontent.com/u/172199972/map.png(huge version https://dl.dropboxusercontent.com/u/172199972/map_huge.png, updated daily). In a blog entryhttp://blog.wikimedia.de/2013/06/04/on-truths-and-lies/, Denny Vrandečić explained his understanding of the relation of Wikidata and the truth. In other news, further development of Wikidata has been supported through a large donation by the search engine company Yandexhttp://blog.wikimedia.de/2013/06/05/grosspende-fur-wikidata/ . Future The engineering management team continues to update the * Deployments https://wikitech.wikimedia.org/wiki/Deployments* page weekly, providing up-to-date information on the upcoming deployments to Wikimedia sites, as well as the *engineering roadmaphttps://www.mediawiki.org/wiki/Roadmap *, listing ongoing and future Wikimedia engineering efforts. *Annual goals*https://www.mediawiki.org/wiki/Wikimedia_Engineering/2013-14_Goalsfor the 2013–2014 fiscal year are currently being drafted. -- This report was reviewed and proofread using VisualEditor.
-- Guillaume Paumier Technical Communications Manager — Wikimedia Foundation https://donate.wikimedia.org