Wikitech-l July 2014

wikitech-l@lists.wikimedia.org

144 participants
151 discussions

Wikimedia engineering report, June 2014
by Guillaume Paumier 15 Jul '14

15 Jul '14

Hi, The report covering Wikimedia engineering activities in June 2014 is now available. Wiki version: https://www.mediawiki.org/wiki/Wikimedia_Engineering/Report/2014/June Blog version: https://blog.wikimedia.org/2014/07/15/engineering-report-june-2014/ We're also proposing a shorter, simpler and translatable version of this report that does not assume specialized technical knowledge: https://www.mediawiki.org/wiki/Wikimedia_Engineering/Report/2014/June/summa… Below is the HTML text of the report. As always, feedback is appreciated on the usefulness of the report and its summary, and on how to improve them. ------------------------------------------------------------------ Major news in June include: - the release of the new Wikipedia for Android app <https://blog.wikimedia.org/2014/06/25/revamped-wikipedia-app-now-available-…>, preceded by its beta version <https://blog.wikimedia.org/2014/06/05/new-wikipedia-android-app-beta/>; - the decision to move away from Bugzilla in favor of Phabricator <https://blog.wikimedia.org/2014/06/10/on-our-way-to-phabricator/>; - A new tablet view for Wikimedia sites <https://blog.wikimedia.org/2014/06/17/wikimedia-sites-get-a-new-look-on-tab…> . *Note: We’re also providing a shorter, simpler and translatable version of this report <https://www.mediawiki.org/wiki/Wikimedia_Engineering/Report/2014/June/summa…> that does not assume specialized technical knowledge.* Engineering metrics in June: - 151 unique committers contributed patchsets of code to MediaWiki. - The total number of unresolved commits <https://gerrit.wikimedia.org/r/#q,status:open+project:%255Emediawiki.*,n,z> went from around 1440 to about 1575. - About 14 shell requests <https://www.mediawiki.org/wiki/Shell_requests> were processed. Contents - Upcoming events <https://blog.wikimedia.org/2014/07/15/engineering-report-june-2014/#Upcomin…> - Personnel <https://blog.wikimedia.org/2014/07/15/engineering-report-june-2014/#Personn…> - Work with us <https://blog.wikimedia.org/2014/07/15/engineering-report-june-2014/#Work_wi…> - Announcements <https://blog.wikimedia.org/2014/07/15/engineering-report-june-2014/#Announc…> - Technical Operations <https://blog.wikimedia.org/2014/07/15/engineering-report-june-2014/#Technic…> - Features Engineering <https://blog.wikimedia.org/2014/07/15/engineering-report-june-2014/#Feature…> - Editor retention: Editing tools <https://blog.wikimedia.org/2014/07/15/engineering-report-june-2014/#Editor_…> - Core Features <https://blog.wikimedia.org/2014/07/15/engineering-report-june-2014/#Core_Fe…> - Growth <https://blog.wikimedia.org/2014/07/15/engineering-report-june-2014/#Growth> - Support <https://blog.wikimedia.org/2014/07/15/engineering-report-june-2014/#Support> - Mobile <https://blog.wikimedia.org/2014/07/15/engineering-report-june-2014/#Mobile> - Language Engineering <https://blog.wikimedia.org/2014/07/15/engineering-report-june-2014/#Languag…> - Platform Engineering <https://blog.wikimedia.org/2014/07/15/engineering-report-june-2014/#Platfor…> - MediaWiki Core <https://blog.wikimedia.org/2014/07/15/engineering-report-june-2014/#MediaWi…> - Quality assurance <https://blog.wikimedia.org/2014/07/15/engineering-report-june-2014/#Quality…> - Multimedia <https://blog.wikimedia.org/2014/07/15/engineering-report-june-2014/#Multime…> - Engineering Community Team <https://blog.wikimedia.org/2014/07/15/engineering-report-june-2014/#Enginee…> - Analytics <https://blog.wikimedia.org/2014/07/15/engineering-report-june-2014/#Analyti…> - Wikidata <https://blog.wikimedia.org/2014/07/15/engineering-report-june-2014/#Wikidata> - Future <https://blog.wikimedia.org/2014/07/15/engineering-report-june-2014/#Future> Personnel Work with us <https://wikimediafoundation.org/wiki/Work_with_us> Are you looking to work for Wikimedia? We have a lot of hiring coming up, and we really love talking to active community members about these roles. - VP of Engineering <http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&pag…> - Software Engineer – Front-end (VisualEditor) <http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&pag…> - Software Engineer – Services <http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&pag…> - Software Engineer – Internationalization <http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&pag…> - Software Engineer – Front-end <http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&pag…> - Software Engineer – Mobile – iOS <http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&pag…> - QA Tester <http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&pag…> - Software Engineer – Full Stack <http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&pag…> - Product Manager <http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&pag…> - Product Manager – Language Engineering <http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&pag…> - Operations Security Engineer <http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&pag…> - Project Coordinator – Engineering <http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&pag…> - Partner Manager – Wikipedia Zero (Latin America) <http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&pag…> Announcements - Elliot Eggleston joined the Wikimedia Foundation as a Features Engineer in the Fundraising-Tech team (announcement <http://lists.wikimedia.org/pipermail/wikitech-l/2014-June/077111.html>). Technical Operations *New Dallas data center* On-site work has started in our new Dallas (Carrollton) data-center (codfw). Racks have been installed, the equipment we moved from Tampa has been racked and cabling work has been mostly completed over the course of the month. We are now awaiting the installation of connectivity to the rest of our network as well as the arrival of the first newly-ordered server equipment, so server & network configuration can commence. *Puppet 3 migration* In July we migrated from Puppet 2 to Puppet 3 on all production servers. Thanks to the hard work of both volunteers and Operations staff on our Puppet repository in the months leading up to this, this migration went very smoothly. Labs metrics in June: - Number of projects: 173 - Number of instances: 424 - Amount of RAM in use (in MBs): 1,741,312 - Amount of allocated storage (in GBs): 19,045 - Number of virtual CPUs in use: 855 - Number of users: 3,356 *Wikimedia Labs* Last month we switched the Labs puppetmaster to Puppet 3; this month all instances switched over as well. Some cleanup work was needed in our puppet manifests to handle Trusty and Puppet 3 properly; everything is fairly stable now but a bit of mopping up remains. Features Engineering <https://www.mediawiki.org/wiki/Wikimedia_Features_engineering> Editor retention: Editing tools *VisualEditor <https://www.mediawiki.org/wiki/VisualEditor>* In June, the VisualEditor team provided a new way to see the context of links and other items when you edit to make this easier, worked on the performance and stability of the editor so that users could more swiftly and reliably make changes to articles, and made some improvements to features focussed on increasing their simplicity and understandability, fixing 94 bugs and tickets <https://bugzilla.wikimedia.org/buglist.cgi?list_id=326277&order=priority%2C…>. The editor now shows with a highlight where dragging-and-dropping content will put it, and works for any content, not just for images. The citation and reference tools had some minor adjustments to guide the user on how they operate, based on feedback and user testing. A lot of fixes to issues with windows opening and closing, and especially the link editing tool, were made, alongside the save dialog, categories, the language editing tool, table styling, template display and highlights on selected items. The mobile version of VisualEditor, currently available for alpha testers, moved towards release, fixing a number of bugs and improving performance. Work to support languages made some significant gains, and work to support Internet Explorer continued. The new visual interface for writing TemplateData was enabled on the Catalan and Hebrew Wikipedias. The deployed version of the code was updated five times in the regular release cycle ( 1.24-wmf8 <https://www.mediawiki.org/wiki/MediaWiki_1.24/wmf8#VisualEditor> , 1.24-wmf9 <https://www.mediawiki.org/wiki/MediaWiki_1.24/wmf9#VisualEditor>, 1.24-wmf10 <https://www.mediawiki.org/wiki/MediaWiki_1.24/wmf10#VisualEditor> and 1.24-wmf11 <https://www.mediawiki.org/wiki/MediaWiki_1.24/wmf11#VisualEditor>). *Parsoid <https://www.mediawiki.org/wiki/Parsoid>* In June, the Parsoid team continued with ongoing bug fixes and bi-weekly deployments; the selective serializer, improving our parsing support for some table-handling edge case, nowiki handling, and parsing performance are some of the areas that saw ongoing work. We began work on supporting language converter markup. We added CSS styling to the HTML to ensure that Parsoid HTML renders like PHP parser output. We continued to tweak the CSS based on rendering differences we found. We also started work on computing visual diffs based on taking screenshots of rendered output of Parsoid and PHP HTML. This initial proof-of-concept will serve as the basis of more larger scale automated testing and identification of rendering diffs. The GSoC 2014 LintTrap project saw good progress and a demo LintBridge application <http://lintbridge.wmflabs.org/_html/issues/> was made available on wmflabs with the wikitext issues detected by LintTrap. We also had our quarterly review this month and contributed to the annual engineering planning process. Core Features *Flow <https://www.mediawiki.org/wiki/Flow/Project_information>* <https://www.mediawiki.org/w/index.php?title=File:Flow_update_for_WMF_Metric…> <https://commons.wikimedia.org/wiki/File:Flow_update_for_WMF_Metrics_present…> Presentation slides on Flow from themetrics meeting for June <https://meta.wikimedia.org/wiki/WMF_Metrics_and_activities_meetings/2014-07> In June, the Flow team finished an architectural re-write for the front-end, so Flow will be easier to keep updating in the future. This will be released to mediawiki.org the first week of July, and Wikipedia the following week. The new feature in this release is the ability to sort topics on a Flow board. There are now two options for the order that topics appear on the board: you can see the most recently created threads at the top (the default), or the most recently updated threads. This new sorting option makes it easier to find the active conversations on the board. We’ve also made a few changes to make Flow discussions easier to read, including: a font size now consistent with other pages; dropdown menus now easier to read; the use of the new button style, and the WikiGlyphs webfont. Growth *Growth <https://www.mediawiki.org/wiki/Growth>* In June, the Growth team completed analysis <https://meta.wikimedia.org/wiki/Research:Anonymous_editor_acquisition/Signu…> of its first round of A/B testing of signup invitations for anonymous editors <https://www.mediawiki.org/wiki/Anonymous_editor_acquisition/Signup_invites> on English, French, German, and Italian Wikipedias. Based on these results, the team prepared a second version <https://www.mediawiki.org/wiki/Anonymous_editor_acquisition/Signup_invites_…> to be A/B tested. Additionally, the team released a major refactor of the GuidedTour extension <https://www.mediawiki.org/wiki/Extension:GuidedTour>‘s API, as well as design enhancements like animations, a new CSS-based way of drawing guider elements, updated button styles, and more. The team also launched GuidedTours on three new Wikipedias: Arabic, Norwegian, and Bengali. Support *Wikipedia Education Program <https://www.mediawiki.org/wiki/Wikipedia_Education_Program>* This month, the Education Program extension again received incremental improvements and bugfixes. Sage Ross of the Wiki Education Foundation <https://meta.wikimedia.org/wiki/Wiki_Education_Foundation> submitted two patches: one that adds information to the API for listing students, and another that lets anonymous users compare course versions. Also, a student from Facebook Open Academy fixed a usability issue in the article assignment feature. Mobile <https://www.mediawiki.org/wiki/Wikimedia_mobile_engineering> *Wikimedia Apps <https://www.mediawiki.org/wiki/Wikimedia_Apps>* The Mobile Apps team released the new Android Wikipedia app <https://play.google.com/store/apps/details?id=org.wikipedia> and it is now available to be downloaded through the Google Play store on Android devices. Core features of the app include the ability to save pages for offline reading, a record of your browsing history, and the ability to edit either as a logged in user or anonymously. Therefore the app is the first mobile platform that allows anonymous editing! The app also supports Wikipedia Zero for participating mobile carriers. Additional work done this month includes the start of implementing night mode for the Android app (by popular demand), creating an onboarding experience which is to be refined and deployed in July, and numerous improvements to the edit workflow. *Mobile web projects <https://www.mediawiki.org/wiki/Mobile_web_projects>* This month, the mobile web team finished work on styling the mobile site to provide a better experience for tablet users. We began redirecting users on tablets, who had previously been sent to the desktop version of all Wikimedia projects, to the new tablet-optimized mobile site on June 17. Our early data suggests that this change had a positive impact on new user signup and new editor activation numbers. We also continued work on VisualEditor features (the linking and citation dialogs) in preparation for releasing the option to edit via VisualEditor to tablet users in the next three months. *Wikipedia Zero <https://www.mediawiki.org/wiki/Wikipedia_Zero>* During the last month, the team deployed the refactored Wikipedia Zero codebase that replaces one monolithic extension with multiple extensions. The JsonConfig extension, which allows a wiki-driven JSON configuration system with data validation and a tiered configuration management architecture, had significant enhancements to make it more general for other use cases. Additionally, the team enabled downsampled thumbnails for a live in-house Wikipedia Zero operator configuration, and finished Wikipedia Zero minimum viable product design and logging polish for the Android and iOS Wikipedia apps. The team also supported the Wikipedia apps development with network connection management enhancements in Android and iOS, with *Find in page* functionality for Android, and response to Wikipedia for Android Google Play reviews. The team facilitated discussions on proxy and small screen device optimization, and examined the HTML5 app landscape for the upcoming fiscal year’s development roadmap. The team also created documentation for operators for enabling zero-rating with different connection scenarios. Bugfixes were issued for the mobile web Wikipedia Zero and the Wikipedia for Firefox OS app user experience. Routine pre- and post-launch configuration changes were made to support operator zero-rating, with routine technical assistance provided to operators and the partner management team to help add zero-rating and address anomalies. Finally, the team participated in recruitment for a third Partners engineering teammate. *Wikipedia Zero (partnerships)* We launched Wikipedia Zero with Airtel in Bangladesh, our third partner in Bangladesh, and our 34th launched partner overall. We participated in the Wiki Indaba conference, the first event of its kind to be held in Africa. The event, organized by Wikimedia South Africa, brought together community members from Tunisia, Egypt, Ghana, Kenya, Namibia, Nigeria, Ethiopia, Malawi and South Africa. The attendees shared experiences and challenges to work in the region and formulated strategies to support and strengthen the movement’s efforts across the continent. While in South Africa, Adele Vrana also met with local operators. Meanwhile, Carolynne Schloeder met with numerous operators and handset manufacturers in India. Carolynne joined Wikimedian RadhaKrishna Arvapally for a presentation at C-DOT, and both participated a blogger event hosted by our partner Aircel, along with other members of Wikimedia India in Bangalore. Smriti Gupta joined the group as Mobile Partnerships Manager, Asia. Language Engineering <https://www.mediawiki.org/wiki/Wikimedia_Language_engineering> *Language tools <https://www.mediawiki.org/wiki/Language_tools>* The Translate extension received numerous bug fixes, including fixing workflow states transitions for fundraising banners. *Content translation <https://www.mediawiki.org/wiki/Content_translation>* The team added support for link adaptation, worked on the infrastructure for machine translation support using Apertium and on hiding templates, images and references that cannot be easily translated. They also prepared for deployment on beta wikis and made multiple bug fixes and design tweaks. Platform Engineering <https://www.mediawiki.org/wiki/Wikimedia_Platform_Engineering> MediaWiki Core *HHVM <https://www.mediawiki.org/wiki/HHVM>* The team has been running HHVM on a single test machine (“osmium”) for the purpose of testing the job queue in production. The machine is only put into production on a very limited basis, while enough bugs are found to keep the team busy for a while, and then it’s disabled again as the team fixes those bugs. We’re planning on having HHVM running on a few job runner machines (continually) in July, then turning our focus toward running HHVM on the main application servers, taking a similar strategy. *Release & QA <https://www.mediawiki.org/wiki/Wikimedia_Release_and_QA_Team>* The Release and QA Team had their mid-quarter check-in on June 27. Phabricator work is progressing nicely. The latest MediaWiki tarball release (1.23) was made and the second RFP started and is close to completion. We are moving to only WMF-hosted Jenkins for all jobs, and we are working with the MediaWiki Core and the Operations teams on HHVM-related integration (both for deployment and for the Beta Cluster). *Admin tools development <https://www.mediawiki.org/wiki/Admin_tools_development>* Work on this project is currently being completed along with the SUL finalisation <https://www.mediawiki.org/wiki/SUL_finalisation> project, including the global rename tool (bug 14862 <https://bugzilla.wikimedia.org/show_bug.cgi?id=14862>) and cleaning up the CentralAuth database (bug 66535 <https://bugzilla.wikimedia.org/show_bug.cgi?id=66535>). *Search <https://www.mediawiki.org/wiki/Search>* CirrusSearch is running as the default search engine on all but the highest traffic wikis at this point. Nik Everett and Chad Horohoe plan to migrate most of the remaining wikis in July, leaving only the German and English Wikipedia to migrate in August. *Auth systems <https://www.mediawiki.org/wiki/Auth_systems>* Continued work on the SOA Authentication RFC <https://www.mediawiki.org/wiki/Requests_for_comment/SOA_Authentication> and Phabricator OAuth integration. We made OAuth compatible with HHVM and made other minor bug fixes. *SUL finalisation <https://www.mediawiki.org/wiki/SUL_finalisation>* The MediaWiki Core team has committed to having the following work completed by the end of September 2014: - Completing the necessary engineering work to carry out the finalisation. - Setting a date on which the finalisation will occur (Note: this date may not be later than September). - Have a communications strategy in place, and community liaisons to carry that out, for the time period between the announcement of the date of the finalisation and the finalisation proper. *Security auditing and response <https://www.mediawiki.org/wiki/Security_auditing_and_response>* We released MediaWiki 1.23.1 to prevent multiple issues caused by loading external SVG resources. We also performed security reviews of the Wikidata property suggester, Extension:Mantle <https://www.mediawiki.org/wiki/Extension:Mantle> for mobile/Flow, and Flow’s templating rewrite. Quality assurance *Quality Assurance <https://www.mediawiki.org/wiki/Quality_Assurance>* This month saw significant improvements to the MediaWiki-Vagrant development environments from new WMF staff member Dan Duvall. We have completed support for running the full suite of browser tests on a Vagrant instance under the VisualEditor role. In the near future, we will extend that support to the MobileFrontend and Flow Vagrant roles, as well as making general improvements to Vagrant overall. Another great QA project is from Google Summer of Code intern Vikas Yaligar, who is using the browser test framework to automate taking screen captures of aspects of VisualEditor (or any other feature) in many different languages, for the purpose of documentation and translation. *Browser testing <https://www.mediawiki.org/wiki/Quality_Assurance/Browser_testing>* After two years of using a third-party host to run browser test builds in Jenkins, this month we have completed the migration of those builds to Jenkins hosted by the Wikimedia Foundation. Hosting our browser test builds ourselves gives us more control over every aspect of running the browser tests, as well as the potential to run them faster than previously possible. Particular thanks to Antoine Musso, whose work made it possible. Simultaneously, we have also ported all of the remaining tests from the /qa/browsertest repository either to/mediawiki/core or to their relevant extension. This gives us the ability to package browser-based acceptance tests with the release of MediaWiki itself. After more than two years evolving the browser testing framework across WMF, the /qa/browsertests repository is retired, and all if its functions now reside in the repositories of the features being tested. Multimedia *Multimedia <https://www.mediawiki.org/wiki/Multimedia>* In June, the multimedia team released Media Viewer v0.2 <https://www.mediawiki.org/wiki/Multimedia/About_Media_Viewer> on all Wikimedia wikis <https://www.mediawiki.org/wiki/Multimedia/Media_Viewer/Release_Plan#Large_W…>, with over 20 million image views per day <http://multimedia-metrics.wmflabs.org/graphs/mmv> on sites we track. Global feedback was generally positive and helped surface a range of issues, many of which were addressed quickly. Based on this feedback, Gilles Dubuc, Mark Holmquist, and Gergő Tisza developed a number of new features, with designs by Pau Giner: view images in full resolution <https://www.mediawiki.org/wiki/Help:Multimedia/Media_Viewer#How_can_I_view_…> , view images in different sizes <https://www.mediawiki.org/wiki/Help:Multimedia/Media_Viewer#How_can_I_view_…> , show more image information <https://www.mediawiki.org/wiki/Help:Multimedia/Media_Viewer#How_can_I_view_…> , edit image file pages <https://www.mediawiki.org/wiki/Help:Multimedia/Media_Viewer#How_can_I_edit_…>, as well as easy disable tools for anonymous users <https://www.mediawiki.org/wiki/Help:Multimedia/Media_Viewer#How_can_I_turn_…> and editors. This month, we started working on the Structured Data <https://www.mediawiki.org/wiki/Multimedia/Structured_Data> project with the Wikidata team <https://www.wikidata.org/wiki/>, to implement machine-readable data on Wikimedia Commons. We are now in a planning phase and aim to start development in Fall. We ramped up our work on UploadWizard <https://www.mediawiki.org/wiki/UploadWizard>, reviewed user feedback, collected metrics, fixed bugs and started code refactoring, with the help of contract engineer Neil Kandalgaonkar. We also kept working on technical debt <https://wikimedia.mingle.thoughtworks.com/projects/multimedia/cards?favorit…> and bug fixes for other multimedia tools, such as image scalers, GWToolset and TimedMediaHandler, with the help of Summer contractor Brian Wolff. As product manager, Fabrice Florin helped plan our next steps, hosting a planning meeting <https://www.mediawiki.org/wiki/Multimedia/Meetings/Next_Cycle_3_Q4_2013-14> and other discussions of our development goals, and led an extensive review of user feedback for Media Viewer <https://docs.google.com/a/wikimedia.org/spreadsheets/d/14LnDVfgHrk4ZbFF1jm8…> and UploadWizard <https://docs.google.com/a/wikimedia.org/spreadsheets/d/1AUKUODbKVQMtFgTbCm8…> with new researcher Abbey Ripstra. Community liaison Keegan Peterzell introduced Media Viewer and responded to user comments throughout the product’s worldwide release <https://www.mediawiki.org/wiki/Multimedia/Media_Viewer/Release_Plan>. To learn more about our work, we invite you to join our discussions on the multimedia mailing list <https://lists.wikimedia.org/mailman/listinfo/multimedia>. Engineering Community Team <https://www.mediawiki.org/wiki/Engineering_Community_Team> *Bug management <https://www.mediawiki.org/wiki/Bug_management>* Apart from gruntwork (handling new tickets; prioritizing tickets; pinging on older tickets) and Andre’s main focus on Phabricator <https://www.mediawiki.org/wiki/Phabricator>, Parent5446, Krinkle and Andre created several requested Bugzilla components <https://bugzilla.wikimedia.org/buglist.cgi?bug_id=66704,66803,66875,66924,6…>, plus moved ‘MediaWiki skins’ to a Bugzilla product of their own <https://bugzilla.wikimedia.org/show_bug.cgi?id=66441>. In Bugzilla’s codebase, Tony and TTO styled Bugzilla’s Alias field differently <https://gerrit.wikimedia.org/r/#/c/124140/>, Tony removed the padlock icons for https links in Bugzilla <https://bugzilla.wikimedia.org/show_bug.cgi?id=59893> and cleaned up the codebase <https://gerrit.wikimedia.org/r/119726>, and Odder fixed a small glitch in Bugzilla’s Weekly Summary <https://bugzilla.wikimedia.org/show_bug.cgi?id=66778> and rendering of custom queries on the Bugzilla frontpage <https://gerrit.wikimedia.org/r/#/c/129671/>. Numerous older tickets with high priority were triaged on a bugday <https://www.mediawiki.org/wiki/Bug_management/Triage/20140624>. *Phabricator migration <https://www.mediawiki.org/wiki/Phabricator/Migration>* Apart from discussions on how to implement certain functionality and settings in Phabricator among team members and stakeholders, Mukunda implemented a MediaWiki OAuth provider in Phabricator (Gerrit changes: 1 <https://gerrit.wikimedia.org/r/#/c/139442/>,2 <https://gerrit.wikimedia.org/r/#/c/139438/>; related ticket <http://fab.wmflabs.org/T314>) and Chase created a Puppet module for Phabricator <https://gerrit.wikimedia.org/r/#/c/132505/>. *Mentorship programs <https://www.mediawiki.org/wiki/Mentorship_programs>* Google Summer of Code <https://www.mediawiki.org/wiki/Google_Summer_of_Code_2014> and FOSS Outreach Program for Women <https://www.mediawiki.org/wiki/FOSS_Outreach_Program_for_Women/Round_8> interns and mentors evaluated each other as part of the mid-term evaluations. Reports are available for all projects: - Tools for mass migration of legacy translated wiki content <https://www.mediawiki.org/wiki/Extension:Translate/Mass_migration_tools/Pro…> - Wikidata annotation tool <https://www.mediawiki.org/wiki/Wikidata_annotation_tool/updates> - Email bounce handling to MediaWiki with VERP <https://www.mediawiki.org/wiki/VERP/GSOC_Progress_Rerport> - Google Books, Internet Archive, Commons upload cycle <https://www.mediawiki.org/wiki/Google_Books,_Internet_Archive,_Commons_uplo…> - UniversalLanguageSelector fonts for Chinese wikis <https://www.mediawiki.org/wiki/Extension:UniversalLanguageSelector/Fonts_fo…> - MassMessage page input list improvements <https://www.mediawiki.org/wiki/Extension:MassMessage/Page_input_list_improv…> - Book management in Wikibooks/Wikisource <https://meta.wikimedia.org/wiki/Book_management_2014/Progress> - Parsoid-based online-detection of broken wikitext <https://www.mediawiki.org/wiki/User:Hardik95/GSoC_2014_Progress_Report> - Usability improvements for the Translate extension <https://www.mediawiki.org/wiki/User:Kunalgrover05/Progress_Report> - A modern, scalable and attractive skin for MediaWiki <https://www.mediawiki.org/wiki/User:Jack_Phoenix/GSoC_2014> - Automatic cross-language screenshots for user documentation <https://www.mediawiki.org/wiki/Automatic_cross-language_screenshots/progress> - Separating skins from core MediaWiki <https://www.mediawiki.org/wiki/Separating_skins_from_core_MediaWiki/Progress> - Chemical Markup support for Wikimedia Commons <https://www.mediawiki.org/wiki/Chemical_Markup_support_for_Wikimedia_Common…> - Improving URL citations on Wikimedia <https://www.mediawiki.org/wiki/User:Mvolz/Weekly_Reports> - Historical OpenStreetMap <https://www.mediawiki.org/wiki/User:JaimeLyn/Weekly_Reports> - Welcoming new contributors to Wikimedia Labs and Tool Labs <https://www.mediawiki.org/wiki/Welcome_to_labs/Progress_Reports> - Evaluating, documenting, and improving MediaWiki web API client libraries <https://www.mediawiki.org/wiki/Evaluating_and_Improving_MediaWiki_web_API_c…> - Feed the Gnomes – Wikidata Outreach <https://www.mediawiki.org/wiki/User:Thepwnco/OPW_Reporting> - Template Matching for RDFIO <https://www.mediawiki.org/wiki/Extension:RDFIO/Template_matching_for_RDFIO/…> - Switching Semantic Forms Autocompletion to Select2 <https://www.mediawiki.org/wiki/Extension:Semantic_Forms/Select2_for_autocom…> - Catalogue for Mediawiki Extensions <https://www.mediawiki.org/wiki/User:Adi.iiita/Gsoc2014/Report#Weekly_Report> - Generic, efficient localisation update service <https://www.mediawiki.org/wiki/Extension:LocalisationUpdate/LUv2/Updates> . *Technical communications <https://www.mediawiki.org/wiki/Technical_communications>* In addition to ongoing communications support <https://www.mediawiki.org/wiki/Technical_communications/Tech_blog_activity> for the engineering staff, Guillaume Paumier <https://www.mediawiki.org/wiki/User:Guillom> focused on information architecture of Wikimedia engineering activities. This notably involved reorganizing the Wikimedia Engineering <https://www.mediawiki.org/wiki/Wikimedia_Engineering> portal (now linked from mediawiki.org’s sidebar) and creating a status dashboard <https://www.mediawiki.org/wiki/Wikimedia_Engineering/Dashboard> that lists the status of all current activities hosted on mediawiki.org. The portal is now also cross-linked with the other main tech spaces (like Tech <https://meta.wikimedia.org/wiki/Tech> and Tech News <https://meta.wikimedia.org/wiki/Tech/News>) and team hubs. *Volunteer coordination and outreach <https://www.mediawiki.org/wiki/Volunteer_coordination_and_outreach>* Volunteers and staff are beginning to add or express interest in topics for the 2014 Wikimania Hackathon in London. The WMUK team is working hard to finalize venue logistics so that we can schedule talks and sessions in specific rooms. Everything is on track for a successful (and very large!) Hackathon. Tech Talks held in June:*How, What, Why of WikiFont* <https://plus.google.com/events/chpgv8usjd6dn38on07njjk28hg> on June 12 and *A Few Python Tips* <https://www.mediawiki.org/wiki/Meetings/2014-06-19> on June 19. A new process has been set up forvolunteers needing to sign an NDA <https://wikitech.wikimedia.org/wiki/Volunteer_NDA> in order to be granted special permissions in Wikimedia servers. On a similar note, we have started a project to implement a Trusted User Tool <http://fab.wmflabs.org/project/view/40/> in Phabricator, in order to register editors of Wikimedia projects that have been granted special permissions after signing a community agreement. *Architecture and Requests for comment process <https://www.mediawiki.org/wiki/Architecture_process>* Developers had several meetings <https://www.mediawiki.org/wiki/Architecture_meetings> on IRC about architectural issues or Requests for comment <https://www.mediawiki.org/wiki/Requests_for_comment>: - 2014-06-02 — Discussion <https://www.mediawiki.org/wiki/Architecture_meetings/RFC_review_2014-06-02> of Requests for comment/Grid system <https://www.mediawiki.org/wiki/Requests_for_comment/Grid_system>; - 2014-06-11 — Discussion <https://www.mediawiki.org/wiki/Architecture_meetings/RFC_review_2014-06-11> of several RfCs (lightning round); - 2014-06-13 — Discussion <https://www.mediawiki.org/wiki/Architecture_meetings/Security_guidelines_di…> of security guidelines draft <https://www.mediawiki.org/wiki/Security_for_developers/Architecture>; - 2014-06-20 — Discussion <https://www.mediawiki.org/wiki/Architecture_meetings/RFC_review_2014-06-20> of revamping MediaWiki’s skin systems: Trevor Parscal’s “Redo skin framework” <https://www.mediawiki.org/wiki/Requests_for_comment/Redo_skin_framework> and Bartosz Dziewoński’s “Separating skins from core MediaWiki” work <https://bugzilla.wikimedia.org/show_bug.cgi?id=65748>; - 2014-06-25 — Discussion <http://lists.wikimedia.org/pipermail/wikitech-l/2014-June/077227.html> of front-end and UX standardization <https://www.mediawiki.org/wiki/UX_standardization>. Analytics <https://www.mediawiki.org/wiki/Analytics> *Wikimetrics <https://www.mediawiki.org/wiki/Analytics/Wikimetrics>* To support Editor Engagement Vital Signs, the team has implemented a new metric: Newly Registered User. There is also a new backup system to preserve user’s reports on cohorts as well as the ability to tag cohorts. A number of bugs have been fixed, including fixing the first run of a recurrent report and preventing the creation of reports with invalid cohorts. *Data Processing <https://www.mediawiki.org/wiki/Analytics/Data_Processing>* The team has now integrated Data Processing as part of its Development Process. New Stories/Features have been identified and tasked. Also, experimentation with Cloudera Hadoop 5 is complete and we are ready to upgrade the cluster in July. *Editor Engagement Vital Signs <https://www.mediawiki.org/wiki/Analytics/Editor_Engagement_Vital_Signs>* The ability to run a metric over an entire project (wiki) in Wikimetrics drives us closer to producing data daily for our first Vital Sign. The team has also iterated on the design of the dashboard and navigation. We added a requirement from executives to have a default view when EEVS is loaded. This view would display metrics for the 7 largest Wikipedias. *EventLogging <https://www.mediawiki.org/wiki/Analytics/EventLogging>* We fixed a serious bug where cookie data was getting captured in the country column. Saved data was scrubbed of the unwanted information and some old and unused tables were dropped. The team also implemented Throughput Monitoring to help catch potential issues in EventLogging. *Research and Data <https://www.mediawiki.org/wiki/Analytics/Research_and_Data>* This month we refined the Editor Model – a proposal to model the main drivers of monthly active editors <https://commons.wikimedia.org/wiki/File:Editor_Model_review.pdf> – and expanded the documentation <https://meta.wikimedia.org/wiki/Research:Rolling_monthly_active_editor> of the corresponding metric definitions. We applied this model to teams designing editor engagement features (Growth, Mobile) and supported them in setting targets for the next fiscal year. We analyzed the early impact of the tablet desktop-to-mobile switchover on traffic <https://trello.com/c/sWoW3yqC/341-tablet-switchover-data-for-the-mobile-team> , edit volume, unique editors <https://trello.com/c/NsBSDPVo/356-tablet-switchover-edit-volume%7Ctotal>, and new editor activation <https://trello.com/c/LKVK7RfL/351-tablet-switchover-editor-activation>. We hosted the June 2014 edition <https://www.mediawiki.org/wiki/Analytics/Research_and_Data/Showcase#June_20…> of the research showcase with two presentations on the effect of early socialization strategies <https://commons.wikimedia.org/wiki/File:Moodbar_--_lightweight_socializatio…> and on predictive modeling of editor retention <https://commons.wikimedia.org/wiki/File:Active_editor_survival.pdf>. We released wikiclass <http://pythonhosted.org/wikiclass/>, a library for performing automated quality assessment of Wikipedia articles. We released longitudinal data on the daily edit volume <https://trello.com/c/NIcjLKjd/328-longitudinal-ve-vs-wt-data-all-wikis> for all wikis with VisualEditor enabled, since the original rollout. We continued work on an updated definition for PageViews. Finally, we held our quarterly review <https://commons.wikimedia.org/wiki/File:Analytics_Quarterly_Review_Q4_2013-…> (Q4-2014) and presented our goals <https://www.mediawiki.org/wiki/Analytics/Research_and_Data/2014-15_Goals#Q1…> for the next quarter (Q1-2015). Wikidata <https://meta.wikimedia.org/wiki/Wikidata> *The Wikidata project is funded and executed by Wikimedia Deutschland <https://meta.wikimedia.org/wiki/Wikimedia_Deutschland/en>.* The team worked on fixing bugs as well as a number of features. These include data access for Wikiquote, support for redirects, the monolingual text datatype as well as further work on queries. Interface messages where reworked to make them easier to understand. First mockups of the new interface design <https://www.wikidata.org/wiki/Wikidata:UI_redesign_input> have been published for comments. The entity suggester a team of students worked on over the last months has been deployed. This makes it easier to add new statements by suggesting what kind of statements are missing on an item. Wikidata the Game <http://tools.wmflabs.org/wikidata-game/> has been extended by Magnus by 2 games to add date of birth and date of death to people as well as to add missing images. FutureThe engineering management team continues to update the *Deployments <https://wikitech.wikimedia.org/wiki/Deployments>* page weekly, providing up-to-date information on the upcoming deployments to Wikimedia sites, as well as the *annual goals <https://www.mediawiki.org/wiki/Wikimedia_Engineering/2014-15_Goals>*, listing ongoing and future Wikimedia engineering efforts. ------------------------------ *This article was written collaboratively by Wikimedia engineers and managers. See revision history <https://www.mediawiki.org/w/index.php?title=Wikimedia_Engineering/Report/20…> and associated status pages. A wiki version <https://www.mediawiki.org/wiki/Wikimedia_Engineering/Report/2014/June> is also available.* -- Guillaume Paumier Technical Communications Manager — Wikimedia Foundation

1 0

RFC: Composer managed libraries for use on WMF cluster
by Bryan Davis 15 Jul '14

15 Jul '14

I have converted my email on using composer to manage a set of library dependencies for MediaWiki-Core [0] into an RFC [1]. Work is continuing on the implementation of this project, but there are still debatable implementation details and the RFC process is meant to not only validate ideas but leave behind a record of the design decisions that have been made and trade offs that were considered in the process. In particular, the current draft RFC omits discussion of the concept of library "ownership" for long term updates and security fixes and could use more detail around the process of forking, patching and subsequently maintaining a external library. I will attempt to fill in some of these details as I see them over the next day or so, but now would be a great time for people with strong ideas or opinions on these aspects to comment on the talk page. [0]: http://www.gossamer-threads.com/lists/wiki/wikitech/467520?page=last [1]: https://www.mediawiki.org/wiki/Requests_for_comment/Composer_managed_librar… Bryan -- Bryan Davis Wikimedia Foundation <bd808(a)wikimedia.org> [[m:User:BDavis_(WMF)]] Sr Software Engineer Boise, ID USA irc: bd808 v:415.839.6885 x6855

1 1

Moving Mathoid to production cluster
by Moritz Schubotz 15 Jul '14

15 Jul '14

Hi, during the last year the math extension achieved a goal defined back in 2003. Support of MathML. In addition there is SVG support for MathML disabled browsers. (See http://arxiv.org/abs/1404.6179 for the details) I would like to give Wikipedia users a chance to test this new long awaited feature. Therefore we would need a mathoid instance that is accessible from the production cluster. Greg Grossmeier already created the required table in the database. (Sorry for the "friction" connected with this process) Currently the MathJax team is working on a phantom.js less method to render texvc to mathml and svg. Some days ago I have tested that it, and it works quite well. I would appreciate a discussion with ops that to figure out how this can be can go to production. The original idea was to use jenkins to build the mathoid debian package. Even though the debian package builds without any issues in the launchpad ppa repo jenkins can not build the package. If there is a reference project that uses jenkins to build debian packages that go to production this would really help to figure out what is different for mathoid and why the package building does not work even though it works on launchpad. Best Physikerwelt PS: I was informed that there is a related RT that I can not access https://rt.wikimedia.org/Ticket/Display.html?id=6077 -- Mit freundlichen Grüßen Moritz Schubotz

3 4

Fwd: Strange mailing list behavior
by James Alexander 15 Jul '14

15 Jul '14

Apologies if this ends up being a duplicate, forwarding it along on Pine's request because he's been having mail issues and is concerned this one is also not getting through. It is also, of course, possible that my email will get stuck too and they will all appear later down the road together. So far I'm not seeing a huge pile of queued messages on ganglia (though there was a bit of a bump earlier). James Alexander Legal and Community Advocacy Wikimedia Foundation (415) 839-6885 x6716 @jamesofur ---------- Forwarded message ---------- From: Pine W <wiki.pine(a)gmail.com> Date: Tue, Jul 15, 2014 at 1:02 AM Subject: Strange mailing list behavior To: "wikitech-l(a)lists.wikimedia.org" <wikitech-l(a)lists.wikimedia.org> So this evening, besides my one email to Wikimedia-l that got through, 2 others disappeared into thin air. Also, I just received an email in my Google account that was sent 11 hours ago. Can someone check for gremlins in the mail system? The problem might be on Google's end or in the mail system, I can't tell, but I suspect the mail system because mail sent to non-list addresses seems to get through ok. Thanks, Pine

1 0

Scribunto, Lua 5.2, and forward compatibility for Module code
by gnosygnu 15 Jul '14

15 Jul '14

Hi all. I came across a strange issue with regards to Lua 5.1 and Lua 5.2 this weekend. The defect was caused by a change in behavior for "tonumber" between 5.1 and 5.2. I detail more below for anyone interested. With that in mind, I wanted to ask a few questions to the group: * Will Scribunto support Lua 5.2 in the future? According to this comment, it may: https://github.com/wikimedia/mediawiki-extensions-Scribunto/blob/master/Scr… * If so, what is the best approach to ensure forward compatibility of Module code for 5.2? Note that code that is valid for 5.1 may "break" in 5.2. (Again, see below for details) ** Should there be a reference page on MediaWiki or in the enwiki Wikipedia Namespace that details these breaking issues for other Module writers? ** Or should 5.1 vs 5.2 issues be corrected on a case-by-case basis for each Module? ** Or should they be left as is, as any future-proofing may be unnecessary and / or premature? * Lastly, is there a general purpose mailing list for Scribunto issues? I've had a few technical questions in the past that have been painful to sort out on my own. I'd like to think that there might be other Module writers who would also be interested in a mailing list as well. Detail on the "tonumber" issue follows below. ---- == In brief == * For a call of "tonumber(213.5616, 10)" ** 5.1 will return back 213.5616 ** 5.2 will return back NIL == In detail == * Template:Weather_box uses Module:WeatherBox which eventually calls Module:BaseConvert * Module:WeatherBox often calls Module:BaseConvert with decimal values. For example, here is a sample call: {{#invoke:BaseConvert|convert|n=213.5615|base = 16|width = 2|precision = 0}} * Module:BaseConvert has the following lines from = from or 10 local num = tonumber(n, from) * Note that if no {{{from}}} is passed, the default is 10. In general, most callers do not specify a {{{from}}} (as seen above) * Lua 5.1 has code that specifically says if tonumber is called with no base or a base of 10, convert it as a number // http://www.lua.org/source/5.1/lbaselib.c.html static int luaB_tonumber (lua_State *L) { int base = luaL_optint(L, 2, 10); if (base == 10) { /* standard conversion */ * In contrast, Lua 5.2 has code that only cares if a base is not specified. // http://www.lua.org/source/5.2/lbaselib.c.html static int luaB_tonumber (lua_State *L) { if (lua_isnoneornil(L, 2)) { /* standard conversion */ ** If a base is specified, Lua 5.2 will not do the standard conversion, and instead try to parse the number ** This parse code only accepts alpha-numeric characters. The dot is considered invalid, and any decimal number is converted to NIL * As a result, for a call of "tonumber(213.5616, 10)" ** 5.1 will return back 213.5616 ** 5.2 will return back NIL * The NIL removes the background (and sometimes text colors) in weather boxes. For those curious, I discovered this in XOWA which uses Luaj -- a Java port of Lua 5.2. I had to backport the 5.1 behavior to get the Weather box to show the right colors again. A proposed fix that would be forward compatible with 5.2 would be as follows: local num = 0; -- {{{from}}} is specified, and {{{from}}} is not base 10 if (from && from <> 10) then -- call overload that will always do parsing num = tonumber(n, from ) -- {{{from}}} is not specified or {{{from}}} is base 10 else -- call overload that will always convert to number num = tonumber(n) from = from or 10 Finally, I've come across a related issue with the varargs operator ("..."). This operator is valid in 5.1, but not in 5.2. I've seen some Modules use this varargs operator, that will presumably just break in 5.2. See https://en.wikipedia.org/wiki/Module:Horizontal_timeline and "function getNotNilValue(...)"

2 2

Who maintains GeoCrumbs? (a changeset waits for review)
by Quim Gil 15 Jul '14

15 Jul '14

Hi, GeoCrumbs is an extension deployed in Wikimedia servers: https://www.mediawiki.org/wiki/Extension:GeoCrumbs It has no official maintainers, what about unofficial ones? Any volunteers? https://www.mediawiki.org/wiki/Developers/Maintainers It also has one of the oldest code contributions waiting to be reviewed by anybody other than Jenkins. https://gerrit.wikimedia.org/r/#/c/65285/ Because of this change, GeoCrumbs currently leads the list of slowest projects reviewing patches. http://korma.wmflabs.org/browser/gerrit_review_queue.html -- Quim Gil Engineering Community Manager @ Wikimedia Foundation http://www.mediawiki.org/wiki/User:Qgil

2 1

Re: [Wikitech-l] [Wikitech-ambassadors] Deprecating print-on-demand functionality
by Erik Moeller 15 Jul '14

15 Jul '14

On Fri, Jul 11, 2014 at 8:45 AM, Luca Martinelli <martinelliluca(a)gmail.com> wrote: > so the Book Creator will still be active, maybe under another name, > maybe with another engine, but still active? Same name and functionality, just the "Order a printed book" feature will disappear. Erik -- Erik Möller VP of Engineering and Product Development, Wikimedia Foundation

2 1

Fwd: Screenshot of Winter on Win7/IE9
by Risker 15 Jul '14

15 Jul '14

Forwarding to the list in case anyone else is interested in seeing the screenshot. Please note the comments below. Risker ---------- Forwarded message ---------- From: Risker <risker.wp(a)gmail.com> Date: 14 July 2014 09:03 Subject: Screenshot of Winter on Win7/IE9 To: bharris(a)wikimedia.org Hi Brandon - attaching this for you, feel free to upload, I release this screenprint under the GDFL and CC-BY-SA. The screenprint is a tiny bit worse than the actual screen - the letters are more broken up in the screenshot. Otherwise, the colour, the spacing and the overlaps are the same. Risker/Anne

2 1

Madison PHP wants speakers, can cover travel
by Sumana Harihareswara 14 Jul '14

14 Jul '14

http://cfp.madisonphpconference.com/ If you've never presented at a conference before, sometimes a smaller regional conference is an easier way to get started. Madison PHP is one day (Saturday, September 13th, 2014) and seeks talks for attendees at all skill levels. And their speaker compensation package includes complimentary airfare/travel and hotel. I suggest you submit talks: * on MediaWiki extension development, like http://opensourcebridge.org/wiki/2014/Extension_Development_with_Mediawiki (slides available) * on HipHop and HHVM * on coding conventions https://www.mediawiki.org/wiki/Manual:Coding_conventions/PHP * on how you do code review The proposal deadline is Monday, July 21. Sumana Harihareswara Senior Technical Writer Wikimedia Foundation

1 0

Tell my favorite conference about your Wikimedia tech
by Roan Kattouw 14 Jul '14

14 Jul '14

The linux.conf.au conference, which I have presented to and love going to, just opened up its call for talks: http://linux.conf.au/cfp . They want talks about all kinds of open source programming stuff, not just Linux: Trevor and I presented about ResourceLoader in 2012, and James and I presented about VisualEditor in 2014. LCA 2015 will be 12-16 January in Auckland, New Zealand. That's summer in the Southern Hemisphere, and the climate there is very moderate, so no sweltering heat like in Australia :) LCA provides travel funding to some speakers, and Wikimedia funds TPS grants for people going to conferences to talk about Wikimedia-related things: https://meta.wikimedia.org/wiki/Grants:TPS . So you could even get to go for free. LCA is basically my favorite conference - the talks are great and of high quality, they treat speakers well, and their audience gets what we're doing. This is a fun chance to show off your project. I encourage you to suggest a talk, or tell someone else that they should talk about their project. Roan P.S.: Thanks to Sumana for writing most of this email, and for talking me into proposing a talk for LCA in the first place back in 2011

5 8

← Newer
1
...
7
8
9
10
11
12
13
...
16
Older →

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Wikitech-l July 2014