Wikitech-l May 2014

wikitech-l@lists.wikimedia.org

132 participants
137 discussions

Security announcement: XSS in MobileFrontend
by Max Semenik 15 May '14

15 May '14

During internal review, an XSS (cross-site scripting) vulnerability was discovered in MobileFrontend extension. Due to an unneeded unescaping of already sanitized section titles, HTML inserted as plaintext into them was injected into DOM. While on ordinary page views only users who have intentionally enabled MobileFrontend's beta mode are in danger, it is possible to construct URLs that enable beta for every user following them. Another requirement for this vulnerability is screen witdth which must be at least 768 pixels. Affected versions include MobileFrontend for MediaWiki 1.23 (branch REL1_23, still in release candidate phase) and 1.24 (master). If you are running a 1.24 WMF branch earlier than wmf/1.24wmf3, please update to a later branch. -- Best regards, Max Semenik ([[User:MaxSem]])

2 2

Re: [Wikitech-l] [Multimedia] Too much open bugs (Wikimedia Commons, Media storage)
by Fabrice Florin 14 May '14

14 May '14

Thanks, Steinsplitter! We really appreciate your call to action to address some of the multimedia issues you list below. You will be glad to hear that our multimedia team is now shifting its focus to more of that technical debt and is taking on an upgrade of the Upload Wizard as our next big project for coming months. We will keep you posted on major developments, but if you are interested in being more involved in this multimedia work, we invite you to join this mailing list, where we actively discuss current tasks, in partnership with community members: https://lists.wikimedia.org/mailman/listinfo/multimedia You can also track our work on this planning board for our current development cycle: http://ur1.ca/h7w5s We look forward to working with you and other community members to address these issues over time. :) Be well, Fabrice _______________________________ Fabrice Florin Product Manager Wikimedia Foundation http://en.wikipedia.org/wiki/User:Fabrice_Florin_(WMF) On May 14, 2014, at 12:40 PM, Greg Grossmeier <greg(a)wikimedia.org> wrote: > FYI. > > ----- Forwarded message from Steinsplitter Wiki <steinsplitter-wiki(a)live.com> ----- > >> Date: Wed, 14 May 2014 21:01:38 +0200 >> From: Steinsplitter Wiki <steinsplitter-wiki(a)live.com> >> To: "wikitech-l(a)lists.wikimedia.org" <wikitech-l(a)lists.wikimedia.org> >> Subject: [Wikitech-l] Too much open bugs (Wikimedia Commons, Media storage) >> Reply-To: Wikimedia developers <wikitech-l(a)lists.wikimedia.org> >> >> Hi, >> I wanted to bring to your attention a few critical issues that are negatively affecting the workof the Commons community and the wider Wikimedia projects. >> First, there are about 36 unresolved bugs [0] in the "Media storage" component. Swift is a vitalcomponent of the projects' ability to show images and other media, and it having so manyopen bugs causes serious ongoing issues, not only on Commons, but everywhere. >> Some of these bugs are high priority, and have been open since 2012. The community of adminsare straining under the load, and it's not getting any better now that additional paths into theupload pipeline are becoming available. >> UploadWizard, too, is fundamentally broken. There are a *lot* of open bugs [1] [2] and the interfaceis in need of design love. >> It would be awesome to see these issues addressed in the coming months. In particular, becauseWiki Loves Monuments will start in September, it would be great to have these issues fixedon or before August 13th, giving Commons about two weeks to test the fixes and changesbefore the contest starts, and plenty of room for backporting urgent fixes. >> Regards,Steinsplitter >> https://commons.wikimedia.org/wiki/User:Steinsplitter >> >> [0] https://bugzilla.wikimedia.org/buglist.cgi?component=Media%20storage&produc… https://commons.wikimedia.org/wiki/Special:PrefixIndex/Commons:Upload_help/… https://bugzilla.wikimedia.org/buglist.cgi?component=UploadWizard&product=M… >> >> _______________________________________________ >> Wikitech-l mailing list >> Wikitech-l(a)lists.wikimedia.org >> https://lists.wikimedia.org/mailman/listinfo/wikitech-l > > ----- End forwarded message ----- > > -- > | Greg Grossmeier GPG: B2FA 27B1 F7EB D327 6B8E | > | identi.ca: @greg A18D 1138 8E47 FAC8 1C7D | > > _______________________________________________ > Multimedia mailing list > Multimedia(a)lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/multimedia

1 0

Too much open bugs (Wikimedia Commons, Media storage)
by Steinsplitter Wiki 14 May '14

14 May '14

Hi, I wanted to bring to your attention a few critical issues that are negatively affecting the workof the Commons community and the wider Wikimedia projects. First, there are about 36 unresolved bugs [0] in the "Media storage" component. Swift is a vitalcomponent of the projects' ability to show images and other media, and it having so manyopen bugs causes serious ongoing issues, not only on Commons, but everywhere. Some of these bugs are high priority, and have been open since 2012. The community of adminsare straining under the load, and it's not getting any better now that additional paths into theupload pipeline are becoming available. UploadWizard, too, is fundamentally broken. There are a *lot* of open bugs [1] [2] and the interfaceis in need of design love. It would be awesome to see these issues addressed in the coming months. In particular, becauseWiki Loves Monuments will start in September, it would be great to have these issues fixedon or before August 13th, giving Commons about two weeks to test the fixes and changesbefore the contest starts, and plenty of room for backporting urgent fixes. Regards,Steinsplitter https://commons.wikimedia.org/wiki/User:Steinsplitter [0] https://bugzilla.wikimedia.org/buglist.cgi?component=Media%20storage&produc… https://commons.wikimedia.org/wiki/Special:PrefixIndex/Commons:Upload_help/… https://bugzilla.wikimedia.org/buglist.cgi?component=UploadWizard&product=M…

1 0

1.23.0: second release candidate ready for download
by Mark A. Hershberger 14 May '14

14 May '14

The second release candidate for 1.23.0 (1.23.0-rc.1) is now available for download. Note that we're making two changes in the release process: 1. We'll be making this and all future releases on Wednesday instead of Thursday or Friday. This will allow people to avoid late Friday nights. 2. We're switching to Semantic Versioning (http://semver.org) for the release candidates and other releases. The changes since 1.23.0rc0 are as follows: * Added pp_sortkey column to page_props table, so pages can be efficiently queried and sorted by property value (bug 58032). * Introduced $wgPagePropsHaveSortkey as a backwards-compatibility switch, for using the old schema of the page_props table, in case the respective schema update was not applied. * (bug 13250) Restored method for clearing a watchlist in web UI so that users with large watchlists don't have to perform contortions to clear them. * (bug 63269) Email notifications were not correctly handling the [[MediaWiki:Helppage]] message being set to a full URL (the default). If you customized [[MediaWiki:Enotif body]] (the text of email notifications), you'll need to edit it locally to include the URL via the new variable $HELPPAGE instead of the parser functions fullurl and canonicalurl; otherwise you don't have to do anything. * $wgProfileToDatabase was removed. Set $wgProfiler to ProfilerSimpleDB in StartProfiler.php instead of using this. * The fix for bug 14323 was pushed to MediaWiki 1.24 since the changes cause problems for Semantic MediaWiki users. * The modules intended for use by custom skins were renamed. Full release notes: https://git.wikimedia.org/blob/mediawiki%2Fcore.git/1.23.0-rc.1/RELEASE-NOT… https://www.mediawiki.org/wiki/Release_notes/1.23 ********************************************************************** Download: http://download.wikimedia.org/mediawiki/1.23/mediawiki-core-1.23.0-rc.1.tar… http://download.wikimedia.org/mediawiki/1.23/mediawiki-1.23.0-rc.1.tar.gz GPG signatures: http://download.wikimedia.org/mediawiki/1.23/mediawiki-core-1.23.0-rc.1.tar… http://download.wikimedia.org/mediawiki/1.23/mediawiki-1.23.0-rc.1.tar.gz.s… Public keys: https://www.mediawiki.org/keys/keys.html Mark A. Hershberger (Release Management Team)

1 0

Wikimedia engineering report, April 2014
by Guillaume Paumier 14 May '14

14 May '14

Hi, The report covering Wikimedia engineering activities in April 2014 is now available. Wiki version: https://www.mediawiki.org/wiki/Wikimedia_engineering_report/2014/April Blog version: https://blog.wikimedia.org/2014/05/14/engineering-report-april-2014/ We're also proposing a shorter, simpler and translatable version of this report that does not assume specialized technical knowledge: https://www.mediawiki.org/wiki/Wikimedia_engineering_report/2014/April/summ… Below is the HTML text of the report. As always, feedback is appreciated on the usefulness of the report and its summary, and on how to improve them. ------------------------------------------------------------------ Major news in April include: - the change of format<https://blog.wikimedia.org/2014/04/10/mediawiki-localization-file-format-ch…> of MediaWiki localization files from PHP to JSON, and the associated modernization of the LocalisationUpdate extension<https://blog.wikimedia.org/2014/04/03/modernising-mediawikis-localisation-u…> ; - the move of Wikimedia Labs<https://blog.wikimedia.org/2014/04/04/migrating-wikimedia-labs-to-a-new-dat…> to a new data center; - the “Heartbleed” security vulnerability<https://blog.wikimedia.org/2014/04/10/wikimedias-response-to-the-heartbleed…> and how the Wikimedia Foundation’s team responded to it; - an explanation of how the Mobile team uses Trello<https://blog.wikimedia.org/2014/04/15/agile-and-trello-the-planning-cycle/> to plan their development sprints; - a project report on a grant to create “gadgets” for VisualEditor<https://blog.wikimedia.org/2014/04/22/visualeditor-gadgets/> . *Note: We’re also providing a shorter, simpler and translatable version of this report <https://www.mediawiki.org/wiki/Wikimedia_engineering_report/2014/April/summ…> that does not assume specialized technical knowledge.* Engineering metrics in April: - 158 unique committers contributed patchsets of code to MediaWiki. - The total number of unresolved commits<https://gerrit.wikimedia.org/r/#q,status:open+project:%255Emediawiki.*,n,z>went from around 1315 to about 1305. - About 30 shell requests <https://www.mediawiki.org/wiki/Shell_requests> were processed. Contents - 1 Personnel<https://blog.wikimedia.org/2014/05/14/engineering-report-april-2014/#Person…> - 1.1 Work with us<https://blog.wikimedia.org/2014/05/14/engineering-report-april-2014/#Work_w…> - 1.2 Announcements<https://blog.wikimedia.org/2014/05/14/engineering-report-april-2014/#Announ…> - 2 Technical Operations<https://blog.wikimedia.org/2014/05/14/engineering-report-april-2014/#Techni…> - 3 Features Engineering<https://blog.wikimedia.org/2014/05/14/engineering-report-april-2014/#Featur…> - 3.1 Editor retention: Editing tools<https://blog.wikimedia.org/2014/05/14/engineering-report-april-2014/#Editor…> - 3.2 Core Features<https://blog.wikimedia.org/2014/05/14/engineering-report-april-2014/#Core_F…> - 3.3 Growth<https://blog.wikimedia.org/2014/05/14/engineering-report-april-2014/#Growth> - 3.4 Support<https://blog.wikimedia.org/2014/05/14/engineering-report-april-2014/#Support> - 4 Mobile<https://blog.wikimedia.org/2014/05/14/engineering-report-april-2014/#Mobile> - 5 Language Engineering<https://blog.wikimedia.org/2014/05/14/engineering-report-april-2014/#Langua…> - 6 Platform Engineering<https://blog.wikimedia.org/2014/05/14/engineering-report-april-2014/#Platfo…> - 6.1 MediaWiki Core<https://blog.wikimedia.org/2014/05/14/engineering-report-april-2014/#MediaW…> - 6.2 Quality assurance<https://blog.wikimedia.org/2014/05/14/engineering-report-april-2014/#Qualit…> - 6.3 Multimedia<https://blog.wikimedia.org/2014/05/14/engineering-report-april-2014/#Multim…> - 6.4 Engineering Community Team<https://blog.wikimedia.org/2014/05/14/engineering-report-april-2014/#Engine…> - 7 Analytics<https://blog.wikimedia.org/2014/05/14/engineering-report-april-2014/#Analyt…> - 8 Kiwix<https://blog.wikimedia.org/2014/05/14/engineering-report-april-2014/#Kiwix> - 9 Wikidata<https://blog.wikimedia.org/2014/05/14/engineering-report-april-2014/#Wikida…> - 10 Future<https://blog.wikimedia.org/2014/05/14/engineering-report-april-2014/#Future> Personnel Work with us <https://wikimediafoundation.org/wiki/Work_with_us> Are you looking to work for Wikimedia? We have a lot of hiring coming up, and we really love talking to active community members about these roles. - VP of Engineering<http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&pag…> - ScrumMaster<http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&pag…> - Software Engineer – VisualEditor Team<http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&pag…> - Software Engineer – Fundraising Team<http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&pag…> - Software Engineer – Internationalization<http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&pag…> - Software Engineer (Partners)<http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&pag…> - QA Tester<http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&pag…> - Research Analyst – Fundraising<http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&pag…> - Product Manager – Language Engineering<http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&pag…> - Operations Security Engineer<http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&pag…> - Data Center Engineer (Contractor)<http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&pag…> Announcements - Giuseppe Lavagetto joined the Operations team as Operations Engineer ( announcement<http://lists.wikimedia.org/pipermail/wikitech-l/2014-April/075569.html> ). - Aaron Schulz is now Senior Performance Engineer (announcement<http://lists.wikimedia.org/pipermail/wikitech-l/2014-April/075764.html> ). - Dmitry Brant joined the the Mobile App Team as Software Developer ( announcement<http://lists.wikimedia.org/pipermail/wikitech-l/2014-April/076128.html> ). - Danny Horn joined the Product Development team as Product Manager ( announcement<http://lists.wikimedia.org/pipermail/wikitech-l/2014-April/076191.html> ). Technical Operations *Datacenter RFP <https://wikimediafoundation.org/wiki/RFP/2013_Datacenter>* The Wikimedia Foundation has chosen a winning RFP bid, a contract has been executed and implementation is underway. A public announcement is being prepared in the upcoming week. *Wikimedia Labs* Labs metrics in April: - Number of projects: 153 - Number of instances: 345 - Amount of RAM in use (in MBs): 1,454,592 - Amount of allocated storage (in GBs): 16,515 - Number of virtual CPUs in use: 716 - Number of users: 3,064 The migration of Labs and Tool Labs<https://blog.wikimedia.org/2014/04/04/migrating-wikimedia-labs-to-a-new-dat…> to the Ashburn data center is complete, and most of the hardware in Tampa has been shut down and packed up for shipping to the new (to be announced) data center. Post-migration, many projects which had public IPs are now relying on the internal Labs web proxy instead. That has caused a few unexpected bugs in project web access, but provides several benefits including HTTPS access and increased user data privacy. *Tampa data center* During the last month, our data center footprint in Tampa has been reduced to just 6 racks, reduced from 24 total. A copy of all essential data remains present in the Tampa facility until we’ve finished setting up the relevant services in our upcoming new data center. Features Engineering<https://www.mediawiki.org/wiki/Wikimedia_Features_engineering> Editor retention: Editing tools *VisualEditor <https://www.mediawiki.org/wiki/VisualEditor>* In April, the VisualEditor team worked to improve the stability of the editor, adding some new features and improving usability so that users can create and edit pages more swiftly and intuitively with VisualEditor than before. Template editing was over-hauled to make adding parameters less busy, showing only a few parameters at first rather than all possible ones, which can number in the dozens or more, especially in the case of some often-used templates like those for citations or infoboxes. Setting the size of images was tweaked to give a more natural set of controls based on feedback from users. The page settings dialog had a number of minor tweaks, leading to the set of options that can be modified inside VisualEditor being completed. VisualEditor’s edit tab is now more consistent with the rest of the MediaWiki interface in a number of noticeable if minor ways, such as on pages to do with the Education Program, on file pages which are hosted on Commons rather than on the local wiki, or on very narrow screens. User testing was carried out on the forthcoming citation dialog and some final simplifications were made, such as adding suggested as well as required parameters, ahead of its pending introduction. Finally, a careful audit of all Wikimedia wikis led to fixing broken local community-written code, to ensure that VisualEditor runs on all of them. The deployed version of the code was updated four times in the regular release cycle (1.23-wmf21<https://www.mediawiki.org/wiki/MediaWiki_1.23/wmf21#VisualEditor> , 1.23-wmf22<https://www.mediawiki.org/wiki/MediaWiki_1.23/wmf22#VisualEditor> , 1.24-wmf1<https://www.mediawiki.org/wiki/MediaWiki_1.24/wmf1#VisualEditor> and 1.24-wmf2<https://www.mediawiki.org/wiki/MediaWiki_1.24/wmf2#VisualEditor> ). *Parsoid <https://www.mediawiki.org/wiki/Parsoid>* In April, the Parsoid team continued to fix bugs and tweak code. Two areas in particular received a lot of attention: template encapsulation and link handling. We ironed out a whole bunch of edge case handling in template encapsulation code and its interaction with fostered content from tables (caused by misnested tags in tables). We also fixed many unhandled scenarios and edge cases parsing and serializing links. In addition to bug fixes, we also improved the performance of the parsing pipeline; some pages like *Barack Obama <https://en.wikipedia.org/wiki/Barack_Obama>* should now parse 30% faster than before. We continued migrating our debugging and tracing code to use our new logger. April also saw additional progress providing support for visual editing of transclusion parameters; this should land on master soon. Core Features *Flow <https://www.mediawiki.org/wiki/Flow/Project_information>* This month, the Flow team focused on back-end changes to improve moderation and history viewing, as well as implementing new front-end JavaScript templating to make Flow more responsive and easier to add new features onto. On the user-facing features side, we released the ability to close and summarize topics. This will allow users to manage active discussions and end ones that have come to a resolution. Flow is now the default discussion experience for many Beta Features discussions on mediawiki.org, and the team is accepting requests to enable Flow on more pages on that wiki for the purpose of testing complex multi-user discussion interactions. Growth *Growth <https://www.mediawiki.org/wiki/Growth>* In April, Growth switched gears to focus on a new experimental area: anonymous editor acquisition<https://www.mediawiki.org/wiki/Anonymous_editor_acquisition>. The team prepared its first two experimental interface changes, aimed at asking anonymous editors to register accounts<https://www.mediawiki.org/wiki/Anonymous_editor_acquisition/Signup_invites>(expected to be launched in early May). The team also will be conducting basic research into the role anonymous editors play in Wikipedia − more at Research:Anonymous phenomena <https://meta.wikimedia.org/wiki/Research:Anonymous_phenomena> and Research:Anonymous editor acquisition<https://meta.wikimedia.org/wiki/Research:Anonymous_editor_acquisition> . Support *Wikipedia Education Program <https://www.mediawiki.org/wiki/Wikipedia_Education_Program>* This month we deployed several bug fixes, including disabling the malfunctioning and little-used *student profiles*feature and setting a sensible default end date for new courses. Thanks to volunteer Tony Thomas, the extension-related preferences were moved into the Appearance tab of Preferences. Progress toward several other improvements was made in April: Sage Ross began implementing an API to generate lists of enrolled student editors from one or more courses, and the Facebook Open Academy students continued their work on new notification features, and also embarked on need-finding research for an improved course activity feed. Mobile <https://www.mediawiki.org/wiki/Wikimedia_Mobile_engineering> *Wikimedia Apps <https://www.mediawiki.org/wiki/Wikimedia_Apps>* The Mobile App team continued moving toward the first market release of the rebooted Wikipedia App for Android and iOS. The team focused on bug fixes, editing refinements, and UX polish. Several issues related to keyboard, navigation bar, edit summary, and abuse filter were fixed. The app now uses the newly created Wikifont which reduced the size of the app and the number of graphical assets. Articles should now look even closer to their mobile web counterparts. Product management switched from Kenan Wang to Maryana Pinchuk due to Kenan’s departure, and the team welcomed Dmitry Brant as an Android software developer. *Mobile web projects <https://www.mediawiki.org/wiki/Mobile_web_projects>* This month, the mobile web team released history and contributions pages, as well as an updated watchlist view, for all users. We also promoted two new features geared toward “humanizing” Wikipedia for readers and new editors: a prominent “last modified” banner that indicates when articles haven’t been edited in a while and may need some attention, and a user profile feature to provide a mobile-friendly snapshot of users’ contributions and activity. For tablets, we updated typography and layout and worked on adding the ability to add and modify links via VisualEditor in beta, in preparation for redirecting tablets to the mobile site later this quarter. *Wikipedia Zero <https://www.mediawiki.org/wiki/Wikipedia_Zero>* <https://www.mediawiki.org/w/index.php?title=File:Wikipedia_Zero_Update_Metr…> <https://commons.wikimedia.org/wiki/File:Wikipedia_Zero_Update_Metrics_May_2…> Presentation slides from the monthly metrics meeting During the last month, the team continued setup tasks on the Partners portal, JSON configuration store, and graceful image quality reduction. The team also updated Android and iOS Wikipedia app reboot visual flourishes for Wikipedia Zero, analyzed anomalous access patterns and proxy-oriented configuration and tech documentation to close gaps, and created bugfixes for unnecessary charge warnings in the “Read in another language” language picker plus direct upload.wikimedia.org image hyperlinks on File: pages. The team also removed some legacy ETL code from the ZeroRatedMobileAccess extension. Yuri did outreach abroad and continued analytics work on SMS/USSD pilot data. The team also generated two custom pageview analyses for an operator to distinguish traffic by high level device access characteristics as part of ongoing discussions. The team also explored legacy Android Wikipedia app trends. Additionally, the team cut Android Wikipedia app alpha builds, worked on User-Agent string and URL format updates for the forthcoming iOS Wikipedia app to ensure pageview logging, and performed app code review. Discussion with the community on MCC-MNC logging to address mobile IP drift was conducted, and it appears it is okay to proceed; the team will reduce the date granularity of log lines to the day (e.g., YYYYMMDD) with a patch to MediaWiki core, though. Routine pre- and post-launch configuration changes were made to support operator zero-rating, and in-depth technical assistance was provided to operators and the partner management team to help add zero-rating and address anomalies. The team emailed further about full-text search in reboots of Wikipedia apps, and may resume investigation of it later. The team also examined requirements for portal and general partners engineering human resources. *Wikipedia Zero (partnerships)* IPKO in Kosovo launched Wikipedia Zero, bringing us to a total of 28 partners in 26 countries. We delivered 68 million free page views in April. Adele Vrana visited South Africa to meet with MTN (current Wikipedia Zero partner), prospective partners, members of Wikimedia South Africa (WMZA) and the Singenjongo High School. This trip was part of a broader strategy to promote Wikipedia in our partners’ corporate social responsibility (CSR) and education initiatives, increasing awareness and impact locally. We are identifying new collaboration opportunities with MTN and local organizations, including the Wikimedia chapter in South Africa and other mission-aligned nonprofits. Additionally, we will continue to support the local initiative created by Sinenjongo High School teachers and students. Language Engineering<https://www.mediawiki.org/wiki/Wikimedia_Language_engineering> *Language tools <https://www.mediawiki.org/wiki/Language_tools>* The team prepared the migration of the translation memory infrastructure from Solr to ElasticSearch. *Milkshake <https://www.mediawiki.org/wiki/Milkshake>* The jquery.webfonts library was adapted for the Typography refresh. An input method for the Batak script was added to jquery.ime, and bugs were fixed in the InScript input method for Hindi, Odia and Gujarati. *Language Engineering Communications and Outreach <https://www.mediawiki.org/wiki/Language_engineering_communications_and_outr…>* The team prepared targeted mini-surveys for readers of Wikipedia in poorly-supported languages, and heldIRC office hours<https://meta.wikimedia.org/wiki/IRC_office_hours/Office_hours_2014-04-09> . *Content translation <https://www.mediawiki.org/wiki/Content_translation>* ContentTranslation was the team’s main effort this month. Source text segmentation was further improved and stabilized. Other developed features include: - A beta feature that shows a red interlanguage link when the article is not translated to the user’s language; - Basic handling of templates and images; - Basic publishing of the translation as a formatted article; - Testing infrastructure for the server. Platform Engineering<https://www.mediawiki.org/wiki/Wikimedia_Platform_Engineering> MediaWiki Core *HHVM <https://www.mediawiki.org/wiki/HHVM>* Work on the Zend plugin compatibility layer is feature complete, and now the team is working on proper packaging of HHVM, and is working toward making HHVM the default PHP implementation on the Beta cluster. *Release & QA <https://www.mediawiki.org/wiki/Wikimedia_Release_and_QA_Team>* <https://commons.wikimedia.org/wiki/File:RelEng_%2B_QA_Quarterly_Reivew_-_20…> <https://commons.wikimedia.org/wiki/File:RelEng_%2B_QA_Quarterly_Reivew_-_20…> Presentation slides of the Release engineering and QA quarterly review. See also the agenda<https://www.mediawiki.org/wiki/Wikimedia_Release_and_QA_Team/Quarterly_revi…> and minutes<https://www.mediawiki.org/wiki/Wikimedia_Release_and_QA_Team/Quarterly_revi…> . The tool that deploys code in production (“scap”) is now used to deploy/update code on the Beta cluster<https://www.mediawiki.org/wiki/Beta_cluster>, removing another difference between the Beta cluster and production along with providing us an environment to safely test changes to our deployment system(s). We converted more scap code to python (*scap-rebuild-cdbs* and *mw-update-l10n*), and moved a ton of Jenkins jobs from the Cloudbees Jenkins to our self-hosted instance; we’re on target to end the use of Cloudbees’ Jenkins in the next two weeks. We also made significant progress on the two open positions (Release Engineer and QA Automation). *Search <https://www.mediawiki.org/wiki/Search>* We deployed Cirrus as a Beta Feature on all wikis that didn’t yet have it. We’re working on deploying a change to how snippets are generated that should be faster and better. We’re also starting to work with Elasticsearch plugins for improved analysis of some languages as well as backup. *Auth systems <https://www.mediawiki.org/wiki/Auth_systems>* We did initial work on Authn/z requirements for RFC architecture, and an initial review of Requests for comment/AuthStack<https://www.mediawiki.org/wiki/Requests_for_comment/AuthStack>. We also investigated the use of MediaWiki’s OAuth for Phabricator, and worked on a proof of concept. *Wikimania Scholarships app <https://www.mediawiki.org/wiki/Wikimania_Scholarships_app>* Several small bug fixes and feature requests were worked on by volunteers applying for GSoC <https://www.mediawiki.org/wiki/GSoC> projects. No operational issues were reported. *Deployment tooling <https://www.mediawiki.org/wiki/Deployment_tooling>* We started investigating changes that may be needed to support the use of HHVM in production. *Security auditing and response <https://www.mediawiki.org/wiki/Security_auditing_and_response>* We helped with the operational response to the Heartbleed vulnerability. Significant work was done on identifying and testing static analysis tools to integrate into the release workflow. We finished reviewing varnishkafka for Analytics, and Compact Personal Bar for UX. MediaWiki releases 1.21.9 and 1.22.6 fixed one security issue. Quality assurance *Quality Assurance <https://www.mediawiki.org/wiki/Quality_Assurance>* This month saw the QA team working closely with the MobileFrontend team to extend and refactor their test suite. We also made great progress in running many of the browser test suites on headless Firefox instances in builds controlled by WMF Jenkins. Work on the WMF Jenkins browser test builds will continue in order to take advantage of the power and flexibility we have there. *Browser testing <https://www.mediawiki.org/wiki/Quality_Assurance/Browser_testing>* The QA team released a number of new browser test features, including the ability to create test data in the target wiki at runtime. This feature was immediately put into use by the MobileFrontend team in their browser test suite. Acomplete list of shared features<https://www.mediawiki.org/wiki/Browser_testing/shared_features> available to any browser tests in any extension repository is available. Multimedia *Multimedia <https://www.mediawiki.org/wiki/Multimedia>* In April, the multimedia team released Media Viewer v0.2<https://www.mediawiki.org/wiki/Multimedia/About_Media_Viewer> on 14 pilot sites, in preparation for a wider deployment<https://www.mediawiki.org/wiki/Multimedia/Media_Viewer/Release_Plan>next month: overall response has been favorable so far, and a growing majority of survey respondents<https://www.mediawiki.org/wiki/Multimedia/Media_Viewer/Survey> are finding this new multimedia browser useful. Gilles Dubuc, Mark Holmquist, Gergő Tisza and Aaron Arcos developed final features for this release, as described on this release’s wall<https://wikimedia.mingle.thoughtworks.com/projects/multimedia/cards?favorit…>, based on designs by Pau Giner. We also developed a set of metrics dashboards<http://multimedia-metrics.wmflabs.org/dashboards/mmv> to track global activity, image load and network performance, as well as local metrics dashboards <https://www.mediawiki.org/wiki/Multimedia/Metrics> for selected sites: first results show a decline in image load time<http://multimedia-metrics.wmflabs.org/dashboards/mmv#overall_network_perfor…>, and suggest that Media Viewer loadsfaster than file description pages<http://multimedia-metrics.wmflabs.org/dashboards/mmv#media_viewer_vs_file_p…>. We invite you to test the latest version<https://www.mediawiki.org/wiki/Lightbox_demo> of Media Viewer (see these testing tips<https://www.mediawiki.org/wiki/Multimedia/About_Media_Viewer#How_can_I_help…>) and share your feedback<https://www.mediawiki.org/wiki/Talk:Multimedia/About_Media_Viewer> . Fabrice Florin led product planning and management, hosting a planning meeting<https://www.mediawiki.org/wiki/Multimedia/Meetings/Next_Cycle_2_Q4_2013-14> for our next development cycle (leading to a wall of tasks<https://wikimedia.mingle.thoughtworks.com/projects/multimedia/cards/grid?ag…>): for the next six weeks, we plan to divide our time between Media Viewer (e.g. serious bugs, basic zoom feature), Technical Debt (e.g. image scalers) and Upload Wizard. Keegan Peterzell and Fabrice announced the gradual release of Media Viewer<https://www.mediawiki.org/wiki/Multimedia/Media_Viewer/Release_Plan> on dozens of wiki sites, starting new discussions in collaboration with our community partners, as well as launching surveys in multiple languages<https://www.mediawiki.org/wiki/Multimedia/Media_Viewer/Survey> to get reader feedback about this tool. For more updates about our multimedia work, we invite you to join the multimedia mailing list<https://lists.wikimedia.org/mailman/listinfo/multimedia> . Engineering Community Team<https://www.mediawiki.org/wiki/Engineering_Community_Team> *Bug management <https://www.mediawiki.org/wiki/Bug_management>* Daniel Zahn and Andre Klapper upgraded Wikimedia Bugzilla<https://bugzilla.wikimedia.org/show_bug.cgi?id=64226> to the latest version 4.4.4. Valhallasw replaced the brittle wikibugs IRC notification bot by pywikibugs <https://github.com/valhallasw/pywikibugs> ( announcement<http://lists.wikimedia.org/pipermail/wikitech-l/2014-April/076140.html>). A bugday <https://www.mediawiki.org/wiki/Bug_management/Triage/20140429> took place updating about 50 MediaWiki General/Unknown tickets. Bugzilla’s “Tools” product was renamed to “Utilities”<https://bugzilla.wikimedia.org/show_bug.cgi?id=53986> to decrease confusion with tools on Tool Labs. Numerous old forgotten “Backport_WMF?” flags on bug reports, older PATCH_TO_REVIEW tickets with all patches merged, and a lot of older WikiEditor tickets were cleaned up. In general, work mostly concentrated on handling the Phabricator RfC<https://www.mediawiki.org/wiki/Requests_for_comment/Phabricator> . *Project management tools review <https://www.mediawiki.org/wiki/Project_management_tools/Review>* On April 14, the Request for comment<https://www.mediawiki.org/wiki/Requests_for_comment/Phabricator> was finalized and a related FAQ<https://www.mediawiki.org/wiki/Requests_for_comment/Phabricator/FAQ> published. The RfC was announced on thewikitech-l<http://lists.wikimedia.org/pipermail/wikitech-l/2014-April/075993.html> , mediawiki-l<http://lists.wikimedia.org/pipermail/mediawiki-l/2014-April/042736.html> and teampractices<http://lists.wikimedia.org/pipermail/teampractices/2014-April/000379.html> mailing lists, asking everybody for feedback on the RfC itself, discussion on its talk page, actively trying out proposed “Phabricator” in a testing instance<http://fab.wmflabs.org/>, and creating tickets in the test instance under the “Wikimedia Phabricator” project for functionality that is missing. *Mentorship programs <https://www.mediawiki.org/wiki/Mentorship_programs>* Sixteen Google Summer of Code<https://www.mediawiki.org/wiki/Google_Summer_of_Code_2014> students and seven FOSS Outreach Program for Women<https://www.mediawiki.org/wiki/FOSS_Outreach_Program_for_Women/Round_8> interns will be busy in the next months working on Wikimedia projects. We got 23 participants in total, two more than a year ago, even if our quality criteria have been more strict this time. *Technical communications <https://www.mediawiki.org/wiki/Technical_communications>* In addition to ongoing communications support<https://www.mediawiki.org/wiki/Technical_communications/Tech_blog_activity> for the engineering staff, Guillaume Paumier<https://www.mediawiki.org/wiki/User:Guillom> organized and archived many team documentation pages<https://www.mediawiki.org/wiki/Template:Engineering_Community_Team> of the Engineering Community Team, like planning pages, reports and meeting notes. He also set up subscription templates on mediawiki.org<https://www.mediawiki.org/wiki/Template:Latest_tech_news> and the English Wikipedia<https://en.wikipedia.org/wiki/Template:Latest_tech_news> that display the latest version of the technical newsletter<https://meta.wikimedia.org/wiki/Tech/News> for users who prefer not to get it delivered on their talk page. *Volunteer coordination and outreach <https://www.mediawiki.org/wiki/Volunteer_coordination_and_outreach>* We restarted the Wikimedia Tech Talks with a light process for scheduling events<https://www.mediawiki.org/wiki/Project:Calendar/How_to_schedule_an_event>, and we help organizing *A preliminary look at Parsoid internals*<https://plus.google.com/u/0/b/103470172168784626509/events/cjo2nmv8jr79k85u…> and *Unit testing for MediaWiki projects*<https://plus.google.com/events/cae6ng1m9o4mhdbpo10u5v05bvg>. The Wikimedia Hackathon in Zürich<https://www.mediawiki.org/wiki/Z%C3%BCrich_Hackathon_2014> is ready to roll on May 9−11, and we co-hosted and info session with Wikimedia Switzerland <https://plus.google.com/events/cj4okkse0n8ealb7mntrc4458a8>, the main organizers of the event. *Architecture and Requests for comment process <https://www.mediawiki.org/wiki/Architecture_process>* We held several Request for Comment review meetings in IRC: - on scoped language converter & square bounding boxes<https://www.mediawiki.org/wiki/Architecture_meetings/RFC_review_2014-04-02> ; - to get quick next actions & validity checks on multiple RfCs<https://www.mediawiki.org/wiki/Architecture_meetings/RFC_review_2014-04-09> ; - on reducing image quality for mobile<https://www.mediawiki.org/wiki/Architecture_meetings/RFC_review_2014-04-16> ; - on associated namespaces<https://www.mediawiki.org/wiki/Architecture_meetings/RFC_review_2014-04-23> ; - on MediaWiki libraries and third-party components<https://www.mediawiki.org/wiki/Architecture_meetings/RFC_review_2014-04-29> . Also, we worked on improving security, architecture, and performance guidelines for developers. We aim to have MediaWiki performance guidelines ready to approve at the Zürich hackathon in mid-May. Analytics <https://www.mediawiki.org/wiki/Analytics> *Wikimetrics <https://www.mediawiki.org/wiki/Analytics/Wikimetrics>* New features have been deployed<http://lists.wikimedia.org/pipermail/wikimetrics/2014-April/000123.html> on Wikimetrics: Scheduled Reports & Public Reports. *Kafka <https://www.mediawiki.org/wiki/Analytics/Logging_infrastructure>* Data from text Varnishes is now being consumed through varnishkafka -> kafka -> camus into hdfs. Kafka now processes Bits, Images and Text data. Kiwix <http://www.kiwix.org/> *The Kiwix project is funded and executed by Wikimedia CH <https://meta.wikimedia.org/wiki/Wikimedia_CH>.* We have finally released a first experimental ZIM file of TED talks. We have packed in a unique 7GB library the 250 talks about business. This includes not only the videos, but also short speaker bios and thousands of subtitles in more than 50 languages. We will fix soon the last critical issues and release other TED ZIM files with talks about entertainment, science, etc. We have also migrated our download server to a better one. Besides providing a better storage system and more bandwidth, it has 9TB of disk space. This was a mandatory step in our ZIM generation industrializing process and therefore necessary to allow us to generate more ZIM files more often. We are also working with an e-reader manufacturer to have Kiwix installed and available on its devices so that theMALeBOOKS<http://malebooks.ml/en> project (eBooks for Mali <https://en.wikipedia.org/wiki/Mali>) can ship e-readers with not only thousands of free eBooks, but also the complete Wikipedia and Wikisource in French. Wikidata <https://meta.wikimedia.org/wiki/Wikidata> <https://www.mediawiki.org/w/index.php?title=File:Wikidata_@_Wikimedia_metri…> <https://commons.wikimedia.org/wiki/File:Wikidata_@_Wikimedia_metrics_and_ac…> Presentation slides on Wikidata from the monthly metrics and activities meeting *The Wikidata project is funded and executed by Wikimedia Deutschland <https://meta.wikimedia.org/wiki/Wikimedia_Deutschland/en>.* The Wikidata team got simple query functionality ready for a first demo at the WMF Metrics and Activities Meeting<https://meta.wikimedia.org/wiki/Metrics_and_activities_meetings/2014-05-01>. The entity suggester a team of students is working on also got finishing touches and should be ready for release soon. Once it is live, it will suggest missing information on an item so it is easier to see what should be added. We also welcomed 2 interns<http://lists.wikimedia.org/pipermail/wikidata-l/2014-April/003753.html> as part of the Outreach Program for Women to help with documentation, social media outreach and mobile app concepts. Wikiquote now manages its language links via Wikidata. Additionally it is now possible to automatically add links<http://lists.wikimedia.org/pipermail/wikidata-l/2014-April/003690.html> to other sister projects in the sidebar of an article using Wikidata. A Wikidata Toolkit<http://lists.wikimedia.org/pipermail/wikidata-l/2014-March/003641.html> was released, as well as a library<http://google-opensource.blogspot.de/2014/04/qlabel-multilingual-content-wi…> that lets you use Wikidata’s labels for translation anywhere on the web. FutureThe engineering management team continues to update the *Deployments <https://wikitech.wikimedia.org/wiki/Deployments>* page weekly, providing up-to-date information on the upcoming deployments to Wikimedia sites, as well as the *annual goals <https://www.mediawiki.org/wiki/Wikimedia_Engineering/2013-14_Goals>*, listing ongoing and future Wikimedia engineering efforts. ------------------------------ *This article was written collaboratively by Wikimedia engineers and managers. See revision history <https://www.mediawiki.org/w/index.php?title=Wikimedia_engineering_report/20…> and associated status pages. A wiki version <https://www.mediawiki.org/wiki/Wikimedia_engineering_report/2014/April> is also available.* -- Guillaume Paumier Technical Communications Manager — Wikimedia Foundation

1 0

Affiliation in username
by Gryllida 14 May '14

14 May '14

On a second thought, do we want to add an optional "affiliation" field to the signup form, so the affiliation goes at the end of username in braces? - DGarry (WMF) - Fred (DesignSolutionsInc) - David (MIT) - ... So the signup form would look like this: ------------------------------------------------------------- | | | [ Username preview in large green font ] | | | | Username: | | ___________________ | | Password: | | ___________________ | | Password 2: | | ___________________ | | Email (optional): | | ___________________ | | Affiliation (optional; if your editing is related to work): | | ___________________ | | | ------------------------------------------------------------- I.e. ------------------------------------------------------------- | | | [ "Gryllida (FOO)" in large green font ] | | | | Username: | | _Gryllida__________ | | Password: | | ___________________ | | Password 2: | | ___________________ | | Email (optional): | | ___________________ | | Affiliation (optional; if your editing is related to work): | | _FOO_______________ | | | ------------------------------------------------------------- Gryllida. On Sun, 23 Feb 2014, at 1:25, rupert THURNER wrote: > hi, > > could wmf please extend the mediawiki software in the following way: > 1. it should knows "groups" > 2. allow users to store an arbitrary number of groups with their profile > 3. allow to select one of the "group"s joined to an edit when saving > 4. add a checkbox "COI" to an edit, meaning "potential conflict of interest" > 5. display and filter edits marked with COI in a different color in history > views > 6. display and filter edits done for a group in a different color in > history views > 7. allow members of a group to receive notifications done on the group page, > or when a group is mentioned in an edit/comment/talk page. > > reason: > currently it is quite cumbersome to participate as an organisation. it is > quite cumbersome for people as well to detect COI edits. the most prominent > examples are employees of the wikimedia foundation, and GLAMs. users tend > to create multiple accounts, and try to create "company accounts". the main > reason for this behaviour are (examples, but of course valid general): > * have a feedback page / notification page for the swiss federal archive > for other users > * make clear that an edit is done private or as wmf employee > > this then would allow the community to create new policies, e.g. the german > community might cease using company accounts, and switch over to this > system. this proposal is purely technical. current policies can still be > applied if people do not need something else, e.g. wmf employees may > continue to use "sue gardner (wmf)" accounts. > > what you think? > > best regards, > rupert > ------------------- > swissGLAMour, http://wikimedia.ch > _______________________________________________ > Wikimedia-l mailing list > Wikimedia-l(a)lists.wikimedia.org > Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, <mailto:wikimedia-l-request@lists.wikimedia.org?subject=unsubscribe>

5 5

Question about the ContentHandler and Extension:Score
by David Cuenca 14 May '14

14 May '14

As a continuation to the community wish expressed last year to start a musical score transcription project [1], I'm investigating what is needed to make it happen. The biggest hurdle seems to be how to separate content from layout. On the one hand users should be able to define which part to work on (a staff, a voice, an instrument, lyrics...), otoh they should be able to select which elements to render for different representations. For instance a single page has all parts but just for one page, a complete score has all the parts for all pages, a piano part has only the piano staffs for all pages, etc. Lilypond can handle different layout representations, but of course the content selection has to be prepared before launching the desired layout rendering. Any ideas about how to deal with this situation? Cheers, Micru [1] https://meta.wikimedia.org/wiki/Requests_for_comment/Musical_score_transcri…

1 0

En.wikipedia redirect loops on everything (??!!)
by George Herbert 14 May '14

14 May '14

I am getting redirect loops on every page I try right now on en.wikipedia -- -george william herbert george.herbert(a)gmail.com

4 5

workflow to add multiple patches to gerrit.wikimedia.org:29418/operations/puppet.git
by Petr Bena 14 May '14

14 May '14

Currently I do: * clone a repo * setup git & hooks # patch 1: * apply my changes * commit * execute git-review # patch 2: * apply my changes * commit FAIL - the new commit it depending on previous commit - I can't push What am I supposed to do in order to push multiple separate patches? GIT-IDIOT way please, no long explanations, just commands and examples. Thanks

6 11

HTML templating systems & MediaWiki - is this summary right?
by Sumana Harihareswara 13 May '14

13 May '14

I'm trying to understand what our current situation is and what our choices are around HTML templating systems and MediaWiki, so I'm gonna note what I think I understand so far in this mail and then would love for people to correct me. TL;DR - did we already consense on a templating system and I just missed it? Description: An HTML templates system (also known as a templating engine) lets you (the programmer) write something that looks more like a document than it looks like code, then has hooks/entry points/macro substitution points (for user input and whatnot) that then invoke code, then emits finished HTML for the browser to render. Examples: PHP itself is kinda a templating language. In the PHP world, Smarty is a somewhat more mature/old-school choice. Mustache.js is a popular modern choice. And in other languages, you'd pick a lot of the MVC frameworks that are popular, e.g. Django or Jinja in Python. Spectrum of approaches: One approach treats HTML as a string ("here's a bunch of bytes to interpolate"). From a security perspective, this is dangerously easy to have vulnerabilities in, because you just naively insert strings. Then on the other end of the spectrum, you have code that always keeps the document object model (DOM) in memory, so the programmer is abstractly manipulating that data model and passing around an object. Sure, it spits out HTML in the end, but inherent in the method for turning those objects into HTML is a sanitization step, so that's inherently more secure. There's some discussion at https://www.mediawiki.org/wiki/Parsoid/Round-trip_testing/Templates . I presume we want the latter, but that the former model is more performant? We talked about this stuff in https://www.mediawiki.org/wiki/Architecture_meetings/RFC_review_2014-02-21 and https://www.mediawiki.org/wiki/Talk:Architecture_Summit_2014/HTML_templatin… . Based on that plus https://www.mediawiki.org/wiki/Architecture_Summit_2014/RFC_clusters#HTML_t… it seems like we are supposed to get consensus on which system(s) to use, and we kind of have four things we could choose: * oojs - https://www.mediawiki.org/wiki/OOjs_UI -- could use this toolkit with one of the template approaches below, or maybe this is enough by itself! Currently used inside VisualEditor and I am not sure whether any other MediaWiki extensions or teams are using it? This is a DOM-based templating system. Template approaches which are competing?: * MVC framework - Wikia has written their own templating library that Wikia uses (Nirvana). Owen Davis is talking about this tomorrow in the RFC review meeting. https://www.mediawiki.org/wiki/Requests_for_comment/MVC_framework * mustache.js stuff - Ryan Kaldari and Chris Steipp mentioned this I think? * Knockout-compatible implementation in Node.js & PHP https://www.mediawiki.org/wiki/Requests_for_comment/HTML_templating_library… and https://www.mediawiki.org/wiki/Requests_for_comment/HTML_templating_library… , being worked on by Gabriel Wicke, Matt Walker, and others. DOM-based. There's also an OutputPage refactor suggested in https://www.mediawiki.org/wiki/Requests_for_comment/OutputPage_refactor that's part of the HTML Templating RFC Cluster https://www.mediawiki.org/wiki/Architecture_Summit_2014/RFC_clusters#HTML_t… . I guess my biggest question right now is whether I have all the big moving parts right in my summary above. Thanks. -- Sumana Harihareswara Senior Technical Writer Wikimedia Foundation

12 37

← Newer
1
...
7
8
9
10
11
12
13
14
Older →

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Wikitech-l May 2014