Wikitech-l April 2017

wikitech-l@lists.wikimedia.org

81 participants
82 discussions

SyntaxHighlight_GeSHi in latest tarball does not include security fixes
by Brian Wolff 29 Apr '17

29 Apr '17

Hello everyone. Unfortunately there's been a mistake in generating the 1.28.1 and 1.27.2 tarball releases, where the wrong version of SyntaxHighlight_GeSHi extension was included. The version of this extension that was included has publically known severe security issues in it. Until such a time as a new release is issued (which will hopefully be soon) I reccomend that people either disable that extension or download a new version from https://www.mediawiki.org/wiki/Special:ExtensionDistributor/SyntaxHighlight… . The version of this extension in git, or from extension distributor is fine. Only the version in the most recent tarball is wrong. The rest of the release is also fine. The 1.23.16 and 1.23.17 release was not affected by this issue. Sorry for the confusion, -- Brian

1 0

Proposal to deprecate parsing and diffs in the action API action=query&prop=revisions
by Brad Jorsch (Anomie) 28 Apr '17

28 Apr '17

TL;DR: I'm proposing deprecating a bunch of parameters. See https://phabricator.wikimedia.org/T164106. In the action API, there are two ways to parse a page/revision: using action=parse, or using the rvparse parameter to action=query&prop=revisions. Similarly, there are two ways to get a diff: using action=compare, or using parameters such as rvdiffto to action=query&prop=revisions. And then there's action=expandtemplates versus the rvexpandtemplates parameter to prop=revisions. This is a somewhat annoying bit of code duplication. Further, the prop=revisions versions of these features have somewhat strange behavior. rvparse forces rvlimit=1. rvdiffto and related parameters will sometimes output "notcached" with no way to directly handle the situation. So, I propose deprecating all of these parameters. The parameters that would be deprecated are the 'rvdifftotext', 'rvdifftotextpst', 'rvdiffto', 'rvexpandtemplates', 'rvgeneratexml', 'rvparse', and 'rvprop=parsetree' parameters to prop=revisions, and the similarly named parameters to prop=deletedrevisions, list=allrevisions, and list=alldeletedrevisions. Following the normal action API deprecation policy, they'd output warnings but would continue to function until usage drops sufficiently or until it becomes too much trouble to fix them, and they wouldn't receive new feature development. If anyone would object to this plan, please reply at https://phabricator.wikimedia.org/T164106, or here if you really hate Phabricator. If there aren't major objections, I'll probably do the deprecation in the next week or two. Thanks. -- Brad Jorsch (Anomie) Senior Software Engineer Wikimedia Foundation

1 0

ArchCom Radar, 2017-04-26
by Daniel Kinzler 28 Apr '17

28 Apr '17

Hi all! Here are the minutes from this week's ArchCom meeting. You can also find the minutes at <https://www.mediawiki.org/wiki/Architecture_committee/2017-04-26>. See also the ArchCom status page at <https://www.mediawiki.org/wiki/Architecture_committee/Status> and the RFC board <https://phabricator.wikimedia.org/tag/mediawiki-rfcs/>. Here are the minutes, for your convenience: * “Canonical data URLs” has been approved: https://phabricator.wikimedia.org/T161527 * “Deprecate unserialize” goes to final call: https://phabricator.wikimedia.org/T161647 * “Long revision comments” to be discussed next Wednesday (May 3), pending confirmation from Kaldari: https://phabricator.wikimedia.org/T153333 Active RFCs: * JS packages / dependency management: some work on implementation https://phabricator.wikimedia.org/T107561 * Multi-file resources: some discussion https://phabricator.wikimedia.org/T133462 Also, discussion started about major overhaul of the the database schema, particularly the revision table. See https://gerrit.wikimedia.org/r/#/c/350097/ and https://phabricator.wikimedia.org/T161671 and https://www.mediawiki.org/wiki/User:Brion_VIBBER/Compacting_the_revision_ta… for context. -- Daniel Kinzler Principal Platform Engineer Wikimedia Deutschland Gesellschaft zur Förderung Freien Wissens e.V.

1 0

Final Call for comments: Deprecate using PHP serialization inside MediaWiki
by Daniel Kinzler 28 Apr '17

28 Apr '17

This is a final call for comments on <https://phabricator.wikimedia.org/T161647>, "Deprecate using PHP serialization inside MediaWiki". This RFC has seen quite a bit of discussion on Phabricator, but has not been discussed in an IRC meeting. ArchCom feels that the discussion was already quite fruitful, and only minor issues are left to resolve. It was agreed that the RFC be put on Final Call: if no new concerns are raised and remain open by May 10, the RFC will be approved for implementation. The RFC in full: Problem statement PHP unserialize() and serialize() can execute code when given malicious input. In most cases this serialization format is unnecessary. As a hardening measure against making a mistake that could result in remote code execution, we should avoid this format. Proposed guideline This RFC proposes the following: * New code SHOULD use JSON instead of PHP serialization whenever possible. * Serialization of primitive values and key-value structures MUST never use PHP serialization. * Any edge cases that require use of serialize or unserialize complicated classes, MUST protect the serialized blob with HMAC (e.g. keyed to $wgSecretKey) to protect against malicious modifications of the blob. In addition to the new guideline for new code, this RFC proposes that we start to (slowly) convert existing uses of PHP serialization. Most likely by using JSON. The eventual goal being to remove all legacy uses of php unserialize() Good first candidates for conversion: * LocalisationCache * MediaHandler metadata. The php serialization output format of the API is outside of the scope of this RFC, since we never ingest it. -- Daniel Kinzler Senior Software Developer Wikimedia Deutschland Gesellschaft zur Förderung Freien Wissens e.V.

1 0

Final Call for comments: RFC Canonical data URLs for machine readable page content
by Daniel Kinzler 28 Apr '17

28 Apr '17

This is a final call for comments on <https://phabricator.wikimedia.org/T161527>, "Canonical data URLs for machine readable page content". This RFC was discussed in a public RFC meeting on the wikimedia-office channel on April 12. It was agreed that the RDF be put on Final Call: if no new pertinent concerns are raised by April 26th, the RFC will be approved for implementation. Meeting log: <https://tools.wmflabs.org/meetbot/wikimedia-office/2017/wikimedia-office.20…> The RFC has been discussed before, and proöposed for final call, but some new concerns where indeed raised, which were then addressed in the above discussion. Log of the original meeting: <https://tools.wmflabs.org/meetbot/wikimedia-office/2017/wikimedia-office.20…> The RDF proposes: * Use URLs of the form https://commons.wikimedia.org/data/main/Data:Avignon_City_Wall.map to identify and retrieve machine readable page content. "main" refers to the main slot, see T107595. * The /data/<slot> path is rewritten to a special page, Special:PageData * Special Special:PageData will redirect (with status 303) to an appropriate (and typically cacheable) URL for retrieving the page data. For now, this will use the action=raw interface. * Special:PageData may apply content negotiation based on the Accept header sent by the client. In the first iteration, it will only check if any accept header sent by the client is compatible with the content model of the requested page. * The 303 redirects are not cecheable for now, because they depend on the Accept header; complex normalization would be needed to allow the cache to vary on the Accept header without causing massive cache fragementation. Please see the phabricator ticket for a summary of the concerns raised and addressed. -- Daniel Kinzler Senior Software Developer Wikimedia Deutschland Gesellschaft zur Förderung Freien Wissens e.V.

1 2

MediaWiki Language Extension Bundle 2017.04 release
by Kartik Mistry 28 Apr '17

28 Apr '17

Hello all, I would like to announce the release of MediaWiki Language Extension Bundle 2017.04. This bundle is The bundle is compatible with MediaWiki 1.27 and 1.28 or above and requires PHP 5.5.9 or above. Next MLEB is expected to be released in 3 months. If there are major changes or important bug fixes, we will do intermediate release. Please give us your feedback at [[Talk:MLEB|https://www.mediawiki.org/wiki/Talk:MLEB]]. * Download: https://translatewiki.net/mleb/MediaWikiLanguageExtensionBundle-2017.04.tar… * sha256sum: 4207398d7ed3ea9c793f35f88e75f29672b33aee4d0baf01afc869517c629c65 Quick links: * Installation instructions are at: https://www.mediawiki.org/wiki/MLEB * Announcements of new releases will be posted to a mailing list: https://lists.wikimedia.org/mailman/listinfo/mediawiki-i18n * Report bugs to: https://phabricator.wikimedia.org/ * Talk with us at: #mediawiki-i18n @ Freenode Release notes for each extension are below. -- Kartik Mistry == Highlights and upgrade notes == == Babel == * TTO fixed disappearing language names on Category: and File: pages. == CLDR == * Reedy updated to CLDR 31. == CleanChanges, LocalisationUpdate == * Localisation updates only. == Translate == * Bartosz Dziewoński fixed "Mark as reviewed" tooltip positioning. * David Causse added support for Elastic5, Multi-DC for TTMServices, freezing writes. * Federico Leva fixed import of the new translations in Special:ImportTranslations. * Geoffrey Mon added insertables suggester class for $1, $2 and allow multiple suggesters. * MarcoAurelio modified log message to match with all other logs listed in Special:Log. * Niklas Laxström added support to allow moving all pages for translation admins (translate-manage). * TTO updated translator-stats to account for expiring user groups. * datguy added confirmation box in Special:ManageTranslatorSandbox when approving or rejecting a translator. (T60706) == UniversalLanguageSelector == * Bartosz Dziewoński fixed language change tooltip position. (T161203) * Niklas Laxström added link to Compact Language Links info page in the beta feature description. * Niklas Laxström fixed broken site picks feature for Compact Language Links. (T157344) === Fonts === * Kartik Mistry added Sundanese (su) font. (T162221) === Input methods === * Amir Aharoni fixed name of Arabic keyboard. * Amir Aharoni and Felix Nartey added keyboard for Ga language. * Amir Aharoni added Akan keyboard also for Twi language. * fgaim added new IMEs for Eritrean languages: Tigrinya (ti), Tigre (tig), Blin (byn). * WikiDiscoverer fixed autonym for Gõychi Konknni (gom) language. -- Kartik Mistry/કાર્તિક મિસ્ત્રી | IRC: kart_ {kartikm, 0x1f1f}.wordpress.com

1 0

Sitelink removal in Wikidata
by Amir Ladsgroup 28 Apr '17

28 Apr '17

Hey, One common form of vandalism in Wikidata is removing sitelinks (we already have an abuse filter flagging them). One of my friends in Persian Wikipedia (who is not a wikidata editor and only cares about Persian Wikipedia) asked me to write a tool that lists all Persian Wikipedia sitelink removals. So I wrote something small and fast but it's usable for any wiki. For example English Wikipedia: http://tools.wmflabs.org/dexbot/tools/deleted_sitelinks.php?wiki=enwiki It's slow due to nature of the database query but once it responds, you might find good things to revert. Since this is the most useful for Wikipedia editors who don't want to patrol Wikidata (in that case, this query <https://www.wikidata.org/w/index.php?namespace=&tagfilter=new+editor+removi…> is the most useful) I'm reaching to wider audiences. Sorry for spamming. HTH Best

5 12

Community Tech team seeking back-end developer for anti-harassment tools
by Ryan Kaldari 27 Apr '17

27 Apr '17

Sorry for the spam, but the Community Tech team at the WMF is currently looking for an experienced back-end developer for the anti-harassment tools team (a sub-team of Community Tech). To read about the sort of work that team will be doing, see https://meta.wikimedia.org/wiki/Community_health_initiative#Anti-Harassment…. We would really, really love to find a developer who is already familiar with Wikipedia and the workflows of the community. If this sounds interesting to you, please submit a resume and cover letter at https://boards.greenhouse.io/wikimedia/jobs/546212?gh_src=lis07f1#.WQKV9cm1… .

1 0

1.29 branched, master is now 1.30
by Chad 27 Apr '17

27 Apr '17

Hi, What it says in the subject :) I've made the REL1_29 branch from master (as of 273eea7357c546d1a1aafd73320e10e8ddac79eb) and have bumped the version to 1.29.0-rc.0 (from alpha). All extensions, skins and vendor have been branched as well. I've just landed the change to master that bumps it to 1.30.0-rc.0. Let the backports begin! (I want to be generous this time too, so bring 'em at me) -Chad

1 0

2017-04-26 Scrum of Scrums meeting notes
by Grace Gellerman 26 Apr '17

26 Apr '17

https://www.mediawiki.org/wiki/Scrum_of_scrums/2017-04-26 *= 2017-04-26 = * contact: https://www.mediawiki.org/wiki/Wikimedia_Engineering == Call outs:== * Datacenter switch back Wednesday, May 3rd 2017 14:00 UTC (user visible, requires read-only mode) ** https://wikitech.wikimedia.org/wiki/Switch_Datacenter * RelEng/Ops: Reading Web needs your help! Config is being ignored and shipping feature that communities have explicitly asked to be disabled and we don't understand why. https://phabricator.wikimedia.org/T163114 == Product == === Reading === ==== Web ==== * Mostly bug fixing and improving code quality for pending Page previews launch * Need help from RelEng/Ops to address `Regression: Fix config to disable related pages where it's not wanted( https://phabricator.wikimedia.org/T163114)`. Config is being ignored and shipping feature that communities have explicitly asked to be disabled and we don't understand why. ==== iOS ==== * Last Week ** Continued work on 5.4.1 - https://phabricator.wikimedia.org/project/view/2600/ *** Regression testing, new public beta *** Crash fixes & performance enhancements ** 5.5 - https://phabricator.wikimedia.org/project/view/2602/ *** Places (UserTesting feedback) *** Article footer content rendered in HTML/CSS rather than native views * This Week ** Submit 5.4.1 to the App Store ** Continue work on 5.5 *** Updates to Places from user feedback ==== Android ==== * Beta released last week containing Wikidata title description editing expanded to many more languages, as well as various offline UX improvements ** Hotfix release Friday (4/21), promoted to prod yesterday: https://lists.wikimedia.org/pipermail/mobile-l/2017-April/010503.html * Planning is underway for implementing offline ZIM compilations (Q4 goal) ** https://phabricator.wikimedia.org/project/view/2723/ * Continuing work on cross-platform consolidation of CSS & JS * Current release board: https://phabricator.wikimedia.org/project/view/2352/ ==== Reading Infrastructure ==== * ORES: working on api.php abuse + DB size issues, hoping to reenable after data center switchback * librarized https://www.mediawiki.org/wiki/Testing-access-wrapper * MCS: stop annoucement of past survey, fix handling of links in section titles === Editing === ==== UI Standardization ==== * This week: ** Continued work to provide WikimediaUI Base variables in core https://phabricator.wikimedia.org/T123359 * Updates: ** OOjs UI: *** Release of v0.21.2 with 11 UI/a11y improvements https://phabricator.wikimedia.org/diffusion/GOJU/browse/master/History.md – among those: **** MediaWiki theme: Full WCAG level AA contrast support of widgets accomplished **** MediaWiki theme: Fix IE 7 oversized buttons **** MediaWiki theme: Improve SearchWidget design **** Set ARIA `role=combobox` on DropdownWidget and LookupElement too (Bartosz Dziewoński) **** Set `aria-owns` for everything with a dropdown list (ARIA `role=combobox`) (Bartosz Dziewoński) ==== Parsing ==== * Parsoid: Audio / video support in place -- we plan to deploy today. * Linter: Improved documentation and guidance for fixing linter errors @ https://www.mediawiki.org/wiki/Help:Extension:Linter * Tidy replacement: Updated documentation @ https://www.mediawiki.org/wiki/Parsing/Replacing_Tidy/FAQ * Language Variant tweaks in preprocessor: Fixups required documented @ https://www.mediawiki.org/wiki/Parsoid/Language_conversion/Preprocessor_fix… ... an editor has been diligently fixing them. We plan to make an announcement about this upcoming change soon and then merge the core patch ( https://gerrit.wikimedia.org/r/#/c/333997/ ) after that. ==== Language ==== * CX reenabled; We're watching logs and ready to disable if needed. Incident report in progress. ** https://phabricator.wikimedia.org/T163344 * OOjs UI migration work in progress. ==== Collaboration ==== * RCFilters: Optimization so if we know a query will return 0 results, we won't do the query at all. Some of these no-result queries have extremely poor performance. * Working on GuidedTour to make people aware of RCFilters beta feature * Working on next generation of RCFilters, including namespace and user filters, saved settings, and more sophisticated time filtering. * DId a deploy Monday to enable RCFilters on English Wikipedia, plus almost all non-ORES wikis. Some issues with preferences, but these have been partly tracked down. * Other bug fixes === Community Tech === No blockers * Rollling out Cookie Blocking to all wikis next Monday * Further polish work on CodeMirror extension (syntax highlighting) * Getting community feedback on LoginNotify extension (currently on Beta Cluster for testing) * User rights expiration is live on all wikis == Technology == === Research === * Reader research surveys are most likely to go out next week ** We will be running these surveys in 14 languages with the help of Reading team ** https://meta.wikimedia.org/wiki/Research_talk:Characterizing_Wikipedia_Read… ** https://phabricator.wikimedia.org/T151835 == Wikidata == * Focusing on the Lexeme extensions UI, example: http://wikidata-lexeme.wmflabs.org/index.php/Lexeme:L2 * Had to work around a change in core that blocked undeleting Wikidata entities: https://phabricator.wikimedia.org/T163144 * Going to deploy Echo notifications when linking pages via Wikidata: https://phabricator.wikimedia.org/T110604 == German Technical Wishlist == * Planning next steps for the book referencing wish: https://phabricator.wikimedia.org/T151301 === Discovery === * No blockers * Building infrastructure for machine learning assisted ranking (aka MjoLniR) * Chinese analyzer seems to be doing well, enabling soon. Working on Hebrew analyzer. * Completed analysis of second sister wiki search A/B test: https://commons.wikimedia.org/wiki/File%3ASecond_Test_Of_Cross-wiki_Search_… * Published notes from discussion on scoring functions: https://www.mediawiki.org/wiki/User:TJones_(WMF)/Notes/Some_Thoughts_on_the… * Updated WDQS dashboard to include traffic from all SPARQL endpoints: https://discovery.wmflabs.org/wdqs/ * Updated the external search dashboard to display non-bot traffic ( https://discovery.wmflabs.org/external/, task T161932) * Portal statistics (task T128546) and translations (task T142582) were updated * Working on Wikidata search improvement * Working on Mediawiki API integration for WDQS === Analytics === * Ongoing work on EventLogging analysis support in Hadoop - Not yet finished * Ongoing work on Wikistats 2.0 data back-end - Finalizing Design * Started to define webrequest tagging project * Daily uniques are in Pivot http://bit.ly/2oZU5gt * Waiting for feedback on Wikistats 2.0 consultation * Dashiki configuration articles on meta still broken, can't fix them until the codfw-related deployment moratorium is over === TechOps === * '''Blocked''' ** None * '''Blocking''' ** None? * '''Updates''' ** Wrapping up switchover action items https://etherpad.wikimedia.org/p/codfw-switchover-AprMay2017 ** ToolsProxy incident report https://wikitech.wikimedia.org/wiki/Incident_documentation/20170419-ToolsPr… === Security === * Reviews ** WikibaseMediaInfo ** TemplateStyles === Services === * Blockers: none * Updates: ** Service-runner doesn't support node 0.1x any more *** https://github.com/wikimedia/service-runner/pull/163 === RelEng === * '''Blocked''' ** None * '''Blocking''' ** None? ** Config symlinks should be touched when they're deployed, Reading callout above https://phabricator.wikimedia.org/T126306 * '''Updates''' ** New version of scap out https://github.com/wikimedia/scap/blob/release/debian/changelog#L1 (config diffs in a basic format, env announce in IRC) ** 1.29 is coming...see wikitech email for more info === Fundraising Tech === * More Paypal Express Checkout fixes * Investigating potential extra session creation on paymentswiki * Planning Ingenico integration changeover, which will include moving a lot of functionality from MW extension to lib * Coordinating with Comms to update the WMF logo in various places: https://phabricator.wikimedia.org/T144254 * CentralNotice: Banner sequence feature is in code review https://phabricator.wikimedia.org/T144453 * CiviCRM: getting rid of the rest of our local core hacks, using upstream buildkit in CI * Found a dozen repos we can delete

1 0

← Newer
1
2
3
4
5
6
7
8
9
Older →

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Wikitech-l April 2017