Wikitech-l January 2019

wikitech-l@lists.wikimedia.org

79 participants
71 discussions

Your Participation Requested - Interview about your local development environment
by Jeena Huneidi 07 Feb '19

07 Feb '19

Hello Everyone, I am working on improving developer productivity! Specifically, I am aiming to improve the local development environment. Satisfaction with the local development environment got the second lowest score in the developer satisfaction survey, and we (Release Engineering) would like to find out which things about it you like, and which are troublesome. In order to do so, we need your help! I’d love to have the opportunity to interview you about your local setup and/or observe how you interact with it. Please send me an email if you’d like to participate, and I or one of my colleagues will schedule some time with you. I’m looking forward to our conversations. This is a chance for you to shape the local development environment, so please don’t hold back :) Outcomes resulting from the interviews (non personally identifiable information) will be made available after analysis has been completed. -- Jeena Huneidi Software Engineer, Release Engineering Wikimedia Foundation

2 4

Sunsetting VP8 version of WebM for video playback
by Brion Vibber 05 Feb '19

05 Feb '19

A few months ago I switched our TimedMediaHandler's config to support the newer, more bandwidth-efficient VP9/Opus variant of WebM and to use these preferably over the older VP8/Vorbis version when creating scaled, playback-ready derivatives. (This does not affect upload support -- you may continue to upload video files in WebM VP9, WebM VP8, or Ogg Theora formats, with either Vorbis or Opus audio.) Conversions of existing files on Commons ran in the background for some weeks, finishing in November. I'm now running a final pass for high-resolution files and any files on other wikis that didn't get a conversion yet, in preparation for removing the VP8 derivatives in the next couple of weeks to free storage space. This should have relatively little visible effect for users, unless someone is relying on the particular derivative files with extensions like ".360p.webm"; the new versions are named like ".360p.vp9.webm". Note that IE 11 users using the "WebM Media Foundation Components for IE <https://www.webmproject.org/ie/>" will not be able to play back the new VP9/Opus files natively, as this driver has never been updated for VP9 or Opus. IE 11 users will receive low-resolution, slow JavaScript-based video playback instead. If you find this is troublesome, the recommended solution is to switch to any other browser. Third-party MediaWiki + TimedMediaHandler users should be aware that the defaults are changing, and in future the VP8-specific support and code paths may be removed. TimedMediaHandler will probably change a lot in the coming months with upcoming WebVTT subtitle format, a streamlined videojs-based player, and hopefully more! -- brion

2 2

The mw.ext construct in lua modules
by John Erling Blad 05 Feb '19

05 Feb '19

At the Extension:Scribunto/Lua reference manual, at several places,[1] it is pointed out that the lua-libs should use the form 'mw.ext.NAME'. This creates visual noise in the code. Any lib included should have a extension page, thus it has already been given an unique name. In addition, only the libraries that need to be preloaded are added to the mw-structure, and those are the extensions. The ext-addition is like saying "this is an extension and it is only extensions that needs to be added to the mw-struct so we make it abundantly clear that this is an extension". The only cases where a name can collide is if some external lib is included, that external lib has the same name as an extension, and if someone in addition preloads the external lib. The chance is quite frankly pretty slim, as there are rather few external libs that makes sense to preload in this environment, especially as preloading imply some kind of interaction with the environment. That means it is an extension. I guess I'm stepping on some toes here… So to make it abundantly clear, not 'mw.ext.NAME' (or 'mw.ext.NaMe', or 'mw.ext.name') but 'mw.name' (lowercase, not camelcase). If the call is a constructor or some kind of builder interface, then 'mw.name(…)' is totally valid. I do not believe it is wise to turn the lib into an instance by the call, but it can return an instance, it can cache previously returned instances, and it can somehow install the instance(s) in the current environment. An extension should have any pure root libs at 'pure/name.lua' and additional libs at 'pure/name/additional.lua', where 'pure' is resolved in the 'ScribuntoExternalLibraries' hook. [1] https://www.mediawiki.org/wiki/Extension:Scribunto/Lua_reference_manual#Ext…

6 13

Re: [Wikitech-l] Changes to the Block paradigm
by 80hnhtv4agou＠bk.ru 04 Feb '19

04 Feb '19

what about abuse by administrators and stewards ? From: Moriel Schottlender Sent: Monday, January 7, 2019 7:53 PM To: Wikimedia developers ; Wikitech-ambassadors(a)lists.wikimedia.org Subject: [Wikitech-l] Changes to the Block paradigm Greetings, Have you written code that deals with or depends on user blocks? Read on. = TL;DR: = The new “Partial Blocks” feature has fundamentally changed the way MediaWiki considers what “block” means; any code that handles blocks should consider whether the questions it is asking are still valid or should adjust its expectations. Please read for more details. = Preamble = A couple of months ago, as part of the Anti Harassment Tools team’s continued work on improving the general experience of our users and providing more robust tools to administrators, an RFC to enable “Partial Blocks[1]” has passed and has been implemented in MediaWiki, affecting the way blocking users operates. While the actual feature, enabling the blocking of users for specific pages and namespaces, will be slowly rolled out as part of our rollout and testing plans, the change has resulted in a complete change of paradigm for what “block” means throughout our code. = Change of paradigm = Until recently, Block was fairly straight forward; whether a block was done on an IP range or a specific user, the question the code would ask is “is the user blocked from this action” and the answer will be a boolean yes or no, depending on whether the user was blocked from the wiki and whether the action was a blockable action. With the new Partial Blocks feature, that question is now more elaborate. We are giving admins and wikis in general much more robust options when deciding to block IPs or users. “Sitewide” block is no longer the only option. Now, a user can be blocked from editing a specific page, and soon from a specific namespace. There are also blocks that prevent a specific action, such as uploading files or creating new pages. This means that the question “is the user blocked” is no longer accurate. In most cases, the question should be “is the user blocked from the action on this page”, because users may receive a block that is not sitewide, but from a specific page or set of pages. = What we worked on = The Anti Harassment Tools team has been working diligently on making sure that the new blocking behavior does not produce obvious regressions in production, and does not add to any still existing inconsistencies. In cases where we identified a clear mismatch, we’ve tried to either fix it outright or alert the code owners to adjust. If we have missed any iteration or use-case, please open a Phabricator ticket and add the ‘anti-harassment’[2] tag to it. If the use-case is sensitive or identifies a current loop-hole in the way blocks work, please make it a security ticket and alert us and the relevant team immediately. = General steps forward = While the team is following up on making the code clear and robust and fixing what we’ve identified as paradigm-changes to deal with, there are still many instances where the changes required to the code are not straight-forward or clear. Some extensions ask whether a user is blocked and may need to change the way that the product’s “business logic” behaves. These are cases where we cannot make the decision for the codebase. We encourage you to look at your product and consider adjusting if necessary. = General guidance = We’ve identified some areas that may help code owners adjust their products to properly take advantage of the new feature and adjust to the new paradigm change. This list is not exhaustive, but may help you spot areas in your code that can easily be changed: * User::isBlocked() has changed its meaning, and its use cases should be re-examined depending on what your code intends to check. For the most part, if there’s a Title object available, User::isBlockedFrom() is a good option. Otherwise, consider using User::getBlock() and Block::isSitewide() * Block::prevents( ‘edit’ ) is an operation that doesn’t make sense anymore, because a block on the ‘edit’ action now depends on context (the title). * Determining whether a block prevents a user from editing their own user talk page has changed. For a sitewide block, if the $wgBlockAllowsUTEdit config is false, then the block prevents the user editing their user talk page, but if it is true, then whether they can edit their user talk page depends on the ipb_allow_usertalk flag on the block. For a partial block to a page or pages, these flags are not taken into account: if the user’s talk page is specified as a blocked page, then they cannot edit their user talk page; if it is not, then they can edit it. Block::prevents( ‘editownuserpage’ ) should therefore not be checked for a partial page block[3]. We plan to deprecate that parameter officially, please consider if this affects your code. * Please check that any classes that extend Action or FormSpecialPage return the correct value for requiresUnblock(). To summarize, in general, the meaning of asking whether a block exists has changed, and any code piece that needs this information should adjust itself to account for partial blocks, depending on its goals. Thank you, Moriel, on behalf of the Anti Harassment Tools Team References: [1] Partial blocks: https://meta.wikimedia.org/wiki/Community_health_initiative/Per-user_page,_… [2] Anti Harassment tools board: https://phabricator.wikimedia.org/project/view/2660/ [3] For a discussion of this, see: https://phabricator.wikimedia.org/T210475 -- Moriel Schottlender (she/her) Senior Software Engineer, Tech Lead Community Tech and Anti Harassment Tools @mooeypoo (IRC / Phabricator) _______________________________________________ Wikitech-l mailing list Wikitech-l(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l

2 1

IE 6/7 MIME type sniffing checks on uploads - is it time to retire them?
by Brion Vibber 01 Feb '19

01 Feb '19

There's been some comments on some old tasks such as T27707 <https://phabricator.wikimedia.org/T27707> about problems with uploading files that include text metadata that looks like HTML elements. Years ago, we added security checks for IE 5/6/7 to work around IE's mime type sniffing: if you went to view a .png file directly in IE (as opposed to in an <img>) the browser would check the first few bytes of the file to detect its type, overriding the HTTP Content-Type header. HTML would be detected with a higher priority than the actual image formats, making it possible to create an actual .png image which when viewed as an image looked like an image, but when viewed as a web page was interpreted as HTML, including any embedded JavaScript. (This was defense in depth in addition to hosting files on a separate domain; among other things, we sometimes serve files out from the main domain when dealing with archived (deleted) versions, and third-party installs are not guaranteed to have a second domain.) Browsers have moved on, but the code remains and it trips up legitimate files containing links in metadata, or sometimes just random compressed data that looks like an element! I've done a quick research check on feasibility: * IE 6 and earlier can no longer access Wikimedia sites due to lack of SNI and TLS 1.0 or later * IE 7 on Windows XP can no longer access Wikimedia sites due to lack of SNI * IE 7 on Windows Vista **can** access Wikimedia sites. * IE 8 and higher support X-Content-Options: nosniff to disable sniffing, which we already use on all MediaWiki requests. At some point Microsoft dropped the sniffing, but I'm not sure if it was a later IE version or an Edge version. No other browsers in reasonably current versions seem to have this problem. So the only remaining browser version that might be affected is IE 7 on Windows Vista, which supports SNI and TLS 1.0. It might or might not still work once we drop TLS 1.0 some time in the future. (Per our TLS dashboard <https://grafana.wikimedia.org/d/000000458/tls-ciphersuite-explorer?orgId=1> about 1.2% of our connections still use TLS 1.0, but this isn't broken down between logged-in-user views and anon views.) Open questions: * Should we drop the anti-sniff checks on upload? * If we do, should we forbid logins with IE 7, or something else to protect the occasional IE 7 logged-in user from a hypothetical targeted drive-by attack? (Is it actually worth doing work and testing it for this?) * Should we add X-Content-Options: nosniff on files served from upload.wikimedia.org too? -- brion

2 2

PHP 7 is now a beta feature
by Giuseppe Lavagetto 31 Jan '19

31 Jan '19

Hi all, as some of you might know, HHVM has decided some time ago to drop support for PHP, choosing to only support Hack (Facebook's own PHP-derivative language)[1]. This forced us to consider alternatives. In particular the last major upgrade to PHP, PHP 7, was supposed to have greatly improved the performance of the runtime, guaranteeing performance on par with HHVM. Given that early tests[2] showed promising performance, we decided to work on PHP7 support and on its rollout in production. I'm happy to announce that PHP 7 is now available as a beta feature on all wikis, and I encourage everyone to try it out and report bugs using the #php7.2-support tag. After this period of beta testing, we will proceed with a progressive rollout to a growing percentage of users, and hopefully we'll complete the transition in the next four months. A huge thank you to all the people who worked hard to reach this goal! Thanks, Giuseppe [1] https://hhvm.com/blog/2017/09/18/the-future-of-hhvm.html [2] https://lists.wikimedia.org/pipermail/wikitech-l/2017-September/088854.html -- Giuseppe Lavagetto Principal Site Reliability Engineer, Wikimedia Foundation

5 5

Wednesday: Technical Advice IRC Meeting
by Michael Schönitzer 29 Jan '19

29 Jan '19

Reminder: Technical Advice IRC meeting **today (Wednesday) 4-5 pm UTC** on #wikimedia-tech. Question can be asked in English, German & Persian. The Technical Advice IRC Meeting is a weekly support event for volunteer developers. Every Wednesday, two full-time developers are available to help you with all your questions about Mediawiki, gadgets, tools and more! This can be anything from "how to get started" over "who would be the best contact for X" to specific questions on your project. If you know already what you would like to discuss or ask, please add your topic to the next meeting: https://www.mediawiki.org/wiki/Technical_Advice_IRC_Meeting Hope to see you there! Michi (for the Technical Advice IRC Meeting crew) -- Michael F. Schönitzer Wikimedia Deutschland e. V. | Tempelhofer Ufer 23-24 | 10963 Berlin Tel. (030) 219 158 26-0 https://wikimedia.de Unsere Vision ist eine Welt, in der alle Menschen am Wissens der Menschheit teilhaben, es nutzen und mehren können. Helfen Sie uns dabei! https://spenden.wikimedia.de Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e. V. Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/029/42207.

1 0

Request for comment: Themes in core
by Isarra Yos 28 Jan '19

28 Jan '19

For those who weren't aware, Jack Phoenix and I have drafted a Request for Comment[1] and related initial patch[2] proposing to merge the logic from the Theme extension[3] into MediaWiki core. From the RfC: This will provide built-in functionality that skins can implement in a simple and consistent manner, enabling user- or site-selection of css variations of a given skin, such as a dark version or a layout with more colours. The RfC itself covers in detail the distinction between skins and themes as well as several use cases and issues surrounding the current situation, but the key point here is thus: with this functionality, we will be able to not just more easily implement night[4], winter, and accessibility[5] modes in skins such as Vector and Timeless, but also much more consistently, and in a manner that can then be developed to better and more consistently use the variables defined by the skins even outside of the themes themselves. Currently, the Theme functionality is limited to the extension itself and a few skins that replicate key parts with varying amounts of fidelity, which not only results in a bit of a mess in terms of code fragmentation, but also limits what we can do with the skins themselves due to the added dependency. We would like to fix this, and thus invite everyone interested to review our proposal and please, comment if you have concerns or see any issues. -I See: [1] https://www.mediawiki.org/wiki/Requests_for_comment/Themes_in_core [2] https://gerrit.wikimedia.org/r/#/c/465451/ [3] https://www.mediawiki.org/wiki/Extension:Theme [4] https://meta.wikimedia.org/wiki/Community_Wishlist_Survey_2019/Reading/Nigh… [5] https://meta.wikimedia.org/wiki/Community_Wishlist_Survey_2019/Reading/Acce…

1 0

Rough Project idea (suggestion)
by K. Kaushik Reddy 28 Jan '19

28 Jan '19

Hi developers, This is Kaushik Reddy. I am currently working on few data science tools, naming few pandas, seaborn, Matplotlib,NumPy etc with python. Can I propose projects for wikimedia dealing with data science? If you people agree to have projects in data science, I would like to propose few within 3 days. Will be waiting for your response. Regards, Kaushik.

2 2

My views on the proposed projects for GSoC '19
by K. Kaushik Reddy 27 Jan '19

27 Jan '19

Hi people, This is Kaushik again. I have gone through the proposed GSoC'19 project ideas, they are all good. https://m.mediawiki.org/wiki/Google_Summer_of_Code/2019 But , I am looking for something which can be done with python. I also think the above projects can be done with python befriended with java script.( Just suggesting, no offense). I am really optimistic that wikimedia would propose some projects dealing with Data science or python for GSoC'19. I actually have many ideas regarding the above mentioned topics. I would bring them in a good form and would like to propose before you people soon. with regards, Kaushik.

4 3

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Wikitech-l January 2019