Wikitech-l September 2017

wikitech-l@lists.wikimedia.org

90 participants
76 discussions

Special Interest Group (SIG) guidance, and the Wiki Replicas SIG
by Nick Wilson (Quiddity) 14 Sep '17

14 Sep '17

Hi all. TL;DR: We've researched and compiled some guidance around creating Special Interest Groups (SIGs), and formed a Wiki Replicas SIG. These are adaptable best-practices for creating a very small task-focused group. Emphasis on adaptable for individual use-cases. Details: Following some discussions at events, on talkpages, and in phabricator,[1] we have compiled the ideas, researched external groups, discussed how to best adapt them for our own diverse use-cases, and placed all the material on mediawikiwiki, https://www.mediawiki.org/wiki/Special_Interest_Groups with notes and ongoing feedback discussions on the talkpage https://www.mediawiki.org/wiki/Talk:Special_Interest_Groups The first live example is the Wiki Replicas Special Interest Group. https://wikitech.wikimedia.org/wiki/Wiki_Replicas_Special_Interest_Group and there is a list of possible SIGs on the talkpage https://www.mediawiki.org/wiki/Topic:Txnngx847m0366kq There are no obligations to use this structure, nor implicit authority associated with them. It's just a set of best-practices for those who appreciate such things. Cheers, Quiddity, on behalf of the many people who've nudged it along. [1] https://phabricator.wikimedia.org/T164551 -- Nick Wilson (Quiddity) Community Liaison, Wikimedia Foundation

1 0

Logs are down
by יגאל חיטרון 13 Sep '17

13 Sep '17

Hi. It looks wrong, at least for me, and it will take hours until somebody will pay attention in tab, so I write here. The sql log tables are dead. In many wikis. Each one in different time. Enwiki yesterday 17:50, hewiki today moring, ruwiki today evening. I'm 98% sure it's wrong. 1% it's always like that and I didn't know, and 1% it's done in purpose. I filed T175487. Thank you. Igal

6 8

TechCom IRC discussion tonight: Job Queue Issues
by Daniel Kinzler 13 Sep '17

13 Sep '17

Hi all! This is a quick reminder that tonight, at the TechCOm IRC hour, we will be talking about the job queue. There have been several issues iwth it lately, and we want to make sure that we have all relevant aspects on the radar. As always, the discussion will take place in the IRC channel #wikimedia-office on Wednesday 21:00 UTC (2pm PDT, 23:00 CEST). This is not an RFC meeting, as there is no concrete proposal. Rather, it's an opportunity to further our understanding of the problems and hand, and to float ideas for possible improvements. I have prepared a quick brain dump of my current understanding of the job queue issues <https://www.mediawiki.org/wiki/User:Daniel_Kinzler_(WMDE)/Job_Queue>. Here's a copy for your convenience, but please comment directly in the document. Observations: * Latest instance of the JQ exploding: https://phabricator.wikimedia.org/T173710 * With 600k jobs in the backlog of commonswiki, only 7k got processed in a day. * For wikis with just a few thousand pages, we sometimes see millions of UpdateHtmlCache jobs sitting in the queue. * Jobs that were triggered months ago were found to continue failing and re-trying Issues and considerations: * Jobs re-trying indefinitely * Deduplication ** mechanism is obscure/undocumented. Some rely on rootJob parameters, some use custom logic. ** Batching prevents deduplication. When and how should jobs do batch operations? Can we automatically break up small batches? ** Delaying jobs may improve deduplication, but support for delayed jobs is limited/obscure. ** Custom coalescing could improve the chance for deduplication. * Scope and purpose of some jobs is unclear. E.g. UpdateHtmlCache invalidates the parser cache, and RefreshLinks re-parse the page - but does not trigger an UpdateHtmlCache, which it probably should. * The throttling mechanism does not take into account the nature and run-time of different job types. * Scaling is achieved by running more cron jobs. * Kafka-based JQ is being tested by Services. Generally saner. Should improve ability to track causality (which job got triggered by which other job). T157088 * No support for recurrent jobs. Should we keep using cron? -- Daniel Kinzler Principal Platform Engineer Wikimedia Deutschland Gesellschaft zur Förderung Freien Wissens e.V.

1 0

Should we plan anything regarding Mozilla’s "Talk" open-source commenting platform
by mathieu stumpf guntz 13 Sep '17

13 Sep '17

Hi, I'm just discovering this blog post Mozilla and the Washington Post Are Reinventing Online Comments <https://blog.mozilla.org/blog/2017/09/06/mozilla-washington-post-reinventin…> and thought it might worth talking about its relevancy for the Wikimedia movement. For those who didn't followed the 2017 Movement strategy <https://meta.wikimedia.org/wiki/Strategy/Wikimedia_movement/2017/Direction>, we now also have wikicomment <http://www.wikicomment.ut.ee/> as a solution related to this topic. Kind regards, mathieu

1 0

Today: Weekly Technical Advice IRC Meeting
by Michael Schönitzer 13 Sep '17

13 Sep '17

Sorry for cross-posting! Reminder: Technical Advice IRC meeting again **today 3-4 pm UTC** on #wikimedia-tech. The Technical Advice IRC meeting is open for all volunteer developers, topics and questions. This can be anything from "how to get started" over "who would be the best contact for X" to specific questions on your project. If you know already what you would like to discuss or ask, please add your topic to the next meeting: https://www.mediawiki.org/ wiki/Technical_Advice_IRC_Meeting This meeting is an offer by WMDE’s tech team. Hosts of todays meeting are: @addshore & @Tobi_WMDE_SW. Hope to see you there! Michi (for WMDE’s tech team) -- Michael F. Schönitzer Wikimedia Deutschland e.V. | Tempelhofer Ufer 23-24 | 10963 Berlin Tel. (030) 219 158 26-0 http://wikimedia.de Stellen Sie sich eine Welt vor, in der jeder Mensch an der Menge allen Wissens frei teilhaben kann. Helfen Sie uns dabei! http://spenden.wikimedia.de/ Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e.V. Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/681/51985.

1 0

Shibboleth authentication
by Sorin Gheorghiu 13 Sep '17

13 Sep '17

Dear developers, we learned that the authentication layer got a major overhaul in 1.27. We are interested in Shibboleth-authentication [1] which derives from AuthPlugin.php, deprecated and superseded by AuthManager.php. The last version of this extension [1] is compatible with 1.26, but it does work with 1.27 and above. It seems the project development has been stopped. While you are accustomed with the new AuthManager classes, could you help us by chance to get a new script working for 1.27? We found the updating tips [3] but unfortunately we don't have any Mediawiki programming experience. [1] https://www.mediawiki.org/wiki/Extension:Shibboleth_Authentication [2] https://github.com/ConsortiumGARR/mediawiki-shibboleth-authentication/blob/… [3] https://www.mediawiki.org/wiki/Manual:SessionManager_and_AuthManager/Updati… Thank you in advance

1 0

Wiki farm configuration
by Manuel Vacelet 13 Sep '17

13 Sep '17

Hi there, I've got questions around Wiki Farm setup, I'm not sure if it's the right place to ask the question. If there is a better channel, feel free to say so. In Wiki Family page [1] there is a quote saying > It is recommended to use a different DB for each wiki (By setting a > different $wgDBname > <https://www.mediawiki.org/wiki/Special:MyLanguage/Manual:$wgDBname> for > each wiki). However if you are limited to a single database, you can use a > different prefix ($wgDBprefix > <https://www.mediawiki.org/wiki/Special:MyLanguage/Manual:$wgDBprefix>) > to separate the different installs. > But it's not detailed why it's recommended. I'd like to know if there is any downside of using the prefix strategy (peformances, upgrades, etc) ? Regards, Manuel [1] https://www.mediawiki.org/wiki/Manual:Wiki_family

3 3

MinusX - a new tool to make sure files aren't executable (+x)
by Legoktm 13 Sep '17

13 Sep '17

Hi, Occasionally someone will unintentionally create a new file or modify an existing one to set the executable bit (+x). This is almost nearly an accident, and usually someone will come along later and fix them en masse[1][2]. With help from Anomie, I've written a tool, MinusX[3], that will search and for executable files that shouldn't be, and optionally fix them. It's written in PHP and should run as part of "composer test", but operates on all types of files, not just PHP. I've proposed adding this tool to all repositories[4] in an automated manner. If you have any suggestions/feature requests/bugs, feel free to create a ticket in Phabricator or reply here. [1] https://phabricator.wikimedia.org/T168659 [2] https://phabricator.wikimedia.org/P5913 [3] https://www.mediawiki.org/wiki/MinusX [4] https://phabricator.wikimedia.org/T175794 Thanks, -- Legoktm

1 0

TechCom Radar, 2017-09-06
by Daniel Kinzler 11 Sep '17

11 Sep '17

Hi all! Find below the minutes of the last meeting of the Technical Committee. * PDF generation for Extension:Collection to be exposed as a stand-alone service <https://phabricator.wikimedia.org/T171965> * Quiddity et.Al. have laid out best practices for Special Interest Groups (SIGs) <https://www.mediawiki.org/wiki/Special_Interest_Groups> (also see the discussion on the talk page). TechCom sees no direct connection to its current operation, but is looking forward to explore different modes of collaboration with and between such groups. * Next week’s IRC discussion: Job Queue - the good, the bad, the ugly. <https://www.mediawiki.org/wiki/User:Daniel_Kinzler_(WMDE)/Job_Queue>. There are several interlocking issues to discuss. We hope the conversation will surface the most pressing issues, and perhaps produce new ideas for solutions. As always, the discussion will take place in the IRC channel #wikimedia-office on Wednesday 21:00 UTC (2pm PDT, 23:00 CEST). You can also find our meeting minutes at <https://www.mediawiki.org/wiki/Wikimedia_Technical_Committee/Minutes> See also the TechCom RFC board <https://phabricator.wikimedia.org/tag/mediawiki-rfcs/>. -- Daniel Kinzler Principal Platform Engineer Wikimedia Deutschland Gesellschaft zur Förderung Freien Wissens e.V.

1 0

Discovery Weekly Update for the week starting 2017-09-04
by Chris Koerner 11 Sep '17

11 Sep '17

Hello, A few updates this week from across the Discovery team. Programming note: There is a new way to follow these weekly updates. You can subscribe to recieve on-wiki (or opt-in email) notifications of the Discovery weekly update. Subscribe to be notified. https://www.mediawiki.org/wiki/Newsletter:Discovery_Weekly == Discussions == * Paul conducted a cartographic review of the new map styles for the Design team on 6 Sep 2017. === Search === * Gehel updated logstash to stop deploying custom elasticsearch plugins that are no longer used and did a rolling restart of elasticsearch/logstash [0] * David upgraded an extra plugin as part of the upgrade to elastic 5.5.x [1] * Erik turned on a third test for the Search Relevance Survey (graded by humans), the test is expected to take a week. [2] * Erik trained MLR models for all wikis with >= 1% of search traffic, in preparation for AB test next week * Erik investigating potential memory leaks in MLR model training which makes training for large wikis (de, en) error prone === Portal === * Jan fixed a bug where the typeahead wasn't working for a single character [3] * Jan also updated the Wikipedia portal statistics and translations on 6 Sep 2017 [4] [5] === Maps === * Gehel finished up reimaging the maps-test servers [6] [0] https://phabricator.wikimedia.org/T174933 [1] https://phabricator.wikimedia.org/T174652 [2] https://phabricator.wikimedia.org/T175046 [3] https://phabricator.wikimedia.org/T173885 [4] https://phabricator.wikimedia.org/T128546 [5] https://phabricator.wikimedia.org/T142582 [6] https://phabricator.wikimedia.org/T169011 ---- The archive of all past updates can be found on MediaWiki.org: https://www.mediawiki.org/wiki/Discovery/Status_updates Interested in getting involved? See tasks marked as "Easy" or "Volunteer needed" in Phabricator. [1] https://phabricator.wikimedia.org/maniphest/query/qW51XhCCd8.7/#R [2] https://phabricator.wikimedia.org/maniphest/query/5KEPuEJh9TPS/#R Yours, Chris Koerner Community Liaison Wikimedia Foundation

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Wikitech-l September 2017