Wikitech-l October 2017

wikitech-l@lists.wikimedia.org

53 participants
75 discussions

JADE needs your feedback
by Keegan Peterzell 16 Oct '17

16 Oct '17

(cross-posting to Wikitech and Wikidata; Wikidata is getting this due to the general interest in data sets and databases) Greetings, The Platform Scoring Team at the Wikimedia Foundation are developing an auditing tool for ORES, the Judgement and Dialogue Engine (JADE) [0]. It's largely inspired by the false positive reporting work that currently occurs by hand on wiki pages. JADE's purpose is to provide human oversight to ORES's AI work. The team is seeking input in two areas as development begins in two technical areas: the database schema [1], and the implementation strategy [2]. If you have thoughts or experience to share in how to best set these up, please have a look over the pages. Questions, comments, concerns are welcome on the talk pages [3][4]. Feedback and stories on using ORES in production is also welcome, to help give the team general ideas about areas of concern and/or improvement that JADE should keep in mind [5]. Thank you for your time, see you on the wikis. 0. https://www.mediawiki.org/wiki/JADE 1. https://www.mediawiki.org/wiki/JADE/Schema 2. https://www.mediawiki.org/wiki/JADE/Implementations 3. https://www.mediawiki.org/wiki/Talk:JADE/Schema 4. https://www.mediawiki.org/wiki/Talk:JADE/Implementations 5. https://www.mediawiki.org/wiki/Talk:JADE -- Keegan Peterzell Technical Collaboration Specialist Wikimedia Foundation

4 5

Fwd: New newsletter: Tech Showcase
by Quim Gil 16 Oct '17

16 Oct '17

Dear Wikimedia developers, TLDR: Please give this a try, subscribe, and forward the link to other mailing lists or projects: https://www.mediawiki.org/ wiki/Newsletter:Tech_Showcase. I am of course biased, but I do believe that this newsletter is interesting basically for any active contributors. The simplest and easiest way to know about interesting software being developed by staff or volunteers for our readers and our contributors. No tech knowledge needed and no time commitment required either. You will just receive one sentence notifications linking to the fresh software showcased. PS: This is a new concept of newsletter for experienced Wikimedians as well. The Technical Collaboration team hopes that these types of newsletters will reach new audiences and new voices beyond the power users following Village Pumps, mailing lists or Phabricator. ---------- Forwarded message ---------- From: Quim Gil <qgil(a)wikimedia.org> Date: Fri, Oct 13, 2017 at 12:18 PM Subject: New newsletter: Tech Showcase To: Wikimedia Mailing List <wikimedia-l(a)lists.wikimedia.org> Tech curious? This is for you: Tech Showcase <https://www.mediawiki.org/wiki/Newsletter:Tech_Showcase> A newsletter (*) about fresh software for Wikimedia caught on the spot. Plans, prototypes, releases... Do you have a scoop? Tell us in the Talk page! Just click "Subscribe" and you will be notified in your preferred Wikimedia wiki. https://www.mediawiki.org/wiki/Newsletter:Tech_Showcase (*) This is a newsletter powered by the Newsletter extension <https://www.mediawiki.org/wiki/Help:Extension:Newsletter>. Expect one sentence notifications linking to the tech showcased, not lengthy articles and prose. -- Quim Gil Engineering Community Manager @ Wikimedia Foundation http://www.mediawiki.org/wiki/User:Qgil

3 4

Invitation to test the new Two Column Edit Conflict view
by Birgit Müller 13 Oct '17

13 Oct '17

(sorry for cross-posting) Dear all, *TL;DR: *If you don't want to wait until you run into a real edit conflict to tryout *Two Column Edit Conflict View* [1], you can now test the new interface on a temporary testpage where you will get simulated edit conflicts. *About Two Column Edit Conflict View* Improving the current UI of the edit conflict resolution page was wish #1 in the 2015 Technical wishes survey of the German-speaking community [2]. The TwoColConflict extension ("Two Column Edit Conflict View") [3] highlights differences between the editor's and the conflicting change directly in the textfield for an easy possibility to copy and paste desired pieces of the text and resolving the conflict. It has been a beta feature on all wikis for 5 months now and users have been invited to test it and give feedback. [4] Here is a rough visualisation of how the Two Column Edit Conflict view works: https://commons.wikimedia.org/wiki/File:Two_Column_Edit_Conflict_View_-_rou… :-) *The challenge* We got some feedback on the beta feature, but not enough to get a good feeling if the product is already great as it is now, or if some changes are still needed. We assume that's because in order to give feedback, people have to have the beta feature activated AND have an edit conflict AND share their feedback - that's a pretty rare combination. *More testing!* We've implemented a temporary Special page that serves as a *test page* [5] where you will get simulated edit conflicts: The new interface can be tested now without actually having an edit conflict, and without the risk of messing up real wiki pages. We'd like to invite everyone to test the new interface with the test page until November 9, 2017! Please go to *https://meta.wikimedia.org/wiki/WMDE_Technical_Wishes/Edit_Conflicts/Feedback_Round_Test_Page <https://meta.wikimedia.org/wiki/WMDE_Technical_Wishes/Edit_Conflicts/Feedba…> *for more informations. If you know people who might be interested in testing, please spread the word! Thank you so much, Birgit (for the Technical Wishes team) Links: [1] Multilingual help page <https://www.mediawiki.org/wiki/Help:Two_Column_Edit_Conflict_View> [2] Main project page <https://meta.wikimedia.org/wiki/WMDE_Technical_Wishes/Edit_Conflicts> [3] Extension manual <https://www.mediawiki.org/wiki/Extension:TwoColConflict> [4] Central feedback page <https://www.mediawiki.org/wiki/Help_talk:Two_Column_Edit_Conflict_View> [5] Ticket on Phab for the Special page <https://phabricator.wikimedia.org/T170375> -- Birgit Müller Community Communications Manager Software Development and Engineering Wikimedia Deutschland e.V. | Tempelhofer Ufer 23-24 | 10963 Berlin Tel. (030) 219 158 26-0 http://wikimedia.de Stellen Sie sich eine Welt vor, in der jeder Mensch an der Menge allen Wissens frei teilhaben kann. Helfen Sie uns dabei! http://spenden.wikimedia.de/ Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e.V. Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/681/51985.

1 0

TechCom Radar, 2017-10-11
by Daniel Kinzler 12 Oct '17

12 Oct '17

Hello all! Tanks to Kevin for taking over while I was on vacation. And apologies for skipping the radar for our meeting on September 27. Here are the minutes from this week's meeting. I have folded in some relevant points from the previous weeks. * SQL backend for EventLogging is to be dropped in favor of the hadoop backend. * OCG has been turned off. Electron is a temporary replacement. * CommTech is seeking guidance on global preference implementation. <https://lists.wikimedia.org/pipermail/wikitech-l/2017-October/088980.html> * Ops had to turn off Wikidata RecentChanges integration on big wikis for performance reasons, looking into workarounds: <https://phabricator.wikimedia.org/T177707> * This week’s IRC discussion: Move rpc/RunJobs.php to core <https://phabricator.wikimedia.org/T175146>. You can find the minutes at <https://tools.wmflabs.org/meetbot/wikimedia-office/2017/wikimedia-office.20…> * Next week’s IRC discussion: Requiring PHP 7 <https://phabricator.wikimedia.org/T172165> As always, the discussion will take place in the IRC channel #wikimedia-office on Wednesday 21:00 UTC (2pm PDT, 23:00 CEST). You can also find our meeting minutes at <https://www.mediawiki.org/wiki/Wikimedia_Technical_Committee/Minutes> See also the TechCom RFC board <https://phabricator.wikimedia.org/tag/mediawiki-rfcs/>. -- Daniel Kinzler Principal Platform Engineer Wikimedia Deutschland Gesellschaft zur Förderung Freien Wissens e.V.

1 0

Fwd: Update on Artifex v. Hancom GNU GPL compliance case
by Federico Leva (Nemo) 12 Oct '17

12 Oct '17

https://www.fsf.org/blogs/licensing/update-on-artifex-v-hancom-gnu-gpl-comp… -------- Messaggio inoltrato -------- Oggetto: Update on Artifex v. Hancom GNU GPL compliance case Data: Thu, 12 Oct 2017 03:05:18 -0400 [...] Hancom here made several arguments against the contract claim, but one is of particular interest. Hancom argued that if any contract claim is allowed, damages should only be considered prior to the date of their initial violation. They argued that since the violation terminated their license, the contract also ended at that point. The judge noted that: /the language of the GPL suggests that Defendant’s obligations persisted beyond termination of its rights to propagate software using Ghostscript ... because the source code or offer of the source code is required each time a “covered work” is conveyed, each time Defendant distributed a product using Ghostscript there was arguably an ensuing obligation to provide or offer to provide the source code./ The judge also found that there was insufficient evidence at this point to rule on this issue, so we can't read too much into it. [...]

1 0

2017-10-11 Scrum of Scrums meeting notes
by Grace Gellerman 11 Oct '17

11 Oct '17

https://www.mediawiki.org/wiki/Scrum_of_scrums/2017-10-11 = 2017-10-11= contact: https://www.mediawiki.org/wiki/Wikimedia_Engineering == Callouts == * Are there any special steps to wire a MediaWiki LoggerFactory into logstash-beta? AWight is unable to find some logs, which are present on fluorine-beta. * CommTech would appreciate help reaching consensus regarding the way preferences are handlled: https://lists.wikimedia.org/pipermail/wikitech-l/2017-October/088980.html * DBAs have disabled Wikidata RC syndication on Commons and ruwiki, and are working on purging existing WD RC rows on those wikis. This is because of performance and table size reasons. https://phabricator.wikimedia.org/T171027 * MediaWiki 1.31.0-wmf.3 didn't go out yesterday, we are getting caught up today. https://phabricator.wikimedia.org/T174359 == Audiences == === Readers === ==== iOS native app ==== * Blocked by: none * Blocking: none * Updates: ** Continuing work on 5.7.0 ( https://phabricator.wikimedia.org/project/view/2899/ ) - Visual updates to onboarding, iOS 11 support, iPhone X support ==== Android native app ==== * Blocked by: none * Blocking: none * Updates: ** On schedule to release beta of E-release this week ( https://phabricator.wikimedia.org/project/view/2839/ ) ** Will start (local) testing of Reading List service. === Discovery === * Continuing work on portal automation, resolving dependencies that fail to build on some systems and cleaning up old code. ==== Maps ==== * Blocked by: none * Blocking: none * Updates: * Got some help debugging nodejs issues * Community is activating <mapframe> on Spanish Wikipedia ==== Reading Infrastructure ==== * ReadingList RESTBase patch under review; hoping to deploy both RESTBase and MediaWiki part to beta soon * MCS: accounting for Parsoid changes: <figure-inline> instead of <span> for inline figures (https://phabricator.wikimedia.org/T177301) ==== Web ==== * Turning off OCG [cc. ops] * Working on replacement for OCG === Contributors === ==== Parsing ==== * cscott's ping to wikidata team: https://gerrit.wikimedia.org/r/#/c/379669/ -- Parsing is not blocked on this per se, but this is something that cscott has been contributing to as part of a longer term project. Using SoS to ping wikidata on a stalled review for him. * Updates ** Parsing team offsite next week -- will be absent at next week's SoS. ** Parsoid code to use the updated Templatedata format now merged (suggested by Thiemo at Wikimania 16). It is in the beta cluster and needs testing before being deployed. ** We are likely to merge both the <section> wrapper code as well as the <figure-inline> code at the same time so that we can bump the HTML version number once. *** Looks like cxserver/CX has patches to handle section output. Is MCS ready to handle Parsoid's section output? C.Scott said he is going to add a patch for VE to handle this, but, if VE folks want to tackle that on your own, please do so. ==== Global Collaboration ==== ===== Language ===== * Compact Language Links out of Beta deployment being scheduled next week for German Wikipedia. * Work continue with VE-CX integration and adpatation. * cxserver has been updated to latest master, debugging issues with Beta cxserver. ===== Collaboration ===== * New filters on RC now the default experience on all wikis except some FlaggedRevs wikis * Looking into Watchlist/ORES slow query issues, but DBAs are also working on that ==== UI Standardization ==== * Blocked: icon feedback appreciated, see below / none * Updates: ** Improved whitespace usage in OOUI HTMLForms https://phabricator.wikimedia.org/T177668 ** OOUI v0.23.4 release in preparation *** Generalized icon/indicator padding/positioning https://phabricator.wikimedia.org/T161177 & streamlined icon/indicator visibility and layout https://phabricator.wikimedia.org/T160593 * Ongoing: ** OOUI: *** icons: Work on icon set to be more harmonious and align to WikimediaUI Style Guide's guidelines https://phabricator.wikimedia.org/T177432 ** Aligning arbitrary, historically grown colors across products to WikimediaUI color palette https://phabricator.wikimedia.org/T148708 === Community Tech === * Blocked: not blocked, but see our callout * Blocking: none * Report: ** Working on GlobalPreferences ** Parser caches are almost populated with Unicode IDs == Technology == === Analytics === * No blocker (AFAIK) * Not blocked *Updates: ** Wikistats 2.0 UI is alpha-level with reading metrics, still missing edit metrics because of missing backend ** Setting up a production data cluster for druid with public data to be the edit-data public APIs backend. We have split our Druid cluster in two, with 3 hosts each. ** Getting Druid Grafana dashboards in place. ** Working on processing eventlogging data in hadoop instead of MySQL ** Still purging data in MySQL eventlogging hosts, slow going ** Work to move metrics to prometheus continues. === Scoring Platform === * Blocked by: none * Blocking: ** Still working on file handle exhaustion, which prevents us from using the new ORES cluster. * Updates: ** Deploying "revscoring 2.0", which could cause some instability in Extension:ORES and in the ORES service. === Search Platform === * Blocked by: none * Blocking: none * Updates: * Messaging fallbacks usage for search disabled ( https://phabricator.wikimedia.org/T147959) - actual switch happens when we reindex * MLR assisted results test analysis published: https://wikimedia-research.github.io/Discovery-Search-Test-InterleavedLTR/, results generally positive * Patch for indexing wikidata statements merged, will index P31 and P279 initially after deployment * Working on porting Selenium tests from Ruby to JS * Working on upgrade to Elastic 5.5 * Working on indexing Wikidata descriptions and adding them to fulltext search === Services === * Blocked: none * Blocking: none ** Updates: *** Deploying cassandra 3 in beta cluster *** Starting testing Parsoid with Cassandra 3 === Technical Operations === * '''Blocked''': ** none * '''Blocking''': ** none * Updates ** Singapore caching DC ongoing ** Quite a bit of puppet refactoring for varnishes ** ganglia slowly being killed (nobody will probably even notice) === Fundraising Tech === * Still trying to figure out db lag, have some leads: https://phabricator.wikimedia.org/T173472 * Still trying to figure our missing data from kafkatee, getting some help: https://phabricator.wikimedia.org/T177331 * More CiviCRM contact de-duplication improvements * Fixing Civi import bugs: https://phabricator.wikimedia.org/T177806, https://phabricator.wikimedia.org/T177803 * Cleaned up contact groups in Civi to improve performance * Diagnosing apparent missing banner impressions (big chunks seem to be bot-related): https://phabricator.wikimedia.org/T176802 == Wikidata == * (Hopefully) unblocked QUnit 2 update: https://phabricator.wikimedia.org/T177764 * We had to turn of RecentChanges integration on big wikis for performance reasons, looking into workarounds: https://phabricator.wikimedia.org/T177707 * Fixed a regression we believe happened because of a jQuery update: https://phabricator.wikimedia.org/T177485 * We turn JS-only components into pure NPM libraries: https://phabricator.wikimedia.org/T177087 * Constraint checks also on qualifiers and references: https://phabricator.wikimedia.org/T176863 * Added new Lua functions to the mw.wikibase library to avoid pulling entire entities: https://phabricator.wikimedia.org/T142903, https://phabricator.wikimedia.org/T173262 * Full URIs for external identifiers will soon be in the RDF export: https://phabricator.wikimedia.org/T121274 * Optimizing runtime of the wikidata.org dump generation: https://phabricator.wikimedia.org/T177486 * wb_entity_per_page table is not updated any more: https://phabricator.wikimedia.org/T140890 * wb_terms table on labs does have a term_full_entity_id column now: https://phabricator.wikimedia.org/T167114 * As usual, we continue working on the Lexeme extension for Wiktionary: https://phabricator.wikimedia.org/T168263 == RelEng == * Blocked: none * Blocking: none? * Updates: ** wmf.3 cut this morning, back on track today group0 and group1 == Security == * Reviews: ** Ex:FileImporter (T160982) ** Ex:WikipediaExtracts (T149424) ** pdfrw

1 0

Person of interest to blame
by יגאל חיטרון 11 Oct '17

11 Oct '17

Hello. There was a patch, or a bug, or a bug fix in the last month that I'm trying to find. Something changed the Special:Watchlist view and broke a couple of gadgets. I wanted to find what was the gerrit patch, or at least some explanation, what exactly was done, but nobody knows (Thanks a lot to Quiddity which spent many time trying to help, you are awesome). The problem is different HTML structure of Special:Watchlist, at least when "Group changes by page in recent changes and watchlist" in preferences is on. I even can't describe the difference, because it's just "looks different". I read a roadmap for the last 50 days, and can't find anything. A couple of people I asked can't help. One of the symptoms is different css inside the revisions groups, bold vs unbold. Maybe somebody can help to find a gerrit to blame? Thank you, Igal (User:IKhitron)

5 5

Discovery Weekly Update for the week starting 2017-10-02
by Chris Koerner 10 Oct '17

10 Oct '17

Hello all, Here is the Discovery weekly update for 2017-10-02. As always, feedback and questions are welcome. ==Highlights== * The week of October 9th, we will deploy code to disable the automatic use of UI message fallback languages for linguistic analysis. In most cases, the messaging fallback language is unrelated to the main language of a wiki (e.g., Wolof falls back to French). A fuller description is on MediaWiki, and you can join the discussion there or on the related Phab ticket. [0] [1] * The Analysis team had several meetings to scope out new work to establish a baseline of metrics for the Structured Data on Commons project. [2] ==Discussions== === Search === * Trey is investigating linguistic analysis software that could be adapted for use on wiki projects. If you know of any open-source linguistic analysis software for languages that don't already have it on their wiki projects, please drop a note on Phab ticket T171652. [3] * David was busy with various tasks: ** relaxing the default "AND" of the retrieval query filter with minimum_should_match [4] ** change loading of cross-wiki configuration to use cirrus-dump-config api call [5] ** finishing up the work to upgrade Elastica to support elastic 5.5.x (will be in production next week [6] ** and adding Special:Search—add additional namespaces to search results for Persian Wikibooks , Hebrew Wikisource, and English Wikibooks [7] [8] [9] * We also stopped the A/B test that was running on 18 wikis with > 1% of search traffic, analysis will follow [10] === Analysis === * Chelsy investigated the full-text search pattern on mobile web and found interesting things: "we think that since people use full-text instead of prefix search results when they have a in-depth information need, and this kind of need is often driven by work/school project, we see more full-text searches on weekdays than weekends on both desktop and mobile web." [11] === Maps === * Paul kicked off the process to regenerate the map tiles in production that will probably finish early next week. We had to do this manually because of a bug in tileshell/node.js that we're working to resolve. [12] [13] [0] https://www.mediawiki.org/wiki/Wikimedia_Discovery/Disabling_Messaging_Fall… [1] https://phabricator.wikimedia.org/T147959 [2] https://phabricator.wikimedia.org/T174519 [3] https://phabricator.wikimedia.org/T171652 [4] https://phabricator.wikimedia.org/T177302 [5] https://phabricator.wikimedia.org/T156497 [6] https://phabricator.wikimedia.org/T174654 [7] https://phabricator.wikimedia.org/T176908 [8] https://phabricator.wikimedia.org/T176907 [9] https://phabricator.wikimedia.org/T176906 [10] https://phabricator.wikimedia.org/T177490 [11] https://phabricator.wikimedia.org/T176815 [12] https://phabricator.wikimedia.org/T176252 [13] https://phabricator.wikimedia.org/T175123 --- Subscribe to receive on-wiki (or opt-in email) notifications of the Discovery weekly update. https://www.mediawiki.org/wiki/Newsletter:Discovery_Weekly The archive of all past updates can be found on MediaWiki.org: https://www.mediawiki.org/wiki/Discovery/Status_updates Interested in getting involved? See tasks marked as "Easy" or "Volunteer needed" in Phabricator. [1] https://phabricator.wikimedia.org/maniphest/query/qW51XhCCd8.7/#R [2] https://phabricator.wikimedia.org/maniphest/query/5KEPuEJh9TPS/#R Yours, Chris Koerner Community Liaison Wikimedia Foundation

1 0

Tomorrow: Weekly Technical Advice IRC Meeting
by Michael Schönitzer 10 Oct '17

10 Oct '17

Sorry for cross-posting! Reminder: Technical Advice IRC meeting again **tomorrow 3-4 pm UTC** on #wikimedia-tech. The Technical Advice IRC meeting is open for all volunteer developers, topics and questions. This can be anything from "how to get started" over "who would be the best contact for X" to specific questions on your project. If you know already what you would like to discuss or ask, please add your topic to the next meeting: https://www.mediawiki.org/ wiki/Technical_Advice_IRC_Meeting This meeting is an offer by WMDE’s tech team. Hope to see you there! Michi (for WMDE’s tech team) -- Michael F. Schönitzer Wikimedia Deutschland e.V. | Tempelhofer Ufer 23-24 | 10963 Berlin Tel. (030) 219 158 26-0 http://wikimedia.de Stellen Sie sich eine Welt vor, in der jeder Mensch an der Menge allen Wissens frei teilhaben kann. Helfen Sie uns dabei! http://spenden.wikimedia.de/ Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e.V. Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/681/51985.

1 0

Language engineering monthly report for August 2017
by Niklas Laxström 09 Oct '17

09 Oct '17

The language engineering monthly report for August 2017 is ready. **Highlights for this month** Language database is now an independent project to make reuse easier. It used to be part of jquery.uls library. It contains over 500 entries detailing basic information about languages, such as their autonym, their writing script, and region(s) of the world where they are spoken. https://github.com/wikimedia/language-data August was, so far, the most active month of 2017 for translatewiki.net with the highest number of translation updates (55k) and translation reviews (15k) by over 350 translators. **Full report** https://www.mediawiki.org/wiki/User:Nikerabbit/Monthly_report/2017-08 For those unfamiliar with this report, its goal is to summarize all technical changes to internationalization, translation tools and other language support products. It also highlights the diversity of contributors to this area and that many of them are volunteers.

1 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Wikitech-l October 2017