Wikitech-l October 2017

wikitech-l@lists.wikimedia.org

53 participants
70 discussions

2017-10-11 Scrum of Scrums meeting notes
by Grace Gellerman 12 Oct '17

12 Oct '17

https://www.mediawiki.org/wiki/Scrum_of_scrums/2017-10-11 = 2017-10-11= contact: https://www.mediawiki.org/wiki/Wikimedia_Engineering == Callouts == * Are there any special steps to wire a MediaWiki LoggerFactory into logstash-beta? AWight is unable to find some logs, which are present on fluorine-beta. * CommTech would appreciate help reaching consensus regarding the way preferences are handlled: https://lists.wikimedia.org/pipermail/wikitech-l/2017-October/088980.html * DBAs have disabled Wikidata RC syndication on Commons and ruwiki, and are working on purging existing WD RC rows on those wikis. This is because of performance and table size reasons. https://phabricator.wikimedia.org/T171027 * MediaWiki 1.31.0-wmf.3 didn't go out yesterday, we are getting caught up today. https://phabricator.wikimedia.org/T174359 == Audiences == === Readers === ==== iOS native app ==== * Blocked by: none * Blocking: none * Updates: ** Continuing work on 5.7.0 ( https://phabricator.wikimedia.org/project/view/2899/ ) - Visual updates to onboarding, iOS 11 support, iPhone X support ==== Android native app ==== * Blocked by: none * Blocking: none * Updates: ** On schedule to release beta of E-release this week ( https://phabricator.wikimedia.org/project/view/2839/ ) ** Will start (local) testing of Reading List service. === Discovery === * Continuing work on portal automation, resolving dependencies that fail to build on some systems and cleaning up old code. ==== Maps ==== * Blocked by: none * Blocking: none * Updates: * Got some help debugging nodejs issues * Community is activating <mapframe> on Spanish Wikipedia ==== Reading Infrastructure ==== * ReadingList RESTBase patch under review; hoping to deploy both RESTBase and MediaWiki part to beta soon * MCS: accounting for Parsoid changes: <figure-inline> instead of <span> for inline figures (https://phabricator.wikimedia.org/T177301) ==== Web ==== * Turning off OCG [cc. ops] * Working on replacement for OCG === Contributors === ==== Parsing ==== * cscott's ping to wikidata team: https://gerrit.wikimedia.org/r/#/c/379669/ -- Parsing is not blocked on this per se, but this is something that cscott has been contributing to as part of a longer term project. Using SoS to ping wikidata on a stalled review for him. * Updates ** Parsing team offsite next week -- will be absent at next week's SoS. ** Parsoid code to use the updated Templatedata format now merged (suggested by Thiemo at Wikimania 16). It is in the beta cluster and needs testing before being deployed. ** We are likely to merge both the <section> wrapper code as well as the <figure-inline> code at the same time so that we can bump the HTML version number once. *** Looks like cxserver/CX has patches to handle section output. Is MCS ready to handle Parsoid's section output? C.Scott said he is going to add a patch for VE to handle this, but, if VE folks want to tackle that on your own, please do so. ==== Global Collaboration ==== ===== Language ===== * Compact Language Links out of Beta deployment being scheduled next week for German Wikipedia. * Work continue with VE-CX integration and adpatation. * cxserver has been updated to latest master, debugging issues with Beta cxserver. ===== Collaboration ===== * New filters on RC now the default experience on all wikis except some FlaggedRevs wikis * Looking into Watchlist/ORES slow query issues, but DBAs are also working on that ==== UI Standardization ==== * Blocked: icon feedback appreciated, see below / none * Updates: ** Improved whitespace usage in OOUI HTMLForms https://phabricator.wikimedia.org/T177668 ** OOUI v0.23.4 release in preparation *** Generalized icon/indicator padding/positioning https://phabricator.wikimedia.org/T161177 & streamlined icon/indicator visibility and layout https://phabricator.wikimedia.org/T160593 * Ongoing: ** OOUI: *** icons: Work on icon set to be more harmonious and align to WikimediaUI Style Guide's guidelines https://phabricator.wikimedia.org/T177432 ** Aligning arbitrary, historically grown colors across products to WikimediaUI color palette https://phabricator.wikimedia.org/T148708 === Community Tech === * Blocked: not blocked, but see our callout * Blocking: none * Report: ** Working on GlobalPreferences ** Parser caches are almost populated with Unicode IDs == Technology == === Analytics === * No blocker (AFAIK) * Not blocked *Updates: ** Wikistats 2.0 UI is alpha-level with reading metrics, still missing edit metrics because of missing backend ** Setting up a production data cluster for druid with public data to be the edit-data public APIs backend. We have split our Druid cluster in two, with 3 hosts each. ** Getting Druid Grafana dashboards in place. ** Working on processing eventlogging data in hadoop instead of MySQL ** Still purging data in MySQL eventlogging hosts, slow going ** Work to move metrics to prometheus continues. === Scoring Platform === * Blocked by: none * Blocking: ** Still working on file handle exhaustion, which prevents us from using the new ORES cluster. * Updates: ** Deploying "revscoring 2.0", which could cause some instability in Extension:ORES and in the ORES service. === Search Platform === * Blocked by: none * Blocking: none * Updates: * Messaging fallbacks usage for search disabled ( https://phabricator.wikimedia.org/T147959) - actual switch happens when we reindex * MLR assisted results test analysis published: https://wikimedia-research.github.io/Discovery-Search-Test-InterleavedLTR/, results generally positive * Patch for indexing wikidata statements merged, will index P31 and P279 initially after deployment * Working on porting Selenium tests from Ruby to JS * Working on upgrade to Elastic 5.5 * Working on indexing Wikidata descriptions and adding them to fulltext search === Services === * Blocked: none * Blocking: none ** Updates: *** Deploying cassandra 3 in beta cluster *** Starting testing Parsoid with Cassandra 3 === Technical Operations === * '''Blocked''': ** none * '''Blocking''': ** none * Updates ** Singapore caching DC ongoing ** Quite a bit of puppet refactoring for varnishes ** ganglia slowly being killed (nobody will probably even notice) === Fundraising Tech === * Still trying to figure out db lag, have some leads: https://phabricator.wikimedia.org/T173472 * Still trying to figure our missing data from kafkatee, getting some help: https://phabricator.wikimedia.org/T177331 * More CiviCRM contact de-duplication improvements * Fixing Civi import bugs: https://phabricator.wikimedia.org/T177806, https://phabricator.wikimedia.org/T177803 * Cleaned up contact groups in Civi to improve performance * Diagnosing apparent missing banner impressions (big chunks seem to be bot-related): https://phabricator.wikimedia.org/T176802 == Wikidata == * (Hopefully) unblocked QUnit 2 update: https://phabricator.wikimedia.org/T177764 * We had to turn of RecentChanges integration on big wikis for performance reasons, looking into workarounds: https://phabricator.wikimedia.org/T177707 * Fixed a regression we believe happened because of a jQuery update: https://phabricator.wikimedia.org/T177485 * We turn JS-only components into pure NPM libraries: https://phabricator.wikimedia.org/T177087 * Constraint checks also on qualifiers and references: https://phabricator.wikimedia.org/T176863 * Added new Lua functions to the mw.wikibase library to avoid pulling entire entities: https://phabricator.wikimedia.org/T142903, https://phabricator.wikimedia.org/T173262 * Full URIs for external identifiers will soon be in the RDF export: https://phabricator.wikimedia.org/T121274 * Optimizing runtime of the wikidata.org dump generation: https://phabricator.wikimedia.org/T177486 * wb_entity_per_page table is not updated any more: https://phabricator.wikimedia.org/T140890 * wb_terms table on labs does have a term_full_entity_id column now: https://phabricator.wikimedia.org/T167114 * As usual, we continue working on the Lexeme extension for Wiktionary: https://phabricator.wikimedia.org/T168263 == RelEng == * Blocked: none * Blocking: none? * Updates: ** wmf.3 cut this morning, back on track today group0 and group1 == Security == * Reviews: ** Ex:FileImporter (T160982) ** Ex:WikipediaExtracts (T149424) ** pdfrw

1 0

Person of interest to blame
by יגאל חיטרון 11 Oct '17

11 Oct '17

Hello. There was a patch, or a bug, or a bug fix in the last month that I'm trying to find. Something changed the Special:Watchlist view and broke a couple of gadgets. I wanted to find what was the gerrit patch, or at least some explanation, what exactly was done, but nobody knows (Thanks a lot to Quiddity which spent many time trying to help, you are awesome). The problem is different HTML structure of Special:Watchlist, at least when "Group changes by page in recent changes and watchlist" in preferences is on. I even can't describe the difference, because it's just "looks different". I read a roadmap for the last 50 days, and can't find anything. A couple of people I asked can't help. One of the symptoms is different css inside the revisions groups, bold vs unbold. Maybe somebody can help to find a gerrit to blame? Thank you, Igal (User:IKhitron)

5 5

Discovery Weekly Update for the week starting 2017-10-02
by Chris Koerner 10 Oct '17

10 Oct '17

Hello all, Here is the Discovery weekly update for 2017-10-02. As always, feedback and questions are welcome. ==Highlights== * The week of October 9th, we will deploy code to disable the automatic use of UI message fallback languages for linguistic analysis. In most cases, the messaging fallback language is unrelated to the main language of a wiki (e.g., Wolof falls back to French). A fuller description is on MediaWiki, and you can join the discussion there or on the related Phab ticket. [0] [1] * The Analysis team had several meetings to scope out new work to establish a baseline of metrics for the Structured Data on Commons project. [2] ==Discussions== === Search === * Trey is investigating linguistic analysis software that could be adapted for use on wiki projects. If you know of any open-source linguistic analysis software for languages that don't already have it on their wiki projects, please drop a note on Phab ticket T171652. [3] * David was busy with various tasks: ** relaxing the default "AND" of the retrieval query filter with minimum_should_match [4] ** change loading of cross-wiki configuration to use cirrus-dump-config api call [5] ** finishing up the work to upgrade Elastica to support elastic 5.5.x (will be in production next week [6] ** and adding Special:Search—add additional namespaces to search results for Persian Wikibooks , Hebrew Wikisource, and English Wikibooks [7] [8] [9] * We also stopped the A/B test that was running on 18 wikis with > 1% of search traffic, analysis will follow [10] === Analysis === * Chelsy investigated the full-text search pattern on mobile web and found interesting things: "we think that since people use full-text instead of prefix search results when they have a in-depth information need, and this kind of need is often driven by work/school project, we see more full-text searches on weekdays than weekends on both desktop and mobile web." [11] === Maps === * Paul kicked off the process to regenerate the map tiles in production that will probably finish early next week. We had to do this manually because of a bug in tileshell/node.js that we're working to resolve. [12] [13] [0] https://www.mediawiki.org/wiki/Wikimedia_Discovery/Disabling_Messaging_Fall… [1] https://phabricator.wikimedia.org/T147959 [2] https://phabricator.wikimedia.org/T174519 [3] https://phabricator.wikimedia.org/T171652 [4] https://phabricator.wikimedia.org/T177302 [5] https://phabricator.wikimedia.org/T156497 [6] https://phabricator.wikimedia.org/T174654 [7] https://phabricator.wikimedia.org/T176908 [8] https://phabricator.wikimedia.org/T176907 [9] https://phabricator.wikimedia.org/T176906 [10] https://phabricator.wikimedia.org/T177490 [11] https://phabricator.wikimedia.org/T176815 [12] https://phabricator.wikimedia.org/T176252 [13] https://phabricator.wikimedia.org/T175123 --- Subscribe to receive on-wiki (or opt-in email) notifications of the Discovery weekly update. https://www.mediawiki.org/wiki/Newsletter:Discovery_Weekly The archive of all past updates can be found on MediaWiki.org: https://www.mediawiki.org/wiki/Discovery/Status_updates Interested in getting involved? See tasks marked as "Easy" or "Volunteer needed" in Phabricator. [1] https://phabricator.wikimedia.org/maniphest/query/qW51XhCCd8.7/#R [2] https://phabricator.wikimedia.org/maniphest/query/5KEPuEJh9TPS/#R Yours, Chris Koerner Community Liaison Wikimedia Foundation

1 0

Tomorrow: Weekly Technical Advice IRC Meeting
by Michael Schönitzer 10 Oct '17

10 Oct '17

Sorry for cross-posting! Reminder: Technical Advice IRC meeting again **tomorrow 3-4 pm UTC** on #wikimedia-tech. The Technical Advice IRC meeting is open for all volunteer developers, topics and questions. This can be anything from "how to get started" over "who would be the best contact for X" to specific questions on your project. If you know already what you would like to discuss or ask, please add your topic to the next meeting: https://www.mediawiki.org/ wiki/Technical_Advice_IRC_Meeting This meeting is an offer by WMDE’s tech team. Hope to see you there! Michi (for WMDE’s tech team) -- Michael F. Schönitzer Wikimedia Deutschland e.V. | Tempelhofer Ufer 23-24 | 10963 Berlin Tel. (030) 219 158 26-0 http://wikimedia.de Stellen Sie sich eine Welt vor, in der jeder Mensch an der Menge allen Wissens frei teilhaben kann. Helfen Sie uns dabei! http://spenden.wikimedia.de/ Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e.V. Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/681/51985.

1 0

Language engineering monthly report for August 2017
by Niklas Laxström 09 Oct '17

09 Oct '17

The language engineering monthly report for August 2017 is ready. **Highlights for this month** Language database is now an independent project to make reuse easier. It used to be part of jquery.uls library. It contains over 500 entries detailing basic information about languages, such as their autonym, their writing script, and region(s) of the world where they are spoken. https://github.com/wikimedia/language-data August was, so far, the most active month of 2017 for translatewiki.net with the highest number of translation updates (55k) and translation reviews (15k) by over 350 translators. **Full report** https://www.mediawiki.org/wiki/User:Nikerabbit/Monthly_report/2017-08 For those unfamiliar with this report, its goal is to summarize all technical changes to internationalization, translation tools and other language support products. It also highlights the diversity of contributors to this area and that many of them are volunteers.

1 0

Wikidata recentchanges on commonswiki and ruwiki not shown
by Jaime Crespo 09 Oct '17

09 Oct '17

Hi all, This is a heads up that I asked, and other developers agreed with the decision, to *temporarily* disable wikidata-originated recentchanges appearing on commonswiki and ruwiki (Commons and Wikipedia in Russian): [0]. I cannot disclose right now all reasons (we will do when things are fixed), but please understand that server admins only do this in case of emergencies such as things being down or in a very broken state. For your tranquillity, this will not affect any wikidata functionality and has no edit or other data loss- it will only make wikidata edits not be *shown* (but they will take effect) on recentchanges and watchlist on other projects (but updates and wikilinks happen as usual) during the time it is disabled. Part of the story can be read at [1], even with only that information, the rollback I think is not much worse than the issues created. We do not discard extending this action to other wikis, too. We apologize for the impact this could have on vandalism patrolling- we will try to make things better soon, so that both errors are minimized but also new features can be active again soon. Regards, [0] <url:https://phabricator.wikimedia.org/source/mediawiki-config/browse/master…> [1] <url:https://phabricator.wikimedia.org/T171027> -- Jaime Crespo <http://wikimedia.org>

1 0

TechCom Radar, 2017-10-04
by Kevin Smith 07 Oct '17

07 Oct '17

Hello all! With Daniel out, I volunteered to send this email. Find below the streamlined minutes of the last meeting of the Technical Committee. When Daniel returns, he'll fill in any details I may have missed. * RFC Approved: Introduce InterruptMutexManager. There was no further discussion, so the committee approved this RFC. <https://phabricator.wikimedia.org/T161749>. * Last week’s IRC discussion covered how long we need/want to keep support for HHVM. IRC meeting log: <https://tools.wmflabs.org/ meetbot/wikimedia-office/2017/wikimedia-office.2017-09-27-21.03.html> Thread on wikitech: <https://lists.gt.net/wiki/wikitech/844711>. RFC about bumping the PHP version (for context) <https://phabricator.wikimedia.org/T172165> * There was no IRC discussion this week (2017-10-04). * Next week's IRC discussion (2017-10-11): Move RunJobs.php to core < https://phabricator.wikimedia.org/T175146> <https://phabricator.wikimedia.org/T175146> As always, the discussion will take place in the IRC channel #wikimedia-office on Wednesday 21:00 UTC (2pm PDT, 23:00 CEST). You can also find our meeting minutes at <https://www.mediawiki.org/wiki/Wikimedia_Technical_Committee/Minutes> See also the TechCom RFC board <https://phabricator.wikimedia.org/tag/mediawiki-rfcs/>. Kevin Smith Engineering Program Manager, Wikimedia Foundation

1 0

Read-only Commons in preparation for 3D file support - 11 October
by Chris Koerner 05 Oct '17

05 Oct '17

Hello, (I posted this on the the Commons Village Pump, apologies for cross-posting) The Multimedia team at the foundation is getting closer to enabling support for uploading and displaying 3D models on Commons. We have to do some work on the database to get ready for 3D file support. This means we'll need to put Commons in read-only mode for about 30 minutes on Wednesday October, 11th at 6:00 UTC. [0] You will be able to read, but not edit Commons for a short period of time. * You will not be able to edit or upload media for approximately 30 minutes on Wednesday, 11 October, starting at 6:00 UTC (07:00 BST, 08:00 CEST, 02:00 EDT, 23:00 PDT). * If you try to edit or save during these times, you will see an error message. We hope that no edits will be lost during these minutes, but we can't guarantee it. If you see the error message, then please wait until everything is back to normal. Then you should be able to save your edit. But, we recommend that you make a copy of your changes first, just in case. Why? The team needs to modify the database primary master so it understands this new file type. This will then allow for the future ability to upload and display 3D models on Commons (and embed in other Wikimedia projects). Want a sneak-peek? Take a look at this programmatically created crystal on Test Wikipedia. [1] It is important to note that the ability to upload 3D files will not be available immediately after the maintenance ends (but at a later date), this only involves the contributor-impacting database work. How can folks monitor? You can keep track of the changes by joining #wikimedia-operations on freenode.net Where do I go with issues? If you have issues after the database work is completed, please report a ticket on the Wikimedia Bug tracker, Phabricator. [2] Please help share this with your community. [0] https://phabricator.wikimedia.org/T168661 [1] https://test.wikipedia.org/wiki/File:Programmatically_created_crystal.stl [2] http://phabricator.wikimedia.org Yours, Chris Koerner Community Liaison Wikimedia Foundation

1 1

Using web fonts with good performance
by Joaquin Oltra Hernandez 05 Oct '17

05 Oct '17

A small TIL, Chrome has been championing this proposal, which I believe would solve the performance concerns that have been raised before about using web fonts, allowing us to have a more fine grained control about the behavior. Right now it is Chrome only, and Firefox and Webkit have it in development. Something to keep in mind for the near future! Links: - Spec proposal <https://tabatkins.github.io/specs/css-font-display/> - Browser usage <http://caniuse.com/#search=font-display> - MDN docs <https://developer.mozilla.org/en-US/docs/Web/CSS/@font-face/font-display> - Feature detection <https://jsfiddle.net/p7g8y7du/> - Controlling Font Performance with font-display <https://developers.google.com/web/updates/2016/02/font-display> - Video [1m40s] Font-Display: Micro Tip #14 - Supercharged <https://youtu.be/-lPTshDcr7o>

1 0

Readers monthly update for September 2017
by Chris Koerner 05 Oct '17

05 Oct '17

Greetings, I'm here again with the monthly update from the Readers department. **New**: You can now subscribe to receive theses updates as on-wiki notifications or opt-in email! Handy if you subscribe to the digest of this mailing list or don't visit the archive frequently. Subscribe on MediaWiki.org: https://www.mediawiki.org/wiki/Newsletter:Readers_Monthly Thanks for reading, as always, feedback and questions are welcome. ==Discussions== === Apps=== * '''Android''': Progress being made with the Offline Library feature. There's a video demo and more information on Mediawiki.org. [0] [1] ===Web=== * We've updated the visual design for hatnotes on the mobile website to distinguish them from the remainder of the content. [2] * Due to issues with our current PDF rendering system, we will be pausing rendering for the book creator for the time being, until we can identify a proper solution. More information can be found on the project page. [3] * We have been planning and making roadmap updates for the updated mobile website project. [4] === Reading Infrastructure === * Reading Lists have been announced to the Wikipedia projects. [5] [6] === New Readers === * New Reader updates can be found at Meta. [7] === Multimedia === * In preparation for 3D object files on commons a short 30 minute read-only period for Commons will happen on October 11th. [8] * Community feedback is wanted on an update to the upload process on Commons for 3D objects [9] === Discovery === The Discovery team provides weekly updates. Here are their updates for the month. * 2017-09-11 [10] * 2017-09-18 [11] * 2017-09-25 [12] -- [0] https://www.youtube.com/watch?v=iESP20HGPiE [1] https://www.mediawiki.org/wiki/Wikimedia_Apps/Offline_support [2] https://phabricator.wikimedia.org/T173600 [3] https://www.mediawiki.org/wiki/Reading/Web/PDF_Functionality [4] https://www.mediawiki.org/wiki/Reading/Web/Projects/NewMobileWebsite [5] https://www.mediawiki.org/wiki/User:CKoerner_(WMF)/Reading_List_Discussion [6] https://www.mediawiki.org/wiki/Wikimedia_Apps/Synced_Reading_Lists [7] https://meta.wikimedia.org/wiki/New_Readers/Updates [8] https://commons.wikimedia.org/wiki/Commons:Village_pump#Read-only_Commons_i… [9] https://commons.wikimedia.org/wiki/Commons:Village_pump#Input_requested_on_… [10] https://www.mediawiki.org/wiki/Discovery/Status_updates/2017-09-11 [11] https://www.mediawiki.org/wiki/Discovery/Status_updates/2017-09-18 [12] https://www.mediawiki.org/wiki/Discovery/Status_updates/2017-09-25 --- The archive of all past updates can be found on MediaWiki.org: https://www.mediawiki.org/wiki/Reading/Status_updates Yours, Chris Koerner Community Liaison Wikimedia Foundation

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Wikitech-l October 2017