Wikitech-l August 2016

wikitech-l@lists.wikimedia.org

78 participants
104 discussions

RESTBase multiple pages with one request
by Toni Hermoso Pulido 07 Aug '16

07 Aug '16

Hello, is it already possible to retrieve data from different pages just by using one request? E.g by combining: https://en.wikipedia.org/api/rest_v1/page/summary/Electron and https://en.wikipedia.org/api/rest_v1/page/summary/Dog Thanks! -- Toni Hermoso Pulido http://www.cau.cat http://www.similis.cc

3 5

Re: [Wikitech-l] JetBrains IDE licenses for MW developers - php, js/node, python, C/C++, puppet, ruby, ...
by Florian Schmidt 05 Aug '16

05 Aug '16

We've an all products Plan, so all IDEA Desktop products are included, so I'm pretty sure, you can use our existing subscriptions. Best, Florian

1 0

Re: [Wikitech-l] JetBrains IDE licenses for MW developers - php, js/node, python, C/C++, puppet, ruby, ...
by Ilya Korniyko 05 Aug '16

05 Aug '16

Did not get the answer, so repeating in the mailing list. We might need Idea license to support https://commons.wikimedia.org/wiki/Commons:WLX_Jury_Tool for Wiki loves monuments as it is the best IDE for Scala and we want to onboard someone except me to support the jury tool. Regards, Ilya On Mon, Nov 2, 2015 at 8:31 PM, Ilya Korniyko <intracer(a)gmail.com> wrote: > Hi Yuri, > Does the license work for Idea 15 that is released today? > > It says: License key is in legacy format. Press Convert button, and after > pressing Convert asks for First name, Last Name and Email > When I enter my name and email, it says that Your license key cannot be > found in the JetBrains database. > >>>>> >>>>> Date: Mon, 19 Oct 2015 23:02:02 +0300 >>>>>> From: Yuri Astrakhan <yastrakhan(a)wikimedia.org> >>>>>> To: Wikimedia developers <wikitech-l(a)lists.wikimedia.org> >>>>>> Subject: [Wikitech-l] JetBrains IDE licenses for MW developers - php, >>>>>> js/node, python, C/C++, puppet, ruby, ... >>>>>> Message-ID: >>>>>> <CALOOOkiP-vsX6he4i451Dg5TcH2vtDvdYkF4cr2 >>>>>> K3c9nFZCmWQ(a)mail.gmail.com> >>>>>> Content-Type: text/plain; charset=UTF-8 >>>>>> >>>>>> Developers, we now have licenses for JetBrains' IDEA Ultimate >>>>>> <https://www.jetbrains.com/idea/> (JavaScript, PHP, Python, ...) >>>>>> and CLion >>>>>> <https://www.jetbrains.com/clion/> (C/C++). The IDE supports >>>>>> debugging in >>>>>> Vagrant and on-the-fly static code analysis. >>>>>> >>>>>> If you are a volunteer developer, and want to use an IDE, contact me >>>>>> for a >>>>>> license. Those with access to the office wiki may register directly >>>>>> at the >>>>>> licenses <https://office.wikimedia.org/wiki/JetBrains> page. >>>>>> >>>>>> Please try it for a bit before getting a license -- just in case you >>>>>> don't >>>>>> like it. >>>>>> >>>>>> P.S. This is only for those who want JetBrains' IDE. If you are happy >>>>>> with >>>>>> Vim, good for you. If you like Atom or Sublime, great. If notepad is >>>>>> the >>>>>> best text editor of all times, all the power to you. Bikeshedding >>>>>> should >>>>>> go into a separate thread )) >>>>>> >>>>>> >>>>>> >>>> >>> >> >

1 0

Autodiscovery of extension unittests
by Legoktm 05 Aug '16

05 Aug '16

Hi, One thing that's always annoyed me about writing tests for an extension is that for the first test, you need to bootstrap it by adding a UnitTestsList hook, and copying the code for it from some other extension. Now that we have a proper extension registry, I wrote <https://gerrit.wikimedia.org/r/#/c/302944/> (merged by Florian!), which will automatically register any tests in an extension's "tests/phpunit/" directory. If you already have a UnitTestsList hook, the tests won't be duplicated or anything, but it is now safe to remove the hook. <https://phabricator.wikimedia.org/T142120> tracks updating extensions to do so by moving their tests into the standardized path and removing the hook. Note: this requires that the extension has already been converted to use extension.json. -- Legoktm

3 2

Workaround for repetitive OAuth prompts for labs tools
by Niharika Kohli 04 Aug '16

04 Aug '16

Hi everyone, If you're tired of repetitively prompting your users to approve your tool (using MediaWiki oauth), here's a trick that might come in handy: you can hit ".../w/index.php?title=Special:OAuth/authenticate" in the second leg of the handshake instead of ".../w/index.php?title=Special:OAuth/authorize" which would cause it to simply jump over to the callback URL if the user has approved your tool at least once in the past. This works for authonly or authonlyprivate grants only. You can read up more about this here: https://www.mediawiki.org/wiki/OAuth/For_Developers#Avoid_repetitive_login_… Note that the mwoauth library was recently updated to include this change so you might just want to update it. Thanks Brad(for writing it), Gergö (for telling Bryan) and Bryan (for telling me). -- Niharika Software Engineer (International) Wikimedia Foundation [User:NiharikaKohli]

1 0

Number of differences
by Bináris 04 Aug '16

04 Aug '16

Hi folks, given a difflink between two states of a page, is there a way to tell by bot, how many modifications there are? I.e. in how many fragments those two differ? I know, this is not an exact question, but the software itself interprets it somehow as it tries to emphasize differences with colors. -- Bináris

4 4

2016-08-03 Scrum of Scrums meeting notes
by Grace Gellerman 03 Aug '16

03 Aug '16

https://www.mediawiki.org/wiki/Scrum_of_scrums/2016-08-03 = 2016-08-03 = == Product == === Reading === ==== iOS native app ==== * 5.0.5 was released last week. ** Reduced crash rate by 90% ** Several problems were identified after release and scheduled for 5.0.6 * 5.0.6 is going through regression today ** Shold be released by end of week ** Working with beta testers to verify fixes https://phabricator.wikimedia.org/tag/ios-app-v5.0.6-hotfix/ * 5.1 is in Development ** Major features are iPad and Find in Page https://phabricator.wikimedia.org/project/board/1736/query/open/ ==== Android native app ==== * Current sprint: https://phabricator.wikimedia.org/project/view/2091/ * Next sprint (not yet full) : https://phabricator.wikimedia.org/project/view/2142/ * Working on navigation overhaul, not blocked/blocking anyone to our knowledge. ==== Mobile Content Service (MCS) ==== * Updating link handling * Adding namespace and last modifier information to mobile-sections responses. * Added a few more languages to 'In the news' feed. ==== Reading Web ==== * Current sprint: https://phabricator.wikimedia.org/project/view/2115/ * Next sprint (not yet full) : https://phabricator.wikimedia.org/project/view/2126/ * Mobile web language switcher button moving from bottom to top (with a tip if user uses button on bottom of page which will remain for another several sprints) * Lazy loaded images for mobile web shipping approximately next 2-4 weeks ==== Reading Infrastructure ==== * Nothing to flag for now. === Community Tech === * Rolling out UCA collation + numeric sorting to English Wikipedia ( https://phabricator.wikimedia.org/T136150) ** Currently testing for German Wikipedia ( https://de.wikipedia.beta.wmflabs.org/wiki/Kategorie:UCA-Sortierung) * Making improvements to InternetArchiveBot * Had more meetings to hash out plan for cross-wiki watchlists ** Would you please add +tgr to any related tix? Mainly in thinking about multiple watchlist support *down the road* (not now, understood that's not part of your charter right now) ** Will probably need a dedicated database shard/hardward === Editing === ==== Collaboration ==== * Blocked - None * Blocking - Finishing up the ConfirmEdit work ( https://phabricator.wikimedia.org/T141300 ) which is blocking ResourceLoader changes. Shouldn't require any action from you. * Updates ** Nothing major to flag ** Some internal Echo-related improvements ==== Language ==== * Blocked: ** [TechOps/DB] Investigate increase in "readonly" saving errors in Content Translation https://phabricator.wikimedia.org/T141090 - Need more guidance on what Language Team can do here. ** [Analytics] Suddenly outrageous higher pageviews for main pages https://phabricator.wikimedia.org/T141506 - Very important in statistics of Compact Language Links. *** Analytics response: we're happy to prioritize any infrastructure issues that come out of this, but right now it needs more research attention, Tilman and others are looking into it * Blocking: None * Updates: ** CX Dumps available soon ** Bug fixes for CX/CLL continue ** Apertium->Jessie in progress; Still lots of packages to go. ==== Parsing ==== * Ongoing migration of Parsoid cluster to Jessie and Node v4 (courtesy Alexandros Kosiaris & Marko Obrovac). All Parsoid deploys on hold until that finishes. * Ongoing work on a bunch of projects, but nothing worth updating here at this point. === Discovery === * No blockers * Working on implementing bm25 searching * Bugfixes for cross-wiki search (sections & namespaces) * Refactoring & cleanup of CirrusSearch code * Set up continuos cleanup process via cron (saneitizer) * Stripping the question mark (?) from search queries is on the train this week (https://phabricator.wikimedia.org/T133711) == Technology == === Analytics === * new AQS cluster is loaded back to February, when we get all the way back to last July we'll switch over ** analytics-l thread on how much history to keep in Pageview API: ( https://lists.wikimedia.org/pipermail/analytics/2016-July/005317.html ) * Mediawiki history reconstruction algorithm scaled to handle enwiki, now being productionized to do all wikis. NOTE to ops / DBAs: we will reach out to you on the list to make sure it's ok to hit analytics-store (dbstore1002 I think) like this * refinery deployment with scap is being tested tomorrow: [[Phab:T129151]] * EventLogging fixed to handle kafka restarts more graciously, being tested in beta right now === Research === * ORES incident during deployment ** https://wikitech.wikimedia.org/wiki/Incident_documentation/20160801-ORES ** deployment to beta didn't show issues due to differences in configuration between beta and prod === Technical Operations === * '''Blocked''' ** None * '''Blocking''' ** None * Updates ** Upgrading parsoid infrastructure to debian jessie, node 4.x ** There is going to be an varnish XKEY meeting ** prometheus work is ongoing ** thumbor is being introduced as a service in the infrastructure === Security === * Mediawiki 1.27.1 release delayed to next week * captcha improvements ** https://phabricator.wikimedia.org/T125132 *** Matt mentions that Collaboration team owns users and logins *** Matt is also doing some captcha refactoring: https://phabricator.wikimedia.org/T141300 ** https://phabricator.wikimedia.org/T141490 === Services === * Parsoid in beta migrated to Jessie and Node 4, prod happening this week, 2 nodes already migrated * Session storage / Auth service - discussion ongoing: https://phabricator.wikimedia.org/T140813 * Maps Cassandra cluster: need to move it to 2.2.6 * Service-runner supports building deploy repo on Mac OS X now === Architecture / ArchCom === ** **https://www.mediawiki.org/wiki/Architecture_committee/Status* <https://www.mediawiki.org/wiki/Architecture_committee/Status> ** Last week: ArchCom meetings 2016W30: 2016-07-27 (Wed) *** 1pm PDT (20 UTC) Planning meeting: [[Phab:*E236]]* *** 2pm PDT (21 UTC) IRC #wikimedia-office: [[Phab:*E237]]* ***** [[Phab:T128602]] - *Create and deploy an extension that implements an authenticated key-value store ** This week: ArchCom meetings 2016W31: 2016-08-03 (Wed) *** 1pm PDT (20 UTC) Planning meeting: [[Phab:E250]] *** 2pm PDT (21 UTC) IRC #wikimedia-office [[Phab:*E251]]* ***** [[Phab:T128351]]: Notifications in core* === RelEng === * '''Blocked''' ** None * '''Blocking''' ** None? * '''Updates''' ** New Scap release that uses logstash_checker.py coming—live on beta already test please ** Lots of folks out this week, next week less folks out. == Wikidata == * structured commons demo: https://commons.wikimedia.org/wiki/Commons_talk:Structured_data#It.27s_aliv… * deploying ArticlePlaceholder on more wikis (cywiki - Welsh) - https://phabricator.wikimedia.org/T140725 == Fundraising Tech == * upgraded to mw 1.27 running on new hardware * Installing new Redis cluster to replace ActiveMQ * Rewriting queue code to work with Redis * CiviCRM performance tuning * CentralNotice cookie cleanup

1 0

Re: [Wikitech-l] [WikimediaMobile] Redlinks in Android app
by Jon Robson 03 Aug '16

03 Aug '16

Please also note the parent bug and rfc which blocks that. I've cced wikitech in case anyone has an update. https://phabricator.wikimedia.org/T39902 On 3 Aug 2016 5:14 a.m., "Stephen Niedzielski" <sniedzielski(a)wikimedia.org> wrote: > Hello! Thank you for the report! This issue is currently being tracked > here[0]. We had hoped to fix this properly but maybe we should consider a > temporary change in the client to handle this more gracefully. > > [0] https://phabricator.wikimedia.org/T119266 > > On Wed, Aug 3, 2016 at 4:21 AM, Andy Mabbett <andy(a)pigsonthewing.org.uk> > wrote: > >> The Android Wikipedia app shows Wikipedia's red links as blue links, >> then returns a user-unfriendly 404 error page, with the somewhat >> dishonest text "an unknown error occurred". >> >> Alternative, and better, behaviours would be one of (in no particular >> order): >> >> * Show a red link; behave like desktop Wikipedia >> >> * Show a red link; when the link is selected, show a customised 404 >> page, which advises users how start a new article >> >> * Show red text, but make it non linking (i.e. "unclickable") >> >> * Remove the link and colour-styling, and render ordinary text >> >> * Fix the erorr page so it says "Sorry, that page does not exist", >> with a link to a guide to starting a new article >> >> -- >> Andy Mabbett >> @pigsonthewing >> http://pigsonthewing.org.uk >> >> _______________________________________________ >> Mobile-l mailing list >> Mobile-l(a)lists.wikimedia.org >> https://lists.wikimedia.org/mailman/listinfo/mobile-l >> > > > _______________________________________________ > Mobile-l mailing list > Mobile-l(a)lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/mobile-l > >

1 0

Save the Date: Wikimedia Developer Summit 2017
by Rachel Farrand 03 Aug '16

03 Aug '16

Hello Everyone! The Wikimedia Developer Summit will be taking place in San Francisco, CA, USA between January 9th - January 11th. If you are interested in attending this year please mark the date on your calendars. More details will be added here soon about location, scholarships, registration, accommodation, etc. https://www.mediawiki.org/wiki/Wikimedia_Developer_Summit_2017?venotify=cre… Please help by forwarding this to any other lists that might be interested in this event.

1 0

Dump frequency
by Bináris 03 Aug '16

03 Aug '16

Hi folks, we in Hungarian Wikipedia have been watching new huwiki dumps by bot since 2011, so this page history: https://hu.wikipedia.org/w/index.php?title=Sablon:A_dump_d%C3%A1tuma&offset… clearly shows the freqency. Back in 2012 it took 8-10 days to create the new dump. Now it takes one month. Are we less develeoped or do we have less hardware than 4 years ago? -- Bináris

2 1

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Wikitech-l August 2016