Wikitech-l August 2016

wikitech-l@lists.wikimedia.org

78 participants
104 discussions

Introduction for Gsoc 2017
by MAYANK JINDAL 10 Dec '16

10 Dec '16

Hi, My name is Mayank Jindal. I am third year undergraduate student currently studying at Indian Institute of Technology, Kharagpur. I want to take part in Gsoc-2017 from Wikimedia. I have knowledge of C, C++, JAVA, Python, Android app development and Web development and beginner in *Machine Learning, Artificial Intelligence.* I am very enthusiastic to learn new skills which would be required. Kindly guide me to proceed further. -- Kind Regards, Mayank Jindal, Third year undergraduate student, Indian Institute of Technology Kharagpur Mobile : +91- 7076670299 || 8875432718

4 5

Deploying the UrlShortener extension to Wikimedia wikis
by Legoktm 12 Nov '16

12 Nov '16

Hi, YuviPanda, prtksxna, and myself (with help from Tim and Aaron) have been working the UrlShortener extension, which is designed to implement the URL shortener RfC[1] (specifically Tim's implementation suggestion). I've filed T108557[2] to deploy the extension to Wikimedia wikis. We'd like to use the "w.wiki" short domain, which the WMF is already in control of. A test wiki has been set up mimicking what Wikimedia's configuration would be like: http://urlshortener.wmflabs.org/, and has an accompanying "short" domain at us.wmflabs.org (e.g. http://us.wmflabs.org/3). Please play with it and report any bugs you might find :) [1] https://www.mediawiki.org/wiki/Requests_for_comment/URL_shortener [2] https://phabricator.wikimedia.org/T108557 Thanks, -- Legoktm

3 5

Need review on ContentHandler implementation, Newsletter extension
by Tony Thomas 18 Oct '16

18 Oct '16

Hello all, We have a patch https://gerrit.wikimedia.org/r/#/c/295670/, which had its last review almost 28 days back, and is a major blocker for the deployment of Newsletter extension in production. The shift is tracked at T138462[1]. The patch is bit lengthy, and enable the extension to use ContentHandler, which help us use a lot of in-wiki features. I had pulled the change to the labs wiki at http://newsletter-test.wmflabs.org/, which is broken as of now, tracked in T138686[2]. It would be great if you devs can take a look at both the labs instance and the contenthandler change. [1] https://phabricator.wikimedia.org/T138462 [1] https://phabricator.wikimedia.org/T138686 Thanks, Tony Thomas <https://www.mediawiki.org/wiki/User:01tonythomas> Home <http://www.thomastony.me> | Blog <http://blog.thomastony.me> | ThinkFOSS <http://www.thinkfoss.com>

2 3

Phab blogs
by Amir E. Aharoni 19 Sep '16

19 Sep '16

Hi, I only now noticed that Phabricator has blogs: https://phabricator.wikimedia.org/phame/blog/ I couldn't find a way to subscribe to them in RSS. Is it possible? -- Amir Elisha Aharoni · אָמִיר אֱלִישָׁע אַהֲרוֹנִי http://aharoni.wordpress.com ‪“We're living in pieces, I want to live in peace.” – T. Moore‬

6 6

Shutting down deprecated rest.wikimedia.org by September 1st
by Gabriel Wicke 06 Sep '16

06 Sep '16

We are going to finally switch off the deprecated domain rest.wikimedia.org by September 1st. This should not affect any REST API users, as this domain has been officially deprecated since January & sunset since April [1]. Since then, requests to that domain have returned an error informing users about the move. Access to the REST API is exclusively through the main project domains, following the following pattern: https://en.wikipedia.org/api/rest_v1/?doc Thank you for your cooperation, -- Gabriel Wicke Principal Engineer, Wikimedia Foundation [1]: https://lists.wikimedia.org/pipermail/wikitech-l/2016-April/085309.html

2 1

Browser testing user satisfaction survey
by Željko Filipin 06 Sep '16

06 Sep '16

In order to improve browser testing tools, release engineering team has created browser testing user satisfaction survey: https://goo.gl/xS6mmV It should take you up to 5 minutes. Most of the questions have simple 5 level linear scale. There are 5 sections, and the last question in each section will be free form text field, so you can leave comments on anything we forgot to ask. For details about the survey, feel free to take a look at phabricator task: https://phabricator.wikimedia.org/T131123 Željko

3 7

Re: [Wikitech-l] Parsoid Exception HTTP 500
by Julian Loferer 03 Sep '16

03 Sep '16

Yeah it looks similar. The link direct me to the right page.

2 6

Grouping phabricator notifications by task
by Joaquin Oltra Hernandez 02 Sep '16

02 Sep '16

Hi, I rely quite a lot on phabricator notifications for checking new activity on my subscribed tasks (I've found to suit me better than email). After using the unread view in phab quite a lot I noticed that I really wanted to see the notifications grouped by the task, since that let's me see the new activity on the task at a glance and open it to read/act if I need to. So I've made a user script that does exactly that: https://gist.github.com/joakin/c0e5dffc23aaf05175a580d24a2adefe There's a gif of what this does in the README and the code is really short. I hope this is useful to some of you, it certainly has been making my life easier. I apply it to https://phabricator.wikimedia.org/notification/query/unread/ only, but I guess there's no reason why it wouldn't work in https://phabricator.wikimedia.org/notification/* Cheers.

3 2

<maplink> to be enabled tomorrow
by Yuri Astrakhan 02 Sep '16

02 Sep '16

Seems we have no more blockers for <maplink> tag on all Wikipedias. We will enable it tomorrow, Sept 1st. <maplink> tag allows editors to add a link to a popup map, complete with extra geojson overlay data. See help page for instructions: https://www.mediawiki.org/wiki/Help:Extension:Kartographer#.3Cmaplink.3E Submit bugs to phabricator: https://phabricator.wikimedia.org/maniphest/task/edit/form/1/?tags=Kartogra…

3 3

2016W35 ArchCom-RFC meeting: Revisiting image/oldimage (T589)
by Rob Lanphier 31 Aug '16

31 Aug '16

Hi everyone, For this week's office hour, we'd like to discuss [T589: schema change for page content language][1] Timo brought this up on the list a couple of weeks ago, and didn't get much of a response. (But thank you Jaime for responding!) We discussed this one fairly recently ([July 13][2]). This area of our system is still in need of simplification and optimization. The discussion is scheduled for 2016-08-31 UTC: Time: Wednesday 21 UTC (2pm PDT, 23 CEST) Place: #wikimedia-office Phab event: [E266][3] [ArchCom/Status][4] Rob [1]: <https://phabricator.wikimedia.org/T589> [3]: <https://phabricator.wikimedia.org/E228> July 13 meeting [3]: <https://phabricator.wikimedia.org/E266> Upcoming meeting [4]: <https://www.mediawiki.org/wiki/Architecture_committee/Status> ---------- Forwarded message ---------- From: Krinkle <krinklemail(a)gmail.com> Date: Wed, Aug 10, 2016 at 1:54 PM Subject: [Wikitech-l] Schema migration for 'image' and 'oldimage' tables To: Wikimedia developers <wikitech-l(a)lists.wikimedia.org> TL;DR: Participate on T589 and help decide what the upcoming schema change should entail, and how we'll migrate existing data. Hey all, Couple weeks ago we dedicated an IRC office hour to https://phabricator.wikimedia.org/T589 (RFC: image and oldimage tables). Updated draft at: https://www.mediawiki.org/wiki/Requests_for_comment/image_and_oldimage_tabl… We clarified scope and purpose of this particular RFC. Other issues are still important but considered orthogonal, and to be dealt with parallelly (or at a later time). Revised problem statement: 1. File revisions should have unique identifiers (better than "current file title + upload timestamp". (Subject to race conditions, hard to index/query, etc.) 2. Uploading new file revisions must not involve rows moving across tables, or rows being replaced. Participants agreed with the revised problem statement, it makes sense not to merely add primary keys to the existing tables ("Proposal 1" on the RFC draft), as that wouldn't adequately solve the Problem 2. The second proposal was to separate information about image revision from the image entity itself. Similar to the page/revisions tables. This was generally accepted as a good idea, but details are still to be determined. The general idea is that all revision-specific information (except for a pointer to the current revision) would no longer live in the 'image' table. Instead, information about all (for both current and past revisions) would live in the same table (instead of being moved around from one table to another when it's no longer the current one). Details at: https://www.mediawiki.org/wiki/Requests_for_comment/image_and_oldimage_tabl… Some open questions I'd like to see discussed on Phabricator (or here on wikitech): 1. Which fields do we keep in the 'image' table (img_id, img_name, img_latest, anything else?). All fields currently being queried from both tables, will probably only stay in the image revision table. But there are a few fields that we intentionally only want to query about current versions. For example 'img_sha1'. For duplicate-detection, we need to only consider the latest revisions. Doing this by keeping img_sha1 means uploading a new revision will involve updating two fields instead of one (img_latest and img_sha1). This isn't unprecedented as we do this for page as well (WikiPage::updateRevisionOn; page_latest, page_touched, page_is_redirect, page_len). Are there other fields we need to keep besides img_sha1? Or should we can solve the img_sha1 use case in a different manner? 2. img_metadata This field is a blob of serialised PHP (typically representing the Exif data of an image). Tim (correct me if I got it wrong) mentioned we could potentially make migration easier by changing img_metadata to be stored in a separate table and change the img_metadata field (in the image revision table) to instead be a pointer to a primary key. This could potentially be done separately later, but if it helps migration, we should consider doing it now. How will this interact with file deletion? Will it be difficult to garbage collect this? Do we need to? (We don't seem to do it for the 'text' table / external store; is it worth moving this an external store?) 3. Migration If we rename both tables (image/oldimage -> file/filerevision), we'd have the ability to run migration in the background without interfering with the live site, and without requiring a long read-only period and/or duplicate and additional code complexity to be developed. Is there a way we can do the migration without creating two new tables? Using the oldimage table as import destination for current rows isn't straight forward as existing scan queries would need to skip the current rows somehow while in the midst of this migration. Seems possible, but is it worth the complexity? (We'd need extra code that knows about that migration field, and how long do we keep that code? Also complicates migration for third-parties using update.php). Is creating the new tables separately viable for the scale of Wikimedia Commons? (and dropping the old ones once finished). Is this a concern from a DBA perspective with regards to storage space? (We'd temporarily need about twice the space for these tables). So far I understood that it wouldn't be a problem per se, but that there are also other options we can explore for Wikimedia. For example we could use a separate set of slaves and alter those while depooled (essentially using entirely separate set of db slaves instead of a separate table within each slave). Do we create the new table(s) separately and switch over once it's caught up? This would require doing multiple passes as we depool slaves one by one (we've done that before at Wikimedia). Switch-over could be done by migrating before the software upgrade, with a very short read-only period after the last pass is finished. It wouldn't require maintaining multiple code paths, which is attractive. Other ideas? -- Timo _______________________________________________ Wikitech-l mailing list Wikitech-l(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l

1 1

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Wikitech-l August 2016