Wikitech-l December 2016

wikitech-l@lists.wikimedia.org

83 participants
84 discussions

productivity of mediawiki developers
by rupert THURNER 29 May '17

29 May '17

hi, is there a statistics about mediawiki developer productivity? i just fell over a couple of pages and i am quite impressed i must say: * gabriel, https://github.com/gwicke, 2'300 commits last year * jeroen, https://github.com/JeroenDeDauw, 3'700 commits a year * ori, https://github.com/atdt, 1'700 * james, https://github.com/jdforrester, 1'200 * yuri, https://github.com/nyurik, 900 * matt: https://github.com/mattflaschen, 400 commits rupert

11 14

Parsoid: node 0.1x deprecated now; node 0.1x support will end March 31st, 2017
by Subramanya Sastry 07 Apr '17

07 Apr '17

The Parsing team at the Wikimedia Foundation that develops the Parsoid service is deprecating support for node 0.1x. Parsoid is the service that powers VisualEditor, Content Translation, and Flow. If you don't run a MediaWiki install that uses VisualEditor, then this announcement does not affect you. Node 0.10 has reached end of life on October 31st, 2016 [1] and node 0.12 is scheduled to reach end of life December 31st, 2016 [1]. Yesterday, we released a 0.6.1 debian package [2] and a 0.6.1 npm version of Parsoid [3]. This will be the last release that will have node 0.1x support. We'll continue to provide any necessary critical bug fixes and security fixes for the 0.6.1 release till March 31st 2017 and will be completely dropping support for all node versions before node v4.x starting April 2017. If you are running a Parsoid service on your wiki and are still using node 0.1x, please upgrade your node version by April 2017. The Wikimedia cluster runs node v4.6 right now and will soon be upgraded to node v6.x [4]. Parsoid has been tested with node 0.1x, node v4.x and node v6.x and works with all these versions. However, we are dropping support for node 0.1x right away from the master branch of Parsoid. Going forward, the Parsoid codebase will adopt ES6 features available in node v4.x and higher which aren't supported in node 0.1x and will constitute a breaking change. Subramanya Sastry (Subbu), Technical Lead and Manager, Parsing Team, Wikimedia Foundation. [1] Node.js Long Term Support schedule @ https://github.com/nodejs/LTS [2] https://www.mediawiki.org/wiki/Parsoid/Releases [3] https://www.npmjs.com/package/parsoid [4] https://phabricator.wikimedia.org/T149331

1 1

Fwd: [Commons-l] Programmatically categorizing media in the Commons with Machine Learning
by Pine W 03 Apr '17

03 Apr '17

Forwarding. Pine ---------- Forwarded message ---------- From: "Jordan Adler" <jmadler(a)google.com> Date: Aug 11, 2016 13:06 Subject: [Commons-l] Programmatically categorizing media in the Commons with Machine Learning To: "commons-l(a)wikimedia.org" <commons-l(a)lists.wikimedia.org> Cc: "Ray Sakai" <rsakai(a)reactive.co.jp>, "Ram Ramanathan" < ramramanathan(a)google.com>, "Kazunori Sato" <kazsato(a)google.com> Hey folks! A few months back a colleague of mine was looking for some unstructured images to analyze as part of a demo for the Google Cloud Vision API <https://cloud.google.com/blog/big-data/2016/05/explore-the-galaxy-of-images…>. Luckily, I knew just the place <https://commons.wikimedia.org/wiki/Category:Media_needing_categories>, and the resulting demo <http://vision-explorer.reactive.ai/>, built by Reactive Inc., is pretty awesome. It was shared on-stage by Jeff Dean during the keynote <https://www.youtube.com/watch?v=HgWHeT_OwHc&feature=youtu.be&t=2h1m19s> at GCP NEXT 2016. I wanted to quickly share the data from the programmatically identified images so it could be used to help categorize the media in the Commons. There's about 80,000 images worth of data: - map.txt <https://storage.googleapis.com/gcs-samples2-explorer/reprocess/map.txt> (5.9MB): A single text file mapping id to filename in a "id : filename" format, one per line - results.tar.gz <https://storage.googleapis.com/gcs-samples2-explorer/reprocess/results.tar.…> (29.6MB): a tgz'd directory of json files representing the output of the API <https://cloud.google.com/vision/reference/rest/v1/images/annotate#response-…>, in the format "${id}.jpg.json" We're making this data available under the CC0 license, and these links will likely be live for at least a few weeks. If you're interested in working with the Cloud Vision API to tag other images in the Commons, talk to the WMF Community Tech team. Thanks for your help! _______________________________________________ Commons-l mailing list Commons-l(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/commons-l

2 1

Final Call for Comments on the Content Model Storage RFC
by Daniel Kinzler 17 Jan '17

17 Jan '17

Hi all! This is a Final Call for Comments on the RFC on Content Model Storage [1][2]. If no new and serious objections are raised within a week, the Architecture Committee will approve this RFC and drive its implementation. The RFC on Content Model Storage was originally approved in 2015, but was then postponed in favor of another RFC, which proposes to create a separate content meta-data table [3] as part of the move towards multi-content revisions (MCR) [4]. However, MCR in turn got stuck on database performance concerns. So we now propose to go ahead with implementing the original RFC. The idea is to assign a number to every content model (and content format), and then use these numbers to refer to the models and formats in the database, instead of repeating the same string millions of times (which is my fault btw, sorry about that). Since the original RFC was already approved, and the situation does not seem to have changed since then, we see no need for another round of discussions. If nobody raises any new and serious objections within a week, this should be good to go. Cheers, Daniel [1] https://phabricator.wikimedia.org/T105652 [2] https://www.mediawiki.org/wiki/Requests_for_comment/Content_model_storage [3] https://phabricator.wikimedia.org/T142980 [4] https://www.mediawiki.org/wiki/Multi-Content_Revisions/Content_Meta-Data#Da… -- Daniel Kinzler Senior Software Developer Wikimedia Deutschland Gesellschaft zur Förderung Freien Wissens e.V.

1 1

Final Call for Comments on the new Deprecation Policy for PHP code
by Daniel Kinzler 17 Jan '17

17 Jan '17

Hi all! At the ArchCom office hour (aka RFC IRC meeting) on December 14, we discussed Legoktm's proposal for a deprecation policy for PHP code [1]. This would supersede the current guideline [2], allowing deprecated code to be removed after a couple of releases, even if it may still used by some extensions. At the IRC meeting, we decided to approve this policy for a final call [3][4]. This means that it will be adopted as official policy if no new pertinent issues are raised within a week. Cheers, Daniel [1] https://www.mediawiki.org/wiki/Requests_for_comment/Deprecation_policy [2] https://www.mediawiki.org/wiki/Deprecation [3] https://tools.wmflabs.org/meetbot/wikimedia-office/2016/wikimedia-office.20… [4] https://tools.wmflabs.org/meetbot/wikimedia-office/2016/wikimedia-office.20… -- Daniel Kinzler Senior Software Developer Wikimedia Deutschland Gesellschaft zur Förderung Freien Wissens e.V.

1 1

Your questions and votes for our CTO and VP of Product Q&A
by Quim Gil 06 Jan '17

06 Jan '17

Hi, On the second day of the Wikimedia Developer Summit (January 10) there will be a Q&A session with Victoria Coleman (Wikimedia Foundation CTO) and Wes Moran (VP of Product). It is a plenary session and it will be video-streamed. The questions for this session are being crowdsourced at http://www.allourideas.org/wikidev17-product-technology-questions. Anyone can propose questions and vote, anonymously, as many times as you want. At the moment, we have 25 questions and 451 votes. An important technical detail: questions posted later have also good chances to make it to the top of the list as long as new voters select them. The ranking is made out of comparisons between questions, not accumulation of votes. For instance, the current top question is in fact one of the last that has been submitted so far. Why posting or voting a good question? One obvious reason is to encourage the Foundation's Technology and Product top managers to bring a good answer in a public session with minutes taken and video recording. :) Beyond that, if the ranking of questions makes sense and is backed by participation numbers, it has a serious chance to influence plans and discussions beyond the Summit. The current ranking does make sense, but maybe you could help covering more areas, other perspectives? 1. How do we deal with the lack of maintainers for all Wikimedia deployed code? 2. Do we have a plan to bring our developer documentation to the level of a top Internet website, a major free software project? 3. For WMF dev teams, what is the right balance between pushing own work versus seeking and supporting volunteer contributors? 4. During the next year or so, what balance do you think we should strike between new projects and technical debt? 5. When are we going to work on a modern talk pages system for good? 6. Whose responsibility is to assure that all MediaWiki core components and the extensions deployed in Wikimedia have active maintainers? 7. How important is to have a well maintained and well promoted catalog of tools, apps, gadgets, bots, templates, extensions...? 8. Will MediaWiki ever become easier to install and manage? (e.g. plugin manager à la Wordpress). How much do we care about enterprise users? 9. What should be the role of the Architecture Committee in WMF planning (priorities, goals, resources...) and are we there yet? 10. In addition to Community Tech, should the other WMF Product teams prioritize their work taking into account the Community Wishlist results? The full list: http://www.allourideas.org/wikidev17-product-technology-questions/results -- Quim Gil Engineering Community Manager @ Wikimedia Foundation http://www.mediawiki.org/wiki/User:Qgil

2 3

pywikibot troubleshooting in recitation-bot
by Anthony Di Franco 05 Jan '17

05 Jan '17

Hi all, I'm doing some renovations on recitation-bot and running into trouble when the time comes for pywikibot to upload article data to wikisource and commons. The thread doing so hangs without any sort of informative error. I made sure that the unix user under which the web service that is using pywikibot is running is logged into each wiki per Max's advice but I still have the problem. I'm going to try to get more information about what's going on but would also appreciate pointers about what might be going wrong. Particularly, the web service is now running under Kubernetes rather than sun grid engine, so I suspect that the login state might not be making it into the container - can anyone advise on where the login state is maintained and whether this will be transferred into the kubernetes container? Thanks, Anthony

2 2

I want to write an extension, let us consult its syntax and API
by Victor Porton 03 Jan '17

03 Jan '17

I want a small popup to be displayed when I hover over a link to a certain namespace. Note that the popup has nothing to do with the page to which it links. So for example, I want a link to A:X and when I hover the mouse over it I want to display B:X in a small popup (where A and B are namespaces). Note that B:X is a short page, it should be displayed in the popup completely (not just its announce or description). Note also that I need the popup to be "stable" enough, so that I could move the mouse (without the tooltip disappearing) and click a link in the tooltip. --- I am considering to write a new extension for this myself. But first I want to consult its syntax and API. For me personally the best mode of functionality is the following: In LocalSettings.php write an associative array mapping of namespaces: A=>B, etc. Then make popups for ALL links to objects in A (on every page which links to namespace A). However, other users may want different API and syntax, for example: <span popup="B:X">[[A:X|X]]</span> Please discuss with me which APIs and syntaxes should be available. -- Victor Porton - http://portonvictor.org

2 1

Now live: Shared structured data
by Yuri Astrakhan 30 Dec '16

30 Dec '16

Gift season! We have launched structured data on Commons, available from all wikis. TLDR; One data store. Use everywhere. Upload table data to Commons, with localization, and use it to create wiki tables, lists, or use directly in graphs. Works for GeoJSON maps too. Must be licensed as CC0. Try this per-state GDP map demo, and select multiple years. More demos at the bottom. US Map state highlight <https://en.wikipedia.org/wiki/Template:Graph:US_Map_state_highlight> Data can now be stored as *.tab and *.map pages in the data namespace on Commons. That data may contain localization, so a table cell could be in multiple languages. And that data is accessible from any wikis, by Lua scripts, Graphs, and Maps. Lua lets you generate wiki tables from the data by filtering, converting, mixing, and formatting the raw data. Lua also lets you generate lists. Or any wiki markup. Graphs can use both .tab and .map directly to visualize the data and let users interact with it. The GDP demo above uses a map from Commons, and colors each segment with the data based on a data table. Kartographer (<maplink>/<mapframe>) can use the .map data as an extra layer on top of the base map. This way we can show endangered species' habitat. == Demo == * Raw data example <https://commons.wikimedia.org/wiki/Data:Weather/New_York_City.tab> * Interactive Weather data <https://en.wikipedia.org/wiki/Template:Graph:Weather_monthly_history> * Same data in Weather template <https://en.wikipedia.org/wiki/User:Yurik/WeatherDemo> * Interactive GDP map <https://en.wikipedia.org/wiki/Template:Graph:US_Map_state_highlight> * Endangered Jemez Mountains salamander - habitat <https://en.wikipedia.org/wiki/Jemez_Mountains_salamander#/maplink/0> * Population history <https://en.wikipedia.org/wiki/Template:Graph:Population_history> * Line chart <https://en.wikipedia.org/wiki/Template:Graph:Lines> == Getting started == * Try creating a page at data:Sandbox/<user>.tab on Commons. Don't forget the .tab extension, or it won't work. * Try using some data with the Line chart graph template A thorough guide is needed, help is welcome! == Documentation links == * Tabular help <https://www.mediawiki.org/wiki/Help:Tabular_Data> * Map help <https://www.mediawiki.org/wiki/Help:Map_Data> If you find a bug, create Phabricator ticket with #tabular-data tag, or comment on the documentation talk pages. == FAQ == * Relation to Wikidata: Wikidata is about "facts" (small pieces of information). Structured data is about "blobs" - large amounts of data like the historical weather or the outline of the state of New York. == TODOs == * Add a nice "table editor" - editing JSON by hand is cruel. T134618 * "What links here" should track data usage across wikis. Will allow quicker auto-refresh of the pages too. T153966 * Support data redirects. T153598 * Mega epic: Support external data feeds.

4 11

Discovery Weekly Update for the week starting 2016-12-19 ⛄️
by Chris Koerner 23 Dec '16

23 Dec '16

Season's Greetings, A few updates from the Discovery department this week. This is the last weekly update from the Discovery department for the year. We'll be skipping next week due to the holiday's and will see you all in January with a fresh 2017 edition. == Highlights == * Secondary result functionality will be available over the search API in early January! Currently, this allows consumers of the search API to benefit from automated language detection [[TextCat]] and forwarding of search queries. [0] [1] == Discussions == === Search === * Secondary result functionality will be available over the search API in early January! Currently, this allows consumers of the search API to benefit from automated language detection [[TextCat]] and forwarding of search queries. [0] [1] * Corrected an error on Hebrew wikis where searches without diacritics could sometimes not find appropriate results that contained diacritics. [3] Feedback and suggestions on this weekly update are welcome. [0] https://www.mediawiki.org/wiki/TextCat [1] https://phabricator.wikimedia.org/T142795 [3] https://phabricator.wikimedia.org/T3836 ---- The full update, and archive of all past updates, can be found on MediaWiki.org: https://www.mediawiki.org/wiki/Discovery/Status_updates Interested in getting involved? See tasks marked as "Easy" or "Volunteer needed" in Phabricator. [1] https://phabricator.wikimedia.org/maniphest/query/qW51XhCCd8.7/#R [2] https://phabricator.wikimedia.org/maniphest/query/5KEPuEJh9TPS/#R Yours, Chris Koerner Community Liaison - Discovery Wikimedia Foundation

1 0

← Newer
1
2
3
4
5
6
7
8
9
Older →

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Wikitech-l December 2016