Wikitech-l December 2016

wikitech-l@lists.wikimedia.org

83 participants
79 discussions

Your questions and votes for our CTO and VP of Product Q&A
by Quim Gil 06 Jan '17

06 Jan '17

Hi, On the second day of the Wikimedia Developer Summit (January 10) there will be a Q&A session with Victoria Coleman (Wikimedia Foundation CTO) and Wes Moran (VP of Product). It is a plenary session and it will be video-streamed. The questions for this session are being crowdsourced at http://www.allourideas.org/wikidev17-product-technology-questions. Anyone can propose questions and vote, anonymously, as many times as you want. At the moment, we have 25 questions and 451 votes. An important technical detail: questions posted later have also good chances to make it to the top of the list as long as new voters select them. The ranking is made out of comparisons between questions, not accumulation of votes. For instance, the current top question is in fact one of the last that has been submitted so far. Why posting or voting a good question? One obvious reason is to encourage the Foundation's Technology and Product top managers to bring a good answer in a public session with minutes taken and video recording. :) Beyond that, if the ranking of questions makes sense and is backed by participation numbers, it has a serious chance to influence plans and discussions beyond the Summit. The current ranking does make sense, but maybe you could help covering more areas, other perspectives? 1. How do we deal with the lack of maintainers for all Wikimedia deployed code? 2. Do we have a plan to bring our developer documentation to the level of a top Internet website, a major free software project? 3. For WMF dev teams, what is the right balance between pushing own work versus seeking and supporting volunteer contributors? 4. During the next year or so, what balance do you think we should strike between new projects and technical debt? 5. When are we going to work on a modern talk pages system for good? 6. Whose responsibility is to assure that all MediaWiki core components and the extensions deployed in Wikimedia have active maintainers? 7. How important is to have a well maintained and well promoted catalog of tools, apps, gadgets, bots, templates, extensions...? 8. Will MediaWiki ever become easier to install and manage? (e.g. plugin manager à la Wordpress). How much do we care about enterprise users? 9. What should be the role of the Architecture Committee in WMF planning (priorities, goals, resources...) and are we there yet? 10. In addition to Community Tech, should the other WMF Product teams prioritize their work taking into account the Community Wishlist results? The full list: http://www.allourideas.org/wikidev17-product-technology-questions/results -- Quim Gil Engineering Community Manager @ Wikimedia Foundation http://www.mediawiki.org/wiki/User:Qgil

2 3

pywikibot troubleshooting in recitation-bot
by Anthony Di Franco 05 Jan '17

05 Jan '17

Hi all, I'm doing some renovations on recitation-bot and running into trouble when the time comes for pywikibot to upload article data to wikisource and commons. The thread doing so hangs without any sort of informative error. I made sure that the unix user under which the web service that is using pywikibot is running is logged into each wiki per Max's advice but I still have the problem. I'm going to try to get more information about what's going on but would also appreciate pointers about what might be going wrong. Particularly, the web service is now running under Kubernetes rather than sun grid engine, so I suspect that the login state might not be making it into the container - can anyone advise on where the login state is maintained and whether this will be transferred into the kubernetes container? Thanks, Anthony

2 2

I want to write an extension, let us consult its syntax and API
by Victor Porton 03 Jan '17

03 Jan '17

I want a small popup to be displayed when I hover over a link to a certain namespace. Note that the popup has nothing to do with the page to which it links. So for example, I want a link to A:X and when I hover the mouse over it I want to display B:X in a small popup (where A and B are namespaces). Note that B:X is a short page, it should be displayed in the popup completely (not just its announce or description). Note also that I need the popup to be "stable" enough, so that I could move the mouse (without the tooltip disappearing) and click a link in the tooltip. --- I am considering to write a new extension for this myself. But first I want to consult its syntax and API. For me personally the best mode of functionality is the following: In LocalSettings.php write an associative array mapping of namespaces: A=>B, etc. Then make popups for ALL links to objects in A (on every page which links to namespace A). However, other users may want different API and syntax, for example: <span popup="B:X">[[A:X|X]]</span> Please discuss with me which APIs and syntaxes should be available. -- Victor Porton - http://portonvictor.org

2 1

Now live: Shared structured data
by Yuri Astrakhan 30 Dec '16

30 Dec '16

Gift season! We have launched structured data on Commons, available from all wikis. TLDR; One data store. Use everywhere. Upload table data to Commons, with localization, and use it to create wiki tables, lists, or use directly in graphs. Works for GeoJSON maps too. Must be licensed as CC0. Try this per-state GDP map demo, and select multiple years. More demos at the bottom. US Map state highlight <https://en.wikipedia.org/wiki/Template:Graph:US_Map_state_highlight> Data can now be stored as *.tab and *.map pages in the data namespace on Commons. That data may contain localization, so a table cell could be in multiple languages. And that data is accessible from any wikis, by Lua scripts, Graphs, and Maps. Lua lets you generate wiki tables from the data by filtering, converting, mixing, and formatting the raw data. Lua also lets you generate lists. Or any wiki markup. Graphs can use both .tab and .map directly to visualize the data and let users interact with it. The GDP demo above uses a map from Commons, and colors each segment with the data based on a data table. Kartographer (<maplink>/<mapframe>) can use the .map data as an extra layer on top of the base map. This way we can show endangered species' habitat. == Demo == * Raw data example <https://commons.wikimedia.org/wiki/Data:Weather/New_York_City.tab> * Interactive Weather data <https://en.wikipedia.org/wiki/Template:Graph:Weather_monthly_history> * Same data in Weather template <https://en.wikipedia.org/wiki/User:Yurik/WeatherDemo> * Interactive GDP map <https://en.wikipedia.org/wiki/Template:Graph:US_Map_state_highlight> * Endangered Jemez Mountains salamander - habitat <https://en.wikipedia.org/wiki/Jemez_Mountains_salamander#/maplink/0> * Population history <https://en.wikipedia.org/wiki/Template:Graph:Population_history> * Line chart <https://en.wikipedia.org/wiki/Template:Graph:Lines> == Getting started == * Try creating a page at data:Sandbox/<user>.tab on Commons. Don't forget the .tab extension, or it won't work. * Try using some data with the Line chart graph template A thorough guide is needed, help is welcome! == Documentation links == * Tabular help <https://www.mediawiki.org/wiki/Help:Tabular_Data> * Map help <https://www.mediawiki.org/wiki/Help:Map_Data> If you find a bug, create Phabricator ticket with #tabular-data tag, or comment on the documentation talk pages. == FAQ == * Relation to Wikidata: Wikidata is about "facts" (small pieces of information). Structured data is about "blobs" - large amounts of data like the historical weather or the outline of the state of New York. == TODOs == * Add a nice "table editor" - editing JSON by hand is cruel. T134618 * "What links here" should track data usage across wikis. Will allow quicker auto-refresh of the pages too. T153966 * Support data redirects. T153598 * Mega epic: Support external data feeds.

4 11

Discovery Weekly Update for the week starting 2016-12-19 ⛄️
by Chris Koerner 24 Dec '16

24 Dec '16

Season's Greetings, A few updates from the Discovery department this week. This is the last weekly update from the Discovery department for the year. We'll be skipping next week due to the holiday's and will see you all in January with a fresh 2017 edition. == Highlights == * Secondary result functionality will be available over the search API in early January! Currently, this allows consumers of the search API to benefit from automated language detection [[TextCat]] and forwarding of search queries. [0] [1] == Discussions == === Search === * Secondary result functionality will be available over the search API in early January! Currently, this allows consumers of the search API to benefit from automated language detection [[TextCat]] and forwarding of search queries. [0] [1] * Corrected an error on Hebrew wikis where searches without diacritics could sometimes not find appropriate results that contained diacritics. [3] Feedback and suggestions on this weekly update are welcome. [0] https://www.mediawiki.org/wiki/TextCat [1] https://phabricator.wikimedia.org/T142795 [3] https://phabricator.wikimedia.org/T3836 ---- The full update, and archive of all past updates, can be found on MediaWiki.org: https://www.mediawiki.org/wiki/Discovery/Status_updates Interested in getting involved? See tasks marked as "Easy" or "Volunteer needed" in Phabricator. [1] https://phabricator.wikimedia.org/maniphest/query/qW51XhCCd8.7/#R [2] https://phabricator.wikimedia.org/maniphest/query/5KEPuEJh9TPS/#R Yours, Chris Koerner Community Liaison - Discovery Wikimedia Foundation

1 0

Correction on Structured Data on Commons project
by Katie Horn 23 Dec '16

23 Dec '16

Hello all, Yesterday, an announcement (Now live: Shared structured data), incorrectly stated that Structured Data had been launched on Commons. The feature which was inaccurately named “Structured Data”, enables users to add tabular data to the data namespace on Commons via the regular page editor and to further display and/or visualize that data from other wikis. This work is unrelated to an ongoing project called Structured Data on Commons. For more on the newly launched feature, see the Tabular Data [1] and Map Data [2] help pages on MediaWiki.org. For information on the Structured Data on Commons project, designed to associate structured data with media files on Commons to improve their discoverability, please visit the project page on Commons. [3] Thank you, -Katie [1] - https://www.mediawiki.org/wiki/Help:Tabular_Data [2] - https://www.mediawiki.org/wiki/Help:Map_Data [3] - https://commons.wikimedia.org/wiki/Commons:Structured_data

1 0

Offering internationalized programming facilities within WM enviroment
by mathieu stumpf guntz 23 Dec '16

23 Dec '16

Hello everybody, Following the suggestion of Andre Klapper <https://phabricator.wikimedia.org/T150933#2886290>, I'm turning to this set of lists to see if it can attract more feedback on the topic of internationalized programming facilities within WM environment <https://phabricator.wikimedia.org/T150933>. As described more extensively in the ticket,the idea is to implement internationalization facilities (if they don't exist yet) in compilers used in the WM infrastructure, and enable contributors to localize them (possibly through Translatewiki), and then let them use localized versions if they wish. Please let me know if you need more details or if you have any question. You can answer on the list or on phabricator, as you wish. Kind regards, Mathieu

7 9

Re: [Wikitech-l] [Maps-l] Now live: Shared structured data
by Susanna Ånäs 23 Dec '16

23 Dec '16

Great work! I'm happy with the new naming, Commons Datasets. For historical maps we have been waiting to have a way to store data about the rectification with the map image. Here it is! It brings us one notch closer to being able to work with zoomable historical maps in Wikipedia. Some have noted that the datasets are contrary to what Wikidata is about. Instead, they are complementary! Not all data is either suitable for Wikidata or licensed openly enough. Or not yet. Many great datasets can be introduced to the Wikimedia online communities. The data owners will pay attention to more open licensing, seeing their data being used. The wikidatans will pick up interesting datasets and work to prepare missing properties for them in Wikidata and ignite their bots. Sometimes the data can just be used as is. This is one part of the puzzle and it will be interesting to see how the pieces fall into their places and evolve further. In the coming few days there'll be time to digest and experiment. Happy holidays Susanna 2016-12-23 5:22 GMT+02:00 Bohdan Melnychuk <base-w(a)yandex.ru>: > Yay :) > > As someone who already has plans to actively use it in both my metapedian > role (e.g. CEE Spring article writing contest statistics data for building > Graphs from being stored like https://meta.wikimedia.org/w/ > index.php?title=User:BaseBot/CEES/MMXVI/Per_country_sums_( > general)&action=edit > <https://meta.wikimedia.org/w/index.php?title=User:BaseBot/CEES/MMXVI/Per_co…> > to a better format of https://commons.wikimedia.org/ > wiki/Data:Wikimedia/CEE_Spring/Statistics/MMXVI/Per_ > country_sums_(general).tab which can be turned by Lua to the same output > but now with it being controlled on wiki rather than bot code part) and > exopedianish for actual articles I think it is wonderful. > > I do think that it needs tight cross linking with Wikidata and perfectly a > way to run queries against both the sources at the same time (e.g. "give me > the weather in all the current capitals in the date the comet Whatever was > closest to the Sun the last time" or whatever else more useful thing may > come into one's mind), but that does not deny the fact that it is very > useful already. > > It can also be used as an intermediate location for data on the way to be > imported to Wikidata, IMHO. > > --Base > > > 22.12.2016, 21:31, "Yuri Astrakhan" <yastrakhan(a)wikimedia.org>: > > Gift season! We have launched structured data on Commons, available from > all wikis. > > TLDR; One data store. Use everywhere. Upload table data to Commons, with > localization, and use it to create wiki tables, lists, or use directly in > graphs. Works for GeoJSON maps too. Must be licensed as CC0. Try this > per-state GDP map demo, and select multiple years. More demos at the bottom. > US Map state highlight > <https://en.wikipedia.org/wiki/Template:Graph:US_Map_state_highlight> > > Data can now be stored as *.tab and *.map pages in the data namespace on > Commons. That data may contain localization, so a table cell could be in > multiple languages. And that data is accessible from any wikis, by Lua > scripts, Graphs, and Maps. > > Lua lets you generate wiki tables from the data by filtering, converting, > mixing, and formatting the raw data. Lua also lets you generate lists. Or > any wiki markup. > > Graphs can use both .tab and .map directly to visualize the data and let > users interact with it. The GDP demo above uses a map from Commons, and > colors each segment with the data based on a data table. > > Kartographer (<maplink>/<mapframe>) can use the .map data as an extra > layer on top of the base map. This way we can show endangered species' > habitat. > > == Demo == > * Raw data example > <https://commons.wikimedia.org/wiki/Data:Weather/New_York_City.tab> > * Interactive Weather data > <https://en.wikipedia.org/wiki/Template:Graph:Weather_monthly_history> > * Same data in Weather template > <https://en.wikipedia.org/wiki/User:Yurik/WeatherDemo> > * Interactive GDP map > <https://en.wikipedia.org/wiki/Template:Graph:US_Map_state_highlight> > * Endangered Jemez Mountains salamander - habitat > <https://en.wikipedia.org/wiki/Jemez_Mountains_salamander#/maplink/0> > * Population history > <https://en.wikipedia.org/wiki/Template:Graph:Population_history> > * Line chart <https://en.wikipedia.org/wiki/Template:Graph:Lines> > > == Getting started == > * Try creating a page at data:Sandbox/<user>.tab on Commons. Don't forget > the .tab extension, or it won't work. > * Try using some data with the Line chart graph template > A thorough guide is needed, help is welcome! > > == Documentation links == > * Tabular help <https://www.mediawiki.org/wiki/Help:Tabular_Data> > * Map help <https://www.mediawiki.org/wiki/Help:Map_Data> > If you find a bug, create Phabricator ticket with #tabular-data tag, or > comment on the documentation talk pages. > > == FAQ == > * Relation to Wikidata: Wikidata is about "facts" (small pieces of > information). Structured data is about "blobs" - large amounts of data like > the historical weather or the outline of the state of New York. > > == TODOs == > * Add a nice "table editor" - editing JSON by hand is cruel. T134618 > * "What links here" should track data usage across wikis. Will allow > quicker auto-refresh of the pages too. T153966 > * Support data redirects. T153598 > * Mega epic: Support external data feeds. > , > > _______________________________________________ > Maps-l mailing list > Maps-l(a)lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/maps-l > > > _______________________________________________ > Maps-l mailing list > Maps-l(a)lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/maps-l > >

1 0

Using OpenStreetMap with Wikipedia account
by Maarten Dammers 23 Dec '16

23 Dec '16

Hi, I noticed a tweet announcing the possibility using your Wikipedia account to edit OpenStreetMap. See the Wikipedia logo at https://www.openstreetmap.org/login . That's really nice! After a bit of digging I found the registration at https://meta.wikimedia.org/w/index.php?title=Special:OAuthListConsumers/vie… and looks like the consumer got one grant: "Applicable grants: User identity verification only with access to real name and email address via Special:OAuth/identify, no ability to read pages or act on a user's behalf." So when used, the emailaddress of the user is shared with a third party. Is this in line with the privacy policy ( https://wikimediafoundation.org/wiki/Privacy_policy ) ? The user isn't warned beforehand, you realize it after granting the right and you get to some registration process with your emailaddress already pre-filled. Don't we want to warn the user what rights are being granted before approving? The current message is "In order to complete your request, OpenStreetMap needs permission to access information on all projects of this site on your behalf. No changes will be made with your account. " Maarten

3 2

Re: [Wikitech-l] [discovery] Now live: Shared structured data
by Yuri Astrakhan 22 Dec '16

22 Dec '16

Svetlana, thanks for suggestion. I think we should create a portal similar to the Structured Data one, and put some examples there. Deciding on the name is difficult :) "Commons Datasets" does sound good. There has been a very prolonged discussion on where to host this feature - https://meta.wikimedia.org/wiki/User:Yurik/Storing_data. Wikidata would have been a good choice, but users expect all the data there to be in public domain, and we may add more licensing choices later. > An inline example with English commentary -- straight on the first page about this new technology without making users click links -- could be nice. The text you typed up does not seem to be on a wiki page, so I am unable to edit it... Which page are you referring to? On Thu, Dec 22, 2016 at 4:03 PM Svetlana Tkachenko <svetlana(a)members.fsf.org> wrote: > Hello, > > Maybe 'commons store' or 'commons datasets' could work? I would suggest > that the name reflects on the fact that the datasets are shared > ('common') and are not on Wikidata. > > If I may ask, why is it in commons.wikimedia.org/wiki/Data:* and not at > Meta (like Global user pages) or Wikidata (like structured data about > lots of things)? > > An inline example with English commentary -- straight on the first page > about this new technology without making users click links -- could be > nice. The text you typed up does not seem to be on a wiki page, so I am > unable to edit it... > > Svetlana. >

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Wikitech-l December 2016