Wikidata January 2016

wikidata@lists.wikimedia.org

48 participants
48 discussions

We plan to drop the wb_entity_per_page table
by hoo 01 Jun '17

01 Jun '17

Hey folks, we plan to drop the wb_entity_per_page table sometime soon[0], because it is just not required (as we will likely always have a programmatic mapping from entity id to page title) and it does not supported non -numeric entity ids as it is now. Due to this removing it is a blocker for the commons metadata. Is anybody using that for their tools (on tool labs)? If so, please tell us so that we can give you instructions and a longer grace period to update your scripts. Cheers, Marius [0]: https://phabricator.wikimedia.org/T95685

5 5

People who died in 2015 who were Dutch
by Gerard Meijssen 31 Aug '16

31 Aug '16

Hoi, Jura1 created a wonderful list of people who died in Brazil in 2015 [1]. It is a page that may update regularly from Wikidata thanks to the ListeriaBot. Obviously, there may be a few more because I am falling ever more behind with my quest for registering deaths in 2015. I have copied his work and created a page for people who died in the Netherlands in 2015 [2]. It is trivially easy to do this and, the result is great. The result looks great, it can be used for any country in any Wikipedia The Dutch Wikipedia indicated that they nowadays maintain important metadata at Wikidata. I am really happy that we can showcase their work. It is important work because as someone reminded me at some stage, this is part of what amounts to the policy of living people... Thanks, GerardM [1] https://www.wikidata.org/wiki/User:Jura1/Recent_deaths_in_Brazil [2] https://www.wikidata.org/wiki/User:Jura1/Recent_deaths_in_the_Netherlands

10 35

Photographers' Identities Catalog (& WikiData)
by David Lowe 04 Apr '16

04 Apr '16

Hello all, The Photographers' Identities Catalog (PIC) is an ongoing project of visualizing photo history through the lives of photographers and photo studios. I have information on 115,000 photographers and studios as of tonight. It is still under construction, but as I've almost completed an initial indexing of the ~12,000 photographers in WikiData, I thought I'd share it with you. We (the New York Public Library) hope to launch it officially in mid to late January. This represents about 12 years worth of my work of researching in NYPL's photography collection, censuses and business directories, and scraping or indexing trusted websites, databases, and published biographical dictionaries pertaining to photo history. Again, please bear in mind that our programmer is still hard at work (and I continue to refine and add to the data*), but we welcome your feedback, questions, critiques, etc. To see the WikiData photographers, select WikiData from the Source dropdown. Have fun! *PIC* <http://mgiraldo.github.io/pic/?address.AddressTypeID=*&address.CountryID=*&…> Thanks, David *Tomorrow, for instance, I'll start mining Wikidata for birth & death locations.

9 24

Freebase to Wikidata: Results from Tpt internship
by Denny Vrandečić 23 Feb '16

23 Feb '16

Hi all, as you know, Tpt has been working as an intern this summer at Google. He finished his work a few weeks ago and I am happy to announce today the publication of all scripts and the resulting data he has been working on. Additionally, we publish a few novel visualizations of the data in Wikidata and Freebase. We are still working on the actual report summarizing the effort and providing numbers on its effectiveness and progress. This will take another few weeks. First, thanks to Tpt for his amazing work! I have not expected to see such rich results. He has exceeded my expectations by far, and produced much more transferable data than I expected. Additionally, he also was working on the primary sources tool directly and helped Marco Fossati to upload a second, sports-related dataset (you can select that by clicking on the gears icon next to the Freebase item link in the sidebar on Wikidata, when you switch on the Primary Sources tool). The scripts that were created and used can be found here: https://github.com/google/freebase-wikidata-converter All scripts are released under the Apache license v2. The following data files are also released. All data is released under the CC0 license (in order to make this explicit, a comment has been added to the start of each file, stating the copyright and the license. If any script dealing with the files hiccups due to that line, simply remove the first line). https://tools.wmflabs.org/wikidata-primary-sources/data/freebase-mapped-mis… The actual missing statements, including URLs for sources, are in this file. This was filtered against statements already existing in Wikidata, and the statements are mapped to Wikidata IDs. This contains about 14.3M statements (214MB gzipped, 831MB unzipped). These are created using the mappings below in addition to the mappings already in Wikidata. The quality of these statements is rather mixed. Additional datasets that we know meet a higher quality bar have been previously released and uploaded directly to Wikidata by Tpt, following community consultation. https://tools.wmflabs.org/wikidata-primary-sources/data/additional-mapping.… Contains additional mappings between Freebase MIDs and Wikidata QIDs, which are not available in Wikidata. These are mappings based on statistical methods and single interwiki links. Unlike the first set of mappings we had created and published previously (which required multiple interwiki links at least), these mappings are expected to have a lower quality - sufficient for a manual process, but probably not sufficient for an automatic upload. This contains about 3.4M mappings (30 MB gzipped, 64MB unzipped). https://tools.wmflabs.org/wikidata-primary-sources/data/freebase-new-labels… This file includes labels and aliases for Wikidata items which seem to be currently missing. The quality of these labels is undetermined. The file contains about 860k labels in about 160 languages, with 33 languages having more than 10k labels each (14MB gzipped, 32MB unzipped). https://tools.wmflabs.org/wikidata-primary-sources/data/freebase-reviewed-m… This is an interesting file as it includes a quality signal for the statements in Freebase. What you will find here are ordered pairs of Freebase mids and properties, each indicating that the given pair were going through a review process and likely have a higher quality on average. This is only for those pairs that are missing from Wikidata. The file includes about 1.4M pairs, and this can be used for importing part of the data directly (6MB gzipped, 52MB unzipped). Now anyone can take the statements, analyse them, slice and dice them, upload them, use them for your own tools and games, etc. They remain available through the primary sources tool as well, which has already led to several thousand new statements in the last few weeks. Additionally, Tpt and I created in the last few days of his internship a few visualizations of the current data in Wikidata and in Freebase. First, the following is a visualization of the whole of Wikidata: https://tools.wmflabs.org/wikidata-primary-sources/data/wikidata-color.png The visualization needs a bit of explanation, I guess. The y-axis (up/down) represents time, the x-axis (left/right) represents space / geolocation. The further down, the closer you are to the present, the further up the more you go in the past. Time is given in a rational scale - the 20th century gets much more space than the 1st century. The x-axis represents longitude, with the prime meridian in the center of the image. Every item is being put at its longitude (averaged, if several) and at its earliest point of time mentioned on the item. For items without either, neighbouring items propagate their value to them (averaging, if necessary). This is done repeatedly until the items are saturated. In order to understand that a bit better, the following image offers a supporting grid: each line from left to right represents a century (up to the first century), and each line from top to bottom represent a meridian (with London in the middle of the graph). https://tools.wmflabs.org/wikidata-primary-sources/data/wikidata-grid-color… The same visualizations has also been created for Freebase: https://tools.wmflabs.org/wikidata-primary-sources/data/freebase-color.png https://tools.wmflabs.org/wikidata-primary-sources/data/freebase-grid-color… In order to compare the two graphs, we also overlaid them over each other. I will leave the interpretation to you, but you can easily see the strengths of weaknesses of both knowledge bases. https://tools.wmflabs.org/wikidata-primary-sources/data/wikidata-red-freeba… https://tools.wmflabs.org/wikidata-primary-sources/data/freebase-red-wikida… The programs for creating the visualizations are all available in the Github repository mentioned above (plenty of RAM is recommended to run it). Enjoy the visualizations, the data and the script! Tpt and I are available to answer questions. I hope this will help with understanding and analysing some of the results of the work that we did this summer. Cheers, Denny

10 10

Wikidata - short biographies
by Hampton Snowball 02 Feb '16

02 Feb '16

Hello, I am interested in a subset of wikidata and I am trying to find the best way to get it without getting a larger dataset then necessary. Is there a way to just get the "bios" that appear on the wikidata pages below the name of the person/organization, as well as the link to the english wikipedia page / or all wikipedia pages? For example from: https://www.wikidata.org/wiki/Q1652291" "Turkish female given name" https://en.wikipedia.org/wiki/H%C3%BClya and optionally https://de.wikipedia.org/wiki/H%C3%BClya I know there is SPARQL which previously this list helped me construct a query, but I know some requests seem to timeout when looking at a large amount of data so I am not sure this would work. The dumps I know are the full dataset, but I am not sure if there's any other subset dumps available or better way of grabbing this data Thanks in advance, HS

4 15

enabling in other projects sidebar on all wikis in January
by Lydia Pintscher 01 Feb '16

01 Feb '16

Hey folks :) The in other projects sidebar is a feature that adds links to articles on other projects to the sidebar. So for example on the Wikivoyage article about Berlin it links to the respective content on Wikipedia, Commons and so on.This works similar to the language links that you already have in the sidebar of an article. It is intended to make more content available to our users and bring our projects closer together. The in other projects sidebar has been around for many months now. It has been a beta feature on all wikis and default on a handful. It can't stay a beta feature forever and we should decide on its faith now. I am happy with the reactions we have been getting about it and believe it adds significant value for our readers and editors on smaller projects. I therefor want to enable it on all projects. This will happen in January. The exact date is still being decided. If for some reason your project decides it does not want this feature please let me know and we'll disable it for you. The ticket for tracking this is https://phabricator.wikimedia.org/T103102 Cheers Lydia -- Lydia Pintscher - http://about.me/lydia.pintscher Product Manager for Wikidata Wikimedia Deutschland e.V. Tempelhofer Ufer 23-24 10963 Berlin www.wikimedia.de Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e. V. Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 Nz. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/681/51985.

2 3

Internal server error after edit / create
by Jérémy Kondi 01 Feb '16

01 Feb '16

Hello everyone, I'm having some issues with my wikidata installation, any help would be appreciated. I'm getting stuck on an "Internal server error" after a Mediawiki + Wikibase repo installation via GIT + update of wikibase via composer, the wikibase is functional but after adding 1 element or 1 property, any other add of elements returns an Internal server error, and I have to wait for several minutes before it's fine again, for only 1 addition of element and then error again. I have no particular errors in my php logs or apache logs, just - error 500. Here are the versions installed : MediaWiki 1.27alpha (b3a82ca) PHP 5.4.45-0+deb7u2 (fpm-fcgi) MySQL 5.5.46-0+deb7u1-log ICU 4.8.1.1 Wikibase Repository 0.5 alpha (62c267) I can add some wiki pages (non-wikibase stuff), modify them, no problems. When I try to add an element or property, the first time I try it works, then a few seconds later I can't add a second element I get an "error 500 - internal server error", and I can't get it work until about 30 minutes / 1 hour then I can add another element and then again nothing ! I think I might have a namespace problem but how to configure it and keep my other wiki pages ? Thanks in advance ! Regards, Jeremy

3 2

Maintenance scripts for clients
by John Erling Blad 01 Feb '16

01 Feb '16

We lack several maintenance scripts for the clients, that is human readable special pages with reports on which pages lacks special treatment. In no particular order we need some way to identify unconnected pages in general (the present one does not work [1]), we need some way to identify pages that are unconnected but has some language links, we need to identify items that are used in some language and lacks labels (almost like [2],but on the client and for items that are somehow connected to pages on the client), and we need to identify items that lacks specific claims and the client pages use a specific template. There are probably more such maintenance pages, these are those that are most urgent. Now users start to create categories to hack around the missing maintenance pages, which create a bunch of categories.[3] At Norwegian Bokmål there are just a few scripts that utilize data from Wikidata, still the number of categories starts to grow large. For us at the "receiving end" this is a show stopper. We can't convince the users that this is a positive addition to the pages without the maintenance scripts, because them we more or less are in the blind when we try to fix errors. We can't use random pages to try to prod the pages to find something that is wrong, we must be able to search for the errors and fix them. This summer we (nowiki) have added about ten (10) properties to the infobokses, some with scripts and some with the property parser function. Most of my time I have not been coding, and I have not been fixing errors. I have been trying to explain to the community why Wikidata is a good idea. At one point the changes was even reverted because someone disagree with what we had done. The whole thing basically revolves around "my article got an Q-id in the infobox and I don't know how to fix it". We know how to fix it, and I have explained that to the editors at nowiki several times. They still don't get it, so we need some way to fix it, and we don't have maintenance scripts to do it. Right now we don't need more wild ideas that will swamp the development for months and years to come, we need maintenance scripts, and we need them now! [1] https://no.wikipedia.org/wiki/Spesial:UnconnectedPages [2] https://www.wikidata.org/wiki/Special:EntitiesWithoutLabel [3] https://no.wikipedia.org/wiki/Spesial:Prefiksindeks/Kategori:Artikler_hvor John Erling Blad /jeblad

4 7

Goal: Establish a framework to engage with data engineers and open data organizations
by Quim Gil 01 Feb '16

01 Feb '16

Hi, it's first of July and I would like to introduce you a quarterly goal that the Engineering Community team has committed to: Establish a framework to engage with data engineers and open data organizations https://phabricator.wikimedia.org/T101950 We are missing a community framework allowing Wikidata content and tech contributors, data engineers, and open data organizations to collaborate effectively. Imagine GLAM applied to data. If all goes well, by the end of September we would like to have basic documentation and community processes for open data engineers and organizations willing to contribute to Wikidata, and ongoing projects with one open data org. If you are interested, get involved! We are looking for * Wikidata contributors with good institutional memory * people that has been in touch with organizations willing to contribute their open data * developers willing to help improving our software and programming missing pieces * also contributors familiar with the GLAM model(s), what works and what didn't work This goal has been created after some conversations with Lydia Pintscher (Wikidata team) and Sylvia Ventura (Strategic Partnerships). Both are on board, Lydia assuring that this work fits into what is technically effective, and Sylvia checking our work against real open data organizations willing to get involved. This email effectively starts the bootstrapping of this project. I will start creating subtasks under that goal based on your feedback and common sense. -- Quim Gil Engineering Community Manager @ Wikimedia Foundation http://www.mediawiki.org/wiki/User:Qgil

12 19

help needed with classifying a bunch of edits
by Lydia Pintscher 29 Jan '16

29 Jan '16

Hey everyone :) Amir, Aaron and others are working hard to make it easier to detect vandalism on Wikidata with the help of ORES. It classifies edits to figure out for example which are the ones that are likely to be vandalism so we can concentrate on those when doing recent changes paroling and so on. It is getting close to having this work in an extension we can deploy on Wikidata so it is all nicely integrated. One of the missing pieces is classifying a bunch more edits to train the classifier. If you can help with getting this done that'd be awesome. Here is the campaign: https://www.wikidata.org/wiki/Wikidata:Edit_labels. Cheers Lydia -- Lydia Pintscher - http://about.me/lydia.pintscher Product Manager for Wikidata Wikimedia Deutschland e.V. Tempelhofer Ufer 23-24 10963 Berlin www.wikimedia.de Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e. V. Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 Nz. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/029/42207.

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

Wikidata January 2016