Wikidata-tech

wikidata-tech@lists.wikimedia.org

626 discussions

[Significant change] New datatype fields in Lexeme JSON output
by Lucas Werkmeister 17 Aug '20

17 Aug '20

Hi all! This is an announcement for a significant (but not breaking) change to the JSON output of WikibaseLexeme entities, specifically of Senses and Forms, when obtained via Special:EntityData <https://www.wikidata.org/wiki/Special:EntityData> or the Wikibase API endpoints like Special:ApiHelp/wbgetentities <https://www.wikidata.org/wiki/Special:ApiHelp/wbgetentities>. The Snak output in the Wikibase JSON serialization <https://doc.wikimedia.org/Wikibase/master/php/md_docs_topics_json.html#json…> usually contains a datatype field for each Snak. Previously, these fields were missing for statements within Senses and Forms of a Lexeme – see the message on Project Chat <https://www.wikidata.org/wiki/Special:PermanentLink/1258923595#[Significant…> for an example (I didn’t want to bloat this email too much by including it here 🙂). Starting on 26 August 2020 (barring unexpected deployment issues; 25 August 2020 on Test Wikidata), these datatype fields will be present there as well. If you have any issue or question, feel free to leave a comment at T249206 <https://phabricator.wikimedia.org/T249206>. Cheers, Lucas -- Lucas Werkmeister (he/er) Full Stack Developer Wikimedia Deutschland e. V. | Tempelhofer Ufer 23-24 | 10963 Berlin Phone: +49 (0)30 219 158 26-0 https://wikimedia.de Imagine a world in which every single human being can freely share in the sum of all knowledge. Help us to achieve our vision! https://spenden.wikimedia.de Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e. V. Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/029/42207.

1 0

Problem with SPARQL query: items which use a specific image (P18)
by Stephan Bösch-Plepelits 24 Jul '20

24 Jul '20

Hi! I have a problem with a SPARQL query. Hopefully you can help me. I'd like to get wikidata articles which use a specific image in the P18 property. I'm using this simple query: SELECT ?item WHERE { ?item wdt:P18 "item.jpg". } This works for some items, but not all. Example 1: SELECT ?item WHERE { ?item wdt:P18 "1936-ChirkovAntonNikolaevich.jpg". } -> returns "wd:Q16718402" (https://www.wikidata.org/wiki/Q16718402) Example 2: SELECT ?item WHERE { ?item wdt:P18 "GuentherZ 2010-02-27 0207 Wien10 Scheunenstrasse2 Bildstock.jpg". } -> returns nothing, although it is used by https://www.wikidata.org/wiki/Q33231166 This might be related to the following fact: Example 1A: SELECT ?item ?image WHERE { ?item wdt:P18 ?image. ?item wdt:P214 "307433452". } -> the field "image" shows a string "1936-ChirkovAntonNikolaevich.jpg" Example 2A: SELECT ?item ?foo WHERE { ?item wdt:P18 ?image. ?item wdt:P2951 "86578". } -> the field "image" shows a link to https://commons.wikimedia.org/wiki/File:GuentherZ%202010-02-27%200207%20Wie… with the text "commons:GuentherZ 2010-02-27 0207 Wien10 Scheunenstrasse2 Bildstock.jpg" (I also tried prefixing the query image name by 'commons:', like "?item wdt:P18 "commons:image.jpg") Please tell me if you have an idea how to solve this problem! greetings, Stephan -- Seid unbequem, seid Sand, nicht Öl im Getriebe der Welt! - Günther Eich ,----------------------------------------------------------------------. | Stephan Bösch-Plepelits ❤ code ❤ urbanism ❤ free software ❤ cycling | | Projects: | | > OpenStreetMap: openstreetbrowser.org > openstreetmap.at | | > Urbanism: Radlobby Wien > Platz für Wien | | Contact: | | > Mail: skunk(a)xover.mud.at > Blog: plepe.at > Code: github.com/plepe | | > Twitter: twitter.com/plepe > Jabber: skunk(a)jabber.at | | > Mastodon: @plepe@en.osm.town | `----------------------------------------------------------------------'

2 1

[Config change] WikibaseClient example config will no longer define an example repo
by Lucas Werkmeister 20 Jul '20

20 Jul '20

Hi everyone! This is an announcement for a change to the example configuration of the WikibaseClient extension, which only affects Client wikis which are not also Repository wikis. If you don’t operate a non-Repo Client wiki, or if you don’t use the example settings, you can ignore this message. (Wikibase-Docker users, in particular, can ignore it.) The change will become effective on the master branch soon, and then in the future 1.36 release (but not yet in the upcoming 1.35 release). Currently, the WikibaseClient example settings (client/ExampleSettings.php or client/config/WikibaseClient.example.php) define an example Repo for the Client if the WikibaseRepository extension is not also being loaded. As we are moving towards using extension registration for Wikibase (T88258 <https://phabricator.wikimedia.org/T88258>), this will no longer be possible (extensions loaded using wfLoadExtension() are processed later, so the example settings file can’t detect if WikibaseRepo is loaded or not). Therefore, we will be removing this code from the Client example settings, and if your wiki is not also a Repo wiki, you will have to configure the Repo manually. (For wikis that are both Repo and Client, the default Client settings are sufficient, and nothing needs to be done.) This means that if your configuration (LocalSettings.php) currently contains a line like require_once "$IP/extensions/Wikibase/client/ExampleSettings.php"; or require_once "$IP/extensions/Wikibase/client/config/WikibaseClient.example.php"; but not a line like require_once "$IP/extensions/Wikibase/repo/Wikibase.php"; then you should copy the example settings <https://github.com/wikimedia/mediawiki-extensions-Wikibase/blob/7613b87c619…> into your configuration and adjust them as needed (at least the repoUrl, http://repo.example.org, is certainly wrong). If you already configure some of these client settings, then you can skip copying the corresponding example settings lines. We will be making this change on the Wikibase master branch very soon, so if you use the latest MediaWiki and Wikibase from Git, then you will probably have to update your configuration ahead of the next update too. However, the change will not be in the upcoming MediaWiki 1.35 release (or the REL1_35 Wikibase branch), so if you are sticking to release versions, you don’t need to change anything yet. That said, you could still update your configuration in preparation for this future change – the only potential problem with copying this block is the repeated define(), which may raise a PHP Notice, so you might want to wrap those two lines <https://github.com/wikimedia/mediawiki-extensions-Wikibase/blob/7613b87c619…> in an if ( !defined( 'WB_REPO_NS_ITEM' ) ) block when copying them (unless you stop loading the example settings altogether, which is also an option). I also want to stress that, while the eventual goal of this work is to load WikibaseRepo and WikibaseClient using extension registration (i. e., JSON files, not PHP entry points), we do not yet recommend that third-party installs do this. When Wikibase is fully ready for extension registration, we will send another announcement, and the PHP entry points will probably still be kept for compatibility for a good while longer even after they’re no longer required. If you have any issue or question, feel free to leave a comment at T257449 <https://phabricator.wikimedia.org/T257449>. For more information, see also T256238 <https://phabricator.wikimedia.org/T256238>. Cheers, Lucas -- Lucas Werkmeister (he/er) Full Stack Developer Wikimedia Deutschland e. V. | Tempelhofer Ufer 23-24 | 10963 Berlin Phone: +49 (0)30 219 158 26-0 https://wikimedia.de Imagine a world in which every single human being can freely share in the sum of all knowledge. Help us to achieve our vision! https://spenden.wikimedia.de Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e. V. Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/029/42207.

1 0

BREAKING CHANGE: removing special pageterms behavior on repo wikis, use entityterms instead
by Lucas Werkmeister 15 Jul '20

15 Jul '20

This is an announcement for a breaking change to the pageterms submodule of the query API module, which only affects Wikibase repository wikis. If you do not use that API module, or only use it on client wikis (e. g. Wikipedias) and not on repository wikis (Wikidata, Wikimedia Commons), you can ignore this message. For years, the pageterms <https://www.wikidata.org/wiki/Special:ApiHelp/query%2Bpageterms> API module has served a double role: on client wikis, it returned the “terms” (Label, Description, Aliases) of the Wikidata Item linked to the given page(s), whereas on repo wikis, it would return the terms of the Item (or other Entity) on that page itself. For example, querying for the Label of Wikipedia:Village pump on English Wikipedia <https://en.wikipedia.org/w/api.php?action=query&prop=pageterms&titles=Wikip…> would return “Project:Village pump” (the Label of Q16503 <https://www.wikidata.org/wiki/Q16503>), but querying for the Label of Wikidata:Project chat on Wikidata <https://www.wikidata.org/w/api.php?action=query&prop=pageterms&titles=Wikid…> would not return anything, even though that page is linked to the same Item – you would have to query for the Label of Q16503 <https://www.wikidata.org/w/api.php?action=query&prop=pageterms&titles=Q1650…> instead. This behavior is inconsistent and also mixes repo and client concerns in a way that makes the Wikibase code harder to maintain. To resolve this, we introduced a new entityterms <https://www.wikidata.org/wiki/Special:ApiHelp/query%2Bentityterms> API module (a submodule of the query module, just like the pageterms module) which has the same behavior as the pageterms module currently has for Item (or other Entity) pages, and which is only available on repo wikis. If you want to get the terms of Q16503, you can now use action=query&prop=entityterms&titles=Q16503 <https://www.wikidata.org/w/api.php?action=query&prop=entityterms&titles=Q16…> instead of action=query&prop=pageterms&titles=Q16503 <https://www.wikidata.org/w/api.php?action=query&prop=pageterms&titles=Q16503>. (You can also use wbgetentities <https://www.wikidata.org/wiki/Special:ApiHelp/wbgetentities>, which gives you much more control over the returned data; pageterms/entityterms may be faster and can also be combined with other submodules of the query module.) On or shortly after 5 August 2020, we will remove the special repo behavior of the pageterms module, and it will then behave just like it always has on client wikis, and return the terms of the Item linked to a page, not the terms of the Item (or other Entity) on a page. (Because the new API module is already available on Wikidata, and you can start using it immediately, we are not making this pageterms behavior change available on Test Wikidata significantly before that date.) If you have any issue or question, feel free to leave a comment at T257658 <https://phabricator.wikimedia.org/T257658>. For more information, see also T115117 <https://phabricator.wikimedia.org/T115117>, T255882 <https://phabricator.wikimedia.org/T255882> and T256255 <https://phabricator.wikimedia.org/T256255>. Cheers, Lucas -- Lucas Werkmeister (he/er) Full Stack Developer Wikimedia Deutschland e. V. | Tempelhofer Ufer 23-24 | 10963 Berlin Phone: +49 (0)30 219 158 26-0 https://wikimedia.de Imagine a world in which every single human being can freely share in the sum of all knowledge. Help us to achieve our vision! https://spenden.wikimedia.de Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e. V. Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/029/42207.

1 0

Uniquely Identifying Items Created by Bot
by Yusra Haider 10 Jul '20

10 Jul '20

Hi, I hope this is an appropriate question for this mailing list. I am a software developer at Reactome and am working on our bot that writes Reactome data to Wikidata. Our bot currently uses the property https://www.wikidata.org/wiki/Property:P3937 (the Reactome ID property) to identify and update Reactome items. We are running into the issue that this property is being used by other people and our bot inadvertently ends up changing items that have not been created by the Reactome bot. Is there any way to uniquely identify the items created by our bot? Thanks and warm regards, Yusra Hader

3 3

Slow loading process
by Leandro Tabares Martín 03 Jul '20

03 Jul '20

Dear all, I'm loading the whole wikidata dataset into Blazegraph using a High Performance Computer. I gave 120 GB RAM and 3 processing cores to the job. After almost 24 hours of load the "wikidata.jnl" file has only 28 GB as size. Initially the process was fast, but as the file increased its size the loading speed has decreased. I realize that only 14 GB of RAM are being used. I already implemented the recomendations given in https://github.com/blazegraph/database/wiki/IOOptimization Do you have some other recommendations to increase the loading speed? Leandro

4 3

Wikidata history and discussion records
by Elisavet Koutsiana 19 Jun '20

19 Jun '20

Hi, I have been studying about wikidata for the last few weeks in order to perform an analysis. My main concern is about the history and the discussion data of every entity. I was wondering whether these information are included in the json files here https://dumps.wikimedia.org/wikidatawiki/entities/ or there are separate files about them. If so, where can I download them? Thanks in advance. Best wishes, Elisavet

2 2

Usage of Blazegraph with Wikidata
by Leandro Tabares Martín 17 Jun '20

17 Jun '20

Hi, I put a "wikidata.jnl" file of almost 60 GB size in the Blazegraph root directory. When I run a query like "select ?s ?p ?o where {?s ?p ?o} limit 10" through the Blazegraph's query tab I get no results at all. Do I need to do something for Blazegraph to recognize the database file? Leandro

2 1

Slow loading process
by Leandro Tabares Martín 11 Jun '20

11 Jun '20

Hi, I have downloaded Blazegraph already compiled from [1]. I also made the optimizations indicated at [2]. For the loading process I'm following the instructions given in the "getting-started.md" file that comes in the "docs" folder of the compiled distribution [1]. That means: 1- Munge the data with: ./munge.sh -f data/wikidata-20150427-all-BETA.ttl.gz -d data/split -l en -s 2- Start the loading process with: ./loadRestAPI.sh -n wdq -d `pwd`/data/split Then the loading process starts with a rate of 84352. However, the rate has been progressively going down till 3362 after 36 hours of loading. I'm running the process on a HPC with SSD and I'm giving to the loading process 3 cores and 120 GB RAM. On the other hand, I notice that the average processor usage doesn't go up over 1.6 and the maximum RAM usage is 14 GB. I also saw [3] and I'm running the loading natively (without containers). I have the difference with [3] that I've reduced the JVM heap to 4GB as [2] suggested. So what else could I do to improve the loading performance. Thanks, Leandro [1] http://search.maven.org/#search%7Cgav%7C1%7Cg%3A%22org.wikidata.query.rdf%2… [2] https://github.com/blazegraph/database/wiki/IOOptimization [3] https://addshore.com/2019/10/your-own-wikidata-query-service-with-no-limits…

3 2

Research on Wikidata dataset
by Leandro Tabares Martín 07 Jun '20

07 Jun '20

Dear all, I am a researcher from Hasselt University performing research on Query Reverse Engineering in the Context of the Semantic Web [1]. I think that the Wikidata dataset could be the ideal one to test the algorithms I have developed. However, due to the limitations of the public SPARQL endpoint [2], I cannot do this online, so I am setting a standalone instance. I realize that with my current computing power, it is not possible to perform the loading process of the dataset to my local Blazegraph instance. Because of the aforementioned reasons, I kindly request your assistance in order to be able to download a Blazegraph instance with the dataset loaded in it. Kind regards, Leandro Tabares Martín [1] https://www.uhasselt.be/UH/Research-groups/en-projecten_DOC/en-project_deta… [2] https://www.mediawiki.org/wiki/Wikidata_Query_Service/User_Manual

2 1

← Newer
1
...
4
5
6
7
8
9
10
...
63
Older →

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

Wikidata-tech