Hi,
I recently set up a nightly cronjob that adds Wikidata links to the YSO
Places SKOS dataset. The way it works is that each time, it runs a
rather simple SPARQL CONSTRUCT query [1] against the Wikidata SPARQL
endpoint (script [2]) that looks for YSO ID properties (P2347), and
stores the result as sorted N-Triples into a file [3] that then gets
committed to GitHub. These triples are then incorporated into the YSO
Places data set [4] and published in Finto.fi.
But I've noticed that each time this SPARQL query is executed, a few
triples that were in the previous version get dropped and others get
reinstated. It seems to me that the Wikidata SPARQL endpoint is randomly
(?) dropping some triples from the result set.
As an example, the skos:closeMatch triple that links yso:p109659
("Laanila, Oulu") to wd:Q11874312 was there yesterday [5] but not in
today's version [6]. The last edit in Wikidata was made 3 weeks ago [7]
so nothing in the RDF data available through the Wikidata SPARQL
endpoint should have changed.
Is this a known problem? Am I missing something here? Is there something
wrong with the approach of running a CONSTRUCT query against the
Wikidata endpoint and expecting to get the same result (of around 4200
triples) each time, unless the underlying data in Wikidata has changed?
-Osma
[1]
https://github.com/NatLibFi/Finto-data/blob/master/vocabularies/yso-paikat/…
[2]
https://github.com/NatLibFi/Finto-data/blob/master/vocabularies/yso-paikat/…
[3]
https://github.com/NatLibFi/Finto-data/blob/master/vocabularies/yso-paikat/…
[4]
https://github.com/NatLibFi/Finto-data/blob/master/vocabularies/yso-paikat/…
[5]
https://github.com/NatLibFi/Finto-data/blob/9ea0e34d0814b1aa7a8ac39597d395a…
[6]
https://github.com/NatLibFi/Finto-data/blob/6af4389c9f85b0b2bd1f201f2940738…
[7]
https://www.wikidata.org/w/index.php?title=Q11874312&action=history
--
Osma Suominen
D.Sc. (Tech), Information Systems Specialist
National Library of Finland
P.O. Box 26 (Kaikukatu 4)
00014 HELSINGIN YLIOPISTO
Tel. +358 50 3199529
osma.suominen(a)helsinki.fi
http://www.nationallibrary.fi