Hi Gerard,
Query performance has never been this bad. Currently the lag is over 6 HOURS.. and rising
My previous question stands.. What is the plan because we do not cope.
how about each hosts their own? This would provide some relief.
Below, I attached a query to https://databus.dbpedia.org/repo/sparql to query the latest download urls for the Wikidata-dbpedia extraction: https://databus.dbpedia.org/dbpedia/wikidata
Here is the yasgui link: https://tinyurl.com/yy768vh3
We have a virtuoso docker image that takes the query, downloads the files and fills a local sparql endpoint:
Download theDockerfile https://github.com/dbpedia/dev.dbpedia.org/raw/master/pics/Dockerfile.docker...
Build|docker build -t databus-dump-triplestore .|
Load any Databus|?file|query: |docker run -p 8890:8890 databus-dump-triplestore $(cat file-with-query.sparql)|
Doing it this way would ease some load and the docker updates each week and can be cronjobbed.
Note that this is for the Wikidata-DBpedia extraction: http://svn.aksw.org/papers/2015/ISWC_Wikidata2DBpedia/public.pdf
Databus is an open platform, so as soon as Wikidata/WMF or somebody else publishes the original wikidata dumps there, you can use the docker to decentralise hosting.
All the best,
Sebastian
QUERY:
PREFIX dataid: http://dataid.dbpedia.org/ns/core# PREFIX dataid-cv: http://dataid.dbpedia.org/ns/cv# PREFIX dct: http://purl.org/dc/terms/ PREFIX dcat: http://www.w3.org/ns/dcat#
# Get all files SELECT DISTINCT ?file WHERE { ?dataset dataid:artifact ?artifact . FILTER (?artifact in ( https://databus.dbpedia.org/dbpedia/wikidata/instance-types, https://databus.dbpedia.org/dbpedia/wikidata/mappingbased-objects-uncleaned, https://databus.dbpedia.org/dbpedia/wikidata/mappingbased-literals, https://databus.dbpedia.org/dbpedia/wikidata/labels, https://databus.dbpedia.org/dbpedia/wikidata/references, https://databus.dbpedia.org/dbpedia/wikidata/ontology-subclassof, https://databus.dbpedia.org/dbpedia/wikidata/sameas-external, https://databus.dbpedia.org/dbpedia/wikidata/images, https://databus.dbpedia.org/dbpedia/wikidata/geo-coordinates, https://databus.dbpedia.org/dbpedia/wikidata/description, https://databus.dbpedia.org/dbpedia/wikidata/mappingbased-properties-reified, https://databus.dbpedia.org/dbpedia/wikidata/properties, https://databus.dbpedia.org/dbpedia/wikidata/redirects, https://databus.dbpedia.org/dbpedia/wikidata/sameas-all-wikis, https://databus.dbpedia.org/dbpedia/wikidata/alias ) ). ?dataset dcat:distribution ?distribution . ?dataset dct:hasVersion ?latestVersion . { SELECT (max(?version) as ?latestVersion) WHERE { ?dataset dataid:artifact ?artifact . FILTER (?artifact in ( https://databus.dbpedia.org/dbpedia/wikidata/instance-types, https://databus.dbpedia.org/dbpedia/wikidata/mappingbased-objects-uncleaned, https://databus.dbpedia.org/dbpedia/wikidata/mappingbased-literals, https://databus.dbpedia.org/dbpedia/wikidata/labels, https://databus.dbpedia.org/dbpedia/wikidata/references, https://databus.dbpedia.org/dbpedia/wikidata/ontology-subclassof, https://databus.dbpedia.org/dbpedia/wikidata/sameas-external, https://databus.dbpedia.org/dbpedia/wikidata/images, https://databus.dbpedia.org/dbpedia/wikidata/geo-coordinates, https://databus.dbpedia.org/dbpedia/wikidata/description, https://databus.dbpedia.org/dbpedia/wikidata/mappingbased-properties-reified, https://databus.dbpedia.org/dbpedia/wikidata/properties, https://databus.dbpedia.org/dbpedia/wikidata/redirects, https://databus.dbpedia.org/dbpedia/wikidata/sameas-all-wikis, https://databus.dbpedia.org/dbpedia/wikidata/alias ) ). ?dataset dct:hasVersion ?version . } } ?distribution dcat:downloadURL ?file .
}