Thanks. Meanwhile, I have also implemented a fall-back query for SQID that does not use the aggregate but only finds distinct results instead. This enables us to update at least the labels automatically from SPARQL (downside: I no longer have reliable statistics on whether the problem is fixed).
Markus
On 05.09.2016 11:27, Guillaume Lederrey wrote:
Hello!
I just had a quick look at some of our graphs [1][2]. It does look like we've had a slight increase in number of queries per second and correlated higher IO. I'm not sure at this point if this is related to the timeout you see. I'll dig a bit deeper...
Thanks for the patience!
[1] https://grafana.wikimedia.org/dashboard/db/wikidata-query-service [2] https://ganglia.wikimedia.org/latest/?r=month&c=Wikidata+Query+Service+e...
On Sun, Sep 4, 2016 at 2:26 PM, Markus Kroetzsch markus.kroetzsch@tu-dresden.de wrote:
Hi,
SQID uses a somewhat challenging SPARQL query to refresh its statistical data for the current usage of classes [1]. This is done once per hour, with one retry after 60sec if the first attempt times out. In the past, timeouts have been common, but it usually worked after a while.
Since a few days, however, the query always times out. In spite of the 48 attempts throughout each day, the query did not succeed once since 8/30/2016, 8:12:28 PM [2].
Possible explanations:
- WDQS experiences more load now (every day, every hour).
- The query got slower since for some reason the overall number of P31
statements increased in a sudden way (or for some reason crossed some threshold).
- There have been technical changes to WDQS that reduce performance.
I don't have statistics on the success rate of the problematic query in past weeks, so I cannot say if the timeout rate had increased before the current week.
Does anybody have further information or obsevations that could help to clarify what is going on? We can rewrite our software to use simpler queries if this one fails now, but it seems like a step backwards.
Best regards,
Markus
[1] Here is the query:
SELECT ?cl ?clLabel ?c WHERE { { SELECT ?cl (count(*) as ?c) WHERE { ?i wdt:P31 ?cl } GROUP BY ?cl } SERVICE wikibase:label { bd:serviceParam wikibase:language "en" . } }
[2] https://tools.wmflabs.org/sqid/#/status
-- Prof. Dr. Markus Kroetzsch Knowledge-Based Systems Group Faculty of Computer Science TU Dresden +49 351 463 38486 https://iccl.inf.tu-dresden.de/web/KBS/en
Wikidata mailing list Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata