Thanks. Meanwhile, I have also implemented a fall-back query for SQID that does not use the aggregate but only finds distinct results instead. This enables us to update at least the labels automatically from SPARQL (downside: I no longer have reliable statistics on whether the problem is fixed).

Markus

On 05.09.2016 11:27, Guillaume Lederrey wrote:

Hello!

I just had a quick look at some of our graphs [1][2]. It does look
like we've had a slight increase in number of queries per second and
correlated higher IO. I'm not sure at this point if this is related to
the timeout you see. I'll dig a bit deeper...

Thanks for the patience!

[1] https://grafana.wikimedia.org/dashboard/db/wikidata-query-service
[2] https://ganglia.wikimedia.org/latest/?r=month&c=Wikidata+Query+Service+eqiad

On Sun, Sep 4, 2016 at 2:26 PM, Markus Kroetzsch
<markus.kroetzsch@tu-dresden.de> wrote:

Hi,

SQID uses a somewhat challenging SPARQL query to refresh its statistical
data for the current usage of classes [1]. This is done once per hour, with
one retry after 60sec if the first attempt times out. In the past, timeouts
have been common, but it usually worked after a while.

Since a few days, however, the query always times out. In spite of the 48
attempts throughout each day, the query did not succeed once since
8/30/2016, 8:12:28 PM [2].

Possible explanations:
* WDQS experiences more load now (every day, every hour).
* The query got slower since for some reason the overall number of P31
statements increased in a sudden way (or for some reason crossed some
threshold).
* There have been technical changes to WDQS that reduce performance.

I don't have statistics on the success rate of the problematic query in past
weeks, so I cannot say if the timeout rate had increased before the current
week.

Does anybody have further information or obsevations that could help to
clarify what is going on? We can rewrite our software to use simpler queries
if this one fails now, but it seems like a step backwards.

Best regards,

Markus


[1] Here is the query:

SELECT ?cl ?clLabel ?c WHERE {
  { SELECT ?cl (count(*) as ?c) WHERE { ?i wdt:P31 ?cl } GROUP BY ?cl }
    SERVICE wikibase:label {
      bd:serviceParam wikibase:language "en" .
  }
}

[2] https://tools.wmflabs.org/sqid/#/status

--
Prof. Dr. Markus Kroetzsch
Knowledge-Based Systems Group
Faculty of Computer Science
TU Dresden
+49 351 463 38486
https://iccl.inf.tu-dresden.de/web/KBS/en

_______________________________________________
Wikidata mailing list
Wikidata@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata

_______________________________________________
Wikidata mailing list
Wikidata@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata