Hi,
is it me or is the SPARQL service very slow right now?
Thanks,
Markus
On Wed, Feb 3, 2016 at 2:57 PM Markus Krötzsch < markus@semantic-mediawiki.org> wrote:
Hi,
is it me or is the SPARQL service very slow right now?
I just tried a few queries of the example queries and it seemed very quick. Any particular queries you are trying?
Cheers Lydia
Here by the way are two dashboard to check on vital signs of the query service: * http://discovery.wmflabs.org/wdqs/#wdqs_usage * https://grafana.wikimedia.org/dashboard/db/wikidata-query-service
Cheers Lydia
Listeria is down for me https://tools.wmflabs.org/listeria Not sure if related.
2016-02-03 15:33 GMT+01:00 Lydia Pintscher Lydia.Pintscher@wikimedia.de:
Here by the way are two dashboard to check on vital signs of the query service:
- http://discovery.wmflabs.org/wdqs/#wdqs_usage
- https://grafana.wikimedia.org/dashboard/db/wikidata-query-service
Cheers Lydia -- Lydia Pintscher - http://about.me/lydia.pintscher Product Manager for Wikidata
Wikimedia Deutschland e.V. Tempelhofer Ufer 23-24 10963 Berlin www.wikimedia.de
Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e. V.
Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 Nz. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/029/42207.
Wikidata mailing list Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Listeria has been down since at least Monday. I e-mailed Magnus because of that yesterday.
Cheers Yellowcard
Emilio J. Rodríguez-Posada schrieb:
Listeria is down for me https://tools.wmflabs.org/listeria Not sure if related.
2016-02-03 15:33 GMT+01:00 Lydia Pintscher <Lydia.Pintscher@wikimedia.de mailto:Lydia.Pintscher@wikimedia.de>:
Here by the way are two dashboard to check on vital signs of the query service: * http://discovery.wmflabs.org/wdqs/#wdqs_usage * https://grafana.wikimedia.org/dashboard/db/wikidata-query-service
Cheers Lydia -- Lydia Pintscher - http://about.me/lydia.pintscher Product Manager for Wikidata
Wikimedia Deutschland e.V. Tempelhofer Ufer 23-24 10963 Berlin www.wikimedia.de http://www.wikimedia.de
Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e. V.
Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 Nz. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/029/42207.
_______________________________________________ Wikidata mailing list Wikidata@lists.wikimedia.org mailto:Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata
_______________________________________________ Wikidata mailing list Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata
Excellent news (for my mental health, not for Wikidata). I was already in despair about my inability to get my query working. Thanks!
Christian
-----Ursprüngliche Nachricht----- Von: Wikidata [mailto:wikidata-bounces@lists.wikimedia.org] Im Auftrag von Franz Deelmann (Yellowcard) Gesendet: Mittwoch, 3. Februar 2016 16:23 An: wikidata@lists.wikimedia.org Betreff: Re: [Wikidata] SPARQL service slow?
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Listeria has been down since at least Monday. I e-mailed Magnus because of that yesterday.
Cheers Yellowcard
Emilio J. Rodríguez-Posada schrieb:
Listeria is down for me https://tools.wmflabs.org/listeria Not sure if related.
2016-02-03 15:33 GMT+01:00 Lydia Pintscher <Lydia.Pintscher@wikimedia.de mailto:Lydia.Pintscher@wikimedia.de>:
Here by the way are two dashboard to check on vital signs of the query service: * http://discovery.wmflabs.org/wdqs/#wdqs_usage * https://grafana.wikimedia.org/dashboard/db/wikidata-query-service
Cheers Lydia -- Lydia Pintscher - http://about.me/lydia.pintscher Product Manager for Wikidata
Wikimedia Deutschland e.V. Tempelhofer Ufer 23-24 10963 Berlin www.wikimedia.de http://www.wikimedia.de
Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e. V.
Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 Nz. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/029/42207.
_______________________________________________ Wikidata mailing list Wikidata@lists.wikimedia.org mailto:Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata
_______________________________________________ Wikidata mailing list Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata
_______________________________________________ Wikidata mailing list Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata
It looks like one of the two query machines has died, and the other has stopped taking updates from wikidata. Thus expect you queries to currently work 50% of the time, but the result to be out of date (currently just over 7 hours lag in the data)
Addshore
On 3 February 2016 at 16:40, christian.hauschke@hs-hannover.de wrote:
Excellent news (for my mental health, not for Wikidata). I was already in despair about my inability to get my query working. Thanks!
Christian
-----Ursprüngliche Nachricht----- Von: Wikidata [mailto:wikidata-bounces@lists.wikimedia.org] Im Auftrag von Franz Deelmann (Yellowcard) Gesendet: Mittwoch, 3. Februar 2016 16:23 An: wikidata@lists.wikimedia.org Betreff: Re: [Wikidata] SPARQL service slow?
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Listeria has been down since at least Monday. I e-mailed Magnus because of that yesterday.
Cheers Yellowcard
Emilio J. Rodríguez-Posada schrieb:
Listeria is down for me https://tools.wmflabs.org/listeria Not sure if related.
2016-02-03 15:33 GMT+01:00 Lydia Pintscher <Lydia.Pintscher@wikimedia.de mailto:Lydia.Pintscher@wikimedia.de>:
Here by the way are two dashboard to check on vital signs of the query service: * http://discovery.wmflabs.org/wdqs/#wdqs_usage * https://grafana.wikimedia.org/dashboard/db/wikidata-query-service
Cheers Lydia -- Lydia Pintscher - http://about.me/lydia.pintscher Product Manager for Wikidata
Wikimedia Deutschland e.V. Tempelhofer Ufer 23-24 10963 Berlin www.wikimedia.de http://www.wikimedia.de
Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e. V.
Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 Nz. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/029/42207.
_______________________________________________ Wikidata mailing list Wikidata@lists.wikimedia.org mailto:Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata
_______________________________________________ Wikidata mailing list Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata
-----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (MingW32)
iQEcBAEBAgAGBQJWshtAAAoJEBK09GQgqEjvztkH/2p2UVOfmHjBlJ1yMhmYF76q 5VJuzFuFWsSASvAYLnasvmAxPcZmQofmf/vEanDvdk98Dn5emuu6+AZv74vTt6Ip eNbb/ZNetmSH/i9haAJSrUXgi+WK1b/KjnQrliN9+kwyanRn1HU8zlRfyI1id7bt J0ucsTr6vgbAQQnzXUIwB5m0yN5VuBtBCsZijKr3RGKuBTB0qw2z8dHaF0g9L9jI IrSr/YuvVOfUDi/mZL0ylE5xxbpuChmGfUEjR3RBPr+ZYVWsVpYrDoKPhFspgZMD 4z8Vdk/hdgJqfXFBfnnXqx6IM3aNRiZQ4tIJ9G5i29vXOTxDtNj+kobLH9yfG3c= =7gtx -----END PGP SIGNATURE-----
Wikidata mailing list Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata _______________________________________________ Wikidata mailing list Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata
Okay, both hosts are now back up and the lag is heading down! :) Lag should be all gone in roughly 1 hour!
Addshore
On 3 February 2016 at 17:35, Addshore addshorewiki@gmail.com wrote:
It looks like one of the two query machines has died, and the other has stopped taking updates from wikidata. Thus expect you queries to currently work 50% of the time, but the result to be out of date (currently just over 7 hours lag in the data)
Addshore
On 3 February 2016 at 16:40, christian.hauschke@hs-hannover.de wrote:
Excellent news (for my mental health, not for Wikidata). I was already in despair about my inability to get my query working. Thanks!
Christian
-----Ursprüngliche Nachricht----- Von: Wikidata [mailto:wikidata-bounces@lists.wikimedia.org] Im Auftrag von Franz Deelmann (Yellowcard) Gesendet: Mittwoch, 3. Februar 2016 16:23 An: wikidata@lists.wikimedia.org Betreff: Re: [Wikidata] SPARQL service slow?
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Listeria has been down since at least Monday. I e-mailed Magnus because of that yesterday.
Cheers Yellowcard
Emilio J. Rodríguez-Posada schrieb:
Listeria is down for me https://tools.wmflabs.org/listeria Not sure if related.
2016-02-03 15:33 GMT+01:00 Lydia Pintscher <Lydia.Pintscher@wikimedia.de mailto:Lydia.Pintscher@wikimedia.de>:
Here by the way are two dashboard to check on vital signs of the query service: * http://discovery.wmflabs.org/wdqs/#wdqs_usage * https://grafana.wikimedia.org/dashboard/db/wikidata-query-service
Cheers Lydia -- Lydia Pintscher - http://about.me/lydia.pintscher Product Manager for Wikidata
Wikimedia Deutschland e.V. Tempelhofer Ufer 23-24 10963 Berlin www.wikimedia.de http://www.wikimedia.de
Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e. V.
Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 Nz. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/029/42207.
_______________________________________________ Wikidata mailing list Wikidata@lists.wikimedia.org mailto:Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata
_______________________________________________ Wikidata mailing list Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata
-----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (MingW32)
iQEcBAEBAgAGBQJWshtAAAoJEBK09GQgqEjvztkH/2p2UVOfmHjBlJ1yMhmYF76q 5VJuzFuFWsSASvAYLnasvmAxPcZmQofmf/vEanDvdk98Dn5emuu6+AZv74vTt6Ip eNbb/ZNetmSH/i9haAJSrUXgi+WK1b/KjnQrliN9+kwyanRn1HU8zlRfyI1id7bt J0ucsTr6vgbAQQnzXUIwB5m0yN5VuBtBCsZijKr3RGKuBTB0qw2z8dHaF0g9L9jI IrSr/YuvVOfUDi/mZL0ylE5xxbpuChmGfUEjR3RBPr+ZYVWsVpYrDoKPhFspgZMD 4z8Vdk/hdgJqfXFBfnnXqx6IM3aNRiZQ4tIJ9G5i29vXOTxDtNj+kobLH9yfG3c= =7gtx -----END PGP SIGNATURE-----
Wikidata mailing list Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata _______________________________________________ Wikidata mailing list Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata
-- Addshore
SPARQL query response is currently excellent. Thanks!
Regards, James Weaver
On Wed, Feb 3, 2016, at 11:55 AM, Addshore wrote:
Okay, both hosts are now back up and the lag is heading down! :) Lag should be all gone in roughly 1 hour!
Addshore
On 3 February 2016 at 17:35, Addshore addshorewiki@gmail.com wrote:
It looks like one of the two query machines has died, and the other has stopped taking updates from wikidata. Thus expect you queries to currently work 50% of the time, but the result to be out of date (currently just over 7 hours lag in the data)
Addshore
On 3 February 2016 at 16:40, <christian.hauschke@hs- hannover.de> wrote:
Excellent news (for my mental health, not for Wikidata). I was already in despair about my inability to get my query working. Thanks!
Christian
-----Ursprüngliche Nachricht-----
Von: Wikidata [mailto:wikidata-bounces@lists.wikimedia.org] Im Auftrag von Franz Deelmann (Yellowcard)
Gesendet: Mittwoch, 3. Februar 2016 16:23
An: wikidata@lists.wikimedia.org
Betreff: Re: [Wikidata] SPARQL service slow?
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Listeria has been down since at least Monday. I e-mailed Magnus because of that yesterday.
Cheers
Yellowcard
Emilio J. Rodríguez-Posada schrieb:
Listeria is down for me https://tools.wmflabs.org/listeria Not sure if
related.
2016-02-03 15:33 GMT+01:00 Lydia Pintscher
<Lydia.Pintscher@wikimedia.de
mailto:Lydia.Pintscher@wikimedia.de>:
Here by the way are two dashboard to check on vital signs of the query
service: * http://discovery.wmflabs.org/wdqs/#wdqs_usage *
https://grafana.wikimedia.org/dashboard/db/wikidata-query-service
Cheers Lydia -- Lydia Pintscher - http://about.me/lydia.pintscher
Product Manager for Wikidata
Wikimedia Deutschland e.V. Tempelhofer Ufer 23-24 10963 Berlin
www.wikimedia.de http://www.wikimedia.de
Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e.
V.
Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg
unter der Nummer 23855 Nz. Als gemeinnützig anerkannt durch das
Finanzamt für Körperschaften I Berlin, Steuernummer 27/029/42207[1].
_______________________________________________ Wikidata mailing list
Wikidata@lists.wikimedia.org mailto:Wikidata@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata
_______________________________________________ Wikidata mailing list
Wikidata@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata
_______________________________________________
Wikidata mailing list
Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata
_______________________________________________
Wikidata mailing list
Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata
-- Addshore
-- Addshore _________________________________________________ Wikidata mailing list Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata
Links:
1. tel:27%2F029%2F42207
Hi!
is it me or is the SPARQL service very slow right now?
I've upgraded it yesterday to Blazegraph 2.0 and it looks like there was some glitch there. I've restarted it and now it seems to be fine. I'll be watching it and see if it repeats.
Nice that the SPARQL service is upgraded. We noticed something was going on and terminated our bots to wait for more stability.
1. Is that stability now reached? 2. Is there some sort of flag-mechanism, that would indicate if the SPARQL endpoint is stable, being upgraded or had change in state?
The SPARQL endpoint is core to our efforts in genewiki's ProteinBoxBot, for a spectrum of reasons. (e.g. concept resolution, identifying data inconsistencies (an item on a gene and protein being incorrectly merged), getting precision and recall numbers on included data sources, etc) Whenever there is an issue, we usually take a step back and retry later. However, this is based on trial and error. It would be quite convenient if we could use a flag-mechanism to decide if we can start or if already started we need to terminate.
Can we do this? Advise on different approaches would be also appreciated.
Regards,
Andra Waagmeester
On Wed, Feb 3, 2016 at 7:53 PM, Stas Malyshev smalyshev@wikimedia.org wrote:
Hi!
is it me or is the SPARQL service very slow right now?
I've upgraded it yesterday to Blazegraph 2.0 and it looks like there was some glitch there. I've restarted it and now it seems to be fine. I'll be watching it and see if it repeats.
-- Stas Malyshev smalyshev@wikimedia.org
Wikidata mailing list Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata
I agree and think that at least a mailing post here and on wikidata-tech would've been great to make sure everyone has the chance to know what's going on. For example, I like the way status updates are posted on labs-l for updates on the labs cluster.
Best regards Bene
Am 03.02.2016 um 21:44 schrieb Andra Waagmeester:
Nice that the SPARQL service is upgraded. We noticed something was going on and terminated our bots to wait for more stability.
- Is that stability now reached?
- Is there some sort of flag-mechanism, that would indicate if the
SPARQL endpoint is stable, being upgraded or had change in state?
The SPARQL endpoint is core to our efforts in genewiki's ProteinBoxBot, for a spectrum of reasons. (e.g. concept resolution, identifying data inconsistencies (an item on a gene and protein being incorrectly merged), getting precision and recall numbers on included data sources, etc) Whenever there is an issue, we usually take a step back and retry later. However, this is based on trial and error. It would be quite convenient if we could use a flag-mechanism to decide if we can start or if already started we need to terminate.
Can we do this? Advise on different approaches would be also appreciated.
Regards,
Andra Waagmeester
On Wed, Feb 3, 2016 at 7:53 PM, Stas Malyshev <smalyshev@wikimedia.org mailto:smalyshev@wikimedia.org> wrote:
Hi! > is it me or is the SPARQL service very slow right now? I've upgraded it yesterday to Blazegraph 2.0 and it looks like there was some glitch there. I've restarted it and now it seems to be fine. I'll be watching it and see if it repeats. -- Stas Malyshev smalyshev@wikimedia.org <mailto:smalyshev@wikimedia.org> _______________________________________________ Wikidata mailing list Wikidata@lists.wikimedia.org <mailto:Wikidata@lists.wikimedia.org> https://lists.wikimedia.org/mailman/listinfo/wikidata
Wikidata mailing list Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata
I agree and think that at least a mailing post here and on wikidata-tech
would've been great to make sure everyone has the chance to know what's going on Well, we only noticed what was up due to this email! Take a look at https://phabricator.wikimedia.org/T119915
So you can always get an overall view of the query service at https://grafana.wikimedia.org/dashboard/db/wikidata-query-service Key here for your issue would be the lag graph in the middle right. This is generated using the following query: SELECT * WHERE { http://www.wikidata.org schema:dateModified ?y } Which gives you the last modified date for the data stored!
Addshore
On 3 February 2016 at 21:44, Andra Waagmeester andra@micelio.be wrote:
Nice that the SPARQL service is upgraded. We noticed something was going on and terminated our bots to wait for more stability.
- Is that stability now reached?
- Is there some sort of flag-mechanism, that would indicate if the SPARQL
endpoint is stable, being upgraded or had change in state?
The SPARQL endpoint is core to our efforts in genewiki's ProteinBoxBot, for a spectrum of reasons. (e.g. concept resolution, identifying data inconsistencies (an item on a gene and protein being incorrectly merged), getting precision and recall numbers on included data sources, etc) Whenever there is an issue, we usually take a step back and retry later. However, this is based on trial and error. It would be quite convenient if we could use a flag-mechanism to decide if we can start or if already started we need to terminate.
Can we do this? Advise on different approaches would be also appreciated.
Regards,
Andra Waagmeester
On Wed, Feb 3, 2016 at 7:53 PM, Stas Malyshev smalyshev@wikimedia.org wrote:
Hi!
is it me or is the SPARQL service very slow right now?
I've upgraded it yesterday to Blazegraph 2.0 and it looks like there was some glitch there. I've restarted it and now it seems to be fine. I'll be watching it and see if it repeats.
-- Stas Malyshev smalyshev@wikimedia.org
Wikidata mailing list Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata
Wikidata mailing list Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata
Hi!
Well, we only noticed what was up due to this email! Take a look at https://phabricator.wikimedia.org/T119915
Yes, we need to look into it. The problem is that the service has two failure modes:
1. Completely dead, rejecting all queries. This would be caught by icinga and alerted.
2. Crawling slow, but still partially alive, just performing very very badly. For this one, we do not have adequate alert system. This failure mode is rare, but we've seen it to happen, both due to somebody sending a torrent of heavy queries and some bug scenarios. Icinga does not catch that because it only checks very basic queries and those are still under timeout.
Hi!
- Is that stability now reached?
- Is there some sort of flag-mechanism, that would indicate if the
SPARQL endpoint is stable, being upgraded or had change in state?
It's supposed to be stable now, though there's at least one known bug (not affecting you unless you run the affected query, but if you see NotMaterializedException you probably hit it, it's being worked on and will be fixed soon, ping me if you want boring details).
In general, the upgrade went fine so I didn't send a note but then apparently (of course, as soon as I waited to ensure everything is going fine, which it was, and went to sleep) something gone wrong and I'm still not sure what exactly - I'll continue to research that, but whatever happened doesn't seem to repeat itself so far. In the future, I'll make announcement when I do potentially risky stuff.
We have a number of plans for implementing new functions which would involve full DB reload at least once - which will mean the DB will be temporarily reset to the state of the latest weekly dump (i.e. 2-3 days behind current state of wikidata) and then will re-sync with the current state over the course of a day or so. I'll write about that additionally before we do it, depending on how development is going could be end of this month or somewhere next month.
Thanks,