I'm looking for two Wikidata-related statistics that I'm having trouble finding. Hoping someone can help.
First, I'd like to find monthly edit counts comparing Wikidata to other Wikimedia projects, ultimately to make the statement that "Wikidata was the x-th most edited WMF site in November 2017". I've scanned stats.wikimedia.org but can't find that summarized. This link [1] seems to say that there were 18726 WD edits in Nov 2017, but that seems exceptionally small (and also I can't put that in context short of visiting the similar table for all other WMF projects).
Second, are there any stats on the uptime of the WDQS SPARQL endpoint? I'd like to make the case that WDQS is among the most stable SPARQL endpoints available (and this site [2] seems to suggest that the bar is pretty low). (is it worth trying to get wikidata directly added to that list? Hmm, seems like it was previously suggested here [3]...)
Pointers appreciated!
thanks, -andrew
[1]: https://stats.wikimedia.org/wikispecial/EN/TablesWikipediaWIKIDATA.htm# editor_activity_levels [2]: http://sparqles.ai.wu.ac.at/availability [3]: https://phabricator.wikimedia.org/T85444
Andrew Su, 20/12/2017 20:11:
I've scanned stats.wikimedia.org http://stats.wikimedia.org but can't find that summarized. This link [1] seems to say that there were 18726 WD edits in Nov 2017
That's the number of *users* making at least one edit. We generally consider the 5+ figure, which is a bit less than half that.
Federico
On Wed, Dec 20, 2017 at 10:14 AM, Federico Leva (Nemo) nemowiki@gmail.com wrote:
Andrew Su, 20/12/2017 20:11:
I've scanned stats.wikimedia.org http://stats.wikimedia.org but can't find that summarized. This link [1] seems to say that there were 18726 WD edits in Nov 2017
That's the number of *users* making at least one edit. We generally consider the 5+ figure, which is a bit less than half that.
Federico
Ahh right, thank you for that clarification. Should have figured that one out. In any case, pointers on the two stats I'm looking for are still welcome!
Best, -andrew
Andrew Su, 20/12/2017 20:19:
In any case, pointers on the two stats I'm looking for are still welcome!
Ok. I don't think the number of edits is especially meaningful, but Wikidata easily wins this metric over any other project, even if you combine all Wikipedias.
https://stats.wikimedia.org/wikispecial/EN/TablesWikipediaWIKIDATA.htm shows 14 M/month (although G>I must be an error) and in https://stats.wikimedia.org/EN/TablesDatabaseEdits.htm under Σ you can see all Wikipedias combined are around 10.
Federico
Thanks Lucas and Federico... I don't have exactly what I was looking for yet, but probably good enough for my current needs. I will say though that I think these metrics I proposed could be useful for me when trying to convince my colleagues in the biomedical domain how awesome Wikidata is. So if there were some future ability to automatically and precisely calculate them, I for one would find it useful...
Best, -andrew
On Wed, Dec 20, 2017 at 10:33 AM, Federico Leva (Nemo) nemowiki@gmail.com wrote:
Andrew Su, 20/12/2017 20:19:
In any case, pointers on the two stats I'm looking for are still welcome!
Ok. I don't think the number of edits is especially meaningful, but Wikidata easily wins this metric over any other project, even if you combine all Wikipedias.
https://stats.wikimedia.org/wikispecial/EN/TablesWikipediaWIKIDATA.htm shows 14 M/month (although G>I must be an error) and in https://stats.wikimedia.org/EN/TablesDatabaseEdits.htm under Σ you can see all Wikipedias combined are around 10.
Federico
There are edit count charts on Wikistats 2.0 (Wikidata [1], English Wikipedia [2]), but I don’t know if there’s a way to see a chart of all wikis simultaneously. As to the Query Service, there are some statistics available on Grafana [3], though I’m not sure how downtime would show up there (probably best in the “Varnish 5xx rate”?).
Cheers, Lucas
[1]: https://stats.wikimedia.org/v2/#/wikidata.org/contributing/edits [2]: https://stats.wikimedia.org/v2/#/en.wikipedia.org/contributing/edits [3]: https://grafana.wikimedia.org/dashboard/db/wikidata-query-service?refresh=1m...
On 20.12.2017 19:11, Andrew Su wrote:
I'm looking for two Wikidata-related statistics that I'm having trouble finding. Hoping someone can help.
First, I'd like to find monthly edit counts comparing Wikidata to other Wikimedia projects, ultimately to make the statement that "Wikidata was the x-th most edited WMF site in November 2017". I've scanned stats.wikimedia.org http://stats.wikimedia.org but can't find that summarized. This link [1] seems to say that there were 18726 WD edits in Nov 2017, but that seems exceptionally small (and also I can't put that in context short of visiting the similar table for all other WMF projects).
Second, are there any stats on the uptime of the WDQS SPARQL endpoint? I'd like to make the case that WDQS is among the most stable SPARQL endpoints available (and this site [2] seems to suggest that the bar is pretty low). (is it worth trying to get wikidata directly added to that list? Hmm, seems like it was previously suggested here [3]...)
Pointers appreciated!
thanks, -andrew
[1]: https://stats.wikimedia.org/wikispecial/EN/TablesWikipediaWIKIDATA.htm#edito... https://stats.wikimedia.org/wikispecial/EN/TablesWikipediaWIKIDATA.htm#editor_activity_levels [2]: http://sparqles.ai.wu.ac.at/availability http://sparqles.ai.wu.ac.at/availability [3]: https://phabricator.wikimedia.org/T85444 https://phabricator.wikimedia.org/T85444
Wikidata mailing list Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata
On Wed, Dec 20, 2017 at 7:22 PM, Lucas Werkmeister mail@lucaswerkmeister.de wrote:
There are edit count charts on Wikistats 2.0 (Wikidata [1], English Wikipedia [2]), but I don’t know if there’s a way to see a chart of all wikis simultaneously. As to the Query Service, there are some statistics available on Grafana [3], though I’m not sure how downtime would show up there (probably best in the “Varnish 5xx rate”?).
Uptime is surprisingly hard to define :)
I'd say that a combination of a rise in 5xx rate AND a drop in 2xx is a good indication that something is seriously wrong. I have just added the 2xx rate on our new dashboard [1], it is an interesting metric in all cases. A raise in 5xx error can be a badly behaved bot which generates a lot of queries in timeout (503 by our nginx proxy), or other similar issues that are more client related than server related (yes, those issues should be reported as a 4xx status code, but we live in an imperfect world).
Another source of information is our Incident Documentation page, which should list all known outages of Wikidata Query Service (and other services as well).
If you can think about other metrics we should collect / expose, let me know! I'll be happy to look into it!
Guillaume
[1] https://grafana.wikimedia.org/dashboard/db/wikidata-query-service-prometheus [2] https://wikitech.wikimedia.org/wiki/Incident_documentation
Cheers, Lucas
On 20.12.2017 19:11, Andrew Su wrote:
I'm looking for two Wikidata-related statistics that I'm having trouble finding. Hoping someone can help.
First, I'd like to find monthly edit counts comparing Wikidata to other Wikimedia projects, ultimately to make the statement that "Wikidata was the x-th most edited WMF site in November 2017". I've scanned stats.wikimedia.org but can't find that summarized. This link [1] seems to say that there were 18726 WD edits in Nov 2017, but that seems exceptionally small (and also I can't put that in context short of visiting the similar table for all other WMF projects).
Second, are there any stats on the uptime of the WDQS SPARQL endpoint? I'd like to make the case that WDQS is among the most stable SPARQL endpoints available (and this site [2] seems to suggest that the bar is pretty low). (is it worth trying to get wikidata directly added to that list? Hmm, seems like it was previously suggested here [3]...)
Pointers appreciated!
thanks, -andrew
Wikidata mailing list Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata
Wikidata mailing list Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata
Hi!
Second, are there any stats on the uptime of the WDQS SPARQL endpoint?
I am not entirely sure how you define "uptime" here? If you try to access query.wikidata.org, it'd be very close to 100%. That said, we had a couple of incidents where one or more servers failed, causing some queries to get stuck or be rejected, see https://wikitech.wikimedia.org/wiki/Incident_documentation/20171018-wdqs and https://wikitech.wikimedia.org/wiki/Incident_documentation/20171130-wdqs These do not take the whole service down, so I am not sure how they qualify uptime-wise.