Dear WikiMedia,
We are using Action API to get recent change updates from www.wikidata.org:
https://www.wikidata.org/w/api.php?action=query&format=json&continue... https://www.wikidata.org/w/api.php?action=query&format=jsonfm&continue=&list=recentchanges&rclimit=500&rcnamespace=0%7C2%7C4%7C6%7C10%7C14%7C120%7C122%7C828&rcprop=comment%7Cflags%7Cids%7Cloginfo%7Coresscores%7Csizes%7Ctags%7Ctimestamp%7Ctitle%7Cuser
But we've found that this API become very slow recently, the response time of one API request is between 15s ~ 25s.
Sometimes the API returns an empty recent change list: [image: SHgNOsVRJPz.png] Other times the length of the response recent change list is always between 35~37, even if we set the rclimit parameter to 500.
Is there any changes related to www.wikidata.org recently?
Sincerely, Wenjie
Hi,
Thanks for the report!
We've just opened a ticket about this: https://phabricator.wikimedia.org/T209459#4745495 where you can follow along for updates.
You'll see that the problem appears to be related to the `oresscores` parameter. If you omit this then you get a timely response.
Best, Tom
On Wed, Nov 14, 2018 at 9:46 AM Wenjie Song wenjies@google.com wrote:
Dear WikiMedia,
We are using Action API to get recent change updates from www.wikidata.org :
https://www.wikidata.org/w/api.php?action=query&format=json&continue... https://www.wikidata.org/w/api.php?action=query&format=jsonfm&continue=&list=recentchanges&rclimit=500&rcnamespace=0%7C2%7C4%7C6%7C10%7C14%7C120%7C122%7C828&rcprop=comment%7Cflags%7Cids%7Cloginfo%7Coresscores%7Csizes%7Ctags%7Ctimestamp%7Ctitle%7Cuser
But we've found that this API become very slow recently, the response time of one API request is between 15s ~ 25s.
Sometimes the API returns an empty recent change list: [image: SHgNOsVRJPz.png] Other times the length of the response recent change list is always between 35~37, even if we set the rclimit parameter to 500.
Is there any changes related to www.wikidata.org recently?
Sincerely, Wenjie
Hi Wenjie,
Can you please check and let us know if you are still experiencing the problem?
Cheers Lydia
Hi!
Can you please check and let us know if you are still experiencing the problem?
We have a task https://phabricator.wikimedia.org/T202764 which I suspect describes the same issue. It is still open, and even though WDQS is running on Kafka in production and thus is not affected by it, I see it every time I run it on Labs (where Kafka stream is not available). So I think the issues with RC API on wikidata are still alive.
There's also a parallel issue of https://phabricator.wikimedia.org/T207718 with RDF fetching, which also still happens.
Also given that it uses oresscores, we recently fixed some performance issues caused by it. Do you still have issues with it?
Best
On Sun, 2 Dec 2018 at 20:47, Stas Malyshev smalyshev@wikimedia.org wrote:
Hi!
Can you please check and let us know if you are still experiencing the problem?
We have a task https://phabricator.wikimedia.org/T202764 which I suspect describes the same issue. It is still open, and even though WDQS is running on Kafka in production and thus is not affected by it, I see it every time I run it on Labs (where Kafka stream is not available). So I think the issues with RC API on wikidata are still alive.
There's also a parallel issue of https://phabricator.wikimedia.org/T207718 with RDF fetching, which also still happens. -- Stas Malyshev smalyshev@wikimedia.org
Wikidata mailing list Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata
Hi!
Also given that it uses oresscores, we recently fixed some performance issues caused by it. Do you still have issues with it?
Yes, the issues I have listed still happen. My API calls do not use ORES. E.g. see: https://logstash.wikimedia.org/goto/63db4ce68fb5da3cdc7828150de10c59
