Hi,
In the last few days my bots get throttled for minutes and minutes (up to 10, on the worse instances) on basically all sites I use (ro.wp, commons, wikidata). I knew about write throttle and I use the default setting there, but it seems that now I get throttled on read operations as well. I use pwb from git, currently synced at 0924e1655ef9e61c86e98884226b18d8250da368.
I checked the server lag and it is 0 most times I see the throttling. Any idea why do I get throttled for such long periods of time?
Thanks, Strainu
Is it Wikidata only it's all sites? Wikidata uses another set of measures for maxlag, like lag between dispatching to other wiki and lag to WDQS.
HTH
On Sat, Jan 25, 2020 at 10:40 PM Strainu strainu10@gmail.com wrote:
Hi,
In the last few days my bots get throttled for minutes and minutes (up to 10, on the worse instances) on basically all sites I use (ro.wp, commons, wikidata). I knew about write throttle and I use the default setting there, but it seems that now I get throttled on read operations as well. I use pwb from git, currently synced at 0924e1655ef9e61c86e98884226b18d8250da368.
I checked the server lag and it is 0 most times I see the throttling. Any idea why do I get throttled for such long periods of time?
Thanks, Strainu
pywikibot mailing list pywikibot@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot
All sites. Currently I also encounter other issues (connection timeouts), but the throttling has been going on for days (at least since 1/20).
În sâm., 25 ian. 2020 la 23:46, Amir Sarabadani ladsgroup@gmail.com a scris:
Is it Wikidata only it's all sites? Wikidata uses another set of measures for maxlag, like lag between dispatching to other wiki and lag to WDQS.
HTH
On Sat, Jan 25, 2020 at 10:40 PM Strainu strainu10@gmail.com wrote:
Hi,
In the last few days my bots get throttled for minutes and minutes (up to 10, on the worse instances) on basically all sites I use (ro.wp, commons, wikidata). I knew about write throttle and I use the default setting there, but it seems that now I get throttled on read operations as well. I use pwb from git, currently synced at 0924e1655ef9e61c86e98884226b18d8250da368.
I checked the server lag and it is 0 most times I see the throttling. Any idea why do I get throttled for such long periods of time?
Thanks, Strainu
pywikibot mailing list pywikibot@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot
-- Amir (he/him)
pywikibot mailing list pywikibot@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot
Actually, now that you mention it, it might be because of wikidata - I read stuff from there in all affected scripts. If I get throttled there 5s for each element, this might account for the long delays. Let me see if I can print some debug data. Any suggestion on what I should look for regarding Wikidata lag?
În sâm., 25 ian. 2020 la 23:48, Strainu strainu10@gmail.com a scris:
All sites. Currently I also encounter other issues (connection timeouts), but the throttling has been going on for days (at least since 1/20).
În sâm., 25 ian. 2020 la 23:46, Amir Sarabadani ladsgroup@gmail.com a scris:
Is it Wikidata only it's all sites? Wikidata uses another set of measures for maxlag, like lag between dispatching to other wiki and lag to WDQS.
HTH
On Sat, Jan 25, 2020 at 10:40 PM Strainu strainu10@gmail.com wrote:
Hi,
In the last few days my bots get throttled for minutes and minutes (up to 10, on the worse instances) on basically all sites I use (ro.wp, commons, wikidata). I knew about write throttle and I use the default setting there, but it seems that now I get throttled on read operations as well. I use pwb from git, currently synced at 0924e1655ef9e61c86e98884226b18d8250da368.
I checked the server lag and it is 0 most times I see the throttling. Any idea why do I get throttled for such long periods of time?
Thanks, Strainu
pywikibot mailing list pywikibot@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot
-- Amir (he/him)
pywikibot mailing list pywikibot@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot
It should be accessible in https://grafana.wikimedia.org/d/000000156/wikidata-dispatch?orgId=1&refr... but I'm not sure why other types of max lag is not showing up.
You can also check maxlag like this: https://www.wikidata.org/w/api.php?action=query&titles=MediaWiki&for...
On Sat, Jan 25, 2020 at 10:53 PM Strainu strainu10@gmail.com wrote:
Actually, now that you mention it, it might be because of wikidata - I read stuff from there in all affected scripts. If I get throttled there 5s for each element, this might account for the long delays. Let me see if I can print some debug data. Any suggestion on what I should look for regarding Wikidata lag?
În sâm., 25 ian. 2020 la 23:48, Strainu strainu10@gmail.com a scris:
All sites. Currently I also encounter other issues (connection timeouts), but the throttling has been going on for days (at least since 1/20).
În sâm., 25 ian. 2020 la 23:46, Amir Sarabadani ladsgroup@gmail.com a
scris:
Is it Wikidata only it's all sites? Wikidata uses another set of
measures for maxlag, like lag between dispatching to other wiki and lag to WDQS.
HTH
On Sat, Jan 25, 2020 at 10:40 PM Strainu strainu10@gmail.com wrote:
Hi,
In the last few days my bots get throttled for minutes and minutes (up to 10, on the worse instances) on basically all sites I use (ro.wp, commons, wikidata). I knew about write throttle and I use the default setting there, but it seems that now I get throttled on read operations as well. I use pwb from git, currently synced at 0924e1655ef9e61c86e98884226b18d8250da368.
I checked the server lag and it is 0 most times I see the throttling. Any idea why do I get throttled for such long periods of time?
Thanks, Strainu
pywikibot mailing list pywikibot@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot
-- Amir (he/him)
pywikibot mailing list pywikibot@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot
pywikibot mailing list pywikibot@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot
I can confirm the throttling is caused by Wikidata. Not sure why I didn't see it before, as I was using that api query that you gave above. Are there some (known) issues currently or should I log a bug on this?
Strainu
În dum., 26 ian. 2020 la 00:01, Amir Sarabadani ladsgroup@gmail.com a scris:
It should be accessible in https://grafana.wikimedia.org/d/000000156/wikidata-dispatch?orgId=1&refr... but I'm not sure why other types of max lag is not showing up.
You can also check maxlag like this: https://www.wikidata.org/w/api.php?action=query&titles=MediaWiki&for...
On Sat, Jan 25, 2020 at 10:53 PM Strainu strainu10@gmail.com wrote:
Actually, now that you mention it, it might be because of wikidata - I read stuff from there in all affected scripts. If I get throttled there 5s for each element, this might account for the long delays. Let me see if I can print some debug data. Any suggestion on what I should look for regarding Wikidata lag?
În sâm., 25 ian. 2020 la 23:48, Strainu strainu10@gmail.com a scris:
All sites. Currently I also encounter other issues (connection timeouts), but the throttling has been going on for days (at least since 1/20).
În sâm., 25 ian. 2020 la 23:46, Amir Sarabadani ladsgroup@gmail.com a scris:
Is it Wikidata only it's all sites? Wikidata uses another set of measures for maxlag, like lag between dispatching to other wiki and lag to WDQS.
HTH
On Sat, Jan 25, 2020 at 10:40 PM Strainu strainu10@gmail.com wrote:
Hi,
In the last few days my bots get throttled for minutes and minutes (up to 10, on the worse instances) on basically all sites I use (ro.wp, commons, wikidata). I knew about write throttle and I use the default setting there, but it seems that now I get throttled on read operations as well. I use pwb from git, currently synced at 0924e1655ef9e61c86e98884226b18d8250da368.
I checked the server lag and it is 0 most times I see the throttling. Any idea why do I get throttled for such long periods of time?
Thanks, Strainu
pywikibot mailing list pywikibot@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot
-- Amir (he/him)
pywikibot mailing list pywikibot@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot
pywikibot mailing list pywikibot@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot
-- Amir (he/him)
pywikibot mailing list pywikibot@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot
I would recommend creating a ticket against search platform team as this is due to WDQS nodes being behind Wikidata's production.
On Sat, Jan 25, 2020 at 11:27 PM Strainu strainu10@gmail.com wrote:
I can confirm the throttling is caused by Wikidata. Not sure why I didn't see it before, as I was using that api query that you gave above. Are there some (known) issues currently or should I log a bug on this?
Strainu
În dum., 26 ian. 2020 la 00:01, Amir Sarabadani ladsgroup@gmail.com a scris:
It should be accessible in
https://grafana.wikimedia.org/d/000000156/wikidata-dispatch?orgId=1&refr... but I'm not sure why other types of max lag is not showing up.
You can also check maxlag like this:
https://www.wikidata.org/w/api.php?action=query&titles=MediaWiki&for...
On Sat, Jan 25, 2020 at 10:53 PM Strainu strainu10@gmail.com wrote:
Actually, now that you mention it, it might be because of wikidata - I read stuff from there in all affected scripts. If I get throttled there 5s for each element, this might account for the long delays. Let me see if I can print some debug data. Any suggestion on what I should look for regarding Wikidata lag?
În sâm., 25 ian. 2020 la 23:48, Strainu strainu10@gmail.com a scris:
All sites. Currently I also encounter other issues (connection timeouts), but the throttling has been going on for days (at least since 1/20).
În sâm., 25 ian. 2020 la 23:46, Amir Sarabadani ladsgroup@gmail.com
a scris:
Is it Wikidata only it's all sites? Wikidata uses another set of
measures for maxlag, like lag between dispatching to other wiki and lag to WDQS.
HTH
On Sat, Jan 25, 2020 at 10:40 PM Strainu strainu10@gmail.com
wrote:
Hi,
In the last few days my bots get throttled for minutes and minutes
(up
to 10, on the worse instances) on basically all sites I use (ro.wp, commons, wikidata). I knew about write throttle and I use the
default
setting there, but it seems that now I get throttled on read operations as well. I use pwb from git, currently synced at 0924e1655ef9e61c86e98884226b18d8250da368.
I checked the server lag and it is 0 most times I see the
throttling.
Any idea why do I get throttled for such long periods of time?
Thanks, Strainu
pywikibot mailing list pywikibot@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot
-- Amir (he/him)
pywikibot mailing list pywikibot@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot
pywikibot mailing list pywikibot@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot
-- Amir (he/him)
pywikibot mailing list pywikibot@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot
pywikibot mailing list pywikibot@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot
Hello,
Have you finally created a ticket for this issue? It is very annoying: sometimes, more than 30 minutes "sleeping".
Regards, Albert.
Albert VILLANOVA DEL MORAL Phone: +33 6 24 20 42 38 Email: albert.villanova@gmail.com
Missatge de Amir Sarabadani ladsgroup@gmail.com del dia ds., 25 de gen. 2020 a les 23:55:
I would recommend creating a ticket against search platform team as this is due to WDQS nodes being behind Wikidata's production.
On Sat, Jan 25, 2020 at 11:27 PM Strainu strainu10@gmail.com wrote:
I can confirm the throttling is caused by Wikidata. Not sure why I didn't see it before, as I was using that api query that you gave above. Are there some (known) issues currently or should I log a bug on this?
Strainu
În dum., 26 ian. 2020 la 00:01, Amir Sarabadani ladsgroup@gmail.com a scris:
It should be accessible in
https://grafana.wikimedia.org/d/000000156/wikidata-dispatch?orgId=1&refr... but I'm not sure why other types of max lag is not showing up.
You can also check maxlag like this:
https://www.wikidata.org/w/api.php?action=query&titles=MediaWiki&for...
On Sat, Jan 25, 2020 at 10:53 PM Strainu strainu10@gmail.com wrote:
Actually, now that you mention it, it might be because of wikidata - I read stuff from there in all affected scripts. If I get throttled there 5s for each element, this might account for the long delays. Let me see if I can print some debug data. Any suggestion on what I should look for regarding Wikidata lag?
În sâm., 25 ian. 2020 la 23:48, Strainu strainu10@gmail.com a scris:
All sites. Currently I also encounter other issues (connection timeouts), but the throttling has been going on for days (at least since 1/20).
În sâm., 25 ian. 2020 la 23:46, Amir Sarabadani ladsgroup@gmail.com
a scris:
Is it Wikidata only it's all sites? Wikidata uses another set of
measures for maxlag, like lag between dispatching to other wiki and lag to WDQS.
HTH
On Sat, Jan 25, 2020 at 10:40 PM Strainu strainu10@gmail.com
wrote:
> > Hi, > > In the last few days my bots get throttled for minutes and
minutes (up
> to 10, on the worse instances) on basically all sites I use
(ro.wp,
> commons, wikidata). I knew about write throttle and I use the
default
> setting there, but it seems that now I get throttled on read > operations as well. I use pwb from git, currently synced at > 0924e1655ef9e61c86e98884226b18d8250da368. > > I checked the server lag and it is 0 most times I see the
throttling.
> Any idea why do I get throttled for such long periods of time? > > Thanks, > Strainu > > _______________________________________________ > pywikibot mailing list > pywikibot@lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/pywikibot
-- Amir (he/him)
pywikibot mailing list pywikibot@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot
pywikibot mailing list pywikibot@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot
-- Amir (he/him)
pywikibot mailing list pywikibot@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot
pywikibot mailing list pywikibot@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot
-- Amir (he/him)
pywikibot mailing list pywikibot@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot
https://phabricator.wikimedia.org/T243701
On Fri, Jan 31, 2020 at 10:01 AM Albert Villanova del Moral albertvillanova@gmail.com wrote:
Hello,
Have you finally created a ticket for this issue? It is very annoying: sometimes, more than 30 minutes "sleeping".
Regards, Albert.
Albert VILLANOVA DEL MORAL Phone: +33 6 24 20 42 38 Email: albert.villanova@gmail.com
Missatge de Amir Sarabadani ladsgroup@gmail.com del dia ds., 25 de gen. 2020 a les 23:55:
I would recommend creating a ticket against search platform team as this is due to WDQS nodes being behind Wikidata's production.
On Sat, Jan 25, 2020 at 11:27 PM Strainu strainu10@gmail.com wrote:
I can confirm the throttling is caused by Wikidata. Not sure why I didn't see it before, as I was using that api query that you gave above. Are there some (known) issues currently or should I log a bug on this?
Strainu
În dum., 26 ian. 2020 la 00:01, Amir Sarabadani ladsgroup@gmail.com a scris:
It should be accessible in https://grafana.wikimedia.org/d/000000156/wikidata-dispatch?orgId=1&refr... but I'm not sure why other types of max lag is not showing up.
You can also check maxlag like this: https://www.wikidata.org/w/api.php?action=query&titles=MediaWiki&for...
On Sat, Jan 25, 2020 at 10:53 PM Strainu strainu10@gmail.com wrote:
Actually, now that you mention it, it might be because of wikidata - I read stuff from there in all affected scripts. If I get throttled there 5s for each element, this might account for the long delays. Let me see if I can print some debug data. Any suggestion on what I should look for regarding Wikidata lag?
În sâm., 25 ian. 2020 la 23:48, Strainu strainu10@gmail.com a scris:
All sites. Currently I also encounter other issues (connection timeouts), but the throttling has been going on for days (at least since 1/20).
În sâm., 25 ian. 2020 la 23:46, Amir Sarabadani ladsgroup@gmail.com a scris: > > Is it Wikidata only it's all sites? Wikidata uses another set of measures for maxlag, like lag between dispatching to other wiki and lag to WDQS. > > HTH > > On Sat, Jan 25, 2020 at 10:40 PM Strainu strainu10@gmail.com wrote: >> >> Hi, >> >> In the last few days my bots get throttled for minutes and minutes (up >> to 10, on the worse instances) on basically all sites I use (ro.wp, >> commons, wikidata). I knew about write throttle and I use the default >> setting there, but it seems that now I get throttled on read >> operations as well. I use pwb from git, currently synced at >> 0924e1655ef9e61c86e98884226b18d8250da368. >> >> I checked the server lag and it is 0 most times I see the throttling. >> Any idea why do I get throttled for such long periods of time? >> >> Thanks, >> Strainu >> >> _______________________________________________ >> pywikibot mailing list >> pywikibot@lists.wikimedia.org >> https://lists.wikimedia.org/mailman/listinfo/pywikibot > > > > -- > Amir (he/him) > > _______________________________________________ > pywikibot mailing list > pywikibot@lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/pywikibot
pywikibot mailing list pywikibot@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot
-- Amir (he/him)
pywikibot mailing list pywikibot@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot
pywikibot mailing list pywikibot@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot
-- Amir (he/him)
pywikibot mailing list pywikibot@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot
pywikibot mailing list pywikibot@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot
Hi,
I encountered what I believe is the same issue as others. This happens on my integraality tool with entirely read-only operations, without any hits on the Query service.
I filed https://phabricator.wikimedia.org/T244030 with my investigations and the tracing showing how Pywikibot spent over 25 minutes doing a page.templatesWithParams().
Hope that helps,
-- Jean-Fred
Le ven. 31 janv. 2020 à 18:32, AntiCompositeNumber < anticompositenumber@gmail.com> a écrit :
https://phabricator.wikimedia.org/T243701
On Fri, Jan 31, 2020 at 10:01 AM Albert Villanova del Moral albertvillanova@gmail.com wrote:
Hello,
Have you finally created a ticket for this issue? It is very annoying:
sometimes, more than 30 minutes "sleeping".
Regards, Albert.
Albert VILLANOVA DEL MORAL Phone: +33 6 24 20 42 38 Email: albert.villanova@gmail.com
Missatge de Amir Sarabadani ladsgroup@gmail.com del dia ds., 25 de
gen. 2020 a les 23:55:
I would recommend creating a ticket against search platform team as
this is due to WDQS nodes being behind Wikidata's production.
On Sat, Jan 25, 2020 at 11:27 PM Strainu strainu10@gmail.com wrote:
I can confirm the throttling is caused by Wikidata. Not sure why I didn't see it before, as I was using that api query that you gave above. Are there some (known) issues currently or should I log a bug on this?
Strainu
În dum., 26 ian. 2020 la 00:01, Amir Sarabadani ladsgroup@gmail.com
a scris:
It should be accessible in
https://grafana.wikimedia.org/d/000000156/wikidata-dispatch?orgId=1&refr... but I'm not sure why other types of max lag is not showing up.
You can also check maxlag like this:
https://www.wikidata.org/w/api.php?action=query&titles=MediaWiki&for...
On Sat, Jan 25, 2020 at 10:53 PM Strainu strainu10@gmail.com
wrote:
Actually, now that you mention it, it might be because of wikidata - I read stuff from there in all affected scripts. If I get throttled there 5s for each element, this might account for the long delays.
Let
me see if I can print some debug data. Any suggestion on what I
should
look for regarding Wikidata lag?
În sâm., 25 ian. 2020 la 23:48, Strainu strainu10@gmail.com a
scris:
> > All sites. Currently I also encounter other issues (connection > timeouts), but the throttling has been going on for days (at least > since 1/20). > > În sâm., 25 ian. 2020 la 23:46, Amir Sarabadani <
ladsgroup@gmail.com> a scris:
> > > > Is it Wikidata only it's all sites? Wikidata uses another set
of measures for maxlag, like lag between dispatching to other wiki and lag to WDQS.
> > > > HTH > > > > On Sat, Jan 25, 2020 at 10:40 PM Strainu strainu10@gmail.com
wrote:
> >> > >> Hi, > >> > >> In the last few days my bots get throttled for minutes and
minutes (up
> >> to 10, on the worse instances) on basically all sites I use
(ro.wp,
> >> commons, wikidata). I knew about write throttle and I use the
default
> >> setting there, but it seems that now I get throttled on read > >> operations as well. I use pwb from git, currently synced at > >> 0924e1655ef9e61c86e98884226b18d8250da368. > >> > >> I checked the server lag and it is 0 most times I see the
throttling.
> >> Any idea why do I get throttled for such long periods of time? > >> > >> Thanks, > >> Strainu > >> > >> _______________________________________________ > >> pywikibot mailing list > >> pywikibot@lists.wikimedia.org > >> https://lists.wikimedia.org/mailman/listinfo/pywikibot > > > > > > > > -- > > Amir (he/him) > > > > _______________________________________________ > > pywikibot mailing list > > pywikibot@lists.wikimedia.org > > https://lists.wikimedia.org/mailman/listinfo/pywikibot
pywikibot mailing list pywikibot@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot
-- Amir (he/him)
pywikibot mailing list pywikibot@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot
pywikibot mailing list pywikibot@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot
-- Amir (he/him)
pywikibot mailing list pywikibot@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot
pywikibot mailing list pywikibot@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot
pywikibot mailing list pywikibot@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot
Jean-Fred, if you want to confirm whether the two are related you can try displaying the site which gets throttled. In my case it was always wikidata.
Strainu
Pe vineri, 31 ianuarie 2020, Jean-Frédéric jeanfrederic.wiki@gmail.com a scris:
Hi,
I encountered what I believe is the same issue as others. This happens on my integraality tool with entirely read-only operations, without any hits on the Query service.
I filed https://phabricator.wikimedia.org/T244030 with my investigations and the tracing showing how Pywikibot spent over 25 minutes doing a page.templatesWithParams().
Hope that helps,
-- Jean-Fred
Le ven. 31 janv. 2020 à 18:32, AntiCompositeNumber < anticompositenumber@gmail.com> a écrit :
https://phabricator.wikimedia.org/T243701
On Fri, Jan 31, 2020 at 10:01 AM Albert Villanova del Moral albertvillanova@gmail.com wrote:
Hello,
Have you finally created a ticket for this issue? It is very annoying:
sometimes, more than 30 minutes "sleeping".
Regards, Albert.
Albert VILLANOVA DEL MORAL Phone: +33 6 24 20 42 38 Email: albert.villanova@gmail.com
Missatge de Amir Sarabadani ladsgroup@gmail.com del dia ds., 25 de
gen. 2020 a les 23:55:
I would recommend creating a ticket against search platform team as
this is due to WDQS nodes being behind Wikidata's production.
On Sat, Jan 25, 2020 at 11:27 PM Strainu strainu10@gmail.com wrote:
I can confirm the throttling is caused by Wikidata. Not sure why I didn't see it before, as I was using that api query that you gave above. Are there some (known) issues currently or should I log a bug on this?
Strainu
În dum., 26 ian. 2020 la 00:01, Amir Sarabadani ladsgroup@gmail.com
a scris:
It should be accessible in https://grafana.wikimedia.org/
d/000000156/wikidata-dispatch?orgId=1&refresh=1m&fullscreen&panelId=22 but I'm not sure why other types of max lag is not showing up.
You can also check maxlag like this: https://www.wikidata.org/w/
api.php?action=query&titles=MediaWiki&format=json&maxlag=-1
On Sat, Jan 25, 2020 at 10:53 PM Strainu strainu10@gmail.com
wrote:
> > Actually, now that you mention it, it might be because of wikidata
> I read stuff from there in all affected scripts. If I get throttled > there 5s for each element, this might account for the long delays.
Let
> me see if I can print some debug data. Any suggestion on what I
should
> look for regarding Wikidata lag? > > În sâm., 25 ian. 2020 la 23:48, Strainu strainu10@gmail.com a
scris:
> > > > All sites. Currently I also encounter other issues (connection > > timeouts), but the throttling has been going on for days (at
least
> > since 1/20). > > > > În sâm., 25 ian. 2020 la 23:46, Amir Sarabadani <
ladsgroup@gmail.com> a scris:
> > > > > > Is it Wikidata only it's all sites? Wikidata uses another set
of measures for maxlag, like lag between dispatching to other wiki and lag to WDQS.
> > > > > > HTH > > > > > > On Sat, Jan 25, 2020 at 10:40 PM Strainu strainu10@gmail.com
wrote:
> > >> > > >> Hi, > > >> > > >> In the last few days my bots get throttled for minutes and
minutes (up
> > >> to 10, on the worse instances) on basically all sites I use
(ro.wp,
> > >> commons, wikidata). I knew about write throttle and I use the
default
> > >> setting there, but it seems that now I get throttled on read > > >> operations as well. I use pwb from git, currently synced at > > >> 0924e1655ef9e61c86e98884226b18d8250da368. > > >> > > >> I checked the server lag and it is 0 most times I see the
throttling.
> > >> Any idea why do I get throttled for such long periods of time? > > >> > > >> Thanks, > > >> Strainu > > >> > > >> _______________________________________________ > > >> pywikibot mailing list > > >> pywikibot@lists.wikimedia.org > > >> https://lists.wikimedia.org/mailman/listinfo/pywikibot > > > > > > > > > > > > -- > > > Amir (he/him) > > > > > > _______________________________________________ > > > pywikibot mailing list > > > pywikibot@lists.wikimedia.org > > > https://lists.wikimedia.org/mailman/listinfo/pywikibot > > _______________________________________________ > pywikibot mailing list > pywikibot@lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/pywikibot
-- Amir (he/him)
pywikibot mailing list pywikibot@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot
pywikibot mailing list pywikibot@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot
-- Amir (he/him)
pywikibot mailing list pywikibot@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot
pywikibot mailing list pywikibot@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot
pywikibot mailing list pywikibot@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot
-- Jean-Frédéric
Le ven. 31 janv. 2020 à 22:11, Strainu strainu10@gmail.com a écrit :
Jean-Fred, if you want to confirm whether the two are related you can try displaying the site which gets throttled. In my case it was always wikidata.
Oh, right, forgot to mention: that particular tool is only interacting with Wikidata (at least for now). (I have added Pastes with the log lines to the Phab ticket too).
Strainu
Pe vineri, 31 ianuarie 2020, Jean-Frédéric jeanfrederic.wiki@gmail.com a scris:
Hi,
I encountered what I believe is the same issue as others. This happens on my integraality tool with entirely read-only operations, without any hits on the Query service.
I filed https://phabricator.wikimedia.org/T244030 with my investigations and the tracing showing how Pywikibot spent over 25 minutes doing a page.templatesWithParams().
Hope that helps,
-- Jean-Fred
Le ven. 31 janv. 2020 à 18:32, AntiCompositeNumber < anticompositenumber@gmail.com> a écrit :
https://phabricator.wikimedia.org/T243701
On Fri, Jan 31, 2020 at 10:01 AM Albert Villanova del Moral albertvillanova@gmail.com wrote:
Hello,
Have you finally created a ticket for this issue? It is very annoying:
sometimes, more than 30 minutes "sleeping".
Regards, Albert.
Albert VILLANOVA DEL MORAL Phone: +33 6 24 20 42 38 Email: albert.villanova@gmail.com
Missatge de Amir Sarabadani ladsgroup@gmail.com del dia ds., 25 de
gen. 2020 a les 23:55:
I would recommend creating a ticket against search platform team as
this is due to WDQS nodes being behind Wikidata's production.
On Sat, Jan 25, 2020 at 11:27 PM Strainu strainu10@gmail.com wrote:
I can confirm the throttling is caused by Wikidata. Not sure why I didn't see it before, as I was using that api query that you gave above. Are there some (known) issues currently or should I log a bug on this?
Strainu
În dum., 26 ian. 2020 la 00:01, Amir Sarabadani ladsgroup@gmail.com
a scris:
> > It should be accessible in
https://grafana.wikimedia.org/d/000000156/wikidata-dispatch?orgId=1&refr... but I'm not sure why other types of max lag is not showing up.
> > You can also check maxlag like this:
https://www.wikidata.org/w/api.php?action=query&titles=MediaWiki&for...
> > On Sat, Jan 25, 2020 at 10:53 PM Strainu strainu10@gmail.com
wrote:
>> >> Actually, now that you mention it, it might be because of
wikidata -
>> I read stuff from there in all affected scripts. If I get
throttled
>> there 5s for each element, this might account for the long
delays. Let
>> me see if I can print some debug data. Any suggestion on what I
should
>> look for regarding Wikidata lag? >> >> În sâm., 25 ian. 2020 la 23:48, Strainu strainu10@gmail.com a
scris:
>> > >> > All sites. Currently I also encounter other issues (connection >> > timeouts), but the throttling has been going on for days (at
least
>> > since 1/20). >> > >> > În sâm., 25 ian. 2020 la 23:46, Amir Sarabadani <
ladsgroup@gmail.com> a scris:
>> > > >> > > Is it Wikidata only it's all sites? Wikidata uses another set
of measures for maxlag, like lag between dispatching to other wiki and lag to WDQS.
>> > > >> > > HTH >> > > >> > > On Sat, Jan 25, 2020 at 10:40 PM Strainu strainu10@gmail.com
wrote:
>> > >> >> > >> Hi, >> > >> >> > >> In the last few days my bots get throttled for minutes and
minutes (up
>> > >> to 10, on the worse instances) on basically all sites I use
(ro.wp,
>> > >> commons, wikidata). I knew about write throttle and I use
the default
>> > >> setting there, but it seems that now I get throttled on read >> > >> operations as well. I use pwb from git, currently synced at >> > >> 0924e1655ef9e61c86e98884226b18d8250da368. >> > >> >> > >> I checked the server lag and it is 0 most times I see the
throttling.
>> > >> Any idea why do I get throttled for such long periods of
time?
>> > >> >> > >> Thanks, >> > >> Strainu >> > >> >> > >> _______________________________________________ >> > >> pywikibot mailing list >> > >> pywikibot@lists.wikimedia.org >> > >> https://lists.wikimedia.org/mailman/listinfo/pywikibot >> > > >> > > >> > > >> > > -- >> > > Amir (he/him) >> > > >> > > _______________________________________________ >> > > pywikibot mailing list >> > > pywikibot@lists.wikimedia.org >> > > https://lists.wikimedia.org/mailman/listinfo/pywikibot >> >> _______________________________________________ >> pywikibot mailing list >> pywikibot@lists.wikimedia.org >> https://lists.wikimedia.org/mailman/listinfo/pywikibot > > > > -- > Amir (he/him) > > _______________________________________________ > pywikibot mailing list > pywikibot@lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/pywikibot
pywikibot mailing list pywikibot@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot
-- Amir (he/him)
pywikibot mailing list pywikibot@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot
pywikibot mailing list pywikibot@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot
pywikibot mailing list pywikibot@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot
-- Jean-Frédéric
pywikibot mailing list pywikibot@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikibot