jenkins-bot has submitted this change and it was merged. (
https://gerrit.wikimedia.org/r/476274 )
Change subject: [bugfix] Increase the delay if maxlag >> retry-after
......................................................................
[bugfix] Increase the delay if maxlag >> retry-after
Normally retry-after value from response_headers is used for site access
throttling if maxlag triggers. But sometime that retry-after value is
too low compared to the lagtime. Increase the delay in this case.
use config.retry_wait as fallback value
doc updated
Bug: T210606
Change-Id: Ide3a434f2e57f1afbba038cd12b93731a21f45f0
---
M pywikibot/throttle.py
1 file changed, 10 insertions(+), 5 deletions(-)
Approvals:
Dvorapa: Looks good to me, approved
jenkins-bot: Verified
diff --git a/pywikibot/throttle.py b/pywikibot/throttle.py
index 23930ab..608f6dc 100644
--- a/pywikibot/throttle.py
+++ b/pywikibot/throttle.py
@@ -1,7 +1,7 @@
# -*- coding: utf-8 -*-
"""Mechanics to slow down wiki read and/or write rate."""
#
-# (C) Pywikibot team, 2008-2018
+# (C) Pywikibot team, 2008-2019
#
# Distributed under the terms of the MIT license.
#
@@ -286,9 +286,12 @@
"""Seize the throttle lock due to server lag.
Usually the self.retry-after value from response_header of the last
- request if available which will be used for wait time. Default value
- set by api and stored in self.retry_after is 5. If neither retry_after
- nor lagtime is set, fallback to 5.
+ request if available which will be used for wait time. Otherwise
+ lagtime from api maxlag is used. If neither retry_after nor lagtime is
+ set, fallback to config.retry_wait.
+
+ If the lagtime is disproportionately high compared to retry-after
+ value, the wait time will be increased.
This method is used by api.request. It will prevent any thread from
accessing this site.
@@ -300,7 +303,9 @@
"""
started = time.time()
with self.lock:
- waittime = self.retry_after or lagtime or 5
+ waittime = lagtime or config.retry_wait
+ if self.retry_after:
+ waittime = max(self.retry_after, waittime / 5)
# wait not more than retry_max seconds
delay = min(waittime, config.retry_max)
# account for any time we waited while acquiring the lock
--
To view, visit
https://gerrit.wikimedia.org/r/476274
To unsubscribe, or for help writing mail filters, visit
https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Ide3a434f2e57f1afbba038cd12b93731a21f45f0
Gerrit-Change-Number: 476274
Gerrit-PatchSet: 5
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: Dalba <dalba.wiki(a)gmail.com>
Gerrit-Reviewer: Dvorapa <dvorapa(a)seznam.cz>
Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl>
Gerrit-Reviewer: Multichill <maarten(a)mdammers.nl>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot (75)