jenkins-bot has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/476274 ) Change subject: [bugfix] Increase the delay if maxlag >> retry-after ...................................................................... [bugfix] Increase the delay if maxlag >> retry-after Normally retry-after value from response_headers is used for site access throttling if maxlag triggers. But sometime that retry-after value is too low compared to the lagtime. Increase the delay in this case. use config.retry_wait as fallback value doc updated Bug: T210606 Change-Id: Ide3a434f2e57f1afbba038cd12b93731a21f45f0 --- M pywikibot/throttle.py 1 file changed, 10 insertions(+), 5 deletions(-) Approvals: Dvorapa: Looks good to me, approved jenkins-bot: Verified diff --git a/pywikibot/throttle.py b/pywikibot/throttle.py index 23930ab..608f6dc 100644 --- a/pywikibot/throttle.py +++ b/pywikibot/throttle.py @@ -1,7 +1,7 @@ # -*- coding: utf-8 -*- """Mechanics to slow down wiki read and/or write rate.""" # -# (C) Pywikibot team, 2008-2018 +# (C) Pywikibot team, 2008-2019 # # Distributed under the terms of the MIT license. # @@ -286,9 +286,12 @@ """Seize the throttle lock due to server lag. Usually the self.retry-after value from response_header of the last - request if available which will be used for wait time. Default value - set by api and stored in self.retry_after is 5. If neither retry_after - nor lagtime is set, fallback to 5. + request if available which will be used for wait time. Otherwise + lagtime from api maxlag is used. If neither retry_after nor lagtime is + set, fallback to config.retry_wait. + + If the lagtime is disproportionately high compared to retry-after + value, the wait time will be increased. This method is used by api.request. It will prevent any thread from accessing this site. @@ -300,7 +303,9 @@ """ started = time.time() with self.lock: - waittime = self.retry_after or lagtime or 5 + waittime = lagtime or config.retry_wait + if self.retry_after: + waittime = max(self.retry_after, waittime / 5) # wait not more than retry_max seconds delay = min(waittime, config.retry_max) # account for any time we waited while acquiring the lock -- To view, visit https://gerrit.wikimedia.org/r/476274 To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ide3a434f2e57f1afbba038cd12b93731a21f45f0 Gerrit-Change-Number: 476274 Gerrit-PatchSet: 5 Gerrit-Owner: Xqt <info(a)gno.de> Gerrit-Reviewer: Dalba <dalba.wiki(a)gmail.com> Gerrit-Reviewer: Dvorapa <dvorapa(a)seznam.cz> Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com> Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl> Gerrit-Reviewer: Multichill <maarten(a)mdammers.nl> Gerrit-Reviewer: Xqt <info(a)gno.de> Gerrit-Reviewer: jenkins-bot (75)