Xqt has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/952531 )
Change subject: [IMPR] remove hints for MediaWiki < 1.27
......................................................................
[IMPR] remove hints for MediaWiki < 1.27
Bug: T306637
Change-Id: I403f3a3cd245d7f3775b99799e93ec39efe0e314
---
M pywikibot/login.py
M pywikibot/site/_generators.py
2 files changed, 17 insertions(+), 7 deletions(-)
Approvals:
jenkins-bot: Verified
Xqt: Looks good to me, approved
diff --git a/pywikibot/login.py b/pywikibot/login.py
index 54c57ee..b808ce1 100644
--- a/pywikibot/login.py
+++ b/pywikibot/login.py
@@ -464,7 +464,7 @@
@deprecated("site.tokens['login']", since='8.0.0')
def get_login_token(self) -> Optional[str]:
- """Fetch login token for MediaWiki 1.27+.
+ """Fetch login token.
.. deprecated:: 8.0
diff --git a/pywikibot/site/_generators.py b/pywikibot/site/_generators.py
index 50b6089..b9fa2b8 100644
--- a/pywikibot/site/_generators.py
+++ b/pywikibot/site/_generators.py
@@ -1709,7 +1709,7 @@
'ususers': usernames, 'usprop': usprop})
return usgen
- def randompages(self, total=None, namespaces=None,
+ def randompages(self, total: Optional[int] = None, namespaces=None,
redirects: Optional[bool] = False, content: bool = False):
"""Iterate a number of random pages.
@@ -1723,11 +1723,11 @@
:type namespaces: iterable of str or Namespace key,
or a single instance of those types. May be a '|' separated
list of namespace identifiers.
- :param redirects: if True, include only redirect pages in results,
- False does not include redirects and None (MW 1.26+) include both
- types. (default: False)
- :param content: if True, load the current content of each iterated page
- (default False)
+ :param redirects: if True, include only redirect pages in
+ results, False does not include redirects and None include
+ both types (default: False).
+ :param content: if True, load the current content of each
+ iterated page (default False).
:raises KeyError: a namespace identifier was not resolved
:raises TypeError: a namespace identifier has an inappropriate
type such as NoneType or bool
--
To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/952531
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Change-Id: I403f3a3cd245d7f3775b99799e93ec39efe0e314
Gerrit-Change-Number: 952531
Gerrit-PatchSet: 1
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot
Gerrit-MessageType: merged
jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/i18n/+/952174 )
Change subject: Localisation updates from https://translatewiki.net.
......................................................................
Localisation updates from https://translatewiki.net.
Change-Id: Id74896f85cf3f151240676e6abefb27ed69f53b9
---
M checkimages/nb.json
1 file changed, 12 insertions(+), 0 deletions(-)
Approvals:
L10n-bot: Looks good to me, approved
jenkins-bot: Verified
diff --git a/checkimages/nb.json b/checkimages/nb.json
index d401889..ef6556d 100644
--- a/checkimages/nb.json
+++ b/checkimages/nb.json
@@ -9,6 +9,9 @@
"checkimages-doubles-file-comment": "Bot: Fila finnes allerede på Commons, kan slettes",
"checkimages-doubles-head": "Duplikatfil",
"checkimages-doubles-talk-comment": "Bot: Fortell at fila allerede finnes på Commons",
+ "checkimages-doubles-talk-text": "Takk for at du lastet opp %(upload)s. Denne fila er imidlertid en kopi av:%(imag)sBotten har markert den nyeste av disse filene som dublett. Hvis du heller ønsker å la den andre fila slettes umiddelbart kan du gjøre det og fjerne slettemalen fra fila til venstre. Dette er en automatisk beskjed fra %(bot)s.",
+ "checkimages-forced-mode": "('''tvungen modus''')",
+ "checkimages-has-duplicates": "har følgende duplikater%(force)s:",
"checkimages-log-comment": "Bot: Oppdaterer loggen",
"checkimages-no-license-head": "Bilde uten lisens",
"checkimages-source-tag-comment": "Robot: Markerer den nylig opplastet umarkerte filen",
--
To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/i18n/+/952174
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/i18n
Gerrit-Branch: master
Gerrit-Change-Id: Id74896f85cf3f151240676e6abefb27ed69f53b9
Gerrit-Change-Number: 952174
Gerrit-PatchSet: 1
Gerrit-Owner: L10n-bot <l10n-bot(a)translatewiki.net>
Gerrit-Reviewer: L10n-bot <l10n-bot(a)translatewiki.net>
Gerrit-Reviewer: jenkins-bot
Gerrit-MessageType: merged
jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/949003 )
Change subject: [IMPR] Add a WaitingMixin to Requests and SparqlQuery class
......................................................................
[IMPR] Add a WaitingMixin to Requests and SparqlQuery class
Add a WaitingMixin class which handles wait cycles to Requests and
SparqlQuery class to prevent code duplication.
Change-Id: Id1f528044a4d95587431b0ccd7fa10330c6a9826
---
M pywikibot/data/sparql.py
M pywikibot/data/__init__.py
M pywikibot/data/api/_requests.py
3 files changed, 81 insertions(+), 45 deletions(-)
Approvals:
Xqt: Looks good to me, approved
jenkins-bot: Verified
diff --git a/pywikibot/data/__init__.py b/pywikibot/data/__init__.py
index 7db53e3..1dd0b02 100644
--- a/pywikibot/data/__init__.py
+++ b/pywikibot/data/__init__.py
@@ -1,6 +1,56 @@
"""Module providing several layers of data access to the wiki."""
#
-# (C) Pywikibot team, 2007-2022
+# (C) Pywikibot team, 2007-2023
#
# Distributed under the terms of the MIT license.
#
+from typing import Optional
+
+import pywikibot
+
+
+class WaitingMixin:
+
+ """A mixin to implement wait cycles.
+
+ .. versionadded:: 8.4
+
+ :ivar int max_retries: Maximum number of times to retry an API
+ request before quitting. Defaults to ``config.max_retries`` if
+ attribute is missing.
+ :ivar int retry_wait: Minimum time to wait before resubmitting a
+ failed API request. Defaults to ``config.retry_wait`` if
+ attribute is missing.
+ :ivar int current_retries: counter of retries made for the current
+ request. Starting with 1 if attribute is missing.
+ """
+
+ def wait(self, delay: Optional[int] = None) -> None:
+ """Determine how long to wait after a failed request.
+
+ :param delay: Minimum time in seconds to wait. Overwrites
+ ``retry_wait`` variable if given. The delay doubles each
+ retry until ``retry_max`` seconds is reached.
+ """
+ if not hasattr(self, 'max_retries'):
+ self.max_retries = pywikibot.config.max_retries
+
+ if not hasattr(self, 'retry_wait'):
+ self.retry_wait = pywikibot.config.retry_wait
+
+ if self.current_retries > self.max_retries:
+ raise pywikibot.exceptions.TimeoutError(
+ 'Maximum retries attempted without success.')
+
+ if not hasattr(self, 'current_retries'):
+ self.current_retries = 1
+ else:
+ self.current_retries += 1
+
+ # double the next wait, but do not exceed config.retry_max seconds
+ delay = delay or self.retry_wait
+ delay *= 2 ** (self.current_retries - 1)
+ delay = min(delay, pywikibot.config.retry_max)
+
+ pywikibot.warning(f'Waiting {delay:.1f} seconds before retrying.')
+ pywikibot.sleep(delay)
diff --git a/pywikibot/data/api/_requests.py b/pywikibot/data/api/_requests.py
index 7e7758c..e13bd6f 100644
--- a/pywikibot/data/api/_requests.py
+++ b/pywikibot/data/api/_requests.py
@@ -24,6 +24,7 @@
from pywikibot import config
from pywikibot.backports import Callable, Dict, Match, Tuple, removeprefix
from pywikibot.comms import http
+from pywikibot.data import WaitingMixin
from pywikibot.exceptions import (
Client414Error,
Error,
@@ -32,7 +33,6 @@
NoUsernameError,
Server504Error,
SiteDefinitionError,
- TimeoutError,
)
from pywikibot.login import LoginStatus
from pywikibot.textlib import removeDisabledParts, removeHTMLParts
@@ -72,7 +72,7 @@
r'Waiting for [\w.: ]+: (?P<lag>\d+(?:\.\d+)?) seconds? lagged')
-class Request(MutableMapping):
+class Request(MutableMapping, WaitingMixin):
"""A request to a Site's api.php interface.
@@ -126,6 +126,9 @@
True
>>> sorted(data['query'])
['namespaces', 'userinfo']
+
+ .. versionchanged:: 8.4
+ inherited from WaitingMixin.
"""
# To make sure the default value of 'parameters' can be identified.
@@ -194,14 +197,9 @@
self.throttle = throttle
self.use_get = use_get
- if max_retries is None:
- self.max_retries = pywikibot.config.max_retries
- else:
+ if max_retries is not None:
self.max_retries = max_retries
- self.current_retries = 0
- if retry_wait is None:
- self.retry_wait = pywikibot.config.retry_wait
- else:
+ if retry_wait is not None:
self.retry_wait = retry_wait
self.json_warning = False
# The only problem with that system is that it won't detect when
@@ -1120,20 +1118,6 @@
raise MaxlagTimeoutError(msg)
- def wait(self, delay=None):
- """Determine how long to wait after a failed request."""
- self.current_retries += 1
- if self.current_retries > self.max_retries:
- raise TimeoutError('Maximum retries attempted without success.')
-
- # double the next wait, but do not exceed config.retry_max seconds
- delay = delay or self.retry_wait
- delay *= 2 ** (self.current_retries - 1)
- delay = min(delay, config.retry_max)
-
- pywikibot.warning(f'Waiting {delay:.1f} seconds before retrying.')
- pywikibot.sleep(delay)
-
class CachedRequest(Request):
diff --git a/pywikibot/data/sparql.py b/pywikibot/data/sparql.py
index 5c4e14e..23ffb1d 100644
--- a/pywikibot/data/sparql.py
+++ b/pywikibot/data/sparql.py
@@ -10,10 +10,11 @@
from requests.exceptions import Timeout
-from pywikibot import Site, config, sleep, warning
+from pywikibot import Site
from pywikibot.backports import Dict, List, removeprefix
from pywikibot.comms import http
-from pywikibot.exceptions import Error, TimeoutError
+from pywikibot.data import WaitingMixin
+from pywikibot.exceptions import Error
try:
@@ -25,11 +26,14 @@
'Accept': 'application/sparql-results+json'}
-class SparqlQuery:
- """
- SPARQL Query class.
+class SparqlQuery(WaitingMixin):
+ """SPARQL Query class.
This class allows to run SPARQL queries against any SPARQL endpoint.
+
+ .. versionchanged:: 8.4
+ inherited from :class:`data.WaitingMixin` which provides a
+ :meth:`data.WaitingMixin.wait` method.
"""
def __init__(self,
@@ -78,13 +82,9 @@
self.last_response = None
- if max_retries is None:
- self.max_retries = config.max_retries
- else:
+ if max_retries is not None:
self.max_retries = max_retries
- if retry_wait is None:
- self.retry_wait = config.retry_wait
- else:
+ if retry_wait is not None:
self.retry_wait = retry_wait
def get_last_response(self):
@@ -158,16 +158,6 @@
return None
- def wait(self):
- """Determine how long to wait after a failed request."""
- self.max_retries -= 1
- if self.max_retries < 0:
- raise TimeoutError('Maximum retries attempted without success.')
- warning(f'Waiting {self.retry_wait} seconds before retrying.')
- sleep(self.retry_wait)
- # double the next wait, but do not exceed config.retry_max seconds
- self.retry_wait = min(config.retry_max, self.retry_wait * 2)
-
def ask(self, query: str,
headers: Optional[Dict[str, str]] = None) -> bool:
"""
--
To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/949003
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Change-Id: Id1f528044a4d95587431b0ccd7fa10330c6a9826
Gerrit-Change-Number: 949003
Gerrit-PatchSet: 2
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot
Gerrit-MessageType: merged
jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/949018 )
Change subject: [IMPR] Improve throttle module
......................................................................
[IMPR] Improve throttle module
- remove dropdelay and releasepid attributes
- introduce a new expiry class attribute instead which holds an
appropriate constant
- change checkdelay attribute to Throttle class attribute
Change-Id: I3a11ca0e705d3444ace46515264383ae3207a15d
---
M pywikibot/throttle.py
1 file changed, 61 insertions(+), 24 deletions(-)
Approvals:
Xqt: Looks good to me, approved
jenkins-bot: Verified
diff --git a/pywikibot/throttle.py b/pywikibot/throttle.py
index 2241059..0daf9fc 100644
--- a/pywikibot/throttle.py
+++ b/pywikibot/throttle.py
@@ -21,12 +21,13 @@
FORMAT_LINE = '{module_id} {pid} {time} {site}\n'
ProcEntry = namedtuple('ProcEntry', ['module_id', 'pid', 'time', 'site'])
-# global process identifier
-#
-# When the first Throttle is instantiated, it will set this variable to a
-# positive integer, which will apply to all throttle objects created by this
-# process.
-pid = False
+pid: Union[bool, int] = False
+"""global process identifier
+
+When the first Throttle is instantiated, it will set this variable to a
+positive integer, which will apply to all throttle objects created by
+this process.
+"""
class Throttle:
@@ -46,6 +47,11 @@
:param writedelay: The write delay
"""
+ # Check throttle file again after this many seconds:
+ checkdelay: int = 300
+ # The number of seconds entries of a process need to be counted
+ expiry: int = 600
+
def __init__(self, site: Union['pywikibot.site.BaseSite', str], *,
mindelay: Optional[int] = None,
maxdelay: Optional[int] = None,
@@ -63,15 +69,6 @@
self.last_write = 0
self.next_multiplicity = 1.0
- # Check logfile again after this many seconds:
- self.checkdelay = 300
-
- # Ignore processes that have not made a check in this many seconds:
- self.dropdelay = 600
-
- # Free the process id after this many seconds:
- self.releasepid = 1200
-
self.retry_after = 0 # set by http.request
self.delay = 0
self.checktime = 0
@@ -83,13 +80,39 @@
@property
@deprecated(since='6.2')
def multiplydelay(self) -> bool:
- """DEPRECATED attribute."""
+ """DEPRECATED attribute.
+
+ .. deprecated:: 6.2
+ """
return True
@multiplydelay.setter
@deprecated(since='6.2')
def multiplydelay(self) -> None:
- """DEPRECATED attribute setter."""
+ """DEPRECATED attribute setter.
+
+ .. deprecated:: 6.2
+ """
+
+ @property
+ @deprecated('expiry', since='8.4')
+ def dropdelay(self):
+ """Ignore processes that have not made a check in this many seconds.
+
+ .. deprecated:: 8.4
+ use *expiry* instead.
+ """
+ return self.expiry
+
+ @property
+ @deprecated('expiry', since='8.4')
+ def releasepid(self):
+ """Free the process id after this many seconds.
+
+ .. deprecated:: 8.4
+ use *expiry* instead.
+ """
+ return self.expiry
@staticmethod
def _module_hash(module=None) -> str:
@@ -152,12 +175,12 @@
now = time.time()
for proc in self._read_file(raise_exc=True):
used_pids.add(proc.pid)
- if now - proc.time > self.releasepid:
- continue # process has expired, drop from file
- if now - proc.time <= self.dropdelay \
- and proc.site == mysite \
- and proc.pid != pid:
+ if now - proc.time > self.expiry:
+ continue # process has expired, drop from file
+
+ if proc.site == mysite and proc.pid != pid:
count += 1
+
if proc.site != mysite or proc.pid != pid:
processes.append(proc)
@@ -178,8 +201,8 @@
self._write_file(sorted(processes, key=lambda p: p.pid))
self.process_multiplicity = count
- pywikibot.log('Found {} {} processes running, including this one.'
- .format(count, mysite))
+ pywikibot.log(f'Found {count} {mysite} processes running,'
+ ' including this one.')
def setDelays(
self,
--
To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/949018
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Change-Id: I3a11ca0e705d3444ace46515264383ae3207a15d
Gerrit-Change-Number: 949018
Gerrit-PatchSet: 2
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot
Gerrit-MessageType: merged
jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/i18n/+/951083 )
Change subject: Localisation updates from https://translatewiki.net.
......................................................................
Localisation updates from https://translatewiki.net.
Change-Id: I78eda005944a609c42485b6defd3ee9e16ae71cc
---
M checkimages/ru.json
M checkimages/fr.json
2 files changed, 13 insertions(+), 3 deletions(-)
Approvals:
L10n-bot: Looks good to me, approved
jenkins-bot: Verified
diff --git a/checkimages/fr.json b/checkimages/fr.json
index 764c298..9303b02 100644
--- a/checkimages/fr.json
+++ b/checkimages/fr.json
@@ -14,7 +14,7 @@
"checkimages-doubles-file-comment": "Robot : fichier déjà sur Commons, peut être supprimé",
"checkimages-doubles-head": "Fichier dupliqué",
"checkimages-doubles-talk-comment": "Robot : notification que le fichier existe déjà sur Commons",
- "checkimages-doubles-talk-text": "Merci d’avoir téléchargé %(upload)s. Toutefois, ce fichier est une copie de %(image)s. Le robot a marqué comme doublon du fichier qui a été le moins utilisé ou le plus récent. Si vous pensez qu’il est plus approprié de laisser le fichier marqué comme devant être supprimé immédiatement, sentez-vous libre de supprimer les fichiers dupliqués et supprimer le modèle de suppression de celui à conserver. Ceci est un message automatique de %(bot)s.",
+ "checkimages-doubles-talk-text": "Merci d’avoir téléchargé %(upload)s. Toutefois, ce fichier est une copie de :%(image)sLe robot a marqué comme doublon du fichier qui a été le moins utilisé ou le plus récent. Si vous pensez qu’il est plus approprié de laisser le fichier marqué comme devant être supprimé immédiatement, sentez-vous libre de supprimer les fichiers dupliqués et supprimer le modèle de suppression de celui à conserver. Ceci est un message automatique de %(bot)s.",
"checkimages-forced-mode": "('''mode forcé''')",
"checkimages-has-duplicates": "a les doublons suivants%(force)s :",
"checkimages-log-comment": "Robot : mise à jour du journal",
diff --git a/checkimages/ru.json b/checkimages/ru.json
index 2f94ef7..e2317ee 100644
--- a/checkimages/ru.json
+++ b/checkimages/ru.json
@@ -3,14 +3,15 @@
"authors": [
"INS Pirat",
"Movses",
- "Okras"
+ "Okras",
+ "Pacha Tchernof"
]
},
"checkimages-deletion-comment": "Бот: Добавление %(adding)s",
"checkimages-doubles-file-comment": "Бот: Файл уже на Викискладе, а потому может быть удалён",
"checkimages-doubles-head": "Дубликат",
"checkimages-doubles-talk-comment": "Бот: Нотификация в случае уже существующего файла на Викискладе",
- "checkimages-doubles-talk-text": "Спасибо за загрузку %(upload)s. Однако этот файл является копией %(image)s. Бот пометил как дубликат файл, который оказался наименее используемым или последним. Если вы считаете более целесообразным оставить файл, помеченный как подлежащий немедленному удалению, не стесняйтесь удалить дубликаты файлов и удалить шаблон удаления из того файла, который нужно оставить. Это автоматическое сообщение от %(bot)s.",
+ "checkimages-doubles-talk-text": "Спасибо за загрузку %(upload)s. Однако этот файл является копией:%(image)sБот пометил как дубликат файл, который оказался наименее используемым или последним. Если вы считаете более целесообразным оставить файл, помеченный как подлежащий немедленному удалению, не стесняйтесь удалить дубликаты файлов и удалить шаблон удаления из того файла, который нужно оставить. Это автоматическое сообщение от %(bot)s.",
"checkimages-forced-mode": "('''принудительный режим''')",
"checkimages-has-duplicates": "имеет следующие дубликаты%(force)s:",
"checkimages-log-comment": "Бот: Обновление журнала",
--
To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/i18n/+/951083
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/i18n
Gerrit-Branch: master
Gerrit-Change-Id: I78eda005944a609c42485b6defd3ee9e16ae71cc
Gerrit-Change-Number: 951083
Gerrit-PatchSet: 1
Gerrit-Owner: L10n-bot <l10n-bot(a)translatewiki.net>
Gerrit-Reviewer: L10n-bot <l10n-bot(a)translatewiki.net>
Gerrit-Reviewer: jenkins-bot
Gerrit-MessageType: merged