jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/1007565?usp=email )
Change subject: [IMPR] backport itertools.batched from Python 3.13
......................................................................
[IMPR] backport itertools.batched from Python 3.13
Python 3.13 batched has a strict option like our own itertools.itergroup
function.
Change-Id: Id3a334ac60ad20128ff14224fc1fc0eb2f5ac52b
---
M pywikibot/backports.py
M pywikibot/tools/itertools.py
2 files changed, 41 insertions(+), 14 deletions(-)
Approvals:
Xqt: Looks good to me, approved
jenkins-bot: Verified
diff --git a/pywikibot/backports.py b/pywikibot/backports.py
index b7a2dab..e49e815 100644
--- a/pywikibot/backports.py
+++ b/pywikibot/backports.py
@@ -5,7 +5,7 @@
instead. The *SimpleQueue* queue; use ``queue.SimpleQueue`` instead.
"""
#
-# (C) Pywikibot team, 2014-2023
+# (C) Pywikibot team, 2014-2024
#
# Distributed under the terms of the MIT license.
#
@@ -135,11 +135,13 @@
# gh-98363
-if PYTHON_VERSION < (3, 12) or SPHINX_RUNNING:
- def batched(iterable, n: int) -> Generator[tuple, None, None]:
+if PYTHON_VERSION < (3, 13) or SPHINX_RUNNING:
+ def batched(iterable, n: int, *,
+ strict: bool = False) -> Generator[tuple, None, None]:
"""Batch data from the *iterable* into tuples of length *n*.
- .. note:: The last batch may be shorter than *n*.
+ .. note:: The last batch may be shorter than *n* if *strict* is
+ True or raise a ValueError otherwise.
Example:
@@ -159,17 +161,32 @@
<library/itertools.html#itertools.batched>`,
backported from Python 3.12.
.. versionadded:: 8.2
+ .. versionchanged:: 9.0
+ Added *strict* option, backported from Python 3.13
:param n: How many items of the iterable to get in one chunk
+ :param strict: raise a ValueError if the final batch is shorter
+ than *n*.
+ :raise ValueError: the final batch is shorter than *n*.
"""
- group = []
- for item in iterable:
- group.append(item)
- if len(group) == n:
+ msg = f'The final batch is shorter than n={n}'
+ if PYTHON_VERSION < (3, 12):
+ group = []
+ for item in iterable:
+ group.append(item)
+ if len(group) == n:
+ yield tuple(group)
+ group.clear()
+ if group:
+ if strict:
+ raise ValueError(msg)
yield tuple(group)
- group.clear()
- if group:
- yield tuple(group)
+ else: # PYTHON_VERSION == (3, 12)
+ from itertools import batched as _batched
+ for group in _batched(iterable, n):
+ if strict and len(group) < n:
+ raise ValueError(msg)
+ yield group
else:
from itertools import batched # type: ignore[no-redef]
diff --git a/pywikibot/tools/itertools.py b/pywikibot/tools/itertools.py
index 4a17c43..41fc5ca 100644
--- a/pywikibot/tools/itertools.py
+++ b/pywikibot/tools/itertools.py
@@ -60,9 +60,7 @@
not divisible by `size`.
:raises ValueError: iterable is not divisible by size
"""
- for group in batched(iterable, size):
- if strict and len(group) < size:
- raise ValueError('iterable is not divisible by size.')
+ for group in batched(iterable, size, strict=strict):
yield list(group)
--
To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/1007565?usp=email
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Change-Id: Id3a334ac60ad20128ff14224fc1fc0eb2f5ac52b
Gerrit-Change-Number: 1007565
Gerrit-PatchSet: 4
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot
Gerrit-MessageType: merged
jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/1004618?usp=email )
Change subject: [IMPR] Raise APIError if the same error comes twice within submit loop
......................................................................
[IMPR] Raise APIError if the same error comes twice within submit loop
api.Request.submit handles API errors within a loop but in some
circumstances the loop will never leaved if the same error comes again
and again. Therefore raise APIError with the previous API error if the
same error occured twice in the same submit loop-
Bug: T357870
Change-Id: Id9f140c9d8815ef622d47cd90a29518f23665a4a
---
M pywikibot/data/api/_requests.py
1 file changed, 42 insertions(+), 1 deletion(-)
Approvals:
JJMC89: Looks good to me, approved
jenkins-bot: Verified
diff --git a/pywikibot/data/api/_requests.py b/pywikibot/data/api/_requests.py
index bc40eec..8facad1 100644
--- a/pywikibot/data/api/_requests.py
+++ b/pywikibot/data/api/_requests.py
@@ -1,6 +1,6 @@
"""Objects representing API requests."""
#
-# (C) Pywikibot team, 2007-2023
+# (C) Pywikibot team, 2007-2024
#
# Distributed under the terms of the MIT license.
#
@@ -945,18 +945,36 @@
self._params['token'] = tokens
return True
+ def wait(self, delay: int | None = None) -> None:
+ """Determine how long to wait after a failed request.
+
+ Also reset last API error with wait cycles.
+
+ .. versionadded: 9.0
+
+ :param delay: Minimum time in seconds to wait. Overwrites
+ ``retry_wait`` variable if given. The delay doubles each
+ retry until ``retry_max`` seconds is reached.
+ """
+ self.last_error = dict.fromkeys(['code', 'info'])
+ super().wait(delay)
+
def submit(self) -> dict:
"""Submit a query and parse the response.
.. versionchanged:: 8.0.4
in addition to *readapidenied* also try to login when API
response is *notloggedin*.
+ .. versionchanged:: 9.0
+ Raise :exc:`pywikibot.exceptions.APIError` if the same error
+ comes twice in a row within the loop.
:return: a dict containing data retrieved from api.php
"""
self._add_defaults()
use_get = self._use_get()
retries = 0
+ self.last_error = dict.fromkeys(['code', 'info'])
while True:
paramstring = self._http_param_string()
@@ -1003,6 +1021,11 @@
code = error.setdefault('code', 'Unknown')
info = error.setdefault('info', None)
+ if (code == self.last_error['code']
+ and info == self.last_error['info']):
+ raise pywikibot.exceptions.APIError(**self.last_error)
+ self.last_error = error
+
if not self._logged_in(code):
continue
@@ -1019,6 +1042,8 @@
lag = float(lag['lag']) if lag else 0.0
self.site.throttle.lag(lag * retries)
+ # reset last error
+ self.last_error = dict.fromkeys(['code', 'info'])
continue
if code == 'help' and self.action == 'help':
@@ -1060,6 +1085,7 @@
pywikibot.error(f'Retrying failed {msg}')
continue
raise NoUsernameError(f'Failed {msg}')
+
if code == 'cirrussearch-too-busy-error': # T170647
self.wait()
continue
--
To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/1004618?usp=email
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Change-Id: Id9f140c9d8815ef622d47cd90a29518f23665a4a
Gerrit-Change-Number: 1004618
Gerrit-PatchSet: 4
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: JJMC89 <JJMC89.Wikimedia(a)gmail.com>
Gerrit-Reviewer: jenkins-bot
Gerrit-MessageType: merged
jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/1005760?usp=email )
Change subject: [bugfix] use require_modules instead of has_module in proofreadpage_tests.py
......................................................................
[bugfix] use require_modules instead of has_module in proofreadpage_tests.py
has_module() does not find 'bs4' but 'beautifulsoup4'.
Use aspects.require_modules decorator instead. Use it on
setUpClass to skip all tests at once; decoration the class
would skip every test method.
Use ModuleNotFoundError instead if ImportError in require_modules
which is raised when a module cannot be found.
This exception was introduced with Python 3.6
Change-Id: I76e17adb2476b02b17ee5e514889fe1b497a1588
---
M tests/aspects.py
M tests/proofreadpage_tests.py
2 files changed, 21 insertions(+), 9 deletions(-)
Approvals:
Xqt: Looks good to me, approved
jenkins-bot: Verified
diff --git a/tests/aspects.py b/tests/aspects.py
index 4ae285e..efd5616 100644
--- a/tests/aspects.py
+++ b/tests/aspects.py
@@ -290,7 +290,7 @@
for required_module in required_modules:
try:
__import__(required_module, globals(), locals(), [], 0)
- except ImportError:
+ except ModuleNotFoundError:
missing += [required_module]
if not missing:
return obj
diff --git a/tests/proofreadpage_tests.py b/tests/proofreadpage_tests.py
index 09a9677..0e2c525 100755
--- a/tests/proofreadpage_tests.py
+++ b/tests/proofreadpage_tests.py
@@ -1,7 +1,7 @@
#!/usr/bin/env python3
"""Tests for the proofreadpage module."""
#
-# (C) Pywikibot team, 2015-2023
+# (C) Pywikibot team, 2015-2024
#
# Distributed under the terms of the MIT license.
#
@@ -21,8 +21,6 @@
ProofreadPage,
TagAttr,
)
-from pywikibot.tools import has_module
-from tests import unittest_print
from tests.aspects import TestCase, require_modules
from tests.basepage import (
BasePageLoadRevisionsCachingTestBase,
@@ -439,13 +437,9 @@
"""Run tests which needs bs4 beeing installed."""
@classmethod
+ @require_modules('bs4')
def setUpClass(cls):
"""Check whether bs4 module is installed already."""
- if not has_module('bs4'):
- unittest_print(
- f'all tests ({__name__}.{cls.__name__})\n{cls.__doc__}.. ',
- end='\n')
- cls.skipTest(cls, 'bs4 not installed')
super().setUpClass()
--
To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/1005760?usp=email
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Change-Id: I76e17adb2476b02b17ee5e514889fe1b497a1588
Gerrit-Change-Number: 1005760
Gerrit-PatchSet: 3
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot
Gerrit-MessageType: merged
Xqt has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/1005719?usp=email )
Change subject: [tests] install bs4 for pywikibot CI tests
......................................................................
[tests] install bs4 for pywikibot CI tests
Most proofreadpage_tests were not executed due to missing bs4.
Install it for tests and code coverage with pywikibot CI action.
Change-Id: I6dd70a6e2538b19ff64d8a22a64c5c818f496f94
---
M .github/workflows/pywikibot-ci.yml
1 file changed, 13 insertions(+), 0 deletions(-)
Approvals:
Xqt: Verified; Looks good to me, approved
diff --git a/.github/workflows/pywikibot-ci.yml b/.github/workflows/pywikibot-ci.yml
index ea447d1..8c91dfb 100644
--- a/.github/workflows/pywikibot-ci.yml
+++ b/.github/workflows/pywikibot-ci.yml
@@ -92,6 +92,7 @@
pip --version
if [ -f dev-requirements.txt ]; then pip install -r dev-requirements.txt; fi
if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
+ pip install "beautifulsoup4>=4.7.1"
pip install wikitextparser
- name: Generate family files
--
To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/1005719?usp=email
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Change-Id: I6dd70a6e2538b19ff64d8a22a64c5c818f496f94
Gerrit-Change-Number: 1005719
Gerrit-PatchSet: 1
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-MessageType: merged
jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/1005516?usp=email )
Change subject: [doc] Update ROADMAP.rst and CHANGELOG.rst
......................................................................
[doc] Update ROADMAP.rst and CHANGELOG.rst
Also wrap lines in HISTORY.rst and remove tests changes there.
Change-Id: I4825991fa052dd8ea30700595383aefe6b4caaa1
---
M HISTORY.rst
M ROADMAP.rst
M scripts/CHANGELOG.rst
3 files changed, 68 insertions(+), 27 deletions(-)
Approvals:
Xqt: Looks good to me, approved
jenkins-bot: Verified
diff --git a/HISTORY.rst b/HISTORY.rst
index 3a6e6ad..94482fd 100644
--- a/HISTORY.rst
+++ b/HISTORY.rst
@@ -53,7 +53,8 @@
and its *preloaded_modules* property was deprecated, the :meth:`data.api.ParamInfo.normalize_paraminfo`
method became a staticmethod (:phab:`T306637`)
* raise ValueError when :class:`pywikibot.FilePage` title doesn't have a valid file extension (:phab:`T345786`)
-* :attr:`site.APISite.file_extensions <pywikibot.site._apisite.APISite.file_extensions>` property was added (:phab:`T345786`)
+* :attr:`site.APISite.file_extensions <pywikibot.site._apisite.APISite.file_extensions>` property was added
+ (:phab:`T345786`)
* ``dropdelay`` and ``releasepid`` attributes of :class:`throttle.Throttle` where deprecated
in favour of `expiry` class attribute
* Add https scheme if missing in url asked by :mod:`pywikibot.scripts.generate_family_file`
@@ -166,7 +167,8 @@
* Raise :exc:`exceptions.NoPageError` when deleting a missing Page (:phab:`T332924`)
* ``text`` parameter of :class:`proofreadpage.PagesTagParser` has a default value
* L10N updates
-* Ignore talk pages with :meth:`APISite.watched_pages()<pywikibot.site._generators.GeneratorsMixin.watched_pages>` (:phab:`T330806`)
+* Ignore talk pages with :meth:`APISite.watched_pages()<pywikibot.site._generators.GeneratorsMixin.watched_pages>`
+ (:phab:`T330806`)
* Load page info when creating a page if not updated previously (:phab:`T330980`)
* Improve flush exception logging (:phab:`T108444`)
@@ -177,7 +179,8 @@
* L10N Updates
* Minimal needed mwparserfromhell was decreased to 0.5.2 (:phab:`T326498`, :phab:`T327600`)
-* No longer lazy load password cookies (:phab:`T271858`, :phab:`T326779`, :phab:`T329132`, :phab:`T330488`, :phab:`T331315`)
+* No longer lazy load password cookies
+ (:phab:`T271858`, :phab:`T326779`, :phab:`T329132`, :phab:`T330488`, :phab:`T331315`)
8.0.3
@@ -401,7 +404,8 @@
* Provide Built Distribution with Pywikibot (:pep:`427`)
* Update `WRITE_ACTIONS` in used by :class:`api.Request<data.api.Request>`
-* Move :func:`get_closest_memento_url<data.memento.get_closest_memento_url>` from weblinkchecker script to memento module.
+* Move :func:`get_closest_memento_url<data.memento.get_closest_memento_url>` from weblinkchecker script to memento
+ module.
* Add :mod:`memento module<data.memento>` to fix memento_client package (:phab:`T185561`)
* L10N and i18n updates
* Fix Flow board topic continuation when iterating in reverse (:phab:`T138323`)
@@ -532,7 +536,8 @@
* Add support for Python 3.11
* Pywikibot supports PyPy 3 (:phab:`T101592`)
* A new method User.is_locked() was added to determine whether the user is currently locked globally (:phab:`T249392`)
-* A new method APISite.is_locked() was added to determine whether a given user or user id is locked globally (:phab:`T249392`)
+* A new method APISite.is_locked() was added to determine whether a given user or user id is locked globally
+ (:phab:`T249392`)
* APISite.get_globaluserinfo() method was added to retrieve globaluserinfo for any user or user id (:phab:`T163629`)
* APISite.globaluserinfo attribute may be deleted to force reload
* APISite.is_blocked() method has a force parameter to reload that info
@@ -544,8 +549,10 @@
* Upload: Retry upload if 'copyuploadbaddomain' API error occurs (:phab:`T294825`)
* Update invisible characters from unicodedata 14.0.0
* Add support for Wikimedia OCR engine with proofreadpage
-* Rewrite :func:`tools.itertools.intersect_generators` which makes it running up to 10'000 times faster. (:phab:`T85623`, :phab:`T293276`)
-* The cached output functionality from compat release was re-implemented (:phab:`T151727`, :phab:`T73646`, :phab:`T74942`, :phab:`T132135`, :phab:`T144698`, :phab:`T196039`, :phab:`T280466`)
+* Rewrite :func:`tools.itertools.intersect_generators` which makes it running up to 10'000 times faster.
+ (:phab:`T85623`, :phab:`T293276`)
+* The cached output functionality from compat release was re-implemented
+ (:phab:`T151727`, :phab:`T73646`, :phab:`T74942`, :phab:`T132135`, :phab:`T144698`, :phab:`T196039`, :phab:`T280466`)
* L10N updates
* Adjust groupsize within pagegenerators.PreloadingGenerator (:phab:`T291770`)
* New "maxlimit" property was added to APISite (:phab:`T291770`)
@@ -575,7 +582,8 @@
^^^^^^^^^^^^^^^^
* Support of Python 3.5.0 - 3.5.2 has been dropped (:phab:`T286867`)
-* generate_user_files.py, generate_user_files.py, shell.py and version.py were moved to pywikibot/scripts and must be used with pwb wrapper script
+* generate_user_files.py, generate_user_files.py, shell.py and version.py were moved to pywikibot/scripts and must be
+ used with pwb wrapper script
* *See also Code cleanups below*
Code cleanups
@@ -606,9 +614,11 @@
* Remove disable_ssl_certificate_validation kwargs in http functions in favour of verify parameter (:phab:`T265206`)
* Deprecated PYWIKIBOT2 environment variables were removed
* version.ParseError was removed in favour of exceptions.VersionParseError
-* specialbots.EditReplacement and specialbots.EditReplacementError were removed in favour of exceptions.EditReplacementError
+* specialbots.EditReplacement and specialbots.EditReplacementError were removed in favour of
+ exceptions.EditReplacementError
* site.PageInUse exception was removed in favour of exceptions.PageInUseError
-* page.UnicodeToAsciiHtml and page.unicode2html were removed in favour of tools.chars.string_to_ascii_html and tools.chars.string2html
+* page.UnicodeToAsciiHtml and page.unicode2html were removed in favour of tools.chars.string_to_ascii_html and
+ tools.chars.string2html
* interwiki_graph.GraphImpossible and login.OAuthImpossible exception were removed in favour of ImportError
* i18n.TranslationError was removed in favour of exceptions.TranslationError
* WikiaFamily was removed in favour of FandomFamily
@@ -863,7 +873,8 @@
* Deprecated Family.known_families dict was removed (:phab:`T89451`)
* Deprecated DataSite.get_* methods was removed
* Deprecated LogEntryFactory.logtypes classproperty was removed
-* Unused comms.threadedhttp module was removed; threadedhttp.HttpRequest was already replaced with requests.Response (:phab:`T265206`)
+* Unused comms.threadedhttp module was removed; threadedhttp.HttpRequest was already replaced with requests.Response
+ (:phab:`T265206`)
Other changes
^^^^^^^^^^^^^
@@ -885,7 +896,8 @@
* Add support for mniwiki and mniwiktionary (:phab:`T273467`, :phab:`T273462`)
* Don't use mime parameter as boolean in api.Request (:phab:`T274723`)
* textlib.removeDisabledPart is able to remove templates (:phab:`T274138`)
-* Create a SiteLink with __getitem__ method and implement lazy load (:phab:`T273386`, :phab:`T245809`, :phab:`T238471`, :phab:`T226157`)
+* Create a SiteLink with __getitem__ method and implement lazy load
+ (:phab:`T273386`, :phab:`T245809`, :phab:`T238471`, :phab:`T226157`)
* Fix date.formats['MonthName'] behaviour (:phab:`T273573`)
* Implement pagegenerators.handle_args() to process all options at once
* Add enabled_options, disabled_options to GeneratorFactory (:phab:`T271320`)
@@ -897,7 +909,8 @@
-----
*24 January 2021*
-* Use string instead of Path-like object with "open" function in UploadRobot for Python 3.5 compatibility (:phab:`T272345`)
+* Use string instead of Path-like object with "open" function in UploadRobot for Python 3.5 compatibility
+ (:phab:`T272345`)
* Add support for trwikivoyage (:phab:`T271263`)
* UI.input_list_choice() has been improved (:phab:`T272237`)
* Global handleArgs() function was removed in favour of handle_args
@@ -916,7 +929,8 @@
-----
*12 January 2021*
-* Add support for niawiki, bclwikt, diqwikt, niawikt (:phab:`T270416`, :phab:`T270282`, :phab:`T270278`, :phab:`T270412`)
+* Add support for niawiki, bclwikt, diqwikt, niawikt
+ (:phab:`T270416`, :phab:`T270282`, :phab:`T270278`, :phab:`T270412`)
* Delete page using pageid instead of title (:phab:`T57072`)
* version.getversion_svn_setuptools function was removed (:phab:`T270393`)
* Add support for "musical notation" data type to wikibase
@@ -948,7 +962,8 @@
* Allow using pywikibot as site-package without user-config.py (:phab:`T270474`)
* Python 3.10 is supported
* Fix AutoFamily scriptpath() call (:phab:`T270370`)
-* Add support for skrwiki, skrwiktionary, eowikivoyage, wawikisource, madwiki (:phab:`T268414`, :phab:`T268460`, :phab:`T269429`, :phab:`T269434`, :phab:`T269442`)
+* Add support for skrwiki, skrwiktionary, eowikivoyage, wawikisource, madwiki
+ (:phab:`T268414`, :phab:`T268460`, :phab:`T269429`, :phab:`T269434`, :phab:`T269442`)
* wikistats methods fetch, raw_cached, csv, xml has been removed
* PageRelatedError.getPage() has been removed in favour of PageRelatedError.page
* DataSite.get_item() method has been removed
@@ -956,7 +971,8 @@
* Property.getType() method has been removed
* Family.server_time() method was removed; it is still available from Site object (:phab:`T89451`)
* All HttpRequest parameters except of charset has been dropped (:phab:`T265206`)
-* A lot of methods and properties of HttpRequest are deprecared in favour of requests.Resonse attributes (:phab:`T265206`)
+* A lot of methods and properties of HttpRequest are deprecared in favour of requests.Resonse attributes
+ (:phab:`T265206`)
* Method and properties of HttpRequest are delegated to requests.Response object (:phab:`T265206`)
* comms.threadedhttp.HttpRequest.raw was replaced by HttpRequest.content property (:phab:`T265206`)
* Desupported version.getfileversion() has been removed
@@ -1027,7 +1043,6 @@
* comms.PywikibotCookieJar and comms.mode_check_decorator were deleted
* Remove deprecated tools classes Stringtypes and UnicodeType
* Remove deprecated tools function open_compressed and signature and UnicodeType class
-* Fix http_tests.LiveFakeUserAgentTestCase (:phab:`T265842`)
* HttpRequest properties were renamed to request.Response identifiers (:phab:`T265206`)
@@ -1043,7 +1058,8 @@
* Accept only valid names in generate_family_file.py (:phab:`T265328`, :phab:`T265353`)
* New plural.plural_rule() function returns a rule for a given language
* Replace deprecated urllib.request.URLopener with http.fetch (:phab:`T255575`)
-* OptionHandler/BaseBot options are accessable as OptionHandler.opt attributes or keyword item (see also :phab:`T264721`)
+* OptionHandler/BaseBot options are accessable as OptionHandler.opt attributes or keyword item
+ (see also :phab:`T264721`)
* pywikibot.setAction() function was removed
* A namedtuple is the result of textlib.extract_sections()
* Prevent circular imports in config2.py and http.py (:phab:`T264500`)
@@ -1209,7 +1225,6 @@
* Do not removeUselessSpaces inside source/syntaxhighlight tags (:phab:`T250469`)
* Restrict Pillow to 6.2.2+ (:phab:`T249911`)
* Fix PetScan generator language and project (:phab:`T249704`)
-* test_family has been removed (:phab:`T228375`, :phab:`T228300`)
* Bugfixes and improvements
* Localisation updates
@@ -1223,7 +1238,8 @@
* Usage of SkipPageError with BaseBot is desupported and may be removed
* Ignore InvalidTitle in textlib.replace_links() (:phab:`T122091`)
* Raise ServerError also if connection to PetScan timeouts
-* pagegenerators.py no longer supports 'oursql' or 'MySQLdb'. It now solely supports PyMySQL (:phab:`T243154`, :phab:`T89976`)
+* pagegenerators.py no longer supports 'oursql' or 'MySQLdb'. It now solely supports PyMySQL
+ (:phab:`T243154`, :phab:`T89976`)
* Disfunctional Family.versionnumber() method was removed
* Refactor login functionality (:phab:`T137805`, :phab:`T224712`, :phab:`T248767`, :phab:`T248768`, :phab:`T248945`)
* Bugfixes and improvements
@@ -1319,7 +1335,8 @@
* Deprecate the ability to login with a secondary sysop account (:phab:`T71283`)
* Enable global args with pwb.py wrapper script (:phab:`T216825`)
* Add a new ConfigParserBot class to set options from the scripts.ini file (:phab:`T223778`)
-* Check a user's rights rather than group memberships; 'sysopnames' will be deprecated (:phab:`T229293`, :phab:`T189126`, :phab:`T122705`, :phab:`T119335`, :phab:`T75545`)
+* Check a user's rights rather than group memberships; 'sysopnames' will be deprecated
+ (:phab:`T229293`, :phab:`T189126`, :phab:`T122705`, :phab:`T119335`, :phab:`T75545`)
* proofreadpage.py: fix footer detection (:phab:`T230301`)
* Add allowusertalk to the User.block() options (:phab:`T229288`)
* botirc module will be removed in next release (:phab:`T212632`)
@@ -1333,7 +1350,6 @@
*22 July 2019*
* Increase the throttling delay if maxlag >> retry-after (:phab:`T210606`)
-* deprecate test_family: Site('test', 'test'), use wikipedia_family: Site('test', 'wikipedia') instead (:phab:`T228375`, :phab:`T228300`)
* Add "user_agent_description" option in config.py
* APISite.fromDBName works for all known dbnames (:phab:`T225590`, 225723, 226960)
* remove the unimplemented "proxy" variable in config.py
@@ -1487,7 +1503,9 @@
* Fix usages of site.namespaces.NAMESPACE_NAME (:phab:`T201969`)
* pywikibot/textlib.py: Fix header regex to allow comments
* Use 'rvslots' when fetching revisions on MW 1.32+ (:phab:`T200955`)
-* Drop the '2' from PYWIKIBOT2_DIR, PYWIKIBOT2_DIR_PWB, and PYWIKIBOT2_NO_USER_CONFIG environment variables. The old names are now deprecated. The other PYWIKIBOT2_* variables which were used only for testing purposes have been renamed without deprecation. (:phab:`T184674`)
+* Drop the '2' from PYWIKIBOT2_DIR, PYWIKIBOT2_DIR_PWB, and PYWIKIBOT2_NO_USER_CONFIG environment variables. The old
+ names are now deprecated. The other PYWIKIBOT2_* variables which were used only for testing purposes have been renamed
+ without deprecation. (:phab:`T184674`)
* Introduce a timestamp in deprecated decorator (:phab:`T106121`)
* textlib.extract_sections: Remove footer from the last section (:phab:`T199751`)
* Don't let WikidataBot crash on save related errors (:phab:`T199642`)
@@ -1537,7 +1555,8 @@
* Pywikibot is following :pep:`396` versioning
* pagegenerators AllpagesPageGenerator, CombinedPageGenerator, UnconnectedPageGenerator are deprecated
* Some DayPageGenerator parameters has been renamed
-* unicodedata2, httpbin and Flask dependency was removed (:phab:`T102461`, :phab:`T108068`, :phab:`T178864`, :phab:`T193383`)
+* unicodedata2, httpbin and Flask dependency was removed
+ (:phab:`T102461`, :phab:`T108068`, :phab:`T178864`, :phab:`T193383`)
* New projects were provided
* Bugfixes and improvements
* Documentation updates
@@ -1651,7 +1670,6 @@
* Only force input_choise if self.always is given (:phab:`T161483`)
* Add colon when replacing category and file weblink (:phab:`T127745`)
* API Request: set uiprop only when ensuring 'userinfo' in meta (:phab:`T169202`)
-* Fix TestLazyLoginNotExistUsername test for Stewardwiki (:phab:`T169458`)
Improvements
^^^^^^^^^^^^
@@ -1830,7 +1848,6 @@
* pywikibot: Store ImportError in imported variable
* Use default tox pip install
* Add asteroids that are being used as locations
-* [bugfix] Fix test_translateMagicWords test
* Fix ID for Rhea
* [bugfix] pass User page object to NotEmailableError
* Allow pywikibot to run on Windows 10 as well
diff --git a/ROADMAP.rst b/ROADMAP.rst
index 0e61df9..bd33c1f 100644
--- a/ROADMAP.rst
+++ b/ROADMAP.rst
@@ -4,6 +4,10 @@
Improvements
^^^^^^^^^^^^
+* A copy button was added to the sphinx documentation.
+* Make :attr:`languages_by_size<family.WikimediaFamily.languages_by_size>` dynamic (:phab:`T78396`). The property is
+ only available for :class:`family.WikimediaFamily` families. The ``wikimedia_sites.py`` maintenance script was
+ removed.
* Add :func:`config.base_dir<config.get_base_dir>` to scripts search path with :mod:`pwb` wrapper (:phab:`T324287`)
* :meth:`pywikibot.WbTime.equal_instant` was added (:phab:`T325248`)
* ``revisions`` parameter of :class:`xmlreader.XmlDump` was introduced to specify parsing method
@@ -11,12 +15,14 @@
* Pass global -nolog argument into bot script from wrapper (:phab:`T328900`)
* Add :meth:`site.APISite.ratelimit()<pywikibot.site._apisite.APISite.ratelimit>` method
and :class:`tools.collections.RateLimit` NamedTuple (:phab:`T304808`)
-* L10N Updates
+* L10N and i18n updates
* Add :class:`pagegenerators.PagePilePageGenerator` (:phab:`T353086`)
Bugfixes
^^^^^^^^
+* Only delegate :mod:`site` methods to public :class:`family.Family` methods which have *code* as first parameter.
+* Use ``str`` instead of ``repr`` for several messages with :class:`family.Family` objects (:phab:`T356782`)
* Add ``hy`` to special languages in :class:`textlib.TimeStripper` (:phab:`T356175`)
* Pass login token when using ``action=login`` (:phab:`T309898`)
* Detect range blocks with :meth:`pywikibot.User.is_blocked` (:phab:`T301282`)
@@ -28,6 +34,8 @@
Breaking changes and code cleanups
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+* Raise ``TypeError`` instead of ``AttributeError`` in :meth:`Site.randompages()
+ <pywikibot.site._generators.GeneratorsMixin.randompages>` if *redirects* parameter is invalid.
* A RuntimeError will be raised if a :class:`family.Family` subclass has an ``__init__`` initializer method.
:meth:`family.Family.__post_init__` classmethod can be used instead.
* :class:`InteractiveReplace<bot_choice.InteractiveReplace>` was moved from :mod:`bot` to :mod:`bot_choice` module
diff --git a/scripts/CHANGELOG.rst b/scripts/CHANGELOG.rst
index d1b701c..4fd41fb 100644
--- a/scripts/CHANGELOG.rst
+++ b/scripts/CHANGELOG.rst
@@ -15,6 +15,11 @@
* provide category_redirect categories for all WM sites (:phab:`T348914`)
+interwiki
+^^^^^^^^^
+
+* Ignore :exc:`exceptions.InvalidTitleError` and :exc:`InvalidPageError` (:phab:`T357953`)
+
touch
^^^^^
--
To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/1005516?usp=email
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Change-Id: I4825991fa052dd8ea30700595383aefe6b4caaa1
Gerrit-Change-Number: 1005516
Gerrit-PatchSet: 2
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot
Gerrit-MessageType: merged
jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/1005124?usp=email )
Change subject: [bugfix] Ignore InvalidTitleError/InvalidPageError
......................................................................
[bugfix] Ignore InvalidTitleError/InvalidPageError
Bug: T357953
Change-Id: I199d46f59d7b0489b02650c0740e1879fa54ae2a
---
M scripts/interwiki.py
1 file changed, 36 insertions(+), 8 deletions(-)
Approvals:
jenkins-bot: Verified
Xqt: Looks good to me, approved
diff --git a/scripts/interwiki.py b/scripts/interwiki.py
index d0a4d4e..ead72d0 100755
--- a/scripts/interwiki.py
+++ b/scripts/interwiki.py
@@ -328,7 +328,7 @@
"""
#
-# (C) Pywikibot team, 2003-2023
+# (C) Pywikibot team, 2003-2024
#
# Distributed under the terms of the MIT license.
#
@@ -357,6 +357,7 @@
from pywikibot.exceptions import (
EditConflictError,
Error,
+ InvalidPageError,
InvalidTitleError,
LockedPageError,
NoCreateError,
@@ -1107,9 +1108,16 @@
def check_page(self, page, counter) -> None:
"""Check whether any iw links should be added to the todo list."""
- if not page.exists():
+ try:
+ ok = page.exists()
+ except InvalidPageError as e: # T357953
+ msg = str(e)
+ ok = False
+ else:
+ msg = f'{page} does not exist.'
+ if not ok:
self.conf.remove.append(str(page))
- self.conf.note(f'{page} does not exist. Skipping.')
+ self.conf.note(f'{msg} Skipping.')
if page == self.origin:
# The page we are working on is the page that does not
# exist. No use in doing any work on it in that case.
@@ -1311,7 +1319,12 @@
# Each value will be a list of pages.
new = defaultdict(list)
for page in self.done:
- if page.exists() and not page.isRedirectPage() \
+ try:
+ ok = page.exists()
+ except InvalidPageError: # T357953
+ continue
+
+ if ok and not page.isRedirectPage() \
and not page.isCategoryRedirect():
site = page.site
if site.family.interwiki_forward:
@@ -1985,10 +1998,15 @@
# Get the content of the assembled list in one blow
gen = site.preloadpages(pageGroup, templates=True, langlinks=True,
pageprops=True, quiet=False)
- for _ in gen:
- # we don't want to do anything with them now. The
- # page contents will be read via the Subject class.
- pass
+ while True:
+ # we don't want to do anything with them now.
+ # The page contents will be read via the Subject class.
+ try:
+ next(gen)
+ except StopIteration:
+ break
+ except InvalidTitleError: # T357953
+ pass
# Tell all of the subjects that the promised work is done
for subject in subjectGroup:
--
To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/1005124?usp=email
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Change-Id: I199d46f59d7b0489b02650c0740e1879fa54ae2a
Gerrit-Change-Number: 1005124
Gerrit-PatchSet: 3
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot
Gerrit-MessageType: merged
jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/996075?usp=email )
Change subject: [IMPR] raise TypeError in Site.randompages if redirects parameter is invalid
......................................................................
[IMPR] raise TypeError in Site.randompages if redirects parameter is invalid
Also add typing hints for GeneratorsMixin
Change-Id: I58ebcceda48a78013006dc6f202e3338736cc426
---
M pywikibot/site/_generators.py
1 file changed, 48 insertions(+), 8 deletions(-)
Approvals:
jenkins-bot: Verified
Xqt: Looks good to me, approved
diff --git a/pywikibot/site/_generators.py b/pywikibot/site/_generators.py
index 763b0d9..aae2e82 100644
--- a/pywikibot/site/_generators.py
+++ b/pywikibot/site/_generators.py
@@ -13,7 +13,7 @@
from typing import TYPE_CHECKING, Any
import pywikibot
-from pywikibot.backports import Generator, Iterable, batched
+from pywikibot.backports import Callable, Generator, Iterable, batched
from pywikibot.data import api
from pywikibot.exceptions import (
APIError,
@@ -34,13 +34,33 @@
if TYPE_CHECKING:
- from pywikibot.site._namespace import SingleNamespaceType
+ from data.api import ParamInfo, Request
+ from pywikibot.site._apisite import _RequestWrapperT
+ from pywikibot.site._namespace import NamespacesDict, SingleNamespaceType
+ from pywikibot.site._tokenwallet import TokenWallet
+ from pywikibot.tools import MediaWikiVersion
class GeneratorsMixin:
"""API generators mixin to MediaWiki site."""
+ if TYPE_CHECKING:
+ _generator: Callable[..., _RequestWrapperT]
+ _paraminfo: ParamInfo
+ _request: Callable[..., Request]
+ assert_valid_iter_params: Callable[..., None]
+ encoding: Callable[[], str]
+ get_property_names: Callable[..., list[str]]
+ has_right: Callable[[str], bool]
+ maxlimit: int
+ mw_version: MediaWikiVersion
+ namespaces: NamespacesDict
+ protection_types: Callable[[], set[str]]
+ sametitle: Callable[[str, str], bool]
+ tokens: TokenWallet
+ user: Callable[[], str | None]
+
def load_pages_from_pageids(
self,
pageids: str | Iterable[int | str],
@@ -1787,11 +1807,14 @@
) -> Iterable[pywikibot.Page]:
"""Iterate a number of random pages.
- .. seealso: :api:`Random`
-
Pages are listed in a fixed sequence, only the starting point is
random.
+ .. seealso: :api:`Random`
+ .. versionchanged:: 9.0
+ Raises ``TypeError`` instead of ``AssertionError`` if
+ *redirects* is invalid.
+
:param total: the maximum number of pages to iterate
:param namespaces: only iterate pages in these namespaces.
:param redirects: if True, include only redirect pages in
@@ -1802,10 +1825,12 @@
:raises KeyError: a namespace identifier was not resolved
:raises TypeError: a namespace identifier has an inappropriate
type such as NoneType or bool
- :raises AssertError: unsupported redirects parameter
+ :raises TypeError: unsupported redirects parameter
"""
mapping = {False: None, True: 'redirects', None: 'all'}
- assert redirects in mapping
+ if redirects not in mapping:
+ raise TypeError(f"Invalid type for 'redirects' parameter: "
+ f'{type(redirects).__name__}({redirects})')
redirects_ = mapping[redirects]
params = {}
if redirects_ is not None:
@@ -1970,9 +1995,13 @@
) -> Iterable[pywikibot.Page]:
"""Iterate Page objects retrieved from Special:{special_page}.
- .. seealso:: :api:`Querypage`
+ Generic function for all special pages supported by the site MW
+ API.
- Generic function for all special pages supported by the site MW API.
+ .. seealso:: :api:`Querypage`
+ .. versionchanged:: 9.0
+ Raises ``ValueError`` instead of ``AssertionError`` if
+ *special_page* is invalid.
:param special_page: Special page to query
:param total: number of pages to return
--
To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/996075?usp=email
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Change-Id: I58ebcceda48a78013006dc6f202e3338736cc426
Gerrit-Change-Number: 996075
Gerrit-PatchSet: 6
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot
Gerrit-MessageType: merged
jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/1004727?usp=email )
Change subject: [doc] Update environment variables in tests documentation
......................................................................
[doc] Update environment variables in tests documentation
Add description for PYWIKIBOT_TEST_NO_RC and PYWIKIBOT_TEST_QUIET
Change-Id: I49e61b30d4c83473a35e5344afd9cdc1e28b7792
---
M tests/README.rst
1 file changed, 25 insertions(+), 1 deletion(-)
Approvals:
Xqt: Looks good to me, approved
jenkins-bot: Verified
diff --git a/tests/README.rst b/tests/README.rst
index bff07ea..e6880d0 100644
--- a/tests/README.rst
+++ b/tests/README.rst
@@ -116,15 +116,28 @@
.. note:: test names must be given without subsequent ``_tests``.
+**PYWIKIBOT_TEST_NO_RC**
+ This environment variable disables recent changes tests and can be used to
+ speed up tests. GitHub actions enables this setting for that purpose::
+
+ PYWIKIBOT_TEST_NO_RC=1
+
**PYWIKIBOT_TEST_OAUTH**
This environment variable holds the Oauth token. It is set by
``oauth_tests-ci.yml`` CI config file and is solely used by
:mod:`tests.oauth_tests`. You can use it for your private tests. The
- environment variabke must contain consumer key and secret and access
+ environment variable must contain consumer key and secret and access
key and secret delimited by ``:`` as::
PYWIKIBOT_TEST_OAUTH=consumer_key:consumer_secret:access_key:access:secret
+**PYWIKIBOT_TEST_QUIET**
+ This environment variable can be set for quit mode. It prevents output by
+ test package, i.e. 'max_retries reduced from x to y'. It is used be the
+ :func:`tests.utils.execute` test runner. To enable it for other tests use::
+
+ PYWIKIBOT_TEST_QUIET=1
+
**PYWIKIBOT_TEST_RUNNING**
This environment variable skips tests instead of raising
:exc:`exceptions.MaxlagTimeoutError` when maximum retries attempted due to
--
To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/1004727?usp=email
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Change-Id: I49e61b30d4c83473a35e5344afd9cdc1e28b7792
Gerrit-Change-Number: 1004727
Gerrit-PatchSet: 2
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot
Gerrit-MessageType: merged
jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/1004659?usp=email )
Change subject: [tests] Show site if skipping tests due to missing rights
......................................................................
[tests] Show site if skipping tests due to missing rights
Change-Id: I2541bdc05e98c2fe8fb563d9486e4797ebcdef79
---
M tests/aspects.py
1 file changed, 12 insertions(+), 3 deletions(-)
Approvals:
jenkins-bot: Verified
Xqt: Looks good to me, approved
diff --git a/tests/aspects.py b/tests/aspects.py
index 4c3d1fb..4ae285e 100644
--- a/tests/aspects.py
+++ b/tests/aspects.py
@@ -675,9 +675,9 @@
for right in cls.rights.split(','):
if not site.has_right(right):
- raise unittest.SkipTest('User "{}" does not have required '
- 'user right "{}"'
- .format(site.user(), right))
+ raise unittest.SkipTest(
+ f'User "{site.user()}" does not have required user '
+ f'right "{right}" on site "{site}"')
class MetaTestCaseClass(type):
--
To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/1004659?usp=email
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Change-Id: I2541bdc05e98c2fe8fb563d9486e4797ebcdef79
Gerrit-Change-Number: 1004659
Gerrit-PatchSet: 2
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot
Gerrit-MessageType: merged
jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/1000779?usp=email )
Change subject: [IMPR] Only delegate site methods to public Family methods
......................................................................
[IMPR] Only delegate site methods to public Family methods
BaseSite.__getattr__() rewritten.
Undefined methods calls are delegated to the Family object.
No longer delegate private methods like _get_cr_templates or
_hostname. Also do not delegate static methods or other methods
which does not have 'code' as first parameter.
Change-Id: Ibca93fbbd14b3a901d816afca9db8cbc9ff4dba6
---
M pywikibot/site/_basesite.py
1 file changed, 39 insertions(+), 14 deletions(-)
Approvals:
jenkins-bot: Verified
Xqt: Looks good to me, approved
diff --git a/pywikibot/site/_basesite.py b/pywikibot/site/_basesite.py
index 813cf91..2eaa581 100644
--- a/pywikibot/site/_basesite.py
+++ b/pywikibot/site/_basesite.py
@@ -1,12 +1,13 @@
"""Objects with site methods independent of the communication interface."""
#
-# (C) Pywikibot team, 2008-2023
+# (C) Pywikibot team, 2008-2024
#
# Distributed under the terms of the MIT license.
#
from __future__ import annotations
import functools
+import inspect
import re
import threading
from warnings import warn
@@ -194,19 +195,27 @@
"""Return the username used for the site."""
return self._username
- def __getattr__(self, attr):
- """Delegate undefined methods calls to the Family object."""
- try:
- method = getattr(self.family, attr)
- if not callable(method):
- raise AttributeError
- f = functools.partial(method, self.code)
- if hasattr(method, '__doc__'):
- f.__doc__ = method.__doc__
- return f
- except AttributeError:
- raise AttributeError(f'{type(self).__name__} instance has no '
- f'attribute {attr!r}') from None
+ def __getattr__(self, name: str):
+ """Delegate undefined methods calls to the Family object.
+
+ .. versionchanged:: 9.0
+ Only delegate to public Family methods which have ``code`` as
+ first parameter.
+ """
+ if not name.startswith('_'):
+ obj = getattr(self.family, name, None)
+ if inspect.ismethod(obj):
+ params = inspect.signature(obj).parameters
+ if params:
+ parameter = next(iter(params))
+ if parameter == 'code':
+ method = functools.partial(obj, self.code)
+ if hasattr(obj, '__doc__'):
+ method.__doc__ = obj.__doc__
+ return method
+
+ raise AttributeError(f'{type(self).__name__} instance has no '
+ f'attribute {name!r}') from None
def __str__(self) -> str:
"""Return string representing this Site's name and code."""
--
To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/1000779?usp=email
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Change-Id: Ibca93fbbd14b3a901d816afca9db8cbc9ff4dba6
Gerrit-Change-Number: 1000779
Gerrit-PatchSet: 6
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot
Gerrit-MessageType: merged
Xqt has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/1003268?usp=email )
Change subject: make the output more verbose
......................................................................
make the output more verbose
Change-Id: Ia15030a3dd62e769b2129164431fcf72f289dc94
---
M .github/workflows/sysop_tests-ci.yml
1 file changed, 10 insertions(+), 1 deletion(-)
Approvals:
Xqt: Verified; Looks good to me, approved
diff --git a/.github/workflows/sysop_tests-ci.yml b/.github/workflows/sysop_tests-ci.yml
index 1de766e..03f082d 100644
--- a/.github/workflows/sysop_tests-ci.yml
+++ b/.github/workflows/sysop_tests-ci.yml
@@ -84,7 +84,7 @@
PYWIKIBOT_TEST_OAUTH: ${{ secrets[format('{0}', steps.token.outputs.uppercase)] }}
run: |
python pwb.py version
- pytest -a rights --cov=.;
+ pytest -v -a rights --cov=.;
- name: Show coverage statistics
run: |
--
To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/1003268?usp=email
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Change-Id: Ia15030a3dd62e769b2129164431fcf72f289dc94
Gerrit-Change-Number: 1003268
Gerrit-PatchSet: 1
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-MessageType: merged
jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/999922?usp=email )
Change subject: [doc] Only use major and minor version numbers within documentation
......................................................................
[doc] Only use major and minor version numbers within documentation
Change-Id: I4afa2e42918f9a6dda3a547be51c363f2f6a878d
---
M pywikibot/data/api/_paraminfo.py
M pywikibot/family.py
M pywikibot/page/_basepage.py
M pywikibot/page/_filepage.py
M pywikibot/site/_apisite.py
M pywikibot/site/_generators.py
M tests/aspects.py
M tests/utils.py
8 files changed, 23 insertions(+), 14 deletions(-)
Approvals:
jenkins-bot: Verified
Xqt: Looks good to me, approved
diff --git a/pywikibot/data/api/_paraminfo.py b/pywikibot/data/api/_paraminfo.py
index eb5b03d..b7cb846 100644
--- a/pywikibot/data/api/_paraminfo.py
+++ b/pywikibot/data/api/_paraminfo.py
@@ -1,6 +1,6 @@
"""Object representing API parameter information."""
#
-# (C) Pywikibot team, 2014-2023
+# (C) Pywikibot team, 2014-2024
#
# Distributed under the terms of the MIT license.
#
@@ -442,7 +442,7 @@
def paraminfo_keys(cls) -> frozenset[str]:
"""Return module types.
- .. deprecated:: 8.4.0
+ .. deprecated:: 8.4
"""
return frozenset(['modules'])
@@ -451,6 +451,6 @@
def preloaded_modules(self) -> frozenset[str] | set[str]:
"""Return set of preloaded modules.
- .. deprecated:: 8.4.0
+ .. deprecated:: 8.4
"""
return self._preloaded_modules
diff --git a/pywikibot/family.py b/pywikibot/family.py
index 4542749..cf48d4f 100644
--- a/pywikibot/family.py
+++ b/pywikibot/family.py
@@ -57,7 +57,7 @@
:attr:`obsolete` setter was removed.
.. versionchanged:: 8.3
Having an initializer method a ``FutureWarning`` will be given.
- .. versionchanged:: 9.0.0
+ .. versionchanged:: 9.0
raises RuntimeError if an initializer method was found;
:meth:`__post_init__` classmethod should be used instead.
"""
diff --git a/pywikibot/page/_basepage.py b/pywikibot/page/_basepage.py
index 52d5248..6af8970 100644
--- a/pywikibot/page/_basepage.py
+++ b/pywikibot/page/_basepage.py
@@ -726,7 +726,7 @@
def editTime(self) -> pywikibot.Timestamp:
"""Return timestamp of last revision to page.
- .. deprecated:: 8.0.0
+ .. deprecated:: 8.0
Use :attr:`latest_revision.timestamp<latest_revision>`
instead.
"""
diff --git a/pywikibot/page/_filepage.py b/pywikibot/page/_filepage.py
index ec71d41..94dc98f 100644
--- a/pywikibot/page/_filepage.py
+++ b/pywikibot/page/_filepage.py
@@ -6,7 +6,7 @@
* FileInfo: a structure holding imageinfo of latest revision of FilePage
"""
#
-# (C) Pywikibot team, 2008-2023
+# (C) Pywikibot team, 2008-2024
#
# Distributed under the terms of the MIT license.
#
@@ -263,7 +263,7 @@
def usingPages(self, **kwargs): # noqa: N802
"""Yield Pages on which the file is displayed.
- .. deprecated:: 7.4.0
+ .. deprecated:: 7.4
Use :meth:`using_pages` instead.
"""
return self.using_pages(**kwargs)
diff --git a/pywikibot/site/_apisite.py b/pywikibot/site/_apisite.py
index 8004a75..33af282 100644
--- a/pywikibot/site/_apisite.py
+++ b/pywikibot/site/_apisite.py
@@ -349,7 +349,7 @@
) -> None:
"""Log the user in if not already logged in.
- .. versionchanged:: 8.0.0
+ .. versionchanged:: 8.0
lazy load cookies when logging in. This was dropped in 8.0.4
.. versionchanged:: 8.0.4
the *cookie_only* parameter was added and cookies are loaded
diff --git a/pywikibot/site/_generators.py b/pywikibot/site/_generators.py
index 763b0d9..93ca190 100644
--- a/pywikibot/site/_generators.py
+++ b/pywikibot/site/_generators.py
@@ -526,9 +526,9 @@
- :meth:`pywikibot.Category.members`
- :meth:`pywikibot.Category.subcategories`
- .. versionchanged:: 4.0.0
+ .. versionchanged:: 4.0
parameters except *category* are keyword arguments only.
- .. versionchanged:: 8.0.0
+ .. versionchanged:: 8.0
raises TypeError instead of Error if no Category is specified
.. seealso:: :api:`Categorymembers`
diff --git a/tests/aspects.py b/tests/aspects.py
index 3a44491..efca054 100644
--- a/tests/aspects.py
+++ b/tests/aspects.py
@@ -312,7 +312,7 @@
for other than the current site e.g. for the related data or image
repositoy of the current site is ot possible.
- .. versionadded:: 8.0.0
+ .. versionadded:: 8.0
:param version_needed: The version needed
:param reason: A reason for skipping the test.
diff --git a/tests/utils.py b/tests/utils.py
index 8057aa6..9465863 100644
--- a/tests/utils.py
+++ b/tests/utils.py
@@ -1,6 +1,6 @@
"""Test utilities."""
#
-# (C) Pywikibot team, 2013-2023
+# (C) Pywikibot team, 2013-2024
#
# Distributed under the terms of the MIT license.
#
@@ -469,7 +469,7 @@
def execute(command: list[str], data_in=None, timeout=None):
"""Execute a command and capture outputs.
- .. versionchanged:: 8.2.0
+ .. versionchanged:: 8.2
*error* parameter was removed.
:param command: executable to run and arguments to use
@@ -518,7 +518,7 @@
overrides: dict[str, str] | None = None) -> dict[str, Any]:
"""Execute the pwb.py script and capture outputs.
- .. versionchanged:: 8.2.0
+ .. versionchanged:: 8.2
the *error* parameter was removed.
:param args: list of arguments for pwb.py
--
To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/999922?usp=email
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Change-Id: I4afa2e42918f9a6dda3a547be51c363f2f6a878d
Gerrit-Change-Number: 999922
Gerrit-PatchSet: 1
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot
Gerrit-MessageType: merged
jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/998295?usp=email )
Change subject: [bugfix] use str instead of repr for family
......................................................................
[bugfix] use str instead of repr for family
family is an object and not a string. Therefore do not use repr
for these variables when printing.
Bug detected with T356782
Bug: T356782
Change-Id: Ifde56185f8c095d6cdccfbe8063301a0ffb8f312
---
M pywikibot/scripts/version.py
M pywikibot/site/_apisite.py
2 files changed, 17 insertions(+), 2 deletions(-)
Approvals:
Xqt: Looks good to me, approved
jenkins-bot: Verified
diff --git a/pywikibot/scripts/version.py b/pywikibot/scripts/version.py
index b0e1156..1178caf 100755
--- a/pywikibot/scripts/version.py
+++ b/pywikibot/scripts/version.py
@@ -93,7 +93,7 @@
for family, usernames in pywikibot.config.usernames.items():
if not usernames:
continue
- pywikibot.info(f'Usernames for family {family!r}:')
+ pywikibot.info(f"Usernames for family '{family}':")
for lang, username in usernames.items():
pywikibot.info(f'\t{lang}: {username}')
diff --git a/pywikibot/site/_apisite.py b/pywikibot/site/_apisite.py
index b66ae01..8004a75 100644
--- a/pywikibot/site/_apisite.py
+++ b/pywikibot/site/_apisite.py
@@ -415,7 +415,7 @@
error_msg = ('No username has been defined in your '
'user config file: you have to add in this '
'file the following line:\n'
- 'usernames[{family!r}][{lang!r}]= {username!r}'
+ "usernames['{family}'][{lang!r}]= {username!r}"
.format(family=self.family,
lang=self.lang,
username=self.userinfo['name']))
--
To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/998295?usp=email
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Change-Id: Ifde56185f8c095d6cdccfbe8063301a0ffb8f312
Gerrit-Change-Number: 998295
Gerrit-PatchSet: 1
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot
Gerrit-MessageType: merged
jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/994696?usp=email )
Change subject: [cleanup] raise RuntimeError if a Family has an initializer
......................................................................
[cleanup] raise RuntimeError if a Family has an initializer
Family class are to be immutable. Therefore the __init__ initializer
is deprecated for more than 5 years. A FutureWarning was given since
Pywikibot 8.3. Now remove this deprecation warning and raise a
RuntimeError instead.
Change-Id: I664ef4ff906b38dbc96d6b01668bebda81db3934
---
M ROADMAP.rst
M docs/api_ref/family.rst
M pywikibot/family.py
3 files changed, 38 insertions(+), 24 deletions(-)
Approvals:
Xqt: Looks good to me, approved
jenkins-bot: Verified
diff --git a/ROADMAP.rst b/ROADMAP.rst
index 329b941..0e61df9 100644
--- a/ROADMAP.rst
+++ b/ROADMAP.rst
@@ -28,6 +28,8 @@
Breaking changes and code cleanups
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+* A RuntimeError will be raised if a :class:`family.Family` subclass has an ``__init__`` initializer method.
+ :meth:`family.Family.__post_init__` classmethod can be used instead.
* :class:`InteractiveReplace<bot_choice.InteractiveReplace>` was moved from :mod:`bot` to :mod:`bot_choice` module
* ``userinterfaces.transliteration.transliterator`` was renamed to :class:`Transliterator
<userinterfaces.transliteration.Transliterator>`
diff --git a/docs/api_ref/family.rst b/docs/api_ref/family.rst
index 3a61f22..9ba9afd 100644
--- a/docs/api_ref/family.rst
+++ b/docs/api_ref/family.rst
@@ -7,17 +7,6 @@
.. autoclass:: Family
- .. method:: __init__()
-
- Initializer
-
- .. deprecated:: 3.0.20180710
- Use :meth:`__post_init__` instead.
- .. versionchanged:: 8.3
- A FutureWarning is printed instead of a ``NotImplementedWarning``.
- The deprecation may be removed in a future release and a
- ``RuntimeError`` will be thrown instead.
-
.. method:: __post_init__()
:classmethod:
diff --git a/pywikibot/family.py b/pywikibot/family.py
index 5c8dd28..968fcef 100644
--- a/pywikibot/family.py
+++ b/pywikibot/family.py
@@ -43,11 +43,22 @@
"""Parent singleton class for all wiki families.
+ Families are immutable and initializer is unsupported. Any class
+ modification should go to :meth:`__post_init__` class method.
+
+ .. versionchanged:: 3.0
+ the family class is immutable. Having an ``__init__`` initializer
+ method a ``NotImplementedWarning`` will be given.
.. versionchanged:: 8.0
``alphabetic``, ``alphabetic_revised`` and ``fyinterwiki``
attributes where removed.
.. versionchanged:: 8.2
:attr:`obsolete` setter was removed.
+ .. versionchanged:: 8.3
+ Having an initializer method a ``FutureWarning`` will be given.
+ .. versionchanged:: 9.0.0
+ raises RuntimeError if an initializer method was found;
+ :meth:`__post_init__` classmethod should be used instead.
"""
def __new__(cls):
@@ -63,17 +74,12 @@
# don't use hasattr() here. consider only the class itself
if '__init__' in cls.__dict__:
- # Initializer deprecated. Families should be immutable and any
- # instance / class modification should go to allocator (__new__).
- cls.__init__ = deprecated(instead='__post_init__() classmethod',
- since='3.0.20180710')(cls.__init__)
+ raise RuntimeError(fill(
+ f'Family class {cls.__module__}.{cls.__name__} cannot be'
+ ' instantiated; use __post_init__() classmethod to modify'
+ ' your family class. Refer the documentation.', width=66))
- # Invoke initializer immediately and make initializer no-op.
- # This is to avoid repeated initializer invocation on repeated
- # invocations of the metaclass's __call__.
- cls.instance.__init__()
- cls.__init__ = lambda self: None # no-op
- elif '__post_init__' not in cls.__dict__:
+ if '__post_init__' not in cls.__dict__:
pass
elif inspect.ismethod(cls.__post_init__): # classmethod check
cls.__post_init__()
@@ -87,11 +93,14 @@
@classproperty
def instance(cls):
- """Get the singleton instance."""
- # This is a placeholder to invoke allocator before it's allocated.
- # Allocator will override this classproperty.
+ """Get the singleton instance.
+
+ This is a placeholder to invoke allocator before it's allocated.
+ Allocator will override this classproperty.
+ """
return cls()
+ #: The family name
name: str | None = None
#: Not open for edits; stewards can still edit.
--
To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/994696?usp=email
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Change-Id: I664ef4ff906b38dbc96d6b01668bebda81db3934
Gerrit-Change-Number: 994696
Gerrit-PatchSet: 6
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot
Gerrit-MessageType: merged
Xqt has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/997899?usp=email )
Change subject: [doc] update ROADMAP.rst
......................................................................
[doc] update ROADMAP.rst
Change-Id: Id688c740fc11780cf39b1dc11c35e49ca152d5bc
---
M ROADMAP.rst
1 file changed, 10 insertions(+), 0 deletions(-)
Approvals:
Xqt: Verified; Looks good to me, approved
diff --git a/ROADMAP.rst b/ROADMAP.rst
index 9b450fd..329b941 100644
--- a/ROADMAP.rst
+++ b/ROADMAP.rst
@@ -28,6 +28,7 @@
Breaking changes and code cleanups
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+* :class:`InteractiveReplace<bot_choice.InteractiveReplace>` was moved from :mod:`bot` to :mod:`bot_choice` module
* ``userinterfaces.transliteration.transliterator`` was renamed to :class:`Transliterator
<userinterfaces.transliteration.Transliterator>`
* ``pywikibot.BaseSite`` and ``pywikibotAPISite`` were dropped. :func:`pywikibot.Site` has to be used to create a
--
To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/997899?usp=email
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Change-Id: Id688c740fc11780cf39b1dc11c35e49ca152d5bc
Gerrit-Change-Number: 997899
Gerrit-PatchSet: 1
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-MessageType: merged
jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/995217?usp=email )
Change subject: [IMPR] Move InteractiveReplace from bot to bot_choice module
......................................................................
[IMPR] Move InteractiveReplace from bot to bot_choice module
InteractiveReplace is a handler for bot_choice.Choice, LinkChoice and
AlwaysChoice. Therefore move it to the bot_choice module.
InteractiveReplace is imported in bot module like other bot_choice
classes; there is no change in importing necessary (yet).
Change-Id: I63ab7e9796a238a53411eab229d1d801eea7121f
---
M pywikibot/bot.py
M pywikibot/bot_choice.py
M pywikibot/specialbots/_unlink.py
3 files changed, 245 insertions(+), 206 deletions(-)
Approvals:
Matěj Suchánek: Looks good to me, approved
jenkins-bot: Verified
diff --git a/pywikibot/bot.py b/pywikibot/bot.py
index 3121ebc..cc583e2 100644
--- a/pywikibot/bot.py
+++ b/pywikibot/bot.py
@@ -57,7 +57,7 @@
``put_current`` is used.
"""
#
-# (C) Pywikibot team, 2008-2023
+# (C) Pywikibot team, 2008-2024
#
# Distributed under the terms of the MIT license.
#
@@ -113,7 +113,7 @@
import pywikibot
from pywikibot import config, daemonize, i18n, version
-from pywikibot.backports import Callable, Dict, Iterable, Mapping, Sequence
+from pywikibot.backports import Callable, Dict, Iterable, Sequence
from pywikibot.bot_choice import (
AlwaysChoice,
Choice,
@@ -121,6 +121,7 @@
ContextOption,
HighlightContextOption,
IntegerOption,
+ InteractiveReplace,
LinkChoice,
ListOption,
MultipleChoiceList,
@@ -172,9 +173,6 @@
if TYPE_CHECKING:
- from typing_extensions import Literal
-
- from pywikibot.page import BaseLink, Link, Page
from pywikibot.site import BaseSite
AnswerType = Iterable[tuple[str, str] | Option] | Option
@@ -641,196 +639,6 @@
force=force)
-class InteractiveReplace:
-
- """
- A callback class for textlib's replace_links.
-
- It shows various options which can be switched on and off:
- * allow_skip_link = True (skip the current link)
- * allow_unlink = True (unlink)
- * allow_replace = False (just replace target, keep section and label)
- * allow_replace_section = False (replace target and section, keep label)
- * allow_replace_label = False (replace target and label, keep section)
- * allow_replace_all = False (replace target, section and label)
- (The boolean values are the default values)
-
- It has also a ``context`` attribute which must be a non-negative
- integer. If it is greater 0 it shows that many characters before and
- after the link in question. The ``context_delta`` attribute can be
- defined too and adds an option to increase ``context`` by the given
- amount each time the option is selected.
-
- Additional choices can be defined using the 'additional_choices' and will
- be amended to the choices defined by this class. This list is mutable and
- the Choice instance returned and created by this class are too.
- """
-
- def __init__(self,
- old_link: Link | Page,
- new_link: Link | Page | Literal[False],
- default: str | None = None,
- automatic_quit: bool = True) -> None:
- """
- Initializer.
-
- :param old_link: The old link which is searched. The label and section
- are ignored.
- :param new_link: The new link with which it should be replaced.
- Depending on the replacement mode it'll use this link's label and
- section. If False it'll unlink all and the attributes beginning
- with allow_replace are ignored.
- :param default: The default answer as the shortcut
- :param automatic_quit: Add an option to quit and raise a
- QuitKeyboardException.
- """
- if isinstance(old_link, pywikibot.Page):
- self._old = old_link._link
- else:
- self._old = old_link
- if isinstance(new_link, pywikibot.Page):
- self._new: BaseLink | Literal[False] = new_link._link
- else:
- self._new = new_link
- self._default = default
- self._quit = automatic_quit
-
- self._current_match: tuple[
- Link | Page,
- str,
- Mapping[str, str],
- tuple[int, int]
- ] | None = None
-
- self.context = 30
- self.context_delta = 0
- self.allow_skip_link = True
- self.allow_unlink = True
- self.allow_replace = False
- self.allow_replace_section = False
- self.allow_replace_label = False
- self.allow_replace_all = False
- # Use list to preserve order
- self._own_choices: list[tuple[str, StandardOption]] = [
- ('skip_link', StaticChoice('Do not change', 'n', None)),
- ('unlink', StaticChoice('Unlink', 'u', False)),
- ]
- if self._new:
- self._own_choices += [
- ('replace', LinkChoice('Change link target', 't', self,
- False, False)),
- ('replace_section', LinkChoice(
- 'Change link target and section', 's', self, True, False)),
- ('replace_label', LinkChoice('Change link target and label',
- 'l', self, False, True)),
- ('replace_all', LinkChoice('Change complete link', 'c', self,
- True, True)),
- ]
-
- self.additional_choices: list[StandardOption] = []
-
- def handle_answer(self, choice: str) -> Any:
- """Return the result for replace_links."""
- for c in self.choices:
- if isinstance(c, Choice) and c.shortcut == choice:
- return c.handle()
-
- raise ValueError(f'Invalid choice "{choice}"')
-
- def __call__(self, link: Link | Page,
- text: str, groups: Mapping[str, str],
- rng: tuple[int, int]) -> Any:
- """Ask user how the selected link should be replaced."""
- if self._old == link:
- self._current_match = (link, text, groups, rng)
- while True:
- try:
- answer = self.handle_link()
- except UnhandledAnswer as e:
- if e.stop:
- raise
- else:
- break
- self._current_match = None # don't reset in case of an exception
- return answer
- return None
-
- @property
- def choices(self) -> tuple[StandardOption, ...]:
- """Return the tuple of choices."""
- choices = []
- for name, choice in self._own_choices:
- if getattr(self, 'allow_' + name):
- choices += [choice]
- if self.context_delta > 0:
- choices += [HighlightContextOption(
- 'more context', 'm', self.current_text, self.context,
- self.context_delta, *self.current_range)]
- choices += self.additional_choices
- return tuple(choices)
-
- def handle_link(self) -> Any:
- """Handle the currently given replacement."""
- choices = self.choices
- for c in choices:
- if isinstance(c, AlwaysChoice) and c.handle_link():
- return c.answer
-
- question = 'Should the link '
- if self.context > 0:
- rng = self.current_range
- text = self.current_text
- # at the beginning of the link, start red color.
- # at the end of the link, reset the color to default
- pywikibot.info(text[max(0, rng[0] - self.context): rng[0]]
- + f'<<lightred>>{text[rng[0]:rng[1]]}<<default>>'
- + text[rng[1]: rng[1] + self.context])
- else:
- question += (
- f'<<lightred>>{self._old.canonical_title()}<<default>> ')
-
- if self._new is False:
- question += 'be unlinked?'
- else:
- question += 'target to <<lightpurple>>{}<<default>>?'.format(
- self._new.canonical_title())
-
- choice = pywikibot.input_choice(question, choices,
- default=self._default,
- automatic_quit=self._quit)
-
- assert isinstance(choice, str)
- return self.handle_answer(choice)
-
- @property
- def current_link(self) -> Link | Page:
- """Get the current link when it's handling one currently."""
- if self._current_match is None:
- raise ValueError('No current link')
- return self._current_match[0]
-
- @property
- def current_text(self) -> str:
- """Get the current text when it's handling one currently."""
- if self._current_match is None:
- raise ValueError('No current text')
- return self._current_match[1]
-
- @property
- def current_groups(self) -> Mapping[str, str]:
- """Get the current groups when it's handling one currently."""
- if self._current_match is None:
- raise ValueError('No current groups')
- return self._current_match[2]
-
- @property
- def current_range(self) -> tuple[int, int]:
- """Get the current range when it's handling one currently."""
- if self._current_match is None:
- raise ValueError('No current range')
- return self._current_match[3]
-
-
# Command line parsing and help
def calledModuleName() -> str:
"""Return the name of the module calling this function.
diff --git a/pywikibot/bot_choice.py b/pywikibot/bot_choice.py
index 5ece69f..035acd8 100644
--- a/pywikibot/bot_choice.py
+++ b/pywikibot/bot_choice.py
@@ -9,10 +9,39 @@
import re
from abc import ABC, abstractmethod
from textwrap import fill
-from typing import Any
+from typing import TYPE_CHECKING, Any
import pywikibot
-from pywikibot.backports import Iterable, Sequence
+from pywikibot.backports import Iterable, Mapping, Sequence
+
+
+__all__ = (
+ 'AlwaysChoice',
+ 'Choice',
+ 'ChoiceException',
+ 'ContextOption',
+ 'HighlightContextOption',
+ 'IntegerOption',
+ 'InteractiveReplace',
+ 'LinkChoice',
+ 'ListOption',
+ 'MultipleChoiceList',
+ 'NestedOption',
+ 'Option',
+ 'OutputProxyOption',
+ 'QuitKeyboardInterrupt',
+ 'ShowingListOption',
+ 'ShowingMultipleChoiceList',
+ 'StandardOption',
+ 'StaticChoice',
+ 'UnhandledAnswer',
+)
+
+
+if TYPE_CHECKING:
+ from typing_extensions import Literal
+
+ from pywikibot.page import BaseLink, Link, Page
class Option(ABC):
@@ -259,14 +288,14 @@
self,
option: str,
shortcut: str,
- replacer: pywikibot.bot.InteractiveReplace | None
+ replacer: InteractiveReplace | None
) -> None:
"""Initializer."""
super().__init__(option, shortcut)
self._replacer = replacer
@property
- def replacer(self) -> pywikibot.bot.InteractiveReplace | None:
+ def replacer(self) -> InteractiveReplace | None:
"""The replacer."""
return self._replacer
@@ -302,7 +331,7 @@
self,
option: str,
shortcut: str,
- replacer: pywikibot.bot.InteractiveReplace | None,
+ replacer: InteractiveReplace | None,
replace_section: bool,
replace_label: bool
) -> None:
@@ -345,7 +374,7 @@
"""Add an option to always apply the default."""
- def __init__(self, replacer: pywikibot.bot.InteractiveReplace | None,
+ def __init__(self, replacer: InteractiveReplace | None,
option: str = 'always', shortcut: str = 'a') -> None:
"""Initializer."""
super().__init__(option, shortcut, replacer)
@@ -601,3 +630,191 @@
def __init__(self) -> None:
"""Constructor using the 'quit' ('q') in input_choice."""
super().__init__('quit', 'q')
+
+
+class InteractiveReplace:
+
+ """A callback class for textlib's replace_links.
+
+ It shows various options which can be switched on and off:
+ * allow_skip_link = True (skip the current link)
+ * allow_unlink = True (unlink)
+ * allow_replace = False (just replace target, keep section and label)
+ * allow_replace_section = False (replace target and section, keep label)
+ * allow_replace_label = False (replace target and label, keep section)
+ * allow_replace_all = False (replace target, section and label)
+ (The boolean values are the default values)
+
+ It has also a ``context`` attribute which must be a non-negative
+ integer. If it is greater 0 it shows that many characters before and
+ after the link in question. The ``context_delta`` attribute can be
+ defined too and adds an option to increase ``context`` by the given
+ amount each time the option is selected.
+
+ Additional choices can be defined using the 'additional_choices' and will
+ be amended to the choices defined by this class. This list is mutable and
+ the Choice instance returned and created by this class are too.
+ """
+
+ def __init__(self,
+ old_link: Link | Page,
+ new_link: Link | Page | Literal[False],
+ default: str | None = None,
+ automatic_quit: bool = True) -> None:
+ """Initializer.
+
+ :param old_link: The old link which is searched. The label and section
+ are ignored.
+ :param new_link: The new link with which it should be replaced.
+ Depending on the replacement mode it'll use this link's label and
+ section. If False it'll unlink all and the attributes beginning
+ with allow_replace are ignored.
+ :param default: The default answer as the shortcut
+ :param automatic_quit: Add an option to quit and raise a
+ QuitKeyboardException.
+ """
+ if isinstance(old_link, pywikibot.Page):
+ self._old = old_link._link
+ else:
+ self._old = old_link
+ if isinstance(new_link, pywikibot.Page):
+ self._new: BaseLink | Literal[False] = new_link._link
+ else:
+ self._new = new_link
+ self._default = default
+ self._quit = automatic_quit
+
+ self._current_match: tuple[
+ Link | Page,
+ str,
+ Mapping[str, str],
+ tuple[int, int]
+ ] | None = None
+
+ self.context = 30
+ self.context_delta = 0
+ self.allow_skip_link = True
+ self.allow_unlink = True
+ self.allow_replace = False
+ self.allow_replace_section = False
+ self.allow_replace_label = False
+ self.allow_replace_all = False
+ # Use list to preserve order
+ self._own_choices: list[tuple[str, StandardOption]] = [
+ ('skip_link', StaticChoice('Do not change', 'n', None)),
+ ('unlink', StaticChoice('Unlink', 'u', False)),
+ ]
+ if self._new:
+ self._own_choices += [
+ ('replace', LinkChoice('Change link target', 't', self,
+ False, False)),
+ ('replace_section', LinkChoice(
+ 'Change link target and section', 's', self, True, False)),
+ ('replace_label', LinkChoice('Change link target and label',
+ 'l', self, False, True)),
+ ('replace_all', LinkChoice('Change complete link', 'c', self,
+ True, True)),
+ ]
+
+ self.additional_choices: list[StandardOption] = []
+
+ def handle_answer(self, choice: str) -> Any:
+ """Return the result for replace_links."""
+ for c in self.choices:
+ if isinstance(c, Choice) and c.shortcut == choice:
+ return c.handle()
+
+ raise ValueError(f'Invalid choice "{choice}"')
+
+ def __call__(self, link: Link | Page,
+ text: str, groups: Mapping[str, str],
+ rng: tuple[int, int]) -> Any:
+ """Ask user how the selected link should be replaced."""
+ if self._old == link:
+ self._current_match = (link, text, groups, rng)
+ while True:
+ try:
+ answer = self.handle_link()
+ except UnhandledAnswer as e:
+ if e.stop:
+ raise
+ else:
+ break
+ self._current_match = None # don't reset in case of an exception
+ return answer
+ return None
+
+ @property
+ def choices(self) -> tuple[StandardOption, ...]:
+ """Return the tuple of choices."""
+ choices = []
+ for name, choice in self._own_choices:
+ if getattr(self, 'allow_' + name):
+ choices += [choice]
+ if self.context_delta > 0:
+ choices += [HighlightContextOption(
+ 'more context', 'm', self.current_text, self.context,
+ self.context_delta, *self.current_range)]
+ choices += self.additional_choices
+ return tuple(choices)
+
+ def handle_link(self) -> Any:
+ """Handle the currently given replacement."""
+ choices = self.choices
+ for c in choices:
+ if isinstance(c, AlwaysChoice) and c.handle_link():
+ return c.answer
+
+ question = 'Should the link '
+ if self.context > 0:
+ rng = self.current_range
+ text = self.current_text
+ # at the beginning of the link, start red color.
+ # at the end of the link, reset the color to default
+ pywikibot.info(text[max(0, rng[0] - self.context): rng[0]]
+ + f'<<lightred>>{text[rng[0]:rng[1]]}<<default>>'
+ + text[rng[1]: rng[1] + self.context])
+ else:
+ question += (
+ f'<<lightred>>{self._old.canonical_title()}<<default>> ')
+
+ if self._new is False:
+ question += 'be unlinked?'
+ else:
+ question += 'target to <<lightpurple>>{}<<default>>?'.format(
+ self._new.canonical_title())
+
+ choice = pywikibot.input_choice(question, choices,
+ default=self._default,
+ automatic_quit=self._quit)
+
+ assert isinstance(choice, str)
+ return self.handle_answer(choice)
+
+ @property
+ def current_link(self) -> Link | Page:
+ """Get the current link when it's handling one currently."""
+ if self._current_match is None:
+ raise ValueError('No current link')
+ return self._current_match[0]
+
+ @property
+ def current_text(self) -> str:
+ """Get the current text when it's handling one currently."""
+ if self._current_match is None:
+ raise ValueError('No current text')
+ return self._current_match[1]
+
+ @property
+ def current_groups(self) -> Mapping[str, str]:
+ """Get the current groups when it's handling one currently."""
+ if self._current_match is None:
+ raise ValueError('No current groups')
+ return self._current_match[2]
+
+ @property
+ def current_range(self) -> tuple[int, int]:
+ """Get the current range when it's handling one currently."""
+ if self._current_match is None:
+ raise ValueError('No current range')
+ return self._current_match[3]
diff --git a/pywikibot/specialbots/_unlink.py b/pywikibot/specialbots/_unlink.py
index 96c6566..dc65dfa 100644
--- a/pywikibot/specialbots/_unlink.py
+++ b/pywikibot/specialbots/_unlink.py
@@ -3,20 +3,19 @@
Do not import classes directly from here but from specialbots.
"""
#
-# (C) Pywikibot team, 2003-2022
+# (C) Pywikibot team, 2003-2024
#
# Distributed under the terms of the MIT license.
#
from __future__ import annotations
-from pywikibot.bot import (
+from pywikibot.bot import AutomaticTWSummaryBot, ExistingPageBot
+from pywikibot.bot_choice import (
AlwaysChoice,
- AutomaticTWSummaryBot,
ChoiceException,
- ExistingPageBot,
InteractiveReplace,
+ UnhandledAnswer,
)
-from pywikibot.bot_choice import UnhandledAnswer
from pywikibot.editor import TextEditor
from pywikibot.textlib import replace_links
--
To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/995217?usp=email
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Change-Id: I63ab7e9796a238a53411eab229d1d801eea7121f
Gerrit-Change-Number: 995217
Gerrit-PatchSet: 6
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: Matěj Suchánek <matejsuchanek97(a)gmail.com>
Gerrit-Reviewer: jenkins-bot
Gerrit-MessageType: merged
jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/996127?usp=email )
Change subject: [IMPR] simplify Siteinfo._get_siteinfo
......................................................................
[IMPR] simplify Siteinfo._get_siteinfo
Decrease nested flow statements after exception cause is either
raised or a return statement leaves it.
Change-Id: I3395dc9b9aa11e26683f98e0ee0aa1fe4bcda158
---
M pywikibot/site/_siteinfo.py
1 file changed, 33 insertions(+), 20 deletions(-)
Approvals:
Xqt: Looks good to me, approved
jenkins-bot: Verified
diff --git a/pywikibot/site/_siteinfo.py b/pywikibot/site/_siteinfo.py
index cf7251a..3283d54 100644
--- a/pywikibot/site/_siteinfo.py
+++ b/pywikibot/site/_siteinfo.py
@@ -137,33 +137,34 @@
except APIError as e:
if e.code == 'siunknown_siprop':
if len(props) == 1:
- pywikibot.log(
- f"Unable to get siprop '{props[0]}'")
+ pywikibot.log(f"Unable to get siprop '{props[0]}'")
return {props[0]: (EMPTY_DEFAULT, False)}
+
pywikibot.log('Unable to get siteinfo, because at least '
- "one property is unknown: '{}'".format(
- "', '".join(props)))
+ "one property is unknown: '{}'"
+ .format("', '".join(props)))
results = {}
for prop in props:
results.update(self._get_siteinfo(prop, expiry))
return results
raise
- else:
- result = {}
- if invalid_properties:
- for prop in invalid_properties:
- result[prop] = (EMPTY_DEFAULT, False)
- pywikibot.log("Unable to get siprop(s) '{}'".format(
- "', '".join(invalid_properties)))
- if 'query' in data:
- # If the request is a CachedRequest, use the _cachetime attr.
- cache_time = getattr(
- request, '_cachetime', None) or datetime.datetime.utcnow()
- for prop in props:
- if prop in data['query']:
- self._post_process(prop, data['query'][prop])
- result[prop] = (data['query'][prop], cache_time)
- return result
+
+ result = {}
+ if invalid_properties:
+ for prop in invalid_properties:
+ result[prop] = (EMPTY_DEFAULT, False)
+ pywikibot.log("Unable to get siprop(s) '{}'"
+ .format("', '".join(invalid_properties)))
+
+ if 'query' in data:
+ # If the request is a CachedRequest, use the _cachetime attr.
+ cache_time = getattr(
+ request, '_cachetime', None) or datetime.datetime.utcnow()
+ for prop in props:
+ if prop in data['query']:
+ self._post_process(prop, data['query'][prop])
+ result[prop] = (data['query'][prop], cache_time)
+ return result
@staticmethod
def _is_expired(cache_date, expire):
--
To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/996127?usp=email
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Change-Id: I3395dc9b9aa11e26683f98e0ee0aa1fe4bcda158
Gerrit-Change-Number: 996127
Gerrit-PatchSet: 1
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot
Gerrit-MessageType: merged
jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/994748?usp=email )
Change subject: [tests] test that https://wikimediafoundation.org/$1 is not a wiki
......................................................................
[tests] test that https://wikimediafoundation.org/$1 is not a wiki
Bug: T356013
Change-Id: I0307691ee34dad5e1c2b7a128ed6b759658fb70a
---
M tests/site_detect_tests.py
1 file changed, 13 insertions(+), 0 deletions(-)
Approvals:
DannyS712: Looks good to me, approved
Xqt: Looks good to me, approved
jenkins-bot: Verified
diff --git a/tests/site_detect_tests.py b/tests/site_detect_tests.py
index 1cafd2c..0f703ff 100755
--- a/tests/site_detect_tests.py
+++ b/tests/site_detect_tests.py
@@ -91,6 +91,9 @@
'http://www.merriam-webster.com/'
'cgi-bin/dictionary?book=Dictionary&va=$1',
'http://arxiv.org/abs/$1',
+ # looks like a MW site but isn't, api is redirected to
+ # https://foundation.wikimedia.org
+ 'https://wikimediafoundation.org/$1',
)
failing_sites = [
--
To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/994748?usp=email
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Change-Id: I0307691ee34dad5e1c2b7a128ed6b759658fb70a
Gerrit-Change-Number: 994748
Gerrit-PatchSet: 3
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: DannyS712 <dannys712.wiki(a)gmail.com>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot
Gerrit-MessageType: merged