jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/1005760?usp=email )
Change subject: [bugfix] use require_modules instead of has_module in proofreadpage_tests.py
......................................................................
[bugfix] use require_modules instead of has_module in proofreadpage_tests.py
has_module() does not find 'bs4' but 'beautifulsoup4'.
Use aspects.require_modules decorator instead. Use it on
setUpClass to skip all tests at once; decoration the class
would skip every test method.
Use ModuleNotFoundError instead if ImportError in require_modules
which is raised when a module cannot be found.
This exception was introduced with Python 3.6
Change-Id: I76e17adb2476b02b17ee5e514889fe1b497a1588
---
M tests/aspects.py
M tests/proofreadpage_tests.py
2 files changed, 21 insertions(+), 9 deletions(-)
Approvals:
Xqt: Looks good to me, approved
jenkins-bot: Verified
diff --git a/tests/aspects.py b/tests/aspects.py
index 4ae285e..efd5616 100644
--- a/tests/aspects.py
+++ b/tests/aspects.py
@@ -290,7 +290,7 @@
for required_module in required_modules:
try:
__import__(required_module, globals(), locals(), [], 0)
- except ImportError:
+ except ModuleNotFoundError:
missing += [required_module]
if not missing:
return obj
diff --git a/tests/proofreadpage_tests.py b/tests/proofreadpage_tests.py
index 09a9677..0e2c525 100755
--- a/tests/proofreadpage_tests.py
+++ b/tests/proofreadpage_tests.py
@@ -1,7 +1,7 @@
#!/usr/bin/env python3
"""Tests for the proofreadpage module."""
#
-# (C) Pywikibot team, 2015-2023
+# (C) Pywikibot team, 2015-2024
#
# Distributed under the terms of the MIT license.
#
@@ -21,8 +21,6 @@
ProofreadPage,
TagAttr,
)
-from pywikibot.tools import has_module
-from tests import unittest_print
from tests.aspects import TestCase, require_modules
from tests.basepage import (
BasePageLoadRevisionsCachingTestBase,
@@ -439,13 +437,9 @@
"""Run tests which needs bs4 beeing installed."""
@classmethod
+ @require_modules('bs4')
def setUpClass(cls):
"""Check whether bs4 module is installed already."""
- if not has_module('bs4'):
- unittest_print(
- f'all tests ({__name__}.{cls.__name__})\n{cls.__doc__}.. ',
- end='\n')
- cls.skipTest(cls, 'bs4 not installed')
super().setUpClass()
--
To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/1005760?usp=email
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Change-Id: I76e17adb2476b02b17ee5e514889fe1b497a1588
Gerrit-Change-Number: 1005760
Gerrit-PatchSet: 3
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot
Gerrit-MessageType: merged
Xqt has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/1005719?usp=email )
Change subject: [tests] install bs4 for pywikibot CI tests
......................................................................
[tests] install bs4 for pywikibot CI tests
Most proofreadpage_tests were not executed due to missing bs4.
Install it for tests and code coverage with pywikibot CI action.
Change-Id: I6dd70a6e2538b19ff64d8a22a64c5c818f496f94
---
M .github/workflows/pywikibot-ci.yml
1 file changed, 13 insertions(+), 0 deletions(-)
Approvals:
Xqt: Verified; Looks good to me, approved
diff --git a/.github/workflows/pywikibot-ci.yml b/.github/workflows/pywikibot-ci.yml
index ea447d1..8c91dfb 100644
--- a/.github/workflows/pywikibot-ci.yml
+++ b/.github/workflows/pywikibot-ci.yml
@@ -92,6 +92,7 @@
pip --version
if [ -f dev-requirements.txt ]; then pip install -r dev-requirements.txt; fi
if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
+ pip install "beautifulsoup4>=4.7.1"
pip install wikitextparser
- name: Generate family files
--
To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/1005719?usp=email
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Change-Id: I6dd70a6e2538b19ff64d8a22a64c5c818f496f94
Gerrit-Change-Number: 1005719
Gerrit-PatchSet: 1
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-MessageType: merged
jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/1005516?usp=email )
Change subject: [doc] Update ROADMAP.rst and CHANGELOG.rst
......................................................................
[doc] Update ROADMAP.rst and CHANGELOG.rst
Also wrap lines in HISTORY.rst and remove tests changes there.
Change-Id: I4825991fa052dd8ea30700595383aefe6b4caaa1
---
M HISTORY.rst
M ROADMAP.rst
M scripts/CHANGELOG.rst
3 files changed, 68 insertions(+), 27 deletions(-)
Approvals:
Xqt: Looks good to me, approved
jenkins-bot: Verified
diff --git a/HISTORY.rst b/HISTORY.rst
index 3a6e6ad..94482fd 100644
--- a/HISTORY.rst
+++ b/HISTORY.rst
@@ -53,7 +53,8 @@
and its *preloaded_modules* property was deprecated, the :meth:`data.api.ParamInfo.normalize_paraminfo`
method became a staticmethod (:phab:`T306637`)
* raise ValueError when :class:`pywikibot.FilePage` title doesn't have a valid file extension (:phab:`T345786`)
-* :attr:`site.APISite.file_extensions <pywikibot.site._apisite.APISite.file_extensions>` property was added (:phab:`T345786`)
+* :attr:`site.APISite.file_extensions <pywikibot.site._apisite.APISite.file_extensions>` property was added
+ (:phab:`T345786`)
* ``dropdelay`` and ``releasepid`` attributes of :class:`throttle.Throttle` where deprecated
in favour of `expiry` class attribute
* Add https scheme if missing in url asked by :mod:`pywikibot.scripts.generate_family_file`
@@ -166,7 +167,8 @@
* Raise :exc:`exceptions.NoPageError` when deleting a missing Page (:phab:`T332924`)
* ``text`` parameter of :class:`proofreadpage.PagesTagParser` has a default value
* L10N updates
-* Ignore talk pages with :meth:`APISite.watched_pages()<pywikibot.site._generators.GeneratorsMixin.watched_pages>` (:phab:`T330806`)
+* Ignore talk pages with :meth:`APISite.watched_pages()<pywikibot.site._generators.GeneratorsMixin.watched_pages>`
+ (:phab:`T330806`)
* Load page info when creating a page if not updated previously (:phab:`T330980`)
* Improve flush exception logging (:phab:`T108444`)
@@ -177,7 +179,8 @@
* L10N Updates
* Minimal needed mwparserfromhell was decreased to 0.5.2 (:phab:`T326498`, :phab:`T327600`)
-* No longer lazy load password cookies (:phab:`T271858`, :phab:`T326779`, :phab:`T329132`, :phab:`T330488`, :phab:`T331315`)
+* No longer lazy load password cookies
+ (:phab:`T271858`, :phab:`T326779`, :phab:`T329132`, :phab:`T330488`, :phab:`T331315`)
8.0.3
@@ -401,7 +404,8 @@
* Provide Built Distribution with Pywikibot (:pep:`427`)
* Update `WRITE_ACTIONS` in used by :class:`api.Request<data.api.Request>`
-* Move :func:`get_closest_memento_url<data.memento.get_closest_memento_url>` from weblinkchecker script to memento module.
+* Move :func:`get_closest_memento_url<data.memento.get_closest_memento_url>` from weblinkchecker script to memento
+ module.
* Add :mod:`memento module<data.memento>` to fix memento_client package (:phab:`T185561`)
* L10N and i18n updates
* Fix Flow board topic continuation when iterating in reverse (:phab:`T138323`)
@@ -532,7 +536,8 @@
* Add support for Python 3.11
* Pywikibot supports PyPy 3 (:phab:`T101592`)
* A new method User.is_locked() was added to determine whether the user is currently locked globally (:phab:`T249392`)
-* A new method APISite.is_locked() was added to determine whether a given user or user id is locked globally (:phab:`T249392`)
+* A new method APISite.is_locked() was added to determine whether a given user or user id is locked globally
+ (:phab:`T249392`)
* APISite.get_globaluserinfo() method was added to retrieve globaluserinfo for any user or user id (:phab:`T163629`)
* APISite.globaluserinfo attribute may be deleted to force reload
* APISite.is_blocked() method has a force parameter to reload that info
@@ -544,8 +549,10 @@
* Upload: Retry upload if 'copyuploadbaddomain' API error occurs (:phab:`T294825`)
* Update invisible characters from unicodedata 14.0.0
* Add support for Wikimedia OCR engine with proofreadpage
-* Rewrite :func:`tools.itertools.intersect_generators` which makes it running up to 10'000 times faster. (:phab:`T85623`, :phab:`T293276`)
-* The cached output functionality from compat release was re-implemented (:phab:`T151727`, :phab:`T73646`, :phab:`T74942`, :phab:`T132135`, :phab:`T144698`, :phab:`T196039`, :phab:`T280466`)
+* Rewrite :func:`tools.itertools.intersect_generators` which makes it running up to 10'000 times faster.
+ (:phab:`T85623`, :phab:`T293276`)
+* The cached output functionality from compat release was re-implemented
+ (:phab:`T151727`, :phab:`T73646`, :phab:`T74942`, :phab:`T132135`, :phab:`T144698`, :phab:`T196039`, :phab:`T280466`)
* L10N updates
* Adjust groupsize within pagegenerators.PreloadingGenerator (:phab:`T291770`)
* New "maxlimit" property was added to APISite (:phab:`T291770`)
@@ -575,7 +582,8 @@
^^^^^^^^^^^^^^^^
* Support of Python 3.5.0 - 3.5.2 has been dropped (:phab:`T286867`)
-* generate_user_files.py, generate_user_files.py, shell.py and version.py were moved to pywikibot/scripts and must be used with pwb wrapper script
+* generate_user_files.py, generate_user_files.py, shell.py and version.py were moved to pywikibot/scripts and must be
+ used with pwb wrapper script
* *See also Code cleanups below*
Code cleanups
@@ -606,9 +614,11 @@
* Remove disable_ssl_certificate_validation kwargs in http functions in favour of verify parameter (:phab:`T265206`)
* Deprecated PYWIKIBOT2 environment variables were removed
* version.ParseError was removed in favour of exceptions.VersionParseError
-* specialbots.EditReplacement and specialbots.EditReplacementError were removed in favour of exceptions.EditReplacementError
+* specialbots.EditReplacement and specialbots.EditReplacementError were removed in favour of
+ exceptions.EditReplacementError
* site.PageInUse exception was removed in favour of exceptions.PageInUseError
-* page.UnicodeToAsciiHtml and page.unicode2html were removed in favour of tools.chars.string_to_ascii_html and tools.chars.string2html
+* page.UnicodeToAsciiHtml and page.unicode2html were removed in favour of tools.chars.string_to_ascii_html and
+ tools.chars.string2html
* interwiki_graph.GraphImpossible and login.OAuthImpossible exception were removed in favour of ImportError
* i18n.TranslationError was removed in favour of exceptions.TranslationError
* WikiaFamily was removed in favour of FandomFamily
@@ -863,7 +873,8 @@
* Deprecated Family.known_families dict was removed (:phab:`T89451`)
* Deprecated DataSite.get_* methods was removed
* Deprecated LogEntryFactory.logtypes classproperty was removed
-* Unused comms.threadedhttp module was removed; threadedhttp.HttpRequest was already replaced with requests.Response (:phab:`T265206`)
+* Unused comms.threadedhttp module was removed; threadedhttp.HttpRequest was already replaced with requests.Response
+ (:phab:`T265206`)
Other changes
^^^^^^^^^^^^^
@@ -885,7 +896,8 @@
* Add support for mniwiki and mniwiktionary (:phab:`T273467`, :phab:`T273462`)
* Don't use mime parameter as boolean in api.Request (:phab:`T274723`)
* textlib.removeDisabledPart is able to remove templates (:phab:`T274138`)
-* Create a SiteLink with __getitem__ method and implement lazy load (:phab:`T273386`, :phab:`T245809`, :phab:`T238471`, :phab:`T226157`)
+* Create a SiteLink with __getitem__ method and implement lazy load
+ (:phab:`T273386`, :phab:`T245809`, :phab:`T238471`, :phab:`T226157`)
* Fix date.formats['MonthName'] behaviour (:phab:`T273573`)
* Implement pagegenerators.handle_args() to process all options at once
* Add enabled_options, disabled_options to GeneratorFactory (:phab:`T271320`)
@@ -897,7 +909,8 @@
-----
*24 January 2021*
-* Use string instead of Path-like object with "open" function in UploadRobot for Python 3.5 compatibility (:phab:`T272345`)
+* Use string instead of Path-like object with "open" function in UploadRobot for Python 3.5 compatibility
+ (:phab:`T272345`)
* Add support for trwikivoyage (:phab:`T271263`)
* UI.input_list_choice() has been improved (:phab:`T272237`)
* Global handleArgs() function was removed in favour of handle_args
@@ -916,7 +929,8 @@
-----
*12 January 2021*
-* Add support for niawiki, bclwikt, diqwikt, niawikt (:phab:`T270416`, :phab:`T270282`, :phab:`T270278`, :phab:`T270412`)
+* Add support for niawiki, bclwikt, diqwikt, niawikt
+ (:phab:`T270416`, :phab:`T270282`, :phab:`T270278`, :phab:`T270412`)
* Delete page using pageid instead of title (:phab:`T57072`)
* version.getversion_svn_setuptools function was removed (:phab:`T270393`)
* Add support for "musical notation" data type to wikibase
@@ -948,7 +962,8 @@
* Allow using pywikibot as site-package without user-config.py (:phab:`T270474`)
* Python 3.10 is supported
* Fix AutoFamily scriptpath() call (:phab:`T270370`)
-* Add support for skrwiki, skrwiktionary, eowikivoyage, wawikisource, madwiki (:phab:`T268414`, :phab:`T268460`, :phab:`T269429`, :phab:`T269434`, :phab:`T269442`)
+* Add support for skrwiki, skrwiktionary, eowikivoyage, wawikisource, madwiki
+ (:phab:`T268414`, :phab:`T268460`, :phab:`T269429`, :phab:`T269434`, :phab:`T269442`)
* wikistats methods fetch, raw_cached, csv, xml has been removed
* PageRelatedError.getPage() has been removed in favour of PageRelatedError.page
* DataSite.get_item() method has been removed
@@ -956,7 +971,8 @@
* Property.getType() method has been removed
* Family.server_time() method was removed; it is still available from Site object (:phab:`T89451`)
* All HttpRequest parameters except of charset has been dropped (:phab:`T265206`)
-* A lot of methods and properties of HttpRequest are deprecared in favour of requests.Resonse attributes (:phab:`T265206`)
+* A lot of methods and properties of HttpRequest are deprecared in favour of requests.Resonse attributes
+ (:phab:`T265206`)
* Method and properties of HttpRequest are delegated to requests.Response object (:phab:`T265206`)
* comms.threadedhttp.HttpRequest.raw was replaced by HttpRequest.content property (:phab:`T265206`)
* Desupported version.getfileversion() has been removed
@@ -1027,7 +1043,6 @@
* comms.PywikibotCookieJar and comms.mode_check_decorator were deleted
* Remove deprecated tools classes Stringtypes and UnicodeType
* Remove deprecated tools function open_compressed and signature and UnicodeType class
-* Fix http_tests.LiveFakeUserAgentTestCase (:phab:`T265842`)
* HttpRequest properties were renamed to request.Response identifiers (:phab:`T265206`)
@@ -1043,7 +1058,8 @@
* Accept only valid names in generate_family_file.py (:phab:`T265328`, :phab:`T265353`)
* New plural.plural_rule() function returns a rule for a given language
* Replace deprecated urllib.request.URLopener with http.fetch (:phab:`T255575`)
-* OptionHandler/BaseBot options are accessable as OptionHandler.opt attributes or keyword item (see also :phab:`T264721`)
+* OptionHandler/BaseBot options are accessable as OptionHandler.opt attributes or keyword item
+ (see also :phab:`T264721`)
* pywikibot.setAction() function was removed
* A namedtuple is the result of textlib.extract_sections()
* Prevent circular imports in config2.py and http.py (:phab:`T264500`)
@@ -1209,7 +1225,6 @@
* Do not removeUselessSpaces inside source/syntaxhighlight tags (:phab:`T250469`)
* Restrict Pillow to 6.2.2+ (:phab:`T249911`)
* Fix PetScan generator language and project (:phab:`T249704`)
-* test_family has been removed (:phab:`T228375`, :phab:`T228300`)
* Bugfixes and improvements
* Localisation updates
@@ -1223,7 +1238,8 @@
* Usage of SkipPageError with BaseBot is desupported and may be removed
* Ignore InvalidTitle in textlib.replace_links() (:phab:`T122091`)
* Raise ServerError also if connection to PetScan timeouts
-* pagegenerators.py no longer supports 'oursql' or 'MySQLdb'. It now solely supports PyMySQL (:phab:`T243154`, :phab:`T89976`)
+* pagegenerators.py no longer supports 'oursql' or 'MySQLdb'. It now solely supports PyMySQL
+ (:phab:`T243154`, :phab:`T89976`)
* Disfunctional Family.versionnumber() method was removed
* Refactor login functionality (:phab:`T137805`, :phab:`T224712`, :phab:`T248767`, :phab:`T248768`, :phab:`T248945`)
* Bugfixes and improvements
@@ -1319,7 +1335,8 @@
* Deprecate the ability to login with a secondary sysop account (:phab:`T71283`)
* Enable global args with pwb.py wrapper script (:phab:`T216825`)
* Add a new ConfigParserBot class to set options from the scripts.ini file (:phab:`T223778`)
-* Check a user's rights rather than group memberships; 'sysopnames' will be deprecated (:phab:`T229293`, :phab:`T189126`, :phab:`T122705`, :phab:`T119335`, :phab:`T75545`)
+* Check a user's rights rather than group memberships; 'sysopnames' will be deprecated
+ (:phab:`T229293`, :phab:`T189126`, :phab:`T122705`, :phab:`T119335`, :phab:`T75545`)
* proofreadpage.py: fix footer detection (:phab:`T230301`)
* Add allowusertalk to the User.block() options (:phab:`T229288`)
* botirc module will be removed in next release (:phab:`T212632`)
@@ -1333,7 +1350,6 @@
*22 July 2019*
* Increase the throttling delay if maxlag >> retry-after (:phab:`T210606`)
-* deprecate test_family: Site('test', 'test'), use wikipedia_family: Site('test', 'wikipedia') instead (:phab:`T228375`, :phab:`T228300`)
* Add "user_agent_description" option in config.py
* APISite.fromDBName works for all known dbnames (:phab:`T225590`, 225723, 226960)
* remove the unimplemented "proxy" variable in config.py
@@ -1487,7 +1503,9 @@
* Fix usages of site.namespaces.NAMESPACE_NAME (:phab:`T201969`)
* pywikibot/textlib.py: Fix header regex to allow comments
* Use 'rvslots' when fetching revisions on MW 1.32+ (:phab:`T200955`)
-* Drop the '2' from PYWIKIBOT2_DIR, PYWIKIBOT2_DIR_PWB, and PYWIKIBOT2_NO_USER_CONFIG environment variables. The old names are now deprecated. The other PYWIKIBOT2_* variables which were used only for testing purposes have been renamed without deprecation. (:phab:`T184674`)
+* Drop the '2' from PYWIKIBOT2_DIR, PYWIKIBOT2_DIR_PWB, and PYWIKIBOT2_NO_USER_CONFIG environment variables. The old
+ names are now deprecated. The other PYWIKIBOT2_* variables which were used only for testing purposes have been renamed
+ without deprecation. (:phab:`T184674`)
* Introduce a timestamp in deprecated decorator (:phab:`T106121`)
* textlib.extract_sections: Remove footer from the last section (:phab:`T199751`)
* Don't let WikidataBot crash on save related errors (:phab:`T199642`)
@@ -1537,7 +1555,8 @@
* Pywikibot is following :pep:`396` versioning
* pagegenerators AllpagesPageGenerator, CombinedPageGenerator, UnconnectedPageGenerator are deprecated
* Some DayPageGenerator parameters has been renamed
-* unicodedata2, httpbin and Flask dependency was removed (:phab:`T102461`, :phab:`T108068`, :phab:`T178864`, :phab:`T193383`)
+* unicodedata2, httpbin and Flask dependency was removed
+ (:phab:`T102461`, :phab:`T108068`, :phab:`T178864`, :phab:`T193383`)
* New projects were provided
* Bugfixes and improvements
* Documentation updates
@@ -1651,7 +1670,6 @@
* Only force input_choise if self.always is given (:phab:`T161483`)
* Add colon when replacing category and file weblink (:phab:`T127745`)
* API Request: set uiprop only when ensuring 'userinfo' in meta (:phab:`T169202`)
-* Fix TestLazyLoginNotExistUsername test for Stewardwiki (:phab:`T169458`)
Improvements
^^^^^^^^^^^^
@@ -1830,7 +1848,6 @@
* pywikibot: Store ImportError in imported variable
* Use default tox pip install
* Add asteroids that are being used as locations
-* [bugfix] Fix test_translateMagicWords test
* Fix ID for Rhea
* [bugfix] pass User page object to NotEmailableError
* Allow pywikibot to run on Windows 10 as well
diff --git a/ROADMAP.rst b/ROADMAP.rst
index 0e61df9..bd33c1f 100644
--- a/ROADMAP.rst
+++ b/ROADMAP.rst
@@ -4,6 +4,10 @@
Improvements
^^^^^^^^^^^^
+* A copy button was added to the sphinx documentation.
+* Make :attr:`languages_by_size<family.WikimediaFamily.languages_by_size>` dynamic (:phab:`T78396`). The property is
+ only available for :class:`family.WikimediaFamily` families. The ``wikimedia_sites.py`` maintenance script was
+ removed.
* Add :func:`config.base_dir<config.get_base_dir>` to scripts search path with :mod:`pwb` wrapper (:phab:`T324287`)
* :meth:`pywikibot.WbTime.equal_instant` was added (:phab:`T325248`)
* ``revisions`` parameter of :class:`xmlreader.XmlDump` was introduced to specify parsing method
@@ -11,12 +15,14 @@
* Pass global -nolog argument into bot script from wrapper (:phab:`T328900`)
* Add :meth:`site.APISite.ratelimit()<pywikibot.site._apisite.APISite.ratelimit>` method
and :class:`tools.collections.RateLimit` NamedTuple (:phab:`T304808`)
-* L10N Updates
+* L10N and i18n updates
* Add :class:`pagegenerators.PagePilePageGenerator` (:phab:`T353086`)
Bugfixes
^^^^^^^^
+* Only delegate :mod:`site` methods to public :class:`family.Family` methods which have *code* as first parameter.
+* Use ``str`` instead of ``repr`` for several messages with :class:`family.Family` objects (:phab:`T356782`)
* Add ``hy`` to special languages in :class:`textlib.TimeStripper` (:phab:`T356175`)
* Pass login token when using ``action=login`` (:phab:`T309898`)
* Detect range blocks with :meth:`pywikibot.User.is_blocked` (:phab:`T301282`)
@@ -28,6 +34,8 @@
Breaking changes and code cleanups
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+* Raise ``TypeError`` instead of ``AttributeError`` in :meth:`Site.randompages()
+ <pywikibot.site._generators.GeneratorsMixin.randompages>` if *redirects* parameter is invalid.
* A RuntimeError will be raised if a :class:`family.Family` subclass has an ``__init__`` initializer method.
:meth:`family.Family.__post_init__` classmethod can be used instead.
* :class:`InteractiveReplace<bot_choice.InteractiveReplace>` was moved from :mod:`bot` to :mod:`bot_choice` module
diff --git a/scripts/CHANGELOG.rst b/scripts/CHANGELOG.rst
index d1b701c..4fd41fb 100644
--- a/scripts/CHANGELOG.rst
+++ b/scripts/CHANGELOG.rst
@@ -15,6 +15,11 @@
* provide category_redirect categories for all WM sites (:phab:`T348914`)
+interwiki
+^^^^^^^^^
+
+* Ignore :exc:`exceptions.InvalidTitleError` and :exc:`InvalidPageError` (:phab:`T357953`)
+
touch
^^^^^
--
To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/1005516?usp=email
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Change-Id: I4825991fa052dd8ea30700595383aefe6b4caaa1
Gerrit-Change-Number: 1005516
Gerrit-PatchSet: 2
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot
Gerrit-MessageType: merged
jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/1005124?usp=email )
Change subject: [bugfix] Ignore InvalidTitleError/InvalidPageError
......................................................................
[bugfix] Ignore InvalidTitleError/InvalidPageError
Bug: T357953
Change-Id: I199d46f59d7b0489b02650c0740e1879fa54ae2a
---
M scripts/interwiki.py
1 file changed, 36 insertions(+), 8 deletions(-)
Approvals:
jenkins-bot: Verified
Xqt: Looks good to me, approved
diff --git a/scripts/interwiki.py b/scripts/interwiki.py
index d0a4d4e..ead72d0 100755
--- a/scripts/interwiki.py
+++ b/scripts/interwiki.py
@@ -328,7 +328,7 @@
"""
#
-# (C) Pywikibot team, 2003-2023
+# (C) Pywikibot team, 2003-2024
#
# Distributed under the terms of the MIT license.
#
@@ -357,6 +357,7 @@
from pywikibot.exceptions import (
EditConflictError,
Error,
+ InvalidPageError,
InvalidTitleError,
LockedPageError,
NoCreateError,
@@ -1107,9 +1108,16 @@
def check_page(self, page, counter) -> None:
"""Check whether any iw links should be added to the todo list."""
- if not page.exists():
+ try:
+ ok = page.exists()
+ except InvalidPageError as e: # T357953
+ msg = str(e)
+ ok = False
+ else:
+ msg = f'{page} does not exist.'
+ if not ok:
self.conf.remove.append(str(page))
- self.conf.note(f'{page} does not exist. Skipping.')
+ self.conf.note(f'{msg} Skipping.')
if page == self.origin:
# The page we are working on is the page that does not
# exist. No use in doing any work on it in that case.
@@ -1311,7 +1319,12 @@
# Each value will be a list of pages.
new = defaultdict(list)
for page in self.done:
- if page.exists() and not page.isRedirectPage() \
+ try:
+ ok = page.exists()
+ except InvalidPageError: # T357953
+ continue
+
+ if ok and not page.isRedirectPage() \
and not page.isCategoryRedirect():
site = page.site
if site.family.interwiki_forward:
@@ -1985,10 +1998,15 @@
# Get the content of the assembled list in one blow
gen = site.preloadpages(pageGroup, templates=True, langlinks=True,
pageprops=True, quiet=False)
- for _ in gen:
- # we don't want to do anything with them now. The
- # page contents will be read via the Subject class.
- pass
+ while True:
+ # we don't want to do anything with them now.
+ # The page contents will be read via the Subject class.
+ try:
+ next(gen)
+ except StopIteration:
+ break
+ except InvalidTitleError: # T357953
+ pass
# Tell all of the subjects that the promised work is done
for subject in subjectGroup:
--
To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/1005124?usp=email
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Change-Id: I199d46f59d7b0489b02650c0740e1879fa54ae2a
Gerrit-Change-Number: 1005124
Gerrit-PatchSet: 3
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot
Gerrit-MessageType: merged
jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/996075?usp=email )
Change subject: [IMPR] raise TypeError in Site.randompages if redirects parameter is invalid
......................................................................
[IMPR] raise TypeError in Site.randompages if redirects parameter is invalid
Also add typing hints for GeneratorsMixin
Change-Id: I58ebcceda48a78013006dc6f202e3338736cc426
---
M pywikibot/site/_generators.py
1 file changed, 48 insertions(+), 8 deletions(-)
Approvals:
jenkins-bot: Verified
Xqt: Looks good to me, approved
diff --git a/pywikibot/site/_generators.py b/pywikibot/site/_generators.py
index 763b0d9..aae2e82 100644
--- a/pywikibot/site/_generators.py
+++ b/pywikibot/site/_generators.py
@@ -13,7 +13,7 @@
from typing import TYPE_CHECKING, Any
import pywikibot
-from pywikibot.backports import Generator, Iterable, batched
+from pywikibot.backports import Callable, Generator, Iterable, batched
from pywikibot.data import api
from pywikibot.exceptions import (
APIError,
@@ -34,13 +34,33 @@
if TYPE_CHECKING:
- from pywikibot.site._namespace import SingleNamespaceType
+ from data.api import ParamInfo, Request
+ from pywikibot.site._apisite import _RequestWrapperT
+ from pywikibot.site._namespace import NamespacesDict, SingleNamespaceType
+ from pywikibot.site._tokenwallet import TokenWallet
+ from pywikibot.tools import MediaWikiVersion
class GeneratorsMixin:
"""API generators mixin to MediaWiki site."""
+ if TYPE_CHECKING:
+ _generator: Callable[..., _RequestWrapperT]
+ _paraminfo: ParamInfo
+ _request: Callable[..., Request]
+ assert_valid_iter_params: Callable[..., None]
+ encoding: Callable[[], str]
+ get_property_names: Callable[..., list[str]]
+ has_right: Callable[[str], bool]
+ maxlimit: int
+ mw_version: MediaWikiVersion
+ namespaces: NamespacesDict
+ protection_types: Callable[[], set[str]]
+ sametitle: Callable[[str, str], bool]
+ tokens: TokenWallet
+ user: Callable[[], str | None]
+
def load_pages_from_pageids(
self,
pageids: str | Iterable[int | str],
@@ -1787,11 +1807,14 @@
) -> Iterable[pywikibot.Page]:
"""Iterate a number of random pages.
- .. seealso: :api:`Random`
-
Pages are listed in a fixed sequence, only the starting point is
random.
+ .. seealso: :api:`Random`
+ .. versionchanged:: 9.0
+ Raises ``TypeError`` instead of ``AssertionError`` if
+ *redirects* is invalid.
+
:param total: the maximum number of pages to iterate
:param namespaces: only iterate pages in these namespaces.
:param redirects: if True, include only redirect pages in
@@ -1802,10 +1825,12 @@
:raises KeyError: a namespace identifier was not resolved
:raises TypeError: a namespace identifier has an inappropriate
type such as NoneType or bool
- :raises AssertError: unsupported redirects parameter
+ :raises TypeError: unsupported redirects parameter
"""
mapping = {False: None, True: 'redirects', None: 'all'}
- assert redirects in mapping
+ if redirects not in mapping:
+ raise TypeError(f"Invalid type for 'redirects' parameter: "
+ f'{type(redirects).__name__}({redirects})')
redirects_ = mapping[redirects]
params = {}
if redirects_ is not None:
@@ -1970,9 +1995,13 @@
) -> Iterable[pywikibot.Page]:
"""Iterate Page objects retrieved from Special:{special_page}.
- .. seealso:: :api:`Querypage`
+ Generic function for all special pages supported by the site MW
+ API.
- Generic function for all special pages supported by the site MW API.
+ .. seealso:: :api:`Querypage`
+ .. versionchanged:: 9.0
+ Raises ``ValueError`` instead of ``AssertionError`` if
+ *special_page* is invalid.
:param special_page: Special page to query
:param total: number of pages to return
--
To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/996075?usp=email
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Change-Id: I58ebcceda48a78013006dc6f202e3338736cc426
Gerrit-Change-Number: 996075
Gerrit-PatchSet: 6
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot
Gerrit-MessageType: merged