jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/680301 )
Change subject: [parser] Make mwparserfromhell or wikitextparser mandatory
......................................................................
[parser] Make mwparserfromhell or wikitextparser mandatory
- check whether mwparserfromhell or wikitextparser is installed and
raise ImportError if no package is available
- update tox.ini
- update some documentation hints
Bug: T106763
Change-Id: Ie89d08074a9d9b6380a0276fac261b8f6a55156f
---
M ROADMAP.rst
M pywikibot/textlib.py
M tests/__init__.py
M tox.ini
4 files changed, 20 insertions(+), 60 deletions(-)
Approvals:
Xqt: Looks good to me, approved
jenkins-bot: Verified
diff --git a/ROADMAP.rst b/ROADMAP.rst
index 36e8705..b242ebb 100644
--- a/ROADMAP.rst
+++ b/ROADMAP.rst
@@ -1,7 +1,7 @@
Current release changes
^^^^^^^^^^^^^^^^^^^^^^^
-* (No changes yet)
+* mwparserfromhell or wikitextparser MediaWiki markup parser is mandatory (T106763)
Deprecations
^^^^^^^^^^^^
@@ -14,7 +14,6 @@
* 6.2.0: empty_iterator will be removed in favour of iter()
* 6.1.0: tools.frozenmap will be removed in favour of types.MappingProxyType
* 6.1.0: tools.DotReadableDict will be removed
-* 6.1.0: mwparserfromhell or wikitextparser MediaWiki markup parser becomes mandatory (T106763)
* 6.1.0: textlib.unescape() function will be removed in favour of html.unescape()
* 6.0.1: Site.undeletepage() and Site.undelete_file_versions() will be removed in favour of Site.undelete() method
* 6.0.1: Site.deletepage() and Site.deleteoldimage() will be removed in favour of Site.delete() method
diff --git a/pywikibot/textlib.py b/pywikibot/textlib.py
index 2b14c97..696c6c3 100644
--- a/pywikibot/textlib.py
+++ b/pywikibot/textlib.py
@@ -37,8 +37,18 @@
except ImportError:
try:
import mwparserfromhell as wikitextparser
- except ImportError as e:
- wikitextparser = e
+ except ImportError:
+ # print required because pywikibot is not imported completely
+ raise ImportError("""
+Pywikibot is missing a MediaWiki markup parser which is necessary.
+Please update the required module with either
+
+ pip install "mwparserfromhell>=0.5.0"
+
+or
+
+ pip install "wikitextparser>=0.47.0"
+""") from None
ETPType = List[Tuple[str, OrderedDictType[str, str]]]
@@ -1585,16 +1595,11 @@
only the last value provided will be returned.
This uses the package L{mwparserfromhell} or L{wikitextparser} as
- MediaWiki markup parser. Otherwise it falls back on a regex based
- implementation but it becomes mandatory that one of them is
+ MediaWiki markup parser. It is mandatory that one of them is
installed.
There are minor differences between the two implementations.
- The two implementations return nested templates in a different
- order, i.e. for `{{a|b={{c}}}}`, parsers returns `[a, c]`, whereas
- regex returns `[c, a]`.
-
The parser packages preserves whitespace in parameter names and
values.
@@ -1613,45 +1618,6 @@
*New in version 6.1:* *wikitextparser* package is supported; either
*wikitextparser* or *mwparserfromhell* is strictly recommended.
"""
- use_regex = isinstance(wikitextparser, ImportError)
-
- if remove_disabled_parts:
- text = removeDisabledParts(text)
-
- if use_regex:
- issue_deprecation_warning("""
-Pywikibot needs a MediaWiki markup parser.
-Please install the requested module with either
-
- pip install "mwparserfromhell>=0.5.0"
-
-or
-
- pip install "wikitextparser>=0.47.0"
-
-Using pywikibot without MediaWiki markup parser""",
- warning_class=FutureWarning,
- since='20210416')
-
- return _extract_templates_and_params_regex(text, False, strip)
- return _extract_templates_and_params_parser(text, strip)
-
-
-def _extract_templates_and_params_parser(text: str,
- strip: bool = False) -> ETPType:
- """
- Extract templates with params using mwparserfromhell.
-
- This function should not be called directly.
-
- Use extract_templates_and_params, which will select this parser
- implementation if the mwparserfromhell or wikitextparser package is
- installed.
-
- @param text: The wikitext from which templates are extracted
- @param strip: if enabled, strip arguments and values of templates
- @return: list of template name and params
- """
def explicit(param):
try:
attr = param.showkey
@@ -1659,6 +1625,9 @@
attr = not param.positional
return attr
+ if remove_disabled_parts:
+ text = removeDisabledParts(text)
+
parser_name = wikitextparser.__name__
pywikibot.log('Using {!r} wikitext parser'.format(parser_name))
@@ -1701,7 +1670,7 @@
global wikitextparser
saved_parser = wikitextparser
import mwparserfromhell as wikitextparser
- result = _extract_templates_and_params_parser(text, strip)
+ result = extract_templates_and_params(text, strip=strip)
wikitextparser = saved_parser
return result
@@ -1723,14 +1692,6 @@
@param strip: if enabled, strip arguments and values of templates
@return: list of template name and params
"""
- return _extract_templates_and_params_regex(text, remove_disabled_parts,
- strip)
-
-
-def _extract_templates_and_params_regex(text: str,
- remove_disabled_parts: bool = True,
- strip: bool = True) -> ETPType:
- """DEPRECATED. Extract templates with params using a regex."""
# remove commented-out stuff etc.
if remove_disabled_parts:
thistxt = removeDisabledParts(text)
diff --git a/tests/__init__.py b/tests/__init__.py
index ebac252..ce88efe 100644
--- a/tests/__init__.py
+++ b/tests/__init__.py
@@ -23,7 +23,7 @@
# Verify that the unit tests have a base working environment:
# - requests is mandatory
# however if unavailable this will fail on use; see pywikibot/tools.py
-# - mwparserfromhell or wikitextparser should be used but the dependency
+# - mwparserfromhell or wikitextparser is mandatory but the dependency
# is checked by textlib already
import requests # noqa: F401
diff --git a/tox.ini b/tox.ini
index 7bf0112..98c63f2 100644
--- a/tox.ini
+++ b/tox.ini
@@ -38,10 +38,10 @@
fasttest: pytest-attrib>=0.1.3
fasttest: pytest-subtests >= 0.3.2
fasttest: mock
+ fasttest: .[mwparserfromhell]
fasttest: .[scripts]
fasttest-py35: .[html]
- fasttest-py35: .[mwparserfromhell]
fasttest-py37: .[wikitextparser]
deeptest: .[html]
--
To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/680301
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Change-Id: Ie89d08074a9d9b6380a0276fac261b8f6a55156f
Gerrit-Change-Number: 680301
Gerrit-PatchSet: 2
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot
Gerrit-MessageType: merged
jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/697061 )
Change subject: [cleanup] Require archivebot durations to have a time unit
......................................................................
[cleanup] Require archivebot durations to have a time unit
This reverts commit 4a350673c4aec0fefb1029f47698f7819430a35c.
Change-Id: I3dfa60dfa66022cdb1f573605d495078407703e1
---
M scripts/archivebot.py
M tests/archivebot_tests.py
2 files changed, 17 insertions(+), 15 deletions(-)
Approvals:
Xqt: Looks good to me, approved
jenkins-bot: Verified
diff --git a/scripts/archivebot.py b/scripts/archivebot.py
index f3c2370..03353a8 100755
--- a/scripts/archivebot.py
+++ b/scripts/archivebot.py
@@ -116,7 +116,6 @@
findmarker,
to_local_digits,
)
-from pywikibot.tools import issue_deprecation_warning
ShouldArchive = Tuple[str, str]
@@ -242,15 +241,19 @@
@return: key and duration extracted form the string
"""
- if string.isdigit():
- key = 's'
- duration = string
- issue_deprecation_warning('Time period without qualifier',
- string + key, 1, FutureWarning,
- since='20161009')
- else:
- key = string[-1]
- duration = string[:-1]
+ if len(string) < 2:
+ raise MalformedConfigError('Time period should be a numeric value '
+ 'followed by its qualifier')
+
+ key, duration = string[-1], string[:-1]
+
+ if key not in MW_KEYS:
+ raise MalformedConfigError('Time period qualifier is unrecognized: {}'
+ .format(string))
+ if not duration.isdigit():
+ raise MalformedConfigError("Time period's duration should be "
+ 'numeric: {}'.format(string))
+
return key, duration
diff --git a/tests/archivebot_tests.py b/tests/archivebot_tests.py
index 9bf1e36..e3a3828 100644
--- a/tests/archivebot_tests.py
+++ b/tests/archivebot_tests.py
@@ -12,7 +12,6 @@
import pywikibot.page
from pywikibot.exceptions import Error
from pywikibot.textlib import TimeStripper
-from pywikibot.tools import suppress_warnings
from scripts import archivebot
from tests.aspects import TestCase
@@ -97,12 +96,12 @@
def test_checkstr(self):
"""Test for extracting key and duration from shorthand notation."""
self.assertEqual(archivebot.checkstr('400s'), ('s', '400'))
- with suppress_warnings('Time period without qualifier', UserWarning):
- self.assertEqual(archivebot.checkstr('3000'), ('s', '3000'))
self.assertEqual(archivebot.checkstr('7d'), ('d', '7'))
self.assertEqual(archivebot.checkstr('3y'), ('y', '3'))
- # Should pass, because the key is verified in str2time
- self.assertEqual(archivebot.checkstr('4000@'), ('@', '4000'))
+
+ for invalid_value in ('', '3000', '4000@'):
+ with self.assertRaises(archivebot.MalformedConfigError):
+ archivebot.checkstr(invalid_value)
def test_str2size(self):
"""Test for parsing the shorthand notation of sizes."""
--
To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/697061
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Change-Id: I3dfa60dfa66022cdb1f573605d495078407703e1
Gerrit-Change-Number: 697061
Gerrit-PatchSet: 2
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: D3r1ck01 <xsavitar.wiki(a)aol.com>
Gerrit-Reviewer: Damian <atagar1(a)gmail.com>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot
Gerrit-MessageType: merged
jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/697058 )
Change subject: [6.3] Prepare next release
......................................................................
[6.3] Prepare next release
Change-Id: I474c51d33f63f6009d58947490c1016cae16fd4e
---
M .appveyor.yml
M HISTORY.rst
M ROADMAP.rst
M pywikibot/__metadata__.py
4 files changed, 48 insertions(+), 41 deletions(-)
Approvals:
Xqt: Looks good to me, approved
jenkins-bot: Verified
diff --git a/.appveyor.yml b/.appveyor.yml
index f8a57d6..534fc84 100644
--- a/.appveyor.yml
+++ b/.appveyor.yml
@@ -1,7 +1,7 @@
image: Visual Studio 2019
clone_depth: 50
skip_tags: true
-version: 6.2.{build}
+version: 6.3.{build}
environment:
PYWIKIBOT_DIR: "%appdata%\\Pywikibot"
diff --git a/HISTORY.rst b/HISTORY.rst
index 1c316df..8348776 100644
--- a/HISTORY.rst
+++ b/HISTORY.rst
@@ -1,6 +1,51 @@
Release history
^^^^^^^^^^^^^^^
+6.2.0
+-----
+*28 May 2021*
+
+Improvements and Bugfixes
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
+* Use different logfiles for multiple processes of the same script (T56685)
+* throttle.pip will be reused as soon as possbile
+* terminal_interface_base.TerminalHandler is subclassed from logging.StreamHandler
+* Fix iterating of SizedKeyCollection (T282865)
+* An abstract base user interface module was added
+* APISite method pagelanglinks() may skip links with empty titles (T223157)
+* Fix Page.getDeletedRevision() method which always returned an empty list
+* Async chunked uploads are supported (T129216, T133443)
+* A new InvalidPageError will be raised if a Page has no version history (T280043)
+* L10N updates
+* Fix __getattr__ for WikibaseEntity (T281389)
+* Handle abusefilter-{disallow,warning} codes (T85656)
+
+Code cleanups
+~~~~~~~~~~~~~
+
+* MultipleSitesBot.site attribute was removed (T283209)
+* Deprecated BaseSite.category_namespaces() method was removed
+* i18n.twntranslate() function was removed in favour of twtranslate()
+* siteinfo must be used as a dictionary ad cannot be called anymore
+* APISite.has_transcluded_data() method was removed
+* Deprecated LogEntry.title() method was removed
+* Deprecated APISite.watchpage() method was removed
+* OptionHandler.options dict has been removed in favour of OptionHandler.opt
+* The toStdout parameter of ui.output has been dropped
+* terminal_interface_base.TerminalFormatter was removed
+* Move page functions UnicodeToAsciiHtml, unicode2html, url2unicode to tools.chars with renaming them
+* Rename _MultiTemplateMatchBuilder to MultiTemplateMatchBuilder
+* User.name() method was removed in favour of User.username property
+* BasePage.getLatestEditors() method was removed in favour of contributors() or revisions()
+* pagenenerators.handleArg() method was renamed to handle_arg() (T271437)
+* CategoryGenerator, FileGenerator, ImageGenerator and ReferringPageGenerator pagegenerator functions were removed
+* Family.ignore_certificate_error() method was removed in favour of verify_SSL_certificate (T265205)
+* tools.is_IP was renamed to is_ip_address due to PEP8
+* config2.py was renamed to config.py
+* Exceptions were renamed having a suffix "Error" due to PEP8 (T280227)
+
+
6.1.0
-----
*17 April 2021*
diff --git a/ROADMAP.rst b/ROADMAP.rst
index a4c4dc5..36e8705 100644
--- a/ROADMAP.rst
+++ b/ROADMAP.rst
@@ -1,45 +1,7 @@
Current release changes
^^^^^^^^^^^^^^^^^^^^^^^
-Improvements and Bugfixes
--------------------------
-
-* Use different logfiles for multiple processes of the same script (T56685)
-* throttle.pip will be reused as soon as possbile
-* terminal_interface_base.TerminalHandler is subclassed from logging.StreamHandler
-* Fix iterating of SizedKeyCollection (T282865)
-* An abstract base user interface module was added
-* APISite method pagelanglinks() may skip links with empty titles (T223157)
-* Fix Page.getDeletedRevision() method which always returned an empty list
-* Async chunked uploads are supported (T129216, T133443)
-* A new InvalidPageError will be raised if a Page has no version history (T280043)
-* L10N updates
-* Fix __getattr__ for WikibaseEntity (T281389)
-* Handle abusefilter-{disallow,warning} codes (T85656)
-
-Code cleanups
--------------
-
-* MultipleSitesBot.site attribute was removed (T283209)
-* Deprecated BaseSite.category_namespaces() method was removed
-* i18n.twntranslate() function was removed in favour of twtranslate()
-* siteinfo must be used as a dictionary ad cannot be called anymore
-* APISite.has_transcluded_data() method was removed
-* Deprecated LogEntry.title() method was removed
-* Deprecated APISite.watchpage() method was removed
-* OptionHandler.options dict has been removed in favour of OptionHandler.opt
-* The toStdout parameter of ui.output has been dropped
-* terminal_interface_base.TerminalFormatter was removed
-* Move page functions UnicodeToAsciiHtml, unicode2html, url2unicode to tools.chars with renaming them
-* Rename _MultiTemplateMatchBuilder to MultiTemplateMatchBuilder
-* User.name() method was removed in favour of User.username property
-* BasePage.getLatestEditors() method was removed in favour of contributors() or revisions()
-* pagenenerators.handleArg() method was renamed to handle_arg() (T271437)
-* CategoryGenerator, FileGenerator, ImageGenerator and ReferringPageGenerator pagegenerator functions were removed
-* Family.ignore_certificate_error() method was removed in favour of verify_SSL_certificate (T265205)
-* tools.is_IP was renamed to is_ip_address due to PEP8
-* config2.py was renamed to config.py
-* Exceptions were renamed having a suffix "Error" due to PEP8 (T280227)
+* (No changes yet)
Deprecations
^^^^^^^^^^^^
diff --git a/pywikibot/__metadata__.py b/pywikibot/__metadata__.py
index 02c154b..f035a4b 100644
--- a/pywikibot/__metadata__.py
+++ b/pywikibot/__metadata__.py
@@ -11,7 +11,7 @@
__name__ = 'pywikibot'
-__version__ = '6.2.0'
+__version__ = '6.3.0.dev0'
__description__ = 'Python MediaWiki Bot Framework'
__maintainer__ = 'The Pywikibot team'
__maintainer_email__ = 'pywikibot(a)lists.wikimedia.org'
--
To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/697058
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Change-Id: I474c51d33f63f6009d58947490c1016cae16fd4e
Gerrit-Change-Number: 697058
Gerrit-PatchSet: 1
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot
Gerrit-MessageType: merged
jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/697053 )
Change subject: [doc] Update docs
......................................................................
[doc] Update docs
Change-Id: I0c13c9634e96641b47dd5ba913a3317f2e854ff7
---
M pywikibot/bot.py
M pywikibot/site/_generators.py
2 files changed, 6 insertions(+), 1 deletion(-)
Approvals:
Xqt: Looks good to me, approved
jenkins-bot: Verified
diff --git a/pywikibot/bot.py b/pywikibot/bot.py
index 619e24a..3112344 100644
--- a/pywikibot/bot.py
+++ b/pywikibot/bot.py
@@ -303,6 +303,9 @@
Accordingly, do **not** use print statements in bot code; instead,
use pywikibot.output function.
+ *New in version 6.2:* different logfiles are uses if multiple
+ processes of the same script are are running.
+
@param strm: Output stream. If None, re-uses the last stream if one
was defined, otherwise uses sys.stderr
@@ -1675,6 +1678,8 @@
The bot should accommodate for that case and not store site specific
information on only one site.
+
+ *New in version 6.2:* site attribute has been dropped.
"""
diff --git a/pywikibot/site/_generators.py b/pywikibot/site/_generators.py
index cfa7bf8..484afb9 100644
--- a/pywikibot/site/_generators.py
+++ b/pywikibot/site/_generators.py
@@ -736,7 +736,7 @@
include_empty_titles: bool = False):
"""Iterate all interlanguage links on page, yielding Link objects.
- *New in 6.2:* *include_empty_titles* parameter was added.
+ *New in version 6.2:* *include_empty_titles* parameter was added.
@see: U{https://www.mediawiki.org/wiki/API:Langlinks}
--
To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/697053
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Change-Id: I0c13c9634e96641b47dd5ba913a3317f2e854ff7
Gerrit-Change-Number: 697053
Gerrit-PatchSet: 1
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot
Gerrit-MessageType: merged
jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/697048 )
Change subject: [doc] Update ROADMAP.rst
......................................................................
[doc] Update ROADMAP.rst
Change-Id: I998865c4dc08a66aa696c72c3f4226677bca4c41
---
M ROADMAP.rst
1 file changed, 14 insertions(+), 1 deletion(-)
Approvals:
Xqt: Looks good to me, approved
jenkins-bot: Verified
diff --git a/ROADMAP.rst b/ROADMAP.rst
index c39575b..a4c4dc5 100644
--- a/ROADMAP.rst
+++ b/ROADMAP.rst
@@ -4,6 +4,9 @@
Improvements and Bugfixes
-------------------------
+* Use different logfiles for multiple processes of the same script (T56685)
+* throttle.pip will be reused as soon as possbile
+* terminal_interface_base.TerminalHandler is subclassed from logging.StreamHandler
* Fix iterating of SizedKeyCollection (T282865)
* An abstract base user interface module was added
* APISite method pagelanglinks() may skip links with empty titles (T223157)
@@ -17,6 +20,16 @@
Code cleanups
-------------
+* MultipleSitesBot.site attribute was removed (T283209)
+* Deprecated BaseSite.category_namespaces() method was removed
+* i18n.twntranslate() function was removed in favour of twtranslate()
+* siteinfo must be used as a dictionary ad cannot be called anymore
+* APISite.has_transcluded_data() method was removed
+* Deprecated LogEntry.title() method was removed
+* Deprecated APISite.watchpage() method was removed
+* OptionHandler.options dict has been removed in favour of OptionHandler.opt
+* The toStdout parameter of ui.output has been dropped
+* terminal_interface_base.TerminalFormatter was removed
* Move page functions UnicodeToAsciiHtml, unicode2html, url2unicode to tools.chars with renaming them
* Rename _MultiTemplateMatchBuilder to MultiTemplateMatchBuilder
* User.name() method was removed in favour of User.username property
@@ -31,6 +44,7 @@
Deprecations
^^^^^^^^^^^^
+* 6.2.0: Bot's availableOptions will be removed in favour of available_options
* 6.2.0: deprecated tools.is_IP will be removed
* 6.2.0: Usage of pywikibot.config2 is deprecated and will be dropped
* 6.2.0: Exceptions must be imported from exceptions namespace (T280227)
@@ -43,5 +57,4 @@
* 6.0.1: Site.undeletepage() and Site.undelete_file_versions() will be removed in favour of Site.undelete() method
* 6.0.1: Site.deletepage() and Site.deleteoldimage() will be removed in favour of Site.delete() method
* 6.0.1: DataSite.createNewItemFromPage() method will be removed in favour of ImagePage.fromPage() (T98663)
-* 5.0.0: OptionHandler.options dict will be removed in favour of OptionHandler.opt
* 5.0.0: Methods deprecated for 5 years or longer will be removed
--
To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/697048
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Change-Id: I998865c4dc08a66aa696c72c3f4226677bca4c41
Gerrit-Change-Number: 697048
Gerrit-PatchSet: 1
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot
Gerrit-MessageType: merged