Pywikibot-commits July 2022

pywikibot-commits@lists.wikimedia.org

1 participants
127 discussions

[Gerrit] ...core[stable]: Merge branch 'master' into stable
by Xqt (Code Review) 22 Jul '22

22 Jul '22

Xqt has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/816098 ) Change subject: Merge branch 'master' into stable ...................................................................... Merge branch 'master' into stable Change-Id: Ibd4a79c93cb02d29236128e379ef7ecb41a7b403 --- 1 file changed, 0 insertions(+), 0 deletions(-) Approvals: Xqt: Verified; Looks good to me, approved -- To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/816098 To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings Gerrit-Project: pywikibot/core Gerrit-Branch: stable Gerrit-Change-Id: Ibd4a79c93cb02d29236128e379ef7ecb41a7b403 Gerrit-Change-Number: 816098 Gerrit-PatchSet: 1 Gerrit-Owner: Xqt <info(a)gno.de> Gerrit-Reviewer: Xqt <info(a)gno.de> Gerrit-Reviewer: jenkins-bot Gerrit-MessageType: merged

1 0

[Gerrit] ...core[master]: [7.5] Publish Pywikibot 7.5
by Xqt (Code Review) 22 Jul '22

22 Jul '22

Xqt has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/816096 ) Change subject: [7.5] Publish Pywikibot 7.5 ...................................................................... [7.5] Publish Pywikibot 7.5 Change-Id: I6f7af5d850d4c36a97f4230b094c5d3a279bd1ce --- M pywikibot/__metadata__.py 1 file changed, 1 insertion(+), 1 deletion(-) Approvals: jenkins-bot: Verified Xqt: Looks good to me, approved diff --git a/pywikibot/__metadata__.py b/pywikibot/__metadata__.py index 8e1887d..0c0e1f8 100644 --- a/pywikibot/__metadata__.py +++ b/pywikibot/__metadata__.py @@ -11,7 +11,7 @@ __name__ = 'pywikibot' -__version__ = '7.5.0.dev0' +__version__ = '7.5.0' __description__ = 'Python MediaWiki Bot Framework' __maintainer__ = 'The Pywikibot team' __maintainer_email__ = 'pywikibot(a)lists.wikimedia.org' -- To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/816096 To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Change-Id: I6f7af5d850d4c36a97f4230b094c5d3a279bd1ce Gerrit-Change-Number: 816096 Gerrit-PatchSet: 1 Gerrit-Owner: Xqt <info(a)gno.de> Gerrit-Reviewer: Xqt <info(a)gno.de> Gerrit-Reviewer: jenkins-bot Gerrit-MessageType: merged

1 0

[Gerrit] ...i18n[master]: Localisation updates from https://translatewiki.net.
by Abijeet Patro (Code Review) 21 Jul '22

21 Jul '22

Abijeet Patro has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/i18n/+/815934 ) Change subject: Localisation updates from https://translatewiki.net. ...................................................................... Localisation updates from https://translatewiki.net. Change-Id: Iaca220878aa17e14173efe81a1823a2b9cde0964 --- A delinker/pt-br.json M noreferences/nqo.json 2 files changed, 12 insertions(+), 2 deletions(-) Approvals: L10n-bot: Looks good to me, approved Abijeet Patro: Verified diff --git a/delinker/pt-br.json b/delinker/pt-br.json new file mode 100644 index 0000000..a9c47c4 --- /dev/null +++ b/delinker/pt-br.json @@ -0,0 +1,8 @@ +{ + "@metadata": { + "authors": [ + "Duke of Wikipädia" + ] + }, + "delinker-delink": "Robô: O arquivo [[%(title)s]] foi removido, pois foi eliminado por [[:User:%(user)s]]: ''%(comment)s''." +} diff --git a/noreferences/nqo.json b/noreferences/nqo.json index b4108e9..ba02fee 100644 --- a/noreferences/nqo.json +++ b/noreferences/nqo.json @@ -1,8 +1,10 @@ { "@metadata": { "authors": [ - "Lancine.kounfantoh.fofana" + "Lancine.kounfantoh.fofana", + "ߒߓߋ߫ ߝߏߝߣߊ߫" ] }, - "noreferences-add-tag": "ߡߐ߰ߡߐ߮: ߦߋ߫ {{ߦߟߌߡߊߛߙߋ߫}} ߞߐߢߌ߬ߣߊ߬ߣߍ߲ ߠߎ߬ ߝߊ߬ߙߊ ߞߊ߲߬" + "noreferences-add-tag": "ߡߐ߰ߡߐ߮: ߦߋ߫ {{ߦߟߌߡߊߛߙߋ߫}} ߞߐߢߌ߬ߣߊ߬ߣߍ߲ ߠߎ߬ ߝߊ߬ߙߊ ߞߊ߲߬", + "noreferences-fix-tag": "ߡߐ߰ߡߐ߮: ߛߓߍߛߋ߲߫ ߓߍ߲߬ߓߊߟߌ ߟߎ߫ ߛߊߞߍߟߌ <references /> ߝߎ߲ߓߟߐ" } -- To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/i18n/+/815934 To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings Gerrit-Project: pywikibot/i18n Gerrit-Branch: master Gerrit-Change-Id: Iaca220878aa17e14173efe81a1823a2b9cde0964 Gerrit-Change-Number: 815934 Gerrit-PatchSet: 1 Gerrit-Owner: L10n-bot <l10n-bot(a)translatewiki.net> Gerrit-Reviewer: Abijeet Patro <apatro(a)wikimedia.org> Gerrit-Reviewer: L10n-bot <l10n-bot(a)translatewiki.net> Gerrit-Reviewer: jenkins-bot Gerrit-MessageType: merged

1 0

[Gerrit] ...core[master]: [tests] Update ui_tests due to changed behaviour of Python 3.11a
by jenkins-bot (Code Review) 21 Jul '22

21 Jul '22

jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/815898 ) Change subject: [tests] Update ui_tests due to changed behaviour of Python 3.11a ...................................................................... [tests] Update ui_tests due to changed behaviour of Python 3.11a Change-Id: I2d9ff415aef75d3c2bb05286fc331d35b6edfbee --- M tests/ui_tests.py 1 file changed, 2 insertions(+), 6 deletions(-) Approvals: Xqt: Looks good to me, approved jenkins-bot: Verified diff --git a/tests/ui_tests.py b/tests/ui_tests.py index 57654eb..cf750fa 100755 --- a/tests/ui_tests.py +++ b/tests/ui_tests.py @@ -23,7 +23,6 @@ VERBOSE, WARNING, ) -from pywikibot.tools import PYTHON_VERSION from pywikibot.userinterfaces import ( terminal_interface_base, terminal_interface_unix, @@ -185,12 +184,9 @@ " raise TestExceptionError('Testing Exception')") end_str = ': Testing Exception' - traceback_line = stderrlines[4 + (PYTHON_VERSION >= (3, 11))] - self.assertTrue(traceback_line.endswith(end_str), + self.assertTrue(stderrlines[-1].endswith(end_str), '\n{!r} does not end with {!r}' - .format(traceback_line, end_str)) - - self.assertNotEqual(stderrlines[-1], '\n') + .format(stderrlines[-1], end_str)) class TestTerminalInput(UITestCase): -- To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/815898 To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Change-Id: I2d9ff415aef75d3c2bb05286fc331d35b6edfbee Gerrit-Change-Number: 815898 Gerrit-PatchSet: 3 Gerrit-Owner: Xqt <info(a)gno.de> Gerrit-Reviewer: Xqt <info(a)gno.de> Gerrit-Reviewer: jenkins-bot Gerrit-MessageType: merged

1 0

[Gerrit] ...core[master]: [doc] Remove rquired external programs hint
by Xqt (Code Review) 21 Jul '22

21 Jul '22

Xqt has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/815899 ) Change subject: [doc] Remove rquired external programs hint ...................................................................... [doc] Remove rquired external programs hint This is not necessary for pip install Change-Id: I3df8998fd33969ece8babd2b390eb22e6c437050 --- M README.rst 1 file changed, 0 insertions(+), 7 deletions(-) Approvals: Xqt: Verified; Looks good to me, approved diff --git a/README.rst b/README.rst index a7ac65f..77c943b 100644 --- a/README.rst +++ b/README.rst @@ -143,13 +143,6 @@ For more documentation on Pywikibot see our `docs <https://doc.wikimedia.org/pywikibot/>`_. -Required external programs ---------------------------- - -It may require the following programs to function properly: - -* `7za`: To extract 7z files - Roadmap ------- -- To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/815899 To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Change-Id: I3df8998fd33969ece8babd2b390eb22e6c437050 Gerrit-Change-Number: 815899 Gerrit-PatchSet: 1 Gerrit-Owner: Xqt <info(a)gno.de> Gerrit-Reviewer: Xqt <info(a)gno.de> Gerrit-MessageType: merged

1 0

[Gerrit] ...core[master]: [IMPR] Add -clear and -nodist options to make_dist.py
by jenkins-bot (Code Review) 21 Jul '22

21 Jul '22

jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/815894 ) Change subject: [IMPR] Add -clear and -nodist options to make_dist.py ...................................................................... [IMPR] Add -clear and -nodist options to make_dist.py Change-Id: Ic10f5e6ec3c30725241617272405efa19466d766 --- M make_dist.py 1 file changed, 36 insertions(+), 3 deletions(-) Approvals: Xqt: Looks good to me, approved jenkins-bot: Verified diff --git a/make_dist.py b/make_dist.py index 21271c3..b975f47 100644 --- a/make_dist.py +++ b/make_dist.py @@ -12,9 +12,13 @@ -remote Upload the package to pypi. This cannot be done if the Pywikibot version is a development release. +-clear Clear old dist folders + -upgrade Upgrade distribution packages pip, setuptools, wheel and twine first +-nodist Do not create a distribution. Useful to -clear or -upgrade only. + Usage:: [pwb] make_dist [options] @@ -27,6 +31,11 @@ - installs pre-releases over stable versions - also creates built distribution together with source distribution - `-upgrade` option was added + +.. versionchanged:: 7.5 + + - `clear` option was added + - `nodist` option was added """ # # (C) Pywikibot team, 2022 @@ -43,6 +52,19 @@ import setup +def clear_old_dist() -> None: + """Delete old dist folders. + + .. versionadded:: 7.5 + """ + info('Removing old dist folders... ', newline=False) + folder = Path().resolve() + shutil.rmtree(folder / 'build', ignore_errors=True) + shutil.rmtree(folder / 'dist', ignore_errors=True) + shutil.rmtree(folder / 'pywikibot.egg-info', ignore_errors=True) + info('done') + + def copy_files() -> None: """Copy code entry point and i18n files to pywikibot.scripts folder. @@ -98,25 +120,36 @@ local = '-local' in sys.argv remote = '-remote' in sys.argv + clear = '-clear' in sys.argv upgrade = '-upgrade' in sys.argv + nodist = '-nodist' in sys.argv + + if nodist: + local, remote = False, False if remote and 'dev' in __version__: warning('Distribution must not be a developmental release to upload.') remote = False sys.argv = [sys.argv[0], 'sdist', 'bdist_wheel'] - return local, remote, upgrade + return local, remote, clear, upgrade, nodist def main() -> None: """Script entry point.""" - local, remote, upgrade = handle_args() + local, remote, clear, upgrade, nodist = handle_args() - copy_files() if upgrade: subprocess.run('python -m pip install --upgrade pip') subprocess.run('pip install --upgrade setuptools wheel twine ') + if clear: + clear_old_dist() + + if nodist: + return + + copy_files() try: setup.main() # create a new package except SystemExit as e: -- To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/815894 To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Change-Id: Ic10f5e6ec3c30725241617272405efa19466d766 Gerrit-Change-Number: 815894 Gerrit-PatchSet: 2 Gerrit-Owner: Xqt <info(a)gno.de> Gerrit-Reviewer: Xqt <info(a)gno.de> Gerrit-Reviewer: jenkins-bot Gerrit-MessageType: merged

1 0

[Gerrit] ...core[master]: [tests] Skip flow_edit_tests on TimeoutError
by jenkins-bot (Code Review) 21 Jul '22

21 Jul '22

jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/815893 ) Change subject: [tests] Skip flow_edit_tests on TimeoutError ...................................................................... [tests] Skip flow_edit_tests on TimeoutError Change-Id: I7ccba690542bd53539f3ae4e4d48e48a39ad0e13 --- M tests/flow_edit_tests.py 1 file changed, 10 insertions(+), 5 deletions(-) Approvals: Xqt: Looks good to me, approved jenkins-bot: Verified diff --git a/tests/flow_edit_tests.py b/tests/flow_edit_tests.py index e8b0644..20e797e 100755 --- a/tests/flow_edit_tests.py +++ b/tests/flow_edit_tests.py @@ -8,9 +8,10 @@ import unittest from contextlib import contextmanager, suppress -from pywikibot.exceptions import LockedPageError +from pywikibot.exceptions import LockedPageError, TimeoutError from pywikibot.flow import Board, Post, Topic from tests.aspects import TestCase +from tests.utils import skipping MODERATION_REASON = 'Pywikibot test' @@ -60,7 +61,8 @@ # Setup content = 'I am a reply to the topic. Replying works!' topic = Topic(self.site, self._topic_title) - old_replies = topic.replies(force=True)[:] + with skipping(TimeoutError): + old_replies = topic.replies(force=True)[:] # Reply reply_post = topic.reply(content, 'wikitext') # Test content @@ -79,7 +81,8 @@ content = ("I am a reply to the topic's root post. " 'Replying still works!') topic = Topic(self.site, self._topic_title) - topic_root = topic.root + with skipping(TimeoutError): + topic_root = topic.root old_replies = topic_root.replies(force=True)[:] # Reply reply_post = topic_root.reply(content, 'wikitext') @@ -99,7 +102,8 @@ content = 'I am a nested reply to a regular post. Still going strong!' topic = Topic(self.site, self._topic_title) root_post = Post(topic, 'stf5bamzx32rj1gt') - old_replies = root_post.replies(force=True)[:] + with skipping(TimeoutError): + old_replies = root_post.replies(force=True)[:] # Reply reply_post = root_post.reply(content, 'wikitext') # Test content @@ -119,7 +123,8 @@ second_content = ('I am a nested reply. This conversation is ' 'getting pretty good!') topic = Topic(self.site, self._topic_title) - topic_root = topic.root + with skipping(TimeoutError): + topic_root = topic.root # First reply old_root_replies = topic_root.replies(force=True)[:] first_reply_post = topic_root.reply(first_content, 'wikitext') -- To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/815893 To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Change-Id: I7ccba690542bd53539f3ae4e4d48e48a39ad0e13 Gerrit-Change-Number: 815893 Gerrit-PatchSet: 1 Gerrit-Owner: Xqt <info(a)gno.de> Gerrit-Reviewer: Xqt <info(a)gno.de> Gerrit-Reviewer: jenkins-bot Gerrit-MessageType: merged

1 0

[Gerrit] ...i18n[master]: build: Updating grunt-banana-checker to 0.10.0
by jenkins-bot (Code Review) 21 Jul '22

21 Jul '22

jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/i18n/+/815821 ) Change subject: build: Updating grunt-banana-checker to 0.10.0 ...................................................................... build: Updating grunt-banana-checker to 0.10.0 Change-Id: If071aab1753fd08e998795c2f6901a868c9cb0b1 --- M package-lock.json M package.json 2 files changed, 8 insertions(+), 8 deletions(-) Approvals: Libraryupgrader: Checked jenkins-bot: Verified Xqt: Looks good to me, approved diff --git a/package-lock.json b/package-lock.json index 62b69dc..822fe7c 100644 --- a/package-lock.json +++ b/package-lock.json @@ -6,7 +6,7 @@ "": { "devDependencies": { "grunt": "1.5.3", - "grunt-banana-checker": "0.9.0" + "grunt-banana-checker": "0.10.0" } }, "node_modules/abbrev": { @@ -388,9 +388,9 @@ } }, "node_modules/grunt-banana-checker": { - "version": "0.9.0", - "resolved": "https://registry.npmjs.org/grunt-banana-checker/-/grunt-banana-checker-0.9.…", - "integrity": "sha512-SqPiB6OazWqR8USL0NymtuT5Br3mD9WBBsM1rHC/3wIi2SrZNM6/+j9CIeuEM5oCn+AtO2Y0+rzzFyOdC9afAg==", + "version": "0.10.0", + "resolved": "https://registry.npmjs.org/grunt-banana-checker/-/grunt-banana-checker-0.10…", + "integrity": "sha512-Sx+P3zWjn4YmBCqzidnCEkYrACe1SLTIT8kKC6C3f21Hu6sm17U/V+re343cuK3U+iGZv15ux6bY+69buIbYrA==", "dev": true, "bin": { "banana-checker": "src/cli.js" @@ -1444,9 +1444,9 @@ } }, "grunt-banana-checker": { - "version": "0.9.0", - "resolved": "https://registry.npmjs.org/grunt-banana-checker/-/grunt-banana-checker-0.9.…", - "integrity": "sha512-SqPiB6OazWqR8USL0NymtuT5Br3mD9WBBsM1rHC/3wIi2SrZNM6/+j9CIeuEM5oCn+AtO2Y0+rzzFyOdC9afAg==", + "version": "0.10.0", + "resolved": "https://registry.npmjs.org/grunt-banana-checker/-/grunt-banana-checker-0.10…", + "integrity": "sha512-Sx+P3zWjn4YmBCqzidnCEkYrACe1SLTIT8kKC6C3f21Hu6sm17U/V+re343cuK3U+iGZv15ux6bY+69buIbYrA==", "dev": true }, "grunt-cli": { diff --git a/package.json b/package.json index ecfd55e..3e83215 100644 --- a/package.json +++ b/package.json @@ -5,6 +5,6 @@ }, "devDependencies": { "grunt": "1.5.3", - "grunt-banana-checker": "0.9.0" + "grunt-banana-checker": "0.10.0" } } -- To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/i18n/+/815821 To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings Gerrit-Project: pywikibot/i18n Gerrit-Branch: master Gerrit-Change-Id: If071aab1753fd08e998795c2f6901a868c9cb0b1 Gerrit-Change-Number: 815821 Gerrit-PatchSet: 1 Gerrit-Owner: Libraryupgrader <tools.libraryupgrader(a)tools.wmflabs.org> Gerrit-Reviewer: Libraryupgrader <tools.libraryupgrader(a)tools.wmflabs.org> Gerrit-Reviewer: Xqt <info(a)gno.de> Gerrit-Reviewer: Zoranzoki21 <zorandori4444(a)gmail.com> Gerrit-Reviewer: jenkins-bot Gerrit-MessageType: merged

1 0

[Gerrit] ...core[master]: [doc] update ROADMAP.rst and CHANGELOG.rst
by Xqt (Code Review) 20 Jul '22

20 Jul '22

Xqt has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/815682 ) Change subject: [doc] update ROADMAP.rst and CHANGELOG.rst ...................................................................... [doc] update ROADMAP.rst and CHANGELOG.rst Change-Id: Iacdebe2381bf77c06c10734ebcc2ea929fa7bd6e --- M ROADMAP.rst M scripts/CHANGELOG.rst 2 files changed, 8 insertions(+), 1 deletion(-) Approvals: jenkins-bot: Verified Xqt: Looks good to me, approved diff --git a/ROADMAP.rst b/ROADMAP.rst index ca3bb1b..405088a 100644 --- a/ROADMAP.rst +++ b/ROADMAP.rst @@ -1,8 +1,12 @@ Current release 7.5.0 ^^^^^^^^^^^^^^^^^^^^^ +* Add support for blkwiki (:phab:`T310875`) +* L10N Updates +* Fix duplicate source detection in :meth:`pywikibot.WikidataBot.user_add_claim_unless_exists` +* :mod:`pywikibot.textlib`.tzoneFixedOffset class was renamed to :class:`pywikibot.time.TZoneFixedOffset` * Wrapper method :meth:`parsevalue()<pywikibot.site._datasite.DataSite.parsevalue>` - around wbparsevalue was added (:phab:`T112140`) + around wbparsevalue was added (:phab:`T112140`, :phab:`T312755`) * L10N updates * Fix cp encodings in :func:`get_charset_from_content_type() <pywikibot.comms.http.get_charset_from_content_type>` (:phab:`T312230`) @@ -21,6 +25,7 @@ Deprecations ^^^^^^^^^^^^ +* 7.5.0: :mod:`pywikibot.textlib`.tzoneFixedOffset class will be removed in favour of :class:`pywikibot.time.TZoneFixedOffset` * 7.4.0: `FilePage.usingPages()` was renamed to :meth:`using_pages()<pywikibot.FilePage.using_pages>` * 7.2.0: ``tb`` parameter of :func:`exception()<pywikibot.exception>` function was renamed to ``exc_info`` * 7.2.0: XMLDumpOldPageGenerator is deprecated in favour of a `content` parameter of `XMLDumpPageGenerator` (:phab:`T306134`) diff --git a/scripts/CHANGELOG.rst b/scripts/CHANGELOG.rst index 88b4b02..4800e20 100644 --- a/scripts/CHANGELOG.rst +++ b/scripts/CHANGELOG.rst @@ -9,6 +9,8 @@ harvest_template ~~~~~~~~~~~~~~~~ +* Support harvesting time values (:phab:`T66503`) +* Do not rely on self.current_page.site * Add ``-inverse`` option for inverse claims (:phab:`T173238`) * Only follow redirects in harvest_template.py if no wikibase item exists (:phab:`T311883`) -- To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/815682 To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Change-Id: Iacdebe2381bf77c06c10734ebcc2ea929fa7bd6e Gerrit-Change-Number: 815682 Gerrit-PatchSet: 1 Gerrit-Owner: Xqt <info(a)gno.de> Gerrit-Reviewer: D3r1ck01 <xsavitar.wiki(a)aol.com> Gerrit-Reviewer: Xqt <info(a)gno.de> Gerrit-Reviewer: jenkins-bot Gerrit-MessageType: merged

1 0

[Gerrit] ...core[master]: [FEAT] Support harvesting time values
by jenkins-bot (Code Review) 20 Jul '22

20 Jul '22

jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/814822 ) Change subject: [FEAT] Support harvesting time values ...................................................................... [FEAT] Support harvesting time values Parse the values using wbparsevalue API calls. Include special handling for dates formatted using wikilinks. Bug: T66503 Change-Id: I5f059edddfe276fbad16dbbc7bf967018516bb70 --- M pywikibot/site/_datasite.py M scripts/harvest_template.py M tests/harvest_template_tests.py 3 files changed, 100 insertions(+), 7 deletions(-) Approvals: Xqt: Looks good to me, approved jenkins-bot: Verified diff --git a/pywikibot/site/_datasite.py b/pywikibot/site/_datasite.py index 2966300..dbe6ee1 100644 --- a/pywikibot/site/_datasite.py +++ b/pywikibot/site/_datasite.py @@ -712,6 +712,7 @@ def parsevalue(self, datatype: str, values: List[str], options: Optional[Dict[str, Any]] = None, + language: Optional[str] = None, validate: bool = False) -> List[Any]: """ Send data values to the wikibase parser for interpretation. @@ -725,19 +726,21 @@ :param values: list of values to be parsed :param options: any additional options for wikibase parser (for time, 'precision' should be specified) + :param language: code of the language to parse the value in :param validate: whether parser should provide data validation as well as parsing :return: list of parsed values :raises ValueError: parsing failed due to some invalid input values """ params = { + 'action': 'wbparsevalue', 'datatype': datatype, 'values': values, 'options': json.dumps(options or {}), 'validate': validate, - 'uselang': 'en', + 'uselang': language or 'en', } - req = self.simple_request(action='wbparsevalue', **params) + req = self.simple_request(**params) try: data = req.submit() except APIError as e: diff --git a/scripts/harvest_template.py b/scripts/harvest_template.py index 6dd4677..48f6dc9 100755 --- a/scripts/harvest_template.py +++ b/scripts/harvest_template.py @@ -101,6 +101,7 @@ # # Distributed under the terms of MIT License. # +import re import signal import sys from typing import Any, Iterator, Optional @@ -108,9 +109,11 @@ import pywikibot from pywikibot import pagegenerators as pg from pywikibot import textlib +from pywikibot import WbTime from pywikibot.backports import List, Tuple from pywikibot.bot import ConfigParserBot, OptionHandler, WikidataBot from pywikibot.exceptions import ( + APIError, InvalidPageError, InvalidTitleError, NoPageError, @@ -192,13 +195,15 @@ self.fields[key] = value else: # backwards compatibility self.fields[key] = (value, PropertyOptionHandler()) - self.cacheSources() - # TODO: Make it a list including the redirects to the template - template_title = template_title.replace('_', ' ') - self.templateTitles = self.getTemplateSynonyms(template_title) + self.template_title = template_title.replace('_', ' ') self.linkR = textlib.compileLinkR() self.create_missing_item = self.opt.create + def setup(self): + """Cache some static data from wikis.""" + self.cacheSources() + self.templateTitles = self.getTemplateSynonyms(self.template_title) + def getTemplateSynonyms(self, title) -> List[str]: """Fetch redirects of the title, so we can check against them.""" temp = pywikibot.Page(pywikibot.Site(), title, ns=10) @@ -369,6 +374,7 @@ .. versionadded:: 7.5 """ + value = value.replace('{{!}}', '|') prop, options = self.fields[field] matched = False @@ -393,6 +399,61 @@ if linked_item: yield linked_item + def handle_time(self, value: str, + site: pywikibot.site.BaseSite, + *args) -> Iterator[WbTime]: + """Handle 'time' claim type. + + .. versionadded:: 7.5 + """ + value = value.replace('{{!}}', '|') + value = value.replace(' ', ' ') + value = re.sub('</?sup>', '', value) + + # Some wikis format dates using wikilinks. We construct + # all possible texts, e.g., "[[A|B]] of [[C]]" becomes + # "A of C" and "B of C", and parse them using the API. + # If the result is same for all the values, we import + # the value. + to_parse = {''} + prev_end = 0 + for match in pywikibot.link_regex.finditer(value): + start, end = match.span() + since_prev_match = value[prev_end:start] + + title = match.group('title').strip() + text = match.group(2) + if text: + text = text[1:].strip() # remove '|' + + new_to_parse = set() + for fragment in to_parse: + fragment += since_prev_match + new_to_parse.add(fragment + title) + if text: + new_to_parse.add(fragment + text) + + to_parse = new_to_parse + prev_end = end + + rest = value[prev_end:] + to_parse = [text + rest for text in to_parse] + + try: + result = self.repo.parsevalue('time', to_parse, language=site.lang) + except (APIError, ValueError): + return + + out = None + for data in result: + if out is None: + out = data + elif out != data: + pywikibot.output('Found ambiguous date: "{}"'.format(value)) + return + + yield WbTime.fromWikibase(out, self.repo) + @staticmethod def handle_string(value, *args) -> Iterator[str]: """Handle 'string' and 'external-id' claim type. @@ -496,6 +557,10 @@ 'Please specify either -template or -transcludes argument') return + if not fields: + pywikibot.error('No template parameters to harvest specified.') + return + if not gen.gens: gen.handle_arg('-transcludes:' + template_title) generator = gen.getCombinedGenerator(preload=True) diff --git a/tests/harvest_template_tests.py b/tests/harvest_template_tests.py index 3909727..742b6f2 100644 --- a/tests/harvest_template_tests.py +++ b/tests/harvest_template_tests.py @@ -8,7 +8,7 @@ import unittest from contextlib import suppress -from pywikibot import ItemPage +from pywikibot import ItemPage, WbTime from scripts.harvest_template import HarvestRobot from tests.aspects import ScriptMainTestCase @@ -48,6 +48,31 @@ self.assertIsInstance(target, ItemPage) self.assertEqual(target.title(), item) + def test_handle_time(self): + """Test handle_time method.""" + bot = HarvestRobot('Foo', {}, site=self.site) + + day = WbTime(2022, 7, 18, precision=11, site=bot.repo) + tests = [ + ('Foo', None), + ('2022', WbTime(2022, 0, 0, precision=9, site=bot.repo)), + ('2022-07-18', day), + ('18. červenec 2022', day), + ('18. července [[2021|2022]]', None), + ('[[18. červenec]] 2022', day), + ('[[18. červenec|18. července]] [[2022]]', day), + ('[[17. červenec|18. července]] [[2022]]', None), + ('44 př. n. l.', + WbTime(-44, 0, 0, precision=9, + calendarmodel='http://www.wikidata.org/entity/Q1985786', + site=bot.repo)), + ] + for text, time in tests: + with self.subTest(text=text, time=time): + gen = bot.handle_time(text, self.site) + out = next(gen, None) + self.assertEqual(time, out) + if __name__ == '__main__': # pragma: no cover with suppress(SystemExit): -- To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/814822 To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Change-Id: I5f059edddfe276fbad16dbbc7bf967018516bb70 Gerrit-Change-Number: 814822 Gerrit-PatchSet: 9 Gerrit-Owner: Matěj Suchánek <matejsuchanek97(a)gmail.com> Gerrit-Reviewer: Xqt <info(a)gno.de> Gerrit-Reviewer: jenkins-bot Gerrit-MessageType: merged

1 0

← Newer
1
2
3
4
5
6
7
8
9
...
13
Older →

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

Pywikibot-commits July 2022