Xqt has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/740644 )
Change subject: [tests] Temporary add commonscat to unrunnable_script_set
......................................................................
[tests] Temporary add commonscat to unrunnable_script_set
This patch is for debug purpose only
Bug: T296204
Change-Id: I193dc74434f5a2f622c642effa06f73695dfa741
---
M tests/script_tests.py
1 file changed, 12 insertions(+), 12 deletions(-)
Approvals:
Xqt: Verified; Looks good to me, approved
diff --git a/tests/script_tests.py b/tests/script_tests.py
index f746518..8c29872 100644
--- a/tests/script_tests.py
+++ b/tests/script_tests.py
@@ -42,7 +42,7 @@
if not check_script_deps(name)}
# scripts which cannot be tested
-unrunnable_script_set = set()
+unrunnable_script_set = {'commonscat'}
def list_scripts(path, exclude=None):
@@ -152,17 +152,17 @@
test_list = ['tests.script_tests.TestScriptHelp.' + name
for name in tests]
- tests = (['test__login']
- + ['test_' + name
- for name in sorted(script_list)
- if name != 'login'
- and name not in failed_dep_script_set
- and name not in unrunnable_script_set
- and (enable_autorun_tests or name not in auto_run_script_list)
- ])
-
- test_list += ['tests.script_tests.TestScriptSimulate.' + name
- for name in tests]
+## tests = (['test__login']
+## + ['test_' + name
+## for name in sorted(script_list)
+## if name != 'login'
+## and name not in failed_dep_script_set
+## and name not in unrunnable_script_set
+## and (enable_autorun_tests or name not in auto_run_script_list)
+## ])
+##
+## test_list += ['tests.script_tests.TestScriptSimulate.' + name
+## for name in tests]
tests = loader.loadTestsFromNames(test_list)
suite = unittest.TestSuite()
--
To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/740644
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Change-Id: I193dc74434f5a2f622c642effa06f73695dfa741
Gerrit-Change-Number: 740644
Gerrit-PatchSet: 1
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot
Gerrit-MessageType: merged
jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/740340 )
Change subject: [bugfix] Don't call check_modules() as a site package
......................................................................
[bugfix] Don't call check_modules() as a site package
Change-Id: I6b09fd146f1a06d5b4c8b31b1a39bc100a1b7042
---
M pwb.py
1 file changed, 1 insertion(+), 1 deletion(-)
Approvals:
Xqt: Looks good to me, approved
jenkins-bot: Verified
diff --git a/pwb.py b/pwb.py
index 321def5..8c5da9f 100755
--- a/pwb.py
+++ b/pwb.py
@@ -439,7 +439,7 @@
help_option = any(arg.startswith('-help:') or arg == '-help'
for arg in script_args)
- if check_modules(filename) or help_option:
+ if site_package or check_modules(filename) or help_option:
run_python_file(filename,
[filename] + script_args,
[Path(filename).stem] + argvu[1:],
--
To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/740340
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Change-Id: I6b09fd146f1a06d5b4c8b31b1a39bc100a1b7042
Gerrit-Change-Number: 740340
Gerrit-PatchSet: 1
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot
Gerrit-MessageType: merged
jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/740339 )
Change subject: [doc] Update ROADMAP.rst and CHANGELOG.md
......................................................................
[doc] Update ROADMAP.rst and CHANGELOG.md
Change-Id: Ib2eafbc075a21b573172b8091f73d39db20f0bf7
---
M ROADMAP.rst
M scripts/CHANGELOG.md
2 files changed, 6 insertions(+), 0 deletions(-)
Approvals:
Xqt: Looks good to me, approved
jenkins-bot: Verified
diff --git a/ROADMAP.rst b/ROADMAP.rst
index 93acb6c..9d32716 100644
--- a/ROADMAP.rst
+++ b/ROADMAP.rst
@@ -4,6 +4,9 @@
Improvements and Bugfixes
-------------------------
+* Make pwb wrapper script a pywikibot entry point for scripts (T139143, T270480)
+* Enable -version and --version with pwb wrapper or code entry point (T101828)
+* Retry upload if 'copyuploadbaddomain' API error occurs (T294825)
* Add `title_delimiter_and_aliases` attribute to family files to support WikiHow family (T294761)
* Only handle query limit if query module is limited (T294836)
* BaseBot has a public collections.Counter for reading, writing and skipping a page
@@ -29,6 +32,8 @@
Code cleanups
-------------
+* Raise a TypeError if BaseBot.init_page return None
+* Remove private upload parameters in UploadRobot.upload_file(), FilePage.upload() and APISite.upload() methods
* Raise an Error exception if 'titles' is still used as where parameter in Site.search()
* Deprecated version.get_module_version() function was removed
* Deprecated setOptions/getOptions OptionHandler methods were removed
diff --git a/scripts/CHANGELOG.md b/scripts/CHANGELOG.md
index b87f57f..b55e5c1 100644
--- a/scripts/CHANGELOG.md
+++ b/scripts/CHANGELOG.md
@@ -43,6 +43,7 @@
* Deprecated "acceptall" and "addedCat" parameters were replaced by "always" and "addcat"
### revertbot
+* Add support for translated dates/times (T102174)
* Deprecated "max" parameter was replaced by "total"
### solve_disambiguation
--
To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/740339
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Change-Id: Ib2eafbc075a21b573172b8091f73d39db20f0bf7
Gerrit-Change-Number: 740339
Gerrit-PatchSet: 1
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: D3r1ck01 <xsavitar.wiki(a)aol.com>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot
Gerrit-MessageType: merged
jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/738576 )
Change subject: [IMP] Make a pywikibot entry point for scripts
......................................................................
[IMP] Make a pywikibot entry point for scripts
- add pwb.py wrapper script to the site-package. This enables scripts to be
invoked whose paths were added to the config.user_script_paths inside
user-config.py. Further implementations can provide entry points of
foreign pypi packages. The pwb.py script will be copied to the new
pywikibot.scripts folder when creating a dist.
- Add a new pywikibot.scripts folder to the framework
- move version.py script from scripts folder to pywikibot.scripts
to enable version call within site-package
- add a code entry point in setup that pwb.py wrapper script can be
always called with "pwb" within every folder
- update documentation in several files
pwb.py
- rename main() function to execute
- move check_modules call to main() function
- add run() function as entry point for site-package and don't check
package dependencies for a site-package because it is already done
during installation
- use base_dir instead of _pwb_dir for testpath path
- add _pwb_dir to script_paths if pywikibot is a site-package,
otherwise add pywikibot.scripts
Bug: T270480
Bug: T139143
Change-Id: I51160dc4df95b59bac98acf9f4e170232c4923a2
---
M README.rst
M docs/api_ref/pywikibot.config.rst
M docs/scripts/scripts.rst
M docs/utilities/index.rst
M pwb.py
M pywikibot/CONTENT.rst
M pywikibot/DIRECTORIES.rst
M pywikibot/config.py
A pywikibot/scripts/__init__.py
R pywikibot/scripts/version.py
M scripts/README.rst
M setup.py
12 files changed, 107 insertions(+), 26 deletions(-)
Approvals:
Xqt: Looks good to me, approved
jenkins-bot: Verified
diff --git a/README.rst b/README.rst
index 4a02555..eebbf97 100644
--- a/README.rst
+++ b/README.rst
@@ -50,6 +50,7 @@
pip install -U setuptools
pip install pywikibot
+ pwb <scriptname>
In addition a MediaWiki markup parser is required. Please install one of them:
diff --git a/docs/api_ref/pywikibot.config.rst b/docs/api_ref/pywikibot.config.rst
index 290636c..c31852a 100644
--- a/docs/api_ref/pywikibot.config.rst
+++ b/docs/api_ref/pywikibot.config.rst
@@ -39,6 +39,8 @@
:start-at: # ############# LOGFILE SETTINGS
:end-before: # #############
+.. _external-script-path-settings:
+
External Script Path Settings
+++++++++++++++++++++++++++++
diff --git a/docs/scripts/scripts.rst b/docs/scripts/scripts.rst
index 6a9371a..86ec7d3 100644
--- a/docs/scripts/scripts.rst
+++ b/docs/scripts/scripts.rst
@@ -252,11 +252,6 @@
.. automodule:: scripts.upload
-version script
---------------
-
-.. automodule:: scripts.version
-
watchlist script
----------------
diff --git a/docs/utilities/index.rst b/docs/utilities/index.rst
index 96131fa..9965397 100644
--- a/docs/utilities/index.rst
+++ b/docs/utilities/index.rst
@@ -6,6 +6,11 @@
.. automodule:: pwb
+version script
+--------------
+
+.. automodule:: pywikibot.scripts.version
+
generate\_family\_file script
-----------------------------
diff --git a/pwb.py b/pwb.py
index b8ddd82..3f23bc3 100755
--- a/pwb.py
+++ b/pwb.py
@@ -2,10 +2,27 @@
# -*- coding: utf-8 -*-
"""Wrapper script to invoke pywikibot-based scripts.
+This wrapper script invokes script by its name in this search order:
+
+1. Scripts listed in `user_script_paths` list inside your `user-config.py`
+ settings file in the given order. Refer
+ :ref:`External Script Path Settings<external-script-path-settings>`.
+2. User scripts residing in `scripts/userscripts` (directory mode only).
+3. Scripts residing in `scripts` folder (directory mode only).
+4. Maintenance scripts residing in `scripts/maintenance` (directory mode only).
+5. Framework scripts residing in `pywikibot/scripts`.
+
+This wrapper script is able to invoke scripts even the script name is
+missspelled. In directory mode it also checks package dependencies.
+
Run scripts with pywikibot in directory mode using::
python pwb.py <pwb options> <name_of_script> <options>
+or run scripts with pywikibot installed as a site package using::
+
+ pwb <pwb options> <name_of_script> <options>
+
This wrapper script uses the package directory to store all user files,
will fix up search paths so the package does not need to be installed, etc.
@@ -13,6 +30,9 @@
for tests to set the default site (see T216825)::
python pwb.py -lang:de bot_tests -v
+
+.. versionchanged:: 7.0
+ pwb wrapper was added to the Python site package lib
"""
# (C) Pywikibot team, 2012-2021
#
@@ -32,6 +52,7 @@
pwb = None
+site_package = False
def check_pwb_versions(package):
@@ -235,12 +256,6 @@
return not missing_requirements
-try:
- if not check_modules():
- raise RuntimeError('') # no further output needed
-except RuntimeError as e: # setup.py may also raise RuntimeError
- sys.exit(e)
-
from pathlib import Path # noqa: E402
@@ -333,9 +348,15 @@
"""Search for the filename in the given script paths."""
from pywikibot import config
- script_paths = ['scripts.userscripts',
- 'scripts',
- 'scripts.maintenance']
+ if site_package:
+ script_paths = [_pwb_dir]
+ else:
+ script_paths = [
+ 'scripts.userscripts',
+ 'scripts',
+ 'scripts.maintenance',
+ 'pywikibot.scripts',
+ ]
if config.user_script_paths:
if isinstance(config.user_script_paths, list):
@@ -349,7 +370,7 @@
for file_package in script_paths:
package = file_package.split('.')
paths = package + [filename]
- testpath = os.path.join(_pwb_dir, *paths)
+ testpath = os.path.join(config.base_dir, *paths)
if os.path.exists(testpath):
filename = testpath
break
@@ -359,8 +380,12 @@
return filename
-def main():
- """Command line entry point."""
+def execute():
+ """Parse arguments, extract filename and run the script.
+
+ .. versionadded:: 7.0
+ renamed from :func:`main`
+ """
global filename
if global_args: # don't use sys.argv
@@ -419,6 +444,32 @@
return True
-if __name__ == '__main__':
- if not main():
+def main():
+ """Script entry point. Print doc if necessary.
+
+ .. versionchanged:: 7.0
+ previous implementation was renamed to :func:`execute`
+ """
+ try:
+ if not check_modules():
+ raise RuntimeError('') # no further output needed
+ except RuntimeError as e: # setup.py may also raise RuntimeError
+ sys.exit(e)
+
+ if not execute():
print(__doc__)
+
+
+def run():
+ """Site package entry point. Print doc if necessary.
+
+ .. versionadded:: 7.0
+ """
+ global site_package
+ site_package = True
+ if not execute():
+ print(__doc__)
+
+
+if __name__ == '__main__':
+ main()
diff --git a/pywikibot/CONTENT.rst b/pywikibot/CONTENT.rst
index e4398a8..deb5c3e 100644
--- a/pywikibot/CONTENT.rst
+++ b/pywikibot/CONTENT.rst
@@ -113,6 +113,16 @@
+----------------------------+------------------------------------------------------+
+ | scripts | Framework helper scripts and entry points |
+ +============================+======================================================+
+ | pwb.py | code entry wrapper script (site-package only) |
+ +----------------------------+------------------------------------------------------+
+ | version.py | Outputs Pywikibot's revision number, Python's |
+ | | version and OS used. |
+ +----------------------------+------------------------------------------------------+
+
+
+ +----------------------------+------------------------------------------------------+
| site | Module with classes for MediaWiki sites |
+============================+======================================================+
| __init__.py | Objects representing MediaWiki sites (wikis) |
diff --git a/pywikibot/DIRECTORIES.rst b/pywikibot/DIRECTORIES.rst
index d2b3c1b..0e930a6 100644
--- a/pywikibot/DIRECTORIES.rst
+++ b/pywikibot/DIRECTORIES.rst
@@ -14,6 +14,8 @@
+---------------------------+------------------------------------------------------+
| pywikibot/page | Module with classes for MediaWiki page content |
+---------------------------+------------------------------------------------------+
+ | pywikibot/scripts | Framework helper scripts |
+ +---------------------------+------------------------------------------------------+
| pywikibot/site | Module with classes for MediaWiki sites |
+---------------------------+------------------------------------------------------+
| pywikibot/specialbots | Module containing special bots reusable by scripts |
diff --git a/pywikibot/config.py b/pywikibot/config.py
index d573ada..beff1fa 100644
--- a/pywikibot/config.py
+++ b/pywikibot/config.py
@@ -572,15 +572,18 @@
# ############# EXTERNAL SCRIPT PATH SETTINGS ##############
# Set your own script path to lookup for your script files.
#
-# Your private script path must be located inside the
-# framework folder, subfolders must be delimited by '.'.
-# every folder must contain an (empty) __init__.py file.
+# Your private script path is relative to your base directory.
+# Subfolders must be delimited by '.'. every folder must contain
+# an (empty) __init__.py file.
#
# The search order is
# 1. user_script_paths in the given order
# 2. scripts/userscripts
# 3. scripts
# 4. scripts/maintenance
+# 5. pywikibot/scripts
+#
+# 2. - 4. are available in directory mode only
#
# sample:
# user_script_paths = ['scripts.myscripts']
diff --git a/pywikibot/scripts/__init__.py b/pywikibot/scripts/__init__.py
new file mode 100644
index 0000000..5a69b88
--- /dev/null
+++ b/pywikibot/scripts/__init__.py
@@ -0,0 +1,6 @@
+"""Folder which holds framework scripts.
+
+When uploading pywikibot to pypi the pwb.py (wrapper script) is copied here.
+
+.. versionadded:: 7.0
+"""
diff --git a/scripts/version.py b/pywikibot/scripts/version.py
similarity index 96%
rename from scripts/version.py
rename to pywikibot/scripts/version.py
index 9a2cf3d..77faad0 100755
--- a/scripts/version.py
+++ b/pywikibot/scripts/version.py
@@ -1,5 +1,9 @@
#!/usr/bin/python
-"""Script to determine the Pywikibot version (tag, revision and date)."""
+"""Script to determine the Pywikibot version (tag, revision and date).
+
+.. versionchanged:: 7.0
+ version script was moved to the framework scripts folder
+"""
#
# (C) Pywikibot team, 2007-2021
#
diff --git a/scripts/README.rst b/scripts/README.rst
index c61f5d9..c4bf53f 100644
--- a/scripts/README.rst
+++ b/scripts/README.rst
@@ -145,9 +145,6 @@
+------------------------+---------------------------------------------------------+
| upload.py | Upload an image to a wiki. |
+------------------------+---------------------------------------------------------+
- | version.py | Outputs Pywikibot's revision number, Python's version |
- | | and OS used. |
- +------------------------+---------------------------------------------------------+
| watchlists.py | Allows access to the account's watchlist. |
+------------------------+---------------------------------------------------------+
| weblinkchecker.py | Check if external links are still working. |
diff --git a/setup.py b/setup.py
index c11224f..9615a85 100644
--- a/setup.py
+++ b/setup.py
@@ -256,6 +256,11 @@
download_url=metadata.__download_url__,
test_suite='tests.collector',
tests_require=test_deps,
+ entry_points={
+ 'console_scripts': [
+ 'pwb = pywikibot.scripts.pwb:run',
+ ],
+ },
classifiers=[
'Development Status :: 5 - Production/Stable',
'Environment :: Console',
--
To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/738576
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Change-Id: I51160dc4df95b59bac98acf9f4e170232c4923a2
Gerrit-Change-Number: 738576
Gerrit-PatchSet: 3
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: AbdealiJK <abdealikothari(a)gmail.com>
Gerrit-Reviewer: D3r1ck01 <xsavitar.wiki(a)aol.com>
Gerrit-Reviewer: Dvorapa <dvorapa(a)seznam.cz>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: Yuvipanda <yuvipanda(a)gmail.com>
Gerrit-Reviewer: jenkins-bot
Gerrit-MessageType: merged
jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/738510 )
Change subject: [IMPR] Add support for translated dates/times
......................................................................
[IMPR] Add support for translated dates/times
This implementation just combines date, year and time from pywikibot.date
lookup. It could be more precise if MonthFormat would have a day_years_format
entry but most of the current year_formats entries are default which just
append the year to the month.
Bug: T102174
Change-Id: I85bb7ed797696d647dd39d4ca744d9bd8bc5133a
---
M scripts/revertbot.py
1 file changed, 13 insertions(+), 1 deletion(-)
Approvals:
Xqt: Looks good to me, approved
jenkins-bot: Verified
diff --git a/scripts/revertbot.py b/scripts/revertbot.py
index d7ab872..b5e63b4 100755
--- a/scripts/revertbot.py
+++ b/scripts/revertbot.py
@@ -43,8 +43,10 @@
from typing import Union
import pywikibot
+
from pywikibot import i18n
from pywikibot.bot import OptionHandler
+from pywikibot.date import format_date, formatYear
from pywikibot.exceptions import APIError, Error
from pywikibot.tools.formatter import color_format
@@ -92,6 +94,16 @@
"""Callback function."""
return 'top' in item
+ def local_timestamp(self, ts) -> str:
+ """Convert Timestamp to a localized timestamp string.
+
+ .. versionadded:: 7.0
+ """
+ year = formatYear(self.site.lang, ts.year)
+ date = format_date(ts.month, ts.day, self.site)
+ *_, time = str(ts).strip('Z').partition('T')
+ return ' '.join((date, year, time))
+
def revert(self, item) -> Union[str, bool]:
"""Revert a single item."""
page = pywikibot.Page(self.site, item['title'])
@@ -110,7 +122,7 @@
self.site, 'revertbot-revert',
{'revid': rev.revid,
'author': rev.user,
- 'timestamp': rev.timestamp})
+ 'timestamp': self.local_timestamp(rev.timestamp)})
if self.opt.comment:
comment += ': ' + self.opt.comment
--
To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/738510
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Change-Id: I85bb7ed797696d647dd39d4ca744d9bd8bc5133a
Gerrit-Change-Number: 738510
Gerrit-PatchSet: 2
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: D3r1ck01 <xsavitar.wiki(a)aol.com>
Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot
Gerrit-MessageType: merged
jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/738603 )
Change subject: [doc] fix doc in scripts/__init__.py
......................................................................
[doc] fix doc in scripts/__init__.py
Change-Id: I2849b53dcd04cccb1b4ed9588cd8b83476469095
---
M scripts/__init__.py
1 file changed, 2 insertions(+), 2 deletions(-)
Approvals:
Xqt: Looks good to me, approved
jenkins-bot: Verified
diff --git a/scripts/__init__.py b/scripts/__init__.py
index c99eb71..3afcca2 100644
--- a/scripts/__init__.py
+++ b/scripts/__init__.py
@@ -1,7 +1,7 @@
"""**Scripts** folder contains predefined scripts easy to use.
-Scripts are only available im Pywikibot if instaled in directory mode
-and not as side package. They can be run in command line using the pwb
+Scripts are only available with Pywikibot if installed in directory mode
+and not as site package. They can be run in command line using the pwb
wrapper script::
python pwb.py <global options> <name_of_script> <options>
--
To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/738603
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Change-Id: I2849b53dcd04cccb1b4ed9588cd8b83476469095
Gerrit-Change-Number: 738603
Gerrit-PatchSet: 1
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: D3r1ck01 <xsavitar.wiki(a)aol.com>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot
Gerrit-MessageType: merged
jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/736272 )
Change subject: [IMPR] Remove private upload parameters
......................................................................
[IMPR] Remove private upload parameters
Site.upload() method has private upload parameters for upload recursion.
- remove these parameter from Page.upload() description
- remove them from UploadRobot.upload_file() method
- reorder Site.upload() parameters and description which is possible
because keyword arguments are required for all parameters except
filepage
Change-Id: I14cddb25f02c63eabcec7c4fe7f29fdbd5de3aef
---
M pywikibot/page/__init__.py
M pywikibot/site/_apisite.py
M pywikibot/specialbots/_upload.py
3 files changed, 17 insertions(+), 30 deletions(-)
Approvals:
Xqt: Looks good to me, approved
jenkins-bot: Verified
diff --git a/pywikibot/page/__init__.py b/pywikibot/page/__init__.py
index ee2a538..6a199aa 100644
--- a/pywikibot/page/__init__.py
+++ b/pywikibot/page/__init__.py
@@ -2380,20 +2380,6 @@
will only upload in chunks, if the chunk size is positive but lower
than the file size.
:type chunk_size: int
- :keyword _file_key: Reuses an already uploaded file using the filekey.
- If None (default) it will upload the file.
- :type _file_key: str or None
- :keyword _offset: When file_key is not None this can be an integer to
- continue a previously canceled chunked upload. If False it treats
- that as a finished upload. If True it requests the stash info from
- the server to determine the offset. By default starts at 0.
- :type _offset: int or bool
- :keyword _verify_stash: Requests the SHA1 and file size uploaded and
- compares it to the local file. Also verifies that _offset is
- matching the file size if the _offset is an int. If _offset is
- False if verifies that the file size match with the local file. If
- None it'll verifies the stash when a file key and offset is given.
- :type _verify_stash: bool or None
:keyword report_success: If the upload was successful it'll print a
success message and if ignore_warnings is set to False it'll
raise an UploadError if a warning occurred. If it's
diff --git a/pywikibot/site/_apisite.py b/pywikibot/site/_apisite.py
index f5607c9..dfe5434 100644
--- a/pywikibot/site/_apisite.py
+++ b/pywikibot/site/_apisite.py
@@ -2449,10 +2449,10 @@
ignore_warnings=False,
chunk_size: int = 0,
asynchronous: bool = False,
+ report_success: Optional[bool] = None,
_file_key: Optional[str] = None,
_offset: Union[bool, int] = 0,
- _verify_stash: Optional[bool] = None,
- report_success: Optional[bool] = None) -> bool:
+ _verify_stash: Optional[bool] = None) -> bool:
"""
Upload a file to the wiki.
@@ -2492,22 +2492,25 @@
but lower than the file size.
:param asynchronous: Make potentially large file operations
asynchronous on the server side when possible.
- :param _file_key: Reuses an already uploaded file using the filekey. If
- None (default) it will upload the file.
- :param _offset: When file_key is not None this can be an integer to
- continue a previously canceled chunked upload. If False it treats
- that as a finished upload. If True it requests the stash info from
- the server to determine the offset. By default starts at 0.
- :param _verify_stash: Requests the SHA1 and file size uploaded and
- compares it to the local file. Also verifies that _offset is
- matching the file size if the _offset is an int. If _offset is
- False if verifies that the file size match with the local file. If
- None it'll verifies the stash when a file key and offset is given.
:param report_success: If the upload was successful it'll print a
success message and if ignore_warnings is set to False it'll
raise an UploadError if a warning occurred. If it's None
(default) it'll be True if ignore_warnings is a bool and False
otherwise. If it's True or None ignore_warnings must be a bool.
+ :param _file_key: Private parameter for upload recurion. Reuses
+ an already uploaded file using the filekey. If None (default)
+ it will upload the file.
+ :param _offset: Private parameter for upload recurion. When
+ file_key is not None this can be an integer to continue a
+ previously canceled chunked upload. If False it treats that
+ as a finished upload. If True it requests the stash info from
+ the server to determine the offset. By default starts at 0.
+ :param _verify_stash: Private parameter for upload recurion.
+ Requests the SHA1 and file size uploaded and compares it to
+ the local file. Also verifies that _offset is matching the
+ file size if the _offset is an int. If _offset is False if
+ verifies that the file size match with the local file. If
+ None it'll verifies the stash when a file key and offset is given.
:return: It returns True if the upload was successful and False
otherwise.
"""
diff --git a/pywikibot/specialbots/_upload.py b/pywikibot/specialbots/_upload.py
index 1aff167..5c08530 100644
--- a/pywikibot/specialbots/_upload.py
+++ b/pywikibot/specialbots/_upload.py
@@ -374,7 +374,7 @@
"""
return self.ignore_warning is True or warn_code in self.ignore_warning
- def upload_file(self, file_url, _file_key=None, _offset=0):
+ def upload_file(self, file_url):
"""
Upload the image at file_url to the target wiki.
@@ -414,8 +414,6 @@
success = imagepage.upload(file_url,
ignore_warnings=ignore_warnings,
chunk_size=self.chunk_size,
- _file_key=_file_key,
- _offset=_offset,
asynchronous=self.asynchronous,
comment=self.summary)
except APIError as error:
--
To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/736272
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Change-Id: I14cddb25f02c63eabcec7c4fe7f29fdbd5de3aef
Gerrit-Change-Number: 736272
Gerrit-PatchSet: 2
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot
Gerrit-MessageType: merged
jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/737201 )
Change subject: [doc] Update ROADMAP.rst
......................................................................
[doc] Update ROADMAP.rst
Change-Id: I0e2207b65781dedf2824806ccade17f147f3d87f
---
M ROADMAP.rst
1 file changed, 3 insertions(+), 0 deletions(-)
Approvals:
Xqt: Looks good to me, approved
jenkins-bot: Verified
diff --git a/ROADMAP.rst b/ROADMAP.rst
index 2c611c2..93acb6c 100644
--- a/ROADMAP.rst
+++ b/ROADMAP.rst
@@ -4,6 +4,8 @@
Improvements and Bugfixes
-------------------------
+* Add `title_delimiter_and_aliases` attribute to family files to support WikiHow family (T294761)
+* Only handle query limit if query module is limited (T294836)
* BaseBot has a public collections.Counter for reading, writing and skipping a page
* Upload: Retry upload if 'copyuploadbaddomain' API error occurs (T294825)
* Upload: Only set filekey/offset for files with names (T294916)
@@ -27,6 +29,7 @@
Code cleanups
-------------
+* Raise an Error exception if 'titles' is still used as where parameter in Site.search()
* Deprecated version.get_module_version() function was removed
* Deprecated setOptions/getOptions OptionHandler methods were removed
* Deprecated from_page() method of CosmeticChangesToolkit was removed
--
To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/737201
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Change-Id: I0e2207b65781dedf2824806ccade17f147f3d87f
Gerrit-Change-Number: 737201
Gerrit-PatchSet: 1
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot
Gerrit-MessageType: merged
jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/614565 )
Change subject: [bugfix] Add title_delimiter_and_alias into family files
......................................................................
[bugfix] Add title_delimiter_and_alias into family files
Titles usually are delimited by a space and the the alias is replaced
to this delimiter; e.g. "Main page" is the title with spaces as
delimiters but "Main_page" also works. Other families like wikihow has
a have setting.
- add title_delimiter_and_alias to family.py as default
- add a different title_delimiter_and_alias to wikihow_family.py
- use this settings for Page.Link and when comparing titles
- tests updated
Bug: T294761
Change-Id: Ib7858b88324376b6bbbf788893308fcf66c4d154
---
M pywikibot/families/wikihow_family.py
M pywikibot/family.py
M pywikibot/page/__init__.py
M pywikibot/site/_basesite.py
M tests/link_tests.py
5 files changed, 61 insertions(+), 12 deletions(-)
Approvals:
Xqt: Looks good to me, approved
jenkins-bot: Verified
diff --git a/pywikibot/families/wikihow_family.py b/pywikibot/families/wikihow_family.py
index 4aa93f7..38d3ab4 100644
--- a/pywikibot/families/wikihow_family.py
+++ b/pywikibot/families/wikihow_family.py
@@ -15,7 +15,7 @@
"""Family class for Wikihow Wiki.
- .. versionaddded: 3.0
+ .. versionadded:: 3.0
"""
name = 'wikihow'
@@ -25,8 +25,12 @@
'ar', 'cs', 'de', 'en', 'es', 'fr', 'hi', 'id', 'it', 'ja', 'ko', 'nl',
'pt', 'ru', 'th', 'tr', 'vi', 'zh',
)
+
removed_wikis = ['ca', 'cy', 'fa', 'he', 'pl', 'ur']
+ title_delimiter_and_aliases = '- '
+ """.. versionadded:: 7.0"""
+
@classproperty
def domains(cls):
"""List of domains used by family wikihow."""
diff --git a/pywikibot/family.py b/pywikibot/family.py
index 006c548..d5240a9 100644
--- a/pywikibot/family.py
+++ b/pywikibot/family.py
@@ -542,6 +542,21 @@
# site. This value can specify this last one with (lang, family) tuple.
shared_urlshortner_wiki = None # type: Optional[Tuple[str, str]]
+ title_delimiter_and_aliases = ' _'
+ """Titles usually are delimited by a space and the alias is replaced
+ to this delimiter; e.g. "Main page" is the title with spaces as
+ delimiters but "Main_page" also works. Other families may have
+ different settings.
+
+ .. note:: The first character is used as delimiter, the others are
+ aliases.
+
+ .. warning:: This attribute is used within ``re.sub()`` method. Use
+ escape sequence if necessary
+
+ .. versionadded:: 7.0
+ """
+
_families = {}
@staticmethod
diff --git a/pywikibot/page/__init__.py b/pywikibot/page/__init__.py
index 59d5781..56841d6 100644
--- a/pywikibot/page/__init__.py
+++ b/pywikibot/page/__init__.py
@@ -5249,6 +5249,8 @@
else:
self._anchor = None
+ self._text = self._text.strip()
+
# Convert URL-encoded characters to unicode
self._text = pywikibot.tools.chars.url2string(
self._text, encodings=self._source.encodings())
@@ -5267,9 +5269,11 @@
'{!r} contains illegal char {!r}'.format(t, '\ufffd'))
# Cleanup whitespace
+ sep = self._source.family.title_delimiter_and_aliases[0]
t = re.sub(
- '[_ \xa0\u1680\u180E\u2000-\u200A\u2028\u2029\u202F\u205F\u3000]+',
- ' ', t)
+ '[{}\xa0\u1680\u180E\u2000-\u200A\u2028\u2029\u202F\u205F\u3000]+'
+ .format(self._source.family.title_delimiter_and_aliases),
+ sep, t)
# Strip spaces at both ends
t = t.strip()
# Remove left-to-right and right-to-left markers.
diff --git a/pywikibot/site/_basesite.py b/pywikibot/site/_basesite.py
index 3a3a605..f92ae18 100644
--- a/pywikibot/site/_basesite.py
+++ b/pywikibot/site/_basesite.py
@@ -378,10 +378,14 @@
return default_ns, title
return ns, name
- # Replace underscores with spaces and multiple combinations of them
- # with only one space
- title1 = re.sub(r'[_ ]+', ' ', title1)
- title2 = re.sub(r'[_ ]+', ' ', title2)
+ # Replace alias characters like underscores with title
+ # delimiters like spaces and multiple combinations of them with
+ # only one delimiter
+ sep = self.family.title_delimiter_and_aliases[0]
+ pattern = re.compile('[{}]+'
+ .format(self.family.title_delimiter_and_aliases))
+ title1 = pattern.sub(sep, title1)
+ title2 = pattern.sub(sep, title2)
if title1 == title2:
return True
diff --git a/tests/link_tests.py b/tests/link_tests.py
index 220bebd..f8e9dbb 100644
--- a/tests/link_tests.py
+++ b/tests/link_tests.py
@@ -64,6 +64,19 @@
default site is using completely different namespaces.
"""
+ def replaced(self, iterable):
+ """Replace family specific title delimiter."""
+ for items in iterable:
+ if isinstance(items, str):
+ items = [items]
+ items = [re.sub(' ',
+ self.site.family.title_delimiter_and_aliases[0],
+ item)
+ for item in items]
+ if len(items) == 1:
+ items = items[0]
+ yield items
+
def test_valid(self):
"""Test that valid titles are correctly normalized."""
title_tests = ['Sandbox', 'A "B"', "A 'B'", '.com', '~', '"', "'",
@@ -87,11 +100,11 @@
site = self.get_site()
- for title in title_tests:
+ for title in self.replaced(title_tests):
with self.subTest(title=title):
self.assertEqual(Link(title, site).title, title)
- for link, title in extended_title_tests:
+ for link, title in self.replaced(extended_title_tests):
with self.subTest(link=link, title=title):
self.assertEqual(Link(link, site).title, title)
@@ -138,7 +151,7 @@
title_tests = [
# Empty title
- (['', ':', '__ __', ' __ '],
+ (['', ':'],
r'^The link \[\[.*\]\] does not contain a page title$'),
(['A [ B', 'A ] B', 'A { B', 'A } B', 'A < B', 'A > B'],
@@ -165,12 +178,21 @@
([('x' * 256), ('Invalid:' + 'X' * 248)],
generate_overlength_exc_regex),
- (['Talk:', 'Category: ', 'Category: #bar'],
+ (['Talk:'],
generate_has_no_title_exc_regex),
]
+ # Known issues with wikihow.
+ if self.site.family.name != 'wikihow':
+ title_tests.extend([
+ (['Category: ', 'Category: #bar'],
+ generate_has_no_title_exc_regex),
+ (['__ __', ' __ '],
+ r'^The link \[\[\]\] does not contain a page title$'),
+ ])
+
for texts_to_test, exception_regex in title_tests:
- for text in texts_to_test:
+ for text in self.replaced(texts_to_test):
with self.subTest(title=text):
if callable(exception_regex):
regex = exception_regex(text)
--
To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/614565
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Change-Id: Ib7858b88324376b6bbbf788893308fcf66c4d154
Gerrit-Change-Number: 614565
Gerrit-PatchSet: 10
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: Matěj Suchánek <matejsuchanek97(a)gmail.com>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot
Gerrit-CC: Lgessler <lukegessler(a)gmail.com>
Gerrit-MessageType: merged
jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/279755 )
Change subject: [IMPR] Rewrite len and bool function for Namespace class
......................................................................
[IMPR] Rewrite len and bool function for Namespace class
- len() is defined as length of the iterable. The iterator uses
Namespace._distinct(). Thus len should use it too instead of calculate it.
This prevents the method could break when Namespace._distinct() or
Namespace.aliases or any other properties will be changed further.
- bool() is derived from len() method but it should be independent from it
and always return True like a generic object class.
- Test added.
Change-Id: I9f37e3d511042317c2c895655c2bb138ef46c18c
---
M pywikibot/site/_namespace.py
M tests/namespace_tests.py
2 files changed, 15 insertions(+), 3 deletions(-)
Approvals:
Matěj Suchánek: Looks good to me, but someone else must approve
Xqt: Looks good to me, approved
jenkins-bot: Verified
diff --git a/pywikibot/site/_namespace.py b/pywikibot/site/_namespace.py
index bafcb76..4f7adba 100644
--- a/pywikibot/site/_namespace.py
+++ b/pywikibot/site/_namespace.py
@@ -144,11 +144,20 @@
return self._contains_lowercase_name(name.lower())
+ def __bool__(self) -> bool:
+ """Obtain boolean method for Namepace class.
+
+ This method is implemented to be independent from __len__ method.
+
+ .. versionadded:: 7.0
+
+ :return: Always return True like generic object class.
+ """
+ return True
+
def __len__(self):
"""Obtain length of the iterable."""
- if self.custom_name == self.canonical_name:
- return len(self.aliases) + 1
- return len(self.aliases) + 2
+ return len(self._distinct())
def __iter__(self):
"""Return an iterator."""
diff --git a/tests/namespace_tests.py b/tests/namespace_tests.py
index 177e8ff..e1cb2d7 100644
--- a/tests/namespace_tests.py
+++ b/tests/namespace_tests.py
@@ -77,6 +77,9 @@
for val in ns.values()
for name in val))
+ # test boolean
+ self.assertTrue(all(x for x in ns.values()))
+
# Use a namespace object as a dict key
self.assertEqual(ns[ns[6]], ns[6])
--
To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/279755
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Change-Id: I9f37e3d511042317c2c895655c2bb138ef46c18c
Gerrit-Change-Number: 279755
Gerrit-PatchSet: 5
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: Matěj Suchánek <matejsuchanek97(a)gmail.com>
Gerrit-Reviewer: Mpaa <mpaa.wiki(a)gmail.com>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot
Gerrit-MessageType: merged
jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/735951 )
Change subject: [cleanup] Raise an exception if 'titles' is still used as where parameter
......................................................................
[cleanup] Raise an exception if 'titles' is still used as where parameter
'titles' value for where parameter was deprecated 5 years ago.
Change-Id: I0b9530be65db2a017cab5cbc4395cf47191a0dff
---
M pywikibot/site/_generators.py
1 file changed, 11 insertions(+), 15 deletions(-)
Approvals:
Matěj Suchánek: Looks good to me, but someone else must approve
Xqt: Looks good to me, approved
jenkins-bot: Verified
diff --git a/pywikibot/site/_generators.py b/pywikibot/site/_generators.py
index ca50567..66d03c9 100644
--- a/pywikibot/site/_generators.py
+++ b/pywikibot/site/_generators.py
@@ -1296,27 +1296,23 @@
:raises TypeError: a namespace identifier has an inappropriate
type such as NoneType or bool
"""
- where_types = ['nearmatch', 'text', 'title', 'titles']
+ where_types = ['nearmatch', 'text', 'title']
if not searchstring:
raise Error('search: searchstring cannot be empty')
if where not in where_types:
raise Error("search: unrecognized 'where' value: {}".format(where))
- if where in ('title', 'titles'):
- if where == 'titles':
- issue_deprecation_warning("where='titles'", "where='title'",
- since='20160224')
- where = 'title'
- if self.has_extension('CirrusSearch') and \
- isinstance(self.family, pywikibot.family.WikimediaFamily):
- # 'title' search was disabled, use intitle instead
- searchstring = 'intitle:' + searchstring
- issue_deprecation_warning(
- "where='{}'".format(where),
- "searchstring='{}'".format(searchstring),
- since='20160224')
+ if where == 'title' \
+ and self.has_extension('CirrusSearch') \
+ and isinstance(self.family, pywikibot.family.WikimediaFamily):
+ # 'title' search was disabled, use intitle instead
+ searchstring = 'intitle:' + searchstring
+ issue_deprecation_warning(
+ "where='{}'".format(where),
+ "searchstring='{}'".format(searchstring),
+ since='20160224')
- where = None # default
+ where = None # default
if not namespaces and namespaces != 0:
namespaces = [ns_id for ns_id in self.namespaces if ns_id >= 0]
--
To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/735951
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Change-Id: I0b9530be65db2a017cab5cbc4395cf47191a0dff
Gerrit-Change-Number: 735951
Gerrit-PatchSet: 1
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: Matěj Suchánek <matejsuchanek97(a)gmail.com>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot
Gerrit-MessageType: merged
jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/736944 )
Change subject: [doc] Update ROADMAP.rst
......................................................................
[doc] Update ROADMAP.rst
Change-Id: Iefc1dce436f8403d7a8e2d8c99d99d291c314dfd
---
M ROADMAP.rst
1 file changed, 10 insertions(+), 0 deletions(-)
Approvals:
Xqt: Looks good to me, approved
jenkins-bot: Verified
diff --git a/ROADMAP.rst b/ROADMAP.rst
index 817a239..2c611c2 100644
--- a/ROADMAP.rst
+++ b/ROADMAP.rst
@@ -4,6 +4,9 @@
Improvements and Bugfixes
-------------------------
+* BaseBot has a public collections.Counter for reading, writing and skipping a page
+* Upload: Retry upload if 'copyuploadbaddomain' API error occurs (T294825)
+* Upload: Only set filekey/offset for files with names (T294916)
* Update invisible characters from unicodedata 14.0.0
* Make site parameter of textlib.replace_links() mandatory (T294649)
* Raise a generic ServerError if the http status code is unofficial (T293208)
@@ -24,6 +27,11 @@
Code cleanups
-------------
+* Deprecated version.get_module_version() function was removed
+* Deprecated setOptions/getOptions OptionHandler methods were removed
+* Deprecated from_page() method of CosmeticChangesToolkit was removed
+* Deprecated diff attribute of CosmeticChangesToolkit was removed in favour of show_diff
+* Deprecated namespace and pageTitle parameter of CosmeticChangesToolkit were removed
* Remove deprecated BaseSite namespace shortcuts
* Remove deprecated Family.get_cr_templates method in favour of Site.category_redirects()
* Remove deprecated Page.put_async() method (T193494)
@@ -66,6 +74,8 @@
Deprecations
^^^^^^^^^^^^
+* 7.0.0: Private BaseBot counters _treat_counter, _save_counter, _skip_counter will be removed in favour of collections.Counter counter attribute
+* 7.0.0: A boolean watch parameter in Page.save() is deprecated and will be desupported
* 7.0.0: baserevid parameter of editSource(), editQualifier(), removeClaims(), removeSources(), remove_qualifiers() DataSite methods will be removed
* 7.0.0: Values of APISite.allpages() parameter filterredir other than True, False and None are deprecated
* 6.5.0: OutputOption.output() method will be removed in favour of OutputOption.out property
--
To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/736944
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Change-Id: Iefc1dce436f8403d7a8e2d8c99d99d291c314dfd
Gerrit-Change-Number: 736944
Gerrit-PatchSet: 1
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot
Gerrit-MessageType: merged