jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/691589 )
Change subject: [bugfix] OutputOption.out must be a property not a method
......................................................................
[bugfix] OutputOption.out must be a property not a method
- Remove deprecation warning for SequenceOutputter.output().
This may be still usable as well as OutputOption.output() but
must always call the "out" property.
- Remove early parsing link introduced with f4ddd1f0 which leads
script to fail.
- Add a more informative string for InvalidTitleError
- Update tests accordingly
Bug: T282936
Change-Id: I7cfd0b4ddeebba83ca1e72e7dd74e98c18fd2315
---
M pywikibot/bot_choice.py
M pywikibot/page/__init__.py
M pywikibot/tools/formatter.py
M tests/link_tests.py
4 files changed, 5 insertions(+), 12 deletions(-)
Approvals:
Xqt: Looks good to me, approved
jenkins-bot: Verified
diff --git a/pywikibot/bot_choice.py b/pywikibot/bot_choice.py
index 5672e1f..ba8518c 100755
--- a/pywikibot/bot_choice.py
+++ b/pywikibot/bot_choice.py
@@ -549,6 +549,7 @@
color = 'lightred'
+ @property
def out(self):
"""Highlighted output section of the text."""
start = max(0, self.start - self.context)
diff --git a/pywikibot/page/__init__.py b/pywikibot/page/__init__.py
index 53ff080..6ddfc9d 100644
--- a/pywikibot/page/__init__.py
+++ b/pywikibot/page/__init__.py
@@ -181,13 +181,6 @@
self._link = Link(title, source=source.site,
default_namespace=ns)
elif isinstance(source, BaseLink):
- if not source.title:
- raise InvalidTitleError(
- '{} title of {} {} cannot be empty.'
- .format(self.__class__.__name__,
- source.__class__.__name__,
- source.astext()))
-
self._link = source
self._revisions = {}
else:
@@ -5374,7 +5367,8 @@
# with a fragment identifier.
if not t.strip(' ') and not self._is_interwiki: # T197642
raise InvalidTitleError(
- 'The link does not contain a page title')
+ 'The link [[{}]] does not contain a page title'
+ .format(self._text))
if self._site.namespaces[self._namespace].case == 'first-letter':
t = first_upper(t)
diff --git a/pywikibot/tools/formatter.py b/pywikibot/tools/formatter.py
index d67d5ec..f629651 100644
--- a/pywikibot/tools/formatter.py
+++ b/pywikibot/tools/formatter.py
@@ -59,8 +59,6 @@
content = ''
return self.prefix + content + self.suffix
- @deprecated('pywikibot.output(SequenceOutputter.out)', since='6.2.0',
- future_warning=True)
def output(self):
"""Output the text of the current sequence."""
output(self.out)
diff --git a/tests/link_tests.py b/tests/link_tests.py
index 14c586b..d94a6ed 100644
--- a/tests/link_tests.py
+++ b/tests/link_tests.py
@@ -139,7 +139,7 @@
title_tests = [
# Empty title
(['', ':', '__ __', ' __ '],
- r'^The link does not contain a page title$'),
+ r'^The link \[\[.*\]\] does not contain a page title$'),
(['A [ B', 'A ] B', 'A { B', 'A } B', 'A < B', 'A > B'],
generate_contains_illegal_chars_exc_regex),
@@ -893,7 +893,7 @@
link = Link('', self.get_site())
with self.assertRaisesRegex(
InvalidTitleError,
- 'The link does not contain a page title'):
+ r'The link \[\[.*\]\] does not contain a page title'):
link.parse()
def test_namespace_lookalike(self):
--
To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/691589
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Change-Id: I7cfd0b4ddeebba83ca1e72e7dd74e98c18fd2315
Gerrit-Change-Number: 691589
Gerrit-PatchSet: 6
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: Matěj Suchánek <matejsuchanek97(a)gmail.com>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot
Gerrit-MessageType: merged
jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/691585 )
Change subject: [docs] Update ROADMAP.rst
......................................................................
[docs] Update ROADMAP.rst
Change-Id: I8e2aba6c3565f780ba98836d333d41209982c960
---
M ROADMAP.rst
M pywikibot/tools/__init__.py
2 files changed, 19 insertions(+), 7 deletions(-)
Approvals:
Xqt: Looks good to me, approved
jenkins-bot: Verified
diff --git a/ROADMAP.rst b/ROADMAP.rst
index 1f6e461..207f939 100644
--- a/ROADMAP.rst
+++ b/ROADMAP.rst
@@ -1,20 +1,32 @@
Current release changes
^^^^^^^^^^^^^^^^^^^^^^^
+Improvements and Bugfixes
+-------------------------
+
+* Fix iterating of SizedKeyCollection (T282865)
+* The cached output functionality from compat relase was re-implemented (T151727, T73646, T74942, T132135, T144698, T280466)
+* An abstract base user interface module was added
* APISite method pagelanglinks() may skip links with empty titles (T223157)
+* Fix Page.getDeletedRevision() method which always returned an empty list
+* Async chunked uploads are supported (T129216, T133443)
+* A new InvalidPageError will be raised if a Page has no version history (T280043)
+* L10N updates
+* Fix __getattr__ for WikibaseEntity (T281389)
+* Handle abusefilter-{disallow,warning} codes (T85656)
+
+Code cleanups
+-------------
+
+* Move page functions UnicodeToAsciiHtml, unicode2html, url2unicode to tools.chars with renaming them
+* Rename _MultiTemplateMatchBuilder to MultiTemplateMatchBuilder
* User.name() method was removed in favour of User.username property
* BasePage.getLatestEditors() method was removed in favour of contributors() or revisions()
* pagenenerators.handleArg() method was renamed to handle_arg() (T271437)
* CategoryGenerator, FileGenerator, ImageGenerator and ReferringPageGenerator pagegenerator functions were removed
* Family.ignore_certificate_error() method was removed in favour of verify_SSL_certificate (T265205)
* tools.is_IP was renamed to is_ip_address due to PEP8
-* Fix Page.getDeletedRevision() method which always returned an empty list
-* Async chunked uploads are supported (T129216, T133443)
-* A new InvalidPageError will be raised if a Page has no version history (T280043)
* config2.py was renamed to config.py
-* L10N updates
-* Fix __getattr__ for WikibaseEntity (T281389)
-* Handle abusefilter-{disallow,warning} codes (T85656)
* Exceptions were renamed having a suffix "Error" due to PEP8 (T280227)
Deprecations
diff --git a/pywikibot/tools/__init__.py b/pywikibot/tools/__init__.py
index dad16f3..218cd5d 100644
--- a/pywikibot/tools/__init__.py
+++ b/pywikibot/tools/__init__.py
@@ -640,7 +640,7 @@
"""
def __init__(self, *args, **kwargs):
- """Initialize RLock."""
+ """Initializer."""
self._lock = threading.RLock(*args, **kwargs)
self._block = threading.Lock()
--
To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/691585
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Change-Id: I8e2aba6c3565f780ba98836d333d41209982c960
Gerrit-Change-Number: 691585
Gerrit-PatchSet: 1
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot
Gerrit-MessageType: merged
jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/688923 )
Change subject: [IMPR] Replace some assert statements in api.py
......................................................................
[IMPR] Replace some assert statements in api.py
assert should not be used outside tests because they can be removed
with -o option; replace it with RuntimeError if applicable.
Change-Id: I6cec8ab6d5b0d5a3e2342753fffe19f38577a7b3
---
M pywikibot/data/api.py
1 file changed, 17 insertions(+), 5 deletions(-)
Approvals:
Xqt: Looks good to me, approved
jenkins-bot: Verified
diff --git a/pywikibot/data/api.py b/pywikibot/data/api.py
index c1de7ac..733ef13 100644
--- a/pywikibot/data/api.py
+++ b/pywikibot/data/api.py
@@ -206,7 +206,8 @@
# the same data available in the paraminfo for query.
query_modules_param = self.parameter('paraminfo', 'querymodules')
- assert('limit' in query_modules_param)
+ if 'limit' not in query_modules_param:
+ raise RuntimeError('"limit" not found in query modules')
self._limit = query_modules_param['limit']
if query_modules_param and 'type' in query_modules_param:
@@ -583,7 +584,10 @@
if not param_data:
return None
- assert(len(param_data) == 1)
+ if len(param_data) != 1:
+ raise RuntimeError(
+ 'parameter data length is eiter empty or not unique.\n{}'
+ .format(param_data))
return param_data[0]
@property
@@ -1132,7 +1136,10 @@
if isinstance(value, datetime.datetime):
return value.strftime(pywikibot.Timestamp.ISO8601Format)
if isinstance(value, pywikibot.page.BasePage):
- assert(value.site == self.site)
+ if value.site != self.site:
+ raise RuntimeError(
+ 'value.site {!r} is different from Request.site {!r}'
+ .format(value.site, self.site))
return value.title(with_section=False)
return str(value)
@@ -1972,7 +1979,9 @@
filename = self._cachefile_path()
with open(filename, 'rb') as f:
uniquedescr, self._data, self._cachetime = pickle.load(f)
- assert(uniquedescr == self._uniquedescriptionstr())
+ if uniquedescr != self._uniquedescriptionstr():
+ raise RuntimeError('Expected unique description for the cache '
+ 'entry is different from file entry.')
if self._expired(self._cachetime):
self._data = None
return False
@@ -3110,7 +3119,10 @@
page.latest_revision_id = pagedict['lastrevid']
if 'imageinfo' in pagedict:
- assert(isinstance(page, pywikibot.FilePage))
+ if not isinstance(page, pywikibot.FilePage):
+ raise RuntimeError(
+ '"imageinfo" found but {} is not a FilePage object'
+ .format(page))
page._load_file_revisions(pagedict['imageinfo'])
if 'categoryinfo' in pagedict:
--
To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/688923
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Change-Id: I6cec8ab6d5b0d5a3e2342753fffe19f38577a7b3
Gerrit-Change-Number: 688923
Gerrit-PatchSet: 1
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot
Gerrit-MessageType: merged
jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/690429 )
Change subject: [IMPR] move character bases functions from page to tools.chars
......................................................................
[IMPR] move character bases functions from page to tools.chars
unicode2html and url2unicode belongs to page titles but is also
independed from other framework parts. Therefore move them to the
module dealing with characters and strings. The page module is too
big and complex already.
- move page.UnicodeToAsciiHtml to tools.chars.string_to_ascii_html
- move page.unicode2html to tools.chars.string2html
- move page.url2unicode to tools.chars.url2string
- deprecate page functions and their direct import from pywikibot
- deprecate pywikibot.site.BaseSite as encodings parameter in url2unicode
function; BaseSite.encodings should be used instead
- update usage of unicode2html in reflinks.py
- update usage of url2unicode in cosmetic_changes.py
Change-Id: Ifab5ab13bf18a2693c014b3d0beeee59e36e0c2a
---
M pywikibot/__init__.py
M pywikibot/cosmetic_changes.py
M pywikibot/page/__init__.py
M pywikibot/tools/chars.py
M scripts/reflinks.py
5 files changed, 102 insertions(+), 57 deletions(-)
Approvals:
Xqt: Looks good to me, approved
jenkins-bot: Verified
diff --git a/pywikibot/__init__.py b/pywikibot/__init__.py
index 4e1204e..cb84609 100644
--- a/pywikibot/__init__.py
+++ b/pywikibot/__init__.py
@@ -1221,7 +1221,6 @@
SiteLink,
User,
html2unicode,
- unicode2html,
url2unicode,
)
@@ -1361,6 +1360,9 @@
since='20200707')
wrapper._add_deprecated_attr('showHelp', show_help,
since='20200705', future_warning=True)
+wrapper._add_deprecated_attr(
+ 'unicode2html', replacement_name='pywikibot.tools.chars.string2html',
+ since='6.2.0', future_warning=True)
# This module aliases many (but not all) pywikibot.exception classes and one
# from pywikibot.data.api. Use of these aliases is deprecated. When removed
diff --git a/pywikibot/cosmetic_changes.py b/pywikibot/cosmetic_changes.py
index 8d2270c..c9d3c73 100755
--- a/pywikibot/cosmetic_changes.py
+++ b/pywikibot/cosmetic_changes.py
@@ -61,7 +61,6 @@
import pywikibot
from pywikibot import textlib
from pywikibot.exceptions import InvalidTitleError
-from pywikibot.page import url2unicode
from pywikibot.textlib import (
FILE_LINK_REGEX,
_get_regexes,
@@ -74,6 +73,7 @@
first_upper,
issue_deprecation_warning,
)
+from pywikibot.tools.chars import url2string
try:
@@ -582,8 +582,8 @@
hadTrailingSpaces = len(titleWithSection) != titleLength
# Convert URL-encoded characters to str
- titleWithSection = url2unicode(titleWithSection,
- encodings=self.site)
+ titleWithSection = url2string(titleWithSection,
+ encodings=self.site.encodings())
if not titleWithSection:
# just skip empty links.
diff --git a/pywikibot/page/__init__.py b/pywikibot/page/__init__.py
index 9cdaa25..19bfd3d 100644
--- a/pywikibot/page/__init__.py
+++ b/pywikibot/page/__init__.py
@@ -25,7 +25,7 @@
from http import HTTPStatus
from itertools import chain
from typing import Any, Optional, Union
-from urllib.parse import quote_from_bytes, unquote_to_bytes
+from urllib.parse import quote_from_bytes
from warnings import warn
import pywikibot
@@ -73,6 +73,8 @@
deprecated_args,
first_upper,
is_ip_address,
+ issue_deprecation_warning,
+ ModuleDeprecationWrapper,
redirect_func,
remove_last_args,
)
@@ -5181,9 +5183,8 @@
self._anchor = None
# Convert URL-encoded characters to unicode
- encodings = [self._source.encoding()] + list(self._source.encodings())
-
- self._text = url2unicode(self._text, encodings=encodings)
+ self._text = pywikibot.tools.chars.url2string(
+ self._text, encodings=self._source.encodings())
# Clean up the name, it can come from anywhere.
# Convert HTML entities to unicode
@@ -5724,40 +5725,12 @@
return _ENTITY_SUB(handle_entity, text)
-def UnicodeToAsciiHtml(string) -> str:
- """Convert unicode to a str using HTML entities."""
- html = []
- for c in string:
- cord = ord(c)
- if 31 < cord < 127:
- html.append(c)
- else:
- html.append('&#{};'.format(cord))
- return ''.join(html)
-
-
-def unicode2html(string: str, encoding: str) -> str:
- """
- Convert unicode string to requested HTML encoding.
-
- Attempt to encode the
- string into the desired format; if that doesn't work, encode the unicode
- into HTML &#; entities. If it does work, return it unchanged.
-
- @param string: String to update
- @param encoding: Encoding to use
- """
- try:
- string.encode(encoding)
- except UnicodeError:
- string = UnicodeToAsciiHtml(string)
- return string
-
-
@deprecated_args(site='encodings')
+@deprecated('pywikibot.tools.chars.url2string', since='6.2.0',
+ future_warning=True)
def url2unicode(title: str, encodings='utf-8') -> str:
"""
- Convert URL-encoded text to unicode using several encoding.
+ DEPRECATED. Convert URL-encoded text to unicode using several encoding.
Uses the first encoding that doesn't cause an error.
@@ -5767,21 +5740,24 @@
@raise UnicodeError: Could not convert using any encoding.
"""
- if isinstance(encodings, str):
- encodings = [encodings]
- elif isinstance(encodings, pywikibot.site.BaseSite):
- # create a list of all possible encodings for both hint sites
- site = encodings
- encodings = [site.encoding()] + list(site.encodings())
+ if isinstance(encodings, pywikibot.site.BaseSite):
+ # use all possible encodings from Site object
+ encodings = encodings.encodings()
+ issue_deprecation_warning(
+ 'Passing BaseSite object to encodings parameter',
+ 'BaseSite.endcodings()',
+ depth=1,
+ warning_class=FutureWarning,
+ since='6.2.0'
+ )
- first_exception = None
- for enc in encodings:
- try:
- t = title.encode(enc)
- t = unquote_to_bytes(t)
- return t.decode(enc)
- except UnicodeError as ex:
- if not first_exception:
- first_exception = ex
- # Couldn't convert, raise the original exception
- raise first_exception
+ return pywikibot.tools.chars.url2string(title, encodings)
+
+
+wrapper = ModuleDeprecationWrapper(__name__)
+wrapper._add_deprecated_attr('UnicodeToAsciiHtml',
+ pywikibot.tools.chars.string_to_ascii_html,
+ since='6.2.0', future_warning=True)
+wrapper._add_deprecated_attr('unicode2html',
+ pywikibot.tools.chars.string2html,
+ since='6.2.0', future_warning=True)
diff --git a/pywikibot/tools/chars.py b/pywikibot/tools/chars.py
index d7799a9..f57a601 100644
--- a/pywikibot/tools/chars.py
+++ b/pywikibot/tools/chars.py
@@ -7,6 +7,11 @@
import re
import sys
+from contextlib import suppress
+from typing import Union
+from urllib.parse import unquote_to_bytes
+
+from pywikibot.backports import List, Tuple
from pywikibot.tools._unidata import _category_cf
@@ -36,3 +41,63 @@
return '<{0:x}>'.format(codepoint)
return INVISIBLE_REGEX.sub(replace, text)
+
+
+def string_to_ascii_html(string: str) -> str:
+ """Convert unicode chars of str to HTML entities if chars are not ASCII."""
+ html = []
+ for c in string:
+ cord = ord(c)
+ if 31 < cord < 127:
+ html.append(c)
+ else:
+ html.append('&#{};'.format(cord))
+ return ''.join(html)
+
+
+def string2html(string: str, encoding: str) -> str:
+ """Convert unicode string to requested HTML encoding.
+
+ Attempt to encode the string into the desired format; if that work
+ return it unchanged. Otherwise encode the non-ASCII characters into
+ HTML &#; entities.
+
+ @param string: String to update
+ @param encoding: Encoding to use
+ """
+ with suppress(UnicodeError):
+ string.encode(encoding)
+ return string
+
+ return string_to_ascii_html(string)
+
+
+def url2string(
+ title: str,
+ encodings: Union[str, List[str], Tuple[str, ...]] = 'utf-8'
+) -> str:
+ """Convert URL-encoded text to unicode using several encoding.
+
+ Uses the first encoding that doesn't cause an error.
+
+ @param title: URL-encoded character data to convert
+ @param encodings: Encodings to attempt to use during conversion.
+
+ @raise UnicodeError: Could not convert using any encoding.
+ """
+ if isinstance(encodings, str):
+ encodings = [encodings]
+
+ first_exception = None
+ for enc in encodings:
+ try:
+ t = title.encode(enc)
+ t = unquote_to_bytes(t)
+ except UnicodeError as e:
+ if not first_exception:
+ first_exception = e
+ else:
+ return t.decode(enc)
+
+ # Couldn't convert, raise the first exception
+ raise first_exception
diff --git a/scripts/reflinks.py b/scripts/reflinks.py
index bb889be..dc437c4 100755
--- a/scripts/reflinks.py
+++ b/scripts/reflinks.py
@@ -65,6 +65,8 @@
)
from pywikibot.textlib import replaceExcept
from pywikibot.tools.formatter import color_format
+from pywikibot.tools.chars import string2html
+
from scripts import noreferences
@@ -246,7 +248,7 @@
self.title = self.title.replace('}}', '}}')
# prevent multiple quotes being interpreted as '' or '''
self.title = self.title.replace("''", "''")
- self.title = pywikibot.unicode2html(self.title, self.site.encoding())
+ self.title = string2html(self.title, self.site.encoding())
# TODO : remove HTML when both opening and closing tags are included
def avoid_uppercase(self):
--
To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/690429
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Change-Id: Ifab5ab13bf18a2693c014b3d0beeee59e36e0c2a
Gerrit-Change-Number: 690429
Gerrit-PatchSet: 2
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: D3r1ck01 <xsavitar.wiki(a)aol.com>
Gerrit-Reviewer: Dalba <dalba.wiki(a)gmail.com>
Gerrit-Reviewer: Dvorapa <dvorapa(a)seznam.cz>
Gerrit-Reviewer: JJMC89 <JJMC89.Wikimedia(a)gmail.com>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot
Gerrit-MessageType: merged
jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/690679 )
Change subject: [IMPR] Improvements fo generate_family_file.py
......................................................................
[IMPR] Improvements fo generate_family_file.py
- Add usage informations to script docstring
- Add typing hints and parameter informations to FamilyFileGenerator
- Ask for url and family name in get_params method; abort script if
they are empty
- introduce -help option to print docstring
Change-Id: I52800f19f48dde321f4798beac15ba65639f37ab
---
M generate_family_file.py
1 file changed, 67 insertions(+), 22 deletions(-)
Approvals:
Xqt: Looks good to me, approved
jenkins-bot: Verified
diff --git a/generate_family_file.py b/generate_family_file.py
index 9c5f33d..0d90366 100755
--- a/generate_family_file.py
+++ b/generate_family_file.py
@@ -1,5 +1,24 @@
#!/usr/bin/python
-"""This script generates a family file from a given URL."""
+"""This script generates a family file from a given URL.
+
+Usage::
+
+ generate_family_file.py [<url>] [<name>] [<dointerwiki>] [<verify>]
+
+Parameters are optional. They must be given consecutively but may be
+omitted if there is no successor parameter. The parameters are::
+
+ <url>: an url from where the family settings are loaded
+ <name>: the family name without "_family.py" tail.
+ <dointerwiki>: predefined answer (y|n) to add multiple language
+ <verify>: disable certificate validaton `(y|n)
+
+Example::
+
+ generate_family_file.py https://www.mywiki.bogus/wiki/Main_Page mywiki
+
+This will create the file mywiki_family.py in pywikibot/families folder
+"""
#
# (C) Pywikibot team, 2010-2021
#
@@ -9,7 +28,9 @@
import os
import string
import sys
+
from os import environ, getenv
+from typing import Optional
from urllib.parse import urlparse
@@ -23,33 +44,59 @@
class FamilyFileGenerator:
- """Family file creator."""
+ """Family file creator object."""
- def __init__(self, url=None, name=None, dointerwiki=None, verify=None):
- """Initializer."""
+ def __init__(self,
+ url: Optional[str] = None,
+ name: Optional[str] = None,
+ dointerwiki: Optional[str] = None,
+ verify: Optional[str] = None):
+ """
+ Parameters are optional. If not given the script asks for the values.
+
+ @param url: an url from where the family settings are loaded
+ @param name: the family name without "_family.py" tail.
+ @param dointerwiki: Predefined answer to add multiple language
+ codes. Pass `Y` or `y` for yes `N` or `n` for no and
+ `E` or `e` if you want to edit the collection of sites.
+ @param verify: If a certificate verification failes, you may
+ pass `Y` or `y` to disable certificate validaton `N` or `n`
+ to keep it enabled.
+ """
# from pywikibot.site_detect import MWSite
# when required but disable user-config checks
# so the family can be created first,
# and then used when generating the user-config
self.Wiki = _import_with_no_user_config(
'pywikibot.site_detect').site_detect.MWSite
- if url is None:
- url = input('Please insert URL to wiki: ')
- if name is None:
- name = input('Please insert a short name (eg: freeciv): ')
- assert all(x in NAME_CHARACTERS for x in name), \
- 'Name of family "{}" must be ASCII letters and digits ' \
- '[a-zA-Z0-9]'.format(name)
-
- self.dointerwiki = dointerwiki
self.base_url = url
self.name = name
+ self.dointerwiki = dointerwiki
self.verify = verify
self.wikis = {} # {'https://wiki/$1': Wiki('https://wiki/$1'), ...}
self.langs = [] # [Wiki('https://wiki/$1'), ...]
+ def get_params(self):
+ """Ask for parameters if necessary."""
+ if self.base_url is None:
+ self.base_url = input('Please insert URL to wiki: ')
+ if not self.base_url:
+ return False
+
+ if self.name is None:
+ self.name = input('Please insert a short name (eg: freeciv): ')
+ if not self.name:
+ return False
+
+ if any(x not in NAME_CHARACTERS for x in self.name):
+ print('ERROR: Name of family "{}" must be ASCII letters and '
+ 'digits [a-zA-Z0-9]'.format(self.name))
+ return False
+
+ return True
+
def get_wiki(self):
"""Get wiki from base_url."""
import pywikibot
@@ -73,6 +120,9 @@
def run(self):
"""Main method, generate family file."""
+ if not self.get_params():
+ return
+
w, verify = self.get_wiki()
if w is None:
return
@@ -241,15 +291,10 @@
def main():
"""Process command line arguments and generate a family file."""
- if len(sys.argv) != 3:
- print("""
-Usage: {module} <url> <short name>
-Example: {module} https://www.mywiki.bogus/wiki/Main_Page mywiki
-This will create the file mywiki_family.py in pywikibot{sep}families"""
- .format(module=sys.argv[0].strip('.' + os.sep),
- sep=os.sep))
-
- FamilyFileGenerator(*sys.argv[1:]).run()
+ if len(sys.argv) > 1 and sys.argv[1] == '-help':
+ print(__doc__)
+ else:
+ FamilyFileGenerator(*sys.argv[1:]).run()
if __name__ == '__main__':
--
To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/690679
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Change-Id: I52800f19f48dde321f4798beac15ba65639f37ab
Gerrit-Change-Number: 690679
Gerrit-PatchSet: 1
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl>
Gerrit-Reviewer: Mpaa <mpaa.wiki(a)gmail.com>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot
Gerrit-MessageType: merged
jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/282818 )
Change subject: [bugfix] Port cache_output from compat
......................................................................
[bugfix] Port cache_output from compat
- compat had a cache_output functionality to fetch all async output during
any input and print it afterwards. This feature is ported to core but in
lowlevel userinterface.
- Add a RLock class to tools which works like RLock to enable nested
locks but it also has a count property and a locked method.
- ui.output() redirects any output stream to a cache and flushes cache
if output to terminal is not locked; cache is implemented as
SimpleQueue
- ui.stream_output() outputs text to terminal
- ui.flush() outputs the cache entries to terminal via stream_output()
- register ui.flush() to be called at termination
- Add SimpleQueue to backports.py
- bot_choice.Option is an abstract base class now
- OutputOption got a new "out" property to be used inside ui.input_choice
- OutputProxyOption uses out property of passed output class
- update SequenceOutputter accordingly
- use color_format with ContextOption and HighlightContextOption
- update CatContextOption in category.py
- revert previous T73646 fix and use standard behaviour of BaseBot
Bug: T73646
Bug: T74942
Bug: T132135
Bug: T144698
Bug: T280466
Bug: T151727
Change-Id: I02d6b7a939a1912a4ebb8694af3e159d7dd7dae5
---
M pywikibot/backports.py
M pywikibot/bot.py
M pywikibot/bot_choice.py
M pywikibot/tools/__init__.py
M pywikibot/tools/formatter.py
M pywikibot/userinterfaces/_interface_base.py
M pywikibot/userinterfaces/terminal_interface_base.py
M scripts/category.py
M scripts/replace.py
M tests/i18n_tests.py
10 files changed, 314 insertions(+), 144 deletions(-)
Approvals:
Xqt: Looks good to me, approved
jenkins-bot: Verified
diff --git a/pywikibot/backports.py b/pywikibot/backports.py
index 69e4057..85bf4a2 100644
--- a/pywikibot/backports.py
+++ b/pywikibot/backports.py
@@ -34,6 +34,13 @@
from contextlib import nullcontext
+# queue
+if PYTHON_VERSION < (3, 7):
+ from queue import Queue as SimpleQueue
+else:
+ from queue import SimpleQueue
+
+
# typing
if PYTHON_VERSION < (3, 5, 2):
from typing import Dict as DefaultDict
diff --git a/pywikibot/bot.py b/pywikibot/bot.py
index 1e2b431..86ab1d7 100644
--- a/pywikibot/bot.py
+++ b/pywikibot/bot.py
@@ -82,6 +82,7 @@
)
+import atexit
import codecs
import configparser
import datetime
@@ -176,6 +177,7 @@
uiModule = __import__('pywikibot.userinterfaces.{}_interface'
.format(config.userinterface), fromlist=['UI'])
ui = uiModule.UI()
+atexit.register(ui.flush)
pywikibot.argvu = ui.argvu()
_GLOBAL_HELP = """
diff --git a/pywikibot/bot_choice.py b/pywikibot/bot_choice.py
index 5385947..5672e1f 100755
--- a/pywikibot/bot_choice.py
+++ b/pywikibot/bot_choice.py
@@ -5,13 +5,21 @@
# Distributed under the terms of the MIT license.
#
import re
+
+from abc import ABC, abstractmethod
from textwrap import fill
from typing import Optional
import pywikibot
+from pywikibot.tools import (
+ deprecated,
+ deprecated_args,
+ issue_deprecation_warning,
+)
-class Option:
+
+class Option(ABC):
"""
A basic option for input_choice.
@@ -72,18 +80,27 @@
"""Return a formatted string for that option."""
raise NotImplementedError()
- def result(self, value):
- """Return the actual value which is associated by the given one."""
- raise NotImplementedError()
-
def test(self, value):
"""Return True whether this option applies."""
raise NotImplementedError()
+ @abstractmethod
+ def result(self, value):
+ """Return the actual value which is associated by the given one.
+
+ *New in version 6.2:* *result()* is an abstract method and must
+ be defined in subclasses
+ """
+ raise NotImplementedError()
+
class OutputOption(Option):
- """An option that never stops and can output on each question."""
+ """An option that never stops and can output on each question.
+
+ :Note: OutputOption must have a an "out" property which returns a
+ string for output method.
+ """
before_question = False
@@ -93,12 +110,25 @@
return False
def result(self, value):
- """Just output the value."""
- self.output()
+ """Just return None."""
+ return None
+
+ @property
+ def out(self) -> str:
+ """String to be used when selected and possibly before the question.
+
+ :Note: This method is used by ui.input_choice instead of output().
+
+ *New in version 6.2.*
+ """
+ return ''
def output(self):
- """Output a string when selected and possibly before the question."""
- raise NotImplementedError()
+ """Output string when selected and possibly before the question.
+
+ :Note: This method should never be overridden.
+ """
+ pywikibot.output(self.out)
class StandardOption(Option):
@@ -140,16 +170,23 @@
class OutputProxyOption(OutputOption, StandardOption):
- """An option which calls output of the given output class."""
+ """An option which calls out property of the given output class."""
def __init__(self, option, shortcut, output, **kwargs):
"""Create a new option for the given sequence."""
super().__init__(option, shortcut, **kwargs)
self._outputter = output
- def output(self):
- """Output the contents."""
- self._outputter.output()
+ @property
+ def out(self) -> str:
+ """Return te contents."""
+ if not hasattr(self._outputter, 'out'):
+ issue_deprecation_warning('{} without "out" property'
+ .format(self.__class__.__name__),
+ warning_class=FutureWarning,
+ since='6.2.0')
+ return self._outputter.output()
+ return self._outputter.out
class NestedOption(OutputOption, StandardOption):
@@ -181,9 +218,10 @@
return super().handled(value)
- def output(self):
- """Output the suboptions."""
- pywikibot.output(self._output)
+ @property
+ def out(self):
+ """Output of suboptions."""
+ return self._output
class ContextOption(OutputOption, StandardOption):
@@ -206,15 +244,19 @@
self.context += self.delta
super().result(value)
- def output(self):
- """Output the context."""
+ @property
+ def out(self):
+ """Output section of the text."""
start = max(0, self.start - self.context)
end = min(len(self.text), self.end + self.context)
- self.output_range(start, end)
+ return self.text[start:end]
- def output_range(self, start_context, end_context):
- """Output a section from the text."""
- pywikibot.output(self.text[start_context:end_context])
+ @deprecated_args(start_context='start', end_context='end')
+ @deprecated('pywikibot.output(ContextOption.out)', since='6.2.0',
+ future_warning=True)
+ def output_range(self, start, end):
+ """DEPRECATED. Output a section from the text."""
+ pywikibot.output(self.text[start:end])
class Choice(StandardOption):
@@ -231,6 +273,7 @@
"""The replacer."""
return self._replacer
+ @abstractmethod
def handle(self):
"""Handle this choice. Must be implemented."""
raise NotImplementedError()
@@ -339,9 +382,9 @@
value = self.parse(value)
except ValueError:
return False
- else:
- return ((self.minimum is None or value >= self.minimum)
- and (self.maximum is None or value <= self.maximum))
+
+ return ((self.minimum is None or value >= self.minimum)
+ and (self.maximum is None or value <= self.maximum))
@property
def minimum(self):
@@ -389,7 +432,7 @@
def result(self, value):
"""Return the value converted into int."""
- return (self.prefix, self.parse(value))
+ return self.prefix, self.parse(value)
class ListOption(IntegerOption):
@@ -419,7 +462,7 @@
def result(self, value):
"""Return a tuple with the prefix and selected value."""
- return (self.prefix, self._list[self.parse(value) - 1])
+ return self.prefix, self._list[self.parse(value) - 1]
class ShowingListOption(ListOption, OutputOption):
@@ -447,15 +490,18 @@
"""Return whether this option stops asking."""
return self._stop
- def output(self):
- """Output the enumerated list."""
+ @property
+ def out(self):
+ """Output text of the enumerated list."""
+ text = ''
if self.pre is not None:
- pywikibot.output(self.pre)
+ text = self.pre + '\n'
width = len(str(self.maximum))
for i, item in enumerate(self._list, self.minimum):
- pywikibot.output('{:>{width}} - {}'.format(i, item, width=width))
+ text += '{:>{width}} - {}\n'.format(i, item, width=width)
if self.post is not None:
- pywikibot.output(self.post)
+ text += self.post + '\n'
+ return text
class MultipleChoiceList(ListOption):
@@ -486,7 +532,7 @@
"""Return a tuple with the prefix and selected values as a list."""
values = (self.parse(val) for val in value.split(','))
result = [self._list[val - 1] for val in values]
- return (self.prefix, result)
+ return self.prefix, result
class ShowingMultipleChoiceList(ShowingListOption, MultipleChoiceList):
@@ -501,11 +547,30 @@
"""Show the original region highlighted."""
+ color = 'lightred'
+
+ def out(self):
+ """Highlighted output section of the text."""
+ start = max(0, self.start - self.context)
+ end = min(len(self.text), self.end + self.context)
+ color_format = pywikibot.tools.formatter.color_format
+ return color_format('{}{%(color)s}{}{default}{}'
+ % {'color': self.color},
+ self.text[start:self.start],
+ self.text[self.start:self.end],
+ self.text[self.end:end])
+
+ @deprecated('pywikibot.output(HighlightContextOption.out)',
+ since='6.2.0', future_warning=True)
def output_range(self, start, end):
- """Show normal context with a red center region."""
- pywikibot.output(self.text[start:self.start] + '\03{lightred}'
- + self.text[self.start:self.end] + '\03{default}'
- + self.text[self.end:end])
+ """DEPRECATED. Show normal context with a highlighted center region."""
+ color_format = pywikibot.tools.formatter.color_format
+ text = color_format('{}{%(color)s}{}{default}{}'
+ % {'color': self.color},
+ self.text[start:self.start],
+ self.text[self.start:self.end],
+ self.text[self.end:end])
+ pywikibot.output(text)
class UnhandledAnswer(Exception):
diff --git a/pywikibot/tools/__init__.py b/pywikibot/tools/__init__.py
index c533af15..bded5bd 100644
--- a/pywikibot/tools/__init__.py
+++ b/pywikibot/tools/__init__.py
@@ -614,6 +614,68 @@
return self._cmp(other) >= 0
+class RLock:
+ """Context manager which implements extended reentrant lock objects.
+
+ This RLock is implicit derived from threading.RLock but provides a
+ locked() method like in threading.Lock and a count attribute which
+ gives the active recursion level of locks.
+
+ Usage:
+
+ >>> from pywikibot.tools import RLock
+ >>> lock = RLock()
+ >>> lock.acquire()
+ True
+ >>> with lock: print(lock.count) # nested lock
+ 2
+ >>> lock.locked()
+ True
+ >>> lock.release()
+ >>> lock.locked()
+ False
+
+ *New in version 6.2*
+ """
+
+ def __init__(self, *args, **kwargs):
+ """Initialize RLock."""
+ self._lock = threading.RLock(*args, **kwargs)
+ self._block = threading.Lock()
+
+ def __enter__(self):
+ """Acquire lock and call atenter."""
+ return self._lock.__enter__()
+
+ def __exit__(self, *exc):
+ """Call atexit and release lock."""
+ return self._lock.__exit__(*exc)
+
+ def __getattr__(self, name):
+ """Delegate attributes and methods to self._lock."""
+ return getattr(self._lock, name)
+
+ def __repr__(self):
+ """Representation of tools.RLock instance."""
+ return repr(self._lock).replace(
+ '_thread.RLock',
+ '{cls.__module__}.{cls.__class__.__name__}'.format(cls=self))
+
+ @property
+ def count(self):
+ """Return number of acquired locks."""
+ with self._block:
+ counter = re.search(r'count=(\d+) ', repr(self))
+ return int(counter.group(1))
+
+ def locked(self):
+ """Return true if the lock is acquired."""
+ with self._block:
+ status = repr(self).split(maxsplit=1)[0][1:]
+ assert status in ('locked', 'unlocked')
+ return status == 'locked'
+
+
class ThreadedGenerator(threading.Thread):
"""Look-ahead generator class.
diff --git a/pywikibot/tools/formatter.py b/pywikibot/tools/formatter.py
index 271c1e2..d67d5ec 100644
--- a/pywikibot/tools/formatter.py
+++ b/pywikibot/tools/formatter.py
@@ -1,6 +1,6 @@
"""Module containing various formatting related utilities."""
#
-# (C) Pywikibot team, 2015-2020
+# (C) Pywikibot team, 2015-2021
#
# Distributed under the terms of the MIT license.
#
@@ -9,6 +9,7 @@
from typing import Any, Mapping, Sequence
from pywikibot.logging import output
+from pywikibot.tools import deprecated
from pywikibot.userinterfaces.terminal_interface_base import colors
@@ -40,7 +41,13 @@
super().__init__()
self.sequence = sequence
+ @deprecated('out', since='6.2.0', future_warning=True)
def format_list(self):
+ """DEPRECATED: Create the text with one item on each line."""
+ return self.out
+
+ @property
+ def out(self):
"""Create the text with one item on each line."""
if self.sequence:
# Width is only defined when the length is greater 0
@@ -52,9 +59,11 @@
content = ''
return self.prefix + content + self.suffix
+ @deprecated('pywikibot.output(SequenceOutputter.out)', since='6.2.0',
+ future_warning=True)
def output(self):
"""Output the text of the current sequence."""
- output(self.format_list())
+ output(self.out)
class _ColorFormatter(Formatter):
diff --git a/pywikibot/userinterfaces/_interface_base.py b/pywikibot/userinterfaces/_interface_base.py
index 856a2a3..b25b160 100644
--- a/pywikibot/userinterfaces/_interface_base.py
+++ b/pywikibot/userinterfaces/_interface_base.py
@@ -30,6 +30,12 @@
"""
return list(sys.argv)
+ def flush(self):
+ """Flush cached output.
+
+ May be passed to atexit.register() to flush any ui cache.
+ """
+
@abstractmethod
def init_handlers(self, *args, **kwargs):
"""Initialize the handlers for user output.
@@ -67,9 +73,3 @@
def output(self, *args, **kwargs) -> None:
"""Output text to a stream."""
print(*args, **kwargs) # noqa: T001
-
- def flush(self):
- """Flush cached output.
-
- May be passed to atexit.register() to flush any ui cache.
- """
diff --git a/pywikibot/userinterfaces/terminal_interface_base.py b/pywikibot/userinterfaces/terminal_interface_base.py
index 0c13fec..9db7fc3 100755
--- a/pywikibot/userinterfaces/terminal_interface_base.py
+++ b/pywikibot/userinterfaces/terminal_interface_base.py
@@ -13,7 +13,7 @@
import pywikibot
from pywikibot import config
-from pywikibot.backports import Sequence
+from pywikibot.backports import Sequence, SimpleQueue
from pywikibot.bot_choice import (
ChoiceException,
Option,
@@ -22,8 +22,9 @@
StandardOption,
)
from pywikibot.logging import INFO, INPUT, STDOUT, VERBOSE, WARNING
-from pywikibot.userinterfaces._interface_base import ABUIC
+from pywikibot.tools import RLock
from pywikibot.userinterfaces import transliteration
+from pywikibot.userinterfaces._interface_base import ABUIC
transliterator = transliteration.transliterator(config.console_encoding)
@@ -54,7 +55,11 @@
class UI(ABUIC):
- """Base for terminal user interfaces."""
+ """Base for terminal user interfaces.
+
+ *New in version 6.2:* subclassed from
+ L{pywikibot.userinterfaces._interface_base.ABUIC}.
+ """
split_col_pat = re.compile(r'(\w+);?(\w+)?')
@@ -74,6 +79,8 @@
self.stderr = sys.stderr
self.stdout = sys.stdout
+ self.cache = SimpleQueue()
+ self.lock = RLock()
def init_handlers(self, root_logger, default_stream='stderr'):
"""Initialize the handlers for user output.
@@ -183,12 +190,38 @@
self.encounter_color(color_stack[-1], target_stream)
def output(self, text, toStdout=False, targetStream=None):
+ """Forward text to cache and flush if output is not locked.
+
+ All input methods locks the output to a stream but collect them
+ in cache. They will be printed with next unlocked output call or
+ at termination time.
+ """
+ self.cache_output(text, toStdout, targetStream)
+ if not self.lock.locked():
+ self.flush()
+
+ def flush(self):
+ """Output cached text."""
+ while not self.cache.empty():
+ args, kwargs = self.cache.get_nowait()
+ self.stream_output(*args, **kwargs)
+
+ def cache_output(self, *args, **kwargs):
+ """Put text to cache.
+
+ *New in version 6.2*
+ """
+ self.cache.put_nowait((args, kwargs))
+
+ def stream_output(self, text, toStdout=False, targetStream=None):
"""
Output text to a stream.
If a character can't be displayed in the encoding used by the user's
terminal, it will be replaced with a question mark or by a
transliteration.
+
+ *New in version 6.2*
"""
if config.transliterate:
# Encode our unicode string in the encoding used by the user's
@@ -264,35 +297,39 @@
@param force: Automatically use the default
"""
assert(not password or not default)
- end_marker = ':'
+
question = question.strip()
- if question[-1] == ':':
+ end_marker = question[-1]
+ if end_marker in (':', '?'):
question = question[:-1]
- elif question[-1] == '?':
- question = question[:-1]
- end_marker = '?'
+ else:
+ end_marker = ':'
+
if default:
- question = question + ' (default: {})'.format(default)
- question = question + end_marker
- if force:
- self.output(question + '\n')
- return default
- # sound the terminal bell to notify the user
- if config.ring_bell:
- sys.stdout.write('\07')
- # TODO: make sure this is logged as well
- while True:
- self.output(question + ' ')
- text = self._input_reraise_cntl_c(password)
+ question += ' (default: {})'.format(default)
+ question += end_marker
- if text is None:
- continue
-
- if text:
- return text
-
- if default is not None:
+ # lock stream output
+ with self.lock:
+ if force:
+ self.stream_output(question + '\n')
return default
+ # sound the terminal bell to notify the user
+ if config.ring_bell:
+ sys.stdout.write('\07')
+ # TODO: make sure this is logged as well
+ while True:
+ self.stream_output(question + ' ')
+ text = self._input_reraise_cntl_c(password)
+
+ if text is None:
+ continue
+
+ if text:
+ return text
+
+ if default is not None:
+ return default
def _input_reraise_cntl_c(self, password):
"""Input and decode, and re-raise Control-C."""
@@ -362,23 +399,27 @@
# TODO: Test for uniquity
handled = False
- while not handled:
- for option in options:
- if isinstance(option, OutputOption) and option.before_question:
- option.output()
- output = Option.formatted(question, options, default)
- if force:
- self.output(output + '\n')
- answer = default
- else:
- answer = self.input(output) or default
- # something entered or default is defined
- if answer:
- for index, option in enumerate(options):
- if option.handled(answer):
- answer = option.result(answer)
- handled = option.stop
- break
+
+ # lock stream output
+ with self.lock:
+ while not handled:
+ for option in options:
+ if isinstance(option, OutputOption) \
+ and option.before_question:
+ self.stream_output(option.out)
+ output = Option.formatted(question, options, default)
+ if force:
+ self.stream_output(output + '\n')
+ answer = default
+ else:
+ answer = self.input(output) or default
+ # something entered or default is defined
+ if answer:
+ for index, option in enumerate(options):
+ if option.handled(answer):
+ answer = option.result(answer)
+ handled = option.stop
+ break
if isinstance(answer, ChoiceException):
raise answer
@@ -398,30 +439,34 @@
@param force: Automatically use the default.
@return: Return a single Sequence entry.
"""
- if not force:
- line_template = '{{0: >{}}}: {{1}}'.format(len(str(len(answers))))
- for i, entry in enumerate(answers, start=1):
- pywikibot.output(line_template.format(i, entry))
+ # lock stream output
+ with self.lock:
+ if not force:
+ line_template = '{{0: >{}}}: {{1}}'.format(
+ len(str(len(answers))))
+ for i, entry in enumerate(answers, start=1):
+ pywikibot.stream_output(line_template.format(i, entry))
- while True:
- choice = self.input(question, default=default, force=force)
+ while True:
+ choice = self.input(question, default=default, force=force)
- try:
- choice = int(choice) - 1
- except (TypeError, ValueError):
- if choice in answers:
- return choice
- choice = -1
+ try:
+ choice = int(choice) - 1
+ except (TypeError, ValueError):
+ if choice in answers:
+ return choice
+ choice = -1
- # User typed choice number
- if 0 <= choice < len(answers):
- return answers[choice]
+ # User typed choice number
+ if 0 <= choice < len(answers):
+ return answers[choice]
- if force:
- raise ValueError('Invalid value "{}" for default during force.'
- .format(default))
+ if force:
+ raise ValueError(
+ 'Invalid value "{}" for default during force.'
+ .format(default))
- pywikibot.error('Invalid response')
+ pywikibot.error('Invalid response')
def editText(self, text: str, jumpIndex: Optional[int] = None,
highlight: Optional[str] = None):
diff --git a/scripts/category.py b/scripts/category.py
index 53b2865..1fea6ec 100755
--- a/scripts/category.py
+++ b/scripts/category.py
@@ -1012,19 +1012,24 @@
class CatContextOption(ContextOption):
"""An option to show more and more context and categories."""
- def output_range(self, start, end) -> None:
- """Output a section and categories from the text."""
- pywikibot.output(self.text[start:end] + '...')
+ @property
+ def out(self) -> str:
+ """Create a section and categories from the text."""
+ start = max(0, self.start - self.context)
+ end = min(len(self.text), self.end + self.context)
+ text = self.text[start:end] + '...'
# if categories weren't visible, show them additionally
if len(self.text) > end:
for cat in member.categories():
if cat != original_cat:
- pywikibot.output(cat.title(as_link=True))
+ text += cat.title(as_link=True)
else:
- pywikibot.output(color_format(
+ text += color_format(
'{lightpurple}{0}{default}',
- current_cat.title(as_link=True)))
+ current_cat.title(as_link=True))
+ text += '\n'
+ return text
class CatIntegerOption(IntegerOption):
"""An option allowing a range of integers."""
diff --git a/scripts/replace.py b/scripts/replace.py
index 513010e..41c9c4c 100755
--- a/scripts/replace.py
+++ b/scripts/replace.py
@@ -146,7 +146,6 @@
import re
from collections.abc import Sequence
from contextlib import suppress
-from queue import Queue
import pywikibot
from pywikibot import editor, fixes, i18n, pagegenerators, textlib
@@ -550,8 +549,6 @@
if self.opt.addcat and isinstance(self.opt.addcat, str):
self.opt.addcat = pywikibot.Category(self.site, self.opt.addcat)
- self._pending_processed_titles = Queue()
-
def isTitleExcepted(self, title, exceptions=None) -> bool:
"""Return True if one of the exceptions applies for the given title."""
if exceptions is None:
@@ -618,16 +615,6 @@
return new_text
- def _replace_async_callback(self, page, err):
- """Log changed titles for display."""
- # This is an async put callback
- if not isinstance(err, Exception):
- self._pending_processed_titles.put((page.title(
- as_link=True), True))
- else: # unsuccessful pages
- self._pending_processed_titles.put((page.title(as_link=True),
- False))
-
def generate_summary(self, applied_replacements):
"""Generate a summary message for the replacements."""
# all replacements which are merged into the default message
@@ -751,14 +738,8 @@
self.opt.always = True
if choice == 'y':
self.save(page, original_text, new_text, applied,
- show_diff=False, quiet=True,
- callback=self._replace_async_callback,
- asynchronous=True)
- while not self._pending_processed_titles.empty():
- proc_title, res = self._pending_processed_titles.get()
- pywikibot.output('Page {}{} saved'
- .format(proc_title,
- '' if res else ' not'))
+ show_diff=False, asynchronous=True)
+
# choice must be 'N'
break
@@ -1081,9 +1062,7 @@
gen = pagegenerators.MySQLPageGenerator(query)
gen = genFactory.getCombinedGenerator(gen, preload=True)
-
- if not gen:
- pywikibot.bot.suggest_help(missing_generator=True)
+ if pywikibot.bot.suggest_help(missing_generator=not gen):
return
bot = ReplaceRobot(gen, replacements, exceptions, site=site,
@@ -1091,10 +1070,6 @@
site.login()
bot.run()
- # Explicitly call pywikibot.stopme(). It will make sure the callback is
- # triggered before replace.py is unloaded.
- pywikibot.stopme()
-
if __name__ == '__main__':
main()
diff --git a/tests/i18n_tests.py b/tests/i18n_tests.py
index e67e0d4..2c9bc72 100644
--- a/tests/i18n_tests.py
+++ b/tests/i18n_tests.py
@@ -514,7 +514,7 @@
self.orig_raw_input = bot.ui._raw_input
self.orig_output = bot.ui.output
bot.ui._raw_input = lambda *args, **kwargs: 'dummy input'
- bot.ui.output = self._capture_output
+ bot.ui.stream_output = self._capture_output
self.old_cc_setting = config.cosmetic_changes_mylang_only
def tearDown(self):
--
To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/282818
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Change-Id: I02d6b7a939a1912a4ebb8694af3e159d7dd7dae5
Gerrit-Change-Number: 282818
Gerrit-PatchSet: 38
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: JJMC89 <JJMC89.Wikimedia(a)gmail.com>
Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: Matěj Suchánek <matejsuchanek97(a)gmail.com>
Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl>
Gerrit-Reviewer: Mpaa <mpaa.wiki(a)gmail.com>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot
Gerrit-CC: Zhuyifei1999 <zhuyifei1999(a)gmail.com>
Gerrit-MessageType: merged
jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/687057 )
Change subject: [IMPR] Add abstract base user interface module to the framework
......................................................................
[IMPR] Add abstract base user interface module to the framework
bot module calls the user interface to be used.
in compat release we had additional interfacec (cgi, tkinter, wxpython)
but all of them failed due to missing interface methods. Therefore add
a abstract base user interface class to ensure that each interface
method is provided. This probably would allow us to implement an
interface for tests and enable assertLogs/assertNoLogs unittest methods
Change-Id: Ic425fe52264b57c44fe97a0a89b9e6260944a8c7
---
M docs/api_ref/pywikibot.userinterfaces.rst
M pywikibot/CONTENT.rst
A pywikibot/userinterfaces/_interface_base.py
M pywikibot/userinterfaces/terminal_interface_base.py
4 files changed, 87 insertions(+), 4 deletions(-)
Approvals:
Xqt: Looks good to me, approved
jenkins-bot: Verified
diff --git a/docs/api_ref/pywikibot.userinterfaces.rst b/docs/api_ref/pywikibot.userinterfaces.rst
index 486ae00..6d0b506 100644
--- a/docs/api_ref/pywikibot.userinterfaces.rst
+++ b/docs/api_ref/pywikibot.userinterfaces.rst
@@ -3,10 +3,10 @@
.. automodule:: pywikibot.userinterfaces
-gui module
-----------
+\_interface\_base module
+------------------------
-.. automodule:: pywikibot.userinterfaces.gui
+.. automodule:: pywikibot.userinterfaces._interface_base
terminal\_interface module
--------------------------
@@ -28,6 +28,11 @@
.. automodule:: pywikibot.userinterfaces.terminal_interface_win32
+gui module
+----------
+
+.. automodule:: pywikibot.userinterfaces.gui
+
transliteration module
----------------------
diff --git a/pywikibot/CONTENT.rst b/pywikibot/CONTENT.rst
index 5929d91..7f977f5 100644
--- a/pywikibot/CONTENT.rst
+++ b/pywikibot/CONTENT.rst
@@ -172,6 +172,8 @@
+-----------------------------------------------------------------------------------+
| User Interface |
+============================+======================================================+
+ | _interface_base.py | Abstract base user interface module |
+ +----------------------------+------------------------------------------------------+
| gui.py | GUI with a unicode textfield where the user can edit |
+----------------------------+------------------------------------------------------+
| terminal_interface.py | Platform independent terminal interface module |
diff --git a/pywikibot/userinterfaces/_interface_base.py b/pywikibot/userinterfaces/_interface_base.py
new file mode 100644
index 0000000..856a2a3
--- /dev/null
+++ b/pywikibot/userinterfaces/_interface_base.py
@@ -0,0 +1,75 @@
+"""Abstract base user interface module.
+
+*New in version 6.2.*
+"""
+#
+# (C) Pywikibot team, 2021
+#
+# Distributed under the terms of the MIT license.
+#
+import sys
+
+from abc import ABC, abstractmethod
+from typing import Any, Union
+
+from pywikibot.backports import List
+
+
+class ABUIC(ABC):
+
+ """Abstract base user interface class.
+
+ Every user interface should derive from it to ensure that all
+ required methods are implemented.
+ """
+
+ def argvu(self) -> List[str]:
+ """Return copy of sys.argv.
+
+ Assigned to pywikibot.argvu in bot module
+ """
+ return list(sys.argv)
+
+ @abstractmethod
+ def init_handlers(self, *args, **kwargs):
+ """Initialize the handlers for user output.
+
+ Called in bot.init_handlers().
+ """
+
+ @abstractmethod
+ def input(self, *args, **kwargs) -> str:
+ """Ask the user a question and return the answer.
+
+ Called by bot.input().
+ """
+ if args:
+ return input(args[0])
+ return input()
+
+ @abstractmethod
+ def input_choice(self, *args, **kwargs) -> Union[int, str]:
+ """Ask the user and returns a value from the options.
+
+ Called by bot.input_choice().
+ """
+ return self.input()
+
+ @abstractmethod
+ def input_list_choice(self, *args, **kwargs) -> Any:
+ """Ask the user to select one entry from a list of entries.
+
+ Called by bot.input_list_choice().
+ """
+ return self.input()
+
+ @abstractmethod
+ def output(self, *args, **kwargs) -> None:
+ """Output text to a stream."""
+ print(*args, **kwargs) # noqa: T001
+
+ def flush(self):
+ """Flush cached output.
+
+ May be passed to atexit.register() to flush any ui cache.
+ """
diff --git a/pywikibot/userinterfaces/terminal_interface_base.py b/pywikibot/userinterfaces/terminal_interface_base.py
index dfbb0ac..0c13fec 100755
--- a/pywikibot/userinterfaces/terminal_interface_base.py
+++ b/pywikibot/userinterfaces/terminal_interface_base.py
@@ -22,6 +22,7 @@
StandardOption,
)
from pywikibot.logging import INFO, INPUT, STDOUT, VERBOSE, WARNING
+from pywikibot.userinterfaces._interface_base import ABUIC
from pywikibot.userinterfaces import transliteration
@@ -51,7 +52,7 @@
colorTagR = re.compile('\03{((:?%s);?(:?%s)?)}' % (_color_pat, _color_pat))
-class UI:
+class UI(ABUIC):
"""Base for terminal user interfaces."""
--
To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/687057
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Change-Id: Ic425fe52264b57c44fe97a0a89b9e6260944a8c7
Gerrit-Change-Number: 687057
Gerrit-PatchSet: 7
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot
Gerrit-MessageType: merged
jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/687203 )
Change subject: [IO] remove outdated register_cp65001() function
......................................................................
[IO] remove outdated register_cp65001() function
cp65001 is an alias for utf-8 since Python 3.8
and was added to the codecs table in Python 3.3 already
https://docs.python.org/3/library/codecs.htmlhttps://docs.python.org/3.7/library/codecs.html
Change-Id: I35277ca3d4a018cc4673c5ac84a8e89e4026cfd0
---
M pywikibot/userinterfaces/win32_unicode.py
1 file changed, 2 insertions(+), 13 deletions(-)
Approvals:
Xqt: Looks good to me, approved
jenkins-bot: Verified
diff --git a/pywikibot/userinterfaces/win32_unicode.py b/pywikibot/userinterfaces/win32_unicode.py
index 7962408..4b858de 100755
--- a/pywikibot/userinterfaces/win32_unicode.py
+++ b/pywikibot/userinterfaces/win32_unicode.py
@@ -1,6 +1,6 @@
"""Stdout, stderr and argv support for unicode."""
#
-# (C) Pywikibot team, 2012-2018
+# (C) Pywikibot team, 2012-2021
#
##############################################
# Support for unicode in Windows cmd.exe
@@ -21,8 +21,8 @@
# Licensed under both CC-BY-SA and the MIT license.
#
################################################
-import codecs
import sys
+
from contextlib import suppress
from ctypes import Structure, byref
from ctypes import c_void_p as LPVOID
@@ -172,13 +172,6 @@
file=original_stderr)
-def register_cp65001():
- """Register codecs cp65001 as utf-8."""
- # Work around <https://bugs.python.org/issue6058>
- codecs.register(lambda name: name == 'cp65001'
- and codecs.lookup('utf-8') or None)
-
-
def force_truetype_console(h_stdout):
"""Force the console to use a TrueType font (Vista+)."""
TMPF_TRUETYPE = 0x04
@@ -331,7 +324,3 @@
.format(e))
return stdin, stdout, stderr, argv
-
-
-if OSWIN32:
- register_cp65001()
--
To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/687203
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Change-Id: I35277ca3d4a018cc4673c5ac84a8e89e4026cfd0
Gerrit-Change-Number: 687203
Gerrit-PatchSet: 3
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot
Gerrit-MessageType: merged
jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/691263 )
Change subject: [doc] Update ROADMAP.rst, update doc strings
......................................................................
[doc] Update ROADMAP.rst, update doc strings
Change-Id: I7dea0f031379ac563fe1f19a6946a177edce461f
---
M ROADMAP.rst
M pywikibot/pagegenerators.py
M pywikibot/site/_generators.py
3 files changed, 15 insertions(+), 6 deletions(-)
Approvals:
Xqt: Looks good to me, approved
jenkins-bot: Verified
diff --git a/ROADMAP.rst b/ROADMAP.rst
index a688629..1f6e461 100644
--- a/ROADMAP.rst
+++ b/ROADMAP.rst
@@ -1,9 +1,15 @@
Current release changes
^^^^^^^^^^^^^^^^^^^^^^^
+* APISite method pagelanglinks() may skip links with empty titles (T223157)
+* User.name() method was removed in favour of User.username property
+* BasePage.getLatestEditors() method was removed in favour of contributors() or revisions()
+* pagenenerators.handleArg() method was renamed to handle_arg() (T271437)
+* CategoryGenerator, FileGenerator, ImageGenerator and ReferringPageGenerator pagegenerator functions were removed
+* Family.ignore_certificate_error() method was removed in favour of verify_SSL_certificate (T265205)
* tools.is_IP was renamed to is_ip_address due to PEP8
* Fix Page.getDeletedRevision() method which always returned an empty list
-* Async chunked uploads are supported (T129216, 133443)
+* Async chunked uploads are supported (T129216, T133443)
* A new InvalidPageError will be raised if a Page has no version history (T280043)
* config2.py was renamed to config.py
* L10N updates
@@ -26,9 +32,5 @@
* 6.0.1: Site.undeletepage() and Site.undelete_file_versions() will be removed in favour of Site.undelete() method
* 6.0.1: Site.deletepage() and Site.deleteoldimage() will be removed in favour of Site.delete() method
* 6.0.1: DataSite.createNewItemFromPage() method will be removed in favour of ImagePage.fromPage() (T98663)
-* 6.0.0: User.name() method will be removed in favour of User.username property
-* 5.6.0: pagenenerators.handleArg() method will be removed in favour of handle_arg() (T271437)
-* 5.6.0: Family.ignore_certificate_error() method will be removed in favour of verify_SSL_certificate() (T265205)
* 5.0.0: OptionHandler.options dict will be removed in favour of OptionHandler.opt
* 5.0.0: Methods deprecated for 5 years or longer will be removed
-* 5.0.0: pagegenerators.ReferringPageGenerator is desupported and will be removed
diff --git a/pywikibot/pagegenerators.py b/pywikibot/pagegenerators.py
index a8c635a..a992b3c 100644
--- a/pywikibot/pagegenerators.py
+++ b/pywikibot/pagegenerators.py
@@ -1215,7 +1215,10 @@
return self._parse_log_events(*params)
def handle_args(self, args: Iterable[str]) -> List[str]:
- """Handle command line arguments and return the rest as a list."""
+ """Handle command line arguments and return the rest as a list.
+
+ *New in version 6.0.*
+ """
return [arg for arg in args if not self.handle_arg(arg)]
def handle_arg(self, arg: str) -> bool:
@@ -1227,6 +1230,8 @@
can try parsing the argument. Call getCombinedGenerator() after all
arguments have been parsed to get the final output generator.
+ *Renamed in version 6.0.*
+
@param arg: Pywikibot argument consisting of -name:value
@return: True if the argument supplied was recognised by the factory
"""
diff --git a/pywikibot/site/_generators.py b/pywikibot/site/_generators.py
index 475d71b..cfa7bf8 100644
--- a/pywikibot/site/_generators.py
+++ b/pywikibot/site/_generators.py
@@ -736,6 +736,8 @@
include_empty_titles: bool = False):
"""Iterate all interlanguage links on page, yielding Link objects.
+ *New in 6.2:* *include_empty_titles* parameter was added.
+
@see: U{https://www.mediawiki.org/wiki/API:Langlinks}
@param include_obsolete: if true, yield even Link objects whose
--
To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/691263
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Change-Id: I7dea0f031379ac563fe1f19a6946a177edce461f
Gerrit-Change-Number: 691263
Gerrit-PatchSet: 1
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot
Gerrit-MessageType: merged
jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/691149 )
Change subject: [IMPR] pagelanglinks() may skip links with empty titles.
......................................................................
[IMPR] pagelanglinks() may skip links with empty titles.
langlinks API call may retrieve site links with empty titles like
[[en:]]. Introduce a new parameter include_empty_titles (default False)
to ignore such links because it cannot be upcasted to a Page object
but link.astitle() would work.
raise an InvalidTitleError if a Page is to be created with empty title
raise an InvalidPageError if a Page has no pageid
Bug: T223157
Change-Id: Iab294e318ff5412ef9a5b7485afcfb85600d531f
---
M pywikibot/page/__init__.py
M pywikibot/site/_generators.py
2 files changed, 19 insertions(+), 4 deletions(-)
Approvals:
Meno25: Looks good to me, approved
jenkins-bot: Verified
diff --git a/pywikibot/page/__init__.py b/pywikibot/page/__init__.py
index 390e23a..5553d95 100644
--- a/pywikibot/page/__init__.py
+++ b/pywikibot/page/__init__.py
@@ -179,6 +179,13 @@
self._link = Link(title, source=source.site,
default_namespace=ns)
elif isinstance(source, BaseLink):
+ if not source.title:
+ raise InvalidTitleError(
+ '{} title of {} {} cannot be empty.'
+ .format(self.__class__.__name__,
+ source.__class__.__name__,
+ source.astext()))
+
self._link = source
self._revisions = {}
else:
@@ -707,7 +714,9 @@
If the title includes a section, return False if this section isn't
found.
"""
- return self.pageid > 0
+ with suppress(AttributeError):
+ return self.pageid > 0
+ raise InvalidPageError(self)
@property
def oldest_revision(self):
diff --git a/pywikibot/site/_generators.py b/pywikibot/site/_generators.py
index 5bd2755..475d71b 100644
--- a/pywikibot/site/_generators.py
+++ b/pywikibot/site/_generators.py
@@ -730,13 +730,18 @@
api.update_page(page, pagedata, rvgen.props)
@deprecated_args(step=True)
- def pagelanglinks(self, page, *, total=None, include_obsolete=False):
+ def pagelanglinks(self, page, *,
+ total: Optional[int] = None,
+ include_obsolete: bool = False,
+ include_empty_titles: bool = False):
"""Iterate all interlanguage links on page, yielding Link objects.
@see: U{https://www.mediawiki.org/wiki/API:Langlinks}
@param include_obsolete: if true, yield even Link objects whose
- site is obsolete
+ site is obsolete
+ @param include_empty_titles: if true, yield even Link objects whose
+ title is empty but redirects to a site like [[en:]]
"""
lltitle = page.title(with_section=False)
llquery = self._generator(api.PropertyGenerator,
@@ -755,7 +760,8 @@
if link.site.obsolete and not include_obsolete:
continue
- yield link
+ if link.title or include_empty_titles:
+ yield link
@deprecated_args(step=True)
def page_extlinks(self, page, *, total=None):
--
To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/691149
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Change-Id: Iab294e318ff5412ef9a5b7485afcfb85600d531f
Gerrit-Change-Number: 691149
Gerrit-PatchSet: 4
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: JJMC89 <JJMC89.Wikimedia(a)gmail.com>
Gerrit-Reviewer: Matěj Suchánek <matejsuchanek97(a)gmail.com>
Gerrit-Reviewer: Meno25 <meno25mail(a)gmail.com>
Gerrit-Reviewer: jenkins-bot
Gerrit-MessageType: merged
jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/689982 )
Change subject: [docs] Update some docs fo sphinx
......................................................................
[docs] Update some docs fo sphinx
Change-Id: I9fdbe2aef30ed1992568ff4fe45104630add04ab
---
M pywikibot/bot.py
M pywikibot/config.py
M pywikibot/cosmetic_changes.py
M pywikibot/textlib.py
M pywikibot/throttle.py
5 files changed, 79 insertions(+), 68 deletions(-)
Approvals:
Xqt: Looks good to me, approved
jenkins-bot: Verified
diff --git a/pywikibot/bot.py b/pywikibot/bot.py
index 0d90522..e1bc0e3 100644
--- a/pywikibot/bot.py
+++ b/pywikibot/bot.py
@@ -1742,19 +1742,19 @@
All options must be predefined in available_options dictionary. The type
of these options is responsible for the correct interpretation of the
options type given by the .ini file. They can be interpreted as bool,
- int, float or str (default). The settings file may be like:
+ int, float or str (default). The settings file may be like::
- [add_text]
- # edit summary for the bot.
- summary = Bot: Aggiungo template Categorizzare
+ [add_text]
+ # edit summary for the bot.
+ summary = Bot: Aggiungo template Categorizzare
- [shell] ; Shell options
- always: true
+ [shell] ; Shell options
+ always: true
The option values are interpreted in this order::
- - available_options default setting
- - script.ini options settings
+ - `available_options` default setting
+ - `script.ini options` settings
- command line arguments
*New in version 3.0.*
diff --git a/pywikibot/config.py b/pywikibot/config.py
index be1002e..fc2f8f2 100644
--- a/pywikibot/config.py
+++ b/pywikibot/config.py
@@ -1,20 +1,20 @@
"""
Module to define and load pywikibot configuration default and user preferences.
-User preferences are loaded from a python file called user-config.py, which
-may be located in directory specified by the environment variable
-PYWIKIBOT_DIR, or the same directory as pwb.py, or in a directory within
-the users home. See get_base_dir for more information.
+User preferences are loaded from a python file called `user-config.py`,
+which may be located in directory specified by the environment variable
+`PYWIKIBOT_DIR`, or the same directory as `pwb.py`, or in a directory
+within the users home. See L{get_base_dir} for more information.
If user-config.py cannot be found in any of those locations, this module
-will fail to load unless the environment variable PYWIKIBOT_NO_USER_CONFIG
-is set to a value other than '0'. i.e. PYWIKIBOT_NO_USER_CONFIG=1 will
-allow config to load without a user-config.py. However, warnings will be
-shown if user-config.py was not loaded. To prevent these warnings, set
-PYWIKIBOT_NO_USER_CONFIG=2. If pywikibot is installed as a site-package
-the behaviour is like PYWIKIBOT_NO_USER_CONFIG=2 is set.
+will fail to load unless the environment variable `PYWIKIBOT_NO_USER_CONFIG`
+is set to a value other than `'0'`. i.e. `PYWIKIBOT_NO_USER_CONFIG=1` will
+allow config to load without a `user-config.py`. However, warnings will be
+shown if `user-config.py` was not loaded. To prevent these warnings, set
+`PYWIKIBOT_NO_USER_CONFIG=2`. If Pywikibot is installed as a site-package
+the behaviour is like `PYWIKIBOT_NO_USER_CONFIG=2` is set.
-Functions made available to user-config:
+Functions made available to `user-config`:
- user_home_path
@@ -283,20 +283,21 @@
r"""Return the directory in which user-specific information is stored.
This is determined in the following order:
- 1. If the script was called with a -dir: argument, use the directory
- provided in this argument.
- 2. If the user has a PYWIKIBOT_DIR environment variable, use the value
- of it.
- 3. If user-config is present in current directory, use the current
- directory.
- 4. If user-config is present in pwb.py directory, use that directory
- 5. Use (and if necessary create) a 'pywikibot' folder under
- 'Application Data' or 'AppData\Roaming' (Windows) or
- '.pywikibot' directory (Unix and similar) under the user's home
- directory.
+ 1. If the script was called with a `-dir:` argument, use the
+ directory provided in this argument.
+ 2. If the user has a `PYWIKIBOT_DIR` environment variable, use the
+ value of it.
+ 3. If `user-config` is present in current directory, use the
+ current directory.
+ 4. If `user-config` is present in `pwb.py` directory, use that
+ directory
+ 5. Use (and if necessary create) a `'pywikibot'` folder under
+ `'Application Data'` or `'AppData\Roaming'` (Windows) or
+ `'.pywikibot'` directory (Unix and similar) under the user's
+ home directory.
- Set PYWIKIBOT_NO_USER_CONFIG=1 to disable loading user-config.py or
- install pywikibot as a site-package.
+ Set `PYWIKIBOT_NO_USER_CONFIG=1` to disable loading `user-config.py`
+ or install Pywikibot as a site-package.
@param test_directory: Assume that a user config file exists in this
directory. Used to test whether placing a user config file in this
diff --git a/pywikibot/cosmetic_changes.py b/pywikibot/cosmetic_changes.py
index da46f8a..8d2270c 100755
--- a/pywikibot/cosmetic_changes.py
+++ b/pywikibot/cosmetic_changes.py
@@ -4,11 +4,11 @@
The changes are not supposed to change the look of the rendered wiki page.
-If you wish to run this as an stand-alone script, use:
+If you wish to run this as an stand-alone script, use::
scripts/cosmetic_changes.py
-For regular use, it is recommended to put this line into your user-config.py:
+For regular use, it is recommended to put this line into your user-config.py::
cosmetic_changes = True
@@ -16,11 +16,11 @@
dictionary cosmetic_changes_enable to your user-config.py. It should contain
a tuple of languages for each site where you wish to enable in addition to
your own langlanguage if cosmetic_changes_mylang_only is True (see below).
-Please set your dictionary by adding such lines to your user-config.py:
+Please set your dictionary by adding such lines to your user-config.py::
cosmetic_changes_enable['wikipedia'] = ('de', 'en', 'fr')
-There is another config variable: You can set
+There is another config variable: You can set::
cosmetic_changes_mylang_only = False
@@ -33,7 +33,7 @@
You may use it with cosmetic_changes_mylang_only is False, but you can also
disable your own language. This also overrides the settings in the dictionary
cosmetic_changes_enable. Please set this dictionary by adding such lines to
-your user-config.py:
+your user-config.py::
cosmetic_changes_disable['wikipedia'] = ('de', 'en', 'fr')
@@ -41,11 +41,11 @@
unwanted scripts to the list cosmetic_changes_deny_script in your
user-config.py. By default it contains cosmetic_changes.py itself and touch.py.
This overrides all other enabling settings for cosmetic changes. Please modify
-the given list by adding such lines to your user-config.py:
+the given list by adding such lines to your user-config.py::
cosmetic_changes_deny_script.append('your_script_name_1')
-or by adding a list to the given one:
+or by adding a list to the given one::
cosmetic_changes_deny_script += ['your_script_name_1',
'your_script_name_2']
@@ -766,12 +766,19 @@
"""
Add a space between the equal signs and the section title.
- Example: ==Section title== becomes == Section title ==
+ Example::
- NOTE: This space is recommended in the syntax help on the English and
- German Wikipedia. It is not wanted on Lojban and English Wiktionary
- (T168399, T169064) and it might be that it is not wanted on other
- wikis. If there are any complaints, please file a bug report.
+ ==Section title==
+
+ becomes::
+
+ == Section title ==
+
+ :NOTE: This space is recommended in the syntax help on the
+ English and German Wikipedia. It is not wanted on Lojban and
+ English Wiktionary (T168399, T169064) and it might be that
+ it is not wanted on other wikis. If there are any complaints,
+ please file a bug report.
"""
if self.site.sitename in ['wiktionary:jbo', 'wiktionary:en']:
return text
@@ -785,9 +792,10 @@
"""
Add a space between the * or # and the text.
- NOTE: This space is recommended in the syntax help on the English,
- German, and French Wikipedia. It might be that it is not wanted on
- other wikis. If there are any complaints, please file a bug report.
+ :NOTE: This space is recommended in the syntax help on the
+ English, German, and French Wikipedia. It might be that it
+ is not wanted on other wikis. If there are any complaints,
+ please file a bug report.
"""
if not self.template:
exceptions = ['comment', 'math', 'nowiki', 'pre',
diff --git a/pywikibot/textlib.py b/pywikibot/textlib.py
index 48cce6b..b278865 100644
--- a/pywikibot/textlib.py
+++ b/pywikibot/textlib.py
@@ -1596,16 +1596,17 @@
There are minor differences between the two implementations.
The two implementations return nested templates in a different
- order, i.e. for {{a|b={{c}}}}, parsers returns [a, c], whereas regex
- returns [c, a].
+ order, i.e. for `{{a|b={{c}}}}`, parsers returns `[a, c]`, whereas
+ regex returns `[c, a]`.
The parser packages preserves whitespace in parameter names and
values.
- If there are multiple numbered parameters in the wikitext for the same
- position, MediaWiki will only use the last parameter value.
- e.g. {{a| foo | 2 <!-- --> = bar | baz }} is {{a|1=foo|2=baz}}
- To replicate that behaviour, enable both remove_disabled_parts and strip.
+ If there are multiple numbered parameters in the wikitext for the
+ same position, MediaWiki will only use the last parameter value.
+ e.g. `{{a| foo | 2 <!-- --> = bar | baz }}` is `{{a|1=foo|2=baz}}`
+ To replicate that behaviour, enable both `remove_disabled_parts`
+ and `strip` parameters.
@param text: The wikitext from which templates are extracted
@param remove_disabled_parts: If enabled, remove disabled wikitext
diff --git a/pywikibot/throttle.py b/pywikibot/throttle.py
index e2a5672..33dff52 100644
--- a/pywikibot/throttle.py
+++ b/pywikibot/throttle.py
@@ -32,11 +32,11 @@
"""Control rate of access to wiki server.
- Calling this object blocks the calling thread until at least 'delay'
- seconds have passed since the previous call.
+ Calling this object blocks the calling thread until at least
+ `'delay'` seconds have passed since the previous call.
- Each Site initiates one Throttle object (site.throttle) to control the
- rate of access.
+ Each Site initiates one Throttle object (`site.throttle`) to control
+ the rate of access.
"""
@@ -256,20 +256,21 @@
def lag(self, lagtime: Optional[float] = None):
"""Seize the throttle lock due to server lag.
- Usually the self.retry-after value from response_header of the last
- request if available which will be used for wait time. Otherwise
- lagtime from api maxlag is used. If neither retry_after nor lagtime is
- set, fallback to config.retry_wait.
+ Usually the `self.retry-after` value from `response_header` of the
+ last request if available which will be used for wait time.
+ Otherwise `lagtime` from api `maxlag` is used. If neither
+ `self.retry_after` nor `lagtime` is set, fallback to
+ `config.retry_wait`.
- If the lagtime is disproportionately high compared to retry-after
- value, the wait time will be increased.
+ If the `lagtime` is disproportionately high compared to
+ `self.retry_after` value, the wait time will be increased.
- This method is used by api.request. It will prevent any thread from
- accessing this site.
+ This method is used by `api.request`. It will prevent any thread
+ from accessing this site.
- @param lagtime: The time to wait for the next request which is the
- last maxlag time from api warning. This is only used as a fallback
- if self.retry-after isn't set.
+ @param lagtime: The time to wait for the next request which is
+ the last `maxlag` time from api warning. This is only used
+ as a fallback if `self.retry_after` isn't set.
"""
started = time.time()
with self.lock:
--
To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/689982
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Change-Id: I9fdbe2aef30ed1992568ff4fe45104630add04ab
Gerrit-Change-Number: 689982
Gerrit-PatchSet: 1
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot
Gerrit-MessageType: merged
jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/689940 )
Change subject: [doc] add pwb.py, generate_user_files.py and generate_family_files to sphinx
......................................................................
[doc] add pwb.py, generate_user_files.py and generate_family_files to sphinx
Change-Id: Ia159179b88a1de40f6a7895cfb1c2812669e200b
---
M docs/index.rst
A docs/utilities/index.rst
M pwb.py
3 files changed, 21 insertions(+), 3 deletions(-)
Approvals:
Xqt: Looks good to me, approved
jenkins-bot: Verified
diff --git a/docs/index.rst b/docs/index.rst
index 2e819d1..dd55ccd 100644
--- a/docs/index.rst
+++ b/docs/index.rst
@@ -49,6 +49,7 @@
:maxdepth: 1
installation
+ utilities/index
scripts/index
getting_help
diff --git a/docs/utilities/index.rst b/docs/utilities/index.rst
new file mode 100644
index 0000000..96131fa
--- /dev/null
+++ b/docs/utilities/index.rst
@@ -0,0 +1,17 @@
+Utility scripts
+===============
+
+pwb wrapper script
+------------------
+
+.. automodule:: pwb
+
+generate\_family\_file script
+-----------------------------
+
+.. automodule:: generate_family_file
+
+generate\_user\_files script
+----------------------------
+
+.. automodule:: generate_user_files
diff --git a/pwb.py b/pwb.py
index 98362e0..9b4873c 100755
--- a/pwb.py
+++ b/pwb.py
@@ -2,15 +2,15 @@
# -*- coding: utf-8 -*-
"""Wrapper script to invoke pywikibot-based scripts.
-Run scripts with pywikibot in directory mode using:
+Run scripts with pywikibot in directory mode using::
python pwb.py <pwb options> <name_of_script> <options>
This wrapper script uses the package directory to store all user files,
will fix up search paths so the package does not need to be installed, etc.
-Currently <pwb options> are global options. This can be used for tests
-to set the default site (see T216825):
+Currently `<pwb options>` are global options. This can be used for tests
+to set the default site (see T216825)::
python pwb.py -lang:de bot_tests -v
"""
--
To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/689940
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Change-Id: Ia159179b88a1de40f6a7895cfb1c2812669e200b
Gerrit-Change-Number: 689940
Gerrit-PatchSet: 1
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot
Gerrit-MessageType: merged