Pywikibot-commits January 2021

pywikibot-commits@lists.wikimedia.org

1 participants
131 discussions

[Gerrit] ...core[master]: [doc] update ROADMAP.rst

by jenkins-bot (Code Review)

jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/656394 ) Change subject: [doc] update ROADMAP.rst ...................................................................... [doc] update ROADMAP.rst Also increase version number to 5.6 Change-Id: I46cd41f027b6ed1fefe20e6a1282fef74a774cec --- M .appveyor.yml M ROADMAP.rst M pywikibot/__metadata__.py 3 files changed, 9 insertions(+), 4 deletions(-) Approvals: Xqt: Looks good to me, approved jenkins-bot: Verified diff --git a/.appveyor.yml b/.appveyor.yml index 021ca49..bf53c50 100644 --- a/.appveyor.yml +++ b/.appveyor.yml @@ -1,6 +1,6 @@ clone_depth: 50 skip_tags: true -version: 5.5.{build} +version: 5.6.{build} environment: APPVEYOR_PYTHON_URL: "https://raw.githubusercontent.com/dvorapa/python-appveyor-demo/master/appve…" diff --git a/ROADMAP.rst b/ROADMAP.rst index 279248e..a622d27 100644 --- a/ROADMAP.rst +++ b/ROADMAP.rst @@ -1,11 +1,17 @@ Current release changes ~~~~~~~~~~~~~~~~~~~~~~~ -* (no changes yet) +* data attribute of http.fetch() response is deprecated (T265206) +* Positional arguments of page.Revision aren't supported any longer (T259428) +* pagenenerators.handleArg() method was renamed to handle_arg() (T271437) +* Page methods deprecated for 6 years were removed +* Create a Site with AutoFamily if a family isn't predefined (T249087) Future release notes ~~~~~~~~~~~~~~~~~~~~ +* 5.6.0: APISite.loadimageinfo will no longer return any content +* 5.6.0: pagenenerators.handleArg() method will be removed in favour of handle_arg() (T271437) * 5.5.0: Site.getuserinfo() method will be dropped in favour of userinfo property * 5.5.0: Site.getglobaluserinfo() method will be dropped in favour of globaluserinfo property * 5.4.0: Support of MediaWiki < 1.23 will be dropped with release 6.0 (T268979) @@ -14,7 +20,6 @@ * 5.3.0: api.ParamInfo.modules property will be removed * 5.3.0: LogEntryFactory.logtypes property will be removed * 5.3.0: stdout parameter of logging.output()/pywikibot.output() function will be desupported -* 5.1.0: Positional arguments of page.Revision must be replaced by keyword arguments (T259428) * 5.0.0: HttpRequest result of http.fetch() will be replaced by requests.Response (T265206) * 5.0.0: OptionHandler.options dict will be removed in favour of OptionHandler.opt * 5.0.0: Methods deprecated for 5 years or longer will be removed diff --git a/pywikibot/__metadata__.py b/pywikibot/__metadata__.py index 3471f87..ab161bb 100644 --- a/pywikibot/__metadata__.py +++ b/pywikibot/__metadata__.py @@ -5,7 +5,7 @@ # Distributed under the terms of the MIT license. # __name__ = 'pywikibot' -__version__ = '5.5.1.dev0' +__version__ = '5.6.0.dev0' __description__ = 'Python MediaWiki Bot Framework' __maintainer__ = 'The Pywikibot team' __maintainer_email__ = 'pywikibot(a)lists.wikimedia.org' -- To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/656394 To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Change-Id: I46cd41f027b6ed1fefe20e6a1282fef74a774cec Gerrit-Change-Number: 656394 Gerrit-PatchSet: 1 Gerrit-Owner: Xqt <info(a)gno.de> Gerrit-Reviewer: Xqt <info(a)gno.de> Gerrit-Reviewer: jenkins-bot Gerrit-MessageType: merged

3 years, 4 months

[Gerrit] ...core[master]: [IMPR] http.fetch() return requests.Response()

by jenkins-bot (Code Review)

jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/651946 ) Change subject: [IMPR] http.fetch() return requests.Response() ...................................................................... [IMPR] http.fetch() return requests.Response() http.fetch() return requests.Response() instead of threadedhttp.HttpRequest(). Done: - move encoding detection in http. - return respone instead of request - deprecate threadedhttp.HttpRequest() with FutureWarning - introduce a temporary class ONLY for deprecation purposes. Bug: T265206 Change-Id: Ia46683bf39f75790baeb6eed8e49e5fcab2de501 --- M pywikibot/comms/http.py M pywikibot/comms/threadedhttp.py M pywikibot/page/__init__.py M pywikibot/site_detect.py M pywikibot/specialbots/_upload.py M scripts/download_dump.py M scripts/imagecopy.py M scripts/reflinks.py M tests/aspects.py M tests/http_tests.py M tests/site_detect_tests.py 11 files changed, 265 insertions(+), 154 deletions(-) Approvals: Xqt: Looks good to me, approved jenkins-bot: Verified diff --git a/pywikibot/comms/http.py b/pywikibot/comms/http.py index 9716b0a..1c7f6e2 100644 --- a/pywikibot/comms/http.py +++ b/pywikibot/comms/http.py @@ -16,6 +16,8 @@ # Distributed under the terms of the MIT license. # import atexit +import codecs +import re import sys from contextlib import suppress @@ -30,7 +32,6 @@ import pywikibot from pywikibot.backports import Tuple -from pywikibot.comms import threadedhttp from pywikibot import config2 as config from pywikibot.exceptions import ( FatalServerError, Server504Error, Server414Error @@ -252,7 +253,7 @@ baseuri = site.base_url(uri) r = fetch(baseuri, headers=headers, **kwargs) - site.throttle.retry_after = int(r.response_headers.get('retry-after', 0)) + site.throttle.retry_after = int(r.headers.get('retry-after', 0)) return r.text @@ -313,15 +314,24 @@ if response.status_code not in (200, 207): warning('Http response status {}'.format(response.status_code)) + if isinstance(response.encoding, UnicodeDecodeError): + error('An error occurred for uri {}: ' + 'no encoding detected!'.format(response.request.url)) + raise response.encoding from None + @deprecated_args(callback=True, body='data') -def fetch(uri, method='GET', headers=None, default_error_handling: bool = True, +def fetch(uri: str, method: str = 'GET', headers: Optional[dict] = None, + default_error_handling: bool = True, use_fake_user_agent: Union[bool, str] = False, **kwargs): """ HTTP request. See L{requests.Session.request} for parameters. + @param uri: URL to send + @param method: HTTP method of the request (default: GET) + @param headers: dictionary of headers of the request @param default_error_handling: Use default error handling @param use_fake_user_agent: Set to True to use fake UA, False to use pywikibot's UA, str to specify own UA. This behaviour might be @@ -335,7 +345,7 @@ @type verify: bool or path to certificates @kwarg callbacks: Methods to call once data is fetched @type callbacks: list of callable - @rtype: L{threadedhttp.HttpRequest} + @rtype: L{requests.Response} """ # Change user agent depending on fake UA settings. # Set header to new UA if needed. @@ -373,11 +383,11 @@ headers['user-agent'] = assign_user_agent(headers.get('user-agent')) callbacks = kwargs.pop('callbacks', []) + # error_handling_callback will be executed first. if default_error_handling: - callbacks.append(error_handling_callback) + callbacks.insert(0, error_handling_callback) charset = kwargs.pop('charset', None) - request = threadedhttp.HttpRequest(charset=charset) auth = get_authentication(uri) if auth is not None and len(auth) == 4: @@ -406,18 +416,107 @@ headers=headers, auth=auth, timeout=timeout, **kwargs) except Exception as e: - request.data = e response = e else: - request.data = response + response.encoding = _decide_encoding(response, charset) for callback in callbacks: callback(response) - # if there's no data in the answer we're in trouble - try: - request.data - except AssertionError as e: - raise e + return _ResponseDeprecationWrapper(response) - return request + +class _ResponseDeprecationWrapper(requests.Response): + + """Helper class for the deprecation of HttpRequests. + + This class will be removed ASAP. Its only purpose is to allow + a graceful deprecation of HttpRequests. + DO NOT USE! + + """ + + def __init__(self, response): + self.__response = response + + def __getattr__(self, attr): + return getattr(self.__response, attr) + + def __setattr__(self, attr, val): + if attr == '_ResponseDeprecationWrapper__response': + object.__setattr__(self, attr, val) + + return setattr(self.__response, attr, val) + + @property + @deprecated('attribute/methods of Response(), ' + 'which is now returned from http.fetch()', + since='20210110', future_warning=True) + def data(self): + return self + + +def _get_encoding_from_response_headers(response): + """Return charset given by the response header.""" + content_type = response.headers.get('content-type') + + if not content_type: + return None + + m = re.search('charset=(?P<charset>.*?$)', content_type) + if m: + header_encoding = m.group('charset') + elif 'json' in content_type: + # application/json | application/sparql-results+json + header_encoding = 'utf-8' + elif 'xml' in content_type: + header = response.content[:100].splitlines()[0] # bytes + m = re.search( + br'encoding=(["\'])(?P<encoding>.+?)\1', header) + if m: + header_encoding = m.group('encoding').decode('utf-8') + else: + header_encoding = 'utf-8' + else: + header_encoding = None + + return header_encoding + + +def _decide_encoding(response, charset): + """Detect the response encoding.""" + def _try_decode(content, encoding): + """Helper function to try decoding.""" + content.decode(encoding) + return encoding + + header_encoding = _get_encoding_from_response_headers(response) + if header_encoding is None: + pywikibot.log('Http response does not contain a charset.') + + if charset is None: + charset = response.request.headers.get('accept-charset') + + # No charset requested, or in request headers or response headers. + # Defaults to latin1. + if charset is None and header_encoding is None: + return _try_decode(response.content, 'latin1') + + if charset is None and header_encoding is not None: + return _try_decode(response.content, header_encoding) + + if charset is not None and header_encoding is None: + return _try_decode(response.content, charset) + + # Both charset and header_encoding are available. + if codecs.lookup(header_encoding) != codecs.lookup(charset): + pywikibot.warning( + 'Encoding "{}" requested but "{}" received in the ' + 'response header.'.format(charset, header_encoding)) + + try: + _encoding = _try_decode(response.content, header_encoding) + except UnicodeDecodeError: + _encoding = _try_decode(response.content, charset) + + return _encoding diff --git a/pywikibot/comms/threadedhttp.py b/pywikibot/comms/threadedhttp.py index 0f0d6e0..d5c26f5 100644 --- a/pywikibot/comms/threadedhttp.py +++ b/pywikibot/comms/threadedhttp.py @@ -17,6 +17,7 @@ deprecated, deprecated_args, issue_deprecation_warning, + ModuleDeprecationWrapper, ) @@ -294,3 +295,8 @@ def __bytes__(self) -> Optional[bytes]: # pragma: no cover """Return the undecoded response.""" return self.content + + +wrapper = ModuleDeprecationWrapper(__name__) +wrapper._add_deprecated_attr('HttpRequest', replacement_name='', + since='20201226', future_warning=True) diff --git a/pywikibot/page/__init__.py b/pywikibot/page/__init__.py index 3dc9015..0b276bd 100644 --- a/pywikibot/page/__init__.py +++ b/pywikibot/page/__init__.py @@ -2528,7 +2528,7 @@ if req.status_code == 200: try: with open(filename, 'wb') as f: - for chunk in req.data.iter_content(chunk_size): + for chunk in req.iter_content(chunk_size): f.write(chunk) except IOError as e: raise e diff --git a/pywikibot/site_detect.py b/pywikibot/site_detect.py index 8b46594..167468d 100644 --- a/pywikibot/site_detect.py +++ b/pywikibot/site_detect.py @@ -43,9 +43,9 @@ r = fetch(fromurl) check_response(r) - if fromurl != r.data.url: - pywikibot.log('{} redirected to {}'.format(fromurl, r.data.url)) - fromurl = r.data.url + if fromurl != r.url: + pywikibot.log('{} redirected to {}'.format(fromurl, r.url)) + fromurl = r.url self.fromurl = fromurl diff --git a/pywikibot/specialbots/_upload.py b/pywikibot/specialbots/_upload.py index 5161adb..cf96e67 100644 --- a/pywikibot/specialbots/_upload.py +++ b/pywikibot/specialbots/_upload.py @@ -132,8 +132,8 @@ with open(path, 'ab') as fd: os.lseek(handle, file_len, 0) try: - r = http.fetch(file_url, stream=True, headers=headers) - response = r.data + response = http.fetch(file_url, stream=True, + headers=headers) response.raise_for_status() # get download info, if available diff --git a/scripts/download_dump.py b/scripts/download_dump.py index b0d1e03..74e9a8d 100644 --- a/scripts/download_dump.py +++ b/scripts/download_dump.py @@ -118,8 +118,7 @@ return with open(file_current_storepath, 'wb') as result_file: - total = int(response.response_headers.get( - 'content-length', -1)) + total = int(response.headers['content-length']) if total == -1: pywikibot.warning("'content-length' missing in " 'response headers') @@ -128,7 +127,7 @@ display_string = '' pywikibot.output('') - for data in response.data.iter_content(100 * 1024): + for data in response.iter_content(100 * 1024): result_file.write(data) if total <= 0: diff --git a/scripts/imagecopy.py b/scripts/imagecopy.py index ae631b4..f949196 100644 --- a/scripts/imagecopy.py +++ b/scripts/imagecopy.py @@ -215,15 +215,14 @@ @return: A CommonHelper description message. @rtype: str """ - gotInfo = False - while not gotInfo: + while True: try: commonsHelperPage = fetch( 'https://commonshelper.toolforge.org/', method='POST', data=parameters) - data = commonsHelperPage.data.content.decode('utf-8') - gotInfo = True + data = commonsHelperPage.content.decode('utf-8') + break except RequestException: pywikibot.output("Got a RequestException, let's try again") return data diff --git a/scripts/reflinks.py b/scripts/reflinks.py index 9ac0292..78ded1a 100755 --- a/scripts/reflinks.py +++ b/scripts/reflinks.py @@ -537,7 +537,7 @@ ref.url, use_fake_user_agent=self._use_fake_user_agent) # Try to get Content-Type from server - content_type = f.response_headers.get('content-type') + content_type = f.headers.get('content-type') if content_type and not self.MIME.search(content_type): if ref.link.lower().endswith('.pdf') \ and not self.opt.ignorepdf: @@ -564,7 +564,7 @@ return # Get the real url where we end (http redirects !) - redir = f.data.url + redir = f.url if redir != ref.link \ and domain.findall(redir) == domain.findall(link): if soft404.search(redir) \ @@ -656,7 +656,7 @@ new_text = new_text.replace(match.group(), repl) return - u = f.data.text + u = f.text # Retrieves the first non empty string inside <title> tags for m in self.TITLE.finditer(u): diff --git a/tests/aspects.py b/tests/aspects.py index f7e3fec..ff35577 100644 --- a/tests/aspects.py +++ b/tests/aspects.py @@ -457,17 +457,12 @@ r = http.fetch(hostname, method='HEAD', default_error_handling=False) - if r.exception: - e = r.exception - else: - if r.status_code not in {200, 301, 302, 303, 307, 308}: - raise ServerError('HTTP status: {}' - .format(r.status_code)) - except Exception as e2: + if r.status_code not in {200, 301, 302, 303, 307, 308}: + raise ServerError('HTTP status: {}'.format(r.status_code)) + except Exception as e: pywikibot.error('{}: accessing {} caused exception:' .format(cls.__name__, hostname)) - pywikibot.exception(e2, tb=True) - e = e2 + pywikibot.exception(e, tb=True) if e: cls._checked_hostnames[hostname] = e diff --git a/tests/http_tests.py b/tests/http_tests.py index ec85e77..d9c0b53 100644 --- a/tests/http_tests.py +++ b/tests/http_tests.py @@ -16,7 +16,8 @@ import pywikibot from pywikibot import config2 as config -from pywikibot.comms import http, threadedhttp + +from pywikibot.comms import http from pywikibot.tools import PYTHON_VERSION, suppress_warnings from tests import join_images_path, patch @@ -41,7 +42,7 @@ def test_fetch(self): """Test http.fetch using http://www.wikipedia.org/.""" r = http.fetch('http://www.wikipedia.org/') - self.assertIsInstance(r, threadedhttp.HttpRequest) + self.assertIsInstance(r, requests.Response) self.assertEqual(r.status_code, 200) self.assertIn('<html lang="mul"', r.text) self.assertIsInstance(r.text, str) @@ -106,9 +107,8 @@ response = http.fetch( 'https://testssl-expire-r2i2.disig.sk/index.en.html', verify=False) - r = response.text - self.assertIsInstance(r, str) - self.assertTrue(re.search(r'<title>.*</title>', r)) + self.assertIsInstance(response.text, str) + self.assertTrue(re.search(r'<title>.*</title>', response.text)) http.session.close() # clear the connection # Verify that it now fails again @@ -166,14 +166,12 @@ # The following will redirect from ' ' -> '_', and maybe to https:// r = http.fetch('http://en.wikipedia.org/wiki/Main%20Page') self.assertEqual(r.status_code, 200) - self.assertIsNotNone(r.data.history) - self.assertIn('//en.wikipedia.org/wiki/Main_Page', - r.data.url) + self.assertIsNotNone(r.history) + self.assertIn('//en.wikipedia.org/wiki/Main_Page', r.url) r = http.fetch('http://en.wikia.com') self.assertEqual(r.status_code, 200) - self.assertEqual(r.data.url, - 'https://www.fandom.com/explore') + self.assertEqual(r.url, 'https://www.fandom.com/explore') class UserAgentTestCase(TestCase): @@ -282,19 +280,19 @@ r = http.fetch( self.get_httpbin_url('/status/200'), headers={'user-agent': 'EXISTING'}) - self.assertEqual(r.headers['user-agent'], 'EXISTING') + self.assertEqual(r.request.headers['user-agent'], 'EXISTING') # Argument value changes r = http.fetch(self.get_httpbin_url('/status/200'), use_fake_user_agent=True) - self.assertNotEqual(r.headers['user-agent'], http.user_agent()) + self.assertNotEqual(r.request.headers['user-agent'], http.user_agent()) r = http.fetch(self.get_httpbin_url('/status/200'), use_fake_user_agent=False) - self.assertEqual(r.headers['user-agent'], http.user_agent()) + self.assertEqual(r.request.headers['user-agent'], http.user_agent()) r = http.fetch( self.get_httpbin_url('/status/200'), use_fake_user_agent='ARBITRARY') - self.assertEqual(r.headers['user-agent'], 'ARBITRARY') + self.assertEqual(r.request.headers['user-agent'], 'ARBITRARY') # Empty value self.assertRaisesRegex(ValueError, @@ -313,7 +311,7 @@ self.get_httpbin_hostname(): 'OVERRIDDEN'} r = http.fetch( self.get_httpbin_url('/status/200'), use_fake_user_agent=False) - self.assertEqual(r.headers['user-agent'], 'OVERRIDDEN') + self.assertEqual(r.request.headers['user-agent'], 'OVERRIDDEN') @require_modules('fake_useragent') def test_fetch_with_fake_useragent(self): @@ -370,153 +368,170 @@ UTF8_BYTES = STR.encode('utf8') @staticmethod - def _create_request(charset=None, data=UTF8_BYTES): + def _create_response(headers=None, data=UTF8_BYTES): """Helper method.""" - req = threadedhttp.HttpRequest(charset=charset) resp = requests.Response() - resp.headers = {'content-type': 'charset=utf-8'} + resp.request = requests.Request() + if headers is not None: + resp.headers = headers + else: + resp.headers = {'content-type': 'charset=utf-8'} resp._content = data[:] - req.data = resp - return req + return resp def test_no_content_type(self): """Test decoding without content-type (and then no charset).""" - req = threadedhttp.HttpRequest('') - resp = requests.Response() - resp.headers = {} - resp._content = CharsetTestCase.LATIN1_BYTES[:] - req._data = resp - self.assertIsNone(req.charset) - self.assertEqual('latin1', req.encoding) - self.assertEqual(req.content, CharsetTestCase.LATIN1_BYTES) - self.assertEqual(req.text, CharsetTestCase.STR) + charset = None + resp = CharsetTestCase._create_response( + headers={}, + data=CharsetTestCase.LATIN1_BYTES) + resp.encoding = http._decide_encoding(resp, charset) + self.assertEqual('latin1', resp.encoding) + self.assertEqual(resp.content, CharsetTestCase.LATIN1_BYTES) + self.assertEqual(resp.text, CharsetTestCase.STR) def test_no_charset(self): """Test decoding without explicit charset.""" - req = threadedhttp.HttpRequest('') - resp = requests.Response() - resp.headers = {'content-type': ''} - resp._content = CharsetTestCase.LATIN1_BYTES[:] - req._data = resp - self.assertIsNone(req.charset) - self.assertEqual('latin1', req.encoding) - self.assertEqual(req.content, CharsetTestCase.LATIN1_BYTES) - self.assertEqual(req.text, CharsetTestCase.STR) + charset = None + resp = CharsetTestCase._create_response( + headers={'content-type': ''}, + data=CharsetTestCase.LATIN1_BYTES) + resp.encoding = http._decide_encoding(resp, charset) + self.assertEqual('latin1', resp.encoding) + self.assertEqual(resp.content, CharsetTestCase.LATIN1_BYTES) + self.assertEqual(resp.text, CharsetTestCase.STR) def test_content_type_application_json_without_charset(self): """Test decoding without explicit charset but JSON content.""" - req = CharsetTestCase._create_request() - resp = requests.Response() - req._data = resp - resp._content = CharsetTestCase.UTF8_BYTES[:] - resp.headers = {'content-type': 'application/json'} - self.assertIsNone(req.charset) - self.assertEqual('utf-8', req.encoding) + charset = None + resp = CharsetTestCase._create_response( + headers={'content-type': 'application/json'}, + data=CharsetTestCase.UTF8_BYTES) + resp.encoding = http._decide_encoding(resp, charset) + self.assertEqual('utf-8', resp.encoding) def test_content_type_sparql_json_without_charset(self): """Test decoding without explicit charset but JSON content.""" - req = CharsetTestCase._create_request() - resp = requests.Response() - req._data = resp - resp._content = CharsetTestCase.UTF8_BYTES[:] - resp.headers = {'content-type': 'application/sparql-results+json'} - self.assertIsNone(req.charset) - self.assertEqual('utf-8', req.encoding) + charset = None + resp = CharsetTestCase._create_response( + headers={'content-type': 'application/sparql-results+json'}, + data=CharsetTestCase.UTF8_BYTES) + resp.encoding = http._decide_encoding(resp, charset) + self.assertEqual('utf-8', resp.encoding) def test_content_type_xml_without_charset(self): """Test decoding without explicit charset but xml content.""" - req = CharsetTestCase._create_request() - resp = requests.Response() - req._data = resp - resp._content = CharsetTestCase.UTF8_BYTES[:] - resp.headers = {'content-type': 'text/xml'} - self.assertIsNone(req.charset) - self.assertEqual('utf-8', req.encoding) + charset = None + resp = CharsetTestCase._create_response( + headers={'content-type': 'application/xml'}, + data=CharsetTestCase.UTF8_BYTES) + resp.encoding = http._decide_encoding(resp, charset) + self.assertEqual('utf-8', resp.encoding) def test_content_type_xml_with_charset(self): """Test xml content with utf-8 encoding given in content.""" - req = CharsetTestCase._create_request() - resp = requests.Response() - req._data = resp - resp._content = '<?xml version="1.0" encoding="UTF-8"?>'.encode( - 'utf-8') - resp.headers = {'content-type': 'text/xml'} - self.assertIsNone(req.charset) - self.assertEqual('UTF-8', req.encoding) + charset = None + resp = CharsetTestCase._create_response( + headers={'content-type': 'application/xml'}, + data='<?xml version="1.0" encoding="UTF-8"?>'.encode('utf-8')) + resp.encoding = http._decide_encoding(resp, charset) + self.assertEqual('UTF-8', resp.encoding) def test_content_type_xml_with_charset_and_more_data(self): """Test xml content with utf-8 encoding given in content.""" - req = CharsetTestCase._create_request() - resp = requests.Response() - req._data = resp - resp._content = ( - '<?xml version="1.0" encoding="UTF-8" someparam="ignored"?>' - .encode('utf-8')) - resp.headers = {'content-type': 'text/xml'} - self.assertIsNone(req.charset) - self.assertEqual('UTF-8', req.encoding) + charset = None + resp = CharsetTestCase._create_response( + headers={'content-type': 'application/xml'}, + data='<?xml version="1.0" encoding="UTF-8" ' + 'someparam="ignored"?>'.encode('utf-8')) + resp.encoding = http._decide_encoding(resp, charset) + self.assertEqual('UTF-8', resp.encoding) def test_content_type_xml_with_variant_charset(self): """Test xml content with latin1 encoding given in content.""" - req = CharsetTestCase._create_request() - resp = requests.Response() - req._data = resp - resp._content = "<?xml version='1.0' encoding='latin1'?>".encode( - 'latin1') - resp.headers = {'content-type': 'text/xml'} - self.assertIsNone(req.charset) - self.assertEqual('latin1', req.encoding) + charset = None + resp = CharsetTestCase._create_response( + headers={'content-type': 'application/xml'}, + data="<?xml version='1.0' encoding='latin1'?>".encode('latin1')) + resp.encoding = http._decide_encoding(resp, charset) + self.assertEqual('latin1', resp.encoding) def test_server_charset(self): """Test decoding with server explicit charset.""" - req = CharsetTestCase._create_request() - self.assertIsNone(req.charset) - self.assertEqual('utf-8', req.encoding) - self.assertEqual(req.content, CharsetTestCase.UTF8_BYTES) - self.assertEqual(req.text, CharsetTestCase.STR) + charset = None + resp = CharsetTestCase._create_response() + resp.encoding = http._decide_encoding(resp, charset) + self.assertEqual('utf-8', resp.encoding) + self.assertEqual(resp.content, CharsetTestCase.UTF8_BYTES) + self.assertEqual(resp.text, CharsetTestCase.STR) def test_same_charset(self): """Test decoding with explicit and equal charsets.""" - req = CharsetTestCase._create_request('utf-8') - self.assertEqual('utf-8', req.charset) - self.assertEqual('utf-8', req.encoding) - self.assertEqual(req.content, CharsetTestCase.UTF8_BYTES) - self.assertEqual(req.text, CharsetTestCase.STR) + charset = 'utf-8' + resp = CharsetTestCase._create_response() + resp.encoding = http._decide_encoding(resp, charset) + self.assertEqual('utf-8', resp.encoding) + self.assertEqual(resp.content, CharsetTestCase.UTF8_BYTES) + self.assertEqual(resp.text, CharsetTestCase.STR) def test_header_charset(self): """Test decoding with different charsets and valid header charset.""" - req = CharsetTestCase._create_request('latin1') - self.assertEqual('latin1', req.charset) + charset = 'latin1' + resp = CharsetTestCase._create_response() + resp.encoding = http._decide_encoding(resp, charset) # Ignore WARNING: Encoding "latin1" requested but "utf-8" received with patch('pywikibot.warning'): - self.assertEqual('utf-8', req.encoding) - self.assertEqual(req.content, CharsetTestCase.UTF8_BYTES) - self.assertEqual(req.text, CharsetTestCase.STR) + self.assertEqual('utf-8', resp.encoding) + self.assertEqual(resp.content, CharsetTestCase.UTF8_BYTES) + self.assertEqual(resp.text, CharsetTestCase.STR) def test_code_charset(self): """Test decoding with different charsets and invalid header charset.""" - req = CharsetTestCase._create_request('latin1', - CharsetTestCase.LATIN1_BYTES) - self.assertEqual('latin1', req.charset) + charset = 'latin1' + resp = CharsetTestCase._create_response( + data=CharsetTestCase.LATIN1_BYTES) + resp.encoding = http._decide_encoding(resp, charset) # Ignore WARNING: Encoding "latin1" requested but "utf-8" received with patch('pywikibot.warning'): - self.assertEqual('latin1', req.encoding) - self.assertEqual(req.content, CharsetTestCase.LATIN1_BYTES) - self.assertEqual(req.text, CharsetTestCase.STR) + self.assertEqual('latin1', resp.encoding) + self.assertEqual(resp.content, CharsetTestCase.LATIN1_BYTES) + self.assertEqual(resp.text, CharsetTestCase.STR) def test_invalid_charset(self): """Test decoding with different and invalid charsets.""" - req = CharsetTestCase._create_request('utf16', - CharsetTestCase.LATIN1_BYTES) - self.assertEqual('utf16', req.charset) + charset = 'utf16' + resp = CharsetTestCase._create_response( + data=CharsetTestCase.LATIN1_BYTES) # Ignore WARNING: Encoding "utf16" requested but "utf-8" received with patch('pywikibot.warning'): self.assertRaisesRegex( UnicodeDecodeError, self.CODEC_CANT_DECODE_RE, - lambda: req.encoding) - self.assertEqual(req.content, CharsetTestCase.LATIN1_BYTES) - self.assertRaisesRegex( - UnicodeDecodeError, self.CODEC_CANT_DECODE_RE, lambda: req.text) + http._decide_encoding, resp, charset) + self.assertEqual(resp.content, CharsetTestCase.LATIN1_BYTES) + + try: + resp.encoding = http._decide_encoding(resp, charset) + except UnicodeDecodeError as e: + resp.encoding = e + + with patch('pywikibot.error'): + self.assertRaisesRegex( + UnicodeDecodeError, self.CODEC_CANT_DECODE_RE, + http.error_handling_callback, resp) + + # TODO: this is a breaking change + # self.assertRaisesRegex( + # UnicodeDecodeError, self.CODEC_CANT_DECODE_RE, lambda: resp.text) + + # Response() would do: + # encoding = UnicodeDecodeError -> str(self.content, errors='replace') + self.assertEqual( + resp.text, str(resp.content, errors='replace')) + # encoding = None -> str(resp.content, resp.encoding, errors='replace') + resp.encoding = None + self.assertEqual( + resp.text, + str(resp.content, resp.apparent_encoding, errors='replace')) class BinaryTestCase(TestCase): diff --git a/tests/site_detect_tests.py b/tests/site_detect_tests.py index e401db9..36f608d 100644 --- a/tests/site_detect_tests.py +++ b/tests/site_detect_tests.py @@ -291,9 +291,7 @@ (object,), {'status_code': 200, 'text': self._responses[parsed_url.path], - 'data': type(str('ResponseData'), - (object,), - {'url': url})}) + 'url': url}) @PatchingTestCase.patched(pywikibot, 'input') def input(self, question, *args, **kwargs): -- To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/651946 To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Change-Id: Ia46683bf39f75790baeb6eed8e49e5fcab2de501 Gerrit-Change-Number: 651946 Gerrit-PatchSet: 20 Gerrit-Owner: Mpaa <mpaa.wiki(a)gmail.com> Gerrit-Reviewer: Xqt <info(a)gno.de> Gerrit-Reviewer: jenkins-bot Gerrit-MessageType: merged

3 years, 4 months

[Gerrit] ...core[master]: [cleanup] remove positional arguments of page.Revision object

by jenkins-bot (Code Review)

jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/655722 ) Change subject: [cleanup] remove positional arguments of page.Revision object ...................................................................... [cleanup] remove positional arguments of page.Revision object Usage of positional arguments is deprecated already. Follow up for T259428 Change-Id: I69ae7cbef8e4f8c4a22ab524985c1ac9dd290fb5 --- M pywikibot/page/_revision.py 1 file changed, 3 insertions(+), 31 deletions(-) Approvals: DannyS712: Looks good to me, but someone else must approve Xqt: Looks good to me, approved jenkins-bot: Verified diff --git a/pywikibot/page/_revision.py b/pywikibot/page/_revision.py index 627fda0..46913f5 100644 --- a/pywikibot/page/_revision.py +++ b/pywikibot/page/_revision.py @@ -1,6 +1,6 @@ """Object representing page revision.""" # -# (C) Pywikibot team, 2008-2020 +# (C) Pywikibot team, 2008-2021 # # Distributed under the terms of the MIT license. # @@ -9,7 +9,7 @@ from collections.abc import Mapping from pywikibot import Timestamp, warning -from pywikibot.tools import deprecated, issue_deprecation_warning +from pywikibot.tools import deprecated class Revision(Mapping): @@ -26,40 +26,12 @@ Sample for Revision access """ - def __init__(self, *args, **kwargs): + def __init__(self, **kwargs): """Initializer.""" - self._clean_args(args, kwargs) self._data = kwargs self._upcast_dict(self._data) super().__init__() - def _clean_args(self, args: tuple, kwargs: dict): - """Cleanup positional arguments. - - Replace positional arguments with keyword arguments for - backwards compatibility. - @param: args: tuple of positional arguments - @param: kwargs: mutable dict of keyword arguments to be updated - """ - keys = ( # postional argument keys by old order - 'revid', 'timestamp', 'user', 'anon', 'comment', 'text', 'minor', - 'rollbacktoken', 'parentid', 'contentmodel', 'sha1', 'slots' - ) - - # replace positional arguments with keyword arguments - for i, (arg, key) in enumerate(zip(args, keys)): # pragma: no cover - issue_deprecation_warning('Positional argument {} ({})' - .format(i + 1, arg), - 'keyword argument "{}={}"' - .format(key, arg), - warning_class=FutureWarning, - since='20200802') - if key in kwargs: - warning('"{}" is given as keyword argument "{}" already; ' - 'ignoring "{}"'.format(key, arg, kwargs[key])) - else: - kwargs[key] = arg - @staticmethod def _upcast_dict(map_): """Upcast dictionary values.""" -- To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/655722 To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Change-Id: I69ae7cbef8e4f8c4a22ab524985c1ac9dd290fb5 Gerrit-Change-Number: 655722 Gerrit-PatchSet: 3 Gerrit-Owner: Xqt <info(a)gno.de> Gerrit-Reviewer: DannyS712 <DannyS712.enwiki(a)gmail.com> Gerrit-Reviewer: Xqt <info(a)gno.de> Gerrit-Reviewer: jenkins-bot Gerrit-MessageType: merged

3 years, 4 months

[Gerrit] ...core[master]: [IMPR] remove IDE warnings from __init__.py

by jenkins-bot (Code Review)

jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/656256 ) Change subject: [IMPR] remove IDE warnings from __init__.py ...................................................................... [IMPR] remove IDE warnings from __init__.py Change-Id: Ia07d3c288862cd76a8aa5c5a3a6e18bd4b64da3a --- M pywikibot/__init__.py 1 file changed, 4 insertions(+), 4 deletions(-) Approvals: Xqt: Looks good to me, approved jenkins-bot: Verified diff --git a/pywikibot/__init__.py b/pywikibot/__init__.py index 88b9c27..47cdd73 100644 --- a/pywikibot/__init__.py +++ b/pywikibot/__init__.py @@ -67,7 +67,7 @@ __all__ = ( '__copyright__', '__description__', '__download_url__', '__license__', - '__maintainer__', '__maintainer_email__', '__name__', '__release__', + '__maintainer__', '__maintainer_email__', '__name__', '__url__', '__version__', 'BadTitle', 'Bot', 'calledModuleName', 'CaptchaError', 'CascadeLockedPage', 'Category', 'CircularRedirect', 'Claim', 'config', 'Coordinate', @@ -80,8 +80,8 @@ 'NoSuchSite', 'NoUsername', 'NoWikibaseEntity', 'OtherPageSaveError', 'output', 'Page', 'PageCreatedConflict', 'PageDeletedConflict', 'PageNotSaved', 'PageRelatedError', 'PageSaveRelatedError', 'PropertyPage', - 'QuitKeyboardInterrupt', 'SectionError', 'Server414Error', - 'Server504Error', 'ServerError', 'showDiff', 'show_help', 'showHelp', + '_QuitKeyboardInterrupt', 'SectionError', 'Server414Error', + 'Server504Error', 'ServerError', 'showDiff', 'show_help', 'Site', 'SiteDefinitionError', 'SiteLink', 'SpamblacklistError', 'stdout', 'Timestamp', 'TitleblacklistError', 'translate', 'ui', 'unicode2html', 'UnknownExtension', 'UnknownFamily', 'UnknownSite', 'UnsupportedPage', @@ -1019,7 +1019,7 @@ _items = ('json',) - def __init__(self, json) -> dict: + def __init__(self, json): """ Create a new WbUnknown object. -- To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/656256 To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Change-Id: Ia07d3c288862cd76a8aa5c5a3a6e18bd4b64da3a Gerrit-Change-Number: 656256 Gerrit-PatchSet: 1 Gerrit-Owner: Mpaa <mpaa.wiki(a)gmail.com> Gerrit-Reviewer: Xqt <info(a)gno.de> Gerrit-Reviewer: jenkins-bot Gerrit-MessageType: merged

3 years, 4 months

[Gerrit] ...core[master]: [cleanup] Deprecate loadimageinfo return statement

by jenkins-bot (Code Review)

jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/655709 ) Change subject: [cleanup] Deprecate loadimageinfo return statement ...................................................................... [cleanup] Deprecate loadimageinfo return statement Change-Id: I626d3b15473d2b3fab8de0ff2cae98d16b5974f9 --- M pywikibot/site/__init__.py M tests/site_tests.py 2 files changed, 34 insertions(+), 4 deletions(-) Approvals: Mpaa: Looks good to me, approved jenkins-bot: Verified diff --git a/pywikibot/site/__init__.py b/pywikibot/site/__init__.py index 348f34d..cb059ef 100644 --- a/pywikibot/site/__init__.py +++ b/pywikibot/site/__init__.py @@ -1,4 +1,3 @@ -# -*- coding: utf-8 -*- """ Objects representing MediaWiki sites (wikis). @@ -6,16 +5,18 @@ groups of wikis on the same topic in different languages. """ # -# (C) Pywikibot team, 2008-2020 +# (C) Pywikibot team, 2008-2021 # # Distributed under the terms of the MIT license. # import datetime import functools import heapq +import inspect import itertools import json import mimetypes +import opcode import os import re import time @@ -1314,6 +1315,23 @@ page, 'loadimageinfo: Query on %s returned no imageinfo') + # inspect whether the result is assigned and print an deprecation + # warning in that case; implementation inspired by + # https://stackoverflow.com/questions/813882/is-there-a-way-to-check-whether-… + try: + frame = inspect.currentframe().f_back + next_opcode = opcode.opname[ + frame.f_code.co_code[frame.f_lasti + 3]] + if next_opcode in ('POP_TOP', 'ROT_TWO', 'ROT_THREE'): + issue_deprecation_warning( + 'APISite.loadimageinfo() result dict and any assignment', + "pageitem['imageinfo'] if history is True " + "else pageitem['imageinfo'][0]", + warning_class=FutureWarning, + since='20210110') + finally: + del frame + return (pageitem['imageinfo'] if history else pageitem['imageinfo'][0]) diff --git a/tests/site_tests.py b/tests/site_tests.py index e3469e5..9a00492 100644 --- a/tests/site_tests.py +++ b/tests/site_tests.py @@ -1,7 +1,6 @@ -# -*- coding: utf-8 -*- """Tests for the site module.""" # -# (C) Pywikibot team, 2008-2020 +# (C) Pywikibot team, 2008-2021 # # Distributed under the terms of the MIT license. # @@ -108,6 +107,19 @@ self.assertEqual(self.site.category_namespaces(), list(self.site.namespace(14, all=True))) + def test_loadimageinfo(self): + """Test deprecation warning if result of loadimageinfo() is used.""" + file = pywikibot.FilePage(self.site, 'foo.jpg') + if not file.file_is_shared(): + self.skipPage('test file is not shared from image repository.') + self.site.loadimageinfo(file) + self.assertNoDeprecation() + result = self.site.loadimageinfo(file) # noqa: F841 + self.assertOneDeprecation() + x = [] + x.append(self.site.loadimageinfo(file)) + self.assertOneDeprecation() + class TestSiteDryDeprecatedFunctions(DefaultDrySiteTestCase, DeprecationTestCase): -- To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/655709 To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Change-Id: I626d3b15473d2b3fab8de0ff2cae98d16b5974f9 Gerrit-Change-Number: 655709 Gerrit-PatchSet: 5 Gerrit-Owner: Xqt <info(a)gno.de> Gerrit-Reviewer: Mpaa <mpaa.wiki(a)gmail.com> Gerrit-Reviewer: jenkins-bot Gerrit-MessageType: merged

3 years, 4 months

[Gerrit] ...core[master]: Renamed handleArg() method to handle_arg() in certain scripts

by jenkins-bot (Code Review)

jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/655900 ) Change subject: Renamed handleArg() method to handle_arg() in certain scripts ...................................................................... Renamed handleArg() method to handle_arg() in certain scripts Bug: T271437 Change-Id: Ib2431deb7c6846b77d5ce53f1031643ea7ae857c --- M pywikibot/pagegenerators.py M scripts/add_text.py M scripts/basic.py M scripts/blockpageschecker.py M scripts/category.py M scripts/change_pagelang.py M scripts/claimit.py M scripts/clean_sandbox.py M scripts/commons_link.py M scripts/commonscat.py M scripts/coordinate_import.py M scripts/cosmetic_changes.py M scripts/create_categories.py M scripts/data_ingestion.py M scripts/delete.py M scripts/fixing_redirects.py M scripts/harvest_template.py M scripts/illustrate_wikidata.py M scripts/imagecopy.py 19 files changed, 43 insertions(+), 38 deletions(-) Approvals: Xqt: Looks good to me, approved jenkins-bot: Verified diff --git a/pywikibot/pagegenerators.py b/pywikibot/pagegenerators.py index 874127d..b2b9a22 100644 --- a/pywikibot/pagegenerators.py +++ b/pywikibot/pagegenerators.py @@ -1179,7 +1179,7 @@ 'Invalid -logevents parameter "{0}"'.format(params[0])) return self._parse_log_events(*params) - def handleArg(self, arg: str) -> bool: + def handle_arg(self, arg: str) -> bool: """Parse one argument at a time. If it is recognized as an argument that specifies a generator, a @@ -1213,6 +1213,11 @@ return False + @deprecated('handle_arg', since='20210113', future_warning=True) + def handleArg(self, arg: str): + """DEPRECATED. Use handle_arg().""" + return self.handle_arg(arg) + def _int_none(v): """Return None if v is None or '' else return int(v).""" diff --git a/scripts/add_text.py b/scripts/add_text.py index 501f329..895d208 100755 --- a/scripts/add_text.py +++ b/scripts/add_text.py @@ -313,7 +313,7 @@ issue_deprecation_warning(arg, new_arg, 2, ArgumentDeprecationWarning, since='20201224') - genFactory.handleArg(new_arg) + genFactory.handle_arg(new_arg) elif option == '-up': up = True elif option == '-noreorder': @@ -323,7 +323,7 @@ elif option in ('-talk', '-talkpage'): talkPage = True else: - genFactory.handleArg(arg) + genFactory.handle_arg(arg) if textfile and not addText: with codecs.open(textfile, 'r', config.textfile_encoding) as f: diff --git a/scripts/basic.py b/scripts/basic.py index 4645978..477b255 100755 --- a/scripts/basic.py +++ b/scripts/basic.py @@ -43,7 +43,7 @@ &params; """ # -# (C) Pywikibot team, 2006-2020 +# (C) Pywikibot team, 2006-2021 # # Distributed under the terms of the MIT license. # @@ -161,7 +161,7 @@ for arg in local_args: # Catch the pagegenerators options - if gen_factory.handleArg(arg): + if gen_factory.handle_arg(arg): continue # nothing to do here # Now pick up your own options diff --git a/scripts/blockpageschecker.py b/scripts/blockpageschecker.py index 0514a63..62de8be 100755 --- a/scripts/blockpageschecker.py +++ b/scripts/blockpageschecker.py @@ -41,7 +41,7 @@ """ # -# (C) Pywikibot team, 2007-2020 +# (C) Pywikibot team, 2007-2021 # # Distributed under the terms of the MIT license. # @@ -241,7 +241,7 @@ if value: namespace = int(value) else: - genFactory.handleArg(arg) + genFactory.handle_arg(arg) if config.mylang not in project_inserted: pywikibot.output('Your project is not supported by this script.\n' diff --git a/scripts/category.py b/scripts/category.py index a0f86dd..d59fd5a 100755 --- a/scripts/category.py +++ b/scripts/category.py @@ -1416,7 +1416,7 @@ elif option == 'prefix': prefix = value else: - gen_factory.handleArg(arg) + gen_factory.handle_arg(arg) bot = None @@ -1429,7 +1429,7 @@ 'Category to add (do not give namespace):') if not gen: # default for backwards compatibility - gen_factory.handleArg('-links') + gen_factory.handle_arg('-links') gen = gen_factory.getCombinedGenerator() # The preloading generator is responsible for downloading multiple # pages from the wiki simultaneously. diff --git a/scripts/change_pagelang.py b/scripts/change_pagelang.py index 4060c7b..0b8e544 100755 --- a/scripts/change_pagelang.py +++ b/scripts/change_pagelang.py @@ -18,7 +18,7 @@ the one set in -setlang (keep the current language). """ # -# (C) Pywikibot team, 2018-2020 +# (C) Pywikibot team, 2018-2021 # # Distributed under the terms of the MIT license. # @@ -141,7 +141,7 @@ elif arg in ('-always', '-never'): options[opt[1:]] = True else: - gen_factory.handleArg(arg) + gen_factory.handle_arg(arg) if not options.get('setlang'): pywikibot.error('No -setlang parameter given.') diff --git a/scripts/claimit.py b/scripts/claimit.py index 58652da..0e30a11 100755 --- a/scripts/claimit.py +++ b/scripts/claimit.py @@ -47,7 +47,7 @@ """ # -# (C) Pywikibot team, 2013-2020 +# (C) Pywikibot team, 2013-2021 # # Distributed under the terms of the MIT license. # @@ -123,7 +123,7 @@ exists_arg = arg.split(':')[1] continue # Handle page generator args - if gen.handleArg(arg): + if gen.handle_arg(arg): continue commandline_claims.append(arg) if len(commandline_claims) % 2: diff --git a/scripts/clean_sandbox.py b/scripts/clean_sandbox.py index 6522866..4b9c0c6 100755 --- a/scripts/clean_sandbox.py +++ b/scripts/clean_sandbox.py @@ -38,7 +38,7 @@ delay: 7 """ # -# (C) Pywikibot team, 2006-2020 +# (C) Pywikibot team, 2006-2021 # # Distributed under the terms of the MIT license. # @@ -285,7 +285,7 @@ elif opt == 'summary': opts[opt] = value or pywikibot.input('Enter the summary:') else: - gen_factory.handleArg(arg) + gen_factory.handle_arg(arg) generator = gen_factory.getCombinedGenerator() diff --git a/scripts/commons_link.py b/scripts/commons_link.py index dccd99c..29ca5b2 100755 --- a/scripts/commons_link.py +++ b/scripts/commons_link.py @@ -23,7 +23,7 @@ &params; """ # -# (C) Pywikibot team, 2006-2020 +# (C) Pywikibot team, 2006-2021 # # Distributed under the terms of the MIT license. # @@ -133,7 +133,7 @@ elif arg == '-always': options['always'] = True else: - gen_factory.handleArg(arg) + gen_factory.handle_arg(arg) gen = gen_factory.getCombinedGenerator(preload=True) if pywikibot.bot.suggest_help(missing_action='action' not in options, diff --git a/scripts/commonscat.py b/scripts/commonscat.py index 1ef48f1..39a6708 100755 --- a/scripts/commonscat.py +++ b/scripts/commonscat.py @@ -35,7 +35,7 @@ # *Found one template. Add this template # *Found more templates. Ask the user <- still have to implement this # -# (C) Pywikibot team, 2008-2020 +# (C) Pywikibot team, 2008-2021 # # Distributed under the terms of the MIT license. # @@ -523,7 +523,7 @@ elif option == 'always': options[option] = True else: - genFactory.handleArg(arg) + genFactory.handle_arg(arg) if checkcurrent: site = pywikibot.Site() diff --git a/scripts/coordinate_import.py b/scripts/coordinate_import.py index 758b9f2..51527ec 100755 --- a/scripts/coordinate_import.py +++ b/scripts/coordinate_import.py @@ -37,7 +37,7 @@ &params; """ # -# (C) Pywikibot team, 2013-2020 +# (C) Pywikibot team, 2013-2021 # # Distributed under the terms of MIT License. # @@ -163,7 +163,7 @@ create_new = False for arg in local_args: - if generator_factory.handleArg(arg): + if generator_factory.handle_arg(arg): continue if arg == '-create': create_new = True diff --git a/scripts/cosmetic_changes.py b/scripts/cosmetic_changes.py index 601eff1..c140cb0 100644 --- a/scripts/cosmetic_changes.py +++ b/scripts/cosmetic_changes.py @@ -28,7 +28,7 @@ For further information see pywikibot/cosmetic_changes.py """ # -# (C) Pywikibot team, 2006-2020 +# (C) Pywikibot team, 2006-2021 # # Distributed under the terms of the MIT license. # @@ -114,7 +114,7 @@ raise ValueError( 'Unknown ignore mode "{0}"!'.format(ignore_mode)) else: - gen_factory.handleArg(arg) + gen_factory.handle_arg(arg) site = pywikibot.Site() diff --git a/scripts/create_categories.py b/scripts/create_categories.py index 7d68442..a7c9c61 100755 --- a/scripts/create_categories.py +++ b/scripts/create_categories.py @@ -31,7 +31,7 @@ to create [[Category:Cultural heritage monuments in Hensies]]. """ # -# (c) Pywikibot team, 2011-2020 +# (c) Pywikibot team, 2011-2021 # # Distributed under the terms of the MIT license. # @@ -117,7 +117,7 @@ if value: options[opt] = value else: - gen_factory.handleArg(arg) + gen_factory.handle_arg(arg) missing = ['-' + arg for arg in ('basename', 'parent') if arg not in options] diff --git a/scripts/data_ingestion.py b/scripts/data_ingestion.py index d4ce7ad..e526b92 100755 --- a/scripts/data_ingestion.py +++ b/scripts/data_ingestion.py @@ -94,7 +94,7 @@ """ # -# (C) Pywikibot team, 2012-2020 +# (C) Pywikibot team, 2012-2021 # # Distributed under the terms of the MIT license. # @@ -326,7 +326,7 @@ if arg.startswith('-csvdir:'): csv_dir = arg[8:] else: - genFactory.handleArg(arg) + genFactory.handle_arg(arg) config_generator = genFactory.getCombinedGenerator() diff --git a/scripts/delete.py b/scripts/delete.py index b5d001c..ed2c0ae 100755 --- a/scripts/delete.py +++ b/scripts/delete.py @@ -52,7 +52,7 @@ python pwb.py delete -cat:"To delete" -always """ # -# (C) Pywikibot team, 2013-2020 +# (C) Pywikibot team, 2013-2021 # # Distributed under the terms of the MIT license. # @@ -259,7 +259,7 @@ namespaces = mysite.namespaces options['orphansonly'] = namespaces else: - gen_factory.handleArg(arg) + gen_factory.handle_arg(arg) found = arg.find(':') + 1 if found: page_name = arg[found:] diff --git a/scripts/fixing_redirects.py b/scripts/fixing_redirects.py index d272888..179b109 100755 --- a/scripts/fixing_redirects.py +++ b/scripts/fixing_redirects.py @@ -10,7 +10,7 @@ &params; """ # -# (C) Pywikibot team, 2004-2020 +# (C) Pywikibot team, 2004-2021 # # Distributed under the terms of the MIT license. # @@ -200,7 +200,7 @@ for arg in local_args: if arg == '-featured': featured = True - elif genFactory.handleArg(arg): + elif genFactory.handle_arg(arg): pass mysite = pywikibot.Site() diff --git a/scripts/harvest_template.py b/scripts/harvest_template.py index e213c79..e96cbfc 100755 --- a/scripts/harvest_template.py +++ b/scripts/harvest_template.py @@ -88,7 +88,7 @@ -multi """ # -# (C) Pywikibot team, 2013-2020 +# (C) Pywikibot team, 2013-2021 # # Distributed under the terms of MIT License. # @@ -379,7 +379,7 @@ template_title = arg[10:] elif arg.startswith('-create'): options['create'] = True - elif gen.handleArg(arg): + elif gen.handle_arg(arg): if arg.startswith('-transcludes:'): template_title = arg[13:] else: @@ -421,7 +421,7 @@ generator = gen.getCombinedGenerator(preload=True) if not generator: - gen.handleArg('-transcludes:' + template_title) + gen.handle_arg('-transcludes:' + template_title) generator = gen.getCombinedGenerator(preload=True) bot = HarvestRobot(generator, template_title, fields, **options) diff --git a/scripts/illustrate_wikidata.py b/scripts/illustrate_wikidata.py index 117cff7..5567304 100755 --- a/scripts/illustrate_wikidata.py +++ b/scripts/illustrate_wikidata.py @@ -14,7 +14,7 @@ &params; """ # -# (C) Pywikibot team, 2013-2020 +# (C) Pywikibot team, 2013-2021 # # Distributed under the terms of MIT License. # @@ -103,7 +103,7 @@ else: wdproperty = arg[10:] else: - generator_factory.handleArg(arg) + generator_factory.handle_arg(arg) generator = generator_factory.getCombinedGenerator(preload=True) if not generator: diff --git a/scripts/imagecopy.py b/scripts/imagecopy.py index ae631b4..547c0f0 100644 --- a/scripts/imagecopy.py +++ b/scripts/imagecopy.py @@ -57,7 +57,7 @@ By default the bot works on your home wiki (set in user-config) """ # -# (C) Pywikibot team, 2003-2020 +# (C) Pywikibot team, 2003-2021 # # Distributed under the terms of the MIT license. # @@ -505,7 +505,7 @@ elif arg == '-delete': delete_after_done = True else: - genFactory.handleArg(arg) + genFactory.handle_arg(arg) pregenerator = genFactory.getCombinedGenerator(preload=True) if not pregenerator: -- To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/655900 To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Change-Id: Ib2431deb7c6846b77d5ce53f1031643ea7ae857c Gerrit-Change-Number: 655900 Gerrit-PatchSet: 4 Gerrit-Owner: Homeboy 445 <akshitsan13(a)gmail.com> Gerrit-Reviewer: D3r1ck01 <xsavitar.wiki(a)aol.com> Gerrit-Reviewer: Xqt <info(a)gno.de> Gerrit-Reviewer: jenkins-bot Gerrit-MessageType: merged

3 years, 4 months

[Gerrit] ...core[master]: [IMPR] simplify three tests in TestClaimSetValue

by jenkins-bot (Code Review)

jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/656177 ) Change subject: [IMPR] simplify three tests in TestClaimSetValue ...................................................................... [IMPR] simplify three tests in TestClaimSetValue Change-Id: I37bb142a7251c6d0e2899f231279a72b3397f53d --- M tests/wikibase_tests.py 1 file changed, 9 insertions(+), 6 deletions(-) Approvals: Xqt: Looks good to me, approved jenkins-bot: Verified diff --git a/tests/wikibase_tests.py b/tests/wikibase_tests.py index bd66528..7cff163 100644 --- a/tests/wikibase_tests.py +++ b/tests/wikibase_tests.py @@ -1630,8 +1630,9 @@ wikidata = self.get_repo() claim = pywikibot.Claim(wikidata, 'P856') self.assertEqual(claim.type, 'url') - claim.setTarget('https://en.wikipedia.org/') - self.assertEqual(claim.target, 'https://en.wikipedia.org/') + target = 'https://en.wikipedia.org/' + claim.setTarget(target) + self.assertEqual(claim.target, target) def test_set_WbMonolingualText(self): """Test setting claim of monolingualtext type.""" @@ -1657,16 +1658,18 @@ wikidata = self.get_repo() claim = pywikibot.Claim(wikidata, 'P2535') self.assertEqual(claim.type, 'math') - claim.setTarget('a^2 + b^2 = c^2') - self.assertEqual(claim.target, 'a^2 + b^2 = c^2') + target = 'a^2 + b^2 = c^2' + claim.setTarget(target) + self.assertEqual(claim.target, target) def test_set_identifier(self): """Test setting claim of external-id type.""" wikidata = self.get_repo() claim = pywikibot.Claim(wikidata, 'P214') self.assertEqual(claim.type, 'external-id') - claim.setTarget('Any string is a valid identifier') - self.assertEqual(claim.target, 'Any string is a valid identifier') + target = 'Any string is a valid identifier' + claim.setTarget(target) + self.assertEqual(claim.target, target) def test_set_date(self): """Test setting claim of time type.""" -- To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/656177 To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Change-Id: I37bb142a7251c6d0e2899f231279a72b3397f53d Gerrit-Change-Number: 656177 Gerrit-PatchSet: 1 Gerrit-Owner: Lokal Profil <andre.costa(a)wikimedia.se> Gerrit-Reviewer: Xqt <info(a)gno.de> Gerrit-Reviewer: jenkins-bot Gerrit-MessageType: merged

3 years, 4 months

[Gerrit] ...core[master]: Bump year of two files updated in 2021

by jenkins-bot (Code Review)

jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/656178 ) Change subject: Bump year of two files updated in 2021 ...................................................................... Bump year of two files updated in 2021 Bumping the dates was missed in 13713379026b1238d1f7b93166fcf20482e92b4b, year in third file was bumped as part of 5cbffbe1dfe57f240a48675f823cb153dff10a59. Change-Id: Icf4d12d346a8571d79b329d29ff04da94de4c7af --- M pywikibot/page/__init__.py M tests/wikistats_tests.py 2 files changed, 2 insertions(+), 2 deletions(-) Approvals: Xqt: Looks good to me, approved jenkins-bot: Verified diff --git a/pywikibot/page/__init__.py b/pywikibot/page/__init__.py index b4d259f..aa2fd6e 100644 --- a/pywikibot/page/__init__.py +++ b/pywikibot/page/__init__.py @@ -11,7 +11,7 @@ """ # -# (C) Pywikibot team, 2008-2020 +# (C) Pywikibot team, 2008-2021 # # Distributed under the terms of the MIT license. # diff --git a/tests/wikistats_tests.py b/tests/wikistats_tests.py index a734807..5d4b206 100644 --- a/tests/wikistats_tests.py +++ b/tests/wikistats_tests.py @@ -1,7 +1,7 @@ # -*- coding: utf-8 -*- """Test cases for the WikiStats dataset.""" # -# (C) Pywikibot team, 2014-2020 +# (C) Pywikibot team, 2014-2021 # # Distributed under the terms of the MIT license. # -- To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/656178 To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Change-Id: Icf4d12d346a8571d79b329d29ff04da94de4c7af Gerrit-Change-Number: 656178 Gerrit-PatchSet: 1 Gerrit-Owner: Lokal Profil <andre.costa(a)wikimedia.se> Gerrit-Reviewer: Xqt <info(a)gno.de> Gerrit-Reviewer: jenkins-bot Gerrit-MessageType: merged

3 years, 4 months

[Gerrit] ...i18n[master]: Localisation updates from https://translatewiki.net.

by Xqt (Code Review)

Xqt has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/i18n/+/656193 ) Change subject: Localisation updates from https://translatewiki.net. ...................................................................... Localisation updates from https://translatewiki.net. Change-Id: I6794f0b22e7da3dcb162c9294127d8fa5ea7283b --- M noreferences/sl.json 1 file changed, 3 insertions(+), 2 deletions(-) Approvals: L10n-bot: Looks good to me, approved Xqt: Verified; Looks good to me, approved diff --git a/noreferences/sl.json b/noreferences/sl.json index bf20f3f..572da3d 100644 --- a/noreferences/sl.json +++ b/noreferences/sl.json @@ -1,9 +1,10 @@ { "@metadata": { "authors": [ - "Dbc334" + "Dbc334", + "Eleassar" ] }, "noreferences-add-tag": "Robot: Dodajanje manjkajoče oznake <references />", - "noreferences-fix-tag": "Bot: Popravek neveljavne skladnje oznake <references />" + "noreferences-fix-tag": "Bot: popravek neveljavne skladnje značke <references />" } -- To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/i18n/+/656193 To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings Gerrit-Project: pywikibot/i18n Gerrit-Branch: master Gerrit-Change-Id: I6794f0b22e7da3dcb162c9294127d8fa5ea7283b Gerrit-Change-Number: 656193 Gerrit-PatchSet: 1 Gerrit-Owner: L10n-bot <l10n-bot(a)translatewiki.net> Gerrit-Reviewer: L10n-bot <l10n-bot(a)translatewiki.net> Gerrit-Reviewer: Xqt <info(a)gno.de> Gerrit-Reviewer: jenkins-bot Gerrit-MessageType: merged

3 years, 4 months

[Gerrit] ...core[master]: [IMPR] remove some IDE warnings for site.py

by jenkins-bot (Code Review)

jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/653977 ) Change subject: [IMPR] remove some IDE warnings for site.py ...................................................................... [IMPR] remove some IDE warnings for site.py Major change is to convert Site.redirectRegex() to the property Site.redirect_regex, to avoid different signature than base method's one. Method has been deprecated with FutureWarning. Change-Id: Icf7ea15683fa8f566acb7b28a7c37b3790bed1a2 --- M pywikibot/cosmetic_changes.py M pywikibot/page/__init__.py M pywikibot/site/__init__.py M scripts/redirect.py M tests/basesite_tests.py 5 files changed, 27 insertions(+), 18 deletions(-) Approvals: Xqt: Looks good to me, approved jenkins-bot: Verified diff --git a/pywikibot/cosmetic_changes.py b/pywikibot/cosmetic_changes.py index 6df193a..f6273ea 100755 --- a/pywikibot/cosmetic_changes.py +++ b/pywikibot/cosmetic_changes.py @@ -788,7 +788,7 @@ if not self.template: exceptions = ['comment', 'math', 'nowiki', 'pre', 'syntaxhighlight', 'template', 'timeline', - self.site.redirectRegex()] + self.site.redirect_regex] text = textlib.replaceExcept( text, r'(?m)' diff --git a/pywikibot/page/__init__.py b/pywikibot/page/__init__.py index b4d259f..b2c38cd 100644 --- a/pywikibot/page/__init__.py +++ b/pywikibot/page/__init__.py @@ -2255,7 +2255,7 @@ raise pywikibot.NoPage(self) if self.exists() and not self.isRedirectPage() and not force: raise pywikibot.IsNotRedirectPage(self) - redirect_regex = self.site.redirectRegex() + redirect_regex = self.site.redirect_regex if self.exists(): old_text = self.get(get_redirect=True) else: diff --git a/pywikibot/site/__init__.py b/pywikibot/site/__init__.py index e29b610..fe62cba 100644 --- a/pywikibot/site/__init__.py +++ b/pywikibot/site/__init__.py @@ -202,7 +202,8 @@ gen.set_maximum_items(total) return gen - def _request_class(self, kwargs): + @staticmethod + def _request_class(kwargs): """ Get the appropriate class. @@ -333,7 +334,7 @@ if login_manager.login(retry=True, autocreate=autocreate): self._username = login_manager.username del self.userinfo # force reloading - self.userinfo # load userinfo + assert self.userinfo['name'] == self.username() # load userinfo self._loginstatus = _LoginStatus.AS_USER else: self._loginstatus = _LoginStatus.NOT_LOGGED_IN # failure @@ -517,7 +518,8 @@ 'articlepath must end with /$1' return self.siteinfo['general']['articlepath'][:-2] - def assert_valid_iter_params(self, msg_prefix, start, end, reverse, + @staticmethod + def assert_valid_iter_params(msg_prefix, start, end, reverse, is_ts=True): """Validate iterating API parameters. @@ -836,7 +838,13 @@ # return the magic word without the preceding '#' character return self.getmagicwords('redirect')[0].lstrip('#') + @deprecated('redirect_regex', since='20210103', future_warning=True) def redirectRegex(self): + """Return a compiled regular expression matching on redirect pages.""" + return self.redirect_regex + + @property + def redirect_regex(self): """Return a compiled regular expression matching on redirect pages. Group 1 in the regex match object will be the target title. @@ -891,9 +899,10 @@ except KeyError: pywikibot.warning( 'Broken namespace alias "{0}" (id: {1}) on {2}'.format( - item['*'], item['id'], self)) - if item['*'] not in namespace: - namespace.aliases.append(item['*']) + item['*'], ns, self)) + else: + if item['*'] not in namespace: + namespace.aliases.append(item['*']) return _namespaces @@ -2227,7 +2236,7 @@ By default, retrieves the last (current) revision of the page, unless any of the optional parameters revids, startid, endid, - starttime, endtime, rvdir, user, excludeuser, or limit are + starttime, endtime, rvdir, user, excludeuser, or total are specified. Unless noted below, all parameters not specified default to False. @@ -2254,9 +2263,10 @@ @keyword starttime: retrieve revisions starting at this Timestamp @keyword endtime: stop upon reaching this Timestamp @keyword rvdir: if false, retrieve newest revisions first (default); - if true, retrieve earliest first + if true, retrieve oldest first @keyword user: retrieve only revisions authored by this user @keyword excludeuser: retrieve all revisions not authored by this user + @keyword total: number of revisions to retrieve @raises ValueError: invalid startid/endid or starttime/endtime values @raises pywikibot.Error: revids belonging to a different page """ @@ -4334,13 +4344,12 @@ raise RuntimeError( 'Unexpected success of upload action without parameters.') - def stash_info(self, file_key, props=False): + def stash_info(self, file_key, props=None): """Get the stash info for a given file key. @see: U{https://www.mediawiki.org/wiki/API:Stashimageinfo} """ - if not props: - props = False + props = props or False req = self._simple_request( action='query', prop='stashimageinfo', siifilekey=file_key, siiprop=props) @@ -5584,8 +5593,8 @@ @see: U{https://www.mediawiki.org/wiki/API:Watchlistraw} - @param force_reload: Reload watchlist - @type force_reload: bool + @param force: Reload watchlist + @type force: bool @param total: if not None, limit the generator to yielding this many items in total @type total: int @@ -6282,7 +6291,7 @@ @param claim: A Claim object to remove the sources from @type claim: pywikibot.Claim @param sources: A list of Claim objects that are sources - @type sources: pywikibot.Claim + @type sources: list @param bot: Whether to mark the edit as a bot edit @type bot: bool @param summary: Edit summary diff --git a/scripts/redirect.py b/scripts/redirect.py index 9b07caf..21584c5 100755 --- a/scripts/redirect.py +++ b/scripts/redirect.py @@ -138,7 +138,7 @@ redict = {} # open xml dump and read page titles out of it dump = xmlreader.XmlDump(xmlFilename) - redirR = self.site.redirectRegex() + redirR = self.site.redirect_regex readPagesCount = 0 pageTitles = set() for entry in dump.parse(): diff --git a/tests/basesite_tests.py b/tests/basesite_tests.py index 392789c..96a8395 100644 --- a/tests/basesite_tests.py +++ b/tests/basesite_tests.py @@ -107,7 +107,7 @@ self.assertEqual(foo, '[[Foo]]') self.assertFalse(mysite.isInterwikiLink('foo')) - self.assertIsInstance(mysite.redirectRegex().pattern, str) + self.assertIsInstance(mysite.redirect_regex.pattern, str) self.assertIsInstance(mysite.category_on_one_line(), bool) self.assertTrue(mysite.sametitle('Template:Test', 'Template:Test')) self.assertTrue(mysite.sametitle('Template: Test', 'Template: Test')) -- To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/653977 To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Change-Id: Icf7ea15683fa8f566acb7b28a7c37b3790bed1a2 Gerrit-Change-Number: 653977 Gerrit-PatchSet: 6 Gerrit-Owner: Mpaa <mpaa.wiki(a)gmail.com> Gerrit-Reviewer: D3r1ck01 <xsavitar.wiki(a)aol.com> Gerrit-Reviewer: Xqt <info(a)gno.de> Gerrit-Reviewer: jenkins-bot Gerrit-MessageType: merged

3 years, 4 months

← Newer
1
...
5
6
7
8
9
10
11
...
14
Older →

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

Pywikibot-commits January 2021