jenkins-bot has submitted this change and it was merged.

Change subject: [bugfix] Workaround UnicodeDecodeError on api error ......................................................................

[bugfix] Workaround UnicodeDecodeError on api error

When an API error happens it tries to log the parameters but it fails as they are in a dict and the str and repr of a dict use the repr of each key and value. If the value is then a Page instance it gets in Python 2 bytes returned encoded with the console encoding. If then the str or repr of the dict is inserted into a unicode it must decode the bytes and uses by default ASCII which fails if the Page's title contains non-ASCII characters.

This patch works around that by manually decoding the bytes in Python 2 using the console encoding whenever an API error happens but does not actually fix Page's repr method so it might still fail at other places. It assumes that the rest of the str is either also encoded using that or is encoded using ASCII and that the console encoding is a superset of ASCII.

Conflicts: pywikibot/data/api.py tests/api_tests.py

Bug: T66958 Change-Id: I298b7594599dd189211a8c268c7e094d042f40e6 --- M pywikibot/data/api.py 1 file changed, 14 insertions(+), 2 deletions(-)

Approvals: Merlijn van Deen: Looks good to me, approved jenkins-bot: Verified

diff --git a/pywikibot/data/api.py b/pywikibot/data/api.py index f0a5831..0d4d01e 100644 --- a/pywikibot/data/api.py +++ b/pywikibot/data/api.py @@ -1666,11 +1666,17 @@ pywikibot.error("Detected MediaWiki API exception %s%s" % (class_name, "; retrying" if retry else "; raising")) + # Due to bug T66958, Page's repr may return non ASCII bytes + # Get as bytes in PY2 and decode with the console encoding as + # the rest should be ASCII anyway. + param_repr = str(self._params) + if sys.version_info[0] == 2: + param_repr = param_repr.decode(config.console_encoding) pywikibot.log(u"MediaWiki exception %s details:\n" u" query=\n%s\n" u" response=\n%s" % (class_name, - pprint.pformat(self._params), + pprint.pformat(param_repr), result))

if retry: @@ -1726,8 +1732,14 @@ for e in user_tokens.items()))) # raise error try: + # Due to bug T66958, Page's repr may return non ASCII bytes + # Get as bytes in PY2 and decode with the console encoding as + # the rest should be ASCII anyway. + param_repr = str(self._params) + if sys.version_info[0] == 2: + param_repr = param_repr.decode(config.console_encoding) pywikibot.log(u"API Error: query=\n%s" - % pprint.pformat(self._params)) + % pprint.pformat(param_repr)) pywikibot.log(u" response=\n%s" % result)