Pywikibot-commits July 2015

pywikibot-commits@lists.wikimedia.org

2 participants
224 discussions

Passed: wikimedia/pywikibot-core#2597 (master - 1db58db)
by Travis CI 30 Jul '15

30 Jul '15

Build Update for wikimedia/pywikibot-core ------------------------------------- Build: #2597 Status: Passed Duration: 29 minutes and 33 seconds Commit: 1db58db (master) Author: John Vandenberg Message: Python issue #10254 Pywikibot had problems with NFC normalizing unicode, resulting in Python issue #10254, which was resolved in Python 2.7.2 and backported to Python 2.6.7. It is unlikely that supporting Python 2.7.0 and 2.7.1 is still necessary, so decommission support for Python 2.7.0 and 2.7.1. Workaround the problem using unicodedata2 backport of unicodedata, which includes Unicode 7.0 support. If unicodedata2 isnt available on Python 2.6.6, the Link constructor raises UnicodeError for titles containing combining characters, which will allow normal usage for many languages, but will fail predictably for page titles which could encounter this error. Moves the unicode test from the version script to the test suite, as the only supported platform with this bug is Python 2.6.6. Bug: T102461 Change-Id: If4a8e8eed682b837dd468b02a5d98b01ebf37584 View the changeset: https://github.com/wikimedia/pywikibot-core/compare/e101dcb2c6e7...1db58dbb… View the full build log and details: https://travis-ci.org/wikimedia/pywikibot-core/builds/73363259 -- You can configure recipients for build notifications in your .travis.yml file. See http://docs.travis-ci.com/user/notifications

1 0

[Gerrit] Python issue #10254 - change (pywikibot/core)
by jenkins-bot (Code Review) 30 Jul '15

30 Jul '15

jenkins-bot has submitted this change and it was merged. Change subject: Python issue #10254 ...................................................................... Python issue #10254 Pywikibot had problems with NFC normalizing unicode, resulting in Python issue #10254, which was resolved in Python 2.7.2 and backported to Python 2.6.7. It is unlikely that supporting Python 2.7.0 and 2.7.1 is still necessary, so decommission support for Python 2.7.0 and 2.7.1. Workaround the problem using unicodedata2 backport of unicodedata, which includes Unicode 7.0 support. If unicodedata2 isnt available on Python 2.6.6, the Link constructor raises UnicodeError for titles containing combining characters, which will allow normal usage for many languages, but will fail predictably for page titles which could encounter this error. Moves the unicode test from the version script to the test suite, as the only supported platform with this bug is Python 2.6.6. Bug: T102461 Change-Id: If4a8e8eed682b837dd468b02a5d98b01ebf37584 --- M .appveyor.yml M pwb.py M pywikibot/bot.py M pywikibot/page.py M pywikibot/version.py M requirements.txt M scripts/version.py M setup.py M tests/link_tests.py A tests/python_tests.py 10 files changed, 193 insertions(+), 41 deletions(-) Approvals: John Vandenberg: Looks good to me, but someone else must approve XZise: Looks good to me, approved jenkins-bot: Verified diff --git a/.appveyor.yml b/.appveyor.yml index acd7c07..1befd5d 100644 --- a/.appveyor.yml +++ b/.appveyor.yml @@ -15,9 +15,9 @@ matrix: - # Pywikibot support matrix suggests 'should run' on Python 2.6.5+ - # Test the lowest release of each major Python version. + # Test the lowest supported release of each major Python version. + # Pywikibot support matrix suggests 'should run' on Python 2.6.5+ # Only Python 2.6.6 is able to be set up on Appveyor. # https://github.com/ogrisel/python-appveyor-demo/issues/10 # fwiw, Redhat Enterprise Linux ships with 2.6.6. @@ -26,11 +26,8 @@ PYTHON_VERSION: "2.6.6" PYTHON_ARCH: "64" - # Python 2.7.0 needs "2.7" as the version instead of "2.7.0" - # https://github.com/ogrisel/python-appveyor-demo/issues/9 - - - PYTHON: "C:\\Python270-x64" - PYTHON_VERSION: "2.7" + - PYTHON: "C:\\Python272-x64" + PYTHON_VERSION: "2.7.2" PYTHON_ARCH: "64" - PYTHON: "C:\\Python330-x64" diff --git a/pwb.py b/pwb.py index 0efba76..3266a85 100755 --- a/pwb.py +++ b/pwb.py @@ -29,6 +29,30 @@ from warnings import warn +PYTHON_VERSION = sys.version_info[:3] +PY2 = (PYTHON_VERSION[0] == 2) +PY26 = (PYTHON_VERSION < (2, 7)) + +versions_required_message = """ +Pywikibot not available on: +%s + +Pywikibot is only supported under Python 2.6.5+, 2.7.2+ or 3.3+ +""" + + +def python_is_supported(): + """Check that Python is supported.""" + # Any change to this must be copied to setup.py + return (PYTHON_VERSION >= (3, 3, 0) or + (PY2 and PYTHON_VERSION >= (2, 7, 2)) or + (PY26 and PYTHON_VERSION >= (2, 6, 5))) + + +if not python_is_supported(): + print(versions_required_message % sys.version) + sys.exit(1) + pwb = None @@ -106,17 +130,6 @@ path = path[0].upper() + path[1:] return path - -if sys.version_info[0] not in (2, 3): - raise RuntimeError("ERROR: Pywikibot only runs under Python 2 " - "or Python 3") -version = tuple(sys.version_info)[:3] -if version < (2, 6, 5): - raise RuntimeError("ERROR: Pywikibot only runs under Python 2.6.5 " - "or higher") -if version >= (3, ) and version < (3, 3): - raise RuntimeError("ERROR: Pywikibot only runs under Python 3.3 " - "or higher") # Establish a normalised path for the directory containing pwb.py. # Either it is '.' if the user's current working directory is the same, diff --git a/pywikibot/bot.py b/pywikibot/bot.py index a110ea1..6c334b6 100644 --- a/pywikibot/bot.py +++ b/pywikibot/bot.py @@ -367,7 +367,11 @@ all_modules = sys.modules.keys() # These are the main dependencies of pywikibot. - check_package_list = ['requests', 'mwparserfromhell'] + check_package_list = [ + 'requests', + 'mwparserfromhell', + 'unicodedata', 'unicodedata2', # T102461 + ] # report all imported packages if config.verbose_output: diff --git a/pywikibot/page.py b/pywikibot/page.py index 278ed54..479eace 100644 --- a/pywikibot/page.py +++ b/pywikibot/page.py @@ -24,7 +24,11 @@ import logging import re import sys -import unicodedata + +try: + import unicodedata2 as unicodedata +except ImportError: + import unicodedata from collections import defaultdict, namedtuple from warnings import warn @@ -53,6 +57,7 @@ UserRightsError, ) from pywikibot.tools import ( + PYTHON_VERSION, MediaWikiVersion, UnicodeMixin, ComparableMixin, DotReadableDict, deprecated, deprecate_arg, deprecated_args, issue_deprecation_warning, first_upper, remove_last_args, _NotImplementedWarning, @@ -4632,6 +4637,10 @@ contain one (defaults to 0) @type defaultNamespace: int + @raises UnicodeError: text could not be converted to unicode. + On Python 2.6.6 without unicodedata2, this could also be raised + if the text contains combining characters. + See https://phabricator.wikimedia.org/T102461 """ source_is_page = isinstance(source, BasePage) @@ -4663,10 +4672,17 @@ t = html2unicode(self._text) # Normalize unicode string to a NFC (composed) format to allow - # proper string comparisons. According to - # https://svn.wikimedia.org/viewvc/mediawiki/branches/REL1_6/phase3/includes/… - # the MediaWiki code normalizes everything to NFC, not NFKC - # (which might result in information loss). + # proper string comparisons to strings output from MediaWiki API. + # Due to Python issue 10254, this is not possible on Python 2.6.6 + # if the string contains combining characters. See T102461. + if (PYTHON_VERSION == (2, 6, 6) and + unicodedata.__name__ != 'unicodedata2' and + any(unicodedata.combining(c) for c in t)): + raise UnicodeError( + 'Link(%r, %s): combining characters detected, which are ' + 'not supported by Pywikibot on Python 2.6.6. See ' + 'https://phabricator.wikimedia.org/T102461' + % (t, self._source)) t = unicodedata.normalize('NFC', t) # This code was adapted from Title.php : secureAndSplit() diff --git a/pywikibot/version.py b/pywikibot/version.py index 45cec8b..4cd65da 100644 --- a/pywikibot/version.py +++ b/pywikibot/version.py @@ -531,6 +531,8 @@ if '__version__' in package.__dict__: info['ver'] = package.__version__ + elif name.startswith('unicodedata'): + info['ver'] = package.unidata_version elif name == 'mwlib': # mwlib 0.14.3 does not include a __init__.py module = __import__(name + '._version', fromlist=['_version'], level=0) diff --git a/requirements.txt b/requirements.txt index 7febc63..4410981 100644 --- a/requirements.txt +++ b/requirements.txt @@ -34,6 +34,9 @@ ipaddr>=2.1.10 ; python_version < '3' +git+https://github.com/jayvdb/unicodedata2@issue_2#egg=unicodedata2-7.0.0 ; python_version < '2.7' +unicodedata2 ; python_version >= '2.7' + # core interwiki_graph.py: git+https://github.com/nlhepler/pydot#egg=pydot-1.0.29 diff --git a/scripts/version.py b/scripts/version.py index 7a90fd8..3d42a86 100755 --- a/scripts/version.py +++ b/scripts/version.py @@ -63,13 +63,7 @@ ' Please reinstall requests!') pywikibot.output('Python: %s' % sys.version) - normalize_text = u'\u092e\u093e\u0930\u094d\u0915 \u091c\u093c\u0941\u0915\u0947\u0930\u092c\u0930\u094d\u0917' - if normalize_text != __import__('unicodedata').normalize( - 'NFC', normalize_text): - pywikibot.output(u' unicode test: triggers problem #3081100') - else: - pywikibot.output(u' unicode test: ok') check_environ('PYWIKIBOT2_DIR') check_environ('PYWIKIBOT2_DIR_PWB') check_environ('PYWIKIBOT2_NO_USER_CONFIG') diff --git a/setup.py b/setup.py index 0e9dab9..01920d1 100644 --- a/setup.py +++ b/setup.py @@ -10,6 +10,29 @@ import os import sys +PYTHON_VERSION = sys.version_info[:3] +PY2 = (PYTHON_VERSION[0] == 2) +PY26 = (PYTHON_VERSION < (2, 7)) + +versions_required_message = """ +Pywikibot not available on: +%s + +Pywikibot is only supported under Python 2.6.5+, 2.7.2+ or 3.3+ +""" + + +def python_is_supported(): + """Check that Python is supported.""" + # Any change to this must be copied to pwb.py + return (PYTHON_VERSION >= (3, 3, 0) or + (PY2 and PYTHON_VERSION >= (2, 7, 2)) or + (PY26 and PYTHON_VERSION >= (2, 6, 5))) + + +if not python_is_supported(): + raise RuntimeError(versions_required_message % sys.version) + test_deps = [] dependencies = ['requests'] @@ -28,9 +51,10 @@ # 0.6.1 supports socket.io 1.0, but WMF is using 0.9 (T91393 and T85716) 'rcstream': ['socketIO-client<0.6.1'], 'security': ['requests[security]'], + 'unicode7': ['unicodedata2'], } -if sys.version_info[0] == 2: +if PY2: # Additional core library dependencies which are only available on Python 2 extra_deps.update({ 'csv': ['unicodecsv'], @@ -69,21 +93,23 @@ ] if sys.version_info[0] == 2: - if sys.version_info < (2, 6, 5): - raise RuntimeError("ERROR: Pywikibot only runs under Python 2.6.5 or higher") - elif sys.version_info[1] == 6: + if PY26: # requests security extra includes pyOpenSSL. cryptography is the # dependency of pyOpenSSL. 0.8.2 is the newest and compatible version # for Python 2.6, which won't raise unexpected DeprecationWarning. extra_deps['security'].append('cryptography<=0.8.2') # work around distutils hardcoded unittest dependency import unittest # noqa - if 'test' in sys.argv and sys.version_info < (2, 7): + if 'test' in sys.argv: import unittest2 sys.modules['unittest'] = unittest2 script_deps['replicate_wiki.py'] = ['argparse'] dependencies.append('future') # provides collections backports + dependency_links.append( + 'git+https://github.com/jayvdb/unicodedata2@issue_2#egg=unicodedata2-7.0.0') + + dependencies += extra_deps['unicode7'] # T102461 workaround # tools.ip does not have a hard dependency on an IP address module, # as it falls back to using regexes if one is not available. @@ -108,11 +134,6 @@ # mwlib is not available for py3 script_deps['patrol'] = ['mwlib'] - -if sys.version_info[0] == 3: - if sys.version_info[1] < 3: - print("ERROR: Python 3.3 or higher is required!") - sys.exit(1) # Some of the ui_tests depend on accessing the console window's menu # to set the console font and copy and paste, achieved using pywinauto diff --git a/tests/link_tests.py b/tests/link_tests.py index f271bb5b..34cd32c 100644 --- a/tests/link_tests.py +++ b/tests/link_tests.py @@ -1,7 +1,7 @@ # -*- coding: utf-8 -*- """Test Link functionality.""" # -# (C) Pywikibot team, 2014 +# (C) Pywikibot team, 2014-2015 # # Distributed under the terms of the MIT license. # @@ -10,9 +10,12 @@ __version__ = '$Id$' import pywikibot + from pywikibot import config2 as config from pywikibot.page import Link, Page from pywikibot.exceptions import Error, InvalidTitle +from pywikibot.tools import PYTHON_VERSION + from tests.aspects import ( unittest, AlteredDefaultSiteTestCase as LinkTestCase, @@ -154,6 +157,35 @@ l = Link('/bar', self.get_site()) self.assertEquals(l.title, '/bar') + +class Issue10254TestCase(DefaultDrySiteTestCase): + + """Test T102461 (Python issue 10254).""" + + def setUp(self): + """Set up test case.""" + super(Issue10254TestCase, self).setUp() + self._orig_unicodedata = pywikibot.page.unicodedata + + def tearDown(self): + """Tear down test case.""" + pywikibot.page.unicodedata = self._orig_unicodedata + super(Issue10254TestCase, self).tearDown() + + def test_no_change(self): + """Test T102461 (Python issue 10254) is not encountered.""" + title = 'Li̍t-sṳ́' + l = Link(title, self.site) + self.assertEqual(l.title, 'Li̍t-sṳ́') + + @unittest.skipIf(PYTHON_VERSION != (2, 6, 6), 'Python 2.6.6-only test') + def test_py266_bug_exception(self): + """Test Python issue 10254 causes an exception.""" + pywikibot.page.unicodedata = __import__('unicodedata') + title = 'Li̍t-sṳ́' + self.assertRaises(UnicodeError, Link, title, self.site) + + # ---- The first set of tests are explicit links, starting with a ':'. diff --git a/tests/python_tests.py b/tests/python_tests.py new file mode 100755 index 0000000..218cc40 --- /dev/null +++ b/tests/python_tests.py @@ -0,0 +1,70 @@ +#!/usr/bin/python +# -*- coding: utf-8 -*- +"""Tests Python features.""" +# +# (C) Pywikibot team, 2015 +# +# Distributed under the terms of the MIT license. +from __future__ import unicode_literals + +__version__ = '$Id$' + +import unicodedata +try: + import unicodedata2 +except ImportError: + unicodedata2 = None + +from pywikibot.tools import PYTHON_VERSION + +from tests.aspects import TestCase, unittest +from tests.utils import expected_failure_if + +# TODO: +# very old +# http://bugs.python.org/issue2517 +# +# unicode +# http://sourceforge.net/p/pywikipediabot/bugs/1246/ +# http://bugs.python.org/issue10254 +# +# ip +# http://bugs.python.org/issue22282 +# +# http://bugs.python.org/issue7559 +# +# diff +# http://bugs.python.org/issue2142 +# http://bugs.python.org/issue11747 +# http://sourceforge.net/p/pywikipediabot/bugs/509/ +# https://phabricator.wikimedia.org/T57329 +# http://bugs.python.org/issue1528074 +# http://bugs.python.org/issue1678345 + + +class PythonTestCase(TestCase): + + """Test Python bugs and features.""" + + net = False + + @expected_failure_if((2, 7, 0) <= PYTHON_VERSION < (2, 7, 2) or + PYTHON_VERSION == (2, 6, 6)) + def test_issue_10254(self): + """Test Python issue #10254.""" + # Python 2.6.6, 2.7.0 and 2.7.1 have a bug in this routine. + # See T102461 and http://bugs.python.org/issue10254 + text = 'Li̍t-sṳ́' + self.assertEqual(text, unicodedata.normalize('NFC', text)) + + @unittest.skipIf(not unicodedata2, 'unicodedata2 not found') + def test_issue_10254_unicodedata2(self): + text = 'Li̍t-sṳ́' + self.assertEqual(text, unicodedata2.normalize('NFC', text)) + + +if __name__ == '__main__': + try: + unittest.main() + except SystemExit: + pass -- To view, visit https://gerrit.wikimedia.org/r/218884 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: If4a8e8eed682b837dd468b02a5d98b01ebf37584 Gerrit-PatchSet: 16 Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Owner: John Vandenberg <jayvdb(a)gmail.com> Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com> Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com> Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl> Gerrit-Reviewer: XZise <CommodoreFabianus(a)gmx.de> Gerrit-Reviewer: Xqt <info(a)gno.de> Gerrit-Reviewer: jenkins-bot <>

1 0

Passed: wikimedia/pywikibot-core#2596 (master - e101dcb)
by Travis CI 30 Jul '15

30 Jul '15

Build Update for wikimedia/pywikibot-core ------------------------------------- Build: #2596 Status: Passed Duration: 26 minutes and 55 seconds Commit: e101dcb (master) Author: jenkins-bot Message: Merge "[bugfix] enable misspelling.py for misspellingTemplate localisation" View the changeset: https://github.com/wikimedia/pywikibot-core/compare/de2c9581bdff...e101dcb2… View the full build log and details: https://travis-ci.org/wikimedia/pywikibot-core/builds/73278196 -- You can configure recipients for build notifications in your .travis.yml file. See http://docs.travis-ci.com/user/notifications

1 0

[Gerrit] [bugfix] enable misspelling.py for misspellingTemplate local... - change (pywikibot/core)
by jenkins-bot (Code Review) 30 Jul '15

30 Jul '15

jenkins-bot has submitted this change and it was merged. Change subject: [bugfix] enable misspelling.py for misspellingTemplate localisation ...................................................................... [bugfix] enable misspelling.py for misspellingTemplate localisation disambPage.templatesWithParams() return the template page and page.title() includes the namespace. The misspellingTemplate names doesn't include the namespace and comparing both does never fit. The generator is assigned after the constructor of the super class was called because it uses the class' site object. - remove obsolete misspellingTemplate entries - Add method docs - shorten lines beneath 80 chars - change lang code index and comparing operator - use placeholders for arguments in setSummaryMessage - enable self.site and inherit mysite, mylang - add script to flake8-docstrings-mandatory Change-Id: I907bb4047d42afa4f9044a870f18cbaa724302c8 --- M scripts/misspelling.py M scripts/solve_disambiguation.py M tox.ini 3 files changed, 35 insertions(+), 21 deletions(-) Approvals: John Vandenberg: Looks good to me, approved jenkins-bot: Verified diff --git a/scripts/misspelling.py b/scripts/misspelling.py index 6285bb2..3ccb4b4 100755 --- a/scripts/misspelling.py +++ b/scripts/misspelling.py @@ -22,7 +22,7 @@ wikipedia, user, etc. namespaces. """ # (C) Daniel Herding, 2007 -# (C) Pywikibot team, 2007-2014 +# (C) Pywikibot team, 2007-2015 # # Distributed under the terms of the MIT license. # @@ -50,11 +50,7 @@ """Spelling bot.""" misspellingTemplate = { - 'da': None, # uses simple redirects 'de': u'Falschschreibung', - 'en': None, # uses simple redirects - 'hu': None, # uses simple redirects - 'nl': None, } # Optional: if there is a category, one can use the -start @@ -68,22 +64,28 @@ } def __init__(self, always, firstPageTitle, main_only): + """Constructor.""" super(MisspellingRobot, self).__init__( - always, [], True, False, - self.createPageGenerator(firstPageTitle), False, main_only) + always, [], True, False, None, False, main_only) + self.generator = self.createPageGenerator(firstPageTitle) def createPageGenerator(self, firstPageTitle): - mysite = pywikibot.Site() - mylang = mysite.code + """ + Generator to retrieve misspelling pages or misspelling redirects. + + @rtype: generator + """ + mylang = self.site.code if mylang in self.misspellingCategory: misspellingCategoryTitle = self.misspellingCategory[mylang] - misspellingCategory = pywikibot.Category(mysite, + misspellingCategory = pywikibot.Category(self.site, misspellingCategoryTitle) generator = pagegenerators.CategorizedPageGenerator( misspellingCategory, recurse=True, start=firstPageTitle) elif mylang in self.misspellingTemplate: - misspellingTemplateName = 'Template:%s' % self.misspellingTemplate[mylang] - misspellingTemplate = pywikibot.Page(mysite, + misspellingTemplateName = ('Template:%s' + % self.misspellingTemplate[mylang]) + misspellingTemplate = pywikibot.Page(self.site, misspellingTemplateName) generator = pagegenerators.ReferringPageGenerator( misspellingTemplate, onlyTemplateInclusion=True) @@ -92,7 +94,7 @@ u'-start parameter unsupported on this wiki because there ' u'is no category for misspellings.') else: - pywikibot.output(HELP_MSG.format(site=mysite)) + pywikibot.output(HELP_MSG.format(site=self.site)) empty_gen = (i for i in []) return empty_gen @@ -100,14 +102,22 @@ preloadingGen = pagegenerators.PreloadingGenerator(generator) return preloadingGen - # Overrides the DisambiguationRobot method. def findAlternatives(self, disambPage): + """ + Append link target to a list of alternative links. + + Overrides the DisambiguationRobot method. + + @return: True if alternate link was appended + @rtype: bool or None + """ if disambPage.isRedirectPage(): self.alternatives.append(disambPage.getRedirectTarget().title()) return True - elif self.misspellingTemplate[disambPage.site.lang] is not None: + if self.misspellingTemplate.get(disambPage.site.code) is not None: for template, params in disambPage.templatesWithParams(): - if template.title() in self.misspellingTemplate[self.mylang]: + if (template.title(withNamespace=False) == + self.misspellingTemplate[disambPage.site.code]): # The correct spelling is in the last paramter. correctSpelling = params[-1] # On de.wikipedia, there are some cases where the @@ -122,12 +132,15 @@ self.alternatives.append(correctSpelling) return True - # Overrides the DisambiguationRobot method. - def setSummaryMessage(self, disambPage, new_targets=[], unlink=False, - dn=False): + def setSummaryMessage(self, disambPage, *args, **kwargs): + """ + Setup the summary message. + + Overrides the DisambiguationRobot method. + """ # TODO: setSummaryMessage() in solve_disambiguation now has parameters # new_targets and unlink. Make use of these here. - self.comment = i18n.twtranslate(self.mysite, 'misspelling-fixing', + self.comment = i18n.twtranslate(self.site, 'misspelling-fixing', {'page': disambPage.title()}) diff --git a/scripts/solve_disambiguation.py b/scripts/solve_disambiguation.py index 0e108c5..2c2dd26 100755 --- a/scripts/solve_disambiguation.py +++ b/scripts/solve_disambiguation.py @@ -567,7 +567,7 @@ self.main_only = main_only self.minimum = minimum - self.mysite = pywikibot.Site() + self.site = self.mysite = pywikibot.Site() self.mylang = self.mysite.lang self.comment = None diff --git a/tox.ini b/tox.ini index ebfd765..5a75cec 100644 --- a/tox.ini +++ b/tox.ini @@ -89,6 +89,7 @@ scripts/listpages.py \ scripts/login.py \ scripts/maintenance/__init__.py \ + scripts/misspelling.py \ scripts/newitem.py \ scripts/noreferences.py \ scripts/pagefromfile.py \ -- To view, visit https://gerrit.wikimedia.org/r/223270 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I907bb4047d42afa4f9044a870f18cbaa724302c8 Gerrit-PatchSet: 8 Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Owner: Xqt <info(a)gno.de> Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com> Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com> Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl> Gerrit-Reviewer: XZise <CommodoreFabianus(a)gmx.de> Gerrit-Reviewer: Xqt <info(a)gno.de> Gerrit-Reviewer: jenkins-bot <>

1 0

[Gerrit] [bugfix] L10N tests for site.lang key - change (pywikibot/core)
by jenkins-bot (Code Review) 30 Jul '15

30 Jul '15

jenkins-bot has submitted this change and it was merged. Change subject: [bugfix] L10N tests for site.lang key ...................................................................... [bugfix] L10N tests for site.lang key Keys of i18n json files has been changed from site.code to site.lang. After updating the L10N test is broken due to some sites has different lang and code property. - Now check templates for all sites in languages_by_size list. - skip known problems - create method name with code - Check whether a given language has a corresponding site - Give more precise message for assert error The original commit in the master branch fdef82ff introduced a problem with testing that it needed to query many Wikipedias to get their language for the translations. The follow-up patch 5e83b6e0 is incorporated into this patch. Co-Author: Fabian Neundorf <CommodoreFabianus(a)gmx.de> Change-Id: I43131490147846d3486a6d457dc055ecb7593077 --- M tests/l10n_tests.py 1 file changed, 45 insertions(+), 21 deletions(-) Approvals: John Vandenberg: Looks good to me, approved jenkins-bot: Verified diff --git a/tests/l10n_tests.py b/tests/l10n_tests.py index 46981a7..294914a 100644 --- a/tests/l10n_tests.py +++ b/tests/l10n_tests.py @@ -29,39 +29,43 @@ def __new__(cls, name, bases, dct): """Create the new class.""" - # this comment is to avoid senseless flake8 warning - - def test_method(msg, code): + def test_method(site): def test_template(self): """Test validity of template.""" + lang = site.lang + if lang not in keys: + return + msg = i18n.twtranslate(lang, package, fallback=False) if msg: # check whether the message contains a template template = re.findall(u'.*?{{(.*?)[|}]', msg) self.assertTrue(template) - if template: - # check whether site is valid - site = pywikibot.Site('en', 'wikipedia') - self.assertTrue(code in site.languages()) - - # check whether template exists - title = template[0] - site = pywikibot.Site(code, 'wikipedia') - page = pywikibot.Page(site, title, ns=10) - self.assertTrue(page.exists()) + # known problem + if site.code == 'simple': + raise unittest.SkipTest( + "'simple' wiki has 'en' language code but " + "missing template. Must be solved by the " + "corresponding script.") + # check whether template exists + title = template[0] + page = pywikibot.Page(site, title, ns=10) + self.assertTrue(page.exists()) return test_template # create test methods for package messages processed by unittest + site = pywikibot.Site(dct['code'], dct['family']) + codes = site.family.languages_by_size + del site for package in PACKAGES: - for lang in i18n.twget_keys(package): - template_msg = i18n.twtranslate(lang, package, fallback=False) - if template_msg is None: - continue - test_name = "test_%s_%s" % (package.replace('-', '_'), lang) - dct[test_name] = test_method(template_msg, lang) - return type.__new__(cls, name, bases, dct) + keys = i18n.twget_keys(package) + for code in codes: + current_site = pywikibot.Site(code, dct['family']) + test_name = ("test_%s_%s" % (package, code)).replace('-', '_') + dct[test_name] = test_method(current_site) + return super(TestValidTemplateMeta, cls).__new__(cls, name, bases, dct) @add_metaclass @@ -71,7 +75,27 @@ __metaclass__ = TestValidTemplateMeta - net = True # magic flag tells jenkins to not run the test. + family = 'wikipedia' + code = 'en' + + +class TestSites(TestCase): + + """Other test L10N cases processed by unittest.""" + + family = 'wikipedia' + code = 'en' + + def test_valid_sites(self): + """Test whether language key has a corresponding site.""" + codes = self.site.family.languages_by_size + languages = [pywikibot.Site(code, self.family).lang for code in codes] + for package in PACKAGES: + keys = i18n.twget_keys(package) + for key in keys: + self.assertIn(key, languages, + "'%s' - json key '%s' is not a site language" + % (package, key)) if __name__ == '__main__': -- To view, visit https://gerrit.wikimedia.org/r/223881 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I43131490147846d3486a6d457dc055ecb7593077 Gerrit-PatchSet: 10 Gerrit-Project: pywikibot/core Gerrit-Branch: 2.0 Gerrit-Owner: Merlijn van Deen <valhallasw(a)arctus.nl> Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com> Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com> Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl> Gerrit-Reviewer: XZise <CommodoreFabianus(a)gmx.de> Gerrit-Reviewer: Xqt <info(a)gno.de> Gerrit-Reviewer: jenkins-bot <>

1 0

Passed: wikimedia/pywikibot-core#2594 (master - de2c958)
by Travis CI 29 Jul '15

29 Jul '15

Build Update for wikimedia/pywikibot-core ------------------------------------- Build: #2594 Status: Passed Duration: 27 minutes and 14 seconds Commit: de2c958 (master) Author: Fabian Neundorf Message: [FIX] aspects: Restore copyright year The copyright year has been changed from 2014 to 2015 in b45d93c4 although the file already existed in 2014. This restores the starting point and makes it an interval. Change-Id: I036d14dd2a941886c9fe85ec1e6c6b61031c5b1e View the changeset: https://github.com/wikimedia/pywikibot-core/compare/a210afdce870...de2c9581… View the full build log and details: https://travis-ci.org/wikimedia/pywikibot-core/builds/73173418 -- You can configure recipients for build notifications in your .travis.yml file. See http://docs.travis-ci.com/user/notifications

1 0

[Gerrit] [FIX] aspects: Restore copyright year - change (pywikibot/core)
by jenkins-bot (Code Review) 29 Jul '15

29 Jul '15

jenkins-bot has submitted this change and it was merged. Change subject: [FIX] aspects: Restore copyright year ...................................................................... [FIX] aspects: Restore copyright year The copyright year has been changed from 2014 to 2015 in b45d93c4 although the file already existed in 2014. This restores the starting point and makes it an interval. Change-Id: I036d14dd2a941886c9fe85ec1e6c6b61031c5b1e --- M tests/aspects.py 1 file changed, 1 insertion(+), 1 deletion(-) Approvals: Xqt: Looks good to me, approved jenkins-bot: Verified diff --git a/tests/aspects.py b/tests/aspects.py index 3e3667a..a0209c8 100644 --- a/tests/aspects.py +++ b/tests/aspects.py @@ -7,7 +7,7 @@ mixin to show cache usage is included. """ # -# (C) Pywikibot team, 2015 +# (C) Pywikibot team, 2014-2015 # # Distributed under the terms of the MIT license. # -- To view, visit https://gerrit.wikimedia.org/r/227680 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I036d14dd2a941886c9fe85ec1e6c6b61031c5b1e Gerrit-PatchSet: 1 Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Owner: XZise <CommodoreFabianus(a)gmx.de> Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com> Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com> Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl> Gerrit-Reviewer: Xqt <info(a)gno.de> Gerrit-Reviewer: jenkins-bot <>

1 0

Passed: wikimedia/pywikibot-core#2593 (master - a210afd)
by Travis CI 29 Jul '15

29 Jul '15

Build Update for wikimedia/pywikibot-core ------------------------------------- Build: #2593 Status: Passed Duration: 26 minutes and 12 seconds Commit: a210afd (master) Author: jenkins-bot Message: Merge "Deprecation warning context handler" View the changeset: https://github.com/wikimedia/pywikibot-core/compare/63d27ac33850...a210afdc… View the full build log and details: https://travis-ci.org/wikimedia/pywikibot-core/builds/73166662 -- You can configure recipients for build notifications in your .travis.yml file. See http://docs.travis-ci.com/user/notifications

1 0

[Gerrit] Deprecation warning context handler - change (pywikibot/core)
by jenkins-bot (Code Review) 29 Jul '15

29 Jul '15

jenkins-bot has submitted this change and it was merged. Change subject: Deprecation warning context handler ...................................................................... Deprecation warning context handler Correctly handle filenames of deprecation warnings occurring during assertRaises and assertRaisesRegex. Improve error when deprecation filename does not match expected filename. Bug: T102365 Change-Id: I23b55f48b0c47c92ee8f5002e81a37f5e8698b57 --- M tests/aspects.py M tests/dry_site_tests.py M tests/utils.py 3 files changed, 120 insertions(+), 18 deletions(-) Approvals: XZise: Looks good to me, approved jenkins-bot: Verified diff --git a/tests/aspects.py b/tests/aspects.py index 8af9ade..a3f33bf 100644 --- a/tests/aspects.py +++ b/tests/aspects.py @@ -7,7 +7,7 @@ mixin to show cache usage is included. """ # -# (C) Pywikibot team, 2014 +# (C) Pywikibot team, 2015 # # Distributed under the terms of the MIT license. # @@ -53,7 +53,10 @@ import tests from tests import unittest, patch_request, unpatch_request -from tests.utils import execute_pwb, DrySite, DryRequest, add_metaclass +from tests.utils import ( + add_metaclass, execute_pwb, DrySite, DryRequest, + WarningSourceSkipContextManager, +) class TestCaseBase(unittest.TestCase): @@ -1303,6 +1306,17 @@ _generic_match = re.compile(r'.* is deprecated(, use .* instead)?\.') + skip_list = [ + unittest.case._AssertRaisesContext, + TestCase.assertRaises, + TestCase.assertRaisesRegex, + TestCase.assertRaisesRegexp, + ] + + # Python 3 component in the call stack of _AssertRaisesContext + if hasattr(unittest.case, '_AssertRaisesBaseContext'): + skip_list.append(unittest.case._AssertRaisesBaseContext) + def __init__(self, *args, **kwargs): super(DeprecationTestCase, self).__init__(*args, **kwargs) self.warning_log = [] @@ -1312,8 +1326,9 @@ self.expect_warning_filename = self.expect_warning_filename[:-1] self._do_test_warning_filename = True + self._ignore_unknown_warning_packages = False - self.context_manager = warnings.catch_warnings(record=True) + self.context_manager = WarningSourceSkipContextManager(self.skip_list) def _reset_messages(self): self._do_test_warning_filename = True @@ -1352,7 +1367,14 @@ def assertDeprecationFile(self, filename): for item in self.warning_log: - self.assertEqual(item.filename, filename) + if (self._ignore_unknown_warning_packages and + 'pywikibot' not in item.filename): + continue + + if item.filename != filename: + self.fail( + 'expected warning filename %s; warning item: %s' + % (filename, item)) def setUp(self): super(DeprecationTestCase, self).setUp() @@ -1375,20 +1397,16 @@ For example C{assertEqual} will do first C{assertEqual} and then C{assertOneDeprecation}. - - With C{check_file} it's possible to enable or disable the check whether the - filename matches the caller's filename. By default it does not match the - filename if the name starts with 'assertRaises' (as it'll call it inside the - assert in that case). """ def process_assert(self, assertion, *args, **kwargs): """Handle assertion and call C{assertOneDeprecation} after it.""" super(AutoDeprecationTestCase, self).process_assert( assertion, *args, **kwargs) - self._do_test_warning_filename = self.check_file(assertion.__name__) self.assertOneDeprecation() - def check_file(self, name): - """Disable filename check on asserted exceptions.""" - return not name.startswith('assertRaises') + skip_list = DeprecationTestCase.skip_list + [ + CapturingTestCase.process_assert, + CapturingTestCase.patch_assert, + process_assert, + ] diff --git a/tests/dry_site_tests.py b/tests/dry_site_tests.py index 4e28476..b97356d 100644 --- a/tests/dry_site_tests.py +++ b/tests/dry_site_tests.py @@ -217,12 +217,9 @@ def test_need_version_fail_with_deprecated(self): """Test order of combined version check and deprecation warning.""" - # As assertRaisesRegex calls the method, unittest is the module - # invoking the method instead of this test module. - self._do_test_warning_filename = False - # FIXME: The deprecation message should be: # __name__ + '.TestNeedVersion.deprecated_unavailable_method + # The outermost decorator is the version check, so no deprecation message. self.assertRaisesRegex( NotImplementedError, diff --git a/tests/utils.py b/tests/utils.py index 27152da..ecf02c2 100644 --- a/tests/utils.py +++ b/tests/utils.py @@ -1,13 +1,14 @@ # -*- coding: utf-8 -*- """Test utilities.""" # -# (C) Pywikibot team, 2013-2014 +# (C) Pywikibot team, 2013-2015 # # Distributed under the terms of the MIT license. # from __future__ import print_function, unicode_literals __version__ = '$Id$' # +import inspect import json import os import re @@ -15,6 +16,7 @@ import sys import time import traceback +import warnings from collections import Mapping from warnings import warn @@ -116,6 +118,91 @@ return gen +class WarningSourceSkipContextManager(warnings.catch_warnings): + + """ + Warning context manager that adjusts source of warning. + + The source of the warning will be moved further down the + stack to skip a list of objects that have been monkey + patched into the call stack. + """ + + def __init__(self, skip_list): + """ + Constructor. + + @param skip_list: List of objects to be skipped + @type skip_list: list of object or (obj, str, int, int) + """ + super(WarningSourceSkipContextManager, self).__init__(record=True) + self.skip_list = skip_list + + @property + def skip_list(self): + """ + Return list of filename and line ranges to skip. + + @rtype: list of (obj, str, int, int) + """ + return self._skip_list + + @skip_list.setter + def skip_list(self, value): + """ + Set list of objects to be skipped. + + @param value: List of objects to be skipped + @type value: list of object or (obj, str, int, int) + """ + self._skip_list = [] + for item in value: + if isinstance(item, tuple): + self._skip_list.append(item) + else: + filename = inspect.getsourcefile(item) + code, first_line = inspect.getsourcelines(item) + last_line = first_line + len(code) + self._skip_list.append( + (item, filename, first_line, last_line)) + + def __enter__(self): + """Enter the context manager.""" + def detailed_show_warning(*args, **kwargs): + """warnings.showwarning replacement handler.""" + entry = warnings.WarningMessage(*args, **kwargs) + + skip_lines = 0 + entry_line_found = False + + for (_, filename, fileno, _, line, _) in inspect.stack(): + if any(start <= fileno <= end + for (_, skip_filename, start, end) in self.skip_list + if skip_filename == filename): + if entry_line_found: + continue + else: + skip_lines += 1 + + if (filename, fileno) == (entry.filename, entry.lineno): + if not skip_lines: + break + entry_line_found = True + + if entry_line_found: + if not skip_lines: + (entry.filename, entry.lineno) = (filename, fileno) + break + else: + skip_lines -= 1 + + log.append(entry) + + log = super(WarningSourceSkipContextManager, self).__enter__() + self._module.showwarning = detailed_show_warning + return log + + class DryParamInfo(dict): """Dummy class to use instead of L{pywikibot.data.api.ParamInfo}.""" -- To view, visit https://gerrit.wikimedia.org/r/225677 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I23b55f48b0c47c92ee8f5002e81a37f5e8698b57 Gerrit-PatchSet: 5 Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Owner: John Vandenberg <jayvdb(a)gmail.com> Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com> Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com> Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl> Gerrit-Reviewer: XZise <CommodoreFabianus(a)gmx.de> Gerrit-Reviewer: jenkins-bot <>

1 0

Passed: wikimedia/pywikibot-core#2592 (master - 63d27ac)
by Travis CI 29 Jul '15

29 Jul '15

Build Update for wikimedia/pywikibot-core ------------------------------------- Build: #2592 Status: Passed Duration: 30 minutes and 15 seconds Commit: 63d27ac (master) Author: jenkins-bot Message: Merge "Use sitename property" View the changeset: https://github.com/wikimedia/pywikibot-core/compare/7fc07aafbe3d...63d27ac3… View the full build log and details: https://travis-ci.org/wikimedia/pywikibot-core/builds/73162180 -- You can configure recipients for build notifications in your .travis.yml file. See http://docs.travis-ci.com/user/notifications

1 0

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

Pywikibot-commits July 2015