Build Update for wikimedia/pywikibot-core
-------------------------------------
Build: #2345
Status: Passed
Duration: 27 minutes and 48 seconds
Commit: 1fa8ca6 (master)
Author: Fabian Neundorf
Message: [FIX] open_compressed: Wrap BZ2File in Py 2.7
Although the documentation specifies that BZ2File is compatible with the
with-statement since Python 2.7 and the code for both gzip and BZ2File were
changed in the same commit in cpython, only since Python 2.7.1 this is
supported.
It also separates the tests so it's easier to see if only one specific
compressor has a problem. It also tests that the wrapper does not block any
exception.
The docstring is also improved to reflect the use_extension parameter.
Bug: T101649
Change-Id: I3f7024746164cc1bb1d89f2ac52dfa41ca08a3a6
View the changeset: https://github.com/wikimedia/pywikibot-core/compare/a1d54f47df8f...1fa8ca6f…
View the full build log and details: https://travis-ci.org/wikimedia/pywikibot-core/builds/65826708
--
You can configure recipients for build notifications in your .travis.yml file. See http://docs.travis-ci.com/user/notifications
jenkins-bot has submitted this change and it was merged.
Change subject: Fix Appveyor builds
......................................................................
Fix Appveyor builds
An extra line from .appveyor.yml needs to be removed
after the switch to python requests (952665a).
Change-Id: I583443f779118683a87b6e3d9216c64da92eab93
---
M .appveyor.yml
1 file changed, 0 insertions(+), 1 deletion(-)
Approvals:
John Vandenberg: Looks good to me, approved
jenkins-bot: Verified
diff --git a/.appveyor.yml b/.appveyor.yml
index bd16e2e..9286aec 100644
--- a/.appveyor.yml
+++ b/.appveyor.yml
@@ -28,7 +28,6 @@
- ps: (new-object net.webclient).DownloadFile('https://raw.githubusercontent.com/pypa/python-packaging-user-guide/master/s…', 'install.ps1')
- "powershell install.ps1"
- ps: (new-object net.webclient).DownloadFile('https://raw.githubusercontent.com/pypa/python-packaging-user-guide/master/s…', 'run_with_compiler.cmd')
- - cd ..\\..
build: off
--
To view, visit https://gerrit.wikimedia.org/r/215861
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I583443f779118683a87b6e3d9216c64da92eab93
Gerrit-PatchSet: 2
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com>
Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl>
Gerrit-Reviewer: jenkins-bot <>
jenkins-bot has submitted this change and it was merged.
Change subject: [FIX] APISite.upload: Support exists-normalized
......................................................................
[FIX] APISite.upload: Support exists-normalized
The upload warning type 'exists-normalized' has been added in MW 1.22 but
wasn't supported in pywikibot yet.
Change-Id: Ic3d4517e1e864e4f7a420d826618fc479ed3361c
---
M pywikibot/site.py
1 file changed, 2 insertions(+), 0 deletions(-)
Approvals:
John Vandenberg: Looks good to me, approved
jenkins-bot: Verified
diff --git a/pywikibot/site.py b/pywikibot/site.py
index 463552c..d1d730f 100644
--- a/pywikibot/site.py
+++ b/pywikibot/site.py
@@ -4980,6 +4980,8 @@
'duplicate': "Uploaded file is a duplicate of %(msg)s.",
'badfilename': "Target filename is invalid.",
'filetype-unwanted-type': "File %(msg)s type is unwanted type.",
+ 'exists-normalized': 'File exists with different extension as '
+ '"%(msg)s".',
}
# check for required user right
--
To view, visit https://gerrit.wikimedia.org/r/216545
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: Ic3d4517e1e864e4f7a420d826618fc479ed3361c
Gerrit-PatchSet: 2
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: XZise <CommodoreFabianus(a)gmx.de>
Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: jenkins-bot <>
jenkins-bot has submitted this change and it was merged.
Change subject: [FIX] open_compressed: Wrap BZ2File in Py 2.7
......................................................................
[FIX] open_compressed: Wrap BZ2File in Py 2.7
Although the documentation specifies that BZ2File is compatible with the
with-statement since Python 2.7 and the code for both gzip and BZ2File were
changed in the same commit in cpython, only since Python 2.7.1 this is
supported.
It also separates the tests so it's easier to see if only one specific
compressor has a problem. It also tests that the wrapper does not block any
exception.
The docstring is also improved to reflect the use_extension parameter.
Bug: T101649
Change-Id: I3f7024746164cc1bb1d89f2ac52dfa41ca08a3a6
---
M pywikibot/tools/__init__.py
M tests/tools_tests.py
2 files changed, 48 insertions(+), 28 deletions(-)
Approvals:
John Vandenberg: Looks good to me, approved
jenkins-bot: Verified
diff --git a/pywikibot/tools/__init__.py b/pywikibot/tools/__init__.py
index 9636c46..8d9c4ff 100644
--- a/pywikibot/tools/__init__.py
+++ b/pywikibot/tools/__init__.py
@@ -750,22 +750,25 @@
they are always available. 7zip is only available when a 7za program is
available.
- The compression is selected via the file ending.
+ The compression is either selected via the magic number or file ending.
@param filename: The filename.
@type filename: str
+ @param use_extension: Use the file extension instead of the magic number
+ to determine the type of compression (default False).
+ @type use_extension: bool
@raises ValueError: When 7za is not available.
@raises OSError: When it's not a 7z archive but the file extension is 7z.
It is also raised by bz2 when its content is invalid. gzip does not
immediately raise that error but only on reading it.
- @return: A file like object returning the uncompressed data in binary mode.
- Before Python 2.7 it's wrapping the object returned by BZ2File and gzip
- in a ContextManagerWrapper so it's advantages/disadvantages apply there.
- @rtype: file like object
+ @return: A file-like object returning the uncompressed data in binary mode.
+ Before Python 2.7 the GzipFile object and before 2.7.1 the BZ2File are
+ wrapped in a ContextManagerWrapper with its advantages/disadvantages.
+ @rtype: file-like object
"""
- def wrap(wrapped):
+ def wrap(wrapped, sub_ver):
"""Wrap in a wrapper when this is below Python version 2.7."""
- if sys.version_info < (2, 7):
+ if sys.version_info < (2, 7, sub_ver):
return ContextManagerWrapper(wrapped)
else:
return wrapped
@@ -787,9 +790,9 @@
extension = ''
if extension == 'bz2':
- return wrap(bz2.BZ2File(filename))
+ return wrap(bz2.BZ2File(filename), 1)
elif extension == 'gz':
- return wrap(gzip.open(filename))
+ return wrap(gzip.open(filename), 0)
elif extension == '7z':
try:
process = subprocess.Popen(['7za', 'e', '-bd', '-so', filename],
diff --git a/tests/tools_tests.py b/tests/tools_tests.py
index ceb42cb..718f589 100644
--- a/tests/tools_tests.py
+++ b/tests/tools_tests.py
@@ -24,26 +24,26 @@
"""Test that ContextManagerWrapper is working correctly."""
+ class DummyClass(object):
+
+ """A dummy class which has some values and a close method."""
+
+ class_var = 42
+
+ def __init__(self):
+ """Create instance with dummy values."""
+ self.instance_var = 1337
+ self.closed = False
+
+ def close(self):
+ """Just store that it has been closed."""
+ self.closed = True
+
net = False
def test_wrapper(self):
"""Create a test instance and verify the wrapper redirects."""
- class DummyClass(object):
-
- """A dummy class which has some values and a close method."""
-
- class_var = 42
-
- def __init__(self):
- """Create instance with dummy values."""
- self.instance_var = 1337
- self.closed = False
-
- def close(self):
- """Just store that it has been closed."""
- self.closed = True
-
- obj = DummyClass()
+ obj = self.DummyClass()
wrapped = tools.ContextManagerWrapper(obj)
self.assertIs(wrapped.class_var, obj.class_var)
self.assertIs(wrapped.instance_var, obj.instance_var)
@@ -52,7 +52,18 @@
with wrapped as unwrapped:
self.assertFalse(obj.closed)
self.assertIs(unwrapped, obj)
+ unwrapped.class_var = 47
self.assertTrue(obj.closed)
+ self.assertEqual(wrapped.class_var, 47)
+
+ def test_exec_wrapper(self):
+ """Check that the wrapper permits exceptions."""
+ wrapper = tools.ContextManagerWrapper(self.DummyClass())
+ self.assertFalse(wrapper.closed)
+ with self.assertRaises(ZeroDivisionError):
+ with wrapper:
+ 1 / 0
+ self.assertTrue(wrapper.closed)
class OpenCompressedTestCase(TestCase):
@@ -82,13 +93,19 @@
with tools.open_compressed(*args) as f:
return f.read()
- def test_open_compressed(self):
- """Test open_compressed with all compressors in the standard library."""
+ def test_open_compressed_normal(self):
+ """Test open_compressed with no compression in the standard library."""
self.assertEqual(self._get_content(self.base_file), self.original_content)
+
+ def test_open_compressed_bz2(self):
+ """Test open_compressed with bz2 compressor in the standard library."""
self.assertEqual(self._get_content(self.base_file + '.bz2'), self.original_content)
- self.assertEqual(self._get_content(self.base_file + '.gz'), self.original_content)
self.assertEqual(self._get_content(self.base_file + '.bz2', True), self.original_content)
+ def test_open_compressed_gz(self):
+ """Test open_compressed with gz compressor in the standard library."""
+ self.assertEqual(self._get_content(self.base_file + '.gz'), self.original_content)
+
def test_open_compressed_7z(self):
"""Test open_compressed with 7za if installed."""
try:
--
To view, visit https://gerrit.wikimedia.org/r/216544
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I3f7024746164cc1bb1d89f2ac52dfa41ca08a3a6
Gerrit-PatchSet: 2
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: XZise <CommodoreFabianus(a)gmx.de>
Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: Ricordisamoa <ricordisamoa(a)openmailbox.org>
Gerrit-Reviewer: XZise <CommodoreFabianus(a)gmx.de>
Gerrit-Reviewer: jenkins-bot <>
jenkins-bot has submitted this change and it was merged.
Change subject: Support Debian python-ipaddr package
......................................................................
Support Debian python-ipaddr package
Prefer ipaddr as the default IP support on Python 2.
Retain support for ipaddress, but recommend installing ipaddr.
Bug: T100603
Change-Id: I531f739ab94a7ba424142041a5e74bf066e6e07d
---
M pywikibot/tools/ip.py
M setup.py
M tests/ipregex_tests.py
3 files changed, 49 insertions(+), 23 deletions(-)
Approvals:
John Vandenberg: Looks good to me, but someone else must approve
Merlijn van Deen: Looks good to me, approved
jenkins-bot: Verified
diff --git a/pywikibot/tools/ip.py b/pywikibot/tools/ip.py
index 808c0cc..93983e7 100644
--- a/pywikibot/tools/ip.py
+++ b/pywikibot/tools/ip.py
@@ -18,8 +18,22 @@
from pywikibot.tools import LazyRegex
+ipaddress_e = ipaddr_e = None
+
try:
from ipaddress import ip_address
+except ImportError as ipaddress_e:
+ ip_address = None
+ pass
+
+if not ip_address or sys.version_info[0] < 3:
+ try:
+ from ipaddr import IPAddress as ip_address
+ ip_address.__T76286__ = False
+ except ImportError as ipaddr_e:
+ pass
+
+if ip_address and ip_address.__module__ == 'ipaddress':
if sys.version_info[0] < 3:
# This backport fails many tests
# https://pypi.python.org/pypi/py2-ipaddress
@@ -27,7 +41,7 @@
try:
ip_address(u'1111')
ip_address = None
- raise ImportError('ipaddress backport is broken')
+ raise ImportError('ipaddress backport is broken; install ipaddr')
except ValueError:
pass
@@ -36,28 +50,32 @@
# However while it rejects u'1111', it will consider '1111' valid
try:
ip_address(b'1111')
- warn('ipaddress backport is defective; patching.', ImportWarning)
+ warn('ipaddress backport is defective; patching; install ipaddr',
+ ImportWarning)
orig_ip_address = ip_address
# force all input to be a unicode object so it validates correctly
- def ip_address(IP):
+ def ip_address_patched(IP):
"""Safe ip_address."""
return orig_ip_address(unicode(IP)) # noqa
+
+ ip_address = ip_address_patched
except ValueError:
# This means ipaddress has correctly determined '1111' is invalid
pass
-except ImportError as e:
- warn('Importing ipaddress.ip_address failed: %s' % e,
- ImportWarning)
+elif not ip_address:
+ warn('Importing ipaddr.IPAddress failed: %s\n'
+ 'Importing ipaddress.ip_address failed: %s\n'
+ 'Please install ipaddr.'
+ % (ipaddr_e, ipaddress_e), ImportWarning)
- def ip_address(IP):
+ def ip_address_fake(IP):
"""Fake ip_address method."""
warn('ipaddress backport not available.', DeprecationWarning)
if ip_regexp.match(IP) is None:
raise ValueError('Invalid IP address')
- # The following flag is used by the unit tests
- ip_address.__fake__ = True
+ ip_address = ip_address_fake
# deprecated IP detector
ip_regexp = LazyRegex()
diff --git a/setup.py b/setup.py
index db13b1e..c74ba98 100644
--- a/setup.py
+++ b/setup.py
@@ -78,12 +78,14 @@
script_deps['replicate_wiki.py'] = ['argparse']
dependencies.append('future') # provides collections backports
- # tools.ip does not depend on an ipaddress module, as it falls back to
- # using regexes if not available, however the pywikibot package should use
- # the functional backport of py3 ipaddress, which is:
+ # tools.ip does not have a hard dependency on an IP address module,
+ # as it falls back to using regexes if one is not available.
+ # The functional backport of py3 ipaddress is acceptable:
# https://pypi.python.org/pypi/ipaddress
+ # However the Debian package python-ipaddr is also supported:
+ # https://pypi.python.org/pypi/ipaddr
# Other backports are likely broken.
- dependencies.append('ipaddress')
+ dependencies.append('ipaddr')
# mwlib is not available for py3
script_deps['patrol'] = ['mwlib']
diff --git a/tests/ipregex_tests.py b/tests/ipregex_tests.py
index 7553684..6e4459e 100644
--- a/tests/ipregex_tests.py
+++ b/tests/ipregex_tests.py
@@ -2,7 +2,7 @@
"""Test IP regex."""
# -*- coding: utf-8 -*-
#
-# (C) Pywikibot team, 2014
+# (C) Pywikibot team, 2015
#
# Distributed under the terms of the MIT license.
from __future__ import unicode_literals
@@ -12,6 +12,7 @@
from pywikibot.tools import ip
from tests.aspects import unittest, TestCase
+from tests.utils import expected_failure_if
class TestIPBase(TestCase):
@@ -620,6 +621,12 @@
self.ipv6test(True, "a:b:c:d:e:f:0::")
self.ipv6test(False, "':10.0.0.1")
+ def _test_T76286_failures(self):
+ """Test known bugs in the ipaddress module."""
+ # The following fail with the ipaddress module. See T76286
+ self.ipv6test(False, "1111:2222:3333:4444:5555:6666:00.00.00.00")
+ self.ipv6test(False, "1111:2222:3333:4444:5555:6666:000.000.000.000")
+
class IPRegexTestCase(TestIPBase):
@@ -631,9 +638,7 @@
def test_regex(self):
"""Test IP regex."""
self._run_tests()
- # The following only work with the IP regex. See below.
- self.ipv6test(False, "1111:2222:3333:4444:5555:6666:00.00.00.00")
- self.ipv6test(False, "1111:2222:3333:4444:5555:6666:000.000.000.000")
+ self._test_T76286_failures()
self.assertEqual(self.fail, 0)
@@ -647,21 +652,22 @@
@classmethod
def setUpClass(cls):
"""Check ipaddress module is available."""
- if hasattr(ip.ip_address, '__fake__'):
+ if ip.ip_address.__name__ == 'ip_address_fake':
raise unittest.SkipTest('module ipaddress not available')
+
super(IPAddressModuleTestCase, cls).setUpClass()
def test_ipaddress_module(self):
"""Test ipaddress module."""
+ print('testing %s' % ip.ip_address.__module__)
self._run_tests()
self.assertEqual(self.fail, 0)
- @unittest.expectedFailure
- def test_ipaddress_module_failures(self):
+ @expected_failure_if(ip.ip_address.__module__ == 'ipaddress' or
+ ip.ip_address.__name__ == 'ip_address_patched')
+ def test_T76286_failures(self):
"""Test known bugs in the ipaddress module."""
- # The following fail with the ipaddress module. See T76286
- self.ipv6test(False, "1111:2222:3333:4444:5555:6666:00.00.00.00")
- self.ipv6test(False, "1111:2222:3333:4444:5555:6666:000.000.000.000")
+ self._test_T76286_failures()
self.assertEqual(self.fail, 0)
if __name__ == "__main__":
--
To view, visit https://gerrit.wikimedia.org/r/214294
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I531f739ab94a7ba424142041a5e74bf066e6e07d
Gerrit-PatchSet: 8
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com>
Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl>
Gerrit-Reviewer: XZise <CommodoreFabianus(a)gmx.de>
Gerrit-Reviewer: jenkins-bot <>