Pywikibot-commits February 2017

pywikibot-commits@lists.wikimedia.org

2 participants
113 discussions

Errored: wikimedia/pywikibot-core#3870 (master - 2f6dc30)
by Travis CI 26 Feb '17

26 Feb '17

Build Update for wikimedia/pywikibot-core ------------------------------------- Build: #3870 Status: Errored Duration: 1 hour, 40 minutes, and 32 seconds Commit: 2f6dc30 (master) Author: jenkins-bot Message: Merge "[Bug]Recognize zero as a valid error bound" View the changeset: https://github.com/wikimedia/pywikibot-core/compare/52d02fbc0e7b...2f6dc302… View the full build log and details: https://travis-ci.org/wikimedia/pywikibot-core/builds/205541767 -- You can configure recipients for build notifications in your .travis.yml file. See https://docs.travis-ci.com/user/notifications

1 0

[Gerrit] pywikibot/core[master]: [Bug]Recognize zero as a valid error bound
by jenkins-bot (Code Review) 26 Feb '17

26 Feb '17

jenkins-bot has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/339939 ) Change subject: [Bug]Recognize zero as a valid error bound ...................................................................... [Bug]Recognize zero as a valid error bound Providing zero for either of the bound would trigger the _require_errors test. Changing from "if bound" to "if bound is not None" type tests allows zero to be recognized a a valid bound. Change-Id: Icf05b0e3fa0d3767ef8577c0a9abc42602f8b4c1 --- M pywikibot/__init__.py 1 file changed, 2 insertions(+), 1 deletion(-) Approvals: Dalba: Looks good to me, approved jenkins-bot: Verified diff --git a/pywikibot/__init__.py b/pywikibot/__init__.py index b980330..f9782ce 100644 --- a/pywikibot/__init__.py +++ b/pywikibot/__init__.py @@ -731,8 +731,9 @@ amount = cls._todecimal(wb['amount']) upperBound = cls._todecimal(wb.get('upperBound')) lowerBound = cls._todecimal(wb.get('lowerBound')) + bounds_provided = (upperBound is not None and lowerBound is not None) error = None - if (upperBound and lowerBound) or cls._require_errors(site): + if bounds_provided or cls._require_errors(site): error = (upperBound - amount, amount - lowerBound) if wb['unit'] == '1': unit = None -- To view, visit https://gerrit.wikimedia.org/r/339939 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: Icf05b0e3fa0d3767ef8577c0a9abc42602f8b4c1 Gerrit-PatchSet: 1 Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Owner: Lokal Profil <lokal.profil(a)gmail.com> Gerrit-Reviewer: Dalba <dalba.wiki(a)gmail.com> Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com> Gerrit-Reviewer: Magul <tomasz.magulski(a)gmail.com> Gerrit-Reviewer: Mpaa <mpaa.wiki(a)gmail.com> Gerrit-Reviewer: Xqt <info(a)gno.de> Gerrit-Reviewer: jenkins-bot <>

1 0

[Gerrit] pywikibot/i18n[master]: Add Old English (ang) localisation redirect.py to i18n
by jenkins-bot (Code Review) 24 Feb '17

24 Feb '17

jenkins-bot has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/339601 ) Change subject: Add Old English (ang) localisation redirect.py to i18n ...................................................................... Add Old English (ang) localisation redirect.py to i18n This hasn't been ported by translater updater bot since months Bug: T158843 Change-Id: I19de890f49e4d6efd98093bdbc452a2dd7a1313d --- A redirect/ang.json 1 file changed, 8 insertions(+), 0 deletions(-) Approvals: jenkins-bot: Verified Siebrand: Looks good to me, approved diff --git a/redirect/ang.json b/redirect/ang.json new file mode 100644 index 0000000..28b1542 --- /dev/null +++ b/redirect/ang.json @@ -0,0 +1,8 @@ +{ + "@metadata": { + "authors": [ + "MarcoAurelio" + ] + }, + "redirect-broken-redirect-template": "{{Ahwit|1=Bot: Þā folgendan edlǣdunga bendaþ tō unedwistlicum trametum.}}" +} -- To view, visit https://gerrit.wikimedia.org/r/339601 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I19de890f49e4d6efd98093bdbc452a2dd7a1313d Gerrit-PatchSet: 3 Gerrit-Project: pywikibot/i18n Gerrit-Branch: master Gerrit-Owner: Xqt <info(a)gno.de> Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com> Gerrit-Reviewer: MarcoAurelio <strigiwm(a)gmail.com> Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl> Gerrit-Reviewer: Siebrand <siebrand(a)kitano.nl> Gerrit-Reviewer: jenkins-bot <>

1 0

Errored: wikimedia/pywikibot-core#3869 (master - 52d02fb)
by Travis CI 24 Feb '17

24 Feb '17

Build Update for wikimedia/pywikibot-core ------------------------------------- Build: #3869 Status: Errored Duration: 47 minutes and 14 seconds Commit: 52d02fb (master) Author: jenkins-bot Message: Merge "page.py: implement FilePage.download()" View the changeset: https://github.com/wikimedia/pywikibot-core/compare/1f4096e4b3a4...52d02fbc… View the full build log and details: https://travis-ci.org/wikimedia/pywikibot-core/builds/204884468 -- You can configure recipients for build notifications in your .travis.yml file. See https://docs.travis-ci.com/user/notifications

1 0

Errored: wikimedia/pywikibot-core#3868 (master - 1f4096e)
by Travis CI 24 Feb '17

24 Feb '17

Build Update for wikimedia/pywikibot-core ------------------------------------- Build: #3868 Status: Errored Duration: 18 minutes and 9 seconds Commit: 1f4096e (master) Author: jenkins-bot Message: Merge "tools: make general function to compute file sha" View the changeset: https://github.com/wikimedia/pywikibot-core/compare/d54db1d40d1c...1f4096e4… View the full build log and details: https://travis-ci.org/wikimedia/pywikibot-core/builds/204884378 -- You can configure recipients for build notifications in your .travis.yml file. See https://docs.travis-ci.com/user/notifications

1 0

[Gerrit] pywikibot/core[master]: page.py: implement FilePage.download()
by jenkins-bot (Code Review) 24 Feb '17

24 Feb '17

jenkins-bot has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/339003 ) Change subject: page.py: implement FilePage.download() ...................................................................... page.py: implement FilePage.download() Add FilePage.download() to save image file to local file. Added tests in file_tests.py Change-Id: I82e9e9fca2e346ddf1ca3af82e60b5234fc49af0 --- M pywikibot/page.py M tests/file_tests.py 2 files changed, 63 insertions(+), 0 deletions(-) Approvals: jenkins-bot: Verified Xqt: Looks good to me, approved diff --git a/pywikibot/page.py b/pywikibot/page.py index a268b1e..6f7c7be 100644 --- a/pywikibot/page.py +++ b/pywikibot/page.py @@ -23,6 +23,7 @@ import hashlib import logging +import os.path import re import sys @@ -67,6 +68,7 @@ from pywikibot.family import Family from pywikibot.site import Namespace, need_version from pywikibot.tools import ( + compute_file_hash, PYTHON_VERSION, MediaWikiVersion, UnicodeMixin, ComparableMixin, DotReadableDict, deprecated, deprecate_arg, deprecated_args, issue_deprecation_warning, @@ -2528,6 +2530,38 @@ return self.site.upload(self, source_filename=filename, source_url=url, **kwargs) + def download(self, filename=None, chunk_size=100 * 1024): + """ + Download to filename file of FilePage. + + @param filename: filename where to save file: + None: self.title(as_filename=True, withNamespace=False) + will be used + str: provided filename will be used. + @type None or str + @return: True if download is successful, False otherwise. + @raise: IOError if filename cannot be written for any reason. + """ + if filename is None: + filename = self.title(as_filename=True, withNamespace=False) + + filename = os.path.expanduser(filename) + + req = http.fetch(self.latest_file_info.url, stream=True) + if req.status == 200: + try: + with open(filename, 'wb') as f: + for chunk in req.data.iter_content(chunk_size): + f.write(chunk) + except IOError as e: + raise e + + sha1 = compute_file_hash(filename) + return sha1 == self.latest_file_info.sha1 + else: + pywikibot.warning('Unsuccesfull request (%s): %s' % (req.status, req.uri)) + return False + wrapper = _ModuleDeprecationWrapper(__name__) wrapper._add_deprecated_attr('ImagePage', FilePage) diff --git a/tests/file_tests.py b/tests/file_tests.py index 91fb22b..6ab6b59 100644 --- a/tests/file_tests.py +++ b/tests/file_tests.py @@ -9,9 +9,13 @@ __version__ = '$Id$' +import os + import pywikibot from pywikibot.tools import UnicodeType as unicode + +from tests import join_images_path from tests.aspects import unittest, TestCase, DeprecationTestCase @@ -180,6 +184,31 @@ self.assertIsInstance(latest[1], unicode) +class TestFilePageDownload(TestCase): + + """Test dowload fo FilePage to local file.""" + + family = 'commons' + code = 'commons' + + cached = True + + def test_successful_download(self): + """Test successful_download.""" + page = pywikibot.FilePage(self.site, 'File:Albert Einstein.jpg') + filename = join_images_path('Albert Einstein.jpg') + status_code = page.download(filename) + self.assertTrue(status_code) + os.unlink(filename) + + def test_not_existing_download(self): + """Test not existing download.""" + page = pywikibot.FilePage(self.site, 'File:Albert Einstein.jpg_notexisting') + filename = join_images_path('Albert Einstein.jpg') + with self.assertRaises(pywikibot.NoPage): + page.download(filename) + + if __name__ == '__main__': # pragma: no cover try: unittest.main() -- To view, visit https://gerrit.wikimedia.org/r/339003 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I82e9e9fca2e346ddf1ca3af82e60b5234fc49af0 Gerrit-PatchSet: 5 Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Owner: Mpaa <mpaa.wiki(a)gmail.com> Gerrit-Reviewer: Magul <tomasz.magulski(a)gmail.com> Gerrit-Reviewer: Xqt <info(a)gno.de> Gerrit-Reviewer: jenkins-bot <>

1 0

[Gerrit] pywikibot/core[master]: tools: make general function to compute file sha
by jenkins-bot (Code Review) 24 Feb '17

24 Feb '17

jenkins-bot has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/339317 ) Change subject: tools: make general function to compute file sha ...................................................................... tools: make general function to compute file sha It can be reused in several places: - site.upload() - Filepage.download() [if/when it will be merged] Change-Id: I756c4d127274f7f6031920127850f30de3964597 --- M pywikibot/site.py M pywikibot/tools/__init__.py M tests/tools_tests.py 3 files changed, 89 insertions(+), 10 deletions(-) Approvals: jenkins-bot: Verified Xqt: Looks good to me, approved diff --git a/pywikibot/site.py b/pywikibot/site.py index bdf1fb4..52e6927 100644 --- a/pywikibot/site.py +++ b/pywikibot/site.py @@ -18,7 +18,6 @@ import copy import datetime import functools -import hashlib import heapq import itertools import json @@ -68,6 +67,7 @@ from pywikibot.family import WikimediaFamily from pywikibot.throttle import Throttle from pywikibot.tools import ( + compute_file_hash, itergroup, UnicodeMixin, ComparableMixin, SelfCallMixin, SelfCallString, deprecated, deprecate_arg, deprecated_args, remove_last_args, redirect_func, issue_deprecation_warning, @@ -6027,15 +6027,7 @@ # The SHA1 was also requested so calculate and compare it assert 'sha1' in stash_info, \ 'sha1 not in stash info: {0}'.format(stash_info) - sha1 = hashlib.sha1() - bytes_to_read = offset - with open(source_filename, 'rb') as f: - while bytes_to_read > 0: - read_bytes = f.read(min(bytes_to_read, 1 << 20)) - assert read_bytes # make sure we actually read bytes - bytes_to_read -= len(read_bytes) - sha1.update(read_bytes) - sha1 = sha1.hexdigest() + sha1 = compute_file_hash(source_filename, bytes_to_read=offset) if sha1 != stash_info['sha1']: raise ValueError( 'The SHA1 of {0} bytes of the stashed "{1}" is {2} ' diff --git a/pywikibot/tools/__init__.py b/pywikibot/tools/__init__.py index 526fd93..261f5ac 100644 --- a/pywikibot/tools/__init__.py +++ b/pywikibot/tools/__init__.py @@ -10,6 +10,7 @@ import collections import gzip +import hashlib import inspect import itertools import os @@ -1714,3 +1715,41 @@ # re-read and check changes if os.stat(filename).st_mode != st_mode: warn(warn_str.format(filename, st_mode - stat.S_IFREG, mode)) + + +def compute_file_hash(filename, sha='sha1', bytes_to_read=None): + """Compute file hash. + + Result is expressed as hexdigest(). + + @param filename: filename path + @type filename: basestring + + @param func: hashing function among the following in hashlib: + md5(), sha1(), sha224(), sha256(), sha384(), and sha512() + function name shall be passed as string, e.g. 'sha1'. + @type filename: basestring + + @param bytes_to_read: only the first bytes_to_read will be considered; + if file size is smaller, the whole file will be considered. + @type bytes_to_read: None or int + + """ + size = os.path.getsize(filename) + if bytes_to_read is None: + bytes_to_read = size + else: + bytes_to_read = min(bytes_to_read, size) + step = 1 << 20 + + shas = ['md5', 'sha1', 'sha224', 'sha256', 'sha384', 'sha512'] + assert sha in shas + sha = getattr(hashlib, sha)() # sha instance + + with open(filename, 'rb') as f: + while bytes_to_read > 0: + read_bytes = f.read(min(bytes_to_read, step)) + assert read_bytes # make sure we actually read bytes + bytes_to_read -= len(read_bytes) + sha.update(read_bytes) + return sha.hexdigest() diff --git a/tests/tools_tests.py b/tests/tools_tests.py index 6e97772..c450966 100644 --- a/tests/tools_tests.py +++ b/tests/tools_tests.py @@ -754,6 +754,54 @@ self.chmod.assert_called_once_with(self.file, 0o600) +class TestFileShaCalculator(TestCase): + + """Test calculator of sha of a file.""" + + net = False + + filename = join_xml_data_path('article-pear-0.10.xml') + + def setUp(self): + """Setup tests.""" + super(TestFileShaCalculator, self).setUp() + + def test_md5_complete_calculation(self): + """"Test md5 of complete file.""" + res = tools.compute_file_hash(self.filename, sha='md5') + self.assertEqual(res, '5d7265e290e6733e1e2020630262a6f3') + + def test_md5_partial_calculation(self): + """"Test md5 of partial file (1024 bytes).""" + res = tools.compute_file_hash(self.filename, sha='md5', + bytes_to_read=1024) + self.assertEqual(res, 'edf6e1accead082b6b831a0a600704bc') + + def test_sha1_complete_calculation(self): + """"Test sha1 of complete file.""" + res = tools.compute_file_hash(self.filename, sha='sha1') + self.assertEqual(res, '1c12696e1119493a625aa818a35c41916ce32d0c') + + def test_sha1_partial_calculation(self): + """"Test sha1 of partial file (1024 bytes).""" + res = tools.compute_file_hash(self.filename, sha='sha1', + bytes_to_read=1024) + self.assertEqual(res, 'e56fa7bd5cfdf6bb7e2d8649dd9216c03e7271e6') + + def test_sha224_complete_calculation(self): + """"Test sha224 of complete file.""" + res = tools.compute_file_hash(self.filename, sha='sha224') + self.assertEqual( + res, '3d350d9d9eca074bd299cb5ffe1b325a9f589b2bcd7ba1c033ab4d33') + + def test_sha224_partial_calculation(self): + """"Test sha224 of partial file (1024 bytes).""" + res = tools.compute_file_hash(self.filename, sha='sha224', + bytes_to_read=1024) + self.assertEqual( + res, 'affa8cb79656a9b6244a079f8af91c9271e382aa9d5aa412b599e169') + + class Foo(object): """Test class to verify classproperty decorator.""" -- To view, visit https://gerrit.wikimedia.org/r/339317 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I756c4d127274f7f6031920127850f30de3964597 Gerrit-PatchSet: 1 Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Owner: Mpaa <mpaa.wiki(a)gmail.com> Gerrit-Reviewer: Magul <tomasz.magulski(a)gmail.com> Gerrit-Reviewer: Xqt <info(a)gno.de> Gerrit-Reviewer: jenkins-bot <>

1 0

Errored: wikimedia/pywikibot-core#3867 (master - d54db1d)
by Travis CI 23 Feb '17

23 Feb '17

Build Update for wikimedia/pywikibot-core ------------------------------------- Build: #3867 Status: Errored Duration: 48 minutes and 38 seconds Commit: d54db1d (master) Author: jenkins-bot Message: Merge "Add missing not-equal comparison for wbtypes" View the changeset: https://github.com/wikimedia/pywikibot-core/compare/864195e8e3b5...d54db1d4… View the full build log and details: https://travis-ci.org/wikimedia/pywikibot-core/builds/204624239 -- You can configure recipients for build notifications in your .travis.yml file. See https://docs.travis-ci.com/user/notifications

1 0

Errored: wikimedia/pywikibot-core#3866 (master - 864195e)
by Travis CI 23 Feb '17

23 Feb '17

Build Update for wikimedia/pywikibot-core ------------------------------------- Build: #3866 Status: Errored Duration: 1 hour, 55 minutes, and 58 seconds Commit: 864195e (master) Author: jenkins-bot Message: Merge "Support adding units to WbQuantity through ItemPage or entity url" View the changeset: https://github.com/wikimedia/pywikibot-core/compare/c005b0e52001...864195e8… View the full build log and details: https://travis-ci.org/wikimedia/pywikibot-core/builds/204595993 -- You can configure recipients for build notifications in your .travis.yml file. See https://docs.travis-ci.com/user/notifications

1 0

[Gerrit] pywikibot/core[master]: Add missing not-equal comparison for wbtypes
by jenkins-bot (Code Review) 23 Feb '17

23 Feb '17

jenkins-bot has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/339399 ) Change subject: Add missing not-equal comparison for wbtypes ...................................................................... Add missing not-equal comparison for wbtypes Bug: T158848 Change-Id: Ib6e992b7ed1c5b4b8feac205758bdbaebda2b09c --- M pywikibot/_wbtypes.py 1 file changed, 3 insertions(+), 0 deletions(-) Approvals: Dalba: Looks good to me, approved jenkins-bot: Verified diff --git a/pywikibot/_wbtypes.py b/pywikibot/_wbtypes.py index b0db7bf..dd3f5a5 100644 --- a/pywikibot/_wbtypes.py +++ b/pywikibot/_wbtypes.py @@ -47,3 +47,6 @@ def __eq__(self, other): return self.__dict__ == other.__dict__ + + def __ne__(self, other): + return not self.__eq__(other) -- To view, visit https://gerrit.wikimedia.org/r/339399 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: Ib6e992b7ed1c5b4b8feac205758bdbaebda2b09c Gerrit-PatchSet: 1 Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Owner: Lokal Profil <lokal.profil(a)gmail.com> Gerrit-Reviewer: Dalba <dalba.wiki(a)gmail.com> Gerrit-Reviewer: Lokal Profil <lokal.profil(a)gmail.com> Gerrit-Reviewer: Magul <tomasz.magulski(a)gmail.com> Gerrit-Reviewer: jenkins-bot <>

1 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

Pywikibot-commits February 2017