Pywikibot-commits January 2016

pywikibot-commits@lists.wikimedia.org

2 participants
115 discussions

Errored: wikimedia/pywikibot-core#3196 (master - e71dcdb)
by Travis CI 21 Jan '16

21 Jan '16

Build Update for wikimedia/pywikibot-core ------------------------------------- Build: #3196 Status: Errored Duration: 1 hour, 16 minutes, and 48 seconds Commit: e71dcdb (master) Author: jenkins-bot Message: Merge "Remove unneeded catching of httplib ValueError" View the changeset: https://github.com/wikimedia/pywikibot-core/compare/59ce9585ec73...e71dcdbc… View the full build log and details: https://travis-ci.org/wikimedia/pywikibot-core/builds/103874212 -- You can configure recipients for build notifications in your .travis.yml file. See https://docs.travis-ci.com/user/notifications

1 0

[Gerrit] color logging fixes - change (pywikibot/core)
by jenkins-bot (Code Review) 21 Jan '16

21 Jan '16

jenkins-bot has submitted this change and it was merged. Change subject: color logging fixes ...................................................................... color logging fixes afe2555d7 implemented color_format, which uses str.format {} syntax, but two messages in reflinks were not converted from %s syntax to {}. Bug: T123887 Change-Id: I8e6916153428689dc4c6410bebfc9c771d8813d8 --- M scripts/reflinks.py 1 file changed, 3 insertions(+), 3 deletions(-) Approvals: Xqt: Looks good to me, approved jenkins-bot: Verified diff --git a/scripts/reflinks.py b/scripts/reflinks.py index 0474deb..814e1ce 100755 --- a/scripts/reflinks.py +++ b/scripts/reflinks.py @@ -38,7 +38,7 @@ one from i18n/reflinks.py """ # (C) Nicolas Dumazet (NicDumZ), 2008 -# (C) Pywikibot team, 2008-2015 +# (C) Pywikibot team, 2008-2016 # # Distributed under the terms of the MIT license. # @@ -587,7 +587,7 @@ # example : http://www.adminet.com/jo/20010615¦/ECOC0100037D.html # in [[fr:Cyanure]] pywikibot.output(color_format( - '{lightred}Bad link{default} : %s in %s', + '{lightred}Bad link{default} : {0} in {1}', ref.url, page.title(asLink=True))) continue except (URLError, @@ -638,7 +638,7 @@ continue elif not self.MIME.search(contentType): pywikibot.output(color_format( - '{lightyellow}WARNING{default} : media : %s ', + '{lightyellow}WARNING{default} : media : {0} ', ref.link)) repl = ref.refLink() new_text = new_text.replace(match.group(), repl) -- To view, visit https://gerrit.wikimedia.org/r/265462 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I8e6916153428689dc4c6410bebfc9c771d8813d8 Gerrit-PatchSet: 1 Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Owner: John Vandenberg <jayvdb(a)gmail.com> Gerrit-Reviewer: MtDu <justin.d128(a)gmail.com> Gerrit-Reviewer: Xqt <info(a)gno.de> Gerrit-Reviewer: jenkins-bot <>

1 0

[Gerrit] Remove old unneeded URL exclusion rules - change (pywikibot/core)
by jenkins-bot (Code Review) 21 Jan '16

21 Jan '16

jenkins-bot has submitted this change and it was merged. Change subject: Remove old unneeded URL exclusion rules ...................................................................... Remove old unneeded URL exclusion rules No problems are experienced now accessing these URLs using requests. Bug: T124015 Change-Id: I1f33969058fc67f8437f2f92325faba636a9c81c --- M scripts/weblinkchecker.py 1 file changed, 1 insertion(+), 12 deletions(-) Approvals: MtDu: Looks good to me, but someone else must approve Xqt: Looks good to me, approved jenkins-bot: Verified diff --git a/scripts/weblinkchecker.py b/scripts/weblinkchecker.py index 51bce05..ed22c3c 100755 --- a/scripts/weblinkchecker.py +++ b/scripts/weblinkchecker.py @@ -133,7 +133,6 @@ import requests -# TODO: Convert to httlib2 if sys.version_info[0] > 2: import http.client as httplib import urllib.parse as urlparse @@ -164,20 +163,10 @@ re.compile(r'.*[\./(a)]example\.org(/.*)?'), # Other special cases - # bot somehow can't handle their redirects: - re.compile(r'.*[\./(a)]gso\.gbv\.de(/.*)?'), re.compile(r'.*[\./(a)]berlinonline\.de(/.*)?'), # above entry to be manually fixed per request at [[de:Benutzer:BLueFiSH.as/BZ]] # bot can't handle their redirects: - re.compile(r'.*[\./(a)]bodo\.kommune\.no(/.*)?'), - re.compile(r'.*[\./(a)]jpl\.nasa\.gov(/.*)?'), # bot rejected on the site - re.compile(r'.*[\./(a)]itis\.gov(/.*)?'), # bot rejected on the site - re.compile(r'.*[\./(a)]cev\.lu(/.*)?'), # bot rejected on the site - # very slow response resulting in bot error: - re.compile(r'.*[\./(a)]science\.ksc\.nasa\.gov(/.*)?'), - re.compile(r'.*[\./(a)]britannica\.com(/.*)?'), # HTTP redirect loop - # bot rejected on the site: - re.compile(r'.*[\./(a)]quickfacts\.census\.gov(/.*)?'), + # bot rejected on the site, already archived re.compile(r'.*[\./(a)]web\.archive\.org(/.*)?'), ] -- To view, visit https://gerrit.wikimedia.org/r/265442 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I1f33969058fc67f8437f2f92325faba636a9c81c Gerrit-PatchSet: 2 Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Owner: John Vandenberg <jayvdb(a)gmail.com> Gerrit-Reviewer: MtDu <justin.d128(a)gmail.com> Gerrit-Reviewer: Xqt <info(a)gno.de> Gerrit-Reviewer: jenkins-bot <>

1 0

Errored: wikimedia/pywikibot-core#3195 (master - 59ce958)
by Travis CI 21 Jan '16

21 Jan '16

Build Update for wikimedia/pywikibot-core ------------------------------------- Build: #3195 Status: Errored Duration: 1 hour, 16 minutes, and 51 seconds Commit: 59ce958 (master) Author: Justin Du Message: Convert weblinkchecker to requests Bug: T113140 Change-Id: Ib9ec7c6294c73f88f562003bdcb30402e0d1fd55 View the changeset: https://github.com/wikimedia/pywikibot-core/compare/f96151900aa1...59ce9585… View the full build log and details: https://travis-ci.org/wikimedia/pywikibot-core/builds/103864863 -- You can configure recipients for build notifications in your .travis.yml file. See https://docs.travis-ci.com/user/notifications

1 0

[Gerrit] Remove unneeded catching of httplib ValueError - change (pywikibot/core)
by jenkins-bot (Code Review) 21 Jan '16

21 Jan '16

jenkins-bot has submitted this change and it was merged. Change subject: Remove unneeded catching of httplib ValueError ...................................................................... Remove unneeded catching of httplib ValueError Bug: T124010 Change-Id: I5f1b5bca386610c4f974358792d45ad01756c20d --- M scripts/reflinks.py 1 file changed, 1 insertion(+), 4 deletions(-) Approvals: John Vandenberg: Looks good to me, approved MtDu: Looks good to me, but someone else must approve jenkins-bot: Verified diff --git a/scripts/reflinks.py b/scripts/reflinks.py index 0474deb..03ac5ca 100755 --- a/scripts/reflinks.py +++ b/scripts/reflinks.py @@ -597,10 +597,7 @@ pywikibot.output(u'Can\'t retrieve page %s : %s' % (ref.url, e)) continue - except ValueError: - # Known bug of httplib, google for : - # "httplib raises ValueError reading chunked content" - continue + # remove <script>/<style>/comments/CDATA tags linkedpagetext = self.NON_HTML.sub(b'', linkedpagetext) -- To view, visit https://gerrit.wikimedia.org/r/264931 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I5f1b5bca386610c4f974358792d45ad01756c20d Gerrit-PatchSet: 1 Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Owner: John Vandenberg <jayvdb(a)gmail.com> Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com> Gerrit-Reviewer: MtDu <justin.d128(a)gmail.com> Gerrit-Reviewer: jenkins-bot <>

1 0

[Gerrit] Convert weblinkchecker to requests - change (pywikibot/core)
by jenkins-bot (Code Review) 21 Jan '16

21 Jan '16

jenkins-bot has submitted this change and it was merged. Change subject: Convert weblinkchecker to requests ...................................................................... Convert weblinkchecker to requests Bug: T113140 Change-Id: Ib9ec7c6294c73f88f562003bdcb30402e0d1fd55 --- M scripts/weblinkchecker.py 1 file changed, 26 insertions(+), 6 deletions(-) Approvals: John Vandenberg: Looks good to me, approved jenkins-bot: Verified diff --git a/scripts/weblinkchecker.py b/scripts/weblinkchecker.py index d1f0834..51bce05 100755 --- a/scripts/weblinkchecker.py +++ b/scripts/weblinkchecker.py @@ -128,7 +128,10 @@ from pywikibot.pagegenerators import ( XMLDumpPageGenerator as _XMLDumpPageGenerator, ) +from pywikibot.tools import deprecated from pywikibot.tools.formatter import color_format + +import requests # TODO: Convert to httlib2 if sys.version_info[0] > 2: @@ -268,6 +271,7 @@ """The link is not an URL.""" +@deprecated('requests') class LinkChecker(object): """ @@ -537,7 +541,17 @@ threading.Thread.__init__(self) self.page = page self.url = url + self._user_agent = comms.http.get_fake_user_agent() self.history = history + self.header = { + 'User-agent': self._user_agent, + 'Accept': 'text/xml,application/xml,application/xhtml+xml,' + 'text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5', + 'Accept-Language': 'de-de,de;q=0.8,en-us;q=0.5,en;q=0.3', + 'Accept-Charset': 'ISO-8859-1,utf-8;q=0.7,*;q=0.7', + 'Keep-Alive': '30', + 'Connection': 'keep-alive', + } # identification for debugging purposes self.setName((u'%s - %s' % (page.title(), url)).encode('utf-8', 'replace')) @@ -545,19 +559,24 @@ self.day = day def run(self): - linkChecker = LinkChecker(self.url, HTTPignore=self.HTTPignore) + ok = False try: - ok, message = linkChecker.check() - except NotAnURLError: - ok = False + header = self.header + timeout = pywikibot.config.socket_timeout + r = requests.get(self.url, headers=header, timeout=timeout) + except requests.exceptions.InvalidURL: message = i18n.twtranslate(self.page.site, 'weblinkchecker-badurl_msg', {'URL': self.url}) - except: pywikibot.output('Exception while processing URL %s in page %s' % (self.url, self.page.title())) raise + if (r.status_code == requests.codes.ok and + str(r.status_code) not in self.HTTPignore): + ok = True + else: + message = '{0} {1}'.format(r.status_code, r.reason) if ok: if self.history.setLinkAlive(self.url): pywikibot.output('*Link to %s in [[%s]] is back alive.' @@ -871,8 +890,9 @@ return i +@deprecated('requests') def check(url): - """Peform a check on URL.""" + """DEPRECATED: Use requests instead. Perform a check on URL.""" c = LinkChecker(url) return c.check() -- To view, visit https://gerrit.wikimedia.org/r/265398 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: Ib9ec7c6294c73f88f562003bdcb30402e0d1fd55 Gerrit-PatchSet: 17 Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Owner: MtDu <justin.d128(a)gmail.com> Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com> Gerrit-Reviewer: MtDu <justin.d128(a)gmail.com> Gerrit-Reviewer: jenkins-bot <>

1 0

Errored: wikimedia/pywikibot-core#3194 (master - f961519)
by Travis CI 20 Jan '16

20 Jan '16

Build Update for wikimedia/pywikibot-core ------------------------------------- Build: #3194 Status: Errored Duration: 1 hour, 12 minutes, and 6 seconds Commit: f961519 (master) Author: Yusuke Matsubara Message: archivebot: conformance to translate() and additional shorthands Use dict as the argument for translate(). Additional shorthand notations for "years" and "weeks". Change-Id: I1ef93cf16f0345ab858532e7a9c2063c76c554c4 View the changeset: https://github.com/wikimedia/pywikibot-core/compare/d486ac657075...f9615190… View the full build log and details: https://travis-ci.org/wikimedia/pywikibot-core/builds/103564828 -- You can configure recipients for build notifications in your .travis.yml file. See https://docs.travis-ci.com/user/notifications

1 0

[Gerrit] archivebot: conformance to translate() and additional shorth... - change (pywikibot/core)
by jenkins-bot (Code Review) 20 Jan '16

20 Jan '16

jenkins-bot has submitted this change and it was merged. Change subject: archivebot: conformance to translate() and additional shorthands ...................................................................... archivebot: conformance to translate() and additional shorthands Use dict as the argument for translate(). Additional shorthand notations for "years" and "weeks". Change-Id: I1ef93cf16f0345ab858532e7a9c2063c76c554c4 --- M scripts/archivebot.py 1 file changed, 17 insertions(+), 7 deletions(-) Approvals: John Vandenberg: Looks good to me, approved jenkins-bot: Verified diff --git a/scripts/archivebot.py b/scripts/archivebot.py index d87e72c..5c925b6 100755 --- a/scripts/archivebot.py +++ b/scripts/archivebot.py @@ -155,12 +155,16 @@ Translates a duration written in the shorthand notation (ex. "24h", "7d") into an expression in the local language of the wiki ("24 hours", "7 days"). """ - if string[-1] == 'd': - template = site.mediawiki_message('Days') - elif string[-1] == 'h': + if string[-1] == 'h': template = site.mediawiki_message('Hours') + elif string[-1] == 'd': + template = site.mediawiki_message('Days') + elif string[-1] == 'w': + template = site.mediawiki_message('Weeks') + elif string[-1] == 'y': + template = site.mediawiki_message('Years') if template: - exp = i18n.translate(site.code, template, int(string[:-1])) + exp = i18n.translate(site.code, template, {'$1': int(string[:-1])}) return to_local_digits(exp.replace('$1', string[:-1]), site.code) else: return to_local_digits(string, site.code) @@ -173,12 +177,18 @@ Accepts a string defining a time period: 7d - 7 days 36h - 36 hours + 2w - 2 weeks (14 days) + 1y - 366 days # to be on the safe side Returns the corresponding timedelta object. """ - if string.endswith('d'): - return datetime.timedelta(days=int(string[:-1])) - elif string.endswith('h'): + if string.endswith('h'): return datetime.timedelta(hours=int(string[:-1])) + elif string.endswith('d'): + return datetime.timedelta(days=int(string[:-1])) + elif string.endswith('w'): + return datetime.timedelta(weeks=int(string[:-1])) + elif string.endswith('y'): + return datetime.timedelta(days=int(string[:-1]) * 366) else: return datetime.timedelta(seconds=int(string)) -- To view, visit https://gerrit.wikimedia.org/r/265222 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I1ef93cf16f0345ab858532e7a9c2063c76c554c4 Gerrit-PatchSet: 3 Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Owner: Whym <whym(a)whym.org> Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com> Gerrit-Reviewer: Whym <whym(a)whym.org> Gerrit-Reviewer: jenkins-bot <>

1 0

Errored: wikimedia/pywikibot-core#3193 (master - d486ac6)
by Travis CI 19 Jan '16

19 Jan '16

Build Update for wikimedia/pywikibot-core ------------------------------------- Build: #3193 Status: Errored Duration: 50 minutes and 22 seconds Commit: d486ac6 (master) Author: jenkins-bot Message: Merge "Add edit tests for WbMonolingualText" View the changeset: https://github.com/wikimedia/pywikibot-core/compare/ae390ce7e8d4...d486ac65… View the full build log and details: https://travis-ci.org/wikimedia/pywikibot-core/builds/103446455 -- You can configure recipients for build notifications in your .travis.yml file. See https://docs.travis-ci.com/user/notifications

1 0

[Gerrit] Add edit tests for WbMonolingualText - change (pywikibot/core)
by jenkins-bot (Code Review) 19 Jan '16

19 Jan '16

jenkins-bot has submitted this change and it was merged. Change subject: Add edit tests for WbMonolingualText ...................................................................... Add edit tests for WbMonolingualText Also adds the monolingualtext type to page.py which is required in order to actually add such claims. Bug: T123856 Change-Id: I88be5687f7df914bceb4497206947324fc593947 --- M pywikibot/page.py M tests/edit_failure_tests.py M tests/wikibase_edit_tests.py M tests/wikibase_tests.py 4 files changed, 57 insertions(+), 2 deletions(-) Approvals: John Vandenberg: Looks good to me, approved jenkins-bot: Verified diff --git a/pywikibot/page.py b/pywikibot/page.py index 48e9c36..82a9028 100644 --- a/pywikibot/page.py +++ b/pywikibot/page.py @@ -4078,6 +4078,7 @@ 'url': basestring, 'time': pywikibot.WbTime, 'quantity': pywikibot.WbQuantity, + 'monolingualtext': pywikibot.WbMonolingualText, } value_types = {'wikibase-item': 'wikibase-entityid', diff --git a/tests/edit_failure_tests.py b/tests/edit_failure_tests.py index 17ed20e..395a516 100644 --- a/tests/edit_failure_tests.py +++ b/tests/edit_failure_tests.py @@ -129,6 +129,28 @@ item = pywikibot.ItemPage(repo, 'Q6') self.assertRaises(pywikibot.PageNotSaved, item.save) + def _make_WbMonolingualText_claim(self, repo, text, language): + """Make a WbMonolingualText and set its value.""" + claim = pywikibot.page.Claim(repo, 'P271', datatype='monolingualtext') + target = pywikibot.WbMonolingualText(text=text, language=language) + claim.setTarget(target) + return claim + + def test_WbMonolingualText_invalid_language(self): + """Attempt adding a monolingual text with an invalid language.""" + repo = self.get_repo() + item = pywikibot.ItemPage(repo, 'Q68') + claim = self._make_WbMonolingualText_claim(repo, text='Test this!', + language='foo') + self.assertAPIError('modification-failed', None, item.addClaim, claim) + + def test_WbMonolingualText_invalid_text(self): + """Attempt adding a monolingual text with an invalid non-string text.""" + repo = self.get_repo() + item = pywikibot.ItemPage(repo, 'Q68') + claim = self._make_WbMonolingualText_claim(repo, text=123456, language='en') + self.assertAPIError('invalid-snak', None, item.addClaim, claim) + if __name__ == '__main__': try: diff --git a/tests/wikibase_edit_tests.py b/tests/wikibase_edit_tests.py index fe9eccc..a350cf7 100644 --- a/tests/wikibase_edit_tests.py +++ b/tests/wikibase_edit_tests.py @@ -1,5 +1,10 @@ # -*- coding: utf-8 -*- -"""Tests for editing Wikibase items.""" +""" +Tests for editing Wikibase items. + +Tests which should fail should instead be in the TestWikibaseSaveTest +class in edit_failiure_tests.py +""" # # (C) Pywikibot team, 2014 # @@ -172,6 +177,33 @@ self.assertEqual(new_item.getRedirectTarget(), target_item) +class TestWbMonolingualText(WikibaseTestCase): + + """Run wikibase write tests for WbMonolingualText.""" + + family = 'wikidata' + code = 'test' + + user = True + write = True + + def test_WbMonolingualText_edit(self): + """Attempt adding a monolingual text with valid input.""" + # Clean the slate in preparation for test.""" + testsite = self.get_repo() + item = pywikibot.ItemPage(testsite, 'Q68') + item.get() + if 'P271' in item.claims: + item.removeClaims(item.claims['P271']) + item.get(force=True) + + # set new claim + claim = pywikibot.page.Claim(testsite, 'P271', datatype='monolingualtext') + target = pywikibot.WbMonolingualText(text='Test this!', language='en') + claim.setTarget(target) + item.addClaim(claim) + + class TestWikibaseRemoveQualifier(WikibaseTestCase): """Run wikibase write tests to remove qualifiers.""" diff --git a/tests/wikibase_tests.py b/tests/wikibase_tests.py index 53d7cee..3d844b7 100644 --- a/tests/wikibase_tests.py +++ b/tests/wikibase_tests.py @@ -244,7 +244,7 @@ q = pywikibot.WbMonolingualText.fromWikibase({'text': 'Test this!', 'language': u'en'}) self.assertEqual(q.toWikibase(), - {'text': 'Test this!', 'language': 'en', }) + {'text': 'Test this!', 'language': 'en'}) def test_WbMonolingualText_errors(self): # test WbMonolingualText error handling -- To view, visit https://gerrit.wikimedia.org/r/264935 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I88be5687f7df914bceb4497206947324fc593947 Gerrit-PatchSet: 5 Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Owner: Lokal Profil <lokal.profil(a)gmail.com> Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com> Gerrit-Reviewer: Lokal Profil <lokal.profil(a)gmail.com> Gerrit-Reviewer: jenkins-bot <>

1 0

← Newer
1
2
3
4
5
6
7
8
...
12
Older →

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

Pywikibot-commits January 2016