September 2016 - Pywikibot-commits

Errored: wikimedia/pywikibot-core#3626 (master - 936170b)
by Travis CI 14 Sep '16

14 Sep '16

Build Update for wikimedia/pywikibot-core ------------------------------------- Build: #3626 Status: Errored Duration: 31 minutes and 8 seconds Commit: 936170b (master) Author: zhuyifei1999 Message: Site.editpage: Moving locking mechanism into a try...finally block Repetition is the root of all evil. One place where self.unlock_page(page) is forgotten can cause a deadlock. Try...finally block is used everywhere else for unlocking in Site except for editpage. Bug: T145633 Change-Id: Ia5738f651fa23ab1a62d0f3ead85e0dbf4856136 View the changeset: https://github.com/wikimedia/pywikibot-core/compare/2d2d51b759ac...936170b7… View the full build log and details: https://travis-ci.org/wikimedia/pywikibot-core/builds/159867784 -- You can configure recipients for build notifications in your .travis.yml file. See https://docs.travis-ci.com/user/notifications

1 0

[Gerrit] pywikibot/core[master]: Site.editpage: Moving locking mechanism into a try...finally...
by jenkins-bot (Code Review) 14 Sep '16

14 Sep '16

jenkins-bot has submitted this change and it was merged. Change subject: Site.editpage: Moving locking mechanism into a try...finally block ...................................................................... Site.editpage: Moving locking mechanism into a try...finally block Repetition is the root of all evil. One place where self.unlock_page(page) is forgotten can cause a deadlock. Try...finally block is used everywhere else for unlocking in Site except for editpage. Bug: T145633 Change-Id: Ia5738f651fa23ab1a62d0f3ead85e0dbf4856136 --- M pywikibot/site.py 1 file changed, 77 insertions(+), 79 deletions(-) Approvals: Xqt: Looks good to me, approved jenkins-bot: Verified diff --git a/pywikibot/site.py b/pywikibot/site.py index 27f45fc..92cf5e1 100644 --- a/pywikibot/site.py +++ b/pywikibot/site.py @@ -4947,7 +4947,6 @@ token = self.tokens['edit'] if bot is None: bot = ("bot" in self.userinfo["rights"]) - self.lock_page(page) params = dict(action='edit', title=page, text=text, token=token, summary=summary, bot=bot, recreate=recreate, createonly=createonly, @@ -4973,92 +4972,91 @@ u"editpage: Invalid watch value '%(watch)s' ignored." % {'watch': watch}) req = self._simple_request(**params) - while True: - try: - result = req.submit() - pywikibot.debug(u"editpage response: %s" % result, - _logger) - except api.APIError as err: - self.unlock_page(page) - if err.code.endswith("anon") and self.logged_in(): + + self.lock_page(page) + try: + while True: + try: + result = req.submit() + pywikibot.debug(u"editpage response: %s" % result, + _logger) + except api.APIError as err: + if err.code.endswith("anon") and self.logged_in(): + pywikibot.debug( + u"editpage: received '%s' even though bot is logged in" + % err.code, + _logger) + if err.code in self._ep_errors: + if isinstance(self._ep_errors[err.code], basestring): + errdata = { + 'site': self, + 'title': page.title(withSection=False), + 'user': self.user(), + 'info': err.info + } + raise Error(self._ep_errors[err.code] % errdata) + else: + raise self._ep_errors[err.code](page) pywikibot.debug( - u"editpage: received '%s' even though bot is logged in" + u"editpage: Unexpected error code '%s' received." % err.code, _logger) - if err.code in self._ep_errors: - if isinstance(self._ep_errors[err.code], basestring): - errdata = { - 'site': self, - 'title': page.title(withSection=False), - 'user': self.user(), - 'info': err.info - } - raise Error(self._ep_errors[err.code] % errdata) - else: - raise self._ep_errors[err.code](page) - pywikibot.debug( - u"editpage: Unexpected error code '%s' received." - % err.code, - _logger) - raise - assert "edit" in result and "result" in result["edit"], result - if result["edit"]["result"] == "Success": - self.unlock_page(page) - if "nochange" in result["edit"]: - # null edit, page not changed - pywikibot.log(u"Page [[%s]] saved without any changes." - % page.title()) + raise + assert "edit" in result and "result" in result["edit"], result + if result["edit"]["result"] == "Success": + if "nochange" in result["edit"]: + # null edit, page not changed + pywikibot.log(u"Page [[%s]] saved without any changes." + % page.title()) + return True + page.latest_revision_id = result["edit"]["newrevid"] + # see https://www.mediawiki.org/wiki/API:Wikimania_2006_API_discussion#Notes + # not safe to assume that saved text is the same as sent + del page.text return True - page.latest_revision_id = result["edit"]["newrevid"] - # see https://www.mediawiki.org/wiki/API:Wikimania_2006_API_discussion#Notes - # not safe to assume that saved text is the same as sent - del page.text - return True - elif result["edit"]["result"] == "Failure": - if "captcha" in result["edit"]: - captcha = result["edit"]["captcha"] - req['captchaid'] = captcha['id'] - if captcha["type"] == "math": - # TODO: Should the input be parsed through eval in py3? - req['captchaword'] = input(captcha["question"]) - continue - elif "url" in captcha: - import webbrowser - webbrowser.open('%s://%s%s' - % (self.protocol(), - self.hostname(), - captcha["url"])) - req['captchaword'] = pywikibot.input( - "Please view CAPTCHA in your browser, " - "then type answer here:") - continue - else: - self.unlock_page(page) + elif result["edit"]["result"] == "Failure": + if "captcha" in result["edit"]: + captcha = result["edit"]["captcha"] + req['captchaid'] = captcha['id'] + if captcha["type"] == "math": + # TODO: Should the input be parsed through eval in py3? + req['captchaword'] = input(captcha["question"]) + continue + elif "url" in captcha: + import webbrowser + webbrowser.open('%s://%s%s' + % (self.protocol(), + self.hostname(), + captcha["url"])) + req['captchaword'] = pywikibot.input( + "Please view CAPTCHA in your browser, " + "then type answer here:") + continue + else: + pywikibot.error( + u"editpage: unknown CAPTCHA response %s, " + u"page not saved" + % captcha) + return False + elif 'spamblacklist' in result['edit']: + raise SpamfilterError(page, result['edit']['spamblacklist']) + elif 'code' in result['edit'] and 'info' in result['edit']: pywikibot.error( - u"editpage: unknown CAPTCHA response %s, " - u"page not saved" - % captcha) + u"editpage: %s\n%s, " + % (result['edit']['code'], result['edit']['info'])) return False - elif 'spamblacklist' in result['edit']: - raise SpamfilterError(page, result['edit']['spamblacklist']) - elif 'code' in result['edit'] and 'info' in result['edit']: - self.unlock_page(page) - pywikibot.error( - u"editpage: %s\n%s, " - % (result['edit']['code'], result['edit']['info'])) - return False + else: + pywikibot.error(u"editpage: unknown failure reason %s" + % str(result)) + return False else: - self.unlock_page(page) - pywikibot.error(u"editpage: unknown failure reason %s" - % str(result)) + pywikibot.error( + u"editpage: Unknown result code '%s' received; " + u"page not saved" % result["edit"]["result"]) + pywikibot.log(str(result)) return False - else: - self.unlock_page(page) - pywikibot.error( - u"editpage: Unknown result code '%s' received; " - u"page not saved" % result["edit"]["result"]) - pywikibot.log(str(result)) - return False + finally: + self.unlock_page(page) OnErrorExc = namedtuple('OnErrorExc', 'exception on_new_page') -- To view, visit https://gerrit.wikimedia.org/r/310531 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: Ia5738f651fa23ab1a62d0f3ead85e0dbf4856136 Gerrit-PatchSet: 2 Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Owner: Zhuyifei1999 <zhuyifei1999(a)gmail.com> Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com> Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl> Gerrit-Reviewer: Xqt <info(a)gno.de> Gerrit-Reviewer: jenkins-bot <>

1 0

Errored: wikimedia/pywikibot-core#3625 (master - 2d2d51b)
by Travis CI 14 Sep '16

14 Sep '16

Build Update for wikimedia/pywikibot-core ------------------------------------- Build: #3625 Status: Errored Duration: 36 minutes and 44 seconds Commit: 2d2d51b (master) Author: jenkins-bot Message: Merge "tools.py: add file_mode_checker()" View the changeset: https://github.com/wikimedia/pywikibot-core/compare/8200a2453361...2d2d51b7… View the full build log and details: https://travis-ci.org/wikimedia/pywikibot-core/builds/159818645 -- You can configure recipients for build notifications in your .travis.yml file. See https://docs.travis-ci.com/user/notifications

1 0

[Gerrit] pywikibot/core[master]: tools.py: add file_mode_checker()
by jenkins-bot (Code Review) 14 Sep '16

14 Sep '16

jenkins-bot has submitted this change and it was merged. Change subject: tools.py: add file_mode_checker() ...................................................................... tools.py: add file_mode_checker() Implement a single function for file permission checks. File permission check is currently done in http.py and login.py, with code repetitions. Tests added (they are skipped if 'mock' module is not available). Change-Id: I47fc750a20b024fce4fad20c4c9a726170a25da0 --- M pywikibot/comms/http.py M pywikibot/login.py M pywikibot/tools/__init__.py M tests/tools_tests.py 4 files changed, 77 insertions(+), 23 deletions(-) Approvals: Xqt: Looks good to me, approved jenkins-bot: Verified diff --git a/pywikibot/comms/http.py b/pywikibot/comms/http.py index 0f6e083..c3acdbc 100644 --- a/pywikibot/comms/http.py +++ b/pywikibot/comms/http.py @@ -23,8 +23,6 @@ __docformat__ = 'epytext' import atexit -import os -import stat import sys from distutils.version import StrictVersion @@ -56,6 +54,7 @@ from pywikibot.logging import critical, debug, error, log, warning from pywikibot.tools import ( deprecate_arg, + file_mode_checker, issue_deprecation_warning, PY2, StringTypes, @@ -77,18 +76,6 @@ config.socket_timeout = min(config.socket_timeout) -def mode_check(filename): - """Check if filename has mode 600 and, if not, set it.""" - mode_600 = stat.S_IRUSR | stat.S_IWUSR - warn_str = 'File {0} had no {1:o} mode; converted to {1:o} mode' - st_mode = os.stat(filename).st_mode - if stat.S_ISREG(st_mode) and (st_mode - stat.S_IFREG != mode_600): - os.chmod(filename, mode_600) - # re-read and check changes - if os.stat(filename).st_mode != st_mode: - pywikibot.warning(warn_str.format(filename, mode_600)) - - def mode_check_decorator(func): """Decorate load()/save() CookieJar methods.""" def wrapper(cls, **kwargs): @@ -97,7 +84,7 @@ except KeyError: filename = cls.filename res = func(cls, **kwargs) - mode_check(filename) + file_mode_checker(filename, mode=0o600) return res return wrapper diff --git a/pywikibot/login.py b/pywikibot/login.py index 4ce223b..dfc5cdf 100644 --- a/pywikibot/login.py +++ b/pywikibot/login.py @@ -3,7 +3,7 @@ """Library to log the bot in to a wiki account.""" # # (C) Rob W.W. Hooft, 2003 -# (C) Pywikibot team, 2003-2015 +# (C) Pywikibot team, 2003-2016 # # Distributed under the terms of the MIT license. # @@ -13,8 +13,9 @@ # import codecs import os -import stat import webbrowser + +from pywikibot.tools import file_mode_checker from warnings import warn @@ -236,12 +237,8 @@ if not os.path.isfile(password_file): password_file = config.password_file - # We fix password file permission first, - # lift upper permission (regular file) from st_mode - # to compare it with private_files_permission. - if os.stat(password_file).st_mode - stat.S_IFREG \ - != config.private_files_permission: - os.chmod(password_file, config.private_files_permission) + # We fix password file permission first. + file_mode_checker(password_file, mode=config.private_files_permission) password_f = codecs.open(password_file, encoding='utf-8') for line_nr, line in enumerate(password_f): diff --git a/pywikibot/tools/__init__.py b/pywikibot/tools/__init__.py index 5e4cb80..ffafbdf 100644 --- a/pywikibot/tools/__init__.py +++ b/pywikibot/tools/__init__.py @@ -12,7 +12,9 @@ import gzip import inspect import itertools +import os import re +import stat import subprocess import sys import threading @@ -168,6 +170,7 @@ class UnicodeMixin(object): + """Mixin class to add __str__ method in Python 2 or 3.""" @py2_encode_utf_8 @@ -1666,3 +1669,21 @@ def open_compressed(filename, use_extension=False): """DEPRECATED: Open a file and uncompress it if needed.""" return open_archive(filename, use_extension=use_extension) + + +def file_mode_checker(filename, mode=0o600): + """Check file mode and update it, if needed. + + @param filename: filename path + @type filename: basestring + @param mode: requested file mode + @type mode: int + + """ + warn_str = 'File {0} had {1:o} mode; converted to {2:o} mode.' + st_mode = os.stat(filename).st_mode + if stat.S_ISREG(st_mode) and (st_mode - stat.S_IFREG != mode): + os.chmod(filename, mode) + # re-read and check changes + if os.stat(filename).st_mode != st_mode: + warn(warn_str.format(filename, st_mode - stat.S_IFREG, mode)) diff --git a/tests/tools_tests.py b/tests/tools_tests.py index b38048b..c6525e9 100644 --- a/tests/tools_tests.py +++ b/tests/tools_tests.py @@ -17,12 +17,19 @@ import tempfile import warnings +try: + import mock +except ImportError as e: + mock = e + from pywikibot import tools from tests import join_xml_data_path + from tests.aspects import ( unittest, require_modules, DeprecationTestCase, TestCase, MetaTestCaseClass ) + from tests.utils import expected_failure_if, add_metaclass @@ -675,6 +682,48 @@ return inspect.getargspec(method) +@require_modules('mock') +class TestFileModeChecker(TestCase): + + """Test parsing password files.""" + + net = False + + def patch(self, name): + """Patch up <name> in self.setUp.""" + patcher = mock.patch(name) + self.addCleanup(patcher.stop) + return patcher.start() + + def setUp(self): + """Patch a variety of dependencies.""" + super(TestFileModeChecker, self).setUp() + self.stat = self.patch('os.stat') + self.chmod = self.patch('os.chmod') + self.file = '~FakeFile' + + def test_auto_chmod_for_dir(self): + """Do not chmod files that have mode private_files_permission.""" + self.stat.return_value.st_mode = 0o040600 # dir + tools.file_mode_checker(self.file, mode=0o600) + self.stat.assert_called_with(self.file) + self.assertFalse(self.chmod.called) + + def test_auto_chmod_OK(self): + """Do not chmod files that have mode private_files_permission.""" + self.stat.return_value.st_mode = 0o100600 # regular file + tools.file_mode_checker(self.file, mode=0o600) + self.stat.assert_called_with(self.file) + self.assertFalse(self.chmod.called) + + def test_auto_chmod_not_OK(self): + """Chmod files that do not have mode private_files_permission.""" + self.stat.return_value.st_mode = 0o100644 # regular file + tools.file_mode_checker(self.file, mode=0o600) + self.stat.assert_called_with(self.file) + self.chmod.assert_called_once_with(self.file, 0o600) + + if __name__ == '__main__': # pragma: no cover try: unittest.main() -- To view, visit https://gerrit.wikimedia.org/r/308354 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I47fc750a20b024fce4fad20c4c9a726170a25da0 Gerrit-PatchSet: 8 Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Owner: Mpaa <mpaa.wiki(a)gmail.com> Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com> Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl> Gerrit-Reviewer: Mpaa <mpaa.wiki(a)gmail.com> Gerrit-Reviewer: Xqt <info(a)gno.de> Gerrit-Reviewer: jenkins-bot <>

1 0

Errored: wikimedia/pywikibot-core#3624 (master - 8200a24)
by Travis CI 13 Sep '16

13 Sep '16

Build Update for wikimedia/pywikibot-core ------------------------------------- Build: #3624 Status: Errored Duration: 43 minutes and 26 seconds Commit: 8200a24 (master) Author: jenkins-bot Message: Merge "Revert "[bugfix] fix tests for wikidata"" View the changeset: https://github.com/wikimedia/pywikibot-core/compare/44be554579de...8200a245… View the full build log and details: https://travis-ci.org/wikimedia/pywikibot-core/builds/159508799 -- You can configure recipients for build notifications in your .travis.yml file. See https://docs.travis-ci.com/user/notifications

1 0

[Gerrit] pywikibot/compat[master]: Remove blockreview.py from framework
by jenkins-bot (Code Review) 13 Sep '16

13 Sep '16

jenkins-bot has submitted this change and it was merged. Change subject: Remove blockreview.py from framework ...................................................................... Remove blockreview.py from framework - This script is not finished and was never used. Put it out of the scope of the framework and delete it. Change-Id: Ic47c233e85bb33b5bc6e460e0b392418a8c058b3 --- D blockreview.py 1 file changed, 0 insertions(+), 330 deletions(-) Approvals: Xqt: Looks good to me, approved jenkins-bot: Verified diff --git a/blockreview.py b/blockreview.py deleted file mode 100644 index b59ed35..0000000 --- a/blockreview.py +++ /dev/null @@ -1,330 +0,0 @@ -#!/usr/bin/python -# -*- coding: utf-8 -*- -""" -This bot implements a blocking review process for de-wiki first. -For other sites this bot script must be changed. - -This script is run by [[de:User:xqt]]. It should -not be run by other users without prior contact. - -The following parameters are supported: - -- - -All other parameters will be regarded as part of the title of a single page, -and the bot will only work on that single page. -""" -# -# (C) xqt, 2010-2013 -# -# Distributed under the terms of the MIT license. -# -__version__ = '$Id$' -# - -import wikipedia as pywikibot -import userlib -import query - - -class BlockreviewBot: - # notes - note_admin = { - 'de': u""" - -== Sperrprüfungswunsch == -Hallo %(admin)s, - -[[%(user)s]] wünscht die Prüfung seiner/ihrer Sperre vom %(time)s über die Dauer von %(duration)s. Kommentar war ''%(comment)s''. Bitte äußere Dich dazu auf der [[%(usertalk)s#%(section)s|Diskussionsseite]]. -~~~~""" - } - - note_project = { - 'de': u""" - -== [[%(user)s]] == -* gesperrt am %(time)s durch {{Benutzer|%(admin)s}} für eine Dauer von %(duration)s. -* Kommentar war ''%(comment)s''. -* [[Benutzer:%(admin)s]] wurde [[Benutzer Diskussion:%(admin)s#Sperrprüfungswunsch|benachrichtigt]]. -* [[%(usertalk)s#%(section)s|Link zur Diskussion]] -:<small>-~~~~</small> -;Antrag entgegengenommen""" - } - - # edit summaries - msg_admin = { - 'de': u'Bot-Benachrichtigung: Sperrprüfungswunsch von [[%(user)s]]', - } - - msg_user = { - 'de': u'Bot: Administrator [[Benutzer:%(admin)s|%(admin)s]] für Sperrprüfung benachrichtigt', - } - - msg_done = { - 'de': u'Bot: Sperrprüfung abgeschlossen. Benutzer ist entsperrt.', - } - - unblock_tpl = { - 'de': u'Benutzer:TAXman/Sperrprüfungsverfahren', - 'pt': u'Predefinição:Discussão de bloqueio', - } - - review_cat = { - 'de': u'Wikipedia:Sperrprüfung', - } - - project_name = { - 'de': u'Benutzer:TAXman/Sperrprüfung Neu', - 'pt': u'Wikipedia:Pedidos a administradores/Discussão de bloqueio', - } - - def __init__(self, dry): - """ - Constructor. Parameters: - * generator - The page generator that determines on which pages - to work on. - * dry - If True, doesn't do any real changes, but only shows - what would have been changed. - """ - self.site = pywikibot.getSite() - self.dry = dry - self.info = None - self.parts = None - - def run(self): - # TODO: change the generator for template to the included category - try: - genPage = pywikibot.Page(self.site, - self.unblock_tpl[self.site.lang], - defaultNamespace=10) - except KeyError: - pywikibot.error(u'Language "%s" not supported by this bot.' - % self.site.lang) - else: - for page in genPage.getReferences(follow_redirects=False, - withTemplateInclusion=True, - onlyTemplateInclusion=True): - if page.namespace() == 3: - self.treat(page) - else: - pywikibot.output(u'Ignoring %s, user namespace required' - % page.title(asLink=True)) - - def treat(self, userPage): - """ - Loads the given page, does some changes, and saves it. - """ - talkText = self.load(userPage) - if not talkText: - # sanity check. No talk page found. - return - unblock_tpl = self.unblock_tpl[self.site.lang] - project_name = self.project_name[self.site.lang] - user = userlib.User(self.site, userPage.title(withNamespace=False)) - saveAdmin = saveProject = False - talkComment = None - for templates in userPage.templatesWithParams(): - if templates[0] == unblock_tpl: - self.getInfo(user) - # Step 1 - # a new template is set on blocked users talk page. - # Notify the blocking admin - if templates[1] == [] or templates[1][0] == u'1': - if self.info['action'] == 'block' or user.isBlocked(): - if self.site.sitename() == 'wikipedia:de': - admin = userlib.User(self.site, self.info['user']) - adminPage = admin.getUserTalkPage() - adminText = adminPage.get() - note = pywikibot.translate(self.site.lang, - self.note_admin, - self.parts) - comment = pywikibot.translate(self.site.lang, - self.msg_admin, - self.parts) - adminText += note - self.save(adminText, adminPage, comment, False) - ### test for pt-wiki - ### just print all sysops talk pages - elif self.site.sitename() == 'wikipedia:pt': - import pagegenerators as pg - gen = pg.PreloadingGenerator(self.SysopGenerator()) - for sysop in gen: - print sysop.title() - - talkText = talkText.replace(u'{{%s}}' % unblock_tpl, - u'{{%s|2}}' % unblock_tpl) - talkText = talkText.replace(u'{{%s|1}}' % unblock_tpl, - u'{{%s|2}}' % unblock_tpl) - talkComment = pywikibot.translate(self.site.lang, - self.msg_user - % self.parts) - - # some test stuff - if pywikibot.logger.isEnabledFor(pywikibot.DEBUG) \ - and self.site().loggedInAs() == u'Xqbot:': - testPage = pywikibot.Page(self.site, - 'Benutzer:Xqt/Test') - test = testPage.get() - test += note - self.save(test, testPage, - '[[WP:BA#SPP-Bot|SPPB-Test]]') - else: - # nicht blockiert. Fall auf DS abschließen - talkText = talkText.replace(u'{{%s}}' % unblock_tpl, - u'{{%s|4}}' % unblock_tpl) - talkText = talkText.replace(u'{{%s|1}}' % unblock_tpl, - u'{{%s|4}}' % unblock_tpl) - talkComment = pywikibot.translate(self.site.lang, - self.msg_done) - # Step 2 - # Admin has been notified. - # Wait for 2 hours, than put a message to the project page - elif templates[1][0] == u'2': - if self.info['action'] == 'block' or user.isBlocked(): - # TODO: check whether wait time is gone - # check whether this entry already esists - project = pywikibot.Page(self.site, project_name) - projText = project.get() - note = pywikibot.translate(self.site.lang, - self.note_project, - self.parts) - comment = pywikibot.translate(self.site.lang, - self.msg_admin, - self.parts) - projText += note - self.save(projText, project, comment, botflag=False) - talkText = talkText.replace(u'{{%s|2}}' % unblock_tpl, - u'{{%s|3}}' % unblock_tpl) - talkComment = u'Bot: [[%s|Wikipedia:Sperrprüfung]] eingetragen' \ - % project_name - else: - # User is unblocked. Review can be closed - talkText = talkText.replace(u'{{%s|2}}' % unblock_tpl, - u'{{%s|4}}' % unblock_tpl) - talkComment = pywikibot.translate(self.site.lang, - self.msg_done) - # Step 3 - # Admin is notified, central project page has a message - # Discussion is going on - # Check whether it can be closed - elif templates[1][0] == u'3': - if self.info['action'] == 'block' or user.isBlocked(): - pass - else: - # User is unblocked. Review can be closed - talkText = talkText.replace(u'{{%s|3}}' % unblock_tpl, - u'{{%s|4}}' % unblock_tpl) - talkComment = pywikibot.translate(self.site.lang, - self.msg_done) - # Step 4 - # Review is closed - elif templates[1][0] == u'4': - # nothing left to do - pass - else: - # wrong template found - pass - - # at last if there is a talk comment, users talk page must be changed - if talkComment: - self.save(talkText, userPage, talkComment) - - def getInfo(self, user): - if not self.info: - self.info = self.site.logpages(1, mode='block', - title=user.getUserPage().title(), - dump=True).next() - self.parts = { - 'admin': self.info['user'], - 'user': self.info['title'], - 'usertalk': user.getUserTalkPage().title(), - 'section': u'Sperrprüfung', - 'time': self.info['timestamp'], - 'duration': self.info['block']['duration'], - 'comment': self.info['comment'], - } - - def SysopGenerator(self): - params = param = { - 'action': 'query', - 'list': 'allusers', - 'augroup': 'sysop', - 'auprop': 'groups', - 'aulimit': 500, - } - data = query.GetData(params, self.site) - for user in data['query']['allusers']: - # exclude sysop bots - if 'bot' not in user['groups']: - # yield the sysop talkpage - yield pywikibot.Page(self.site, user['name'], - defaultNamespace=3) - - def load(self, page): - """ - Loads the given page, does some changes, and saves it. - """ - try: - # Load the page - text = page.get() - except pywikibot.NoPage: - pywikibot.output(u"Page %s does not exist; skipping." - % page.title(asLink=True)) - except pywikibot.IsRedirectPage: - pywikibot.output(u"Page %s is a redirect; skipping." - % page.title(asLink=True)) - else: - return text - return None - - def save(self, text, page, comment, minorEdit=True, botflag=True): - if text != page.get(): - # Show the title of the page we're working on. - # Highlight the title in purple. - pywikibot.output(u"\n\n>>> \03{lightpurple}%s\03{default} <<<" - % page.title()) - # show what was changed - pywikibot.showDiff(page.get(), text) - pywikibot.output(u'Comment: %s' % comment) - if not self.dry: - choice = pywikibot.inputChoice( - u'Do you want to accept these changes?', - ['Yes', 'No'], ['y', 'N'], 'N') - if choice == 'y': - try: - # Save the page - page.put(text, comment=comment, minorEdit=minorEdit, - botflag=botflag) - except pywikibot.LockedPage: - pywikibot.output(u"Page %s is locked; skipping." - % page.title(asLink=True)) - except pywikibot.EditConflict: - pywikibot.output( - u'Skipping %s because of edit conflict' - % (page.title())) - except pywikibot.SpamfilterError, error: - pywikibot.output( - u'Cannot change %s because of spam blacklist entry ' - u'%s' % (page.title(), error.url)) - else: - return True - return - - -def main(): - show = False - - # Parse command line arguments - for arg in pywikibot.handleArgs(): - show = True - - if not show: - bot = BlockreviewBot(pywikibot.simulate) - bot.run() - else: - pywikibot.showHelp() - -if __name__ == "__main__": - try: - main() - finally: - pywikibot.stopme() -- To view, visit https://gerrit.wikimedia.org/r/307326 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: Ic47c233e85bb33b5bc6e460e0b392418a8c058b3 Gerrit-PatchSet: 1 Gerrit-Project: pywikibot/compat Gerrit-Branch: master Gerrit-Owner: Xqt <info(a)gno.de> Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com> Gerrit-Reviewer: Xqt <info(a)gno.de> Gerrit-Reviewer: jenkins-bot <>

1 0

Errored: wikimedia/pywikibot-core#3623 (master - 44be554)
by Travis CI 13 Sep '16

13 Sep '16

Build Update for wikimedia/pywikibot-core ------------------------------------- Build: #3623 Status: Errored Duration: 44 minutes and 37 seconds Commit: 44be554 (master) Author: Translation updater bot Message: Updated git submodules Project: pywikibot/i18n master 9e2988327bba4bc1374e6991e592e77d1f93420b Localisation updates from https://translatewiki.net. Change-Id: Ia58d58836095b1e47190c8dfd7f0965906d7208d View the changeset: https://github.com/wikimedia/pywikibot-core/compare/96848bd656c9...44be5545… View the full build log and details: https://travis-ci.org/wikimedia/pywikibot-core/builds/159504567 -- You can configure recipients for build notifications in your .travis.yml file. See https://docs.travis-ci.com/user/notifications

1 0

[Gerrit] pywikibot/core[master]: Revert "[bugfix] fix tests for wikidata"
by jenkins-bot (Code Review) 13 Sep '16

13 Sep '16

jenkins-bot has submitted this change and it was merged. Change subject: Revert "[bugfix] fix tests for wikidata" ...................................................................... Revert "[bugfix] fix tests for wikidata" This reverts commit 8816913e4320dbac94c0d18926803ee8f5e8d44b. Bug: T134655 Change-Id: Ib435c5c64a3744f43aef639364dfbbd3b6712b08 --- M tests/wikibase_tests.py 1 file changed, 0 insertions(+), 17 deletions(-) Approvals: John Vandenberg: Looks good to me, but someone else must approve Xqt: Looks good to me, approved jenkins-bot: Verified diff --git a/tests/wikibase_tests.py b/tests/wikibase_tests.py index 6cfceaf..7b747e3 100644 --- a/tests/wikibase_tests.py +++ b/tests/wikibase_tests.py @@ -1131,30 +1131,13 @@ }, } - def test_own_repository(self, key): - """Test that a data repository family is its own repository.""" - site = self.get_site(key) - self.assertEqual(site, site.data_repository()) - self.assertTrue(site.is_data_repository) - def test_own_client(self, key): """Test that a data repository family can be its own client.""" site = self.get_site(key) - self.assertTrue(site.has_data_repository) - def test_item_exists(self): - """Test that a ItemPage exists for wikidata:wikidata.""" - site = self.get_site('wikidata') page = pywikibot.Page(site, 'Wikidata:Main Page') item = pywikibot.ItemPage.fromPage(page) self.assertEqual(item.site, site) - - def test_item_not_exists(self): - """Test that a ItemPage does not exists for test:wikidata.""" - site = self.get_site('wikidatatest') - page = pywikibot.Page(site, 'Wikidata:Main Page') - with self.assertRaises(pywikibot.NoPage): - pywikibot.ItemPage.fromPage(page) class TestUnconnectedClient(TestCase): -- To view, visit https://gerrit.wikimedia.org/r/298452 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: Ib435c5c64a3744f43aef639364dfbbd3b6712b08 Gerrit-PatchSet: 2 Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Owner: John Vandenberg <jayvdb(a)gmail.com> Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com> Gerrit-Reviewer: Xqt <info(a)gno.de> Gerrit-Reviewer: jenkins-bot <>

1 0

Errored: wikimedia/pywikibot-core#3621 (master - 96848bd)
by Travis CI 11 Sep '16

11 Sep '16

Build Update for wikimedia/pywikibot-core ------------------------------------- Build: #3621 Status: Errored Duration: 25 minutes and 2 seconds Commit: 96848bd (master) Author: jenkins-bot Message: Merge "Re-enable test_allpages_langlinks_enabled test" View the changeset: https://github.com/wikimedia/pywikibot-core/compare/a19edfad0f75...96848bd6… View the full build log and details: https://travis-ci.org/wikimedia/pywikibot-core/builds/159088998 -- You can configure recipients for build notifications in your .travis.yml file. See https://docs.travis-ci.com/user/notifications

1 0

Errored: wikimedia/pywikibot-core#3620 (master - a19edfa)
by Travis CI 11 Sep '16

11 Sep '16

Build Update for wikimedia/pywikibot-core ------------------------------------- Build: #3620 Status: Errored Duration: 33 minutes and 21 seconds Commit: a19edfa (master) Author: jenkins-bot Message: Merge "Print additional warning when starting a thread fails" View the changeset: https://github.com/wikimedia/pywikibot-core/compare/49f6c7ecdab1...a19edfad… View the full build log and details: https://travis-ci.org/wikimedia/pywikibot-core/builds/159086691 -- You can configure recipients for build notifications in your .travis.yml file. See https://docs.travis-ci.com/user/notifications

1 0

[Gerrit] pywikibot/core[master]: Re-enable test_allpages_langlinks_enabled test
by jenkins-bot (Code Review) 11 Sep '16

11 Sep '16

jenkins-bot has submitted this change and it was merged. Change subject: Re-enable test_allpages_langlinks_enabled test ...................................................................... Re-enable test_allpages_langlinks_enabled test - re-enable this test without allowed_failure decorator since T78276 is fixed. Bug: T145319 Change-Id: I5dcb5a0cbe67d126c927e9b20629efc0dc95a538 --- M tests/site_tests.py 1 file changed, 0 insertions(+), 1 deletion(-) Approvals: Mpaa: Looks good to me, approved jenkins-bot: Verified diff --git a/tests/site_tests.py b/tests/site_tests.py index 840f9b5..2ddd451 100644 --- a/tests/site_tests.py +++ b/tests/site_tests.py @@ -588,7 +588,6 @@ self.assertEqual(page.namespace(), 0) self.assertFalse(page.isRedirectPage()) - @allowed_failure # T78276 def test_allpages_langlinks_enabled(self): """Test allpages with langlinks enabled.""" mysite = self.get_site() -- To view, visit https://gerrit.wikimedia.org/r/309817 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I5dcb5a0cbe67d126c927e9b20629efc0dc95a538 Gerrit-PatchSet: 1 Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Owner: Xqt <info(a)gno.de> Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com> Gerrit-Reviewer: Mpaa <mpaa.wiki(a)gmail.com> Gerrit-Reviewer: jenkins-bot <>

1 0

[Gerrit] pywikibot/core[master]: Print additional warning when starting a thread fails
by jenkins-bot (Code Review) 11 Sep '16

11 Sep '16

jenkins-bot has submitted this change and it was merged. Change subject: Print additional warning when starting a thread fails ...................................................................... Print additional warning when starting a thread fails - print a warning to decrease max_external_links - increase sleeping time before starting a new thread to config.retry_wait value to wait a longer time until a previous thread was finished. Bug: T145273 Change-Id: I1cebcfa6e49f0e68975d7e9062e1dd3825b9a7ac --- M scripts/weblinkchecker.py 1 file changed, 10 insertions(+), 3 deletions(-) Approvals: Mpaa: Looks good to me, approved jenkins-bot: Verified diff --git a/scripts/weblinkchecker.py b/scripts/weblinkchecker.py index 9f5abb5..44e3296 100755 --- a/scripts/weblinkchecker.py +++ b/scripts/weblinkchecker.py @@ -862,13 +862,20 @@ # Limit the number of threads started at the same time. Each # thread will check one page, then die. while threading.activeCount() >= config.max_external_links: - # wait 100 ms - time.sleep(0.1) + time.sleep(config.retry_wait) thread = LinkCheckThread(page, url, self.history, self.HTTPignore, self.day) # thread dies when program terminates thread.setDaemon(True) - thread.start() + try: + thread.start() + except threading.ThreadError: + pywikibot.warning( + "Can't start a new thread.\nPlease decrease " + "max_external_links in your user-config.py or use\n" + "'-max_external_links:' option with a smaller value. " + "Default is 50.") + raise def RepeatPageGenerator(): -- To view, visit https://gerrit.wikimedia.org/r/309730 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I1cebcfa6e49f0e68975d7e9062e1dd3825b9a7ac Gerrit-PatchSet: 2 Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Owner: Xqt <info(a)gno.de> Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com> Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl> Gerrit-Reviewer: Mpaa <mpaa.wiki(a)gmail.com> Gerrit-Reviewer: jenkins-bot <>

1 0

Errored: wikimedia/pywikibot-core#3619 (master - 49f6c7e)
by Travis CI 11 Sep '16

11 Sep '16

Build Update for wikimedia/pywikibot-core ------------------------------------- Build: #3619 Status: Errored Duration: 34 minutes and 22 seconds Commit: 49f6c7e (master) Author: John Vandenberg Message: Use released flake8-mock 0.3 Change-Id: I10d134d8059236ea758065566a64edc390aeaab0 View the changeset: https://github.com/wikimedia/pywikibot-core/compare/5ad518163b4a...49f6c7ec… View the full build log and details: https://travis-ci.org/wikimedia/pywikibot-core/builds/159075334 -- You can configure recipients for build notifications in your .travis.yml file. See https://docs.travis-ci.com/user/notifications

1 0

[Gerrit] pywikibot/core[master]: Use released flake8-mock 0.3
by jenkins-bot (Code Review) 11 Sep '16

11 Sep '16

jenkins-bot has submitted this change and it was merged. Change subject: Use released flake8-mock 0.3 ...................................................................... Use released flake8-mock 0.3 Change-Id: I10d134d8059236ea758065566a64edc390aeaab0 --- M dev-requirements.txt M tox.ini 2 files changed, 2 insertions(+), 2 deletions(-) Approvals: Xqt: Looks good to me, approved jenkins-bot: Verified diff --git a/dev-requirements.txt b/dev-requirements.txt index e8bab19..edd5005 100644 --- a/dev-requirements.txt +++ b/dev-requirements.txt @@ -17,7 +17,7 @@ pyflakes>=0.9 flake8 flake8-docstrings>=0.2.6 -git+https://github.com/jayvdb/flake8-mock@use-ascii-readme +flake8-mock>=0.3 codecov coverage mock ; python_version < '3' diff --git a/tox.ini b/tox.ini index c436cf8..f053a96 100644 --- a/tox.ini +++ b/tox.ini @@ -62,7 +62,7 @@ flake8-import-order flake8-tuple>=0.2.8 flake8-print>=2.0.1 - git+https://github.com/jayvdb/flake8-mock@use-ascii-readme + flake8-mock>=0.3 pep8-naming [testenv:nose] -- To view, visit https://gerrit.wikimedia.org/r/309816 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I10d134d8059236ea758065566a64edc390aeaab0 Gerrit-PatchSet: 1 Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Owner: John Vandenberg <jayvdb(a)gmail.com> Gerrit-Reviewer: Xqt <info(a)gno.de> Gerrit-Reviewer: jenkins-bot <>

1 0

Errored: wikimedia/pywikibot-core#3618 (master - 5ad5181)
by Travis CI 10 Sep '16

10 Sep '16

Build Update for wikimedia/pywikibot-core ------------------------------------- Build: #3618 Status: Errored Duration: 39 minutes and 1 second Commit: 5ad5181 (master) Author: Alex Monk Message: Default wikimedia rcstream to https on port 443 Instead of http on port 80 This commit itself is not tested (I'm not a pywikibot user), though I have checked my own little socketIO_client script and it should be okay. Bug: T145244 Change-Id: I4a01817ba7e525f3ebdf82cab939ac8bbcdf1835 View the changeset: https://github.com/wikimedia/pywikibot-core/compare/6453e15141d4...5ad51816… View the full build log and details: https://travis-ci.org/wikimedia/pywikibot-core/builds/158849183 -- You can configure recipients for build notifications in your .travis.yml file. See https://docs.travis-ci.com/user/notifications

1 0

[Gerrit] pywikibot/core[master]: Default wikimedia rcstream to https on port 443
by jenkins-bot (Code Review) 10 Sep '16

10 Sep '16

jenkins-bot has submitted this change and it was merged. Change subject: Default wikimedia rcstream to https on port 443 ...................................................................... Default wikimedia rcstream to https on port 443 Instead of http on port 80 This commit itself is not tested (I'm not a pywikibot user), though I have checked my own little socketIO_client script and it should be okay. Bug: T145244 Change-Id: I4a01817ba7e525f3ebdf82cab939ac8bbcdf1835 --- M pywikibot/comms/rcstream.py M pywikibot/family.py 2 files changed, 10 insertions(+), 5 deletions(-) Approvals: Merlijn van Deen: Looks good to me, approved jenkins-bot: Verified diff --git a/pywikibot/comms/rcstream.py b/pywikibot/comms/rcstream.py index 3e67ed2..823597a 100644 --- a/pywikibot/comms/rcstream.py +++ b/pywikibot/comms/rcstream.py @@ -45,7 +45,7 @@ is passed to rcstream using a 'subscribe' command. Pass '*' to listen to all wikis for a given rc host. @param rchost: the recent changes stream host to connect to. For Wikimedia - wikis, this is 'stream.wikimedia.org' + wikis, this is 'https://stream.wikimedia.org' @param rcport: the port to connect to (default: 80) @param rcpath: the sockets.io path. For Wikimedia wikis, this is '/rc'. (default: '/rc') @@ -58,7 +58,7 @@ Usage: - >>> t = RcListenerThread('en.wikipedia.org', 'stream.wikimedia.org') + >>> t = RcListenerThread('en.wikipedia.org', 'https://stream.wikimedia.org') >>> t.start() >>> change = t.queue.get() >>> change @@ -164,7 +164,7 @@ is passed to rcstream using a 'subscribe' command. Pass '*' to listen to all wikis for a given rc host. @param rchost: the recent changes stream host to connect to. For Wikimedia - wikis, this is 'stream.wikimedia.org' + wikis, this is 'https://stream.wikimedia.org' @param rcport: the port to connect to (default: 80) @param rcpath: the sockets.io path. For Wikimedia wikis, this is '/rc'. (default: '/rc') @@ -218,5 +218,6 @@ return rc_listener( wikihost=site.hostname(), rchost=site.rcstream_host(), + rcport=site.rcstream_port(), total=total, ) diff --git a/pywikibot/family.py b/pywikibot/family.py index c95f2e5..6058e6d 100644 --- a/pywikibot/family.py +++ b/pywikibot/family.py @@ -1612,8 +1612,12 @@ return 'https' def rcstream_host(self, code): - """Return 'stream.wikimedia.org' as the RCStream hostname.""" - return 'stream.wikimedia.org' + """Return 'https://stream.wikimedia.org' as the RCStream hostname.""" + return 'https://stream.wikimedia.org' + + def rcstream_port(self, code): + """Return 443 as the RCStream port number.""" + return 443 class WikimediaOrgFamily(SingleSiteFamily, WikimediaFamily): -- To view, visit https://gerrit.wikimedia.org/r/309632 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I4a01817ba7e525f3ebdf82cab939ac8bbcdf1835 Gerrit-PatchSet: 1 Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Owner: Alex Monk <alex(a)wikimedia.org> Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com> Gerrit-Reviewer: Legoktm <legoktm.wikipedia(a)gmail.com> Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl> Gerrit-Reviewer: Mpaa <mpaa.wiki(a)gmail.com> Gerrit-Reviewer: Xqt <info(a)gno.de> Gerrit-Reviewer: jenkins-bot <>

1 0

Errored: wikimedia/pywikibot-core#3617 (master - 6453e15)
by Travis CI 10 Sep '16

10 Sep '16

Build Update for wikimedia/pywikibot-core ------------------------------------- Build: #3617 Status: Errored Duration: 30 minutes and 33 seconds Commit: 6453e15 (master) Author: jenkins-bot Message: Merge "Match 'score' in textlib" View the changeset: https://github.com/wikimedia/pywikibot-core/compare/287ee3196449...6453e151… View the full build log and details: https://travis-ci.org/wikimedia/pywikibot-core/builds/158791824 -- You can configure recipients for build notifications in your .travis.yml file. See https://docs.travis-ci.com/user/notifications

1 0

[Gerrit] pywikibot/core[master]: Match 'score' in textlib
by jenkins-bot (Code Review) 10 Sep '16

10 Sep '16

jenkins-bot has submitted this change and it was merged. Change subject: Match 'score' in textlib ...................................................................... Match 'score' in textlib cherry-picked from compat Ic69777f3db3c235380eabd778a5052403b0f8ce6 Change-Id: I68206e3dc4f22e333e69e5dfd8c3def4b3978566 --- M pywikibot/textlib.py 1 file changed, 1 insertion(+), 0 deletions(-) Approvals: Mpaa: Looks good to me, approved jenkins-bot: Verified diff --git a/pywikibot/textlib.py b/pywikibot/textlib.py index 4492d8c..5eb0064 100644 --- a/pywikibot/textlib.py +++ b/pywikibot/textlib.py @@ -219,6 +219,7 @@ # preformatted text 'pre': re.compile(r'(?ism)<pre>.*?</pre>'), 'source': re.compile(r'(?is)<source .*?</source>'), + 'score': re.compile(r'(?ism)<score[ >].*?</score>'), # inline references 'ref': re.compile(r'(?ism)<ref[ >].*?</ref>'), 'template': NESTED_TEMPLATE_REGEX, -- To view, visit https://gerrit.wikimedia.org/r/308804 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I68206e3dc4f22e333e69e5dfd8c3def4b3978566 Gerrit-PatchSet: 2 Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Owner: Xqt <info(a)gno.de> Gerrit-Reviewer: Bináris <wikiposta(a)gmail.com> Gerrit-Reviewer: Bináris2 <wikiposta+wikitech(a)gmail.com> Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com> Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl> Gerrit-Reviewer: Mpaa <mpaa.wiki(a)gmail.com> Gerrit-Reviewer: Xqt <info(a)gno.de> Gerrit-Reviewer: jenkins-bot <>

1 0

Errored: wikimedia/pywikibot-core#3616 (master - 287ee31)
by Travis CI 08 Sep '16

08 Sep '16

Build Update for wikimedia/pywikibot-core ------------------------------------- Build: #3616 Status: Errored Duration: 38 minutes and 24 seconds Commit: 287ee31 (master) Author: Translation updater bot Message: Updated git submodules Project: pywikibot/i18n master 0ebae593f6d8552a66d3bcccfcbb409c25d659a1 Localisation updates from https://translatewiki.net. Change-Id: Ia36aca33a947c69c216e1c76ae85f34b4981938b View the changeset: https://github.com/wikimedia/pywikibot-core/compare/a1dd98eed93d...287ee319… View the full build log and details: https://travis-ci.org/wikimedia/pywikibot-core/builds/158394198 -- You can configure recipients for build notifications in your .travis.yml file. See https://docs.travis-ci.com/user/notifications

1 0

Errored: wikimedia/pywikibot-core#3614 (master - a1dd98e)
by Travis CI 08 Sep '16

08 Sep '16

Build Update for wikimedia/pywikibot-core ------------------------------------- Build: #3614 Status: Errored Duration: 25 minutes and 22 seconds Commit: a1dd98e (master) Author: jenkins-bot Message: Merge "[IMPR] Provide global step parameter via config2.py" View the changeset: https://github.com/wikimedia/pywikibot-core/compare/c4715d687745...a1dd98ee… View the full build log and details: https://travis-ci.org/wikimedia/pywikibot-core/builds/158273846 -- You can configure recipients for build notifications in your .travis.yml file. See https://docs.travis-ci.com/user/notifications

1 0

[Gerrit] pywikibot/core[master]: [IMPR] Provide global step parameter via config2.py
by jenkins-bot (Code Review) 08 Sep '16

08 Sep '16

jenkins-bot has submitted this change and it was merged. Change subject: [IMPR] Provide global step parameter via config2.py ...................................................................... [IMPR] Provide global step parameter via config2.py step parameter is reimplemented as config setting used by api. It could be set either in user_config.py or given as global option -step:<value>. As before, default is 50 and setting could be overwriten by set_query_increment() during runtime. Bug: T109208 Change-Id: I02c6ea9c1a8fc6c3cd4fdf1e39f7317b671ffa10 --- M pywikibot/bot.py M pywikibot/config2.py M pywikibot/data/api.py M pywikibot/pagegenerators.py 4 files changed, 23 insertions(+), 16 deletions(-) Approvals: Mpaa: Looks good to me, approved jenkins-bot: Verified diff --git a/pywikibot/bot.py b/pywikibot/bot.py index 895e56f..bb986e2 100644 --- a/pywikibot/bot.py +++ b/pywikibot/bot.py @@ -908,7 +908,7 @@ daemonize.daemonize(redirect_std=redirect_std) else: # the argument depends on numerical config settings - # e.g. -maxlag: + # e.g. -maxlag and -step: try: _arg = option[1:] # explicitly check for int (so bool doesn't match) diff --git a/pywikibot/config2.py b/pywikibot/config2.py index 0491846..de34c9b 100644 --- a/pywikibot/config2.py +++ b/pywikibot/config2.py @@ -618,6 +618,10 @@ # running solve_disambiguation.py with the -primary argument. special_page_limit = 500 +# Maximum of pages which can be retrieved at one time from wiki server. +# -1 indicates limit by api restriction +step = -1 + # Maximum number of times to retry an API request before quitting. max_retries = 25 # Minimum time to wait before resubmitting a failed API request. diff --git a/pywikibot/data/api.py b/pywikibot/data/api.py index a42caf6..1db496c 100644 --- a/pywikibot/data/api.py +++ b/pywikibot/data/api.py @@ -1,7 +1,7 @@ # -*- coding: utf-8 -*- """Interface to Mediawiki's api.php.""" # -# (C) Pywikibot team, 2007-2015 +# (C) Pywikibot team, 2007-2016 # # Distributed under the terms of the MIT license. # @@ -2383,7 +2383,10 @@ self.limit_name = limit_name self.data_name = data_name - self.query_increment = 50 + if config.step > 0: + self.query_increment = config.step + else: + self.query_increment = None self.limit = None self.starting_offset = kwargs['parameters'].pop(self.continue_name, 0) self.request = self.request_class(**kwargs) @@ -2393,7 +2396,7 @@ """ Set the maximum number of items to be retrieved per API query. - If not called, the default is 50. + If not called, the default is config.step. @param value: The value of maximum number of items to be retrieved per API request to set. @@ -2544,7 +2547,10 @@ else: self.request[prefix + 'limit'] = int(param["max"]) - self.api_limit = None + if config.step > 0: + self.api_limit = config.step + else: + self.api_limit = None if self.limited_module: self.prefix = self.site._paraminfo['query+' + self.limited_module]['prefix'] @@ -2607,13 +2613,15 @@ param = self.site._paraminfo.parameter('query+' + self.limited_module, 'limit') if self.site.logged_in() and self.site.has_right('apihighlimits'): - self.api_limit = int(param["highmax"]) + limit = int(param['highmax']) else: - self.api_limit = int(param["max"]) - pywikibot.debug(u"%s: Set query_limit to %i." - % (self.__class__.__name__, - self.api_limit), - _logger) + limit = int(param['max']) + if self.api_limit is None or limit < self.api_limit: + self.api_limit = limit + pywikibot.debug( + '{0}: Set query_limit to {1}.'.format(self.__class__.__name__, + self.api_limit), + _logger) def set_namespace(self, namespaces): """Set a namespace filter on this query. diff --git a/pywikibot/pagegenerators.py b/pywikibot/pagegenerators.py index 6a59a1d..4507db4 100644 --- a/pywikibot/pagegenerators.py +++ b/pywikibot/pagegenerators.py @@ -696,11 +696,6 @@ u'What namespace are you filtering on?') self._namespaces += value.split(",") return True - elif arg == '-step': - issue_deprecation_warning( - 'The usage of "{0}"'.format(arg), None, 2, - ArgumentDeprecationWarning) - return False elif arg == '-limit': if not value: value = pywikibot.input('What is the limit value?') -- To view, visit https://gerrit.wikimedia.org/r/274991 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I02c6ea9c1a8fc6c3cd4fdf1e39f7317b671ffa10 Gerrit-PatchSet: 10 Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Owner: Xqt <info(a)gno.de> Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com> Gerrit-Reviewer: Mpaa <mpaa.wiki(a)gmail.com> Gerrit-Reviewer: Xqt <info(a)gno.de> Gerrit-Reviewer: jenkins-bot <>

1 0

Errored: wikimedia/pywikibot-core#3613 (master - c4715d6)
by Travis CI 08 Sep '16

08 Sep '16

Build Update for wikimedia/pywikibot-core ------------------------------------- Build: #3613 Status: Errored Duration: 38 minutes and 55 seconds Commit: c4715d6 (master) Author: xqt Message: Simplify replaceExcept predefined exceptions - fix spelling mistake - simplify format string - remove recheck of else clause Change-Id: I82f4ad991fad98df5decd02ccb869a617353bf2d View the changeset: https://github.com/wikimedia/pywikibot-core/compare/a6c66789f31a...c4715d68… View the full build log and details: https://travis-ci.org/wikimedia/pywikibot-core/builds/158242300 -- You can configure recipients for build notifications in your .travis.yml file. See https://docs.travis-ci.com/user/notifications

1 0

[Gerrit] pywikibot/core[master]: Simplify replaceExcept predefined exceptions
by jenkins-bot (Code Review) 08 Sep '16

08 Sep '16

jenkins-bot has submitted this change and it was merged. Change subject: Simplify replaceExcept predefined exceptions ...................................................................... Simplify replaceExcept predefined exceptions - fix spelling mistake - simplify format string - remove recheck of else clause Change-Id: I82f4ad991fad98df5decd02ccb869a617353bf2d --- M pywikibot/textlib.py 1 file changed, 3 insertions(+), 4 deletions(-) Approvals: Mpaa: Looks good to me, approved jenkins-bot: Verified diff --git a/pywikibot/textlib.py b/pywikibot/textlib.py index 4492d8c..e186cbe 100644 --- a/pywikibot/textlib.py +++ b/pywikibot/textlib.py @@ -281,11 +281,10 @@ else: result.append(_regex_cache[exc]) else: - # nowiki, noinclude, includeonly, timeline, math ond other + # nowiki, noinclude, includeonly, timeline, math and other # extensions - if exc not in _regex_cache: - _regex_cache[exc] = re.compile(r'(?is)<%s>.*?</%s>' - % (exc, exc)) + _regex_cache[exc] = re.compile( + r'(?is)<{0}>.*?</{0}>'.format(exc)) result.append(_regex_cache[exc]) # handle alias if exc == 'source': -- To view, visit https://gerrit.wikimedia.org/r/308681 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I82f4ad991fad98df5decd02ccb869a617353bf2d Gerrit-PatchSet: 3 Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Owner: Xqt <info(a)gno.de> Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com> Gerrit-Reviewer: Mpaa <mpaa.wiki(a)gmail.com> Gerrit-Reviewer: Xqt <info(a)gno.de> Gerrit-Reviewer: jenkins-bot <>

1 0

[Gerrit] pywikibot/compat[master]: Match 'score' in textlib
by jenkins-bot (Code Review) 07 Sep '16

07 Sep '16

jenkins-bot has submitted this change and it was merged. Change subject: Match 'score' in textlib ...................................................................... Match 'score' in textlib Change-Id: Ic69777f3db3c235380eabd778a5052403b0f8ce6 --- M pywikibot/textlib.py 1 file changed, 2 insertions(+), 1 deletion(-) Approvals: Xqt: Looks good to me, approved jenkins-bot: Verified diff --git a/pywikibot/textlib.py b/pywikibot/textlib.py index 41c61ab..06edb36 100644 --- a/pywikibot/textlib.py +++ b/pywikibot/textlib.py @@ -7,7 +7,7 @@ """ # -# (C) Pywikibot team, 2004-2013 +# (C) Pywikibot team, 2004-2016 # # Distributed under the terms of the MIT license. # @@ -69,6 +69,7 @@ # preformatted text 'pre': re.compile(r'(?ism)<pre>.*?</pre>'), 'source': re.compile(r'(?is)<source .*?</source>'), + 'score': re.compile(r'(?is)<score.*?</score>'), # inline references 'ref': re.compile(r'(?ism)<ref[ >].*?</ref>'), # lines that start with a space are shown in a monospace font and -- To view, visit https://gerrit.wikimedia.org/r/308606 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: Ic69777f3db3c235380eabd778a5052403b0f8ce6 Gerrit-PatchSet: 2 Gerrit-Project: pywikibot/compat Gerrit-Branch: master Gerrit-Owner: Gerrit Patch Uploader <gerritpatchuploader(a)gmail.com> Gerrit-Reviewer: Bináris <wikiposta(a)gmail.com> Gerrit-Reviewer: Bináris2 <wikiposta+wikitech(a)gmail.com> Gerrit-Reviewer: Gerrit Patch Uploader <gerritpatchuploader(a)gmail.com> Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com> Gerrit-Reviewer: Xqt <info(a)gno.de> Gerrit-Reviewer: jenkins-bot <>

1 0

Errored: wikimedia/pywikibot-core#3612 (master - a6c6678)
by Travis CI 05 Sep '16

05 Sep '16

Build Update for wikimedia/pywikibot-core ------------------------------------- Build: #3612 Status: Errored Duration: 29 minutes and 17 seconds Commit: a6c6678 (master) Author: jenkins-bot Message: Merge "[L10N] Enable unusedfiles.py at meta-wiki" View the changeset: https://github.com/wikimedia/pywikibot-core/compare/37460fa911b3...a6c66789… View the full build log and details: https://travis-ci.org/wikimedia/pywikibot-core/builds/157630734 -- You can configure recipients for build notifications in your .travis.yml file. See https://docs.travis-ci.com/user/notifications

1 0

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

Pywikibot-commits September 2016