Pywikibot-commits May 2017

pywikibot-commits@lists.wikimedia.org

2 participants
99 discussions

[Gerrit] pywikibot/core[master]: upload.py: remove "Do you want to change this description?" ...
by jenkins-bot (Code Review) 03 May '17

03 May '17

jenkins-bot has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/348485 ) Change subject: upload.py: remove "Do you want to change this description?" when it's empty ...................................................................... upload.py: remove "Do you want to change this description?" when it's empty - If no description, ask if user want to add one or quit, and loop until one is filled. - If self.verifyDescription, ask if user want to change it or continue. - If neither of them, continues Bug: T163108 Change-Id: Iec43845e293721aee70d8f1129b323d7fbceb8b8 --- M pywikibot/specialbots.py 1 file changed, 16 insertions(+), 12 deletions(-) Approvals: jenkins-bot: Verified Xqt: Looks good to me, approved diff --git a/pywikibot/specialbots.py b/pywikibot/specialbots.py index 72e079c..f18e381 100644 --- a/pywikibot/specialbots.py +++ b/pywikibot/specialbots.py @@ -332,24 +332,26 @@ # A proper description for the submission. # Empty descriptions are not accepted. - pywikibot.output(u'The suggested description is:\n%s' - % self.description) - - # Description must be set and verified - if not self.description: - self.verifyDescription = True + if self.description: + pywikibot.output('The suggested description is:\n%s' + % self.description) while not self.description or self.verifyDescription: if not self.description: pywikibot.output(color_format( '{lightred}It is not possible to upload a file ' - 'without a summary/description.{default}')) - + 'without a description.{default}')) assert not always - # if no description, default is 'yes' - if pywikibot.input_yn( - u'Do you want to change this description?', - default=not self.description): + # if no description, ask if user want to add one or quit, + # and loop until one is filled. + # if self.verifyDescription, ask if user want to change it + # or continue. + if self.description: + question = 'Do you want to change this description?' + else: + question = 'No description was given. Add one?' + if pywikibot.input_yn(question, default=not self.description, + automatic_quit=self.description): from pywikibot import editor as editarticle editor = editarticle.TextEditor() try: @@ -360,6 +362,8 @@ # if user saved / didn't press Cancel if newDescription: self.description = newDescription + elif not self.description: + raise QuitKeyboardInterrupt self.verifyDescription = False return filename -- To view, visit https://gerrit.wikimedia.org/r/348485 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: Iec43845e293721aee70d8f1129b323d7fbceb8b8 Gerrit-PatchSet: 10 Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Owner: Gerrit Patch Uploader <gerritpatchuploader(a)gmail.com> Gerrit-Reviewer: Framawiki <framawiki(a)tools.wmflabs.org> Gerrit-Reviewer: Gerrit Patch Uploader <gerritpatchuploader(a)gmail.com> Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com> Gerrit-Reviewer: Magul <tomasz.magulski(a)gmail.com> Gerrit-Reviewer: Matěj Suchánek <matejsuchanek97(a)gmail.com> Gerrit-Reviewer: Xqt <info(a)gno.de> Gerrit-Reviewer: jenkins-bot <>

1 0

[Gerrit] pywikibot/core[master]: SiteDetectionTestCase.assertSite: Skip tests on ConnectionError
by jenkins-bot (Code Review) 03 May '17

03 May '17

jenkins-bot has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/351625 ) Change subject: SiteDetectionTestCase.assertSite: Skip tests on ConnectionError ...................................................................... SiteDetectionTestCase.assertSite: Skip tests on ConnectionError Bug: T164362 Change-Id: I041c49fc0c24ac070502e87fff3c92a863667b77 --- M tests/site_detect_tests.py 1 file changed, 1 insertion(+), 1 deletion(-) Approvals: jenkins-bot: Verified Xqt: Looks good to me, approved diff --git a/tests/site_detect_tests.py b/tests/site_detect_tests.py index a3cbfe8..cc3a2df 100644 --- a/tests/site_detect_tests.py +++ b/tests/site_detect_tests.py @@ -32,7 +32,7 @@ """ try: self.assertIsInstance(MWSite(url), MWSite) - except (ServerError, Timeout) as e: + except (ServerError, Timeout, ConnectionError) as e: self.skipTest(e) def assertNoSite(self, url): -- To view, visit https://gerrit.wikimedia.org/r/351625 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I041c49fc0c24ac070502e87fff3c92a863667b77 Gerrit-PatchSet: 1 Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Owner: Dalba <dalba.wiki(a)gmail.com> Gerrit-Reviewer: Magul <tomasz.magulski(a)gmail.com> Gerrit-Reviewer: Xqt <info(a)gno.de> Gerrit-Reviewer: jenkins-bot <>

1 0

[Gerrit] pywikibot/core[master]: Accept QuitKeyboardInterrupt in specialbots.Uploadbot
by jenkins-bot (Code Review) 03 May '17

03 May '17

jenkins-bot has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/351325 ) Change subject: Accept QuitKeyboardInterrupt in specialbots.Uploadbot ...................................................................... Accept QuitKeyboardInterrupt in specialbots.Uploadbot - call Bot.exit() when script terminates - Adjust _treat_counter and _save_counter Bug: T163970 Change-Id: Icfd50925bfea739316b405b63f7066ff83ed4eb5 --- M pywikibot/specialbots.py 1 file changed, 21 insertions(+), 5 deletions(-) Approvals: Dalba: Looks good to me, approved jenkins-bot: Verified diff --git a/pywikibot/specialbots.py b/pywikibot/specialbots.py index bd6c4d2..232e7e3 100644 --- a/pywikibot/specialbots.py +++ b/pywikibot/specialbots.py @@ -21,7 +21,7 @@ from pywikibot import config -from pywikibot.bot import BaseBot +from pywikibot.bot import BaseBot, QuitKeyboardInterrupt from pywikibot.tools import PY2, deprecated from pywikibot.tools.formatter import color_format @@ -439,6 +439,7 @@ if success: # No warning, upload complete. pywikibot.output(u"Upload of %s successful." % filename) + self._save_counter += 1 return filename # data['filename'] else: pywikibot.output(u"Upload aborted.") @@ -459,7 +460,22 @@ "User '%s' does not have upload rights on site %s." % (self.targetSite.user(), self.targetSite)) return - if isinstance(self.url, basestring): - return self.upload_file(self.url) - for file_url in self.url: - self.upload_file(file_url) + + try: + if isinstance(self.url, basestring): + self._treat_counter = 1 + return self.upload_file(self.url) + for file_url in self.url: + self.upload_file(file_url) + self._treat_counter += 1 + except QuitKeyboardInterrupt: + pywikibot.output('\nUser quit %s bot run...' % + self.__class__.__name__) + except KeyboardInterrupt: + if config.verbose_output: + raise + else: + pywikibot.output('\nKeyboardInterrupt during %s bot run...' % + self.__class__.__name__) + finally: + self.exit() -- To view, visit https://gerrit.wikimedia.org/r/351325 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: Icfd50925bfea739316b405b63f7066ff83ed4eb5 Gerrit-PatchSet: 2 Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Owner: Xqt <info(a)gno.de> Gerrit-Reviewer: Dalba <dalba.wiki(a)gmail.com> Gerrit-Reviewer: Magul <tomasz.magulski(a)gmail.com> Gerrit-Reviewer: jenkins-bot <>

1 0

[Gerrit] pywikibot/core[master]: .travis.yml, setup.py: Don't use mwoauth-0.3.1
by jenkins-bot (Code Review) 03 May '17

03 May '17

jenkins-bot has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/351622 ) Change subject: .travis.yml, setup.py: Don't use mwoauth-0.3.1 ...................................................................... .travis.yml, setup.py: Don't use mwoauth-0.3.1 There seems to be a bug in this version. Bug: T164154 Change-Id: Ie874df68ba0bbcfb11e581121e9099c7cc466be9 --- M .travis.yml M setup.py 2 files changed, 2 insertions(+), 2 deletions(-) Approvals: jenkins-bot: Verified Xqt: Looks good to me, approved diff --git a/.travis.yml b/.travis.yml index 6f6ec19..1541421 100644 --- a/.travis.yml +++ b/.travis.yml @@ -55,7 +55,7 @@ script: # Install security packages for requests to support HTTPS in site_detect - if [[ "$PYSETUP_TEST_EXTRAS" != '1' ]]; then - pip install mwoauth -r requests-requirements.txt ; + pip install mwoauth!=0.3.1 -r requests-requirements.txt ; fi - mkdir ~/.pywikibot diff --git a/setup.py b/setup.py index d5167c7..fe99201 100644 --- a/setup.py +++ b/setup.py @@ -63,7 +63,7 @@ # 0.6.1 supports socket.io 1.0, but WMF is using 0.9 (T91393 and T85716) 'rcstream': ['socketIO-client<0.6.1'], 'security': ['requests[security]', 'pycparser!=2.14'], - 'mwoauth': ['mwoauth>=0.2.4'], + 'mwoauth': ['mwoauth>=0.2.4,!=0.3.1'], 'html': ['BeautifulSoup4'], } -- To view, visit https://gerrit.wikimedia.org/r/351622 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: Ie874df68ba0bbcfb11e581121e9099c7cc466be9 Gerrit-PatchSet: 1 Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Owner: Dalba <dalba.wiki(a)gmail.com> Gerrit-Reviewer: Xqt <info(a)gno.de> Gerrit-Reviewer: jenkins-bot <>

1 0

[Gerrit] pywikibot/core[master]: Import unicode_literals in user-fixes.py.sample
by jenkins-bot (Code Review) 02 May '17

02 May '17

jenkins-bot has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/351262 ) Change subject: Import unicode_literals in user-fixes.py.sample ...................................................................... Import unicode_literals in user-fixes.py.sample This indicates to use py3 compatible unicode literals Bug:T164253 Change-Id: Iaf4380dec34c77928d30cc9de08aa35b6800b83f --- M user-fixes.py.sample 1 file changed, 3 insertions(+), 2 deletions(-) Approvals: Dalba: Looks good to me, approved jenkins-bot: Verified diff --git a/user-fixes.py.sample b/user-fixes.py.sample index a0dfa5b..0007844 100644 --- a/user-fixes.py.sample +++ b/user-fixes.py.sample @@ -3,14 +3,15 @@ # # This is only an example. Don't use it. # +from __future__ import unicode_literals fixes['example'] = { 'regex': True, 'msg': { - '_default': u'no summary specified', + '_default': 'no summary specified', }, 'replacements': [ - (r'\bword\b', u'two words'), + (r'\bword\b', 'two words'), ] } -- To view, visit https://gerrit.wikimedia.org/r/351262 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: Iaf4380dec34c77928d30cc9de08aa35b6800b83f Gerrit-PatchSet: 1 Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Owner: Xqt <info(a)gno.de> Gerrit-Reviewer: Dalba <dalba.wiki(a)gmail.com> Gerrit-Reviewer: Magul <tomasz.magulski(a)gmail.com> Gerrit-Reviewer: jenkins-bot <>

1 0

[Gerrit] pywikibot/core[master]: Some compat apispec functionality could be used in core
by jenkins-bot (Code Review) 02 May '17

02 May '17

jenkins-bot has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/267435 ) Change subject: Some compat apispec functionality could be used in core ...................................................................... Some compat apispec functionality could be used in core apispec.py is not used in compat but some methods are usable for core too. Below is a list where you find the old apispec functionality (left) in core (right) added to README-conversion.txt. iso -> Timestamp.isoformat uniso -> Timestamp.strftime('%Y-%m-%d %H:%M:%S') dt -> Timestamp.totimestampformat duration -> BlockEntry.duration Blocks.__init__ -> (* obsolete constructor *) Blocks.empty -> (* obsolete parameter cleanup *) Blocks.query -> site.blocks or site.logevents('block') Blocks.IPsortkey -> (* sort key, not needed * ) Blocks.allblocks -> site.blocks or site.logevents('block') Blocks.user -> site.blocks(user=user) Blocks.IP -> site.blocks(iprange=IP) Bug: T66835 Change-Id: I310d2b28389d143051afcef5137e60ba20e535ae --- M README-conversion.txt 1 file changed, 16 insertions(+), 0 deletions(-) Approvals: Dalba: Looks good to me, approved jenkins-bot: Verified diff --git a/README-conversion.txt b/README-conversion.txt index d67b702..d2fd9ee 100644 --- a/README-conversion.txt +++ b/README-conversion.txt @@ -123,6 +123,22 @@ - contributions(): returns a pywikibot.Timestamp object instead of a Mediawiki one +== apispec library and Blocks objects == + +Some apispec functionality could be replaced with other methods: + + iso() -> Timestamp.isoformat() + uniso() -> Timestamp.strftime('%Y-%m-%d %H:%M:%S') + dt() -> Timestamp.totimestampformat() + duration() -> BlockEntry.duration() + + Blocks.empty() -> (* obsolete parameter cleanup *) + Blocks.query() -> site.blocks() or site.logevents('block') + Blocks.IPsortkey() -> (* sort key, not needed * ) + Blocks.allblocks() -> site.blocks() or site.logevents('block') + Blocks.user() -> site.blocks(user=user) + Blocks.IP() -> site.blocks(iprange=IP) + === Command-line arguments === Scripts that supported unnamed arguments as titles of pages on which to work, -- To view, visit https://gerrit.wikimedia.org/r/267435 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I310d2b28389d143051afcef5137e60ba20e535ae Gerrit-PatchSet: 14 Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Owner: Xqt <info(a)gno.de> Gerrit-Reviewer: Bináris <wikiposta(a)gmail.com> Gerrit-Reviewer: Dalba <dalba.wiki(a)gmail.com> Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com> Gerrit-Reviewer: Magul <tomasz.magulski(a)gmail.com> Gerrit-Reviewer: Xqt <info(a)gno.de> Gerrit-Reviewer: jenkins-bot <>

1 0

[Gerrit] pywikibot/core[master]: getInternetArchiveURL: Retry http.fetch if there is a Connec...
by jenkins-bot (Code Review) 02 May '17

02 May '17

jenkins-bot has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/351241 ) Change subject: getInternetArchiveURL: Retry http.fetch if there is a ConnectionError ...................................................................... getInternetArchiveURL: Retry http.fetch if there is a ConnectionError The server sometimes returns an invalid response. Retrying should mitigate the issue. Skip the related test on ConnectionError. `max_retries` is reduced to 1 during tests and it might not be enough. Bug: T164208 Change-Id: I19e287bacb5f6165ed50d8cb82e92b7880347066 --- M pywikibot/weblib.py M tests/weblib_tests.py 2 files changed, 24 insertions(+), 2 deletions(-) Approvals: jenkins-bot: Verified Xqt: Looks good to me, approved diff --git a/pywikibot/weblib.py b/pywikibot/weblib.py index 4c27358..136fc22 100644 --- a/pywikibot/weblib.py +++ b/pywikibot/weblib.py @@ -11,6 +11,7 @@ import json import sys +from time import sleep import xml.etree.ElementTree as ET if sys.version_info[0] > 2: @@ -18,7 +19,10 @@ else: from urllib import urlencode +from requests.exceptions import ConnectionError as RequestsConnectionError + from pywikibot.comms import http +from pywikibot import config2 from pywikibot.tools import deprecated @@ -42,7 +46,19 @@ query['timestamp'] = timestamp uri = uri + urlencode(query) - jsontext = http.fetch(uri).content + + retry_count = 0 + while retry_count <= config2.max_retries: + try: + jsontext = http.fetch(uri).content + break + except RequestsConnectionError as e: + error = e + retry_count += 1 + sleep(config2.retry_wait) + else: + raise error + if "closest" in jsontext: data = json.loads(jsontext) return data['archived_snapshots']['closest']['url'] diff --git a/tests/weblib_tests.py b/tests/weblib_tests.py index 33f88bb..e4b5414 100644 --- a/tests/weblib_tests.py +++ b/tests/weblib_tests.py @@ -7,6 +7,8 @@ # from __future__ import absolute_import, unicode_literals +from requests.exceptions import ConnectionError as RequestsConnectionError + from pywikibot.tools import PY2 if not PY2: @@ -38,7 +40,11 @@ def _get_archive_url(self, url, date_string=None): with PatchedHttp(weblib, False) as p: p.after_fetch = self._test_response - archivedversion = weblib.getInternetArchiveURL(url, date_string) + try: + archivedversion = weblib.getInternetArchiveURL( + url, date_string) + except RequestsConnectionError as e: + self.skipTest(e) self.assertOneDeprecation() return archivedversion -- To view, visit https://gerrit.wikimedia.org/r/351241 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I19e287bacb5f6165ed50d8cb82e92b7880347066 Gerrit-PatchSet: 3 Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Owner: Dalba <dalba.wiki(a)gmail.com> Gerrit-Reviewer: Dalba <dalba.wiki(a)gmail.com> Gerrit-Reviewer: Magul <tomasz.magulski(a)gmail.com> Gerrit-Reviewer: Xqt <info(a)gno.de> Gerrit-Reviewer: jenkins-bot <>

1 0

[Gerrit] pywikibot/core[master]: [IMPR] Provide preloading via GeneratorFactory.getCombinedGe...
by jenkins-bot (Code Review) 02 May '17

02 May '17

jenkins-bot has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/288919 ) Change subject: [IMPR] Provide preloading via GeneratorFactory.getCombinedGenerator() ...................................................................... [IMPR] Provide preloading via GeneratorFactory.getCombinedGenerator() - replace PreloadingGenerator if possible - also disable PreloadingGenerator when -liverecentchanges is selected Bug: T135331 Change-Id: I52777a8b2405686ad7a3dffdf631789752afb7ff --- M pywikibot/pagegenerators.py M scripts/basic.py M scripts/blockpageschecker.py M scripts/capitalize_redirects.py M scripts/commons_link.py M scripts/commonscat.py M scripts/coordinate_import.py M scripts/cosmetic_changes.py M scripts/harvest_template.py M scripts/illustrate_wikidata.py M scripts/imagecopy.py M scripts/imagecopy_self.py M scripts/imageuncat.py M scripts/interwikidata.py M scripts/isbn.py M scripts/movepages.py M scripts/newitem.py M scripts/piper.py M scripts/reflinks.py M scripts/replace.py M scripts/selflink.py M scripts/table2wiki.py M scripts/template.py M scripts/touch.py M scripts/weblinkchecker.py 25 files changed, 95 insertions(+), 95 deletions(-) Approvals: Dalba: Looks good to me, approved jenkins-bot: Verified diff --git a/pywikibot/pagegenerators.py b/pywikibot/pagegenerators.py index d5a56f0..37936bc 100644 --- a/pywikibot/pagegenerators.py +++ b/pywikibot/pagegenerators.py @@ -394,6 +394,7 @@ self._site = site self._positional_arg_name = positional_arg_name self._sparql = None + self.nopreload = False @property def site(self): @@ -436,10 +437,16 @@ self.site.namespaces.resolve(self._namespaces)) return self._namespaces - def getCombinedGenerator(self, gen=None): + def getCombinedGenerator(self, gen=None, preload=False): """Return the combination of all accumulated generators. Only call this after all arguments have been parsed. + + @param gen: Another generator to be combined with + @type gen: iterator + @param preload: preload pages using PreloadingGenerator + unless self.nopreload is True + @type preload: bool """ if gen: self.gens.insert(0, gen) @@ -509,6 +516,12 @@ if self.catfilter_list: dupfiltergen = CategoryFilterPageGenerator( dupfiltergen, self.catfilter_list, self.site) + + if preload and not self.nopreload: + if isinstance(dupfiltergen, DequeGenerator): + dupfiltergen = DequePreloadingGenerator(dupfiltergen) + else: + dupfiltergen = PreloadingGenerator(dupfiltergen) return dupfiltergen @@ -740,7 +753,9 @@ _filter_unique=self._filter_unique) elif arg == '-liverecentchanges': + self.nopreload = True gen = LiveRCPageGenerator(site=self.site, total=intNone(value)) + elif arg == '-file': if not value: value = pywikibot.input('Please enter the local file name:') diff --git a/scripts/basic.py b/scripts/basic.py index 0f3879b..d056f38 100755 --- a/scripts/basic.py +++ b/scripts/basic.py @@ -25,7 +25,7 @@ -summary: Set the action summary message for the edit. """ # -# (C) Pywikibot team, 2006-2016 +# (C) Pywikibot team, 2006-2017 # # Distributed under the terms of the MIT license. # @@ -189,11 +189,10 @@ else: options[option] = True - gen = genFactory.getCombinedGenerator() + # The preloading option is responsible for downloading multiple + # pages from the wiki simultaneously. + gen = genFactory.getCombinedGenerator(preload=True) if gen: - # The preloading generator is responsible for downloading multiple - # pages from the wiki simultaneously. - gen = pagegenerators.PreloadingGenerator(gen) # pass generator and private options to the bot bot = BasicBot(gen, **options) bot.run() # guess what it does diff --git a/scripts/blockpageschecker.py b/scripts/blockpageschecker.py index acacada..5fe9446 100755 --- a/scripts/blockpageschecker.py +++ b/scripts/blockpageschecker.py @@ -50,7 +50,7 @@ # (C) Monobi a.k.a. Wikihermit, 2007 # (C) Filnik, 2007-2011 # (C) Nicolas Dumazet (NicDumZ), 2008-2009 -# (C) Pywikibot team, 2007-2016 +# (C) Pywikibot team, 2007-2017 # # Distributed under the terms of the MIT license. # @@ -284,8 +284,10 @@ generator.append(pageCat) pywikibot.output(u'Categories loaded, start!') # Main Loop - preloadingGen = pagegenerators.PreloadingGenerator(generator, groupsize=60) - for page in preloadingGen: + if not genFactory.nopreload: + generator = pagegenerators.PreloadingGenerator(generator, + groupsize=60) + for page in generator: pagename = page.title(asLink=True) pywikibot.output('Loading %s...' % pagename) try: diff --git a/scripts/capitalize_redirects.py b/scripts/capitalize_redirects.py index 7a7d75d..6576e9e 100755 --- a/scripts/capitalize_redirects.py +++ b/scripts/capitalize_redirects.py @@ -22,7 +22,7 @@ """ # # (C) Yrithinnd, 2006 -# (C) Pywikibot team, 2007-2015 +# (C) Pywikibot team, 2007-2017 # # Distributed under the terms of the MIT license. # @@ -111,10 +111,9 @@ else: genFactory.handleArg(arg) - gen = genFactory.getCombinedGenerator() + gen = genFactory.getCombinedGenerator(preload=True) if gen: - preloadingGen = pagegenerators.PreloadingGenerator(gen) - bot = CapitalizeBot(preloadingGen, **options) + bot = CapitalizeBot(gen, **options) bot.run() return True else: diff --git a/scripts/commons_link.py b/scripts/commons_link.py index 925cdd1..3ddffaf 100755 --- a/scripts/commons_link.py +++ b/scripts/commons_link.py @@ -24,7 +24,7 @@ """ # # (C) Leonardo Gregianin, 2006 -# (C) Pywikibot team, 2007-2016 +# (C) Pywikibot team, 2007-2017 # # Distributed under the terms of the MIT license. # @@ -140,9 +140,8 @@ else: genFactory.handleArg(arg) - gen = genFactory.getCombinedGenerator() + gen = genFactory.getCombinedGenerator(preload=True) if 'action' in options and gen: - gen = pagegenerators.PreloadingGenerator(gen) bot = CommonsLinkBot(gen, **options) bot.run() return True diff --git a/scripts/commonscat.py b/scripts/commonscat.py index 44a62d7..fc710ae 100755 --- a/scripts/commonscat.py +++ b/scripts/commonscat.py @@ -52,8 +52,8 @@ # November 2013 # # (C) Multichill, 2008-2009 -# (C) Xqt, 2009-2015 -# (C) Pywikibot team, 2008-2015 +# (C) Xqt, 2009-2017 +# (C) Pywikibot team, 2008-2017 # # Distributed under the terms of the MIT license. # @@ -536,8 +536,9 @@ generator = genFactory.getCombinedGenerator() if generator: - pregenerator = pagegenerators.PreloadingGenerator(generator) - bot = CommonscatBot(pregenerator, **options) + if not genFactory.nopreload: + generator = pagegenerators.PreloadingGenerator(generator) + bot = CommonscatBot(generator, **options) bot.run() return True else: diff --git a/scripts/coordinate_import.py b/scripts/coordinate_import.py index 7ed163d..9000b31 100755 --- a/scripts/coordinate_import.py +++ b/scripts/coordinate_import.py @@ -25,7 +25,7 @@ """ # # (C) Multichill, 2014 -# (C) Pywikibot team, 2013-2016 +# (C) Pywikibot team, 2013-2017 # # Distributed under the terms of MIT License. # @@ -49,7 +49,7 @@ @param generator: A generator that yields Page objects. """ super(CoordImportRobot, self).__init__() - self.generator = pagegenerators.PreloadingGenerator(generator) + self.generator = generator self.cacheSources() self.prop = 'P625' @@ -123,7 +123,7 @@ if generator_factory.handleArg(arg): continue - generator = generator_factory.getCombinedGenerator() + generator = generator_factory.getCombinedGenerator(preload=True) if generator: coordbot = CoordImportRobot(generator) diff --git a/scripts/cosmetic_changes.py b/scripts/cosmetic_changes.py index 9117d5a..bc90df3 100644 --- a/scripts/cosmetic_changes.py +++ b/scripts/cosmetic_changes.py @@ -26,7 +26,7 @@ For further information see pywikibot/cosmetic_changes.py """ # -# (C) xqt, 2009-2013 +# (C) xqt, 2009-2017 # (C) Pywikibot team, 2006-2017 # # Distributed under the terms of the MIT license. @@ -123,14 +123,13 @@ options['summary'] = i18n.twtranslate(site, 'cosmetic_changes-standalone') - gen = genFactory.getCombinedGenerator() + gen = genFactory.getCombinedGenerator(preload=True) if gen: if options.get('always') or config.simulate or pywikibot.input_yn( warning + '\nDo you really want to continue?', default=False, automatic_quit=False): site.login() - preloadingGen = pagegenerators.PreloadingGenerator(gen) - bot = CosmeticChangesBot(preloadingGen, **options) + bot = CosmeticChangesBot(gen, **options) bot.run() return True else: diff --git a/scripts/harvest_template.py b/scripts/harvest_template.py index 62d24c6..653c101 100755 --- a/scripts/harvest_template.py +++ b/scripts/harvest_template.py @@ -72,7 +72,7 @@ """ super(HarvestRobot, self).__init__() - self.generator = pg.PreloadingGenerator(generator) + self.generator = generator self.templateTitle = templateTitle.replace(u'_', u' ') # TODO: Make it a list which also includes the redirects to the template self.fields = fields @@ -262,10 +262,10 @@ for i in range(0, len(commandline_arguments), 2): fields[commandline_arguments[i]] = commandline_arguments[i + 1] - generator = gen.getCombinedGenerator() + generator = gen.getCombinedGenerator(preload=True) if not generator: gen.handleArg(u'-transcludes:' + template_title) - generator = gen.getCombinedGenerator() + generator = gen.getCombinedGenerator(preload=True) bot = HarvestRobot(generator, template_title, fields) bot.run() diff --git a/scripts/illustrate_wikidata.py b/scripts/illustrate_wikidata.py index 6e0f91a..c48eef5 100755 --- a/scripts/illustrate_wikidata.py +++ b/scripts/illustrate_wikidata.py @@ -15,7 +15,7 @@ """ # # (C) Multichill, 2014 -# (C) Pywikibot team, 2013-2016 +# (C) Pywikibot team, 2013-2017 # # Distributed under the terms of MIT License. # @@ -45,7 +45,7 @@ @type wdproperty: str """ super(IllustrateRobot, self).__init__() - self.generator = pagegenerators.PreloadingGenerator(generator) + self.generator = generator self.wdproperty = wdproperty self.cacheSources() @@ -119,7 +119,7 @@ else: generator_factory.handleArg(arg) - generator = generator_factory.getCombinedGenerator() + generator = generator_factory.getCombinedGenerator(preload=True) if not generator: pywikibot.bot.suggest_help(missing_generator=True) return False diff --git a/scripts/imagecopy.py b/scripts/imagecopy.py index 49ce90b..5a5700a 100644 --- a/scripts/imagecopy.py +++ b/scripts/imagecopy.py @@ -77,7 +77,7 @@ # # Another rewrite by: # (C) Multichill 2008-2011 -# (C) Pywikibot team, 2007-2016 +# (C) Pywikibot team, 2007-2017 # # Distributed under the terms of the MIT license. # @@ -494,7 +494,6 @@ def main(*args): """Process command line arguments and invoke bot.""" - generator = None imagepage = None always = False category = u'' @@ -510,12 +509,10 @@ else: genFactory.handleArg(arg) - generator = genFactory.getCombinedGenerator() - if not generator: + pregenerator = genFactory.getCombinedGenerator(preload=True) + if not pregenerator: pywikibot.bot.suggest_help(missing_generator=True) return False - - pregenerator = pagegenerators.PreloadingGenerator(generator) for page in pregenerator: skip = False diff --git a/scripts/imagecopy_self.py b/scripts/imagecopy_self.py index 0f2e12f..32e7954 100644 --- a/scripts/imagecopy_self.py +++ b/scripts/imagecopy_self.py @@ -45,7 +45,7 @@ # English Wikipedia specific bot by: # (C) Multichill 2010-2012 # -# (C) Pywikibot team, 2010-2016 +# (C) Pywikibot team, 2010-2017 # # Distributed under the terms of the MIT license. # @@ -1022,7 +1022,6 @@ def main(*args): """Process command line arguments and invoke bot.""" - generator = None autonomous = False checkTemplate = True @@ -1037,8 +1036,8 @@ else: genFactory.handleArg(arg) - generator = genFactory.getCombinedGenerator() - if not generator: + pregenerator = genFactory.getCombinedGenerator(preload=True) + if not pregenerator: pywikibot.bot.suggest_help(missing_generator=True) return False @@ -1050,8 +1049,6 @@ pywikibot.warning(u'It will only work on self published work images') pywikibot.warning(u'This bot is still full of bugs') pywikibot.warning(u'Use at your own risk!') - - pregenerator = pagegenerators.PreloadingGenerator(generator) prefetchQueue = Queue(maxsize=50) uploadQueue = Queue(maxsize=200) diff --git a/scripts/imageuncat.py b/scripts/imageuncat.py index 2e3693c..7bb9db1 100755 --- a/scripts/imageuncat.py +++ b/scripts/imageuncat.py @@ -1366,14 +1366,13 @@ else: genFactory.handleArg(arg) - generator = genFactory.getCombinedGenerator(gen=generator) + generator = genFactory.getCombinedGenerator(gen=generator, preload=True) if not generator: pywikibot.bot.suggest_help(missing_generator=True) return False else: - pregenerator = pagegenerators.PreloadingGenerator(generator) site.login() - for page in pregenerator: + for page in generator: pywikibot.output(page.title()) if page.exists() and (page.namespace() == 6) \ and (not page.isRedirectPage()): diff --git a/scripts/interwikidata.py b/scripts/interwikidata.py index 63dce6e..e2fea57 100644 --- a/scripts/interwikidata.py +++ b/scripts/interwikidata.py @@ -21,7 +21,7 @@ -summary: Use your own edit summary for cleaning the page. """ -# (C) Pywikibot team, 2015-2016 +# (C) Pywikibot team, 2015-2017 # # Distributed under the terms of the MIT license. # @@ -197,9 +197,8 @@ site = pywikibot.Site() - generator = genFactory.getCombinedGenerator() + generator = genFactory.getCombinedGenerator(preload=True) if generator: - generator = pagegenerators.PreloadingGenerator(generator) bot = IWBot(generator=generator, site=site, **options) bot.run() else: diff --git a/scripts/isbn.py b/scripts/isbn.py index cfd1fbb..415df16 100755 --- a/scripts/isbn.py +++ b/scripts/isbn.py @@ -35,7 +35,7 @@ """ # -# (C) Pywikibot team, 2009-2015 +# (C) Pywikibot team, 2009-2017 # # Distributed under the terms of the MIT license. # @@ -1662,13 +1662,12 @@ else: genFactory.handleArg(arg) - gen = genFactory.getCombinedGenerator() + gen = genFactory.getCombinedGenerator(preload=True) if gen: - preloadingGen = pagegenerators.PreloadingGenerator(gen) if use_wikibase: - bot = IsbnWikibaseBot(preloadingGen, **options) + bot = IsbnWikibaseBot(gen, **options) else: - bot = IsbnBot(preloadingGen, **options) + bot = IsbnBot(gen, **options) bot.run() return True else: diff --git a/scripts/movepages.py b/scripts/movepages.py index 5a61392..b4ad5bd 100755 --- a/scripts/movepages.py +++ b/scripts/movepages.py @@ -34,7 +34,7 @@ # # (C) Leonardo Gregianin, 2006 # (C) Andreas J. Schwab, 2007 -# (C) Pywikibot team, 2006-2016 +# (C) Pywikibot team, 2006-2017 # # Distributed under the terms of the MIT license. # @@ -183,7 +183,6 @@ @param args: command line arguments @type args: list of unicode """ - gen = None oldName = None options = {} fromToPairs = [] @@ -253,11 +252,9 @@ bot = MovePagesBot(None, **options) bot.moveOne(page, pair[1]) - if not gen: - gen = genFactory.getCombinedGenerator() + gen = genFactory.getCombinedGenerator(preload=True) if gen: - preloadingGen = pagegenerators.PreloadingGenerator(gen) - bot = MovePagesBot(preloadingGen, **options) + bot = MovePagesBot(gen, **options) bot.run() return True diff --git a/scripts/newitem.py b/scripts/newitem.py index 9d83d45..401de40 100755 --- a/scripts/newitem.py +++ b/scripts/newitem.py @@ -20,7 +20,7 @@ """ # # (C) Multichill, 2014 -# (C) Pywikibot team, 2014-2016 +# (C) Pywikibot team, 2014-2017 # # Distributed under the terms of the MIT license. # @@ -48,7 +48,7 @@ }) super(NewItemRobot, self).__init__(**kwargs) - self.generator = pagegenerators.PreloadingGenerator(generator) + self.generator = generator self.pageAge = self.getOption('pageage') self.lastEdit = self.getOption('lastedit') self.pageAgeBefore = self.repo.getcurrenttime() - timedelta( @@ -160,7 +160,7 @@ else: options[arg[1:].lower()] = True - generator = gen.getCombinedGenerator() + generator = gen.getCombinedGenerator(preload=True) if not generator: pywikibot.bot.suggest_help(missing_generator=True) return False diff --git a/scripts/piper.py b/scripts/piper.py index d15e4c8..757d4b5 100755 --- a/scripts/piper.py +++ b/scripts/piper.py @@ -142,11 +142,10 @@ options['filters'] = filters - gen = genFactory.getCombinedGenerator() + gen = genFactory.getCombinedGenerator(preload=True) if gen: # The preloading generator is responsible for downloading multiple # pages from the wiki simultaneously. - gen = pagegenerators.PreloadingGenerator(gen) bot = PiperBot(gen, **options) bot.run() return True diff --git a/scripts/reflinks.py b/scripts/reflinks.py index fd3352f..44e9d10 100755 --- a/scripts/reflinks.py +++ b/scripts/reflinks.py @@ -38,7 +38,7 @@ one from i18n/reflinks.py """ # (C) Nicolas Dumazet (NicDumZ), 2008 -# (C) Pywikibot team, 2008-2016 +# (C) Pywikibot team, 2008-2017 # # Distributed under the terms of the MIT license. # @@ -786,8 +786,8 @@ if not generator: pywikibot.bot.suggest_help(missing_generator=True) return False - - generator = pagegenerators.PreloadingGenerator(generator) + if not genFactory.nopreload: + generator = pagegenerators.PreloadingGenerator(generator) generator = pagegenerators.RedirectFilterPageGenerator(generator) bot = ReferencesRobot(generator, **options) bot.run() diff --git a/scripts/replace.py b/scripts/replace.py index f235c95..66a15da 100755 --- a/scripts/replace.py +++ b/scripts/replace.py @@ -1142,14 +1142,13 @@ LIMIT 200""" % (whereClause, exceptClause) gen = pagegenerators.MySQLPageGenerator(query) - gen = genFactory.getCombinedGenerator(gen) + gen = genFactory.getCombinedGenerator(gen, preload=True) if not gen: pywikibot.bot.suggest_help(missing_generator=True) return False - preloadingGen = pagegenerators.PreloadingGenerator(gen) - bot = ReplaceRobot(preloadingGen, replacements, exceptions, + bot = ReplaceRobot(gen, replacements, exceptions, allowoverlap, recursive, add_cat, sleep, edit_summary, always=acceptall, site=site) site.login() diff --git a/scripts/selflink.py b/scripts/selflink.py index 116b6a1..69a728f 100755 --- a/scripts/selflink.py +++ b/scripts/selflink.py @@ -11,7 +11,7 @@ ATTENTION: Use this with care! """ # -# (C) Pywikibot team, 2006-2015 +# (C) Pywikibot team, 2006-2017 # # Distributed under the terms of the MIT license. # @@ -20,8 +20,7 @@ import pywikibot from pywikibot.bot import Choice, MultipleSitesBot -from pywikibot.pagegenerators import GeneratorFactory, PreloadingGenerator, \ - parameterHelp +from pywikibot.pagegenerators import GeneratorFactory, parameterHelp from scripts.unlink import BaseUnlinkBot @@ -96,13 +95,12 @@ else: genFactory.handleArg(arg) - gen = genFactory.getCombinedGenerator() + gen = genFactory.getCombinedGenerator(preload=True) if not gen: pywikibot.bot.suggest_help(missing_generator=True) return False - preloadingGen = PreloadingGenerator(gen) - bot = SelflinkBot(preloadingGen, **botArgs) + bot = SelflinkBot(gen, **botArgs) bot.run() return True diff --git a/scripts/table2wiki.py b/scripts/table2wiki.py index 87aa6dc..7559ee5 100644 --- a/scripts/table2wiki.py +++ b/scripts/table2wiki.py @@ -48,7 +48,7 @@ """ # # (C) 2003 Thomas R. Koll, <tomk32(a)tomk32.de> -# (C) Pywikibot team, 2003-2013 +# (C) Pywikibot team, 2003-2017 # # Distributed under the terms of the MIT license. # @@ -594,8 +594,9 @@ if gen: if namespaces != []: gen = pagegenerators.NamespaceFilterPageGenerator(gen, namespaces) - preloadingGen = pagegenerators.PreloadingGenerator(gen) - bot = Table2WikiRobot(preloadingGen, quietMode) + if not genFactory.nopreload: + gen = pagegenerators.PreloadingGenerator(gen) + bot = Table2WikiRobot(gen, quietMode) bot.run() else: pywikibot.showHelp('table2wiki') diff --git a/scripts/template.py b/scripts/template.py index b4bd2aa..7ccffa3 100755 --- a/scripts/template.py +++ b/scripts/template.py @@ -105,8 +105,8 @@ # # (C) Daniel Herding, 2004 # (C) Rob W.W. Hooft, 2003-2005 -# (C) xqt, 2009-2016 -# (C) Pywikibot team, 2004-2016 +# (C) xqt, 2009-2017 +# (C) Pywikibot team, 2004-2017 # # Distributed under the terms of the MIT license. # @@ -356,9 +356,10 @@ # make sure that proper namespace filtering etc. is handled gen = genFactory.getCombinedGenerator(gen) - preloadingGen = pagegenerators.PreloadingGenerator(gen) + if not genFactory.nopreload: + gen = pagegenerators.PreloadingGenerator(gen) - bot = TemplateRobot(preloadingGen, templates, site=site, **options) + bot = TemplateRobot(gen, templates, site=site, **options) bot.run() if __name__ == "__main__": diff --git a/scripts/touch.py b/scripts/touch.py index 87a9c61..88d8b5f 100755 --- a/scripts/touch.py +++ b/scripts/touch.py @@ -15,7 +15,7 @@ """ # -# (C) Pywikibot team, 2009-2015 +# (C) Pywikibot team, 2009-2017 # # Distributed under the terms of the MIT license. # @@ -97,10 +97,9 @@ # -botflag options[arg[1:].lower()] = True - gen = genFactory.getCombinedGenerator() + gen = genFactory.getCombinedGenerator(preload=True) if gen: - preloadingGen = pagegenerators.PreloadingGenerator(gen) - bot = bot_class(generator=preloadingGen, **options) + bot = bot_class(generator=gen, **options) pywikibot.Site().login() bot.run() return True diff --git a/scripts/weblinkchecker.py b/scripts/weblinkchecker.py index b8cb323..9fd0274 100755 --- a/scripts/weblinkchecker.py +++ b/scripts/weblinkchecker.py @@ -95,7 +95,7 @@ """ # # (C) Daniel Herding, 2005 -# (C) Pywikibot team, 2005-2016 +# (C) Pywikibot team, 2005-2017 # # Distributed under the terms of the MIT license. # @@ -976,10 +976,11 @@ if not gen: gen = genFactory.getCombinedGenerator() if gen: - # fetch at least 240 pages simultaneously from the wiki, but more if - # a high thread number is set. - pageNumber = max(240, config.max_external_links * 2) - gen = pagegenerators.PreloadingGenerator(gen, groupsize=pageNumber) + if not genFactory.nopreload: + # fetch at least 240 pages simultaneously from the wiki, but more + # if a high thread number is set. + pageNumber = max(240, config.max_external_links * 2) + gen = pagegenerators.PreloadingGenerator(gen, groupsize=pageNumber) gen = pagegenerators.RedirectFilterPageGenerator(gen) bot = WeblinkCheckerRobot(gen, HTTPignore, config.weblink_dead_days) try: -- To view, visit https://gerrit.wikimedia.org/r/288919 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I52777a8b2405686ad7a3dffdf631789752afb7ff Gerrit-PatchSet: 11 Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Owner: Xqt <info(a)gno.de> Gerrit-Reviewer: Dalba <dalba.wiki(a)gmail.com> Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com> Gerrit-Reviewer: Magul <tomasz.magulski(a)gmail.com> Gerrit-Reviewer: Matěj Suchánek <matejsuchanek97(a)gmail.com> Gerrit-Reviewer: Xqt <info(a)gno.de> Gerrit-Reviewer: jenkins-bot <>

1 0

[Gerrit] pywikibot/core[master]: Assign object to Tkdialog if it cannot be imported from gui ...
by jenkins-bot (Code Review) 01 May '17

01 May '17

jenkins-bot has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/351102 ) Change subject: Assign object to Tkdialog if it cannot be imported from gui module ...................................................................... Assign object to Tkdialog if it cannot be imported from gui module Bug: T164163 Change-Id: I62d28206c4061ec1907b5e6b852aab3d0901a761 --- M scripts/imagecopy.py M scripts/imagecopy_self.py 2 files changed, 2 insertions(+), 2 deletions(-) Approvals: jenkins-bot: Verified Xqt: Looks good to me, approved diff --git a/scripts/imagecopy.py b/scripts/imagecopy.py index 1e2f454..fc5b6b9 100644 --- a/scripts/imagecopy.py +++ b/scripts/imagecopy.py @@ -113,7 +113,7 @@ try: from pywikibot.userinterfaces.gui import Tkdialog except ImportError as _tk_error: - Tkdialog = None + Tkdialog = object NL = '' diff --git a/scripts/imagecopy_self.py b/scripts/imagecopy_self.py index f0ec351..36d883e 100644 --- a/scripts/imagecopy_self.py +++ b/scripts/imagecopy_self.py @@ -80,7 +80,7 @@ try: from pywikibot.userinterfaces.gui import Tkdialog except ImportError as _tk_error: - Tkdialog = None + Tkdialog = object NL = '' -- To view, visit https://gerrit.wikimedia.org/r/351102 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I62d28206c4061ec1907b5e6b852aab3d0901a761 Gerrit-PatchSet: 1 Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Owner: Dalba <dalba.wiki(a)gmail.com> Gerrit-Reviewer: Magul <tomasz.magulski(a)gmail.com> Gerrit-Reviewer: Xqt <info(a)gno.de> Gerrit-Reviewer: jenkins-bot <>

1 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

Pywikibot-commits May 2017