September 2013 - Pywikibot-commits

[Gerrit] Limit ItemPage preloading to existing pages - change (pywikibot/core)
by jenkins-bot (Code Review) 05 Sep '13

05 Sep '13

jenkins-bot has submitted this change and it was merged. Change subject: Limit ItemPage preloading to existing pages ...................................................................... Limit ItemPage preloading to existing pages [[d:Q6]] does not exist, so just go up to and including [[d:Q5]]. Change-Id: I712d0138529af20dd1c88c726aa1122ae3567b46 --- M tests/site_tests.py 1 file changed, 1 insertion(+), 1 deletion(-) Approvals: Merlijn van Deen: Looks good to me, approved jenkins-bot: Verified diff --git a/tests/site_tests.py b/tests/site_tests.py index 14720f8..6d41c98 100644 --- a/tests/site_tests.py +++ b/tests/site_tests.py @@ -202,7 +202,7 @@ datasite = mysite.data_repository() - items = [pywikibot.ItemPage(datasite, 'q' + str(num)) for num in range(1, 11)] + items = [pywikibot.ItemPage(datasite, 'q' + str(num)) for num in range(1, 6)] for page in datasite.preloaditempages(items): self.assertTrue(hasattr(page, '_content')) -- To view, visit https://gerrit.wikimedia.org/r/82891 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I712d0138529af20dd1c88c726aa1122ae3567b46 Gerrit-PatchSet: 2 Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Owner: Legoktm <legoktm.wikipedia(a)gmail.com> Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com> Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl> Gerrit-Reviewer: jenkins-bot

1 0

[Gerrit] Implement a preloading generator for ItemPages - change (pywikibot/core)
by jenkins-bot (Code Review) 05 Sep '13

05 Sep '13

jenkins-bot has submitted this change and it was merged. Change subject: Implement a preloading generator for ItemPages ...................................................................... Implement a preloading generator for ItemPages Since it becomes near impossible to find which specific item lines up with which original page object, we just create new objects and yield those. WikibasePage.__defined_by had to be renamed so it is no longer a private method. Change-Id: I05917eb982f41dcdc92d8e45292ba6c27eac47f7 --- M pywikibot/page.py M pywikibot/pagegenerators.py M pywikibot/site.py M tests/site_tests.py 4 files changed, 69 insertions(+), 2 deletions(-) Approvals: Merlijn van Deen: Looks good to me, approved jenkins-bot: Verified diff --git a/pywikibot/page.py b/pywikibot/page.py index 7956ae1..1244a72 100644 --- a/pywikibot/page.py +++ b/pywikibot/page.py @@ -2293,7 +2293,11 @@ del self._link._title return Page(self).title(**kwargs) + @deprecated("_defined_by") def __defined_by(self, singular=False): + return self._defined_by(singular=singular) + + def _defined_by(self, singular=False): """ returns the parameters needed by the API to identify an item. Once an item's "p/q##" is looked up, that will be used for all future @@ -2342,7 +2346,7 @@ args can be used to specify custom props. """ if force or not hasattr(self, '_content'): - data = self.repo.loadcontent(self.__defined_by(), *args) + data = self.repo.loadcontent(self._defined_by(), *args) self.id = data.keys()[0] self._content = data[self.id] if 'lastrevid' in self._content: @@ -2429,7 +2433,7 @@ baserevid = self.lastrevid else: baserevid = None - updates = self.repo.editEntity(self.__defined_by(singular=True), data, + updates = self.repo.editEntity(self._defined_by(singular=True), data, baserevid=baserevid, **kwargs) self.lastrevid = updates['entity']['lastrevid'] diff --git a/pywikibot/pagegenerators.py b/pywikibot/pagegenerators.py index c6e8c7d..0fbfa63 100644 --- a/pywikibot/pagegenerators.py +++ b/pywikibot/pagegenerators.py @@ -797,16 +797,44 @@ site = page.site sites.setdefault(site, []).append(page) if len(sites[site]) >= step: + # if this site is at the step, process it group = sites[site] sites[site] = [] for i in site.preloadpages(group, step): yield i for site in sites: if sites[site]: + # process any leftover sites that never reached the step for i in site.preloadpages(sites[site], step): yield i +def PreloadingItemGenerator(generator, step=50): + """ + Yield preloaded pages taken from another generator. + + Function basically is copied from above, but for ItemPage's + + @param generator: pages to iterate over + @param step: how many pages to preload at once + """ + sites = {} + for page in generator: + site = page.site + sites.setdefault(site, []).append(page) + if len(sites[site]) >= step: + # if this site is at the step, process it + group = sites[site] + sites[site] = [] + for i in site.preloaditempages(group, step): + yield i + for site in sites: + if sites[site]: + # process any leftover sites that never reached the step + for i in site.preloaditempages(sites[site], step): + yield i + + def NewimagesPageGenerator(step=None, total=None, site=None): if site is None: site = pywikibot.Site() diff --git a/pywikibot/site.py b/pywikibot/site.py index ef00047..920b414 100644 --- a/pywikibot/site.py +++ b/pywikibot/site.py @@ -3398,6 +3398,32 @@ raise pywikibot.data.api.APIError, data['errors'] return data['entities'] + def preloaditempages(self, pagelist, groupsize=50): + """Yields ItemPages with content prefilled. + + Note that pages will be iterated in a different order + than in the underlying pagelist. + + @param pagelist: an iterable that yields ItemPage objects + @param groupsize: how many pages to query at a time + @type groupsize: int + """ + from pywikibot.tools import itergroup + for sublist in itergroup(pagelist, groupsize): + req = {'ids': [], 'titles': [], 'sites': []} + for p in sublist: + ident = p._defined_by() + for key in ident: + req[key].append(ident[key]) + + req = api.Request(site=self, action='wbgetentities', **req) + data = req.submit() + for qid in data['entities']: + item = pywikibot.ItemPage(self, qid) + item._content = data['entities'][qid] + item.get() # This parses the json and preloads the various properties + yield item + def getPropertyType(self, prop): """ This is used sepecifically because we can cache diff --git a/tests/site_tests.py b/tests/site_tests.py index cb29a0e..bf9e33f 100644 --- a/tests/site_tests.py +++ b/tests/site_tests.py @@ -202,6 +202,15 @@ if count >= 5: break + def testItemPreload(self): + """Test that ItemPage preloading works""" + + datasite = mysite.data_repository() + + items = [pywikibot.ItemPage(datasite, 'q' + str(num)) for num in range(1, 11)] + for page in datasite.preloaditempages(items): + self.assertTrue(hasattr(page, '_content')) + def testLinkMethods(self): """Test site methods for getting links to and from a page""" -- To view, visit https://gerrit.wikimedia.org/r/80789 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I05917eb982f41dcdc92d8e45292ba6c27eac47f7 Gerrit-PatchSet: 5 Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Owner: Legoktm <legoktm.wikipedia(a)gmail.com> Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com> Gerrit-Reviewer: Legoktm <legoktm.wikipedia(a)gmail.com> Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl> Gerrit-Reviewer: jenkins-bot

1 0

[Gerrit] Assume utf-8 encoding for password file - change (pywikibot/core)
by jenkins-bot (Code Review) 05 Sep '13

05 Sep '13

jenkins-bot has submitted this change and it was merged. Change subject: Assume utf-8 encoding for password file ...................................................................... Assume utf-8 encoding for password file Change-Id: I7fe9fe9842cdb61b8ecb373a2c071b2def54bff2 --- M pywikibot/login.py 1 file changed, 8 insertions(+), 4 deletions(-) Approvals: Merlijn van Deen: Looks good to me, approved jenkins-bot: Verified diff --git a/pywikibot/login.py b/pywikibot/login.py index e5b9f97..49cebaa 100644 --- a/pywikibot/login.py +++ b/pywikibot/login.py @@ -126,22 +126,26 @@ DO NOT FORGET TO REMOVE READ ACCESS FOR OTHER USERS!!! Use chmod 600 password-file. + All lines below should be valid Python tuples in the form (code, family, username, password) or (username, password) to set a default password for an username. Default usernames should occur above specific usernames. + If the username or password contain non-ascii characters, they + should be stored using the utf-8 encoding. + Example: - ("my_username", "my_default_password") - ("my_sysop_user", "my_sysop_password") - ("en", "wikipedia", "my_en_user", "my_en_pass") + (u"my_username", u"my_default_password") + (u"my_sysop_user", u"my_sysop_password") + (u"en", u"wikipedia", u"my_en_user", u"my_en_pass") """ password_f = open(config.password_file) for line in password_f: if not line.strip(): continue - entry = eval(line) + entry = eval(line.decode('utf-8')) if len(entry) == 2: # for default userinfo if entry[0] == self.username: self.password = entry[1] -- To view, visit https://gerrit.wikimedia.org/r/81520 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I7fe9fe9842cdb61b8ecb373a2c071b2def54bff2 Gerrit-PatchSet: 2 Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Owner: Toto Azéro <totoazero(a)yahoo.fr> Gerrit-Reviewer: DrTrigon <dr.trigon(a)surfeu.ch> Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com> Gerrit-Reviewer: Legoktm <legoktm.wikipedia(a)gmail.com> Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl> Gerrit-Reviewer: Toto Azéro <totoazero(a)yahoo.fr> Gerrit-Reviewer: jenkins-bot

1 0

[Gerrit] Have most other tests subclass PywikibotTestCase - change (pywikibot/core)
by jenkins-bot (Code Review) 05 Sep '13

05 Sep '13

jenkins-bot has submitted this change and it was merged. Change subject: Have most other tests subclass PywikibotTestCase ...................................................................... Have most other tests subclass PywikibotTestCase PywikibotTestCase now patches the request to use the cached version Change-Id: I0fe73bde40b456091fd451841438fd9bfcaabc17 --- M tests/api_tests.py M tests/site_tests.py M tests/utils.py M tests/wikibase_tests.py 4 files changed, 14 insertions(+), 12 deletions(-) Approvals: Merlijn van Deen: Looks good to me, approved jenkins-bot: Verified diff --git a/tests/api_tests.py b/tests/api_tests.py index 2954683..7c21ff0 100644 --- a/tests/api_tests.py +++ b/tests/api_tests.py @@ -7,9 +7,10 @@ __version__ = '$Id$' import datetime -import unittest import pywikibot import pywikibot.data.api as api +import unittest +from utils import PywikibotTestCase mysite = pywikibot.Site('en', 'wikipedia') @@ -33,8 +34,9 @@ self.assert_(all(len(item) == 2 for item in req.iteritems())) -class TestPageGenerator(unittest.TestCase): +class TestPageGenerator(PywikibotTestCase): def setUp(self): + super(TestPageGenerator, self).setUp() self.gen = api.PageGenerator(site=mysite, generator="links", titles="User:R'n'B") diff --git a/tests/site_tests.py b/tests/site_tests.py index 01457ee..89f076c 100644 --- a/tests/site_tests.py +++ b/tests/site_tests.py @@ -14,32 +14,24 @@ import pywikibot import warnings from tests import patch_request, unpatch_request +from utils import PywikibotTestCase mysite = None mainpage = None imagepage = None -class TestSiteObject(unittest.TestCase): +class TestSiteObject(PywikibotTestCase): """Test cases for Site methods.""" family = "wikipedia" code = "en" @classmethod def setUpClass(cls): - patch_request() global mysite, mainpage, imagepage mysite = pywikibot.Site(cls.code, cls.family) mainpage = pywikibot.Page(pywikibot.Link("Main Page", mysite)) imagepage = iter(mainpage.imagelinks()).next() # 1st image on main page - - @classmethod - def tearDownClass(cls): - unpatch_request - - def assertType(self, obj, cls): - """Assert that obj is an instance of type cls""" - return self.assertTrue(isinstance(obj, cls)) def testBaseMethods(self): """Test cases for BaseSite methods""" diff --git a/tests/utils.py b/tests/utils.py index 2aa8307..adb31b7 100644 --- a/tests/utils.py +++ b/tests/utils.py @@ -1,7 +1,14 @@ import unittest +from tests import patch_request, unpatch_request class PywikibotTestCase(unittest.TestCase): def assertType(self, obj, cls): """Assert that obj is an instance of type cls""" return self.assertTrue(isinstance(obj, cls)) + + def setUp(self): + patch_request() + + def tearDown(self): + unpatch_request() diff --git a/tests/wikibase_tests.py b/tests/wikibase_tests.py index 878de0a..fa3facd 100644 --- a/tests/wikibase_tests.py +++ b/tests/wikibase_tests.py @@ -58,6 +58,7 @@ class TestLinks(PywikibotTestCase): """Test cases to test links stored in wikidata""" def setUp(self): + super(TestLinks, self).setUp() self.wdp = pywikibot.ItemPage(wikidata, 'Q60') self.wdp.id = 'q60' self.wdp._content = json.load(open(os.path.join(os.path.split(__file__)[0], 'pages', 'Q60_only_sitelinks.wd'))) -- To view, visit https://gerrit.wikimedia.org/r/82794 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I0fe73bde40b456091fd451841438fd9bfcaabc17 Gerrit-PatchSet: 1 Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Owner: Legoktm <legoktm.wikipedia(a)gmail.com> Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com> Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl> Gerrit-Reviewer: jenkins-bot

1 0

[Gerrit] Support to add a reference with multiple claims. - change (pywikibot/core)
by Legoktm (Code Review) 05 Sep '13

05 Sep '13

Legoktm has submitted this change and it was merged. Change subject: Support to add a reference with multiple claims. ...................................................................... Support to add a reference with multiple claims. site.editSource() accepts now either a single claim (=backward-compatible) for a single claim reference or a list of claims for a multi claim reference page.addSources() is new and takes a list of claims in contrast to page.addSource() Claim.sources is a list of dicts now. Thus, it is possible to check if a property is contained. Change-Id: Iac583043531ba3f30985244ffad2aef2ad549f51 --- M pywikibot/page.py M pywikibot/site.py 2 files changed, 65 insertions(+), 34 deletions(-) Approvals: Legoktm: Verified; Looks good to me, approved jenkins-bot: Verified diff --git a/pywikibot/page.py b/pywikibot/page.py index 8816da2..bb87d46 100644 --- a/pywikibot/page.py +++ b/pywikibot/page.py @@ -87,8 +87,8 @@ self._revisions = {} else: raise pywikibot.Error( - "Invalid argument type '%s' in Page constructor: %s" - % (type(source), source)) + "Invalid argument type '%s' in Page constructor: %s" + % (type(source), source)) @property def site(self): @@ -2741,9 +2741,15 @@ bit differently, and require some more handling. """ - mainsnak = data['snaks'].values()[0][0] - wrap = {'mainsnak': mainsnak, 'hash': data['hash']} - return Claim.fromJSON(site, wrap) + source = {} + for prop in data['snaks'].values(): + for claimsnak in prop: + claim = Claim.fromJSON(site, {'mainsnak': claimsnak, 'hash': data['hash']}) + if claim.getID() in source: + source[claim.getID()].append(claim) + else: + source[claim.getID()] = [claim] + return source @staticmethod def qualifierFromJSON(site, data): @@ -2754,7 +2760,6 @@ """ wrap = {'mainsnak': data} return Claim.fromJSON(site, wrap) - def setTarget(self, value): """ @@ -2815,18 +2820,33 @@ def getSources(self): """ - Returns a list of Claims + Returns a list of sources. Each source is a list of Claims. """ return self.sources - def addSource(self, source, **kwargs): + def addSource(self, claim, **kwargs): """ - source is a Claim. - adds it as a reference. + Adds the claim as a source. + @param claim: the claim to add + @type claim: pywikibot.Claim """ - data = self.repo.editSource(self, source, new=True, **kwargs) - source.hash = data['reference']['hash'] - self.on_item.lastrevid = data['pageinfo']['lastrevid'] + self.addSources([claim], **kwargs) + + def addSources(self, claims, **kwargs): + """ + Adds the claims as one source. + @param claims: the claims to add + @type claims: list of pywikibot.Claim + """ + data = self.repo.editSource(self, claims, new=True, **kwargs) + source = {} + for claim in claims: + claim.hash = data['reference']['hash'] + self.on_item.lastrevid = data['pageinfo']['lastrevid'] + if claim.getID() in source: + source[claim.getID()].append(claim) + else: + source[claim.getID()] = [claim] self.sources.append(source) def _formatDataValue(self): @@ -2845,8 +2865,6 @@ else: raise NotImplementedError('%s datatype is not supported yet.' % self.getType()) return value - - class Revision(object): diff --git a/pywikibot/site.py b/pywikibot/site.py index 4e5dfaf..2d8142e 100644 --- a/pywikibot/site.py +++ b/pywikibot/site.py @@ -3385,7 +3385,7 @@ """get the data for multiple Wikibase items""" if type(source) == int or \ isinstance(source, basestring) and source.isdigit(): - ids = 'q'+str(source) + ids = 'q' + str(source) wbrequest = api.Request(site=self, action="wbgetentities", ids=ids, **params) wbdata = wbrequest.submit() @@ -3435,7 +3435,7 @@ #Store it for 100 years req = api.CachedRequest(expiry, site=self, **params) data = req.submit() - dtype = data['entities'][prop.getID()]['datatype'] + dtype = data['entities'][prop.getID()]['datatype'] if dtype == 'globe-coordinate': dtype = 'globecoordinate' #TODO Fix this @@ -3535,25 +3535,38 @@ if bot: params['bot'] = 1 params['token'] = self.token(claim, 'edit') - if not new and hasattr(source, 'hash'): - params['reference'] = source.hash #build up the snak - if source.getType() == 'wikibase-item': - datavalue = {'type': 'wikibase-entityid', - 'value': source._formatDataValue(), - } - elif source.getType() == 'string': - datavalue = {'type': 'string', - 'value': source._formatDataValue(), - } + if isinstance(source, list): + sources = source else: - raise NotImplementedError('%s datatype is not supported yet.' % claim.getType()) - snak = {source.getID(): [{'snaktype': 'value', - 'property': source.getID(), - 'datavalue': datavalue, - }, - ], - } + sources = [source] + + snak = {} + for sourceclaim in sources: + if sourceclaim.getType() == 'wikibase-item': + datavalue = {'type': 'wikibase-entityid', + 'value': sourceclaim._formatDataValue(), + } + elif sourceclaim.getType() == 'string': + datavalue = {'type': 'string', + 'value': sourceclaim._formatDataValue(), + } + else: + raise NotImplementedError('%s datatype is not supported yet.' % sourceclaim.getType()) + valuesnaks = [] + if sourceclaim.getID() in snak: + valuesnaks = snak[sourceclaim.getID()] + valuesnaks.append({'snaktype': 'value', + 'property': sourceclaim.getID(), + 'datavalue': datavalue, + }, + ) + + snak[sourceclaim.getID()] = valuesnaks + # set the hash if the source should be changed. if present, all claims of one source have the same hash + if not new and hasattr(sourceclaim, 'hash'): + params['reference'] = sourceclaim.hash + params['snaks'] = json.dumps(snak) for arg in kwargs: if arg in ['bot', 'lastrevid']: -- To view, visit https://gerrit.wikimedia.org/r/81219 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: Iac583043531ba3f30985244ffad2aef2ad549f51 Gerrit-PatchSet: 7 Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Owner: FelixReimann <felix(a)fex-it.de> Gerrit-Reviewer: FelixReimann <felix(a)fex-it.de> Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com> Gerrit-Reviewer: Legoktm <legoktm.wikipedia(a)gmail.com> Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl> Gerrit-Reviewer: Multichill <maarten(a)mdammers.nl> Gerrit-Reviewer: jenkins-bot

1 0

[Gerrit] replace spaces with tabs to prevent indentation errors - change (pywikibot/i18n)
by Xqt (Code Review) 04 Sep '13

04 Sep '13

Xqt has submitted this change and it was merged. Change subject: replace spaces with tabs to prevent indentation errors ...................................................................... replace spaces with tabs to prevent indentation errors Change-Id: I5c31e3abcfea5f32abcad31a51c738e7400a8ad7 --- M blockpageschecker.py 1 file changed, 3 insertions(+), 3 deletions(-) Approvals: Xqt: Looks good to me, approved diff --git a/blockpageschecker.py b/blockpageschecker.py index 04eedb2..ae61c9d 100644 --- a/blockpageschecker.py +++ b/blockpageschecker.py @@ -1,8 +1,8 @@ # -*- coding: utf-8 -*- msg = { - 'en': { - 'blockpageschecker-summary': u'Bot: Deleting out-dated template', - }, + 'en': { + 'blockpageschecker-summary': u'Bot: Deleting out-dated template', + }, 'qqq': { 'blockpageschecker-summary': u'Edit summary for blockpageschecker report', }, -- To view, visit https://gerrit.wikimedia.org/r/82608 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I5c31e3abcfea5f32abcad31a51c738e7400a8ad7 Gerrit-PatchSet: 2 Gerrit-Project: pywikibot/i18n Gerrit-Branch: master Gerrit-Owner: Xqt <info(a)gno.de> Gerrit-Reviewer: Xqt <info(a)gno.de> Gerrit-Reviewer: jenkins-bot

1 0

[Gerrit] PEP8 fixes - change (pywikibot/core)
by jenkins-bot (Code Review) 04 Sep '13

04 Sep '13

jenkins-bot has submitted this change and it was merged. Change subject: PEP8 fixes ...................................................................... PEP8 fixes This commit adds some fixes to reduce the number of PEP8 warnings produced from each commit. Change-Id: I3c2da53919fdaae34dc16dc012ae6589c4d7147a --- M tests/dry_site_tests.py M tests/i18n/test.py M tests/page_tests.py M tests/pwb/print_locals.py M tests/pwb_tests.py M tests/site_tests.py 6 files changed, 68 insertions(+), 56 deletions(-) Approvals: Legoktm: Looks good to me, approved jenkins-bot: Verified diff --git a/tests/dry_site_tests.py b/tests/dry_site_tests.py index affb8a9..27ca886 100644 --- a/tests/dry_site_tests.py +++ b/tests/dry_site_tests.py @@ -45,15 +45,17 @@ def test_must_be_user(self): x = SiteMock() wrapped_inner = pywikibot.site.must_be(group='user')(x.inner_fn) - self.assertEqual(wrapped_inner(x,1,2,3,a='a', b='b'), ((x,1,2,3), {'a': 'a', 'b': 'b'})) - self.assertEqual(x.last_fn_called, ((x,1,2,3), {'a': 'a', 'b': 'b'})) + self.assertEqual(wrapped_inner(x, 1, 2, 3, a='a', b='b'), ((x, 1, 2, 3), + {'a': 'a', 'b': 'b'})) + self.assertEqual(x.last_fn_called, ((x, 1, 2, 3), {'a': 'a', 'b': 'b'})) self.assertEqual(x.last_login, 'user') def test_must_be_sysop(self): x = SiteMock() wrapped_inner = pywikibot.site.must_be(group='sysop')(x.inner_fn) - self.assertEqual(wrapped_inner(x,1,2,3,a='a', b='b'), ((x,1,2,3), {'a': 'a', 'b': 'b'})) - self.assertEqual(x.last_fn_called, ((x,1,2,3), {'a': 'a', 'b': 'b'})) + self.assertEqual(wrapped_inner(x, 1, 2, 3, a='a', b='b'), ((x, 1, 2, 3), + {'a': 'a', 'b': 'b'})) + self.assertEqual(x.last_fn_called, ((x, 1, 2, 3), {'a': 'a', 'b': 'b'})) self.assertEqual(x.last_login, 'sysop') if __name__ == '__main__': diff --git a/tests/i18n/test.py b/tests/i18n/test.py index f93769a..e82bcd0 100644 --- a/tests/i18n/test.py +++ b/tests/i18n/test.py @@ -1,12 +1,18 @@ # -*- coding=utf-8 -*- msg = { - 'en': {'test-localized': u'test-localized EN', - 'test-semi-localized': u'test-semi-localized EN', - 'test-non-localized': u'test-non-localized EN' - }, - 'nl': {'test-localized': u'test-localized NL', - 'test-semi-localized': u'test-semi-localized NL', - }, - 'fy': {'test-localized': u'test-localized FY'}, - 'ja': {'test-no-english': u'test-no-english JA'} + 'en': { + 'test-localized': u'test-localized EN', + 'test-semi-localized': u'test-semi-localized EN', + 'test-non-localized': u'test-non-localized EN' + }, + 'nl': { + 'test-localized': u'test-localized NL', + 'test-semi-localized': u'test-semi-localized NL', + }, + 'fy': { + 'test-localized': u'test-localized FY' + }, + 'ja': { + 'test-no-english': u'test-no-english JA' } +} diff --git a/tests/page_tests.py b/tests/page_tests.py index 13289d8..f828c33 100644 --- a/tests/page_tests.py +++ b/tests/page_tests.py @@ -72,11 +72,11 @@ """Test that Link() normalizes namespace names""" for num in self.namespaces: for prefix in self.namespaces[num]: - l = pywikibot.page.Link(prefix+self.titles.keys()[0], + l = pywikibot.page.Link(prefix + self.titles.keys()[0], self.enwiki) self.assertEqual(l.namespace, num) # namespace prefixes are case-insensitive - m = pywikibot.page.Link(prefix.lower()+self.titles.keys()[1], + m = pywikibot.page.Link(prefix.lower() + self.titles.keys()[1], self.enwiki) self.assertEqual(m.namespace, num) @@ -84,7 +84,7 @@ """Test that Link() normalizes titles""" for title in self.titles: for num in (0, 1): - l = pywikibot.page.Link(self.namespaces[num][0]+title) + l = pywikibot.page.Link(self.namespaces[num][0] + title) self.assertEqual(l.title, self.titles[title]) # prefixing name with ":" shouldn't change result m = pywikibot.page.Link(":" + self.namespaces[num][0] + title) diff --git a/tests/pwb/print_locals.py b/tests/pwb/print_locals.py index 1b84ac0..8bb8da2 100644 --- a/tests/pwb/print_locals.py +++ b/tests/pwb/print_locals.py @@ -1,4 +1,4 @@ """docstring""" -for k,v in locals().copy().iteritems(): +for k, v in locals().copy().iteritems(): print repr(k), ":", repr(v) diff --git a/tests/pwb_tests.py b/tests/pwb_tests.py index 506b02f..bd30b33 100644 --- a/tests/pwb_tests.py +++ b/tests/pwb_tests.py @@ -16,8 +16,9 @@ pypath = sys.executable basepath = os.path.split(os.path.split(__file__)[0])[0] -pwbpath = os.path.join(basepath, 'pwb.py') +pwbpath = os.path.join(basepath, 'pwb.py') testbasepath = os.path.join(basepath, 'tests', 'pwb') + class TestPwb(unittest.TestCase): def setUp(self): @@ -35,8 +36,8 @@ test = os.path.join(testbasepath, 'print_locals.py') direct = subprocess.check_output([pypath, test]) - vpwb = subprocess.check_output([pypath, pwbpath, test]) + vpwb = subprocess.check_output([pypath, pwbpath, test]) self.assertEqual(direct, vpwb) -if __name__=="__main__": +if __name__ == "__main__": unittest.main(verbosity=10) diff --git a/tests/site_tests.py b/tests/site_tests.py index 0877653..01457ee 100644 --- a/tests/site_tests.py +++ b/tests/site_tests.py @@ -19,10 +19,12 @@ mainpage = None imagepage = None + class TestSiteObject(unittest.TestCase): """Test cases for Site methods.""" family = "wikipedia" code = "en" + @classmethod def setUpClass(cls): patch_request() @@ -92,21 +94,22 @@ def testNamespaceMethods(self): """Test cases for methods manipulating namespace names""" - builtins = {'Talk': 1, # these should work in any MW wiki - 'User': 2, - 'User talk': 3, - 'Project': 4, - 'Project talk': 5, - 'Image': 6, - 'Image talk': 7, - 'MediaWiki': 8, - 'MediaWiki talk': 9, - 'Template': 10, - 'Template talk': 11, - 'Help': 12, - 'Help talk': 13, - 'Category': 14, - 'Category talk': 15, + builtins = { + 'Talk': 1, # these should work in any MW wiki + 'User': 2, + 'User talk': 3, + 'Project': 4, + 'Project talk': 5, + 'Image': 6, + 'Image talk': 7, + 'MediaWiki': 8, + 'MediaWiki talk': 9, + 'Template': 10, + 'Template talk': 11, + 'Help': 12, + 'Help talk': 13, + 'Category': 14, + 'Category talk': 15, } self.assertTrue(all(mysite.ns_index(b) == builtins[b] for b in builtins)) @@ -150,7 +153,7 @@ self.assertFalse(mysite.has_group("nonexistent_group", True)) except pywikibot.NoUsername: pywikibot.warning( - "Cannot test Site methods for sysop; no sysop account configured.") + "Cannot test Site methods for sysop; no sysop account configured.") for msg in ("1movedto2", "about", "aboutpage", "aboutsite", "accesskey-n-portal"): self.assertTrue(mysite.has_mediawiki_message(msg)) @@ -239,19 +242,19 @@ self.assertTrue(ref in backlinks or ref in embedded) # test embeddedin arguments self.assertTrue(embedded.issuperset( - set(mysite.page_embeddedin(mainpage, filterRedirects=True, - namespaces=[0])))) + set(mysite.page_embeddedin(mainpage, filterRedirects=True, + namespaces=[0])))) self.assertTrue(embedded.issuperset( - set(mysite.page_embeddedin(mainpage, filterRedirects=False, - namespaces=[0])))) + set(mysite.page_embeddedin(mainpage, filterRedirects=False, + namespaces=[0])))) self.assertTrue(embedded.issubset( - set(mysite.page_embeddedin(mainpage, namespaces=[0, 2])))) + set(mysite.page_embeddedin(mainpage, namespaces=[0, 2])))) links = set(mysite.pagelinks(mainpage)) for pl in links: self.assertType(pl, pywikibot.Page) # test links arguments self.assertTrue(links.issuperset( - set(mysite.pagelinks(mainpage, namespaces=[0, 1])))) + set(mysite.pagelinks(mainpage, namespaces=[0, 1])))) for target in mysite.preloadpages( mysite.pagelinks(mainpage, follow_redirects=True, total=5)): @@ -470,7 +473,7 @@ # timestamps should be in descending order timestamps = [block['timestamp'] for block in bl] for t in xrange(1, len(timestamps)): - self.assertTrue(timestamps[t] <= timestamps[t-1]) + self.assertTrue(timestamps[t] <= timestamps[t - 1]) b2 = list(mysite.blocks(total=10, reverse=True)) self.assertTrue(len(b2) <= 10) @@ -481,7 +484,7 @@ # timestamps should be in ascending order timestamps = [block['timestamp'] for block in b2] for t in xrange(1, len(timestamps)): - self.assertTrue(timestamps[t] >= timestamps[t-1]) + self.assertTrue(timestamps[t] >= timestamps[t - 1]) for block in mysite.blocks(starttime="2008-07-01T00:00:01Z", total=5): self.assertType(block, dict) @@ -534,7 +537,7 @@ self.assertTrue(len(iu) <= 10) self.assertTrue(all(isinstance(link, pywikibot.Page) for link in iu)) - for using in mysite.imageusage(imagepage, namespaces=[3,4], total=5): + for using in mysite.imageusage(imagepage, namespaces=[3, 4], total=5): self.assertType(using, pywikibot.Page) self.assertTrue(imagepage in list(using.imagelinks())) for using in mysite.imageusage(imagepage, filterredir=True, total=5): @@ -629,14 +632,14 @@ self.assertRaises(pywikibot.Error, mysite.recentchanges, start="2008-02-03T23:59:59Z", end="2008-02-03T00:00:01Z", reverse=True, total=5) - for change in mysite.recentchanges(namespaces=[6,7], total=5): + for change in mysite.recentchanges(namespaces=[6, 7], total=5): self.assertType(change, dict) self.assertTrue("title" in change and "ns" in change) title = change['title'] self.assertTrue(":" in title) prefix = title[ : title.index(":")] - self.assertTrue(mysite.ns_index(prefix) in [6,7]) - self.assertTrue(change["ns"] in [6,7]) + self.assertTrue(mysite.ns_index(prefix) in [6, 7]) + self.assertTrue(change["ns"] in [6, 7]) if mysite.versionnumber() <= 14: for change in mysite.recentchanges(pagelist=[mainpage, imagepage], total=5): @@ -691,12 +694,12 @@ for hit in mysite.search("common", namespaces=4, total=5): self.assertType(hit, pywikibot.Page) self.assertEqual(hit.namespace(), 4) - for hit in mysite.search("word", namespaces=[5,6,7], total=5): + for hit in mysite.search("word", namespaces=[5, 6, 7], total=5): self.assertType(hit, pywikibot.Page) - self.assertTrue(hit.namespace() in [5,6,7]) + self.assertTrue(hit.namespace() in [5, 6, 7]) for hit in mysite.search("another", namespaces="8|9|10", total=5): self.assertType(hit, pywikibot.Page) - self.assertTrue(hit.namespace() in [8,9,10]) + self.assertTrue(hit.namespace() in [8, 9, 10]) for hit in mysite.search("wiki", namespaces=0, total=10, getredirects=True): self.assertType(hit, pywikibot.Page) @@ -762,7 +765,7 @@ self.assertTrue("title" in contrib) self.assertTrue(contrib["title"].startswith(mysite.namespace(14))) for contrib in mysite.usercontribs(user=mysite.user(), - namespaces=[10,11], total=5): + namespaces=[10, 11], total=5): self.assertType(contrib, dict) self.assertTrue("title" in contrib) self.assertTrue(contrib["ns"] in (10, 11)) @@ -818,14 +821,14 @@ self.assertRaises(pywikibot.Error, mysite.watchlist_revs, start="2008-09-03T23:59:59Z", end="2008-09-03T00:00:01Z", reverse=True, total=5) - for rev in mysite.watchlist_revs(namespaces=[6,7], total=5): + for rev in mysite.watchlist_revs(namespaces=[6, 7], total=5): self.assertType(rev, dict) self.assertTrue("title" in rev and "ns" in rev) title = rev['title'] self.assertTrue(":" in title) prefix = title[ : title.index(":")] - self.assertTrue(mysite.ns_index(prefix) in [6,7]) - self.assertTrue(rev["ns"] in [6,7]) + self.assertTrue(mysite.ns_index(prefix) in [6, 7]) + self.assertTrue(rev["ns"] in [6, 7]) for rev in mysite.watchlist_revs(showMinor=True, total=5): self.assertType(rev, dict) self.assertTrue("minor" in rev) @@ -851,7 +854,7 @@ mysite.login(True) except pywikibot.NoUsername: pywikibot.warning( - "Cannot test Site.deleted_revs; no sysop account configured.") + "Cannot test Site.deleted_revs; no sysop account configured.") return dr = list(mysite.deletedrevs(total=10, page=mainpage)) self.assertTrue(len(dr) <= 10) -- To view, visit https://gerrit.wikimedia.org/r/82566 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I3c2da53919fdaae34dc16dc012ae6589c4d7147a Gerrit-PatchSet: 1 Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Owner: Hazard-SJ <hazard_sj(a)yahoo.com> Gerrit-Reviewer: Legoktm <legoktm.wikipedia(a)gmail.com> Gerrit-Reviewer: jenkins-bot

1 0

[Gerrit] pwb_tests: set PYWIKIBOT2_DIR from base_dir - change (pywikibot/core)
by jenkins-bot (Code Review) 03 Sep '13

03 Sep '13

jenkins-bot has submitted this change and it was merged. Change subject: pwb_tests: set PYWIKIBOT2_DIR from base_dir ...................................................................... pwb_tests: set PYWIKIBOT2_DIR from base_dir If the tests are run from setup.py test without PYWIKIBOT2_DIR set, it will use base_dir='~/.pywikibot'. This is OK for all tests, except when calling pwb.py -- which will assume base_dir is the checkout dir instead, and thus ask for information to create a new user-config.py. This commit makes sure PYWIKIBOT2_DIR is set to base_dir (='~/.pywikibot') before calling pwb.py, which will make pwb.py check that directory instead. Change-Id: I0a277425b4f9df5b64c325b7d2712f4ba007a260 Fixes-Change-Id: Ic8522e1f08c359fff4085c172e696e62ca969a70 Fixes-Change-Id: Iaeaaf1b34d3e2bcacf8308d2c3b74bd8f711ec6a Fixes-Change-Id: If9458fca50f07f08441dbb6e06f78bdbae2065de --- M tests/pwb_tests.py 1 file changed, 10 insertions(+), 4 deletions(-) Approvals: Legoktm: Looks good to me, approved jenkins-bot: Verified diff --git a/tests/pwb_tests.py b/tests/pwb_tests.py index 5a97483..506b02f 100644 --- a/tests/pwb_tests.py +++ b/tests/pwb_tests.py @@ -12,17 +12,23 @@ import unittest +import pywikibot + pypath = sys.executable basepath = os.path.split(os.path.split(__file__)[0])[0] pwbpath = os.path.join(basepath, 'pwb.py') testbasepath = os.path.join(basepath, 'tests', 'pwb') class TestPwb(unittest.TestCase): - @unittest.skip("""Skipping test due to broken Travis run. + def setUp(self): + self.oldenviron = os.environ.copy() + os.environ['PYWIKIBOT2_DIR'] = pywikibot.config.base_dir -Probably the shelling out causes an issue, but we have to investigate this. -See https://gerrit.wikimedia.org/r/#/c/76486/ and - https://gerrit.wikimedia.org/r/#/c/82370/ for details. """) + def tearDown(self): + del os.environ['PYWIKIBOT2_DIR'] + if 'PYWIKIBOT2_DIR' in self.oldenviron: + os.environ['PYWIKIBOT2_DIR'] = self.oldenviron['PYWIKIBOT2_DIR'] + def testScriptEnvironment(self): """Make sure the environment is not contaminated, and is the same as the environment we get when directly running a script.""" -- To view, visit https://gerrit.wikimedia.org/r/82425 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I0a277425b4f9df5b64c325b7d2712f4ba007a260 Gerrit-PatchSet: 1 Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Owner: Merlijn van Deen <valhallasw(a)arctus.nl> Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com> Gerrit-Reviewer: Legoktm <legoktm.wikipedia(a)gmail.com> Gerrit-Reviewer: jenkins-bot

1 0

[Gerrit] Fix 'Disable pwb_tests to prevent Travis build failures' - change (pywikibot/core)
by jenkins-bot (Code Review) 03 Sep '13

03 Sep '13

jenkins-bot has submitted this change and it was merged. Change subject: Fix 'Disable pwb_tests to prevent Travis build failures' ...................................................................... Fix 'Disable pwb_tests to prevent Travis build failures' Now with actual closing bracket. Change-Id: Ic8522e1f08c359fff4085c172e696e62ca969a70 Fixes-Change-Id: Iaeaaf1b34d3e2bcacf8308d2c3b74bd8f711ec6a --- M tests/pwb_tests.py 1 file changed, 1 insertion(+), 1 deletion(-) Approvals: Legoktm: Looks good to me, approved jenkins-bot: Verified diff --git a/tests/pwb_tests.py b/tests/pwb_tests.py index af44203..5a97483 100644 --- a/tests/pwb_tests.py +++ b/tests/pwb_tests.py @@ -22,7 +22,7 @@ Probably the shelling out causes an issue, but we have to investigate this. See https://gerrit.wikimedia.org/r/#/c/76486/ and - https://gerrit.wikimedia.org/r/#/c/82370/ for details. """ + https://gerrit.wikimedia.org/r/#/c/82370/ for details. """) def testScriptEnvironment(self): """Make sure the environment is not contaminated, and is the same as the environment we get when directly running a script.""" -- To view, visit https://gerrit.wikimedia.org/r/82422 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: Ic8522e1f08c359fff4085c172e696e62ca969a70 Gerrit-PatchSet: 1 Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Owner: Merlijn van Deen <valhallasw(a)arctus.nl> Gerrit-Reviewer: Legoktm <legoktm.wikipedia(a)gmail.com> Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl> Gerrit-Reviewer: jenkins-bot

1 0

[Gerrit] Disable pwb_tests to prevent Travis build failures - change (pywikibot/core)
by jenkins-bot (Code Review) 03 Sep '13

03 Sep '13

jenkins-bot has submitted this change and it was merged. Change subject: Disable pwb_tests to prevent Travis build failures ...................................................................... Disable pwb_tests to prevent Travis build failures I'm fairly certain this is a testing issue related to shelling out, or possibly to environment variables. However, the test passes for me, so I think the most reasonable option for now is to disable the test and then to see why Travis fails. Change-Id: Iaeaaf1b34d3e2bcacf8308d2c3b74bd8f711ec6a --- M tests/pwb_tests.py 1 file changed, 5 insertions(+), 0 deletions(-) Approvals: Legoktm: Looks good to me, approved jenkins-bot: Verified diff --git a/tests/pwb_tests.py b/tests/pwb_tests.py index 4cb9759..af44203 100644 --- a/tests/pwb_tests.py +++ b/tests/pwb_tests.py @@ -18,6 +18,11 @@ testbasepath = os.path.join(basepath, 'tests', 'pwb') class TestPwb(unittest.TestCase): + @unittest.skip("""Skipping test due to broken Travis run. + +Probably the shelling out causes an issue, but we have to investigate this. +See https://gerrit.wikimedia.org/r/#/c/76486/ and + https://gerrit.wikimedia.org/r/#/c/82370/ for details. """ def testScriptEnvironment(self): """Make sure the environment is not contaminated, and is the same as the environment we get when directly running a script.""" -- To view, visit https://gerrit.wikimedia.org/r/82370 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: Iaeaaf1b34d3e2bcacf8308d2c3b74bd8f711ec6a Gerrit-PatchSet: 2 Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Owner: Merlijn van Deen <valhallasw(a)arctus.nl> Gerrit-Reviewer: Legoktm <legoktm.wikipedia(a)gmail.com> Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl> Gerrit-Reviewer: jenkins-bot

1 0

[Gerrit] Fix and document parameters that Site.newpages was passing t... - change (pywikibot/core)
by jenkins-bot (Code Review) 03 Sep '13

03 Sep '13

jenkins-bot has submitted this change and it was merged. Change subject: Fix and document parameters that Site.newpages was passing to Site.recentchanges ...................................................................... Fix and document parameters that Site.newpages was passing to Site.recentchanges Also added type hinting to the various parameters. Change-Id: Icbc9f988a6c533788f1d1d9867411eda901017c2 --- M pywikibot/site.py 1 file changed, 22 insertions(+), 1 deletion(-) Approvals: Merlijn van Deen: Looks good to me, approved jenkins-bot: Verified diff --git a/pywikibot/site.py b/pywikibot/site.py index 1b1f1d6..1c0b144 100644 --- a/pywikibot/site.py +++ b/pywikibot/site.py @@ -2242,29 +2242,43 @@ namespaces=None, pagelist=None, changetype=None, showMinor=None, showBot=None, showAnon=None, showRedirects=None, showPatrolled=None, topOnly=False, - step=None, total=None): + step=None, total=None, user=None, excludeuser=None): """Iterate recent changes. @param start: Timestamp to start listing from + @type start: pywikibot.Timestamp @param end: Timestamp to end listing at + @type end: pywikibot.Timestamp @param reverse: if True, start with oldest changes (default: newest) + @type reverse: bool @param pagelist: iterate changes to pages in this list only @param pagelist: list of Pages @param changetype: only iterate changes of this type ("edit" for edits to existing pages, "new" for new pages, "log" for log entries) + @type changetype: basestring @param showMinor: if True, only list minor edits; if False (and not None), only list non-minor edits + @type showMinor: bool @param showBot: if True, only list bot edits; if False (and not None), only list non-bot edits + @type showBot: bool @param showAnon: if True, only list anon edits; if False (and not None), only list non-anon edits + @type showAnon: bool @param showRedirects: if True, only list edits to redirect pages; if False (and not None), only list edits to non-redirect pages + @type showRedirects: bool @param showPatrolled: if True, only list patrolled edits; if False (and not None), only list non-patrolled edits + @type showPatrolled: bool @param topOnly: if True, only list changes that are the latest revision (default False) + @type topOnly: bool + @param user: if not None, only list edits by this user or users + @type user: basestring|list + @param excludeuser: if not None, exclude edits by this user or users + @type excludeuser: basestring|list """ if start and end: @@ -2312,6 +2326,13 @@ rcshow.append(filters[item] and item or ("!" + item)) if rcshow: rcgen.request["rcshow"] = "|".join(rcshow) + + if user: + rcgen.request['rcuser'] = user + + if excludeuser: + rcgen.request['rcexcludeuser'] = excludeuser + return rcgen @deprecate_arg("number", "limit") -- To view, visit https://gerrit.wikimedia.org/r/82220 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: Icbc9f988a6c533788f1d1d9867411eda901017c2 Gerrit-PatchSet: 2 Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Owner: Legoktm <legoktm.wikipedia(a)gmail.com> Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com> Gerrit-Reviewer: Legoktm <legoktm.wikipedia(a)gmail.com> Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl> Gerrit-Reviewer: jenkins-bot

1 0

[Gerrit] Note missing user-config.py before starting generate_user_files - change (pywikibot/core)
by jenkins-bot (Code Review) 03 Sep '13

03 Sep '13

jenkins-bot has submitted this change and it was merged. Change subject: Note missing user-config.py before starting generate_user_files ...................................................................... Note missing user-config.py before starting generate_user_files Change-Id: I8be54f76e6cb9e7352bd1b19d51515b3aa0b1340 --- M pwb.py 1 file changed, 4 insertions(+), 2 deletions(-) Approvals: Ladsgroup: Looks good to me, approved jenkins-bot: Verified diff --git a/pwb.py b/pwb.py index a684c70..cfd9358 100644 --- a/pwb.py +++ b/pwb.py @@ -80,8 +80,10 @@ sys.argv.pop(i) break - -if not os.path.exists(os.path.join(os.environ["PYWIKIBOT2_DIR"], "user-config.py")): +user_config_path = os.path.join(os.environ["PYWIKIBOT2_DIR"], "user-config.py") +if not os.path.exists(user_config_path): + print "NOTE:", user_config_path, "was not found!" + print "Please follow the prompts to create it:" run_python_file('generate_user_files.py', ['generate_user_files.py']) if len(sys.argv) > 1: -- To view, visit https://gerrit.wikimedia.org/r/81681 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I8be54f76e6cb9e7352bd1b19d51515b3aa0b1340 Gerrit-PatchSet: 2 Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Owner: Merlijn van Deen <valhallasw(a)arctus.nl> Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com> Gerrit-Reviewer: Legoktm <legoktm.wikipedia(a)gmail.com> Gerrit-Reviewer: Xqt <info(a)gno.de> Gerrit-Reviewer: jenkins-bot

1 0

[Gerrit] Updated pwb.py to better mirror direct script runs - change (pywikibot/core)
by Xqt (Code Review) 02 Sep '13

02 Sep '13

Xqt has submitted this change and it was merged. Change subject: Updated pwb.py to better mirror direct script runs ...................................................................... Updated pwb.py to better mirror direct script runs Before, execfile() ran the script in environment that pwb.py has, which included os and path imports, and several other variables. This commit uses run_python_file from Ned Batchelors' coverage.py [1], available under the BSD license [2]. See also his blog posts on the subject [3,4]. This commit also adds a test that shells out to compare the locals() for runs through pwb.py as well as to a script directly. After this change, the effects of changeset 76484 can be seen using pwb.py [1] https://bitbucket.org/ned/coveragepy/src/b5abcee50dbe/coverage/execfile.py [2] https://bitbucket.org/ned/coveragepy/src/2c5fb3a8b81cc56d8ad57dd1bd83ef7740… [3] http://nedbatchelder.com/blog/200904/running_a_python_file_as_main.html [4] http://nedbatchelder.com/blog/200905/running_a_python_file_as_main_take_2.h… Change-Id: If9458fca50f07f08441dbb6e06f78bdbae2065de --- M pwb.py A tests/pwb/print_locals.py A tests/pwb_tests.py 3 files changed, 89 insertions(+), 13 deletions(-) Approvals: Xqt: Looks good to me, approved jenkins-bot: Verified diff --git a/pwb.py b/pwb.py index ae732ed..94fef0c 100644 --- a/pwb.py +++ b/pwb.py @@ -12,8 +12,50 @@ # Distributed under the terms of the MIT license. # -import sys + +# The following snippet was developed by Ned Batchelder (and others) +# for coverage.py [1], and is available under the BSD license (see [2]) +# [1] https://bitbucket.org/ned/coveragepy/src/b5abcee50dbe/coverage/execfile.py +# [2] https://bitbucket.org/ned/coveragepy/src/2c5fb3a8b81cc56d8ad57dd1bd83ef7740… + +import imp import os +import sys + + +def run_python_file(filename, args): + """Run a python file as if it were the main program on the command line. + + `filename` is the path to the file to execute, it need not be a .py file. + `args` is the argument array to present as sys.argv, including the first + element representing the file being executed. + + """ + # Create a module to serve as __main__ + old_main_mod = sys.modules['__main__'] + main_mod = imp.new_module('__main__') + sys.modules['__main__'] = main_mod + main_mod.__file__ = filename + main_mod.__builtins__ = sys.modules['__builtin__'] + + # Set sys.argv and the first path element properly. + old_argv = sys.argv + old_path0 = sys.path[0] + sys.argv = args + sys.path[0] = os.path.dirname(filename) + + try: + source = open(filename).read() + exec compile(source, filename, "exec") in main_mod.__dict__ + finally: + # Restore the old __main__ + sys.modules['__main__'] = old_main_mod + + # Restore the old argv and path + sys.argv = old_argv + sys.path[0] = old_path0 + +#### end of snippet if sys.version_info[0] != 2: raise RuntimeError("ERROR: Pywikipediabot only runs under Python 2") @@ -33,21 +75,20 @@ os.environ["PYWIKIBOT2_DIR"] = os.path.split(__file__)[0] if not os.path.exists(os.path.join(os.environ["PYWIKIBOT2_DIR"], "user-config.py")): - execfile('generate_user_files.py') + run_python_file('generate_user_files.py', ['generate_user_files.py']) -sys.argv.pop(0) -if len(sys.argv) > 0: - if not os.path.exists(sys.argv[0]): - testpath = os.path.join(os.path.split(__file__)[0], 'scripts', sys.argv[0]) +if len(sys.argv) > 1: + fn = sys.argv[1] + args = sys.argv[1:] + + if not os.path.exists(fn): + testpath = os.path.join(os.path.split(__file__)[0], 'scripts', fn) if os.path.exists(testpath): - sys.argv[0] = testpath + fn = testpath else: testpath = testpath + '.py' if os.path.exists(testpath): - sys.argv[0] = testpath + fn = testpath else: - raise Exception("%s not found!" % sys.argv[0]) - sys.path.append(os.path.split(sys.argv[0])[0]) - execfile(sys.argv[0]) -else: - sys.argv.append('') + raise Exception("%s not found!" % fn) + run_python_file(fn, args) diff --git a/tests/pwb/print_locals.py b/tests/pwb/print_locals.py new file mode 100644 index 0000000..1b84ac0 --- /dev/null +++ b/tests/pwb/print_locals.py @@ -0,0 +1,4 @@ +"""docstring""" + +for k,v in locals().copy().iteritems(): + print repr(k), ":", repr(v) diff --git a/tests/pwb_tests.py b/tests/pwb_tests.py new file mode 100644 index 0000000..4cb9759 --- /dev/null +++ b/tests/pwb_tests.py @@ -0,0 +1,31 @@ +# -*- coding: utf-8 -*- +# +# (C) Pywikipedia bot team, 2007 +# +# Distributed under the terms of the MIT license. +# +__version__ = '$Id$' + +import os +import sys +import subprocess + +import unittest + +pypath = sys.executable +basepath = os.path.split(os.path.split(__file__)[0])[0] +pwbpath = os.path.join(basepath, 'pwb.py') +testbasepath = os.path.join(basepath, 'tests', 'pwb') + +class TestPwb(unittest.TestCase): + def testScriptEnvironment(self): + """Make sure the environment is not contaminated, and is the same as + the environment we get when directly running a script.""" + test = os.path.join(testbasepath, 'print_locals.py') + + direct = subprocess.check_output([pypath, test]) + vpwb = subprocess.check_output([pypath, pwbpath, test]) + self.assertEqual(direct, vpwb) + +if __name__=="__main__": + unittest.main(verbosity=10) -- To view, visit https://gerrit.wikimedia.org/r/76486 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: If9458fca50f07f08441dbb6e06f78bdbae2065de Gerrit-PatchSet: 6 Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Owner: Merlijn van Deen <valhallasw(a)arctus.nl> Gerrit-Reviewer: DrTrigon <dr.trigon(a)surfeu.ch> Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com> Gerrit-Reviewer: Legoktm <legoktm.wikipedia(a)gmail.com> Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl> Gerrit-Reviewer: Xqt <info(a)gno.de> Gerrit-Reviewer: jenkins-bot

1 0

[Gerrit] Set correct language as source - change (pywikibot/core)
by Xqt (Code Review) 01 Sep '13

01 Sep '13

Xqt has submitted this change and it was merged. Change subject: Set correct language as source ...................................................................... Set correct language as source Patch is basically the same as d90be6a8765. Bug: https://sourceforge.net/p/pywikipediabot/feature-requests/346/ Change-Id: Ic2a6010903a3586803687f350f95ce33fa6abbb2 --- M scripts/claimit.py 1 file changed, 5 insertions(+), 5 deletions(-) Approvals: Xqt: Looks good to me, approved jenkins-bot: Verified diff --git a/scripts/claimit.py b/scripts/claimit.py index 02bdf35..0f50db0 100755 --- a/scripts/claimit.py +++ b/scripts/claimit.py @@ -37,15 +37,15 @@ self.setSource(pywikibot.Site().language()) def setSource(self, lang): - ''' + """ Get the source - ''' + """ page = pywikibot.Page(self.repo, 'Wikidata:List of wikis/python') source_values = json.loads(page.get()) source_values = source_values['wikipedia'] - for lang in source_values: - source_values[lang] = pywikibot.ItemPage(self.repo, - source_values[lang]) + for source_lang in source_values: + source_values[source_lang] = pywikibot.ItemPage(self.repo, + source_values[source_lang]) if lang in source_values: self.source = pywikibot.Claim(self.repo, 'p143') -- To view, visit https://gerrit.wikimedia.org/r/82106 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: Ic2a6010903a3586803687f350f95ce33fa6abbb2 Gerrit-PatchSet: 1 Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Owner: Legoktm <legoktm.wikipedia(a)gmail.com> Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com> Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl> Gerrit-Reviewer: Xqt <info(a)gno.de> Gerrit-Reviewer: jenkins-bot

1 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

Pywikibot-commits September 2013