Pywikibot-commits August 2014

pywikibot-commits@lists.wikimedia.org

1 participants
196 discussions

[Gerrit] page.py: base isEmptyCategory on cat.categoryinfo - change (pywikibot/core)
by jenkins-bot (Code Review) 31 Aug '14

31 Aug '14

jenkins-bot has submitted this change and it was merged. Change subject: page.py: base isEmptyCategory on cat.categoryinfo ...................................................................... page.py: base isEmptyCategory on cat.categoryinfo No change of functionality. Test is based on check of properties rather than try to retrieve content. Tests added. Change-Id: I6070c72882fbcd94a5526a158a9827738386bf51 --- M pywikibot/page.py M tests/page_tests.py 2 files changed, 13 insertions(+), 3 deletions(-) Approvals: John Vandenberg: Looks good to me, approved jenkins-bot: Verified diff --git a/pywikibot/page.py b/pywikibot/page.py index e77b23e..5d8c8f7 100644 --- a/pywikibot/page.py +++ b/pywikibot/page.py @@ -2121,9 +2121,8 @@ def isEmptyCategory(self): """Return True if category has no members (including subcategories).""" - for member in self.site.categorymembers(self, total=1): - return False - return True + ci = self.categoryinfo + return sum(ci[k] for k in ['files', 'pages', 'subcats']) == 0 def isHiddenCategory(self): """Return True if the category is hidden.""" diff --git a/tests/page_tests.py b/tests/page_tests.py index ebc37d6..05d3228 100644 --- a/tests/page_tests.py +++ b/tests/page_tests.py @@ -400,6 +400,17 @@ # def contributingUsers(self): +class TestCategoryObject(PywikibotTestCase): + + def test_isEmptyCategory(self): + """Test if category is empty or not""" + site = pywikibot.Site('en', 'wikipedia') + cat_empty = pywikibot.Category(site, u'Category:foooooo') + cat_not_empty = pywikibot.Category(site, u'Category:Wikipedia categories') + self.assertTrue(cat_empty.isEmptyCategory()) + self.assertFalse(cat_not_empty.isEmptyCategory()) + + if __name__ == '__main__': try: unittest.main() -- To view, visit https://gerrit.wikimedia.org/r/157293 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I6070c72882fbcd94a5526a158a9827738386bf51 Gerrit-PatchSet: 3 Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Owner: Mpaa <mpaa.wiki(a)gmail.com> Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com> Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com> Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl> Gerrit-Reviewer: Mpaa <mpaa.wiki(a)gmail.com> Gerrit-Reviewer: XZise <CommodoreFabianus(a)gmx.de> Gerrit-Reviewer: jenkins-bot <>

1 0

[Gerrit] Wikibase pages can be redirects - change (pywikibot/core)
by jenkins-bot (Code Review) 31 Aug '14

31 Aug '14

jenkins-bot has submitted this change and it was merged. Change subject: Wikibase pages can be redirects ...................................................................... Wikibase pages can be redirects some tests have been added Change-Id: I9898a4a6312ca5c2b2dd6d02d177b0ba4611f052 --- M pywikibot/page.py M tests/wikibase_tests.py 2 files changed, 16 insertions(+), 1 deletion(-) Approvals: John Vandenberg: Looks good to me, approved XZise: Looks good to me, but someone else must approve jenkins-bot: Verified diff --git a/pywikibot/page.py b/pywikibot/page.py index e77b23e..87791be 100644 --- a/pywikibot/page.py +++ b/pywikibot/page.py @@ -2616,7 +2616,6 @@ raise TypeError("site must be a pywikibot.site.DataSite object") Page.__init__(self, site, title, **kwargs) self.repo = self.site - self._isredir = False # Wikibase pages cannot be a redirect def title(self, **kwargs): """ Page title. diff --git a/tests/wikibase_tests.py b/tests/wikibase_tests.py index fafb515..d629e7e 100644 --- a/tests/wikibase_tests.py +++ b/tests/wikibase_tests.py @@ -371,6 +371,22 @@ self.assertRaisesRegexp(pywikibot.NoPage, 'Test page', item.get) +class TestRedirects(PywikibotTestCase): + + """Test redirect and non-redirect items.""" + + def test_normal_item(self): + item = pywikibot.ItemPage(wikidata, 'Q1') + self.assertFalse(item.isRedirectPage()) + self.assertRaises(pywikibot.IsNotRedirectPage, item.getRedirectTarget) + + def test_redirect_item(self): + item = pywikibot.ItemPage(wikidata, 'Q10008448') + target = pywikibot.ItemPage(wikidata, 'Q8422626') + self.assertTrue(item.isRedirectPage()) + self.assertEqual(item.getRedirectTarget(), target) + + class TestPropertyPage(PywikibotTestCase): def test_property_empty_property(self): -- To view, visit https://gerrit.wikimedia.org/r/157626 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I9898a4a6312ca5c2b2dd6d02d177b0ba4611f052 Gerrit-PatchSet: 1 Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Owner: Ricordisamoa <ricordisamoa(a)openmailbox.org> Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com> Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com> Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl> Gerrit-Reviewer: Ricordisamoa <ricordisamoa(a)openmailbox.org> Gerrit-Reviewer: XZise <CommodoreFabianus(a)gmx.de> Gerrit-Reviewer: jenkins-bot <>

1 0

[Gerrit] [IMPROV] Use proper asserts - change (pywikibot/core)
by jenkins-bot (Code Review) 31 Aug '14

31 Aug '14

jenkins-bot has submitted this change and it was merged. Change subject: [IMPROV] Use proper asserts ...................................................................... [IMPROV] Use proper asserts - assert(Not)Equal instead of assert(Not)Equals - assertIn(X, Y) instead of assertTrue(X in Y) - assertNotIn(X, Y) instead of assertTrue(X not in Y) and instead of assertFalse(X in Y) - assertEqual(X, Y) instead of assertEqual(X == Y) Change-Id: I1bbef226a75d6ddc2e536ca94459595b198d81cc --- M tests/http_tests.py M tests/namespace_tests.py M tests/page_tests.py M tests/pagegenerators_tests.py M tests/site_tests.py M tests/wikibase_tests.py M tests/wikidataquery_tests.py M tests/xmlreader_tests.py 8 files changed, 263 insertions(+), 263 deletions(-) Approvals: Nullzero: Looks good to me, approved jenkins-bot: Verified diff --git a/tests/http_tests.py b/tests/http_tests.py index 4a200e5..e139fbb 100644 --- a/tests/http_tests.py +++ b/tests/http_tests.py @@ -19,31 +19,31 @@ def test_get(self): r = http.request(site=None, uri='http://www.wikipedia.org/') self.assertIsInstance(r, str) - self.assertTrue('<html lang="mul"' in r) + self.assertIn('<html lang="mul"', r) def test_request(self): o = threadedhttp.Http() r = o.request('http://www.wikipedia.org/') self.assertIsInstance(r, tuple) self.assertIsInstance(r[0], dict) - self.assertTrue('status' in r[0]) + self.assertIn('status', r[0]) self.assertIsInstance(r[0]['status'], str) - self.assertEquals(r[0]['status'], '200') + self.assertEqual(r[0]['status'], '200') self.assertIsInstance(r[1], str) - self.assertTrue('<html lang="mul"' in r[1]) - self.assertEquals(int(r[0]['content-length']), len(r[1])) + self.assertIn('<html lang="mul"', r[1]) + self.assertEqual(int(r[0]['content-length']), len(r[1])) def test_gzip(self): o = threadedhttp.Http() r = o.request('http://www.wikipedia.org/') - self.assertTrue('-content-encoding' in r[0]) - self.assertEquals(r[0]['-content-encoding'], 'gzip') + self.assertIn('-content-encoding', r[0]) + self.assertEqual(r[0]['-content-encoding'], 'gzip') url = 'https://test.wikidata.org/w/api.php?action=query&meta=siteinfo' r = o.request(url) - self.assertTrue('-content-encoding' in r[0]) - self.assertEquals(r[0]['-content-encoding'], 'gzip') + self.assertIn('-content-encoding', r[0]) + self.assertEqual(r[0]['-content-encoding'], 'gzip') if __name__ == '__main__': diff --git a/tests/namespace_tests.py b/tests/namespace_tests.py index de7b216..b8458e4 100644 --- a/tests/namespace_tests.py +++ b/tests/namespace_tests.py @@ -84,19 +84,19 @@ y = Namespace(id=6, custom_name=u'dummy', canonical_name=u'File', aliases=[u'Image', u'Immagine'], **kwargs) - self.assertEquals(y.id, 6) - self.assertEquals(y.custom_name, u'dummy') - self.assertEquals(y.canonical_name, u'File') + self.assertEqual(y.id, 6) + self.assertEqual(y.custom_name, u'dummy') + self.assertEqual(y.canonical_name, u'File') - self.assertNotEquals(y.custom_name, u'Dummy') - self.assertNotEquals(y.canonical_name, u'file') + self.assertNotEqual(y.custom_name, u'Dummy') + self.assertNotEqual(y.canonical_name, u'file') self.assertIn(u'Image', y.aliases) self.assertIn(u'Immagine', y.aliases) - self.assertEquals(len(y), 4) - self.assertEquals(list(y), ['dummy', u'File', u'Image', u'Immagine']) - self.assertEquals(y.case, u'first-letter') + self.assertEqual(len(y), 4) + self.assertEqual(list(y), ['dummy', u'File', u'Image', u'Immagine']) + self.assertEqual(y.case, u'first-letter') def testNamespaceNameCase(self): """Namespace names are always case-insensitive.""" @@ -115,30 +115,30 @@ def testNamespaceToString(self): ns = Namespace.builtin_namespaces(use_image_name=False) - self.assertEquals(str(ns[0]), ':') - self.assertEquals(str(ns[1]), 'Talk:') - self.assertEquals(str(ns[6]), ':File:') + self.assertEqual(str(ns[0]), ':') + self.assertEqual(str(ns[1]), 'Talk:') + self.assertEqual(str(ns[6]), ':File:') - self.assertEquals(unicode(ns[0]), u':') - self.assertEquals(unicode(ns[1]), u'Talk:') - self.assertEquals(unicode(ns[6]), u':File:') + self.assertEqual(unicode(ns[0]), u':') + self.assertEqual(unicode(ns[1]), u'Talk:') + self.assertEqual(unicode(ns[6]), u':File:') kwargs = {u'case': u'first-letter'} y = Namespace(id=6, custom_name=u'ملف', canonical_name=u'File', aliases=[u'Image', u'Immagine'], **kwargs) if sys.version_info[0] == 2: - self.assertEquals(str(y), ':File:') - self.assertEquals(unicode(y), u':ملف:') + self.assertEqual(str(y), ':File:') + self.assertEqual(unicode(y), u':ملف:') else: - self.assertEquals(str(y), u':ملف:') + self.assertEqual(str(y), u':ملف:') def testNamespaceCompare(self): a = Namespace(id=0, canonical_name=u'') - self.assertEquals(a, 0) - self.assertEquals(a, '') - self.assertEquals(a, None) + self.assertEqual(a, 0) + self.assertEqual(a, '') + self.assertEqual(a, None) x = Namespace(id=6, custom_name=u'dummy', canonical_name=u'File', aliases=[u'Image', u'Immagine']) @@ -147,23 +147,23 @@ z = Namespace(id=7, custom_name=u'dummy', canonical_name=u'File', aliases=[u'Image', u'Immagine']) - self.assertEquals(x, x) - self.assertEquals(x, y) - self.assertNotEquals(x, a) - self.assertNotEquals(x, z) + self.assertEqual(x, x) + self.assertEqual(x, y) + self.assertNotEqual(x, a) + self.assertNotEqual(x, z) - self.assertEquals(x, 6) - self.assertEquals(x, u'dummy') - self.assertEquals(x, u'Dummy') - self.assertEquals(x, u'file') - self.assertEquals(x, u'File') - self.assertEquals(x, u':File') - self.assertEquals(x, u':File:') - self.assertEquals(x, u'File:') - self.assertEquals(x, u'image') - self.assertEquals(x, u'Image') + self.assertEqual(x, 6) + self.assertEqual(x, u'dummy') + self.assertEqual(x, u'Dummy') + self.assertEqual(x, u'file') + self.assertEqual(x, u'File') + self.assertEqual(x, u':File') + self.assertEqual(x, u':File:') + self.assertEqual(x, u'File:') + self.assertEqual(x, u'image') + self.assertEqual(x, u'Image') - self.assertEquals(y, u'ملف') + self.assertEqual(y, u'ملف') # FIXME: Namespace is missing operators required for py3 if sys.version_info[0] > 2: @@ -174,48 +174,48 @@ self.assertTrue(z > x) def testNamespaceNormalizeName(self): - self.assertEquals(Namespace.normalize_name(u'File'), u'File') - self.assertEquals(Namespace.normalize_name(u':File'), u'File') - self.assertEquals(Namespace.normalize_name(u'File:'), u'File') - self.assertEquals(Namespace.normalize_name(u':File:'), u'File') + self.assertEqual(Namespace.normalize_name(u'File'), u'File') + self.assertEqual(Namespace.normalize_name(u':File'), u'File') + self.assertEqual(Namespace.normalize_name(u'File:'), u'File') + self.assertEqual(Namespace.normalize_name(u':File:'), u'File') - self.assertEquals(Namespace.normalize_name(u''), u'') + self.assertEqual(Namespace.normalize_name(u''), u'') - self.assertEquals(Namespace.normalize_name(u':'), False) - self.assertEquals(Namespace.normalize_name(u'::'), False) - self.assertEquals(Namespace.normalize_name(u':::'), False) - self.assertEquals(Namespace.normalize_name(u':File::'), False) - self.assertEquals(Namespace.normalize_name(u'::File:'), False) - self.assertEquals(Namespace.normalize_name(u'::File::'), False) + self.assertEqual(Namespace.normalize_name(u':'), False) + self.assertEqual(Namespace.normalize_name(u'::'), False) + self.assertEqual(Namespace.normalize_name(u':::'), False) + self.assertEqual(Namespace.normalize_name(u':File::'), False) + self.assertEqual(Namespace.normalize_name(u'::File:'), False) + self.assertEqual(Namespace.normalize_name(u'::File::'), False) def test_repr(self): a = Namespace(id=0, canonical_name=u'Foo') s = repr(a) r = "Namespace(id=0, custom_name=%r, canonical_name=%r, aliases=[])" \ % (unicode('Foo'), unicode('Foo')) - self.assertEquals(s, r) + self.assertEqual(s, r) a.info['defaultcontentmodel'] = 'bar' r = {'defaultcontentmodel': 'bar'} - self.assertEquals(a.info, r) + self.assertEqual(a.info, r) s = repr(a) r = "Namespace(id=0, custom_name=%r, canonical_name=%r, aliases=[], defaultcontentmodel='bar')" \ % (unicode('Foo'), unicode('Foo')) - self.assertEquals(s, r) + self.assertEqual(s, r) a.info['case'] = 'upper' r = {'defaultcontentmodel': 'bar', 'case': 'upper'} - self.assertEquals(a.info, r) + self.assertEqual(a.info, r) s = repr(a) r = "Namespace(id=0, custom_name=%r, canonical_name=%r, aliases=[], case='upper', defaultcontentmodel='bar')" \ % (unicode('Foo'), unicode('Foo')) - self.assertEquals(s, r) + self.assertEqual(s, r) b = eval(repr(a)) - self.assertEquals(a, b) - self.assertEquals(a.info, b.info) + self.assertEqual(a, b) + self.assertEqual(a.info, b.info) if __name__ == '__main__': diff --git a/tests/page_tests.py b/tests/page_tests.py index ebc37d6..ec8f6f4 100644 --- a/tests/page_tests.py +++ b/tests/page_tests.py @@ -347,7 +347,7 @@ self.assertIsInstance(p, pywikibot.Link) for p2 in mainpage.interwiki(expand=False): self.assertIsInstance(p2, pywikibot.Link) - self.assertTrue(p2 in iw) + self.assertIn(p2, iw) for p in mainpage.langlinks(): self.assertIsInstance(p, pywikibot.Link) for p in mainpage.imagelinks(): diff --git a/tests/pagegenerators_tests.py b/tests/pagegenerators_tests.py index c564e6a..706ea11 100644 --- a/tests/pagegenerators_tests.py +++ b/tests/pagegenerators_tests.py @@ -43,74 +43,74 @@ self.assertTrue(hasattr(getattr(pagegenerators, obj), '__call__')) def test_module_import(self): - self.assertTrue("pywikibot.pagegenerators" in sys.modules) + self.assertIn("pywikibot.pagegenerators", sys.modules) def test_PagesFromTitlesGenerator(self): self.assertFunction("PagesFromTitlesGenerator") gen = pagegenerators.PagesFromTitlesGenerator(self.titles) - self.assertTrue(len(self.titles) == len(tuple(gen))) + self.assertEqual(len(self.titles), len(tuple(gen))) def test_NamespaceFilterPageGenerator(self): self.assertFunction("NamespaceFilterPageGenerator") gen = pagegenerators.PagesFromTitlesGenerator(self.titles) gen = pagegenerators.NamespaceFilterPageGenerator(gen, 0) - self.assertTrue(len(tuple(gen)) == 3) + self.assertEqual(len(tuple(gen)), 3) gen = pagegenerators.PagesFromTitlesGenerator(self.titles) gen = pagegenerators.NamespaceFilterPageGenerator(gen, 1) - self.assertTrue(len(tuple(gen)) == 4) + self.assertEqual(len(tuple(gen)), 4) gen = pagegenerators.PagesFromTitlesGenerator(self.titles) gen = pagegenerators.NamespaceFilterPageGenerator(gen, 10) - self.assertTrue(len(tuple(gen)) == 6) + self.assertEqual(len(tuple(gen)), 6) gen = pagegenerators.PagesFromTitlesGenerator(self.titles) gen = pagegenerators.NamespaceFilterPageGenerator(gen, (1, 10)) - self.assertTrue(len(tuple(gen)) == 10) + self.assertEqual(len(tuple(gen)), 10) def test_RegexFilterPageGenerator(self): self.assertFunction("RegexFilterPageGenerator") gen = pagegenerators.PagesFromTitlesGenerator(self.titles) gen = pagegenerators.RegexFilterPageGenerator(gen, '/doc') - self.assertTrue(len(tuple(gen)) == 2) + self.assertEqual(len(tuple(gen)), 2) gen = pagegenerators.PagesFromTitlesGenerator(self.titles) gen = pagegenerators.RegexFilterPageGenerator(gen, '/doc', quantifier='none') - self.assertTrue(len(tuple(gen)) == 11) + self.assertEqual(len(tuple(gen)), 11) gen = pagegenerators.PagesFromTitlesGenerator(self.titles) gen = pagegenerators.RegexFilterPageGenerator(gen, ['/doc', '/meta']) - self.assertTrue(len(tuple(gen)) == 4) + self.assertEqual(len(tuple(gen)), 4) gen = pagegenerators.PagesFromTitlesGenerator(self.titles) gen = pagegenerators.RegexFilterPageGenerator(gen, ['/doc', '/meta'], quantifier='none') - self.assertTrue(len(tuple(gen)) == 9) + self.assertEqual(len(tuple(gen)), 9) gen = pagegenerators.PagesFromTitlesGenerator(self.titles) gen = pagegenerators.RegexFilterPageGenerator(gen, ['/doc', '/meta'], quantifier='all') - self.assertTrue(len(tuple(gen)) == 0) + self.assertEqual(len(tuple(gen)), 0) gen = pagegenerators.PagesFromTitlesGenerator(self.titles) gen = pagegenerators.RegexFilterPageGenerator(gen, ['Template', '/meta'], quantifier='all') - self.assertTrue(len(tuple(gen)) == 1) + self.assertEqual(len(tuple(gen)), 1) gen = pagegenerators.PagesFromTitlesGenerator(self.titles) gen = pagegenerators.RegexFilterPageGenerator(gen, ['template', '/meta'], quantifier='any') - self.assertTrue(len(tuple(gen)) == 4) + self.assertEqual(len(tuple(gen)), 4) gen = pagegenerators.PagesFromTitlesGenerator(self.titles, site=self.site) gen = pagegenerators.RegexFilterPageGenerator(gen, ['template', '/meta'], quantifier='any', ignore_namespace=False) - self.assertTrue(len(tuple(gen)) == 6) + self.assertEqual(len(tuple(gen)), 6) gen = pagegenerators.PagesFromTitlesGenerator(self.titles, site=self.site) gen = pagegenerators.RegexFilterPageGenerator(gen, ['template', '/meta'], quantifier='all', ignore_namespace=False) - self.assertTrue(len(tuple(gen)) == 2) + self.assertEqual(len(tuple(gen)), 2) gen = pagegenerators.PagesFromTitlesGenerator(self.titles, site=self.site) gen = pagegenerators.RegexFilterPageGenerator(gen, ['template', '/meta'], quantifier='none', ignore_namespace=False) - self.assertTrue(len(tuple(gen)) == 7) + self.assertEqual(len(tuple(gen)), 7) def test_RegexBodyFilterPageGenerator(self): self.assertFunction("RegexBodyFilterPageGenerator") @@ -121,12 +121,12 @@ p.text = u"This is the content of %s as a sample" % p.title() pages.append(p) gen = pagegenerators.RegexBodyFilterPageGenerator(iter(pages), '/doc') - self.assertTrue(len(tuple(gen)) == 2) + self.assertEqual(len(tuple(gen)), 2) gen = pagegenerators.RegexBodyFilterPageGenerator(iter(pages), 'This') - self.assertTrue(len(tuple(gen)) == 13) + self.assertEqual(len(tuple(gen)), 13) gen = pagegenerators.RegexBodyFilterPageGenerator(iter(pages), 'talk', quantifier='none') - self.assertTrue(len(tuple(gen)) == 9) + self.assertEqual(len(tuple(gen)), 9) if __name__ == "__main__": try: diff --git a/tests/site_tests.py b/tests/site_tests.py index 3477872..bac24c9 100644 --- a/tests/site_tests.py +++ b/tests/site_tests.py @@ -80,14 +80,14 @@ """Test cases for languages() and related methods""" langs = mysite.languages() self.assertIsInstance(langs, list) - self.assertTrue(mysite.code in langs) + self.assertIn(mysite.code, langs) mysite.family.obsolete ipf = mysite.interwiki_putfirst() if ipf: # Not all languages use this self.assertIsInstance(ipf, list) for item in mysite.validLanguageLinks(): - self.assertTrue(item in langs, item) + self.assertIn(item, langs) def testNamespaceMethods(self): """Test cases for methods manipulating namespace names.""" @@ -134,25 +134,25 @@ def testNamespaceCase(self): site = pywikibot.Site('en', 'wiktionary') main_namespace = site.namespaces()[0] - self.assertEquals(main_namespace.case, 'case-sensitive') + self.assertEqual(main_namespace.case, 'case-sensitive') user_namespace = site.namespaces()[2] - self.assertEquals(user_namespace.case, 'first-letter') + self.assertEqual(user_namespace.case, 'first-letter') def testNamespaceAliases(self): site = pywikibot.Site('nn', 'wikipedia') namespaces = site.namespaces() image_namespace = namespaces[6] - self.assertEquals(image_namespace.custom_name, 'Fil') - self.assertEquals(image_namespace.canonical_name, 'File') - self.assertEquals(str(image_namespace), ':File:') - self.assertEquals(unicode(image_namespace), ':Fil:') - self.assertEquals(image_namespace.aliases, ['Image']) - self.assertEquals(len(image_namespace), 3) + self.assertEqual(image_namespace.custom_name, 'Fil') + self.assertEqual(image_namespace.canonical_name, 'File') + self.assertEqual(str(image_namespace), ':File:') + self.assertEqual(unicode(image_namespace), ':Fil:') + self.assertEqual(image_namespace.aliases, ['Image']) + self.assertEqual(len(image_namespace), 3) - self.assertEquals(len(namespaces[1].aliases), 0) - self.assertEquals(len(namespaces[4].aliases), 1) - self.assertEquals(namespaces[4].aliases[0], 'WP') + self.assertEqual(len(namespaces[1].aliases), 0) + self.assertEqual(len(namespaces[4].aliases), 1) + self.assertEqual(namespaces[4].aliases[0], 'WP') self.assertIn('WP', namespaces[4]) def testApiMethods(self): @@ -194,7 +194,7 @@ # Load all messages and check that '*' is not a valid key. self.assertIsInstance(mysite.mediawiki_messages('*'), dict) self.assertTrue(len(mysite.mediawiki_messages(['*'])) > 10) - self.assertFalse('*' in mysite.mediawiki_messages(['*'])) + self.assertNotIn('*', mysite.mediawiki_messages(['*'])) self.assertIsInstance(mysite.getcurrenttime(), pywikibot.Timestamp) ts = mysite.getcurrenttimestamp() @@ -278,10 +278,10 @@ self.assertTrue(embedded.issubset(refs)) for bl in backlinks: self.assertIsInstance(bl, pywikibot.Page) - self.assertTrue(bl in refs) + self.assertIn(bl, refs) for ei in embedded: self.assertIsInstance(ei, pywikibot.Page) - self.assertTrue(ei in refs) + self.assertIn(ei, refs) for ref in refs: self.assertTrue(ref in backlinks or ref in embedded) # test embeddedin arguments @@ -382,13 +382,13 @@ for page in mysite.allpages(protect_type="edit", total=5): self.assertIsInstance(page, pywikibot.Page) self.assertTrue(mysite.page_exists(page)) - self.assertTrue("edit" in page._protection) + self.assertIn("edit", page._protection) for page in mysite.allpages(protect_type="edit", protect_level="sysop", total=5): self.assertIsInstance(page, pywikibot.Page) self.assertTrue(mysite.page_exists(page)) - self.assertTrue("edit" in page._protection) - self.assertTrue("sysop" in page._protection["edit"]) + self.assertIn("edit", page._protection) + self.assertIn("sysop", page._protection["edit"]) def testAllLinks(self): """Test the site.alllinks() method""" @@ -439,27 +439,27 @@ self.assertTrue(len(au) <= 10) for user in au: self.assertIsInstance(user, dict) - self.assertTrue("name" in user) - self.assertTrue("editcount" in user) - self.assertTrue("registration" in user) + self.assertIn("name", user) + self.assertIn("editcount", user) + self.assertIn("registration", user) for user in mysite.allusers(start="B", total=5): self.assertIsInstance(user, dict) - self.assertTrue("name" in user) + self.assertIn("name", user) self.assertTrue(user["name"] >= "B") - self.assertTrue("editcount" in user) - self.assertTrue("registration" in user) + self.assertIn("editcount", user) + self.assertIn("registration", user) for user in mysite.allusers(prefix="C", total=5): self.assertIsInstance(user, dict) - self.assertTrue("name" in user) + self.assertIn("name", user) self.assertTrue(user["name"].startswith("C")) - self.assertTrue("editcount" in user) - self.assertTrue("registration" in user) + self.assertIn("editcount", user) + self.assertIn("registration", user) for user in mysite.allusers(prefix="D", group="sysop", total=5): self.assertIsInstance(user, dict) - self.assertTrue("name" in user) + self.assertIn("name", user) self.assertTrue(user["name"].startswith("D")) - self.assertTrue("editcount" in user) - self.assertTrue("registration" in user) + self.assertIn("editcount", user) + self.assertIn("registration", user) self.assertTrue("groups" in user and "sysop" in user["groups"]) def testAllImages(self): @@ -498,7 +498,7 @@ for block in bl: self.assertIsInstance(block, dict) for prop in props: - self.assertTrue(prop in block) + self.assertIn(prop, block) # timestamps should be in descending order timestamps = [block['timestamp'] for block in bl] for t in range(1, len(timestamps)): @@ -509,7 +509,7 @@ for block in b2: self.assertIsInstance(block, dict) for prop in props: - self.assertTrue(prop in block) + self.assertIn(prop, block) # timestamps should be in ascending order timestamps = [block['timestamp'] for block in b2] for t in range(1, len(timestamps)): @@ -518,23 +518,23 @@ for block in mysite.blocks(starttime="2008-07-01T00:00:01Z", total=5): self.assertIsInstance(block, dict) for prop in props: - self.assertTrue(prop in block) + self.assertIn(prop, block) for block in mysite.blocks(endtime="2008-07-31T23:59:59Z", total=5): self.assertIsInstance(block, dict) for prop in props: - self.assertTrue(prop in block) + self.assertIn(prop, block) for block in mysite.blocks(starttime="2008-08-02T00:00:01Z", endtime="2008-08-02T23:59:59Z", reverse=True, total=5): self.assertIsInstance(block, dict) for prop in props: - self.assertTrue(prop in block) + self.assertIn(prop, block) for block in mysite.blocks(starttime="2008-08-03T23:59:59Z", endtime="2008-08-03T00:00:01Z", total=5): self.assertIsInstance(block, dict) for prop in props: - self.assertTrue(prop in block) + self.assertIn(prop, block) # starttime earlier than endtime self.assertRaises(pywikibot.Error, mysite.blocks, starttime="2008-08-03T00:00:01Z", @@ -556,7 +556,7 @@ for link in eu)) for link in mysite.exturlusage(url, namespaces=[2, 3], total=5): self.assertIsInstance(link, pywikibot.Page) - self.assertTrue(link.namespace() in (2, 3)) + self.assertIn(link.namespace(), (2, 3)) def testImageusage(self): """Test the site.imageusage() method""" @@ -566,7 +566,7 @@ for link in iu)) for using in mysite.imageusage(imagepage, namespaces=[3, 4], total=5): self.assertIsInstance(using, pywikibot.Page) - self.assertTrue(imagepage in list(using.imagelinks())) + self.assertIn(imagepage, list(using.imagelinks())) for using in mysite.imageusage(imagepage, filterredir=True, total=5): self.assertIsInstance(using, pywikibot.Page) self.assertTrue(using.isRedirectPage()) @@ -585,9 +585,9 @@ for entry in mysite.logevents(logtype=typ, total=3): self.assertEqual(entry.type(), typ) for entry in mysite.logevents(page=mainpage, total=3): - self.assertTrue(entry.title().title() == mainpage.title()) + self.assertEqual(entry.title().title(), mainpage.title()) for entry in mysite.logevents(user=mysite.user(), total=3): - self.assertTrue(entry.user() == mysite.user()) + self.assertEqual(entry.user(), mysite.user()) for entry in mysite.logevents(start="2008-09-01T00:00:01Z", total=5): self.assertIsInstance(entry, pywikibot.logentries.LogEntry) self.assertTrue(str(entry.timestamp()) <= "2008-09-01T00:00:01Z") @@ -661,52 +661,52 @@ self.assertIsInstance(change, dict) self.assertTrue("title" in change and "ns" in change) title = change['title'] - self.assertTrue(":" in title) + self.assertIn(":", title) prefix = title[:title.index(":")] - self.assertTrue(mysite.ns_index(prefix) in [6, 7]) - self.assertTrue(change["ns"] in [6, 7]) + self.assertIn(mysite.ns_index(prefix), [6, 7]) + self.assertIn(change["ns"], [6, 7]) if LV(mysite.version()) <= LV("1.14"): for change in mysite.recentchanges(pagelist=[mainpage, imagepage], total=5): self.assertIsInstance(change, dict) - self.assertTrue("title" in change) - self.assertTrue(change["title"] in (mainpage.title(), - imagepage.title())) + self.assertIn("title", change) + self.assertIn(change["title"], (mainpage.title(), + imagepage.title())) for typ in ("edit", "new", "log"): for change in mysite.recentchanges(changetype=typ, total=5): self.assertIsInstance(change, dict) - self.assertTrue("type" in change) + self.assertIn("type", change) self.assertEqual(change["type"], typ) for change in mysite.recentchanges(showMinor=True, total=5): self.assertIsInstance(change, dict) - self.assertTrue("minor" in change) + self.assertIn("minor", change) for change in mysite.recentchanges(showMinor=False, total=5): self.assertIsInstance(change, dict) - self.assertTrue("minor" not in change) + self.assertNotIn("minor", change) for change in mysite.recentchanges(showBot=True, total=5): self.assertIsInstance(change, dict) - self.assertTrue("bot" in change) + self.assertIn("bot", change) for change in mysite.recentchanges(showBot=False, total=5): self.assertIsInstance(change, dict) - self.assertTrue("bot" not in change) + self.assertNotIn("bot", change) for change in mysite.recentchanges(showAnon=True, total=5): self.assertIsInstance(change, dict) for change in mysite.recentchanges(showAnon=False, total=5): self.assertIsInstance(change, dict) for change in mysite.recentchanges(showRedirects=True, total=5): self.assertIsInstance(change, dict) - self.assertTrue("redirect" in change) + self.assertIn("redirect", change) for change in mysite.recentchanges(showRedirects=False, total=5): self.assertIsInstance(change, dict) - self.assertTrue("redirect" not in change) + self.assertNotIn("redirect", change) for change in mysite.recentchanges(showPatrolled=True, total=5): self.assertIsInstance(change, dict) if mysite.has_right('patrol'): - self.assertTrue("patrolled" in change) + self.assertIn("patrolled", change) for change in mysite.recentchanges(showPatrolled=False, total=5): self.assertIsInstance(change, dict) if mysite.has_right('patrol'): - self.assertTrue("patrolled" not in change) + self.assertNotIn("patrolled", change) def testSearch(self): """Test the site.search() method""" @@ -721,10 +721,10 @@ self.assertEqual(hit.namespace(), 4) for hit in mysite.search("word", namespaces=[5, 6, 7], total=5): self.assertIsInstance(hit, pywikibot.Page) - self.assertTrue(hit.namespace() in [5, 6, 7]) + self.assertIn(hit.namespace(), [5, 6, 7]) for hit in mysite.search("another", namespaces="8|9|10", total=5): self.assertIsInstance(hit, pywikibot.Page) - self.assertTrue(hit.namespace() in [8, 9, 10]) + self.assertIn(hit.namespace(), [8, 9, 10]) for hit in mysite.search("wiki", namespaces=0, total=10, getredirects=True): self.assertIsInstance(hit, pywikibot.Page) @@ -746,7 +746,7 @@ for contrib in mysite.usercontribs(userprefix="John", total=5): self.assertIsInstance(contrib, dict) for key in ("user", "title", "ns", "pageid", "revid"): - self.assertTrue(key in contrib) + self.assertIn(key, contrib) self.assertTrue(contrib["user"].startswith("John")) for contrib in mysite.usercontribs(userprefix="Jane", start="2008-10-06T01:02:03Z", @@ -790,21 +790,21 @@ for contrib in mysite.usercontribs(user=mysite.user(), namespaces=14, total=5): self.assertIsInstance(contrib, dict) - self.assertTrue("title" in contrib) + self.assertIn("title", contrib) self.assertTrue(contrib["title"].startswith(mysite.namespace(14))) for contrib in mysite.usercontribs(user=mysite.user(), namespaces=[10, 11], total=5): self.assertIsInstance(contrib, dict) - self.assertTrue("title" in contrib) - self.assertTrue(contrib["ns"] in (10, 11)) + self.assertIn("title", contrib) + self.assertIn(contrib["ns"], (10, 11)) for contrib in mysite.usercontribs(user=mysite.user(), showMinor=True, total=5): self.assertIsInstance(contrib, dict) - self.assertTrue("minor" in contrib) + self.assertIn("minor", contrib) for contrib in mysite.usercontribs(user=mysite.user(), showMinor=False, total=5): self.assertIsInstance(contrib, dict) - self.assertTrue("minor" not in contrib) + self.assertNotIn("minor", contrib) def testWatchlistrevs(self): """Test the site.watchlist_revs() method""" @@ -852,22 +852,22 @@ self.assertIsInstance(rev, dict) self.assertTrue("title" in rev and "ns" in rev) title = rev['title'] - self.assertTrue(":" in title) + self.assertIn(":", title) prefix = title[:title.index(":")] - self.assertTrue(mysite.ns_index(prefix) in [6, 7]) - self.assertTrue(rev["ns"] in [6, 7]) + self.assertIn(mysite.ns_index(prefix), [6, 7]) + self.assertIn(rev["ns"], [6, 7]) for rev in mysite.watchlist_revs(showMinor=True, total=5): self.assertIsInstance(rev, dict) - self.assertTrue("minor" in rev) + self.assertIn("minor", rev) for rev in mysite.watchlist_revs(showMinor=False, total=5): self.assertIsInstance(rev, dict) - self.assertTrue("minor" not in rev) + self.assertNotIn("minor", rev) for rev in mysite.watchlist_revs(showBot=True, total=5): self.assertIsInstance(rev, dict) - self.assertTrue("bot" in rev) + self.assertIn("bot", rev) for rev in mysite.watchlist_revs(showBot=False, total=5): self.assertIsInstance(rev, dict) - self.assertTrue("bot" not in rev) + self.assertNotIn("bot", rev) for rev in mysite.watchlist_revs(showAnon=True, total=5): self.assertIsInstance(rev, dict) for rev in mysite.watchlist_revs(showAnon=False, total=5): @@ -954,7 +954,7 @@ self.assertTrue(rndpage.isRedirectPage()) for rndpage in mysite.randompages(total=5, namespaces=[6, 7]): self.assertIsInstance(rndpage, pywikibot.Page) - self.assertTrue(rndpage.namespace() in [6, 7]) + self.assertIn(rndpage.namespace(), [6, 7]) # TODO: test newimages, longpages, shortpages, ancientpages, unwatchedpages # and the other following methods in site.py @@ -1018,16 +1018,16 @@ return False self.assertIsInstance(mysite.siteinfo.get('restrictions'), dict) - self.assertTrue('restrictions' in mysite.siteinfo) + self.assertIn('restrictions', mysite.siteinfo) # the following line only works in 1.23+ self.assertTrue(mysite.siteinfo.is_recognised('restrictions')) del mysite.siteinfo._cache['restrictions'] self.assertIsInstance(mysite.siteinfo.get('restrictions', cache=False), dict) - self.assertFalse('restrictions' in mysite.siteinfo) + self.assertNotIn('restrictions', mysite.siteinfo) not_exists = 'this-property-does-not-exist' self.assertRaises(KeyError, mysite.siteinfo.__getitem__, not_exists) - self.assertFalse(not_exists in mysite.siteinfo) + self.assertNotIn(not_exists, mysite.siteinfo) self.assertEqual(len(mysite.siteinfo.get(not_exists)), 0) self.assertFalse(entered_loop(mysite.siteinfo.get(not_exists))) self.assertFalse(entered_loop(mysite.siteinfo.get(not_exists).iteritems())) @@ -1063,7 +1063,7 @@ self.mysite.loadrevisions(self.mainpage, total=15) self.assertTrue(hasattr(self.mainpage, "_revid")) self.assertTrue(hasattr(self.mainpage, "_revisions")) - self.assertTrue(self.mainpage._revid in self.mainpage._revisions) + self.assertIn(self.mainpage._revid, self.mainpage._revisions) self.assertEqual(len(self.mainpage._revisions), 15) self.assertEqual(self.mainpage._text, None) @@ -1082,10 +1082,10 @@ self.assertTrue(all(rev in self.mainpage._revisions for rev in [139994, 139995])) # revids as int self.mysite.loadrevisions(self.mainpage, revids=140000) - self.assertTrue(140000 in self.mainpage._revisions) + self.assertIn(140000, self.mainpage._revisions) # revids as str self.mysite.loadrevisions(self.mainpage, revids='140001') - self.assertTrue(140001 in self.mainpage._revisions) + self.assertIn(140001, self.mainpage._revisions) # revids belonging to a different page raises Exception self.assertRaises(pywikibot.Error, self.mysite.loadrevisions, self.mainpage, revids=130000) @@ -1174,7 +1174,7 @@ # which forwards interwikis to wikipedia ll = next(self.site.pagelanglinks(self.mainpage)) self.assertIsInstance(ll, pywikibot.Link) - self.assertEquals(ll.site.family.name, 'wikipedia') + self.assertEqual(ll.site.family.name, 'wikipedia') if __name__ == '__main__': diff --git a/tests/wikibase_tests.py b/tests/wikibase_tests.py index 8f33435..fafb515 100644 --- a/tests/wikibase_tests.py +++ b/tests/wikibase_tests.py @@ -46,10 +46,10 @@ self.assertIsInstance(item, pywikibot.ItemPage) self.assertEqual(item.getID(), 'Q5296') self.assertEqual(item.title(), 'Q5296') - self.assertTrue('en' in item.labels) + self.assertIn('en', item.labels) self.assertTrue(item.labels['en'].lower().endswith('main page')) - self.assertTrue('en' in item.aliases) - self.assertTrue('HomePage' in item.aliases['en']) + self.assertIn('en', item.aliases) + self.assertIn('HomePage', item.aliases['en']) self.assertEqual(item.namespace(), 0) item2 = pywikibot.ItemPage(repo, 'q5296') self.assertEqual(item2.getID(), 'Q5296') @@ -160,30 +160,30 @@ """ def test_item_normal(self): item = pywikibot.ItemPage(wikidata, 'Q60') - self.assertEquals(item._link._title, 'Q60') - self.assertEquals(item.id, 'Q60') - self.assertEquals(hasattr(item, '_title'), False) - self.assertEquals(hasattr(item, '_site'), False) - self.assertEquals(item.title(), 'Q60') - self.assertEquals(item.getID(), 'Q60') - self.assertEquals(item.getID(numeric=True), 60) - self.assertEquals(hasattr(item, '_content'), False) + self.assertEqual(item._link._title, 'Q60') + self.assertEqual(item.id, 'Q60') + self.assertEqual(hasattr(item, '_title'), False) + self.assertEqual(hasattr(item, '_site'), False) + self.assertEqual(item.title(), 'Q60') + self.assertEqual(item.getID(), 'Q60') + self.assertEqual(item.getID(numeric=True), 60) + self.assertEqual(hasattr(item, '_content'), False) item.get() - self.assertEquals(hasattr(item, '_content'), True) + self.assertEqual(hasattr(item, '_content'), True) def test_load_item_set_id(self): """Test setting item.id attribute on empty item.""" item = pywikibot.ItemPage(wikidata, '-1') - self.assertEquals(item._link._title, '-1') + self.assertEqual(item._link._title, '-1') item.id = 'Q60' - self.assertEquals(hasattr(item, '_content'), False) - self.assertEquals(item.getID(), 'Q60') - self.assertEquals(hasattr(item, '_content'), False) + self.assertEqual(hasattr(item, '_content'), False) + self.assertEqual(item.getID(), 'Q60') + self.assertEqual(hasattr(item, '_content'), False) item.get() - self.assertEquals(hasattr(item, '_content'), True) - self.assertTrue('en' in item.labels) - self.assertEquals(item.labels['en'], 'New York City') - self.assertEquals(item.title(), 'Q60') + self.assertEqual(hasattr(item, '_content'), True) + self.assertIn('en', item.labels) + self.assertEqual(item.labels['en'], 'New York City') + self.assertEqual(item.title(), 'Q60') def test_reuse_item_set_id(self): """ @@ -195,19 +195,19 @@ """ item = pywikibot.ItemPage(wikidata, 'Q60') item.get() - self.assertEquals(item.labels['en'], 'New York City') + self.assertEqual(item.labels['en'], 'New York City') # When the id attribute is modified, the ItemPage goes into # an inconsistent state. item.id = 'Q5296' # The title is updated correctly - self.assertEquals(item.title(), 'Q5296') + self.assertEqual(item.title(), 'Q5296') # This del has no effect on the test; it is here to demonstrate that # it doesnt help to clear this piece of saved state. del item._content # The labels are not updated; assertion showing undesirable behaviour: - self.assertEquals(item.labels['en'], 'New York City') + self.assertEqual(item.labels['en'], 'New York City') # TODO: This is the assertion that this test should be using: # self.assertTrue(item.labels['en'].lower().endswith('main page')) @@ -224,13 +224,13 @@ ucfirst_title = title[0].upper() + title[1:] else: ucfirst_title = title - self.assertEquals(item._link._title, ucfirst_title) - self.assertEquals(item.id, title.upper()) - self.assertEquals(item.title(), title.upper()) - self.assertEquals(hasattr(item, '_content'), False) + self.assertEqual(item._link._title, ucfirst_title) + self.assertEqual(item.id, title.upper()) + self.assertEqual(item.title(), title.upper()) + self.assertEqual(hasattr(item, '_content'), False) self.assertRaises(exception, item.get) - self.assertEquals(hasattr(item, '_content'), False) - self.assertEquals(item.title(), title.upper()) + self.assertEqual(hasattr(item, '_content'), False) + self.assertEqual(item.title(), title.upper()) check('', KeyError) @@ -240,57 +240,57 @@ def test_item_untrimmed_title(self): # spaces in the title cause an error item = pywikibot.ItemPage(wikidata, ' Q60 ') - self.assertEquals(item._link._title, 'Q60') - self.assertEquals(item.title(), ' Q60 ') + self.assertEqual(item._link._title, 'Q60') + self.assertEqual(item.title(), ' Q60 ') self.assertRaises(APIError, item.get) def test_item_missing(self): # this item is deleted item = pywikibot.ItemPage(wikidata, 'Q404') - self.assertEquals(item._link._title, 'Q404') - self.assertEquals(item.title(), 'Q404') - self.assertEquals(hasattr(item, '_content'), False) - self.assertEquals(item.id, 'Q404') - self.assertEquals(item.getID(), 'Q404') - self.assertEquals(item.getID(numeric=True), 404) - self.assertEquals(hasattr(item, '_content'), False) + self.assertEqual(item._link._title, 'Q404') + self.assertEqual(item.title(), 'Q404') + self.assertEqual(hasattr(item, '_content'), False) + self.assertEqual(item.id, 'Q404') + self.assertEqual(item.getID(), 'Q404') + self.assertEqual(item.getID(numeric=True), 404) + self.assertEqual(hasattr(item, '_content'), False) self.assertRaises(pywikibot.NoPage, item.get) - self.assertEquals(hasattr(item, '_content'), True) - self.assertEquals(item._link._title, 'Q404') - self.assertEquals(item.title(), 'Q404') - self.assertEquals(item.exists(), False) + self.assertEqual(hasattr(item, '_content'), True) + self.assertEqual(item._link._title, 'Q404') + self.assertEqual(item.title(), 'Q404') + self.assertEqual(item.exists(), False) def test_fromPage_noprops(self): page = pywikibot.Page(pywikibot.page.Link("New York City", site)) item = pywikibot.ItemPage.fromPage(page) - self.assertEquals(item._link._title, 'Null') # not good - self.assertEquals(hasattr(item, 'id'), False) - self.assertEquals(hasattr(item, '_content'), False) - self.assertEquals(item.title(), 'Q60') - self.assertEquals(hasattr(item, '_content'), True) - self.assertEquals(item.id, 'Q60') - self.assertEquals(item.getID(), 'Q60') - self.assertEquals(item.getID(numeric=True), 60) + self.assertEqual(item._link._title, 'Null') # not good + self.assertEqual(hasattr(item, 'id'), False) + self.assertEqual(hasattr(item, '_content'), False) + self.assertEqual(item.title(), 'Q60') + self.assertEqual(hasattr(item, '_content'), True) + self.assertEqual(item.id, 'Q60') + self.assertEqual(item.getID(), 'Q60') + self.assertEqual(item.getID(numeric=True), 60) item.get() - self.assertEquals(item.exists(), True) + self.assertEqual(item.exists(), True) def test_fromPage_props(self): page = pywikibot.Page(pywikibot.page.Link("New York City", site)) # fetch page properties page.properties() item = pywikibot.ItemPage.fromPage(page) - self.assertEquals(item._link._title, 'Q60') - self.assertEquals(item.id, 'Q60') - self.assertEquals(hasattr(item, '_content'), False) - self.assertEquals(item.title(), 'Q60') - self.assertEquals(hasattr(item, '_content'), False) - self.assertEquals(item.id, 'Q60') - self.assertEquals(item.getID(), 'Q60') - self.assertEquals(item.getID(numeric=True), 60) - self.assertEquals(hasattr(item, '_content'), False) + self.assertEqual(item._link._title, 'Q60') + self.assertEqual(item.id, 'Q60') + self.assertEqual(hasattr(item, '_content'), False) + self.assertEqual(item.title(), 'Q60') + self.assertEqual(hasattr(item, '_content'), False) + self.assertEqual(item.id, 'Q60') + self.assertEqual(item.getID(), 'Q60') + self.assertEqual(item.getID(numeric=True), 60) + self.assertEqual(hasattr(item, '_content'), False) item.get() - self.assertEquals(hasattr(item, '_content'), True) - self.assertEquals(item.exists(), True) + self.assertEqual(hasattr(item, '_content'), True) + self.assertEqual(item.exists(), True) def test_fromPage_invalid_title(self): page = pywikibot.Page(pywikibot.page.Link("[]", site)) @@ -313,15 +313,15 @@ page.properties() item = pywikibot.ItemPage.fromPage(page) - self.assertEquals(hasattr(item, 'id'), False) - self.assertEquals(hasattr(item, '_title'), True) - self.assertEquals(hasattr(item, '_site'), True) - self.assertEquals(hasattr(item, '_content'), False) + self.assertEqual(hasattr(item, 'id'), False) + self.assertEqual(hasattr(item, '_title'), True) + self.assertEqual(hasattr(item, '_site'), True) + self.assertEqual(hasattr(item, '_content'), False) - self.assertEquals(item._link._title, 'Null') + self.assertEqual(item._link._title, 'Null') # the method 'exists' does not raise an exception if method == 'exists': - self.assertEquals(item.exists(), False) + self.assertEqual(item.exists(), False) else: self.assertRaises(pywikibot.NoPage, getattr(item, method)) @@ -333,11 +333,11 @@ # local data is modified in this scenario. This case is # separately tested in test_fromPage_missing_lazy. if link.title != 'Test page': - self.assertEquals(item._link._title, '-1') + self.assertEqual(item._link._title, '-1') - self.assertEquals(hasattr(item, '_content'), True) + self.assertEqual(hasattr(item, '_content'), True) - self.assertEquals(item.exists(), False) + self.assertEqual(item.exists(), False) def test_fromPage_redirect(self): # this is a redirect, and should not have a wikidata item @@ -378,29 +378,29 @@ def test_globe_coordinate(self): property_page = pywikibot.PropertyPage(wikidata, 'P625') - self.assertEquals(property_page.type, 'globe-coordinate') - self.assertEquals(property_page.getType(), 'globecoordinate') + self.assertEqual(property_page.type, 'globe-coordinate') + self.assertEqual(property_page.getType(), 'globecoordinate') claim = pywikibot.Claim(wikidata, 'P625') - self.assertEquals(claim.type, 'globe-coordinate') - self.assertEquals(claim.getType(), 'globecoordinate') + self.assertEqual(claim.type, 'globe-coordinate') + self.assertEqual(claim.getType(), 'globecoordinate') class TestClaimSetValue(PywikibotTestCase): def test_set_website(self): claim = pywikibot.Claim(wikidata, 'P856') - self.assertEquals(claim.type, 'url') + self.assertEqual(claim.type, 'url') claim.setTarget('https://en.wikipedia.org/') - self.assertEquals(claim.target, 'https://en.wikipedia.org/') + self.assertEqual(claim.target, 'https://en.wikipedia.org/') def test_set_date(self): claim = pywikibot.Claim(wikidata, 'P569') - self.assertEquals(claim.type, 'time') + self.assertEqual(claim.type, 'time') claim.setTarget(pywikibot.WbTime(year=2001, month=1, day=1, site=wikidata)) - self.assertEquals(claim.target.year, 2001) - self.assertEquals(claim.target.month, 1) - self.assertEquals(claim.target.day, 1) + self.assertEqual(claim.target.year, 2001) + self.assertEqual(claim.target.month, 1) + self.assertEqual(claim.target.day, 1) def test_set_incorrect_target_value(self): claim = pywikibot.Claim(wikidata, 'P569') @@ -417,8 +417,8 @@ self.wdp = pywikibot.ItemPage(wikidatatest, 'Q6') self.assertRaises(pywikibot.PageNotSaved, self.wdp.save) self.wdp.previousRevision() - self.assertEquals(self.wdp.langlinks(), []) - self.assertEquals(self.wdp.templates(), []) + self.assertEqual(self.wdp.langlinks(), []) + self.assertEqual(self.wdp.templates(), []) self.assertFalse(self.wdp.isCategoryRedirect()) self.wdp.templatesWithParams() @@ -436,14 +436,14 @@ def test_iterlinks_page_object(self): page = [pg for pg in self.wdp.iterlinks() if pg.site.language() == 'af'][0] - self.assertEquals(page, pywikibot.Page(pywikibot.Site('af', 'wikipedia'), u'New York Stad')) + self.assertEqual(page, pywikibot.Page(pywikibot.Site('af', 'wikipedia'), u'New York Stad')) def test_iterlinks_filtering(self): wikilinks = list(self.wdp.iterlinks('wikipedia')) wvlinks = list(self.wdp.iterlinks('wikivoyage')) - self.assertEquals(len(wikilinks), 3) - self.assertEquals(len(wvlinks), 2) + self.assertEqual(len(wikilinks), 3) + self.assertEqual(len(wvlinks), 2) class TestWriteNormalizeLang(PywikibotTestCase): @@ -462,12 +462,12 @@ lang_in = {self.site: 'foo'} response = WikibasePage._normalizeLanguages(lang_in) - self.assertEquals(response, self.lang_out) + self.assertEqual(response, self.lang_out) def test_normalized_lang(self): response = WikibasePage._normalizeData( copy.deepcopy(self.lang_out)) - self.assertEquals(response, self.lang_out) + self.assertEqual(response, self.lang_out) class TestWriteNormalizeData(PywikibotTestCase): @@ -501,12 +501,12 @@ } response = WikibasePage._normalizeData(data_in) - self.assertEquals(response, self.data_out) + self.assertEqual(response, self.data_out) def test_normalized_data(self): response = WikibasePage._normalizeData( copy.deepcopy(self.data_out)) - self.assertEquals(response, self.data_out) + self.assertEqual(response, self.data_out) if __name__ == '__main__': diff --git a/tests/wikidataquery_tests.py b/tests/wikidataquery_tests.py index 52c8416..07d72de 100644 --- a/tests/wikidataquery_tests.py +++ b/tests/wikidataquery_tests.py @@ -245,8 +245,8 @@ self.assertTrue(os.path.exists(cacheFile)) - self.assertTrue('status' in data) - self.assertTrue('items' in data) + self.assertIn('status', data) + self.assertIn('items', data) t1 = time.time() data = w.query(q) diff --git a/tests/xmlreader_tests.py b/tests/xmlreader_tests.py index 955261c..2f9264c 100644 --- a/tests/xmlreader_tests.py +++ b/tests/xmlreader_tests.py @@ -24,22 +24,22 @@ xmlreader.XmlDump(os.path.join(self.path, 'data', "article-pear.xml"), allrevisions=True).parse()] - self.assertEquals(4, len(pages)) - self.assertEquals(u"Automated conversion", pages[0].comment) - self.assertEquals(u"Pear", pages[0].title) - self.assertEquals(u"24278", pages[0].id) + self.assertEqual(4, len(pages)) + self.assertEqual(u"Automated conversion", pages[0].comment) + self.assertEqual(u"Pear", pages[0].title) + self.assertEqual(u"24278", pages[0].id) self.assertTrue(pages[0].text.startswith('Pears are [[tree]]s of')) - self.assertEquals(u"Quercusrobur", pages[1].username) - self.assertEquals(u"Pear", pages[0].title) + self.assertEqual(u"Quercusrobur", pages[1].username) + self.assertEqual(u"Pear", pages[0].title) def test_XmlDumpFirstRev(self): pages = [r for r in xmlreader.XmlDump(os.path.join(self.path, 'data', "article-pear.xml")).parse()] - self.assertEquals(1, len(pages)) - self.assertEquals(u"Automated conversion", pages[0].comment) - self.assertEquals(u"Pear", pages[0].title) - self.assertEquals(u"24278", pages[0].id) + self.assertEqual(1, len(pages)) + self.assertEqual(u"Automated conversion", pages[0].comment) + self.assertEqual(u"Pear", pages[0].title) + self.assertEqual(u"24278", pages[0].id) self.assertTrue(pages[0].text.startswith('Pears are [[tree]]s of')) self.assertTrue(not pages[0].isredirect) -- To view, visit https://gerrit.wikimedia.org/r/157482 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I1bbef226a75d6ddc2e536ca94459595b198d81cc Gerrit-PatchSet: 5 Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Owner: XZise <CommodoreFabianus(a)gmx.de> Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com> Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl> Gerrit-Reviewer: Mpaa <mpaa.wiki(a)gmail.com> Gerrit-Reviewer: Nullzero <nullzero.free(a)gmail.com> Gerrit-Reviewer: XZise <CommodoreFabianus(a)gmx.de> Gerrit-Reviewer: jenkins-bot <>

1 0

[Gerrit] [FIX] Safely close the files - change (pywikibot/core)
by jenkins-bot (Code Review) 31 Aug '14

31 Aug '14

jenkins-bot has submitted this change and it was merged. Change subject: [FIX] Safely close the files ...................................................................... [FIX] Safely close the files Change-Id: I0b3f7fcca3b79474fd3cc21257ccb6e82087590b --- M pywikibot/version.py 1 file changed, 7 insertions(+), 5 deletions(-) Approvals: Nullzero: Looks good to me, approved jenkins-bot: Verified diff --git a/pywikibot/version.py b/pywikibot/version.py index 8826a4d..0b7d873 100644 --- a/pywikibot/version.py +++ b/pywikibot/version.py @@ -143,7 +143,8 @@ # some windows git versions provide git.cmd instead of git.exe cmd = 'git.cmd' - tag = open(os.path.join(_program_dir, '.git/config'), 'r').read() + with open(os.path.join(_program_dir, '.git/config'), 'r') as f: + tag = f.read() # Try 'origin' and then 'gerrit' as remote name; bail if can't find either. remote_pos = tag.find('[remote "origin"]') if remote_pos == -1: @@ -213,10 +214,11 @@ mtime = None fn = os.path.join(_program_dir, filename) if os.path.exists(fn): - for line in open(fn, 'r').readlines(): - if line.find('__version__') == 0: - exec(line) - break + with open(fn, 'r') as f: + for line in f.readlines(): + if line.find('__version__') == 0: + exec(line) + break stat = os.stat(fn) mtime = datetime.datetime.fromtimestamp(stat.st_mtime).isoformat(' ') if mtime and __version__: -- To view, visit https://gerrit.wikimedia.org/r/157487 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I0b3f7fcca3b79474fd3cc21257ccb6e82087590b Gerrit-PatchSet: 1 Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Owner: XZise <CommodoreFabianus(a)gmx.de> Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com> Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl> Gerrit-Reviewer: Nullzero <nullzero.free(a)gmail.com> Gerrit-Reviewer: jenkins-bot <>

1 0

[Gerrit] Removing deprecated arguments and using standard ones in int... - change (pywikibot/core)
by jenkins-bot (Code Review) 30 Aug '14

30 Aug '14

jenkins-bot has submitted this change and it was merged. Change subject: Removing deprecated arguments and using standard ones in interwiki.py ...................................................................... Removing deprecated arguments and using standard ones in interwiki.py It causes this error: https://bugzilla.wikimedia.org/show_bug.cgi?id=57050#c3 Bug 57050 Change-Id: Ifef3e1982d217613235ff0888825d7c4ce3bc81a --- M scripts/interwiki.py 1 file changed, 2 insertions(+), 2 deletions(-) Approvals: XZise: Looks good to me, but someone else must approve Mpaa: Looks good to me, approved jenkins-bot: Verified diff --git a/scripts/interwiki.py b/scripts/interwiki.py index 1691c4f..85394d7 100755 --- a/scripts/interwiki.py +++ b/scripts/interwiki.py @@ -2516,8 +2516,8 @@ namespaces = [] else: ns = 'all' - hintlessPageGen = pagegenerators.NewpagesPageGenerator(newPages, - namespace=ns) + hintlessPageGen = pagegenerators.NewpagesPageGenerator(total=newPages, + namespaces=ns) elif optRestore or optContinue or globalvar.restoreAll: dumpFileName = pywikibot.config.datafilepath( -- To view, visit https://gerrit.wikimedia.org/r/157340 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: Ifef3e1982d217613235ff0888825d7c4ce3bc81a Gerrit-PatchSet: 2 Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Owner: Ladsgroup <ladsgroup(a)gmail.com> Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl> Gerrit-Reviewer: Mpaa <mpaa.wiki(a)gmail.com> Gerrit-Reviewer: XZise <CommodoreFabianus(a)gmx.de> Gerrit-Reviewer: jenkins-bot <>

1 0

[Gerrit] category.py: consider also files, not only pages - change (pywikibot/core)
by jenkins-bot (Code Review) 30 Aug '14

30 Aug '14

jenkins-bot has submitted this change and it was merged. Change subject: category.py: consider also files, not only pages ...................................................................... category.py: consider also files, not only pages In CategoryTidyRobot(), a category is considered empty only if no pages are present, while number of files is not considered. This patch fixes it. Change-Id: I1a340e2337d44caeedd435a40160736ea943d7cb --- M scripts/category.py 1 file changed, 10 insertions(+), 8 deletions(-) Approvals: XZise: Looks good to me, but someone else must approve Ladsgroup: Looks good to me, approved jenkins-bot: Verified diff --git a/scripts/category.py b/scripts/category.py index fa7a3ea..cf07768 100755 --- a/scripts/category.py +++ b/scripts/category.py @@ -887,15 +887,17 @@ def run(self): cat = pywikibot.Category(self.site, self.catTitle) - if cat.categoryinfo['pages'] == 0: - pywikibot.output(u'There are no articles in category %s' + empty = True + preloadingGen = pagegenerators.PreloadingGenerator(cat.articles()) + for article in preloadingGen: + empty = False + pywikibot.output('') + pywikibot.output(u'=' * 67) + self.move_to_category(article, cat, cat) + + if empty: + pywikibot.output(u'There are no articles or files in category %s' % self.catTitle) - else: - preloadingGen = pagegenerators.PreloadingGenerator(cat.articles()) - for article in preloadingGen: - pywikibot.output('') - pywikibot.output(u'=' * 67) - self.move_to_category(article, cat, cat) class CategoryTreeRobot: -- To view, visit https://gerrit.wikimedia.org/r/157292 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I1a340e2337d44caeedd435a40160736ea943d7cb Gerrit-PatchSet: 2 Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Owner: Mpaa <mpaa.wiki(a)gmail.com> Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com> Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl> Gerrit-Reviewer: XZise <CommodoreFabianus(a)gmx.de> Gerrit-Reviewer: jenkins-bot <>

1 0

[Gerrit] Workaround flake8 2.2.3 breaking with pep257 - change (pywikibot/core)
by jenkins-bot (Code Review) 29 Aug '14

29 Aug '14

jenkins-bot has submitted this change and it was merged. Change subject: Workaround flake8 2.2.3 breaking with pep257 ...................................................................... Workaround flake8 2.2.3 breaking with pep257 pep257 yields some error which apparently cause flake8 2.2.3 to stall completely waiting for some input. That can be worked around by disabling multiprocessing (turned on by default with 2.2.3). Simply pass --jobs=1 to flake8 in the flake8-docstrings job. Upstream issue: https://bitbucket.org/tarek/flake8/issue/167/flake8-223-stall-with-multipro… Change-Id: Iff84c926b42846b4c36db30507851859a827e103 --- M tox.ini 1 file changed, 8 insertions(+), 1 deletion(-) Approvals: Ladsgroup: Looks good to me, approved jenkins-bot: Verified diff --git a/tox.ini b/tox.ini index 9fae601..00fcf0d 100644 --- a/tox.ini +++ b/tox.ini @@ -14,8 +14,15 @@ deps = flake8 [testenv:flake8-docstrings] -commands = flake8 --select=D {posargs} +commands = flake8 --select=D {posargs} --jobs=1 deps = flake8-docstrings +# Note: flake8 is run here with --jobs=1 to disable multiprocessing which stall +# the run with flake8-docstrings / pep257. +# That appeared with flake8 2.2.3 which turns multiprocessing on by default. +# +# See upstream issue: +# https://bitbucket.org/tarek/flake8/issue/167/flake8-223-stall-with-multipro… + [testenv:nose] setenv = PYWIKIBOT2_NO_USER_CONFIG=1 -- To view, visit https://gerrit.wikimedia.org/r/157291 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: Iff84c926b42846b4c36db30507851859a827e103 Gerrit-PatchSet: 1 Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Owner: Hashar <hashar(a)free.fr> Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com> Gerrit-Reviewer: XZise <CommodoreFabianus(a)gmx.de> Gerrit-Reviewer: jenkins-bot <>

1 0

[Gerrit] site.py: raise NoPage instead of Error - change (pywikibot/core)
by jenkins-bot (Code Review) 29 Aug '14

29 Aug '14

jenkins-bot has submitted this change and it was merged. Change subject: site.py: raise NoPage instead of Error ...................................................................... site.py: raise NoPage instead of Error In site.movepage() raise NoPage instead of Error if no page exists. This can be a problem for scripts that expect the proper exception. E.g. movepages.py expects NoPage to be raised. Added test in site_tests.py. Also reordered imports in site_tests.py. Change-Id: Ie89ec7dc8bd4ee99af4c8e51a779fedb52e098da --- M pywikibot/site.py M tests/site_tests.py 2 files changed, 15 insertions(+), 5 deletions(-) Approvals: XZise: Looks good to me, but someone else must approve Ladsgroup: Looks good to me, approved jenkins-bot: Verified diff --git a/pywikibot/site.py b/pywikibot/site.py index a19afa5..b125b54 100644 --- a/pywikibot/site.py +++ b/pywikibot/site.py @@ -3530,8 +3530,8 @@ raise Error("Cannot move page %s to its own title." % oldtitle) if not page.exists(): - raise Error("Cannot move page %s because it does not exist on %s." - % (oldtitle, self)) + raise NoPage("Cannot move page %s because it does not exist on %s." + % (oldtitle, self)) token = self.token(page, "move") self.lock_page(page) req = api.Request(site=self, action="move", to=newtitle, diff --git a/tests/site_tests.py b/tests/site_tests.py index b4e0540..3477872 100644 --- a/tests/site_tests.py +++ b/tests/site_tests.py @@ -8,14 +8,16 @@ __version__ = '$Id$' +import sys from distutils.version import LooseVersion as LV from collections import Iterable -import pywikibot -from tests.utils import PywikibotTestCase, unittest from datetime import datetime import re -import sys +import pywikibot +from pywikibot.exceptions import Error, NoPage +from tests.utils import PywikibotTestCase, unittest + if sys.version_info[0] > 2: basestring = (str, ) @@ -1035,6 +1037,14 @@ self.assertFalse(entered_loop(mysite.siteinfo.get(not_exists).values())) self.assertFalse(entered_loop(mysite.siteinfo.get(not_exists).keys())) + def test_movepage(self): + self.assertRaises(Error, mysite.movepage, mainpage, 'Main Page', 'test') + + page_from = pywikibot.Page(mysite, 'Not exiting page') + if not page_from.exists(): + self.assertRaises(NoPage, mysite.movepage, + page_from, 'Main Page', 'test') + class TestSiteLoadRevisions(PywikibotTestCase): -- To view, visit https://gerrit.wikimedia.org/r/156707 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: Ie89ec7dc8bd4ee99af4c8e51a779fedb52e098da Gerrit-PatchSet: 3 Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Owner: Mpaa <mpaa.wiki(a)gmail.com> Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com> Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com> Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl> Gerrit-Reviewer: XZise <CommodoreFabianus(a)gmx.de> Gerrit-Reviewer: jenkins-bot <>

1 0

[Gerrit] Wild card '*' in user-config does not allow login - change (pywikibot/core)
by jenkins-bot (Code Review) 29 Aug '14

29 Aug '14

jenkins-bot has submitted this change and it was merged. Change subject: Wild card '*' in user-config does not allow login ...................................................................... Wild card '*' in user-config does not allow login Allow login with: usernames['family']['*'] or sysopnames['family']['*'] in user-config.py. Bug: 69342 Change-Id: Ic0dc3a614df40ffdb400e15122cadaa82ed86266 --- M pywikibot/login.py 1 file changed, 7 insertions(+), 5 deletions(-) Approvals: John Vandenberg: Looks good to me, approved jenkins-bot: Verified diff --git a/pywikibot/login.py b/pywikibot/login.py index 46afba3..0026aba 100644 --- a/pywikibot/login.py +++ b/pywikibot/login.py @@ -44,8 +44,9 @@ self.username = user elif sysop: try: - self.username = config.sysopnames[ - self.site.family.name][self.site.code] + family_sysopnames = config.sysopnames[self.site.family.name] + self.username = family_sysopnames.get(self.site.code, None) + self.username = self.username or family_sysopnames['*'] except KeyError: raise NoUsername(u"""\ ERROR: Sysop username for %(fam_name)s:%(wiki_code)s is undefined. @@ -56,8 +57,9 @@ 'wiki_code': self.site.code}) else: try: - self.username = config.usernames[ - self.site.family.name][self.site.code] + family_usernames = config.usernames[self.site.family.name] + self.username = family_usernames.get(self.site.code, None) + self.username = self.username or family_usernames['*'] except: raise NoUsername(u"""\ ERROR: Username for %(fam_name)s:%(wiki_code)s is undefined. @@ -196,7 +198,7 @@ # % {'name': self.username, # 'page': botList[self.site.family.name][self.site.code]}) # logger.error( -#"Please make sure you are allowed to use the robot before actually using it!") +# "Please make sure you are allowed to use the robot before actually using it!") # return False return True -- To view, visit https://gerrit.wikimedia.org/r/156987 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: Ic0dc3a614df40ffdb400e15122cadaa82ed86266 Gerrit-PatchSet: 2 Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Owner: Mpaa <mpaa.wiki(a)gmail.com> Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com> Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com> Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl> Gerrit-Reviewer: jenkins-bot <>

1 0

[Gerrit] [FEAT] Chunked uploads - change (pywikibot/core)
by Mpaa (Code Review) 28 Aug '14

28 Aug '14

Mpaa has submitted this change and it was merged. Change subject: [FEAT] Chunked uploads ...................................................................... [FEAT] Chunked uploads This allows chunked uploads by setting the parameter 'chunk_size' to a value between 0 and the file size (both exclusive). It will also only work if the version is 1.20 or newer. The upload.py script supports this mode via the '-chunked' parameter. This also adds the capability to run the API request without throttle so that it don't has to wait after each request. See: https://www.mediawiki.org/wiki/API:Upload#Chunked_uploading Change-Id: I80b2bba9e63832173d5b697db1f4ea419ca1122f --- M pywikibot/data/api.py M pywikibot/site.py M scripts/upload.py 3 files changed, 156 insertions(+), 36 deletions(-) Approvals: Mpaa: Looks good to me, approved diff --git a/pywikibot/data/api.py b/pywikibot/data/api.py index 8132a27..11df17b 100644 --- a/pywikibot/data/api.py +++ b/pywikibot/data/api.py @@ -129,6 +129,8 @@ @param site: The Site to which the request will be submitted. If not supplied, uses the user's configured default Site. @param mime: If true, send in "multipart/form-data" format (default False) + @param mime_params: A dictionary of parameter which should only be + transferred via mime mode. If not None sets mime to True. @param max_retries: (optional) Maximum number of times to retry after errors, defaults to 25 @param retry_wait: (optional) Minimum time to wait after an error, @@ -143,7 +145,15 @@ self.site = kwargs.pop("site") except KeyError: self.site = pywikibot.Site() - self.mime = kwargs.pop("mime", False) + if 'mime_params' in kwargs: + self.mime_params = kwargs.pop('mime_params') + # mime may not be different from mime_params + if 'mime' in kwargs and kwargs.pop('mime') != self.mime: + raise ValueError('If mime_params is set, mime may not differ ' + 'from it.') + else: + self.mime = kwargs.pop('mime', False) + self.throttle = kwargs.pop('throttle', False) self.max_retries = kwargs.pop("max_retries", pywikibot.config.max_retries) self.retry_wait = kwargs.pop("retry_wait", pywikibot.config.retry_wait) self.params = {} @@ -210,6 +220,23 @@ def iteritems(self): return iter(self.params.items()) + @property + def mime(self): + """Return whether mime parameters are defined.""" + return self.mime_params is not None + + @mime.setter + def mime(self, value): + """ + Change whether mime parameter should be defined. + + This will clear the mime parameters. + """ + try: + self.mime_params = dict(value) + except TypeError: + self.mime_params = {} if value else None + def http_params(self): """Return the parameters formatted for inclusion in an HTTP request. @@ -218,7 +245,9 @@ unicode (may be |-separated list) str in site encoding (may be |-separated list) """ - + if self.mime_params and set(self.params.keys()) & set(self.mime_params.keys()): + raise ValueError('The mime_params and params may not share the ' + 'same keys.') for key in self.params: if isinstance(self.params[key], bytes): self.params[key] = self.params[key].decode(self.site.encoding()) @@ -296,6 +325,23 @@ message = None return message == ERR_MSG + @staticmethod + def _generate_MIME_part(key, content, keytype, headers): + if not keytype: + try: + content.encode("ascii") + keytype = ("text", "plain") + except UnicodeError: + keytype = ("application", "octet-stream") + submsg = MIMENonMultipart(*keytype) + content_headers = {'name': key} + if headers: + content_headers.update(headers) + submsg.add_header("Content-disposition", "form-data", + **content_headers) + submsg.set_payload(content) + return submsg + def submit(self): """Submit a query and parse the response. @@ -308,7 +354,10 @@ simulate = self._simulate(action) if simulate: return simulate - self.site.throttle(write=self.write) + if self.throttle: + self.site.throttle(write=self.write) + else: + pywikibot.log("Action '{0}' is submitted not throttled.".format(action)) uri = self.site.scriptpath() + "/api.php" ssl = False if self.site.family.name in config.available_ssl_project: @@ -328,22 +377,15 @@ filetype = mimetypes.guess_type(local_filename)[0] \ or 'application/octet-stream' file_content = file(local_filename, "rb").read() - submsg = MIMENonMultipart(*filetype.split("/")) - submsg.add_header("Content-disposition", - "form-data", name=key, - filename=local_filename) - submsg.set_payload(file_content) + submsg = Request._generate_MIME_part( + key, file_content, filetype.split('/'), + {'filename': local_filename}) else: - try: - self.params[key].encode("ascii") - keytype = ("text", "plain") - except UnicodeError: - keytype = ("application", "octet-stream") - submsg = MIMENonMultipart(*keytype) - submsg.add_header("Content-disposition", "form-data", - name=key) - submsg.set_payload(self.params[key]) + submsg = Request._generate_MIME_part( + key, self.params[key], None, None) container.attach(submsg) + for key, value in self.mime_params.items(): + container.attach(Request._generate_MIME_part(key, *value)) # strip the headers to get the HTTP message body body = container.as_string() marker = "\n\n" # separates headers from body diff --git a/pywikibot/site.py b/pywikibot/site.py index fb8202d..21e0989 100644 --- a/pywikibot/site.py +++ b/pywikibot/site.py @@ -3858,7 +3858,8 @@ @deprecate_arg('imagepage', 'filepage') def upload(self, filepage, source_filename=None, source_url=None, - comment=None, text=None, watch=False, ignore_warnings=False): + comment=None, text=None, watch=False, ignore_warnings=False, + chunk_size=0): """Upload a file to the wiki. Either source_filename or source_url, but not both, must be provided. @@ -3875,7 +3876,11 @@ @param watch: If true, add filepage to the bot user's watchlist @param ignore_warnings: if true, ignore API warnings and force upload (for example, to overwrite an existing file); default False - + @param chunk_size: The chunk size in bytesfor chunked uploading (see + U{https://www.mediawiki.org/wiki/API:Upload#Chunked_uploading}). It + will only upload in chunks, if the version number is 1.20 or higher + and the chunk size is positive but lower than the file size. + @type chunk_size: int """ upload_warnings = { # map API warning codes to user error messages @@ -3909,18 +3914,51 @@ if not text: text = comment token = self.token(filepage, "edit") + result = None if source_filename: # upload local file # make sure file actually exists if not os.path.isfile(source_filename): raise ValueError("File '%s' does not exist." % source_filename) - # TODO: if file size exceeds some threshold (to be determined), - # upload by chunks (--> os.path.getsize(source_filename)) + additional_parameters = {} + throttle = True + filesize = os.path.getsize(source_filename) + if (chunk_size > 0 and chunk_size < filesize and + LV(self.version()) >= LV('1.20')): + offset = 0 + file_key = None + with open(source_filename, 'rb') as f: + while True: + f.seek(offset) + chunk = f.read(chunk_size) + req = api.Request(site=self, action='upload', token=token, + stash='1', offset=offset, filesize=filesize, + filename=filepage.title(withNamespace=False), + mime_params={}, throttle=throttle) + req.mime_params['chunk'] = (chunk, None, {'filename': req.params['filename']}) + if file_key: + req['filekey'] = file_key + # TODO: Proper error and warning handling + data = req.submit()['upload'] + if 'warnings' in data: + result = data + break + file_key = data['filekey'] + throttle = False + new_offset = int(data['offset']) + if offset + len(chunk) != new_offset: + pywikibot.warning('Unexpected offset.') + offset = new_offset + if data['result'] != 'Continue': # finished + additional_parameters['filekey'] = file_key + break + else: + additional_parameters = {'file': source_filename, 'mime': True} req = api.Request(site=self, action="upload", token=token, filename=filepage.title(withNamespace=False), - file=source_filename, comment=comment, - text=text, mime=True) + comment=comment, text=text, throttle=throttle, + **additional_parameters) else: # upload by URL if "upload_by_url" not in self.userinfo["rights"]: @@ -3930,16 +3968,17 @@ req = api.Request(site=self, action="upload", token=token, filename=filepage.title(withNamespace=False), url=source_url, comment=comment, text=text) - if watch: - req["watch"] = "" - if ignore_warnings: - req["ignorewarnings"] = "" - try: - result = req.submit() - except api.APIError: - # TODO: catch and process foreseeable errors - raise - result = result["upload"] + if not result: + if watch: + req["watch"] = "" + if ignore_warnings: + req["ignorewarnings"] = "" + try: + result = req.submit() + except api.APIError: + # TODO: catch and process foreseeable errors + raise + result = result["upload"] pywikibot.debug(result, _logger) if "warnings" in result: warning = list(result["warnings"].keys())[0] diff --git a/scripts/upload.py b/scripts/upload.py index 8a9a44d..39f9df5 100755 --- a/scripts/upload.py +++ b/scripts/upload.py @@ -11,6 +11,15 @@ is given -abortonwarn: Abort upload on the specified warning type. If no warning type is specified abort on all warnings. + -chunked: Upload the file in chunks (more overhead, but restartable). If + no value is specified the chunk size is 1 MiB. The value must + be a number which can be preceded by a suffix. The units are: + No suffix: Bytes + 'k': Kilobytes (1000 B) + 'M': Megabytes (1000000 B) + 'Ki': Kibibytes (1024 B) + 'Mi': Mebibytes (1024x1024 B) + The suffixes are case insenstive. If any other arguments are given, the first is the URL or filename to upload, and the rest is a proposed description to go with the upload. If none of these @@ -34,6 +43,8 @@ import urllib import urlparse import tempfile +import re +import math import pywikibot import pywikibot.data.api from pywikibot import config @@ -43,7 +54,7 @@ def __init__(self, url, urlEncoding=None, description=u'', useFilename=None, keepFilename=False, verifyDescription=True, ignoreWarning=False, - targetSite=None, uploadByUrl=False, aborts=[]): + targetSite=None, uploadByUrl=False, aborts=[], chunk_size=0): """ @param ignoreWarning: Set this to True if you want to upload even if another file would be overwritten or another mistake would be @@ -58,6 +69,7 @@ self.verifyDescription = verifyDescription self.ignoreWarning = ignoreWarning self.aborts = aborts + self.chunk_size = chunk_size if config.upload_to_commons: self.targetSite = targetSite or pywikibot.Site('commons', 'commons') @@ -224,7 +236,8 @@ else: temp = self.url site.upload(imagepage, source_filename=temp, - ignore_warnings=self.ignoreWarning) + ignore_warnings=self.ignoreWarning, + chunk_size=self.chunk_size) except pywikibot.data.api.UploadWarning as warn: pywikibot.output(u"We got a warning message: {0}".format(warn.message)) @@ -266,6 +279,8 @@ useFilename = None verifyDescription = True aborts = set() + chunk_size = 0 + chunk_size_regex = re.compile(r'^-chunked(?::(\d+(?:\.\d+)?)[ \t]*(k|ki|m|mi)?b?)?$', re.I) # process all global bot args # returns a list of non-global args, i.e. args for upload.py @@ -282,6 +297,30 @@ aborts.add(arg[len('-abortonwarn:'):]) else: aborts = True + elif arg.startswith('-chunked'): + match = chunk_size_regex.match(arg) + if match: + if match.group(1): # number was in there + base = float(match.group(1)) + if match.group(2): # suffix too + suffix = match.group(2).lower() + if suffix == "k": + suffix = 1000 + elif suffix == "m": + suffix = 1000000 + elif suffix == "ki": + suffix = 1 << 10 + elif suffix == "mi": + suffix = 1 << 20 + else: + pass # huh? + else: + suffix = 1 + chunk_size = math.trunc(base * suffix) + else: + chunk_size = 1 << 20 # default to 1 MiB + else: + pywikibot.error('Chunk size parameter is not valid.') elif url == u'': url = arg else: @@ -290,7 +329,7 @@ bot = UploadRobot(url, description=description, useFilename=useFilename, keepFilename=keepFilename, verifyDescription=verifyDescription, - aborts=aborts) + aborts=aborts, chunk_size=chunk_size) bot.run() if __name__ == "__main__": -- To view, visit https://gerrit.wikimedia.org/r/156030 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I80b2bba9e63832173d5b697db1f4ea419ca1122f Gerrit-PatchSet: 4 Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Owner: XZise <CommodoreFabianus(a)gmx.de> Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com> Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com> Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl> Gerrit-Reviewer: Mpaa <mpaa.wiki(a)gmail.com> Gerrit-Reviewer: XZise <CommodoreFabianus(a)gmx.de> Gerrit-Reviewer: jenkins-bot <>

1 0

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

Pywikibot-commits August 2014