jenkins-bot has submitted this change and it was merged.
Change subject: Unit tests for Site.{long,short,ancient,unwatched}pages()
......................................................................
Unit tests for Site.{long,short,ancient,unwatched}pages()
More unit tests for uncovered Site functions.
Bug: T60941
Change-Id: I7a7ee263b56c4cab8c73ae5db701dc595de562e8
---
M tests/site_tests.py
1 file changed, 47 insertions(+), 0 deletions(-)
Approvals:
John Vandenberg: Looks good to me, but someone else must approve
Mpaa: Looks good to me, approved
jenkins-bot: Verified
diff --git a/tests/site_tests.py b/tests/site_tests.py
index d2cfcd1..9ec0b45 100644
--- a/tests/site_tests.py
+++ b/tests/site_tests.py
@@ -738,6 +738,53 @@
self.assertTrue(all(isinstance(tup[2], unicode) for tup in the_list))
self.assertTrue(all(isinstance(tup[3], unicode) for tup in the_list))
+ def test_longpages(self):
+ """Test the site.longpages() method."""
+ mysite = self.get_site()
+ longpages = list(mysite.longpages(total=10))
+
+ # Make sure each object returned by site.longpages() is
+ # a tuple of a Page object and an int
+ self.assertTrue(all(isinstance(tup, tuple) and len(tup) == 2) for tup in longpages)
+ self.assertTrue(all(isinstance(tup[0], pywikibot.Page) for tup in longpages))
+ self.assertTrue(all(isinstance(tup[1], int) for tup in longpages))
+
+ def test_shortpages(self):
+ """Test the site.shortpages() method."""
+ mysite = self.get_site()
+ shortpages = list(mysite.shortpages(total=10))
+
+ # Make sure each object returned by site.shortpages() is
+ # a tuple of a Page object and an int
+ self.assertTrue(all(isinstance(tup, tuple) and len(tup) == 2) for tup in shortpages)
+ self.assertTrue(all(isinstance(tup[0], pywikibot.Page) for tup in shortpages))
+ self.assertTrue(all(isinstance(tup[1], int) for tup in shortpages))
+
+ def test_ancientpages(self):
+ """Test the site.ancientpages() method."""
+ mysite = self.get_site()
+ ancientpages = list(mysite.ancientpages(total=10))
+
+ # Make sure each object returned by site.ancientpages() is
+ # a tuple of a Page object and a Timestamp object
+ self.assertTrue(all(isinstance(tup, tuple) and len(tup) == 2) for tup in ancientpages)
+ self.assertTrue(all(isinstance(tup[0], pywikibot.Page) for tup in ancientpages))
+ self.assertTrue(all(isinstance(tup[1], pywikibot.Timestamp) for tup in ancientpages))
+
+ def test_unwatchedpages(self):
+ """Test the site.unwatchedpages() method."""
+ mysite = self.get_site()
+ try:
+ unwatchedpages = list(mysite.unwatchedpages(total=10))
+ except api.APIError as error:
+ if error.code == 'gqpspecialpage-cantexecute':
+ # User must have correct permissions to use Special:UnwatchedPages
+ raise unittest.SkipTest(error)
+ raise
+
+ # Make sure each object returned by site.unwatchedpages() is a Page object
+ self.assertTrue(all(isinstance(p, pywikibot.Page) for p in unwatchedpages))
+
def testBlocks(self):
"""Test the site.blocks() method."""
mysite = self.get_site()
--
To view, visit https://gerrit.wikimedia.org/r/261866
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I7a7ee263b56c4cab8c73ae5db701dc595de562e8
Gerrit-PatchSet: 4
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: Sn1per <geofbot(a)gmail.com>
Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com>
Gerrit-Reviewer: Mpaa <mpaa.wiki(a)gmail.com>
Gerrit-Reviewer: Sn1per <geofbot(a)gmail.com>
Gerrit-Reviewer: jenkins-bot <>
jenkins-bot has submitted this change and it was merged.
Change subject: Set url to None if it contains only "*" in the weblink argument
......................................................................
Set url to None if it contains only "*" in the weblink argument
When "-weblink:*" is the argument passed, "*" is passed as the euquery url
to the MediaWiki API which raises a geubad_query: Invalid query. So, a
check is made on url in handleArg() if the argument starts with "-weblink"
and if it is a * then it is set to None. Now, the MediaWiki API will
generate every page with any URL.
Bug: T113142
Change-Id: I3a324942f6ac4a187179a6db5584361d65d71e04
---
M pywikibot/pagegenerators.py
1 file changed, 4 insertions(+), 0 deletions(-)
Approvals:
John Vandenberg: Looks good to me, but someone else must approve
Mpaa: Looks good to me, approved
jenkins-bot: Verified
diff --git a/pywikibot/pagegenerators.py b/pywikibot/pagegenerators.py
index d3a15a9..3f185d4 100644
--- a/pywikibot/pagegenerators.py
+++ b/pywikibot/pagegenerators.py
@@ -700,6 +700,10 @@
if not url:
url = pywikibot.input(
u'Pages with which weblink should be processed?')
+ # If url is * we make it None in order to search for every page
+ # with any URL.
+ if url == "*":
+ url = None
gen = LinksearchPageGenerator(url, site=self.site)
elif arg.startswith('-transcludes'):
transclusionPageTitle = arg[len('-transcludes:'):]
--
To view, visit https://gerrit.wikimedia.org/r/261767
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I3a324942f6ac4a187179a6db5584361d65d71e04
Gerrit-PatchSet: 6
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: Ani310 <anirudh.gp(a)gmail.com>
Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com>
Gerrit-Reviewer: Mpaa <mpaa.wiki(a)gmail.com>
Gerrit-Reviewer: jenkins-bot <>
jenkins-bot has submitted this change and it was merged.
Change subject: pagegenerators.py: Allow filtering by category
......................................................................
pagegenerators.py: Allow filtering by category
Allow generated pages to be filtered by category.
A category may have large number of pages, while using -intersect filtering
pages from other generators will is more efficient than getting all pages in
the category.
Add related tests in page_generators_tests.py.
Bug: T122392
Change-Id: Ib760712be4b5acc84b09b80e7eaf6b9c11e4c870
---
M pywikibot/pagegenerators.py
M tests/pagegenerators_tests.py
2 files changed, 61 insertions(+), 3 deletions(-)
Approvals:
Mpaa: Looks good to me, approved
Xqt: Looks good to me, but someone else must approve
jenkins-bot: Verified
diff --git a/pywikibot/pagegenerators.py b/pywikibot/pagegenerators.py
index d3a15a9..f198766 100644
--- a/pywikibot/pagegenerators.py
+++ b/pywikibot/pagegenerators.py
@@ -62,6 +62,9 @@
parameterHelp = u"""\
+-catfilter Filter the page generator to only yield pages in the
+ specified category. See -cat for argument format.
+
-cat Work on all pages which are in a specific category.
Argument can also be given as "-cat:categoryname" or
as "-cat:categoryname|fromtitle" (using # instead of |
@@ -350,6 +353,7 @@
self.articlefilter_list = []
self.titlefilter_list = []
self.claimfilter_list = []
+ self.catfilter_list = []
self.intersect = False
self.subpage_max_depth = None
self._site = site
@@ -422,6 +426,7 @@
if (self.titlefilter_list or
self.articlefilter_list or
self.claimfilter_list or
+ self.catfilter_list or
self.subpage_max_depth is not None or
self.qualityfilter_list):
pywikibot.warning(
@@ -466,11 +471,14 @@
dupfiltergen = RegexBodyFilterPageGenerator(
PreloadingGenerator(dupfiltergen), self.articlefilter_list)
+ if self.catfilter_list:
+ dupfiltergen = CategoryFilterPageGenerator(
+ dupfiltergen, self.catfilter_list, self.site)
+
return dupfiltergen
- def getCategoryGen(self, arg, recurse=False, content=False,
- gen_func=None):
- """Return generator based on Category defined by arg and gen_func."""
+ def getCategory(self, arg):
+ """Return Category and start as defined by arg."""
categoryname = arg.partition(':')[2]
if not categoryname:
categoryname = i18n.input(
@@ -491,6 +499,12 @@
categoryname)
cat = pywikibot.Category(pywikibot.Link(categoryname,
defaultNamespace=14))
+ return cat, startfrom
+
+ def getCategoryGen(self, arg, recurse=False, content=False,
+ gen_func=None):
+ """Return generator based on Category defined by arg and gen_func."""
+ cat, startfrom = self.getCategory(arg)
return gen_func(cat,
start=startfrom,
@@ -651,6 +665,10 @@
elif arg.startswith('-catr'):
gen = self.getCategoryGen(arg, recurse=True,
gen_func=CategorizedPageGenerator)
+ elif arg.startswith('-catfilter'):
+ cat, _ = self.getCategory(arg)
+ self.catfilter_list.append(cat)
+ return True
elif arg.startswith('-category'):
gen = self.getCategoryGen(arg, gen_func=CategorizedPageGenerator)
elif arg.startswith('-cat'):
@@ -1550,6 +1568,23 @@
else:
yield page
+
+def CategoryFilterPageGenerator(generator, category_list, site=None):
+ """
+ Wrap a generator to filter pages by categories specified.
+
+ @param generator: A generator object
+ @param category_list: categories used to filter generated pages
+ @type category_list: list of category objects
+
+ """
+ if site is None:
+ site = pywikibot.Site()
+ for page in generator:
+ if all(x in site.pagecategories(page) for x in category_list):
+ yield page
+
+
# name the generator methods
RegexFilterPageGenerator = RegexFilter.titlefilter
RegexBodyFilterPageGenerator = RegexFilter.contentfilter
diff --git a/tests/pagegenerators_tests.py b/tests/pagegenerators_tests.py
index ed38066..76d1c28 100755
--- a/tests/pagegenerators_tests.py
+++ b/tests/pagegenerators_tests.py
@@ -183,6 +183,29 @@
self.assertEqual(len(tuple(gen)), 9)
+class TestCategoryFilterPageGenerator(TestCase):
+
+ """Test CategoryFilterPageGenerator method."""
+
+ family = 'wikisource'
+ code = 'en'
+
+ base_title = 'Page:06-24-1920 -The Story of the Jones County Calf Case.pdf/%s'
+ category_list = ['Category:Validated']
+
+ def setUp(self):
+ super(TestCategoryFilterPageGenerator, self).setUp()
+ self.site = self.get_site()
+ self.titles = [self.base_title % i for i in range(1, 11)]
+ self.catfilter_list = [pywikibot.Category(self.site, cat) for cat in self.category_list]
+
+ def test_CategoryFilterPageGenerator(self):
+ site = self.site
+ gen = pagegenerators.PagesFromTitlesGenerator(self.titles, site)
+ gen = pagegenerators.CategoryFilterPageGenerator(gen, self.catfilter_list, site)
+ self.assertEqual(len(tuple(gen)), 7)
+
+
class TestQualityFilterPageGenerator(TestCase):
"""Test QualityFilterPageGenerator methods."""
--
To view, visit https://gerrit.wikimedia.org/r/261877
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: Ib760712be4b5acc84b09b80e7eaf6b9c11e4c870
Gerrit-PatchSet: 3
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: Vadiraja.k <vadi.fedx(a)gmail.com>
Gerrit-Reviewer: Billinghurst <billinghurstwiki(a)gmail.com>
Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com>
Gerrit-Reviewer: Mpaa <mpaa.wiki(a)gmail.com>
Gerrit-Reviewer: Siebrand <siebrand(a)kitano.nl>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot <>
jenkins-bot has submitted this change and it was merged.
Change subject: Use _siteinfo in is_uploaddisabled
......................................................................
Use _siteinfo in is_uploaddisabled
Change-Id: I2ce1f3c4410467216eda3d71577b54bd18d2581a
---
M pywikibot/site.py
1 file changed, 4 insertions(+), 0 deletions(-)
Approvals:
John Vandenberg: Looks good to me, approved
jenkins-bot: Verified
diff --git a/pywikibot/site.py b/pywikibot/site.py
index 7e0fae9..a7c0bd4 100644
--- a/pywikibot/site.py
+++ b/pywikibot/site.py
@@ -1384,6 +1384,7 @@
'writeapi',
'variantarticlepath',
'misermode',
+ 'uploadsenabled',
],
'namespaces': [ # for each namespace
'subpages',
@@ -5473,10 +5474,13 @@
def is_uploaddisabled(self):
"""Return True if upload is disabled on site.
+ When the version is at least 1.27wmf9, uses general siteinfo.
If not called directly, it is cached by the first attempted
upload action.
"""
+ if MediaWikiVersion(self.version()) >= MediaWikiVersion('1.27wmf9'):
+ return not self._siteinfo.get('general')['uploadsenabled']
if hasattr(self, '_uploaddisabled'):
return self._uploaddisabled
else:
--
To view, visit https://gerrit.wikimedia.org/r/261745
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I2ce1f3c4410467216eda3d71577b54bd18d2581a
Gerrit-PatchSet: 5
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: Yujwey <rawzausho(a)gmail.com>
Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com>
Gerrit-Reviewer: Yujwey <rawzausho(a)gmail.com>
Gerrit-Reviewer: jenkins-bot <>