jenkins-bot has submitted this change and it was merged.
Change subject: Prevent AttributeError for when filename is None
......................................................................
Prevent AttributeError for when filename is None
>From line 172, filename can be None.
Such a case would raise an AttributeError
when handling the RuntimeError.
Bug: T121835
Change-Id: I02510d6e69269881dc1ae4b3ed0b9c65156cddbc
---
M pwb.py
1 file changed, 1 insertion(+), 1 deletion(-)
Approvals:
Mpaa: Looks good to me, approved
Hazard-SJ: Looks good to me, but someone else must approve
jenkins-bot: Verified
diff --git a/pwb.py b/pwb.py
index 601c2d4..fe2eeec 100755
--- a/pwb.py
+++ b/pwb.py
@@ -206,7 +206,7 @@
except RuntimeError as err:
# user-config.py to be created
print("NOTE: 'user-config.py' was not found!")
- if not filename.startswith('generate_'):
+ if filename is not None and not filename.startswith('generate_'):
print("Please follow the prompts to create it:")
run_python_file('generate_user_files.py',
['generate_user_files.py'],
--
To view, visit https://gerrit.wikimedia.org/r/260719
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I02510d6e69269881dc1ae4b3ed0b9c65156cddbc
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: 2.0
Gerrit-Owner: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: Hazard-SJ <hazardsjwiki(a)gmail.com>
Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com>
Gerrit-Reviewer: Mpaa <mpaa.wiki(a)gmail.com>
Gerrit-Reviewer: jenkins-bot <>
jenkins-bot has submitted this change and it was merged.
Change subject: Prevent AttributeError for when filename is None
......................................................................
Prevent AttributeError for when filename is None
>From line 172, filename can be None.
Such a case would raise an AttributeError
when handling the RuntimeError.
Bug: T121835
Change-Id: I02510d6e69269881dc1ae4b3ed0b9c65156cddbc
---
M pwb.py
1 file changed, 1 insertion(+), 1 deletion(-)
Approvals:
John Vandenberg: Looks good to me, approved
jenkins-bot: Verified
diff --git a/pwb.py b/pwb.py
index 84ca3aa..81ecb85 100755
--- a/pwb.py
+++ b/pwb.py
@@ -187,7 +187,7 @@
except RuntimeError as err:
# user-config.py to be created
print("NOTE: 'user-config.py' was not found!")
- if not filename.startswith('generate_'):
+ if filename is not None and not filename.startswith('generate_'):
print("Please follow the prompts to create it:")
run_python_file('generate_user_files.py',
['generate_user_files.py'],
--
To view, visit https://gerrit.wikimedia.org/r/260717
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I02510d6e69269881dc1ae4b3ed0b9c65156cddbc
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: Hazard-SJ <hazardsjwiki(a)gmail.com>
Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot <>
jenkins-bot has submitted this change and it was merged.
Change subject: Fix limit argument parser
......................................................................
Fix limit argument parser
'-limit' was parsed as including a value instead of prompting for
a value, causing a ValueError.
Change-Id: Iea57e1bb8cc5823cc01420636d04752a5fb2f3f7
---
M scripts/checkimages.py
1 file changed, 1 insertion(+), 1 deletion(-)
Approvals:
Xqt: Looks good to me, approved
jenkins-bot: Verified
diff --git a/scripts/checkimages.py b/scripts/checkimages.py
index 762f6d6..91468f6 100755
--- a/scripts/checkimages.py
+++ b/scripts/checkimages.py
@@ -1595,7 +1595,7 @@
# Here below there are the parameters.
for arg in pywikibot.handle_args(args):
if arg.startswith('-limit'):
- if len(arg) == 7:
+ if len(arg) == 6:
limit = int(pywikibot.input(
u'How many files do you want to check?'))
else:
--
To view, visit https://gerrit.wikimedia.org/r/257848
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: Iea57e1bb8cc5823cc01420636d04752a5fb2f3f7
Gerrit-PatchSet: 2
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com>
Gerrit-Reviewer: Murfel <murnatty(a)gmail.com>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot <>
jenkins-bot has submitted this change and it was merged.
Change subject: Subpage filter generator
......................................................................
Subpage filter generator
* Filter that excludes subpages that have a high depth
i.e. how many parents
* Siteinfo converts boolean properties (that we use) into actual
boolean values
* Page.depth returns the subpage depth
* Page._namespace_obj returns the Namespace object, rather than
the int Namespace id
* Update interwiki script to use 'content' property of Namespace
(which is now a boolean) for page_empty_check()
* Unit tests for above
Bug: T121323
Change-Id: Ia53580cf8ad7387c14d6ca3bf4fcf5b35f53edd4
---
M pywikibot/page.py
M pywikibot/pagegenerators.py
M pywikibot/site.py
M scripts/interwiki.py
M tests/page_tests.py
M tests/pagegenerators_tests.py
M tests/site_tests.py
7 files changed, 161 insertions(+), 3 deletions(-)
Approvals:
John Vandenberg: Looks good to me, approved
jenkins-bot: Verified
diff --git a/pywikibot/page.py b/pywikibot/page.py
index aa90752..c35c6cf 100644
--- a/pywikibot/page.py
+++ b/pywikibot/page.py
@@ -176,6 +176,12 @@
return self._link.namespace
@property
+ def _namespace_obj(self):
+ """Return the namespace object of the page."""
+ # TODO: T104864: Temporary until Page.namespace() is consistent
+ return self.site.namespaces[self.namespace()]
+
+ @property
def content_model(self):
"""Return the content model for this page.
@@ -187,6 +193,20 @@
self.site.loadpageinfo(self)
return self._contentmodel
+ @property
+ def depth(self):
+ """Return the depth/subpage level of the page."""
+ if not hasattr(self, '_depth'):
+ # Check if the namespace allows subpages
+ if self._namespace_obj.subpages:
+ # Count how many '/'s we have in the title
+ _depth = len(list(re.finditer('/', self.title())))
+ else:
+ # Does not allow subpages, which means depth is always 0
+ _depth = 0
+
+ return _depth
+
@deprecated_args(decode=None, savetitle="asUrl")
def title(self, underscore=False, withNamespace=True,
withSection=True, asUrl=False, asLink=False,
diff --git a/pywikibot/pagegenerators.py b/pywikibot/pagegenerators.py
index 9198dca..e205ff6 100644
--- a/pywikibot/pagegenerators.py
+++ b/pywikibot/pagegenerators.py
@@ -190,6 +190,10 @@
of pages, only retrieve n pages at a time from the wiki
server.
+-subpage:n Filters pages to only those that have depth n
+ i.e. a depth of 0 filters out all pages that are subpages, and
+ a depth of 1 filters out all pages that are subpages of subpages.
+
-titleregex A regular expression that needs to match the article title
otherwise the page won't be returned.
Multiple -titleregex:regexpr can be provided and the page will
@@ -339,6 +343,7 @@
self.titlefilter_list = []
self.claimfilter_list = []
self.intersect = False
+ self.subpage_max_depth = None
self._site = site
@property
@@ -406,9 +411,10 @@
if self.limit:
self.gens[i] = itertools.islice(self.gens[i], self.limit)
if len(self.gens) == 0:
- if self.titlefilter_list or self.articlefilter_list:
+ if self.titlefilter_list or self.articlefilter_list or \
+ self.claimfilter_list or self.subpage_max_depth is not None:
pywikibot.warning(
- 'grep/titleregex filters specified but no generators.')
+ 'filter(s) specified but no generators.')
return None
elif len(self.gens) == 1:
gensList = self.gens[0]
@@ -424,6 +430,11 @@
else:
gensList = CombinedPageGenerator(self.gens)
dupfiltergen = self._filter_unique(gensList)
+
+ # Add on subpage filter generator
+ if self.subpage_max_depth is not None:
+ dupfiltergen = SubpageFilterGenerator(
+ dupfiltergen, self.subpage_max_depth)
if self.claimfilter_list:
dupfiltergen = PreloadingItemGenerator(dupfiltergen)
@@ -799,6 +810,13 @@
gen = MySQLPageGenerator(query, site=self.site)
elif arg.startswith('-intersect'):
self.intersect = True
+ return True
+ elif arg.startswith('-subpage'):
+ max_depth = arg[len('-subpage:'):]
+ if not max_depth:
+ max_depth = pywikibot.input(
+ 'Maximum subpage depth:')
+ self.subpage_max_depth = int(max_depth)
return True
elif arg.startswith('-logevents:'):
gen = self._parse_log_events(*arg[len('-logevents:'):].split(','))
@@ -1376,6 +1394,32 @@
ItemClaimFilterPageGenerator = ItemClaimFilter.filter
+def SubpageFilterGenerator(generator, max_depth=0, show_filtered=False):
+ """
+ Generator which filters out subpages based on depth.
+
+ It looks at the namespace of each page and checks if that namespace has
+ subpages enabled. If so, pages with forward slashes ('/') are excluded.
+
+ @param generator: A generator object
+ @type generator: any generator or iterator
+ @param max_depth: Max depth of subpages to yield, at least zero
+ @type max_depth: int
+ @param show_filtered: Output a message for each page not yielded
+ @type show_filtered: bool
+ """
+ assert max_depth >= 0, 'Max subpage depth must be at least 0'
+
+ for page in generator:
+ if page.depth <= max_depth:
+ yield page
+ else:
+ if show_filtered:
+ pywikibot.output(
+ 'Page %s is a subpage that is too deep. Skipping.'
+ % page)
+
+
class RegexFilter(object):
"""Regex filter."""
diff --git a/pywikibot/site.py b/pywikibot/site.py
index 0c486d7..0d7623d 100644
--- a/pywikibot/site.py
+++ b/pywikibot/site.py
@@ -1372,6 +1372,29 @@
WARNING_REGEX = re.compile(u"^Unrecognized values? for parameter "
u"'siprop': ([^,]+(?:, [^,]+)*)$")
+ # Until we get formatversion=2, we have to convert empty-string properties
+ # into booleans so they are easier to use.
+ BOOLEAN_PROPS = {
+ 'general': [
+ 'imagewhitelistenabled',
+ 'langconversion',
+ 'titleconversion',
+ 'rtl',
+ 'readonly',
+ 'writeapi',
+ 'variantarticlepath',
+ 'misermode',
+ ],
+ 'namespaces': [ # for each namespace
+ 'subpages',
+ 'content',
+ 'nonincludable',
+ ],
+ 'magicwords': [ # for each magicword
+ 'case-sensitive',
+ ],
+ }
+
def __init__(self, site):
"""Initialise it with an empty cache."""
self._site = site
@@ -1412,12 +1435,32 @@
"""Do some default handling of data. Directly modifies data."""
# Be careful with version tests inside this here as it might need to
# query this method to actually get the version number
+
if prop == 'general':
if 'articlepath' not in data: # Introduced in 1.16.0
# Old version of MediaWiki, extract from base
path = urlparse(data['base'])[2].rsplit('/', 1)[0] + '/$1'
data['articlepath'] = path
+ # Convert boolean props from empty strings to actual boolean values
+ if prop in Siteinfo.BOOLEAN_PROPS.keys():
+ # siprop=namespaces and magicwords has properties per item in result
+ if prop == 'namespaces' or prop == 'magicwords':
+ for index, value in enumerate(data):
+ # namespaces uses a dict, while magicwords uses a list
+ key = index if type(data) is list else value
+ for p in Siteinfo.BOOLEAN_PROPS[prop]:
+ if p in data[key]:
+ data[key][p] = True
+ else:
+ data[key][p] = False
+ else:
+ for p in Siteinfo.BOOLEAN_PROPS[prop]:
+ if p in data:
+ data[p] = True
+ else:
+ data[p] = False
+
def _get_siteinfo(self, prop, expiry):
"""
Retrieve a siteinfo property.
diff --git a/scripts/interwiki.py b/scripts/interwiki.py
index dcd2095..5bf3dee 100755
--- a/scripts/interwiki.py
+++ b/scripts/interwiki.py
@@ -2466,7 +2466,7 @@
@rtype: bool
"""
# Check if the page is in content namespace
- if page.namespace() == 0:
+ if page._namespace_obj.content:
# Check if the page contains at least 50 characters
return len(page.text) < 50
else:
diff --git a/tests/page_tests.py b/tests/page_tests.py
index 73bb2d3..1cc6248 100644
--- a/tests/page_tests.py
+++ b/tests/page_tests.py
@@ -516,6 +516,21 @@
page_copy.isDisambig()
self.assertTrue(page_copy.isRedirectPage())
+ def test_depth(self):
+ """Test page depth calculation."""
+ site = self.get_site()
+ page_d0 = pywikibot.Page(site, '/home/test/')
+ if site.namespaces[0].subpages:
+ self.assertEqual(page_d0.depth, 3)
+ else:
+ self.assertEqual(page_d0.depth, 0)
+
+ page_user_d0 = pywikibot.Page(site, 'User:Sn1per')
+ self.assertEqual(page_user_d0.depth, 0)
+
+ page_d3 = pywikibot.Page(site, 'User:Sn1per/ProtectTest1/test/test')
+ self.assertEqual(page_d3.depth, 3)
+
class TestPageDeprecation(DefaultSiteTestCase, DeprecationTestCase):
diff --git a/tests/pagegenerators_tests.py b/tests/pagegenerators_tests.py
index 5c5e207..3936fca 100755
--- a/tests/pagegenerators_tests.py
+++ b/tests/pagegenerators_tests.py
@@ -22,6 +22,7 @@
from pywikibot.pagegenerators import (
PagesFromTitlesGenerator,
PreloadingGenerator,
+ CategorizedPageGenerator
)
from tests import join_data_path
@@ -233,6 +234,33 @@
self.assertEqual(len(list(gen)), 0)
+class SubpageFilterGeneratorTestCase(TestCase):
+
+ """Test SubpageFilterGenerator."""
+
+ family = 'test'
+ code = 'test'
+
+ def test_subpage_filter(self):
+ site = self.get_site()
+ test_cat = pywikibot.Category(site, 'Subpage testing')
+
+ gen = CategorizedPageGenerator(test_cat)
+ gen = pagegenerators.SubpageFilterGenerator(gen, 0)
+ expect_0 = ('/home/test',)
+ self.assertPagelistTitles(gen, titles=expect_0, site=site)
+
+ gen = CategorizedPageGenerator(test_cat)
+ gen = pagegenerators.SubpageFilterGenerator(gen, 3)
+ expect_3 = (
+ '/home/test',
+ 'User:Sn1per/ProtectTest1/test',
+ 'User:Sn1per/ProtectTest1/test/test',
+ 'User:Sn1per/sandbox',
+ )
+ self.assertPagelistTitles(gen, titles=expect_3, site=site)
+
+
class TestRepeatingGenerator(RecentChangesTestCase):
"""Test RepeatingGenerator."""
diff --git a/tests/site_tests.py b/tests/site_tests.py
index ab547f7..d2cfcd1 100644
--- a/tests/site_tests.py
+++ b/tests/site_tests.py
@@ -2015,6 +2015,14 @@
self.assertRegex(mysite.siteinfo['timezone'], "([A-Z]{3,4}|[A-Z][a-z]+/[A-Z][a-z]+)")
self.assertIn(mysite.siteinfo['case'], ["first-letter", "case-sensitive"])
+ def test_siteinfo_boolean(self):
+ """Test conversion of boolean properties from empty strings to True/False."""
+ mysite = self.get_site()
+ self.assertIsInstance(mysite.siteinfo['titleconversion'], bool)
+
+ self.assertIsInstance(mysite.namespaces[0].subpages, bool)
+ self.assertIsInstance(mysite.namespaces[0].content, bool)
+
def test_siteinfo_v1_16(self):
"""Test v.16+ siteinfo values."""
if MediaWikiVersion(self.site.version()) < MediaWikiVersion('1.16'):
--
To view, visit https://gerrit.wikimedia.org/r/258422
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: Ia53580cf8ad7387c14d6ca3bf4fcf5b35f53edd4
Gerrit-PatchSet: 20
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: Sn1per <geofbot(a)gmail.com>
Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: Mpaa <mpaa.wiki(a)gmail.com>
Gerrit-Reviewer: jenkins-bot <>
jenkins-bot has submitted this change and it was merged.
Change subject: [FIX] Support changed _vformat signature of 3.5.1
......................................................................
[FIX] Support changed _vformat signature of 3.5.1
With Python 3.5.1 the signature of `Formatter._vformat` changed. This is
supporting this change and now returns the tuple if it had returned a tuple
first.
This is not deactivating the custom `_vformat` method in Python 3 because the
patch introduced that change might be backported to Python 2.7 (see also
https://bugs.python.org/issue25034).
Bug: T121684
Change-Id: I0d1e4f062882ed03dc7bb2040984a67970df6c52
---
M pywikibot/tools/formatter.py
1 file changed, 12 insertions(+), 0 deletions(-)
Approvals:
Merlijn van Deen: Looks good to me, approved
jenkins-bot: Verified
diff --git a/pywikibot/tools/formatter.py b/pywikibot/tools/formatter.py
index ee34057..a571ee4 100644
--- a/pywikibot/tools/formatter.py
+++ b/pywikibot/tools/formatter.py
@@ -123,6 +123,18 @@
@rtype: unicode
"""
result = super(_ColorFormatter, self)._vformat(*args, **kwargs)
+ if isinstance(result, tuple):
+ additional_params = result[1:]
+ result = result[0]
+ else:
+ additional_params = tuple()
+ result = self._convert_bytes(result)
+ if additional_params:
+ result = (result, ) + additional_params
+ return result
+
+ def _convert_bytes(self, result):
+ """Convert everything into unicode."""
if PY2 and isinstance(result, str):
assert result == b''
result = '' # This is changing it into a unicode
--
To view, visit https://gerrit.wikimedia.org/r/259549
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I0d1e4f062882ed03dc7bb2040984a67970df6c52
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: XZise <CommodoreFabianus(a)gmx.de>
Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com>
Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl>
Gerrit-Reviewer: jenkins-bot <>
jenkins-bot has submitted this change and it was merged.
Change subject: Pywikibot 2.0rc4
......................................................................
Pywikibot 2.0rc4
Change-Id: I028472a8fe0c322981c1ef817a68218a7cf7305a
---
M ChangeLog
M docs/conf.py
M pywikibot/__init__.py
M setup.py
4 files changed, 47 insertions(+), 4 deletions(-)
Approvals:
XZise: Looks good to me, approved
jenkins-bot: Verified
diff --git a/ChangeLog b/ChangeLog
index c0deb8a..17a191d 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,48 @@
+Release 2.0rc4 (15 December 2015)
+=================================
+
+Bugfixes (core)
+---------------
+e42891d Remove dependency on pYsearch
+ecc39ce Require google>=0.7
+fa431a4 Desupport Python 2.6 for Pywikibot 2.0 release branch
+b1f6f39 config: Don't crash on later get_base_dir calls
+0925c51 cosmetic_changes: merge similar regexes
+4bf39f3 Update revId upon claim change
+ea11626 Update WOW hostnames
+2a5a4fa Mark site.patrol() as a user write action
+4f568c5 Fix interwikiFormat support for Link
+7cb6962 Changes are wrongly detected in the last langlink
+5c5d658 getLanguageLinks: Skip own site
+7ee8ec2 fix intersection of sets of namespaces
+94b0c43 Import textlib.TimeStripper
+584d7c1 Change "PyWikiBot" to "Pywikibot"
+1f8d57e Stop crashing item loads due to support of units
+c19dcfa __all__ items must be bytes on Python 2
+e6aa6ca Omit includeredirects parameter for allpages generator
+687ec9e Performance fix for sites using interwiki_putfirst option
+28db3d2 Fix Persian Wikipedia configuration
+3944f5e rollback: Use Revision instance properly
+860af34 Add must_be to DataSite write actions
+f8af8e0 Remove unneeded site argument to AutoFamily
+bf49a7d Fix ComparableMixin
+1d36e45 Deprecate ParamInfo.query_modules_with_limits
+e83c6a3 be-x-old is renamed to be-tarask
+19bbe7e Correctly identify qualifier from JSON
+
+Bugfixes (scripts)
+------------------
+4e78c27 checkimages: Expect NoPage exception
+fbb6266 interwiki: Optimize/remove unneeded calls to Page.isEmpty()
+428420a interwiki: Allow Subject.originPage to be updated
+a305d7b interwiki: Allow titletranslate.translate page to be None
+a786226 interwiki: '-ignore' parameter throws exception
+ab7abf5 interwiki: check for category before emptiness
+0db2e87 interwiki: do not automatically log in
+47ed5d5 nowcommons: Query at most one users
+
Release 2.0rc3 (30 September 2015)
-============================
+==================================
Bugfixes (core)
---------------
diff --git a/docs/conf.py b/docs/conf.py
index 2e59f19..f6b5f94 100644
--- a/docs/conf.py
+++ b/docs/conf.py
@@ -52,7 +52,7 @@
# The short X.Y version.
version = '2.0'
# The full version, including alpha/beta/rc tags.
-release = '2.0rc1'
+release = '2.0rc4'
# The language for content autogenerated by Sphinx. Refer to documentation
# for a list of supported languages.
diff --git a/pywikibot/__init__.py b/pywikibot/__init__.py
index 1191b6e..e26392a 100644
--- a/pywikibot/__init__.py
+++ b/pywikibot/__init__.py
@@ -7,7 +7,7 @@
#
from __future__ import unicode_literals
-__release__ = '2.0rc3'
+__release__ = '2.0rc4'
__version__ = '$Id$'
import datetime
diff --git a/setup.py b/setup.py
index de7bb95..397dac1 100644
--- a/setup.py
+++ b/setup.py
@@ -160,7 +160,7 @@
from setuptools import setup, find_packages
name = 'pywikibot'
-version = '2.0rc3'
+version = '2.0rc4'
github_url = 'https://github.com/wikimedia/pywikibot-core'
setup(
--
To view, visit https://gerrit.wikimedia.org/r/259215
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I028472a8fe0c322981c1ef817a68218a7cf7305a
Gerrit-PatchSet: 2
Gerrit-Project: pywikibot/core
Gerrit-Branch: 2.0
Gerrit-Owner: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com>
Gerrit-Reviewer: XZise <CommodoreFabianus(a)gmx.de>
Gerrit-Reviewer: jenkins-bot <>