jenkins-bot has submitted this change and it was merged.
Change subject: port PageTitleFilterPageGenerator from compat
......................................................................
port PageTitleFilterPageGenerator from compat
changed re.match to re.search for more precise comparison
bug: 55007
Change-Id: I3ff0bebff3e4bfc62fde290811d8d6504532e9c0
---
M pywikibot/pagegenerators.py
1 file changed, 26 insertions(+), 0 deletions(-)
Approvals:
John Vandenberg: Looks good to me, approved
jenkins-bot: Verified
diff --git a/pywikibot/pagegenerators.py b/pywikibot/pagegenerators.py
index b79dfda..fab7ccd 100644
--- a/pywikibot/pagegenerators.py
+++ b/pywikibot/pagegenerators.py
@@ -918,6 +918,32 @@
yield page
+@deprecated_args(ignoreList='ignore_list')
+def PageTitleFilterPageGenerator(generator, ignore_list):
+ """
+ Yield only those pages are not listed in the ignore list.
+
+ @param ignore_list: family names are mapped to dictionaries
+ in which language codes are mapped to lists of page titles
+ @type ignore_list: dict
+
+ """
+ def is_ignored(page):
+ if page.site.family.name in ignore_list and \
+ page.site.code in ignore_list[page.site.family.name]:
+ for ig in ignore_list[page.site.family.name][page.site.code]:
+ if re.search(ig, page.title()):
+ return True
+ return False
+
+ for page in generator:
+ if is_ignored(page):
+ if config.verbose_output:
+ pywikibot.output('Ignoring page %s' % page.title())
+ else:
+ yield page
+
+
def RedirectFilterPageGenerator(generator, no_redirects=True):
"""Yield pages from another generator that are redirects or not."""
for page in generator:
--
To view, visit https://gerrit.wikimedia.org/r/135407
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I3ff0bebff3e4bfc62fde290811d8d6504532e9c0
Gerrit-PatchSet: 8
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: Ricordisamoa <ricordisamoa(a)openmailbox.org>
Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com>
Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl>
Gerrit-Reviewer: Ricordisamoa <ricordisamoa(a)openmailbox.org>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot <>
jenkins-bot has submitted this change and it was merged.
Change subject: Cosmetic changes fixup
......................................................................
Cosmetic changes fixup
f0543d6 heavily revised to flow and error control of the script
cosmetic_changes. However a bug meant the changes made by the
change methods were discarded.
Change-Id: I2adecb09876bdf64ff4e9e03dd095368fa6135b2
---
M scripts/cosmetic_changes.py
1 file changed, 1 insertion(+), 1 deletion(-)
Approvals:
XZise: Looks good to me, approved
Dalba: Looks good to me, but someone else must approve
jenkins-bot: Verified
diff --git a/scripts/cosmetic_changes.py b/scripts/cosmetic_changes.py
index f0c892d..07f9c33 100755
--- a/scripts/cosmetic_changes.py
+++ b/scripts/cosmetic_changes.py
@@ -206,7 +206,7 @@
pywikibot.exception(e)
else:
raise
- return text if result is None else text
+ return text if result is None else result
@staticmethod
def isbn_execute(text):
--
To view, visit https://gerrit.wikimedia.org/r/169651
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I2adecb09876bdf64ff4e9e03dd095368fa6135b2
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: Dalba <dalba.wiki(a)gmail.com>
Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com>
Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl>
Gerrit-Reviewer: XZise <CommodoreFabianus(a)gmx.de>
Gerrit-Reviewer: jenkins-bot <>
jenkins-bot has submitted this change and it was merged.
Change subject: [FIX] DequeGenerator tests: always add talk page
......................................................................
[FIX] DequeGenerator tests: always add talk page
The tests for wikidata and wiktionary fail because their main pages are
not in the main namespace.
Change-Id: I46c4e507f0839023fe355161bb299e74c42c5367
---
M tests/pagegenerators_tests.py
1 file changed, 3 insertions(+), 3 deletions(-)
Approvals:
John Vandenberg: Looks good to me, approved
jenkins-bot: Verified
diff --git a/tests/pagegenerators_tests.py b/tests/pagegenerators_tests.py
index dd85f1b..96e5dfa 100755
--- a/tests/pagegenerators_tests.py
+++ b/tests/pagegenerators_tests.py
@@ -173,14 +173,14 @@
for page in gen:
pages_out.append(page)
# Add a page to the generator
- if page.namespace() == 0:
+ if not page.isTalkPage():
pages.extend([page.toggleTalkPage()])
self.assertTrue(all(isinstance(page, pywikibot.Page) for page in pages_out))
- self.assertEqual(len(pages_out), 2)
- self.assertEqual(pages_out[1].namespace(), 1)
self.assertIn(mainpage, pages_out)
self.assertIn(mainpage.toggleTalkPage(), pages_out)
+ self.assertEqual(len(pages_out), 2)
+ self.assertTrue(pages_out[1].isTalkPage())
class TestPreloadingItemGenerator(WikidataTestCase):
--
To view, visit https://gerrit.wikimedia.org/r/169647
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I46c4e507f0839023fe355161bb299e74c42c5367
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: XZise <CommodoreFabianus(a)gmx.de>
Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com>
Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl>
Gerrit-Reviewer: jenkins-bot <>
jenkins-bot has submitted this change and it was merged.
Change subject: [FEAT] Diff: Use PatchManager for showDiff
......................................................................
[FEAT] Diff: Use PatchManager for showDiff
This uses PatchManager for pywikibot.showDiff and also includes now a
header.
It also fixes the missing space in front of a line if it was added or
removed. This makes it consistent with lines where only parts of it were
changed which use difflib.ndiff. And it fixes a crash if both texts are
equal and not empty.
Also a minor change that the state which stores if it ends with the
default color is now represented as a boolean.
Change-Id: Ic59df944b4c64b8536a5178e1aabc0b1adfeab47
---
M pywikibot/__init__.py
M pywikibot/diff.py
2 files changed, 18 insertions(+), 65 deletions(-)
Approvals:
John Vandenberg: Looks good to me, approved
jenkins-bot: Verified
diff --git a/pywikibot/__init__.py b/pywikibot/__init__.py
index d70dc86..fc1a708 100644
--- a/pywikibot/__init__.py
+++ b/pywikibot/__init__.py
@@ -9,7 +9,6 @@
__version__ = '$Id$'
import datetime
-import difflib
import math
import re
import sys
@@ -49,6 +48,7 @@
from pywikibot.tools import UnicodeMixin, redirect_func
from pywikibot.i18n import translate
from pywikibot.data.api import UploadWarning
+from pywikibot.diff import PatchManager
import pywikibot.textlib as textlib
import pywikibot.tools
@@ -602,60 +602,7 @@
The differences are highlighted (only on compatible systems) to show which
changes were made.
"""
- # This is probably not portable to non-terminal interfaces....
- # For information on difflib, see http://pydoc.org/2.1/difflib.html
- color = {
- '+': 'lightgreen',
- '-': 'lightred',
- }
- diff = u''
- colors = []
- # This will store the last line beginning with + or -.
- lastline = None
- # For testing purposes only: show original, uncolored diff
- # for line in difflib.ndiff(oldtext.splitlines(), newtext.splitlines()):
- # print line
- for line in difflib.ndiff(oldtext.splitlines(), newtext.splitlines()):
- if line.startswith('?'):
- # initialize color vector with None, which means default color
- lastcolors = [None for c in lastline]
- # colorize the + or - sign
- lastcolors[0] = color[lastline[0]]
- # colorize changed parts in red or green
- for i in range(min(len(line), len(lastline))):
- if line[i] != ' ':
- lastcolors[i] = color[lastline[0]]
- diff += lastline + '\n'
- # append one None (default color) for the newline character
- colors += lastcolors + [None]
- elif lastline:
- diff += lastline + '\n'
- # colorize the + or - sign only
- lastcolors = [None for c in lastline]
- lastcolors[0] = color[lastline[0]]
- colors += lastcolors + [None]
- lastline = None
- if line[0] in ('+', '-'):
- lastline = line
- # there might be one + or - line left that wasn't followed by a ? line.
- if lastline:
- diff += lastline + '\n'
- # colorize the + or - sign only
- lastcolors = [None for c in lastline]
- lastcolors[0] = color[lastline[0]]
- colors += lastcolors + [None]
-
- result = u''
- lastcolor = None
- for i in range(len(diff)):
- if colors[i] != lastcolor:
- if lastcolor is None:
- result += '\03{%s}' % colors[i]
- else:
- result += '\03{default}'
- lastcolor = colors[i]
- result += diff[i]
- output(result)
+ PatchManager(oldtext, newtext).print_hunks()
# Throttle and thread handling
diff --git a/pywikibot/diff.py b/pywikibot/diff.py
index 7ac30dc..e118770 100644
--- a/pywikibot/diff.py
+++ b/pywikibot/diff.py
@@ -77,15 +77,17 @@
return l
for tag, i1, i2, j1, j2 in self.group:
+ # equal/delete/insert add additional space after the sign as it's
+ # what difflib.ndiff does do too.
if tag == 'equal':
for line in self.a[i1:i2]:
- yield ' ' + check_line(line)
+ yield ' ' + check_line(line)
if tag in ('delete'):
for line in self.a[i1:i2]:
- yield '-' + check_line(line)
+ yield '- ' + check_line(line)
if tag in ('insert'):
for line in self.b[j1:j2]:
- yield '+' + check_line(line)
+ yield '+ ' + check_line(line)
if tag in ('replace'):
for line in difflib.ndiff(self.a[i1:i2], self.b[j1:j2]):
yield check_line(line)
@@ -132,20 +134,20 @@
return line
colored_line = u''
- state = 'Close'
+ color_closed = True
for char, char_ref in zip_longest(line, line_ref.strip(), fillvalue=' '):
char_tagged = char
- if state == 'Close':
+ if color_closed:
if char_ref != ' ':
char_tagged = '\03{%s}%s' % (self.colors[color], char)
- state = 'Open'
- elif state == 'Open':
+ color_closed = False
+ else:
if char_ref == ' ':
char_tagged = '\03{default}%s' % char
- state = 'Close'
+ color_closed = True
colored_line += char_tagged
- if state == 'Open':
+ if not color_closed:
colored_line += '\03{default}'
return colored_line
@@ -228,11 +230,15 @@
# there is a section of unchanged text at the end of a, b.
if i2 < len(self.a):
- rng = (-1, (last[2], len(self.a)), (-1, -1))
+ rng = (-1, (i2, len(self.a)), (-1, -1))
blocks.append(rng)
return blocks
+ def print_hunks(self):
+ for hunk in self.hunks:
+ pywikibot.output(hunk.header + hunk.diff_text)
+
def review_hunks(self):
"Review hunks."
--
To view, visit https://gerrit.wikimedia.org/r/168967
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: Ic59df944b4c64b8536a5178e1aabc0b1adfeab47
Gerrit-PatchSet: 4
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: XZise <CommodoreFabianus(a)gmx.de>
Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com>
Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl>
Gerrit-Reviewer: Mpaa <mpaa.wiki(a)gmail.com>
Gerrit-Reviewer: XZise <CommodoreFabianus(a)gmx.de>
Gerrit-Reviewer: jenkins-bot <>
jenkins-bot has submitted this change and it was merged.
Change subject: [IMPROV] TestBaseCase: Only add backward compatible aliases if needed
......................................................................
[IMPROV] TestBaseCase: Only add backward compatible aliases if needed
Instead of assuming that Python 2 won't support currently Python 3
specific methods this just adds a wrapper for the old name if the new
name isn't implemented.
Change-Id: I1022d27fcecdc516b3ebf42dc3380458163b2aa4
---
M tests/aspects.py
1 file changed, 2 insertions(+), 1 deletion(-)
Approvals:
John Vandenberg: Looks good to me, approved
jenkins-bot: Verified
diff --git a/tests/aspects.py b/tests/aspects.py
index d6da7b8..40edb6d 100644
--- a/tests/aspects.py
+++ b/tests/aspects.py
@@ -49,7 +49,7 @@
"""Base class for all tests."""
- if sys.version_info[0] == 2:
+ if not hasattr(unittest.TestCase, 'assertRaisesRegex'):
def assertRaisesRegex(self, *args, **kwargs):
"""
Wrapper of unittest.assertRaisesRegexp for Python 2 unittest.
@@ -58,6 +58,7 @@
"""
return self.assertRaisesRegexp(*args, **kwargs)
+ if not hasattr(unittest.TestCase, 'assertRegex'):
def assertRegex(self, *args, **kwargs):
"""
Wrapper of unittest.assertRegexpMatches for Python 2 unittest.
--
To view, visit https://gerrit.wikimedia.org/r/169645
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I1022d27fcecdc516b3ebf42dc3380458163b2aa4
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: XZise <CommodoreFabianus(a)gmx.de>
Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com>
Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl>
Gerrit-Reviewer: jenkins-bot <>
jenkins-bot has submitted this change and it was merged.
Change subject: Simplify initialisation logic
......................................................................
Simplify initialisation logic
Change-Id: Ia8df25bc2d06c7a53c18e869aae87bac901321ac
---
M scripts/makecat.py
1 file changed, 12 insertions(+), 8 deletions(-)
Approvals:
Nullzero: Looks good to me, approved
jenkins-bot: Verified
diff --git a/scripts/makecat.py b/scripts/makecat.py
index cb2acf4..0b28870 100644
--- a/scripts/makecat.py
+++ b/scripts/makecat.py
@@ -235,10 +235,13 @@
except IOError:
# File does not exist
excludefile = codecs.open(filename, 'w', encoding=mysite.encoding())
+
+ # Get parent categories in order to `removeparent`
try:
parentcats = workingcat.categories()
except pywikibot.Error:
parentcats = []
+
# Do not include articles already in subcats; only checking direct subcats
subcatlist = list(workingcat.subcategories())
if subcatlist:
@@ -247,14 +250,12 @@
artlist = list(cat.articles())
for page in artlist:
checked[page] = page
- list = [x for x in workingcat.articles()]
- if list:
- for pl in list:
- checked[pl] = pl
- list = pagegenerators.PreloadingGenerator(list)
- for pl in list:
- include(pl)
- else:
+
+ # Fetch articles in category, and mark as already checked (seen)
+ # If category is empty, ask user if they want to look for pages
+ # in a diferent category.
+ articles = list(workingcat.articles(content=True))
+ if not articles:
pywikibot.output(
u"Category %s does not exist or is empty. Which page to start with?"
% workingcatname)
@@ -263,6 +264,9 @@
answer = workingcatname
pywikibot.output(u'' + answer)
pl = pywikibot.Page(mysite, answer)
+ articles = [pl]
+
+ for pl in articles:
checked[pl] = pl
include(pl)
--
To view, visit https://gerrit.wikimedia.org/r/169640
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: Ia8df25bc2d06c7a53c18e869aae87bac901321ac
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com>
Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl>
Gerrit-Reviewer: Nullzero <nullzero.free(a)gmail.com>
Gerrit-Reviewer: jenkins-bot <>