jenkins-bot has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/434301 )
Change subject: Add 'startcolon' rule in textlib.py
......................................................................
Add 'startcolon' rule in textlib.py
startcolon is a rule that will be used to detect lines
that start with a colon or more, the colon handles the indentation
in the rendered version.
Change-Id: I3393515099874ab7db4e679efaaa3c630727908a
---
M pywikibot/textlib.py
M tests/textlib_tests.py
2 files changed, 5 insertions(+), 0 deletions(-)
Approvals:
John Vandenberg: Looks good to me, approved
jenkins-bot: Verified
diff --git a/pywikibot/textlib.py b/pywikibot/textlib.py
index 9ba2b5f..17480c4 100644
--- a/pywikibot/textlib.py
+++ b/pywikibot/textlib.py
@@ -258,6 +258,8 @@
# lines that start with a space are shown in a monospace font and
# have whitespace preserved.
'startspace': re.compile(r'(?m)^ (.*?)$'),
+ # lines that start with a colon or more will be indented
+ 'startcolon': re.compile(r'(?m)^:(.*?)$'),
# tables often have whitespace that is used to improve wiki
# source code readability.
# TODO: handle nested tables.
diff --git a/tests/textlib_tests.py b/tests/textlib_tests.py
index 648bcb1..45c79c6 100644
--- a/tests/textlib_tests.py
+++ b/tests/textlib_tests.py
@@ -1193,6 +1193,9 @@
self.assertEqual(textlib.replaceExcept(' xA ', 'x', 'y',
['startspace'], site=self.site),
' xA ')
+ self.assertEqual(textlib.replaceExcept(':xA ', 'x', 'y',
+ ['startcolon'], site=self.site),
+ ':xA ')
self.assertEqual(textlib.replaceExcept('<table>x</table>', 'x', 'y',
['table'], site=self.site),
'<table>x</table>')
--
To view, visit https://gerrit.wikimedia.org/r/434301
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I3393515099874ab7db4e679efaaa3c630727908a
Gerrit-Change-Number: 434301
Gerrit-PatchSet: 2
Gerrit-Owner: Rafidaslam <rafidteam(a)gmail.com>
Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: Zoranzoki21 <zorandori4444(a)gmail.com>
Gerrit-Reviewer: jenkins-bot <>
jenkins-bot has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/434205 )
Change subject: [bugfix] Revert getting members back to articles in category tidy
......................................................................
[bugfix] Revert getting members back to articles in category tidy
Until 765965cb1223e46fee42fc11835b5082105af707 category tidy was unable
to tidy also subcategories or even files. But since that commit category
tidy fails completely due to different behavior of Category.articles and
Category.members. This should be reverted once the bug is resolved.
Bug: T195062
Change-Id: Id41bdef269f6fb2ebf485ec22775eb7051966b6b
---
M scripts/category.py
1 file changed, 1 insertion(+), 1 deletion(-)
Approvals:
Xqt: Looks good to me, approved
jenkins-bot: Verified
diff --git a/scripts/category.py b/scripts/category.py
index cf5c209..57102cb 100755
--- a/scripts/category.py
+++ b/scripts/category.py
@@ -979,7 +979,7 @@
self.cat = pywikibot.Category(site, catTitle)
super(CategoryTidyRobot, self).__init__(
generator=pagegenerators.PreloadingGenerator(
- self.cat.members(namespaces=namespaces)))
+ self.cat.articles(namespaces=namespaces)))
@deprecated_args(article='member')
def move_to_category(self, member, original_cat, current_cat):
--
To view, visit https://gerrit.wikimedia.org/r/434205
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Id41bdef269f6fb2ebf485ec22775eb7051966b6b
Gerrit-Change-Number: 434205
Gerrit-PatchSet: 1
Gerrit-Owner: Dvorapa <dvorapa(a)seznam.cz>
Gerrit-Reviewer: Dvorapa <dvorapa(a)seznam.cz>
Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: Zoranzoki21 <zorandori4444(a)gmail.com>
Gerrit-Reviewer: jenkins-bot <>
jenkins-bot has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/428144 )
Change subject: [IMPR] Introduce setup and teardown methods
......................................................................
[IMPR] Introduce setup and teardown methods
setup and teardown methods can be used to initalize and cleanup the bot.
setup is called in when the bot.run starts working whereas teardown is called
from exit method. This is much more clearer than doing this inside a derived
run method.
setup allows some inital work like reading from live wiki or io stuff
which might be unwanted in the initializer of the instance. teardown allows
some cleanups like saving content to a file when the bots stops running and
there is no need to derive the exit method.
Change-Id: Id18281feb67be8a39f79e409a752f9502e736850
---
M pywikibot/bot.py
1 file changed, 15 insertions(+), 1 deletion(-)
Approvals:
JJMC89: Looks good to me, but someone else must approve
Framawiki: Looks good to me, approved
Dvorapa: Looks good to me, approved
jenkins-bot: Verified
diff --git a/pywikibot/bot.py b/pywikibot/bot.py
index 3007f70..e6a058b 100644
--- a/pywikibot/bot.py
+++ b/pywikibot/bot.py
@@ -1373,6 +1373,7 @@
terminated gracefully or was halted by exception.
May be overridden by subclasses.
"""
+ self.teardown()
pywikibot.output("\n%i pages read"
"\n%i pages written"
% (self._treat_counter, self._save_counter))
@@ -1408,6 +1409,19 @@
"""Return whether treat should be executed for the page."""
pass
+ def setup(self):
+ """Some inital setup before run operation starts.
+
+ This can be used for reading huge parts from life wiki or file
+ operation which is more than just initialize the instance.
+ Invoked by run() before running through generator loop.
+ """
+ pass
+
+ def teardown(self):
+ """Some cleanups after run operation. Invoked by exit()."""
+ pass
+
def run(self):
"""Process all pages in generator."""
self._start_ts = pywikibot.Timestamp.now()
@@ -1422,7 +1436,7 @@
# Python 2 does not clear previous exceptions and method `exit`
# relies on sys.exc_info returning exceptions occurring in `run`.
sys.exc_clear()
-
+ self.setup()
try:
for page in self.generator:
try:
--
To view, visit https://gerrit.wikimedia.org/r/428144
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Id18281feb67be8a39f79e409a752f9502e736850
Gerrit-Change-Number: 428144
Gerrit-PatchSet: 5
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: Dvorapa <dvorapa(a)seznam.cz>
Gerrit-Reviewer: Framawiki <framawiki(a)tools.wmflabs.org>
Gerrit-Reviewer: JJMC89 <JJMC89.Wikimedia(a)gmail.com>
Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: Zoranzoki21 <zorandori4444(a)gmail.com>
Gerrit-Reviewer: jenkins-bot <>
jenkins-bot has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/433361 )
Change subject: [test] enable -help test for imageharvest
......................................................................
[test] enable -help test for imageharvest
- enable -help test for imageharvest but there is a remaining failure
for -simulate the script (T167726)
- enable -help tests for isbn and weblinkchecker scripts
- some lists are convered to sets
- some dependencies are updated
Bug: T193973
Change-Id: Ib2326d77fa92767a0cab582da396fdd6b0623140
---
M tests/script_tests.py
1 file changed, 27 insertions(+), 31 deletions(-)
Approvals:
Dvorapa: Looks good to me, approved
jenkins-bot: Verified
diff --git a/tests/script_tests.py b/tests/script_tests.py
index 7ade52a..f84c4b1 100644
--- a/tests/script_tests.py
+++ b/tests/script_tests.py
@@ -32,11 +32,13 @@
script_deps = {
'script_wui': ['crontab', 'lua'],
# Note: package 'lunatic-python' provides module 'lua'
- 'flickrripper': ['flickrapi'],
+ 'flickrripper': ['flickrapi', 'Pillow'],
'imageharvest': ['bs4'],
+ 'isbn': ['python-stdnum'],
'match_images': ['PIL.ImageTk'],
'states_redirect': ['pycountry'],
- 'patrol': ['mwparserfromhell'],
+ 'patrol': ['mwparserfromhell>=0.3.3'],
+ 'weblinkchecker': ['memento_client>=0.5.1,!=0.6.0'],
}
if PY2:
@@ -56,15 +58,13 @@
return True
-failed_dep_script_list = [name
- for name in script_deps
- if not check_script_deps(name)]
+failed_dep_script_set = {name for name in script_deps
+ if not check_script_deps(name)}
-unrunnable_script_list = [
+unrunnable_script_set = {
'version', # does not use global args
'script_wui', # depends on lua compiling
- 'imageharvest', # T167726
-]
+}
def list_scripts(path, exclude=None):
@@ -81,8 +81,7 @@
list_scripts(archive_path))
runnable_script_list = (
- ['login'] + sorted(set(script_list)
- - {'login'} - set(unrunnable_script_list)))
+ ['login'] + sorted(set(script_list) - {'login'} - unrunnable_script_set))
script_input = {
'catall': 'q\n', # q for quit
@@ -180,9 +179,9 @@
# cause the loader to fallback to its own
# discover() ordering of unit tests.
- if unrunnable_script_list:
- unittest_print('Skipping execution of unrunnable scripts:\n %r'
- % unrunnable_script_list)
+ if unrunnable_script_set:
+ unittest_print('Skipping execution of unrunnable scripts:\n {!r}'
+ .format(unrunnable_script_set))
if not enable_autorun_tests:
unittest_print('Skipping execution of auto-run scripts '
@@ -193,7 +192,7 @@
['test_' + name
for name in sorted(script_list)
if name != 'login'
- and name not in unrunnable_script_list
+ and name not in unrunnable_script_set
])
test_list = ['tests.script_tests.TestScriptHelp.' + name
@@ -203,8 +202,8 @@
['test_' + name
for name in sorted(script_list)
if name != 'login'
- and name not in failed_dep_script_list
- and name not in unrunnable_script_list
+ and name not in failed_dep_script_set
+ and name not in unrunnable_script_set
and (enable_autorun_tests or name not in auto_run_script_list)
])
@@ -358,7 +357,7 @@
dct[test_name] = allowed_failure(dct[test_name])
# Disable test by default in nosetests
- if script_name in unrunnable_script_list:
+ if script_name in unrunnable_script_set:
# flag them as an expectedFailure due to py.test (T135594)
dct[test_name] = unittest.expectedFailure(dct[test_name])
dct[test_name].__test__ = False
@@ -378,12 +377,11 @@
net = False
- _expected_failures = failed_dep_script_list
- # -help supported not explicitly
- try:
- _expected_failures.remove('imageharvest')
- except ValueError:
- pass
+ _expected_failures = failed_dep_script_set
+ # -help tests may pass even when packages are required
+ _expected_failures.discard('imageharvest')
+ _expected_failures.discard('isbn')
+ _expected_failures.discard('weblinkchecker')
_allowed_failures = []
_argument = 'help'
@@ -405,20 +403,18 @@
user = True
- _expected_failures = [
+ _expected_failures = {
'catall', # stdout user interaction
'flickrripper', # Requires a flickr api key
'upload', # raises custom ValueError
- ] + failed_dep_script_list
+ }.union(failed_dep_script_set)
_allowed_failures = [
'disambredir',
- # T94681
- 'misspelling',
- # T77965
- 'watchlist',
- # T94680: uses exit code 1
- 'lonelypages',
+ 'imageharvest', # T167726
+ 'misspelling', # T94681
+ 'watchlist', # T77965
+ 'lonelypages', # T94680: uses exit code 1
]
_argument = 'simulate'
--
To view, visit https://gerrit.wikimedia.org/r/433361
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Ib2326d77fa92767a0cab582da396fdd6b0623140
Gerrit-Change-Number: 433361
Gerrit-PatchSet: 7
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: Dvorapa <dvorapa(a)seznam.cz>
Gerrit-Reviewer: Framawiki <framawiki(a)tools.wmflabs.org>
Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: Zoranzoki21 <zorandori4444(a)gmail.com>
Gerrit-Reviewer: jenkins-bot <>