jenkins-bot has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/524673 )
Change subject: [bugfix] Fix the comparison in archivebot
......................................................................
[bugfix] Fix the comparison in archivebot
Bug: T228587
Change-Id: I30da3c19521d24391924e3cb1c1d7ee20abb8d47
---
M scripts/archivebot.py
1 file changed, 1 insertion(+), 1 deletion(-)
Approvals:
Xqt: Looks good to me, approved
jenkins-bot: Verified
diff --git a/scripts/archivebot.py b/scripts/archivebot.py
index 97c0bb9..c0381f8 100755
--- a/scripts/archivebot.py
+++ b/scripts/archivebot.py
@@ -572,7 +572,7 @@
pywikibot.output('Looking for: {{%s}} in %s' % (self.tpl.title(),
self.page))
for tpl in self.page.raw_extracted_templates:
- if tpl[0] == self.tpl.title():
+ if tpl[0] == self.tpl.title(with_ns=False):
for item, value in tpl[1].items():
self.set_attr(item.strip(), value.strip())
break
--
To view, visit https://gerrit.wikimedia.org/r/524673
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I30da3c19521d24391924e3cb1c1d7ee20abb8d47
Gerrit-Change-Number: 524673
Gerrit-PatchSet: 3
Gerrit-Owner: Dvorapa <dvorapa(a)seznam.cz>
Gerrit-Reviewer: D3r1ck01 <xsavitar.wiki(a)aol.com>
Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot (75)
jenkins-bot has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/476274 )
Change subject: [bugfix] Increase the delay if maxlag >> retry-after
......................................................................
[bugfix] Increase the delay if maxlag >> retry-after
Normally retry-after value from response_headers is used for site access
throttling if maxlag triggers. But sometime that retry-after value is
too low compared to the lagtime. Increase the delay in this case.
use config.retry_wait as fallback value
doc updated
Bug: T210606
Change-Id: Ide3a434f2e57f1afbba038cd12b93731a21f45f0
---
M pywikibot/throttle.py
1 file changed, 10 insertions(+), 5 deletions(-)
Approvals:
Dvorapa: Looks good to me, approved
jenkins-bot: Verified
diff --git a/pywikibot/throttle.py b/pywikibot/throttle.py
index 23930ab..608f6dc 100644
--- a/pywikibot/throttle.py
+++ b/pywikibot/throttle.py
@@ -1,7 +1,7 @@
# -*- coding: utf-8 -*-
"""Mechanics to slow down wiki read and/or write rate."""
#
-# (C) Pywikibot team, 2008-2018
+# (C) Pywikibot team, 2008-2019
#
# Distributed under the terms of the MIT license.
#
@@ -286,9 +286,12 @@
"""Seize the throttle lock due to server lag.
Usually the self.retry-after value from response_header of the last
- request if available which will be used for wait time. Default value
- set by api and stored in self.retry_after is 5. If neither retry_after
- nor lagtime is set, fallback to 5.
+ request if available which will be used for wait time. Otherwise
+ lagtime from api maxlag is used. If neither retry_after nor lagtime is
+ set, fallback to config.retry_wait.
+
+ If the lagtime is disproportionately high compared to retry-after
+ value, the wait time will be increased.
This method is used by api.request. It will prevent any thread from
accessing this site.
@@ -300,7 +303,9 @@
"""
started = time.time()
with self.lock:
- waittime = self.retry_after or lagtime or 5
+ waittime = lagtime or config.retry_wait
+ if self.retry_after:
+ waittime = max(self.retry_after, waittime / 5)
# wait not more than retry_max seconds
delay = min(waittime, config.retry_max)
# account for any time we waited while acquiring the lock
--
To view, visit https://gerrit.wikimedia.org/r/476274
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Ide3a434f2e57f1afbba038cd12b93731a21f45f0
Gerrit-Change-Number: 476274
Gerrit-PatchSet: 5
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: Dalba <dalba.wiki(a)gmail.com>
Gerrit-Reviewer: Dvorapa <dvorapa(a)seznam.cz>
Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl>
Gerrit-Reviewer: Multichill <maarten(a)mdammers.nl>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot (75)
jenkins-bot has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/494898 )
Change subject: [bugfix] Namespace identifier can be ommitted for -start option
......................................................................
[bugfix] Namespace identifier can be ommitted for -start option
- in contrary to the documentation a Namespace identifier like 'File:'
or 'Image:' must be given with the -start command argument.
This patch solves this issue.
- handle global args first. This ensures the right site is used with
-start, -cat and -ref options
- check whether an namespace identifier is given with -start option and
remove it
- use allimages instead of allpages because images are used with this
script only
- print a warning if an unknown option is given and leave the scripts then
- finally return True in main() due to PEP8
Bug: T217824
Change-Id: I83a594947d8b205ca20b978b4c16e1cbb788b4a1
---
M scripts/checkimages.py
1 file changed, 30 insertions(+), 18 deletions(-)
Approvals:
Dvorapa: Looks good to me, approved
jenkins-bot: Verified
diff --git a/scripts/checkimages.py b/scripts/checkimages.py
index 08a4b55..54c1545 100755
--- a/scripts/checkimages.py
+++ b/scripts/checkimages.py
@@ -38,7 +38,7 @@
-skip[:#] The bot skip the first [:#] images (default: 0)
--start[:#] Use allpages() as generator
+-start[:#] Use allimages() as generator
(it starts already from File:[:#])
-cat[:#] Use a category as generator
@@ -93,11 +93,12 @@
import pywikibot
-from pywikibot import i18n
-from pywikibot import pagegenerators as pg
-
+from pywikibot.bot import suggest_help
from pywikibot.exceptions import ArgumentDeprecationWarning, NotEmailableError
from pywikibot.family import Family
+from pywikibot import i18n
+from pywikibot import pagegenerators as pg
+from pywikibot.site import Namespace
from pywikibot.tools import issue_deprecation_warning
###############################################################################
@@ -1528,9 +1529,12 @@
sendemailActive = False # Use the send-email
logFullError = True # Raise an error when the log is full
generator = None
+ unknown = [] # unknown parameters
- # Here below there are the parameters.
- for arg in pywikibot.handle_args(args):
+ local_args = pywikibot.handle_args(args)
+ site = pywikibot.Site()
+ # Here below there are the local parameters.
+ for arg in local_args:
if arg.startswith('-limit'):
if len(arg) == 6:
limit = int(pywikibot.input(
@@ -1597,9 +1601,11 @@
'From which page do you want to start?')
elif len(arg) > 6:
firstPageTitle = arg[7:]
- firstPageTitle = firstPageTitle.split(':')[1:]
- generator = pywikibot.Site().allpages(start=firstPageTitle,
- namespace=6)
+ namespaces = tuple(ns + ':'
+ for ns in site.namespace(Namespace.FILE, all))
+ if firstPageTitle.startswith(namespaces):
+ firstPageTitle = firstPageTitle.split(':')[1]
+ generator = site.allimages(start=firstPageTitle)
repeat = False
elif arg.startswith('-page'):
if len(arg) == 5:
@@ -1631,7 +1637,7 @@
catName = str(pywikibot.input('In which category do I work?'))
elif len(arg) > 4:
catName = str(arg[5:])
- catSelected = pywikibot.Category(pywikibot.Site(),
+ catSelected = pywikibot.Category(site,
'Category:{}'.format(catName))
generator = catSelected.articles(namespaces=[6])
repeat = False
@@ -1641,23 +1647,28 @@
'The references of what page should I parse?'))
elif len(arg) > 4:
refName = str(arg[5:])
- ref = pywikibot.Page(pywikibot.Site(), refName)
+ ref = pywikibot.Page(site, refName)
generator = ref.getReferences(namespaces=[6])
repeat = False
+ else:
+ unknown.append(arg)
if not generator:
normal = True
- site = pywikibot.Site()
skip = skip_number > 0
- # A little block-statement to ensure that the bot will not start with
- # en-parameters
+ # Ensure that the bot is localized and right command args are given
if site.code not in project_inserted:
- pywikibot.output('Your project is not supported by this script.\n'
- 'To allow your project in the script you have to '
- 'add a localization into the script and add your '
- 'project to the "project_inserted" list!')
+ additional_text = ('Your project is not supported by this script.\n'
+ 'To allow your project in the script you have to '
+ 'add a localization into the script and add your '
+ 'project to the "project_inserted" list!')
+ else:
+ additional_text = ''
+ if unknown or additional_text:
+ suggest_help(unknown_parameters=unknown,
+ additional_text=additional_text)
return False
# Reading the log of the new images if another generator is not given.
@@ -1716,6 +1727,7 @@
pywikibot.sleep(time_sleep)
else:
break
+ return True
if __name__ == '__main__':
--
To view, visit https://gerrit.wikimedia.org/r/494898
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I83a594947d8b205ca20b978b4c16e1cbb788b4a1
Gerrit-Change-Number: 494898
Gerrit-PatchSet: 7
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: D3r1ck01 <xsavitar.wiki(a)aol.com>
Gerrit-Reviewer: Dalba <dalba.wiki(a)gmail.com>
Gerrit-Reviewer: Dvorapa <dvorapa(a)seznam.cz>
Gerrit-Reviewer: Framawiki <framawiki(a)tools.wmflabs.org>
Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: Lokal Profil <andre.costa(a)wikimedia.se>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: Zoranzoki21 <zorandori4444(a)gmail.com>
Gerrit-Reviewer: jenkins-bot (75)
jenkins-bot has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/524641 )
Change subject: test(utils.py): redesign the timeout workflow
......................................................................
test(utils.py): redesign the timeout workflow
The implentation of timeout is complex and confusingly just returns
empty strings if a timeout occurs, causing T224364.
Use a new implementation for Python 2 which raises TimeoutError and
use the default one in Python 3.
In this design, infinite timeouts are indicated by None, not 0.
Change the script_tests accordingly.
Increase the timeout of test_one_similar_script to 10 seconds.
It occasionally failed for me with the previous 6s timeout.
Bug: T224364
Change-Id: Icc69d3df8ca41b86061f40fcaef0da7fa46d494e
---
M tests/aspects.py
M tests/pwb_tests.py
M tests/script_tests.py
M tests/utils.py
4 files changed, 22 insertions(+), 33 deletions(-)
Approvals:
Xqt: Looks good to me, approved
jenkins-bot: Verified
diff --git a/tests/aspects.py b/tests/aspects.py
index fa93d87..328a279 100644
--- a/tests/aspects.py
+++ b/tests/aspects.py
@@ -1490,7 +1490,7 @@
if self.orig_pywikibot_dir:
os.environ[str('PYWIKIBOT_DIR')] = self.orig_pywikibot_dir
- def _execute(self, args, data_in=None, timeout=0, error=None):
+ def _execute(self, args, data_in=None, timeout=None, error=None):
site = self.get_site()
args = args + ['-family:' + site.family.name,
diff --git a/tests/pwb_tests.py b/tests/pwb_tests.py
index 54f0b33..cac124f 100644
--- a/tests/pwb_tests.py
+++ b/tests/pwb_tests.py
@@ -16,8 +16,6 @@
import io
import sys
-from pywikibot.tools import PY2
-
from tests import join_tests_path, create_path_func
from tests.utils import execute, execute_pwb
from tests.aspects import unittest, PwbTestCase
@@ -91,7 +89,6 @@
self.assertEqual(stderr.readline().strip(),
'ERROR: pywikibot.py not found! Misspelling?')
- @unittest.skipIf(PY2, 'cannot be safely tested with Python 2 (T224364)')
def test_one_similar_script(self):
"""Test shell.py script call which gives one similar result."""
result = [
@@ -99,7 +96,7 @@
'NOTE: Starting the most similar script shell.py',
'in 5.0 seconds; type CTRL-C to stop.',
]
- stream = execute_pwb(['hello'], data_in=chr(3), timeout=6)
+ stream = execute_pwb(['hello'], data_in=chr(3), timeout=10)
stderr = io.StringIO(stream['stderr'])
with self.subTest(line=0):
self.assertEqual(stderr.readline().strip(), result[0])
diff --git a/tests/script_tests.py b/tests/script_tests.py
index ef3f18c..d82c938 100644
--- a/tests/script_tests.py
+++ b/tests/script_tests.py
@@ -234,9 +234,10 @@
data_in = script_input.get(script_name)
- timeout = 0
if is_autorun:
timeout = 5
+ else:
+ timeout = None
if self._results and script_name in self._results:
error = self._results[script_name]
diff --git a/tests/utils.py b/tests/utils.py
index 7fdaa1b..89f54e8 100644
--- a/tests/utils.py
+++ b/tests/utils.py
@@ -13,7 +13,6 @@
import re
from subprocess import PIPE, Popen
import sys
-import time
import traceback
import warnings
@@ -21,6 +20,8 @@
from collections.abc import Mapping
except ImportError: # Python 2.7
from collections import Mapping
+ from multiprocessing import TimeoutError
+ from threading import Timer
from types import ModuleType
try:
@@ -636,7 +637,7 @@
self._module.http = self._old_http
-def execute(command, data_in=None, timeout=0, error=None):
+def execute(command, data_in=None, timeout=None, error=None):
"""
Execute a command and capture outputs.
@@ -699,35 +700,25 @@
p.stdin.write(data_in.encode(config.console_encoding))
p.stdin.flush() # _communicate() otherwise has a broken pipe
- stderr_lines = b''
- waited = 0
- while (error or (waited < timeout)) and p.poll() is None:
- # In order to kill 'shell' and others early, read only a single
- # line per second, and kill the process as soon as the expected
- # output has been seen.
- # Additional lines will be collected later with p.communicate()
- if error:
- line = p.stderr.readline()
- stderr_lines += line
- if error in line.decode(config.console_encoding):
- break
- time.sleep(1)
- waited += 1
+ if PY2: # subprocess.communicate does not support timeout
+ def timeout_handler():
+ p.kill()
+ raise TimeoutError
- if (timeout or error) and p.poll() is None:
- p.kill()
-
- if p.poll() is not None:
- stderr_lines += p.stderr.read()
-
- data_out = p.communicate()
+ timer = Timer(timeout, timeout_handler)
+ timer.start()
+ try:
+ stdout_data, stderr_data = p.communicate()
+ finally:
+ timer.cancel()
+ else:
+ stdout_data, stderr_data = p.communicate(timeout=timeout)
return {'exit_code': p.returncode,
- 'stdout': data_out[0].decode(config.console_encoding),
- 'stderr': (stderr_lines + data_out[1])
- .decode(config.console_encoding)}
+ 'stdout': stdout_data.decode(config.console_encoding),
+ 'stderr': stderr_data.decode(config.console_encoding)}
-def execute_pwb(args, data_in=None, timeout=0, error=None, overrides=None):
+def execute_pwb(args, data_in=None, timeout=None, error=None, overrides=None):
"""
Execute the pwb.py script and capture outputs.
--
To view, visit https://gerrit.wikimedia.org/r/524641
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Icc69d3df8ca41b86061f40fcaef0da7fa46d494e
Gerrit-Change-Number: 524641
Gerrit-PatchSet: 8
Gerrit-Owner: Dalba <dalba.wiki(a)gmail.com>
Gerrit-Reviewer: Dalba <dalba.wiki(a)gmail.com>
Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot (75)
jenkins-bot has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/524640 )
Change subject: refactor(tests.utlis.execute): do not create an options dict
......................................................................
refactor(tests.utlis.execute): do not create an options dict
No need for that. It's easier to follow if the values are passed
directly to Popen.
Change-Id: Ibb3591c91554df2bf8d6f80519ee387d877fcde3
---
M tests/utils.py
1 file changed, 4 insertions(+), 10 deletions(-)
Approvals:
Xqt: Looks good to me, approved
jenkins-bot: Verified
diff --git a/tests/utils.py b/tests/utils.py
index 40f63bd..7fdaa1b 100644
--- a/tests/utils.py
+++ b/tests/utils.py
@@ -11,7 +11,7 @@
import json
import os
import re
-import subprocess
+from subprocess import PIPE, Popen
import sys
import time
import traceback
@@ -672,16 +672,10 @@
# Set EDITOR to an executable that ignores all arguments and does nothing.
env[str('EDITOR')] = str('call' if OSWIN32 else 'true')
-
- options = {
- 'stdout': subprocess.PIPE,
- 'stderr': subprocess.PIPE
- }
- if data_in is not None:
- options['stdin'] = subprocess.PIPE
-
try:
- p = subprocess.Popen(command, env=env, **options)
+ p = Popen(
+ command, env=env, stdout=PIPE, stderr=PIPE,
+ stdin=PIPE if data_in is not None else None)
except TypeError as e:
# Generate a more informative error
if OSWIN32 and PY2:
--
To view, visit https://gerrit.wikimedia.org/r/524640
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Ibb3591c91554df2bf8d6f80519ee387d877fcde3
Gerrit-Change-Number: 524640
Gerrit-PatchSet: 2
Gerrit-Owner: Dalba <dalba.wiki(a)gmail.com>
Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot (75)
jenkins-bot has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/516502 )
Change subject: [bugfix] Allow user with 0 contributions to be welcomed
......................................................................
[bugfix] Allow user with 0 contributions to be welcomed
Also validate option settings for numbers
Bug: T225434
Change-Id: Ia60bea2476a1aeae79729595e7cf87f3e590a7be
---
M scripts/welcome.py
1 file changed, 28 insertions(+), 27 deletions(-)
Approvals:
Dalba: Looks good to me, approved
jenkins-bot: Verified
diff --git a/scripts/welcome.py b/scripts/welcome.py
index f36beff..a9f3d7f 100755
--- a/scripts/welcome.py
+++ b/scripts/welcome.py
@@ -783,31 +783,26 @@
"""Check whether the user is to be skipped."""
if users.isBlocked():
showStatus(3)
- pywikibot.output('{} has been blocked!'
- .format(users.username))
+ pywikibot.output('{} has been blocked!'.format(users.username))
elif 'bot' in users.groups():
showStatus(3)
- pywikibot.output('{} is a bot!'
- .format(users.username))
+ pywikibot.output('{} is a bot!'.format(users.username))
elif 'bot' in users.username.lower():
showStatus(3)
pywikibot.output('{} might be a global bot!'
.format(users.username))
- elif users.editCount() == 0:
- if not globalvar.quiet:
+ elif users.editCount() < globalvar.attachEditCount:
+ if not users.editCount() == 0:
+ showStatus(1)
+ pywikibot.output('{0} has only {1} contributions.'
+ .format(users.username, users.editCount()))
+ elif not globalvar.quiet:
showStatus(1)
pywikibot.output('{} has no contributions.'
.format(users.username))
-
- elif users.editCount() < globalvar.attachEditCount:
- showStatus(1)
- pywikibot.output('{0} has only {1} contributions.'
- .format(users.username,
- users.editCount()))
-
else:
return super(WelcomeBot, self).skip_page(users)
@@ -971,17 +966,20 @@
for arg in pywikibot.handle_args(args):
arg, _, val = arg.partition(':')
if arg == '-edit':
- globalvar.attachEditCount = int(val or pywikibot.input(
- 'After how many edits would you like to welcome new users? '
- '(0 is allowed)'))
+ globalvar.attachEditCount = int(
+ val if val.isdigit() else pywikibot.input(
+ 'After how many edits would you like to welcome new users?'
+ ' (0 is allowed)'))
elif arg == '-timeoffset':
- globalvar.timeoffset = int(val or pywikibot.input(
- 'Which time offset (in minutes) for new users would you like '
- 'to use?'))
+ globalvar.timeoffset = int(
+ val if val.isdigit() else pywikibot.input(
+ 'Which time offset (in minutes) for new users would you '
+ 'like to use?'))
elif arg == '-time':
- globalvar.timeRecur = int(val or pywikibot.input(
- 'For how many seconds would you like to bot to sleep before '
- 'checking again?'))
+ globalvar.timeRecur = int(
+ val if val.isdigit() else pywikibot.input(
+ 'For how many seconds would you like to bot to sleep '
+ 'before checking again?'))
elif arg == '-offset':
_handle_offset(val)
elif arg == '-file':
@@ -993,12 +991,15 @@
'Which signature to use?')
globalvar.defaultSign += timeselected
elif arg == '-limit':
- globalvar.queryLimit = int(val or pywikibot.input(
- 'How many of the latest new users would you like to load?'))
+ globalvar.queryLimit = int(
+ val if val.isdigit() else pywikibot.input(
+ 'How many of the latest new users would you like to '
+ 'load?'))
elif arg == '-numberlog':
- globalvar.dumpToLog = int(val or pywikibot.input(
- 'After how many welcomed users would you like to update the '
- 'welcome log?'))
+ globalvar.dumpToLog = int(
+ val if val.isdigit() else pywikibot.input(
+ 'After how many welcomed users would you like to update '
+ 'the welcome log?'))
elif arg in mapping:
setattr(globalvar, *mapping[arg])
else:
--
To view, visit https://gerrit.wikimedia.org/r/516502
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Ia60bea2476a1aeae79729595e7cf87f3e590a7be
Gerrit-Change-Number: 516502
Gerrit-PatchSet: 4
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: D3r1ck01 <xsavitar.wiki(a)aol.com>
Gerrit-Reviewer: Dalba <dalba.wiki(a)gmail.com>
Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: Meno25 <meno25mail(a)gmail.com>
Gerrit-Reviewer: jenkins-bot (75)
jenkins-bot has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/524210 )
Change subject: fix(test_family): deprecate test_family
......................................................................
fix(test_family): deprecate test_family
We have two families that both define test.wikipedia.org and this
has created some issues. (see the tasks below)
test_family.py:
- deprecate this module
- replace the Family with a new class that inherits from the
wikipedia_family.Family
family.py:
- test_family.Family was the only family using the private
`_ignore_from_url` attribute. Remove all the usages of this attriube.
family_tests.py:
- remove the assertions of test family that do not apply now that
test family is inheriting from wikipedia family.
- remove usages of `_ignore_from_url` attribute.
pywikibot.__init__:
- to avoid the newly added deprecation warning of test family when
`pywikibot.Site(url='https://test.wikipedia.org/w/index.php')` is
used, prevent loading that family and let the code continue to
look for the right family (wikipedia).
generate_user_files_test.py:
- Update test_get_site_and_lang to use wikisource. (we could also just
change self.assertEqual(code, 'test') to self.assertEqual(code, 'en'),
but that will live the DeprecationWarning.
link_test.py:
- Use Site('test', 'wikipedia') instead of the deprecated
pywikibot.Site('test', 'test'). Note these two are still not equal.
Bug: T228375
Bug: T228300
Change-Id: I7b65fca0a451422e5acbe64f80549f6f49646482
---
M pywikibot/__init__.py
M pywikibot/families/test_family.py
M pywikibot/family.py
M tests/family_tests.py
M tests/generate_user_files_tests.py
M tests/link_tests.py
6 files changed, 18 insertions(+), 29 deletions(-)
Approvals:
Xqt: Looks good to me, approved
jenkins-bot: Verified
diff --git a/pywikibot/__init__.py b/pywikibot/__init__.py
index 7a6b267..662e746 100644
--- a/pywikibot/__init__.py
+++ b/pywikibot/__init__.py
@@ -1174,6 +1174,8 @@
# Iterate through all families and look, which does apply to
# the given URL
for fam in config.family_files:
+ if fam == 'test': # test_family.py is deprecated
+ continue
family = Family.load(fam)
code = family.from_url(url)
if code is not None:
diff --git a/pywikibot/families/test_family.py b/pywikibot/families/test_family.py
index a1ea3e0..51b4480 100644
--- a/pywikibot/families/test_family.py
+++ b/pywikibot/families/test_family.py
@@ -7,17 +7,17 @@
#
from __future__ import absolute_import, division, unicode_literals
-from pywikibot import family
+from pywikibot.families.wikipedia_family import Family
+from pywikibot.tools import issue_deprecation_warning
+
+issue_deprecation_warning(
+ 'test_family', 'wikipedia_family', since='20190718')
+# Also remove the ``if fam == 'test':`` condition in pywikibot.__init__
+# whenever this module is removed.
-# The test wikipedia family
-class Family(family.SingleSiteFamily, family.WikimediaFamily):
+class Family(Family):
"""Family class for test.wikipedia.org."""
name = 'test'
- domain = 'test.wikipedia.org'
- interwiki_forward = 'wikipedia'
-
- # 'test' family should resolve to be in the 'wikipedia' family
- _ignore_from_url = True
diff --git a/pywikibot/family.py b/pywikibot/family.py
index d34938a..421d70e 100644
--- a/pywikibot/family.py
+++ b/pywikibot/family.py
@@ -1264,9 +1264,6 @@
return config.site_interface
- # List of codes which aren't returned by from_url; True returns None always
- _ignore_from_url = []
-
def from_url(self, url):
"""
Return whether this family matches the given url.
@@ -1291,9 +1288,6 @@
which would work with the given URL.
@raises ValueError: When text is present after $1.
"""
- if self._ignore_from_url is True:
- return None
-
parsed = urlparse.urlparse(url)
if not re.match('(https?)?$', parsed.scheme):
return None
@@ -1316,8 +1310,6 @@
matched_sites = []
for code in chain(self.codes, getattr(self, 'test_codes', ())):
- if code in self._ignore_from_url:
- continue
if self._hostname(code)[1] == parsed.netloc:
# Use the code and family instead of the url
# This is only creating a Site instance if domain matches
diff --git a/tests/family_tests.py b/tests/family_tests.py
index 8361090..aad18c9 100644
--- a/tests/family_tests.py
+++ b/tests/family_tests.py
@@ -124,8 +124,6 @@
family = Family.load('test')
self.assertIn('dk', family.obsolete)
self.assertIn('dk', family.interwiki_replacements)
- self.assertEqual(family.obsolete, family.interwiki_replacements)
- self.assertEqual(family.interwiki_removals, set())
def test_set_obsolete(self):
"""Test obsolete can be set."""
@@ -251,11 +249,7 @@
family.path(code)))
# Families can switch off if they want to be detected using
# URL. This applies for test:test (there is test:wikipedia)
- if (family._ignore_from_url
- or code in family._ignore_from_url):
- self.assertIsNone(family.from_url(url))
- else:
- self.assertEqual(family.from_url(url), code)
+ self.assertEqual(family.from_url(url), code)
class TestOldFamilyMethod(DeprecationTestCase):
diff --git a/tests/generate_user_files_tests.py b/tests/generate_user_files_tests.py
index f0625cb..dd0efe4 100644
--- a/tests/generate_user_files_tests.py
+++ b/tests/generate_user_files_tests.py
@@ -61,10 +61,11 @@
def test_get_site_and_lang(self):
"""Test get_site_and_lang function with parameters."""
family, code, user = guf.get_site_and_lang(
- default_family='test', default_lang='foo', default_username='bar',
+ default_family='wikisource', default_lang='foo',
+ default_username='bar',
force=True)
- self.assertEqual(family, 'test')
- self.assertEqual(code, 'test')
+ self.assertEqual(family, 'wikisource')
+ self.assertEqual(code, 'en')
self.assertEqual(user, 'bar')
def test_parse_sections(self):
diff --git a/tests/link_tests.py b/tests/link_tests.py
index 180dcf5..1307ac8 100644
--- a/tests/link_tests.py
+++ b/tests/link_tests.py
@@ -528,7 +528,7 @@
'code': 'en'
},
'test.wp': {
- 'family': 'test',
+ 'family': 'wikipedia',
'code': 'test'
},
}
@@ -891,7 +891,7 @@
config.family = 'wikipedia'
link = Link('wikidata:testwiki:Q6')
link.parse()
- self.assertEqual(link.site, pywikibot.Site('test', 'test'))
+ self.assertEqual(link.site, pywikibot.Site('test', 'wikipedia'))
self.assertEqual(link.title, 'Q6')
self.assertEqual(link.namespace, 0)
@@ -901,7 +901,7 @@
config.family = 'wikipedia'
link = Link('wikidata:testwiki:Talk:Q6')
link.parse()
- self.assertEqual(link.site, pywikibot.Site('test', 'test'))
+ self.assertEqual(link.site, pywikibot.Site('test', 'wikipedia'))
self.assertEqual(link.title, 'Q6')
self.assertEqual(link.namespace, 1)
--
To view, visit https://gerrit.wikimedia.org/r/524210
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I7b65fca0a451422e5acbe64f80549f6f49646482
Gerrit-Change-Number: 524210
Gerrit-PatchSet: 8
Gerrit-Owner: Dalba <dalba.wiki(a)gmail.com>
Gerrit-Reviewer: Dalba <dalba.wiki(a)gmail.com>
Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot (75)
jenkins-bot has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/524209 )
Change subject: test(test_family_load_valid): do not care if languages_by_size is empty
......................................................................
test(test_family_load_valid): do not care if languages_by_size is empty
I've created a few custom family files, e.g. for wpbeta, but they fail
this test because the autogenerated family files does not have a
languages_by_size attribute.
We have fixed and changed this part of test several times during these
years. I think it is not worth maintaning and is safe to remove.
Change-Id: Icffca034d26a82d9114da6c0668ac32bc07c060e
---
M tests/family_tests.py
1 file changed, 0 insertions(+), 4 deletions(-)
Approvals:
Xqt: Looks good to me, approved
jenkins-bot: Verified
diff --git a/tests/family_tests.py b/tests/family_tests.py
index 8361090..6e24bee 100644
--- a/tests/family_tests.py
+++ b/tests/family_tests.py
@@ -52,10 +52,6 @@
self.assertEqual(f.name, name)
self.assertIsInstance(f.languages_by_size, list)
self.assertGreaterEqual(set(f.langs), set(f.languages_by_size))
- if len(f.langs) > 2 and f.name not in {
- 'wikimediachapter', 'vikidia', 'wikidata', 'wikimania'
- }:
- self.assertNotEqual(f.languages_by_size, [])
if isinstance(f, SingleSiteFamily):
self.assertIsNotNone(f.code)
self.assertIsNotNone(f.domain)
--
To view, visit https://gerrit.wikimedia.org/r/524209
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Icffca034d26a82d9114da6c0668ac32bc07c060e
Gerrit-Change-Number: 524209
Gerrit-PatchSet: 2
Gerrit-Owner: Dalba <dalba.wiki(a)gmail.com>
Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot (75)