jenkins-bot has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/517186 )
Change subject: [FEAT] Use generator from target site for transfer pages
......................................................................
[FEAT] Use generator from target site for transfer pages
For example copy wanted templates of German Wikipedia
from English Wikipedia to German Wikipedia
The command line is:
pwb transferbot -family:wikipedia -lang:en -tolang:de -wantedtemplates -target
Bug: T223713
Change-Id: Ie3f29c0b4a8be1ef6cd71bdee48f8feeb5a963cf
---
M scripts/transferbot.py
1 file changed, 40 insertions(+), 18 deletions(-)
Approvals:
Framawiki: Looks good to me, approved
jenkins-bot: Verified
diff --git a/scripts/transferbot.py b/scripts/transferbot.py
index a3e3e0a..eff9df1 100755
--- a/scripts/transferbot.py
+++ b/scripts/transferbot.py
@@ -16,6 +16,9 @@
-overwrite: Existing pages are skipped by default. Use this option to
overwrite pages.
+-target Use page generator of the target site
+
+
Internal links are *not* repaired!
Pages to work on can be specified using any of:
@@ -37,6 +40,10 @@
python pwb.py transferbot -family:wikipedia -lang:en \
-tofamily:wiktionary -tolang:ar -page:"Template:Query service"
+Copy 10 wanted templates of German Wikipedia from English Wikipedia to German
+ python pwb.py transferbot -family:wikipedia -lang:en \
+ -tolang:de -wantedtemplates:10 -target
+
"""
#
# (C) Merlijn van Deen, 2014
@@ -69,14 +76,10 @@
tofamily = fromsite.family.name
prefix = ''
overwrite = False
+ target = False
gen_args = []
- gen_factory = pagegenerators.GeneratorFactory()
-
for arg in local_args:
- if gen_factory.handleArg(arg):
- gen_args.append(arg)
- continue
if arg.startswith('-tofamily'):
tofamily = arg[len('-tofamily:'):]
elif arg.startswith('-tolang'):
@@ -85,34 +88,53 @@
prefix = arg[len('-prefix:'):]
elif arg == '-overwrite':
overwrite = True
-
- gen = gen_factory.getCombinedGenerator()
+ elif arg == '-target':
+ target = True
+ else:
+ gen_args.append(arg)
tosite = pywikibot.Site(tolang, tofamily)
additional_text = ('Target site not different from source site.'
if fromsite == tosite else '')
+ gen_factory = pagegenerators.GeneratorFactory(site=tosite if target
+ else fromsite)
+ unknown_args = [arg for arg in gen_args if not gen_factory.handleArg(arg)]
+
+ if unknown_args:
+ for item in unknown_args:
+ gen_args.remove(item)
+
+ gen = gen_factory.getCombinedGenerator()
+
+ suggest_help(missing_generator=not gen,
+ additional_text=additional_text,
+ unknown_parameters=unknown_args)
if additional_text or not gen:
- suggest_help(missing_generator=not gen,
- additional_text=additional_text)
return
gen_args = ' '.join(gen_args)
pywikibot.output("""
Page transfer configuration
---------------------------
- Source: %(fromsite)r
- Target: %(tosite)r
+ Source: {fromsite}
+ Target: {tosite}
- Generator of pages to transfer: %(gen_args)s
-
- Prefix for transferred pages: %(prefix)s
- """ % {'fromsite': fromsite, 'tosite': tosite,
- 'gen_args': gen_args, 'prefix': prefix if prefix else '(none)'})
+ Generator of pages to transfer: {gen_args}
+ {target}
+ Prefix for transferred pages: {prefix}
+ """.format(fromsite=fromsite, tosite=tosite, gen_args=gen_args,
+ prefix=prefix if prefix else '(none)',
+ target='from target site\n' if target else ''))
for page in gen:
- target_title = (prefix + page.namespace().canonical_prefix()
- + page.title(with_ns=False))
+ title = page.namespace().canonical_prefix() + page.title(with_ns=False)
+ if target:
+ # page is at target site, fetch it from source
+ target_title = prefix + page.title()
+ page = pywikibot.Page(fromsite, title)
+ else:
+ target_title = (prefix + title)
targetpage = pywikibot.Page(tosite, target_title)
edithistpage = pywikibot.Page(tosite, target_title + '/edithistory')
--
To view, visit https://gerrit.wikimedia.org/r/517186
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Ie3f29c0b4a8be1ef6cd71bdee48f8feeb5a963cf
Gerrit-Change-Number: 517186
Gerrit-PatchSet: 5
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: D3r1ck01 <xsavitar.wiki(a)aol.com>
Gerrit-Reviewer: Dalba <dalba.wiki(a)gmail.com>
Gerrit-Reviewer: Dvorapa <dvorapa(a)seznam.cz>
Gerrit-Reviewer: Framawiki <framawiki(a)tools.wmflabs.org>
Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: Nicolas NALLET <contact(a)semantiki.fr>
Gerrit-Reviewer: jenkins-bot (75)
jenkins-bot has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/551798 )
Change subject: [IMPR] Use a set for _locked_pages lookup
......................................................................
[IMPR] Use a set for _locked_pages lookup
Change-Id: I2781e3dab1d261364db149473b321b637054f64d
---
M pywikibot/site.py
1 file changed, 3 insertions(+), 3 deletions(-)
Approvals:
D3r1ck01: Looks good to me, but someone else must approve
Framawiki: Looks good to me, approved
jenkins-bot: Verified
diff --git a/pywikibot/site.py b/pywikibot/site.py
index ffa61ff..29493b1 100644
--- a/pywikibot/site.py
+++ b/pywikibot/site.py
@@ -789,7 +789,7 @@
# following are for use with lock_page and unlock_page methods
self._pagemutex = threading.Lock()
- self._locked_pages = []
+ self._locked_pages = set()
@deprecated(since='20141225')
def has_api(self):
@@ -1085,7 +1085,7 @@
time.sleep(.25)
self._pagemutex.acquire()
- self._locked_pages.append(title)
+ self._locked_pages.add(title)
finally:
# time.sleep may raise an exception from signal handler (eg:
# KeyboardInterrupt) while the lock is released, and there is no
@@ -1107,7 +1107,7 @@
"""
self._pagemutex.acquire()
try:
- self._locked_pages.remove(page.title(with_section=False))
+ self._locked_pages.discard(page.title(with_section=False))
finally:
self._pagemutex.release()
--
To view, visit https://gerrit.wikimedia.org/r/551798
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I2781e3dab1d261364db149473b321b637054f64d
Gerrit-Change-Number: 551798
Gerrit-PatchSet: 1
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: D3r1ck01 <xsavitar.wiki(a)aol.com>
Gerrit-Reviewer: Dalba <dalba.wiki(a)gmail.com>
Gerrit-Reviewer: Dvorapa <dvorapa(a)seznam.cz>
Gerrit-Reviewer: Framawiki <framawiki(a)tools.wmflabs.org>
Gerrit-Reviewer: jenkins-bot (75)
jenkins-bot has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/538257 )
Change subject: [IMPR] derive LoginStatus from IntEnum
......................................................................
[IMPR] derive LoginStatus from IntEnum
Removing name class method might be a breaking change
but I think this is never used elsewhere and can easily
replace by the IntEnum name attribute:
Instead LoginStatus.name(0) you may use LoginStatus(0).name
Bug: T213287
Change-Id: Id92f64671bde4c19463f516e1f273a8d33c67dcb
---
M pywikibot/site.py
1 file changed, 11 insertions(+), 19 deletions(-)
Approvals:
Huji: Looks good to me, approved
jenkins-bot: Verified
diff --git a/pywikibot/site.py b/pywikibot/site.py
index bd8d070..283133e 100644
--- a/pywikibot/site.py
+++ b/pywikibot/site.py
@@ -31,6 +31,7 @@
except ImportError: # Python 2.7
from collections import Iterable, Container, Mapping
from collections import namedtuple
+from enum import IntEnum
from warnings import warn
import pywikibot
@@ -101,18 +102,22 @@
"""Page cannot be reserved for writing due to existing lock."""
-class LoginStatus(object):
+class LoginStatus(IntEnum):
"""
Enum for Login statuses.
>>> LoginStatus.NOT_ATTEMPTED
- -3
- >>> LoginStatus.AS_USER
+ LoginStatus(-3)
+ >>> LoginStatus.IN_PROGRESS.value
+ -2
+ >>> LoginStatus.NOT_LOGGED_IN.name
+ NOT_LOGGED_IN
+ >>> int(LoginStatus.AS_USER)
0
- >>> LoginStatus.name(-3)
+ >>> LoginStatus(-3).name
'NOT_ATTEMPTED'
- >>> LoginStatus.name(0)
+ >>> LoginStatus(0).name
'AS_USER'
"""
@@ -122,22 +127,9 @@
AS_USER = 0
AS_SYSOP = 1
- @classmethod
- def name(cls, search_value):
- """Return the name of a LoginStatus by it's value."""
- for key, value in cls.__dict__.items():
- if key == key.upper() and value == search_value:
- return key
- raise KeyError('Value %r could not be found in this enum'
- % search_value)
-
- def __init__(self, state):
- """Initializer."""
- self.state = state
-
def __repr__(self):
"""Return internal representation."""
- return 'LoginStatus(%s)' % (LoginStatus.name(self.state))
+ return 'LoginStatus({})'.format(self)
Family = redirect_func(pywikibot.family.Family.load,
--
To view, visit https://gerrit.wikimedia.org/r/538257
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Id92f64671bde4c19463f516e1f273a8d33c67dcb
Gerrit-Change-Number: 538257
Gerrit-PatchSet: 3
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: Huji <huji.huji(a)gmail.com>
Gerrit-Reviewer: jenkins-bot (75)
jenkins-bot has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/538244 )
Change subject: [Python3] require enum34 for Python 2.7
......................................................................
[Python3] require enum34 for Python 2.7
Bug: T213287
Change-Id: I25916860cc19ae4a9b52b432a021798adf3149df
---
M HISTORY.rst
M requirements.txt
M setup.py
3 files changed, 3 insertions(+), 2 deletions(-)
Approvals:
Huji: Looks good to me, approved
jenkins-bot: Verified
diff --git a/HISTORY.rst b/HISTORY.rst
index 02893b9..27968bc 100644
--- a/HISTORY.rst
+++ b/HISTORY.rst
@@ -4,6 +4,7 @@
Current release
---------------
+* enum34 package is mandatory for Python 2.7
* Implement deletedrevisions api call (T75370)
* assert_valid_iter_params may raise AssertionError instead of pywikibot.Error (T233582)
* Upcast getRedirectTarget result and return the appropriate page subclass (T233392)
diff --git a/requirements.txt b/requirements.txt
index 5f56895..c666b1e 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -20,7 +20,7 @@
# mandatory; see README.conversion.txt
requests>=2.20.0
-
+enum34
# requests security extra
requests[security] ; python_full_version > '2.7.6' and python_full_version < '2.7.9'
diff --git a/setup.py b/setup.py
index 9c4b393..de86399 100644
--- a/setup.py
+++ b/setup.py
@@ -94,7 +94,7 @@
extra_deps.update(script_deps)
# ------- setup install_requires ------- #
-dependencies = ['requests>=2.20.0']
+dependencies = ['requests>=2.20.0', 'enum34>=1.1.6; python_version < "3"']
# tools.ip does not have a hard dependency on an IP address module,
# as it falls back to using regexes if one is not available.
# The functional backport of py3 ipaddress is acceptable:
--
To view, visit https://gerrit.wikimedia.org/r/538244
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I25916860cc19ae4a9b52b432a021798adf3149df
Gerrit-Change-Number: 538244
Gerrit-PatchSet: 2
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: Dvorapa <dvorapa(a)seznam.cz>
Gerrit-Reviewer: Huji <huji.huji(a)gmail.com>
Gerrit-Reviewer: jenkins-bot (75)
jenkins-bot has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/548768 )
Change subject: [tests] Use allowed_failure with Python 2 behaviour
......................................................................
[tests] Use allowed_failure with Python 2 behaviour
With Python 3 unittest.expectedFailure fails tests if they passes unexpectedly.
In some circumstances tests fail or pass if the test parameters aren't
deterministic. In such cases the test suite should be able to pass like
it does with Python 2.
- Don't care about exceptions other than AssertionError; they should
always fail.
- Unfortunately this decorator does not work with subtests
- Remove deprecation warnings and allow this decorator to be used in
such cases described above
- Additional documentation
Bug: T223030
Bug: T233484
Change-Id: Iab611d29c655b49ad442384e3c8c1edcfeccca9c
---
M tests/generate_family_files_tests.py
M tests/utils.py
2 files changed, 17 insertions(+), 15 deletions(-)
Approvals:
Xqt: Looks good to me, approved
jenkins-bot: Verified
diff --git a/tests/generate_family_files_tests.py b/tests/generate_family_files_tests.py
index 7c4f008..54deeaa 100644
--- a/tests/generate_family_files_tests.py
+++ b/tests/generate_family_files_tests.py
@@ -12,6 +12,7 @@
from pywikibot import Site
from tests.aspects import unittest, DefaultSiteTestCase
+from tests.utils import allowed_failure
import generate_family_file
@@ -52,7 +53,7 @@
self.assertIsInstance(self.generator_instance.wikis, dict)
self.assertIsInstance(self.generator_instance.langs, list)
- @unittest.expectedFailure # T194138
+ @allowed_failure # T194138
def test_attributes_after_run(self):
"""Test FamilyFileGenerator attributes after run()."""
self.generator_instance.run()
@@ -61,9 +62,8 @@
self.assertIn(lang, self.generator_instance.wikis)
for i in range(10):
lang = choice(self.generator_instance.langs)
- with self.subTest(lang=lang['prefix']):
- site = Site(url=lang['url'])
- self.assertEqual(site.lang, lang['prefix'])
+ site = Site(url=lang['url'])
+ self.assertEqual(site.lang, lang['prefix'])
if __name__ == '__main__': # pragma: no cover
diff --git a/tests/utils.py b/tests/utils.py
index 76b4091..9b57c5a 100644
--- a/tests/utils.py
+++ b/tests/utils.py
@@ -38,7 +38,6 @@
from pywikibot.data.api import Request as _original_Request
from pywikibot.site import Namespace
from pywikibot.tools import (
- deprecated,
PY2, PYTHON_VERSION,
UnicodeType as unicode,
)
@@ -73,23 +72,27 @@
return lambda orig: orig
-@deprecated('unittest.expectedFailure', since='20190512')
def allowed_failure(func):
"""
Unit test decorator to allow failure.
- Test runners each have different interpretations of what should be
- the result of an @expectedFailure test if it succeeds. Some consider
- it to be a pass; others a failure.
+ This decorator runs the test and, if it is an Assertion failure,
+ reports the result and considers it a skipped test. This is a
+ similar behaviour like expectedFailure in Python 2. Passing a test
+ will not count as failure (unexpected success) which Python 3 does.
- This decorator runs the test and, if it is a failure, reports the result
- and considers it a skipped test.
+ This decorator should be used if a test passes or fails due to
+ random test parameters. If tests fails deterministic expectedFailure
+ decorator should be used instead.
+
+ @note: This decorator does not support subTest content manager.
"""
@wraps(func)
def wrapper(*args, **kwargs):
try:
func(*args, **kwargs)
except AssertionError:
+ pywikibot.exception(tb=False)
tb = traceback.extract_tb(sys.exc_info()[2])
for depth, line in enumerate(tb):
if re.match('assert[A-Z]', line[2]):
@@ -97,9 +100,6 @@
tb = traceback.format_list(tb[:depth])
pywikibot.error('\n' + ''.join(tb)[:-1]) # remove \n at the end
raise unittest.SkipTest('Test is allowed to fail.')
- except Exception:
- pywikibot.exception(tb=True)
- raise unittest.SkipTest('Test is allowed to fail.')
if PY2:
return unittest.expectedFailure(func)
@@ -107,11 +107,13 @@
return wrapper
-@deprecated('expected_failure_if', since='20190512')
def allowed_failure_if(expect):
"""
Unit test decorator to allow failure under conditions.
+ See allowed_failure method for more details.
+
+ @note: This decorator does not support subTest content manager.
@param expect: Flag to check if failure is allowed
@type expect: bool
"""
--
To view, visit https://gerrit.wikimedia.org/r/548768
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Iab611d29c655b49ad442384e3c8c1edcfeccca9c
Gerrit-Change-Number: 548768
Gerrit-PatchSet: 3
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: Dvorapa <dvorapa(a)seznam.cz>
Gerrit-Reviewer: Framawiki <framawiki(a)tools.wmflabs.org>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot (75)
Gerrit-CC: Mpaa <mpaa.wiki(a)gmail.com>
jenkins-bot has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/551866 )
Change subject: [bugfix] call LoginManager with keyword arguments
......................................................................
[bugfix] call LoginManager with keyword arguments
Bug: T237501
Change-Id: I498b581aa92e3121ee5d13d19ee93eed66663697
---
M pywikibot/login.py
1 file changed, 2 insertions(+), 1 deletion(-)
Approvals:
Xqt: Looks good to me, approved
jenkins-bot: Verified
diff --git a/pywikibot/login.py b/pywikibot/login.py
index 389ac12..5d0f51c 100644
--- a/pywikibot/login.py
+++ b/pywikibot/login.py
@@ -377,7 +377,8 @@
if isinstance(mwoauth, ImportError):
raise OAuthImpossible('mwoauth is not installed: %s.' % mwoauth)
assert password is not None and user is not None
- super(OauthLoginManager, self).__init__(None, False, site, None)
+ super(OauthLoginManager, self).__init__(password=None, site=site,
+ user=None)
if self.password:
pywikibot.warn('Password exists in password file for %s:%s.'
'Password is unnecessary and should be removed '
--
To view, visit https://gerrit.wikimedia.org/r/551866
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I498b581aa92e3121ee5d13d19ee93eed66663697
Gerrit-Change-Number: 551866
Gerrit-PatchSet: 1
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: Huji <huji.huji(a)gmail.com>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot (75)
jenkins-bot has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/550928 )
Change subject: [doc] Fix typo in transferbot doc
......................................................................
[doc] Fix typo in transferbot doc
Change-Id: Ie61673a93b31fdc4df90678bad6996f4c757515c
---
M scripts/transferbot.py
1 file changed, 1 insertion(+), 1 deletion(-)
Approvals:
Xqt: Looks good to me, approved
jenkins-bot: Verified
diff --git a/scripts/transferbot.py b/scripts/transferbot.py
index 590bc83..a3e3e0a 100755
--- a/scripts/transferbot.py
+++ b/scripts/transferbot.py
@@ -13,7 +13,7 @@
-prefix: Page prefix on the new site.
--overwrite: Existing pages are skipped by default. Use his option to
+-overwrite: Existing pages are skipped by default. Use this option to
overwrite pages.
Internal links are *not* repaired!
--
To view, visit https://gerrit.wikimedia.org/r/550928
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Ie61673a93b31fdc4df90678bad6996f4c757515c
Gerrit-Change-Number: 550928
Gerrit-PatchSet: 2
Gerrit-Owner: Dvorapa <dvorapa(a)seznam.cz>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot (75)
jenkins-bot has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/550385 )
Change subject: component: Deriving from tools.UnicodeMixin to BaseLink
......................................................................
component: Deriving from tools.UnicodeMixin to BaseLink
Deleted the __str__ method in BaseLink and the if else
statement differenciating between PY2 and PY3. The
__str__ method is now being derived from UnicodeMixin
and if using PY2 the mixin class encodes to utf-8.
Bug: T223894
Change-Id: I0318ff8c106e340b4d9028a9759af2835e7941fe
---
M pywikibot/page.py
1 file changed, 1 insertion(+), 14 deletions(-)
Approvals:
Zhuyifei1999: Looks good to me, approved
jenkins-bot: Verified
diff --git a/pywikibot/page.py b/pywikibot/page.py
index 5fa4c69..b889df3 100644
--- a/pywikibot/page.py
+++ b/pywikibot/page.py
@@ -5703,7 +5703,7 @@
return self.__dict__ == other.__dict__
-class BaseLink(ComparableMixin):
+class BaseLink(UnicodeMixin, ComparableMixin):
"""
A MediaWiki link (local or interwiki).
@@ -5865,19 +5865,6 @@
return '[[%s:%s]]' % (self.site.code, title)
return '[[%s:%s:%s]]' % (self.site.family.name, self.site.code, title)
- if not PY2:
- def __str__(self):
- """Return a string representation."""
- return self.__unicode__()
- else: # PY2
- def __str__(self):
- """Return a string representation."""
- return self.__bytes__()
-
- def __bytes__(self):
- """Return a byte representation."""
- return self.astext().encode('ascii', 'backslashreplace')
-
def _cmpkey(self):
"""
Key for comparison of BaseLink objects.
--
To view, visit https://gerrit.wikimedia.org/r/550385
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I0318ff8c106e340b4d9028a9759af2835e7941fe
Gerrit-Change-Number: 550385
Gerrit-PatchSet: 2
Gerrit-Owner: Zack Khalid <zkhalido(a)msudenver.edu>
Gerrit-Reviewer: Zhuyifei1999 <zhuyifei1999(a)gmail.com>
Gerrit-Reviewer: jenkins-bot (75)
Gerrit-CC: Welcome, new contributor! <ssethi(a)wikimedia.org>
jenkins-bot has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/548817 )
Change subject: component: docstring update patch for comments
......................................................................
component: docstring update patch for comments
Update per feedback
Bug: T184115
Change-Id: I3047f6b880308ee8adfb6d931916958a76646255
---
M pywikibot/login.py
M scripts/add_text.py
M scripts/category.py
M scripts/imagetransfer.py
4 files changed, 94 insertions(+), 8 deletions(-)
Approvals:
Zhuyifei1999: Looks good to me, approved
jenkins-bot: Verified
diff --git a/pywikibot/login.py b/pywikibot/login.py
index 9b9ed2f..389ac12 100644
--- a/pywikibot/login.py
+++ b/pywikibot/login.py
@@ -173,9 +173,10 @@
"""
Store cookie data.
- The argument data is the raw data, as returned by getCookie().
+ @param data: The raw data as returned by getCookie()
+ @type data: str
- Returns nothing.
+ @return: None
"""
# THIS IS OVERRIDDEN IN data/api.py
filename = config.datafilepath('pywikibot.lwp')
diff --git a/scripts/add_text.py b/scripts/add_text.py
index 386dd44..3eec3a8 100755
--- a/scripts/add_text.py
+++ b/scripts/add_text.py
@@ -76,7 +76,19 @@
def get_text(page, old, create):
- """Get the old text."""
+ """
+ Get text on page. If old is not None, return old.
+
+ @param page: The page to get text from
+ @type page: pywikibot.page.BasePage
+ @param old: If not None, this parameter is returned instead
+ of fetching text from the page
+ @type old: str
+ @param create: Create the page if it doesn't exist
+ @type create: bool
+ @return: The page's text or old parameter if not None
+ @rtype: str
+ """
if old is None:
try:
text = page.get()
@@ -98,7 +110,23 @@
def put_text(page, new, summary, count, asynchronous=False):
- """Save the new text."""
+ """
+ Save the new text.
+
+ @param page: The page to update and save
+ @type page: pywikibot.page.BasePage
+ @param new: The new text for the page
+ @type new: str
+ @param summary: Summary of page changes.
+ @type summary: str
+ @param count: Maximum num attempts to reach the server
+ @type count: int
+ @param asynchronous: Save the page asynchronously
+ @type asynchronous: bool
+ @return: True if successful, False if unsuccessful, None if
+ waiting for server
+ @rtype: bool / None
+ """
page.text = new
try:
page.save(summary=summary, asynchronous=asynchronous,
@@ -132,7 +160,34 @@
"""
Add text to a page.
- @rtype: tuple of (text, newtext, always)
+ @param page: The page to add text to
+ @type page: pywikibot.page.BasePage
+ @param addText: Text to add
+ @type addText: str
+ @param summary: Summary of changes. If None, beginning of addText is used.
+ @type summary: str
+ @param regexSkip: Abort if text on page matches
+ @type regexSkip: str
+ @param regexSkipUrl: Abort if full url matches
+ @type regexSkipUrl: str
+ @param always: Always add text without user confirmation
+ @type always: bool
+ @param up: If True, add text to top of page, else add at bottom.
+ @type up: bool
+ @param putText: If True, save changes to the page, else return
+ (text, newtext, always)
+ @type putText: bool
+ @param oldTextGiven: If None fetch page text, else use this text
+ @type oldTextGiven: str
+ @param reorderEnabled: If True place text above categories and
+ interwiki, else place at page bottom. No effect if up = False.
+ @type reorderEnabled: bool
+ @param create: Create page if it does not exist
+ @type create: bool
+ @return: If putText=True: (success, success, always)
+ else: (text, newtext, always)
+ @rtype: Tuple of (bool, bool, bool) or (str, str, bool)
+ depending on value of putText parameter
"""
site = page.site
if not summary:
diff --git a/scripts/category.py b/scripts/category.py
index e07d71e..cc67ebc 100755
--- a/scripts/category.py
+++ b/scripts/category.py
@@ -651,7 +651,8 @@
self.move_comment = move_comment if move_comment else self.comment
def run(self):
- """The main bot function that does all the work.
+ """
+ The main bot function that does all the work.
For readability it is split into several helper functions:
- _movecat()
@@ -718,6 +719,12 @@
Do not use this function from outside the class. Automatically marks
the pages if they can't be removed due to missing permissions.
+
+ @param moved_page: Category page to delete
+ @param moved_talk: Talk page to delete
+ @type moved_page: pywikibot.page.BasePage
+ @type moved_talk: pywikibot.page.BasePage
+
"""
if moved_page and self.oldcat.exists():
self.oldcat.delete(self.deletion_comment, not self.batch,
@@ -755,7 +762,19 @@
@staticmethod
def check_move(name, old_page, new_page):
- """Return if the old page can be safely moved to the new page."""
+ """Return if the old page can be safely moved to the new page.
+
+ @param name: Title of the new page
+ @type name: str
+ @param old_page: Page to be moved
+ @type old_page: pywikibot.page.BasePage
+ @param new_page: Page to be moved to
+ @type new_page: pywikibot.page.BasePage
+ @return: True if possible to move page, False if not page move
+ not possible
+ @rtype: bool
+
+ """
move_possible = True
if new_page and new_page.exists():
pywikibot.warning("The {0} target '{1}' already exists."
diff --git a/scripts/imagetransfer.py b/scripts/imagetransfer.py
index 5968112..f89e8ba 100755
--- a/scripts/imagetransfer.py
+++ b/scripts/imagetransfer.py
@@ -126,7 +126,18 @@
def __init__(self, generator, targetSite=None, interwiki=False,
keep_name=False, ignore_warning=False):
- """Initializer."""
+ """Initializer.
+
+ @param generator: the pages to work on
+ @type generator: iterable
+ @param targetSite: Site to send image to, default none
+ @type targetSite: pywikibot.site.APISite
+ @param interwiki: Look for images in interwiki links, default false
+ @type interwiki: boolean
+ @param keep_name: Keep the filename and do not verify description
+ while replacing, default false
+ @type keep_name: boolean
+ """
self.generator = generator
self.interwiki = interwiki
self.targetSite = targetSite
--
To view, visit https://gerrit.wikimedia.org/r/548817
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I3047f6b880308ee8adfb6d931916958a76646255
Gerrit-Change-Number: 548817
Gerrit-PatchSet: 4
Gerrit-Owner: Morgan11235 <morgan11235(a)hotmail.com>
Gerrit-Reviewer: Zhuyifei1999 <zhuyifei1999(a)gmail.com>
Gerrit-Reviewer: jenkins-bot (75)
Gerrit-CC: Dvorapa <dvorapa(a)seznam.cz>
Gerrit-CC: Xqt <info(a)gno.de>
Gerrit-CC: Zoranzoki21 <zorandori4444(a)gmail.com>
jenkins-bot has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/375448 )
Change subject: Make _flush aware of _putthread ongoing tasks
......................................................................
Make _flush aware of _putthread ongoing tasks
Make sure _flush() is aware if _putthread() is idle or working on a
task.
Relying on page_put_queue.qsize() > 0 only, as a condition to keep the while
loop alive is not safe as the last item in the queue can be fetched by
async_manager() before page_put_queue.qsize() is checked and then found empty.
In this case _flush() returns before all pending requests are served.
A new Queue is introduced as a form of synchronisation between
MainThread in _flush() and asyc_manager(), signalling when async_manager
is actually working and not idle waiting for a request.
Bug: T147178
Change-Id: I184b445c43aa44cb000bfacc1cad18d2de265c48
---
M pywikibot/__init__.py
1 file changed, 7 insertions(+), 1 deletion(-)
Approvals:
Zhuyifei1999: Looks good to me, approved
jenkins-bot: Verified
diff --git a/pywikibot/__init__.py b/pywikibot/__init__.py
index 9c9632d..6fe82cc 100644
--- a/pywikibot/__init__.py
+++ b/pywikibot/__init__.py
@@ -1372,7 +1372,9 @@
'{lightblue}Waiting for {num} pages to be put. '
'Estimated time remaining: {sec}{default}', num=num, sec=sec))
- while _putthread.is_alive() and page_put_queue.qsize() > 0:
+ while (_putthread.is_alive()
+ and (page_put_queue.qsize() > 0
+ or page_put_queue_busy.qsize() > 0)):
try:
_putthread.join(1)
except KeyboardInterrupt:
@@ -1398,10 +1400,12 @@
"""Daemon; take requests from the queue and execute them in background."""
while True:
(request, args, kwargs) = page_put_queue.get()
+ page_put_queue_busy.put(None)
if request is None:
break
request(*args, **kwargs)
page_put_queue.task_done()
+ page_put_queue_busy.get()
def async_request(request, *args, **kwargs):
@@ -1420,6 +1424,8 @@
# queue to hold pending requests
page_put_queue = Queue(config.max_queue_size)
+# queue to signal that async_manager is working on a request. See T147178.
+page_put_queue_busy = Queue(config.max_queue_size)
# set up the background thread
_putthread = threading.Thread(target=async_manager)
# identification for debugging purposes
--
To view, visit https://gerrit.wikimedia.org/r/375448
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I184b445c43aa44cb000bfacc1cad18d2de265c48
Gerrit-Change-Number: 375448
Gerrit-PatchSet: 4
Gerrit-Owner: Mpaa <mpaa.wiki(a)gmail.com>
Gerrit-Reviewer: Dalba <dalba.wiki(a)gmail.com>
Gerrit-Reviewer: Dvorapa <dvorapa(a)seznam.cz>
Gerrit-Reviewer: Framawiki <framawiki(a)tools.wmflabs.org>
Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: Magul <tomasz.magulski(a)gmail.com>
Gerrit-Reviewer: Matěj Suchánek <matejsuchanek97(a)gmail.com>
Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl>
Gerrit-Reviewer: Mpaa <mpaa.wiki(a)gmail.com>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: Zhuyifei1999 <zhuyifei1999(a)gmail.com>
Gerrit-Reviewer: jenkins-bot (75)
jenkins-bot has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/550039 )
Change subject: Create family file for foundation wiki
......................................................................
Create family file for foundation wiki
Bug: T237888
Change-Id: I58552265916491402ff861928c7d8741572f3ea7
---
A pywikibot/families/foundation_family.py
1 file changed, 21 insertions(+), 0 deletions(-)
Approvals:
Zhuyifei1999: Looks good to me, approved
jenkins-bot: Verified
diff --git a/pywikibot/families/foundation_family.py b/pywikibot/families/foundation_family.py
new file mode 100644
index 0000000..14ef42e
--- /dev/null
+++ b/pywikibot/families/foundation_family.py
@@ -0,0 +1,21 @@
+# -*- coding: utf-8 -*-
+"""Family module for Foundation wiki."""
+#
+# (C) Pywikibot team, 2019
+#
+# Distributed under the terms of the MIT license.
+#
+from __future__ import absolute_import, division, unicode_literals
+
+from pywikibot import family
+
+
+# The Foundation family
+class Family(family.WikimediaFamily, family.SingleSiteFamily):
+
+ """Family class for Foundation wiki."""
+
+ name = 'foundation'
+ domain = 'foundation.wikimedia.org'
+
+ interwiki_forward = 'wmf'
--
To view, visit https://gerrit.wikimedia.org/r/550039
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I58552265916491402ff861928c7d8741572f3ea7
Gerrit-Change-Number: 550039
Gerrit-PatchSet: 2
Gerrit-Owner: Zoranzoki21 <zorandori4444(a)gmail.com>
Gerrit-Reviewer: Mpaa <mpaa.wiki(a)gmail.com>
Gerrit-Reviewer: Zhuyifei1999 <zhuyifei1999(a)gmail.com>
Gerrit-Reviewer: Zoranzoki21 <zorandori4444(a)gmail.com>
Gerrit-Reviewer: jenkins-bot (75)
Gerrit-CC: Dvorapa <dvorapa(a)seznam.cz>
Gerrit-CC: Xqt <info(a)gno.de>