jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/699333 )
Change subject: [IMPR] add add_text function to textlib
......................................................................
[IMPR] add add_text function to textlib
textlib.add_text adds text to a page content above categories and
interwiki links.
Add add_text function from add_text script to textlib but
without reading/writing the page. The trivial cases of adding
a text on top or on bottom of the page aren't supported.
Bug: T284388
Change-Id: Ia1b378c37272d0f8ba057f21633b306340a45360
---
M pywikibot/textlib.py
M tests/textlib_tests.py
2 files changed, 42 insertions(+), 0 deletions(-)
Approvals:
Damian: Looks good to me, but someone else must approve
JJMC89: Looks good to me, approved
jenkins-bot: Verified
diff --git a/pywikibot/textlib.py b/pywikibot/textlib.py
index 534cb78..9deac38 100644
--- a/pywikibot/textlib.py
+++ b/pywikibot/textlib.py
@@ -832,6 +832,36 @@
return text
+def add_text(text: str, add: str, *, site=None) -> str:
+ """Add text to a page content above categories and interwiki.
+
+ :param text: The page content to add text to.
+ :param add: Text to add.
+ :param site: The site that the text is coming from. Required for
+ reorder of categories and interlanguage links. Te default site
+ is used otherwise.
+ :type site: pywikibot.Site
+ """
+ # Translating the \\n (e.g. from command line) into binary \n
+ add = add.replace('\\n', '\n')
+
+ # Getting the categories
+ categories_inside = getCategoryLinks(text, site)
+ # Deleting the categories
+ text = removeCategoryLinks(text, site)
+ # Getting the interwiki
+ interwiki_inside = getLanguageLinks(text, site)
+ # Removing the interwiki
+ text = removeLanguageLinks(text, site)
+
+ # Adding the text
+ text += '\n' + add
+ # Reputting the categories
+ text = replaceCategoryLinks(text, categories_inside, site, addOnly=True)
+ # Adding the interwiki
+ return replaceLanguageLinks(text, interwiki_inside, site)
+
+
# -------------------------------
# Functions dealing with sections
# -------------------------------
diff --git a/tests/textlib_tests.py b/tests/textlib_tests.py
index 02bcbaa..80f7274 100644
--- a/tests/textlib_tests.py
+++ b/tests/textlib_tests.py
@@ -173,6 +173,18 @@
textlib.categoryFormat(data, self.site))
+class TestAddText(DefaultDrySiteTestCase):
+
+ """Test add_text function."""
+
+ def test_add_text(self):
+ """Test adding text."""
+ self.assertEqual(
+ textlib.add_text('foo\n[[Category:Foo]]', 'bar', site=self.site),
+ 'foo\nbar\n\n[[Category:Foo]]'
+ )
+
+
class TestCategoryRearrangement(DefaultDrySiteTestCase):
"""
--
To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/699333
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Change-Id: Ia1b378c37272d0f8ba057f21633b306340a45360
Gerrit-Change-Number: 699333
Gerrit-PatchSet: 5
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: Damian <atagar1(a)gmail.com>
Gerrit-Reviewer: JJMC89 <JJMC89.Wikimedia(a)gmail.com>
Gerrit-Reviewer: jenkins-bot
Gerrit-MessageType: merged
jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/699328 )
Change subject: [cleanup] Suppress deprecation warnings while building docs
......................................................................
[cleanup] Suppress deprecation warnings while building docs
The sphinx.util.inspect.safe_getattr() function triggers FutureWarnings for
every deprecated class as they're processed. This reduces the output of 'make
clean; make html' from 120 to 48 lines.
Change-Id: I6118efb78391c4c700d38ebb3d3a97b83c88ce8e
---
M docs/conf.py
1 file changed, 7 insertions(+), 0 deletions(-)
Approvals:
Xqt: Looks good to me, approved
jenkins-bot: Verified
diff --git a/docs/conf.py b/docs/conf.py
index d3e68ae..6a10812 100644
--- a/docs/conf.py
+++ b/docs/conf.py
@@ -23,9 +23,16 @@
import os
import re
import sys
+import warnings
+
from os.path import abspath, dirname, join
+# Deprecated classes will generate warnings as Sphinx processes them. Ignoring
+# them.
+
+warnings.simplefilter(action='ignore', category=FutureWarning)
+
docs_dir = dirname(__file__)
repo_dir = abspath(join(docs_dir, '..'))
sys.path.insert(0, repo_dir)
--
To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/699328
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Change-Id: I6118efb78391c4c700d38ebb3d3a97b83c88ce8e
Gerrit-Change-Number: 699328
Gerrit-PatchSet: 1
Gerrit-Owner: Damian <atagar1(a)gmail.com>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot
Gerrit-MessageType: merged
jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/699152 )
Change subject: [bugfix] don't try to retrieve data until logged in
......................................................................
[bugfix] don't try to retrieve data until logged in
mw_version Site property does an API call but this may fail
if a user is not logged in already. Therefore api.LoginManager
should not make an api call except for login purposes.
- move parameters preparation to api.LoginManager.parameters method.
- call get_login_token() to determine the mw version dispatching
- generate a fresh token if status in ('NeedToken', 'WrongToken', 'badtoken')
Bug: T284577
Change-Id: Ib7c24513fb08c475dcebe28ebd6b0411b2822372
---
M pywikibot/data/api.py
1 file changed, 55 insertions(+), 58 deletions(-)
Approvals:
Xqt: Looks good to me, approved
jenkins-bot: Verified
diff --git a/pywikibot/data/api.py b/pywikibot/data/api.py
index 2b8ebee..5bbaf38 100644
--- a/pywikibot/data/api.py
+++ b/pywikibot/data/api.py
@@ -28,7 +28,6 @@
from pywikibot.backports import Tuple, removeprefix
from pywikibot.comms import http
from pywikibot.exceptions import (
- CaptchaError,
Error,
FatalServerError,
InvalidTitleError,
@@ -1240,8 +1239,9 @@
self['prop'] = sorted(prop)
# When neither 'continue' nor 'rawcontinue' is present and the
# version number is at least 1.25wmf5 we add a dummy rawcontinue
- # parameter. Querying siteinfo is save as it adds 'continue'.
- if ('continue' not in self._params
+ # parameter. Querying siteinfo is save as it adds 'continue'
+ # except for 'tokens' (T284577)
+ if ('tokens' not in meta and 'continue' not in self._params
and self.site.mw_version >= '1.25wmf5'):
self._params.setdefault('rawcontinue', [''])
elif self.action == 'help' and self.site.mw_version > '1.24':
@@ -2848,32 +2848,16 @@
"""Get API keyword from mapping."""
return self.mapping[key][self.action != 'login']
- def login_to_site(self) -> None:
- """Login to the site.
-
- Note, this doesn't do anything with cookies. The http module
- takes care of all the cookie stuff. Throws exception on failure.
- """
- if hasattr(self, '_waituntil'):
- if datetime.datetime.now() < self._waituntil:
- diff = self._waituntil - datetime.datetime.now()
- pywikibot.warning(
- 'Too many tries, waiting {} seconds before retrying.'
- .format(diff.seconds))
- pywikibot.sleep(diff.seconds)
-
- below_mw_1_27 = self.site.mw_version < '1.27'
-
- if '@' in self.login_name or '@' in self.password or below_mw_1_27:
- # Since MW 1.27 only for bot passwords.
- # Bot passwords username contains @,
- # otherwise @ is not allowed in usernames.
- # @ in bot password is deprecated,
- # but we don't want to break bots using it.
- self.action = 'login'
- else:
- # Standard login request since MW 1.27
- self.action = 'clientlogin'
+ def parameters(self, botpassword: bool) -> dict:
+ """Return login parameters."""
+ # Since MW 1.27 only for bot passwords.
+ self.action = 'login'
+ if not botpassword:
+ # get token using meta=tokens if supported
+ token = self.get_login_token()
+ if token:
+ # Standard login request since MW 1.27
+ self.action = 'clientlogin'
# prepare default login parameters
parameters = {'action': self.action,
@@ -2884,20 +2868,41 @@
# clientlogin requires non-empty loginreturnurl
parameters['loginreturnurl'] = 'https://example.com'
parameters['rememberMe'] = '1'
+ parameters['logintoken'] = token
+
+ if self.site.family.ldapDomain:
+ parameters[self.keyword('ldap')] = self.site.family.ldapDomain
+
+ return parameters
+
+ def login_to_site(self) -> None:
+ """Login to the site.
+
+ Note, this doesn't do anything with cookies. The http module
+ takes care of all the cookie stuff. Throws exception on failure.
+ """
+ self.below_mw_1_27 = False
+ if hasattr(self, '_waituntil'):
+ if datetime.datetime.now() < self._waituntil:
+ diff = self._waituntil - datetime.datetime.now()
+ pywikibot.warning(
+ 'Too many tries, waiting {} seconds before retrying.'
+ .format(diff.seconds))
+ pywikibot.sleep(diff.seconds)
+
+ self.site._loginstatus = LoginStatus.IN_PROGRESS
+
+ # Bot passwords username contains @,
+ # otherwise @ is not allowed in usernames.
+ # @ in bot password is deprecated,
+ # but we don't want to break bots using it.
+ parameters = self.parameters(
+ botpassword='@' in self.login_name or '@' in self.password)
# base login request
login_request = self.site._request(use_get=False,
parameters=parameters)
-
- if self.site.family.ldapDomain:
- login_request[self.keyword('ldap')] = self.site.family.ldapDomain
-
- self.site._loginstatus = LoginStatus.IN_PROGRESS
while True:
- # get token using meta=tokens if supported
- if not below_mw_1_27:
- login_request[self.keyword('token')] = self.get_login_token()
-
# try to login
try:
login_result = login_request.submit()
@@ -2921,11 +2926,10 @@
if status in ('NeedToken', 'WrongToken', 'badtoken'):
token = response.get('token')
- if token and below_mw_1_27:
+ if token and self.below_mw_1_27:
# fetched token using action=login
login_request['lgtoken'] = token
- pywikibot.log('Received login token, '
- 'proceed with login.')
+ pywikibot.log('Received login token, proceed with login.')
else:
# if incorrect login token was used,
# force relogin and generate fresh one
@@ -2933,22 +2937,13 @@
'Forcing re-login.')
# invalidate superior wiki cookies (T224712)
_invalidate_superior_cookies(self.site.family)
+ login_request[
+ self.keyword('token')] = self.get_login_token()
continue
# messagecode was introduced with 1.29.0-wmf.14
- login_throttled = False
- if self.site.mw_version >= '1.29.0-wmf.14':
- try:
- login_throttled = \
- response['messagecode'] == 'login-throttled'
- except KeyError:
- pywikibot.log(
- "Missing 'messagecode' key encountered\n"
- 'Please file the following debug message to '
- 'Phabricator task T261061:\n{}'
- .format(login_result))
- raise CaptchaError('Captcha encountered which cannot be '
- 'handled:\n{}'.format(response))
+ # but older wikis are still supported
+ login_throttled = response.get('messagecode') == 'login-throttled'
if (status == 'Throttled' or status == self.keyword('fail')
and (login_throttled or 'wait' in fail_reason)):
@@ -2978,15 +2973,17 @@
:return: login token
"""
- if self.site.mw_version < '1.27':
- raise NotImplementedError('The method get_login_token() requires '
- 'at least MediaWiki version 1.27.')
login_token_request = self.site._request(
use_get=False,
parameters={'action': 'query', 'meta': 'tokens', 'type': 'login'},
)
login_token_result = login_token_request.submit()
- return login_token_result['query']['tokens'].get('logintoken')
+ # check if we have to use old implementation of mw < 1.27
+ if 'query' in login_token_result:
+ return login_token_result['query']['tokens'].get('logintoken')
+
+ self.below_mw_1_27 = True
+ return None
def encode_url(query) -> str:
--
To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/699152
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Change-Id: Ib7c24513fb08c475dcebe28ebd6b0411b2822372
Gerrit-Change-Number: 699152
Gerrit-PatchSet: 3
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot
Gerrit-CC: Chris Blom <chrisblom(a)chrisblom.net>
Gerrit-CC: Dvorapa <dvorapa(a)seznam.cz>
Gerrit-CC: JJMC89 <JJMC89.Wikimedia(a)gmail.com>
Gerrit-CC: Mpaa <mpaa.wiki(a)gmail.com>
Gerrit-MessageType: merged
jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/697994 )
Change subject: [IMPR] Rename compat2core.py to update_script.py
......................................................................
[IMPR] Rename compat2core.py to update_script.py
Change-Id: I86de7fdef40bb0daab45f566779f6823f698f72c
---
M HISTORY.rst
M README-conversion.rst
M scripts/README.rst
R scripts/maintenance/update_script.py
R tests/update_script_tests.py
5 files changed, 26 insertions(+), 19 deletions(-)
Approvals:
Xqt: Looks good to me, approved
jenkins-bot: Verified
diff --git a/HISTORY.rst b/HISTORY.rst
index 031c036..2973a39 100644
--- a/HISTORY.rst
+++ b/HISTORY.rst
@@ -95,7 +95,6 @@
* getSite() function was removed in favour of Site() constructor
* Page.fileUrl() was removed in favour of Page.get_file_url()
* Deprecated getuserinfo and getglobaluserinfo Site methods were removed
-* compat2core.py script was archived
6.0.1
diff --git a/README-conversion.rst b/README-conversion.rst
index 726686e..c3e081a 100644
--- a/README-conversion.rst
+++ b/README-conversion.rst
@@ -10,7 +10,7 @@
There is also a helper script which does a lot of changes automatically.
Just call it::
- pwb.py compat2core [<script to convert>]
+ pwb.py update_script [<script to convert>]
and follow the instructions and hints.
diff --git a/scripts/README.rst b/scripts/README.rst
index 801fedd..34b3173 100644
--- a/scripts/README.rst
+++ b/scripts/README.rst
@@ -246,8 +246,8 @@
+------------------------+---------------------------------------------------------+
| colors.py | Utility to show pywikibot colors. |
+------------------------+---------------------------------------------------------+
- | compat2core.py | Helper script to convert compat 1.0 scripts to the core |
- | | 3.0 framework. Also works for newer Pywikibot releases. |
+ | update_script.py | Helper script to update scripts and replace deprecated |
+ | | code parts. |
+------------------------+---------------------------------------------------------+
| make_i18n_dict.py | Generate an i18n file from a given script. |
+------------------------+---------------------------------------------------------+
diff --git a/scripts/maintenance/compat2core.py b/scripts/maintenance/update_script.py
similarity index 91%
rename from scripts/maintenance/compat2core.py
rename to scripts/maintenance/update_script.py
index 347e913..cc245e5 100755
--- a/scripts/maintenance/compat2core.py
+++ b/scripts/maintenance/update_script.py
@@ -1,6 +1,8 @@
#!/usr/bin/python
"""
-A helper script to convert compat 1.0 scripts to the new core 3.0 framework.
+A helper script to update scripts and replace deprecated parts.
+
+This also convert compat 1.0 scripts to the new core framework.
NOTE: Please be aware that this script is not able to convert your codes
completely. It may support you with some automatic replacements and it gives
@@ -9,7 +11,7 @@
finally.
The scripts asks for the .py file and converts it to
-<scriptname>-core.py in the same directory. The following option is supported:
+<scriptname>-new.py in the same directory. The following option is supported:
-warnonly Do not convert the source but show warning messages. This is good
to check already merged scripts.
@@ -18,11 +20,11 @@
to convert a script and show warnings about deprecated methods:
- python pwb.py compat2core <scriptname>
+ python pwb.py update_script <scriptname>
to show warnings about deprecated methods:
- python pwb.py compat2core <scriptname> -warnonly
+ python pwb.py update_script <scriptname> -warnonly
"""
#
# (C) Pywikibot team, 2014-2021
@@ -39,10 +41,13 @@
# be careful with replacement order!
replacements = (
+ #############################
+ # compat 2 core replacements
+ #############################
# doc strings
('#\r?\n__version__.*\r?\n',
'#\n'
- '# Automatically ported from compat branch by compat2core.py script\n'),
+ '# Automatically ported from compat branch by update_script.py script\n'),
('Pywikipedia bot team', 'Pywikibot team'),
# importing changes
('import wikipedia(?: as pywikibot)?', 'import pywikibot'),
@@ -88,6 +93,9 @@
# some warnings which must be changed manually
warnings = (
+ #############################
+ # compat 2 core warnings
+ #############################
('pywikibot.setAction(',
'setAction() no longer works; you must pass an explicit edit summary\n'
'message to save() or put()'),
@@ -171,7 +179,7 @@
def get_dest(self):
"""Ask for destination script name."""
- self.dest = '{}-core.{}'.format(*self.source.rsplit('.', 1))
+ self.dest = '{}-new.{}'.format(*self.source.rsplit('.', 1))
if not self.warnonly and not pywikibot.input_yn(
'Destination file is {}.'.format(self.dest),
default=True, automatic_quit=False):
diff --git a/tests/compat2core_tests.py b/tests/update_script_tests.py
similarity index 66%
rename from tests/compat2core_tests.py
rename to tests/update_script_tests.py
index bed2e9a..9047031 100644
--- a/tests/compat2core_tests.py
+++ b/tests/update_script_tests.py
@@ -1,4 +1,4 @@
-"""compat2core.py tests."""
+"""update_script.py tests."""
#
# (C) Pywikibot team, 2019-2021
#
@@ -6,33 +6,33 @@
#
import unittest
-import scripts.maintenance.compat2core as c2c
+import scripts.maintenance.update_script as us
from tests.aspects import TestCase
class Compat2CoreTests(TestCase):
- """Validate compat2core script."""
+ """Validate update_script script."""
net = False
def test_replacements(self):
- """Test compat2core replacements."""
- for item in c2c.replacements:
+ """Test update_script replacements."""
+ for item in us.replacements:
self.assertLength(item, 2)
self.assertIsInstance(item[0], str)
self.assertIsInstance(item[1], str)
def test_warnings(self):
- """Test compat2core warnings."""
- for item in c2c.warnings:
+ """Test update_script warnings."""
+ for item in us.warnings:
self.assertLength(item, 2)
self.assertIsInstance(item[0], str)
self.assertIsInstance(item[1], str)
def test_bot(self):
- """Test compat2core bot."""
- bot = c2c.ConvertBot(warnonly=True)
+ """Test update_script bot."""
+ bot = us.ConvertBot(warnonly=True)
self.assertIsNone(bot.source)
self.assertTrue(bot.warnonly)
--
To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/697994
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Change-Id: I86de7fdef40bb0daab45f566779f6823f698f72c
Gerrit-Change-Number: 697994
Gerrit-PatchSet: 1
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: D3r1ck01 <xsavitar.wiki(a)aol.com>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot
Gerrit-MessageType: merged