jenkins-bot has submitted this change and it was merged.
Change subject: Verifies tokens from paraminfo
......................................................................
Verifies tokens from paraminfo
Resolves the API warnings due to tokens which only concern global accounts.
Instead of hard coded variables TOKENS_[012] , now using the following:
For mwVersion < 1.20 gets the tokens from action=paraminfo&querymodules=info
For mwVersion < 1.24wmf19 and >= 1.20 from action=paraminfo&modules=tokens
For mwVersion > 1.24wmf19 from action=paraminfo&querymodules=tokens
and replacing those required and present in action=paraminfo&querymodules=info
or action=paraminfo&modules=tokens by 'csrf'
Bug: T72965
Bug: T85725
Change-Id: I3ff70dd8b9ee33fde15bd13d7af15db408aefc7d
---
M pywikibot/site.py
1 file changed, 27 insertions(+), 56 deletions(-)
Approvals:
John Vandenberg: Looks good to me, approved
jenkins-bot: Verified
diff --git a/pywikibot/site.py b/pywikibot/site.py
index 222145d..fdecbe0 100644
--- a/pywikibot/site.py
+++ b/pywikibot/site.py
@@ -1492,50 +1492,6 @@
# Pages; see method docs for details) --
#
- # Constants for token management.
- # For all MediaWiki versions prior to 1.20.
- # 'patrol' is indirectly supported via 'edit' token or recentchanges.
- # It will be converted in site.validate_tokens()/site.get_tokens().
- TOKENS_0 = set(['edit',
- 'delete',
- 'protect',
- 'move',
- 'block',
- 'unblock',
- 'email',
- 'import',
- 'watch',
- 'patrol',
- ])
-
- # For all MediaWiki versions, with 1.20 <= version < 1.24wmf19
- TOKENS_1 = set(['block',
- 'centralauth',
- 'delete',
- 'deleteglobalaccount',
- 'undelete',
- 'edit',
- 'email',
- 'import',
- 'move',
- 'options',
- 'patrol',
- 'protect',
- 'setglobalaccountstatus',
- 'unblock',
- 'watch',
- ])
-
- # For all MediaWiki versions >= 1.24wmf19
- TOKENS_2 = set(['csrf',
- 'deleteglobalaccount',
- 'patrol',
- 'rollback',
- 'setglobalaccountstatus',
- 'userrights',
- 'watch',
- ])
-
def __init__(self, code, fam=None, user=None, sysop=None):
"""Constructor."""
BaseSite.__init__(self, code, fam, user, sysop)
@@ -2646,7 +2602,10 @@
"""
_version = MediaWikiVersion(self.version())
if _version < MediaWikiVersion('1.20'):
- valid_types = [token for token in types if token in self.TOKENS_0]
+ types_wiki = self._paraminfo.parameter('query+info',
+ 'token')['type']
+ types_wiki.append('patrol')
+ valid_types = [token for token in types if token in types_wiki]
# Pre 1.17, preload token was the same as the edit token.
if _version < MediaWikiVersion('1.17'):
@@ -2654,16 +2613,22 @@
valid_types.append('edit')
elif _version < MediaWikiVersion('1.24wmf19'):
- valid_types = [token for token in types if token in self.TOKENS_1]
+ types_wiki = self._paraminfo.parameter('tokens',
+ 'type')['type']
+ valid_types = [token for token in types if token in types_wiki]
else:
- valid_types = []
+ types_wiki_old = self._paraminfo.parameter('query+info',
+ 'token')['type']
+ types_wiki_action = self._paraminfo.parameter('tokens',
+ 'type')['type']
+ types_wiki = self._paraminfo.parameter('query+tokens',
+ 'type')['type']
+ valid_types = [token for token in types if token in types_wiki]
for token in types:
- if ((token in self.TOKENS_0 or token in self.TOKENS_1) and
- token not in self.TOKENS_2):
- token = 'csrf'
- if token in self.TOKENS_2:
- valid_types.append(token)
-
+ if (token not in valid_types and
+ (token in types_wiki_old or
+ token in types_wiki_action)):
+ valid_types.append('csrf')
return valid_types
def get_tokens(self, types, all=False):
@@ -2708,7 +2673,9 @@
_version = MediaWikiVersion(self.version())
if _version < MediaWikiVersion('1.20'):
if all:
- types.extend(self.TOKENS_0)
+ types_wiki = self._paraminfo.parameter('query+info',
+ 'token')['type']
+ types.extend(types_wiki)
valid_tokens = set(self.validate_tokens(types))
# don't request patrol
query = api.PropertyGenerator('info',
@@ -2746,12 +2713,16 @@
else:
if _version < MediaWikiVersion('1.24wmf19'):
if all is not False:
- types.extend(self.TOKENS_1)
+ types_wiki = self._paraminfo.parameter('action+tokens',
+ 'type')['type']
+ types.extend(types_wiki)
req = api.Request(site=self, action='tokens',
type='|'.join(self.validate_tokens(types)))
else:
if all is not False:
- types.extend(self.TOKENS_2)
+ types_wiki = self._paraminfo.parameter('query+tokens',
+ 'type')['type']
+ types.extend(types_wiki)
req = api.Request(site=self, action='query', meta='tokens',
type='|'.join(self.validate_tokens(types)))
--
To view, visit https://gerrit.wikimedia.org/r/196450
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I3ff70dd8b9ee33fde15bd13d7af15db408aefc7d
Gerrit-PatchSet: 11
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: Akashagarwal <akashslg(a)gmail.com>
Gerrit-Reviewer: Akashagarwal <akashslg(a)gmail.com>
Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com>
Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl>
Gerrit-Reviewer: Mpaa <mpaa.wiki(a)gmail.com>
Gerrit-Reviewer: XZise <CommodoreFabianus(a)gmx.de>
Gerrit-Reviewer: jenkins-bot <>
jenkins-bot has submitted this change and it was merged.
Change subject: API help support complete help
......................................................................
API help support complete help
If the help module pre 1.25 is used without parameters, it returns
the complete help using the API error data structure.
If the client requests this information, do not raise an
APIError and instead return a dict containing the requested string
similar to the 1.25+ help data structure.
Bug: T93192
Change-Id: Ie1e6e9c69d7b71229926fed65a7b38331f11ceac
---
M pywikibot/data/api.py
1 file changed, 6 insertions(+), 0 deletions(-)
Approvals:
John Vandenberg: Looks good to me, but someone else must approve
XZise: Looks good to me, approved
jenkins-bot: Verified
diff --git a/pywikibot/data/api.py b/pywikibot/data/api.py
index 741a75b..934af87 100644
--- a/pywikibot/data/api.py
+++ b/pywikibot/data/api.py
@@ -1296,6 +1296,12 @@
u"Pausing due to database lag: " + info)
self.site.throttle.lag(int(lag.group("lag")))
continue
+ elif code == 'help' and self.action == 'help':
+ # The help module returns an error result with the complete
+ # API information. As this data was requested, return the
+ # data instead of raising an exception.
+ return {'help': {'mime': 'text/plain',
+ 'help': result['error']['help']}}
if code.startswith(u'internal_api_error_'):
class_name = code[len(u'internal_api_error_'):]
--
To view, visit https://gerrit.wikimedia.org/r/197903
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: Ie1e6e9c69d7b71229926fed65a7b38331f11ceac
Gerrit-PatchSet: 2
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com>
Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl>
Gerrit-Reviewer: XZise <CommodoreFabianus(a)gmx.de>
Gerrit-Reviewer: jenkins-bot <>
jenkins-bot has submitted this change and it was merged.
Change subject: ParamInfo sub-module support
......................................................................
ParamInfo sub-module support
ParamInfo was initially designed with actions and query modules
flattened. A submodule 'tokens' has been added to the query module,
causing a name clash with the deprecated 'tokens' action.
The API now provides a 'path' for each module, with submodules.
While this change doesnt remove all special handling of action and query
modules, the internal data structure of ParamInfo now uses the module path
as the cache key, preventing clashes, and emulates the path for older MW
releases which do not provide a path.
ParamInfo data can now be obtained using any valid submodule path,
like 'query+revisions', however 'revisions' will still obtain paraminfo
for the query sub-module.
ParamInfo for 'tokens' now returns data for the deprecated action 'tokens'
on MW 1.20+, whereas previously it returned data for the query 'tokens'.
To obtain the non-deprecated tokens paraminfo, use 'query+tokens'.
Bug: T92626
Change-Id: I7db5a2d5dd84ebbab47b310c447bee59346b194a
---
M pywikibot/data/api.py
M tests/api_tests.py
M tests/dry_api_tests.py
3 files changed, 194 insertions(+), 82 deletions(-)
Approvals:
John Vandenberg: Looks good to me, but someone else must approve
XZise: Looks good to me, approved
jenkins-bot: Verified
diff --git a/pywikibot/data/api.py b/pywikibot/data/api.py
index 741a75b..fd3428b 100644
--- a/pywikibot/data/api.py
+++ b/pywikibot/data/api.py
@@ -23,6 +23,8 @@
import traceback
import time
+from warnings import warn
+
import pywikibot
from pywikibot import config, login
from pywikibot.tools import MediaWikiVersion, deprecated, itergroup
@@ -155,7 +157,6 @@
'mainmodule', 'pagesetmodule'])
root_modules = frozenset(['main', 'pageset'])
- root_module_keys = frozenset(['mainmodule', 'pagesetmodule'])
init_modules = frozenset(['main', 'paraminfo'])
@@ -258,6 +259,15 @@
generator_param['type']
)
+ _reused_module_names = self._action_modules & self._query_modules
+
+ # The only name clash in core between actions and query submodules is
+ # action=tokens and actions=query&meta=tokens, and this will warn if
+ # any new ones appear.
+ if _reused_module_names > set(['tokens']):
+ warn('Unexpected overlap between action and query submodules: %s'
+ % (_reused_module_names - set(['tokens'])), UserWarning)
+
self.__inited = True
def _emulate_pageset(self):
@@ -288,12 +298,8 @@
if 'paraminfo' not in self._paraminfo and not _init:
self._init()
- # Users will supply the wrong type, and expect it to work.
- if not isinstance(modules, set):
- if isinstance(modules, basestring):
- modules = set(modules.split('|'))
- else:
- modules = set(modules)
+ if self.__inited:
+ modules = self._normalize_modules(modules)
modules = modules - set(self._paraminfo.keys())
if not modules:
@@ -326,16 +332,13 @@
}
if self.modules_only_mode:
- params['modules'] = ['query+' + mod
- if mod in self._query_modules
- else mod
- for mod in module_batch]
+ params['modules'] = module_batch
else:
params['modules'] = [mod for mod in module_batch
- if mod not in self._query_modules and
+ if not mod.startswith('query+') and
mod not in self.root_modules]
- params['querymodules'] = [mod for mod in module_batch
- if mod in self._query_modules]
+ params['querymodules'] = [mod[6:] for mod in module_batch
+ if mod.startswith('query+')]
for mod in set(module_batch) & self.root_modules:
params[mod + 'module'] = 1
@@ -350,31 +353,106 @@
if self.modules_only_mode and 'pageset' in modules:
self._emulate_pageset()
+ def _normalize_modules(self, modules):
+ """Add query+ to any query module name not also in action modules."""
+ # Users will supply the wrong type, and expect it to work.
+ if isinstance(modules, basestring):
+ modules = set(modules.split('|'))
+
+ assert(self._action_modules)
+
+ return set('query+' + mod if '+' not in mod and
+ mod in self._query_modules and
+ mod not in self._action_modules
+ else mod
+ for mod in modules)
+
+ def normalize_modules(self, modules):
+ """
+ Convert the modules into module paths.
+
+ Add query+ to any query module name not also in action modules.
+
+ @return: The modules converted into a module paths
+ @rtype: set
+ """
+ if not self.__inited:
+ self._init()
+ return self._normalize_modules(modules)
+
@classmethod
def normalize_paraminfo(cls, data):
"""Convert both old and new API JSON into a new-ish data structure."""
- return dict([(mod_data['name'] if 'name' in mod_data else
- 'main' if mod_data['classname'] == 'ApiMain' else
- 'pageset' if mod_data['classname'] == 'ApiPageSet' else
- '<unknown>:' + mod_data['classname'],
- mod_data)
- for modules_data in
- [[modules_data] if paraminfo_key in cls.root_module_keys
- else modules_data
- for paraminfo_key, modules_data
- in data['paraminfo'].items()
- if modules_data and paraminfo_key in cls.paraminfo_keys]
- for mod_data in modules_data
- if 'missing' not in mod_data])
+ result_data = {}
+ for paraminfo_key, modules_data in data['paraminfo'].items():
+ if not modules_data:
+ continue
+
+ if paraminfo_key[:-len('module')] in cls.root_modules:
+ modules_data = [modules_data]
+ elif not paraminfo_key.endswith('modules'):
+ continue
+
+ for mod_data in modules_data:
+ if 'missing' in mod_data:
+ continue
+
+ name = mod_data.get('name')
+ php_class = mod_data.get('classname')
+
+ if not name and php_class:
+ if php_class == 'ApiMain':
+ name = 'main'
+ elif php_class == 'ApiPageSet':
+ name = 'pageset'
+ else:
+ pywikibot.warning('Unknown paraminfo module "{0}"'.format(
+ php_class))
+ name = '<unknown>:' + php_class
+
+ mod_data['name'] = name
+
+ if 'path' not in mod_data:
+ # query modules often contain 'ApiQuery' and have a suffix.
+ # 'ApiQuery' alone is the action 'query'
+ if 'querytype' in mod_data or (
+ php_class and len(php_class) > 8 and
+ 'ApiQuery' in php_class):
+ mod_data['path'] = 'query+' + name
+ else:
+ mod_data['path'] = name
+
+ path = mod_data['path']
+
+ result_data[path] = mod_data
+
+ return result_data
def __getitem__(self, key):
- """Return a paraminfo property, caching it."""
+ """
+ Return a paraminfo module for the module path, caching it.
+
+ Use the module path, such as 'query+x', to obtain the paraminfo for
+ submodule 'x' in the query module.
+
+ If the key does not include a '+' and is not present in the top level
+ of the API, it will fallback to looking for the key 'query+x'.
+ """
self.fetch(set([key]))
- return self._paraminfo[key]
+ if key in self._paraminfo:
+ return self._paraminfo[key]
+ elif '+' not in key:
+ return self._paraminfo['query+' + key]
+ else:
+ raise KeyError(key)
def __contains__(self, key):
- """Return whether the value is cached."""
- return key in self._paraminfo
+ """Return whether the key is valid."""
+ try:
+ self[key]
+ return True
+ except KeyError:
+ return False
def __len__(self):
"""Obtain length of the iterable."""
@@ -398,21 +476,31 @@
# data and discard the entire received paraminfo structure. There are
# also params which are common to many modules, such as those provided
# by the ApiPageSet php class: titles, pageids, redirects, etc.
- self.fetch(set([module]))
- if module not in self._paraminfo:
+ try:
+ module = self[module]
+ except KeyError:
raise ValueError("paraminfo for '%s' not loaded" % module)
- if 'parameters' not in self._paraminfo[module]:
+
+ if 'parameters' not in module:
pywikibot.warning("module '%s' has no parameters" % module)
return
- params = self._paraminfo[module]['parameters']
+ params = module['parameters']
param_data = [param for param in params
if param['name'] == param_name]
return param_data[0] if len(param_data) else None
@property
def modules(self):
- """Set of all modules."""
+ """
+ Set of all module names without path prefixes.
+
+ Only includes one 'tokens', even if it appears as both a
+ action and a query submodule.
+
+ @return: module names
+ @rtype: set of str
+ """
if not self.__inited:
self._init()
return self._action_modules | self._query_modules
@@ -426,10 +514,15 @@
@property
def query_modules(self):
- """Set of all query modules."""
+ """Set of all query module names without query+ path prefix."""
if not self.__inited:
self._init()
return self._query_modules
+
+ @staticmethod
+ def _prefix_submodules(modules, prefix):
+ """Prefix submodules with path."""
+ return set('{0}+{1}'.format(prefix, mod) for mod in modules)
@property
def prefixes(self):
@@ -450,23 +543,28 @@
@type attribute: basestring
@param modules: modules to include (default: all modules)
@type modules: set
- @rtype: dict
+ @rtype: dict using modules as keys
"""
if modules is None:
- modules = self.modules
+ # TODO: The keys for modules with a clash are path prefixed
+ # which is different from all other keys.
+ modules = self.modules | self._prefix_submodules(
+ self.query_modules & self.action_modules, 'query')
+
self.fetch(modules)
- return dict([(mod, self._paraminfo[mod][attribute])
- for mod in modules
- if self._paraminfo[mod][attribute]])
+
+ return dict((mod, self[mod][attribute])
+ for mod in modules
+ if self[mod][attribute])
@property
def query_modules_with_limits(self):
"""Set of all query modules which have limits."""
if not self._with_limits:
- self.fetch(self.query_modules)
+ self.fetch(self._prefix_submodules(self.query_modules, 'query'))
self._with_limits = frozenset(
[mod for mod in self.query_modules
- if self.parameter(mod, 'limit')])
+ if self.parameter('query+' + mod, 'limit')])
return self._with_limits
diff --git a/tests/api_tests.py b/tests/api_tests.py
index 215c750..8a495fc 100644
--- a/tests/api_tests.py
+++ b/tests/api_tests.py
@@ -68,8 +68,8 @@
self.assertEqual(len(pi), 0)
pi._init()
- self.assertIn('main', pi)
- self.assertIn('paraminfo', pi)
+ self.assertIn('main', pi._paraminfo)
+ self.assertIn('paraminfo', pi._paraminfo)
self.assertEqual(len(pi),
len(pi.preloaded_modules))
@@ -84,15 +84,15 @@
self.assertEqual(len(pi), 0)
pi._init()
- self.assertIn('main', pi)
- self.assertIn('paraminfo', pi)
- self.assertIn('pageset', pi)
+ self.assertIn('main', pi._paraminfo)
+ self.assertIn('paraminfo', pi._paraminfo)
+ self.assertIn('pageset', pi._paraminfo)
if 'query' in pi.preloaded_modules:
- self.assertIn('query', pi)
+ self.assertIn('query', pi._paraminfo)
self.assertEqual(len(pi), 4)
else:
- self.assertNotIn('query', pi)
+ self.assertNotIn('query', pi._paraminfo)
self.assertEqual(len(pi), 3)
self.assertEqual(len(pi),
@@ -109,10 +109,10 @@
self.assertEqual(len(pi), 0)
pi._init()
- self.assertIn('main', pi)
- self.assertIn('paraminfo', pi)
- self.assertIn('pageset', pi)
- self.assertIn('query', pi)
+ self.assertIn('main', pi._paraminfo)
+ self.assertIn('paraminfo', pi._paraminfo)
+ self.assertIn('pageset', pi._paraminfo)
+ self.assertIn('query', pi._paraminfo)
if MediaWikiVersion(site.version()) >= MediaWikiVersion("1.21"):
# 'generator' was added to 'pageset' in 1.21
@@ -126,10 +126,10 @@
pi = api.ParamInfo(site)
self.assertEqual(len(pi), 0)
pi.fetch(['info'])
- self.assertIn('info', pi)
+ self.assertIn('query+info', pi._paraminfo)
- self.assertIn('main', pi)
- self.assertIn('paraminfo', pi)
+ self.assertIn('main', pi._paraminfo)
+ self.assertIn('paraminfo', pi._paraminfo)
self.assertEqual(len(pi),
1 + len(pi.preloaded_modules))
@@ -149,10 +149,10 @@
pi = api.ParamInfo(site)
self.assertEqual(len(pi), 0)
pi.fetch(['revisions'])
- self.assertIn('revisions', pi)
+ self.assertIn('query+revisions', pi._paraminfo)
- self.assertIn('main', pi)
- self.assertIn('paraminfo', pi)
+ self.assertIn('main', pi._paraminfo)
+ self.assertIn('paraminfo', pi._paraminfo)
self.assertEqual(len(pi),
1 + len(pi.preloaded_modules))
@@ -172,11 +172,11 @@
pi = api.ParamInfo(site)
self.assertEqual(len(pi), 0)
pi.fetch(['info', 'revisions'])
- self.assertIn('info', pi)
- self.assertIn('revisions', pi)
+ self.assertIn('query+info', pi._paraminfo)
+ self.assertIn('query+revisions', pi._paraminfo)
- self.assertIn('main', pi)
- self.assertIn('paraminfo', pi)
+ self.assertIn('main', pi._paraminfo)
+ self.assertIn('paraminfo', pi._paraminfo)
self.assertEqual(len(pi),
2 + len(pi.preloaded_modules))
@@ -185,10 +185,10 @@
pi = api.ParamInfo(site)
self.assertEqual(len(pi), 0)
pi.fetch('foobar')
- self.assertNotIn('foobar', pi)
+ self.assertNotIn('foobar', pi._paraminfo)
- self.assertIn('main', pi)
- self.assertIn('paraminfo', pi)
+ self.assertIn('main', pi._paraminfo)
+ self.assertIn('paraminfo', pi._paraminfo)
self.assertEqual(len(pi),
len(pi.preloaded_modules))
@@ -218,10 +218,10 @@
site = self.get_site()
pi = api.ParamInfo(site, modules_only_mode=False)
pi.fetch(['info'])
- self.assertIn('info', pi)
+ self.assertIn('query+info', pi._paraminfo)
- self.assertIn('main', pi)
- self.assertIn('paraminfo', pi)
+ self.assertIn('main', pi._paraminfo)
+ self.assertIn('paraminfo', pi._paraminfo)
self.assertEqual(len(pi),
1 + len(pi.preloaded_modules))
@@ -235,10 +235,10 @@
% site.version())
pi = api.ParamInfo(site, modules_only_mode=True)
pi.fetch(['info'])
- self.assertIn('info', pi)
+ self.assertIn('query+info', pi._paraminfo)
- self.assertIn('main', pi)
- self.assertIn('paraminfo', pi)
+ self.assertIn('main', pi._paraminfo)
+ self.assertIn('paraminfo', pi._paraminfo)
self.assertEqual(len(pi),
1 + len(pi.preloaded_modules))
diff --git a/tests/dry_api_tests.py b/tests/dry_api_tests.py
index eb260c1..de14294 100644
--- a/tests/dry_api_tests.py
+++ b/tests/dry_api_tests.py
@@ -276,11 +276,23 @@
"querytype": "prop"
}
+ edit_action_param_data = {
+ 'name': 'edit',
+ 'path': 'edit'
+ }
+
def setUp(self):
"""Add a real ParamInfo to the DrySite."""
super(ParamInfoDictTests, self).setUp()
site = self.get_site()
site._paraminfo = ParamInfo(site)
+ # Pretend that paraminfo has been loaded
+ for mod in site._paraminfo.init_modules:
+ site._paraminfo._paraminfo[mod] = {}
+ site._paraminfo._query_modules = ['info']
+ site._paraminfo._action_modules = ['edit']
+ # TODO: remove access of this private member of ParamInfo
+ site._paraminfo._ParamInfo__inited = True
def test_new_format(self):
pi = self.get_site()._paraminfo
@@ -291,14 +303,16 @@
'paraminfo': {
'modules': [
self.prop_info_param_data,
- {'name': 'edit'}
+ self.edit_action_param_data,
]
}
})
pi._paraminfo.update(data)
- self.assertIn('info', pi._paraminfo)
self.assertIn('edit', pi._paraminfo)
+ self.assertIn('query+info', pi._paraminfo)
+ self.assertIn('edit', pi)
+ self.assertIn('info', pi)
def test_old_format(self):
pi = self.get_site()._paraminfo
@@ -308,13 +322,15 @@
data = pi.normalize_paraminfo({
'paraminfo': {
'querymodules': [self.prop_info_param_data],
- 'modules': [{'name': 'edit'}]
+ 'modules': [self.edit_action_param_data],
}
})
pi._paraminfo.update(data)
- self.assertIn('info', pi._paraminfo)
self.assertIn('edit', pi._paraminfo)
+ self.assertIn('query+info', pi._paraminfo)
+ self.assertIn('edit', pi)
+ self.assertIn('info', pi)
def test_attribute(self):
pi = self.get_site()._paraminfo
@@ -329,8 +345,8 @@
pi._paraminfo.update(data)
- self.assertEqual(pi._paraminfo['info']['prefix'], 'in')
- self.assertEqual(pi._paraminfo['info']['querytype'], 'prop')
+ self.assertEqual(pi._paraminfo['query+info']['querytype'], 'prop')
+ self.assertEqual(pi['info']['prefix'], 'in')
def test_parameter(self):
pi = self.get_site()._paraminfo
@@ -344,8 +360,6 @@
})
pi._paraminfo.update(data)
- # Pretend that paraminfo has been loaded
- pi._paraminfo['paraminfo'] = {}
param = pi.parameter('info', 'token')
self.assertIsInstance(param, dict)
--
To view, visit https://gerrit.wikimedia.org/r/196642
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I7db5a2d5dd84ebbab47b310c447bee59346b194a
Gerrit-PatchSet: 14
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: XZise <CommodoreFabianus(a)gmx.de>
Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com>
Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl>
Gerrit-Reviewer: XZise <CommodoreFabianus(a)gmx.de>
Gerrit-Reviewer: jenkins-bot <>
jenkins-bot has submitted this change and it was merged.
Change subject: [FEAT] version: actually interpret the __version__
......................................................................
[FEAT] version: actually interpret the __version__
Instead of just reading the line with __version__ it uses the
__version__ variable of the module. Thus it's required that the file is
imported, but this doesn't accidentially executes any code.
Bug: T73788
Change-Id: I1a5691e8492e204388dae4b2031016b7d2c3bab7
---
M pywikibot/bot.py
M pywikibot/version.py
2 files changed, 61 insertions(+), 15 deletions(-)
Approvals:
John Vandenberg: Looks good to me, approved
jenkins-bot: Verified
diff --git a/pywikibot/bot.py b/pywikibot/bot.py
index 22e614f..be83074 100644
--- a/pywikibot/bot.py
+++ b/pywikibot/bot.py
@@ -352,10 +352,12 @@
# imported modules
log(u'MODULES:')
- for item in list(all_modules):
- ver = version.getfileversion('%s.py' % item.replace('.', '/'))
- if ver:
- log(u' %s' % ver)
+ for module in sys.modules.values():
+ filename = version.get_module_filename(module)
+ ver = version.get_module_version(module)
+ mtime = version.get_module_mtime(module)
+ if filename and ver and mtime:
+ log(u' {0} {1} {2}'.format(filename, ver[:7], mtime.isoformat(' ')))
if config.log_pywiki_repo_version:
log(u'PYWIKI REPO VERSION: %s' % version.getversion_onlinerepo())
diff --git a/pywikibot/version.py b/pywikibot/version.py
index 723a221..c7a4d18 100644
--- a/pywikibot/version.py
+++ b/pywikibot/version.py
@@ -11,11 +11,14 @@
#
import os
+import sys
import time
import datetime
import subprocess
import codecs
+import pywikibot
+from pywikibot.tools import deprecated
import pywikibot.config2 as config
cache = None
@@ -68,18 +71,11 @@
(tag, rev, date, hsh) = getversion_nightly()
except Exception:
try:
- version = getfileversion('pywikibot/__init__.py')
- if not version:
- # fall-back in case everything breaks (should not be used)
- import pywikibot
- version = getfileversion(pywikibot.__file__[:-1])
+ hsh = get_module_version(pywikibot)
+ date = get_module_mtime(pywikibot)
- file, hsh_short, date, ts = version.split(' ')
tag = 'pywikibot/__init__.py'
rev = '-1 (unknown)'
- ts = ts.split('.')[0]
- date = time.strptime('%sT%s' % (date, ts), '%Y-%m-%dT%H:%M:%S')
- hsh = hsh_short + ('?' * 33) # enhance the short hash w. '?'
except:
# nothing worked; version unknown (but suppress exceptions)
# the value is most likely '$Id' + '$', it means that
@@ -254,6 +250,7 @@
return hsh
+@deprecated('get_module_version, get_module_filename and get_module_mtime')
def getfileversion(filename):
"""Retrieve revision number of file.
@@ -284,6 +281,55 @@
return None
+def get_module_version(module):
+ """
+ Retrieve __version__ variable from an imported module.
+
+ @param module: The module instance.
+ @type module: module
+ @return: The version hash without the surrounding text. If not present None.
+ @rtype: str or None
+ """
+ if hasattr(module, '__version__'):
+ return module.__version__[5:-1]
+
+
+def get_module_filename(module):
+ """
+ Retrieve filename from an imported pywikibot module.
+
+ It uses the __file__ attribute of the module. If it's file extension ends
+ with py and another character the last character is discarded when the py
+ file exist.
+
+ @param module: The module instance.
+ @type module: module
+ @return: The filename if it's a pywikibot module otherwise None.
+ @rtype: str or None
+ """
+ if hasattr(module, '__file__') and os.path.exists(module.__file__):
+ filename = module.__file__
+ if filename[-4:-1] == '.py' and os.path.exists(filename[:-1]):
+ filename = filename[:-1]
+ program_dir = _get_program_dir()
+ if filename[:len(program_dir)] == program_dir:
+ return filename
+
+
+def get_module_mtime(module):
+ """
+ Retrieve the modification time from an imported module.
+
+ @param module: The module instance.
+ @type module: module
+ @return: The modification time if it's a pywikibot module otherwise None.
+ @rtype: datetime or None
+ """
+ filename = get_module_filename(module)
+ if filename:
+ return datetime.datetime.fromtimestamp(os.stat(filename).st_mtime)
+
+
def package_versions(modules=None, builtins=False, standard_lib=None):
"""Retrieve package version information.
@@ -297,8 +343,6 @@
@param standard_lib: Include standard library packages
@type standard_lib: Boolean, or None for automatic selection
"""
- import sys
-
if not modules:
modules = sys.modules.keys()
--
To view, visit https://gerrit.wikimedia.org/r/192082
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I1a5691e8492e204388dae4b2031016b7d2c3bab7
Gerrit-PatchSet: 2
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: XZise <CommodoreFabianus(a)gmx.de>
Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com>
Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl>
Gerrit-Reviewer: XZise <CommodoreFabianus(a)gmx.de>
Gerrit-Reviewer: jenkins-bot <>
jenkins-bot has submitted this change and it was merged.
Change subject: Dont pass None as api param value
......................................................................
Dont pass None as api param value
Change-Id: I43cb53e4c61182031ef25a14589cd10b1f799bbd
---
M pywikibot/data/api.py
1 file changed, 2 insertions(+), 2 deletions(-)
Approvals:
John Vandenberg: Looks good to me, but someone else must approve
XZise: Looks good to me, but someone else must approve
Xqt: Looks good to me, approved
jenkins-bot: Verified
diff --git a/pywikibot/data/api.py b/pywikibot/data/api.py
index 741a75b..7f4210e 100644
--- a/pywikibot/data/api.py
+++ b/pywikibot/data/api.py
@@ -996,8 +996,8 @@
value = values[0]
if value is True:
values = ['']
- elif value is False:
- # False booleans are not in the http URI
+ elif value is False or value is None:
+ # False and None are not included in the http URI
continue
iterator = iter(values)
value = u'|'.join(self._format_value(value) for value in iterator)
--
To view, visit https://gerrit.wikimedia.org/r/197813
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I43cb53e4c61182031ef25a14589cd10b1f799bbd
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com>
Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl>
Gerrit-Reviewer: XZise <CommodoreFabianus(a)gmx.de>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot <>