jenkins-bot has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/524305 )
Change subject: archivebot.py: don't reorder template parameters
......................................................................
archivebot.py: don't reorder template parameters
Page.raw_extracted_templates is based on an ordered dict, which preserves
order of parameters.
As opposite of Page.templatesWithParams(), which uses plain dictionaries.
Bug: T220828
Change-Id: I0ac00fe1a6251833c8ae3ae94bea8f59f46d9a59
---
M scripts/archivebot.py
1 file changed, 3 insertions(+), 4 deletions(-)
Approvals:
Xqt: Looks good to me, approved
jenkins-bot: Verified
diff --git a/scripts/archivebot.py b/scripts/archivebot.py
index 5ea63ac..97c0bb9 100755
--- a/scripts/archivebot.py
+++ b/scripts/archivebot.py
@@ -571,10 +571,9 @@
"""Load and validate archiver template."""
pywikibot.output('Looking for: {{%s}} in %s' % (self.tpl.title(),
self.page))
- for tpl in self.page.templatesWithParams():
- if tpl[0] == pywikibot.Page(self.site, self.tpl.title(), ns=10):
- for param in tpl[1]:
- item, value = param.split('=', 1)
+ for tpl in self.page.raw_extracted_templates:
+ if tpl[0] == self.tpl.title():
+ for item, value in tpl[1].items():
self.set_attr(item.strip(), value.strip())
break
else:
--
To view, visit https://gerrit.wikimedia.org/r/524305
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I0ac00fe1a6251833c8ae3ae94bea8f59f46d9a59
Gerrit-Change-Number: 524305
Gerrit-PatchSet: 2
Gerrit-Owner: Mpaa <mpaa.wiki(a)gmail.com>
Gerrit-Reviewer: D3r1ck01 <xsavitar.wiki(a)aol.com>
Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot (75)
jenkins-bot has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/524091 )
Change subject: [bugfix] ignore test_family.py if resolving family from url
......................................................................
[bugfix] ignore test_family.py if resolving family from url
test.wikipedia.org has it's own family file but this family is available
in wikipedia_family too. To prevent multiple matches in _code_fam_from_url
when resolve test.wikipedia.org url ignore test_family and
use wikpedia_family instead
Fix tests accordingly.
Bug: T228300
Change-Id: I1f36d4e7bb3261dcca0d0298bfb1992e48e89a1c
---
M pywikibot/families/test_family.py
M tests/site_tests.py
2 files changed, 4 insertions(+), 1 deletion(-)
Approvals:
Dalba: Looks good to me, approved
jenkins-bot: Verified
diff --git a/pywikibot/families/test_family.py b/pywikibot/families/test_family.py
index 0e89716..a1ea3e0 100644
--- a/pywikibot/families/test_family.py
+++ b/pywikibot/families/test_family.py
@@ -18,3 +18,6 @@
name = 'test'
domain = 'test.wikipedia.org'
interwiki_forward = 'wikipedia'
+
+ # 'test' family should resolve to be in the 'wikipedia' family
+ _ignore_from_url = True
diff --git a/tests/site_tests.py b/tests/site_tests.py
index 650a77d..91204f2 100644
--- a/tests/site_tests.py
+++ b/tests/site_tests.py
@@ -330,7 +330,7 @@
['commonswiki', ('commons', 'commons')],
['wikidatawiki', ('wikidata', 'wikidata')],
['testwikidatawiki', ('test', 'wikidata')],
- ['testwiki', ('test', 'test')], # see T225729
+ ['testwiki', ('test', 'wikipedia')], # see T225729, T228300
['test2wiki', ('test2', 'wikipedia')], # see T225729
['sourceswiki', ('mul', 'wikisource')], # see T226960
]
--
To view, visit https://gerrit.wikimedia.org/r/524091
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I1f36d4e7bb3261dcca0d0298bfb1992e48e89a1c
Gerrit-Change-Number: 524091
Gerrit-PatchSet: 1
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: Dalba <dalba.wiki(a)gmail.com>
Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: jenkins-bot (75)
jenkins-bot has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/508071 )
Change subject: [cleanup] Reduce code complexity of Family.from_url
......................................................................
[cleanup] Reduce code complexity of Family.from_url
Reduce code complexity of Family.from_url method from C17 to C15
- remove "ignored" variable and use "_ignore_from_url" attribute directly
- remove start marker "^" from match pattern
- exit from_url method if domain was not found in parsed.netloc
- don't use "elsif"/"else" statement if method was leaved previously
Change-Id: I123ea6ed4180a22e07a4c95202f885464e831230
---
M pywikibot/family.py
1 file changed, 23 insertions(+), 23 deletions(-)
Approvals:
Dalba: Looks good to me, approved
jenkins-bot: Verified
diff --git a/pywikibot/family.py b/pywikibot/family.py
index b0be37b..d34938a 100644
--- a/pywikibot/family.py
+++ b/pywikibot/family.py
@@ -1293,12 +1293,11 @@
"""
if self._ignore_from_url is True:
return None
- else:
- ignored = self._ignore_from_url
parsed = urlparse.urlparse(url)
- if not re.match('^(https?)?$', parsed.scheme):
+ if not re.match('(https?)?$', parsed.scheme):
return None
+
path = parsed.path
if parsed.query:
path += '?' + parsed.query
@@ -1309,35 +1308,36 @@
raise ValueError('Text after the $1 placeholder is not supported '
'(T111513).')
- matched_sites = []
for domain in self.domains:
if domain in parsed.netloc:
break
else:
- domain = False
- if domain is not False:
- for code in chain(self.codes, getattr(self, 'test_codes', ())):
- if code in ignored:
- continue
- if self._hostname(code)[1] == parsed.netloc:
- # Use the code and family instead of the url
- # This is only creating a Site instance if domain matches
- site = pywikibot.Site(code, self.name)
- pywikibot.log('Found candidate {0}'.format(site))
+ return None
- for iw_url in site._interwiki_urls():
- if path.startswith(iw_url):
- matched_sites += [site]
- break
+ matched_sites = []
+ for code in chain(self.codes, getattr(self, 'test_codes', ())):
+ if code in self._ignore_from_url:
+ continue
+ if self._hostname(code)[1] == parsed.netloc:
+ # Use the code and family instead of the url
+ # This is only creating a Site instance if domain matches
+ site = pywikibot.Site(code, self.name)
+ pywikibot.log('Found candidate {0}'.format(site))
+
+ for iw_url in site._interwiki_urls():
+ if path.startswith(iw_url):
+ matched_sites += [site]
+ break
if len(matched_sites) == 1:
return matched_sites[0].code
- elif not matched_sites:
+
+ if not matched_sites:
return None
- else:
- raise RuntimeError(
- 'Found multiple matches for URL "{0}": {1}'
- .format(url, ', '.join(str(s) for s in matched_sites)))
+
+ raise RuntimeError(
+ 'Found multiple matches for URL "{0}": {1}'
+ .format(url, ', '.join(str(s) for s in matched_sites)))
def maximum_GET_length(self, code):
"""Return the maximum URL length for GET instead of POST."""
--
To view, visit https://gerrit.wikimedia.org/r/508071
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I123ea6ed4180a22e07a4c95202f885464e831230
Gerrit-Change-Number: 508071
Gerrit-PatchSet: 5
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: Dalba <dalba.wiki(a)gmail.com>
Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: Zhuyifei1999 <zhuyifei1999(a)gmail.com>
Gerrit-Reviewer: jenkins-bot (75)
jenkins-bot has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/523985 )
Change subject: Revert "[bugfix] site.lang is different from site.code"
......................................................................
Revert "[bugfix] site.lang is different from site.code"
This reverts commit d9a0c49c4a73e08b2df30c680d250d8f8fdd9d9a.
Bug: T228322
Change-Id: Ic8a1a7b580dff8471dbb5cfb844dc9e329297373
---
M pywikibot/comms/http.py
1 file changed, 1 insertion(+), 1 deletion(-)
Approvals:
Xqt: Looks good to me, approved
jenkins-bot: Verified
diff --git a/pywikibot/comms/http.py b/pywikibot/comms/http.py
index 069237a..c1e1bc1 100644
--- a/pywikibot/comms/http.py
+++ b/pywikibot/comms/http.py
@@ -216,7 +216,7 @@
values.update({
'family': site.family.name if site else '',
'code': site.code if site else '',
- 'lang': site.lang if site else '',
+ 'lang': site.code if site else '', # TODO: use site.lang, if known
'site': str(site) if site else '',
'username': username,
'script_comments': '; '.join(script_comments)
--
To view, visit https://gerrit.wikimedia.org/r/523985
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Ic8a1a7b580dff8471dbb5cfb844dc9e329297373
Gerrit-Change-Number: 523985
Gerrit-PatchSet: 1
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: Dalba <dalba.wiki(a)gmail.com>
Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot (75)
jenkins-bot has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/523801 )
Change subject: Add user_agent_description option
......................................................................
Add user_agent_description option
https://meta.wikimedia.org/wiki/User-Agent_policy suggests that the client
should use descriptive name with contact info, etc. however pywikibot does
not provide an easy way to do it unless you use specific username (which
is impossible for SPARQL bots for example) or override whole user agent string.
script_name etc. only base on filenames and thus does not allow to add
more descriptive text or contact info.
Adding user_agent_description would allow to add one string to user-config.py like:
ser_agent_description = "https://example.org/CoolTool/; CoolTool(a)example.org"
and have it become part of the user-agent string.
Change-Id: I28a0446e117dab7fc778aebeecb1f9f0702f2502
---
M pywikibot/comms/http.py
M pywikibot/config2.py
2 files changed, 6 insertions(+), 0 deletions(-)
Approvals:
Xqt: Looks good to me, approved
jenkins-bot: Verified
diff --git a/pywikibot/comms/http.py b/pywikibot/comms/http.py
index c1e1bc1..b95eadd 100644
--- a/pywikibot/comms/http.py
+++ b/pywikibot/comms/http.py
@@ -203,6 +203,8 @@
script_comments = []
username = ''
+ if config.user_agent_description:
+ script_comments.append(config.user_agent_description)
if site:
script_comments.append(str(site))
diff --git a/pywikibot/config2.py b/pywikibot/config2.py
index b405b79..8ef04fc 100644
--- a/pywikibot/config2.py
+++ b/pywikibot/config2.py
@@ -156,6 +156,10 @@
user_agent_format = ('{script_product} ({script_comments}) {pwb} ({revision}) '
'{http_backend} {python}')
+# User agent description
+# This is a free-form string that can be user to describe specific bot/tool,
+# provide contact information, etc.
+user_agent_description = None
# Fake user agent.
# Some external websites reject bot-like user agents. It is possible to use
# fake user agents in requests to these websites.
--
To view, visit https://gerrit.wikimedia.org/r/523801
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I28a0446e117dab7fc778aebeecb1f9f0702f2502
Gerrit-Change-Number: 523801
Gerrit-PatchSet: 3
Gerrit-Owner: Smalyshev <smalyshev(a)wikimedia.org>
Gerrit-Reviewer: Dalba <dalba.wiki(a)gmail.com>
Gerrit-Reviewer: Dvorapa <dvorapa(a)seznam.cz>
Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: Smalyshev <smalyshev(a)wikimedia.org>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot (75)
jenkins-bot has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/523964 )
Change subject: [tests] Additional tests for APISite.fromDBName
......................................................................
[tests] Additional tests for APISite.fromDBName
- Take into account that 'test2wiki' is a wikpedia site and does not
have its own family file (other than 'testwiki' does)
- Take into account that 'testwikidatawiki' is a wikidata site
- Use a test list for testing APISite.fromDBName which is easier to expand
- use subTest for each test in the list
- Additional tests for APISite.fromDBName
Bug: T225723
Bug: T225590
Change-Id: I92bf81833c5e9611389fd13226f1086b5a6d71e5
---
M tests/site_tests.py
1 file changed, 18 insertions(+), 18 deletions(-)
Approvals:
Dalba: Looks good to me, approved
jenkins-bot: Verified
diff --git a/tests/site_tests.py b/tests/site_tests.py
index 7382d75..650a77d 100644
--- a/tests/site_tests.py
+++ b/tests/site_tests.py
@@ -321,28 +321,28 @@
def test_constructors(self):
"""Test cases for site constructors."""
+ test_list = [
+ ['enwiki', ('en', 'wikipedia')],
+ ['eswikisource', ('es', 'wikisource')],
+ ['dewikinews', ('de', 'wikinews')],
+ ['ukwikivoyage', ('uk', 'wikivoyage')],
+ ['metawiki', ('meta', 'meta')],
+ ['commonswiki', ('commons', 'commons')],
+ ['wikidatawiki', ('wikidata', 'wikidata')],
+ ['testwikidatawiki', ('test', 'wikidata')],
+ ['testwiki', ('test', 'test')], # see T225729
+ ['test2wiki', ('test2', 'wikipedia')], # see T225729
+ ['sourceswiki', ('mul', 'wikisource')], # see T226960
+ ]
if isinstance(self.site.family, pywikibot.family.WikimediaFamily):
site = self.site
else:
site = None
- self.assertEqual(pywikibot.site.APISite.fromDBName('enwiki', site),
- pywikibot.Site('en', 'wikipedia'))
- self.assertEqual(
- pywikibot.site.APISite.fromDBName('eswikisource', site),
- pywikibot.Site('es', 'wikisource'))
- self.assertEqual(pywikibot.site.APISite.fromDBName('dewikinews', site),
- pywikibot.Site('de', 'wikinews'))
- self.assertEqual(
- pywikibot.site.APISite.fromDBName('ukwikivoyage', site),
- pywikibot.Site('uk', 'wikivoyage'))
- self.assertEqual(pywikibot.site.APISite.fromDBName('metawiki', site),
- pywikibot.Site('meta', 'meta'))
- self.assertEqual(
- pywikibot.site.APISite.fromDBName('commonswiki', site),
- pywikibot.Site('commons', 'commons'))
- self.assertEqual(
- pywikibot.site.APISite.fromDBName('wikidatawiki', site),
- pywikibot.Site('wikidata', 'wikidata'))
+ for dbname, site_tuple in test_list:
+ with self.subTest(dbname=dbname):
+ self.assertEqual(
+ pywikibot.site.APISite.fromDBName(dbname, site),
+ pywikibot.Site(*site_tuple))
def test_language_methods(self):
"""Test cases for languages() and related methods."""
--
To view, visit https://gerrit.wikimedia.org/r/523964
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I92bf81833c5e9611389fd13226f1086b5a6d71e5
Gerrit-Change-Number: 523964
Gerrit-PatchSet: 2
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: Dalba <dalba.wiki(a)gmail.com>
Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: jenkins-bot (75)
jenkins-bot has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/523933 )
Change subject: [bugfix] site.lang is different from site.code
......................................................................
[bugfix] site.lang is different from site.code
Change-Id: I06c5d9663ddee2d6a579a370032516b827b9209f
---
M pywikibot/comms/http.py
1 file changed, 1 insertion(+), 1 deletion(-)
Approvals:
Dalba: Looks good to me, approved
jenkins-bot: Verified
diff --git a/pywikibot/comms/http.py b/pywikibot/comms/http.py
index c1e1bc1..069237a 100644
--- a/pywikibot/comms/http.py
+++ b/pywikibot/comms/http.py
@@ -216,7 +216,7 @@
values.update({
'family': site.family.name if site else '',
'code': site.code if site else '',
- 'lang': site.code if site else '', # TODO: use site.lang, if known
+ 'lang': site.lang if site else '',
'site': str(site) if site else '',
'username': username,
'script_comments': '; '.join(script_comments)
--
To view, visit https://gerrit.wikimedia.org/r/523933
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I06c5d9663ddee2d6a579a370032516b827b9209f
Gerrit-Change-Number: 523933
Gerrit-PatchSet: 1
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: Dalba <dalba.wiki(a)gmail.com>
Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: jenkins-bot (75)
jenkins-bot has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/523949 )
Change subject: fix(APISite.fromDBName): work for all known dbnames
......................................................................
fix(APISite.fromDBName): work for all known dbnames
APISite.fromDBName used to use `APISite(sitecode, sitecode)` for
instantiating special sites. But some of those sites, like test2wiki,
cannot be instantiated this way because pywikibot does not have any
family with that code (test2).
APISite.fromDBName:
* Use pywikibot.Site(url=...)
Family.from_url:
* Include test codes when searching for a domain.
wikidata_family.py and test_family.py:
* Remove test_codes attribute. Otherwise Family.from_url will end up
with duplicate codes in Family.from_url and raises
RuntimeError('Found multiple matches for URL ...').
I found two issues with this patch:
>>> Site('test', 'wikipedia') == Site('test', 'test') # should be True
False
>>> Site().fromDBName('testwiki') # should not warn
WARNING: Found multiple matches for URL \
"https://test.wikipedia.org/w/index.php": \
('test', Family("test")), ('test', Family("wikipedia")) (use first)
APISite("test", "test")
Note that the first issue is not introduced by this patch (
it already exists on master).
The cause of the mentioned issues is that we have two separate family
files that contain the same site. IOW, test.wikipeida.org is defined
both in test_family.py and in wikipedia_family.py. It should be removed
from one of them, but that sounds like a big breaking change and requires
a deprecation period. This patch is not going to get into that matter.
Bug: T225590
Bug: T225723
Bug: T226960
Change-Id: I42c7f34abd42c3eced06f1332177f3f7b91b0c40
---
M pywikibot/families/test_family.py
M pywikibot/families/wikidata_family.py
M pywikibot/family.py
M pywikibot/site.py
4 files changed, 4 insertions(+), 7 deletions(-)
Approvals:
Xqt: Looks good to me, approved
jenkins-bot: Verified
diff --git a/pywikibot/families/test_family.py b/pywikibot/families/test_family.py
index c2cc2f4..0e89716 100644
--- a/pywikibot/families/test_family.py
+++ b/pywikibot/families/test_family.py
@@ -17,7 +17,4 @@
name = 'test'
domain = 'test.wikipedia.org'
-
- test_codes = ('test', )
-
interwiki_forward = 'wikipedia'
diff --git a/pywikibot/families/wikidata_family.py b/pywikibot/families/wikidata_family.py
index fe11d15..e8012c4 100644
--- a/pywikibot/families/wikidata_family.py
+++ b/pywikibot/families/wikidata_family.py
@@ -17,7 +17,6 @@
"""Family class for Wikidata."""
name = 'wikidata'
- test_codes = ('test', 'beta')
langs = {
'wikidata': 'www.wikidata.org',
diff --git a/pywikibot/family.py b/pywikibot/family.py
index 77904e2..b0be37b 100644
--- a/pywikibot/family.py
+++ b/pywikibot/family.py
@@ -9,6 +9,7 @@
import collections
from importlib import import_module
+from itertools import chain
import logging
from os.path import basename, dirname, splitext
import re
@@ -1315,7 +1316,7 @@
else:
domain = False
if domain is not False:
- for code in self.codes:
+ for code in chain(self.codes, getattr(self, 'test_codes', ())):
if code in ignored:
continue
if self._hostname(code)[1] == parsed.netloc:
diff --git a/pywikibot/site.py b/pywikibot/site.py
index ee2c84c..bef3935 100644
--- a/pywikibot/site.py
+++ b/pywikibot/site.py
@@ -1886,10 +1886,10 @@
if site['code'] == 'wiki':
site['code'] = 'wikipedia'
return cls(lang, site['code'])
- else:
+ else: # key == 'specials'
for site in val:
if site['dbname'] == dbname:
- return cls(site['code'], site['code'])
+ return pywikibot.Site(url=site['url'] + '/w/index.php')
raise ValueError('Cannot parse a site out of %s.' % dbname)
@deprecated(since='20141225')
--
To view, visit https://gerrit.wikimedia.org/r/523949
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I42c7f34abd42c3eced06f1332177f3f7b91b0c40
Gerrit-Change-Number: 523949
Gerrit-PatchSet: 3
Gerrit-Owner: Dalba <dalba.wiki(a)gmail.com>
Gerrit-Reviewer: Dalba <dalba.wiki(a)gmail.com>
Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot (75)