jenkins-bot has submitted this change and it was merged.
Change subject: Category.subcategories() should yield objects that have the same properties, not just the same title, as the objects yielded by Category.members()
......................................................................
Category.subcategories() should yield objects that have the same properties,
not just the same title, as the objects yielded by Category.members()
Change-Id: I4e31a22393ba1ce3216e0c6a65978fae8ec0c6d4
---
M pywikibot/page.py
1 file changed, 1 insertion(+), 1 deletion(-)
Approvals:
Xqt: Looks good to me, approved
jenkins-bot: Verified
diff --git a/pywikibot/page.py b/pywikibot/page.py
index 141224c..ad5c935 100644
--- a/pywikibot/page.py
+++ b/pywikibot/page.py
@@ -1803,7 +1803,7 @@
for member in self.site.categorymembers(
self, namespaces=[14], step=step,
total=total, content=content):
- subcat = Category(self.site, member.title())
+ subcat = Category(member)
self._subcats.append(subcat)
yield subcat
if total is not None:
--
To view, visit https://gerrit.wikimedia.org/r/131644
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I4e31a22393ba1ce3216e0c6a65978fae8ec0c6d4
Gerrit-PatchSet: 2
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: Russell Blau <russblau(a)imapmail.org>
Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com>
Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl>
Gerrit-Reviewer: Ricordisamoa <ricordisamoa(a)live.it>
Gerrit-Reviewer: Russell Blau <russblau(a)imapmail.org>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot <>
jenkins-bot has submitted this change and it was merged.
Change subject: (bug 64997) Bugfix - Note: self.limit could be -1
......................................................................
(bug 64997) Bugfix - Note: self.limit could be -1
Partly revert of https://gerrit.wikimedia.org/r/#/c/130019/
testing of bool(self.limit) is True when it is -1.
We have to check whether self.limit > 0
Change-Id: If25add4c3ba7892416195337d5a79993baf6d0d9
---
M pywikibot/data/api.py
1 file changed, 2 insertions(+), 1 deletion(-)
Approvals:
Merlijn van Deen: Looks good to me, approved
jenkins-bot: Verified
diff --git a/pywikibot/data/api.py b/pywikibot/data/api.py
index de2865e..1a85255 100644
--- a/pywikibot/data/api.py
+++ b/pywikibot/data/api.py
@@ -760,7 +760,8 @@
# otherwise we proceed as usual
else:
count += 1
- if self.limit and count >= self.limit:
+ # note: self.limit could be -1
+ if self.limit > 0 and count >= self.limit:
return
if self.module == "random" and self.limit:
# "random" module does not return "query-continue"
--
To view, visit https://gerrit.wikimedia.org/r/131991
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: If25add4c3ba7892416195337d5a79993baf6d0d9
Gerrit-PatchSet: 2
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com>
Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl>
Gerrit-Reviewer: Russell Blau <russblau(a)imapmail.org>
Gerrit-Reviewer: jenkins-bot <>
jenkins-bot has submitted this change and it was merged.
Change subject: remove obsolete function ItemPage.__make_site()
......................................................................
remove obsolete function ItemPage.__make_site()
it is hackish, as it only supports Wikipedia, and unused,
since ItemPage.iterlinks() uses APISite.fromDBName()
Change-Id: I13c9756b84137449f283153b45999e331b01000c
---
M pywikibot/page.py
1 file changed, 0 insertions(+), 14 deletions(-)
Approvals:
Xqt: Looks good to me, approved
jenkins-bot: Verified
diff --git a/pywikibot/page.py b/pywikibot/page.py
index 66371e2..711acf4 100644
--- a/pywikibot/page.py
+++ b/pywikibot/page.py
@@ -2635,15 +2635,6 @@
i._title = page.title()
return i
- def __make_site(self, dbname):
- """
- Converts a Site.dbName() into a Site object.
- Rather hackish method that only works for WMF sites
- """
- lang = dbname.replace('wiki', '')
- lang = lang.replace('_', '-')
- return pywikibot.Site(lang, 'wikipedia')
-
def get(self, force=False, *args):
"""
Fetches all page data, and caches it
@@ -2666,11 +2657,6 @@
self.sitelinks = {}
if 'sitelinks' in self._content:
for dbname in self._content['sitelinks']:
- # Due to issues with locked/obsolete sites
- # this part is commented out
- #site = self.__make_site(dbname)
- #self.sitelinks[site] = pywikibot.Page(
- # site, self._content['sitelinks'][dbname]['title'])
self.sitelinks[dbname] = self._content[
'sitelinks'][dbname]['title']
--
To view, visit https://gerrit.wikimedia.org/r/131716
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I13c9756b84137449f283153b45999e331b01000c
Gerrit-PatchSet: 2
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: Ricordisamoa <ricordisamoa(a)live.it>
Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com>
Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot <>
jenkins-bot has submitted this change and it was merged.
Change subject: Make tests/api_tests.py run on py3k
......................................................................
Make tests/api_tests.py run on py3k
Syntactic/import changes:
comms/http.py syntax/import
data/wikidataquery.py import
pywikibot/textlib.py syntax
More elaborate changes:
comms/threadedhttp.py: import, plus adapt DummyResponse/DummyRequest
to have Python3 urllib's required parameters
pywikibot/throttle.py: close throttle file after use
userinterfaces/terminal_interface_unix.py: only encode when target stream
is a bytes stream (i.e. doesn't have encoding
property)
tests/api_tests.py: cleanup to more specific assertions.
Change-Id: Ib639e7dc70ffc5f192aefacf64e2770c80629c8a
---
M pywikibot/comms/http.py
M pywikibot/comms/threadedhttp.py
M pywikibot/data/wikidataquery.py
M pywikibot/textlib.py
M pywikibot/throttle.py
M pywikibot/userinterfaces/terminal_interface_unix.py
M tests/api_tests.py
7 files changed, 34 insertions(+), 15 deletions(-)
Approvals:
Xqt: Looks good to me, approved
jenkins-bot: Verified
diff --git a/pywikibot/comms/http.py b/pywikibot/comms/http.py
index a3a25fc..97f5f40 100644
--- a/pywikibot/comms/http.py
+++ b/pywikibot/comms/http.py
@@ -35,7 +35,7 @@
else:
from ssl import SSLError as SSLHandshakeError
import queue as Queue
- import urllib as urlparse
+ import urllib.parse as urlparse
from http import cookiejar as cookielib
from pywikibot import config
@@ -76,7 +76,7 @@
# Build up HttpProcessors
-pywikibot.log('Starting %(numthreads)i threads...' % locals())
+pywikibot.log(u'Starting %(numthreads)i threads...' % locals())
for i in range(numthreads):
proc = threadedhttp.HttpProcessor(http_queue, cookie_jar, connection_pool)
proc.setDaemon(True)
@@ -92,7 +92,7 @@
message = u'Waiting for %i network thread(s) to finish. Press ctrl-c to abort' % len(threads)
if hasattr(sys, 'last_type'):
# we quit because of an exception
- print sys.last_type
+ print(sys.last_type)
pywikibot.critical(message)
else:
pywikibot.log(message)
diff --git a/pywikibot/comms/threadedhttp.py b/pywikibot/comms/threadedhttp.py
index 3b6f6a8..f59060d 100644
--- a/pywikibot/comms/threadedhttp.py
+++ b/pywikibot/comms/threadedhttp.py
@@ -31,8 +31,10 @@
if sys.version_info[0] == 2:
import cookielib
+ from urllib import splittype, splithost, unquote
else:
from http import cookiejar as cookielib
+ from urllib.parse import splittype, splithost, unquote
import pywikibot
from pywikibot import config
@@ -392,10 +394,10 @@
self.url = url
self.headers = headers
self.origin_req_host = cookielib.request_host(self)
- self.type, r = urllib.splittype(url)
- self.host, r = urllib.splithost(r)
+ self.type, r = splittype(url)
+ self.host, r = splithost(r)
if self.host:
- self.host = urllib.unquote(self.host)
+ self.host = unquote(self.host)
def get_full_url(self):
return self.url
@@ -424,6 +426,8 @@
# TODO to match urllib2, this should be set to True when the
# request is the result of a redirect
return False
+
+ unverifiable = property(is_unverifiable)
class DummyResponse(object):
@@ -457,3 +461,9 @@
# to split carefully here - header.split(',') won't do it.
HEADERVAL = re.compile(r'\s*(([^,]|(,\s*\d))+)')
return [h[0] for h in HEADERVAL.findall(self.response[k])]
+
+ def get_all(self, k, failobj=None):
+ rv = self.getheaders(k)
+ if not rv:
+ return failobj
+ return rv
diff --git a/pywikibot/data/wikidataquery.py b/pywikibot/data/wikidataquery.py
index fdd66c7..79a315a 100644
--- a/pywikibot/data/wikidataquery.py
+++ b/pywikibot/data/wikidataquery.py
@@ -8,7 +8,11 @@
# Distributed under the terms of the MIT license.
import json
-import urllib2
+import sys
+if sys.version_info[0] == 2:
+ from urllib2 import quote
+else:
+ from urllib.parse import quote
from pywikibot.comms import http
import pickle
import os
@@ -421,7 +425,7 @@
Get the query string for a given query or queryset
@return query string including lables and props
"""
- qStr = "q=%s" % urllib2.quote(str(q))
+ qStr = "q=%s" % quote(str(q))
if labels:
qStr += "&labels=%s" % ','.join(labels)
diff --git a/pywikibot/textlib.py b/pywikibot/textlib.py
index 4535b4d..15e29b5 100644
--- a/pywikibot/textlib.py
+++ b/pywikibot/textlib.py
@@ -1180,7 +1180,7 @@
timeR = r'(?P<time>(?P<hour>[0-2]\d)[:\.h](?P<minute>[0-5]\d))'
timeznR = r'\((?P<tzinfo>[A-Z]+)\)'
yearR = r'(?P<year>(19|20)\d\d)'
- monthR = ur'(?P<month>(%s))' % (u'|'.join(self.origNames2monthNum))
+ monthR = r'(?P<month>(%s))' % (u'|'.join(self.origNames2monthNum))
dayR = r'(?P<day>(3[01]|[12]\d|0?[1-9]))'
self.ptimeR = re.compile(timeR)
diff --git a/pywikibot/throttle.py b/pywikibot/throttle.py
index 4e56464..95cc6b1 100644
--- a/pywikibot/throttle.py
+++ b/pywikibot/throttle.py
@@ -108,6 +108,7 @@
'site': this_site})
if not pid and this_pid >= my_pid:
my_pid = this_pid + 1 # next unused process id
+ f.close()
if not pid:
pid = my_pid
diff --git a/pywikibot/userinterfaces/terminal_interface_unix.py b/pywikibot/userinterfaces/terminal_interface_unix.py
index 1da2a6a..30e64ec 100755
--- a/pywikibot/userinterfaces/terminal_interface_unix.py
+++ b/pywikibot/userinterfaces/terminal_interface_unix.py
@@ -42,4 +42,7 @@
# just to be sure, reset the color
text += unixColors['default']
- targetStream.write(text.encode(self.encoding, 'replace'))
+ if hasattr(targetStream, 'encoding'):
+ targetStream.write(text)
+ else:
+ targetStream.write(text.encode(self.encoding, 'replace'))
diff --git a/tests/api_tests.py b/tests/api_tests.py
index f7401f2..7d7e14e 100644
--- a/tests/api_tests.py
+++ b/tests/api_tests.py
@@ -19,9 +19,9 @@
def testObjectCreation(self):
"""Test that api.Request() creates an object with desired attributes"""
req = api.Request(site=mysite, action="test", foo="", bar="test")
- self.assert_(req)
+ self.assertTrue(req)
self.assertEqual(req.site, mysite)
- self.assert_("foo" in req.params)
+ self.assertIn("foo", req.params)
self.assertEqual(req["bar"], "test")
# test item assignment
req["one"] = "1"
@@ -29,8 +29,9 @@
# test compliance with dict interface
# req.keys() should contain "action", "foo", "bar", "one"
self.assertEqual(len(req.keys()), 4)
- self.assert_("test" in req.values())
- self.assert_(all(len(item) == 2 for item in req.items()))
+ self.assertIn("test", req.values())
+ for item in req.items():
+ self.assertEqual(len(item), 2, item)
class TestPageGenerator(PywikibotTestCase):
@@ -70,7 +71,7 @@
for page in results:
self.assertEqual(type(page), pywikibot.Page)
self.assertEqual(page.site, mysite)
- self.assert_(page.title() in titles)
+ self.assertIn(page.title(), titles)
class TestCachedRequest(unittest.TestCase):
--
To view, visit https://gerrit.wikimedia.org/r/130023
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: Ib639e7dc70ffc5f192aefacf64e2770c80629c8a
Gerrit-PatchSet: 3
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: Merlijn van Deen <valhallasw(a)arctus.nl>
Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com>
Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot <>
jenkins-bot has submitted this change and it was merged.
Change subject: Make __str__ python3 compatible
......................................................................
Make __str__ python3 compatible
To make a python 2 & 3 compatible code base, we need to have
a __str__ function that returns bytes on python 2 and unicode
on python 3. This Unicodemixin handles this, by using
__unicode__, and encoding that to utf-8 to get bytes.
Change-Id: I7229b884b630c0d05e2ebe171edec81511d7940b
---
M pywikibot/exceptions.py
M pywikibot/page.py
M pywikibot/tools.py
3 files changed, 22 insertions(+), 18 deletions(-)
Approvals:
Xqt: Looks good to me, approved
jenkins-bot: Verified
diff --git a/pywikibot/exceptions.py b/pywikibot/exceptions.py
index f029147..7d4ba61 100644
--- a/pywikibot/exceptions.py
+++ b/pywikibot/exceptions.py
@@ -10,23 +10,16 @@
__version__ = '$Id$'
-from pywikibot import config
+from pywikibot.tools import UnicodeMixin
# TODO: These are copied from wikipedia.py; not certain that all of them
# will be needed in the rewrite.
-class Error(Exception):
+class Error(Exception, UnicodeMixin):
"""Wikipedia error"""
def __init__(self, arg):
self.unicode = arg
- try:
- self.string = arg.encode(config.console_encoding, "xmlcharrefreplace")
- except (AttributeError, TypeError):
- self.string = arg.encode("ascii", "xmlcharrefreplace")
-
- def __str__(self):
- return self.string
def __unicode__(self):
return self.unicode
diff --git a/pywikibot/page.py b/pywikibot/page.py
index 1b9ec62..5f68c9c 100644
--- a/pywikibot/page.py
+++ b/pywikibot/page.py
@@ -42,7 +42,7 @@
# Note: Link objects (defined later on) represent a wiki-page's title, while
# Page objects (defined here) represent the page itself, including its contents.
-class Page(object):
+class Page(pywikibot.UnicodeMixin):
"""Page: A MediaWiki page
This object only implements internally methods that do not require
@@ -214,12 +214,6 @@
"""
return self._link.section
-
- def __str__(self):
- """Return a console representation of the pagelink."""
- return self.title(asLink=True, forceInterwiki=True
- ).encode(config.console_encoding,
- "xmlcharrefreplace")
def __unicode__(self):
return self.title(asLink=True, forceInterwiki=True)
diff --git a/pywikibot/tools.py b/pywikibot/tools.py
index 299a248..3b5671b 100644
--- a/pywikibot/tools.py
+++ b/pywikibot/tools.py
@@ -10,7 +10,24 @@
import sys
import threading
import time
-import Queue
+
+if sys.version_info[0] > 2:
+ import queue as Queue
+else:
+ import Queue
+
+
+class UnicodeMixin(object):
+ """Mixin class to handle defining the proper __str__/__unicode__
+ methods in Python 2 or 3.
+ """
+
+ if sys.version_info[0] >= 3:
+ def __str__(self):
+ return self.__unicode__()
+ else:
+ def __str__(self):
+ return self.__unicode__().encode('utf8')
class ThreadedGenerator(threading.Thread):
@@ -141,7 +158,7 @@
...
"""
- def __init__(self, limit=sys.maxint, *args):
+ def __init__(self, limit=128, *args):
self.limit = limit
list.__init__(self, *args)
for item in list(self):
--
To view, visit https://gerrit.wikimedia.org/r/130021
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I7229b884b630c0d05e2ebe171edec81511d7940b
Gerrit-PatchSet: 4
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: Merlijn van Deen <valhallasw(a)arctus.nl>
Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com>
Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot <>
jenkins-bot has submitted this change and it was merged.
Change subject: Add mwparserfromhell as dependency
......................................................................
Add mwparserfromhell as dependency
Even though mwparserfromhell is technically optional,
having mwpfh as dependency will make sure people who
setup.py install pywikibot will have it installed.
People using pwb.py are not affected by this, although we
should probably add a warning at some point.
Change-Id: Iaa77c0f7e0f7885f08c90581bde86c2c2aa88720
---
M setup.py
1 file changed, 2 insertions(+), 1 deletion(-)
Approvals:
Xqt: Looks good to me, approved
jenkins-bot: Verified
diff --git a/setup.py b/setup.py
index 580f9b1..cf14a97 100644
--- a/setup.py
+++ b/setup.py
@@ -63,7 +63,8 @@
for package in find_packages()
if package.startswith('pywikibot.')],
install_requires=[
- 'httplib2>=0.6.0'
+ 'httplib2>=0.6.0',
+ 'mwparserfromhell>=0.3.3'
],
dependency_links=[
'https://git.wikimedia.org/zip/?r=pywikibot/externals/httplib2.git&format=gz…'
--
To view, visit https://gerrit.wikimedia.org/r/131263
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: Iaa77c0f7e0f7885f08c90581bde86c2c2aa88720
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: Merlijn van Deen <valhallasw(a)arctus.nl>
Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com>
Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl>
Gerrit-Reviewer: Ricordisamoa <ricordisamoa(a)live.it>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot <>