Bugs item #3160376, was opened at 2011-01-18 00:08 Message generated for change (Tracker Item Submitted) made by nobody You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=603138&aid=3160376...
Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: Non-ascii cookies not supported
Initial Comment: Not fully clear whether this is a pwb or a python bug.
Reported by: Creeper == http://callofduty.wikia.com/wiki/User:Ikin Wiki: callofduty.wikia.com, autogenerated family file Location: Somewhere where wikia puts an å in the cookie.
Initial output: C:\pywikipedia>redirect.py double unicode test: triggers problem #3081100 Retrieving special page... 'ascii' codec can't decode byte 0xe5 in position 282: ordinal not in range(128) WARNING: Could not open 'http://callofduty.wikia.com/api.php?action=query&format =json&meta=userinfo&uiprop=blockinfo'. Maybe the server or your connection is down. Retrying in 1 minutes...
After removing two catch-alls: 00:10 < valhalla1w> open wikipedia.py, search for output(u'%s' %e) 00:10 < valhalla1w> change that to 'raise' 00:11 < valhalla1w> replace output(...) with raise 00:11 < valhalla1w> then search again and replace again 00:12 < valhalla1w> (there are two hits)
and
00:41 < valhalla1w> open query.py, search for Error downloading data 00:41 < valhalla1w> change that line to 'raise', too, and try again
and some pdb magic: C:\pywikipedia> c:\python26\python -m pdb redirect.py double
c:\pywikipedia\redirect.py(53)<module>()
-> """ (Pdb) c unicode test: triggers problem #3081100 Retrieving special page... Traceback (most recent call last): File "c:\python26\lib\pdb.py", line 1296, in main pdb._runscript(mainpyfile) File "c:\python26\lib\pdb.py", line 1215, in _runscript self.run(statement) File "c:\python26\lib\bdb.py", line 372, in run exec cmd in globals, locals File "<string>", line 1, in <module> File "redirect.py", line 911, in <module> main() File "redirect.py", line 907, in main bot.run() File "redirect.py", line 823, in run self.fix_double_redirects() File "redirect.py", line 650, in fix_double_redirects for redir_name in self.generator.retrieve_double_redirects(): File "redirect.py", line 463, in retrieve_double_redirects maintenance_txt = self.site.getUrl(path) File "wikipedia.py", line 5469, in getUrl self._getUserDataOld(text, sysop = sysop) File "wikipedia.py", line 5597, in _getUserDataOld blocked = self._getBlock(sysop = sysop) File "wikipedia.py", line 4910, in _getBlock data = query.GetData(params, self) File "query.py", line 127, in GetData jsontext = site.getUrl( path, retry=True, sysop=sysop, data=data) File "wikipedia.py", line 5365, in getUrl f = MyURLopener.open(request) File "c:\python26\lib\urllib2.py", line 391, in open response = self._open(req, data) File "c:\python26\lib\urllib2.py", line 409, in _open '_open', req) File "c:\python26\lib\urllib2.py", line 369, in _call_chain result = func(*args) File "c:\python26\lib\urllib2.py", line 1170, in http_open return self.do_open(httplib.HTTPConnection, req) File "c:\python26\lib\urllib2.py", line 1142, in do_open h.request(req.get_method(), req.get_selector(), req.data, headers) File "c:\python26\lib\httplib.py", line 914, in request self._send_request(method, url, body, headers) File "c:\python26\lib\httplib.py", line 951, in _send_request self.endheaders() File "c:\python26\lib\httplib.py", line 908, in endheaders self._send_output() File "c:\python26\lib\httplib.py", line 778, in _send_output msg = "\r\n".join(self._buffer) UnicodeDecodeError: 'ascii' codec can't decode byte 0xe5 in position 291: ordina l not in range(128) Uncaught exception. Entering post mortem debugging Running 'cont' or 'step' will restart the program
c:\python26\lib\httplib.py(778)_send_output()
-> msg = "\r\n".join(self._buffer) (Pdb) (Pdb) self._buffer [u'GET /api.php?action=query&format=json&meta=userinfo&uiprop=blockinfo HTTP/1.1 ', u'Host: callofduty.wikia.com', 'User-Agent: PythonWikipediaBot/1.0', 'Cookie: cookieprefix=None; wikicitiesUserName=xxxt; loadtime=S1295308051.857639313,VS0 ,AS69,MD14,DD102,AE538,VE753; wikicitiesToken=7xxxxxxx; wikicities_session=xxxxx; varnish-stat=/server/LON/va rnish-l8-LON/SESSION/; Geo = {"city":"Kabelv\xe5g","country":"NO","continent":"E U"}; wikicitiesUserID=3116284; ', 'Connection: close', 'Accept-Encoding: gzip', '', '']
Testing this directly confirms the problem:
buffer = [u'GET /api.php?action=query&format=json&meta=userinfo&uiprop=blockinfo HTTP/1.1', u'Host: callofduty.wikia.com', 'User-Agent: PythonWikipediaBot/1.0', 'Cookie: cookieprefix=None; wikicitiesUserName=xxxx; loadtime=S1295308051.857639313,VS0,AS69,MD14,DD102,AE538,VE753; wikicitiesToken=xxxx;wikicities_session=xxx; varnish-stat=/server/LON/varnish-l8-LON/SESSION/; Geo = {"city":"Kabelv\xe5g","country":"NO","continent":"EU"}; wikicitiesUserID=xxx; ', 'Connection: close', 'Accept-Encoding: gzip','', ''] "\r\n".join(buffer)
Traceback (most recent call last): File "<stdin>", line 1, in <module> UnicodeDecodeError: 'ascii' codec can't decode byte 0xe5 in position 290: ordinal not in range(128)
As a temporary solution, I suggested editing c:\python26\lib\httplib.py and replacing line msg = "\r\n".join(self._buffer) with "\r\n".join([x.encode('utf-8') if isinstance(x, unicode) else x for x in self._buffer])
----------------------------------------------------------------------
You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=603138&aid=3160376...