[Pywikipedia-bugs] [ pywikipediabot-Bugs-3428346 ] utf8 error in category.py move

Tue Oct 25 17:58:37 UTC 2011

Bugs item #3428346, was opened at 2011-10-25 17:58
Message generated for change (Tracker Item Submitted) made by nobody
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=603138&aid=3428346&group_id=93107

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Nobody/Anonymous (nobody)
Assigned to: Nobody/Anonymous (nobody)
Summary: utf8 error in category.py move

Initial Comment:
System Environment: Windows 7 Home Basic X64

Action: category.py move
Exact command from attached commands.log:
2011-10-25 19:36:06 r9648 Python 2.7.2 category.py "-ns:0" "summary:fix_category" "move"

Old Category name: कैलण्डर
Old category name input in cmd as: %E0%A4%95%E0%A5%88%E0%A4%B2%E0%A4%A3%E0%A5%8D%E0%A4%A1%E0%A4%B0

New Category name: वर्ष
New category name input in cmd as: %E0%A4%B5%E0%A4%B0%E0%A5%8D%E0%A4%B7

The old category page did not exist, the new category page did. Both category names were in hindi(devanagari script). The old category contained approximately 9000 articles. The bot didn't use the edit summary I gave and instead used the default summary(Maybe I did something wrong?) After 600 edits, the bot gave the following output in cmd(the gibberish is probably devanagari as rendered by cmd)(the windows username has been changed for anonymity):

Getting 60 pages from wikipedia:hi...
'utf8' codec can't decode bytes in position 127815-127816: unexpected end of dat
a
Traceback (most recent call last):
  File "C:\Users\Me\Downloads\pywikipedia-nightly\pywikipedia\pagegenerators.py
", line 1191, in __iter__
    for loaded_page in self.preload(somePages):
  File "C:\Users\Me\Downloads\pywikipedia-nightly\pywikipedia\pagegenerators.py
", line 1216, in preload
    pywikibot.getall(site, pagesThisSite)
  File "C:\Users\Me\Downloads\pywikipedia-nightly\pywikipedia\wikipedia.py", li
ne 4440, in getall
    _GetAll(site, pages, throttle, force).run()
  File "C:\Users\Me\Downloads\pywikipedia-nightly\pywikipedia\wikipedia.py", li
ne 4071, in run
    data = self.getData()
  File "C:\Users\Me\Downloads\pywikipedia-nightly\pywikipedia\wikipedia.py", li
ne 4253, in getData
    response, data = self.site.postForm(address, predata)
  File "C:\Users\Me\Downloads\pywikipedia-nightly\pywikipedia\wikipedia.py", li
ne 5279, in postForm
    cookies=self.cookies(sysop = sysop))
  File "C:\Users\Me\Downloads\pywikipedia-nightly\pywikipedia\wikipedia.py", li
ne 5408, in postData
    % (self.protocol(), self.hostname(), path))
NameError: global name 'path' is not defined
global name 'path' is not defined
Getting [[cnfnrenii:qailnfnrr]]...
Getting [[cnfnrenii:qailnfnrr]] list from cmcontinue=page|e0a4aae0a582e0a4b0e0a5
8de0a4b52c20e0a5a8e0a5a6e0a5abe0a5ab20e0a488e0a4b8e0a4be0ae0a5a8e0a5a6e0a5abe0a5
ab20e0a488e0a4b8e0a4be20e0a4aae0a582e0a4b0e0a58de0a4b5|230770...
Getting [[cnfnrenii:qailnfnrr]] list from cmcontinue=page|e0a4aae0a582e0a4b0e0a5
8de0a4b52c20e0a5a8e0a5abe0a5a6e0a5ab20e0a488e0a4b8e0a4be0ae0a5a8e0a5abe0a5a6e0a5
ab20e0a488e0a4b8e0a4be20e0a4aae0a582e0a4b0e0a58de0a4b5|231220...
Getting [[cnfnrenii:qailnfnrr]] list from cmcontinue=page|e0a4aae0a582e0a4b0e0a5
8de0a4b52c20e0a5a8e0a5afe0a5abe0a5ac20e0a488e0a4b8e0a4be0ae0a5a8e0a5afe0a5abe0a5
ac20e0a488e0a4b8e0a4be20e0a4aae0a582e0a4b0e0a58de0a4b5|231671...
Getting [[cnfnrenii:qailnfnrr]] list from cmcontinue=page|e0a4aae0a582e0a4b0e0a5
8de0a4b52c20e0a5a9e0a5aae0a5a6e0a5ae20e0a488e0a4b8e0a4be0ae0a5a9e0a5aae0a5a6e0a5
ae20e0a488e0a4b8e0a4be20e0a4aae0a582e0a4b0e0a58de0a4b5|232122...
Getting [[cnfnrenii:qailnfnrr]] list from cmcontinue=page|e0a4aae0a582e0a4b0e0a5
8de0a4b52c20e0a5a9e0a5aee0a5abe0a5af20e0a488e0a4b8e0a4be0ae0a5a9e0a5aee0a5abe0a5
af20e0a488e0a4b8e0a4be20e0a4aae0a582e0a4b0e0a58de0a4b5|232573...
Getting [[cnfnrenii:qailnfnrr]] list from cmcontinue=page|e0a4aae0a582e0a4b0e0a5
8de0a4b52c20e0a5aae0a5a9e0a5a6e0a5ae20e0a488e0a4b8e0a4be0ae0a5aae0a5a9e0a5a6e0a5
ae20e0a488e0a4b8e0a4be20e0a4aae0a582e0a4b0e0a58de0a4b5|235031...
Getting [[cnfnrenii:qailnfnrr]] list from cmcontinue=page|e0a4aae0a582e0a4b0e0a5
8de0a4b52c20e0a5aae0a5ade0a5abe0a5af20e0a488e0a4b8e0a4be0ae0a5aae0a5ade0a5abe0a5
af20e0a488e0a4b8e0a4be20e0a4aae0a582e0a4b0e0a58de0a4b5|235482...
Getting [[cnfnrenii:qailnfnrr]] list from cmcontinue=page|e0a4aae0a582e0a4b0e0a5
8de0a4b52c20e0a5abe0a5a8e0a5a6e0a5af20e0a488e0a4b8e0a4be0ae0a5abe0a5a8e0a5a6e0a5
af20e0a488e0a4b8e0a4be20e0a4aae0a582e0a4b0e0a58de0a4b5|232923...
Getting [[cnfnrenii:qailnfnrr]] list from cmcontinue=page|e0a4aae0a582e0a4b0e0a5
8de0a4b52c20e0a5abe0a5ace0a5ace0a5a620e0a488e0a4b8e0a4be0ae0a5abe0a5ace0a5ace0a5
a620e0a488e0a4b8e0a4be20e0a4aae0a582e0a4b0e0a58de0a4b5|233373...
Getting [[cnfnrenii:qailnfnrr]] list from cmcontinue=page|e0a4aae0a582e0a4b0e0a5
8de0a4b52c20e0a5ace0a5a7e0a5a720e0a488e0a4b8e0a4be0ae0a5ace0a5a7e0a5a720e0a488e0
a4b8e0a4be20e0a4aae0a582e0a4b0e0a58de0a4b5|229329...
Getting [[cnfnrenii:qailnfnrr]] list from cmcontinue=page|e0a4aae0a582e0a4b0e0a5
8de0a4b52c20e0a5ace0a5abe0a5ace0a5a620e0a488e0a4b8e0a4be0ae0a5ace0a5abe0a5ace0a5
a620e0a488e0a4b8e0a4be20e0a4aae0a582e0a4b0e0a58de0a4b5|234283...
Getting [[cnfnrenii:qailnfnrr]] list from cmcontinue=page|e0a4aae0a582e0a4b0e0a5
8de0a4b52c20e0a5ade0a5a6e0a5a720e0a488e0a4b8e0a4be0ae0a5ade0a5a6e0a5a720e0a488e0
a4b8e0a4be20e0a4aae0a582e0a4b0e0a58de0a4b5|229419...
Getting [[cnfnrenii:qailnfnrr]] list from cmcontinue=page|e0a4aae0a582e0a4b0e0a5
8de0a4b52c20e0a5ade0a5aae0a5ace0a5a620e0a488e0a4b8e0a4be0ae0a5ade0a5aae0a5ace0a5
a620e0a488e0a4b8e0a4be20e0a4aae0a582e0a4b0e0a58de0a4b5|236183...
Getting [[cnfnrenii:qailnfnrr]] list from cmcontinue=page|e0a4aae0a582e0a4b0e0a5
8de0a4b52c20e0a5ade0a5afe0a5a7e0a5a620e0a488e0a4b8e0a4be0ae0a5ade0a5afe0a5a7e0a5
a620e0a488e0a4b8e0a4be20e0a4aae0a582e0a4b0e0a58de0a4b5|236634...
Getting [[cnfnrenii:qailnfnrr]] list from cmcontinue=page|e0a4aae0a582e0a4b0e0a5
8de0a4b52c20e0a5aee0a5a9e0a5ace0a5a720e0a488e0a4b8e0a4be0ae0a5aee0a5a9e0a5ace0a5
a720e0a488e0a4b8e0a4be20e0a4aae0a582e0a4b0e0a58de0a4b5|237086...
Getting [[cnfnrenii:qailnfnrr]] list from cmcontinue=page|e0a4aae0a582e0a4b0e0a5
8de0a4b52c20e0a5aee0a5aee0a5a7e0a5a720e0a488e0a4b8e0a4be0ae0a5aee0a5aee0a5a7e0a5
a720e0a488e0a4b8e0a4be20e0a4aae0a582e0a4b0e0a58de0a4b5|237536...
Getting 15 pages from wikipedia:hi...
Updating page [[cnfnrenii:2007|32303037]] via API
Sleeping for 6.3 seconds, 2011-10-25 21:52:30
Updating page [[cnfnrenii:mqfntnuubr|e0a485e0a495e0a58de0a4a4e0a582e0a4ace0a4b0]
] via API
Updating page [[cnfnrenii:mgsfntn|e0a485e0a497e0a4b8e0a58de0a4a4]] via API
Updating page [[cnfnrenii:mpafnrail|e0a485e0a4aae0a58de0a4b0e0a588e0a4b2]] via A
PI
Updating page [[cnfnrenii:isfnlaamii qailenfnrr|e0a487e0a4b8e0a58de0a4b2e0a4bee0
a4aee0a58020e0a495e0a588e0a4b2e0a587e0a4a3e0a58de0a4a1e0a4b0]] via API
Sleeping for 7.3 seconds, 2011-10-25 21:53:09
Updating page [[cnfnrenii:znvrii|e0a49ce0a4a8e0a4b5e0a4b0e0a580]] via API
Sleeping for 5.7 seconds, 2011-10-25 21:53:20
Updating page [[cnfnrenii:zulaaii|e0a49ce0a581e0a4b2e0a4bee0a488]] via API
Sleeping for 5.5 seconds, 2011-10-25 21:53:31
Updating page [[cnfnrenii:zuun|e0a49ce0a582e0a4a8]] via API
Updating page [[cnfnrenii:dnis?br|e0a4a6e0a4bfe0a4b8e0a482e0a4ace0a4b0]] via API

Sleeping for 4.3 seconds, 2011-10-25 21:53:52
Updating page [[cnfnrenii:nv?br|e0a4a8e0a4b5e0a482e0a4ace0a4b0]] via API
Sleeping for 5.4 seconds, 2011-10-25 21:54:01
Updating page [[cnfnrenii:frvrii|e0a4abe0a4b0e0a4b5e0a4b0e0a580]] via API
Sleeping for 5.5 seconds, 2011-10-25 21:54:11
Updating page [[cnfnrenii:mii|e0a4aee0a488]] via API
Sleeping for 5.1 seconds, 2011-10-25 21:54:21
Updating page [[cnfnrenii:maarfnc|e0a4aee0a4bee0a4b0e0a58de0a49a]] via API
Updating page [[cnfnrenii:sitn?br|e0a4b8e0a4bfe0a4a4e0a482e0a4ace0a4b0]] via API

Sleeping for 5.7 seconds, 2011-10-25 21:54:40
Updating page [[cnfnrenii:fninfndnuu qaile?rr|e0a4b9e0a4bfe0a4a8e0a58de0a4a6e0a5
8220e0a495e0a588e0a4b2e0a587e0a482e0a4a1e0a4b0]] via API

After this, the bot stopped even though there were a lot of articles remaining in the old category. Note that the last 15 edits done by the bot are actually subcategory moves, not article moves as originally intended.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=603138&aid=3428346&group_id=93107