https://bugzilla.wikimedia.org/show_bug.cgi?id=66619
Bug ID: 66619 Summary: TypeError: list indices must be integers, not str Product: Pywikibot Version: core (2.0) Hardware: All OS: All Status: NEW Severity: normal Priority: Unprioritized Component: Wikidata Assignee: Pywikipedia-bugs@lists.wikimedia.org Reporter: maarten@mdammers.nl Web browser: --- Mobile Platform: ---
File "C:\pywikibot\coredev\rijksmonumenten_import.py", line 106, in run result = self.repo.editEntity(identification, data, summary=summary) File "C:\pywikibot\coredev\pywikibot\site.py", line 566, in callee return fn(self, *args, **kwargs) File "C:\pywikibot\coredev\pywikibot\site.py", line 3750, in editEntity data = req.submit() File "C:\pywikibot\coredev\pywikibot\data\api.py", line 414, in submit message = result["error"]["messages"]["0"]["name"] TypeError: list indices must be integers, not str <type 'exceptions.TypeError'> CRITICAL: Waiting for 1 network thread(s) to finish. Press ctrl-c to abort
We use ["0"] and that used to work. Maybe the upstream api was changed? This is a bit hard to reproduce.
https://bugzilla.wikimedia.org/show_bug.cgi?id=66619
--- Comment #1 from Gerrit Notification Bot gerritadmin@wikimedia.org --- Change 139794 had a related patch set uploaded by Xqt: (bug 66619) Ignore TypeError for bug 46535
https://gerrit.wikimedia.org/r/139794
https://bugzilla.wikimedia.org/show_bug.cgi?id=66619
Gerrit Notification Bot gerritadmin@wikimedia.org changed:
What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |PATCH_TO_REVIEW
https://bugzilla.wikimedia.org/show_bug.cgi?id=66619
xqt info@gno.de changed:
What |Removed |Added ---------------------------------------------------------------------------- CC| |info@gno.de
--- Comment #2 from xqt info@gno.de --- rijksmonumenten_import.py is not part of the framework. Could you give more information about values of editEntity(identification, data, summary=summary)?
https://bugzilla.wikimedia.org/show_bug.cgi?id=66619
--- Comment #3 from xqt info@gno.de --- Or place a print statement in api.py line 412:
print result["error"]
to get the whole error result
https://bugzilla.wikimedia.org/show_bug.cgi?id=66619
--- Comment #4 from Maarten Dammers maarten@mdammers.nl --- I haven't been able to reproduce this. Bot has been running for quite some time.
My assumption would be that the 'result["error"]["messages"]["0"]["name"]' causes the error because it's not four layers of dicts, but has a list in there somewhere.
Anyone a clue how to trigger an error to see if the format of error messages might have changed?
The relevant snippet that triggered tis bug:
data = {'labels': {monument.get('lang'): {'language': monument.get('lang'), 'value': monumentName} } } identification = {} summary = u'Creating new item with data from %s' % (monument.get('source'),) pywikibot.output(summary) result = self.repo.editEntity(identification, data, summary=summary)
https://bugzilla.wikimedia.org/show_bug.cgi?id=66619
Maarten Dammers maarten@mdammers.nl changed:
What |Removed |Added ---------------------------------------------------------------------------- Status|PATCH_TO_REVIEW |NEW CC| |daniel.kinzler@wikimedia.de | |, | |legoktm.wikipedia@gmail.com | |, | |lydia.pintscher@wikimedia.d | |e
--- Comment #5 from Maarten Dammers maarten@mdammers.nl --- Daniel probably broke it in https://gerrit.wikimedia.org/r/#/c/124323/
I asked him what to expect back in april, but never got a response http://lists.wikimedia.org/pipermail/wikidata-tech/2014-April/000473.html
The Wikidata technical documentation is a mess so I can't find what to expect.
Wikidata people, could you please provide us, your downstream users, with some decent documentation on what to expect?
https://bugzilla.wikimedia.org/show_bug.cgi?id=66619
Lydia Pintscher lydia.pintscher@wikimedia.de changed:
What |Removed |Added ---------------------------------------------------------------------------- CC| |wikidata-bugs@lists.wikimed | |ia.org Whiteboard| |u=dev c=backend p=0
https://bugzilla.wikimedia.org/show_bug.cgi?id=66619
--- Comment #6 from xqt info@gno.de --- Maarten, as you declined just to ignore the TypeError, I've uploaded a new path that should log the api request and its error content.
https://bugzilla.wikimedia.org/show_bug.cgi?id=66619
--- Comment #7 from xqt info@gno.de --- See also bug 45277
https://bugzilla.wikimedia.org/show_bug.cgi?id=66619
--- Comment #8 from Thiemo Mättig thiemo.maettig@wikimedia.de --- I don't know exactly what's happening here but it seems I have a very similar patch in review at https://gerrit.wikimedia.org/r/#/c/138660/
Please note that result["error"]["messages"] is not an array but an object (at least that's what I see in the JavaScript code my patch touches). I wonder why Python complains. Accessing the elements via ["0"] should work.
https://bugzilla.wikimedia.org/show_bug.cgi?id=66619
--- Comment #9 from Maarten Dammers maarten@mdammers.nl --- "0" is a string. If I would have a dict and a list:
somedict = { "0" : "bla", "1" : "more bla" } somelist = ["bla", "more bla"]
I would access the "bla with somedict["0"] ("0" is a string here) and in the list I would do somelist[0] to access "bla.
somedict = { "0" : "bla", "1" : "more bla" } somelist = ["bla", "more bla"] print somedict["0"]
bla
print somelist["0"]
Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: list indices must be integers, not str
print somelist[0]
bla
So if someone changed the output from a dict ("0") to list (0), that would explain this error.
https://bugzilla.wikimedia.org/show_bug.cgi?id=66619
--- Comment #10 from xqt info@gno.de --- In past we had a structure like the following for the error:
{u'error': {u'code': u'failed-save', u'info': u'Another item (Q1151870) already has label "disambiguation" and description "Wikimedia disambiguation page" associated with language code en.', u'messages': {u'0': {u'name': u'wikibase-error-label-not-unique-item', u'parameters': [u'disambiguation', u'en', {}, u'Wikimedia disambiguation page'], u'type': u'error'}, u'html': {u'*': u'<p>Another item (Q1151870) already has label "disambiguation" and description "Wikimedia disambiguation page" associated with language code en.\n</p>'}}}, u'servedby': u'mw1199'}
now result['error']['messages']['0']['name'] does not work anymore to get the u'wikibase-error-label-not-unique-item',
https://bugzilla.wikimedia.org/show_bug.cgi?id=66619
--- Comment #11 from Thiemo Mättig thiemo.maettig@wikimedia.de --- Guys, you realize somebody needs to post an example API request that contains an error message and shows the wrong structure we are talking about? As I said, from what I know there is nothing wrong with the API. At the moment this looks like a coding error in Pywikibot to me. Which is nothing the Wikidata team can fix.
https://bugzilla.wikimedia.org/show_bug.cgi?id=66619
--- Comment #12 from Maarten Dammers maarten@mdammers.nl --- Yup, someone changed the output of the api, bot just crashed with this message:
{u'servedby': u'mw1129', u'error': {u'messages': [{u'html': {u'*': u'Could not create a new page.\nIt already exists.'}, u'name': u'edit-already-exists', u'parameters': []}]}}
https://bugzilla.wikimedia.org/show_bug.cgi?id=66619
--- Comment #13 from Thiemo Mättig thiemo.maettig@wikimedia.de --- (In reply to Maarten Dammers from comment #12)
bot just crashed with this message:
Again: Example URL?
https://bugzilla.wikimedia.org/show_bug.cgi?id=66619
John Mark Vandenberg jayvdb@gmail.com changed:
What |Removed |Added ---------------------------------------------------------------------------- CC| |jayvdb@gmail.com
--- Comment #14 from John Mark Vandenberg jayvdb@gmail.com --- (In reply to Maarten Dammers from comment #12)
Yup, someone changed the output of the api, bot just crashed with this message:
{u'servedby': u'mw1129', u'error': {u'messages': [{u'html': {u'*': u'Could not create a new page.\nIt already exists.'}, u'name': u'edit-already-exists', u'parameters': []}]}}
Has anyone seen this bug except with 'edit-already-exists'? It may be that one error is problematic, as it is a lower level part of the code which emits this error.
https://bugzilla.wikimedia.org/show_bug.cgi?id=66619
--- Comment #15 from Maarten Dammers maarten@mdammers.nl --- (In reply to Thiemo Mättig from comment #13)
(In reply to Maarten Dammers from comment #12)
bot just crashed with this message:
Again: Example URL?
Not possible. All POST and intermittent.
https://bugzilla.wikimedia.org/show_bug.cgi?id=66619
xqt info@gno.de changed:
What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |PATCH_TO_REVIEW
--- Comment #16 from xqt info@gno.de --- I uploaded a new PS to fix the API breaking change in gerrit 139794
https://bugzilla.wikimedia.org/show_bug.cgi?id=66619
--- Comment #17 from Maarten Dammers maarten@mdammers.nl --- I was thinking. Both cases might be mixed depending on what module is throwing the error. Maybe we should grab ['error']['messages'] and check if it's a list or a dict and depending on that, grab the message.
That way we don't care any more about the format, we just accept both formats.
https://bugzilla.wikimedia.org/show_bug.cgi?id=66619
--- Comment #18 from xqt info@gno.de --- Yes, have a look at my PS.
https://bugzilla.wikimedia.org/show_bug.cgi?id=66619
--- Comment #19 from Thiemo Mättig thiemo.maettig@wikimedia.de --- Ok, I had a chance to talk to the rest of the Wikidata team. The fact that "messages" was not an array was a bug. It should be an array (with numeric keys, obviously).
The problem is that there are many different places where error messages are build. We started unifying them by introducing Wikibase ApiErrorReporter but this may not have cleaned up all instances. If you find situations where "messages" still is an object (with string keys) please report them.
https://bugzilla.wikimedia.org/show_bug.cgi?id=66619
John F. Lewis johnflewis93@gmail.com changed:
What |Removed |Added ---------------------------------------------------------------------------- Priority|Unprioritized |Normal
https://bugzilla.wikimedia.org/show_bug.cgi?id=66619
--- Comment #20 from Gerrit Notification Bot gerritadmin@wikimedia.org --- Change 152036 had a related patch set uploaded by XZise: API change: error message is a list not a dict
https://gerrit.wikimedia.org/r/152036
https://bugzilla.wikimedia.org/show_bug.cgi?id=66619
--- Comment #21 from Gerrit Notification Bot gerritadmin@wikimedia.org --- Change 139794 merged by jenkins-bot: (bug 66619) API change: error message is a list not a dict
https://gerrit.wikimedia.org/r/139794
https://bugzilla.wikimedia.org/show_bug.cgi?id=66619
xqt info@gno.de changed:
What |Removed |Added ---------------------------------------------------------------------------- Status|PATCH_TO_REVIEW |RESOLVED Resolution|--- |FIXED
https://bugzilla.wikimedia.org/show_bug.cgi?id=66619
--- Comment #22 from Gerrit Notification Bot gerritadmin@wikimedia.org --- Change 152036 abandoned by Xqt: API change: error message is a list not a dict
Reason: https://gerrit.wikimedia.org/r/#/c/139794/ is already merged.
https://gerrit.wikimedia.org/r/152036
pywikipedia-bugs@lists.wikimedia.org