Unicodesnowman created this task. Unicodesnowman added a subscriber: Unicodesnowman. Unicodesnowman added a project: pywikibot-core. Unicodesnowman changed Security from none to none.
TASK DESCRIPTION Category:Female_Wikipedians has a subcategory, Category:Lesbian_Wikipedians. Category.members() lists it, however Category.members(namespaces=14) doesn't, nor does Category.subcategories().
``` $ python pwb.py shell
import pywikibot site = pywikibot.Site('en', 'wikipedia') cat = pywikibot.Category('Category:Female_Wikipedians') cat.categoryinfo
{u'files': 0, u'subcats': 1, u'pages': 2595, u'size': 2596}
list(cat.subcategories())
[]
list(cat.members(namespaces=14))
[] ```
jayvcb isn't missing Category:Lesbian_Wikipedians, however he's missing Category:Gay_Wikipedians from Category:Male_Wikipedians.
I've been unable to find other pages that demonstrate this bug.
TASK DETAIL https://phabricator.wikimedia.org/T84860
REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign <username>.
EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: Unicodesnowman Cc: Aklapper, Unicodesnowman, jayvdb, pywikipedia-bugs
Unicodesnowman changed the title from "Category.subcategories() or Category.members(namespaces=14) failing for certain articles" to "Category.subcategories() or Category.members(namespaces=14) missing certain subcategories".
TASK DETAIL https://phabricator.wikimedia.org/T84860
REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign <username>.
EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: Unicodesnowman Cc: Aklapper, Unicodesnowman, jayvdb, pywikipedia-bugs
XZise added a subscriber: XZise. XZise added a comment.
Okay I'm not sure if that is specific to categorymembers, but when you request all members of a category which are categories it still tries to return non-category pages so that the category is returned in the last batch:
https://en.wikipedia.org/w/api.php?action=query&list=categorymembers&...
Maybe our QueryGenerator stops requesting pages because it got an empty result set:
https://en.wikipedia.org/w/api.php?action=query&list=categorymembers&...
TASK DETAIL https://phabricator.wikimedia.org/T84860
REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign <username>.
EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: XZise Cc: Aklapper, Unicodesnowman, XZise, jayvdb, pywikipedia-bugs
XZise edited the task description.
TASK DETAIL https://phabricator.wikimedia.org/T84860
REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign <username>.
EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: XZise Cc: Aklapper, Unicodesnowman, XZise, jayvdb, pywikipedia-bugs
XZise added a comment.
Okay I think I'm closer to it now: The 'query' entry is missing because no pages are returned but on the first request the userinfo was returned which generated a 'query' entry. For some reason it's not on the second so that entry is missing and it stops iteration:
- `{'batchcomplete': '', 'continue': {'gcmcontinue': 'page|434c45414e555042414245|25600657', 'continue': 'gcmcontinue||userinfo'}, 'query': {'userinfo': {'id': 7818389, 'name': 'XZise'}}}` - `{'batchcomplete': '', 'continue': {'gcmcontinue': 'page|494d4d4143554c4154454845415254|5038287', 'continue': 'gcmcontinue||userinfo'}}`
So the problem is that QueryGenerator must continue even if 'query' is missing. Or the MediaWiki itself has a bug and it should return a query entry even if it's empty.
TASK DETAIL https://phabricator.wikimedia.org/T84860
REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign <username>.
EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: XZise Cc: Aklapper, Unicodesnowman, XZise, jayvdb, pywikipedia-bugs
XZise added a comment.
Okay I tested it on a 1.21 wiki and got results immediately:
- http://wiki.kerbalspaceprogram.com/w/api.php?action=query&generator=cate... - http://wiki.kerbalspaceprogram.com/w/api.php?action=query&generator=cate...
I query only one page and on the English Wikipedia it would be either a template or a category so I'd only get a result in one request but not both. And both use the simplified continuation so that is not the cause of the difference.
TASK DETAIL https://phabricator.wikimedia.org/T84860
REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign <username>.
EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: XZise Cc: Aklapper, Unicodesnowman, XZise, jayvdb, pywikipedia-bugs
gerritbot added a project: Patch-For-Review. gerritbot added a comment.
Change 180779 had a related patch set uploaded (by XZise): [FIX] QueryGenerator: Allow missing 'query' entry
https://gerrit.wikimedia.org/r/180779
https://phabricator.wikimedia.org/tag/patch-for-review/
TASK DETAIL https://phabricator.wikimedia.org/T84860
REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign <username>.
EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: gerritbot Cc: Aklapper, Unicodesnowman, XZise, jayvdb, pywikipedia-bugs
gerritbot added a comment.
Change 180779 merged by jenkins-bot: [FIX] QueryGenerator: Allow missing 'query' entry
https://gerrit.wikimedia.org/r/180779
TASK DETAIL https://phabricator.wikimedia.org/T84860
REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign <username>.
EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: gerritbot Cc: Aklapper, Unicodesnowman, XZise, jayvdb, pywikipedia-bugs
XZise closed this task as "Resolved". XZise claimed this task. XZise removed a project: Patch-For-Review.
TASK DETAIL https://phabricator.wikimedia.org/T84860
REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign <username>.
EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: XZise Cc: Aklapper, Unicodesnowman, XZise, jayvdb, pywikipedia-bugs
pywikipedia-bugs@lists.wikimedia.org