Daviskr created this task. Daviskr added a subscriber: Daviskr. Daviskr added a project: pywikibot-core.
TASK DESCRIPTION Page generators can be iterated over multiple times when expected to be empty and stop after the first time. The following iterations can contain erratic results.
For example: It is possible to iterate over a generator with a for loop and then iterate over that same generator with a later for loop.
If the generator is called multiple times without `step` specified, the generator repeats itself if iterated over again. If `step` is specified, the generator shifts results erratically when called multiple times.
Below is REPL output from the `test:test` site using `PrefixingPageGenerator()`.
```
ppg = PrefixingPageGenerator('a', step=2, total=3) for _ in range(3):
... print list(ppg) ... [Page(A), Page(AAA), Page(AF Test)] [Page(AF Test), Page(AKlapper2), Page(API output)] [Page(AKlapper2), Page(API output), Page(API page move test)] ```
Expected output would be `[]` after the first one.
TASK DETAIL https://phabricator.wikimedia.org/T85204
REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign <username>.
EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: Daviskr Cc: Aklapper, Daviskr, jayvdb, pywikipedia-bugs
valhallasw added a subscriber: valhallasw. valhallasw added a comment.
It has to do with the hairy business of iterators versus generators in python:
## current code
class IteratorExample(object): def __iter__(self): yield 1 i = IteratorExample() print list(i) print list(i) # this is confusing
## 'typical' use of __iter__
class IteratorExample(object): def __init__(self): self.returnvalues = [1] def __iter__(self): return self def next(self): try: return self.returnvalues.pop() except: raise StopIteration()
i = IteratorExample() print list(i) print list(i) # this now makes sense
## how to go from (1) to (2) with minimal changes?
class IteratorExample(object): def __init__(self): self.gen = self._gen() def _gen(self): yield 1 def __iter__(self): return self def next(self): return self.gen.next()
i = IteratorExample() print list(i) print list(i) # this now makes sense, but the boilerplate code is... meh.
TASK DETAIL https://phabricator.wikimedia.org/T85204
REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign <username>.
EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: valhallasw Cc: Aklapper, Daviskr, valhallasw, jayvdb, pywikipedia-bugs
valhallasw added a comment.
Reading a bit more about it, I think the issue is that we save state in `data.api.QueryGenerator.__iter__`, instead of in `QueryGenerator` //itself//.
https://github.com/wikimedia/pywikibot-core/blob/fb5af89cbdb465e207e9f21a60c...
TASK DETAIL https://phabricator.wikimedia.org/T85204
REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign <username>.
EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: valhallasw Cc: Aklapper, Daviskr, valhallasw, jayvdb, pywikipedia-bugs
jayvdb added a subscriber: jayvdb. jayvdb added a comment.
This doesnt quite solve the bug, but it almost does by making __iter__ stateless . that should mean the iterator's second run is identical to the first, unless data has changed.
https://gerrit.wikimedia.org/r/#/c/176013/
TASK DETAIL https://phabricator.wikimedia.org/T85204
REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign <username>.
EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: jayvdb Cc: Aklapper, Daviskr, valhallasw, jayvdb, pywikipedia-bugs
jayvdb edited projects, added Pywikibot-pagegenerators; removed pywikibot-core. jayvdb set Security to none.
TASK DETAIL https://phabricator.wikimedia.org/T85204
REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign <username>.
EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: jayvdb Cc: Aklapper, Daviskr, valhallasw, jayvdb, pywikipedia-bugs
pywikipedia-bugs@lists.wikimedia.org