jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/608947 )

Change subject: [IMPR] Wait for _putthread is done in BaseBot.exit() ......................................................................

[IMPR] Wait for _putthread is done in BaseBot.exit()

Re-implement waiting for for _putthread is done in BaseBot.exit() but use stopme() function instead of _flush.

The reason for the revert of 87cfda61cd98 was that async put was blocked: With _flush a tuple containing None was added to the page_put_queue and the async_manager function leaved the loop and no longer processed any queue items which were put later by async_request. This means all following wheren't pocessed anymore i.e. no page was written to site. in result the page_put_queue increases up to its max_queue_size and blocks after that including KeyboardInterrupt wasn't funtional.

This problem only occures if the bot itself runs in a loop.

This partly reverts commit 4291f36321606 but re-implements most of the 87cfda61cd98 patch except that _flush was replaced by stopme,

Change-Id: Ic26928eafdf19844c950f40993073a4e6b9dd687 --- M pywikibot/bot.py 1 file changed, 15 insertions(+), 7 deletions(-)

Approvals: Xqt: Looks good to me, approved jenkins-bot: Verified

diff --git a/pywikibot/bot.py b/pywikibot/bot.py index 9a5b42e..2d22767 100644 --- a/pywikibot/bot.py +++ b/pywikibot/bot.py @@ -1290,28 +1290,36 @@ May be overridden by subclasses. """ self.teardown() + if hasattr(self, '_start_ts'): + read_delta = pywikibot.Timestamp.now() - self._start_ts + read_seconds = int(read_delta.total_seconds()) + + # wait until pending threads finished but don't close the queue + pywikibot.stopme() + pywikibot.output('\n{} pages read' '\n{} pages written' '\n{} pages skipped' .format(self._treat_counter, self._save_counter, self._skip_counter)) + if hasattr(self, '_start_ts'): - delta = (pywikibot.Timestamp.now() - self._start_ts) - seconds = int(delta.total_seconds()) - if delta.days: + write_delta = pywikibot.Timestamp.now() - self._start_ts + write_seconds = int(write_delta.total_seconds()) + if write_delta.days: pywikibot.output( 'Execution time: {d.days} days, {d.seconds} seconds' - .format(d=delta)) + .format(d=write_delta)) else: pywikibot.output('Execution time: {} seconds' - .format(delta.seconds)) + .format(write_delta.seconds)) if self._treat_counter: pywikibot.output('Read operation time: {:.1f} seconds' - .format(seconds / self._treat_counter)) + .format(read_seconds / self._treat_counter)) if self._save_counter: pywikibot.output('Write operation time: {:.1f} seconds' - .format(seconds / self._save_counter)) + .format(write_seconds / self._save_counter))

# exc_info contains exception from self.run() while terminating exc_info = sys.exc_info()