jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/1221451?usp=email )

Change subject: IMPR: use RLock instead of Queue to signal async_manager activity ......................................................................

IMPR: use RLock instead of Queue to signal async_manager activity

Use an RLock instead of an auxiliary Queue to signal whether async_manager is currently processing a request.

With Python 3.14, threading.RLock gained a public locked() method. This method is backported for older Python versions and available in Pywikibot since 10.2 via the backports module (and since 6.2 via the tools module).

The lock is held while a request is being executed and released afterwards. _flush() can reliably determine whether the worker thread is still busy by checking _page_put_queue_busy.locked() in addition to page_put_queue.empty().

Changes:

- Replace page_put_queue_busy with an internal RLock (_page_put_queue_busy) - Use the lock to signal active request processing - Remove the undocumented page_put_queue_busy Queue

This preserves the original synchronization semantics while reducing overhead and avoiding the need for a second Queue.

Bug: T147178 Change-Id: I7f25e34cc807ccf61987431db691858d910dfd21 --- M ROADMAP.rst M pywikibot/__init__.py 2 files changed, 16 insertions(+), 10 deletions(-)

Approvals: jenkins-bot: Verified Xqt: Looks good to me, approved

diff --git a/ROADMAP.rst b/ROADMAP.rst index 0905cf3..4ed43d8 100644 --- a/ROADMAP.rst +++ b/ROADMAP.rst @@ -3,6 +3,7 @@

**Improvements**

+* Use :class:`backports.RLock` instead of Queue to signal async_manager activity (:phab:`T147178`) * Add :meth:`User.is_partial_blocked()<pywikibot.User.is_partial_blocked>` and methods :meth:`APISite.is_partial_blocked()<pywikibot.site._apisite.APISite.is_partial_blocked>` to detect partial blocks (:phab:`T412613`) @@ -28,6 +29,7 @@

**Code cleanups**

+* The undocumented ``page_put_queue_busy`` was removed without deprecation period. * Dysfunctional :meth:`APISite.alllinks() <pywikibot.site._generators.GeneratorsMixin.alllinks>` was removed. (:phab:`T359427`, :phab:`T407708`) diff --git a/pywikibot/__init__.py b/pywikibot/__init__.py index 7dc246a..abdd4a1 100644 --- a/pywikibot/__init__.py +++ b/pywikibot/__init__.py @@ -32,6 +32,7 @@ WbTime, WbUnknown, ) +from pywikibot.backports import RLock from pywikibot.bot import ( Bot, CurrentPageBot, @@ -353,8 +354,8 @@

exit_queue = None if _putthread is not threading.current_thread(): - while _putthread.is_alive() and not (page_put_queue.empty() - and page_put_queue_busy.empty()): + while _putthread.is_alive() and (not page_put_queue.empty() + or _page_put_queue_busy.locked()): try: _putthread.join(1) except KeyboardInterrupt: @@ -393,13 +394,14 @@ while True: if not block and page_put_queue.empty(): break + (request, args, kwargs) = page_put_queue.get(block) - page_put_queue_busy.put(None) - if request is None: - break - request(*args, **kwargs) - page_put_queue.task_done() - page_put_queue_busy.get() + + with _page_put_queue_busy: + if request is None: + break + request(*args, **kwargs) + page_put_queue.task_done()

def async_request(request: Callable, *args: Any, **kwargs: Any) -> None: @@ -408,14 +410,16 @@ # ignore RuntimeError if start() is called more than once with page_put_queue.mutex, suppress(RuntimeError): _putthread.start() + page_put_queue.put((request, args, kwargs))

#: Queue to hold pending requests page_put_queue: Queue = Queue(_config.max_queue_size)

-# queue to signal that async_manager is working on a request. See T147178. -page_put_queue_busy: Queue = Queue(_config.max_queue_size) +# RLock to signal that async_manager is working on a request. See T147178. +_page_put_queue_busy: RLock = RLock() + # set up the background thread _putthread = threading.Thread(target=async_manager, name='Put-Thread', # for debugging purposes