jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/692617 )
Change subject: [IMPR] re-use throttle PID (Step 1) ......................................................................
[IMPR] re-use throttle PID (Step 1)
After dropping a throttle PID it can be reused for a next process. PID (Process IDentifier) is to be used for logging filenames to prevent WindowsError/PermissionError if runnng more than one script with the same name.
Bug: T56685 Change-Id: I7588e81c15c77b63fe919c653a33430f71f5c538 --- M pywikibot/throttle.py 1 file changed, 10 insertions(+), 5 deletions(-)
Approvals: Xqt: Looks good to me, approved jenkins-bot: Verified
diff --git a/pywikibot/throttle.py b/pywikibot/throttle.py index 33dff52..edb6625 100644 --- a/pywikibot/throttle.py +++ b/pywikibot/throttle.py @@ -4,14 +4,17 @@ # # Distributed under the terms of the MIT license. # +import itertools import math import threading import time + from collections import namedtuple from contextlib import suppress from typing import Optional, Union
import pywikibot + from pywikibot import config
@@ -117,11 +120,12 @@ _logger) with self.lock: processes = [] - my_pid = pid or 1 # start at 1 if global pid not yet set + used_pids = set() count = 1
now = time.time() for proc in self._read_file(raise_exc=True): + used_pids.add(proc.pid) if now - proc.time > self.releasepid: continue # process has expired, drop from file if now - proc.time <= self.dropdelay \ @@ -130,16 +134,17 @@ count += 1 if proc.site != self.mysite or proc.pid != pid: processes.append(proc) - if not pid and proc.pid >= my_pid: - my_pid = proc.pid + 1 # next unused process id
+ free_pid = (i for i in itertools.count(start=1) + if i not in used_pids) if not pid: - pid = my_pid + pid = next(free_pid) + self.checktime = time.time() processes.append( ProcEntry(pid=pid, time=self.checktime, site=mysite))
- self._write_file(processes) + self._write_file(sorted(processes, key=lambda x: x.pid))
self.process_multiplicity = count pywikibot.log('Found {} {} processes running, including this one.'