jenkins-bot submitted this change.

View Change

Approvals: Mpaa: Looks good to me, approved jenkins-bot: Verified
[IMPR] Increase minimum workers

doubling the workers to have one free for each family is a good value

Change-Id: I5fa697c8959ed5c70e8a76cf684981ff53dac55f
---
M scripts/maintenance/preload_sites.py
1 file changed, 7 insertions(+), 4 deletions(-)

diff --git a/scripts/maintenance/preload_sites.py b/scripts/maintenance/preload_sites.py
index 3c0afcd..5e0743b 100755
--- a/scripts/maintenance/preload_sites.py
+++ b/scripts/maintenance/preload_sites.py
@@ -73,15 +73,18 @@

.. versionchanged:: 7.3
Default of worker is calculated like for Python 3.8 but preserves
- at least one worker more than families_list elements to ensure a
- worker can be added in :func:`preload_family`.
+ at least one worker for each element in families_list for better
+ performance.
"""
start = datetime.now()
if worker is None:
# Python 3.8 default
worker = min(32, (os.cpu_count() or 1) + 4)
- # allow to add futures in preload_family
- worker = max(len(families) + 1, worker)
+ # to allow adding futures in preload_family the workers must be one
+ # more than families are handled
+ worker = max(len(families) * 2, worker)
+ pywikibot.output('Using {} workers to process {} families'
+ .format(worker, len(families)))
with ThreadPoolExecutor(worker) as executor:
futures = {executor.submit(preload_family, family, executor)
for family in families}

To view, visit change 789587. To unsubscribe, or for help writing mail filters, visit settings.

Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Change-Id: I5fa697c8959ed5c70e8a76cf684981ff53dac55f
Gerrit-Change-Number: 789587
Gerrit-PatchSet: 2
Gerrit-Owner: Xqt <info@gno.de>
Gerrit-Reviewer: D3r1ck01 <xsavitar.wiki@aol.com>
Gerrit-Reviewer: Mpaa <mpaa.wiki@gmail.com>
Gerrit-Reviewer: jenkins-bot
Gerrit-MessageType: merged