I recently migrated most of pywikibot jobs from grid engine to k8s and to
my surprise, it was actually quite easy. I have lots of tasks (been running
bots since 2008). So much that it showed redaction in the total number of
jobs in SGE.
One thing that helped me that I collapsed everything into a bash file, like
hourly.sh and made that a job (it's better to have one pod doing a batch of
work as overhead of creating a pod is bigger than a SGE job)
Also, make sure your bash file is readable by others (see the doc, chmod
ug+x hourlysh), that would have saved me a bit of time.
Thank you for doing this. I use only the new fancy infrastructure from now
on ^^
On Fri, Feb 18, 2022 at 8:28 PM Arturo Borrero Gonzalez <
aborrero(a)wikimedia.org> wrote:
On 2/16/22 17:34, Russell Blau wrote:
Also, it is not possible to load Pywikibot in the
tf-python39 runtime
because a required module (requests,
fromhttps://python-requests.org) is
not available. What is the process for requesting (no pun intended) that
this (or any other resource) be added to the image?
See some documentation here:
https://wikitech.wikimedia.org/wiki/Help:Toolforge/Python#Kubernetes_python…
I just created it, and may need some polishing, but it should work!
We will review pywikibot specific workflows and documents soon.
regards.
--
Arturo Borrero Gonzalez
Site Reliability Engineer
Wikimedia Cloud Services
Wikimedia Foundation
_______________________________________________
Cloud mailing list -- cloud(a)lists.wikimedia.org
List information:
https://lists.wikimedia.org/postorius/lists/cloud.lists.wikimedia.org/
--
Amir (he/him)