October 2020 - Cloud - lists.wikimedia.org

Venv
by maxime delzenne 11 Oct '20

11 Oct '20

Hello, I am still a bit stuck with my worker. It worked briefly last week, but I suspect that my environment was not consistent. I suppose the source of my problems is my bad understanding of the virtual environment (see also my email last week about python 3.5 and 3.7), and of the "root" path. Today, I cleaned everything and started from scratch (for the 10th time :/). I made some observations that are not clear to me. As a recall, my website uses django and has a celery/redis worker. *Try 1: following* https://wikitech.wikimedia.org/wiki/Help:Toolforge/Web/Python So I install the venv using "webservice --backend=kubernetes python3.7 shell", which means using the interactive pod. When I am in the interactive pod, with venv activated, kubectl is not found (bash: kubectl: command not found). The packages are however found, for instance "celery" in the shell returns no error. It is also possible to migrate the app. There is a folder venv/lib/python3.7... There is a celery and a django folder inside. Then if I "exit out of webservice shell" and re-activate the venv. Kubectl is found, but celery sends me: Traceback (most recent call last): File "/data/project/cycling-init-bot/www/python/venv/bin/celery", line 5, in <module> from celery.__main__ import main ImportError: No module named 'celery' If I go to www/python/src (or don't actually) and try to migrate, I get: Traceback (most recent call last): File "manage.py", line 10, in main from django.core.management import execute_from_command_line ImportError: No module named 'django' The "webservice --backend=kubernetes python3.7 start", works. So I don't quite understand what are the discrepancies between interactive and normal pod. We install things in one context but then they are not found in the other one... Shall I do something with the PYTHONPATH maybe? *Try 2: without interactive* Now I don't use interactive at all. When typing "python3 -m venv $HOME/www/python/venv". It creates a python3.5 venv (lib/python3.5/). In this case, celery command and migrations works. However by starting the webservice "webservice --backend=kubernetes python3.5 start", I get in uwsgi.logs *** Operational MODE: preforking *** mounting /data/project/cycling-init-bot/www/python/src/app.py on /cycling-init-bot Traceback (most recent call last): File "/data/project/cycling-init-bot/www/python/src/app.py", line 5, in <module> from django.core.wsgi import get_wsgi_application ModuleNotFoundError: No module named 'django' For try 1 and try 2, the worker didn't start (python image version was adapted/variated, it changed this time nothing). As kubectl logs is empty and the describe deployment gives no information, it is quite difficult to describe what happens. I noticed that https://github.com/Wikidata/editgroups uses a separate venv at the root of the project for celery. Psemdel

1 0

October 2020 Technical Community Newsletter
by Sarah R 08 Oct '20

08 Oct '20

Hi Everyone, We’re happy to announce the October 2020 edition of the Technical Community Newsletter <https://www.mediawiki.org/wiki/Technical_Community_Newsletter/2020/October> is now available. The newsletter is compiled by the Wikimedia Developer Advocacy Team. It aims to share highlights, news, and information of interest from and about the Wikimedia technical community. Check it out, and learn about what technical contributors have been up to this past quarter, upcoming conferences & calls for papers, and how to get involved. The Wikimedia Technical Community is large and diverse, and we know we can't capture everything perfectly. We welcome your ideas for future newsletters. Let us know what you would like to see or highlights you would like us to include. Subscribe to the Technical Community Newsletter <https://www.mediawiki.org/wiki/Newsletter:Technical_Community_Newsletter>, if you'd like to keep up with essential updates and information Kindly, Sarah R. Rodlund Senior Technical Writer, Developer Advocacy <https://meta.wikimedia.org/wiki/Developer_Advocacy> srodlund(a)wikimedia.org

1 0

Python3.7
by maxime delzenne 07 Oct '20

07 Oct '20

Hello, First of all, thank you to Chico Venancio for the explanation about "kubectl describe". However it still did not start... It seems to me that there is a problem with python3.7. When I install the venv for python3.7 (web and base, same problem) and I do a command "hello Kubernetes!", it crashes (deployment file here https://paste.toolforge.org/view/89fc0b13). If I replace 3.7 through 3.5, then It works and returns the hello Kubernetes. After bringing back everything to Python 3.5, the Celery worker is running (at last!). Psemdel

3 7

Pywikibot: solved
by maxime delzenne 07 Oct '20

07 Oct '20

Hello, third message (very sorry) Problem, with Pywikibot were solved on the chat in the evening. It turned out, that I was not understanding the functioning of BotPasswords (but was able to make work my bot for 3 years...). Psemdel

1 0

Pywikibot and login
by maxime delzenne 07 Oct '20

07 Oct '20

Hello, second message Today I am making the setting of Pywikibot and I get a small issue by login. I followed the steps from https://wikitech.wikimedia.org/wiki/Help:Toolforge/Pywikibot#Submitting,_ma…... <https://wikitech.wikimedia.org/wiki/Help:Toolforge/Pywikibot#Submitting,_ma…> to create the user_config.py and user_password.py, this one should work as I used it previously on other platforms. In my Django app I have some import pywikibot... However when I try to make pwb.py login, it asks my password instead to read user_password.py (nevertheless it created the throttle file). When Django tries to run a script, it wants the password to be typed which is impossible, so it returns an error. It must be a small detail but I don't know how to solve that right now. Psemdel

1 0

[Cloud-announce] cloud-vps vulnerability discovered -- please check project membership
by Andrew Bogott 06 Oct '20

06 Oct '20

Earlier today a vulnerability was uncovered that might have permitted unauthorized users to manipulate project membership or create or delete VMs within cloud-vps. That issue has since been resolved, and there is currently no evidence that anyone exploited it. Nevertheless, out of an abundance of caution: if you are a project admin, please review the members and projectadmins of your project. If you see any unknown or untrusted users or unexpected instances, remove them and notify me directly about what you found and what you've done. Thank you! -Andrew + the WMCS team _______________________________________________ Wikimedia Cloud Services announce mailing list Cloud-announce(a)lists.wikimedia.org (formerly labs-announce(a)lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/cloud-announce

1 0

unscheduled keystone maintenance
by Arturo Borrero Gonzalez 06 Oct '20

06 Oct '20

Hi there, we need to perform some unscheduled keystone maintenance right now. Authentication to some cloud services, in particular Horizon, might be interrupted during this maintenance period. We expect such maintenance to don't last more than 1h. regards. -- Arturo Borrero Gonzalez SRE / Wikimedia Cloud Services Wikimedia Foundation

1 0

Deployment issue
by maxime delzenne 05 Oct '20

05 Oct '20

Hello, thank you for the explanations about the deployment file. I imitated the file from https://wikitech.wikimedia.org/wiki/Help:Toolforge/Kubernetes#Kubernetes_co… here is my deployment file: https://paste.toolforge.org/view/f44f8065 But somehow, I always get the following error (I do "kubectl describe cycling-init-bot.worker" to see it): error: the server doesn't have a resource type "cycling-init-bot" How do I know which resources the server has? Psemdel

2 1

Procfile
by maxime delzenne 02 Oct '20

02 Oct '20

Hello everyone, I have an easy question Today. In which file shall I write the instructions (bash) that are performed during the start of the webserver? In my case, to start celery. I mean like the "Procfile" for Heroku. I cannot find that in the help. Thank you. With best regards Psemdel

2 1

2024

2023

2022

2021

2020

2019

2018

2017

Cloud October 2020