Cloud April 2022

cloud@lists.wikimedia.org

21 participants
20 discussions

Removal of revision_actor_temp table
by Amir Sarabadani 20 Apr '22

20 Apr '22

Hello, (If you don't query revision_actor_temp table, feel free to ignore this email) We will remove revision_actor_temp in two weeks, this table was only built for temporary use inside mediawiki. Cloud replicas provide a view that you can query the revision table directly and instead of joining the revision table with revision_actor_temp, you can simply get the value of rev_actor field. We have finally backfilled the value of rev_actor in production and that can be used directly (thus we will remove the view soon). If you query this table in your tools, reports, etc. You need to change it ASAP. Hopefully this should make your queries much simpler. Keep in mind a similar work will happen on revision_comment_temp table in the future. You can follow the work in https://phabricator.wikimedia.org/T275246 Best -- *Amir Sarabadani (he/him)* Staff Database Architect Wikimedia Foundation <https://wikimediafoundation.org/>

1 0

checkuser via a tool?
by Roy Smith 20 Apr '22

20 Apr '22

I've been experimenting with adding some checkuser functionality to spi-tools, and scratching my head over why I'm getting permissiondenied errors. I think I finally figured it out. It looks like my OAuth consumer <https://meta.wikimedia.org/wiki/Special:OAuthListConsumers/view/35ecb3674a0…> doesn't have the checkuser right. So, how do I move forward? Can I add the checkuser right to my consumer key, or do I just throw my consumer key away and create a new one with the checkuser right added? And, just to double-check, I'm assuming that adding the checkuser right to my consumer doesn't actually do anything unless the user who goes through the OAuth flow also has checkuser themselves, right?

2 1

[Cloud-announce] Toolsdb outage this Tuesday, 2022-04-19 at 15:00 UTC
by Andrew Bogott 20 Apr '22

20 Apr '22

Next week (Tuesday, 2022-04-19 15:00 UTC) we will be upgrading the operating system that hosts the shared Toolsdb servers. This upgrade may take an hour or more, during which time the databases will not be available. This outage will be VERY DISRUPTIVE to many toolforge tools, as all database access will fail during the upgrade. Toolforge users may want to disable tools before the outage and/or check in to verify proper recovery after service is restored. There is likely to be a similar (or longer) outage in subsequent weeks as we also need to upgrade the database servers themselves. Toolsdb has grown to an ungainly size and can't be easily handled using standard rolling upgrade procedures; the WMCS team is in ongoing discussions about long-term solutions for this issue. In the meantime you can help us out by engaging in periodic cleanup of your database usage and dumping or dropping data that's no longer of use. -Andrew + the WMCS team _______________________________________________ Cloud-announce mailing list -- cloud-announce(a)lists.wikimedia.org List information: https://lists.wikimedia.org/postorius/lists/cloud-announce.lists.wikimedia.…

1 3

Third Pywikibot workshop on Friday, April 29th, 16:00 UTC
by Srishti Sethi 16 Apr '22

16 Apr '22

Hello everyone, The third workshop on the topic of "Writing Pywikibot scripts" is coming up - it will take place on Friday, April 29th at 16:00 UTC. You can find more details on the workshop and a link to join here: < https://meta.wikimedia.org/wiki/Small_wiki_toolkits/Workshops#How_to_write_…> [1]. This workshop will introduce participants to writing basic scripts via the Pywikibot framework. We will be focusing on examples of scripts that participants have requested to cover in the workshop (e.g., finding and replacing content, archiving discussions, etc.). You can add your ideas to the ongoing discussion in the etherpad doc linked from the workshops page. If you missed attending the previous two workshops, going through the workshop materials beforehand would be beneficial. We look forward to your participation! Best, Srishti On behalf of the SWT Workshops Organization team [1] https://meta.wikimedia.org/wiki/Small_wiki_toolkits/Workshops#How_to_write_… *Srishti Sethi* Senior Developer Advocate Wikimedia Foundation <https://wikimediafoundation.org/>

1 0

No space left when doing tab-autocomplete
by Maciej Jaros 15 Apr '22

15 Apr '22

I get this weird error when trying to autocomplete file and folder names: > tools.dna@tools-sgebastion-07:~$ cat /-bash: cannot create temp file for here-document: No space left on device > -bash: cannot create temp file for here-document: No space left on device Not sure if tab-autocomplete was blocked intentionally or if something is wrong... I think this used to work on the Toolforge. Cheers, Nux.

4 5

[Cloud-announce] Brief service outage for some VMs
by Andrew Bogott 07 Apr '22

07 Apr '22

1 0

Kubernetes-based jobs engine -- how to use Python virtual environments?
by Martin Urbanec 06 Apr '22

06 Apr '22

Hello, I just received a dozen emails about grid engine migration. I tried to migrate my personal tool (tool.martin-urbanec) first. This tool currently generates a Jupyter-notebook based report daily. I do that by calling jupyter nbconvert --to html --execute community_configuration_usage.ipynb from a virtual environment where Jupyter is installed, together with a couple of other Python modules. I managed to create new virtual environment that works from the new Buster bastion, and it works when executed directly from the bastion, but I can't get it to execute via the k8s-based engine: tools.martin-urbanec@tools-sgebastion-10 ~ $ cat ~/public_html/growth-reports/community_configuration_usage/update_report_buster.sh #!/bin/bash source ~/jupyter-buster/bin/activate cd ~/public_html/growth-reports/community_configuration_usage jupyter nbconvert --to html --execute community_configuration_usage.ipynb tools.martin-urbanec@tools-sgebastion-10 ~ $ toolforge-jobs run k8s-community-configuration-update-report --image 'tf-python37-DEPRECATED' --command 'bash ~/public_html/growth-reports/community_configuration_usage/update_report_buster.sh' --mem '1Gi' --wait [toolforge-jobs] ERROR: job 'k8s-community-configuration-update-report' failed: +------------+-----------------------------------------------------------------------------------------+ | Job name: | k8s-community-configuration-update-report | +------------+-----------------------------------------------------------------------------------------+ | Command: | bash ~/public_html/growth-reports/community_configuration_usage/update_report_buster.sh | +------------+-----------------------------------------------------------------------------------------+ | Job type: | normal | +------------+-----------------------------------------------------------------------------------------+ | Container: | tf-python37-DEPRECATED | +------------+-----------------------------------------------------------------------------------------+ | File log: | yes | +------------+-----------------------------------------------------------------------------------------+ | Emails: | none | +------------+-----------------------------------------------------------------------------------------+ | Resources: | mem: 1Gi, cpu: default | +------------+-----------------------------------------------------------------------------------------+ | Status: | Failed | +------------+-----------------------------------------------------------------------------------------+ | Hints: | Last run at 2022-04-02T12:43:04Z. Pod in 'Pending' phase. State | | | 'waiting'. Reason 'ContainerCreating'. | +------------+-----------------------------------------------------------------------------------------+ tools.martin-urbanec@tools-sgebastion-10 ~ $ cat k8s-community-configuration-update-report.err /data/project/martin-urbanec/public_html/growth-reports/community_configuration_usage/update_report_buster.sh: line 6: jupyter: command not found /data/project/martin-urbanec/public_html/growth-reports/community_configuration_usage/update_report_buster.sh: line 6: jupyter: command not found tools.martin-urbanec@tools-sgebastion-10 ~ $ I tried to execute the job using other images (incl. the non-deprecated tf-python39), but the behavior was the same every time. I think this is happening, because I need to create the virtual environment from within the container, rather than via the bastion, because of differences between environment available in the container and at the bastion. I don't know how to do that. With websevices, I would do a webservice python3.9 shell and create the environment from there, but I can't find a comparable solution with toolforge-jobs. How can I fix this issue? A related issue is that previously, the HTML version of the notebook looked like this: [image: image.png] While after I ran the update_report_buster.sh script, it looked like this: [image: image.png] Due to a different HTML structure, the "Show code" / "Hide code" button stopped working :/. Thanks for any help, Martin Urbanec

6 6

Network operations today 2022-04-06
by Arturo Borrero Gonzalez 06 Apr '22

06 Apr '22

Hi there, Today 2022-04-06 we're performing some network maintenance operations on Cloud VPS that could affect all cloud egress/ingress traffic, including Toolforge. The cuts, if noticeable, should last a few minutes at most. Some operations were also conducted yesterday (without this email notice), and some unexpected hiccups occurred. That's why the email today. regards. -- Arturo Borrero Gonzalez Site Reliability Engineer Wikimedia Cloud Services Wikimedia Foundation

1 0

[Toolforge] some updates on toolforge-jobs command line interface
by Arturo Borrero Gonzalez 04 Apr '22

04 Apr '22

Hi there, wanted to share a few small updates for the toolforge-jobs command line interface. The changes are being deployed right now. 1) listing jobs now shows less columns, use --long to show all columns. 2) the `containers` action has been renamed to `images`. A compatibilty period will exists, and you will see a warning if you use `containers`. 3) when listing images, the table header no longer mentions "Docker". These changes should be mostly cosmetic, and no functional or behavioral change is expected. Please report any problems you may find. regards. -- Arturo Borrero Gonzalez Site Reliability Engineer Wikimedia Cloud Services Wikimedia Foundation

1 0

[Cloud-announce] PAWS upgrading to pywikibot 7.0.0 on 2022-03-28
by Vivian Rook 04 Apr '22

04 Apr '22

PAWS upgrading to pywikibot 7.0.0 there are some breaking changes: Support of Python 3.5.0 - 3.5.2 has been dropped (T286867) generate_user_files.py, generate_user_files.py, shell.py and version.py were moved to pywikibot/scripts and must be used with pwb wrapper script With some more in the Code cleanups section of the changelog: https://doc.wikimedia.org/pywikibot/stable/changelog.html -- *Vivian Rook (They/Them)* Site Reliability Engineer Wikimedia Foundation <https://wikimediafoundation.org/> _______________________________________________ Cloud-announce mailing list -- cloud-announce(a)lists.wikimedia.org List information: https://lists.wikimedia.org/postorius/lists/cloud-announce.lists.wikimedia.…

2 1

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

Cloud April 2022