Cloud-announce September 2022

cloud-announce@lists.wikimedia.org

5 participants
6 discussions

Eliminating Debian Stretch in Cloud VPS
by Andrew Bogott 13 Oct '22

13 Oct '22

Debian Stretch's security support ends in mid 2022, and the Foundation's OS policy already discourages use of existing Stretch machines. That means that it's time for all project admins to start rebuilding your VMs with Bullseye (or, if you must, Buster.) Any webservices running in Kubernetes created in the last year or two are most likely using Buster images already, so there's no action needed for those. Older kubernetes jobs should be refreshed to use more modern images whenever possible. If you are still using the grid engine for webservices, we strongly encourage you to migrate your jobs to Kubernetes. For other grid uses, watch this space for future announcements about grid engine migration; we don't yet have a solution prepared for that. Details about the what and why for this process can be found here: https://wikitech.wikimedia.org/wiki/News/Stretch_deprecation Here is the deprecation timeline: March 2021: Stretch VM creation disabled in most projects July 6, 2021: Active support of Stretch ends, Stretch moves into LTS <- You are Here -> January 1st, 2022: Stretch VM creation disabled in all projects, deprecation nagging begins in earnest. Stretch alternatives will be available for tool migration in Toolforge May 1, 2022: All active Stretch VMs will be shut down (but not deleted) by WMCS admins. This includes Toolforge grid exec nodes. June 30, 2022: LTS support for Debian Stretch ends, all Stretch VMs will be deleted by WMCS admins

1 1

Breaking change for Toolforge tools that use wikimedia dumps
by Andrew Bogott 26 Sep '22

26 Sep '22

TL;DR: If your tool makes use of Wikimedia Dumps, please restart your tool today to avoid breakage next week. --- As part of a hardware upgrades[0], the mount points hosting the dumps is moving. In order to support this move we've needed to update the base Kubernetes containers to know about the new mount points. For the vast majority of use cases, a simple tool restart will pick up the new container and ensure a smooth transition. For rare non-conforming use cases, here are a few more details. The old mountpoints were: /mnt/nfs/dumps-labstore1006.wikimedia.org /mnt/nfs/dumps-labstore1007.wikimedia.org The new mountpoints are: /mnt/nfs/dumps-clouddumps1001.wikimedia.org /mnt/nfs/dumps-clouddumps1002.wikimedia.org Currently all four mountpoints are present on toolforge nodes, but the older mountpoints will be disabled next week. If your tool contains references to ANY of the above paths, I encourage you to update your code to use /public/dumps/ instead, which is a persistent server-neutral link. Tools using the old mountpoints or accessing dumps via old containers will start breaking a week from today on Monday, October 3. [0] https://phabricator.wikimedia.org/T309346

1 0

Cloud VPS web proxy now enforces an IP-based rate limit
by Taavi Väänänen 08 Sep '22

08 Sep '22

Hi all, We've just configured a simple rate limiter on the Cloud VPS web proxy that limits traffic from a single IP address to 100 requests/second. The limit is designed to be high enough that it shouldn't affect most normal workflows while also being low enough to properly block excess traffic. This is in response to some misbehaving crawlers that were causing instability in the proxy service. We applied a similar rate limit to Toolforge earlier this year. Please be in touch if this is causing issues for your project. Thanks! -- Taavi

1 0

[TOOLHUB] Result of Community Feedback on Toolhub Taxonomy
by Seyram Komla Sapaty 08 Sep '22

08 Sep '22

Hello! Thanks to everyone who took the time to provide their valuable feedback on the Toolhub taxonomy[0]! After a round of community feedback and input, we made the following decisions about which categories and values to implement in the first productionized version of the taxonomy. === Summary of Changes === 1. Exclude the proposed Programming language attribute. 2. Exclude the proposed Platform attribute 3. Revise the Tasks attribute values: 1. Remove "Creating or uploading content" 2. Add "Creating new content" 3. Rename "Generating and recommending content" to "Recommending content" 4. Revise the Content types attribute values: 1. Add additional level of hierarchy to group content types and enable both broad or specific values to be applied. 2. Remove "Files". 3. Split "Maps" and "Geographic Data" Find more details of additional changes on the decision record log page[0] === Next Steps === The team will continue to observe and improve the taxonomy as the community continues to use Toolhub. We will monitor tags and community created lists to determine if certain attributes would be useful or feasible in the future. [0]: https://meta.wikimedia.org/wiki/Toolhub/Data_model#Taxonomy_v2 [1]: https://meta.wikimedia.org/wiki/Toolhub/Decision_record#Taxonomy_v2 -- Seyram Komla Sapaty Developer Advocate Wikimedia Cloud Services

1 0

[Toolforge] Moving git repos from Diffusion to GitLab starting 2022-09-06
by Bryan Davis 07 Sep '22

07 Sep '22

What: Diffusion git hosting moving to GitLab [0][1] When: Tuesday 2022-09-06 between 15:00 - 23:00 UTC Why: Unblocking sunsetting work for Differential/Diffusion [2] What you can do: If you have not logged into https://gitlab.wikimedia.org/ yet to attach your Developer account and look around, now would be a great time! Toolforge tool maintainers can use Striker (<https://toolsadmin.wikimedia.org/>) to create git repositories for each of their tools. Today these git repositories are hosted by <https://phabricator.wikimedia.org/diffusion/>. Starting on 2022-09-06 new repositories will be hosted under <https://gitlab.wikimedia.org/toolforge-repos> instead. We will also be migrating the 474 existing Striker created repositories from Diffusion to GitLab starting on Tuesday 2022-09-06. This process will involve making each existing Diffusion repository read-only, copying it to GitLab, and finally configuring the Diffusion repo to be a read-only mirror of the GitLab repo. We hope that this set of operations will be the least disruptive way to migrate the repositories to the new hosting platform. For tool maintainers with git repos that are migrating who *do not* yet have their Developer account attached at https://gitlab.wikimedia.org/, GitLab will send an email invitation to join the new repo. Because of some quirks of the login process that we are using for our GitLab service, the link in this email needs to be used *after* you have attached your account in order to grant you access [3]. [0]: https://phabricator.wikimedia.org/T296893 [1]: https://phabricator.wikimedia.org/T315706 [2]: https://phabricator.wikimedia.org/T191182 [3]: https://phabricator.wikimedia.org/T313366#8203450 Bryan, on behalf of the Toolforge administrators -- Bryan Davis Technical Engagement Wikimedia Foundation Principal Software Engineer Boise, ID USA [[m:User:BDavis_(WMF)]] irc: bd808

1 2

quarry moving to github
by Vivian Rook 07 Sep '22

07 Sep '22

In order to enable building containers on code changes, Quarry will be moving to github on 2022-09-09 https://github.com/toolforge/quarry The existing gerrit repo will be set to readonly at that point. Thank you -- *Vivian Rook (They/Them)* Site Reliability Engineer Wikimedia Foundation <https://wikimediafoundation.org/>

1 0

2024

2023

2022

2021

2020

2019

2018

2017

Cloud-announce September 2022