Cloud February 2020

cloud@lists.wikimedia.org

12 participants
15 discussions

Hi
by Michael A Peoni 29 Feb '20

29 Feb '20

Should be Whitelisted mapco2(a)outlook.com still getting spam wiki please advise. Michael P. Mapco2 Get Outlook for iOS<https://aka.ms/o0ukef>

1 0

[Cloud-announce] Planned NFS maintenance 2020-02-26@1800 UTC
by Brooke Storm 27 Feb '20

27 Feb '20

On Wed 26-Feb, we are making a large change to how NFS is mounted in Cloud Services https://gerrit.wikimedia.org/r/c/operations/puppet/+/571821. This will impact any Cloud VPS projects that mount NFS for home directories, project directories and scratch, including Toolforge. During this change, NFS will become unresponsive for a short time. Some NFS clients will recover on their own with little impact. Where needed, WMCS will reboot or remount NFS clients. This change will improve future NFS management and hopefully reduce future disruptions from maintenance. _______________________________________________ Wikimedia Cloud Services announce mailing list Cloud-announce(a)lists.wikimedia.org (formerly labs-announce(a)lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/cloud-announce

3 5

[Cloud-announce] [Toolforge] 2020 Kubernetes cluster automatic migration phase beginning
by Bryan Davis 25 Feb '20

25 Feb '20

Following a beta testing period [0] and a general use self-migration period [1], the Toolforge administration team is ready to begin the final phase of automatic migration of tools currently running on the legacy Kubernetes cluster to the 2020 Kubernetes cluster. The migration process will involve Toolforge administrators running `webservice migrate` for each tool in the same way that self-migration happens [2]. A small number of tools are using the legacy Kubernetes cluster outside of the `webservice` system. These tools will be moved using a more manual process after move all webservices. We are currently planning on doing these migrations in several batches so that we can monitor the load and capacity of the 2020 Kubernetes cluster as we move ~640 more tools over from the legacy cluster. Once the tools have all been moved to the 2020 cluster, we will continue with additional clean up and default configuration changes which will allow us to fully decommission the legacy cluster. We will also be updating various documentation on Wikitech during this final phase. We hope to complete this entire process by 2020-03-06 at the latest. [0]: https://lists.wikimedia.org/pipermail/cloud-announce/2020-January/000247.ht… [1]: https://lists.wikimedia.org/pipermail/cloud-announce/2020-January/000252.ht… [2]: https://wikitech.wikimedia.org/wiki/News/2020_Kubernetes_cluster_migration#… Bryan (on behalf of the Toolforge admins and the Cloud Services team) -- Bryan Davis Technical Engagement Wikimedia Foundation Principal Software Engineer Boise, ID USA [[m:User:BDavis_(WMF)]] irc: bd808 _______________________________________________ Wikimedia Cloud Services announce mailing list Cloud-announce(a)lists.wikimedia.org (formerly labs-announce(a)lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/cloud-announce

3 6

Workflow for kubernetes?
by Roy Smith 22 Feb '20

22 Feb '20

The bastion hosts don't have python3.7, and the k8s hosts don't have ssh, so "git pull" and "git push" don't work. Which means I need to keep flipping back and forth between the two during development. Is there some way to get ssh working on the k8s hosts, so I can do everything in one place? The other annoyance is that when you connect with "webservice --backend=kubernetes python3.7 shell", the connection eats control-p's Is there some way to turn that off, or at least reconfigure it to use a different command character?

2 4

[Cloud-announce] VM reboots on Thursday
by Andrew Bogott 21 Feb '20

21 Feb '20

The barrage of hardware failures continues! Next week the eqiad staff will be repairing cloudvirt1014; to prepare I'll be draining it on this Thursday (2020-02-20), beginning around 15:00 UTC. Affected instances will be down for a few minutes and then rebooted. Toolforge users should be largely unaffected by this maintenance. Here is the list of affected VMs: traffic-cache-atsupload-buster canary1014-01 util-abogott-buster filippo-log-buster01 staging opusmt mw01 grantreview-04 cloud-puppetmaster-03 commtech-wikiwho toolsbeta-test-k8s-haproxy-2 debmonitor-pm toolsbeta-test-k8s-control-1 toolsbeta-test-k8s-etcd-2 toolsbeta-test-k8s-etcd-1 jmm-debm-01 xtools-dev05 ores-web-05 roebling ores-web-04 cloudinfra-db02 Krypton discovery-production-02 maps-tiles1 wikitextexp-base-1002 accounts-appserver4 tofawiki02 packagist-mirror1 deployment-elastic06 deployment-changeprop deployment-restbase02 deployment-imagescaler01 deployment-kafka-jumbo-1 deployment-memc07 deployment-eventlog05 deployment-cpjobqueue deployment-mediawiki-07 deployment-chromium01 deployment-cache-text05 whgi wikilabels gitservices wikilabels-02 af-puppetdb02 missing-sections ores-lb-03 matrix-synapse-01 captcha-tf-43 k4-2 _______________________________________________ Wikimedia Cloud Services announce mailing list Cloud-announce(a)lists.wikimedia.org (formerly labs-announce(a)lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/cloud-announce

1 1

[Cloud-announce] Brief interruption in grid engine cronjobs tomorrow
by Andrew Bogott 20 Feb '20

20 Feb '20

Tomorrow I need to relocate 'tools-sgecron-01,' the VM that is in charge of starting cron jobs on the grid. The host will be down for 5-10 minutes, during which time no cronjobs will start. I'm going to make the move around 16:00 UTC, although I'll wait until a few minutes after so that on-the-hour jobs still happen. -Andrew _______________________________________________ Wikimedia Cloud Services announce mailing list Cloud-announce(a)lists.wikimedia.org (formerly labs-announce(a)lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/cloud-announce

1 0

wm-bot instance moved + static movable VIP address question
by Petr Bena 20 Feb '20

20 Feb '20

Hello, FYI, I know most of you may have no idea what wm-bot is, but I don't know any better channel to use for this kind of announcement so, I had to move wm-bot to a new instance as old one was using deprecated OS, which means, that anyone who is using its "netcat" feature - relaying TCP messages directly to IRC channels https://meta.wikimedia.org/wiki/Wm-bot#Relay_of_messages_from_scripts_and_o… will have to change the IP / hostname. To avoid this kind of unnecessary work in the future: is it possible to create some VIP in cloud environment so that I can simply move this address to a new instance when I perform this kind of migration? Also, as many users are using the hostname instead of IP, what's the best way to move the hostname? Would it be possible to implement some instance renaming feature? Thanks

3 2

[Cloud-announce] Keystone/Horizon sessions will be reset on Tuesday
by Andrew Bogott 17 Feb '20

17 Feb '20

On Tuesday morning I'm going to switch the OpenStack Keystone token engine from UUID tokens to fernet tokens[0]. The changeover[1] will be abrupt and cause all existing sessions to reset (e.g. if you're using Horizon you'll have to log back in, and if you're in the middle of creating a VM that creation will probably fail.) Should you encounter this interruption, just try again in a few minutes and everything should be fine. The new tokens will generally behave the same from a user standpoint, but will allow us to simplify and modernize things on our end a bit. The switch will happen at 15:00 UTC on this Tuesday, 2020-02-18. That's 7AM, Pacific Coast time. [0] https://docs.openstack.org/keystone/pike/admin/identity-fernet-token-faq.ht… [1] https://phabricator.wikimedia.org/T243418 _______________________________________________ Wikimedia Cloud Services announce mailing list Cloud-announce(a)lists.wikimedia.org (formerly labs-announce(a)lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/cloud-announce

1 0

[Cloud-announce] VM downtimes later today
by Andrew Bogott 14 Feb '20

14 Feb '20

In order to repair some ailing hardware, I'm going to migrate several cloud-vps instances later today. Each will be down for a few minutes (or longer, depending on disk size) and then rebooted. Toolforge users will be unaffected by this change. Affected VMs will be: vanadium cloudstore-client-2 cloudstore-client-1 cn-staging-2 wikispore-test wm-bot traffic-cache-atstext-buster etherpad traffic-cache-atsupload jbond-buster commons-corruption-checker-main codesearch6 canary1022-01 ores-web-06 cloud-cumin-01 petscan4 thanos-prom01 thanos-be01 extdist-04 _______________________________________________ Wikimedia Cloud Services announce mailing list Cloud-announce(a)lists.wikimedia.org (formerly labs-announce(a)lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/cloud-announce

1 1

Invitation to apply to Outreachy and Google Summer of Code program with Wikimedia!
by Srishti Sethi 11 Feb '20

11 Feb '20

*(feel free to forward the message as is to your friends, family members & colleagues)* Hello folks, We would like to invite you to apply to the Outreachy and Google Summer of Code program with the Wikimedia Foundation (a non-profit organization behind Wikipedia)! *About the Outreachy program* Wikimedia will be mentoring ~7 projects in the Outreachy program in the May to August 2020 round. The initial applications are due February 25th at 4 pm UTC. Apply today: https://www.outreachy.org/apply/ Outreachy offers three-month internships to work remotely in Free and Open Source Software (FOSS) projects with experienced mentors. The internships may include programming, user experience, documentation, illustration and graphic design, or data science. Outreachy internships run twice a year – from May to August and December to March. Interns are paid a stipend of $5,500 USD for the three months of work. They also have a $500 USD stipend to travel to conferences and events. Interns often find employment after their internship with Outreachy sponsors or in jobs that use the skills they learned during their internship. Outreachy is open to both students and non-students. Outreachy expressly invites the following people to apply: * Women (both cis and trans), trans men, and genderqueer people. * Anyone who faces under-representation, systematic bias, or discrimination in the technology industry in their country or residence is invited to apply. * Residents and nationals of the United States of any gender who are Black/African American, Hispanic/Latin@, Native American/American Indian, Alaska Native, Native Hawaiian, or Pacific Islander. *About the Google Summer of Code program* Wikimedia is planning to mentor 8-10 projects in 2020’s Google Summer of Code (GSoC) program. Beginning February 20th, pending Wikimedia’s acceptance as a mentoring organization, applicants can begin discussing ideas with the mentors! The student application will be due on March 31st 18:00 UTC: https://summerofcode.withgoogle.com/ Google Summer of Code now in its 16th year, is Google's summer program for university students who want to get involved in open source software. Over 15,960 students from 109 countries have already participated. Google Summer of Code is a unique program that pairs students with mentors who introduce them to the open-source community and provide guidance while they work on real-world open-source projects during their summer break from university. Projects cover a wide range of fields including: Cloud, Operating Systems, Graphics, Medicine, Programming Languages, Robotics, Science, Security and many more. It's a highly competitive program (and this year is expected to be even bigger than last year), so don't wait until the last minute to prepare! *About the Wikimedia Foundation* The Wikimedia Foundation <https://wikimediafoundation.org/> is the nonprofit organization that hosts and operates Wikipedia and the other Wikimedia free knowledge projects <https://wikimediafoundation.org/our-work/wikimedia-projects/>. Our vision is a world in which every single human can freely share in the sum of all knowledge. We believe that everyone has the potential to contribute something to our shared knowledge, and that everyone should be able to access that knowledge, free of interference. We host the Wikimedia projects, build software experiences for reading, contributing, and sharing Wikimedia content, support the volunteer communities and partners who make Wikimedia possible, and advocate for policies that enable Wikimedia and free knowledge to thrive. *Resources* * Browse through the participants’ guides, to learn more about the application process steps: https://www.mediawiki.org/wiki/Outreachy/Participants https://www.mediawiki.org/wiki/Google_Summer_of_Code/Participants * All the projects will be showcased here: https://www.outreachy.org/communities/cfp/wikimedia/ https://www.mediawiki.org/wiki/Google_Summer_of_Code/2020 We hope you will help us spread the word about Wikimedia’s participation in these programs: https://twitter.com/mediawiki/status/1224790357361098752 (by retweeting the post in the link or by sharing this email) Looking forward to your participation! Cheers, Srishti & Pavithra (Wikimedia organization administrators for GSoC & Outreachy) *Srishti Sethi* Developer Advocate Wikimedia Foundation <https://wikimediafoundation.org/>

1 0

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

Cloud February 2020