May 2022 - Cloud - lists.wikimedia.org

[Cloud-announce] Eliminating Debian Stretch in Cloud VPS
by Andrew Bogott 13 Oct '22

13 Oct '22

Debian Stretch's security support ends in mid 2022, and the Foundation's OS policy already discourages use of existing Stretch machines. That means that it's time for all project admins to start rebuilding your VMs with Bullseye (or, if you must, Buster.) Any webservices running in Kubernetes created in the last year or two are most likely using Buster images already, so there's no action needed for those. Older kubernetes jobs should be refreshed to use more modern images whenever possible. If you are still using the grid engine for webservices, we strongly encourage you to migrate your jobs to Kubernetes. For other grid uses, watch this space for future announcements about grid engine migration; we don't yet have a solution prepared for that. Details about the what and why for this process can be found here: https://wikitech.wikimedia.org/wiki/News/Stretch_deprecation Here is the deprecation timeline: March 2021: Stretch VM creation disabled in most projects July 6, 2021: Active support of Stretch ends, Stretch moves into LTS <- You are Here -> January 1st, 2022: Stretch VM creation disabled in all projects, deprecation nagging begins in earnest. Stretch alternatives will be available for tool migration in Toolforge May 1, 2022: All active Stretch VMs will be shut down (but not deleted) by WMCS admins. This includes Toolforge grid exec nodes. June 30, 2022: LTS support for Debian Stretch ends, all Stretch VMs will be deleted by WMCS admins _______________________________________________ Cloud-announce mailing list -- cloud-announce(a)lists.wikimedia.org List information: https://lists.wikimedia.org/postorius/lists/cloud-announce.lists.wikimedia.…

1 1

Changes in schema of MediaWiki links tables
by Amir Sarabadani 12 Aug '22

12 Aug '22

(If you don’t work with links tables such as templatelinks, pagelinks and so on, feel free to ignore this message) TLDR: The schema of links tables (starting with templatelinks) will change to have numeric id pointing to linktarget table instead of repeating namespace and title. Hello, The current schema and storage of most links tables are: page id (the source), namespace id of the target link and title of the target. For example, if a page with id of 1 uses Template:Foo, the row in the database would be 1, 6, and Foo (Template namespace has id of 6) Repeating the target’s title is not sustainable, for example more than half of Wikimedia Commons database is just three links tables. The sheer size of these tables makes a considerable portion of all queries slower, backups and dumps taking longer and taking much more space than needed due to unnecessary duplication. In Wikimedia Commons, on average a title is duplicated around 100 times for templatelinks and around 20 times for pagelinks. The numbers for other wikis depend on the usage patterns. Moving forward, these tables will be normalized, meaning a typical row will hold mapping of page id to linktarget id instead. Linktarget is a new table deployed in production and contains immutable records of namespace id and string. The major differences between page and linktarget tables are: 1- linktarget values won’t change (unlike page records that change with page move) 2- linktarget values can point to non-existent pages (=red links). The first table being done is templatelinks, then pagelinks, imagelinks and categorylinks will follow. During the migration phase both values will be accessible but we will turn off writing to the old columns once the values are backfilled and switched to be read from the new schema. We will announce any major changes beforehand but this is to let you know these changes are coming. While the normalization of all links tables will take several years to finish, templatelinks will finish in the next few months and is the most pressing one. So if you: - … rely on the schema of these tables in cloud replicas, you will need to change your tools. - … rely on dumps of these tables, you will need to change your scripts. Currently, templatelinks writes to both data schemes for new rows in most wikis. This week we will start backfilling the data with the new schema but it will take months to finish in large wikis. You can keep track of the general long-term work in https://phabricator.wikimedia.org/T300222 and the specific work for templatelinks in https://phabricator.wikimedia.org/T299417. You can also read more on the reasoning in https://phabricator.wikimedia.org/T222224. Thanks -- *Amir Sarabadani (he/him)* Staff Database Architect Wikimedia Foundation <https://wikimediafoundation.org/>

1 2

[Reminder] Your Hackathon feedback is wanted (until May 29th)
by Andre Klapper 01 Jun '22

01 Jun '22

Hi, this is a friendly reminder that we would love to hear from you about your experience at last weekend's Hackathon. Please fill in the survey until Sunday, May 29th and help us improve. See the links below. Thanks a lot in advance! andre -------- Quoted Message -------- From: Haley Lepp Date: Sun, 22 May 2022 11:13:39 -0700 On behalf of the 2022 Wikimedia Hackathon Committee, we would like to thank you for coming to the Wikimedia Hackathon! Please consider giving us feedback on the Hackathon and your suggestions for improvement. There are two ways to give feedback: 1. Fill out the Wikimedia Hackathon Survey <https://wikimedia.qualtrics.com/jfe/form/SV_cGbCKj4xyP0H3wi>. For more information on privacy and data-handling, see the survey privacy statement <https://foundation.wikimedia.org/wiki/Wikimedia_Hackathon_Post-Event_Survey… >. The survey will remain open until May 29, 2022. 2. If you would like to share feedback but do not wish to take the Qualtrics survey, you can leave feedback on the Etherpad <https://etherpad.wikimedia.org/p/Wikimedia_Hackathon_2022_Feedback >. Finally, check out the badges <https://www.mediawiki.org/wiki/Wikimedia_Hackathon_2022/How_to#Joining_a_se… > the committee made. You can put them on your userpages to show your participation. Thank you again for joining us! It was so much fun to meet everyone and hack together. See you at the Wikimania Hackathon in August! Haley, on behalf of the 2022 Wikimedia Hackathon Team -- Andre Klapper (he/him) | Bugwrangler / Developer Advocate https://blogs.gnome.org/aklapper/ _______________________________________________ Wikitech-ambassadors mailing list -- wikitech-ambassadors(a)lists.wikimedia.org To unsubscribe send an email to wikitech-ambassadors-leave(a)lists.wikimedia.org

2 1

Custom puppet class?
by Roy Smith 01 Jun '22

01 Jun '22

I want to set up a custom rsyslog config on my spi-tools VPS instance. I know what I want to end up with, but I'm trying to get it puppetized. It's not really clear what I need to do. Do I really need to set up my own standalone puppetmaster <https://wikitech.wikimedia.org/wiki/Help:Standalone_puppetmaster>? That seems like overkill.

2 1

Connecting from Toolforge to VPS?
by Roy Smith 27 May '22

27 May '22

I want to send data from a process running on toolforge to a VPS host. I tried the obvious: On the VPS host (puppet-test.spi-tools.eqiad1.wikimedia.cloud): > $ nc -4 -l -p 23001 On tools-sgebastion-11: > echo foo | nc -v -4 puppet-test.spi-tools.eqiad1.wikimedia.cloud 23001 > nc: connect to puppet-test.spi-tools.eqiad1.wikimedia.cloud port 23001 (tcp) failed: Connection timed out I'm assuming I need to configure a security group in horizon to allow ingress on that port, is that correct?

2 3

[Cloud-announce] Upgrading kubernetes on PAWS 2022-06-01
by Vivian Rook 25 May '22

25 May '22

On 2022-06-01 we will be upgrading kubernetes (from 1.20 to 1.21) on the PAWS cluster. This should take no action on your part. -- *Vivian Rook (They/Them)* Site Reliability Engineer Wikimedia Foundation <https://wikimediafoundation.org/> _______________________________________________ Cloud-announce mailing list -- cloud-announce(a)lists.wikimedia.org List information: https://lists.wikimedia.org/postorius/lists/cloud-announce.lists.wikimedia.…

1 0

Hackathon thank you and feedback!
by Haley Lepp 23 May '22

23 May '22

Hi Everyone, On behalf of the 2022 Wikimedia Hackathon Committee, we would like to thank you for coming to the Wikimedia Hackathon! Please consider giving us feedback on the Hackathon and your suggestions for improvement. There are two ways to give feedback: 1. Fill out the Wikimedia Hackathon Survey < https://wikimedia.qualtrics.com/jfe/form/SV_cGbCKj4xyP0H3wi >. For more information on privacy and data-handling, see the survey privacy statement <https://foundation.wikimedia.org/wiki/Wikimedia_Hackathon_Post-Event_Survey…>. The survey will remain open until May 29, 2022. 2. If you would like to share feedback but do not wish to take the Qualtrics survey, you can leave feedback on the Etherpad <https://etherpad.wikimedia.org/p/Wikimedia_Hackathon_2022_Feedback>. Finally, check out the badges <https://www.mediawiki.org/wiki/Wikimedia_Hackathon_2022/How_to#Joining_a_se…> the committee made. You can put them on your userpages to show your participation. Thank you again for joining us! It was so much fun to meet everyone and hack together. See you at the Wikimania Hackathon in August! Haley, on behalf of the 2022 Wikimedia Hackathon Team

1 0

Wikimedia Hackathon May 20-22: Info you need for tomorrow’s event
by Melinda Seckington 20 May '22

20 May '22

Hi everyone, We hope you’re ready for this three-day event, because the event starts in 10 hours! The main hackathon will take place over the weekend (Friday through Sunday), with two sets of core hours for sessions, social events, and hacking. These core hours are: - 3:00 <https://iw.toolforge.org/zonestamp/1653102000> - 6:00 <https://iw.toolforge.org/zonestamp/1653112800>UTC (Note: this is tonight for some time zones!) - 15:00 <https://iw.toolforge.org/zonestamp/1653145200>- 19:0 <https://iw.toolforge.org/zonestamp/1653159600>0 UTC We’re expecting the virtual space to be the busiest at these times. Outside of those core hours, you’re welcome to stay online to hack on projects, collaborate with others, or hang out in the virtual space. The goal with this schedule is to allow time for breaks and to accommodate as many time zones as possible. You are not expected to attend both sets of core hours - choose whichever hours work for you! For more info, see the Schedule <https://www.mediawiki.org/wiki/Wikimedia_Hackathon_2022/Schedule>. When will the event start? The opening ceremony will happen twice - once at 3:00 UTC, and once at 15:00 UTC on May 20. Find the links on the schedule <https://www.mediawiki.org/wiki/Wikimedia_Hackathon_2022/Schedule>! How can I join the virtual space? We’ll be using an online game-style space for the Hackathon. The links will be published shortly before the event on the hackathon page on MediaWiki.org <https://www.mediawiki.org/wiki/Wikimedia_Hackathon_2022>. There will be rooms for hacking and for sessions. Feel free to explore the virtual space and join any room - they’re open for everyone! What happens if I need help? Once the platform goes live, you will be able to find a Help Desk where you can ask questions, report any incidents, or just consult useful information about the event. There are also discussion channels <https://www.mediawiki.org/wiki/Wikimedia_Hackathon_2022/Discussions> that you can participate in. How can I work on a project? If you have an idea, you will be able to add your own projects <https://phabricator.wikimedia.org/project/board/5802/> on Phabricator. If you don’t know yet what to work on, see what projects <https://phabricator.wikimedia.org/project/view/5802/> others will be working on. You might find a project to join or get inspiration for your own idea! If you have any other questions, please check our FAQ <https://www.mediawiki.org/wiki/Wikimedia_Hackathon_2022/FAQ> section or leave a comment on the talk page. See you soon! Melinda, for the Hackathon Committee -- Melinda Seckington Developer Advocacy Manager Wikimedia Foundation <https://wikimediafoundation.org/>

1 1

[Toolhub @ Wikimedia Hackathon, May 20-22 2022]
by Slavina Stefanova 18 May '22

18 May '22

Hello all, On behalf of the Toolhub <https://toolhub.wikimedia.org/> team, I’m happy to announce our participation in the Wikimedia Hackathon 2022! New to Toolhub or curious about the latest features? Come join one of the introductory <https://phabricator.wikimedia.org/T308392> live sessions <https://phabricator.wikimedia.org/T307615> and find out how you can use Toolhub to search for tools to help with your workflows. For ways to contribute, check out the Hackathon 2022 project board <https://phabricator.wikimedia.org/project/board/5802/> and keep an eye out for tasks tagged “Toolhub”. Read more here: Toolhub Hackathon 2022 page <https://meta.wikimedia.org/wiki/Toolhub/Hackathon_2022>. Got questions or feedback? Leave them on our talk page <https://meta.wikimedia.org/w/index.php?title=Talk:Toolhub/Hackathon_2022&ac…>, and we’ll get back to you as fast as we can. See you soon! -- Slavina Stefanova (she/her) Software Engineer - Technical Engagement Wikimedia Foundation

1 0

[Small wiki toolkits] Fourth Bots workshop at Wikimedia Hackathon 2022 (May 20-22nd)
by Srishti Sethi 17 May '22

17 May '22

Hello everyone, The fourth workshop on the topic of "Running popular bots on wikis" is coming up - several sessions will be run as part of it at the upcoming Wikimedia Hackathon 2022 (May 20-22nd). You can find more details on the workshop here: < https://meta.wikimedia.org/wiki/Small_wiki_toolkits/Workshops#How_to_run_a_…> [1]. This workshop will introduce participants to a few popular bots in the Wikimedia ecosystem, how to set them up on the wikis, and tackle problems concerning the setup. This workshop will organize three sessions around InternetArchiveBot, YoBot, and SuggestBot. See more details < https://phabricator.wikimedia.org/T306593> [2]. We look forward to your participation! Best, Srishti On behalf of the SWT Workshops Organization team [1] https://meta.wikimedia.org/wiki/Small_wiki_toolkits/Workshops#How_to_run_a_… [2] https://phabricator.wikimedia.org/T306593 *Srishti Sethi* Senior Developer Advocate Wikimedia Foundation <https://wikimediafoundation.org/>

1 0

2024

2023

2022

2021

2020

2019

2018

2017

Cloud May 2022