April 2022 - Cloud - lists.wikimedia.org

[Cloud-announce] Eliminating Debian Stretch in Cloud VPS
by Andrew Bogott 13 Oct '22

13 Oct '22

Debian Stretch's security support ends in mid 2022, and the Foundation's OS policy already discourages use of existing Stretch machines. That means that it's time for all project admins to start rebuilding your VMs with Bullseye (or, if you must, Buster.) Any webservices running in Kubernetes created in the last year or two are most likely using Buster images already, so there's no action needed for those. Older kubernetes jobs should be refreshed to use more modern images whenever possible. If you are still using the grid engine for webservices, we strongly encourage you to migrate your jobs to Kubernetes. For other grid uses, watch this space for future announcements about grid engine migration; we don't yet have a solution prepared for that. Details about the what and why for this process can be found here: https://wikitech.wikimedia.org/wiki/News/Stretch_deprecation Here is the deprecation timeline: March 2021: Stretch VM creation disabled in most projects July 6, 2021: Active support of Stretch ends, Stretch moves into LTS <- You are Here -> January 1st, 2022: Stretch VM creation disabled in all projects, deprecation nagging begins in earnest. Stretch alternatives will be available for tool migration in Toolforge May 1, 2022: All active Stretch VMs will be shut down (but not deleted) by WMCS admins. This includes Toolforge grid exec nodes. June 30, 2022: LTS support for Debian Stretch ends, all Stretch VMs will be deleted by WMCS admins _______________________________________________ Cloud-announce mailing list -- cloud-announce(a)lists.wikimedia.org List information: https://lists.wikimedia.org/postorius/lists/cloud-announce.lists.wikimedia.…

1 1

Changes in schema of MediaWiki links tables
by Amir Sarabadani 11 Aug '22

11 Aug '22

(If you don’t work with links tables such as templatelinks, pagelinks and so on, feel free to ignore this message) TLDR: The schema of links tables (starting with templatelinks) will change to have numeric id pointing to linktarget table instead of repeating namespace and title. Hello, The current schema and storage of most links tables are: page id (the source), namespace id of the target link and title of the target. For example, if a page with id of 1 uses Template:Foo, the row in the database would be 1, 6, and Foo (Template namespace has id of 6) Repeating the target’s title is not sustainable, for example more than half of Wikimedia Commons database is just three links tables. The sheer size of these tables makes a considerable portion of all queries slower, backups and dumps taking longer and taking much more space than needed due to unnecessary duplication. In Wikimedia Commons, on average a title is duplicated around 100 times for templatelinks and around 20 times for pagelinks. The numbers for other wikis depend on the usage patterns. Moving forward, these tables will be normalized, meaning a typical row will hold mapping of page id to linktarget id instead. Linktarget is a new table deployed in production and contains immutable records of namespace id and string. The major differences between page and linktarget tables are: 1- linktarget values won’t change (unlike page records that change with page move) 2- linktarget values can point to non-existent pages (=red links). The first table being done is templatelinks, then pagelinks, imagelinks and categorylinks will follow. During the migration phase both values will be accessible but we will turn off writing to the old columns once the values are backfilled and switched to be read from the new schema. We will announce any major changes beforehand but this is to let you know these changes are coming. While the normalization of all links tables will take several years to finish, templatelinks will finish in the next few months and is the most pressing one. So if you: - … rely on the schema of these tables in cloud replicas, you will need to change your tools. - … rely on dumps of these tables, you will need to change your scripts. Currently, templatelinks writes to both data schemes for new rows in most wikis. This week we will start backfilling the data with the new schema but it will take months to finish in large wikis. You can keep track of the general long-term work in https://phabricator.wikimedia.org/T300222 and the specific work for templatelinks in https://phabricator.wikimedia.org/T299417. You can also read more on the reasoning in https://phabricator.wikimedia.org/T222224. Thanks -- *Amir Sarabadani (he/him)* Staff Database Architect Wikimedia Foundation <https://wikimediafoundation.org/>

1 2

Toolforge account and replica databases
by Travis Briggs 29 Apr '22

29 Apr '22

Hello, As part of the WP 1.0 bot/tool project, we are no longer using webservices or grid/cron jobs on Toolforge. However we still access the English Wikipedia replica database and our tool specific database hosted on tools.db.svc.wikimedia.cloud. It has been suggested by team members that we delete our toolforge account since we're not using any resources on the toolforge server. However my question is: Would we still be able to access the databases mentioned above? If there's a lighter weight way of accessing those databases without having a toolforge login we'd be interested in that as well. Thanks, -Travis

2 1

[Cloud-announce] Reminder: All Debian Stretch VMs should be removed by May of 2022
by Andrew Bogott 29 Apr '22

29 Apr '22

There are still dozens of VMs in cloud-vps running Debian Stretch. All of these hosts will need to be deleted and replaced with VMs running either Buster or Bullseye in the next few months. Beginning in May we will begin to shut down Stretch instances. Please check this page for your projects, and take whatever steps are necessary to move off of Stretch: https://os-deprecation.toolforge.org/ Don't hesitate to reach out for help on IRC or mailing list if you need help with this migration. You may find Cinder volumes (https://wikitech.wikimedia.org/wiki/Help:Adding_Disk_Space_to_Cloud_VPS_ins…) especially useful for transferring data between VMs. Note that we do NOT recommend in-place OS upgrades of VMs; it is almost always better to start with a fresh host and transfer workloads over. Details about the what and why for this process can be found here: https://wikitech.wikimedia.org/wiki/News/Stretch_deprecation Here is the remaining deprecation timeline: May 1, 2022: All active Stretch VMs will be shut down (but not deleted) by WMCS admins. June 30, 2022: LTS support for Debian Stretch ends, all Stretch VMs will be deleted by WMCS admins _______________________________________________ Cloud-announce mailing list -- cloud-announce(a)lists.wikimedia.org List information: https://lists.wikimedia.org/postorius/lists/cloud-announce.lists.wikimedia.…

1 1

[IMPORTANT] login.toolforge.org now points to Buster Grid Engine
by Seyram Komla Sapaty 29 Apr '22

29 Apr '22

Hello, As WMCS continue to work on deprecating Toolforge Stretch Grid Engine[0], we are today switching login.toolforge.org to point to Buster Grid Engine[1]. This is a minor change and it's not expected to disrupt current workflows. === What Does This Change Mean For Existing Users === This change does not directly impact any existing jobs. Users will now have access to the new toolforge-jobs framework and the latest versions of our runtime environments. Note that this change will result in ssh fingerprint warnings when you first connect. The new fingerprints can be found at https://wikitech.wikimedia.org/wiki/Help:SSH_Fingerprints/login.toolforge.o… and https://wikitech.wikimedia.org/wiki/Help:SSH_Fingerprints/dev.toolforge.org. [0]: https://wikitech.wikimedia.org/wiki/News/Toolforge_Stretch_deprecation [1]: https://wikitech.wikimedia.org/wiki/News/Toolforge_Stretch_deprecation#Time… -- Seyram Komla Sapaty Developer Advocate Wikimedia Cloud Services

1 0

Re: [Wikimedia-l] Third Pywikibot workshop on Friday, April 29th, 16:00 UTC
by Srishti Sethi 27 Apr '22

27 Apr '22

Hello Handgod, We currently do not have the resources to offer the workshop content in different languages. We are investigating the possibility of taking this up in future projects. One thing, though, is that we are presently exploring Google Meet's live captioning feature, and interestingly, it offers translations in French. As I have not tested this feature myself, particularly for non-English translations, I am unsure how reliable it is, something you might want to try out! Best, Srishti *Srishti Sethi* Senior Developer Advocate Wikimedia Foundation <https://wikimediafoundation.org/> On Fri, Apr 15, 2022 at 1:31 PM Handgod Abraham <sambayo23(a)gmail.com> wrote: > Thanks ! Is a French translation planned please ? > > Le ven. 15 avr. 2022 à 15:42, Srishti Sethi <ssethi(a)wikimedia.org> a > écrit : > >> Hello everyone, >> >> >> The third workshop on the topic of "Writing Pywikibot scripts" is coming >> up - it will take place on Friday, April 29th at 16:00 UTC. You can find >> more details on the workshop and a link to join here: < >> https://meta.wikimedia.org/wiki/Small_wiki_toolkits/Workshops#How_to_write_…> >> [1]. >> >> >> This workshop will introduce participants to writing basic scripts via >> the Pywikibot framework. We will be focusing on examples of scripts that >> participants have requested to cover in the workshop (e.g., finding and >> replacing content, archiving discussions, etc.). You can add your ideas to >> the ongoing discussion in the etherpad doc linked from the workshops page. If >> you missed attending the previous two workshops, going through the workshop >> materials beforehand would be beneficial. >> >> >> We look forward to your participation! >> >> >> Best, >> >> Srishti >> >> >> On behalf of the SWT Workshops Organization team >> >> >> [1] >> https://meta.wikimedia.org/wiki/Small_wiki_toolkits/Workshops#How_to_write_… >> >> >> *Srishti Sethi* >> Senior Developer Advocate >> Wikimedia Foundation <https://wikimediafoundation.org/> >> >> _______________________________________________ >> Wikimedia-l mailing list -- wikimedia-l(a)lists.wikimedia.org, guidelines >> at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and >> https://meta.wikimedia.org/wiki/Wikimedia-l >> Public archives at >> https://lists.wikimedia.org/hyperkitty/list/wikimedia-l@lists.wikimedia.org… >> To unsubscribe send an email to wikimedia-l-leave(a)lists.wikimedia.org > >

1 1

Wikimedia Hackathon: Call for Sessions
by Haley Lepp 27 Apr '22

27 Apr '22

Hello all! Are you planning to attend the Wikimedia Hackathon <https://www.mediawiki.org/wiki/Wikimedia_Hackathon_2022> from May 20-22 next month? We hope to see you there! The main event will be held online. We have an open call for sessions on our schedule page <https://www.mediawiki.org/wiki/Wikimedia_Hackathon_2022/Schedule>. If you'd like to host a session, you can simply pick an open slot in the category which best fits your topic. The developer advocacy team also put together some suggestions <https://www.mediawiki.org/wiki/Wikimedia_Hackathon_2022/Schedule/Sessions> for how to create a fun session. You can also add project ideas to the Phabricator Board. <https://phabricator.wikimedia.org/project/view/5802/> We will share more information soon about how to join the online space and where to find local meetups. Take care! Haley and the Hackathon Committee

2 1

[Cloud-announce] Upgrading Openstack on 2022-05-02
by Vivian Rook 26 Apr '22

26 Apr '22

We will be running an Openstack upgrade from Victoria to Wallaby on 2022-05-02 Some networking interruptions will occur during the upgrade at 18:00 UTC https://zonestamp.toolforge.org/1651510817 -- *Vivian Rook (They/Them)* Site Reliability Engineer Wikimedia Foundation <https://wikimediafoundation.org/> _______________________________________________ Cloud-announce mailing list -- cloud-announce(a)lists.wikimedia.org List information: https://lists.wikimedia.org/postorius/lists/cloud-announce.lists.wikimedia.…

1 0

Heartbeat monitoring system?
by Sascha Brawer 25 Apr '22

25 Apr '22

Is there some heartbeat monitoring system in the Wikimedia cloud? I’m running a periodic cronjob (on toolforge/kubernetes) that I’d like to monitor. Ideally, my cronjob could send a heartbeat to some monitoring system for each successful run. For example, it would send an HTTP POST request to monitoring.wmcloud.org/mycronjob or whatever. When no heartbeat has been received for >3 weeks, I’d like to receive an email alert to some configured email address. Does Wikimedia operate some heartbeat monitoring system like this? Best, — Sascha

3 3

Code of Conduct Committee – call for new members
by Martin Urbanec 24 Apr '22

24 Apr '22

Hello all, It's coming close to the time for annual appointments of community members to serve on the Code of Conduct committee (CoCC). The Code of Conduct Committee is a team of five trusted individuals (plus five auxiliary members) with diverse affiliations responsible for general enforcement of the Code of conduct for Wikimedia technical spaces. Committee members are in charge of processing complaints, discussing with the parties affected, agreeing on resolutions, and following up on their enforcement. For more on their duties and roles, see https://www.mediawiki.org/wiki/Code_of_Conduct/Committee. This is a call for community members interested in volunteering for appointment to this committee. Volunteers serving in this role should be experienced Wikimedians or have had experience serving in a similar position before. The current committee is doing the selection and will research and discuss candidates. Six weeks before the beginning of the next Committee term, meaning 07 May 2022, they will publish their candidate slate (a list of candidates) on-wiki. The community can provide feedback on these candidates, via private email to the group choosing the next Committee. The feedback period will be two weeks. The current Committee will then either finalize the slate, or update the candidate slate in response to concerns raised. If the candidate slate changes, there will be another two week feedback period covering the newly proposed members. After the selections are finalized, there will be a training period, after which the new Committee is appointed. The current Committee continues to serve until the feedback, selection, and training process is complete. If you are interested in serving on this committee or like to nominate a candidate, please write an email to techconductcandidates AT wikimedia.org with details of your experience on the projects, your thoughts on the code of conduct and the committee and what you hope to bring to the role and whether you have a preference in being auxiliary or main member of the committee. The committee consists of five main members plus five auxiliary members and they will serve for a year; all applications are appreciated and will be carefully considered. The deadline for applications is *the end of day on 30 April 2022*. Please feel free to pass this invitation along to any users who you think may be qualified and interested. Best, Martin Urbanec, on behalf of the Code of Conduct Committee

1 1

2024

2023

2022

2021

2020

2019

2018

2017

Cloud April 2022