Debian Stretch's security support ends in mid 2022, and the Foundation's
OS policy already discourages use of existing Stretch machines. That
means that it's time for all project admins to start rebuilding your VMs
with Bullseye (or, if you must, Buster.)
Any webservices running in Kubernetes created in the last year or two
are most likely using Buster images already, so there's no action needed
for those. Older kubernetes jobs should be refreshed to use more modern
images whenever possible.
If you are still using the grid engine for webservices, we strongly
encourage you to migrate your jobs to Kubernetes. For other grid uses,
watch this space for future announcements about grid engine migration;
we don't yet have a solution prepared for that.
Details about the what and why for this process can be found here:
https://wikitech.wikimedia.org/wiki/News/Stretch_deprecation
Here is the deprecation timeline:
March 2021: Stretch VM creation disabled in most projects
July 6, 2021: Active support of Stretch ends, Stretch moves into LTS
<- You are Here ->
January 1st, 2022: Stretch VM creation disabled in all projects,
deprecation nagging begins in earnest. Stretch alternatives will be
available for tool migration in Toolforge
May 1, 2022: All active Stretch VMs will be shut down (but not deleted)
by WMCS admins. This includes Toolforge grid exec nodes.
June 30, 2022: LTS support for Debian Stretch ends, all Stretch VMs will
be deleted by WMCS admins
_______________________________________________
Cloud-announce mailing list -- cloud-announce(a)lists.wikimedia.org
List information: https://lists.wikimedia.org/postorius/lists/cloud-announce.lists.wikimedia.…
Hello everyone,
The sixth workshop on the topic of "How to maintain bots" is coming up - it
will take place on Friday, July 29th at 16:00 UTC. You can find more
details on the workshop and a link to join here: <
https://meta.wikimedia.org/wiki/Small_wiki_toolkits/Workshops#How_to_mainta…>
[1].
This session will focus on best practices for maintaining bots and tools in
the Wikimedia ecosystem. It will cover a few practices that can help
developers run a bot or a tool with help from others, such as picking a
license, adding co-maintainers to the project, publishing source code,
writing docs, and much more.
To participate in this workshop, you would need basic familiarity with bots
or tools development. You can add your discussion ideas in the etherpad doc
linked from the workshops page.
We look forward to your participation!
Best,
Srishti
On behalf of the SWT Workshops Organization team
[1]
https://meta.wikimedia.org/wiki/Small_wiki_toolkits/Workshops#How_to_mainta…
*Srishti Sethi*
Senior Developer Advocate
Wikimedia Foundation <https://wikimediafoundation.org/>
Hello all,
The Wikimania Hackathon is August 12-14, just over 2 weeks away! Details
are listed below.
Registration
Registration for the Wikimania Hackathon has now opened [1]! To
register, submit
your information to the Wikimania organizers, which will give you access to
the Wikimania platform [2]. You can also optionally add your name to
the participants
page on the Wikimania wiki [3].
Platform
The Hackathon will take place virtually on Pheedloop, the Wikimania
platform [4]. This platform complies with WCAG 2.1 AA, and will support
screen readers, font adjustments, and many other accessibility features.
Video sessions will be held in Jitsi through this platform.
Format of the event
The Hackathon consists of events spread over three days [5]:
-
On the first day, there will be a pre-Hacking showcase to share project
ideas and find collaborators. Anyone can present a project, and anyone can
come as an observer.
-
Throughout the next two days, there will be open hacking, social events,
and technical sessions. Anyone can offer a session; just claim a slot on
the schedule!
-
Finally, there will be a final showcase to share the projects worked on
during the Hackathon.
Preparing for the Hackathon
There are many ways to take part in the event. Think about what project you
might want to work on (see examples from past Hackathons [6]), add your
idea to Phabricator [7], and consider presenting at the pre-Hacking
showcase. Host a session by adding information to the schedule [5]. Check
out information for newcomers [8]. And don’t forget to register [2]!
Best wishes,
Haley and the Developer Advocacy Team
[1] https://wikimania.wikimedia.org/wiki/Hackathon
[2] https://pheedloop.com/register/wikimania2022/attendee/
[3] https://wikimania.wikimedia.org/wiki/Hackathon/Participants
[4]
https://diff.wikimedia.org/2022/07/20/the-platform-powering-wikimania-2022/
[5] https://wikimania.wikimedia.org/wiki/Hackathon/Schedule
[6] https://www.mediawiki.org/wiki/Wikimedia_Hackathon_2022/Showcase
[7] https://phabricator.wikimedia.org/project/board/6030/
[8] https://wikimania.wikimedia.org/wiki/Hackathon/Newcomers
(If you don’t work with links tables such as templatelinks, pagelinks and
so on, feel free to ignore this message)
TLDR: The schema of links tables (starting with templatelinks) will change
to have numeric id pointing to linktarget table instead of repeating
namespace and title.
Hello,
The current schema and storage of most links tables are: page id (the
source), namespace id of the target link and title of the target. For
example, if a page with id of 1 uses Template:Foo, the row in the database
would be 1, 6, and Foo (Template namespace has id of 6)
Repeating the target’s title is not sustainable, for example more than half
of Wikimedia Commons database is just three links tables. The sheer size of
these tables makes a considerable portion of all queries slower, backups
and dumps taking longer and taking much more space than needed due to
unnecessary duplication. In Wikimedia Commons, on average a title is
duplicated around 100 times for templatelinks and around 20 times for
pagelinks. The numbers for other wikis depend on the usage patterns.
Moving forward, these tables will be normalized, meaning a typical row will
hold mapping of page id to linktarget id instead. Linktarget is a new table
deployed in production and contains immutable records of namespace id and
string. The major differences between page and linktarget tables are: 1-
linktarget values won’t change (unlike page records that change with page
move) 2- linktarget values can point to non-existent pages (=red links).
The first table being done is templatelinks, then pagelinks, imagelinks and
categorylinks will follow. During the migration phase both values will be
accessible but we will turn off writing to the old columns once the values
are backfilled and switched to be read from the new schema. We will
announce any major changes beforehand but this is to let you know these
changes are coming.
While the normalization of all links tables will take several years to
finish, templatelinks will finish in the next few months and is the most
pressing one.
So if you:
-
… rely on the schema of these tables in cloud replicas, you will need to
change your tools.
-
… rely on dumps of these tables, you will need to change your scripts.
Currently, templatelinks writes to both data schemes for new rows in most
wikis. This week we will start backfilling the data with the new schema but
it will take months to finish in large wikis.
You can keep track of the general long-term work in
https://phabricator.wikimedia.org/T300222 and the specific work for
templatelinks in https://phabricator.wikimedia.org/T299417. You can also
read more on the reasoning in https://phabricator.wikimedia.org/T222224.
Thanks
--
*Amir Sarabadani (he/him)*
Staff Database Architect
Wikimedia Foundation <https://wikimediafoundation.org/>
Hi everyone!
The 2021 survey collected feedback from Toolforge project members and Cloud
VPS project administrators on how the services offered can be improved to
help their development and maintenance needs. It ran from 2021-12-21 to
2022-02-05, and had 118 participants.
Please find below the published
result:https://meta.wikimedia.org/wiki/Research:Cloud_Services_Annual_Surve…
Thanks to everyone who participated and provided input and comments. They
are very useful and instrumental in shaping the future and improvements for
the cloud services.
We will launch the 2022 Cloud Services survey in a couple of months
following a similar methodology.
Thank you!
--
Seyram Komla Sapaty
Developer Advocate
Wikimedia Cloud Services
Hello everyone!
Mark your calendars for the Wikimania Hackathon! The Wikimania 2022
Hackathon is a free, online event open to the general public to work
together on technical projects, learn new skills, and meet other technical
contributors. You can find more information about this on the Wikimania page
<https://wikimania.wikimedia.org/wiki/2022:Hackathon>, which will continue
to grow over the next few weeks. For more details, read below.
*Schedule: *
The Hackathon will take place virtually in three time blocks:
16- 22 UTC August 12
12-17 UTC August 13
A final showcase on August 14.
*Format:*
At the beginning of the event, anyone can present a technical project they
plan to work on at the Pre-Event Showcase. This session will provide a
space for people to find teams, brainstorm new ideas, and make plans for
what to hack during the Hackathon.
Throughout the following hours, there will be social sessions and training
sessions run by the community, but the majority of the time will be spent
hacking together. Attendees can also attend other Wikimania activities.
At the end of the event, there will be a final showcase to present advances
or new technical projects built during the Hackathon to all of Wikimania.
*Accessibility:*If you have any accessibility or translation requests,
please contact hlepp(a)wikimedia.org.
Looking forward to spending time hacking together!
Cheers,
Haley and the WMF Developer Advocacy Team