Wikitech-l August 2023

wikitech-l@lists.wikimedia.org

33 participants
32 discussions

Datacenter Switchover process change
by Alexandros Kosiaris 31 Aug '23

31 Aug '23

Hello, I 'd like to inform everyone of what we consider a big change (for the better) in the Datacenter Switchover process. The full rationale, planning and implementation is documented at https://wikitech.wikimedia.org/wiki/Switch_Datacenter/Recurring,_Equinox-ba… and it includes a TL;DR that I am pasting below for everyone's convenience: Site Reliability Engineering will, starting September 2023, run a data center Switchover every 6 months, in the week of the solar Equinox <https://en.wikipedia.org/wiki/Equinox>, namely the *work weeks containing March 21st and September 21st*. If you are interested to learn more about Switchovers and why we perform them, or already know what they are and want to learn more about how this proposal would impact your workflows or the Wikimedia Movement, please read on. We hope that making the Switchover dates and duration predictable will allow the teams involved and/or utilizing a Switchover, as well as the entire movement reap the benefits we anticipate and document in the doc linked above. Regards, -- Alexandros Kosiaris Principal Site Reliability Engineer Wikimedia Foundation

1 0

Moving Wikimedia Developer Account creation
by Simon Lyngshede 31 Aug '23

31 Aug '23

Hi all, We have removed the developer account signup from Wikitech and are now directing new users towards https://idm.wikimedia.org instead. This is the first step in the implementation of our new identity manager, and a small step towards making Wikitech a little less special. Part of our documentation was updated yesterday, August 30th, and the first accounts have already been successfully created. Should there be a need to re-enable the account creation process on Wikitech, this can be done by reverting: https://gerrit.wikimedia.org/r/c/operations/mediawiki-config/+/952209/ If you have any questions, concerns or documentation that needs updating, please reach out to me or the infrastructure foundation team. Thank you to everyone who helped us get this far. -- Simon Lyngshede Site Reliability Engineer / Infrastructure Foundations Wikimedia Foundation

1 1

September 2023 Datacenter Switchover
by Kamila Součková 30 Aug '23

30 Aug '23

Dear Wikitechians, On *Wednesday September 20th*, the SRE team will run a planned data center switchover, moving all wikis from our data center in Virginia to the data center in Texas. This is an important periodic test of our tools and procedures, to ensure the wikis will continue to be available even in the event of major technical issues in our primary home. It also gives all our SRE and ops teams a chance to do maintenance and upgrades on systems in Virginia that normally run 24 hours a day. The switchover process requires a *brief read-only period for all Foundation-hosted wikis*, which will start at *14:00 UTC on Wednesday **September 20th*, and will last for a few minutes while we execute the migration as efficiently as possible. All our public and private wikis will be continuously available for reading as usual, but no one will be able to save edits during the process. Users will see a notification of the upcoming maintenance, and anyone still editing will be asked to try again in a few minutes. CommRel will soon begin notifying communities of the read-only window. If you like, you can follow along on the day in the public #wikimedia-operations channel on IRC (instructions for joining here <https://meta.wikimedia.org/wiki/IRC/Instructions>). To report any issues, you can reach us in #wikimedia-sre on IRC, or file a Phabricator ticket with the *datacenter-switchover* tag (pre-filled form here <https://phabricator.wikimedia.org/maniphest/task/edit/form/1/?projects=Data…>); we'll be monitoring closely for reports of trouble during and after the switchover. (If you're new to Phab, there's more information at Phabricator/Help.) The switchover and its preparation will be tracked in Phabricator task T345263 <https://phabricator.wikimedia.org/T345263>. On behalf of the SRE team, please excuse the disruption, and our thanks to everyone in a number of departments who've been involved in planning this work. Feel free to reply directly to me with any questions. Thank you, Kamila Součková (they/them) Senior SRE Wikimedia Foundation

1 0

Global Lua modules and templates
by Felipe Schenone 30 Aug '23

30 Aug '23

Hi! As many of you know, a central global repository for Lua modules and templates has been a frequent request since the early days of the movement. This year, I programmed a JavaScript tool called Synchronizer https://www.mediawiki.org/wiki/Synchronizer (inspired on a previous tool called DiBabel by User:Yurik) The tool allows to synchronize (that is, automatically copy) Lua modules across Wikimedia wikis, and provides other features to help developers update and maintain global modules. I also re-wrote the documentation at https://www.mediawiki.org/wiki/Multilingual_Templates_and_Modules to account for the new tool. It basically describes how to develop a Lua module that can be copied unchanged to any wiki, by abstracting things like user-readable strings and config. Admittedly, this is a "poor man's version" of a proper solution to the problem, but one I find invaluable while developing and maintaining modules. Hopefully some of you may find it useful too!

3 4

Help needed in finding a replacement to the mobile-friendly page content Rest API for my web app
by harsh410＠hotmail.com 29 Aug '23

29 Aug '23

Help needed in finding a replacement to the mobile-friendly page content Rest API for my web app Hello All, For my website webnomads.in I have been using mobile-friendly page content Rest API for years . Now that it has been stopped I am struggling to find a new suitable API as I was displaying the content in sections . On various pages I read about making use of parsoids . While I have explored the different API offerings , I could not find any API which is delivering the text based on sections . I will be very thankful if someone can guide me or point me to any article showing the use of Parsoid or any other way to deliver page content in text form and in sections . Thanks in advance . Regards Harsh

3 2

contint2002 Jenkins maintenance Tue, 29 Aug 2023 08:00-8:15 UTC Inbox
by Jelto Wodstrcil 28 Aug '23

28 Aug '23

Hi all, We will perform a short maintenance on contint2002 host tomorrow 29 Aug 2023 08:00-8:15 UTC. The machine hosts integration.wikimedia.org, an instance of jenkins and zuul. Services have to be stopped and restarted, so expect some unavailability for around 15 minutes. Alias/cname contint.wikimedia.org also maps to this machine. Greetings Jelto

1 0

New mediawiki.page_change.v1 event stream publicly available
by Luke Bowmaker 25 Aug '23

25 Aug '23

The Wikimedia Data Engineering team is pleased to announce that a new event stream, mediawiki.page_change.v1, is now publicly available at stream.wikimedia.org (here <https://stream.wikimedia.org/v2/ui/#/?streams=mediawiki.page_change.v1>). The new event stream models page changes using a consolidated changelog data model, whereas existing streams, like page-create and revision-create, model each type of page change as a separate stream. With the current model, you would have to consume multiple streams to understand how a MediaWiki page changed. With the new stream, when a page is created, edited, or deleted, an event captures the new state, as well as the state prior to the change. For more information on what fields you can expect to see in the events, see the schema definition here <https://schema.wikimedia.org/repositories//primary/jsonschema/mediawiki/pag…> . Starting now, new streams will be suffixed with a major version and will not use hyphens in stream names. For more information, see here <https://wikitech.wikimedia.org/wiki/Event_Platform/Stream_Configuration#Str…> . Benefits: - Only one stream to consume (instead of having to consume page-create, page-delete, and revision-create streams to have a full picture of page changes) - Events are ordered for a given page_id (delete will not come before create) - Latest event has current and prior state The existing event streams, such as page-create, revision-create, page-delete, etc will continue to remain available, for now. We encourage people to migrate to the new consolidated stream when they can as we plan to mark the existing streams as deprecated within the next year. We will send another communication about this when the plan is decided. If you have any questions or issues please drop a Phabricator ticket here <https://phabricator.wikimedia.org/project/board/6628/> -- Luke Bowmaker Data Engineering - Product Manager

1 0

Pabricator downtime: Friday 25th August, 09:00 UTC
by Eoghan Gaffney 24 Aug '23

24 Aug '23

Hi friends, Phabricator will be unavailable tomorrow morning, Friday 25th August at 09:00 UTC. This should take no more than 15 minutes. If you have any questions or concerns, please let us know! --Eoghan

1 0

Brief Phabricator downtime incoming for update to Phorge @ 15:00 UTC
by Brennen Bearnes 23 Aug '23

23 Aug '23

Background: https://phabricator.wikimedia.org/T333885 -- Brennen Bearnes Release Engineering Wikimedia Foundation

1 0

MediaWiki Insights - first monthly email and upcoming interviews
by Birgit Müller 23 Aug '23

23 Aug '23

Hello All, Following up from conversations at the Wikimedia Hackathon in May <https://phabricator.wikimedia.org/T336990>, I’d like to give an update on how we’re approaching the work around MediaWiki and what to expect in the upcoming months! Since the beginning of July, the Foundation has dedicated MediaWiki product leadership and a new MediaWiki engineering group <https://www.mediawiki.org/wiki/MediaWiki_Engineering_Group>. We’re in the process of onboarding (i.e. team processes, scope and operational questions, getting familiar with the domain), have been spreading out to help with specific projects as consultants and/or code contributors and are now kicking off broader research to inform a strategy for MediaWiki. This email is the first edition of a monthly email report to wikitech-l and mediawiki-l. Reports can also be found on <https://www.mediawiki.org/wiki/MediaWiki_Product_Insights/Reports>this page <https://www.mediawiki.org/wiki/MediaWiki_Product_Insights/Reports>. The first 12 months are about building out the foundations for the following years: - Building up the new MW Engineering group and MW Product function {{doing}} - Developing a strategy for MediaWiki - by June 30th, 2024 [Wikimedia Foundation Annual Plan, WE3 <https://meta.wikimedia.org/wiki/Wikimedia_Foundation_Annual_Plan/2023-2024/…>] {{kicking off}} - Reaching a 20% increase of authors to selected MediaWiki repositories deployed in Wikimedia production - by June 30th, 2024 [WE3.2] {{preparing}} - Investing in developer experiences and reduce fragmentation of developer workflows - continuous work with specific deliverables in 2023/24 [WE 3.1] {{doing}} - Exploring and resolving a set of questions around stewardship and Open Source strategy (goes beyond MediaWiki) [WE3.3] {{preparing}} The approach that we’re trying to take is to work problem-focused (start from the problem, usage and needs), data driven (conduct research to inform decisions), step by step (prioritize and scope things, since we can’t address all questions at once), and take people along (share regularly about the work, initiate exchange and invite people in). The research to inform this work <https://www.mediawiki.org/wiki/MediaWiki_Product_Insights> consists of multiple pillars and phases: - MediaWiki within Wikimedia’s ecosystem: Key challenges and opportunities. Includes interviews and review of existing write-ups (pre-kick off at the Hackathon in May, continue in July) - MediaWiki usage and development beyond Wikimedia: Synergy effects, opportunities and challenges (start in November) - Components-workflow mapping exercise. How are core workflows such as edit or patrol powered and enabled by MW core components, extensions, modules, gadgets, tools as of today? Initial focus is on prototyping this for 1-2 Wikipedias and potentially Wikidata (start in September) - Exploration of MW platform designs - what changes could we make to increase the sustainability of the platform, better serve our core workflows and needs and provide easier, faster and cohesive paths to feature development? (October-June) - Ways to establish effective and responsible stewardship for the platform (October-June) MediaWiki within Wikimedia’s ecosystem: Interviews A key part of research is talking to folks. We’ve kicked this off with first interviews with MediaWiki developers <https://phabricator.wikimedia.org/T336990> at the Hackathon in May, have surveyed ourselves (= MediaWiki Engineering group) and plan to conduct more interviews with a cross-section of people over the next 6 weeks. The main focus of this round is on MediaWiki within Wikimedia’s ecosystem and we’ll be reaching out to a few more MediaWiki volunteer developers starting this week. We’re planning to organize broader MediaWiki ecosystem conversations in the upcoming quarters. Project snapshot: Onboarding via Code Mob As we’re building up the new MediaWiki Engineering Group, we’re also investing in onboarding team members to MediaWiki development and MediaWiki APIs who are just getting started. Bill Pirkle has come up with an onboarding concept and is hosting weekly “Code Mob” sessions for folks on the teams across MW Engineering - if you’re interested in “learning along”, you can find a list of recordings and topics on this page <https://meta.wikimedia.org/wiki/User:BPirkle_(WMF)/Code_Mobs_2023>. Reporting and documentation We will share about our work through the monthly emails and in 3-4 MediaWiki Product Insights reports published throughout the Foundations’ fiscal year (July-June). Over the next weeks, we will also create or update team pages and clean up + create Phabricator workboards to make it easier to follow along (and for us to keep track/focus :-). Your ideas, support, patience as we’re settling in, experiences and insights in challenges and opportunities really matter for this to work out. A big thanks to all the folks who already have offered support, shared their experiences, or otherwise helped out so far! <3 If you have specific questions or thoughts on this edition, this talk page <https://www.mediawiki.org/wiki/Talk:MediaWiki_Product_Insights> is a good place to connect :-) Thanks all for reading! Birgit -- Birgit Müller (she/her) Director of Product, MediaWiki and Developer Experiences Wikimedia Foundation <https://wikimediafoundation.org/>

1 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Wikitech-l August 2023