Small mistake in the above email, on-call is Cathal and Eoghan

On Tue, Aug 22, 2023 at 11:09 AM Clément Goubert cgoubert@wikimedia.org wrote:

Hi y'all,

I'm about to start sending 2% of global traffic to mw-on-k8s https://phabricator.wikimedia.org/T341780#9108656 On-call has been made aware (it's me and slyngs), emergency rollback is:

1. Revert https://gerrit.wikimedia.org/r/c/operations/puppet/+/951131
2. sudo cumin -b 20 'A:cp-text and P{P:trafficserver::backend}'

'run-puppet-agent -q'

If you want to see what happens, you can check out the mw-on-k8s dashboard https://grafana.wikimedia.org/d/U7JT--knk/mw-on-k8s This will impact the mw-web and mw-api-ext deployments.

Cheers,

-- Clément Goubert (they/them) Senior SRE Wikimedia Foundation