Hi all!
In response to COVID-19, we are putting in place stricter guidelines around
deployments with an emphasis on site reliability.
To support this change, we have the following guidelines for software
development:
-
While we are not going to go into full emergency or holiday mode (i.e., no
releases), we do think it is necessary to de-risk the deployment train by
adding some additional scrutiny into the process. Our ask is that you take
extra precautions as outlined in our deployment guidelines below. Most
importantly, if you know you have limited availability to support a
deployment, don’t put your code on the train. When in doubt, ask.
-
Please review the COVID-19 deployment guidelines at
https://wikitech.wikimedia.org/wiki/Deployments/Covid-19
-
SWAT (emergency hot-fix) deploys will continue as is
-
We are limiting the frequency of onsite data center work to help
minimize the exposure of our team members who travel in and out of our data
center facilities. This will result in the general delay of hardware
installations and repairs, though we will continue being immediately
available for emergencies associated with uptime and critical
redundancies. We are still finalizing what this means and will provide
additional guidance when we have it.
Please err on the side of caution with the changes you merge.
Considerations (from the wikitech page)
-
Can you roll back this change without lasting impact?
-
A recovery plan is required as this will help identify our capacity
for recovering from the failure
-
THIS IS A KEY QUESTION, if you can’t answer it, you shouldn’t deploy
-
Is specialized knowledge required to support this change in production?
-
Are there multiple people with this knowledge?
-
Is there a way to increase confidence about the correctness of this
change?
-
Reviews (Design, Code, etc)
-
Testing coverage (unit tests, integration tests)
-
Manual testing (e.g. Beta, vagrant, docker)
We’re hosting office hours on Mondays at 17:00 UTC in #wikimedia-office
where you can ask questions regarding what is a good choice vs not.
Thank you all in advance for your understanding and empathy over the next
few weeks.
<3
-- Your Local (Internet) Neighborhood Release Engineers