How’d we do in our strive for operational excellence last month?
Incidents
3 documented incidents. That's lower than June in the previous five years
where the month saw 5-9 incidents. I've added a new panel ⭐️ to the Incident
statistics <https://codepen.io/Krinkle/full/wbYMZK> tool. This one plots
monthly statistics on top of previous years, to more easily compare them.
Learn more from the Incident documents
<https://wikitech.wikimedia.org/wiki/Incident_status> on Wikitech, and
remember to review and schedule Incident Follow-up
<https://phabricator.wikimedia.org/project/view/4758/> in Phabricator,
which are preventive measures and other action items filed after an
incident.
-------
Trends
In June, work on production errors appears to have stagnated a bit. Or more
precisely, the work only resulted in relatively few tasks being resolved.
15 of the 26 new tasks are still open as of writing.
Of the tasks from previous months, only 11 were resolved, leaving most
columns unchanged. See the table further down for a more detailed breakdown
and links to Phabricator queries for the tasks in question.
With the 15 remaining new tasks, and the 11 tasks resolved from our
backlog, this raises the chart from 150 to 154 tasks.
Figure 1, Figure 2: Unresolved error reports stacked by month.
<https://phabricator.wikimedia.org/phame/post/view/240/production_excellence_33_june_2021/#trends>
Month-over-month plots based on spreadsheet data
<https://docs.google.com/spreadsheets/d/e/2PACX-1vTrUCAI10hIroYDU-i5_8s7pony8M71ATXrFRiXXV7t5-tITZYrTRLGch-3iJbmeG41ZMcj1vGfzZ70/pubhtml>
.
-------
Outstanding errors
Take a look at the workboard and look for tasks that could use your help.
→
https://phabricator.wikimedia.org/tag/wikimedia-production-error/
Summary over recent months:
Jan 2020 (1 of 7 left) ⚠️ Unchanged (over one year old).
Mar 2020 (2 of 2 left) ⚠️ Unchanged (over one year old).
Apr 2020 (4 of 14 left) ⚠️ Unchanged (over one year old).
May 2020 (5 of 14 left) ⚠️ Unchanged (over one year old).
Jun 2020 (5 of 14 left) ⚠️ Unchanged (over one year old).
Jul 2020 (4 of 24 issues) ⚠️ Unchanged (over one year old).
Aug 2020 (11 of 53 issues) ⬇️ One task resolved. -1
Sep 2020 (7 of 33 issues) ⚠️ Unchanged (over one year old).
Oct 2020 (19 of 69 issues) ⚠️ Unchanged (over one year old).
Nov 2020 (8 of 38 issues) ⚠️ Unchanged (over one year old).
Dec 2020 (7 of 33 issues) ⚠️ Unchanged (over one year old).
Jan 2021 (3 of 50 issues
<https://phabricator.wikimedia.org/maniphest/query/WIP9W8q54HB6/#R>) ⚠️
Unchanged (over one year old).
Feb 2021 (6 of 20 issues
<https://phabricator.wikimedia.org/maniphest/query/5MzPJAb5oJgv/#R>) ⬇️ One
task resolved. -1
Mar 2021 (13 of 48 issues
<https://phabricator.wikimedia.org/maniphest/query/RsVPep46KRY4/#R>) ⬇️ One
task resolved. -1
Apr 2021 (19 of 42 issues
<https://phabricator.wikimedia.org/maniphest/query/rYyMt_gYYymb/#R>) ⬇️
Four tasks resolved. -4
May 2021 (25 of 54 issues
<https://phabricator.wikimedia.org/maniphest/query/tmkGqt0C93YG/#R>) ⬇️
Four tasks resolved. -4
June 2021 (15 of 26 issues
<https://phabricator.wikimedia.org/maniphest/query/roL0TaxtcaLQ/#R>) 📌 26
new issues, of which 11 were closed. +26, -11
-------
Tally
150 issues open, as of Excellence #32 (May 2021)
<https://phabricator.wikimedia.org/phame/post/view/236/production_excellence_32_may_2021/>
.
-11 issues closed, of the previous 150 open issues.
+15 new issues that survived June 2021.
154 issues open as of yesterday.
-------
Thanks!
Thank you to everyone who helped by reporting, investigating, or resolving
problems in Wikimedia production. Thanks!
Until next time,
– Timo Tijhof
🔗 Share or read later via
https://phabricator.wikimedia.org/phame/post/view/240/