How’d we do in our strive for operational excellence last month? Read on to find out! Incidents
Zero documented incidents last month. Isn't that something! (Incident graphs https://codepen.io/Krinkle/full/wbYMZK)
Learn about past incidents at Incident status https://wikitech.wikimedia.org/wiki/Incident_status on Wikitech. Remember to review and schedule Incident Follow-up https://phabricator.wikimedia.org/project/view/4758/ in Phabricator, which are preventive measures and other action items to learn from. Trends
In August we resolved 18 of the 156 reports that carried over from previous months, and reported 46 new failures in production. Of the new ones, 17 remain unresolved as of writing and will carry over to next month.
The number of new errors reports in August was fairly high at 46, compared to 31 reports in July, and 26 reports in June.
The backlog of "Old" issues saw no progress this past month and remained constant at 146 open error reports.
Figure 1, Figure 2: Unresolved error reports, stacked by month. https://phabricator.wikimedia.org/phame/post/view/248/production_excellence_35_august_2021/#trends
💡 *Did you know*: You can zoom in to your team's error reports by using the appropriate "Filter" link in the sidebar of our shared workboard?
Take a look at the workboard and look for tasks that could use your help.
→ https://phabricator.wikimedia.org/tag/wikimedia-production-error/ Progress
Last few months in review: Jan 2021 (50 issues https://phabricator.wikimedia.org/maniphest/query/gn7TOpf2LdVE/#R) 3 left. Feb 2021 (20 issues https://phabricator.wikimedia.org/maniphest/query/xQxnXZys4q97/#R) 6 > 5 left. Mar 2021 (48 issues https://phabricator.wikimedia.org/maniphest/query/To0edISjsA9s/#R) 13 > 10 left. Apr 2021 (42 issues https://phabricator.wikimedia.org/maniphest/query/ORxSVxnJBlLc/#R) 18 > 17 left. May 2021 (54 issues https://phabricator.wikimedia.org/maniphest/query/9y.PWGoGgWbK/#R) 22 > 20 left. Jun 2021 (26 issues https://phabricator.wikimedia.org/maniphest/query/DlpqBkLj0aP4/#R) 11 > 10 left. Jul 2021 (31 issues https://phabricator.wikimedia.org/maniphest/query/qQAV178rYaJ_/#R) 16 > 12 left. Aug 2021 (46 issues https://phabricator.wikimedia.org/maniphest/query/_VlOsgZ9On4g/#R) +17 new unresolved issues.
Tally: 156 issues open, as of Excellence #34 https://phabricator.wikimedia.org/phame/post/view/247/production_excellence_34_july_2021/ (July 2021). -18 issues closed, of the previously open issues. +17 new issues that survived August 2021. 155 issues open, as of today (3 Sep 2021).
For more month-over-month numbers refer to the spreadsheet https://docs.google.com/spreadsheets/d/e/2PACX-1vTrUCAI10hIroYDU-i5_8s7pony8M71ATXrFRiXXV7t5-tITZYrTRLGch-3iJbmeG41ZMcj1vGfzZ70/pubhtml . Thanks!
Thank you to everyone who helped by reporting, investigating, or resolving problems in Wikimedia production. Thanks!
Until next time,
– Timo Tijhof
🔗 Share or read later via https://phabricator.wikimedia.org/phame/post/view/248/