[Wikitech-l] Wikimedia production excellence (July 2019)

📘 Read on Phabricator at https://phabricator.wikimedia.org/phame/post/view/164 ------- How’re we doing on that strive for operational excellence? Read this first anniversary edition to find out! #### 📊 Month in numbers - 5 documented incidents. [1] - 53 new Wikimedia-prod-error reports. [2] - 44 closed Wikimedia-prod-error reports. [3] - 218 currently open Wikimedia-prod-error reports in total. [4] The number of recorded incidents over the past month, at five, is equal to the median number of incidents per month (2016-2019). – Explore the data at https://codepen.io/Krinkle/full/wbYMZK To read more about these incidents, their investigations, and pending actionables; check https://wikitech.wikimedia.org/wiki/Incident_documentation#2019 ------- #### *️⃣One year of Excellent adventures! Exactly one year ago [5] this periodical started to provide regular insights on production stability. The idea was to shorten the feedback cycle between deployment of code that leads to fatal errors and the discovery of those errors. This allows more people to find reports earlier, which (hopefully) prevents them from sneaking into a growing pile of “normal” errors. 576 reports were created between 15 July 2018 and 31 July 2019 (tagged Wikimedia-prod-error). 425 reports got closed over that same time period. ------- #### 📉 Outstanding reports Take a look at the workboard and look for tasks that might need your help. The workboard lists error reports, grouped by the month in which they were first observed. → https://phabricator.wikimedia.org/tag/wikimedia-production-error/ Or help someone who already started with their patch: → https://phabricator.wikimedia.org/maniphest/query/pzVPXPeMfRIz/#R Breakdown of recent months (past two weeks not included): - November: 1 report left (unchanged). ⚠️ - December: 3 reports left (unchanged). ⚠️ - January: 1 report left (unchanged). - February: 2 reports left (unchanged). - March: 4 reports left (unchanged). - April: 10 of 14 reports left (unchanged). - May: 2 reports got fixed! (4 of 10 reports left). ✨ - June: 2 reports got fixed! (9 of 11 reports left). ✨️ - July: 18 new reports from last month remain unsolved. ------- #### 🎉 Thanks! Thank you to Aaron Schulz, Ariel Glenn, Brad Jorsch, CScott, Daimona, David Barratt, David Causse, EBernhardson, Gergő Tisza, James Forrester, Jeena Huneidi, MarcoAurelio, Roan Kattouw, Stephane Bisson, Tchanders, Tim Starling, Urbanecm; and everyone else who helped by finding, investigating, or resolving error reports in Wikimedia production. Thanks! Until next time, – Timo Tijhof ------- Footnotes: [1] Incidents. – https://wikitech.wikimedia.org/wiki/Special:PrefixIndex?prefix=Incident+doc… [2] Tasks created. – https://phabricator.wikimedia.org/maniphest/query/9m.NNvBAvuHF/#R [3] Tasks closed. – https://phabricator.wikimedia.org/maniphest/query/6sekLiUhHCmq/#R [4] Open tasks. https://phabricator.wikimedia.org/maniphest/query/47MGY8BUDvRD/#R [5] The initial e-mail. – https://lists.wikimedia.org/pipermail/wikitech-l/2018-July/090363.html