On Mon, 2013-05-06 at 12:52 -0700, Greg Grossmeier wrote:
Also, the majority of bugs that are in the Highest/Immediate priority level (from my gut assessment, I don't have the data here) are found after a deploy to non-WP projects.
I agree with that impression: We don't get many (manually found) highest/immediate prio bug reports after the first deployment phase, most of them after phase 2, and a few after phase 3 (e.g. when we failed to understand the explosive force of an issue).
Backing that impression up with Bugzilla data: <tl;dr>: That's hard.
Long version: I tried a Bugzilla query for tickets created in the last four months, that at some point in their lifetime had Priority = {Highest | Immediate}, restricted it to the products {MediaWiki, MediaWiki extensions, Wikimedia}, made buglist.cgi display the "Opened" column (via "Change columns" at the bottom); dropped the last 9 characters of the "Opened" column (to get rid of the time and only have the date, though that's UTC so does not perfectly fit our deployment *time*), imported the resulting CSV into OOCalc, cumulated a bit, and summed up all those tickets that got filed in a certain deployment phase && at *some* point became highest/immediate. See attachment.
The results don't back up my impression. One potential reason: Development teams file tickets *at some point* and don't see priority immediately, and when tickets get triaged they get higher priority at some point later on. Maybe results would look different if I the query excluded reporters that are employees? Don't want to spend too much time trying though.
andre