On Mon, Jul 12, 2021 at 10:47 PM Risker <risker.wp@gmail.com> wrote:
On Mon, 12 Jul 2021 at 19:26, Tyler Cipriani <tcipriani@wikimedia.org> wrote:
* Can we do something to improve the speed from "a user notices an issue with the site" to "the right team/owner is aware of it and acts on it"?

Or can we do something to improve how many issues users notice? :)


As someone who's been around for a long time as an editor, I can say honestly that having most of the issues addressed before they hit the really big projects has resulted in a huge improvement.  The train really works, and the only challenge I really see is what Jon mentions in his original post.  Some of those issues aren't really that significant in the great scheme of things, but there's a big leap when something takes two business days to fix from the Tuesday deployment and two business days to fix from the Thursday deployment. 

It's not always possible for even the best developer and the best testing systems to catch an issue that will be spotted by a hands-on user, several of whom are much more familiar with the purpose, expected outcomes and change impact on extensions than the people who have written them or QA'd them.  That's why there will always be plenty of issues that are identified by users, and it is in no way a problem that a small number of them (compared to what we saw 10-15 years ago) get through to the end of the train before being identified as needing to be addressed (for different values of "addressed"). 

Thank you for this response! The train existed before I started thinking about MediaWiki-software deployment. The impression that it has had a positive impact on the number of problems seen by users is important information. Your response is a fantastic answer to a different question I wonder about a lot: why does the train process give us confidence in the code being released?

The next part of that question is: are there ways we can gain this confidence with less disruption? I'd be interested in trying to catalog the types of problems that are only spotted by hands-on users in the interest of seeing if patterns emerge.

Thank you again!
– Tyler