I had a great conversation with Liam Wyatt at Wikimania (cc'ing him, in
case he doesn't follow this list). We talked about strategies for deploying
new products on Wikimedia projects: what works, what doesn't. He held up
the design/deployment process for Vector as an example of *good* process,
one that we should (re)adopt.
Vector was created based on extensive user research and community
consultation[1]. Then WMF made a beta, and invited people across projects
to opt-in and try it out on prototype wikis[2]. The product team set public
criteria for when it would release the product as default across
production projects: retention of 80% of the Beta users who had opted in,
after a certain amount of time. When a beta tester opted out, they were
sent a survey to find out why[3]. The product team attempted to triage the
issues reported in these surveys, address them in the next iteration, or
(if they couldn't/wouldn't fix them), at least publicly acknowledge the
feedback. Then they created a phased deployment schedule, and stuck to
it[4].
This was, according to Liam (who's been around the movement a lot longer
than most of us at WMF), a successful strategy. It built trust, and engaged
volunteers as both evangelists and co-designers. I am personally very eager
to hear from other community members who were around at the time what they
thought of the process, and/or whether there are other examples of good WMF
product deployments that we could crib from as we re-assess our current
process. From what I've seen, we still follow many good practices in our
product deployments, but we follow them haphazardly and inconsistently.
I agree wholeheartedly with your email. But I wonder if this part is a
bit looking at the past through rose coloured glasses. Vector roll out
was certainly better than some other feature rollouts, but... it was
hardly without pain if I remember correctly. Although it was a long
time ago, and before I was involved on the dev side, so my memory is a
bit fuzzy.
--
-bawolff