On Mon, May 6, 2013 at 7:43 PM, Erik Moeller erik@wikimedia.org wrote:
On Mon, May 6, 2013 at 7:20 PM, MZMcBride z@mzmcbride.com wrote:
The reason I ask about a distinction is that there have been a lot of changes to Wikimedia wikis lately and likely more to come, as the Wikimedia Foundation has gotten larger and has more dedicated tech resources. Overall, this is great. But big new features come with big changes, and these changes sometimes need a bit of breathing room. I've read a lot of pushback lately against rapid changes (usurping usernames, getting rid of the new message indicator, etc.). A lot of this seems mostly outside the scope how often to deploy (and I don't want to shift the focus of this thread), but it gets confusing (to me, at least) to make a distinction between new code/features on Wikimedia wikis and how often to branch MediaWiki core/extensions.
A lot of this could potentially be addressed in a consistent manner across wikis if we applied the alpha->beta->prod (or just beta->prod for starters) channel model that's used on the Wikimedia mobile sites. Then features (whether in core or extensions) could be flagged for alpha or beta readiness, and users would only get them if they've decided to opt into either of those channels. We could still flip the switch from beta->prod, but that decision could be decoupled from the weekly deployment cycle.
This would likely be done for features & changes which have significant user-facing impact, and where segregation into "on" and "off" modes is possible (not always the case).
I think this is awesome for features ... but if we're putting work into this, I would love even more to have a clustered a+b production environment, such that 10% of folks are put on the new release (cluster a) and then it gets pushed over to cluster b. Then we can also test performance in a real world environment, and breakages only happen for 10% (PS the 10% number was pulled out of thin air).
The opt-in beta is much too limited, as well as being inapplicable to the vast majority of our traffic (logged in users are such a small percentage) to make proper comparisons. You could also see the impact of features on usage for average users.
We may want to consider at least putting some such scaffolding for beta->prod desktop modes into place before shifting to weekly deployments, although if that holds up this change significantly, I'd be in favor of making the shift first and then iterating.
Right now we have lots of individual "experimental" prefs, some dark launch URL parameters (&useNew=1 for the account creation forms etc.), some changes that are announced widely but then rolled out immediately (section edit link change), etc. What would be the disadvantage of having a single "I'd like the latest and greatest changes once they come in" preference for our users? The main disadvantage I see is that we'd need to temporarily retain two codepaths for significant user-facing changes, potentially increasing code complexity a fair bit, but perhaps reducing post-launch cost in return. And we'd need to consider more carefully if/when to make the beta/prod switch -- not necessarily a bad thing. ;-)
Have there been any negative experiences with this model on the mobile sites?
Erik
-- Erik Möller VP of Engineering and Product Development, Wikimedia Foundation
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
-- Leslie Carr Wikimedia Foundation AS 14907, 43821 http://as14907.peeringdb.com/