On Mon, May 6, 2013 at 7:43 PM, Erik Moeller erik@wikimedia.org wrote:
We may want to consider at least putting some such scaffolding for beta->prod desktop modes into place before shifting to weekly deployments, although if that holds up this change significantly, I'd be in favor of making the shift first and then iterating.
Right now we have lots of individual "experimental" prefs, some dark launch URL parameters (&useNew=1 for the account creation forms etc.), some changes that are announced widely but then rolled out immediately (section edit link change), etc. What would be the disadvantage of having a single "I'd like the latest and greatest changes once they come in" preference for our users? The main disadvantage I see is that we'd need to temporarily retain two codepaths for significant user-facing changes, potentially increasing code complexity a fair bit, but perhaps reducing post-launch cost in return. And we'd need to consider more carefully if/when to make the beta/prod switch -- not necessarily a bad thing. ;-)
Have there been any negative experiences with this model on the mobile sites?
For the most part, this model has been awesome for mobile. It allows us to iterate on feature ideas quickly, try a lot of different things, and focus on the things that really work. Getting a partially-polished feature in front of a group of users who expect to have things not always work 100% is incredibly valuable - and takes off the pressure of having to get something totally right. If it turns out the feature idea was a bust, it's generally no big deal for us (the mobile team or the users) to either can it or tweak it to make it better. Having an 'alpha' that is essentially a developer sandobx (little to no productization happens for our alpha) and some of the best mobile features have grown from here.
We've had occasional issues with beta or alpha features bleeding into a mode they weren't supposed to - though this is generally quick to fix. Ultimately, the issues we've had with the approach have been minor.
If this were something adopted by Mediawiki more generally, I would want to see it carefully built into core. Ori brings up a good point about increased code complexity, which is a guarantee with this kind of approach but could certainly be mitigated if the plumbing were well architected.
I think thoughtful development of something like this into core would be awesome and would allow us to collectively build better, more useful features, faster.