Hello all,
------ tl;dr: There's a difference between how the WMF Platform team and Feature Teams commit to master. Why? How can we unify and/or make testing from development branches easier for everyone? ------
Two weeks ago Ops and Platform both had all staff local in SF for a week for a slew of meetings. I took that opportunity to make even more meetings with people/teams regarding our dev and deployment process. (I promise I'll be nicer in the future.)
You can see the fruits of that labor here: https://wikitech.wikimedia.org/wiki/Deployments/Features_Process/General_Fee...
(More info linked from: https://wikitech.wikimedia.org/wiki/Deployments/Features_Process )
One of the things that became apparent is that the Features Teams (eg: E2, E3, Visual Editor) tend to have separate branches that they develop on and only merge to master closer to their deploy windows. There is some variation among them, of course, but the general idea stands.
This contrasts with the way the WMF Platform team develops; we predominately commit to master as we develop with any new features set as disabled in a config until it is ready. The Features Teams also use the "disabled in a config until ready" bit, of course.
What is the reasoning for the use of this development/deployment distinction on the Feature Teams? Mostly it is testing, from what I could tell. Many teams have the development branch running on a test instance that they, well, test against. Then, as things stabilize they merge to master via Gerrit.
(Feature Teams members, please correct me if I'm wrong anywhere above, generally: specifics may differ, of course, but is the overall notion accurate?)
How do we encourage a more unified development, testing, and deployment process across all WMF teams and community members (sometimes they are one and the same)?
This issue should also be looked at with betalabs in mind; how can we utilize betalabs to be the best test environment it can be? What code should be running there and how often should it be updated? What tests should be automatically run against it? These questions are almost out of scope of this thread, but a solution to this thread should also have an eye towards betalabs.
Thanks,
Greg
On 03/12/2013 01:44 PM, Greg Grossmeier wrote:
One of the things that became apparent is that the Features Teams (eg: E2, E3, Visual Editor) tend to have separate branches that they develop on and only merge to master closer to their deploy windows. There is some variation among them, of course, but the general idea stands.
We should clarify whether we're talking about core, extensions, or both.
To my knowledge, E3 does not have any server-side branches for core. In other words, we have what Gerrit calls topics, but not what Gerrit calls branches. Even for big changes like a major update to core login and account creation (https://gerrit.wikimedia.org/r/#/c/30637/), it's still just a big single Gerrit change on master.
For extensions, for the most part we also use master and topics. However, there was recently a notable exception for v2 (a major change) of https://www.mediawiki.org/wiki/Extension:GettingStarted .
This contrasts with the way the WMF Platform team develops; we predominately commit to master as we develop with any new features set as disabled in a config until it is ready. The Features Teams also use the "disabled in a config until ready" bit, of course.
Right, the loging/creation change (Gerrit link above) is using that kind of gating (as well as a query-string override).
What is the reasoning for the use of this development/deployment distinction on the Feature Teams? Mostly it is testing, from what I could tell. Many teams have the development branch running on a test instance that they, well, test against. Then, as things stabilize they merge to master via Gerrit.
In the case of the GettingStarted extension v2, we used a feature branch because it was a user-facing change that took a few weeks to get ready for deployment. Because it was user-facing, there were interactions, and we wanted to test it, we wanted to do it in one shot.
Matt Flaschen
On Tue, Mar 12, 2013 at 2:21 PM, Matthew Flaschen mflaschen@wikimedia.orgwrote:
In the case of the GettingStarted extension v2, we used a feature branch because it was a user-facing change that took a few weeks to get ready for deployment. Because it was user-facing, there were interactions, and we wanted to test it, we wanted to do it in one shot.
If there were a convenient mechanism to do it, this is a great example of something I would like to see enabled on beta labs during development but disabled in production until ready.
On 03/12/2013 04:43 PM, Chris McMahon wrote:
On Tue, Mar 12, 2013 at 2:21 PM, Matthew Flaschen mflaschen@wikimedia.orgwrote:
In the case of the GettingStarted extension v2, we used a feature branch because it was a user-facing change that took a few weeks to get ready for deployment. Because it was user-facing, there were interactions, and we wanted to test it, we wanted to do it in one shot.
If there were a convenient mechanism to do it, this is a great example of something I would like to see enabled on beta labs during development but disabled in production until ready.
The problem is, gating can become a maintenance burden when you're making larger changes. You have a long strings of code in if/else blocks, sometimes necessary files.
In such cases, I think branching might be better. In our case, we tested on http://toro.wmflabs.org/wiki/Main_Page . If it were considered alright to test branches on Beta labs, that would have been an option.
Matt Flaschen
wikitech-l@lists.wikimedia.org