My $0.02: planning to determine high-level goals for feature flags, off the top of my head:

- As a developer, I can regularly merge my work into master while developing a feature, without exposing it to users or causing side effects (IOW users have to opt-in for experimental features)

- As a product owner, I can regularly ship builds with minor fixes while developing major features in parallel without paying a high synchronization cost between feature & maintenance development

Then, each platform can chart out a road map w/ estimates for how long it might take.  There's a fair amount to consider here, at least IMO we should try to knock out a couple of birds w/ this stone (i.e. those "global state" and "modularization" things I mentioned earlier).

On Thu, Mar 12, 2015 at 5:01 PM, Dan Garry <dgarry@wikimedia.org> wrote:
So it sounds like we have agreement on using feature flags for our features, with the aim of increasing our cadence.

What are our next steps now?

Dan

On 11 March 2015 at 14:01, Dmitry Brant <dbrant@wikimedia.org> wrote:
Yeah, I'll make sure to listen to the podcast, but generally I'm on the side of feature flags.



On Wed, Mar 11, 2015 at 4:58 PM, Corey Floyd <cfloyd@wikimedia.org> wrote:
Yeah - after listening to the podcast I was on the fence about it too… I think they bring up good points on the tradeoffs of both (even the hosts were split on their preference at the end).

I was initially attracted towards the git methodology as it gets these business decisions out of the code base - but I also think that having these flags that work at runtime (from a debug menu) is really valuable for testing purposes.

One thing I am not convinced on is that there will be "less work at the end" when integrating features into a release with the feature flag methodology - I feel like that could still be a problem if you are not merging/rebasing. They discussed this in the podcast almost as an aside - but it is entirely possible to develop conflicting features using feature flags if developers do not coordinate (I feel this is really an orthogonal issue to enabling features).

Still - I think the feature flag solution has promise - but I am way more interested in it if we do it as run time option to get the testing/QA benefits.


On Wed, Mar 11, 2015 at 4:08 PM, Bernd Sitzmann <bernd@wikimedia.org> wrote:
I have not listened to the podcast yet, but my opinion is similar to what Brian described. 
So, +1 for feature flags.

Bernd

On Wed, Mar 11, 2015 at 1:40 PM, Brian Gerstle <bgerstle@wikimedia.org> wrote:
Listened to it this morning, gotta say I think I'm on "Team Flag," if it's done well.  IMO we should try one (or both) and see how it goes.  Here's my take on each approach:

Branches are cheap to implement, but come at the a potentially high cost if you don't continually rebase and try to keep the work scope as small as possible.  At a previous job we used git submodules as pseudo feature branches.  There were common problems w/ dependencies between branches and between branches and the main repo, which as the hosts mention are often pushed later in the process.

Flags are more expensive to implement up front, but allow for truly continuous integration and delivery—as well as the potential for gradual rollout.  IMO it could also lead to better architected code since feature flags require you to codify the boundaries between the "platform" and the features (and between the features themselves).  You also need to limit global state and have good test coverage (which we should do anyway) in order to keep undesired side-effects to a minimum.  My previous job also switched to this model and was able to improve their release cadence and sync between features (IIRC, don't have any concrete evidence to back it up unfortunately).


On Tue, Mar 10, 2015 at 11:16 PM, Corey Floyd <cfloyd@wikimedia.org> wrote:
As luck would have it, a very good tech podcast I subscribe to recently discussed this subject. It is a pretty good listen and discusses trade offs for both feature flags and branching. 


(Topic is covered in the 2nd half of the show)


On Monday, March 9, 2015, Dan Duvall <dduvall@wikimedia.org> wrote:
One very simple model that I think works well with distributed software (and should play somewhat nice with Gerrit) is to branch per minor release (assuming somewhat semantic versioning). In this model, master can continue to serve as an integration branch, bug fixes are developed against the release branch and merged following each patch release, and features are developed against master per usual. With Gerrit in the mix, you're essentially left with a two-stage merge for bug fixes which can be a bit of extra work to get them merged into master, but at least the pipeline is greased for getting them released. 

I can't say whether this would fit your team's workflow and, as Corey mentioned, there are many different branching models to consider, each with its own focus and drawbacks.[1][2] The one I've outlined above is geared more for stability and maintenance but can probably be tweaked for more frequent releases of features as well. I'm happy to brainstorm further.



On Mon, Mar 9, 2015 at 2:30 PM, Corey Floyd <cfloyd@wikimedia.org> wrote:
Feature flags would help, but they also add an extra development investment to make sure all features are engineered in a way that a flag can shut them off without other bad things happening (not necessarily a bad thing, but require more effort).

Another route to go is to manage this in git using the branches. There are several methodologies for this, and your branching strategy will depend mostly on how the team wants to operate. Pretty much all of them boil down to NOT merging features into master that are not going to be deployed after the current sprint.




On Mon, Mar 9, 2015 at 5:19 PM, Tomasz Finc <tfinc@wikimedia.org> wrote:
Excited to see this. In order for this to be successful we'll want to
developer a health dashboard for the app and set alarming for it
whenever we dip above/below certain thresholds. One of the challenges
that you face when releasing often is that the amount of attention you
keep during small iterative releases. It's easy to keep very focused
for 2-3 releases but after that attention can drift to just the major
releases. And while it's great to read reviews and find out a
subjective metric of how we're doing we need to get in front of it
with objective metrics.

Thus having an app health dashboard showcasing: search, readership,
editing, etc can easily show you if you've had any regressions. These
would not only be useful for small bug fix releases but would also
help validate our major product releases.

I'll leave it with you guys to define what metrics are necessary to
define a healthy app.

--tomasz

On Mon, Mar 9, 2015 at 1:31 PM, Dan Garry <dgarry@wikimedia.org> wrote:
> Hi everyone,
>
> There is a long time between production release on Android. The reason for
> this is because we have featured merged into master that are sound from an
> engineering standpoint, but aren't quite ready for release yet. These
> features often block production releases.
>
> As product owner, pushing out regular bug fix builds would make me very
> happy! But there is a requirement that we are able to not push out features
> that are merged but not ready for production yet. This can probably me
> managed by feature flags.
>
> Dan Duvall (on cc) from Release Engineering will consult to help Dmitry and
> Bernd figure out what their process should be for maximum effectiveness.
>
> Thanks!
>
> Dan
>
> --
> Dan Garry
> Associate Product Manager, Mobile Apps
> Wikimedia Foundation
>
> _______________________________________________
> Mobile-l mailing list
> Mobile-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/mobile-l
>

_______________________________________________
Mobile-l mailing list
Mobile-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mobile-l



--
Corey Floyd
Software Engineer 
Mobile Apps / iOS 
Wikimedia Foundation

_______________________________________________
Mobile-l mailing list
Mobile-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mobile-l




--
Dan Duvall
Automation Engineer


--
Corey Floyd
Software Engineer 
Mobile Apps / iOS 
Wikimedia Foundation


_______________________________________________
Mobile-l mailing list
Mobile-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mobile-l




--
EN Wikipedia user page: https://en.wikipedia.org/wiki/User:Brian.gerstle
IRC: bgerstle

_______________________________________________
Mobile-l mailing list
Mobile-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mobile-l




_______________________________________________
Mobile-l mailing list
Mobile-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mobile-l



_______________________________________________
Mobile-l mailing list
Mobile-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mobile-l




--
Dan Garry
Associate Product Manager, Mobile Apps
Wikimedia Foundation

_______________________________________________
Mobile-l mailing list
Mobile-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mobile-l




--
EN Wikipedia user page: https://en.wikipedia.org/wiki/User:Brian.gerstle
IRC: bgerstle