The (rough) epic definition is already on Phabricator: https://phabricator.wikimedia.org/T98970.  

I've defined some metrics there already, but admittedly—and thanks for calling us out on this—we don't really have baselines*.  I think there are some feasible ways to get a rough starting point, which I can brainstorm w/ the team.  We were planning (or I should say, I was hoping) to gather more code metrics anyway, so I'm glad to have an excuse to hook it up sooner ;-).

FWIW, I also think having patches tested as part of code review would also work as a sufficient definition of success.  Our goal here is to do that as quickly, easily, and cheaply as possible so we can get back to focusing on the app.

* I think it's fair to say that the coverage at point of migration was already low (~10% based on my Travis-covered fork) and hasn't changed much.

On Thu, Jul 23, 2015 at 2:04 PM, Greg Grossmeier <greg@wikimedia.org> wrote:

Brian and the Reading team:

On Wed, Jul 22, 2015 at 3:40 AM, Brian Gerstle <bgerstle@wikimedia.org> wrote:

By using GitHub with Travis CI, the team believes it will work faster, improve testing, grow developer confidence in making code changes, and, most importantly, deploy fewer bugs to production.

Given that, I am requesting you/your team create a set of KPIs to review in 3 or 4 months to determine if this change had the intended outcome. It's hard to make these things quantifiable as useful KPIs (that prevent eg gaming the system) but I think it'd be a good exercise for your team given your team's decision making process thus far.

Please post those KPIs somewhere public and trackable (wiki or phab).

Thank you,

Greg

--

Greg Grossmeier
Release Team Manager

--

EN Wikipedia user page: https://en.wikipedia.org/wiki/User:Brian.gerstle
IRC: bgerstle