I'll be sure to share with the rest of the team before we move forward on anything.

I think doing clean-up might be premature as the WikiGrok UI may still have some significant UI changes coming soon. We had a meeting with Leila last week before I left town and discussed some challenges with implementing a usable aggregation model for WikiGrok version B. The main issue is that it is unclear how to treat null responses in the aggregation model. If they are treated as a purely negative response (i.e. the equivalent of saying that a tag is false), the test data doesn't converge on an accurate response. If they are completely ignored, all possible tags eventually tend towards true. The null response can be weighted as being somewhere in between (between -1 and 0), but how to weight it will likely depend on the campaign. For example, some campaigns have mutually exclusive tags (live album vs. studio album) which would need a heavy weighting for null, while other campaigns have non-exclusive tags (tv actor and film actor) which would need a light weighting for null. I have a meeting scheduled with Analytics and Design this afternoon to talk about ideas for how to address this. It's possible that we may decide to implement a new UI for WikiGrok that is a hybrid of the A and B designs or modify the WikiGrok v B interface to allow the user to submit more information regarding the non-positive responses. This is just a preliminary meeting though. Whatever ideas we come up with I'll be sure to share with the rest of the team before we move forward on anything. If there are some light-weight things that can be cleaned-up without too much refactoring, that would be fine, but I would suggest avoiding any significant refactoring (other than the WikiGrok version consolidation) until we have a clean idea of how we're going to move forward with the UI. Kaldari On Thu, Feb 19, 2015 at 7:46 AM, Sam Smith <samsmith(a)wikimedia.org> wrote: