On Mon, Jun 9, 2014 at 11:20 AM, Aaron Halfaker <ahalfaker(a)wikimedia.org>
wrote:
Hmm... It's hard to evaluate your strategy without
more context. Why are
you limiting your query to users with more than 10k lifetime edits? Are
your trying to generate a proportion of a subset of users? If so, what's
the denominator?
I ran the query with various editcount limits (0, 1K, 10K, 100K, 1M); the
goal was to see whether power users are affected differently. (They are,
users with 100K+ edits have the highest opt-out rates.) I am basically
taking the ratio of ( <users with more than X total edits who have been
active since the rollout and disabled MediaViewer> / <users with more than
X total edits who have been active since the rollout> ) which in this case
should be ( <count(*) for up_value = ' ' group> / ( <count(*) for
up_value
= ' ' group> + <count(*) for up_value is NULL ' group>) )
Also, opt-out rates tend to be low no matter how
obvious and desired they
are. If the goal of this analysis is to find out if opt-out rates are high
(or low), then I'd recommend comparing them with opt-out rates for another
feature.
Any tip for what that feature should be? Are there features for which
opt-out rates have been recorded a few days after they have been deployed?
(I imagine one problem is that opt-out happens slowly - although in the
past two days our opt-out count grew about 20%, but the ratio to active
users has been remarkably stable.)