>a) aim to track total users who enable/disable Media Viewer, rather than just events
>b) switch to a 3-state preference setting: enabled / disabled / default
>c) try to measure the total number of users in each group (instead of daily events)

I assume we are talking about logging stuff for logged in users and not logged in users. 
If that is the case this strategy will dwarf editor data with reader data, in the RFC most concerns seem to be about editor workflow. Could we sample only logged-in users? That way we will get a sample of data that better represents editor's interaction with media viewer and it will be a lot easier to quantify our results.


>Once we figure out a practical way to implement this request together, we will present it to community members to >discuss its impact on the RfC, and agree on acceptance criteria (e.g. keep Media Viewer enabled by default unless >a 51% majority of users disables the feature on Enwiki by September 30?).

Perhaps if we want the analytics to impact the RFC we should work with the community to come up with a survey strategy they find aceptable? 









On Tue, Jul 15, 2014 at 11:59 PM, Fabrice Florin <fflorin@wikimedia.org> wrote:
Dear Analytics Team,

As discussed with Toby, Dario and Aaron, we would be grateful for your guidance to collect user preference data for Media Viewer (1), which is being challenged in RfCs on English (2) and German (3) Wikipedias, as well as on Wikimedia Commons (4). 

The English RfC closed last week, requesting that we disable Media Viewer right away. The multimedia team, in consultation with community and legal teams, is not comfortable disabling the feature on the basis of this small RfC, which we believe does not accurately represent the views of the millions of users whom we serve. 

To address this issue, we would like to conduct a wider outreach to collect more accurate data about user preferences than either this RfC or our optional surveys can provide. We propose to develop a more prominent viewing options panel (5) that would make it very easy to switch quickly to your preferred viewing mode, as shown in this prototype (6). Right after launch of this new feature, all users would be shown this prominent panel and asked to select their favorite viewing option. Once we have collected and analyzed that data over a period of a month or two, we would be able to make more informed decisions with our community on whether or not to keep the feature enabled -- based on actual responses from all users, rather than speculation. 

With your help, we have already developed a basic dashboard that tracks Media Viewer opt-in/out events (8), based on clicks on the less prominent ‘Disable’ feature at the bottom of the metadata panel. We propose to modify this dashboard to support this new initiative, as outlined in this ticket (7), with these changes:
a) aim to track total users who enable/disable Media Viewer, rather than just events
b) switch to a 3-state preference setting: enabled / disabled / default
c) try to measure the total number of users in each group (instead of daily events)

Also note that most users would start in the default state, showing them Media Viewer, since it is enabled by default — but after 10 or so image views, the viewing options panel would appear automatically, asking them to either enable or disable the feature. After they have made their selection, the panel would remain accessible (but much less prominent) and they can still switch state (but can't go back to default).

We could use your guidance on the right way of logging total logged-out users, so we can use them as a basis for percentages: should we log enabled/disabled state on some specified event/action: page load? thumbnail click? first site visit of the day? — or do you recommend another method? We would want to collect this data for a month or two, so we don’t only capture the immediate responses from the most active users, but also those of less frequent visitors.

We would also appreciate your comments on the specific tickets (5) and (6), with any recommendations for improvement. Keep in mind that we would like to respond quickly to our community’s concerns and have limited resources, so we would prefer to get this work done in the next week or two. And it will take a couple months to develop, release and collect enough data -- so even if we start tomorrow, we may not have conclusive data to share until September. So time is of the essence. :)

Gergo is spearheading this project, and may have more technical questions for you. But I wanted to give an overview from a product perspective, before we dive in to the implementation details. Once we figure out a practical way to implement this request together, we will present it to community members to discuss its impact on the RfC, and agree on acceptance criteria (e.g. keep Media Viewer enabled by default unless a 51% majority of users disables the feature on Enwiki by September 30?).

Thanks in advance for your advice. We look forward to working with you soon on this important project, which could impact other features now in development.

All the best,



Fabrice


(1) https://www.mediawiki.org/wiki/Multimedia/About_Media_Viewer
(2) https://en.wikipedia.org/wiki/Wikipedia:Media_Viewer/June_2014_RfC
(3) https://de.wikipedia.org/wiki/Wikipedia:Meinungsbilder/Medienbetrachter
(4) https://commons.wikimedia.org/wiki/Commons:Requests_for_comment/Media_Viewer_software_feature
(5) https://wikimedia.mingle.thoughtworks.com/projects/multimedia/cards/787
(6) http://pauginer.github.io/prototypes/media-viewer/desk/disabling-settings/index.html
(7) https://wikimedia.mingle.thoughtworks.com/projects/multimedia/cards/793


_______________________________

Fabrice Florin
Product Manager, Multimedia
Wikimedia Foundation





_______________________________________________
Analytics mailing list
Analytics@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/analytics