Hello, Hovercards, currently a beta feature, is nearing release. [0] We'd like to encourage communities to adopt Hovercards as a default option for logged-out readers. [1] Hovercards provide a preview of any linked article, giving readers a quick understanding of a related article without leaving the current page. The Reading web team recently completed a series of A/B tests to gather information on how Hovercards are used.
* April 2015 - Catalan and Greek A/B test results [2] * Summer/Fall 2016 Hungarian, Italian, and Russian Wikipedia A/B test results [3] * 2016 Reading team UX research [4]
As you can see the results of the tests were generally positive. A few key highlights:
* During the Catalan and Greek A/B test, 79% of respondents either Agreed or Strongly Agreed that Hovercards were easy to use, 76% of respondents either Agreed or Strongly Agreed that Hovercards were useful for their needs. * Users are viewing approximately 0.99 hovercards per session, and interacting with approximately 31% more pages each session. [5] * The disable rate was very low: the rate of clicking the settings cog for Hovercards was 0.02% for Hungarian, 0.034% for Italian, and 0.016% for Russian Wikipedia. The rates for disabling the feature were even lower. * In our qualitative tests, 13 out of 15 questionnaire participants reported positive experiences with Hovercards.
== Rollout plan == The next step for Hovercards is to develop a plan for rolling the feature out to all projects. We have created a draft proposal for which projects to discuss Hovercards with and approximate timeline for deployments. [6]
The Reading Web team will be reaching out to the communities in the order listed above to discuss how to implement the feature. We want to enable the feature by default for logged-out users and we will respect the current beta feature preferences for logged-in users.
Questions and feedback welcome at Beta Features/Hovercards. [7]
Thank you for your time.
[0] https://www.mediawiki.org/wiki/Beta_Features/Hovercards [1] Logged-out users would see Hovercards by default on wikis that opt-in. Logged-out users may turn Hovercards off via the cog-icon displayed at the bottom of each Hovercard. Logged-in users see Hovercards if they have enabled the feature. New accounts created after the launch of Hovercards would have it on by default (mimicking the default for logged-out users) with the option to disable in Special:Preferences. [2] https://www.mediawiki.org/wiki/Beta_Features/Hovercards/GreekCatalanTest [3] https://www.mediawiki.org/wiki/Beta_Features/Hovercards/2016_A/B_Tests [4] https://www.mediawiki.org/wiki/Wikimedia_Research/Design_Research/Reading_Te... [5] With page interactions defined as the sum of average hovercards viewed per session and average pages viewed per session [6] https://www.mediawiki.org/wiki/Beta_Features/Hovercards#Rollout_Plan [7] https://www.mediawiki.org/wiki/Talk:Beta_Features/Hovercards
Yours, Chris Koerner Community Liaison - Discovery Wikimedia Foundation
I'm glad to see progress on this front. I do however have one concern, and that is that multiple of the issues identified in A/B testing of 2015, are still open. I know there has been progress in several areas (esp. some of the objections raised in the en.wp consultation of early 2016), but if issues were important enough to mention them in the Greek Catalan report, and 80% of those issues are still open today, then this seems worthy evaluating to me.
I also see little management and sorting of issues on the Phabricator dashboard, which gives me no ability to assess potential impact and make distinction between bugs that might impact the release, vs things that are enhancements, non-issues etc etc.
DJ
On Thu, Nov 17, 2016 at 6:00 PM, Chris Koerner ckoerner@wikimedia.org wrote:
Hello, Hovercards, currently a beta feature, is nearing release. [0] We'd like to encourage communities to adopt Hovercards as a default option for logged-out readers. [1] Hovercards provide a preview of any linked article, giving readers a quick understanding of a related article without leaving the current page. The Reading web team recently completed a series of A/B tests to gather information on how Hovercards are used.
- April 2015 - Catalan and Greek A/B test results [2]
- Summer/Fall 2016 Hungarian, Italian, and Russian Wikipedia A/B test
results [3]
- 2016 Reading team UX research [4]
As you can see the results of the tests were generally positive. A few key highlights:
- During the Catalan and Greek A/B test, 79% of respondents either Agreed
or Strongly Agreed that Hovercards were easy to use, 76% of respondents either Agreed or Strongly Agreed that Hovercards were useful for their needs.
- Users are viewing approximately 0.99 hovercards per session, and
interacting with approximately 31% more pages each session. [5]
- The disable rate was very low: the rate of clicking the settings cog for
Hovercards was 0.02% for Hungarian, 0.034% for Italian, and 0.016% for Russian Wikipedia. The rates for disabling the feature were even lower.
- In our qualitative tests, 13 out of 15 questionnaire participants
reported positive experiences with Hovercards.
== Rollout plan == The next step for Hovercards is to develop a plan for rolling the feature out to all projects. We have created a draft proposal for which projects to discuss Hovercards with and approximate timeline for deployments. [6]
The Reading Web team will be reaching out to the communities in the order listed above to discuss how to implement the feature. We want to enable the feature by default for logged-out users and we will respect the current beta feature preferences for logged-in users.
Questions and feedback welcome at Beta Features/Hovercards. [7]
Thank you for your time.
[0] https://www.mediawiki.org/wiki/Beta_Features/Hovercards [1] Logged-out users would see Hovercards by default on wikis that opt-in. Logged-out users may turn Hovercards off via the cog-icon displayed at the bottom of each Hovercard. Logged-in users see Hovercards if they have enabled the feature. New accounts created after the launch of Hovercards would have it on by default (mimicking the default for logged-out users) with the option to disable in Special:Preferences. [2] https://www.mediawiki.org/wiki/Beta_Features/Hovercards/ GreekCatalanTest [3] https://www.mediawiki.org/wiki/Beta_Features/Hovercards/2016_A/B_Tests [4] https://www.mediawiki.org/wiki/Wikimedia_Research/ Design_Research/Reading_Team_UX_Research/Hovercards_Usability [5] With page interactions defined as the sum of average hovercards viewed per session and average pages viewed per session [6] https://www.mediawiki.org/wiki/Beta_Features/Hovercards#Rollout_Plan [7] https://www.mediawiki.org/wiki/Talk:Beta_Features/Hovercards
Yours, Chris Koerner Community Liaison - Discovery Wikimedia Foundation _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Hi all, hi DJ
Thank you for your comments! Those are some valid concerns. A few clarifications and related updates:
1. Current bugs: We have been tracking all current issues from the Greek/Catalan test. The page for the results also has some updates on each task here https://www.mediawiki.org/wiki/Beta_Features/Hovercards/GreekCatalanTest#Major_Bugs_Reported. I'd also like to point out that we do not plan on releasing the feature before all these issues are resolved (except for the ones marked for future implementation) and thoroughly tested. 2. Expected Functionality: The list of acceptance criteria before release, in terms of functionality, can be found here https://www.mediawiki.org/wiki/Beta_Features/Hovercards/Functionality. All criteria necessary prior to release is noted on this page. The page also has some ideas/new functionality which are marked for future iterations. 3. Phabricator: since we are refactoring the hovercards code, we have decided to stall current bugs until after the refactor - we are hoping that most of them will no longer be applicable and can be immediately closed, but we're currently keeping them around to account for any edge cases that might occur. The phabricator tasks currently lined up/in progress are the following: 1. Refactoring and accounting for basic functionality: https://phabricator.wikimedia.org/T149801 https://phabricator.wikimedia.org/T149801 2. Visual design: https://phabricator.wikimedia.org/T150814 3. Accounting for edge cases where previews are not available: https://phabricator.wikimedia.org/T151054 4. Building preferences for logged-in users: https://phabricator.wikimedia.org/T146889 5. Identifying service that will allow optimal summaries: https://phabricator.wikimedia.org/T113094, https://phabricator.wikimedia.org/T151055, https://phabricator.wikimedia.org/T141766 https://phabricator.wikimedia.org/T141766 and https://phabricator.wikimedia.org/T141651 6. Allowing hovercards to use RESTbase API endpoint: https://phabricator.wikimedia.org/T145429
Cheers,
Olga
On Tue, Nov 22, 2016 at 12:50 PM Derk-Jan Hartman < d.j.hartman+wmf_ml@gmail.com> wrote:
I'm glad to see progress on this front. I do however have one concern, and that is that multiple of the issues identified in A/B testing of 2015, are still open. I know there has been progress in several areas (esp. some of the objections raised in the en.wp consultation of early 2016), but if issues were important enough to mention them in the Greek Catalan report, and 80% of those issues are still open today, then this seems worthy evaluating to me.
I also see little management and sorting of issues on the Phabricator dashboard, which gives me no ability to assess potential impact and make distinction between bugs that might impact the release, vs things that are enhancements, non-issues etc etc.
DJ
On Thu, Nov 17, 2016 at 6:00 PM, Chris Koerner ckoerner@wikimedia.org wrote:
Hello, Hovercards, currently a beta feature, is nearing release. [0] We'd like
to
encourage communities to adopt Hovercards as a default option for logged-out readers. [1] Hovercards provide a preview of any linked
article,
giving readers a quick understanding of a related article without leaving the current page. The Reading web team recently completed a series of A/B tests to gather information on how Hovercards are used.
- April 2015 - Catalan and Greek A/B test results [2]
- Summer/Fall 2016 Hungarian, Italian, and Russian Wikipedia A/B test
results [3]
- 2016 Reading team UX research [4]
As you can see the results of the tests were generally positive. A few
key
highlights:
- During the Catalan and Greek A/B test, 79% of respondents either Agreed
or Strongly Agreed that Hovercards were easy to use, 76% of respondents either Agreed or Strongly Agreed that Hovercards were useful for their needs.
- Users are viewing approximately 0.99 hovercards per session, and
interacting with approximately 31% more pages each session. [5]
- The disable rate was very low: the rate of clicking the settings cog
for
Hovercards was 0.02% for Hungarian, 0.034% for Italian, and 0.016% for Russian Wikipedia. The rates for disabling the feature were even lower.
- In our qualitative tests, 13 out of 15 questionnaire participants
reported positive experiences with Hovercards.
== Rollout plan == The next step for Hovercards is to develop a plan for rolling the feature out to all projects. We have created a draft proposal for which projects
to
discuss Hovercards with and approximate timeline for deployments. [6]
The Reading Web team will be reaching out to the communities in the order listed above to discuss how to implement the feature. We want to enable
the
feature by default for logged-out users and we will respect the current beta feature preferences for logged-in users.
Questions and feedback welcome at Beta Features/Hovercards. [7]
Thank you for your time.
[0] https://www.mediawiki.org/wiki/Beta_Features/Hovercards [1] Logged-out users would see Hovercards by default on wikis that
opt-in.
Logged-out users may turn Hovercards off via the cog-icon displayed at
the
bottom of each Hovercard. Logged-in users see Hovercards if they have enabled the feature. New accounts created after the launch of Hovercards would have it on by default (mimicking the default for logged-out users) with the option to disable in Special:Preferences. [2] https://www.mediawiki.org/wiki/Beta_Features/Hovercards/ GreekCatalanTest [3]
https://www.mediawiki.org/wiki/Beta_Features/Hovercards/2016_A/B_Tests
[4] https://www.mediawiki.org/wiki/Wikimedia_Research/ Design_Research/Reading_Team_UX_Research/Hovercards_Usability [5] With page interactions defined as the sum of average hovercards
viewed
per session and average pages viewed per session [6] https://www.mediawiki.org/wiki/Beta_Features/Hovercards#Rollout_Plan [7] https://www.mediawiki.org/wiki/Talk:Beta_Features/Hovercards
Yours, Chris Koerner Community Liaison - Discovery Wikimedia Foundation _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
wikitech-l@lists.wikimedia.org