Hi all - some interesting analysis on the share-a-fact feature from the mobile team.
-Toby
Begin forwarded message:
From: Adam Baso abaso@wikimedia.org Date: May 21, 2015 at 12:05:29 PDT To: mobile-l mobile-l@lists.wikimedia.org Subject: [WikimediaMobile] Share a Fact Initial Analysis
Hello all,
We’ve been looking at some initial results from the Share a Fact feature introduced on the Wikipedia apps for Android and iOS in its basic "minimal viable product" implementation. Here’s some analysis, using data from one day (20150512) with respect to the latest stable versions of the apps (2.0-r-2015-04-23 on Android and 4.1.2 on iOS) for that day.
On iOS, when a user initiates the first step of the default sharing workflow - tapping the up-arrow box share button (6,194 non-highlighting instances for the day under question) - about 11.7% of the time it yielded successful sharing.
On Android, it’s not possible to easily tell when the sharing workflow was carried through to successful share, but we anticipate the Android success rate is currently much higher, as general engagement percentage up to the point of picking an app for sharing is higher on Android than on iOS.
On Android, when presented with the share card preview, 28.0% of the time the ‘Share as image’ button was tapped and 55.5% of the time the 'Share as text' button was tapped, whereas on iOS it was 8.4% ‘Share as image’ and 16.8% ‘Share as text’.
The forthcoming 4.1.4 version of the iOS app will relax its default sharing snippet generation rules and be more like the Android version in that respect. We anticipate this will result in higher engagement with both the ‘Share as image’ and ‘Share as text’ buttons on iOS, and we should be able to verify this once the 4.1.4 iOS version is released and generally adopted (usually takes 4-5 days after release; the 4.1.4 release isn’t released yet).
On the Android app the ‘Share’ option is located on the overflow menu, not as part of the main set of UI buttons. This potentially increases the likelihood of Android users being primed to step through the workflow. On the iOS app, the share button (up-arrow box) is plainly visible from the main UI and not an overflow menu, and this probably creates a different priming dynamic for the iOS demographic.
When users on iOS tapped on the ‘Share as image’ or ‘Share as text’ buttons, there is a pretty sharp drop off at the next stage - the system sharesheet. Once the sharesheet was presented to iOS users, 41.6% of the time it resulted in active abandonment. We believe this probably has something to do with the relatively small set of default apps listed on the sharesheet and the extra work involved with exposing additional social apps for sharing in that context. As with the Android app, the labels of ‘Share as image’ and ’Share as text’ may also pose something of a hurdle at least for first time users of the feature. To this end, there is an onboarding tutorial planned at least on Android.
For a one hour period (2015051201) there were about 100 pageviews in some sense attributable to Share a Fact using a provenance parameter available on the latest stable versions of the apps at that time; this may slightly overstate the number of pageviews attributable to the two specific apps reviewed in this analysis, but probably not too much (n.b., previously a different source parameter was used than the new wprov provenance parameter). Pageviews are not the sole motivation for the feature, but following the trendline over the long run should be interesting. Impact on social media and the destinations of shares is a little harder to capture directly, but https://twitter.com/search?f=realtime&q=%40wikipedia%20-%40itzwikipedia%... gives one a sense about image shares, at least.
A couple potential options for increasing sharing include:
** Trying to add support for sharing to the Photos app on iOS. People may be interested in using images from the Photos apps for various workflows, as Dan Garry has noted.
** Offering a more concise app picklist, in particular explicitly adding the native OS app components (namely, Twitter and Facebook, and as mentioned, Photos if possible), with an option to expose the sharesheet for additional options if necessary. This is probably also somewhat confined to iOS, although conceivably a similar approach could be possible on Android. On Android the full list of applications in its equivalent of the sharesheet is by default readily available to the user, though.
** On Android, exposing the diagonal arrow share button on the main interface akin to how the iOS version of the app shows the up-arrow share button. This may introduce more opportunities for sharing (and thus numbers of abandons would go up in tandem with numbers of shares), but would also partially clutter the interface and probably increase abandon. A controlled experiment may be useful for observing the impact of such an approach.
As a point of reference, for the app versions in scope for this analysis over a single day, there appeared to be approximately 3.78 million Wikipedia for Android pageviews and 1.19 Wikipedia Mobile for iOS app pageviews. There were about 6.73 million app pageviews on the “modern” versions of these apps total for this particular day, meaning there were about 1.75 million pageviews on other modern versions of the app.
Examination of the categories of successful shares on iOS showed the following distributions:
Images: 48.5% messaging 25.5% sharesheet copy 22.9% social 1.8% productivity 0.9% reading
Text: 53.6% messaging 31.9% sharesheet copy 7.1% social 5.4% reading 2.0% productivity
Here were some queries used in the analysis:
== SHARE A FACT ATTRIBUTABLE PAGEVIEWS FOR ONE HOUR ==
select wprov, uri_host, count(*) from (select x_analytics_map['wprov'] as wprov, uri_host from webrequest where year = 2015 and month = 5 and day = 12 and hour = 1 and is_pageview = true and uri_host like '%.wikipedia.org' and x_analytics_map['wprov'] is not null) t group by wprov, uri_host;
== PAGE VIEWS FOR THE DAY FOR THE “MODERN” VERSIONS OF THE APPS ==
SELECT user_agent, count(*) FROM wmf.webrequest tablesample(BUCKET 1 OUT OF 100 ON rand()) WHERE YEAR = 2015 AND MONTH = 5 AND DAY = 12 AND is_pageview = TRUE AND lower(uri_host) like '%.wikipedia.org' AND user_agent like 'WikipediaApp%' GROUP BY user_agent;
== HIGHLIGHTING SESSION CASE FOR SPECIFIC VERSIONS OF THE APPS == select CASE WHEN t2.userAgent LIKE 'WikipediaApp/2.0-r-2015-04-23%' THEN 'Android' WHEN t2.userAgent LIKE 'WikipediaApp/4.1.2%' THEN 'iOS' END AS 'ua', t1.event_action, t1.event_sharemode, t1.event_target, count(*) from MobileWikiAppShareAFact_11331974 t1 inner join MobileWikiAppShareAFact_11331974 t2 on t1.event_shareSessionToken = t2.event_shareSessionToken where t1.timestamp > '20150512' and t1.timestamp < '20150513' and t2.timestamp > '20150512' and t2.timestamp < '20150513' and t1.event_action != 'highlight' and t2.event_action = 'highlight' and (t2.userAgent like 'WikipediaApp/2.0-r-2015-04-23%' or t2.userAgent like 'WikipediaApp/4.1.2%') group by ua, t1.event_action, t1.event_sharemode, t1.event_target;
== NON-HIGHLIGHTING SESSION CASE FOR SPECIFIC VERSIONS OF THE APPS == n.b., subtract the highlighting cases from the non-highlighting cases to arrive at the default sharing behavior. Technically, inner joins can be used to do more comprehensive session analysis, but the queries take a long time.
select CASE WHEN userAgent LIKE 'WikipediaApp/2.0-r-2015-04-23%' THEN 'Android' WHEN userAgent LIKE 'WikipediaApp/4.1.2%' THEN 'iOS' END AS 'ua', event_action, event_sharemode, event_target, count(*) from MobileWikiAppShareAFact_11331974 where timestamp > '20150512' and timestamp < '20150513' and (userAgent like 'WikipediaApp/2.0-r-2015-04-23%' or userAgent like 'WikipediaApp/4.1.2%') group by ua, event_action, event_sharemode, event_target;
-Adam _______________________________________________ Mobile-l mailing list Mobile-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mobile-l
Thanks for sharing this, Adam. Aside from engagement/funnel data, the critical question for this feature is: does it bring back eyeballs to the site from social media? It looks like it doesn’t yet, at least not in a substantial way, even with the caveat that App traffic is a very small fraction of total mobile traffic.
Having looked into referrals for this feature before and after comparing them to Twitter’s own engagement analytics (and finding some big discrepancy), you should consider removing spiders/crawlers from the data (see [1]) to avoid inflating pageviews with non-human activity.
I’m a big fan of this feature and look forward to seeing how you guys intend to scale it.
Dario
[1] https://github.com/ewulczyn/wmf/blob/b9f726ee3468852c3fed2780af1d8ac0004eda7... https://github.com/ewulczyn/wmf/blob/b9f726ee3468852c3fed2780af1d8ac0004eda73/mc/oozie/hive_query.sql#L60
On May 21, 2015, at 12:37 PM, Toby Negrin tnegrin@wikimedia.org wrote:
Hi all - some interesting analysis on the share-a-fact feature from the mobile team.
-Toby
Begin forwarded message:
From: Adam Baso <abaso@wikimedia.org mailto:abaso@wikimedia.org> Date: May 21, 2015 at 12:05:29 PDT To: mobile-l <mobile-l@lists.wikimedia.org mailto:mobile-l@lists.wikimedia.org> Subject: [WikimediaMobile] Share a Fact Initial Analysis
Hello all,
We’ve been looking at some initial results from the Share a Fact feature introduced on the Wikipedia apps for Android and iOS in its basic "minimal viable product" implementation. Here’s some analysis, using data from one day (20150512) with respect to the latest stable versions of the apps (2.0-r-2015-04-23 on Android and 4.1.2 on iOS) for that day.
On iOS, when a user initiates the first step of the default sharing workflow - tapping the up-arrow box share button (6,194 non-highlighting instances for the day under question) - about 11.7% of the time it yielded successful sharing.
On Android, it’s not possible to easily tell when the sharing workflow was carried through to successful share, but we anticipate the Android success rate is currently much higher, as general engagement percentage up to the point of picking an app for sharing is higher on Android than on iOS.
On Android, when presented with the share card preview, 28.0% of the time the ‘Share as image’ button was tapped and 55.5% of the time the 'Share as text' button was tapped, whereas on iOS it was 8.4% ‘Share as image’ and 16.8% ‘Share as text’.
The forthcoming 4.1.4 version of the iOS app will relax its default sharing snippet generation rules and be more like the Android version in that respect. We anticipate this will result in higher engagement with both the ‘Share as image’ and ‘Share as text’ buttons on iOS, and we should be able to verify this once the 4.1.4 iOS version is released and generally adopted (usually takes 4-5 days after release; the 4.1.4 release isn’t released yet).
On the Android app the ‘Share’ option is located on the overflow menu, not as part of the main set of UI buttons. This potentially increases the likelihood of Android users being primed to step through the workflow. On the iOS app, the share button (up-arrow box) is plainly visible from the main UI and not an overflow menu, and this probably creates a different priming dynamic for the iOS demographic.
When users on iOS tapped on the ‘Share as image’ or ‘Share as text’ buttons, there is a pretty sharp drop off at the next stage - the system sharesheet. Once the sharesheet was presented to iOS users, 41.6% of the time it resulted in active abandonment. We believe this probably has something to do with the relatively small set of default apps listed on the sharesheet and the extra work involved with exposing additional social apps for sharing in that context. As with the Android app, the labels of ‘Share as image’ and ’Share as text’ may also pose something of a hurdle at least for first time users of the feature. To this end, there is an onboarding tutorial planned at least on Android.
For a one hour period (2015051201) there were about 100 pageviews in some sense attributable to Share a Fact using a provenance parameter available on the latest stable versions of the apps at that time; this may slightly overstate the number of pageviews attributable to the two specific apps reviewed in this analysis, but probably not too much (n.b., previously a different source parameter was used than the new wprov provenance parameter). Pageviews are not the sole motivation for the feature, but following the trendline over the long run should be interesting. Impact on social media and the destinations of shares is a little harder to capture directly, but https://twitter.com/search?f=realtime&q=%40wikipedia%20-%40itzwikipedia%... https://twitter.com/search?f=realtime&q=%40wikipedia%20-%40itzwikipedia%20filter%3Amedia gives one a sense about image shares, at least.
A couple potential options for increasing sharing include:
** Trying to add support for sharing to the Photos app on iOS. People may be interested in using images from the Photos apps for various workflows, as Dan Garry has noted.
** Offering a more concise app picklist, in particular explicitly adding the native OS app components (namely, Twitter and Facebook, and as mentioned, Photos if possible), with an option to expose the sharesheet for additional options if necessary. This is probably also somewhat confined to iOS, although conceivably a similar approach could be possible on Android. On Android the full list of applications in its equivalent of the sharesheet is by default readily available to the user, though.
** On Android, exposing the diagonal arrow share button on the main interface akin to how the iOS version of the app shows the up-arrow share button. This may introduce more opportunities for sharing (and thus numbers of abandons would go up in tandem with numbers of shares), but would also partially clutter the interface and probably increase abandon. A controlled experiment may be useful for observing the impact of such an approach.
As a point of reference, for the app versions in scope for this analysis over a single day, there appeared to be approximately 3.78 million Wikipedia for Android pageviews and 1.19 Wikipedia Mobile for iOS app pageviews. There were about 6.73 million app pageviews on the “modern” versions of these apps total for this particular day, meaning there were about 1.75 million pageviews on other modern versions of the app.
Examination of the categories of successful shares on iOS showed the following distributions:
Images: 48.5% messaging 25.5% sharesheet copy 22.9% social 1.8% productivity 0.9% reading
Text: 53.6% messaging 31.9% sharesheet copy 7.1% social 5.4% reading 2.0% productivity
Here were some queries used in the analysis:
== SHARE A FACT ATTRIBUTABLE PAGEVIEWS FOR ONE HOUR ==
select wprov, uri_host, count(*) from (select x_analytics_map['wprov'] as wprov, uri_host from webrequest where year = 2015 and month = 5 and day = 12 and hour = 1 and is_pageview = true and uri_host like '%.wikipedia.org http://wikipedia.org/' and x_analytics_map['wprov'] is not null) t group by wprov, uri_host;
== PAGE VIEWS FOR THE DAY FOR THE “MODERN” VERSIONS OF THE APPS ==
SELECT user_agent, count(*) FROM wmf.webrequest tablesample(BUCKET 1 OUT OF 100 ON rand()) WHERE YEAR = 2015 AND MONTH = 5 AND DAY = 12 AND is_pageview = TRUE AND lower(uri_host) like '%.wikipedia.org http://wikipedia.org/' AND user_agent like 'WikipediaApp%' GROUP BY user_agent;
== HIGHLIGHTING SESSION CASE FOR SPECIFIC VERSIONS OF THE APPS == select CASE WHEN t2.userAgent LIKE 'WikipediaApp/2.0-r-2015-04-23%' THEN 'Android' WHEN t2.userAgent LIKE 'WikipediaApp/4.1.2%' THEN 'iOS' END AS 'ua', t1.event_action, t1.event_sharemode, t1.event_target, count(*) from MobileWikiAppShareAFact_11331974 t1 inner join MobileWikiAppShareAFact_11331974 t2 on t1.event_shareSessionToken = t2.event_shareSessionToken where t1.timestamp > '20150512' and t1.timestamp < '20150513' and t2.timestamp > '20150512' and t2.timestamp < '20150513' and t1.event_action != 'highlight' and t2.event_action = 'highlight' and (t2.userAgent like 'WikipediaApp/2.0-r-2015-04-23%' or t2.userAgent like 'WikipediaApp/4.1.2%') group by ua, t1.event_action, t1.event_sharemode, t1.event_target;
== NON-HIGHLIGHTING SESSION CASE FOR SPECIFIC VERSIONS OF THE APPS == n.b., subtract the highlighting cases from the non-highlighting cases to arrive at the default sharing behavior. Technically, inner joins can be used to do more comprehensive session analysis, but the queries take a long time.
select CASE WHEN userAgent LIKE 'WikipediaApp/2.0-r-2015-04-23%' THEN 'Android' WHEN userAgent LIKE 'WikipediaApp/4.1.2%' THEN 'iOS' END AS 'ua', event_action, event_sharemode, event_target, count(*) from MobileWikiAppShareAFact_11331974 where timestamp > '20150512' and timestamp < '20150513' and (userAgent like 'WikipediaApp/2.0-r-2015-04-23%' or userAgent like 'WikipediaApp/4.1.2%') group by ua, event_action, event_sharemode, event_target;
-Adam _______________________________________________ Mobile-l mailing list Mobile-l@lists.wikimedia.org mailto:Mobile-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mobile-l https://lists.wikimedia.org/mailman/listinfo/mobile-l
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
Thanks. Right, good point, will need to re-run the pageview count to exclude spiders.
On Friday, May 22, 2015, Dario Taraborelli dtaraborelli@wikimedia.org wrote:
Thanks for sharing this, Adam. Aside from engagement/funnel data, the critical question for this feature is: does it bring back eyeballs to the site from social media? It looks like it doesn’t yet, at least not in a substantial way, even with the caveat that App traffic is a very small fraction of total mobile traffic.
Having looked into referrals for this feature before and after comparing them to Twitter’s own engagement analytics (and finding some big discrepancy), you should consider removing spiders/crawlers from the data (see [1]) to avoid inflating pageviews with non-human activity.
I’m a big fan of this feature and look forward to seeing how you guys intend to scale it.
Dario
[1] https://github.com/ewulczyn/wmf/blob/b9f726ee3468852c3fed2780af1d8ac0004eda7...
On May 21, 2015, at 12:37 PM, Toby Negrin <tnegrin@wikimedia.org javascript:_e(%7B%7D,'cvml','tnegrin@wikimedia.org');> wrote:
Hi all - some interesting analysis on the share-a-fact feature from the mobile team.
-Toby
Begin forwarded message:
*From:* Adam Baso <abaso@wikimedia.org javascript:_e(%7B%7D,'cvml','abaso@wikimedia.org');> *Date:* May 21, 2015 at 12:05:29 PDT *To:* mobile-l <mobile-l@lists.wikimedia.org javascript:_e(%7B%7D,'cvml','mobile-l@lists.wikimedia.org');> *Subject:* *[WikimediaMobile] Share a Fact Initial Analysis*
Hello all,
We’ve been looking at some initial results from the Share a Fact feature introduced on the Wikipedia apps for Android and iOS in its basic "minimal viable product" implementation. Here’s some analysis, using data from one day (20150512) with respect to the latest stable versions of the apps (2.0-r-2015-04-23 on Android and 4.1.2 on iOS) for that day.
- On iOS, when a user initiates the first step of the default sharing
workflow - tapping the up-arrow box share button (6,194 non-highlighting instances for the day under question) - about 11.7% of the time it yielded successful sharing.
- On Android, it’s not possible to easily tell when the sharing workflow
was carried through to successful share, but we anticipate the Android success rate is currently much higher, as general engagement percentage up to the point of picking an app for sharing is higher on Android than on iOS.
- On Android, when presented with the share card preview, 28.0% of the
time the ‘Share as image’ button was tapped and 55.5% of the time the 'Share as text' button was tapped, whereas on iOS it was 8.4% ‘Share as image’ and 16.8% ‘Share as text’.
- The forthcoming 4.1.4 version of the iOS app will relax its default
sharing snippet generation rules and be more like the Android version in that respect. We anticipate this will result in higher engagement with both the ‘Share as image’ and ‘Share as text’ buttons on iOS, and we should be able to verify this once the 4.1.4 iOS version is released and generally adopted (usually takes 4-5 days after release; the 4.1.4 release isn’t released yet).
- On the Android app the ‘Share’ option is located on the overflow menu,
not as part of the main set of UI buttons. This potentially increases the likelihood of Android users being primed to step through the workflow. On the iOS app, the share button (up-arrow box) is plainly visible from the main UI and not an overflow menu, and this probably creates a different priming dynamic for the iOS demographic.
- When users on iOS tapped on the ‘Share as image’ or ‘Share as text’
buttons, there is a pretty sharp drop off at the next stage - the system sharesheet. Once the sharesheet was presented to iOS users, 41.6% of the time it resulted in active abandonment. We believe this probably has something to do with the relatively small set of default apps listed on the sharesheet and the extra work involved with exposing additional social apps for sharing in that context. As with the Android app, the labels of ‘Share as image’ and ’Share as text’ may also pose something of a hurdle at least for first time users of the feature. To this end, there is an onboarding tutorial planned at least on Android.
- For a one hour period (2015051201) there were about 100 pageviews in
some sense attributable to Share a Fact using a provenance parameter available on the latest stable versions of the apps at that time; this may slightly overstate the number of pageviews attributable to the two specific apps reviewed in this analysis, but probably not too much (n.b., previously a different source parameter was used than the new wprov provenance parameter). Pageviews are not the sole motivation for the feature, but following the trendline over the long run should be interesting. Impact on social media and the destinations of shares is a little harder to capture directly, but https://twitter.com/search?f=realtime&q=%40wikipedia%20-%40itzwikipedia%... gives one a sense about image shares, at least.
- A couple potential options for increasing sharing include:
** Trying to add support for sharing to the Photos app on iOS. People may be interested in using images from the Photos apps for various workflows, as Dan Garry has noted.
** Offering a more concise app picklist, in particular explicitly adding the native OS app components (namely, Twitter and Facebook, and as mentioned, Photos if possible), with an option to expose the sharesheet for additional options if necessary. This is probably also somewhat confined to iOS, although conceivably a similar approach could be possible on Android. On Android the full list of applications in its equivalent of the sharesheet is by default readily available to the user, though.
** On Android, exposing the diagonal arrow share button on the main interface akin to how the iOS version of the app shows the up-arrow share button. This may introduce more opportunities for sharing (and thus numbers of abandons would go up in tandem with numbers of shares), but would also partially clutter the interface and probably increase abandon. A controlled experiment may be useful for observing the impact of such an approach.
- As a point of reference, for the app versions in scope for this analysis
over a single day, there appeared to be approximately 3.78 million Wikipedia for Android pageviews and 1.19 Wikipedia Mobile for iOS app pageviews. There were about 6.73 million app pageviews on the “modern” versions of these apps total for this particular day, meaning there were about 1.75 million pageviews on other modern versions of the app.
- Examination of the categories of successful shares on iOS showed the
following distributions:
Images: 48.5% messaging 25.5% sharesheet copy 22.9% social 1.8% productivity 0.9% reading
Text: 53.6% messaging 31.9% sharesheet copy 7.1% social 5.4% reading 2.0% productivity
Here were some queries used in the analysis:
== SHARE A FACT ATTRIBUTABLE PAGEVIEWS FOR ONE HOUR ==
select wprov, uri_host, count(*) from (select x_analytics_map['wprov'] as wprov, uri_host from webrequest where year = 2015 and month = 5 and day = 12 and hour = 1 and is_pageview = true and uri_host like '%.wikipedia.org' and x_analytics_map['wprov'] is not null) t group by wprov, uri_host;
== PAGE VIEWS FOR THE DAY FOR THE “MODERN” VERSIONS OF THE APPS ==
SELECT user_agent, count(*) FROM wmf.webrequest tablesample(BUCKET 1 OUT OF 100 ON rand()) WHERE YEAR = 2015 AND MONTH = 5 AND DAY = 12 AND is_pageview = TRUE AND lower(uri_host) like '%.wikipedia.org' AND user_agent like 'WikipediaApp%' GROUP BY user_agent;
== HIGHLIGHTING SESSION CASE FOR SPECIFIC VERSIONS OF THE APPS == select CASE WHEN t2.userAgent LIKE 'WikipediaApp/2.0-r-2015-04-23%' THEN 'Android' WHEN t2.userAgent LIKE 'WikipediaApp/4.1.2%' THEN 'iOS' END AS 'ua', t1.event_action, t1.event_sharemode, t1.event_target, count(*) from MobileWikiAppShareAFact_11331974 t1 inner join MobileWikiAppShareAFact_11331974 t2 on t1.event_shareSessionToken = t2.event_shareSessionToken where t1.timestamp > '20150512' and t1.timestamp < '20150513' and t2.timestamp > '20150512' and t2.timestamp < '20150513' and t1.event_action != 'highlight' and t2.event_action = 'highlight' and (t2.userAgent like 'WikipediaApp/2.0-r-2015-04-23%' or t2.userAgent like 'WikipediaApp/4.1.2%') group by ua, t1.event_action, t1.event_sharemode, t1.event_target;
== NON-HIGHLIGHTING SESSION CASE FOR SPECIFIC VERSIONS OF THE APPS == n.b., subtract the highlighting cases from the non-highlighting cases to arrive at the default sharing behavior. Technically, inner joins can be used to do more comprehensive session analysis, but the queries take a long time.
select CASE WHEN userAgent LIKE 'WikipediaApp/2.0-r-2015-04-23%' THEN 'Android' WHEN userAgent LIKE 'WikipediaApp/4.1.2%' THEN 'iOS' END AS 'ua', event_action, event_sharemode, event_target, count(*) from MobileWikiAppShareAFact_11331974 where timestamp > '20150512' and timestamp < '20150513' and (userAgent like 'WikipediaApp/2.0-r-2015-04-23%' or userAgent like 'WikipediaApp/4.1.2%') group by ua, event_action, event_sharemode, event_target;
-Adam
Mobile-l mailing list Mobile-l@lists.wikimedia.org javascript:_e(%7B%7D,'cvml','Mobile-l@lists.wikimedia.org'); https://lists.wikimedia.org/mailman/listinfo/mobile-l
Analytics mailing list Analytics@lists.wikimedia.org javascript:_e(%7B%7D,'cvml','Analytics@lists.wikimedia.org'); https://lists.wikimedia.org/mailman/listinfo/analytics
Okay, I ran a different query [1], and it looks like it was about 87 pageviews for that one hour in question. Some other hours had more pageviews in a sense attributable to Share a Fact (based on the provenance parameter) - there were about 3,616 pageviews for the day in question. Google Sheet here for more info, which has a pivot table tab.
https://docs.google.com/spreadsheets/d/1fGv7_aTf7oo9jxcVk8NqHNi9q8kpO_JwUJXT...
-Adam
[1]
SELECT hour, uri_host, x_analytics_map['wprov'], geocoded_data['country_code'], count(*) FROM wmf.webrequest WHERE x_analytics_map['wprov'] is not null AND year = 2015 AND month = 5 AND day = 12 AND is_pageview = TRUE AND agent_type = 'user' AND uri_host like '%.wikipedia.org' GROUP BY hour, uri_host, x_analytics_map['wprov'], geocoded_data['country_code'] ;
On Fri, May 22, 2015 at 12:11 PM, Adam Baso abaso@wikimedia.org wrote:
Thanks. Right, good point, will need to re-run the pageview count to exclude spiders.
On Friday, May 22, 2015, Dario Taraborelli dtaraborelli@wikimedia.org wrote:
Thanks for sharing this, Adam. Aside from engagement/funnel data, the critical question for this feature is: does it bring back eyeballs to the site from social media? It looks like it doesn’t yet, at least not in a substantial way, even with the caveat that App traffic is a very small fraction of total mobile traffic.
Having looked into referrals for this feature before and after comparing them to Twitter’s own engagement analytics (and finding some big discrepancy), you should consider removing spiders/crawlers from the data (see [1]) to avoid inflating pageviews with non-human activity.
I’m a big fan of this feature and look forward to seeing how you guys intend to scale it.
Dario
[1] https://github.com/ewulczyn/wmf/blob/b9f726ee3468852c3fed2780af1d8ac0004eda7...
On May 21, 2015, at 12:37 PM, Toby Negrin tnegrin@wikimedia.org wrote:
Hi all - some interesting analysis on the share-a-fact feature from the mobile team.
-Toby
Begin forwarded message:
*From:* Adam Baso abaso@wikimedia.org *Date:* May 21, 2015 at 12:05:29 PDT *To:* mobile-l mobile-l@lists.wikimedia.org *Subject:* *[WikimediaMobile] Share a Fact Initial Analysis*
Hello all,
We’ve been looking at some initial results from the Share a Fact feature introduced on the Wikipedia apps for Android and iOS in its basic "minimal viable product" implementation. Here’s some analysis, using data from one day (20150512) with respect to the latest stable versions of the apps (2.0-r-2015-04-23 on Android and 4.1.2 on iOS) for that day.
- On iOS, when a user initiates the first step of the default sharing
workflow - tapping the up-arrow box share button (6,194 non-highlighting instances for the day under question) - about 11.7% of the time it yielded successful sharing.
- On Android, it’s not possible to easily tell when the sharing workflow
was carried through to successful share, but we anticipate the Android success rate is currently much higher, as general engagement percentage up to the point of picking an app for sharing is higher on Android than on iOS.
- On Android, when presented with the share card preview, 28.0% of the
time the ‘Share as image’ button was tapped and 55.5% of the time the 'Share as text' button was tapped, whereas on iOS it was 8.4% ‘Share as image’ and 16.8% ‘Share as text’.
- The forthcoming 4.1.4 version of the iOS app will relax its default
sharing snippet generation rules and be more like the Android version in that respect. We anticipate this will result in higher engagement with both the ‘Share as image’ and ‘Share as text’ buttons on iOS, and we should be able to verify this once the 4.1.4 iOS version is released and generally adopted (usually takes 4-5 days after release; the 4.1.4 release isn’t released yet).
- On the Android app the ‘Share’ option is located on the overflow menu,
not as part of the main set of UI buttons. This potentially increases the likelihood of Android users being primed to step through the workflow. On the iOS app, the share button (up-arrow box) is plainly visible from the main UI and not an overflow menu, and this probably creates a different priming dynamic for the iOS demographic.
- When users on iOS tapped on the ‘Share as image’ or ‘Share as text’
buttons, there is a pretty sharp drop off at the next stage - the system sharesheet. Once the sharesheet was presented to iOS users, 41.6% of the time it resulted in active abandonment. We believe this probably has something to do with the relatively small set of default apps listed on the sharesheet and the extra work involved with exposing additional social apps for sharing in that context. As with the Android app, the labels of ‘Share as image’ and ’Share as text’ may also pose something of a hurdle at least for first time users of the feature. To this end, there is an onboarding tutorial planned at least on Android.
- For a one hour period (2015051201) there were about 100 pageviews in
some sense attributable to Share a Fact using a provenance parameter available on the latest stable versions of the apps at that time; this may slightly overstate the number of pageviews attributable to the two specific apps reviewed in this analysis, but probably not too much (n.b., previously a different source parameter was used than the new wprov provenance parameter). Pageviews are not the sole motivation for the feature, but following the trendline over the long run should be interesting. Impact on social media and the destinations of shares is a little harder to capture directly, but https://twitter.com/search?f=realtime&q=%40wikipedia%20-%40itzwikipedia%... gives one a sense about image shares, at least.
- A couple potential options for increasing sharing include:
** Trying to add support for sharing to the Photos app on iOS. People may be interested in using images from the Photos apps for various workflows, as Dan Garry has noted.
** Offering a more concise app picklist, in particular explicitly adding the native OS app components (namely, Twitter and Facebook, and as mentioned, Photos if possible), with an option to expose the sharesheet for additional options if necessary. This is probably also somewhat confined to iOS, although conceivably a similar approach could be possible on Android. On Android the full list of applications in its equivalent of the sharesheet is by default readily available to the user, though.
** On Android, exposing the diagonal arrow share button on the main interface akin to how the iOS version of the app shows the up-arrow share button. This may introduce more opportunities for sharing (and thus numbers of abandons would go up in tandem with numbers of shares), but would also partially clutter the interface and probably increase abandon. A controlled experiment may be useful for observing the impact of such an approach.
- As a point of reference, for the app versions in scope for this
analysis over a single day, there appeared to be approximately 3.78 million Wikipedia for Android pageviews and 1.19 Wikipedia Mobile for iOS app pageviews. There were about 6.73 million app pageviews on the “modern” versions of these apps total for this particular day, meaning there were about 1.75 million pageviews on other modern versions of the app.
- Examination of the categories of successful shares on iOS showed the
following distributions:
Images: 48.5% messaging 25.5% sharesheet copy 22.9% social 1.8% productivity 0.9% reading
Text: 53.6% messaging 31.9% sharesheet copy 7.1% social 5.4% reading 2.0% productivity
Here were some queries used in the analysis:
== SHARE A FACT ATTRIBUTABLE PAGEVIEWS FOR ONE HOUR ==
select wprov, uri_host, count(*) from (select x_analytics_map['wprov'] as wprov, uri_host from webrequest where year = 2015 and month = 5 and day = 12 and hour = 1 and is_pageview = true and uri_host like '%.wikipedia.org' and x_analytics_map['wprov'] is not null) t group by wprov, uri_host;
== PAGE VIEWS FOR THE DAY FOR THE “MODERN” VERSIONS OF THE APPS ==
SELECT user_agent, count(*) FROM wmf.webrequest tablesample(BUCKET 1 OUT OF 100 ON rand()) WHERE YEAR = 2015 AND MONTH = 5 AND DAY = 12 AND is_pageview = TRUE AND lower(uri_host) like '%.wikipedia.org' AND user_agent like 'WikipediaApp%' GROUP BY user_agent;
== HIGHLIGHTING SESSION CASE FOR SPECIFIC VERSIONS OF THE APPS == select CASE WHEN t2.userAgent LIKE 'WikipediaApp/2.0-r-2015-04-23%' THEN 'Android' WHEN t2.userAgent LIKE 'WikipediaApp/4.1.2%' THEN 'iOS' END AS 'ua', t1.event_action, t1.event_sharemode, t1.event_target, count(*) from MobileWikiAppShareAFact_11331974 t1 inner join MobileWikiAppShareAFact_11331974 t2 on t1.event_shareSessionToken = t2.event_shareSessionToken where t1.timestamp > '20150512' and t1.timestamp < '20150513' and t2.timestamp > '20150512' and t2.timestamp < '20150513' and t1.event_action != 'highlight' and t2.event_action = 'highlight' and (t2.userAgent like 'WikipediaApp/2.0-r-2015-04-23%' or t2.userAgent like 'WikipediaApp/4.1.2%') group by ua, t1.event_action, t1.event_sharemode, t1.event_target;
== NON-HIGHLIGHTING SESSION CASE FOR SPECIFIC VERSIONS OF THE APPS == n.b., subtract the highlighting cases from the non-highlighting cases to arrive at the default sharing behavior. Technically, inner joins can be used to do more comprehensive session analysis, but the queries take a long time.
select CASE WHEN userAgent LIKE 'WikipediaApp/2.0-r-2015-04-23%' THEN 'Android' WHEN userAgent LIKE 'WikipediaApp/4.1.2%' THEN 'iOS' END AS 'ua', event_action, event_sharemode, event_target, count(*) from MobileWikiAppShareAFact_11331974 where timestamp > '20150512' and timestamp < '20150513' and (userAgent like 'WikipediaApp/2.0-r-2015-04-23%' or userAgent like 'WikipediaApp/4.1.2%') group by ua, event_action, event_sharemode, event_target;
-Adam
Mobile-l mailing list Mobile-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mobile-l
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics