Hi all,
I wanted to provide some advanced notification that as of May 31st, we no
longer plan to support the pilot social media traffic report that we have
been running. As a reminder, in response to a request in a Disinformation
Meetup during Wikimania 2019 [1], we have been publishing daily reports [2]
that list articles on English Wikipedia that received over 500 pageviews
the day before from readers coming from one of four external platforms
(Youtube, Twitter, Reddit, Facebook) [3]. As of May 31st, we (the WMF
Research team [4]) will no longer maintain the job that produces these
daily reports.
Factors in our decision:
We ran some basic analyses [5] based on the first two months of data (3324
unique articles) but do not see strong evidence that this data has had a
clearly valuable impact on patrolling:
* We saw no significant change in the number of edits: between the two
weeks prior to an article being published to the report (10.7; 99%
confidence interval: [8.3-13.7]) and the two weeks following (10.8; 99%
confidence interval: [8.5-13.3]).
* We saw no significant change in the number of reverts: between the two
weeks prior to an article being published to the report (0.83 [0.70-0.99])
and the two weeks following (0.83 [0.70-0.98]).
* We saw a slight uptick in the amount of page restrictions: 64 (2%) of the
articles saw an increase in protections while the rest remained the same
(we did not verify whether this is "normal" for similar articles over the
time range we are studying).
* We saw steady but low traffic to the report:
https://tools.wmflabs.org/pageviews/?project=en.wikipedia.org&platform=…
* We are aware that a large number of users -- even outside of this
listserv -- received notice of the report:
https://en.wikipedia.org/w/index.php?title=Special:WhatLinksHere/User:HostB…
* We received some good feedback early on in the pilot that we have
captured:
https://meta.wikimedia.org/wiki/Research_talk:Social_media_traffic_report_p…
We have seen a number of positive things come out of this work:
* The above analyses suggest that the organic traffic coming from external
platforms like Youtube and Facebook that link to Wikipedia articles as
context for fact-checking is not having a significant deleterious impact on
Wikipedia or placing an additional burden on patrollers.
* We now have a working process that allows us to deploy reports like this
one, which will make it much easier to prototype additional data releases
of this sort in the future.
* Though the early evidence suggests that the Social Media Traffic Report
as an intervention has not led to a substantial change in patrolling
behavior around these articles, we now have a public dataset of
externally-referred traffic for two months that can support further
research into the impact of platforms (and users on those platforms) that
link to Wikipedia articles.
If you have a compelling reason to extend the pilot, any questions,
thoughts, or other feedback, don't hesitate to reach out or post on the
project talk page [6]. We remain committed to support you in your important
patrolling efforts and will continue to prototype new interventions where
we can.
[1]
https://wikimania.wikimedia.org/wiki/2019:Meetups/Disinformation
[2]
https://en.wikipedia.org/wiki/User:HostBot/Social_media_traffic_report
[3]
https://meta.wikimedia.org/wiki/Research:Social_media_traffic_report_pilot/…
[4]
https://research.wikimedia.org/team.html
[5]
https://phabricator.wikimedia.org/T241768#6152394
[6]
https://meta.wikimedia.org/wiki/Research_talk:Social_media_traffic_report_p…
Thanks,
Isaac
On Mon, Mar 23, 2020 at 2:24 PM Jonathan Morgan <jmorgan(a)wikimedia.org>
wrote:
The WMF Research team has published a new pageview
report of inbound
traffic coming from Facebook, Twitter, YouTube, and Reddit.[1]
The report contains a list of all articles that received at least 500
views from one or more of these platforms (i.e. someone clicked a link on
Twitter that sent them directly to a Wikipedia article). The report is
available on-wiki and will be updated daily at around 14:00 UTC with
traffic counts from the previous calendar day.
We believe this report provides editors with a valuable new information
source. Daily inbound social media traffic stats can help editors monitor
edits to articles that are going viral on social media sites and/or are
being linked to by the social media platform itself in order to fact-check
disinformation and other controversial content[2][3].
The social media traffic report also contains additional public article
metadata that may be useful in the context of monitoring articles that are
receiving unexpected attention from social media sites, such as...
- the total number of pageviews (from all sources) that article
received in the same period of time
- the number of pageviews the article received from the same platform
(e.g. Facebook) the previous day (two days ago)
- the number of editors who have the page on their watchlist
- the number of editors who have watchlisted the page AND recently
visited it
We want your feedback! We have some ideas of our own for how to improve
the report, but we want to hear yours! If you have feature suggestions,
please add them here.[4] We intend to maintain this daily report for at
least the next two months. If we receive feedback that the report is
useful, we are considering making it available indefinitely.
If you have other questions about the report, please first check out our
(still growing) FAQ [5]. All questions, comments, concerns, ideas, etc. are
welcome on the project talkpage on Meta.[4]
1.
https://en.wikipedia.org/wiki/User:HostBot/Social_media_traffic_report
2.
https://www.engadget.com/2018/03/15/wikipedia-unaware-would-be-youtube-fact…
3.
https://mashable.com/2017/10/05/facebook-wikipedia-context-articles-news-fe…
4.
https://meta.wikimedia.org/wiki/Research_talk:Social_media_traffic_report_p…
5.
https://meta.wikimedia.org/wiki/Research:Social_media_traffic_report_pilot/…
Cheers,
Jonathan
--
Jonathan T. Morgan
Senior Design Researcher
Wikimedia Foundation
User:Jmorgan (WMF) <https://meta.wikimedia.org/wiki/User:Jmorgan_(WMF)>
(Uses He/Him)
*Please note that I do not expect a response from you on evenings or
weekends*
_______________________________________________
Analytics mailing list
Analytics(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/analytics
--
Isaac Johnson (he/him/his) -- Research Scientist -- Wikimedia Foundation