Hi all,
Can someone help me with my failing memory and remind me what the current state of affairs is re: unique visitors -- we're not counting them anymore? We are counting them but not via comscore? Something else?
Just putting together a talk and wanted the latest numbers.
thanks, -- Phoebe
Hi Phoebe,
We can't use the comScore numbers any more because they were not tracking mobile devices. Since a lot of our traffic is now coming from mobile devices, comScore numbers were showing a sharp decline, but this was not a reflection of reality. We've been working on a way to measure uniques that's sensitive to privacy, and we're very close to release that data, we're just working on the blog post now. One of the drawbacks is that we can't report on a single total number across all our projects. We'll have an announcement shortly but until then I'll see if we can share the numbers with you privately. Since we can't compute one total number, would you give us a list of wikis that you would like numbers for?
Dan
On Thu, Mar 17, 2016 at 12:52 PM, phoebe ayers phoebe.wiki@gmail.com wrote:
Hi all,
Can someone help me with my failing memory and remind me what the current state of affairs is re: unique visitors -- we're not counting them anymore? We are counting them but not via comscore? Something else?
Just putting together a talk and wanted the latest numbers.
thanks, -- Phoebe
--
- I use this address for lists; send personal messages to phoebe.ayers
<at> gmail.com *
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
On Thu, Mar 17, 2016 at 1:04 PM, Dan Andreescu dandreescu@wikimedia.org wrote:
Hi Phoebe,
We can't use the comScore numbers any more because they were not tracking mobile devices. Since a lot of our traffic is now coming from mobile devices, comScore numbers were showing a sharp decline, but this was not a reflection of reality.
Right, I thought this was the case but didn't know the current state of affairs. (Missed the announcement from Kevin).
We've been working on a way to measure uniques
that's sensitive to privacy, and we're very close to release that data, we're just working on the blog post now.
Awesome, that is great to hear! Congratulations!
One of the drawbacks is that we
can't report on a single total number across all our projects.
Hmm. That's unfortunate for understanding reach -- if nothing else, the idea that "half a billion people access Wikipedia" (eg from earlier comscore reports) was a PR-friendly way of giving an idea of the scale of our readership. But I can see why it would be tricky to measure. Since this is the research list: I suspect there's still lots to be done in understanding just how multilingual people use different language editions of Wikipedia, too.
We'll have
an announcement shortly but until then I'll see if we can share the numbers with you privately. Since we can't compute one total number, would you give us a list of wikis that you would like numbers for?
I was thinking about total numbers, but for this particular project maybe English, Spanish & Arabic -- I'm interested in global languages. Thank you!
Phoebe
Dan
On Thu, Mar 17, 2016 at 12:52 PM, phoebe ayers phoebe.wiki@gmail.com wrote:
Hi all,
Can someone help me with my failing memory and remind me what the current state of affairs is re: unique visitors -- we're not counting them anymore? We are counting them but not via comscore? Something else?
Just putting together a talk and wanted the latest numbers.
thanks, -- Phoebe
--
- I use this address for lists; send personal messages to phoebe.ayers
<at> gmail.com *
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Hey Phoebe,
On Thu, Mar 17, 2016 at 12:40 PM, phoebe ayers phoebe.wiki@gmail.com wrote:
On Thu, Mar 17, 2016 at 1:04 PM, Dan Andreescu dandreescu@wikimedia.org wrote:
One of the drawbacks is that we
can't report on a single total number across all our projects.
Hmm. That's unfortunate for understanding reach -- if nothing else, the idea that "half a billion people access Wikipedia" (eg from earlier comscore reports) was a PR-friendly way of giving an idea of the scale of our readership. But I can see why it would be tricky to measure. Since this is the research list: I suspect there's still lots to be done in understanding just how multilingual people use different language editions of Wikipedia, too.
I'm not sure how ComScore estimated unique visitors/people, and whether they estimated unique users/people or unique devices. I'd like to point out that the current approach for counting uniques that Dan has referred to will tell you the unique device count, and not unique user/people count. There may be big differences in those numbers, for example, I access Wikipedia with at least two devices everyday and I know I'm not unique. ;) Erik Zachte and I have had some very early discussions about how to go from the unique device number to unique user number, but we are nowhere close to any solution. In the mean time, I suggest replacing unique users with unique devices in your communications.
Good luck with your presentation. :)
Best, Leila
Phoebe
Dan
On Thu, Mar 17, 2016 at 12:52 PM, phoebe ayers phoebe.wiki@gmail.com wrote:
Hi all,
Can someone help me with my failing memory and remind me what the current state of affairs is re: unique visitors -- we're not counting them anymore? We are counting them but not via comscore? Something else?
Just putting together a talk and wanted the latest numbers.
thanks, -- Phoebe
--
- I use this address for lists; send personal messages to phoebe.ayers
<at> gmail.com *
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
--
- I use this address for lists; send personal messages to phoebe.ayers
<at> gmail.com *
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
On 17 March 2016 at 19:40, phoebe ayers phoebe.wiki@gmail.com wrote:
One of the drawbacks is that we can't report on a single total number across all our projects.
Hmm. That's unfortunate for understanding reach -- if nothing else, the idea that "half a billion people access Wikipedia" (eg from earlier comscore reports) was a PR-friendly way of giving an idea of the scale of our readership. But I can see why it would be tricky to measure. Since this is the research list: I suspect there's still lots to be done in understanding just how multilingual people use different language editions of Wikipedia, too.
Building on this question a little: with the information we currently have, is it actively *wrong* for us to keep using the "half a billion" figure as a very rough first-order estimate? (Like Phoebe, I think I keep trotting it out when giving talks). Do the new figures give us reason to think it's substantially higher or lower than that, or even not meaningfully answerable?
Hi Andrew, Phoebe -- here's what the Communication department here is comfortable with:
Wikipedia and the other projects operated by the Wikimedia Foundation
receive hundreds of millions of unique visitors per month
The numbers of unique devices for enwiki are far greater than 500mm. But of course, people use more than one device and/or shared computers. There are datasets out there about average number of devices per person so we could potentially use this as a scaling factor to get a higher level of confidence but IMO the mapping from device to actual human is always going to be dicey.
For comparison, I worked at Yahoo for a long time and generally understand their tech stack -- in their 2014 annual report, Yahoo speaks to "more than 1 billion MAUs".[1] From my experience, I really don't know how they could measure this with any certainty without estimation or other statistical techniques because they have the same measurement issues that we do. Only Facebook or other sites where personalization is necessary for the site to work can report on reach without some sort of qualification.
-Toby
[1] http://static.tumblr.com/7drgjla/386nnw4n9/yahoo_inc._2014_annual_report.pdf
On Fri, Mar 18, 2016 at 4:09 AM, Andrew Gray andrew.gray@dunelm.org.uk wrote:
On 17 March 2016 at 19:40, phoebe ayers phoebe.wiki@gmail.com wrote:
One of the drawbacks is that we can't report on a single total number across all our projects.
Hmm. That's unfortunate for understanding reach -- if nothing else, the idea that "half a billion people access Wikipedia" (eg from earlier comscore reports) was a PR-friendly way of giving an idea of the scale of our readership. But I can see why it would be tricky to measure. Since this is the research list: I suspect there's still lots to be done in understanding just how multilingual people use different language editions of Wikipedia, too.
Building on this question a little: with the information we currently have, is it actively *wrong* for us to keep using the "half a billion" figure as a very rough first-order estimate? (Like Phoebe, I think I keep trotting it out when giving talks). Do the new figures give us reason to think it's substantially higher or lower than that, or even not meaningfully answerable?
--
- Andrew Gray andrew.gray@dunelm.org.uk
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
On Fri, Mar 18, 2016 at 9:45 AM, Toby Negrin tnegrin@wikimedia.org wrote:
Hi Andrew, Phoebe -- here's what the Communication department here is comfortable with:
Wikipedia and the other projects operated by the Wikimedia Foundation receive hundreds of millions of unique visitors per month
Thanks Toby. I'll start using that and add it to my presentation toolkit.
The numbers of unique devices for enwiki are far greater than 500mm. But of course, people use more than one device and/or shared computers. There are datasets out there about average number of devices per person so we could potentially use this as a scaling factor to get a higher level of confidence but IMO the mapping from device to actual human is always going to be dicey.
Sounds like a research project! Like Andrew, for communication purposes I'm less interested in exactitude than I am in order-of-magnitude. (The kinds of things I use these numbers for: comparisons against the online population, against the population reached by libraries, etc. -- all of which is deeply qualified, of course.)
Semi-off topic thoughts about "reach" as a metric:
I wonder if there's a qualitative project somewhere in here about *types* of use -- e.g. if I'm using WP on my phone & my work pc is that really equivalent use? Perhaps I am using them for different kinds of information seeking, e.g. looking up terms related to work vs looking up info on movie stars -- does this different kind of use matter for how we construct and present information, or count "use"?
Can we build testable hypotheses about use patterns & needs for people who do straight device swapping (phone to tablet to pc, for the same purposes) versus people who have devices for different purposes (i.e. work v. personal) versus people who share devices? (Obviously, all this goes well beyond just Wikipedia use).
I also think there's something in here about levels of access related to language, which relates to multilingual use of Wikipedia. Someone who speaks a language served by a Wikipedia with 100K articles can not access Wikipedia to the same depth or level that a person who can use English Wikipedia with 5M articles can.
In other words, though we talk about reach, not all reach is the same. The depth to which Wikipedia reaches me -- someone with unlimited data on multiple devices and 24/7 device access for all purposes, who reads English well and a couple other languages poorly -- is way different from the depth which someone with part-time access on a mobile phone who speaks an underserved language is reached by our projects. This may be pretty obvious, but I hadn't thought about the implications for claiming "we reach x millions" before.
-- phoebe
For comparison, I worked at Yahoo for a long time and generally understand their tech stack -- in their 2014 annual report, Yahoo speaks to "more than 1 billion MAUs".[1] From my experience, I really don't know how they could measure this with any certainty without estimation or other statistical techniques because they have the same measurement issues that we do. Only Facebook or other sites where personalization is necessary for the site to work can report on reach without some sort of qualification.
-Toby
[1] http://static.tumblr.com/7drgjla/386nnw4n9/yahoo_inc._2014_annual_report.pdf
On Fri, Mar 18, 2016 at 4:09 AM, Andrew Gray andrew.gray@dunelm.org.uk wrote:
On 17 March 2016 at 19:40, phoebe ayers phoebe.wiki@gmail.com wrote:
One of the drawbacks is that we can't report on a single total number across all our projects.
Hmm. That's unfortunate for understanding reach -- if nothing else, the idea that "half a billion people access Wikipedia" (eg from earlier comscore reports) was a PR-friendly way of giving an idea of the scale of our readership. But I can see why it would be tricky to measure. Since this is the research list: I suspect there's still lots to be done in understanding just how multilingual people use different language editions of Wikipedia, too.
Building on this question a little: with the information we currently have, is it actively *wrong* for us to keep using the "half a billion" figure as a very rough first-order estimate? (Like Phoebe, I think I keep trotting it out when giving talks). Do the new figures give us reason to think it's substantially higher or lower than that, or even not meaningfully answerable?
--
- Andrew Gray andrew.gray@dunelm.org.uk
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Hi Phoebe --
Allow me to direct you to the eminent statistician George Box's aphorism: "All models are wrong but some are useful"[1] :)
Right now the Reading team models our success on three axes:
Reach: how many people read our content Engagement: how much do they read Retention: how often do they come back
Just as metrics for games measure "fun", we think of ours as measuring "learning" but there's a lot of nuance left on the table, particularly in the areas that you highlight above.
We're slowly building up our quantitive and qualitative understanding of our Readers. Leila and team have been doing some really interesting work https://meta.wikimedia.org/wiki/Research:Characterizing_Wikipedia_Reader_Behaviour on Reader intention and we've made some serious strides in our qualitative research as well. Every new project and exploration adds another dimension of understanding but clearly we need to balance complexity and effort with expressive value.
-Toby
[1] https://en.wikipedia.org/wiki/All_models_are_wrong
On Fri, Mar 18, 2016 at 7:45 AM, phoebe ayers phoebe.wiki@gmail.com wrote:
On Fri, Mar 18, 2016 at 9:45 AM, Toby Negrin tnegrin@wikimedia.org wrote:
Hi Andrew, Phoebe -- here's what the Communication department here is comfortable with:
Wikipedia and the other projects operated by the Wikimedia Foundation receive hundreds of millions of unique visitors per month
Thanks Toby. I'll start using that and add it to my presentation toolkit.
The numbers of unique devices for enwiki are far greater than 500mm.
But of
course, people use more than one device and/or shared computers. There
are
datasets out there about average number of devices per person so we could potentially use this as a scaling factor to get a higher level of
confidence
but IMO the mapping from device to actual human is always going to be
dicey.
Sounds like a research project! Like Andrew, for communication purposes I'm less interested in exactitude than I am in order-of-magnitude. (The kinds of things I use these numbers for: comparisons against the online population, against the population reached by libraries, etc. -- all of which is deeply qualified, of course.)
Semi-off topic thoughts about "reach" as a metric:
I wonder if there's a qualitative project somewhere in here about *types* of use -- e.g. if I'm using WP on my phone & my work pc is that really equivalent use? Perhaps I am using them for different kinds of information seeking, e.g. looking up terms related to work vs looking up info on movie stars -- does this different kind of use matter for how we construct and present information, or count "use"?
Can we build testable hypotheses about use patterns & needs for people who do straight device swapping (phone to tablet to pc, for the same purposes) versus people who have devices for different purposes (i.e. work v. personal) versus people who share devices? (Obviously, all this goes well beyond just Wikipedia use).
I also think there's something in here about levels of access related to language, which relates to multilingual use of Wikipedia. Someone who speaks a language served by a Wikipedia with 100K articles can not access Wikipedia to the same depth or level that a person who can use English Wikipedia with 5M articles can.
In other words, though we talk about reach, not all reach is the same. The depth to which Wikipedia reaches me -- someone with unlimited data on multiple devices and 24/7 device access for all purposes, who reads English well and a couple other languages poorly -- is way different from the depth which someone with part-time access on a mobile phone who speaks an underserved language is reached by our projects. This may be pretty obvious, but I hadn't thought about the implications for claiming "we reach x millions" before.
-- phoebe
For comparison, I worked at Yahoo for a long time and generally
understand
their tech stack -- in their 2014 annual report, Yahoo speaks to "more
than
1 billion MAUs".[1] From my experience, I really don't know how they
could
measure this with any certainty without estimation or other statistical techniques because they have the same measurement issues that we do. Only Facebook or other sites where personalization is necessary for the site
to
work can report on reach without some sort of qualification.
-Toby
[1]
http://static.tumblr.com/7drgjla/386nnw4n9/yahoo_inc._2014_annual_report.pdf
On Fri, Mar 18, 2016 at 4:09 AM, Andrew Gray andrew.gray@dunelm.org.uk wrote:
On 17 March 2016 at 19:40, phoebe ayers phoebe.wiki@gmail.com wrote:
One of the drawbacks is that we can't report on a single total number across all our projects.
Hmm. That's unfortunate for understanding reach -- if nothing else, the idea that "half a billion people access Wikipedia" (eg from earlier comscore reports) was a PR-friendly way of giving an idea of the scale of our readership. But I can see why it would be tricky to measure. Since this is the research list: I suspect there's still lots to be done in understanding just how multilingual people use different language editions of Wikipedia, too.
Building on this question a little: with the information we currently have, is it actively *wrong* for us to keep using the "half a billion" figure as a very rough first-order estimate? (Like Phoebe, I think I keep trotting it out when giving talks). Do the new figures give us reason to think it's substantially higher or lower than that, or even not meaningfully answerable?
--
- Andrew Gray andrew.gray@dunelm.org.uk
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
--
- I use this address for lists; send personal messages to phoebe.ayers
<at> gmail.com *
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
On Fri, Mar 18, 2016 at 8:44 AM, Toby Negrin tnegrin@wikimedia.org wrote:
We're slowly building up our quantitive and qualitative understanding of our Readers. Leila and team have been doing some really interesting work https://meta.wikimedia.org/wiki/Research:Characterizing_Wikipedia_Reader_Behaviour on Reader intention and we've made some serious strides in our qualitative research as well.
Phoebe, the page Toby mentioned is well documented and we are updating it as we learn more. If you are interested to get a quick sense of what we know about reader motivations, depth of information they are seeking, and their prior knowledge of the topic before reading an article, please look at the analysis section in https://meta.wikimedia.org/wiki/Research:Characterizing_Wikipedia_Reader_Beh.... Those results are based on a survey we ran in November 2015. We have run another larger survey in March 2016, and we are currently analyzing that survey. It will take months for us to finish this work, but we are hoping to make significant progress in this area and answer some basic but fundamental questions we don't have answers for today.
Of course, ping if you want to chat more about this research. :)
Best, L
-Toby
[1] https://en.wikipedia.org/wiki/All_models_are_wrong
On Fri, Mar 18, 2016 at 7:45 AM, phoebe ayers phoebe.wiki@gmail.com wrote:
On Fri, Mar 18, 2016 at 9:45 AM, Toby Negrin tnegrin@wikimedia.org wrote:
Hi Andrew, Phoebe -- here's what the Communication department here is comfortable with:
Wikipedia and the other projects operated by the Wikimedia Foundation receive hundreds of millions of unique visitors per month
Thanks Toby. I'll start using that and add it to my presentation toolkit.
The numbers of unique devices for enwiki are far greater than 500mm.
But of
course, people use more than one device and/or shared computers. There
are
datasets out there about average number of devices per person so we
could
potentially use this as a scaling factor to get a higher level of
confidence
but IMO the mapping from device to actual human is always going to be
dicey.
Sounds like a research project! Like Andrew, for communication purposes I'm less interested in exactitude than I am in order-of-magnitude. (The kinds of things I use these numbers for: comparisons against the online population, against the population reached by libraries, etc. -- all of which is deeply qualified, of course.)
Semi-off topic thoughts about "reach" as a metric:
I wonder if there's a qualitative project somewhere in here about *types* of use -- e.g. if I'm using WP on my phone & my work pc is that really equivalent use? Perhaps I am using them for different kinds of information seeking, e.g. looking up terms related to work vs looking up info on movie stars -- does this different kind of use matter for how we construct and present information, or count "use"?
Can we build testable hypotheses about use patterns & needs for people who do straight device swapping (phone to tablet to pc, for the same purposes) versus people who have devices for different purposes (i.e. work v. personal) versus people who share devices? (Obviously, all this goes well beyond just Wikipedia use).
I also think there's something in here about levels of access related to language, which relates to multilingual use of Wikipedia. Someone who speaks a language served by a Wikipedia with 100K articles can not access Wikipedia to the same depth or level that a person who can use English Wikipedia with 5M articles can.
In other words, though we talk about reach, not all reach is the same. The depth to which Wikipedia reaches me -- someone with unlimited data on multiple devices and 24/7 device access for all purposes, who reads English well and a couple other languages poorly -- is way different from the depth which someone with part-time access on a mobile phone who speaks an underserved language is reached by our projects. This may be pretty obvious, but I hadn't thought about the implications for claiming "we reach x millions" before.
-- phoebe
For comparison, I worked at Yahoo for a long time and generally
understand
their tech stack -- in their 2014 annual report, Yahoo speaks to "more
than
1 billion MAUs".[1] From my experience, I really don't know how they
could
measure this with any certainty without estimation or other statistical techniques because they have the same measurement issues that we do.
Only
Facebook or other sites where personalization is necessary for the site
to
work can report on reach without some sort of qualification.
-Toby
[1]
http://static.tumblr.com/7drgjla/386nnw4n9/yahoo_inc._2014_annual_report.pdf
On Fri, Mar 18, 2016 at 4:09 AM, Andrew Gray <andrew.gray@dunelm.org.uk
wrote:
On 17 March 2016 at 19:40, phoebe ayers phoebe.wiki@gmail.com wrote:
One of the drawbacks is that we can't report on a single total number across all our projects.
Hmm. That's unfortunate for understanding reach -- if nothing else, the idea that "half a billion people access Wikipedia" (eg from earlier comscore reports) was a PR-friendly way of giving an idea of the scale of our readership. But I can see why it would be tricky to measure. Since this is the research list: I suspect there's still
lots
to be done in understanding just how multilingual people use
different
language editions of Wikipedia, too.
Building on this question a little: with the information we currently have, is it actively *wrong* for us to keep using the "half a billion" figure as a very rough first-order estimate? (Like Phoebe, I think I keep trotting it out when giving talks). Do the new figures give us reason to think it's substantially higher or lower than that, or even not meaningfully answerable?
--
- Andrew Gray andrew.gray@dunelm.org.uk
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
--
- I use this address for lists; send personal messages to phoebe.ayers
<at> gmail.com *
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
I'm sorry, I forgot to mention this: The link I shared is what we know about English Wikipedia. We have also run 2 surveys in Spanish and Persian Wikipedias that were ground-work for running a similar survey as in S3-English in those languages. It is important for us to understand if reader characteristics are different in other languages, and if so, we need to be able to characterize how they are different than English.
L
Leila Zia Research Scientist Wikimedia Foundation
On Fri, Mar 18, 2016 at 9:21 AM, Leila Zia leila@wikimedia.org wrote:
On Fri, Mar 18, 2016 at 8:44 AM, Toby Negrin tnegrin@wikimedia.org wrote:
We're slowly building up our quantitive and qualitative understanding of our Readers. Leila and team have been doing some really interesting work https://meta.wikimedia.org/wiki/Research:Characterizing_Wikipedia_Reader_Behaviour on Reader intention and we've made some serious strides in our qualitative research as well.
Phoebe, the page Toby mentioned is well documented and we are updating it as we learn more. If you are interested to get a quick sense of what we know about reader motivations, depth of information they are seeking, and their prior knowledge of the topic before reading an article, please look at the analysis section in https://meta.wikimedia.org/wiki/Research:Characterizing_Wikipedia_Reader_Beh.... Those results are based on a survey we ran in November 2015. We have run another larger survey in March 2016, and we are currently analyzing that survey. It will take months for us to finish this work, but we are hoping to make significant progress in this area and answer some basic but fundamental questions we don't have answers for today.
Of course, ping if you want to chat more about this research. :)
Best, L
-Toby
[1] https://en.wikipedia.org/wiki/All_models_are_wrong
On Fri, Mar 18, 2016 at 7:45 AM, phoebe ayers phoebe.wiki@gmail.com wrote:
On Fri, Mar 18, 2016 at 9:45 AM, Toby Negrin tnegrin@wikimedia.org wrote:
Hi Andrew, Phoebe -- here's what the Communication department here is comfortable with:
Wikipedia and the other projects operated by the Wikimedia Foundation receive hundreds of millions of unique visitors per month
Thanks Toby. I'll start using that and add it to my presentation toolkit.
The numbers of unique devices for enwiki are far greater than 500mm.
But of
course, people use more than one device and/or shared computers. There
are
datasets out there about average number of devices per person so we
could
potentially use this as a scaling factor to get a higher level of
confidence
but IMO the mapping from device to actual human is always going to be
dicey.
Sounds like a research project! Like Andrew, for communication purposes I'm less interested in exactitude than I am in order-of-magnitude. (The kinds of things I use these numbers for: comparisons against the online population, against the population reached by libraries, etc. -- all of which is deeply qualified, of course.)
Semi-off topic thoughts about "reach" as a metric:
I wonder if there's a qualitative project somewhere in here about *types* of use -- e.g. if I'm using WP on my phone & my work pc is that really equivalent use? Perhaps I am using them for different kinds of information seeking, e.g. looking up terms related to work vs looking up info on movie stars -- does this different kind of use matter for how we construct and present information, or count "use"?
Can we build testable hypotheses about use patterns & needs for people who do straight device swapping (phone to tablet to pc, for the same purposes) versus people who have devices for different purposes (i.e. work v. personal) versus people who share devices? (Obviously, all this goes well beyond just Wikipedia use).
I also think there's something in here about levels of access related to language, which relates to multilingual use of Wikipedia. Someone who speaks a language served by a Wikipedia with 100K articles can not access Wikipedia to the same depth or level that a person who can use English Wikipedia with 5M articles can.
In other words, though we talk about reach, not all reach is the same. The depth to which Wikipedia reaches me -- someone with unlimited data on multiple devices and 24/7 device access for all purposes, who reads English well and a couple other languages poorly -- is way different from the depth which someone with part-time access on a mobile phone who speaks an underserved language is reached by our projects. This may be pretty obvious, but I hadn't thought about the implications for claiming "we reach x millions" before.
-- phoebe
For comparison, I worked at Yahoo for a long time and generally
understand
their tech stack -- in their 2014 annual report, Yahoo speaks to "more
than
1 billion MAUs".[1] From my experience, I really don't know how they
could
measure this with any certainty without estimation or other statistical techniques because they have the same measurement issues that we do.
Only
Facebook or other sites where personalization is necessary for the
site to
work can report on reach without some sort of qualification.
-Toby
[1]
http://static.tumblr.com/7drgjla/386nnw4n9/yahoo_inc._2014_annual_report.pdf
On Fri, Mar 18, 2016 at 4:09 AM, Andrew Gray <
andrew.gray@dunelm.org.uk>
wrote:
On 17 March 2016 at 19:40, phoebe ayers phoebe.wiki@gmail.com
wrote:
> One of the drawbacks is that we > can't report on a single total number across all our projects.
Hmm. That's unfortunate for understanding reach -- if nothing else, the idea that "half a billion people access Wikipedia" (eg from earlier comscore reports) was a PR-friendly way of giving an idea of the scale of our readership. But I can see why it would be tricky to measure. Since this is the research list: I suspect there's still
lots
to be done in understanding just how multilingual people use
different
language editions of Wikipedia, too.
Building on this question a little: with the information we currently have, is it actively *wrong* for us to keep using the "half a billion" figure as a very rough first-order estimate? (Like Phoebe, I think I keep trotting it out when giving talks). Do the new figures give us reason to think it's substantially higher or lower than that, or even not meaningfully answerable?
--
- Andrew Gray andrew.gray@dunelm.org.uk
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
--
- I use this address for lists; send personal messages to phoebe.ayers
<at> gmail.com *
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
phoebe ayers phoebe.wiki@gmail.com writes:
I wonder if there's a qualitative project somewhere in here about *types* of use -- e.g. if I'm using WP on my phone & my work pc is that really equivalent use? Perhaps I am using them for different kinds of information seeking, e.g. looking up terms related to work vs looking up info on movie stars -- does this different kind of use matter for how we construct and present information, or count "use"?
Beyond the issue of devices, I think this is important in part because the raw traffic counts (and reach numbers and similar) paint a very specific story of what Wikimedia is doing and is successful at. (And what you measure influences what you tend to optimize for.)
Specifically, a small slice of content, mainly English Wikipedia articles on pop culture, recent news events, and U.S. politics, contribute a disproportionate share of views. (A weekly top-25 list for enwiki is at https://en.wikipedia.org/wiki/Wikipedia:Top_25_Report ). So if you're measuring aggregate numbers, you're measuring mainly that specific type of content. If the goal is really simply to reach as many people as possible, have high page views and unique visitor counts, etc., then this subset of articles is really the only important part of Wikimedia's mission--- articles on, say, mathematics, don't contribute anything to moving the needle if that's the metric.
-Mark
wiki-research-l@lists.wikimedia.org