It is reasonable to presume watchlists are important for all active
Wikimedians on all projects, having a self-selected 'user sample
space' of 87, seems small. I know the survey was mentioned on this
email list and posted on some noticeboards, but it would have made
sense to hold open the survey, and send around a couple more notices,
until a few hundred people took part.[1]
To put this in context, there are currently more than 75,000 "active"
editors,[2] so a participation rate in a survey targeted at active
system contributors of 0.1% is statistically poor. Though a small
sample can highlight problems with the user experience, and even
discover bugs, you would probably need to aim for 0.5% to 1% of the
population to make statistically credible conclusions about what the
editor community prefers, or to justify investment in non-critical
system improvement. It is interesting to observe that if the sample
were engineered to be less self-selected, then statistically it might
be valid to draw conclusions from a relatively small sample.
I note that in the survey feedback, the home projects of users is not
reported. This is a pity as it is hard to judge if the survey was
biased to English speakers/writers, possibly as a consequence of the
languages of notices and emails posted.
Links
1.
https://meta.wikimedia.org/wiki/Community_Tech/Cross-wiki_watchlist
2.
http://reportcard.wmflabs.org
Fae
On 23 May 2016 at 15:34, Bohdan Melnychuk <base-w(a)yandex.ru> wrote:
Hi!
How could one participate in the survey? I've just scrolled through past issues of
Tech News and did not find any mention of the survey, nor I remember it mentioned in any
other place. Was there some way to find about it besides from this list? Which readers are
clearly not the most Wikimedia movement representative selection.
Yours,
--Base
21.05.2016, 04:21, "Johan Jönsson" <jjonsson(a)wikimedia.org>rg>:
> On Sat, Mar 26, 2016 at 1:06 AM, Danny Horn <dhorn(a)wikimedia.org> wrote:
>> Hi everyone,
>>
>> WMF's Community Tech team is starting to work on a Cross-wiki watchlist,
>> one of the top 10 wishes in the Community Wishlist Survey that we conducted
>> at the end of last year. [1]
>>
>> We're running a survey on how people use their watchlists, to help inform
>> our work.
>
> Hi everyone,
>
> A couple of months ago, the Community Tech team ran a survey to gather
> information on how Wikimedians use their watchlists. You can see the
> results here:
>
https://meta.wikimedia.org/wiki/Community_Tech/Cross-wiki_watchlist#Survey_…
>
> If you're interested, there are also some very early and rough
> wireframes available on the project page:
>
https://meta.wikimedia.org/wiki/Community_Tech/Cross-wiki_watchlist#Current…
>
> //Johan Jönsson
faewik(a)gmail.com
https://commons.wikimedia.org/wiki/User:Fae