It is reasonable to presume watchlists are important for all active Wikimedians on all projects, having a self-selected 'user sample space' of 87, seems small. I know the survey was mentioned on this email list and posted on some noticeboards, but it would have made sense to hold open the survey, and send around a couple more notices, until a few hundred people took part.[1]
To put this in context, there are currently more than 75,000 "active" editors,[2] so a participation rate in a survey targeted at active system contributors of 0.1% is statistically poor. Though a small sample can highlight problems with the user experience, and even discover bugs, you would probably need to aim for 0.5% to 1% of the population to make statistically credible conclusions about what the editor community prefers, or to justify investment in non-critical system improvement. It is interesting to observe that if the sample were engineered to be less self-selected, then statistically it might be valid to draw conclusions from a relatively small sample.
I note that in the survey feedback, the home projects of users is not reported. This is a pity as it is hard to judge if the survey was biased to English speakers/writers, possibly as a consequence of the languages of notices and emails posted.
Links 1. https://meta.wikimedia.org/wiki/Community_Tech/Cross-wiki_watchlist 2. http://reportcard.wmflabs.org
Fae
On 23 May 2016 at 15:34, Bohdan Melnychuk base-w@yandex.ru wrote:
Hi!
How could one participate in the survey? I've just scrolled through past issues of Tech News and did not find any mention of the survey, nor I remember it mentioned in any other place. Was there some way to find about it besides from this list? Which readers are clearly not the most Wikimedia movement representative selection.
Yours, --Base
21.05.2016, 04:21, "Johan Jönsson" jjonsson@wikimedia.org:
On Sat, Mar 26, 2016 at 1:06 AM, Danny Horn dhorn@wikimedia.org wrote:
Hi everyone,
WMF's Community Tech team is starting to work on a Cross-wiki watchlist, one of the top 10 wishes in the Community Wishlist Survey that we conducted at the end of last year. [1]
We're running a survey on how people use their watchlists, to help inform our work.
Hi everyone,
A couple of months ago, the Community Tech team ran a survey to gather information on how Wikimedians use their watchlists. You can see the results here: https://meta.wikimedia.org/wiki/Community_Tech/Cross-wiki_watchlist#Survey_r...
If you're interested, there are also some very early and rough wireframes available on the project page: https://meta.wikimedia.org/wiki/Community_Tech/Cross-wiki_watchlist#Current_...
//Johan Jönsson
faewik@gmail.com https://commons.wikimedia.org/wiki/User:Fae