Samuel Klein, 30/04/2014 05:35:
Asking 1/1000 users of tool X a single open-ended question ("please give us feedback on X" or "how is X working for you"?) can be a handy way to encourage brief input from a cross-section of users,
We do have such a feature in MediaWiki, though: mediawiki.feedback.js. It's just a JavaScrip popup which saves the comment to a page on the wiki.
many of which would not otherwise comment at all. And for some tools (such as UploadWizard) there is no obvious place to leave comments, and opening Bugzilla is a new-tab + multi-step process away.
UploadWizard (like VisualEditor) uses what above. Maybe it needs an option to be offered more prominently under some conditions? This is probably the most viable option here, almost no technical effort and more value in output.
Tilman Bayer, 29/04/2014 21:58:
might be worth revisiting LimeSurvey, which appears to have undergone a complete rewrite since that installation was removed from WMF servers for security concerns around 2011.
+1. It will need to be done anyway, at some point, e.g. if a general editor survey is tried again.
multilingual support than other solutions [...] lack of integrated language support in Surveymonkey, or just because the focus was on per-project results anyway?
Agreed on all the rest but this point specifically. It seems surveymonkey is really out of question. However, how many languages does Qualtrics support? LimeSurvey says 50; it is translated on a public instance of GlotPress. GlotPress is from Automattic and is used to make some Wordpress locale, hence some translatewiki.net have experience with it. However I wasn't able to gather much information about it, I only know that it's yet another web tool for .po format; maybe Stu can put us in contact with someone with more insight (especially on how much it's used and how prioritary for Automattic)?
Nemo
On 30 April 2014 08:52, Federico Leva (Nemo) nemowiki@gmail.com wrote:
We do have such a feature in MediaWiki, though: mediawiki.feedback.js. It's just a JavaScrip popup which saves the comment to a page on the wiki.
many of which would not otherwise comment at all. And for some tools (such as UploadWizard) there is no obvious place to leave comments, and opening Bugzilla is a new-tab + multi-step process away.
UploadWizard (like VisualEditor) uses what above. Maybe it needs an option to be offered more prominently under some conditions? This is probably the most viable option here, almost no technical effort and more value in output.
It's certainly simpler to implement, but anything that involves on-wiki recording has two main problems:
* friction in saving the entry (eg edit conflicts, login required, user IP blocked) * privacy problems (comments are public and effectively attributed)
This isn't much of an issue for things like "please give us feedback on the new fancy upload tool" - where everyone can be expected to have a functioning account and aware of how the wiki works, but if you're going to be gathering feedback on reader-focused things it breaks down.
Andrew Gray, 30/04/2014 21:47:
It's certainly simpler to implement, but anything that involves on-wiki recording has two main problems:
- friction in saving the entry (eg edit conflicts, login required,
user IP blocked)
Edit conflicts are supposed to be avoided, when appending text to a page via API. Please file a bug if this is not working, we have some reports but they're all quite incomplete. The rest may skew a survey from a scientifical point of view, but won't significantly spoil a simple collection of comments IMHO.
- privacy problems (comments are public and effectively attributed)
On the other hand you don't have to worry about private data, you're left only with public data. Of course users need to be informed.
This isn't much of an issue for things like "please give us feedback on the new fancy upload tool" - where everyone can be expected to have a functioning account and aware of how the wiki works,
Sure. In fact the goal is «capture user feedback about Upload Wizard as soon as they have completed their upload, while it’s still fresh on their minds» and that's exactly what mediawiki.feedback.js does currently within Special:UploadWizard, just not very prominently. (Even MoodBar does/did that I suppose.)
but if you're going to be gathering feedback on reader-focused things it breaks down.
True. However the surveys we're talking about here are in the order of 10³ respondents. It's a lot compared to e.g. the few hundreds per year in http://vs.aka-online.de/cgi-bin/wppagehiststat.pl?lang=commons.wikimedia&page=Commons:Upload_Wizard_feedback, but not so explosive. pl.wiki's http://vs.aka-online.de/cgi-bin/wppagehiststat.pl?lang=pl.wikipedia&page=Wikipedia:Zgłoś_błąd_w_artykule, fed by a popup in sidebar, seems to work well.
Nemo
On 1 May 2014 08:07, Federico Leva (Nemo) nemowiki@gmail.com wrote:
- friction in saving the entry (eg edit conflicts, login required,
user IP blocked)
Edit conflicts are supposed to be avoided, when appending text to a page via API. Please file a bug if this is not working, we have some reports but they're all quite incomplete. The rest may skew a survey from a scientifical point of view, but won't significantly spoil a simple collection of comments IMHO.
Interesting - edit conflicts are pretty rare in any case, so I can't speak to this with confidence, but I'm happy to take your word for it :-) I've only had one problem that I attributed to an edit conflict (something involving commons deletion-nomination scripts a year or two ago?) but it's always possible it was something else. So put this down as just relating to account/blocked-IP problems...
I suspect this sort of issue won't significantly affect the outcomes of the survey (probably not in a statistically significant way), they will annoy or distress readers who encounter them. "I tried to do something and it gave me all sorts of weird alarming messages about not being allowed to" - it's a very harsh way to make them annoyed with us, and so I think there is a definite risk of a downside if we start using this mechanism for general readers.
- privacy problems (comments are public and effectively attributed)
On the other hand you don't have to worry about private data, you're left only with public data. Of course users need to be informed.
Indeed - but that informing starts adding complexity and may deter some responses. (The private data may also be collected in such a way that makes it *less* sensitive - eg by not logging IP addresses of respondents - which we're not currently able to do with a mediawiki solution.)
Sure. In fact the goal is «capture user feedback about Upload Wizard as soon as they have completed their upload, while it’s still fresh on their minds» and that's exactly what mediawiki.feedback.js does currently within Special:UploadWizard, just not very prominently. (Even MoodBar does/did that I suppose.)
Definitely agree it's the most suitable tool for cases like this.