Hi Kerry, yes, we took into account the possibility for self-selection when we designed the study. That's actually quite a crucial aspect. That is: is MoodBar conducive to increased productivity and retention (because by using it people can seek help from other editors), or is it just that those who were more motivated and perhaps tech-savvy to begin with were also more likely to use MoodBar to report feedback? For example, many people who managed to complete their first edit reported a "happy" mood, usually with a nice sweet message thanking Wikipedia.This is all discussed in the paper, which you are more than welcome to download and read: http://arxiv.org/abs/1409.1496I also mentioned it in the talk, which you can watch here: https://www.youtube.com/watch?v=Rn4-cBYxttACheers,GGiovanni Luca Ciampaglia✉ gciampag@indiana.edu
✎ 919 E 10th ∙ Bloomington 47408 IN ∙ USA
☞ http://www.glciampaglia.com/
✆ +1 812 855-72612014-09-17 17:39 GMT-04:00 Kerry Raymond <kerry.raymond@gmail.com>:Well, it’s interesting that there was a 24% higher edit counts for those with an email address. The question is what does it tell us. Is there a dependency? Are they both dependent on something else we don’t know about?
It might mean (as I am speculating) that those with email addresses were more likely to actually see the notifications of responses to their feedback and more likely to edit more productively from receiving those responses leading to higher edit counts.
But it might also mean something completely different. For example, those who envisaged themselves as making many contributions might have thought it worth the extra keystrokes to provide their email address on signup, whereas those who envisaged themselves as just fixing a couple of things might not have thought it worth the effort. In which case, the 24% difference in edit count might just reflect the difference in intention (self-selecting).
Kerry
From: Giovanni Luca Ciampaglia [mailto:gciampag@indiana.edu]
Sent: Wednesday, 17 September 2014 11:48 PM
To: kerry.raymond@gmail.com; Research into Wikimedia content and communities
Subject: Re: [Wiki-research-l] FW: What works for increasing editor engagement?
Hi Kerry, this is an interesting question but unfortunately no such data was collected from MoodBar. However, we knew whether users had an authenticated email address, and found that this is associated to a 24% higher edit count, as measured in the first 30 days, compared to other users who reported feedback. MoodBar users were also reminded of the possibility of registering an email address, in case they had none, and in this case we observed no difference. Don't know if this answers your question.
Cheers,
G
Giovanni Luca Ciampaglia
✎ 919 E 10th ∙ Bloomington 47408 IN ∙ USA
☞ http://www.glciampaglia.com/
✆ +1 812 855-7261
2014-09-16 0:47 GMT-04:00 Kerry Raymond <kerry.raymond@gmail.com>:
From: Kerry Raymond [mailto:kerry.raymond@gmail.com]
Sent: Tuesday, 16 September 2014 12:23 PM
To: Research into Wikimedia content and communities
Cc: Editor Engagement
Subject: Re: [Wiki-research-l] What works for increasing editor engagement?
With the mood bar, the communication back to the editor was through their user page and email (when known). Do you have any data to show where they saw it (or from where they responded to it)? I've long suspected that new users don't know about User Talk and this frustrates our efforts to communicate with them. so I would be interested to know if there was any difference in reaction from those communicated with via user talk alone and those who also got email and what that might say about user talk as a means to communicate with new users. I note that on the mobile interface running on my ipad, I cannot find a way to get to my User Talk page (as far as I can see), short of entering the URL manually or switching to the desktop interface, which makes user talk pretty useless way of communicating with mobile users.
Sent from my iPad
On 16 Sep 2014, at 6:02 am, Giovanni Luca Ciampaglia <gciampag@indiana.edu> wrote:Hi Pine,
to answer your question on results about improving editor retention, there is a new paper authored by me and Dario coming out soon about MoodBar, an early EE experiment whose aim was to elicit feedback from newly registered editors, that shows that lightweight socialization (e.g. reporting feedback about editing experience and receiving help from more experienced users) improves long-term editor retention.
The pre-print of the paper is up on the arxiv http://arxiv.org/abs/1409.1496
I also gave a talk about it at the Mediawiki metrics meeting earlier this summer: https://www.youtube.com/watch?v=Rn4-cBYxttA
Cheers,
Giovanni Luca Ciampaglia
✎ 919 E 10th ∙ Bloomington 47408 IN ∙ USA
☞ http://www.glciampaglia.com/
✆ +1 812 855-7261
2014-09-11 2:00 GMT-04:00 Pine W <wiki.pine@gmail.com>:
Hello research colleagues,
When I look at the WMF Report Card, it appears to me that the global active editor stats and the number of new accounts being registered per month has been relatively flat since at least 2011.
Those of you who work in EE research and analytics, I would like to ask if there is a summary of techniques that you have found that do produce statistically significant results in improving editor retention. I know that some of you write tools, design projects, or pull and analyze data about editors. It looks to me like WMF is investing significant effort in research and tool creation, but we're not moving the needle to create the results that we had hoped to achieve. So I'd like to ask what have we learned from all of our time working on editor engagement about techniques and programs that do improve the EE stats significant ways, so that we can hopefully accelerate the implementation of programs and techniques that have demonstrated success.
I'd also like to ask what barriers you think prevent us from becoming more effective at improving the number of users who register and the number of active editors. For example, are users who go through GettingStarted often being deterred by quickly being confronted by experienced editors in ways that make the newbies want to leave? If that is a significant problem, how do you suggest addressing this?
One of my concerns about investing further in developing Flow, analytics tools like like WIkimetrics, and further complex editor engagement research projects, is that the most important challenges related to editor engagement may be problems that can only be solved through primarily interpersonal and social means rather than the use of software tools and mass communications. I like Wikimetrics and I use it, and I think there's an important place for analytics and tool development in EE work, but I wonder if WMF should scale up the emphasis on grassroots social and interpersonal efforts, particularly in the context of the 2015+ Strategic Plan and Jimmy's speech at the 2014 Wikimania. What do you think,and if your answer is yes, how do you think WMF can do this while respecting the autonomy and social processes of the volunteer projects?
Thanks,
Pine
_______________________________________________
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
_______________________________________________
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
_______________________________________________
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
_______________________________________________
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l