[Wikimedia-l] Let's report and learn from non-successes (more often)

[Maryana Pinchuk]

On Wed, Jul 3, 2013 at 5:05 PM, Asaf Bartov <abartov at wikimedia.org> wrote:

> Dear colleagues,
>
> I wanted to remind us all of the importance of learning from when our plans
> and efforts don't succeed, whether due to mistakes or just an experimental
> hypothesis turning out to not be the case.
>
> We at the Wikimedia Foundation have had such an experiment a few months
> ago, and an idea we thought had potential turned out to have none of the
> hoped-for impact.  We wrote it up and drew some (I think) interesting
> learning points from it, including possibly next steps for those who may
> want to experiment along similar lines.
>
> The learning is featured in a Wikimedia Blog post here:
>
> http://blog.wikimedia.org/2013/07/02/english-wikipedia-editor-pilot-philippines/
>
> which also links to the full report on Meta.  I am also pleased at the
> discussion the blog post has already generated.
> Wikimedia-l mailing list
> Wikimedia-l at lists.wikimedia.org
> Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l,
> <mailto:wikimedia-l-request at lists.wikimedia.org?subject=unsubscribe>


Yes! Big plus-one! Though I would refer to it as "confirming the null
hypothesis"[1] rather than "non-success" :)

Publication bias[2] and systematic underreporting of non-significant
effects are huge problems in all scholarly fields, so it's not like we're
unusual in this respect. But we miss out on really great opportunities to
advance our institutional learning when we bury our non-successes.. err..
null hypothesis confirmations! Keep em' coming, folks.

1. https://en.wikipedia.org/wiki/Null_hypothesis
2. https://en.wikipedia.org/wiki/Publication_bias

-- 
Maryana Pinchuk
Associate Product Manager, Wikimedia Foundation
wikimediafoundation.org