I'd love to do some sort of editing frequency/time clustering. One thing that might be interesting is to take a cohort of users that started editing in, say Jan 2008, and map out that cohort's edit frequency by month from Jan 2008-present and see if meaningful clusters emerge. And then repeat for different start dates.
This type of analysis should help us get a better understanding of what it means to "leave." In the Former Contributor's Survey, a large percentage of users self-identified as "not having left", but I'm very curious as to whether they actually came back. I will try to get that stat. But on a more general level, it would be great to know "for user has been editing for x months, with an average of x edits/month, if s/he stops editing for 3 months, there's an x% chance that s/he is not coming back."
Howie
On 8/10/10 3:13 PM, Samuel Klein wrote:
It would be interesting to see both a description of current editors in terms of their editing pattern and frequency, and a description of the editing histories of everyone who has edited in the past year.
To do this, it would be useful to cluster editing patterns into
- classification of editing style by recent editing pattern and frequency.
- classification of editing histories by their shape and change over time.
Editing style give a sense of the type of editor+editing over a given period [week? month?]. Editing history suggests an editor lifecycle and where in that cycle the editor would be.
average time a user who does an edit has been registered.
I like this measure.
SJ
On Tue, Aug 10, 2010 at 5:49 PM, Luca de Alfaroluca@dealfaro.org wrote:
It's difficult to tell when a person leaves, because ... you never know if a contribution they made is the last one. A measure would be "how many users have done an edit in the last month", and this is actually an incredibly simple DB query to run (how fast it runs, is another question). This can tell you how the number of users is evolving. Hey, I could run this on my wikitrust database, if the Foundation does not wish to do this :-) Another measure, which is slightly harder but not much to compute, is the average time a user who does an edit has been registered. Together these two figures (across time) could give you a pretty good picture of what is going on. Luca
On Tue, Aug 10, 2010 at 2:38 PM, Felipe Ortegaglimmer_phoenix@yahoo.es wrote:
Hello guys.
First of all, kudos for this initiative! It's great that all researchers in this list can get to know the names and interests of WMF staff working on same topics.
Additional context for Piotr. Believe me, it's really challenging to define a set of clear, and *exact* conditions to consider that any wikipedian ceased to contribute.
For our analysis published by WSJ last November, we followed similar requirements to those in the Former Contributors Survey. In particular, we established 3 months of inactivity as a "reasonable" period to consider that an editor took a long break. The main difference is that in the Survey they focus on editors who reached at least a reasonable number of lifetime revisions (20-99), while we included everyone.
I already broke down the net gain curve for different cohorts, according to number of edits, and there is no significant change in the trends (I believe that the meaningful info is the slope, not the numbers).
For what is worth, I think the best constructive critic we received about this approach came from Jimmy Wales. Jimmy explained a useful twist to the methodology, that they seem to be applying for internal metrics at Wikia.
Instead of trying to measure how many people "left", which will always have methodological drawbacks, we can ask the following question: what percentage of editors survived up to a certain age?
For instance: what's the percentage of editors who made at least 20 lifetime edits who are still active one month later? Three months later? And then: is that percentage improving, constant, or getting worse over time?
Indeed, limiting the scope to recorded revisions (the only event we can certainly measure) we avoid many of these methodological problems.
I'm still spending time with flagged-revisions, but in case Howie or anybody else is interested, it shouldn't be difficult to have a look at this.
BTW, Howie thanks for uploading the survey slides. Terrific the work you did, guys.
Cheers, Felipe.
--- El mar, 10/8/10, Piotr Koniecznypiokon@post.pl escribió:
De: Piotr Koniecznypiokon@post.pl Asunto: Re: [Wiki-research-l] WMF Staff Introductions. Para: "Research into Wikimedia content and communities" wiki-research-l@lists.wikimedia.org Fecha: martes, 10 de agosto, 2010 20:21 Welcome!
I have to say that http://strategy.wikimedia.org/wiki/Former_Contributors_Survey_Results of which I've just learned from your post is an excellent piece of research, one that was needed for a very long time.
One question comes to mind: we now, roughly, how many editors we are gaining per months. Are there any estimates on how many we are losing (per month, year, total)? I cannot find such numbers in that survey.
-- Piotr Konieczny
Parul Vora wrote:
Hello everyone,
We (most of the current staff at the Wikimedia
Foundation currently
engaging in research) had a chance to meet some of you
at Wikisym and
Wikimania this year and thought it would be nice to
introduce ourselves
and say hi to all of you! All of us have joined WMF in
the past two
years and are working on projects or research
questions that may be
relevant or of interest to all of you. Also, as far as
I know, we are
all new to this list and will hopefully be talking and
collaborating
with you more in the future - both here and on the
Meta Research page.
So, in no particular order, some introductions from
all of us:
From Nimishhttp://wikimediafoundation.org/wiki/User:Nimish_Gautam (ngautam@wikimedia.org):
Hi, I'm Nimish Gautam. I started with the
foundation in 2009 doing development for the Usability
Initiative, which
focused on new editors. I like analyzing user behavior
to figure out how
people use the tools we give them (turning templates
into a programming
language, who would've guessed?) and spotting trends
so we can improve
those tools to help people accomplish what it is
they're trying to do.
Currently I'm doing qualitative and quantitative
research on user
behavior for the foundation and its various projects,
and very
interested in finding ways of chunking all this
information together to
make pretty, compelling, informative resources so
people know what's
going on in the wikiverse and hopefully want to be a
part of it.
From Howie http://wikimediafoundation.org/wiki/Staff#User_Experience_Programs (hfung@wikimedia.org):
Hello! I'm Howie and I'm a Senior Product Manager
at the Wikimedia Foundation. As a product person, I'd
like to work with
the community towards more data-driven decision
making. One area I'm
particularly interested in is getting a better
understanding of our
user's lifecycle with our projects -- how they come to
the projects, how
they start contributing, their experiences as a
contributor, why they
leave, and why they return. I like to use both
quantitative and
qualitative methods to obtain as complete a picture as
we need to guide
our decisions. On the quantitative side, I'm working
on getting better
web analytics for our projects. I'm also interested in
any data mining
projects along these lines (e.g., contribution
behavior, user lifecycle
patterns, etc.). On the qualitative side, I worked on
the "Why Editors
Leave Wikipedia" survey and would be interested in
other qualitative
measurements (e.g., interviews, surveys, focus
groups). If you're
interested in any of the above topics, please drop me
a line.
From Amy http://wikimediafoundation.org/wiki/Staff#Public_Policy_Initiative (aroth@wikimedia.org):
Hi, I'm Amy, the Research Analyst for the Public
Policy Initiative http://outreach.wikimedia.org/wiki/Public_Policy_Initiative.
My task
is to assess the project's impact on: U.S. public
policy article
quality, public policy categorization, new articles,
and new
contributors. Through the project I have focused on
article quality
assessment, and worked with the community to add a
quantitative value to
the current article assessment
http://en.wikipedia.org/wiki/Wikipedia:WikiProject_United_States_Public_Policy/Quality_rating#Rubric. As a data analyst, I am interested in improving data
accessibility from
Wikipedia. In my dreams, I envision data from the
assessment tools that
exist within Wikipedia are captured in a real-time
database, so that we
can observe what is currently happening in Wikipedia
and how it is
evolving in the present, rather than having to use
data dumps to get
snapshots of the state of Wikipedia. I have experience
analyzing and
designing surveys and would like to use that
experience to take a more
in depth look at contributor demographics and
motivations. I am excited
to be a part of this huge collaborative project with a
mission to make
knowledge accessible.
From me, Parul Vora
http://wikimediafoundation.org/wiki/User:Parulvora (pvora@wikimedia.org):
Hi Everyone! I'm a researcher and designer with a
focus on participatory and collaborative spaces. I
started at the
Wikimedia Foundation in 2009 and moving forward have
interest in:
creating new forms of participation (beyond editing)
on the projects
that better engage a wider audience with the content
and each other;
assessing, evaluating and addressing the demographic
and cultural biases
in our projects; and exploring location, culture and
language as they
affect the development patterns of different language
Wikipedias in an
effort to identify potential for experimentation and
catalysis in
younger projects. I'm currently exploring the
potential effect feedback
systems (article ratings, expert reviews,
visualizations of an article's
history or a user's contributions) can have on the
engagement of
readers, actions of editors, and the quality of
content over time. I
like infovis, ux research, and unresearched innovation
and I am
interested in learning more about research with
wikipedia on motivation,
behavioral economic modeling and/or game theory, using
geolocative data,
mobile experiences, and profiling and trend
visualizations......and your
work too!
Let us know if you're interested in learning more,
participating in, or
contributing to our efforts. And drop any of us a line
if we could learn
from or contribute to what you've been working
on.......
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l