Speaking of specialized lists, I'd like to suggest that this discussion would be well suited to Research-l, where many people who are interested in these kinds of questions read and write about them more frequently than they do on Wikimedia-l. I'm boldly adding that list to the recipients for this thread.
I have some thoughts about the substance of this discussion but they're a bit rushed at the moment. I may write more later.
Regards, Pine On Jun 2, 2015 4:32 AM, "Milos Rancic" millosh@gmail.com wrote:
Luis, I have to say that you are the first person on WMF side who has substantially engaged into this issue and I am very glad to see that :)
The products of your work are of the highest importance, as the community is the most important part of our movement, not to say that it's the movement itself.
I am finally relieved to know that we are on the path to rationally understand what's going on inside of the community after short 14.5 years.
It would be good if you'd share your results with the rest of us.
As for this list: As MZ said, this list is important. However, there is no doubt that it's far from being the only or even the most important indicator of community health. It is just about one of the rare publicly accessible data which could give a clue of what's going on inside of the community, but could mislead, as well. On Jun 2, 2015 04:39, "Luis Villa" lvilla@wikimedia.org wrote:
On Mon, Jun 1, 2015 at 10:57 AM, Milos Rancic millosh@gmail.com wrote:
On Mon, Jun 1, 2015 at 7:51 PM, Luis Villa lvilla@wikimedia.org
wrote:
On Mon, Jun 1, 2015 at 9:26 AM, Andrew Lih andrew.lih@gmail.com
wrote:
- Participation in the mailing list may be a misleading indicator
of
activity or interest, as other regional or specialized forums (eg. Facebook, GLAM-oriented lists, etc) have emerged in recent years.
Let me second this. My department is thinking about community health metrics (constructive suggestions welcome!), but I would not
personally
propose mailing list participation (especially this list) as a good
metric
- decreased participation here may reflect many, many things, only
some
of
which are actually negative.
This is not the only one indicator, but it's pretty consistent since 2011 (take a look into [1]). In other words, something happened in May. Maybe it's actually about the elections because people used other means of communication for that.
Looking briefly at some of the highest-traffic months, it could simply be that people got tired of discussing high-controversy topics here. (Flamewars are good for traffic volume; not so great for community
health.)
I'm sure Facebook's increased acceptance also has a role. I suspect also that some announcements that used to come here now go to other, more specialized mailing lists.
That last one points to a key thing: as MZ says, many people are
subscribed
to this list, but many don't read and don't participate, because this mailing list has an *awful* reputation, and people who want to get things done are going elsewhere. So "the decline of wikimedia-l" may be a sign
of
bad health of the overall community, or it may simply mean that the
healthy
and constructive parts of the community has moved elsewhere.
To re-iterate what I said in the last email, I'm all ears for suggestions on creative community metrics. I'll add here that I'm also very open to suggestions on what a new wikimedia-l might look like. (I know some FOSS communities are having good experiences with discourse.org, for
example.)
No commitment that WMF can act on either immediately, of course, but I think it is worth starting both of those discussions.
Luis
-- Luis Villa Sr. Director of Community Engagement Wikimedia Foundation *Working towards a world in which every single human being can freely
share
in the sum of all knowledge.* _______________________________________________ Wikimedia-l mailing list, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines Wikimedia-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, mailto:wikimedia-l-request@lists.wikimedia.org?subject=unsubscribe
Wikimedia-l mailing list, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines Wikimedia-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, mailto:wikimedia-l-request@lists.wikimedia.org?subject=unsubscribe
Hi Pine,
Can you provide us with a quick summary of the start of this conversation and what the " these kinds of questions " might be?
Thanks! -Aaron
On Tue, Jun 2, 2015 at 10:04 AM, Pine W wiki.pine@gmail.com wrote:
Speaking of specialized lists, I'd like to suggest that this discussion would be well suited to Research-l, where many people who are interested in these kinds of questions read and write about them more frequently than they do on Wikimedia-l. I'm boldly adding that list to the recipients for this thread.
I have some thoughts about the substance of this discussion but they're a bit rushed at the moment. I may write more later.
Regards, Pine On Jun 2, 2015 4:32 AM, "Milos Rancic" millosh@gmail.com wrote:
Luis, I have to say that you are the first person on WMF side who has substantially engaged into this issue and I am very glad to see that :)
The products of your work are of the highest importance, as the community is the most important part of our movement, not to say that it's the movement itself.
I am finally relieved to know that we are on the path to rationally understand what's going on inside of the community after short 14.5 years.
It would be good if you'd share your results with the rest of us.
As for this list: As MZ said, this list is important. However, there is no doubt that it's far from being the only or even the most important indicator of community health. It is just about one of the rare publicly accessible data which could give a clue of what's going on inside of the community, but could mislead, as well. On Jun 2, 2015 04:39, "Luis Villa" lvilla@wikimedia.org wrote:
On Mon, Jun 1, 2015 at 10:57 AM, Milos Rancic millosh@gmail.com
wrote:
On Mon, Jun 1, 2015 at 7:51 PM, Luis Villa lvilla@wikimedia.org
wrote:
On Mon, Jun 1, 2015 at 9:26 AM, Andrew Lih andrew.lih@gmail.com
wrote:
- Participation in the mailing list may be a misleading indicator
of
activity or interest, as other regional or specialized forums (eg. Facebook, GLAM-oriented lists, etc) have emerged in recent years.
Let me second this. My department is thinking about community health metrics (constructive suggestions welcome!), but I would not
personally
propose mailing list participation (especially this list) as a good
metric
- decreased participation here may reflect many, many things, only
some
of
which are actually negative.
This is not the only one indicator, but it's pretty consistent since 2011 (take a look into [1]). In other words, something happened in May. Maybe it's actually about the elections because people used other means of communication for that.
Looking briefly at some of the highest-traffic months, it could simply
be
that people got tired of discussing high-controversy topics here. (Flamewars are good for traffic volume; not so great for community
health.)
I'm sure Facebook's increased acceptance also has a role. I suspect also that some announcements that used to come here now go to other, more specialized mailing lists.
That last one points to a key thing: as MZ says, many people are
subscribed
to this list, but many don't read and don't participate, because this mailing list has an *awful* reputation, and people who want to get
things
done are going elsewhere. So "the decline of wikimedia-l" may be a sign
of
bad health of the overall community, or it may simply mean that the
healthy
and constructive parts of the community has moved elsewhere.
To re-iterate what I said in the last email, I'm all ears for
suggestions
on creative community metrics. I'll add here that I'm also very open to suggestions on what a new wikimedia-l might look like. (I know some FOSS communities are having good experiences with discourse.org, for
example.)
No commitment that WMF can act on either immediately, of course, but I think it is worth starting both of those discussions.
Luis
-- Luis Villa Sr. Director of Community Engagement Wikimedia Foundation *Working towards a world in which every single human being can freely
share
in the sum of all knowledge.* _______________________________________________ Wikimedia-l mailing list, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines Wikimedia-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, mailto:wikimedia-l-request@lists.wikimedia.org?subject=unsubscribe
Wikimedia-l mailing list, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines Wikimedia-l@lists.wikimedia.org https://meta.wikimedia.org/wiki/Mailing_lists/GuidelinesWikimedia-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, mailto:wikimedia-l-request@lists.wikimedia.org?subject=unsubscribe
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Context: reduced traffic on wikimedia-l. Is this a sign of poor community health? To what extent has community energy and activity moved to other fora? Is list activity (for certain lists) a useful measure of activity and enthusiasm (about certain parts of the projects)?
On Thu, Jun 4, 2015 at 7:18 AM, Aaron Halfaker aaron.halfaker@gmail.com wrote:
Hi Pine,
Can you provide us with a quick summary of the start of this conversation and what the " these kinds of questions " might be?
Thanks! -Aaron
On Tue, Jun 2, 2015 at 10:04 AM, Pine W wiki.pine@gmail.com wrote:
Speaking of specialized lists, I'd like to suggest that this discussion would be well suited to Research-l, where many people who are interested in these kinds of questions read and write about them more frequently than they do on Wikimedia-l. I'm boldly adding that list to the recipients for this thread.
I have some thoughts about the substance of this discussion but they're a bit rushed at the moment. I may write more later.
Regards, Pine On Jun 2, 2015 4:32 AM, "Milos Rancic" millosh@gmail.com wrote:
Luis, I have to say that you are the first person on WMF side who has substantially engaged into this issue and I am very glad to see that :)
The products of your work are of the highest importance, as the community is the most important part of our movement, not to say that it's the movement itself.
I am finally relieved to know that we are on the path to rationally understand what's going on inside of the community after short 14.5 years.
It would be good if you'd share your results with the rest of us.
As for this list: As MZ said, this list is important. However, there is no doubt that it's far from being the only or even the most important indicator of community health. It is just about one of the rare publicly accessible data which could give a clue of what's going on inside of the community, but could mislead, as well. On Jun 2, 2015 04:39, "Luis Villa" lvilla@wikimedia.org wrote:
On Mon, Jun 1, 2015 at 10:57 AM, Milos Rancic millosh@gmail.com
wrote:
On Mon, Jun 1, 2015 at 7:51 PM, Luis Villa lvilla@wikimedia.org
wrote:
On Mon, Jun 1, 2015 at 9:26 AM, Andrew Lih andrew.lih@gmail.com
wrote:
> 3. Participation in the mailing list may be a misleading
indicator of
> activity or interest, as other regional or specialized forums (eg. > Facebook, GLAM-oriented lists, etc) have emerged in recent years. >
Let me second this. My department is thinking about community
health
metrics (constructive suggestions welcome!), but I would not
personally
propose mailing list participation (especially this list) as a good
metric
- decreased participation here may reflect many, many things, only
some
of
which are actually negative.
This is not the only one indicator, but it's pretty consistent since 2011 (take a look into [1]). In other words, something happened in May. Maybe it's actually about the elections because people used
other
means of communication for that.
Looking briefly at some of the highest-traffic months, it could simply
be
that people got tired of discussing high-controversy topics here. (Flamewars are good for traffic volume; not so great for community
health.)
I'm sure Facebook's increased acceptance also has a role. I suspect
also
that some announcements that used to come here now go to other, more specialized mailing lists.
That last one points to a key thing: as MZ says, many people are
subscribed
to this list, but many don't read and don't participate, because this mailing list has an *awful* reputation, and people who want to get
things
done are going elsewhere. So "the decline of wikimedia-l" may be a
sign of
bad health of the overall community, or it may simply mean that the
healthy
and constructive parts of the community has moved elsewhere.
To re-iterate what I said in the last email, I'm all ears for
suggestions
on creative community metrics. I'll add here that I'm also very open to suggestions on what a new wikimedia-l might look like. (I know some
FOSS
communities are having good experiences with discourse.org, for
example.)
No commitment that WMF can act on either immediately, of course, but I think it is worth starting both of those discussions.
Luis
-- Luis Villa Sr. Director of Community Engagement Wikimedia Foundation *Working towards a world in which every single human being can freely
share
in the sum of all knowledge.* _______________________________________________ Wikimedia-l mailing list, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines Wikimedia-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, mailto:wikimedia-l-request@lists.wikimedia.org?subject=unsubscribe
Wikimedia-l mailing list, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines Wikimedia-l@lists.wikimedia.org https://meta.wikimedia.org/wiki/Mailing_lists/GuidelinesWikimedia-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, mailto:wikimedia-l-request@lists.wikimedia.org?subject=unsubscribe
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Am 04.06.2015 um 16:33 schrieb Samuel Klein meta.sj@gmail.com:
Context: reduced traffic on wikimedia-l. Is this a sign of poor community health?
Reduced traffic on Wikimedia-l is mostly due to list moderation. All critical content has been filtered for a while. I became aware of it only recently when I posted a critical remark which was rejected.
The Foundation has not only introduced Superprotect and Superban, it has also got a firm grip on all communication channels whatsoever. This will intensify as, we have just learned, new staff will be hired for communication. The WMF no longer needs the community, it does all the traffic itself (staff, chapters, etc.).
Of course this shift from crowdsourcing to staff has to be paid for, hence the interest in an ever-increasing flow of donations and hence the interest in the Alexa ranking of Wikipedia.
Best, Jürgen.
Hi Juergen, That's an interesting hypothesis. Do you have any evidence to support it?
On Thu, Jun 4, 2015 at 9:50 AM, Juergen Fenn jfenn@gmx.net wrote:
Am 04.06.2015 um 16:33 schrieb Samuel Klein meta.sj@gmail.com:
Context: reduced traffic on wikimedia-l. Is this a sign of poor community health?
Reduced traffic on Wikimedia-l is mostly due to list moderation. All critical content has been filtered for a while. I became aware of it only recently when I posted a critical remark which was rejected.
The Foundation has not only introduced Superprotect and Superban, it has also got a firm grip on all communication channels whatsoever. This will intensify as, we have just learned, new staff will be hired for communication. The WMF no longer needs the community, it does all the traffic itself (staff, chapters, etc.).
Of course this shift from crowdsourcing to staff has to be paid for, hence the interest in an ever-increasing flow of donations and hence the interest in the Alexa ranking of Wikipedia.
Best, Jürgen. _______________________________________________ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Am 04.06.2015 um 16:55 schrieb Aaron Halfaker aaron.halfaker@gmail.com:
Hi Juergen, That's an interesting hypothesis. Do you have any evidence to support it?
Of course, I cannot tell which guidelines are in force from the Foundation's point of view. As I said, I can tell from my personal experience. I used to post to several lists for many years. I mostly refrained from participation due to the shift in policy that was performed at London Wikimania without seeking prior consent from the community. The board declared it no longer needs the community; those critical of the superprotect turn should take a wikibreak. In fact, most of them quit the project altogether. Wikimedia-l became more official a channel than ever. No more debates, mostly a channel for announcements and thank yous from staff and affiliates. Now I tried to take part in it lately and found out why my posting did not appear on the list. Ah. We are already at step two: a rather strict moderation. It's no more critics, only a forum for applause. Been there, seen that. But it all makes perfect sense.
Best, Jürgen.
Hi Aaron, which of Juergen's statements do you mean?
my question is: do you have any evidence for the contrary?
best, Claudia
---------- Original Message ----------- From:Aaron Halfaker aaron.halfaker@gmail.com To:Research into Wikimedia content and communities <wiki-research- l@lists.wikimedia.org> Sent:Thu, 4 Jun 2015 09:55:02 -0500 Subject:Re: [Wiki-research-l] Community health (retitled thread)
Hi Juergen, That's an interesting hypothesis. Do you have any evidence to support it?
On Thu, Jun 4, 2015 at 9:50 AM, Juergen Fenn jfenn@gmx.net wrote:
Am 04.06.2015 um 16:33 schrieb Samuel Klein meta.sj@gmail.com:
Context: reduced traffic on wikimedia-l. Is this a sign of poor community health?
Reduced traffic on Wikimedia-l is mostly due to list moderation. All critical content has been filtered for a while. I became aware of it only recently when I posted a critical remark which was rejected.
The Foundation has not only introduced Superprotect and Superban, it
has
also got a firm grip on all communication channels whatsoever. This will intensify as, we have just learned, new staff will be hired for communication. The WMF no longer needs the community, it does all the traffic itself (staff, chapters, etc.).
Of course this shift from crowdsourcing to staff has to be paid for, hence the interest in an ever-increasing flow of donations and hence the
interest
in the Alexa ranking of Wikipedia.
Best, Jürgen. _______________________________________________ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
------- End of Original Message -------
Hi Claudia,
which of Juergen's statements do you mean?
All of them, but mostly the explanation for the drop in traffic.
do you have any evidence for the contrary?
Please don't assume that my call for evidence suggests my disagreement. I'm an empiricist and this is the research mailing list. IMO, claims need evidence or should be carefully framed as speculation or hypothesis. In this context, it is good practice to request that those making statements of fact produce justification.
It seems like it would be helpful to move this conversation forward if someone were to find the dates of the policy change and compare it to the rate of moderated messages.
I also suggest that any further discussion about WMF policies or board decisions (outside of their measurable effects, theoretical implications, etc.) be taken to a more appropriate forum.
-Aaron
On Thu, Jun 4, 2015 at 10:05 AM, koltzenburg@w4w.net wrote:
Hi Aaron, which of Juergen's statements do you mean?
my question is: do you have any evidence for the contrary?
best, Claudia
---------- Original Message ----------- From:Aaron Halfaker aaron.halfaker@gmail.com To:Research into Wikimedia content and communities <wiki-research- l@lists.wikimedia.org> Sent:Thu, 4 Jun 2015 09:55:02 -0500 Subject:Re: [Wiki-research-l] Community health (retitled thread)
Hi Juergen, That's an interesting hypothesis. Do you have any evidence to support it?
On Thu, Jun 4, 2015 at 9:50 AM, Juergen Fenn jfenn@gmx.net wrote:
Am 04.06.2015 um 16:33 schrieb Samuel Klein meta.sj@gmail.com:
Context: reduced traffic on wikimedia-l. Is this a sign of poor community health?
Reduced traffic on Wikimedia-l is mostly due to list moderation. All critical content has been filtered for a while. I became aware of it
only
recently when I posted a critical remark which was rejected.
The Foundation has not only introduced Superprotect and Superban, it
has
also got a firm grip on all communication channels whatsoever. This
will
intensify as, we have just learned, new staff will be hired for communication. The WMF no longer needs the community, it does all the traffic itself (staff, chapters, etc.).
Of course this shift from crowdsourcing to staff has to be paid for,
hence
the interest in an ever-increasing flow of donations and hence the
interest
in the Alexa ranking of Wikipedia.
Best, Jürgen. _______________________________________________ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
------- End of Original Message -------
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Anecdata, but: as someone who no longer posts to wikimedia-l, I stopped posting because I find it a fundamentally toxic place to be.
On 4 June 2015 at 13:55, Aaron Halfaker aaron.halfaker@gmail.com wrote:
Hi Claudia,
which of Juergen's statements do you mean?
All of them, but mostly the explanation for the drop in traffic.
do you have any evidence for the contrary?
Please don't assume that my call for evidence suggests my disagreement. I'm an empiricist and this is the research mailing list. IMO, claims need evidence or should be carefully framed as speculation or hypothesis. In this context, it is good practice to request that those making statements of fact produce justification.
It seems like it would be helpful to move this conversation forward if someone were to find the dates of the policy change and compare it to the rate of moderated messages.
I also suggest that any further discussion about WMF policies or board decisions (outside of their measurable effects, theoretical implications, etc.) be taken to a more appropriate forum.
-Aaron
On Thu, Jun 4, 2015 at 10:05 AM, koltzenburg@w4w.net wrote:
Hi Aaron, which of Juergen's statements do you mean?
my question is: do you have any evidence for the contrary?
best, Claudia
---------- Original Message ----------- From:Aaron Halfaker aaron.halfaker@gmail.com To:Research into Wikimedia content and communities <wiki-research- l@lists.wikimedia.org> Sent:Thu, 4 Jun 2015 09:55:02 -0500 Subject:Re: [Wiki-research-l] Community health (retitled thread)
Hi Juergen, That's an interesting hypothesis. Do you have any evidence to support it?
On Thu, Jun 4, 2015 at 9:50 AM, Juergen Fenn jfenn@gmx.net wrote:
Am 04.06.2015 um 16:33 schrieb Samuel Klein meta.sj@gmail.com:
Context: reduced traffic on wikimedia-l. Is this a sign of poor community health?
Reduced traffic on Wikimedia-l is mostly due to list moderation. All critical content has been filtered for a while. I became aware of it only recently when I posted a critical remark which was rejected.
The Foundation has not only introduced Superprotect and Superban, it
has
also got a firm grip on all communication channels whatsoever. This will intensify as, we have just learned, new staff will be hired for communication. The WMF no longer needs the community, it does all the traffic itself (staff, chapters, etc.).
Of course this shift from crowdsourcing to staff has to be paid for, hence the interest in an ever-increasing flow of donations and hence the
interest
in the Alexa ranking of Wikipedia.
Best, Jürgen. _______________________________________________ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
------- End of Original Message -------
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
On Thu, Jun 4, 2015 at 2:53 PM, Oliver Keyes okeyes@wikimedia.org wrote:
Anecdata, but: as someone who no longer posts to wikimedia-l, I stopped posting because I find it a fundamentally toxic place to be.
And related to this, when was the last time anyone recommended to a newbie: subscribe to Wikimedia-L?
The WMF board of trustees election results (any hour now) will be interesting, because it appears voter turnout was double the 2013 numbers while most other indicators of participation in our community have been flat.
Hi Luis, Aaron and all,
Here's a list of possible metrics that we could use for measuring community health.
Introductory notes: * I emphasized the number of unique contributors rather than number of contributions. * All of these metrics can be calculated over a variety of time-frames, although I suggest monthly because that is our existing default for many metrics. * "Number of contributors" == at least 1 contribution during the time-frame * "Number of active contributors" == at least 5 contributions during the time-frame
My suggested list:
* Number of new accounts created that are not blocked within 31 days * Number of new accounts that make 1 edit and are not blocked within 31 days * Number of new accounts that make 5 edits and are not blocked within 31 days * Number of active editors * Number of rolling surviving active new editors (see https://meta.wikimedia.org/wiki/Research:Rolling_monthly_active_editor)
* Percentage of active editors who self-identify as non-male
* Number of articles and files on all wikis that have passed quality reviews
* Number of volunteer publicly logged non-edit actions
* Number of contributors to all public mailing lists * Number of contributors to social media discussions on all platforms * Aggregate sentiment of all discussions relevant to Wikimedia on all platforms
* Number of unique senders of OTRS requests * Number of unique responders to OTRS requests * Number of files with OTRS permission tags * Average wait time for resolution of OTRS requests * Maximum wait time for resolution of OTRS requests
* Number of contributors to Phabricator * Average Phabricator "unbreak now" task wait time to closure * Average Phabricator high priority task wait time to closure * Average Phabricator task wait time to closure * Number of contributors to Gerrit * Number of patches created * Number of code reviewers * Average patch code review wait time * Maximum patch code review wait time
* Number of wikis with at least 100 unique editors active * Number of wikis with at least 1000 unique editors active * Number of wikis with at least 10,000 unique editors active * Number of active admins on all wikis
* Number of files uploaded that are not deleted within 31 days * Number of media files in use on wikis other than Commons
* Number of members of affiliates * Number of volunteers of affiliates (this includes online and offline activities) * Number of affiliate organizational partnerships * Number of active affiliate organizations * Percentage of affiliate funding from sources outside WMF
* Number of unique donors to affiliates * Number of unique donors to WMF * Number of repeat donors to affiliates * Number of repeat donors to WMF * Number of WMF organizational partnerships * Number of active WMF grants * Number of unique active WMF grantees
Additional comments:
Note that the Learning and Evaluation team independently seemed to be doing some work about relevant metrics on https://meta.wikimedia.org/wiki/Grants:Evaluation/Measures_for_evaluation in September 2014.
Finally, I would like to suggest that research about community development in general may be relevant to work on Wikimedia community health online. Suggested reading:
* https://en.wikipedia.org/wiki/Social_psychology * https://en.wikipedia.org/wiki/Social_capital * https://en.wikipedia.org/wiki/Community_building * https://en.wikipedia.org/wiki/Community_development * https://en.wikipedia.org/wiki/Community_indicators * https://en.wikipedia.org/wiki/Community_of_practice * https://en.wikipedia.org/wiki/Urban_planning * https://en.wikipedia.org/wiki/Community_policing * https://en.wikipedia.org/wiki/Sense_of_community * https://en.wikipedia.org/wiki/Social_network https://en.wikipedia.org/wiki/Bowling_Alone I hope that these suggestions help to move the community health conversation forward.
Pine
Here's a list of possible metrics that we could use for measuring
community health.
That's a great list, with some great metrics. I'd be included to add some silo-breaking metrics which measure activity across projects or across silos within projects:
* Number of editors with actions/edits on more than N wikis (N=2, N=3, etc) * Number of editors with actions/edits on more than N namespaces on the same wiki (N=2, N=3, etc) ...
cheers stuart
-- ...let us be heard from red core to black sky
Hello,
As far as it is about me, I can say that I left wikimedia-l twice or three times. I left mainly because of the high amount of mails, also often not very useful mails, "witty" remarks in 1-2 lines for example.
But I think that this is a good example for a quantitative research that should later lead you to a qualitative look. And maybe it is indeed an indicator for something. In systems theory, one might think that the social system shows an internal differentiation so that people go to more specialized lists.
Isnt't there literature about the traffic on mailing lists?
Kind regards, Ziko
2015-06-05 3:27 GMT+02:00 Stuart A. Yeates syeates@gmail.com:
Here's a list of possible metrics that we could use for measuring community health.
That's a great list, with some great metrics. I'd be included to add some silo-breaking metrics which measure activity across projects or across silos within projects:
- Number of editors with actions/edits on more than N wikis (N=2, N=3, etc)
- Number of editors with actions/edits on more than N namespaces on the same
wiki (N=2, N=3, etc) ...
cheers stuart
-- ...let us be heard from red core to black sky
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Hello Ziko,
Am 05.06.2015 um 09:33 schrieb Ziko van Dijk zvandijk@gmail.com:
But I think that this is a good example for a quantitative research that should later lead you to a qualitative look. And maybe it is indeed an indicator for something. In systems theory, one might think that the social system shows an internal differentiation so that people go to more specialized lists.
From the point of view of systems theory what matters is how system Wikimedia draws the line between itself and its environment because that is what constitutes Wikimedia. In other words, how open is Wikimedia still to newbies, different-minded contributors, criticism from within, etc.
What is it that leads to changes in this differentiation between inside and outside the system? Is it due to moderation or to the subscribers leaving, following their interest in certain subjects?
Systems theory deals with an objective description of developments, while the latter would be a matter for those interested in the individual motives for any changes.
Most important: There is no metrics for that, we definitely need a qualitative approach for that.
Isnt't there literature about the traffic on mailing lists?
Of course, there is. ;) Mailing lists have been there since 1972, IIRC. E.g., a search for "mailing list" in First Monday yields 117 articles. Mailing lists are the oldest type of all virtual communities.
Best, Jürgen.
The number one problem with Wikipedia seems to be the assessment of newbies and the communication with them. We often don't have enough information in order to see whether a contribution was made in good or bad faith. We usually simply revert. If the contribution was made in bad faith, that reaction is probably the best. If the contribution was made in good faith, the reaction should be different, trying to pull the newbie into the boat. WMF researchers once examined the "revert ratio" and found out that many new editor contributions are simply reverted. The communication with them consists only of prepared, general texts, if at all. The researchers said: You community must communicate better and write personal texts, that works better. But why do the experienced community members don't like to communicate personally with the newbies? Because they don't a response in 99% of the cases. Communicating especially with bad faith contributors is a waste of time. Also, for technical reasons the newbies usually don't see feedback: they don't know the version history or the talk pages. One way to solve the problem is to make it more likely that communication takes place, and make it easier to asses newbies. Kind regards Ziko
2015-06-05 14:46 GMT+02:00 Juergen Fenn jfenn@gmx.net:
Hello Ziko,
Am 05.06.2015 um 09:33 schrieb Ziko van Dijk zvandijk@gmail.com:
But I think that this is a good example for a quantitative research that should later lead you to a qualitative look. And maybe it is indeed an indicator for something. In systems theory, one might think that the social system shows an internal differentiation so that people go to more specialized lists.
From the point of view of systems theory what matters is how system Wikimedia draws the line between itself and its environment because that is what constitutes Wikimedia. In other words, how open is Wikimedia still to newbies, different-minded contributors, criticism from within, etc.
What is it that leads to changes in this differentiation between inside and outside the system? Is it due to moderation or to the subscribers leaving, following their interest in certain subjects?
Systems theory deals with an objective description of developments, while the latter would be a matter for those interested in the individual motives for any changes.
Most important: There is no metrics for that, we definitely need a qualitative approach for that.
Isnt't there literature about the traffic on mailing lists?
Of course, there is. ;) Mailing lists have been there since 1972, IIRC. E.g., a search for "mailing list" in First Monday yields 117 articles. Mailing lists are the oldest type of all virtual communities.
Best, Jürgen. _______________________________________________ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
HI Ziko,
I agree. That sounds like a TL;DR of my research agenda. :D
- It started with http://www-users.cs.umn.edu/~halfak/publications/The_Rise_and_Decline/ - So I tied to make assessing newcomers easier http://www-users.cs.umn.edu/~halfak/publications/Snuggle/halfaker14snuggle-p... - See also https://en.wikipedia.org/wiki/Wikipedia:Snuggle - And now I'm working on https://meta.wikimedia.org/wiki/Research:Revision_scoring_as_a_service
Feedback and collaboration welcome.
-Aaron
On Fri, Jun 5, 2015 at 11:55 AM, Ziko van Dijk zvandijk@gmail.com wrote:
The number one problem with Wikipedia seems to be the assessment of newbies and the communication with them. We often don't have enough information in order to see whether a contribution was made in good or bad faith. We usually simply revert. If the contribution was made in bad faith, that reaction is probably the best. If the contribution was made in good faith, the reaction should be different, trying to pull the newbie into the boat. WMF researchers once examined the "revert ratio" and found out that many new editor contributions are simply reverted. The communication with them consists only of prepared, general texts, if at all. The researchers said: You community must communicate better and write personal texts, that works better. But why do the experienced community members don't like to communicate personally with the newbies? Because they don't a response in 99% of the cases. Communicating especially with bad faith contributors is a waste of time. Also, for technical reasons the newbies usually don't see feedback: they don't know the version history or the talk pages. One way to solve the problem is to make it more likely that communication takes place, and make it easier to asses newbies. Kind regards Ziko
2015-06-05 14:46 GMT+02:00 Juergen Fenn jfenn@gmx.net:
Hello Ziko,
Am 05.06.2015 um 09:33 schrieb Ziko van Dijk zvandijk@gmail.com:
But I think that this is a good example for a quantitative research that should later lead you to a qualitative look. And maybe it is indeed an indicator for something. In systems theory, one might think that the social system shows an internal differentiation so that people go to more specialized lists.
From the point of view of systems theory what matters is how system
Wikimedia draws the line between itself and its environment because that is what constitutes Wikimedia. In other words, how open is Wikimedia still to newbies, different-minded contributors, criticism from within, etc.
What is it that leads to changes in this differentiation between inside
and outside the system? Is it due to moderation or to the subscribers leaving, following their interest in certain subjects?
Systems theory deals with an objective description of developments,
while the latter would be a matter for those interested in the individual motives for any changes.
Most important: There is no metrics for that, we definitely need a
qualitative approach for that.
Isnt't there literature about the traffic on mailing lists?
Of course, there is. ;) Mailing lists have been there since 1972, IIRC.
E.g., a search for "mailing list" in First Monday yields 117 articles. Mailing lists are the oldest type of all virtual communities.
Best, Jürgen. _______________________________________________ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Yes, but may I also point out that one of our biggest problems on EN wiki is that even good faith newbies will often have their edits reverted. If you add uncited facts to a page you are now much more likely to have your edit reverted than to have someone add <citation needed> so I would suggest a metric that includes persistence v reversion of edits that are not vandalism.
Another issue worth measuring is the number of edit conflicts and the frequency that having an edit conflict triggers a newbies departure. This would require WMF help as I don't think that edit conflicts are publicly logged. But some research on this might resolve the divide between those who consider this a minor issue deserving only the lowest priority at bugzilla, and those such as myself who suspect this is one of the most toxic features of the pedia and reducing edit conflicts the easiest major improvement that could be made.
By contrast commons is a relatively lonely place. From my experience you can do hundreds of thousands of edits there without ever needing to archive your talkpage. It would be interesting to see some community health metrics that looked at how many interactions people have with other editors, whether thanks or talkpage messages. My suspicion is that editor retention will vary by interaction level, and there will be a sweet spot which is best for retention, above this interaction level some people finding things distracting, and below this level people leave because they feel ignored.
Another metric, and probably one best derived from polling organisations who survey the general public would be to identify how many of our readers would fix an error if they spotted it. One of the arguments that our perceived decline in editor recruitment is a cost of quality is the theory that readers who are willing to fix obvious errors are finding fewer errors per hour of reading Wikipedia. I know that casual readers are less likely to spot typos and vandalism than they were a few years ago, but I'm not sure the best way to measure this phenomenon
Regards
Jonathan Cardy
On 5 Jun 2015, at 02:27, Stuart A. Yeates syeates@gmail.com wrote:
Here's a list of possible metrics that we could use for measuring community health.
That's a great list, with some great metrics. I'd be included to add some silo-breaking metrics which measure activity across projects or across silos within projects:
- Number of editors with actions/edits on more than N wikis (N=2, N=3, etc)
- Number of editors with actions/edits on more than N namespaces on the same wiki (N=2, N=3, etc)
...
cheers stuart
-- ...let us be heard from red core to black sky
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
WereSpielChequers werespielchequers@gmail.com writes:
Yes, but may I also point out that one of our biggest problems on EN wiki is that even good faith newbies will often have their edits reverted. [...] By contrast commons is a relatively lonely place.
This comment reminds me that for large wikis things can be quite heterogenous across different parts of the wiki (different subject matter, popular vs. obscure articles, etc.), which entire-wiki-level analyses tend to obscure. For example in my own editing, EN-wiki is also "a relatively lonely place"--- most articles I've created *years* ago have gotten no feedback at all in the time since (no comments on the talk page, no non-housekeeping edits, etc.). For other people, of course, it feels like a crowded place where someone is always stepping on your toes, presumably because they edit different kinds of articles. To understand exactly what's going on with editor culture/retention/etc., I think we need to analyze therefore at a finer level of granularity than "the English Wikipedia" (or "the German Wikipedia", etc.).
-Mark
Am 04.06.2015 um 19:11 schrieb Federico Leva (Nemo) nemowiki@gmail.com:
Reduced traffic on Wikimedia-l is mostly due to list moderation.
That's plausible. "Most" people on wikimedia-l are moderated by now; I and others unsubscribed due to tyrannical moderation, too.
Well, not exactly tyrannical, as there obviously is a plan behind it not to let anything critical sound on a list that no longer serves the community, but that is just another tool for corporate communication.
Of course, I also unsubscribed from the list, and I will read it from the archive only because I don't subscribe to corporate communication lists.
I agree to Aaron that we should concentrate on factual data for research. However, I'd like to give you an idea of what nowadays is no longer possible on Wikimedia-l because this also serves as an indicator for the "health" of Wikipedia.
The text of my censored email read: "Jan-Bart, you might be aware that it was you that drove many "talented candidates" out of the movement last year. So, no more comment."
And the moderator gave this comment for his decision: "I could not find anything positive in your message. How would it help the goals of the Wikimedia movement? Regards, Richard."
That's the way Wikipedia works in 2015. So, no more comments about "community health" or whoever's health. I'm off.
Best, Jürgen.
"We should concentrate on factual data for research" in a long email about how everything is ruined forever because a moderator couldn't find anything of value in an uncited claim that Jan-Bart actively drove people away?
This must be what people mean by "mixed methods" ;)
On 4 June 2015 at 18:29, Juergen Fenn jfenn@gmx.net wrote:
Am 04.06.2015 um 19:11 schrieb Federico Leva (Nemo) nemowiki@gmail.com:
Reduced traffic on Wikimedia-l is mostly due to list moderation.
That's plausible. "Most" people on wikimedia-l are moderated by now; I and others unsubscribed due to tyrannical moderation, too.
Well, not exactly tyrannical, as there obviously is a plan behind it not to let anything critical sound on a list that no longer serves the community, but that is just another tool for corporate communication.
Of course, I also unsubscribed from the list, and I will read it from the archive only because I don't subscribe to corporate communication lists.
I agree to Aaron that we should concentrate on factual data for research. However, I'd like to give you an idea of what nowadays is no longer possible on Wikimedia-l because this also serves as an indicator for the "health" of Wikipedia.
The text of my censored email read: "Jan-Bart, you might be aware that it was you that drove many "talented candidates" out of the movement last year. So, no more comment."
And the moderator gave this comment for his decision: "I could not find anything positive in your message. How would it help the goals of the Wikimedia movement? Regards, Richard."
That's the way Wikipedia works in 2015. So, no more comments about "community health" or whoever's health. I'm off.
Best, Jürgen. _______________________________________________ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Am 05.06.2015 um 01:05 schrieb Oliver Keyes okeyes@wikimedia.org:
"We should concentrate on factual data for research" in a long email about how everything is ruined forever because a moderator couldn't find anything of value in an uncited claim that Jan-Bart actively drove people away?
This must be what people mean by "mixed methods" ;)
It has really been a very long road of frustration that lies behind the Community and what they call the "Wikimedia movement". I'd bet someone from within the Wikimedia apparatus with a Wikimedia email address would argue like that.
I think what Nemo (Nemo, please correct me if I'm wrong) and I wanted to get across was that you have no right to speak like that vis-à-vis a volunteer who, as far as I am concerned, has spent ten years of his life building up a project that today has grown up to make /your/ living. It was yesterday exactly ten years ago that I made my first edit on Wikipedia. This is what /we/ call the community. You lack respect and tact towards the volunteers. I am sorry that I have to say so.
The point is that Wikimedia-l does not have a blacklist, but rather a Whitelist. Moderation happened after I subscribed to the list with a different email address. It's really that closed a circle.
Now, where should we go to discuss this now that there is no more Wikimedia-l left for that? I for one will not post any more to the research list about this matter because I gather its readers are in fact interested in other things. So, it's really EOD from my side.
Best, Jürgen.
wiki-research-l@lists.wikimedia.org