Context: reduced traffic on wikimedia-l. Is this a sign of poor community health? To what extent has community energy and activity moved to other fora? Is list activity (for certain lists) a useful measure of activity and enthusiasm (about certain parts of the projects)? On Thu, Jun 4, 2015 at 7:18 AM, Aaron Halfaker <aaron.halfaker(a)gmail.com> wrote: -- Samuel Klein @metasj w:user:sj +1 617 529 4266

Hi Juergen, That's an interesting hypothesis. Do you have any evidence to support it? On Thu, Jun 4, 2015 at 9:50 AM, Juergen Fenn <jfenn(a)gmx.net> wrote:

Hi Aaron, which of Juergen's statements do you mean? my question is: do you have any evidence for the contrary? best, Claudia ---------- Original Message ----------- From:Aaron Halfaker &lt;aaron.halfaker(a)gmail.com&gt; To:Research into Wikimedia content and communities <wiki-research- l(a)lists.wikimedia.org&gt; Sent:Thu, 4 Jun 2015 09:55:02 -0500 Subject:Re: [Wiki-research-l] Community health (retitled thread) has interest ------- End of Original Message -------

Anecdata, but: as someone who no longer posts to wikimedia-l, I stopped posting because I find it a fundamentally toxic place to be. On 4 June 2015 at 13:55, Aaron Halfaker &lt;aaron.halfaker(a)gmail.com&gt; wrote: -- Oliver Keyes Research Analyst Wikimedia Foundation

Hi Luis, Aaron and all, Here's a list of possible metrics that we could use for measuring community health. Introductory notes: * I emphasized the number of unique contributors rather than number of contributions. * All of these metrics can be calculated over a variety of time-frames, although I suggest monthly because that is our existing default for many metrics. * "Number of contributors" == at least 1 contribution during the time-frame * "Number of active contributors" == at least 5 contributions during the time-frame My suggested list: * Number of new accounts created that are not blocked within 31 days * Number of new accounts that make 1 edit and are not blocked within 31 days * Number of new accounts that make 5 edits and are not blocked within 31 days * Number of active editors * Number of rolling surviving active new editors (see https://meta.wikimedia.org/wiki/Research:Rolling_monthly_active_editor) * Percentage of active editors who self-identify as non-male * Number of articles and files on all wikis that have passed quality reviews * Number of volunteer publicly logged non-edit actions * Number of contributors to all public mailing lists * Number of contributors to social media discussions on all platforms * Aggregate sentiment of all discussions relevant to Wikimedia on all platforms * Number of unique senders of OTRS requests * Number of unique responders to OTRS requests * Number of files with OTRS permission tags * Average wait time for resolution of OTRS requests * Maximum wait time for resolution of OTRS requests * Number of contributors to Phabricator * Average Phabricator "unbreak now" task wait time to closure * Average Phabricator high priority task wait time to closure * Average Phabricator task wait time to closure * Number of contributors to Gerrit * Number of patches created * Number of code reviewers * Average patch code review wait time * Maximum patch code review wait time * Number of wikis with at least 100 unique editors active * Number of wikis with at least 1000 unique editors active * Number of wikis with at least 10,000 unique editors active * Number of active admins on all wikis * Number of files uploaded that are not deleted within 31 days * Number of media files in use on wikis other than Commons * Number of members of affiliates * Number of volunteers of affiliates (this includes online and offline activities) * Number of affiliate organizational partnerships * Number of active affiliate organizations * Percentage of affiliate funding from sources outside WMF * Number of unique donors to affiliates * Number of unique donors to WMF * Number of repeat donors to affiliates * Number of repeat donors to WMF * Number of WMF organizational partnerships * Number of active WMF grants * Number of unique active WMF grantees Additional comments: Note that the Learning and Evaluation team independently seemed to be doing some work about relevant metrics on https://meta.wikimedia.org/wiki/Grants:Evaluation/Measures_for_evaluation in September 2014. Finally, I would like to suggest that research about community development in general may be relevant to work on Wikimedia community health online. Suggested reading: * https://en.wikipedia.org/wiki/Social_psychology * https://en.wikipedia.org/wiki/Social_capital * https://en.wikipedia.org/wiki/Community_building * https://en.wikipedia.org/wiki/Community_development * https://en.wikipedia.org/wiki/Community_indicators * https://en.wikipedia.org/wiki/Community_of_practice * https://en.wikipedia.org/wiki/Urban_planning * https://en.wikipedia.org/wiki/Community_policing * https://en.wikipedia.org/wiki/Sense_of_community * https://en.wikipedia.org/wiki/Social_network <https://en.wikipedia.org/wiki/Bowling_Alone> I hope that these suggestions help to move the community health conversation forward. Pine

Hello, As far as it is about me, I can say that I left wikimedia-l twice or three times. I left mainly because of the high amount of mails, also often not very useful mails, "witty" remarks in 1-2 lines for example. But I think that this is a good example for a quantitative research that should later lead you to a qualitative look. And maybe it is indeed an indicator for something. In systems theory, one might think that the social system shows an internal differentiation so that people go to more specialized lists. Isnt't there literature about the traffic on mailing lists? Kind regards, Ziko 2015-06-05 3:27 GMT+02:00 Stuart A. Yeates &lt;syeates(a)gmail.com&gt;om>:

The number one problem with Wikipedia seems to be the assessment of newbies and the communication with them. We often don't have enough information in order to see whether a contribution was made in good or bad faith. We usually simply revert. If the contribution was made in bad faith, that reaction is probably the best. If the contribution was made in good faith, the reaction should be different, trying to pull the newbie into the boat. WMF researchers once examined the "revert ratio" and found out that many new editor contributions are simply reverted. The communication with them consists only of prepared, general texts, if at all. The researchers said: You community must communicate better and write personal texts, that works better. But why do the experienced community members don't like to communicate personally with the newbies? Because they don't a response in 99% of the cases. Communicating especially with bad faith contributors is a waste of time. Also, for technical reasons the newbies usually don't see feedback: they don't know the version history or the talk pages. One way to solve the problem is to make it more likely that communication takes place, and make it easier to asses newbies. Kind regards Ziko 2015-06-05 14:46 GMT+02:00 Juergen Fenn &lt;jfenn(a)gmx.net&gt;et>:

Yes, but may I also point out that one of our biggest problems on EN wiki is that even good faith newbies will often have their edits reverted. If you add uncited facts to a page you are now much more likely to have your edit reverted than to have someone add <citation needed> so I would suggest a metric that includes persistence v reversion of edits that are not vandalism. Another issue worth measuring is the number of edit conflicts and the frequency that having an edit conflict triggers a newbies departure. This would require WMF help as I don't think that edit conflicts are publicly logged. But some research on this might resolve the divide between those who consider this a minor issue deserving only the lowest priority at bugzilla, and those such as myself who suspect this is one of the most toxic features of the pedia and reducing edit conflicts the easiest major improvement that could be made. By contrast commons is a relatively lonely place. From my experience you can do hundreds of thousands of edits there without ever needing to archive your talkpage. It would be interesting to see some community health metrics that looked at how many interactions people have with other editors, whether thanks or talkpage messages. My suspicion is that editor retention will vary by interaction level, and there will be a sweet spot which is best for retention, above this interaction level some people finding things distracting, and below this level people leave because they feel ignored. Another metric, and probably one best derived from polling organisations who survey the general public would be to identify how many of our readers would fix an error if they spotted it. One of the arguments that our perceived decline in editor recruitment is a cost of quality is the theory that readers who are willing to fix obvious errors are finding fewer errors per hour of reading Wikipedia. I know that casual readers are less likely to spot typos and vandalism than they were a few years ago, but I'm not sure the best way to measure this phenomenon Regards Jonathan Cardy

Juergen Fenn, 04/06/2015 16:50: That's plausible. "Most" people on wikimedia-l are moderated by now; I and others unsubscribed due to tyrannical moderation, too. Nemo

"We should concentrate on factual data for research" in a long email about how everything is ruined forever because a moderator couldn't find anything of value in an uncited claim that Jan-Bart actively drove people away? This must be what people mean by "mixed methods" ;) On 4 June 2015 at 18:29, Juergen Fenn &lt;jfenn(a)gmx.net&gt; wrote: -- Oliver Keyes Research Analyst Wikimedia Foundation