Relevant: https://meta.wikimedia.org/wiki/Chapters_Dialogue

On Mon, Jan 9, 2017 at 12:23 PM, Jonathan Morgan jmorgan@wikimedia.org wrote:

Hi Aisha,

Interesting question. I haven't read anything that fits this description, but you may want to take a look at the work of Iolanda Pensa[1] and Darius Jemielniak[2], both of whom are researchers and also active in Movement governance.

1. http://repository.supsi.ch/2138/
2. http://www.sup.org/books/title/?id=24010

On Sun, Jan 8, 2017 at 5:41 PM, Aisha Brady aishabrady@gmail.com wrote:

...

Hi!

Could anyone point me towards any papers relevant to Wikimedia chapters (how they function, the work they do, whether they have been successful or otherwise)?

Thank you! :)

Aisha

---

Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

-- Jonathan T. Morgan Senior Design Researcher Wikimedia Foundation User:Jmorgan (WMF) https://meta.wikimedia.org/wiki/User:Jmorgan_(WMF)

---

Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

My personal 10c on this having been a chapter member for several years and a chapter committee member for some of those years is that there are the chapters who get annual funding and those who don’t. If you don’t get annual funding, then you have no staff member who can do the day-to-day administrative work (every organisation has to submit forms to their government, organise auditing, keep the web site updated, do the bookkeeping, etc) so this work has to be farmed out to the members, which means that sometimes you have nobody with the right skills (responsibilities of treasurers make it a particularly difficult role to fill) and that you use up all of people’s time and goodwill in doing the day-to-day stuff instead of doing the exciting projects you hoped you’d be doing as a chapter member. Contrary to what WMF think ,there is a lot of work involved in writing grant applications and, when you are doing it with lots of volunteers each with randoms skills and only a certain amount of spare time, generally some people let you down (family issues, busy at work, or maybe just don’t know how to write the section allocated to them) and it doesn’t get finished to meet the deadline, which is then a waste of the time of the people who did their share of the work. The net result is a somewhat demoralising downward spiral with fewer members, burned-out committee people, and fewer achievements. I’ve pretty much abandoned trying to work chapter-wide and just try to do what I can in my own local area.

WMF strongly pushes you to use volunteer time in a chapter, but overlooks practical realities. Engagement with GLAMs almost always involves weekday meetings; most volunteers are not available on weekdays due to their own employment. I have 7 upcoming GLAM sessions in the next 3 weeks (all for 1Lib1Ref) all on weekdays and despite my call for help to both chapter members and the Australian noticeboard, nobody is volunteering; I guess I am doing them all myself (assuming I don’t have conflicting commitments). Even committee meetings are very hard to schedule across 4 time zones with everyone with different working hours, different commitments to family events etc on the weekends, and technology problems with phones/computers often waste a lot of the meeting time (some people can’t get Hangouts to work for them, other people’s microphones cut out randomly, etc). Our chapter has never met face to face.

It’s really hard to get a chapter off the ground with a small population over a large geography without any staff. You get some stupid suggestions like “why do you need to be incorporated, ditch that and save all that time and admin cost” by people who clearly do not understand that without a legal entity to take out public liability insurance, the committee members or organisers of chapter events would bear the risk personally of anything going wrong, i.e. under our law, your house could be sold to pay for someone’s injuries.

I agree with Pine’s comments about impact. Given that we have no way of knowing which editors on en.WP are from our country, we have no way to measure impact of anything we do other than small group events where the individuals are willing to disclose their user IDs. Take the upcoming 1Lib1Ref sessions, I have no way of knowing the user IDs of the librarians I will be addressing or who the recipients of my electronically-distributed material are. I will not be able to tell if there is 1 or 1,000 citations added as a result of my efforts. 1Lib1Ref can presumably analyse edit summaries world-wide to see how successful it was overall but we can’t easily attribute it to specific countries. When you have a chapter that is closely aligned with a specific language of Wikipedia, you may have some chance of seeing impact by looking at overall behaviour on that Wikipedia. When you are a small nation that is one of many English speaking countries, you have no way of knowing. How do you define a success metric in these circumstances? And the lack of any indicator that your efforts are successful makes it difficult for people to sustain enthusiasm to make these efforts (I could be just wasting my time). Unless WMF is willing to create tools that map IP addresses to country/city so we can do some kind of query about readership or contribution from our area, how can we measure impact, understand trends etc. But, hey, we don’t even get a chance to discuss these issues with anyone; I’m still waiting for several months for anyone from WMF to respond to my question of who has responsibility for training.

My overall point here is that chapters are very different. I suspect if you consider chapters across the range of issues I’ve outlined, you will find very few are directly comparable in terms of how they operate and there is virtually no way to measure their impact.

Kerry

From: Wiki-research-l [mailto:wiki-research-l-bounces@lists.wikimedia.org] On Behalf Of Pine W Sent: Tuesday, 10 January 2017 6:46 AM To: Research into Wikimedia content and communities wiki-research-l@lists.wikimedia.org Cc: Jaime Anstee janstee@wikimedia.org; Katy Love klove@wikimedia.org Subject: Re: [Wiki-research-l] Chapters

Hi Aisha,

I suggest that you contact Jaime Anstee and/or Katy Love (cc'd here) about this subject, because they are WMF staff who do a lot of work with grantmaking and performance evaluation for chapters. They might know of some analyses that could help you.

Discussions about what kinds of resources, and what quantities of resources, to allocate to the chapters vs. smaller affiliates, other kinds of grants, and WMF-run work that focuses on content and community development, have been happening for years, and are likely to continue for the foreseeable future.

Different chapters function differently, partly because of varied cultural and legal contexts, so there is not a monolithic model of how a chapter should run. The definition of "successful" varies from affiliate to affiliate.

There has been a discussion for years about how to define and quantify affiliate "impact"; my personal preference is to abolish are use of that word. (:

Pine

On Sun, Jan 8, 2017 at 5:41 PM, Aisha Brady <aishabrady@gmail.com mailto:aishabrady@gmail.com > wrote:

Hi!

Could anyone point me towards any papers relevant to Wikimedia chapters (how they function, the work they do, whether they have been successful or otherwise)?

Thank you! :)

Aisha

_______________________________________________ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org mailto:Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Hi, Kerry.

Thanks for sharing these thoughts. I know what you're talking about, and I think they are important to express, for the benefit of those who do not have experience with the kinds of activities running a chapter requires. (I do.)

Some comments, inline:

(Pardon the length of this e-mail. I thought it a good opportunity to engage with this important topic, and I hope I offer some new thoughts to at least some of you.)

On Mon, Jan 9, 2017 at 4:50 PM Kerry Raymond kerry.raymond@gmail.com wrote:

My personal 10c on this having been a chapter member for several years and a chapter committee member for some of those years is that there are the chapters who get annual funding and those who don’t.

And there are chapters (and non-chapters) that get funding on a project basis, which *can* include some compensation for lost days of employment. If you think this could help WMAU, I encourage you to discuss it with one of WMF's grant program officers. See [[m:Grants:Start]].

Also, chapters can transition from project funding to annual funding, given a track record of accountable spending and organizational maturity. Some example of chapters that have made that transition in the last couple of years are Wikimedia Ukraine and Wikimedia Spain.

If you don’t get annual funding, then you have no staff member who can do the day-to-day administrative work (every organisation has to submit forms to their government, organise auditing, keep the web site updated, do the bookkeeping, etc) so this work has to be farmed out to the members, which means that sometimes you have nobody with the right skills (responsibilities of treasurers make it a particularly difficult role to fill) and that you use up all of people’s time and goodwill in doing the day-to-day stuff instead of doing the exciting projects you hoped you’d be doing as a chapter member.

In addition to the possibility of getting reimbursed for time off work, grants proposals can be written to include paying contractors for services like bookkeeping, auditing, and Web site management. I recognize it *usually* falls to the (active among the) board members of a small chapter, but it doesn't have to, with a bit of planning.

Contrary to what WMF think ,there is a lot of work involved in writing grant applications

I wonder what makes you say that. As a former grantmaker at WMF, I know I have never considered writing grant applications easy (or fun), and certainly never said anything of the sort. I do think WMF has taken some pains over the years to make the process _as_ painless as possible (not pain-free, to be sure), and I think we're doing fairly well compared to most traditional grantmakers. But I know I speak for my team when I say none of us thinks this is not a lot of work.

If, on the other hand, you meant to say "it's more work than a group of volunteers can be expected to do" -- that I do disagree with. There is ample empirical evidence that many groups of volunteers *do* manage to write grant applications, and with some care and good judgment, they also manage to grow the organization, both in active and engaged volunteer base, and in organizational capacity up to and including hiring employees.

and, when you are doing it with lots of volunteers each with randoms skills and only a certain amount of spare time, generally some people let you down (family issues, busy at work, or maybe just don’t know how to write the section allocated to them) and it doesn’t get finished to meet the deadline, which is then a waste of the time of the people who did their share of the work. The net result is a somewhat demoralising downward spiral with fewer members, burned-out committee people, and fewer achievements. I’ve pretty much abandoned trying to work chapter-wide and just try to do what I can in my own local area.

This is certainly demoralizing, and I am sorry to hear this has been your experience. I have heard similar experiences from others in Australia. There is no doubt that in addition to objective difficulties (huge country, widely distributed community), there have also been some difficulties caused by some internal strife and poor decisions in WMAU's past, which must have contributed to the downward spiral, as you aptly call it, of frustration, demotivation, and burnout.

The only known cure to such a downward spiral is resisting it with an upward spiral: finding new people, or at least new energy, to put in work despite the frustration and despite past disappointments, utilizing all available resources (e.g. those I described above, and several others that had perhaps not been available in previous years), and creating an upward spiral of small successes, motivating and drawing others, and growing anew. Perhaps you are on your way to doing this, with the activities you mentioned taking on; perhaps you do not feel up to doing this, and I am not laying that burden on your shoulders; I am merely sharing a thought about what it would take to stop the downward spiral.

WMF strongly pushes you to use volunteer time in a chapter, but overlooks

practical realities. Engagement with GLAMs almost always involves weekday meetings; most volunteers are not available on weekdays due to their own employment. I have 7 upcoming GLAM sessions in the next 3 weeks (all for 1Lib1Ref) all on weekdays and despite my call for help to both chapter members and the Australian noticeboard, nobody is volunteering; I guess I am doing them all myself (assuming I don’t have conflicting commitments). Even committee meetings are very hard to schedule across 4 time zones with everyone with different working hours, different commitments to family events etc on the weekends, and technology problems with phones/computers often waste a lot of the meeting time (some people can’t get Hangouts to work for them, other people’s microphones cut out randomly, etc). Our chapter has never met face to face.

Again, I am sorry to hear you have not been able to get help from your colleagues. I am interested in seeing what might be done to help WMAU. I encourage you to consider me a contact point at WMF, and we can discuss some options and plans, if you'd like.

I agree with Pine’s comments about impact. Given that we have no way of knowing which editors on en.WP are from our country, we have no way to measure impact of anything we do other than small group events where the individuals are willing to disclose their user IDs.

I hear you. I think it's a major problem for many chapters. However, we (at WMF) *do*, in fact, have a way to measure the amount of active editors in a given country. I have been advocating that WMF make this data available publicly for more than 3 years now, but, despite at one point getting the Legal dept.'s blessing, have not been able to overcome internal concerns that a sufficiently-determined attacker could use this data to determine an individual editor's country. You can see some public evidence of this discussion in this Phabricator ticket: https://phabricator.wikimedia.org/T131280

In the meantime, I do have access to those numbers, and can tell you that the average active editor (5+ edits/month) count on English Wikipedia from Australia is between 1210 and 1250 editors. (This broad bucket is a precaution. I am happy to share a precise figure with you privately, as a trusted user, until we manage to make the data [probably bucketed like above] public.) The very active editor (100+ edits/month) count of the same is between 140 and 170. WMF has these numbers going back to 2012. I am happy to help you get some snapshots of those numbers to help you assess the size of the editing community in Australia in the past few years.

My overall point here is that chapters are very different. I suspect if you consider chapters across the range of issues I’ve outlined, you will find very few are directly comparable in terms of how they operate and there is virtually no way to measure their impact.

It's difficult, to be sure, but we're not quite prepared to give up on at least attempting to measure impact, or at least some aspects of impact. We have adjusted our expectations after several years of thinking about impact so that we no longer pretend we have a good grasp of what impact looks like in every community or program type. But we try, and we hope we are improving our tools and perspectives for measuring at least some aspects. My colleagues at the Learning and Evaluation team are eager thought-partners for developing a better understanding of impact.

A.

Hoi, With logic like "return on investment" you favour big over important. So no, please no. Thanks, GerardM

On 10 January 2017 at 07:23, Pine W wiki.pine@gmail.com wrote:

To clarify my earlier comment about the term "impact": this has been used as a term of art by WMF in ways that I think are difficult even for native English speakers to grasp without specific instruction in how WMF uses the term. In practice, among grantees, the term seems to be used to mean a variety of things: "outcome", "output", "success", etc. I am hopeful that we can discontinue use of the word "impact" because of its confusing and varied uses in practice.

I am in favor of attempting to quantify how much return on investment is received on the money and time (including precious volunteer time) invested in and by the affiliates and the people who participate in affiliate work. I suggest using terms other than "impact" to describe these returns on investment.

I share a number of Kerry's frustrations with WMF grantmaking for affiliates; some of those frustrations were factors in my decision to significantly decrease my involvement in Cascadia Wikimedians, although there were other significant factors as well.

Pine

---

Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Hoi, Return on investment is in our context all too arbitrary. Ask yourself; is investing in gender gap important but does it make the best return on investment. At the time I invested in documenting every person who died in Wikidata. It was a good investment because now people have taken over from me. Now at years end they use Wikidata to know who the "famous" people are.

The question is not bang for the buck. The question is where are we weak and how can we change this. My current project is adding information about the nobility, the monarchs of particularly Asia. I am learning as I am doing this and I blog about it. All the investments in students working on Wikipedia does not make the quality of the subjects I write about better. Many of the stuff I am involved with has a point of view that I find is hardly neutral. It is however not the subjects students are taught.

When a chapter, a community finds that a specific area is important to them, they should be able to do so. Their relevance and work / investment is not to be mistaken for a provable "return on investment". Because of the gender gap I do give more time to the women I find. That awareness is something you cannot measure but it does have a bearing. People with proper historic knowledge could do much more; they would study the relationship between marriages and peace between countries when they are ruled by monarchs. They would bring this out. At this time we do not even have many of the important battles and wars from the past... I am not saying this is more important but it paints the picture.

When you want return on investment, there are the things people do not care about because it means that it changes the way things are. The best return on investment for Wikidata is by replacing red links and wiki links with references to Wikidata items. I dare anyone to find an argument how it will not bring more quality to any Wikipedia.

My point is that we will only look into the things that we know and care for and in the process forget what we do it for. Money / investment is more of the same. I prefer that we trust more and do not measure using our own yard stick.

NB I am into meters and metric myself :) Thanks, GerardM

On 10 January 2017 at 11:30, Jane Darnell jane023@gmail.com wrote:

What's wrong with "return on investment"? And what is a "term of art" exactly? I agree with Kerry and Pine both about the frustrations, but I also agree with Asaf in terms of all the improvements WMF has made. The problem with making a yearly chapter plan is the lack of knowledge on what "impact" (still better than any other word) was achieved the previous year, making estimation nearly impossible. For the Dutch chapter, the various projects (WLM etc) have been able to come up with their own measurements over time. The problem with any new project is that there is never anything to base estimates on. I am a terrible estimator myself (even when I have pretty good data to base my estimate on), but I enjoy finding creative ways to measure things. Right now we are in general terrible at measuring project-related chapter stuff, and the stuff we are good at measuring is hard to share with the people who need it most (see Asaf's comments about active editors).

Last night I had a long skype-chat with my gendergap friends in NL and we were plotting what we can measure now as a way of being able to measure impact after some (soon-to-be-dreamed-up) international women's day editing event in March. One of the problems with measuring edits is the need for anonymity that Asaf and Kerry talk about. So we need to somehow capture aggregated measurements, but how can we do this and how do we define a "gendergap-related edit"? Theoretically this is an edit made either by a new or existing editor -or- about a woman, and either one is prompted not by something random (organic growth model of Wikimedia projects such as Wikipedia), but specifically by something in our gendergap workgroup "output" (whatever that is). The return on investment (=what we get for giving our personal time) is the increase in such edits over time. At the end of the day, we need to measure "our" increase of aggregated edits against the "normal" increase in aggregated edits, and if we can never measure this, why don't we all just shut up and go back to editing? Well I believe that these efforts will at some point become measurable and I have good faith that these efforts are not just "drops in the bucket". Sometimes it helps to just keep trying to reinvent the wheel, and until we do, we keep at least a list of new and improved articles that we are sure were prompted by our efforts (though these are certainly not 100% of all the edits we have inspired).

On Tue, Jan 10, 2017 at 7:52 AM, Gerard Meijssen < gerard.meijssen@gmail.com> wrote:

...

Hoi, With logic like "return on investment" you favour big over important. So no, please no. Thanks, GerardM

On 10 January 2017 at 07:23, Pine W wiki.pine@gmail.com wrote:

...

To clarify my earlier comment about the term "impact": this has been used as a term of art by WMF in ways that I think are difficult even for native English speakers to grasp without specific instruction in how WMF uses the term. In practice, among grantees, the term seems to be used to mean a variety of things: "outcome", "output", "success", etc. I am hopeful that we can discontinue use of the word "impact" because of its confusing and varied uses in practice.

I am in favor of attempting to quantify how much return on investment is received on the money and time (including precious volunteer time) invested in and by the affiliates and the people who participate in affiliate work. I suggest using terms other than "impact" to describe these returns on investment.

I share a number of Kerry's frustrations with WMF grantmaking for affiliates; some of those frustrations were factors in my decision to significantly decrease my involvement in Cascadia Wikimedians, although there were other significant factors as well.

Pine

---

Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

---

Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

---

Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Hoi,

From my perspective, this endless talk, these ever shifting sands prevent

chapters in many ways to branch out and do things that are not necessarily the best from a global point of view but are the best from a local point of view. Do appreciate that many of these discussions are not happening on a level playing field with too much consideration given to the Anglo Saxon point of view and practice.

When I observe the funding and the allocation of money to chapters it is a case in point. For regulatory purposes the Dutch chapter cannot use "Wikipedia" in its funding mission because it is exclusively used by the WMF. At the same time, the Dutch chapter is asked to support fundraising in the Netherlands AND is asked to substantially do its own fundraising. Other chapters do not need funding from the WMF and they do as they see fit, they are not restricted by all this continuous talk.

I have also observed that the WMF has its own agenda and when projects fail because of said agenda, it is still the others who are to blame. This is something I observed in a project that I got funding for. To make it worse, the reason why part of my project failed is remembered but not the part where my project got screwed because prerequisites needed from the WMF were not met.

Ask yourself, why are projects and practices to be adopted by other languages and why is there so little that goes the other way? Do appreciate that English is less than 50% of our traffic. Thanks, GerardM

On 9 January 2017 at 21:45, Pine W wiki.pine@gmail.com wrote:

Hi Aisha,

I suggest that you contact Jaime Anstee and/or Katy Love (cc'd here) about this subject, because they are WMF staff who do a lot of work with grantmaking and performance evaluation for chapters. They might know of some analyses that could help you.

Discussions about what kinds of resources, and what quantities of resources, to allocate to the chapters vs. smaller affiliates, other kinds of grants, and WMF-run work that focuses on content and community development, have been happening for years, and are likely to continue for the foreseeable future.

Different chapters function differently, partly because of varied cultural and legal contexts, so there is not a monolithic model of how a chapter should run. The definition of "successful" varies from affiliate to affiliate.

There has been a discussion for years about how to define and quantify affiliate "impact"; my personal preference is to abolish are use of that word. (:

Pine

On Sun, Jan 8, 2017 at 5:41 PM, Aisha Brady aishabrady@gmail.com wrote:

...

Hi!

Could anyone point me towards any papers relevant to Wikimedia chapters (how they function, the work they do, whether they have been successful or otherwise)?

Thank you! :)

Aisha

---

Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

---

Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l