Hi folks, Sorry to have been absent from this discussion thus far: I didn't realize Sam was going to post the targets when he did, and so I am playing a little catch-up here. Below are some questions and answers re the targets that might be helpful for the discussion. (Erik wrote most of this, and I've just now added a few bits.) If you read this and there are still issues that you want addressed, please just say so :-) Thanks, Sue http://wikimediafoundation.org/wiki/Resolution:Five-year_targets What’s the purpose of these five-year targets? The July 2015 targets approved by the Board represent [[big hairy audacious goal]]s for Wikimedia. They are intended to reflect Wikimedia’s core mission -- “to empower and engage people around the world to collect and develop free educational content and to disseminate it effectively and globally” -- and to capture it in terms of concrete, ambitious outcomes that are very hard --but ideally not impossible-- to achieve. The development of very ambitious five-year targets is intended to help inspire and focus energy across the Wikimedia movement, in connection with the priorities identified through Wikimedia’s strategic planning process. They’re also intended to help persuade people who aren’t yet active members of the Wikimedia movement (for example, readers who don’t yet edit or donate, grant-making institutions, “GLAM” organizations) to get involved and support us. How do you expect these targets to be used? These targets will be part of the printed “summary” version of the strategy plan that we will distribute internally and to external interested parties -- e.g., Wikipedia Academy attendees, grant-making institutions, etc. We will give out the document, including the targets, so that people understand where the Wikimedia movement is focusing its energy, and how they can help. The targets will also help us, we hope, to focus our own work, and measure whether we’re being successful. In the context of an overall Wikimedia dashboard, we can begin to highlight if key performance indicators are significantly deviating from our expectations -- e.g. if overall article growth flattens, or the number of editors declines. Are these the only targets used by the Wikimedia Foundation? No. These five targets are called out as very ambitious long-term outcomes which, if we achieve them together, will indicate that Wikimedia has made great strides in serving its mission. They are intentionally high level, focused on information and the people who develop and receive it, as opposed to operations. There are other key performance indicators which we must examine on an ongoing basis, including but not limited to: * engagement and retention of the editor community * site uptime and load times in different geographies * financial health of the Wikimedia movement * availability of secure off-site copies of all data * number and quality of multimedia files in all our projects * number and quality of information in Wikimedia’s other projects * demographic composition of the editor community * our collective ability to develop and operationalize innovative technology The Wikimedia report-card, at http://stats.wikimedia.org/reportcard/ , is our primary instrument for tracking key performance indicators; its formats and the indicators which are included are still evolving. The Wikimedia report-card is much more detailed than the five-year targets: it tracks more information at a more granular level, more frequently. How were these targets developed? The development of five-year targets was the last major piece of work that needed to be concluded before the strategy project wrapped up. To that end, the targets were developed primarily by the staff of the Wikimedia Foundation (mainly Sue Gardner, Erik Moeller, and Barry Newstead), for Board approval. The first source for material was the discussions held on the strategy wiki, focused on performance indicators and goals (e.g., <http://strategy.wikimedia.org/wiki/Strategic_Plan/Movement_Priorities>). Another source was the set of internal metrics that the Wikimedia community has used for a long time, including “number of articles” and “active editors”, as well as first baseline estimates for gender participation and country-of-origin breakdown that have been established more recently through surveys, log analysis, and other methods. People’s general views about target-setting, as well as their assessment of the value of different possible measures, were surfaced through surveys of foundation-l and internal-l readers, Advisory Board members, Board members and staff members [1]. That helped shape both our general approach to target-setting, and the actual measures and numbers we wanted to use. Once a draft set of targets was created, Sue gave it to the Board, where it was discussed at some length and then approved. [1] See Sue’s blog post at <http://suegardner.org/2010/08/16/how-wikimedia-will-measure-success-over-the-next-five-years/> for some background. Can these targets be modified? While other key performance indicators (see above) are intended to be flexibly adjusted as we better understand which indicators are useful and which ones aren’t, the five-year targets voted on by the Board are intended to be stable. That’s why they are so high-level. The underlying measurements and methodologies for each target will continue to be refined over time -- more so in categories where we lack clear and consistent baselines, such as article quality. Why was “number of Wikipedia articles” used as a target for “amount of information we offer”? We were looking for a single, clearly understandable indicator that we weren’t just reaching more people, growing our community and increasing its diversity, and improving perceived quality of content, but that we were also successfully expanding the breadth and depth of information available to readers. We fully understand that the number of Wikipedia articles is an imperfect measure -- it’s especially imperfect if taken by itself. On the other hand, it’s fairly well understood what a Wikipedia article is, what it isn’t, and what the potential challenges with counting Wikipedia articles are (such as mass-creation of articles based on some data source): we’ve used article counts in Wikipedia since the project started, and have developed a collective expertise how to manage and how to interpret them. Taken by itself, the number of Wikipedia articles answers one question, and one question only: How likely is it that Wikimedia’s flagship project, Wikipedia, is going to have any information on a given term? It doesn’t answer whether the information is useful, of high quality, or even whether it should be in Wikipedia at all. But the other targets help to answer that question: We are measuring whether we are increasing the reach of our projects, the perceived quality of the information provided, and the number and diversity of contributors. If we succeed along all those dimensions, the number of Wikipedia articles is a useful indicator of our overall breadth and depth. This is not to diminish the work done in other categories of content (such as media files), or the work done in Wikipedia’s sister projects. The “number of Wikipedia articles” is expected to be useful as an _indicator_ of the overall amount of information we offer: that is its primary usefulness. What does a “25% increase in quality” actually mean? We’ve not yet established which assessment systems will be the most useful, scalable, and effective, and until we have done so, this measure should be seen principally as a placeholder for a large, ambitious, relative increase in perceived quality of a large number of assessed articles over the five-year time period (most simply, in the context of a scoring system like <http://www.mediawiki.org/wiki/Article_feedback/Public_Policy_Pilot>, a 25% increase of average scores for assessed articles over the five-year time period). Wikimedia projects to-date have primarily used internal measures of quality (“good articles”, “featured articles”, patrolled edits, WikiProject classification, etc.). These measures are of critical importance to the organization of Wikimedia’s editorial efforts. At the same time, they only measure what we, ourselves, think about the quality of the product that we’ve built. It’s equally important that we obtain measures of what others think about the quality provided -- this includes our audience, but it also includes individuals with verifiable expertise regarding the subject matter domain(s) an article relates to. -- Sue Gardner Executive Director Wikimedia Foundation 415 839 6885 office 415 816 9967 cell Imagine a world in which every single human being can freely share in the sum of all knowledge.  Help us make it a reality! http://wikimediafoundation.org/wiki/Donate

Thank you for your replies. Sue has kindly answered some questions on IRC, as well: http://meta.wikimedia.org/wiki/IRC_office_hours/Office_hours_2010-10-14 Sue Gardner, 13/10/2010 03:58: Is Thomas' interpretation correct? Thomas Dalton, 12/10/2010 13:37: The FAQ Sue posted here doesn't answer at all to my question. «The “number of Wikipedia articles” is expected to be useful as an _indicator_ of the overall amount of information we offer»: it just can't. But this could change. For instance, I think that people spend much more time reading/studying dictionaries and other books than encyclopedias (in whatever form). You can't say what it's possible. As a dictionaries addict, I used to believe (in 2005-2006), for many reasons, that a wiki dictionary wasn't going to be very useful (and I was a Wiktionary admin, I cared about it!); now en.wikt is IMHO better than many traditional English dictionaries, and it could attract most pageviews of its competitors if only people knew it. And I don't see why en, fr and some other Wiktionaries should continue to be the only successful ones. But this is only an example. A very small improvement to Wikipedia may cost much, much more than a huge improvement to projects where we've never invested almost anything. And I could give you a list of examples (from the past) and proposals (for the future), but we would be OT. John Vandenberg, 11/10/2010 05:20: I want to stress this, again. Samuel J Klein, 12/10/2010 20:52: This? http://strategy.wikimedia.org/wiki/Strategic_Plan/Role_of_the_WMF (Or a later version of it.) I don't think it's enough. Yes, we often overlook the other users of MediaWiki, as well (at least, I do); and actually, it's quite surprising that the long-term proposal of setting up a "MediaWiki Foundation", which has been discussed several times (if I remember correctly), didn't come up on strategywiki. Perhaps it's something to think about for the next five-years plan. :-p That's exactly what I want to underline. Wikipedia isn't enough. And if we focus our efforts only on Wikipedia, we could waste our energies.[1] Therefore, I think that our main measure for the second goal should include all WMF projects. For instance, the number of entries on all projects. (This doesn't include MediaWiki, though.) This is a good example. And there are many. Michael Peel, 10/10/2010 19:54: Yes, but I'm also interested in WMF's priorities. :-) And you forgot chapters. ;-) Nemo [1] We could reverse Thomas's argument and be even more harsh than he is: Wikipedia is our most developed project (and our main source of income); it can't grow much. Let's use the income from our core product to conquer other (big) markets.

Hello, 2010/10/12 Thomas Dalton &lt;thomas.dalton(a)gmail.com&gt;om>: That's an unproven assumption. It might even be the opposite, i.e. reinforcing Wikipedia might only increase the gap between the projects. That's a bad decision. The WMF should try to balance the projects, when the community has not done it alone. Regards, Yann