On 7/28/12 5:58 AM, Tilman Bayer wrote:
Hi all,
the Wikimedia Foundation's 2012-13 Annual Plan has just been published at
https://wikimediafoundation.org/wiki/File:2012-13_Wikimedia_Foundation_Plan_...
accompanied by a Q&A:
https://wikimediafoundation.org/wiki/2012-2013_Annual_Plan_Questions_and_Ans...
The plan was approved by the Board of Trustees at its meeting in Washington, DC, at Wikimania, and previously outlined to the Foundation staff and interested community members at the monthly staff meeting on July 5, 2012. We were planning to publish the video recording of that meeting at this point, but encountered technical difficulties; the video will hopefully become available soon.
Slide 8 : "How are we doing against the 2012 targets"
I was stopped by
"The Global Education Program is now the largest-ever systematic effort of the Wikimedia mouvement to boost high quality content creation, with a projected addition of 19 million characters to Wikipedia through student assignements 2011-2012"
OF COURSE, we all know that WMF needs to glorify what it is actually initiating/in charge of. And that's fair enough.
But seriously... I would feel fine with us trying to claim that the GEP is the largest system effort to INCREASE the number of articles. It is probably true.
But we all know that the result is... so and so. Possibly good content, but also lot's of crap being reverted and deleted afterwards. Claiming it is the largest effort to boost high quality content is not only disingenous... but I actually find it counter productive and a tiny bit offensive toward the actual community.
High quality content simply does NOT come from newbie students.
Florence
Florence Devouard, 29/07/2012 13:37:
But seriously... I would feel fine with us trying to claim that the GEP is the largest system effort to INCREASE the number of articles. It is probably true.
But we all know that the result is... so and so. Possibly good content, but also lot's of crap being reverted and deleted afterwards. Claiming it is the largest effort to boost high quality content is not only disingenous... but I actually find it counter productive and a tiny bit offensive toward the actual community.
Another passage is interesting too: «We've successfully brought in thousands of students [...] they typically make a small number of edits [...] and then leave. [...] initiatives like "Wiki Loves Monuments" have also achieved very high multimedia participation in spike months (22K uploaders in September 2011, when WLM took place, returning to normal levels immediately afterwards).»
Where are the statistics validating this interpretation? I didn't find anyone able to answer me how many of the thousands participants to WLM contributed to Commons or another project afterwards. Their motivation is very different from the students'.
Nemo
On Sun, Jul 29, 2012 at 6:58 AM, Federico Leva (Nemo) nemowiki@gmail.com wrote:
Florence Devouard, 29/07/2012 13:37:
But seriously... I would feel fine with us trying to claim that the GEP is the largest system effort to INCREASE the number of articles. It is probably true.
But we all know that the result is... so and so. Possibly good content, but also lot's of crap being reverted and deleted afterwards. Claiming it is the largest effort to boost high quality content is not only disingenous... but I actually find it counter productive and a tiny bit offensive toward the actual community.
Another passage is interesting too: «We've successfully brought in thousands of students [...] they typically make a small number of edits [...] and then leave. [...] initiatives like "Wiki Loves Monuments" have also achieved very high multimedia participation in spike months (22K uploaders in September 2011, when WLM took place, returning to normal levels immediately afterwards).»
Where are the statistics validating this interpretation? I didn't find anyone able to answer me how many of the thousands participants to WLM contributed to Commons or another project afterwards. Their motivation is very different from the students'.
Regarding the "normal levels", I suppose you haven't yet had a chance to look at http://reportcard.wmflabs.org/graphs/active_editors ?
Also, recently Lodewijk, with the help of WMF data analyst Erik Zachte, posted this interesting analysis: http://www.wikilovesmonuments.org/new-editors-thanks-to-wiki-loves-monuments...
If I read it correctly, from the newbies among the WLM participants, 61 were still active in May 2012. This compares to altogether 7053 active editors on Commons during that month (the latter number is from http://stats.wikimedia.org/wikispecial/EN/TablesWikipediaCOMMONS.htm ; note that a user who makes just one edit or one upload during a month falls below the threshold for the currently used active editors metric). But as the blog post notes, there are efforts underway to improve retention of new contributors in this year's WLM.
Nemo
Wikimedia-l mailing list Wikimedia-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l
Tilman Bayer, 29/07/2012 18:28:
Regarding the "normal levels", I suppose you haven't yet had a chance to look at http://reportcard.wmflabs.org/graphs/active_editors ?
Yes and it shows that there's still an increase over the pre-WLM situation. Actually I was reading http://stats.wikimedia.org/wikispecial/EN/TablesWikipediaCOMMONS.htm which shows the numbers better but still doesn't have the total number of uploaders/ussers with at least one edit in a given month.
Also, recently Lodewijk, with the help of WMF data analyst Erik Zachte, posted this interesting analysis: http://www.wikilovesmonuments.org/new-editors-thanks-to-wiki-loves-monuments...
If I read it correctly, from the newbies among the WLM participants, 61 were still active in May 2012. This compares to altogether 7053 active editors on Commons during that month (the latter number is from http://stats.wikimedia.org/wikispecial/EN/TablesWikipediaCOMMONS.htm ; note that a user who makes just one edit or one upload during a month falls below the threshold for the currently used active editors metric). But as the blog post notes, there are efforts underway to improve retention of new contributors in this year's WLM.
Thanks, I had indeed missed this post for some reason. 231 or 6,6 % with some activity after the end and 61 very active editors seems to be better than what the university students do? This is also acknowledged later on, at p. 25: «[...] multimedia is where early usability efforts (UploadWizard), especially alongside programs like Wiki Loves Monuments, have paid off. (Commons is one of the few areas where active editors are growing -- 25% year over year, with a spike to 9.37K from 6.97K in September 2011 due to the WLM competition.)».
Tilman Bayer, 29/07/2012 23:17:
Someone from the Education Program will be able to give a more thorough overview of the efforts to evaluate its results, but for example I'm aware of
https://blog.wikimedia.org/2012/04/19/wikipedia-education-program-stats-fall...
. The quantitative method used there has its limitations, but similar methods are employed in independent (i.e non-WMF) research about Wikipedia in the academic literature.
Still, even if we consider only quantity, 19 millions characters is not that much, and with some guesstimate I'm not sure it's more than what some WikiProjects or edit drives have done in the past, e.g. the addition of all Italian municipalities on it.wiki back in 2005 or so. That passage would have been clearer by excluding all "normal" volunteer (individual or organized) activity from the comparison, otherwise it's easy to mix things up.
Nemo
On Sun, Jul 29, 2012 at 11:18 PM, Federico Leva (Nemo) nemowiki@gmail.com wrote:
Tilman Bayer, 29/07/2012 18:28:
Regarding the "normal levels", I suppose you haven't yet had a chance to look at http://reportcard.wmflabs.org/graphs/active_editors ?
Yes and it shows that there's still an increase over the pre-WLM situation.
Given the size of the normal monthly fluctuations (e.g. July-August 2011: +0.3K, August-October 2011: +0.2K), and the overall upwards trend during 2011-12, I find it hard to understand the objections to the interpretation "returning to normal levels".
Actually I was reading http://stats.wikimedia.org/wikispecial/EN/TablesWikipediaCOMMONS.htm which shows the numbers better but still doesn't have the total number of uploaders/ussers with at least one edit in a given month.
It does show the number of users with at least one upload, and those with at least one mainspace edit (look further down). As an aside, it also contains numbers for uploads made using UploadWizard, strongly supporting the statement that much of the 2011-12 growth was due to this usability improvement, cf. the statement on slide 25 you already cited below.
Also, recently Lodewijk, with the help of WMF data analyst Erik Zachte, posted this interesting analysis:
http://www.wikilovesmonuments.org/new-editors-thanks-to-wiki-loves-monuments...
If I read it correctly, from the newbies among the WLM participants, 61 were still active in May 2012. This compares to altogether 7053 active editors on Commons during that month (the latter number is from http://stats.wikimedia.org/wikispecial/EN/TablesWikipediaCOMMONS.htm ; note that a user who makes just one edit or one upload during a month falls below the threshold for the currently used active editors metric). But as the blog post notes, there are efforts underway to improve retention of new contributors in this year's WLM.
Thanks, I had indeed missed this post for some reason. 231 or 6,6 % with some activity after the end and 61 very active editors
That's not quite what the blog post said. 61 was the number of all *active* editors left during the latest month examined (May), and it doesn't say how the average number of edits is distributed among these. That being said, it's of course absolutely great that WLM appears to have brought in at least some very active contributors, among them one who has already done 20,000 edits so far.
seems to be better than what the university students do?
If "what the university students do" refers to the Education Program, note that boosting the number of active editors by those students isn't its primary goal, and neither has WLM been focused on that metric.
This is also acknowledged later on, at p. 25: «[...] multimedia is where early usability efforts (UploadWizard), especially alongside programs like Wiki Loves Monuments, have paid off. (Commons is one of the few areas where active editors are growing -- 25% year over year, with a spike to 9.37K from 6.97K in September 2011 due to the WLM competition.)».
Again, I'm not quite sure what "This" in "This is also acknowledged later on" refers to. See e.g. https://en.wiktionary.org/wiki/spike for the meaning of "spike".
Tilman Bayer, 08/04/2012 12:02 AM:
On Sun, Jul 29, 2012 at 11:18 PM, Federico Leva (Nemo) wrote:
Tilman Bayer, 29/07/2012 18:28:
Regarding the "normal levels", I suppose you haven't yet had a chance to look at http://reportcard.wmflabs.org/graphs/active_editors ?
Yes and it shows that there's still an increase over the pre-WLM situation.
Given the size of the normal monthly fluctuations (e.g. July-August 2011: +0.3K, August-October 2011: +0.2K), and the overall upwards trend during 2011-12, I find it hard to understand the objections to the interpretation "returning to normal levels".
I wasn't objecting to it, I only asked where to find the data. :-) To me, it seems to have stabilized on a slightly higher level.
Actually I was reading http://stats.wikimedia.org/wikispecial/EN/TablesWikipediaCOMMONS.htm which shows the numbers better but still doesn't have the total number of uploaders/ussers with at least one edit in a given month.
It does show the number of users with at least one upload, and those with at least one mainspace edit (look further down). As an aside, it also contains numbers for uploads made using UploadWizard, strongly supporting the statement that much of the 2011-12 growth was due to this usability improvement, cf. the statement on slide 25 you already cited below.
Ah, thank you, I forgot the Commons-specific part of the page: that's what I was looking for, with the 22642 uploaders number. :-) http://stats.wikimedia.org/wikispecial/EN/TablesWikipediaCOMMONS.htm#uploader_activity_levels To me, this table shows that uploaders were stabilized (for a few months) at about: * 15k total / 0k with UW "before the UploadWizard"; * 16k / 9k after it; * 18k / 13k after WLM. Which would show a progressive switch to UW (also by very active uploaders), boosted by WLM, and a stable increase of uploaders after UW+WLM. This poor man reading of the tables doesn't provide reliable indications, of course; I do have a slightly different opinion than the one the annual plan seems to be implying (speaking of WLM "only" as one of the short-term activity boosters), but the conclusion is the same: WLM and UW had a wonderful synergy and are the way to go (with the mobile app this year).
Nemo
On Sun, Jul 29, 2012 at 4:37 AM, Florence Devouard anthere9@yahoo.com wrote:
On 7/28/12 5:58 AM, Tilman Bayer wrote:
Hi all,
the Wikimedia Foundation's 2012-13 Annual Plan has just been published at
https://wikimediafoundation.org/wiki/File:2012-13_Wikimedia_Foundation_Plan_...
accompanied by a Q&A:
https://wikimediafoundation.org/wiki/2012-2013_Annual_Plan_Questions_and_Ans...
The plan was approved by the Board of Trustees at its meeting in Washington, DC, at Wikimania, and previously outlined to the Foundation staff and interested community members at the monthly staff meeting on July 5, 2012. We were planning to publish the video recording of that meeting at this point, but encountered technical difficulties; the video will hopefully become available soon.
Slide 8 : "How are we doing against the 2012 targets"
I was stopped by
"The Global Education Program is now the largest-ever systematic effort of the Wikimedia mouvement to boost high quality content creation, with a projected addition of 19 million characters to Wikipedia through student assignements 2011-2012"
OF COURSE, we all know that WMF needs to glorify what it is actually initiating/in charge of. And that's fair enough.
But seriously... I would feel fine with us trying to claim that the GEP is the largest system effort to INCREASE the number of articles. It is probably true.
But we all know that the result is... so and so. Possibly good content, but also lot's of crap being reverted and deleted afterwards. Claiming it is the largest effort to boost high quality content is not only disingenous... but I actually find it counter productive and a tiny bit offensive toward the actual community.
High quality content simply does NOT come from newbie students.
Over the last years, the Foundation has been trying to base decisions and evaluations more often on objective data and research rather than on personal opinions and impressions.
Of course, here the term "high quality" does not necessarily mean, say, featured content (e.g. on the English Wikipedia, featured articles currently make up less than 0.1% of the total articles), but instead refers to comparisons with average contributions.
Someone from the Education Program will be able to give a more thorough overview of the efforts to evaluate its results, but for example I'm aware of https://blog.wikimedia.org/2012/04/19/wikipedia-education-program-stats-fall... . The quantitative method used there has its limitations, but similar methods are employed in independent (i.e non-WMF) research about Wikipedia in the academic literature.
Which research methodology did you use to arrive at your conclusions above?
Florence
Wikimedia-l mailing list Wikimedia-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l
On Jul 30, 2012 7:18 AM, "Tilman Bayer" tbayer@wikimedia.org wrote:
On Sun, Jul 29, 2012 at 4:37 AM, Florence Devouard anthere9@yahoo.com
wrote:
On 7/28/12 5:58 AM, Tilman Bayer wrote:
Hi all,
the Wikimedia Foundation's 2012-13 Annual Plan has just been published
at
https://wikimediafoundation.org/wiki/File:2012-13_Wikimedia_Foundation_Plan_...
accompanied by a Q&A:
https://wikimediafoundation.org/wiki/2012-2013_Annual_Plan_Questions_and_Ans...
The plan was approved by the Board of Trustees at its meeting in Washington, DC, at Wikimania, and previously outlined to the Foundation staff and interested community members at the monthly staff meeting on July 5, 2012. We were planning to publish the video recording of that meeting at this point, but encountered technical difficulties; the video will hopefully become available soon.
Slide 8 : "How are we doing against the 2012 targets"
I was stopped by
"The Global Education Program is now the largest-ever systematic effort
of
the Wikimedia mouvement to boost high quality content creation, with a projected addition of 19 million characters to Wikipedia through student assignements 2011-2012"
OF COURSE, we all know that WMF needs to glorify what it is actually initiating/in charge of. And that's fair enough.
But seriously... I would feel fine with us trying to claim that the GEP
is
the largest system effort to INCREASE the number of articles. It is
probably
true.
But we all know that the result is... so and so. Possibly good content,
but
also lot's of crap being reverted and deleted afterwards. Claiming it
is the
largest effort to boost high quality content is not only disingenous...
but
I actually find it counter productive and a tiny bit offensive toward
the
actual community.
High quality content simply does NOT come from newbie students.
Over the last years, the Foundation has been trying to base decisions and evaluations more often on objective data and research rather than on personal opinions and impressions.
Of course, here the term "high quality" does not necessarily mean, say, featured content (e.g. on the English Wikipedia, featured articles currently make up less than 0.1% of the total articles), but instead refers to comparisons with average contributions.
Someone from the Education Program will be able to give a more thorough overview of the efforts to evaluate its results, but for example I'm aware of
https://blog.wikimedia.org/2012/04/19/wikipedia-education-program-stats-fall...
Ive asked for more info at
http://meta.wikimedia.org/wiki/Research_talk:Wikipedia_Education_Program_eva...
. The quantitative method used there has its limitations, but similar methods are employed in independent (i.e non-WMF) research about Wikipedia in the academic literature.
Do you have links to any relevant studies of the GEP?
-- John Vandenberg
Hi John,
On Sun, Jul 29, 2012 at 2:39 PM, John Vandenberg jayvdb@gmail.com wrote:
Ive asked for more info at
http://meta.wikimedia.org/wiki/Research_talk:Wikipedia_Education_Program_eva...
I did my best to answer your question there.
. The quantitative method used there has its limitations, but similar methods are employed in independent (i.e non-WMF) research about Wikipedia in the academic literature.
Do you have links to any relevant studies of the GEP?
For the English Wikipedia, you might be interested in the Article Quality Improvement section of the Public Policy Initiative Learning Points document[1] (that project was the pilot of the U.S. Education Program). Last term, because of some of the limitations Tilman referenced above, I worked with English Wikipedia editors Mike Christie ([[User:Mike Christie]]) and Doc James ([[User:Jmh649]]) to run a modified version of the Public Policy research, where a corps of volunteer Wikipedians edited a random sample of student work from last term. We hired an outside researcher to cross-tabulate the good classes (as determined by the quality improvement shown) with a series of factors present in the various classes (so, for example, how many Ambassadors did the class have? Were they undergraduate or graduate classes? Did the professor edit Wikipedia? etc.). We're expecting the results of that research in the next two weeks. You can see more about it here: [2].
For the Brazil and Egypt pilots, the number of students is so low that it's easy to see the improvements by hand. For example, students in the Cairo Pilot wrote the article on Laura Restrepo by hand [3] and translated the article on Civil disobedience from the French Wikipedia [4]. All the articles students work on are listed on their course pages; links to the various language Wikipedia course pages are at http://education.wikimedia.org
I'm happy to answer other questions about the Education Program either here or on wiki.
LiAnna
[1] http://outreach.wikimedia.org/wiki/Public_Policy_Initiative_Learning_Points#...
[2] http://en.wikipedia.org/wiki/Wikipedia:Ambassadors/Research
[3] http://ar.wikipedia.org/wiki/%D9%84%D9%88%D8%B1%D8%A7_%D8%B1%D9%8A%D8%B3%D8%...
[4] http://ar.wikipedia.org/wiki/%D8%B9%D8%B5%D9%8A%D8%A7%D9%86_%D9%85%D8%AF%D9%...
On Jul 31, 2012 1:43 AM, "LiAnna Davis" ldavis@wikimedia.org wrote:
Hi John,
On Sun, Jul 29, 2012 at 2:39 PM, John Vandenberg jayvdb@gmail.com wrote:
Ive asked for more info at
http://meta.wikimedia.org/wiki/Research_talk:Wikipedia_Education_Program_eva...
I did my best to answer your question there.
Ive replied with more specific questions.
This research was mentioned because of bold statements in the annual plan, and Tilman Bayer mentioned this blog post:
https://blog.wikimedia.org/2012/04/19/wikipedia-education-program-stats-fall...
Which says U.S. Education Program users are three times better than other users.
-- JV
On Mon, Jul 30, 2012 at 3:11 PM, John Vandenberg jayvdb@gmail.com wrote:
Ive replied with more specific questions.
This research was mentioned because of bold statements in the annual plan, and Tilman Bayer mentioned this blog post:
https://blog.wikimedia.org/2012/04/19/wikipedia-education-program-stats-fall...
Which says U.S. Education Program users are three times better than other users.
Just to clarify, it says three times better than other *new* editors, not established editors! New is a very key word there.
I'm afraid I don't know the answers to the questions you posted as I was not involved in any of the methodology of this particular research, but I will see if I can find answers for you.
LiAnna
On Sun, Jul 29, 2012 at 10:17 PM, Tilman Bayer tbayer@wikimedia.org wrote:
Of course, here the term "high quality" does not necessarily mean, say, featured content (e.g. on the English Wikipedia, featured articles currently make up less than 0.1% of the total articles), but instead refers to comparisons with average contributions.
Someone from the Education Program will be able to give a more thorough overview of the efforts to evaluate its results, but for example I'm aware of
https://blog.wikimedia.org/2012/04/19/wikipedia-education-program-stats-fall... . The quantitative method used there has its limitations, but similar methods are employed in independent (i.e non-WMF) research about Wikipedia in the academic literature.
It certainly does have limitations. Let's look at what it says:
---o0o---
In the Wikipedia Education Program, professors assign their students to edit Wikipedia articles as a grade for class, assisted by volunteer Wikipedia Ambassadors. In fall 2011, 55 courses participated in the program in the United States, with students editing articles on the English Wikipedia. On average, these students added 1855 bytes of content that stayed on Wikipedia, compared to only 491 for a randomly chosen sample of new users who joined English Wikipedia in September 2011. These numbers establish that students who participate in the Wikipedia Education Program contribute significantly more quality content that stays on Wikipedia than other new users.
---o0o---
Apart from John's very salient question about how the random sample of editors was selected, another very obvious issue is the traffic the edited pages attract. A random sample of users might include contributors to very popular and heavily edited pages, while students' edits are more likely to be to specialised pages on scholarly niche topics that get very few views, and attract few edits.
Content on little watched pages always stays longer than content on highly watched pages with a high edit turnover. This is quite irrespective of edit quality. Just look at some Wikipedia pages on Indian villages ... their content is crap, with outstanding long-term stability. :)
So until the analysis also factors in page viewing statistics and average edits per month on each page, the variables are hopelessly confounded, and the conclusions are nothing but wishful thinking (not to say lying with statistics).
In other words, it's impossible to conclude that content staying on Wikipedia is a reflection of edit quality, rather than a reflection of said content being on a very obscure page that no one reads or edits.
If the Foundation has an interest in producing meaningful statistical analyses, I would suggest actually employing a statistician who can give such posts a look-over and point out the obvious fallacies.
http://meta.wikimedia.org/wiki/Research_talk:Wikipedia_Education_Program_eva...
wikimedia-l@lists.wikimedia.org