It sounds interesting, Laura. There are a couple of parameters that may
create issues, specifically the gender and geolocation of editors of the
articles (most editors do not gender-identify or specify their geographic
location), but it's a pretty thorough review of the interrelationships
between the various WMF projects and with external media.
Risker/Anne
On 17 June 2012 21:58, Laura Hale <laura(a)fanhistory.com> wrote:
I think I brought this up in another post about
developing a methodology
to determine this based on a number of offline conversations I was having.
As this might be of interest to people doing research on women's
involvement for Wikipedia, I'm crossposting this here from my blog. The
entry can be found at
http://ozziesport.com/2012/06/measuring-the-influence-of-wikipedia-on-publi….
It isn't necessarily the one I will ultimately use when I set out to do
this, but working to set up some of the tools to allow measuring for this
in the near future. If anyone has any thoughts on how to better measure
the influence of Wikipedia for public thought formation, especially as it
pertains to women's issues, that would be much appreciated.
*
*
*Measuring the influence of Wikipedia on public thinking (in Australian
women’s sport): A proposed methodology*
This entry was posted by Laura on Monday, 18 June, 2012
I spend a lot of time thinking about Wikipedia and talking to people in
Australia’s sport sector about Wikipedia, Wikinews and Commons. Some of
those I have talked to tend to agree that Wikipedia has value, potentially
more value than a news story from a traditional outlet where the story
quickly disappear. There is still a question of: “Why does Wikipedia
matter?” The most obvious answer is “Page views demonstrate meaning.
People are going to Wikipedia for information about sport, Australian
sport, and women’s sport. The page views clearly demonstrate that
Wikipedia matters and is worth contributing to as an organisation.” This
argument doesn’t always work and I’ve been challenged to demonstrate
Wikipedia’s influence on the topic of Australian women’s sport.
This is in some ways a frustrating endeavor. How do you measure
Wikipedia’s influence beyond page views? As a person who loves research
design, I have a number of ideas but how to implement and analyze
information is still something I am struggling with because I keep coming
back to the need to possibly use a qualitative approach reliant on survey
research… and that always makes me nervous. Still, with this in mind, the
following methodology is one I have been leaning towards trying to write up
more formally.
*
Benchmark English Wikipedia, English Wikinews and Commons coverage of
Australian women’s sport:*
Develop a list of all existing articles pertaining to Australian
women’s sport on English Wikipedia. The list would include biographies,
sport teams, leagues, organisations and people connected to administering
women’s sport, competitions, articles about women’s sport in the country,
articles about sport in the country, general sport articles that broadly
intersect with Australian women’s sport such as women’s basketball and the
Olympic games. Once the list is created, benchmark the following for each
article:
Determine the existing size of the article.
Identify projects the article is part of.
Identify current article assessment.
Date article was created and who created it.
Determine the number of contributors to the article.
Determine the gender ratio for article contributors.
Determine the geographic location of article contributors.
Determine the historic page views for the article.
Determine if the article has appeared at Did you Know, and average
traffic before and after it appeared.
Determine if there is a spoken word version of the article.
Count the number of pictures on each article.
Develop a list of all existing news stories about Australian women’s
sport on English Wikinews.
Determine the historical traffic to these articles.
Identify the categories these articles are included in.
Identify if the article is synthesis or original research.
Identify all Wikipedia articles the story is linked on.
Develop a list of all photographs pertaining to Australian women’s
sport on Commons.
Develop a list what articles these images are used on across
Wikimedia Foundation projects.
Develop a list of contributors for images in this space.
This data will provide a framework for understanding the story of
Australian women’s sport coverage on Wikipedia, and will enable case
studies to be developed around any potential editing efforts affiliated
with a study. This data is largely background, which can help to
contextualise data around Wikipedia as it pertains to influence thought
formation.
*Benchmark interest in English Wikipedia, English Wikinews and Commons
coverage of Australian women’s sport:*
The next step is possibly the more difficult one: How do we contextualise
Wikipedia articles to understand where they sit as resources people turn to
for information? Below are quantitative, web based measures to try to
determine this:
The News
Find the number of articles in the media that link to/mention an
article on Wikipedia about a topic.
Check articles about a topic to find examples of media plagiarism
of Wikipedia articles.
Academia
Identify the number of times an academic text uses Wikipedia as a
source.
Identify which academics reference Wikipedia.
Sport Institutions
Count the number of links to Wikimedia content on their website.
Count the number of links to Wikimedia content on their social
media related portals.
The Community
Facebook
Measure the number of likes for a Wikipedia article on
Facebook.
Twitter
Measure the number of links to an article.
Identify who is tweeting about the article, get their
individual Twitter metric data.
Get the metric data for all the followers of a person tweeting
about a topic.
Google+
Measure the number of links to an article.
Identify who is tweeting about the article, get their
individual Google+ information.
LiveJournal and clones
Measure the number of mentions for an article on LiveJournal
and its clones.
Get the community or user metrics for including the links.
Google search
Determine the ranking of the Wikipedia and Wikinews articles
on Google.
Using Google Trends, determine the relative number of searches
for these topics and where they are origination from, both with and without
Wikipedia in the search phrase.
Yahoo!Answers
Measure the number of questions about the topic. (Closed, and
open.)
Measure the number of questions on Wikipedia about articles.
Measure the number of answers that reference the article.
While these can help measure a certain level of influence, they do not
necessarily explain why people are citing or how this forms thinking.
Results would just suggest current levels of awareness but not necessarily
influence. The two are really separate points.
*Survey attitudes towards English Wikipedia, English Wikinews and Commons
coverage of Australian women’s sport:*
A third methodology is needed to complement the previous two. As much as
it pains me, a qualitative methodology needs to be used: Survey work needs
to be completed. The following groups need to be surveyed with questions
seeking specific information in certain areas. The following are some
broad themes for this group.
The media
As a journalist, what is their opinion of Wikipedia in this area?
Have they used Wikipedia, Wikinews or other projects as a resource
when working on a story?
Academics and students
As an academic or student, what is their opinion of Wikipedia in
this area?
Have they used Wikipedia, Wikinews or other projects as a resource
when working on papers?
Sport industry participants including athletes
As members of the sport industry, what is their opinion of
Wikipedia in this area?
Have they actively sought to improve content related to themselves
or their sport organisation on WMF projects?
Sport fans and participants
Do they use Wikipedia for information and how reliable do they
consider it to be?
Do they contribute to Wikipedia?
This information would then need to be wrapped around existing research
that discusses the influence of the media, in this case in an Australian
and sport context, academia and industry influence on forming public
perception. An approach also needs to be developed which can include a
before and after treatment for a content improvement drive in this area.
In doing the first part, it should hopefully become apparent where there is
a need for articles to be improved or created to develop a class of
articles for improvement.
Sincerely,
Laura Hale
--
twitter: purplepopple
blog:
ozziesport.com
_______________________________________________
Gendergap mailing list
Gendergap(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/gendergap