Hi Laura, I very much like the topic and question of your research. Bravo. The result that pops out at me I is 23.5% of gendered-content by super-users is Woman-related. The sample size of 0.4% of editors is small, too small to draw conclusions about the super users themselves, but the sample of articles, I think starts to approach being representative. Well at least it is in line with what I found, which was that looking at Biography articles only, 18% of English Wikipedia is about Women [1]. One way to scale up this kind of sentiment analysis would be Mechanical Turk[2] , to "automate" categorizing pages into gender. That would take some money, but maybe there's a research grant for it. With a statiscally significant sample, it seems like a result that would be popular in the media. One could also compare the the likelihood to write gendered articles of the super-users versus others. The conclusion is quite punchy - finding people to write about women is not as important as convincing people who currently write, to write about women. I would be very intrigued to collaborate and help on this research. Do you have ideas about what you want "future directions" to look like? [1] http://hangingtogether.org/?p=2877 [2] https://www.mturk.com/mturk/welcome Maximilian Klein Wikipedian in Residence, OCLC +17074787023 ________________________________ From: wiki-research-l-bounces(a)lists.wikimedia.org <wiki-research-l-bounces(a)lists.wikimedia.org> on behalf of Kristin Dagmar Eckert <keckert(a)umd.edu> Sent: Sunday, February 16, 2014 7:24 PM To: Research into Wikimedia content and communities Subject: Re: [Wiki-research-l] The role of English Wikipedia's top content creators in perpetuating gender bias Thank you very much again for this insightful article on Wikipedia, gender and content creation!! We are very happy we could post it on our blog and are looking forward to responses. Sincerely, Stine Eckert Stine Eckert Ph.D. Candidate Philip Merrill College of Journalism University of Maryland 2100N Knight Hall www.stineeckert.com @stineeckert ________________________________ From: wiki-research-l-bounces(a)lists.wikimedia.org [wiki-research-l-bounces(a)lists.wikimedia.org] on behalf of Laura Hale [laura(a)fanhistory.com] Sent: Sunday, February 16, 2014 7:59 AM To: Research into Wikimedia content and communities Subject: [Wiki-research-l] The role of English Wikipedia's top content creators in perpetuating gender bias http://wikinewsreporter.wordpress.com/2014/02/10/the-role-of-english-wikipe… is some research I did recently about the role of English Wikipedia's top content creators in terms of perpetuating gender bias. Any feedback appreciated. Sincerely, Laura Hale -- twitter: purplepopple

On Mon, Feb 17, 2014 at 2:28 PM, Stuart A. Yeates <syeates(a)gmail.com> wrote: Why do you want categories rather than structured data about gender, religion, and race? We should be moving towards replacing our entire category system with a better implementation of structured data. Categories, while broadly useful, are an incomplete, arbitrary, and arbitrarily heirarchical subset of structured data. Almost none of the arguments against 'categorization' apply to gathering this data, structuring it, and making it easily searchable and filterable. Sam.

Why do you want categories in the first place? Why not extract whatever semantic meaning you need (e.g., about genderbread) by parsing the sentences in each article? On Mon, Feb 17, 2014 at 3:08 PM, Stuart A. Yeates <syeates(a)gmail.com> wrote: Self-definition is fine, and compatible with what I think of as structured data: large numbers of high-granularity data points associated with each [article]. Category data is a narrow subset of structured data that happens to support a tree structure, in the MediaWiki implementation. ... so many assumptions you just made there :-) If the concept of fa'afafine exists in our knowledge-set (and it does: anything that passes some low bar of verifiability can be included in what our projects consider knowledge), then it can be noted as a data point applying to some other topic [article]. There is little that is Western or English about our verifiability standards; though if you are talking about the English-language Wikipedia, having an English-language source increases the verifiability of a data-claim. We can create a category for every data-attribute -- in this case, [[category:fa'afafine]] (which does not yet exist) or [[category:kathoey]] (which does). If we didn't have wikidata, that would be the clear solution. But now it is enough to capture the attribute of "self-identifies as fa'afafina", whether or not there is an associated category. In particular, arguments about "how many category-intersections of the fa'afafine gender and other traits deserve their own category" are red herrings. All that matters is identifying these (self-defined) attributes in a way that is easy to process in bulk. Why is this a problem? The attribute "gender according to DNB" is a) useful historical data, b) verifiable, and c) easy to add to wikidata. I believe you can have "DNB-gender" as one of the variations on the global "gender" attribute. Most articles (unless they are talking about the DNB specifically) would likely refer to the global attribute. But this way you can have both datasets globally accessible. Then after the import is done, people can write bulk data-cleaning scripts to help humans review those articles where the two differ. And in cases where there is a years-long edit war about what the global attribute should be, you can keep track of what the input source-data is from various sources. Sam.

A "normal user" might not notice that Wikipedia is "too male" or "too female". If that is so and Wikipedia has bias, then doesn't it follow that the "normal user" will uncritically absorb that bias? If the normal user is reading about tennis (say) and reads many long article about male tennis players and a few short articles about female tennis players, isn't the normal user forming the impression that men are the important players of tennis? Isn't this why we worry about bias in the first place because we risk perpetuating it? I agree it is difficult in Wikipedia to: * measure bias in number, size, quality of articles * correlate bias in articles to skewed demographics of editors contributing to those articles (probably adjusted by frequency, size, or nature of edits) * determine if editors creating observable bias in the articles are doing so deliberately or unconsciously * postulate ways to address the bias But since we are here to discuss research, then let's discuss what would be a set of experiments that would help to answer these questions? Having said all of that, I would be very interested to have a "bias dashboard" set up for certain topics that shows simple (preferably visually) stats like number, length, quality of "gendered" articles (the first of my bullet points above). Often just making bias visible creates change. Kerry -----Original Message----- From: wiki-research-l-bounces(a)lists.wikimedia.org [mailto:wiki-research-l-bounces@lists.wikimedia.org] On Behalf Of Federico Leva (Nemo) Sent: Tuesday, 18 February 2014 6:44 AM To: Research into Wikimedia content and communities Subject: Re: [Wiki-research-l] The role of English Wikipedia's top content creators in perpetuating gender bias Given that the discussion is all happening here anyway, I'll copy the comment I left on the blog. :) ---- Nemo February 16, 2014 at 2:43 pm Your comment is awaiting moderation. Interesting trivia to munch, but little nutritional value IMHO. Everyone in every field always complains that there are too many articles about pokémon and French municipalities, too few about X very important topic/aspect. All such considerations are worthless because they don’t consider the actual relative *impact* of those articles, e.g. how many other people edited or discussed them after creation and how many page views they had. One could even argue that the “perception problems related to Wikipedia being male and cliquey” is made worse by posts like this, ;) but that would be trolling. I don’t see how the mere number of articles on one topic or another is going to make Wikipedia feel “too male” or “anti-female” to a normal user who certainly doesn’t notice such things. If you have to dig for it, it doesn’t contribute to public perception; at most it can be a possible symptom of something else that may be contributing to public perception. Even disregarding the impact, to assess the bias of the contributors themselves a more precise research, comparative in nature, would be needed. For instance, if one writes articles on parliament members in country X, and 70 % of articles are about males, that’s only biased if the actual percentage of male MP is less than 70 %. The same should be done with all the sources for each topic. And it’s nothing compared to the systemic bias towards the western and anglo-saxon point of view which writing in English and using (mostly online) English sources encourages, let alone languages less global in nature. _______________________________________________ Wiki-research-l mailing list Wiki-research-l(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Stuart A. Yeates, 18/02/2014 01:48: As a fallback to "serious" research, a handful questions may be added to the next editors' survey whenever one happens. I recommend to copy this suggestion to a suitable Meta-Wiki page (e.g. <https://meta.wikimedia.org/wiki/Talk:General_User_Survey>) so that it stays available for the years to come. Nemo

As one of the English Wikipedia's top content creators [1], let me weigh in here with a shameless plug that I would like to present my work at Wikimania this year and am hoping for sponsorship from the WMF in order to do that.[2] I agree heartily with Laura that <quote>One of the things that really needs to be done is to develop a list of inherently notable matching pairs and then see what the patterns are, because general lists of notable women are unlikely to answer the question of how real the systematic bias is. Further, gendered lists where the percentages of men and women present should also be attained and the red links percentages for each gender should also be looked at.</quote> My corner of the Wikiverse is art, and I have been busily matching up various lists of artists and have become well aware of a huge difference between "documented artists" and "art market artists". I find lots more women in the second group than the first group, and there has been a lot of previous work done on the why of this gender bias. This was in fact one of the main reasons for people to join the recent Art&Feminism edit-a-thon. The gender bias in the world of art museums is much easier to document and track, because it is so overwhelmingly obvious. Pick any top art museum in the world and do a "weenie count" [3]. The percentage of works by women actually hanging in public display galleries is way less than 10%. The main reason is of course that people want the biggest bang for their buck in any situation, and art museum curators are no exception. They would rather buy one Anthony van Dijk than 50 works by his lesser known male or female contemporaries. If you prioritize any "would like to buy list" that any art museum curator is holding at any given auction, the percentage of female names on those lists will probably be zero. That said, I have discovered that most art museums do have quality works by women in their collections, but these are not on show. I have been resurrecting these works wherever I find them virtually on Wikipedia as a way to illustrate biographies on these artists. [1] https://en.wikipedia.org/wiki/Wikipedia:Wikipedia_Signpost/2014-01-22/Speci… [2] https://wikimania2014.wikimedia.org/wiki/Submissions/Women_on_Wikipedia [3] https://en.wikipedia.org/wiki/Guerrilla_Girls 2014-02-20 13:16 GMT+01:00, Laura Hale &lt;laura(a)fanhistory.com&gt;om>:

Your example of sports is very interesting. I agree that above a certain level, any team etc. is inherently notable. There is still, however, the question of available sources, I'm not a sports connoisseur, but if I go to a typical library, there is a range of books on popular *male* sports. I can find biographies of famous players, histories of major clubs, etc. A quick Amazon search for instance shows me that they have a book on sale about the successes of AS Saint-Etienne in the French football (that's soccer for Americans) championship in 1976! I do not find such books on female sports. In fact, if I look for a book on the French women's soccer team on Amazon, I find something... extracted from Wikipedia! (Recall that football is the most popular sport in France...) In short, for certain topics (e.g. male sports), there is a gazillion books, biographies, and other source material readily available, while for others (e.g. female sports) such sources are more difficult to find. What I would like to understand is how much the bias is caused by such imbalances in sources. A possible evaluation method would be to consider female and male personalities (e.g. writers) equal in notoriety (e.g. according to scholars from that field), and to compare the length and quality of the biographies. What do you think?

On Thu, Feb 20, 2014 at 7:55 PM, David Monniaux &lt;David.Monniaux(a)free.fr&gt;wrote;wrote: On the issue of notability and availability of men's versus women's top level teams, there are a number of issues at play. There are regional notability issues that make sourcing even more difficult, and then biases by a dominant editing pool that become enshrined. (Look at the men's football league notability guidelines.) There might be a lot of sources about Norwegian women fútbol players in Norwegian newspapers. These sources may be online, and easily accessible to anyone looking at Norwegian language newspapers. But there is an inherent bias against automatic notability for leagues and players from non-English speaking countries, where English speaking players are not playing, and where English speaking fans are not following the league. But most sports people I know wouldn't make their first stop at a bookstore unless they were looking for historical information. Most of the sport related activity happens around current players. Hence, newspaper sources tend to be relied on heavily. In the case of sports and for the most highly visible (re: national teams), this is not an issue of readily available but the relative volume of one compared to another. http://www.google.com/trends/explore?hl=en-US&q=/m/01l3vx,+/m/06drlz&am… the Google trends for the French men's national team versus the women's national team. I went to Google news and searched for french men national soccer team and had 57,000 results. I searched for french women national soccer team and had 43,200 results. There is not an availability of sources issue. In fact, there is likely sooooo much information that you actually need to filter. The basics for national teams for many sports can also generally be filled out by the sport's governing bodies. (FIFA is really, really excellent in this regards. FIBA is also great.) I personally think that this is often an excuse because for certain topics, the sources are there but people are not utilizing them to create articles about women while they are out there creating them about men. Or when they aren't as accessible for both men and women, people are willing to go the extra length to try to create articles about the men's national team. A quick search on Google news for Gabon men's national basketball team pulls up zero results. Another search for Gabon women's national basketball team pulls up zero results. Guess which team has an article on English Wikipedia? And how big of a role does that play anyway in article creation? A while back, I looked at the Australian women's national soccer team players versus the men's national team players. There were an average of 6.7 sources on women's articles and 19.6 for men. Women's articles are more poorly sourced when looked at head to head, and I can guarantee you there is waaaaaay more available sources to take all articles about both sets all to FA if people were motivated, but no one appears to be motivated enough to do so even with available sources. I would love to see research on source availability as a factor leading to the lack of creation of articles about women in head to head situations with their direct male counterparts. Sincerely, Laura Hale -- twitter: purplepopple

Thanks Kerry! I have been working on lists of painters per museum collection in order to show how few women artists are represented in major collections. With all of the work we do for GLAMS, it is interesting to note that they themselves are highly successful at perpetuating systemic bias. Magnus is able to collect data on all the museums on the BBC's "Your Paintings" website, and with his data I just created a list of painters of the National Gallery, London. I was surprised to see that there is not even one female artist from Britain represented (though the British men are also underepresented, with only 18 out of 750 names). Lists like these can help generate demographic data for all sorts of diversity issues, as the amount of art in the museum is overwhelmingly Italian, Dutch/Netherlandish, and French, without any art at all from areas outside Europe. The list is here: https://en.wikipedia.org/wiki/Catalog_of_paintings_in_the_National_Gallery,… 2014-02-21 8:32 GMT+01:00, David Monniaux &lt;David.Monniaux(a)free.fr&gt;fr>:

Funny you should interpret me that way. I think my point is twofold; namely Wikipedia is only a reflection of its sources, and that it's sources show an odd set of demographics no matter how you slice and dice the data. Sent from my iPad On Feb 23, 2014, at 2:59 AM, "Federico Leva (Nemo)" &lt;nemowiki(a)gmail.com&gt; wrote:

Laura, I am of course only a sample of one, but in my experience based on the issues discussed at length in workshops and on the gendergap mailing list, often what is perceived to be anti-female behavior is enough to drive women away. Men, when perceiving anti-male behavior tend to do the opposite, namely they become aggressive and stand their ground. The key way to entice more women to contribute is to give them the tips and tricks they need to 1) feel their contribution is appreciated 2) see their contributions are reused in the normal ways of "wiki magic" The main problem with keeping women on board is because men are more tech-savvy in general and can get both points much easier than women, who see both points much more easily on fb, pinterest, and other social media sites. Jane Sent from my iPad On Feb 23, 2014, at 10:13 AM, Laura Hale &lt;laura(a)fanhistory.com&gt; wrote:

Nemo, that is so cool you said that, because it proves as I have long suspected that I have developed some sort of blind eye to such things. I recognized your snippy response as only being a knee-jerk reaction to a perceived anti-Italian remark, not to an anti-male remark! Anyway as I said in my first mail in this thread I am also reacting to a self-interpreted attack for being perceived for the shameful underrepresentation on women's topics (I think way less than 10% of what I have contributed onwiki is female-oriented). For the record, I am a huge Glam fan and i think Caravaggio rocks, so don't shoot me for these observations, which are I think fascinating. I will continue to compile these lists on museum collections, because exploring the data of the institutions Wikipedia bases its own content on is important to our understanding of the perceived biases. I am somewhat confused about your comment on warfare and trenches. I simply advocate one-on-one coaching of women in the tips and tricks that male contributors discover easily on their own. Like you I am also interested in learning more about such social dynamics and feel a yearly editor survey would be a good place to start. Jane Sent from my iPad On Feb 23, 2014, at 12:44 PM, "Federico Leva (Nemo)" &lt;nemowiki(a)gmail.com&gt; wrote:

David, I think we are both on the same page, but I am a bit farther in my thinking about HOW you can illustrate the biases. The mobile team coined the phrase 'ghost edits' to mean the edits we don't make while on mobile devices (I am on my iPad right now and experienced a ghost edit which will definitely not get done now because I don't care about it enough). We need another term for the edits that are never considered, because we don't have the people to even make those 'ghost' considerations. By creating lists of topics and working with Wikidata, we can eventually make statements possible such as in the 20th century, Harvard only appointed x women to full professorship, an x percentage of the faculty, which compares favorably or not to the Sorbonne, etc. Jane Sent from my iPad On Feb 23, 2014, at 11:38 AM, David Monniaux &lt;David.Monniaux(a)free.fr&gt; wrote:

Stuart, Yes there are lots of institutions we could work with, given the proper funding and volunteers to monitor efforts. However, we can also use the GLAM contacts we already have. I believe the proper channel to propose something like this is the Wikiproject Women artists. We need to be careful though about partner selection, because we don't want to get involved in political discussions with activist groups. The Guerrilla girls don't seem to be an institution, but more a group of performance artists. When you look at GLAMs that we have worked with in the past, generally the work relationships are through image donations and/or data donations. I advocate using the lists we already have access to as a basis for making statements about "black holes" in our data. Making a statement about how many women vs. men there are in any given list of names (french engravers, dutch architects, and so forth) is useless, because those lists are not based on any finite list from an institution. When you publish a list of "french engravers in the collection of the British library" then you can make valid statements about that specific set of data. I am a bit unclear about what you mean about a non-straight-white-male-wheel, because the list I published is from the PCF (via Magnus) and there is nothing about it that re-invents a wheel (though we should probably be doing that anyway as well). Jane On Feb 23, 2014, at 8:06 PM, Stuart A. Yeates wrote: