Wiki-research-l

wiki-research-l@lists.wikimedia.org

8 participants
2988 discussions

Call for contributors, A survey of wiki sites
by Houcemeddine A. Turki 29 Aug '19

29 Aug '19

Dear all, I thank you for your efforts. I am managing to begin writing a survey about wiki sites and publish it in an appropriate journal... This survey will involve the software used (Mediawiki or other), the size, reference support, topic... I am looking for contributors to this project. Anyone who has published two papers about wikis in a research journal is invited to join the initiative. Yours Sincerely, Houcemeddine Turki (he/him) Medical Student, Faculty of Medicine of Sfax, University of Sfax, Tunisia Undergraduate Researcher, UR12SP36 GLAM and Education Coordinator, Wikimedia TN User Group Member, WikiResearch Tunisia Member, Wiki Project Med Member, WikiIndaba Steering Committee Member, Wikimedia and Library User Group Steering Committee Co-Founder, WikiLingua Maghreb Founder, TunSci _____________________ +21629499418

1 0

Re: [Wiki-research-l] gender balance of Wikipedia citations
by Greg 28 Aug '19

28 Aug '19

Hi Jane, Thanks for the link. It's clear that there is a lot of work being done, and even more left to do. I've been thinking about what you said about second authors and was wondering if instead of fixing it (or in addition to fixing it), it would make sense to put some sort of tag on the page itself (like the ones I see questioning notability or requests for additional citations). Something along the lines of authors missing from a particular citation and how to fix that, or no work by women cited in this article (if this is the case). It strikes me that by fixing it yourself, you are doing great work, but that maybe it also makes sense to spread awareness about these issues to the broader editing community so more people are thinking about it/doing it. At any rate, I thought I'd float the idea. Such a tag/the response (if any), could also be interesting to study, though perhaps something like this already exists and I'm just not aware of it, or perhaps there is good reason not to do it. All best, Greg On Tue, Aug 27, 2019 at 5:00 AM <wiki-research-l-request(a)lists.wikimedia.org> wrote: > Send Wiki-research-l mailing list submissions to > wiki-research-l(a)lists.wikimedia.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://lists.wikimedia.org/mailman/listinfo/wiki-research-l > or, via email, send a message with subject or body 'help' to > wiki-research-l-request(a)lists.wikimedia.org > > You can reach the person managing the list at > wiki-research-l-owner(a)lists.wikimedia.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Wiki-research-l digest..." > > > Today's Topics: > > 1. Re: gender balance of wikipedia citations (Greg) > 2. Re: gender balance of Wikipedia citations (Jane Darnell) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Mon, 26 Aug 2019 18:56:12 -0700 > From: Greg <thenatureprogram(a)gmail.com> > To: Isaac Johnson <isaac(a)wikimedia.org> > Cc: Research into Wikimedia content and communities > <wiki-research-l(a)lists.wikimedia.org> > Subject: Re: [Wiki-research-l] gender balance of wikipedia citations > Message-ID: > < > CAOO9DNv92bVR2COT2XmpHDU5kJOvD0yD3bahG+6Fkuma+HYDEg(a)mail.gmail.com> > Content-Type: text/plain; charset="UTF-8" > > Thanks, Isaac and Federico. These notes and links are very helpful--and > will require some time to process. As for how many years I have to work on > this, I'm retired! In truth, I keep hoping that someone on this list will > express interest in working on these matters. The questions are all very > interesting and quite relevant. The idea of studying removed citations is > both complex and compelling. > > Greg > > On Mon, Aug 26, 2019 at 6:49 AM Isaac Johnson <isaac(a)wikimedia.org> wrote: > > > Regarding data, I have not been a part of these projects but I think that > > I can help a bit with working links: > > * The (I believe) original dataset can also be found here: > > > https://analytics.wikimedia.org/datasets/archive/public-datasets/all/mwrefs/ > > * A newer version of this dataset was produced that also included > > information about whether the source was openly available and its topic: > > ** Meta page: > > > https://meta.wikimedia.org/wiki/Research:Towards_Modeling_Citation_Quality > > ** Figshare: > > > https://figshare.com/articles/Accessibility_and_topics_of_citations_with_id… > > > > On Mon, Aug 26, 2019 at 3:53 AM Federico Leva (Nemo) <nemowiki(a)gmail.com > > > > wrote: > > > >> Greg, 22/08/19 06:19: > >> > I do not know the current status of wikicite or if/when this > >> > could be used for this inquiry--either to examine all, or a sensible > >> subset > >> > of the citations. > >> > >> If I see correctly, you still did not receive an answer on the data > >> available. > >> > >> It's true that the Figshare item for > >> < > >> > https://meta.wikimedia.org/wiki/Research:Scholarly_article_citations_in_Wik… > > > >> > >> was deleted (I've asked about it on the talk page), but it's trivial to > >> run https://pypi.org/project/mwcites/ and extract the data yourself, at > >> least for citations which use an identifier. > >> > >> Some example datasets produced this way: > >> https://zenodo.org/record/15871 > >> https://zenodo.org/record/55004 > >> https://zenodo.org/record/54799 > >> > >> Once you extract the list of works, the fun begins. You'll need to > >> intersect with other data sources (Wikidata, ORCID, other?) and account > >> for a number of factors until you manage to find a subset of the data > >> which has a sufficiently high signal:noise ratio. For instance you might > >> need to filter or normalise by > >> * year of publication (some year recent enough to have good data but old > >> enough to allow the work to be cited elsewhere, be archived after > >> embargos); > >> * country or institution (some probably have better ORCID coverage); > >> * field/discipline and language; > >> * open access status (per Unpaywall); > >> * number of expected pageviews and clicks (for instance using > >> <https://wikitech.wikimedia.org/wiki/Analytics/AQS/Pageviews> and > >> < > https://meta.wikimedia.org/wiki/Research:Wikipedia_clickstream#Releases>; > >> > >> a link from 10k articles on asteroids or proteins is not the same as > >> being the lone link from a popular article which is not the same as a > >> link buried among a thousand others on a big article); > >> * time or duration of the addition (with one of the various diff > >> extraction libraries, content persistence data or possibly historical > >> eventstream if such a thing is available). > >> > >> To avoid having to invent everything yourself, maybe you can reuse the > >> method of some similar study, for instance the one on the open access > >> citation advantage or one of the many which studied the gender imbalance > >> of citations and peer review in journals. > >> > >> However, it's very possible that the noise is just too much for a > >> general computational method. You might consider a more manual approach > >> on a sample of relevant events, for instance the *removal* of citations, > >> which is in my opinion more significant than the addition.* You might > >> extract all the diffs which removed a citation from an article in the > >> last N years (probably they'll be in the order of 10^5 rather than > >> 10^6), remove some massive events or outliers, sample 500-1000 of them > >> randomly and verify the required data manually. > >> > >> As usual it will be impossible to have an objective assessment of > >> whether that citation was really (in)appropriate in that context > >> according to the (English or whatever) Wikipedia guidelines. To test > >> that too, you should replicate one of the various studies of the gender > >> imbalance of peer review, perhaps one of those which tried to assess the > >> impact of a double blind peer review system on the gender imbalance. > >> However, because the sources are already published, you'd need to > >> provide the agendered information yourself and make sure the > >> participants perform their assessment in some controlled environment > >> where they don't have access to any gendered information (i.e. where you > >> cut them off the internet). > >> > >> How many years do you have to work on this project? :-) > >> > >> Federico > >> > >> (*) I might add a citation just because it's the first result a popular > >> search engine gives me, after glancing at the abstract and maybe the > >> journal home page; but if I remove an existing citation, hopefully I've > >> at least assessed its content and made a judgement about it, apart from > >> cases of mass removals for specific problems with certain articles or > >> publication venues. > >> > >> _______________________________________________ > >> Wiki-research-l mailing list > >> Wiki-research-l(a)lists.wikimedia.org > >> https://lists.wikimedia.org/mailman/listinfo/wiki-research-l > >> > > > > > > -- > > Isaac Johnson -- Research Scientist -- Wikimedia Foundation > > > > > ------------------------------ > > Message: 2 > Date: Tue, 27 Aug 2019 08:00:45 +0200 > From: Jane Darnell <jane023(a)gmail.com> > To: Research into Wikimedia content and communities > <wiki-research-l(a)lists.wikimedia.org> > Subject: Re: [Wiki-research-l] gender balance of Wikipedia citations > Message-ID: > <CAFVcA-HqVicR0k65J4iox0PD= > oc3HBPMZLfXVO5zqkFD+EnSxQ(a)mail.gmail.com> > Content-Type: text/plain; charset="UTF-8" > > Greg, > Yes that's what I meant. On Wikipedia you get what you measure, so many > Wikipedians are page-creators and page-hit junkies because we can measure > that. The trick to motivating editors is giving them other measurements for > progress. Here is the link to the Women writers Wikiproject and as you > scroll down you can see what is measured. > https://en.wikipedia.org/wiki/Wikipedia:WikiProject_Women_writers > Jane > > On Tue, Aug 27, 2019 at 3:39 AM Greg <thenatureprogram(a)gmail.com> wrote: > > > Thanks for sharing your experience and thoughts, Jane. I did not know > this > > was happening--I'm hardly an expert, so that's not surprising, and yet > it's > > still very troubling to hear. I'm not sure what you mean by setting up a > > Wikiproject. Do you mean of ways for how to study this gap--i.e., the > ideas > > that have been floated in this thread to this point? Or are you thinking > of > > something else? > > > > Greg > > > > On Mon, Aug 26, 2019 at 5:00 AM < > > wiki-research-l-request(a)lists.wikimedia.org> > > wrote: > > > > > Send Wiki-research-l mailing list submissions to > > > wiki-research-l(a)lists.wikimedia.org > > > > > > To subscribe or unsubscribe via the World Wide Web, visit > > > https://lists.wikimedia.org/mailman/listinfo/wiki-research-l > > > or, via email, send a message with subject or body 'help' to > > > wiki-research-l-request(a)lists.wikimedia.org > > > > > > You can reach the person managing the list at > > > wiki-research-l-owner(a)lists.wikimedia.org > > > > > > When replying, please edit your Subject line so it is more specific > > > than "Re: Contents of Wiki-research-l digest..." > > > > > > > > > Today's Topics: > > > > > > 1. Re: gender balance of Wikipedia citations (WereSpielChequers) > > > 2. Re: gender balance of Wikipedia citations (Greg) > > > 3. Re: sockpuppets and how to find them sooner (Federico Leva > (Nemo)) > > > 4. Re: gender balance of Wikipedia citations (Jane Darnell) > > > 5. Re: gender balance of wikipedia citations (Federico Leva (Nemo)) > > > > > > > > > ---------------------------------------------------------------------- > > > > > > Message: 1 > > > Date: Sun, 25 Aug 2019 14:28:25 +0100 > > > From: WereSpielChequers <werespielchequers(a)gmail.com> > > > To: Research into Wikimedia content and communities > > > <wiki-research-l(a)lists.wikimedia.org> > > > Subject: Re: [Wiki-research-l] gender balance of Wikipedia citations > > > Message-ID: > > > <CAAanWP3qJnMpLB4tr9Eqt4EJLg2kCihkb50UY-d8= > > > ShNONhSAA(a)mail.gmail.com> > > > Content-Type: text/plain; charset="UTF-8" > > > > > > Hi Greg, > > > > > > One of the major step changes in the early growth of the English > > Wikipedia > > > was when a bot called RamBot created stub articles on US places. I > think > > > they were cited to the census. Others have created articles on rivers > in > > > countries and various other topics by similar programmatic means. > > Nowadays > > > such article creation is unlikely to get consensus on the English > > > Wikipedia, but there are some languages which are very open to such > > > creations and have them by the million. > > > > > > I'm not sure if the fastest updating of existing articles is automated > or > > > just semiautomated. But looking at the bot requests page, it certainly > > > looks like some people are running such maintenance bots "updating GDP > by > > > country" is a current bot request. > > > https://en.wikipedia.org/wiki/Wikipedia:Bot_requests. > > > > > > I'm not sure how "the ease of a source for purposes of converting into > a > > > table and generating a separate article for each row" relates to > gender. > > > But i suspect "number of times cited in wikipedia" deserves less kudos > > than > > > "number of times cited in academia". > > > > > > WSC > > > > > > On Sun, 25 Aug 2019 at 05:22, Greg <thenatureprogram(a)gmail.com> wrote: > > > > > > > Thanks again, Kerry. I am hoping that someone with access to more > > > resources > > > > (knowledge, support, etc) than I have will look into this. > > > > > > > > A few more thoughts/questions: > > > > > > > > 1. The link to the citation dataset from the Medium article ("What > are > > > the > > > > ten most cited sources on Wikipedia? Let’s ask the data.") is broken. > > > > 2. As far as I can tell, every named author in the top ten most cited > > > > sources on Wikipedia is male. One piece is by a working group > > > > 3. This line from the Medium piece struck me: "Many of these > > publications > > > > have been cited by Wikipedians across large series of articles using > > > > powerful bots and automated tools." > > > > > > > > Are citations being added by bots? I'm not sure that I understand > that > > > line > > > > correctly. > > > > > > > > Greg > > > > > > > > > > > > > > > > > > > > > > > > > > > > > ------------------------------ > > > > > > Message: 2 > > > Date: Sun, 25 Aug 2019 21:16:25 -0700 > > > From: Greg <thenatureprogram(a)gmail.com> > > > To: wiki-research-l(a)lists.wikimedia.org > > > Subject: Re: [Wiki-research-l] gender balance of Wikipedia citations > > > Message-ID: > > > <CAOO9DNvGyfvJkzyRq60cSQi-T80mAkUa= > > > vCPkzFbEysfGQqnVg(a)mail.gmail.com> > > > Content-Type: text/plain; charset="UTF-8" > > > > > > Thanks, WSC. All very interesting. > > > > > > I've been thinking about Wiklpedia citations less in terms of kudos and > > > more in terms of a feedback loop. The cited sources get a significant > > > amount of attention (1 click per 200 pageviews is the number I saw > > > recently). When I imagine total Wikipedia traffic, that's huge. How > many > > > students are finding sources this way? How many academics? And how many > > of > > > these citations are finding their way back into academic publications > via > > > this mechanism? > > > > > > Assuming this is happening to some degree, the gender imbalance of the > > > citations is also reflected. If the Wikipedia imbalance is the same as > > the > > > one in academia, that's one thing; if it is better on Wikipedia than it > > is > > > in academia, that's reason to celebrate; if the balance is worse, > that's > > > concerning. In fact, if the gender imbalance conforms to my fears > instead > > > of my hopes, and is magnified by the massive website traffic, I imagine > > it > > > could even explain the growth in the citation disparity researchers > note > > in > > > their study of political science texts. (I link to that study in a > > previous > > > post; it was mentioned in the Washington Post recently) > > > > > > There is a very real possibility that Wikipedia is making the citation > > > gender gap worse. I think we need to understand what is happening and > > take > > > immediate action if the news is not good. > > > > > > Greg > > > > > > > > > > > > > > > > > > > > > > > > ------------------------------ > > > > > > Message: 3 > > > Date: Mon, 26 Aug 2019 10:59:07 +0300 > > > From: "Federico Leva (Nemo)" <nemowiki(a)gmail.com> > > > To: Research into Wikimedia content and communities > > > <wiki-research-l(a)lists.wikimedia.org>, Aaron Halfaker > > > <ahalfaker(a)wikimedia.org>, Kerry Raymond < > > kerry.raymond(a)gmail.com> > > > Subject: Re: [Wiki-research-l] sockpuppets and how to find them sooner > > > Message-ID: <cf2734ff-d2cf-3108-691f-8ecf46125ed7(a)gmail.com> > > > Content-Type: text/plain; charset=utf-8; format=flowed > > > > > > Please everyone avoid using jargon specific to the English Wikipedia on > > > this cross-language and cross-wiki mailing list. > > > > > > Aaron Halfaker, 23/08/19 17:36: > > > > I think embeddings[1] would be a nice way to create a signature. > > > > > > There is some discussion of acceptable user fingerprinting (presumably > > > to be available to CheckUsers only), other than the usual over-reliance > > > on IP addresses, in particular at > > > < > > > > > > https://meta.wikimedia.org/wiki/Talk:IP_Editing:_Privacy_Enhancement_and_Ab… > > > >. > > > > > > Federico > > > > > > > > > > > > ------------------------------ > > > > > > Message: 4 > > > Date: Mon, 26 Aug 2019 10:17:46 +0200 > > > From: Jane Darnell <jane023(a)gmail.com> > > > To: Research into Wikimedia content and communities > > > <wiki-research-l(a)lists.wikimedia.org> > > > Subject: Re: [Wiki-research-l] gender balance of Wikipedia citations > > > Message-ID: > > > <CAFVcA-G87k26nBMr=-e-+C8o6eG0KQvVihH= > > > f4M40faVNbKkqw(a)mail.gmail.com> > > > Content-Type: text/plain; charset="UTF-8" > > > > > > Greg, > > > Thanks for worrying. This is a known problem and yes, Wikipedia > > contributes > > > to the Gendergap in citations and no, it's not an easy fix, since it is > > the > > > fault of systemic bias in academia. So fewer women are head author on > > > scientific publications, and it is generally only the head author that > > gets > > > cited on Wikipedia. This is not just a problem with written works in > the > > > field of politics. I spend most of my time working on paintings and > > their > > > documented catalogs, so generally I only notice and fix this problem in > > art > > > catalogs. Women rarely appear as lead author mentioned. I will always > add > > > them in to descriptions when I add items for their works on Wikidata, > > but I > > > can not always find them! Sometimes I can't even create items for them > > > because all I have is a name and a work and nothing else available > online > > > anywhere. You see this most often with women who spent entire careers > > > working at a single institution and the institution doesn't bother to > > > promote their work or even list them in exhibition catalogs. With luck > > > there might be a local obituary, but not always. If you have > suggestions > > > how to set up a Wikiproject to tackle this it would be a good idea. In > my > > > onwiki experience the Women-in-Red community can be very positive in > > their > > > response to gendergap-related issues for women writers. > > > Jane > > > > > > On Mon, Aug 26, 2019 at 6:17 AM Greg <thenatureprogram(a)gmail.com> > wrote: > > > > > > > Thanks, WSC. All very interesting. > > > > > > > > I've been thinking about Wiklpedia citations less in terms of kudos > and > > > > more in terms of a feedback loop. The cited sources get a significant > > > > amount of attention (1 click per 200 pageviews is the number I saw > > > > recently). When I imagine total Wikipedia traffic, that's huge. How > > many > > > > students are finding sources this way? How many academics? And how > many > > > of > > > > these citations are finding their way back into academic publications > > via > > > > this mechanism? > > > > > > > > Assuming this is happening to some degree, the gender imbalance of > the > > > > citations is also reflected. If the Wikipedia imbalance is the same > as > > > the > > > > one in academia, that's one thing; if it is better on Wikipedia than > it > > > is > > > > in academia, that's reason to celebrate; if the balance is worse, > > that's > > > > concerning. In fact, if the gender imbalance conforms to my fears > > instead > > > > of my hopes, and is magnified by the massive website traffic, I > imagine > > > it > > > > could even explain the growth in the citation disparity researchers > > note > > > in > > > > their study of political science texts. (I link to that study in a > > > previous > > > > post; it was mentioned in the Washington Post recently) > > > > > > > > There is a very real possibility that Wikipedia is making the > citation > > > > gender gap worse. I think we need to understand what is happening and > > > take > > > > immediate action if the news is not good. > > > > > > > > Greg > > > > > > > > > > > > > > > > > > > > > > > _______________________________________________ > > > > Wiki-research-l mailing list > > > > Wiki-research-l(a)lists.wikimedia.org > > > > https://lists.wikimedia.org/mailman/listinfo/wiki-research-l > > > > > > > > > > > > > ------------------------------ > > > > > > Message: 5 > > > Date: Mon, 26 Aug 2019 11:45:09 +0300 > > > From: "Federico Leva (Nemo)" <nemowiki(a)gmail.com> > > > To: Research into Wikimedia content and communities > > > <wiki-research-l(a)lists.wikimedia.org>, Greg > > > <thenatureprogram(a)gmail.com> > > > Subject: Re: [Wiki-research-l] gender balance of wikipedia citations > > > Message-ID: <835202af-4653-641e-782e-c619458bdd7f(a)gmail.com> > > > Content-Type: text/plain; charset=utf-8; format=flowed > > > > > > Greg, 22/08/19 06:19: > > > > I do not know the current status of wikicite or if/when this > > > > could be used for this inquiry--either to examine all, or a sensible > > > subset > > > > of the citations. > > > > > > If I see correctly, you still did not receive an answer on the data > > > available. > > > > > > It's true that the Figshare item for > > > < > > > > > > https://meta.wikimedia.org/wiki/Research:Scholarly_article_citations_in_Wik… > > > > > > > > > was deleted (I've asked about it on the talk page), but it's trivial to > > > run https://pypi.org/project/mwcites/ and extract the data yourself, > at > > > least for citations which use an identifier. > > > > > > Some example datasets produced this way: > > > https://zenodo.org/record/15871 > > > https://zenodo.org/record/55004 > > > https://zenodo.org/record/54799 > > > > > > Once you extract the list of works, the fun begins. You'll need to > > > intersect with other data sources (Wikidata, ORCID, other?) and account > > > for a number of factors until you manage to find a subset of the data > > > which has a sufficiently high signal:noise ratio. For instance you > might > > > need to filter or normalise by > > > * year of publication (some year recent enough to have good data but > old > > > enough to allow the work to be cited elsewhere, be archived after > > > embargos); > > > * country or institution (some probably have better ORCID coverage); > > > * field/discipline and language; > > > * open access status (per Unpaywall); > > > * number of expected pageviews and clicks (for instance using > > > <https://wikitech.wikimedia.org/wiki/Analytics/AQS/Pageviews> and > > > < > https://meta.wikimedia.org/wiki/Research:Wikipedia_clickstream#Releases > > >; > > > > > > a link from 10k articles on asteroids or proteins is not the same as > > > being the lone link from a popular article which is not the same as a > > > link buried among a thousand others on a big article); > > > * time or duration of the addition (with one of the various diff > > > extraction libraries, content persistence data or possibly historical > > > eventstream if such a thing is available). > > > > > > To avoid having to invent everything yourself, maybe you can reuse the > > > method of some similar study, for instance the one on the open access > > > citation advantage or one of the many which studied the gender > imbalance > > > of citations and peer review in journals. > > > > > > However, it's very possible that the noise is just too much for a > > > general computational method. You might consider a more manual approach > > > on a sample of relevant events, for instance the *removal* of > citations, > > > which is in my opinion more significant than the addition.* You might > > > extract all the diffs which removed a citation from an article in the > > > last N years (probably they'll be in the order of 10^5 rather than > > > 10^6), remove some massive events or outliers, sample 500-1000 of them > > > randomly and verify the required data manually. > > > > > > As usual it will be impossible to have an objective assessment of > > > whether that citation was really (in)appropriate in that context > > > according to the (English or whatever) Wikipedia guidelines. To test > > > that too, you should replicate one of the various studies of the gender > > > imbalance of peer review, perhaps one of those which tried to assess > the > > > impact of a double blind peer review system on the gender imbalance. > > > However, because the sources are already published, you'd need to > > > provide the agendered information yourself and make sure the > > > participants perform their assessment in some controlled environment > > > where they don't have access to any gendered information (i.e. where > you > > > cut them off the internet). > > > > > > How many years do you have to work on this project? :-) > > > > > > Federico > > > > > > (*) I might add a citation just because it's the first result a popular > > > search engine gives me, after glancing at the abstract and maybe the > > > journal home page; but if I remove an existing citation, hopefully I've > > > at least assessed its content and made a judgement about it, apart from > > > cases of mass removals for specific problems with certain articles or > > > publication venues. > > > > > > > > > > > > ------------------------------ > > > > > > Subject: Digest Footer > > > > > > _______________________________________________ > > > Wiki-research-l mailing list > > > Wiki-research-l(a)lists.wikimedia.org > > > https://lists.wikimedia.org/mailman/listinfo/wiki-research-l > > > > > > > > > ------------------------------ > > > > > > End of Wiki-research-l Digest, Vol 168, Issue 20 > > > ************************************************ > > > > > _______________________________________________ > > Wiki-research-l mailing list > > Wiki-research-l(a)lists.wikimedia.org > > https://lists.wikimedia.org/mailman/listinfo/wiki-research-l > > > > > ------------------------------ > > Subject: Digest Footer > > _______________________________________________ > Wiki-research-l mailing list > Wiki-research-l(a)lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/wiki-research-l > > > ------------------------------ > > End of Wiki-research-l Digest, Vol 168, Issue 22 > ************************************************ >

2 1

soweego 1.1 is under way
by Marco Fossati 28 Aug '19

28 Aug '19

[You can skip this message if you've already read it in the Wikidata mailing list, and sorry for the disturb] Hi everyone, Wearing the soweego project lead hat, I'm pleased to announce that the Wikimedia Foundation has approved the *soweego 1.1* proposal: https://meta.wikimedia.org/wiki/Grants:Project/Rapid/Hjfocs/soweego_1.1 The main goal is to put together different machine learning algorithms and get the highest-quality links between Wikidata and large external catalogs. Stay tuned for more rock'n'roll at: https://github.com/Wikidata/soweego And while you're there, why don't you give a star? Cheers, Marco

1 0

[Input requested] Data Lake Edit release input request
by Leila Zia 27 Aug '19

27 Aug '19

In a nutshell: We are asking for your input to help us learn how to release the historical edit data of Wikimedia projects in a more efficient way. Please provide your feedback via https://docs.google.com/forms/d/e/1FAIpQLScc15eSeFrVvAh_ydpX_1p0v6-WSx2qe3E… by 2019-09-03. ****** Dear researchers, The Analytics team at Wikimedia Foundation [1] has been working on building a data lake [2] for Wikimedia edits [3] to enable the research and analysis of Wikimedia's edit data in a more efficient way. This data is a history of activity on Wikimedia projects as complete and research-friendly as possible. Edits have context, such as whether they were reverted, in the same line as the edit itself. So you can focus more on what you want to find out instead of writing code to wrestle the data. Each line of the data released will include the following and more (see full specification [3a], [3b], [3c]): * editor edit count, groups, blocks, bot status, name, current and historical (time of edit) * seconds since this editor's last edit * page context, current and historical (namespace, seconds since last revision, etc.) * seconds to identity revert or deletion, if applicable * revision tags (mobile edit, ve edit, etc.) The first instance of this data will be released in the coming months and to make this release as useful as possible for you all, the users of the data, the team needs to hear your thoughts on how to slice and dice the data at publishing time. You can provide your input at https://docs.google.com/forms/d/e/1FAIpQLScc15eSeFrVvAh_ydpX_1p0v6-WSx2qe3E… . Please provide your input to this survey no later than 2019-09-03. Best, Leila [1] https://wikitech.wikimedia.org/wiki/Analytics [2] https://en.wikipedia.org/wiki/Data_lake [3] https://wikitech.wikimedia.org/wiki/Analytics/Data_Lake/Edits a) https://wikitech.wikimedia.org/wiki/Analytics/Data_Lake/Edits/Mediawiki_his… b) https://wikitech.wikimedia.org/wiki/Analytics/Data_Lake/Edits/Mediawiki_use… c) https://wikitech.wikimedia.org/wiki/Analytics/Data_Lake/Edits/Mediawiki_pag… -- Leila Zia Principal Research Scientist, Head of Research Wikimedia Foundation

2 3

Looking for a senior full-stack developer (Contractor) for Scribe (Wikimedia-funded project)
by Lucie-Aimée Kaffee 27 Aug '19

27 Aug '19

Hello everyone, We are looking for a senior full-stack developer to work with us as a contractor on Scribe. Scribe is a Wikimedia Foundation funded software and research project. We are building a tool to support newcomer Wikipedia editors in creating new articles in under-resourced languages (e.g. Arabic, Hindi). The project is open-source and will be integrated on low-resource Wikipedias so you will be working on a great cause of making knowledge accessible to everyone. Plus your work will have great visibility and impact on the community. *You:* We are looking for a senior software engineer contractor to realize the Scribe project in a part-time position for 9 months starting on the 14th of October. You will work remotely with us. We are looking for someone with *3+ years of experience in Full-stack development and community-focused applications*. *Your future team:* The project leads are Lucie and Hady, researchers in Computer Science. So you will be part of our three-person team and able to influence the project itself. Lucie is a PhD researcher at the University of Southampton and TIB Hannover. She previously worked at Wikimedia Germany. Hady holds a PhD in Natural Language Processing and Machine Learning and currently works as a researcher in Naver Labs Europe. We live in France and Germany/UK respectively, so we are used to working remotely and effective communication. *How to apply: * *Make sure to read the full job description here:* https://meta.wikimedia.org/wiki/Scribe/Job Then drop us an email (scribe.wikimedia(a)gmail.com) showing your interest in the project with your *CV* and *portfolio attached*. We also appreciate *any other kind of information in any format* that shows your relevant experience, creativity and problem-solving skills. Note: Interviews will be scheduled on a first come first served basis until the position gets filled, so please consider to submit your profile as soon as possible. Looking forward to hearing from you, Lucie-Aimée Kaffee and Hady Elsahar -- Lucie-Aimée Kaffee Web and Internet Science Group School of Electronics and Computer Science University of Southampton

1 0

Re: [Wiki-research-l] gender balance of Wikipedia citations
by Greg 27 Aug '19

27 Aug '19

Thanks for sharing your experience and thoughts, Jane. I did not know this was happening--I'm hardly an expert, so that's not surprising, and yet it's still very troubling to hear. I'm not sure what you mean by setting up a Wikiproject. Do you mean of ways for how to study this gap--i.e., the ideas that have been floated in this thread to this point? Or are you thinking of something else? Greg On Mon, Aug 26, 2019 at 5:00 AM <wiki-research-l-request(a)lists.wikimedia.org> wrote: > Send Wiki-research-l mailing list submissions to > wiki-research-l(a)lists.wikimedia.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://lists.wikimedia.org/mailman/listinfo/wiki-research-l > or, via email, send a message with subject or body 'help' to > wiki-research-l-request(a)lists.wikimedia.org > > You can reach the person managing the list at > wiki-research-l-owner(a)lists.wikimedia.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Wiki-research-l digest..." > > > Today's Topics: > > 1. Re: gender balance of Wikipedia citations (WereSpielChequers) > 2. Re: gender balance of Wikipedia citations (Greg) > 3. Re: sockpuppets and how to find them sooner (Federico Leva (Nemo)) > 4. Re: gender balance of Wikipedia citations (Jane Darnell) > 5. Re: gender balance of wikipedia citations (Federico Leva (Nemo)) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Sun, 25 Aug 2019 14:28:25 +0100 > From: WereSpielChequers <werespielchequers(a)gmail.com> > To: Research into Wikimedia content and communities > <wiki-research-l(a)lists.wikimedia.org> > Subject: Re: [Wiki-research-l] gender balance of Wikipedia citations > Message-ID: > <CAAanWP3qJnMpLB4tr9Eqt4EJLg2kCihkb50UY-d8= > ShNONhSAA(a)mail.gmail.com> > Content-Type: text/plain; charset="UTF-8" > > Hi Greg, > > One of the major step changes in the early growth of the English Wikipedia > was when a bot called RamBot created stub articles on US places. I think > they were cited to the census. Others have created articles on rivers in > countries and various other topics by similar programmatic means. Nowadays > such article creation is unlikely to get consensus on the English > Wikipedia, but there are some languages which are very open to such > creations and have them by the million. > > I'm not sure if the fastest updating of existing articles is automated or > just semiautomated. But looking at the bot requests page, it certainly > looks like some people are running such maintenance bots "updating GDP by > country" is a current bot request. > https://en.wikipedia.org/wiki/Wikipedia:Bot_requests. > > I'm not sure how "the ease of a source for purposes of converting into a > table and generating a separate article for each row" relates to gender. > But i suspect "number of times cited in wikipedia" deserves less kudos than > "number of times cited in academia". > > WSC > > On Sun, 25 Aug 2019 at 05:22, Greg <thenatureprogram(a)gmail.com> wrote: > > > Thanks again, Kerry. I am hoping that someone with access to more > resources > > (knowledge, support, etc) than I have will look into this. > > > > A few more thoughts/questions: > > > > 1. The link to the citation dataset from the Medium article ("What are > the > > ten most cited sources on Wikipedia? Let’s ask the data.") is broken. > > 2. As far as I can tell, every named author in the top ten most cited > > sources on Wikipedia is male. One piece is by a working group > > 3. This line from the Medium piece struck me: "Many of these publications > > have been cited by Wikipedians across large series of articles using > > powerful bots and automated tools." > > > > Are citations being added by bots? I'm not sure that I understand that > line > > correctly. > > > > Greg > > > > > > > > > > > > > ------------------------------ > > Message: 2 > Date: Sun, 25 Aug 2019 21:16:25 -0700 > From: Greg <thenatureprogram(a)gmail.com> > To: wiki-research-l(a)lists.wikimedia.org > Subject: Re: [Wiki-research-l] gender balance of Wikipedia citations > Message-ID: > <CAOO9DNvGyfvJkzyRq60cSQi-T80mAkUa= > vCPkzFbEysfGQqnVg(a)mail.gmail.com> > Content-Type: text/plain; charset="UTF-8" > > Thanks, WSC. All very interesting. > > I've been thinking about Wiklpedia citations less in terms of kudos and > more in terms of a feedback loop. The cited sources get a significant > amount of attention (1 click per 200 pageviews is the number I saw > recently). When I imagine total Wikipedia traffic, that's huge. How many > students are finding sources this way? How many academics? And how many of > these citations are finding their way back into academic publications via > this mechanism? > > Assuming this is happening to some degree, the gender imbalance of the > citations is also reflected. If the Wikipedia imbalance is the same as the > one in academia, that's one thing; if it is better on Wikipedia than it is > in academia, that's reason to celebrate; if the balance is worse, that's > concerning. In fact, if the gender imbalance conforms to my fears instead > of my hopes, and is magnified by the massive website traffic, I imagine it > could even explain the growth in the citation disparity researchers note in > their study of political science texts. (I link to that study in a previous > post; it was mentioned in the Washington Post recently) > > There is a very real possibility that Wikipedia is making the citation > gender gap worse. I think we need to understand what is happening and take > immediate action if the news is not good. > > Greg > > > > > > > > > > ------------------------------ > > Message: 3 > Date: Mon, 26 Aug 2019 10:59:07 +0300 > From: "Federico Leva (Nemo)" <nemowiki(a)gmail.com> > To: Research into Wikimedia content and communities > <wiki-research-l(a)lists.wikimedia.org>, Aaron Halfaker > <ahalfaker(a)wikimedia.org>, Kerry Raymond <kerry.raymond(a)gmail.com> > Subject: Re: [Wiki-research-l] sockpuppets and how to find them sooner > Message-ID: <cf2734ff-d2cf-3108-691f-8ecf46125ed7(a)gmail.com> > Content-Type: text/plain; charset=utf-8; format=flowed > > Please everyone avoid using jargon specific to the English Wikipedia on > this cross-language and cross-wiki mailing list. > > Aaron Halfaker, 23/08/19 17:36: > > I think embeddings[1] would be a nice way to create a signature. > > There is some discussion of acceptable user fingerprinting (presumably > to be available to CheckUsers only), other than the usual over-reliance > on IP addresses, in particular at > < > https://meta.wikimedia.org/wiki/Talk:IP_Editing:_Privacy_Enhancement_and_Ab… > >. > > Federico > > > > ------------------------------ > > Message: 4 > Date: Mon, 26 Aug 2019 10:17:46 +0200 > From: Jane Darnell <jane023(a)gmail.com> > To: Research into Wikimedia content and communities > <wiki-research-l(a)lists.wikimedia.org> > Subject: Re: [Wiki-research-l] gender balance of Wikipedia citations > Message-ID: > <CAFVcA-G87k26nBMr=-e-+C8o6eG0KQvVihH= > f4M40faVNbKkqw(a)mail.gmail.com> > Content-Type: text/plain; charset="UTF-8" > > Greg, > Thanks for worrying. This is a known problem and yes, Wikipedia contributes > to the Gendergap in citations and no, it's not an easy fix, since it is the > fault of systemic bias in academia. So fewer women are head author on > scientific publications, and it is generally only the head author that gets > cited on Wikipedia. This is not just a problem with written works in the > field of politics. I spend most of my time working on paintings and their > documented catalogs, so generally I only notice and fix this problem in art > catalogs. Women rarely appear as lead author mentioned. I will always add > them in to descriptions when I add items for their works on Wikidata, but I > can not always find them! Sometimes I can't even create items for them > because all I have is a name and a work and nothing else available online > anywhere. You see this most often with women who spent entire careers > working at a single institution and the institution doesn't bother to > promote their work or even list them in exhibition catalogs. With luck > there might be a local obituary, but not always. If you have suggestions > how to set up a Wikiproject to tackle this it would be a good idea. In my > onwiki experience the Women-in-Red community can be very positive in their > response to gendergap-related issues for women writers. > Jane > > On Mon, Aug 26, 2019 at 6:17 AM Greg <thenatureprogram(a)gmail.com> wrote: > > > Thanks, WSC. All very interesting. > > > > I've been thinking about Wiklpedia citations less in terms of kudos and > > more in terms of a feedback loop. The cited sources get a significant > > amount of attention (1 click per 200 pageviews is the number I saw > > recently). When I imagine total Wikipedia traffic, that's huge. How many > > students are finding sources this way? How many academics? And how many > of > > these citations are finding their way back into academic publications via > > this mechanism? > > > > Assuming this is happening to some degree, the gender imbalance of the > > citations is also reflected. If the Wikipedia imbalance is the same as > the > > one in academia, that's one thing; if it is better on Wikipedia than it > is > > in academia, that's reason to celebrate; if the balance is worse, that's > > concerning. In fact, if the gender imbalance conforms to my fears instead > > of my hopes, and is magnified by the massive website traffic, I imagine > it > > could even explain the growth in the citation disparity researchers note > in > > their study of political science texts. (I link to that study in a > previous > > post; it was mentioned in the Washington Post recently) > > > > There is a very real possibility that Wikipedia is making the citation > > gender gap worse. I think we need to understand what is happening and > take > > immediate action if the news is not good. > > > > Greg > > > > > > > > > > > > > _______________________________________________ > > Wiki-research-l mailing list > > Wiki-research-l(a)lists.wikimedia.org > > https://lists.wikimedia.org/mailman/listinfo/wiki-research-l > > > > > ------------------------------ > > Message: 5 > Date: Mon, 26 Aug 2019 11:45:09 +0300 > From: "Federico Leva (Nemo)" <nemowiki(a)gmail.com> > To: Research into Wikimedia content and communities > <wiki-research-l(a)lists.wikimedia.org>, Greg > <thenatureprogram(a)gmail.com> > Subject: Re: [Wiki-research-l] gender balance of wikipedia citations > Message-ID: <835202af-4653-641e-782e-c619458bdd7f(a)gmail.com> > Content-Type: text/plain; charset=utf-8; format=flowed > > Greg, 22/08/19 06:19: > > I do not know the current status of wikicite or if/when this > > could be used for this inquiry--either to examine all, or a sensible > subset > > of the citations. > > If I see correctly, you still did not receive an answer on the data > available. > > It's true that the Figshare item for > < > https://meta.wikimedia.org/wiki/Research:Scholarly_article_citations_in_Wik…> > > was deleted (I've asked about it on the talk page), but it's trivial to > run https://pypi.org/project/mwcites/ and extract the data yourself, at > least for citations which use an identifier. > > Some example datasets produced this way: > https://zenodo.org/record/15871 > https://zenodo.org/record/55004 > https://zenodo.org/record/54799 > > Once you extract the list of works, the fun begins. You'll need to > intersect with other data sources (Wikidata, ORCID, other?) and account > for a number of factors until you manage to find a subset of the data > which has a sufficiently high signal:noise ratio. For instance you might > need to filter or normalise by > * year of publication (some year recent enough to have good data but old > enough to allow the work to be cited elsewhere, be archived after > embargos); > * country or institution (some probably have better ORCID coverage); > * field/discipline and language; > * open access status (per Unpaywall); > * number of expected pageviews and clicks (for instance using > <https://wikitech.wikimedia.org/wiki/Analytics/AQS/Pageviews> and > <https://meta.wikimedia.org/wiki/Research:Wikipedia_clickstream#Releases>; > > a link from 10k articles on asteroids or proteins is not the same as > being the lone link from a popular article which is not the same as a > link buried among a thousand others on a big article); > * time or duration of the addition (with one of the various diff > extraction libraries, content persistence data or possibly historical > eventstream if such a thing is available). > > To avoid having to invent everything yourself, maybe you can reuse the > method of some similar study, for instance the one on the open access > citation advantage or one of the many which studied the gender imbalance > of citations and peer review in journals. > > However, it's very possible that the noise is just too much for a > general computational method. You might consider a more manual approach > on a sample of relevant events, for instance the *removal* of citations, > which is in my opinion more significant than the addition.* You might > extract all the diffs which removed a citation from an article in the > last N years (probably they'll be in the order of 10^5 rather than > 10^6), remove some massive events or outliers, sample 500-1000 of them > randomly and verify the required data manually. > > As usual it will be impossible to have an objective assessment of > whether that citation was really (in)appropriate in that context > according to the (English or whatever) Wikipedia guidelines. To test > that too, you should replicate one of the various studies of the gender > imbalance of peer review, perhaps one of those which tried to assess the > impact of a double blind peer review system on the gender imbalance. > However, because the sources are already published, you'd need to > provide the agendered information yourself and make sure the > participants perform their assessment in some controlled environment > where they don't have access to any gendered information (i.e. where you > cut them off the internet). > > How many years do you have to work on this project? :-) > > Federico > > (*) I might add a citation just because it's the first result a popular > search engine gives me, after glancing at the abstract and maybe the > journal home page; but if I remove an existing citation, hopefully I've > at least assessed its content and made a judgement about it, apart from > cases of mass removals for specific problems with certain articles or > publication venues. > > > > ------------------------------ > > Subject: Digest Footer > > _______________________________________________ > Wiki-research-l mailing list > Wiki-research-l(a)lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/wiki-research-l > > > ------------------------------ > > End of Wiki-research-l Digest, Vol 168, Issue 20 > ************************************************ >

2 1

gender balance of wikipedia citations
by Greg 27 Aug '19

27 Aug '19

Greetings! I was looking for information about the gender balance of Wikipedia citations and no one I've asked knows of any work on this topic. Do you? I think this is an important question. Here's what I've learned so far: Wikipedia citations are currently in the form of text strings. There is also an initiative to place citations in an annotated structured repository (wikicite). I do not know the current status of wikicite or if/when this could be used for this inquiry--either to examine all, or a sensible subset of the citations. My perspective is that understanding the gender balance is necessary and urgent. The balance could be better, the same, or worse than the citation balances we already know, and the scale of the effect is quite large. Is this a line of inquiry that the wikimedia/wikicite community is interested in pursuing? If so, what is the best way to get started? Does the WMF have the resources and interest to look into this matter inhouse? Thanks for your thoughts. Greg

4 4

sockpuppets and how to find them sooner
by Kerry Raymond 26 Aug '19

26 Aug '19

Currently, to open a sockpuppet investigation, you must name the two (or more) accounts that you believe to be sockpuppets with "clear, behavioural evidence of sock puppetry" which is typically in the form of pairs of edits that demonstrate similar edit behaviours that are unlikely to naturally occur. Now if you spend enough time on-wiki, you develop an intuition about behaviours you see on your watchlist and in article edit histories. Often I am highly suspicious that an account is a sockpuppet, but I cannot report them because I don't know which other account is involved. As a example, I recently encounted User:Shelati an account about 1 day old at that time with nearly 100 edits in that day all about 1-2 minutes apart, mostly making a similar change to a large number of Australian place infoboxes. https://en.wikipedia.org/w/index.php?title=Special:Contributions/Shelati <https://en.wikipedia.org/w/index.php?title=Special:Contributions/Shelati&of fset=20190728053057&limit=100&target=Shelati> &offset=20190728053057&limit=100&target=Shelati Genuine new users do not edit that quickly, do not use templates and do not mess structurally with infoboxes (at most they try to change the values). It "smelled" like a sockpuppet. However, as I did not recognise that pattern of edit behaviour as being that of any other user I was familiar with, it wasn't something I could report for sockpuppet investigation. Anyhow after about 2 weeks, the user was blocked as a sockpuppet. Someone must have noticed and figured out the other account: https://en.wikipedia.org/wiki/Wikipedia:Sockpuppet_investigations/Meganesia/ Archive Two weeks and 1,279 edits later . that's over 1000 possibly problematic edits after I first suspected them. But that's nothing compared with another ongoing situation in which a very large number of different IPs are engaged in a pattern of problem edits on mostly Australian articles (a few different types of edits but an obvious "quack like a duck" situation). The IP number changes frequently (and one assumes deliberately). The edits potentially go back to 2013 but appear to have intensified in 2018/2019. Here's one user's summary of all the IP addresses involved, and the extent to which they have been cleaned up, given many thousands of edits are involved, see: https://en.wikipedia.org/wiki/User:IamNotU/History_cleanup As well as the damage done to the content (which harms the readers), these IP sockpuppets are consuming enormous amounts of effort to track them down and revert them, which could be more productively used to improve the content. We need better tools to foil these pests. So I want to put that challenge out to this list. Kerry

5 7

Re: [Wiki-research-l] sockpuppets and how to find them sooner
by Kerry Raymond 26 Aug '19

26 Aug '19

To reply to my own question . Can we find a way to create a "signature" of an account's pattern of editing? Perhaps it might be a set of signatures, maybe one for the categories that the account appears to be active in, another for the type of edit, etc. Then if these signatures were calculated for all banned accounts or currently blocked accounts (or at least ones with a long enough contribution history to make it worthwhile - we're not interested in one-edit vandals), then we could have a tool that could be run to quickly compare one account against the signatures of banned/blocked accounts as well as the cumulative edits of a set of known sockpuppets (i.e. treat them as a single account) to determine if this may be a sockpuppet case meriting further investigation. I imagine that it would be too expensive computationally to actually run comparisons of the contribution histories of all "bad guy" accounts against the suspicious account which is why I propose a "signature" approach (but I'm happy to be told otherwise). If we had such a tool and it proves reasonably reliable in identifying likely sockpuppets (not asking for guarantees but close enough not to be a waste of time to investigate), then we could routinely use it on new accounts or reactivating accounts (i.e. possible sleeper accounts) once they have a long enough editing history to enable the tool to operate effectively to provide automated early warning of new/reactivating accounts appearing suspiciously similar to "bad guy" accounts. But this is a hard problem, both technically and socially (Assume Good Faith, Privacy, etc), so I welcome the thoughts of others. Kerry

4 3

Re: [Wiki-research-l] gender balance of Wikipedia citations
by Greg 26 Aug '19

26 Aug '19

Thanks, WSC. All very interesting. I've been thinking about Wiklpedia citations less in terms of kudos and more in terms of a feedback loop. The cited sources get a significant amount of attention (1 click per 200 pageviews is the number I saw recently). When I imagine total Wikipedia traffic, that's huge. How many students are finding sources this way? How many academics? And how many of these citations are finding their way back into academic publications via this mechanism? Assuming this is happening to some degree, the gender imbalance of the citations is also reflected. If the Wikipedia imbalance is the same as the one in academia, that's one thing; if it is better on Wikipedia than it is in academia, that's reason to celebrate; if the balance is worse, that's concerning. In fact, if the gender imbalance conforms to my fears instead of my hopes, and is magnified by the massive website traffic, I imagine it could even explain the growth in the citation disparity researchers note in their study of political science texts. (I link to that study in a previous post; it was mentioned in the Washington Post recently) There is a very real possibility that Wikipedia is making the citation gender gap worse. I think we need to understand what is happening and take immediate action if the news is not good. Greg > > >

2 1

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

Wiki-research-l